Mathematics Foundations/21.5 Radon-Nikodym Theorem

21.5 Radon-Nikodym Theorem

Introduction

The Radon-Nikodym theorem is a fundamental result in measure theory that establishes conditions under which one measure can be represented as an integral with respect to another measure. Named after Johann Radon and Otto Nikodym, this theorem provides a powerful tool for analyzing relationships between measures and has wide-ranging applications in probability theory, statistics, and functional analysis.

Absolute Continuity

Before stating the Radon-Nikodym theorem, we need to understand the concept of absolute continuity.

Definition: Let $\mu$ and $\nu$ be measures on a measurable space $(X,{\mathcal {F}})$. We say that $\nu$ is absolutely continuous with respect to $\mu$, denoted $\nu \ll \mu$, if for every measurable set $E\in {\mathcal {F}}$:

$\mu (E)=0\implies \nu (E)=0$

In other words, $\nu$ assigns zero measure to any set that $\mu$ assigns zero measure.

The Radon-Nikodym Theorem

Theorem (Radon-Nikodym): Let $(X,{\mathcal {F}},\mu )$ be a σ-finite measure space and let $\nu$ be a σ-finite measure on $(X,{\mathcal {F}})$ that is absolutely continuous with respect to $\mu$. Then there exists a measurable function $f:X\rightarrow [0,\infty )$, unique up to sets of $\mu$-measure zero, such that for all $E\in {\mathcal {F}}$:

$\nu (E)=\int _{E}f\,d\mu$

The function $f$ is called the Radon-Nikodym derivative of $\nu$ with respect to $\mu$ and is often denoted by ${\frac {d\nu }{d\mu }}$.

Proof Sketch

The proof of the Radon-Nikodym theorem involves several key steps:

1. First, we reduce to the case where both $\mu$ and $\nu$ are finite measures. 2. We define a functional on the space of simple functions and show it is bounded. 3. Using the Riesz representation theorem, we establish the existence of the derivative function. 4. Finally, we verify uniqueness up to sets of measure zero.

The complete proof requires techniques from functional analysis and is beyond the scope of this section, but the above outline captures the essential strategy.

Properties of the Radon-Nikodym Derivative

The Radon-Nikodym derivative satisfies several important properties:

1. Linearity: If $\nu _{1}\ll \mu$ and $\nu _{2}\ll \mu$, then for any constants $a,b\geq 0$:

  ${\frac {d(a\nu _{1}+b\nu _{2})}{d\mu }}=a{\frac {d\nu _{1}}{d\mu }}+b{\frac {d\nu _{2}}{d\mu }}$

2. Chain Rule: If $\nu \ll \lambda$ and $\lambda \ll \mu$, then $\nu \ll \mu$ and:

  ${\frac {d\nu }{d\mu }}={\frac {d\nu }{d\lambda }}\cdot {\frac {d\lambda }{d\mu }}$

3. Change of Measure: For any measurable function $g$, if $g$ is integrable with respect to $\nu$, then:

  $\int _{X}g\,d\nu =\int _{X}g\cdot {\frac {d\nu }{d\mu }}\,d\mu$

Applications

The Radon-Nikodym theorem has numerous applications:

1. Probability Theory: The theorem provides the theoretical foundation for likelihood ratios and probability density functions.

2. Statistics: It underlies the theory of sufficient statistics and parameter estimation.

3. Functional Analysis: The theorem is used to represent bounded linear functionals on $L^{p}$ spaces.

4. Mathematical Finance: It plays a crucial role in the change of measure techniques used in option pricing.

Examples

Example 1: Let $\mu$ be the Lebesgue measure on $\mathbb {R}$ and define $\nu (E)=\int _{E}x^{2}\,d\mu$. Then $\nu \ll \mu$ and ${\frac {d\nu }{d\mu }}(x)=x^{2}$.

Example 2: Let $\mu$ be the Lebesgue measure on $[0,1]$ and $\nu$ be the probability measure with density $f(x)=2x$. Then ${\frac {d\nu }{d\mu }}(x)=2x$ for $x\in [0,1]$.

Exercises

1. Prove that if $\nu \ll \mu$ and $\mu \ll \nu$, then there exists a strictly positive measurable function $h$ such that $\nu (E)=\int _{E}h\,d\mu$ for all measurable sets $E$.

2. Let $\mu$ be the Lebesgue measure on $[0,1]$ and $\nu$ be the measure defined by $\nu (E)=\int _{E}\sin ^{2}(\pi x)\,d\mu (x)$. Find the Radon-Nikodym derivative ${\frac {d\nu }{d\mu }}$.

3. Show that if $\nu$ is not absolutely continuous with respect to $\mu$, then the Radon-Nikodym derivative does not exist.

4. Let $P$ and $Q$ be probability measures on $(\Omega ,{\mathcal {F}})$ with $Q\ll P$. Define the Kullback-Leibler divergence as $D_{KL}(Q||P)=\int _{\Omega }\log \left({\frac {dQ}{dP}}\right)\,dQ$. Prove that $D_{KL}(Q||P)\geq 0$ with equality if and only if $P=Q$.

5. Let $\mu$ be a finite measure and $\nu =\mu +\delta _{a}$ where $\delta _{a}$ is the Dirac measure at point $a$. Determine whether $\nu$ is absolutely continuous with respect to $\mu$ and explain why.