Neyman-Pearson Lemma

We will review useful Neyman-Pearson lemma, see [FS04]. It is often used to resolve optimization problem arised from efficient hedging problems.

Let ${\mathbb{P}}$ and ${\mathbb{Q}}$ be two probability measures on ${(\Omega, \mathcal{F})}$ . We say ${\mathbb{Q}}$ is absolutely continuous w.r.t. ${\mathbb{P}}$ and write ${\mathbb{Q} << \mathbb{P}}$ , if for all ${A\in \mathcal{F}}$

$\displaystyle \mathbb{P}(A) = 0 \hbox{ implies } \mathbb{Q}(A) = 0.$

By Radon-Nikodym theorem, ${\mathbb{Q} << \mathbb{P}}$ if and only if there exists ${\mathcal{F}}$ -measurable function ${\varphi}$ satisfying

$\displaystyle \int F d \mathbb{Q} = \int F \varphi d \mathbb{P}, \quad \forall \mathcal{F}\hbox{-measurable r.v. } F.$

We often write ${ \frac{d \mathbb{Q}}{d \mathbb{P}}:= \varphi}$ , and refer this to density or Radon-Nikodym derivative.

Hence, if ${\mathbb{Q} << \mathbb{P}}$ , then ${\mathbb{Q}(A) = \int_A \frac{d \mathbb{Q}}{d \mathbb{P}} d \mathbb{P}}$ . In general, if ${\mathbb{Q}<<\mathbb{P}}$ does not hold, then we have Lesbegue decomposition instead: there exists ${N\in \mathcal{F}}$ with ${\mathbb{P}(N) = 0}$ satisfying

$\displaystyle \mathbb{Q}(A) = \mathbb{Q}(A \cap N) + \int_{A\cap N^c} \varphi d \mathbb{P}, \quad \forall A\in \mathcal{F}$

for some ${\mathcal{F}}$ -measurable function ${\varphi}$ . In this case, we can generalize Radon-Nikodym derivative by ${ \frac{d \mathbb{Q}}{d \mathbb{P}} (\omega) := \left\{ \begin{array}{ll} \varphi(\omega) & \omega \in N^c,\\ \infty & \hbox{otherwise}. \end{array} \right.}$

Proposition 1 (Neyman-Pearson Lemma) Let ${\mathbb{P}}$ and ${\mathbb{Q}}$ be two probability measures on ${(\Omega, \mathcal{F})}$ . Suppose ${A^0\in \mathcal{F}}$ satisfies ${\{\frac{d \mathbb{P}}{d \mathbb{Q}} > c \} \subset A^0 \subset \{\frac{d \mathbb{P}}{d \mathbb{Q}} \ge c \}}$ for some fixed ${c\ge 0}$ . Then, for arbitrary ${A\in \mathcal{F}}$ ,

$\displaystyle \mathbb{Q}(A) \le \mathbb{Q}(A^0) \hbox{ implies } \mathbb{P}(A) \le \mathbb{P}(A^0).$

Proof: Let ${F = I_{A^0} - I_A}$ . Define ${N = \{\frac{d \mathbb{P}}{d \mathbb{Q}} = \infty \}.}$ Then, ${A^0 \supset N}$ , and ${F\ge 0}$ on ${N}$ . Also, ${F \cdot (\frac{d \mathbb{P}}{d \mathbb{Q}} - c) \ge 0}$ . Now, we have

$\displaystyle \begin{array}{ll} \mathbb{P}(A^0) - \mathbb{P}(A) &= \displaystyle \int F d \mathbb{P} = \int_N F d \mathbb{P} + \int_{N^c} F \frac{d \mathbb{P}}{d \mathbb{Q}} d \mathbb{Q} \\ & \ge c \int_{N^c} F d \mathbb{Q} = c (Q(A^0) - Q(A)). \end{array}$

This completes the proof. $\Box$

Remark The above Proposition 1 generalized the Neyman-Pearson Lemma given by [FS04], in which ${A^0 = \{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c \}}$ .

Let ${\mathcal{I}}$ be the collection of all ${\mathcal{F}}$ -measurable indicator functions. Proposition 1 provides a partial solution to the following optimization problem [NP]: Given ${\alpha \in (0,1)}$ , find ${\psi \in \mathcal{I}}$ , which maximizes

$\displaystyle \mathbb{E}^\mathbb{P} [ \psi]$

under constraint

$\displaystyle \mathbb{E}^\mathbb{Q} [ \psi ] \le \alpha.$

By 1, if there exists ${A^0\in \mathcal{F}}$ satisfies ${\{\frac{d \mathbb{P}}{d \mathbb{Q}} > c \} \subset A^0 \subset \{\frac{d \mathbb{P}}{d \mathbb{Q}} \ge c \}}$ with ${\mathbb{Q}(A_0) = \alpha}$ , then ${I_{A^0}}$ solves the problem [NP]. Now, we generalize Neyman-Pearson Lemma from indicator function to the randomized function. Let ${\mathcal{R}}$ be the space of all randomized test function ${\psi: \Omega/ \mathcal{F} \rightarrow [0, 1]/\mathcal{B}([0,1])}$ .

Proposition 2 Let ${\mathbb{P}}$ , ${\mathbb{Q}}$ and ${\mu}$ be three probability measures on ${(\Omega, \mathcal{F})}$ , and ${\frac 1 2 (\mathbb{P} + \mathbb{Q}) << \mu}$ . Consider optimization problem [GNP]: Given fixed ${\alpha \in (0,1)}$ , find ${\psi\in \mathcal{R}}$ , which maximizes

$\displaystyle \mathbb{E}^{\mathbb{P}} [\psi]$

under constraint

$\displaystyle \mathbb{E}^{\mathbb{Q}} [ \psi ] \le \alpha.$

Then, there exisits ${\psi^0 \in \mathcal{R}}$ solves the above problem [GNP], and ${\psi^0}$ must satisfy

$\displaystyle \mathbb{E}^{\mathbb{Q}} [ \psi^0 ] = \alpha, \ \ \ \ \ (1)$

and take the form of

$\displaystyle \psi^0 = I_{\{\frac{d \mathbb{P}}{d \mathbb{Q}} >c\}} + B I_{\{\frac{d \mathbb{P}}{d \mathbb{Q}} = c\}}, \ \mu-a.s. \ \ \ \ \ (2)$

for the constant ${c}$ given by

$\displaystyle c = \inf\Big\{c_1: \mathbb{Q}\Big(\frac{d \mathbb{P}}{d \mathbb{Q}}>c_1\Big) \le \alpha\Big\} \ \ \ \ \ (3)$

and some random variable ${B\in \mathcal{R}}$ . In particular, ${B}$ can simply take a non-random as of

$\displaystyle B = \left\{ \begin{array}{ll} 0 & \hbox{ if } \mathbb{Q}\Big(\frac{d \mathbb{P}}{d \mathbb{Q}}=c \Big) = 0 \\ \frac{\alpha - \mathbb{Q}( \frac{d \mathbb{P}}{d \mathbb{Q}}>c)}{ \mathbb{Q}( \frac{d \mathbb{P}}{d \mathbb{Q}}=c)} & \hbox{ otherwise }. \end{array} \right. \ \ \ \ \ (4)$

For the optimization problem [GNP], optimizer ${\psi^0}$ given by (2) with ${c}$ of (3) and (4) may not be unique one.

Proposition 3 Assume there exists ${Y: \Omega/\mathcal{F} \rightarrow \mathbb{R}/\mathcal{B}(\mathbb{R})}$ with continuous cdf under ${\mathbb{Q}}$ is continuous. Then, there exists an optimizer ${\psi^0 \in \mathcal{I}}$ both for the problem [NP] and [GNP], i.e. the value of the problem [NP] and [GNP] are equal to each other.

Proof: Let ${c}$ be given by (3). We can construct ${\psi^0}$ in each of the following three cases.

If ${\mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} \ge c\} = \alpha}$ , then ${\psi^0}$ may be of (2) with ${B = 1}$ .
If ${\mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c\} = \alpha}$ , then ${\psi^0}$ may be of (2) with ${B = 0}$ .
If , then . Let . Then, satisfies
1. ${ G(y) \rightarrow 0}$ as ${y\rightarrow -\infty}$ , and ${ G(y) \rightarrow 1}$ as ${y\rightarrow \infty}$ .
2. ${G(y)}$ is continuous.
Hence, there exists ${y^0}$ such that ${G(y^0) = \alpha - \mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c\} }$ . Now, ${\psi^0}$ may be of (2) with ${B = I_{\{Y \le y^0\}}}$ .