Neyman-Pearson Lemma

We will review useful Neyman-Pearson lemma, see [FS04]. It is often used to resolve optimization problem arised from efficient hedging problems.

Let {\mathbb{P}} and {\mathbb{Q}} be two probability measures on {(\Omega, \mathcal{F})}. We say {\mathbb{Q}} is absolutely continuous w.r.t. {\mathbb{P}} and write {\mathbb{Q} << \mathbb{P}}, if for all {A\in \mathcal{F}}

\displaystyle \mathbb{P}(A) = 0 \hbox{ implies } \mathbb{Q}(A) = 0.

By Radon-Nikodym theorem, {\mathbb{Q} << \mathbb{P}} if and only if there exists {\mathcal{F}}-measurable function {\varphi} satisfying

\displaystyle \int F d \mathbb{Q} = \int F \varphi d \mathbb{P}, \quad \forall \mathcal{F}\hbox{-measurable r.v. } F.

We often write { \frac{d \mathbb{Q}}{d \mathbb{P}}:= \varphi}, and refer this to density or Radon-Nikodym derivative.

Hence, if {\mathbb{Q} << \mathbb{P}}, then {\mathbb{Q}(A) = \int_A \frac{d \mathbb{Q}}{d \mathbb{P}} d \mathbb{P}}. In general, if {\mathbb{Q}<<\mathbb{P}} does not hold, then we have Lesbegue decomposition instead: there exists {N\in \mathcal{F}} with {\mathbb{P}(N) = 0} satisfying

\displaystyle \mathbb{Q}(A) = \mathbb{Q}(A \cap N) + \int_{A\cap N^c} \varphi d \mathbb{P}, \quad \forall A\in \mathcal{F}

for some {\mathcal{F}}-measurable function {\varphi}. In this case, we can generalize Radon-Nikodym derivative by { \frac{d \mathbb{Q}}{d \mathbb{P}} (\omega) := \left\{ \begin{array}{ll} \varphi(\omega) & \omega \in N^c,\\ \infty & \hbox{otherwise}. \end{array} \right.}

Proposition 1 (Neyman-Pearson Lemma) Let {\mathbb{P}} and {\mathbb{Q}} be two probability measures on{(\Omega, \mathcal{F})}. Suppose {A^0\in \mathcal{F}} satisfies {\{\frac{d \mathbb{P}}{d \mathbb{Q}} > c \} \subset A^0 \subset \{\frac{d \mathbb{P}}{d \mathbb{Q}} \ge c \}} for some fixed {c\ge 0}. Then, for arbitrary {A\in \mathcal{F}},

\displaystyle \mathbb{Q}(A) \le \mathbb{Q}(A^0) \hbox{ implies } \mathbb{P}(A) \le \mathbb{P}(A^0).

Proof: Let {F = I_{A^0} - I_A}. Define {N = \{\frac{d \mathbb{P}}{d \mathbb{Q}} = \infty \}.} Then, {A^0 \supset N}, and {F\ge 0} on {N}. Also, {F \cdot (\frac{d \mathbb{P}}{d \mathbb{Q}} - c) \ge 0}. Now, we have

\displaystyle  \begin{array}{ll} \mathbb{P}(A^0) - \mathbb{P}(A) &= \displaystyle \int F d \mathbb{P} = \int_N F d \mathbb{P} + \int_{N^c} F \frac{d \mathbb{P}}{d \mathbb{Q}} d \mathbb{Q} \\ & \ge c \int_{N^c} F d \mathbb{Q} = c (Q(A^0) - Q(A)). \end{array}

This completes the proof. \Box

Remark The above Proposition 1 generalized the Neyman-Pearson Lemma given by [FS04], in which {A^0 = \{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c \}}.

Let {\mathcal{I}} be the collection of all {\mathcal{F}}-measurable indicator functions. Proposition 1 provides a partial solution to the following optimization problem [NP]: Given {\alpha \in (0,1)}, find {\psi \in \mathcal{I}}, which maximizes

\displaystyle \mathbb{E}^\mathbb{P} [ \psi]

under constraint

\displaystyle \mathbb{E}^\mathbb{Q} [ \psi ] \le \alpha.

By 1, if there exists {A^0\in \mathcal{F}} satisfies {\{\frac{d \mathbb{P}}{d \mathbb{Q}} > c \} \subset A^0 \subset \{\frac{d \mathbb{P}}{d \mathbb{Q}} \ge c \}} with {\mathbb{Q}(A_0) = \alpha}, then {I_{A^0}} solves the problem [NP]. Now, we generalize Neyman-Pearson Lemma from indicator function to the randomized function. Let {\mathcal{R}} be the space of all randomized test function {\psi: \Omega/ \mathcal{F} \rightarrow [0, 1]/\mathcal{B}([0,1])}.

Proposition 2 Let {\mathbb{P}}, {\mathbb{Q}} and {\mu} be three probability measures on{(\Omega, \mathcal{F})}, and {\frac 1 2 (\mathbb{P} + \mathbb{Q}) << \mu}. Consider optimization problem [GNP]: Given fixed {\alpha \in (0,1)}, find {\psi\in \mathcal{R}}, which maximizes

\displaystyle \mathbb{E}^{\mathbb{P}} [\psi]

under constraint

\displaystyle \mathbb{E}^{\mathbb{Q}} [ \psi ] \le \alpha.

Then, there exisits {\psi^0 \in \mathcal{R}} solves the above problem [GNP], and {\psi^0} must satisfy

\displaystyle   \mathbb{E}^{\mathbb{Q}} [ \psi^0 ] = \alpha, \ \ \ \ \ (1)

and take the form of

\displaystyle   \psi^0 = I_{\{\frac{d \mathbb{P}}{d \mathbb{Q}} >c\}} + B I_{\{\frac{d \mathbb{P}}{d \mathbb{Q}} = c\}}, \ \mu-a.s. \ \ \ \ \ (2)

for the constant {c} given by

\displaystyle   c = \inf\Big\{c_1: \mathbb{Q}\Big(\frac{d \mathbb{P}}{d \mathbb{Q}}>c_1\Big) \le \alpha\Big\} \ \ \ \ \ (3)

and some random variable {B\in \mathcal{R}}. In particular, {B} can simply take a non-random as of

\displaystyle   B = \left\{ \begin{array}{ll} 0 & \hbox{ if } \mathbb{Q}\Big(\frac{d \mathbb{P}}{d \mathbb{Q}}=c \Big) = 0 \\ \frac{\alpha - \mathbb{Q}( \frac{d \mathbb{P}}{d \mathbb{Q}}>c)}{ \mathbb{Q}( \frac{d \mathbb{P}}{d \mathbb{Q}}=c)} & \hbox{ otherwise }. \end{array} \right. \ \ \ \ \ (4)

For the optimization problem [GNP], optimizer {\psi^0} given by (2) with {c} of (3) and (4) may not be unique one.

Proposition 3 Assume there exists {Y: \Omega/\mathcal{F} \rightarrow \mathbb{R}/\mathcal{B}(\mathbb{R})} with continuous cdf under {\mathbb{Q}} is continuous. Then, there exists an optimizer {\psi^0 \in \mathcal{I}} both for the problem [NP] and [GNP], i.e. the value of the problem [NP] and [GNP] are equal to each other.

Proof: Let {c} be given by (3). We can construct {\psi^0} in each of the following three cases.

  1. If {\mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} \ge c\} = \alpha}, then {\psi^0} may be of (2) with {B = 1}.
  2. If {\mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c\} = \alpha}, then {\psi^0} may be of (2) with {B = 0}.
  3. If {\mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c\} < \alpha < \mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} \ge c\}}, then {0< \alpha - \mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c\} < \mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} = c \}}. Let {G(y) = \mathbb{Q} ( \{ Y\le y \} \cap \{ \frac{d \mathbb{P}}{d \mathbb{Q}} = c \})}. Then, {G(\cdot)} satisfies
    1. { G(y) \rightarrow 0} as {y\rightarrow -\infty}, and { G(y) \rightarrow 1} as {y\rightarrow \infty}.
    2. {G(y)} is continuous.

    Hence, there exists {y^0} such that {G(y^0) = \alpha - \mathbb{Q}\{ \frac{d \mathbb{P}}{d \mathbb{Q}} > c\} }. Now, {\psi^0} may be of (2) with {B = I_{\{Y \le y^0\}}}.

One can use Proposition 1 and 2 to verify {\psi^0} given above is the optimizer for both problems. \Box

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s