Lecture 28: The Neyman-Pearson Lemma and Its Consequences

The Neyman-Pearson Lemma allows us to show that for tests of a simple alternative against a simple hypothesis, one can do no better than constructing a Likelihood Ratio Test. For the proofs of the theorems below, see Basic Probability Theory by Robert Ash.

In what follows, $H_0 = \{f_0\}$ and $H_a = \{f_a\}$ and we assume that either f₀ and f_a are both densities or both mass functions of the same random variable R.

Theorem 201

For any $\alpha\in[0,1]$ there is a Likelihood Ratio Test whose probability of Type I error is $\alpha$.

The sticky part of the proof involves handling the case where the distribution function of the likelihood ratio is not a continuous function.

Theorem 217 (Neyman-Pearson lemma)

Let $\phi_\lambda$ be a Likelihood Ratio test with parameter $\lambda$. Assume the probability of Type I error for $\phi_\lambda$ is $\alpha_\lambda$ and the probability of Type II error for $\phi_\lambda$ is $\beta_\lambda$. Let $\phi$ be another test with the probability of Type I error for $\phi$ equal to $\alpha_\lambda$ and the probability of Type II error for $\phi$ equal to $\beta_\lambda$. Then if $\alpha = \alpha_\lambda$ then $\beta \geq \beta_\lambda$, and if if $\alpha < \alpha_\lambda$ then $\beta \gt \beta_\lambda$.

This is slightly more specific than then version given by Ash, but is proved the same way. It says that for a given size test, the Likelihood Ratio Test will be most powerful.

We say that a test $\phi$ with error probabilities $\alpha$ and $\beta$ is inadmissible if there is another test $\phi_o$ with error probabilities $\alpha_o \leq \alpha$ and $\beta_o \leq \beta$ and either $\alpha_o < \alpha$or $\beta_o < \beta$. We say that a test is admissible if it is not inadmissible. These definitions can be extended to composite hypotheses. It is clear that we don't want to use inadmissible tests.

Theorem 226

Every Likelihood Ratio Test is admissible.

This is a direct consequence of the Neyman-Pearson Lemma.

Theorem 228

If $\phi$ is an admissible test then there is a Likelihood Ratio Test with the same error probabilities.

For a given pair H₀ and H_a we can consider the set of all possible tests. If $\phi$ is one of these tests, and $\alpha(\phi)$ and $\beta(\phi)$ are its error probabilities, the ordered pair $(\alpha(\phi),\beta(\phi))$ is called a risk point. The set of all risk points is called the risk set. We see now that the risk set is the set of risk points of all Likelihood Ratio tests. We see also that it is a convex subset of the unit square [0,1]², and it is symmetric about the point (1/2,1/2). The geometry of the risk set helps us construct various types of tests. For example, for a most powerful test at level $\alpha$ we want to find the risk point (a,b) with $a = \alpha$ and the smallest value of b. Given such (a,b) we can construct a Likelihood Ratio test with error probabilities $\alpha$ and b.

Another popular test is a bf minimax test in which we choose at test whose risk point (a,b) minimizes the function $\min(\max(a,b))$. A minimax test must have a = b if it is to be admissible, so we have to find the risk point of the form (a,a) with the smallest value of a. We need to find where the line y=x intersects the ``southwest'' boundary of the risk set.