Lecture 24: Bayes Tests

We will present here one way to find a test of hypothesis which has an optimal property.

Suppose that I take a sample of size 1 from one of two populations, with density functions f₀ and f₁ respectively. Let the null hypothesis be that the sample is from f₀, that is $H_0 = \{f_0\}$, and the alternative hypothesis be that the sample is from f₁, that is $H_1 = \{f_1\}$. We define a decision function $\Phi$ in the following way. If x is observed then $\Phi(x) \in [0,1]$, where we interpret $\Phi(x)$ to be the probability that reject the null hypothesis if x is observed. To implement this, we act as follows:

• If $\Phi(x) = 1$ we reject H₀ whenever x is observed.
• If $\Phi(x) = 0$ we accept H₀ whenever x is observed.
• If $\Phi(x) \in (0,1)$ toss a coin which comes up heads with probability $\Phi(x)$ once, and if it comes up heads, reject H₀ and if it comes up tails accept H₀.

Our problem is to find a suitable $\Phi$.

First observe that for a given $\Phi$ the probability of Type I error, $\alpha(\Phi)$, is given by

$$\alpha(\Phi) \equiv \int \Phi(x)\;f_0(x)\;dx$$

and the probability of Type II error, $\beta(\Phi)$, is

$$\beta(\Phi) \equiv \int (1-\Phi(x))\;f_1(x)\;dx = 1 - \int \Phi(x)\;f_1(x)\;dx.$$

Now suppose that there is a penalty c₁> 0 for making a Type I error and a penalty c₂ > 0 for making a Type II error, and no penalty for making the right decision. Furthermore, suppose that there is a probability $p \in (0,1)$that the null hypothesis is actually correct. Then the average penalty for using $\Phi$, call it $B(\Phi)$, is

$$B(\Phi) = pc_1\alpha(\Phi) + (1-p)c_2\beta(\Phi).$$

The goal is to find $\Phi$ that gives the smallest penalty $B(\Phi)$. This penalty is called the Bayes Risk associated with $\Phi$, and the best $\Phi$ is called the Bayes Test corresponding to (p,c₁,c₂,f₀,f₁).

We can actually compute the Bayes Test! By using the definitions of $\alpha(\Phi)$ and $\beta(\Phi)$ we can write

$$B(\Phi) = \int \Phi(x)[pc_1f_0(x) - (1-p)c_2f_1(x)]\;dx + (1-p)c_2$$

(See BPT for more details.) We see we have to minimize

$$A(\Phi) \equiv \int \Phi(x)[pc_1f_0(x) - (1-p)c_2f_1(x)]\;dx$$

where $\Phi(x) \in [0,1]$.

Let

$$\begin{array} {rcl} S_- & = & \{x : [pc_1f_0(x) - (1-p)c_2f_... ... S_0 & = & \{x : [pc_1f_0(x) - (1-p)c_2f_1(x)] = 0\}\end{array}$$

The way to make $A(\Phi)$ smallest is to have

$$\Phi(x) = \left\{ \begin{array} {rcl} 1 & {\rm if}& x\in S_-\ 0 & {\rm if}& x\in S_+\end{array}\right.$$

and it does not matter how we define $\Phi(x)$ for $x\in S_0$, so we could take $\Phi(x) = 1$ on S₀ too.

One convenient way to give the Bayes Test is in terms of the likelihood ratio L(x) defined by

$$L(x) = \left\{ \begin{array} {rcl} \displaystyle{\frac{f_1(... ...0(x) \neq 0\ \infty & {\rm if} & f_0(x) = 0.\end{array}\right.$$

Then our test takes the form

• Reject H₀ if

  $$L(x) \gt \frac{pc_1}{(1-p)c_2}.$$

• Accept H₀ if

  $$L(x) < \frac{pc_1}{(1-p)c_2}.$$

• Do whatever you want if

  $$L(x) = \frac{pc_1}{(1-p)c_2}.$$

Tests based on comparing L(x) to a constant are called Likelihood Ratio Tests. One important theorem is that for H₀ and H₁ of the type above, given any $\alpha\in[0,1]$ there is a Likelyhood Ratio Test $\Phi$ with $\alpha(\Phi) = \alpha$.