Lecture 26: Attained Significance Levels

Suppose that one had constructed a test of hypotheses for $H_0 = \{\theta = \theta_0\}$ versus $H_1=\{\theta \gt \theta_0\}$ of the form ``accept H<sub>0</sub> if $T(X) < \lambda$ with a level of $\alpha$. That means that if H<sub>0</sub> is true then $\Pr(T(X) \gt \lambda) = \alpha$. Now the data, x, is collected, and T(x) is computed to be $T(x) = \lambda_x$. If $\lambda_x < \lambda$ you report that H<sub>0</sub> is accepted, and if $\lambda_x \gt \lambda$ you report that H<sub>0</sub> is rejected. So far, so good, but the client may want to know if it was a ``close call'' or not. That is, if the level, $\alpha$ were different, might you have rejected or accepted H₀ based on the same data. One way to convey this is report the level of a test of the form ``accept H₀ if $T(X) < \lambda_x$. That is, compute $\Pr(T(X) \geq \lambda_x)$ under the assumtion that the null hypothesis is true. This is called the attained significance level of the test.