Lecture 30: Convergence to Standard Normal

A great many statistical tests are based on the standard normal distribution, and go along the following lines. A statistic is decided upon, such as the number of runs. Its theoretical mean and variance are computed under the null hypothesis. The statistic is normalized by subtracting the theoretical mean and dividing by the theoretical standard deviation. It is then proved that as some parameters diverge to infinity, the distribution functions of the statistics converge to the standard normal distribution, and for certain ranges of parameters, the distributions are well approximated by the standard normal.

How are such theorems proven? Let us look at the most famous case, the DeMoivre-Laplace limit theorem for symmetric binomial distributions. This theorem is the forerunner of the Central Limit Theorem.

Before doing so, we will make some general observations. Suppose that F_n(t) is the distribution function of a random variable X_n with mean and standard deviation 1. We are trying to see if F_n(t) converges to

$$\int_{-\infty}^t \frac{1}{\sqrt{2\pi}}\exp(-u^2/2)\;du$$

as $n\rightarrow\infty$. This already poses technical difficulties as the integral above is an inproper integral, and as such, is a limit of a limit. So our first step is to try to show that for all a < b

$$\lim_{n\rightarrow\infty}F_n(b)-F_n(a) = \int_{a}^b \frac{1}{\sqrt{2\pi}}\exp(-u^2/2)\;du$$

Our second simpification will be to try to look only at those a and b where the F_n are continuous. Monotinicity properties of distribution functions will take care of the other values.

Now we see that if the X_n are discrete random variables, there is some hope of proving such results, since the lefthand side is a limit of sums of positive terms which are less than 1, and so is the righthand side. Thus the terms on the left hand side do have the form $(2\pi)^{-1/2}\exp(-u^2/2)\Delta u$.Now on to a specific example. Suppose that X_n has the binomial distributions with

$$\Pr(X_n = k) = \mbox{$\left(\begin{array} {c}2n\\ k\end{array}\right)$}\left(\frac{1}{2}\right)^{2n}.$$

We have ${\rm E}[X_n] = n$ and ${\rm Var}[X_n] = n/2$.Put

$$T_n = \frac{X_n-n}{\sqrt{n/2}}$$

and

$$F_n(t) = \Pr(T_n \leq t) = \Pr(X_n \leq n + t\sqrt{n/2}).$$

From considerations of symmetry it will be sufficient to show

$$\lim_{n\rightarrow\infty} F_n(b)-F_n(0) = \int_0^b \frac{1}{\sqrt{2\pi}}\exp(-u^2/2)\;du$$

where b > 0.

We now adopt the convention that if $b\sqrt{n/2}$ is not an integer then interpret it as the greatest integer in $b\sqrt{n/2}$.

We have

$$F_n(b) - F_n(0) = \sum_{k=n+1}^{n+b\sqrt{n/2}}\mbox{$\left(\... ...y} {c}2n\\ n+k\end{array}\right)$}\left(\frac{1}{2}\right)^{2n}$$

We can write

$$\mbox{$\left(\begin{array} {c}2n\\ n+k\end{array}\right)$} =... ...s(n+1)}\mbox{$\left(\begin{array} {c}2n\\ n\end{array}\right)$}$$

Recall that Stirling's Formula says that

$$\lim_{H\rightarrow\infty}\frac{H!}{H^He^{-H}\sqrt{2\pi H}} = 1.$$

Therefore we should write

$$\mbox{$\left(\begin{array} {c}2n\\ n\end{array}\right)$} = \... ...-n}\sqrt{2\pi n})^2} = A(n)\frac{2^{2n}}{\sqrt{2\pi}\sqrt{n/2}}$$

where

$$A(n) = \frac{(2n)!(n^ne^{-n}\sqrt{2\pi n})^2}{(n!)^2(2n)^{2n}e^{-2n}\sqrt{4\pi n}}.$$

Note that Stirling's Formula guarentees that $A(n)\rightarrow 1$ as $n\rightarrow\infty$, and it is easy to check that A(n) is increasing in n by computing A(n+1)/A(n) and seeing that it is larger than 1. For example, $A(32) \approx 0.9961$. So now we can write

$$F_n(b)-F_n(0) = \sum_{k=1}^{b\sqrt{n/2}} A(n)\frac{n(n-1)\cd... ...-1))}{(n+k)(n+k-1)\cdots(n+1)} \frac{1}{\sqrt{2\pi}\sqrt{n/2}}.$$

So far, so good. We see the $1/\sqrt{2\pi}$, and that we can ignore A(n). If we let $\Delta u = 1/\sqrt{n/2}$ we see that we have the makings of a Riemann sum for dividing up the interval [0,b] into $b\sqrt{n/2}$ parts, each of width $\Delta u$. The problem remains to identify the integrand, which must come from the expression

$$\frac{n(n-1)\cdots(n-(k-1))}{(n+k)(n+k-1)\cdots(n+1)} = \fr... ... {\left(1+\frac{k}{n}\right)\cdots\left(1+\frac{k+1}{n}\right)}$$

Now from Taylor's Theorem we know that for |x| < 1/2 we have

$$\log(1+x) = x + C_xx^2$$

where $\vert C_x\vert < C < \infty$ independent of x. We will apply this with $x = \pm j/n$ where $0 \leq j \leq b\sqrt{n/2}$ and n large enough to get |x| < 1/2. We can then write

1+x = e^xe^C_xx2

and get

$$\frac{\left(1-\frac{1}{n}\right)\cdots\left(1-\frac{k-1}{n}\... ...(-\frac{(k-1)k}{2n}-\frac{k(k+1)}{2n} + C'(k,n)\frac{k^3}{n^2})$$

and

$$\exp(-\frac{(k-1)k}{2n}-\frac{k(k+1)}{2n} + C'(k,n)\frac{k^3}{n^2}) = \exp(-\frac{k^2}{n})\exp(C'(k,n)\frac{k^3}{n^2})$$

where C'(k,n) is bounded in n and k so long as $k\leq b\sqrt{n/2}$.This shows us that for the range of k we have,

$$\exp(C'(k,n)\frac{k^3}{n^2})\rightarrow 1$$

as $n\rightarrow\infty$, uniformly in k. Therefore we can ignore this term just as we did A(n) and write

$$\exp(-\frac{k^2}{n}) = \exp(-\left(\frac{k}{\sqrt{n}}\right)^2) = \exp(-(k\Delta u)^2/2)$$

giving us

$$F_n(b)-F_n(0) = \sum_{k=1}^{b\sqrt{n/2}} A(n)B(n) \frac{1}{\sqrt{2\pi}} \exp(-(k\Delta u)^2/2)\Delta u$$

where $A(n)B(n)\rightarrow 1$ as $n\rightarrow\infty$, so that

$$\lim_{n\rightarrow\infty}F_n(b)-F_n(0) = \int_0^b\frac{1}{\sqrt{2\pi}}\exp(-u^2/2)\;du.$$