Lecture 12: Statistics and Functions of Vector Valued Random Variable

We have now seen that computing maximum likelihood estimators of parameters leads to function of the data. For example, in the case of random sample of size N with normally distributed components, if

$$V = (V_1,\dots,V_N) = (x_1,\dots,x_N)$$

then the maximum likelihood estimate of $(\mu,\sigma^2)$ was given by

$$\left(\frac{1}{N}\sum_{j=1}^N x_j,\frac{1}{N}\sum_{k=1}^N x_j^2 - \left(\frac{1}{N}\sum_{j=1}^N x_j\right)^2\right)$$

Since each x_j is a value of the corresponding random variable V_j, we have constructed some new random variables:

$$\frac{1}{N}\sum_{j=1}^N V_j$$

and

$$\frac{1}{N}\sum_{k=1}^N V_j^2 - \left(\frac{1}{N}\sum_{j=1}^N V_j\right)^2.$$

More generally, if we have any random sample V, a statistic based on V is simply a function of V which is a random variable. Naturally we want to study interesting statistics based on V, but we also need to remember that the term statistic has this broader meaning.

Unbiased Statistics

Suppose the problem at hand is to collect some data to estimate a quantity z. A statistic T based on the random sample V is called unbiased for z if

E[T(V)] = z.

For example, the maximum likelihood estimate of $\mu$ in the case of normally distributed random samples is unbiased, since

$$T(V) = \frac{1}{N}\sum_{j=1}^N V_j$$

and

$$E[T(V)] = E\left[\frac{1}{N}\sum_{j=1}^N V_j\right] = \frac{1}{N}\sum_{j=1}^N E[V_j] = \frac{1}{N}\sum_{j=1}^N \mu = \mu$$