Lecture 14: More Functions of Random Variables

Since a statistic is a function of a vector valued random variable, it will be useful to be able to find the distribution function, density or probability mass function of a function of a vector valued random variable. The general rule of thumb is that if the vector valued random variable has a density it is best to try to find the distribution function of the statistic in terms of this density. If the vector valued random variable is discrete, one should try to find the probability mass function of the statistic. Rather than preceding with generalities, let us look at some examples.

Example 1: Suppose that R has a density r_R and that S = aR + b, where a > 0. Express the density of S in terms of the density of R.

Solution: We express the distribution function of S in terms of the distribution function of R:

$$\Pr(S \leq y) = \Pr(aR + b \leq y) = \Pr(R \leq (y-b)/a) = \int_{-\infty}^{(y-b)/a}f_R(x)\;dx$$

and now we change variables in the integral to $x = (z-b)/a, dx = (1/a)\;dz$to get

$$\Pr(S \leq y) = \int_{-\infty}^{y}f_R((z-b)/a)(1/a)\;dz$$

so that we see that the density of S is (1/a)f_R((z-b)/a). For example, if R has a Gamma density with $\beta = 1$, then for any b > 0, bR has a gamma density with $\beta = b$.

Example 2: Suppose that R and S have gamma distributions with a common value of $\beta$. If R and S are independent find the density of R + S.

Solution: Our first example shows we might as well assume that the commmon value is 1. Then the vector random variable V = (R, S) has the density

$$f_V(x,y) = x^{r-1}\exp(-x)/\Gamma(r)y^{s-1}\exp(-y)/\Gamma(s)$$

if x > 0 and y > 0, and is zero otherwise. We propose to compute $\Pr(R + S \leq z)$ for any positive number z. Recall that $R + S \leq z$ is shorthand for

$$\{\omega: (R(\omega),S(\omega))\in\{(x,y) : x + y \leq z\}\}$$

so we have to compute

$$\int\int_{\{(x,y) : x + y \leq z\}}f_V(x,y)\;dA = \int_{-\in... ...infty}^\infty\left(\int_{-\infty}^{z} f_V(x,u-x)\;du\right)\;dx$$

by making the change of variables y = u - x in the inner integral. Since we can now easily invert the order of integration we have

$$\int\int_{\{(x,y) : x + y \leq z\}}f_V(x,y)\;dA = \int_{-\infty}^z\left(\int_{-\infty}^{\infty} f_V(x,u-x)\;dx\right)\;du$$

from which we see that the density of R + S must be given by  

$$f_{R+S}(u) = \int_{-\infty}^{\infty} f_V(x,u-x)\;dx.$$ (1)

So far we have used nothing about R and S save that they have a joint density, so (1) holds for all ordered pairs of random variables V with joint density f_V. Since we have additional information, we can continue with

$$f_{R+S}(u) = \frac{1}{\Gamma(r)\Gamma(s)}\int_{0}^{u}x^{r-1}... ...c{\exp(-u)}{\Gamma(r)\Gamma(s)}\int_{0}^{u}x^{r-1}(u-x)^{s-1}dx$$

The remaining integral is just the Beta function in disguise. Let $x = uw, dx = u\;dw$ and we get

$$\int_{0}^{u}x^{r-1}(u-x)^{s-1}dx = \int_0^1 u^{r+s-1}w^{r-1}... ...r+s-1}B(r,s) = u^{r+s-1}\frac{\Gamma(r)\Gamma(s)}{\Gamma(r+s)}$$

so

$$f_{R+S}(u) = u^{r+s-1}e^{-u}/\Gamma(r+s).$$

In other words, Gamma densities add if the ``$\beta$'' parameters are equal!

Here is an important application of this observation.

The Poisson process

Suppose that events occur at at random times, and the times between events are independent random variables, each with an exponential distribution with mean 1/b. Let N_t be the number of events observed by time t assuming that no events have been observed at time 0. Find the probability mass function of N_t.

Let $S_1, S_2, \dots$ be the times between the arrivals. The key is the observation that for any non-negative integer k,

$$\{N_t \leq k\} = \{S_1 + S_2 +\cdots +S_{k+1} \gt t\}.$$

so  

$$\Pr(N_t \leq k) = \Pr(S_1 + S_2 +\cdots +S_{k+1} \gt t).$$ (2)

Since the S_j are independent and exponential we know that the sum $S_1 + \dots + S_{k+1}$ has a gamma distribution with $\alpha = k+1$ and $\beta =1/b$we have

$$\Pr(S_1 + \cdots S_{k+1} \gt t) = \int_t^\infty \frac{b^{k+1}u^k\exp(bu)}{\Gamma(k+1)} \;du.$$

Now one integration by parts gives

$$\Pr(S_1 + \cdots S_{k+1} \gt t) = \frac{(bt)^k\exp(-bt)}{k!} + \int_t^\infty \frac{b^{k}u^{k-1}\exp(bu)}{\Gamma(k)} \;du,$$

so applying (2) we see

$$\Pr(N_t \leq k) = \frac{(bt)^k\exp(-bt)}{k!} + \Pr(N_t \leq k-1).$$

Since N_t only takes integer values we see

$$\Pr(N_t = k) = \Pr(N_t \leq k) - \Pr(N_t \leq k-1) = \frac{(bt)^k\exp(-bt)}{k!}$$

so that N_t has the so-called Poisson distribution.

Example 3: Suppose that $V = (V_1,\dots,V_N)$ is random sample from a distribution with distribution function F. Put $M(V) = \max(V_1,\dots,V_N)$. Find the distribution function of M(V).

Solution: Since V is a random sample its components are independent and each component has F as its distribution function. Therefore

$$F_{M(V)}(t) = \Pr(\max(V_1,\dots,V_N) \leq t) = \Pr(V_1 \leq t,\dots,V_N\leq t) = F(t)^N.$$

Application: Suppose that V is a random sample from a uniform distribution on (1, b). M(V) is a reasonable estimate of b. Determine if M(V) is unbiased for b.

We need to compute E[M(V)]. To do so we need the density for V to avoid using the Law of the Unconscious Statistician and an N dimensional integral. It is clear that the density of M(V) is zero off of the interval (1,b). On the interval (1,b) the distribution function of each component of V is given by F(t) = (t-1)/(b-1), so the distribution function of M(V) given by

$$F_{M(V)}(t) = \left(\frac{t-1}{b-1}\right)^N$$

on (1,b). Hence the density of M(V) is given by

$$f_{M(V)}(t) = N\left(\frac{t-1}{b-1}\right)^{N-1}\frac{1}{b-1}$$

on (1,b). Therefore

$$E[M(V)] = \int_0^b tN\left(\frac{t-1}{b-1}\right)^{N-1}\frac{1}{b-1}\;dt = \frac{N}{N+1}b + \frac{1}{N+1}$$

Hence M(V) is only assymptotically unbiased for b.

$$T(V) = \frac{N+1}{N}M(V) - \frac{1}{N}$$

is an unbiased estimate of b. What is more, we can compute the variance of T(V) and show that it goes to 0 like 1/N.