Lecture 15: Transformations of Random Vectors

Considered together, the maximum likelihood estimates of the mean and variance of a normal distribution constitute a vector valued statistic. We will now look at trying to compute the distribution functions of such statistics. We shall confine ourselves to the case where the original random variables have a distribution given by a density. The computations are greatly aided by the change of variables theorem for multiple integrals.

Change of variables in multiple integrals

Recall from calculus that if $\phi:[a,b]\rightarrow(-\infty,\infty)$ is an increasing continuous function and $\phi$ is differentiable on (a,b) then

$$\int_{\phi(a)}^{\phi(b)}f(x)\;dx = \int_a^b f(\phi(u))\phi'(u)\;du$$

while if $\phi:[a,b]\rightarrow[c,d]$ is a decreasing continuous function and $\phi$ is differentiable on (a,b) then

$$\int_{\phi(b)}^{\phi(a)}f(x)\;dx = \int_b^a f(\phi(u))\phi'(u)\;du = \int_a^b f(\phi(u))(-\phi'(u))\;du.$$

These two formulae can be combined into one if one adopts the convention that $\phi([a,b])$ stands for the interval whose endpoints are $\phi(a)$ and $\phi(b)$. Since $\phi$ decreasing means $\phi'(u)\leq 0$, and $\phi$increasing means $\phi'(u) \geq 0$, we can just write

$$\int_{\phi([a,b])}f(x)\;dx = \int_{[a,b]} f(\phi(u))\vert\phi'(u)\vert\;du.$$

For example, consider

$$\int_0^{1/2}\sqrt{1-x^2}\;dx = \int_{[0,1/2]}\sqrt{1-x^2}\;dx$$

Note that $\cos:[\pi/3,\pi/2]\rightarrow[0,1/2]$ and $\cos$ is decreasing. In calculus you learned to write

$$\int_0^{1/2}\sqrt{1-x^2}\;dx = \int_{\pi/2}^{\pi/3}\sqrt{1-\... ...t))\;dt = \int_{\pi/3}^{\pi/2}\sqrt{1-\cos^2(t)}(\sin(t))\;dt.$$

Simply observe that on the interval $[\pi/3,\pi/2]$, $\vert\cos'(u)\vert = \sin(u)$and that our new version of the change of variables theorem eliminates one step:

$$\int_{[0,1/2]}\sqrt{1-x^2}\;dx = \int_{[\pi/3 ,\pi/2]}\sqrt{1-\cos^2(t)}(\sin(t))\;dt.$$

If we had made a sine substitution, the formulae are equally efficient.

We want to apply this technique to multiple integrals. What makes it work is that when we divide up [a,b] into subintervals, the function $\phi$ tells us how to divide up $\phi([a,b])$ into intervals, by mapping endpoints to endpoints. The appearance of the factor $\phi'$ reminds us that the mean value theorem can be used to get the length of each new subinterval in terms of the old one, since it tells us that

$$\vert\phi(u_{k+1})-\phi(u_k)\vert = \vert\phi'(u_k^*)\vert\vert u_{k+1}-u_k\vert$$

for each subinterval $[u_k,u_{k+1}]\subset[a,b]$. If we now consider at closed and bounded set $C\in R^d$ and a continuous function $\phi:C\rightarrow R^d$which is differentiable and invertible on the interior of C, we see that

$$\int_{\phi(C)}f(x) dx = \int_C f(\phi(u))\vert\det(\phi'(u))\vert du$$

where $\phi'$ is the $d\times d$ matrix of derivatives of $\phi$, and x and u represent d-tuples.

Here is a familiar example. Suppose that $D = \{(x,y): x^2 + y^2 \leq A^2\}$,that is, the disk in the plane of radius A. We want to integrate some function f(x,y) over D. If we put $C = \{(r,\theta):r\in[0,A], \theta \in [0,2\pi]\}$,and $\phi(r,\theta) = (r\cos(\theta),r\sin(\theta))$, then $\phi(C) = D$, and $\phi$ satisfies the conditions of the change of variables theorem.

$$\phi'(r,\theta) = \left[\begin{array} {cc} \cos(\theta) & -r\sin(\theta)\ \sin(\theta) & r\cos(\theta)\end{array}\right]$$

which has determinant $r \geq 0$. Hence the change of variables formula says

$$\int_D f(x,y)\;dxdy = \int_C f(r\cos(\theta),r\sin(\theta))\;r\;drd\theta.$$

Since C is a rectangle, the latter integral is easily rewritten as an iterated integral:

$$\int_C f(r\cos(\theta),r\sin(\theta))\;r\;drd\theta = \int_... ...ft(\int_0^A f(r\cos(\theta),r\sin(\theta))\;r\;dr\right)d\theta$$

yielding the familiar polar coordinate formula for evaluating double integrals.

Some applications in statistics

Problem 1: Suppose that R and S are independent gamma random variables with $\alpha$equal r and s respectively, and $\beta = 1$. Put

$$T(R,S) = \left(R+S,\frac{R}{R+S}\right).$$

Let us find the distribution and density of T(R,S).

Solution: Let f_R and f_S denote the densities of R and S, respectively. We want to compute

$$\Pr\left(R+S \leq a, \frac{R}{R+S}\leq b\right).$$

Let the vector valued random variable (U,V) be defined by

$$(U,V) = \left(R+S,\frac{R}{R+S}\right) \equiv T(R,S)$$

Note that since R and S take only non-negative values, the range of T is is the rectangle $C\equiv[0,\infty)\times [0,1]$. The inverse of T, call it $\phi$ is given by

$$\phi(u,v) = (uv,u(1-v))$$

Therefore, for a > 0 and $b\in (0,1)$,

$$\begin{array} {rcl} \left\{R+S \leq a \frac{R}{R+S}\leq b\ri... ...(\omega),S(\omega))\in \{(x,y):(x,y) \in \phi(C) \}\end{array}$$

Therefore,

$$\Pr\left(R+S \leq a \frac{R}{R+S}\leq b\right) = \int_{\phi(C)}f_R(x)f_S(y)dx\;dy$$

and by applying the change of variables theorem we get

$$\Pr\left(R+S \leq a \frac{R}{R+S}\leq b\right) = \int_C f_R(uv)f_S(u(1-v))\vert\det( \phi'(u,v))\vert\;du\;dv.$$

Since C is a rectangle with sides parallel to the coordinate axes, this last integral is easily written as an iterated integral and evaluated as soon as we evaluate $\det( \phi'(u,v))\vert$. This is easy as

$$\phi'(u,v) = \left[\begin{array} {cc} v & u\ 1-v & -u\end{array}\right]$$

so $\det( \phi'(u,v))\vert = \vert-u\vert = u$ since we know $u\geq 0$. Using the explicit formulae for the gamma densities we get

$$f_R(uv)f_S(u(1-v))\vert\det( \phi'(u,v))\vert = \frac{(uv)^{r-1}\exp(-uv)(u(1-v))^{s-1}\exp(-u(1-v))}{\Gamma(r)\Gamma(s)}u$$

which simplifies to

$$\frac{B(r,s)\Gamma(r+s)}{\Gamma(r)\Gamma(s)} \frac{u^{r+s-1}\exp(-u)}{\Gamma(r+s)}\frac{v^{r-1}(1-v)^{s-1}}{B(r,s)}$$

for $(u,v)\in C$. Therefore

$$\Pr\left(R+S \leq a \frac{R}{R+S}\leq b\right) = \frac{B(r,s... ...{\Gamma(r+s)}\;du \int_0^b\frac{v^{r-1}(1-v)^{s-1}}{B(r,s)}\;dv$$

simultaneously showing

• $${\frac{B(r,s)\Gamma(r+s)}{\Gamma(r)\Gamma(s)}} = 1$$;
• R + S has a gamma distribution;
• $${\frac{R}{R+S}}$$ has a beta distribution;
• R + S and $${\frac{R}{R+S}}$$ are independent.

Problem 2: Suppose that R and S are independent random variables, each with a standard normal distribution. Let t be a fixed real number, and define the random vector (U,V) as the rotations of the random vector (R,S) through the angle t. Discuss the properties of the random vector (U,V).

Solution: Let f_R and f_S denote the densities of R and S, respectively. We want to compute

$$\Pr(U \leq a, V\leq b)$$

where

$$(U,V) = (\cos(t)R+\sin(t)S,-\sin(t)R+\cos(t)S) \equiv T(R,S)$$

Note that since R and S take any real values, the range of T is is the rectangle the whole plane. The inverse of T, call it $\phi$ is given by

$$\phi(u,v) = (\cos(t)u-\sin(t)v,\sin(t)u + \cos(t)v).$$

Therefore,

$$\begin{array} {rcl} \{U \leq a, V\leq b\} & = & \{T(R,S)\in(... ...(\omega),S(\omega))\in \{(x,y):(x,y) \in \phi(Q) \}\end{array}$$

where Q is the quadrant $(-\infty, a]\times (-\infty, b]$.Since

$$\Pr(U \leq a, V\leq b) = \int_{\phi(Q)}f_R(x)f_S(y)dx\;dy$$

the change of variables theorem implies

$$\Pr(U \leq a, V\leq b) = \int_Q f_R(\cos(t)u-\sin(t)v)f_S(\sin(t)u+\cos(t)v)\vert\det( \phi'(u,v))\vert\;du\;dv.$$

Since Q is a quadrant with sides parallel to the coordinate axes, this last integral is easily written as an iterated integral and evaluated as soon as we evaluate $\det( \phi'(u,v))\vert$. This is easy as

$$\phi'(u,v) = \left[\begin{array} {cc} \cos(t) & -\sin(t)\ \sin(t) & \cos(t)\end{array}\right]$$

so $\det( \phi'(u,v))\vert = 1$. Furthermore,

$$f_R(\cos(t)u-\sin(t)v)f_S(\sin(t)u+\cos(t)v) = \frac{1}{2\pi}\exp(-(u^2 + v^2)/2)$$

so

$$\Pr(U \leq a, V\leq b) = \int_{-\infty}^a\frac{\exp(-u^2)}{\sqrt{2\pi}}du \int_{-\infty}^b\frac{\exp(-v^2)}{\sqrt{2\pi}}dv.$$

This shows that U and V are also independent random variables with the standard normal distribution.

The preceding are examples of the following corollary to the change of variables theorem. It gives us a formula for the density of the transformation of a random vector V by the function g in terms of the density of V and the inverse of g.
Theorem Suppose that V is a k dimensional vector valued random variable with density f_V and S is an open subset of R^k such that $\Pr(V \in S) = 1$. Suppose that $g:S\rightarrow R^k$ is one-to-one and has a continous invertible first derivative on S. Let h be the inverse of g. Then the density of the vector random variable g(V) is given by

$$f_{g(V)}(\vec{u}) = f_V(h(\vec{u}))\vert\det(h'(\vec{u}))\vert$$

Here is a very interesting example. Suppose that V is a random sample of size k from a normal population with mean 0 and variance $\sigma^2$. Let O be any $k\times k$ matrix with the property that O^tO = I. Let us find the density of V0. The density of V is

$$f_V(x_1,\dots, x_k) = \left(\frac{1}{\sqrt{2\pi\sigma^2}}\ri... ...t{2\pi\sigma^2}}\right)^k\exp(-\vec{x}\cdot\vec{x}/(2\sigma^2))$$

where $\vec{x}$ is $(x_1,\dots,x_k)$ and $\cdot$ means dot product. Here g(V) = OV so h(Y) = YO^-1 = YO^t, so h'(Y) = O^t. Since O^t = O^-1 we see that $\vert\det(h'(\vec{u}))\vert = 1$ and $h(\vec{u})\cdot h(\vec{u}) = \vec{u}\cdot \vec{u}$, so

$$f_V(h(\vec{u}))\vert\det(h'(\vec{u}))\vert = \left(\frac{1}{... ...\sigma^2}}\right)^k\exp(-(u_1^2 + \cdots + u_k^2)/(2\sigma^2) )$$

so we see that the components of VO are again a random sample of size k from a normal population with mean 0 and variance $\sigma^2$.