Lecture 8: Distribution functions

Not all random variables are discrete, nor, for modeling purposes, would we want to restrict our attention to discrete random variables. If we return, for a moment, to the problem of modeling ``draw a real number at random from [0,1]'' we noted that given any interval ${\cal I}\subset[0,1]$, the event that the number drawn is in $\cal I$ should have probability equal to the length of $\cal I$. Hence we want a probability space where $\Omega = [0,1]$, where the sigma field contains all subintervals of [0,1], and where the probability measure of an interval is its length. It is shown in courses on measure theory that such a probability space exists. (See, for example, Real and Complex Analysis by Walter Rudin.) The relevant random variable for this this example is $R(\omega) = \omega$. Here the range of R is all of [0,1], and as we explained earlier,

$$p_R(y) = \Pr(\{\omega:R(\omega) = y\}) = 0$$

for every y! Hence there is nothing to be gained from looking at the probability mass function. The solution is look at something different, called the distribution function of R, denoted by F_R, defined by

$$F_R(t) = \Pr(\{\omega:R(\omega) \leq t\}).$$

Every random variable has a distribution function, and all the information about probablities involving the random variable can be derived from its distribution function.

For the moment, observe that

$$\Pr(R \in (a,b]) = \Pr(R \leq b) - \Pr(R \leq a) = F_R(b) - F_R(a).$$

Also notice that any distribution function is non-decreasing and has range contained in $(-\infty,\infty)$.

We will have to have another look at properties of probability measures to show how to use the distribution function to compute $\Pr(R \in {\cal I})$ for any interval.

Continuity properties of probability measures and distribution functions

Consider a sequence of events, $A_1 \subset A_2 \subset A_3\cdots$. We have

$$\bigcup_{j=1}^N A_j = A_N.$$

Define the limit of these sets, denoted by $\lim_{j\rightarrow \infty}A_j$ by

$$\lim_{j\rightarrow \infty}A_j = \bigcup_{j=1}^\infty A_j.$$

One justification for considering this to be a limit is that the sets A_n expand out to give us all of the union of the A_n. The following is an important consequence of the additivity axiom for probability measures:
If $A_1 \subset A_2 \subset A_3\cdots$, then

$$\lim_{n\rightarrow\infty}\Pr(A_n) = \Pr(\lim_{n\rightarrow\infty}A_n).$$

This is called the continuity property of probability measures, and is in fact, equivalent to the additivity axiom. The proof is relatively straightforward.

Put B₁=A₁ and B_n = A_n-A_n-1 for n > 1. The events B_n are disjoint and for each positive integer n

$$A_n = B_1 \cup B_2 \cup \cdots \cup B_n$$

From this we see that

$$\lim_{n\rightarrow\infty}A_n = B_1 \cup B_2 \cup \cdots.$$

Therefore

$$\begin{array} {rcl} \Pr(\lim_{n\rightarrow\infty}A_n) & = & ... ...s\cup B_n)\ & = & \lim_{n\rightarrow\infty}\Pr(A_n)\end{array}$$

as we claimed.

It is also the case that if we have events $\cdots C_n\subset C_{n-1}\subset\cdots\subset C_1$ and we define

$$\lim_{n\rightarrow} C_n = C_1\cap C_2 \cap \cdots$$

then

$$\lim_{\rightarrow\infty}\Pr(C_n) = \Pr(\lim_{n\rightarrow} C_n)$$

This is easily derived from the previous result by using Boole's identities.

These results are used to investigate the continuity of distribution functions. Let F_R be the distribution function of the random variable R. Since distribution functions are non-decreasing they have left and right hand limits at all points. To see if F_R is continuous at t it is suffient to check to see if

$$\begin{array} {c} \lim_{n\rightarrow\infty} F_R(t + n^{-1})\ \lim_{n\rightarrow\infty} F_R(t - n^{-1})\end{array}$$

converge to F_R(t) when n takes only integer values. Fix a value t and let

$$C_n = R \in (-\infty,t+n^{-1}]$$

Observe that these sets are a nested decreasing sequence of events, and

$$\lim_{n\rightarrow\infty}C_n \equiv C_1\cap C_2 \cap C_3 \cap \cdots = R \leq t.$$

Therefore

$$\lim_{n\rightarrow\infty}F_R(t+n^{-1}) = \lim{n\rightarrow\i... ...) = \Pr(\lim_{n\rightarrow\infty}C_n) = \Pr(R \leq t) = F_R(t),$$

so a distribution function is always continuous from the right.

Now put

$$A_n = R \in (-\infty,t-n^{-1}]$$

Observe that these sets are a nested increasing sequence of events, and

$$\lim_{n\rightarrow\infty}A_n \equiv A_1\cap A_2 \cap A_3 \cap \cdots = R < t.$$

Therefore

$$\lim_{n\rightarrow\infty}F_R(t-n^{-1}) = \lim{n\rightarrow\infty}\Pr(A_n) = \Pr(\lim_{n\rightarrow\infty}A_n) = \Pr(R < t),$$

and since

$$\begin{array} {rcl} R \leq t &=& (R < t)\cup (R = t)\ F_R(t) &=& \Pr(R<t) + p_R(t)\end{array}$$

we have

$$\lim_{x\rightarrow t^-}F_R(t) = F_R(t) - p_R(t)$$

so that F_R is continuous from the right at t if and only if the probability that R equals t is zero. Therefore, F_R is continuous at t if and only if the probability that R equals t is zero, and the jump in F_R at t is equal to the probability that R equals t. The values of t for which F_R jumps are called the atoms of the distribution of R. The limit from below at t of F_R is denoted F_R(t^-). With this notation we have $\Pr(R < t) = F_R(t^-)$. Therefore,

$$\begin{array} {rcl} \Pr(a < R \leq b) & = & F_R(b) - F_R(a)\... ...(a)\ \Pr(a \leq R \leq b) & = & F_R(b^-) - F_R(a^-)\end{array}$$

so we have represented the probability that R is in any type of interval in terms of the distribution function of R.

In fact, a distribution function F is characterized by the following properties

• $${\lim_{t\rightarrow -\infty}F(t) = 0}$$and $${\lim_{t\rightarrow \infty}F(t) = 1}$$;
• F is non-decreasing;
• F is continuous from the right with lefthand limits.

It can be shown that for any function F with these properties there is a probability model $(\Omega,{\cal F},\Pr)$ and and a random variable R so that F is the distribution function of R.

Independence of random variables in general

We can show that any pair of random variables R and S is independent if for any pair of real numbers x and y,

$$\Pr(R < x, S < y) = F_R(x)F_S(y)$$

The function of two variables, F_R,S defined by

$$F_{R,S}(x,y) = \Pr(R < x, S < y)$$

is called the joint distribution function of R and S. More generally, if $V = (V_1, \cdots, V_n)$ is a vector-valued random variable, its joint distribution function, F_V, is the function on $(-\infty,\infty)^n$ defined by

$$F_V(x) = \Pr(V_1 \leq x_1, \cdots, V_n \leq x_n).$$

It can be shown that the components of V are independent if and only if

$$F_V(x) = \Pr(V_1 \leq x_1, \cdots, V_n \leq x_n) = \prod_{j=1}^n \Pr(V_j \leq x_j) = \prod_{j=1}^n F_{V_j}(x_j).$$