Lecture 9: Absolutely continuous distribution functions

As we have seen, not all distribution functions are continuous. In fact, the distribution function of a discrete random variable has a jump discontinuity at every point where the probability mass function is not 0, and the size of the jump is the value of the probability mass function.

Let us look at the other extreme. We will say that a random variable, R, has a continuous distribution if its distribution function, F_R, is a continuous function. Thus a random variable has a continuous distribution if and only if its probability mass function is always 0. Therefore, if R has a continuous distribution we have

$$\Pr(R\in (a,b)) = \Pr(R\in[a,b]) = \Pr(R\in(a,b]) = \Pr(R\in[a,b))$$

We will say that the random variable R has an absolutely continuous distribution if there is a function f_R, called the probability density of R, so that for every $t\in (-\infty,\infty)$

$$F_R(t) = \int_{-\infty}^t f_R(u)\;du$$

It is clear from calculus that a random variable with an absolutely continuous distribution is a random variable with a continuous distribution.

Expected value of an absolutely continuous random variable

If R is a random variable with density f_R and

$$\int_{-\infty}^{\infty}\vert u\vert f_R(u)\;du < \infty$$

then we define the expected value of R, denoted by E[R], by

$$E[R] = \int_{-\infty}^{\infty}u\;f_R(u)\;du$$

This is a reasonable extension of our definition of expected value for a discrete random variable. For example, suppose that R has as its density the function f_R where f_R(u) = 0 for $u \in [0,1]^c$ and f_R is continuous. Let R_n be the random variable which is R rounded down to the nearest 10^-n. Clearly R_n increases to R as $n\rightarrow\infty$, so it would be reasonable to define $E[R] = \lim_{n\rightarrow\infty}E[R_n]$. On the other hand,

$$\{R_n = k/10^n\} = \{R \in [k/10^n, (k+1)/10^n)\}$$

so

$$\Pr(R_n = k/10^n) = \Pr(R \in [k/10^n, (k+1)/10^n)) = F_R((k+1)/10^n)-F_R( k/10^n).$$

Therefore

$$E[R_n] = \sum_{k=0}^{10^n-1}\frac{k}{10^n}(F_R((k+1)/10^n)-F_R(k/10^n)) = \sum_{k=0}^{10^n-1}\frac{k}{10^n}f(x_k)(1/10^n)$$

where $x_k \in (k/10^n, (k+1)/10^n)$ is chosen according to the mean value theorem. Then as $n\rightarrow\infty$, we see that

$$\lim_{n\rightarrow\infty}E[R_n] = \lim_{n\rightarrow\infty... ... \int_0^1 u\;f_r(u)\;du = \int_{-\infty}^\infty u\;f_r(u)\;du.$$

As expected, there is a version of the law of the unconscious statistician for random variables with densities:
If R has density f_R, if h(R) is a random variable, and if

$$\int_{-\infty}^\infty \vert h(u)\vert\;f_R(u)\;du < \infty$$

then

$$E[h(R)] = \int_{-\infty}^\infty h(u)\;f_R(u)\;du.$$

The variance and standard deviation of such random variables is defined in the same way as for discrete random variables.

Vector valued random variables

The preceding can be generalized to vector valued random variables. The vector valued random variable $V = (V_1, \dots, V_n)$ is said to have a continuous distribution if its joint distribution function F_V is continuous. It is said to have an absolutely continuous distribution if there is a function f_V, called the joint density, where, with $t = (t_1, \dots, t_n)$,

$$F_V(t) = \Pr(V_1 \leq t_1, \dots, V_n \leq t_n) = \int_{-\i... ...\cdots \int_{-\infty}^{t_n}f_V(x_1,\dots,x_n)\;dx_n\cdots dx_1$$

It follows from this formula that all of the components of V are also absolutely continuous random variables. For example,

$$\Pr(V_1 \leq t) = \int_{-\infty}^t \left( \int_{-\infty}^{\i... ...}^{\infty}f_V(x_1,x_2,\dots,x_n)\;dx_n\cdots dx_2\right)\;dx_1$$

so the density of V₁, f_V1(u) is given by

$$f_{V_1}(u) = \int_{-\infty}^{\infty}\cdots \int_{-\infty}^{\infty}f_V(u,x_2,\dots,x_n)\;dx_n\cdots dx_2.$$

Since, roughly speaking, the density is the derivative of the distribution function, we see that the components of V are independent if and only if

$$f_V(x_1,\dots,x_n) = \prod_{j=1}^n f_{V_j}(x_j).$$