Lecture 5: Conditional Probability

Not all events are independent, nor should we want this to be so in our models. For example, if the experiment is to draw two cards from a deck of playing cards, one after the other, the event that the first card is the queen of hearts should not be independent of the event that the second card is the queen of hearts. When we have partial information about the outcomes, we should use this information to reduce the sample space. Since it is mathematically inconvenient to keep redefining the sample space, we simply define a new probability measure which will assign probability 0 to sets that do not not meet the requirements of the partial information. In the extreme example cited above, if we know that the first card drawn is the queen of hearts, we only want to consider sample points whose first element is the queen of hearts. To do this, we want to assign probability 0 to all events which do not intersect the event that the first card is the queen of hearts. Since the probability of the empty set is 0, if we let QH1 be the event that the first card is the queen of hearts, and B is any other event, then the function

$$f(B) = \Pr(B\cap QH1)$$

has the property that f(B) = 0 if B and QH1 have no points in common. f is not a probability measure since

$$f(\Omega) = \Pr(QH1) = 1/52$$

if $\Pr$ is the classical probability measure. We can remedy this defect then by dividing f by $\Pr(QH1)$. This leads us to the following definition:
Suppose that $(\Omega,{\cal F},\Pr)$ is a probability model, and $\Pr(F) \gt 0$.Then the conditional probability of an event B given F, denoted either by $\Pr(B \vert F)$ or by $\Pr_F(B)$ is defined by

$$\Pr(B \vert F) = \frac{\Pr(B\cap F)}{\Pr(F)}.$$

As a function of B, $\Pr_F(B)$ is a probability measure.

Note that we have not defined what it means to condition on an event F whose probability is 0. It will be convenient, however, to adopt the following convention. For all events F, whether they have probability or not,

$$\Pr(B \vert F)\Pr(F) = \Pr(B\cap F).$$

When F has positive probability, this follows from the definition of conditional probability. When F has probability 0, so does $\Pr(B\cap F)$,so the lefthand side is of the form ``something times 0'', and will be taken to be 0 by fiat.

The law of total probability

Suppose that X is a countable set, and $\{F_x,x\in X\}$ is a collection of pairwise disjoint events. Such a collection is called a partition of $\Omega$. If B is any other event then

$$B = \bigcup_{x\in X}(B\cap F_x)$$

from which it follows that

$$\Pr(B) = \sum_{x\in X}\Pr(B\cap F_x).$$

This is called the law of total probability, and is simply another way of formulating our ``divide and conquer'' for computing probabilities. In terms of conditional probabilities, the law of total probability may be written

$$\Pr(B) = \sum_{x\in X}\Pr(B \vert F_x)\Pr(F_x).$$

The most common case is for X to have two elements, so that there is some event A where F₁ = A and F₂ = A^c.

Bayes' Theorem

The following formula, called Bayes' Theorem, turns out to be quite useful. Suppose that $\{F_x,x\in X\}$ is a partition of $\Omega$. Let $a\in X$ be an element of X, and let B be any event with positive probability. Then

$$\Pr(F_a \vert B) \ \frac{\Pr(B \vert F_a)\Pr(F_a)}{\sum_{x\in X}\Pr(B \vert F_x)\Pr(F_x)}$$

Again, the typical application has X a set of two elements, as we now illustrate.

Medical test have what are called false positive and false negative rates, which are usually expressed as conditional probabilities. For example, suppose the test is for HIV positivity. Let TP be the event that the test is positive, let TN be the event that the test is negative, and let HP be the event that the person tested is HIV positive and let HN be the event that the person is HIV negative. The false positive rate is $\Pr(TP \vert HN)$ and the false negative rate is $\Pr(TN \vert HP)$. These rates are are assumed to be known, and (hopefully) relatively small. Let us, for illustration sake, assume that the false positive rate is 0.02, that the false negative rate is 0.01, and that the rate of HIV positivity in the population under consideration is 0.10. The question is, if a person is tested for HIV and the test comes back positive, what is the probability that the person really is HIV positive? In probability language, what is $\Pr(HP \vert TP)$? According to Bayes' Formula,

$$\Pr(HP \vert TP) = \frac{\Pr(TP \vert HP)\Pr(HP)}{\Pr(TP \... ...\Pr(HN) } = \frac{(1-0.01)(0.10)}{(1-0.01)(0.10)+(0.02)(0.90)}$$

which is about 0.846. However, if the rate of HIV in the population is 0.01, then

$$\Pr(HP \vert TP) = 1/3.$$

If we denote the rate of HIV in the population by p then

$$\Pr(HP \vert TP) = \frac{99p}{97p+2}.$$