Lecture 2: Sigma Algebras and Probability Measures.

First, some elmentary algebra of sets. If A and B are sets then the intersection of A and B, denoted $A\cap B$, is the set of all x which are both in A and in B, while the union of A and B, denoted $A\cup B$, is the set of all x which are either in A or in B or in $A\cap B$. The operations of intersection and union are both associative and commutative. The relation, A is a subset of B, denoted $A \subset B$ means each element of A is an element of B.

The complement of a set A is a little trickier to define, as we have to keep in mind some set that contains A. For our purposes, we shall always be thinking of some sample space $\Omega$, and when we speak of the complement of A, denoted A^c, we shall mean all those elements of $\Omega$ which are not elements of A. The set containing no elements, called the empty set, is denoted by $\emptyset$. Do not use this symbol for zero.

There is a distributive law for intersection over union. If ${\cal A}$ is an index set then

$$B\bigcap \left(\bigcup_{\alpha\in{\cal A}}A_\alpha \right) = \bigcup_{\alpha\in{\cal A}}\left(B\bigcap A_\alpha \right)$$

There are also the two DeMorgan's laws:

$$\left(\bigcup_{\alpha\in{\cal A}}A_\alpha \right)^c = \bigcap_{\alpha\in{\cal A}}A_\alpha^c,$$

and

$$\left(\bigcap_{\alpha\in{\cal A}}A_\alpha \right)^c = \bigcup_{\alpha\in{\cal A}}A_\alpha^c.$$

For the proofs of these results see BPT or take Math 241.

Algebra with sets is introduced so that we may deal with sets of outcomes from statistical experiments. Some examples of this were given in the first lecture. We noted there that if we were to consider a set of subsets of the sample space it would be important that for each set in our set of subsets, the complement of the subset appear also. Naturally, the entire sample space should be included, and therefore, the empty set as well. We also noted indirectly that we wanted to look at unions of subsets, when we computed the probability of the number of heads lying between 35 and 45 by consider 35, 36, and so on, heads individually. As it turns out, these are really the only properties that we need to assume. We have the following definition: A set of subsets of $\Omega$, denote it by ${\cal F}$, is called a sigma field if

1.

$\Omega \in {\cal F}$;

2.

If $A\in {\cal F}$ then $A^c \in {\cal F}$;

3.

If $A_1, A_2, \dots$ is an infinite sequence of elements of ${\cal F}$then

$$\bigcup_{j=1}^\infty A_j \in {\cal F}.$$

As a consequence of these three properties and general properties of the algebra of sets, there are many other properties that the elements of a sigma field enjoy. For example, it is a consequence of the second property that the empty set is an element of every sigma field. It is then a consequence of the third property and DeMorgan's Laws that the intersection of a countable number of elements of the sigma field is in the sigma field.

Now we can talk about assigning probability. It turns out to be convenient (mathematically) to talk about assigning probability to the elements of the sigma field, called events, rather than to the elements of the sample space. To convince yourself of why, consider the problem of modeling the experiment: ``Choose a real number at random from between 0 and 1.'' If all numbers should have the same probability the only probability that could be assigned would be 0. This would then lead us to the conclusion that any set should have probability 0. Therefore it would be impossible to choose a number!!

What should govern the assignment of probabilities? It is traditional that probabilities be numbers between 0 and 1 inclusive. It is also traditional that the probability of the whole sample space be 1. It is commonly accepted that probability should add if the events do not overlap. This is all we need to assume, and we make the following definition, noting that in mathematical terms, an assignment of probabilities to the sigma field is a function whose domain is the sigma field and whose range is contained in the closed interval [0,1]. A function

$$\Pr:{\cal F} \rightarrow [0,1]$$

is called a probability measure if

1.

$\Pr(\Omega) = 1$;

2.

If $A_1, A_2, \dots$ is a countable sequence of events and $A_i \cap A_j = \emptyset$ for $i\neq j$ then

$$\Pr\left(\bigcup_{j=1}^\infty A_j\right) = \sum_{j=1}^\infty \Pr(A_j)$$

In many ways, measures of probability behave like measures of length, area and volume. These analogies give us clues about how to model experiment such as choosing a real number at random between 0 and 1.

Many other properties that we would expect any reasonable way of measuring probability follow from the three properties we assume. For example:

1.

$\Pr(\emptyset) = 0$.

2.

$\Pr(A\cup B) = \Pr(A) + \Pr(B) - \Pr(A\cap B)$

3.

If $A \subset B$ then $\Pr(A) \leq \Pr(B)$.

4.

If $A_1, A_2, \dots$ is any sequence of events then

$$\Pr\left(\bigcup_{j=1}^\infty A_j\right) \leq \sum_{j=1}^\infty \Pr(A_j)$$

where we adopt the convention that a divergent series of non-negative terms is bigger than any real number.

Recommended reading

• Pages 1 through 14 of BPT.
• Pages 17 through 30 of MSA.