4.7. Chapter Summary#
The primary purpose of this chapter is to formalize our language and techniques for dealing with random phenomena. Our framework for doing that is a probability space \((S, \mathcal{F}, P)\), which consists of:
A sample space \(S\), which contains all of the outcomes for a random experiment.
An event class, \(\mathcal{F}\), which contains sets of events to which we will assign probability.
A probability measure, \(P\), which is a set function that assigns probabilities to the events. The event class has to be a \(\sigma\)-algebra, which basically means that we can combine any events using set operations and the result will still be in the event class. The properties of the probability measure come from the three Axioms of Probability.
We showed how to calculate probabilities for fair experiments using counting. However, counting the number of outcomes in an event is sometimes nontrivial, and I introduced several counting techniques from combinatorics to help with this.
We will see some of these same formulas again when we introduce random variables in Chapter 8.
Now that we have better terminology and tools, we take a deep dive into null hypothesis testing with resampling in Chapter 5.
Self Assessment Exercises
Answer the questions below to assess your understanding of the material in this chapter. Because there are many questions for this chapter, a random sample of 10 questions are shown. Reload this page to get a new random sample of 10 questions.
Terminology Review
Use the flashcards below to help you review the terminology introduced in this chapter. \(~~~~ ~~~~ ~~~~\)
Spaced Repetition
4.7.1. Key take-aways:#
Probability can be defined in several ways. Three considered in this chapter are:
Using relative frequency
Using analysis for fair experiments
Using Axiomatic Theory
Axiomatic Theory is the most general and powerful of these.
Outcomes, Sample Spaces, and Events
An outcome is a result of a random experiment that cannot be further decomposed.
The sample space of a random experiment is the set of all outcomes.
An event is a set of outcomes. The event occurs if any of the outcomes in that set occcurs.
Relative Frequencies and Probabilities
A compound experiment consists of multiple sub-experiments.
A trial is one of the sub-experiments in a compound experiment.
A repeated experiment is a compound experiment in which the trials are identical and independent of each other.
The relative frequency of an event is defined over some number of repeated trials. It is the proportion of times that event is seen during those trials.
Relative frequencies take values between 0 and 1. The relative frequencies of the outcomes of an experiment sum to 1.
An experiment possesses statistical regularity if the relative frequencies converge to some fixed values.
If the relative frequencies converge to fixed values, we define those fixed values as the probabilities of the corresponding events.
Probabilities defined in this way satisfy the same properties as relative frequencies: probabilities take values between 0 and 1, and the probabilities of the outcomes sum to 1.
There are many types of experiments that can’t be repeated, and the limit cannot be defined in a formal mathematical sense. Thus, this approach to defining probabilities is limiting.
Fair Experiments
An experiment is fair if the outcomes are equally likely (have equal probability of occurring).
For fair experiments, we can define probabilities analytically. Let \(S\) be the sample space, where \(|S| < \infty\). Then:
The probability of any outcome is \( 1/|S|\).
The probability of an event \(E\) is \(P(E) = |E|/|S|\).
Axiomatic Probability
A probability space is a triple \((S, \mathcal{F}, P)\), where \(S\) is the sample space, \(\mathcal{F}\) is the event class, and \(P\) is the probability measure.
The event class is a collection of events to which we will assign probability.
The event class must contain the sample space and be closed under finite unions and intersections.
For finite sample spaces, the event class is usually the power set of the sample space, \(2^S\).
A probability measure is a set function that assigns a probability to every event in \(\mathcal{F}\).
The probability measure must obey the 3 Axioms of Probability:
\(P(E) \ge 0\) (probabilities are non-negative)
\(P(S)=1\) (the probability that some outcome occurs is equal to 1)
\(P(E \cup F) = P(E) + P(F)\) if \(E\) and \(F\) are disjoint events
Corollaries to the Axioms of Probability
There are many Corollaries of Probability, which can be used to help solve problems.
Combinatorics
Combinatorics is the mathematics of counting.
Combinatorics is often helpful in determining the cardinalities of sample spaces and events.
The
itertoolsPython library can be used to enumerate sample spaces and events.In sampling \(k\) items from a set of \(N\) items with replacement and with ordering, there are \(N^k\) possible samples.
For \(k\) items, there are \(k!\) orderings, called permutations.
In Python, factorial value can be computed using the
factorial()function form SciPy.special. To get an exact (integer) result, pass the keyword argumentexact=True.In sampling \(k\) items from a set of \(N\) items without replacement and with ordering, there are \(N!/k!\) possible samples.
In sampling \(k\) items from a set of \(N\) items without replacement and without ordering, there are \(\binom{N}{k}\) possible samples, where \begin{equation*} \binom{N}{k} = \frac{N!}{(N-k)!~k!} . \end{equation*}
In Python, the binomial coefficient can be computed using the
binom()function from SciPy.special.Combinatorics can be quite tricky in practice. A common problem is overcounting, in which some equivalent results are counted more than one time.