Bayes' theorem
Bayes' theorem, also referred to as Bayes' law or Bayes' rule, is a formula that can be used to determine the probability of an event based on prior knowledge of conditions that may affect the event. In other words, it is a way to calculate a conditional probability, which is the probability of one event occurring given that another has already occurred.
As an example, imagine a bag of marbles. If there are 3 blue marbles in the bag and 2 red marbles, there is a 3/5 chance of drawing a blue marble from the bag. If you then draw 1 red marble from the bag, and don't replace it, the probability of drawing a blue marble increases to 3/4. The probability of drawing a blue or red marble on the second draw is a conditional probability based on what color marble was drawn first. Bayes' theorem is denoted as:
| P(A) is the probability of event A |
| P(B) is the probability of event B |
| P(A|B) is the probability of A given that event B has occurred |
| P(B|A) is the probability of B given that event A has occurred |
Bayes' theorem can be derived from the definition of conditional probability (proof below), which involves knowing the joint probability of the events. In some cases, this probability can be difficult to determine, and where possible, Bayes' theorem can be used instead.
Example
Use Bayes' theorem to determine the probability of choosing a queen from a standard deck of 52 cards given that the chosen card is a face card.
This problem requires knowledge of the number of types of cards in a standard deck. There are 3 different types of face cards (king, queen, jack) in a standard deck, and there are 4 of each type. Thus, there are a total of 12 face cards in a standard deck. It is possible to determine the probability in question using this information alone; since there are 4 queens and 12 face cards, if a face card is chosen, there is a 4/12, or 1/3 chance of the card being a queen.
To use Bayes' theorem to solve the problem, we need to determine each value in the theorem. P(A|B) is the probability of choosing a queen given that a face card is selected. Let:
- P(A) = probability of choosing a queen
- P(B) = probability of choosing a face card
- P(B|A) = probability of the chosen card being a face card given that a queen was selected
There are 52 cards in a standard deck, 40 of which are not face cards, 12 of which are face cards, and 4 of which are queens. Thus:
- P(A) = 4/52 = 1/13
- P(B) = 12/52 = 3/13
- P(B|A) = 1
Using Bayes' theorem:
This confirms the probability we determined above.
Derivation of Bayes' theorem
Bayes' theorem is derived from the definition of conditional probability:
P(A ∩ B) is the probability of both events A and B occurring. Note that P(A ∩ B) is the same as P(B ∩ A), or:
The probability of both events occurring can also be expressed as the product of the probability of event A occurring and the probability of B occurring given A, or:
Equating the two yields Bayes' theorem:
