Bayes' Theorem Calculator

Probability of event A before evidence B is known.
Probability of evidence B given that A is true.
Probability of evidence B given that A is false.
Proportion of the population with the disease.
Probability of a positive test given the disease is present.
Probability of a negative test given the disease is absent.
Hypothesis 1
Hypothesis 2
Hypothesis 3

What is Bayes' Theorem?

Bayes' Theorem is a fundamental rule in probability theory that describes how to update the probability of a hypothesis based on new evidence. It allows us to calculate the probability of an event A given that another event B has occurred, using the probability of B given A and the individual probabilities of A and B.

Definition: Bayes' Theorem provides a way to revise existing predictions or theories given new or additional evidence.

The Formula

Bayes' Theorem is expressed mathematically as:

P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}

Where:

  • P(A|B) is the posterior probability of A given B
  • P(B|A) is the likelihood of B given A
  • P(A) is the prior probability of A
  • P(B) is the marginal probability of B

The denominator P(B) can be expanded using the law of total probability:

P(B) = P(B|A) \cdot P(A) + P(B|\neg A) \cdot P(\neg A)

This gives us the expanded form of Bayes' Theorem:

P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B|A) \cdot P(A) + P(B|\neg A) \cdot P(\neg A)}

Components of Bayes' Theorem

Prior Probability P(A)

The initial probability of event A before considering any evidence. This represents our initial belief about the hypothesis.

Likelihood P(B|A)

The probability of observing evidence B if the hypothesis A is true. This measures how well the hypothesis predicts the observed evidence.

Posterior Probability P(A|B)

The updated probability of hypothesis A after observing evidence B. This is what we're solving for with Bayes' Theorem.

Marginal Probability P(B)

The total probability of observing evidence B under all possible hypotheses. Acts as a normalizing constant.

Visual Representation of Bayes' Theorem

Bayes' Theorem Visualization

Total Population
Hypothesis A
P(A)
Evidence B
P(B)
A∩B
P(A|B) = P(A∩B) / P(B)
= [P(A) × P(B|A)] / P(B)

The diagram shows how Bayes' Theorem calculates the probability of hypothesis A given evidence B by focusing on the intersection of A and B relative to all of B.

Examples of Bayes' Theorem

Example 1: Medical Testing

A medical test for a disease has the following characteristics:

  • 1% of the population has the disease (prevalence)
  • The test correctly identifies 95% of people who have the disease (sensitivity)
  • The test correctly identifies 90% of people who don't have the disease (specificity)

Question: If a person tests positive, what is the probability that they actually have the disease?

Solution:

Let's define our events:

  • D = "person has the disease"
  • T = "test is positive"

We need to find P(D|T) using Bayes' Theorem:

P(D|T) = \frac{P(T|D) \cdot P(D)}{P(T)}

We know:

  • P(D) = 0.01 (prevalence)
  • P(T|D) = 0.95 (sensitivity)
  • P(T|¬D) = 0.10 (1 - specificity)

Using the law of total probability:

P(T) = P(T|D) \cdot P(D) + P(T|\neg D) \cdot P(\neg D)
P(T) = 0.95 \cdot 0.01 + 0.10 \cdot 0.99 = 0.0095 + 0.099 = 0.1085

Now we can calculate P(D|T):

P(D|T) = \frac{0.95 \cdot 0.01}{0.1085} \approx 0.0876

Therefore, the probability that someone has the disease given a positive test result is only about 8.76%.

This example illustrates the "base rate fallacy" - even with a good test, if the disease is rare, a positive test result may not indicate a high probability of having the disease.

Example 2: Email Spam Filter

Consider a spam filter with the following statistics:

  • 30% of all emails are spam
  • The filter correctly identifies 90% of spam emails as spam
  • The filter incorrectly marks 5% of legitimate emails as spam

Question: If an email is marked as spam by the filter, what is the probability that it's actually spam?

Solution:

Let's define our events:

  • S = "email is spam"
  • M = "email is marked as spam by the filter"

We need to find P(S|M) using Bayes' Theorem:

P(S|M) = \frac{P(M|S) \cdot P(S)}{P(M)}

We know:

  • P(S) = 0.30 (spam prevalence)
  • P(M|S) = 0.90 (filter correctly marks spam)
  • P(M|¬S) = 0.05 (filter incorrectly marks legitimate emails)

Using the law of total probability:

P(M) = P(M|S) \cdot P(S) + P(M|\neg S) \cdot P(\neg S)
P(M) = 0.90 \cdot 0.30 + 0.05 \cdot 0.70 = 0.27 + 0.035 = 0.305

Now we can calculate P(S|M):

P(S|M) = \frac{0.90 \cdot 0.30}{0.305} \approx 0.8852

Therefore, if the filter marks an email as spam, there's about an 88.52% chance that it's actually spam.

This illustrates why spam filters are generally reliable - the prior probability of spam is relatively high (unlike rare diseases), which leads to a higher posterior probability.

Bayes' Theorem with Multiple Hypotheses

Bayes' Theorem can be extended to situations with more than two possibilities (i.e., more than just a hypothesis and its negation). For n mutually exclusive and exhaustive hypotheses H₁, H₂, ..., Hₙ, and evidence E, the posterior probability of hypothesis Hᵢ is:

P(H_i|E) = \frac{P(E|H_i) \cdot P(H_i)}{\sum_{j=1}^{n} P(E|H_j) \cdot P(H_j)}

This formula allows us to update the probabilities of multiple competing hypotheses when we observe new evidence.

Note: The denominator serves as a normalizing constant to ensure that the posterior probabilities sum to 1 across all hypotheses.

Example with Multiple Hypotheses

Example: Weather Prediction

A meteorologist is trying to predict tomorrow's weather. Based on historical data, the prior probabilities are:

  • P(Sunny) = 0.50
  • P(Cloudy) = 0.30
  • P(Rainy) = 0.20

The meteorologist observes that the barometric pressure is falling. The likelihood of this evidence given each weather condition is:

  • P(Falling pressure | Sunny) = 0.10
  • P(Falling pressure | Cloudy) = 0.30
  • P(Falling pressure | Rainy) = 0.75

Question: Given the falling pressure, what are the updated probabilities for each weather condition?

Solution:

Let's define E as "falling barometric pressure" and calculate the posterior probability for each weather condition.

First, we calculate the denominator (total probability of the evidence):

P(E) = \sum_{j=1}^{3} P(E|H_j) \cdot P(H_j)
P(E) = 0.10 \cdot 0.50 + 0.30 \cdot 0.30 + 0.75 \cdot 0.20 = 0.05 + 0.09 + 0.15 = 0.29

Now we can calculate the posterior probability for each hypothesis:

For Sunny weather:

P(Sunny|E) = \frac{0.10 \cdot 0.50}{0.29} \approx 0.1724

For Cloudy weather:

P(Cloudy|E) = \frac{0.30 \cdot 0.30}{0.29} \approx 0.3103

For Rainy weather:

P(Rainy|E) = \frac{0.75 \cdot 0.20}{0.29} \approx 0.5172

We can verify that these probabilities sum to 1:

P(Sunny|E) + P(Cloudy|E) + P(Rainy|E) \approx 0.1724 + 0.3103 + 0.5172 \approx 0.9999 \approx 1

Therefore, given the falling barometric pressure, the updated probabilities are:

  • P(Sunny | Falling pressure) ≈ 17.24%
  • P(Cloudy | Falling pressure) ≈ 31.03%
  • P(Rainy | Falling pressure) ≈ 51.72%

Notice how the probability of rain has significantly increased from 20% to about 52% based on the new evidence.

Practice Problems

Test your understanding of Bayes' Theorem with these practice problems.

Problem 1

A certain university has 60% female students and 40% male students. Statistics show that 30% of female students and 60% of male students regularly attend basketball games. If a randomly selected student is observed at a basketball game, what is the probability that the student is female?

Problem 2

A factory has three machines (A, B, and C) that produce widgets. Machine A produces 20% of the widgets and has a 5% defect rate. Machine B produces 30% of the widgets and has a 3% defect rate. Machine C produces 50% of the widgets and has a 2% defect rate. If a randomly selected widget is found to be defective, what is the probability it was produced by Machine A?

Real-World Applications of Bayes' Theorem

Medical Diagnosis

Doctors use Bayesian reasoning to update the probability of a diagnosis based on test results, symptoms, and patient history.

Example: Calculating the probability a patient has a disease given a positive test result.

Spam Filtering

Email services use Bayesian classifiers to determine the probability that a message is spam based on the words it contains.

Example: Gmail's spam filter uses Bayesian inference to identify and filter out unwanted emails.

Machine Learning

Bayesian methods are fundamental in many machine learning algorithms for classification, prediction, and decision-making.

Example: Naive Bayes classifiers for text categorization and recommendation systems.

Search & Rescue

Bayesian methods help optimize search patterns by updating location probabilities based on where searches have already been conducted.

Example: The Coast Guard uses Bayesian search theory to locate missing vessels.

Legal Reasoning

Bayes' Theorem helps evaluate the strength of evidence in legal cases by updating prior beliefs with new evidence.

Example: Assessing the probability of guilt given forensic evidence.

Finance

Investors use Bayesian methods to update their beliefs about market trends based on new economic data.

Example: Algorithmic trading systems that use Bayesian inference to predict market movements.

Further Reading

To deepen your understanding of Bayes' Theorem and related concepts, explore these topics:

Conditional Probability

Learn about how probabilities change when additional information is known.

Learn More

Sample Space & Events

Understand the foundations of probability theory.

Learn More

Independence

Explore when events don't affect each other's probabilities.

Learn More