Probability for Statistics

Probability rules, expected value, variance, and distribution models used in statistical inference.

Statistics uses probability to reason about what random data might do. You do not need advanced probability to begin statistics, but you do need the core language: events, random variables, distributions, expected value, variance, and common models.

Learning objectives

After this lesson you should be able to:

  • use probability language in statistical contexts;
  • distinguish sample data from a probability model;
  • compute expected value for a simple random variable;
  • interpret variance and standard deviation of a random variable;
  • recognize binomial and normal models;
  • explain why probability is the foundation for standard error and inference.

Probability models

A probability model describes possible outcomes and how likely they are. In statistics, we often imagine data as the result of a random process.

Example: If a poll randomly samples voters, each person’s response can be treated as random before it is observed. The sample proportion then varies from sample to sample.

Events and complements

An event is a set of outcomes. If A is an event, its complement not A contains all outcomes where A does not occur.

P(not A) = 1 - P(A)

This rule is often useful for proportions: if 60% support a policy, then 40% do not support it.

Random variables

A random variable assigns a number to each outcome.

Examples:

  • X = number of heads in 3 coin flips;
  • Y = waiting time until a bus arrives;
  • P_hat = sample proportion who answer yes.

A distribution tells us how probability is assigned to values of the random variable.

Expected value

Expected value is a probability-weighted average.

E(X) = sum(value * probability)

Expected value is not necessarily a value that happens on one trial. It is a long-run average over many repetitions.

Variance and standard deviation

Variance measures average squared distance from expected value:

Var(X) = E((X - mean)^2)

Standard deviation is the square root of variance. In statistics, standard deviation describes variation in raw values, while standard error describes variation in a statistic.

Common distribution models

Bernoulli

A Bernoulli variable has two outcomes: success or failure. Example: one voter supports a policy or does not.

Binomial

A binomial variable counts successes in a fixed number of independent Bernoulli trials with the same success probability.

Examples:

  • number of heads in 10 fair coin flips;
  • number of supporters in a sample of 100 voters, approximately, if conditions are met.

Normal

A normal distribution is symmetric and bell-shaped. It appears often in statistics because of the central limit theorem and because many measurement errors are roughly symmetric.

Worked example 1: expected value

A game pays $0 with probability 0.7 and $10 with probability 0.3.

Step 1: Multiply each value by its probability.

0 * 0.7 = 0
10 * 0.3 = 3

Step 2: Add.

E(X) = 0 + 3 = 3

Interpretation: In the long run, the average payout is $3 per play. This does not mean any single play pays exactly $3.

Worked example 2: binomial setting

A machine produces defective items with probability 0.04. A quality inspector checks 20 independent items.

Question: Is the number of defective items binomial?

Check the conditions:

  1. Fixed number of trials: 20 items. Yes.
  2. Two outcomes per item: defective or not defective. Yes.
  3. Same probability: 0.04. Yes, if the machine process is stable.
  4. Independence: assumed or approximately true. Yes, if checking one item does not affect another.

So the count of defective items can be modeled as binomial.

Worked example 3: probability and inference

Suppose the true support proportion is p=0.50, and we take a random sample of 100 people. We should not expect every sample to have exactly 50 supporters. Random sampling creates variability.

Probability lets us answer:

How unusual is it to observe 60 or more supporters if p = 0.50?

That question is the beginning of hypothesis testing.

Common pitfalls

  • Treating expected value as a guaranteed single-trial result.
  • Using a binomial model when trials are not independent.
  • Forgetting that probabilities must be between 0 and 1 and sum to 1.
  • Confusing standard deviation of raw data with standard error of an estimate.
  • Assuming all data are normal without checking context and shape.

Practice exercises

  1. If P(A)=0.7, find P(not A).
  2. A random variable takes values 1, 2, 3 with probabilities 0.2, 0.5, 0.3. Find its expected value.
  3. Name the distribution for one yes/no trial.
  4. Name the distribution for the number of successes in n independent yes/no trials.
  5. Why is probability needed for confidence intervals?
  1. P(not A)=1-0.7=0.3.
  2. E(X)=1(0.2)+2(0.5)+3(0.3)=0.2+1.0+0.9=2.1.
  3. Bernoulli.
  4. Binomial.
  5. Confidence intervals use probability models to describe how estimates vary from sample to sample.

Subtopic guided practice and checkpoints

Lab 1: check probabilities

Guess first. Can probabilities 0.2, 0.5, 0.4 form a complete distribution?

Guided exercise.

Add them:

0.2 + 0.5 + 0.4 = 1.1

They cannot form a complete distribution because they sum to more than 1.

Checkpoint. Do 0.25, 0.25, 0.50 form a valid distribution?

Lab 2: compute expected value

Guess first. If the only possible values are 0 and 10, can expected value be 4?

Guided exercise.

Yes. Expected value is a long-run average, not necessarily a possible single outcome.

Checkpoint. Find the expected value of 0 with probability 0.6 and 5 with probability 0.4.

Lab 3: identify a distribution

Guess first. Is the number of heads in 12 coin flips Bernoulli or binomial?

Guided exercise.

It is binomial because it counts successes across multiple fixed trials.

Checkpoint. Is one coin flip being heads Bernoulli or binomial?

Lab 4: connect to statistics

Guess first. Why can two samples from the same population have different means?

Guided exercise.

The sample is random. Different selected individuals produce different sample means. Probability models describe that variation.

Checkpoint. What word describes the sample-to-sample variation of a statistic?

  1. Yes. They are nonnegative and sum to 1.
  2. 0(0.6)+5(0.4)=2.
  3. Bernoulli.
  4. Sampling variability.

Probability for statistics guessing game

Using this lesson with edumath and SymPy

Use Python arithmetic to compute expected values.

values = [1, 2, 3]
probabilities = [0.2, 0.5, 0.3]
sum(value * probability for value, probability in zip(values, probabilities))
2.1

Use SymPy to keep probability formulas symbolic.

import sympy as sp

p = sp.symbols("p")
variance_bernoulli = p * (1 - p) ** 2 + (1 - p) * (0 - p) ** 2
sp.simplify(variance_bernoulli)

\(\displaystyle p \left(1 - p\right)\)