๐Ÿ“Š Master Reference ยท Mathematics

Probability &
Statistics

A complete study companion โ€” from sample spaces and classical rules through Bayes' theorem and probability distributions, with worked examples throughout.

Probability Random Variables Bayes' Theorem Distributions Normal Curve

Contents

01 Foundations of Probability
05 Conditional Probability
02 Types & Terminology
06 Bayes' Theorem
03 Key Rules
07 Random Variables & Stats
04 Practice Examples
08 Probability Distributions

Chapter 01

Foundations of Probability

Probability quantifies uncertainty โ€” it measures how likely an event is to occur on a scale from 0 (impossible) to 1 (certain).

0 0.5 1 Impossible Even Chance Certain Snow in July Coin flip Sun rises

Theoretical Probability

Derived from logical reasoning about equally likely outcomes โ€” no experimentation required.

P(E) = n(E) / n(S)

Empirical Probability

Computed from observed data. As trials increase, empirical probability converges toward theoretical probability.

P(E) = freq(E) / total trials

Chapter 02

Key Terminology

Sample Space (S) The complete set of all possible outcomes of an experiment. For a die: S = {1, 2, 3, 4, 5, 6}
Event (E) Any subset of the sample space. Rolling an even number: E = {2, 4, 6}
Certain Event An event guaranteed to occur. P = 1. Example: rolling a number โ‰ค 6 on a standard die.
Impossible Event An event that cannot occur. P = 0. Example: rolling a 7 on a standard die.
Complementary Aแถœ is everything in S that is not in A. P(A) + P(Aแถœ) = 1 always.
Mutually Exclusive Events A and B cannot both occur simultaneously. A โˆฉ B = โˆ…
Independent Events Occurrence of A does not affect probability of B. P(AโˆฉB) = P(A)ยทP(B)

Chapter 03

Rules of Probability

๐Ÿ”„

Complementary Rule

P(Aแถœ) = 1 โˆ’ P(A) P(S) = 1 ยท P(โˆ…) = 0
โž•

Addition Rule

P(AโˆชB) = P(A) + P(B) if mutually exclusive P(AโˆชB) = P(A) + P(B) โˆ’ P(AโˆฉB) if non-exclusive
โœ–๏ธ

Multiplication Rule

P(AโˆฉB) = P(A)ยทP(B) if independent P(AโˆฉB) = P(A)ยทP(B|A) if dependent
Visual Guide

Seeing Probability in Action

Data visualizations bridge the gap between abstract formulas and intuitive understanding. Probability distributions, sample spaces, and event intersections all become clear when represented graphically โ€” whether through Venn diagrams, histograms, or scatter plots.

Probability and statistics visualization

Chapter 04

Practice Examples

Experiment Sample Space Example Event Probability
๐ŸŽฒ Roll a die {1, 2, 3, 4, 5, 6} Roll a 4 P(4) = 1/6 โ‰ˆ 0.167
๐Ÿƒ Draw a card 52 cards, 4 suits Draw a heart P(โ™ฅ) = 13/52 = 1/4
๐Ÿช™ Flip 2 coins {HH, HT, TH, TT} Both tails (TT) P(TT) = 1/4 = 0.25
๐Ÿญ Factory defect Machine A + B Defective item P(D) = 0.032

Chapter 05

Conditional Probability

The probability of B given that A has already occurred:

Core Formula
P(B | A) = P(A โˆฉ B) / P(A)
// Read as "probability of B given A"
๐Ÿ“Œ Worked Example

In a class of 40 students, 25 passed both math and physics. The probability that a student passed physics given they passed math = 25/40 = 0.625

Law of Total Probability

When event B can occur through several mutually exclusive pathways Aโ‚, Aโ‚‚, โ€ฆ, Aโ‚™:

Total Probability
P(B) = ฮฃ P(B | Aแตข) ยท P(Aแตข)
๐Ÿญ Factory Example

Machine A produces 60% of parts (2% defective). Machine B produces 40% (5% defective).
P(Defective) = (0.02 ร— 0.6) + (0.05 ร— 0.4) = 0.012 + 0.020 = 0.032

Bayes theorem tree diagram
Bayes' Theorem

Updating Beliefs with Evidence

Bayes' theorem is the engine of rational inference. A tree diagram reveals how prior probabilities combine with new evidence to produce updated (posterior) probabilities โ€” the foundation of machine learning, medical diagnostics, and spam filters.

Chapter 06

Bayes' Theorem

Reverses conditional probability โ€” lets you update the probability of a cause given that you've observed an effect.

Bayes' Formula
P(A | B) = [ P(B | A) ยท P(A) ] / P(B)
// P(A) = prior ยท P(B|A) = likelihood ยท P(A|B) = posterior
๐Ÿ“ง Spam Filter Example

20% of emails are spam. 90% of spam contains "free". 10% of legitimate email also contains "free".
โ†’ P(spam | "free") = (0.9 ร— 0.2) / 0.22 โ‰ˆ 0.818

Chapter 07

Random Variables & Descriptive Statistics

Discrete Random Variable

Takes a countable number of values. Each value has a specific probability.

  • Number of heads in 10 coin flips
  • Number of defects in a batch

Continuous Random Variable

Takes any value in a range. Probabilities are defined over intervals (areas under a curve).

  • Human height or weight
  • Temperature readings

Measures of Central Tendency & Spread

MeasureDescriptionFormula
Mean (ฮผ)Arithmetic averageฮผ = ฮฃxแตข / n
Weighted MeanAverage with weightsฮผ = ฮฃwแตขxแตข / ฮฃwแตข
MedianMiddle value when sortedmiddle of sorted list
ModeMost frequent valuemax frequency
Variance (ฯƒยฒ)Average squared deviationฯƒยฒ = ฮฃ(xแตขโˆ’ฮผ)ยฒ / n
Std. Dev. (ฯƒ)Square root of varianceฯƒ = โˆšฯƒยฒ

Chapter 08

Probability Distributions

A probability distribution describes how probability is spread across all possible values of a random variable.

๐ŸŽฏ Binomial Distribution

Counts successes in n independent trials, each with probability p.

P(X=k) = C(n,k) ยท pแต ยท (1โˆ’p)โฟโปแต
  • Fixed number of trials
  • Two outcomes only (success / failure)
  • Constant probability per trial
  • Independent trials

๐ŸŽฒ Uniform Distribution

All outcomes are equally likely โ€” maximum entropy distribution.

P(X = xแตข) = 1/n (discrete)
f(x) = 1/(bโˆ’a) (continuous)
  • Standard die roll
  • Random number generator
Normal Distribution

The Bell Curve

The most important distribution in all of statistics. Symmetric around the mean, with probabilities concentrated near the center and tailing off symmetrically. The empirical rule lets you quickly estimate how much data falls within 1, 2, or 3 standard deviations.

Normal distribution bell curve with standard deviations

๐Ÿ”” Normal Distribution Properties

  • Mean = Median = Mode (perfect symmetry)
  • Total area under the curve = 1
  • Defined by two parameters: ฮผ (mean) and ฯƒ (std dev)
  • Asymptotically approaches zero at extremes
PDF Formula
f(x) = (1 / ฯƒโˆš2ฯ€) ยท e^(โˆ’(xโˆ’ฮผ)ยฒ/2ฯƒยฒ)

โšก The Empirical Rule (68โ€“95โ€“99.7)

68%
within ยฑ1ฯƒ
95%
within ยฑ2ฯƒ
99.7%
within ยฑ3ฯƒ

Conclusion

Key Takeaways

Core Concepts

  • Probability lives on [0, 1] โ€” always
  • Addition rule handles union of events
  • Multiplication rule handles intersection
  • Conditional probability updates with evidence

Advanced Topics

  • Bayes' theorem inverts conditional probability
  • Binomial distribution models success counts
  • Normal distribution is universal in nature
  • ฯƒ and ฮผ fully describe the normal curve
โญ

Pro Tip: The best way to internalize probability is through daily problem-solving. Start with classical problems (dice, cards, coins), then move to real-world applications like quality control, medical testing, and machine learning classification โ€” where Bayes' theorem and distributions dominate.