Introduction to Probability

Part I: The Fundamentals

The videos in Part I introduce the general framework of probability models, multiple discrete or continuous random variables, expectations, conditional distributions, and various powerful tools of general applicability.

The textbook for this subject is Bertsekas, Dimitri, and John Tsitsiklis. Introduction to Probability. 2nd ed. Athena Scientific, 2008. ISBN: 9781886529236.

The authors have made this Selected Summary Material (PDF) available for OCW users.

L = Lecture Content

S = Supplemental Content

SES # & TOPICS	SLIDES
Lecture 1: Probability Models and Axioms
› View Lecture Videos L01.1 Lecture Overview L01.2 Sample Space L01.3 Sample Space Examples L01.4 Probability Axioms L01.5 Simple Properties of Probabilities L01.6 More Properties of Probabilities L01.7 A Discrete Example L01.8 A Continuous Example L01.9 Countable Additivity L01.10 Interpretations & Uses of Probabilities	Lecture 1 Slides (PDF - 1.5MB) Lecture 1 Slides Annotated (PDF)
Lecture 1 Supplement: Mathematical Background
› View Lecture Videos S01.0 Mathematical Background Overview S01.1 Sets S01.2 De Morgan’s Laws S01.3 Sequences and their Limits S01.4 When Does a Sequence Converge? S01.5 Infinite Series S01.6 The Geometric Series S01.7 About the Order of Summation in Series with Multiple Indices S01.8 Countable and Uncountable Sets S01.9 Proof that a Set of Real Numbers is Uncountable S01.10 Bonferroni’s Inequality	Math Overview Slides (PDF)
Lecture 2: Conditioning and Bayes’ Rule
› View Lecture Videos L02.1 Lecture Overview L02.2 Conditional Probabilities L02.3 A Die Roll Example L02.4 Conditional Probabilities Obey the Same Axioms L02.5 A Radar Example and Three Basic Tools L02.6 The Multiplication Rule L02.7 Total Probability Theorem L02.8 Bayes’ Rule	Lecture 2 Slides (PDF - 3.3MB) Lecture 2 Slides Annotated (PDF)
Lecture 3: Independence
› View Lecture Videos L03.1 Lecture Overview L03.2 A Coin Tossing Example L03.3 Independence of Two Events L03.4 Independence of Event Complements L03.5 Conditional Independence L03.6 Independence Versus Conditional Independence L03.7 Independence of a Collection of Events L03.8 Independence Versus Pairwise Independence L03.9 Reliability L03.10 The King’s Sibling	Lecture 3 Slides (PDF) Lecture 3 Slides Annotated (PDF)
Lecture 4: Counting
› View Lecture Videos L04.1 Lecture Overview L04.2 The Counting Principle L04.3 Die Roll Example L04.4 Combinations L04.5 Binomial Probabilities L04.6 A Coin Tossing Example L04.7 Partitions L04.8 Each Person Gets An Ace L04.9 Multinomial Probabilities	Lecture 4 Slides (PDF - 1.0MB) Lecture 4 Slides Annotated (PDF)
Lecture 5: Discrete Random Variables Part I
› View Lecture Videos L05.1 Lecture Overview L05.2 Definition of Random Variables L05.3 Probability Mass Functions L05.4 Bernoulli & Indicator Random Variables L05.5 Uniform Random Variables L05.6 Binomial Random Variables L05.7 Geometric Random Variables L05.8 Expectation L05.9 Elementary Properties of Expectation L05.10 The Expected Value Rule L05.11 Linearity of Expectations S05.1 Supplement: Functions	Lecture 5 Slides (PDF - 1.9MB) Lecture 5 Slides Annotated (PDF - 1.1MB)
Lecture 6: Discrete Random Variables Part II
› View Lecture Videos L06.1 Lecture Overview L06.2 Variance L06.3 The Variance of the Bernoulli & the Uniform L06.4 Conditional PMFs & Expectations Given an Event L06.5 Total Expectation Theorem L06.6 Geometric PMF Memorylessness & Expectation L06.7 Joint PMFs and the Expected Value Rule L06.8 Linearity of Expectations & the Mean of the Binomial	Lecture 6 Slides (PDF - 1.4MB) Lecture 6 Slides Annotated (PDF - 1.1MB)
Lecture 7: Discrete Random Variables Part III
› View Lecture Videos L07.1 Lecture Overview L07.2 Conditional PMFs L07.3 Conditional Expectation & the Total Expectation Theorem L07.4 Independence of Random Variables L07.5 Example L07.6 Independence & Expectations L07.7 Independence, Variances & the Binomial Variance L07.8 The Hat Problem S07.1 The Inclusion-Exclusion Formula S07.2 The Variance of the Geometric S07.3 Independence of Random Variables Versus Independence of Events	Lecture 7 Slides (PDF - 1.1MB) Lecture 7 Slides Annotated (PDF)
Lecture 8: Continuous Random Variables Part I
› View Lecture Videos L08.1 Lecture Overview L08.2 Probability Density Functions L08.3 Uniform & Piecewise Constant PDFs L08.4 Means & Variances L08.5 Mean & Variance of the Uniform L08.6 Exponential Random Variables L08.7 Cumulative Distribution Functions L08.8 Normal Random Variables L08.9 Calculation of Normal Probabilities	Lecture 8 Slides (PDF - 3.2MB) Lecture 8 Slides Annotated (PDF - 1.2MB)
Lecture 9: Continuous Random Variables Part II
› View Lecture Videos L09.1 Lecture Overview L09.2 Conditioning A Continuous Random Variable on an Event L09.3 Conditioning Example L09.4 Memorylessness of the Exponential PDF L09.5 Total Probability & Expectation Theorems L09.6 Mixed Random Variables L09.7 Joint PDFs L09.8 From The Joint to the Marginal L09.9 Continuous Analogs of Various Properties L09.10 Joint CDFs S09.1 Buffon’s Needle & Monte Carlo Simulation	Lecture 9 Slides (PDF - 1.7MB) Lecture 9 Slides Annotated (PDF)
Lecture 10: Continuous Random Variables Part III
› View Lecture Videos L10.1 Lecture Overview L10.2 Conditional PDFs L10.3 Comments on Conditional PDFs L10.4 Total Probability & Total Expectation Theorems L10.5 Independence L10.6 Stick-Breaking Example L10.7 Independent Normals L10.8 Bayes Rule Variations L10.9 Mixed Bayes Rule L10.10 Detection of a Binary Signal L10.11 Inference of the Bias of a Coin	Lecture 10 Slides (PDF - 1.5MB) Lecture 10 Slides Annotated (PDF)
Lecture 11: Derived Distributions
› View Lecture Videos L11.1 Lecture Overview L11.2 The PMF of a Function of a Discrete Random Variable L11.3 A Linear Function of a Continuous Random Variable L11.4 A Linear Function of a Normal Random Variable L11.5 The PDF of a General Function L11.6 The Monotonic Case L11.7 The Intuition for the Monotonic Case L11.8 A Nonmonotonic Example L11.9 The PDF of a Function of Multiple Random Variables S11.1 Simulation	Lecture 11 Slides (PDF) Lecture 11 Slides Annotated (PDF)
Lecture 12: Sum of Independent R.V.s. Covariance and Correlation
› View Lecture Videos L12.1 Lecture Overview L12.2 The Sum of Independent Discrete Random Variables L12.3 The Sum of Independent Continuous Random Variables L12.4 The Sum of Independent Normal Random Variables L12.5 Covariance L12.6 Covariance Properties L12.7 The Variance of the Sum of Random Variables L12.8 The Correlation Coefficient L12.9 Proof of Key Properties of the Correlation Coefficient L12.10 Interpreting the Correlation Coefficient L12.11 Correlations Matter	Lecture 12 Slides (PDF - 2.0MB) Lecture 12 Slides Annotated (PDF)
Lecture 13: Conditional Expectation & Variance Revisited; Sum of a Random Number of Independent R.V.s
› View Lecture Videos L13.1 Lecture Overview L13.2 Conditional Expectation as a Random Variable L13.3 The Law of Iterated Expectations L13.4 Stick-Breaking Revisited L13.5 Forecast Revisions L13.6 The Conditional Variance L13.7 Derivation of the Law of Total Variance L13.8 A Simple Example L13.9 Section Means and Variances L13.10 Mean of the Sum of a Random Number of Random Variables L13.11 Variance of the Sum of a Random Number of Random Variables S13.1 Conditional Expectation Properties	Lecture 13 Slides (PDF - 1.2MB) Lecture 13 Slides Annotated (PDF)