Introduction to Probability

Part II: Inference & Limit Theorems

The videos in Part II describe the laws of large numbers and introduce the main tools of Bayesian inference methods.

The textbook for this subject is Bertsekas, Dimitri, and John Tsitsiklis. Introduction to Probability. 2nd ed. Athena Scientific, 2008. ISBN: 9781886529236.

The authors have made this Selected Summary Material (PDF) available for OCW users. 

L = Lecture Content

S = Supplemental Content

SES # & TOPICS	SLIDES
Lecture 14: Introduction to Bayesian Inference
› View Lecture Videos L14.1 Lecture Overview L14.2 Overview of some Application Domains L14.3 Types of Inference Problems L14.4 The Bayesian Inference Framework L14.5 Discrete Parameter, Discrete Observation L14.6 Discrete Parameter, Continuous Observation L14.7 Continuous Parameter, Continuous Observation L14.8 Inferring the Unknown Bias of a Coin and the Beta Distribution L14.9 Inferring the Unknown Bias of a Coin—Point Estimates L14.10 Summary S14.1 The Beta Formula	Lecture 14 Slides (PDF - 2.2MB) Lecture 14 Slides Annotated (PDF - 1.3MB)
Lecture 15: Linear Models With Normal Noise
› View Lecture Videos L15.1 Lecture Overview L15.2 Recognizing Normal PDFs L15.3 Estimating a Normal Random Variable in the Presence of Additive Noise L15.4 The Case of Multiple Observations L15.5 The Mean Squared Error L15.6 Multiple Parameters; Trajectory Estimation L15.7 Linear Normal Models L15.8 Trajectory Estimation Illustration	Lecture 15 Slides (PDF - 1.8MB) Lecture 15 Slides Annotated (PDF - 1.8MB)
Lecture 16: Least Mean Squares (LMS) Estimation
› View Lecture Videos L16.1 Lecture Overview L16.2 LMS Estimation in the Absence of Observations L16.3 LMS Estimation of One Random Variable Based on Another L16.4 LMS Performance Evaluation L16.5 Example: The LMS Estimate L16.6 Example Continued: LMS Performance Evaluation L16.7 LMS Estimation with Multiple Observations or Unknowns L16.8 Properties of the LMS Estimation Error	Lecture 16 Slides (PDF - 1.0MB) Lecture 16 Slides Annotated (PDF - 1.2MB)
Lecture 17: Linear Least Mean Squares (LLMS) Estimation
› View Lecture Videos L17.1 Lecture Overview L17.2 LLMS Formulation L17.3 Solution to the LLMS Problem L17.4 Remarks on the LLMS Solution and on the Error Variance L17.5 LLMS Example L17.6 LLMS for Inferring the Parameter of a Coin L17.7 LLMS with Multiple Observations L17.8 The Simplest LLMS Example with Multiple Observations L17.9 The Representation of the Data Matters in LLMS	Lecture 17 Slides (PDF - 1.0MB) Lecture 17 Slides Annotated (PDF - 1.2MB)
Lecture 18: Inequalities, Convergence, and the Weak Law of Large Numbers
› View Lecture Videos L18.1 Lecture Overview L18.2 The Markov Inequality L18.3 The Chebyshev Inequality L18.4 The Weak Law of Large Numbers L18.5 Polling L18.6 Convergence in Probability L18.7 Convergence in Probability Examples L18.8 Related Topics S18.1 Convergence in Probability of the Sum of Two Random Variables S18.2 Jensen’s Inequality S18.3 Hoeffding’s Inequality	Lecture 18 Slides (PDF - 1.3MB) Lecture 18 Slides Annotated (PDF - 1.9MB)
Lecture 19: The Central Limit Theorem (CLT)
› View Lecture Videos L19.1 Lecture Overview L19.2 The Central Limit Theorem L19.3 Discussion of the CLT L19.4 Illustration of the CLT L19.5 CLT Examples L19.6 Normal Approximation to the Binomial L19.7 Polling Revisited	Lecture 19 Slides (PDF - 2.1MB) Lecture 19 Slides Annotated (PDF - 2.0MB)
Lecture 20: An Introduction to Classical Statistics
› View Lecture Videos L20.1 Lecture Overview L20.2 Overview of the Classical Statistical Framework L20.3 The Sample Mean and Some Terminology L20.4 On the Mean Squared Error of an Estimator L20.5 Confidence Intervals L20.6 Confidence Intervals for the Estimation of the Mean L20.7 Confidence Intervals for the Mean, When the Variance is Unknown L20.8 Other Natural Estimators L20.9 Maximum Likelihood Estimation L20.10 Maximum Likelihood Estimation Examples	Lecture 20 Slides (PDF - 1.3MB) Lecture 20 Slides Annotated