Introduction to Probability

Part III: Random Processes

The videos in Part III provide an introduction to both classical statistical methods and to random processes (Poisson processes and Markov chains).

The textbook for this subject is Bertsekas, Dimitri, and John Tsitsiklis. Introduction to Probability. 2nd ed. Athena Scientific, 2008. ISBN: 9781886529236.

The authors have made this Selected Summary Material (PDF) available for OCW users.

L = Lecture Content

S = Supplemental Content

SES # & TOPICS	SLIDES
Lecture 21: The Bernoulli Process
› View Lecture Videos L21.1 Lecture Overview L21.2 The Bernoulli Process L21.3 Stochastic Processes L21.4 Review of Known Properties of the Bernoulli Process L21.5 The Fresh Start Property L21.6 Example: The Distribution of a Busy Period L21.7 The Time of the K-th Arrival L21.8 Merging of Bernoulli Processes L21.9 Splitting a Bernoulli Process L21.10 The Poisson Approximation to the Binomial	Lecture 21 Slides (PDF - 1.7MB) Lecture 21 Slides Annotated (PDF - 1.3MB)
Lecture 22: The Poisson Process Part I
› View Lecture Videos L22.1 Lecture Overview L22.2 Definition of the Poisson Process L22.3 Applications of the Poisson Process L22.4 The Poisson PMF for the Number of Arrivals L22.5 The Mean and Variance of the Number of Arrivals L22.6 A Simple Example L22.7 Time of the K–th Arrival L22.8 The Fresh Start Property and Its Implications L22.9 Summary of Results L22.10 An Example	Lecture 22 Slides (PDF - 4.6MB) Lecture 22 Slides Annotated (PDF - 1.5MB)
Lecture 23: The Poisson Process Part II
› View Lecture Videos L23.1 Lecture Overview L23.2 The Sum of Independent Poisson Random Variables L23.3 Merging Independent Poisson Processes L23.4 Where is an Arrival of the Merged Process Coming From? L23.5 The Time Until the First (or Last) Lightbulb Burns Out L23.6 Splitting a Poisson Process L23.7 Random Incidence in the Poisson Process L23.8 Random Incidence in a Non–Poisson Process L23.9 Different Sampling Methods Can Give Different Results S23.1 Poisson Versus Normal Approximations to the Binomial S23.2 Poisson Arrivals During an Exponential Interval	Lecture 23 Slides (PDF - 3.0MB) Lecture 23 Slides Annotated (PDF - 1.2MB)
Lecture 24: Finite-State Markov Chains
› View Lecture Videos L24.1 Lecture Overview L24.2 Introduction to Markov Processes L24.3 Checkout Counter Example L24.4 Discrete-Time Finite-State Markov Chains L24.5 N–Step Transition Probabilities L24.6 A Numerical Example — Part I L24.7 Generic Convergence Questions L24.8 Recurrent and Transient States	Lecture 24 Slides (PDF) Lecture 24 Slides Annotated (PDF)
Lecture 25: Steady–State Behavior of Markov Chains
› View Lecture Videos L25.1 Brief Introduction L25.2 Lecture Overview L25.3 Markov Chain Review L25.4 The Probability of a Path L25.5 Recurrent and Transient States: Review L25.6 Periodic States L25.7 Steady-State Probabilities and Convergence L25.8 A Numerical Example — Part II L25.9 Visit Frequency Interpretation of Steady-State Probabilities L25.10 Birth-Death Processes — Part I L25.11 Birth-Death Processes — Part II	Lecture 25 Slides (PDF) Lecture 25 Slides Annotated (PDF - 1.2MB)
Lecture 26: Absorption Probabilities and Expected Time to Absorption
› View Lecture Videos L26.1 Brief Introduction L26.2 Lecture Overview L26.3 Review of Steady-State Behavior L26.4 A Numerical Example — Part III L26.5 Design of a Phone System L26.6 Absorption Probabilities L26.7 Expected Time to Absorption L26.8 Mean First Passage Time L26.9 Gambler’s Ruin	Lecture 26 Slides (PDF - 1.1MB) Lecture 26 Slides Annotated (PDF - 1.2MB)