Discrete Stochastic Processes

Video Lectures

Lecture 1: Introduction and Probability Review

Lecture 2: More Review; The Bernoulli Process

Lecture 3: Law of Large Numbers, Convergence

Lecture 4: Poisson (The Perfect Arrival Process)

Lecture 5: Poisson Combining and Splitting

Lecture 6: From Poisson to Markov

Lecture 7: Finite-state Markov Chains; The Matrix Approach

Lecture 8: Markov Eigenvalues and Eigenvectors

Lecture 9: Markov Rewards and Dynamic Programming

Lecture 10: Renewals and the Strong Law of Large Numbers

Lecture 11: Renewals: Strong Law and Rewards

Lecture 12: Renewal Rewards, Stopping Trials, and Wald's Inequality

Lecture 13: Little, M/G/1, Ensemble Averages

Lecture 14: Review

Lecture 15: The Last Renewal

Lecture 16: Renewals and Countable-state Markov

Lecture 17: Countable-state Markov Chains

Lecture 18: Countable-state Markov Chains and Processes

Lecture 19: Countable-state Markov Processes

Lecture 20: Markov Processes and Random Walks

Lecture 21: Hypothesis Testing and Random Walks

Lecture 22: Random Walks and Thresholds

Lecture 23: Martingales (Plain, Sub, and Super)

Lecture 24: Martingales: Stopping and Converging

Lecture 25: Putting It All Together