| 1 | Introduction and probability review | (PDF) |
| 2 | More review; the Bernoulli process | (PDF) |
| 3 | Laws of large numbers, convergence | (PDF) |
| 4 | Poisson (the perfect arrival process) | (PDF) |
| 5 | Poisson combining and splitting | (PDF) |
| 6 | From Poisson to Markov | (PDF) |
| 7 | Finite-state Markov chains; the matrix approach | (PDF) |
| 8 | Markov eigenvalues and eigenvectors | (PDF) |
| 9 | Markov rewards and dynamic programming | (PDF) |
| 10 | Renewals and the strong law of large numbers (SLLN) | (PDF) |
| 11 | Renewals: strong law and rewards | (PDF) |
| 12 | Renewal rewards, stopping trials, and Wald's equality | (PDF) |
| 13 | Little, M/G/1, ensemble averages | (PDF) |
| 14 | Review | (PDF) |
| 15 | The last renewal | (PDF) |
| 16 | Renewals and countable state Markov | (PDF) |
| 17 | Countable-state Markov chains | (PDF) |
| 18 | Countable-state Markov chains and processes | (PDF) |
| 19 | Countable-state Markov processes | (PDF) |
| 20 | Markov processes and random walks | (PDF) |
| 21 | Hypothesis testing and random walks | (PDF) |
| 22 | Random walks and thresholds | (PDF) |
| 23 | Martingales (plain, sub and super) | (PDF) |
| 24 | Martingales: stopping and converging | (PDF) |
| 25 | Putting it all together | (PDF) |
