1 Introduction and probability review  
2 More review; the Bernoulli process  
3 Laws of large numbers, convergence Problem set 1 due
4 Poisson (the perfect arrival process)  
5 Poisson combining and splitting Problem set 2 due
6 From Poisson to Markov  
7 Finite-state Markov chains; the matrix approach Problem set 3 due
8 Markov eigenvalues and eigenvectors  
9 Markov rewards and dynamic programming Problem set 4 due
10 Renewals and the strong law of large numbers (SLLN)  
11 Renewals: strong law and rewards Problem set 5 due
12 Renewal rewards, stopping trials, and Wald’s equality  
13 Little, M/G/1, ensemble averages Problem set 6 due
14 Review  
15 The last renewal Problem set 7 due
16 Renewals and countable state Markov  
17 Countable-state Markov chains  
18 Countable-state Markov chains and processes Problem set 8 due
19 Countable-state Markov processes Problem set 9 due
20 Markov processes and random walks  
21 Hypothesis testing and random walks Problem set 10 due
22 Random walks and thresholds  
23 Martingales (plain, sub and super) Problem set 11 due
24 Martingales: stopping and converging  
25 Putting it all together  

Course Info

Learning Resource Types

assignment_turned_in Problem Sets with Solutions
grading Exams with Solutions
menu_book Online Textbook
theaters Lecture Videos