LEC #  TOPICS  KEY DATES 

1  Introduction and probability review  
2  More review; the Bernoulli process  
3  Laws of large numbers, convergence  Problem set 1 due 
4  Poisson (the perfect arrival process)  
5  Poisson combining and splitting  Problem set 2 due 
6  From Poisson to Markov  
7  Finitestate Markov chains; the matrix approach  Problem set 3 due 
8  Markov eigenvalues and eigenvectors  
9  Markov rewards and dynamic programming  Problem set 4 due 
10  Renewals and the strong law of large numbers (SLLN)  
11  Renewals: strong law and rewards  Problem set 5 due 
12  Renewal rewards, stopping trials, and Wald’s equality  
13  Little, M/G/1, ensemble averages  Problem set 6 due 
14  Review  
15  The last renewal  Problem set 7 due 
Quiz  
16  Renewals and countable state Markov  
17  Countablestate Markov chains  
18  Countablestate Markov chains and processes  Problem set 8 due 
19  Countablestate Markov processes  Problem set 9 due 
20  Markov processes and random walks  
21  Hypothesis testing and random walks  Problem set 10 due 
22  Random walks and thresholds  
23  Martingales (plain, sub and super)  Problem set 11 due 
24  Martingales: stopping and converging  
25  Putting it all together 
Calendar
Instructor:  
Course Number: 

Departments:  
As Taught In:  Spring 2011 
Level: 
Graduate

Learning Resource Types
assignment_turned_in
Problem Sets with Solutions
grading
Exams with Solutions
menu_book
Online Textbook
theaters
Lecture Videos