Discrete Stochastic Processes

Calendar

LEC #	TOPICS	KEY DATES
1	Introduction and probability review
2	More review; the Bernoulli process
3	Laws of large numbers, convergence	Problem set 1 due
4	Poisson (the perfect arrival process)
5	Poisson combining and splitting	Problem set 2 due
6	From Poisson to Markov
7	Finite-state Markov chains; the matrix approach	Problem set 3 due
8	Markov eigenvalues and eigenvectors
9	Markov rewards and dynamic programming	Problem set 4 due
10	Renewals and the strong law of large numbers (SLLN)
11	Renewals: strong law and rewards	Problem set 5 due
12	Renewal rewards, stopping trials, and Wald’s equality
13	Little, M/G/1, ensemble averages	Problem set 6 due
14	Review
15	The last renewal	Problem set 7 due
	Quiz
16	Renewals and countable state Markov
17	Countable-state Markov chains
18	Countable-state Markov chains and processes	Problem set 8 due
19	Countable-state Markov processes	Problem set 9 due
20	Markov processes and random walks
21	Hypothesis testing and random walks	Problem set 10 due
22	Random walks and thresholds
23	Martingales (plain, sub and super)	Problem set 11 due
24	Martingales: stopping and converging
25	Putting it all together