Help   |   Contact Us

 

Lecture Notes

This section includes two versions of the course notes. The complete notes are from the MIT course, while the short notes are from a series of lectures given by Prof. Bertsekas in Summer 2012.

Course Notes

Readings are from the course textbook:

Buy at Amazon Bertsekas, Dimitri P. Dynamic Programming and Optimal Control, Volume I. 3rd ed. Athena Scientific, 2005. ISBN: 9781886529267.

Buy at Amazon ———. Dynamic Programming and Optimal Control, Volume II: Approximate Dynamic Programming. 4th ed. Athena Scientific, 2012. ISBN: 9781886529441.

Complete course notes (PDF - 1.4MB)

LEC # LECTURE NOTES READINGS
Finite Horizon Problems (Volume 1, Chapters 1–6)
1 The DP algorithm (PDF) Chapter 1
2 The DP algorithm (cont.) (PDF)
3 Deterministic finite-state problems (PDF) Chapter 2
4 Deterministic continuous-time problems (PDF) Chapter 3
5 Stochastic DP problems (PDF) Chapter 4
6 Stochastic DP problems (cont.) (PDF)
7 Imperfect state information problems (PDF) Chapter 5
8 Imperfect state information problems (cont.) (PDF)
9 Suboptimal control (PDF) Chapter 6
10 Suboptimal control (cont.) (PDF)
Infinite Horizon Problems—Simple (Volume 1, Chapter 7)
11 Infinite horizon problems (PDF) Chapter 7
12 Infinite horizon problems (cont.) (PDF)
13 Infinite horizon problems (cont.) (PDF)
Infinite Horizon Problems—Advanced (Volume 2)
14 Discounted problems: computational methods (PDF) Chapter 1
15 Discounted problems: computational methods (cont.) (PDF)
16 Discounted problems: computational methods (cont.) (PDF)
17 Stochastic shortest path problems (PDF) Chapter 2
18 Approximate DP (PDF) Chapter 6
19 Approximate DP (cont.) (PDF)
20 Approximate DP (cont.) (PDF)
21 Approximate DP (cont.) (PDF)
22 Approximate DP (cont.) (PDF)
23 Approximate DP (cont.) (PDF)

Short Notes

These notes are from a condensed, more research-oriented version of the course.

Short course notes (PDF - 1.5MB)

LEC # LECTURE NOTES
1 Exact DP: infinite horizon problems (PDF)
2 Exact DP: large-scale computational methods (PDF)
3 General issues of approximation and simulation (PDF)
4 Temporal differences (TD), projected equations, Galerkin approximation (PDF)
5 Aggregation methods (PDF)
6 Stochastic approximation, Q-learning, and other methods (PDF)
7 Monte Carlo methods (PDF)