 When you click the Amazon logo to the left of any citation and purchase the book (or other media) from Amazon.com, MIT OpenCourseWare will receive up to 10% of this purchase and any other purchases you make during that visit. This will not increase the cost of your purchase. Links provided are to the US Amazon site, but you can also support OCW through Amazon sites in other regions. Learn more. |
In the table below, some topics are given along with sections in the course textbook:
Bertsekas, Dimitri. Dynamic Programming and Optimal Control. Vol. I and II. 3rd ed. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083.
Course calendar.
| SES # |
TOPICS |
TEXTBOOK SECTIONS |
KEY DATES |
| 1 |
Introduction to dynamic programming; examples and formulation |
|
|
| 2 |
The dynamic programming algorithm |
|
Problem set 1 out |
| 3 |
Deterministic systems and the shortest path problem |
|
|
| 4 |
Shortest path algorithms |
|
Problem set 1 due
Problem set 2 out
|
| 5 |
Deterministic continuous-time optimal control |
|
|
| 6 |
Stopping and scheduling problems |
|
Problem set 2 due
Problem set 3 out
|
| 7 |
Linear systems with quadratic costs and inventory control |
|
|
| 8 |
Problems with imperfect state information |
|
Problem set 3 due
Problem set 4 out
|
| 9 |
Sufficient statistics |
|
|
| 10 |
Suboptimal control |
|
Problem set 4 due
Problem set 5 out
|
| 11 |
Rollout algorithms |
|
|
| 12 |
More on suboptimal control |
|
|
| 13 |
Infinite horizon I: stochastic shortest path problems |
Sections 7.1-7.2, vol. I |
|
| 14 |
Infinite horizon II: discounted problems |
Section 7.3, vol. I |
Problem set 5 due
Problem set 6 out
|
| 15 |
Infinite horizon III: average cost problems |
Section 7.4, vol. I |
|
| 16 |
Semi-Markov problems |
Section 7.5, vol. I |
Problem set 6 due
Problem set 7 out
|
| 17 |
Infinite horizon: discounted problems I |
Sections 1.1-1.3, vol. II |
|
| 18 |
Infinite horizon: discounted problems II |
Sections 1.3-1.4, vol. II |
Problem set 7 due |
|
Midterm |
|
|
| 19 |
Stochastic shortest path problems |
Chapter 2, vol. II |
|
| 20 |
Overview of main approaches in approximate dynamic programming |
|
Problem set 8 out |
| 21 |
Cost approximation: discounted cost |
Section 6.2, vol. II |
|
| 22 |
Projected equation methods |
Section 6.3, vol. II and references |
Problem set 8 due
Problem set 9 out
|
| 23 |
More on projected equations: Q-learning |
Sections 6.3-6.4, vol. II and references |
|
| 24 |
Extensions to stochastic shortest path and average cost |
Sections 6.5-6.6, vol. II and references |
Problem set 9 due |
| 25 |
Gradient methods for approximation in policy space |
Section 6.7, vol. II and references |
|
| 26 |
Project presentations I |
|
|
| 27 |
Project presentations II |
|
|