This section is divided into recommended textbooks, readings by session, and course notes. There are no required textbooks for this course.
Recommended Textbooks
A list of relevant textbooks is given below:
Strogatz, Steven H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Boulder, CO: Westview Press, 2001. ISBN: 9780738204536.
Slotine, Jean-Jacques E., and Weiping Li. Applied Nonlinear Control. Upper Saddle River, NJ: Prentice Hall, 1991. ISBN: 9780130408907.
Fantoni, Isabelle, and Rogelio Lozano. Non-linear Control for Underactuated Mechanical Systems. New York, NY: Springer-Verlag, 2002. ISBN: 9781852334239.
Bertsekas, Dimitri P. Dynamic Programming and Optimal Control. 3rd ed. Vols. I and II. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083 (set).
Sutton, Richard S., and Andrew G. Barto. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. ISBN: 9780262193986.
Bertsekas, Dimitri P., and John N. Tsitsiklis. Neuro-Dynamic Programming. Nashua, NH: Athena Scientific, 1996. ISBN: 9781886529106.
LaValle, Steven M. Planning Algorithms. New York, NY: Cambridge University Press, 2006. ISBN: 9780521862059.
Readings by Session
The readings below come from the course notes “Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines.”
| SES # | TOPICS | READINGS |
|---|---|---|
| 1 |
Fully- vs. under-actuated systems Preliminaries |
Chapter 1 and Appendix A |
| 2 | Nonlinear dynamics of the simple pendulum | Chapter 2 |
| 3 |
Introduction to optimal control Double-integrator examples |
Chapter 9 |
| 4 |
Double integrator (cont.) Quadratic regulator (Hamilton-Jacobi-Bellman (HJB) sufficiency), min-time control (Pontryagin) |
Chapter 10 |
| 5 | Dynamic programming and value interation: grid world, double integrator, and pendulum examples |
Chapter 9 (cont.) |
| 6 | Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping | Chapter 3 |
| 7 | Acrobot and cart-pole (cont.) | Chapter 3 (cont.) |
| 8 | Policy search: open-loop optimal control, direct methods, and indirect methods | Chapter 12 |
| 9 | Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP) | Chapter 12 (cont.) |
| 10 | Simple walking models: rimless wheel, compass gait, kneed compass gait | Chapter 5 |
| 11 | Feedback control for simple walking models | Chapter 5 (cont.) |
| 12 | Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers | Chapter 6 |
| 13 | Motion planning: Dijkstra’s, A-star | Chapter 13 |
| 14 | Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps | Chapter 13 (cont.) |
| 15 | Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees | Chapter 14 |
| 16 | Function approximation and system identification | Chapter 8 and Appendix B |
| 17 | Model systems with uncertainty: state distribution dynamics and state estimation | Chapter 8 (cont.) |
| 18 | Stochastic optimal control | Chapter 15 |
| 19 | Aircraft | Chapter 7 |
| 20 | Swimming and flapping flight | Chapter 7 (cont.) |
| 21 | Randomized policy gradient | Chapter 17 |
| 22 | Randomized policy gradient (cont.) | Chapter 17 (cont.) |
| 23 | Model-free value methods: temporal difference learning and Q-learning | Chapter 16 |
| 24 |
Actor-critic methods Final project presentations |
Chapter 18 |
| 25 | Final project presentations |
Course Notes
Selected chapters from the course notes are available below. Updated revisions of the course notes are available here.
| CHAPTERS | TOPICS |
|---|---|
| Front | Title page, table of contents, and preface (PDF) |
| 1 | Fully actuated vs. underactuated systems (PDF) |
| I. Nonlinear dynamics and control | |
| 2 | The simple pendulum (PDF) |
| 3 | The acrobot and cart-pole (PDF) |
| 4 | Manipulation |
| 5 | Walking (PDF) |
| 6 | Running |
| 7 | Flight |
| 8 | Model systems with stochasticity |
| II. Optimal control and motion planning | |
| 9 | Dynamic programming (PDF) |
| 10 | Analytical optimal control with the Hamilton-Jacobi-Bellman sufficiency theorem (PDF) |
| 11 | Analytical optimal control with Pontryagin’s minimum principle |
| 12 | Trajectory optimization (PDF) |
| 13 | Feasible motion planning |
| 14 | Global policies from local policies |
| 15 | Stochastic optimal control |
| 16 | Model-free value methods |
| 17 | Model-free policy search (PDF) |
| 18 | Actor-critic methods |
| IV. Applications and extensions | |
| 19 | Learning case studies and course wrap-up |
| Appendix |
A. Robotics preliminaries (PDF) B. Machine learning preliminaries |
| Back | References (PDF) |
