This section is divided into recommended textbooks, readings by session, and course notes. There are no required textbooks for this course.
Recommended Textbooks
A list of relevant textbooks is given below:
Strogatz, Steven H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Boulder, CO: Westview Press, 2001. ISBN: 9780738204536.
Slotine, Jean-Jacques E., and Weiping Li. Applied Nonlinear Control. Upper Saddle River, NJ: Prentice Hall, 1991. ISBN: 9780130408907.
Fantoni, Isabelle, and Rogelio Lozano. Non-linear Control for Underactuated Mechanical Systems. New York, NY: Springer-Verlag, 2002. ISBN: 9781852334239.
Bertsekas, Dimitri P. Dynamic Programming and Optimal Control. 3rd ed. Vols. I and II. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083 (set).
Sutton, Richard S., and Andrew G. Barto. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. ISBN: 9780262193986.
Bertsekas, Dimitri P., and John N. Tsitsiklis. Neuro-Dynamic Programming. Nashua, NH: Athena Scientific, 1996. ISBN: 9781886529106.
LaValle, Steven M. Planning Algorithms. New York, NY: Cambridge University Press, 2006. ISBN: 9780521862059.
Readings by Session
The readings below come from the course notes “Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines.”
SES # | TOPICS | READINGS |
---|---|---|
1 |
Fully- vs. under-actuated systems Preliminaries |
Chapter 1 and Appendix A |
2 | Nonlinear dynamics of the simple pendulum | Chapter 2 |
3 |
Introduction to optimal control Double-integrator examples |
Chapter 9 |
4 |
Double integrator (cont.) Quadratic regulator (Hamilton-Jacobi-Bellman (HJB) sufficiency), min-time control (Pontryagin) |
Chapter 10 |
5 | Dynamic programming and value interation: grid world, double integrator, and pendulum examples |
Chapter 9 (cont.) |
6 | Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping | Chapter 3 |
7 | Acrobot and cart-pole (cont.) | Chapter 3 (cont.) |
8 | Policy search: open-loop optimal control, direct methods, and indirect methods | Chapter 12 |
9 | Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP) | Chapter 12 (cont.) |
10 | Simple walking models: rimless wheel, compass gait, kneed compass gait | Chapter 5 |
11 | Feedback control for simple walking models | Chapter 5 (cont.) |
12 | Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers | Chapter 6 |
13 | Motion planning: Dijkstra’s, A-star | Chapter 13 |
14 | Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps | Chapter 13 (cont.) |
15 | Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees | Chapter 14 |
16 | Function approximation and system identification | Chapter 8 and Appendix B |
17 | Model systems with uncertainty: state distribution dynamics and state estimation | Chapter 8 (cont.) |
18 | Stochastic optimal control | Chapter 15 |
19 | Aircraft | Chapter 7 |
20 | Swimming and flapping flight | Chapter 7 (cont.) |
21 | Randomized policy gradient | Chapter 17 |
22 | Randomized policy gradient (cont.) | Chapter 17 (cont.) |
23 | Model-free value methods: temporal difference learning and Q-learning | Chapter 16 |
24 |
Actor-critic methods Final project presentations |
Chapter 18 |
25 | Final project presentations |
Course Notes
Selected chapters from the course notes are available below. Updated revisions of the course notes are available here.
CHAPTERS | TOPICS |
---|---|
Front | Title page, table of contents, and preface (PDF) |
1 | Fully actuated vs. underactuated systems (PDF) |
I. Nonlinear dynamics and control | |
2 | The simple pendulum (PDF) |
3 | The acrobot and cart-pole (PDF) |
4 | Manipulation |
5 | Walking (PDF) |
6 | Running |
7 | Flight |
8 | Model systems with stochasticity |
II. Optimal control and motion planning | |
9 | Dynamic programming (PDF) |
10 | Analytical optimal control with the Hamilton-Jacobi-Bellman sufficiency theorem (PDF) |
11 | Analytical optimal control with Pontryagin’s minimum principle |
12 | Trajectory optimization (PDF) |
13 | Feasible motion planning |
14 | Global policies from local policies |
15 | Stochastic optimal control |
16 | Model-free value methods |
17 | Model-free policy search (PDF) |
18 | Actor-critic methods |
IV. Applications and extensions | |
19 | Learning case studies and course wrap-up |
Appendix |
A. Robotics preliminaries (PDF) B. Machine learning preliminaries |
Back | References (PDF) |