Readings

This section is divided into recommended textbooks, readings by session, and course notes. There are no required textbooks for this course.

Recommended Textbooks

A list of relevant textbooks is given below:

Strogatz, Steven H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering. Boulder, CO: Westview Press, 2001. ISBN: 9780738204536.

Slotine, Jean-Jacques E., and Weiping Li. Applied Nonlinear Control. Upper Saddle River, NJ: Prentice Hall, 1991. ISBN: 9780130408907.

Fantoni, Isabelle, and Rogelio Lozano. Non-linear Control for Underactuated Mechanical Systems. New York, NY: Springer-Verlag, 2002. ISBN: 9781852334239.

Bertsekas, Dimitri P. Dynamic Programming and Optimal Control. 3rd ed. Vols. I and II. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083 (set).

Buy at MIT Press Sutton, Richard S., and Andrew G. Barto. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998. ISBN: 9780262193986.

Bertsekas, Dimitri P., and John N. Tsitsiklis. Neuro-Dynamic Programming. Nashua, NH: Athena Scientific, 1996. ISBN: 9781886529106.

LaValle, Steven M. Planning Algorithms. New York, NY: Cambridge University Press, 2006. ISBN: 9780521862059.

Readings by Session

The readings below come from the course notes “Underactuated Robotics: Learning, Planning, and Control for Efficient and Agile Machines.”

SES # TOPICS READINGS
1

Fully- vs. under-actuated systems

Preliminaries

Chapter 1 and Appendix A
2 Nonlinear dynamics of the simple pendulum Chapter 2
3

Introduction to optimal control

Double-integrator examples

Chapter 9
4

Double integrator (cont.)

Quadratic regulator (Hamilton-Jacobi-Bellman (HJB) sufficiency), min-time control (Pontryagin)

Chapter 10
5 Dynamic programming and value interation: grid world, double integrator, and pendulum examples

Chapter 9 (cont.)

6 Acrobot and cart-pole: controllability, partial feedback linearization (PFL), and energy shaping Chapter 3
7 Acrobot and cart-pole (cont.) Chapter 3 (cont.)
8 Policy search: open-loop optimal control, direct methods, and indirect methods Chapter 12
9 Policy search (cont.): trajectory stabilization, iterative linear quadratic regulator (iLQR), differential dynamic programming (DDP) Chapter 12 (cont.)
10 Simple walking models: rimless wheel, compass gait, kneed compass gait Chapter 5
11 Feedback control for simple walking models Chapter 5 (cont.)
12 Simple running models: spring-loaded inverted pendulum (SLIP), Raibert hoppers Chapter 6
13 Motion planning: Dijkstra’s, A-star Chapter 13
14 Randomized motion planning: rapidly-exploring randomized trees and probabilistic road maps Chapter 13 (cont.)
15 Feedback motion planning: planning with funnels, linear quadratic regulator (LQR) trees Chapter 14
16 Function approximation and system identification Chapter 8 and Appendix B
17 Model systems with uncertainty: state distribution dynamics and state estimation Chapter 8 (cont.)
18 Stochastic optimal control Chapter 15
19 Aircraft Chapter 7
20 Swimming and flapping flight Chapter 7 (cont.)
21 Randomized policy gradient Chapter 17
22 Randomized policy gradient (cont.) Chapter 17 (cont.)
23 Model-free value methods: temporal difference learning and Q-learning Chapter 16
24

Actor-critic methods

Final project presentations

Chapter 18
25 Final project presentations  

Course Notes

Selected chapters from the course notes are available below. Updated revisions of the course notes are available  here.

CHAPTERS TOPICS
Front Title page, table of contents, and preface ( PDF)
1 Fully actuated vs. underactuated systems ( PDF)
I. Nonlinear dynamics and control
2 The simple pendulum ( PDF)
3 The acrobot and cart-pole ( PDF)
4 Manipulation
5 Walking ( PDF)
6 Running
7 Flight
8 Model systems with stochasticity
II. Optimal control and motion planning
9 Dynamic programming ( PDF)
10 Analytical optimal control with the Hamilton-Jacobi-Bellman sufficiency theorem ( PDF)
11 Analytical optimal control with Pontryagin’s minimum principle
12 Trajectory optimization ( PDF)
13 Feasible motion planning
14 Global policies from local policies
15 Stochastic optimal control
16 Model-free value methods
17 Model-free policy search ( PDF)
18 Actor-critic methods
IV. Applications and extensions
19 Learning case studies and course wrap-up
Appendix

A. Robotics preliminaries ( PDF)

B. Machine learning preliminaries

Back References ( PDF)

Course Info

Learning Resource Types

grading Exams
theaters Lecture Videos
group_work Projects
assignment_turned_in Problem Sets with Solutions
assignment_turned_in Programming Assignments with Examples