16.412J | Spring 2016 | Graduate

Cognitive Robotics


Course Meeting Times

Lectures: 2 sessions / week, 1.5 hours / session


Welcome to Cognitive Robotics! This is a class about applying autonomy to real-world systems. The overarching theme uniting the many different topics in this course will center around programming a cognitive robot. Examples of cognitive robots include humanoids, deep-space explorers, and even the Hal 9000 and SkyNet. This class takes the approach of introducing new reasoning techniques and ideas incrementally. We start with the current paradigm of embedded programming that you’re likely familiar with, and evolve it over the semester—continually adding in new features and reasoning capabilities—ending with a robust, intelligent system. These techniques and topics will include algorithms for allowing a robot to: Monitor itself for potential problems (both observable and hidden), scheduling tasks in time, coming up with novel plans to achieve desired goals over time, dealing with the continuous world, collaborating with other (autonomous) agents, dealing with risk, and more. Many of the topics discussed in this class are new and cutting edge, and some have evolved directly from research from MIT CSAIL’s Model-based Embedded and Robotics Systems (MERS) group, headed by the professor. Thus you should take this course because you are interested in an exploratory, “bleeding-edge” course on robotics, rather than a traditional course that teaches the fundamentals.

The course lecturer is Prof. Brian Williams, and the course TA is Steve Levine. There will also be a number of other guest lecturers throughout the term.


Students must attend all lectures! Attending lecture attentively is vital to learning the emerging field of Cognitive Robotics, and will also seriously help you with your problem sets, so plan on coming to all lectures. As a new field, just looking at the lecture slides afterwards is usually insufficient. You are expected to participate actively in lecture discussions, and to read assigned material before class. We will have randomly-scheduled, 5-minute mini-quizzes before some classes on the topic from a previous lecture. These are not meant to be difficult or stressful, but rather as a way to gauge your understanding, and as a carrot to ensure you are participating.


This class focuses on applying autonomy and autonomy algorithms, as opposed to autonomy theory. Problem sets will ask you to perform modeling exercises, use existing autonomy tools, and implement algorithms (sometimes from scratch). The lectures are meant to give sufficient background into the inner workings of the algorithms, but you are expected to delve into the literature to understand the details.

Cognitive Robotics culminates in a Grand Challenge at the end of the semester. This will involve modeling, algorithm implementation, and debugging in a simulated environment as well as a real robotic testbed. The grand challenge will have a modified “Orienteering” theme—autonomously exploring areas, completing tasks, and accumulating rewards.

Students will give advanced lectures to the class in groups of 5–6. These full-class, 80-minute long lectures are intended to give students a taste of what it’s like doing research in the field, by self-learning a topic on their own (searching the existing literature and extracting relevant information), communicating those ideas clearly to an audience (the rest of the class). Lecture teams will also provide a software implementation with a tutorial / documentation of the capability you lecture about for the rest of the class to use in the grand challenge.

We recommend you do the required readings and preparation before starting on assignments. Past students report that not doing so can cause assignments to take much more time. We also recommend you start assignments earlier—remember that there is a larger demand for help from the teaching staff just before assignments are due!

This course will not have any exams, and will not have a final exam.

Electronics Use Policy

We allow the use of computers and tablets in class for note-taking purposes only. We expect students to be attentive during class and not use their computers for anything else (no checking email, facebook, browsing the web, watching cat videos, etc.). Research has shown that not only is it distracting to the user, but also to nearby classmates who can see you and your screen.


Your grade for 16.412 will be approximately based on the following weighting (the course staff reserves the right to consider other factors and adjust formula):

Participation & attendance (mini quizzes) 10%
Problem sets 40%
Advanced lecture & implementation 30%
Grand challenge 20%

Note: You must do all assignments to pass the course; a passing grade based on all other assignments may be converted to a failing grade if you do not turn in an assignment, where turning in an assignment means a good faith effort to complete it.

Policy on Collaboration

Many people learn more effectively when they study in small groups and cooperate in various ways on homework. This can be particularly true in programming assignments, where working with a partner often helps to avoid careless errors. We are very much in favor of this kind of cooperation, so long as all participants are actively involved in all aspects of the work; do not split up the assignment and have each team member do only a fraction. When you hand in any material with your name on it, we assume that you are certifying that this is your work and that you were involved in all aspects of it. You are expected to write your own solutions and your own code. Do not copy any material from another class member. We may use tools to check for copying on electronic submissions.

We are structuring the work this term into three types: Problem sets, team lectures and projects. Please abide by the following guidelines with regard to these different types of work. Problem sets are designed to reinforce key concepts. These should be completed by each student individually (though seeking tutoring help from the staff is perfectly appropriate). Advanced lectures and projects are designed to be larger scale activities in which team activity is key. For these, we encourage you to work with others. When you turn in your lecture or project, you must identify your collaborators. We expect, however, that you are involved in all aspects of the lecture and project.

Here’s an example of how a good collaboration might work: Both (all) of you sit down with pencil and paper and together plan how you’re going to solve things. You go to a cluster and sit at adjacent machines. When one of you has a problem, the others look over your shoulder. You check after each problem to make sure that the others are all caught up. But in each case you write your own solution, seeking help from the others when you have difficulties. On the write-up, each of you lists the names of all of your collaborators.

Not listing the name of a collaborator will be deemed cheating. Similarly, copying another person’s work and representing it as one’s own work is a serious academic offense and will be treated as such.

We prohibit the use of course bibles in this course.

To summarize, we strongly encourage you to work as a group. It’s a very effective way of catching conceptual and other errors, and of refining one’s thinking and understanding.

And most of all, have fun!

Learning Resource Types
Lecture Videos
Programming Assignments
Lecture Notes
Instructor Insights