Visual Navigation for Autonomous Vehicles (VNAV)

Syllabus

Course Meeting Times

Lectures: 3 sessions / week, 1 hour / session

Labs: 1 session / week, 2 hours / session

Prerequisites

• Real-time embedded programming (16.35 Real-Time Systems and Software or similar — we are going to code in C++)
• Optimal estimation and control (16.32 Principles of Optimal Control and Estimation or similar)
• Linear algebra (18.06 Linear Algebra or similar)
• MIT students can get permission of the instructors.
• Assignments are largely based on ROS and openCV and are all in C++. Previous experience with robotics (16.405 Robotics: Science and Systems or similar) is a plus.

Description

This course covers the mathematical foundations and state-of-the-art implementations of algorithms for vision-based navigation of autonomous vehicles (e.g., mobile robots, self-driving cars, drones). Topics include geometric control, 3D vision, visual-inertial navigation, place recognition, and simultaneous localization and mapping. It provides students with a rigorous but pragmatic overview of differential geometry and optimization on manifolds and knowledge of the fundamentals of 2-view and multi-view geometric vision for real-time motion estimation, calibration, localization, and mapping. The theoretical foundations are complemented with hands-on labs based on state-of-the-art mini race car and drone platforms. It culminates in a critical review of recent advances in the field and a team project aimed at advancing the state-of-the-art.

Resources

There is no required textbook. The following are recommended books (also available online):

• Barfoot, Timothy. State Estimation for Robotics. Cambridge University Press, 2017. ISBN: 9781107159396. [Preview with Google Books]
• Ma, Yi et al. An Invitation to 3-D Vision: From Images to Geometric Models. Springer, 2003. ISBN: 9780387008936. [Preview with Google Books]

Pointer to other relevant references (tutorial/survey papers as well as technical papers) will be provided during each lecture.

Assignments

Graded assignments include lab exercises and a final project. Lab exercises include a set of questions covering both theoretical aspects (discussed during the lectures) and the result of implementation and testing of algorithms (assigned during the labs). The code written in some of the labs will also be subject to evaluation. Assignments for Lab 1 and 2 are individual. Labs 3–9 include a mix of team assignments (we expect teams of 2 students) and individual assignments.

The final project will be completed in teams formed according to the students’ interests. The grading for the final project will be based on:

1. Technical report, formatted according to ICRA guidelines
2. Final demo, showcasing the outcome of the project
3. Team presentation, including videos describing the project and its implementation

Grading

ACTIVITIES	PERCENTAGES
Lab Exercises Set of questions covering both theoretical aspects (discussed during the lectures) and the result of implementation and testing of algorithms (assigned during the labs).	60%
Final Project Report, Demo, Presentation A final demo, together with a technical report (formatted according to ICRA guidelines) detailing and motivating the choice of final project and your approach to address and solve the corresponding problem. A final presentation reporting on the outcome of your implementation, including experimental results and videos describing the overall project.	25%
Participation, TA evaluation Your participation in the lectures and labs, including attendance and evaluation of your performance in the labs by the VNAV staff.	10%
Team Members’ Assessment Assessment of your performance by your teammates.	5%