18.S096 | January IAP 2023 | Undergraduate

Matrix Calculus for Machine Learning and Beyond

Course Description

We all know that calculus courses such as 18.01 Single Variable Calculus and 18.02 Multivariable Calculus cover univariate and vector calculus, respectively. Modern applications such as machine learning and large-scale optimization require the next big step, “matrix calculus” and calculus on arbitrary …

We all know that calculus courses such as 18.01 Single Variable Calculus and 18.02 Multivariable Calculus cover univariate and vector calculus, respectively. Modern applications such as machine learning and large-scale optimization require the next big step, “matrix calculus” and calculus on arbitrary vector spaces.

This class covers a coherent approach to matrix calculus showing techniques that allow you to think of a matrix holistically (not just as an array of scalars), generalize and compute derivatives of important matrix factorizations and many other complicated-looking operations, and understand how differentiation formulas must be reimagined in large-scale computing.

Learning Resource Types
Lecture Videos
Lecture Notes
Problem Sets with Solutions
Readings
Instructor Insights
The text "So, you think you know how to take derivatives?" surround by derivatives.
You may think you mastered derivatives after learning a few simple rules, but you could be bewildered if you are faced with more complicated functions like a matrix determinant (what is a derivative “with respect to a matrix”?), the solution of a differential equation, or other huge practical calculations.  We address such topics by re-emphasizing what a derivative really is — linearization — and refocusing differential calculus on the linear algebra at its heart. (Image courtesy of Prof. Steven G. Johnson.)