Instructor Interview
Below, Prof. Steven G. Johnson describes various aspects of how he and Prof. Alan Edelman taught 18.S096 Matrix Calculus for Machine Learning and Beyond during the 2023 Independent Activities Period (IAP).
OCW: How do the practicalities of teaching an IAP class differ from those of teaching a full-semester course? (Aside from the fact that you can’t cover as much material, naturally!)
Steven Johnson: The main difference from a full semester is that it’s more of an “enrichment” course, focused on exposing students to material they might not see otherwise, with less concern about performance feedback or grading. In such a short time period, it’s not practical to give exams or quizzes, and with time for only two homework assignments the students don’t get practice or feedback on all of the material (none at all on the last week of the course). This makes it a lower-stress course for the students (we hope), however, and frees us to survey more advanced topics at a high level (especially in the later part of the course) even though further study would be required for practical application.
OCW: Students in this class have already studied both calculus and matrix mathematics. What are the challenges they face in learning to apply techniques from the one field to the other?
Steven Johnson: In both first-year calculus and first-semester linear algebra, it’s easy for students to become overly focused on the mechanical procedures—chain/product/quotient rules, Gaussian elimination, eigenvalue formulas, etc.—rather than on the underlying concepts. In our matrix-calculus course, the hand calculations are mostly simple, and the challenging part is more a conceptual rethinking of what calculus means, which may require students to revisit portions of their education that they had previously thought were less important (such as abstract linear operators and vector spaces).
OCW: How do you and Prof. Edelman share the teaching responsibilities for this team-taught course?
Steven Johnson: We tried to split the lectures roughly 50–50% (while heckling each other’s lectures with commentary), each contributing homework problems and other materials, and at the same time we had extensive discussions to converge on notation and conceptual frameworks.
OCW: What is your approach to constructing problem sets? Do they require students primarily to demonstrate mastery of a set of techniques, or do they entail an element of creative problem-solving?
Steven Johnson: Although there are certainly some techniques in which we try go give useful practice, many of our problems are designed more to challenge a student’s understanding of the new generalizations of old concepts (like derivative, gradient, and Jacobian) via questions in which they will get stuck (even simply for understanding what the question means) if they try to mechanically apply their familiar rules.
Often, I also draw inspiration from questions I’ve seen arising in real-world contexts, from my own research group to online forums such as StackOverflow or Julia Discourse, and I try to construct a simplified version for homework. In many cases, this leads to multi-part questions that require students to successfully combine various concepts and techniques, since real-life challenges are rarely resolved by a single formula.
Instructor Video
Prof. Johnson’s talk “So You Think You Know How to Take Derivatives” (YouTube), which he delivered at a special Math & Computation conference in July 2023 in celebration of Prof. Edelman’s 60th birthday, describes in more detail why he and Prof. Edelman decided to create this course.
Curriculum Information
Prerequisites
Courses in linear algebra (such as 18.06 Linear Algebra) and multivariate calculus (such as 18.02 Multivariable Calculus)
Requirements Satisfied
None
Offered
Various versions of 18.S096 Special Subject in Mathematics, each on a different topic, are offered most fall and spring semesters, and occasionally also during the Independent Activities Period (IAP).
Assessment
The course grade was based on performance on the two homework assignments.
Student Information
Enrollment
40 students
Breakdown by Year
15% of the class were first-year students; the remainder were higher-level undergraduates.
Breakdown by Major
First-year students at MIT have not yet declared a major. Of the other students in the class, about 35% were majoring in Electrical Engineering and Computer Science, about 29% in Mathematics, and the remainder in other departments.
How Student Time Was Spent
During an average week, students were expected to spend 24 hours on the course, roughly divided as follows:
In Class (6 hours)
Met 3 times per week for 2 hours per session; mandatory attendance
Out of Class (18 hours)
Outside of class, students completed two problem sets.
