Readings

Reading assignments are all in the textbook: Strang, Gilbert. Linear Algebra and Learning from Data. Wellesley-Cambridge Press, 2018. ISBN: 9780692196380.

Professor Strang created a website for the book, including a link to the Table of Contents (PDF) and sample chapters.

LEC # TOPICS READINGS
1 The Column Space of \(A\) Contains All Vectors \(A\boldsymbol{x}\) Section I.1: Multiplication \(A\boldsymbol{x}\) Using Columns of \(A\)
2 Multiplying and Factoring Matrices  Section I.2: Matrix-Matrix Multiplication \(AB\)
3 Orthonormal Columns in \(Q\) Give \(Q’Q= I\) Section I.5: Orthogonal Matrices and Subspaces
4 Eigenvalues and Eigenvectors Section I.6: Eigenvalues and Eigenvectors
5 Positive Definite and Semidefinite Matrices Section I.7: Symmetric Positive Definite Matrices
6 Singular Value Decomposition (SVD) Section I.8: Singular Values and Singular Vectors in the SVD
7 Eckart-Young: The Closest Rank \(k\) Matrix to \(A\) Section I.9: Principal Components and the Best Low Rank Matrix
8 Norms of Vectors and Matrices Section I.11: Norms of Vectors and Functions and Matrices
9 Four Ways to Solve Least Squares Problems Section II.2: Least Squares: Four Ways
10 Survey of Difficulties with \(A\boldsymbol{x} = \boldsymbol{b}\) Intro Chapter 2: Introduction to Computations with Large Matrices
11 Minimizing \(‖\boldsymbol{x}‖\) Subject to \(A\boldsymbol{x} = \boldsymbol{b}\) Section I.11: Norms of Vectors and Functions and Matrices
12 Computing Eigenvalues and Singular Values Section II.1: Numerical Linear Algebra
13 Randomized Matrix Multiplication Section II.4: Randomized Linear Algebra
14 Low Rank Changes in \(A\) and Its Inverse Section III.1: Changes in \(A^{-1}\) from Changes in \(A\)
15 Matrices \(A(t)\) Depending on \(t\), Derivative = \(dA/dt\) Section III.1: Changes in \(A^{-1}\) from Changes in \(A\)
Section III.2: Interlacing Eigenvalues and Low Rank Signals
16 Derivatives of Inverse and Singular Values Section III.1: Changes in \(A^{-1}\) from Changes in \(A\)
Section III.2: Interlacing Eigenvalues and Low Rank Signals
17 Rapidly Decreasing Singular Values Section III.3: Rapidly Decaying Singular Values
18 Counting Parameters in SVD, LU, QR, Saddle Points Section III.2: Interlacing Eigenvalues and Low Rank Signals
19 Saddle Points Continued, Maxmin Principle Section III.2: Interlacing Eigenvalues and Low Rank Signals
Section V.1: Mean, Variance, and Probability
20 Definitions and Inequalities  
21 Minimizing a Function Step by Step Section VI.1: Minimum Problems: Convexity and Newton’s Method
Section VI.4: Gradient Descent Toward the Minimum
22 Gradient Descent: Downhill to a Minimum Section VI.4: Gradient Descent Toward the Minimum
23 Accelerating Gradient Descent (Use Momentum) Section VI.4: Gradient Descent Toward the Minimum
24 Linear Programming and Two-Person Games Section VI.2: Lagrange Multipliers = Derivatives of the Cost
Section VI.3: Linear Programming, Game Theory, and Duality
25 Stochastic Gradient Descent Section VI.5: Stochastic Gradient Descent and ADAM
26 Structure of Neural Nets for Deep Learning Section VII.1: The Construction of Deep Neural Networks
27 Backpropagation: Find Partial Derivatives Section VII.3: Backpropagation and the Chain Rule
28 Computing in Class [No video available] Section VII.2: Convolutional Neural Nets
29 Computing in Class (cont.) [No video available]  
30 Completing a Rank One Matrix, Circulants! Section IV.8: Completing Rank One Matrices
Section IV.2: Shift Matrices and Circulant Matrices
31 Eigenvectors of Circulant Matrices: Fourier Matrix Section IV.2: Shift Matrices and Circulant Matrices
32 ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule Section IV.2: Shift Matrices and Circulant Matrices
33 Neural Nets and the Learning Function Section VII.1: The Construction of Deep Neural Networks
Section IV.10: Distance Matrices
34 Distance Matrices, Procrustes Problem Section IV.9: The Orthogonal Procrustes Problem
Section IV.10: Distance Matrices
35 Finding Clusters in Graphs Section IV.6: Graphs and Laplacians and Kirchhoff’s Laws
Section IV.7: Clustering by Spectral Methods and \(k\)-means
36 Alan Edelman and Julia Language Section III.3: Rapidly Decaying Singular Values
Section VII.2: Convolutional Neural Nets

Course Info

Departments
As Taught In
Spring 2018
Learning Resource Types
Lecture Videos
Problem Sets
Instructor Insights