Reading assignments are all in the textbook: Strang, Gilbert. Linear Algebra and Learning from Data. Wellesley-Cambridge Press, 2018. ISBN: 9780692196380.
Professor Strang created a website for the book, including a link to the Table of Contents (PDF) and sample chapters.
| LEC # | TOPICS | READINGS |
|---|---|---|
| 1 | The Column Space of \(A\) Contains All Vectors \(A\boldsymbol{x}\) | Section I.1: Multiplication \(A\boldsymbol{x}\) Using Columns of \(A\) |
| 2 | Multiplying and Factoring Matrices | Section I.2: Matrix-Matrix Multiplication \(AB\) |
| 3 | Orthonormal Columns in \(Q\) Give \(Q’Q= I\) | Section I.5: Orthogonal Matrices and Subspaces |
| 4 | Eigenvalues and Eigenvectors | Section I.6: Eigenvalues and Eigenvectors |
| 5 | Positive Definite and Semidefinite Matrices | Section I.7: Symmetric Positive Definite Matrices |
| 6 | Singular Value Decomposition (SVD) | Section I.8: Singular Values and Singular Vectors in the SVD |
| 7 | Eckart-Young: The Closest Rank \(k\) Matrix to \(A\) | Section I.9: Principal Components and the Best Low Rank Matrix |
| 8 | Norms of Vectors and Matrices | Section I.11: Norms of Vectors and Functions and Matrices |
| 9 | Four Ways to Solve Least Squares Problems | Section II.2: Least Squares: Four Ways |
| 10 | Survey of Difficulties with \(A\boldsymbol{x} = \boldsymbol{b}\) | Intro Chapter 2: Introduction to Computations with Large Matrices |
| 11 | Minimizing \(‖\boldsymbol{x}‖\) Subject to \(A\boldsymbol{x} = \boldsymbol{b}\) | Section I.11: Norms of Vectors and Functions and Matrices |
| 12 | Computing Eigenvalues and Singular Values | Section II.1: Numerical Linear Algebra |
| 13 | Randomized Matrix Multiplication | Section II.4: Randomized Linear Algebra |
| 14 | Low Rank Changes in \(A\) and Its Inverse | Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) |
| 15 | Matrices \(A(t)\) Depending on \(t\), Derivative = \(dA/dt\) |
Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) Section III.2: Interlacing Eigenvalues and Low Rank Signals |
| 16 | Derivatives of Inverse and Singular Values |
Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) Section III.2: Interlacing Eigenvalues and Low Rank Signals |
| 17 | Rapidly Decreasing Singular Values | Section III.3: Rapidly Decaying Singular Values |
| 18 | Counting Parameters in SVD, LU, QR, Saddle Points | Section III.2: Interlacing Eigenvalues and Low Rank Signals |
| 19 | Saddle Points Continued, Maxmin Principle |
Section III.2: Interlacing Eigenvalues and Low Rank Signals Section V.1: Mean, Variance, and Probability |
| 20 | Definitions and Inequalities | |
| 21 | Minimizing a Function Step by Step |
Section VI.1: Minimum Problems: Convexity and Newton’s Method Section VI.4: Gradient Descent Toward the Minimum |
| 22 | Gradient Descent: Downhill to a Minimum | Section VI.4: Gradient Descent Toward the Minimum |
| 23 | Accelerating Gradient Descent (Use Momentum) | Section VI.4: Gradient Descent Toward the Minimum |
| 24 | Linear Programming and Two-Person Games |
Section VI.2: Lagrange Multipliers = Derivatives of the Cost Section VI.3: Linear Programming, Game Theory, and Duality |
| 25 | Stochastic Gradient Descent | Section VI.5: Stochastic Gradient Descent and ADAM |
| 26 | Structure of Neural Nets for Deep Learning | Section VII.1: The Construction of Deep Neural Networks |
| 27 | Backpropagation: Find Partial Derivatives | Section VII.3: Backpropagation and the Chain Rule |
| 28 | Computing in Class [No video available] | Section VII.2: Convolutional Neural Nets |
| 29 | Computing in Class (cont.) [No video available] | |
| 30 | Completing a Rank One Matrix, Circulants! |
Section IV.8: Completing Rank One Matrices Section IV.2: Shift Matrices and Circulant Matrices |
| 31 | Eigenvectors of Circulant Matrices: Fourier Matrix | Section IV.2: Shift Matrices and Circulant Matrices |
| 32 | ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule | Section IV.2: Shift Matrices and Circulant Matrices |
| 33 | Neural Nets and the Learning Function |
Section VII.1: The Construction of Deep Neural Networks Section IV.10: Distance Matrices |
| 34 | Distance Matrices, Procrustes Problem |
Section IV.9: The Orthogonal Procrustes Problem Section IV.10: Distance Matrices |
| 35 | Finding Clusters in Graphs |
Section IV.6: Graphs and Laplacians and Kirchhoff’s Laws Section IV.7: Clustering by Spectral Methods and \(k\)-means |
| 36 | Alan Edelman and Julia Language |
Section III.3: Rapidly Decaying Singular Values Section VII.2: Convolutional Neural Nets |
