Reading assignments are all in the textbook: Strang, Gilbert. *Linear Algebra and Learning from Data*. Wellesley-Cambridge Press, 2018. ISBN: 9780692196380.

Professor Strang created a website for the book, including a link to the Table of Contents (PDF) and sample chapters.

LEC # | TOPICS | READINGS |
---|---|---|

1 | The Column Space of \(A\) Contains All Vectors \(A\boldsymbol{x}\) | Section I.1: Multiplication \(A\boldsymbol{x}\) Using Columns of \(A\) |

2 | Multiplying and Factoring Matrices | Section I.2: Matrix-Matrix Multiplication \(AB\) |

3 | Orthonormal Columns in \(Q\) Give \(Q’Q= I\) | Section I.5: Orthogonal Matrices and Subspaces |

4 | Eigenvalues and Eigenvectors | Section I.6: Eigenvalues and Eigenvectors |

5 | Positive Definite and Semidefinite Matrices | Section I.7: Symmetric Positive Definite Matrices |

6 | Singular Value Decomposition (SVD) | Section I.8: Singular Values and Singular Vectors in the SVD |

7 | Eckart-Young: The Closest Rank \(k\) Matrix to \(A\) | Section I.9: Principal Components and the Best Low Rank Matrix |

8 | Norms of Vectors and Matrices | Section I.11: Norms of Vectors and Functions and Matrices |

9 | Four Ways to Solve Least Squares Problems | Section II.2: Least Squares: Four Ways |

10 | Survey of Difficulties with \(A\boldsymbol{x} = \boldsymbol{b}\) | Intro Chapter 2: Introduction to Computations with Large Matrices |

11 | Minimizing \(‖\boldsymbol{x}‖\) Subject to \(A\boldsymbol{x} = \boldsymbol{b}\) | Section I.11: Norms of Vectors and Functions and Matrices |

12 | Computing Eigenvalues and Singular Values | Section II.1: Numerical Linear Algebra |

13 | Randomized Matrix Multiplication | Section II.4: Randomized Linear Algebra |

14 | Low Rank Changes in \(A\) and Its Inverse | Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) |

15 | Matrices \(A(t)\) Depending on \(t\), Derivative = \(dA/dt\) | Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) Section III.2: Interlacing Eigenvalues and Low Rank Signals |

16 | Derivatives of Inverse and Singular Values | Section III.1: Changes in \(A^{-1}\) from Changes in \(A\) Section III.2: Interlacing Eigenvalues and Low Rank Signals |

17 | Rapidly Decreasing Singular Values | Section III.3: Rapidly Decaying Singular Values |

18 | Counting Parameters in SVD, LU, QR, Saddle Points | Section III.2: Interlacing Eigenvalues and Low Rank Signals |

19 | Saddle Points Continued, Maxmin Principle | Section III.2: Interlacing Eigenvalues and Low Rank Signals Section V.1: Mean, Variance, and Probability |

20 | Definitions and Inequalities | |

21 | Minimizing a Function Step by Step | Section VI.1: Minimum Problems: Convexity and Newton's Method Section VI.4: Gradient Descent Toward the Minimum |

22 | Gradient Descent: Downhill to a Minimum | Section VI.4: Gradient Descent Toward the Minimum |

23 | Accelerating Gradient Descent (Use Momentum) | Section VI.4: Gradient Descent Toward the Minimum |

24 | Linear Programming and Two-Person Games | Section VI.2: Lagrange Multipliers = Derivatives of the Cost Section VI.3: Linear Programming, Game Theory, and Duality |

25 | Stochastic Gradient Descent | Section VI.5: Stochastic Gradient Descent and ADAM |

26 | Structure of Neural Nets for Deep Learning | Section VII.1: The Construction of Deep Neural Networks |

27 | Backpropagation: Find Partial Derivatives | Section VII.3: Backpropagation and the Chain Rule |

28 | Computing in Class [No video available] | Section VII.2: Convolutional Neural Nets |

29 | Computing in Class (cont.) [No video available] | |

30 | Completing a Rank One Matrix, Circulants! | Section IV.8: Completing Rank One Matrices Section IV.2: Shift Matrices and Circulant Matrices |

31 | Eigenvectors of Circulant Matrices: Fourier Matrix | Section IV.2: Shift Matrices and Circulant Matrices |

32 | ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule | Section IV.2: Shift Matrices and Circulant Matrices |

33 | Neural Nets and the Learning Function | Section VII.1: The Construction of Deep Neural Networks Section IV.10: Distance Matrices |

34 | Distance Matrices, Procrustes Problem | Section IV.9: The Orthogonal Procrustes Problem Section IV.10: Distance Matrices |

35 | Finding Clusters in Graphs | Section IV.6: Graphs and Laplacians and Kirchhoff's Laws Section IV.7: Clustering by Spectral Methods and \(k\)-means |

36 | Alan Edelman and Julia Language | Section III.3: Rapidly Decaying Singular Values Section VII.2: Convolutional Neural Nets |