# Assignments Related to Lectures and Readings

All problem sets in one file (PDF)

1 The Column Space of $$A$$ Contains All Vectors $$A\boldsymbol{x}$$ Section I.1 Problem Set I.1 (PDF)
2 Multiplying and Factoring Matrices  Section I.2 Problem Set I.2 (PDF)
3 Orthonormal Columns in $$Q$$ Give $$Q’Q= I$$ Section I.5 Problem Set I.5 (PDF)
4 Eigenvalues and Eigenvectors Section I.6 Problem Set I.6 (PDF)
5 Positive Definite and Semidefinite Matrices Section I.7 Problem Set I.7 (PDF)
6 Singular Value Decomposition (SVD) Section I.8 Problem Set I.8 (PDF)
7 Eckart-Young: The Closest Rank $$k$$ Matrix to $$A$$ Section I.9 Problem Set I.9 (PDF)
8 Norms of Vectors and Matrices Section I.11 Problem Set I.11 (PDF)
9 Four Ways to Solve Least Squares Problems  Section II.2 Problem Set II.2 Problems 2, 8, 9 (PDF)
10 Survey of Difficulties with $$A\boldsymbol{x} = \boldsymbol{b}$$ Introduction Chapter 2 Problem Set II.2 Problems 12, 17 (PDF)
11 Minimizing $$‖\boldsymbol{x}‖$$ Subject to $$A\boldsymbol{x} = \boldsymbol{b}$$

Section I.11

Problem Set I.11 Problem 6
Problem Set II.2 Problem 10 (PDF)
12 Computing Eigenvalues and Singular Values  Section II.1 Problem Set II.1 (PDF)
13 Randomized Matrix Multiplication Section II.4 Problem Set II.4 (PDF)
14 Low Rank Changes in $$A$$ and Its Inverse Section III.1 Problem Set III.1 (PDF)
15 Matrices $$A(t)$$ Depending on $$t$$, Derivative = $$dA/dt$$ Sections III.1–III.2 Problem Set III.2 Problems 1, 2, 5 (PDF)
16 Derivatives of Inverse and Singular Values Sections III.1–III.2 Problem Set III.2 Problems 3, 12 (PDF)
17 Rapidly Decreasing Singular Values Section III.3 Problem Set III.3 (PDF)
18 Counting Parameters in SVD, LU, QR, Saddle Points Appendix, Section III.2 Problem Set III.2 (PDF)
19 Saddle Points Continued, Maxmin Principle Sections III.2, V.1 Problem Set V.1 Problems 3, 8 (PDF)
20 Definitions and Inequalities

Sections V.1, V.3

Problem Set V.1 Problems 10. 12
Problem Set V.3 Problem 3 (PDF)
21 Minimizing a Function Step by Step Sections VI.1, VI.4 Problem Set VI.1 (PDF)
22 Gradient Descent: Downhill to a Minimum Section VI.4 Problem Set VI.4 Problems 1, 6 (PDF)
23 Accelerating Gradient Descent (Use Momentum) Section VI.4 Problem Set VI.4 Problem 5 (PDF)
24 Linear Programming and Two-Person Games Sections VI.2–VI.3

Problem Set VI.2 Problem 1
Problem Set VI.3 Problems 2, 5 (PDF)

25 Stochastic Gradient Descent Section VI.5 Problem Set VI.5 (PDF)
26 Structure of Neural Nets for Deep Learning Section VII.1 Problem Set VII.1 (PDF)
27 Backpropagation to Find Derivative of the Learning Function Section VII.2 Problem Set VII.2 (PDF)
28 Computing in Class [No video available] Section VII.2 and Appendix 3 [No Problems Assigned]
29 Computing in Class (cont.) [No video available] [No Readings] [No Problems Assigned]
30 Completing a Rank-One Matrix, Circulants! Sections IV.8, IV.2

Problem Set IV.8
Problem Set IV.2 (PDF)

31 Eigenvectors of Circulant Matrices: Fourier Matrix Section IV.2 Problem Set IV.2 Problems 3, 5 (PDF)
32 ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule Section IV.2 Problem Set IV.2 Problem 4 (PDF)
33 Neural Nets and the Learning Function Sections VII.1, IV.10

Problem Set VII.1
Problem Set IV.10 (PDF)

34 Distance Matrices, Procrustes Problem Sections IV.9–IV.10 Problem Set IV.9 (PDF)
35 Finding Clusters in Graphs Sections IV.6–IV.7 Problem Set IV.6 (PDF)
36 Alan Edelman and Julia Language Sections III.3, VII.2 [No Problems Assigned]