Lecture 16: Derivatives of Inverse and Singular Values
Description
In this lecture, Professor Strang reviews how to find the derivatives of inverse and singular values. Later in the lecture, he discusses LASSO optimization, the nuclear norm, matrix completion, and compressed sensing.
Summary
Derivative of \(A^2\) is \(A(dA/dt)+(dA/dt)A\): NOT \(2A(dA/dt)\).
The inverse of \(A\) has derivative \(-A^{-1}(dA/dt)A^{-1}\).
Derivative of singular values \(= u(dA/dt)v^{\mathtt{T}} \)
Interlacing of eigenvalues / Weyl inequalities
Related section in textbook: III.1-2
Instructor: Prof. Gilbert Strang
Problems for Lecture 16
From textbook Sections III.1 - III.2
-
Find the eigenvalues \(\lambda_1(t)\) and \(\lambda_2(t)\) of \( A=\begin{bmatrix}2 & 1\\ 1 & 0\end{bmatrix}+ t\,\begin{bmatrix}1 & 1\\ 1 & 1\end{bmatrix}\).
-
At \(t=0\), find the eigenvectors of \(A(0)\) and verify \(\dfrac{d\lambda}{dt}=y^{\mathtt{T}} \dfrac{dA}{dt}\,x\).
-
Check that the change \(A(t)-A(0)\) is positive semidefinite for \(t>0\). Then verify the interlacing law \(\lambda_1(t)\geq\lambda_1(0)\geq\lambda_2(t)\geq\lambda_2(0)\).
12. If \(x^{\mathtt{T}} Sx>0\) for all \(x\neq 0\) and \(C\) is invertible, why is \((Cy)^{\mathtt{T}} S(Cy)\) also positive? This shows again that if \(S\) has all positive eigenvalues, so does \(C^{\mathtt{T}} SC\).
