Lecture 14: Low Rank Changes in A and Its Inverse
Description
In this lecture, Professor Strang introduces the concept of low rank matrices. He demonstrates how using the Sherman-Morrison-Woodbury formula is useful to efficiently compute how small changes in a matrix affect its inverse.
Summary
If \(A\) is changed by a rank-one matrix, so is its inverse.
Woodbury-Morrison formula for those changes
New data in least squares will produce these changes.
Avoid recomputing over again with all data
Note: Formula in class is correct in the textbook.
Related section in textbook: III.1
Instructor: Prof. Gilbert Strang
Problems for Lecture 14
From textbook Section III.1
1. Another approach to \((I-uv^{\mathtt{T}})^{-1}\) starts with the formula for a geometric series:
\((1-x)^{-1}=1+x+x^2+x^3+\cdots\) Apply that formula when \(x = uv^{\mathtt{T}} =\) matrix:
\begin{eqnarray} \nonumber (I-uv^{\mathtt{T}})^{-1} &=& I+uv^{\mathtt{T}}+uv^{\mathtt{T}} uv^{\mathtt{T}}+uv^{\mathtt{T}} uv^{\mathtt{T}} uv^{\mathtt{T}}+\cdots \\ \nonumber &=& I+u\,[1+v^{\mathtt{T}} u+v^{\mathtt{T}} uv^{\mathtt{T}} u+\cdots]\,v^{\mathtt{T}} \end{eqnarray}
Take \(x=v^{\mathtt{T}} u\) to see \(I+\dfrac{uv^{\mathtt{T}}}{1-v^{\mathtt{T}} u}\). This is exactly equation (1) for \((I-uv^{\mathtt{T}})^{-1}\).
4. Problem 3 found the inverse matrix \(M^{-1}=(A-uv^{\mathtt{T}})^{-1}\). In solving the equation \(My=b\), we compute only the solution \(y\) and not the whole inverse matrix \(M^{-1}\). You can find \(y\) in two easy steps:
\(\quad\) Step 1: Solve \(Ax=b\) and \(Az=u\). Compute \(D=1-v^{\mathtt{T}} z\).
\(\quad\) Step 2: Then \(y=x+\dfrac{v^{\mathtt{T}} x}{D}z\,\) is the solution to \(My=(A-uv^{\mathtt{T}})y=b\).
Verify \((A-uv^{\mathtt{T}})y=b\). We solved two equations using \(A\), no equations using \(M\).
