Lecture 7: Eckart-Young: The Closest Rank k Matrix to A

Course Info

Instructor

Prof. Gilbert Strang

Departments

Mathematics

As Taught In

Spring 2018

Level

Undergraduate

Topics

Learning Resource Types

Lecture Videos

Problem Sets

Instructor Insights

Download Course

Video Lectures

Description

In this lecture, Professor Strang reviews Principal Component Analysis (PCA), which is a major tool in understanding a matrix of data. In particular, he focuses on the Eckart-Young low rank approximation theorem.

Summary

$A_{k} = σ_{1} u_{1} v_{1}^{T} + \dots + σ_{k} u_{k} v_{k}^{T}$ (larger $σ$ ’s from $A = U Σ V^{T}$ )
The norm of $A - A_{k}$ is below the norm of all other $A - B_{k}$ .
Frobenius norm squared = sum of squares of all entries
The idea of Principal Component Analysis (PCA)

Related section in textbook: I.9

Instructor: Prof. Gilbert Strang

Problems for Lecture 7
From textbook Section I.9

2. Find a closest rank-1 approximation to these matrices ( $L^{2}$ or Frobenius norm) :

$A = [\begin{matrix} 3 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 1 \end{matrix}] A = [\begin{matrix} 0 & 3 \\ 2 & 0 \end{matrix}] A = [\begin{matrix} 2 & 1 \\ 1 & 2 \end{matrix}]$