Lecture 11: Minimizing ‖x‖ Subject to Ax = b

Course Info

Instructor

Prof. Gilbert Strang

Departments

Mathematics

As Taught In

Spring 2018

Level

Undergraduate

Topics

Learning Resource Types

Lecture Videos

Problem Sets

Instructor Insights

Download Course

Video Lectures

Description

In this lecture, Professor Strang revisits the ways to solve least squares problems. In particular, he focuses on the Gram-Schmidt process that finds orthogonal vectors.

Summary

Picture the shortest $x$ in $ℓ^{1}$ and $ℓ^{2}$ and $ℓ^{\infty}$ norms
The $ℓ^{1}$ norm gives a sparse solution $x$ .
Details of Gram-Schmidt orthogonalization and $A = Q R$
Orthogonal vectors in $Q$ from independent vectors in $A$

Related section in textbook: I.11

Instructor: Prof. Gilbert Strang

Problems for Lecture 11
From textbook Section I.11

6. The first page of I.11 shows unit balls for the $ℓ^{1}$ and $ℓ^{2}$ and $ℓ^{\infty}$ norms. Those are the three sets of vectors $v = (v_{1}, v_{2})$ with $| | v | |_{1} \leq 1, | | v | |_{2} \leq 1, | | v | |_{\infty} \leq 1$ . Unit balls are always convex because of the triangle inequality for vector norms:

$If | | v | | \leq 1 and | | w | | \leq 1 show that | | \frac{v}{2} + \frac{w}{2} | | \leq 1$

10. What multiple of $a = [\begin{matrix} 1 \\ 1 \end{matrix}]$ should be subtracted from $b = [\begin{matrix} 4 \\ 0 \end{matrix}]$ to make the result $A_{2}$ orthogonal to $a$ ? Sketch a figure to show $a$ , $b$ , and $A_{2}$ .