1
00:00:06,870 --> 00:00:09,060
PROFESSOR: Hey, we're back.
2
00:00:09,060 --> 00:00:11,870
Today we're going to do a
singular value decomposition
3
00:00:11,870 --> 00:00:13,510
question.
4
00:00:13,510 --> 00:00:16,630
The problem is really
simple to state:
5
00:00:16,630 --> 00:00:19,970
find the singular value
decomposition of this matrix
6
00:00:19,970 --> 00:00:25,150
C equals [5, 5; -1, 7].
7
00:00:25,150 --> 00:00:28,680
Hit pause, try it yourself,
I'll be back in a minute
8
00:00:28,680 --> 00:00:29,920
and we can do it together.
9
00:00:40,360 --> 00:00:44,400
All right, we're back,
now let's do it together.
10
00:00:44,400 --> 00:00:46,050
Now, I know Professor
Strang has done
11
00:00:46,050 --> 00:00:50,600
a couple of these in lecture,
but as he pointed out there,
12
00:00:50,600 --> 00:00:52,430
it's really easy
to make a mistake,
13
00:00:52,430 --> 00:00:59,460
so you can never do enough
examples of finding the SVD.
14
00:00:59,460 --> 00:01:02,690
So, what does the SVD look like?
15
00:01:02,690 --> 00:01:05,190
What do we want to end up with?
16
00:01:05,190 --> 00:01:16,620
Well, we want a decomposition
C equals U sigma V transpose.
17
00:01:16,620 --> 00:01:20,620
U and V are going to be
orthogonal matrices, that
18
00:01:20,620 --> 00:01:25,690
is, their columns
are orthonormal sets.
19
00:01:25,690 --> 00:01:28,760
Sigma is going to
be a diagonal matrix
20
00:01:28,760 --> 00:01:31,690
with non-negative entries.
21
00:01:31,690 --> 00:01:32,610
OK, good.
22
00:01:32,610 --> 00:01:34,930
So now, how do we find
this decomposition?
23
00:01:34,930 --> 00:01:39,640
Well, we need two equations, OK?
24
00:01:39,640 --> 00:01:52,070
One is C transpose C is equal
to V, sigma transpose, sigma,
25
00:01:52,070 --> 00:01:54,260
V transpose.
26
00:01:54,260 --> 00:01:57,390
And you get this just by
plugging in C transpose C
27
00:01:57,390 --> 00:02:00,940
here and noticing that U
transpose U is 1, since U
28
00:02:00,940 --> 00:02:02,752
is an orthogonal matrix.
29
00:02:02,752 --> 00:02:04,200
Okay.
30
00:02:04,200 --> 00:02:08,620
And the second equation is just
noticing that V transpose is V
31
00:02:08,620 --> 00:02:11,640
inverse, and moving it to the
other side of the equation,
32
00:02:11,640 --> 00:02:15,930
which is C*V equals U*sigma.
33
00:02:15,930 --> 00:02:18,400
OK, so these are
the two equations
34
00:02:18,400 --> 00:02:24,470
we need to use to find
V, sigma, and U. OK,
35
00:02:24,470 --> 00:02:28,040
so let's start
with the first one.
36
00:02:28,040 --> 00:02:33,070
Let's compute C transpose
C. So C transpose C
37
00:02:33,070 --> 00:02:47,220
is that-- Well, if
you compute, we'll
38
00:02:47,220 --> 00:03:00,300
get a 26, an 18, an
18, and a 74, great.
39
00:03:00,300 --> 00:03:03,100
Now, what you notice
about this equation
40
00:03:03,100 --> 00:03:06,680
is this is just a
diagonalization of C transpose
41
00:03:06,680 --> 00:03:10,460
C. So we need to find
the eigenvalues-- those
42
00:03:10,460 --> 00:03:13,780
will be the entries
of sigma transpose
43
00:03:13,780 --> 00:03:16,160
sigma-- and the
eigenvectors which will be
44
00:03:16,160 --> 00:03:19,910
the columns of a V. Okay, good.
45
00:03:22,490 --> 00:03:24,810
So how do we find those?
46
00:03:24,810 --> 00:03:30,690
Well, we look at the determinant
of C transpose C minus lambda
47
00:03:30,690 --> 00:03:35,730
times the identity,
which is the determinant
48
00:03:35,730 --> 00:03:48,710
of 26 minus lambda, 18, 18,
and 74-- 74 minus lambda,
49
00:03:48,710 --> 00:03:49,320
thank you.
50
00:03:55,520 --> 00:03:59,860
Good, OK, and what
is that polynomial?
51
00:03:59,860 --> 00:04:06,460
Well, we get a lambda squared,
now the 26 plus 74 is 100,
52
00:04:06,460 --> 00:04:09,130
so minus 100*lambda.
53
00:04:09,130 --> 00:04:14,310
And I'll let you do 26 times 74
minus 18 squared on your own,
54
00:04:14,310 --> 00:04:22,100
but you'll see you get 1,600,
and this easily factors
55
00:04:22,100 --> 00:04:25,080
as lambda minus 20
times lambda minus 80.
56
00:04:27,880 --> 00:04:31,550
So the eigenvalues
are 20 and 80.
57
00:04:31,550 --> 00:04:33,150
Now what are the eigenvectors?
58
00:04:33,150 --> 00:04:39,850
Well, you take C transpose C
minus 20 times the identity,
59
00:04:39,850 --> 00:04:47,486
and you get 6, 18, 18 and 54.
60
00:04:50,400 --> 00:04:54,550
To find the eigenvector
with eigenvalue 20,
61
00:04:54,550 --> 00:04:58,940
we need to find a vector in
the null space of this matrix.
62
00:04:58,940 --> 00:05:02,650
Note that the second
column is three times
63
00:05:02,650 --> 00:05:08,430
the first column, so our
first vector, v_1, we can just
64
00:05:08,430 --> 00:05:16,220
take that to be, well, we
could take it to be [-3, 1],
65
00:05:16,220 --> 00:05:21,040
but we'd like it to be
a unit vector, right?
66
00:05:21,040 --> 00:05:23,960
Remember the columns of
v should be unit vectors
67
00:05:23,960 --> 00:05:25,600
because they're orthonormal.
68
00:05:25,600 --> 00:05:28,699
So 3 squared plus
1 squared is 10,
69
00:05:28,699 --> 00:05:30,490
we have to divide by
the square root of 10.
70
00:05:34,260 --> 00:05:43,580
OK, similarly, we do C transpose
C minus 80 times the identity,
71
00:05:43,580 --> 00:05:56,400
we'll get -54, 18; 18,
and -6, and similarly
72
00:05:56,400 --> 00:06:05,000
we can find that v_2 will
be 1 over square root of 10,
73
00:06:05,000 --> 00:06:08,360
3 over the square root of 10.
74
00:06:08,360 --> 00:06:11,550
Great, OK, so what
information do we have now?
75
00:06:11,550 --> 00:06:15,070
we have our V matrix, which
is just made up of these two
76
00:06:15,070 --> 00:06:18,720
columns, and we actually
have our sigma matrix too,
77
00:06:18,720 --> 00:06:23,670
because the squares of the
diagonal entries of sigma
78
00:06:23,670 --> 00:06:26,250
are 20 and 80.
79
00:06:26,250 --> 00:06:33,130
Good, so let's write those
down, write down what we have.
80
00:06:33,130 --> 00:06:40,640
So we have V-- I just
add these vectors
81
00:06:40,640 --> 00:06:43,120
and make them the
columns of my matrix.
82
00:06:43,120 --> 00:06:48,770
Square root of 10, 1
over square root of 10;
83
00:06:48,770 --> 00:06:55,570
1 over square root of 10,
3 over square root of 10.
84
00:06:55,570 --> 00:07:02,490
And sigma, this is just the
square roots of 20 and 80,
85
00:07:02,490 --> 00:07:08,085
which is just 2 root 5 and
4 root 5 along the diagonal.
86
00:07:11,200 --> 00:07:14,500
Squeezing it in here, I hope
you all can see these two.
87
00:07:17,104 --> 00:07:21,830
Good, so these are two of the
three parts of my singular
88
00:07:21,830 --> 00:07:24,940
value decomposition.
89
00:07:24,940 --> 00:07:27,190
The last thing I
need to find is u,
90
00:07:27,190 --> 00:07:31,600
and for that I need to use this
second equation right here.
91
00:07:31,600 --> 00:07:37,470
So you need to multiply
C times V, okay so So
92
00:07:37,470 --> 00:07:47,550
c is [5, 5; -1, 7],
let's multiply it by V,
93
00:07:47,550 --> 00:07:54,380
1 over root 10, 3 over
square root of 10.
94
00:07:54,380 --> 00:07:56,960
What do we get?
95
00:07:56,960 --> 00:08:01,920
Well, I'll let you
work out the details,
96
00:08:01,920 --> 00:08:05,000
but it's not hard here.
97
00:08:05,000 --> 00:08:09,650
You get -10 over root 10, which
is just negative square root
98
00:08:09,650 --> 00:08:12,180
of 10 here.
99
00:08:12,180 --> 00:08:32,940
Then I just get 2 square
root of 10, and then I get--
100
00:08:32,940 --> 00:08:44,760
1 is 2 square root of 10 and--
101
00:08:44,760 --> 00:08:49,730
I think I made an error here.
102
00:08:49,730 --> 00:08:55,350
Give me a second to look
through my computation again.
103
00:08:55,350 --> 00:08:58,300
AUDIENCE: [INAUDIBLE]
104
00:08:58,300 --> 00:09:03,410
PROFESSOR: The (2, 1) entry
should be-- oh, yes, thank you.
105
00:09:03,410 --> 00:09:05,660
The (2, 1) entry should
be the square root of 10.
106
00:09:05,660 --> 00:09:20,980
Good, yes, that's what I was
hoping, yes, because we get--
107
00:09:20,980 --> 00:09:24,380
Yes, I did it in
the wrong order,
108
00:09:24,380 --> 00:09:28,460
right, so your recitation
instructor should
109
00:09:28,460 --> 00:09:33,970
know how to multiply matrices,
great, yes, thank you.
110
00:09:33,970 --> 00:09:37,580
You multiply this first, then
this, then this, and then this,
111
00:09:37,580 --> 00:09:44,030
and if you do it correctly
you will get this matrix here.
112
00:09:44,030 --> 00:09:45,510
Good, great.
113
00:09:45,510 --> 00:09:49,980
So now I'd like this
to be my U matrix,
114
00:09:49,980 --> 00:09:55,990
but it's actually U times sigma,
so I need to make these entries
115
00:09:55,990 --> 00:09:57,570
unit length.
116
00:09:57,570 --> 00:10:08,260
OK, so I get -1 over root 2, 1
over root 2, 1 over root 2, 1
117
00:10:08,260 --> 00:10:12,120
over root 2, times
my sigma matrix
118
00:10:12,120 --> 00:10:17,740
here, which is, remember,
2 square root of 5,
119
00:10:17,740 --> 00:10:22,130
4 square root of 5,
and these constants
120
00:10:22,130 --> 00:10:25,390
are just what I needed to
divide these columns by in order
121
00:10:25,390 --> 00:10:27,840
to make them unit vectors.
122
00:10:27,840 --> 00:10:33,630
So now, here's my U matrix,
1 over square root of 2,
123
00:10:33,630 --> 00:10:38,250
-1 over square root of 2;
1 over square root of 2,
124
00:10:38,250 --> 00:10:41,160
1 over square root of 2, good.
125
00:10:41,160 --> 00:10:46,710
So now I have all three
matrices, U, V, and sigma
126
00:10:46,710 --> 00:10:49,340
and despite some little
errors here and there,
127
00:10:49,340 --> 00:10:51,950
I think this is actually right.
128
00:10:51,950 --> 00:10:54,340
You should go check it
yourself, because if you're
129
00:10:54,340 --> 00:10:59,370
at all like me, you've screwed
up several times by now.
130
00:10:59,370 --> 00:11:02,190
But anyway, this is
a good illustration
131
00:11:02,190 --> 00:11:05,120
of how to find the singular
value decomposition.
132
00:11:05,120 --> 00:11:07,880
Recall that you're
looking for U sigma V
133
00:11:07,880 --> 00:11:11,990
transpose where u and v
are orthogonal matrices,
134
00:11:11,990 --> 00:11:16,410
and sigma is diagonal
with non-negative entries.
135
00:11:16,410 --> 00:11:21,290
And you find it using
these two equations,
136
00:11:21,290 --> 00:11:26,540
you compute C transpose C,
that's V sigma transpose sigma
137
00:11:26,540 --> 00:11:32,340
times V transpose, and you
also have C*V is U*sigma.
138
00:11:32,340 --> 00:11:35,100
I hope this was a
helpful illustration.