1
00:00:14,160 --> 00:00:14,860
Okay.
2
00:00:14,860 --> 00:00:21,760
This is the lecture on the
singular value decomposition.
3
00:00:21,760 --> 00:00:27,030
But everybody calls it the SVD.
4
00:00:27,030 --> 00:00:34,350
So this is the final and best
factorization of a matrix.
5
00:00:34,350 --> 00:00:37,270
Let me tell you what's coming.
6
00:00:37,270 --> 00:00:43,330
The factors will be, orthogonal
matrix, diagonal matrix,
7
00:00:43,330 --> 00:00:45,610
orthogonal matrix.
8
00:00:45,610 --> 00:00:48,610
So it's things that
we've seen before,
9
00:00:48,610 --> 00:00:53,180
these special good matrices,
orthogonal diagonal.
10
00:00:53,180 --> 00:00:58,810
The new point is that we
need two orthogonal matrices.
11
00:00:58,810 --> 00:01:02,720
A can be any matrix whatsoever.
12
00:01:02,720 --> 00:01:07,140
Any matrix whatsoever has this
singular value decomposition,
13
00:01:07,140 --> 00:01:11,870
so a diagonal one in the middle,
but I need two different --
14
00:01:11,870 --> 00:01:17,030
probably different orthogonal
matrices to be able to do this.
15
00:01:17,030 --> 00:01:17,560
Okay.
16
00:01:17,560 --> 00:01:22,030
And this factorization
has jumped into importance
17
00:01:22,030 --> 00:01:27,240
and is properly, I think,
maybe the bringing together
18
00:01:27,240 --> 00:01:29,620
of everything in this course.
19
00:01:29,620 --> 00:01:36,970
One thing we'll bring together
is the very good family
20
00:01:36,970 --> 00:01:39,070
of matrices that
we just studied,
21
00:01:39,070 --> 00:01:41,120
symmetric, positive, definite.
22
00:01:41,120 --> 00:01:43,730
Do you remember the
stories with those guys?
23
00:01:43,730 --> 00:01:49,160
Because they were symmetric,
their eigenvectors were
24
00:01:49,160 --> 00:01:53,880
orthogonal, so I could produce
an orthogonal matrix --
25
00:01:53,880 --> 00:01:56,070
this is my usual one.
26
00:01:56,070 --> 00:01:59,630
My usual one is the
eigenvectors and eigenvalues In
27
00:01:59,630 --> 00:02:05,540
the symmetric case, the
eigenvectors are orthogonal,
28
00:02:05,540 --> 00:02:09,430
so I've got the good --
my ordinary s has become
29
00:02:09,430 --> 00:02:12,180
an especially good Q.
30
00:02:12,180 --> 00:02:15,610
And positive definite,
my ordinary lambda
31
00:02:15,610 --> 00:02:18,260
has become a positive lambda.
32
00:02:18,260 --> 00:02:24,420
So that's the singular
value decomposition in case
33
00:02:24,420 --> 00:02:28,240
our matrix is symmetric
positive definite --
34
00:02:28,240 --> 00:02:30,650
in that case, I
don't need two --
35
00:02:30,650 --> 00:02:35,795
U and a V -- one orthogonal
matrix will do for both sides.
36
00:02:38,970 --> 00:02:41,260
So this would be
no good in general,
37
00:02:41,260 --> 00:02:45,190
because usually the eigenvector
matrix isn't orthogonal.
38
00:02:45,190 --> 00:02:49,900
So that's not what I'm after.
39
00:02:49,900 --> 00:02:56,790
I'm looking for orthogonal
times diagonal times orthogonal.
40
00:02:56,790 --> 00:03:00,250
And let me show you what that
means and where it comes from.
41
00:03:00,250 --> 00:03:01,880
Okay.
42
00:03:01,880 --> 00:03:02,680
What does it mean?
43
00:03:05,440 --> 00:03:10,310
You remember the picture of
any linear transformation.
44
00:03:10,310 --> 00:03:16,670
This was, like, the
most important figure in
45
00:03:16,670 --> 00:03:19,810
And what I looking for now?
46
00:03:19,810 --> 00:03:26,020
A typical vector
in the row space --
47
00:03:26,020 --> 00:03:28,940
typical vector,
let me call it v1,
48
00:03:28,940 --> 00:03:35,470
gets taken over to some vector
in the column space, say u1.
49
00:03:35,470 --> 00:03:37,850
So u1 is Av1.
50
00:03:41,350 --> 00:03:43,320
Okay.
51
00:03:43,320 --> 00:03:48,800
Now, another vector gets
taken over here somewhere.
52
00:03:48,800 --> 00:03:50,290
What I looking for?
53
00:03:50,290 --> 00:03:54,590
In this SVD, this singular
value decomposition,
54
00:03:54,590 --> 00:03:59,950
what I'm looking for is
an orthogonal basis here
55
00:03:59,950 --> 00:04:06,130
that gets knocked over into an
orthogonal basis over there.
56
00:04:06,130 --> 00:04:11,810
See that's pretty special,
to have an orthogonal basis
57
00:04:11,810 --> 00:04:18,180
in the row space that goes over
into an orthogonal basis --
58
00:04:18,180 --> 00:04:21,490
so this is like a right angle
and this is a right angle --
59
00:04:21,490 --> 00:04:25,700
into an orthogonal basis
in the column space.
60
00:04:25,700 --> 00:04:30,940
So that's our
goal, is to find --
61
00:04:30,940 --> 00:04:34,230
do you see how things
are coming together?
62
00:04:34,230 --> 00:04:37,590
First of all, can I
find an orthogonal basis
63
00:04:37,590 --> 00:04:39,960
for this row space?
64
00:04:39,960 --> 00:04:41,100
Of course.
65
00:04:41,100 --> 00:04:44,210
No big deal to find
an orthogonal basis.
66
00:04:44,210 --> 00:04:46,870
Graham Schmidt tells
me how to do it.
67
00:04:46,870 --> 00:04:50,710
Start with any old basis and
grind through Graham Schmidt,
68
00:04:50,710 --> 00:04:54,100
out comes an orthogonal basis.
69
00:04:54,100 --> 00:04:58,280
But then, if I just take any
old orthogonal basis, then
70
00:04:58,280 --> 00:05:00,990
when I multiply by
A, there's no reason
71
00:05:00,990 --> 00:05:04,770
why it should be
orthogonal over here.
72
00:05:04,770 --> 00:05:06,800
So I'm looking for
this special set
73
00:05:06,800 --> 00:05:13,850
up where A takes these basis
vectors into orthogonal vectors
74
00:05:13,850 --> 00:05:14,780
over there.
75
00:05:14,780 --> 00:05:18,570
Now, you might have
noticed that the null space
76
00:05:18,570 --> 00:05:19,400
I didn't include.
77
00:05:19,400 --> 00:05:22,400
Why don't I stick that in?
78
00:05:22,400 --> 00:05:25,770
You remember our usual figure
had a little null space
79
00:05:25,770 --> 00:05:28,730
and a little null space.
80
00:05:28,730 --> 00:05:31,650
And those are no problems.
81
00:05:31,650 --> 00:05:34,000
Those null spaces
are going to show up
82
00:05:34,000 --> 00:05:37,510
as zeroes on the
diagonal of sigma,
83
00:05:37,510 --> 00:05:43,730
so that doesn't
present any difficulty.
84
00:05:43,730 --> 00:05:46,250
Our difficulty is to find these.
85
00:05:46,250 --> 00:05:48,220
So do you see what
this will mean?
86
00:05:48,220 --> 00:05:57,950
This will mean that A times
these v-s, v1, v2, up to --
87
00:05:57,950 --> 00:06:01,460
what's the dimension
of this row space?
88
00:06:01,460 --> 00:06:03,180
Vr.
89
00:06:03,180 --> 00:06:07,390
Sorry, make that V a
little smaller -- up to vr.
90
00:06:10,650 --> 00:06:12,990
So that's --
91
00:06:12,990 --> 00:06:16,680
Av1 is going to be
the first column,
92
00:06:16,680 --> 00:06:20,250
so here's what I'm achieving.
93
00:06:20,250 --> 00:06:24,390
Oh, I'm not only going
to make these orthogonal,
94
00:06:24,390 --> 00:06:27,030
but why not make
them orthonormal?
95
00:06:27,030 --> 00:06:28,880
Make them unit vectors.
96
00:06:28,880 --> 00:06:34,380
So maybe the unit vector
is here, is the u1,
97
00:06:34,380 --> 00:06:38,510
and this might be
a multiple of it.
98
00:06:38,510 --> 00:06:45,560
So really, what's happening
is Av1 is some multiple of u1,
99
00:06:45,560 --> 00:06:46,660
right?
100
00:06:46,660 --> 00:06:50,040
These guys will be unit
vectors and they'll
101
00:06:50,040 --> 00:06:52,940
go over into multiples
of unit vectors
102
00:06:52,940 --> 00:06:56,820
and the multiple I'm not
going to call lambda anymore.
103
00:06:56,820 --> 00:06:58,090
I'm calling it sigma.
104
00:06:58,090 --> 00:07:01,480
So that's the number --
the stretching number.
105
00:07:01,480 --> 00:07:06,000
And similarly, Av2
is sigma two u2.
106
00:07:06,000 --> 00:07:09,000
This is my goal.
107
00:07:09,000 --> 00:07:13,730
And now I want to express
that goal in matrix language.
108
00:07:13,730 --> 00:07:15,340
That's the usual step.
109
00:07:15,340 --> 00:07:18,400
Think of what you want
and then express it
110
00:07:18,400 --> 00:07:20,420
as a matrix multiplication.
111
00:07:20,420 --> 00:07:25,800
So Av1 is sigma one u1 --
112
00:07:25,800 --> 00:07:27,960
actually, here we go.
113
00:07:27,960 --> 00:07:30,710
Let me pull out these --
114
00:07:30,710 --> 00:07:36,765
u1, u2 to ur and then a
matrix with the sigmas.
115
00:07:40,460 --> 00:07:45,970
Everything now is going
to be in that little part
116
00:07:45,970 --> 00:07:46,720
of the blackboard.
117
00:07:46,720 --> 00:07:52,880
Do you see that this
equation says what I'm
118
00:07:52,880 --> 00:07:55,740
trying to do with my figure.
119
00:07:55,740 --> 00:08:00,770
A times the first basis vector
should be sigma one times
120
00:08:00,770 --> 00:08:04,580
the other basis -- the
other first basis vector.
121
00:08:04,580 --> 00:08:08,150
These are the basis
vectors in the row space,
122
00:08:08,150 --> 00:08:11,470
these are the basis
vectors in the column space
123
00:08:11,470 --> 00:08:14,680
and these are the
multiplying factors.
124
00:08:14,680 --> 00:08:23,660
So Av2 is sigma two times
u2, Avr is sigma r times ur.
125
00:08:23,660 --> 00:08:27,170
And then we've got a whole lot
of zeroes and maybe some zeroes
126
00:08:27,170 --> 00:08:31,480
at the end, but that's
the heart of it.
127
00:08:31,480 --> 00:08:34,064
And now if I express that in --
128
00:08:40,000 --> 00:08:43,350
as matrices, because you
knew that was coming --
129
00:08:43,350 --> 00:08:45,140
that's what I have.
130
00:08:45,140 --> 00:08:51,520
So, this is my goal, to
find an orthogonal basis
131
00:08:51,520 --> 00:08:55,500
in the orthonormal, even
-- basis in the row space
132
00:08:55,500 --> 00:09:00,420
and an orthonormal basis in the
column space so that I've sort
133
00:09:00,420 --> 00:09:02,590
of diagonalized the matrix.
134
00:09:02,590 --> 00:09:05,530
The matrix A is, like,
getting converted
135
00:09:05,530 --> 00:09:08,660
to this diagonal matrix sigma.
136
00:09:08,660 --> 00:09:13,140
And you notice that usually
I have to allow myself
137
00:09:13,140 --> 00:09:15,980
two different bases.
138
00:09:15,980 --> 00:09:19,230
My little comment about
symmetric positive definite
139
00:09:19,230 --> 00:09:24,860
was the one case where
it's A Q equal Q sigma,
140
00:09:24,860 --> 00:09:27,430
where V and U are the same Q.
141
00:09:27,430 --> 00:09:32,310
But mostly, you know, I'm going
to take a matrix like -- oh,
142
00:09:32,310 --> 00:09:38,240
let me take a matrix like
four four minus three three.
143
00:09:38,240 --> 00:09:39,540
Okay.
144
00:09:39,540 --> 00:09:42,420
There's a two by two matrix.
145
00:09:42,420 --> 00:09:48,430
It's invertible,
so it has rank two.
146
00:09:48,430 --> 00:09:51,210
So I'm going to look
for two vectors,
147
00:09:51,210 --> 00:09:55,230
v1 and v2 in the
row space, and U --
148
00:09:55,230 --> 00:10:00,770
so I'm going to look for
v1, v2 in the row space,
149
00:10:00,770 --> 00:10:05,350
which of course is R^2.
150
00:10:05,350 --> 00:10:10,280
And I'm going to look for
u1, u2 in the column space,
151
00:10:10,280 --> 00:10:15,620
which of course is also R^2, and
I'm going to look for numbers
152
00:10:15,620 --> 00:10:21,160
sigma one and sigma two so
that it all comes out right.
153
00:10:21,160 --> 00:10:26,060
So these guys are orthonormal,
these guys are orthonormal
154
00:10:26,060 --> 00:10:28,820
and these are the
scaling factors.
155
00:10:28,820 --> 00:10:33,510
So I'll do that example as
soon as I get the matrix
156
00:10:33,510 --> 00:10:35,030
picture straight.
157
00:10:35,030 --> 00:10:36,280
Okay.
158
00:10:36,280 --> 00:10:39,380
Do you see that this
expresses what I want?
159
00:10:39,380 --> 00:10:45,360
Can I just say two
words about null spaces?
160
00:10:45,360 --> 00:10:52,010
If there's some
null space, then we
161
00:10:52,010 --> 00:10:56,080
want to stick in a basis
for those, for that.
162
00:10:56,080 --> 00:11:02,700
So here comes a basis for the
null space, v(r+1) down to vm.
163
00:11:02,700 --> 00:11:06,590
So if we only had an r
dimensional row space
164
00:11:06,590 --> 00:11:11,160
and the other n-r dimensions
were in the null space -- okay,
165
00:11:11,160 --> 00:11:14,360
we'll take an orthogonal
-- orthonormal basis there.
166
00:11:14,360 --> 00:11:15,430
No problem.
167
00:11:15,430 --> 00:11:18,500
And then we'll just get zeroes.
168
00:11:18,500 --> 00:11:21,700
So, actually, w- those
zeroes will come out
169
00:11:21,700 --> 00:11:25,040
on the diagonal matrix.
170
00:11:25,040 --> 00:11:33,250
So I'll complete that to an
orthonormal basis for the whole
171
00:11:33,250 --> 00:11:35,420
space, R^m.
172
00:11:35,420 --> 00:11:38,080
I complete this to an
orthonormal basis for the whole
173
00:11:38,080 --> 00:11:42,420
space R^n and I complete
that with zeroes.
174
00:11:42,420 --> 00:11:46,870
Null spaces are no problem here.
175
00:11:46,870 --> 00:11:52,570
So really the true problem
is in a matrix like that,
176
00:11:52,570 --> 00:11:56,170
which isn't symmetric, so I
can't use its eigenvectors,
177
00:11:56,170 --> 00:11:58,110
they're not orthogonal --
178
00:11:58,110 --> 00:12:02,740
but somehow I have to get
these orthogonal -- in fact,
179
00:12:02,740 --> 00:12:09,470
orthonormal guys
that make it work.
180
00:12:09,470 --> 00:12:13,990
I have to find these orthonormal
guys, these orthonormal guys
181
00:12:13,990 --> 00:12:22,960
and I want Av1 to be sigma one
u1 and Av2 to be sigma two u2.
182
00:12:22,960 --> 00:12:23,460
Okay.
183
00:12:26,790 --> 00:12:28,260
That's my goal.
184
00:12:28,260 --> 00:12:34,310
Here's the matrices that
are going to get me there.
185
00:12:34,310 --> 00:12:36,500
Now these are
orthogonal matrices.
186
00:12:36,500 --> 00:12:41,610
I can put that -- if I multiply
on both sides by V inverse,
187
00:12:41,610 --> 00:12:47,740
I have A equals U
sigma V inverse,
188
00:12:47,740 --> 00:12:53,230
and of course you know the
other way I can write V inverse.
189
00:12:53,230 --> 00:12:57,200
This is one of those
square orthogonal matrices,
190
00:12:57,200 --> 00:13:02,731
so it's the same as
U sigma V transpose.
191
00:13:02,731 --> 00:13:03,230
Okay.
192
00:13:06,020 --> 00:13:08,900
Here's my problem.
193
00:13:08,900 --> 00:13:14,580
I've got two orthogonal
matrices here.
194
00:13:14,580 --> 00:13:17,330
And I don't want to
find them both at once.
195
00:13:17,330 --> 00:13:21,120
So I want to cook up
some expression that
196
00:13:21,120 --> 00:13:26,630
will make the Us disappear.
197
00:13:26,630 --> 00:13:29,010
I would like to make the
Us disappear and leave me
198
00:13:29,010 --> 00:13:31,070
only with the Vs.
199
00:13:31,070 --> 00:13:34,200
And here's how to do it.
200
00:13:34,200 --> 00:13:37,520
It's the same combination
that keeps showing up
201
00:13:37,520 --> 00:13:41,040
whenever we have a general
rectangular matrix,
202
00:13:41,040 --> 00:13:46,640
then it's A transpose A,
that's the great matrix.
203
00:13:46,640 --> 00:13:48,760
That's the great matrix.
204
00:13:48,760 --> 00:13:50,530
That's the matrix
that's symmetric,
205
00:13:50,530 --> 00:13:53,910
and in fact positive
definite or at least
206
00:13:53,910 --> 00:13:55,240
positive semi-definite.
207
00:13:55,240 --> 00:13:58,300
This is the matrix with nice
properties, so let's see what
208
00:13:58,300 --> 00:13:59,230
will it be?
209
00:13:59,230 --> 00:14:03,100
So if I took the transpose,
then, I would have --
210
00:14:03,100 --> 00:14:06,140
A transpose A will be what?
211
00:14:06,140 --> 00:14:06,920
What do I have?
212
00:14:06,920 --> 00:14:12,580
If I transpose that I have V
sigma transpose U transpose,
213
00:14:12,580 --> 00:14:14,900
that's the A transpose.
214
00:14:14,900 --> 00:14:18,350
Now the A --
215
00:14:18,350 --> 00:14:19,290
and what have I got?
216
00:14:21,930 --> 00:14:25,960
Looks like worse, because it's
got six things now together,
217
00:14:25,960 --> 00:14:30,660
but it's going to collapse
into something good.
218
00:14:30,660 --> 00:14:34,300
What does U transpose
U collapse into?
219
00:14:34,300 --> 00:14:35,770
I, the identity.
220
00:14:35,770 --> 00:14:37,330
So that's the key point.
221
00:14:37,330 --> 00:14:42,050
This is the identity and
we don't have U anymore.
222
00:14:42,050 --> 00:14:44,380
And sigma transpose
times sigma, those
223
00:14:44,380 --> 00:14:48,300
are diagonal matrixes,
so their product is just
224
00:14:48,300 --> 00:14:50,590
going to have sigma
squareds on the diagonal.
225
00:14:50,590 --> 00:14:52,650
So do you see what
we've got here?
226
00:14:52,650 --> 00:14:57,440
This is V times this
easy matrix sigma
227
00:14:57,440 --> 00:15:03,080
one squared sigma two
squared times V transpose.
228
00:15:03,080 --> 00:15:05,840
This is the A transpose A.
229
00:15:05,840 --> 00:15:07,590
This is -- let me copy down --
230
00:15:07,590 --> 00:15:09,185
A transpose A is that.
231
00:15:12,760 --> 00:15:14,170
Us are out of the picture, now.
232
00:15:14,170 --> 00:15:17,900
I'm only having to choose the
Vs, and what are these Vs?
233
00:15:17,900 --> 00:15:19,370
And what are these sigmas?
234
00:15:19,370 --> 00:15:26,750
Do you know what the Vs are?
235
00:15:26,750 --> 00:15:28,650
They're the eigenvectors
that -- see,
236
00:15:28,650 --> 00:15:34,120
this is a perfect
eigenvector, eigenvalue,
237
00:15:34,120 --> 00:15:41,920
Q lambda Q transpose for
the matrix A transpose A.
238
00:15:41,920 --> 00:15:45,020
A itself is nothing special.
239
00:15:45,020 --> 00:15:48,620
But A transpose A
will be special.
240
00:15:48,620 --> 00:15:50,700
It'll be symmetric
positive definite,
241
00:15:50,700 --> 00:15:55,340
so this will be its eigenvectors
and this'll be its eigenvalues.
242
00:15:55,340 --> 00:15:59,100
And the eigenvalues'll be
positive because this thing's
243
00:15:59,100 --> 00:16:02,400
positive definite.
244
00:16:02,400 --> 00:16:04,650
So this is my method.
245
00:16:04,650 --> 00:16:07,310
This tells me what the Vs are.
246
00:16:07,310 --> 00:16:09,210
And how I going to find the Us?
247
00:16:11,730 --> 00:16:18,570
Well, one way would be
to look at A A transpose.
248
00:16:18,570 --> 00:16:21,650
Multiply A by A transpose
in the opposite order.
249
00:16:21,650 --> 00:16:24,340
That will stick the
Vs in the middle,
250
00:16:24,340 --> 00:16:27,130
knock them out, and
leave me with the Us.
251
00:16:27,130 --> 00:16:29,620
So here's the overall
picture, then.
252
00:16:29,620 --> 00:16:36,270
The Vs are the eigenvectors
of A transpose A.
253
00:16:36,270 --> 00:16:38,180
The Us are the
eigenvectors of A A
254
00:16:38,180 --> 00:16:39,860
transpose, which are different.
255
00:16:39,860 --> 00:16:43,940
And the sigmas are
the square roots
256
00:16:43,940 --> 00:16:48,730
of these and the
positive square roots,
257
00:16:48,730 --> 00:16:50,580
so we have positive sigmas.
258
00:16:50,580 --> 00:16:52,780
Let me do it for that example.
259
00:16:52,780 --> 00:16:56,570
This is really what
you should know
260
00:16:56,570 --> 00:17:01,460
and be able to do for the SVD.
261
00:17:01,460 --> 00:17:02,520
Okay.
262
00:17:02,520 --> 00:17:03,740
Let me take that matrix.
263
00:17:03,740 --> 00:17:06,550
So what's my first step?
264
00:17:06,550 --> 00:17:12,191
Compute A transpose A, because
I want its eigenvectors.
265
00:17:12,191 --> 00:17:12,690
Okay.
266
00:17:12,690 --> 00:17:16,690
So I have to compute
A transpose A.
267
00:17:16,690 --> 00:17:22,240
So A transpose is four
four minus three three,
268
00:17:22,240 --> 00:17:26,970
and A is four four
minus three three,
269
00:17:26,970 --> 00:17:30,810
and I do that multiplication
and I get sixteen --
270
00:17:30,810 --> 00:17:32,930
I get twenty five --
271
00:17:32,930 --> 00:17:36,300
I get sixteen minus nine --
272
00:17:36,300 --> 00:17:37,870
is that seven?
273
00:17:37,870 --> 00:17:40,260
And it better come
out symmetric.
274
00:17:40,260 --> 00:17:43,190
And -- oh, okay, and
then it comes out 25.
275
00:17:43,190 --> 00:17:43,690
Okay.
276
00:17:47,360 --> 00:17:52,030
So, I want its eigenvectors
and its eigenvalues.
277
00:17:52,030 --> 00:17:55,680
Its eigenvectors will be
the Vs, its eigenvalues
278
00:17:55,680 --> 00:17:58,990
will be the squares
of the sigmas.
279
00:17:58,990 --> 00:17:59,570
Okay.
280
00:17:59,570 --> 00:18:04,900
What are the eigenvalues and
eigenvectors of this guy?
281
00:18:04,900 --> 00:18:10,690
Have you seen that two by two
example enough to recognize
282
00:18:10,690 --> 00:18:16,970
that the eigenvectors are --
that one one is an eigenvector?
283
00:18:16,970 --> 00:18:19,210
So this here is A transpose A.
284
00:18:19,210 --> 00:18:22,300
I'm looking for
its eigenvectors.
285
00:18:22,300 --> 00:18:27,620
So its eigenvectors, I think,
are one one and one minus one,
286
00:18:27,620 --> 00:18:29,800
because if I
multiply that matrix
287
00:18:29,800 --> 00:18:32,390
by one one, what do I get?
288
00:18:32,390 --> 00:18:37,720
If I multiply that matrix
by one one, I get 32 32,
289
00:18:37,720 --> 00:18:41,530
which is 32 of one one.
290
00:18:41,530 --> 00:18:45,240
So there's the
first eigenvector,
291
00:18:45,240 --> 00:18:49,360
and there's the eigenvalue
for A transpose A.
292
00:18:49,360 --> 00:18:59,120
So I'm going to take its
square root for sigma.
293
00:18:59,120 --> 00:18:59,620
Okay.
294
00:18:59,620 --> 00:19:01,310
What's the eigenvector
that goes --
295
00:19:01,310 --> 00:19:03,410
eigenvalue that
goes with this one?
296
00:19:03,410 --> 00:19:05,730
If I do that multiplication,
what do I get?
297
00:19:05,730 --> 00:19:12,420
I get some multiple of one minus
one, and what is that multiple?
298
00:19:12,420 --> 00:19:13,250
Looks like 18.
299
00:19:16,470 --> 00:19:17,340
Okay.
300
00:19:17,340 --> 00:19:20,880
So those are the two
eigenvectors, but -- oh,
301
00:19:20,880 --> 00:19:24,300
just a moment, I
didn't normalize them.
302
00:19:24,300 --> 00:19:27,310
To make everything
absolutely right,
303
00:19:27,310 --> 00:19:30,180
I ought to normalize
these eigenvectors,
304
00:19:30,180 --> 00:19:33,370
divide by their length,
square root of two.
305
00:19:33,370 --> 00:19:42,890
So all these guys should be true
unit vectors and, of course,
306
00:19:42,890 --> 00:19:46,900
that normalization didn't
change the 32 and the 18.
307
00:19:46,900 --> 00:19:48,500
Okay.
308
00:19:48,500 --> 00:19:51,980
So I'm happy with the Vs.
309
00:19:51,980 --> 00:19:53,380
Here are the Vs.
310
00:19:53,380 --> 00:19:57,490
So now let me put
together the pieces here.
311
00:19:57,490 --> 00:19:59,340
Here's my A.
312
00:19:59,340 --> 00:20:01,060
Here's my A.
313
00:20:01,060 --> 00:20:03,550
Let me write down A again.
314
00:20:07,860 --> 00:20:16,150
If life is right, we should get
U, which I don't yet know --
315
00:20:16,150 --> 00:20:19,870
U I don't yet know,
sigma I do now know.
316
00:20:19,870 --> 00:20:21,040
What's sigma?
317
00:20:21,040 --> 00:20:24,080
So I'm looking for a
U sigma V transpose.
318
00:20:24,080 --> 00:20:28,115
U, the diagonal guy
and V transpose.
319
00:20:31,500 --> 00:20:32,030
Okay.
320
00:20:32,030 --> 00:20:33,740
Let's just see that
come out right.
321
00:20:33,740 --> 00:20:36,780
So what are the sigmas?
322
00:20:36,780 --> 00:20:39,130
They're the square
roots of these things.
323
00:20:39,130 --> 00:20:43,820
So square root of 32
and square root of 18.
324
00:20:47,900 --> 00:20:49,120
Zero zero.
325
00:20:49,120 --> 00:20:50,210
Okay.
326
00:20:50,210 --> 00:20:52,040
What are the Vs?
327
00:20:52,040 --> 00:20:53,810
They're these two.
328
00:20:53,810 --> 00:20:56,900
And I have to transpose --
329
00:20:56,900 --> 00:20:59,450
maybe that just
leaves me with ones --
330
00:20:59,450 --> 00:21:03,790
with one over square root of two
in that row and the other one
331
00:21:03,790 --> 00:21:06,870
is one over square
root of two minus one
332
00:21:06,870 --> 00:21:08,010
over square root of two.
333
00:21:11,590 --> 00:21:15,030
Now finally, I've
got to know the Us.
334
00:21:15,030 --> 00:21:18,570
Well, actually, one way to do --
since I now know all the other
335
00:21:18,570 --> 00:21:21,100
pieces, I could put those
together and figure out what
336
00:21:21,100 --> 00:21:22,110
the Us are.
337
00:21:22,110 --> 00:21:25,780
But let me do it the
A A transpose way.
338
00:21:25,780 --> 00:21:26,280
Okay.
339
00:21:26,280 --> 00:21:27,380
Find the Us now.
340
00:21:31,390 --> 00:21:33,070
u1 and u2.
341
00:21:33,070 --> 00:21:34,650
And what are they?
342
00:21:37,220 --> 00:21:41,240
I look at A A transpose --
343
00:21:41,240 --> 00:21:47,230
so A is supposed to be U
sigma V transpose, and then
344
00:21:47,230 --> 00:21:52,652
when I transpose that I get V
sigma transpose U transpose.
345
00:21:57,210 --> 00:21:59,160
So I'm just doing it
in the opposite order,
346
00:21:59,160 --> 00:22:03,470
A times A transpose, and
what's the good part here?
347
00:22:03,470 --> 00:22:10,480
That in the middle, V transpose
V is going to be the identity.
348
00:22:10,480 --> 00:22:14,990
So this is just U
sigma sigma transpose,
349
00:22:14,990 --> 00:22:22,340
that's some diagonal matrix with
sigma squareds and U transpose.
350
00:22:22,340 --> 00:22:24,480
So what I seeing here?
351
00:22:24,480 --> 00:22:29,280
I'm seeing here, again, a
symmetric positive definite
352
00:22:29,280 --> 00:22:32,000
or at least semi-definite
matrix and I'm
353
00:22:32,000 --> 00:22:36,600
seeing its eigenvectors
and its eigenvalues.
354
00:22:36,600 --> 00:22:41,590
So if I compute A A
transpose, its eigenvectors
355
00:22:41,590 --> 00:22:44,060
will be the things
that go into U.
356
00:22:44,060 --> 00:22:47,470
Okay, so I need to
compute A A transpose.
357
00:22:47,470 --> 00:22:50,970
I guess I'm going
to have to go --
358
00:22:50,970 --> 00:22:53,450
can I move that
up just a little?
359
00:22:53,450 --> 00:22:56,580
Maybe a little more
and do A A transpose.
360
00:22:59,880 --> 00:23:01,820
So what's A?
361
00:23:01,820 --> 00:23:05,930
Four four minus three and three.
362
00:23:05,930 --> 00:23:07,550
And what's A transpose?
363
00:23:07,550 --> 00:23:10,310
Four four minus three and three.
364
00:23:10,310 --> 00:23:15,530
And when I do that
multiplication, what do I get?
365
00:23:15,530 --> 00:23:18,750
Sixteen and sixteen, thirty two.
366
00:23:18,750 --> 00:23:21,630
Uh, that one comes out zero.
367
00:23:21,630 --> 00:23:26,940
Oh, so this is a lucky case
and that one comes out 18.
368
00:23:26,940 --> 00:23:31,560
So this is an accident
that A A transpose
369
00:23:31,560 --> 00:23:38,030
happens to come out diagonal, so
we know easily its eigenvectors
370
00:23:38,030 --> 00:23:38,990
and eigenvalues.
371
00:23:38,990 --> 00:23:43,130
So its eigenvectors -- what's
the first eigenvector for this
372
00:23:43,130 --> 00:23:45,150
A A transpose matrix?
373
00:23:45,150 --> 00:23:49,940
It's just one zero, and when
I do that multiplication,
374
00:23:49,940 --> 00:23:54,020
I get 32 times one zero.
375
00:23:54,020 --> 00:23:57,380
And the other eigenvector
is just zero one
376
00:23:57,380 --> 00:24:00,350
and when I multiply
by that I get 18.
377
00:24:00,350 --> 00:24:04,720
So this is A A transpose.
378
00:24:04,720 --> 00:24:08,910
Multiplying that gives
me the 32 A A transpose.
379
00:24:08,910 --> 00:24:14,860
Multiplying this guy
gives me First of all,
380
00:24:14,860 --> 00:24:18,590
I got 32 and 18 again.
381
00:24:18,590 --> 00:24:19,740
Am I surprised?
382
00:24:19,740 --> 00:24:24,420
You know, it's clearly
not an accident.
383
00:24:24,420 --> 00:24:29,190
The eigenvalues of A A
transpose were exactly the same
384
00:24:29,190 --> 00:24:37,820
as the eigenvalues of --
this one was A transpose A.
385
00:24:37,820 --> 00:24:40,140
That's no surprise at all.
386
00:24:40,140 --> 00:24:47,140
The eigenvalues of A B are the
same as the eigenvalues of B A.
387
00:24:47,140 --> 00:24:50,530
That's a very nice
fact, that eigenvalues
388
00:24:50,530 --> 00:24:55,310
stay the same if I switch
the order of multiplication.
389
00:24:55,310 --> 00:25:01,650
So no surprise to see
32 and What I learned --
390
00:25:01,650 --> 00:25:05,550
first the check that things
were numerically correct,
391
00:25:05,550 --> 00:25:07,950
but now I've learned
these eigenvectors,
392
00:25:07,950 --> 00:25:11,980
and actually they're
about as nice as can be.
393
00:25:11,980 --> 00:25:16,045
They're the best orthogonal
matrix, just the identity.
394
00:25:18,511 --> 00:25:19,010
Okay.
395
00:25:21,940 --> 00:25:26,400
So my claim is that it
ought to all fit together,
396
00:25:26,400 --> 00:25:31,560
that these numbers
should come out right.
397
00:25:31,560 --> 00:25:33,440
The numbers should
come out right
398
00:25:33,440 --> 00:25:41,070
because the matrix
multiplications use
399
00:25:41,070 --> 00:25:42,450
the properties that we want.
400
00:25:42,450 --> 00:25:42,970
Okay.
401
00:25:42,970 --> 00:25:44,370
Shall we just check that?
402
00:25:44,370 --> 00:25:47,020
Here's the identity, so
not doing anything --
403
00:25:47,020 --> 00:25:50,530
square root of 32 is
multiplying that row,
404
00:25:50,530 --> 00:25:53,670
so that square root of 32
divided by square root of two
405
00:25:53,670 --> 00:25:58,150
means square root of
16, four, correct?
406
00:25:58,150 --> 00:26:01,680
And square root of 18 is
divided by square root of two,
407
00:26:01,680 --> 00:26:07,570
so that leaves me square root
of 9, which is three, but --
408
00:26:07,570 --> 00:26:11,100
well, Professor Strang,
you see the problem?
409
00:26:11,100 --> 00:26:12,740
Why is that --
410
00:26:12,740 --> 00:26:13,240
okay.
411
00:26:13,240 --> 00:26:16,790
Why I getting minus
three three here
412
00:26:16,790 --> 00:26:19,965
and here I'm getting
three minus three?
413
00:26:24,640 --> 00:26:26,240
Phooey.
414
00:26:26,240 --> 00:26:27,105
I don't know why.
415
00:26:30,980 --> 00:26:34,650
It shouldn't have
happened, but it did.
416
00:26:34,650 --> 00:26:38,402
Now, okay, you could
say, well, just --
417
00:26:41,300 --> 00:26:43,560
the eigenvector
there could have --
418
00:26:43,560 --> 00:26:47,410
I could have had the minus
sign here for that eigenvector,
419
00:26:47,410 --> 00:26:49,710
but I'm not happy about that.
420
00:26:49,710 --> 00:26:50,210
Hmm.
421
00:26:50,210 --> 00:26:50,710
Okay.
422
00:26:55,740 --> 00:26:58,480
So I realize there's a
little catch here somewhere
423
00:26:58,480 --> 00:27:02,630
and I may not see
it until Wednesday.
424
00:27:02,630 --> 00:27:04,830
Which then gives you a
very important reason
425
00:27:04,830 --> 00:27:09,930
to come back on Wednesday, to
catch that sine difference.
426
00:27:09,930 --> 00:27:14,310
So what did I do illegally?
427
00:27:14,310 --> 00:27:22,550
I think I put the eigenvectors
in that matrix V transpose --
428
00:27:22,550 --> 00:27:24,160
okay, I'm going
to have to think.
429
00:27:24,160 --> 00:27:29,460
Why did that come out with
with the opposite sines?
430
00:27:29,460 --> 00:27:30,510
So you see --
431
00:27:30,510 --> 00:27:35,000
I mean, if I had a minus
there, I would be all right,
432
00:27:35,000 --> 00:27:36,490
but I don't want that.
433
00:27:36,490 --> 00:27:45,331
I want positive entries down
the diagonal of sigma squared.
434
00:27:45,331 --> 00:27:45,830
Okay.
435
00:27:45,830 --> 00:27:51,590
It'll come to me, but, I'm
going to leave this example
436
00:27:51,590 --> 00:27:57,600
to finish.
437
00:27:57,600 --> 00:27:58,920
Okay.
438
00:27:58,920 --> 00:28:02,560
And the beauty of,
these sliding boards
439
00:28:02,560 --> 00:28:05,780
is I can make that go away.
440
00:28:05,780 --> 00:28:10,910
Can I,-- let me not
do it, though, yet.
441
00:28:10,910 --> 00:28:15,090
Let me take a second example.
442
00:28:15,090 --> 00:28:18,720
Let me take a second example
where the matrix is singular.
443
00:28:21,940 --> 00:28:24,090
So rank one.
444
00:28:24,090 --> 00:28:32,390
Okay, so let me take
as an example two,
445
00:28:32,390 --> 00:28:38,770
where my matrix A is going
to be rectangular again --
446
00:28:38,770 --> 00:28:43,220
let me just make it
four three eight six.
447
00:28:47,590 --> 00:28:48,090
Okay.
448
00:28:48,090 --> 00:28:50,830
That's a rank one matrix.
449
00:28:50,830 --> 00:28:57,430
So that has a null space and
only a one dimensional row
450
00:28:57,430 --> 00:28:59,370
space and column space.
451
00:28:59,370 --> 00:29:05,510
So actually, my picture
becomes easy for this matrix,
452
00:29:05,510 --> 00:29:09,510
because what's my row
space for this one?
453
00:29:09,510 --> 00:29:12,450
So this is two by two.
454
00:29:12,450 --> 00:29:17,000
So my pictures are
both two dimensional.
455
00:29:17,000 --> 00:29:22,390
My row space is all multiples
of that vector four three.
456
00:29:22,390 --> 00:29:24,760
So the whole -- the row
space is just a line, right?
457
00:29:27,770 --> 00:29:29,380
That's the row space.
458
00:29:29,380 --> 00:29:33,050
And the null space, of course,
is the perpendicular line.
459
00:29:33,050 --> 00:29:46,480
So the row space for this matrix
is multiples of four three.
460
00:29:46,480 --> 00:29:47,680
Typical row.
461
00:29:47,680 --> 00:29:48,520
Okay.
462
00:29:48,520 --> 00:29:50,060
What's the column space?
463
00:29:50,060 --> 00:29:55,980
The columns are all multiples of
four eight, three six, one two.
464
00:29:55,980 --> 00:30:00,315
The column space, then, goes
in, like, this direction.
465
00:30:03,040 --> 00:30:07,490
So the column space --
466
00:30:07,490 --> 00:30:09,650
when I look at those
columns, the column space --
467
00:30:09,650 --> 00:30:12,660
so it's only one dimensional,
because the rank is one.
468
00:30:12,660 --> 00:30:21,480
It's multiples of four eight.
469
00:30:21,480 --> 00:30:22,370
Okay.
470
00:30:22,370 --> 00:30:26,360
And what's the null
space of A transpose?
471
00:30:26,360 --> 00:30:30,270
It's the perpendicular guy.
472
00:30:30,270 --> 00:30:35,150
So this was the null
space of A and this is
473
00:30:35,150 --> 00:30:38,921
the null space of A transpose.
474
00:30:38,921 --> 00:30:39,420
Okay.
475
00:30:42,260 --> 00:30:48,650
What I want to say here is that
choosing these orthogonal bases
476
00:30:48,650 --> 00:30:53,750
for the row space and the column
space is, like, no problem.
477
00:30:53,750 --> 00:30:55,800
They're only one dimensional.
478
00:30:55,800 --> 00:30:58,020
So what should V be?
479
00:30:58,020 --> 00:31:02,620
V should be -- v1, but
-- yes, v1, rather --
480
00:31:02,620 --> 00:31:05,540
v1 is supposed to
be a unit vector.
481
00:31:05,540 --> 00:31:08,640
There's only one
v1 to choose here,
482
00:31:08,640 --> 00:31:11,210
only one dimension
in the row space.
483
00:31:11,210 --> 00:31:13,800
I just want to make
it a unit vector.
484
00:31:13,800 --> 00:31:17,180
So v1 will be --
485
00:31:17,180 --> 00:31:24,570
it'll be this vector, but made
into a unit vector, so four --
486
00:31:24,570 --> 00:31:25,970
point eight point six.
487
00:31:28,660 --> 00:31:30,670
Four fifths, three fifths.
488
00:31:30,670 --> 00:31:33,040
And what will be u1?
489
00:31:33,040 --> 00:31:35,740
u1 will be the
unit vector there.
490
00:31:35,740 --> 00:31:40,820
So I want to turn four eight
or one two into a unit vector,
491
00:31:40,820 --> 00:31:44,150
so u1 will be --
492
00:31:44,150 --> 00:31:47,760
let's see, if it's one two,
then what multiple of one two
493
00:31:47,760 --> 00:31:49,230
do I want?
494
00:31:49,230 --> 00:31:51,240
That has length
square root of five,
495
00:31:51,240 --> 00:31:53,460
so I have to divide by
square root of five.
496
00:31:56,140 --> 00:31:58,470
Let me complete
the singular value
497
00:31:58,470 --> 00:32:01,660
decomposition for this matrix.
498
00:32:01,660 --> 00:32:09,540
So this matrix, four
three eight six, is --
499
00:32:09,540 --> 00:32:11,520
so I know what u1 --
500
00:32:11,520 --> 00:32:19,570
here's A and I want to get U
the basis in the column space.
501
00:32:19,570 --> 00:32:24,020
And it has to start
with this guy, one
502
00:32:24,020 --> 00:32:27,290
over square root of five two
over square root of five.
503
00:32:30,520 --> 00:32:36,350
Then I want the sigma.
504
00:32:36,350 --> 00:32:37,540
Okay.
505
00:32:37,540 --> 00:32:40,370
What are we expecting
now for sigma?
506
00:32:44,400 --> 00:32:47,010
This is only a rank one matrix.
507
00:32:47,010 --> 00:32:51,720
We're only expecting a sigma
one, which I have to find,
508
00:32:51,720 --> 00:32:55,040
but zeroes here.
509
00:32:55,040 --> 00:32:55,540
Okay.
510
00:32:55,540 --> 00:32:57,420
So what's sigma one?
511
00:32:57,420 --> 00:33:02,950
It should be the --
512
00:33:02,950 --> 00:33:05,480
where did these
sigmas come from?
513
00:33:05,480 --> 00:33:08,370
They came from A
transpose A, so I --
514
00:33:08,370 --> 00:33:10,750
can I do that little
calculation over here?
515
00:33:10,750 --> 00:33:19,840
A transpose A is four three --
four three eight six times four
516
00:33:19,840 --> 00:33:23,340
three eight six.
517
00:33:23,340 --> 00:33:26,240
This had better -- this
is a rank one matrix,
518
00:33:26,240 --> 00:33:29,170
this is going to be -- the
whole thing will have rank one,
519
00:33:29,170 --> 00:33:40,240
that's 16 and 64 is 80, 12
and 48 is 60, 12 and 48 is 60,
520
00:33:40,240 --> 00:33:43,880
9 and 36 is 45.
521
00:33:43,880 --> 00:33:45,520
Okay.
522
00:33:45,520 --> 00:33:47,340
It's a rank one matrix.
523
00:33:47,340 --> 00:33:48,230
Of course.
524
00:33:48,230 --> 00:33:52,450
Every row is a
multiple of four three.
525
00:33:52,450 --> 00:33:56,820
And what's the eigen -- what are
the eigenvalues of that matrix?
526
00:33:56,820 --> 00:33:59,730
So this is the calculation
-- this is like practicing,
527
00:33:59,730 --> 00:34:00,230
now.
528
00:34:00,230 --> 00:34:04,510
What are the eigenvalues
of this rank one matrix?
529
00:34:04,510 --> 00:34:08,389
Well, tell me one eigenvalue,
since the rank is only one,
530
00:34:08,389 --> 00:34:11,920
one eigenvalue is
going to be zero.
531
00:34:11,920 --> 00:34:14,320
And then you know that
the other eigenvalue
532
00:34:14,320 --> 00:34:19,330
is going to be a
hundred and twenty five.
533
00:34:19,330 --> 00:34:22,760
So that's sigma squared,
right, in A transpose A.
534
00:34:22,760 --> 00:34:28,780
So this will be the square root
of a hundred and twenty five.
535
00:34:28,780 --> 00:34:34,900
And then finally,
the V transpose --
536
00:34:34,900 --> 00:34:37,750
the Vs will be --
537
00:34:37,750 --> 00:34:41,830
there's v1, and what's v2?
538
00:34:41,830 --> 00:34:45,170
What's v2 in the --
539
00:34:45,170 --> 00:34:50,739
how do I make this into
an orthonormal basis?
540
00:34:50,739 --> 00:34:55,480
Well, v2 is, in the
null space direction.
541
00:34:55,480 --> 00:34:59,810
It's perpendicular to that,
so point six and minus point
542
00:34:59,810 --> 00:35:00,790
eight.
543
00:35:00,790 --> 00:35:04,400
So those are the
Vs that go in here.
544
00:35:04,400 --> 00:35:11,920
Point eight, point six and
point six minus point eight.
545
00:35:11,920 --> 00:35:12,420
Okay.
546
00:35:14,960 --> 00:35:17,410
And I guess I better
finish this guy.
547
00:35:17,410 --> 00:35:21,580
So this guy, all I want is to
complete the orthonormal basis
548
00:35:21,580 --> 00:35:23,670
-- it'll be coming from there.
549
00:35:23,670 --> 00:35:28,770
It'll be a two over square
root of five and a minus one
550
00:35:28,770 --> 00:35:31,330
over square root of five.
551
00:35:31,330 --> 00:35:35,100
Let me take square root
of five out of that matrix
552
00:35:35,100 --> 00:35:38,080
to make it look better.
553
00:35:38,080 --> 00:35:44,850
So one over square root of five
times one two two minus one.
554
00:35:47,560 --> 00:35:49,920
Okay.
555
00:35:49,920 --> 00:35:53,160
So there I have -- including
the square root of five --
556
00:35:53,160 --> 00:35:56,870
that's an orthogonal matrix,
that's an orthogonal matrix,
557
00:35:56,870 --> 00:36:01,190
that's a diagonal matrix
and its rank is only one.
558
00:36:01,190 --> 00:36:03,800
And now if I do
that multiplication,
559
00:36:03,800 --> 00:36:07,550
I pray that it comes out right.
560
00:36:10,200 --> 00:36:12,022
The square root of
five will cancel
561
00:36:12,022 --> 00:36:13,480
into that square
root of one twenty
562
00:36:13,480 --> 00:36:16,390
five and leave me with the
square root of 25, which
563
00:36:16,390 --> 00:36:20,190
is five, and five will
multiply these numbers
564
00:36:20,190 --> 00:36:24,410
and I'll get whole numbers
and out will come A.
565
00:36:24,410 --> 00:36:25,230
Okay.
566
00:36:25,230 --> 00:36:31,110
That's like a second example
showing how the null space guy
567
00:36:31,110 --> 00:36:39,680
-- so this -- this vector
and this one were multiplied
568
00:36:39,680 --> 00:36:40,620
by this zero.
569
00:36:40,620 --> 00:36:46,370
So they were easy to deal with.
570
00:36:46,370 --> 00:36:50,960
Tthe key ones are the ones in
the column space and the row
571
00:36:50,960 --> 00:36:51,480
space.
572
00:36:51,480 --> 00:36:57,920
Do you see how I'm getting
columns here, diagonal here,
573
00:36:57,920 --> 00:37:02,330
rows here, coming
together to produce A.
574
00:37:02,330 --> 00:37:06,790
Okay, that's the singular
value decomposition.
575
00:37:06,790 --> 00:37:13,370
So, let me think what I want
to add to complete this topic.
576
00:37:18,130 --> 00:37:22,970
So that's two examples.
577
00:37:22,970 --> 00:37:25,740
And now let's think
what we're really doing.
578
00:37:25,740 --> 00:37:33,770
We're choosing the right
basis for the four subspaces
579
00:37:33,770 --> 00:37:35,010
of linear algebra.
580
00:37:35,010 --> 00:37:39,640
Let me write this down.
581
00:37:39,640 --> 00:37:52,785
So v1 up to vr is an orthonormal
basis for the row space.
582
00:37:58,220 --> 00:38:05,570
u1 up to ur is an orthonormal
basis for the column space.
583
00:38:09,420 --> 00:38:14,930
And then I just finish
those out by v(r+1),
584
00:38:14,930 --> 00:38:20,350
the rest up to vn is an
orthonormal basis for the null
585
00:38:20,350 --> 00:38:20,850
space.
586
00:38:24,510 --> 00:38:35,220
And finally, u(r+1) up to is an
orthonormal basis for the null
587
00:38:35,220 --> 00:38:36,320
space of A transpose.
588
00:38:39,670 --> 00:38:45,360
Do you see that we finally
got the bases right?
589
00:38:45,360 --> 00:38:51,170
They're right because they're
orthonormal, and also --
590
00:38:51,170 --> 00:38:55,100
again, Graham Schmidt would
have done this in chapter four.
591
00:38:55,100 --> 00:39:00,920
Here we needed eigenvalues,
because these bases
592
00:39:00,920 --> 00:39:03,070
make the matrix diagonal.
593
00:39:03,070 --> 00:39:09,400
A times V I is a
multiple of U I.
594
00:39:09,400 --> 00:39:11,250
So I'll put "and" --
595
00:39:14,540 --> 00:39:16,640
the matrix has
been made diagonal.
596
00:39:16,640 --> 00:39:24,730
When we choose these bases,
there's no coupling between Vs
597
00:39:24,730 --> 00:39:26,740
and no coupling between Us.
598
00:39:26,740 --> 00:39:30,550
Each A -- A times each
V is in the direction
599
00:39:30,550 --> 00:39:31,980
of the corresponding U.
600
00:39:31,980 --> 00:39:36,700
So it's exactly the
right basis for the four
601
00:39:36,700 --> 00:39:38,130
fundamental subspaces.
602
00:39:38,130 --> 00:39:41,570
And of course, their
dimensions are what we know.
603
00:39:41,570 --> 00:39:44,850
The dimension of
the row space is
604
00:39:44,850 --> 00:39:49,450
the rank r, and so is the
dimension of the column space.
605
00:39:49,450 --> 00:39:51,190
The dimension of
the null space is
606
00:39:51,190 --> 00:39:54,620
n-r, that's how many
vectors we need,
607
00:39:54,620 --> 00:39:59,790
and m-r basis vectors for the
left null space, the null space
608
00:39:59,790 --> 00:40:01,960
of A transpose.
609
00:40:01,960 --> 00:40:04,620
Okay.
610
00:40:04,620 --> 00:40:05,850
I'm going to stop there.
611
00:40:05,850 --> 00:40:10,720
I could develop
further from the SVD,
612
00:40:10,720 --> 00:40:14,140
but we'll see it again
in the very last lectures
613
00:40:14,140 --> 00:40:14,949
of the course.
614
00:40:14,949 --> 00:40:15,740
So there's the SVD.
615
00:40:15,740 --> 00:40:17,290
Thanks.