1
00:00:01 --> 00:00:12
OK, here's the last lecture in
the chapter on orthogonality.
2
00:00:12 --> 00:00:20
So we met orthogonal vectors,
two vectors,
3
00:00:20 --> 00:00:31
we met orthogonal subspaces,
like the row space and null
4
00:00:31 --> 00:00:32
space.
5
00:00:32 --> 00:00:44
Now today we meet an orthogonal
basis, and an orthogonal matrix.
6
00:00:44 --> 00:00:51
So we really --
this chapter cleans up
7
00:00:51 --> 00:00:52
orthogonality.
8
00:00:52 --> 00:00:57
And really I want -- I should
use the word orthonormal.
9
00:00:57 --> 00:01:03
Orthogonal is -- so my vectors
are q1,q2 up to qn -- I use the
10
00:01:03 --> 00:01:06
letter "q", here,
to remind me,
11
00:01:06 --> 00:01:12
I'm talking about orthogonal
things, not just any vectors,
12
00:01:12 --> 00:01:15
but orthogonal ones.
13
00:01:15 --> 00:01:17
So what does that mean?
14
00:01:17 --> 00:01:23
That means that every q is
orthogonal to every other q.
15
00:01:23 --> 00:01:28
It's a natural idea,
to have a basis that's headed
16
00:01:28 --> 00:01:33
off at ninety-degree angles,
the inner products are all
17
00:01:33 --> 00:01:34
zero.
18
00:01:34 --> 00:01:41
Of course if q is -- certainly
qi is not orthogonal to itself.
19
00:01:41 --> 00:01:44
But there we'll make the best
choice again,
20
00:01:44 --> 00:01:46
make it a unit vector.
21
00:01:46 --> 00:01:51
Then qi transpose qi is one,
for a unit vector.
22
00:01:51 --> 00:01:53
The length squared is one.
23
00:01:53 --> 00:01:57
And that's what I would use the
word normal.
24
00:01:57 --> 00:02:00
So for this part,
normalized,
25
00:02:00 --> 00:02:03
unit length for this part.
26
00:02:03 --> 00:02:03
OK.
27
00:02:03 --> 00:02:09
So first part of the lecture is
how does having an orthonormal
28
00:02:09 --> 00:02:11
basis make things nice?
29
00:02:11 --> 00:02:12
It certainly does.
30
00:02:12 --> 00:02:18
It makes all the calculations
better, a whole lot of numerical
31
00:02:18 --> 00:02:23
linear algebra is built around
working with orthonormal
32
00:02:23 --> 00:02:27
vectors,
because they never get out of
33
00:02:27 --> 00:02:30
hand, they never overflow or
underflow.
34
00:02:30 --> 00:02:35
And I'll put them into a matrix
Q, and then the second part of
35
00:02:35 --> 00:02:40
the lecture will be suppose my
basis, my columns of A are not
36
00:02:40 --> 00:02:41
orthonormal.
37
00:02:41 --> 00:02:43
How do I make them so?
38
00:02:43 --> 00:02:48.41
And the two names associated
with that simple idea are Graham
39
00:02:48.41 --> 00:02:50.36
and Schmidt.
40
00:02:50.36 --> 00:02:56
So the first part is we've got
a basis like this.
41
00:02:56 --> 00:03:02
Let's put those into the
columns of a matrix.
42
00:03:02 --> 00:03:10
So a matrix Q that has -- I'll
put these orthonormal vectors,
43
00:03:10 --> 00:03:17
q1 will be the first column,
qn will be the n-th column.
44
00:03:17 --> 00:03:24
And I want to say,
I want to write this property,
45
00:03:24 --> 00:03:30
qi transpose qj being zero,
I want to put that in a matrix
46
00:03:30 --> 00:03:31
form.
47
00:03:31 --> 00:03:36
And just the right thing is to
look at Q transpose Q.
48
00:03:36 --> 00:03:42
So this chapter has been
looking at A transpose A.
49
00:03:42 --> 00:03:47
So it's natural to look at Q
transpose Q.
50
00:03:47 --> 00:03:51
And the beauty is it comes out
perfectly.
51
00:03:51 --> 00:03:55
Because Q transpose has these
vectors in its rows,
52
00:03:55 --> 00:04:01
the first row is q1 transpose,
the nth row is qn transpose.
53
00:04:01 --> 00:04:03
So that's Q transpose.
54
00:04:03 --> 00:04:06
And now I want to multiply by
Q.
55
00:04:06 --> 00:04:10
That has q1 along to qn in the
columns.
56
00:04:10 --> 00:04:12
That's Q.
57
00:04:12 --> 00:04:14
And what do I get?
58
00:04:14 --> 00:04:19
You really -- this is the first
simplest most basic fact,
59
00:04:19 --> 00:04:25
that how do orthonormal
vectors, orthonormal columns in
60
00:04:25 --> 00:04:30
a matrix, what happens if I
compute Q transpose Q?
61
00:04:30 --> 00:04:32
Do you see it?
62
00:04:32 --> 00:04:37
If I take the first row times
the first column,
63
00:04:37 --> 00:04:39
what do I get?
64
00:04:39 --> 00:04:40.41
A one.
65
00:04:40.41 --> 00:04:44
If I take the first row times
the second column,
66
00:04:44 --> 00:04:45
what do I get?
67
00:04:45 --> 00:04:46
Zero.
68
00:04:46 --> 00:04:48
That's the orthogonality.
69
00:04:48 --> 00:04:52
The first row times the last
column is zero.
70
00:04:52 --> 00:04:57
And so I'm getting ones on the
diagonal and I'm getting zeroes
71
00:04:57 --> 00:04:58
everywhere else.
72
00:04:58 --> 00:05:01
I'm getting the identity
matrix.
73
00:05:01 --> 00:05:06.54
You see how that's --
it's just like the right
74
00:05:06.54 --> 00:05:08
calculation to do.
75
00:05:08 --> 00:05:13
If you have orthonormal
columns, and the matrix doesn't
76
00:05:13 --> 00:05:15.08
have to be square here.
77
00:05:15.08 --> 00:05:17
We might have just two columns.
78
00:05:17 --> 00:05:21
And they might have four,
lots of components.
79
00:05:21 --> 00:05:26
So but they're orthonormal,
and when we do Q transpose
80
00:05:26 --> 00:05:30
times Q,
that Q transpose times Q or A
81
00:05:30 --> 00:05:35
transpose A just asks for all
those dot products.
82
00:05:35 --> 00:05:36
Rows times columns.
83
00:05:36 --> 00:05:41
And in this orthonormal case,
we get the best possible
84
00:05:41 --> 00:05:43
answer, the identity.
85
00:05:43 --> 00:05:48
OK, so this is -- so I mean now
we have a new bunch of important
86
00:05:48 --> 00:05:49
matrices.
87
00:05:49 --> 00:05:53
What have we seen previously?
88
00:05:53 --> 00:05:59
We've seen in the distant past
we had triangular matrices,
89
00:05:59 --> 00:06:04
diagonal matrices,
permutation matrices,
90
00:06:04 --> 00:06:10
that was early chapters,
then we had row echelon forms,
91
00:06:10 --> 00:06:15
then in this chapter we've
already seen projection
92
00:06:15 --> 00:06:20
matrices,
and now we're seeing this new
93
00:06:20 --> 00:06:23.99
class of matrices with
orthonormal columns.
94
00:06:23.99 --> 00:06:26
That's a very long expression.
95
00:06:26 --> 00:06:31
I sorry that I can't just call
them orthogonal matrices.
96
00:06:31 --> 00:06:36
But that word orthogonal
matrices -- or maybe I should be
97
00:06:36 --> 00:06:41.06
able to call it orthonormal
matrices, why don't we call it
98
00:06:41.06 --> 00:06:45
orthonormal --
I mean that would be an
99
00:06:45 --> 00:06:47
absolutely perfect name.
100
00:06:47 --> 00:06:51
For Q, call it an orthonormal
matrix because its columns are
101
00:06:51 --> 00:06:52
orthonormal.
102
00:06:52 --> 00:06:58.06
OK, but the convention is that
we only use that name orthogonal
103
00:06:58.06 --> 00:07:02
matrix, we only use this -- this
word orthogonal,
104
00:07:02 --> 00:07:07
we don't even say orthonormal
for some unknown reason,
105
00:07:07 --> 00:07:10
matrix when it's square.
106
00:07:10 --> 00:07:16
So in the case when this is a
square matrix,
107
00:07:16 --> 00:07:23
that's the case we call it an
orthogonal matrix.
108
00:07:23 --> 00:07:30
And what's special about the
case when it's square?
109
00:07:30 --> 00:07:36
When it's a square matrix,
we've got its inverse,
110
00:07:36 --> 00:07:43
so --
so in the case if Q is square,
111
00:07:43 --> 00:07:51
then Q transpose Q equals I
tells us -- let me write that
112
00:07:51 --> 00:07:58
underneath -- tells us that Q
transpose is Q inverse.
113
00:07:58 --> 00:08:06.45
There we have the easy to
remember property for a square
114
00:08:06.45 --> 00:08:11
matrix with orthonormal columns.
115
00:08:11 --> 00:08:17
That -- I need to write some
examples down.
116
00:08:17 --> 00:08:20
Let's see.
117
00:08:20 --> 00:08:24
Some examples like if I take
any -- so examples,
118
00:08:24 --> 00:08:27
let's do some examples.
119
00:08:27 --> 00:08:32
Any permutation matrix,
let me take just some random
120
00:08:32 --> 00:08:34
permutation matrix.
121
00:08:34 --> 00:08:39
Permutation Q equals let's say
oh, make it three by three,
122
00:08:39 --> 00:08:42
say zero, zero,
one, one, zero,
123
00:08:42 --> 00:08:46
zero, zero, one,
zero.
124
00:08:46 --> 00:08:46
OK.
125
00:08:46 --> 00:08:51
That certainly has unit vectors
in its columns.
126
00:08:51 --> 00:08:57.67
Those vectors are certainly
perpendicular to each other.
127
00:08:57.67 --> 00:09:00
And if I -- and so that's it.
128
00:09:00 --> 00:09:02
That makes it a Q.
129
00:09:02 --> 00:09:09
And -- if I took its transpose,
if I multiplied by Q transpose,
130
00:09:09 --> 00:09:15
shall I do that --
and let me stick in Q transpose
131
00:09:15 --> 00:09:15
here.
132
00:09:15 --> 00:09:20
Just to do that multiplication
once more, transpose it'll put
133
00:09:20 --> 00:09:24
the -- make that into a column,
make that into a column,
134
00:09:24 --> 00:09:26
make that into a column.
135
00:09:26 --> 00:09:29
And the transpose is also --
another Q.
136
00:09:29 --> 00:09:31
Another orthonormal matrix.
137
00:09:31 --> 00:09:35
And when I multiply that
product I get I.
138
00:09:35 --> 00:09:37
OK, so there's an example.
139
00:09:37 --> 00:09:41
And actually there's a second
example.
140
00:09:41 --> 00:09:45
But those are real easy
examples, right,
141
00:09:45 --> 00:09:50.39
I mean to get orthogonal
columns by just putting ones in
142
00:09:50.39 --> 00:09:53
different places is like too
easy.
143
00:09:53 --> 00:09:56
So let me keep going with
examples.
144
00:09:56 --> 00:10:01
So here's another simple
example.
145
00:10:01 --> 00:10:05
Cos theta sine theta,
there's a unit vector,
146
00:10:05 --> 00:10:08
oh, let me even take it,
well, yeah.
147
00:10:08 --> 00:10:14
Cos theta sine theta and now
the other way I want sine theta
148
00:10:14 --> 00:10:15
cos theta.
149
00:10:15 --> 00:10:19
But I want the inner product to
be zero.
150
00:10:19 --> 00:10:23
And if I put a minus there,
it'll do it.
151
00:10:23 --> 00:10:27
So that's --
unit vector,
152
00:10:27 --> 00:10:29
that's a unit vector.
153
00:10:29 --> 00:10:34
And if I take the dot product,
I get minus plus zero.
154
00:10:34 --> 00:10:34
OK.
155
00:10:34 --> 00:10:39
For example Q equals say one,
one, one, minus one,
156
00:10:39 --> 00:10:42
is that an orthogonal matrix?
157
00:10:42 --> 00:10:48
I've got orthogonal columns
there, but it's not quite an
158
00:10:48 --> 00:10:51.01
orthogonal matrix.
159
00:10:51.01 --> 00:10:54
How shall I fix it to be an
orthogonal matrix?
160
00:10:54 --> 00:10:59
Well, what's the length of
those column vectors,
161
00:10:59 --> 00:11:04
the dot product with themselves
is -- right now it's two,
162
00:11:04 --> 00:11:06
right, the -- the length
squared.
163
00:11:06 --> 00:11:11
The length squared would be one
plus one would be two,
164
00:11:11 --> 00:11:15
the length would be square root
of two,
165
00:11:15 --> 00:11:19
so I better divide by square
root of two.
166
00:11:19 --> 00:11:19
OK.
167
00:11:19 --> 00:11:23
So there's a -- there now I
have got an orthogonal matrix,
168
00:11:23 --> 00:11:27
in fact, it's this one -- when
theta is pi over four.
169
00:11:27 --> 00:11:32
The cosines and well almost,
I guess the minus sine is down
170
00:11:32 --> 00:11:34
there, so maybe,
I don't know,
171
00:11:34 --> 00:11:38.76
maybe minus pi over four or
something.
172
00:11:38.76 --> 00:11:39
OK.
173
00:11:39 --> 00:11:45
Let me do one final example,
just to show that you can get
174
00:11:45 --> 00:11:46
bigger ones.
175
00:11:46 --> 00:11:53
Q equals let me take that
matrix up in the corner and I'll
176
00:11:53 --> 00:11:58
sort of repeat that pattern,
repeat it again,
177
00:11:58 --> 00:12:01
and then minus it down here.
178
00:12:01 --> 00:12:09
That's one of the world's
favorite orthogonal matrices.
179
00:12:09 --> 00:12:12
I hope I got it right,
is -- can you see whether -- if
180
00:12:12 --> 00:12:16
I take the inner product of one
column with another one,
181
00:12:16 --> 00:12:21
let's see, if I take the inner
product of that column with that
182
00:12:21 --> 00:12:24
I have two minuses and two
pluses, that's good.
183
00:12:24 --> 00:12:29
When I take the inner product
of that with that I have a plus
184
00:12:29 --> 00:12:32
and a minus, a minus and a plus.
185
00:12:32 --> 00:12:32
Good.
186
00:12:32 --> 00:12:35
I think it all works out.
187
00:12:35 --> 00:12:39.03
And what do I have to divide by
now?
188
00:12:39.03 --> 00:12:42
To make those into unit
vectors.
189
00:12:42 --> 00:12:47
Right now the vector one,
one, one, one has length two.
190
00:12:47 --> 00:12:49
Square root of four.
191
00:12:49 --> 00:12:55
So I have to divide by two to
make it unit vector,
192
00:12:55 --> 00:12:58
so there's another.
193
00:12:58 --> 00:13:03
That's my entire array of
simple examples.
194
00:13:03 --> 00:13:10
This construction is named
after a guy called Adhemar and
195
00:13:10 --> 00:13:16
we know how to do it for two,
four, sixteen,
196
00:13:16 --> 00:13:22
sixty-four and so on,
but we -- nobody knows exactly
197
00:13:22 --> 00:13:30
which size matrices have --
which size -- which sizes allow
198
00:13:30 --> 00:13:34
orthogonal matrices of ones and
minus ones.
199
00:13:34 --> 00:13:38
So Adhemar matrix is an
orthogonal matrix that's got
200
00:13:38 --> 00:13:42
ones and minus ones,
and a lot of ones -- some we
201
00:13:42 --> 00:13:47
know, some other sizes,
there couldn't be a five by
202
00:13:47 --> 00:13:48
five I think.
203
00:13:48 --> 00:13:54
But there are some sizes
that nobody yet knows whether
204
00:13:54 --> 00:13:58
there could be or can't be a
matrix like that.
205
00:13:58 --> 00:13:59.13
OK.
206
00:13:59.13 --> 00:14:02
You see those orthogonal
matrices.
207
00:14:02 --> 00:14:07
Now let me ask what -- why is
it good to have orthogonal
208
00:14:07 --> 00:14:08
matrices?
209
00:14:08 --> 00:14:11
What calculation is made easy?
210
00:14:11 --> 00:14:14
If I have an orthogonal matrix.
211
00:14:14 --> 00:14:19
And -- let me remember that
the matrix could be
212
00:14:19 --> 00:14:21
rectangular.
213
00:14:21 --> 00:14:26
Shall I put down -- I better
put a rectangular example down.
214
00:14:26 --> 00:14:29
So the -- these were all square
examples.
215
00:14:29 --> 00:14:34.62
Can I put down just -- a
rectangular one just to be sure
216
00:14:34.62 --> 00:14:37
that we realize that this is
possible.
217
00:14:37 --> 00:14:39
let's help me out.
218
00:14:39 --> 00:14:43.55
Let's see,
if I put like a one,
219
00:14:43.55 --> 00:14:47
two, two and a minus two,
minus one, two.
220
00:14:47 --> 00:14:54
That's -- a matrix -- oh its
columns aren't normalized yet.
221
00:14:54 --> 00:14:57
I always have to remember to do
that.
222
00:14:57 --> 00:15:02
I always do that last because
it's easy to do.
223
00:15:02 --> 00:15:06
What's the length of those
columns?
224
00:15:06 --> 00:15:12
So if I wanted them --
if I wanted them to be length
225
00:15:12 --> 00:15:16
one, I should divide by their
length, which is -- so I'd look
226
00:15:16 --> 00:15:20
at one squared plus two squared
plus two squared,
227
00:15:20 --> 00:15:24
that's one and four and four is
nine, so I take the square root
228
00:15:24 --> 00:15:27
and I need to divide by three.
229
00:15:27 --> 00:15:27
OK.
230
00:15:27 --> 00:15:29.75
So there is -- well,
without that,
231
00:15:29.75 --> 00:15:33
I've got one orthonormal
vector.
232
00:15:33 --> 00:15:35
I mean just one unit vector.
233
00:15:35 --> 00:15:37
Now put that guy in.
234
00:15:37 --> 00:15:42
Now I have a basis for the
column space for a
235
00:15:42 --> 00:15:46
two-dimensional space,
an orthonormal basis,
236
00:15:46 --> 00:15:47
right?
237
00:15:47 --> 00:15:52.27
These two columns are
orthonormal, they would be an
238
00:15:52.27 --> 00:15:58
orthonormal basis for this
two-dimensional space that they
239
00:15:58 --> 00:15:59
span.
240
00:15:59 --> 00:16:04
Orthonormal vectors by the way
have got to be independent.
241
00:16:04 --> 00:16:09
It's easy to show that
orthonormal vectors since
242
00:16:09 --> 00:16:14
they're headed off all at ninety
degrees there's no combination
243
00:16:14 --> 00:16:16
that gives zero.
244
00:16:16 --> 00:16:21
Now if I wanted to create now a
third one, I could either just
245
00:16:21 --> 00:16:26
put in some third vector that
was independent
246
00:16:26 --> 00:16:32
and go to this Graham-Schmidt
calculation that I'm going to
247
00:16:32 --> 00:16:36.9
explain, or I could be inspired
and say look,
248
00:16:36.9 --> 00:16:42
that -- with that pattern,
why not put a one in there,
249
00:16:42 --> 00:16:45
and a two in there,
and a two in there,
250
00:16:45 --> 00:16:50
and try to fix up the signs so
that they worked.
251
00:16:50 --> 00:16:51
Hmm.
252
00:16:51 --> 00:16:56
I don't know if I've done this
too brilliantly.
253
00:16:56 --> 00:16:59
Let's see, what signs,
that's minus,
254
00:16:59 --> 00:17:04
maybe I'd make a minus sign
there, how would that be?
255
00:17:04 --> 00:17:06
Yeah, maybe that works.
256
00:17:06 --> 00:17:11
I think that those three
columns are orthonormal and they
257
00:17:11 --> 00:17:17
-- the beauty of this -- this is
the last example I'll probably
258
00:17:17 --> 00:17:22
find where there's no square
root, the -- the punishing thing
259
00:17:22 --> 00:17:26
in Graham-Schmidt,
maybe we better know that in
260
00:17:26 --> 00:17:30
advance, is that because I want
these vectors to be unit
261
00:17:30 --> 00:17:34
vectors, I'm always running into
square roots.
262
00:17:34 --> 00:17:37
I'm always dividing by lengths.
263
00:17:37 --> 00:17:40
And those lengths are square
roots.
264
00:17:40 --> 00:17:46
So you'll see as soon as I do a
Graham-Schmidt example,
265
00:17:46 --> 00:17:50
square roots are going to show
up.
266
00:17:50 --> 00:17:56
But here are some examples
where we did it without any
267
00:17:56 --> 00:17:57
square root.
268
00:17:57 --> 00:17:57
OK.
269
00:17:57 --> 00:17:58
OK.
270
00:17:58 --> 00:17:59
So -- so great.
271
00:17:59 --> 00:18:06
Now next question is what's the
good of having a Q?
272
00:18:06 --> 00:18:09
What formulas become easier?
273
00:18:09 --> 00:18:15.59
Suppose I want to project,
so suppose Q -- suppose Q has
274
00:18:15.59 --> 00:18:17
orthonormal columns.
275
00:18:17 --> 00:18:24
I'm using the letter Q to mean
this, I'll write it this one
276
00:18:24 --> 00:18:29
more time, but I always mean
when I write a Q,
277
00:18:29 --> 00:18:35
I always mean that it has
orthonormal columns.
278
00:18:35 --> 00:18:41
So suppose I want to project
onto its column space.
279
00:18:41 --> 00:18:45
So what's the projection
matrix?
280
00:18:45 --> 00:18:52
What's the projection matrix is
I project onto a column space?
281
00:18:52 --> 00:18:59
OK, that gives me a chance to
review the projection section,
282
00:18:59 --> 00:19:07
including that big formula,
which used to be -- those four
283
00:19:07 --> 00:19:13
As in a row, but now it's got
Qs, because I'm projecting onto
284
00:19:13 --> 00:19:18
the column space of Q,
so do you remember what it was?
285
00:19:18 --> 00:19:22
It's Q Q transpose Q inverse Q
transpose.
286
00:19:22 --> 00:19:24
That's my four Qs in a row.
287
00:19:24 --> 00:19:26
But what's good here?
288
00:19:26 --> 00:19:32
What --
what makes this formula nice if
289
00:19:32 --> 00:19:38
I'm projecting onto a column
space when I have orthonormal
290
00:19:38 --> 00:19:41
basis for that space?
291
00:19:41 --> 00:19:46
What makes it nice is this is
the identity.
292
00:19:46 --> 00:19:49
I don't have to do any
inversion.
293
00:19:49 --> 00:19:52
I just get Q Q transpose.
294
00:19:52 --> 00:19:57
So Q Q transpose is a
projection matrix.
295
00:19:57 --> 00:20:02
Oh, I can't help --
I can't resist just checking
296
00:20:02 --> 00:20:06
the properties,
what are the properties of a
297
00:20:06 --> 00:20:07
projection matrix?
298
00:20:07 --> 00:20:12
There are two properties to
know for any projection matrix.
299
00:20:12 --> 00:20:16
And I'm saying that this is the
right projection matrix when
300
00:20:16 --> 00:20:21
we've got this orthonormal basis
in the columns.
301
00:20:21 --> 00:20:21
OK.
302
00:20:21 --> 00:20:24
So there's the projection
matrix.
303
00:20:24 --> 00:20:27
Suppose the matrix is square.
304
00:20:27 --> 00:20:30
First just tell me first this
extreme case.
305
00:20:30 --> 00:20:36
If my matrix is square and it's
got these orthonormal columns,
306
00:20:36 --> 00:20:38
then what's the column space?
307
00:20:38 --> 00:20:43
If I have a square matrix and I
have independent columns,
308
00:20:43 --> 00:20:49
and even orthonormal columns,
then the column space is the
309
00:20:49 --> 00:20:51
whole space, right?
310
00:20:51 --> 00:20:56
And what's the projection
matrix onto the whole space?
311
00:20:56 --> 00:20:58
The identity matrix.
312
00:20:58 --> 00:21:04
If I'm projecting in the whole
space, every vector B is right
313
00:21:04 --> 00:21:09
where it's supposed to be and I
don't have to move it by
314
00:21:09 --> 00:21:11
projection.
315
00:21:11 --> 00:21:18
So this would be -- I'll put in
parentheses this is I if Q is
316
00:21:18 --> 00:21:19
square.
317
00:21:19 --> 00:21:22
Well that we said that already.
318
00:21:22 --> 00:21:27
If Q is square,
that's the case where Q
319
00:21:27 --> 00:21:33
transpose is Q inverse,
we can put it on the right,
320
00:21:33 --> 00:21:38
we can put it on the left,
we always get the identity
321
00:21:38 --> 00:21:40
matrix, if it's square.
322
00:21:40 --> 00:21:46
But if it's not a square matrix
then it's not -- we don't get
323
00:21:46 --> 00:21:48
the identity matrix.
324
00:21:48 --> 00:21:53
We have Q Q transpose,
and just again what are those
325
00:21:53 --> 00:21:57.41
two properties of a projection
matrix?
326
00:21:57.41 --> 00:21:59.63
First of all,
it's symmetric.
327
00:21:59.63 --> 00:22:03.04
OK, no problem,
that's certainly a symmetric
328
00:22:03.04 --> 00:22:03
matrix.
329
00:22:03 --> 00:22:07
So what's that second property
of a projection?
330
00:22:07 --> 00:22:12
That if you project and project
again you don't move the second
331
00:22:12 --> 00:22:12
time.
332
00:22:12 --> 00:22:17
So the other property of a
projection matrix should be that
333
00:22:17 --> 00:22:21
Q Q transpose twice should be
the same
334
00:22:21 --> 00:22:23
as Q Q transpose once.
335
00:22:23 --> 00:22:25
That's projection matrices.
336
00:22:25 --> 00:22:31
And that property better fall
out right away because from the
337
00:22:31 --> 00:22:36
fact we know about orthonormal
matrices, Q transpose Q is I.
338
00:22:36 --> 00:22:37
OK, you see it.
339
00:22:37 --> 00:22:42.52
In the middle here is sitting Q
Q t- Q transpose Q,
340
00:22:42.52 --> 00:22:46
sorry, that's what I meant to
say,
341
00:22:46 --> 00:22:48
Q transpose Q is I.
342
00:22:48 --> 00:22:53
So that's sitting right in the
middle, that cancels out,
343
00:22:53 --> 00:22:57.13
to give the identity,
we're left with one Q Q
344
00:22:57.13 --> 00:22:59
transpose, and we're all set.
345
00:22:59 --> 00:23:00
OK.
346
00:23:00 --> 00:23:04
So this is the projection
matrix -- all the equation --
347
00:23:04 --> 00:23:11
all the messy equations of this
chapter become trivial when our
348
00:23:11 --> 00:23:16
matrix -- when we have this
orthonormal basis.
349
00:23:16 --> 00:23:20
I mean what do I mean by all
the equations?
350
00:23:20 --> 00:23:24
Well, the most important
equation was the normal
351
00:23:24 --> 00:23:30
equation, do you remember old A
transpose A x hat equals A
352
00:23:30 --> 00:23:31
transpose b?
353
00:23:31 --> 00:23:33
But now -- now A is Q.
354
00:23:33 --> 00:23:41
Now I'm thinking I
have Q transpose Q X hat equals
355
00:23:41 --> 00:23:42
Q transpose b.
356
00:23:42 --> 00:23:46
And what's good about that?
357
00:23:46 --> 00:23:54
What's good is that matrix on
the left side is the identity.
358
00:23:54 --> 00:24:01
The matrix on the left is the
identity, Q transpose Q,
359
00:24:01 --> 00:24:07
normally it isn't,
normally it's that matrix of
360
00:24:07 --> 00:24:12
inner products and you've to
compute all those dopey inner
361
00:24:12 --> 00:24:15
products and -- and -- and solve
the system.
362
00:24:15 --> 00:24:19
Here the inner products are all
one or zero.
363
00:24:19 --> 00:24:21
This is the identity matrix.
364
00:24:21 --> 00:24:22
It's gone.
365
00:24:22 --> 00:24:24
And there's the answer.
366
00:24:24 --> 00:24:27
There's no inversion involved.
367
00:24:27 --> 00:24:29
Each component of x is a Q
times b.
368
00:24:29 --> 00:24:36
What that equation
is saying is that the i-th
369
00:24:36 --> 00:24:41
component is the i-th basis
vector times b.
370
00:24:41 --> 00:24:49
That's -- probably the most
important formula in some major
371
00:24:49 --> 00:24:55
parts of mathematics,
that if we have orthonormal
372
00:24:55 --> 00:25:02
basis, then the component
in the -- in the i-th,
373
00:25:02 --> 00:25:07
along the i-th -- the
projection on the i-th basis
374
00:25:07 --> 00:25:11
vector is just qi transpose b.
375
00:25:11 --> 00:25:16
That number x that we look for
is just a dot product.
376
00:25:16 --> 00:25:17
OK.
377
00:25:17 --> 00:25:23
OK, so I'm ready now for the
sort of like second half of the
378
00:25:23 --> 00:25:25
lecture.
379
00:25:25 --> 00:25:30
Where we don't start with an
orthogonal matrix,
380
00:25:30 --> 00:25:32
orthonormal vectors.
381
00:25:32 --> 00:25:39
We just start with independent
vectors and we want to make them
382
00:25:39 --> 00:25:40
orthonormal.
383
00:25:40 --> 00:25:44
So I'm going to -- can I do
that now?
384
00:25:44 --> 00:25:48
Now here comes Graham-Schmidt.
385
00:25:48 --> 00:25:50
So -- Graham-Schmidt.
386
00:25:50 --> 00:25:54.11
So this
is a calculation,
387
00:25:54.11 --> 00:26:00
I won't say -- I can't quite
say it's like elimination,
388
00:26:00 --> 00:26:05
because it's different,
our goal isn't triangular
389
00:26:05 --> 00:26:06
anymore.
390
00:26:06 --> 00:26:12
With elimination our goal was
make the matrix triangular.
391
00:26:12 --> 00:26:18
Now our goal is make the matrix
orthogonal.
392
00:26:18 --> 00:26:22.2
Make those columns orthonormal.
393
00:26:22.2 --> 00:26:25.92
So let me start with two
columns.
394
00:26:25.92 --> 00:26:29
So I start with vectors a and
b.
395
00:26:29 --> 00:26:34
And they're just like -- here,
let me draw them.
396
00:26:34 --> 00:26:36
Here's a.
397
00:26:36 --> 00:26:37
Here's b.
398
00:26:37 --> 00:26:38
For example.
399
00:26:38 --> 00:26:44
A isn't specially horizontal,
wasn't meant to be,
400
00:26:44 --> 00:26:48
just a is one
vector, b is another.
401
00:26:48 --> 00:26:53.7
I want to produce those two
vectors, they might be in
402
00:26:53.7 --> 00:26:57
twelve-dimensional space,
or they might be in
403
00:26:57 --> 00:26:59
two-dimensional space.
404
00:26:59 --> 00:27:02
They're independent,
anyway.
405
00:27:02 --> 00:27:04
So I better be sure I say that.
406
00:27:04 --> 00:27:07
I start with independent
vectors.
407
00:27:07 --> 00:27:12
And I want to produce out of
that q
408
00:27:12 --> 00:27:17.4
1 and q2, I want to produce
orthonormal vectors.
409
00:27:17.4 --> 00:27:21
And Graham and Schmidt tell me
how.
410
00:27:21 --> 00:27:21
OK.
411
00:27:21 --> 00:27:27
Well, actually you could tell
me how, we don't need --
412
00:27:27 --> 00:27:33
frankly, I don't know -- there's
only one idea here,
413
00:27:33 --> 00:27:39
if Graham had the idea,
I don't know what Schmidt did.
414
00:27:39 --> 00:27:41
But OK.
415
00:27:41 --> 00:27:43
So you'll see it.
416
00:27:43 --> 00:27:47
We don't need either of them,
actually.
417
00:27:47 --> 00:27:50
OK, so what I going to do.
418
00:27:50 --> 00:27:54
I'll take that -- this first
guy.
419
00:27:54 --> 00:27:54.8
OK.
420
00:27:54.8 --> 00:27:56.65
Well, he's fine.
421
00:27:56.65 --> 00:28:02
That direction is fine except
-- yeah, I'll say OK,
422
00:28:02 --> 00:28:06
I'll settle for that direction.
423
00:28:06 --> 00:28:10
So I'm going to --
I'm going to get,
424
00:28:10 --> 00:28:15
so what I going to -- my goal
is I'm going to get orthogonal
425
00:28:15 --> 00:28:19
vectors and I'll call those
capital A and B.
426
00:28:19 --> 00:28:24
So that's the key step is to
get from any two vectors to two
427
00:28:24 --> 00:28:26
orthogonal vectors.
428
00:28:26 --> 00:28:29
And then at the end,
no problem, I'll get
429
00:28:29 --> 00:28:33
orthonormal vectors,
how will --
430
00:28:33 --> 00:28:39
what will those will be my qs,
q1 and q2, and what will they
431
00:28:39 --> 00:28:39
be?
432
00:28:39 --> 00:28:43
Once I've got A and B
orthogonal, well,
433
00:28:43 --> 00:28:49
look, it's no big deal -- maybe
that's what Schmidt did,
434
00:28:49 --> 00:28:54
he, brilliant Schmidt,
thought OK, divide by the
435
00:28:54 --> 00:28:56
length, all right.
436
00:28:56 --> 00:28:59
That's Schmidt's contribution.
437
00:28:59 --> 00:28:59
OK.
438
00:28:59 --> 00:29:05
But Graham had a little more
thinking to do,
439
00:29:05 --> 00:29:06
right?
440
00:29:06 --> 00:29:09
We haven't done Graham's part.
441
00:29:09 --> 00:29:13
This part except OK,
I'm happy with A,
442
00:29:13 --> 00:29:14.65
A can be A.
443
00:29:14.65 --> 00:29:17
That first direction is fine.
444
00:29:17 --> 00:29:21
Why should -- no complaint
about that.
445
00:29:21 --> 00:29:27
The trouble is the second
direction is not fine.
446
00:29:27 --> 00:29:31
Because it's not orthogonal to
the first.
447
00:29:31 --> 00:29:38
I'm looking for a vector
that's -- starts with B,
448
00:29:38 --> 00:29:41
but makes it orthogonal to A.
449
00:29:41 --> 00:29:43.32
What's the vector?
450
00:29:43.32 --> 00:29:45
How do I do that?
451
00:29:45 --> 00:29:51
How do I produce from this
vector a piece that's orthogonal
452
00:29:51 --> 00:29:52
to this one?
453
00:29:52 --> 00:29:58
And the -- remember these
vectors might be in two
454
00:29:58 --> 00:30:03
dimensions or they might be in
twelve
455
00:30:03 --> 00:30:04
dimensions.
456
00:30:04 --> 00:30:06
I'm just looking for the idea.
457
00:30:06 --> 00:30:07
So what's the idea?
458
00:30:07 --> 00:30:12
Where did we have orthogonal --
a vector showing up that was
459
00:30:12 --> 00:30:14
orthogonal to this guy?
460
00:30:14 --> 00:30:18
Well, that was the first basic
calculation of the whole
461
00:30:18 --> 00:30:19
chapter.
462
00:30:19 --> 00:30:23
We -- we did a projection and
the projection gave us this
463
00:30:23 --> 00:30:27
part,
which was the part in the A
464
00:30:27 --> 00:30:28
direction.
465
00:30:28 --> 00:30:33
Now, the part we want is the
other part, the e part.
466
00:30:33 --> 00:30:34
This part.
467
00:30:34 --> 00:30:39
This is going to be our -- that
guy is that guy.
468
00:30:39 --> 00:30:41
This is our vector B.
469
00:30:41 --> 00:30:46
That gives us that
ninety-degree angle.
470
00:30:46 --> 00:30:53
So B is you could say -- B is
really what we previously called
471
00:30:53 --> 00:30:53
e.
472
00:30:53 --> 00:30:55
The error vector.
473
00:30:55 --> 00:30:57
And what is it?
474
00:30:57 --> 00:31:01
I mean what do I -- what is B
here?
475
00:31:01 --> 00:31:03
A is A, no problem.
476
00:31:03 --> 00:31:09
B is -- OK, what's this error
piece?
477
00:31:09 --> 00:31:10.72
Do you remember?
478
00:31:10.72 --> 00:31:16
It's I start with the original
B and I take away what?
479
00:31:16 --> 00:31:18
Its projection,
this P.
480
00:31:18 --> 00:31:22
This -- the vector B,
this error vector,
481
00:31:22 --> 00:31:27
is the original vector removing
the projection.
482
00:31:27 --> 00:31:31
So instead of wanting the
projection,
483
00:31:31 --> 00:31:34
now that's what I want to throw
away.
484
00:31:34 --> 00:31:38
I want to get the part that's
perpendicular.
485
00:31:38 --> 00:31:41
And there will be a
perpendicular part,
486
00:31:41 --> 00:31:42.82
it won't be zero.
487
00:31:42.82 --> 00:31:47
Because these vectors were
independent, so B -- if B was
488
00:31:47 --> 00:31:52
along the direction of A,
then if the original B and A
489
00:31:52 --> 00:31:56
were in the same direction,
then I'm -- I've only got one
490
00:31:56 --> 00:31:57.08
direction.
491
00:31:57.08 --> 00:32:00
But here they're in two
independent directions and all
492
00:32:00 --> 00:32:03
I'm doing is getting that guy.
493
00:32:03 --> 00:32:04
So what's its formula?
494
00:32:04 --> 00:32:09
What's the formula for that if
-- I want to subtract
495
00:32:09 --> 00:32:14
the projection,
so do you remember the
496
00:32:14 --> 00:32:15
projection?
497
00:32:15 --> 00:32:21
It's some multiple of A and
what's that multiple?
498
00:32:21 --> 00:32:29
It's -- it's that thing we
called x in the very very first
499
00:32:29 --> 00:32:32
lecture on this chapter.
500
00:32:32 --> 00:32:39
There's an A transpose A in the
bottom and there's an A
501
00:32:39 --> 00:32:44
transpose B, isn't that it?
502
00:32:44 --> 00:32:46
I think that's Graham's
formula.
503
00:32:46 --> 00:32:47
Or Graham-Schmidt.
504
00:32:47 --> 00:32:49
No, that's Graham.
505
00:32:49 --> 00:32:53
Schmidt has got to divide the
whole thing by the length,
506
00:32:53 --> 00:32:57
so he -- his formula makes a
mess which I'm not willing to
507
00:32:57 --> 00:32:58
write down.
508
00:32:58 --> 00:33:01
So let's just see that what I
saying here?
509
00:33:01 --> 00:33:06
I'm saying that this vector is
perpendicular to A.
510
00:33:06 --> 00:33:08
That these are orthogonal.
511
00:33:08 --> 00:33:10
A is perpendicular to B.
512
00:33:10 --> 00:33:11
Can you check that?
513
00:33:11 --> 00:33:16
How do you see that yes,
of course, we -- our picture is
514
00:33:16 --> 00:33:19
telling us, yes,
we did it right.
515
00:33:19 --> 00:33:23
How would I check that this
matrix is perpendicular to A?
516
00:33:23 --> 00:33:29
I would multiply by A transpose
and I better get zero,
517
00:33:29 --> 00:33:30
right?
518
00:33:30 --> 00:33:32
I should check that.
519
00:33:32 --> 00:33:35
A transpose B should come out
zero.
520
00:33:35 --> 00:33:41.71
So this is A transpose times --
now what did we say B was?
521
00:33:41.71 --> 00:33:46
We start with the original B,
and we take away this
522
00:33:46 --> 00:33:51
projection, and that should come
out zero.
523
00:33:51 --> 00:33:56
Well, here we get an A
transpose B minus -- and here is
524
00:33:56 --> 00:34:03
another A transpose B,
and the -- and it's an A
525
00:34:03 --> 00:34:09
transpose A over A transpose A,
a one, those cancel,
526
00:34:09 --> 00:34:12
and we do get zero.
527
00:34:12 --> 00:34:13
Right.
528
00:34:13 --> 00:34:17
Now I guess I can do numbers in
there.
529
00:34:17 --> 00:34:24
But I have to take a third
vector to be sure we've got this
530
00:34:24 --> 00:34:26.44
system down.
531
00:34:26.44 --> 00:34:34
So now I have to say if I have
independent vectors A,
532
00:34:34 --> 00:34:39
B and C, I'm looking for
orthogonal vectors A,
533
00:34:39 --> 00:34:44
B and capital C,
and then of course the third
534
00:34:44 --> 00:34:50
guy will just be C over its
length, the unit vector.
535
00:34:50 --> 00:34:54
So this is now the problem.
536
00:34:54 --> 00:34:55
I got B here.
537
00:34:55 --> 00:34:58
I got A very easily.
538
00:34:58 --> 00:35:05
And now -- if you see the idea,
we could figure out a formula
539
00:35:05 --> 00:35:07
for C.
540
00:35:07 --> 00:35:13
So now that -- so this is like
a typical homework quiz problem.
541
00:35:13 --> 00:35:16.81
I give you two vectors,
you do this,
542
00:35:16.81 --> 00:35:21
I give you three vectors,
and you have to make them
543
00:35:21 --> 00:35:22
orthonormal.
544
00:35:22 --> 00:35:27
So you do this again,
the first vector's fine,
545
00:35:27 --> 00:35:31
the second vector is
perpendicular
546
00:35:31 --> 00:35:35
to the first,
and now I need a third vector
547
00:35:35 --> 00:35:39
that's perpendicular to the
first one and the second one.
548
00:35:39 --> 00:35:40
Right?
549
00:35:40 --> 00:35:44
Tthis is the end of a -- the
lecture is to find this guy.
550
00:35:44 --> 00:35:49
Find this vector -- this vector
C, that's perpendicular we n- at
551
00:35:49 --> 00:35:52
this point we know A and B.
552
00:35:52 --> 00:35:58
But C, the little c that we're
given, is off in some -- it's
553
00:35:58 --> 00:36:02.82
got to come out of the
blackboard to be independent,
554
00:36:02.82 --> 00:36:08
so -- so can I sort of draw off
-- off comes a c somewhere.
555
00:36:08 --> 00:36:12
I don't know,
where I going to put the darn
556
00:36:12 --> 00:36:12
thing?
557
00:36:12 --> 00:36:17
Maybe I'll put it off,
oh, I don't know,
558
00:36:17 --> 00:36:21
like that somehow,
C, little c.
559
00:36:21 --> 00:36:26
And I already know that
perpendicular direction,
560
00:36:26 --> 00:36:29.17
that one and that one.
561
00:36:29.17 --> 00:36:31
So now what's the idea?
562
00:36:31 --> 00:36:36
Give me the Graham-Schmidt
formula for C.
563
00:36:36 --> 00:36:38
What is this C here?
564
00:36:38 --> 00:36:40
Equals what?
565
00:36:40 --> 00:36:43
What I going to do?
566
00:36:43 --> 00:36:46
I'll start with the given one.
567
00:36:46 --> 00:36:47
As before.
568
00:36:47 --> 00:36:47
Right?
569
00:36:47 --> 00:36:50
I start with the vector I'm
given.
570
00:36:50 --> 00:36:52
And what do I do with it?
571
00:36:52 --> 00:36:56
I want to remove out of it,
I want to subtract off,
572
00:36:56 --> 00:37:01
so I'll put a minus sign in,
I want to subtract off its
573
00:37:01 --> 00:37:04
components in the A,
capital A and capital B
574
00:37:04 --> 00:37:06
directions.
575
00:37:06 --> 00:37:11
I just want to get those out of
there.
576
00:37:11 --> 00:37:14
Well, I know how to do that.
577
00:37:14 --> 00:37:16
I did it with B.
578
00:37:16 --> 00:37:23
So I'll just -- so let me take
away -- what if I do this?
579
00:37:23 --> 00:37:24
What have I done?
580
00:37:24 --> 00:37:31
I've got little c and what have
I subtracted from it?
581
00:37:31 --> 00:37:36
Its component,
its projection if
582
00:37:36 --> 00:37:39
you like, in the A direction.
583
00:37:39 --> 00:37:46
And now I've got to subtract
off its component B transpose C
584
00:37:46 --> 00:37:50
over B transpose B,
that multiple of B,
585
00:37:50 --> 00:37:55
is its component in the B
direction.
586
00:37:55 --> 00:38:02
And that gives me the vector
capital C that if anything is --
587
00:38:02 --> 00:38:09.56
if there's any justice,
this C should be perpendicular
588
00:38:09.56 --> 00:38:14
to A and it should be
perpendicular to B.
589
00:38:14 --> 00:38:19
And the only thing it's --
hasn't got is unit vector,
590
00:38:19 --> 00:38:24.68
so we divide by its length to
get that too.
591
00:38:24.68 --> 00:38:25
OK.
592
00:38:25 --> 00:38:27
Let me do an example.
593
00:38:27 --> 00:38:33
Can I -- I'll make my life
easy, I'll just take two
594
00:38:33 --> 00:38:35
vectors.
595
00:38:35 --> 00:38:38
So let me do a numerical
example.
596
00:38:38 --> 00:38:43
If I'll give you two vectors,
you give me back the
597
00:38:43 --> 00:38:48
Graham-Schmidt orthonormal
basis, and we'll see how to
598
00:38:48 --> 00:38:51
express that in matrix form.
599
00:38:51 --> 00:38:51
OK.
600
00:38:51 --> 00:38:55
So let me give you the two
vectors.
601
00:38:55 --> 00:39:00
So I'll take the vector A
equals let's say
602
00:39:00 --> 00:39:03
one, one, one,
why not?
603
00:39:03 --> 00:39:08
And B equals let's say one,
zero, two, OK?
604
00:39:08 --> 00:39:15
I didn't want to cheat and make
them orthogonal in the first
605
00:39:15 --> 00:39:20
place because then
Graham-Schmidt wouldn't be
606
00:39:20 --> 00:39:21
needed.
607
00:39:21 --> 00:39:22
OK.
608
00:39:22 --> 00:39:25
So those are not orthogonal.
609
00:39:25 --> 00:39:29
So what is capital A?
610
00:39:29 --> 00:39:32
Well that's the same as big A.
611
00:39:32 --> 00:39:33
That was fine.
612
00:39:33 --> 00:39:34
What's B?
613
00:39:34 --> 00:39:40
So B is this b -- is the
original B, and then I subtract
614
00:39:40 --> 00:39:42
off some multiple of the A.
615
00:39:42 --> 00:39:44.99
And what's the multiple?
616
00:39:44.99 --> 00:39:46
What goes in here?
617
00:39:46 --> 00:39:52
B -- here's the A -- this is
the -- this is the little b,
618
00:39:52 --> 00:39:56
this is the big A,
also the little a,
619
00:39:56 --> 00:40:03
and I want to multiply it by
that right -- that right ratio,
620
00:40:03 --> 00:40:07
which has A transpose A,
here's my ratio.
621
00:40:07 --> 00:40:09
I'm just doing this.
622
00:40:09 --> 00:40:14
So it's A transpose B,
what is A transpose B,
623
00:40:14 --> 00:40:16.97
it looks like three.
624
00:40:16.97 --> 00:40:23
And what is A -- oh,
my -- what's A transpose A?
625
00:40:23 --> 00:40:23.8
Three.
626
00:40:23.8 --> 00:40:24
I'm sorry.
627
00:40:24 --> 00:40:28
I didn't know that was going to
happen.
628
00:40:28 --> 00:40:28
OK.
629
00:40:28 --> 00:40:30
But it happened.
630
00:40:30 --> 00:40:32
Why should we knock it?
631
00:40:32 --> 00:40:32
OK.
632
00:40:32 --> 00:40:35
So do you see it all right?
633
00:40:35 --> 00:40:39
That's A transpose B,
there's A transpose A,
634
00:40:39 --> 00:40:44
that's the fraction,
so I take this away,
635
00:40:44 --> 00:40:50
and I get one take away one is
a zero, zero minus this one is a
636
00:40:50 --> 00:40:53
minus one, and two minus the one
is a one.
637
00:40:53 --> 00:40:54
OK.
638
00:40:54 --> 00:40:58
And what's this vector that we
finally found?
639
00:40:58 --> 00:40:59
This is B.
640
00:40:59 --> 00:41:01
And how do I know it's right?
641
00:41:01 --> 00:41:05.03
How do I know I've got a vector
I want?
642
00:41:05.03 --> 00:41:09
I check that B is perpendicular
to -- that A and B are
643
00:41:09 --> 00:41:12
perpendicular.
644
00:41:12 --> 00:41:15
That A is perpendicular to B.
645
00:41:15 --> 00:41:17
Just look at that.
646
00:41:17 --> 00:41:22
That one -- the dot product of
that with that is zero.
647
00:41:22 --> 00:41:22
OK.
648
00:41:22 --> 00:41:25
So now what is my q1 and q2?
649
00:41:25 --> 00:41:28
Why don't I put them in a
matrix?
650
00:41:28 --> 00:41:29
Of course.
651
00:41:29 --> 00:41:36
Since I'm always putting these
-- so the Q, I'll put the q1 and
652
00:41:36 --> 00:41:39.14
the
q2 in a matrix.
653
00:41:39.14 --> 00:41:41
And what are they?
654
00:41:41 --> 00:41:46
Now when I'm writing q-s I'm
supposed to make things
655
00:41:46 --> 00:41:47
normalized.
656
00:41:47 --> 00:41:51
I'm supposed to make things
unit vectors.
657
00:41:51 --> 00:41:58
So I'm going to take that A but
I'm going to divide it by square
658
00:41:58 --> 00:41:59
root of three.
659
00:41:59 --> 00:42:04
And I'm going to take this B
but I'm
660
00:42:04 --> 00:42:10
going to divide it by square
root of two to make it a unit
661
00:42:10 --> 00:42:13
vector, and there is my matrix.
662
00:42:13 --> 00:42:18.93
That's my matrix with
orthonormal columns coming from
663
00:42:18.93 --> 00:42:24
Graham-Schmidt and it sort of it
-- it came from the original
664
00:42:24 --> 00:42:27
one, one, one,
one, zero, two,
665
00:42:27 --> 00:42:28
right?
666
00:42:28 --> 00:42:31
That was my original guys.
667
00:42:31 --> 00:42:35.71
These were the two I started
with.
668
00:42:35.71 --> 00:42:39
These are the two that I'm
happy to end with.
669
00:42:39 --> 00:42:42
Because those are orthonormal.
670
00:42:42 --> 00:42:45
So that's what Graham-Schmidt
did.
671
00:42:45 --> 00:42:51
It -- well, tell me about the
column spaces of these matrices.
672
00:42:51 --> 00:42:57
How is the column space of Q
related to the column space of
673
00:42:57 --> 00:42:57
A?
674
00:42:57 --> 00:43:03
So I'm always asking you things
like this, and that makes you
675
00:43:03 --> 00:43:07
think,
OK, the column space is all
676
00:43:07 --> 00:43:10
combinations of the columns,
it's that plane,
677
00:43:10 --> 00:43:10
right?
678
00:43:10 --> 00:43:14.74
I've got two vectors in
three-dimensional space,
679
00:43:14.74 --> 00:43:19
their column space is a plane,
the column space of this matrix
680
00:43:19 --> 00:43:23
is a plane, what's the relation
between the planes?
681
00:43:23 --> 00:43:25.97
Between the two column spaces?
682
00:43:25.97 --> 00:43:29
They're one and the same,
right?
683
00:43:29 --> 00:43:32
It's the same column space.
684
00:43:32 --> 00:43:37
All I'm taking is here this B
thing that I computed,
685
00:43:37 --> 00:43:43.26
this B thing that I computed is
a combination of B and A,
686
00:43:43.26 --> 00:43:48
and A was little A,
so I'm always working here with
687
00:43:48 --> 00:43:50.77
this in the same space.
688
00:43:50.77 --> 00:43:56
I'm just like getting
ninety-degree angles in there.
689
00:43:56 --> 00:44:02.53
Where my original column space
had a perfectly good basis,
690
00:44:02.53 --> 00:44:07
but it wasn't as good as this
basis, because it wasn't
691
00:44:07 --> 00:44:08
orthonormal.
692
00:44:08 --> 00:44:13
Now this one is orthonormal,
and I have a basis then that --
693
00:44:13 --> 00:44:17.07
so now projections,
all the calculations I would
694
00:44:17.07 --> 00:44:21
ever want to do are -- are a
cinch with this orthonormal
695
00:44:21 --> 00:44:22
basis.
696
00:44:22 --> 00:44:23
One final point.
697
00:44:23 --> 00:44:27.77
One final point
in this chapter.
698
00:44:27.77 --> 00:44:31
And it's -- just like
elimination.
699
00:44:31 --> 00:44:36
We learned how to do
elimination, we know all the
700
00:44:36 --> 00:44:38.96
steps, we can do it.
701
00:44:38.96 --> 00:44:45
But then I came back to it and
said look at it as a matrix in
702
00:44:45 --> 00:44:52
matrix language and elimination
gave me -- what was elimination
703
00:44:52 --> 00:44:55
in
matrix language?
704
00:44:55 --> 00:44:57
I'll just put it up there.
705
00:44:57 --> 00:44:58
A was LU.
706
00:44:58 --> 00:45:01
That was matrix,
that was elimination.
707
00:45:01 --> 00:45:05
Now, I want to do the same for
Graham-Schmidt.
708
00:45:05 --> 00:45:11.13
Everybody who works in linear
algebra isn't going to write out
709
00:45:11.13 --> 00:45:14
the columns are orthogonal,
or orthonormal.
710
00:45:14 --> 00:45:19
And isn't going to
write out these formulas.
711
00:45:19 --> 00:45:24.88
They're going to write out the
connection between the matrix A
712
00:45:24.88 --> 00:45:26
and the matrix Q.
713
00:45:26 --> 00:45:30
And the two matrices have the
same column space,
714
00:45:30 --> 00:45:35.58
but there's some -- some matrix
is taking the -- and I'm going
715
00:45:35.58 --> 00:45:39.86
to call it R,
so A equals QR is the magic
716
00:45:39.86 --> 00:45:41
formula here.
717
00:45:41 --> 00:45:46
It's the expression of
Graham-Schmidt.
718
00:45:46 --> 00:45:51
And I'll -- let me just capture
that.
719
00:45:51 --> 00:45:57
So that's the -- my final step
then is A equal QR.
720
00:45:57 --> 00:46:02.09
Maybe I can squeeze it in here.
721
00:46:02.09 --> 00:46:08
So A has columns,
let's say a1 and a2.
722
00:46:08 --> 00:46:12
Let me suppose n is two,
just two vectors.
723
00:46:12 --> 00:46:13
OK.
724
00:46:13 --> 00:46:17
So that's some combination of
q1 and q2.
725
00:46:17 --> 00:46:19
And times some matrix R.
726
00:46:19 --> 00:46:23.14
They have the same column
space.
727
00:46:23.14 --> 00:46:29
This is just -- this matrix
just includes in it whatever
728
00:46:29 --> 00:46:35.33
these numbers like three over
three and one over square root
729
00:46:35.33 --> 00:46:40
of three and
one over square root of two,
730
00:46:40 --> 00:46:43
probably that's what it's got.
731
00:46:43 --> 00:46:49
One over square root of three,
one over square root of two,
732
00:46:49 --> 00:46:53
something there,
but actually it's got a zero
733
00:46:53 --> 00:46:54
there.
734
00:46:54 --> 00:47:00
So the main point about this A
equal QR is this R turns out to
735
00:47:00 --> 00:47:03
be upper
triangular.
736
00:47:03 --> 00:47:08
It turns out that this zero is
upper triangular.
737
00:47:08 --> 00:47:10
We could see why.
738
00:47:10 --> 00:47:16.05
Let me see, I can put in
general formulas for what these
739
00:47:16.05 --> 00:47:16
are.
740
00:47:16 --> 00:47:23
This I think in here should be
the inner product of a1 with q1.
741
00:47:23 --> 00:47:30
And this one should be the --
the inner product of a1 with
742
00:47:30 --> 00:47:31
q2.
743
00:47:31 --> 00:47:35
And that's what I believe is
zero.
744
00:47:35 --> 00:47:42
This will be something here,
and this will be something here
745
00:47:42 --> 00:47:49
with inner -- a1 transpose q2,
sorry a2 transpose q1 and a2
746
00:47:49 --> 00:47:50
transpose q2.
747
00:47:50 --> 00:47:53
But why is that guy zero?
748
00:47:53 --> 00:47:57
Why is a1 q2 zero?
749
00:47:57 --> 00:48:03.39
That's the key to this being --
this R here being upper
750
00:48:03.39 --> 00:48:04
triangular.
751
00:48:04 --> 00:48:10
You know why a1q2 is zero,
because a1 -- that was my --
752
00:48:10 --> 00:48:13
this was really a and b here.
753
00:48:13 --> 00:48:16
This was really a and b.
754
00:48:16 --> 00:48:19
So this is a transpose q2.
755
00:48:19 --> 00:48:25
And the whole point of
Graham-Schmidt was that we
756
00:48:25 --> 00:48:31
constructed these later q-s to
be perpendicular to the earlier
757
00:48:31 --> 00:48:36
vectors, to the earlier -- all
the earlier vectors.
758
00:48:36 --> 00:48:40.21
So that's why we get a
triangular matrix.
759
00:48:40.21 --> 00:48:44
The -- result is extremely
satisfactory.
760
00:48:44 --> 00:48:49
That if I have a matrix
with independent columns,
761
00:48:49 --> 00:48:55
the Graham-Schmidt produces a
matrix with orthonormal columns,
762
00:48:55 --> 00:49:01
and the connection between
those is a triangular matrix.
763
00:49:01 --> 00:49:05
That last point,
that the connection is a
764
00:49:05 --> 00:49:09
triangular matrix,
please look in the book,
765
00:49:09 --> 00:49:12
you have to see that one more
time.
766
00:49:12 --> 00:49:13
OK.
767
00:49:13 --> 00:49:16
Thanks, that's great.