WEBVTT
00:00:07.270 --> 00:00:12.540
OK, here's the last lecture in
the chapter on orthogonality.
00:00:12.540 --> 00:00:16.900
So we met orthogonal
vectors, two vectors,
00:00:16.900 --> 00:00:22.530
we met orthogonal subspaces,
like the row space and null
00:00:22.530 --> 00:00:23.590
space.
00:00:23.590 --> 00:00:28.720
Now today we meet
an orthogonal basis,
00:00:28.720 --> 00:00:31.370
and an orthogonal matrix.
00:00:31.370 --> 00:00:32.920
So we really --
00:00:32.920 --> 00:00:36.170
this chapter cleans
up orthogonality.
00:00:36.170 --> 00:00:38.930
And really I want --
00:00:38.930 --> 00:00:43.880
I should use the
word orthonormal.
00:00:43.880 --> 00:00:51.690
Orthogonal is -- so my
vectors are q1,q2 up to qn --
00:00:51.690 --> 00:00:56.180
I use the letter "q",
here, to remind me,
00:00:56.180 --> 00:01:01.270
I'm talking about orthogonal
things, not just any vectors,
00:01:01.270 --> 00:01:02.690
but orthogonal ones.
00:01:02.690 --> 00:01:04.230
So what does that mean?
00:01:04.230 --> 00:01:08.600
That means that every q is
orthogonal to every other q.
00:01:08.600 --> 00:01:13.880
It's a natural idea,
to have a basis that's
00:01:13.880 --> 00:01:17.060
headed off at
ninety-degree angles,
00:01:17.060 --> 00:01:19.120
the inner products are all zero.
00:01:19.120 --> 00:01:26.260
Of course if q is -- certainly
qi is not orthogonal to itself.
00:01:26.260 --> 00:01:29.200
But there we'll make
the best choice again,
00:01:29.200 --> 00:01:31.050
make it a unit vector.
00:01:31.050 --> 00:01:35.610
Then qi transpose qi is
one, for a unit vector.
00:01:35.610 --> 00:01:37.590
The length squared is one.
00:01:37.590 --> 00:01:41.430
And that's what I would
use the word normal.
00:01:41.430 --> 00:01:48.479
So for this part, normalized,
unit length for this part.
00:01:48.479 --> 00:01:48.979
OK.
00:01:48.979 --> 00:01:55.120
So first part of
the lecture is how
00:01:55.120 --> 00:01:59.320
does having an orthonormal
basis make things nice?
00:01:59.320 --> 00:02:00.330
It certainly does.
00:02:00.330 --> 00:02:02.490
It makes all the
calculations better,
00:02:02.490 --> 00:02:05.150
a whole lot of
numerical linear algebra
00:02:05.150 --> 00:02:10.520
is built around working
with orthonormal vectors,
00:02:10.520 --> 00:02:12.230
because they never
get out of hand,
00:02:12.230 --> 00:02:16.220
they never overflow
or underflow.
00:02:16.220 --> 00:02:20.390
And I'll put them
into a matrix Q,
00:02:20.390 --> 00:02:22.430
and then the second
part of the lecture
00:02:22.430 --> 00:02:26.630
will be suppose my
basis, my columns of A
00:02:26.630 --> 00:02:28.710
are not orthonormal.
00:02:28.710 --> 00:02:30.660
How do I make them so?
00:02:30.660 --> 00:02:35.110
And the two names associated
with that simple idea
00:02:35.110 --> 00:02:36.840
are Graham and Schmidt.
00:02:36.840 --> 00:02:43.410
So the first part is we've
got a basis like this.
00:02:43.410 --> 00:02:46.040
Let's put those into
the columns of a matrix.
00:02:48.620 --> 00:02:53.990
So a matrix Q that has --
00:02:53.990 --> 00:02:56.680
I'll put these
orthonormal vectors,
00:02:56.680 --> 00:03:01.650
q1 will be the first column,
qn will be the n-th column.
00:03:04.350 --> 00:03:09.610
And I want to say, I want
to write this property,
00:03:09.610 --> 00:03:13.890
qi transpose qj
being zero, I want
00:03:13.890 --> 00:03:17.640
to put that in a matrix form.
00:03:17.640 --> 00:03:23.390
And just the right thing is
to look at Q transpose Q.
00:03:23.390 --> 00:03:27.520
So this chapter has been
looking at A transpose A.
00:03:27.520 --> 00:03:30.470
So it's natural to
look at Q transpose Q.
00:03:30.470 --> 00:03:33.490
And the beauty is it
comes out perfectly.
00:03:33.490 --> 00:03:38.160
Because Q transpose has
these vectors in its rows,
00:03:38.160 --> 00:03:45.510
the first row is q1 transpose,
the nth row is qn transpose.
00:03:45.510 --> 00:03:49.070
So that's Q transpose.
00:03:49.070 --> 00:03:51.610
And now I want to multiply by Q.
00:03:51.610 --> 00:03:56.810
That has q1 along to
qn in the columns.
00:03:56.810 --> 00:03:58.430
That's Q.
00:03:58.430 --> 00:03:59.410
And what do I get?
00:04:01.940 --> 00:04:06.240
You really -- this is the
first simplest most basic fact,
00:04:06.240 --> 00:04:10.930
that how do orthonormal
vectors, orthonormal columns
00:04:10.930 --> 00:04:17.660
in a matrix, what happens
if I compute Q transpose Q?
00:04:17.660 --> 00:04:18.850
Do you see it?
00:04:18.850 --> 00:04:23.060
If I take the first row
times the first column,
00:04:23.060 --> 00:04:25.170
what do I get?
00:04:25.170 --> 00:04:26.680
A one.
00:04:26.680 --> 00:04:29.570
If I take the first row
times the second column,
00:04:29.570 --> 00:04:31.330
what do I get?
00:04:31.330 --> 00:04:32.280
Zero.
00:04:32.280 --> 00:04:34.170
That's the orthogonality.
00:04:34.170 --> 00:04:37.460
The first row times the
last column is zero.
00:04:37.460 --> 00:04:40.630
And so I'm getting
ones on the diagonal
00:04:40.630 --> 00:04:43.620
and I'm getting zeroes
everywhere else.
00:04:43.620 --> 00:04:45.080
I'm getting the identity matrix.
00:04:47.880 --> 00:04:50.920
You see how that's -- it's
just like the right calculation
00:04:50.920 --> 00:04:51.740
to do.
00:04:51.740 --> 00:04:55.540
If you have orthonormal
columns, and the matrix
00:04:55.540 --> 00:04:59.420
doesn't have to be square here.
00:04:59.420 --> 00:05:02.290
We might have just two columns.
00:05:02.290 --> 00:05:05.670
And they might have
four, lots of components.
00:05:05.670 --> 00:05:13.370
So but they're orthonormal, and
when we do Q transpose times Q,
00:05:13.370 --> 00:05:17.110
that Q transpose times
Q or A transpose A
00:05:17.110 --> 00:05:22.680
just asks for all
those dot products.
00:05:22.680 --> 00:05:24.530
Rows times columns.
00:05:24.530 --> 00:05:29.390
And in this orthonormal case,
we get the best possible answer,
00:05:29.390 --> 00:05:30.850
the identity.
00:05:30.850 --> 00:05:33.330
OK, so this is --
00:05:33.330 --> 00:05:39.690
so I mean now we have a new
bunch of important matrices.
00:05:39.690 --> 00:05:41.070
What have we seen previously?
00:05:41.070 --> 00:05:43.440
We've seen in the
distant past we
00:05:43.440 --> 00:05:48.180
had triangular matrices,
diagonal matrices, permutation
00:05:48.180 --> 00:05:51.760
matrices, that was
early chapters,
00:05:51.760 --> 00:05:59.190
then we had row echelon
forms, then in this chapter
00:05:59.190 --> 00:06:02.900
we've already seen
projection matrices,
00:06:02.900 --> 00:06:08.830
and now we're seeing this
new class of matrices
00:06:08.830 --> 00:06:11.490
with orthonormal columns.
00:06:11.490 --> 00:06:13.510
That's a very long expression.
00:06:13.510 --> 00:06:19.020
I sorry that I can't just
call them orthogonal matrices.
00:06:19.020 --> 00:06:22.090
But that word
orthogonal matrices --
00:06:22.090 --> 00:06:25.440
or maybe I should be able to
call it orthonormal matrices,
00:06:25.440 --> 00:06:28.090
why don't we call
it orthonormal --
00:06:28.090 --> 00:06:31.290
I mean that would be an
absolutely perfect name.
00:06:31.290 --> 00:06:33.860
For Q, call it an
orthonormal matrix
00:06:33.860 --> 00:06:36.370
because its columns
are orthonormal.
00:06:36.370 --> 00:06:42.270
OK, but the convention is
that we only use that name
00:06:42.270 --> 00:06:46.930
orthogonal matrix,
we only use this --
00:06:46.930 --> 00:06:49.670
this word orthogonal,
we don't even
00:06:49.670 --> 00:06:51.950
say orthonormal for
some unknown reason,
00:06:51.950 --> 00:06:54.530
matrix when it's square.
00:06:59.130 --> 00:07:04.740
So in the case when this is a
square matrix, that's the case
00:07:04.740 --> 00:07:07.830
we call it an orthogonal matrix.
00:07:07.830 --> 00:07:12.290
And what's special about
the case when it's square?
00:07:12.290 --> 00:07:19.390
When it's a square matrix,
we've got its inverse, so --
00:07:19.390 --> 00:07:33.850
so in the case if Q is square,
then Q transpose Q equals I
00:07:33.850 --> 00:07:35.730
tells us --
00:07:35.730 --> 00:07:37.820
let me write that underneath --
00:07:37.820 --> 00:07:47.670
tells us that Q
transpose is Q inverse.
00:07:47.670 --> 00:07:51.000
There we have the
easy to remember
00:07:51.000 --> 00:07:57.440
property for a square matrix
with orthonormal columns.
00:07:57.440 --> 00:08:01.960
That -- I need to write
some examples down.
00:08:01.960 --> 00:08:04.180
Let's see.
00:08:04.180 --> 00:08:07.750
Some examples like if I
take any -- so examples,
00:08:07.750 --> 00:08:08.720
let's do some examples.
00:08:12.100 --> 00:08:14.830
Any permutation matrix,
let me take just
00:08:14.830 --> 00:08:17.360
some random permutation matrix.
00:08:17.360 --> 00:08:23.910
Permutation Q equals let's say
oh, make it three by three,
00:08:23.910 --> 00:08:30.050
say zero, zero, one, one,
zero, zero, zero, one, zero.
00:08:30.050 --> 00:08:30.550
OK.
00:08:36.470 --> 00:08:41.860
That certainly has unit
vectors in its columns.
00:08:41.860 --> 00:08:45.940
Those vectors are certainly
perpendicular to each other.
00:08:45.940 --> 00:08:48.700
And if I -- and so that's it.
00:08:48.700 --> 00:08:50.440
That makes it a Q.
00:08:50.440 --> 00:08:56.360
And -- if I took its transpose,
if I multiplied by Q transpose,
00:08:56.360 --> 00:08:59.640
shall I do that -- and let
me stick in Q transpose
00:08:59.640 --> 00:09:00.400
here.
00:09:00.400 --> 00:09:02.990
Just to do that
multiplication once more,
00:09:02.990 --> 00:09:05.200
transpose it'll put the --
00:09:05.200 --> 00:09:08.720
make that into a column,
make that into a column,
00:09:08.720 --> 00:09:10.920
make that into a column.
00:09:10.920 --> 00:09:14.200
And the transpose is also --
00:09:14.200 --> 00:09:15.690
another Q.
00:09:15.690 --> 00:09:17.070
Another orthonormal matrix.
00:09:17.070 --> 00:09:22.110
And when I multiply that
product I get I. OK,
00:09:22.110 --> 00:09:23.890
so there's an example.
00:09:23.890 --> 00:09:26.860
And actually there's
a second example.
00:09:26.860 --> 00:09:29.420
But those are real
easy examples, right,
00:09:29.420 --> 00:09:34.780
I mean to get orthogonal
columns by just
00:09:34.780 --> 00:09:40.630
putting ones in different
places is like too easy.
00:09:40.630 --> 00:09:43.030
So let me keep
going with examples.
00:09:43.030 --> 00:09:46.170
So here's another
simple example.
00:09:46.170 --> 00:09:51.170
Cos theta sine theta,
there's a unit vector,
00:09:51.170 --> 00:09:53.920
oh, let me even
take it, well, yeah.
00:09:53.920 --> 00:09:58.500
Cos theta sine theta
and now the other way
00:09:58.500 --> 00:10:01.790
I want sine theta cos theta.
00:10:01.790 --> 00:10:05.700
But I want the inner
product to be zero.
00:10:05.700 --> 00:10:09.150
And if I put a minus
there, it'll do it.
00:10:09.150 --> 00:10:12.760
So that's -- unit vector,
that's a unit vector.
00:10:12.760 --> 00:10:16.920
And if I take the dot product,
I get minus plus zero.
00:10:19.470 --> 00:10:19.970
OK.
00:10:19.970 --> 00:10:27.230
For example Q equals say
one, one, one, minus one,
00:10:27.230 --> 00:10:30.710
is that an orthogonal matrix?
00:10:33.280 --> 00:10:35.600
I've got orthogonal
columns there,
00:10:35.600 --> 00:10:37.920
but it's not quite
an orthogonal matrix.
00:10:37.920 --> 00:10:43.000
How shall I fix it to
be an orthogonal matrix?
00:10:43.000 --> 00:10:45.780
Well, what's the length
of those column vectors,
00:10:45.780 --> 00:10:51.540
the dot product with themselves
is -- right now it's two,
00:10:51.540 --> 00:10:52.760
right, the --
00:10:52.760 --> 00:10:55.040
the length squared.
00:10:55.040 --> 00:10:57.710
The length squared would be
one plus one would be two,
00:10:57.710 --> 00:10:59.340
the length would be
square root of two,
00:10:59.340 --> 00:11:02.790
so I better divide by
square root of two.
00:11:02.790 --> 00:11:03.750
OK.
00:11:03.750 --> 00:11:08.080
So there's a -- there now I
have got an orthogonal matrix,
00:11:08.080 --> 00:11:13.350
in fact, it's this one --
when theta is pi over four.
00:11:13.350 --> 00:11:16.210
The cosines and
well almost, I guess
00:11:16.210 --> 00:11:19.610
the minus sine is down
there, so maybe, I
00:11:19.610 --> 00:11:23.650
don't know, maybe minus
pi over four or something.
00:11:23.650 --> 00:11:24.150
OK.
00:11:26.760 --> 00:11:28.760
Let me do one
final example, just
00:11:28.760 --> 00:11:32.000
to show that you
can get bigger ones.
00:11:32.000 --> 00:11:38.270
Q equals let me take that
matrix up in the corner
00:11:38.270 --> 00:11:41.830
and I'll sort of
repeat that pattern,
00:11:41.830 --> 00:11:47.010
repeat it again, and
then minus it down here.
00:11:51.110 --> 00:11:57.800
That's one of the world's
favorite orthogonal matrices.
00:11:57.800 --> 00:12:00.090
I hope I got it right, is --
00:12:00.090 --> 00:12:02.500
can you see whether --
00:12:02.500 --> 00:12:05.890
if I take the inner product of
one column with another one,
00:12:05.890 --> 00:12:07.990
let's see, if I take
the inner product
00:12:07.990 --> 00:12:11.070
of that column with that I have
two minuses and two pluses,
00:12:11.070 --> 00:12:11.960
that's good.
00:12:11.960 --> 00:12:14.280
When I take the inner
product of that with that
00:12:14.280 --> 00:12:17.390
I have a plus and a
minus, a minus and a plus.
00:12:17.390 --> 00:12:17.890
Good.
00:12:17.890 --> 00:12:19.530
I think it all works out.
00:12:19.530 --> 00:12:22.210
And what do I have
to divide by now?
00:12:22.210 --> 00:12:24.500
To make those into unit vectors.
00:12:27.320 --> 00:12:34.520
Right now the vector one,
one, one, one has length two.
00:12:34.520 --> 00:12:35.660
Square root of four.
00:12:35.660 --> 00:12:39.680
So I have to divide by two
to make it unit vector,
00:12:39.680 --> 00:12:41.190
so there's another.
00:12:41.190 --> 00:12:45.550
That's my entire array
of simple examples.
00:12:49.190 --> 00:12:56.610
This construction is named after
a guy called Adhemar and we
00:12:56.610 --> 00:13:02.960
know how to do it for
two, four, sixteen,
00:13:02.960 --> 00:13:10.670
sixty-four and so on, but we --
nobody knows exactly which size
00:13:10.670 --> 00:13:12.970
matrices have --
00:13:12.970 --> 00:13:18.250
which size -- which sizes allow
orthogonal matrices of ones
00:13:18.250 --> 00:13:19.300
and minus ones.
00:13:19.300 --> 00:13:24.450
So Adhemar matrix is an
orthogonal matrix that's got
00:13:24.450 --> 00:13:30.070
ones and minus ones, and a
lot of ones -- some we know,
00:13:30.070 --> 00:13:34.130
some other sizes, there couldn't
be a five by five I think.
00:13:34.130 --> 00:13:35.970
But there are some
sizes that nobody
00:13:35.970 --> 00:13:42.220
yet knows whether there could be
or can't be a matrix like that.
00:13:42.220 --> 00:13:43.160
OK.
00:13:43.160 --> 00:13:47.860
You see those
orthogonal matrices.
00:13:47.860 --> 00:13:54.910
Now let me ask what -- why
is it good to have orthogonal
00:13:54.910 --> 00:13:56.050
matrices?
00:13:56.050 --> 00:13:59.830
What calculation is made easy?
00:13:59.830 --> 00:14:02.040
If I have an orthogonal matrix.
00:14:02.040 --> 00:14:06.660
And -- let me remember that the
matrix could be rectangular.
00:14:06.660 --> 00:14:07.880
Shall I put down --
00:14:07.880 --> 00:14:10.680
I better put a
rectangular example down.
00:14:10.680 --> 00:14:13.270
So the -- these were
all square examples.
00:14:13.270 --> 00:14:14.910
Can I put down just --
00:14:14.910 --> 00:14:18.190
a rectangular one
just to be sure
00:14:18.190 --> 00:14:22.080
that we realize that
this is possible.
00:14:22.080 --> 00:14:23.930
let's help me out.
00:14:23.930 --> 00:14:33.870
Let's see, if I put like a
one, two, two and a minus two,
00:14:33.870 --> 00:14:35.630
minus one, two.
00:14:40.530 --> 00:14:45.340
That's -- a matrix -- oh its
columns aren't normalized yet.
00:14:45.340 --> 00:14:47.950
I always have to
remember to do that.
00:14:47.950 --> 00:14:50.590
I always do that last
because it's easy to do.
00:14:50.590 --> 00:14:53.460
What's the length
of those columns?
00:14:53.460 --> 00:14:56.420
So if I wanted them -- if I
wanted them to be length one,
00:14:56.420 --> 00:14:59.840
I should divide by their
length, which is --
00:14:59.840 --> 00:15:03.440
so I'd look at one squared plus
two squared plus two squared,
00:15:03.440 --> 00:15:06.420
that's one and four
and four is nine,
00:15:06.420 --> 00:15:11.420
so I take the square root and
I need to divide by three.
00:15:11.420 --> 00:15:11.920
OK.
00:15:11.920 --> 00:15:14.780
So there is --
00:15:14.780 --> 00:15:21.320
well, without that, I've
got one orthonormal vector.
00:15:21.320 --> 00:15:24.060
I mean just one unit vector.
00:15:24.060 --> 00:15:26.130
Now put that guy in.
00:15:26.130 --> 00:15:29.230
Now I have a basis
for the column
00:15:29.230 --> 00:15:34.980
space for a two-dimensional
space, an orthonormal basis,
00:15:34.980 --> 00:15:35.500
right?
00:15:35.500 --> 00:15:38.140
These two columns
are orthonormal,
00:15:38.140 --> 00:15:40.660
they would be an
orthonormal basis
00:15:40.660 --> 00:15:45.190
for this two-dimensional
space that they span.
00:15:45.190 --> 00:15:48.700
Orthonormal vectors by the way
have got to be independent.
00:15:48.700 --> 00:15:52.910
It's easy to show that
orthonormal vectors
00:15:52.910 --> 00:15:55.640
since they're headed off
all at ninety degrees
00:15:55.640 --> 00:15:58.630
there's no combination
that gives zero.
00:15:58.630 --> 00:16:06.970
Now if I wanted to
create now a third one,
00:16:06.970 --> 00:16:13.220
I could either just put in
some third vector that was
00:16:13.220 --> 00:16:18.800
independent and go to this
Graham-Schmidt calculation that
00:16:18.800 --> 00:16:22.860
I'm going to explain, or I
could be inspired and say look,
00:16:22.860 --> 00:16:26.990
that -- with that pattern,
why not put a one in there,
00:16:26.990 --> 00:16:29.650
and a two in there,
and a two in there,
00:16:29.650 --> 00:16:33.260
and try to fix up the
signs so that they worked.
00:16:36.530 --> 00:16:37.070
Hmm.
00:16:37.070 --> 00:16:41.250
I don't know if I've done
this too brilliantly.
00:16:41.250 --> 00:16:43.230
Let's see, what
signs, that's minus,
00:16:43.230 --> 00:16:49.270
maybe I'd make a minus sign
there, how would that be?
00:16:49.270 --> 00:16:52.990
Yeah, maybe that works.
00:16:52.990 --> 00:17:00.170
I think that those three columns
are orthonormal and they --
00:17:00.170 --> 00:17:03.720
the beauty of this -- this is
the last example I'll probably
00:17:03.720 --> 00:17:08.089
find where there's no
square root, the --
00:17:08.089 --> 00:17:11.250
the punishing thing
in Graham-Schmidt,
00:17:11.250 --> 00:17:14.720
maybe we better know
that in advance,
00:17:14.720 --> 00:17:19.950
is that because I want these
vectors to be unit vectors,
00:17:19.950 --> 00:17:21.900
I'm always running
into square roots.
00:17:21.900 --> 00:17:24.270
I'm always dividing by lengths.
00:17:24.270 --> 00:17:26.109
And those lengths
are square roots.
00:17:26.109 --> 00:17:29.900
So you'll see as soon as I
do a Graham-Schmidt example,
00:17:29.900 --> 00:17:32.190
square roots are
going to show up.
00:17:32.190 --> 00:17:34.960
But here are some
examples where we did it
00:17:34.960 --> 00:17:36.850
without any square root.
00:17:36.850 --> 00:17:38.900
OK.
00:17:38.900 --> 00:17:42.110
So -- so great.
00:17:42.110 --> 00:17:50.760
Now next question is what's
the good of having a Q?
00:17:50.760 --> 00:17:52.860
What formulas become easier?
00:17:52.860 --> 00:17:57.540
Suppose I want to
project, so suppose Q --
00:17:57.540 --> 00:18:03.010
suppose Q has
orthonormal columns.
00:18:03.010 --> 00:18:05.150
I'm using the letter
Q to mean this,
00:18:05.150 --> 00:18:07.130
I'll write it this
one more time,
00:18:07.130 --> 00:18:12.120
but I always mean
when I write a Q,
00:18:12.120 --> 00:18:14.910
I always mean that it
has orthonormal columns.
00:18:14.910 --> 00:18:25.910
So suppose I want to project
onto its column space.
00:18:31.590 --> 00:18:33.170
So what's the projection matrix?
00:18:36.480 --> 00:18:40.760
What's the projection matrix is
I project onto a column space?
00:18:40.760 --> 00:18:46.310
OK, that gives me a chance to
review the projection section,
00:18:46.310 --> 00:18:50.770
including that big formula,
which used to be --
00:18:50.770 --> 00:18:53.580
those four As in a
row, but now it's
00:18:53.580 --> 00:18:57.350
got Qs, because I'm projecting
onto the column space of Q,
00:18:57.350 --> 00:18:58.780
so do you remember what it was?
00:18:58.780 --> 00:19:05.700
It's Q Q transpose Q
inverse Q transpose.
00:19:08.500 --> 00:19:12.390
That's my four Qs in a row.
00:19:12.390 --> 00:19:13.690
But what's good here?
00:19:16.490 --> 00:19:21.040
What -- what makes this formula
nice if I'm projecting onto
00:19:21.040 --> 00:19:24.660
a column space when I have
orthonormal basis for that
00:19:24.660 --> 00:19:25.440
space?
00:19:25.440 --> 00:19:29.110
What makes it nice is
this is the identity.
00:19:29.110 --> 00:19:31.360
I don't have to
do any inversion.
00:19:31.360 --> 00:19:33.190
I just get Q Q transpose.
00:19:40.170 --> 00:19:44.250
So Q Q transpose is
a projection matrix.
00:19:44.250 --> 00:19:45.590
Oh, I can't help --
00:19:45.590 --> 00:19:47.970
I can't resist just
checking the properties,
00:19:47.970 --> 00:19:52.640
what are the properties
of a projection matrix?
00:19:52.640 --> 00:19:57.280
There are two properties to
know for any projection matrix.
00:19:57.280 --> 00:20:00.560
And I'm saying that this
is the right projection
00:20:00.560 --> 00:20:04.500
matrix when we've got
this orthonormal basis
00:20:04.500 --> 00:20:06.980
in the columns.
00:20:06.980 --> 00:20:07.540
OK.
00:20:07.540 --> 00:20:10.650
So there's the
projection matrix.
00:20:10.650 --> 00:20:13.670
Suppose the matrix is square.
00:20:13.670 --> 00:20:17.090
First just tell me
first this extreme case.
00:20:17.090 --> 00:20:22.190
If my matrix is square and it's
got these orthonormal columns,
00:20:22.190 --> 00:20:26.170
then what's the column space?
00:20:26.170 --> 00:20:31.270
If I have a square matrix and
I have independent columns,
00:20:31.270 --> 00:20:34.950
and even orthonormal columns,
then the column space
00:20:34.950 --> 00:20:37.170
is the whole space, right?
00:20:37.170 --> 00:20:41.250
And what's the projection
matrix onto the whole space?
00:20:41.250 --> 00:20:43.790
The identity matrix.
00:20:43.790 --> 00:20:45.360
If I'm projecting
in the whole space,
00:20:45.360 --> 00:20:49.690
every vector B is right
where it's supposed to be
00:20:49.690 --> 00:20:52.810
and I don't have to
move it by projection.
00:20:52.810 --> 00:20:57.800
So this would be --
00:20:57.800 --> 00:21:02.390
I'll put in parentheses
this is I if Q is square.
00:21:07.890 --> 00:21:10.360
Well that we said that already.
00:21:10.360 --> 00:21:14.990
If Q is square, that's the case
where Q transpose is Q inverse,
00:21:14.990 --> 00:21:18.140
we can put it on the right,
we can put it on the left,
00:21:18.140 --> 00:21:23.000
we always get the identity
matrix, if it's square.
00:21:23.000 --> 00:21:29.070
But if it's not a square
matrix then it's not --
00:21:29.070 --> 00:21:32.510
we don't get the
identity matrix.
00:21:32.510 --> 00:21:37.710
We have Q Q transpose,
and just again
00:21:37.710 --> 00:21:41.050
what are those two properties
of a projection matrix?
00:21:41.050 --> 00:21:44.400
First of all, it's symmetric.
00:21:44.400 --> 00:21:48.870
OK, no problem, that's
certainly a symmetric So what's
00:21:48.870 --> 00:21:50.860
that second property
of a projection?
00:21:50.860 --> 00:21:51.360
matrix.
00:21:51.360 --> 00:21:55.010
That if you project and project
again you don't move the second
00:21:55.010 --> 00:21:55.760
time.
00:21:55.760 --> 00:21:58.220
So the other property
of a projection matrix
00:21:58.220 --> 00:22:04.070
should be that Q
Q transpose twice
00:22:04.070 --> 00:22:09.320
should be the same as
Q Q transpose once.
00:22:09.320 --> 00:22:11.550
That's projection matrices.
00:22:11.550 --> 00:22:14.170
And that property
better fall out
00:22:14.170 --> 00:22:18.540
right away because
from the fact we
00:22:18.540 --> 00:22:24.060
know about orthonormal matrices,
Q transpose Q is I. OK,
00:22:24.060 --> 00:22:25.010
you see it.
00:22:25.010 --> 00:22:30.320
In the middle here is sitting
Q Q t- Q transpose Q, sorry,
00:22:30.320 --> 00:22:34.030
that's what I meant to
say, Q transpose Q is I.
00:22:34.030 --> 00:22:37.000
So that's sitting right in
the middle, that cancels out,
00:22:37.000 --> 00:22:40.760
to give the identity, we're
left with one Q Q transpose,
00:22:40.760 --> 00:22:43.480
and we're all set.
00:22:43.480 --> 00:22:44.180
OK.
00:22:44.180 --> 00:22:48.840
So this is the
projection matrix --
00:22:48.840 --> 00:22:54.670
all the equation -- all the
messy equations of this chapter
00:22:54.670 --> 00:22:59.370
become trivial
when our matrix --
00:22:59.370 --> 00:23:02.540
when we have this
orthonormal basis.
00:23:02.540 --> 00:23:04.630
I mean what do I mean
by all the equations?
00:23:04.630 --> 00:23:06.280
Well, the most
important equation
00:23:06.280 --> 00:23:10.680
was the normal equation, do
you remember old A transpose
00:23:10.680 --> 00:23:15.620
A x hat equals A transpose b?
00:23:15.620 --> 00:23:22.200
But now -- now A is Q.
00:23:22.200 --> 00:23:28.140
Now I'm thinking I have
Q transpose Q X hat
00:23:28.140 --> 00:23:32.000
equals Q transpose b.
00:23:32.000 --> 00:23:33.280
And what's good about that?
00:23:37.220 --> 00:23:42.950
What's good is that matrix on
the left side is the identity.
00:23:42.950 --> 00:23:46.110
The matrix on the left is
the identity, Q transpose Q,
00:23:46.110 --> 00:23:49.510
normally it isn't, normally it's
that matrix of inner products
00:23:49.510 --> 00:23:53.870
and you've to compute all those
dopey inner products and --
00:23:53.870 --> 00:23:56.030
and -- and solve the system.
00:23:56.030 --> 00:23:59.950
Here the inner products
are all one or zero.
00:23:59.950 --> 00:24:01.540
This is the identity matrix.
00:24:01.540 --> 00:24:03.280
It's gone.
00:24:03.280 --> 00:24:05.910
And there's the answer.
00:24:05.910 --> 00:24:09.140
There's no inversion involved.
00:24:09.140 --> 00:24:15.730
Each component of
x is a Q times b.
00:24:15.730 --> 00:24:21.240
What that equation is saying
is that the i-th component is
00:24:21.240 --> 00:24:26.420
the i-th basis vector times b.
00:24:26.420 --> 00:24:34.760
That's -- probably the most
important formula in some major
00:24:34.760 --> 00:24:40.720
parts of mathematics, that
if we have orthonormal basis,
00:24:40.720 --> 00:24:47.660
then the component in the --
in the i-th, along the i-th --
00:24:47.660 --> 00:24:54.630
the projection on the i-th basis
vector is just qi transpose b.
00:24:54.630 --> 00:25:00.580
That number x that we look
for is just a dot product.
00:25:00.580 --> 00:25:01.080
OK.
00:25:04.050 --> 00:25:10.040
OK, so I'm ready now for
the sort of like second half
00:25:10.040 --> 00:25:11.530
of the lecture.
00:25:11.530 --> 00:25:16.320
Where we don't start with
an orthogonal matrix,
00:25:16.320 --> 00:25:18.910
orthonormal vectors.
00:25:18.910 --> 00:25:21.610
We just start with
independent vectors
00:25:21.610 --> 00:25:25.370
and we want to make
them orthonormal.
00:25:25.370 --> 00:25:27.860
So I'm going to --
can I do that now?
00:25:27.860 --> 00:25:29.940
Now here comes Graham-Schmidt.
00:25:29.940 --> 00:25:31.490
So -- Graham-Schmidt.
00:25:39.460 --> 00:25:43.990
So this is a calculation,
I won't say --
00:25:43.990 --> 00:25:53.190
I can't quite say it's like
elimination, because it's
00:25:53.190 --> 00:25:56.630
different, our goal
isn't triangular anymore.
00:25:56.630 --> 00:26:00.780
With elimination our goal was
make the matrix triangular.
00:26:00.780 --> 00:26:04.200
Now our goal is make
the matrix orthogonal.
00:26:04.200 --> 00:26:07.580
Make those columns orthonormal.
00:26:07.580 --> 00:26:10.240
So let me start
with two columns.
00:26:10.240 --> 00:26:13.000
So I start with vectors a and b.
00:26:16.960 --> 00:26:20.430
And they're just like --
here, let me draw them.
00:26:20.430 --> 00:26:22.780
Here's a.
00:26:22.780 --> 00:26:23.400
Here's b.
00:26:26.370 --> 00:26:27.580
For example.
00:26:27.580 --> 00:26:29.700
A isn't specially
horizontal, wasn't
00:26:29.700 --> 00:26:34.150
meant to be, just a is
one vector, b is another.
00:26:34.150 --> 00:26:38.040
I want to produce
those two vectors,
00:26:38.040 --> 00:26:40.760
they might be in
twelve-dimensional space,
00:26:40.760 --> 00:26:43.940
or they might be in
two-dimensional space.
00:26:43.940 --> 00:26:46.300
They're independent, anyway.
00:26:46.300 --> 00:26:49.500
So I better be sure I say that.
00:26:49.500 --> 00:26:51.330
I start with
independent vectors.
00:26:54.860 --> 00:26:58.500
And I want to produce
out of that q 1 and q2,
00:26:58.500 --> 00:27:00.970
I want to produce
orthonormal vectors.
00:27:03.880 --> 00:27:09.730
And Graham and
Schmidt tell me how.
00:27:09.730 --> 00:27:10.260
OK.
00:27:10.260 --> 00:27:14.330
Well, actually you could tell me
how, we don't need -- frankly,
00:27:14.330 --> 00:27:17.800
I don't know -- there's
only one idea here,
00:27:17.800 --> 00:27:24.850
if Graham had the idea, I
don't know what Schmidt did.
00:27:24.850 --> 00:27:28.140
But OK.
00:27:28.140 --> 00:27:29.470
So you'll see it.
00:27:29.470 --> 00:27:31.900
We don't need either
of them, actually.
00:27:31.900 --> 00:27:33.630
OK, so what I going to do.
00:27:33.630 --> 00:27:36.630
I'll take that --
this first guy.
00:27:36.630 --> 00:27:37.970
OK.
00:27:37.970 --> 00:27:39.790
Well, he's fine.
00:27:43.480 --> 00:27:46.620
That direction is fine except --
00:27:46.620 --> 00:27:50.300
yeah, I'll say OK, I'll
settle for that direction.
00:27:50.300 --> 00:27:51.410
So I'm going to --
00:27:51.410 --> 00:27:53.630
I'm going to get, so
what I going to --
00:27:53.630 --> 00:27:59.370
my goal is I'm going to
get orthogonal vectors
00:27:59.370 --> 00:28:02.780
and I'll call those
capital A and B.
00:28:02.780 --> 00:28:07.590
So that's the key step is
to get from any two vectors
00:28:07.590 --> 00:28:09.430
to two orthogonal vectors.
00:28:09.430 --> 00:28:14.530
And then at the end, no problem,
I'll get orthonormal vectors,
00:28:14.530 --> 00:28:20.630
how will -- what will those
will be my qs, q1 and q2,
00:28:20.630 --> 00:28:21.450
and what will they
00:28:21.450 --> 00:28:21.950
be?
00:28:25.980 --> 00:28:30.270
Once I've got A and B
orthogonal, well, look,
00:28:30.270 --> 00:28:35.390
it's no big deal -- maybe
that's what Schmidt did, he,
00:28:35.390 --> 00:28:38.840
brilliant Schmidt, thought
OK, divide by the length,
00:28:38.840 --> 00:28:40.310
all right.
00:28:40.310 --> 00:28:42.820
That's Schmidt's contribution.
00:28:45.640 --> 00:28:46.140
OK.
00:28:51.250 --> 00:28:55.620
But Graham had a little
more thinking to do, right?
00:28:55.620 --> 00:28:58.780
We haven't done Graham's part.
00:28:58.780 --> 00:29:03.050
This part except OK,
I'm happy with A,
00:29:03.050 --> 00:29:07.170
A can be A. That first
direction is fine.
00:29:07.170 --> 00:29:09.620
Why should -- no
complaint about that.
00:29:09.620 --> 00:29:13.620
The trouble is the second
direction is not fine.
00:29:13.620 --> 00:29:18.170
Because it's not
orthogonal to the first.
00:29:18.170 --> 00:29:23.710
I'm looking for a vector
that's -- starts with B,
00:29:23.710 --> 00:29:29.550
but makes it orthogonal to A.
00:29:29.550 --> 00:29:30.980
What's the vector?
00:29:30.980 --> 00:29:32.870
How do I do that?
00:29:32.870 --> 00:29:35.960
How do I produce
from this vector
00:29:35.960 --> 00:29:41.820
a piece that's
orthogonal to this one?
00:29:41.820 --> 00:29:44.900
And the -- remember these
vectors might be in two
00:29:44.900 --> 00:29:48.570
dimensions or they might
be in twelve dimensions.
00:29:48.570 --> 00:29:51.210
I'm just looking for the idea.
00:29:51.210 --> 00:29:53.840
So what's the idea?
00:29:53.840 --> 00:29:57.110
Where did we have orthogonal --
00:29:57.110 --> 00:30:01.440
a vector showing up that
was orthogonal to this guy?
00:30:01.440 --> 00:30:03.990
Well, that was the
first basic calculation
00:30:03.990 --> 00:30:05.780
of the whole chapter.
00:30:05.780 --> 00:30:10.850
We -- we did a projection and
the projection gave us this
00:30:10.850 --> 00:30:15.890
part, which was the
part in the A direction.
00:30:15.890 --> 00:30:19.500
Now, the part we want is
the other part, the e part.
00:30:19.500 --> 00:30:21.050
This part.
00:30:21.050 --> 00:30:23.790
This is going to be our --
00:30:23.790 --> 00:30:25.450
that guy is that guy.
00:30:25.450 --> 00:30:31.110
This is our vector B. That gives
us that ninety-degree angle.
00:30:31.110 --> 00:30:33.240
So B is you could say --
00:30:33.240 --> 00:30:35.730
B is really what we
previously called
00:30:35.730 --> 00:30:37.140
e.
00:30:37.140 --> 00:30:41.060
The error vector.
00:30:41.060 --> 00:30:42.950
And what is it?
00:30:42.950 --> 00:30:45.950
I mean what do I
-- what is B here?
00:30:45.950 --> 00:30:47.690
A is A, no problem.
00:30:47.690 --> 00:30:52.410
B is --
00:30:52.410 --> 00:30:54.120
OK, what's this error piece?
00:30:54.120 --> 00:30:56.530
Do you remember?
00:30:56.530 --> 00:31:04.150
It's I start with the original
B and I take away what?
00:31:04.150 --> 00:31:08.670
Its projection, this P.
This -- the vector B,
00:31:08.670 --> 00:31:12.160
this error vector, is the
original vector removing
00:31:12.160 --> 00:31:12.890
the projection.
00:31:12.890 --> 00:31:15.590
So instead of wanting
the projection,
00:31:15.590 --> 00:31:20.260
now that's what I
want to throw away.
00:31:20.260 --> 00:31:22.780
I want to get the part
that's perpendicular.
00:31:22.780 --> 00:31:24.720
And there will be a
perpendicular part,
00:31:24.720 --> 00:31:25.450
it won't be zero.
00:31:28.890 --> 00:31:32.880
Because these vectors
were independent, so B --
00:31:32.880 --> 00:31:34.880
if B was along the
direction of A,
00:31:34.880 --> 00:31:37.830
then if the original B and A
were in the same direction,
00:31:37.830 --> 00:31:38.760
then I'm --
00:31:38.760 --> 00:31:40.500
I've only got one direction.
00:31:40.500 --> 00:31:43.440
But here they're in two
independent directions
00:31:43.440 --> 00:31:46.190
and all I'm doing
is getting that guy.
00:31:46.190 --> 00:31:49.550
So what's its formula?
00:31:49.550 --> 00:31:53.510
What's the formula
for that if --
00:31:53.510 --> 00:31:55.540
I want to subtract
the projection,
00:31:55.540 --> 00:31:57.450
so do you remember
the projection?
00:31:57.450 --> 00:32:03.490
It's some multiple of A
and what's that multiple?
00:32:03.490 --> 00:32:07.030
It's -- it's that thing we
called x in the very very first
00:32:07.030 --> 00:32:10.150
lecture on this chapter.
00:32:10.150 --> 00:32:16.540
There's an A transpose
A in the bottom
00:32:16.540 --> 00:32:23.990
and there's an A transpose
B, isn't that it?
00:32:29.780 --> 00:32:32.240
I think that's Graham's formula.
00:32:32.240 --> 00:32:33.300
Or Graham-Schmidt.
00:32:33.300 --> 00:32:34.600
No, that's Graham.
00:32:34.600 --> 00:32:38.120
Schmidt has got to divide the
whole thing by the length,
00:32:38.120 --> 00:32:39.660
so he --
00:32:39.660 --> 00:32:43.280
his formula makes a mess which
I'm not willing to write down.
00:32:43.280 --> 00:32:48.070
So let's just see that
what I saying here?
00:32:48.070 --> 00:32:51.670
I'm saying that this vector
is perpendicular to A.
00:32:51.670 --> 00:32:53.260
That these are orthogonal.
00:32:53.260 --> 00:32:56.710
A is perpendicular to B.
00:32:56.710 --> 00:32:58.490
Can you check that?
00:32:58.490 --> 00:33:01.430
How do you see that
yes, of course, we --
00:33:01.430 --> 00:33:04.160
our picture is telling
us, yes, we did it right.
00:33:04.160 --> 00:33:08.720
How would I check that this
matrix is perpendicular to A?
00:33:11.390 --> 00:33:16.740
I would multiply by A transpose
and I better get zero, right?
00:33:16.740 --> 00:33:18.410
I should check that.
00:33:18.410 --> 00:33:22.930
A transpose B should
come out zero.
00:33:22.930 --> 00:33:26.920
So this is A transpose times
-- now what did we say B was?
00:33:26.920 --> 00:33:30.370
We start with the original
B, and we take away
00:33:30.370 --> 00:33:38.090
this projection, and that
should come out zero.
00:33:38.090 --> 00:33:42.950
Well, here we get an
A transpose B minus --
00:33:42.950 --> 00:33:46.050
and here is another A
transpose B, and the --
00:33:46.050 --> 00:33:50.120
and it's an A transpose A
over A transpose A, a one,
00:33:50.120 --> 00:33:53.840
those cancel, and
we do get zero.
00:33:53.840 --> 00:33:54.340
Right.
00:33:58.360 --> 00:34:05.940
Now I guess I can
do numbers in there.
00:34:05.940 --> 00:34:09.420
But I have to take
a third vector
00:34:09.420 --> 00:34:13.330
to be sure we've got
this system down.
00:34:13.330 --> 00:34:20.250
So now I have to say if I have
independent vectors A, B and C,
00:34:20.250 --> 00:34:25.820
I'm looking for orthogonal
vectors A, B and capital C,
00:34:25.820 --> 00:34:29.170
and then of course the
third guy will just
00:34:29.170 --> 00:34:32.969
be C over its length,
the unit vector.
00:34:35.860 --> 00:34:39.980
So this is now the problem.
00:34:39.980 --> 00:34:42.929
I got B here.
00:34:42.929 --> 00:34:45.889
I got A very easily.
00:34:45.889 --> 00:34:52.610
And now -- if you see the idea,
we could figure out a formula
00:34:52.610 --> 00:35:01.390
for C. So now that -- so this
is like a typical homework quiz
00:35:01.390 --> 00:35:02.150
problem.
00:35:02.150 --> 00:35:07.260
I give you two vectors, you do
this, I give you three vectors,
00:35:07.260 --> 00:35:10.460
and you have to make
them orthonormal.
00:35:10.460 --> 00:35:13.520
So you do this again,
the first vector's fine,
00:35:13.520 --> 00:35:17.160
the second vector is
perpendicular to the first,
00:35:17.160 --> 00:35:19.720
and now I need a
third vector that's
00:35:19.720 --> 00:35:23.240
perpendicular to the first
one and the second one.
00:35:23.240 --> 00:35:25.560
Right?
00:35:25.560 --> 00:35:29.660
Tthis is the end of a -- the
lecture is to find this guy.
00:35:29.660 --> 00:35:33.980
Find this vector -- this vector
C, that's perpendicular we n-
00:35:33.980 --> 00:35:39.410
at this point we know A and B.
00:35:39.410 --> 00:35:44.540
But C, the little c that
we're given, is off in some --
00:35:44.540 --> 00:35:48.030
it's got to come out of the
blackboard to be independent,
00:35:48.030 --> 00:35:52.440
so -- so can I sort of draw
off -- off comes a c somewhere.
00:35:52.440 --> 00:35:54.240
I don't know, where I
going to put the darn
00:35:54.240 --> 00:35:55.050
thing?
00:35:55.050 --> 00:35:59.450
Maybe I'll put it
off, oh, I don't know,
00:35:59.450 --> 00:36:02.160
like that somehow, C, little c.
00:36:05.610 --> 00:36:08.930
And I already know that
perpendicular direction,
00:36:08.930 --> 00:36:10.750
that one and that one.
00:36:10.750 --> 00:36:13.800
So now what's the idea?
00:36:13.800 --> 00:36:17.080
Give me the Graham-Schmidt
formula for C.
00:36:17.080 --> 00:36:20.700
What is this C here?
00:36:20.700 --> 00:36:21.520
Equals what?
00:36:27.340 --> 00:36:28.160
What I going to do?
00:36:28.160 --> 00:36:31.300
I'll start with the given one.
00:36:31.300 --> 00:36:32.730
As before.
00:36:32.730 --> 00:36:33.280
Right?
00:36:33.280 --> 00:36:36.410
I start with the
vector I'm given.
00:36:36.410 --> 00:36:38.400
And what do I do with it?
00:36:38.400 --> 00:36:42.060
I want to remove out of
it, I want to subtract off,
00:36:42.060 --> 00:36:46.510
so I'll put a minus sign
in, I want to subtract off
00:36:46.510 --> 00:36:53.010
its components in the A, capital
A and capital B directions.
00:36:53.010 --> 00:36:55.850
I just want to get
those out of there.
00:36:55.850 --> 00:36:57.290
Well, I know how to do that.
00:36:57.290 --> 00:36:58.710
I did it with B.
00:36:58.710 --> 00:37:02.410
So I'll just -- so
let me take away --
00:37:02.410 --> 00:37:03.440
what if I do this?
00:37:07.960 --> 00:37:08.850
What have I done?
00:37:12.060 --> 00:37:16.520
I've got little c and what
have I subtracted from it?
00:37:16.520 --> 00:37:22.280
Its component, its projection
if you like, in the A direction.
00:37:25.010 --> 00:37:30.820
And now I've got to subtract
off its component B transpose
00:37:30.820 --> 00:37:35.200
C over B transpose B,
that multiple of B,
00:37:35.200 --> 00:37:37.570
is its component
in the B direction.
00:37:40.100 --> 00:37:47.480
And that gives me the vector
capital C that if anything is
00:37:47.480 --> 00:37:48.160
--
00:37:48.160 --> 00:37:54.700
if there's any justice, this
C should be perpendicular to A
00:37:54.700 --> 00:37:58.750
and it should be
perpendicular to B.
00:37:58.750 --> 00:38:02.660
And the only thing it's --
hasn't got is unit vector,
00:38:02.660 --> 00:38:05.830
so we divide by its
length to get that too.
00:38:08.660 --> 00:38:10.390
OK.
00:38:10.390 --> 00:38:14.940
Let me do an example.
00:38:14.940 --> 00:38:16.410
Can I --
00:38:16.410 --> 00:38:20.990
I'll make my life easy,
I'll just take two vectors.
00:38:20.990 --> 00:38:23.880
So let me do a
numerical example.
00:38:23.880 --> 00:38:26.480
If I'll give you
two vectors, you
00:38:26.480 --> 00:38:31.220
give me back the Graham-Schmidt
orthonormal basis,
00:38:31.220 --> 00:38:35.990
and we'll see how to
express that in matrix form.
00:38:35.990 --> 00:38:36.530
OK.
00:38:36.530 --> 00:38:41.080
So let me give you
the two vectors.
00:38:41.080 --> 00:38:46.420
So I'll take the vector A
equals let's say one, one, one,
00:38:46.420 --> 00:38:47.910
why not?
00:38:47.910 --> 00:38:55.810
And B equals let's say
one, zero, two, OK?
00:39:02.400 --> 00:39:05.300
I didn't want to cheat
and make them orthogonal
00:39:05.300 --> 00:39:07.330
in the first place because
then Graham-Schmidt
00:39:07.330 --> 00:39:08.430
wouldn't be needed.
00:39:08.430 --> 00:39:08.930
OK.
00:39:08.930 --> 00:39:10.760
So those are not orthogonal.
00:39:10.760 --> 00:39:12.330
So what is capital A?
00:39:12.330 --> 00:39:14.280
Well that's the same as big A.
00:39:14.280 --> 00:39:15.350
That was fine.
00:39:15.350 --> 00:39:18.590
What's B?
00:39:18.590 --> 00:39:21.600
So B is this b --
is the original B,
00:39:21.600 --> 00:39:29.680
and then I subtract off
some multiple of the A.
00:39:29.680 --> 00:39:30.900
And what's the multiple?
00:39:33.810 --> 00:39:36.520
What goes in here?
00:39:36.520 --> 00:39:41.240
B -- here's the A -- this is
the -- this is the little b,
00:39:41.240 --> 00:39:45.430
this is the big A, also
the little a, and I want
00:39:45.430 --> 00:39:48.680
to multiply it by that
right -- that right ratio,
00:39:48.680 --> 00:39:53.480
which has A transpose
A, here's my ratio.
00:39:53.480 --> 00:39:57.100
I'm just doing this.
00:39:57.100 --> 00:40:00.610
So it's A transpose B,
what is A transpose B,
00:40:00.610 --> 00:40:02.790
it looks like three.
00:40:02.790 --> 00:40:06.780
And what is A -- oh, my --
00:40:06.780 --> 00:40:08.470
what's A transpose A?
00:40:08.470 --> 00:40:08.970
Three.
00:40:11.540 --> 00:40:12.690
I'm sorry.
00:40:12.690 --> 00:40:14.990
I didn't know that
was going to happen.
00:40:14.990 --> 00:40:15.490
OK.
00:40:15.490 --> 00:40:16.440
But it happened.
00:40:16.440 --> 00:40:18.750
Why should we knock it?
00:40:18.750 --> 00:40:19.610
OK.
00:40:19.610 --> 00:40:21.430
So do you see it all right?
00:40:21.430 --> 00:40:25.040
That's A transpose B,
there's A transpose A, that's
00:40:25.040 --> 00:40:28.640
the fraction, so
I take this away,
00:40:28.640 --> 00:40:33.640
and I get one take away one
is a zero, zero minus this one
00:40:33.640 --> 00:40:39.180
is a minus one, and two
minus the one is a one.
00:40:39.180 --> 00:40:39.930
OK.
00:40:39.930 --> 00:40:42.330
And what's this vector
that we finally found?
00:40:42.330 --> 00:40:47.020
This is B.
00:40:47.020 --> 00:40:48.580
And how do I know it's right?
00:40:51.930 --> 00:40:54.690
How do I know I've
got a vector I want?
00:40:54.690 --> 00:40:57.700
I check that B is
perpendicular to --
00:40:57.700 --> 00:40:59.980
that A and B are perpendicular.
00:40:59.980 --> 00:41:02.280
That A is perpendicular to B.
00:41:02.280 --> 00:41:03.190
Just look at that.
00:41:03.190 --> 00:41:06.550
That one -- the dot product
of that with that is zero.
00:41:06.550 --> 00:41:07.180
OK.
00:41:07.180 --> 00:41:10.850
So now what is my q1 and q2?
00:41:14.720 --> 00:41:17.700
Why don't I put
them in a matrix?
00:41:17.700 --> 00:41:18.480
Of course.
00:41:18.480 --> 00:41:20.970
Since I'm always putting
these -- so the Q,
00:41:20.970 --> 00:41:24.260
I'll put the q1 and
the q2 in a matrix.
00:41:24.260 --> 00:41:25.720
And what are they?
00:41:29.270 --> 00:41:32.700
Now when I'm writing
q-s I'm supposed
00:41:32.700 --> 00:41:34.200
to make things normalized.
00:41:34.200 --> 00:41:36.450
I'm supposed to make
things unit vectors.
00:41:36.450 --> 00:41:39.740
So I'm going to take that A
but I'm going to divide it
00:41:39.740 --> 00:41:41.650
by square root of three.
00:41:46.920 --> 00:41:48.780
And I'm going to
take this B but I'm
00:41:48.780 --> 00:41:53.370
going to divide it
by square root of two
00:41:53.370 --> 00:41:57.720
to make it a unit vector,
and there is my matrix.
00:42:01.000 --> 00:42:05.430
That's my matrix with
orthonormal columns coming from
00:42:05.430 --> 00:42:08.790
Graham-Schmidt and
it sort of it --
00:42:08.790 --> 00:42:14.740
it came from the original
one, one, one, one, zero, two,
00:42:14.740 --> 00:42:15.240
right?
00:42:15.240 --> 00:42:16.620
That was my original guys.
00:42:20.880 --> 00:42:23.200
These were the two
I started with.
00:42:23.200 --> 00:42:25.950
These are the two that
I'm happy to end with.
00:42:25.950 --> 00:42:30.110
Because those are orthonormal.
00:42:30.110 --> 00:42:33.490
So that's what
Graham-Schmidt did.
00:42:33.490 --> 00:42:38.050
It -- well, tell me about
the column spaces of these
00:42:38.050 --> 00:42:40.260
matrices.
00:42:40.260 --> 00:42:44.180
How is the column space of Q
related to the column space of
00:42:44.180 --> 00:42:44.900
A?
00:42:44.900 --> 00:42:47.150
So I'm always asking
you things like this,
00:42:47.150 --> 00:42:49.970
and that makes you think,
OK, the column space
00:42:49.970 --> 00:42:54.550
is all combinations of the
columns, it's that plane,
00:42:54.550 --> 00:42:55.540
right?
00:42:55.540 --> 00:42:58.770
I've got two vectors in
three-dimensional space,
00:42:58.770 --> 00:43:03.630
their column space is a plane,
the column space of this matrix
00:43:03.630 --> 00:43:08.200
is a plane, what's the
relation between the planes?
00:43:08.200 --> 00:43:09.740
Between the two column spaces?
00:43:12.750 --> 00:43:15.190
They're one and the same, right?
00:43:15.190 --> 00:43:17.760
It's the same column space.
00:43:17.760 --> 00:43:23.580
All I'm taking is here this
B thing that I computed,
00:43:23.580 --> 00:43:30.120
this B thing that I computed
is a combination of B and A,
00:43:30.120 --> 00:43:34.630
and A was little A, so
I'm always working here
00:43:34.630 --> 00:43:36.920
with this in the same space.
00:43:36.920 --> 00:43:42.450
I'm just like getting
ninety-degree angles in there.
00:43:42.450 --> 00:43:47.450
Where my original column space
had a perfectly good basis,
00:43:47.450 --> 00:43:50.470
but it wasn't as
good as this basis,
00:43:50.470 --> 00:43:53.610
because it wasn't orthonormal.
00:43:53.610 --> 00:43:59.560
Now this one is orthonormal,
and I have a basis then that --
00:43:59.560 --> 00:44:03.050
so now projections, all the
calculations I would ever want
00:44:03.050 --> 00:44:09.570
to do are -- are a cinch
with this orthonormal basis.
00:44:09.570 --> 00:44:12.710
One final point.
00:44:12.710 --> 00:44:14.480
One final point in this chapter.
00:44:17.200 --> 00:44:21.240
And it's -- just
like elimination.
00:44:21.240 --> 00:44:23.530
We learned how to
do elimination,
00:44:23.530 --> 00:44:26.160
we know all the
steps, we can do it.
00:44:26.160 --> 00:44:34.720
But then I came back to it and
said look at it as a matrix
00:44:34.720 --> 00:44:40.140
in matrix language and
elimination gave me --
00:44:40.140 --> 00:44:42.480
what was elimination
in matrix language?
00:44:42.480 --> 00:44:44.000
I'll just put it up there.
00:44:44.000 --> 00:44:46.240
A was LU.
00:44:46.240 --> 00:44:49.720
That was matrix,
that was elimination.
00:44:49.720 --> 00:44:53.180
Now, I want to do the
same for Graham-Schmidt.
00:44:53.180 --> 00:44:56.200
Everybody who works
in linear algebra
00:44:56.200 --> 00:44:58.530
isn't going to write
out the columns
00:44:58.530 --> 00:45:01.220
are orthogonal, or orthonormal.
00:45:01.220 --> 00:45:04.530
And isn't going to write
out these formulas.
00:45:04.530 --> 00:45:08.820
They're going to write out the
connection between the matrix A
00:45:08.820 --> 00:45:11.320
and the matrix Q.
00:45:11.320 --> 00:45:14.230
And the two matrices have
the same column space,
00:45:14.230 --> 00:45:17.720
but there's some -- some
matrix is taking the --
00:45:17.720 --> 00:45:25.110
and I'm going to call it R, so
A equals QR is the magic formula
00:45:25.110 --> 00:45:25.610
here.
00:45:28.200 --> 00:45:30.260
It's the expression
of Graham-Schmidt.
00:45:32.860 --> 00:45:38.430
And I'll -- let me
just capture that.
00:45:38.430 --> 00:45:42.240
So that's the -- my final
step then is A equal QR.
00:45:42.240 --> 00:45:44.310
Maybe I can squeeze it in here.
00:45:47.300 --> 00:45:50.960
So A has columns,
let's say a1 and a2.
00:45:55.420 --> 00:45:59.150
Let me suppose n is
two, just two vectors.
00:45:59.150 --> 00:46:00.290
OK.
00:46:00.290 --> 00:46:06.300
So that's some
combination of q1 and q2.
00:46:06.300 --> 00:46:13.170
And times some matrix R.
00:46:13.170 --> 00:46:16.330
They have the same column space.
00:46:16.330 --> 00:46:20.420
This is just -- this matrix just
includes in it whatever these
00:46:20.420 --> 00:46:23.430
numbers like three over three
and one over square root
00:46:23.430 --> 00:46:25.360
of three and one over
square root of two,
00:46:25.360 --> 00:46:28.400
probably that's what it's got.
00:46:28.400 --> 00:46:31.140
One over square root of three,
one over square root of two,
00:46:31.140 --> 00:46:34.390
something there, but actually
it's got a zero there.
00:46:37.690 --> 00:46:45.260
So the main point about
this A equal QR is this R
00:46:45.260 --> 00:46:48.270
turns out to be
upper triangular.
00:46:48.270 --> 00:46:50.750
It turns out that this
zero is upper triangular.
00:46:53.510 --> 00:46:56.440
We could see why.
00:46:56.440 --> 00:47:00.640
Let me see, I can put in
general formulas for what these
00:47:00.640 --> 00:47:05.190
This I think in here should
be the inner product of a1
00:47:05.190 --> 00:47:05.740
with q1. are.
00:47:08.600 --> 00:47:12.160
And this one should be the --
00:47:12.160 --> 00:47:16.230
the inner product of a1 with q2.
00:47:16.230 --> 00:47:18.970
And that's what I
believe is zero.
00:47:21.710 --> 00:47:25.110
This will be something here,
and this will be something here
00:47:25.110 --> 00:47:33.960
with inner -- a1 transpose q2,
sorry a2 transpose q1 and a2
00:47:33.960 --> 00:47:35.060
transpose q2.
00:47:35.060 --> 00:47:37.130
But why is that guy zero?
00:47:40.100 --> 00:47:43.900
Why is a1 q2 zero?
00:47:43.900 --> 00:47:47.580
That's the key to this being
-- this R here being upper
00:47:47.580 --> 00:47:49.110
triangular.
00:47:49.110 --> 00:47:55.100
You know why a1q2 is
zero, because a1 --
00:47:55.100 --> 00:47:57.310
that was my --
00:47:57.310 --> 00:48:00.180
this was really a and b here.
00:48:00.180 --> 00:48:02.700
This was really a and b.
00:48:02.700 --> 00:48:05.780
So this is a transpose q2.
00:48:05.780 --> 00:48:09.380
And the whole point of
Graham-Schmidt was that we
00:48:09.380 --> 00:48:14.820
constructed these later q-s to
be perpendicular to the earlier
00:48:14.820 --> 00:48:19.030
vectors, to the earlier --
all the earlier vectors.
00:48:19.030 --> 00:48:21.090
So that's why we get
a triangular matrix.
00:48:23.800 --> 00:48:29.750
The -- result is
extremely satisfactory.
00:48:32.530 --> 00:48:37.450
That if I have a matrix
with independent columns,
00:48:37.450 --> 00:48:40.540
the Graham-Schmidt
produces a matrix
00:48:40.540 --> 00:48:44.780
with orthonormal columns, and
the connection between those
00:48:44.780 --> 00:48:48.230
is a triangular matrix.
00:48:48.230 --> 00:48:51.200
That last point, that the
connection is a triangular
00:48:51.200 --> 00:48:53.510
matrix, please look
in the book, you
00:48:53.510 --> 00:48:56.390
have to see that one more time.
00:48:56.390 --> 00:48:56.930
OK.
00:48:56.930 --> 00:48:59.170
Thanks, that's great.