WEBVTT
00:00:10.240 --> 00:00:17.470
OK, guys the -- we're almost
ready to make this lecture
00:00:17.470 --> 00:00:19.310
immortal.
00:00:19.310 --> 00:00:21.340
OK.
00:00:21.340 --> 00:00:22.720
Are we on?
00:00:22.720 --> 00:00:23.490
All right.
00:00:23.490 --> 00:00:26.820
This is an important lecture.
00:00:26.820 --> 00:00:30.170
It's about projections.
00:00:30.170 --> 00:00:36.430
Let me start by just projecting
a vector b down on a vector a.
00:00:36.430 --> 00:00:41.400
So just to so you see what the
geometry looks like in when
00:00:41.400 --> 00:00:44.060
I'm in -- in just
two dimensions,
00:00:44.060 --> 00:00:51.670
I'd like to find the point along
this line so that line through
00:00:51.670 --> 00:00:55.080
a is a one-dimensional subspace,
so I'm starting with one
00:00:55.080 --> 00:00:56.270
dimension.
00:00:56.270 --> 00:01:00.940
I'd like to find the point
on that line closest to a.
00:01:00.940 --> 00:01:03.390
Can I just take that
problem first and then I'll
00:01:03.390 --> 00:01:06.600
explain why I want to
do it and why I want
00:01:06.600 --> 00:01:09.620
to project on other subspaces.
00:01:09.620 --> 00:01:14.780
So where's the point closest
to b that's on that line?
00:01:14.780 --> 00:01:17.480
It's somewhere there.
00:01:17.480 --> 00:01:24.780
And let me connect that and
-- and what's the whole point
00:01:24.780 --> 00:01:25.800
of my picture now?
00:01:25.800 --> 00:01:29.440
What's the -- where does
orthogonality come into this
00:01:29.440 --> 00:01:30.350
picture?
00:01:30.350 --> 00:01:33.860
The whole point is that
this best point, that's
00:01:33.860 --> 00:01:44.200
the projection, P, of b onto
the line, where's orthogonality?
00:01:44.200 --> 00:01:46.820
It's the fact that
that's a right angle.
00:01:46.820 --> 00:01:50.990
That this -- the error -- this
is like how much I'm wrong
00:01:50.990 --> 00:01:51.940
by --
00:01:51.940 --> 00:01:57.730
this is the difference between
b and P, the whole point is --
00:01:57.730 --> 00:02:01.090
that's perpendicular to a.
00:02:01.090 --> 00:02:03.660
That's got to give
us the equation.
00:02:03.660 --> 00:02:07.370
That's got to tell us --
that's the one fact we know,
00:02:07.370 --> 00:02:11.090
that's got to tell us
where that projection is.
00:02:11.090 --> 00:02:14.200
Let me also say, look --
00:02:14.200 --> 00:02:16.970
I've drawn a triangle there.
00:02:16.970 --> 00:02:19.220
So if we were doing
trigonometry we
00:02:19.220 --> 00:02:25.680
would do like we would have
angles theta and distances that
00:02:25.680 --> 00:02:29.330
would involve sine
theta and cos theta that
00:02:29.330 --> 00:02:33.120
leads to lousy formulas
compared to linear algebra.
00:02:33.120 --> 00:02:40.750
The formula that we want comes
out nicely and what's the --
00:02:40.750 --> 00:02:42.260
what do we know?
00:02:42.260 --> 00:02:49.040
We know that P, this projection,
is some multiple of a, right?
00:02:49.040 --> 00:02:50.690
It's on that line.
00:02:50.690 --> 00:02:54.550
So we know it's in that
one-dimensional subspace, it's
00:02:54.550 --> 00:02:57.870
some multiple, let me
call that multiple x,
00:02:57.870 --> 00:02:59.060
of a.
00:02:59.060 --> 00:03:03.530
So really it's that
number x I'd like to find.
00:03:03.530 --> 00:03:07.500
So this is going to
be simple in 1-D,
00:03:07.500 --> 00:03:11.130
so let's just carry it
through, and then see
00:03:11.130 --> 00:03:13.890
how it goes in high dimensions.
00:03:13.890 --> 00:03:14.510
OK.
00:03:14.510 --> 00:03:17.120
The key fact is --
00:03:17.120 --> 00:03:21.000
the key to everything
is that perpendicular.
00:03:21.000 --> 00:03:31.150
The fact that a is perpendicular
to a is perpendicular to e.
00:03:31.150 --> 00:03:34.470
Which is (b-ax), xa.
00:03:34.470 --> 00:03:36.510
I don't care what --
00:03:36.510 --> 00:03:37.010
xa.
00:03:37.010 --> 00:03:41.950
That that equals zero.
00:03:41.950 --> 00:03:44.010
Do you see that as
the central equation,
00:03:44.010 --> 00:03:50.100
that's saying that this a
is perpendicular to this --
00:03:50.100 --> 00:03:53.700
correction, that's going
to tell us what x is.
00:03:53.700 --> 00:03:57.890
Let me just raise the board
and simplify that and out will
00:03:57.890 --> 00:03:59.790
come x.
00:03:59.790 --> 00:04:00.290
OK.
00:04:00.290 --> 00:04:06.540
So if I simplify that, let's
see, I'll move one to --
00:04:06.540 --> 00:04:09.580
one term to one side, the other
term will be on the other side,
00:04:09.580 --> 00:04:14.690
it looks to me like
x times a transpose a
00:04:14.690 --> 00:04:18.560
is equal to a transpose b.
00:04:21.370 --> 00:04:22.210
Right?
00:04:22.210 --> 00:04:25.240
I have a transpose b
as one f- one term,
00:04:25.240 --> 00:04:27.510
a transpose a as
the other, so right
00:04:27.510 --> 00:04:30.590
away here's my a transpose a.
00:04:30.590 --> 00:04:32.900
But it's just a number now.
00:04:32.900 --> 00:04:35.400
And I divide by it.
00:04:35.400 --> 00:04:42.200
And I get the answer. x is a
transpose b over a transpose a.
00:04:45.220 --> 00:04:48.320
And P, the projection
I wanted, is --
00:04:48.320 --> 00:04:50.530
that's the right multiple.
00:04:50.530 --> 00:04:52.720
That's got a cosine
theta built in.
00:04:52.720 --> 00:04:55.560
But we don't need
to look at angles.
00:04:55.560 --> 00:04:58.040
It's -- we've just
got vectors here.
00:04:58.040 --> 00:05:02.140
And the projection is
P is a times that x.
00:05:07.450 --> 00:05:08.349
Or x times that a.
00:05:08.349 --> 00:05:10.640
But I'm really going to --
eventually I'm going to want
00:05:10.640 --> 00:05:13.380
that x coming on
the right-hand side.
00:05:13.380 --> 00:05:20.630
So do you see that I've got two
of the three formulas already,
00:05:20.630 --> 00:05:24.750
right here, I've got the --
00:05:24.750 --> 00:05:29.140
that's the equation -- that
leads me to the answer,
00:05:29.140 --> 00:05:34.230
here's the answer for x,
and here's the projection.
00:05:34.230 --> 00:05:34.870
OK.
00:05:34.870 --> 00:05:37.520
can I do add just
one more thing to
00:05:37.520 --> 00:05:39.880
this one-dimensional problem?
00:05:39.880 --> 00:05:45.150
One more like lift it up into
linear algebra, into matrices.
00:05:48.460 --> 00:05:50.260
Here's the last
thing I want to do --
00:05:50.260 --> 00:05:52.120
but don't forget those formulas.
00:05:52.120 --> 00:05:54.760
a transpose b over a transpose
00:05:54.760 --> 00:05:55.680
a.
00:05:55.680 --> 00:06:06.020
Actually let's look at
that for a moment first.
00:06:06.020 --> 00:06:09.390
Suppose -- Let me
take this next step.
00:06:09.390 --> 00:06:11.490
So P is a times x.
00:06:11.490 --> 00:06:13.170
So can I write that then?
00:06:13.170 --> 00:06:20.800
P is a times this neat number, a
transpose b over a transpose a.
00:06:24.130 --> 00:06:26.690
That's our projection.
00:06:26.690 --> 00:06:30.060
Can I ask a couple of questions
about it, just while we look,
00:06:30.060 --> 00:06:32.510
get that digest that formula.
00:06:32.510 --> 00:06:37.020
Suppose b is doubled.
00:06:37.020 --> 00:06:41.600
Suppose I change b to two b.
00:06:41.600 --> 00:06:42.935
What happens to the projection?
00:06:46.200 --> 00:06:48.950
So suppose I instead of
that vector b that I drew
00:06:48.950 --> 00:06:51.920
on the board make it
two b, twice as long --
00:06:51.920 --> 00:06:55.540
what's the projection now?
00:06:55.540 --> 00:06:57.550
It's doubled too, right?
00:06:57.550 --> 00:07:01.060
It's going to be twice as
far, if b goes twice as far,
00:07:01.060 --> 00:07:03.030
the projection will
go twice as far.
00:07:03.030 --> 00:07:04.160
And you see it there.
00:07:04.160 --> 00:07:09.180
If I put in an extra factor
two, then P's got that factor
00:07:09.180 --> 00:07:09.840
too.
00:07:09.840 --> 00:07:14.040
Now what about if I double a?
00:07:14.040 --> 00:07:19.850
What if I double the vector
a that I'm projecting onto?
00:07:19.850 --> 00:07:23.220
What changes?
00:07:23.220 --> 00:07:25.320
The projection
doesn't change at all.
00:07:25.320 --> 00:07:25.820
Right?
00:07:25.820 --> 00:07:28.890
Because I'm just --
the line didn't change.
00:07:28.890 --> 00:07:31.620
If I double a or I take minus a.
00:07:31.620 --> 00:07:33.430
It's still that same line.
00:07:33.430 --> 00:07:35.310
The projection's still
in the same place.
00:07:35.310 --> 00:07:40.540
And of course if I double
a I get a four up above,
00:07:40.540 --> 00:07:43.600
and I get a four -- an extra
four below, they cancel out,
00:07:43.600 --> 00:07:45.580
and the projection is the same.
00:07:45.580 --> 00:07:46.350
OK.
00:07:46.350 --> 00:07:48.970
So really, this --
00:07:48.970 --> 00:07:54.750
I want to look at this
as the projection --
00:07:54.750 --> 00:07:57.630
there's a matrix here.
00:07:57.630 --> 00:08:01.960
The projection is carried
out by some matrix
00:08:01.960 --> 00:08:05.090
that I'm going to call
the projection matrix.
00:08:05.090 --> 00:08:07.970
And in other words
the projection
00:08:07.970 --> 00:08:12.240
is some matrix that
acts on this guy b
00:08:12.240 --> 00:08:14.100
and produces the projection.
00:08:14.100 --> 00:08:21.270
The projection P is the
projection matrix acting
00:08:21.270 --> 00:08:24.230
on whatever the input is.
00:08:24.230 --> 00:08:28.130
The input is b, the
projection matrix is P. OK.
00:08:28.130 --> 00:08:30.850
Actually you can tell
me right away what
00:08:30.850 --> 00:08:31.970
this projection matrix
00:08:31.970 --> 00:08:32.470
is.
00:08:32.470 --> 00:08:37.530
So this is a pretty
interesting matrix.
00:08:37.530 --> 00:08:40.650
What matrix is multiplying b?
00:08:40.650 --> 00:08:42.690
I'm just -- just
from my formula --
00:08:42.690 --> 00:08:44.120
I see what P is.
00:08:44.120 --> 00:08:51.650
P, this projection matrix, is --
00:08:51.650 --> 00:08:52.360
what is it?
00:08:52.360 --> 00:08:59.515
I see a a transpose above,
and I see a transpose a below.
00:09:02.470 --> 00:09:04.560
And those don't cancel.
00:09:04.560 --> 00:09:06.620
That's not one.
00:09:06.620 --> 00:09:07.260
Right?
00:09:07.260 --> 00:09:08.230
That's a matrix.
00:09:08.230 --> 00:09:10.580
Because down here,
the a transpose a,
00:09:10.580 --> 00:09:13.690
that's just a number,
a transpose a, that's
00:09:13.690 --> 00:09:18.060
the length of a squared, and
up above is a column times
00:09:18.060 --> 00:09:19.040
a row.
00:09:19.040 --> 00:09:22.050
Column times a row is a matrix.
00:09:22.050 --> 00:09:26.450
So this is a full-scale
n by n matrix, if I --
00:09:26.450 --> 00:09:28.880
if I'm in n dimensions.
00:09:28.880 --> 00:09:32.270
And it's kind of
an interesting one.
00:09:32.270 --> 00:09:36.510
And it's the one which if I
multiply by b then I get this,
00:09:36.510 --> 00:09:39.750
you see once again I'm
putting parentheses
00:09:39.750 --> 00:09:41.140
in different places.
00:09:41.140 --> 00:09:46.480
I'm putting the
parentheses right there.
00:09:46.480 --> 00:09:49.280
I'm saying OK, that's
really the matrix
00:09:49.280 --> 00:09:55.990
that produces this projection.
00:09:55.990 --> 00:09:56.980
OK.
00:09:56.980 --> 00:10:00.280
Now, tell me --
00:10:00.280 --> 00:10:02.910
all right, what are the
properties of that matrix?
00:10:05.420 --> 00:10:08.900
I'm just using letters here, a
and b, I could put in numbers,
00:10:08.900 --> 00:10:12.100
but I think it's -- for once,
it's clearer with letters,
00:10:12.100 --> 00:10:16.790
because all formulas are simple,
a transpose b over a transpose
00:10:16.790 --> 00:10:21.770
a -- that's the number that
multiplies the a, and then I
00:10:21.770 --> 00:10:26.160
see wait a minute, there's a
matrix here and what's the rank
00:10:26.160 --> 00:10:29.260
of that matrix, by the way?
00:10:29.260 --> 00:10:31.360
What's the rank of
that matrix, yeah --
00:10:31.360 --> 00:10:33.260
let me just ask you
about that matrix.
00:10:33.260 --> 00:10:38.250
Which looks a little strange,
a a transpose over this number.
00:10:38.250 --> 00:10:46.180
But well, I could ask
you its column space.
00:10:46.180 --> 00:10:48.920
Yeah, let me ask you
its column space.
00:10:48.920 --> 00:10:52.180
So what's the column
space of a matrix?
00:10:52.180 --> 00:10:56.460
If you multiply that
matrix by anything
00:10:56.460 --> 00:10:59.810
you always get in the
column space, right?
00:10:59.810 --> 00:11:04.900
The column space of a matrix
is when you multiply any vector
00:11:04.900 --> 00:11:09.900
by that matrix -- any
vector b, by the matrix,
00:11:09.900 --> 00:11:13.360
you always land in
the column space.
00:11:13.360 --> 00:11:16.320
That's what column spaces work.
00:11:16.320 --> 00:11:20.060
Now what space do
we always land in?
00:11:20.060 --> 00:11:22.110
What's the column space of --
00:11:22.110 --> 00:11:26.160
what's the result when I
multiply this any vector
00:11:26.160 --> 00:11:29.550
b by my matrix?
00:11:29.550 --> 00:11:34.710
So I have P times b, where I?
00:11:34.710 --> 00:11:36.170
I'm on that line, right?
00:11:36.170 --> 00:11:40.610
The column space, so here
are facts about this matrix.
00:11:40.610 --> 00:11:46.220
The column space of P, of
this projection matrix,
00:11:46.220 --> 00:11:49.190
is the line through a.
00:11:52.620 --> 00:12:03.720
And the rank of this matrix is
you can all say it at once one.
00:12:03.720 --> 00:12:04.520
Right.
00:12:04.520 --> 00:12:06.590
The rank is one.
00:12:06.590 --> 00:12:08.300
This is a rank one matrix.
00:12:08.300 --> 00:12:10.960
Actually it's exactly
the form that we're
00:12:10.960 --> 00:12:15.130
familiar with a rank one matrix.
00:12:15.130 --> 00:12:20.840
A column times a row, that's
a rank one matrix, that column
00:12:20.840 --> 00:12:27.400
is the basis for
the column space.
00:12:27.400 --> 00:12:28.770
Just one dimension.
00:12:28.770 --> 00:12:29.550
OK.
00:12:29.550 --> 00:12:32.560
So I know that much
about the matrix.
00:12:32.560 --> 00:12:35.490
But now there are two more
facts about the matrix
00:12:35.490 --> 00:12:40.230
that I want to notice.
00:12:40.230 --> 00:12:42.980
First of all is the
matrix symmetric?
00:12:42.980 --> 00:12:45.580
That's a natural
question for matrices.
00:12:45.580 --> 00:12:48.380
And the answer is yes.
00:12:48.380 --> 00:12:52.620
If I take the transpose of this
-- there's a number down there,
00:12:52.620 --> 00:12:57.300
the transpose of a a
transpose is a a transpose.
00:12:57.300 --> 00:12:59.200
So P is symmetric.
00:12:59.200 --> 00:13:04.500
P transpose equals P. So
this is a key property.
00:13:04.500 --> 00:13:07.730
That the projection
matrix is symmetric.
00:13:07.730 --> 00:13:12.430
One more property now
and this is the real one.
00:13:12.430 --> 00:13:18.570
What happens if I do
the projection twice?
00:13:18.570 --> 00:13:21.740
So I'm looking for
something, some information
00:13:21.740 --> 00:13:24.220
about P squared.
00:13:24.220 --> 00:13:27.530
But just give me in terms
of that picture, in terms
00:13:27.530 --> 00:13:33.940
my picture, I take any
vector b, I multiply it
00:13:33.940 --> 00:13:36.280
by my projection
matrix, and that
00:13:36.280 --> 00:13:41.010
puts me there, so this is Pb.
00:13:41.010 --> 00:13:45.170
And now I project again.
00:13:45.170 --> 00:13:46.790
What happens now?
00:13:46.790 --> 00:13:49.620
What happens when I apply
the projection matrix
00:13:49.620 --> 00:13:51.880
a second time?
00:13:51.880 --> 00:13:56.160
To this, so I'm applying
it once brings me here
00:13:56.160 --> 00:14:01.420
and the second time
brings me I stay put.
00:14:01.420 --> 00:14:02.170
Right?
00:14:02.170 --> 00:14:05.490
The projection for a point
on this line the projection
00:14:05.490 --> 00:14:08.530
is right where it is.
00:14:08.530 --> 00:14:10.610
The projection is
the same point.
00:14:10.610 --> 00:14:16.340
So that means that
if I project twice,
00:14:16.340 --> 00:14:21.430
I get the same answer as I
did in the first projection.
00:14:21.430 --> 00:14:26.510
So those are the
two properties that
00:14:26.510 --> 00:14:30.630
tell me I'm looking at
a projection matrix.
00:14:30.630 --> 00:14:35.070
It's symmetric and
it's square is itself.
00:14:35.070 --> 00:14:37.500
Because if I project
a second time,
00:14:37.500 --> 00:14:40.740
it's the same result
as the first result.
00:14:40.740 --> 00:14:41.940
OK.
00:14:41.940 --> 00:14:49.120
So that's -- and then here's the
exact formula when I know what
00:14:49.120 --> 00:14:53.450
I'm projecting onto, that line
through a, then I know what P
00:14:53.450 --> 00:14:54.460
is.
00:14:54.460 --> 00:14:57.070
So do you see that I
have all the pieces here
00:14:57.070 --> 00:14:59.990
for projection on a line?
00:14:59.990 --> 00:15:04.880
Now, and those --
please remember those.
00:15:04.880 --> 00:15:07.330
So there are three
formulas to remember.
00:15:07.330 --> 00:15:11.650
The formula for x, the formula
for P, which is just ax,
00:15:11.650 --> 00:15:17.680
and then the formula for
capital P, which is the matrix.
00:15:17.680 --> 00:15:18.430
Good.
00:15:18.430 --> 00:15:18.970
Good.
00:15:18.970 --> 00:15:20.400
OK.
00:15:20.400 --> 00:15:26.780
Now I want to move
to more dimensions.
00:15:26.780 --> 00:15:30.610
So we're going to have three
formulas again but you'll have
00:15:30.610 --> 00:15:32.170
to --
00:15:32.170 --> 00:15:35.830
they'll be a little different
because we won't have a single
00:15:35.830 --> 00:15:40.990
line but -- a plane
or three-dimensional
00:15:40.990 --> 00:15:44.530
or a n-dimensional subspace.
00:15:44.530 --> 00:15:45.230
OK.
00:15:45.230 --> 00:15:49.960
So now I'll move to
the next question.
00:15:49.960 --> 00:16:04.020
Maybe -- let me say first
why I want this projection,
00:16:04.020 --> 00:16:06.200
and then we'll figure
out what it is,
00:16:06.200 --> 00:16:11.040
we'll go completely in parallel
there, and then we'll use it.
00:16:11.040 --> 00:16:14.420
OK, why do I want
this projection?
00:16:14.420 --> 00:16:17.083
Well, so why project?
00:16:21.600 --> 00:16:32.010
It's because I'm as I mentioned
last time this new chapter
00:16:32.010 --> 00:16:47.890
deals with equations Ax=b
may have no solution.
00:16:54.540 --> 00:16:57.380
So that's really my
problem, that I'm
00:16:57.380 --> 00:17:00.240
given a bunch of
equations probably too
00:17:00.240 --> 00:17:04.450
many equations, more
equations than unknowns,
00:17:04.450 --> 00:17:08.119
and I can't solve them.
00:17:08.119 --> 00:17:09.079
OK.
00:17:09.079 --> 00:17:11.050
So what do I do?
00:17:11.050 --> 00:17:15.359
I solve the closest
problem that I can solve.
00:17:15.359 --> 00:17:17.730
And what's the closest one?
00:17:17.730 --> 00:17:21.460
Well, ax will always be
in the column space of a.
00:17:21.460 --> 00:17:22.970
That's my problem.
00:17:22.970 --> 00:17:27.190
My problem is ax has to
be in the column space
00:17:27.190 --> 00:17:31.130
and b is probably not
in the column space.
00:17:31.130 --> 00:17:36.620
So I change b to what?
00:17:36.620 --> 00:17:40.820
I choose the closest
vector in the column space,
00:17:40.820 --> 00:17:50.350
so I'll solve Ax
equal P instead.
00:17:50.350 --> 00:17:52.660
That one I can do.
00:17:52.660 --> 00:17:57.220
Where P is this
is the projection
00:17:57.220 --> 00:18:04.460
of b onto the column space.
00:18:04.460 --> 00:18:07.010
That's why I want to
be able to do this.
00:18:07.010 --> 00:18:10.460
Because I have to
find a solution here,
00:18:10.460 --> 00:18:13.370
and I'm going to put
a little hat there
00:18:13.370 --> 00:18:17.400
to indicate that
it's not the x, it's
00:18:17.400 --> 00:18:21.660
not the x that
doesn't exist, it's
00:18:21.660 --> 00:18:27.000
the x hat that's best possible.
00:18:27.000 --> 00:18:32.170
So I must be able
to figure out what's
00:18:32.170 --> 00:18:35.310
the good projection there.
00:18:35.310 --> 00:18:38.600
What's the good right-hand side
that is in the column space
00:18:38.600 --> 00:18:42.330
that's as close as possible
to b and then I'm --
00:18:42.330 --> 00:18:43.820
then I know what to do.
00:18:43.820 --> 00:18:44.620
OK.
00:18:44.620 --> 00:18:47.500
So now I've got that problem.
00:18:47.500 --> 00:18:50.190
So that's why I have
the problem again
00:18:50.190 --> 00:18:54.050
but now let me say I'm
in three dimensions,
00:18:54.050 --> 00:18:58.630
so I have a plane
maybe for example,
00:18:58.630 --> 00:19:06.390
and I have a vector b
that's not in the plane.
00:19:06.390 --> 00:19:10.840
And I want to project
b down into the plane.
00:19:13.420 --> 00:19:15.510
OK.
00:19:15.510 --> 00:19:17.540
So there's my question.
00:19:17.540 --> 00:19:19.890
How do I project a
vector and I'm --
00:19:19.890 --> 00:19:22.670
what I'm looking for
is a nice formula,
00:19:22.670 --> 00:19:28.120
and I'm counting on linear
algebra to just come out right,
00:19:28.120 --> 00:19:34.940
a nice formula for the
projection of b into the plane.
00:19:34.940 --> 00:19:36.310
The nearest point.
00:19:36.310 --> 00:19:41.990
So this again a right angle
is going to be crucial.
00:19:41.990 --> 00:19:42.570
OK.
00:19:42.570 --> 00:19:46.630
Now so what's -- first of all
I've got to say what is that
00:19:46.630 --> 00:19:48.100
plane.
00:19:48.100 --> 00:19:51.460
To get a formula I have to
tell you what the plane is.
00:19:51.460 --> 00:19:53.990
How I going to tell you a plane?
00:19:53.990 --> 00:19:56.580
I'll tell you a
basis for the plane,
00:19:56.580 --> 00:20:03.476
I'll tell you two vectors a one
and a two that give you a basis
00:20:03.476 --> 00:20:05.100
for the plane, so
that -- let us say --
00:20:05.100 --> 00:20:12.510
say there's an a one and
here's an a -- a vector a two.
00:20:12.510 --> 00:20:15.690
They don't have to
be perpendicular.
00:20:15.690 --> 00:20:18.921
But they better be independent,
because then that tells me the
00:20:18.921 --> 00:20:19.420
plane.
00:20:19.420 --> 00:20:24.880
The plane is the -- is the
plane of a one and a two.
00:20:31.760 --> 00:20:34.320
And actually going back to my --
00:20:34.320 --> 00:20:39.620
to this connection, this
plane is a column space,
00:20:39.620 --> 00:20:47.990
it's the column
space of what matrix?
00:20:47.990 --> 00:20:56.610
What matrix, so how do I
connect the two questions?
00:20:56.610 --> 00:21:01.220
I'm thinking how do I
project onto a plane
00:21:01.220 --> 00:21:06.030
and I want to get
a matrix in here.
00:21:06.030 --> 00:21:09.760
Everything's cleaner if I
write it in terms of a matrix.
00:21:09.760 --> 00:21:14.680
So what matrix has these
-- has that column space?
00:21:14.680 --> 00:21:17.270
Well of course it's
just the matrix
00:21:17.270 --> 00:21:20.240
that has a one in
the first column
00:21:20.240 --> 00:21:23.790
and a two in the second column.
00:21:23.790 --> 00:21:27.820
Right, just just let's be
sure we've got the question
00:21:27.820 --> 00:21:30.060
before we get to the answer.
00:21:30.060 --> 00:21:34.290
So I'm looking for
again I'm given
00:21:34.290 --> 00:21:38.140
a matrix a with two columns.
00:21:38.140 --> 00:21:44.110
And really I'm ready once I
get to two I'm ready for n.
00:21:44.110 --> 00:21:47.610
So it could be two columns,
it could be n columns.
00:21:47.610 --> 00:21:50.930
I'll write the answer in
terms of the matrix a.
00:21:50.930 --> 00:21:56.590
And the point will be those
two columns describe the plane,
00:21:56.590 --> 00:22:01.331
they describe the column
space, and I want to project.
00:22:01.331 --> 00:22:01.830
OK.
00:22:01.830 --> 00:22:06.190
And I'm given a vector b that's
probably not in the column
00:22:06.190 --> 00:22:06.770
space.
00:22:06.770 --> 00:22:11.280
Of course, if b is in the column
space, my projection is simple,
00:22:11.280 --> 00:22:13.420
it's just b.
00:22:13.420 --> 00:22:19.680
But most likely I have an
error e, this b minus P
00:22:19.680 --> 00:22:24.500
part, which is
probably not zero.
00:22:24.500 --> 00:22:25.560
OK.
00:22:25.560 --> 00:22:30.330
But the beauty is that I know --
00:22:30.330 --> 00:22:35.880
from geometry or I could get it
from calculus or I could get it
00:22:35.880 --> 00:22:38.724
from linear algebra that
that this this vector --
00:22:42.600 --> 00:22:45.830
this is the part
of b that's that's
00:22:45.830 --> 00:22:48.600
perpendicular to the plane.
00:22:48.600 --> 00:22:55.390
That e is perpendicular is
perpendicular to the plane.
00:22:58.030 --> 00:23:03.530
If your intuition is saying
that that's the crucial fact.
00:23:03.530 --> 00:23:06.900
That's going to
give us the answer.
00:23:06.900 --> 00:23:07.540
OK.
00:23:07.540 --> 00:23:10.320
So let me, that's the problem.
00:23:10.320 --> 00:23:14.550
Now for the answer.
00:23:14.550 --> 00:23:18.910
So this is a lecture that's
really like moving along.
00:23:18.910 --> 00:23:24.940
Because I'm just plotting that
problem up there and asking you
00:23:24.940 --> 00:23:27.130
what combination --
00:23:27.130 --> 00:23:29.920
now, yeah, so what is it?
00:23:29.920 --> 00:23:31.110
What is this projection P?
00:23:31.110 --> 00:23:42.300
P. This is projection P, is
some multiple of these basis
00:23:42.300 --> 00:23:46.740
guys, right, some
multiple of the columns.
00:23:46.740 --> 00:23:51.700
But I don't like writing out
x one a one plus x two a two,
00:23:51.700 --> 00:23:54.265
I would rather right that as ax.
00:23:56.790 --> 00:24:00.512
Well, actually I should put
if I'm really doing everything
00:24:00.512 --> 00:24:02.220
right, I should put
a little hat on it --
00:24:02.220 --> 00:24:05.500
to remember that this x --
00:24:05.500 --> 00:24:08.190
that those are the numbers
and I could have a put
00:24:08.190 --> 00:24:14.470
a hat way back there
is right, so this
00:24:14.470 --> 00:24:20.360
is this is the projection,
P. P is ax bar.
00:24:20.360 --> 00:24:22.830
And I'm looking for x bar.
00:24:22.830 --> 00:24:25.195
So that's what I
want an equation for.
00:24:28.410 --> 00:24:31.630
So now I've got
hold of the problem.
00:24:31.630 --> 00:24:36.330
The problem is find the right
combination of the columns
00:24:36.330 --> 00:24:44.070
so that the error vector is
perpendicular to the plane.
00:24:44.070 --> 00:24:48.840
Now let me turn that
into an equation.
00:24:48.840 --> 00:24:53.250
So I'll raise the board
and just turn that --
00:24:53.250 --> 00:24:54.910
what we've just done
into an equation.
00:24:58.490 --> 00:25:02.020
So let me I'll write
again the main point.
00:25:02.020 --> 00:25:06.600
The projection is ax b- x hat.
00:25:06.600 --> 00:25:08.770
And our problem is find x hat.
00:25:12.670 --> 00:25:23.060
And the key is that b minus
ax hat, that's the error.
00:25:23.060 --> 00:25:24.800
This is the e.
00:25:24.800 --> 00:25:30.390
Is perpendicular to the plane.
00:25:35.050 --> 00:25:38.490
That's got to give me
well what I looking
00:25:38.490 --> 00:25:40.720
for, I'm looking for
two equations now
00:25:40.720 --> 00:25:42.845
because I've got an
x one and an x two.
00:25:45.640 --> 00:25:49.910
And I'll get two equations
because so this thing e
00:25:49.910 --> 00:25:51.680
is perpendicular to the plane.
00:25:55.470 --> 00:25:56.650
So what does that mean?
00:25:56.650 --> 00:26:00.060
I guess it means it's
perpendicular to a one
00:26:00.060 --> 00:26:02.540
and also to a two.
00:26:02.540 --> 00:26:06.320
Right, those are two vectors
in the plane and the things
00:26:06.320 --> 00:26:08.240
that are perpendicular
to the plane
00:26:08.240 --> 00:26:10.810
are perpendicular
to a one and a two.
00:26:10.810 --> 00:26:12.350
Let me just repeat.
00:26:12.350 --> 00:26:15.480
This this guy then is
perpendicular to the plane
00:26:15.480 --> 00:26:18.460
so it's perpendicular to
that vector and that vector.
00:26:18.460 --> 00:26:22.000
Not -- it's perpendicular
to that of course.
00:26:22.000 --> 00:26:25.940
But it's perpendicular to
everything I the plane.
00:26:25.940 --> 00:26:31.500
And the plane is really
told me by a one and a two.
00:26:31.500 --> 00:26:39.150
So really I have the equations
a one transpose b minus ax
00:26:39.150 --> 00:26:41.050
is zero.
00:26:41.050 --> 00:26:48.130
And also a two transpose
b minus ax is zero.
00:26:52.321 --> 00:26:53.445
Those are my two equations.
00:26:56.950 --> 00:27:02.410
But I want those in matrix form.
00:27:02.410 --> 00:27:04.430
I want to put those
two equations together
00:27:04.430 --> 00:27:08.600
as a matrix equation and
it's just comes out right.
00:27:08.600 --> 00:27:12.100
Look at the matrix a transpose.
00:27:12.100 --> 00:27:16.050
Put a one a one transpose
is its first row,
00:27:16.050 --> 00:27:25.770
a two transpose is its second
row, that multiplies this b-ax,
00:27:25.770 --> 00:27:28.440
and gives me the
zero and the zero.
00:27:28.440 --> 00:27:29.410
I'm you see the --
00:27:39.220 --> 00:27:43.050
this is one way -- to come
up with this equation.
00:27:43.050 --> 00:27:44.600
So the equation
I'm coming up with
00:27:44.600 --> 00:27:52.550
is a transpose b-ax hat is zero.
00:27:55.250 --> 00:27:57.520
OK.
00:27:57.520 --> 00:27:59.791
That's my equation.
00:27:59.791 --> 00:28:00.290
All right.
00:28:00.290 --> 00:28:03.790
Now I want to stop for a
moment before I solve it
00:28:03.790 --> 00:28:06.280
and just think about it.
00:28:06.280 --> 00:28:13.230
First of all do you see that
that equation back in the very
00:28:13.230 --> 00:28:17.440
first problem I solved
on a line, what was --
00:28:17.440 --> 00:28:23.610
what was on a line the
matrix a only had one column,
00:28:23.610 --> 00:28:27.210
it was just little a.
00:28:27.210 --> 00:28:29.980
So in the first
problem I solved,
00:28:29.980 --> 00:28:33.680
projecting on a line,
this for capital
00:28:33.680 --> 00:28:35.510
a you just change
that to little a
00:28:35.510 --> 00:28:39.160
and you have the same equation
that we solved before.
00:28:39.160 --> 00:28:42.670
a transpose e equals zero.
00:28:42.670 --> 00:28:43.300
OK.
00:28:43.300 --> 00:28:48.500
Now a second thing,
second comment.
00:28:48.500 --> 00:28:54.220
I would like to since I know
about these four subspaces,
00:28:54.220 --> 00:28:57.720
I would like to get
them into this picture.
00:29:00.890 --> 00:29:07.270
So let me ask the question,
what subspace is this thing in?
00:29:07.270 --> 00:29:11.330
Which of the four subspaces
is that error vector e,
00:29:11.330 --> 00:29:13.920
this is this is nothing but e --
00:29:13.920 --> 00:29:21.204
this is this guy, coming in
down perpendicular to the plane.
00:29:21.204 --> 00:29:22.120
What subspace is e in?
00:29:22.120 --> 00:29:22.911
From this equation.
00:29:22.911 --> 00:29:34.470
Well the equation is saying
a transpose e is zero.
00:29:34.470 --> 00:29:39.630
So I'm learning here that
e is in the null space
00:29:39.630 --> 00:29:40.330
of a transpose.
00:29:44.660 --> 00:29:45.360
Right?
00:29:45.360 --> 00:29:47.000
That's my equation.
00:29:47.000 --> 00:29:51.300
And now I just want to see hey
of course that that was right.
00:29:51.300 --> 00:29:57.320
Because things that are in
the null space of a transpose,
00:29:57.320 --> 00:30:00.310
what do we know about the
null space of a transpose?
00:30:03.170 --> 00:30:06.320
So that last lecture
gave us the sort
00:30:06.320 --> 00:30:09.210
of the geometry of
these subspaces.
00:30:09.210 --> 00:30:11.200
And the orthogonality of them.
00:30:11.200 --> 00:30:12.950
And do you remember what it was?
00:30:12.950 --> 00:30:16.910
What on the right
side of our big figure
00:30:16.910 --> 00:30:20.200
we always have the null
space of a transpose
00:30:20.200 --> 00:30:27.270
and the column space of
a, and they're orthogonal.
00:30:27.270 --> 00:30:31.750
So e in the null
space of a transpose
00:30:31.750 --> 00:30:38.730
is saying e is perpendicular
to the column space of a.
00:30:38.730 --> 00:30:39.230
Yes.
00:30:44.850 --> 00:30:49.100
I just feel OK, the damn
thing came out right.
00:30:49.100 --> 00:30:58.140
The equation for the equation
that I struggled to find for e
00:30:58.140 --> 00:31:03.980
really said what I
wanted, that the error
00:31:03.980 --> 00:31:08.290
e is perpendicular to the
column space of a, just right.
00:31:08.290 --> 00:31:10.710
And from our four
fundamental subspaces
00:31:10.710 --> 00:31:14.780
we knew that that
is the same as that.
00:31:14.780 --> 00:31:17.530
To say e is in the null
space of a transpose says
00:31:17.530 --> 00:31:19.451
e's perpendicular
to the column space.
00:31:19.451 --> 00:31:19.950
OK.
00:31:19.950 --> 00:31:21.900
So we've got this equation.
00:31:21.900 --> 00:31:23.360
Now let's just solve it.
00:31:23.360 --> 00:31:23.860
All right.
00:31:23.860 --> 00:31:28.130
Let me just rewrite
it as a transpose
00:31:28.130 --> 00:31:34.100
a x hat equals a transpose b.
00:31:34.100 --> 00:31:37.700
That's our equation.
00:31:37.700 --> 00:31:42.500
That gives us x.
00:31:42.500 --> 00:31:48.140
And -- allow me to keep
remembering the one-dimensional
00:31:48.140 --> 00:31:48.970
case.
00:31:48.970 --> 00:31:54.260
The one-dimensional
case, this was little a.
00:31:54.260 --> 00:31:56.620
So this was just a number,
little a transpose,
00:31:56.620 --> 00:32:02.910
a transpose a was just a vector
row times a column, a number.
00:32:02.910 --> 00:32:04.560
And this was a number.
00:32:04.560 --> 00:32:07.480
And x was the ratio
of those numbers.
00:32:07.480 --> 00:32:12.350
But now we've got matrices,
this one is n by n.
00:32:12.350 --> 00:32:15.211
a transpose a is
an n by n matrix.
00:32:15.211 --> 00:32:15.710
OK.
00:32:15.710 --> 00:32:20.260
So can I move to the next
board for the solution?
00:32:26.260 --> 00:32:27.470
OK.
00:32:27.470 --> 00:32:30.900
This is the -- the key equation.
00:32:30.900 --> 00:32:35.160
Now I'm ready for the formulas
that we have to remember.
00:32:35.160 --> 00:32:38.010
What's x hat?
00:32:38.010 --> 00:32:41.570
What's the projection,
what's the projection matrix,
00:32:41.570 --> 00:32:43.620
those are my three questions.
00:32:43.620 --> 00:32:46.050
That we answered in
the 1-D case and now
00:32:46.050 --> 00:32:49.720
we're ready for in the
n-dimensional case.
00:32:49.720 --> 00:32:50.920
So what is x hat?
00:32:50.920 --> 00:32:55.930
Well, what can I
say but a transpose
00:32:55.930 --> 00:33:01.075
a inverse, a transpose b.
00:33:03.750 --> 00:33:07.840
That's the solution
to -- to our equation.
00:33:07.840 --> 00:33:08.890
OK.
00:33:08.890 --> 00:33:09.940
What's the projection?
00:33:09.940 --> 00:33:11.430
That's more interesting.
00:33:11.430 --> 00:33:13.500
What's the projection?
00:33:13.500 --> 00:33:18.520
The projection is a x hat.
00:33:18.520 --> 00:33:21.700
That's how x hat got into the
picture in the first place.
00:33:21.700 --> 00:33:28.050
x hat was the was the
combination of columns
00:33:28.050 --> 00:33:31.440
in the I had to look for those
numbers and now I found them.
00:33:31.440 --> 00:33:34.020
Was the combination
of the columns of a
00:33:34.020 --> 00:33:35.510
that gave me the projection.
00:33:35.510 --> 00:33:37.060
OK.
00:33:37.060 --> 00:33:40.000
So now I know what this guy is.
00:33:40.000 --> 00:33:42.260
So it's just I multiply by a.
00:33:42.260 --> 00:33:50.110
a a transpose a
inverse a transpose b.
00:33:56.010 --> 00:34:00.145
That's looking a little
messy but it's not bad.
00:34:03.270 --> 00:34:07.600
That that combination is is
our like magic combination.
00:34:07.600 --> 00:34:14.860
This is the thing which is which
use which is like what's it
00:34:14.860 --> 00:34:16.504
like, what was it
in one dimension?
00:34:19.130 --> 00:34:20.820
What was that we
had this we must
00:34:20.820 --> 00:34:25.139
have had this thing way back at
the beginning of the lecture.
00:34:25.139 --> 00:34:29.929
What did we -- oh that a was
just a column so it was little
00:34:29.929 --> 00:34:36.210
a, little a transpose
over a transpose a, right,
00:34:36.210 --> 00:34:46.110
that's what it was in 1-D. You
see what's happened in more
00:34:46.110 --> 00:34:47.520
dimensions, I --
00:34:47.520 --> 00:34:49.810
I'm not allowed
to to just divide
00:34:49.810 --> 00:34:52.440
because because I don't have a
number, I have to put inverse,
00:34:52.440 --> 00:34:55.320
because I have an n by n matrix.
00:34:55.320 --> 00:34:56.440
But same formula.
00:34:59.120 --> 00:35:03.320
And now tell me what's
the projection matrix?
00:35:03.320 --> 00:35:08.955
What matrix is multiplying
b to give the projection?
00:35:12.300 --> 00:35:13.660
Right there.
00:35:13.660 --> 00:35:15.010
Because there it --
00:35:15.010 --> 00:35:17.390
I even already underlined
it by accident.
00:35:17.390 --> 00:35:24.100
The projection matrix which
I use capital P is this,
00:35:24.100 --> 00:35:28.360
it's it's that thing, shall
I write it again, a times
00:35:28.360 --> 00:35:32.770
a transpose a inverse
times a transpose.
00:35:41.610 --> 00:35:49.080
Now if you'll bear with me I'll
think about what have I done
00:35:49.080 --> 00:35:49.640
here.
00:35:49.640 --> 00:35:53.030
I've got this formula.
00:35:53.030 --> 00:36:00.100
Now the first thing that
occurs to me is something bad.
00:36:00.100 --> 00:36:04.360
Look why don't I
just you know here's
00:36:04.360 --> 00:36:08.100
a product of two matrices
and I want its inverse,
00:36:08.100 --> 00:36:10.460
why don't I just
use the formula I
00:36:10.460 --> 00:36:14.030
know for the inverse of
a product and say OK,
00:36:14.030 --> 00:36:22.110
that's a inverse times a
transpose inverse, what
00:36:22.110 --> 00:36:25.480
will happen if I do that?
00:36:25.480 --> 00:36:29.370
What will happen
if I say hey this
00:36:29.370 --> 00:36:36.060
is a inverse times a transpose
inverse, then shall I do it?
00:36:36.060 --> 00:36:41.100
It's going to go on videotape
if I do it, and I don't --
00:36:41.100 --> 00:36:43.780
all right, I'll put it
there, but just like
00:36:43.780 --> 00:36:48.010
don't take the videotape
quite so carefully.
00:36:48.010 --> 00:36:48.770
OK.
00:36:48.770 --> 00:36:52.440
So if I put that thing it
-- it would be a a inverse
00:36:52.440 --> 00:36:59.387
a transpose inverse a
transpose and what's that?
00:36:59.387 --> 00:37:00.220
That's the identity.
00:37:03.610 --> 00:37:06.660
But what's going on?
00:37:06.660 --> 00:37:14.059
So why -- you see my question is
somehow I did something wrong.
00:37:14.059 --> 00:37:15.100
That that wasn't allowed.
00:37:15.100 --> 00:37:19.100
And and and why is that?
00:37:19.100 --> 00:37:23.530
Because a is not
a square matrix.
00:37:23.530 --> 00:37:25.060
a is not a square matrix.
00:37:25.060 --> 00:37:28.580
It doesn't have an inverse.
00:37:28.580 --> 00:37:31.850
So I have to leave it that way.
00:37:31.850 --> 00:37:33.310
This is not OK.
00:37:33.310 --> 00:37:36.950
If if a was a square
invertible matrix, then then
00:37:36.950 --> 00:37:37.800
I couldn't complain.
00:37:40.980 --> 00:37:44.280
Yeah I think -- let me
think about that case.
00:37:44.280 --> 00:37:46.740
But you but my main
case, the whole reason
00:37:46.740 --> 00:37:51.240
I'm doing all this, is
that a is this matrix that
00:37:51.240 --> 00:37:56.360
has x too many rows, it's
just got a couple of columns,
00:37:56.360 --> 00:38:01.280
like a one and a two,
but lots of rows.
00:38:01.280 --> 00:38:02.390
Not square.
00:38:02.390 --> 00:38:07.810
And if it's not square, this
a transpose a is square but I
00:38:07.810 --> 00:38:10.530
can't pull it apart like this --
00:38:10.530 --> 00:38:16.650
I'm not allowed to do this pull
apart, except if a was square.
00:38:16.650 --> 00:38:19.180
Now if a is square
what's what's going on
00:38:19.180 --> 00:38:20.780
if a is a square matrix?
00:38:20.780 --> 00:38:25.070
a nice square inv-
invertible matrix.
00:38:25.070 --> 00:38:26.290
Think.
00:38:26.290 --> 00:38:29.790
What's up with that
what's with that case.
00:38:29.790 --> 00:38:34.300
So this is that the formula
ought to work then too.
00:38:34.300 --> 00:38:38.910
If a is a nice square invertible
matrix what's its column space,
00:38:38.910 --> 00:38:44.070
so it's a nice n by n invertible
everything great matrix,
00:38:44.070 --> 00:38:50.670
what's its column
space, the whole of R^n.
00:38:50.670 --> 00:38:54.190
So what's the projection
matrix if I'm projecting
00:38:54.190 --> 00:38:56.900
onto the whole space?
00:38:56.900 --> 00:39:00.510
It's the identity matrix right?
00:39:00.510 --> 00:39:04.800
If I'm projecting b
onto the whole space,
00:39:04.800 --> 00:39:08.090
not just onto a plane,
but onto all of 3-D,
00:39:08.090 --> 00:39:11.550
then b is already
in the column space,
00:39:11.550 --> 00:39:15.930
the projection is the
identity, and this is gives me
00:39:15.930 --> 00:39:18.720
the correct formula, P is I.
00:39:18.720 --> 00:39:23.000
But if I'm projecting
onto a subspace then
00:39:23.000 --> 00:39:27.310
I can't split those apart and I
have to stay with that formula.
00:39:27.310 --> 00:39:30.040
OK.
00:39:30.040 --> 00:39:37.240
And what can I say if -- so I
remember this formula for 1-D
00:39:37.240 --> 00:39:40.760
and that's what it looks
like in n dimensions.
00:39:40.760 --> 00:39:44.350
And what are the properties that
I expected for any projection
00:39:44.350 --> 00:39:44.970
matrix?
00:39:44.970 --> 00:39:47.370
And I still expect for this one?
00:39:47.370 --> 00:39:50.170
That matrix should be
symmetric and it is.
00:39:50.170 --> 00:39:55.160
P transpose is P. Because
if I transpose this,
00:39:55.160 --> 00:39:59.280
this guy's symmetric, and
its inverse is symmetric,
00:39:59.280 --> 00:40:03.340
and if I transpose this
one when I transpose
00:40:03.340 --> 00:40:08.050
it will pop up there, become
a, that a transpose will pop up
00:40:08.050 --> 00:40:11.440
here, and I'm back to P again.
00:40:11.440 --> 00:40:14.720
And do we dare try
the other property
00:40:14.720 --> 00:40:18.182
which is P squared equal P?
00:40:24.790 --> 00:40:25.750
It's got to be right.
00:40:28.670 --> 00:40:33.540
Because we know geometrically
that the first projection pops
00:40:33.540 --> 00:40:37.090
us onto the column space and the
second one leaves us where we
00:40:37.090 --> 00:40:37.680
are.
00:40:37.680 --> 00:40:43.180
So I expect that if I
multiply by let me do it --
00:40:43.180 --> 00:40:48.030
if I multiply by another P,
so there's another a, another
00:40:48.030 --> 00:40:59.040
a transpose a inverse
a transpose, can you --
00:40:59.040 --> 00:41:03.960
eight (a)-s in a row
is quite obscene but --
00:41:03.960 --> 00:41:05.415
do you see that it works?
00:41:08.160 --> 00:41:11.240
So I'm squaring that so
what do I do-- how do I
00:41:11.240 --> 00:41:12.610
see that multiplication?
00:41:12.610 --> 00:41:17.480
Well, yeah, I just want to put
parentheses in good places,
00:41:17.480 --> 00:41:21.910
so I see what's happening, yeah,
here's an a transpose a sitting
00:41:21.910 --> 00:41:25.380
together -- so when that a
transpose a multiplies its
00:41:25.380 --> 00:41:29.840
inverse, all that
stuff goes, right.
00:41:29.840 --> 00:41:32.300
And leaves just the a
transpose at the end,
00:41:32.300 --> 00:41:35.480
which is just what we want.
00:41:35.480 --> 00:41:39.090
So P squared equals P.
So sure enough those two
00:41:39.090 --> 00:41:40.200
properties hold.
00:41:40.200 --> 00:41:40.730
OK.
00:41:40.730 --> 00:41:45.510
OK we really have got
now all the formulas.
00:41:45.510 --> 00:41:50.370
x hat, the projection P, and
the projection matrix capital
00:41:50.370 --> 00:41:58.710
P. And now my job
is to use them.
00:41:58.710 --> 00:41:59.210
OK.
00:41:59.210 --> 00:42:05.940
So when would I have
a bunch of equations,
00:42:05.940 --> 00:42:12.250
too many equations and yet I
want the best answer and the --
00:42:12.250 --> 00:42:19.210
the most important example,
the most common example is if I
00:42:19.210 --> 00:42:25.130
have points so here's the
-- here's the application.
00:42:25.130 --> 00:42:25.850
v squared.
00:42:25.850 --> 00:42:27.800
Fitting by a line.
00:42:27.800 --> 00:42:28.300
OK.
00:42:42.170 --> 00:42:46.700
So I'll start this application
today and there's more in it
00:42:46.700 --> 00:42:50.240
than I can do in
this same lecture.
00:42:50.240 --> 00:42:53.800
So that'll give me a chance
to recap the formulas
00:42:53.800 --> 00:42:59.910
and there they are,
and recap the ideas.
00:42:59.910 --> 00:43:03.050
So let me start
the problem today.
00:43:03.050 --> 00:43:10.500
I'm given a bunch
of data points.
00:43:10.500 --> 00:43:14.430
And they lie close to a
line but not on a line.
00:43:14.430 --> 00:43:15.660
Let me take that.
00:43:15.660 --> 00:43:19.960
Say a t equal to
one, two and three,
00:43:19.960 --> 00:43:25.740
I have one, and
two and two again.
00:43:25.740 --> 00:43:31.890
So my data points are this is
the like the time direction
00:43:31.890 --> 00:43:37.200
and this is like well let me
call that b or y or something.
00:43:37.200 --> 00:43:43.790
I'm given these three points and
I want to fit them by a line.
00:43:43.790 --> 00:43:46.130
By the best straight line.
00:43:46.130 --> 00:43:55.700
So the problem is fit the points
one, one is the first point --
00:43:55.700 --> 00:44:02.720
the second point is t
equals two, b equal one,
00:44:02.720 --> 00:44:06.735
and the third point is t
equal three, b equal to two.
00:44:09.570 --> 00:44:14.920
So those are my three points,
t equal sorry,that's two.
00:44:14.920 --> 00:44:17.400
Yeah, OK.
00:44:17.400 --> 00:44:19.380
So this is the point one, one.
00:44:19.380 --> 00:44:22.900
This is the point two, two, and
that's the point three, two.
00:44:22.900 --> 00:44:27.430
And of course there isn't a --
a line that goes through them.
00:44:27.430 --> 00:44:29.120
So I'm looking
for the best line.
00:44:29.120 --> 00:44:33.670
I'm looking for a line that
probably goes somewhere,
00:44:33.670 --> 00:44:38.080
do you think it goes
somewhere like that?
00:44:38.080 --> 00:44:41.300
I didn't mean to make it go
through that point, it won't.
00:44:41.300 --> 00:44:42.720
It'll kind of --
00:44:42.720 --> 00:44:47.180
it'll go between so the error
there and the error there
00:44:47.180 --> 00:44:54.780
and the error there are as
small as I can get them.
00:44:54.780 --> 00:45:00.200
OK, what I'd like to do
is find the matrix a.
00:45:00.200 --> 00:45:01.710
Because once I've
found the matrix
00:45:01.710 --> 00:45:05.030
a the formulas take over.
00:45:05.030 --> 00:45:11.110
So what I'm looking for
this line, b is C+Dt.
00:45:11.110 --> 00:45:16.970
So this is in the homework
that I sent out for today.
00:45:16.970 --> 00:45:18.310
Find the best line.
00:45:18.310 --> 00:45:21.170
So I'm looking
for these numbers.
00:45:21.170 --> 00:45:23.820
C and D.
00:45:23.820 --> 00:45:27.130
That tell me the
line and I want them
00:45:27.130 --> 00:45:30.560
to be as close to going
through those three points
00:45:30.560 --> 00:45:32.270
as I can get.
00:45:32.270 --> 00:45:35.320
I can't get exactly so
there are three equations
00:45:35.320 --> 00:45:37.870
to go through the three points.
00:45:37.870 --> 00:45:41.110
It would it will go
exactly through that point
00:45:41.110 --> 00:45:45.050
if let's see that first
point has t equal to one,
00:45:45.050 --> 00:45:48.710
so that would say C+D equaled 1.
00:45:48.710 --> 00:45:51.890
This is the one, one.
00:45:51.890 --> 00:45:55.250
The second point t is two.
00:45:55.250 --> 00:46:00.760
So C+2D should come
out to equal 2.
00:46:00.760 --> 00:46:05.190
But I also want to get the third
equation in and at that third
00:46:05.190 --> 00:46:11.400
equation t is three
so C+3D equals only 2.
00:46:17.400 --> 00:46:19.160
That's the key.
00:46:19.160 --> 00:46:23.450
Is to write down what equations
we would like to solve
00:46:23.450 --> 00:46:25.010
but can't.
00:46:25.010 --> 00:46:28.080
Reason we if we could solve them
that would mean that we could
00:46:28.080 --> 00:46:33.470
put a line through all
three points and of course
00:46:33.470 --> 00:46:39.100
if these numbers one, two, two
were different, we could do it.
00:46:39.100 --> 00:46:42.130
But with those numbers,
one, two, two, we can't.
00:46:42.130 --> 00:46:49.300
So what is our equation Ax equal
Ax equal b that we can't solve?
00:46:49.300 --> 00:46:52.840
I just want to say
what's the matrix here,
00:46:52.840 --> 00:46:56.090
what's the unknown x, and
what's the right-hand side.
00:46:56.090 --> 00:47:00.975
So this is the matrix is one,
one, one, one, two, three.
00:47:03.980 --> 00:47:07.650
The unknown is C and D.
00:47:07.650 --> 00:47:10.350
And the right-hand
side if one, two, two.
00:47:10.350 --> 00:47:18.080
Right, I've just
taken my equations
00:47:18.080 --> 00:47:28.810
and I've said what
is Ax and what is b.
00:47:28.810 --> 00:47:33.640
Then there's no solution, this
is the typical case where I
00:47:33.640 --> 00:47:36.860
have three equations --
two unknowns, no solution,
00:47:36.860 --> 00:47:40.640
but I'm still looking
for the best solution.
00:47:40.640 --> 00:47:45.780
And the best
solution is taken is
00:47:45.780 --> 00:47:49.390
is to solve not this
equation Ax equal
00:47:49.390 --> 00:47:55.960
b which has which has no
solution but the equation that
00:47:55.960 --> 00:48:01.350
does have a solution,
which was this one.
00:48:01.350 --> 00:48:03.720
So that's the equation to solve.
00:48:03.720 --> 00:48:06.110
That's the central
equation of the subject.
00:48:06.110 --> 00:48:12.540
I can't solve Ax=b but magically
when I multiply both sides
00:48:12.540 --> 00:48:22.280
by a transpose I get an equation
that I can solve and its
00:48:22.280 --> 00:48:29.770
solution gives me x, the
best x, the best projection,
00:48:29.770 --> 00:48:36.510
and I discover what's the
matrix that's behind it.
00:48:36.510 --> 00:48:37.260
OK.
00:48:37.260 --> 00:48:44.010
So next time I'll complete an
example, numerical example.
00:48:44.010 --> 00:48:49.250
today was all letters,
numbers next time.
00:48:49.250 --> 00:48:50.800
Thanks.