1
00:00:07 --> 00:00:14
OK, guys the -- we're almost
ready to make this lecture
2
00:00:14 --> 00:00:15
immortal.
3
00:00:15 --> 00:00:15
OK.
4
00:00:15 --> 00:00:16
Are we on?
5
00:00:16 --> 00:00:17
All right.
6
00:00:17 --> 00:00:21
This is an important lecture.
7
00:00:21 --> 00:00:23
It's about projections.
8
00:00:23 --> 00:00:30
Let me start by just projecting
a vector b down on a vector a.
9
00:00:30 --> 00:00:38.07
So just to so you see what the
geometry looks like in when I'm
10
00:00:38.07 --> 00:00:41
in --
in just two dimensions,
11
00:00:41 --> 00:00:46
I'd like to find the point
along this line so that line
12
00:00:46 --> 00:00:51
through a is a one-dimensional
subspace, so I'm starting with
13
00:00:51 --> 00:00:52
one dimension.
14
00:00:52 --> 00:00:56
I'd like to find the point on
that line closest to a.
15
00:00:56 --> 00:01:00
Can I just take that problem
first and then I'll explain why
16
00:01:00 --> 00:01:06
I want to do it and why I want
to project on other subspaces.
17
00:01:06 --> 00:01:12
So where's the point closest to
b that's on that line?
18
00:01:12 --> 00:01:15.16
It's somewhere there.
19
00:01:15.16 --> 00:01:22
And let me connect that and --
and what's the whole point of my
20
00:01:22 --> 00:01:23
picture now?
21
00:01:23 --> 00:01:29
What's the -- where does
orthogonality come into this
22
00:01:29 --> 00:01:31
picture?
23
00:01:31 --> 00:01:35
The whole point is that this
best point, that's the
24
00:01:35 --> 00:01:38
projection, P,
of b onto the line,
25
00:01:38 --> 00:01:40
where's orthogonality?
26
00:01:40 --> 00:01:43
It's the fact that that's a
right angle.
27
00:01:43 --> 00:01:49
That this -- the error -- this
is like how much I'm wrong by --
28
00:01:49 --> 00:01:55
this is the difference between b
and P, the whole point is --
29
00:01:55 --> 00:01:57
that's perpendicular to a.
30
00:01:57 --> 00:02:01
That's got to give us the
equation.
31
00:02:01 --> 00:02:06
That's got to tell us -- that's
the one fact we know,
32
00:02:06 --> 00:02:10
that's got to tell us where
that projection is.
33
00:02:10 --> 00:02:14
Let me also say,
look -- I've drawn a triangle
34
00:02:14 --> 00:02:16
there.
35
00:02:16 --> 00:02:21
So if we were doing
trigonometry we would do like we
36
00:02:21 --> 00:02:27
would have angles theta and
distances that would involve
37
00:02:27 --> 00:02:33
sine theta and cos theta that
leads to lousy formulas compared
38
00:02:33 --> 00:02:35
to linear algebra.
39
00:02:35 --> 00:02:42
The formula that we want comes
out nicely and what's the --
40
00:02:42 --> 00:02:43
what do we know?
41
00:02:43 --> 00:02:46
We know that P,
this projection,
42
00:02:46 --> 00:02:49
is some multiple of a,
right?
43
00:02:49 --> 00:02:50
It's on that line.
44
00:02:50 --> 00:02:54
So we know it's in that
one-dimensional subspace,
45
00:02:54 --> 00:02:59
it's some multiple,
let me call that multiple x,
46
00:02:59 --> 00:02:59
of a.
47
00:02:59 --> 00:03:04
So really it's that number x
I'd like to find.
48
00:03:04 --> 00:03:11
So this is going to be simple
in 1-D, so let's just carry it
49
00:03:11 --> 00:03:16
through, and then see how it
goes in high dimensions.
50
00:03:16 --> 00:03:17
OK.
51
00:03:17 --> 00:03:22
The key fact is -- the key to
everything is that
52
00:03:22 --> 00:03:23
perpendicular.
53
00:03:23 --> 00:03:28.27
The fact that a is
perpendicular to a is
54
00:03:28.27 --> 00:03:31
perpendicular to e.
55
00:03:31 --> 00:03:33
Which is (b-ax),
xa.
56
00:03:33 --> 00:03:36
I don't care what -- xa.
57
00:03:36 --> 00:03:38
That that equals zero.
58
00:03:38 --> 00:03:44
Do you see that as the central
equation, that's saying that
59
00:03:44 --> 00:03:48
this a is perpendicular to this
-- correction,
60
00:03:48 --> 00:03:52
that's going to tell us what x
is.
61
00:03:52 --> 00:03:58
Let me just raise the board and
simplify that and out will come
62
00:03:58 --> 00:04:00
x.
63
00:04:00 --> 00:04:00
OK.
64
00:04:00 --> 00:04:05
So if I simplify that,
let's see, I'll move one to --
65
00:04:05 --> 00:04:10
one term to one side,
the other term will be on the
66
00:04:10 --> 00:04:15
other side, it looks to me like
x times a transpose a is equal
67
00:04:15 --> 00:04:17
to a transpose b.
68
00:04:17 --> 00:04:17
Right?
69
00:04:17 --> 00:04:23
I have a transpose b as one f-
one term, a transpose a as the
70
00:04:23 --> 00:04:28
other, so right away here's my a
transpose a.
71
00:04:28 --> 00:04:31
But it's just a number now.
72
00:04:31 --> 00:04:33
And I divide by it.
73
00:04:33 --> 00:04:38
And I get the answer.
x is a transpose b over a
74
00:04:38 --> 00:04:39
transpose a.
75
00:04:39 --> 00:04:44.73
And P, the projection I wanted,
is -- that's the right
76
00:04:44.73 --> 00:04:45
multiple.
77
00:04:45 --> 00:04:49
That's got a cosine theta built
in.
78
00:04:49 --> 00:04:53
But we don't need to look at
angles.
79
00:04:53 --> 00:04:56
It's -- we've just got vectors
here.
80
00:04:56 --> 00:05:00
And the projection is P is a
times that x.
81
00:05:00 --> 00:05:02
Or x times that a.
82
00:05:02 --> 00:05:07
But I'm really going to --
eventually I'm going to want
83
00:05:07 --> 00:05:10
that x coming on the right-hand
side.
84
00:05:10 --> 00:05:16
So do you see that I've got two
of the three formulas already,
85
00:05:16 --> 00:05:23
right here, I've got the --
that's the equation -- that
86
00:05:23 --> 00:05:28.66
leads me to the answer,
here's the answer for x,
87
00:05:28.66 --> 00:05:31
and here's the projection.
88
00:05:31 --> 00:05:35
OK.
can I do add just one more
89
00:05:35 --> 00:05:40.6
thing to this one-dimensional
problem?
90
00:05:40.6 --> 00:05:45
One more like lift it up into
linear algebra,
91
00:05:45 --> 00:05:46
into matrices.
92
00:05:46 --> 00:05:53
Here's the last thing I want to
do -- but don't forget those
93
00:05:53 --> 00:05:57
formulas.
a transpose b over a transpose
94
00:05:57 --> 00:05:57
a.
95
00:05:57 --> 00:06:02
Actually let's look at that for
a moment first.
96
00:06:02 --> 00:06:08
Suppose --
Let me take this next step.
97
00:06:08 --> 00:06:09
So P is a times x.
98
00:06:09 --> 00:06:12.55
So can I write that then?
99
00:06:12.55 --> 00:06:18
P is a times this neat number,
a transpose b over a transpose
100
00:06:18 --> 00:06:19
a.
101
00:06:19 --> 00:06:21.4
That's our projection.
102
00:06:21.4 --> 00:06:27
Can I ask a couple of questions
about it, just while we look,
103
00:06:27 --> 00:06:31
get that digest that formula.
104
00:06:31 --> 00:06:34
Suppose b is doubled.
105
00:06:34 --> 00:06:37
Suppose I change b to two b.
106
00:06:37 --> 00:06:40
What happens to the projection?
107
00:06:40 --> 00:06:46
So suppose I instead of that
vector b that I drew on the
108
00:06:46 --> 00:06:52
board make it two b,
twice as long -- what's the
109
00:06:52 --> 00:06:53
projection now?
110
00:06:53 --> 00:06:57
It's doubled too,
right?
111
00:06:57 --> 00:07:02
It's going to be twice as far,
if b goes twice as far,
112
00:07:02 --> 00:07:05
the projection will go twice as
far.
113
00:07:05 --> 00:07:07
And you see it there.
114
00:07:07 --> 00:07:12
If I put in an extra factor
two, then P's got that factor
115
00:07:12 --> 00:07:12
too.
116
00:07:12 --> 00:07:15.35
Now what about if I double a?
117
00:07:15.35 --> 00:07:20
What if I double the vector a
that I'm projecting onto?
118
00:07:20 --> 00:07:22
What changes?
119
00:07:22 --> 00:07:25
The projection doesn't change
at all.
120
00:07:25 --> 00:07:25
Right?
121
00:07:25 --> 00:07:29
Because I'm just -- the line
didn't change.
122
00:07:29 --> 00:07:31
If I double a or I take minus
a.
123
00:07:31 --> 00:07:33
It's still that same line.
124
00:07:33 --> 00:07:36
The projection's still in the
same place.
125
00:07:36 --> 00:07:40
And of course if I double a I
get a four up above,
126
00:07:40 --> 00:07:45
and I get a four --
an extra four below,
127
00:07:45 --> 00:07:50
they cancel out,
and the projection is the same.
128
00:07:50 --> 00:07:50
OK.
129
00:07:50 --> 00:07:57
So really, this -- I want to
look at this as the projection
130
00:07:57 --> 00:07:59
-- there's a matrix here.
131
00:07:59 --> 00:08:06.58
The projection is carried out
by some matrix that I'm going to
132
00:08:06.58 --> 00:08:10.62
call the projection matrix.
133
00:08:10.62 --> 00:08:14
And in other words the
projection is some matrix that
134
00:08:14 --> 00:08:18.28
acts on this guy b and produces
the projection.
135
00:08:18.28 --> 00:08:22
The projection P is the
projection matrix acting on
136
00:08:22 --> 00:08:23
whatever the input is.
137
00:08:23 --> 00:08:27
The input is b,
the projection matrix is P.
138
00:08:27 --> 00:08:27
OK.
139
00:08:27 --> 00:08:32
Actually you can tell me right
away what this projection matrix
140
00:08:32 --> 00:08:32
is.
141
00:08:32 --> 00:08:36
So this is a pretty interesting
matrix.
142
00:08:36 --> 00:08:40
What matrix is multiplying b?
143
00:08:40 --> 00:08:47
I'm just -- just from my
formula -- I see what P is.
144
00:08:47 --> 00:08:53
P, this projection matrix,
is -- what is it?
145
00:08:53 --> 00:09:01
I see a a transpose above,
and I see a transpose a below.
146
00:09:01 --> 00:09:05
And those don't cancel.
147
00:09:05 --> 00:09:06
That's not one.
148
00:09:06 --> 00:09:07
Right?
149
00:09:07 --> 00:09:08
That's a matrix.
150
00:09:08 --> 00:09:11.42
Because down here,
the a transpose a,
151
00:09:11.42 --> 00:09:14
that's just a number,
a transpose a,
152
00:09:14 --> 00:09:19
that's the length of a squared,
and up above is a column times
153
00:09:19 --> 00:09:19
a row.
154
00:09:19 --> 00:09:22.3
Column times a row is a matrix.
155
00:09:22.3 --> 00:09:27
So this is a full-scale n by n
matrix, if I --
156
00:09:27 --> 00:09:29
if I'm in n dimensions.
157
00:09:29 --> 00:09:32.32
And it's kind of an interesting
one.
158
00:09:32.32 --> 00:09:37
And it's the one which if I
multiply by b then I get this,
159
00:09:37 --> 00:09:43
you see once again I'm putting
parentheses in different places.
160
00:09:43 --> 00:09:46.69
I'm putting the parentheses
right there.
161
00:09:46.69 --> 00:09:50
I'm saying OK,
that's really the matrix that
162
00:09:50 --> 00:09:53
produces this projection.
163
00:09:53 --> 00:09:54
OK.
164
00:09:54 --> 00:09:59.75
Now, tell me -- all right,
what are the properties of that
165
00:09:59.75 --> 00:10:00
matrix?
166
00:10:00 --> 00:10:05.34
I'm just using letters here,
a and b, I could put in
167
00:10:05.34 --> 00:10:11
numbers, but I think it's -- for
once, it's clearer with letters,
168
00:10:11 --> 00:10:16
because all formulas are
simple, a transpose b over a
169
00:10:16 --> 00:10:20
transpose a --
that's the number that
170
00:10:20 --> 00:10:24
multiplies the a,
and then I see wait a minute,
171
00:10:24 --> 00:10:28
there's a matrix here and
what's the rank of that matrix,
172
00:10:28 --> 00:10:29
by the way?
173
00:10:29 --> 00:10:34
What's the rank of that matrix,
yeah -- let me just ask you
174
00:10:34 --> 00:10:35
about that matrix.
175
00:10:35 --> 00:10:40
Which looks a little strange,
a a transpose over this number.
176
00:10:40 --> 00:10:44
But well, I could ask you its
column space.
177
00:10:44 --> 00:10:47
Yeah, let me ask you its column
space.
178
00:10:47 --> 00:10:50
So what's the column space of a
matrix?
179
00:10:50 --> 00:10:54
If you multiply that matrix by
anything you always get in the
180
00:10:54 --> 00:10:56
column space,
right?
181
00:10:56 --> 00:11:01
The column space of a matrix is
when you multiply any vector by
182
00:11:01 --> 00:11:04
that matrix --
any vector b,
183
00:11:04 --> 00:11:09
by the matrix,
you always land in the column
184
00:11:09 --> 00:11:10.44
space.
185
00:11:10.44 --> 00:11:14
That's what column spaces work.
186
00:11:14 --> 00:11:18.26
Now what space do we always
land in?
187
00:11:18.26 --> 00:11:24.54
What's the column space of --
what's the result when I
188
00:11:24.54 --> 00:11:30
multiply this any vector b by my
matrix?
189
00:11:30 --> 00:11:34.01
So I have P times b,
where I?
190
00:11:34.01 --> 00:11:36
I'm on that line,
right?
191
00:11:36 --> 00:11:42.62
The column space,
so here are facts about this
192
00:11:42.62 --> 00:11:43
matrix.
193
00:11:43 --> 00:11:49
The column space of P,
of this projection matrix,
194
00:11:49 --> 00:11:52
is the line through a.
195
00:11:52 --> 00:12:00
And the rank of this matrix is
you can all say it at once one.
196
00:12:00 --> 00:12:00
Right.
197
00:12:00 --> 00:12:04.65
The rank is one.
198
00:12:04.65 --> 00:12:07
This is a rank one matrix.
199
00:12:07 --> 00:12:14
Actually it's exactly the form
that we're familiar with a rank
200
00:12:14 --> 00:12:15
one matrix.
201
00:12:15 --> 00:12:20.25
A column times a row,
that's a rank one matrix,
202
00:12:20.25 --> 00:12:25
that column is the basis for
the column space.
203
00:12:25 --> 00:12:28
Just one dimension.
204
00:12:28 --> 00:12:28
OK.
205
00:12:28 --> 00:12:31
So I know that much about the
matrix.
206
00:12:31 --> 00:12:37
But now there are two more
facts about the matrix that I
207
00:12:37 --> 00:12:38
want to notice.
208
00:12:38 --> 00:12:42
First of all is the matrix
symmetric?
209
00:12:42 --> 00:12:46.13
That's a natural question for
matrices.
210
00:12:46.13 --> 00:12:48
And the answer is yes.
211
00:12:48 --> 00:12:52
If I take the transpose of this
--
212
00:12:52 --> 00:12:58
there's a number down there,
the transpose of a a transpose
213
00:12:58 --> 00:12:59
is a a transpose.
214
00:12:59 --> 00:13:01
So P is symmetric.
215
00:13:01 --> 00:13:03
P transpose equals P.
216
00:13:03 --> 00:13:05
So this is a key property.
217
00:13:05 --> 00:13:09
That the projection matrix is
symmetric.
218
00:13:09 --> 00:13:13
One more property now and this
is the real one.
219
00:13:13 --> 00:13:18
What happens if I do the
projection twice?
220
00:13:18 --> 00:13:23
So I'm looking for something,
some information about P
221
00:13:23 --> 00:13:24
squared.
222
00:13:24 --> 00:13:28
But just give me in terms of
that picture,
223
00:13:28 --> 00:13:32
in terms my picture,
I take any vector b,
224
00:13:32 --> 00:13:39
I multiply it by my projection
matrix, and that puts me there,
225
00:13:39 --> 00:13:40
so this is Pb.
226
00:13:40 --> 00:13:44
And now I project again.
227
00:13:44 --> 00:13:46
What happens now?
228
00:13:46 --> 00:13:52
What happens when I apply the
projection matrix a second time?
229
00:13:52 --> 00:13:58.45
To this, so I'm applying it
once brings me here and the
230
00:13:58.45 --> 00:14:01
second time brings me I stay
put.
231
00:14:01 --> 00:14:02
Right?
232
00:14:02 --> 00:14:08
The projection for a point on
this line the projection is
233
00:14:08 --> 00:14:11
right where it is.
234
00:14:11 --> 00:14:14
The projection is the same
point.
235
00:14:14 --> 00:14:19
So that means that if I project
twice, I get the same answer as
236
00:14:19 --> 00:14:22
I did in the first projection.
237
00:14:22 --> 00:14:27
So those are the two properties
that tell me I'm looking at a
238
00:14:27 --> 00:14:28
projection matrix.
239
00:14:28 --> 00:14:32
It's symmetric and it's square
is itself.
240
00:14:32 --> 00:14:37
Because if I project a second
time, it's the same result as
241
00:14:37 --> 00:14:39
the first result.
242
00:14:39 --> 00:14:40.25
OK.
243
00:14:40.25 --> 00:14:46
So that's -- and then here's
the exact formula when I know
244
00:14:46 --> 00:14:50.92
what I'm projecting onto,
that line through a,
245
00:14:50.92 --> 00:14:53
then I know what P is.
246
00:14:53 --> 00:14:59
So do you see that I have all
the pieces here for projection
247
00:14:59 --> 00:15:00
on a line?
248
00:15:00 --> 00:15:05
Now, and those --
please remember those.
249
00:15:05 --> 00:15:08
So there are three formulas to
remember.
250
00:15:08 --> 00:15:12
The formula for x,
the formula for P,
251
00:15:12 --> 00:15:16
which is just ax,
and then the formula for
252
00:15:16 --> 00:15:18
capital P, which is the matrix.
253
00:15:18 --> 00:15:19
Good.
254
00:15:19 --> 00:15:19
Good.
255
00:15:19 --> 00:15:20
OK.
256
00:15:20 --> 00:15:23
Now I want to move to more
dimensions.
257
00:15:23 --> 00:15:28
So we're going to have three
formulas again but you'll have
258
00:15:28 --> 00:15:35
to --
they'll be a little different
259
00:15:35 --> 00:15:43
because we won't have a single
line but -- a plane or
260
00:15:43 --> 00:15:50
three-dimensional or a
n-dimensional subspace.
261
00:15:50 --> 00:15:50
OK.
262
00:15:50 --> 00:15:56
So now I'll move to the next
question.
263
00:15:56 --> 00:16:06
Maybe -- let me say first why I
want this projection,
264
00:16:06 --> 00:16:16
and then we'll figure out what
it is, we'll go completely in
265
00:16:16 --> 00:16:22.27
parallel there,
and then we'll use it.
266
00:16:22.27 --> 00:16:27
OK, why do I want this
projection?
267
00:16:27 --> 00:16:31
Well, so why project?
268
00:16:31 --> 00:16:41
It's because I'm as I mentioned
last time this new chapter deals
269
00:16:41 --> 00:16:50
with equations Ax=b may have no
solution.
270
00:16:50 --> 00:16:56
So that's really my problem,
that I'm given a bunch of
271
00:16:56 --> 00:17:02
equations probably too many
equations, more equations than
272
00:17:02 --> 00:17:06.7
unknowns, and I can't solve
them.
273
00:17:06.7 --> 00:17:07
OK.
274
00:17:07 --> 00:17:08
So what do I do?
275
00:17:08 --> 00:17:14
I solve the closest problem
that I can solve.
276
00:17:14 --> 00:17:18
And what's the closest one?
277
00:17:18 --> 00:17:24
Well, ax will always be in the
column space of a.
278
00:17:24 --> 00:17:27
That's my problem.
279
00:17:27 --> 00:17:34
My problem is ax has to be in
the column space and b is
280
00:17:34 --> 00:17:38
probably not in the column
space.
281
00:17:38 --> 00:17:41
So I change b to what?
282
00:17:41 --> 00:17:47
I choose the closest vector in
the column space,
283
00:17:47 --> 00:17:51.48
so I'll solve Ax equal P
instead.
284
00:17:51.48 --> 00:17:55
That one I can do.
285
00:17:55 --> 00:18:00
Where P is this is the
projection of b onto the column
286
00:18:00 --> 00:18:01
space.
287
00:18:01 --> 00:18:05
That's why I want to be able to
do this.
288
00:18:05 --> 00:18:09
Because I have to find a
solution here,
289
00:18:09 --> 00:18:15.1
and I'm going to put a little
hat there to indicate that it's
290
00:18:15.1 --> 00:18:19
not the x, it's not the x that
doesn't exist,
291
00:18:19 --> 00:18:24
it's the x hat that's best
possible.
292
00:18:24 --> 00:18:30
So I must be able to figure out
what's the good projection
293
00:18:30 --> 00:18:31
there.
294
00:18:31 --> 00:18:38
What's the good right-hand side
that is in the column space
295
00:18:38 --> 00:18:45
that's as close as possible to b
and then I'm -- then I know what
296
00:18:45 --> 00:18:46
to do.
297
00:18:46 --> 00:18:47
OK.
298
00:18:47 --> 00:18:51
So now I've got that problem.
299
00:18:51 --> 00:18:58
So that's why I have the
problem again but now let me say
300
00:18:58 --> 00:19:03
I'm in three dimensions,
so I have a plane maybe for
301
00:19:03 --> 00:19:10
example, and I have a vector b
that's not in the plane.
302
00:19:10 --> 00:19:14
And I want to project b down
into the plane.
303
00:19:14 --> 00:19:15
OK.
304
00:19:15 --> 00:19:17
So there's my question.
305
00:19:17 --> 00:19:23
How do I project a vector and
I'm --
306
00:19:23 --> 00:19:28
what I'm looking for is a nice
formula, and I'm counting on
307
00:19:28 --> 00:19:33
linear algebra to just come out
right, a nice formula for the
308
00:19:33 --> 00:19:36
projection of b into the plane.
309
00:19:36 --> 00:19:37
The nearest point.
310
00:19:37 --> 00:19:42
So this again a right angle is
going to be crucial.
311
00:19:42 --> 00:19:42
OK.
312
00:19:42 --> 00:19:47
Now so what's -- first of all
I've got to say what is that
313
00:19:47 --> 00:19:48
plane.
314
00:19:48 --> 00:19:53
To get a formula I have to tell
you what the plane is.
315
00:19:53 --> 00:19:55
How I going to tell you a
plane?
316
00:19:55 --> 00:20:01
I'll tell you a basis for the
plane, I'll tell you two vectors
317
00:20:01 --> 00:20:05
a one and a two that give you a
basis for the plane,
318
00:20:05 --> 00:20:10
so that -- let us say -- say
there's an a one and here's an a
319
00:20:10 --> 00:20:12
-- a vector a two.
320
00:20:12 --> 00:20:17
They don't have to be
perpendicular.
321
00:20:17 --> 00:20:24
But they better be independent,
because then that tells me the
322
00:20:24 --> 00:20:24
plane.
323
00:20:24 --> 00:20:30
The plane is the -- is the
plane of a one and a two.
324
00:20:30 --> 00:20:37
And actually going back to my
-- to this connection,
325
00:20:37 --> 00:20:43
this plane is a column space,
it's the column space of what
326
00:20:43 --> 00:20:46
matrix?
327
00:20:46 --> 00:20:50
What matrix,
so how do I connect the two
328
00:20:50 --> 00:20:52
questions?
329
00:20:52 --> 00:20:59
I'm thinking how do I project
onto a plane and I want to get a
330
00:20:59 --> 00:21:01
matrix in here.
331
00:21:01 --> 00:21:08
Everything's cleaner if I write
it in terms of a matrix.
332
00:21:08 --> 00:21:14
So what matrix has these --
has that column space?
333
00:21:14 --> 00:21:19.65
Well of course it's just the
matrix that has a one in the
334
00:21:19.65 --> 00:21:23
first column and a two in the
second column.
335
00:21:23 --> 00:21:29
Right, just just let's be sure
we've got the question before we
336
00:21:29 --> 00:21:31
get to the answer.
337
00:21:31 --> 00:21:35
So I'm looking for again I'm
given a matrix a with two
338
00:21:35 --> 00:21:37
columns.
339
00:21:37 --> 00:21:42
And really I'm ready once I get
to two I'm ready for n.
340
00:21:42 --> 00:21:46
So it could be two columns,
it could be n columns.
341
00:21:46 --> 00:21:51
I'll write the answer in terms
of the matrix a.
342
00:21:51 --> 00:21:56
And the point will be those two
columns describe the plane,
343
00:21:56 --> 00:22:01
they describe the column space,
and I want to project.
344
00:22:01 --> 00:22:02
OK.
345
00:22:02 --> 00:22:07
And I'm given a vector b that's
probably not in the column
346
00:22:07 --> 00:22:07
space.
347
00:22:07 --> 00:22:11
Of course, if b is in the
column space,
348
00:22:11 --> 00:22:14
my projection is simple,
it's just b.
349
00:22:14 --> 00:22:19
But most likely I have an error
e, this b minus P part,
350
00:22:19 --> 00:22:21
which is probably not zero.
351
00:22:21 --> 00:22:21.84
OK.
352
00:22:21.84 --> 00:22:27
But the beauty is that I know
-- from geometry or I could get
353
00:22:27 --> 00:22:32
it from calculus or I could get
it from linear algebra that that
354
00:22:32 --> 00:22:40
this this vector --
this is the part of b that's
355
00:22:40 --> 00:22:45
that's perpendicular to the
plane.
356
00:22:45 --> 00:22:54
That e is perpendicular is
perpendicular to the plane.
357
00:22:54 --> 00:23:02
If your intuition is saying
that that's the crucial fact.
358
00:23:02 --> 00:23:09
That's going to give us the
answer.
359
00:23:09 --> 00:23:10
OK.
360
00:23:10 --> 00:23:13
So let me, that's the problem.
361
00:23:13 --> 00:23:15
Now for the answer.
362
00:23:15 --> 00:23:21
So this is a lecture that's
really like moving along.
363
00:23:21 --> 00:23:29
Because I'm just plotting that
problem up there and asking you
364
00:23:29 --> 00:23:34.32
what combination -- now,
yeah, so what is it?
365
00:23:34.32 --> 00:23:34
P.
366
00:23:34 --> 00:23:38
What is this projection P?
367
00:23:38 --> 00:23:43
This is projection P,
is some multiple of these basis
368
00:23:43 --> 00:23:47
guys, right, some multiple of
the columns.
369
00:23:47 --> 00:23:52
But I don't like writing out x
one a one plus x two a two,
370
00:23:52 --> 00:23:55.7
I would rather right that as
ax.
371
00:23:55.7 --> 00:24:00
Well, actually I should put if
I'm really doing everything
372
00:24:00 --> 00:24:05
right, I should put a little hat
on it --
373
00:24:05 --> 00:24:11
to remember that this x -- that
those are the numbers and I
374
00:24:11 --> 00:24:16
could have a put a hat way back
there is right,
375
00:24:16 --> 00:24:20
so this is this is the
projection, P.
376
00:24:20 --> 00:24:21
P is ax bar.
377
00:24:21 --> 00:24:23
And I'm looking for x bar.
378
00:24:23 --> 00:24:27
So that's what I want an
equation for.
379
00:24:27 --> 00:24:32
So now I've got hold of the
problem.
380
00:24:32 --> 00:24:39
The problem is find the right
combination of the columns so
381
00:24:39 --> 00:24:45
that the error vector is
perpendicular to the plane.
382
00:24:45 --> 00:24:49
Now let me turn that into an
equation.
383
00:24:49 --> 00:24:56
So I'll raise the board and
just turn that -- what we've
384
00:24:56 --> 00:24:59
just done into an equation.
385
00:24:59 --> 00:25:05
So let me I'll write again the
main point.
386
00:25:05 --> 00:25:09
The projection is ax b- x hat.
387
00:25:09 --> 00:25:13
And our problem is find x hat.
388
00:25:13 --> 00:25:21
And the key is that b minus ax
hat, that's the error.
389
00:25:21 --> 00:25:23
This is the e.
390
00:25:23 --> 00:25:27
Is perpendicular to the plane.
391
00:25:27 --> 00:25:33
That's got to give me well what
I looking for,
392
00:25:33 --> 00:25:42
I'm looking for two equations
now because I've got an x one
393
00:25:42 --> 00:25:45
and an x two.
394
00:25:45 --> 00:25:49
And I'll get two equations
because so this thing e is
395
00:25:49 --> 00:25:51
perpendicular to the plane.
396
00:25:51 --> 00:25:53
So what does that mean?
397
00:25:53 --> 00:25:57
I guess it means it's
perpendicular to a one and also
398
00:25:57 --> 00:25:58
to a two.
399
00:25:58 --> 00:26:03.27
Right, those are two vectors in
the plane and the things that
400
00:26:03.27 --> 00:26:08.18
are perpendicular to the plane
are perpendicular to a one and a
401
00:26:08.18 --> 00:26:09
two.
402
00:26:09 --> 00:26:11
Let me just repeat.
403
00:26:11 --> 00:26:15
This this guy then is
perpendicular to the plane so
404
00:26:15 --> 00:26:19
it's perpendicular to that
vector and that vector.
405
00:26:19 --> 00:26:23
Not -- it's perpendicular to
that of course.
406
00:26:23 --> 00:26:27
But it's perpendicular to
everything I the plane.
407
00:26:27 --> 00:26:32
And the plane is really told me
by a one and a two.
408
00:26:32 --> 00:26:40
So really I have the equations
a one transpose b minus ax is
409
00:26:40 --> 00:26:41
zero.
410
00:26:41 --> 00:26:47
And also a two transpose b
minus ax is zero.
411
00:26:47 --> 00:26:51
Those are my two equations.
412
00:26:51 --> 00:26:55
But I want those in matrix
form.
413
00:26:55 --> 00:27:02
I want to put those two
equations together as a matrix
414
00:27:02 --> 00:27:09
equation and it's just comes out
right.
415
00:27:09 --> 00:27:14
Look at the matrix a transpose.
416
00:27:14 --> 00:27:21
Put a one a one transpose is
its first row,
417
00:27:21 --> 00:27:30
a two transpose is its second
row, that multiplies this b-ax,
418
00:27:30 --> 00:27:36
and gives me the zero and the
zero.
419
00:27:36 --> 00:27:45
I'm you see the -- this is one
way -- to come up with this
420
00:27:45 --> 00:27:48.22
equation.
421
00:27:48.22 --> 00:27:54
So the equation I'm coming up
with is a transpose b-ax hat is
422
00:27:54 --> 00:27:55
zero.
423
00:27:55 --> 00:27:55
OK.
424
00:27:55 --> 00:27:57
That's my equation.
425
00:27:57 --> 00:27:58
All right.
426
00:27:58 --> 00:28:04
Now I want to stop for a moment
before I solve it and just think
427
00:28:04 --> 00:28:05
about it.
428
00:28:05 --> 00:28:11
First of all do you see that
that equation back in the very
429
00:28:11 --> 00:28:17
first problem I solved on a
line, what was --
430
00:28:17 --> 00:28:22.73
what was on a line the matrix a
only had one column,
431
00:28:22.73 --> 00:28:24
it was just little a.
432
00:28:24 --> 00:28:29
So in the first problem I
solved, projecting on a line,
433
00:28:29 --> 00:28:35
this for capital a you just
change that to little a and you
434
00:28:35 --> 00:28:39.64
have the same equation that we
solved before.
435
00:28:39.64 --> 00:28:43
a transpose e equals zero.
436
00:28:43 --> 00:28:43
OK.
437
00:28:43 --> 00:28:47
Now a second thing,
second comment.
438
00:28:47 --> 00:28:53
I would like to since I know
about these four subspaces,
439
00:28:53 --> 00:28:57
I would like to get them into
this picture.
440
00:28:57 --> 00:29:03
So let me ask the question,
what subspace is this thing in?
441
00:29:03 --> 00:29:09.27
Which of the four subspaces is
that error vector e,
442
00:29:09.27 --> 00:29:14
this is this is nothing but e
--
443
00:29:14 --> 00:29:20
this is this guy,
coming in down perpendicular to
444
00:29:20 --> 00:29:21
the plane.
445
00:29:21 --> 00:29:24
What subspace is e in?
446
00:29:24 --> 00:29:26
From this equation.
447
00:29:26 --> 00:29:33
Well the equation is saying a
transpose e is zero.
448
00:29:33 --> 00:29:39
So I'm learning here that e is
in the null space of a
449
00:29:39 --> 00:29:41
transpose.
450
00:29:41 --> 00:29:41
Right?
451
00:29:41 --> 00:29:45
That's my equation.
452
00:29:45 --> 00:29:51
And now I just want to see hey
of course that that was right.
453
00:29:51 --> 00:29:56
Because things that are in the
null space of a transpose,
454
00:29:56 --> 00:30:01
what do we know about the null
space of a transpose?
455
00:30:01 --> 00:30:07
So that last lecture gave us
the sort of the geometry of
456
00:30:07 --> 00:30:09
these subspaces.
457
00:30:09 --> 00:30:12
And the orthogonality of them.
458
00:30:12 --> 00:30:15
And do you remember what it
was?
459
00:30:15 --> 00:30:22
What on the right side of our
big figure we always have the
460
00:30:22 --> 00:30:27
null space of a transpose and
the column space of a,
461
00:30:27 --> 00:30:29.91
and they're orthogonal.
462
00:30:29.91 --> 00:30:35
So e in the null space of a
transpose is saying e is
463
00:30:35 --> 00:30:40
perpendicular to the column
space of a.
464
00:30:40 --> 00:30:41
Yes.
465
00:30:41 --> 00:30:47
I just feel OK,
the damn thing came out right.
466
00:30:47 --> 00:30:55
The equation for the equation
that I struggled to find for e
467
00:30:55 --> 00:31:01
really said what I wanted,
that the error e is
468
00:31:01 --> 00:31:08
perpendicular to the column
space of a, just right.
469
00:31:08 --> 00:31:14
And from our four fundamental
subspaces we knew that that is
470
00:31:14 --> 00:31:15
the same as that.
471
00:31:15 --> 00:31:20
To say e is in the null space
of a transpose says e's
472
00:31:20 --> 00:31:23
perpendicular to the column
space.
473
00:31:23 --> 00:31:23
OK.
474
00:31:23 --> 00:31:25
So we've got this equation.
475
00:31:25 --> 00:31:27.87
Now let's just solve it.
476
00:31:27.87 --> 00:31:28
All right.
477
00:31:28 --> 00:31:33
Let me just rewrite it as a
transpose a x hat equals a
478
00:31:33 --> 00:31:35
transpose b.
479
00:31:35 --> 00:31:37
That's our equation.
480
00:31:37 --> 00:31:39
That gives us x.
481
00:31:39 --> 00:31:44
And -- allow me to keep
remembering the one-dimensional
482
00:31:44 --> 00:31:44
case.
483
00:31:44 --> 00:31:49
The one-dimensional case,
this was little a.
484
00:31:49 --> 00:31:53
So this was just a number,
little a transpose,
485
00:31:53 --> 00:31:58
a transpose a was just a vector
row times a column,
486
00:31:58 --> 00:32:00
a number.
487
00:32:00 --> 00:32:03
And this was a number.
488
00:32:03 --> 00:32:07
And x was the ratio of those
numbers.
489
00:32:07 --> 00:32:12
But now we've got matrices,
this one is n by n.
490
00:32:12 --> 00:32:16
a transpose a is an n by n
matrix.
491
00:32:16 --> 00:32:17
OK.
492
00:32:17 --> 00:32:22
So can I move to the next board
for the solution?
493
00:32:22 --> 00:32:23
OK.
494
00:32:23 --> 00:32:28
This is the --
the key equation.
495
00:32:28 --> 00:32:33
Now I'm ready for the formulas
that we have to remember.
496
00:32:33 --> 00:32:35
What's x hat?
497
00:32:35 --> 00:32:40
What's the projection,
what's the projection matrix,
498
00:32:40 --> 00:32:43
those are my three questions.
499
00:32:43 --> 00:32:49
That we answered in the 1-D
case and now we're ready for in
500
00:32:49 --> 00:32:52
the n-dimensional case.
501
00:32:52 --> 00:32:54
So what is x hat?
502
00:32:54 --> 00:32:59
Well, what can I say but a
transpose a inverse,
503
00:32:59 --> 00:33:01.25
a transpose b.
504
00:33:01.25 --> 00:33:01
OK.
505
00:33:01 --> 00:33:06
That's the solution to -- to
our equation.
506
00:33:06 --> 00:33:08
What's the projection?
507
00:33:08 --> 00:33:11
That's more interesting.
508
00:33:11 --> 00:33:13
What's the projection?
509
00:33:13 --> 00:33:17
The projection is a x hat.
510
00:33:17 --> 00:33:23.41
That's how x hat got into the
picture in the first place.
511
00:33:23.41 --> 00:33:28
x hat was the was the
combination of columns in the I
512
00:33:28 --> 00:33:33.31
had to look for those numbers
and now I found them.
513
00:33:33.31 --> 00:33:38
Was the combination of the
columns of a that gave me the
514
00:33:38 --> 00:33:39
projection.
515
00:33:39 --> 00:33:40
OK.
516
00:33:40 --> 00:33:44
So now I know what this guy is.
517
00:33:44 --> 00:33:51
So it's just I multiply by a.
a a transpose a inverse a
518
00:33:51 --> 00:33:52
transpose b.
519
00:33:52 --> 00:33:58
That's looking a little messy
but it's not bad.
520
00:33:58 --> 00:34:05
That that combination is is our
like magic combination.
521
00:34:05 --> 00:34:13
This is the thing which is
which use which is like what's
522
00:34:13 --> 00:34:19
it like, what was it in one
dimension?
523
00:34:19 --> 00:34:25
What was that we had this we
must have had this thing way
524
00:34:25 --> 00:34:28
back at the beginning of the
lecture.
525
00:34:28 --> 00:34:34
What did we -- oh that a was
just a column so it was little
526
00:34:34 --> 00:34:39
a, little a transpose over a
transpose a, right,
527
00:34:39 --> 00:34:42
that's what it was in 1-D.
528
00:34:42 --> 00:34:47
You see what's happened in more
dimensions, I --
529
00:34:47 --> 00:34:53
I'm not allowed to to just
divide because because I don't
530
00:34:53 --> 00:34:56
have a number,
I have to put inverse,
531
00:34:56 --> 00:34:59
because I have an n by n
matrix.
532
00:34:59 --> 00:35:01
But same formula.
533
00:35:01 --> 00:35:05
And now tell me what's the
projection matrix?
534
00:35:05 --> 00:35:10
What matrix is multiplying b to
give the projection?
535
00:35:10 --> 00:35:12
Right there.
536
00:35:12 --> 00:35:19.59
Because there it --
I even already underlined it by
537
00:35:19.59 --> 00:35:20
accident.
538
00:35:20 --> 00:35:28
The projection matrix which I
use capital P is this,
539
00:35:28 --> 00:35:34
it's it's that thing,
shall I write it again,
540
00:35:34 --> 00:35:40
a times a transpose a inverse
times a transpose.
541
00:35:40 --> 00:35:49
Now if you'll bear with me I'll
think about what have I done
542
00:35:49 --> 00:35:49
here.
543
00:35:49 --> 00:35:54
I've got this formula.
544
00:35:54 --> 00:35:59
Now the first thing that occurs
to me is something bad.
545
00:35:59 --> 00:36:05
Look why don't I just you know
here's a product of two matrices
546
00:36:05 --> 00:36:09.51
and I want its inverse,
why don't I just use the
547
00:36:09.51 --> 00:36:14.45
formula I know for the inverse
of a product and say OK,
548
00:36:14.45 --> 00:36:18
that's a inverse times a
transpose inverse,
549
00:36:18 --> 00:36:22
what will happen if I do that?
550
00:36:22 --> 00:36:27
What will happen if I say hey
this is a inverse times a
551
00:36:27 --> 00:36:31
transpose inverse,
then shall I do it?
552
00:36:31 --> 00:36:37
It's going to go on videotape
if I do it, and I don't -- all
553
00:36:37 --> 00:36:43
right, I'll put it there,
but just like don't take the
554
00:36:43 --> 00:36:47
videotape quite so carefully.
555
00:36:47 --> 00:36:47
OK.
556
00:36:47 --> 00:36:55
So if I put that thing it -- it
would be a a inverse a transpose
557
00:36:55 --> 00:36:59.71
inverse a transpose and what's
that?
558
00:36:59.71 --> 00:37:02
That's the identity.
559
00:37:02 --> 00:37:04
But what's going on?
560
00:37:04 --> 00:37:11
So why -- you see my question
is somehow I did something
561
00:37:11 --> 00:37:11.92
wrong.
562
00:37:11.92 --> 00:37:16
That that wasn't allowed.
563
00:37:16 --> 00:37:18
And and and why is that?
564
00:37:18 --> 00:37:22.21
Because a is not a square
matrix.
565
00:37:22.21 --> 00:37:24.86
a is not a square matrix.
566
00:37:24.86 --> 00:37:27
It doesn't have an inverse.
567
00:37:27 --> 00:37:31
So I have to leave it that way.
568
00:37:31 --> 00:37:32
This is not OK.
569
00:37:32 --> 00:37:38
If if a was a square invertible
matrix, then then I couldn't
570
00:37:38 --> 00:37:39
complain.
571
00:37:39 --> 00:37:45
Yeah I think -- let me think
about that case.
572
00:37:45 --> 00:37:50
But you but my main case,
the whole reason I'm doing all
573
00:37:50 --> 00:37:54
this, is that a is this matrix
that has x too many rows,
574
00:37:54 --> 00:37:58
it's just got a couple of
columns, like a one and a two,
575
00:37:58 --> 00:38:00
but lots of rows.
576
00:38:00 --> 00:38:00
Not square.
577
00:38:00 --> 00:38:04
And if it's not square,
this a transpose a is square
578
00:38:04 --> 00:38:08
but I can't pull it apart like
this --
579
00:38:08 --> 00:38:15
I'm not allowed to do this pull
apart, except if a was square.
580
00:38:15 --> 00:38:21
Now if a is square what's
what's going on if a is a square
581
00:38:21 --> 00:38:24
matrix?
a nice square inv- invertible
582
00:38:24 --> 00:38:25
matrix.
583
00:38:25 --> 00:38:26
Think.
584
00:38:26 --> 00:38:31
What's up with that what's with
that case.
585
00:38:31 --> 00:38:36
So this is that the formula
ought to work then too.
586
00:38:36 --> 00:38:41
If a is a nice square
invertible matrix what's its
587
00:38:41 --> 00:38:44
column space,
so it's a nice n by n
588
00:38:44 --> 00:38:50
invertible everything great
matrix, what's its column space,
589
00:38:50 --> 00:38:51
the whole of R^n.
590
00:38:51 --> 00:38:57
So what's the projection matrix
if I'm projecting onto the whole
591
00:38:57 --> 00:38:59
space?
592
00:38:59 --> 00:39:01
It's the identity matrix right?
593
00:39:01 --> 00:39:06
If I'm projecting b onto the
whole space, not just onto a
594
00:39:06 --> 00:39:10
plane, but onto all of 3-D,
then b is already in the column
595
00:39:10 --> 00:39:15
space, the projection is the
identity, and this is gives me
596
00:39:15 --> 00:39:17
the correct formula,
P is I.
597
00:39:17 --> 00:39:21
But if I'm projecting onto a
subspace then I can't split
598
00:39:21 --> 00:39:26
those apart and I have to stay
with that formula.
599
00:39:26 --> 00:39:26
OK.
600
00:39:26 --> 00:39:32.52
And what can I say if -- so I
remember this formula for 1-D
601
00:39:32.52 --> 00:39:36
and that's what it looks like in
n dimensions.
602
00:39:36 --> 00:39:41
And what are the properties
that I expected for any
603
00:39:41 --> 00:39:43
projection matrix?
604
00:39:43 --> 00:39:46
And I still expect for this
one?
605
00:39:46 --> 00:39:50
That matrix should be symmetric
and it is.
606
00:39:50 --> 00:39:53.19
P transpose is P.
607
00:39:53.19 --> 00:39:57
Because if I transpose this,
this guy's symmetric,
608
00:39:57 --> 00:40:03
and its inverse is symmetric,
and if I transpose this one
609
00:40:03 --> 00:40:07
when I transpose it will pop up
there, become a,
610
00:40:07 --> 00:40:13
that a transpose will pop up
here, and I'm back to P again.
611
00:40:13 --> 00:40:18
And do we dare try the other
property which is P squared
612
00:40:18 --> 00:40:19
equal P?
613
00:40:19 --> 00:40:22
It's got to be right.
614
00:40:22 --> 00:40:30
Because we know geometrically
that the first projection pops
615
00:40:30 --> 00:40:38
us onto the column space and the
second one leaves us where we
616
00:40:38 --> 00:40:38
are.
617
00:40:38 --> 00:40:47
So I expect that if I multiply
by let me do it -- if I multiply
618
00:40:47 --> 00:40:51
by another P,
so there's another a,
619
00:40:51 --> 00:40:59
another a transpose a inverse a
transpose, can you --
620
00:40:59 --> 00:41:05
eight (a)-s in a row is quite
obscene but -- do you see that
621
00:41:05 --> 00:41:06
it works?
622
00:41:06 --> 00:41:10.92
So I'm squaring that so what do
I do-- how do I see that
623
00:41:10.92 --> 00:41:12
multiplication?
624
00:41:12 --> 00:41:17
Well, yeah, I just want to put
parentheses in good places,
625
00:41:17 --> 00:41:22
so I see what's happening,
yeah, here's an a transpose a
626
00:41:22 --> 00:41:27
sitting together --
so when that a transpose a
627
00:41:27 --> 00:41:31
multiplies its inverse,
all that stuff goes,
628
00:41:31 --> 00:41:32
right.
629
00:41:32 --> 00:41:37
And leaves just the a transpose
at the end, which is just what
630
00:41:37 --> 00:41:38
we want.
631
00:41:38 --> 00:41:40
So P squared equals P.
632
00:41:40 --> 00:41:44
So sure enough those two
properties hold.
633
00:41:44 --> 00:41:44
OK.
634
00:41:44 --> 00:41:50
OK we really have got now all
the formulas.
635
00:41:50 --> 00:41:54
x hat, the projection P,
and the projection matrix
636
00:41:54 --> 00:41:55
capital P.
637
00:41:55 --> 00:41:58
And now my job is to use them.
638
00:41:58 --> 00:41:59.13
OK.
639
00:41:59.13 --> 00:42:04
So when would I have a bunch of
equations, too many equations
640
00:42:04 --> 00:42:10
and yet I want the best answer
and the -- the most important
641
00:42:10 --> 00:42:16
example, the most common example
is if I have points so here's
642
00:42:16 --> 00:42:22
the --
here's the application.
643
00:42:22 --> 00:42:23
v squared.
644
00:42:23 --> 00:42:26
Fitting by a line.
645
00:42:26 --> 00:42:27
OK.
646
00:42:27 --> 00:42:36
So I'll start this application
today and there's more in it
647
00:42:36 --> 00:42:41
than I can do in this same
lecture.
648
00:42:41 --> 00:42:51
So that'll give me a chance to
recap the formulas and there
649
00:42:51 --> 00:42:57
they are, and recap the ideas.
650
00:42:57 --> 00:43:02
So let me start the problem
today.
651
00:43:02 --> 00:43:06
I'm given a bunch of data
points.
652
00:43:06 --> 00:43:11
And they lie close to a line
but not on a line.
653
00:43:11 --> 00:43:13
Let me take that.
654
00:43:13 --> 00:43:18
Say a t equal to one,
two and three,
655
00:43:18 --> 00:43:23
I have one, and two and two
again.
656
00:43:23 --> 00:43:30
So my data points are this is
the like the time direction and
657
00:43:30 --> 00:43:37
this is like well let me call
that b or y or something.
658
00:43:37 --> 00:43:43
I'm given these three points
and I want to fit them by a
659
00:43:43 --> 00:43:44.4
line.
660
00:43:44.4 --> 00:43:47
By the best straight line.
661
00:43:47 --> 00:43:53
So the problem is fit the
points one, one is the first
662
00:43:53 --> 00:43:58
point --
the second point is t equals
663
00:43:58 --> 00:44:03
two, b equal one,
and the third point is t equal
664
00:44:03 --> 00:44:06
three, b equal to two.
665
00:44:06 --> 00:44:11
So those are my three points,
t equal sorry,that's two.
666
00:44:11 --> 00:44:12
Yeah, OK.
667
00:44:12 --> 00:44:15.79
So this is the point one,
one.
668
00:44:15.79 --> 00:44:20
This is the point two,
two, and that's the point
669
00:44:20 --> 00:44:21
three, two.
670
00:44:21 --> 00:44:29
And of course there isn't a --
a line that goes through them.
671
00:44:29 --> 00:44:32
So I'm looking for the best
line.
672
00:44:32 --> 00:44:37
I'm looking for a line that
probably goes somewhere,
673
00:44:37 --> 00:44:40
do you think it goes somewhere
like that?
674
00:44:40 --> 00:44:45
I didn't mean to make it go
through that point,
675
00:44:45 --> 00:44:46
it won't.
676
00:44:46 --> 00:44:51
It'll kind of --
it'll go between so the error
677
00:44:51 --> 00:44:57
there and the error there and
the error there are as small as
678
00:44:57 --> 00:44:58.65
I can get them.
679
00:44:58.65 --> 00:45:02
OK, what I'd like to do is find
the matrix a.
680
00:45:02 --> 00:45:08
Because once I've found the
matrix a the formulas take over.
681
00:45:08 --> 00:45:13
So what I'm looking for this
line, b is C+Dt.
682
00:45:13 --> 00:45:18
So this is in the homework that
I sent out for today.
683
00:45:18 --> 00:45:19
Find the best line.
684
00:45:19 --> 00:45:22
So I'm looking for these
numbers.
685
00:45:22 --> 00:45:23
C and D.
686
00:45:23 --> 00:45:28
That tell me the line and I
want them to be as close to
687
00:45:28 --> 00:45:32
going through those three points
as I can get.
688
00:45:32 --> 00:45:36
I can't get exactly so there
are three equations to go
689
00:45:36 --> 00:45:39
through the three points.
690
00:45:39 --> 00:45:45
It would it will go exactly
through that point if let's see
691
00:45:45 --> 00:45:50
that first point has t equal to
one, so that would say C+D
692
00:45:50 --> 00:45:51
equaled 1.
693
00:45:51 --> 00:45:52.82
This is the one,
one.
694
00:45:52.82 --> 00:45:55
The second point t is two.
695
00:45:55 --> 00:45:58
So C+2D should come out to
equal 2.
696
00:45:58 --> 00:46:03
But I also want to get the
third equation in and at that
697
00:46:03 --> 00:46:08
third equation t is three so
C+3D equals only 2.
698
00:46:08 --> 00:46:09.96
That's the key.
699
00:46:09.96 --> 00:46:15
Is to write down what equations
we would like to solve but
700
00:46:15 --> 00:46:16
can't.
701
00:46:16 --> 00:46:22.12
Reason we if we could solve
them that would mean that we
702
00:46:22.12 --> 00:46:27
could put a line through all
three points and of course if
703
00:46:27 --> 00:46:32
these numbers one,
two, two were different,
704
00:46:32 --> 00:46:35
we could do it.
705
00:46:35 --> 00:46:38
But with those numbers,
one, two, two,
706
00:46:38 --> 00:46:39
we can't.
707
00:46:39 --> 00:46:45.78
So what is our equation Ax
equal Ax equal b that we can't
708
00:46:45.78 --> 00:46:46
solve?
709
00:46:46 --> 00:46:52
I just want to say what's the
matrix here, what's the unknown
710
00:46:52 --> 00:46:57
x, and what's the right-hand
side.
711
00:46:57 --> 00:47:02
So this is the matrix is one,
one, one, one,
712
00:47:02 --> 00:47:03
two, three.
713
00:47:03 --> 00:47:06
The unknown is C and D.
714
00:47:06 --> 00:47:10
And the right-hand side if one,
two, two.
715
00:47:10 --> 00:47:17
Right, I've just taken my
equations and I've said what is
716
00:47:17 --> 00:47:19
Ax and what is b.
717
00:47:19 --> 00:47:25
Then there's no solution,
this is the typical case where
718
00:47:25 --> 00:47:31
I have three equations --
two unknowns,
719
00:47:31 --> 00:47:36
no solution,
but I'm still looking for the
720
00:47:36 --> 00:47:38
best solution.
721
00:47:38 --> 00:47:45
And the best solution is taken
is is to solve not this equation
722
00:47:45 --> 00:47:52
Ax equal b which has which has
no solution but the equation
723
00:47:52 --> 00:47:57
that does have a solution,
which was this one.
724
00:47:57 --> 00:48:03
So that's the equation to
solve.
725
00:48:03 --> 00:48:07
That's the central equation of
the subject.
726
00:48:07 --> 00:48:11
I can't solve Ax=b but
magically when I multiply both
727
00:48:11 --> 00:48:17
sides by a transpose I get an
equation that I can solve and
728
00:48:17 --> 00:48:21
its solution gives me x,
the best x, the best
729
00:48:21 --> 00:48:26
projection, and I discover
what's the matrix that's behind
730
00:48:26 --> 00:48:27
it.
731
00:48:27 --> 00:48:27
OK.
732
00:48:27 --> 00:48:32
So next time I'll complete an
example, numerical example.
733
00:48:32 --> 00:48:36
today was all letters,
numbers next time.
734
00:48:36 --> 00:48:39
Thanks.