1
00:00:07 --> 00:00:14
OK, guys the -- we're almost
ready to make this lecture

2
00:00:14 --> 00:00:15
immortal.

3
00:00:15 --> 00:00:15
OK.

4
00:00:15 --> 00:00:16
Are we on?

5
00:00:16 --> 00:00:17
All right.

6
00:00:17 --> 00:00:21
This is an important lecture.

7
00:00:21 --> 00:00:23
It's about projections.

8
00:00:23 --> 00:00:30
Let me start by just projecting
a vector b down on a vector a.

9
00:00:30 --> 00:00:38.07
So just to so you see what the
geometry looks like in when I'm

10
00:00:38.07 --> 00:00:41
in --
in just two dimensions,

11
00:00:41 --> 00:00:46
I'd like to find the point
along this line so that line

12
00:00:46 --> 00:00:51
through a is a one-dimensional
subspace, so I'm starting with

13
00:00:51 --> 00:00:52
one dimension.

14
00:00:52 --> 00:00:56
I'd like to find the point on
that line closest to a.

15
00:00:56 --> 00:01:00
Can I just take that problem
first and then I'll explain why

16
00:01:00 --> 00:01:06
I want to do it and why I want
to project on other subspaces.

17
00:01:06 --> 00:01:12
So where's the point closest to
b that's on that line?

18
00:01:12 --> 00:01:15.16
It's somewhere there.

19
00:01:15.16 --> 00:01:22
And let me connect that and --
and what's the whole point of my

20
00:01:22 --> 00:01:23
picture now?

21
00:01:23 --> 00:01:29
What's the -- where does
orthogonality come into this

22
00:01:29 --> 00:01:31
picture?

23
00:01:31 --> 00:01:35
The whole point is that this
best point, that's the

24
00:01:35 --> 00:01:38
projection, P,
of b onto the line,

25
00:01:38 --> 00:01:40
where's orthogonality?

26
00:01:40 --> 00:01:43
It's the fact that that's a
right angle.

27
00:01:43 --> 00:01:49
That this -- the error -- this
is like how much I'm wrong by --

28
00:01:49 --> 00:01:55
this is the difference between b
and P, the whole point is --

29
00:01:55 --> 00:01:57
that's perpendicular to a.

30
00:01:57 --> 00:02:01
That's got to give us the
equation.

31
00:02:01 --> 00:02:06
That's got to tell us -- that's
the one fact we know,

32
00:02:06 --> 00:02:10
that's got to tell us where
that projection is.

33
00:02:10 --> 00:02:14
Let me also say,
look -- I've drawn a triangle

34
00:02:14 --> 00:02:16
there.

35
00:02:16 --> 00:02:21
So if we were doing
trigonometry we would do like we

36
00:02:21 --> 00:02:27
would have angles theta and
distances that would involve

37
00:02:27 --> 00:02:33
sine theta and cos theta that
leads to lousy formulas compared

38
00:02:33 --> 00:02:35
to linear algebra.

39
00:02:35 --> 00:02:42
The formula that we want comes
out nicely and what's the --

40
00:02:42 --> 00:02:43
what do we know?

41
00:02:43 --> 00:02:46
We know that P,
this projection,

42
00:02:46 --> 00:02:49
is some multiple of a,
right?

43
00:02:49 --> 00:02:50
It's on that line.

44
00:02:50 --> 00:02:54
So we know it's in that
one-dimensional subspace,

45
00:02:54 --> 00:02:59
it's some multiple,
let me call that multiple x,

46
00:02:59 --> 00:02:59
of a.

47
00:02:59 --> 00:03:04
So really it's that number x
I'd like to find.

48
00:03:04 --> 00:03:11
So this is going to be simple
in 1-D, so let's just carry it

49
00:03:11 --> 00:03:16
through, and then see how it
goes in high dimensions.

50
00:03:16 --> 00:03:17
OK.

51
00:03:17 --> 00:03:22
The key fact is -- the key to
everything is that

52
00:03:22 --> 00:03:23
perpendicular.

53
00:03:23 --> 00:03:28.27
The fact that a is
perpendicular to a is

54
00:03:28.27 --> 00:03:31
perpendicular to e.

55
00:03:31 --> 00:03:33
Which is (b-ax),
xa.

56
00:03:33 --> 00:03:36
I don't care what -- xa.

57
00:03:36 --> 00:03:38
That that equals zero.

58
00:03:38 --> 00:03:44
Do you see that as the central
equation, that's saying that

59
00:03:44 --> 00:03:48
this a is perpendicular to this
-- correction,

60
00:03:48 --> 00:03:52
that's going to tell us what x
is.

61
00:03:52 --> 00:03:58
Let me just raise the board and
simplify that and out will come

62
00:03:58 --> 00:04:00
x.

63
00:04:00 --> 00:04:00
OK.

64
00:04:00 --> 00:04:05
So if I simplify that,
let's see, I'll move one to --

65
00:04:05 --> 00:04:10
one term to one side,
the other term will be on the

66
00:04:10 --> 00:04:15
other side, it looks to me like
x times a transpose a is equal

67
00:04:15 --> 00:04:17
to a transpose b.

68
00:04:17 --> 00:04:17
Right?

69
00:04:17 --> 00:04:23
I have a transpose b as one f-
one term, a transpose a as the

70
00:04:23 --> 00:04:28
other, so right away here's my a
transpose a.

71
00:04:28 --> 00:04:31
But it's just a number now.

72
00:04:31 --> 00:04:33
And I divide by it.

73
00:04:33 --> 00:04:38
And I get the answer.
x is a transpose b over a

74
00:04:38 --> 00:04:39
transpose a.

75
00:04:39 --> 00:04:44.73
And P, the projection I wanted,
is -- that's the right

76
00:04:44.73 --> 00:04:45
multiple.

77
00:04:45 --> 00:04:49
That's got a cosine theta built
in.

78
00:04:49 --> 00:04:53
But we don't need to look at
angles.

79
00:04:53 --> 00:04:56
It's -- we've just got vectors
here.

80
00:04:56 --> 00:05:00
And the projection is P is a
times that x.

81
00:05:00 --> 00:05:02
Or x times that a.

82
00:05:02 --> 00:05:07
But I'm really going to --
eventually I'm going to want

83
00:05:07 --> 00:05:10
that x coming on the right-hand
side.

84
00:05:10 --> 00:05:16
So do you see that I've got two
of the three formulas already,

85
00:05:16 --> 00:05:23
right here, I've got the --
that's the equation -- that

86
00:05:23 --> 00:05:28.66
leads me to the answer,
here's the answer for x,

87
00:05:28.66 --> 00:05:31
and here's the projection.

88
00:05:31 --> 00:05:35
OK.
can I do add just one more

89
00:05:35 --> 00:05:40.6
thing to this one-dimensional
problem?

90
00:05:40.6 --> 00:05:45
One more like lift it up into
linear algebra,

91
00:05:45 --> 00:05:46
into matrices.

92
00:05:46 --> 00:05:53
Here's the last thing I want to
do -- but don't forget those

93
00:05:53 --> 00:05:57
formulas.
a transpose b over a transpose

94
00:05:57 --> 00:05:57
a.

95
00:05:57 --> 00:06:02
Actually let's look at that for
a moment first.

96
00:06:02 --> 00:06:08
Suppose --
Let me take this next step.

97
00:06:08 --> 00:06:09
So P is a times x.

98
00:06:09 --> 00:06:12.55
So can I write that then?

99
00:06:12.55 --> 00:06:18
P is a times this neat number,
a transpose b over a transpose

100
00:06:18 --> 00:06:19
a.

101
00:06:19 --> 00:06:21.4
That's our projection.

102
00:06:21.4 --> 00:06:27
Can I ask a couple of questions
about it, just while we look,

103
00:06:27 --> 00:06:31
get that digest that formula.

104
00:06:31 --> 00:06:34
Suppose b is doubled.

105
00:06:34 --> 00:06:37
Suppose I change b to two b.

106
00:06:37 --> 00:06:40
What happens to the projection?

107
00:06:40 --> 00:06:46
So suppose I instead of that
vector b that I drew on the

108
00:06:46 --> 00:06:52
board make it two b,
twice as long -- what's the

109
00:06:52 --> 00:06:53
projection now?

110
00:06:53 --> 00:06:57
It's doubled too,
right?

111
00:06:57 --> 00:07:02
It's going to be twice as far,
if b goes twice as far,

112
00:07:02 --> 00:07:05
the projection will go twice as
far.

113
00:07:05 --> 00:07:07
And you see it there.

114
00:07:07 --> 00:07:12
If I put in an extra factor
two, then P's got that factor

115
00:07:12 --> 00:07:12
too.

116
00:07:12 --> 00:07:15.35
Now what about if I double a?

117
00:07:15.35 --> 00:07:20
What if I double the vector a
that I'm projecting onto?

118
00:07:20 --> 00:07:22
What changes?

119
00:07:22 --> 00:07:25
The projection doesn't change
at all.

120
00:07:25 --> 00:07:25
Right?

121
00:07:25 --> 00:07:29
Because I'm just -- the line
didn't change.

122
00:07:29 --> 00:07:31
If I double a or I take minus
a.

123
00:07:31 --> 00:07:33
It's still that same line.

124
00:07:33 --> 00:07:36
The projection's still in the
same place.

125
00:07:36 --> 00:07:40
And of course if I double a I
get a four up above,

126
00:07:40 --> 00:07:45
and I get a four --
an extra four below,

127
00:07:45 --> 00:07:50
they cancel out,
and the projection is the same.

128
00:07:50 --> 00:07:50
OK.

129
00:07:50 --> 00:07:57
So really, this -- I want to
look at this as the projection

130
00:07:57 --> 00:07:59
-- there's a matrix here.

131
00:07:59 --> 00:08:06.58
The projection is carried out
by some matrix that I'm going to

132
00:08:06.58 --> 00:08:10.62
call the projection matrix.

133
00:08:10.62 --> 00:08:14
And in other words the
projection is some matrix that

134
00:08:14 --> 00:08:18.28
acts on this guy b and produces
the projection.

135
00:08:18.28 --> 00:08:22
The projection P is the
projection matrix acting on

136
00:08:22 --> 00:08:23
whatever the input is.

137
00:08:23 --> 00:08:27
The input is b,
the projection matrix is P.

138
00:08:27 --> 00:08:27
OK.

139
00:08:27 --> 00:08:32
Actually you can tell me right
away what this projection matrix

140
00:08:32 --> 00:08:32
is.

141
00:08:32 --> 00:08:36
So this is a pretty interesting
matrix.

142
00:08:36 --> 00:08:40
What matrix is multiplying b?

143
00:08:40 --> 00:08:47
I'm just -- just from my
formula -- I see what P is.

144
00:08:47 --> 00:08:53
P, this projection matrix,
is -- what is it?

145
00:08:53 --> 00:09:01
I see a a transpose above,
and I see a transpose a below.

146
00:09:01 --> 00:09:05
And those don't cancel.

147
00:09:05 --> 00:09:06
That's not one.

148
00:09:06 --> 00:09:07
Right?

149
00:09:07 --> 00:09:08
That's a matrix.

150
00:09:08 --> 00:09:11.42
Because down here,
the a transpose a,

151
00:09:11.42 --> 00:09:14
that's just a number,
a transpose a,

152
00:09:14 --> 00:09:19
that's the length of a squared,
and up above is a column times

153
00:09:19 --> 00:09:19
a row.

154
00:09:19 --> 00:09:22.3
Column times a row is a matrix.

155
00:09:22.3 --> 00:09:27
So this is a full-scale n by n
matrix, if I --

156
00:09:27 --> 00:09:29
if I'm in n dimensions.

157
00:09:29 --> 00:09:32.32
And it's kind of an interesting
one.

158
00:09:32.32 --> 00:09:37
And it's the one which if I
multiply by b then I get this,

159
00:09:37 --> 00:09:43
you see once again I'm putting
parentheses in different places.

160
00:09:43 --> 00:09:46.69
I'm putting the parentheses
right there.

161
00:09:46.69 --> 00:09:50
I'm saying OK,
that's really the matrix that

162
00:09:50 --> 00:09:53
produces this projection.

163
00:09:53 --> 00:09:54
OK.

164
00:09:54 --> 00:09:59.75
Now, tell me -- all right,
what are the properties of that

165
00:09:59.75 --> 00:10:00
matrix?

166
00:10:00 --> 00:10:05.34
I'm just using letters here,
a and b, I could put in

167
00:10:05.34 --> 00:10:11
numbers, but I think it's -- for
once, it's clearer with letters,

168
00:10:11 --> 00:10:16
because all formulas are
simple, a transpose b over a

169
00:10:16 --> 00:10:20
transpose a --
that's the number that

170
00:10:20 --> 00:10:24
multiplies the a,
and then I see wait a minute,

171
00:10:24 --> 00:10:28
there's a matrix here and
what's the rank of that matrix,

172
00:10:28 --> 00:10:29
by the way?

173
00:10:29 --> 00:10:34
What's the rank of that matrix,
yeah -- let me just ask you

174
00:10:34 --> 00:10:35
about that matrix.

175
00:10:35 --> 00:10:40
Which looks a little strange,
a a transpose over this number.

176
00:10:40 --> 00:10:44
But well, I could ask you its
column space.

177
00:10:44 --> 00:10:47
Yeah, let me ask you its column
space.

178
00:10:47 --> 00:10:50
So what's the column space of a
matrix?

179
00:10:50 --> 00:10:54
If you multiply that matrix by
anything you always get in the

180
00:10:54 --> 00:10:56
column space,
right?

181
00:10:56 --> 00:11:01
The column space of a matrix is
when you multiply any vector by

182
00:11:01 --> 00:11:04
that matrix --
any vector b,

183
00:11:04 --> 00:11:09
by the matrix,
you always land in the column

184
00:11:09 --> 00:11:10.44
space.

185
00:11:10.44 --> 00:11:14
That's what column spaces work.

186
00:11:14 --> 00:11:18.26
Now what space do we always
land in?

187
00:11:18.26 --> 00:11:24.54
What's the column space of --
what's the result when I

188
00:11:24.54 --> 00:11:30
multiply this any vector b by my
matrix?

189
00:11:30 --> 00:11:34.01
So I have P times b,
where I?

190
00:11:34.01 --> 00:11:36
I'm on that line,
right?

191
00:11:36 --> 00:11:42.62
The column space,
so here are facts about this

192
00:11:42.62 --> 00:11:43
matrix.

193
00:11:43 --> 00:11:49
The column space of P,
of this projection matrix,

194
00:11:49 --> 00:11:52
is the line through a.

195
00:11:52 --> 00:12:00
And the rank of this matrix is
you can all say it at once one.

196
00:12:00 --> 00:12:00
Right.

197
00:12:00 --> 00:12:04.65
The rank is one.

198
00:12:04.65 --> 00:12:07
This is a rank one matrix.

199
00:12:07 --> 00:12:14
Actually it's exactly the form
that we're familiar with a rank

200
00:12:14 --> 00:12:15
one matrix.

201
00:12:15 --> 00:12:20.25
A column times a row,
that's a rank one matrix,

202
00:12:20.25 --> 00:12:25
that column is the basis for
the column space.

203
00:12:25 --> 00:12:28
Just one dimension.

204
00:12:28 --> 00:12:28
OK.

205
00:12:28 --> 00:12:31
So I know that much about the
matrix.

206
00:12:31 --> 00:12:37
But now there are two more
facts about the matrix that I

207
00:12:37 --> 00:12:38
want to notice.

208
00:12:38 --> 00:12:42
First of all is the matrix
symmetric?

209
00:12:42 --> 00:12:46.13
That's a natural question for
matrices.

210
00:12:46.13 --> 00:12:48
And the answer is yes.

211
00:12:48 --> 00:12:52
If I take the transpose of this
--

212
00:12:52 --> 00:12:58
there's a number down there,
the transpose of a a transpose

213
00:12:58 --> 00:12:59
is a a transpose.

214
00:12:59 --> 00:13:01
So P is symmetric.

215
00:13:01 --> 00:13:03
P transpose equals P.

216
00:13:03 --> 00:13:05
So this is a key property.

217
00:13:05 --> 00:13:09
That the projection matrix is
symmetric.

218
00:13:09 --> 00:13:13
One more property now and this
is the real one.

219
00:13:13 --> 00:13:18
What happens if I do the
projection twice?

220
00:13:18 --> 00:13:23
So I'm looking for something,
some information about P

221
00:13:23 --> 00:13:24
squared.

222
00:13:24 --> 00:13:28
But just give me in terms of
that picture,

223
00:13:28 --> 00:13:32
in terms my picture,
I take any vector b,

224
00:13:32 --> 00:13:39
I multiply it by my projection
matrix, and that puts me there,

225
00:13:39 --> 00:13:40
so this is Pb.

226
00:13:40 --> 00:13:44
And now I project again.

227
00:13:44 --> 00:13:46
What happens now?

228
00:13:46 --> 00:13:52
What happens when I apply the
projection matrix a second time?

229
00:13:52 --> 00:13:58.45
To this, so I'm applying it
once brings me here and the

230
00:13:58.45 --> 00:14:01
second time brings me I stay
put.

231
00:14:01 --> 00:14:02
Right?

232
00:14:02 --> 00:14:08
The projection for a point on
this line the projection is

233
00:14:08 --> 00:14:11
right where it is.

234
00:14:11 --> 00:14:14
The projection is the same
point.

235
00:14:14 --> 00:14:19
So that means that if I project
twice, I get the same answer as

236
00:14:19 --> 00:14:22
I did in the first projection.

237
00:14:22 --> 00:14:27
So those are the two properties
that tell me I'm looking at a

238
00:14:27 --> 00:14:28
projection matrix.

239
00:14:28 --> 00:14:32
It's symmetric and it's square
is itself.

240
00:14:32 --> 00:14:37
Because if I project a second
time, it's the same result as

241
00:14:37 --> 00:14:39
the first result.

242
00:14:39 --> 00:14:40.25
OK.

243
00:14:40.25 --> 00:14:46
So that's -- and then here's
the exact formula when I know

244
00:14:46 --> 00:14:50.92
what I'm projecting onto,
that line through a,

245
00:14:50.92 --> 00:14:53
then I know what P is.

246
00:14:53 --> 00:14:59
So do you see that I have all
the pieces here for projection

247
00:14:59 --> 00:15:00
on a line?

248
00:15:00 --> 00:15:05
Now, and those --
please remember those.

249
00:15:05 --> 00:15:08
So there are three formulas to
remember.

250
00:15:08 --> 00:15:12
The formula for x,
the formula for P,

251
00:15:12 --> 00:15:16
which is just ax,
and then the formula for

252
00:15:16 --> 00:15:18
capital P, which is the matrix.

253
00:15:18 --> 00:15:19
Good.

254
00:15:19 --> 00:15:19
Good.

255
00:15:19 --> 00:15:20
OK.

256
00:15:20 --> 00:15:23
Now I want to move to more
dimensions.

257
00:15:23 --> 00:15:28
So we're going to have three
formulas again but you'll have

258
00:15:28 --> 00:15:35
to --
they'll be a little different

259
00:15:35 --> 00:15:43
because we won't have a single
line but -- a plane or

260
00:15:43 --> 00:15:50
three-dimensional or a
n-dimensional subspace.

261
00:15:50 --> 00:15:50
OK.

262
00:15:50 --> 00:15:56
So now I'll move to the next
question.

263
00:15:56 --> 00:16:06
Maybe -- let me say first why I
want this projection,

264
00:16:06 --> 00:16:16
and then we'll figure out what
it is, we'll go completely in

265
00:16:16 --> 00:16:22.27
parallel there,
and then we'll use it.

266
00:16:22.27 --> 00:16:27
OK, why do I want this
projection?

267
00:16:27 --> 00:16:31
Well, so why project?

268
00:16:31 --> 00:16:41
It's because I'm as I mentioned
last time this new chapter deals

269
00:16:41 --> 00:16:50
with equations Ax=b may have no
solution.

270
00:16:50 --> 00:16:56
So that's really my problem,
that I'm given a bunch of

271
00:16:56 --> 00:17:02
equations probably too many
equations, more equations than

272
00:17:02 --> 00:17:06.7
unknowns, and I can't solve
them.

273
00:17:06.7 --> 00:17:07
OK.

274
00:17:07 --> 00:17:08
So what do I do?

275
00:17:08 --> 00:17:14
I solve the closest problem
that I can solve.

276
00:17:14 --> 00:17:18
And what's the closest one?

277
00:17:18 --> 00:17:24
Well, ax will always be in the
column space of a.

278
00:17:24 --> 00:17:27
That's my problem.

279
00:17:27 --> 00:17:34
My problem is ax has to be in
the column space and b is

280
00:17:34 --> 00:17:38
probably not in the column
space.

281
00:17:38 --> 00:17:41
So I change b to what?

282
00:17:41 --> 00:17:47
I choose the closest vector in
the column space,

283
00:17:47 --> 00:17:51.48
so I'll solve Ax equal P
instead.

284
00:17:51.48 --> 00:17:55
That one I can do.

285
00:17:55 --> 00:18:00
Where P is this is the
projection of b onto the column

286
00:18:00 --> 00:18:01
space.

287
00:18:01 --> 00:18:05
That's why I want to be able to
do this.

288
00:18:05 --> 00:18:09
Because I have to find a
solution here,

289
00:18:09 --> 00:18:15.1
and I'm going to put a little
hat there to indicate that it's

290
00:18:15.1 --> 00:18:19
not the x, it's not the x that
doesn't exist,

291
00:18:19 --> 00:18:24
it's the x hat that's best
possible.

292
00:18:24 --> 00:18:30
So I must be able to figure out
what's the good projection

293
00:18:30 --> 00:18:31
there.

294
00:18:31 --> 00:18:38
What's the good right-hand side
that is in the column space

295
00:18:38 --> 00:18:45
that's as close as possible to b
and then I'm -- then I know what

296
00:18:45 --> 00:18:46
to do.

297
00:18:46 --> 00:18:47
OK.

298
00:18:47 --> 00:18:51
So now I've got that problem.

299
00:18:51 --> 00:18:58
So that's why I have the
problem again but now let me say

300
00:18:58 --> 00:19:03
I'm in three dimensions,
so I have a plane maybe for

301
00:19:03 --> 00:19:10
example, and I have a vector b
that's not in the plane.

302
00:19:10 --> 00:19:14
And I want to project b down
into the plane.

303
00:19:14 --> 00:19:15
OK.

304
00:19:15 --> 00:19:17
So there's my question.

305
00:19:17 --> 00:19:23
How do I project a vector and
I'm --

306
00:19:23 --> 00:19:28
what I'm looking for is a nice
formula, and I'm counting on

307
00:19:28 --> 00:19:33
linear algebra to just come out
right, a nice formula for the

308
00:19:33 --> 00:19:36
projection of b into the plane.

309
00:19:36 --> 00:19:37
The nearest point.

310
00:19:37 --> 00:19:42
So this again a right angle is
going to be crucial.

311
00:19:42 --> 00:19:42
OK.

312
00:19:42 --> 00:19:47
Now so what's -- first of all
I've got to say what is that

313
00:19:47 --> 00:19:48
plane.

314
00:19:48 --> 00:19:53
To get a formula I have to tell
you what the plane is.

315
00:19:53 --> 00:19:55
How I going to tell you a
plane?

316
00:19:55 --> 00:20:01
I'll tell you a basis for the
plane, I'll tell you two vectors

317
00:20:01 --> 00:20:05
a one and a two that give you a
basis for the plane,

318
00:20:05 --> 00:20:10
so that -- let us say -- say
there's an a one and here's an a

319
00:20:10 --> 00:20:12
-- a vector a two.

320
00:20:12 --> 00:20:17
They don't have to be
perpendicular.

321
00:20:17 --> 00:20:24
But they better be independent,
because then that tells me the

322
00:20:24 --> 00:20:24
plane.

323
00:20:24 --> 00:20:30
The plane is the -- is the
plane of a one and a two.

324
00:20:30 --> 00:20:37
And actually going back to my
-- to this connection,

325
00:20:37 --> 00:20:43
this plane is a column space,
it's the column space of what

326
00:20:43 --> 00:20:46
matrix?

327
00:20:46 --> 00:20:50
What matrix,
so how do I connect the two

328
00:20:50 --> 00:20:52
questions?

329
00:20:52 --> 00:20:59
I'm thinking how do I project
onto a plane and I want to get a

330
00:20:59 --> 00:21:01
matrix in here.

331
00:21:01 --> 00:21:08
Everything's cleaner if I write
it in terms of a matrix.

332
00:21:08 --> 00:21:14
So what matrix has these --
has that column space?

333
00:21:14 --> 00:21:19.65
Well of course it's just the
matrix that has a one in the

334
00:21:19.65 --> 00:21:23
first column and a two in the
second column.

335
00:21:23 --> 00:21:29
Right, just just let's be sure
we've got the question before we

336
00:21:29 --> 00:21:31
get to the answer.

337
00:21:31 --> 00:21:35
So I'm looking for again I'm
given a matrix a with two

338
00:21:35 --> 00:21:37
columns.

339
00:21:37 --> 00:21:42
And really I'm ready once I get
to two I'm ready for n.

340
00:21:42 --> 00:21:46
So it could be two columns,
it could be n columns.

341
00:21:46 --> 00:21:51
I'll write the answer in terms
of the matrix a.

342
00:21:51 --> 00:21:56
And the point will be those two
columns describe the plane,

343
00:21:56 --> 00:22:01
they describe the column space,
and I want to project.

344
00:22:01 --> 00:22:02
OK.

345
00:22:02 --> 00:22:07
And I'm given a vector b that's
probably not in the column

346
00:22:07 --> 00:22:07
space.

347
00:22:07 --> 00:22:11
Of course, if b is in the
column space,

348
00:22:11 --> 00:22:14
my projection is simple,
it's just b.

349
00:22:14 --> 00:22:19
But most likely I have an error
e, this b minus P part,

350
00:22:19 --> 00:22:21
which is probably not zero.

351
00:22:21 --> 00:22:21.84
OK.

352
00:22:21.84 --> 00:22:27
But the beauty is that I know
-- from geometry or I could get

353
00:22:27 --> 00:22:32
it from calculus or I could get
it from linear algebra that that

354
00:22:32 --> 00:22:40
this this vector --
this is the part of b that's

355
00:22:40 --> 00:22:45
that's perpendicular to the
plane.

356
00:22:45 --> 00:22:54
That e is perpendicular is
perpendicular to the plane.

357
00:22:54 --> 00:23:02
If your intuition is saying
that that's the crucial fact.

358
00:23:02 --> 00:23:09
That's going to give us the
answer.

359
00:23:09 --> 00:23:10
OK.

360
00:23:10 --> 00:23:13
So let me, that's the problem.

361
00:23:13 --> 00:23:15
Now for the answer.

362
00:23:15 --> 00:23:21
So this is a lecture that's
really like moving along.

363
00:23:21 --> 00:23:29
Because I'm just plotting that
problem up there and asking you

364
00:23:29 --> 00:23:34.32
what combination -- now,
yeah, so what is it?

365
00:23:34.32 --> 00:23:34
P.

366
00:23:34 --> 00:23:38
What is this projection P?

367
00:23:38 --> 00:23:43
This is projection P,
is some multiple of these basis

368
00:23:43 --> 00:23:47
guys, right, some multiple of
the columns.

369
00:23:47 --> 00:23:52
But I don't like writing out x
one a one plus x two a two,

370
00:23:52 --> 00:23:55.7
I would rather right that as
ax.

371
00:23:55.7 --> 00:24:00
Well, actually I should put if
I'm really doing everything

372
00:24:00 --> 00:24:05
right, I should put a little hat
on it --

373
00:24:05 --> 00:24:11
to remember that this x -- that
those are the numbers and I

374
00:24:11 --> 00:24:16
could have a put a hat way back
there is right,

375
00:24:16 --> 00:24:20
so this is this is the
projection, P.

376
00:24:20 --> 00:24:21
P is ax bar.

377
00:24:21 --> 00:24:23
And I'm looking for x bar.

378
00:24:23 --> 00:24:27
So that's what I want an
equation for.

379
00:24:27 --> 00:24:32
So now I've got hold of the
problem.

380
00:24:32 --> 00:24:39
The problem is find the right
combination of the columns so

381
00:24:39 --> 00:24:45
that the error vector is
perpendicular to the plane.

382
00:24:45 --> 00:24:49
Now let me turn that into an
equation.

383
00:24:49 --> 00:24:56
So I'll raise the board and
just turn that -- what we've

384
00:24:56 --> 00:24:59
just done into an equation.

385
00:24:59 --> 00:25:05
So let me I'll write again the
main point.

386
00:25:05 --> 00:25:09
The projection is ax b- x hat.

387
00:25:09 --> 00:25:13
And our problem is find x hat.

388
00:25:13 --> 00:25:21
And the key is that b minus ax
hat, that's the error.

389
00:25:21 --> 00:25:23
This is the e.

390
00:25:23 --> 00:25:27
Is perpendicular to the plane.

391
00:25:27 --> 00:25:33
That's got to give me well what
I looking for,

392
00:25:33 --> 00:25:42
I'm looking for two equations
now because I've got an x one

393
00:25:42 --> 00:25:45
and an x two.

394
00:25:45 --> 00:25:49
And I'll get two equations
because so this thing e is

395
00:25:49 --> 00:25:51
perpendicular to the plane.

396
00:25:51 --> 00:25:53
So what does that mean?

397
00:25:53 --> 00:25:57
I guess it means it's
perpendicular to a one and also

398
00:25:57 --> 00:25:58
to a two.

399
00:25:58 --> 00:26:03.27
Right, those are two vectors in
the plane and the things that

400
00:26:03.27 --> 00:26:08.18
are perpendicular to the plane
are perpendicular to a one and a

401
00:26:08.18 --> 00:26:09
two.

402
00:26:09 --> 00:26:11
Let me just repeat.

403
00:26:11 --> 00:26:15
This this guy then is
perpendicular to the plane so

404
00:26:15 --> 00:26:19
it's perpendicular to that
vector and that vector.

405
00:26:19 --> 00:26:23
Not -- it's perpendicular to
that of course.

406
00:26:23 --> 00:26:27
But it's perpendicular to
everything I the plane.

407
00:26:27 --> 00:26:32
And the plane is really told me
by a one and a two.

408
00:26:32 --> 00:26:40
So really I have the equations
a one transpose b minus ax is

409
00:26:40 --> 00:26:41
zero.

410
00:26:41 --> 00:26:47
And also a two transpose b
minus ax is zero.

411
00:26:47 --> 00:26:51
Those are my two equations.

412
00:26:51 --> 00:26:55
But I want those in matrix
form.

413
00:26:55 --> 00:27:02
I want to put those two
equations together as a matrix

414
00:27:02 --> 00:27:09
equation and it's just comes out
right.

415
00:27:09 --> 00:27:14
Look at the matrix a transpose.

416
00:27:14 --> 00:27:21
Put a one a one transpose is
its first row,

417
00:27:21 --> 00:27:30
a two transpose is its second
row, that multiplies this b-ax,

418
00:27:30 --> 00:27:36
and gives me the zero and the
zero.

419
00:27:36 --> 00:27:45
I'm you see the -- this is one
way -- to come up with this

420
00:27:45 --> 00:27:48.22
equation.

421
00:27:48.22 --> 00:27:54
So the equation I'm coming up
with is a transpose b-ax hat is

422
00:27:54 --> 00:27:55
zero.

423
00:27:55 --> 00:27:55
OK.

424
00:27:55 --> 00:27:57
That's my equation.

425
00:27:57 --> 00:27:58
All right.

426
00:27:58 --> 00:28:04
Now I want to stop for a moment
before I solve it and just think

427
00:28:04 --> 00:28:05
about it.

428
00:28:05 --> 00:28:11
First of all do you see that
that equation back in the very

429
00:28:11 --> 00:28:17
first problem I solved on a
line, what was --

430
00:28:17 --> 00:28:22.73
what was on a line the matrix a
only had one column,

431
00:28:22.73 --> 00:28:24
it was just little a.

432
00:28:24 --> 00:28:29
So in the first problem I
solved, projecting on a line,

433
00:28:29 --> 00:28:35
this for capital a you just
change that to little a and you

434
00:28:35 --> 00:28:39.64
have the same equation that we
solved before.

435
00:28:39.64 --> 00:28:43
a transpose e equals zero.

436
00:28:43 --> 00:28:43
OK.

437
00:28:43 --> 00:28:47
Now a second thing,
second comment.

438
00:28:47 --> 00:28:53
I would like to since I know
about these four subspaces,

439
00:28:53 --> 00:28:57
I would like to get them into
this picture.

440
00:28:57 --> 00:29:03
So let me ask the question,
what subspace is this thing in?

441
00:29:03 --> 00:29:09.27
Which of the four subspaces is
that error vector e,

442
00:29:09.27 --> 00:29:14
this is this is nothing but e
--

443
00:29:14 --> 00:29:20
this is this guy,
coming in down perpendicular to

444
00:29:20 --> 00:29:21
the plane.

445
00:29:21 --> 00:29:24
What subspace is e in?

446
00:29:24 --> 00:29:26
From this equation.

447
00:29:26 --> 00:29:33
Well the equation is saying a
transpose e is zero.

448
00:29:33 --> 00:29:39
So I'm learning here that e is
in the null space of a

449
00:29:39 --> 00:29:41
transpose.

450
00:29:41 --> 00:29:41
Right?

451
00:29:41 --> 00:29:45
That's my equation.

452
00:29:45 --> 00:29:51
And now I just want to see hey
of course that that was right.

453
00:29:51 --> 00:29:56
Because things that are in the
null space of a transpose,

454
00:29:56 --> 00:30:01
what do we know about the null
space of a transpose?

455
00:30:01 --> 00:30:07
So that last lecture gave us
the sort of the geometry of

456
00:30:07 --> 00:30:09
these subspaces.

457
00:30:09 --> 00:30:12
And the orthogonality of them.

458
00:30:12 --> 00:30:15
And do you remember what it
was?

459
00:30:15 --> 00:30:22
What on the right side of our
big figure we always have the

460
00:30:22 --> 00:30:27
null space of a transpose and
the column space of a,

461
00:30:27 --> 00:30:29.91
and they're orthogonal.

462
00:30:29.91 --> 00:30:35
So e in the null space of a
transpose is saying e is

463
00:30:35 --> 00:30:40
perpendicular to the column
space of a.

464
00:30:40 --> 00:30:41
Yes.

465
00:30:41 --> 00:30:47
I just feel OK,
the damn thing came out right.

466
00:30:47 --> 00:30:55
The equation for the equation
that I struggled to find for e

467
00:30:55 --> 00:31:01
really said what I wanted,
that the error e is

468
00:31:01 --> 00:31:08
perpendicular to the column
space of a, just right.

469
00:31:08 --> 00:31:14
And from our four fundamental
subspaces we knew that that is

470
00:31:14 --> 00:31:15
the same as that.

471
00:31:15 --> 00:31:20
To say e is in the null space
of a transpose says e's

472
00:31:20 --> 00:31:23
perpendicular to the column
space.

473
00:31:23 --> 00:31:23
OK.

474
00:31:23 --> 00:31:25
So we've got this equation.

475
00:31:25 --> 00:31:27.87
Now let's just solve it.

476
00:31:27.87 --> 00:31:28
All right.

477
00:31:28 --> 00:31:33
Let me just rewrite it as a
transpose a x hat equals a

478
00:31:33 --> 00:31:35
transpose b.

479
00:31:35 --> 00:31:37
That's our equation.

480
00:31:37 --> 00:31:39
That gives us x.

481
00:31:39 --> 00:31:44
And -- allow me to keep
remembering the one-dimensional

482
00:31:44 --> 00:31:44
case.

483
00:31:44 --> 00:31:49
The one-dimensional case,
this was little a.

484
00:31:49 --> 00:31:53
So this was just a number,
little a transpose,

485
00:31:53 --> 00:31:58
a transpose a was just a vector
row times a column,

486
00:31:58 --> 00:32:00
a number.

487
00:32:00 --> 00:32:03
And this was a number.

488
00:32:03 --> 00:32:07
And x was the ratio of those
numbers.

489
00:32:07 --> 00:32:12
But now we've got matrices,
this one is n by n.

490
00:32:12 --> 00:32:16
a transpose a is an n by n
matrix.

491
00:32:16 --> 00:32:17
OK.

492
00:32:17 --> 00:32:22
So can I move to the next board
for the solution?

493
00:32:22 --> 00:32:23
OK.

494
00:32:23 --> 00:32:28
This is the --
the key equation.

495
00:32:28 --> 00:32:33
Now I'm ready for the formulas
that we have to remember.

496
00:32:33 --> 00:32:35
What's x hat?

497
00:32:35 --> 00:32:40
What's the projection,
what's the projection matrix,

498
00:32:40 --> 00:32:43
those are my three questions.

499
00:32:43 --> 00:32:49
That we answered in the 1-D
case and now we're ready for in

500
00:32:49 --> 00:32:52
the n-dimensional case.

501
00:32:52 --> 00:32:54
So what is x hat?

502
00:32:54 --> 00:32:59
Well, what can I say but a
transpose a inverse,

503
00:32:59 --> 00:33:01.25
a transpose b.

504
00:33:01.25 --> 00:33:01
OK.

505
00:33:01 --> 00:33:06
That's the solution to -- to
our equation.

506
00:33:06 --> 00:33:08
What's the projection?

507
00:33:08 --> 00:33:11
That's more interesting.

508
00:33:11 --> 00:33:13
What's the projection?

509
00:33:13 --> 00:33:17
The projection is a x hat.

510
00:33:17 --> 00:33:23.41
That's how x hat got into the
picture in the first place.

511
00:33:23.41 --> 00:33:28
x hat was the was the
combination of columns in the I

512
00:33:28 --> 00:33:33.31
had to look for those numbers
and now I found them.

513
00:33:33.31 --> 00:33:38
Was the combination of the
columns of a that gave me the

514
00:33:38 --> 00:33:39
projection.

515
00:33:39 --> 00:33:40
OK.

516
00:33:40 --> 00:33:44
So now I know what this guy is.

517
00:33:44 --> 00:33:51
So it's just I multiply by a.
a a transpose a inverse a

518
00:33:51 --> 00:33:52
transpose b.

519
00:33:52 --> 00:33:58
That's looking a little messy
but it's not bad.

520
00:33:58 --> 00:34:05
That that combination is is our
like magic combination.

521
00:34:05 --> 00:34:13
This is the thing which is
which use which is like what's

522
00:34:13 --> 00:34:19
it like, what was it in one
dimension?

523
00:34:19 --> 00:34:25
What was that we had this we
must have had this thing way

524
00:34:25 --> 00:34:28
back at the beginning of the
lecture.

525
00:34:28 --> 00:34:34
What did we -- oh that a was
just a column so it was little

526
00:34:34 --> 00:34:39
a, little a transpose over a
transpose a, right,

527
00:34:39 --> 00:34:42
that's what it was in 1-D.

528
00:34:42 --> 00:34:47
You see what's happened in more
dimensions, I --

529
00:34:47 --> 00:34:53
I'm not allowed to to just
divide because because I don't

530
00:34:53 --> 00:34:56
have a number,
I have to put inverse,

531
00:34:56 --> 00:34:59
because I have an n by n
matrix.

532
00:34:59 --> 00:35:01
But same formula.

533
00:35:01 --> 00:35:05
And now tell me what's the
projection matrix?

534
00:35:05 --> 00:35:10
What matrix is multiplying b to
give the projection?

535
00:35:10 --> 00:35:12
Right there.

536
00:35:12 --> 00:35:19.59
Because there it --
I even already underlined it by

537
00:35:19.59 --> 00:35:20
accident.

538
00:35:20 --> 00:35:28
The projection matrix which I
use capital P is this,

539
00:35:28 --> 00:35:34
it's it's that thing,
shall I write it again,

540
00:35:34 --> 00:35:40
a times a transpose a inverse
times a transpose.

541
00:35:40 --> 00:35:49
Now if you'll bear with me I'll
think about what have I done

542
00:35:49 --> 00:35:49
here.

543
00:35:49 --> 00:35:54
I've got this formula.

544
00:35:54 --> 00:35:59
Now the first thing that occurs
to me is something bad.

545
00:35:59 --> 00:36:05
Look why don't I just you know
here's a product of two matrices

546
00:36:05 --> 00:36:09.51
and I want its inverse,
why don't I just use the

547
00:36:09.51 --> 00:36:14.45
formula I know for the inverse
of a product and say OK,

548
00:36:14.45 --> 00:36:18
that's a inverse times a
transpose inverse,

549
00:36:18 --> 00:36:22
what will happen if I do that?

550
00:36:22 --> 00:36:27
What will happen if I say hey
this is a inverse times a

551
00:36:27 --> 00:36:31
transpose inverse,
then shall I do it?

552
00:36:31 --> 00:36:37
It's going to go on videotape
if I do it, and I don't -- all

553
00:36:37 --> 00:36:43
right, I'll put it there,
but just like don't take the

554
00:36:43 --> 00:36:47
videotape quite so carefully.

555
00:36:47 --> 00:36:47
OK.

556
00:36:47 --> 00:36:55
So if I put that thing it -- it
would be a a inverse a transpose

557
00:36:55 --> 00:36:59.71
inverse a transpose and what's
that?

558
00:36:59.71 --> 00:37:02
That's the identity.

559
00:37:02 --> 00:37:04
But what's going on?

560
00:37:04 --> 00:37:11
So why -- you see my question
is somehow I did something

561
00:37:11 --> 00:37:11.92
wrong.

562
00:37:11.92 --> 00:37:16
That that wasn't allowed.

563
00:37:16 --> 00:37:18
And and and why is that?

564
00:37:18 --> 00:37:22.21
Because a is not a square
matrix.

565
00:37:22.21 --> 00:37:24.86
a is not a square matrix.

566
00:37:24.86 --> 00:37:27
It doesn't have an inverse.

567
00:37:27 --> 00:37:31
So I have to leave it that way.

568
00:37:31 --> 00:37:32
This is not OK.

569
00:37:32 --> 00:37:38
If if a was a square invertible
matrix, then then I couldn't

570
00:37:38 --> 00:37:39
complain.

571
00:37:39 --> 00:37:45
Yeah I think -- let me think
about that case.

572
00:37:45 --> 00:37:50
But you but my main case,
the whole reason I'm doing all

573
00:37:50 --> 00:37:54
this, is that a is this matrix
that has x too many rows,

574
00:37:54 --> 00:37:58
it's just got a couple of
columns, like a one and a two,

575
00:37:58 --> 00:38:00
but lots of rows.

576
00:38:00 --> 00:38:00
Not square.

577
00:38:00 --> 00:38:04
And if it's not square,
this a transpose a is square

578
00:38:04 --> 00:38:08
but I can't pull it apart like
this --

579
00:38:08 --> 00:38:15
I'm not allowed to do this pull
apart, except if a was square.

580
00:38:15 --> 00:38:21
Now if a is square what's
what's going on if a is a square

581
00:38:21 --> 00:38:24
matrix?
a nice square inv- invertible

582
00:38:24 --> 00:38:25
matrix.

583
00:38:25 --> 00:38:26
Think.

584
00:38:26 --> 00:38:31
What's up with that what's with
that case.

585
00:38:31 --> 00:38:36
So this is that the formula
ought to work then too.

586
00:38:36 --> 00:38:41
If a is a nice square
invertible matrix what's its

587
00:38:41 --> 00:38:44
column space,
so it's a nice n by n

588
00:38:44 --> 00:38:50
invertible everything great
matrix, what's its column space,

589
00:38:50 --> 00:38:51
the whole of R^n.

590
00:38:51 --> 00:38:57
So what's the projection matrix
if I'm projecting onto the whole

591
00:38:57 --> 00:38:59
space?

592
00:38:59 --> 00:39:01
It's the identity matrix right?

593
00:39:01 --> 00:39:06
If I'm projecting b onto the
whole space, not just onto a

594
00:39:06 --> 00:39:10
plane, but onto all of 3-D,
then b is already in the column

595
00:39:10 --> 00:39:15
space, the projection is the
identity, and this is gives me

596
00:39:15 --> 00:39:17
the correct formula,
P is I.

597
00:39:17 --> 00:39:21
But if I'm projecting onto a
subspace then I can't split

598
00:39:21 --> 00:39:26
those apart and I have to stay
with that formula.

599
00:39:26 --> 00:39:26
OK.

600
00:39:26 --> 00:39:32.52
And what can I say if -- so I
remember this formula for 1-D

601
00:39:32.52 --> 00:39:36
and that's what it looks like in
n dimensions.

602
00:39:36 --> 00:39:41
And what are the properties
that I expected for any

603
00:39:41 --> 00:39:43
projection matrix?

604
00:39:43 --> 00:39:46
And I still expect for this
one?

605
00:39:46 --> 00:39:50
That matrix should be symmetric
and it is.

606
00:39:50 --> 00:39:53.19
P transpose is P.

607
00:39:53.19 --> 00:39:57
Because if I transpose this,
this guy's symmetric,

608
00:39:57 --> 00:40:03
and its inverse is symmetric,
and if I transpose this one

609
00:40:03 --> 00:40:07
when I transpose it will pop up
there, become a,

610
00:40:07 --> 00:40:13
that a transpose will pop up
here, and I'm back to P again.

611
00:40:13 --> 00:40:18
And do we dare try the other
property which is P squared

612
00:40:18 --> 00:40:19
equal P?

613
00:40:19 --> 00:40:22
It's got to be right.

614
00:40:22 --> 00:40:30
Because we know geometrically
that the first projection pops

615
00:40:30 --> 00:40:38
us onto the column space and the
second one leaves us where we

616
00:40:38 --> 00:40:38
are.

617
00:40:38 --> 00:40:47
So I expect that if I multiply
by let me do it -- if I multiply

618
00:40:47 --> 00:40:51
by another P,
so there's another a,

619
00:40:51 --> 00:40:59
another a transpose a inverse a
transpose, can you --

620
00:40:59 --> 00:41:05
eight (a)-s in a row is quite
obscene but -- do you see that

621
00:41:05 --> 00:41:06
it works?

622
00:41:06 --> 00:41:10.92
So I'm squaring that so what do
I do-- how do I see that

623
00:41:10.92 --> 00:41:12
multiplication?

624
00:41:12 --> 00:41:17
Well, yeah, I just want to put
parentheses in good places,

625
00:41:17 --> 00:41:22
so I see what's happening,
yeah, here's an a transpose a

626
00:41:22 --> 00:41:27
sitting together --
so when that a transpose a

627
00:41:27 --> 00:41:31
multiplies its inverse,
all that stuff goes,

628
00:41:31 --> 00:41:32
right.

629
00:41:32 --> 00:41:37
And leaves just the a transpose
at the end, which is just what

630
00:41:37 --> 00:41:38
we want.

631
00:41:38 --> 00:41:40
So P squared equals P.

632
00:41:40 --> 00:41:44
So sure enough those two
properties hold.

633
00:41:44 --> 00:41:44
OK.

634
00:41:44 --> 00:41:50
OK we really have got now all
the formulas.

635
00:41:50 --> 00:41:54
x hat, the projection P,
and the projection matrix

636
00:41:54 --> 00:41:55
capital P.

637
00:41:55 --> 00:41:58
And now my job is to use them.

638
00:41:58 --> 00:41:59.13
OK.

639
00:41:59.13 --> 00:42:04
So when would I have a bunch of
equations, too many equations

640
00:42:04 --> 00:42:10
and yet I want the best answer
and the -- the most important

641
00:42:10 --> 00:42:16
example, the most common example
is if I have points so here's

642
00:42:16 --> 00:42:22
the --
here's the application.

643
00:42:22 --> 00:42:23
v squared.

644
00:42:23 --> 00:42:26
Fitting by a line.

645
00:42:26 --> 00:42:27
OK.

646
00:42:27 --> 00:42:36
So I'll start this application
today and there's more in it

647
00:42:36 --> 00:42:41
than I can do in this same
lecture.

648
00:42:41 --> 00:42:51
So that'll give me a chance to
recap the formulas and there

649
00:42:51 --> 00:42:57
they are, and recap the ideas.

650
00:42:57 --> 00:43:02
So let me start the problem
today.

651
00:43:02 --> 00:43:06
I'm given a bunch of data
points.

652
00:43:06 --> 00:43:11
And they lie close to a line
but not on a line.

653
00:43:11 --> 00:43:13
Let me take that.

654
00:43:13 --> 00:43:18
Say a t equal to one,
two and three,

655
00:43:18 --> 00:43:23
I have one, and two and two
again.

656
00:43:23 --> 00:43:30
So my data points are this is
the like the time direction and

657
00:43:30 --> 00:43:37
this is like well let me call
that b or y or something.

658
00:43:37 --> 00:43:43
I'm given these three points
and I want to fit them by a

659
00:43:43 --> 00:43:44.4
line.

660
00:43:44.4 --> 00:43:47
By the best straight line.

661
00:43:47 --> 00:43:53
So the problem is fit the
points one, one is the first

662
00:43:53 --> 00:43:58
point --
the second point is t equals

663
00:43:58 --> 00:44:03
two, b equal one,
and the third point is t equal

664
00:44:03 --> 00:44:06
three, b equal to two.

665
00:44:06 --> 00:44:11
So those are my three points,
t equal sorry,that's two.

666
00:44:11 --> 00:44:12
Yeah, OK.

667
00:44:12 --> 00:44:15.79
So this is the point one,
one.

668
00:44:15.79 --> 00:44:20
This is the point two,
two, and that's the point

669
00:44:20 --> 00:44:21
three, two.

670
00:44:21 --> 00:44:29
And of course there isn't a --
a line that goes through them.

671
00:44:29 --> 00:44:32
So I'm looking for the best
line.

672
00:44:32 --> 00:44:37
I'm looking for a line that
probably goes somewhere,

673
00:44:37 --> 00:44:40
do you think it goes somewhere
like that?

674
00:44:40 --> 00:44:45
I didn't mean to make it go
through that point,

675
00:44:45 --> 00:44:46
it won't.

676
00:44:46 --> 00:44:51
It'll kind of --
it'll go between so the error

677
00:44:51 --> 00:44:57
there and the error there and
the error there are as small as

678
00:44:57 --> 00:44:58.65
I can get them.

679
00:44:58.65 --> 00:45:02
OK, what I'd like to do is find
the matrix a.

680
00:45:02 --> 00:45:08
Because once I've found the
matrix a the formulas take over.

681
00:45:08 --> 00:45:13
So what I'm looking for this
line, b is C+Dt.

682
00:45:13 --> 00:45:18
So this is in the homework that
I sent out for today.

683
00:45:18 --> 00:45:19
Find the best line.

684
00:45:19 --> 00:45:22
So I'm looking for these
numbers.

685
00:45:22 --> 00:45:23
C and D.

686
00:45:23 --> 00:45:28
That tell me the line and I
want them to be as close to

687
00:45:28 --> 00:45:32
going through those three points
as I can get.

688
00:45:32 --> 00:45:36
I can't get exactly so there
are three equations to go

689
00:45:36 --> 00:45:39
through the three points.

690
00:45:39 --> 00:45:45
It would it will go exactly
through that point if let's see

691
00:45:45 --> 00:45:50
that first point has t equal to
one, so that would say C+D

692
00:45:50 --> 00:45:51
equaled 1.

693
00:45:51 --> 00:45:52.82
This is the one,
one.

694
00:45:52.82 --> 00:45:55
The second point t is two.

695
00:45:55 --> 00:45:58
So C+2D should come out to
equal 2.

696
00:45:58 --> 00:46:03
But I also want to get the
third equation in and at that

697
00:46:03 --> 00:46:08
third equation t is three so
C+3D equals only 2.

698
00:46:08 --> 00:46:09.96
That's the key.

699
00:46:09.96 --> 00:46:15
Is to write down what equations
we would like to solve but

700
00:46:15 --> 00:46:16
can't.

701
00:46:16 --> 00:46:22.12
Reason we if we could solve
them that would mean that we

702
00:46:22.12 --> 00:46:27
could put a line through all
three points and of course if

703
00:46:27 --> 00:46:32
these numbers one,
two, two were different,

704
00:46:32 --> 00:46:35
we could do it.

705
00:46:35 --> 00:46:38
But with those numbers,
one, two, two,

706
00:46:38 --> 00:46:39
we can't.

707
00:46:39 --> 00:46:45.78
So what is our equation Ax
equal Ax equal b that we can't

708
00:46:45.78 --> 00:46:46
solve?

709
00:46:46 --> 00:46:52
I just want to say what's the
matrix here, what's the unknown

710
00:46:52 --> 00:46:57
x, and what's the right-hand
side.

711
00:46:57 --> 00:47:02
So this is the matrix is one,
one, one, one,

712
00:47:02 --> 00:47:03
two, three.

713
00:47:03 --> 00:47:06
The unknown is C and D.

714
00:47:06 --> 00:47:10
And the right-hand side if one,
two, two.

715
00:47:10 --> 00:47:17
Right, I've just taken my
equations and I've said what is

716
00:47:17 --> 00:47:19
Ax and what is b.

717
00:47:19 --> 00:47:25
Then there's no solution,
this is the typical case where

718
00:47:25 --> 00:47:31
I have three equations --
two unknowns,

719
00:47:31 --> 00:47:36
no solution,
but I'm still looking for the

720
00:47:36 --> 00:47:38
best solution.

721
00:47:38 --> 00:47:45
And the best solution is taken
is is to solve not this equation

722
00:47:45 --> 00:47:52
Ax equal b which has which has
no solution but the equation

723
00:47:52 --> 00:47:57
that does have a solution,
which was this one.

724
00:47:57 --> 00:48:03
So that's the equation to
solve.

725
00:48:03 --> 00:48:07
That's the central equation of
the subject.

726
00:48:07 --> 00:48:11
I can't solve Ax=b but
magically when I multiply both

727
00:48:11 --> 00:48:17
sides by a transpose I get an
equation that I can solve and

728
00:48:17 --> 00:48:21
its solution gives me x,
the best x, the best

729
00:48:21 --> 00:48:26
projection, and I discover
what's the matrix that's behind

730
00:48:26 --> 00:48:27
it.

731
00:48:27 --> 00:48:27
OK.

732
00:48:27 --> 00:48:32
So next time I'll complete an
example, numerical example.

733
00:48:32 --> 00:48:36
today was all letters,
numbers next time.

734
00:48:36 --> 00:48:39
Thanks.