1
00:00:00 --> 00:00:06
OK, this is the lecture on
positive definite matrices.

2
00:00:06 --> 00:00:12
I made a start on those briefly
in a previous lecture.

3
00:00:12 --> 00:00:19
One point I wanted to make was
the way that this topic brings

4
00:00:19 --> 00:00:25
the whole course together,
pivots, determinants,

5
00:00:25 --> 00:00:29
eigenvalues,
and something new- four plot

6
00:00:29 --> 00:00:34
instability and then something
new

7
00:00:34 --> 00:00:38
in this expression,
x transpose Ax,

8
00:00:38 --> 00:00:43
actually that's the guy to
watch in this lecture.

9
00:00:43 --> 00:00:47
So, so the topic is positive
definite matrix,

10
00:00:47 --> 00:00:49
and what's my goal?

11
00:00:49 --> 00:00:54
First, first goal is,
how can I tell if a matrix is

12
00:00:54 --> 00:00:56
positive definite?

13
00:00:56 --> 00:01:01
So I would like to have tests
to see

14
00:01:01 --> 00:01:04
if you give me a,
a five by five matrix,

15
00:01:04 --> 00:01:07
how do I tell if it's positive
definite?

16
00:01:07 --> 00:01:10
More important is,
what does it mean?

17
00:01:10 --> 00:01:14
Why are we so interested in
this property of positive

18
00:01:14 --> 00:01:15
definiteness?

19
00:01:15 --> 00:01:18
And then, at the end comes some
geometry.

20
00:01:18 --> 00:01:24
Ellipses are connected
with positive definite things.

21
00:01:24 --> 00:01:29
Hyperbolas are not connected
with positive definite things,

22
00:01:29 --> 00:01:32
so we- it's this,
we, there's a geometry too,

23
00:01:32 --> 00:01:38
but mostly it's linear algebra
and -- this application of how

24
00:01:38 --> 00:01:43
do you recognize 'em when you
have a minim is pretty neat.

25
00:01:43 --> 00:01:43.33
OK.

26
00:01:43.33 --> 00:01:47
I'm gonna begin with two by
two.

27
00:01:47 --> 00:01:50
All matrices are symmetric,
right?

28
00:01:50 --> 00:01:54
That's understood;
the matrix is symmetric,

29
00:01:54 --> 00:01:59
now my question is,
is it positive definite?

30
00:01:59 --> 00:02:05
Now, here are some -- each one
of these is a complete test for

31
00:02:05 --> 00:02:07
positive definiteness.

32
00:02:07 --> 00:02:14
If I know the eigenvalues,
my test is are they positive?

33
00:02:14 --> 00:02:16
Are they all positive?

34
00:02:16 --> 00:02:21
If I know these -- so,
A is really -- I look at that

35
00:02:21 --> 00:02:25
number A, here,
as the, as the one by one

36
00:02:25 --> 00:02:30
determinant, and here's the two
by two determinant.

37
00:02:30 --> 00:02:34
So this is the determinant
test.

38
00:02:34 --> 00:02:39.11
This is the eigenvalue test,
this is the determinant test.

39
00:02:39.11 --> 00:02:43
Are the determinants growing in
s- of all, of all end,

40
00:02:43 --> 00:02:47
sort of, can I call them
leading submatrices,

41
00:02:47 --> 00:02:51
they're the first ones the
northwest,

42
00:02:51 --> 00:02:55
Seattle submatrices coming down
from from there,

43
00:02:55 --> 00:02:59.6
they all, all those
determinants have to be

44
00:02:59.6 --> 00:03:03
positive, and then another test
is the pivots.

45
00:03:03 --> 00:03:08
The pivots of a two by two
matrix are the number A for

46
00:03:08 --> 00:03:13
sure, and, since the product is
the determinant,

47
00:03:13 --> 00:03:18
the second pivot must be the
determinant divided by A.

48
00:03:18 --> 00:03:24
And then in here is gonna come
my favorite and my new idea,

49
00:03:24 --> 00:03:30
the, the, the the one to catch,
about x transpose Ax being

50
00:03:30 --> 00:03:31
positive.

51
00:03:31 --> 00:03:34
But we'll have to look at this
guy.

52
00:03:34 --> 00:03:39
This gets, like a star,
because for most,

53
00:03:39 --> 00:03:43
presentations,
the definition of positive

54
00:03:43 --> 00:03:48
definiteness would be this
number four and these numbers

55
00:03:48 --> 00:03:51
one two three would be test
four.

56
00:03:51 --> 00:03:51
OK.

57
00:03:51 --> 00:03:55
Maybe I'll tuck this,
where, you know,

58
00:03:55 --> 00:03:55
OK.

59
00:03:55 --> 00:03:59
So I'll have to look at this x
transpose Ax.

60
00:03:59 --> 00:04:04
Can you, can we just be sure,
how do we know that the

61
00:04:04 --> 00:04:10
eigenvalue
test and the determinant test,

62
00:04:10 --> 00:04:18
pick out the same matrices,
and let me, let's just do a few

63
00:04:18 --> 00:04:19
examples.

64
00:04:19 --> 00:04:21
Some examples.

65
00:04:21 --> 00:04:27
Let me pick the matrix two,
six, six, tell me,

66
00:04:27 --> 00:04:34.82
what number do I have to put
there for the matrix to be

67
00:04:34.82 --> 00:04:38
positive definite?

68
00:04:38 --> 00:04:42
Tell me a sufficiently large
number that would make it

69
00:04:42 --> 00:04:43
positive definite?

70
00:04:43 --> 00:04:47
Let's just practice with these
conditions in the two by two

71
00:04:47 --> 00:04:48
case.

72
00:04:48 --> 00:04:51
Now, when I ask you that,
you don't wanna find the

73
00:04:51 --> 00:04:55
eigenvalues, you would use the
determinant test for that,

74
00:04:55 --> 00:04:59
so,
the first or the pivot test,

75
00:04:59 --> 00:05:04
that, that guy is certainly
positive, that had to happen,

76
00:05:04 --> 00:05:05
and it's OK.

77
00:05:05 --> 00:05:10
How large a number here -- the
number had better be more than

78
00:05:10 --> 00:05:10
what?

79
00:05:10 --> 00:05:15
More than eighteen,
right, because if it's eight --

80
00:05:15 --> 00:05:15
no.

81
00:05:15 --> 00:05:16
More than what?

82
00:05:16 --> 00:05:19
Nineteen, is it?

83
00:05:19 --> 00:05:25
If I have a nineteen here,
is that positive definite?

84
00:05:25 --> 00:05:30
I get thirty eight minus thirty
six, that's OK.

85
00:05:30 --> 00:05:36
If I had an eighteen,
let me play it really close.

86
00:05:36 --> 00:05:42
If I have an eighteen there,
then I positive definite?

87
00:05:42 --> 00:05:43.48
Not quite.

88
00:05:43.48 --> 00:05:51
I would call this guy positive,
so it's useful just to see that

89
00:05:51 --> 00:05:53
this the borderline.

90
00:05:53 --> 00:05:58
That matrix is on the
borderline, I would call that

91
00:05:58 --> 00:06:01
matrix positive semi-definite.

92
00:06:01 --> 00:06:07
And what are the eigenvalues of
that matrix, just since we're

93
00:06:07 --> 00:06:14
given eigenvalues of two by
twos, when it's semi-definite,

94
00:06:14 --> 00:06:18
but not definite,
then the -- I'm squeezing this

95
00:06:18 --> 00:06:24
eigenvalue test down,
-- what's the eigenvalue that I

96
00:06:24 --> 00:06:26
know this matrix has?

97
00:06:26 --> 00:06:30
What kind of a matrix have I
got here?

98
00:06:30 --> 00:06:36
It's a singular matrix,
one of its eigenvalues is zero.

99
00:06:36 --> 00:06:42
That has an eigenvalue zero,
and the other eigenvalue is --

100
00:06:42 --> 00:06:44
from the trace,
twenty.

101
00:06:44 --> 00:06:44
OK.

102
00:06:44 --> 00:06:49
So that, that matrix has
eigenvalues greater than or

103
00:06:49 --> 00:06:52
equal to zero,
and it's that "equal to" that

104
00:06:52 --> 00:06:55
brought this word
"semi-definite" in.

105
00:06:55 --> 00:07:00.47
And, the what are the pivots of
that matrix?

106
00:07:00.47 --> 00:07:04
So the pivots,
so the eigenvalues are zero and

107
00:07:04 --> 00:07:09
twenty, the pivots are,
well, the pivot is two,

108
00:07:09 --> 00:07:11
and what's the next pivot?

109
00:07:11 --> 00:07:13
There isn't one.

110
00:07:13 --> 00:07:18
We got a singular matrix here,
it'll only have one pivot.

111
00:07:18 --> 00:07:25
You see that that's a rank one
matrix, two six is a -- six

112
00:07:25 --> 00:07:32
eighteen is a multiple of two
six, the matrix is singular it

113
00:07:32 --> 00:07:38.21
only has one pivot,
so the pivot test doesn't quite

114
00:07:38.21 --> 00:07:38
pass.

115
00:07:38 --> 00:07:42
The -- let me do the x
transpose Ax.

116
00:07:42 --> 00:07:46
Now this is -- the novelty now.

117
00:07:46 --> 00:07:46
OK.

118
00:07:46 --> 00:07:54.06
What I looking at when I
look at this new combination,

119
00:07:54.06 --> 00:07:57
x transpose Ax.
x is any vector now,

120
00:07:57 --> 00:08:01
so lemme compute,
so any vector,

121
00:08:01 --> 00:08:06
lemme call its components x1
and x2, so that's x.

122
00:08:06 --> 00:08:08
And I put in here A.

123
00:08:08 --> 00:08:15.37
Let's, let's use this example
two six, six eighteen,

124
00:08:15.37 --> 00:08:18
and here's x transposed,
so x1 x2.

125
00:08:18 --> 00:08:24
We're back to real matrices,
after that last lecture that-

126
00:08:24 --> 00:08:27
that said what to do in the
complex case,

127
00:08:27 --> 00:08:30
let's come back to real
matrices.

128
00:08:30 --> 00:08:36
So here's x transpose Ax,
and what I'm interested in is,

129
00:08:36 --> 00:08:41
what do I get if I multiply
those together?

130
00:08:41 --> 00:08:46
I get some function of x1 and
x2, and what is it?

131
00:08:46 --> 00:08:50.62
Let's see, if I do this
multiplication,

132
00:08:50.62 --> 00:08:55
so I do it, lemme,
just, I'll just do it slowly,

133
00:08:55 --> 00:09:00
x1, x2, if I multiply that
matrix, this is 2x1,

134
00:09:00 --> 00:09:06
and 6x2s, and the next row is
6x1s and 18x2s,

135
00:09:06 --> 00:09:10
and now I do this final step
and what do I have?

136
00:09:10 --> 00:09:15.92
I've got 2x1 squareds,
got 2X1 squareds is coming from

137
00:09:15.92 --> 00:09:20
this two, I've got this one
gives me eighteen,

138
00:09:20 --> 00:09:24
well, shall I do the ones in
the middle?

139
00:09:24 --> 00:09:26
How many x1 x2s do I have?

140
00:09:26 --> 00:09:31
Let's see,
x1 times that 6x2 would be six

141
00:09:31 --> 00:09:36
of 'em, and then x2 times this
one will be six more,

142
00:09:36 --> 00:09:38
I get twelve.

143
00:09:38 --> 00:09:42
So, in here is going,
this is the number a,

144
00:09:42 --> 00:09:47
this is the number 2b,
and in here is -- x2 times

145
00:09:47 --> 00:09:53
eighteen x2 will be eighteen x2
squareds and this

146
00:09:53 --> 00:09:55
is the number c.

147
00:09:55 --> 00:09:59
So it's ax1 -- it's like ax
squared.

148
00:09:59 --> 00:10:01.23
2bxy and cy squared.

149
00:10:01.23 --> 00:10:07
For my first point that I
wanted to make by just doing out

150
00:10:07 --> 00:10:12
a multiplication is,
that is as soon as you give me

151
00:10:12 --> 00:10:17
the matrix, as soon as you give
me the matrix,

152
00:10:17 --> 00:10:21
I can --
those are the numbers that

153
00:10:21 --> 00:10:25
appear in -- I'll call this
thing a quadratic,

154
00:10:25 --> 00:10:27
you see, it's not linear
anymore.

155
00:10:27 --> 00:10:31
Ax is linear,
but now I've got an x transpose

156
00:10:31 --> 00:10:35
coming in, I'm up to degree two,
up to degree two,

157
00:10:35 --> 00:10:39
maybe quadratic is the -- use
the word.

158
00:10:39 --> 00:10:40
A quadratic form.

159
00:10:40 --> 00:10:44
It's purely degree two,
there's no linear part,

160
00:10:44 --> 00:10:49
there's no constant part,
there certainly no cubes or

161
00:10:49 --> 00:10:53
fourth powers,
it's all second degree.

162
00:10:53 --> 00:10:58
And my question is -- is that
quantity positive or not?

163
00:10:58 --> 00:11:04
That's -- for every x1 and x2,
that is my new definition --

164
00:11:04 --> 00:11:10
that's my definition of a
positive definite matrix.

165
00:11:10 --> 00:11:16
If this quantity is positive,
if, if, if, it's positive for

166
00:11:16 --> 00:11:21
all x's and y's,
all x1 x2s, then I call them --

167
00:11:21 --> 00:11:26
then that's the matrix is
positive definite.

168
00:11:26 --> 00:11:31
Now, is this guy passing our
test?

169
00:11:31 --> 00:11:36
Well we have,
we anticipated the answer here

170
00:11:36 --> 00:11:41.87
by, by asking first about
eigenvalues and pivots,

171
00:11:41.87 --> 00:11:43.97
and what happened?

172
00:11:43.97 --> 00:11:45
It failed.

173
00:11:45 --> 00:11:47
It barely failed.

174
00:11:47 --> 00:11:52
If I had made this eighteen
down to a seven,

175
00:11:52 --> 00:11:55
it would've totally failed.

176
00:11:55 --> 00:12:01
I do that with the eraser,
and then I'll put back

177
00:12:01 --> 00:12:06
eighteen, because,
seven is such a total disaster,

178
00:12:06 --> 00:12:10
but if -- I'll keep seven for a
second.

179
00:12:10 --> 00:12:14
Is that thing in any way
positive definite?

180
00:12:14 --> 00:12:16
No, absolutely not.

181
00:12:16 --> 00:12:22
I don't know its eigenvalues,
but I know for sure one of them

182
00:12:22 --> 00:12:22.98
is negative.

183
00:12:22.98 --> 00:12:26
Its pivots are two and then the
next pivot would be the

184
00:12:26 --> 00:12:30
determinant over two,
and the determinant is -- what,

185
00:12:30 --> 00:12:33
what's the determinant of this
thing?

186
00:12:33 --> 00:12:37
Fourteen minus thirty six,
I've got a determinant minus

187
00:12:37 --> 00:12:38
twenty two.

188
00:12:38 --> 00:12:41
The next pivot will be -- the
pivots

189
00:12:41 --> 00:12:45
now, of this thing are two and
minus eleven or something.

190
00:12:45 --> 00:12:49
Their product being minus
twenty two the determinant.

191
00:12:49 --> 00:12:52
This thing is not positive
definite.

192
00:12:52 --> 00:12:56
What would be -- let me look at
the x transpose Ax for this guy.

193
00:12:56 --> 00:12:59
What's -- if I do out this
multiplication,

194
00:12:59 --> 00:13:04.5
this eighteen is temporarily
changing to a seven.

195
00:13:04.5 --> 00:13:10
This eighteen is temporarily
changing to a seven,

196
00:13:10 --> 00:13:17
and I know that there's some
numbers x1 and x2 for which that

197
00:13:17 --> 00:13:21
thing, that function,
is negative.

198
00:13:21 --> 00:13:26
And I'm desperately trying to
think what they are.

199
00:13:26 --> 00:13:29
Maybe you can see.

200
00:13:29 --> 00:13:36
Can you tell me a value of x1
and x2 that makes this quantity

201
00:13:36 --> 00:13:37
negative?

202
00:13:37 --> 00:13:40
Oh, maybe one and minus one?

203
00:13:40 --> 00:13:44
Yes, that's -- in this case,
that, will work,

204
00:13:44 --> 00:13:49
right, if I took x1 to be one,
and x2 to be minus one,

205
00:13:49 --> 00:13:54
then I always get something
positive, the two,

206
00:13:54 --> 00:13:59
and the seven minus one
squared, but this would be

207
00:13:59 --> 00:14:04
minus twelve and the whole thing
would be negative;

208
00:14:04 --> 00:14:10
I would have two minus twelve
plus seven, a negative.

209
00:14:10 --> 00:14:15
If I drew the graph,
can I get the little picture in

210
00:14:15 --> 00:14:15
here?

211
00:14:15 --> 00:14:18
If I draw the graph of this
thing?

212
00:14:18 --> 00:14:23
So, graphs, of the function
f(x,y),

213
00:14:23 --> 00:14:31
or f(x), so I say here f(x,y)
equal this -- x transpose Ax,

214
00:14:31 --> 00:14:37
this, this this ax squared,
2bxy, and cy squared.

215
00:14:37 --> 00:14:44
And, let's take the example,
with these numbers.

216
00:14:44 --> 00:14:49
OK, so here's the x axis,
here's the y axis,

217
00:14:49 --> 00:14:54
and here's -- up is the
function.

218
00:14:54 --> 00:14:58.18
z, if you like,
or f.

219
00:14:58.18 --> 00:15:01
I apologize,
and let me, just once in my

220
00:15:01 --> 00:15:05
life here, put an arrow over
these, these,

221
00:15:05 --> 00:15:07
axes so you see them.

222
00:15:07 --> 00:15:13
That's the vector and I just,
see, instead of x1 and x2,

223
00:15:13 --> 00:15:16
I made them x- the components x
and y.

224
00:15:16 --> 00:15:16
OK.

225
00:15:16 --> 00:15:21
So, so, what's a graph of 2x
squared,

226
00:15:21 --> 00:15:25
twelve xy, and seven y squared?

227
00:15:25 --> 00:15:30
I'd like to see -- I not the
greatest artist,

228
00:15:30 --> 00:15:37
but let's -- tell me something
about this graph of this

229
00:15:37 --> 00:15:38.87
function.

230
00:15:38.87 --> 00:15:44
Whoa, tell me one point that it
goes through.

231
00:15:44 --> 00:15:45
The origin.

232
00:15:45 --> 00:15:46
Right?

233
00:15:46 --> 00:15:52
Even this artist can get this
thing

234
00:15:52 --> 00:15:56
to go through the origin,
when these are zero,

235
00:15:56 --> 00:15:58
I, I certainly get zero.

236
00:15:58 --> 00:15:58
OK.

237
00:15:58 --> 00:16:00
Some more points.

238
00:16:00 --> 00:16:04
If x is one and y is zero,
then I'm going upwards,

239
00:16:04 --> 00:16:08
so I'm going up this way,
and I'm, I'm going up,

240
00:16:08 --> 00:16:13
like, two x squared in that
direction.

241
00:16:13 --> 00:16:16
So -- that's meant to be a
parabola.

242
00:16:16 --> 00:16:19
And, suppose x stays zero and y
increases.

243
00:16:19 --> 00:16:24
Well, y could be positive or
negative; it's seven y squared.

244
00:16:24 --> 00:16:27.42
Is this function going upward?

245
00:16:27.42 --> 00:16:32
In the x direction it's going
upward, and in the y direction

246
00:16:32 --> 00:16:37
it's going upwards,
and if x equals y then the

247
00:16:37 --> 00:16:42
forty-five degree direction is
certainly going upwards;

248
00:16:42 --> 00:16:47
because then we'd have what,
about, everything would be

249
00:16:47 --> 00:16:49
positive, but what?

250
00:16:49 --> 00:16:54
This function -- what's the
graph of this function?

251
00:16:54 --> 00:16:55
Look like?

252
00:16:55 --> 00:16:59
Tell me the word that
describes the graph of this

253
00:16:59 --> 00:17:00
function.

254
00:17:00 --> 00:17:03
This is the non-positive
definite here,

255
00:17:03 --> 00:17:06
everybody's with me here,
for some reason got started in

256
00:17:06 --> 00:17:10
a negative direction,
your case that isn't positive

257
00:17:10 --> 00:17:11
definite.

258
00:17:11 --> 00:17:13
And what's the graph look like
that goes up,

259
00:17:13 --> 00:17:17
but does it -- do we have a
minimum here,

260
00:17:17 --> 00:17:20
does it go from,
from the origin?

261
00:17:20 --> 00:17:21.58
Completely?

262
00:17:21.58 --> 00:17:25
No, because we just checked
that this thing failed.

263
00:17:25 --> 00:17:31
It failed along the direction
when x was minus y -- we have a

264
00:17:31 --> 00:17:33
saddle point,
let me put myself,

265
00:17:33 --> 00:17:38
let me, to the least,
tell you the word.

266
00:17:38 --> 00:17:46
This thing, goes up in some
directions, but down in other

267
00:17:46 --> 00:17:55
directions, and if we actually
knew what a saddle looked like

268
00:17:55 --> 00:18:02
or thinks saddles do that -- the
way your legs go is,

269
00:18:02 --> 00:18:07
like, down, up,
the way,

270
00:18:07 --> 00:18:10
you, looking like,
forward, and,

271
00:18:10 --> 00:18:16
the, and drawing the thing is
even worse than describing --

272
00:18:16 --> 00:18:22
I'm just going to say in some
directions we go up and in other

273
00:18:22 --> 00:18:28
directions, there is,
a saddle -- Now I'm sorry I put

274
00:18:28 --> 00:18:34
that on the front board,
you have no way to cover it,

275
00:18:34 --> 00:18:36
but it's a saddle.

276
00:18:36 --> 00:18:36
OK.

277
00:18:36 --> 00:18:40.48
And, and this is a saddle
point, it's the,

278
00:18:40.48 --> 00:18:45
it's the point that's at the
maximum in some directions and

279
00:18:45 --> 00:18:49
at the minimum in other
directions.

280
00:18:49 --> 00:18:53
And actually,
the perfect directions to look

281
00:18:53 --> 00:18:56
are the eigenvector directions.

282
00:18:56 --> 00:18:57
We'll see that.

283
00:18:57 --> 00:19:03
So this is,
not a positive definite matrix.

284
00:19:03 --> 00:19:03
OK.

285
00:19:03 --> 00:19:08
Now I'm coming back to this
example, getting rid of this

286
00:19:08 --> 00:19:12
seven, let's move it up to
twenty.

287
00:19:12 --> 00:19:18
Let's, let's let's make the
thing really positive definite.

288
00:19:18 --> 00:19:18
OK.

289
00:19:18 --> 00:19:22
So this is, this number's now
twenty.

290
00:19:22 --> 00:19:23
c is now twenty.

291
00:19:23 --> 00:19:24
OK.

292
00:19:24 --> 00:19:30
Now that passes the test,
which I haven't proved,

293
00:19:30 --> 00:19:33
of course, it passes the test
for positive pivots.

294
00:19:33 --> 00:19:37
It passes the test for positive
eigenvalues.

295
00:19:37 --> 00:19:41
How can you tell that the
eigenvalues of that matrix are

296
00:19:41 --> 00:19:44
positive without actually
finding them?

297
00:19:44 --> 00:19:48
Of course,
two by two I could find them,

298
00:19:48 --> 00:19:51
but can you see -- how do I
know they're positive?

299
00:19:51 --> 00:19:55
I know that their product is --
I know that lambda one times

300
00:19:55 --> 00:19:56
lambda two is positive,
why?

301
00:19:56 --> 00:20:00
Because that's the determinant,
right, lambda one times lambda

302
00:20:00 --> 00:20:04
two is the determinant,
which is forty minus thirty-six

303
00:20:04 --> 00:20:05
is
four.

304
00:20:05 --> 00:20:08
So the determinant is four.

305
00:20:08 --> 00:20:12
And the trace,
the sum down the diagonal,

306
00:20:12 --> 00:20:13.44
is twenty-two.

307
00:20:13.44 --> 00:20:16
So, they multiply to give four.

308
00:20:16 --> 00:20:22
So that leaves the possibility
they're either both positive or

309
00:20:22 --> 00:20:23
both negative.

310
00:20:23 --> 00:20:29.08
But if they're both negative,
the trace couldn't be

311
00:20:29.08 --> 00:20:29
twenty-two.

312
00:20:29 --> 00:20:31
So they're both positive.

313
00:20:31 --> 00:20:35
So both of the eigenvalues that
are positive,

314
00:20:35 --> 00:20:39.73
both of the pivots are positive
-- the determinants are

315
00:20:39.73 --> 00:20:43
positive, and I believe that
this function is positive

316
00:20:43 --> 00:20:47
everywhere except at zero,
zero, of course.

317
00:20:47 --> 00:20:50
When I write down this
condition,

318
00:20:50 --> 00:20:55
So I believe that x transposed,
let me copy,

319
00:20:55 --> 00:21:00
x transpose Ax is positive,
except, of course,

320
00:21:00 --> 00:21:04
at the minimum point,
at zero, of course,

321
00:21:04 --> 00:21:07
I don't expect miracles.

322
00:21:07 --> 00:21:12
So what does its graph look
like, and how do I check,

323
00:21:12 --> 00:21:18
and how do I check that this
really is positive?

324
00:21:18 --> 00:21:23
So we take it's graph for a
minute.

325
00:21:23 --> 00:21:28
What would be the graph of that
function -- it does not have a

326
00:21:28 --> 00:21:29
saddle point.

327
00:21:29 --> 00:21:34
Let me -- I'll raise the board,
here, and stay with this

328
00:21:34 --> 00:21:36
example for a while.

329
00:21:36 --> 00:21:40
So I want to do the graph of --
here's my function,

330
00:21:40 --> 00:21:42
two x squared,
twelve xy-s,

331
00:21:42 --> 00:21:48.69
that could be positive or
negative, and twenty y squared.

332
00:21:48.69 --> 00:21:52.62
But my point is,
so you're seeing the underlying

333
00:21:52.62 --> 00:21:55
point is, that,
the things are positive

334
00:21:55 --> 00:22:00
definite when in some way,
these, these pure squares,

335
00:22:00 --> 00:22:04
squares we know to be positive,
and when those kind of

336
00:22:04 --> 00:22:08
overwhelm this guy,
who could be m- positive or

337
00:22:08 --> 00:22:12
negative,
because some like or have same

338
00:22:12 --> 00:22:18
or have same or different signs,
when these are big enough they

339
00:22:18 --> 00:22:24
overwhelm this guy and make the
total thing positive,

340
00:22:24 --> 00:22:27
and what would the graph now
look like?

341
00:22:27 --> 00:22:32
Let me draw the x - well,
let me draw the x direction,

342
00:22:32 --> 00:22:36
the y direction,
and the origin,

343
00:22:36 --> 00:22:41
at zero, zero,
I'm there, where do I go as I

344
00:22:41 --> 00:22:44
move away from the origin?

345
00:22:44 --> 00:22:49
Where do I go as I move away
from the origin?

346
00:22:49 --> 00:22:52
I'm sure that I go up.

347
00:22:52 --> 00:22:58
The origin, the center point
here, is a minim because this

348
00:22:58 --> 00:23:02
thing I believe,
and we better see why,

349
00:23:02 --> 00:23:08
it's, the graph is like a bowl,
the

350
00:23:08 --> 00:23:13
graph is like a bowl shape,
it's -- here's the minimum.

351
00:23:13 --> 00:23:19
And because we've got a pure
quadratic, we know it sits at

352
00:23:19 --> 00:23:26
the origin, we know it's tangent
plane, the first derivatives are

353
00:23:26 --> 00:23:29
zero, so, we know,
first derivatives,

354
00:23:29 --> 00:23:36
First derivatives are all zero,
but that's not enough for a

355
00:23:36 --> 00:23:37
minimum.

356
00:23:37 --> 00:23:40
It's first derivatives were
zero here.

357
00:23:40 --> 00:23:45
So, the partial derivatives,
the first derivatives,

358
00:23:45 --> 00:23:46
are zero.

359
00:23:46 --> 00:23:51
Again, because first
derivatives are gonna have an x

360
00:23:51 --> 00:23:56
or an a y, or a y in them,
they'll be zero at the origin.

361
00:23:56 --> 00:24:00
It's the second derivatives
that

362
00:24:00 --> 00:24:02
control everything.

363
00:24:02 --> 00:24:07
It's the second derivatives
that this matrix is telling us,

364
00:24:07 --> 00:24:09
and somehow -- here's my point.

365
00:24:09 --> 00:24:13
You remember in Calculus,
how did you decide on a

366
00:24:13 --> 00:24:14
minimum?

367
00:24:14 --> 00:24:18
First requirement was,
that the derivative had to be

368
00:24:18 --> 00:24:18
zero.

369
00:24:18 --> 00:24:22
But then you didn't know if you
had

370
00:24:22 --> 00:24:24
a minimum or a maximum.

371
00:24:24 --> 00:24:28
To know that you had a minimum,
you had to look at the second

372
00:24:28 --> 00:24:29.67
derivative.

373
00:24:29.67 --> 00:24:34
The second derivative had to be
positive, the slope had to be

374
00:24:34 --> 00:24:37
increasing as you went through
the minimum point.

375
00:24:37 --> 00:24:41
The curvature had to go
upwards, and that's what we're

376
00:24:41 --> 00:24:45.1
doing now in
two dimensions,

377
00:24:45.1 --> 00:24:47
and in n dimensions.

378
00:24:47 --> 00:24:50
So we're doing what we did in
Calculus.

379
00:24:50 --> 00:24:56
Second derivative positive,
m- will now become that the

380
00:24:56 --> 00:25:00
matrix of second derivatives is
positive definite.

381
00:25:00 --> 00:25:05
Can I just -- like a
translation of -- this is how

382
00:25:05 --> 00:25:12.52
minimum are coming in,
ithe beginning of Calculus --

383
00:25:12.52 --> 00:25:19
we had a minimum was associated
with second derivative,

384
00:25:19 --> 00:25:21
being positive.

385
00:25:21 --> 00:25:26
And first derivative zero,
of course.

386
00:25:26 --> 00:25:32
Derivative, first derivative,
but it was the second

387
00:25:32 --> 00:25:38
derivative that told us we had a
minimum.

388
00:25:38 --> 00:25:42
And now, in 18.06,
in linear algebra,

389
00:25:42 --> 00:25:47
we'll have a minim for our
function now,

390
00:25:47 --> 00:25:53
our function will have,
for your function be a function

391
00:25:53 --> 00:25:58
not of just x but several
variables, the way functions

392
00:25:58 --> 00:26:04
really are in real life,
the minimum will be when the

393
00:26:04 --> 00:26:12
matrix of second derivatives,
the matrix here was one by one,

394
00:26:12 --> 00:26:18.83
there was just one second
derivative, now we've got lots.

395
00:26:18.83 --> 00:26:21
Is positive definite.

396
00:26:21 --> 00:26:26.49
So positive for a number
translates into positive

397
00:26:26.49 --> 00:26:28
definite for a matrix.

398
00:26:28 --> 00:26:34.04
And it this brings everything
you check pivots,

399
00:26:34.04 --> 00:26:39.48
you check determinants,
you check all your values,

400
00:26:39.48 --> 00:26:44
or you check this minimum
stuff.

401
00:26:44 --> 00:26:44
OK.

402
00:26:44 --> 00:26:47
Let me come back to this graph.

403
00:26:47 --> 00:26:49
That graph goes upwards.

404
00:26:49 --> 00:26:51
And I'll have to see why.

405
00:26:51 --> 00:26:56.81
How do I know that this,
that this function is always

406
00:26:56.81 --> 00:26:57
positive?

407
00:26:57 --> 00:27:02
Can you look at that and tell
that it's always positive?

408
00:27:02 --> 00:27:06
Maybe two by two,
you could feel pretty sure,

409
00:27:06 --> 00:27:13.02
but what's the good way to show
that this thing is always

410
00:27:13.02 --> 00:27:13
positive?

411
00:27:13 --> 00:27:17
If we can express it,
as, in terms of squares,

412
00:27:17 --> 00:27:22
because that's what we know for
any x and y, whatever,

413
00:27:22 --> 00:27:27
if we're squaring something we
certainly are not negative.

414
00:27:27 --> 00:27:31
So I believe that this
expression, this function,

415
00:27:31 --> 00:27:35.53
could be written as a sum of
squares.

416
00:27:35.53 --> 00:27:39
Can you tell me -- see,
because all the problems,

417
00:27:39 --> 00:27:43
the headaches are coming from
this xy term.

418
00:27:43 --> 00:27:48
If we can get expressions -- if
we can get that inside a square,

419
00:27:48 --> 00:27:52
so actually,
what we're doing is something

420
00:27:52 --> 00:27:57
called, that you've seen called
completing the square.

421
00:27:57 --> 00:28:01
Let me start the
square and you complete it.

422
00:28:01 --> 00:28:07
OK, I think we have two of x
plus, now I don't remember how

423
00:28:07 --> 00:28:11
many y-s we need,
but you'll figure it out,

424
00:28:11 --> 00:28:12.41
squared.

425
00:28:12.41 --> 00:28:18
How many y-s should I put in
here, to make -- what do I want

426
00:28:18 --> 00:28:22
to do, the two x squared-s will
be correct, right?

427
00:28:22 --> 00:28:29.5
What I want to do is put in the
right number of y-s to get

428
00:28:29.5 --> 00:28:31.39
the twelve xy correct.

429
00:28:31.39 --> 00:28:34
And what is that number of y-s?

430
00:28:34 --> 00:28:39
Let's see, I've got two times,
and so I really want six xy-s

431
00:28:39 --> 00:28:43
to come out of here,
I think maybe if I put three

432
00:28:43 --> 00:28:46
there, does that look right to
you?

433
00:28:46 --> 00:28:49
I have two- this is,
we can mentally,

434
00:28:49 --> 00:28:52
multiply out,
that's X squared,

435
00:28:52 --> 00:28:55
that's right,
that's six X Y,

436
00:28:55 --> 00:29:00
times the two gives from,
right, and how many Y squareds

437
00:29:00 --> 00:29:01
have I now got?

438
00:29:01 --> 00:29:05
How many Y squareds have I now
got from this term?

439
00:29:05 --> 00:29:06
Eighteen.

440
00:29:06 --> 00:29:09
Eighteen was the key number,
remember?

441
00:29:09 --> 00:29:13
Now if I want to make it
twenty,

442
00:29:13 --> 00:29:16
then I've got two left.

443
00:29:16 --> 00:29:17
Two y squared-s.

444
00:29:17 --> 00:29:24
That's completing the square,
and it's, now I can see that

445
00:29:24 --> 00:29:30
that function is positive,
because it's all squares.

446
00:29:30 --> 00:29:34.07
I've got two squares,
added together,

447
00:29:34.07 --> 00:29:36.64
I couldn't go negative.

448
00:29:36.64 --> 00:29:41
What if I went back to that
seven?

449
00:29:41 --> 00:29:44
If instead of twenty that
number was a seven,

450
00:29:44 --> 00:29:46
then what would happen?

451
00:29:46 --> 00:29:50
This would still be correct,
I'd still have this square,

452
00:29:50 --> 00:29:53
to get the two x squared and
the twelve xy,

453
00:29:53 --> 00:29:57
and I'd have eighteen y squared
and then what would I do here?

454
00:29:57 --> 00:30:00
I'd have to remove eleven y
squared-s, right,

455
00:30:00 --> 00:30:05
if I only had a seven here,
then instead of -- when I had a

456
00:30:05 --> 00:30:09
twenty
I had two more to put in,

457
00:30:09 --> 00:30:15
when I had an eighteen,
which was the marginal case,

458
00:30:15 --> 00:30:18
I had no more to put in.

459
00:30:18 --> 00:30:23
When I had a seven,
which was the case below zero,

460
00:30:23 --> 00:30:28.51
the indefinite case,
I had minus eleven.

461
00:30:28.51 --> 00:30:28
OK.

462
00:30:28 --> 00:30:35
Now, so, you can see now,
that this thing is a bowl.

463
00:30:35 --> 00:30:39
It's going upwards,
if I cut it at a plane,

464
00:30:39 --> 00:30:42.25
z equal to one,
say, I would get,

465
00:30:42.25 --> 00:30:46
I would get a curve,
what would be the equation for

466
00:30:46 --> 00:30:47
that curve?

467
00:30:47 --> 00:30:52
If I cut it at height one,
the equation would be this

468
00:30:52 --> 00:30:55.57
thing equal to one.

469
00:30:55.57 --> 00:30:58
And that curve would be an
ellipse.

470
00:30:58 --> 00:31:02
So actually,
already, I've blocked into the

471
00:31:02 --> 00:31:07
lecture, the different pieces
that we're aiming for.

472
00:31:07 --> 00:31:11
We're aiming for the tests,
which this passed;

473
00:31:11 --> 00:31:15
we're aiming for the connection
to a minimum,

474
00:31:15 --> 00:31:19.64
which this -- which we see in
the graph,

475
00:31:19.64 --> 00:31:24
and if we chop it up,
if we set this thing equal to

476
00:31:24 --> 00:31:29
one, if I set that thing equal
to one, that -- what that gives

477
00:31:29 --> 00:31:31
me is, the cross-section.

478
00:31:31 --> 00:31:36
It gives me this,
this curve, and its equation is

479
00:31:36 --> 00:31:39
this thing equals one,
and that's an ellipse.

480
00:31:39 --> 00:31:43
Whereas if I cut through a
saddle

481
00:31:43 --> 00:31:46.09
point, I get a hyperbola.

482
00:31:46.09 --> 00:31:46
OK.

483
00:31:46 --> 00:31:51
But this minimum stuff is
really what I'm most interested

484
00:31:51 --> 00:31:51
in.

485
00:31:51 --> 00:31:52
OK.

486
00:31:52 --> 00:31:55
By -- I just have to ask,
do you recognize,

487
00:31:55 --> 00:32:01
I mean, these numbers here,
the two that appeared outside,

488
00:32:01 --> 00:32:07
the three that appeared inside,
the two that appeared there --

489
00:32:07 --> 00:32:12
actually, those numbers come
from elimination.

490
00:32:12 --> 00:32:18
Completing the square is our
good old method of Gaussian

491
00:32:18 --> 00:32:24
elimination, in expressed in
terms of these squares.

492
00:32:24 --> 00:32:27
The -- let me show you what I
mean.

493
00:32:27 --> 00:32:33
I just think those numbers are
no accident,

494
00:32:33 --> 00:32:38
If I take my matrix two,
six, six, and twenty,

495
00:32:38 --> 00:32:43
and I do elimination,
then the pivot is two and I

496
00:32:43 --> 00:32:46
take three, what's the
multiplier?

497
00:32:46 --> 00:32:51.41
How much of row one do I take
away from row two?

498
00:32:51.41 --> 00:32:52
Three.

499
00:32:52 --> 00:32:58
So what you're seeing in this,
completing the square,

500
00:32:58 --> 00:33:02
is the pivots outside and the
multiplier inside.

501
00:33:02 --> 00:33:04
Just do that again?

502
00:33:04 --> 00:33:07
The pivot is two,
three -- three of those away

503
00:33:07 --> 00:33:11
from that gives me two,
six, zero, and what's the

504
00:33:11 --> 00:33:12
second pivot?

505
00:33:12 --> 00:33:17
Three of this away from this,
three sixes'll be eighteen,

506
00:33:17 --> 00:33:21
and the second pivot will also
be a

507
00:33:21 --> 00:33:21
two.

508
00:33:21 --> 00:33:26
So that's the U,
this is the A,

509
00:33:26 --> 00:33:34.85
and of course the L was one,
zero, one, and the multiplier

510
00:33:34.85 --> 00:33:36.34
was three.

511
00:33:36.34 --> 00:33:42
So, completing the square is
elimination.

512
00:33:42 --> 00:33:48
Why I happy to see,
happy to see that coming

513
00:33:48 --> 00:33:50
together?

514
00:33:50 --> 00:33:56
Because I know about
elimination

515
00:33:56 --> 00:33:58
for m by m matrices.

516
00:33:58 --> 00:34:03
I just started talking about
completing the square,

517
00:34:03 --> 00:34:05
here, for two by twos.

518
00:34:05 --> 00:34:08
But now I see what's going on.

519
00:34:08 --> 00:34:13
Completing the square really
amounts to splitting this thing

520
00:34:13 --> 00:34:18
into a sum of squares,
so what's the critical thing --

521
00:34:18 --> 00:34:24
I have a lot of squares,
and inside those squares are

522
00:34:24 --> 00:34:28
multipliers but they're squares,
and the question is,

523
00:34:28 --> 00:34:31.37
what's outside these squares?

524
00:34:31.37 --> 00:34:36
When I complete the square,
what are the numbers that go

525
00:34:36 --> 00:34:36
outside?

526
00:34:36 --> 00:34:38
They're the pivots.

527
00:34:38 --> 00:34:43
They're the pivots,
and that's why positive pivots

528
00:34:43 --> 00:34:46
give me sum of squares.

529
00:34:46 --> 00:34:49.55
Positive pivots,
those pivots are the numbers

530
00:34:49.55 --> 00:34:53
that go outside the squares,
so positive pivots,

531
00:34:53 --> 00:34:56
sum of squares,
everything positive,

532
00:34:56 --> 00:34:59
graph goes up,
a minimum at the origin,

533
00:34:59 --> 00:35:03
it's all connected together;
all connected together.

534
00:35:03 --> 00:35:07
And in the two by two case,
you can see those connections,

535
00:35:07 --> 00:35:11
but linear algebra now can go
up to

536
00:35:11 --> 00:35:13
three by three,
m by m.

537
00:35:13 --> 00:35:15
Let's do that next.

538
00:35:15 --> 00:35:20.52
Can I just, before I leave two
by two, I've written this

539
00:35:20.52 --> 00:35:26
expression "matrix of second
derivatives." What's the matrix

540
00:35:26 --> 00:35:28
of second derivatives?

541
00:35:28 --> 00:35:32
That's one second derivative
now, but if I'm in two

542
00:35:32 --> 00:35:37
dimensions,
I have a two by two matrix,

543
00:35:37 --> 00:35:44.04
it's the second x derivative,
the second x derivative goes

544
00:35:44.04 --> 00:35:48.88
there -- shall I write it --
fxx, if you like,

545
00:35:48.88 --> 00:35:54
fxx, that means the second
derivative of f in the x

546
00:35:54 --> 00:35:59
direction.
fyy, second derivative in the y

547
00:35:59 --> 00:36:00
direction.

548
00:36:00 --> 00:36:05
Those are the pure derivatives,
second derivatives.

549
00:36:05 --> 00:36:07
They have to be positive.

550
00:36:07 --> 00:36:09
For a minimum.

551
00:36:09 --> 00:36:13
This number has to be positive
for a minimum.

552
00:36:13 --> 00:36:17.75
That number has to be positive
for a minimum.

553
00:36:17.75 --> 00:36:19
But, that's not enough.

554
00:36:19 --> 00:36:26
Those numbers have to somehow
be big enough to overcome this

555
00:36:26 --> 00:36:30
cross-derivative,
Why is the matrix symmetric?

556
00:36:30 --> 00:36:35
Because the second derivative f
with respect to x and y is equal

557
00:36:35 --> 00:36:40
to -- I can, that's the
beautiful fact about second

558
00:36:40 --> 00:36:45
derivatives, is I can do those
in either order and I get the

559
00:36:45 --> 00:36:46
same thing.

560
00:36:46 --> 00:36:52
So this is the same as that,
and so, that's the matrix of

561
00:36:52 --> 00:36:54
second derivatives.

562
00:36:54 --> 00:36:58
And the test is,
it has to be positive definite.

563
00:36:58 --> 00:37:02.72
You might remember,
from, tucked in somewhere near

564
00:37:02.72 --> 00:37:07
the end of eighteen o' two or at
least in the book,

565
00:37:07 --> 00:37:12
was the condition for a
minimum, For a function of two

566
00:37:12 --> 00:37:13.9
variables.

567
00:37:13.9 --> 00:37:16
Let's -- when do you have a
minimum?

568
00:37:16 --> 00:37:20
For a function of two
variables, believe me,

569
00:37:20 --> 00:37:22
that's what Calculus is for.

570
00:37:22 --> 00:37:26
The condition is first
derivatives have to be zero.

571
00:37:26 --> 00:37:30
And the matrix of second
derivatives has to be positive

572
00:37:30 --> 00:37:31
definite.

573
00:37:31 --> 00:37:36
So you maybe remember there was
an fxx times an fyy

574
00:37:36 --> 00:37:40
that had to be bigger than an
an fxy squared,

575
00:37:40 --> 00:37:44
that's just our determinant,
two by two.

576
00:37:44 --> 00:37:49
But now, we now know the answer
for three by three,

577
00:37:49 --> 00:37:55
m by m, because we can do
elimination by m by m matrices,

578
00:37:55 --> 00:38:00
we can connect eigenvalues of m
by m matrices,

579
00:38:00 --> 00:38:08
we can do sum of squares,
sum of m squares instead of

580
00:38:08 --> 00:38:15
only two squares;
and so let's take a,

581
00:38:15 --> 00:38:24
let me go over here to do a
three by three example.

582
00:38:24 --> 00:38:29
So, three by three example.

583
00:38:29 --> 00:38:29
OK.

584
00:38:29 --> 00:38:37
Oh, let me -- shall I use my
favorite matrix?

585
00:38:37 --> 00:38:45
You've seen this matrix before.

586
00:38:45 --> 00:38:50
Yes, let's use the good matrix,
four by one,

587
00:38:50 --> 00:38:51
oops, open.

588
00:38:51 --> 00:38:54
Is that matrix positive
definite?

589
00:38:54 --> 00:39:01
What's -- so I'm going to ask
questions about this matrix,

590
00:39:01 --> 00:39:05
is it positive definite,
first of all?

591
00:39:05 --> 00:39:10
What's the function associated
with that matrix,

592
00:39:10 --> 00:39:14
what's the x transpose Ax?

593
00:39:14 --> 00:39:18
Is -- do we have a minimum for
that function,

594
00:39:18 --> 00:39:18
at zero?

595
00:39:18 --> 00:39:21
And then even what's the
geometry?

596
00:39:21 --> 00:39:21
OK.

597
00:39:21 --> 00:39:24
First of all,
is the matrix positive

598
00:39:24 --> 00:39:29
definite, now I've given you the
numbers there so you can take

599
00:39:29 --> 00:39:32
the determinants,
maybe that's the quickest,

600
00:39:32 --> 00:39:36
is that what you would do
mentally,

601
00:39:36 --> 00:39:40
if I give you all a matrix on a
quiz and say is it positive

602
00:39:40 --> 00:39:42.02
definite or not?

603
00:39:42.02 --> 00:39:46
I would take that determinant
and I'd give the answer two.

604
00:39:46 --> 00:39:50
I would take that determinant
and I would give the answer for

605
00:39:50 --> 00:39:55
that two by two determinant,
I'd give the answer three,

606
00:39:55 --> 00:39:58
and anybody remember the answer
for

607
00:39:58 --> 00:40:01
the three by three determinant?

608
00:40:01 --> 00:40:06
It was four,
you remember for these special

609
00:40:06 --> 00:40:10.04
matrices, when we do
determinants,

610
00:40:10.04 --> 00:40:13
they went up two,
three, four,

611
00:40:13 --> 00:40:17
five, six, they just went up
linearly.

612
00:40:17 --> 00:40:22
So that matrix has -- the
determinants are two,

613
00:40:22 --> 00:40:24
three, and four.

614
00:40:24 --> 00:40:26
Pivots.

615
00:40:26 --> 00:40:28
What are the pivots for that
matrix?

616
00:40:28 --> 00:40:32
I'll tell you,
they're -- the first pivot is

617
00:40:32 --> 00:40:37
two, the next pivot is three
over two, the next pivot is four

618
00:40:37 --> 00:40:38
over three.

619
00:40:38 --> 00:40:42
Because, the product of the
pivots has to give me those

620
00:40:42 --> 00:40:43
determinants.

621
00:40:43 --> 00:40:47
The product of these two pivots
gives me that determinant;

622
00:40:47 --> 00:40:53
the
product of all the pivots gives

623
00:40:53 --> 00:40:55
me that determinant.

624
00:40:55 --> 00:40:59
What are the eigenvalues?

625
00:40:59 --> 00:41:01
Oh, I don't know.

626
00:41:01 --> 00:41:09
The eigenvalues I've got,
what do I have a cubic equation

627
00:41:09 --> 00:41:12
-- a degree three equation?

628
00:41:12 --> 00:41:17
There are three eigenvalues to
find.

629
00:41:17 --> 00:41:24
If I believe what I've said
today, what do I know about

630
00:41:24 --> 00:41:27.91
these
eigenvalues,

631
00:41:27.91 --> 00:41:31
even though I don't know the
exact numbers.

632
00:41:31 --> 00:41:34
I -- I remember the numbers.

633
00:41:34 --> 00:41:39
Because these matrices are so
important that people figure

634
00:41:39 --> 00:41:40
them.

635
00:41:40 --> 00:41:45
But -- what do you believe to
be true about these three

636
00:41:45 --> 00:41:51
eigenvalues -- you believe that
they are all positive.

637
00:41:51 --> 00:41:52
They're all positive.

638
00:41:52 --> 00:41:56
I think that they are two minus
square root of two,

639
00:41:56 --> 00:41:59
two, and two plus the square
root of two.

640
00:41:59 --> 00:42:00
I think.

641
00:42:00 --> 00:42:04
Let me just -- I can't write
those numbers down without

642
00:42:04 --> 00:42:09
checking the simple checks,
what the first simple check is

643
00:42:09 --> 00:42:13
the trace, so if I add those
numbers I

644
00:42:13 --> 00:42:16
get six and if I add those
numbers I get six.

645
00:42:16 --> 00:42:21
The other simple test is the
determinant, if I -- can you do

646
00:42:21 --> 00:42:24
this, can you multiply those
numbers together?

647
00:42:24 --> 00:42:27
I guess we can multiply by two
out there.

648
00:42:27 --> 00:42:32
What's two minus square root of
two times two plus square root

649
00:42:32 --> 00:42:35
of
two, that'll be four minus two,

650
00:42:35 --> 00:42:39.07
that'll be two,
yeah, two times two,

651
00:42:39.07 --> 00:42:43
that's got the determinant,
right, so it's got,

652
00:42:43 --> 00:42:47
it's got a chance of being
correct and I think it is.

653
00:42:47 --> 00:42:50
Now, what's the x transpose Ax?

654
00:42:50 --> 00:42:54.07
I better give myself enough
room

655
00:42:54.07 --> 00:42:57
for that.
x transpose Ax for this guy.

656
00:42:57 --> 00:43:01
It's two x1 squareds,
and two x2 squareds,

657
00:43:01 --> 00:43:03
and two x3 squareds.

658
00:43:03 --> 00:43:08
Those come from the diagonal,
those were easy.

659
00:43:08 --> 00:43:12
Now off the diagonal there's a
minus and a minus,

660
00:43:12 --> 00:43:17
they come together there'll be
a minus two

661
00:43:17 --> 00:43:19
minus two whats?

662
00:43:19 --> 00:43:23
Are coming from this one two
and two one position,

663
00:43:23 --> 00:43:24
is the x1 x2.

664
00:43:24 --> 00:43:29
I'm doing mentally a
multiplication of this matrix

665
00:43:29 --> 00:43:34
times a row vector on the left
times a column vector on the

666
00:43:34 --> 00:43:40.9
right, and I know that these
numbers show up in the answer.

667
00:43:40.9 --> 00:43:46
The diagonal is the perfect
square, this off diagonal is a

668
00:43:46 --> 00:43:50
minus two x1 x2,
and there are no x1 x3-s,

669
00:43:50 --> 00:43:54
and there're minus two x2 x3-s.

670
00:43:54 --> 00:43:59.48
And I believe that that
expression is always positive.

671
00:43:59.48 --> 00:44:04
I believe that that curve,
that graph, really,

672
00:44:04 --> 00:44:09
of that function,
this is my function f,

673
00:44:09 --> 00:44:14
and I'm in more dimensions now
than I can draw,

674
00:44:14 --> 00:44:20
it -- but the graph of that
function goes upwards.

675
00:44:20 --> 00:44:22
It's a bowl.

676
00:44:22 --> 00:44:29
Or maybe the right word is --
just forgot, what's a long word

677
00:44:29 --> 00:44:30.87
for bowl?

678
00:44:30.87 --> 00:44:38
Hm, maybe paraboloid,
I think, paraboloid comes in.

679
00:44:38 --> 00:44:42
I'll edit the tape and get that
word in.

680
00:44:42 --> 00:44:45
Bowl, let's say,
is, that, so that,

681
00:44:45 --> 00:44:49
and if I can -- I could
complete the squares,

682
00:44:49 --> 00:44:54
I could write that as the sum
of three squares,

683
00:44:54 --> 00:44:59
and those three squares would
get multiplied by the three

684
00:44:59 --> 00:45:00
pivots.

685
00:45:00 --> 00:45:04
And the pivots are all
positive.

686
00:45:04 --> 00:45:07
So I would have positive pivots
times squares,

687
00:45:07 --> 00:45:12
the net result would be a
positive function and a bowl

688
00:45:12 --> 00:45:13
which goes upwards.

689
00:45:13 --> 00:45:17.29
And then, finally,
if I cut -- if I slice through

690
00:45:17.29 --> 00:45:21
this bowl, if I -- now I'm
asking you to stretch your

691
00:45:21 --> 00:45:25
visualization here,
because I'm in four dimensions,

692
00:45:25 --> 00:45:29
I've
got x1 x2 x3 in the base,

693
00:45:29 --> 00:45:33.26
and this function is z,
or f, or something.

694
00:45:33.26 --> 00:45:35
And its graph is going up.

695
00:45:35 --> 00:45:41
But I'm in four dimensions,
because I've got three in the

696
00:45:41 --> 00:45:46
base and then the upward
direction, but now if I cut

697
00:45:46 --> 00:45:51
through this four-dimensional
picture, at level one,

698
00:45:51 --> 00:45:57
so, suppose I cut
through this thing at height

699
00:45:57 --> 00:45:58
one.

700
00:45:58 --> 00:46:03
So I take all the points that
are at height one.

701
00:46:03 --> 00:46:08
That gives me -- it gave me an
ellipse over there,

702
00:46:08 --> 00:46:13
in that two by two case,
in this case,

703
00:46:13 --> 00:46:19
this will be the equation of an
ellipsoid, a football in other

704
00:46:19 --> 00:46:20
words.

705
00:46:20 --> 00:46:24
Well, not quite a football.

706
00:46:24 --> 00:46:26
A lopsided football.

707
00:46:26 --> 00:46:30
What would be,
can I try to describe to you

708
00:46:30 --> 00:46:35.18
what the ellipsoid will look
like, this ellipsoid,

709
00:46:35.18 --> 00:46:39
I'm sorry that the,
that I've ended the matrix

710
00:46:39 --> 00:46:44
right -- at the point,
let's -- let me be sure you've

711
00:46:44 --> 00:46:45
seen the equation.

712
00:46:45 --> 00:46:48.62
Two x1 squared,
two x2 squared,

713
00:46:48.62 --> 00:46:54
two x3
squared, minus two of the cross

714
00:46:54 --> 00:46:57
parts, equal what?

715
00:46:57 --> 00:47:04.99
That is the equation of a
football, so what do I mean by a

716
00:47:04.99 --> 00:47:08
football or an ellipsoid?

717
00:47:08 --> 00:47:13
I mean that,
well, I'll draw a few.

718
00:47:13 --> 00:47:17
It's like that,
it's got a center,

719
00:47:17 --> 00:47:24
and it's got it's got three
principal directions.

720
00:47:24 --> 00:47:26
This ellipsoid.

721
00:47:26 --> 00:47:32
So -- you see what I'm saying,
if we have a sphere then all

722
00:47:32 --> 00:47:35
directions would be the same.

723
00:47:35 --> 00:47:41
If we had a true football,
or it's closer to a rugby ball,

724
00:47:41 --> 00:47:46
really, because it's more
curved than a football,

725
00:47:46 --> 00:47:52
it would have one long
direction and the other two

726
00:47:52 --> 00:47:53
would be equal.

727
00:47:53 --> 00:47:58
That would be like having a
matrix that had one eigenvalue

728
00:47:58 --> 00:47:59
repeated.

729
00:47:59 --> 00:48:01
And then one other different.

730
00:48:01 --> 00:48:06
So this sphere comes from,
like, the identity matrix,

731
00:48:06 --> 00:48:08
all eigenvalues the same.

732
00:48:08 --> 00:48:12.99
Our rugby ball comes from a
case where --

733
00:48:12.99 --> 00:48:17.05
three, the three,
two of the three eigenvalues

734
00:48:17.05 --> 00:48:18
are the same.

735
00:48:18 --> 00:48:22
But we know how the case where
-- the typical case,

736
00:48:22 --> 00:48:26
where the three eigenvalues
were all different.

737
00:48:26 --> 00:48:31
So this will have -- How do I
say it, if I look at this

738
00:48:31 --> 00:48:36
ellipsoid correctly,
it'll have a major axis,

739
00:48:36 --> 00:48:41
it'll have a middle axis,
and it'll have a minor axis.

740
00:48:41 --> 00:48:45
And those three axes will be in
the direction of the

741
00:48:45 --> 00:48:47.11
eigenvectors.

742
00:48:47.11 --> 00:48:51
And the lengths of those axes
will be determined by the

743
00:48:51 --> 00:48:52
eigenvalues.

744
00:48:52 --> 00:48:57
I can get -- turn this all into
linear algebra,

745
00:48:57 --> 00:49:02.3
because we have -- the right
thing we know about

746
00:49:02.3 --> 00:49:07
eigenvectors and eigenvalues,
for that matrix is what?

747
00:49:07 --> 00:49:12
Of -- let me just tell you
that, repeat the main linear

748
00:49:12 --> 00:49:13
algebra point.

749
00:49:13 --> 00:49:17.5
How could we turn what I said
into algebra;

750
00:49:17.5 --> 00:49:22
we would write this A as Q,
the eigenvector matrix,

751
00:49:22 --> 00:49:26.36
times lambda,
the eigenvalue matrix

752
00:49:26.36 --> 00:49:27
times Q transposed.

753
00:49:27 --> 00:49:31
The principal axis theorem,
we'll call it,

754
00:49:31 --> 00:49:31
now.

755
00:49:31 --> 00:49:36
The eigenvectors tell us the
directions of the principal

756
00:49:36 --> 00:49:36
axes.

757
00:49:36 --> 00:49:40.88
The eigenvalues tell us the
lengths of those axes,

758
00:49:40.88 --> 00:49:44
actually the lengths,
or the half-lengths,

759
00:49:44 --> 00:49:48
or one over the eigenvalues,
it turns out.

760
00:49:48 --> 00:49:53.62
And that is the matrix
factorization which is the most

761
00:49:53.62 --> 00:49:58
important matrix factorization
in our eigenvalue material so

762
00:49:58 --> 00:49:59
far.

763
00:49:59 --> 00:50:03
That's diagonalization for a
symmetric matrix,

764
00:50:03 --> 00:50:08
so instead of the inverse I can
write the transposed.

765
00:50:08 --> 00:50:09
OK.

766
00:50:09 --> 00:50:13
I've -- so what I've tried
today is to tell you the --

767
00:50:13 --> 00:50:18
what's going on with positive
definite matrices.

768
00:50:18 --> 00:50:23
Ah, you see all how all these
pieces are there and linear

769
00:50:23 --> 00:50:25
algebra connects them.

770
00:50:25 --> 00:50:25
OK.

771
00:50:25 --> 00:50:28
See you on Friday.