1
00:00:00 --> 00:00:06
OK, this is the lecture on
positive definite matrices.
2
00:00:06 --> 00:00:12
I made a start on those briefly
in a previous lecture.
3
00:00:12 --> 00:00:19
One point I wanted to make was
the way that this topic brings
4
00:00:19 --> 00:00:25
the whole course together,
pivots, determinants,
5
00:00:25 --> 00:00:29
eigenvalues,
and something new- four plot
6
00:00:29 --> 00:00:34
instability and then something
new
7
00:00:34 --> 00:00:38
in this expression,
x transpose Ax,
8
00:00:38 --> 00:00:43
actually that's the guy to
watch in this lecture.
9
00:00:43 --> 00:00:47
So, so the topic is positive
definite matrix,
10
00:00:47 --> 00:00:49
and what's my goal?
11
00:00:49 --> 00:00:54
First, first goal is,
how can I tell if a matrix is
12
00:00:54 --> 00:00:56
positive definite?
13
00:00:56 --> 00:01:01
So I would like to have tests
to see
14
00:01:01 --> 00:01:04
if you give me a,
a five by five matrix,
15
00:01:04 --> 00:01:07
how do I tell if it's positive
definite?
16
00:01:07 --> 00:01:10
More important is,
what does it mean?
17
00:01:10 --> 00:01:14
Why are we so interested in
this property of positive
18
00:01:14 --> 00:01:15
definiteness?
19
00:01:15 --> 00:01:18
And then, at the end comes some
geometry.
20
00:01:18 --> 00:01:24
Ellipses are connected
with positive definite things.
21
00:01:24 --> 00:01:29
Hyperbolas are not connected
with positive definite things,
22
00:01:29 --> 00:01:32
so we- it's this,
we, there's a geometry too,
23
00:01:32 --> 00:01:38
but mostly it's linear algebra
and -- this application of how
24
00:01:38 --> 00:01:43
do you recognize 'em when you
have a minim is pretty neat.
25
00:01:43 --> 00:01:43.33
OK.
26
00:01:43.33 --> 00:01:47
I'm gonna begin with two by
two.
27
00:01:47 --> 00:01:50
All matrices are symmetric,
right?
28
00:01:50 --> 00:01:54
That's understood;
the matrix is symmetric,
29
00:01:54 --> 00:01:59
now my question is,
is it positive definite?
30
00:01:59 --> 00:02:05
Now, here are some -- each one
of these is a complete test for
31
00:02:05 --> 00:02:07
positive definiteness.
32
00:02:07 --> 00:02:14
If I know the eigenvalues,
my test is are they positive?
33
00:02:14 --> 00:02:16
Are they all positive?
34
00:02:16 --> 00:02:21
If I know these -- so,
A is really -- I look at that
35
00:02:21 --> 00:02:25
number A, here,
as the, as the one by one
36
00:02:25 --> 00:02:30
determinant, and here's the two
by two determinant.
37
00:02:30 --> 00:02:34
So this is the determinant
test.
38
00:02:34 --> 00:02:39.11
This is the eigenvalue test,
this is the determinant test.
39
00:02:39.11 --> 00:02:43
Are the determinants growing in
s- of all, of all end,
40
00:02:43 --> 00:02:47
sort of, can I call them
leading submatrices,
41
00:02:47 --> 00:02:51
they're the first ones the
northwest,
42
00:02:51 --> 00:02:55
Seattle submatrices coming down
from from there,
43
00:02:55 --> 00:02:59.6
they all, all those
determinants have to be
44
00:02:59.6 --> 00:03:03
positive, and then another test
is the pivots.
45
00:03:03 --> 00:03:08
The pivots of a two by two
matrix are the number A for
46
00:03:08 --> 00:03:13
sure, and, since the product is
the determinant,
47
00:03:13 --> 00:03:18
the second pivot must be the
determinant divided by A.
48
00:03:18 --> 00:03:24
And then in here is gonna come
my favorite and my new idea,
49
00:03:24 --> 00:03:30
the, the, the the one to catch,
about x transpose Ax being
50
00:03:30 --> 00:03:31
positive.
51
00:03:31 --> 00:03:34
But we'll have to look at this
guy.
52
00:03:34 --> 00:03:39
This gets, like a star,
because for most,
53
00:03:39 --> 00:03:43
presentations,
the definition of positive
54
00:03:43 --> 00:03:48
definiteness would be this
number four and these numbers
55
00:03:48 --> 00:03:51
one two three would be test
four.
56
00:03:51 --> 00:03:51
OK.
57
00:03:51 --> 00:03:55
Maybe I'll tuck this,
where, you know,
58
00:03:55 --> 00:03:55
OK.
59
00:03:55 --> 00:03:59
So I'll have to look at this x
transpose Ax.
60
00:03:59 --> 00:04:04
Can you, can we just be sure,
how do we know that the
61
00:04:04 --> 00:04:10
eigenvalue
test and the determinant test,
62
00:04:10 --> 00:04:18
pick out the same matrices,
and let me, let's just do a few
63
00:04:18 --> 00:04:19
examples.
64
00:04:19 --> 00:04:21
Some examples.
65
00:04:21 --> 00:04:27
Let me pick the matrix two,
six, six, tell me,
66
00:04:27 --> 00:04:34.82
what number do I have to put
there for the matrix to be
67
00:04:34.82 --> 00:04:38
positive definite?
68
00:04:38 --> 00:04:42
Tell me a sufficiently large
number that would make it
69
00:04:42 --> 00:04:43
positive definite?
70
00:04:43 --> 00:04:47
Let's just practice with these
conditions in the two by two
71
00:04:47 --> 00:04:48
case.
72
00:04:48 --> 00:04:51
Now, when I ask you that,
you don't wanna find the
73
00:04:51 --> 00:04:55
eigenvalues, you would use the
determinant test for that,
74
00:04:55 --> 00:04:59
so,
the first or the pivot test,
75
00:04:59 --> 00:05:04
that, that guy is certainly
positive, that had to happen,
76
00:05:04 --> 00:05:05
and it's OK.
77
00:05:05 --> 00:05:10
How large a number here -- the
number had better be more than
78
00:05:10 --> 00:05:10
what?
79
00:05:10 --> 00:05:15
More than eighteen,
right, because if it's eight --
80
00:05:15 --> 00:05:15
no.
81
00:05:15 --> 00:05:16
More than what?
82
00:05:16 --> 00:05:19
Nineteen, is it?
83
00:05:19 --> 00:05:25
If I have a nineteen here,
is that positive definite?
84
00:05:25 --> 00:05:30
I get thirty eight minus thirty
six, that's OK.
85
00:05:30 --> 00:05:36
If I had an eighteen,
let me play it really close.
86
00:05:36 --> 00:05:42
If I have an eighteen there,
then I positive definite?
87
00:05:42 --> 00:05:43.48
Not quite.
88
00:05:43.48 --> 00:05:51
I would call this guy positive,
so it's useful just to see that
89
00:05:51 --> 00:05:53
this the borderline.
90
00:05:53 --> 00:05:58
That matrix is on the
borderline, I would call that
91
00:05:58 --> 00:06:01
matrix positive semi-definite.
92
00:06:01 --> 00:06:07
And what are the eigenvalues of
that matrix, just since we're
93
00:06:07 --> 00:06:14
given eigenvalues of two by
twos, when it's semi-definite,
94
00:06:14 --> 00:06:18
but not definite,
then the -- I'm squeezing this
95
00:06:18 --> 00:06:24
eigenvalue test down,
-- what's the eigenvalue that I
96
00:06:24 --> 00:06:26
know this matrix has?
97
00:06:26 --> 00:06:30
What kind of a matrix have I
got here?
98
00:06:30 --> 00:06:36
It's a singular matrix,
one of its eigenvalues is zero.
99
00:06:36 --> 00:06:42
That has an eigenvalue zero,
and the other eigenvalue is --
100
00:06:42 --> 00:06:44
from the trace,
twenty.
101
00:06:44 --> 00:06:44
OK.
102
00:06:44 --> 00:06:49
So that, that matrix has
eigenvalues greater than or
103
00:06:49 --> 00:06:52
equal to zero,
and it's that "equal to" that
104
00:06:52 --> 00:06:55
brought this word
"semi-definite" in.
105
00:06:55 --> 00:07:00.47
And, the what are the pivots of
that matrix?
106
00:07:00.47 --> 00:07:04
So the pivots,
so the eigenvalues are zero and
107
00:07:04 --> 00:07:09
twenty, the pivots are,
well, the pivot is two,
108
00:07:09 --> 00:07:11
and what's the next pivot?
109
00:07:11 --> 00:07:13
There isn't one.
110
00:07:13 --> 00:07:18
We got a singular matrix here,
it'll only have one pivot.
111
00:07:18 --> 00:07:25
You see that that's a rank one
matrix, two six is a -- six
112
00:07:25 --> 00:07:32
eighteen is a multiple of two
six, the matrix is singular it
113
00:07:32 --> 00:07:38.21
only has one pivot,
so the pivot test doesn't quite
114
00:07:38.21 --> 00:07:38
pass.
115
00:07:38 --> 00:07:42
The -- let me do the x
transpose Ax.
116
00:07:42 --> 00:07:46
Now this is -- the novelty now.
117
00:07:46 --> 00:07:46
OK.
118
00:07:46 --> 00:07:54.06
What I looking at when I
look at this new combination,
119
00:07:54.06 --> 00:07:57
x transpose Ax.
x is any vector now,
120
00:07:57 --> 00:08:01
so lemme compute,
so any vector,
121
00:08:01 --> 00:08:06
lemme call its components x1
and x2, so that's x.
122
00:08:06 --> 00:08:08
And I put in here A.
123
00:08:08 --> 00:08:15.37
Let's, let's use this example
two six, six eighteen,
124
00:08:15.37 --> 00:08:18
and here's x transposed,
so x1 x2.
125
00:08:18 --> 00:08:24
We're back to real matrices,
after that last lecture that-
126
00:08:24 --> 00:08:27
that said what to do in the
complex case,
127
00:08:27 --> 00:08:30
let's come back to real
matrices.
128
00:08:30 --> 00:08:36
So here's x transpose Ax,
and what I'm interested in is,
129
00:08:36 --> 00:08:41
what do I get if I multiply
those together?
130
00:08:41 --> 00:08:46
I get some function of x1 and
x2, and what is it?
131
00:08:46 --> 00:08:50.62
Let's see, if I do this
multiplication,
132
00:08:50.62 --> 00:08:55
so I do it, lemme,
just, I'll just do it slowly,
133
00:08:55 --> 00:09:00
x1, x2, if I multiply that
matrix, this is 2x1,
134
00:09:00 --> 00:09:06
and 6x2s, and the next row is
6x1s and 18x2s,
135
00:09:06 --> 00:09:10
and now I do this final step
and what do I have?
136
00:09:10 --> 00:09:15.92
I've got 2x1 squareds,
got 2X1 squareds is coming from
137
00:09:15.92 --> 00:09:20
this two, I've got this one
gives me eighteen,
138
00:09:20 --> 00:09:24
well, shall I do the ones in
the middle?
139
00:09:24 --> 00:09:26
How many x1 x2s do I have?
140
00:09:26 --> 00:09:31
Let's see,
x1 times that 6x2 would be six
141
00:09:31 --> 00:09:36
of 'em, and then x2 times this
one will be six more,
142
00:09:36 --> 00:09:38
I get twelve.
143
00:09:38 --> 00:09:42
So, in here is going,
this is the number a,
144
00:09:42 --> 00:09:47
this is the number 2b,
and in here is -- x2 times
145
00:09:47 --> 00:09:53
eighteen x2 will be eighteen x2
squareds and this
146
00:09:53 --> 00:09:55
is the number c.
147
00:09:55 --> 00:09:59
So it's ax1 -- it's like ax
squared.
148
00:09:59 --> 00:10:01.23
2bxy and cy squared.
149
00:10:01.23 --> 00:10:07
For my first point that I
wanted to make by just doing out
150
00:10:07 --> 00:10:12
a multiplication is,
that is as soon as you give me
151
00:10:12 --> 00:10:17
the matrix, as soon as you give
me the matrix,
152
00:10:17 --> 00:10:21
I can --
those are the numbers that
153
00:10:21 --> 00:10:25
appear in -- I'll call this
thing a quadratic,
154
00:10:25 --> 00:10:27
you see, it's not linear
anymore.
155
00:10:27 --> 00:10:31
Ax is linear,
but now I've got an x transpose
156
00:10:31 --> 00:10:35
coming in, I'm up to degree two,
up to degree two,
157
00:10:35 --> 00:10:39
maybe quadratic is the -- use
the word.
158
00:10:39 --> 00:10:40
A quadratic form.
159
00:10:40 --> 00:10:44
It's purely degree two,
there's no linear part,
160
00:10:44 --> 00:10:49
there's no constant part,
there certainly no cubes or
161
00:10:49 --> 00:10:53
fourth powers,
it's all second degree.
162
00:10:53 --> 00:10:58
And my question is -- is that
quantity positive or not?
163
00:10:58 --> 00:11:04
That's -- for every x1 and x2,
that is my new definition --
164
00:11:04 --> 00:11:10
that's my definition of a
positive definite matrix.
165
00:11:10 --> 00:11:16
If this quantity is positive,
if, if, if, it's positive for
166
00:11:16 --> 00:11:21
all x's and y's,
all x1 x2s, then I call them --
167
00:11:21 --> 00:11:26
then that's the matrix is
positive definite.
168
00:11:26 --> 00:11:31
Now, is this guy passing our
test?
169
00:11:31 --> 00:11:36
Well we have,
we anticipated the answer here
170
00:11:36 --> 00:11:41.87
by, by asking first about
eigenvalues and pivots,
171
00:11:41.87 --> 00:11:43.97
and what happened?
172
00:11:43.97 --> 00:11:45
It failed.
173
00:11:45 --> 00:11:47
It barely failed.
174
00:11:47 --> 00:11:52
If I had made this eighteen
down to a seven,
175
00:11:52 --> 00:11:55
it would've totally failed.
176
00:11:55 --> 00:12:01
I do that with the eraser,
and then I'll put back
177
00:12:01 --> 00:12:06
eighteen, because,
seven is such a total disaster,
178
00:12:06 --> 00:12:10
but if -- I'll keep seven for a
second.
179
00:12:10 --> 00:12:14
Is that thing in any way
positive definite?
180
00:12:14 --> 00:12:16
No, absolutely not.
181
00:12:16 --> 00:12:22
I don't know its eigenvalues,
but I know for sure one of them
182
00:12:22 --> 00:12:22.98
is negative.
183
00:12:22.98 --> 00:12:26
Its pivots are two and then the
next pivot would be the
184
00:12:26 --> 00:12:30
determinant over two,
and the determinant is -- what,
185
00:12:30 --> 00:12:33
what's the determinant of this
thing?
186
00:12:33 --> 00:12:37
Fourteen minus thirty six,
I've got a determinant minus
187
00:12:37 --> 00:12:38
twenty two.
188
00:12:38 --> 00:12:41
The next pivot will be -- the
pivots
189
00:12:41 --> 00:12:45
now, of this thing are two and
minus eleven or something.
190
00:12:45 --> 00:12:49
Their product being minus
twenty two the determinant.
191
00:12:49 --> 00:12:52
This thing is not positive
definite.
192
00:12:52 --> 00:12:56
What would be -- let me look at
the x transpose Ax for this guy.
193
00:12:56 --> 00:12:59
What's -- if I do out this
multiplication,
194
00:12:59 --> 00:13:04.5
this eighteen is temporarily
changing to a seven.
195
00:13:04.5 --> 00:13:10
This eighteen is temporarily
changing to a seven,
196
00:13:10 --> 00:13:17
and I know that there's some
numbers x1 and x2 for which that
197
00:13:17 --> 00:13:21
thing, that function,
is negative.
198
00:13:21 --> 00:13:26
And I'm desperately trying to
think what they are.
199
00:13:26 --> 00:13:29
Maybe you can see.
200
00:13:29 --> 00:13:36
Can you tell me a value of x1
and x2 that makes this quantity
201
00:13:36 --> 00:13:37
negative?
202
00:13:37 --> 00:13:40
Oh, maybe one and minus one?
203
00:13:40 --> 00:13:44
Yes, that's -- in this case,
that, will work,
204
00:13:44 --> 00:13:49
right, if I took x1 to be one,
and x2 to be minus one,
205
00:13:49 --> 00:13:54
then I always get something
positive, the two,
206
00:13:54 --> 00:13:59
and the seven minus one
squared, but this would be
207
00:13:59 --> 00:14:04
minus twelve and the whole thing
would be negative;
208
00:14:04 --> 00:14:10
I would have two minus twelve
plus seven, a negative.
209
00:14:10 --> 00:14:15
If I drew the graph,
can I get the little picture in
210
00:14:15 --> 00:14:15
here?
211
00:14:15 --> 00:14:18
If I draw the graph of this
thing?
212
00:14:18 --> 00:14:23
So, graphs, of the function
f(x,y),
213
00:14:23 --> 00:14:31
or f(x), so I say here f(x,y)
equal this -- x transpose Ax,
214
00:14:31 --> 00:14:37
this, this this ax squared,
2bxy, and cy squared.
215
00:14:37 --> 00:14:44
And, let's take the example,
with these numbers.
216
00:14:44 --> 00:14:49
OK, so here's the x axis,
here's the y axis,
217
00:14:49 --> 00:14:54
and here's -- up is the
function.
218
00:14:54 --> 00:14:58.18
z, if you like,
or f.
219
00:14:58.18 --> 00:15:01
I apologize,
and let me, just once in my
220
00:15:01 --> 00:15:05
life here, put an arrow over
these, these,
221
00:15:05 --> 00:15:07
axes so you see them.
222
00:15:07 --> 00:15:13
That's the vector and I just,
see, instead of x1 and x2,
223
00:15:13 --> 00:15:16
I made them x- the components x
and y.
224
00:15:16 --> 00:15:16
OK.
225
00:15:16 --> 00:15:21
So, so, what's a graph of 2x
squared,
226
00:15:21 --> 00:15:25
twelve xy, and seven y squared?
227
00:15:25 --> 00:15:30
I'd like to see -- I not the
greatest artist,
228
00:15:30 --> 00:15:37
but let's -- tell me something
about this graph of this
229
00:15:37 --> 00:15:38.87
function.
230
00:15:38.87 --> 00:15:44
Whoa, tell me one point that it
goes through.
231
00:15:44 --> 00:15:45
The origin.
232
00:15:45 --> 00:15:46
Right?
233
00:15:46 --> 00:15:52
Even this artist can get this
thing
234
00:15:52 --> 00:15:56
to go through the origin,
when these are zero,
235
00:15:56 --> 00:15:58
I, I certainly get zero.
236
00:15:58 --> 00:15:58
OK.
237
00:15:58 --> 00:16:00
Some more points.
238
00:16:00 --> 00:16:04
If x is one and y is zero,
then I'm going upwards,
239
00:16:04 --> 00:16:08
so I'm going up this way,
and I'm, I'm going up,
240
00:16:08 --> 00:16:13
like, two x squared in that
direction.
241
00:16:13 --> 00:16:16
So -- that's meant to be a
parabola.
242
00:16:16 --> 00:16:19
And, suppose x stays zero and y
increases.
243
00:16:19 --> 00:16:24
Well, y could be positive or
negative; it's seven y squared.
244
00:16:24 --> 00:16:27.42
Is this function going upward?
245
00:16:27.42 --> 00:16:32
In the x direction it's going
upward, and in the y direction
246
00:16:32 --> 00:16:37
it's going upwards,
and if x equals y then the
247
00:16:37 --> 00:16:42
forty-five degree direction is
certainly going upwards;
248
00:16:42 --> 00:16:47
because then we'd have what,
about, everything would be
249
00:16:47 --> 00:16:49
positive, but what?
250
00:16:49 --> 00:16:54
This function -- what's the
graph of this function?
251
00:16:54 --> 00:16:55
Look like?
252
00:16:55 --> 00:16:59
Tell me the word that
describes the graph of this
253
00:16:59 --> 00:17:00
function.
254
00:17:00 --> 00:17:03
This is the non-positive
definite here,
255
00:17:03 --> 00:17:06
everybody's with me here,
for some reason got started in
256
00:17:06 --> 00:17:10
a negative direction,
your case that isn't positive
257
00:17:10 --> 00:17:11
definite.
258
00:17:11 --> 00:17:13
And what's the graph look like
that goes up,
259
00:17:13 --> 00:17:17
but does it -- do we have a
minimum here,
260
00:17:17 --> 00:17:20
does it go from,
from the origin?
261
00:17:20 --> 00:17:21.58
Completely?
262
00:17:21.58 --> 00:17:25
No, because we just checked
that this thing failed.
263
00:17:25 --> 00:17:31
It failed along the direction
when x was minus y -- we have a
264
00:17:31 --> 00:17:33
saddle point,
let me put myself,
265
00:17:33 --> 00:17:38
let me, to the least,
tell you the word.
266
00:17:38 --> 00:17:46
This thing, goes up in some
directions, but down in other
267
00:17:46 --> 00:17:55
directions, and if we actually
knew what a saddle looked like
268
00:17:55 --> 00:18:02
or thinks saddles do that -- the
way your legs go is,
269
00:18:02 --> 00:18:07
like, down, up,
the way,
270
00:18:07 --> 00:18:10
you, looking like,
forward, and,
271
00:18:10 --> 00:18:16
the, and drawing the thing is
even worse than describing --
272
00:18:16 --> 00:18:22
I'm just going to say in some
directions we go up and in other
273
00:18:22 --> 00:18:28
directions, there is,
a saddle -- Now I'm sorry I put
274
00:18:28 --> 00:18:34
that on the front board,
you have no way to cover it,
275
00:18:34 --> 00:18:36
but it's a saddle.
276
00:18:36 --> 00:18:36
OK.
277
00:18:36 --> 00:18:40.48
And, and this is a saddle
point, it's the,
278
00:18:40.48 --> 00:18:45
it's the point that's at the
maximum in some directions and
279
00:18:45 --> 00:18:49
at the minimum in other
directions.
280
00:18:49 --> 00:18:53
And actually,
the perfect directions to look
281
00:18:53 --> 00:18:56
are the eigenvector directions.
282
00:18:56 --> 00:18:57
We'll see that.
283
00:18:57 --> 00:19:03
So this is,
not a positive definite matrix.
284
00:19:03 --> 00:19:03
OK.
285
00:19:03 --> 00:19:08
Now I'm coming back to this
example, getting rid of this
286
00:19:08 --> 00:19:12
seven, let's move it up to
twenty.
287
00:19:12 --> 00:19:18
Let's, let's let's make the
thing really positive definite.
288
00:19:18 --> 00:19:18
OK.
289
00:19:18 --> 00:19:22
So this is, this number's now
twenty.
290
00:19:22 --> 00:19:23
c is now twenty.
291
00:19:23 --> 00:19:24
OK.
292
00:19:24 --> 00:19:30
Now that passes the test,
which I haven't proved,
293
00:19:30 --> 00:19:33
of course, it passes the test
for positive pivots.
294
00:19:33 --> 00:19:37
It passes the test for positive
eigenvalues.
295
00:19:37 --> 00:19:41
How can you tell that the
eigenvalues of that matrix are
296
00:19:41 --> 00:19:44
positive without actually
finding them?
297
00:19:44 --> 00:19:48
Of course,
two by two I could find them,
298
00:19:48 --> 00:19:51
but can you see -- how do I
know they're positive?
299
00:19:51 --> 00:19:55
I know that their product is --
I know that lambda one times
300
00:19:55 --> 00:19:56
lambda two is positive,
why?
301
00:19:56 --> 00:20:00
Because that's the determinant,
right, lambda one times lambda
302
00:20:00 --> 00:20:04
two is the determinant,
which is forty minus thirty-six
303
00:20:04 --> 00:20:05
is
four.
304
00:20:05 --> 00:20:08
So the determinant is four.
305
00:20:08 --> 00:20:12
And the trace,
the sum down the diagonal,
306
00:20:12 --> 00:20:13.44
is twenty-two.
307
00:20:13.44 --> 00:20:16
So, they multiply to give four.
308
00:20:16 --> 00:20:22
So that leaves the possibility
they're either both positive or
309
00:20:22 --> 00:20:23
both negative.
310
00:20:23 --> 00:20:29.08
But if they're both negative,
the trace couldn't be
311
00:20:29.08 --> 00:20:29
twenty-two.
312
00:20:29 --> 00:20:31
So they're both positive.
313
00:20:31 --> 00:20:35
So both of the eigenvalues that
are positive,
314
00:20:35 --> 00:20:39.73
both of the pivots are positive
-- the determinants are
315
00:20:39.73 --> 00:20:43
positive, and I believe that
this function is positive
316
00:20:43 --> 00:20:47
everywhere except at zero,
zero, of course.
317
00:20:47 --> 00:20:50
When I write down this
condition,
318
00:20:50 --> 00:20:55
So I believe that x transposed,
let me copy,
319
00:20:55 --> 00:21:00
x transpose Ax is positive,
except, of course,
320
00:21:00 --> 00:21:04
at the minimum point,
at zero, of course,
321
00:21:04 --> 00:21:07
I don't expect miracles.
322
00:21:07 --> 00:21:12
So what does its graph look
like, and how do I check,
323
00:21:12 --> 00:21:18
and how do I check that this
really is positive?
324
00:21:18 --> 00:21:23
So we take it's graph for a
minute.
325
00:21:23 --> 00:21:28
What would be the graph of that
function -- it does not have a
326
00:21:28 --> 00:21:29
saddle point.
327
00:21:29 --> 00:21:34
Let me -- I'll raise the board,
here, and stay with this
328
00:21:34 --> 00:21:36
example for a while.
329
00:21:36 --> 00:21:40
So I want to do the graph of --
here's my function,
330
00:21:40 --> 00:21:42
two x squared,
twelve xy-s,
331
00:21:42 --> 00:21:48.69
that could be positive or
negative, and twenty y squared.
332
00:21:48.69 --> 00:21:52.62
But my point is,
so you're seeing the underlying
333
00:21:52.62 --> 00:21:55
point is, that,
the things are positive
334
00:21:55 --> 00:22:00
definite when in some way,
these, these pure squares,
335
00:22:00 --> 00:22:04
squares we know to be positive,
and when those kind of
336
00:22:04 --> 00:22:08
overwhelm this guy,
who could be m- positive or
337
00:22:08 --> 00:22:12
negative,
because some like or have same
338
00:22:12 --> 00:22:18
or have same or different signs,
when these are big enough they
339
00:22:18 --> 00:22:24
overwhelm this guy and make the
total thing positive,
340
00:22:24 --> 00:22:27
and what would the graph now
look like?
341
00:22:27 --> 00:22:32
Let me draw the x - well,
let me draw the x direction,
342
00:22:32 --> 00:22:36
the y direction,
and the origin,
343
00:22:36 --> 00:22:41
at zero, zero,
I'm there, where do I go as I
344
00:22:41 --> 00:22:44
move away from the origin?
345
00:22:44 --> 00:22:49
Where do I go as I move away
from the origin?
346
00:22:49 --> 00:22:52
I'm sure that I go up.
347
00:22:52 --> 00:22:58
The origin, the center point
here, is a minim because this
348
00:22:58 --> 00:23:02
thing I believe,
and we better see why,
349
00:23:02 --> 00:23:08
it's, the graph is like a bowl,
the
350
00:23:08 --> 00:23:13
graph is like a bowl shape,
it's -- here's the minimum.
351
00:23:13 --> 00:23:19
And because we've got a pure
quadratic, we know it sits at
352
00:23:19 --> 00:23:26
the origin, we know it's tangent
plane, the first derivatives are
353
00:23:26 --> 00:23:29
zero, so, we know,
first derivatives,
354
00:23:29 --> 00:23:36
First derivatives are all zero,
but that's not enough for a
355
00:23:36 --> 00:23:37
minimum.
356
00:23:37 --> 00:23:40
It's first derivatives were
zero here.
357
00:23:40 --> 00:23:45
So, the partial derivatives,
the first derivatives,
358
00:23:45 --> 00:23:46
are zero.
359
00:23:46 --> 00:23:51
Again, because first
derivatives are gonna have an x
360
00:23:51 --> 00:23:56
or an a y, or a y in them,
they'll be zero at the origin.
361
00:23:56 --> 00:24:00
It's the second derivatives
that
362
00:24:00 --> 00:24:02
control everything.
363
00:24:02 --> 00:24:07
It's the second derivatives
that this matrix is telling us,
364
00:24:07 --> 00:24:09
and somehow -- here's my point.
365
00:24:09 --> 00:24:13
You remember in Calculus,
how did you decide on a
366
00:24:13 --> 00:24:14
minimum?
367
00:24:14 --> 00:24:18
First requirement was,
that the derivative had to be
368
00:24:18 --> 00:24:18
zero.
369
00:24:18 --> 00:24:22
But then you didn't know if you
had
370
00:24:22 --> 00:24:24
a minimum or a maximum.
371
00:24:24 --> 00:24:28
To know that you had a minimum,
you had to look at the second
372
00:24:28 --> 00:24:29.67
derivative.
373
00:24:29.67 --> 00:24:34
The second derivative had to be
positive, the slope had to be
374
00:24:34 --> 00:24:37
increasing as you went through
the minimum point.
375
00:24:37 --> 00:24:41
The curvature had to go
upwards, and that's what we're
376
00:24:41 --> 00:24:45.1
doing now in
two dimensions,
377
00:24:45.1 --> 00:24:47
and in n dimensions.
378
00:24:47 --> 00:24:50
So we're doing what we did in
Calculus.
379
00:24:50 --> 00:24:56
Second derivative positive,
m- will now become that the
380
00:24:56 --> 00:25:00
matrix of second derivatives is
positive definite.
381
00:25:00 --> 00:25:05
Can I just -- like a
translation of -- this is how
382
00:25:05 --> 00:25:12.52
minimum are coming in,
ithe beginning of Calculus --
383
00:25:12.52 --> 00:25:19
we had a minimum was associated
with second derivative,
384
00:25:19 --> 00:25:21
being positive.
385
00:25:21 --> 00:25:26
And first derivative zero,
of course.
386
00:25:26 --> 00:25:32
Derivative, first derivative,
but it was the second
387
00:25:32 --> 00:25:38
derivative that told us we had a
minimum.
388
00:25:38 --> 00:25:42
And now, in 18.06,
in linear algebra,
389
00:25:42 --> 00:25:47
we'll have a minim for our
function now,
390
00:25:47 --> 00:25:53
our function will have,
for your function be a function
391
00:25:53 --> 00:25:58
not of just x but several
variables, the way functions
392
00:25:58 --> 00:26:04
really are in real life,
the minimum will be when the
393
00:26:04 --> 00:26:12
matrix of second derivatives,
the matrix here was one by one,
394
00:26:12 --> 00:26:18.83
there was just one second
derivative, now we've got lots.
395
00:26:18.83 --> 00:26:21
Is positive definite.
396
00:26:21 --> 00:26:26.49
So positive for a number
translates into positive
397
00:26:26.49 --> 00:26:28
definite for a matrix.
398
00:26:28 --> 00:26:34.04
And it this brings everything
you check pivots,
399
00:26:34.04 --> 00:26:39.48
you check determinants,
you check all your values,
400
00:26:39.48 --> 00:26:44
or you check this minimum
stuff.
401
00:26:44 --> 00:26:44
OK.
402
00:26:44 --> 00:26:47
Let me come back to this graph.
403
00:26:47 --> 00:26:49
That graph goes upwards.
404
00:26:49 --> 00:26:51
And I'll have to see why.
405
00:26:51 --> 00:26:56.81
How do I know that this,
that this function is always
406
00:26:56.81 --> 00:26:57
positive?
407
00:26:57 --> 00:27:02
Can you look at that and tell
that it's always positive?
408
00:27:02 --> 00:27:06
Maybe two by two,
you could feel pretty sure,
409
00:27:06 --> 00:27:13.02
but what's the good way to show
that this thing is always
410
00:27:13.02 --> 00:27:13
positive?
411
00:27:13 --> 00:27:17
If we can express it,
as, in terms of squares,
412
00:27:17 --> 00:27:22
because that's what we know for
any x and y, whatever,
413
00:27:22 --> 00:27:27
if we're squaring something we
certainly are not negative.
414
00:27:27 --> 00:27:31
So I believe that this
expression, this function,
415
00:27:31 --> 00:27:35.53
could be written as a sum of
squares.
416
00:27:35.53 --> 00:27:39
Can you tell me -- see,
because all the problems,
417
00:27:39 --> 00:27:43
the headaches are coming from
this xy term.
418
00:27:43 --> 00:27:48
If we can get expressions -- if
we can get that inside a square,
419
00:27:48 --> 00:27:52
so actually,
what we're doing is something
420
00:27:52 --> 00:27:57
called, that you've seen called
completing the square.
421
00:27:57 --> 00:28:01
Let me start the
square and you complete it.
422
00:28:01 --> 00:28:07
OK, I think we have two of x
plus, now I don't remember how
423
00:28:07 --> 00:28:11
many y-s we need,
but you'll figure it out,
424
00:28:11 --> 00:28:12.41
squared.
425
00:28:12.41 --> 00:28:18
How many y-s should I put in
here, to make -- what do I want
426
00:28:18 --> 00:28:22
to do, the two x squared-s will
be correct, right?
427
00:28:22 --> 00:28:29.5
What I want to do is put in the
right number of y-s to get
428
00:28:29.5 --> 00:28:31.39
the twelve xy correct.
429
00:28:31.39 --> 00:28:34
And what is that number of y-s?
430
00:28:34 --> 00:28:39
Let's see, I've got two times,
and so I really want six xy-s
431
00:28:39 --> 00:28:43
to come out of here,
I think maybe if I put three
432
00:28:43 --> 00:28:46
there, does that look right to
you?
433
00:28:46 --> 00:28:49
I have two- this is,
we can mentally,
434
00:28:49 --> 00:28:52
multiply out,
that's X squared,
435
00:28:52 --> 00:28:55
that's right,
that's six X Y,
436
00:28:55 --> 00:29:00
times the two gives from,
right, and how many Y squareds
437
00:29:00 --> 00:29:01
have I now got?
438
00:29:01 --> 00:29:05
How many Y squareds have I now
got from this term?
439
00:29:05 --> 00:29:06
Eighteen.
440
00:29:06 --> 00:29:09
Eighteen was the key number,
remember?
441
00:29:09 --> 00:29:13
Now if I want to make it
twenty,
442
00:29:13 --> 00:29:16
then I've got two left.
443
00:29:16 --> 00:29:17
Two y squared-s.
444
00:29:17 --> 00:29:24
That's completing the square,
and it's, now I can see that
445
00:29:24 --> 00:29:30
that function is positive,
because it's all squares.
446
00:29:30 --> 00:29:34.07
I've got two squares,
added together,
447
00:29:34.07 --> 00:29:36.64
I couldn't go negative.
448
00:29:36.64 --> 00:29:41
What if I went back to that
seven?
449
00:29:41 --> 00:29:44
If instead of twenty that
number was a seven,
450
00:29:44 --> 00:29:46
then what would happen?
451
00:29:46 --> 00:29:50
This would still be correct,
I'd still have this square,
452
00:29:50 --> 00:29:53
to get the two x squared and
the twelve xy,
453
00:29:53 --> 00:29:57
and I'd have eighteen y squared
and then what would I do here?
454
00:29:57 --> 00:30:00
I'd have to remove eleven y
squared-s, right,
455
00:30:00 --> 00:30:05
if I only had a seven here,
then instead of -- when I had a
456
00:30:05 --> 00:30:09
twenty
I had two more to put in,
457
00:30:09 --> 00:30:15
when I had an eighteen,
which was the marginal case,
458
00:30:15 --> 00:30:18
I had no more to put in.
459
00:30:18 --> 00:30:23
When I had a seven,
which was the case below zero,
460
00:30:23 --> 00:30:28.51
the indefinite case,
I had minus eleven.
461
00:30:28.51 --> 00:30:28
OK.
462
00:30:28 --> 00:30:35
Now, so, you can see now,
that this thing is a bowl.
463
00:30:35 --> 00:30:39
It's going upwards,
if I cut it at a plane,
464
00:30:39 --> 00:30:42.25
z equal to one,
say, I would get,
465
00:30:42.25 --> 00:30:46
I would get a curve,
what would be the equation for
466
00:30:46 --> 00:30:47
that curve?
467
00:30:47 --> 00:30:52
If I cut it at height one,
the equation would be this
468
00:30:52 --> 00:30:55.57
thing equal to one.
469
00:30:55.57 --> 00:30:58
And that curve would be an
ellipse.
470
00:30:58 --> 00:31:02
So actually,
already, I've blocked into the
471
00:31:02 --> 00:31:07
lecture, the different pieces
that we're aiming for.
472
00:31:07 --> 00:31:11
We're aiming for the tests,
which this passed;
473
00:31:11 --> 00:31:15
we're aiming for the connection
to a minimum,
474
00:31:15 --> 00:31:19.64
which this -- which we see in
the graph,
475
00:31:19.64 --> 00:31:24
and if we chop it up,
if we set this thing equal to
476
00:31:24 --> 00:31:29
one, if I set that thing equal
to one, that -- what that gives
477
00:31:29 --> 00:31:31
me is, the cross-section.
478
00:31:31 --> 00:31:36
It gives me this,
this curve, and its equation is
479
00:31:36 --> 00:31:39
this thing equals one,
and that's an ellipse.
480
00:31:39 --> 00:31:43
Whereas if I cut through a
saddle
481
00:31:43 --> 00:31:46.09
point, I get a hyperbola.
482
00:31:46.09 --> 00:31:46
OK.
483
00:31:46 --> 00:31:51
But this minimum stuff is
really what I'm most interested
484
00:31:51 --> 00:31:51
in.
485
00:31:51 --> 00:31:52
OK.
486
00:31:52 --> 00:31:55
By -- I just have to ask,
do you recognize,
487
00:31:55 --> 00:32:01
I mean, these numbers here,
the two that appeared outside,
488
00:32:01 --> 00:32:07
the three that appeared inside,
the two that appeared there --
489
00:32:07 --> 00:32:12
actually, those numbers come
from elimination.
490
00:32:12 --> 00:32:18
Completing the square is our
good old method of Gaussian
491
00:32:18 --> 00:32:24
elimination, in expressed in
terms of these squares.
492
00:32:24 --> 00:32:27
The -- let me show you what I
mean.
493
00:32:27 --> 00:32:33
I just think those numbers are
no accident,
494
00:32:33 --> 00:32:38
If I take my matrix two,
six, six, and twenty,
495
00:32:38 --> 00:32:43
and I do elimination,
then the pivot is two and I
496
00:32:43 --> 00:32:46
take three, what's the
multiplier?
497
00:32:46 --> 00:32:51.41
How much of row one do I take
away from row two?
498
00:32:51.41 --> 00:32:52
Three.
499
00:32:52 --> 00:32:58
So what you're seeing in this,
completing the square,
500
00:32:58 --> 00:33:02
is the pivots outside and the
multiplier inside.
501
00:33:02 --> 00:33:04
Just do that again?
502
00:33:04 --> 00:33:07
The pivot is two,
three -- three of those away
503
00:33:07 --> 00:33:11
from that gives me two,
six, zero, and what's the
504
00:33:11 --> 00:33:12
second pivot?
505
00:33:12 --> 00:33:17
Three of this away from this,
three sixes'll be eighteen,
506
00:33:17 --> 00:33:21
and the second pivot will also
be a
507
00:33:21 --> 00:33:21
two.
508
00:33:21 --> 00:33:26
So that's the U,
this is the A,
509
00:33:26 --> 00:33:34.85
and of course the L was one,
zero, one, and the multiplier
510
00:33:34.85 --> 00:33:36.34
was three.
511
00:33:36.34 --> 00:33:42
So, completing the square is
elimination.
512
00:33:42 --> 00:33:48
Why I happy to see,
happy to see that coming
513
00:33:48 --> 00:33:50
together?
514
00:33:50 --> 00:33:56
Because I know about
elimination
515
00:33:56 --> 00:33:58
for m by m matrices.
516
00:33:58 --> 00:34:03
I just started talking about
completing the square,
517
00:34:03 --> 00:34:05
here, for two by twos.
518
00:34:05 --> 00:34:08
But now I see what's going on.
519
00:34:08 --> 00:34:13
Completing the square really
amounts to splitting this thing
520
00:34:13 --> 00:34:18
into a sum of squares,
so what's the critical thing --
521
00:34:18 --> 00:34:24
I have a lot of squares,
and inside those squares are
522
00:34:24 --> 00:34:28
multipliers but they're squares,
and the question is,
523
00:34:28 --> 00:34:31.37
what's outside these squares?
524
00:34:31.37 --> 00:34:36
When I complete the square,
what are the numbers that go
525
00:34:36 --> 00:34:36
outside?
526
00:34:36 --> 00:34:38
They're the pivots.
527
00:34:38 --> 00:34:43
They're the pivots,
and that's why positive pivots
528
00:34:43 --> 00:34:46
give me sum of squares.
529
00:34:46 --> 00:34:49.55
Positive pivots,
those pivots are the numbers
530
00:34:49.55 --> 00:34:53
that go outside the squares,
so positive pivots,
531
00:34:53 --> 00:34:56
sum of squares,
everything positive,
532
00:34:56 --> 00:34:59
graph goes up,
a minimum at the origin,
533
00:34:59 --> 00:35:03
it's all connected together;
all connected together.
534
00:35:03 --> 00:35:07
And in the two by two case,
you can see those connections,
535
00:35:07 --> 00:35:11
but linear algebra now can go
up to
536
00:35:11 --> 00:35:13
three by three,
m by m.
537
00:35:13 --> 00:35:15
Let's do that next.
538
00:35:15 --> 00:35:20.52
Can I just, before I leave two
by two, I've written this
539
00:35:20.52 --> 00:35:26
expression "matrix of second
derivatives." What's the matrix
540
00:35:26 --> 00:35:28
of second derivatives?
541
00:35:28 --> 00:35:32
That's one second derivative
now, but if I'm in two
542
00:35:32 --> 00:35:37
dimensions,
I have a two by two matrix,
543
00:35:37 --> 00:35:44.04
it's the second x derivative,
the second x derivative goes
544
00:35:44.04 --> 00:35:48.88
there -- shall I write it --
fxx, if you like,
545
00:35:48.88 --> 00:35:54
fxx, that means the second
derivative of f in the x
546
00:35:54 --> 00:35:59
direction.
fyy, second derivative in the y
547
00:35:59 --> 00:36:00
direction.
548
00:36:00 --> 00:36:05
Those are the pure derivatives,
second derivatives.
549
00:36:05 --> 00:36:07
They have to be positive.
550
00:36:07 --> 00:36:09
For a minimum.
551
00:36:09 --> 00:36:13
This number has to be positive
for a minimum.
552
00:36:13 --> 00:36:17.75
That number has to be positive
for a minimum.
553
00:36:17.75 --> 00:36:19
But, that's not enough.
554
00:36:19 --> 00:36:26
Those numbers have to somehow
be big enough to overcome this
555
00:36:26 --> 00:36:30
cross-derivative,
Why is the matrix symmetric?
556
00:36:30 --> 00:36:35
Because the second derivative f
with respect to x and y is equal
557
00:36:35 --> 00:36:40
to -- I can, that's the
beautiful fact about second
558
00:36:40 --> 00:36:45
derivatives, is I can do those
in either order and I get the
559
00:36:45 --> 00:36:46
same thing.
560
00:36:46 --> 00:36:52
So this is the same as that,
and so, that's the matrix of
561
00:36:52 --> 00:36:54
second derivatives.
562
00:36:54 --> 00:36:58
And the test is,
it has to be positive definite.
563
00:36:58 --> 00:37:02.72
You might remember,
from, tucked in somewhere near
564
00:37:02.72 --> 00:37:07
the end of eighteen o' two or at
least in the book,
565
00:37:07 --> 00:37:12
was the condition for a
minimum, For a function of two
566
00:37:12 --> 00:37:13.9
variables.
567
00:37:13.9 --> 00:37:16
Let's -- when do you have a
minimum?
568
00:37:16 --> 00:37:20
For a function of two
variables, believe me,
569
00:37:20 --> 00:37:22
that's what Calculus is for.
570
00:37:22 --> 00:37:26
The condition is first
derivatives have to be zero.
571
00:37:26 --> 00:37:30
And the matrix of second
derivatives has to be positive
572
00:37:30 --> 00:37:31
definite.
573
00:37:31 --> 00:37:36
So you maybe remember there was
an fxx times an fyy
574
00:37:36 --> 00:37:40
that had to be bigger than an
an fxy squared,
575
00:37:40 --> 00:37:44
that's just our determinant,
two by two.
576
00:37:44 --> 00:37:49
But now, we now know the answer
for three by three,
577
00:37:49 --> 00:37:55
m by m, because we can do
elimination by m by m matrices,
578
00:37:55 --> 00:38:00
we can connect eigenvalues of m
by m matrices,
579
00:38:00 --> 00:38:08
we can do sum of squares,
sum of m squares instead of
580
00:38:08 --> 00:38:15
only two squares;
and so let's take a,
581
00:38:15 --> 00:38:24
let me go over here to do a
three by three example.
582
00:38:24 --> 00:38:29
So, three by three example.
583
00:38:29 --> 00:38:29
OK.
584
00:38:29 --> 00:38:37
Oh, let me -- shall I use my
favorite matrix?
585
00:38:37 --> 00:38:45
You've seen this matrix before.
586
00:38:45 --> 00:38:50
Yes, let's use the good matrix,
four by one,
587
00:38:50 --> 00:38:51
oops, open.
588
00:38:51 --> 00:38:54
Is that matrix positive
definite?
589
00:38:54 --> 00:39:01
What's -- so I'm going to ask
questions about this matrix,
590
00:39:01 --> 00:39:05
is it positive definite,
first of all?
591
00:39:05 --> 00:39:10
What's the function associated
with that matrix,
592
00:39:10 --> 00:39:14
what's the x transpose Ax?
593
00:39:14 --> 00:39:18
Is -- do we have a minimum for
that function,
594
00:39:18 --> 00:39:18
at zero?
595
00:39:18 --> 00:39:21
And then even what's the
geometry?
596
00:39:21 --> 00:39:21
OK.
597
00:39:21 --> 00:39:24
First of all,
is the matrix positive
598
00:39:24 --> 00:39:29
definite, now I've given you the
numbers there so you can take
599
00:39:29 --> 00:39:32
the determinants,
maybe that's the quickest,
600
00:39:32 --> 00:39:36
is that what you would do
mentally,
601
00:39:36 --> 00:39:40
if I give you all a matrix on a
quiz and say is it positive
602
00:39:40 --> 00:39:42.02
definite or not?
603
00:39:42.02 --> 00:39:46
I would take that determinant
and I'd give the answer two.
604
00:39:46 --> 00:39:50
I would take that determinant
and I would give the answer for
605
00:39:50 --> 00:39:55
that two by two determinant,
I'd give the answer three,
606
00:39:55 --> 00:39:58
and anybody remember the answer
for
607
00:39:58 --> 00:40:01
the three by three determinant?
608
00:40:01 --> 00:40:06
It was four,
you remember for these special
609
00:40:06 --> 00:40:10.04
matrices, when we do
determinants,
610
00:40:10.04 --> 00:40:13
they went up two,
three, four,
611
00:40:13 --> 00:40:17
five, six, they just went up
linearly.
612
00:40:17 --> 00:40:22
So that matrix has -- the
determinants are two,
613
00:40:22 --> 00:40:24
three, and four.
614
00:40:24 --> 00:40:26
Pivots.
615
00:40:26 --> 00:40:28
What are the pivots for that
matrix?
616
00:40:28 --> 00:40:32
I'll tell you,
they're -- the first pivot is
617
00:40:32 --> 00:40:37
two, the next pivot is three
over two, the next pivot is four
618
00:40:37 --> 00:40:38
over three.
619
00:40:38 --> 00:40:42
Because, the product of the
pivots has to give me those
620
00:40:42 --> 00:40:43
determinants.
621
00:40:43 --> 00:40:47
The product of these two pivots
gives me that determinant;
622
00:40:47 --> 00:40:53
the
product of all the pivots gives
623
00:40:53 --> 00:40:55
me that determinant.
624
00:40:55 --> 00:40:59
What are the eigenvalues?
625
00:40:59 --> 00:41:01
Oh, I don't know.
626
00:41:01 --> 00:41:09
The eigenvalues I've got,
what do I have a cubic equation
627
00:41:09 --> 00:41:12
-- a degree three equation?
628
00:41:12 --> 00:41:17
There are three eigenvalues to
find.
629
00:41:17 --> 00:41:24
If I believe what I've said
today, what do I know about
630
00:41:24 --> 00:41:27.91
these
eigenvalues,
631
00:41:27.91 --> 00:41:31
even though I don't know the
exact numbers.
632
00:41:31 --> 00:41:34
I -- I remember the numbers.
633
00:41:34 --> 00:41:39
Because these matrices are so
important that people figure
634
00:41:39 --> 00:41:40
them.
635
00:41:40 --> 00:41:45
But -- what do you believe to
be true about these three
636
00:41:45 --> 00:41:51
eigenvalues -- you believe that
they are all positive.
637
00:41:51 --> 00:41:52
They're all positive.
638
00:41:52 --> 00:41:56
I think that they are two minus
square root of two,
639
00:41:56 --> 00:41:59
two, and two plus the square
root of two.
640
00:41:59 --> 00:42:00
I think.
641
00:42:00 --> 00:42:04
Let me just -- I can't write
those numbers down without
642
00:42:04 --> 00:42:09
checking the simple checks,
what the first simple check is
643
00:42:09 --> 00:42:13
the trace, so if I add those
numbers I
644
00:42:13 --> 00:42:16
get six and if I add those
numbers I get six.
645
00:42:16 --> 00:42:21
The other simple test is the
determinant, if I -- can you do
646
00:42:21 --> 00:42:24
this, can you multiply those
numbers together?
647
00:42:24 --> 00:42:27
I guess we can multiply by two
out there.
648
00:42:27 --> 00:42:32
What's two minus square root of
two times two plus square root
649
00:42:32 --> 00:42:35
of
two, that'll be four minus two,
650
00:42:35 --> 00:42:39.07
that'll be two,
yeah, two times two,
651
00:42:39.07 --> 00:42:43
that's got the determinant,
right, so it's got,
652
00:42:43 --> 00:42:47
it's got a chance of being
correct and I think it is.
653
00:42:47 --> 00:42:50
Now, what's the x transpose Ax?
654
00:42:50 --> 00:42:54.07
I better give myself enough
room
655
00:42:54.07 --> 00:42:57
for that.
x transpose Ax for this guy.
656
00:42:57 --> 00:43:01
It's two x1 squareds,
and two x2 squareds,
657
00:43:01 --> 00:43:03
and two x3 squareds.
658
00:43:03 --> 00:43:08
Those come from the diagonal,
those were easy.
659
00:43:08 --> 00:43:12
Now off the diagonal there's a
minus and a minus,
660
00:43:12 --> 00:43:17
they come together there'll be
a minus two
661
00:43:17 --> 00:43:19
minus two whats?
662
00:43:19 --> 00:43:23
Are coming from this one two
and two one position,
663
00:43:23 --> 00:43:24
is the x1 x2.
664
00:43:24 --> 00:43:29
I'm doing mentally a
multiplication of this matrix
665
00:43:29 --> 00:43:34
times a row vector on the left
times a column vector on the
666
00:43:34 --> 00:43:40.9
right, and I know that these
numbers show up in the answer.
667
00:43:40.9 --> 00:43:46
The diagonal is the perfect
square, this off diagonal is a
668
00:43:46 --> 00:43:50
minus two x1 x2,
and there are no x1 x3-s,
669
00:43:50 --> 00:43:54
and there're minus two x2 x3-s.
670
00:43:54 --> 00:43:59.48
And I believe that that
expression is always positive.
671
00:43:59.48 --> 00:44:04
I believe that that curve,
that graph, really,
672
00:44:04 --> 00:44:09
of that function,
this is my function f,
673
00:44:09 --> 00:44:14
and I'm in more dimensions now
than I can draw,
674
00:44:14 --> 00:44:20
it -- but the graph of that
function goes upwards.
675
00:44:20 --> 00:44:22
It's a bowl.
676
00:44:22 --> 00:44:29
Or maybe the right word is --
just forgot, what's a long word
677
00:44:29 --> 00:44:30.87
for bowl?
678
00:44:30.87 --> 00:44:38
Hm, maybe paraboloid,
I think, paraboloid comes in.
679
00:44:38 --> 00:44:42
I'll edit the tape and get that
word in.
680
00:44:42 --> 00:44:45
Bowl, let's say,
is, that, so that,
681
00:44:45 --> 00:44:49
and if I can -- I could
complete the squares,
682
00:44:49 --> 00:44:54
I could write that as the sum
of three squares,
683
00:44:54 --> 00:44:59
and those three squares would
get multiplied by the three
684
00:44:59 --> 00:45:00
pivots.
685
00:45:00 --> 00:45:04
And the pivots are all
positive.
686
00:45:04 --> 00:45:07
So I would have positive pivots
times squares,
687
00:45:07 --> 00:45:12
the net result would be a
positive function and a bowl
688
00:45:12 --> 00:45:13
which goes upwards.
689
00:45:13 --> 00:45:17.29
And then, finally,
if I cut -- if I slice through
690
00:45:17.29 --> 00:45:21
this bowl, if I -- now I'm
asking you to stretch your
691
00:45:21 --> 00:45:25
visualization here,
because I'm in four dimensions,
692
00:45:25 --> 00:45:29
I've
got x1 x2 x3 in the base,
693
00:45:29 --> 00:45:33.26
and this function is z,
or f, or something.
694
00:45:33.26 --> 00:45:35
And its graph is going up.
695
00:45:35 --> 00:45:41
But I'm in four dimensions,
because I've got three in the
696
00:45:41 --> 00:45:46
base and then the upward
direction, but now if I cut
697
00:45:46 --> 00:45:51
through this four-dimensional
picture, at level one,
698
00:45:51 --> 00:45:57
so, suppose I cut
through this thing at height
699
00:45:57 --> 00:45:58
one.
700
00:45:58 --> 00:46:03
So I take all the points that
are at height one.
701
00:46:03 --> 00:46:08
That gives me -- it gave me an
ellipse over there,
702
00:46:08 --> 00:46:13
in that two by two case,
in this case,
703
00:46:13 --> 00:46:19
this will be the equation of an
ellipsoid, a football in other
704
00:46:19 --> 00:46:20
words.
705
00:46:20 --> 00:46:24
Well, not quite a football.
706
00:46:24 --> 00:46:26
A lopsided football.
707
00:46:26 --> 00:46:30
What would be,
can I try to describe to you
708
00:46:30 --> 00:46:35.18
what the ellipsoid will look
like, this ellipsoid,
709
00:46:35.18 --> 00:46:39
I'm sorry that the,
that I've ended the matrix
710
00:46:39 --> 00:46:44
right -- at the point,
let's -- let me be sure you've
711
00:46:44 --> 00:46:45
seen the equation.
712
00:46:45 --> 00:46:48.62
Two x1 squared,
two x2 squared,
713
00:46:48.62 --> 00:46:54
two x3
squared, minus two of the cross
714
00:46:54 --> 00:46:57
parts, equal what?
715
00:46:57 --> 00:47:04.99
That is the equation of a
football, so what do I mean by a
716
00:47:04.99 --> 00:47:08
football or an ellipsoid?
717
00:47:08 --> 00:47:13
I mean that,
well, I'll draw a few.
718
00:47:13 --> 00:47:17
It's like that,
it's got a center,
719
00:47:17 --> 00:47:24
and it's got it's got three
principal directions.
720
00:47:24 --> 00:47:26
This ellipsoid.
721
00:47:26 --> 00:47:32
So -- you see what I'm saying,
if we have a sphere then all
722
00:47:32 --> 00:47:35
directions would be the same.
723
00:47:35 --> 00:47:41
If we had a true football,
or it's closer to a rugby ball,
724
00:47:41 --> 00:47:46
really, because it's more
curved than a football,
725
00:47:46 --> 00:47:52
it would have one long
direction and the other two
726
00:47:52 --> 00:47:53
would be equal.
727
00:47:53 --> 00:47:58
That would be like having a
matrix that had one eigenvalue
728
00:47:58 --> 00:47:59
repeated.
729
00:47:59 --> 00:48:01
And then one other different.
730
00:48:01 --> 00:48:06
So this sphere comes from,
like, the identity matrix,
731
00:48:06 --> 00:48:08
all eigenvalues the same.
732
00:48:08 --> 00:48:12.99
Our rugby ball comes from a
case where --
733
00:48:12.99 --> 00:48:17.05
three, the three,
two of the three eigenvalues
734
00:48:17.05 --> 00:48:18
are the same.
735
00:48:18 --> 00:48:22
But we know how the case where
-- the typical case,
736
00:48:22 --> 00:48:26
where the three eigenvalues
were all different.
737
00:48:26 --> 00:48:31
So this will have -- How do I
say it, if I look at this
738
00:48:31 --> 00:48:36
ellipsoid correctly,
it'll have a major axis,
739
00:48:36 --> 00:48:41
it'll have a middle axis,
and it'll have a minor axis.
740
00:48:41 --> 00:48:45
And those three axes will be in
the direction of the
741
00:48:45 --> 00:48:47.11
eigenvectors.
742
00:48:47.11 --> 00:48:51
And the lengths of those axes
will be determined by the
743
00:48:51 --> 00:48:52
eigenvalues.
744
00:48:52 --> 00:48:57
I can get -- turn this all into
linear algebra,
745
00:48:57 --> 00:49:02.3
because we have -- the right
thing we know about
746
00:49:02.3 --> 00:49:07
eigenvectors and eigenvalues,
for that matrix is what?
747
00:49:07 --> 00:49:12
Of -- let me just tell you
that, repeat the main linear
748
00:49:12 --> 00:49:13
algebra point.
749
00:49:13 --> 00:49:17.5
How could we turn what I said
into algebra;
750
00:49:17.5 --> 00:49:22
we would write this A as Q,
the eigenvector matrix,
751
00:49:22 --> 00:49:26.36
times lambda,
the eigenvalue matrix
752
00:49:26.36 --> 00:49:27
times Q transposed.
753
00:49:27 --> 00:49:31
The principal axis theorem,
we'll call it,
754
00:49:31 --> 00:49:31
now.
755
00:49:31 --> 00:49:36
The eigenvectors tell us the
directions of the principal
756
00:49:36 --> 00:49:36
axes.
757
00:49:36 --> 00:49:40.88
The eigenvalues tell us the
lengths of those axes,
758
00:49:40.88 --> 00:49:44
actually the lengths,
or the half-lengths,
759
00:49:44 --> 00:49:48
or one over the eigenvalues,
it turns out.
760
00:49:48 --> 00:49:53.62
And that is the matrix
factorization which is the most
761
00:49:53.62 --> 00:49:58
important matrix factorization
in our eigenvalue material so
762
00:49:58 --> 00:49:59
far.
763
00:49:59 --> 00:50:03
That's diagonalization for a
symmetric matrix,
764
00:50:03 --> 00:50:08
so instead of the inverse I can
write the transposed.
765
00:50:08 --> 00:50:09
OK.
766
00:50:09 --> 00:50:13
I've -- so what I've tried
today is to tell you the --
767
00:50:13 --> 00:50:18
what's going on with positive
definite matrices.
768
00:50:18 --> 00:50:23
Ah, you see all how all these
pieces are there and linear
769
00:50:23 --> 00:50:25
algebra connects them.
770
00:50:25 --> 00:50:25
OK.
771
00:50:25 --> 00:50:28
See you on Friday.