1
00:00:05,560 --> 00:00:14,460
OK, this is the lecture on
positive definite matrices.
2
00:00:14,460 --> 00:00:20,770
I made a start on those
briefly in a previous lecture.
3
00:00:20,770 --> 00:00:26,180
One point I wanted to make was
the way that this topic brings
4
00:00:26,180 --> 00:00:30,300
the whole course together,
pivots, determinants,
5
00:00:30,300 --> 00:00:36,190
eigenvalues, and something
new- four plot instability
6
00:00:36,190 --> 00:00:42,050
and then something new in this
expression, x transpose Ax,
7
00:00:42,050 --> 00:00:46,040
actually that's the guy
to watch in this lecture.
8
00:00:46,040 --> 00:00:50,410
So, so the topic is
positive definite matrix,
9
00:00:50,410 --> 00:00:53,580
and what's my goal?
10
00:00:53,580 --> 00:00:58,440
First, first goal is, how
can I tell if a matrix is
11
00:00:58,440 --> 00:00:59,970
positive definite?
12
00:00:59,970 --> 00:01:02,490
So I would like to
have tests to see
13
00:01:02,490 --> 00:01:06,100
if you give me a, a
five by five matrix,
14
00:01:06,100 --> 00:01:09,000
how do I tell if it's
positive definite?
15
00:01:09,000 --> 00:01:12,020
More important is,
what does it mean?
16
00:01:12,020 --> 00:01:14,700
Why are we so interested
in this property
17
00:01:14,700 --> 00:01:16,440
of positive definiteness?
18
00:01:16,440 --> 00:01:21,410
And then, at the end
comes some geometry.
19
00:01:21,410 --> 00:01:25,080
Ellipses are connected with
positive definite things.
20
00:01:25,080 --> 00:01:28,520
Hyperbolas are not connected
with positive definite things,
21
00:01:28,520 --> 00:01:33,350
so we- it's this, we,
there's a geometry too,
22
00:01:33,350 --> 00:01:38,410
but mostly it's
linear algebra and --
23
00:01:38,410 --> 00:01:43,020
this application of how do you
recognize 'em when you have
24
00:01:43,020 --> 00:01:45,181
a minim is pretty neat.
25
00:01:45,181 --> 00:01:45,680
OK.
26
00:01:45,680 --> 00:01:50,090
I'm gonna begin with two by two.
27
00:01:50,090 --> 00:01:52,280
All matrices are
symmetric, right?
28
00:01:52,280 --> 00:01:55,180
That's understood; the
matrix is symmetric,
29
00:01:55,180 --> 00:02:00,650
now my question is, is
it positive definite?
30
00:02:00,650 --> 00:02:04,030
Now, here are some --
31
00:02:04,030 --> 00:02:09,570
each one of these is a complete
test for positive definiteness.
32
00:02:09,570 --> 00:02:15,010
If I know the eigenvalues,
my test is are they positive?
33
00:02:15,010 --> 00:02:18,140
Are they all positive?
34
00:02:18,140 --> 00:02:21,650
If I know these --
35
00:02:21,650 --> 00:02:24,170
so, A is really --
36
00:02:24,170 --> 00:02:27,570
I look at that number A,
here, as the, as the one
37
00:02:27,570 --> 00:02:33,420
by one determinant, and here's
the two by two determinant.
38
00:02:33,420 --> 00:02:35,510
So this is the determinant test.
39
00:02:35,510 --> 00:02:39,280
This is the eigenvalue test,
this is the determinant test.
40
00:02:39,280 --> 00:02:46,040
Are the determinants growing in
s- of all, of all end, sort of,
41
00:02:46,040 --> 00:02:48,640
can I call them
leading submatrices,
42
00:02:48,640 --> 00:02:52,140
they're the first
ones the northwest,
43
00:02:52,140 --> 00:02:56,510
Seattle submatrices coming
down from from there,
44
00:02:56,510 --> 00:02:59,570
they all, all those determinants
have to be positive,
45
00:02:59,570 --> 00:03:03,020
and then another
test is the pivots.
46
00:03:03,020 --> 00:03:06,370
The pivots of a
two by two matrix
47
00:03:06,370 --> 00:03:13,070
are the number A for sure,
and, since the product
48
00:03:13,070 --> 00:03:15,410
is the determinant,
the second pivot
49
00:03:15,410 --> 00:03:18,980
must be the determinant
divided by A.
50
00:03:18,980 --> 00:03:24,970
And then in here is gonna come
my favorite and my new idea,
51
00:03:24,970 --> 00:03:30,450
the, the, the the one to
catch, about x transpose Ax
52
00:03:30,450 --> 00:03:32,640
being positive.
53
00:03:32,640 --> 00:03:35,280
But we'll have to
look at this guy.
54
00:03:35,280 --> 00:03:42,780
This gets, like a star, because
for most, presentations,
55
00:03:42,780 --> 00:03:45,610
the definition of
positive definiteness
56
00:03:45,610 --> 00:03:49,370
would be this number four and
these numbers one two three
57
00:03:49,370 --> 00:03:51,260
would be test four.
58
00:03:51,260 --> 00:03:51,760
OK.
59
00:03:51,760 --> 00:03:56,160
Maybe I'll tuck this,
where, you know,
60
00:03:56,160 --> 00:03:56,660
OK.
61
00:03:56,660 --> 00:04:00,000
So I'll have to look
at this x transpose Ax.
62
00:04:00,000 --> 00:04:07,760
Can you, can we
just be sure, how
63
00:04:07,760 --> 00:04:13,130
do we know that the eigenvalue
test and the determinant test,
64
00:04:13,130 --> 00:04:17,170
pick out the same
matrices, and let me,
65
00:04:17,170 --> 00:04:19,797
let's just do a few examples.
66
00:04:19,797 --> 00:04:20,380
Some examples.
67
00:04:24,610 --> 00:04:31,220
Let me pick the matrix
two, six, six, tell me,
68
00:04:31,220 --> 00:04:36,360
what number do I have to
put there for the matrix
69
00:04:36,360 --> 00:04:37,440
to be positive definite?
70
00:04:40,150 --> 00:04:42,090
Tell me a sufficiently
large number
71
00:04:42,090 --> 00:04:45,020
that would make it
positive definite?
72
00:04:45,020 --> 00:04:47,830
Let's just practice with
these conditions in the two
73
00:04:47,830 --> 00:04:49,370
by two case.
74
00:04:49,370 --> 00:04:51,490
Now, when I ask
you that, you don't
75
00:04:51,490 --> 00:04:56,270
wanna find the eigenvalues, you
would use the determinant test
76
00:04:56,270 --> 00:05:00,340
for that, so, the first
or the pivot test,
77
00:05:00,340 --> 00:05:03,750
that, that guy is certainly
positive, that had to happen,
78
00:05:03,750 --> 00:05:05,070
and it's OK.
79
00:05:05,070 --> 00:05:09,030
How large a number here -- the
number had better be more than
80
00:05:09,030 --> 00:05:10,200
what?
81
00:05:10,200 --> 00:05:14,350
More than eighteen, right,
because if it's eight --
82
00:05:14,350 --> 00:05:15,400
no.
83
00:05:15,400 --> 00:05:18,550
More than what?
84
00:05:18,550 --> 00:05:21,390
Nineteen, is it?
85
00:05:21,390 --> 00:05:28,965
If I have a nineteen here,
is that positive definite?
86
00:05:32,590 --> 00:05:36,160
I get thirty eight minus
thirty six, that's OK.
87
00:05:36,160 --> 00:05:39,900
If I had an eighteen, let
me play it really close.
88
00:05:39,900 --> 00:05:44,850
If I have an eighteen there,
then I positive definite?
89
00:05:44,850 --> 00:05:46,640
Not quite.
90
00:05:46,640 --> 00:05:49,830
I would call this
guy positive, so it's
91
00:05:49,830 --> 00:05:53,550
useful just to see that
this the borderline.
92
00:05:53,550 --> 00:05:55,640
That matrix is on
the borderline,
93
00:05:55,640 --> 00:05:58,880
I would call that matrix
positive semi-definite.
94
00:06:04,670 --> 00:06:07,900
And what are the
eigenvalues of that matrix,
95
00:06:07,900 --> 00:06:11,690
just since we're given
eigenvalues of two by twos,
96
00:06:11,690 --> 00:06:19,460
when it's semi-definite, but
not definite, then the --
97
00:06:19,460 --> 00:06:22,540
I'm squeezing this
eigenvalue test down, --
98
00:06:22,540 --> 00:06:28,370
what's the eigenvalue that
I know this matrix has?
99
00:06:28,370 --> 00:06:31,320
What kind of a matrix
have I got here?
100
00:06:31,320 --> 00:06:37,680
It's a singular matrix, one
of its eigenvalues is zero.
101
00:06:37,680 --> 00:06:43,570
That has an eigenvalue zero,
and the other eigenvalue is --
102
00:06:43,570 --> 00:06:47,020
from the trace, twenty.
103
00:06:47,020 --> 00:06:47,620
OK.
104
00:06:47,620 --> 00:06:51,560
So that, that matrix has
eigenvalues greater than
105
00:06:51,560 --> 00:06:55,710
or equal to zero, and it's that
"equal to" that brought this
106
00:06:55,710 --> 00:06:58,000
word "semi-definite" in.
107
00:06:58,000 --> 00:07:01,240
And, the what are the
pivots of that matrix?
108
00:07:01,240 --> 00:07:03,810
So the pivots, so
the eigenvalues
109
00:07:03,810 --> 00:07:10,580
are zero and twenty, the pivots
are, well, the pivot is two,
110
00:07:10,580 --> 00:07:12,090
and what's the next pivot?
111
00:07:15,840 --> 00:07:17,430
There isn't one.
112
00:07:17,430 --> 00:07:20,900
We got a singular matrix here,
it'll only have one pivot.
113
00:07:20,900 --> 00:07:24,920
You see that that's a rank
one matrix, two six is a --
114
00:07:24,920 --> 00:07:27,710
six eighteen is a
multiple of two six,
115
00:07:27,710 --> 00:07:32,380
the matrix is singular
it only has one pivot,
116
00:07:32,380 --> 00:07:38,050
so the pivot test doesn't quite
117
00:07:38,050 --> 00:07:40,710
The -- let me do
the x transpose Ax.
118
00:07:40,710 --> 00:07:41,210
pass.
119
00:07:41,210 --> 00:07:45,280
Now this is --
120
00:07:45,280 --> 00:07:49,130
the novelty now.
121
00:07:49,130 --> 00:07:49,960
OK.
122
00:07:49,960 --> 00:07:54,470
What I looking at when I
look at this new combination,
123
00:07:54,470 --> 00:07:56,360
x transpose Ax.
124
00:07:56,360 --> 00:08:02,140
x is any vector now, so
lemme compute, so any vector,
125
00:08:02,140 --> 00:08:08,960
lemme call its components
x1 and x2, so that's x.
126
00:08:08,960 --> 00:08:11,300
And I put in here A.
127
00:08:11,300 --> 00:08:15,500
Let's, let's use this example
two six, six eighteen,
128
00:08:15,500 --> 00:08:20,500
and here's x
transposed, so x1 x2.
129
00:08:20,500 --> 00:08:25,580
We're back to real matrices,
after that last lecture
130
00:08:25,580 --> 00:08:29,480
that- that said what to do
in the complex case, let's
131
00:08:29,480 --> 00:08:32,010
come back to real matrices.
132
00:08:32,010 --> 00:08:36,780
So here's x transpose Ax,
and what I'm interested
133
00:08:36,780 --> 00:08:42,720
in is, what do I get if I
multiply those together?
134
00:08:42,720 --> 00:08:48,410
I get some function of x1
and x2, and what is it?
135
00:08:48,410 --> 00:08:51,700
Let's see, if I do this
multiplication, so I do it,
136
00:08:51,700 --> 00:08:55,220
lemme, just, I'll just
do it slowly, x1, x2,
137
00:08:55,220 --> 00:09:01,980
if I multiply that matrix,
this is 2x1, and 6x2s,
138
00:09:01,980 --> 00:09:07,400
and the next row
is 6x1s and 18x2s,
139
00:09:07,400 --> 00:09:11,530
and now I do this final
step and what do I have?
140
00:09:11,530 --> 00:09:16,810
I've got 2x1 squareds,
got 2X1 squareds
141
00:09:16,810 --> 00:09:23,080
is coming from this two, I've
got this one gives me eighteen,
142
00:09:23,080 --> 00:09:25,630
well, shall I do the
ones in the middle?
143
00:09:25,630 --> 00:09:29,530
How many x1 x2s do I have?
144
00:09:29,530 --> 00:09:33,330
Let's see, x1 times that
6x2 would be six of 'em,
145
00:09:33,330 --> 00:09:38,300
and then x2 times this one
will be six more, I get twelve.
146
00:09:38,300 --> 00:09:44,290
So, in here is going, this
is the number a, this is
147
00:09:44,290 --> 00:09:49,190
the number 2b, and in here is --
148
00:09:49,190 --> 00:09:54,960
x2 times eighteen x2 will be
eighteen x2 squareds and this
149
00:09:54,960 --> 00:09:57,630
is the number c.
150
00:09:57,630 --> 00:10:02,010
So it's ax1 -- it's
like ax squared.
151
00:10:02,010 --> 00:10:05,140
2bxy and cy squared.
152
00:10:05,140 --> 00:10:11,280
For my first point that I
wanted to make by just doing out
153
00:10:11,280 --> 00:10:15,720
a multiplication is, that is as
soon as you give me the matrix,
154
00:10:15,720 --> 00:10:19,240
as soon as you give me
the matrix, I can --
155
00:10:19,240 --> 00:10:22,860
those are the numbers
that appear in --
156
00:10:22,860 --> 00:10:25,580
I'll call this
thing a quadratic,
157
00:10:25,580 --> 00:10:29,140
you see, it's not
linear anymore.
158
00:10:29,140 --> 00:10:32,360
Ax is linear, but now I've
got an x transpose coming
159
00:10:32,360 --> 00:10:35,680
in, I'm up to degree
two, up to degree two,
160
00:10:35,680 --> 00:10:38,880
maybe quadratic is the --
161
00:10:38,880 --> 00:10:39,480
use the word.
162
00:10:39,480 --> 00:10:41,980
A quadratic form.
163
00:10:41,980 --> 00:10:46,610
It's purely degree two,
there's no linear part,
164
00:10:46,610 --> 00:10:48,440
there's no constant
part, there certainly
165
00:10:48,440 --> 00:10:53,540
no cubes or fourth powers,
it's all second degree.
166
00:10:53,540 --> 00:10:55,780
And my question is --
167
00:10:55,780 --> 00:11:00,120
is that quantity
positive or not?
168
00:11:00,120 --> 00:11:08,250
That's -- for every x1 and x2,
that is my new definition --
169
00:11:08,250 --> 00:11:11,920
that's my definition of a
positive definite matrix.
170
00:11:11,920 --> 00:11:17,970
If this quantity is positive,
if, if, if, it's positive
171
00:11:17,970 --> 00:11:25,140
for all x's and y's, all
x1 x2s, then I call them --
172
00:11:25,140 --> 00:11:29,170
then that's the matrix
is positive definite.
173
00:11:29,170 --> 00:11:34,340
Now, is this guy
passing our test?
174
00:11:34,340 --> 00:11:38,120
Well we have, we anticipated
the answer here by,
175
00:11:38,120 --> 00:11:42,420
by asking first about
eigenvalues and pivots,
176
00:11:42,420 --> 00:11:44,820
and what happened?
177
00:11:44,820 --> 00:11:46,350
It failed.
178
00:11:46,350 --> 00:11:49,490
It barely failed.
179
00:11:49,490 --> 00:11:53,670
If I had made this
eighteen down to a seven,
180
00:11:53,670 --> 00:11:58,120
it would've totally failed.
181
00:11:58,120 --> 00:12:01,540
I do that with the eraser, and
then I'll put back eighteen,
182
00:12:01,540 --> 00:12:07,170
because, seven is such a
total disaster, but if --
183
00:12:07,170 --> 00:12:10,240
I'll keep seven for a second.
184
00:12:10,240 --> 00:12:15,170
Is that thing in any
way positive definite?
185
00:12:15,170 --> 00:12:18,480
No, absolutely not.
186
00:12:18,480 --> 00:12:21,990
I don't know its eigenvalues,
but I know for sure one of them
187
00:12:21,990 --> 00:12:24,230
is negative.
188
00:12:24,230 --> 00:12:28,940
Its pivots are two and then
the next pivot would be
189
00:12:28,940 --> 00:12:32,250
the determinant over two, and
the determinant is -- what,
190
00:12:32,250 --> 00:12:35,090
what's the determinant
of this thing?
191
00:12:35,090 --> 00:12:37,620
Fourteen minus
thirty six, I've got
192
00:12:37,620 --> 00:12:39,820
a determinant minus twenty two.
193
00:12:39,820 --> 00:12:43,330
The next pivot will
be -- the pivots now,
194
00:12:43,330 --> 00:12:48,480
of this thing are two and
minus eleven or something.
195
00:12:48,480 --> 00:12:51,320
Their product being minus
twenty two the determinant.
196
00:12:51,320 --> 00:12:53,620
This thing is not
positive definite.
197
00:12:53,620 --> 00:12:57,200
What would be -- let me look
at the x transpose Ax for this
198
00:12:57,200 --> 00:12:57,700
guy.
199
00:12:57,700 --> 00:13:00,460
What's -- if I do out
this multiplication,
200
00:13:00,460 --> 00:13:04,560
this eighteen is temporarily
changing to a seven.
201
00:13:04,560 --> 00:13:08,130
This eighteen is temporarily
changing to a seven,
202
00:13:08,130 --> 00:13:16,610
and I know that there's
some numbers x1 and x2
203
00:13:16,610 --> 00:13:24,880
for which that thing, that
function, is negative.
204
00:13:24,880 --> 00:13:28,640
And I'm desperately trying
to think what they are.
205
00:13:28,640 --> 00:13:30,040
Maybe you can see.
206
00:13:30,040 --> 00:13:33,940
Can you tell me a
value of x1 and x2
207
00:13:33,940 --> 00:13:36,675
that makes this
quantity negative?
208
00:13:40,400 --> 00:13:43,180
Oh, maybe one and minus one?
209
00:13:43,180 --> 00:13:49,270
Yes, that's -- in this case,
that, will work, right,
210
00:13:49,270 --> 00:13:54,150
if I took x1 to be one,
and x2 to be minus one,
211
00:13:54,150 --> 00:13:57,170
then I always get something
positive, the two,
212
00:13:57,170 --> 00:14:01,367
and the seven minus one squared,
but this would be minus twelve
213
00:14:01,367 --> 00:14:02,950
and the whole thing
would be negative;
214
00:14:02,950 --> 00:14:07,620
I would have two minus twelve
plus seven, a negative.
215
00:14:07,620 --> 00:14:11,550
If I drew the graph, can I
get the little picture in
216
00:14:11,550 --> 00:14:12,050
here?
217
00:14:12,050 --> 00:14:16,430
If I draw the graph
of this thing?
218
00:14:16,430 --> 00:14:22,730
So, graphs, of the
function f(x,y), or f(x),
219
00:14:22,730 --> 00:14:29,620
so I say here f(x,y) equal
this -- x transpose Ax, this,
220
00:14:29,620 --> 00:14:32,040
this this ax squared,
2bxy, and cy squared.
221
00:14:32,040 --> 00:14:46,600
And, let's take the
example, with these numbers.
222
00:14:49,210 --> 00:14:54,650
OK, so here's the x axis,
here's the y axis, and here's --
223
00:14:54,650 --> 00:14:56,380
up is the function.
224
00:14:56,380 --> 00:14:59,390
z, if you like, or f.
225
00:14:59,390 --> 00:15:03,940
I apologize, and let me,
just once in my life here,
226
00:15:03,940 --> 00:15:08,780
put an arrow over these,
these, axes so you see them.
227
00:15:08,780 --> 00:15:13,300
That's the vector and I just,
see, instead of x1 and x2,
228
00:15:13,300 --> 00:15:16,220
I made them x- the
components x and y.
229
00:15:16,220 --> 00:15:16,930
OK.
230
00:15:16,930 --> 00:15:23,660
So, so, what's a graph of 2x
squared, twelve xy, and seven y
231
00:15:23,660 --> 00:15:25,710
squared?
232
00:15:25,710 --> 00:15:27,660
I'd like to see --
233
00:15:27,660 --> 00:15:31,260
I not the greatest
artist, but let's --
234
00:15:31,260 --> 00:15:38,400
tell me something about
this graph of this function.
235
00:15:42,560 --> 00:15:44,740
Whoa, tell me one point
that it goes through.
236
00:15:47,970 --> 00:15:49,470
The origin.
237
00:15:49,470 --> 00:15:50,990
Right?
238
00:15:50,990 --> 00:15:56,450
Even this artist can get this
thing to go through the origin,
239
00:15:56,450 --> 00:16:00,421
when these are zero, I,
I certainly get zero.
240
00:16:00,421 --> 00:16:00,920
OK.
241
00:16:00,920 --> 00:16:02,540
Some more points.
242
00:16:02,540 --> 00:16:06,570
If x is one and y is zero,
then I'm going upwards,
243
00:16:06,570 --> 00:16:09,060
so I'm going up this
way, and I'm, I'm
244
00:16:09,060 --> 00:16:12,180
going up, like, two x
squared in that direction.
245
00:16:12,180 --> 00:16:14,766
So -- that's meant
to be a parabola.
246
00:16:18,200 --> 00:16:23,150
And, suppose x stays
zero and y increases.
247
00:16:23,150 --> 00:16:26,760
Well, y could be positive or
negative; it's seven y squared.
248
00:16:26,760 --> 00:16:30,530
Is this function going upward?
249
00:16:30,530 --> 00:16:35,640
In the x direction it's going
upward, and in the y direction
250
00:16:35,640 --> 00:16:39,900
it's going upwards,
and if x equals y
251
00:16:39,900 --> 00:16:42,140
then the forty-five degree
direction is certainly
252
00:16:42,140 --> 00:16:44,810
going upwards; because
then we'd have what,
253
00:16:44,810 --> 00:16:49,600
about, everything would
be positive, but what?
254
00:16:49,600 --> 00:16:54,810
This function -- what's
the graph of this function?
255
00:16:54,810 --> 00:16:55,310
Look like?
256
00:16:55,310 --> 00:17:00,450
Tell me the word that describes
the graph of this function.
257
00:17:00,450 --> 00:17:02,980
This is the non-positive
definite here,
258
00:17:02,980 --> 00:17:07,660
everybody's with me
here, for some reason
259
00:17:07,660 --> 00:17:10,740
got started in a negative
direction, your case that
260
00:17:10,740 --> 00:17:12,750
isn't positive definite.
261
00:17:12,750 --> 00:17:17,190
And what's the graph look like
that goes up, but does it --
262
00:17:17,190 --> 00:17:22,710
do we have a minimum here, does
it go from, from the origin?
263
00:17:22,710 --> 00:17:24,500
Completely?
264
00:17:24,500 --> 00:17:28,710
No, because we just checked
that this thing failed.
265
00:17:28,710 --> 00:17:33,490
It failed along the direction
when x was minus y --
266
00:17:33,490 --> 00:17:37,080
we have a saddle point,
let me put myself, let me,
267
00:17:37,080 --> 00:17:39,350
to the least, tell you the word.
268
00:17:39,350 --> 00:17:46,470
This thing, goes up
in some directions,
269
00:17:46,470 --> 00:17:53,330
but down in other directions,
and if we actually knew what
270
00:17:53,330 --> 00:17:59,300
a saddle looked like or
thinks saddles do that --
271
00:17:59,300 --> 00:18:06,560
the way your legs go is,
like, down, up, the way, you,
272
00:18:06,560 --> 00:18:16,200
looking like, forward, and, the,
and drawing the thing is even
273
00:18:16,200 --> 00:18:17,330
worse than describing --
274
00:18:17,330 --> 00:18:20,760
I'm just going to say in
some directions we go up
275
00:18:20,760 --> 00:18:28,290
and in other directions,
there is, a saddle --
276
00:18:28,290 --> 00:18:30,330
Now I'm sorry I put
that on the front board,
277
00:18:30,330 --> 00:18:34,450
you have no way to cover
it, but it's a saddle.
278
00:18:34,450 --> 00:18:35,360
OK.
279
00:18:35,360 --> 00:18:38,950
And, and this is a
saddle point, it's
280
00:18:38,950 --> 00:18:43,900
the, it's the point that's at
the maximum in some directions
281
00:18:43,900 --> 00:18:46,770
and at the minimum
in other directions.
282
00:18:46,770 --> 00:18:50,840
And actually, the perfect
directions to look
283
00:18:50,840 --> 00:18:53,670
are the eigenvector directions.
284
00:18:53,670 --> 00:18:55,260
We'll see that.
285
00:18:55,260 --> 00:19:03,730
So this is, not a
positive definite matrix.
286
00:19:03,730 --> 00:19:04,280
OK.
287
00:19:04,280 --> 00:19:08,190
Now I'm coming back
to this example,
288
00:19:08,190 --> 00:19:14,400
getting rid of this seven,
let's move it up to twenty.
289
00:19:14,400 --> 00:19:18,420
Let's, let's let's make the
thing really positive definite.
290
00:19:18,420 --> 00:19:19,470
OK.
291
00:19:19,470 --> 00:19:22,990
So this is, this
number's now twenty.
292
00:19:22,990 --> 00:19:24,360
c is now twenty.
293
00:19:24,360 --> 00:19:24,860
OK.
294
00:19:24,860 --> 00:19:30,800
Now that passes the test, which
I haven't proved, of course,
295
00:19:30,800 --> 00:19:34,960
it passes the test
for positive pivots.
296
00:19:34,960 --> 00:19:40,270
It passes the test for
positive eigenvalues.
297
00:19:40,270 --> 00:19:43,160
How can you tell that the
eigenvalues of that matrix
298
00:19:43,160 --> 00:19:45,850
are positive without
actually finding them?
299
00:19:45,850 --> 00:19:49,400
Of course, two by two I could
find them, but can you see --
300
00:19:49,400 --> 00:19:51,530
how do I know they're positive?
301
00:19:51,530 --> 00:19:53,270
I know that their product is --
302
00:19:53,270 --> 00:19:58,020
I know that lambda one times
lambda two is positive, why?
303
00:19:58,020 --> 00:20:01,750
Because that's the
determinant, right,
304
00:20:01,750 --> 00:20:06,000
lambda one times lambda two is
the determinant, which is forty
305
00:20:06,000 --> 00:20:07,730
minus thirty-six is four.
306
00:20:07,730 --> 00:20:11,120
So the determinant is four.
307
00:20:11,120 --> 00:20:16,160
And the trace, the sum down
the diagonal, is twenty-two.
308
00:20:16,160 --> 00:20:20,630
So, they multiply to give four.
309
00:20:20,630 --> 00:20:22,850
So that leaves the
possibility they're either
310
00:20:22,850 --> 00:20:25,730
both positive or both negative.
311
00:20:25,730 --> 00:20:29,979
But if they're both negative,
the trace couldn't be
312
00:20:29,979 --> 00:20:31,520
So they're both
positive. twenty-two.
313
00:20:31,520 --> 00:20:34,170
So both of the eigenvalues
that are positive,
314
00:20:34,170 --> 00:20:36,220
both of the pivots
are positive --
315
00:20:36,220 --> 00:20:39,820
the determinants are positive,
and I believe that this
316
00:20:39,820 --> 00:20:47,260
function is positive everywhere
except at zero, zero,
317
00:20:47,260 --> 00:20:48,240
of course.
318
00:20:48,240 --> 00:20:52,330
When I write down
this condition,
319
00:20:52,330 --> 00:20:54,670
So I believe that
x transposed, let
320
00:20:54,670 --> 00:21:00,280
me copy, x transpose Ax is
positive, except, of course,
321
00:21:00,280 --> 00:21:07,450
at the minimum point,
at zero, of course,
322
00:21:07,450 --> 00:21:08,830
I don't expect miracles.
323
00:21:11,760 --> 00:21:15,720
So what does its graph look
like, and how do I check,
324
00:21:15,720 --> 00:21:18,730
and how do I check that
this really is positive?
325
00:21:22,300 --> 00:21:24,960
So we take it's
graph for a minute.
326
00:21:24,960 --> 00:21:27,200
What would be the graph
of that function --
327
00:21:27,200 --> 00:21:29,400
it does not have a saddle point.
328
00:21:29,400 --> 00:21:31,280
Let me -- I'll raise
the board, here,
329
00:21:31,280 --> 00:21:33,320
and stay with this
example for a while.
330
00:21:36,280 --> 00:21:40,660
So I want to do the graph
of -- here's my function,
331
00:21:40,660 --> 00:21:46,940
two x squared, twelve xy-s, that
could be positive or negative,
332
00:21:46,940 --> 00:21:48,510
and twenty y squared.
333
00:21:48,510 --> 00:21:56,290
But my point is, so you're
seeing the underlying point is,
334
00:21:56,290 --> 00:21:59,910
that, the things are
positive definite
335
00:21:59,910 --> 00:22:05,730
when in some way, these,
these pure squares, squares
336
00:22:05,730 --> 00:22:10,180
we know to be positive, and
when those kind of overwhelm
337
00:22:10,180 --> 00:22:13,527
this guy, who could be
m- positive or negative,
338
00:22:13,527 --> 00:22:15,110
because some like
or have same or have
339
00:22:15,110 --> 00:22:19,760
same or different signs,
when these are big enough
340
00:22:19,760 --> 00:22:22,850
they overwhelm this guy and
make the total thing positive,
341
00:22:22,850 --> 00:22:25,750
and what would the
graph now look like?
342
00:22:25,750 --> 00:22:32,940
Let me draw the x - well, let
me draw the x direction, the y
343
00:22:32,940 --> 00:22:40,970
direction, and the origin,
at zero, zero, I'm there,
344
00:22:40,970 --> 00:22:45,650
where do I go as I move
away from the origin?
345
00:22:45,650 --> 00:22:50,780
Where do I go as I move
away from the origin?
346
00:22:50,780 --> 00:22:53,240
I'm sure that I go up.
347
00:22:53,240 --> 00:22:56,660
The origin, the
center point here,
348
00:22:56,660 --> 00:23:01,730
is a minim because this thing I
believe, and we better see why,
349
00:23:01,730 --> 00:23:07,620
it's, the graph is like a bowl,
the graph is like a bowl shape,
350
00:23:07,620 --> 00:23:09,805
it's -- here's the minimum.
351
00:23:15,660 --> 00:23:19,030
And because we've
got a pure quadratic,
352
00:23:19,030 --> 00:23:23,410
we know it sits at the origin,
we know it's tangent plane,
353
00:23:23,410 --> 00:23:30,970
the first derivatives are zero,
so, we know, first derivatives,
354
00:23:30,970 --> 00:23:37,410
First derivatives are
all zero, but that's
355
00:23:37,410 --> 00:23:38,560
not enough for a minimum.
356
00:23:38,560 --> 00:23:43,020
It's first derivatives
were zero here.
357
00:23:45,780 --> 00:23:51,200
So, the partial derivatives,
the first derivatives, are zero.
358
00:23:51,200 --> 00:23:56,720
Again, because first derivatives
are gonna have an x or an a y,
359
00:23:56,720 --> 00:23:59,730
or a y in them, they'll
be zero at the origin.
360
00:23:59,730 --> 00:24:03,340
It's the second derivatives
that control everything.
361
00:24:03,340 --> 00:24:07,810
It's the second derivatives
that this matrix is telling us,
362
00:24:07,810 --> 00:24:10,630
and somehow --
363
00:24:10,630 --> 00:24:11,780
here's my point.
364
00:24:11,780 --> 00:24:16,710
You remember in Calculus, how
did you decide on a minimum?
365
00:24:16,710 --> 00:24:20,010
First requirement was, that
the derivative had to be
366
00:24:20,010 --> 00:24:20,940
zero.
367
00:24:20,940 --> 00:24:25,890
But then you didn't know if
you had a minimum or a maximum.
368
00:24:25,890 --> 00:24:27,500
To know that you
had a minimum, you
369
00:24:27,500 --> 00:24:30,300
had to look at the
second derivative.
370
00:24:30,300 --> 00:24:33,400
The second derivative
had to be positive,
371
00:24:33,400 --> 00:24:36,900
the slope had to be
increasing as you
372
00:24:36,900 --> 00:24:40,010
went through the minimum point.
373
00:24:40,010 --> 00:24:43,470
The curvature had to
go upwards, and that's
374
00:24:43,470 --> 00:24:46,910
what we're doing now
in two dimensions,
375
00:24:46,910 --> 00:24:49,280
and in n dimensions.
376
00:24:49,280 --> 00:24:52,030
So we're doing what
we did in Calculus.
377
00:24:52,030 --> 00:24:55,150
Second derivative
positive, m- will now
378
00:24:55,150 --> 00:24:58,590
become that the matrix
of second derivatives
379
00:24:58,590 --> 00:25:00,810
is positive definite.
380
00:25:00,810 --> 00:25:02,650
Can I just --
381
00:25:02,650 --> 00:25:05,670
like a translation of --
382
00:25:05,670 --> 00:25:11,890
this is how minimum are coming
in, ithe beginning of Calculus
383
00:25:11,890 --> 00:25:14,640
--
384
00:25:14,640 --> 00:25:22,730
we had a minimum was associated
with second derivative,
385
00:25:22,730 --> 00:25:24,170
being positive.
386
00:25:24,170 --> 00:25:27,340
And first derivative
zero, of course.
387
00:25:27,340 --> 00:25:36,120
Derivative, first
derivative, but it
388
00:25:36,120 --> 00:25:39,870
was the second derivative
that told us we had a minimum.
389
00:25:39,870 --> 00:25:43,510
And now, in 18.06,
in linear algebra,
390
00:25:43,510 --> 00:25:47,260
we'll have a minim
for our function now,
391
00:25:47,260 --> 00:25:53,040
our function will have, for
your function be a function not
392
00:25:53,040 --> 00:26:00,290
of just x but several variables,
the way functions really
393
00:26:00,290 --> 00:26:03,370
are in real life,
the minimum will
394
00:26:03,370 --> 00:26:15,140
be when the matrix of second
derivatives, the matrix
395
00:26:15,140 --> 00:26:17,590
here was one by one, there was
just one second derivative,
396
00:26:17,590 --> 00:26:20,560
now we've got lots.
397
00:26:20,560 --> 00:26:25,335
Is positive definite.
398
00:26:29,070 --> 00:26:31,450
So positive for a
number translates
399
00:26:31,450 --> 00:26:34,450
into positive
definite for a matrix.
400
00:26:34,450 --> 00:26:38,860
And it this brings
everything you check pivots,
401
00:26:38,860 --> 00:26:41,680
you check determinants,
you check all your values,
402
00:26:41,680 --> 00:26:44,910
or you check this minimum stuff.
403
00:26:44,910 --> 00:26:45,410
OK.
404
00:26:45,410 --> 00:26:49,190
Let me come back to this graph.
405
00:26:49,190 --> 00:26:50,595
That graph goes upwards.
406
00:26:53,220 --> 00:26:54,740
And I'll have to see why.
407
00:26:54,740 --> 00:26:58,360
How do I know that this,
that this function is always
408
00:26:58,360 --> 00:26:59,980
positive?
409
00:26:59,980 --> 00:27:04,770
Can you look at that and tell
that it's always positive?
410
00:27:04,770 --> 00:27:08,710
Maybe two by two, you
could feel pretty sure,
411
00:27:08,710 --> 00:27:14,530
but what's the good way to
show that this thing is always
412
00:27:14,530 --> 00:27:20,630
If we can express it, as, in
terms of squares, positive?
413
00:27:20,630 --> 00:27:24,010
because that's what we
know for any x and y,
414
00:27:24,010 --> 00:27:26,760
whatever, if we're
squaring something
415
00:27:26,760 --> 00:27:29,470
we certainly are not negative.
416
00:27:29,470 --> 00:27:32,690
So I believe that this
expression, this function,
417
00:27:32,690 --> 00:27:37,190
could be written as
a sum of squares.
418
00:27:37,190 --> 00:27:41,170
Can you tell me --
419
00:27:41,170 --> 00:27:43,460
see, because all the
problems, the headaches
420
00:27:43,460 --> 00:27:47,230
are coming from this xy term.
421
00:27:47,230 --> 00:27:50,500
If we can get expressions
-- if we can get that inside
422
00:27:50,500 --> 00:27:53,660
a square, so actually, what
we're doing is something
423
00:27:53,660 --> 00:27:58,160
called, that you've seen
called completing the square.
424
00:27:58,160 --> 00:28:01,210
Let me start the square
and you complete it.
425
00:28:01,210 --> 00:28:05,820
OK, I think we
have two of x plus,
426
00:28:05,820 --> 00:28:09,400
now I don't remember how
many y-s we need, but you'll
427
00:28:09,400 --> 00:28:10,725
figure it out, squared.
428
00:28:13,750 --> 00:28:20,470
How many y-s should I
put in here, to make --
429
00:28:20,470 --> 00:28:23,950
what do I want to do, the two
x squared-s will be correct,
430
00:28:23,950 --> 00:28:25,770
right?
431
00:28:25,770 --> 00:28:28,990
What I want to do is put
in the right number of y-s
432
00:28:28,990 --> 00:28:33,270
to get the twelve xy correct.
433
00:28:33,270 --> 00:28:34,885
And what is that number of y-s?
434
00:28:37,430 --> 00:28:39,170
Let's see, I've got
two times, and so
435
00:28:39,170 --> 00:28:41,770
I really want six xy-s
to come out of here,
436
00:28:41,770 --> 00:28:44,650
I think maybe if
I put three there,
437
00:28:44,650 --> 00:28:48,590
does that look right to you?
438
00:28:48,590 --> 00:28:53,940
I have two- this is, we
can mentally, multiply out,
439
00:28:53,940 --> 00:28:56,420
that's X squared,
that's right, that's
440
00:28:56,420 --> 00:28:59,580
six X Y, times the
two gives from, right,
441
00:28:59,580 --> 00:29:02,660
and how many Y squareds
have I now got?
442
00:29:02,660 --> 00:29:05,950
How many Y squareds have
I now got from this term?
443
00:29:05,950 --> 00:29:07,750
Eighteen.
444
00:29:07,750 --> 00:29:12,000
Eighteen was the key
number, remember?
445
00:29:12,000 --> 00:29:16,630
Now if I want to make it
twenty, then I've got two left.
446
00:29:16,630 --> 00:29:18,100
Two y squared-s.
447
00:29:18,100 --> 00:29:25,660
That's completing the
square, and it's, now
448
00:29:25,660 --> 00:29:28,120
I can see that that
function is positive,
449
00:29:28,120 --> 00:29:30,090
because it's all squares.
450
00:29:33,340 --> 00:29:35,840
I've got two squares,
added together,
451
00:29:35,840 --> 00:29:38,110
I couldn't go negative.
452
00:29:38,110 --> 00:29:42,070
What if I went
back to that seven?
453
00:29:42,070 --> 00:29:45,080
If instead of twenty that
number was a seven, then
454
00:29:45,080 --> 00:29:47,610
what would happen?
455
00:29:47,610 --> 00:29:51,010
This would still be correct,
I'd still have this square,
456
00:29:51,010 --> 00:29:53,670
to get the two x squared
and the twelve xy,
457
00:29:53,670 --> 00:29:58,800
and I'd have eighteen y squared
and then what would I do here?
458
00:29:58,800 --> 00:30:03,760
I'd have to remove eleven
y squared-s, right,
459
00:30:03,760 --> 00:30:08,200
if I only had a seven
here, then instead of --
460
00:30:08,200 --> 00:30:13,040
when I had a twenty I had two
more to put in, when I had
461
00:30:13,040 --> 00:30:16,740
an eighteen, which
was the marginal case,
462
00:30:16,740 --> 00:30:19,780
I had no more to put in.
463
00:30:19,780 --> 00:30:23,370
When I had a seven, which
was the case below zero,
464
00:30:23,370 --> 00:30:29,870
the indefinite case,
I had minus eleven.
465
00:30:29,870 --> 00:30:36,360
Now, so, you can see now,
that this thing is a bowl.
466
00:30:36,360 --> 00:30:36,860
OK.
467
00:30:36,860 --> 00:30:42,200
It's going upwards, if I cut
it at a plane, z equal to one,
468
00:30:42,200 --> 00:30:47,830
say, I would get, I would
get a curve, what would
469
00:30:47,830 --> 00:30:50,280
be the equation for that curve?
470
00:30:50,280 --> 00:30:53,190
If I cut it at height
one, the equation
471
00:30:53,190 --> 00:30:56,980
would be this
thing equal to one.
472
00:30:56,980 --> 00:30:59,910
And that curve
would be an ellipse.
473
00:30:59,910 --> 00:31:02,560
So actually,
already, I've blocked
474
00:31:02,560 --> 00:31:09,380
into the lecture, the different
pieces that we're aiming for.
475
00:31:09,380 --> 00:31:12,620
We're aiming for the
tests, which this passed;
476
00:31:12,620 --> 00:31:17,360
we're aiming for the connection
to a minimum, which this --
477
00:31:17,360 --> 00:31:22,420
which we see in the graph,
and if we chop it up,
478
00:31:22,420 --> 00:31:25,180
if we set this
thing equal to one,
479
00:31:25,180 --> 00:31:27,560
if I set that thing
equal to one, that --
480
00:31:27,560 --> 00:31:30,490
what that gives me
is, the cross-section.
481
00:31:30,490 --> 00:31:37,750
It gives me this, this
curve, and its equation
482
00:31:37,750 --> 00:31:41,200
is this thing equals one,
and that's an ellipse.
483
00:31:41,200 --> 00:31:44,660
Whereas if I cut
through a saddle point,
484
00:31:44,660 --> 00:31:47,900
I get a hyperbola.
485
00:31:47,900 --> 00:31:53,820
But this minimum stuff is
really what I'm most interested
486
00:31:53,820 --> 00:31:54,610
OK.
487
00:31:54,610 --> 00:31:55,110
in.
488
00:31:55,110 --> 00:31:55,610
OK.
489
00:31:55,610 --> 00:32:00,300
By -- I just have to ask,
do you recognize, I mean,
490
00:32:00,300 --> 00:32:04,460
these numbers here, the
two that appeared outside,
491
00:32:04,460 --> 00:32:08,090
the three that appeared inside,
the two that appeared there --
492
00:32:08,090 --> 00:32:13,690
actually, those numbers
come from elimination.
493
00:32:13,690 --> 00:32:18,890
Completing the square
is our good old method
494
00:32:18,890 --> 00:32:24,910
of Gaussian elimination,
in expressed
495
00:32:24,910 --> 00:32:27,280
in terms of these squares.
496
00:32:27,280 --> 00:32:30,600
The -- let me show
you what I mean.
497
00:32:30,600 --> 00:32:34,050
I just think those
numbers are no accident,
498
00:32:34,050 --> 00:32:40,050
If I take my matrix two,
six, six, and twenty,
499
00:32:40,050 --> 00:32:45,050
and I do elimination,
then the pivot is two
500
00:32:45,050 --> 00:32:50,150
and I take three,
what's the multiplier?
501
00:32:50,150 --> 00:32:54,180
How much of row one do I
take away from row two?
502
00:32:54,180 --> 00:32:55,360
Three.
503
00:32:55,360 --> 00:32:58,700
So what you're seeing in
this, completing the square,
504
00:32:58,700 --> 00:33:05,180
is the pivots outside and
the multiplier inside.
505
00:33:05,180 --> 00:33:06,680
Just do that again?
506
00:33:06,680 --> 00:33:12,840
The pivot is two, three -- three
of those away from that gives
507
00:33:12,840 --> 00:33:18,100
me two, six, zero, and
what's the second pivot?
508
00:33:18,100 --> 00:33:21,550
Three of this away from this,
three sixes'll be eighteen,
509
00:33:21,550 --> 00:33:23,380
and the second
pivot will also be a
510
00:33:23,380 --> 00:33:24,040
two.
511
00:33:24,040 --> 00:33:32,490
So that's the U, this is
the A, and of course the L
512
00:33:32,490 --> 00:33:37,900
was one, zero, one, and
the multiplier was three.
513
00:33:40,540 --> 00:33:48,210
So, completing the
square is elimination.
514
00:33:48,210 --> 00:33:54,230
Why I happy to see, happy
to see that coming together?
515
00:33:54,230 --> 00:34:01,190
Because I know about
elimination for m by m matrices.
516
00:34:01,190 --> 00:34:07,740
I just started talking about
completing the square, here,
517
00:34:07,740 --> 00:34:10,320
for two by twos.
518
00:34:10,320 --> 00:34:12,989
But now I see what's going on.
519
00:34:12,989 --> 00:34:17,909
Completing the square really
amounts to splitting this thing
520
00:34:17,909 --> 00:34:21,610
into a sum of squares, so
what's the critical thing --
521
00:34:21,610 --> 00:34:25,380
I have a lot of squares,
and inside those squares
522
00:34:25,380 --> 00:34:28,020
are multipliers but
they're squares,
523
00:34:28,020 --> 00:34:31,790
and the question is, what's
outside these squares?
524
00:34:31,790 --> 00:34:35,210
When I complete the square,
what are the numbers that go
525
00:34:35,210 --> 00:34:36,179
outside?
526
00:34:36,179 --> 00:34:37,342
They're the pivots.
527
00:34:39,889 --> 00:34:45,510
They're the pivots, and that's
why positive pivots give me
528
00:34:45,510 --> 00:34:47,409
sum of squares.
529
00:34:47,409 --> 00:34:50,270
Positive pivots, those
pivots are the numbers
530
00:34:50,270 --> 00:34:53,600
that go outside the
squares, so positive pivots,
531
00:34:53,600 --> 00:34:57,850
sum of squares, everything
positive, graph goes up,
532
00:34:57,850 --> 00:35:03,050
a minimum at the origin,
it's all connected together;
533
00:35:03,050 --> 00:35:04,710
all connected together.
534
00:35:04,710 --> 00:35:08,110
And in the two by two case,
you can see those connections,
535
00:35:08,110 --> 00:35:16,100
but linear algebra now can go
up to three by three, m by m.
536
00:35:16,100 --> 00:35:17,950
Let's do that next.
537
00:35:17,950 --> 00:35:20,680
Can I just, before
I leave two by two,
538
00:35:20,680 --> 00:35:24,940
I've written this expression
"matrix of second derivatives."
539
00:35:24,940 --> 00:35:27,165
What's the matrix of
second derivatives?
540
00:35:29,690 --> 00:35:31,970
That's one second
derivative now,
541
00:35:31,970 --> 00:35:40,020
but if I'm in two dimensions,
I have a two by two matrix,
542
00:35:40,020 --> 00:35:46,480
it's the second x derivative,
the second x derivative goes
543
00:35:46,480 --> 00:35:49,070
there --
544
00:35:49,070 --> 00:35:56,360
shall I write it --
fxx, if you like, fxx,
545
00:35:56,360 --> 00:36:00,170
that means the second derivative
of f in the x direction.
546
00:36:00,170 --> 00:36:04,500
fyy, second derivative
in the y direction.
547
00:36:04,500 --> 00:36:07,970
Those are the pure derivatives,
second derivatives.
548
00:36:07,970 --> 00:36:10,950
They have to be positive.
549
00:36:10,950 --> 00:36:13,430
For a minimum.
550
00:36:13,430 --> 00:36:15,590
This number has to be
positive for a minimum.
551
00:36:15,590 --> 00:36:17,730
That number has to be
positive for a minimum.
552
00:36:17,730 --> 00:36:20,410
But, that's not enough.
553
00:36:20,410 --> 00:36:23,680
Those numbers have to
somehow be big enough
554
00:36:23,680 --> 00:36:30,090
to overcome this
cross-derivative,
555
00:36:30,090 --> 00:36:31,780
Why is the matrix symmetric?
556
00:36:31,780 --> 00:36:35,730
Because the second derivative
f with respect to x and y is
557
00:36:35,730 --> 00:36:37,950
equal to --
558
00:36:37,950 --> 00:36:42,170
I can, that's the beautiful fact
about second derivatives, is
559
00:36:42,170 --> 00:36:46,480
I can do those in either order
and I get the same thing.
560
00:36:46,480 --> 00:36:54,410
So this is the same as
that, and so, that's
561
00:36:54,410 --> 00:36:57,300
the matrix of
second derivatives.
562
00:36:57,300 --> 00:37:01,280
And the test is, it has
to be positive definite.
563
00:37:01,280 --> 00:37:05,960
You might remember,
from, tucked in somewhere
564
00:37:05,960 --> 00:37:08,990
near the end of eighteen o'
two or at least in the book,
565
00:37:08,990 --> 00:37:13,590
was the condition for a
minimum, For a function
566
00:37:13,590 --> 00:37:15,470
of two variables.
567
00:37:15,470 --> 00:37:17,180
Let's -- when do
you have a minimum?
568
00:37:17,180 --> 00:37:19,740
For a function of two
variables, believe me,
569
00:37:19,740 --> 00:37:24,200
that's what Calculus is for.
570
00:37:24,200 --> 00:37:29,270
The condition is first
derivatives have to be zero.
571
00:37:29,270 --> 00:37:32,630
And the matrix of
second derivatives
572
00:37:32,630 --> 00:37:35,280
has to be positive definite.
573
00:37:35,280 --> 00:37:39,650
So you maybe remember there
was an fxx times an fyy that
574
00:37:39,650 --> 00:37:42,610
had to be bigger than
an an fxy squared,
575
00:37:42,610 --> 00:37:46,630
that's just our
determinant, two by two.
576
00:37:46,630 --> 00:37:51,480
But now, we now know the
answer for three by three,
577
00:37:51,480 --> 00:37:57,770
m by m, because we can do
elimination by m by m matrices,
578
00:37:57,770 --> 00:38:01,660
we can connect eigenvalues
of m by m matrices,
579
00:38:01,660 --> 00:38:06,070
we can do sum of squares, sum
of m squares instead of only two
580
00:38:06,070 --> 00:38:11,470
squares; and so
let's take a, let
581
00:38:11,470 --> 00:38:14,000
me go over here to do a
three by three example.
582
00:38:17,110 --> 00:38:18,510
So, three by three example.
583
00:38:23,445 --> 00:38:23,945
OK.
584
00:38:27,060 --> 00:38:29,113
Oh, let me --
585
00:38:32,980 --> 00:38:34,670
shall I use my favorite matrix?
586
00:38:37,210 --> 00:38:39,080
You've seen this matrix before.
587
00:38:45,340 --> 00:38:49,680
Yes, let's use the good matrix,
four by one, oops, open.
588
00:38:56,130 --> 00:38:57,950
Is that matrix
positive definite?
589
00:39:01,300 --> 00:39:04,440
What's -- so I'm going to ask
questions about this matrix,
590
00:39:04,440 --> 00:39:08,260
is it positive
definite, first of all?
591
00:39:08,260 --> 00:39:10,970
What's the function
associated with that matrix,
592
00:39:10,970 --> 00:39:12,780
what's the x transpose Ax?
593
00:39:16,260 --> 00:39:18,860
Is -- do we have a
minimum for that function,
594
00:39:18,860 --> 00:39:19,865
at zero?
595
00:39:22,460 --> 00:39:25,380
And then even
what's the geometry?
596
00:39:25,380 --> 00:39:26,060
OK.
597
00:39:26,060 --> 00:39:29,200
First of all, is the
matrix positive definite,
598
00:39:29,200 --> 00:39:33,460
now I've given you the
numbers there so you can take
599
00:39:33,460 --> 00:39:35,370
the determinants, maybe
that's the quickest,
600
00:39:35,370 --> 00:39:36,910
is that what you
would do mentally,
601
00:39:36,910 --> 00:39:39,660
if I give you all a
matrix on a quiz and say
602
00:39:39,660 --> 00:39:43,010
is it positive definite or not?
603
00:39:43,010 --> 00:39:47,240
I would take that determinant
and I'd give the answer two.
604
00:39:47,240 --> 00:39:49,130
I would take that
determinant and I
605
00:39:49,130 --> 00:39:55,380
would give the answer for
that two by two determinant,
606
00:39:55,380 --> 00:39:57,400
I'd give the answer
three, and anybody
607
00:39:57,400 --> 00:40:04,220
remember the answer for the
three by three determinant?
608
00:40:04,220 --> 00:40:08,220
It was four, you remember
for these special matrices,
609
00:40:08,220 --> 00:40:13,190
when we do determinants, they
went up two, three, four, five,
610
00:40:13,190 --> 00:40:15,950
six, they just went up linearly.
611
00:40:15,950 --> 00:40:22,940
So that matrix has -- the
determinants are two, three,
612
00:40:22,940 --> 00:40:24,615
and four.
613
00:40:24,615 --> 00:40:25,115
Pivots.
614
00:40:27,630 --> 00:40:31,160
What are the pivots
for that matrix?
615
00:40:31,160 --> 00:40:34,470
I'll tell you, they're --
the first pivot is two,
616
00:40:34,470 --> 00:40:37,520
the next pivot is
three over two,
617
00:40:37,520 --> 00:40:39,730
the next pivot is
four over three.
618
00:40:42,590 --> 00:40:45,170
Because, the product
of the pivots
619
00:40:45,170 --> 00:40:47,550
has to give me
those determinants.
620
00:40:47,550 --> 00:40:50,370
The product of these two pivots
gives me that determinant;
621
00:40:50,370 --> 00:40:53,574
the product of all the pivots
gives me that determinant.
622
00:40:53,574 --> 00:40:54,615
What are the eigenvalues?
623
00:41:03,020 --> 00:41:05,717
Oh, I don't know.
624
00:41:08,600 --> 00:41:12,860
The eigenvalues I've got, what
do I have a cubic equation --
625
00:41:12,860 --> 00:41:14,320
a degree three equation?
626
00:41:17,230 --> 00:41:19,350
There are three
eigenvalues to find.
627
00:41:23,350 --> 00:41:25,200
If I believe what
I've said today,
628
00:41:25,200 --> 00:41:27,800
what do I know about
these eigenvalues,
629
00:41:27,800 --> 00:41:30,710
even though I don't
know the exact numbers.
630
00:41:30,710 --> 00:41:34,535
I -- I remember the numbers.
631
00:41:37,100 --> 00:41:39,740
Because these matrices
are so important
632
00:41:39,740 --> 00:41:43,750
that people figure them.
633
00:41:43,750 --> 00:41:47,440
But -- what do you believe
to be true about these three
634
00:41:47,440 --> 00:41:52,950
eigenvalues -- you believe
that they are all positive.
635
00:41:52,950 --> 00:41:54,270
They're all positive.
636
00:41:54,270 --> 00:42:00,910
I think that they are two
minus square root of two, two,
637
00:42:00,910 --> 00:42:02,630
and two plus the
square root of two.
638
00:42:02,630 --> 00:42:03,720
I think.
639
00:42:03,720 --> 00:42:05,490
Let me just --
640
00:42:05,490 --> 00:42:08,280
I can't write those numbers
down without checking
641
00:42:08,280 --> 00:42:10,870
the simple checks, what
the first simple check is
642
00:42:10,870 --> 00:42:15,110
the trace, so if I add
those numbers I get six
643
00:42:15,110 --> 00:42:18,970
and if I add those
numbers I get six.
644
00:42:18,970 --> 00:42:24,150
The other simple test is
the determinant, if I --
645
00:42:24,150 --> 00:42:26,930
can you do this, can you
multiply those numbers
646
00:42:26,930 --> 00:42:29,240
together?
647
00:42:29,240 --> 00:42:32,610
I guess we can multiply
by two out there.
648
00:42:32,610 --> 00:42:35,010
What's two minus square
root of two times two
649
00:42:35,010 --> 00:42:39,530
plus square root of two,
that'll be four minus two,
650
00:42:39,530 --> 00:42:41,610
that'll be two,
yeah, two times two,
651
00:42:41,610 --> 00:42:46,280
that's got the determinant,
right, so it's got,
652
00:42:46,280 --> 00:42:49,810
it's got a chance of being
correct and I think it is.
653
00:42:49,810 --> 00:42:51,990
Now, what's the x transpose Ax?
654
00:42:51,990 --> 00:42:54,610
I better give myself
enough room for that.
655
00:42:54,610 --> 00:42:59,180
x transpose Ax for this guy.
656
00:42:59,180 --> 00:43:07,290
It's two x1 squareds, and two x2
squareds, and two x3 squareds.
657
00:43:07,290 --> 00:43:10,800
Those come from the
diagonal, those were easy.
658
00:43:10,800 --> 00:43:13,490
Now off the diagonal
there's a minus and a minus,
659
00:43:13,490 --> 00:43:20,010
they come together there'll be
a minus two minus two whats?
660
00:43:20,010 --> 00:43:26,390
Are coming from this one two and
two one position, is the x1 x2.
661
00:43:26,390 --> 00:43:30,900
I'm doing mentally
a multiplication
662
00:43:30,900 --> 00:43:34,300
of this matrix
times a row vector
663
00:43:34,300 --> 00:43:37,680
on the left times a column
vector on the right,
664
00:43:37,680 --> 00:43:41,730
and I know that these numbers
show up in the answer.
665
00:43:41,730 --> 00:43:45,060
The diagonal is
the perfect square,
666
00:43:45,060 --> 00:43:49,110
this off diagonal is
a minus two x1 x2,
667
00:43:49,110 --> 00:43:55,280
and there are no x1 x3-s, and
there're minus two x2 x3-s.
668
00:43:55,280 --> 00:43:58,860
And I believe that that
expression is always positive.
669
00:44:01,670 --> 00:44:06,010
I believe that that
curve, that graph, really,
670
00:44:06,010 --> 00:44:09,720
of that function,
this is my function f,
671
00:44:09,720 --> 00:44:14,770
and I'm in more dimensions
now than I can draw, it --
672
00:44:14,770 --> 00:44:20,080
but the graph of that
function goes upwards.
673
00:44:20,080 --> 00:44:22,320
It's a bowl.
674
00:44:22,320 --> 00:44:26,320
Or maybe the right word is --
675
00:44:26,320 --> 00:44:34,210
just forgot, what's
a long word for bowl?
676
00:44:34,210 --> 00:44:42,570
Hm, maybe paraboloid, I
think, paraboloid comes in.
677
00:44:42,570 --> 00:44:46,150
I'll edit the tape
and get that word in.
678
00:44:46,150 --> 00:44:54,410
Bowl, let's say, is, that,
so that, and if I can --
679
00:44:54,410 --> 00:44:56,700
I could complete the
squares, I could write that
680
00:44:56,700 --> 00:44:59,740
as the sum of three squares,
and those three squares
681
00:44:59,740 --> 00:45:02,780
would get multiplied
by the three pivots.
682
00:45:02,780 --> 00:45:05,440
And the pivots are all positive.
683
00:45:05,440 --> 00:45:08,760
So I would have positive
pivots times squares,
684
00:45:08,760 --> 00:45:11,630
the net result would
be a positive function
685
00:45:11,630 --> 00:45:14,520
and a bowl which goes upwards.
686
00:45:14,520 --> 00:45:19,230
And then, finally, if I cut --
if I slice through this bowl,
687
00:45:19,230 --> 00:45:23,730
if I -- now I'm asking you
to stretch your visualization
688
00:45:23,730 --> 00:45:26,680
here, because I'm
in four dimensions,
689
00:45:26,680 --> 00:45:33,610
I've got x1 x2 x3 in the base,
and this function is z, or f,
690
00:45:33,610 --> 00:45:35,000
or something.
691
00:45:35,000 --> 00:45:38,320
And its graph is going up.
692
00:45:38,320 --> 00:45:40,530
But I'm in four dimensions,
because I've got three
693
00:45:40,530 --> 00:45:43,410
in the base and then
the upward direction,
694
00:45:43,410 --> 00:45:49,640
but now if I cut through this
four-dimensional picture,
695
00:45:49,640 --> 00:45:55,720
at level one, so, suppose
I cut through this thing
696
00:45:55,720 --> 00:45:58,100
at height one.
697
00:45:58,100 --> 00:46:01,470
So I take all the points
that are at height one.
698
00:46:04,980 --> 00:46:07,770
That gives me --
699
00:46:07,770 --> 00:46:14,180
it gave me an ellipse over
there, in that two by two case,
700
00:46:14,180 --> 00:46:19,350
in this case, this will be
the equation of an ellipsoid,
701
00:46:19,350 --> 00:46:20,830
a football in other words.
702
00:46:23,770 --> 00:46:25,170
Well, not quite a football.
703
00:46:25,170 --> 00:46:26,450
A lopsided football.
704
00:46:26,450 --> 00:46:30,960
What would be, can I try
to describe to you what
705
00:46:30,960 --> 00:46:35,590
the ellipsoid will look
like, this ellipsoid,
706
00:46:35,590 --> 00:46:40,440
I'm sorry that the, that I've
ended the matrix right --
707
00:46:40,440 --> 00:46:46,230
at the point, let's -- let me be
sure you've seen the equation.
708
00:46:46,230 --> 00:46:51,130
Two x1 squared, two x2 squared,
two x3 squared, minus two
709
00:46:51,130 --> 00:46:57,000
of the cross parts, equal what?
710
00:46:57,000 --> 00:47:00,720
That is the equation
of a football, so what
711
00:47:00,720 --> 00:47:03,780
do I mean by a football
or an ellipsoid?
712
00:47:03,780 --> 00:47:12,520
I mean that, well,
I'll draw a few.
713
00:47:12,520 --> 00:47:21,750
It's like that,
it's got a center,
714
00:47:21,750 --> 00:47:31,990
and it's got it's got
three principal directions.
715
00:47:31,990 --> 00:47:32,830
This ellipsoid.
716
00:47:32,830 --> 00:47:35,660
So -- you see what I'm saying,
if we have a sphere then all
717
00:47:35,660 --> 00:47:36,890
directions would be the same.
718
00:47:39,530 --> 00:47:44,750
If we had a true football, or
it's closer to a rugby ball,
719
00:47:44,750 --> 00:47:48,110
really, because it's more
curved than a football,
720
00:47:48,110 --> 00:47:52,270
it would have one long
direction and the other two
721
00:47:52,270 --> 00:47:54,020
would be equal.
722
00:47:54,020 --> 00:47:56,240
That would be like
having a matrix
723
00:47:56,240 --> 00:47:59,700
that had one
eigenvalue repeated.
724
00:47:59,700 --> 00:48:02,080
And then one other different.
725
00:48:02,080 --> 00:48:04,940
So this sphere comes from,
like, the identity matrix,
726
00:48:04,940 --> 00:48:07,600
all eigenvalues the same.
727
00:48:07,600 --> 00:48:12,370
Our rugby ball comes
from a case where --
728
00:48:12,370 --> 00:48:17,070
three, the three, two of the
three eigenvalues are the same.
729
00:48:17,070 --> 00:48:20,000
But we know how the case
where -- the typical case,
730
00:48:20,000 --> 00:48:23,340
where the three eigenvalues
were all different.
731
00:48:23,340 --> 00:48:25,551
So this will have --
732
00:48:28,440 --> 00:48:31,830
How do I say it, if I look
at this ellipsoid correctly,
733
00:48:31,830 --> 00:48:37,200
it'll have a major axis,
it'll have a middle axis,
734
00:48:37,200 --> 00:48:41,520
and it'll have a minor axis.
735
00:48:41,520 --> 00:48:44,940
And those three axes
will be in the direction
736
00:48:44,940 --> 00:48:47,450
of the eigenvectors.
737
00:48:47,450 --> 00:48:50,760
And the lengths of
those axes will be
738
00:48:50,760 --> 00:48:52,496
determined by the eigenvalues.
739
00:48:55,260 --> 00:48:56,770
I can get --
740
00:48:56,770 --> 00:49:02,520
turn this all into linear
algebra, because we have --
741
00:49:02,520 --> 00:49:06,410
the right thing we know about
eigenvectors and eigenvalues,
742
00:49:06,410 --> 00:49:08,120
for that matrix is what?
743
00:49:08,120 --> 00:49:11,970
Of -- let me just tell you that,
repeat the main linear algebra
744
00:49:11,970 --> 00:49:13,070
point.
745
00:49:13,070 --> 00:49:18,970
How could we turn what
I said into algebra;
746
00:49:18,970 --> 00:49:25,050
we would write this A as
Q, the eigenvector matrix,
747
00:49:25,050 --> 00:49:30,370
times lambda, the eigenvalue
matrix times Q transposed.
748
00:49:30,370 --> 00:49:33,010
The principal axis
theorem, we'll call it,
749
00:49:33,010 --> 00:49:33,900
now.
750
00:49:33,900 --> 00:49:37,980
The eigenvectors tell us the
directions of the principal
751
00:49:37,980 --> 00:49:39,110
axes.
752
00:49:39,110 --> 00:49:43,240
The eigenvalues tell us the
lengths of those axes, actually
753
00:49:43,240 --> 00:49:45,160
the lengths, or
the half-lengths,
754
00:49:45,160 --> 00:49:48,970
or one over the
eigenvalues, it turns out.
755
00:49:48,970 --> 00:49:53,260
And that is the
matrix factorization
756
00:49:53,260 --> 00:49:55,490
which is the most
important matrix
757
00:49:55,490 --> 00:50:00,540
factorization in our
eigenvalue material so far.
758
00:50:00,540 --> 00:50:04,320
That's diagonalization
for a symmetric matrix,
759
00:50:04,320 --> 00:50:09,350
so instead of the inverse
I can write the transposed.
760
00:50:09,350 --> 00:50:09,930
OK.
761
00:50:09,930 --> 00:50:14,140
I've -- so what I've tried
today is to tell you the --
762
00:50:14,140 --> 00:50:20,510
what's going on with
positive definite matrices.
763
00:50:20,510 --> 00:50:23,020
Ah, you see all how all
these pieces are there
764
00:50:23,020 --> 00:50:25,721
and linear algebra
connects them.
765
00:50:25,721 --> 00:50:26,220
OK.
766
00:50:28,760 --> 00:50:30,680
See you on Friday.