1
00:00:00,000 --> 00:00:00,030
2
00:00:00,030 --> 00:00:02,330
The following content is
provided under a Creative
3
00:00:02,330 --> 00:00:03,610
Commons license.
4
00:00:03,610 --> 00:00:05,770
Your support will help
MIT OpenCourseWare
5
00:00:05,770 --> 00:00:09,930
continue to offer high quality
educational resources for free.
6
00:00:09,930 --> 00:00:12,530
To make a donation or to
view additional materials
7
00:00:12,530 --> 00:00:15,620
from hundreds of MIT courses
visit MIT OpenCourseWare
8
00:00:15,620 --> 00:00:20,920
at ocw.mit.edu.
9
00:00:20,920 --> 00:00:25,670
PROFESSOR STRANG: Finally we get
to positive definite matrices.
10
00:00:25,670 --> 00:00:29,180
I've used the word and now
it's time to pin it down.
11
00:00:29,180 --> 00:00:33,620
And so this would be my
thank you for staying with it
12
00:00:33,620 --> 00:00:37,490
while we do this important
preliminary stuff
13
00:00:37,490 --> 00:00:39,780
about linear algebra.
14
00:00:39,780 --> 00:00:41,850
So starting the
next lecture we'll
15
00:00:41,850 --> 00:00:46,660
really make a big start on
engineering applications.
16
00:00:46,660 --> 00:00:51,250
But these matrices are going
to be the key to everything.
17
00:00:51,250 --> 00:00:58,290
And I'll call these matrices
K. And positive definite,
18
00:00:58,290 --> 00:01:03,200
I will only use that word
about a symmetric matrix.
19
00:01:03,200 --> 00:01:07,130
So the matrix is
already symmetric
20
00:01:07,130 --> 00:01:09,730
and that means it
has real eigenvalues
21
00:01:09,730 --> 00:01:12,820
and many other
important properties,
22
00:01:12,820 --> 00:01:14,930
orthogonal eigenvectors.
23
00:01:14,930 --> 00:01:17,960
And now we're asking for more.
24
00:01:17,960 --> 00:01:25,630
And it's that extra
bit that is terrific
25
00:01:25,630 --> 00:01:28,260
in all kinds of applications.
26
00:01:28,260 --> 00:01:31,090
So if I can do this
bit of linear algebra.
27
00:01:31,090 --> 00:01:38,880
So what's coming then, my review
session this afternoon at four,
28
00:01:38,880 --> 00:01:41,460
I'm very happy that
we've got, I think,
29
00:01:41,460 --> 00:01:47,150
the best MATLAB problem ever
invented, in 18.085 anyway.
30
00:01:47,150 --> 00:01:51,410
So that should get onto the
website probably by tomorrow.
31
00:01:51,410 --> 00:01:55,670
And Peter Buchak is
like the MATLAB person.
32
00:01:55,670 --> 00:01:59,590
So his review sessions
are Friday at noon.
33
00:01:59,590 --> 00:02:02,890
And I just saw him and
suggested Friday at noon
34
00:02:02,890 --> 00:02:06,750
he might as well
just stay in here.
35
00:02:06,750 --> 00:02:09,740
And knowing that
that isn't maybe
36
00:02:09,740 --> 00:02:11,050
a good hour for everybody.
37
00:02:11,050 --> 00:02:16,390
So you could see him also
outside of that hour.
38
00:02:16,390 --> 00:02:19,670
But that's the hour
he will be ready
39
00:02:19,670 --> 00:02:23,520
for all kinds of questions about
MATLAB or about the homeworks.
40
00:02:23,520 --> 00:02:28,720
Actually you'll be probably
thinking more about, also
41
00:02:28,720 --> 00:02:35,680
about the homework
questions on this topic.
42
00:02:35,680 --> 00:02:40,910
Ready for positive definite?
43
00:02:40,910 --> 00:02:43,750
You said yes, right?
44
00:02:43,750 --> 00:02:50,180
And you have a hint
about these things.
45
00:02:50,180 --> 00:02:54,880
So we have a symmetric
matrix and the beauty
46
00:02:54,880 --> 00:02:58,190
is that it brings together
all of linear algebra.
47
00:02:58,190 --> 00:03:01,670
Including elimination,
that's when we see pivots.
48
00:03:01,670 --> 00:03:05,930
Including determinants which are
closely related to the pivots.
49
00:03:05,930 --> 00:03:08,180
And what do I mean
by upper left?
50
00:03:08,180 --> 00:03:13,310
I mean that if I have a three
by three symmetric matrix
51
00:03:13,310 --> 00:03:16,350
and I want to test it
for positive definite,
52
00:03:16,350 --> 00:03:18,690
and I guess actually
this would be the easiest
53
00:03:18,690 --> 00:03:23,140
test if I had a tiny
matrix, three by three,
54
00:03:23,140 --> 00:03:27,120
and I had numbers, then
this would be a good test.
55
00:03:27,120 --> 00:03:31,413
The determinants-- By upper left
determinants I mean that one
56
00:03:31,413 --> 00:03:33,170
by one determinant.
57
00:03:33,170 --> 00:03:36,220
So just that first number
has to be positive.
58
00:03:36,220 --> 00:03:39,900
Then the two by two determinant,
that times that minus that
59
00:03:39,900 --> 00:03:42,230
times that has to be positive.
60
00:03:42,230 --> 00:03:44,420
Oh I've already
been saying that.
61
00:03:44,420 --> 00:03:46,290
Let me just put in some letters.
62
00:03:46,290 --> 00:03:48,100
So a has to be positive.
63
00:03:48,100 --> 00:03:51,620
This is symmetric,
so a times c has
64
00:03:51,620 --> 00:03:54,310
to be bigger than b squared.
65
00:03:54,310 --> 00:03:57,340
So that will tell us.
66
00:03:57,340 --> 00:03:59,230
And then for two
by two we finish.
67
00:03:59,230 --> 00:04:03,256
For three by three we would
also require the three
68
00:04:03,256 --> 00:04:05,880
by three determinant
to be positive.
69
00:04:05,880 --> 00:04:08,440
But already here
you're seeing one point
70
00:04:08,440 --> 00:04:11,470
about a positive
definite matrix.
71
00:04:11,470 --> 00:04:14,430
Its diagonal will
have to be positive.
72
00:04:14,430 --> 00:04:20,020
And somehow its diagonal has
to be not just above zero,
73
00:04:20,020 --> 00:04:25,140
but somehow it has
to defeat b squared.
74
00:04:25,140 --> 00:04:30,690
So the diagonal has to
be somehow more positive
75
00:04:30,690 --> 00:04:33,890
than whatever
negative stuff might
76
00:04:33,890 --> 00:04:35,780
come from off the diagonal.
77
00:04:35,780 --> 00:04:41,250
That's why I would need
a*c > b squared.
78
00:04:41,250 --> 00:04:43,120
So both of those
will be positive
79
00:04:43,120 --> 00:04:48,600
and their product has to be
bigger than the other guy.
80
00:04:48,600 --> 00:04:52,650
And then finally, a third test
is that all the eigenvalues
81
00:04:52,650 --> 00:04:53,970
are positive.
82
00:04:53,970 --> 00:04:56,500
And of course if I give you
a three by three matrix,
83
00:04:56,500 --> 00:04:58,340
that's probably not
the easiest test
84
00:04:58,340 --> 00:05:00,670
since you'd have to
find the eigenvalues.
85
00:05:00,670 --> 00:05:05,490
Much easier to find the
determinants or the pivots.
86
00:05:05,490 --> 00:05:10,400
Actually, just while I'm at it,
so the first pivot of course
87
00:05:10,400 --> 00:05:12,170
is a itself.
88
00:05:12,170 --> 00:05:15,140
No difficulty there.
89
00:05:15,140 --> 00:05:19,580
The second pivot turns out
to be the ratio of a*c -
90
00:05:19,580 --> 00:05:22,140
b squared to a.
91
00:05:22,140 --> 00:05:25,840
So the connection between
pivots and determinants
92
00:05:25,840 --> 00:05:28,210
is just really close.
93
00:05:28,210 --> 00:05:30,320
Pivots are ratios
of determinants,
94
00:05:30,320 --> 00:05:31,250
if you work it out.
95
00:05:31,250 --> 00:05:34,930
The second pivot, maybe
I would call that d_2,
96
00:05:34,930 --> 00:05:40,110
is the ratio of a*c
- b squared over a.
97
00:05:40,110 --> 00:05:43,040
In other words it's
c minus b^2 / a.
98
00:05:43,040 --> 00:05:48,250
99
00:05:48,250 --> 00:05:51,790
Determinants are
positive and vice versa.
100
00:05:51,790 --> 00:05:54,130
Then it's fantastic
that the eigenvalues
101
00:05:54,130 --> 00:05:56,740
come into the picture.
102
00:05:56,740 --> 00:06:00,510
So those are three ways,
three important properties
103
00:06:00,510 --> 00:06:02,870
of a positive definite matrix.
104
00:06:02,870 --> 00:06:08,770
But I'd like to make the
definition something different.
105
00:06:08,770 --> 00:06:11,200
Now I'm coming to the meaning.
106
00:06:11,200 --> 00:06:14,420
If I think of those as
the tests, that's done.
107
00:06:14,420 --> 00:06:24,370
Now the meaning of
positive definite.
108
00:06:24,370 --> 00:06:27,230
The meaning of positive
definite and the applications
109
00:06:27,230 --> 00:06:32,540
are closely related to
looking for a minimum.
110
00:06:32,540 --> 00:06:39,900
And so what I'm going to bring
in here-- So it's symmetric.
111
00:06:39,900 --> 00:06:47,610
Now for a symmetric matrix I
want to introduce the energy.
112
00:06:47,610 --> 00:06:51,580
So this is the reason why
it has so many applications
113
00:06:51,580 --> 00:06:54,750
and such important
physical meaning is
114
00:06:54,750 --> 00:06:56,910
that what I'm
about to introduce,
115
00:06:56,910 --> 00:07:01,600
which is a function
of x, and here
116
00:07:01,600 --> 00:07:07,940
it is, it's x transpose
times A, not A,
117
00:07:07,940 --> 00:07:16,770
I'm sticking with K
for my matrix, times x.
118
00:07:16,770 --> 00:07:20,870
I think of that as some f(x).
119
00:07:20,870 --> 00:07:24,430
And let's just see what it
would be if the matrix was
120
00:07:24,430 --> 00:07:29,510
two by two, [a, b; b, c].
121
00:07:29,510 --> 00:07:33,330
Suppose that's my matrix.
122
00:07:33,330 --> 00:07:36,770
We want to get a handle on
what, this is the first time
123
00:07:36,770 --> 00:07:42,040
I've ever written something
that has x's times x's.
124
00:07:42,040 --> 00:07:45,190
So it's going to be quadratic.
125
00:07:45,190 --> 00:07:48,330
They're going to
be x's times x's.
126
00:07:48,330 --> 00:07:52,190
And x is a general
vector of the right size
127
00:07:52,190 --> 00:07:55,100
so it's got components x_1, x_2.
128
00:07:55,100 --> 00:07:57,850
And there it's
transposed, so it's a row.
129
00:07:57,850 --> 00:08:01,920
And now I put in
the [a, b; b, c].
130
00:08:01,920 --> 00:08:05,680
And then I put in x again.
131
00:08:05,680 --> 00:08:09,090
This is going to give me a
very nice, simple, important
132
00:08:09,090 --> 00:08:11,830
expression.
133
00:08:11,830 --> 00:08:14,470
Depending on x_1 and x_2.
134
00:08:14,470 --> 00:08:18,090
Now what is, can we do
that multiplication?
135
00:08:18,090 --> 00:08:25,210
Maybe above I'll do the
multiplication of this pair
136
00:08:25,210 --> 00:08:28,370
and then I have the
other guy to bring in.
137
00:08:28,370 --> 00:08:31,230
So here, that
would be ax_1+bx_2.
138
00:08:31,230 --> 00:08:35,040
139
00:08:35,040 --> 00:08:36,850
And this would be bx_1+cx_2.
140
00:08:36,850 --> 00:08:39,550
141
00:08:39,550 --> 00:08:44,722
So that's the first,
that's this times this.
142
00:08:44,722 --> 00:08:45,680
What am I going to get?
143
00:08:45,680 --> 00:08:48,910
What shape, what size is
this result going to be?
144
00:08:48,910 --> 00:08:56,140
This K is n by n. x is a column
vector. n by one. x transpose,
145
00:08:56,140 --> 00:08:58,800
what's the shape of x transpose?
146
00:08:58,800 --> 00:09:00,070
One by n?
147
00:09:00,070 --> 00:09:02,400
So what's the total result?
148
00:09:02,400 --> 00:09:03,100
One by one.
149
00:09:03,100 --> 00:09:04,270
Just a number.
150
00:09:04,270 --> 00:09:05,070
Just a function.
151
00:09:05,070 --> 00:09:06,560
It's a number.
152
00:09:06,560 --> 00:09:11,450
But it depends on the x's
and the matrix inside.
153
00:09:11,450 --> 00:09:12,830
Can we do it now?
154
00:09:12,830 --> 00:09:16,280
So I've got this to
multiply by this.
155
00:09:16,280 --> 00:09:19,900
Do you see an x_1
squared showing up?
156
00:09:19,900 --> 00:09:21,590
Yes, from there times there.
157
00:09:21,590 --> 00:09:24,770
And what's it multiplied by?
158
00:09:24,770 --> 00:09:26,260
The a.
159
00:09:26,260 --> 00:09:30,830
The first term is this times
the ax_1 is a x_1 squared.
160
00:09:30,830 --> 00:09:33,480
So that's our first quadratic.
161
00:09:33,480 --> 00:09:36,360
Now there'd be an x_1, x_2.
162
00:09:36,360 --> 00:09:39,820
Let me leave that for a minute
and find the x_2 squared
163
00:09:39,820 --> 00:09:41,350
because it's easy.
164
00:09:41,350 --> 00:09:43,440
So where am I going
to get x_2 squared?
165
00:09:43,440 --> 00:09:46,650
I'm going to get that
from x_2, second guy
166
00:09:46,650 --> 00:09:49,150
here times second guy here.
167
00:09:49,150 --> 00:09:54,830
There's a c x_2 squared.
168
00:09:54,830 --> 00:09:58,630
So you're seeing already
where the diagonal shows up.
169
00:09:58,630 --> 00:10:02,330
The diagonal a, c,
whatever, is multiplying
170
00:10:02,330 --> 00:10:04,600
the perfect squares.
171
00:10:04,600 --> 00:10:08,820
And it'll be the off-diagonal
that multiplies the x_1 x_2.
172
00:10:08,820 --> 00:10:11,210
We might call those
the cross terms.
173
00:10:11,210 --> 00:10:13,160
And what do we
get for that then?
174
00:10:13,160 --> 00:10:16,180
We have x_1 times this guy.
175
00:10:16,180 --> 00:10:20,200
So that's a cross
term. bx_1*x_2, right?
176
00:10:20,200 --> 00:10:24,270
And here's another one coming
from x_2 times this guy.
177
00:10:24,270 --> 00:10:27,780
And what's that one?
178
00:10:27,780 --> 00:10:30,080
It's also bx_1*x_2.
179
00:10:30,080 --> 00:10:33,750
So x_1, multiply that,
x_2 multiply that,
180
00:10:33,750 --> 00:10:37,210
and so what do we have
for this cross term here?
181
00:10:37,210 --> 00:10:39,600
Two of them.
182
00:10:39,600 --> 00:10:40,670
2bx_1*x_2.
183
00:10:40,670 --> 00:10:43,350
184
00:10:43,350 --> 00:10:48,160
In other words, that b
and that b came together
185
00:10:48,160 --> 00:10:50,490
in the 2bx_1*x_2.
186
00:10:50,490 --> 00:10:56,280
So here's my energy.
187
00:10:56,280 --> 00:10:58,400
Can I just loosely
call it energy?
188
00:10:58,400 --> 00:11:02,960
And then as we get to
applications we'll see why.
189
00:11:02,960 --> 00:11:06,400
So I'm interested
in that because it
190
00:11:06,400 --> 00:11:12,390
has important meaning.
191
00:11:12,390 --> 00:11:16,770
Well, so now I'm ready to define
positive definite matrices.
192
00:11:16,770 --> 00:11:19,690
So I'll call that number four.
193
00:11:19,690 --> 00:11:23,860
But I'm going to
give it a big star.
194
00:11:23,860 --> 00:11:29,670
Even more because
it's the sort of key.
195
00:11:29,670 --> 00:11:34,010
So the test will be, you
can probably guess it,
196
00:11:34,010 --> 00:11:41,630
I look at this expression,
that x transpose Ax.
197
00:11:41,630 --> 00:11:45,250
And if it's a positive
definite matrix
198
00:11:45,250 --> 00:11:47,340
and this represents
energy, the key
199
00:11:47,340 --> 00:11:50,750
will be that this
should be positive.
200
00:11:50,750 --> 00:11:57,120
This one should be
positive for all x's.
201
00:11:57,120 --> 00:11:59,700
Well, with one
exception, of course.
202
00:11:59,700 --> 00:12:07,530
All x's except, which vector
is it? x=0 would just give me--
203
00:12:07,530 --> 00:12:17,390
See, I put K. My default for a
matrix, but should be, it's K.
204
00:12:17,390 --> 00:12:22,340
Except x=0, except
the zero vector.
205
00:12:22,340 --> 00:12:23,730
Of course.
206
00:12:23,730 --> 00:12:29,940
If x_1 and x_2 are both zero.
207
00:12:29,940 --> 00:12:34,720
Now that looks a little
maybe less straightforward,
208
00:12:34,720 --> 00:12:37,760
I would say, because it's
a statement about this
209
00:12:37,760 --> 00:12:41,940
is true for all x_1 and x_2.
210
00:12:41,940 --> 00:12:44,760
And we better do some
examples and draw a picture.
211
00:12:44,760 --> 00:12:51,670
Let me draw a
picture right away.
212
00:12:51,670 --> 00:12:54,750
So here's x_1 direction.
213
00:12:54,750 --> 00:12:56,420
Here's x_2 direction.
214
00:12:56,420 --> 00:13:05,960
And here is the x
transpose Ax, my function.
215
00:13:05,960 --> 00:13:09,160
So this depends
on two variables.
216
00:13:09,160 --> 00:13:13,370
So it's going to be a sort
of a surface if I draw it.
217
00:13:13,370 --> 00:13:16,650
Now, what point do
we absolutely know?
218
00:13:16,650 --> 00:13:21,050
And I put A again.
219
00:13:21,050 --> 00:13:29,560
I am so sorry.
220
00:13:29,560 --> 00:13:30,840
Well, we know one point.
221
00:13:30,840 --> 00:13:34,660
It's there whatever
that matrix might be.
222
00:13:34,660 --> 00:13:35,680
It's there.
223
00:13:35,680 --> 00:13:37,240
Zero, right?
224
00:13:37,240 --> 00:13:40,300
You just told me that if
both x's are zero then
225
00:13:40,300 --> 00:13:42,760
we automatically get zero.
226
00:13:42,760 --> 00:13:47,100
Now what do you think
the shape of this curve,
227
00:13:47,100 --> 00:13:51,280
the shape of this graph
is going to look like?
228
00:13:51,280 --> 00:13:54,920
The point is, if we're
positive definite now.
229
00:13:54,920 --> 00:13:58,980
So I'm drawing the picture
for positive definite.
230
00:13:58,980 --> 00:14:04,610
So my definition is
that the energy goes up.
231
00:14:04,610 --> 00:14:06,650
It's positive, right?
232
00:14:06,650 --> 00:14:10,320
When I leave, when I move away
from that point I go upwards.
233
00:14:10,320 --> 00:14:13,540
That point will be a minimum.
234
00:14:13,540 --> 00:14:17,740
Let me just draw it roughly.
235
00:14:17,740 --> 00:14:23,500
So it sort of goes up like this.
236
00:14:23,500 --> 00:14:30,670
These cheap 2-D boards and I've
got a three-dimensional picture
237
00:14:30,670 --> 00:14:34,370
here.
238
00:14:34,370 --> 00:14:35,780
But you see it somehow?
239
00:14:35,780 --> 00:14:40,300
What word or what's
your visualization?
240
00:14:40,300 --> 00:14:43,100
It has a minimum there.
241
00:14:43,100 --> 00:14:45,840
That's why minimization,
which was like,
242
00:14:45,840 --> 00:14:49,140
the core problem in
calculus, is here now.
243
00:14:49,140 --> 00:14:54,870
But for functions
of two x's or n x's.
244
00:14:54,870 --> 00:14:58,930
We're up the dimension over
the basic minimum problem
245
00:14:58,930 --> 00:15:03,730
of calculus.
246
00:15:03,730 --> 00:15:05,260
It's sort of like a parabola.
247
00:15:05,260 --> 00:15:07,960
Its cross-sections cutting
down through the thing
248
00:15:07,960 --> 00:15:10,550
would be just parabolas
because of the x squared.
249
00:15:10,550 --> 00:15:12,890
I'm going to call this a bowl.
250
00:15:12,890 --> 00:15:16,050
It's a short word.
251
00:15:16,050 --> 00:15:16,840
Do you see it?
252
00:15:16,840 --> 00:15:18,280
It opens up.
253
00:15:18,280 --> 00:15:20,870
That's the key point,
that it opens upward.
254
00:15:20,870 --> 00:15:22,920
And let's do some examples.
255
00:15:22,920 --> 00:15:26,000
Tell me some positive definite.
256
00:15:26,000 --> 00:15:29,170
So positive definite
and then let
257
00:15:29,170 --> 00:15:35,910
me here put some not
positive definite cases.
258
00:15:35,910 --> 00:15:38,360
Tell me a matrix.
259
00:15:38,360 --> 00:15:41,050
Well, what's the
easiest, first matrix
260
00:15:41,050 --> 00:15:44,710
that occurs to you as a
positive definite matrix?
261
00:15:44,710 --> 00:15:49,230
The identity.
262
00:15:49,230 --> 00:15:52,090
That passes all our tests,
its eigenvalues are one,
263
00:15:52,090 --> 00:15:54,890
its pivots are one, the
determinants are one.
264
00:15:54,890 --> 00:15:58,830
And the function is
x_1 squared plus x_2
265
00:15:58,830 --> 00:16:05,520
squared with no b in it.
266
00:16:05,520 --> 00:16:08,590
It's just a perfect bowl,
perfectly symmetric,
267
00:16:08,590 --> 00:16:12,190
the way it would come
off a potter's wheel.
268
00:16:12,190 --> 00:16:16,190
Now let me take one
that's maybe not so,
269
00:16:16,190 --> 00:16:18,410
let me put a nine there.
270
00:16:18,410 --> 00:16:20,310
So I'm off to a
reasonable start.
271
00:16:20,310 --> 00:16:24,090
I have an x_1 squared
and a nine x_2 squared.
272
00:16:24,090 --> 00:16:25,890
And now I want to
ask you, what could I
273
00:16:25,890 --> 00:16:30,410
put in there that would
leave it positive definite?
274
00:16:30,410 --> 00:16:33,430
Well, give me a couple
of possibilities.
275
00:16:33,430 --> 00:16:37,800
What's a nice, not too
big now, that's the thing.
276
00:16:37,800 --> 00:16:38,550
Two.
277
00:16:38,550 --> 00:16:39,850
Two would be fine.
278
00:16:39,850 --> 00:16:42,540
So if I had a two there and
a two there I would have
279
00:16:42,540 --> 00:16:47,000
a 4x_1*x_2 and it
would, like, this,
280
00:16:47,000 --> 00:16:54,720
instead of being a circle,
which it was for the identity,
281
00:16:54,720 --> 00:16:59,230
the plane there would cut
out a ellipse instead.
282
00:16:59,230 --> 00:17:02,750
But it would be a good ellipse.
283
00:17:02,750 --> 00:17:05,730
Because we're doing
squares, we're really,
284
00:17:05,730 --> 00:17:08,960
the Greeks understood
these second degree things
285
00:17:08,960 --> 00:17:18,540
and they would have known this
would have been an ellipse.
286
00:17:18,540 --> 00:17:23,070
How high can I go with that
two or where do I have to stop?
287
00:17:23,070 --> 00:17:27,660
Where would I have to, if
I wanted to change the two,
288
00:17:27,660 --> 00:17:33,080
let me just focus on that one,
suppose I wanted to change it.
289
00:17:33,080 --> 00:17:36,510
First of all, give
me one that's,
290
00:17:36,510 --> 00:17:38,900
how about the borderline.
291
00:17:38,900 --> 00:17:40,610
Three would be the borderline.
292
00:17:40,610 --> 00:17:41,410
Why's that?
293
00:17:41,410 --> 00:17:48,100
Because at three we have nine
minus nine for the determinant.
294
00:17:48,100 --> 00:17:51,500
So the determinant is zero.
295
00:17:51,500 --> 00:17:53,350
Of course it passed
the first test.
296
00:17:53,350 --> 00:17:54,920
One by one was okay.
297
00:17:54,920 --> 00:18:03,840
But two by two was not,
was at the borderline.
298
00:18:03,840 --> 00:18:07,850
What else should I think?
299
00:18:07,850 --> 00:18:11,230
Oh, that's a very
interesting case.
300
00:18:11,230 --> 00:18:13,320
The borderline.
301
00:18:13,320 --> 00:18:15,660
You know, it almost makes it.
302
00:18:15,660 --> 00:18:21,860
But can you tell me the
eigenvalues of that matrix?
303
00:18:21,860 --> 00:18:25,100
Don't do any
quadratic equations.
304
00:18:25,100 --> 00:18:28,220
How do I know, what's one
eigenvalue of a matrix?
305
00:18:28,220 --> 00:18:30,330
You made it singular, right?
306
00:18:30,330 --> 00:18:31,770
You made that matrix singular.
307
00:18:31,770 --> 00:18:32,730
Determinant zero.
308
00:18:32,730 --> 00:18:36,030
So one of its
eigenvalues is zero.
309
00:18:36,030 --> 00:18:40,180
And the other one is visible
by looking at the trace.
310
00:18:40,180 --> 00:18:44,090
I just quickly mentioned
that if I add the diagonal,
311
00:18:44,090 --> 00:18:47,850
I get the same answer as if
I add the two eigenvalues.
312
00:18:47,850 --> 00:18:51,180
So that other
eigenvalue must be ten.
313
00:18:51,180 --> 00:18:54,940
And this is entirely
typical, that ten and zero,
314
00:18:54,940 --> 00:18:58,640
the extreme eigenvalues,
lambda_max and lambda_min,
315
00:18:58,640 --> 00:19:04,300
are bigger than-- these
diagonal guys are inside.
316
00:19:04,300 --> 00:19:08,180
They're inside,
between zero and ten
317
00:19:08,180 --> 00:19:12,580
and it's these terms that
enter somehow and gave us
318
00:19:12,580 --> 00:19:15,880
an eigenvalue of ten and
an eigenvalue of zero.
319
00:19:15,880 --> 00:19:20,320
I guess I'm tempted to
try to draw that figure.
320
00:19:20,320 --> 00:19:25,560
Just to get a feeling
of what's with that one.
321
00:19:25,560 --> 00:19:30,140
It always helps to get
the borderline case.
322
00:19:30,140 --> 00:19:32,320
So what's with this one?
323
00:19:32,320 --> 00:19:35,630
Let me see what my
quadratic would be.
324
00:19:35,630 --> 00:19:37,760
Can I just change it up here?
325
00:19:37,760 --> 00:19:38,970
Rather than rewriting it.
326
00:19:38,970 --> 00:19:42,000
So I'm going to,
I'll put it up here.
327
00:19:42,000 --> 00:19:46,240
So I have to change
that four to what?
328
00:19:46,240 --> 00:19:48,810
Now that I'm looking
at this matrix.
329
00:19:48,810 --> 00:19:51,960
That four is now a six.
330
00:19:51,960 --> 00:19:53,900
Six.
331
00:19:53,900 --> 00:19:56,880
This is my guy for this one.
332
00:19:56,880 --> 00:19:58,862
Which is not positive definite.
333
00:19:58,862 --> 00:20:00,320
Let me tell you
right away the word
334
00:20:00,320 --> 00:20:02,260
that I would use for this one.
335
00:20:02,260 --> 00:20:06,820
I would call it positive
semi-definite because it's
336
00:20:06,820 --> 00:20:09,500
almost there, but not quite.
337
00:20:09,500 --> 00:20:15,300
So semi-definite allows
the matrix to be singular.
338
00:20:15,300 --> 00:20:18,180
So semi-definite,
maybe I'll do it
339
00:20:18,180 --> 00:20:22,350
in green what
semi-definite would be.
340
00:20:22,350 --> 00:20:31,700
Semi-def would be eigenvalues
greater than or equal zero.
341
00:20:31,700 --> 00:20:35,160
Determinants greater
than or equal zero.
342
00:20:35,160 --> 00:20:39,440
Pivots greater than zero
if they're there or then
343
00:20:39,440 --> 00:20:41,410
we run out of pivots.
344
00:20:41,410 --> 00:20:44,490
You could say greater than
or equal to zero then.
345
00:20:44,490 --> 00:20:48,740
And energy, greater
than or equal to zero
346
00:20:48,740 --> 00:20:53,110
for semi-definite.
347
00:20:53,110 --> 00:20:58,380
And when would the energy, what
x's, what would be the like,
348
00:20:58,380 --> 00:21:01,090
you could say the ground
states or something,
349
00:21:01,090 --> 00:21:04,420
what x's-- So greater
than or equal to zero,
350
00:21:04,420 --> 00:21:08,080
emphasize that possibility
of equal in the semi-definite
351
00:21:08,080 --> 00:21:10,100
case.
352
00:21:10,100 --> 00:21:17,300
Suppose I have a semi-definite
matrix, yeah, I've got one.
353
00:21:17,300 --> 00:21:19,690
But it's singular.
354
00:21:19,690 --> 00:21:26,160
So that means a singular matrix
takes some vector x to zero.
355
00:21:26,160 --> 00:21:27,700
Right?
356
00:21:27,700 --> 00:21:30,080
If my matrix is
actually singular,
357
00:21:30,080 --> 00:21:32,990
then there'll be an
x where Kx is zero.
358
00:21:32,990 --> 00:21:35,000
And then, of course,
multiplying by x transpose,
359
00:21:35,000 --> 00:21:36,350
I'm still at zero.
360
00:21:36,350 --> 00:21:41,900
So the x's, the zero energy
guys, this is straightforward,
361
00:21:41,900 --> 00:21:49,180
the zero energy guys, the ones
where x transpose Kx is zero,
362
00:21:49,180 --> 00:21:56,730
will happen when Kx is zero.
363
00:21:56,730 --> 00:22:03,210
If Kx is zero, and we'll
see it in that example.
364
00:22:03,210 --> 00:22:05,140
Let's see it in that example.
365
00:22:05,140 --> 00:22:12,300
What's the x for
which, I could say
366
00:22:12,300 --> 00:22:15,170
in the null space,
what's the vector x
367
00:22:15,170 --> 00:22:18,830
that that matrix kills?
368
00:22:18,830 --> 00:22:21,490
369
00:22:21,490 --> 00:22:23,146
[3, -1], right?
370
00:22:23,146 --> 00:22:24,770
The vector [3, -1].
371
00:22:24,770 --> 00:22:30,600
[3, -1] gives me [0, 0].
372
00:22:30,600 --> 00:22:35,360
That's the vector that-- So
I get 3-3, the zero, 9-9,
373
00:22:35,360 --> 00:22:36,710
the zero.
374
00:22:36,710 --> 00:22:40,700
So I believe that
this thing will
375
00:22:40,700 --> 00:22:44,750
be-- Is it zero at [3, -1]?
376
00:22:44,750 --> 00:22:47,530
I think that it
has to be, right?
377
00:22:47,530 --> 00:22:52,040
If I take x_1 to be three
and x_2 to be minus one,
378
00:22:52,040 --> 00:22:54,400
I think I've got
zero energy here.
379
00:22:54,400 --> 00:22:59,630
Do I? x_1 squared will be
nine and nine x_2 squared
380
00:22:59,630 --> 00:23:01,210
will be nine more.
381
00:23:01,210 --> 00:23:04,160
And what will be this 6x_1*x_2?
382
00:23:04,160 --> 00:23:09,250
What will that come out
for this x_1 and x_2?
383
00:23:09,250 --> 00:23:10,530
Minus 18.
384
00:23:10,530 --> 00:23:11,950
Had to, right?
385
00:23:11,950 --> 00:23:15,620
So I'd get nine from there,
nine from there, minus 18, zero.
386
00:23:15,620 --> 00:23:18,660
So the graph for this
positive semi-definite
387
00:23:18,660 --> 00:23:21,050
will look a bit like this.
388
00:23:21,050 --> 00:23:26,060
There'll be a direction
in which it doesn't climb.
389
00:23:26,060 --> 00:23:29,560
It doesn't go below
the base, right?
390
00:23:29,560 --> 00:23:31,340
It's never negative.
391
00:23:31,340 --> 00:23:33,190
This is now the
semi-definite picture.
392
00:23:33,190 --> 00:23:36,260
But it can run along the base.
393
00:23:36,260 --> 00:23:40,390
And it will for the
vector x_1=3, x_2=-1,
394
00:23:40,390 --> 00:23:42,520
I don't know where that
is, one, two, three,
395
00:23:42,520 --> 00:23:45,910
and then maybe minus one.
396
00:23:45,910 --> 00:23:51,520
Along some line here
the graph doesn't go up.
397
00:23:51,520 --> 00:23:56,940
It's sitting, can you imagine
that sitting in the base?
398
00:23:56,940 --> 00:24:05,870
I'm not Rembrandt here, but in
the other direction it goes up.
399
00:24:05,870 --> 00:24:08,240
Oh, the hell with that one.
400
00:24:08,240 --> 00:24:10,650
Do you see, sort of?
401
00:24:10,650 --> 00:24:14,270
It's like a trough,
would you say?
402
00:24:14,270 --> 00:24:16,740
I mean, it's like
a, you know, a bit
403
00:24:16,740 --> 00:24:19,720
of a drainpipe or something.
404
00:24:19,720 --> 00:24:27,300
It's running along the ground,
along this [3, -1] direction
405
00:24:27,300 --> 00:24:30,180
and in the other
directions it does go up.
406
00:24:30,180 --> 00:24:35,480
So it's shaped like this
with the base not climbing.
407
00:24:35,480 --> 00:24:39,020
Whereas here, there's
no bad direction.
408
00:24:39,020 --> 00:24:40,900
Climbs every way you go.
409
00:24:40,900 --> 00:24:45,440
So that's positive definite and
that's positive semi-definite.
410
00:24:45,440 --> 00:24:50,770
Well suppose I push
it a little further.
411
00:24:50,770 --> 00:24:56,510
Let me make a place here
for a matrix that isn't
412
00:24:56,510 --> 00:25:01,350
even positive semi-definite.
413
00:25:01,350 --> 00:25:05,180
Now it's just going
to go down somewhere.
414
00:25:05,180 --> 00:25:07,010
I'll start again
with one and nine
415
00:25:07,010 --> 00:25:09,850
and tell me what to put in now.
416
00:25:09,850 --> 00:25:13,480
So this is going to be a case
where the off-diagonal is
417
00:25:13,480 --> 00:25:15,740
too big, it wins.
418
00:25:15,740 --> 00:25:18,160
And prevents positive definite.
419
00:25:18,160 --> 00:25:21,410
So what number
would you like here?
420
00:25:21,410 --> 00:25:22,770
Five?
421
00:25:22,770 --> 00:25:27,960
Five is certainly plenty.
422
00:25:27,960 --> 00:25:30,600
So now I have [1, 5; 5, 9].
423
00:25:30,600 --> 00:25:37,440
Let me take a little space
on a board just to show you.
424
00:25:37,440 --> 00:25:42,190
Sorry about that.
425
00:25:42,190 --> 00:25:44,450
So I'm going to do
the [1, 5; 5, 9]
426
00:25:44,450 --> 00:25:46,800
just because they're
all important,
427
00:25:46,800 --> 00:25:49,480
but then we're coming
back to positive definite.
428
00:25:49,480 --> 00:25:57,510
So if it's [1, 5; 5, 9] and
I do that usual x transpose
429
00:25:57,510 --> 00:26:03,890
Kx and I do the multiplication
out, I see the one x_1 squared
430
00:26:03,890 --> 00:26:06,750
and I see the nine x_2 squareds.
431
00:26:06,750 --> 00:26:11,690
And how many x_1*x_2's do I see?
432
00:26:11,690 --> 00:26:15,480
Five from there,
five from there, ten.
433
00:26:15,480 --> 00:26:20,800
And I believe that
can be negative.
434
00:26:20,800 --> 00:26:23,220
The fact of having
all nice plus signs
435
00:26:23,220 --> 00:26:25,970
is not going to help it
because we can choose,
436
00:26:25,970 --> 00:26:30,030
as we already did, x_1 to be
like a negative number and x_2
437
00:26:30,030 --> 00:26:31,350
to be a positive.
438
00:26:31,350 --> 00:26:35,180
And we can get this guy to
be negative and make it,
439
00:26:35,180 --> 00:26:41,040
in this case we can make it
defeat these positive parts.
440
00:26:41,040 --> 00:26:43,590
What choice would do it?
441
00:26:43,590 --> 00:26:46,560
Let me take x_1 to be
minus one and tell me
442
00:26:46,560 --> 00:26:53,760
an x_2 that's good enough to
show that this thing is not
443
00:26:53,760 --> 00:26:56,520
positive definite or
even semi-definite,
444
00:26:56,520 --> 00:26:58,360
it goes downhill.
445
00:26:58,360 --> 00:26:59,570
Take x_2 equal?
446
00:26:59,570 --> 00:27:04,030
What do you say?
447
00:27:04,030 --> 00:27:05,460
1/2?
448
00:27:05,460 --> 00:27:07,250
Yeah, I don't want
too big an x_2
449
00:27:07,250 --> 00:27:10,910
because if I have too big an
x_2, then this'll be important.
450
00:27:10,910 --> 00:27:14,550
Does 1/2 do it?
451
00:27:14,550 --> 00:27:18,040
So I've got 1/4, that's
positive, but not very.
452
00:27:18,040 --> 00:27:23,610
9/4, so I'm up to 10/4,
but this guy is what?
453
00:27:23,610 --> 00:27:26,531
Ten and the minus is minus five.
454
00:27:26,531 --> 00:27:27,030
Yeah.
455
00:27:27,030 --> 00:27:35,370
So that absolutely goes, at this
one I come out less than zero.
456
00:27:35,370 --> 00:27:37,170
And I might as well complete.
457
00:27:37,170 --> 00:27:43,890
So this is the case where
I would call it indefinite.
458
00:27:43,890 --> 00:27:45,190
Indefinite.
459
00:27:45,190 --> 00:27:49,240
It goes up, like if
x_2 is zero, then
460
00:27:49,240 --> 00:27:52,170
it's just got x_1
squared, that's up.
461
00:27:52,170 --> 00:27:55,750
If x_1 is zero, it's only
got x_2 squared, that's up.
462
00:27:55,750 --> 00:27:58,890
But there are other directions
where it goes downhill.
463
00:27:58,890 --> 00:28:02,100
So it goes either up-- it
goes both up in some ways
464
00:28:02,100 --> 00:28:03,160
and down in others.
465
00:28:03,160 --> 00:28:07,760
And what kind of a graph,
what kind of a surface
466
00:28:07,760 --> 00:28:12,160
would I now have for x
transpose for this x transpose,
467
00:28:12,160 --> 00:28:15,690
this indefinite guy?
468
00:28:15,690 --> 00:28:26,400
So up in some ways
and down in others.
469
00:28:26,400 --> 00:28:35,150
This gets really hard to draw.
470
00:28:35,150 --> 00:28:39,890
I believe that if
you ride horses
471
00:28:39,890 --> 00:28:43,090
you have an edge on
visualizing this.
472
00:28:43,090 --> 00:28:45,760
So it's called, what kind
of a point's it called?
473
00:28:45,760 --> 00:28:50,260
Saddle point, it's
called a saddle point.
474
00:28:50,260 --> 00:28:53,890
So what's a saddle point?
475
00:28:53,890 --> 00:28:56,460
That's not bad, right?
476
00:28:56,460 --> 00:28:58,650
So this is a direction
where it went up.
477
00:28:58,650 --> 00:29:02,210
This is a direction
where it went down.
478
00:29:02,210 --> 00:29:09,440
And so it sort of
fills in somehow.
479
00:29:09,440 --> 00:29:18,330
Or maybe, if you don't, I
mean, who rides horses now?
480
00:29:18,330 --> 00:29:25,190
Actually maybe something we
do do is drive over mountains.
481
00:29:25,190 --> 00:29:35,380
So the path, if the road
is sort of well-chosen,
482
00:29:35,380 --> 00:29:39,250
the road will go,
it'll look for the,
483
00:29:39,250 --> 00:29:43,540
this would be-- Yeah,
here's our road.
484
00:29:43,540 --> 00:29:46,310
We would do as little
climbing as possible.
485
00:29:46,310 --> 00:29:48,860
The mountain would go
like this, sort of.
486
00:29:48,860 --> 00:29:52,320
So this would be
like, the bottom part
487
00:29:52,320 --> 00:29:55,630
looking along the
peaks of the mountains.
488
00:29:55,630 --> 00:29:59,410
But it's the top part looking
along the driving direction.
489
00:29:59,410 --> 00:30:05,590
So driving, it's a maximum, but
in the mountain range direction
490
00:30:05,590 --> 00:30:06,320
it's a minimum.
491
00:30:06,320 --> 00:30:10,500
So it's a saddle point.
492
00:30:10,500 --> 00:30:14,810
So that's what you get from
a typical symmetric matrix.
493
00:30:14,810 --> 00:30:19,010
And if it was minus five it
would still be the same saddle
494
00:30:19,010 --> 00:30:23,480
point, would still be
9-25, it would still
495
00:30:23,480 --> 00:30:27,130
be negative and a saddle.
496
00:30:27,130 --> 00:30:30,200
Positive guys are our thing.
497
00:30:30,200 --> 00:30:32,000
Alright.
498
00:30:32,000 --> 00:30:36,230
So now back to
positive definite.
499
00:30:36,230 --> 00:30:40,650
With these four tests
and then the discussion
500
00:30:40,650 --> 00:30:45,680
of semi-definite.
501
00:30:45,680 --> 00:30:49,150
Very key, that energy.
502
00:30:49,150 --> 00:30:51,970
Let me just look ahead a moment.
503
00:30:51,970 --> 00:30:58,650
Most physical problems,
many, many physical problems,
504
00:30:58,650 --> 00:31:00,010
you have an option.
505
00:31:00,010 --> 00:31:04,810
Either you solve some equations,
either you find the solution
506
00:31:04,810 --> 00:31:10,040
from our equations,
Ku=f, typically.
507
00:31:10,040 --> 00:31:12,670
Matrix equation or
differential equation.
508
00:31:12,670 --> 00:31:20,940
Or there's another option
of minimizing some function.
509
00:31:20,940 --> 00:31:23,640
Some energy.
510
00:31:23,640 --> 00:31:25,850
And it gives the same equations.
511
00:31:25,850 --> 00:31:30,680
So this minimizing energy
will be a second way
512
00:31:30,680 --> 00:31:36,720
to describe the applications.
513
00:31:36,720 --> 00:31:40,760
Now can I get a number five?
514
00:31:40,760 --> 00:31:42,660
There's an important
number five and then
515
00:31:42,660 --> 00:31:48,130
you know really all you need to
know about symmetric matrices.
516
00:31:48,130 --> 00:31:51,630
This gives me, about
positive definite matrices,
517
00:31:51,630 --> 00:32:01,750
this gives me a chance to recap.
518
00:32:01,750 --> 00:32:06,400
So I'm going to put
down a number five.
519
00:32:06,400 --> 00:32:16,540
Because this is where
the matrices come from.
520
00:32:16,540 --> 00:32:17,480
Really important.
521
00:32:17,480 --> 00:32:20,390
And it's where they'll come
from in all these applications
522
00:32:20,390 --> 00:32:23,230
that chapter two is going
to be all about, that we're
523
00:32:23,230 --> 00:32:25,600
going to start.
524
00:32:25,600 --> 00:32:28,500
So they come, these
positive definite matrices,
525
00:32:28,500 --> 00:32:34,290
so this is another
way to, it's a test
526
00:32:34,290 --> 00:32:39,180
for positive definite matrices
and it's, actually, it's
527
00:32:39,180 --> 00:32:40,930
where they come from.
528
00:32:40,930 --> 00:32:44,340
So here's a positive
definite matrix.
529
00:32:44,340 --> 00:32:54,270
They come from A transpose
A. A fundamental message is
530
00:32:54,270 --> 00:32:57,090
that if I have just
an average matrix,
531
00:32:57,090 --> 00:33:01,840
possibly rectangular, could
be a square but not symmetric,
532
00:33:01,840 --> 00:33:07,220
then sooner or later,
in fact usually sooner,
533
00:33:07,220 --> 00:33:10,430
you end up looking
at A transpose A.
534
00:33:10,430 --> 00:33:11,940
We've seen that already.
535
00:33:11,940 --> 00:33:15,390
And we already know that
A transpose A is square,
536
00:33:15,390 --> 00:33:17,950
we already know it's
symmetric and now
537
00:33:17,950 --> 00:33:20,980
we're going to know that
it's positive definite.
538
00:33:20,980 --> 00:33:25,140
So matrices like A transpose
A are positive definite
539
00:33:25,140 --> 00:33:28,140
or possibly semi-definite.
540
00:33:28,140 --> 00:33:29,660
There's that possibility.
541
00:33:29,660 --> 00:33:31,920
If A was the zero
matrix, of course,
542
00:33:31,920 --> 00:33:33,820
we would just get
the zero matrix which
543
00:33:33,820 --> 00:33:37,460
would be only
semi-definite, or other ways
544
00:33:37,460 --> 00:33:42,120
to get a semi-definite.
545
00:33:42,120 --> 00:33:46,720
So I'm saying that if K, if
I have a matrix, any matrix,
546
00:33:46,720 --> 00:33:50,770
and I form A transpose A, I
get a positive definite matrix
547
00:33:50,770 --> 00:33:56,780
or maybe just semi-definite,
but not indefinite.
548
00:33:56,780 --> 00:34:01,850
Can we see why?
549
00:34:01,850 --> 00:34:11,900
Why is this positive
definite or semi-?
550
00:34:11,900 --> 00:34:13,740
So that's my question.
551
00:34:13,740 --> 00:34:16,490
And the answer is
really worth-- it's
552
00:34:16,490 --> 00:34:19,070
just neat and worth seeing.
553
00:34:19,070 --> 00:34:23,970
So do I want to look at the
pivots of A transpose A?
554
00:34:23,970 --> 00:34:25,970
No.
555
00:34:25,970 --> 00:34:27,910
They're something,
but whatever they are,
556
00:34:27,910 --> 00:34:30,460
I can't really
follow those well.
557
00:34:30,460 --> 00:34:34,870
Or the eigenvalues very
well, or the determinants.
558
00:34:34,870 --> 00:34:36,790
None of those come out nicely.
559
00:34:36,790 --> 00:34:41,690
But the real guy
works perfectly.
560
00:34:41,690 --> 00:34:46,060
So look at x transpose Kx.
561
00:34:46,060 --> 00:34:48,580
562
00:34:48,580 --> 00:34:57,190
So I'm just doing--
following my instinct here.
563
00:34:57,190 --> 00:35:00,590
So if K is A
transpose A, my claim
564
00:35:00,590 --> 00:35:07,050
is, what am I saying
then about this energy?
565
00:35:07,050 --> 00:35:13,090
What is it that I want to
discover and understand?
566
00:35:13,090 --> 00:35:15,630
Why it's positive.
567
00:35:15,630 --> 00:35:18,820
Why does taking any
matrix, multiplying
568
00:35:18,820 --> 00:35:27,670
by its transpose produce
something that's positive?
569
00:35:27,670 --> 00:35:30,520
Can you see any reason
why that quantity,
570
00:35:30,520 --> 00:35:33,930
which looks kind
of messy, I just
571
00:35:33,930 --> 00:35:37,530
want to look at it
the right way to see
572
00:35:37,530 --> 00:35:41,510
why that should be positive,
that should come out positive.
573
00:35:41,510 --> 00:35:44,820
So I'm not going to
get into numbers,
574
00:35:44,820 --> 00:35:47,640
I'm not going to get into
diagonals and off-diagonals.
575
00:35:47,640 --> 00:35:52,260
I'm just going to do
one thing to understand
576
00:35:52,260 --> 00:35:57,990
that particular combination,
x transpose A transpose Ax.
577
00:35:57,990 --> 00:35:59,870
What shall I do?
578
00:35:59,870 --> 00:36:06,130
Anybody see what I might do?
579
00:36:06,130 --> 00:36:10,190
Yeah, you're seeing here
if you look at it again,
580
00:36:10,190 --> 00:36:12,210
what are you seeing here?
581
00:36:12,210 --> 00:36:14,220
Tell me again.
582
00:36:14,220 --> 00:36:20,500
If I take Ax together,
then what's the other half?
583
00:36:20,500 --> 00:36:23,090
It's the transpose of Ax.
584
00:36:23,090 --> 00:36:25,900
So I just want to write
that as, I just want
585
00:36:25,900 --> 00:36:29,000
to think of it that way, as Ax.
586
00:36:29,000 --> 00:36:32,270
And here's the transpose of Ax.
587
00:36:32,270 --> 00:36:33,040
Right?
588
00:36:33,040 --> 00:36:36,050
Because transposes
of Ax, so transpose
589
00:36:36,050 --> 00:36:39,890
guys in the opposite order,
and the multiplication--
590
00:36:39,890 --> 00:36:41,230
This is the great.
591
00:36:41,230 --> 00:36:44,380
I call these proof
by parenthesis
592
00:36:44,380 --> 00:36:47,150
because I'm just putting
parentheses in the right place,
593
00:36:47,150 --> 00:36:53,940
but the key law of
matrix multiplication
594
00:36:53,940 --> 00:36:58,920
is that, that I can put
(AB)C is the same as A(BC).
595
00:36:58,920 --> 00:37:01,810
596
00:37:01,810 --> 00:37:04,170
That rule, which is
just multiply it out
597
00:37:04,170 --> 00:37:06,150
and you see that
parentheses are not
598
00:37:06,150 --> 00:37:08,430
needed because if you keep
them in the right order
599
00:37:08,430 --> 00:37:12,300
you can do this first,
or you can do this first.
600
00:37:12,300 --> 00:37:13,720
Same answer.
601
00:37:13,720 --> 00:37:15,480
What do I learn from that?
602
00:37:15,480 --> 00:37:17,020
What was the point?
603
00:37:17,020 --> 00:37:20,510
This is some vector, I don't
know especially what it is,
604
00:37:20,510 --> 00:37:21,900
times its transpose.
605
00:37:21,900 --> 00:37:24,910
So that's the length squared.
606
00:37:24,910 --> 00:37:27,210
What's the key fact about that?
607
00:37:27,210 --> 00:37:30,000
That it is never negative.
608
00:37:30,000 --> 00:37:41,070
It's always greater than
zero or possibly equal.
609
00:37:41,070 --> 00:37:44,280
When does that
quantity equal zero?
610
00:37:44,280 --> 00:37:45,580
When Ax is zero.
611
00:37:45,580 --> 00:37:47,060
When Ax is zero.
612
00:37:47,060 --> 00:37:49,050
Because this is a vector.
613
00:37:49,050 --> 00:37:50,830
That's the same
vector transposed.
614
00:37:50,830 --> 00:37:52,500
And everybody's
got that picture.
615
00:37:52,500 --> 00:37:56,970
When I take any y
transpose y, I get
616
00:37:56,970 --> 00:38:00,440
y_1 squared plus y_2
squared through y_n squared.
617
00:38:00,440 --> 00:38:05,450
And I get a positive answer
except if the vector is zero.
618
00:38:05,450 --> 00:38:11,720
So it's zero when Ax is zero.
619
00:38:11,720 --> 00:38:13,990
So that's going to be the key.
620
00:38:13,990 --> 00:38:16,680
If I pick any matrix
A, and I can just
621
00:38:16,680 --> 00:38:21,670
take an example, but
chapter-- the applications
622
00:38:21,670 --> 00:38:23,730
are just going to
be full of examples.
623
00:38:23,730 --> 00:38:29,110
Where the problem begins
with a matrix A and then
624
00:38:29,110 --> 00:38:34,960
A transpose shows up and it's
the combination A transpose A
625
00:38:34,960 --> 00:38:36,100
that we work with.
626
00:38:36,100 --> 00:38:40,150
And we're just learning
that it's positive definite.
627
00:38:40,150 --> 00:38:46,440
Unless, shall I just hang
on since I've got here,
628
00:38:46,440 --> 00:38:53,470
I have to say when is it, have
to get these two possibilities.
629
00:38:53,470 --> 00:38:56,950
Positive definite or
only semi-definite.
630
00:38:56,950 --> 00:39:05,820
So what's the key to
that borderline question?
631
00:39:05,820 --> 00:39:09,680
This thing will be
only semi-definite
632
00:39:09,680 --> 00:39:12,270
if there's a solution to Ax=0.
633
00:39:12,270 --> 00:39:16,000
634
00:39:16,000 --> 00:39:23,560
If there is an x, well,
there's always the zero vector.
635
00:39:23,560 --> 00:39:26,730
Zero vector I can't
expect to be positive.
636
00:39:26,730 --> 00:39:31,300
So I'm looking for
if there's an x so
637
00:39:31,300 --> 00:39:42,130
that Ax is zero
but x is not zero,
638
00:39:42,130 --> 00:39:48,420
then I'll only be semi-definite.
639
00:39:48,420 --> 00:39:50,320
That's the test.
640
00:39:50,320 --> 00:39:52,730
If there is a solution to Ax=0.
641
00:39:52,730 --> 00:39:55,420
642
00:39:55,420 --> 00:39:58,320
When we see applications
that'll mean
643
00:39:58,320 --> 00:40:03,110
there's a displacement
with no stretching.
644
00:40:03,110 --> 00:40:09,580
We might have a line
of springs and when
645
00:40:09,580 --> 00:40:16,570
could the line of springs
displace with no stretching?
646
00:40:16,570 --> 00:40:18,390
When it's free-free, right?
647
00:40:18,390 --> 00:40:24,010
If I have a line of springs
and no supports at the ends,
648
00:40:24,010 --> 00:40:26,860
then that would be the case
where it could shift over
649
00:40:26,860 --> 00:40:29,350
by the [1, 1, 1] vector.
650
00:40:29,350 --> 00:40:33,230
So that would be the case where
the matrix is only singular.
651
00:40:33,230 --> 00:40:34,410
We know that.
652
00:40:34,410 --> 00:40:37,440
The matrix is now
positive semi-definite.
653
00:40:37,440 --> 00:40:38,880
We just learned that.
654
00:40:38,880 --> 00:40:43,330
So the free-free
matrix, like B, both
655
00:40:43,330 --> 00:40:48,690
ends free, or C.
So our answer is
656
00:40:48,690 --> 00:40:56,170
going to be that K and
T are positive definite.
657
00:40:56,170 --> 00:40:59,470
And our other two guys, the
singular ones, of course,
658
00:40:59,470 --> 00:41:00,590
just don't make it.
659
00:41:00,590 --> 00:41:04,000
B at both ends, the
free-free line of springs,
660
00:41:04,000 --> 00:41:07,040
it can shift without stretching.
661
00:41:07,040 --> 00:41:10,810
Since Ax will measure the
stretching when it just
662
00:41:10,810 --> 00:41:13,630
shifts rigid motion,
the Ax is zero
663
00:41:13,630 --> 00:41:16,110
and we see only
positive definite.
664
00:41:16,110 --> 00:41:19,030
And also C, the circular one.
665
00:41:19,030 --> 00:41:21,460
There it can displace
with no stretching
666
00:41:21,460 --> 00:41:24,360
because it can just
turn in the circle.
667
00:41:24,360 --> 00:41:45,220
So these guys will be only
positive semi-definite.
668
00:41:45,220 --> 00:41:49,340
Maybe I better say
this another way.
669
00:41:49,340 --> 00:41:51,790
When is this positive definite?
670
00:41:51,790 --> 00:41:54,930
Can I use just a
different sentence
671
00:41:54,930 --> 00:41:57,270
to describe this possibility?
672
00:41:57,270 --> 00:42:03,830
This is positive
definite provided,
673
00:42:03,830 --> 00:42:08,270
so what I'm going to write now
is to remove this possibility
674
00:42:08,270 --> 00:42:10,140
and get positive definite.
675
00:42:10,140 --> 00:42:16,010
This is positive
definite provided, now,
676
00:42:16,010 --> 00:42:17,490
I could say it this way.
677
00:42:17,490 --> 00:42:25,130
The A has independent columns.
678
00:42:25,130 --> 00:42:29,080
So I just needed to give you
another way of looking at this
679
00:42:29,080 --> 00:42:33,400
Ax=0 question.
680
00:42:33,400 --> 00:42:37,220
If A has independent
columns, what does that mean?
681
00:42:37,220 --> 00:42:40,920
That means that the only
solution to Ax=0 is the zero
682
00:42:40,920 --> 00:42:42,800
solution.
683
00:42:42,800 --> 00:42:47,640
In other words, it means that
this thing works perfectly
684
00:42:47,640 --> 00:42:50,520
and gives me positive.
685
00:42:50,520 --> 00:42:53,080
When A has independent columns.
686
00:42:53,080 --> 00:43:05,470
Let's just remember our K,
T, B, C. So here's a matrix,
687
00:43:05,470 --> 00:43:11,150
so let me take the T matrix,
that's this one, this guy.
688
00:43:11,150 --> 00:43:15,600
And then the third
column is [0, -1, 2].
689
00:43:15,600 --> 00:43:19,960
Those three columns
are independent.
690
00:43:19,960 --> 00:43:21,250
They point off.
691
00:43:21,250 --> 00:43:23,110
They don't lie in a plane.
692
00:43:23,110 --> 00:43:27,140
They point off in three
different directions.
693
00:43:27,140 --> 00:43:34,820
And then there are no solutions
to, no x's that go Kx=0.
694
00:43:34,820 --> 00:43:39,140
695
00:43:39,140 --> 00:43:41,840
So that would be a case
of independent columns.
696
00:43:41,840 --> 00:43:45,000
Let me make a case
of dependent columns.
697
00:43:45,000 --> 00:43:47,730
So, and I'm going
to make it B now.
698
00:43:47,730 --> 00:43:51,750
Now the columns of that
matrix are dependent.
699
00:43:51,750 --> 00:43:54,280
There's a combination
of them that give zero.
700
00:43:54,280 --> 00:43:56,530
They all lie in the same plane.
701
00:43:56,530 --> 00:44:00,210
There's a solution to that
matrix times x equal zero.
702
00:44:00,210 --> 00:44:02,440
What combination
of those columns
703
00:44:02,440 --> 00:44:05,850
shows me that they
are dependent?
704
00:44:05,850 --> 00:44:09,000
That some combination
of those three columns,
705
00:44:09,000 --> 00:44:10,880
some amount of this
plus some amount
706
00:44:10,880 --> 00:44:12,630
of this plus some
amount of that column
707
00:44:12,630 --> 00:44:15,350
gives me the zero vector.
708
00:44:15,350 --> 00:44:17,600
You see the combination.
709
00:44:17,600 --> 00:44:21,550
What should I take?
[1, 1, 1] again.
710
00:44:21,550 --> 00:44:22,300
No surprise.
711
00:44:22,300 --> 00:44:26,050
That's the vector
[1, 1, 1] that we
712
00:44:26,050 --> 00:44:29,840
know is in the-- everything
shifting the same amount,
713
00:44:29,840 --> 00:44:36,680
nothing stretching.
714
00:44:36,680 --> 00:44:40,430
Talking fast here about
positive definite matrices.
715
00:44:40,430 --> 00:44:42,360
This is the key.
716
00:44:42,360 --> 00:44:44,980
Let's just ask a few questions
about positive definite
717
00:44:44,980 --> 00:44:49,510
matrices as a way to practice.
718
00:44:49,510 --> 00:44:50,940
Suppose I had one.
719
00:44:50,940 --> 00:44:52,560
Positive definite.
720
00:44:52,560 --> 00:44:57,390
What about its inverse?
721
00:44:57,390 --> 00:45:02,340
Is that positive
definite or not?
722
00:45:02,340 --> 00:45:06,160
So I've got a positive definite
one, it's not singular,
723
00:45:06,160 --> 00:45:09,990
it's got positive
eigenvalues, everything else.
724
00:45:09,990 --> 00:45:14,240
It's inverse will
be symmetric, so I'm
725
00:45:14,240 --> 00:45:16,330
allowed to think about it.
726
00:45:16,330 --> 00:45:20,530
Will it be positive definite?
727
00:45:20,530 --> 00:45:23,670
What do you think?
728
00:45:23,670 --> 00:45:27,000
Well, you've got a
whole bunch of tests
729
00:45:27,000 --> 00:45:30,090
to sort of mentally run through.
730
00:45:30,090 --> 00:45:35,580
Pivots of the inverse, you
don't want to touch that stuff.
731
00:45:35,580 --> 00:45:36,690
Determinants, no.
732
00:45:36,690 --> 00:45:39,230
What about eigenvalues?
733
00:45:39,230 --> 00:45:41,090
What would be the
eigenvalues if I
734
00:45:41,090 --> 00:45:43,640
have this positive
definite symmetric matrix,
735
00:45:43,640 --> 00:45:46,800
its eigenvalues are
one, four, five.
736
00:45:46,800 --> 00:45:49,700
What can you tell me
about the eigenvalues
737
00:45:49,700 --> 00:45:53,030
of the inverse matrix?
738
00:45:53,030 --> 00:45:54,080
They're the inverses.
739
00:45:54,080 --> 00:45:56,690
So those three eigenvalues are?
740
00:45:56,690 --> 00:46:00,890
1, 1/4, 1/5, what's
the conclusion here?
741
00:46:00,890 --> 00:46:02,080
It is positive definite.
742
00:46:02,080 --> 00:46:04,430
Those are all positive,
it is positive definite.
743
00:46:04,430 --> 00:46:07,940
So if I invert a
positive definite matrix,
744
00:46:07,940 --> 00:46:11,350
I'm still positive definite.
745
00:46:11,350 --> 00:46:13,660
All the tests
would have to pass.
746
00:46:13,660 --> 00:46:17,720
It's just I'm looking each
time for the easiest test.
747
00:46:17,720 --> 00:46:22,380
Let me look now, for the
easiest test on K_1+K_2.
748
00:46:22,380 --> 00:46:25,470
749
00:46:25,470 --> 00:46:29,980
Suppose that's positive definite
and that's positive definite.
750
00:46:29,980 --> 00:46:33,670
What if I add them?
751
00:46:33,670 --> 00:46:35,940
What do you think?
752
00:46:35,940 --> 00:46:38,680
Well, we hope so.
753
00:46:38,680 --> 00:46:42,840
But we have to say which of
my one, two, three, four, five
754
00:46:42,840 --> 00:46:45,660
would be a good way to see it.
755
00:46:45,660 --> 00:46:48,300
Would be a good way to see it.
756
00:46:48,300 --> 00:46:50,830
Good question.
757
00:46:50,830 --> 00:46:53,560
Four?
758
00:46:53,560 --> 00:46:55,280
We certainly don't
want to touch pivots
759
00:46:55,280 --> 00:46:58,860
and now we don't want to
touch eigenvalues either.
760
00:46:58,860 --> 00:47:03,400
Of course, if number four
works, others will also work.
761
00:47:03,400 --> 00:47:05,650
The eigenvalues will
come out positive.
762
00:47:05,650 --> 00:47:08,170
But not too easy to
say what they are.
763
00:47:08,170 --> 00:47:14,450
Let's try test number four.
764
00:47:14,450 --> 00:47:15,060
So K_1.
765
00:47:15,060 --> 00:47:18,760
766
00:47:18,760 --> 00:47:20,020
What's the test?
767
00:47:20,020 --> 00:47:23,430
So test number four
tells us that this part,
768
00:47:23,430 --> 00:47:28,670
x transpose K_1*x, that that
part is positive, right?
769
00:47:28,670 --> 00:47:30,900
That that part is positive.
770
00:47:30,900 --> 00:47:33,040
If we know that's
positive definite.
771
00:47:33,040 --> 00:47:37,030
Now, about K_2 we also know
that for every x, you see it's
772
00:47:37,030 --> 00:47:38,720
for every x, that helps.
773
00:47:38,720 --> 00:47:40,900
Don't let me put x_2 there.
774
00:47:40,900 --> 00:47:47,670
For every x, this
will be positive.
775
00:47:47,670 --> 00:47:52,670
And now what's the
step I want to take?
776
00:47:52,670 --> 00:47:57,030
To get some information
on the matrix K_1+K_2.
777
00:47:57,030 --> 00:47:59,590
778
00:47:59,590 --> 00:48:01,280
I should add.
779
00:48:01,280 --> 00:48:06,330
If I add these guys, you
see that it just, then I
780
00:48:06,330 --> 00:48:14,160
can write that as, I
can write that this way.
781
00:48:14,160 --> 00:48:17,650
And what have I learned?
782
00:48:17,650 --> 00:48:19,890
I've learned that that's
positive, even greater than,
783
00:48:19,890 --> 00:48:21,680
except for the zero vector.
784
00:48:21,680 --> 00:48:23,930
Because this was greater
than, this is greater than.
785
00:48:23,930 --> 00:48:27,170
If I add two positive numbers,
the energies are positive
786
00:48:27,170 --> 00:48:29,380
and the energies just add.
787
00:48:29,380 --> 00:48:34,660
The energies just add.
788
00:48:34,660 --> 00:48:40,170
So that definition four was the
good way, just nice, easy way
789
00:48:40,170 --> 00:48:44,960
to see that if I have a couple
of positive definite matrices,
790
00:48:44,960 --> 00:48:47,020
a couple of positive
energies, I'm really
791
00:48:47,020 --> 00:48:49,570
coupling the two systems.
792
00:48:49,570 --> 00:48:53,030
This is associated somehow.
793
00:48:53,030 --> 00:48:55,190
I've got two systems,
I'm putting them together
794
00:48:55,190 --> 00:49:00,260
and the energy is just
even more positive.
795
00:49:00,260 --> 00:49:05,860
It's more positive either of
these guys because I'm adding.
796
00:49:05,860 --> 00:49:09,070
As I'm speaking
here, will you allow
797
00:49:09,070 --> 00:49:14,870
me to try test number five,
this A transpose A business?
798
00:49:14,870 --> 00:49:21,271
Suppose K_1 was A transpose
A. If it's positive definite,
799
00:49:21,271 --> 00:49:21,770
it will.
800
00:49:21,770 --> 00:49:31,190
Be And suppose K_2
is B transpose B.
801
00:49:31,190 --> 00:49:33,280
If it's positive
definite, it will be.
802
00:49:33,280 --> 00:49:41,390
Now I would like to
write the sum somehow as,
803
00:49:41,390 --> 00:49:43,990
in this something
transpose something.
804
00:49:43,990 --> 00:49:46,220
And I just do it
now because I think
805
00:49:46,220 --> 00:49:50,020
it's like, you won't perhaps
have thought of this way
806
00:49:50,020 --> 00:49:54,000
to do it.
807
00:49:54,000 --> 00:49:56,070
Watch.
808
00:49:56,070 --> 00:50:01,830
Suppose I create
the matrix [A; B].
809
00:50:01,830 --> 00:50:03,210
That'll be my new matrix.
810
00:50:03,210 --> 00:50:11,100
Say, call it C. Am I
allowed to do that?
811
00:50:11,100 --> 00:50:13,210
I mean, that creates a matrix?
812
00:50:13,210 --> 00:50:18,110
These A and B, they had the
same number of columns, n.
813
00:50:18,110 --> 00:50:20,300
So I can put one over
the other and I still
814
00:50:20,300 --> 00:50:22,370
have something with n columns.
815
00:50:22,370 --> 00:50:26,800
So that's my new matrix C.
And now I want C transpose.
816
00:50:26,800 --> 00:50:31,490
By the way, I'd call
that a block matrix.
817
00:50:31,490 --> 00:50:35,990
You know, instead of numbers,
it's got two blocks in there.
818
00:50:35,990 --> 00:50:37,980
Block matrices are really handy.
819
00:50:37,980 --> 00:50:43,310
Now what's the transpose
of that block matrix?
820
00:50:43,310 --> 00:50:47,490
You just have faith, just
have faith with blocks.
821
00:50:47,490 --> 00:50:48,880
It's just like numbers.
822
00:50:48,880 --> 00:50:55,970
If I had a matrix [1; 5]
then I'd get a row one, five.
823
00:50:55,970 --> 00:50:57,810
But what do you think?
824
00:50:57,810 --> 00:51:01,270
This is worth thinking
about even after class.
825
00:51:01,270 --> 00:51:05,470
What would be, if this C
matrix is this block A above B,
826
00:51:05,470 --> 00:51:07,870
what do you think
for C transpose?
827
00:51:07,870 --> 00:51:11,510
A transpose, B
transpose side by side.
828
00:51:11,510 --> 00:51:15,720
Just put in numbers
and you'd see it.
829
00:51:15,720 --> 00:51:17,990
And now I'm going
to take C transpose
830
00:51:17,990 --> 00:51:23,880
times C. I'm calling
it C now instead of A
831
00:51:23,880 --> 00:51:26,360
because I've used the
A in the first guy
832
00:51:26,360 --> 00:51:31,780
and I've used B in the second
one and now I'm ready for C.
833
00:51:31,780 --> 00:51:35,490
How do you multiply
block matrices?
834
00:51:35,490 --> 00:51:37,980
Again, you just have faith.
835
00:51:37,980 --> 00:51:39,750
What do you think?
836
00:51:39,750 --> 00:51:41,820
Tell me the answer.
837
00:51:41,820 --> 00:51:43,880
A transpose, I
multiply that by that
838
00:51:43,880 --> 00:51:47,180
just as if they were numbers.
839
00:51:47,180 --> 00:51:52,020
And I add that times that
just as if they were numbers.
840
00:51:52,020 --> 00:51:55,072
And what do I have?
841
00:51:55,072 --> 00:51:55,780
I've got K_1+K_2.
842
00:51:55,780 --> 00:51:58,600
843
00:51:58,600 --> 00:52:05,000
So I've written K_1, this is
K_1+K_2 and this is in my form
844
00:52:05,000 --> 00:52:09,080
C transpose C that I was looking
for, that number five was
845
00:52:09,080 --> 00:52:10,860
looking for.
846
00:52:10,860 --> 00:52:12,920
So it's done it.
847
00:52:12,920 --> 00:52:13,660
It's done it.
848
00:52:13,660 --> 00:52:19,310
The fact of getting A-- K_1 in
this form, K_2 in this form.
849
00:52:19,310 --> 00:52:21,720
And I just made a block
matrix and I got K_1+K_2.
850
00:52:21,720 --> 00:52:25,730
851
00:52:25,730 --> 00:52:29,730
That's not a big deal in
itself, but block matrices
852
00:52:29,730 --> 00:52:32,060
are really handy.
853
00:52:32,060 --> 00:52:36,260
It's good to take that
step with matrices.
854
00:52:36,260 --> 00:52:39,930
Think of, possibly, the
entries as coming in blocks
855
00:52:39,930 --> 00:52:42,600
and not just one at a time.
856
00:52:42,600 --> 00:52:44,330
Well, thank you, okay.
857
00:52:44,330 --> 00:52:50,700
I swear Friday we'll
start applications
858
00:52:50,700 --> 00:52:53,000
in all kinds of
engineering problems
859
00:52:53,000 --> 00:52:55,277
and you'll have
new applications.
860
00:52:55,277 --> 00:52:55,777