1
00:00:10,510 --> 00:00:11,010
OK.
2
00:00:11,010 --> 00:00:11,635
Shall we start?
3
00:00:11,635 --> 00:00:15,360
This is the second
lecture on eigenvalues.
4
00:00:15,360 --> 00:00:18,350
So the first lecture was --
5
00:00:18,350 --> 00:00:22,350
reached the key equation,
A x equal lambda x.
6
00:00:22,350 --> 00:00:27,090
x is the eigenvector and
lambda's the eigenvalue.
7
00:00:27,090 --> 00:00:30,220
Now to use that.
8
00:00:30,220 --> 00:00:36,330
And the, the good way
to, after we've found --
9
00:00:36,330 --> 00:00:39,400
so, so job one is to
find the eigenvalues
10
00:00:39,400 --> 00:00:41,990
and find the eigenvectors.
11
00:00:41,990 --> 00:00:44,750
Now after we've found them,
what do we do with them?
12
00:00:44,750 --> 00:00:50,360
Well, the good way to see that
is diagonalize the matrix.
13
00:00:50,360 --> 00:00:52,850
So the matrix is A.
14
00:00:52,850 --> 00:00:55,130
And I want to show
-- first of all,
15
00:00:55,130 --> 00:00:57,190
this is like the basic fact.
16
00:00:57,190 --> 00:00:58,720
This, this formula.
17
00:00:58,720 --> 00:01:03,210
That's, that's the key
to today's lecture.
18
00:01:03,210 --> 00:01:06,810
This matrix A, I
put its eigenvectors
19
00:01:06,810 --> 00:01:09,660
in the columns of a matrix S.
20
00:01:09,660 --> 00:01:12,740
So S will be the
eigenvector matrix.
21
00:01:12,740 --> 00:01:19,560
And I want to look at this
magic combination S inverse A S.
22
00:01:19,560 --> 00:01:22,830
So can I show you how that --
23
00:01:22,830 --> 00:01:24,200
what happens there?
24
00:01:24,200 --> 00:01:27,900
And notice, there's
an S inverse.
25
00:01:27,900 --> 00:01:33,690
We have to be able to invert
this eigenvector matrix S.
26
00:01:33,690 --> 00:01:38,330
So for that, we need n
independent eigenvectors.
27
00:01:38,330 --> 00:01:41,440
So that's the, that's the case.
28
00:01:41,440 --> 00:01:42,130
OK.
29
00:01:42,130 --> 00:01:53,740
So suppose we have n linearly
independent eigenvectors
30
00:01:53,740 --> 00:02:11,830
of A. Put them in the
columns of this matrix S.
31
00:02:11,830 --> 00:02:15,430
So I'm naturally going to call
that the eigenvector matrix,
32
00:02:15,430 --> 00:02:18,730
because it's got the
eigenvectors in its columns.
33
00:02:18,730 --> 00:02:21,730
And all I want to do is
show you what happens
34
00:02:21,730 --> 00:02:24,960
when you multiply A times S.
35
00:02:24,960 --> 00:02:29,200
So A times S.
36
00:02:29,200 --> 00:02:34,490
So this is A times the matrix
with the first eigenvector
37
00:02:34,490 --> 00:02:37,300
in its first column,
the second eigenvector
38
00:02:37,300 --> 00:02:42,510
in its second column, the n-th
eigenvector in its n-th column.
39
00:02:42,510 --> 00:02:46,300
And how I going to do this
matrix multiplication?
40
00:02:46,300 --> 00:02:49,890
Well, certainly I'll do
it a column at a time.
41
00:02:49,890 --> 00:02:52,510
And what do I get.
42
00:02:52,510 --> 00:02:55,480
A times the first column
gives me the first column
43
00:02:55,480 --> 00:02:57,820
of the answer, but what is it?
44
00:02:57,820 --> 00:03:00,010
That's an eigenvector.
45
00:03:00,010 --> 00:03:04,280
A times x1 is equal to
the lambda times the x1.
46
00:03:04,280 --> 00:03:06,870
And that lambda's we're
-- we'll call lambda one,
47
00:03:06,870 --> 00:03:09,590
of course.
48
00:03:09,590 --> 00:03:11,420
So that's the first column.
49
00:03:11,420 --> 00:03:14,000
Ax1 is the same
as lambda one x1.
50
00:03:14,000 --> 00:03:16,730
A x2 is lambda two x2.
51
00:03:16,730 --> 00:03:22,120
So on, along to in the n-th
column we now how lambda n xn.
52
00:03:24,690 --> 00:03:27,110
Looking good, but
the next step is even
53
00:03:27,110 --> 00:03:28,120
better.
54
00:03:28,120 --> 00:03:31,280
So for the next step,
I want to separate out
55
00:03:31,280 --> 00:03:36,100
those eigenvalues, those,
those multiplying numbers,
56
00:03:36,100 --> 00:03:38,880
from the x-s.
57
00:03:38,880 --> 00:03:41,920
So then I'll have
just what I want.
58
00:03:41,920 --> 00:03:42,530
OK.
59
00:03:42,530 --> 00:03:44,990
So how, how I going
to separate out?
60
00:03:44,990 --> 00:03:47,150
So that, that
number lambda one is
61
00:03:47,150 --> 00:03:49,760
multiplying the first column.
62
00:03:49,760 --> 00:03:52,840
So if I want to factor it
out of the first column,
63
00:03:52,840 --> 00:03:55,400
I better put --
64
00:03:55,400 --> 00:03:58,830
here is going to
be x1, and that's
65
00:03:58,830 --> 00:04:02,180
going to multiply
this matrix lambda
66
00:04:02,180 --> 00:04:05,050
one in the first
entry and all zeros.
67
00:04:05,050 --> 00:04:08,230
Do you see that that,
that's going to come out
68
00:04:08,230 --> 00:04:10,220
right for the first column?
69
00:04:10,220 --> 00:04:13,370
Because w- we remember how --
how we're going back to that
70
00:04:13,370 --> 00:04:15,310
original punchline.
71
00:04:15,310 --> 00:04:21,260
That if I want a number
to multiply x1 then
72
00:04:21,260 --> 00:04:24,310
I can do it by putting
x1 in that column,
73
00:04:24,310 --> 00:04:27,280
in the first column, and
putting that number there.
74
00:04:27,280 --> 00:04:28,850
Th- u- what I
going to have here?
75
00:04:28,850 --> 00:04:30,040
I'm going to have lambda --
76
00:04:30,040 --> 00:04:33,800
I'm going to have x1, x2, ...
77
00:04:33,800 --> 00:04:34,890
,xn.
78
00:04:34,890 --> 00:04:37,150
These are going to
be my columns again.
79
00:04:37,150 --> 00:04:39,740
I'm getting S back again.
80
00:04:39,740 --> 00:04:41,380
I'm getting S back again.
81
00:04:41,380 --> 00:04:44,560
But now what's it multiplied
by, on the right it's
82
00:04:44,560 --> 00:04:48,080
multiplied by?
83
00:04:48,080 --> 00:04:54,730
If I want lambda n xn in the
last column, how do I do it?
84
00:04:54,730 --> 00:04:57,670
Well, the last column
here will be --
85
00:04:57,670 --> 00:05:02,630
I'll take the last column,
use these coefficients,
86
00:05:02,630 --> 00:05:05,050
put the lambda n
down there, and it
87
00:05:05,050 --> 00:05:09,970
will multiply that n-th column
and give me lambda n xn.
88
00:05:09,970 --> 00:05:13,130
There, there you see matrix
multiplication just working
89
00:05:13,130 --> 00:05:14,850
for us.
90
00:05:14,850 --> 00:05:17,410
So I started with A S.
91
00:05:17,410 --> 00:05:21,010
I wrote down what it meant,
A times each eigenvector.
92
00:05:21,010 --> 00:05:23,690
That gave me lambda
time the eigenvector.
93
00:05:23,690 --> 00:05:25,700
And then when I peeled
off the lambdas,
94
00:05:25,700 --> 00:05:30,690
they were on the right-hand
side, so I've got S, my matrix,
95
00:05:30,690 --> 00:05:31,750
back again.
96
00:05:31,750 --> 00:05:45,500
And this matrix, this diagonal
matrix, the eigenvalue matrix,
97
00:05:45,500 --> 00:05:49,210
and I call it capital lambda.
98
00:05:49,210 --> 00:05:52,830
Using capital letters
for matrices and lambda
99
00:05:52,830 --> 00:05:54,980
to prompt me that
it's, that it's
100
00:05:54,980 --> 00:05:56,850
eigenvalues that are in there.
101
00:05:56,850 --> 00:05:58,840
So you see that the
eigenvalues are just
102
00:05:58,840 --> 00:06:01,480
sitting down that diagonal?
103
00:06:01,480 --> 00:06:07,340
If I had a column x2 here,
I would want the lambda two
104
00:06:07,340 --> 00:06:10,970
in the two two position,
in the diagonal position,
105
00:06:10,970 --> 00:06:15,140
to multiply that x2 and
give me the lambda two x2.
106
00:06:15,140 --> 00:06:16,730
That's my formula.
107
00:06:16,730 --> 00:06:19,770
A S is S lambda.
108
00:06:23,000 --> 00:06:23,720
OK.
109
00:06:23,720 --> 00:06:27,530
That's the -- you see,
it's just a calculation.
110
00:06:27,530 --> 00:06:31,390
Now -- I mentioned, and
I have to mention again,
111
00:06:31,390 --> 00:06:37,210
this business about n
independent eigenvectors.
112
00:06:37,210 --> 00:06:40,050
As it stands, this is
all fine, whether --
113
00:06:40,050 --> 00:06:43,240
I mean, I could be repeating
the same eigenvector, but --
114
00:06:43,240 --> 00:06:45,550
I'm not interested in that.
115
00:06:45,550 --> 00:06:50,600
I want to be able to invert S,
and that's where this comes in.
116
00:06:50,600 --> 00:06:53,690
This n independent
eigenvectors business
117
00:06:53,690 --> 00:06:56,990
comes in to tell me that
that matrix is invertible.
118
00:06:56,990 --> 00:07:01,380
So let me, on the next board,
write down what I've got.
119
00:07:01,380 --> 00:07:06,780
A S equals S lambda.
120
00:07:06,780 --> 00:07:11,890
And now I'm, I can multiply
on the left by S inverse.
121
00:07:11,890 --> 00:07:13,839
So this is really --
122
00:07:17,970 --> 00:07:20,660
I can do that, provided
S is invertible.
123
00:07:20,660 --> 00:07:26,320
Provided my assumption of n
independent eigenvectors is
124
00:07:26,320 --> 00:07:27,260
satisfied.
125
00:07:27,260 --> 00:07:31,470
And I mentioned at the end of
last time, and I'll say again,
126
00:07:31,470 --> 00:07:35,930
that there's a small
number of matrices for --
127
00:07:35,930 --> 00:07:38,680
that don't have n
independent eigenvectors.
128
00:07:38,680 --> 00:07:42,580
So I've got to discuss
that, that technical point.
129
00:07:42,580 --> 00:07:48,030
But the great -- the most
matrices that we see have n di-
130
00:07:48,030 --> 00:07:51,050
n independent eigenvectors,
and we can diagonalize.
131
00:07:51,050 --> 00:07:54,090
This is diagonalization.
132
00:07:54,090 --> 00:07:57,890
I could also write
it, and I often will,
133
00:07:57,890 --> 00:07:59,680
the other way round.
134
00:07:59,680 --> 00:08:03,140
If I multiply on the
right by S inverse,
135
00:08:03,140 --> 00:08:06,130
if I took this equation at the
top and multiplied on the right
136
00:08:06,130 --> 00:08:08,090
by S inverse, I could --
137
00:08:08,090 --> 00:08:10,900
I would have A left here.
138
00:08:10,900 --> 00:08:15,790
Now S inverse is
coming from the right.
139
00:08:15,790 --> 00:08:18,530
So can you keep
those two straight?
140
00:08:18,530 --> 00:08:21,491
A multiplies its eigenvectors,
that's how I keep them
141
00:08:21,491 --> 00:08:21,990
straight.
142
00:08:21,990 --> 00:08:24,530
So A multiplies S.
143
00:08:24,530 --> 00:08:26,020
A multiplies S.
144
00:08:26,020 --> 00:08:30,700
And then this S inverse makes
the whole thing diagonal.
145
00:08:30,700 --> 00:08:33,750
And this is another way
of saying the same thing,
146
00:08:33,750 --> 00:08:36,700
putting the Ss on the
other side of the equation.
147
00:08:36,700 --> 00:08:40,179
A is S lambda S inverse.
148
00:08:40,179 --> 00:08:44,520
So that's the, that's
the new factorization.
149
00:08:44,520 --> 00:08:50,550
That's the replacement for L U
from elimination or Q R for --
150
00:08:50,550 --> 00:08:52,210
from Gram-Schmidt.
151
00:08:52,210 --> 00:08:56,850
And notice that the matrix --
so it's, it's a matrix times
152
00:08:56,850 --> 00:09:00,420
a diagonal matrix times the
inverse of the first one.
153
00:09:00,420 --> 00:09:02,410
It's, that's the
combination that we'll
154
00:09:02,410 --> 00:09:05,530
see throughout this chapter.
155
00:09:05,530 --> 00:09:09,650
This combination with
an S and an S inverse.
156
00:09:09,650 --> 00:09:10,250
OK.
157
00:09:10,250 --> 00:09:14,080
Can I just begin to use that?
158
00:09:14,080 --> 00:09:17,730
For example, what
about A squared?
159
00:09:17,730 --> 00:09:20,880
What are the eigenvalues and
eigenvectors of A squared?
160
00:09:20,880 --> 00:09:23,040
That's a straightforward
question with a,
161
00:09:23,040 --> 00:09:25,380
with an absolutely clean answer.
162
00:09:25,380 --> 00:09:28,930
So let me, let me
consider A squared.
163
00:09:28,930 --> 00:09:33,560
So I start with A
x equal lambda x.
164
00:09:33,560 --> 00:09:36,070
And I'm headed for A squared.
165
00:09:36,070 --> 00:09:38,610
So let me multiply
both sides by A.
166
00:09:38,610 --> 00:09:41,290
That's one way to get
A squared on the left.
167
00:09:41,290 --> 00:09:46,190
So -- I should write
these if-s in here.
168
00:09:46,190 --> 00:09:51,010
If A x equals lambda x,
then I multiply by A,
169
00:09:51,010 --> 00:09:56,200
so I get A squared x equals
-- well, I'm multiplying by A,
170
00:09:56,200 --> 00:09:58,670
so that's lambda A x.
171
00:09:58,670 --> 00:10:02,520
That lambda was a number, so
I just put it on the left.
172
00:10:02,520 --> 00:10:06,740
And what do I -- tell me how
to make that look better.
173
00:10:06,740 --> 00:10:10,220
What have I got
here for if, if A
174
00:10:10,220 --> 00:10:13,220
has the eigenvalue
lambda and eigenvector
175
00:10:13,220 --> 00:10:16,010
x, what's up with A squared?
176
00:10:16,010 --> 00:10:18,430
A squared x, I just
multiplied by A,
177
00:10:18,430 --> 00:10:23,070
but now for Ax I'm going
to substitute lambda x.
178
00:10:23,070 --> 00:10:25,010
So I've got lambda squared x.
179
00:10:28,120 --> 00:10:31,370
So from that simple
calculation, I --
180
00:10:31,370 --> 00:10:37,330
my conclusion is that the
eigenvalues of A squared are
181
00:10:37,330 --> 00:10:39,310
lambda squared.
182
00:10:39,310 --> 00:10:40,384
And the eigenvectors --
183
00:10:40,384 --> 00:10:41,550
I always think about both of
184
00:10:41,550 --> 00:10:42,150
those.
185
00:10:42,150 --> 00:10:44,390
What can I say about
the eigenvalues?
186
00:10:44,390 --> 00:10:45,560
They're squared.
187
00:10:45,560 --> 00:10:48,220
What can I say about
the eigenvectors?
188
00:10:48,220 --> 00:10:49,730
They're the same.
189
00:10:49,730 --> 00:10:54,090
The same x as in -- as for A.
190
00:10:54,090 --> 00:11:00,320
Now let me see that
also from this formula.
191
00:11:00,320 --> 00:11:05,160
How can I see what A squared is
looking like from this formula?
192
00:11:05,160 --> 00:11:08,480
So let me -- that
was one way to do it.
193
00:11:08,480 --> 00:11:12,510
Let me do it by just
taking A squared from that.
194
00:11:12,510 --> 00:11:18,990
A squared is S lambda S
inverse -- that's A --
195
00:11:18,990 --> 00:11:22,620
times S lambda S inverse
-- that's A, which is?
196
00:11:25,530 --> 00:11:30,170
This is the beauty of
eigenvalues, eigenvectors.
197
00:11:30,170 --> 00:11:34,000
Having that S inverse
and S is the identity,
198
00:11:34,000 --> 00:11:40,220
so I've got S lambda
squared S inverse.
199
00:11:40,220 --> 00:11:43,380
Do you see what
that's telling me?
200
00:11:43,380 --> 00:11:46,570
It's, it's telling me the same
thing that I just learned here,
201
00:11:46,570 --> 00:11:49,630
but in the -- in a matrix form.
202
00:11:49,630 --> 00:11:52,720
It's telling me that
the S is the same,
203
00:11:52,720 --> 00:11:57,130
the eigenvectors are the
same, but the eigenvalues
204
00:11:57,130 --> 00:11:58,530
are squared.
205
00:11:58,530 --> 00:12:01,020
Because this is --
what's lambda squared?
206
00:12:01,020 --> 00:12:02,800
That's still diagonal.
207
00:12:02,800 --> 00:12:05,180
It's got little
lambda one squared,
208
00:12:05,180 --> 00:12:07,190
lambda two squared,
down to lambda n
209
00:12:07,190 --> 00:12:09,280
squared o- on that diagonal.
210
00:12:09,280 --> 00:12:13,760
Those are the eigenvalues, as
we just learned, of A squared.
211
00:12:13,760 --> 00:12:14,500
OK.
212
00:12:14,500 --> 00:12:19,590
So -- somehow those eigenvalues
and eigenvectors are really
213
00:12:19,590 --> 00:12:25,060
giving you a way to --
214
00:12:25,060 --> 00:12:29,460
see what's going
on inside a matrix.
215
00:12:29,460 --> 00:12:32,100
Of course I can
continue that for --
216
00:12:32,100 --> 00:12:36,270
to the K-th power,
A to the K-th power.
217
00:12:36,270 --> 00:12:39,370
If I multiply, if I have
K of these together,
218
00:12:39,370 --> 00:12:42,920
do you see how S inverse
S will keep canceling
219
00:12:42,920 --> 00:12:44,900
in the, in the inside?
220
00:12:44,900 --> 00:12:48,740
I'll have the S outside
at the far left,
221
00:12:48,740 --> 00:12:54,390
and lambda will be in there
K times, and S inverse.
222
00:12:54,390 --> 00:12:56,670
So what's that telling me?
223
00:12:56,670 --> 00:12:59,420
That's telling me
that the eigenvalues
224
00:12:59,420 --> 00:13:04,340
of A, of A to the K-th
power are the K-th powers.
225
00:13:04,340 --> 00:13:08,390
The eigenvalues of A cubed are
the cubes of the eigenvalues of
226
00:13:08,390 --> 00:13:15,000
A. And the eigenvectors
are the same, the same.
227
00:13:15,000 --> 00:13:15,600
OK.
228
00:13:15,600 --> 00:13:20,750
In other words, eigenvalues
and eigenvectors
229
00:13:20,750 --> 00:13:25,910
give a great way to understand
the powers of a matrix.
230
00:13:25,910 --> 00:13:28,380
If I take the
square of a matrix,
231
00:13:28,380 --> 00:13:30,540
or the hundredth
power of a matrix,
232
00:13:30,540 --> 00:13:34,780
the pivots are all
over the place.
233
00:13:34,780 --> 00:13:39,690
L U, if I multiply L U times
L U times L U times L U
234
00:13:39,690 --> 00:13:44,610
a hundred times, I've
got a hundred L Us.
235
00:13:44,610 --> 00:13:46,370
I can't do anything with them.
236
00:13:46,370 --> 00:13:50,180
But when I multiply S
lambda S inverse by itself,
237
00:13:50,180 --> 00:13:54,720
when I look at the eigenvector
picture a hundred times,
238
00:13:54,720 --> 00:13:59,810
I get a hundred or ninety-nine
of these guys canceling out
239
00:13:59,810 --> 00:14:03,390
inside, and I get
A to the hundredth
240
00:14:03,390 --> 00:14:05,900
is S lambda to the
hundredth S inverse.
241
00:14:05,900 --> 00:14:09,870
I mean, eigenvalues
tell you about powers
242
00:14:09,870 --> 00:14:16,290
of a matrix in a way that we had
no way to approach previously.
243
00:14:16,290 --> 00:14:21,220
For example, when does --
244
00:14:21,220 --> 00:14:24,790
when do the powers of
a matrix go to zero?
245
00:14:24,790 --> 00:14:29,330
I would call that
matrix stable, maybe.
246
00:14:29,330 --> 00:14:31,940
So I could write down a theorem.
247
00:14:31,940 --> 00:14:36,720
I'll write it as a theorem
just to use that word
248
00:14:36,720 --> 00:14:40,350
to emphasize that here I'm
getting this great fact
249
00:14:40,350 --> 00:14:42,610
from this eigenvalue picture.
250
00:14:42,610 --> 00:14:43,230
OK.
251
00:14:43,230 --> 00:14:53,720
A to the K approaches zero as K
goes, as K gets bigger if what?
252
00:14:53,720 --> 00:14:57,840
What's the w- how can
I tell, for a matrix A,
253
00:14:57,840 --> 00:14:59,850
if its powers go to zero?
254
00:15:02,650 --> 00:15:07,520
What's -- somewhere inside that
matrix is that information.
255
00:15:07,520 --> 00:15:11,510
That information is not
present in the pivots.
256
00:15:11,510 --> 00:15:13,270
It's present in the eigenvalues.
257
00:15:13,270 --> 00:15:17,010
What do I need for the -- to
know that if I take higher
258
00:15:17,010 --> 00:15:20,170
and higher powers of A, that
this matrix gets smaller
259
00:15:20,170 --> 00:15:21,030
and smaller?
260
00:15:21,030 --> 00:15:24,330
Well, S and S inverse
are not moving.
261
00:15:24,330 --> 00:15:26,810
So it's this guy that
has to get small.
262
00:15:26,810 --> 00:15:29,720
And that's easy to
-- to understand.
263
00:15:29,720 --> 00:15:32,595
The requirement is
all eigenvalues --
264
00:15:35,330 --> 00:15:38,040
so what is the requirement?
265
00:15:38,040 --> 00:15:41,170
The eigenvalues have
to be less than one.
266
00:15:41,170 --> 00:15:45,310
Now I have to wrote
that absolute value,
267
00:15:45,310 --> 00:15:48,360
because those eigenvalues
could be negative,
268
00:15:48,360 --> 00:15:50,550
they could be complex numbers.
269
00:15:50,550 --> 00:15:53,170
So I'm taking the
absolute value.
270
00:15:53,170 --> 00:15:56,750
If all of those are below one.
271
00:15:56,750 --> 00:16:05,000
That's, in fact, we
practically see why.
272
00:16:05,000 --> 00:16:13,100
And let me just say that I'm
operating on one assumption
273
00:16:13,100 --> 00:16:15,740
here, and I got to
keep remembering
274
00:16:15,740 --> 00:16:18,690
that that assumption
is still present.
275
00:16:18,690 --> 00:16:21,420
That assumption was that
I had a full set of,
276
00:16:21,420 --> 00:16:24,370
of n independent eigenvectors.
277
00:16:24,370 --> 00:16:30,830
If I don't have that, then
this approach is not working.
278
00:16:30,830 --> 00:16:37,090
So again, a pure eigenvalue
approach, eigenvector approach,
279
00:16:37,090 --> 00:16:40,470
needs n independent
eigenvectors.
280
00:16:40,470 --> 00:16:42,900
If we don't have n
independent eigenvectors,
281
00:16:42,900 --> 00:16:46,490
we can't diagonalize the matrix.
282
00:16:46,490 --> 00:16:50,820
We can't get to a
diagonal matrix.
283
00:16:50,820 --> 00:16:55,960
This diagonalization
is only possible
284
00:16:55,960 --> 00:16:58,581
if S inverse makes sense.
285
00:16:58,581 --> 00:16:59,080
OK.
286
00:16:59,080 --> 00:17:02,800
Can I, can I follow
up on that point now?
287
00:17:02,800 --> 00:17:07,099
So you see why -- what we
get and, and why we want it,
288
00:17:07,099 --> 00:17:11,490
because we get information about
the powers of a matrix just
289
00:17:11,490 --> 00:17:14,940
immediately from
the eigenvalues.
290
00:17:14,940 --> 00:17:15,500
OK.
291
00:17:15,500 --> 00:17:22,329
Now let me follow up on this,
business of which matrices
292
00:17:22,329 --> 00:17:25,030
are diagonalizable.
293
00:17:25,030 --> 00:17:28,220
Sorry about that long word.
294
00:17:28,220 --> 00:17:32,940
So a matrix is, is sure -- so
here's, here's the main point.
295
00:17:32,940 --> 00:17:37,600
A is sure to be --
296
00:17:37,600 --> 00:17:50,110
to have N independent
eigenvectors and, and be --
297
00:17:50,110 --> 00:18:00,210
now here comes that word
-- diagonalizable if, if --
298
00:18:00,210 --> 00:18:05,810
so we might as well get the
nice case out in the open.
299
00:18:05,810 --> 00:18:13,330
The nice case is when -- if
all the lambdas are different.
300
00:18:19,520 --> 00:18:28,326
That means, that means
no repeated eigenvalues.
301
00:18:30,920 --> 00:18:32,000
OK.
302
00:18:32,000 --> 00:18:34,410
That's the nice case.
303
00:18:34,410 --> 00:18:39,970
If my matrix, and most -- if
I do a random matrix in Matlab
304
00:18:39,970 --> 00:18:43,140
and compute its eigenvalues --
305
00:18:43,140 --> 00:18:55,540
so if I computed if I took
eig of rand of ten ten, gave,
306
00:18:55,540 --> 00:18:58,640
gave that Matlab command, the --
307
00:18:58,640 --> 00:19:01,190
we'd get a random
ten by ten matrix,
308
00:19:01,190 --> 00:19:03,990
we would get a list of
its ten eigenvalues,
309
00:19:03,990 --> 00:19:08,040
and they would be different.
310
00:19:08,040 --> 00:19:10,470
They would be distinct
is the best word.
311
00:19:10,470 --> 00:19:13,880
I would have -- a random matrix
will have ten distinct --
312
00:19:13,880 --> 00:19:18,090
a ten by ten matrix will have
ten distinct eigenvalues.
313
00:19:18,090 --> 00:19:25,500
And if it does, the eigenvectors
are automatically independent.
314
00:19:25,500 --> 00:19:26,930
So that's a nice fact.
315
00:19:26,930 --> 00:19:29,730
I'll refer you to the
text for the proof.
316
00:19:29,730 --> 00:19:36,360
That, that A is sure to have
n independent eigenvectors
317
00:19:36,360 --> 00:19:41,610
if the eigenvalues
are different, if.
318
00:19:41,610 --> 00:19:43,890
If all the, if all
eigenvalues are different.
319
00:19:43,890 --> 00:19:47,810
It's just if some
lambdas are repeated,
320
00:19:47,810 --> 00:19:50,560
then I have to
look more closely.
321
00:19:50,560 --> 00:19:55,050
If an eigenvalue is repeated, I
have to look, I have to count,
322
00:19:55,050 --> 00:19:56,260
I have to check.
323
00:19:56,260 --> 00:19:59,560
Has it got -- say it's
repeated three times.
324
00:19:59,560 --> 00:20:02,030
So what's a
possibility for the --
325
00:20:02,030 --> 00:20:05,893
so here is the, here is
the repeated possibility.
326
00:20:11,000 --> 00:20:16,490
And, and let me
emphasize the conclusion.
327
00:20:16,490 --> 00:20:21,410
That if I have repeated
eigenvalues, I may or may not,
328
00:20:21,410 --> 00:20:35,271
I may or may not have, have
n independent eigenvectors.
329
00:20:35,271 --> 00:20:35,770
I might.
330
00:20:35,770 --> 00:20:41,240
I, I, you know, this isn't
a completely negative case.
331
00:20:41,240 --> 00:20:43,260
The identity matrix --
332
00:20:43,260 --> 00:20:46,380
suppose I take the ten
by ten identity matrix.
333
00:20:46,380 --> 00:20:50,750
What are the eigenvalues
of that matrix?
334
00:20:50,750 --> 00:20:55,370
So just, just take the
easiest matrix, the identity.
335
00:20:55,370 --> 00:21:00,470
If I look for its
eigenvalues, they're all ones.
336
00:21:00,470 --> 00:21:04,410
So that eigenvalue one
is repeated ten times.
337
00:21:04,410 --> 00:21:07,340
But there's no shortage of
eigenvectors for the identity
338
00:21:07,340 --> 00:21:08,250
matrix.
339
00:21:08,250 --> 00:21:10,760
In fact, every vector
is an eigenvector.
340
00:21:10,760 --> 00:21:13,610
So I can take ten
independent vectors.
341
00:21:13,610 --> 00:21:16,530
Oh, well, what happens
to everything --
342
00:21:16,530 --> 00:21:18,590
if A is the identity
matrix, let's
343
00:21:18,590 --> 00:21:21,650
just think that one
through in our head.
344
00:21:21,650 --> 00:21:27,440
If A is the identity
matrix, then it's
345
00:21:27,440 --> 00:21:28,830
got plenty of eigenvectors.
346
00:21:28,830 --> 00:21:30,910
I choose ten
independent vectors.
347
00:21:30,910 --> 00:21:32,560
They're the columns of S.
348
00:21:32,560 --> 00:21:37,380
And, and what do I get
from S inverse A S?
349
00:21:37,380 --> 00:21:39,400
I get I again, right?
350
00:21:39,400 --> 00:21:42,210
If A is the identity -- and
of course that's the correct
351
00:21:42,210 --> 00:21:43,790
lambda.
352
00:21:43,790 --> 00:21:46,790
The matrix was already diagonal.
353
00:21:46,790 --> 00:21:48,970
So if the matrix is
already diagonal,
354
00:21:48,970 --> 00:21:53,800
then the, the lambda is
the same as the matrix.
355
00:21:53,800 --> 00:21:56,380
A diagonal matrix has
got its eigenvalues
356
00:21:56,380 --> 00:21:59,170
sitting right there
in front of you.
357
00:21:59,170 --> 00:22:01,790
Now if it's triangular,
the eigenvalues
358
00:22:01,790 --> 00:22:04,820
are still sitting
there, but so let's
359
00:22:04,820 --> 00:22:08,460
take a case where
it's triangular.
360
00:22:08,460 --> 00:22:14,870
Suppose A is like,
two one two zero.
361
00:22:17,920 --> 00:22:23,290
So there's a case that's
going to be trouble.
362
00:22:23,290 --> 00:22:25,040
There's a case that's
going to be trouble.
363
00:22:25,040 --> 00:22:26,360
First of all, what are the --
364
00:22:26,360 --> 00:22:29,410
I mean, we just --
365
00:22:29,410 --> 00:22:31,190
if we start with a
matrix, the first thing
366
00:22:31,190 --> 00:22:32,890
we do, practically
without thinking
367
00:22:32,890 --> 00:22:36,130
is compute the eigenvalues
and eigenvectors.
368
00:22:36,130 --> 00:22:36,630
OK.
369
00:22:36,630 --> 00:22:38,360
So what are the eigenvalues?
370
00:22:38,360 --> 00:22:41,130
You can tell me right
away what they are.
371
00:22:41,130 --> 00:22:43,880
They're two and two, right.
372
00:22:43,880 --> 00:22:47,770
It's a triangular matrix, so
when I do this determinant,
373
00:22:47,770 --> 00:22:51,680
shall I do this determinant
of A minus lambda I?
374
00:22:51,680 --> 00:22:59,990
I'll get this two minus lambda
one zero two minus lambda,
375
00:22:59,990 --> 00:23:01,700
right?
376
00:23:01,700 --> 00:23:06,890
I take that determinant, so I
make those into vertical bars
377
00:23:06,890 --> 00:23:09,130
to mean determinant.
378
00:23:09,130 --> 00:23:10,810
And what's the determinant?
379
00:23:10,810 --> 00:23:13,140
It's two minus lambda squared.
380
00:23:13,140 --> 00:23:14,570
What are the roots?
381
00:23:14,570 --> 00:23:17,410
Lambda equal two twice.
382
00:23:17,410 --> 00:23:22,640
So the eigenvalues are
lambda equals two and two.
383
00:23:22,640 --> 00:23:23,580
OK, fine.
384
00:23:23,580 --> 00:23:26,810
Now the next step,
find the eigenvectors.
385
00:23:26,810 --> 00:23:31,940
So I look for eigenvectors, and
what do I find for this guy?
386
00:23:31,940 --> 00:23:33,530
Eigenvectors for
this guy, when I
387
00:23:33,530 --> 00:23:38,340
subtract two minus the
identity, so A minus two
388
00:23:38,340 --> 00:23:42,280
I has zeros here.
389
00:23:45,420 --> 00:23:48,740
And I'm looking
for the null space.
390
00:23:48,740 --> 00:23:50,390
What's, what are
the eigenvectors?
391
00:23:50,390 --> 00:23:56,540
They're the -- the null
space of A minus lambda I.
392
00:23:56,540 --> 00:23:59,200
The null space is
only one dimensional.
393
00:23:59,200 --> 00:24:03,700
This is a case where I don't
have enough eigenvectors.
394
00:24:03,700 --> 00:24:07,940
My algebraic
multiplicity is two.
395
00:24:07,940 --> 00:24:10,520
I would say, when
I see, when I count
396
00:24:10,520 --> 00:24:16,210
how often the
eigenvalue is repeated,
397
00:24:16,210 --> 00:24:18,410
that's the algebraic
multiplicity.
398
00:24:18,410 --> 00:24:20,650
That's the multiplicity,
how many times
399
00:24:20,650 --> 00:24:22,770
is it the root of
the polynomial?
400
00:24:22,770 --> 00:24:28,340
My polynomial is two
minus lambda squared.
401
00:24:28,340 --> 00:24:30,040
It's a double root.
402
00:24:30,040 --> 00:24:33,240
So my algebraic
multiplicity is two.
403
00:24:33,240 --> 00:24:37,110
But the geometric multiplicity,
which looks for vectors,
404
00:24:37,110 --> 00:24:42,130
looks for eigenvectors, and
-- which means the null space
405
00:24:42,130 --> 00:24:46,500
of this thing, and the
only eigenvector is one
406
00:24:46,500 --> 00:24:47,360
zero.
407
00:24:47,360 --> 00:24:50,140
That's in the null space.
408
00:24:50,140 --> 00:24:52,600
Zero one is not
in the null space.
409
00:24:52,600 --> 00:24:54,540
The null space is
only one dimensional.
410
00:24:54,540 --> 00:24:58,960
So there's a matrix, my --
this A or the original A,
411
00:24:58,960 --> 00:25:02,310
that are not diagonalizable.
412
00:25:02,310 --> 00:25:06,360
I can't find two
independent eigenvectors.
413
00:25:06,360 --> 00:25:08,090
There's only one.
414
00:25:08,090 --> 00:25:08,700
OK.
415
00:25:08,700 --> 00:25:11,710
So that's the case that I'm --
416
00:25:11,710 --> 00:25:15,520
that's a case that I'm
not really handling.
417
00:25:15,520 --> 00:25:19,590
For example, when I
wrote down up here
418
00:25:19,590 --> 00:25:24,490
that the powers went to zero if
the eigenvalues were below one,
419
00:25:24,490 --> 00:25:29,210
I didn't really handle that
case of repeated eigenvalues,
420
00:25:29,210 --> 00:25:33,980
because my reasoning was
based on this formula.
421
00:25:33,980 --> 00:25:36,420
And this formula is based on
n independent eigenvectors.
422
00:25:36,420 --> 00:25:36,920
OK.
423
00:25:36,920 --> 00:25:45,610
Just to say then, there are
some matrices that we're, that,
424
00:25:45,610 --> 00:25:48,730
that we don't cover
through diagonalization,
425
00:25:48,730 --> 00:25:51,070
but the great majority we do.
426
00:25:51,070 --> 00:25:51,720
OK.
427
00:25:51,720 --> 00:25:54,040
And we, we're
always OK if we have
428
00:25:54,040 --> 00:25:56,550
different distinct eigenvalues.
429
00:25:56,550 --> 00:26:02,390
OK, that's the, like,
the typical case.
430
00:26:02,390 --> 00:26:04,660
Because for each
eigenvalue there's
431
00:26:04,660 --> 00:26:07,140
at least one eigenvector.
432
00:26:07,140 --> 00:26:11,530
The algebraic multiplicity here
is one for every eigenvalue
433
00:26:11,530 --> 00:26:14,080
and the geometric
multiplicity is one.
434
00:26:14,080 --> 00:26:15,580
There's one eigenvector.
435
00:26:15,580 --> 00:26:17,650
And they are independent.
436
00:26:17,650 --> 00:26:18,150
OK.
437
00:26:18,150 --> 00:26:18,650
OK.
438
00:26:21,690 --> 00:26:26,390
Now let me come back to
the important case, when,
439
00:26:26,390 --> 00:26:27,770
when we're OK.
440
00:26:27,770 --> 00:26:31,820
The important case, when
we are diagonalizable.
441
00:26:31,820 --> 00:26:38,060
Let me, look at --
442
00:26:38,060 --> 00:26:42,455
so -- let me solve
this equation.
443
00:26:46,680 --> 00:26:49,460
The equation will be each --
444
00:26:49,460 --> 00:26:57,146
I start with some -- start
with a given vector u0.
445
00:27:02,390 --> 00:27:06,080
And then my equation
is at every step,
446
00:27:06,080 --> 00:27:11,600
I multiply what I have by A.
447
00:27:11,600 --> 00:27:16,550
That, that equation ought
to be simple to handle.
448
00:27:19,940 --> 00:27:21,940
And I'd like to be
able to solve it.
449
00:27:21,940 --> 00:27:26,840
How would I find -- if I start
with a vector u0 and I multiply
450
00:27:26,840 --> 00:27:31,470
by A a hundred times,
what have I got?
451
00:27:31,470 --> 00:27:35,310
Well, I could certainly write
down a formula for the answer,
452
00:27:35,310 --> 00:27:39,295
so what, what -- so u1 is A u0.
453
00:27:42,170 --> 00:27:45,800
And u2 is -- what's u2 then?
454
00:27:45,800 --> 00:27:52,350
u2, I multiply -- u2 I get from
u1 by another multiplying by A,
455
00:27:52,350 --> 00:27:55,830
so I've got A twice.
456
00:27:55,830 --> 00:28:02,120
And my formula is
uk, after k steps,
457
00:28:02,120 --> 00:28:07,580
I've multiplied by A k
times the original u0.
458
00:28:07,580 --> 00:28:11,220
You see what I'm doing?
459
00:28:11,220 --> 00:28:14,050
The next section is
going to solve systems
460
00:28:14,050 --> 00:28:17,550
of differential equations.
461
00:28:17,550 --> 00:28:19,690
I'm going to have derivatives.
462
00:28:19,690 --> 00:28:23,370
This section is the nice one.
463
00:28:23,370 --> 00:28:26,190
It solves difference equations.
464
00:28:26,190 --> 00:28:28,550
I would call that a
difference equation.
465
00:28:28,550 --> 00:28:33,550
It's -- at first order, I would
call that a first-order system,
466
00:28:33,550 --> 00:28:40,150
because it connects only --
it only goes up one level.
467
00:28:40,150 --> 00:28:43,180
And I -- it's a system
because these are vectors
468
00:28:43,180 --> 00:28:45,960
and that's a matrix.
469
00:28:45,960 --> 00:28:48,470
And the solution is just that.
470
00:28:48,470 --> 00:28:49,090
OK.
471
00:28:49,090 --> 00:28:55,160
But, that's a nice formula.
472
00:28:55,160 --> 00:28:57,500
That's the, like, the
most compact formula
473
00:28:57,500 --> 00:29:01,630
I could ever get. u100 would
be A to the one hundred u0.
474
00:29:01,630 --> 00:29:06,480
But how would I
actually find u100?
475
00:29:06,480 --> 00:29:11,520
How would I find -- how would
I discover what u100 is?
476
00:29:11,520 --> 00:29:13,760
Let me, let me show you how.
477
00:29:16,620 --> 00:29:18,630
Here's the idea.
478
00:29:18,630 --> 00:29:23,090
If -- so to solve, to
really solve -- shall I say,
479
00:29:23,090 --> 00:29:26,920
to really solve --
480
00:29:26,920 --> 00:29:34,500
to really solve it, I would
take this initial vector u0
481
00:29:34,500 --> 00:29:39,420
and I would write it as a
combination of eigenvectors.
482
00:29:39,420 --> 00:29:47,320
To really solve, write u
nought as a combination,
483
00:29:47,320 --> 00:29:50,660
say certain amount of
the first eigenvector
484
00:29:50,660 --> 00:29:53,480
plus a certain amount of
the second eigenvector
485
00:29:53,480 --> 00:29:55,820
plus a certain amount
of the last eigenvector.
486
00:30:01,790 --> 00:30:04,740
Now multiply by A.
487
00:30:04,740 --> 00:30:07,350
You want to -- you got to
see the magic of eigenvectors
488
00:30:07,350 --> 00:30:08,520
working here.
489
00:30:08,520 --> 00:30:10,360
Multiply by A.
490
00:30:10,360 --> 00:30:13,910
So Au0 is what?
491
00:30:13,910 --> 00:30:16,930
So A times that.
492
00:30:16,930 --> 00:30:18,800
A times -- so what's A --
493
00:30:18,800 --> 00:30:21,390
I can separate it out
into n separate pieces,
494
00:30:21,390 --> 00:30:23,430
and that's the whole point.
495
00:30:23,430 --> 00:30:28,800
That each of those pieces is
going in its own merry way.
496
00:30:28,800 --> 00:30:31,320
Each of those pieces
is an eigenvector,
497
00:30:31,320 --> 00:30:35,810
and when I multiply by A,
what does this piece become?
498
00:30:35,810 --> 00:30:38,450
So that's some amount
of the first --
499
00:30:38,450 --> 00:30:41,030
let's suppose the eigenvectors
are normalized to be unit
500
00:30:41,030 --> 00:30:41,530
vectors.
501
00:30:44,750 --> 00:30:48,530
So that says what
the eigenvector is.
502
00:30:48,530 --> 00:30:51,340
It's a --
503
00:30:51,340 --> 00:30:55,220
And I need some multiple
of it to produce u0.
504
00:30:55,220 --> 00:30:56,120
OK.
505
00:30:56,120 --> 00:30:59,470
Now when I multiply
by A, what do I get?
506
00:30:59,470 --> 00:31:04,350
I get c1, which is just
a factor, times Ax1,
507
00:31:04,350 --> 00:31:07,865
but Ax1 is lambda one x1.
508
00:31:10,780 --> 00:31:17,060
When I multiply this by
A, I get c2 lambda two x2.
509
00:31:17,060 --> 00:31:20,740
And here I get cn lambda n xn.
510
00:31:20,740 --> 00:31:27,980
And suppose I multiply by A
to the hundredth power now.
511
00:31:27,980 --> 00:31:30,840
Can we, having done it,
multiplied by A, let's
512
00:31:30,840 --> 00:31:32,890
multiply by A to the hundredth.
513
00:31:32,890 --> 00:31:36,380
What happens to this first term
when I multiply by A to the one
514
00:31:36,380 --> 00:31:38,130
hundredth?
515
00:31:38,130 --> 00:31:41,620
It's got that factor
lambda to the hundredth.
516
00:31:41,620 --> 00:31:42,890
That's the key.
517
00:31:42,890 --> 00:31:48,440
That -- that's what I mean
by going its own merry way.
518
00:31:48,440 --> 00:31:52,320
It, it is pure eigenvector.
519
00:31:52,320 --> 00:31:55,850
It's exactly in a direction
where multiplication by A
520
00:31:55,850 --> 00:31:59,200
just brings in a scalar
factor, lambda one.
521
00:31:59,200 --> 00:32:02,240
So a hundred times brings
in this a hundred times.
522
00:32:02,240 --> 00:32:06,080
Hundred times lambda two,
hundred times lambda n.
523
00:32:06,080 --> 00:32:08,830
Actually, we're -- what
are we seeing here?
524
00:32:08,830 --> 00:32:15,040
We're seeing, this
same, lambda capital
525
00:32:15,040 --> 00:32:19,570
lambda to the hundredth as in
the, as in the diagonalization.
526
00:32:19,570 --> 00:32:22,350
And we're seeing
the S matrix, the,
527
00:32:22,350 --> 00:32:24,730
the matrix S of eigenvectors.
528
00:32:24,730 --> 00:32:29,440
That's what this has got to
-- this has got to amount to.
529
00:32:29,440 --> 00:32:40,030
A lambda to the hundredth power
times an S times this vector c
530
00:32:40,030 --> 00:32:43,490
that's telling us
how much of each one
531
00:32:43,490 --> 00:32:45,010
is in the original thing.
532
00:32:45,010 --> 00:32:49,010
So if, if I had to really
find the hundredth power,
533
00:32:49,010 --> 00:32:54,200
I would take u0, I would
expand it as a combination
534
00:32:54,200 --> 00:32:57,210
of eigenvectors --
this is really S,
535
00:32:57,210 --> 00:33:01,680
the eigenvector matrix, times
c, the, the coefficient vector.
536
00:33:04,240 --> 00:33:07,310
And then I would
immediately then,
537
00:33:07,310 --> 00:33:10,950
by inserting these hundredth
powers of eigenvalues,
538
00:33:10,950 --> 00:33:15,490
I'd have the answer.
539
00:33:15,490 --> 00:33:17,880
So -- huh, there must be --
540
00:33:17,880 --> 00:33:20,570
oh, let's see, OK.
541
00:33:20,570 --> 00:33:22,970
It's -- so, yeah.
542
00:33:22,970 --> 00:33:30,790
So if u100 is A to the hundredth
times u0, and u0 is S c --
543
00:33:30,790 --> 00:33:36,160
then you see this formula
is just this formula,
544
00:33:36,160 --> 00:33:40,840
which is the way I would
actually get hold of this,
545
00:33:40,840 --> 00:33:44,690
of this u100, which is --
546
00:33:44,690 --> 00:33:47,180
let me put it here.
547
00:33:47,180 --> 00:33:48,030
u100.
548
00:33:48,030 --> 00:33:51,070
The way I would actually
get hold of that, see what,
549
00:33:51,070 --> 00:33:57,400
what the solution is after
a hundred steps, would be --
550
00:33:57,400 --> 00:34:05,960
expand the initial vector
into eigenvectors and let each
551
00:34:05,960 --> 00:34:10,020
eigenvector go its own way,
multiplying by a hundred at --
552
00:34:10,020 --> 00:34:13,400
by lambda at every step,
and therefore by lambda
553
00:34:13,400 --> 00:34:16,030
to the hundredth power
after a hundred steps.
554
00:34:16,030 --> 00:34:18,050
Can I do an example?
555
00:34:18,050 --> 00:34:20,260
So that's the formulas.
556
00:34:20,260 --> 00:34:22,540
Now let me take an example.
557
00:34:22,540 --> 00:34:29,090
I'll use the Fibonacci
sequence as an example.
558
00:34:29,090 --> 00:34:31,590
So, so Fibonacci example.
559
00:34:39,830 --> 00:34:43,050
You remember the
Fibonacci numbers?
560
00:34:43,050 --> 00:34:48,150
If we start with one
and one as F0 -- oh,
561
00:34:48,150 --> 00:34:50,280
I think I start
with zero, maybe.
562
00:34:50,280 --> 00:34:54,550
Let zero and one
be the first ones.
563
00:34:54,550 --> 00:34:58,550
So there's F0 and F1, the
first two Fibonacci numbers.
564
00:34:58,550 --> 00:35:02,840
Then what's the rule
for Fibonacci numbers?
565
00:35:02,840 --> 00:35:04,130
Ah, they're the sum.
566
00:35:04,130 --> 00:35:08,030
The next one is the sum
of those, so it's one.
567
00:35:08,030 --> 00:35:11,110
The next one is the sum
of those, so it's two.
568
00:35:11,110 --> 00:35:14,010
The next one is the sum
of those, so it's three.
569
00:35:14,010 --> 00:35:16,190
Well, it looks like one
two three four five,
570
00:35:16,190 --> 00:35:19,350
but somehow it's not
going to do that way.
571
00:35:19,350 --> 00:35:21,380
The next one is five, right.
572
00:35:21,380 --> 00:35:22,640
Two and three makes five.
573
00:35:22,640 --> 00:35:26,090
The next one is eight.
574
00:35:26,090 --> 00:35:28,370
The next one is thirteen.
575
00:35:28,370 --> 00:35:33,245
And the one hundredth
Fibonacci number is what?
576
00:35:35,920 --> 00:35:37,600
That's my question.
577
00:35:37,600 --> 00:35:40,680
How could I get a formula
for the hundredth number?
578
00:35:40,680 --> 00:35:44,470
And, for example, how could
I answer the question,
579
00:35:44,470 --> 00:35:47,740
how fast are they growing?
580
00:35:47,740 --> 00:35:52,650
How fast are those
Fibonacci numbers growing?
581
00:35:52,650 --> 00:35:54,070
They're certainly growing.
582
00:35:54,070 --> 00:35:56,270
It's not a stable case.
583
00:35:56,270 --> 00:35:59,030
Whatever the eigenvalues
of whatever matrix it is,
584
00:35:59,030 --> 00:36:00,720
they're not smaller than one.
585
00:36:00,720 --> 00:36:02,540
These numbers are growing.
586
00:36:02,540 --> 00:36:04,450
But how fast are they growing?
587
00:36:04,450 --> 00:36:10,070
The answer lies
in the eigenvalue.
588
00:36:10,070 --> 00:36:12,450
So I've got to find the
matrix, so let me write down
589
00:36:12,450 --> 00:36:14,495
the Fibonacci rule.
590
00:36:17,610 --> 00:36:22,245
F(k+2) = F(k+1)+F k, right?
591
00:36:25,210 --> 00:36:28,280
Now that's not in my --
592
00:36:28,280 --> 00:36:32,420
I want to write that
as uk plus one and Auk.
593
00:36:32,420 --> 00:36:38,920
But right now what I've got is
a single equation, not a system,
594
00:36:38,920 --> 00:36:41,140
and it's second-order.
595
00:36:41,140 --> 00:36:44,290
It's like having a second-order
differential equation
596
00:36:44,290 --> 00:36:45,810
with second derivatives.
597
00:36:45,810 --> 00:36:47,580
I want to get first derivatives.
598
00:36:47,580 --> 00:36:49,200
Here I want to get
first differences.
599
00:36:49,200 --> 00:36:55,910
So the way, the way to do it
is to introduce uk will be
600
00:36:55,910 --> 00:36:57,960
a vector --
601
00:36:57,960 --> 00:36:59,125
see, a small trick.
602
00:37:01,920 --> 00:37:05,330
Let uk be a vector,
F(k+1) and Fk.
603
00:37:08,230 --> 00:37:12,680
So I'm going to get a two
by two system, first order,
604
00:37:12,680 --> 00:37:16,890
instead of a one -- instead of
a scalar system, second order,
605
00:37:16,890 --> 00:37:18,300
by a simple trick.
606
00:37:18,300 --> 00:37:22,820
I'm just going to add in an
equation F(k+1) equals F(k+1).
607
00:37:22,820 --> 00:37:28,980
That will be my second equation.
608
00:37:28,980 --> 00:37:33,940
Then this is my system,
this is my unknown,
609
00:37:33,940 --> 00:37:38,690
and what's my one step equation?
610
00:37:38,690 --> 00:37:45,120
So, so now u(k+1), that's --
so u(k+1) is the left side,
611
00:37:45,120 --> 00:37:47,620
and what have I got
here on the right side?
612
00:37:47,620 --> 00:37:52,530
I've got some matrix
multiplying uk.
613
00:37:52,530 --> 00:37:56,510
Can you, do -- can you
see that all right?
614
00:37:56,510 --> 00:37:59,450
if you can see it, then you
can tell me what the matrix is.
615
00:37:59,450 --> 00:38:02,860
Do you see that I'm
taking my system here.
616
00:38:02,860 --> 00:38:06,550
I artificially made
it into a system.
617
00:38:06,550 --> 00:38:10,540
I artificially made the
unknown into a vector.
618
00:38:10,540 --> 00:38:14,260
And now I'm ready to look
at and see what the matrix
619
00:38:14,260 --> 00:38:15,020
is.
620
00:38:15,020 --> 00:38:20,240
So do you see the left side,
u(k+1) is F(k+2) F(k+1),
621
00:38:20,240 --> 00:38:21,940
that's just what I want.
622
00:38:21,940 --> 00:38:25,590
On the right side, this
remember, this uk here --
623
00:38:25,590 --> 00:38:29,960
let me for the moment
put it as F(k+1) Fk.
624
00:38:29,960 --> 00:38:33,080
So what's the matrix?
625
00:38:33,080 --> 00:38:41,380
Well, that has a one and a one,
and that has a one and a zero.
626
00:38:41,380 --> 00:38:43,080
There's the matrix.
627
00:38:43,080 --> 00:38:47,880
Do you see that that gives
me the right-hand side?
628
00:38:47,880 --> 00:38:52,360
So there's the matrix A.
629
00:38:52,360 --> 00:38:56,810
And this is our friend uk.
630
00:38:56,810 --> 00:39:00,650
So we've got -- so
that simple trick --
631
00:39:00,650 --> 00:39:03,900
changed the second-order
scalar problem
632
00:39:03,900 --> 00:39:05,730
to a first-order system.
633
00:39:05,730 --> 00:39:08,750
Two b- u- with two unknowns.
634
00:39:08,750 --> 00:39:10,040
With a matrix.
635
00:39:10,040 --> 00:39:13,100
And now what do I do?
636
00:39:13,100 --> 00:39:16,240
Well, before I even think,
I find its eigenvalues
637
00:39:16,240 --> 00:39:18,170
and eigenvectors.
638
00:39:18,170 --> 00:39:21,170
So what are the eigenvalues and
eigenvectors of that matrix?
639
00:39:23,820 --> 00:39:24,320
Let's see.
640
00:39:24,320 --> 00:39:27,083
I always -- first let me just,
like, think for a minute.
641
00:39:29,720 --> 00:39:35,440
It's two by two, so this
shouldn't be impossible to do.
642
00:39:35,440 --> 00:39:37,020
Let's do it.
643
00:39:37,020 --> 00:39:37,670
OK.
644
00:39:37,670 --> 00:39:43,170
So my matrix, again,
is one one one zero.
645
00:39:46,170 --> 00:39:49,070
It's symmetric, by the way.
646
00:39:49,070 --> 00:39:56,070
So what I will eventually
know about symmetric matrices
647
00:39:56,070 --> 00:39:59,140
is that the eigenvalues
will come out real.
648
00:39:59,140 --> 00:40:02,290
I won't get any
complex numbers here.
649
00:40:02,290 --> 00:40:06,210
And the eigenvectors,
once I get those,
650
00:40:06,210 --> 00:40:08,520
actually will be orthogonal.
651
00:40:08,520 --> 00:40:11,190
But two by two, I'm
more interested in what
652
00:40:11,190 --> 00:40:13,740
the actual numbers are.
653
00:40:13,740 --> 00:40:16,230
What do I know about
the two numbers?
654
00:40:16,230 --> 00:40:18,190
Well, should do
you want me to find
655
00:40:18,190 --> 00:40:19,820
this determinant of A minus
656
00:40:19,820 --> 00:40:20,629
lambda I?
657
00:40:20,629 --> 00:40:21,128
Sure.
658
00:40:23,880 --> 00:40:27,910
So it's the determinant of
one minus lambda one one zero,
659
00:40:27,910 --> 00:40:28,410
right?
660
00:40:31,900 --> 00:40:33,400
Minus lambda, yes.
661
00:40:33,400 --> 00:40:33,994
God.
662
00:40:33,994 --> 00:40:34,493
OK.
663
00:40:38,030 --> 00:40:40,110
OK.
664
00:40:40,110 --> 00:40:42,240
There'll be two eigenvalues.
665
00:40:42,240 --> 00:40:45,160
What will -- tell me again
what I know about the two
666
00:40:45,160 --> 00:40:47,550
eigenvalues before
I go any further.
667
00:40:47,550 --> 00:40:49,590
Tell me something about
these two eigenvalues.
668
00:40:49,590 --> 00:40:51,770
What do they add up to?
669
00:40:51,770 --> 00:40:55,860
Lambda one plus lambda two is?
670
00:40:55,860 --> 00:41:02,390
Is the same as the trace down
the diagonal of the matrix.
671
00:41:02,390 --> 00:41:04,660
One and zero is one.
672
00:41:04,660 --> 00:41:08,320
So lambda one plus lambda two
should come out to be one.
673
00:41:08,320 --> 00:41:10,710
And lambda one times
lambda one times lambda two
674
00:41:10,710 --> 00:41:13,300
should come out to be
the determinant, which
675
00:41:13,300 --> 00:41:15,360
is minus one.
676
00:41:15,360 --> 00:41:18,440
So I'm expecting the
eigenvalues to add to one
677
00:41:18,440 --> 00:41:20,570
and to multiply to minus one.
678
00:41:20,570 --> 00:41:22,720
But let's just see
it happen here.
679
00:41:22,720 --> 00:41:26,680
If I multiply this out, I get --
that times that'll be a lambda
680
00:41:26,680 --> 00:41:30,290
squared minus lambda minus one.
681
00:41:30,290 --> 00:41:30,790
Good.
682
00:41:33,830 --> 00:41:36,570
Lambda squared minus
lambda minus one.
683
00:41:36,570 --> 00:41:43,250
Actually, I -- you see the b-
compare that with the original
684
00:41:43,250 --> 00:41:48,655
equation that I started with.
685
00:41:48,655 --> 00:41:49,780
F(k+2) - F(k+1)-Fk is zero.
686
00:41:49,780 --> 00:42:00,330
The recursion that -- that the
Fibonacci numbers satisfy is
687
00:42:00,330 --> 00:42:05,140
somehow showing up directly here
for the eigenvalues when we set
688
00:42:05,140 --> 00:42:06,230
that to zero.
689
00:42:06,230 --> 00:42:06,730
WK.
690
00:42:06,730 --> 00:42:09,530
Let's solve.
691
00:42:09,530 --> 00:42:14,200
Well, I would like to be able
to factor that, that quadratic,
692
00:42:14,200 --> 00:42:17,450
but I'm better off to use
the quadratic formula.
693
00:42:17,450 --> 00:42:19,880
Lambda is -- let's see.
694
00:42:19,880 --> 00:42:25,860
Minus b is one plus or minus
the square root of b squared,
695
00:42:25,860 --> 00:42:30,650
which is one, minus four
times that times that,
696
00:42:30,650 --> 00:42:33,740
which is plus four, over two.
697
00:42:37,614 --> 00:42:39,030
So that's the
square root of five.
698
00:42:42,600 --> 00:42:50,230
So the eigenvalues are
lambda one is one half of one
699
00:42:50,230 --> 00:42:57,000
plus square root of five, and
lambda two is one half of one
700
00:42:57,000 --> 00:42:59,220
minus square root of five.
701
00:42:59,220 --> 00:43:04,630
And sure enough, they -- those
add up to one and they multiply
702
00:43:04,630 --> 00:43:06,890
to give minus one.
703
00:43:06,890 --> 00:43:07,390
OK.
704
00:43:07,390 --> 00:43:09,400
Those are the two eigenvalues.
705
00:43:09,400 --> 00:43:12,250
How -- what are those
numbers approximately?
706
00:43:12,250 --> 00:43:18,060
Square root of five,
well, it's more than two
707
00:43:18,060 --> 00:43:19,030
but less than three.
708
00:43:19,030 --> 00:43:19,860
Hmm.
709
00:43:19,860 --> 00:43:25,330
It'd be nice to
know these numbers.
710
00:43:25,330 --> 00:43:30,480
I think, I think that -- so that
number comes out bigger than
711
00:43:30,480 --> 00:43:30,980
one, right?
712
00:43:30,980 --> 00:43:31,640
That's right.
713
00:43:31,640 --> 00:43:35,070
This number comes
out bigger than one.
714
00:43:35,070 --> 00:43:38,210
It's about one point six
one eight or something.
715
00:43:42,610 --> 00:43:44,700
Not exactly, but.
716
00:43:44,700 --> 00:43:48,300
And suppose it's one point six.
717
00:43:48,300 --> 00:43:52,390
Just, like, I think so.
718
00:43:52,390 --> 00:43:54,800
Then what's lambda two?
719
00:43:54,800 --> 00:43:57,870
Is, is lambda two
positive or negative?
720
00:43:57,870 --> 00:44:01,430
Negative, right, because I'm
-- it's, obviously negative,
721
00:44:01,430 --> 00:44:07,420
and I knew that the
-- so it's minus --
722
00:44:07,420 --> 00:44:16,720
and they add up to one, so minus
point six one eight, I guess.
723
00:44:16,720 --> 00:44:17,220
OK.
724
00:44:17,220 --> 00:44:17,800
A- and some more.
725
00:44:17,800 --> 00:44:18,520
Those are the two eigenvalues.
726
00:44:18,520 --> 00:44:19,830
One eigenvalue bigger than one,
one eigenvalue smaller than
727
00:44:19,830 --> 00:44:20,330
one.
728
00:44:20,330 --> 00:44:22,480
Actually, that's a great
situation to be in.
729
00:44:22,480 --> 00:44:25,430
Of course, the
eigenvalues are different,
730
00:44:25,430 --> 00:44:29,340
so there's no doubt whatever --
is this matrix diagonalizable?
731
00:44:32,270 --> 00:44:35,280
Is this matrix diagonalizable,
that original matrix A?
732
00:44:35,280 --> 00:44:35,990
Sure.
733
00:44:35,990 --> 00:44:38,030
We've got two
distinct eigenvalues
734
00:44:38,030 --> 00:44:44,294
and we can find the
eigenvectors in a moment.
735
00:44:44,294 --> 00:44:46,460
But they'll be independent,
we'll be diagonalizable.
736
00:44:46,460 --> 00:44:54,790
And now, you, you can already
answer my very first question.
737
00:44:54,790 --> 00:44:59,530
How fast are those Fibonacci
numbers increasing?
738
00:44:59,530 --> 00:45:01,080
How -- those --
they're increasing,
739
00:45:01,080 --> 00:45:01,810
right?
740
00:45:01,810 --> 00:45:03,970
They're not doubling
at every step.
741
00:45:03,970 --> 00:45:07,330
Let me -- let's look
again at these numbers.
742
00:45:07,330 --> 00:45:09,580
Five, eight, thirteen,
it's not obvious.
743
00:45:09,580 --> 00:45:14,060
The next one would be
twenty-one, thirty-four.
744
00:45:14,060 --> 00:45:20,924
So to get some idea of
what F one hundred is,
745
00:45:20,924 --> 00:45:21,840
can you give me any --
746
00:45:21,840 --> 00:45:24,820
I mean the crucial number --
747
00:45:24,820 --> 00:45:32,280
so it -- these --
it's approximately --
748
00:45:32,280 --> 00:45:37,970
what's controlling the growth
of these Fibonacci numbers?
749
00:45:37,970 --> 00:45:39,630
It's the eigenvalues.
750
00:45:39,630 --> 00:45:43,031
And which eigenvalue is
controlling that growth?
751
00:45:43,031 --> 00:45:43,530
The big one.
752
00:45:43,530 --> 00:45:50,380
So F100 will be approximately
some constant, c1 I guess,
753
00:45:50,380 --> 00:45:56,110
times this lambda one, this
one plus square root of five
754
00:45:56,110 --> 00:46:01,300
over two, to the
hundredth power.
755
00:46:01,300 --> 00:46:04,560
And the two hundredth F -- in
other words, the eigenvalue --
756
00:46:04,560 --> 00:46:08,950
the Fibonacci numbers are
growing by about that factor.
757
00:46:08,950 --> 00:46:13,780
Do you see that we, we've got
precise information about the,
758
00:46:13,780 --> 00:46:18,230
about the Fibonacci numbers
out of the eigenvalues?
759
00:46:18,230 --> 00:46:18,940
OK.
760
00:46:18,940 --> 00:46:21,880
And again, why is that true?
761
00:46:21,880 --> 00:46:26,750
Let me go over to this board
and s- show what I'm doing here.
762
00:46:26,750 --> 00:46:30,720
The -- the original initial
value is some combination
763
00:46:30,720 --> 00:46:31,730
of eigenvectors.
764
00:46:35,520 --> 00:46:39,470
And then when we start -- when
we start going out the theories
765
00:46:39,470 --> 00:46:42,580
of Fibonacci numbers, when
we start multiplying by A
766
00:46:42,580 --> 00:46:45,980
a hundred times, it's this
lambda one to the hundredth.
767
00:46:45,980 --> 00:46:51,070
This term is, is the
one that's taking over.
768
00:46:51,070 --> 00:46:54,920
It's -- I mean, that's big, like
one point six to the hundredth
769
00:46:54,920 --> 00:46:55,880
power.
770
00:46:55,880 --> 00:47:00,610
The second term is
practically nothing, right?
771
00:47:00,610 --> 00:47:04,300
The point six, or minus point
six, to the hundredth power
772
00:47:04,300 --> 00:47:08,180
is an extremely small,
extremely small number.
773
00:47:08,180 --> 00:47:11,410
So this is -- there're
only two terms,
774
00:47:11,410 --> 00:47:13,020
because we're two by two.
775
00:47:13,020 --> 00:47:16,430
This number is -- this
piece of it is there,
776
00:47:16,430 --> 00:47:21,270
but it's, it's disappearing,
where this piece is there
777
00:47:21,270 --> 00:47:23,890
and it's growing and
controlling everything.
778
00:47:23,890 --> 00:47:27,230
So, so really the --
we're doing, like,
779
00:47:27,230 --> 00:47:29,100
problems that are evolving.
780
00:47:29,100 --> 00:47:33,390
We're doing dynamic
u- instead of Ax=b,
781
00:47:33,390 --> 00:47:35,440
that's a static problem.
782
00:47:35,440 --> 00:47:36,930
We're now we're doing dynamics.
783
00:47:36,930 --> 00:47:39,740
A, A squared, A cubed,
things are evolving in
784
00:47:39,740 --> 00:47:40,440
time.
785
00:47:40,440 --> 00:47:44,660
And the eigenvalues are
the crucial, numbers.
786
00:47:44,660 --> 00:47:45,640
OK.
787
00:47:45,640 --> 00:47:52,490
I guess to complete
this, I better
788
00:47:52,490 --> 00:47:56,420
write down the eigenvectors.
789
00:47:56,420 --> 00:47:59,160
So we should complete
the, the whole process
790
00:47:59,160 --> 00:48:01,200
by finding the eigenvectors.
791
00:48:01,200 --> 00:48:03,820
OK, well, I have to --
up in the corner, then,
792
00:48:03,820 --> 00:48:07,670
I have to look at
A minus lambda I.
793
00:48:07,670 --> 00:48:15,800
So A minus lambda I is this one
minus lambda one one and minus
794
00:48:15,800 --> 00:48:16,930
lambda.
795
00:48:16,930 --> 00:48:19,990
And now can we spot an
eigenvector out of that?
796
00:48:19,990 --> 00:48:23,070
That's, that's, for
these two lambdas,
797
00:48:23,070 --> 00:48:24,415
this matrix is singular.
798
00:48:27,380 --> 00:48:30,350
I guess the eigenvector -- two
by two ought to be, I mean,
799
00:48:30,350 --> 00:48:31,260
easy.
800
00:48:31,260 --> 00:48:33,960
So if I know that this
matrix is singular,
801
00:48:33,960 --> 00:48:37,100
then u- seems to
me the eigenvector
802
00:48:37,100 --> 00:48:41,340
has to be lambda and one,
because that multiplication
803
00:48:41,340 --> 00:48:43,500
will give me the zero.
804
00:48:43,500 --> 00:48:47,240
And this multiplication gives
me -- better give me also zero.
805
00:48:47,240 --> 00:48:48,650
Do you see why it does?
806
00:48:48,650 --> 00:48:52,670
This is the minus lambda
squared plus lambda plus one.
807
00:48:52,670 --> 00:48:56,360
It's the thing that's zero
because these lambdas are
808
00:48:56,360 --> 00:48:56,930
special.
809
00:48:56,930 --> 00:48:58,490
There's the eigenvector.
810
00:48:58,490 --> 00:49:07,900
x1 is lambda one one,
and x2 is lambda two one.
811
00:49:07,900 --> 00:49:12,520
I did that as a little trick
that was available in the two
812
00:49:12,520 --> 00:49:14,130
by two case.
813
00:49:14,130 --> 00:49:17,680
So now I finally have to --
814
00:49:17,680 --> 00:49:20,390
oh, I have to take
the initial u0 now.
815
00:49:20,390 --> 00:49:22,710
So to complete this
example entirely,
816
00:49:22,710 --> 00:49:26,680
I have to say, OK, what was u0?
817
00:49:26,680 --> 00:49:28,740
u0 was F1 F0.
818
00:49:28,740 --> 00:49:40,630
So u0, the starting vector is
F1 F0, and those were one and
819
00:49:40,630 --> 00:49:41,130
zero.
820
00:49:43,910 --> 00:49:47,150
So I have to use that vector.
821
00:49:47,150 --> 00:49:50,390
So I have to look
for, for a multiple
822
00:49:50,390 --> 00:49:56,100
of the first eigenvector and
the second to produce u0,
823
00:49:56,100 --> 00:49:58,070
the one zero
824
00:49:58,070 --> 00:49:58,570
vector.
825
00:49:58,570 --> 00:50:05,430
This is what will find c1
and c2, and then I'm done.
826
00:50:05,430 --> 00:50:10,470
Do you -- so let me instead
of, in the last five seconds,
827
00:50:10,470 --> 00:50:14,920
grinding out a formula,
let me repeat the idea.
828
00:50:14,920 --> 00:50:19,100
Because I'd really -- it's
the idea that's central.
829
00:50:19,100 --> 00:50:21,610
When things are
evolving in time --
830
00:50:21,610 --> 00:50:25,730
let me come back to this board,
because the ideas are here.
831
00:50:25,730 --> 00:50:30,400
When things are evolving in
time by a first-order system,
832
00:50:30,400 --> 00:50:34,480
starting from an
original u0, the key
833
00:50:34,480 --> 00:50:39,956
is find the eigenvalues
and eigenvectors of A.
834
00:50:39,956 --> 00:50:41,580
That will tell --
those eigenvectors --
835
00:50:41,580 --> 00:50:46,710
the eigenvalues will already
tell you what's happening.
836
00:50:46,710 --> 00:50:48,540
Is the solution
blowing up, is it
837
00:50:48,540 --> 00:50:51,390
going to zero, what's it doing.
838
00:50:51,390 --> 00:50:56,240
And then to, to find
out exactly a formula,
839
00:50:56,240 --> 00:50:59,290
you have to take
your u0 and write it
840
00:50:59,290 --> 00:51:03,270
as a combination of
eigenvectors and then
841
00:51:03,270 --> 00:51:05,820
follow each
eigenvector separately.
842
00:51:05,820 --> 00:51:10,930
And that's really what this
formula, the formula for, --
843
00:51:10,930 --> 00:51:15,190
that's what the formula
for A to the K is doing.
844
00:51:15,190 --> 00:51:17,180
So remember that
formula for A to the K
845
00:51:17,180 --> 00:51:21,770
is S lambda to the K S inverse.
846
00:51:21,770 --> 00:51:22,280
OK.
847
00:51:22,280 --> 00:51:24,590
That's, that's
difference equations.
848
00:51:24,590 --> 00:51:33,460
And you just have to -- so the,
the homework will give some
849
00:51:33,460 --> 00:51:41,180
examples, different from
Fibonacci, to follow through.
850
00:51:41,180 --> 00:51:48,900
And next time will be
differential equations.
851
00:51:48,900 --> 00:51:50,450
Thanks.