WEBVTT
00:00:10.510 --> 00:00:11.010
OK.
00:00:11.010 --> 00:00:11.635
Shall we start?
00:00:11.635 --> 00:00:15.360
This is the second
lecture on eigenvalues.
00:00:15.360 --> 00:00:18.350
So the first lecture was --
00:00:18.350 --> 00:00:22.350
reached the key equation,
A x equal lambda x.
00:00:22.350 --> 00:00:27.090
x is the eigenvector and
lambda's the eigenvalue.
00:00:27.090 --> 00:00:30.220
Now to use that.
00:00:30.220 --> 00:00:36.330
And the, the good way
to, after we've found --
00:00:36.330 --> 00:00:39.400
so, so job one is to
find the eigenvalues
00:00:39.400 --> 00:00:41.990
and find the eigenvectors.
00:00:41.990 --> 00:00:44.750
Now after we've found them,
what do we do with them?
00:00:44.750 --> 00:00:50.360
Well, the good way to see that
is diagonalize the matrix.
00:00:50.360 --> 00:00:52.850
So the matrix is A.
00:00:52.850 --> 00:00:55.130
And I want to show
-- first of all,
00:00:55.130 --> 00:00:57.190
this is like the basic fact.
00:00:57.190 --> 00:00:58.720
This, this formula.
00:00:58.720 --> 00:01:03.210
That's, that's the key
to today's lecture.
00:01:03.210 --> 00:01:06.810
This matrix A, I
put its eigenvectors
00:01:06.810 --> 00:01:09.660
in the columns of a matrix S.
00:01:09.660 --> 00:01:12.740
So S will be the
eigenvector matrix.
00:01:12.740 --> 00:01:19.560
And I want to look at this
magic combination S inverse A S.
00:01:19.560 --> 00:01:22.830
So can I show you how that --
00:01:22.830 --> 00:01:24.200
what happens there?
00:01:24.200 --> 00:01:27.900
And notice, there's
an S inverse.
00:01:27.900 --> 00:01:33.690
We have to be able to invert
this eigenvector matrix S.
00:01:33.690 --> 00:01:38.330
So for that, we need n
independent eigenvectors.
00:01:38.330 --> 00:01:41.440
So that's the, that's the case.
00:01:41.440 --> 00:01:42.130
OK.
00:01:42.130 --> 00:01:53.740
So suppose we have n linearly
independent eigenvectors
00:01:53.740 --> 00:02:11.830
of A. Put them in the
columns of this matrix S.
00:02:11.830 --> 00:02:15.430
So I'm naturally going to call
that the eigenvector matrix,
00:02:15.430 --> 00:02:18.730
because it's got the
eigenvectors in its columns.
00:02:18.730 --> 00:02:21.730
And all I want to do is
show you what happens
00:02:21.730 --> 00:02:24.960
when you multiply A times S.
00:02:24.960 --> 00:02:29.200
So A times S.
00:02:29.200 --> 00:02:34.490
So this is A times the matrix
with the first eigenvector
00:02:34.490 --> 00:02:37.300
in its first column,
the second eigenvector
00:02:37.300 --> 00:02:42.510
in its second column, the n-th
eigenvector in its n-th column.
00:02:42.510 --> 00:02:46.300
And how I going to do this
matrix multiplication?
00:02:46.300 --> 00:02:49.890
Well, certainly I'll do
it a column at a time.
00:02:49.890 --> 00:02:52.510
And what do I get.
00:02:52.510 --> 00:02:55.480
A times the first column
gives me the first column
00:02:55.480 --> 00:02:57.820
of the answer, but what is it?
00:02:57.820 --> 00:03:00.010
That's an eigenvector.
00:03:00.010 --> 00:03:04.280
A times x1 is equal to
the lambda times the x1.
00:03:04.280 --> 00:03:06.870
And that lambda's we're
-- we'll call lambda one,
00:03:06.870 --> 00:03:09.590
of course.
00:03:09.590 --> 00:03:11.420
So that's the first column.
00:03:11.420 --> 00:03:14.000
Ax1 is the same
as lambda one x1.
00:03:14.000 --> 00:03:16.730
A x2 is lambda two x2.
00:03:16.730 --> 00:03:22.120
So on, along to in the n-th
column we now how lambda n xn.
00:03:24.690 --> 00:03:27.110
Looking good, but
the next step is even
00:03:27.110 --> 00:03:28.120
better.
00:03:28.120 --> 00:03:31.280
So for the next step,
I want to separate out
00:03:31.280 --> 00:03:36.100
those eigenvalues, those,
those multiplying numbers,
00:03:36.100 --> 00:03:38.880
from the x-s.
00:03:38.880 --> 00:03:41.920
So then I'll have
just what I want.
00:03:41.920 --> 00:03:42.530
OK.
00:03:42.530 --> 00:03:44.990
So how, how I going
to separate out?
00:03:44.990 --> 00:03:47.150
So that, that
number lambda one is
00:03:47.150 --> 00:03:49.760
multiplying the first column.
00:03:49.760 --> 00:03:52.840
So if I want to factor it
out of the first column,
00:03:52.840 --> 00:03:55.400
I better put --
00:03:55.400 --> 00:03:58.830
here is going to
be x1, and that's
00:03:58.830 --> 00:04:02.180
going to multiply
this matrix lambda
00:04:02.180 --> 00:04:05.050
one in the first
entry and all zeros.
00:04:05.050 --> 00:04:08.230
Do you see that that,
that's going to come out
00:04:08.230 --> 00:04:10.220
right for the first column?
00:04:10.220 --> 00:04:13.370
Because w- we remember how --
how we're going back to that
00:04:13.370 --> 00:04:15.310
original punchline.
00:04:15.310 --> 00:04:21.260
That if I want a number
to multiply x1 then
00:04:21.260 --> 00:04:24.310
I can do it by putting
x1 in that column,
00:04:24.310 --> 00:04:27.280
in the first column, and
putting that number there.
00:04:27.280 --> 00:04:28.850
Th- u- what I
going to have here?
00:04:28.850 --> 00:04:30.040
I'm going to have lambda --
00:04:30.040 --> 00:04:33.800
I'm going to have x1, x2, ...
00:04:33.800 --> 00:04:34.890
,xn.
00:04:34.890 --> 00:04:37.150
These are going to
be my columns again.
00:04:37.150 --> 00:04:39.740
I'm getting S back again.
00:04:39.740 --> 00:04:41.380
I'm getting S back again.
00:04:41.380 --> 00:04:44.560
But now what's it multiplied
by, on the right it's
00:04:44.560 --> 00:04:48.080
multiplied by?
00:04:48.080 --> 00:04:54.730
If I want lambda n xn in the
last column, how do I do it?
00:04:54.730 --> 00:04:57.670
Well, the last column
here will be --
00:04:57.670 --> 00:05:02.630
I'll take the last column,
use these coefficients,
00:05:02.630 --> 00:05:05.050
put the lambda n
down there, and it
00:05:05.050 --> 00:05:09.970
will multiply that n-th column
and give me lambda n xn.
00:05:09.970 --> 00:05:13.130
There, there you see matrix
multiplication just working
00:05:13.130 --> 00:05:14.850
for us.
00:05:14.850 --> 00:05:17.410
So I started with A S.
00:05:17.410 --> 00:05:21.010
I wrote down what it meant,
A times each eigenvector.
00:05:21.010 --> 00:05:23.690
That gave me lambda
time the eigenvector.
00:05:23.690 --> 00:05:25.700
And then when I peeled
off the lambdas,
00:05:25.700 --> 00:05:30.690
they were on the right-hand
side, so I've got S, my matrix,
00:05:30.690 --> 00:05:31.750
back again.
00:05:31.750 --> 00:05:45.500
And this matrix, this diagonal
matrix, the eigenvalue matrix,
00:05:45.500 --> 00:05:49.210
and I call it capital lambda.
00:05:49.210 --> 00:05:52.830
Using capital letters
for matrices and lambda
00:05:52.830 --> 00:05:54.980
to prompt me that
it's, that it's
00:05:54.980 --> 00:05:56.850
eigenvalues that are in there.
00:05:56.850 --> 00:05:58.840
So you see that the
eigenvalues are just
00:05:58.840 --> 00:06:01.480
sitting down that diagonal?
00:06:01.480 --> 00:06:07.340
If I had a column x2 here,
I would want the lambda two
00:06:07.340 --> 00:06:10.970
in the two two position,
in the diagonal position,
00:06:10.970 --> 00:06:15.140
to multiply that x2 and
give me the lambda two x2.
00:06:15.140 --> 00:06:16.730
That's my formula.
00:06:16.730 --> 00:06:19.770
A S is S lambda.
00:06:23.000 --> 00:06:23.720
OK.
00:06:23.720 --> 00:06:27.530
That's the -- you see,
it's just a calculation.
00:06:27.530 --> 00:06:31.390
Now -- I mentioned, and
I have to mention again,
00:06:31.390 --> 00:06:37.210
this business about n
independent eigenvectors.
00:06:37.210 --> 00:06:40.050
As it stands, this is
all fine, whether --
00:06:40.050 --> 00:06:43.240
I mean, I could be repeating
the same eigenvector, but --
00:06:43.240 --> 00:06:45.550
I'm not interested in that.
00:06:45.550 --> 00:06:50.600
I want to be able to invert S,
and that's where this comes in.
00:06:50.600 --> 00:06:53.690
This n independent
eigenvectors business
00:06:53.690 --> 00:06:56.990
comes in to tell me that
that matrix is invertible.
00:06:56.990 --> 00:07:01.380
So let me, on the next board,
write down what I've got.
00:07:01.380 --> 00:07:06.780
A S equals S lambda.
00:07:06.780 --> 00:07:11.890
And now I'm, I can multiply
on the left by S inverse.
00:07:11.890 --> 00:07:13.839
So this is really --
00:07:17.970 --> 00:07:20.660
I can do that, provided
S is invertible.
00:07:20.660 --> 00:07:26.320
Provided my assumption of n
independent eigenvectors is
00:07:26.320 --> 00:07:27.260
satisfied.
00:07:27.260 --> 00:07:31.470
And I mentioned at the end of
last time, and I'll say again,
00:07:31.470 --> 00:07:35.930
that there's a small
number of matrices for --
00:07:35.930 --> 00:07:38.680
that don't have n
independent eigenvectors.
00:07:38.680 --> 00:07:42.580
So I've got to discuss
that, that technical point.
00:07:42.580 --> 00:07:48.030
But the great -- the most
matrices that we see have n di-
00:07:48.030 --> 00:07:51.050
n independent eigenvectors,
and we can diagonalize.
00:07:51.050 --> 00:07:54.090
This is diagonalization.
00:07:54.090 --> 00:07:57.890
I could also write
it, and I often will,
00:07:57.890 --> 00:07:59.680
the other way round.
00:07:59.680 --> 00:08:03.140
If I multiply on the
right by S inverse,
00:08:03.140 --> 00:08:06.130
if I took this equation at the
top and multiplied on the right
00:08:06.130 --> 00:08:08.090
by S inverse, I could --
00:08:08.090 --> 00:08:10.900
I would have A left here.
00:08:10.900 --> 00:08:15.790
Now S inverse is
coming from the right.
00:08:15.790 --> 00:08:18.530
So can you keep
those two straight?
00:08:18.530 --> 00:08:21.491
A multiplies its eigenvectors,
that's how I keep them
00:08:21.491 --> 00:08:21.990
straight.
00:08:21.990 --> 00:08:24.530
So A multiplies S.
00:08:24.530 --> 00:08:26.020
A multiplies S.
00:08:26.020 --> 00:08:30.700
And then this S inverse makes
the whole thing diagonal.
00:08:30.700 --> 00:08:33.750
And this is another way
of saying the same thing,
00:08:33.750 --> 00:08:36.700
putting the Ss on the
other side of the equation.
00:08:36.700 --> 00:08:40.179
A is S lambda S inverse.
00:08:40.179 --> 00:08:44.520
So that's the, that's
the new factorization.
00:08:44.520 --> 00:08:50.550
That's the replacement for L U
from elimination or Q R for --
00:08:50.550 --> 00:08:52.210
from Gram-Schmidt.
00:08:52.210 --> 00:08:56.850
And notice that the matrix --
so it's, it's a matrix times
00:08:56.850 --> 00:09:00.420
a diagonal matrix times the
inverse of the first one.
00:09:00.420 --> 00:09:02.410
It's, that's the
combination that we'll
00:09:02.410 --> 00:09:05.530
see throughout this chapter.
00:09:05.530 --> 00:09:09.650
This combination with
an S and an S inverse.
00:09:09.650 --> 00:09:10.250
OK.
00:09:10.250 --> 00:09:14.080
Can I just begin to use that?
00:09:14.080 --> 00:09:17.730
For example, what
about A squared?
00:09:17.730 --> 00:09:20.880
What are the eigenvalues and
eigenvectors of A squared?
00:09:20.880 --> 00:09:23.040
That's a straightforward
question with a,
00:09:23.040 --> 00:09:25.380
with an absolutely clean answer.
00:09:25.380 --> 00:09:28.930
So let me, let me
consider A squared.
00:09:28.930 --> 00:09:33.560
So I start with A
x equal lambda x.
00:09:33.560 --> 00:09:36.070
And I'm headed for A squared.
00:09:36.070 --> 00:09:38.610
So let me multiply
both sides by A.
00:09:38.610 --> 00:09:41.290
That's one way to get
A squared on the left.
00:09:41.290 --> 00:09:46.190
So -- I should write
these if-s in here.
00:09:46.190 --> 00:09:51.010
If A x equals lambda x,
then I multiply by A,
00:09:51.010 --> 00:09:56.200
so I get A squared x equals
-- well, I'm multiplying by A,
00:09:56.200 --> 00:09:58.670
so that's lambda A x.
00:09:58.670 --> 00:10:02.520
That lambda was a number, so
I just put it on the left.
00:10:02.520 --> 00:10:06.740
And what do I -- tell me how
to make that look better.
00:10:06.740 --> 00:10:10.220
What have I got
here for if, if A
00:10:10.220 --> 00:10:13.220
has the eigenvalue
lambda and eigenvector
00:10:13.220 --> 00:10:16.010
x, what's up with A squared?
00:10:16.010 --> 00:10:18.430
A squared x, I just
multiplied by A,
00:10:18.430 --> 00:10:23.070
but now for Ax I'm going
to substitute lambda x.
00:10:23.070 --> 00:10:25.010
So I've got lambda squared x.
00:10:28.120 --> 00:10:31.370
So from that simple
calculation, I --
00:10:31.370 --> 00:10:37.330
my conclusion is that the
eigenvalues of A squared are
00:10:37.330 --> 00:10:39.310
lambda squared.
00:10:39.310 --> 00:10:40.384
And the eigenvectors --
00:10:40.384 --> 00:10:41.550
I always think about both of
00:10:41.550 --> 00:10:42.150
those.
00:10:42.150 --> 00:10:44.390
What can I say about
the eigenvalues?
00:10:44.390 --> 00:10:45.560
They're squared.
00:10:45.560 --> 00:10:48.220
What can I say about
the eigenvectors?
00:10:48.220 --> 00:10:49.730
They're the same.
00:10:49.730 --> 00:10:54.090
The same x as in -- as for A.
00:10:54.090 --> 00:11:00.320
Now let me see that
also from this formula.
00:11:00.320 --> 00:11:05.160
How can I see what A squared is
looking like from this formula?
00:11:05.160 --> 00:11:08.480
So let me -- that
was one way to do it.
00:11:08.480 --> 00:11:12.510
Let me do it by just
taking A squared from that.
00:11:12.510 --> 00:11:18.990
A squared is S lambda S
inverse -- that's A --
00:11:18.990 --> 00:11:22.620
times S lambda S inverse
-- that's A, which is?
00:11:25.530 --> 00:11:30.170
This is the beauty of
eigenvalues, eigenvectors.
00:11:30.170 --> 00:11:34.000
Having that S inverse
and S is the identity,
00:11:34.000 --> 00:11:40.220
so I've got S lambda
squared S inverse.
00:11:40.220 --> 00:11:43.380
Do you see what
that's telling me?
00:11:43.380 --> 00:11:46.570
It's, it's telling me the same
thing that I just learned here,
00:11:46.570 --> 00:11:49.630
but in the -- in a matrix form.
00:11:49.630 --> 00:11:52.720
It's telling me that
the S is the same,
00:11:52.720 --> 00:11:57.130
the eigenvectors are the
same, but the eigenvalues
00:11:57.130 --> 00:11:58.530
are squared.
00:11:58.530 --> 00:12:01.020
Because this is --
what's lambda squared?
00:12:01.020 --> 00:12:02.800
That's still diagonal.
00:12:02.800 --> 00:12:05.180
It's got little
lambda one squared,
00:12:05.180 --> 00:12:07.190
lambda two squared,
down to lambda n
00:12:07.190 --> 00:12:09.280
squared o- on that diagonal.
00:12:09.280 --> 00:12:13.760
Those are the eigenvalues, as
we just learned, of A squared.
00:12:13.760 --> 00:12:14.500
OK.
00:12:14.500 --> 00:12:19.590
So -- somehow those eigenvalues
and eigenvectors are really
00:12:19.590 --> 00:12:25.060
giving you a way to --
00:12:25.060 --> 00:12:29.460
see what's going
on inside a matrix.
00:12:29.460 --> 00:12:32.100
Of course I can
continue that for --
00:12:32.100 --> 00:12:36.270
to the K-th power,
A to the K-th power.
00:12:36.270 --> 00:12:39.370
If I multiply, if I have
K of these together,
00:12:39.370 --> 00:12:42.920
do you see how S inverse
S will keep canceling
00:12:42.920 --> 00:12:44.900
in the, in the inside?
00:12:44.900 --> 00:12:48.740
I'll have the S outside
at the far left,
00:12:48.740 --> 00:12:54.390
and lambda will be in there
K times, and S inverse.
00:12:54.390 --> 00:12:56.670
So what's that telling me?
00:12:56.670 --> 00:12:59.420
That's telling me
that the eigenvalues
00:12:59.420 --> 00:13:04.340
of A, of A to the K-th
power are the K-th powers.
00:13:04.340 --> 00:13:08.390
The eigenvalues of A cubed are
the cubes of the eigenvalues of
00:13:08.390 --> 00:13:15.000
A. And the eigenvectors
are the same, the same.
00:13:15.000 --> 00:13:15.600
OK.
00:13:15.600 --> 00:13:20.750
In other words, eigenvalues
and eigenvectors
00:13:20.750 --> 00:13:25.910
give a great way to understand
the powers of a matrix.
00:13:25.910 --> 00:13:28.380
If I take the
square of a matrix,
00:13:28.380 --> 00:13:30.540
or the hundredth
power of a matrix,
00:13:30.540 --> 00:13:34.780
the pivots are all
over the place.
00:13:34.780 --> 00:13:39.690
L U, if I multiply L U times
L U times L U times L U
00:13:39.690 --> 00:13:44.610
a hundred times, I've
got a hundred L Us.
00:13:44.610 --> 00:13:46.370
I can't do anything with them.
00:13:46.370 --> 00:13:50.180
But when I multiply S
lambda S inverse by itself,
00:13:50.180 --> 00:13:54.720
when I look at the eigenvector
picture a hundred times,
00:13:54.720 --> 00:13:59.810
I get a hundred or ninety-nine
of these guys canceling out
00:13:59.810 --> 00:14:03.390
inside, and I get
A to the hundredth
00:14:03.390 --> 00:14:05.900
is S lambda to the
hundredth S inverse.
00:14:05.900 --> 00:14:09.870
I mean, eigenvalues
tell you about powers
00:14:09.870 --> 00:14:16.290
of a matrix in a way that we had
no way to approach previously.
00:14:16.290 --> 00:14:21.220
For example, when does --
00:14:21.220 --> 00:14:24.790
when do the powers of
a matrix go to zero?
00:14:24.790 --> 00:14:29.330
I would call that
matrix stable, maybe.
00:14:29.330 --> 00:14:31.940
So I could write down a theorem.
00:14:31.940 --> 00:14:36.720
I'll write it as a theorem
just to use that word
00:14:36.720 --> 00:14:40.350
to emphasize that here I'm
getting this great fact
00:14:40.350 --> 00:14:42.610
from this eigenvalue picture.
00:14:42.610 --> 00:14:43.230
OK.
00:14:43.230 --> 00:14:53.720
A to the K approaches zero as K
goes, as K gets bigger if what?
00:14:53.720 --> 00:14:57.840
What's the w- how can
I tell, for a matrix A,
00:14:57.840 --> 00:14:59.850
if its powers go to zero?
00:15:02.650 --> 00:15:07.520
What's -- somewhere inside that
matrix is that information.
00:15:07.520 --> 00:15:11.510
That information is not
present in the pivots.
00:15:11.510 --> 00:15:13.270
It's present in the eigenvalues.
00:15:13.270 --> 00:15:17.010
What do I need for the -- to
know that if I take higher
00:15:17.010 --> 00:15:20.170
and higher powers of A, that
this matrix gets smaller
00:15:20.170 --> 00:15:21.030
and smaller?
00:15:21.030 --> 00:15:24.330
Well, S and S inverse
are not moving.
00:15:24.330 --> 00:15:26.810
So it's this guy that
has to get small.
00:15:26.810 --> 00:15:29.720
And that's easy to
-- to understand.
00:15:29.720 --> 00:15:32.595
The requirement is
all eigenvalues --
00:15:35.330 --> 00:15:38.040
so what is the requirement?
00:15:38.040 --> 00:15:41.170
The eigenvalues have
to be less than one.
00:15:41.170 --> 00:15:45.310
Now I have to wrote
that absolute value,
00:15:45.310 --> 00:15:48.360
because those eigenvalues
could be negative,
00:15:48.360 --> 00:15:50.550
they could be complex numbers.
00:15:50.550 --> 00:15:53.170
So I'm taking the
absolute value.
00:15:53.170 --> 00:15:56.750
If all of those are below one.
00:15:56.750 --> 00:16:05.000
That's, in fact, we
practically see why.
00:16:05.000 --> 00:16:13.100
And let me just say that I'm
operating on one assumption
00:16:13.100 --> 00:16:15.740
here, and I got to
keep remembering
00:16:15.740 --> 00:16:18.690
that that assumption
is still present.
00:16:18.690 --> 00:16:21.420
That assumption was that
I had a full set of,
00:16:21.420 --> 00:16:24.370
of n independent eigenvectors.
00:16:24.370 --> 00:16:30.830
If I don't have that, then
this approach is not working.
00:16:30.830 --> 00:16:37.090
So again, a pure eigenvalue
approach, eigenvector approach,
00:16:37.090 --> 00:16:40.470
needs n independent
eigenvectors.
00:16:40.470 --> 00:16:42.900
If we don't have n
independent eigenvectors,
00:16:42.900 --> 00:16:46.490
we can't diagonalize the matrix.
00:16:46.490 --> 00:16:50.820
We can't get to a
diagonal matrix.
00:16:50.820 --> 00:16:55.960
This diagonalization
is only possible
00:16:55.960 --> 00:16:58.581
if S inverse makes sense.
00:16:58.581 --> 00:16:59.080
OK.
00:16:59.080 --> 00:17:02.800
Can I, can I follow
up on that point now?
00:17:02.800 --> 00:17:07.099
So you see why -- what we
get and, and why we want it,
00:17:07.099 --> 00:17:11.490
because we get information about
the powers of a matrix just
00:17:11.490 --> 00:17:14.940
immediately from
the eigenvalues.
00:17:14.940 --> 00:17:15.500
OK.
00:17:15.500 --> 00:17:22.329
Now let me follow up on this,
business of which matrices
00:17:22.329 --> 00:17:25.030
are diagonalizable.
00:17:25.030 --> 00:17:28.220
Sorry about that long word.
00:17:28.220 --> 00:17:32.940
So a matrix is, is sure -- so
here's, here's the main point.
00:17:32.940 --> 00:17:37.600
A is sure to be --
00:17:37.600 --> 00:17:50.110
to have N independent
eigenvectors and, and be --
00:17:50.110 --> 00:18:00.210
now here comes that word
-- diagonalizable if, if --
00:18:00.210 --> 00:18:05.810
so we might as well get the
nice case out in the open.
00:18:05.810 --> 00:18:13.330
The nice case is when -- if
all the lambdas are different.
00:18:19.520 --> 00:18:28.326
That means, that means
no repeated eigenvalues.
00:18:30.920 --> 00:18:32.000
OK.
00:18:32.000 --> 00:18:34.410
That's the nice case.
00:18:34.410 --> 00:18:39.970
If my matrix, and most -- if
I do a random matrix in Matlab
00:18:39.970 --> 00:18:43.140
and compute its eigenvalues --
00:18:43.140 --> 00:18:55.540
so if I computed if I took
eig of rand of ten ten, gave,
00:18:55.540 --> 00:18:58.640
gave that Matlab command, the --
00:18:58.640 --> 00:19:01.190
we'd get a random
ten by ten matrix,
00:19:01.190 --> 00:19:03.990
we would get a list of
its ten eigenvalues,
00:19:03.990 --> 00:19:08.040
and they would be different.
00:19:08.040 --> 00:19:10.470
They would be distinct
is the best word.
00:19:10.470 --> 00:19:13.880
I would have -- a random matrix
will have ten distinct --
00:19:13.880 --> 00:19:18.090
a ten by ten matrix will have
ten distinct eigenvalues.
00:19:18.090 --> 00:19:25.500
And if it does, the eigenvectors
are automatically independent.
00:19:25.500 --> 00:19:26.930
So that's a nice fact.
00:19:26.930 --> 00:19:29.730
I'll refer you to the
text for the proof.
00:19:29.730 --> 00:19:36.360
That, that A is sure to have
n independent eigenvectors
00:19:36.360 --> 00:19:41.610
if the eigenvalues
are different, if.
00:19:41.610 --> 00:19:43.890
If all the, if all
eigenvalues are different.
00:19:43.890 --> 00:19:47.810
It's just if some
lambdas are repeated,
00:19:47.810 --> 00:19:50.560
then I have to
look more closely.
00:19:50.560 --> 00:19:55.050
If an eigenvalue is repeated, I
have to look, I have to count,
00:19:55.050 --> 00:19:56.260
I have to check.
00:19:56.260 --> 00:19:59.560
Has it got -- say it's
repeated three times.
00:19:59.560 --> 00:20:02.030
So what's a
possibility for the --
00:20:02.030 --> 00:20:05.893
so here is the, here is
the repeated possibility.
00:20:11.000 --> 00:20:16.490
And, and let me
emphasize the conclusion.
00:20:16.490 --> 00:20:21.410
That if I have repeated
eigenvalues, I may or may not,
00:20:21.410 --> 00:20:35.271
I may or may not have, have
n independent eigenvectors.
00:20:35.271 --> 00:20:35.770
I might.
00:20:35.770 --> 00:20:41.240
I, I, you know, this isn't
a completely negative case.
00:20:41.240 --> 00:20:43.260
The identity matrix --
00:20:43.260 --> 00:20:46.380
suppose I take the ten
by ten identity matrix.
00:20:46.380 --> 00:20:50.750
What are the eigenvalues
of that matrix?
00:20:50.750 --> 00:20:55.370
So just, just take the
easiest matrix, the identity.
00:20:55.370 --> 00:21:00.470
If I look for its
eigenvalues, they're all ones.
00:21:00.470 --> 00:21:04.410
So that eigenvalue one
is repeated ten times.
00:21:04.410 --> 00:21:07.340
But there's no shortage of
eigenvectors for the identity
00:21:07.340 --> 00:21:08.250
matrix.
00:21:08.250 --> 00:21:10.760
In fact, every vector
is an eigenvector.
00:21:10.760 --> 00:21:13.610
So I can take ten
independent vectors.
00:21:13.610 --> 00:21:16.530
Oh, well, what happens
to everything --
00:21:16.530 --> 00:21:18.590
if A is the identity
matrix, let's
00:21:18.590 --> 00:21:21.650
just think that one
through in our head.
00:21:21.650 --> 00:21:27.440
If A is the identity
matrix, then it's
00:21:27.440 --> 00:21:28.830
got plenty of eigenvectors.
00:21:28.830 --> 00:21:30.910
I choose ten
independent vectors.
00:21:30.910 --> 00:21:32.560
They're the columns of S.
00:21:32.560 --> 00:21:37.380
And, and what do I get
from S inverse A S?
00:21:37.380 --> 00:21:39.400
I get I again, right?
00:21:39.400 --> 00:21:42.210
If A is the identity -- and
of course that's the correct
00:21:42.210 --> 00:21:43.790
lambda.
00:21:43.790 --> 00:21:46.790
The matrix was already diagonal.
00:21:46.790 --> 00:21:48.970
So if the matrix is
already diagonal,
00:21:48.970 --> 00:21:53.800
then the, the lambda is
the same as the matrix.
00:21:53.800 --> 00:21:56.380
A diagonal matrix has
got its eigenvalues
00:21:56.380 --> 00:21:59.170
sitting right there
in front of you.
00:21:59.170 --> 00:22:01.790
Now if it's triangular,
the eigenvalues
00:22:01.790 --> 00:22:04.820
are still sitting
there, but so let's
00:22:04.820 --> 00:22:08.460
take a case where
it's triangular.
00:22:08.460 --> 00:22:14.870
Suppose A is like,
two one two zero.
00:22:17.920 --> 00:22:23.290
So there's a case that's
going to be trouble.
00:22:23.290 --> 00:22:25.040
There's a case that's
going to be trouble.
00:22:25.040 --> 00:22:26.360
First of all, what are the --
00:22:26.360 --> 00:22:29.410
I mean, we just --
00:22:29.410 --> 00:22:31.190
if we start with a
matrix, the first thing
00:22:31.190 --> 00:22:32.890
we do, practically
without thinking
00:22:32.890 --> 00:22:36.130
is compute the eigenvalues
and eigenvectors.
00:22:36.130 --> 00:22:36.630
OK.
00:22:36.630 --> 00:22:38.360
So what are the eigenvalues?
00:22:38.360 --> 00:22:41.130
You can tell me right
away what they are.
00:22:41.130 --> 00:22:43.880
They're two and two, right.
00:22:43.880 --> 00:22:47.770
It's a triangular matrix, so
when I do this determinant,
00:22:47.770 --> 00:22:51.680
shall I do this determinant
of A minus lambda I?
00:22:51.680 --> 00:22:59.990
I'll get this two minus lambda
one zero two minus lambda,
00:22:59.990 --> 00:23:01.700
right?
00:23:01.700 --> 00:23:06.890
I take that determinant, so I
make those into vertical bars
00:23:06.890 --> 00:23:09.130
to mean determinant.
00:23:09.130 --> 00:23:10.810
And what's the determinant?
00:23:10.810 --> 00:23:13.140
It's two minus lambda squared.
00:23:13.140 --> 00:23:14.570
What are the roots?
00:23:14.570 --> 00:23:17.410
Lambda equal two twice.
00:23:17.410 --> 00:23:22.640
So the eigenvalues are
lambda equals two and two.
00:23:22.640 --> 00:23:23.580
OK, fine.
00:23:23.580 --> 00:23:26.810
Now the next step,
find the eigenvectors.
00:23:26.810 --> 00:23:31.940
So I look for eigenvectors, and
what do I find for this guy?
00:23:31.940 --> 00:23:33.530
Eigenvectors for
this guy, when I
00:23:33.530 --> 00:23:38.340
subtract two minus the
identity, so A minus two
00:23:38.340 --> 00:23:42.280
I has zeros here.
00:23:45.420 --> 00:23:48.740
And I'm looking
for the null space.
00:23:48.740 --> 00:23:50.390
What's, what are
the eigenvectors?
00:23:50.390 --> 00:23:56.540
They're the -- the null
space of A minus lambda I.
00:23:56.540 --> 00:23:59.200
The null space is
only one dimensional.
00:23:59.200 --> 00:24:03.700
This is a case where I don't
have enough eigenvectors.
00:24:03.700 --> 00:24:07.940
My algebraic
multiplicity is two.
00:24:07.940 --> 00:24:10.520
I would say, when
I see, when I count
00:24:10.520 --> 00:24:16.210
how often the
eigenvalue is repeated,
00:24:16.210 --> 00:24:18.410
that's the algebraic
multiplicity.
00:24:18.410 --> 00:24:20.650
That's the multiplicity,
how many times
00:24:20.650 --> 00:24:22.770
is it the root of
the polynomial?
00:24:22.770 --> 00:24:28.340
My polynomial is two
minus lambda squared.
00:24:28.340 --> 00:24:30.040
It's a double root.
00:24:30.040 --> 00:24:33.240
So my algebraic
multiplicity is two.
00:24:33.240 --> 00:24:37.110
But the geometric multiplicity,
which looks for vectors,
00:24:37.110 --> 00:24:42.130
looks for eigenvectors, and
-- which means the null space
00:24:42.130 --> 00:24:46.500
of this thing, and the
only eigenvector is one
00:24:46.500 --> 00:24:47.360
zero.
00:24:47.360 --> 00:24:50.140
That's in the null space.
00:24:50.140 --> 00:24:52.600
Zero one is not
in the null space.
00:24:52.600 --> 00:24:54.540
The null space is
only one dimensional.
00:24:54.540 --> 00:24:58.960
So there's a matrix, my --
this A or the original A,
00:24:58.960 --> 00:25:02.310
that are not diagonalizable.
00:25:02.310 --> 00:25:06.360
I can't find two
independent eigenvectors.
00:25:06.360 --> 00:25:08.090
There's only one.
00:25:08.090 --> 00:25:08.700
OK.
00:25:08.700 --> 00:25:11.710
So that's the case that I'm --
00:25:11.710 --> 00:25:15.520
that's a case that I'm
not really handling.
00:25:15.520 --> 00:25:19.590
For example, when I
wrote down up here
00:25:19.590 --> 00:25:24.490
that the powers went to zero if
the eigenvalues were below one,
00:25:24.490 --> 00:25:29.210
I didn't really handle that
case of repeated eigenvalues,
00:25:29.210 --> 00:25:33.980
because my reasoning was
based on this formula.
00:25:33.980 --> 00:25:36.420
And this formula is based on
n independent eigenvectors.
00:25:36.420 --> 00:25:36.920
OK.
00:25:36.920 --> 00:25:45.610
Just to say then, there are
some matrices that we're, that,
00:25:45.610 --> 00:25:48.730
that we don't cover
through diagonalization,
00:25:48.730 --> 00:25:51.070
but the great majority we do.
00:25:51.070 --> 00:25:51.720
OK.
00:25:51.720 --> 00:25:54.040
And we, we're
always OK if we have
00:25:54.040 --> 00:25:56.550
different distinct eigenvalues.
00:25:56.550 --> 00:26:02.390
OK, that's the, like,
the typical case.
00:26:02.390 --> 00:26:04.660
Because for each
eigenvalue there's
00:26:04.660 --> 00:26:07.140
at least one eigenvector.
00:26:07.140 --> 00:26:11.530
The algebraic multiplicity here
is one for every eigenvalue
00:26:11.530 --> 00:26:14.080
and the geometric
multiplicity is one.
00:26:14.080 --> 00:26:15.580
There's one eigenvector.
00:26:15.580 --> 00:26:17.650
And they are independent.
00:26:17.650 --> 00:26:18.150
OK.
00:26:18.150 --> 00:26:18.650
OK.
00:26:21.690 --> 00:26:26.390
Now let me come back to
the important case, when,
00:26:26.390 --> 00:26:27.770
when we're OK.
00:26:27.770 --> 00:26:31.820
The important case, when
we are diagonalizable.
00:26:31.820 --> 00:26:38.060
Let me, look at --
00:26:38.060 --> 00:26:42.455
so -- let me solve
this equation.
00:26:46.680 --> 00:26:49.460
The equation will be each --
00:26:49.460 --> 00:26:57.146
I start with some -- start
with a given vector u0.
00:27:02.390 --> 00:27:06.080
And then my equation
is at every step,
00:27:06.080 --> 00:27:11.600
I multiply what I have by A.
00:27:11.600 --> 00:27:16.550
That, that equation ought
to be simple to handle.
00:27:19.940 --> 00:27:21.940
And I'd like to be
able to solve it.
00:27:21.940 --> 00:27:26.840
How would I find -- if I start
with a vector u0 and I multiply
00:27:26.840 --> 00:27:31.470
by A a hundred times,
what have I got?
00:27:31.470 --> 00:27:35.310
Well, I could certainly write
down a formula for the answer,
00:27:35.310 --> 00:27:39.295
so what, what -- so u1 is A u0.
00:27:42.170 --> 00:27:45.800
And u2 is -- what's u2 then?
00:27:45.800 --> 00:27:52.350
u2, I multiply -- u2 I get from
u1 by another multiplying by A,
00:27:52.350 --> 00:27:55.830
so I've got A twice.
00:27:55.830 --> 00:28:02.120
And my formula is
uk, after k steps,
00:28:02.120 --> 00:28:07.580
I've multiplied by A k
times the original u0.
00:28:07.580 --> 00:28:11.220
You see what I'm doing?
00:28:11.220 --> 00:28:14.050
The next section is
going to solve systems
00:28:14.050 --> 00:28:17.550
of differential equations.
00:28:17.550 --> 00:28:19.690
I'm going to have derivatives.
00:28:19.690 --> 00:28:23.370
This section is the nice one.
00:28:23.370 --> 00:28:26.190
It solves difference equations.
00:28:26.190 --> 00:28:28.550
I would call that a
difference equation.
00:28:28.550 --> 00:28:33.550
It's -- at first order, I would
call that a first-order system,
00:28:33.550 --> 00:28:40.150
because it connects only --
it only goes up one level.
00:28:40.150 --> 00:28:43.180
And I -- it's a system
because these are vectors
00:28:43.180 --> 00:28:45.960
and that's a matrix.
00:28:45.960 --> 00:28:48.470
And the solution is just that.
00:28:48.470 --> 00:28:49.090
OK.
00:28:49.090 --> 00:28:55.160
But, that's a nice formula.
00:28:55.160 --> 00:28:57.500
That's the, like, the
most compact formula
00:28:57.500 --> 00:29:01.630
I could ever get. u100 would
be A to the one hundred u0.
00:29:01.630 --> 00:29:06.480
But how would I
actually find u100?
00:29:06.480 --> 00:29:11.520
How would I find -- how would
I discover what u100 is?
00:29:11.520 --> 00:29:13.760
Let me, let me show you how.
00:29:16.620 --> 00:29:18.630
Here's the idea.
00:29:18.630 --> 00:29:23.090
If -- so to solve, to
really solve -- shall I say,
00:29:23.090 --> 00:29:26.920
to really solve --
00:29:26.920 --> 00:29:34.500
to really solve it, I would
take this initial vector u0
00:29:34.500 --> 00:29:39.420
and I would write it as a
combination of eigenvectors.
00:29:39.420 --> 00:29:47.320
To really solve, write u
nought as a combination,
00:29:47.320 --> 00:29:50.660
say certain amount of
the first eigenvector
00:29:50.660 --> 00:29:53.480
plus a certain amount of
the second eigenvector
00:29:53.480 --> 00:29:55.820
plus a certain amount
of the last eigenvector.
00:30:01.790 --> 00:30:04.740
Now multiply by A.
00:30:04.740 --> 00:30:07.350
You want to -- you got to
see the magic of eigenvectors
00:30:07.350 --> 00:30:08.520
working here.
00:30:08.520 --> 00:30:10.360
Multiply by A.
00:30:10.360 --> 00:30:13.910
So Au0 is what?
00:30:13.910 --> 00:30:16.930
So A times that.
00:30:16.930 --> 00:30:18.800
A times -- so what's A --
00:30:18.800 --> 00:30:21.390
I can separate it out
into n separate pieces,
00:30:21.390 --> 00:30:23.430
and that's the whole point.
00:30:23.430 --> 00:30:28.800
That each of those pieces is
going in its own merry way.
00:30:28.800 --> 00:30:31.320
Each of those pieces
is an eigenvector,
00:30:31.320 --> 00:30:35.810
and when I multiply by A,
what does this piece become?
00:30:35.810 --> 00:30:38.450
So that's some amount
of the first --
00:30:38.450 --> 00:30:41.030
let's suppose the eigenvectors
are normalized to be unit
00:30:41.030 --> 00:30:41.530
vectors.
00:30:44.750 --> 00:30:48.530
So that says what
the eigenvector is.
00:30:48.530 --> 00:30:51.340
It's a --
00:30:51.340 --> 00:30:55.220
And I need some multiple
of it to produce u0.
00:30:55.220 --> 00:30:56.120
OK.
00:30:56.120 --> 00:30:59.470
Now when I multiply
by A, what do I get?
00:30:59.470 --> 00:31:04.350
I get c1, which is just
a factor, times Ax1,
00:31:04.350 --> 00:31:07.865
but Ax1 is lambda one x1.
00:31:10.780 --> 00:31:17.060
When I multiply this by
A, I get c2 lambda two x2.
00:31:17.060 --> 00:31:20.740
And here I get cn lambda n xn.
00:31:20.740 --> 00:31:27.980
And suppose I multiply by A
to the hundredth power now.
00:31:27.980 --> 00:31:30.840
Can we, having done it,
multiplied by A, let's
00:31:30.840 --> 00:31:32.890
multiply by A to the hundredth.
00:31:32.890 --> 00:31:36.380
What happens to this first term
when I multiply by A to the one
00:31:36.380 --> 00:31:38.130
hundredth?
00:31:38.130 --> 00:31:41.620
It's got that factor
lambda to the hundredth.
00:31:41.620 --> 00:31:42.890
That's the key.
00:31:42.890 --> 00:31:48.440
That -- that's what I mean
by going its own merry way.
00:31:48.440 --> 00:31:52.320
It, it is pure eigenvector.
00:31:52.320 --> 00:31:55.850
It's exactly in a direction
where multiplication by A
00:31:55.850 --> 00:31:59.200
just brings in a scalar
factor, lambda one.
00:31:59.200 --> 00:32:02.240
So a hundred times brings
in this a hundred times.
00:32:02.240 --> 00:32:06.080
Hundred times lambda two,
hundred times lambda n.
00:32:06.080 --> 00:32:08.830
Actually, we're -- what
are we seeing here?
00:32:08.830 --> 00:32:15.040
We're seeing, this
same, lambda capital
00:32:15.040 --> 00:32:19.570
lambda to the hundredth as in
the, as in the diagonalization.
00:32:19.570 --> 00:32:22.350
And we're seeing
the S matrix, the,
00:32:22.350 --> 00:32:24.730
the matrix S of eigenvectors.
00:32:24.730 --> 00:32:29.440
That's what this has got to
-- this has got to amount to.
00:32:29.440 --> 00:32:40.030
A lambda to the hundredth power
times an S times this vector c
00:32:40.030 --> 00:32:43.490
that's telling us
how much of each one
00:32:43.490 --> 00:32:45.010
is in the original thing.
00:32:45.010 --> 00:32:49.010
So if, if I had to really
find the hundredth power,
00:32:49.010 --> 00:32:54.200
I would take u0, I would
expand it as a combination
00:32:54.200 --> 00:32:57.210
of eigenvectors --
this is really S,
00:32:57.210 --> 00:33:01.680
the eigenvector matrix, times
c, the, the coefficient vector.
00:33:04.240 --> 00:33:07.310
And then I would
immediately then,
00:33:07.310 --> 00:33:10.950
by inserting these hundredth
powers of eigenvalues,
00:33:10.950 --> 00:33:15.490
I'd have the answer.
00:33:15.490 --> 00:33:17.880
So -- huh, there must be --
00:33:17.880 --> 00:33:20.570
oh, let's see, OK.
00:33:20.570 --> 00:33:22.970
It's -- so, yeah.
00:33:22.970 --> 00:33:30.790
So if u100 is A to the hundredth
times u0, and u0 is S c --
00:33:30.790 --> 00:33:36.160
then you see this formula
is just this formula,
00:33:36.160 --> 00:33:40.840
which is the way I would
actually get hold of this,
00:33:40.840 --> 00:33:44.690
of this u100, which is --
00:33:44.690 --> 00:33:47.180
let me put it here.
00:33:47.180 --> 00:33:48.030
u100.
00:33:48.030 --> 00:33:51.070
The way I would actually
get hold of that, see what,
00:33:51.070 --> 00:33:57.400
what the solution is after
a hundred steps, would be --
00:33:57.400 --> 00:34:05.960
expand the initial vector
into eigenvectors and let each
00:34:05.960 --> 00:34:10.020
eigenvector go its own way,
multiplying by a hundred at --
00:34:10.020 --> 00:34:13.400
by lambda at every step,
and therefore by lambda
00:34:13.400 --> 00:34:16.030
to the hundredth power
after a hundred steps.
00:34:16.030 --> 00:34:18.050
Can I do an example?
00:34:18.050 --> 00:34:20.260
So that's the formulas.
00:34:20.260 --> 00:34:22.540
Now let me take an example.
00:34:22.540 --> 00:34:29.090
I'll use the Fibonacci
sequence as an example.
00:34:29.090 --> 00:34:31.590
So, so Fibonacci example.
00:34:39.830 --> 00:34:43.050
You remember the
Fibonacci numbers?
00:34:43.050 --> 00:34:48.150
If we start with one
and one as F0 -- oh,
00:34:48.150 --> 00:34:50.280
I think I start
with zero, maybe.
00:34:50.280 --> 00:34:54.550
Let zero and one
be the first ones.
00:34:54.550 --> 00:34:58.550
So there's F0 and F1, the
first two Fibonacci numbers.
00:34:58.550 --> 00:35:02.840
Then what's the rule
for Fibonacci numbers?
00:35:02.840 --> 00:35:04.130
Ah, they're the sum.
00:35:04.130 --> 00:35:08.030
The next one is the sum
of those, so it's one.
00:35:08.030 --> 00:35:11.110
The next one is the sum
of those, so it's two.
00:35:11.110 --> 00:35:14.010
The next one is the sum
of those, so it's three.
00:35:14.010 --> 00:35:16.190
Well, it looks like one
two three four five,
00:35:16.190 --> 00:35:19.350
but somehow it's not
going to do that way.
00:35:19.350 --> 00:35:21.380
The next one is five, right.
00:35:21.380 --> 00:35:22.640
Two and three makes five.
00:35:22.640 --> 00:35:26.090
The next one is eight.
00:35:26.090 --> 00:35:28.370
The next one is thirteen.
00:35:28.370 --> 00:35:33.245
And the one hundredth
Fibonacci number is what?
00:35:35.920 --> 00:35:37.600
That's my question.
00:35:37.600 --> 00:35:40.680
How could I get a formula
for the hundredth number?
00:35:40.680 --> 00:35:44.470
And, for example, how could
I answer the question,
00:35:44.470 --> 00:35:47.740
how fast are they growing?
00:35:47.740 --> 00:35:52.650
How fast are those
Fibonacci numbers growing?
00:35:52.650 --> 00:35:54.070
They're certainly growing.
00:35:54.070 --> 00:35:56.270
It's not a stable case.
00:35:56.270 --> 00:35:59.030
Whatever the eigenvalues
of whatever matrix it is,
00:35:59.030 --> 00:36:00.720
they're not smaller than one.
00:36:00.720 --> 00:36:02.540
These numbers are growing.
00:36:02.540 --> 00:36:04.450
But how fast are they growing?
00:36:04.450 --> 00:36:10.070
The answer lies
in the eigenvalue.
00:36:10.070 --> 00:36:12.450
So I've got to find the
matrix, so let me write down
00:36:12.450 --> 00:36:14.495
the Fibonacci rule.
00:36:17.610 --> 00:36:22.245
F(k+2) = F(k+1)+F k, right?
00:36:25.210 --> 00:36:28.280
Now that's not in my --
00:36:28.280 --> 00:36:32.420
I want to write that
as uk plus one and Auk.
00:36:32.420 --> 00:36:38.920
But right now what I've got is
a single equation, not a system,
00:36:38.920 --> 00:36:41.140
and it's second-order.
00:36:41.140 --> 00:36:44.290
It's like having a second-order
differential equation
00:36:44.290 --> 00:36:45.810
with second derivatives.
00:36:45.810 --> 00:36:47.580
I want to get first derivatives.
00:36:47.580 --> 00:36:49.200
Here I want to get
first differences.
00:36:49.200 --> 00:36:55.910
So the way, the way to do it
is to introduce uk will be
00:36:55.910 --> 00:36:57.960
a vector --
00:36:57.960 --> 00:36:59.125
see, a small trick.
00:37:01.920 --> 00:37:05.330
Let uk be a vector,
F(k+1) and Fk.
00:37:08.230 --> 00:37:12.680
So I'm going to get a two
by two system, first order,
00:37:12.680 --> 00:37:16.890
instead of a one -- instead of
a scalar system, second order,
00:37:16.890 --> 00:37:18.300
by a simple trick.
00:37:18.300 --> 00:37:22.820
I'm just going to add in an
equation F(k+1) equals F(k+1).
00:37:22.820 --> 00:37:28.980
That will be my second equation.
00:37:28.980 --> 00:37:33.940
Then this is my system,
this is my unknown,
00:37:33.940 --> 00:37:38.690
and what's my one step equation?
00:37:38.690 --> 00:37:45.120
So, so now u(k+1), that's --
so u(k+1) is the left side,
00:37:45.120 --> 00:37:47.620
and what have I got
here on the right side?
00:37:47.620 --> 00:37:52.530
I've got some matrix
multiplying uk.
00:37:52.530 --> 00:37:56.510
Can you, do -- can you
see that all right?
00:37:56.510 --> 00:37:59.450
if you can see it, then you
can tell me what the matrix is.
00:37:59.450 --> 00:38:02.860
Do you see that I'm
taking my system here.
00:38:02.860 --> 00:38:06.550
I artificially made
it into a system.
00:38:06.550 --> 00:38:10.540
I artificially made the
unknown into a vector.
00:38:10.540 --> 00:38:14.260
And now I'm ready to look
at and see what the matrix
00:38:14.260 --> 00:38:15.020
is.
00:38:15.020 --> 00:38:20.240
So do you see the left side,
u(k+1) is F(k+2) F(k+1),
00:38:20.240 --> 00:38:21.940
that's just what I want.
00:38:21.940 --> 00:38:25.590
On the right side, this
remember, this uk here --
00:38:25.590 --> 00:38:29.960
let me for the moment
put it as F(k+1) Fk.
00:38:29.960 --> 00:38:33.080
So what's the matrix?
00:38:33.080 --> 00:38:41.380
Well, that has a one and a one,
and that has a one and a zero.
00:38:41.380 --> 00:38:43.080
There's the matrix.
00:38:43.080 --> 00:38:47.880
Do you see that that gives
me the right-hand side?
00:38:47.880 --> 00:38:52.360
So there's the matrix A.
00:38:52.360 --> 00:38:56.810
And this is our friend uk.
00:38:56.810 --> 00:39:00.650
So we've got -- so
that simple trick --
00:39:00.650 --> 00:39:03.900
changed the second-order
scalar problem
00:39:03.900 --> 00:39:05.730
to a first-order system.
00:39:05.730 --> 00:39:08.750
Two b- u- with two unknowns.
00:39:08.750 --> 00:39:10.040
With a matrix.
00:39:10.040 --> 00:39:13.100
And now what do I do?
00:39:13.100 --> 00:39:16.240
Well, before I even think,
I find its eigenvalues
00:39:16.240 --> 00:39:18.170
and eigenvectors.
00:39:18.170 --> 00:39:21.170
So what are the eigenvalues and
eigenvectors of that matrix?
00:39:23.820 --> 00:39:24.320
Let's see.
00:39:24.320 --> 00:39:27.083
I always -- first let me just,
like, think for a minute.
00:39:29.720 --> 00:39:35.440
It's two by two, so this
shouldn't be impossible to do.
00:39:35.440 --> 00:39:37.020
Let's do it.
00:39:37.020 --> 00:39:37.670
OK.
00:39:37.670 --> 00:39:43.170
So my matrix, again,
is one one one zero.
00:39:46.170 --> 00:39:49.070
It's symmetric, by the way.
00:39:49.070 --> 00:39:56.070
So what I will eventually
know about symmetric matrices
00:39:56.070 --> 00:39:59.140
is that the eigenvalues
will come out real.
00:39:59.140 --> 00:40:02.290
I won't get any
complex numbers here.
00:40:02.290 --> 00:40:06.210
And the eigenvectors,
once I get those,
00:40:06.210 --> 00:40:08.520
actually will be orthogonal.
00:40:08.520 --> 00:40:11.190
But two by two, I'm
more interested in what
00:40:11.190 --> 00:40:13.740
the actual numbers are.
00:40:13.740 --> 00:40:16.230
What do I know about
the two numbers?
00:40:16.230 --> 00:40:18.190
Well, should do
you want me to find
00:40:18.190 --> 00:40:19.820
this determinant of A minus
00:40:19.820 --> 00:40:20.629
lambda I?
00:40:20.629 --> 00:40:21.128
Sure.
00:40:23.880 --> 00:40:27.910
So it's the determinant of
one minus lambda one one zero,
00:40:27.910 --> 00:40:28.410
right?
00:40:31.900 --> 00:40:33.400
Minus lambda, yes.
00:40:33.400 --> 00:40:33.994
God.
00:40:33.994 --> 00:40:34.493
OK.
00:40:38.030 --> 00:40:40.110
OK.
00:40:40.110 --> 00:40:42.240
There'll be two eigenvalues.
00:40:42.240 --> 00:40:45.160
What will -- tell me again
what I know about the two
00:40:45.160 --> 00:40:47.550
eigenvalues before
I go any further.
00:40:47.550 --> 00:40:49.590
Tell me something about
these two eigenvalues.
00:40:49.590 --> 00:40:51.770
What do they add up to?
00:40:51.770 --> 00:40:55.860
Lambda one plus lambda two is?
00:40:55.860 --> 00:41:02.390
Is the same as the trace down
the diagonal of the matrix.
00:41:02.390 --> 00:41:04.660
One and zero is one.
00:41:04.660 --> 00:41:08.320
So lambda one plus lambda two
should come out to be one.
00:41:08.320 --> 00:41:10.710
And lambda one times
lambda one times lambda two
00:41:10.710 --> 00:41:13.300
should come out to be
the determinant, which
00:41:13.300 --> 00:41:15.360
is minus one.
00:41:15.360 --> 00:41:18.440
So I'm expecting the
eigenvalues to add to one
00:41:18.440 --> 00:41:20.570
and to multiply to minus one.
00:41:20.570 --> 00:41:22.720
But let's just see
it happen here.
00:41:22.720 --> 00:41:26.680
If I multiply this out, I get --
that times that'll be a lambda
00:41:26.680 --> 00:41:30.290
squared minus lambda minus one.
00:41:30.290 --> 00:41:30.790
Good.
00:41:33.830 --> 00:41:36.570
Lambda squared minus
lambda minus one.
00:41:36.570 --> 00:41:43.250
Actually, I -- you see the b-
compare that with the original
00:41:43.250 --> 00:41:48.655
equation that I started with.
00:41:48.655 --> 00:41:49.780
F(k+2) - F(k+1)-Fk is zero.
00:41:49.780 --> 00:42:00.330
The recursion that -- that the
Fibonacci numbers satisfy is
00:42:00.330 --> 00:42:05.140
somehow showing up directly here
for the eigenvalues when we set
00:42:05.140 --> 00:42:06.230
that to zero.
00:42:06.230 --> 00:42:06.730
WK.
00:42:06.730 --> 00:42:09.530
Let's solve.
00:42:09.530 --> 00:42:14.200
Well, I would like to be able
to factor that, that quadratic,
00:42:14.200 --> 00:42:17.450
but I'm better off to use
the quadratic formula.
00:42:17.450 --> 00:42:19.880
Lambda is -- let's see.
00:42:19.880 --> 00:42:25.860
Minus b is one plus or minus
the square root of b squared,
00:42:25.860 --> 00:42:30.650
which is one, minus four
times that times that,
00:42:30.650 --> 00:42:33.740
which is plus four, over two.
00:42:37.614 --> 00:42:39.030
So that's the
square root of five.
00:42:42.600 --> 00:42:50.230
So the eigenvalues are
lambda one is one half of one
00:42:50.230 --> 00:42:57.000
plus square root of five, and
lambda two is one half of one
00:42:57.000 --> 00:42:59.220
minus square root of five.
00:42:59.220 --> 00:43:04.630
And sure enough, they -- those
add up to one and they multiply
00:43:04.630 --> 00:43:06.890
to give minus one.
00:43:06.890 --> 00:43:07.390
OK.
00:43:07.390 --> 00:43:09.400
Those are the two eigenvalues.
00:43:09.400 --> 00:43:12.250
How -- what are those
numbers approximately?
00:43:12.250 --> 00:43:18.060
Square root of five,
well, it's more than two
00:43:18.060 --> 00:43:19.030
but less than three.
00:43:19.030 --> 00:43:19.860
Hmm.
00:43:19.860 --> 00:43:25.330
It'd be nice to
know these numbers.
00:43:25.330 --> 00:43:30.480
I think, I think that -- so that
number comes out bigger than
00:43:30.480 --> 00:43:30.980
one, right?
00:43:30.980 --> 00:43:31.640
That's right.
00:43:31.640 --> 00:43:35.070
This number comes
out bigger than one.
00:43:35.070 --> 00:43:38.210
It's about one point six
one eight or something.
00:43:42.610 --> 00:43:44.700
Not exactly, but.
00:43:44.700 --> 00:43:48.300
And suppose it's one point six.
00:43:48.300 --> 00:43:52.390
Just, like, I think so.
00:43:52.390 --> 00:43:54.800
Then what's lambda two?
00:43:54.800 --> 00:43:57.870
Is, is lambda two
positive or negative?
00:43:57.870 --> 00:44:01.430
Negative, right, because I'm
-- it's, obviously negative,
00:44:01.430 --> 00:44:07.420
and I knew that the
-- so it's minus --
00:44:07.420 --> 00:44:16.720
and they add up to one, so minus
point six one eight, I guess.
00:44:16.720 --> 00:44:17.220
OK.
00:44:17.220 --> 00:44:17.800
A- and some more.
00:44:17.800 --> 00:44:18.520
Those are the two eigenvalues.
00:44:18.520 --> 00:44:19.830
One eigenvalue bigger than one,
one eigenvalue smaller than
00:44:19.830 --> 00:44:20.330
one.
00:44:20.330 --> 00:44:22.480
Actually, that's a great
situation to be in.
00:44:22.480 --> 00:44:25.430
Of course, the
eigenvalues are different,
00:44:25.430 --> 00:44:29.340
so there's no doubt whatever --
is this matrix diagonalizable?
00:44:32.270 --> 00:44:35.280
Is this matrix diagonalizable,
that original matrix A?
00:44:35.280 --> 00:44:35.990
Sure.
00:44:35.990 --> 00:44:38.030
We've got two
distinct eigenvalues
00:44:38.030 --> 00:44:44.294
and we can find the
eigenvectors in a moment.
00:44:44.294 --> 00:44:46.460
But they'll be independent,
we'll be diagonalizable.
00:44:46.460 --> 00:44:54.790
And now, you, you can already
answer my very first question.
00:44:54.790 --> 00:44:59.530
How fast are those Fibonacci
numbers increasing?
00:44:59.530 --> 00:45:01.080
How -- those --
they're increasing,
00:45:01.080 --> 00:45:01.810
right?
00:45:01.810 --> 00:45:03.970
They're not doubling
at every step.
00:45:03.970 --> 00:45:07.330
Let me -- let's look
again at these numbers.
00:45:07.330 --> 00:45:09.580
Five, eight, thirteen,
it's not obvious.
00:45:09.580 --> 00:45:14.060
The next one would be
twenty-one, thirty-four.
00:45:14.060 --> 00:45:20.924
So to get some idea of
what F one hundred is,
00:45:20.924 --> 00:45:21.840
can you give me any --
00:45:21.840 --> 00:45:24.820
I mean the crucial number --
00:45:24.820 --> 00:45:32.280
so it -- these --
it's approximately --
00:45:32.280 --> 00:45:37.970
what's controlling the growth
of these Fibonacci numbers?
00:45:37.970 --> 00:45:39.630
It's the eigenvalues.
00:45:39.630 --> 00:45:43.031
And which eigenvalue is
controlling that growth?
00:45:43.031 --> 00:45:43.530
The big one.
00:45:43.530 --> 00:45:50.380
So F100 will be approximately
some constant, c1 I guess,
00:45:50.380 --> 00:45:56.110
times this lambda one, this
one plus square root of five
00:45:56.110 --> 00:46:01.300
over two, to the
hundredth power.
00:46:01.300 --> 00:46:04.560
And the two hundredth F -- in
other words, the eigenvalue --
00:46:04.560 --> 00:46:08.950
the Fibonacci numbers are
growing by about that factor.
00:46:08.950 --> 00:46:13.780
Do you see that we, we've got
precise information about the,
00:46:13.780 --> 00:46:18.230
about the Fibonacci numbers
out of the eigenvalues?
00:46:18.230 --> 00:46:18.940
OK.
00:46:18.940 --> 00:46:21.880
And again, why is that true?
00:46:21.880 --> 00:46:26.750
Let me go over to this board
and s- show what I'm doing here.
00:46:26.750 --> 00:46:30.720
The -- the original initial
value is some combination
00:46:30.720 --> 00:46:31.730
of eigenvectors.
00:46:35.520 --> 00:46:39.470
And then when we start -- when
we start going out the theories
00:46:39.470 --> 00:46:42.580
of Fibonacci numbers, when
we start multiplying by A
00:46:42.580 --> 00:46:45.980
a hundred times, it's this
lambda one to the hundredth.
00:46:45.980 --> 00:46:51.070
This term is, is the
one that's taking over.
00:46:51.070 --> 00:46:54.920
It's -- I mean, that's big, like
one point six to the hundredth
00:46:54.920 --> 00:46:55.880
power.
00:46:55.880 --> 00:47:00.610
The second term is
practically nothing, right?
00:47:00.610 --> 00:47:04.300
The point six, or minus point
six, to the hundredth power
00:47:04.300 --> 00:47:08.180
is an extremely small,
extremely small number.
00:47:08.180 --> 00:47:11.410
So this is -- there're
only two terms,
00:47:11.410 --> 00:47:13.020
because we're two by two.
00:47:13.020 --> 00:47:16.430
This number is -- this
piece of it is there,
00:47:16.430 --> 00:47:21.270
but it's, it's disappearing,
where this piece is there
00:47:21.270 --> 00:47:23.890
and it's growing and
controlling everything.
00:47:23.890 --> 00:47:27.230
So, so really the --
we're doing, like,
00:47:27.230 --> 00:47:29.100
problems that are evolving.
00:47:29.100 --> 00:47:33.390
We're doing dynamic
u- instead of Ax=b,
00:47:33.390 --> 00:47:35.440
that's a static problem.
00:47:35.440 --> 00:47:36.930
We're now we're doing dynamics.
00:47:36.930 --> 00:47:39.740
A, A squared, A cubed,
things are evolving in
00:47:39.740 --> 00:47:40.440
time.
00:47:40.440 --> 00:47:44.660
And the eigenvalues are
the crucial, numbers.
00:47:44.660 --> 00:47:45.640
OK.
00:47:45.640 --> 00:47:52.490
I guess to complete
this, I better
00:47:52.490 --> 00:47:56.420
write down the eigenvectors.
00:47:56.420 --> 00:47:59.160
So we should complete
the, the whole process
00:47:59.160 --> 00:48:01.200
by finding the eigenvectors.
00:48:01.200 --> 00:48:03.820
OK, well, I have to --
up in the corner, then,
00:48:03.820 --> 00:48:07.670
I have to look at
A minus lambda I.
00:48:07.670 --> 00:48:15.800
So A minus lambda I is this one
minus lambda one one and minus
00:48:15.800 --> 00:48:16.930
lambda.
00:48:16.930 --> 00:48:19.990
And now can we spot an
eigenvector out of that?
00:48:19.990 --> 00:48:23.070
That's, that's, for
these two lambdas,
00:48:23.070 --> 00:48:24.415
this matrix is singular.
00:48:27.380 --> 00:48:30.350
I guess the eigenvector -- two
by two ought to be, I mean,
00:48:30.350 --> 00:48:31.260
easy.
00:48:31.260 --> 00:48:33.960
So if I know that this
matrix is singular,
00:48:33.960 --> 00:48:37.100
then u- seems to
me the eigenvector
00:48:37.100 --> 00:48:41.340
has to be lambda and one,
because that multiplication
00:48:41.340 --> 00:48:43.500
will give me the zero.
00:48:43.500 --> 00:48:47.240
And this multiplication gives
me -- better give me also zero.
00:48:47.240 --> 00:48:48.650
Do you see why it does?
00:48:48.650 --> 00:48:52.670
This is the minus lambda
squared plus lambda plus one.
00:48:52.670 --> 00:48:56.360
It's the thing that's zero
because these lambdas are
00:48:56.360 --> 00:48:56.930
special.
00:48:56.930 --> 00:48:58.490
There's the eigenvector.
00:48:58.490 --> 00:49:07.900
x1 is lambda one one,
and x2 is lambda two one.
00:49:07.900 --> 00:49:12.520
I did that as a little trick
that was available in the two
00:49:12.520 --> 00:49:14.130
by two case.
00:49:14.130 --> 00:49:17.680
So now I finally have to --
00:49:17.680 --> 00:49:20.390
oh, I have to take
the initial u0 now.
00:49:20.390 --> 00:49:22.710
So to complete this
example entirely,
00:49:22.710 --> 00:49:26.680
I have to say, OK, what was u0?
00:49:26.680 --> 00:49:28.740
u0 was F1 F0.
00:49:28.740 --> 00:49:40.630
So u0, the starting vector is
F1 F0, and those were one and
00:49:40.630 --> 00:49:41.130
zero.
00:49:43.910 --> 00:49:47.150
So I have to use that vector.
00:49:47.150 --> 00:49:50.390
So I have to look
for, for a multiple
00:49:50.390 --> 00:49:56.100
of the first eigenvector and
the second to produce u0,
00:49:56.100 --> 00:49:58.070
the one zero
00:49:58.070 --> 00:49:58.570
vector.
00:49:58.570 --> 00:50:05.430
This is what will find c1
and c2, and then I'm done.
00:50:05.430 --> 00:50:10.470
Do you -- so let me instead
of, in the last five seconds,
00:50:10.470 --> 00:50:14.920
grinding out a formula,
let me repeat the idea.
00:50:14.920 --> 00:50:19.100
Because I'd really -- it's
the idea that's central.
00:50:19.100 --> 00:50:21.610
When things are
evolving in time --
00:50:21.610 --> 00:50:25.730
let me come back to this board,
because the ideas are here.
00:50:25.730 --> 00:50:30.400
When things are evolving in
time by a first-order system,
00:50:30.400 --> 00:50:34.480
starting from an
original u0, the key
00:50:34.480 --> 00:50:39.956
is find the eigenvalues
and eigenvectors of A.
00:50:39.956 --> 00:50:41.580
That will tell --
those eigenvectors --
00:50:41.580 --> 00:50:46.710
the eigenvalues will already
tell you what's happening.
00:50:46.710 --> 00:50:48.540
Is the solution
blowing up, is it
00:50:48.540 --> 00:50:51.390
going to zero, what's it doing.
00:50:51.390 --> 00:50:56.240
And then to, to find
out exactly a formula,
00:50:56.240 --> 00:50:59.290
you have to take
your u0 and write it
00:50:59.290 --> 00:51:03.270
as a combination of
eigenvectors and then
00:51:03.270 --> 00:51:05.820
follow each
eigenvector separately.
00:51:05.820 --> 00:51:10.930
And that's really what this
formula, the formula for, --
00:51:10.930 --> 00:51:15.190
that's what the formula
for A to the K is doing.
00:51:15.190 --> 00:51:17.180
So remember that
formula for A to the K
00:51:17.180 --> 00:51:21.770
is S lambda to the K S inverse.
00:51:21.770 --> 00:51:22.280
OK.
00:51:22.280 --> 00:51:24.590
That's, that's
difference equations.
00:51:24.590 --> 00:51:33.460
And you just have to -- so the,
the homework will give some
00:51:33.460 --> 00:51:41.180
examples, different from
Fibonacci, to follow through.
00:51:41.180 --> 00:51:48.900
And next time will be
differential equations.
00:51:48.900 --> 00:51:50.450
Thanks.