WEBVTT

00:00:00.230 --> 00:00:01.021
GILBERT STRANG: OK.

00:00:01.021 --> 00:00:07.090
So this video is about using
eigenvectors and eigenvalues

00:00:07.090 --> 00:00:09.560
to take powers of
a matrix, and I'll

00:00:09.560 --> 00:00:13.100
show you why we want to
take powers of a matrix.

00:00:13.100 --> 00:00:16.655
And then the next
video would be using

00:00:16.655 --> 00:00:21.330
eigenvalues and eigenvectors to
solve differential equations.

00:00:21.330 --> 00:00:24.840
The two big applications.

00:00:24.840 --> 00:00:26.980
So here's the first application.

00:00:26.980 --> 00:00:29.670
Let me remember the main facts.

00:00:29.670 --> 00:00:32.700
That if A-- if.

00:00:32.700 --> 00:00:34.220
This is an important point.

00:00:34.220 --> 00:00:40.180
Not every matrix has n
independent eigenvectors that

00:00:40.180 --> 00:00:42.570
would go into matrix
V. You remember

00:00:42.570 --> 00:00:47.040
V is the eigenvector
matrix, and I

00:00:47.040 --> 00:00:51.930
need n independent eigenvectors
in order to have a V inverse,

00:00:51.930 --> 00:00:53.920
to make that formula correct.

00:00:53.920 --> 00:00:56.870
So that's the key
formula for using

00:00:56.870 --> 00:00:58.870
eigenvalues and eigenvectors.

00:00:58.870 --> 00:01:06.080
And the case where we might
run short of eigenvectors

00:01:06.080 --> 00:01:11.010
is when maybe one
eigenvalue is repeated.

00:01:11.010 --> 00:01:13.530
It's a double eigenvalue,
and maybe there's

00:01:13.530 --> 00:01:16.080
only one eigenvector
to go with it.

00:01:16.080 --> 00:01:20.320
Every eigenvalue's got at
least one line of eigenvectors.

00:01:20.320 --> 00:01:25.160
But we might not have two when
the eigenvalue is repeated

00:01:25.160 --> 00:01:27.680
or we might.

00:01:27.680 --> 00:01:33.000
So there are cases when
this formula doesn't apply.

00:01:33.000 --> 00:01:35.590
Because I must be able
to take V inverse,

00:01:35.590 --> 00:01:38.570
I need n independent
columns there.

00:01:38.570 --> 00:01:39.220
OK.

00:01:39.220 --> 00:01:42.790
But when it works,
it really works.

00:01:42.790 --> 00:01:46.850
So the n-th power,
just remembering,

00:01:46.850 --> 00:01:51.410
is V lambda V inverse, V
lambda V inverse, n times.

00:01:51.410 --> 00:01:54.130
But every time I have
a V inverse and a V,

00:01:54.130 --> 00:01:55.760
that's the identity.

00:01:55.760 --> 00:01:58.380
So I move V out
at the beginning.

00:01:58.380 --> 00:02:01.990
I have lambda, lambda,
lambda, n of those,

00:02:01.990 --> 00:02:04.570
and a V inverse at the very end.

00:02:04.570 --> 00:02:11.240
So that's the nice result for
the n-th power of a matrix.

00:02:11.240 --> 00:02:13.860
Now I have to show you
how to use that formula,

00:02:13.860 --> 00:02:16.790
how to use eigenvalues
and eigenvectors.

00:02:16.790 --> 00:02:17.580
OK.

00:02:17.580 --> 00:02:22.340
So we know we can take
powers of a matrix.

00:02:22.340 --> 00:02:26.910
So first of all, what
kind of equation?

00:02:26.910 --> 00:02:29.530
There's an equation.

00:02:29.530 --> 00:02:31.460
That's called a
difference equation.

00:02:31.460 --> 00:02:36.830
It goes from step k to step
k plus 1 to step k plus 2.

00:02:36.830 --> 00:02:41.920
It steps one at a time and
every time multiplies by A.

00:02:41.920 --> 00:02:47.550
So after k steps, I've
multiplied by A k times

00:02:47.550 --> 00:02:50.270
from the original u0.

00:02:50.270 --> 00:02:54.490
So instead of a
differential equation,

00:02:54.490 --> 00:03:00.150
it's a step difference
equation with u0 given.

00:03:02.750 --> 00:03:05.670
And there's the solution.

00:03:05.670 --> 00:03:08.950
That's the quickest
form of the solution.

00:03:08.950 --> 00:03:12.390
A to the k-th power,
that's what we want.

00:03:12.390 --> 00:03:15.350
But just writing
A to the k, if we

00:03:15.350 --> 00:03:18.382
had a big matrix, to
take its hundredth power

00:03:18.382 --> 00:03:19.215
would be ridiculous.

00:03:22.500 --> 00:03:24.690
But with eigenvalues
and eigenvectors,

00:03:24.690 --> 00:03:26.560
we have that formula.

00:03:26.560 --> 00:03:27.940
OK.

00:03:27.940 --> 00:03:31.010
But now I want to think.

00:03:31.010 --> 00:03:35.250
Let me try to turn that
formula into something

00:03:35.250 --> 00:03:38.000
that you just naturally see.

00:03:38.000 --> 00:03:39.570
And we know what happens.

00:03:39.570 --> 00:03:44.490
If u0 is an eigenvector,
if u0 is an eigenvector,

00:03:44.490 --> 00:03:47.880
that probably won't happen
because there are just

00:03:47.880 --> 00:03:50.660
n eigenvector directions.

00:03:50.660 --> 00:03:54.290
But if it happened to be an
eigenvector, then every step

00:03:54.290 --> 00:04:00.590
we'd multiply by lambda, and
we'd have the answer, lambda k

00:04:00.590 --> 00:04:01.830
times.

00:04:01.830 --> 00:04:08.580
But what do we do for all
the initial vectors u0 which

00:04:08.580 --> 00:04:12.350
are maybe not an eigenvector?

00:04:15.130 --> 00:04:17.310
How do I proceed?

00:04:17.310 --> 00:04:23.250
How do I use eigenvectors when
my original starting vector

00:04:23.250 --> 00:04:25.352
is not an eigenvector?

00:04:25.352 --> 00:04:31.600
And the answer is, it will be
a combination of eigenvectors.

00:04:31.600 --> 00:04:37.530
So making this formula
real starts with this.

00:04:37.530 --> 00:04:46.016
So I write u0 as a combination
of the eigenvectors.

00:04:54.820 --> 00:04:57.230
And I can do it
because if I have

00:04:57.230 --> 00:05:01.090
n independent eigenvectors,
that will be a basis.

00:05:01.090 --> 00:05:04.260
Every vector can be
written in the basis.

00:05:04.260 --> 00:05:09.750
So I'm looking there at a
combination of eigenvectors.

00:05:09.750 --> 00:05:17.650
And now the point is that as
I take these steps to u1--

00:05:17.650 --> 00:05:19.396
what will u1 be?

00:05:19.396 --> 00:05:24.270
u1 will be Au0.

00:05:24.270 --> 00:05:28.190
So I'm multiplying by A. So
when I multiply this by A,

00:05:28.190 --> 00:05:28.880
what happens?

00:05:28.880 --> 00:05:30.430
That's the whole point.

00:05:30.430 --> 00:05:37.120
c1, A times x1 is
lambda 1 times x1.

00:05:37.120 --> 00:05:39.250
It's an eigenvector.

00:05:39.250 --> 00:05:43.790
c2 tells me how much of the
second eigenvector I have.

00:05:43.790 --> 00:05:49.030
When I multiply by A, that
multiplies by lambda 2,

00:05:49.030 --> 00:05:54.241
and so on, cn lambda n xn.

00:05:58.756 --> 00:06:00.140
And that's the thing.

00:06:00.140 --> 00:06:03.490
Each eigenvector
goes its own way,

00:06:03.490 --> 00:06:06.260
and I just add them together.

00:06:06.260 --> 00:06:06.760
OK.

00:06:06.760 --> 00:06:12.760
And what about A
to the k-th power?

00:06:12.760 --> 00:06:16.100
Now, that will give me uk.

00:06:16.100 --> 00:06:20.930
And what happens if
I do this k times?

00:06:20.930 --> 00:06:24.070
You've seen what I got
after doing it one time.

00:06:24.070 --> 00:06:27.910
If I do it k times,
that lambda 1

00:06:27.910 --> 00:06:32.125
that multiplies its eigenvector
will happen k times.

00:06:32.125 --> 00:06:35.585
So I'll have lambda
1 to the k-th power.

00:06:35.585 --> 00:06:38.396
Do you see that?

00:06:38.396 --> 00:06:41.870
Every step brings
another factor lambda 1.

00:06:41.870 --> 00:06:44.830
Every step brings
another factor lambda 2.

00:06:44.830 --> 00:06:48.180
Every step brings--
that's the answer.

00:06:48.180 --> 00:06:56.911
That is-- well, that answer
must be the same as this answer.

00:06:56.911 --> 00:06:59.800
And I'll do an
example in a minute.

00:06:59.800 --> 00:07:02.550
Right now, I'm just getting
the formulas straight.

00:07:02.550 --> 00:07:05.790
So I have the quickest
possible formula,

00:07:05.790 --> 00:07:07.540
but it doesn't help me much.

00:07:07.540 --> 00:07:10.690
I have the using the
eigenvectors and eigenvalue

00:07:10.690 --> 00:07:11.550
formula.

00:07:11.550 --> 00:07:14.410
And here I have it
that, really, it's

00:07:14.410 --> 00:07:22.490
the same thing written out as
a combination of eigenvectors.

00:07:22.490 --> 00:07:24.750
And then this is my answer.

00:07:24.750 --> 00:07:30.740
That's my answer to the--
that's my solution uk.

00:07:30.740 --> 00:07:31.240
That's it.

00:07:31.240 --> 00:07:34.590
So that must be
the same as that.

00:07:34.590 --> 00:07:38.612
Do you want to just
think for one minute

00:07:38.612 --> 00:07:44.200
why this answer is the
same as that answer?

00:07:44.200 --> 00:07:46.250
Well, we need to know
what are the c's?

00:07:46.250 --> 00:07:50.520
Well, the c's came from u0.

00:07:50.520 --> 00:07:55.330
And if I write that equation
for the c's-- do you see what I

00:07:55.330 --> 00:07:57.930
have as an equation for the c's?

00:07:57.930 --> 00:08:03.970
u0 is this combination
of eigenvectors.

00:08:03.970 --> 00:08:06.090
That's a matrix multiplication.

00:08:06.090 --> 00:08:11.100
That's the eigenvector matrix
multiplied by the vector

00:08:11.100 --> 00:08:14.770
c of coefficients, right?

00:08:14.770 --> 00:08:17.430
That's how a matrix
multiplies a vector.

00:08:17.430 --> 00:08:24.465
The columns, which are the x's,
multiply the numbers c1, c2,

00:08:24.465 --> 00:08:25.260
cn.

00:08:25.260 --> 00:08:25.970
There it is.

00:08:25.970 --> 00:08:27.430
That's the same as that.

00:08:27.430 --> 00:08:30.670
So u0 is Vc.

00:08:30.670 --> 00:08:35.230
So c is V inverse u0.

00:08:35.230 --> 00:08:36.919
Oh, that's nice.

00:08:39.580 --> 00:08:42.740
That's telling us what
are the coefficients, what

00:08:42.740 --> 00:08:45.710
are the numbers, what
amount of each eigenvector

00:08:45.710 --> 00:08:48.410
is present in u0.

00:08:48.410 --> 00:08:49.680
This is the equation.

00:08:49.680 --> 00:08:52.760
But look, you see
there that V inverse

00:08:52.760 --> 00:08:57.550
u0, that's the first part
there of the formula.

00:08:57.550 --> 00:09:01.380
I'm trying to match this
formula with that one.

00:09:01.380 --> 00:09:05.050
And I'm taking one
step to recognize

00:09:05.050 --> 00:09:09.320
that this part of the
formula is exactly c.

00:09:09.320 --> 00:09:11.270
You might want to
think about that.

00:09:11.270 --> 00:09:14.630
Run this video once more
just to see that step.

00:09:14.630 --> 00:09:15.810
Now what do we do?

00:09:15.810 --> 00:09:18.000
We've got the lambdas.

00:09:18.000 --> 00:09:22.240
So I'm taking care of
the c's, you could say.

00:09:22.240 --> 00:09:25.310
Now I need the lambda
to the k-th power--

00:09:25.310 --> 00:09:28.560
lambda 1 to the k-th, lambda
2 to the k-th, lambda n

00:09:28.560 --> 00:09:29.450
to the k-th.

00:09:29.450 --> 00:09:33.730
That's exactly
what goes in here.

00:09:33.730 --> 00:09:38.620
So that factor is producing
the lambdas to the k-th power.

00:09:38.620 --> 00:09:44.640
And finally, this factor has--
everybody's remembering here.

00:09:44.640 --> 00:09:50.170
V is the eigenvector
matrix x1, x2, to xn.

00:09:54.000 --> 00:09:58.520
And when I multiply by V,
it's a matrix times a vector.

00:09:58.520 --> 00:09:59.800
This is a matrix.

00:09:59.800 --> 00:10:01.240
This is a vector.

00:10:01.240 --> 00:10:06.460
And I get the combination--
I'm adding up.

00:10:06.460 --> 00:10:09.730
I'm reconstructing the solution.

00:10:09.730 --> 00:10:13.580
So first I break
up u0 into the x's.

00:10:13.580 --> 00:10:16.310
I multiply them by the
lambdas, and then I

00:10:16.310 --> 00:10:17.520
put them all together.

00:10:17.520 --> 00:10:21.230
I reconstruct uk.

00:10:21.230 --> 00:10:23.020
I hope you like that.

00:10:23.020 --> 00:10:26.180
This formula, which it's
like common sense formula,

00:10:26.180 --> 00:10:32.740
is exactly what that algebra
formula, matrix formula, says.

00:10:32.740 --> 00:10:33.240
OK.

00:10:33.240 --> 00:10:34.670
I have to do an example.

00:10:34.670 --> 00:10:36.730
Let me finish with an example.

00:10:36.730 --> 00:10:38.260
OK.

00:10:38.260 --> 00:10:40.536
Here's a matrix example.

00:10:45.080 --> 00:10:49.860
A equals-- this'll
be a special matrix.

00:10:49.860 --> 00:10:54.150
I'm going to make the
first column add up to 1,

00:10:54.150 --> 00:10:59.600
and I'm going to make the
second column add up to 1.

00:10:59.600 --> 00:11:03.800
And I'm using positive numbers.

00:11:03.800 --> 00:11:05.580
They're adding to 1.

00:11:05.580 --> 00:11:08.430
And that's called
a Markov matrix.

00:11:08.430 --> 00:11:12.120
So it's nice to know that
name-- Markov matrix.

00:11:18.360 --> 00:11:20.280
One of the beauties
of linear algebra

00:11:20.280 --> 00:11:25.200
is the variety of matrices--
orthogonal matrices,

00:11:25.200 --> 00:11:28.030
symmetric matrices.

00:11:28.030 --> 00:11:30.350
We'll see more and
more kinds of matrices.

00:11:30.350 --> 00:11:33.120
And sometimes they're
named after somebody

00:11:33.120 --> 00:11:35.760
who understood that
they were important

00:11:35.760 --> 00:11:38.120
and found their
special properties.

00:11:38.120 --> 00:11:42.760
So a Markov matrix is a matrix
with the columns adding up

00:11:42.760 --> 00:11:50.530
to 1 and no negative numbers
involved, no negative numbers.

00:11:50.530 --> 00:11:51.640
OK.

00:11:51.640 --> 00:11:53.850
That's just by the way.

00:11:53.850 --> 00:11:58.370
But it tells us something
about the eigenvalues here.

00:11:58.370 --> 00:12:01.490
Well, we could find
those two eigenvalues.

00:12:01.490 --> 00:12:04.420
We could do the determinant.

00:12:04.420 --> 00:12:06.290
You remember how to
find eigenvalues.

00:12:06.290 --> 00:12:12.400
The determinant of lambda I
minus A will be something.

00:12:12.400 --> 00:12:13.850
Could easily figure it out.

00:12:13.850 --> 00:12:17.110
There's always a lambda squared,
because it's two by two,

00:12:17.110 --> 00:12:19.032
minus the trace.

00:12:19.032 --> 00:12:24.880
0.8 and 0.7 is 1.5 lambda,
plus the determinant.

00:12:24.880 --> 00:12:30.230
0.56 minus 0.06 is 0.50, 0.5.

00:12:30.230 --> 00:12:31.610
And you set that to 0.

00:12:34.390 --> 00:12:37.890
And you get a result that
one of the eigenvalues

00:12:37.890 --> 00:12:45.840
is-- this factors into lambda
minus 1, lambda minus 1/2.

00:12:45.840 --> 00:12:48.940
And the cool fact
about Markov matrices

00:12:48.940 --> 00:12:52.490
is lambda equal 1 is
always an eigenvalue.

00:12:52.490 --> 00:12:55.820
So lambda equal 1
is an eigenvalue.

00:12:55.820 --> 00:12:57.600
Let's call that lambda 1.

00:12:57.600 --> 00:13:01.100
And lambda 2 is an
eigenvalue, and that

00:13:01.100 --> 00:13:07.080
depends on the numbers,
and it's 1/2, 0.5, 0.5.

00:13:07.080 --> 00:13:08.560
Those are the eigenvalues.

00:13:08.560 --> 00:13:12.090
1 plus 1/2 is 1.5.

00:13:12.090 --> 00:13:15.980
The trace is 0.8 plus 0.7, 1.5.

00:13:15.980 --> 00:13:17.950
Are we good for those
two eigenvalues?

00:13:17.950 --> 00:13:19.370
Yes.

00:13:19.370 --> 00:13:22.400
And then we find the
eigenvectors that go with them.

00:13:22.400 --> 00:13:29.520
I think that this eigenvector
turns out to be 0.6, 0.4.

00:13:29.520 --> 00:13:30.250
I could check.

00:13:30.250 --> 00:13:36.030
If I multiply, I get
0.48 plus 0.12 is 0.60,

00:13:36.030 --> 00:13:39.150
and that's the same as 0.6.

00:13:39.150 --> 00:13:41.600
And that goes with eigenvalue 1.

00:13:41.600 --> 00:13:46.810
And I think that this
eigenvector is 1, minus 1.

00:13:46.810 --> 00:13:50.210
Maybe that's always for a
two-by-two Markov matrix.

00:13:50.210 --> 00:13:52.790
Maybe that's always
the second eigenvector.

00:13:52.790 --> 00:13:54.620
I think that's probably good.

00:13:54.620 --> 00:13:55.140
Right.

00:13:55.140 --> 00:13:56.003
OK.

00:13:56.003 --> 00:13:56.850
Yeah.

00:13:56.850 --> 00:13:59.840
All right.

00:13:59.840 --> 00:14:01.920
What now?

00:14:01.920 --> 00:14:02.700
What now?

00:14:02.700 --> 00:14:05.700
I want to use the
eigenvalues and eigenvectors,

00:14:05.700 --> 00:14:10.620
and I'm going to
write out now uk.

00:14:10.620 --> 00:14:17.250
So if I apply A k
times to u0, I get uk.

00:14:17.250 --> 00:14:23.990
And that's c1 1 to
the k-- this lambda 1

00:14:23.990 --> 00:14:32.600
is 1-- times its eigenvector
0.6, 0.4 plus c2,

00:14:32.600 --> 00:14:36.090
however much of the
second eigenvector

00:14:36.090 --> 00:14:40.510
is in there, times
its eigenvalue,

00:14:40.510 --> 00:14:45.010
1/2 to the k-th power
times its eigenvector,

00:14:45.010 --> 00:14:48.956
the second eigenvector,
1, negative 1.

00:14:48.956 --> 00:14:51.720
That is a formula.

00:14:51.720 --> 00:14:56.985
c1 lambda 1 to the k-th
power x1 plus c2 lambda 2

00:14:56.985 --> 00:14:59.660
to the k-th power x2.

00:14:59.660 --> 00:15:04.540
And c1 and c2 would
be determined by u0,

00:15:04.540 --> 00:15:07.330
which I haven't picked a u0.

00:15:07.330 --> 00:15:08.060
I could.

00:15:08.060 --> 00:15:11.420
But I can make the
point, because the point

00:15:11.420 --> 00:15:18.790
I want to make is true for
every u0, every example.

00:15:18.790 --> 00:15:20.410
And here's the point.

00:15:20.410 --> 00:15:22.760
What happens as k gets large?

00:15:22.760 --> 00:15:27.500
What happens if Markov
multiplies his matrix over

00:15:27.500 --> 00:15:32.000
and over again, which is what
happens in a Markov process,

00:15:32.000 --> 00:15:33.770
a Markov process?

00:15:33.770 --> 00:15:37.800
This is like-- actually,
the whole Google algorithm

00:15:37.800 --> 00:15:42.610
for page rank is based
on a Markov matrix.

00:15:42.610 --> 00:15:46.540
So that's like a
multi-billion-dollar company

00:15:46.540 --> 00:15:52.240
that's based on the
properties of a Markov matrix.

00:15:52.240 --> 00:15:55.060
And you repeat it and repeat it.

00:15:55.060 --> 00:16:00.500
That just means that Google
is looping through the web,

00:16:00.500 --> 00:16:06.680
and if it sees a website more
often, the ranking goes up.

00:16:06.680 --> 00:16:09.670
And if it never sees
my website, then

00:16:09.670 --> 00:16:14.240
for that, when it was
googling some special subject,

00:16:14.240 --> 00:16:20.080
it never came to your website
and mine, we didn't get ranked.

00:16:20.080 --> 00:16:20.870
OK.

00:16:20.870 --> 00:16:22.840
So this goes to 0.

00:16:22.840 --> 00:16:26.590
1/2 to the-- it goes
fast to 0, quickly to 0.

00:16:26.590 --> 00:16:28.870
So that goes to 0.

00:16:28.870 --> 00:16:31.900
And of course, that stays
exactly where it is.

00:16:31.900 --> 00:16:34.870
So there's a steady state.

00:16:34.870 --> 00:16:40.500
What happens if page rank had
only two websites to rank,

00:16:40.500 --> 00:16:43.270
if Google was just
ranking two websites?

00:16:43.270 --> 00:16:48.000
Then its initial ranking,
they don't know what it is.

00:16:48.000 --> 00:16:51.980
But by repeating the
Markov matrix and this part

00:16:51.980 --> 00:16:55.590
going to 0, right, goes
to 0 because of 1/2

00:16:55.590 --> 00:17:00.530
to the k-th power, there
is the ranking, 0.6, 0.4.

00:17:00.530 --> 00:17:04.130
That's where Google-- so
this first website would

00:17:04.130 --> 00:17:06.980
be ranked above the second one.

00:17:06.980 --> 00:17:07.530
OK.

00:17:07.530 --> 00:17:12.180
There's an example of a process
that's repeated and repeated,

00:17:12.180 --> 00:17:16.910
and so a Markov matrix comes in.

00:17:16.910 --> 00:17:20.599
This business of adding up to
1 means that nothing is lost.

00:17:20.599 --> 00:17:21.760
Nothing is created.

00:17:21.760 --> 00:17:23.130
You're just moving.

00:17:23.130 --> 00:17:26.550
At every step, you
take a Markov step.

00:17:26.550 --> 00:17:29.420
And the question is,
where do you end up?

00:17:29.420 --> 00:17:32.950
Well, you keep moving,
but this vector

00:17:32.950 --> 00:17:36.060
tells you how much of
the time you're spending

00:17:36.060 --> 00:17:40.905
in the two possible locations.

00:17:40.905 --> 00:17:43.100
And this one goes to 0.

00:17:43.100 --> 00:17:43.940
OK.

00:17:43.940 --> 00:17:47.450
Powers of a matrix,
powers of a Markov matrix.

00:17:47.450 --> 00:17:49.230
Thank you.