WEBVTT
00:00:00.000 --> 00:00:11.240
-- two, one and -- okay.
00:00:11.240 --> 00:00:18.910
Here is a lecture on the
applications of eigenvalues
00:00:18.910 --> 00:00:24.070
and, if I can -- so that
will be Markov matrices.
00:00:24.070 --> 00:00:27.260
I'll tell you what
a Markov matrix is,
00:00:27.260 --> 00:00:32.070
so this matrix A will
be a Markov matrix
00:00:32.070 --> 00:00:38.890
and I'll explain how they
come in applications.
00:00:38.890 --> 00:00:42.060
And -- and then if I have time,
I would like to say a little
00:00:42.060 --> 00:00:46.060
bit about Fourier series, which
is a fantastic application
00:00:46.060 --> 00:00:48.540
of the projection chapter.
00:00:48.540 --> 00:00:49.100
Okay.
00:00:49.100 --> 00:00:50.930
What's a Markov matrix?
00:00:50.930 --> 00:01:01.940
Can I just write down a typical
Markov matrix, say .1, .2, .7,
00:01:01.940 --> 00:01:08.820
.01, .99 0, let's
say, .3, .3, .4.
00:01:08.820 --> 00:01:09.970
Okay.
00:01:09.970 --> 00:01:13.922
There's a -- a totally just
invented Markov matrix.
00:01:16.640 --> 00:01:19.540
What makes it a Markov matrix?
00:01:19.540 --> 00:01:22.510
Two properties that
this -- this matrix has.
00:01:22.510 --> 00:01:24.970
So two properties are --
00:01:24.970 --> 00:01:30.430
one, every entry is
greater equal zero.
00:01:30.430 --> 00:01:38.720
All entries greater
than or equal to zero.
00:01:38.720 --> 00:01:43.100
And, of course, when
I square the matrix,
00:01:43.100 --> 00:01:45.660
the entries will still
be greater/equal zero.
00:01:45.660 --> 00:01:50.150
I'm going to be interested
in the powers of this matrix.
00:01:50.150 --> 00:01:54.250
And this property, of course,
is going to -- stay there.
00:01:54.250 --> 00:01:58.870
It -- really Markov matrices
you'll see are connected
00:01:58.870 --> 00:02:04.230
to probability ideas and
probabilities are never
00:02:04.230 --> 00:02:05.120
negative.
00:02:05.120 --> 00:02:09.669
The other property -- do you
see the other property in there?
00:02:09.669 --> 00:02:13.590
If I add down the columns,
what answer do I get?
00:02:13.590 --> 00:02:14.490
One.
00:02:14.490 --> 00:02:17.640
So all columns add to one.
00:02:17.640 --> 00:02:25.380
All columns add to one.
00:02:28.900 --> 00:02:31.580
And actually when I
square the matrix,
00:02:31.580 --> 00:02:33.820
that will be true again.
00:02:33.820 --> 00:02:38.920
So that the powers
of my matrix are all
00:02:38.920 --> 00:02:42.330
Markov matrices,
and I'm interested
00:02:42.330 --> 00:02:46.520
in, always, the eigenvalues
and the eigenvectors.
00:02:46.520 --> 00:02:50.780
And this question of
steady state will come up.
00:02:50.780 --> 00:02:53.860
You remember we had steady
state for differential equations
00:02:53.860 --> 00:02:54.940
last time?
00:02:54.940 --> 00:02:57.080
When -- what was
the steady state --
00:02:57.080 --> 00:02:59.100
what was the eigenvalue?
00:02:59.100 --> 00:03:03.240
What was the eigenvalue in
the differential equation case
00:03:03.240 --> 00:03:05.620
that led to a steady state?
00:03:05.620 --> 00:03:08.360
It was lambda equals zero.
00:03:08.360 --> 00:03:11.210
When -- you remember that
we did an example and one
00:03:11.210 --> 00:03:15.180
of the eigenvalues was lambda
equals zero, and that --
00:03:15.180 --> 00:03:20.140
so then we had an E to the
zero T, a constant one --
00:03:20.140 --> 00:03:24.430
as time went on, there
that thing stayed steady.
00:03:24.430 --> 00:03:30.760
Now what -- in the powers case,
it's not a zero eigenvalue.
00:03:30.760 --> 00:03:33.740
Actually with powers of a
matrix, a zero eigenvalue,
00:03:33.740 --> 00:03:36.570
that part is going
to die right away.
00:03:36.570 --> 00:03:41.710
It's an eigenvalue of
one that's all important.
00:03:41.710 --> 00:03:44.540
So this steady state
will correspond --
00:03:44.540 --> 00:03:49.000
will be totally connected with
an eigenvalue of one and its
00:03:49.000 --> 00:03:49.910
eigenvector.
00:03:49.910 --> 00:03:53.510
In fact, the steady state
will be the eigenvector
00:03:53.510 --> 00:03:55.260
for that eigenvalue.
00:03:55.260 --> 00:03:56.670
Okay.
00:03:56.670 --> 00:03:58.580
So that's what's coming.
00:03:58.580 --> 00:04:04.390
Now, for some reason
then that we have to see,
00:04:04.390 --> 00:04:08.210
this matrix has an
eigenvalue of one.
00:04:08.210 --> 00:04:12.040
This property, that the
columns all add to one --
00:04:12.040 --> 00:04:16.680
turns out -- guarantees
that one is an eigenvalue,
00:04:16.680 --> 00:04:19.600
so that you can actually
find the eigenvalue --
00:04:19.600 --> 00:04:24.330
find that eigenvalue of a Markov
matrix without computing any
00:04:24.330 --> 00:04:27.640
determinants of A
minus lambda I --
00:04:27.640 --> 00:04:30.310
that matrix will have
an eigenvalue of one,
00:04:30.310 --> 00:04:32.270
and we want to see why.
00:04:32.270 --> 00:04:35.850
And then the other thing is --
00:04:35.850 --> 00:04:39.190
so the key points -- let me --
let me write these underneath.
00:04:39.190 --> 00:04:45.837
The key points are -- the key
points are lambda equal one is
00:04:45.837 --> 00:04:46.420
an eigenvalue.
00:04:53.760 --> 00:04:56.410
I'll add in a little --
an additional -- well,
00:04:56.410 --> 00:04:59.030
a thing about eigenvalues --
00:04:59.030 --> 00:05:02.400
key point two, the other
eigenval- values --
00:05:02.400 --> 00:05:12.340
all other eigenvalues are, in
magnitude, smaller than one --
00:05:12.340 --> 00:05:14.630
in absolute value,
smaller than one.
00:05:14.630 --> 00:05:18.390
Well, there could be some
exceptional case when --
00:05:18.390 --> 00:05:22.380
when an eigen -- another
eigenvalue might have magnitude
00:05:22.380 --> 00:05:23.300
equal one.
00:05:23.300 --> 00:05:26.690
It never has an eigenvalue
larger than one.
00:05:26.690 --> 00:05:29.230
So these two facts --
somehow we ought to --
00:05:29.230 --> 00:05:32.600
linear algebra ought to tell us.
00:05:32.600 --> 00:05:35.440
And then, of course, linear
algebra is going to tell us
00:05:35.440 --> 00:05:40.040
what the -- what's -- what
happens if I take -- if --
00:05:40.040 --> 00:05:42.720
you remember when I solve --
00:05:42.720 --> 00:05:47.980
when I multiply by A time
after time the K-th thing is A
00:05:47.980 --> 00:05:54.940
to the K u0 and I'm asking
what's special about this --
00:05:54.940 --> 00:06:01.240
these powers of A, and very
likely the quiz will have
00:06:01.240 --> 00:06:06.440
a problem to computer s- to
computer some powers of A or --
00:06:06.440 --> 00:06:08.570
or applied to an initial vector.
00:06:08.570 --> 00:06:11.040
So, you remember
the general form?
00:06:11.040 --> 00:06:14.180
The general form is
that there's some amount
00:06:14.180 --> 00:06:17.660
of the first eigenvalue
to the K-th power
00:06:17.660 --> 00:06:22.790
times the first eigenvector,
and another amount
00:06:22.790 --> 00:06:25.340
of the second eigenvalue
to the K-th power
00:06:25.340 --> 00:06:27.430
times the second
eigenvector and so on.
00:06:27.430 --> 00:06:30.700
A -- just --
00:06:30.700 --> 00:06:34.380
my conscience always makes me
say at least once per lecture
00:06:34.380 --> 00:06:40.060
that this requires a
complete set of eigenvectors,
00:06:40.060 --> 00:06:43.080
otherwise we might not
be able to expand u0
00:06:43.080 --> 00:06:45.840
in the eigenvectors and
we couldn't get started.
00:06:45.840 --> 00:06:49.410
But once we're started
with u0 when K is zero,
00:06:49.410 --> 00:06:53.880
then every A brings
in these lambdas.
00:06:53.880 --> 00:06:56.830
And now you can see what the
steady state is going to be.
00:06:56.830 --> 00:07:01.630
If lambda one is one --
so lambda one equals one
00:07:01.630 --> 00:07:07.900
to the K-th power and these
other eigenvalues are smaller
00:07:07.900 --> 00:07:10.280
than one --
00:07:10.280 --> 00:07:14.560
so I've sort of scratched
over the equation there to --
00:07:14.560 --> 00:07:18.260
we had this term, but what
happens to this term --
00:07:18.260 --> 00:07:21.180
if the lambda's smaller than
one, then the -- when --
00:07:21.180 --> 00:07:25.160
as we take powers, as
we iterate as we --
00:07:25.160 --> 00:07:30.662
as we go forward in time, this
goes to zero, Can I just --
00:07:30.662 --> 00:07:32.370
having scratched over
it, I might as well
00:07:32.370 --> 00:07:34.050
scratch right? further.
00:07:34.050 --> 00:07:36.880
That term and all the other
terms are going to zero
00:07:36.880 --> 00:07:39.750
because all the other
eigenvalues are smaller than
00:07:39.750 --> 00:07:45.440
one and the steady state that
we're approaching is just --
00:07:45.440 --> 00:07:48.780
whatever there was -- this
was -- this was the --
00:07:48.780 --> 00:07:56.990
this is the x1 part of un- of
the initial condition u0 --
00:07:56.990 --> 00:07:59.150
is the steady state.
00:08:05.120 --> 00:08:08.350
This much we know from
general -- from -- you know,
00:08:08.350 --> 00:08:11.390
what we've already done.
00:08:11.390 --> 00:08:15.740
So I want to see why -- let's
at least see number one,
00:08:15.740 --> 00:08:18.160
why one is an eigenvalue.
00:08:18.160 --> 00:08:20.580
And then there's actually --
00:08:20.580 --> 00:08:23.370
in this chapter we're interested
not only in eigenvalues,
00:08:23.370 --> 00:08:25.390
but also eigenvectors.
00:08:25.390 --> 00:08:28.760
And there's something special
about the eigenvector.
00:08:28.760 --> 00:08:30.800
Let me write down what that is.
00:08:30.800 --> 00:08:36.700
The eigenvector x1 --
00:08:36.700 --> 00:08:41.909
x1 is the eigenvector and all
its components are positive,
00:08:41.909 --> 00:08:49.100
so the steady state is
positive, if the start was.
00:08:49.100 --> 00:08:51.480
If the start was -- so --
00:08:51.480 --> 00:08:54.780
well, actually, in general, I --
00:08:54.780 --> 00:09:00.710
this might have a -- might have
some component zero always,
00:09:00.710 --> 00:09:03.880
but no negative components
in that eigenvector.
00:09:03.880 --> 00:09:04.410
Okay.
00:09:04.410 --> 00:09:08.470
Can I come to that point?
00:09:08.470 --> 00:09:11.370
How can I look at that matrix --
00:09:11.370 --> 00:09:15.280
so that was just an example.
00:09:15.280 --> 00:09:21.000
How could I be sure -- how
can I see that a matrix --
00:09:21.000 --> 00:09:24.750
if the columns add to zero
-- add to one, sorry --
00:09:24.750 --> 00:09:30.930
if the columns add to one,
this property means that lambda
00:09:30.930 --> 00:09:32.780
equal one is an eigenvalue.
00:09:32.780 --> 00:09:33.540
Okay.
00:09:33.540 --> 00:09:37.080
So let's just
think that through.
00:09:37.080 --> 00:09:41.760
What I saying about -- let
me ca- let me look at A,
00:09:41.760 --> 00:09:44.940
and if I believe that
one is an eigenvalue,
00:09:44.940 --> 00:09:47.810
then I should be able to
subtract off one times
00:09:47.810 --> 00:09:56.150
the identity and then I would
get a matrix that's, what, -.9,
00:09:56.150 --> 00:09:59.160
-.01 and -.6 --
00:09:59.160 --> 00:10:02.700
wh- I took the ones away and
the other parts, of course,
00:10:02.700 --> 00:10:11.430
are still what they were, and
this is still .2 and .7 and --
00:10:11.430 --> 00:10:14.200
okay, what's --
00:10:14.200 --> 00:10:15.700
what's up with this matrix now?
00:10:15.700 --> 00:10:19.380
I've shifted the matrix,
this Markov matrix by one,
00:10:19.380 --> 00:10:24.350
by the identity, and
what do I want to prove?
00:10:24.350 --> 00:10:28.940
I -- what is it that I
believe this matrix --
00:10:28.940 --> 00:10:30.760
about this matrix?
00:10:30.760 --> 00:10:33.190
I believe it's singular.
00:10:33.190 --> 00:10:36.580
Singular will -- if A
minus I is singular,
00:10:36.580 --> 00:10:40.020
that tells me that one
is an eigenvalue, right?
00:10:40.020 --> 00:10:43.420
The eigenvalues are the
numbers that I subtract off --
00:10:43.420 --> 00:10:45.010
the shifts --
00:10:45.010 --> 00:10:47.880
the numbers that I subtract
from the diagonal --
00:10:47.880 --> 00:10:48.810
to make it singular.
00:10:48.810 --> 00:10:50.610
Now why is that matrix singular?
00:10:50.610 --> 00:10:55.570
I -- we could compute
its determinant,
00:10:55.570 --> 00:11:00.000
but we want to see a reason
that would work for every Markov
00:11:00.000 --> 00:11:05.550
matrix not just this
particular random example.
00:11:05.550 --> 00:11:07.780
So what is it about that matrix?
00:11:07.780 --> 00:11:11.040
Well, I guess you could
look at its columns now --
00:11:11.040 --> 00:11:12.890
what do they add up to?
00:11:12.890 --> 00:11:13.390
Zero.
00:11:13.390 --> 00:11:21.660
The columns add to
zero, so all columns --
00:11:21.660 --> 00:11:25.240
let me put all columns
now of -- of --
00:11:25.240 --> 00:11:32.450
of A minus I add
to zero, and then I
00:11:32.450 --> 00:11:40.305
want to realize that this
means A minus I is singular.
00:11:45.400 --> 00:11:47.790
Okay.
00:11:47.790 --> 00:11:49.560
Why?
00:11:49.560 --> 00:11:52.210
So I could I -- you know,
that could be a quiz question,
00:11:52.210 --> 00:11:54.280
a sort of theoretical
quiz question.
00:11:54.280 --> 00:11:56.070
If I give you a
matrix and I tell you
00:11:56.070 --> 00:12:00.740
all its columns add to
zero, give me a reason,
00:12:00.740 --> 00:12:06.270
because it is true, that
the matrix is singular.
00:12:06.270 --> 00:12:06.940
Okay.
00:12:06.940 --> 00:12:09.290
I guess actually -- now what --
00:12:09.290 --> 00:12:11.680
I think of -- you know, I'm
thinking of two or three ways
00:12:11.680 --> 00:12:12.480
to see that.
00:12:17.370 --> 00:12:18.410
How would you do it?
00:12:18.410 --> 00:12:21.880
We don't want to take
its determinant somehow.
00:12:21.880 --> 00:12:24.520
For the matrix to
be singular, well,
00:12:24.520 --> 00:12:31.180
it means that these three
columns are dependent, right?
00:12:31.180 --> 00:12:33.530
The determinant will be zero
when those three columns
00:12:33.530 --> 00:12:34.260
are dependent.
00:12:34.260 --> 00:12:36.650
You see, we're -- we're at
a point in this course, now,
00:12:36.650 --> 00:12:40.100
where we have several
ways to look at an idea.
00:12:40.100 --> 00:12:42.390
We can take the determinant
-- here we don't want to.
00:12:42.390 --> 00:12:45.810
B- but we met singular
before that --
00:12:45.810 --> 00:12:47.830
those columns are dependent.
00:12:47.830 --> 00:12:50.670
So how do I see that those
columns are dependent?
00:12:50.670 --> 00:12:51.925
They all add to zero.
00:12:57.020 --> 00:13:00.440
Let's see, whew --
00:13:00.440 --> 00:13:05.070
well, oh, actually, what --
00:13:05.070 --> 00:13:07.010
another thing I
know is that the --
00:13:07.010 --> 00:13:11.410
I would like to be able to show
is that the rows are dependent.
00:13:11.410 --> 00:13:14.250
Maybe that's easier.
00:13:14.250 --> 00:13:15.940
If I know that all
the columns add
00:13:15.940 --> 00:13:18.480
to zero, that's my
information, how
00:13:18.480 --> 00:13:25.120
do I see that those three
rows are linearly dependent?
00:13:25.120 --> 00:13:30.210
What -- what combination of
those rows gives the zero row?
00:13:30.210 --> 00:13:32.920
How -- how could I combine
those three rows --
00:13:32.920 --> 00:13:38.930
those three row vectors to
produce the zero row vector?
00:13:38.930 --> 00:13:42.360
And that would tell me
those rows are dependent,
00:13:42.360 --> 00:13:43.910
therefore the columns
are dependent,
00:13:43.910 --> 00:13:46.940
the matrix is singular, the
determinant is zero -- well,
00:13:46.940 --> 00:13:47.890
you see it.
00:13:47.890 --> 00:13:49.010
I just add the rows.
00:13:49.010 --> 00:13:52.610
One times that row plus one
times that row plus one times
00:13:52.610 --> 00:13:53.770
that row --
00:13:53.770 --> 00:13:56.990
it's the zero row.
00:13:56.990 --> 00:13:58.960
The rows are dependent.
00:13:58.960 --> 00:14:03.470
In a way, that one one one,
because it's multiplying
00:14:03.470 --> 00:14:07.810
the rows, is like an
eigenvector in the --
00:14:07.810 --> 00:14:10.700
it's in the left
null space, right?
00:14:10.700 --> 00:14:12.870
One one one is in
the left null space.
00:14:12.870 --> 00:14:23.190
It's singular because
the rows are dependent --
00:14:23.190 --> 00:14:25.380
and can I just keep
the reasoning going?
00:14:25.380 --> 00:14:32.790
Because this vector
one one one is --
00:14:32.790 --> 00:14:34.940
it's not in the null
space of the matrix,
00:14:34.940 --> 00:14:37.410
but it's in the null
space of the transpose --
00:14:37.410 --> 00:14:43.160
is in the null space
of the transpose.
00:14:43.160 --> 00:14:45.600
And that's good enough.
00:14:45.600 --> 00:14:48.890
If we have a square matrix
-- if we have a square matrix
00:14:48.890 --> 00:14:52.930
and the rows are dependent,
that matrix is singular.
00:14:52.930 --> 00:14:58.630
So it turned out that the
immediate guy we could identify
00:14:58.630 --> 00:15:00.470
was one one one.
00:15:00.470 --> 00:15:04.650
Of course, the --
00:15:04.650 --> 00:15:06.480
there will be somebody
in the null space,
00:15:06.480 --> 00:15:07.300
too.
00:15:07.300 --> 00:15:10.280
And actually, who will it be?
00:15:10.280 --> 00:15:13.110
So what's -- so --
00:15:13.110 --> 00:15:15.830
so now I want to ask
about the null space of --
00:15:15.830 --> 00:15:17.400
of the matrix itself.
00:15:17.400 --> 00:15:20.890
What combination of
the columns gives zero?
00:15:20.890 --> 00:15:23.450
I -- I don't want to compute
it because I just made up this
00:15:23.450 --> 00:15:24.830
matrix and --
00:15:24.830 --> 00:15:28.020
it will -- it would
take me a while --
00:15:28.020 --> 00:15:31.390
it looks sort of doable because
it's three by three but wh-
00:15:31.390 --> 00:15:34.720
my point is, what --
00:15:34.720 --> 00:15:37.350
what vector is it if we
-- once we've found it,
00:15:37.350 --> 00:15:41.360
what have we got that's in
the -- in the null space of A?
00:15:41.360 --> 00:15:44.060
It's the eigenvector, right?
00:15:44.060 --> 00:15:46.060
That's where we find X one.
00:15:46.060 --> 00:15:54.350
Then X one, the eigenvector,
is in the null space of A.
00:15:54.350 --> 00:15:57.690
That's the eigenvector
corresponding to the eigenvalue
00:15:57.690 --> 00:15:58.470
one.
00:15:58.470 --> 00:15:58.980
Right?
00:15:58.980 --> 00:16:01.330
That's how we find eigenvectors.
00:16:01.330 --> 00:16:05.280
So those three columns
must be dependent --
00:16:05.280 --> 00:16:08.090
some combination of columns
-- of those three columns is
00:16:08.090 --> 00:16:11.930
the zero column and that --
the three components in that
00:16:11.930 --> 00:16:14.020
combination are the eigenvector.
00:16:14.020 --> 00:16:17.250
And that guy is
the steady state.
00:16:17.250 --> 00:16:18.280
Okay.
00:16:18.280 --> 00:16:21.110
So I'm happy about the --
00:16:21.110 --> 00:16:26.010
the thinking here,
but I haven't given --
00:16:26.010 --> 00:16:29.620
I haven't completed it
because I haven't found x1.
00:16:29.620 --> 00:16:32.560
But it's there.
00:16:32.560 --> 00:16:36.330
Can I -- another thought came
to me as I was doing this,
00:16:36.330 --> 00:16:39.280
another little
comment that -- you --
00:16:39.280 --> 00:16:41.500
about eigenvalues
and eigenvectors,
00:16:41.500 --> 00:16:45.330
because of A and A transpose.
00:16:45.330 --> 00:16:50.800
What can you tell me
about eigenvalues of A --
00:16:50.800 --> 00:16:56.275
of A and eigenvalues
of A transpose?
00:17:01.160 --> 00:17:01.660
Whoops.
00:17:04.952 --> 00:17:05.660
They're the same.
00:17:05.660 --> 00:17:11.079
They're -- so this is a little
comment -- we -- it's useful,
00:17:11.079 --> 00:17:14.550
since eigenvalues are
generally not easy to find --
00:17:14.550 --> 00:17:19.180
it's always useful to know some
cases where you've got them,
00:17:19.180 --> 00:17:20.210
where --
00:17:20.210 --> 00:17:23.089
and this is -- if you
know the eigenvalues of A,
00:17:23.089 --> 00:17:25.260
then you know the
eigenvalues of A transpose.
00:17:25.260 --> 00:17:27.500
eigenvalues of A
transpose are the same.
00:17:32.560 --> 00:17:37.890
And can I just, like,
review why that is?
00:17:37.890 --> 00:17:44.280
So to find the eigenvalues of A,
this would be determinate of A
00:17:44.280 --> 00:17:53.490
minus lambda I equals zero, that
gives me an eigenvalue of A --
00:17:53.490 --> 00:17:58.520
now how can I get A transpose
into the picture here?
00:17:58.520 --> 00:18:00.690
I'll use the fact
that the determinant
00:18:00.690 --> 00:18:06.690
of a matrix and the determinant
of its transpose are the same.
00:18:06.690 --> 00:18:09.550
The determinant of a matrix
equals the determinant of a --
00:18:09.550 --> 00:18:10.520
of the transpose.
00:18:10.520 --> 00:18:13.950
That was property
ten, the very last guy
00:18:13.950 --> 00:18:15.860
in our determinant list.
00:18:15.860 --> 00:18:18.280
So I'll transpose that matrix.
00:18:18.280 --> 00:18:21.680
This leads to --
00:18:21.680 --> 00:18:24.690
I just take the matrix
and transpose it,
00:18:24.690 --> 00:18:29.520
but now what do I get
when I transpose lambda I?
00:18:29.520 --> 00:18:33.920
I just get lambda I.
00:18:33.920 --> 00:18:37.800
So that's -- that's all
there was to the reasoning.
00:18:37.800 --> 00:18:40.000
The reasoning is that
the eigenvalues of A
00:18:40.000 --> 00:18:42.370
solved that equation.
00:18:42.370 --> 00:18:44.530
The determinant of a
matrix is the determinant
00:18:44.530 --> 00:18:47.480
of its transpose, so that
gives me this equation
00:18:47.480 --> 00:18:50.160
and that tells me
that the same lambdas
00:18:50.160 --> 00:18:53.240
are eigenvalues of A transpose.
00:18:53.240 --> 00:18:56.660
So that, backing up
to the Markov case,
00:18:56.660 --> 00:19:00.880
one is an eigenvalue of A
transpose and we actually found
00:19:00.880 --> 00:19:05.480
its eigenvector, one one one,
and that tell us that one is
00:19:05.480 --> 00:19:08.090
also an eigenvalue of
A -- but, of course,
00:19:08.090 --> 00:19:10.360
it has a different
eigenvector, the --
00:19:10.360 --> 00:19:13.530
the left null space isn't the
same as the null space and we
00:19:13.530 --> 00:19:14.750
would have to find it.
00:19:14.750 --> 00:19:20.960
So there's some vector here
which is x1 that produces zero
00:19:20.960 --> 00:19:22.460
zero zero.
00:19:22.460 --> 00:19:24.900
Actually, it wouldn't be that
hard to find, you know, I --
00:19:24.900 --> 00:19:26.520
as I'm talking I'm
thinking, okay,
00:19:26.520 --> 00:19:29.440
I going to follow through
and actually find it?
00:19:29.440 --> 00:19:32.750
Well, I can tell from
this one -- look,
00:19:32.750 --> 00:19:37.650
if I put a point six there
and a point seven there,
00:19:37.650 --> 00:19:43.510
that's what -- then I'll be
okay in the last row, right?
00:19:43.510 --> 00:19:47.840
Now I only -- remains
to find one guy.
00:19:47.840 --> 00:19:49.860
And let me take the
first row, then.
00:19:49.860 --> 00:19:53.500
Minus point 54 plus point 21 --
00:19:53.500 --> 00:19:56.790
there's some big number
going in there, right?
00:19:56.790 --> 00:20:00.320
So I have -- just to make
the first row come out zero,
00:20:00.320 --> 00:20:03.980
I'm getting minus
point 54 plus point 21,
00:20:03.980 --> 00:20:09.420
so that was minus point 33
and what -- what do I want?
00:20:09.420 --> 00:20:11.350
Like thirty three hundred?
00:20:11.350 --> 00:20:13.980
This is the first time in
the history of linear algebra
00:20:13.980 --> 00:20:16.980
that an eigenvector has
every had a component
00:20:16.980 --> 00:20:18.980
thirty three hundred.
00:20:18.980 --> 00:20:21.030
But I guess it's true.
00:20:21.030 --> 00:20:24.760
Because then I multiply by minus
one over a hundred -- oh no,
00:20:24.760 --> 00:20:26.830
it was point 33.
00:20:26.830 --> 00:20:29.180
So is this just -- oh, shoot.
00:20:29.180 --> 00:20:30.800
Only 33.
00:20:30.800 --> 00:20:31.880
Okay.
00:20:31.880 --> 00:20:33.030
Only 33.
00:20:33.030 --> 00:20:35.610
Okay, so there's
the eigenvector.
00:20:35.610 --> 00:20:39.150
Oh, and notice that it -- that
it turned -- did turn out,
00:20:39.150 --> 00:20:42.940
at least, to be all positive.
00:20:42.940 --> 00:20:45.990
So that was, like, the theory
-- predicts that part, too.
00:20:45.990 --> 00:20:48.500
I won't give the
proof of that part.
00:20:48.500 --> 00:20:50.600
So 30 -- 33 --
00:20:50.600 --> 00:20:52.770
point six 33 point seven.
00:20:52.770 --> 00:20:53.420
Okay.
00:20:53.420 --> 00:20:58.750
Now those are the ma- that's
the linear algebra part.
00:20:58.750 --> 00:21:00.980
Can I get to the applications?
00:21:00.980 --> 00:21:03.120
Where do these Markov
matrices come from?
00:21:03.120 --> 00:21:06.360
Because that's -- that's part of
this course and absolutely part
00:21:06.360 --> 00:21:07.570
of this lecture.
00:21:07.570 --> 00:21:08.080
Okay.
00:21:08.080 --> 00:21:12.270
So where's -- what's an
application of Markov matrices?
00:21:12.270 --> 00:21:12.770
Okay.
00:21:17.600 --> 00:21:21.180
Markov matrices --
so, my equation, then,
00:21:21.180 --> 00:21:24.842
that I'm solving and studying
is this equation u(k+1)=Auk.
00:21:28.290 --> 00:21:31.280
And now A is a Markov matrix.
00:21:31.280 --> 00:21:31.930
A is Markov.
00:21:36.060 --> 00:21:39.110
And I want to give an example.
00:21:39.110 --> 00:21:41.190
Can I just create an example?
00:21:41.190 --> 00:21:44.120
It'll be two by two.
00:21:44.120 --> 00:21:48.870
And it's one I've used
before because it seems
00:21:48.870 --> 00:21:50.990
to me to bring out the idea.
00:21:50.990 --> 00:21:55.740
It's -- because we have two
by two, we have two states,
00:21:55.740 --> 00:22:01.510
let's say California
and Massachusetts.
00:22:01.510 --> 00:22:05.310
And I'm looking at the
populations in those two
00:22:05.310 --> 00:22:08.720
states, the people in those
two states, California
00:22:08.720 --> 00:22:10.900
and Massachusetts.
00:22:10.900 --> 00:22:17.040
And my matrix A is going to
tell me in a -- in a year,
00:22:17.040 --> 00:22:19.230
some movement has happened.
00:22:19.230 --> 00:22:21.180
Some people stayed
in Massachusetts,
00:22:21.180 --> 00:22:24.050
some people moved to
California, some smart people
00:22:24.050 --> 00:22:26.300
moved from California
to Massachusetts,
00:22:26.300 --> 00:22:29.240
some people stayed in
California and made a billion.
00:22:29.240 --> 00:22:29.740
Okay.
00:22:29.740 --> 00:22:36.150
So that -- there's a matrix
there with four entries
00:22:36.150 --> 00:22:41.044
and those tell me the
fractions of my population --
00:22:41.044 --> 00:22:41.710
so I'm making --
00:22:41.710 --> 00:22:45.140
I'm going to use fractions,
so they won't be negative,
00:22:45.140 --> 00:22:48.500
of course, because -- because
only positive people are
00:22:48.500 --> 00:22:51.550
in- involved here -- and
they'll add up to one,
00:22:51.550 --> 00:22:54.620
because I'm accounting
for all people.
00:22:54.620 --> 00:22:57.830
So that's why I have
these two key properties.
00:22:57.830 --> 00:22:59.650
The entries are
greater equal zero
00:22:59.650 --> 00:23:02.890
because I'm looking
at probabilities.
00:23:02.890 --> 00:23:06.050
Do they move, do they stay?
00:23:06.050 --> 00:23:09.130
Those probabilities are
all between zero and one.
00:23:09.130 --> 00:23:12.290
And the probabilities add
to one because everybody's
00:23:12.290 --> 00:23:12.990
accounted for.
00:23:12.990 --> 00:23:17.970
I'm not losing anybody, gaining
anybody in this Markov chain.
00:23:17.970 --> 00:23:22.490
It's -- it conserves
the total population.
00:23:22.490 --> 00:23:22.990
Okay.
00:23:22.990 --> 00:23:25.450
So what would be a
typical matrix, then?
00:23:25.450 --> 00:23:34.820
So this would be u, California
and u Massachusetts at time t
00:23:34.820 --> 00:23:36.740
equal k+1.
00:23:36.740 --> 00:23:40.680
And it's some
matrix, which we'll
00:23:40.680 --> 00:23:48.730
think of, times u California
and u Massachusetts at time k.
00:23:51.280 --> 00:23:54.460
And notice this matrix is
going to stay the same,
00:23:54.460 --> 00:23:57.480
you know, forever.
00:23:57.480 --> 00:24:01.940
So that's a severe
limitation on the example.
00:24:01.940 --> 00:24:05.090
The example has a --
the same Markov matrix,
00:24:05.090 --> 00:24:08.510
the same probabilities
act at every time.
00:24:08.510 --> 00:24:09.010
Okay.
00:24:09.010 --> 00:24:11.030
So what's a reasonable, say --
00:24:11.030 --> 00:24:16.750
say point nine of the people
in California at time k
00:24:16.750 --> 00:24:19.410
stay there.
00:24:19.410 --> 00:24:24.440
And point one of the
people in California
00:24:24.440 --> 00:24:26.930
move to Massachusetts.
00:24:26.930 --> 00:24:28.900
Notice why that
column added to one,
00:24:28.900 --> 00:24:32.170
because we've now accounted for
all the people in California
00:24:32.170 --> 00:24:33.890
at time k.
00:24:33.890 --> 00:24:36.250
Nine tenths of them are
still in California,
00:24:36.250 --> 00:24:41.220
one tenth are here at time k+1.
00:24:41.220 --> 00:24:42.130
Okay.
00:24:42.130 --> 00:24:45.120
What about the people
who are in Massachusetts?
00:24:45.120 --> 00:24:47.530
This is going to multiply
column two, right,
00:24:47.530 --> 00:24:52.370
by our fundamental rule of
multiplying matrix by vector,
00:24:52.370 --> 00:24:57.050
it's the -- it's the
population in Massachusetts.
00:24:57.050 --> 00:25:06.350
Shall we say that -- that
after the Red Sox, fail again,
00:25:06.350 --> 00:25:11.630
eight -- only 80 percent of the
people in Massachusetts stay
00:25:11.630 --> 00:25:15.120
and 20 percent
move to California.
00:25:15.120 --> 00:25:15.870
Okay.
00:25:15.870 --> 00:25:18.960
So again, this
adds to one, which
00:25:18.960 --> 00:25:23.070
accounts for all people in
Massachusetts where they are.
00:25:23.070 --> 00:25:26.770
So there is a Markov matrix.
00:25:26.770 --> 00:25:28.400
Non-negative entries
adding to one.
00:25:28.400 --> 00:25:29.980
What's the steady state?
00:25:29.980 --> 00:25:33.314
If everybody started in
Massachusetts, say, at --
00:25:33.314 --> 00:25:34.980
you know, when the
Pilgrims showed up or
00:25:34.980 --> 00:25:35.600
something.
00:25:35.600 --> 00:25:40.280
Then where are they now?
00:25:40.280 --> 00:25:45.211
Where are they at time
100, let's say, or maybe --
00:25:45.211 --> 00:25:47.210
I don't know, how many
years since the Pilgrims?
00:25:47.210 --> 00:25:49.090
300 and something.
00:25:49.090 --> 00:25:51.520
Or -- and actually where
will they be, like,
00:25:51.520 --> 00:25:54.820
way out a million
years from now?
00:25:54.820 --> 00:26:00.980
I -- I could multiply --
00:26:00.980 --> 00:26:03.360
take the powers of this matrix.
00:26:03.360 --> 00:26:07.040
In fact, you'll -- you would --
ought to be able to figure out
00:26:07.040 --> 00:26:10.490
what is the hundredth
power of that matrix?
00:26:10.490 --> 00:26:12.520
Why don't we do that?
00:26:12.520 --> 00:26:15.050
But let me follow
the steady state.
00:26:15.050 --> 00:26:17.760
So what -- what's my starting --
00:26:17.760 --> 00:26:28.560
my starting u Cal, u Mass at
time zero is, shall we say --
00:26:28.560 --> 00:26:30.400
shall we put anybody
in California?
00:26:30.400 --> 00:26:32.920
Let's make -- let's
make zero there,
00:26:32.920 --> 00:26:36.400
and say the population
of Massachusetts is --
00:26:36.400 --> 00:26:38.780
let's say a thousand
just to -- okay.
00:26:43.330 --> 00:26:46.030
So the population is --
00:26:46.030 --> 00:26:49.400
so the populations are zero
and a thousand at the start.
00:26:49.400 --> 00:26:52.790
What can you tell me about
this population after --
00:26:52.790 --> 00:26:55.990
after k steps?
00:26:55.990 --> 00:27:01.420
What will u Cal
plus u Mass add to?
00:27:01.420 --> 00:27:02.790
A thousand.
00:27:02.790 --> 00:27:06.260
Those thousand people
are always accounted for.
00:27:06.260 --> 00:27:10.590
But -- so u Mass will start
dropping from a thousand and u
00:27:10.590 --> 00:27:11.930
Cal will start growing.
00:27:11.930 --> 00:27:15.060
Actually, we could see -- why
don't we figure out what it is
00:27:15.060 --> 00:27:16.420
after one?
00:27:16.420 --> 00:27:22.100
After one time step, what are
the populations at time one?
00:27:24.890 --> 00:27:26.530
So what happens in one step?
00:27:26.530 --> 00:27:30.230
You multiply once by that
matrix and, let's see,
00:27:30.230 --> 00:27:33.460
zero times this column -- so
it's just a thousand times this
00:27:33.460 --> 00:27:39.940
column, so I think we're
getting 200 and 800.
00:27:39.940 --> 00:27:42.860
So after the first
step, 200 people have --
00:27:42.860 --> 00:27:44.760
are in California.
00:27:44.760 --> 00:27:49.620
Now at the following step, I'll
multiply again by this matrix
00:27:49.620 --> 00:27:52.740
-- more people will
move to California.
00:27:52.740 --> 00:27:54.340
Some people will move back.
00:27:54.340 --> 00:28:00.390
Twenty people will
come back and, the --
00:28:00.390 --> 00:28:03.010
the net result will be that the
California population will be
00:28:03.010 --> 00:28:08.310
above 200 and the Massachusetts
below 800 and they'll still add
00:28:08.310 --> 00:28:10.191
up to a thousand.
00:28:10.191 --> 00:28:10.690
Okay.
00:28:10.690 --> 00:28:14.100
I do that a few times.
00:28:14.100 --> 00:28:15.500
I do that 100 times.
00:28:15.500 --> 00:28:18.220
What's the population?
00:28:18.220 --> 00:28:20.770
Well, okay, to answer
any question like that,
00:28:20.770 --> 00:28:22.700
I need the eigenvalues
and eigenvectors,
00:28:22.700 --> 00:28:23.200
right?
00:28:23.200 --> 00:28:24.090
As soon as I've --
00:28:24.090 --> 00:28:26.340
I've created an
example, but as soon
00:28:26.340 --> 00:28:28.570
as I want to solve
anything, I have
00:28:28.570 --> 00:28:31.610
to find eigenvalues and
eigenvectors of that matrix.
00:28:31.610 --> 00:28:32.230
Okay.
00:28:32.230 --> 00:28:33.890
So let's do it.
00:28:33.890 --> 00:28:40.010
So there's the matrix
.9, .2, .1, .8 and tell
00:28:40.010 --> 00:28:43.150
me its eigenvalues.
00:28:43.150 --> 00:28:46.390
Lambda equals -- so
tell me one eigenvalue?
00:28:46.390 --> 00:28:48.680
One, thanks.
00:28:48.680 --> 00:28:49.960
And tell me the other one.
00:28:53.120 --> 00:28:55.300
What's the other eigenvalue --
00:28:55.300 --> 00:28:59.370
from the trace or the
determinant -- from the --
00:28:59.370 --> 00:29:02.510
I -- the trace is what
-- is, like, easier.
00:29:02.510 --> 00:29:06.110
So the trace of that
matrix is one point seven.
00:29:06.110 --> 00:29:10.040
So the other eigenvalue
is point seven.
00:29:10.040 --> 00:29:13.540
And it -- notice that
it's less than one.
00:29:13.540 --> 00:29:17.780
And notice that that
determinant is point 72-.02,
00:29:17.780 --> 00:29:18.960
which is point seven.
00:29:18.960 --> 00:29:19.490
Right.
00:29:19.490 --> 00:29:20.190
Okay.
00:29:20.190 --> 00:29:21.900
Now to find the eigenvectors.
00:29:21.900 --> 00:29:26.120
This is -- so that's lambda
one and the eigenvector --
00:29:26.120 --> 00:29:29.950
I'll subtract one from
the diagonal, right?
00:29:29.950 --> 00:29:33.440
So can I do that in light
let -- in light here?
00:29:33.440 --> 00:29:35.300
Subtract one from
the diagonal, I
00:29:35.300 --> 00:29:38.410
have minus point one
and minus point two,
00:29:38.410 --> 00:29:39.730
and of course these are still
00:29:39.730 --> 00:29:40.940
there.
00:29:40.940 --> 00:29:44.220
And I'm looking for its --
00:29:44.220 --> 00:29:47.360
here's -- here's --
this is going to be x1.
00:29:47.360 --> 00:29:52.170
It's the null
space of A minus I.
00:29:52.170 --> 00:29:56.820
Okay, everybody sees
that it's two and one.
00:29:56.820 --> 00:29:57.940
Okay?
00:29:57.940 --> 00:29:59.820
And now how about --
so that -- and it --
00:29:59.820 --> 00:30:03.160
notice that that
eigenvector is positive.
00:30:03.160 --> 00:30:08.770
And actually, we can jump
to infinity right now.
00:30:08.770 --> 00:30:14.100
What's the population
at infinity?
00:30:14.100 --> 00:30:17.170
It's a multiple -- this is
-- this eigenvector is giving
00:30:17.170 --> 00:30:19.910
the steady state.
00:30:19.910 --> 00:30:23.280
It's some multiple of this, and
how is that multiple decided?
00:30:23.280 --> 00:30:26.940
By adding up to a
thousand people.
00:30:26.940 --> 00:30:31.200
So the steady state, the
c1x1 -- this is the x1,
00:30:31.200 --> 00:30:36.740
but that adds up to three,
so I really want two --
00:30:36.740 --> 00:30:39.820
it's going to be two thirds
of a thousand and one third
00:30:39.820 --> 00:30:42.980
of a thousand, making
a total of the thousand
00:30:42.980 --> 00:30:43.550
people.
00:30:43.550 --> 00:30:45.480
That'll be the steady state.
00:30:45.480 --> 00:30:48.390
That's really all I need
to know at infinity.
00:30:48.390 --> 00:30:50.160
But if I want to
know what's happened
00:30:50.160 --> 00:30:53.280
after just a finite
number like 100 steps,
00:30:53.280 --> 00:30:55.380
I'd better find
this eigenvector.
00:30:55.380 --> 00:30:57.840
So can I -- can I look at --
00:30:57.840 --> 00:31:00.130
I'll subtract point
seven time -- ti-
00:31:00.130 --> 00:31:06.250
from the diagonal and I'll get
that and I'll look at the null
00:31:06.250 --> 00:31:11.280
space of that one and I -- and
this is going to give me x2,
00:31:11.280 --> 00:31:13.540
now, and what is it?
00:31:13.540 --> 00:31:16.380
So what's in the null space of
-- that's certainly singular,
00:31:16.380 --> 00:31:20.920
so I know my calculation
is right, and --
00:31:20.920 --> 00:31:24.290
one and minus one.
00:31:24.290 --> 00:31:26.240
One and minus one.
00:31:26.240 --> 00:31:30.580
So I'm prepared
now to write down
00:31:30.580 --> 00:31:32.470
the solution after 100 time
00:31:32.470 --> 00:31:33.180
steps.
00:31:33.180 --> 00:31:35.670
The -- the populations
after 100 time steps,
00:31:35.670 --> 00:31:36.420
right?
00:31:36.420 --> 00:31:39.290
Can -- can we remember
the point one -- the --
00:31:39.290 --> 00:31:43.100
the one with this two one
eigenvector and the point seven
00:31:43.100 --> 00:31:44.930
with the minus one
one eigenvector.
00:31:44.930 --> 00:31:46.160
So I'll -- let me --
00:31:46.160 --> 00:31:48.960
I'll just write it above here.
00:31:48.960 --> 00:31:53.810
u after k steps is
some multiple of one
00:31:53.810 --> 00:31:57.610
to the k times the
two one eigenvector
00:31:57.610 --> 00:32:04.230
and some multiple of point seven
to the k times the minus one
00:32:04.230 --> 00:32:07.430
one eigenvector.
00:32:07.430 --> 00:32:09.910
Right?
00:32:09.910 --> 00:32:11.290
That's -- I --
00:32:11.290 --> 00:32:15.010
this is how I take -- how
powers of a matrix work.
00:32:15.010 --> 00:32:21.360
When I apply those powers to
a u0, what I -- so it's u0,
00:32:21.360 --> 00:32:26.390
which was zero a thousand --
00:32:26.390 --> 00:32:29.220
that has to be corrected k=0.
00:32:29.220 --> 00:32:35.720
So I'm plugging in k=0 and I get
c1 times two one and c2 times
00:32:35.720 --> 00:32:38.640
minus one one.
00:32:38.640 --> 00:32:44.990
Two equations, two
constants, certainly
00:32:44.990 --> 00:32:49.710
independent eigenvectors,
so there's a solution
00:32:49.710 --> 00:32:51.770
and you see what it is?
00:32:51.770 --> 00:32:56.450
Let's see, I guess we already
figured that c1 was a thousand
00:32:56.450 --> 00:32:59.390
over three, I think -- did we
think that had to be a thousand
00:32:59.390 --> 00:33:02.200
over three?
00:33:02.200 --> 00:33:05.840
And maybe c2 would be --
00:33:05.840 --> 00:33:07.920
excuse me, let --
get an eraser --
00:33:07.920 --> 00:33:08.930
we can --
00:33:08.930 --> 00:33:11.330
I just -- I think
we've -- get it here.
00:33:11.330 --> 00:33:15.590
c2, we want to get a
zero here, so maybe we
00:33:15.590 --> 00:33:21.400
need plus two
thousand over three.
00:33:21.400 --> 00:33:22.840
I think that has to work.
00:33:22.840 --> 00:33:26.010
Two times a thousand over
three minus two thousand
00:33:26.010 --> 00:33:29.160
over three, that'll give
us the zero and a thousand
00:33:29.160 --> 00:33:32.020
over three and the two thousand
over three will give us
00:33:32.020 --> 00:33:33.290
three thousand over three,
00:33:33.290 --> 00:33:34.160
the thousand.
00:33:34.160 --> 00:33:37.820
So this is what we approach --
00:33:37.820 --> 00:33:42.000
this part, with the point
seven to the k-th power
00:33:42.000 --> 00:33:45.050
is the part that's disappearing.
00:33:45.050 --> 00:33:48.270
That's -- that's
Markov matrices.
00:33:48.270 --> 00:33:48.770
Okay.
00:33:48.770 --> 00:33:52.610
That's an example of
where they come from,
00:33:52.610 --> 00:34:00.220
from modeling movement of
people with no gain or loss,
00:34:00.220 --> 00:34:03.690
with total -- total
count conserved.
00:34:03.690 --> 00:34:04.440
Okay.
00:34:04.440 --> 00:34:07.260
I -- just if I can
add one more comment,
00:34:07.260 --> 00:34:11.550
because you'll see Markov
matrices in electrical
00:34:11.550 --> 00:34:16.949
engineering courses and often
you'll see them -- sorry,
00:34:16.949 --> 00:34:19.679
here's my little comment.
00:34:19.679 --> 00:34:22.280
Sometimes -- in a lot of
applications they prefer
00:34:22.280 --> 00:34:25.590
to work with row vectors.
00:34:25.590 --> 00:34:29.480
So they -- instead of -- this
was natural for us, right?
00:34:29.480 --> 00:34:32.429
For all the eigenvectors
to be column vectors.
00:34:32.429 --> 00:34:37.139
So our columns added to
one in the Markov matrix.
00:34:37.139 --> 00:34:39.889
Just so you don't
think, well, what --
00:34:39.889 --> 00:34:41.820
what's going on?
00:34:41.820 --> 00:34:48.170
If we work with row vectors and
we multiply vector times matrix
00:34:48.170 --> 00:34:51.179
-- so we're multiplying
from the left --
00:34:51.179 --> 00:34:55.679
then it'll be the then we'll
be using the transpose of --
00:34:55.679 --> 00:35:00.100
of this matrix and it'll be
the rows that add to one.
00:35:00.100 --> 00:35:05.390
So in other textbooks,
you'll see -- instead of col-
00:35:05.390 --> 00:35:08.200
columns adding to one,
you'll see rows add to one.
00:35:08.200 --> 00:35:09.320
Okay.
00:35:09.320 --> 00:35:10.410
Fine.
00:35:10.410 --> 00:35:13.030
Okay, that's what I wanted
to say about Markov,
00:35:13.030 --> 00:35:17.560
now I want to say something
about projections and even
00:35:17.560 --> 00:35:22.190
leading in -- a little
into Fourier series.
00:35:22.190 --> 00:35:24.960
Because -- but before
any Fourier stuff,
00:35:24.960 --> 00:35:28.150
let me make a comment
about projections.
00:35:28.150 --> 00:35:33.310
This -- so this is a comment
about projections onto --
00:35:33.310 --> 00:35:36.526
with an orthonormal basis.
00:35:42.420 --> 00:35:49.770
So, of course, the basis
vectors are q1 up to qn.
00:35:49.770 --> 00:35:50.860
Okay.
00:35:50.860 --> 00:35:52.330
I have a vector b.
00:35:52.330 --> 00:35:58.700
Let -- let me imagine -- let
me imagine this is a basis.
00:35:58.700 --> 00:36:00.640
Let -- let's say I'm in n by n.
00:36:00.640 --> 00:36:06.660
I'm -- I've got, eh,
n orthonormal vectors,
00:36:06.660 --> 00:36:09.350
I'm in n dimensional space
so they're a complete --
00:36:09.350 --> 00:36:11.110
they're a basis --
00:36:11.110 --> 00:36:16.170
any vector v could be
expanded in this basis.
00:36:16.170 --> 00:36:22.050
So any vector v is some
combination, some amount of q1
00:36:22.050 --> 00:36:28.060
plus some amount of q2
plus some amount of qn.
00:36:28.060 --> 00:36:35.220
So -- so any v.
00:36:35.220 --> 00:36:41.740
I just want you to tell
me what those amounts are.
00:36:41.740 --> 00:36:46.510
What are x1 -- what's
x1, for example?
00:36:46.510 --> 00:36:49.980
So I'm looking
for the expansion.
00:36:49.980 --> 00:36:51.900
This is -- this is
really our projection.
00:36:51.900 --> 00:36:56.330
I could -- I could really
use the word expansion.
00:36:56.330 --> 00:37:01.450
I'm expanding the
vector in the basis.
00:37:01.450 --> 00:37:05.770
And the special thing about the
basis is that it's orthonormal.
00:37:05.770 --> 00:37:10.470
So that should give me a
special formula for the answer,
00:37:10.470 --> 00:37:12.220
for the coefficients.
00:37:12.220 --> 00:37:14.010
So how do I get x1?
00:37:14.010 --> 00:37:15.510
What -- what's a formula for x1?
00:37:18.250 --> 00:37:23.040
I could -- I can go
through the projection --
00:37:23.040 --> 00:37:26.940
the Q transpose Q, all that --
00:37:26.940 --> 00:37:31.450
normal equations, but --
00:37:31.450 --> 00:37:32.140
and I'll get --
00:37:32.140 --> 00:37:33.960
I'll come out with
this nice answer
00:37:33.960 --> 00:37:36.060
that I think I can
see right away.
00:37:36.060 --> 00:37:38.370
How can I pick --
00:37:38.370 --> 00:37:42.920
get a hold of x1 and get these
other x-s out of the equation?
00:37:42.920 --> 00:37:47.460
So how can I get a nice,
simple formula for x1?
00:37:47.460 --> 00:37:50.751
And then we want to see, sure,
we knew that all the time.
00:37:50.751 --> 00:37:51.250
Okay.
00:37:51.250 --> 00:37:52.610
So what's x1?
00:37:52.610 --> 00:37:59.020
The good way is take the inner
product of everything with q1.
00:37:59.020 --> 00:38:02.430
Take the inner product of that
whole equation, every term,
00:38:02.430 --> 00:38:03.850
with q1.
00:38:03.850 --> 00:38:08.370
What will happen
to that last term?
00:38:08.370 --> 00:38:11.560
The inner product -- when -- if
I take the dot product with q1
00:38:11.560 --> 00:38:13.970
I get zero, right?
00:38:13.970 --> 00:38:17.160
Because this basis
was orthonormal.
00:38:17.160 --> 00:38:20.330
If I take the dot product
with q2 I get zero.
00:38:20.330 --> 00:38:24.570
If I take the dot product
with q1 I get one.
00:38:24.570 --> 00:38:28.740
So that tells me what
x1 is. q1 transpose
00:38:28.740 --> 00:38:31.250
v, that's taking
the dot product,
00:38:31.250 --> 00:38:41.060
is x1 times q1 transpose
q1 plus a bunch of zeroes.
00:38:41.060 --> 00:38:45.330
And this is a one,
so I can forget that.
00:38:45.330 --> 00:38:47.820
I get x1 immediately.
00:38:47.820 --> 00:38:49.840
So -- do you see
what I'm saying --
00:38:49.840 --> 00:38:53.070
is that I have an
orthonormal basis,
00:38:53.070 --> 00:38:58.760
then the coefficient that I
need for each basis vector is
00:38:58.760 --> 00:38:59.340
a cinch to
00:38:59.340 --> 00:39:00.070
find.
00:39:00.070 --> 00:39:02.310
Let me -- let me just --
00:39:02.310 --> 00:39:05.580
I have to put this into
matrix language, too,
00:39:05.580 --> 00:39:07.350
so you'll see it there also.
00:39:07.350 --> 00:39:10.690
If I write that first equation
in matrix language, what --
00:39:10.690 --> 00:39:12.010
what is it?
00:39:12.010 --> 00:39:13.870
I'm writing -- in
matrix language,
00:39:13.870 --> 00:39:19.020
this equation says I'm taking
these columns -- are --
00:39:19.020 --> 00:39:20.620
are you guys good at this now?
00:39:20.620 --> 00:39:29.180
I'm taking those columns times
the Xs and getting V, right?
00:39:29.180 --> 00:39:30.440
That's the matrix form.
00:39:30.440 --> 00:39:37.464
Okay, that's the
matrix Q. Qx is v.
00:39:37.464 --> 00:39:39.005
What's the solution
to that equation?
00:39:41.570 --> 00:39:44.920
It's -- of course, it's
x equal Q inverse v.
00:39:44.920 --> 00:39:51.000
So x is Q inverse v,
but what's the point?
00:39:51.000 --> 00:39:54.060
Q inverse in this case
is going to -- is simple.
00:39:54.060 --> 00:39:58.220
I don't have to work to
invert this matrix Q,
00:39:58.220 --> 00:40:03.110
because of the fact that the --
these columns are orthonormal,
00:40:03.110 --> 00:40:05.110
I know the inverse to that.
00:40:05.110 --> 00:40:10.070
And it is Q transpose.
00:40:10.070 --> 00:40:15.050
When you see a Q, a square
matrix with that letter Q,
00:40:15.050 --> 00:40:17.090
the -- that just triggers --
00:40:17.090 --> 00:40:19.670
Q inverse is the
same as Q transpose.
00:40:19.670 --> 00:40:22.100
So the first component, then --
00:40:22.100 --> 00:40:25.970
the first component of
x is the first row times
00:40:25.970 --> 00:40:29.550
v, and what's that?
00:40:29.550 --> 00:40:32.650
The first component
of this answer
00:40:32.650 --> 00:40:36.570
is the first row of Q transpose.
00:40:36.570 --> 00:40:42.110
That's just -- that's
just q1 transpose times v.
00:40:42.110 --> 00:40:46.320
So that's what we
concluded here, too.
00:40:46.320 --> 00:40:46.820
Okay.
00:40:49.590 --> 00:40:55.090
So -- so nothing Fourier here.
00:40:55.090 --> 00:40:59.650
The -- the key ingredient
here was that the q-s are
00:40:59.650 --> 00:41:01.460
orthonormal.
00:41:01.460 --> 00:41:04.900
And now that's what Fourier
series are built on.
00:41:04.900 --> 00:41:08.520
So now, in the
remaining time, let
00:41:08.520 --> 00:41:11.760
me say something
about Fourier series.
00:41:11.760 --> 00:41:12.400
Okay.
00:41:12.400 --> 00:41:20.120
So Fourier series is --
00:41:20.120 --> 00:41:25.150
well, we've got a
function f of x.
00:41:25.150 --> 00:41:28.160
And we want to write it
as a combination of --
00:41:28.160 --> 00:41:30.900
maybe it has a constant term.
00:41:30.900 --> 00:41:34.910
And then it has
some cos(x) in it.
00:41:34.910 --> 00:41:38.120
And it has some sin(x) in it.
00:41:38.120 --> 00:41:42.330
And it has some cos(2x) in it.
00:41:42.330 --> 00:41:45.230
And a -- and some
sin(2x), and forever.
00:41:50.780 --> 00:41:54.600
So what's -- what's the
difference between this type
00:41:54.600 --> 00:41:56.900
problem and the one above it?
00:41:56.900 --> 00:42:02.090
This one's infinite,
but the key property
00:42:02.090 --> 00:42:06.110
of things being
orthogonal is still
00:42:06.110 --> 00:42:09.790
true for sines and cosines,
so it's the property that
00:42:09.790 --> 00:42:11.170
makes Fourier series work.
00:42:11.170 --> 00:42:12.860
So that's called
a Fourier series.
00:42:12.860 --> 00:42:14.620
Better write his name up.
00:42:14.620 --> 00:42:15.530
Fourier series.
00:42:22.170 --> 00:42:25.960
So it was Joseph Fourier
who realized that, hey, I
00:42:25.960 --> 00:42:29.870
could work in function space.
00:42:29.870 --> 00:42:33.970
Instead of a vector v, I
could have a function f of x.
00:42:33.970 --> 00:42:38.590
Instead of orthogonal
vectors, q1, q2 , q3,
00:42:38.590 --> 00:42:42.180
I could have orthogonal
functions, the constant,
00:42:42.180 --> 00:42:45.370
the cos(x), the
sin(x), the s- cos(2x),
00:42:45.370 --> 00:42:47.450
but infinitely many of them.
00:42:47.450 --> 00:42:49.930
I need infinitely
many, because my space
00:42:49.930 --> 00:42:52.440
is infinite dimensional.
00:42:52.440 --> 00:42:56.970
So this is, like, the moment
in which we leave finite
00:42:56.970 --> 00:43:00.680
dimensional vector spaces and go
to infinite dimensional vector
00:43:00.680 --> 00:43:03.090
spaces and our basis --
00:43:03.090 --> 00:43:07.250
so the vectors are
now functions --
00:43:07.250 --> 00:43:09.650
and of course, there are so
many functions that it's --
00:43:09.650 --> 00:43:13.620
that we've got an infin-
infinite dimensional space --
00:43:13.620 --> 00:43:17.320
and the basis vectors
are functions, too.
00:43:17.320 --> 00:43:25.400
a0, the constant function one
-- so my basis is one cos(x),
00:43:25.400 --> 00:43:29.190
sin(x), cos(2x),
sin(2x) and so on.
00:43:29.190 --> 00:43:32.710
And the reason Fourier
series is a success
00:43:32.710 --> 00:43:35.170
is that those are orthogonal.
00:43:35.170 --> 00:43:35.760
Okay.
00:43:35.760 --> 00:43:37.190
Now what do I mean
by orthogonal?
00:43:40.110 --> 00:43:44.490
I know what it means for two
vectors to be orthogonal --
00:43:44.490 --> 00:43:46.870
y transpose x
equals zero, right?
00:43:46.870 --> 00:43:48.600
Dot product equals zero.
00:43:48.600 --> 00:43:52.070
But what's the dot
product of functions?
00:43:52.070 --> 00:43:55.340
I'm claiming that whatever
it is, the dot product --
00:43:55.340 --> 00:43:59.840
or we would more likely use
the word inner product of, say,
00:43:59.840 --> 00:44:02.430
cos(x) with sin(x) is zero.
00:44:02.430 --> 00:44:06.340
And cos(x) with
cos(2x), also zero.
00:44:06.340 --> 00:44:08.780
So I -- let me tell you
what I mean by that,
00:44:08.780 --> 00:44:10.410
by that dot product.
00:44:10.410 --> 00:44:12.850
Well, how do I
compute a dot product?
00:44:12.850 --> 00:44:16.970
So, let's just remember
for vectors v trans-
00:44:16.970 --> 00:44:23.050
v transpose w for vectors,
so this was vectors,
00:44:23.050 --> 00:44:30.145
v transpose w was
v1w1 +...+vnwn.
00:44:33.510 --> 00:44:34.140
Okay.
00:44:34.140 --> 00:44:34.750
Now functions.
00:44:40.100 --> 00:44:42.720
Now I have two functions,
let's call them f and g.
00:44:45.260 --> 00:44:46.700
What's with them now?
00:44:46.700 --> 00:44:49.730
The vectors had n
components, but the functions
00:44:49.730 --> 00:44:53.060
have a whole, like, continuum.
00:44:53.060 --> 00:44:55.640
To graph the function, I
just don't have n points,
00:44:55.640 --> 00:44:57.780
I've got this whole graph.
00:44:57.780 --> 00:44:59.780
So I have functions --
00:44:59.780 --> 00:45:01.590
I'm really trying
to ask you what's
00:45:01.590 --> 00:45:03.840
the inner product
of this function
00:45:03.840 --> 00:45:05.170
f with another function
00:45:05.170 --> 00:45:06.130
g?
00:45:06.130 --> 00:45:11.870
And I want to make it parallel
to this the best I can.
00:45:11.870 --> 00:45:20.080
So the best parallel is to
multiply f (x) times g(x)
00:45:20.080 --> 00:45:23.060
at every x --
00:45:23.060 --> 00:45:25.120
and here I just had
n multiplications,
00:45:25.120 --> 00:45:28.330
but here I'm going to
have a whole range of x-s,
00:45:28.330 --> 00:45:31.980
and here I added the results.
00:45:31.980 --> 00:45:34.600
What do I do here?
00:45:34.600 --> 00:45:38.400
So what's the analog of
addition when you have --
00:45:38.400 --> 00:45:40.350
when you're in a continuum?
00:45:40.350 --> 00:45:41.830
It's integration.
00:45:41.830 --> 00:45:47.630
So that the -- the dot product
of two functions will be
00:45:47.630 --> 00:45:49.525
the integral of
those functions, dx.
00:45:52.240 --> 00:45:55.010
Now I have to say -- say,
well, what are the limits
00:45:55.010 --> 00:45:56.800
of integration?
00:45:56.800 --> 00:46:03.980
And for this Fourier series,
this function f(x) --
00:46:03.980 --> 00:46:08.000
if I'm going to have -- if that
right hand side is going to be
00:46:08.000 --> 00:46:11.920
f(x), that function that
I'm seeing on the right,
00:46:11.920 --> 00:46:16.130
all those sines and cosines,
they're all periodic, with --
00:46:16.130 --> 00:46:18.210
with period two pi.
00:46:18.210 --> 00:46:21.650
So -- so that's what
f(x) had better be.
00:46:21.650 --> 00:46:24.870
So I'll integrate
from zero to two pi.
00:46:24.870 --> 00:46:29.160
My -- all -- everything -- is
on the interval zero two pi now,
00:46:29.160 --> 00:46:33.690
because if I'm going to use
these sines and cosines,
00:46:33.690 --> 00:46:39.950
then f(x) is equal to f(x+2pi).
00:46:39.950 --> 00:46:42.428
This is periodic --
00:46:45.180 --> 00:46:48.250
periodic functions.
00:46:48.250 --> 00:46:49.800
Okay.
00:46:49.800 --> 00:46:52.770
So now I know what --
00:46:52.770 --> 00:46:56.120
I've got all the
right words now.
00:46:56.120 --> 00:47:00.490
I've got a vector space, but
the vectors are functions.
00:47:00.490 --> 00:47:05.450
I've got inner products and
-- and the inner product gives
00:47:05.450 --> 00:47:07.760
a number, all right.
00:47:07.760 --> 00:47:12.260
It just happens to be an
integral instead of a sum.
00:47:12.260 --> 00:47:15.230
I've got -- and that -- then I
have the idea of orthogonality
00:47:15.230 --> 00:47:15.860
--
00:47:15.860 --> 00:47:18.470
because, actually, just
-- let's just check.
00:47:18.470 --> 00:47:21.910
Orthogonality -- if I take
the integral -- s- I --
00:47:21.910 --> 00:47:25.890
let me do sin(x) times cos(x) --
00:47:25.890 --> 00:47:31.106
sin(x) times cos(x) dx
from zero to two pi --
00:47:34.680 --> 00:47:36.700
I think we get zero.
00:47:36.700 --> 00:47:40.670
That's the differential of
that, so it would be one half
00:47:40.670 --> 00:47:42.890
sine x squared, was that right?
00:47:47.220 --> 00:47:50.100
Between zero and two pi --
00:47:50.100 --> 00:47:52.910
and, of course, we get zero.
00:47:52.910 --> 00:47:58.150
And the same would be
true with a little more --
00:47:58.150 --> 00:48:02.430
some trig identities to help
us out -- of every other pair.
00:48:02.430 --> 00:48:05.650
So we have now an
orthonormal infinite
00:48:05.650 --> 00:48:10.160
basis for function space,
and all we want to do
00:48:10.160 --> 00:48:12.980
is express a function in that
00:48:12.980 --> 00:48:14.090
basis.
00:48:14.090 --> 00:48:20.120
And so I -- the end of my
lecture is, okay, what is a1?
00:48:20.120 --> 00:48:24.210
What's the coefficient --
how much cos(x) is there
00:48:24.210 --> 00:48:30.230
in a function compared
to the other harmonics?
00:48:30.230 --> 00:48:32.940
How much constant
is in that function?
00:48:32.940 --> 00:48:35.310
That'll -- that would
be an easy question.
00:48:35.310 --> 00:48:39.110
The answer a0 will come out
to be the average value of f.
00:48:39.110 --> 00:48:40.620
That's the amount
of the constant
00:48:40.620 --> 00:48:42.630
that's in there,
its average value.
00:48:42.630 --> 00:48:45.350
But let's take a1
as more typical.
00:48:45.350 --> 00:48:48.170
How will I get -- here's the
end of the lecture, then --
00:48:48.170 --> 00:48:49.400
how do I get a1?
00:48:52.070 --> 00:48:54.720
The first Fourier coefficient.
00:48:54.720 --> 00:48:56.350
Okay.
00:48:56.350 --> 00:48:59.790
I do just as I did
in the vector case.
00:48:59.790 --> 00:49:03.850
I take the inner product
of everything with cos(x)
00:49:03.850 --> 00:49:07.010
Take the inner product of
everything with cos(x).
00:49:07.010 --> 00:49:08.800
Then on the left --
00:49:08.800 --> 00:49:13.990
on the left I have -- the inner
product is the integral of f(x)
00:49:13.990 --> 00:49:15.365
times cos(x) cx.
00:49:18.740 --> 00:49:22.080
And on the right,
what do I have?
00:49:22.080 --> 00:49:24.970
When I -- so what I -- when I
say take the inner product with
00:49:24.970 --> 00:49:28.850
cos(x), let me put it in
ordinary calculus words.
00:49:28.850 --> 00:49:32.610
Multiply by cos(x)
and integrate.
00:49:32.610 --> 00:49:34.540
That's what inner products are.
00:49:34.540 --> 00:49:36.940
So if I multiply that
whole thing by cos(x)
00:49:36.940 --> 00:49:40.990
and I integrate, I get
a whole lot of zeroes.
00:49:40.990 --> 00:49:45.830
The only thing that
survives is that term.
00:49:45.830 --> 00:49:47.250
All the others disappear.
00:49:47.250 --> 00:49:53.950
So -- and that term is a1 times
the integral of cos(x) squared
00:49:53.950 --> 00:50:01.580
dx zero to 2pi equals -- so this
was the left side and this is
00:50:01.580 --> 00:50:04.840
all that's left on
the right-hand side.
00:50:04.840 --> 00:50:09.700
And this is not zero of
course, because it's the length
00:50:09.700 --> 00:50:13.570
of the function squared, it's
the inner product with itself,
00:50:13.570 --> 00:50:18.050
and -- and a simple calculation
gives that answer to be pi.
00:50:18.050 --> 00:50:23.360
So that's an easy integral
and it turns out to be pi,
00:50:23.360 --> 00:50:31.900
so that a1 is one over pi times
there -- times this integral.
00:50:31.900 --> 00:50:35.490
So there is, actually --
that's Euler's famous formula
00:50:35.490 --> 00:50:39.110
for the -- or maybe
Fourier found it --
00:50:39.110 --> 00:50:41.405
for the coefficients
in a Fourier series.
00:50:43.980 --> 00:50:47.940
And you see that it's
exactly an expansion
00:50:47.940 --> 00:50:50.790
in an orthonormal basis.
00:50:50.790 --> 00:50:51.540
Okay, thanks.
00:50:51.540 --> 00:50:56.660
So I'll do a quiz review on
Monday and then the quiz itself
00:50:56.660 --> 00:50:59.200
in Walker on Wednesday.
00:50:59.200 --> 00:51:00.590
Okay, see you Monday.
00:51:00.590 --> 00:51:02.140
Thanks.