1
00:00:06 --> 00:00:09
-- two, one and -- okay.
2
00:00:09 --> 00:00:16
Here is a lecture on the
applications of eigenvalues and,
3
00:00:16 --> 00:00:21.55
if I can -- so that will be
Markov matrices.
4
00:00:21.55 --> 00:00:28
I'll tell you what a Markov
matrix is, so this matrix A will
5
00:00:28 --> 00:00:35
be a Markov matrix and I'll
explain how they come in
6
00:00:35 --> 00:00:38
applications.
7
00:00:38 --> 00:00:43
And -- and then if I have time,
I would like to say a little
8
00:00:43 --> 00:00:48
bit about Fourier series,
which is a fantastic
9
00:00:48 --> 00:00:51
application of the projection
chapter.
10
00:00:51 --> 00:00:52
Okay.
11
00:00:52 --> 00:00:54
What's a Markov matrix?
12
00:00:54 --> 00:00:58.81
Can I just write down a typical
Markov matrix,
13
00:00:58.81 --> 00:01:01
say .1, .2, .7,
.01,
14
00:01:01 --> 00:01:05
.99 0, let's say,
.3, .3, .4.
15
00:01:05 --> 00:01:05
Okay.
16
00:01:05 --> 00:01:11
There's a -- a totally just
invented Markov matrix.
17
00:01:11 --> 00:01:14
What makes it a Markov matrix?
18
00:01:14 --> 00:01:19
Two properties that this --
this matrix has.
19
00:01:19 --> 00:01:27
So two properties are -- one,
every entry is greater equal
20
00:01:27 --> 00:01:28
zero.
21
00:01:28 --> 00:01:32
All entries greater than or
equal to zero.
22
00:01:32 --> 00:01:37
And, of course,
when I square the matrix,
23
00:01:37 --> 00:01:42
the entries will still be
greater/equal zero.
24
00:01:42 --> 00:01:48
I'm going to be interested in
the powers of this matrix.
25
00:01:48 --> 00:01:53
And this property,
of course, is going to -- stay
26
00:01:53 --> 00:01:54
there.
27
00:01:54 --> 00:01:59.05
It -- really Markov matrices
you'll see are connected to
28
00:01:59.05 --> 00:02:02.74
probability ideas and
probabilities are never
29
00:02:02.74 --> 00:02:03
negative.
30
00:02:03 --> 00:02:08
The other property -- do you
see the other property in there?
31
00:02:08 --> 00:02:14
If I add down the columns,
what answer do I get?
32
00:02:14 --> 00:02:14.81
One.
33
00:02:14.81 --> 00:02:18
So all columns add to one.
34
00:02:18 --> 00:02:21.21
All columns add to one.
35
00:02:21.21 --> 00:02:29
And actually when I square the
matrix, that will be true again.
36
00:02:29 --> 00:02:36
So that the powers of my matrix
are all Markov matrices,
37
00:02:36 --> 00:02:43
and I'm interested in,
always, the eigenvalues and the
38
00:02:43 --> 00:02:46.55
eigenvectors.
39
00:02:46.55 --> 00:02:49
And this question of steady
state will come up.
40
00:02:49 --> 00:02:54
You remember we had steady
state for differential equations
41
00:02:54 --> 00:02:54
last time?
42
00:02:54 --> 00:02:58
When -- what was the steady
state -- what was the
43
00:02:58 --> 00:02:59
eigenvalue?
44
00:02:59 --> 00:03:03
What was the eigenvalue in the
differential equation case that
45
00:03:03 --> 00:03:05
led to a steady state?
46
00:03:05 --> 00:03:08
It was lambda equals zero.
47
00:03:08 --> 00:03:12
When -- you remember that we
did an example and one of the
48
00:03:12 --> 00:03:17
eigenvalues was lambda equals
zero, and that -- so then we had
49
00:03:17 --> 00:03:22
an E to the zero T,
a constant one -- as time went
50
00:03:22 --> 00:03:24
on, there that thing stayed
steady.
51
00:03:24 --> 00:03:30
Now what -- in the powers case,
it's not a zero eigenvalue.
52
00:03:30 --> 00:03:34
Actually with powers of a
matrix, a zero eigenvalue,
53
00:03:34 --> 00:03:37
that part is going to die right
away.
54
00:03:37 --> 00:03:40
It's an eigenvalue of one
that's all important.
55
00:03:40 --> 00:03:45
So this steady state will
correspond -- will be totally
56
00:03:45 --> 00:03:49
connected with an eigenvalue of
one and its eigenvector.
57
00:03:49 --> 00:03:52
In fact, the steady state will
be
58
00:03:52 --> 00:03:56
the eigenvector for that
eigenvalue.
59
00:03:56 --> 00:03:56
Okay.
60
00:03:56 --> 00:03:58
So that's what's coming.
61
00:03:58 --> 00:04:02
Now, for some reason then that
we have to see,
62
00:04:02 --> 00:04:06
this matrix has an eigenvalue
of one.
63
00:04:06 --> 00:04:10
This property,
that the columns all add to one
64
00:04:10 --> 00:04:15
-- turns out -- guarantees that
one is an eigenvalue,
65
00:04:15 --> 00:04:20
so that you can
actually find the eigenvalue --
66
00:04:20 --> 00:04:24
find that eigenvalue of a Markov
matrix without computing any
67
00:04:24 --> 00:04:28
determinants of A minus lambda I
-- that matrix will have an
68
00:04:28 --> 00:04:31
eigenvalue of one,
and we want to see why.
69
00:04:31 --> 00:04:36
And then the other thing is --
so the key points -- let me --
70
00:04:36 --> 00:04:39
let me write these underneath.
71
00:04:39 --> 00:04:46.11
The key points are -- the key
points are lambda equal one is
72
00:04:46.11 --> 00:04:47
an eigenvalue.
73
00:04:47 --> 00:04:53
I'll add in a little -- an
additional -- well,
74
00:04:53 --> 00:04:58.14
a thing about eigenvalues --
key point two,
75
00:04:58.14 --> 00:05:04
the other eigenval- values --
all other eigenvalues are,
76
00:05:04 --> 00:05:08
in magnitude,
smaller than one -- in absolute
77
00:05:08 --> 00:05:11
value, smaller than one.
78
00:05:11 --> 00:05:16
Well, there could be some
exceptional case when -- when an
79
00:05:16 --> 00:05:21
eigen -- another eigenvalue
might have magnitude equal one.
80
00:05:21 --> 00:05:25
It never has an eigenvalue
larger than one.
81
00:05:25 --> 00:05:30
So these two facts --
somehow we ought to -- linear
82
00:05:30 --> 00:05:32
algebra ought to tell us.
83
00:05:32 --> 00:05:36
And then, of course,
linear algebra is going to tell
84
00:05:36 --> 00:05:40
us what the -- what's -- what
happens if I take -- if -- you
85
00:05:40 --> 00:05:45
remember when I solve -- when I
multiply by A time after time
86
00:05:45 --> 00:05:51
the K-th thing is A to the K u0
and I'm asking what's special
87
00:05:51 --> 00:05:57
about this -- these powers of A,
and very likely the quiz will
88
00:05:57 --> 00:06:03
have a problem to computer s- to
computer some powers of A or --
89
00:06:03 --> 00:06:06
or applied to an initial vector.
90
00:06:06 --> 00:06:09
So, you remember the general
form?
91
00:06:09 --> 00:06:13
The general form is that
there's
92
00:06:13 --> 00:06:17
some amount of the first
eigenvalue to the K-th power
93
00:06:17 --> 00:06:21
times the first eigenvector,
and another amount of the
94
00:06:21 --> 00:06:26
second eigenvalue to the K-th
power times the second
95
00:06:26 --> 00:06:27
eigenvector and so on.
96
00:06:27 --> 00:06:32
A -- just -- my conscience
always makes me say at least
97
00:06:32 --> 00:06:36
once per lecture that this
requires
98
00:06:36 --> 00:06:40.92
a complete set of eigenvectors,
otherwise we might not be able
99
00:06:40.92 --> 00:06:45
to expand u0 in the eigenvectors
and we couldn't get started.
100
00:06:45 --> 00:06:48
But once we're started with u0
when K is zero,
101
00:06:48 --> 00:06:51
then every A brings in these
lambdas.
102
00:06:51 --> 00:06:56
And now you can see what the
steady state is going to be.
103
00:06:56 --> 00:07:01
If lambda one is one -- so
lambda one equals one to the
104
00:07:01 --> 00:07:06
K-th power and these other
eigenvalues are smaller than one
105
00:07:06 --> 00:07:12
-- so I've sort of scratched
over the equation there to -- we
106
00:07:12 --> 00:07:16
had this term,
but what happens to this term
107
00:07:16 --> 00:07:20
-- if the lambda's smaller than
one,
108
00:07:20 --> 00:07:24
then the -- when -- as we take
powers, as we iterate as we --
109
00:07:24 --> 00:07:27.38
as we go forward in time,
this goes to zero,
110
00:07:27.38 --> 00:07:27
right?
111
00:07:27 --> 00:07:31
Can I just -- having scratched
over it, I might as well scratch
112
00:07:31 --> 00:07:32
further.
113
00:07:32 --> 00:07:36
That term and all the other
terms are going to zero because
114
00:07:36 --> 00:07:42
all the other eigenvalues
are smaller than one and the
115
00:07:42 --> 00:07:49
steady state that we're
approaching is just -- whatever
116
00:07:49 --> 00:07:57
there was -- this was -- this
was the -- this is the x1 part
117
00:07:57 --> 00:08:05
of un- of the initial condition
u0 -- is the steady state.
118
00:08:05 --> 00:08:09
This much we know from general
-- from -- you know,
119
00:08:09 --> 00:08:10
what we've already done.
120
00:08:10 --> 00:08:14.76
So I want to see why -- let's
at least see number one,
121
00:08:14.76 --> 00:08:16
why one is an eigenvalue.
122
00:08:16 --> 00:08:21.1
And then there's actually -- in
this chapter we're interested
123
00:08:21.1 --> 00:08:25
not only in eigenvalues,
but also eigenvectors.
124
00:08:25 --> 00:08:30
And there's something special
about the eigenvector.
125
00:08:30 --> 00:08:33
Let me write down what that is.
126
00:08:33 --> 00:08:39
The eigenvector x1 -- x1 is the
eigenvector and all its
127
00:08:39 --> 00:08:44
components are positive,
so the steady state is
128
00:08:44 --> 00:08:47
positive, if the start was.
129
00:08:47 --> 00:08:52
If the start was -- so -- well,
actually,
130
00:08:52 --> 00:08:57
in general, I -- this might
have a -- might have some
131
00:08:57 --> 00:09:02.67
component zero always,
but no negative components in
132
00:09:02.67 --> 00:09:04
that eigenvector.
133
00:09:04 --> 00:09:04
Okay.
134
00:09:04 --> 00:09:07
Can I come to that point?
135
00:09:07 --> 00:09:12
How can I look at that matrix
-- so that was just an example.
136
00:09:12 --> 00:09:19
How could I be sure -- how can
I see that a matrix --
137
00:09:19 --> 00:09:24
if the columns add to zero --
add to one, sorry -- if the
138
00:09:24 --> 00:09:29
columns add to one,
this property means that lambda
139
00:09:29 --> 00:09:32
equal one is an eigenvalue.
140
00:09:32 --> 00:09:32
Okay.
141
00:09:32 --> 00:09:35
So let's just think that
through.
142
00:09:35 --> 00:09:41.87
What I saying about -- let me
ca- let me look at A,
143
00:09:41.87 --> 00:09:47.34
and if I believe that one is an
eigenvalue, then I should be
144
00:09:47.34 --> 00:09:52
able to subtract off one times
the identity and then I would
145
00:09:52 --> 00:09:57
get a matrix that's,
what, -.9, -.01 and -.6 -- wh-
146
00:09:57 --> 00:10:02
I took the ones away and the
other parts, of course,
147
00:10:02 --> 00:10:08
are still what they were,
and this is still .2 and .7 and
148
00:10:08 --> 00:10:13
-- okay, what's -- what's up
with this matrix now?
149
00:10:13 --> 00:10:18
I've shifted the matrix,
this Markov matrix by one,
150
00:10:18 --> 00:10:23.2
by the identity,
and what do I want to prove?
151
00:10:23.2 --> 00:10:28
I -- what is it that I believe
this matrix -- about this
152
00:10:28 --> 00:10:30
matrix?
153
00:10:30 --> 00:10:32
I believe it's singular.
154
00:10:32 --> 00:10:36
Singular will -- if A minus I
is singular, that tells me that
155
00:10:36 --> 00:10:38
one is an eigenvalue,
right?
156
00:10:38 --> 00:10:42
The eigenvalues are the numbers
that I subtract off -- the
157
00:10:42 --> 00:10:47
shifts -- the numbers that I
subtract from the diagonal -- to
158
00:10:47 --> 00:10:48
make it singular.
159
00:10:48 --> 00:10:51
Now why is that matrix
singular?
160
00:10:51 --> 00:10:56
I -- we could compute its
determinant, but we want to see
161
00:10:56 --> 00:11:00
a reason that would work for
every Markov matrix not just
162
00:11:00 --> 00:11:03
this particular random example.
163
00:11:03 --> 00:11:05
So what is it about that
matrix?
164
00:11:05 --> 00:11:11
Well, I guess you could look at
its columns now -- what
165
00:11:11 --> 00:11:14
do they add up to?
166
00:11:14 --> 00:11:15
Zero.
167
00:11:15 --> 00:11:24
The columns add to zero,
so all columns -- let me put
168
00:11:24 --> 00:11:33
all columns now of -- of -- of A
minus I add to zero,
169
00:11:33 --> 00:11:42
and then I want to realize that
this means A minus I is
170
00:11:42 --> 00:11:44
singular.
171
00:11:44 --> 00:11:44
Okay.
172
00:11:44 --> 00:11:45
Why?
173
00:11:45 --> 00:11:52
So I could I --
you know, that could be a quiz
174
00:11:52 --> 00:11:55
question, a sort of theoretical
quiz question.
175
00:11:55 --> 00:11:59
If I give you a matrix and I
tell you all its columns add to
176
00:11:59 --> 00:12:02
zero, give me a reason,
because it is true,
177
00:12:02 --> 00:12:04
that the matrix is singular.
178
00:12:04 --> 00:12:04.35
Okay.
179
00:12:04.35 --> 00:12:07
I guess actually -- now what --
I think of -- you know,
180
00:12:07 --> 00:12:11
I'm thinking of two or three
ways to
181
00:12:11 --> 00:12:12.06
see that.
182
00:12:12.06 --> 00:12:13
How would you do it?
183
00:12:13 --> 00:12:18
We don't want to take its
determinant somehow.
184
00:12:18 --> 00:12:24
For the matrix to be singular,
well, it means that these three
185
00:12:24 --> 00:12:26
columns are dependent,
right?
186
00:12:26 --> 00:12:32
The determinant will be zero
when those three columns are
187
00:12:32 --> 00:12:33.12
dependent.
188
00:12:33.12 --> 00:12:37
You see, we're --
we're at a point in this
189
00:12:37 --> 00:12:40
course, now, where we have
several ways to look at an idea.
190
00:12:40 --> 00:12:44
We can take the determinant --
here we don't want to.
191
00:12:44 --> 00:12:47.39
B- but we met singular before
that -- those columns are
192
00:12:47.39 --> 00:12:48
dependent.
193
00:12:48 --> 00:12:50
So how do I see that those
columns are dependent?
194
00:12:50 --> 00:12:52
They all add to zero.
195
00:12:52 --> 00:12:56
Let's see, whew -- well,
oh, actually,
196
00:12:56 --> 00:13:03
what -- another thing I know is
that the -- I would like to be
197
00:13:03 --> 00:13:07
able to show is that the rows
are dependent.
198
00:13:07 --> 00:13:10
Maybe that's easier.
199
00:13:10 --> 00:13:15
If I know that all the columns
add to zero, that's my
200
00:13:15 --> 00:13:21
information, how do I see that
those three rows
201
00:13:21 --> 00:13:23
are linearly dependent?
202
00:13:23 --> 00:13:28
What -- what combination of
those rows gives the zero row?
203
00:13:28 --> 00:13:33
How -- how could I combine
those three rows -- those three
204
00:13:33 --> 00:13:36
row vectors to produce the zero
row vector?
205
00:13:36 --> 00:13:40
And that would tell me those
rows are dependent,
206
00:13:40 --> 00:13:44
therefore the columns are
dependent,
207
00:13:44 --> 00:13:47
the matrix is singular,
the determinant is zero --
208
00:13:47 --> 00:13:49
well, you see it.
209
00:13:49 --> 00:13:50
I just add the rows.
210
00:13:50 --> 00:13:54
One times that row plus one
times that row plus one times
211
00:13:54 --> 00:13:57.27
that row -- it's the zero row.
212
00:13:57.27 --> 00:13:59
The rows are dependent.
213
00:13:59 --> 00:14:03
In a way, that one one one,
because it's multiplying the
214
00:14:03 --> 00:14:07
rows,
is like an eigenvector in the
215
00:14:07 --> 00:14:11
-- it's in the left null space,
right?
216
00:14:11 --> 00:14:15
One one one is in the left null
space.
217
00:14:15 --> 00:14:22
It's singular because the rows
are dependent -- and can I just
218
00:14:22 --> 00:14:24.76
keep the reasoning going?
219
00:14:24.76 --> 00:14:29
Because this vector one one one
is --
220
00:14:29 --> 00:14:34
it's not in the null space of
the matrix, but it's in the null
221
00:14:34 --> 00:14:40
space of the transpose -- is in
the null space of the transpose.
222
00:14:40 --> 00:14:42
And that's good enough.
223
00:14:42 --> 00:14:47.56
If we have a square matrix --
if we have a square matrix and
224
00:14:47.56 --> 00:14:50
the rows are dependent,
that
225
00:14:50 --> 00:14:52
matrix is singular.
226
00:14:52 --> 00:14:57
So it turned out that the
immediate guy we could identify
227
00:14:57 --> 00:14:59
was one one one.
228
00:14:59 --> 00:15:05
Of course, the -- there will be
somebody in the null space,
229
00:15:05 --> 00:15:05
too.
230
00:15:05 --> 00:15:08.09
And actually,
who will it be?
231
00:15:08.09 --> 00:15:13
So what's -- so -- so now I
want to ask about the
232
00:15:13 --> 00:15:16
null space of -- of the matrix
itself.
233
00:15:16 --> 00:15:19
What combination of the columns
gives zero?
234
00:15:19 --> 00:15:23.11
I -- I don't want to compute it
because I just made up this
235
00:15:23.11 --> 00:15:26
matrix and -- it will -- it
would take me a while -- it
236
00:15:26 --> 00:15:31
looks sort of doable
because it's three by three but
237
00:15:31 --> 00:15:34
wh- my point is,
what -- what vector is it if we
238
00:15:34 --> 00:15:39
-- once we've found it,
what have we got that's in the
239
00:15:39 --> 00:15:41
-- in the null space of A?
240
00:15:41 --> 00:15:43
It's the eigenvector,
right?
241
00:15:43 --> 00:15:46
That's where we find X one.
242
00:15:46 --> 00:15:51
Then X one, the eigenvector,
is in the null space of A.
243
00:15:51 --> 00:15:56
That's the eigenvector
corresponding to the eigenvalue
244
00:15:56 --> 00:15:56
one.
245
00:15:56 --> 00:15:57
Right?
246
00:15:57 --> 00:16:00
That's how we find
eigenvectors.
247
00:16:00 --> 00:16:05
So those three columns must be
dependent -- some combination of
248
00:16:05 --> 00:16:10.29
columns --
of those three columns is the
249
00:16:10.29 --> 00:16:14
zero column and that -- the
three components in that
250
00:16:14 --> 00:16:17
combination are the eigenvector.
251
00:16:17 --> 00:16:19
And that guy is the steady
state.
252
00:16:19 --> 00:16:20
Okay.
253
00:16:20 --> 00:16:23
So I'm happy about the -- the
thinking here,
254
00:16:23 --> 00:16:28
but I haven't given -- I
haven't completed it because I
255
00:16:28 --> 00:16:29
haven't found x1.
256
00:16:29 --> 00:16:32
But it's there.
257
00:16:32 --> 00:16:39
Can I -- another thought came
to me as I was doing this,
258
00:16:39 --> 00:16:46
another little comment that --
you -- about eigenvalues and
259
00:16:46 --> 00:16:51
eigenvectors,
because of A and A transpose.
260
00:16:51 --> 00:16:58
What can you tell me about
eigenvalues of A -- of A and
261
00:16:58 --> 00:17:01
eigenvalues of A transpose?
262
00:17:01 --> 00:17:02
Whoops.
263
00:17:02 --> 00:17:06
They're the same.
264
00:17:06 --> 00:17:10
They're -- so this is a little
comment -- we -- it's useful,
265
00:17:10 --> 00:17:14
since eigenvalues are generally
not easy to find -- it's always
266
00:17:14 --> 00:17:17
useful to know some cases where
you've got them,
267
00:17:17 --> 00:17:21
where -- and this is -- if you
know the eigenvalues of A,
268
00:17:21 --> 00:17:25
then you know the eigenvalues
of A transpose.
269
00:17:25 --> 00:17:30
eigenvalues of A transpose are
the same.
270
00:17:30 --> 00:17:35
And can I just,
like, review why that is?
271
00:17:35 --> 00:17:42
So to find the eigenvalues of
A, this would be determinate of
272
00:17:42 --> 00:17:49
A minus lambda I equals zero,
that gives me an eigenvalue of
273
00:17:49 --> 00:17:57
A -- now how can I get A
transpose into the picture here?
274
00:17:57 --> 00:18:02
I'll use the fact that the
determinant of a matrix and the
275
00:18:02 --> 00:18:05
determinant of its transpose are
the same.
276
00:18:05 --> 00:18:09
The determinant of a matrix
equals the determinant of a --
277
00:18:09 --> 00:18:10.39
of the transpose.
278
00:18:10.39 --> 00:18:13
That was property ten,
the very last guy in our
279
00:18:13 --> 00:18:15
determinant list.
280
00:18:15 --> 00:18:18
So I'll transpose that matrix.
281
00:18:18 --> 00:18:24.43
This leads to -- I just take
the matrix and transpose it,
282
00:18:24.43 --> 00:18:29
but now what do I get when I
transpose lambda I?
283
00:18:29 --> 00:18:31
I just get lambda I.
284
00:18:31 --> 00:18:35
So that's -- that's all there
was to the
285
00:18:35 --> 00:18:36
reasoning.
286
00:18:36 --> 00:18:40
The reasoning is that the
eigenvalues of A solved that
287
00:18:40 --> 00:18:41
equation.
288
00:18:41 --> 00:18:44
The determinant of a matrix is
the determinant of its
289
00:18:44 --> 00:18:49.16
transpose, so that gives me this
equation and that tells me that
290
00:18:49.16 --> 00:18:53
the same lambdas are eigenvalues
of A transpose.
291
00:18:53 --> 00:18:56
So that, backing up to the
Markov case, one is an
292
00:18:56 --> 00:19:01
eigenvalue of A transpose and we
actually found its eigenvector,
293
00:19:01 --> 00:19:04
one one one,
and that tell us that one is
294
00:19:04 --> 00:19:08
also an eigenvalue of A -- but,
of course, it has a different
295
00:19:08 --> 00:19:11.66
eigenvector,
the -- the left null space
296
00:19:11.66 --> 00:19:15
isn't the same as the null space
and we would have to find it.
297
00:19:15 --> 00:19:19
So there's some vector here
which is x1 that produces zero
298
00:19:19 --> 00:19:19
zero zero.
299
00:19:19 --> 00:19:22
Actually, it wouldn't be that
hard to find,
300
00:19:22 --> 00:19:25
you know, I -- as I'm talking
I'm thinking,
301
00:19:25 --> 00:19:29
okay, I going to follow through
and actually find it?
302
00:19:29 --> 00:19:34
Well, I can tell from this one
-- look, if I put a point six
303
00:19:34 --> 00:19:39
there and a point seven there,
that's what -- then I'll be
304
00:19:39 --> 00:19:41
okay in the last row,
right?
305
00:19:41 --> 00:19:45
Now I only -- remains to find
one guy.
306
00:19:45 --> 00:19:48
And let me take the first row,
then.
307
00:19:48 --> 00:19:54
Minus point 54 plus point 21 --
there's some big number going in
308
00:19:54 --> 00:19:55.54
there, right?
309
00:19:55.54 --> 00:19:59
So I have -- just to make the
first row come out zero,
310
00:19:59 --> 00:20:04
I'm getting minus point 54 plus
point 21, so that was minus
311
00:20:04 --> 00:20:07
point 33 and what -- what do I
want?
312
00:20:07 --> 00:20:09
Like thirty three hundred?
313
00:20:09 --> 00:20:13
This is the first time in the
history of linear algebra that
314
00:20:13 --> 00:20:17
an
eigenvector has every had a
315
00:20:17 --> 00:20:19
component thirty three hundred.
316
00:20:19 --> 00:20:21
But I guess it's true.
317
00:20:21 --> 00:20:26
Because then I multiply by
minus one over a hundred -- oh
318
00:20:26 --> 00:20:27
no, it was point 33.
319
00:20:27 --> 00:20:30
So is this just -- oh,
shoot.
320
00:20:30 --> 00:20:30
Only 33.
321
00:20:30 --> 00:20:31
Okay.
322
00:20:31 --> 00:20:31
Only 33.
323
00:20:31 --> 00:20:35
Okay, so there's the
eigenvector.
324
00:20:35 --> 00:20:39.74
Oh, and notice that it -- that
it turned -- did turn out,
325
00:20:39.74 --> 00:20:41
at least, to be all positive.
326
00:20:41 --> 00:20:45
So that was,
like, the theory -- predicts
327
00:20:45 --> 00:20:46
that part, too.
328
00:20:46 --> 00:20:48
I won't give the proof of that
part.
329
00:20:48 --> 00:20:51
So 30 -- 33 -- point six 33
point seven.
330
00:20:51 --> 00:20:52
Okay.
331
00:20:52 --> 00:20:57
Now those are the ma- that's
the linear algebra part.
332
00:20:57 --> 00:20:59
Can I get to the applications?
333
00:20:59 --> 00:21:01
Where do these Markov matrices
come from?
334
00:21:01 --> 00:21:05
Because that's -- that's part
of this course and absolutely
335
00:21:05 --> 00:21:07
part of this lecture.
336
00:21:07 --> 00:21:07
Okay.
337
00:21:07 --> 00:21:12
So where's -- what's an
application of Markov matrices?
338
00:21:12 --> 00:21:12
Okay.
339
00:21:12 --> 00:21:17
Markov matrices -- so,
my equation,
340
00:21:17 --> 00:21:24
then, that I'm solving and
studying is this equation
341
00:21:24 --> 00:21:25
u(k+1)=Auk.
342
00:21:25 --> 00:21:29.57
And now A is a Markov matrix.
343
00:21:29.57 --> 00:21:31
A is Markov.
344
00:21:31 --> 00:21:35
And I want to give an example.
345
00:21:35 --> 00:21:39
Can I just create an example?
346
00:21:39 --> 00:21:43
It'll be two by two.
347
00:21:43 --> 00:21:48
And it's one I've used before
because it seems to me to bring
348
00:21:48 --> 00:21:50
out the idea.
349
00:21:50 --> 00:21:55
It's -- because we have two by
two, we have two states,
350
00:21:55 --> 00:21:58
let's say California and
Massachusetts.
351
00:21:58 --> 00:22:03
And I'm looking at the
populations in those two states,
352
00:22:03 --> 00:22:10
the people in those two states,
California and Massachusetts.
353
00:22:10 --> 00:22:14
And my matrix A is going to
tell me in a -- in a year,
354
00:22:14 --> 00:22:16
some movement has happened.
355
00:22:16 --> 00:22:18
Some people stayed in
Massachusetts,
356
00:22:18 --> 00:22:22
some people moved to
California, some smart people
357
00:22:22 --> 00:22:24
moved from California to
Massachusetts,
358
00:22:24 --> 00:22:29
some people stayed in
California and made a billion.
359
00:22:29 --> 00:22:29
Okay.
360
00:22:29 --> 00:22:33
So that -- there's a matrix
there with four entries and
361
00:22:33 --> 00:22:38
those tell me the fractions of
my population -- so I'm making
362
00:22:38 --> 00:22:42
-- I'm going to use fractions,
so they won't be negative,
363
00:22:42 --> 00:22:47
of course, because -- because
only positive people are in-
364
00:22:47 --> 00:22:51
involved here -- and they'll
add up to one,
365
00:22:51 --> 00:22:53
because I'm accounting for all
people.
366
00:22:53 --> 00:22:57
So that's why I have these two
key properties.
367
00:22:57 --> 00:23:01
The entries are greater equal
zero because I'm looking at
368
00:23:01 --> 00:23:02
probabilities.
369
00:23:02 --> 00:23:05
Do they move,
do they stay?
370
00:23:05 --> 00:23:09
Those probabilities are all
between zero and one.
371
00:23:09 --> 00:23:12
And the probabilities add to
one because everybody's
372
00:23:12 --> 00:23:13
accounted for.
373
00:23:13 --> 00:23:17
I'm not losing anybody,
gaining anybody in this Markov
374
00:23:17 --> 00:23:18
chain.
375
00:23:18 --> 00:23:21
It's -- it conserves the total
population.
376
00:23:21 --> 00:23:21
Okay.
377
00:23:21 --> 00:23:25
So what would be a typical
matrix, then?
378
00:23:25 --> 00:23:31
So this would be u,
California and u Massachusetts
379
00:23:31 --> 00:23:34
at time t equal k+1.
380
00:23:34 --> 00:23:39
And it's some matrix,
which we'll think of,
381
00:23:39 --> 00:23:45.83
times u California and u
Massachusetts at time k.
382
00:23:45.83 --> 00:23:51
And notice this matrix is going
to stay the same,
383
00:23:51 --> 00:23:54
you know, forever.
384
00:23:54 --> 00:24:00
So that's a severe limitation
on the example.
385
00:24:00 --> 00:24:05
The example has a -- the same
Markov matrix,
386
00:24:05 --> 00:24:09
the same probabilities act at
every time.
387
00:24:09 --> 00:24:09
Okay.
388
00:24:09 --> 00:24:14.78
So what's a reasonable,
say -- say point nine of the
389
00:24:14.78 --> 00:24:18
people in California at time k
stay there.
390
00:24:18 --> 00:24:23
And point one of the people in
California move to
391
00:24:23 --> 00:24:26
Massachusetts.
392
00:24:26 --> 00:24:31
Notice why that column added to
one, because we've now accounted
393
00:24:31 --> 00:24:34.31
for all the people in California
at time k.
394
00:24:34.31 --> 00:24:37
Nine tenths of them are still
in California,
395
00:24:37 --> 00:24:39
one tenth are here at time k+1.
396
00:24:39 --> 00:24:40
Okay.
397
00:24:40 --> 00:24:43
What about the people who are
in Massachusetts?
398
00:24:43 --> 00:24:46
This is going to multiply
column two, right,
399
00:24:46 --> 00:24:51
by our
fundamental rule of multiplying
400
00:24:51 --> 00:24:56
matrix by vector,
it's the -- it's the population
401
00:24:56 --> 00:24:58
in Massachusetts.
402
00:24:58 --> 00:25:04.25
Shall we say that -- that after
the Red Sox, fail again,
403
00:25:04.25 --> 00:25:10
eight -- only 80 percent of the
people in Massachusetts stay and
404
00:25:10 --> 00:25:14
20 percent
move to California.
405
00:25:14 --> 00:25:14
Okay.
406
00:25:14 --> 00:25:19
So again, this adds to one,
which accounts for all people
407
00:25:19 --> 00:25:21
in Massachusetts where they are.
408
00:25:21 --> 00:25:23
So there is a Markov matrix.
409
00:25:23 --> 00:25:26
Non-negative entries adding to
one.
410
00:25:26 --> 00:25:28
What's the steady state?
411
00:25:28 --> 00:25:32
If everybody started in
Massachusetts,
412
00:25:32 --> 00:25:36
say, at -- you know,
when the Pilgrims showed up or
413
00:25:36 --> 00:25:36
something.
414
00:25:36 --> 00:25:38
Then where are they now?
415
00:25:38 --> 00:25:43
Where are they at time 100,
let's say, or maybe -- I don't
416
00:25:43 --> 00:25:46
know, how many years since the
Pilgrims?
417
00:25:46 --> 00:25:47
300 and something.
418
00:25:47 --> 00:25:51
Or -- and actually where will
they be, like,
419
00:25:51 --> 00:25:54
way out a million
years from now?
420
00:25:54 --> 00:25:59
I -- I could multiply -- take
the powers of this matrix.
421
00:25:59 --> 00:26:05
In fact, you'll -- you would --
ought to be able to figure out
422
00:26:05 --> 00:26:09
what is the hundredth power of
that matrix?
423
00:26:09 --> 00:26:11
Why don't we do that?
424
00:26:11 --> 00:26:15
But let me follow the steady
state.
425
00:26:15 --> 00:26:20
So what -- what's my starting
-- my starting u Cal,
426
00:26:20 --> 00:26:26
u Mass at time zero is,
shall we say -- shall we put
427
00:26:26 --> 00:26:28.8
anybody in California?
428
00:26:28.8 --> 00:26:35
Let's make -- let's make zero
there, and say the population of
429
00:26:35 --> 00:26:41
Massachusetts is --
let's say a thousand just to --
430
00:26:41 --> 00:26:42
okay.
431
00:26:42 --> 00:26:47.68
So the population is -- so the
populations are zero and a
432
00:26:47.68 --> 00:26:49
thousand at the start.
433
00:26:49 --> 00:26:55
What can you tell me about this
population after -- after k
434
00:26:55 --> 00:26:56
steps?
435
00:26:56 --> 00:26:59
What will u Cal plus u Mass add
to?
436
00:26:59 --> 00:27:00.7
A thousand.
437
00:27:00.7 --> 00:27:06
Those thousand people are
always accounted for.
438
00:27:06 --> 00:27:10
But -- so u Mass will start
dropping from a thousand and u
439
00:27:10 --> 00:27:12
Cal will start growing.
440
00:27:12 --> 00:27:17
Actually, we could see -- why
don't we figure out what it is
441
00:27:17 --> 00:27:18
after one?
442
00:27:18 --> 00:27:22
After one time step,
what are the populations at
443
00:27:22 --> 00:27:23
time one?
444
00:27:23 --> 00:27:26
So what happens in one step?
445
00:27:26 --> 00:27:30
You multiply once by that
matrix and, let's see,
446
00:27:30 --> 00:27:35
zero times this column -- so
it's just a thousand times this
447
00:27:35 --> 00:27:38
column, so I think we're getting
200 and 800.
448
00:27:38 --> 00:27:42
So after the first step,
200 people have -- are in
449
00:27:42 --> 00:27:43
California.
450
00:27:43 --> 00:27:48
Now at the following step,
I'll multiply again by this
451
00:27:48 --> 00:27:52
matrix -- more people will move
to California.
452
00:27:52 --> 00:27:54.64
Some people will move back.
453
00:27:54.64 --> 00:27:59
Twenty people will come back
and, the -- the net result will
454
00:27:59 --> 00:28:03
be that the California
population will be above 200 and
455
00:28:03 --> 00:28:08
the Massachusetts below 800 and
they'll still add up to a
456
00:28:08 --> 00:28:09
thousand.
457
00:28:09 --> 00:28:10.3
Okay.
458
00:28:10.3 --> 00:28:11
I do that a few times.
459
00:28:11 --> 00:28:13
I do that 100 times.
460
00:28:13 --> 00:28:15
What's the population?
461
00:28:15 --> 00:28:18
Well, okay, to answer any
question like that,
462
00:28:18 --> 00:28:21
I need the eigenvalues and
eigenvectors,
463
00:28:21 --> 00:28:21
right?
464
00:28:21 --> 00:28:26
As soon as I've -- I've created
an example, but as soon as I
465
00:28:26 --> 00:28:31
want to solve anything,
I have to find eigenvalues and
466
00:28:31 --> 00:28:34
eigenvectors of that matrix.
467
00:28:34 --> 00:28:34.85
Okay.
468
00:28:34.85 --> 00:28:36
So let's do it.
469
00:28:36 --> 00:28:41.09
So there's the matrix .9,
.2, .1, .8 and tell me its
470
00:28:41.09 --> 00:28:42
eigenvalues.
471
00:28:42 --> 00:28:46
Lambda equals -- so tell me one
eigenvalue?
472
00:28:46 --> 00:28:48
One, thanks.
473
00:28:48 --> 00:28:50.66
And tell me the other one.
474
00:28:50.66 --> 00:28:55.15
What's the other eigenvalue --
from the trace or the
475
00:28:55.15 --> 00:28:59
determinant -- from the -- I --
the trace is what -- is,
476
00:28:59 --> 00:29:01
like, easier.
477
00:29:01 --> 00:29:05
So the trace of that matrix is
one point seven.
478
00:29:05 --> 00:29:08
So the other eigenvalue is
point seven.
479
00:29:08 --> 00:29:12
And it -- notice that
it's less than one.
480
00:29:12 --> 00:29:16
And notice that that
determinant is point 72-.02,
481
00:29:16 --> 00:29:18
which is point seven.
482
00:29:18 --> 00:29:18
Right.
483
00:29:18 --> 00:29:19.21
Okay.
484
00:29:19.21 --> 00:29:21
Now to find the eigenvectors.
485
00:29:21 --> 00:29:26
This is -- so that's lambda one
and the eigenvector -- I'll
486
00:29:26 --> 00:29:29
subtract one from the diagonal,
right?
487
00:29:29 --> 00:29:33
So can I do that in light let
-- in light here?
488
00:29:33 --> 00:29:38
Subtract one from the diagonal,
I have minus point one and
489
00:29:38 --> 00:29:42
minus point two,
and of course these are still
490
00:29:42 --> 00:29:42
there.
491
00:29:42 --> 00:29:47
And I'm looking for its --
here's -- here's -- this is
492
00:29:47 --> 00:29:48
going to be x1.
493
00:29:48 --> 00:29:51
It's the null space of A minus
I.
494
00:29:51 --> 00:29:55
Okay, everybody sees that it's
two and one.
495
00:29:55 --> 00:29:56
Okay?
496
00:29:56 --> 00:30:01
And now how about -- so that --
and it -- notice that that
497
00:30:01 --> 00:30:03
eigenvector is positive.
498
00:30:03 --> 00:30:07
And actually,
we can jump to infinity right
499
00:30:07 --> 00:30:08
now.
500
00:30:08 --> 00:30:11
What's the population at
infinity?
501
00:30:11 --> 00:30:16.8
It's a multiple -- this is --
this eigenvector is
502
00:30:16.8 --> 00:30:18
giving the steady state.
503
00:30:18 --> 00:30:22
It's some multiple of this,
and how is that multiple
504
00:30:22 --> 00:30:23
decided?
505
00:30:23 --> 00:30:26
By adding up to a thousand
people.
506
00:30:26 --> 00:30:29
So the steady state,
the c1x1 -- this is the x1,
507
00:30:29 --> 00:30:34
but that adds up to three,
so I really want two -- it's
508
00:30:34 --> 00:30:38
going to be two thirds of a
thousand and one third of a
509
00:30:38 --> 00:30:42
thousand,
making a total of the thousand
510
00:30:42 --> 00:30:42
people.
511
00:30:42 --> 00:30:44
That'll be the steady state.
512
00:30:44 --> 00:30:47.73
That's really all I need to
know at infinity.
513
00:30:47.73 --> 00:30:51
But if I want to know what's
happened after just a finite
514
00:30:51 --> 00:30:54
number like 100 steps,
I'd better find this
515
00:30:54 --> 00:30:55
eigenvector.
516
00:30:55 --> 00:30:59
So can I --
can I look at -- I'll subtract
517
00:30:59 --> 00:31:04
point seven time -- ti- from the
diagonal and I'll get that and
518
00:31:04 --> 00:31:08
I'll look at the null space of
that one and I -- and this is
519
00:31:08 --> 00:31:11
going to give me x2,
now, and what is it?
520
00:31:11 --> 00:31:16
So what's in the null space of
-- that's certainly singular,
521
00:31:16 --> 00:31:20
so I know my calculation is
right,
522
00:31:20 --> 00:31:22
and -- one and minus one.
523
00:31:22 --> 00:31:23
One and minus one.
524
00:31:23 --> 00:31:28
So I'm prepared now to write
down the solution after 100 time
525
00:31:28 --> 00:31:28
steps.
526
00:31:28 --> 00:31:32
The -- the populations after
100 time steps,
527
00:31:32 --> 00:31:32
right?
528
00:31:32 --> 00:31:37
Can -- can we remember the
point one -- the -- the one with
529
00:31:37 --> 00:31:43
this two one eigenvector and the
point seven with the minus
530
00:31:43 --> 00:31:45
one one eigenvector.
531
00:31:45 --> 00:31:50
So I'll -- let me -- I'll just
write it above here.
532
00:31:50 --> 00:31:56
u after k steps is some
multiple of one to the k times
533
00:31:56 --> 00:32:03
the two one eigenvector and some
multiple of point seven to the k
534
00:32:03 --> 00:32:06
times the minus one one
eigenvector.
535
00:32:06 --> 00:32:07
Right?
536
00:32:07 --> 00:32:12
That's --
I -- this is how I take -- how
537
00:32:12 --> 00:32:15
powers of a matrix work.
538
00:32:15 --> 00:32:21.82
When I apply those powers to a
u0, what I -- so it's u0,
539
00:32:21.82 --> 00:32:28
which was zero a thousand --
that has to be corrected k=0.
540
00:32:28 --> 00:32:34
So I'm plugging in k=0 and I
get c1 times two one and c2
541
00:32:34 --> 00:32:37
times minus one one.
542
00:32:37 --> 00:32:40
Two equations,
two constants,
543
00:32:40 --> 00:32:43
certainly independent
eigenvectors,
544
00:32:43 --> 00:32:48
so there's a solution and you
see what it is?
545
00:32:48 --> 00:32:54
Let's see, I guess we already
figured that c1 was a thousand
546
00:32:54 --> 00:32:59
over three, I think -- did we
think that had to
547
00:32:59 --> 00:33:01
be a thousand over three?
548
00:33:01 --> 00:33:07
And maybe c2 would be -- excuse
me, let -- get an eraser -- we
549
00:33:07 --> 00:33:11
can -- I just -- I think we've
-- get it here.
550
00:33:11 --> 00:33:16
c2, we want to get a zero here,
so maybe we need plus two
551
00:33:16 --> 00:33:18
thousand over three.
552
00:33:18 --> 00:33:21.03
I think that has to work.
553
00:33:21.03 --> 00:33:26
Two times a thousand over three
minus two thousand over three,
554
00:33:26 --> 00:33:31
that'll give us the zero and a
thousand over three and the two
555
00:33:31 --> 00:33:35
thousand over three will give us
three thousand over three,
556
00:33:35 --> 00:33:35
the thousand.
557
00:33:35 --> 00:33:40
So this is what we approach --
this part, with the point seven
558
00:33:40 --> 00:33:44.18
to the k-th power is the part
that's disappearing.
559
00:33:44.18 --> 00:33:44
Okay.
560
00:33:44 --> 00:33:48
That's -- that's Markov
matrices.
561
00:33:48 --> 00:33:53
That's an example of where they
come from, from modeling
562
00:33:53 --> 00:33:59
movement of people with no gain
or loss, with total -- total
563
00:33:59 --> 00:34:01
count conserved.
564
00:34:01 --> 00:34:02
Okay.
565
00:34:02 --> 00:34:07
I -- just if I can add one more
comment,
566
00:34:07 --> 00:34:11
because you'll see Markov
matrices in electrical
567
00:34:11 --> 00:34:15.94
engineering courses and often
you'll see them -- sorry,
568
00:34:15.94 --> 00:34:18
here's my little comment.
569
00:34:18 --> 00:34:22
Sometimes -- in a lot of
applications they prefer to work
570
00:34:22 --> 00:34:24
with row vectors.
571
00:34:24 --> 00:34:28
So they -- instead of -- this
was natural for us,
572
00:34:28 --> 00:34:29
right?
573
00:34:29 --> 00:34:33
For all the eigenvectors to be
column vectors.
574
00:34:33 --> 00:34:37
So our columns added to one in
the Markov matrix.
575
00:34:37 --> 00:34:41
Just so you don't think,
well, what -- what's going on?
576
00:34:41 --> 00:34:47
If we work with row vectors and
we multiply vector times matrix
577
00:34:47 --> 00:34:52
-- so we're multiplying from
the left -- then it'll be the
578
00:34:52 --> 00:34:57
then we'll be using the
transpose of -- of this matrix
579
00:34:57 --> 00:35:00
and it'll be the rows that add
to one.
580
00:35:00 --> 00:35:04
So in other textbooks,
you'll see -- instead of col-
581
00:35:04 --> 00:35:08
columns adding to one,
you'll see rows add to one.
582
00:35:08 --> 00:35:08
Okay.
583
00:35:08 --> 00:35:10
Fine.
584
00:35:10 --> 00:35:14
Okay, that's what I wanted to
say about Markov,
585
00:35:14 --> 00:35:18
now I want to say something
about projections and even
586
00:35:18 --> 00:35:22
leading in -- a little into
Fourier series.
587
00:35:22 --> 00:35:25
Because -- but before any
Fourier stuff,
588
00:35:25 --> 00:35:29.37
let me make a comment about
projections.
589
00:35:29.37 --> 00:35:35
This -- so this is a comment
about projections onto --
590
00:35:35 --> 00:35:38
with an orthonormal basis.
591
00:35:38 --> 00:35:44
So, of course,
the basis vectors are q1 up to
592
00:35:44 --> 00:35:44
qn.
593
00:35:44 --> 00:35:45
Okay.
594
00:35:45 --> 00:35:48.11
I have a vector b.
595
00:35:48.11 --> 00:35:55
Let -- let me imagine -- let me
imagine this is a basis.
596
00:35:55 --> 00:35:59.78
Let -- let's say I'm in n by n.
597
00:35:59.78 --> 00:36:03
I'm -- I've got,
eh,
598
00:36:03 --> 00:36:08
n orthonormal vectors,
I'm in n dimensional space so
599
00:36:08 --> 00:36:15.14
they're a complete -- they're a
basis -- any vector v could be
600
00:36:15.14 --> 00:36:17
expanded in this basis.
601
00:36:17 --> 00:36:22
So any vector v is some
combination, some amount of q1
602
00:36:22 --> 00:36:28
plus some amount of q2 plus some
amount of qn.
603
00:36:28 --> 00:36:30
So -- so any v.
604
00:36:30 --> 00:36:35
I just want you to tell me what
those amounts are.
605
00:36:35 --> 00:36:40
What are x1 -- what's x1,
for example?
606
00:36:40 --> 00:36:43
So I'm looking for the
expansion.
607
00:36:43 --> 00:36:48
This is -- this is really our
projection.
608
00:36:48 --> 00:36:55
I could -- I could really use
the word expansion.
609
00:36:55 --> 00:36:59
I'm expanding the vector in the
basis.
610
00:36:59 --> 00:37:05
And the special thing about the
basis is that it's orthonormal.
611
00:37:05 --> 00:37:10
So that should give me a
special formula for the answer,
612
00:37:10 --> 00:37:12
for the coefficients.
613
00:37:12 --> 00:37:14
So how do I get x1?
614
00:37:14 --> 00:37:17
What -- what's a formula for
x1?
615
00:37:17 --> 00:37:23
I could -- I can go through the
projection -- the Q transpose
616
00:37:23 --> 00:37:28
Q, all that -- normal equations,
but -- and I'll get -- I'll
617
00:37:28 --> 00:37:33
come out with this nice answer
that I think I can see right
618
00:37:33 --> 00:37:34
away.
619
00:37:34 --> 00:37:39
How can I pick -- get a hold of
x1 and get these other x-s out
620
00:37:39 --> 00:37:41
of the equation?
621
00:37:41 --> 00:37:44
So how can I get a nice,
simple
622
00:37:44 --> 00:37:46
formula for x1?
623
00:37:46 --> 00:37:50
And then we want to see,
sure, we knew that all the
624
00:37:50 --> 00:37:51
time.
625
00:37:51 --> 00:37:51
Okay.
626
00:37:51 --> 00:37:52
So what's x1?
627
00:37:52 --> 00:37:58.01
The good way is take the inner
product of everything with q1.
628
00:37:58.01 --> 00:38:02
Take the inner product of that
whole equation,
629
00:38:02 --> 00:38:03
every term, with q1.
630
00:38:03 --> 00:38:06
What will happen to that last
term?
631
00:38:06 --> 00:38:12.2
The inner product --
when -- if I take the dot
632
00:38:12.2 --> 00:38:15
product with q1 I get zero,
right?
633
00:38:15 --> 00:38:18
Because this basis was
orthonormal.
634
00:38:18 --> 00:38:22
If I take the dot product with
q2 I get zero.
635
00:38:22 --> 00:38:26
If I take the dot product with
q1 I get one.
636
00:38:26 --> 00:38:30
So that tells me what x1 is.
q1 transpose v,
637
00:38:30 --> 00:38:36
that's taking the dot product,
is x1 times q1 transpose q1
638
00:38:36 --> 00:38:38
plus a bunch of zeroes.
639
00:38:38 --> 00:38:42
And this is a one,
so I can forget that.
640
00:38:42 --> 00:38:44
I get x1 immediately.
641
00:38:44 --> 00:38:49
So -- do you see what I'm
saying -- is that I have an
642
00:38:49 --> 00:38:54.05
orthonormal basis,
then the coefficient that I
643
00:38:54.05 --> 00:38:58
need for
each basis vector is a cinch to
644
00:38:58 --> 00:38:58
find.
645
00:38:58 --> 00:39:02
Let me -- let me just -- I have
to put this into matrix
646
00:39:02 --> 00:39:06
language, too,
so you'll see it there also.
647
00:39:06 --> 00:39:10
If I write that first equation
in matrix language,
648
00:39:10 --> 00:39:11
what -- what is it?
649
00:39:11 --> 00:39:15
I'm writing --
in matrix language,
650
00:39:15 --> 00:39:21
this equation says I'm taking
these columns -- are -- are you
651
00:39:21 --> 00:39:23
guys good at this now?
652
00:39:23 --> 00:39:29
I'm taking those columns times
the Xs and getting V,
653
00:39:29 --> 00:39:29.83
right?
654
00:39:29.83 --> 00:39:32
That's the matrix form.
655
00:39:32 --> 00:39:34
Okay, that's the matrix Q.
656
00:39:34 --> 00:39:35
Qx is v.
657
00:39:35 --> 00:39:40.38
What's the solution
to that equation?
658
00:39:40.38 --> 00:39:44
It's -- of course,
it's x equal Q inverse v.
659
00:39:44 --> 00:39:48
So x is Q inverse v,
but what's the point?
660
00:39:48 --> 00:39:53
Q inverse in this case is going
to -- is simple.
661
00:39:53 --> 00:39:57
I don't have to work to invert
this matrix Q,
662
00:39:57 --> 00:40:03
because of the fact that the --
these columns are orthonormal,
663
00:40:03 --> 00:40:06.39
I know the inverse to that.
664
00:40:06.39 --> 00:40:08
And it is Q transpose.
665
00:40:08 --> 00:40:12
When you see a Q,
a square matrix with that
666
00:40:12 --> 00:40:16
letter Q, the -- that just
triggers -- Q inverse is the
667
00:40:16 --> 00:40:18
same as Q transpose.
668
00:40:18 --> 00:40:23
So the first component,
then -- the first component of
669
00:40:23 --> 00:40:29
x is the first row times
v, and what's that?
670
00:40:29 --> 00:40:35
The first component of this
answer is the first row of Q
671
00:40:35 --> 00:40:37.23
transpose.
672
00:40:37.23 --> 00:40:43
That's just -- that's just q1
transpose times v.
673
00:40:43 --> 00:40:47
So that's what we concluded
here, too.
674
00:40:47 --> 00:40:48
Okay.
675
00:40:48 --> 00:40:52
So -- so nothing Fourier here.
676
00:40:52 --> 00:40:57
The -- the key ingredient here
was
677
00:40:57 --> 00:41:01.23
that the q-s are orthonormal.
678
00:41:01.23 --> 00:41:06
And now that's what Fourier
series are built on.
679
00:41:06 --> 00:41:12
So now, in the remaining time,
let me say something about
680
00:41:12 --> 00:41:14
Fourier series.
681
00:41:14 --> 00:41:15
Okay.
682
00:41:15 --> 00:41:21
So Fourier series is -- well,
we've got a function f of x.
683
00:41:21 --> 00:41:28
And we want to write it as a
combination of -- maybe
684
00:41:28 --> 00:41:31
it has a constant term.
685
00:41:31 --> 00:41:34
And then it has some cos(x) in
it.
686
00:41:34 --> 00:41:38
And it has some sin(x) in it.
687
00:41:38 --> 00:41:41
And it has some cos(2x) in it.
688
00:41:41 --> 00:41:46
And a -- and some sin(2x),
and forever.
689
00:41:46 --> 00:41:51
So what's -- what's the
difference between this type
690
00:41:51 --> 00:41:55.26
problem and the one above it?
691
00:41:55.26 --> 00:42:02
This one's infinite,
but the key property of things
692
00:42:02 --> 00:42:07.83
being orthogonal is still true
for sines and cosines,
693
00:42:07.83 --> 00:42:13.43
so it's the property that makes
Fourier series work.
694
00:42:13.43 --> 00:42:17
So that's called a Fourier
series.
695
00:42:17 --> 00:42:19
Better write his name up.
696
00:42:19 --> 00:42:22
Fourier series.
697
00:42:22 --> 00:42:26
So it was Joseph Fourier who
realized that,
698
00:42:26 --> 00:42:28
hey, I could work in function
space.
699
00:42:28 --> 00:42:33
Instead of a vector v,
I could have a function f of x.
700
00:42:33 --> 00:42:37
Instead of orthogonal vectors,
q1, q2 , q3,
701
00:42:37 --> 00:42:41
I could have orthogonal
functions, the constant,
702
00:42:41 --> 00:42:45
the cos(x), the sin(x),
the s- cos(2x),
703
00:42:45 --> 00:42:47
but infinitely many of them.
704
00:42:47 --> 00:42:51
I need infinitely many,
because my space is infinite
705
00:42:51 --> 00:42:52
dimensional.
706
00:42:52 --> 00:42:55.39
So this is, like,
the moment in which we leave
707
00:42:55.39 --> 00:42:59
finite dimensional vector spaces
and go to infinite dimensional
708
00:42:59 --> 00:43:04
vector spaces and our basis --
so the vectors are now
709
00:43:04 --> 00:43:09
functions -- and of course,
there are so many functions
710
00:43:09 --> 00:43:13
that it's -- that we've got an
infin- infinite dimensional
711
00:43:13 --> 00:43:17
space -- and the basis vectors
are functions,
712
00:43:17 --> 00:43:19.79
too.
a0, the constant function one
713
00:43:19.79 --> 00:43:24
-- so my basis is one cos(x),
sin(x), cos(2x),
714
00:43:24 --> 00:43:26
sin(2x) and so on.
715
00:43:26 --> 00:43:32
And the reason Fourier series
is a success is that those are
716
00:43:32 --> 00:43:33
orthogonal.
717
00:43:33 --> 00:43:33
Okay.
718
00:43:33 --> 00:43:37
Now what do I mean by
orthogonal?
719
00:43:37 --> 00:43:42
I know what it means for two
vectors to be orthogonal -- y
720
00:43:42 --> 00:43:46
transpose x equals zero,
right?
721
00:43:46 --> 00:43:48
Dot product equals zero.
722
00:43:48 --> 00:43:51.72
But what's the dot product of
functions?
723
00:43:51.72 --> 00:43:56
I'm claiming that whatever it
is, the dot product -- or we
724
00:43:56 --> 00:43:59
would more likely use the word
inner product of,
725
00:43:59 --> 00:44:02
say, cos(x) with sin(x) is
zero.
726
00:44:02 --> 00:44:04
And cos(x) with cos(2x),
also zero.
727
00:44:04 --> 00:44:08
So I -- let me tell you what I
mean
728
00:44:08 --> 00:44:12
by that, by that dot product.
729
00:44:12 --> 00:44:17
Well, how do I compute a dot
product?
730
00:44:17 --> 00:44:24
So, let's just remember for
vectors v trans- v transpose w
731
00:44:24 --> 00:44:32
for vectors, so this was
vectors, v transpose w was v1w1
732
00:44:32 --> 00:44:33.59
+...+vnwn.
733
00:44:33.59 --> 00:44:34
Okay.
734
00:44:34 --> 00:44:36
Now functions.
735
00:44:36 --> 00:44:42
Now I have two functions,
let's call them f and g.
736
00:44:42 --> 00:44:44
What's with them now?
737
00:44:44 --> 00:44:49
The vectors had n components,
but the functions have a whole,
738
00:44:49 --> 00:44:50
like, continuum.
739
00:44:50 --> 00:44:54
To graph the function,
I just don't have n points,
740
00:44:54 --> 00:44:56
I've got this whole graph.
741
00:44:56 --> 00:45:01
So I have functions -- I'm
really trying to ask you what's
742
00:45:01 --> 00:45:07
the inner product of this
function f with another function
743
00:45:07 --> 00:45:07
g?
744
00:45:07 --> 00:45:13
And I want to make it parallel
to this the best I can.
745
00:45:13 --> 00:45:18
So the best parallel is to
multiply f (x) times g(x) at
746
00:45:18 --> 00:45:23.23
every x -- and here I just had n
multiplications,
747
00:45:23.23 --> 00:45:27
but here I'm going to have a
whole range of x-s,
748
00:45:27 --> 00:45:31
and here
I added the results.
749
00:45:31 --> 00:45:33.48
What do I do here?
750
00:45:33.48 --> 00:45:38
So what's the analog of
addition when you have -- when
751
00:45:38 --> 00:45:40
you're in a continuum?
752
00:45:40 --> 00:45:42
It's integration.
753
00:45:42 --> 00:45:47
So that the -- the dot product
of two functions will be the
754
00:45:47 --> 00:45:51
integral of those functions,
dx.
755
00:45:51 --> 00:45:56
Now I have to say -- say,
well, what are the limits of
756
00:45:56 --> 00:45:57
integration?
757
00:45:57 --> 00:46:02.01
And for this Fourier series,
this function f(x) -- if I'm
758
00:46:02.01 --> 00:46:07
going to have -- if that right
hand side is going to be f(x),
759
00:46:07 --> 00:46:10
that function that I'm seeing
on the right,
760
00:46:10 --> 00:46:13
all those
sines and cosines,
761
00:46:13 --> 00:46:17
they're all periodic,
with -- with period two pi.
762
00:46:17 --> 00:46:20
So -- so that's what f(x) had
better be.
763
00:46:20 --> 00:46:23
So I'll integrate from zero to
two pi.
764
00:46:23 --> 00:46:28
My -- all -- everything -- is
on the interval zero two pi
765
00:46:28 --> 00:46:35
now, because if I'm going to
use these sines and cosines,
766
00:46:35 --> 00:46:39
then f(x) is equal to f(x+2pi).
767
00:46:39 --> 00:46:44
This is periodic -- periodic
functions.
768
00:46:44 --> 00:46:45
Okay.
769
00:46:45 --> 00:46:51
So now I know what -- I've got
all the right words now.
770
00:46:51 --> 00:47:00
I've got a vector space,
but the vectors are functions.
771
00:47:00 --> 00:47:04
I've got inner products and --
and the inner product gives a
772
00:47:04 --> 00:47:06
number, all right.
773
00:47:06 --> 00:47:09
It just happens to be an
integral instead of a sum.
774
00:47:09 --> 00:47:14
I've got -- and that -- then I
have the idea of orthogonality
775
00:47:14 --> 00:47:18
-- because, actually,
just -- let's just check.
776
00:47:18 --> 00:47:25
Orthogonality -- if I take the
integral -- s- I -- let me do
777
00:47:25 --> 00:47:32
sin(x) times cos(x) -- sin(x)
times cos(x) dx from zero to two
778
00:47:32 --> 00:47:35
pi -- I think we get zero.
779
00:47:35 --> 00:47:41
That's the differential of
that, so it would be one half
780
00:47:41 --> 00:47:45.29
sine x squared,
was that right?
781
00:47:45.29 --> 00:47:50
Between zero and two pi --
and, of course,
782
00:47:50 --> 00:47:52
we get zero.
783
00:47:52 --> 00:47:57.65
And the same would be true with
a little more -- some trig
784
00:47:57.65 --> 00:48:02
identities to help us out -- of
every other pair.
785
00:48:02 --> 00:48:07.81
So we have now an orthonormal
infinite basis for function
786
00:48:07.81 --> 00:48:13
space, and all we want to do is
express a function in that
787
00:48:13 --> 00:48:13
basis.
788
00:48:13 --> 00:48:18
And so I --
the end of my lecture is,
789
00:48:18 --> 00:48:19
okay, what is a1?
790
00:48:19 --> 00:48:24
What's the coefficient -- how
much cos(x) is there in a
791
00:48:24 --> 00:48:27
function compared to the other
harmonics?
792
00:48:27 --> 00:48:31
How much constant is in that
function?
793
00:48:31 --> 00:48:35
That'll -- that would be an
easy question.
794
00:48:35 --> 00:48:39
The answer a0 will come out to
be the average value of f.
795
00:48:39 --> 00:48:42
That's the amount of the
constant that's in there,
796
00:48:42 --> 00:48:43
its average value.
797
00:48:43 --> 00:48:45
But let's take a1 as more
typical.
798
00:48:45 --> 00:48:49
How will I get -- here's the
end of the lecture,
799
00:48:49 --> 00:48:50
then -- how do I get a1?
800
00:48:50 --> 00:48:53
The first Fourier coefficient.
801
00:48:53 --> 00:48:54
Okay.
802
00:48:54 --> 00:48:57
I do just as I did in the
vector case.
803
00:48:57 --> 00:49:03
I take the inner product of
everything with cos(x) Take the
804
00:49:03 --> 00:49:06.96
inner product of everything with
cos(x).
805
00:49:06.96 --> 00:49:12
Then on the left -- on the left
I have -- the inner product is
806
00:49:12 --> 00:49:16.92
the integral of
f(x) times cos(x) cx.
807
00:49:16.92 --> 00:49:19
And on the right,
what do I have?
808
00:49:19 --> 00:49:24
When I -- so what I -- when I
say take the inner product with
809
00:49:24 --> 00:49:27
cos(x), let me put it in
ordinary calculus words.
810
00:49:27 --> 00:49:30
Multiply by cos(x) and
integrate.
811
00:49:30 --> 00:49:32
That's what inner products are.
812
00:49:32 --> 00:49:36
So if I multiply that whole
thing by
813
00:49:36 --> 00:49:41
cos(x) and I integrate,
I get a whole lot of zeroes.
814
00:49:41 --> 00:49:45
The only thing that survives is
that term.
815
00:49:45 --> 00:49:47
All the others disappear.
816
00:49:47 --> 00:49:53
So -- and that term is a1 times
the integral of cos(x) squared
817
00:49:53 --> 00:49:58
dx zero to 2pi equals -- so this
was the left side and this is
818
00:49:58 --> 00:50:03.2
all that's left on
the right-hand side.
819
00:50:03.2 --> 00:50:08
And this is not zero of course,
because it's the length of the
820
00:50:08 --> 00:50:13
function squared,
it's the inner product with
821
00:50:13 --> 00:50:18
itself, and -- and a simple
calculation gives that answer to
822
00:50:18 --> 00:50:19
be pi.
823
00:50:19 --> 00:50:23.95
So that's an easy integral and
it turns out to be pi,
824
00:50:23.95 --> 00:50:30
so that a1 is one over pi times
there -- times this integral.
825
00:50:30 --> 00:50:34
So there is,
actually -- that's Euler's
826
00:50:34 --> 00:50:39
famous formula for the -- or
maybe Fourier found it -- for
827
00:50:39 --> 00:50:43
the coefficients in a Fourier
series.
828
00:50:43 --> 00:50:48.8
And you see that it's exactly
an expansion in an orthonormal
829
00:50:48.8 --> 00:50:49
basis.
830
00:50:49 --> 00:50:51
Okay, thanks.
831
00:50:51 --> 00:50:57
So I'll do a quiz review on
Monday and then the quiz itself
832
00:50:57 --> 00:50:59
in Walker on Wednesday.
833
00:50:59 --> 00:51:01
Okay, see you Monday.
834
00:51:01 --> 00:51:04
Thanks.