1
00:00:06 --> 00:00:09
-- two, one and -- okay.

2
00:00:09 --> 00:00:16
Here is a lecture on the
applications of eigenvalues and,

3
00:00:16 --> 00:00:21.55
if I can -- so that will be
Markov matrices.

4
00:00:21.55 --> 00:00:28
I'll tell you what a Markov
matrix is, so this matrix A will

5
00:00:28 --> 00:00:35
be a Markov matrix and I'll
explain how they come in

6
00:00:35 --> 00:00:38
applications.

7
00:00:38 --> 00:00:43
And -- and then if I have time,
I would like to say a little

8
00:00:43 --> 00:00:48
bit about Fourier series,
which is a fantastic

9
00:00:48 --> 00:00:51
application of the projection
chapter.

10
00:00:51 --> 00:00:52
Okay.

11
00:00:52 --> 00:00:54
What's a Markov matrix?

12
00:00:54 --> 00:00:58.81
Can I just write down a typical
Markov matrix,

13
00:00:58.81 --> 00:01:01
say .1, .2, .7,
.01,

14
00:01:01 --> 00:01:05
.99 0, let's say,
.3, .3, .4.

15
00:01:05 --> 00:01:05
Okay.

16
00:01:05 --> 00:01:11
There's a -- a totally just
invented Markov matrix.

17
00:01:11 --> 00:01:14
What makes it a Markov matrix?

18
00:01:14 --> 00:01:19
Two properties that this --
this matrix has.

19
00:01:19 --> 00:01:27
So two properties are -- one,
every entry is greater equal

20
00:01:27 --> 00:01:28
zero.

21
00:01:28 --> 00:01:32
All entries greater than or
equal to zero.

22
00:01:32 --> 00:01:37
And, of course,
when I square the matrix,

23
00:01:37 --> 00:01:42
the entries will still be
greater/equal zero.

24
00:01:42 --> 00:01:48
I'm going to be interested in
the powers of this matrix.

25
00:01:48 --> 00:01:53
And this property,
of course, is going to -- stay

26
00:01:53 --> 00:01:54
there.

27
00:01:54 --> 00:01:59.05
It -- really Markov matrices
you'll see are connected to

28
00:01:59.05 --> 00:02:02.74
probability ideas and
probabilities are never

29
00:02:02.74 --> 00:02:03
negative.

30
00:02:03 --> 00:02:08
The other property -- do you
see the other property in there?

31
00:02:08 --> 00:02:14
If I add down the columns,
what answer do I get?

32
00:02:14 --> 00:02:14.81
One.

33
00:02:14.81 --> 00:02:18
So all columns add to one.

34
00:02:18 --> 00:02:21.21
All columns add to one.

35
00:02:21.21 --> 00:02:29
And actually when I square the
matrix, that will be true again.

36
00:02:29 --> 00:02:36
So that the powers of my matrix
are all Markov matrices,

37
00:02:36 --> 00:02:43
and I'm interested in,
always, the eigenvalues and the

38
00:02:43 --> 00:02:46.55
eigenvectors.

39
00:02:46.55 --> 00:02:49
And this question of steady
state will come up.

40
00:02:49 --> 00:02:54
You remember we had steady
state for differential equations

41
00:02:54 --> 00:02:54
last time?

42
00:02:54 --> 00:02:58
When -- what was the steady
state -- what was the

43
00:02:58 --> 00:02:59
eigenvalue?

44
00:02:59 --> 00:03:03
What was the eigenvalue in the
differential equation case that

45
00:03:03 --> 00:03:05
led to a steady state?

46
00:03:05 --> 00:03:08
It was lambda equals zero.

47
00:03:08 --> 00:03:12
When -- you remember that we
did an example and one of the

48
00:03:12 --> 00:03:17
eigenvalues was lambda equals
zero, and that -- so then we had

49
00:03:17 --> 00:03:22
an E to the zero T,
a constant one -- as time went

50
00:03:22 --> 00:03:24
on, there that thing stayed
steady.

51
00:03:24 --> 00:03:30
Now what -- in the powers case,
it's not a zero eigenvalue.

52
00:03:30 --> 00:03:34
Actually with powers of a
matrix, a zero eigenvalue,

53
00:03:34 --> 00:03:37
that part is going to die right
away.

54
00:03:37 --> 00:03:40
It's an eigenvalue of one
that's all important.

55
00:03:40 --> 00:03:45
So this steady state will
correspond -- will be totally

56
00:03:45 --> 00:03:49
connected with an eigenvalue of
one and its eigenvector.

57
00:03:49 --> 00:03:52
In fact, the steady state will
be

58
00:03:52 --> 00:03:56
the eigenvector for that
eigenvalue.

59
00:03:56 --> 00:03:56
Okay.

60
00:03:56 --> 00:03:58
So that's what's coming.

61
00:03:58 --> 00:04:02
Now, for some reason then that
we have to see,

62
00:04:02 --> 00:04:06
this matrix has an eigenvalue
of one.

63
00:04:06 --> 00:04:10
This property,
that the columns all add to one

64
00:04:10 --> 00:04:15
-- turns out -- guarantees that
one is an eigenvalue,

65
00:04:15 --> 00:04:20
so that you can
actually find the eigenvalue --

66
00:04:20 --> 00:04:24
find that eigenvalue of a Markov
matrix without computing any

67
00:04:24 --> 00:04:28
determinants of A minus lambda I
-- that matrix will have an

68
00:04:28 --> 00:04:31
eigenvalue of one,
and we want to see why.

69
00:04:31 --> 00:04:36
And then the other thing is --
so the key points -- let me --

70
00:04:36 --> 00:04:39
let me write these underneath.

71
00:04:39 --> 00:04:46.11
The key points are -- the key
points are lambda equal one is

72
00:04:46.11 --> 00:04:47
an eigenvalue.

73
00:04:47 --> 00:04:53
I'll add in a little -- an
additional -- well,

74
00:04:53 --> 00:04:58.14
a thing about eigenvalues --
key point two,

75
00:04:58.14 --> 00:05:04
the other eigenval- values --
all other eigenvalues are,

76
00:05:04 --> 00:05:08
in magnitude,
smaller than one -- in absolute

77
00:05:08 --> 00:05:11
value, smaller than one.

78
00:05:11 --> 00:05:16
Well, there could be some
exceptional case when -- when an

79
00:05:16 --> 00:05:21
eigen -- another eigenvalue
might have magnitude equal one.

80
00:05:21 --> 00:05:25
It never has an eigenvalue
larger than one.

81
00:05:25 --> 00:05:30
So these two facts --
somehow we ought to -- linear

82
00:05:30 --> 00:05:32
algebra ought to tell us.

83
00:05:32 --> 00:05:36
And then, of course,
linear algebra is going to tell

84
00:05:36 --> 00:05:40
us what the -- what's -- what
happens if I take -- if -- you

85
00:05:40 --> 00:05:45
remember when I solve -- when I
multiply by A time after time

86
00:05:45 --> 00:05:51
the K-th thing is A to the K u0
and I'm asking what's special

87
00:05:51 --> 00:05:57
about this -- these powers of A,
and very likely the quiz will

88
00:05:57 --> 00:06:03
have a problem to computer s- to
computer some powers of A or --

89
00:06:03 --> 00:06:06
or applied to an initial vector.

90
00:06:06 --> 00:06:09
So, you remember the general
form?

91
00:06:09 --> 00:06:13
The general form is that
there's

92
00:06:13 --> 00:06:17
some amount of the first
eigenvalue to the K-th power

93
00:06:17 --> 00:06:21
times the first eigenvector,
and another amount of the

94
00:06:21 --> 00:06:26
second eigenvalue to the K-th
power times the second

95
00:06:26 --> 00:06:27
eigenvector and so on.

96
00:06:27 --> 00:06:32
A -- just -- my conscience
always makes me say at least

97
00:06:32 --> 00:06:36
once per lecture that this
requires

98
00:06:36 --> 00:06:40.92
a complete set of eigenvectors,
otherwise we might not be able

99
00:06:40.92 --> 00:06:45
to expand u0 in the eigenvectors
and we couldn't get started.

100
00:06:45 --> 00:06:48
But once we're started with u0
when K is zero,

101
00:06:48 --> 00:06:51
then every A brings in these
lambdas.

102
00:06:51 --> 00:06:56
And now you can see what the
steady state is going to be.

103
00:06:56 --> 00:07:01
If lambda one is one -- so
lambda one equals one to the

104
00:07:01 --> 00:07:06
K-th power and these other
eigenvalues are smaller than one

105
00:07:06 --> 00:07:12
-- so I've sort of scratched
over the equation there to -- we

106
00:07:12 --> 00:07:16
had this term,
but what happens to this term

107
00:07:16 --> 00:07:20
-- if the lambda's smaller than
one,

108
00:07:20 --> 00:07:24
then the -- when -- as we take
powers, as we iterate as we --

109
00:07:24 --> 00:07:27.38
as we go forward in time,
this goes to zero,

110
00:07:27.38 --> 00:07:27
right?

111
00:07:27 --> 00:07:31
Can I just -- having scratched
over it, I might as well scratch

112
00:07:31 --> 00:07:32
further.

113
00:07:32 --> 00:07:36
That term and all the other
terms are going to zero because

114
00:07:36 --> 00:07:42
all the other eigenvalues
are smaller than one and the

115
00:07:42 --> 00:07:49
steady state that we're
approaching is just -- whatever

116
00:07:49 --> 00:07:57
there was -- this was -- this
was the -- this is the x1 part

117
00:07:57 --> 00:08:05
of un- of the initial condition
u0 -- is the steady state.

118
00:08:05 --> 00:08:09
This much we know from general
-- from -- you know,

119
00:08:09 --> 00:08:10
what we've already done.

120
00:08:10 --> 00:08:14.76
So I want to see why -- let's
at least see number one,

121
00:08:14.76 --> 00:08:16
why one is an eigenvalue.

122
00:08:16 --> 00:08:21.1
And then there's actually -- in
this chapter we're interested

123
00:08:21.1 --> 00:08:25
not only in eigenvalues,
but also eigenvectors.

124
00:08:25 --> 00:08:30
And there's something special
about the eigenvector.

125
00:08:30 --> 00:08:33
Let me write down what that is.

126
00:08:33 --> 00:08:39
The eigenvector x1 -- x1 is the
eigenvector and all its

127
00:08:39 --> 00:08:44
components are positive,
so the steady state is

128
00:08:44 --> 00:08:47
positive, if the start was.

129
00:08:47 --> 00:08:52
If the start was -- so -- well,
actually,

130
00:08:52 --> 00:08:57
in general, I -- this might
have a -- might have some

131
00:08:57 --> 00:09:02.67
component zero always,
but no negative components in

132
00:09:02.67 --> 00:09:04
that eigenvector.

133
00:09:04 --> 00:09:04
Okay.

134
00:09:04 --> 00:09:07
Can I come to that point?

135
00:09:07 --> 00:09:12
How can I look at that matrix
-- so that was just an example.

136
00:09:12 --> 00:09:19
How could I be sure -- how can
I see that a matrix --

137
00:09:19 --> 00:09:24
if the columns add to zero --
add to one, sorry -- if the

138
00:09:24 --> 00:09:29
columns add to one,
this property means that lambda

139
00:09:29 --> 00:09:32
equal one is an eigenvalue.

140
00:09:32 --> 00:09:32
Okay.

141
00:09:32 --> 00:09:35
So let's just think that
through.

142
00:09:35 --> 00:09:41.87
What I saying about -- let me
ca- let me look at A,

143
00:09:41.87 --> 00:09:47.34
and if I believe that one is an
eigenvalue, then I should be

144
00:09:47.34 --> 00:09:52
able to subtract off one times
the identity and then I would

145
00:09:52 --> 00:09:57
get a matrix that's,
what, -.9, -.01 and -.6 -- wh-

146
00:09:57 --> 00:10:02
I took the ones away and the
other parts, of course,

147
00:10:02 --> 00:10:08
are still what they were,
and this is still .2 and .7 and

148
00:10:08 --> 00:10:13
-- okay, what's -- what's up
with this matrix now?

149
00:10:13 --> 00:10:18
I've shifted the matrix,
this Markov matrix by one,

150
00:10:18 --> 00:10:23.2
by the identity,
and what do I want to prove?

151
00:10:23.2 --> 00:10:28
I -- what is it that I believe
this matrix -- about this

152
00:10:28 --> 00:10:30
matrix?

153
00:10:30 --> 00:10:32
I believe it's singular.

154
00:10:32 --> 00:10:36
Singular will -- if A minus I
is singular, that tells me that

155
00:10:36 --> 00:10:38
one is an eigenvalue,
right?

156
00:10:38 --> 00:10:42
The eigenvalues are the numbers
that I subtract off -- the

157
00:10:42 --> 00:10:47
shifts -- the numbers that I
subtract from the diagonal -- to

158
00:10:47 --> 00:10:48
make it singular.

159
00:10:48 --> 00:10:51
Now why is that matrix
singular?

160
00:10:51 --> 00:10:56
I -- we could compute its
determinant, but we want to see

161
00:10:56 --> 00:11:00
a reason that would work for
every Markov matrix not just

162
00:11:00 --> 00:11:03
this particular random example.

163
00:11:03 --> 00:11:05
So what is it about that
matrix?

164
00:11:05 --> 00:11:11
Well, I guess you could look at
its columns now -- what

165
00:11:11 --> 00:11:14
do they add up to?

166
00:11:14 --> 00:11:15
Zero.

167
00:11:15 --> 00:11:24
The columns add to zero,
so all columns -- let me put

168
00:11:24 --> 00:11:33
all columns now of -- of -- of A
minus I add to zero,

169
00:11:33 --> 00:11:42
and then I want to realize that
this means A minus I is

170
00:11:42 --> 00:11:44
singular.

171
00:11:44 --> 00:11:44
Okay.

172
00:11:44 --> 00:11:45
Why?

173
00:11:45 --> 00:11:52
So I could I --
you know, that could be a quiz

174
00:11:52 --> 00:11:55
question, a sort of theoretical
quiz question.

175
00:11:55 --> 00:11:59
If I give you a matrix and I
tell you all its columns add to

176
00:11:59 --> 00:12:02
zero, give me a reason,
because it is true,

177
00:12:02 --> 00:12:04
that the matrix is singular.

178
00:12:04 --> 00:12:04.35
Okay.

179
00:12:04.35 --> 00:12:07
I guess actually -- now what --
I think of -- you know,

180
00:12:07 --> 00:12:11
I'm thinking of two or three
ways to

181
00:12:11 --> 00:12:12.06
see that.

182
00:12:12.06 --> 00:12:13
How would you do it?

183
00:12:13 --> 00:12:18
We don't want to take its
determinant somehow.

184
00:12:18 --> 00:12:24
For the matrix to be singular,
well, it means that these three

185
00:12:24 --> 00:12:26
columns are dependent,
right?

186
00:12:26 --> 00:12:32
The determinant will be zero
when those three columns are

187
00:12:32 --> 00:12:33.12
dependent.

188
00:12:33.12 --> 00:12:37
You see, we're --
we're at a point in this

189
00:12:37 --> 00:12:40
course, now, where we have
several ways to look at an idea.

190
00:12:40 --> 00:12:44
We can take the determinant --
here we don't want to.

191
00:12:44 --> 00:12:47.39
B- but we met singular before
that -- those columns are

192
00:12:47.39 --> 00:12:48
dependent.

193
00:12:48 --> 00:12:50
So how do I see that those
columns are dependent?

194
00:12:50 --> 00:12:52
They all add to zero.

195
00:12:52 --> 00:12:56
Let's see, whew -- well,
oh, actually,

196
00:12:56 --> 00:13:03
what -- another thing I know is
that the -- I would like to be

197
00:13:03 --> 00:13:07
able to show is that the rows
are dependent.

198
00:13:07 --> 00:13:10
Maybe that's easier.

199
00:13:10 --> 00:13:15
If I know that all the columns
add to zero, that's my

200
00:13:15 --> 00:13:21
information, how do I see that
those three rows

201
00:13:21 --> 00:13:23
are linearly dependent?

202
00:13:23 --> 00:13:28
What -- what combination of
those rows gives the zero row?

203
00:13:28 --> 00:13:33
How -- how could I combine
those three rows -- those three

204
00:13:33 --> 00:13:36
row vectors to produce the zero
row vector?

205
00:13:36 --> 00:13:40
And that would tell me those
rows are dependent,

206
00:13:40 --> 00:13:44
therefore the columns are
dependent,

207
00:13:44 --> 00:13:47
the matrix is singular,
the determinant is zero --

208
00:13:47 --> 00:13:49
well, you see it.

209
00:13:49 --> 00:13:50
I just add the rows.

210
00:13:50 --> 00:13:54
One times that row plus one
times that row plus one times

211
00:13:54 --> 00:13:57.27
that row -- it's the zero row.

212
00:13:57.27 --> 00:13:59
The rows are dependent.

213
00:13:59 --> 00:14:03
In a way, that one one one,
because it's multiplying the

214
00:14:03 --> 00:14:07
rows,
is like an eigenvector in the

215
00:14:07 --> 00:14:11
-- it's in the left null space,
right?

216
00:14:11 --> 00:14:15
One one one is in the left null
space.

217
00:14:15 --> 00:14:22
It's singular because the rows
are dependent -- and can I just

218
00:14:22 --> 00:14:24.76
keep the reasoning going?

219
00:14:24.76 --> 00:14:29
Because this vector one one one
is --

220
00:14:29 --> 00:14:34
it's not in the null space of
the matrix, but it's in the null

221
00:14:34 --> 00:14:40
space of the transpose -- is in
the null space of the transpose.

222
00:14:40 --> 00:14:42
And that's good enough.

223
00:14:42 --> 00:14:47.56
If we have a square matrix --
if we have a square matrix and

224
00:14:47.56 --> 00:14:50
the rows are dependent,
that

225
00:14:50 --> 00:14:52
matrix is singular.

226
00:14:52 --> 00:14:57
So it turned out that the
immediate guy we could identify

227
00:14:57 --> 00:14:59
was one one one.

228
00:14:59 --> 00:15:05
Of course, the -- there will be
somebody in the null space,

229
00:15:05 --> 00:15:05
too.

230
00:15:05 --> 00:15:08.09
And actually,
who will it be?

231
00:15:08.09 --> 00:15:13
So what's -- so -- so now I
want to ask about the

232
00:15:13 --> 00:15:16
null space of -- of the matrix
itself.

233
00:15:16 --> 00:15:19
What combination of the columns
gives zero?

234
00:15:19 --> 00:15:23.11
I -- I don't want to compute it
because I just made up this

235
00:15:23.11 --> 00:15:26
matrix and -- it will -- it
would take me a while -- it

236
00:15:26 --> 00:15:31
looks sort of doable
because it's three by three but

237
00:15:31 --> 00:15:34
wh- my point is,
what -- what vector is it if we

238
00:15:34 --> 00:15:39
-- once we've found it,
what have we got that's in the

239
00:15:39 --> 00:15:41
-- in the null space of A?

240
00:15:41 --> 00:15:43
It's the eigenvector,
right?

241
00:15:43 --> 00:15:46
That's where we find X one.

242
00:15:46 --> 00:15:51
Then X one, the eigenvector,
is in the null space of A.

243
00:15:51 --> 00:15:56
That's the eigenvector
corresponding to the eigenvalue

244
00:15:56 --> 00:15:56
one.

245
00:15:56 --> 00:15:57
Right?

246
00:15:57 --> 00:16:00
That's how we find
eigenvectors.

247
00:16:00 --> 00:16:05
So those three columns must be
dependent -- some combination of

248
00:16:05 --> 00:16:10.29
columns --
of those three columns is the

249
00:16:10.29 --> 00:16:14
zero column and that -- the
three components in that

250
00:16:14 --> 00:16:17
combination are the eigenvector.

251
00:16:17 --> 00:16:19
And that guy is the steady
state.

252
00:16:19 --> 00:16:20
Okay.

253
00:16:20 --> 00:16:23
So I'm happy about the -- the
thinking here,

254
00:16:23 --> 00:16:28
but I haven't given -- I
haven't completed it because I

255
00:16:28 --> 00:16:29
haven't found x1.

256
00:16:29 --> 00:16:32
But it's there.

257
00:16:32 --> 00:16:39
Can I -- another thought came
to me as I was doing this,

258
00:16:39 --> 00:16:46
another little comment that --
you -- about eigenvalues and

259
00:16:46 --> 00:16:51
eigenvectors,
because of A and A transpose.

260
00:16:51 --> 00:16:58
What can you tell me about
eigenvalues of A -- of A and

261
00:16:58 --> 00:17:01
eigenvalues of A transpose?

262
00:17:01 --> 00:17:02
Whoops.

263
00:17:02 --> 00:17:06
They're the same.

264
00:17:06 --> 00:17:10
They're -- so this is a little
comment -- we -- it's useful,

265
00:17:10 --> 00:17:14
since eigenvalues are generally
not easy to find -- it's always

266
00:17:14 --> 00:17:17
useful to know some cases where
you've got them,

267
00:17:17 --> 00:17:21
where -- and this is -- if you
know the eigenvalues of A,

268
00:17:21 --> 00:17:25
then you know the eigenvalues
of A transpose.

269
00:17:25 --> 00:17:30
eigenvalues of A transpose are
the same.

270
00:17:30 --> 00:17:35
And can I just,
like, review why that is?

271
00:17:35 --> 00:17:42
So to find the eigenvalues of
A, this would be determinate of

272
00:17:42 --> 00:17:49
A minus lambda I equals zero,
that gives me an eigenvalue of

273
00:17:49 --> 00:17:57
A -- now how can I get A
transpose into the picture here?

274
00:17:57 --> 00:18:02
I'll use the fact that the
determinant of a matrix and the

275
00:18:02 --> 00:18:05
determinant of its transpose are
the same.

276
00:18:05 --> 00:18:09
The determinant of a matrix
equals the determinant of a --

277
00:18:09 --> 00:18:10.39
of the transpose.

278
00:18:10.39 --> 00:18:13
That was property ten,
the very last guy in our

279
00:18:13 --> 00:18:15
determinant list.

280
00:18:15 --> 00:18:18
So I'll transpose that matrix.

281
00:18:18 --> 00:18:24.43
This leads to -- I just take
the matrix and transpose it,

282
00:18:24.43 --> 00:18:29
but now what do I get when I
transpose lambda I?

283
00:18:29 --> 00:18:31
I just get lambda I.

284
00:18:31 --> 00:18:35
So that's -- that's all there
was to the

285
00:18:35 --> 00:18:36
reasoning.

286
00:18:36 --> 00:18:40
The reasoning is that the
eigenvalues of A solved that

287
00:18:40 --> 00:18:41
equation.

288
00:18:41 --> 00:18:44
The determinant of a matrix is
the determinant of its

289
00:18:44 --> 00:18:49.16
transpose, so that gives me this
equation and that tells me that

290
00:18:49.16 --> 00:18:53
the same lambdas are eigenvalues
of A transpose.

291
00:18:53 --> 00:18:56
So that, backing up to the
Markov case, one is an

292
00:18:56 --> 00:19:01
eigenvalue of A transpose and we
actually found its eigenvector,

293
00:19:01 --> 00:19:04
one one one,
and that tell us that one is

294
00:19:04 --> 00:19:08
also an eigenvalue of A -- but,
of course, it has a different

295
00:19:08 --> 00:19:11.66
eigenvector,
the -- the left null space

296
00:19:11.66 --> 00:19:15
isn't the same as the null space
and we would have to find it.

297
00:19:15 --> 00:19:19
So there's some vector here
which is x1 that produces zero

298
00:19:19 --> 00:19:19
zero zero.

299
00:19:19 --> 00:19:22
Actually, it wouldn't be that
hard to find,

300
00:19:22 --> 00:19:25
you know, I -- as I'm talking
I'm thinking,

301
00:19:25 --> 00:19:29
okay, I going to follow through
and actually find it?

302
00:19:29 --> 00:19:34
Well, I can tell from this one
-- look, if I put a point six

303
00:19:34 --> 00:19:39
there and a point seven there,
that's what -- then I'll be

304
00:19:39 --> 00:19:41
okay in the last row,
right?

305
00:19:41 --> 00:19:45
Now I only -- remains to find
one guy.

306
00:19:45 --> 00:19:48
And let me take the first row,
then.

307
00:19:48 --> 00:19:54
Minus point 54 plus point 21 --
there's some big number going in

308
00:19:54 --> 00:19:55.54
there, right?

309
00:19:55.54 --> 00:19:59
So I have -- just to make the
first row come out zero,

310
00:19:59 --> 00:20:04
I'm getting minus point 54 plus
point 21, so that was minus

311
00:20:04 --> 00:20:07
point 33 and what -- what do I
want?

312
00:20:07 --> 00:20:09
Like thirty three hundred?

313
00:20:09 --> 00:20:13
This is the first time in the
history of linear algebra that

314
00:20:13 --> 00:20:17
an
eigenvector has every had a

315
00:20:17 --> 00:20:19
component thirty three hundred.

316
00:20:19 --> 00:20:21
But I guess it's true.

317
00:20:21 --> 00:20:26
Because then I multiply by
minus one over a hundred -- oh

318
00:20:26 --> 00:20:27
no, it was point 33.

319
00:20:27 --> 00:20:30
So is this just -- oh,
shoot.

320
00:20:30 --> 00:20:30
Only 33.

321
00:20:30 --> 00:20:31
Okay.

322
00:20:31 --> 00:20:31
Only 33.

323
00:20:31 --> 00:20:35
Okay, so there's the
eigenvector.

324
00:20:35 --> 00:20:39.74
Oh, and notice that it -- that
it turned -- did turn out,

325
00:20:39.74 --> 00:20:41
at least, to be all positive.

326
00:20:41 --> 00:20:45
So that was,
like, the theory -- predicts

327
00:20:45 --> 00:20:46
that part, too.

328
00:20:46 --> 00:20:48
I won't give the proof of that
part.

329
00:20:48 --> 00:20:51
So 30 -- 33 -- point six 33
point seven.

330
00:20:51 --> 00:20:52
Okay.

331
00:20:52 --> 00:20:57
Now those are the ma- that's
the linear algebra part.

332
00:20:57 --> 00:20:59
Can I get to the applications?

333
00:20:59 --> 00:21:01
Where do these Markov matrices
come from?

334
00:21:01 --> 00:21:05
Because that's -- that's part
of this course and absolutely

335
00:21:05 --> 00:21:07
part of this lecture.

336
00:21:07 --> 00:21:07
Okay.

337
00:21:07 --> 00:21:12
So where's -- what's an
application of Markov matrices?

338
00:21:12 --> 00:21:12
Okay.

339
00:21:12 --> 00:21:17
Markov matrices -- so,
my equation,

340
00:21:17 --> 00:21:24
then, that I'm solving and
studying is this equation

341
00:21:24 --> 00:21:25
u(k+1)=Auk.

342
00:21:25 --> 00:21:29.57
And now A is a Markov matrix.

343
00:21:29.57 --> 00:21:31
A is Markov.

344
00:21:31 --> 00:21:35
And I want to give an example.

345
00:21:35 --> 00:21:39
Can I just create an example?

346
00:21:39 --> 00:21:43
It'll be two by two.

347
00:21:43 --> 00:21:48
And it's one I've used before
because it seems to me to bring

348
00:21:48 --> 00:21:50
out the idea.

349
00:21:50 --> 00:21:55
It's -- because we have two by
two, we have two states,

350
00:21:55 --> 00:21:58
let's say California and
Massachusetts.

351
00:21:58 --> 00:22:03
And I'm looking at the
populations in those two states,

352
00:22:03 --> 00:22:10
the people in those two states,
California and Massachusetts.

353
00:22:10 --> 00:22:14
And my matrix A is going to
tell me in a -- in a year,

354
00:22:14 --> 00:22:16
some movement has happened.

355
00:22:16 --> 00:22:18
Some people stayed in
Massachusetts,

356
00:22:18 --> 00:22:22
some people moved to
California, some smart people

357
00:22:22 --> 00:22:24
moved from California to
Massachusetts,

358
00:22:24 --> 00:22:29
some people stayed in
California and made a billion.

359
00:22:29 --> 00:22:29
Okay.

360
00:22:29 --> 00:22:33
So that -- there's a matrix
there with four entries and

361
00:22:33 --> 00:22:38
those tell me the fractions of
my population -- so I'm making

362
00:22:38 --> 00:22:42
-- I'm going to use fractions,
so they won't be negative,

363
00:22:42 --> 00:22:47
of course, because -- because
only positive people are in-

364
00:22:47 --> 00:22:51
involved here -- and they'll
add up to one,

365
00:22:51 --> 00:22:53
because I'm accounting for all
people.

366
00:22:53 --> 00:22:57
So that's why I have these two
key properties.

367
00:22:57 --> 00:23:01
The entries are greater equal
zero because I'm looking at

368
00:23:01 --> 00:23:02
probabilities.

369
00:23:02 --> 00:23:05
Do they move,
do they stay?

370
00:23:05 --> 00:23:09
Those probabilities are all
between zero and one.

371
00:23:09 --> 00:23:12
And the probabilities add to
one because everybody's

372
00:23:12 --> 00:23:13
accounted for.

373
00:23:13 --> 00:23:17
I'm not losing anybody,
gaining anybody in this Markov

374
00:23:17 --> 00:23:18
chain.

375
00:23:18 --> 00:23:21
It's -- it conserves the total
population.

376
00:23:21 --> 00:23:21
Okay.

377
00:23:21 --> 00:23:25
So what would be a typical
matrix, then?

378
00:23:25 --> 00:23:31
So this would be u,
California and u Massachusetts

379
00:23:31 --> 00:23:34
at time t equal k+1.

380
00:23:34 --> 00:23:39
And it's some matrix,
which we'll think of,

381
00:23:39 --> 00:23:45.83
times u California and u
Massachusetts at time k.

382
00:23:45.83 --> 00:23:51
And notice this matrix is going
to stay the same,

383
00:23:51 --> 00:23:54
you know, forever.

384
00:23:54 --> 00:24:00
So that's a severe limitation
on the example.

385
00:24:00 --> 00:24:05
The example has a -- the same
Markov matrix,

386
00:24:05 --> 00:24:09
the same probabilities act at
every time.

387
00:24:09 --> 00:24:09
Okay.

388
00:24:09 --> 00:24:14.78
So what's a reasonable,
say -- say point nine of the

389
00:24:14.78 --> 00:24:18
people in California at time k
stay there.

390
00:24:18 --> 00:24:23
And point one of the people in
California move to

391
00:24:23 --> 00:24:26
Massachusetts.

392
00:24:26 --> 00:24:31
Notice why that column added to
one, because we've now accounted

393
00:24:31 --> 00:24:34.31
for all the people in California
at time k.

394
00:24:34.31 --> 00:24:37
Nine tenths of them are still
in California,

395
00:24:37 --> 00:24:39
one tenth are here at time k+1.

396
00:24:39 --> 00:24:40
Okay.

397
00:24:40 --> 00:24:43
What about the people who are
in Massachusetts?

398
00:24:43 --> 00:24:46
This is going to multiply
column two, right,

399
00:24:46 --> 00:24:51
by our
fundamental rule of multiplying

400
00:24:51 --> 00:24:56
matrix by vector,
it's the -- it's the population

401
00:24:56 --> 00:24:58
in Massachusetts.

402
00:24:58 --> 00:25:04.25
Shall we say that -- that after
the Red Sox, fail again,

403
00:25:04.25 --> 00:25:10
eight -- only 80 percent of the
people in Massachusetts stay and

404
00:25:10 --> 00:25:14
20 percent
move to California.

405
00:25:14 --> 00:25:14
Okay.

406
00:25:14 --> 00:25:19
So again, this adds to one,
which accounts for all people

407
00:25:19 --> 00:25:21
in Massachusetts where they are.

408
00:25:21 --> 00:25:23
So there is a Markov matrix.

409
00:25:23 --> 00:25:26
Non-negative entries adding to
one.

410
00:25:26 --> 00:25:28
What's the steady state?

411
00:25:28 --> 00:25:32
If everybody started in
Massachusetts,

412
00:25:32 --> 00:25:36
say, at -- you know,
when the Pilgrims showed up or

413
00:25:36 --> 00:25:36
something.

414
00:25:36 --> 00:25:38
Then where are they now?

415
00:25:38 --> 00:25:43
Where are they at time 100,
let's say, or maybe -- I don't

416
00:25:43 --> 00:25:46
know, how many years since the
Pilgrims?

417
00:25:46 --> 00:25:47
300 and something.

418
00:25:47 --> 00:25:51
Or -- and actually where will
they be, like,

419
00:25:51 --> 00:25:54
way out a million
years from now?

420
00:25:54 --> 00:25:59
I -- I could multiply -- take
the powers of this matrix.

421
00:25:59 --> 00:26:05
In fact, you'll -- you would --
ought to be able to figure out

422
00:26:05 --> 00:26:09
what is the hundredth power of
that matrix?

423
00:26:09 --> 00:26:11
Why don't we do that?

424
00:26:11 --> 00:26:15
But let me follow the steady
state.

425
00:26:15 --> 00:26:20
So what -- what's my starting
-- my starting u Cal,

426
00:26:20 --> 00:26:26
u Mass at time zero is,
shall we say -- shall we put

427
00:26:26 --> 00:26:28.8
anybody in California?

428
00:26:28.8 --> 00:26:35
Let's make -- let's make zero
there, and say the population of

429
00:26:35 --> 00:26:41
Massachusetts is --
let's say a thousand just to --

430
00:26:41 --> 00:26:42
okay.

431
00:26:42 --> 00:26:47.68
So the population is -- so the
populations are zero and a

432
00:26:47.68 --> 00:26:49
thousand at the start.

433
00:26:49 --> 00:26:55
What can you tell me about this
population after -- after k

434
00:26:55 --> 00:26:56
steps?

435
00:26:56 --> 00:26:59
What will u Cal plus u Mass add
to?

436
00:26:59 --> 00:27:00.7
A thousand.

437
00:27:00.7 --> 00:27:06
Those thousand people are
always accounted for.

438
00:27:06 --> 00:27:10
But -- so u Mass will start
dropping from a thousand and u

439
00:27:10 --> 00:27:12
Cal will start growing.

440
00:27:12 --> 00:27:17
Actually, we could see -- why
don't we figure out what it is

441
00:27:17 --> 00:27:18
after one?

442
00:27:18 --> 00:27:22
After one time step,
what are the populations at

443
00:27:22 --> 00:27:23
time one?

444
00:27:23 --> 00:27:26
So what happens in one step?

445
00:27:26 --> 00:27:30
You multiply once by that
matrix and, let's see,

446
00:27:30 --> 00:27:35
zero times this column -- so
it's just a thousand times this

447
00:27:35 --> 00:27:38
column, so I think we're getting
200 and 800.

448
00:27:38 --> 00:27:42
So after the first step,
200 people have -- are in

449
00:27:42 --> 00:27:43
California.

450
00:27:43 --> 00:27:48
Now at the following step,
I'll multiply again by this

451
00:27:48 --> 00:27:52
matrix -- more people will move
to California.

452
00:27:52 --> 00:27:54.64
Some people will move back.

453
00:27:54.64 --> 00:27:59
Twenty people will come back
and, the -- the net result will

454
00:27:59 --> 00:28:03
be that the California
population will be above 200 and

455
00:28:03 --> 00:28:08
the Massachusetts below 800 and
they'll still add up to a

456
00:28:08 --> 00:28:09
thousand.

457
00:28:09 --> 00:28:10.3
Okay.

458
00:28:10.3 --> 00:28:11
I do that a few times.

459
00:28:11 --> 00:28:13
I do that 100 times.

460
00:28:13 --> 00:28:15
What's the population?

461
00:28:15 --> 00:28:18
Well, okay, to answer any
question like that,

462
00:28:18 --> 00:28:21
I need the eigenvalues and
eigenvectors,

463
00:28:21 --> 00:28:21
right?

464
00:28:21 --> 00:28:26
As soon as I've -- I've created
an example, but as soon as I

465
00:28:26 --> 00:28:31
want to solve anything,
I have to find eigenvalues and

466
00:28:31 --> 00:28:34
eigenvectors of that matrix.

467
00:28:34 --> 00:28:34.85
Okay.

468
00:28:34.85 --> 00:28:36
So let's do it.

469
00:28:36 --> 00:28:41.09
So there's the matrix .9,
.2, .1, .8 and tell me its

470
00:28:41.09 --> 00:28:42
eigenvalues.

471
00:28:42 --> 00:28:46
Lambda equals -- so tell me one
eigenvalue?

472
00:28:46 --> 00:28:48
One, thanks.

473
00:28:48 --> 00:28:50.66
And tell me the other one.

474
00:28:50.66 --> 00:28:55.15
What's the other eigenvalue --
from the trace or the

475
00:28:55.15 --> 00:28:59
determinant -- from the -- I --
the trace is what -- is,

476
00:28:59 --> 00:29:01
like, easier.

477
00:29:01 --> 00:29:05
So the trace of that matrix is
one point seven.

478
00:29:05 --> 00:29:08
So the other eigenvalue is
point seven.

479
00:29:08 --> 00:29:12
And it -- notice that
it's less than one.

480
00:29:12 --> 00:29:16
And notice that that
determinant is point 72-.02,

481
00:29:16 --> 00:29:18
which is point seven.

482
00:29:18 --> 00:29:18
Right.

483
00:29:18 --> 00:29:19.21
Okay.

484
00:29:19.21 --> 00:29:21
Now to find the eigenvectors.

485
00:29:21 --> 00:29:26
This is -- so that's lambda one
and the eigenvector -- I'll

486
00:29:26 --> 00:29:29
subtract one from the diagonal,
right?

487
00:29:29 --> 00:29:33
So can I do that in light let
-- in light here?

488
00:29:33 --> 00:29:38
Subtract one from the diagonal,
I have minus point one and

489
00:29:38 --> 00:29:42
minus point two,
and of course these are still

490
00:29:42 --> 00:29:42
there.

491
00:29:42 --> 00:29:47
And I'm looking for its --
here's -- here's -- this is

492
00:29:47 --> 00:29:48
going to be x1.

493
00:29:48 --> 00:29:51
It's the null space of A minus
I.

494
00:29:51 --> 00:29:55
Okay, everybody sees that it's
two and one.

495
00:29:55 --> 00:29:56
Okay?

496
00:29:56 --> 00:30:01
And now how about -- so that --
and it -- notice that that

497
00:30:01 --> 00:30:03
eigenvector is positive.

498
00:30:03 --> 00:30:07
And actually,
we can jump to infinity right

499
00:30:07 --> 00:30:08
now.

500
00:30:08 --> 00:30:11
What's the population at
infinity?

501
00:30:11 --> 00:30:16.8
It's a multiple -- this is --
this eigenvector is

502
00:30:16.8 --> 00:30:18
giving the steady state.

503
00:30:18 --> 00:30:22
It's some multiple of this,
and how is that multiple

504
00:30:22 --> 00:30:23
decided?

505
00:30:23 --> 00:30:26
By adding up to a thousand
people.

506
00:30:26 --> 00:30:29
So the steady state,
the c1x1 -- this is the x1,

507
00:30:29 --> 00:30:34
but that adds up to three,
so I really want two -- it's

508
00:30:34 --> 00:30:38
going to be two thirds of a
thousand and one third of a

509
00:30:38 --> 00:30:42
thousand,
making a total of the thousand

510
00:30:42 --> 00:30:42
people.

511
00:30:42 --> 00:30:44
That'll be the steady state.

512
00:30:44 --> 00:30:47.73
That's really all I need to
know at infinity.

513
00:30:47.73 --> 00:30:51
But if I want to know what's
happened after just a finite

514
00:30:51 --> 00:30:54
number like 100 steps,
I'd better find this

515
00:30:54 --> 00:30:55
eigenvector.

516
00:30:55 --> 00:30:59
So can I --
can I look at -- I'll subtract

517
00:30:59 --> 00:31:04
point seven time -- ti- from the
diagonal and I'll get that and

518
00:31:04 --> 00:31:08
I'll look at the null space of
that one and I -- and this is

519
00:31:08 --> 00:31:11
going to give me x2,
now, and what is it?

520
00:31:11 --> 00:31:16
So what's in the null space of
-- that's certainly singular,

521
00:31:16 --> 00:31:20
so I know my calculation is
right,

522
00:31:20 --> 00:31:22
and -- one and minus one.

523
00:31:22 --> 00:31:23
One and minus one.

524
00:31:23 --> 00:31:28
So I'm prepared now to write
down the solution after 100 time

525
00:31:28 --> 00:31:28
steps.

526
00:31:28 --> 00:31:32
The -- the populations after
100 time steps,

527
00:31:32 --> 00:31:32
right?

528
00:31:32 --> 00:31:37
Can -- can we remember the
point one -- the -- the one with

529
00:31:37 --> 00:31:43
this two one eigenvector and the
point seven with the minus

530
00:31:43 --> 00:31:45
one one eigenvector.

531
00:31:45 --> 00:31:50
So I'll -- let me -- I'll just
write it above here.

532
00:31:50 --> 00:31:56
u after k steps is some
multiple of one to the k times

533
00:31:56 --> 00:32:03
the two one eigenvector and some
multiple of point seven to the k

534
00:32:03 --> 00:32:06
times the minus one one
eigenvector.

535
00:32:06 --> 00:32:07
Right?

536
00:32:07 --> 00:32:12
That's --
I -- this is how I take -- how

537
00:32:12 --> 00:32:15
powers of a matrix work.

538
00:32:15 --> 00:32:21.82
When I apply those powers to a
u0, what I -- so it's u0,

539
00:32:21.82 --> 00:32:28
which was zero a thousand --
that has to be corrected k=0.

540
00:32:28 --> 00:32:34
So I'm plugging in k=0 and I
get c1 times two one and c2

541
00:32:34 --> 00:32:37
times minus one one.

542
00:32:37 --> 00:32:40
Two equations,
two constants,

543
00:32:40 --> 00:32:43
certainly independent
eigenvectors,

544
00:32:43 --> 00:32:48
so there's a solution and you
see what it is?

545
00:32:48 --> 00:32:54
Let's see, I guess we already
figured that c1 was a thousand

546
00:32:54 --> 00:32:59
over three, I think -- did we
think that had to

547
00:32:59 --> 00:33:01
be a thousand over three?

548
00:33:01 --> 00:33:07
And maybe c2 would be -- excuse
me, let -- get an eraser -- we

549
00:33:07 --> 00:33:11
can -- I just -- I think we've
-- get it here.

550
00:33:11 --> 00:33:16
c2, we want to get a zero here,
so maybe we need plus two

551
00:33:16 --> 00:33:18
thousand over three.

552
00:33:18 --> 00:33:21.03
I think that has to work.

553
00:33:21.03 --> 00:33:26
Two times a thousand over three
minus two thousand over three,

554
00:33:26 --> 00:33:31
that'll give us the zero and a
thousand over three and the two

555
00:33:31 --> 00:33:35
thousand over three will give us
three thousand over three,

556
00:33:35 --> 00:33:35
the thousand.

557
00:33:35 --> 00:33:40
So this is what we approach --
this part, with the point seven

558
00:33:40 --> 00:33:44.18
to the k-th power is the part
that's disappearing.

559
00:33:44.18 --> 00:33:44
Okay.

560
00:33:44 --> 00:33:48
That's -- that's Markov
matrices.

561
00:33:48 --> 00:33:53
That's an example of where they
come from, from modeling

562
00:33:53 --> 00:33:59
movement of people with no gain
or loss, with total -- total

563
00:33:59 --> 00:34:01
count conserved.

564
00:34:01 --> 00:34:02
Okay.

565
00:34:02 --> 00:34:07
I -- just if I can add one more
comment,

566
00:34:07 --> 00:34:11
because you'll see Markov
matrices in electrical

567
00:34:11 --> 00:34:15.94
engineering courses and often
you'll see them -- sorry,

568
00:34:15.94 --> 00:34:18
here's my little comment.

569
00:34:18 --> 00:34:22
Sometimes -- in a lot of
applications they prefer to work

570
00:34:22 --> 00:34:24
with row vectors.

571
00:34:24 --> 00:34:28
So they -- instead of -- this
was natural for us,

572
00:34:28 --> 00:34:29
right?

573
00:34:29 --> 00:34:33
For all the eigenvectors to be
column vectors.

574
00:34:33 --> 00:34:37
So our columns added to one in
the Markov matrix.

575
00:34:37 --> 00:34:41
Just so you don't think,
well, what -- what's going on?

576
00:34:41 --> 00:34:47
If we work with row vectors and
we multiply vector times matrix

577
00:34:47 --> 00:34:52
-- so we're multiplying from
the left -- then it'll be the

578
00:34:52 --> 00:34:57
then we'll be using the
transpose of -- of this matrix

579
00:34:57 --> 00:35:00
and it'll be the rows that add
to one.

580
00:35:00 --> 00:35:04
So in other textbooks,
you'll see -- instead of col-

581
00:35:04 --> 00:35:08
columns adding to one,
you'll see rows add to one.

582
00:35:08 --> 00:35:08
Okay.

583
00:35:08 --> 00:35:10
Fine.

584
00:35:10 --> 00:35:14
Okay, that's what I wanted to
say about Markov,

585
00:35:14 --> 00:35:18
now I want to say something
about projections and even

586
00:35:18 --> 00:35:22
leading in -- a little into
Fourier series.

587
00:35:22 --> 00:35:25
Because -- but before any
Fourier stuff,

588
00:35:25 --> 00:35:29.37
let me make a comment about
projections.

589
00:35:29.37 --> 00:35:35
This -- so this is a comment
about projections onto --

590
00:35:35 --> 00:35:38
with an orthonormal basis.

591
00:35:38 --> 00:35:44
So, of course,
the basis vectors are q1 up to

592
00:35:44 --> 00:35:44
qn.

593
00:35:44 --> 00:35:45
Okay.

594
00:35:45 --> 00:35:48.11
I have a vector b.

595
00:35:48.11 --> 00:35:55
Let -- let me imagine -- let me
imagine this is a basis.

596
00:35:55 --> 00:35:59.78
Let -- let's say I'm in n by n.

597
00:35:59.78 --> 00:36:03
I'm -- I've got,
eh,

598
00:36:03 --> 00:36:08
n orthonormal vectors,
I'm in n dimensional space so

599
00:36:08 --> 00:36:15.14
they're a complete -- they're a
basis -- any vector v could be

600
00:36:15.14 --> 00:36:17
expanded in this basis.

601
00:36:17 --> 00:36:22
So any vector v is some
combination, some amount of q1

602
00:36:22 --> 00:36:28
plus some amount of q2 plus some
amount of qn.

603
00:36:28 --> 00:36:30
So -- so any v.

604
00:36:30 --> 00:36:35
I just want you to tell me what
those amounts are.

605
00:36:35 --> 00:36:40
What are x1 -- what's x1,
for example?

606
00:36:40 --> 00:36:43
So I'm looking for the
expansion.

607
00:36:43 --> 00:36:48
This is -- this is really our
projection.

608
00:36:48 --> 00:36:55
I could -- I could really use
the word expansion.

609
00:36:55 --> 00:36:59
I'm expanding the vector in the
basis.

610
00:36:59 --> 00:37:05
And the special thing about the
basis is that it's orthonormal.

611
00:37:05 --> 00:37:10
So that should give me a
special formula for the answer,

612
00:37:10 --> 00:37:12
for the coefficients.

613
00:37:12 --> 00:37:14
So how do I get x1?

614
00:37:14 --> 00:37:17
What -- what's a formula for
x1?

615
00:37:17 --> 00:37:23
I could -- I can go through the
projection -- the Q transpose

616
00:37:23 --> 00:37:28
Q, all that -- normal equations,
but -- and I'll get -- I'll

617
00:37:28 --> 00:37:33
come out with this nice answer
that I think I can see right

618
00:37:33 --> 00:37:34
away.

619
00:37:34 --> 00:37:39
How can I pick -- get a hold of
x1 and get these other x-s out

620
00:37:39 --> 00:37:41
of the equation?

621
00:37:41 --> 00:37:44
So how can I get a nice,
simple

622
00:37:44 --> 00:37:46
formula for x1?

623
00:37:46 --> 00:37:50
And then we want to see,
sure, we knew that all the

624
00:37:50 --> 00:37:51
time.

625
00:37:51 --> 00:37:51
Okay.

626
00:37:51 --> 00:37:52
So what's x1?

627
00:37:52 --> 00:37:58.01
The good way is take the inner
product of everything with q1.

628
00:37:58.01 --> 00:38:02
Take the inner product of that
whole equation,

629
00:38:02 --> 00:38:03
every term, with q1.

630
00:38:03 --> 00:38:06
What will happen to that last
term?

631
00:38:06 --> 00:38:12.2
The inner product --
when -- if I take the dot

632
00:38:12.2 --> 00:38:15
product with q1 I get zero,
right?

633
00:38:15 --> 00:38:18
Because this basis was
orthonormal.

634
00:38:18 --> 00:38:22
If I take the dot product with
q2 I get zero.

635
00:38:22 --> 00:38:26
If I take the dot product with
q1 I get one.

636
00:38:26 --> 00:38:30
So that tells me what x1 is.
q1 transpose v,

637
00:38:30 --> 00:38:36
that's taking the dot product,
is x1 times q1 transpose q1

638
00:38:36 --> 00:38:38
plus a bunch of zeroes.

639
00:38:38 --> 00:38:42
And this is a one,
so I can forget that.

640
00:38:42 --> 00:38:44
I get x1 immediately.

641
00:38:44 --> 00:38:49
So -- do you see what I'm
saying -- is that I have an

642
00:38:49 --> 00:38:54.05
orthonormal basis,
then the coefficient that I

643
00:38:54.05 --> 00:38:58
need for
each basis vector is a cinch to

644
00:38:58 --> 00:38:58
find.

645
00:38:58 --> 00:39:02
Let me -- let me just -- I have
to put this into matrix

646
00:39:02 --> 00:39:06
language, too,
so you'll see it there also.

647
00:39:06 --> 00:39:10
If I write that first equation
in matrix language,

648
00:39:10 --> 00:39:11
what -- what is it?

649
00:39:11 --> 00:39:15
I'm writing --
in matrix language,

650
00:39:15 --> 00:39:21
this equation says I'm taking
these columns -- are -- are you

651
00:39:21 --> 00:39:23
guys good at this now?

652
00:39:23 --> 00:39:29
I'm taking those columns times
the Xs and getting V,

653
00:39:29 --> 00:39:29.83
right?

654
00:39:29.83 --> 00:39:32
That's the matrix form.

655
00:39:32 --> 00:39:34
Okay, that's the matrix Q.

656
00:39:34 --> 00:39:35
Qx is v.

657
00:39:35 --> 00:39:40.38
What's the solution
to that equation?

658
00:39:40.38 --> 00:39:44
It's -- of course,
it's x equal Q inverse v.

659
00:39:44 --> 00:39:48
So x is Q inverse v,
but what's the point?

660
00:39:48 --> 00:39:53
Q inverse in this case is going
to -- is simple.

661
00:39:53 --> 00:39:57
I don't have to work to invert
this matrix Q,

662
00:39:57 --> 00:40:03
because of the fact that the --
these columns are orthonormal,

663
00:40:03 --> 00:40:06.39
I know the inverse to that.

664
00:40:06.39 --> 00:40:08
And it is Q transpose.

665
00:40:08 --> 00:40:12
When you see a Q,
a square matrix with that

666
00:40:12 --> 00:40:16
letter Q, the -- that just
triggers -- Q inverse is the

667
00:40:16 --> 00:40:18
same as Q transpose.

668
00:40:18 --> 00:40:23
So the first component,
then -- the first component of

669
00:40:23 --> 00:40:29
x is the first row times
v, and what's that?

670
00:40:29 --> 00:40:35
The first component of this
answer is the first row of Q

671
00:40:35 --> 00:40:37.23
transpose.

672
00:40:37.23 --> 00:40:43
That's just -- that's just q1
transpose times v.

673
00:40:43 --> 00:40:47
So that's what we concluded
here, too.

674
00:40:47 --> 00:40:48
Okay.

675
00:40:48 --> 00:40:52
So -- so nothing Fourier here.

676
00:40:52 --> 00:40:57
The -- the key ingredient here
was

677
00:40:57 --> 00:41:01.23
that the q-s are orthonormal.

678
00:41:01.23 --> 00:41:06
And now that's what Fourier
series are built on.

679
00:41:06 --> 00:41:12
So now, in the remaining time,
let me say something about

680
00:41:12 --> 00:41:14
Fourier series.

681
00:41:14 --> 00:41:15
Okay.

682
00:41:15 --> 00:41:21
So Fourier series is -- well,
we've got a function f of x.

683
00:41:21 --> 00:41:28
And we want to write it as a
combination of -- maybe

684
00:41:28 --> 00:41:31
it has a constant term.

685
00:41:31 --> 00:41:34
And then it has some cos(x) in
it.

686
00:41:34 --> 00:41:38
And it has some sin(x) in it.

687
00:41:38 --> 00:41:41
And it has some cos(2x) in it.

688
00:41:41 --> 00:41:46
And a -- and some sin(2x),
and forever.

689
00:41:46 --> 00:41:51
So what's -- what's the
difference between this type

690
00:41:51 --> 00:41:55.26
problem and the one above it?

691
00:41:55.26 --> 00:42:02
This one's infinite,
but the key property of things

692
00:42:02 --> 00:42:07.83
being orthogonal is still true
for sines and cosines,

693
00:42:07.83 --> 00:42:13.43
so it's the property that makes
Fourier series work.

694
00:42:13.43 --> 00:42:17
So that's called a Fourier
series.

695
00:42:17 --> 00:42:19
Better write his name up.

696
00:42:19 --> 00:42:22
Fourier series.

697
00:42:22 --> 00:42:26
So it was Joseph Fourier who
realized that,

698
00:42:26 --> 00:42:28
hey, I could work in function
space.

699
00:42:28 --> 00:42:33
Instead of a vector v,
I could have a function f of x.

700
00:42:33 --> 00:42:37
Instead of orthogonal vectors,
q1, q2 , q3,

701
00:42:37 --> 00:42:41
I could have orthogonal
functions, the constant,

702
00:42:41 --> 00:42:45
the cos(x), the sin(x),
the s- cos(2x),

703
00:42:45 --> 00:42:47
but infinitely many of them.

704
00:42:47 --> 00:42:51
I need infinitely many,
because my space is infinite

705
00:42:51 --> 00:42:52
dimensional.

706
00:42:52 --> 00:42:55.39
So this is, like,
the moment in which we leave

707
00:42:55.39 --> 00:42:59
finite dimensional vector spaces
and go to infinite dimensional

708
00:42:59 --> 00:43:04
vector spaces and our basis --
so the vectors are now

709
00:43:04 --> 00:43:09
functions -- and of course,
there are so many functions

710
00:43:09 --> 00:43:13
that it's -- that we've got an
infin- infinite dimensional

711
00:43:13 --> 00:43:17
space -- and the basis vectors
are functions,

712
00:43:17 --> 00:43:19.79
too.
a0, the constant function one

713
00:43:19.79 --> 00:43:24
-- so my basis is one cos(x),
sin(x), cos(2x),

714
00:43:24 --> 00:43:26
sin(2x) and so on.

715
00:43:26 --> 00:43:32
And the reason Fourier series
is a success is that those are

716
00:43:32 --> 00:43:33
orthogonal.

717
00:43:33 --> 00:43:33
Okay.

718
00:43:33 --> 00:43:37
Now what do I mean by
orthogonal?

719
00:43:37 --> 00:43:42
I know what it means for two
vectors to be orthogonal -- y

720
00:43:42 --> 00:43:46
transpose x equals zero,
right?

721
00:43:46 --> 00:43:48
Dot product equals zero.

722
00:43:48 --> 00:43:51.72
But what's the dot product of
functions?

723
00:43:51.72 --> 00:43:56
I'm claiming that whatever it
is, the dot product -- or we

724
00:43:56 --> 00:43:59
would more likely use the word
inner product of,

725
00:43:59 --> 00:44:02
say, cos(x) with sin(x) is
zero.

726
00:44:02 --> 00:44:04
And cos(x) with cos(2x),
also zero.

727
00:44:04 --> 00:44:08
So I -- let me tell you what I
mean

728
00:44:08 --> 00:44:12
by that, by that dot product.

729
00:44:12 --> 00:44:17
Well, how do I compute a dot
product?

730
00:44:17 --> 00:44:24
So, let's just remember for
vectors v trans- v transpose w

731
00:44:24 --> 00:44:32
for vectors, so this was
vectors, v transpose w was v1w1

732
00:44:32 --> 00:44:33.59
+...+vnwn.

733
00:44:33.59 --> 00:44:34
Okay.

734
00:44:34 --> 00:44:36
Now functions.

735
00:44:36 --> 00:44:42
Now I have two functions,
let's call them f and g.

736
00:44:42 --> 00:44:44
What's with them now?

737
00:44:44 --> 00:44:49
The vectors had n components,
but the functions have a whole,

738
00:44:49 --> 00:44:50
like, continuum.

739
00:44:50 --> 00:44:54
To graph the function,
I just don't have n points,

740
00:44:54 --> 00:44:56
I've got this whole graph.

741
00:44:56 --> 00:45:01
So I have functions -- I'm
really trying to ask you what's

742
00:45:01 --> 00:45:07
the inner product of this
function f with another function

743
00:45:07 --> 00:45:07
g?

744
00:45:07 --> 00:45:13
And I want to make it parallel
to this the best I can.

745
00:45:13 --> 00:45:18
So the best parallel is to
multiply f (x) times g(x) at

746
00:45:18 --> 00:45:23.23
every x -- and here I just had n
multiplications,

747
00:45:23.23 --> 00:45:27
but here I'm going to have a
whole range of x-s,

748
00:45:27 --> 00:45:31
and here
I added the results.

749
00:45:31 --> 00:45:33.48
What do I do here?

750
00:45:33.48 --> 00:45:38
So what's the analog of
addition when you have -- when

751
00:45:38 --> 00:45:40
you're in a continuum?

752
00:45:40 --> 00:45:42
It's integration.

753
00:45:42 --> 00:45:47
So that the -- the dot product
of two functions will be the

754
00:45:47 --> 00:45:51
integral of those functions,
dx.

755
00:45:51 --> 00:45:56
Now I have to say -- say,
well, what are the limits of

756
00:45:56 --> 00:45:57
integration?

757
00:45:57 --> 00:46:02.01
And for this Fourier series,
this function f(x) -- if I'm

758
00:46:02.01 --> 00:46:07
going to have -- if that right
hand side is going to be f(x),

759
00:46:07 --> 00:46:10
that function that I'm seeing
on the right,

760
00:46:10 --> 00:46:13
all those
sines and cosines,

761
00:46:13 --> 00:46:17
they're all periodic,
with -- with period two pi.

762
00:46:17 --> 00:46:20
So -- so that's what f(x) had
better be.

763
00:46:20 --> 00:46:23
So I'll integrate from zero to
two pi.

764
00:46:23 --> 00:46:28
My -- all -- everything -- is
on the interval zero two pi

765
00:46:28 --> 00:46:35
now, because if I'm going to
use these sines and cosines,

766
00:46:35 --> 00:46:39
then f(x) is equal to f(x+2pi).

767
00:46:39 --> 00:46:44
This is periodic -- periodic
functions.

768
00:46:44 --> 00:46:45
Okay.

769
00:46:45 --> 00:46:51
So now I know what -- I've got
all the right words now.

770
00:46:51 --> 00:47:00
I've got a vector space,
but the vectors are functions.

771
00:47:00 --> 00:47:04
I've got inner products and --
and the inner product gives a

772
00:47:04 --> 00:47:06
number, all right.

773
00:47:06 --> 00:47:09
It just happens to be an
integral instead of a sum.

774
00:47:09 --> 00:47:14
I've got -- and that -- then I
have the idea of orthogonality

775
00:47:14 --> 00:47:18
-- because, actually,
just -- let's just check.

776
00:47:18 --> 00:47:25
Orthogonality -- if I take the
integral -- s- I -- let me do

777
00:47:25 --> 00:47:32
sin(x) times cos(x) -- sin(x)
times cos(x) dx from zero to two

778
00:47:32 --> 00:47:35
pi -- I think we get zero.

779
00:47:35 --> 00:47:41
That's the differential of
that, so it would be one half

780
00:47:41 --> 00:47:45.29
sine x squared,
was that right?

781
00:47:45.29 --> 00:47:50
Between zero and two pi --
and, of course,

782
00:47:50 --> 00:47:52
we get zero.

783
00:47:52 --> 00:47:57.65
And the same would be true with
a little more -- some trig

784
00:47:57.65 --> 00:48:02
identities to help us out -- of
every other pair.

785
00:48:02 --> 00:48:07.81
So we have now an orthonormal
infinite basis for function

786
00:48:07.81 --> 00:48:13
space, and all we want to do is
express a function in that

787
00:48:13 --> 00:48:13
basis.

788
00:48:13 --> 00:48:18
And so I --
the end of my lecture is,

789
00:48:18 --> 00:48:19
okay, what is a1?

790
00:48:19 --> 00:48:24
What's the coefficient -- how
much cos(x) is there in a

791
00:48:24 --> 00:48:27
function compared to the other
harmonics?

792
00:48:27 --> 00:48:31
How much constant is in that
function?

793
00:48:31 --> 00:48:35
That'll -- that would be an
easy question.

794
00:48:35 --> 00:48:39
The answer a0 will come out to
be the average value of f.

795
00:48:39 --> 00:48:42
That's the amount of the
constant that's in there,

796
00:48:42 --> 00:48:43
its average value.

797
00:48:43 --> 00:48:45
But let's take a1 as more
typical.

798
00:48:45 --> 00:48:49
How will I get -- here's the
end of the lecture,

799
00:48:49 --> 00:48:50
then -- how do I get a1?

800
00:48:50 --> 00:48:53
The first Fourier coefficient.

801
00:48:53 --> 00:48:54
Okay.

802
00:48:54 --> 00:48:57
I do just as I did in the
vector case.

803
00:48:57 --> 00:49:03
I take the inner product of
everything with cos(x) Take the

804
00:49:03 --> 00:49:06.96
inner product of everything with
cos(x).

805
00:49:06.96 --> 00:49:12
Then on the left -- on the left
I have -- the inner product is

806
00:49:12 --> 00:49:16.92
the integral of
f(x) times cos(x) cx.

807
00:49:16.92 --> 00:49:19
And on the right,
what do I have?

808
00:49:19 --> 00:49:24
When I -- so what I -- when I
say take the inner product with

809
00:49:24 --> 00:49:27
cos(x), let me put it in
ordinary calculus words.

810
00:49:27 --> 00:49:30
Multiply by cos(x) and
integrate.

811
00:49:30 --> 00:49:32
That's what inner products are.

812
00:49:32 --> 00:49:36
So if I multiply that whole
thing by

813
00:49:36 --> 00:49:41
cos(x) and I integrate,
I get a whole lot of zeroes.

814
00:49:41 --> 00:49:45
The only thing that survives is
that term.

815
00:49:45 --> 00:49:47
All the others disappear.

816
00:49:47 --> 00:49:53
So -- and that term is a1 times
the integral of cos(x) squared

817
00:49:53 --> 00:49:58
dx zero to 2pi equals -- so this
was the left side and this is

818
00:49:58 --> 00:50:03.2
all that's left on
the right-hand side.

819
00:50:03.2 --> 00:50:08
And this is not zero of course,
because it's the length of the

820
00:50:08 --> 00:50:13
function squared,
it's the inner product with

821
00:50:13 --> 00:50:18
itself, and -- and a simple
calculation gives that answer to

822
00:50:18 --> 00:50:19
be pi.

823
00:50:19 --> 00:50:23.95
So that's an easy integral and
it turns out to be pi,

824
00:50:23.95 --> 00:50:30
so that a1 is one over pi times
there -- times this integral.

825
00:50:30 --> 00:50:34
So there is,
actually -- that's Euler's

826
00:50:34 --> 00:50:39
famous formula for the -- or
maybe Fourier found it -- for

827
00:50:39 --> 00:50:43
the coefficients in a Fourier
series.

828
00:50:43 --> 00:50:48.8
And you see that it's exactly
an expansion in an orthonormal

829
00:50:48.8 --> 00:50:49
basis.

830
00:50:49 --> 00:50:51
Okay, thanks.

831
00:50:51 --> 00:50:57
So I'll do a quiz review on
Monday and then the quiz itself

832
00:50:57 --> 00:50:59
in Walker on Wednesday.

833
00:50:59 --> 00:51:01
Okay, see you Monday.

834
00:51:01 --> 00:51:04
Thanks.