1
00:00:00 --> 00:00:01
 
 

2
00:00:01 --> 00:00:02
The following content is
provided under a Creative

3
00:00:02 --> 00:00:03
Commons license.
 

4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:09
offer high-quality educational
resources for free.

6
00:00:09 --> 00:00:13
To make a donation, or to view
additional materials from

7
00:00:13 --> 00:00:15
hundreds of MIT courses, visit
MIT OpenCourseWare

8
00:00:15 --> 00:00:20
at ocw.mit.edu.
 

9
00:00:20 --> 00:00:25
PROFESSOR STRANG: OK, so I
thought I'd quickly list the

10
00:00:25 --> 00:00:29
four central topics for the
Fourier part of the course, and

11
00:00:29 --> 00:00:32
I would guess that there'll be
a question on each of those

12
00:00:32 --> 00:00:36
four in the final quiz.
 

13
00:00:36 --> 00:00:39
Final exam.
 

14
00:00:39 --> 00:00:43
And we covered 4.1,
Fourier series.

15
00:00:43 --> 00:00:50
I don't plan to do discussion
in 4.2 of other series like

16
00:00:50 --> 00:00:53
Bessel and Legendre and so on.
 

17
00:00:53 --> 00:00:56
If I can, I'll come
back to those in the

18
00:00:56 --> 00:00:58
very last lectures.
 

19
00:00:58 --> 00:01:06
But I want to pick up now on
the second key example, which

20
00:01:06 --> 00:01:10
is the discrete Fourier series
that has only N terms.

21
00:01:10 --> 00:01:13
Instead this series, as
you're looking at it, has

22
00:01:13 --> 00:01:15
infinitely many terms.
 

23
00:01:15 --> 00:01:20
But I'm going to cut
back to n terms.

24
00:01:20 --> 00:01:25
So this K, well I can go
one to N, but the best

25
00:01:25 --> 00:01:27
way is zero to N-1.
 

26
00:01:29 --> 00:01:34
So that's the discrete series.
 

27
00:01:34 --> 00:01:38
We're going to be dealing with
a vector, c_0 up to c_(n-1).

28
00:01:40 --> 00:01:44
And a vector of
function values.

29
00:01:44 --> 00:01:46
Vector to vector.
 

30
00:01:46 --> 00:01:49
So a matrix is going to be
the key to everything here.

31
00:01:49 --> 00:01:51
An n by n matrix.
 

32
00:01:51 --> 00:01:52
So that's what's coming.
 

33
00:01:52 --> 00:01:58
Then, after that probably
by Friday will be the

34
00:01:58 --> 00:01:59
integral transform.
 

35
00:01:59 --> 00:02:04
And early next week, and a
little after Thanksgiving

36
00:02:04 --> 00:02:08
would be this key idea of
convolution, which is really

37
00:02:08 --> 00:02:12
important in so many
applications and needs

38
00:02:12 --> 00:02:14
to be separated out.
 

39
00:02:14 --> 00:02:17
Convolution will apply
to all of these.

40
00:02:17 --> 00:02:20
And you'll see the point there.
 

41
00:02:20 --> 00:02:24
OK, ready for the idea
of the discrete one.

42
00:02:24 --> 00:02:28
OK, so the key idea is we only
have N terms, so I can't expect

43
00:02:28 --> 00:02:30
to reproduce a whole function.
 

44
00:02:30 --> 00:02:35
I can get n function value,
so I'll get this at

45
00:02:35 --> 00:02:37
n different points.
 

46
00:02:37 --> 00:02:42
And if I'm thinking of my F(x),
on, let me take zero to 2pi

47
00:02:42 --> 00:02:47
now, and I'm thinking again
of an F(x) that's periodic.

48
00:02:47 --> 00:02:51
So it goes, does whatever.
 

49
00:02:51 --> 00:02:57
If it was like that, if that
was my F(x), just remember

50
00:02:57 --> 00:03:00
about Fourier series, what
could you tell me about the

51
00:03:00 --> 00:03:02
Fourier series for that F(x)?
 

52
00:03:03 --> 00:03:07
Well, when I continue,
periodically am I

53
00:03:07 --> 00:03:09
going to see a jump?
 

54
00:03:09 --> 00:03:10
Yes.
 

55
00:03:10 --> 00:03:18
There'll be a jump at every
2pi, so the underlying function

56
00:03:18 --> 00:03:28
is not continuous, has a jump,
so that our special examples

57
00:03:28 --> 00:03:34
with jumps and slow decay, 1/k
decay rate for the Fourier

58
00:03:34 --> 00:03:36
coefficients would apply here.
 

59
00:03:36 --> 00:03:39
OK, that's remembering
Fourier series.

60
00:03:39 --> 00:03:42
Now, Fourier integrals,
I'm just going to take,

61
00:03:42 --> 00:03:43
let me just take N=4.
 

62
00:03:43 --> 00:03:46
 
 

63
00:03:46 --> 00:03:49
So I'll take this point,
you know what four points

64
00:03:49 --> 00:03:50
am I going to take?
 

65
00:03:50 --> 00:03:54
I'm not going to repeat, ah.
 

66
00:03:54 --> 00:04:00
Just for the heck of it,
let's make it look periodic.

67
00:04:00 --> 00:04:02
Yeah, let's make
it look periodic.

68
00:04:02 --> 00:04:04
I never should have
done it otherwise.

69
00:04:04 --> 00:04:08
OK, so there's one point.
 

70
00:04:08 --> 00:04:12
I'm just going to take four
equally spaced, equally

71
00:04:12 --> 00:04:17
spaced being key,
zero, one, two, three.

72
00:04:17 --> 00:04:21
So N-1 is three.
 

73
00:04:21 --> 00:04:28
And these four values
I can get right.

74
00:04:28 --> 00:04:30
So those are my four x's.
 

75
00:04:30 --> 00:04:33
The four x's that
I'm going to take.

76
00:04:33 --> 00:04:36
So this is x=0, of course.
 

77
00:04:36 --> 00:04:41
And this one is, what's
the x there? x is

78
00:04:41 --> 00:04:43
2pi, the whole deal.
 

79
00:04:43 --> 00:04:46
The whole interval, but
it's divided into N

80
00:04:46 --> 00:04:47
pieces, so 2pi/N.
 

81
00:04:47 --> 00:04:50
 
 

82
00:04:50 --> 00:04:54
So we are going to see a lot
of fractions involving pi/N.

83
00:04:55 --> 00:04:56
Or 2pi/N.
 

84
00:04:57 --> 00:04:59
They're just going to
come up everywhere.

85
00:04:59 --> 00:05:04
Because that's the delta
x, the delta h, the step

86
00:05:04 --> 00:05:08
in the discrete form.
 

87
00:05:08 --> 00:05:08
OK.
 

88
00:05:08 --> 00:05:11
So those are the x's.
 

89
00:05:11 --> 00:05:22
So I'm only going to get this
at x=0, 2pi/N, 4pi/N, up to

90
00:05:22 --> 00:05:24
whatever it comes out
to, (N-1)*2pi/N.

91
00:05:24 --> 00:05:28
 
 

92
00:05:28 --> 00:05:31
That doesn't look elegant
so we probably will try to

93
00:05:31 --> 00:05:32
avoid writing that again.
 

94
00:05:32 --> 00:05:39
But this was the three times
2pi/4 that got us to there.

95
00:05:39 --> 00:05:45
So this is 2pi/8, pi/4.
 

96
00:05:46 --> 00:05:50
Sorry, 2pi/4, 4pi/4,
6pi - oh, ok.

97
00:05:50 --> 00:05:54
So we're only going to get
equality at those x's.

98
00:05:54 --> 00:05:58
So it's those x's that
I should plug in here.

99
00:05:58 --> 00:06:00
OK, so let me do that.
 

100
00:06:00 --> 00:06:05
Let me plus in, what does
this series look like

101
00:06:05 --> 00:06:09
at these four points?
 

102
00:06:09 --> 00:06:10
Let me see.
 

103
00:06:10 --> 00:06:17
So writing out all four
equations, F(0) is going

104
00:06:17 --> 00:06:18
to match the right side.
 

105
00:06:18 --> 00:06:21
At zero, because zero
is one of my points.

106
00:06:21 --> 00:06:23
So I get c_0+c_1+c_2+c_3.
 

107
00:06:30 --> 00:06:35
At x=0, right, all the e to
the whatevers were one.

108
00:06:35 --> 00:06:44
Now at the next x, 2pi/N, now
I'm matching the series with a

109
00:06:44 --> 00:06:46
function at that value, there.
 

110
00:06:46 --> 00:06:46
2pi/N.
 

111
00:06:48 --> 00:06:54
So now I have k=0, so
what does k=0 give me?

112
00:06:54 --> 00:06:58
When k is zero, what's
my term look like?

113
00:06:58 --> 00:07:03
It's just c_0 times one.
 

114
00:07:03 --> 00:07:07
Because if k is zero, e to
the whatever is again one.

115
00:07:07 --> 00:07:09
So this is just c_0.
 

116
00:07:09 --> 00:07:11
Now, here's the key.
 

117
00:07:11 --> 00:07:17
What is that business
there at x=2pi/N?

118
00:07:19 --> 00:07:22
Here comes the important number
in this whole business.

119
00:07:22 --> 00:07:25
The important number this
whole business is, I'll

120
00:07:25 --> 00:07:28
call it w, is e^(2pi*i)/N.
 

121
00:07:28 --> 00:07:33
 
 

122
00:07:33 --> 00:07:40
That's a complex number,
and we'll place it in

123
00:07:40 --> 00:07:42
the complex plane.
 

124
00:07:42 --> 00:07:46
So you must get to
know that number. w.

125
00:07:46 --> 00:07:48
Because when I plug in x equal
 

126
00:07:48 --> 00:07:51
- k is now one here,
so I have c_1.

127
00:07:53 --> 00:08:01
Times this e to the i one,
2pi/N, what is that? w.

128
00:08:01 --> 00:08:02
So it's w*c_1.
 

129
00:08:04 --> 00:08:08
And what do I get from the
next term in this series?

130
00:08:08 --> 00:08:08
I have c_2*e^(i2x).
 

131
00:08:08 --> 00:08:14
 
 

132
00:08:14 --> 00:08:20
x is 2pi/N, do you see
what's happening? w

133
00:08:20 --> 00:08:22
squared is e^(4pi*i/N).
 

134
00:08:22 --> 00:08:26
 
 

135
00:08:26 --> 00:08:29
We're using the beauty
of exponentials, that's

136
00:08:29 --> 00:08:30
making everything go.
 

137
00:08:30 --> 00:08:32
It's w squared.
 

138
00:08:32 --> 00:08:35
And this guy would
be w cubed, c_3.

139
00:08:35 --> 00:08:38
 
 

140
00:08:38 --> 00:08:41
That's the series.
 

141
00:08:41 --> 00:08:45
At our points.
 

142
00:08:45 --> 00:08:48
So this, like, we'll
just see this once.

143
00:08:48 --> 00:08:49
What were we doing?
 

144
00:08:49 --> 00:08:51
But this is what we're
going to live with here.

145
00:08:51 --> 00:08:57
So I'll rewrite it all in terms
of w, so you just, you've got

146
00:08:57 --> 00:08:58
the idea of what are we doing?
 

147
00:08:58 --> 00:09:04
We're matching at N points and
now we get the good notation w.

148
00:09:04 --> 00:09:07
OK, and at the next
point, 4pi/N.

149
00:09:09 --> 00:09:12
Well, again, k=0
term, it's c_0.

150
00:09:13 --> 00:09:19
The k=1 term, can you tell me
what I get when k is one, I

151
00:09:19 --> 00:09:30
have something times c_1, it's
e^(i*4pi/N), what's that? w

152
00:09:30 --> 00:09:35
squared. e^(i*4pi/N)
is w squared.

153
00:09:35 --> 00:09:39
Look, this matrix is
coming out symmetric.

154
00:09:39 --> 00:09:45
And this will be w^4, and
that'll turn out to be w^6, and

155
00:09:45 --> 00:09:50
then the last row of the matrix
is going to come from matching

156
00:09:50 --> 00:09:56
at 6pi/N, and it will turn
out to be c_0, and it'll

157
00:09:56 --> 00:10:02
be w^3, w^6, and w^9.
 

158
00:10:02 --> 00:10:06
 
 

159
00:10:06 --> 00:10:10
So I did the last row
or two a bit fast.

160
00:10:10 --> 00:10:15
Just because the patterns
there, and you'll spot it.

161
00:10:15 --> 00:10:22
Now I want to pull
out of the matrix.

162
00:10:22 --> 00:10:26
The Fourier matrix, the four
by four Fourier matrix

163
00:10:26 --> 00:10:29
that multiplies the c's
and gives me the F's.

164
00:10:29 --> 00:10:36
Let me call these numbers are
y_0, y_1, y_2, and y_3, just

165
00:10:36 --> 00:10:41
to have a, so it's a vector
of values of the function.

166
00:10:41 --> 00:10:43
And this is a vector
of coefficients.

167
00:10:43 --> 00:10:47
So it's wise to - the Fourier
transform goes between

168
00:10:47 --> 00:10:56
y's and c's, and y's.
 

169
00:10:56 --> 00:11:02
Connects the vector, and this
is N values, N function

170
00:11:02 --> 00:11:06
values in physical space.
 

171
00:11:06 --> 00:11:11
These are N coefficients in
frequency space, and one way is

172
00:11:11 --> 00:11:14
the discrete Fourier transform
and the other way is

173
00:11:14 --> 00:11:17
the inverse discrete
Fourier transform.

174
00:11:17 --> 00:11:20
So, and it's a little
bit confused, which

175
00:11:20 --> 00:11:22
is which, actually.
 

176
00:11:22 --> 00:11:25
There was no question with
Fourier series, it was easy to

177
00:11:25 --> 00:11:28
tell the function from the
coefficients because the

178
00:11:28 --> 00:11:30
function was f x and the
coefficients were a

179
00:11:30 --> 00:11:31
bunch of numbers.
 

180
00:11:31 --> 00:11:34
Here we only have N numbers.
 

181
00:11:34 --> 00:11:40
N y's and N c's and we're
transforming back and forth.

182
00:11:40 --> 00:11:45
So I'm finding right now the
matrix, this Fourier matrix

183
00:11:45 --> 00:11:48
is going to take the c's
and give me the y's.

184
00:11:48 --> 00:11:52
So I'm talking here
about the matrix F.

185
00:11:52 --> 00:11:54
And let's see what F is.
 

186
00:11:54 --> 00:12:04
Can you just see what, that was
y_0, to y_3, equals this matrix

187
00:12:04 --> 00:12:06
F, the four by four matrix.
 

188
00:12:06 --> 00:12:08
Times c_0 to c_3.
 

189
00:12:10 --> 00:12:14
This is really the inverse.
 

190
00:12:14 --> 00:12:14
DFT.
 

191
00:12:14 --> 00:12:20
 
 

192
00:12:20 --> 00:12:27
It reconstructs the y's from
the c's, or an easier way to

193
00:12:27 --> 00:12:32
say that is add up the series.
 

194
00:12:32 --> 00:12:39
So that's the usual DFT is
taking us, starting with the

195
00:12:39 --> 00:12:41
y's and giving us the c's.
 

196
00:12:41 --> 00:12:46
So this would be the
discrete Fourier transform.

197
00:12:46 --> 00:12:50
Take a function, produce
its coefficients.

198
00:12:50 --> 00:12:56
The reverse step is take the
coefficients, add back, add up

199
00:12:56 --> 00:12:59
the series, get the function.
 

200
00:12:59 --> 00:13:02
And that's what F does.
 

201
00:13:02 --> 00:13:05
So I'm going to focus on F.
 

202
00:13:05 --> 00:13:09
Which is the ones that
involves this w guy.

203
00:13:09 --> 00:13:13
OK, can you read off, peel off
from here what the matrix

204
00:13:13 --> 00:13:15
F is, the Fourier matrix?
 

205
00:13:15 --> 00:13:20
Its first row is all ones.
 

206
00:13:20 --> 00:13:24
Its first column is all ones.
 

207
00:13:24 --> 00:13:31
And then it's one, w,
w^2, w^3, w^2, w^4,

208
00:13:31 --> 00:13:35
w^6, w^3, w^6, and w^9.
 

209
00:13:37 --> 00:13:39
To the ninth power.
 

210
00:13:39 --> 00:13:43
OK, so it's made up of
these powers of w.

211
00:13:43 --> 00:13:49
Zero first, second, and third
power and then higher powers.

212
00:13:49 --> 00:13:52
But higher powers are -
let's draw a picture.

213
00:13:52 --> 00:13:54
Where is w?
 

214
00:13:54 --> 00:13:57
So this is the complex plane.
 

215
00:13:57 --> 00:14:00
Here's the real, and here's
the imaginary direction.

216
00:14:00 --> 00:14:05
And let me take N to
be, here N was four.

217
00:14:05 --> 00:14:10
Let me take N to be eight in my
picture, just to have some,

218
00:14:10 --> 00:14:13
then I'll be able to spot
the fours and the eights.

219
00:14:13 --> 00:14:16
If N is eight, where
would w^8 be?

220
00:14:16 --> 00:14:32
So w^8 is meant to be
the e^(2pi*i/8), now.

221
00:14:32 --> 00:14:38
Where's that number
in that picture?

222
00:14:38 --> 00:14:43
Is it on the circle?
 

223
00:14:43 --> 00:14:46
Absolutely.
 

224
00:14:46 --> 00:14:51
This discrete Fourier transform
never gets off the circle.

225
00:14:51 --> 00:14:55
And in fact, it never gets off
eight points on the circle.

226
00:14:55 --> 00:14:58
Which are the eight
equally spaced points.

227
00:14:58 --> 00:15:02
This says go 1/8 of the way
of the whole way around,

228
00:15:02 --> 00:15:04
which will be here.
 

229
00:15:04 --> 00:15:05
So there's w^8.
 

230
00:15:05 --> 00:15:08
 
 

231
00:15:08 --> 00:15:11
When I square it, what's
the square of w^8?

232
00:15:12 --> 00:15:13
The angle.
 

233
00:15:13 --> 00:15:17
What happens to the angle
when I multiply numbers?

234
00:15:17 --> 00:15:19
I add the exponents.
 

235
00:15:19 --> 00:15:22
Here, if I'm multiplying
by itself, it'll

236
00:15:22 --> 00:15:23
double the exponent.
 

237
00:15:23 --> 00:15:24
Double the angle.
 

238
00:15:24 --> 00:15:34
It'll be, there's w^8 squared,
so that point is w^8 squared.

239
00:15:34 --> 00:15:37
And it's the same as
w^4 because it's 1/4

240
00:15:37 --> 00:15:39
of the way around.
 

241
00:15:39 --> 00:15:45
And what is that number
in ordinary language? i.

242
00:15:45 --> 00:15:47
It's i, right?
 

243
00:15:47 --> 00:15:53
It's on the unit
circle at 90 degrees.

244
00:15:53 --> 00:15:56
And on the imaginary
axis it's just i.

245
00:15:56 --> 00:15:59
So all this here
was i, i^2 i^3.

246
00:16:00 --> 00:16:01
i^2, i^4, i^6.
 

247
00:16:02 --> 00:16:02
i^3, i^6, i^9.
 

248
00:16:04 --> 00:16:09
The four by four Fourier
matrix is just powers of i.

249
00:16:09 --> 00:16:12
And the N by N, the eight
by eight Fourier matrix

250
00:16:12 --> 00:16:13
is powers of w^8.
 

251
00:16:15 --> 00:16:16
And let's find all the
rest of the powers.

252
00:16:16 --> 00:16:21
So there's w^8, w^8 squared,
where's w^8 cubed?

253
00:16:21 --> 00:16:22
Here.
 

254
00:16:22 --> 00:16:25
I multiply those guys, I add
the angle, I'm out to here.

255
00:16:25 --> 00:16:28
Here's the fourth power, here's
the fifth power, sixth power,

256
00:16:28 --> 00:16:34
seventh power and finally we
get to w^8, to the eighth

257
00:16:34 --> 00:16:37
power, which is one.
 

258
00:16:37 --> 00:16:40
Because if I take the eighth
power here, I've got

259
00:16:40 --> 00:16:45
e^(2pi*8), and that's one.
 

260
00:16:45 --> 00:16:50
So that's the picture
to remember.

261
00:16:50 --> 00:16:57
And while looking at that
picture, could you tell me the

262
00:16:57 --> 00:17:00
sum of those eight numbers?
 

263
00:17:00 --> 00:17:03
What do those eight
members add up to?

264
00:17:03 --> 00:17:13
So I just want to ask what do
w^1 plus w up to w^7, when I'm

265
00:17:13 --> 00:17:17
doing the eighth
guy, adds up to?

266
00:17:17 --> 00:17:21
So the sum of those eight
numbers, well I wouldn't ask

267
00:17:21 --> 00:17:24
the question if it didn't
have a nice answer.

268
00:17:24 --> 00:17:24
Zero.
 

269
00:17:24 --> 00:17:29
That's of course the
usual answer in math.

270
00:17:29 --> 00:17:34
We do all this work
and we get zero.

271
00:17:34 --> 00:17:37
But now, of course the
real math is why.

272
00:17:37 --> 00:17:43
I mean, so math is, you get
equations but then always

273
00:17:43 --> 00:17:45
there's that kicker, why.
 

274
00:17:45 --> 00:17:48
And why do they add up to zero?
 

275
00:17:48 --> 00:17:51
Here.
 

276
00:17:51 --> 00:17:54
Can you see that?
 

277
00:17:54 --> 00:18:00
Why those eight numbers
would add to zero?

278
00:18:00 --> 00:18:02
Well, yeah.
 

279
00:18:02 --> 00:18:04
So what do these add up to?
 

280
00:18:04 --> 00:18:06
Just those two guys?
 

281
00:18:06 --> 00:18:09
One and minus one add to zero.
 

282
00:18:09 --> 00:18:13
That and that add to zero,
I can pair them off.

283
00:18:13 --> 00:18:15
This and this add to zero.
 

284
00:18:15 --> 00:18:17
That and that add to zero.
 

285
00:18:17 --> 00:18:19
Yeah, so that's one
way of seeing it.

286
00:18:19 --> 00:18:24
But actually, even if N was
three or five or some odd

287
00:18:24 --> 00:18:27
number, and I had - so should
I do the picture for N=3?

288
00:18:28 --> 00:18:31
Just a little, doesn't need
a very big picture for N=3.

289
00:18:32 --> 00:18:34
So where would the
roots be for N=3?

290
00:18:35 --> 00:18:37
One of them is always one.
 

291
00:18:37 --> 00:18:41
These are the three
cube roots of one.

292
00:18:41 --> 00:18:44
That I'm going to draw.
 

293
00:18:44 --> 00:18:48
Those were the eight
eighth roots of one.

294
00:18:48 --> 00:18:51
Because if I take any one of
those numbers its eighth

295
00:18:51 --> 00:18:55
power brings me back to one.
 

296
00:18:55 --> 00:18:59
So where are these guys, so
this is three numbers equally

297
00:18:59 --> 00:19:00
spaced around the circle.
 

298
00:19:00 --> 00:19:02
Where do you think they are?
 

299
00:19:02 --> 00:19:10
Well, it's gotta be at 120
degrees, 240 degrees and 360.

300
00:19:10 --> 00:19:14
And again, those
will add to zero.

301
00:19:14 --> 00:19:18
You can't pair them off because
we got an odd number, but it's

302
00:19:18 --> 00:19:23
safely zero and we
could see why.

303
00:19:23 --> 00:19:33
OK, so this section, this topic
is all about that matrix.

304
00:19:33 --> 00:19:37
It's all about that four by
four, or N by N matrix.

305
00:19:37 --> 00:19:39
Which has powers of w.
 

306
00:19:39 --> 00:19:43
Notice that the matrix is,
it hasn't got any zeroes.

307
00:19:43 --> 00:19:46
Hasn't even got anything
close to zero.

308
00:19:46 --> 00:19:50
All those numbers are
on the unit circle.

309
00:19:50 --> 00:19:58
So multiplying by that matrix,
which is the same as adding up

310
00:19:58 --> 00:20:06
the Fourier series at the four
points, doing this calculation,

311
00:20:06 --> 00:20:10
finding these y's from the c's,
reconstructing the y's, given

312
00:20:10 --> 00:20:16
the coefficients, has 16 terms.
 

313
00:20:16 --> 00:20:18
Right?
 

314
00:20:18 --> 00:20:22
The matrix has got 16 entries,
all the entries in the matrix

315
00:20:22 --> 00:20:25
are showing up here
in powers of w.

316
00:20:25 --> 00:20:28
We've got 16 multiplications.
 

317
00:20:28 --> 00:20:33
Well, maybe a couple of them
were easy, but basically 16

318
00:20:33 --> 00:20:36
multiplications and additions.
 

319
00:20:36 --> 00:20:39
Pretty much N squared work.
 

320
00:20:39 --> 00:20:50
And that's not bad.
 

321
00:20:50 --> 00:20:54
Except if you want to do it
a million times, right?

322
00:20:54 --> 00:20:58
Suppose we have a typical
N might be 1024.

323
00:20:58 --> 00:21:03
So the matrix has then got,
is 1024 squared numbers,

324
00:21:03 --> 00:21:05
which is about a million.
 

325
00:21:05 --> 00:21:09
So if we have a matrix with a
million entries, a million

326
00:21:09 --> 00:21:16
operations to do, and then we
do it a million times then it's

327
00:21:16 --> 00:21:21
getting up to money that
Congress would give

328
00:21:21 --> 00:21:23
to the banks.
 

329
00:21:23 --> 00:21:24
Whatever.
 

330
00:21:24 --> 00:21:25
Anyway.
 

331
00:21:25 --> 00:21:27
Serious money.
 

332
00:21:27 --> 00:21:31
Serious computing time.
 

333
00:21:31 --> 00:21:39
And the wonderful thing is
that there is a faster

334
00:21:39 --> 00:21:43
way than n squared.
 

335
00:21:43 --> 00:21:48
There's a faster way
to do these 16 terms

336
00:21:48 --> 00:21:50
than the obvious way.
 

337
00:21:50 --> 00:21:51
You might say, well,
they're pretty nice.

338
00:21:51 --> 00:21:53
And they are.
 

339
00:21:53 --> 00:21:55
But they're nicer
than you can know.

340
00:21:55 --> 00:21:58
I mean, nicer than you
can see immediately.

341
00:21:58 --> 00:22:08
This matrix will break up into
simple steps with lots of

342
00:22:08 --> 00:22:15
zeroes in each step, and the
final result is that instead of

343
00:22:15 --> 00:22:19
n squared, capital N squared,
which is done the old

344
00:22:19 --> 00:22:22
fashioned way, let's say.
 

345
00:22:22 --> 00:22:26
That that there's a fast
Fourier transform.

346
00:22:26 --> 00:22:33
So the FFT, I should maybe find
a better space for the most

347
00:22:33 --> 00:22:42
important algorithm in the last
hundred years, and here it is.

348
00:22:42 --> 00:22:45
Right there.
 

349
00:22:45 --> 00:22:49
So my one goal for Wednesday
will be to give you some

350
00:22:49 --> 00:22:52
insight into the FFT.
 

351
00:22:52 --> 00:22:55
But here I'm just
looking at the result.

352
00:22:55 --> 00:22:59
The number of computations
is, instead of N squared,

353
00:22:59 --> 00:23:03
it's N times log N.
 

354
00:23:03 --> 00:23:08
It's log to the base two,
or maybe 1/2 N log N.

355
00:23:08 --> 00:23:11
So I'll erase some of the
other things close by

356
00:23:11 --> 00:23:14
so you can see it.
 

357
00:23:14 --> 00:23:20
So that, where n squared was,
if N was a thousand, N squared

358
00:23:20 --> 00:23:26
was a million, If N is a
thousand, this is a thousand,

359
00:23:26 --> 00:23:30
and what's the logarithm of a
thousand, to the base two?

360
00:23:30 --> 00:23:33
That's the ten, right?
 

361
00:23:33 --> 00:23:36
The 2^10 gave us that 1,024.
 

362
00:23:36 --> 00:23:37
So this is 10^4.
 

363
00:23:39 --> 00:23:45
So N is 10^3, the logarithm
of 1,024 more exactly, the

364
00:23:45 --> 00:23:48
logarithm is ten, so we're
10^4 instead of 10^6.

365
00:23:51 --> 00:23:54
So that's a saving of a
factor of 100 that's true.

366
00:23:54 --> 00:24:00
I mean it just comes from doing
the addition and multiplication

367
00:24:00 --> 00:24:02
in the right order.
 

368
00:24:02 --> 00:24:05
You get this incredible saving.
 

369
00:24:05 --> 00:24:08
And it's sort of makes whole
calculations that were

370
00:24:08 --> 00:24:16
previously impossible are now
possible because instead of

371
00:24:16 --> 00:24:24
where it might have been 100
minutes, it's now one minute.

372
00:24:24 --> 00:24:29
So everything focuses on this
Fourier matrix and its inverse,

373
00:24:29 --> 00:24:35
which we still have to find but
it'll come out beautifully.

374
00:24:35 --> 00:24:38
In fact, I could tell you
what F inverse, just

375
00:24:38 --> 00:24:41
so you see it coming.
 

376
00:24:41 --> 00:24:45
F inverse will, so what
happened to w, that

377
00:24:45 --> 00:24:48
all-important number w?
 

378
00:24:48 --> 00:24:59
Well, let me repeat what w is.
 

379
00:24:59 --> 00:25:02
I'll put it on this board.
 

380
00:25:02 --> 00:25:06
F has powers of w.
w is e^(2pi*i/N).

381
00:25:06 --> 00:25:12
 
 

382
00:25:12 --> 00:25:14
And what power has it got?
 

383
00:25:14 --> 00:25:19
In the j,k position?
 

384
00:25:19 --> 00:25:25
So now you can see the
formula for these entries.

385
00:25:25 --> 00:25:29
Only, you have to let me
start the count at j

386
00:25:29 --> 00:25:31
and k equals zero.
 

387
00:25:31 --> 00:25:35
This is zero based, where
MATLAB is one based.

388
00:25:35 --> 00:25:40
And to do this stuff you always
have to shift things by one.

389
00:25:40 --> 00:25:44
Some other, more recent,
software like Python on

390
00:25:44 --> 00:25:50
is zero based, because
this is so common.

391
00:25:50 --> 00:25:54
So here's the formula.
 

392
00:25:54 --> 00:25:59
It's w to the power j times k.
 

393
00:25:59 --> 00:26:01
That's neat.
 

394
00:26:01 --> 00:26:06
That's what goes
into the matrix F.

395
00:26:06 --> 00:26:09
This here is row three.
 

396
00:26:09 --> 00:26:11
It looks like row four but
I'm starting with zero,

397
00:26:11 --> 00:26:13
so that's row three.
 

398
00:26:13 --> 00:26:14
That's column three.
 

399
00:26:14 --> 00:26:21
This is w^3 times three.
w^9, which by the way

400
00:26:21 --> 00:26:23
is the same as what?
 

401
00:26:23 --> 00:26:32
If w is i in this four by four,
it is. w is i, what's w^9,

402
00:26:32 --> 00:26:36
what's i to the 9th power?
 

403
00:26:36 --> 00:26:38
Same as?
 

404
00:26:38 --> 00:26:41
Can you do i^9 in your head?
 

405
00:26:41 --> 00:26:42
Sure.
 

406
00:26:42 --> 00:26:45
Because you're starting here.
 

407
00:26:45 --> 00:26:50
You're taking its ninth power,
so one, two, three, four,

408
00:26:50 --> 00:26:53
you're back to one. i^4 is one.
 

409
00:26:53 --> 00:26:57
One, two, three, four
more, i^8 is one. i^9.

410
00:26:57 --> 00:27:02
That will actually
be just i. i^9.

411
00:27:02 --> 00:27:04
So they're all powers of i.
 

412
00:27:04 --> 00:27:06
But this is the good
way to see it.

413
00:27:06 --> 00:27:09
It's three times three,
because it's in.

414
00:27:09 --> 00:27:12
And this, of course, is three
times two, and two times

415
00:27:12 --> 00:27:14
two, and three times one.
 

416
00:27:14 --> 00:27:15
You see the entries of F?
 

417
00:27:15 --> 00:27:16
Yeah.
 

418
00:27:16 --> 00:27:19
So now I'd better write down,
I promised to write this

419
00:27:19 --> 00:27:22
formula in a better form.
 

420
00:27:22 --> 00:27:27
All those formulas in a better
- nobody is going to write this

421
00:27:27 --> 00:27:31
out for matrices of
order 1000, right?

422
00:27:31 --> 00:27:36
So we've got to write
the decent formula.

423
00:27:36 --> 00:27:42
So the decent formula is that
the jth y, because it's going

424
00:27:42 --> 00:27:48
to come in equation number j,
will be the sum on k=0 to

425
00:27:48 --> 00:27:55
N-1, of c_k times w^jk.
 

426
00:27:55 --> 00:27:58
 
 

427
00:27:58 --> 00:28:01
Those are the entries,
those are the c's that it

428
00:28:01 --> 00:28:03
multiplies and the y's.
 

429
00:28:03 --> 00:28:06
This is just vector
y's, this is y=Fc.

430
00:28:06 --> 00:28:09
 
 

431
00:28:09 --> 00:28:12
Now we've finally got
a good notation.

432
00:28:12 --> 00:28:16
So this was horrible notation.
 

433
00:28:16 --> 00:28:19
I mean, that's exhausting.
 

434
00:28:19 --> 00:28:22
This shows you the algebra, but
you have to be sort of prepared

435
00:28:22 --> 00:28:25
- well, the information is here
because this is telling you

436
00:28:25 --> 00:28:30
what the entries of F, of the
matrix, are But this is with

437
00:28:30 --> 00:28:35
indices, and this is
with whole vectors.

438
00:28:35 --> 00:28:40
So that would be y, in MATLAB
that would be the inverse

439
00:28:40 --> 00:28:49
fast Fourier transform of c.
 

440
00:28:49 --> 00:28:52
So you see that
that's our matrix.

441
00:28:52 --> 00:28:58
Now, I was going to say before
I show why, I was going to

442
00:28:58 --> 00:29:01
say what about the inverse?
 

443
00:29:01 --> 00:29:08
Because if the FFT gave us a
fast way to multiply by F, we

444
00:29:08 --> 00:29:11
want it also to give us a
fast way to do the inverse.

445
00:29:11 --> 00:29:14
We have to go both ways here.
 

446
00:29:14 --> 00:29:19
We get our data, we put it into
frequency space, we look at

447
00:29:19 --> 00:29:22
it in frequency space, we
understand what's going on.

448
00:29:22 --> 00:29:24
By separating out
the frequencies.

449
00:29:24 --> 00:29:28
We maybe smooth it, we maybe
convolve it, whatever we do.

450
00:29:28 --> 00:29:29
Compress it.
 

451
00:29:29 --> 00:29:34
And then we've got to go
back to physical space.

452
00:29:34 --> 00:29:36
So we have to go both ways.
 

453
00:29:36 --> 00:29:41
And the question is
what about F inverse?

454
00:29:41 --> 00:29:52
And let me just say, what
goes into F inverse?

455
00:29:52 --> 00:29:54
The point is that the
number that goes into F

456
00:29:54 --> 00:29:59
inverse is just like w.
 

457
00:29:59 --> 00:30:04
But it's the complex conjugate,
and I'll call it w bar.

458
00:30:04 --> 00:30:05
And that's e^(-2pi*i/N).
 

459
00:30:05 --> 00:30:10
 
 

460
00:30:10 --> 00:30:13
So the thing that's going to
go into the inverse matrix

461
00:30:13 --> 00:30:16
is the powers of w bar.
 

462
00:30:16 --> 00:30:21
The complex conjugate. jk.
 

463
00:30:21 --> 00:30:28
Let me just say right away,
that just as there were 2pi's

464
00:30:28 --> 00:30:34
in the Fourier series world,
that got in our way,

465
00:30:34 --> 00:30:38
and physicists can't
stand those 2pi's.

466
00:30:39 --> 00:30:41
They try to get rid of
them but course they

467
00:30:41 --> 00:30:43
can't, they're there.
 

468
00:30:43 --> 00:30:47
So they put it up
into the exponent.

469
00:30:47 --> 00:30:50
Well, I guess, I'm blaming
physicists, I've done it too.

470
00:30:50 --> 00:30:52
I've put the 2pi up there.
 

471
00:30:52 --> 00:30:53
For the Fourier series.
 

472
00:30:53 --> 00:30:57
Anyway whatever, 2pi's
are all in your hair

473
00:30:57 --> 00:30:59
in the Fourier series.
 

474
00:30:59 --> 00:31:03
And in this, for the discrete
one, the corresponding

475
00:31:03 --> 00:31:05
thing is an N.
 

476
00:31:05 --> 00:31:08
There's a factor N that
I have to deal with.

477
00:31:08 --> 00:31:13
Because look, here's
what I'm saying.

478
00:31:13 --> 00:31:16
Let me multiply F by F inverse.
 

479
00:31:16 --> 00:31:17
OK.
 

480
00:31:17 --> 00:31:23
Say, 1, 1, 1, 1, 1 i, i^2,
i^3, this is my F, right?

481
00:31:23 --> 00:31:24
Whoops.
 

482
00:31:24 --> 00:31:31
Don't ever let me do that. i^2,
fourth, sixth. i^3, i^6, i^9.

483
00:31:33 --> 00:31:39
Now, my claim is that F inverse
is going to involve the complex

484
00:31:39 --> 00:31:41
conjugate, w bar. i bar.
 

485
00:31:41 --> 00:31:44
What is the complex
conjugate of i?

486
00:31:44 --> 00:31:48
You see that we really have to
use complex numbers here, but

487
00:31:48 --> 00:31:50
they're on the unit circle.
 

488
00:31:50 --> 00:31:51
You couldn't ask
for a better line.

489
00:31:51 --> 00:31:54
What's the complex conjugate
that's going to go

490
00:31:54 --> 00:31:56
into F inverse?
 

491
00:31:56 --> 00:32:03
If I take i and I take its
conjugate, what does that mean?

492
00:32:03 --> 00:32:07
So the conjugate of that is
just, flip it across the

493
00:32:07 --> 00:32:10
real axis to the other one.
 

494
00:32:10 --> 00:32:12
You see, it's great?
 

495
00:32:12 --> 00:32:16
The complex conjugate
of i is minus i.

496
00:32:16 --> 00:32:19
Here's w^8, here's w bar^8.
 

497
00:32:19 --> 00:32:22
 
 

498
00:32:22 --> 00:32:24
It's just across the axis.
 

499
00:32:24 --> 00:32:29
So it still has size one.
 

500
00:32:29 --> 00:32:31
And, of course, it's e to the?
 

501
00:32:31 --> 00:32:34
This angle, which is
just minus this angle.

502
00:32:34 --> 00:32:37
So that's why we
get this minus.

503
00:32:37 --> 00:32:42
Because when I flip it across,
i changes to minus i, angles

504
00:32:42 --> 00:32:44
change it to minus angle.
 

505
00:32:44 --> 00:32:46
And finally I was going to
do this multiplication

506
00:32:46 --> 00:32:47
just to see.
 

507
00:32:47 --> 00:32:50
So now I'm claiming that
this is 1, 1, 1 1, 1

508
00:32:50 --> 00:32:55
1, 1, -i, -i^2, -i^3.
 

509
00:32:55 --> 00:33:08
So on; -i^2, -i^3, fourth,
sixth, sixth, and ninth.

510
00:33:08 --> 00:33:12
And now, if you do that
multiplication, you will

511
00:33:12 --> 00:33:13
get the identity matrix.
 

512
00:33:13 --> 00:33:15
That's the great thing.
 

513
00:33:15 --> 00:33:16
You get the identity.
 

514
00:33:16 --> 00:33:20
1, 1, 1, 1.
 

515
00:33:20 --> 00:33:23
Except for a factor N.
 

516
00:33:23 --> 00:33:27
And do you see that coming?
 

517
00:33:27 --> 00:33:31
When I multiply this by
this, what do I get?

518
00:33:31 --> 00:33:34
Four.
 

519
00:33:34 --> 00:33:39
If they were N by N, I'd have
N ones against N ones, so I'd

520
00:33:39 --> 00:33:45
have to expect an N there.
 

521
00:33:45 --> 00:33:51
So F times F inverse, so, in
other words if I want to get

522
00:33:51 --> 00:33:57
the identity, I'd better
divide this by N.

523
00:33:57 --> 00:34:01
So now I have F F
inverse equal I.

524
00:34:01 --> 00:34:07
So F inverse is just like F,
except complex conjugate

525
00:34:07 --> 00:34:09
and divide by N.
 

526
00:34:09 --> 00:34:12
So if I want this formula
to be really correct,

527
00:34:12 --> 00:34:16
I have to divide by N.
 

528
00:34:16 --> 00:34:17
OK?
 

529
00:34:17 --> 00:34:21
So and that's what
w and w bar is.

530
00:34:21 --> 00:34:25
And another letter that
often gets used in

531
00:34:25 --> 00:34:27
this topic is omega.
 

532
00:34:27 --> 00:34:34
And I figure that's a good way
to separate out the complex

533
00:34:34 --> 00:34:39
conjugates, let's call this guy
w and call this guy omega.

534
00:34:39 --> 00:34:42
So this is w and here's omega.
 

535
00:34:42 --> 00:34:44
I don't know if that helps.
 

536
00:34:44 --> 00:34:48
But you do have to keep
straight whether you're

537
00:34:48 --> 00:34:50
talking about this
number or that number.

538
00:34:50 --> 00:34:55
And this number could be called
w, because omega wouldn't

539
00:34:55 --> 00:34:56
be easy to type.
 

540
00:34:56 --> 00:35:01
So you have to pay attention,
which one you're doing

541
00:35:01 --> 00:35:05
and when, somewhere you
have to divide by N.

542
00:35:05 --> 00:35:08
But otherwise it's fantastic.
 

543
00:35:08 --> 00:35:17
It's just beautiful.
 

544
00:35:17 --> 00:35:21
Can we do that multiplication,
just to see that

545
00:35:21 --> 00:35:23
it really works?
 

546
00:35:23 --> 00:35:25
Well, the ones worked fine.
 

547
00:35:25 --> 00:35:32
What about the ones times,
or what about the ones

548
00:35:32 --> 00:35:36
times that column.
 

549
00:35:36 --> 00:35:42
So I'm looking at the (1,2)
entry in this multiplication.

550
00:35:42 --> 00:35:45
Or rather, the (0,1),
entry I should say.

551
00:35:45 --> 00:35:48
This is the zeroth row, and
this is the row number one, and

552
00:35:48 --> 00:35:51
what do I get out of that?
 

553
00:35:51 --> 00:35:54
Zero, right.
 

554
00:35:54 --> 00:35:55
That was my point.
 

555
00:35:55 --> 00:36:00
That this is adding up,
oh well, going the

556
00:36:00 --> 00:36:01
other way around.
 

557
00:36:01 --> 00:36:04
Because it's the complex
conjugate, but it's adding up

558
00:36:04 --> 00:36:12
the four, same four numbers
and giving me that zero.

559
00:36:12 --> 00:36:15
And now how about the
ones on the diagonal?

560
00:36:15 --> 00:36:20
When I do this times this,
so this is the same,

561
00:36:20 --> 00:36:23
the row times itself.
 

562
00:36:23 --> 00:36:26
What am I seeing there?
 

563
00:36:26 --> 00:36:28
What are these terms?
 

564
00:36:28 --> 00:36:33
That's certainly one, what
is i times minus i? i times

565
00:36:33 --> 00:36:35
minus i is one again.
 

566
00:36:35 --> 00:36:38
A number's getting multiplied
by its conjugate.

567
00:36:38 --> 00:36:41
That gives you something real
and positive when you multiply

568
00:36:41 --> 00:36:44
a number by its conjugate.
 

569
00:36:44 --> 00:36:47
In fact, we get one every time
so we get four ones, we divide

570
00:36:47 --> 00:36:57
by four and we get that.
 

571
00:36:57 --> 00:37:03
Maybe you actually see here,
just so you remember, so F

572
00:37:03 --> 00:37:11
inverse is, I'm concluding
that F inverse, I'm looking

573
00:37:11 --> 00:37:13
now, what is F inverse?
 

574
00:37:13 --> 00:37:20
It's F conjugate divided by
this nuisance number N.

575
00:37:20 --> 00:37:23
I could put square root of N
with F, and then square root of

576
00:37:23 --> 00:37:27
N would show up with F inverse,
just the way I could have done,

577
00:37:27 --> 00:37:30
square the 2pi with one, and
square the 2pi with the other.

578
00:37:30 --> 00:37:37
But let's leave it like so.
 

579
00:37:37 --> 00:37:40
So that's not proved in a way.
 

580
00:37:40 --> 00:37:48
Just, checked for a couple
of cases with N=4,

581
00:37:48 --> 00:37:51
but it always works.
 

582
00:37:51 --> 00:37:52
That's the beauty.
 

583
00:37:52 --> 00:37:58
In other words, the two
directions, the y to c and the

584
00:37:58 --> 00:38:04
c to y, are both going to
be speeded up by the FFT.

585
00:38:04 --> 00:38:07
The FFT, whatever it did,
whatever it could do with the

586
00:38:07 --> 00:38:12
w and powers of w, it can
do the same with w bar.

587
00:38:12 --> 00:38:14
And powers of w bar.
 

588
00:38:14 --> 00:38:18
If the FFT is a fast way to to
multiply by that, it'll also

589
00:38:18 --> 00:38:24
give me a fast way to multiply,
to do the other transform.

590
00:38:24 --> 00:38:29
So the FFT is what makes every
- this would all be important

591
00:38:29 --> 00:38:32
if the FFT didn't exist.
 

592
00:38:32 --> 00:38:34
And of course actually,
Fourier didn't notice it.

593
00:38:34 --> 00:38:42
Gauss again had the idea, but
he had so many ideas it wasn't

594
00:38:42 --> 00:38:46
very clear in his notes.
 

595
00:38:46 --> 00:38:48
The FFT idea.
 

596
00:38:48 --> 00:38:56
But then Cooley and Tukey,
guys at Bell Labs in the

597
00:38:56 --> 00:38:59
1950s discovered it.
 

598
00:38:59 --> 00:39:04
In a small paper, I don't
think they had any idea

599
00:39:04 --> 00:39:06
how important it would be.
 

600
00:39:06 --> 00:39:10
But they published their paper
and people saw what was

601
00:39:10 --> 00:39:13
happening, and realized that
they should, that that gave the

602
00:39:13 --> 00:39:16
right way to do the transforms.
 

603
00:39:16 --> 00:39:24
OK, I was going to say here, do
you notice that this one, when

604
00:39:24 --> 00:39:30
I take a vector, that vectors
and its complex conjugate,

605
00:39:30 --> 00:39:33
what am I getting?
 

606
00:39:33 --> 00:39:35
I'm getting the length squared.
 

607
00:39:35 --> 00:39:37
I'm getting the right
inner product.

608
00:39:37 --> 00:39:41
This is the right inner product
when things are complex.

609
00:39:41 --> 00:39:42
Something times its conjugate.
 

610
00:39:42 --> 00:39:47
I'm just reminding you that the
inner product of two vectors,

611
00:39:47 --> 00:39:54
uv, u dot v or u comma v,
should now be the sum

612
00:39:54 --> 00:39:57
- it used to be the
sum of u_i*v_i.

613
00:39:59 --> 00:40:02
But now we're in the complex
case, so we should take

614
00:40:02 --> 00:40:04
the conjugate one of them.
 

615
00:40:04 --> 00:40:08
And here I guess that's the
v, and this is the u, and

616
00:40:08 --> 00:40:15
so the point is that we're
taking the right, yeah.

617
00:40:15 --> 00:40:22
We have, oh yeah. let me
just say it this way.

618
00:40:22 --> 00:40:26
These columns are orthogonal.
 

619
00:40:26 --> 00:40:29
Those four columns, the columns
of the Fourier matrix,

620
00:40:29 --> 00:40:31
are orthogonal.
 

621
00:40:31 --> 00:40:36
In other words, we have
an orthogonal basis.

622
00:40:36 --> 00:40:38
And it's those four columns.
 

623
00:40:38 --> 00:40:43
I almost missed saying that.
 

624
00:40:43 --> 00:40:47
But you remember from last
week, the whole point about

625
00:40:47 --> 00:40:51
sines and cosines and
exponentials in the function

626
00:40:51 --> 00:40:53
case, was orthogonal.
 

627
00:40:53 --> 00:40:55
That's what made everything
work, and it makes

628
00:40:55 --> 00:40:58
everything work here.
 

629
00:40:58 --> 00:41:01
Here we have vectors,
not functions.

630
00:41:01 --> 00:41:04
Orthogonal just means
dot products are zero.

631
00:41:04 --> 00:41:09
But they're complex, so that in
the dot product, one factor or

632
00:41:09 --> 00:41:11
the other has to
get conjugated.

633
00:41:11 --> 00:41:15
Anyway, that's an
orthogonal matrix.

634
00:41:15 --> 00:41:20
Apart from this N,
this capital N thing.

635
00:41:20 --> 00:41:24
So if I normalize it, if I take
out one over square root of

636
00:41:24 --> 00:41:28
N, now the length of
every column is one.

637
00:41:28 --> 00:41:31
So now I have
orthonormal columns.

638
00:41:31 --> 00:41:34
One squared, one squared, one
squared, one squared is four,

639
00:41:34 --> 00:41:38
divided by four is one.
 

640
00:41:38 --> 00:41:45
And every column has length is
a vector in four-dimensional

641
00:41:45 --> 00:41:47
complex space.
 

642
00:41:47 --> 00:41:49
Of length one.
 

643
00:41:49 --> 00:41:51
And orthogonal.
 

644
00:41:51 --> 00:41:52
So that means orthonormal.
 

645
00:41:52 --> 00:41:55
It's a fantastic matrix.
 

646
00:41:55 --> 00:41:59
That's the best basis we'll
ever have for complex

647
00:41:59 --> 00:42:00
four-dimensional space.
 

648
00:42:00 --> 00:42:06
That basis.
 

649
00:42:06 --> 00:42:09
Well, actually when I say
the best basis we'll ever

650
00:42:09 --> 00:42:13
have, I realize I wrote
a book about wavelets.

651
00:42:13 --> 00:42:16
And the idea of wavelets
is, there's another basis.

652
00:42:16 --> 00:42:18
A wavelet basis.
 

653
00:42:18 --> 00:42:21
And I hope to tell you
about that at the very

654
00:42:21 --> 00:42:24
end of the semester.
 

655
00:42:24 --> 00:42:26
So wavelets would be another
basis, would you like to

656
00:42:26 --> 00:42:29
know the wavelet basis?
 

657
00:42:29 --> 00:42:32
Have we got?
 

658
00:42:32 --> 00:42:37
Just in case the end of the
semester doesn't come,

659
00:42:37 --> 00:42:40
can I write down here
the wavelet basis?

660
00:42:40 --> 00:42:41
OK.
 

661
00:42:41 --> 00:42:45
And you can decide you
maybe like it better.

662
00:42:45 --> 00:42:49
So the four by four
wavelet basis, alright.

663
00:42:49 --> 00:42:53
So this will be the
wavelet matrix.

664
00:42:53 --> 00:42:56
Well, this was too
good to lose, right?

665
00:42:56 --> 00:42:59
That's a terrific vector.
 

666
00:42:59 --> 00:43:02
All ones, constant vector.
 

667
00:43:02 --> 00:43:06
It's the thing that gives us
the average, this c_0 is

668
00:43:06 --> 00:43:13
going to again be the
average of the y's.

669
00:43:13 --> 00:43:17
How do I get that average?
 

670
00:43:17 --> 00:43:21
Somehow I'm doing, the c_0, the
constant term is again I'm

671
00:43:21 --> 00:43:24
going to take one of everything
and I'll divide by four and

672
00:43:24 --> 00:43:25
I'll have the average.
 

673
00:43:25 --> 00:43:27
So I like that vector.
 

674
00:43:27 --> 00:43:30
That's the DC vector,
if I'm talking about

675
00:43:30 --> 00:43:31
direct current vector.
 

676
00:43:31 --> 00:43:36
If I'm talking about networks.
 

677
00:43:36 --> 00:43:39
In an image, it would be the
whole, you could call it

678
00:43:39 --> 00:43:41
the blackboard vector.
 

679
00:43:41 --> 00:43:46
The whole image of a empty
blackboard would be constant.

680
00:43:46 --> 00:43:49
OK, so that's too good to lose.
 

681
00:43:49 --> 00:43:52
Now, the obvious thing
orthogonal to that would

682
00:43:52 --> 00:43:56
be .
 

683
00:43:56 --> 00:43:57
Right?
 

684
00:43:57 --> 00:44:00
Now I've got two
orthogonal columns.

685
00:44:00 --> 00:44:03
And probably that's pretty
close to something

686
00:44:03 --> 00:44:05
I have over here.
 

687
00:44:05 --> 00:44:08
Now, here's the point
about wavelets.

688
00:44:08 --> 00:44:10
You'll see it with
the next guy.

689
00:44:10 --> 00:44:14
The next one, you saw that
those two are orthogonal.

690
00:44:14 --> 00:44:17
And real, and simple,
nice numbers.

691
00:44:17 --> 00:44:24
Now, the next one will
be .

692
00:44:24 --> 00:44:28
Do you see that
that's orthogonal?

693
00:44:28 --> 00:44:30
Obviously, because its
dot product with that is

694
00:44:30 --> 00:44:33
zero and that is zero.
 

695
00:44:33 --> 00:44:35
And you want to tell me the
last column of the wavelet, of

696
00:44:35 --> 00:44:37
the simple wavelet matrix?
 

697
00:44:37 --> 00:44:47
This would be the Haar, named
after Haar, the Haar wavelets.

698
00:44:47 --> 00:44:50
The whole point about
wavelets was a new basis.

699
00:44:50 --> 00:44:53
Somebody realized, well
actually, Haar lived a long

700
00:44:53 --> 00:44:57
time ago, but they got improved
and improved, but here's

701
00:44:57 --> 00:44:57
the simplest one.
 

702
00:44:57 --> 00:45:00
What's the good fourth column?
 

703
00:45:00 --> 00:45:05
Fourth orthogonal vector here?
 

704
00:45:05 --> 00:45:06
What shall I start with?
 

705
00:45:06 --> 00:45:17
Zeroes, and then I'll
put one minus one.

706
00:45:17 --> 00:45:23
So the beauty of wavelets is,
I've got this averaging vector.

707
00:45:23 --> 00:45:28
And then I've got this
difference, sort of on a big

708
00:45:28 --> 00:45:31
scale, and then I've got
differences on a little scale

709
00:45:31 --> 00:45:36
with zero, differences here
with zeroes so as I keep going

710
00:45:36 --> 00:45:42
I could do eight by eight, 16
by 16, it would - I get

711
00:45:42 --> 00:45:44
the averaging guy.
 

712
00:45:44 --> 00:45:47
And then I get differencing, I
get large scale differences and

713
00:45:47 --> 00:45:49
then I get finer differences.
 

714
00:45:49 --> 00:45:53
So coarse differences,
finer differences, finer

715
00:45:53 --> 00:45:54
finer differences.
 

716
00:45:54 --> 00:46:03
And that's a really good way
to - if your function, if

717
00:46:03 --> 00:46:06
your signal has jumps.
 

718
00:46:06 --> 00:46:11
We saw that Fourier
wasn't perfect, right?

719
00:46:11 --> 00:46:13
And a lot of signals
have jumps.

720
00:46:13 --> 00:46:13
Right?
 

721
00:46:13 --> 00:46:18
Your shocks, whatever you're
trying to represent.

722
00:46:18 --> 00:46:22
This is a better basis
to represent a jump, a

723
00:46:22 --> 00:46:25
discontinuity, than Fourier.
 

724
00:46:25 --> 00:46:29
Because Fourier, at a
discontinuity, does its best

725
00:46:29 --> 00:46:32
but it's not a local basis.
 

726
00:46:32 --> 00:46:37
It can't, like, especially
focus on that jump point.

727
00:46:37 --> 00:46:42
And and the result is the
coefficients decay very slowly

728
00:46:42 --> 00:46:44
and it's hard to compute with.
 

729
00:46:44 --> 00:46:49
Whereas the wavelet is great
for signals that have jumps.

730
00:46:49 --> 00:46:52
If something happens
suddenly at a certain time.

731
00:46:52 --> 00:46:56
Anyway, that would be another,
and of course I haven't made

732
00:46:56 --> 00:47:00
these unit vectors, I'd have to
divide that by the square root

733
00:47:00 --> 00:47:03
of four and divide these guys
by the square root of two

734
00:47:03 --> 00:47:05
just to normalize them.
 

735
00:47:05 --> 00:47:08
But normalizing
isn't important.

736
00:47:08 --> 00:47:11
Orthogonal is what
is important.

737
00:47:11 --> 00:47:20
OK, so that's competition,
you could say.

738
00:47:20 --> 00:47:22
That's the competition.
 

739
00:47:22 --> 00:47:29
OK, back to main guy, so this
is the famous formula that

740
00:47:29 --> 00:47:34
gives the y's from the c's, and
what's the other formula that

741
00:47:34 --> 00:47:40
gives the c's from the y's?
 

742
00:47:40 --> 00:47:44
So now this is the transform
in the other direction.

743
00:47:44 --> 00:47:49
So it's going to be a
sum from j=0 to N-1.

744
00:47:49 --> 00:47:53
 
 

745
00:47:53 --> 00:47:57
And what's going
to go in there?

746
00:47:57 --> 00:48:01
Have you got the idea
of the two transforms?

747
00:48:01 --> 00:48:04
In one direction this is e^i.
 

748
00:48:05 --> 00:48:07
You see why we don't
want to write it.

749
00:48:07 --> 00:48:10
That is, that number
is e^(2pi*i*j*k/N).

750
00:48:10 --> 00:48:17
 
 

751
00:48:17 --> 00:48:23
That's what that number is,
in the Fourier matrix.

752
00:48:23 --> 00:48:28
So you see using w
makes it much cleaner.

753
00:48:28 --> 00:48:34
OK, what goes there? w bar.
 

754
00:48:34 --> 00:48:37
The complex conjugate, e to
the minus all that stuff. w

755
00:48:37 --> 00:48:41
bar to the same power, jk.
 

756
00:48:42 --> 00:48:45
And then I have to remember
to divide by N somewhere.

757
00:48:45 --> 00:48:48
I can put it here, or
I can put it there.

758
00:48:48 --> 00:48:55
But the all ones vector has
length squared equal to N

759
00:48:55 --> 00:48:59
and I've got to divide
by an N somewhere.

760
00:48:59 --> 00:49:01
OK, those are the two
formulas that many people

761
00:49:01 --> 00:49:02
would just remember.
 

762
00:49:02 --> 00:49:06
For them that's the discrete
Fourier transform and the

763
00:49:06 --> 00:49:08
inverse, and maybe they
would put the N over there.

764
00:49:08 --> 00:49:11
Probably that would
be more common.

765
00:49:11 --> 00:49:16
I got started with the Fourier
matrix with this side, and

766
00:49:16 --> 00:49:24
then the DFT matrix comes
here, and I needed the N.

767
00:49:24 --> 00:49:26
So those are the formulas.
 

768
00:49:26 --> 00:49:29
I don't know how you'd
prefer to remember these.

769
00:49:29 --> 00:49:31
Somehow you have to commit
something to memory here,

770
00:49:31 --> 00:49:33
certainly what w is.
 

771
00:49:33 --> 00:49:37
And the fact that
it's w and w bar?

772
00:49:37 --> 00:49:40
And the fact that the
sums start at zero.

773
00:49:40 --> 00:49:43
And the fact that you have just
these simple expressions for

774
00:49:43 --> 00:49:46
the entries in the matrix.
 

775
00:49:46 --> 00:49:48
And this, then would
be c=F inverse*y.

776
00:49:48 --> 00:49:53
 
 

777
00:49:53 --> 00:49:59
That's what I've written
down there in detail.

778
00:49:59 --> 00:50:05
OK, so that's the discrete
Fourier transform, and I

779
00:50:05 --> 00:50:06
think we're out of time.
 

780
00:50:06 --> 00:50:09
I've got more to say in
a little bit about the

781
00:50:09 --> 00:50:12
FFT, how that works.