1
00:00:00 --> 00:00:01
 
 

2
00:00:01 --> 00:00:03
The following content is
provided under a Creative

3
00:00:03 --> 00:00:03
Commons license.
 

4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:09
offer high-quality educational
resources for free.

6
00:00:09 --> 00:00:12
To make a donation, or to view
additional materials from

7
00:00:12 --> 00:00:16
hundreds of MIT courses, visit
MIT OpenCourseWare

8
00:00:16 --> 00:00:21
at ocw.mit.edu.
 

9
00:00:21 --> 00:00:23
OK, so.
 

10
00:00:23 --> 00:00:25
Pretty full day again.
 

11
00:00:25 --> 00:00:30
I had last time introduced
the Fourier matrix, the

12
00:00:30 --> 00:00:33
discrete Fourier transform.
 

13
00:00:33 --> 00:00:37
Well, more strictly, the
discrete Fourier transform

14
00:00:37 --> 00:00:38
is usually this one.
 

15
00:00:38 --> 00:00:43
It takes the function values
and produces the coefficients.

16
00:00:43 --> 00:00:47
And then I started with the
coefficients added back,

17
00:00:47 --> 00:00:51
added up the series to
get the function values.

18
00:00:51 --> 00:00:53
So F or F inverse.
 

19
00:00:53 --> 00:00:56
So we didn't do examples yet.
 

20
00:00:56 --> 00:01:03
And one natural example is the
discrete delta function that

21
00:01:03 --> 00:01:06
has a one in the zero position.
 

22
00:01:06 --> 00:01:07
That's easy to do.
 

23
00:01:07 --> 00:01:12
And then we should do also a
shift, to see what's the

24
00:01:12 --> 00:01:15
effect, if you shift the
function, what happens

25
00:01:15 --> 00:01:17
to the transform.
 

26
00:01:17 --> 00:01:19
That's an important rule,
important for Fourier series

27
00:01:19 --> 00:01:21
and Fourier integrals, too.
 

28
00:01:21 --> 00:01:26
Because often you do that, and
it's going to be a simple rule.

29
00:01:26 --> 00:01:31
If you shift the function, the
transform does something nice.

30
00:01:31 --> 00:01:35
OK, and then I want to describe
a little about the FFT and

31
00:01:35 --> 00:01:38
then start on the next
section, convolutions.

32
00:01:38 --> 00:01:42
So that's fun and
that's a big deal.

33
00:01:42 --> 00:01:47
OK, about reviews, I'll
be here as usual today.

34
00:01:47 --> 00:01:53
I think maybe the 26th, just
hours before Thanksgiving we

35
00:01:53 --> 00:01:56
can give ourselves a holiday.
 

36
00:01:56 --> 00:02:01
So not next Wednesday but then
certainly the Wednesday of the

37
00:02:01 --> 00:02:08
following week would be a sort
of major quiz review on

38
00:02:08 --> 00:02:10
in the review session.
 

39
00:02:10 --> 00:02:11
And in class.
 

40
00:02:11 --> 00:02:13
OK, ready to go?
 

41
00:02:13 --> 00:02:19
On this example gives us a
chance just to remember what

42
00:02:19 --> 00:02:22
the matrix looks like, because
we're just going to, if I

43
00:02:22 --> 00:02:26
multiply this inverse matrix by
that vector it's just going to

44
00:02:26 --> 00:02:30
pick off the first column and
it'll be totally easy

45
00:02:30 --> 00:02:31
so let's just do it.
 

46
00:02:31 --> 00:02:36
So if y is this one, I
want to know about f.

47
00:02:36 --> 00:02:42
What are the Fourier
coefficients of the

48
00:02:42 --> 00:02:44
delta function?
 

49
00:02:44 --> 00:02:45
Discrete delta function?
 

50
00:02:45 --> 00:02:49
OK, before I even do it,
we got a pretty good

51
00:02:49 --> 00:02:51
idea what to expect.
 

52
00:02:51 --> 00:02:54
Because we remember what
happened to the ordinary delta

53
00:02:54 --> 00:02:57
function, in continuous time.
 

54
00:02:57 --> 00:03:00
Or rather, I guess it was the
periodic delta function.

55
00:03:00 --> 00:03:02
Do you remember the
coefficients, what were

56
00:03:02 --> 00:03:05
the coefficients for the
periodic delta function?

57
00:03:05 --> 00:03:07
You remember those?
 

58
00:03:07 --> 00:03:11
We took the integral from minus
pi to pi of our function,

59
00:03:11 --> 00:03:12
which was delta(x).
 

60
00:03:13 --> 00:03:18
And then we had to remember to
divide by 2pi, and you remember

61
00:03:18 --> 00:03:19
the coefficients are e^(-ikx).
 

62
00:03:19 --> 00:03:23
 
 

63
00:03:23 --> 00:03:29
This is c_k in the periodic
case, the 2pi periodic case the

64
00:03:29 --> 00:03:32
function is the delta function,
and you remember that if we

65
00:03:32 --> 00:03:35
want coefficient k we
multiply by e^(-ikx).

66
00:03:37 --> 00:03:42
That's the thing that will pick
out the e^(+ikx) term and of

67
00:03:42 --> 00:03:47
course everybody knows what we
get here, this delta, this

68
00:03:47 --> 00:03:53
spike at x=0, means we take the
value of that function at zero,

69
00:03:53 --> 00:03:55
which is one, so we
just get 1/2pi.

70
00:03:55 --> 00:03:57
 
 

71
00:03:57 --> 00:04:00
For all the Fourier
coefficients of the

72
00:04:00 --> 00:04:02
delta function.
 

73
00:04:02 --> 00:04:05
The point being that
they're all the same.

74
00:04:05 --> 00:04:08
That all frequencies are
in the delta function

75
00:04:08 --> 00:04:09
to the same amount.
 

76
00:04:09 --> 00:04:11
I mean that's kind of nice.
 

77
00:04:11 --> 00:04:15
That we created the delta
function for other reasons, but

78
00:04:15 --> 00:04:21
then here in Fourier space
it's just clean as could be.

79
00:04:21 --> 00:04:23
And we'll expect
something here, too.

80
00:04:23 --> 00:04:25
You remember what F inverse is?
 

81
00:04:25 --> 00:04:27
F inverse 1/N.
 

82
00:04:28 --> 00:04:34
Instead of 1/2pi, and then the
entries of F inverse come

83
00:04:34 --> 00:04:38
from F bar, the conjugate.
 

84
00:04:38 --> 00:04:42
So it's just one, one, one,
minus - well, I've made

85
00:04:42 --> 00:04:44
it four by four here.
 

86
00:04:44 --> 00:04:50
This is minus omega - no,
it isn't minus omega.

87
00:04:50 --> 00:04:58
It's it's omega bar, which
is minus i, in this case.

88
00:04:58 --> 00:05:02
Omega bar, so it's minus
i, that's omega bar.

89
00:05:02 --> 00:05:04
And the next one would
be omega bar squared,

90
00:05:04 --> 00:05:09
and cubed, and so on.
 

91
00:05:09 --> 00:05:12
All the way up to
the ninth power.

92
00:05:12 --> 00:05:14
But we're multiplying
by one, zero, zero, so

93
00:05:14 --> 00:05:19
none of that matters.
 

94
00:05:19 --> 00:05:24
What's the answer?
 

95
00:05:24 --> 00:05:28
I'm doing this discrete Fourier
transform, so I'm multiplying

96
00:05:28 --> 00:05:32
by the matrix with the
complex conjugate guys.

97
00:05:32 --> 00:05:35
But I'm multiplying by that
simple thing so it's just going

98
00:05:35 --> 00:05:37
to pick out the 0th column.
 

99
00:05:37 --> 00:05:40
In other words, constant.
 

100
00:05:40 --> 00:05:44
All the Fourier discrete
Fourier coefficients of the

101
00:05:44 --> 00:05:46
discrete delta are the same.
 

102
00:05:46 --> 00:05:47
Just again.
 

103
00:05:47 --> 00:05:49
And what are they?
 

104
00:05:49 --> 00:05:52
So it picks out this column,
but of course it divides by

105
00:05:52 --> 00:05:54
N, so the answer was 1/N.
 

106
00:05:56 --> 00:05:58
.
 

107
00:05:58 --> 00:06:03
It's just constant with the
1/N, where in the continuous

108
00:06:03 --> 00:06:04
case we had 1/2pi.
 

109
00:06:05 --> 00:06:06
No problem.
 

110
00:06:06 --> 00:06:08
OK.
 

111
00:06:08 --> 00:06:12
And, of course, everybody
knows, suppose that I now start

112
00:06:12 --> 00:06:16
with these coefficients and add
back to get the function.

113
00:06:16 --> 00:06:20
What would I get?
 

114
00:06:20 --> 00:06:24
Because just to be sure that we
believe that F and F inverse

115
00:06:24 --> 00:06:26
really are what they
are supposed to be.

116
00:06:26 --> 00:06:33
If I start with these
coefficients, add back to put

117
00:06:33 --> 00:06:39
those in here and reconstruct,
so 1/N, <1, 1, 1,

118
00:06:39 --> 00:06:43
1>, what will I get?
 

119
00:06:43 --> 00:06:46
Well, what am I
supposed to get?

120
00:06:46 --> 00:06:47
The delta, right?
 

121
00:06:47 --> 00:06:50
I'm supposed to get back to y.
 

122
00:06:50 --> 00:06:55
If I started with that, did F
inverse to get the

123
00:06:55 --> 00:06:58
coefficients, that was the
discrete Fourier transform, now

124
00:06:58 --> 00:07:03
I add back to get, add the
Fourier series up again to come

125
00:07:03 --> 00:07:07
back here, well I'll certainly
get ,

126
00:07:07 --> 00:07:09
and you see why?
 

127
00:07:09 --> 00:07:15
If I multiply F, that zeroth
row of F is , times

128
00:07:15 --> 00:07:18
 will give me N.
 

129
00:07:18 --> 00:07:19
The N will cancel.
 

130
00:07:19 --> 00:07:21
I get the one.
 

131
00:07:21 --> 00:07:25
And all the other
guys add to zeroes.

132
00:07:25 --> 00:07:31
So, sure enough, it works.
 

133
00:07:31 --> 00:07:34
We're really just seeing
an example, an important

134
00:07:34 --> 00:07:37
example of the DFT.
 

135
00:07:37 --> 00:07:40
And the homework, then, would
have some other examples.

136
00:07:40 --> 00:07:44
But I've forgotten whether the
homework has this example.

137
00:07:44 --> 00:07:47
But let's think about it now.
 

138
00:07:47 --> 00:07:54
Suppose that's my
function value instead.

139
00:07:54 --> 00:07:59
OK, so now I'm starting
with the delta.

140
00:07:59 --> 00:08:02
Again it's a delta,
but it's moved over.

141
00:08:02 --> 00:08:06
And I could ask, I really
should ask first, in the

142
00:08:06 --> 00:08:11
continuous case, suppose
I, I can do it with just

143
00:08:11 --> 00:08:13
a little erasing here.
 

144
00:08:13 --> 00:08:16
Let me do the continuous case
for the delta that we met

145
00:08:16 --> 00:08:18
first in this course.
 

146
00:08:18 --> 00:08:20
I'll shift it to a.
 

147
00:08:20 --> 00:08:26
If I shift the delta function
to a point a, well, I said

148
00:08:26 --> 00:08:29
we'd met the delta function
first in this course.

149
00:08:29 --> 00:08:31
At a, good.
 

150
00:08:31 --> 00:08:35
But when we did it
wasn't 2pi periodic.

151
00:08:35 --> 00:08:41
So we still have, in fact, the
Fourier integrals next week.

152
00:08:41 --> 00:08:42
Will have a similar formula.
 

153
00:08:42 --> 00:08:46
The integral will go from minus
infinity to infinity and

154
00:08:46 --> 00:08:49
then we'll have the real
delta, not periodic.

155
00:08:49 --> 00:08:53
Here, we have, and people
call it a train of deltas.

156
00:08:53 --> 00:08:55
A train of spikes.
 

157
00:08:55 --> 00:08:57
Sort of you have one every 2pi.
 

158
00:08:58 --> 00:09:00
Anyway, that's what we've got.
 

159
00:09:00 --> 00:09:03
Now, you can see the answer.
 

160
00:09:03 --> 00:09:07
This is like in perfect
practice in doing an

161
00:09:07 --> 00:09:09
integral with a delta.
 

162
00:09:09 --> 00:09:15
What's the integral equal?
 

163
00:09:15 --> 00:09:16
Well, the spike is at x=a.
 

164
00:09:18 --> 00:09:24
So it picks this function
at x=a, which is e^(-ika).

165
00:09:24 --> 00:09:30
 
 

166
00:09:30 --> 00:09:32
So not constant any more.
 

167
00:09:32 --> 00:09:35
They depend on k.
 

168
00:09:35 --> 00:09:38
The 1/2pi's still there.
 

169
00:09:38 --> 00:09:43
So it's, but still, the delta
function shifted over.

170
00:09:43 --> 00:09:45
I mean, it didn't
change energy.

171
00:09:45 --> 00:09:49
It didn't change, it just
changed phase, so to speak.

172
00:09:49 --> 00:10:00
And we see that, I would call
this like a modulation.

173
00:10:00 --> 00:10:04
So it's staying of absolute
value one, still.

174
00:10:04 --> 00:10:07
But it's not the number one,
it's going around the circle.

175
00:10:07 --> 00:10:10
Going around the unit circle.
 

176
00:10:10 --> 00:10:13
So it's a phase factor, right.
 

177
00:10:13 --> 00:10:16
And that's what I'm going
to expect to see here in

178
00:10:16 --> 00:10:17
the discrete case too.
 

179
00:10:17 --> 00:10:22
If I do this multiplication
by one there, it picks

180
00:10:22 --> 00:10:24
out this column, right?
 

181
00:10:24 --> 00:10:28
That one will pick out this
column, so you see it's

182
00:10:28 --> 00:10:31
maybe I come up here now.
 

183
00:10:31 --> 00:10:41
Shall I just, when I pick out
that column, the answer then,

184
00:10:41 --> 00:10:46
I guess I've got the column
circle, there it is minus i.

185
00:10:46 --> 00:10:49
Minus i squared, minus i cubed.
 

186
00:10:49 --> 00:10:53
You see it's like k equals
zero, one, two, three.

187
00:10:53 --> 00:10:58
Just the way here, we had k,
well we had all integers.

188
00:10:58 --> 00:11:01
k in that function case.
 

189
00:11:01 --> 00:11:04
Here we've got four integers, k
equals zero, one, two, and

190
00:11:04 --> 00:11:10
three, but again it's the minus
i, it's the e to the,

191
00:11:10 --> 00:11:13
it's the w bar.
 

192
00:11:13 --> 00:11:18
In other words, the answer
was one w bar w bar

193
00:11:18 --> 00:11:19
squared w bar cubed.
 

194
00:11:19 --> 00:11:24
Just the powers of w
with this factor 1/N.

195
00:11:25 --> 00:11:28
Here we had a modulation.
 

196
00:11:28 --> 00:11:31
It's the same picture.
 

197
00:11:31 --> 00:11:33
Absolute value's one.
 

198
00:11:33 --> 00:11:34
And and what about energy?
 

199
00:11:34 --> 00:11:42
Having mentioned energy, so
that's another key rule.

200
00:11:42 --> 00:11:47
The key rules for the Fourier
series, just let's think back.

201
00:11:47 --> 00:11:49
What were the key rules?
 

202
00:11:49 --> 00:11:52
First, the rule to find
the coefficients.

203
00:11:52 --> 00:11:53
Good.
 

204
00:11:53 --> 00:11:56
Then the rule for
the derivatives.

205
00:11:56 --> 00:11:59
This is so important.
 

206
00:11:59 --> 00:12:03
These are rules.
 

207
00:12:03 --> 00:12:05
Let's say, for Fourier series.
 

208
00:12:05 --> 00:12:08
For Fourier series.
 

209
00:12:08 --> 00:12:11
Let's just make this
a quick review.

210
00:12:11 --> 00:12:14
What were the important rules?
 

211
00:12:14 --> 00:12:18
The important rules were, if I
had the Fourier series of f.

212
00:12:18 --> 00:12:20
Start with the
Fourier series of f.

213
00:12:20 --> 00:12:24
Then the question was, what's
the Fourier series of df/dx.

214
00:12:24 --> 00:12:28
 
 

215
00:12:28 --> 00:12:32
And now I'm saying the next
important rule is the Fourier

216
00:12:32 --> 00:12:38
series of f, shifted.
 

217
00:12:38 --> 00:12:45
And then the last important
rule is the energy.

218
00:12:45 --> 00:12:50
OK, and let's just, maybe this
is a bad idea to, since we're

219
00:12:50 --> 00:12:57
kind of doing all of Fourier
in, it's coming in three parts.

220
00:12:57 --> 00:13:05
Functions, discrete, integrals,
but they all match.

221
00:13:05 --> 00:13:10
So this is what happens
to the function.

222
00:13:10 --> 00:13:12
What happens to
the coefficient?

223
00:13:12 --> 00:13:19
So this starts with coefficient
c_k, for f, what are the

224
00:13:19 --> 00:13:22
coefficients for the
derivative, just remind me?

225
00:13:22 --> 00:13:28
If f(x), so I'm starting with
f(x) equals sum of c_k*e^(ikx).

226
00:13:28 --> 00:13:32
 
 

227
00:13:32 --> 00:13:33
Start with that.
 

228
00:13:33 --> 00:13:35
And now take the derivative.
 

229
00:13:35 --> 00:13:37
When I take the derivative,
down comes ik.

230
00:13:39 --> 00:13:41
So you remember that rule.
 

231
00:13:41 --> 00:13:44
Those are the Fourier
coefficients of the derivative.

232
00:13:44 --> 00:13:47
Now what's the Fourier
coefficients of the shift?

233
00:13:47 --> 00:13:51
If I've just shifted,
translated the function, if

234
00:13:51 --> 00:13:56
my original x was this,
now let me look at f(x-a).

235
00:13:56 --> 00:13:59
You'll see it.
 

236
00:13:59 --> 00:14:02
It'll jump out at us, it'll
be a sum of the same

237
00:14:02 --> 00:14:04
c_k's e^ik(x-a).
 

238
00:14:04 --> 00:14:09
 
 

239
00:14:09 --> 00:14:12
So what are the Fourier
coefficients of that?

240
00:14:12 --> 00:14:13
Well there is the e^(ikx).
 

241
00:14:15 --> 00:14:17
Whatever's multiplying it
has got to be the Fourier

242
00:14:17 --> 00:14:21
coefficient, and we see it
as c_k times e^ik(-a).

243
00:14:23 --> 00:14:28
e^(-ika), times c_k.
 

244
00:14:30 --> 00:14:32
Right?
 

245
00:14:32 --> 00:14:35
And, of course, that's just
what we discovered here.

246
00:14:35 --> 00:14:38
That's just what we found
there, that when we shifted the

247
00:14:38 --> 00:14:43
delta, we've multiplied by this
modulation, this phase factor

248
00:14:43 --> 00:14:46
came into the Fourier
coefficients.

249
00:14:46 --> 00:14:49
And now finally,
the energy stuff.

250
00:14:49 --> 00:14:52
You remember the energy
was, what's the energy?

251
00:14:52 --> 00:14:59
The integral from minus pi to
pi, of f(x) squared. dx is the

252
00:14:59 --> 00:15:04
same as the sum from minus
infinity to infinity of

253
00:15:04 --> 00:15:06
the coefficient squared.
 

254
00:15:06 --> 00:15:11
And somebody correctly sent me
an email to say energy and

255
00:15:11 --> 00:15:14
length squared are you really,
is there much difference?

256
00:15:14 --> 00:15:15
No.
 

257
00:15:15 --> 00:15:16
No.
 

258
00:15:16 --> 00:15:18
You could say length squared
here, I'm just using

259
00:15:18 --> 00:15:19
the word energy.
 

260
00:15:19 --> 00:15:23
Now, I left a space because
I know that there's a

261
00:15:23 --> 00:15:26
stupid 2pi somewhere.
 

262
00:15:26 --> 00:15:29
Where does it come?
 

263
00:15:29 --> 00:15:32
You remember how to get this?
 

264
00:15:32 --> 00:15:37
You put that whole series in
there, multiply by its complex

265
00:15:37 --> 00:15:40
conjugate to get squared.
 

266
00:15:40 --> 00:15:42
And integrate.
 

267
00:15:42 --> 00:15:42
Right?
 

268
00:15:42 --> 00:15:43
That was the idea.
 

269
00:15:43 --> 00:15:46
Isn't that how we figured
out, we got to this?

270
00:15:46 --> 00:15:50
We started with this,
length squared.

271
00:15:50 --> 00:15:53
We plugged in the
Fourier series.

272
00:15:53 --> 00:15:56
This is f times f bar,
so that's this times

273
00:15:56 --> 00:15:57
its conjugate.
 

274
00:15:57 --> 00:16:01
And we integrated, and all
the cross terms vanished.

275
00:16:01 --> 00:16:08
And only the ones were, e^(ikx)
multiplied e^(-ikx) came.

276
00:16:08 --> 00:16:10
And those had c_k squared.
 

277
00:16:10 --> 00:16:14
And when we integrated that
one, we probably got a 2pi.

278
00:16:14 --> 00:16:17
 
 

279
00:16:17 --> 00:16:20
So that's the energy
inequality, right.

280
00:16:20 --> 00:16:21
For functions.
 

281
00:16:21 --> 00:16:26
And now what I was going to say
is, you shouldn't miss the fact

282
00:16:26 --> 00:16:32
that in the discrete case,
there'll be a similar

283
00:16:32 --> 00:16:33
energy inequality.
 

284
00:16:33 --> 00:16:40
So we had y was the
Fourier matrix times c.

285
00:16:40 --> 00:16:47
Now, if I take the length
squared of both, y, so

286
00:16:47 --> 00:16:49
I'm going to right?
 

287
00:16:49 --> 00:16:52
That's the same as that.
 

288
00:16:52 --> 00:16:56
Now I'm going to do
the same as this.

289
00:16:56 --> 00:16:59
I'm going to find the
length squared, which

290
00:16:59 --> 00:17:02
will be y transpose y.
 

291
00:17:02 --> 00:17:04
No, it won't be y transpose y.
 

292
00:17:04 --> 00:17:09
What is length squared?
y bar transpose y.

293
00:17:09 --> 00:17:11
I have to do that right.
 

294
00:17:11 --> 00:17:16
That will be, substituting
that's c bar F bar

295
00:17:16 --> 00:17:20
transpose times y is Fc.
 

296
00:17:22 --> 00:17:26
Just plugged it in, and
now what do I use?

297
00:17:26 --> 00:17:31
The key fact, the fact that
the columns are orthogonal.

298
00:17:31 --> 00:17:34
That's what made all these
integrals simple, right?

299
00:17:34 --> 00:17:37
When I put that into there,
a whole lot of integrals

300
00:17:37 --> 00:17:39
had to be zero.
 

301
00:17:39 --> 00:17:42
When I put this in, a
whole lot of dot products

302
00:17:42 --> 00:17:42
have to be zero.
 

303
00:17:42 --> 00:17:47
Rows of F bar times
columns of F, all zero.

304
00:17:47 --> 00:17:51
Except when I'm hitting the
same row, and when I'm hitting

305
00:17:51 --> 00:17:53
that same row I get an N.
 

306
00:17:53 --> 00:17:59
So I get this, is N
c bar transpose c.

307
00:17:59 --> 00:18:04
And that's c squared.
 

308
00:18:04 --> 00:18:08
That's the energy inequality,
it's just orthogonality

309
00:18:08 --> 00:18:10
once again.
 

310
00:18:10 --> 00:18:15
Everything in these weeks is
coming out of orthogonality.

311
00:18:15 --> 00:18:20
Orthogonality is the fact that
this is N times the identity.

312
00:18:20 --> 00:18:23
Right?
 

313
00:18:23 --> 00:18:30
Well, OK that's a quick
recall of a bunch of

314
00:18:30 --> 00:18:32
stuff for functions.
 

315
00:18:32 --> 00:18:36
And just seeing maybe
for the first time

316
00:18:36 --> 00:18:38
the discrete analogs.
 

317
00:18:38 --> 00:18:42
I guess I don't have a
brilliant idea for the discrete

318
00:18:42 --> 00:18:45
analog of the derivative.
 

319
00:18:45 --> 00:18:48
Well, guess there's a natural
idea, it would be a finite

320
00:18:48 --> 00:18:53
difference, but somehow that
isn't a rule that gets

321
00:18:53 --> 00:18:56
like, high marks.
 

322
00:18:56 --> 00:19:02
But we saw the discrete analog
of the shift and now we see the

323
00:19:02 --> 00:19:07
energy inequality is just that
the length of the function

324
00:19:07 --> 00:19:11
squared is equal to N
times the length of the

325
00:19:11 --> 00:19:13
coefficient squared.
 

326
00:19:13 --> 00:19:15
OK with that?
 

327
00:19:15 --> 00:19:18
Lots of formulas here.
 

328
00:19:18 --> 00:19:25
Let's see, and do
some examples.

329
00:19:25 --> 00:19:27
I mean, these were
simple examples.

330
00:19:27 --> 00:19:30
And I think the homework
gives you some more.

331
00:19:30 --> 00:19:34
You should be able to take
the Fourier transform

332
00:19:34 --> 00:19:37
and go backwards.
 

333
00:19:37 --> 00:19:42
And when we do convolution in a
few minutes, we're certainly

334
00:19:42 --> 00:19:45
going to be taking the Fourier,
we're going to be

335
00:19:45 --> 00:19:47
going both ways.
 

336
00:19:47 --> 00:19:51
And use all these facts.
 

337
00:19:51 --> 00:19:52
OK, I'll pause a moment.
 

338
00:19:52 --> 00:19:55
That's topic one.
 

339
00:19:55 --> 00:19:57
Topic two, fast
Fourier transform.

340
00:19:57 --> 00:20:00
Wow.
 

341
00:20:00 --> 00:20:07
What's the good way to - I
mean, any decent machine comes

342
00:20:07 --> 00:20:10
with the FFT hardwired in.
 

343
00:20:10 --> 00:20:13
So you might say, OK
I'll just use it.

344
00:20:13 --> 00:20:16
And that seems totally
reasonable to me.

345
00:20:16 --> 00:20:24
But you might like to
see just on one board,

346
00:20:24 --> 00:20:26
what's the key idea?
 

347
00:20:26 --> 00:20:30
What's the little bit of
algebra that makes it work?

348
00:20:30 --> 00:20:35
So I'll just have one board
here for the FFT and a

349
00:20:35 --> 00:20:38
little bit of algebra.
 

350
00:20:38 --> 00:20:46
Simple, simple but once it hit
the world, well, computer

351
00:20:46 --> 00:20:50
scientists just love the
recursion that comes in there.

352
00:20:50 --> 00:20:58
So they look for that in every
possible other algorithm, now.

353
00:20:58 --> 00:21:01
OK, let me see that point.
 

354
00:21:01 --> 00:21:03
So here's the main point.
 

355
00:21:03 --> 00:21:12
That if I want to take the
multiply by F of size 1,024,

356
00:21:12 --> 00:21:19
the fast Fourier transform
connects that full matrix

357
00:21:19 --> 00:21:21
to a half-full matrix.
 

358
00:21:21 --> 00:21:24
It connects that to the
half-full matrix that

359
00:21:24 --> 00:21:30
takes the half-size
transforms separately.

360
00:21:30 --> 00:21:36
So it's half-full because
of these zeroes.

361
00:21:36 --> 00:21:37
That's the point.
 

362
00:21:37 --> 00:21:44
That the 1,024 matrix is
connected to the 512 matrix.

363
00:21:44 --> 00:21:47
And what's underlying that?
 

364
00:21:47 --> 00:21:56
The 1,024 matrix is full
of e^(2pi*i), the w,

365
00:21:56 --> 00:21:59
over a 1,024, right?
 

366
00:21:59 --> 00:22:01
That's the w for this guy.
 

367
00:22:01 --> 00:22:07
And then the w for this
guy, for both of these,

368
00:22:07 --> 00:22:14
is e^(2pi*i) over 512.
 

369
00:22:14 --> 00:22:16
So if there's a connection
between that matrix and this

370
00:22:16 --> 00:22:19
matrix, there'd better be a
connection between that

371
00:22:19 --> 00:22:21
number and that number.
 

372
00:22:21 --> 00:22:24
Because this is the number that
fills this one, and this is

373
00:22:24 --> 00:22:26
the number that fills these.
 

374
00:22:26 --> 00:22:30
So what's the connection?
 

375
00:22:30 --> 00:22:32
It's just perfect, right?
 

376
00:22:32 --> 00:22:38
If I take this number, which is
one part of the whole circle,

377
00:22:38 --> 00:22:43
1/1,024, a fraction of the
whole circle, what do

378
00:22:43 --> 00:22:47
I do to get this guy?
 

379
00:22:47 --> 00:22:51
To get 1/512th of the
way around the circle?

380
00:22:51 --> 00:22:55
I square it.
 

381
00:22:55 --> 00:23:01
The square of this w is this w.
 

382
00:23:01 --> 00:23:07
Let me call this w_N,
and this one w_M.

383
00:23:07 --> 00:23:12
Maybe I'll use caps, yeah I'm
using cap N, this is my w.

384
00:23:12 --> 00:23:14
This is the one I want.
 

385
00:23:14 --> 00:23:21
The w_N, the N by N one, N is
1,024, and the point is,

386
00:23:21 --> 00:23:24
everybody saw that, when I
squared it I doubled the angle?

387
00:23:24 --> 00:23:31
When I doubled that angle the
two over that gave me 1/512.

388
00:23:31 --> 00:23:32
Fantastic.
 

389
00:23:32 --> 00:23:40
Of course, that doesn't make
this equal to that, but it

390
00:23:40 --> 00:23:44
suggests that there is
a close connection.

391
00:23:44 --> 00:23:49
So let me finish here the key
idea of the Fourier transform

392
00:23:49 --> 00:23:53
in block matrix form.
 

393
00:23:53 --> 00:23:55
What's the key idea?
 

394
00:23:55 --> 00:23:59
So instead of doing the big
transform, the full size, I'm

395
00:23:59 --> 00:24:01
going to do two half-sizes.
 

396
00:24:01 --> 00:24:08
But what am I going
to apply those to?

397
00:24:08 --> 00:24:10
Here's the trick.
 

398
00:24:10 --> 00:24:15
These two separate guys
apply to the odd-numbered

399
00:24:15 --> 00:24:16
coefficients.
 

400
00:24:16 --> 00:24:18
The odd-numbered component.
 

401
00:24:18 --> 00:24:19
And the even.
 

402
00:24:19 --> 00:24:25
So I have to first do a
little permutation, and

403
00:24:25 --> 00:24:27
even comes first, always.
 

404
00:24:27 --> 00:24:32
Even means zero, two,
four, up to a 1,022.

405
00:24:32 --> 00:24:38
So this is zero, two,
zero, up to 1,022.

406
00:24:38 --> 00:24:40
And then come all the odd guys.
 

407
00:24:40 --> 00:24:44
One up to 1,023.
 

408
00:24:44 --> 00:24:48
So this is a permutation,
a simple permutation.

409
00:24:48 --> 00:24:52
Just take your 1,024
numbers, pick out the even

410
00:24:52 --> 00:24:55
ones, put them on top.
 

411
00:24:55 --> 00:24:56
Right?
 

412
00:24:56 --> 00:24:58
In other words, put them
on top where 512 is

413
00:24:58 --> 00:25:00
going to act on it.
 

414
00:25:00 --> 00:25:03
Put the odd ones at the
bottom, the last 512, that

415
00:25:03 --> 00:25:05
guy will act on that.
 

416
00:25:05 --> 00:25:09
So there's 512 numbers
there, with the even

417
00:25:09 --> 00:25:10
coefficients, this acts.
 

418
00:25:10 --> 00:25:14
OK, now we've got two
half-size transforms.

419
00:25:14 --> 00:25:21
Because we're applying this to
the y, to a to a typical y.

420
00:25:21 --> 00:25:22
OK.
 

421
00:25:22 --> 00:25:25
But I've just written F
without a y so I don't

422
00:25:25 --> 00:25:27
really need a y here.
 

423
00:25:27 --> 00:25:30
This is a matrix identity.
 

424
00:25:30 --> 00:25:33
It's a matrix identity that's
got a bunch of zeroes there.

425
00:25:33 --> 00:25:36
Of course, that matrix
is full of zeroes.

426
00:25:36 --> 00:25:41
I mean, this is instant speed
to do that permutation.

427
00:25:41 --> 00:25:45
Grab the evens, put them
in front of the odds.

428
00:25:45 --> 00:25:49
OK, so now I've got two
half-sizes, but then I have

429
00:25:49 --> 00:25:53
to put them back together to
get the full-size matrix.

430
00:25:53 --> 00:25:55
And that is also a matrix.
 

431
00:25:55 --> 00:25:59
Turns out to be a diagonal
there, and a minus the

432
00:25:59 --> 00:26:05
diagonal goes there.
 

433
00:26:05 --> 00:26:09
So actually, that looks
great too, right?

434
00:26:09 --> 00:26:11
Full of zeroes, the identity.
 

435
00:26:11 --> 00:26:13
No multiplications whatever.
 

436
00:26:13 --> 00:26:16
Well, these are the only
multiplications, because

437
00:26:16 --> 00:26:19
sometimes they're called
twiddle factors, give

438
00:26:19 --> 00:26:21
it a fancy name.
 

439
00:26:21 --> 00:26:25
Official sounding name,
twiddle factors.

440
00:26:25 --> 00:26:29
OK, so that diagonal
matrix D, what's that?

441
00:26:29 --> 00:26:34
That diagonal matrix D happens
to be just the powers

442
00:26:34 --> 00:26:37
of w, sit along D.
 

443
00:26:37 --> 00:26:44
1 w up to w to the, this is
W_N, we're talking, the big

444
00:26:44 --> 00:26:48
W, and it's only half-size
so it goes up to M-1.

445
00:26:50 --> 00:26:53
Half, M is half of N.
 

446
00:26:53 --> 00:26:55
Everybody's got that, right?
 

447
00:26:55 --> 00:26:57
M is half of N here.
 

448
00:26:57 --> 00:27:01
I'll get that written
on the board.

449
00:27:01 --> 00:27:07
M is 512, N is 1,024, here
we have the powers of

450
00:27:07 --> 00:27:10
this guy up to 511.
 

451
00:27:10 --> 00:27:13
The total size being
512 because that's a

452
00:27:13 --> 00:27:17
512 by 512 matrix.
 

453
00:27:17 --> 00:27:21
I guess I can remember
somewhere, being at a

454
00:27:21 --> 00:27:24
conference, this was
probably soon after the

455
00:27:24 --> 00:27:29
FFT became sort of famous.
 

456
00:27:29 --> 00:27:34
And then somebody who was just
presenting the idea and as

457
00:27:34 --> 00:27:38
soon as it was presented
that way, I was happy.

458
00:27:38 --> 00:27:39
I guess.
 

459
00:27:39 --> 00:27:43
I thought OK, there
you see the idea.

460
00:27:43 --> 00:27:46
Permutation, reorder
the even-odd.

461
00:27:46 --> 00:27:50
Two half-size transforms,
put them back together.

462
00:27:50 --> 00:27:52
And what's happened here?
 

463
00:27:52 --> 00:27:59
The work of this matrix,
multiplying by this matrix,

464
00:27:59 --> 00:28:04
which would be 1,024 squared is
now practically cut in half.

465
00:28:04 --> 00:28:06
Because this is nothing.
 

466
00:28:06 --> 00:28:10
And we have just this diagonal
multiplication to do, and of

467
00:28:10 --> 00:28:12
course this is the same as
that, just with minus signs.

468
00:28:12 --> 00:28:16
So the total multiplications we
have to do, the total number of

469
00:28:16 --> 00:28:22
twiddle factors, is just 512,
twelve and then we're golden.

470
00:28:22 --> 00:28:27
So we've got half the work
plus 512, 512 operations.

471
00:28:27 --> 00:28:30
That's pretty good.
 

472
00:28:30 --> 00:28:33
And of course it gets better.
 

473
00:28:33 --> 00:28:35
How?
 

474
00:28:35 --> 00:28:38
Once you have the idea of
getting down to 512, what

475
00:28:38 --> 00:28:39
are you going to do now?
 

476
00:28:39 --> 00:28:46
This is the computer
scientist's favorite idea.

477
00:28:46 --> 00:28:47
Do it again.
 

478
00:28:47 --> 00:28:48
That's what it comes to.
 

479
00:28:48 --> 00:28:53
Whatever worked for 1,024 to
get to 512 is going to work.

480
00:28:53 --> 00:28:58
So now I'll split this up
into, well, each 512, so

481
00:28:58 --> 00:29:01
now I have to do, yeah.
 

482
00:29:01 --> 00:29:07
Let me write F_512 will
be, it's now smaller.

483
00:29:07 --> 00:29:17
An I, an I, and a D and a minus
D for the 512 size of F_256,

484
00:29:17 --> 00:29:25
256 and then the permutation,
the odd-even permutation P.

485
00:29:25 --> 00:29:30
So we're doing this idea
in there, and in there.

486
00:29:30 --> 00:29:38
So it's just recursive.
 

487
00:29:38 --> 00:29:39
Recursive.
 

488
00:29:39 --> 00:29:42
And now, if we go all the
way, so you see why I keep

489
00:29:42 --> 00:29:46
taking powers of two.
 

490
00:29:46 --> 00:29:48
It's not natural to
have powers of two.

491
00:29:48 --> 00:29:49
Two or three.
 

492
00:29:49 --> 00:29:51
Three is also good.
 

493
00:29:51 --> 00:29:55
I mean, all this gets so
optimized that powers of two

494
00:29:55 --> 00:29:57
or three are pretty good.
 

495
00:29:57 --> 00:30:00
And you just use the same idea.
 

496
00:30:00 --> 00:30:06
There'd be a similar idea here
for, if I was doing instead

497
00:30:06 --> 00:30:12
of odd-even, even-odd I was
doing maybe three groups.

498
00:30:12 --> 00:30:19
But stick with
two, that's fine.

499
00:30:19 --> 00:30:25
Then you might ask, if you were
a worrier I guess you might ask

500
00:30:25 --> 00:30:30
what if it's not
a power of two.

501
00:30:30 --> 00:30:32
I think you just add in zeroes.
 

502
00:30:32 --> 00:30:41
Just pad it out to be
the next power of two.

503
00:30:41 --> 00:30:43
Nothing difficult there.
 

504
00:30:43 --> 00:30:44
I think that's right.
 

505
00:30:44 --> 00:30:49
Hope that's right.
 

506
00:30:49 --> 00:30:55
And once this idea came out, of
course, people started looking.

507
00:30:55 --> 00:31:00
What if the number
here was prime?

508
00:31:00 --> 00:31:06
And found another neat bit
of algebra that worked

509
00:31:06 --> 00:31:08
OK for prime numbers.
 

510
00:31:08 --> 00:31:11
Using a little bit
of number theory.

511
00:31:11 --> 00:31:16
But the ultimate
winner was this one.

512
00:31:16 --> 00:31:25
So maybe I'll just refer you to
those pages in the book, which

513
00:31:25 --> 00:31:29
were, you'll spot this
matrix equality.

514
00:31:29 --> 00:31:33
And then next to it is the
algebra that you have

515
00:31:33 --> 00:31:37
to do to check it.
 

516
00:31:37 --> 00:31:42
I can just say, because I want
you to look to the right spot

517
00:31:42 --> 00:31:47
there, maybe I'll take out the
great - this is sometimes

518
00:31:47 --> 00:31:53
called after Parseval,
or some other person.

519
00:31:53 --> 00:31:56
Yeah, the algebra.
 

520
00:31:56 --> 00:32:02
Let me start the algebra
that made this thing work.

521
00:32:02 --> 00:32:08
We want the sum, when we
multiply Fourier F times

522
00:32:08 --> 00:32:14
something, we want the
sum of w^jk, right?

523
00:32:14 --> 00:32:20
That's the coefficient, that's
the entry of F, times c_k.

524
00:32:23 --> 00:32:25
Sum from k=0 to N-1.
 

525
00:32:26 --> 00:32:30
To 1,023.
 

526
00:32:30 --> 00:32:40
That's the y_j that we're
trying to compute.

527
00:32:40 --> 00:32:45
We're computing a 1,024 y's
from 1,024 c's by adding

528
00:32:45 --> 00:32:48
up the Fourier series
when we multiply by F.

529
00:32:48 --> 00:33:00
This is F, this is the equation
y=Fc written with subscripts.

530
00:33:00 --> 00:33:03
So this is what the matrices
are doing, and now where

531
00:33:03 --> 00:33:07
do I find that 512 thing.
 

532
00:33:07 --> 00:33:12
How do I get M into the
picture, remembering that the

533
00:33:12 --> 00:33:16
w_N squared was w_M, right?
 

534
00:33:16 --> 00:33:20
That's what we saw.
 

535
00:33:20 --> 00:33:24
This is the big number, this
is half of it, so this is a

536
00:33:24 --> 00:33:26
little bit of the part
around the circle.

537
00:33:26 --> 00:33:29
When I go twice as far I
get to the other one.

538
00:33:29 --> 00:33:34
So that's the thing
that we've got to use.

539
00:33:34 --> 00:33:37
Everything is going
to depend on that.

540
00:33:37 --> 00:33:41
So this was w_N, of course.
 

541
00:33:41 --> 00:33:44
This is the N by N transpose
So now comes what,

542
00:33:44 --> 00:33:47
what's the key idea?
 

543
00:33:47 --> 00:33:49
The key idea is split
into even and odd.

544
00:33:49 --> 00:33:51
And then use this.
 

545
00:33:51 --> 00:33:55
So split into the even
ones and the odd ones.

546
00:33:55 --> 00:34:01
So I write this as two separate
sums, a sum for the even ones

547
00:34:01 --> 00:34:09
w_N, now, so now the even
c_k, so I'm going to

548
00:34:09 --> 00:34:12
multiply by c_2k.
 

549
00:34:14 --> 00:34:17
These are the ones with even,
and the sum is only going to

550
00:34:17 --> 00:34:23
go from zero to M-1, right?
 

551
00:34:23 --> 00:34:25
This is only half of the terms.
 

552
00:34:25 --> 00:34:31
And then look on the other
half, plus the same sum of,

553
00:34:31 --> 00:34:32
but I didn't finish here.
 

554
00:34:32 --> 00:34:33
Let me finish.
 

555
00:34:33 --> 00:34:36
So I'm just picking out,
I'm taking, instead

556
00:34:36 --> 00:34:37
of k I'm doing 2k.
 

557
00:34:38 --> 00:34:43
So I have j times 2k here.
 

558
00:34:43 --> 00:34:49
And now these will be
the odd ones. c_(2k+1),

559
00:34:49 --> 00:34:49
and omega_N^j(2k+1).
 

560
00:34:49 --> 00:35:00
 
 

561
00:35:00 --> 00:35:03
It's a lot to ask you.
 

562
00:35:03 --> 00:35:11
To focus on this
bit of algebra.

563
00:35:11 --> 00:35:15
I hope you're going to go
away feeling, well it's

564
00:35:15 --> 00:35:17
pretty darn simple.
 

565
00:35:17 --> 00:35:20
I mean there'll be a lot
of indices, and if I push

566
00:35:20 --> 00:35:22
it all the way through
there'll be a few more.

567
00:35:22 --> 00:35:25
But the point is, it's
pretty darn simple.

568
00:35:25 --> 00:35:26
For example, this term.
 

569
00:35:26 --> 00:35:29
What have I got there?
 

570
00:35:29 --> 00:35:33
Look at that term,
that's beautiful.

571
00:35:33 --> 00:35:36
That's w_N squared.
 

572
00:35:36 --> 00:35:40
What is w_N squared?
 

573
00:35:40 --> 00:35:41
It's w_N.
 

574
00:35:42 --> 00:35:45
So instead of w_N squared,
I'm going to replace

575
00:35:45 --> 00:35:48
that w_N squared by w_M.
 

576
00:35:49 --> 00:35:52
And the sum goes from
zero to M-1, and what

577
00:35:52 --> 00:35:56
does that represent?
 

578
00:35:56 --> 00:35:59
That represents the F_512
multiplication, the

579
00:35:59 --> 00:36:00
half-size transform.
 

580
00:36:00 --> 00:36:03
It goes halfway, it
operates on the even

581
00:36:03 --> 00:36:08
ones, and it uses the M.
 

582
00:36:08 --> 00:36:13
It's just perfectly, so this is
nothing but the Fourier matrix,

583
00:36:13 --> 00:36:21
the M by M Fourier matrix,
acting on the even c's.

584
00:36:21 --> 00:36:23
That's what that is.
 

585
00:36:23 --> 00:36:26
And that's why we get
these identities.

586
00:36:26 --> 00:36:29
That's why we get these
identities, because they're

587
00:36:29 --> 00:36:37
acting on the even - the
top half is the even guy.

588
00:36:37 --> 00:36:40
OK, this is almost as good.
 

589
00:36:40 --> 00:36:42
This is the odd c's.
 

590
00:36:42 --> 00:36:47
Here I have, now do
I have w_M here?

591
00:36:47 --> 00:36:49
I've got to have w_M.
 

592
00:36:49 --> 00:36:52
 
 

593
00:36:52 --> 00:36:54
Well, you can see.
 

594
00:36:54 --> 00:36:56
It's not quite coming
out right, and that's

595
00:36:56 --> 00:36:58
the twiddle factor.
 

596
00:36:58 --> 00:37:05
I have to take out a w_N to the
power j to make things good.

597
00:37:05 --> 00:37:09
And when I take out that w_N to
the power j, that's what goes

598
00:37:09 --> 00:37:14
in the D, in the
diagonal matrix.

599
00:37:14 --> 00:37:16
Yeah.
 

600
00:37:16 --> 00:37:18
I won't go more than that.
 

601
00:37:18 --> 00:37:24
You couldn't, there's no reason
to do everything here on the

602
00:37:24 --> 00:37:27
board when the main
point is there.

603
00:37:27 --> 00:37:30
And then the point was
recursion and then, oh, let

604
00:37:30 --> 00:37:33
me complete the recursion.
 

605
00:37:33 --> 00:37:38
So I recurse down to
256, then 128, then 64.

606
00:37:38 --> 00:37:42
And what do I get in the end?
 

607
00:37:42 --> 00:37:45
What do I get altogether,
once I've got all the

608
00:37:45 --> 00:37:48
way down to size two?
 

609
00:37:48 --> 00:37:58
I have a whole lot of these
factors down in the middle, the

610
00:37:58 --> 00:38:01
part that used to be hard is
now down to F_2 or

611
00:38:01 --> 00:38:03
F_1 or something.
 

612
00:38:03 --> 00:38:07
So let's say F_1, one by
one, just the identity.

613
00:38:07 --> 00:38:09
So I go all the way.
 

614
00:38:09 --> 00:38:14
Ten steps from 1024,
512, 256, every time I

615
00:38:14 --> 00:38:16
get twiddle factors.
 

616
00:38:16 --> 00:38:19
Every time I get P's.
 

617
00:38:19 --> 00:38:25
A lot of P's, but the F_512,
what used to be the hard part,

618
00:38:25 --> 00:38:29
is gone to the easy part.
 

619
00:38:29 --> 00:38:31
And then what do I have?
 

620
00:38:31 --> 00:38:33
I've got just a
permutation there.

621
00:38:33 --> 00:38:38
And this is the actual work.
 

622
00:38:38 --> 00:38:45
This is the only work left,
is the matrices like this,

623
00:38:45 --> 00:38:49
for different sizes.
 

624
00:38:49 --> 00:38:50
And I have to do those.
 

625
00:38:50 --> 00:38:52
I have to do all those
twiddle factors.

626
00:38:52 --> 00:38:58
So how many matrices
are there, there?

627
00:38:58 --> 00:39:00
It's the log, right?
 

628
00:39:00 --> 00:39:02
Every time I divided by two.
 

629
00:39:02 --> 00:39:07
So if I started at 1,024, I do
ten times, I have ten of those

630
00:39:07 --> 00:39:12
matrices and have me down to
N=1. - So I've got ten of

631
00:39:12 --> 00:39:19
these, and each one takes 1,024
or maybe only 1/2 of that.

632
00:39:19 --> 00:39:22
Actually, only half because
this is a copy of that.

633
00:39:22 --> 00:39:27
I think the final count, and
can I just put in here, this is

634
00:39:27 --> 00:39:35
the great number, is each of
these took n multiplications.

635
00:39:35 --> 00:39:38
But there were only log to
the base two - oh, no.

636
00:39:38 --> 00:39:41
Each of them took half of N
multiplications, because the

637
00:39:41 --> 00:39:46
D and the minus D are just,
I don't have to repeat.

638
00:39:46 --> 00:39:50
So half N for each factor and
the number of factors is

639
00:39:50 --> 00:39:53
log to the base two of N.
 

640
00:39:53 --> 00:39:55
Ten, for 1,024.
 

641
00:39:55 --> 00:40:00
So that's the magic of the FFT.
 

642
00:40:00 --> 00:40:02
OK.
 

643
00:40:02 --> 00:40:06
It's almost all on one board,
one and a half boards.

644
00:40:06 --> 00:40:21
To tell you the key point, odds
and evens, recursion, twiddle

645
00:40:21 --> 00:40:25
factors, getting down to the
point where you only have

646
00:40:25 --> 00:40:30
twiddle factors left and then
those multiplications

647
00:40:30 --> 00:40:33
are only N log N.
 

648
00:40:33 --> 00:40:37
Good?
 

649
00:40:37 --> 00:40:38
Yes.
 

650
00:40:38 --> 00:40:40
Right, OK.
 

651
00:40:40 --> 00:40:47
Now. that's discrete transform.
 

652
00:40:47 --> 00:40:52
The theory behind it and
the fantastic algorithm

653
00:40:52 --> 00:40:59
that executes it.
 

654
00:40:59 --> 00:41:00
Are you ready for
a convolution?

655
00:41:00 --> 00:41:04
Can we start on a topic that's
really quite nice, and

656
00:41:04 --> 00:41:10
then Friday will be the
focus on convolution.

657
00:41:10 --> 00:41:14
Friday will certainly be
all convolution day.

658
00:41:14 --> 00:41:18
But maybe it's not a
bad idea to see now,

659
00:41:18 --> 00:41:21
what's the question.
 

660
00:41:21 --> 00:41:25
Let me ask that question.
 

661
00:41:25 --> 00:41:27
OK, convolution.
 

662
00:41:27 --> 00:41:32
So we're into the next
section of the book,

663
00:41:32 --> 00:41:42
Section 4.4, it must be.
 

664
00:41:42 --> 00:41:46
And let me do it first
for a Fourier series.

665
00:41:46 --> 00:41:50
I have convolution of series,
convolution of discrete.

666
00:41:50 --> 00:41:57
Convolution of integrals, but
we haven't got there yet.

667
00:41:57 --> 00:42:02
So I'll do this
one, this series.

668
00:42:02 --> 00:42:06
So let me start with a couple
of series. f(x) is the

669
00:42:06 --> 00:42:07
sum of c_k*e^(ikx).
 

670
00:42:07 --> 00:42:11
 
 

671
00:42:11 --> 00:42:18
g(x) is the sum of some
other coefficients.

672
00:42:18 --> 00:42:21
And I'm going to ask
you a simple question.

673
00:42:21 --> 00:42:27
What are the Fourier
coefficients of f times g?

674
00:42:27 --> 00:42:39
If I multiply those
functions, equals something.

675
00:42:39 --> 00:42:43
And let me call those
coefficients, h maybe.

676
00:42:43 --> 00:42:52
h_k*e^(ikx), and my question
is what are the coefficients

677
00:42:52 --> 00:42:59
h_k of f times g?
 

678
00:42:59 --> 00:43:10
That's the question that
convolution answers.

679
00:43:10 --> 00:43:14
Actually, both this series
and the discrete series

680
00:43:14 --> 00:43:17
are highly interesting.
 

681
00:43:17 --> 00:43:20
Highly interesting.
 

682
00:43:20 --> 00:43:25
So here I wrote it
for this series.

683
00:43:25 --> 00:43:34
If I write it for the discrete
ones, you'll see, so let me

684
00:43:34 --> 00:43:36
do it over here for
the discrete one.

685
00:43:36 --> 00:43:39
Because I can write it out
for the discrete ones.

686
00:43:39 --> 00:43:51
My y's are c_0+c_1 e - no, w
I have this nice notation w,

687
00:43:51 --> 00:43:58
plus c_N-1*w^(N-1), right?
 

688
00:43:58 --> 00:44:03
That's the - ooh.
 

689
00:44:03 --> 00:44:05
What's that?
 

690
00:44:05 --> 00:44:08
I haven't got that right.
 

691
00:44:08 --> 00:44:12
Yes, what do I want now?
 

692
00:44:12 --> 00:44:15
I need, yep.
 

693
00:44:15 --> 00:44:16
Sorry, I'm looking.
 

694
00:44:16 --> 00:44:20
Really, I'm looking at y_j,
the j'th component of y.

695
00:44:20 --> 00:44:24
So I need a w^j, w^j(N-1).
 

696
00:44:24 --> 00:44:26
 
 

697
00:44:26 --> 00:44:32
Yeah, OK, let me - OK.
 

698
00:44:32 --> 00:44:33
Alright.
 

699
00:44:33 --> 00:44:36
And so that's my f.
 

700
00:44:36 --> 00:44:44
I'll come back to that,
let me stay with this.

701
00:44:44 --> 00:44:49
I'll stay with this to make the
main point, and then Friday

702
00:44:49 --> 00:44:53
we'll see it in a neat way
for the discrete one.

703
00:44:53 --> 00:44:57
So I'm coming back to this.
f has its Fourier series

704
00:44:57 --> 00:44:59
g has its Fourier series.
 

705
00:44:59 --> 00:45:02
I multiply.
 

706
00:45:02 --> 00:45:10
What happens when I
multiply this times this?

707
00:45:10 --> 00:45:12
I'm not going to integrate.
 

708
00:45:12 --> 00:45:15
I mean, when I do this
multiplication, I'm going

709
00:45:15 --> 00:45:18
to get a mass of terms.
 

710
00:45:18 --> 00:45:21
A real lot of terms.
 

711
00:45:21 --> 00:45:23
And I'm not going to
integrate them away.

712
00:45:23 --> 00:45:25
So they're all there.
 

713
00:45:25 --> 00:45:27
So what am I asking?
 

714
00:45:27 --> 00:45:31
I'm asking to pick out all
the terms that have the

715
00:45:31 --> 00:45:36
same exponential with them.
 

716
00:45:36 --> 00:45:38
Like, what's h_0?
 

717
00:45:38 --> 00:45:42
Yes, tell me what h_0 is?
 

718
00:45:42 --> 00:45:45
If you can pick out h_0
here, you'll get the

719
00:45:45 --> 00:45:50
idea of convolution.
 

720
00:45:50 --> 00:45:55
What's the constant term if I
multiply this mess times this

721
00:45:55 --> 00:46:04
mess, and I look for the
constant term, h_0, where do I

722
00:46:04 --> 00:46:07
get the constant terms when
I multiply that by that?

723
00:46:07 --> 00:46:08
Just think about that.
 

724
00:46:08 --> 00:46:10
Where do I get a
constant, without any

725
00:46:10 --> 00:46:13
k, without an e^(ikx)?
 

726
00:46:13 --> 00:46:16
If I multiply that by that.
 

727
00:46:16 --> 00:46:21
Well tell me one place I get
something. c_0 times? d_0.

728
00:46:21 --> 00:46:25
Good.
 

729
00:46:25 --> 00:46:28
Is that the end of the story?
 

730
00:46:28 --> 00:46:29
No.
 

731
00:46:29 --> 00:46:32
If you thought that multiplying
the functions, I just

732
00:46:32 --> 00:46:34
multiplied the Fourier
coefficients, the

733
00:46:34 --> 00:46:36
first point is no.
 

734
00:46:36 --> 00:46:38
There's more stuff.
 

735
00:46:38 --> 00:46:41
Where else do I get a
constant out of this?

736
00:46:41 --> 00:46:45
Just look at it, do that
multiplication and ask yourself

737
00:46:45 --> 00:46:48
where's the constant.
 

738
00:46:48 --> 00:46:49
Another one, yep.
 

739
00:46:49 --> 00:46:54
You were going to say
it is? c_1 times d_-1.

740
00:46:54 --> 00:46:55
Right.
 

741
00:46:55 --> 00:46:58
Right. c_1 times d_-1.
 

742
00:46:59 --> 00:47:04
And tell me all of them,
now. c_2 times d_-2.

743
00:47:05 --> 00:47:07
And what about c_-1?
 

744
00:47:09 --> 00:47:09
There's a c_-1.
 

745
00:47:09 --> 00:47:12
 
 

746
00:47:12 --> 00:47:14
It multiplies d_1.
 

747
00:47:16 --> 00:47:19
And onwards.
 

748
00:47:19 --> 00:47:26
So the coefficient comes
from, now how could

749
00:47:26 --> 00:47:29
you describe that?
 

750
00:47:29 --> 00:47:35
I guess I'll describe it as,
I'll need a sum to multiply.

751
00:47:35 --> 00:47:39
This will be the sum
of c_k times d_1.

752
00:47:39 --> 00:47:42
 
 

753
00:47:42 --> 00:47:45
Minus k, right?
 

754
00:47:45 --> 00:47:47
That's what you told me.
 

755
00:47:47 --> 00:47:52
Piece at the start, and that's
the pattern that keeps going.

756
00:47:52 --> 00:47:56
OK, that's h_0, the sum
of c_k times d_-k.

757
00:47:58 --> 00:48:05
Now, we have just time
to do the next one.

758
00:48:05 --> 00:48:07
We've got time but not
space, where the heck

759
00:48:07 --> 00:48:09
am I going to put it?
 

760
00:48:09 --> 00:48:14
I want to do h_k, I guess.
 

761
00:48:14 --> 00:48:18
Or I better use a different
letter h. l, let me use the

762
00:48:18 --> 00:48:22
letter h_l, and God
there's no space.

763
00:48:22 --> 00:48:31
Alright, so can I - yes. h_l.
 

764
00:48:31 --> 00:48:32
OK.
 

765
00:48:32 --> 00:48:36
So this was h_0, let me keep
things sort of looking

766
00:48:36 --> 00:48:37
right for the moment.
 

767
00:48:37 --> 00:48:39
OK, now you're
going to fix h_l.

768
00:48:39 --> 00:48:44
So what does c_0 multiply
if I'm looking for h_l,

769
00:48:44 --> 00:48:46
I'm looking for the
coefficient of e^(ilx).

770
00:48:47 --> 00:48:51
So ask yourself how do I
get e^(ilx) when that

771
00:48:51 --> 00:48:54
multiplies that?
 

772
00:48:54 --> 00:48:57
When that multiplies that, and
I'm looking for an e^(ilx), I

773
00:48:57 --> 00:49:02
get one when c_0 all
multiplies what?

774
00:49:02 --> 00:49:05
This is it. dl, right.
 

775
00:49:05 --> 00:49:07
And what about for c_1?
 

776
00:49:07 --> 00:49:09
 
 

777
00:49:09 --> 00:49:11
Think of here, I
have a c_1*e^(ilx).

778
00:49:11 --> 00:49:14
 
 

779
00:49:14 --> 00:49:17
What does it multiply
down here to get the

780
00:49:17 --> 00:49:20
exponential to be l? ilx?
 

781
00:49:22 --> 00:49:25
What doesn't multiply d_-1.
 

782
00:49:25 --> 00:49:30
It multiplies, c_1
multiplies? d_(l-1).

783
00:49:31 --> 00:49:37
Good, good. l-1, right. l-1,
and what are you noticing

784
00:49:37 --> 00:49:40
here? c minus, I'll
have to fill that in.

785
00:49:40 --> 00:49:45
But you're seeing
the pattern here?

786
00:49:45 --> 00:49:47
And what was the pattern here?
 

787
00:49:47 --> 00:49:51
Those numbers added
to this number.

788
00:49:51 --> 00:49:55
And now these numbers add to l
Those numbers add to l,

789
00:49:55 --> 00:50:03
whatever it is, the two indices
have to add to l, so that when

790
00:50:03 --> 00:50:06
I multiply the exponential
they'll add to e^(ilx).

791
00:50:08 --> 00:50:08
They'll multiply to e^(ilx).
 

792
00:50:08 --> 00:50:11
 
 

793
00:50:11 --> 00:50:14
So what goes there?
 

794
00:50:14 --> 00:50:16
It's probably l+1, right?
 

795
00:50:16 --> 00:50:22
So that l+1 combined with
minus one gives me the l.

796
00:50:22 --> 00:50:26
If you tell me what goes
there, I'm a happy person.

797
00:50:26 --> 00:50:27
Let's make it h_l.
 

798
00:50:27 --> 00:50:33
We're ready for the final
formula for convolutions.

799
00:50:33 --> 00:50:39
Big star.
 

800
00:50:39 --> 00:50:45
To find h_l, the coefficient of
e^(ilx), when you multiply that

801
00:50:45 --> 00:50:53
by that, you look at c_k, and
which d is going to show up in

802
00:50:53 --> 00:50:59
the e^(ilx) term? l-k,
is that what you said?

803
00:50:59 --> 00:51:02
I hope, yeah. l-k.
 

804
00:51:02 --> 00:51:04
Right, that's it.
 

805
00:51:04 --> 00:51:05
That's it.
 

806
00:51:05 --> 00:51:10
So we've got a lot of
computation here.

807
00:51:10 --> 00:51:17
But we've got the idea of
what, we've got a formula.

808
00:51:17 --> 00:51:20
And most of all we
have the magic rule.

809
00:51:20 --> 00:51:27
In convolutions, convolutions
are, things multiply

810
00:51:27 --> 00:51:29
but indices add.
 

811
00:51:29 --> 00:51:33
Things multiply, numbers
multiply, while

812
00:51:33 --> 00:51:34
their indices add.
 

813
00:51:34 --> 00:51:36
That's the key idea of
convolution that we'll

814
00:51:36 --> 00:51:40
see clearly and
completely on Friday.

815
00:51:40 --> 00:51:41
OK.