1
00:00:00 --> 00:00:01
2
00:00:01 --> 00:00:03
The following content is
provided under a Creative
3
00:00:03 --> 00:00:03
Commons license.
4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to
5
00:00:06 --> 00:00:09
offer high-quality educational
resources for free.
6
00:00:09 --> 00:00:12
To make a donation, or to view
additional materials from
7
00:00:12 --> 00:00:16
hundreds of MIT courses, visit
MIT OpenCourseWare
8
00:00:16 --> 00:00:21
at ocw.mit.edu.
9
00:00:21 --> 00:00:23
OK, so.
10
00:00:23 --> 00:00:25
Pretty full day again.
11
00:00:25 --> 00:00:30
I had last time introduced
the Fourier matrix, the
12
00:00:30 --> 00:00:33
discrete Fourier transform.
13
00:00:33 --> 00:00:37
Well, more strictly, the
discrete Fourier transform
14
00:00:37 --> 00:00:38
is usually this one.
15
00:00:38 --> 00:00:43
It takes the function values
and produces the coefficients.
16
00:00:43 --> 00:00:47
And then I started with the
coefficients added back,
17
00:00:47 --> 00:00:51
added up the series to
get the function values.
18
00:00:51 --> 00:00:53
So F or F inverse.
19
00:00:53 --> 00:00:56
So we didn't do examples yet.
20
00:00:56 --> 00:01:03
And one natural example is the
discrete delta function that
21
00:01:03 --> 00:01:06
has a one in the zero position.
22
00:01:06 --> 00:01:07
That's easy to do.
23
00:01:07 --> 00:01:12
And then we should do also a
shift, to see what's the
24
00:01:12 --> 00:01:15
effect, if you shift the
function, what happens
25
00:01:15 --> 00:01:17
to the transform.
26
00:01:17 --> 00:01:19
That's an important rule,
important for Fourier series
27
00:01:19 --> 00:01:21
and Fourier integrals, too.
28
00:01:21 --> 00:01:26
Because often you do that, and
it's going to be a simple rule.
29
00:01:26 --> 00:01:31
If you shift the function, the
transform does something nice.
30
00:01:31 --> 00:01:35
OK, and then I want to describe
a little about the FFT and
31
00:01:35 --> 00:01:38
then start on the next
section, convolutions.
32
00:01:38 --> 00:01:42
So that's fun and
that's a big deal.
33
00:01:42 --> 00:01:47
OK, about reviews, I'll
be here as usual today.
34
00:01:47 --> 00:01:53
I think maybe the 26th, just
hours before Thanksgiving we
35
00:01:53 --> 00:01:56
can give ourselves a holiday.
36
00:01:56 --> 00:02:01
So not next Wednesday but then
certainly the Wednesday of the
37
00:02:01 --> 00:02:08
following week would be a sort
of major quiz review on
38
00:02:08 --> 00:02:10
in the review session.
39
00:02:10 --> 00:02:11
And in class.
40
00:02:11 --> 00:02:13
OK, ready to go?
41
00:02:13 --> 00:02:19
On this example gives us a
chance just to remember what
42
00:02:19 --> 00:02:22
the matrix looks like, because
we're just going to, if I
43
00:02:22 --> 00:02:26
multiply this inverse matrix by
that vector it's just going to
44
00:02:26 --> 00:02:30
pick off the first column and
it'll be totally easy
45
00:02:30 --> 00:02:31
so let's just do it.
46
00:02:31 --> 00:02:36
So if y is this one, I
want to know about f.
47
00:02:36 --> 00:02:42
What are the Fourier
coefficients of the
48
00:02:42 --> 00:02:44
delta function?
49
00:02:44 --> 00:02:45
Discrete delta function?
50
00:02:45 --> 00:02:49
OK, before I even do it,
we got a pretty good
51
00:02:49 --> 00:02:51
idea what to expect.
52
00:02:51 --> 00:02:54
Because we remember what
happened to the ordinary delta
53
00:02:54 --> 00:02:57
function, in continuous time.
54
00:02:57 --> 00:03:00
Or rather, I guess it was the
periodic delta function.
55
00:03:00 --> 00:03:02
Do you remember the
coefficients, what were
56
00:03:02 --> 00:03:05
the coefficients for the
periodic delta function?
57
00:03:05 --> 00:03:07
You remember those?
58
00:03:07 --> 00:03:11
We took the integral from minus
pi to pi of our function,
59
00:03:11 --> 00:03:12
which was delta(x).
60
00:03:13 --> 00:03:18
And then we had to remember to
divide by 2pi, and you remember
61
00:03:18 --> 00:03:19
the coefficients are e^(-ikx).
62
00:03:19 --> 00:03:23
63
00:03:23 --> 00:03:29
This is c_k in the periodic
case, the 2pi periodic case the
64
00:03:29 --> 00:03:32
function is the delta function,
and you remember that if we
65
00:03:32 --> 00:03:35
want coefficient k we
multiply by e^(-ikx).
66
00:03:37 --> 00:03:42
That's the thing that will pick
out the e^(+ikx) term and of
67
00:03:42 --> 00:03:47
course everybody knows what we
get here, this delta, this
68
00:03:47 --> 00:03:53
spike at x=0, means we take the
value of that function at zero,
69
00:03:53 --> 00:03:55
which is one, so we
just get 1/2pi.
70
00:03:55 --> 00:03:57
71
00:03:57 --> 00:04:00
For all the Fourier
coefficients of the
72
00:04:00 --> 00:04:02
delta function.
73
00:04:02 --> 00:04:05
The point being that
they're all the same.
74
00:04:05 --> 00:04:08
That all frequencies are
in the delta function
75
00:04:08 --> 00:04:09
to the same amount.
76
00:04:09 --> 00:04:11
I mean that's kind of nice.
77
00:04:11 --> 00:04:15
That we created the delta
function for other reasons, but
78
00:04:15 --> 00:04:21
then here in Fourier space
it's just clean as could be.
79
00:04:21 --> 00:04:23
And we'll expect
something here, too.
80
00:04:23 --> 00:04:25
You remember what F inverse is?
81
00:04:25 --> 00:04:27
F inverse 1/N.
82
00:04:28 --> 00:04:34
Instead of 1/2pi, and then the
entries of F inverse come
83
00:04:34 --> 00:04:38
from F bar, the conjugate.
84
00:04:38 --> 00:04:42
So it's just one, one, one,
minus - well, I've made
85
00:04:42 --> 00:04:44
it four by four here.
86
00:04:44 --> 00:04:50
This is minus omega - no,
it isn't minus omega.
87
00:04:50 --> 00:04:58
It's it's omega bar, which
is minus i, in this case.
88
00:04:58 --> 00:05:02
Omega bar, so it's minus
i, that's omega bar.
89
00:05:02 --> 00:05:04
And the next one would
be omega bar squared,
90
00:05:04 --> 00:05:09
and cubed, and so on.
91
00:05:09 --> 00:05:12
All the way up to
the ninth power.
92
00:05:12 --> 00:05:14
But we're multiplying
by one, zero, zero, so
93
00:05:14 --> 00:05:19
none of that matters.
94
00:05:19 --> 00:05:24
What's the answer?
95
00:05:24 --> 00:05:28
I'm doing this discrete Fourier
transform, so I'm multiplying
96
00:05:28 --> 00:05:32
by the matrix with the
complex conjugate guys.
97
00:05:32 --> 00:05:35
But I'm multiplying by that
simple thing so it's just going
98
00:05:35 --> 00:05:37
to pick out the 0th column.
99
00:05:37 --> 00:05:40
In other words, constant.
100
00:05:40 --> 00:05:44
All the Fourier discrete
Fourier coefficients of the
101
00:05:44 --> 00:05:46
discrete delta are the same.
102
00:05:46 --> 00:05:47
Just again.
103
00:05:47 --> 00:05:49
And what are they?
104
00:05:49 --> 00:05:52
So it picks out this column,
but of course it divides by
105
00:05:52 --> 00:05:54
N, so the answer was 1/N.
106
00:05:56 --> 00:05:58
.
107
00:05:58 --> 00:06:03
It's just constant with the
1/N, where in the continuous
108
00:06:03 --> 00:06:04
case we had 1/2pi.
109
00:06:05 --> 00:06:06
No problem.
110
00:06:06 --> 00:06:08
OK.
111
00:06:08 --> 00:06:12
And, of course, everybody
knows, suppose that I now start
112
00:06:12 --> 00:06:16
with these coefficients and add
back to get the function.
113
00:06:16 --> 00:06:20
What would I get?
114
00:06:20 --> 00:06:24
Because just to be sure that we
believe that F and F inverse
115
00:06:24 --> 00:06:26
really are what they
are supposed to be.
116
00:06:26 --> 00:06:33
If I start with these
coefficients, add back to put
117
00:06:33 --> 00:06:39
those in here and reconstruct,
so 1/N, <1, 1, 1,
118
00:06:39 --> 00:06:43
1>, what will I get?
119
00:06:43 --> 00:06:46
Well, what am I
supposed to get?
120
00:06:46 --> 00:06:47
The delta, right?
121
00:06:47 --> 00:06:50
I'm supposed to get back to y.
122
00:06:50 --> 00:06:55
If I started with that, did F
inverse to get the
123
00:06:55 --> 00:06:58
coefficients, that was the
discrete Fourier transform, now
124
00:06:58 --> 00:07:03
I add back to get, add the
Fourier series up again to come
125
00:07:03 --> 00:07:07
back here, well I'll certainly
get ,
126
00:07:07 --> 00:07:09
and you see why?
127
00:07:09 --> 00:07:15
If I multiply F, that zeroth
row of F is , times
128
00:07:15 --> 00:07:18
will give me N.
129
00:07:18 --> 00:07:19
The N will cancel.
130
00:07:19 --> 00:07:21
I get the one.
131
00:07:21 --> 00:07:25
And all the other
guys add to zeroes.
132
00:07:25 --> 00:07:31
So, sure enough, it works.
133
00:07:31 --> 00:07:34
We're really just seeing
an example, an important
134
00:07:34 --> 00:07:37
example of the DFT.
135
00:07:37 --> 00:07:40
And the homework, then, would
have some other examples.
136
00:07:40 --> 00:07:44
But I've forgotten whether the
homework has this example.
137
00:07:44 --> 00:07:47
But let's think about it now.
138
00:07:47 --> 00:07:54
Suppose that's my
function value instead.
139
00:07:54 --> 00:07:59
OK, so now I'm starting
with the delta.
140
00:07:59 --> 00:08:02
Again it's a delta,
but it's moved over.
141
00:08:02 --> 00:08:06
And I could ask, I really
should ask first, in the
142
00:08:06 --> 00:08:11
continuous case, suppose
I, I can do it with just
143
00:08:11 --> 00:08:13
a little erasing here.
144
00:08:13 --> 00:08:16
Let me do the continuous case
for the delta that we met
145
00:08:16 --> 00:08:18
first in this course.
146
00:08:18 --> 00:08:20
I'll shift it to a.
147
00:08:20 --> 00:08:26
If I shift the delta function
to a point a, well, I said
148
00:08:26 --> 00:08:29
we'd met the delta function
first in this course.
149
00:08:29 --> 00:08:31
At a, good.
150
00:08:31 --> 00:08:35
But when we did it
wasn't 2pi periodic.
151
00:08:35 --> 00:08:41
So we still have, in fact, the
Fourier integrals next week.
152
00:08:41 --> 00:08:42
Will have a similar formula.
153
00:08:42 --> 00:08:46
The integral will go from minus
infinity to infinity and
154
00:08:46 --> 00:08:49
then we'll have the real
delta, not periodic.
155
00:08:49 --> 00:08:53
Here, we have, and people
call it a train of deltas.
156
00:08:53 --> 00:08:55
A train of spikes.
157
00:08:55 --> 00:08:57
Sort of you have one every 2pi.
158
00:08:58 --> 00:09:00
Anyway, that's what we've got.
159
00:09:00 --> 00:09:03
Now, you can see the answer.
160
00:09:03 --> 00:09:07
This is like in perfect
practice in doing an
161
00:09:07 --> 00:09:09
integral with a delta.
162
00:09:09 --> 00:09:15
What's the integral equal?
163
00:09:15 --> 00:09:16
Well, the spike is at x=a.
164
00:09:18 --> 00:09:24
So it picks this function
at x=a, which is e^(-ika).
165
00:09:24 --> 00:09:30
166
00:09:30 --> 00:09:32
So not constant any more.
167
00:09:32 --> 00:09:35
They depend on k.
168
00:09:35 --> 00:09:38
The 1/2pi's still there.
169
00:09:38 --> 00:09:43
So it's, but still, the delta
function shifted over.
170
00:09:43 --> 00:09:45
I mean, it didn't
change energy.
171
00:09:45 --> 00:09:49
It didn't change, it just
changed phase, so to speak.
172
00:09:49 --> 00:10:00
And we see that, I would call
this like a modulation.
173
00:10:00 --> 00:10:04
So it's staying of absolute
value one, still.
174
00:10:04 --> 00:10:07
But it's not the number one,
it's going around the circle.
175
00:10:07 --> 00:10:10
Going around the unit circle.
176
00:10:10 --> 00:10:13
So it's a phase factor, right.
177
00:10:13 --> 00:10:16
And that's what I'm going
to expect to see here in
178
00:10:16 --> 00:10:17
the discrete case too.
179
00:10:17 --> 00:10:22
If I do this multiplication
by one there, it picks
180
00:10:22 --> 00:10:24
out this column, right?
181
00:10:24 --> 00:10:28
That one will pick out this
column, so you see it's
182
00:10:28 --> 00:10:31
maybe I come up here now.
183
00:10:31 --> 00:10:41
Shall I just, when I pick out
that column, the answer then,
184
00:10:41 --> 00:10:46
I guess I've got the column
circle, there it is minus i.
185
00:10:46 --> 00:10:49
Minus i squared, minus i cubed.
186
00:10:49 --> 00:10:53
You see it's like k equals
zero, one, two, three.
187
00:10:53 --> 00:10:58
Just the way here, we had k,
well we had all integers.
188
00:10:58 --> 00:11:01
k in that function case.
189
00:11:01 --> 00:11:04
Here we've got four integers, k
equals zero, one, two, and
190
00:11:04 --> 00:11:10
three, but again it's the minus
i, it's the e to the,
191
00:11:10 --> 00:11:13
it's the w bar.
192
00:11:13 --> 00:11:18
In other words, the answer
was one w bar w bar
193
00:11:18 --> 00:11:19
squared w bar cubed.
194
00:11:19 --> 00:11:24
Just the powers of w
with this factor 1/N.
195
00:11:25 --> 00:11:28
Here we had a modulation.
196
00:11:28 --> 00:11:31
It's the same picture.
197
00:11:31 --> 00:11:33
Absolute value's one.
198
00:11:33 --> 00:11:34
And and what about energy?
199
00:11:34 --> 00:11:42
Having mentioned energy, so
that's another key rule.
200
00:11:42 --> 00:11:47
The key rules for the Fourier
series, just let's think back.
201
00:11:47 --> 00:11:49
What were the key rules?
202
00:11:49 --> 00:11:52
First, the rule to find
the coefficients.
203
00:11:52 --> 00:11:53
Good.
204
00:11:53 --> 00:11:56
Then the rule for
the derivatives.
205
00:11:56 --> 00:11:59
This is so important.
206
00:11:59 --> 00:12:03
These are rules.
207
00:12:03 --> 00:12:05
Let's say, for Fourier series.
208
00:12:05 --> 00:12:08
For Fourier series.
209
00:12:08 --> 00:12:11
Let's just make this
a quick review.
210
00:12:11 --> 00:12:14
What were the important rules?
211
00:12:14 --> 00:12:18
The important rules were, if I
had the Fourier series of f.
212
00:12:18 --> 00:12:20
Start with the
Fourier series of f.
213
00:12:20 --> 00:12:24
Then the question was, what's
the Fourier series of df/dx.
214
00:12:24 --> 00:12:28
215
00:12:28 --> 00:12:32
And now I'm saying the next
important rule is the Fourier
216
00:12:32 --> 00:12:38
series of f, shifted.
217
00:12:38 --> 00:12:45
And then the last important
rule is the energy.
218
00:12:45 --> 00:12:50
OK, and let's just, maybe this
is a bad idea to, since we're
219
00:12:50 --> 00:12:57
kind of doing all of Fourier
in, it's coming in three parts.
220
00:12:57 --> 00:13:05
Functions, discrete, integrals,
but they all match.
221
00:13:05 --> 00:13:10
So this is what happens
to the function.
222
00:13:10 --> 00:13:12
What happens to
the coefficient?
223
00:13:12 --> 00:13:19
So this starts with coefficient
c_k, for f, what are the
224
00:13:19 --> 00:13:22
coefficients for the
derivative, just remind me?
225
00:13:22 --> 00:13:28
If f(x), so I'm starting with
f(x) equals sum of c_k*e^(ikx).
226
00:13:28 --> 00:13:32
227
00:13:32 --> 00:13:33
Start with that.
228
00:13:33 --> 00:13:35
And now take the derivative.
229
00:13:35 --> 00:13:37
When I take the derivative,
down comes ik.
230
00:13:39 --> 00:13:41
So you remember that rule.
231
00:13:41 --> 00:13:44
Those are the Fourier
coefficients of the derivative.
232
00:13:44 --> 00:13:47
Now what's the Fourier
coefficients of the shift?
233
00:13:47 --> 00:13:51
If I've just shifted,
translated the function, if
234
00:13:51 --> 00:13:56
my original x was this,
now let me look at f(x-a).
235
00:13:56 --> 00:13:59
You'll see it.
236
00:13:59 --> 00:14:02
It'll jump out at us, it'll
be a sum of the same
237
00:14:02 --> 00:14:04
c_k's e^ik(x-a).
238
00:14:04 --> 00:14:09
239
00:14:09 --> 00:14:12
So what are the Fourier
coefficients of that?
240
00:14:12 --> 00:14:13
Well there is the e^(ikx).
241
00:14:15 --> 00:14:17
Whatever's multiplying it
has got to be the Fourier
242
00:14:17 --> 00:14:21
coefficient, and we see it
as c_k times e^ik(-a).
243
00:14:23 --> 00:14:28
e^(-ika), times c_k.
244
00:14:30 --> 00:14:32
Right?
245
00:14:32 --> 00:14:35
And, of course, that's just
what we discovered here.
246
00:14:35 --> 00:14:38
That's just what we found
there, that when we shifted the
247
00:14:38 --> 00:14:43
delta, we've multiplied by this
modulation, this phase factor
248
00:14:43 --> 00:14:46
came into the Fourier
coefficients.
249
00:14:46 --> 00:14:49
And now finally,
the energy stuff.
250
00:14:49 --> 00:14:52
You remember the energy
was, what's the energy?
251
00:14:52 --> 00:14:59
The integral from minus pi to
pi, of f(x) squared. dx is the
252
00:14:59 --> 00:15:04
same as the sum from minus
infinity to infinity of
253
00:15:04 --> 00:15:06
the coefficient squared.
254
00:15:06 --> 00:15:11
And somebody correctly sent me
an email to say energy and
255
00:15:11 --> 00:15:14
length squared are you really,
is there much difference?
256
00:15:14 --> 00:15:15
No.
257
00:15:15 --> 00:15:16
No.
258
00:15:16 --> 00:15:18
You could say length squared
here, I'm just using
259
00:15:18 --> 00:15:19
the word energy.
260
00:15:19 --> 00:15:23
Now, I left a space because
I know that there's a
261
00:15:23 --> 00:15:26
stupid 2pi somewhere.
262
00:15:26 --> 00:15:29
Where does it come?
263
00:15:29 --> 00:15:32
You remember how to get this?
264
00:15:32 --> 00:15:37
You put that whole series in
there, multiply by its complex
265
00:15:37 --> 00:15:40
conjugate to get squared.
266
00:15:40 --> 00:15:42
And integrate.
267
00:15:42 --> 00:15:42
Right?
268
00:15:42 --> 00:15:43
That was the idea.
269
00:15:43 --> 00:15:46
Isn't that how we figured
out, we got to this?
270
00:15:46 --> 00:15:50
We started with this,
length squared.
271
00:15:50 --> 00:15:53
We plugged in the
Fourier series.
272
00:15:53 --> 00:15:56
This is f times f bar,
so that's this times
273
00:15:56 --> 00:15:57
its conjugate.
274
00:15:57 --> 00:16:01
And we integrated, and all
the cross terms vanished.
275
00:16:01 --> 00:16:08
And only the ones were, e^(ikx)
multiplied e^(-ikx) came.
276
00:16:08 --> 00:16:10
And those had c_k squared.
277
00:16:10 --> 00:16:14
And when we integrated that
one, we probably got a 2pi.
278
00:16:14 --> 00:16:17
279
00:16:17 --> 00:16:20
So that's the energy
inequality, right.
280
00:16:20 --> 00:16:21
For functions.
281
00:16:21 --> 00:16:26
And now what I was going to say
is, you shouldn't miss the fact
282
00:16:26 --> 00:16:32
that in the discrete case,
there'll be a similar
283
00:16:32 --> 00:16:33
energy inequality.
284
00:16:33 --> 00:16:40
So we had y was the
Fourier matrix times c.
285
00:16:40 --> 00:16:47
Now, if I take the length
squared of both, y, so
286
00:16:47 --> 00:16:49
I'm going to right?
287
00:16:49 --> 00:16:52
That's the same as that.
288
00:16:52 --> 00:16:56
Now I'm going to do
the same as this.
289
00:16:56 --> 00:16:59
I'm going to find the
length squared, which
290
00:16:59 --> 00:17:02
will be y transpose y.
291
00:17:02 --> 00:17:04
No, it won't be y transpose y.
292
00:17:04 --> 00:17:09
What is length squared?
y bar transpose y.
293
00:17:09 --> 00:17:11
I have to do that right.
294
00:17:11 --> 00:17:16
That will be, substituting
that's c bar F bar
295
00:17:16 --> 00:17:20
transpose times y is Fc.
296
00:17:22 --> 00:17:26
Just plugged it in, and
now what do I use?
297
00:17:26 --> 00:17:31
The key fact, the fact that
the columns are orthogonal.
298
00:17:31 --> 00:17:34
That's what made all these
integrals simple, right?
299
00:17:34 --> 00:17:37
When I put that into there,
a whole lot of integrals
300
00:17:37 --> 00:17:39
had to be zero.
301
00:17:39 --> 00:17:42
When I put this in, a
whole lot of dot products
302
00:17:42 --> 00:17:42
have to be zero.
303
00:17:42 --> 00:17:47
Rows of F bar times
columns of F, all zero.
304
00:17:47 --> 00:17:51
Except when I'm hitting the
same row, and when I'm hitting
305
00:17:51 --> 00:17:53
that same row I get an N.
306
00:17:53 --> 00:17:59
So I get this, is N
c bar transpose c.
307
00:17:59 --> 00:18:04
And that's c squared.
308
00:18:04 --> 00:18:08
That's the energy inequality,
it's just orthogonality
309
00:18:08 --> 00:18:10
once again.
310
00:18:10 --> 00:18:15
Everything in these weeks is
coming out of orthogonality.
311
00:18:15 --> 00:18:20
Orthogonality is the fact that
this is N times the identity.
312
00:18:20 --> 00:18:23
Right?
313
00:18:23 --> 00:18:30
Well, OK that's a quick
recall of a bunch of
314
00:18:30 --> 00:18:32
stuff for functions.
315
00:18:32 --> 00:18:36
And just seeing maybe
for the first time
316
00:18:36 --> 00:18:38
the discrete analogs.
317
00:18:38 --> 00:18:42
I guess I don't have a
brilliant idea for the discrete
318
00:18:42 --> 00:18:45
analog of the derivative.
319
00:18:45 --> 00:18:48
Well, guess there's a natural
idea, it would be a finite
320
00:18:48 --> 00:18:53
difference, but somehow that
isn't a rule that gets
321
00:18:53 --> 00:18:56
like, high marks.
322
00:18:56 --> 00:19:02
But we saw the discrete analog
of the shift and now we see the
323
00:19:02 --> 00:19:07
energy inequality is just that
the length of the function
324
00:19:07 --> 00:19:11
squared is equal to N
times the length of the
325
00:19:11 --> 00:19:13
coefficient squared.
326
00:19:13 --> 00:19:15
OK with that?
327
00:19:15 --> 00:19:18
Lots of formulas here.
328
00:19:18 --> 00:19:25
Let's see, and do
some examples.
329
00:19:25 --> 00:19:27
I mean, these were
simple examples.
330
00:19:27 --> 00:19:30
And I think the homework
gives you some more.
331
00:19:30 --> 00:19:34
You should be able to take
the Fourier transform
332
00:19:34 --> 00:19:37
and go backwards.
333
00:19:37 --> 00:19:42
And when we do convolution in a
few minutes, we're certainly
334
00:19:42 --> 00:19:45
going to be taking the Fourier,
we're going to be
335
00:19:45 --> 00:19:47
going both ways.
336
00:19:47 --> 00:19:51
And use all these facts.
337
00:19:51 --> 00:19:52
OK, I'll pause a moment.
338
00:19:52 --> 00:19:55
That's topic one.
339
00:19:55 --> 00:19:57
Topic two, fast
Fourier transform.
340
00:19:57 --> 00:20:00
Wow.
341
00:20:00 --> 00:20:07
What's the good way to - I
mean, any decent machine comes
342
00:20:07 --> 00:20:10
with the FFT hardwired in.
343
00:20:10 --> 00:20:13
So you might say, OK
I'll just use it.
344
00:20:13 --> 00:20:16
And that seems totally
reasonable to me.
345
00:20:16 --> 00:20:24
But you might like to
see just on one board,
346
00:20:24 --> 00:20:26
what's the key idea?
347
00:20:26 --> 00:20:30
What's the little bit of
algebra that makes it work?
348
00:20:30 --> 00:20:35
So I'll just have one board
here for the FFT and a
349
00:20:35 --> 00:20:38
little bit of algebra.
350
00:20:38 --> 00:20:46
Simple, simple but once it hit
the world, well, computer
351
00:20:46 --> 00:20:50
scientists just love the
recursion that comes in there.
352
00:20:50 --> 00:20:58
So they look for that in every
possible other algorithm, now.
353
00:20:58 --> 00:21:01
OK, let me see that point.
354
00:21:01 --> 00:21:03
So here's the main point.
355
00:21:03 --> 00:21:12
That if I want to take the
multiply by F of size 1,024,
356
00:21:12 --> 00:21:19
the fast Fourier transform
connects that full matrix
357
00:21:19 --> 00:21:21
to a half-full matrix.
358
00:21:21 --> 00:21:24
It connects that to the
half-full matrix that
359
00:21:24 --> 00:21:30
takes the half-size
transforms separately.
360
00:21:30 --> 00:21:36
So it's half-full because
of these zeroes.
361
00:21:36 --> 00:21:37
That's the point.
362
00:21:37 --> 00:21:44
That the 1,024 matrix is
connected to the 512 matrix.
363
00:21:44 --> 00:21:47
And what's underlying that?
364
00:21:47 --> 00:21:56
The 1,024 matrix is full
of e^(2pi*i), the w,
365
00:21:56 --> 00:21:59
over a 1,024, right?
366
00:21:59 --> 00:22:01
That's the w for this guy.
367
00:22:01 --> 00:22:07
And then the w for this
guy, for both of these,
368
00:22:07 --> 00:22:14
is e^(2pi*i) over 512.
369
00:22:14 --> 00:22:16
So if there's a connection
between that matrix and this
370
00:22:16 --> 00:22:19
matrix, there'd better be a
connection between that
371
00:22:19 --> 00:22:21
number and that number.
372
00:22:21 --> 00:22:24
Because this is the number that
fills this one, and this is
373
00:22:24 --> 00:22:26
the number that fills these.
374
00:22:26 --> 00:22:30
So what's the connection?
375
00:22:30 --> 00:22:32
It's just perfect, right?
376
00:22:32 --> 00:22:38
If I take this number, which is
one part of the whole circle,
377
00:22:38 --> 00:22:43
1/1,024, a fraction of the
whole circle, what do
378
00:22:43 --> 00:22:47
I do to get this guy?
379
00:22:47 --> 00:22:51
To get 1/512th of the
way around the circle?
380
00:22:51 --> 00:22:55
I square it.
381
00:22:55 --> 00:23:01
The square of this w is this w.
382
00:23:01 --> 00:23:07
Let me call this w_N,
and this one w_M.
383
00:23:07 --> 00:23:12
Maybe I'll use caps, yeah I'm
using cap N, this is my w.
384
00:23:12 --> 00:23:14
This is the one I want.
385
00:23:14 --> 00:23:21
The w_N, the N by N one, N is
1,024, and the point is,
386
00:23:21 --> 00:23:24
everybody saw that, when I
squared it I doubled the angle?
387
00:23:24 --> 00:23:31
When I doubled that angle the
two over that gave me 1/512.
388
00:23:31 --> 00:23:32
Fantastic.
389
00:23:32 --> 00:23:40
Of course, that doesn't make
this equal to that, but it
390
00:23:40 --> 00:23:44
suggests that there is
a close connection.
391
00:23:44 --> 00:23:49
So let me finish here the key
idea of the Fourier transform
392
00:23:49 --> 00:23:53
in block matrix form.
393
00:23:53 --> 00:23:55
What's the key idea?
394
00:23:55 --> 00:23:59
So instead of doing the big
transform, the full size, I'm
395
00:23:59 --> 00:24:01
going to do two half-sizes.
396
00:24:01 --> 00:24:08
But what am I going
to apply those to?
397
00:24:08 --> 00:24:10
Here's the trick.
398
00:24:10 --> 00:24:15
These two separate guys
apply to the odd-numbered
399
00:24:15 --> 00:24:16
coefficients.
400
00:24:16 --> 00:24:18
The odd-numbered component.
401
00:24:18 --> 00:24:19
And the even.
402
00:24:19 --> 00:24:25
So I have to first do a
little permutation, and
403
00:24:25 --> 00:24:27
even comes first, always.
404
00:24:27 --> 00:24:32
Even means zero, two,
four, up to a 1,022.
405
00:24:32 --> 00:24:38
So this is zero, two,
zero, up to 1,022.
406
00:24:38 --> 00:24:40
And then come all the odd guys.
407
00:24:40 --> 00:24:44
One up to 1,023.
408
00:24:44 --> 00:24:48
So this is a permutation,
a simple permutation.
409
00:24:48 --> 00:24:52
Just take your 1,024
numbers, pick out the even
410
00:24:52 --> 00:24:55
ones, put them on top.
411
00:24:55 --> 00:24:56
Right?
412
00:24:56 --> 00:24:58
In other words, put them
on top where 512 is
413
00:24:58 --> 00:25:00
going to act on it.
414
00:25:00 --> 00:25:03
Put the odd ones at the
bottom, the last 512, that
415
00:25:03 --> 00:25:05
guy will act on that.
416
00:25:05 --> 00:25:09
So there's 512 numbers
there, with the even
417
00:25:09 --> 00:25:10
coefficients, this acts.
418
00:25:10 --> 00:25:14
OK, now we've got two
half-size transforms.
419
00:25:14 --> 00:25:21
Because we're applying this to
the y, to a to a typical y.
420
00:25:21 --> 00:25:22
OK.
421
00:25:22 --> 00:25:25
But I've just written F
without a y so I don't
422
00:25:25 --> 00:25:27
really need a y here.
423
00:25:27 --> 00:25:30
This is a matrix identity.
424
00:25:30 --> 00:25:33
It's a matrix identity that's
got a bunch of zeroes there.
425
00:25:33 --> 00:25:36
Of course, that matrix
is full of zeroes.
426
00:25:36 --> 00:25:41
I mean, this is instant speed
to do that permutation.
427
00:25:41 --> 00:25:45
Grab the evens, put them
in front of the odds.
428
00:25:45 --> 00:25:49
OK, so now I've got two
half-sizes, but then I have
429
00:25:49 --> 00:25:53
to put them back together to
get the full-size matrix.
430
00:25:53 --> 00:25:55
And that is also a matrix.
431
00:25:55 --> 00:25:59
Turns out to be a diagonal
there, and a minus the
432
00:25:59 --> 00:26:05
diagonal goes there.
433
00:26:05 --> 00:26:09
So actually, that looks
great too, right?
434
00:26:09 --> 00:26:11
Full of zeroes, the identity.
435
00:26:11 --> 00:26:13
No multiplications whatever.
436
00:26:13 --> 00:26:16
Well, these are the only
multiplications, because
437
00:26:16 --> 00:26:19
sometimes they're called
twiddle factors, give
438
00:26:19 --> 00:26:21
it a fancy name.
439
00:26:21 --> 00:26:25
Official sounding name,
twiddle factors.
440
00:26:25 --> 00:26:29
OK, so that diagonal
matrix D, what's that?
441
00:26:29 --> 00:26:34
That diagonal matrix D happens
to be just the powers
442
00:26:34 --> 00:26:37
of w, sit along D.
443
00:26:37 --> 00:26:44
1 w up to w to the, this is
W_N, we're talking, the big
444
00:26:44 --> 00:26:48
W, and it's only half-size
so it goes up to M-1.
445
00:26:50 --> 00:26:53
Half, M is half of N.
446
00:26:53 --> 00:26:55
Everybody's got that, right?
447
00:26:55 --> 00:26:57
M is half of N here.
448
00:26:57 --> 00:27:01
I'll get that written
on the board.
449
00:27:01 --> 00:27:07
M is 512, N is 1,024, here
we have the powers of
450
00:27:07 --> 00:27:10
this guy up to 511.
451
00:27:10 --> 00:27:13
The total size being
512 because that's a
452
00:27:13 --> 00:27:17
512 by 512 matrix.
453
00:27:17 --> 00:27:21
I guess I can remember
somewhere, being at a
454
00:27:21 --> 00:27:24
conference, this was
probably soon after the
455
00:27:24 --> 00:27:29
FFT became sort of famous.
456
00:27:29 --> 00:27:34
And then somebody who was just
presenting the idea and as
457
00:27:34 --> 00:27:38
soon as it was presented
that way, I was happy.
458
00:27:38 --> 00:27:39
I guess.
459
00:27:39 --> 00:27:43
I thought OK, there
you see the idea.
460
00:27:43 --> 00:27:46
Permutation, reorder
the even-odd.
461
00:27:46 --> 00:27:50
Two half-size transforms,
put them back together.
462
00:27:50 --> 00:27:52
And what's happened here?
463
00:27:52 --> 00:27:59
The work of this matrix,
multiplying by this matrix,
464
00:27:59 --> 00:28:04
which would be 1,024 squared is
now practically cut in half.
465
00:28:04 --> 00:28:06
Because this is nothing.
466
00:28:06 --> 00:28:10
And we have just this diagonal
multiplication to do, and of
467
00:28:10 --> 00:28:12
course this is the same as
that, just with minus signs.
468
00:28:12 --> 00:28:16
So the total multiplications we
have to do, the total number of
469
00:28:16 --> 00:28:22
twiddle factors, is just 512,
twelve and then we're golden.
470
00:28:22 --> 00:28:27
So we've got half the work
plus 512, 512 operations.
471
00:28:27 --> 00:28:30
That's pretty good.
472
00:28:30 --> 00:28:33
And of course it gets better.
473
00:28:33 --> 00:28:35
How?
474
00:28:35 --> 00:28:38
Once you have the idea of
getting down to 512, what
475
00:28:38 --> 00:28:39
are you going to do now?
476
00:28:39 --> 00:28:46
This is the computer
scientist's favorite idea.
477
00:28:46 --> 00:28:47
Do it again.
478
00:28:47 --> 00:28:48
That's what it comes to.
479
00:28:48 --> 00:28:53
Whatever worked for 1,024 to
get to 512 is going to work.
480
00:28:53 --> 00:28:58
So now I'll split this up
into, well, each 512, so
481
00:28:58 --> 00:29:01
now I have to do, yeah.
482
00:29:01 --> 00:29:07
Let me write F_512 will
be, it's now smaller.
483
00:29:07 --> 00:29:17
An I, an I, and a D and a minus
D for the 512 size of F_256,
484
00:29:17 --> 00:29:25
256 and then the permutation,
the odd-even permutation P.
485
00:29:25 --> 00:29:30
So we're doing this idea
in there, and in there.
486
00:29:30 --> 00:29:38
So it's just recursive.
487
00:29:38 --> 00:29:39
Recursive.
488
00:29:39 --> 00:29:42
And now, if we go all the
way, so you see why I keep
489
00:29:42 --> 00:29:46
taking powers of two.
490
00:29:46 --> 00:29:48
It's not natural to
have powers of two.
491
00:29:48 --> 00:29:49
Two or three.
492
00:29:49 --> 00:29:51
Three is also good.
493
00:29:51 --> 00:29:55
I mean, all this gets so
optimized that powers of two
494
00:29:55 --> 00:29:57
or three are pretty good.
495
00:29:57 --> 00:30:00
And you just use the same idea.
496
00:30:00 --> 00:30:06
There'd be a similar idea here
for, if I was doing instead
497
00:30:06 --> 00:30:12
of odd-even, even-odd I was
doing maybe three groups.
498
00:30:12 --> 00:30:19
But stick with
two, that's fine.
499
00:30:19 --> 00:30:25
Then you might ask, if you were
a worrier I guess you might ask
500
00:30:25 --> 00:30:30
what if it's not
a power of two.
501
00:30:30 --> 00:30:32
I think you just add in zeroes.
502
00:30:32 --> 00:30:41
Just pad it out to be
the next power of two.
503
00:30:41 --> 00:30:43
Nothing difficult there.
504
00:30:43 --> 00:30:44
I think that's right.
505
00:30:44 --> 00:30:49
Hope that's right.
506
00:30:49 --> 00:30:55
And once this idea came out, of
course, people started looking.
507
00:30:55 --> 00:31:00
What if the number
here was prime?
508
00:31:00 --> 00:31:06
And found another neat bit
of algebra that worked
509
00:31:06 --> 00:31:08
OK for prime numbers.
510
00:31:08 --> 00:31:11
Using a little bit
of number theory.
511
00:31:11 --> 00:31:16
But the ultimate
winner was this one.
512
00:31:16 --> 00:31:25
So maybe I'll just refer you to
those pages in the book, which
513
00:31:25 --> 00:31:29
were, you'll spot this
matrix equality.
514
00:31:29 --> 00:31:33
And then next to it is the
algebra that you have
515
00:31:33 --> 00:31:37
to do to check it.
516
00:31:37 --> 00:31:42
I can just say, because I want
you to look to the right spot
517
00:31:42 --> 00:31:47
there, maybe I'll take out the
great - this is sometimes
518
00:31:47 --> 00:31:53
called after Parseval,
or some other person.
519
00:31:53 --> 00:31:56
Yeah, the algebra.
520
00:31:56 --> 00:32:02
Let me start the algebra
that made this thing work.
521
00:32:02 --> 00:32:08
We want the sum, when we
multiply Fourier F times
522
00:32:08 --> 00:32:14
something, we want the
sum of w^jk, right?
523
00:32:14 --> 00:32:20
That's the coefficient, that's
the entry of F, times c_k.
524
00:32:23 --> 00:32:25
Sum from k=0 to N-1.
525
00:32:26 --> 00:32:30
To 1,023.
526
00:32:30 --> 00:32:40
That's the y_j that we're
trying to compute.
527
00:32:40 --> 00:32:45
We're computing a 1,024 y's
from 1,024 c's by adding
528
00:32:45 --> 00:32:48
up the Fourier series
when we multiply by F.
529
00:32:48 --> 00:33:00
This is F, this is the equation
y=Fc written with subscripts.
530
00:33:00 --> 00:33:03
So this is what the matrices
are doing, and now where
531
00:33:03 --> 00:33:07
do I find that 512 thing.
532
00:33:07 --> 00:33:12
How do I get M into the
picture, remembering that the
533
00:33:12 --> 00:33:16
w_N squared was w_M, right?
534
00:33:16 --> 00:33:20
That's what we saw.
535
00:33:20 --> 00:33:24
This is the big number, this
is half of it, so this is a
536
00:33:24 --> 00:33:26
little bit of the part
around the circle.
537
00:33:26 --> 00:33:29
When I go twice as far I
get to the other one.
538
00:33:29 --> 00:33:34
So that's the thing
that we've got to use.
539
00:33:34 --> 00:33:37
Everything is going
to depend on that.
540
00:33:37 --> 00:33:41
So this was w_N, of course.
541
00:33:41 --> 00:33:44
This is the N by N transpose
So now comes what,
542
00:33:44 --> 00:33:47
what's the key idea?
543
00:33:47 --> 00:33:49
The key idea is split
into even and odd.
544
00:33:49 --> 00:33:51
And then use this.
545
00:33:51 --> 00:33:55
So split into the even
ones and the odd ones.
546
00:33:55 --> 00:34:01
So I write this as two separate
sums, a sum for the even ones
547
00:34:01 --> 00:34:09
w_N, now, so now the even
c_k, so I'm going to
548
00:34:09 --> 00:34:12
multiply by c_2k.
549
00:34:14 --> 00:34:17
These are the ones with even,
and the sum is only going to
550
00:34:17 --> 00:34:23
go from zero to M-1, right?
551
00:34:23 --> 00:34:25
This is only half of the terms.
552
00:34:25 --> 00:34:31
And then look on the other
half, plus the same sum of,
553
00:34:31 --> 00:34:32
but I didn't finish here.
554
00:34:32 --> 00:34:33
Let me finish.
555
00:34:33 --> 00:34:36
So I'm just picking out,
I'm taking, instead
556
00:34:36 --> 00:34:37
of k I'm doing 2k.
557
00:34:38 --> 00:34:43
So I have j times 2k here.
558
00:34:43 --> 00:34:49
And now these will be
the odd ones. c_(2k+1),
559
00:34:49 --> 00:34:49
and omega_N^j(2k+1).
560
00:34:49 --> 00:35:00
561
00:35:00 --> 00:35:03
It's a lot to ask you.
562
00:35:03 --> 00:35:11
To focus on this
bit of algebra.
563
00:35:11 --> 00:35:15
I hope you're going to go
away feeling, well it's
564
00:35:15 --> 00:35:17
pretty darn simple.
565
00:35:17 --> 00:35:20
I mean there'll be a lot
of indices, and if I push
566
00:35:20 --> 00:35:22
it all the way through
there'll be a few more.
567
00:35:22 --> 00:35:25
But the point is, it's
pretty darn simple.
568
00:35:25 --> 00:35:26
For example, this term.
569
00:35:26 --> 00:35:29
What have I got there?
570
00:35:29 --> 00:35:33
Look at that term,
that's beautiful.
571
00:35:33 --> 00:35:36
That's w_N squared.
572
00:35:36 --> 00:35:40
What is w_N squared?
573
00:35:40 --> 00:35:41
It's w_N.
574
00:35:42 --> 00:35:45
So instead of w_N squared,
I'm going to replace
575
00:35:45 --> 00:35:48
that w_N squared by w_M.
576
00:35:49 --> 00:35:52
And the sum goes from
zero to M-1, and what
577
00:35:52 --> 00:35:56
does that represent?
578
00:35:56 --> 00:35:59
That represents the F_512
multiplication, the
579
00:35:59 --> 00:36:00
half-size transform.
580
00:36:00 --> 00:36:03
It goes halfway, it
operates on the even
581
00:36:03 --> 00:36:08
ones, and it uses the M.
582
00:36:08 --> 00:36:13
It's just perfectly, so this is
nothing but the Fourier matrix,
583
00:36:13 --> 00:36:21
the M by M Fourier matrix,
acting on the even c's.
584
00:36:21 --> 00:36:23
That's what that is.
585
00:36:23 --> 00:36:26
And that's why we get
these identities.
586
00:36:26 --> 00:36:29
That's why we get these
identities, because they're
587
00:36:29 --> 00:36:37
acting on the even - the
top half is the even guy.
588
00:36:37 --> 00:36:40
OK, this is almost as good.
589
00:36:40 --> 00:36:42
This is the odd c's.
590
00:36:42 --> 00:36:47
Here I have, now do
I have w_M here?
591
00:36:47 --> 00:36:49
I've got to have w_M.
592
00:36:49 --> 00:36:52
593
00:36:52 --> 00:36:54
Well, you can see.
594
00:36:54 --> 00:36:56
It's not quite coming
out right, and that's
595
00:36:56 --> 00:36:58
the twiddle factor.
596
00:36:58 --> 00:37:05
I have to take out a w_N to the
power j to make things good.
597
00:37:05 --> 00:37:09
And when I take out that w_N to
the power j, that's what goes
598
00:37:09 --> 00:37:14
in the D, in the
diagonal matrix.
599
00:37:14 --> 00:37:16
Yeah.
600
00:37:16 --> 00:37:18
I won't go more than that.
601
00:37:18 --> 00:37:24
You couldn't, there's no reason
to do everything here on the
602
00:37:24 --> 00:37:27
board when the main
point is there.
603
00:37:27 --> 00:37:30
And then the point was
recursion and then, oh, let
604
00:37:30 --> 00:37:33
me complete the recursion.
605
00:37:33 --> 00:37:38
So I recurse down to
256, then 128, then 64.
606
00:37:38 --> 00:37:42
And what do I get in the end?
607
00:37:42 --> 00:37:45
What do I get altogether,
once I've got all the
608
00:37:45 --> 00:37:48
way down to size two?
609
00:37:48 --> 00:37:58
I have a whole lot of these
factors down in the middle, the
610
00:37:58 --> 00:38:01
part that used to be hard is
now down to F_2 or
611
00:38:01 --> 00:38:03
F_1 or something.
612
00:38:03 --> 00:38:07
So let's say F_1, one by
one, just the identity.
613
00:38:07 --> 00:38:09
So I go all the way.
614
00:38:09 --> 00:38:14
Ten steps from 1024,
512, 256, every time I
615
00:38:14 --> 00:38:16
get twiddle factors.
616
00:38:16 --> 00:38:19
Every time I get P's.
617
00:38:19 --> 00:38:25
A lot of P's, but the F_512,
what used to be the hard part,
618
00:38:25 --> 00:38:29
is gone to the easy part.
619
00:38:29 --> 00:38:31
And then what do I have?
620
00:38:31 --> 00:38:33
I've got just a
permutation there.
621
00:38:33 --> 00:38:38
And this is the actual work.
622
00:38:38 --> 00:38:45
This is the only work left,
is the matrices like this,
623
00:38:45 --> 00:38:49
for different sizes.
624
00:38:49 --> 00:38:50
And I have to do those.
625
00:38:50 --> 00:38:52
I have to do all those
twiddle factors.
626
00:38:52 --> 00:38:58
So how many matrices
are there, there?
627
00:38:58 --> 00:39:00
It's the log, right?
628
00:39:00 --> 00:39:02
Every time I divided by two.
629
00:39:02 --> 00:39:07
So if I started at 1,024, I do
ten times, I have ten of those
630
00:39:07 --> 00:39:12
matrices and have me down to
N=1. - So I've got ten of
631
00:39:12 --> 00:39:19
these, and each one takes 1,024
or maybe only 1/2 of that.
632
00:39:19 --> 00:39:22
Actually, only half because
this is a copy of that.
633
00:39:22 --> 00:39:27
I think the final count, and
can I just put in here, this is
634
00:39:27 --> 00:39:35
the great number, is each of
these took n multiplications.
635
00:39:35 --> 00:39:38
But there were only log to
the base two - oh, no.
636
00:39:38 --> 00:39:41
Each of them took half of N
multiplications, because the
637
00:39:41 --> 00:39:46
D and the minus D are just,
I don't have to repeat.
638
00:39:46 --> 00:39:50
So half N for each factor and
the number of factors is
639
00:39:50 --> 00:39:53
log to the base two of N.
640
00:39:53 --> 00:39:55
Ten, for 1,024.
641
00:39:55 --> 00:40:00
So that's the magic of the FFT.
642
00:40:00 --> 00:40:02
OK.
643
00:40:02 --> 00:40:06
It's almost all on one board,
one and a half boards.
644
00:40:06 --> 00:40:21
To tell you the key point, odds
and evens, recursion, twiddle
645
00:40:21 --> 00:40:25
factors, getting down to the
point where you only have
646
00:40:25 --> 00:40:30
twiddle factors left and then
those multiplications
647
00:40:30 --> 00:40:33
are only N log N.
648
00:40:33 --> 00:40:37
Good?
649
00:40:37 --> 00:40:38
Yes.
650
00:40:38 --> 00:40:40
Right, OK.
651
00:40:40 --> 00:40:47
Now. that's discrete transform.
652
00:40:47 --> 00:40:52
The theory behind it and
the fantastic algorithm
653
00:40:52 --> 00:40:59
that executes it.
654
00:40:59 --> 00:41:00
Are you ready for
a convolution?
655
00:41:00 --> 00:41:04
Can we start on a topic that's
really quite nice, and
656
00:41:04 --> 00:41:10
then Friday will be the
focus on convolution.
657
00:41:10 --> 00:41:14
Friday will certainly be
all convolution day.
658
00:41:14 --> 00:41:18
But maybe it's not a
bad idea to see now,
659
00:41:18 --> 00:41:21
what's the question.
660
00:41:21 --> 00:41:25
Let me ask that question.
661
00:41:25 --> 00:41:27
OK, convolution.
662
00:41:27 --> 00:41:32
So we're into the next
section of the book,
663
00:41:32 --> 00:41:42
Section 4.4, it must be.
664
00:41:42 --> 00:41:46
And let me do it first
for a Fourier series.
665
00:41:46 --> 00:41:50
I have convolution of series,
convolution of discrete.
666
00:41:50 --> 00:41:57
Convolution of integrals, but
we haven't got there yet.
667
00:41:57 --> 00:42:02
So I'll do this
one, this series.
668
00:42:02 --> 00:42:06
So let me start with a couple
of series. f(x) is the
669
00:42:06 --> 00:42:07
sum of c_k*e^(ikx).
670
00:42:07 --> 00:42:11
671
00:42:11 --> 00:42:18
g(x) is the sum of some
other coefficients.
672
00:42:18 --> 00:42:21
And I'm going to ask
you a simple question.
673
00:42:21 --> 00:42:27
What are the Fourier
coefficients of f times g?
674
00:42:27 --> 00:42:39
If I multiply those
functions, equals something.
675
00:42:39 --> 00:42:43
And let me call those
coefficients, h maybe.
676
00:42:43 --> 00:42:52
h_k*e^(ikx), and my question
is what are the coefficients
677
00:42:52 --> 00:42:59
h_k of f times g?
678
00:42:59 --> 00:43:10
That's the question that
convolution answers.
679
00:43:10 --> 00:43:14
Actually, both this series
and the discrete series
680
00:43:14 --> 00:43:17
are highly interesting.
681
00:43:17 --> 00:43:20
Highly interesting.
682
00:43:20 --> 00:43:25
So here I wrote it
for this series.
683
00:43:25 --> 00:43:34
If I write it for the discrete
ones, you'll see, so let me
684
00:43:34 --> 00:43:36
do it over here for
the discrete one.
685
00:43:36 --> 00:43:39
Because I can write it out
for the discrete ones.
686
00:43:39 --> 00:43:51
My y's are c_0+c_1 e - no, w
I have this nice notation w,
687
00:43:51 --> 00:43:58
plus c_N-1*w^(N-1), right?
688
00:43:58 --> 00:44:03
That's the - ooh.
689
00:44:03 --> 00:44:05
What's that?
690
00:44:05 --> 00:44:08
I haven't got that right.
691
00:44:08 --> 00:44:12
Yes, what do I want now?
692
00:44:12 --> 00:44:15
I need, yep.
693
00:44:15 --> 00:44:16
Sorry, I'm looking.
694
00:44:16 --> 00:44:20
Really, I'm looking at y_j,
the j'th component of y.
695
00:44:20 --> 00:44:24
So I need a w^j, w^j(N-1).
696
00:44:24 --> 00:44:26
697
00:44:26 --> 00:44:32
Yeah, OK, let me - OK.
698
00:44:32 --> 00:44:33
Alright.
699
00:44:33 --> 00:44:36
And so that's my f.
700
00:44:36 --> 00:44:44
I'll come back to that,
let me stay with this.
701
00:44:44 --> 00:44:49
I'll stay with this to make the
main point, and then Friday
702
00:44:49 --> 00:44:53
we'll see it in a neat way
for the discrete one.
703
00:44:53 --> 00:44:57
So I'm coming back to this.
f has its Fourier series
704
00:44:57 --> 00:44:59
g has its Fourier series.
705
00:44:59 --> 00:45:02
I multiply.
706
00:45:02 --> 00:45:10
What happens when I
multiply this times this?
707
00:45:10 --> 00:45:12
I'm not going to integrate.
708
00:45:12 --> 00:45:15
I mean, when I do this
multiplication, I'm going
709
00:45:15 --> 00:45:18
to get a mass of terms.
710
00:45:18 --> 00:45:21
A real lot of terms.
711
00:45:21 --> 00:45:23
And I'm not going to
integrate them away.
712
00:45:23 --> 00:45:25
So they're all there.
713
00:45:25 --> 00:45:27
So what am I asking?
714
00:45:27 --> 00:45:31
I'm asking to pick out all
the terms that have the
715
00:45:31 --> 00:45:36
same exponential with them.
716
00:45:36 --> 00:45:38
Like, what's h_0?
717
00:45:38 --> 00:45:42
Yes, tell me what h_0 is?
718
00:45:42 --> 00:45:45
If you can pick out h_0
here, you'll get the
719
00:45:45 --> 00:45:50
idea of convolution.
720
00:45:50 --> 00:45:55
What's the constant term if I
multiply this mess times this
721
00:45:55 --> 00:46:04
mess, and I look for the
constant term, h_0, where do I
722
00:46:04 --> 00:46:07
get the constant terms when
I multiply that by that?
723
00:46:07 --> 00:46:08
Just think about that.
724
00:46:08 --> 00:46:10
Where do I get a
constant, without any
725
00:46:10 --> 00:46:13
k, without an e^(ikx)?
726
00:46:13 --> 00:46:16
If I multiply that by that.
727
00:46:16 --> 00:46:21
Well tell me one place I get
something. c_0 times? d_0.
728
00:46:21 --> 00:46:25
Good.
729
00:46:25 --> 00:46:28
Is that the end of the story?
730
00:46:28 --> 00:46:29
No.
731
00:46:29 --> 00:46:32
If you thought that multiplying
the functions, I just
732
00:46:32 --> 00:46:34
multiplied the Fourier
coefficients, the
733
00:46:34 --> 00:46:36
first point is no.
734
00:46:36 --> 00:46:38
There's more stuff.
735
00:46:38 --> 00:46:41
Where else do I get a
constant out of this?
736
00:46:41 --> 00:46:45
Just look at it, do that
multiplication and ask yourself
737
00:46:45 --> 00:46:48
where's the constant.
738
00:46:48 --> 00:46:49
Another one, yep.
739
00:46:49 --> 00:46:54
You were going to say
it is? c_1 times d_-1.
740
00:46:54 --> 00:46:55
Right.
741
00:46:55 --> 00:46:58
Right. c_1 times d_-1.
742
00:46:59 --> 00:47:04
And tell me all of them,
now. c_2 times d_-2.
743
00:47:05 --> 00:47:07
And what about c_-1?
744
00:47:09 --> 00:47:09
There's a c_-1.
745
00:47:09 --> 00:47:12
746
00:47:12 --> 00:47:14
It multiplies d_1.
747
00:47:16 --> 00:47:19
And onwards.
748
00:47:19 --> 00:47:26
So the coefficient comes
from, now how could
749
00:47:26 --> 00:47:29
you describe that?
750
00:47:29 --> 00:47:35
I guess I'll describe it as,
I'll need a sum to multiply.
751
00:47:35 --> 00:47:39
This will be the sum
of c_k times d_1.
752
00:47:39 --> 00:47:42
753
00:47:42 --> 00:47:45
Minus k, right?
754
00:47:45 --> 00:47:47
That's what you told me.
755
00:47:47 --> 00:47:52
Piece at the start, and that's
the pattern that keeps going.
756
00:47:52 --> 00:47:56
OK, that's h_0, the sum
of c_k times d_-k.
757
00:47:58 --> 00:48:05
Now, we have just time
to do the next one.
758
00:48:05 --> 00:48:07
We've got time but not
space, where the heck
759
00:48:07 --> 00:48:09
am I going to put it?
760
00:48:09 --> 00:48:14
I want to do h_k, I guess.
761
00:48:14 --> 00:48:18
Or I better use a different
letter h. l, let me use the
762
00:48:18 --> 00:48:22
letter h_l, and God
there's no space.
763
00:48:22 --> 00:48:31
Alright, so can I - yes. h_l.
764
00:48:31 --> 00:48:32
OK.
765
00:48:32 --> 00:48:36
So this was h_0, let me keep
things sort of looking
766
00:48:36 --> 00:48:37
right for the moment.
767
00:48:37 --> 00:48:39
OK, now you're
going to fix h_l.
768
00:48:39 --> 00:48:44
So what does c_0 multiply
if I'm looking for h_l,
769
00:48:44 --> 00:48:46
I'm looking for the
coefficient of e^(ilx).
770
00:48:47 --> 00:48:51
So ask yourself how do I
get e^(ilx) when that
771
00:48:51 --> 00:48:54
multiplies that?
772
00:48:54 --> 00:48:57
When that multiplies that, and
I'm looking for an e^(ilx), I
773
00:48:57 --> 00:49:02
get one when c_0 all
multiplies what?
774
00:49:02 --> 00:49:05
This is it. dl, right.
775
00:49:05 --> 00:49:07
And what about for c_1?
776
00:49:07 --> 00:49:09
777
00:49:09 --> 00:49:11
Think of here, I
have a c_1*e^(ilx).
778
00:49:11 --> 00:49:14
779
00:49:14 --> 00:49:17
What does it multiply
down here to get the
780
00:49:17 --> 00:49:20
exponential to be l? ilx?
781
00:49:22 --> 00:49:25
What doesn't multiply d_-1.
782
00:49:25 --> 00:49:30
It multiplies, c_1
multiplies? d_(l-1).
783
00:49:31 --> 00:49:37
Good, good. l-1, right. l-1,
and what are you noticing
784
00:49:37 --> 00:49:40
here? c minus, I'll
have to fill that in.
785
00:49:40 --> 00:49:45
But you're seeing
the pattern here?
786
00:49:45 --> 00:49:47
And what was the pattern here?
787
00:49:47 --> 00:49:51
Those numbers added
to this number.
788
00:49:51 --> 00:49:55
And now these numbers add to l
Those numbers add to l,
789
00:49:55 --> 00:50:03
whatever it is, the two indices
have to add to l, so that when
790
00:50:03 --> 00:50:06
I multiply the exponential
they'll add to e^(ilx).
791
00:50:08 --> 00:50:08
They'll multiply to e^(ilx).
792
00:50:08 --> 00:50:11
793
00:50:11 --> 00:50:14
So what goes there?
794
00:50:14 --> 00:50:16
It's probably l+1, right?
795
00:50:16 --> 00:50:22
So that l+1 combined with
minus one gives me the l.
796
00:50:22 --> 00:50:26
If you tell me what goes
there, I'm a happy person.
797
00:50:26 --> 00:50:27
Let's make it h_l.
798
00:50:27 --> 00:50:33
We're ready for the final
formula for convolutions.
799
00:50:33 --> 00:50:39
Big star.
800
00:50:39 --> 00:50:45
To find h_l, the coefficient of
e^(ilx), when you multiply that
801
00:50:45 --> 00:50:53
by that, you look at c_k, and
which d is going to show up in
802
00:50:53 --> 00:50:59
the e^(ilx) term? l-k,
is that what you said?
803
00:50:59 --> 00:51:02
I hope, yeah. l-k.
804
00:51:02 --> 00:51:04
Right, that's it.
805
00:51:04 --> 00:51:05
That's it.
806
00:51:05 --> 00:51:10
So we've got a lot of
computation here.
807
00:51:10 --> 00:51:17
But we've got the idea of
what, we've got a formula.
808
00:51:17 --> 00:51:20
And most of all we
have the magic rule.
809
00:51:20 --> 00:51:27
In convolutions, convolutions
are, things multiply
810
00:51:27 --> 00:51:29
but indices add.
811
00:51:29 --> 00:51:33
Things multiply, numbers
multiply, while
812
00:51:33 --> 00:51:34
their indices add.
813
00:51:34 --> 00:51:36
That's the key idea of
convolution that we'll
814
00:51:36 --> 00:51:40
see clearly and
completely on Friday.
815
00:51:40 --> 00:51:41
OK.