1
00:00:00 --> 00:00:01
2
00:00:01 --> 00:00:03
The following content is
provided under a Creative
3
00:00:03 --> 00:00:03
Commons license.
4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to
5
00:00:06 --> 00:00:10
offer high-quality educational
resources for free.
6
00:00:10 --> 00:00:12
To make a donation, or to view
additional materials from
7
00:00:12 --> 00:00:17
hundreds of MIT courses, visit
MIT OpenCourseWare
8
00:00:17 --> 00:00:20
at ocw.mit.edu.
9
00:00:20 --> 00:00:22
PROFESSOR STRANG: So,
thanks for coming today.
10
00:00:22 --> 00:00:30
This is a key lecture in the
application of Fourier,
11
00:00:30 --> 00:00:33
you could say.
12
00:00:33 --> 00:00:35
So convolution is the big word.
13
00:00:35 --> 00:00:40
And a major application of
convolution is filtering
14
00:00:40 --> 00:00:43
signal processing.
15
00:00:43 --> 00:00:46
So we'll develop
that application.
16
00:00:46 --> 00:00:48
But it's nothing
but convolution.
17
00:00:48 --> 00:00:53
So the key idea is these
convolution rules,
18
00:00:53 --> 00:00:55
where they come from.
19
00:00:55 --> 00:00:59
And what these new symbols,
that's the symbol for the
20
00:00:59 --> 00:01:03
convolution of two functions.
21
00:01:03 --> 00:01:05
They could be functions here.
22
00:01:05 --> 00:01:11
Here they're long vectors
of coefficients, and so
23
00:01:11 --> 00:01:14
these are the rules.
24
00:01:14 --> 00:01:16
So it's just a little
bit of algebra.
25
00:01:16 --> 00:01:21
But it just is so central
to all this subject.
26
00:01:21 --> 00:01:25
Signal processing is
certainly the most important
27
00:01:25 --> 00:01:27
little thing to know.
28
00:01:27 --> 00:01:29
That if I multiply two
functions, so that's where
29
00:01:29 --> 00:01:31
we started last time.
30
00:01:31 --> 00:01:36
If I have a function f with
Fourier coefficient c, and a
31
00:01:36 --> 00:01:44
function g with coefficients
d, then, oh, wrong.
32
00:01:44 --> 00:01:48
I convolve the
coefficients, right.
33
00:01:48 --> 00:01:53
So f has coefficient c,
g has coefficient d.
34
00:01:53 --> 00:01:57
So if I multiply the functions
I definitely do not just
35
00:01:57 --> 00:02:00
multiply each c times the d.
36
00:02:00 --> 00:02:03
I do this convolution
operation, which we
37
00:02:03 --> 00:02:04
have to remember.
38
00:02:04 --> 00:02:07
That's our main first
step is to remember
39
00:02:07 --> 00:02:09
what that was about.
40
00:02:09 --> 00:02:13
And then this is the
other direction.
41
00:02:13 --> 00:02:15
So we didn't see this before.
42
00:02:15 --> 00:02:20
That I mean, there's always
this fantastic symmetry
43
00:02:20 --> 00:02:26
between physical space
and frequency space.
44
00:02:26 --> 00:02:30
And convolution in one is
multiplication in the other.
45
00:02:30 --> 00:02:33
It's so easy to remember.
46
00:02:33 --> 00:02:36
If I multiply in one
space, I do a convolution
47
00:02:36 --> 00:02:38
in the other space.
48
00:02:38 --> 00:02:42
If I do a convolution
of functions, I do a
49
00:02:42 --> 00:02:45
multiplication of coefficients.
50
00:02:45 --> 00:02:49
So that's the rule to know.
51
00:02:49 --> 00:02:53
And now if we expand on
it, by sort of seeing
52
00:02:53 --> 00:02:54
again what it means.
53
00:02:54 --> 00:02:59
So let me do a quick repeat
of this first step to
54
00:02:59 --> 00:03:04
remember what this symbol,
convolution symbol, means.
55
00:03:04 --> 00:03:07
And then another
thing I have to do.
56
00:03:07 --> 00:03:11
Here, I'm talking about the
infinite case, functions
57
00:03:11 --> 00:03:15
and with a whole infinite
sequence of coefficients.
58
00:03:15 --> 00:03:20
I've got to do the
cyclic case, too.
59
00:03:20 --> 00:03:23
Which goes with the
discrete transform.
60
00:03:23 --> 00:03:26
OK, but let's start with
the infinite case as
61
00:03:26 --> 00:03:29
we did last time.
62
00:03:29 --> 00:03:31
Maybe, here's something.
63
00:03:31 --> 00:03:32
Here's a suggestion.
64
00:03:32 --> 00:03:34
We had f(x).
65
00:03:36 --> 00:03:39
We started with
f(x), as the sum.
66
00:03:39 --> 00:03:43
And remember to have nice
formulas, we're doing the
67
00:03:43 --> 00:03:45
complex version. c_k*e^(ikx).
68
00:03:47 --> 00:03:49
Let me suggest something.
69
00:03:49 --> 00:03:51
Let me write z for e^(ikx).
70
00:03:52 --> 00:03:56
So I'm going to just write
that at the sum of c_k*z^k.
71
00:03:57 --> 00:04:01
z to So z is e^(ix).
72
00:04:01 --> 00:04:05
73
00:04:05 --> 00:04:09
This is like a good
point to make anyway.
74
00:04:09 --> 00:04:14
When we have this e^(ikx), it's
natural to think of that as a
75
00:04:14 --> 00:04:18
complex number on
the unit circle.
76
00:04:18 --> 00:04:19
Right?
77
00:04:19 --> 00:04:21
That's always the message,
e to the i real is
78
00:04:21 --> 00:04:23
on the unit circle.
79
00:04:23 --> 00:04:25
Absolute value one.
80
00:04:25 --> 00:04:29
And this is great for
periodic function.
81
00:04:29 --> 00:04:33
Because if these functions have
period 2pi, and if we think of
82
00:04:33 --> 00:04:40
our function as being on the
circle, it obviously
83
00:04:40 --> 00:04:41
has period 2pi.
84
00:04:42 --> 00:04:44
I mean, the picture says, yes.
85
00:04:44 --> 00:04:47
If I go 2pi, I come back.
86
00:04:47 --> 00:04:53
So x is the angle, right? x is
the in this picture, a little
87
00:04:53 --> 00:04:59
bit unusual maybe, x there
would be the angle.
88
00:04:59 --> 00:05:03
And it's just a little
bit easier to write.
89
00:05:03 --> 00:05:06
And maybe it even has an
official name, the z transform.
90
00:05:06 --> 00:05:07
How do you like that?
91
00:05:07 --> 00:05:17
You learn a transform just, in
fourteen seconds. z transform.
92
00:05:17 --> 00:05:21
It'll make it easier to
multiply by g, so g is going
93
00:05:21 --> 00:05:23
to be the sum of d_k*z^k.
94
00:05:26 --> 00:05:30
So just think of these as
long, well I'm tempted to
95
00:05:30 --> 00:05:32
say, long polynomials.
96
00:05:32 --> 00:05:37
I mean, very long, because
they can be infinite series.
97
00:05:37 --> 00:05:40
Infinite negative powers
and positive powers.
98
00:05:40 --> 00:05:44
But just think of it as a
bunch of powers of z times
99
00:05:44 --> 00:05:46
a bunch of powers of z.
100
00:05:46 --> 00:05:51
And if you multiply a couple
of polynomials, you're
101
00:05:51 --> 00:05:54
doing convolution.
102
00:05:54 --> 00:05:56
I guess my message is, you've
been doing convolution
103
00:05:56 --> 00:05:59
since the second grade.
104
00:05:59 --> 00:06:01
That's the real message.
105
00:06:01 --> 00:06:04
Here, let me show you.
106
00:06:04 --> 00:06:06
This is convolution, too.
107
00:06:06 --> 00:06:14
Suppose I have to multiply
a 123 times 456?
108
00:06:14 --> 00:06:17
OK, so what do you do?
109
00:06:17 --> 00:06:22
Remember back, it's a long
way back but we can do this.
110
00:06:22 --> 00:06:25
One, two, three times
four, five, six.
111
00:06:25 --> 00:06:30
So I multiply the three, oh
there's a little point.
112
00:06:30 --> 00:06:33
Where that second-grade
teacher's going to panic.
113
00:06:33 --> 00:06:37
I'm going to write that as 18.
114
00:06:37 --> 00:06:41
And that's 15, and that's 12.
115
00:06:41 --> 00:06:43
Sorry about that, yeah.
116
00:06:43 --> 00:06:48
And 12, ten and eight, right?
117
00:06:48 --> 00:06:50
Four, or five and six.
118
00:06:50 --> 00:06:54
So you see the nine
multiplications that
119
00:06:54 --> 00:06:55
you have to do?
120
00:06:55 --> 00:06:58
Nine multiplications,
three times three.
121
00:06:58 --> 00:07:05
OK, right now imagine
those were, they could
122
00:07:05 --> 00:07:06
have been longer.
123
00:07:06 --> 00:07:13
But they were finite length
filters, we could say.
124
00:07:13 --> 00:07:17
And now, what does
convolution do?
125
00:07:17 --> 00:07:19
Just what you did
in multiplication.
126
00:07:19 --> 00:07:23
When I add 12, 10 and
6, what am I doing?
127
00:07:23 --> 00:07:26
I'm putting together the three
times the four, the two
128
00:07:26 --> 00:07:29
times the five, and
the one times the six.
129
00:07:29 --> 00:07:31
That's what convolution,
those are the things that
130
00:07:31 --> 00:07:33
convolution puts together.
131
00:07:33 --> 00:07:40
So we got an 18, a 27, 28.
132
00:07:40 --> 00:07:45
13, and four.
133
00:07:45 --> 00:07:49
So I guess I'm saying that,
alright, here's what
134
00:07:49 --> 00:07:51
I really want to say.
135
00:07:51 --> 00:07:57
I want to say that the
convolution of those, with
136
00:07:57 --> 00:08:04
these, four, five, six, is
this sequence here - oh.
137
00:08:04 --> 00:08:04
Yeah, that's right.
138
00:08:04 --> 00:08:09
Four, 13, 28.
139
00:08:09 --> 00:08:13
27, and 18.
140
00:08:13 --> 00:08:15
If you just look at that.
141
00:08:15 --> 00:08:17
Where did that 13 come from?
142
00:08:17 --> 00:08:21
Let's just remember, where
does that 13 come from?
143
00:08:21 --> 00:08:23
That came from, this was z^0.
144
00:08:25 --> 00:08:29
This 13 is 13z to the
first power, right?
145
00:08:29 --> 00:08:33
We're just checking
all the powers here.
146
00:08:33 --> 00:08:35
There's 13z to the first power.
147
00:08:35 --> 00:08:37
Where do we get a first power?
148
00:08:37 --> 00:08:40
We get a z^0 times 5z^1.
149
00:08:41 --> 00:08:42
So that's 5z^1.
150
00:08:43 --> 00:08:47
And we also have
2z^1, times 4z^0.
151
00:08:49 --> 00:08:50
Right?
152
00:08:50 --> 00:08:52
Two times four gave the eight.
153
00:08:52 --> 00:08:54
So the eight from there
and the five from
154
00:08:54 --> 00:08:56
there produce that 13.
155
00:08:56 --> 00:09:00
And that's just what you
did in multiplication.
156
00:09:00 --> 00:09:01
Right?
157
00:09:01 --> 00:09:06
So that's multiplication
of two series.
158
00:09:06 --> 00:09:08
That's not cyclic.
159
00:09:08 --> 00:09:11
This is definitely not yet
cyclic, and but we'll make
160
00:09:11 --> 00:09:13
it cyclic in a minute.
161
00:09:13 --> 00:09:16
This is the infinite
one, except that we
162
00:09:16 --> 00:09:20
had all zeroes beyond.
163
00:09:20 --> 00:09:24
OK, so that if you in
non-cyclic convolution like
164
00:09:24 --> 00:09:30
this, if I have length m
and length n, then I get
165
00:09:30 --> 00:09:32
length m+n, maybe m+n-1.
166
00:09:34 --> 00:09:37
OK, so that's convolution.
167
00:09:37 --> 00:09:40
Without carrying numbers.
168
00:09:40 --> 00:09:42
Without doing it right.
169
00:09:42 --> 00:09:46
OK, so and what does
that correspond to?
170
00:09:46 --> 00:09:48
Let me just, so you
see it every way.
171
00:09:48 --> 00:09:55
That corresponds to 1+2z+3z^2,
multiplying 4+5z+6z^2.
172
00:10:00 --> 00:10:02
And it gave, that's times.
173
00:10:02 --> 00:10:08
And it gave this
thing up to 18z^4.
174
00:10:10 --> 00:10:12
Just, exactly the
multiplication that
175
00:10:12 --> 00:10:14
you've always done.
176
00:10:14 --> 00:10:17
OK, so that's the idea.
177
00:10:17 --> 00:10:20
Over on that board I'm going
to put a formula for this
178
00:10:20 --> 00:10:22
convolution operation.
179
00:10:22 --> 00:10:27
But my point on this board
is, you've done it always.
180
00:10:27 --> 00:10:30
When you multiply a couple
of polynomials, you
181
00:10:30 --> 00:10:31
collect powers.
182
00:10:31 --> 00:10:34
And that's all convolution
is doing, collecting
183
00:10:34 --> 00:10:36
each power separately.
184
00:10:36 --> 00:10:38
OK, let's do it.
185
00:10:38 --> 00:10:47
So then f(x), g(x), is, when I
multiply that polynomial or
186
00:10:47 --> 00:10:53
series by that polynomial, I
get some polynomial in, with
187
00:10:53 --> 00:10:58
coefficients, oh I was
changed to l, just to have a
188
00:10:58 --> 00:11:00
different symbol there.
189
00:11:00 --> 00:11:03
And what was the formula for l?
190
00:11:03 --> 00:11:03
For h_l?
191
00:11:05 --> 00:11:08
What is the coefficient of z^l?
192
00:11:08 --> 00:11:10
193
00:11:10 --> 00:11:15
If I multiply that by that, do
you remember the story there?
194
00:11:15 --> 00:11:20
When I multiply that by
that and I looked for the
195
00:11:20 --> 00:11:22
terms that gave me z^l?
196
00:11:23 --> 00:11:26
OK, well that means that
this power times this
197
00:11:26 --> 00:11:27
power is going to be z^l.
198
00:11:29 --> 00:11:33
So that the index, do you
remember what happened?
199
00:11:33 --> 00:11:38
It was a lot of different, just
the way this h_l is here.
200
00:11:38 --> 00:11:41
Here's h_2 or something.
201
00:11:41 --> 00:11:43
I've got to do an addition.
202
00:11:43 --> 00:11:49
Because a bunch of c's come
in with different d's, and
203
00:11:49 --> 00:11:53
what's the deal then? c_k
comes in with which d?
204
00:11:53 --> 00:11:56
This is the magic number there.
205
00:11:56 --> 00:12:02
What's the subscript that if
I look at the coefficient of
206
00:12:02 --> 00:12:07
z^l, I look at each of these.
207
00:12:07 --> 00:12:11
And then I pick out the one of
these that will give me a z^l.
208
00:12:13 --> 00:12:14
And which one is it?
209
00:12:14 --> 00:12:14
d_(l-k).
210
00:12:14 --> 00:12:18
211
00:12:18 --> 00:12:23
It's that magic quantity that
the eye spots perfectly. k and
212
00:12:23 --> 00:12:32
l-k, adding to l simply because
z to the k is z^(l-k),
213
00:12:32 --> 00:12:33
multiplies to z^l.
214
00:12:35 --> 00:12:36
Same thing.
215
00:12:36 --> 00:12:37
Right?
216
00:12:37 --> 00:12:43
OK, this is, now I'll
use that notation.
217
00:12:43 --> 00:12:49
This is c convolved with the
d. c convolved with d is h.
218
00:12:49 --> 00:12:50
c convolved with d is h.
219
00:12:50 --> 00:12:54
So this is the l'th
component. c convolved
220
00:12:54 --> 00:12:55
with d is my symbol.
221
00:12:55 --> 00:12:58
This h is the convolution.
222
00:12:58 --> 00:13:01
And that's the
convolution rule.
223
00:13:01 --> 00:13:05
It's just whatever operation
you have to do to get
224
00:13:05 --> 00:13:06
the right answer.
225
00:13:06 --> 00:13:11
The right answer
when you multiply.
226
00:13:11 --> 00:13:13
So that one.
227
00:13:13 --> 00:13:16
Ready for the discrete case?
228
00:13:16 --> 00:13:18
The finite case?
229
00:13:18 --> 00:13:25
The case when, you have
power, when z becomes w.
230
00:13:25 --> 00:13:27
The discrete case.
231
00:13:27 --> 00:13:28
The cyclic case, sorry.
232
00:13:28 --> 00:13:33
Maybe emphasize the cyclic
case, meaning it circles
233
00:13:33 --> 00:13:38
around, is the case when z
becomes this very special
234
00:13:38 --> 00:13:42
z, on the unit circle
that we know as w.
235
00:13:42 --> 00:13:48
OK, and we have to say, and of
course it's w_N, I have to tell
236
00:13:48 --> 00:13:54
you in the cyclic case
how long the cycle is.
237
00:13:54 --> 00:13:57
So this would be a case.
238
00:13:57 --> 00:14:02
Watch what you do here
to make this cyclic.
239
00:14:02 --> 00:14:09
OK, I have three inputs,
so N is three here.
240
00:14:09 --> 00:14:13
Now I'm going to do the cyclic.
241
00:14:13 --> 00:14:16
So instead of z's, I
should be putting w's.
242
00:14:16 --> 00:14:18
I will.
243
00:14:18 --> 00:14:23
Just to emphasize, it's good to
think of it with the w there,
244
00:14:23 --> 00:14:27
because the w has this special
property that's critical to
245
00:14:27 --> 00:14:32
everything, OK so now I'm going
to, I think of this is as
246
00:14:32 --> 00:14:32
1w^0, 2w^1, and 3w^2.
247
00:14:32 --> 00:14:38
248
00:14:38 --> 00:14:40
I'm thinking of the
same multiplication
249
00:14:40 --> 00:14:45
here, but now w's.
250
00:14:45 --> 00:14:51
OK, so I'll end up with 18w^4,
and four was the constant
251
00:14:51 --> 00:14:55
and 3w's and so on.
252
00:14:55 --> 00:14:59
All these numbers.
253
00:14:59 --> 00:15:01
What's the difference?
254
00:15:01 --> 00:15:04
Ready for the key point?
255
00:15:04 --> 00:15:07
Now, what's happened
in this cyclic case?
256
00:15:07 --> 00:15:10
Well, the difference
is what is w^4?
257
00:15:12 --> 00:15:16
If we're in the cyclic case, N
is three now, our guys have
258
00:15:16 --> 00:15:24
length three, our circle is w
now is 1/3 of the way around.
259
00:15:24 --> 00:15:27
So that's my w,
here's my w squared.
260
00:15:27 --> 00:15:29
Here is my one.
261
00:15:29 --> 00:15:32
But here it is also w cubed.
262
00:15:32 --> 00:15:38
So w is the same as w^4,
w^2 is the same as w^5.
263
00:15:39 --> 00:15:44
So, what's the difference?
264
00:15:44 --> 00:15:48
What can I do now?
265
00:15:48 --> 00:15:55
If I'm in this discrete
case, then my inputs are a
266
00:15:55 --> 00:15:58
vector of length N, three.
267
00:15:58 --> 00:16:00
A vector of length N, three.
268
00:16:00 --> 00:16:06
And I want to get out to a
vector of length three.
269
00:16:06 --> 00:16:10
I'm not happy with that in
the cyclic case, because
270
00:16:10 --> 00:16:12
I'm not happy with w^4.
271
00:16:13 --> 00:16:17
So now tell me again that last,
when I do the multiplication
272
00:16:17 --> 00:16:21
and I just do it, there's no
difference except in how
273
00:16:21 --> 00:16:23
I write the answer.
274
00:16:23 --> 00:16:35
18w^4 is the same as? w. w^4 is
the same as w, when N is three.
275
00:16:35 --> 00:16:41
So that 18w^4 cycles
back in, with this 13.
276
00:16:41 --> 00:16:45
So now if I do the, can
I show you the symbol?
277
00:16:45 --> 00:16:47
I'll just do a little
circle there.
278
00:16:47 --> 00:16:52
To say this is now the
cyclic convolution.
279
00:16:52 --> 00:16:54
Then this isn't the
answer any more.
280
00:16:54 --> 00:16:57
The answer now for cyclic
convolution, I only
281
00:16:57 --> 00:16:59
want three numbers.
282
00:16:59 --> 00:17:01
And if you can tell me
what those three numbers
283
00:17:01 --> 00:17:04
are, we've got it.
284
00:17:04 --> 00:17:06
Move those over a little
to make room for
285
00:17:06 --> 00:17:07
the three numbers.
286
00:17:07 --> 00:17:11
So there's the answer, the
space for the answer.
287
00:17:11 --> 00:17:14
What do I write in?
288
00:17:14 --> 00:17:19
How many, what's
the constant term?
289
00:17:19 --> 00:17:21
It's 31, right.
290
00:17:21 --> 00:17:22
It's 31.
291
00:17:22 --> 00:17:25
Where did 31 come from?
292
00:17:25 --> 00:17:31
It came from, you could
say, cycling that 27w^3
293
00:17:31 --> 00:17:33
back with the four.
294
00:17:33 --> 00:17:36
Because there's no w^3
is the same as one.
295
00:17:36 --> 00:17:40
So I've gone around the circle
when I come around to w^3.
296
00:17:40 --> 00:17:45
So that 27 and four
combined into that 31.
297
00:17:45 --> 00:17:49
And let's see so so
what multiplications
298
00:17:49 --> 00:17:50
am I doing here?
299
00:17:50 --> 00:17:53
One times four gives
me the constant.
300
00:17:53 --> 00:18:00
Two times six, that's
2w's, and six w squareds,
301
00:18:00 --> 00:18:03
that's 12w^3, that's 12.
302
00:18:03 --> 00:18:09
And then 3w^2, and
five w's is 15w^3.
303
00:18:10 --> 00:18:13
But that's the same as 15.
304
00:18:13 --> 00:18:15
So that's why we get 31.
305
00:18:15 --> 00:18:20
We got four, we've got
12 and we've got 15.
306
00:18:20 --> 00:18:25
OK, now what's the second
component, the w component
307
00:18:25 --> 00:18:32
of the cyclic convolution?
308
00:18:32 --> 00:18:36
Tell me what number do I write
in, in that middle position?
309
00:18:36 --> 00:18:39
How many w's do I have?
310
00:18:39 --> 00:18:41
31 again.
311
00:18:41 --> 00:18:47
This 28, this 18 is
coming back by three.
312
00:18:47 --> 00:18:48
To 13.
313
00:18:48 --> 00:18:53
You see, I could have done
that multiplication.
314
00:18:53 --> 00:18:57
So coming back to 31, am I
going to get another 31?
315
00:18:57 --> 00:19:01
No, what's the w^2 guy?
316
00:19:01 --> 00:19:01
28.
317
00:19:01 --> 00:19:02
Yeah.
318
00:19:02 --> 00:19:09
28, because there's no to
the fifth to come back.
319
00:19:09 --> 00:19:15
So 28 uses three
multiplications.
320
00:19:15 --> 00:19:21
The 31 there used the
four and these two.
321
00:19:21 --> 00:19:24
This came back over to here.
322
00:19:24 --> 00:19:28
And this 31 used,
this 18 came back.
323
00:19:28 --> 00:19:30
I could have put the 18 here.
324
00:19:30 --> 00:19:33
You know, I could have
lined it up just three.
325
00:19:33 --> 00:19:35
So I'll write a formula for it.
326
00:19:35 --> 00:19:37
So that's the answer.
327
00:19:37 --> 00:19:41
31, 31, 28.
328
00:19:41 --> 00:19:44
Could I just suggest a
little check on that?
329
00:19:44 --> 00:19:46
Just to check on the numbers.
330
00:19:46 --> 00:19:52
I think that if I add up
these numbers, I get six.
331
00:19:52 --> 00:19:57
And if I add up those
numbers I get 15.
332
00:19:57 --> 00:20:01
And if I multiply
that, I get 90.
333
00:20:01 --> 00:20:04
And if I add those
numbers I get 90, right?
334
00:20:04 --> 00:20:08
So those add to 90.
335
00:20:08 --> 00:20:13
So I'm just saying the miracle
check is add these, multiply
336
00:20:13 --> 00:20:15
by the sum of those.
337
00:20:15 --> 00:20:17
And you get the sum of those.
338
00:20:17 --> 00:20:20
Why is that?
339
00:20:20 --> 00:20:23
Somehow seems
right, doesn't it?
340
00:20:23 --> 00:20:29
Because somehow I've taken all
nine products here, and so when
341
00:20:29 --> 00:20:33
I add all the results, I'll
have the sum of all nine of
342
00:20:33 --> 00:20:35
these possible products.
343
00:20:35 --> 00:20:37
So I'll have six times 15.
344
00:20:37 --> 00:20:41
Actually, here's a good
way to look at it.
345
00:20:41 --> 00:20:43
In doing this, I just set w=1.
346
00:20:43 --> 00:20:48
347
00:20:48 --> 00:20:51
I just set w=1 in
the polynomial.
348
00:20:51 --> 00:20:55
When I set w to one, this
becomes six, this becomes
349
00:20:55 --> 00:20:59
15 and the answer becomes
90, when w is one.
350
00:20:59 --> 00:21:00
Yeah.
351
00:21:00 --> 00:21:02
So that's another way to see.
352
00:21:02 --> 00:21:08
And actually, this
multiplication, the second
353
00:21:08 --> 00:21:14
grade version, had w, well,
I'm almost going to say
354
00:21:14 --> 00:21:19
w=10, but not quite that.
355
00:21:19 --> 00:21:22
Because it's written in the
opposite order, right?
356
00:21:22 --> 00:21:25
If w was ten, this
would be one.
357
00:21:25 --> 00:21:28
Well, anyway, w can't be
ten, it's got to stay
358
00:21:28 --> 00:21:31
on the unit circle, so.
359
00:21:31 --> 00:21:34
So somewhere in the non-cyclic
case, it's something like
360
00:21:34 --> 00:21:38
w=10, or w=1/10, maybe.
361
00:21:38 --> 00:21:39
Whatever.
362
00:21:39 --> 00:21:40
OK.
363
00:21:40 --> 00:21:43
Could you take the convolution
now, let me give you
364
00:21:43 --> 00:21:45
just another example.
365
00:21:45 --> 00:21:49
Do it mentally.
366
00:21:49 --> 00:21:55
What's the convolution
of , let me
367
00:21:55 --> 00:21:57
make a little longer.
368
00:21:57 --> 00:22:02
So I take the, first of all,
the non-cyclic convolution
369
00:22:02 --> 00:22:05
of .
370
00:22:05 --> 00:22:07
OK.
371
00:22:07 --> 00:22:11
What's the ordinary - how
long is the answer now?
372
00:22:11 --> 00:22:13
This is just practice.
373
00:22:13 --> 00:22:16
How many components am I going
to have in the ordinary
374
00:22:16 --> 00:22:22
convolution of those two
guys of length four?
375
00:22:22 --> 00:22:24
I think it's seven.
376
00:22:24 --> 00:22:26
I think it'll be seven.
377
00:22:26 --> 00:22:30
Because we'll have here one,
we have no, we have it
378
00:22:30 --> 00:22:34
z^0, z^1, z^2, z^3.
379
00:22:35 --> 00:22:39
And here we'll have again the
same, it would go up to z^6 but
380
00:22:39 --> 00:22:43
remember there's a z^0, so
that's why we have seven.
381
00:22:43 --> 00:22:47
And what will it be?
382
00:22:47 --> 00:22:49
What will it be?
383
00:22:49 --> 00:22:53
I guess, actually, this wasn't
a brilliant example, was it?
384
00:22:53 --> 00:22:55
But let's finish it.
385
00:22:55 --> 00:23:01
So I'm just multiplying z
by z squared, so what do
386
00:23:01 --> 00:23:02
you get for an answer?
387
00:23:02 --> 00:23:07
I think the one shows
up in the z^3.
388
00:23:07 --> 00:23:13
389
00:23:13 --> 00:23:14
Is that right?
390
00:23:14 --> 00:23:22
Yeah. z^1, here z^2 here, z^3
here, and we always have
391
00:23:22 --> 00:23:27
to remember everything in
Chapter 4 starts at zero.
392
00:23:27 --> 00:23:30
Zero, z^0's the first one.
393
00:23:30 --> 00:23:34
OK, that's not too great an
example, because what happens
394
00:23:34 --> 00:23:40
if I do the circular
cyclic convolution?
395
00:23:40 --> 00:23:42
What would be the
cyclic convolution?
396
00:23:42 --> 00:23:47
Now I'm expecting four
guys only, right?
397
00:23:47 --> 00:23:50
A cyclic keeps the same length.
398
00:23:50 --> 00:23:55
And what would be the answer?
399
00:23:55 --> 00:23:57
Well, there's nobody to
fold back, so it would
400
00:23:57 --> 00:23:59
be just .
401
00:23:59 --> 00:24:06
So let me update this a
little bit with one here.
402
00:24:06 --> 00:24:08
OK, just to practice.
403
00:24:08 --> 00:24:12
So suppose I do
that convolution.
404
00:24:12 --> 00:24:15
Un-cyclic, first.
405
00:24:15 --> 00:24:18
What do I change here now?
406
00:24:18 --> 00:24:22
I've now got a z^2 and I've
also got a z^3, but I only
407
00:24:22 --> 00:24:25
have a single one there.
408
00:24:25 --> 00:24:27
Let's make it a little
more interesting.
409
00:24:27 --> 00:24:30
OK, make it like so.
410
00:24:30 --> 00:24:36
Alright, z+z^2 is what
we're looking at, z+z^2
411
00:24:36 --> 00:24:37
is multiplying z^2+z^3.
412
00:24:39 --> 00:24:43
And in the long form,
what do I get?
413
00:24:43 --> 00:24:47
Let me make space, tell me
what numbers to put in.
414
00:24:47 --> 00:24:52
If I multiply
z+z^2+z^3, I get what?
415
00:24:52 --> 00:24:54
1z^3, how many z^4?
416
00:24:54 --> 00:24:57
417
00:24:57 --> 00:24:58
Two of them.
418
00:24:58 --> 00:24:58
How many z^5?
419
00:25:00 --> 00:25:01
One, and nobody there.
420
00:25:01 --> 00:25:02
OK.
421
00:25:02 --> 00:25:09
And now the cyclic
version would be what?
422
00:25:09 --> 00:25:12
What's what's my answer now
for the cyclic version?
423
00:25:12 --> 00:25:15
Let me take those out.
424
00:25:15 --> 00:25:21
So the cyclic version would
bring the two back to the zero.
425
00:25:21 --> 00:25:24
Would bring that
one back so there.
426
00:25:24 --> 00:25:27
That zero will still be zero.
427
00:25:27 --> 00:25:32
And I checked that I haven't
missed anything by adding those
428
00:25:32 --> 00:25:37
up to get four, and adding
this up to get two times two.
429
00:25:37 --> 00:25:38
Yeah.
430
00:25:38 --> 00:25:42
OK, so that's the rule.
431
00:25:42 --> 00:25:47
And it's a lot cleaner to
see these answers than
432
00:25:47 --> 00:25:52
to see this formula.
433
00:25:52 --> 00:25:57
And I need, actually of course,
now mentioning that formula,
434
00:25:57 --> 00:25:59
I need a cyclic formula.
435
00:25:59 --> 00:26:03
So can I write above it
the cyclic formula?
436
00:26:03 --> 00:26:13
What do I get when I'm, instead
of this sum, which went from
437
00:26:13 --> 00:26:21
k equal minus infinity to
infinity, in the cyclic case,
438
00:26:21 --> 00:26:29
h_k is just going to be a sum
from zero to N-1, and there'll
439
00:26:29 --> 00:26:33
be a c_k, and a d something.
440
00:26:33 --> 00:26:42
And now this is the cyclic
case, so I guess this makes us,
441
00:26:42 --> 00:26:46
I think what our situation now
is we understand the cyclic
442
00:26:46 --> 00:26:47
case from examples.
443
00:26:47 --> 00:26:52
And now we just have the job of
how do I put it into algebra.
444
00:26:52 --> 00:26:55
How do I put it into symbols?
445
00:26:55 --> 00:27:02
What's the point?
c_k d_n, let's say.
446
00:27:02 --> 00:27:08
But oh no, I'm looking
for h_l, yeah.
447
00:27:08 --> 00:27:10
So what's the deal?
448
00:27:10 --> 00:27:14
Here l was k.
449
00:27:14 --> 00:27:17
That one is the sum of
that one and that one.
450
00:27:17 --> 00:27:21
So here, l is the sum
of k and n, but.
451
00:27:21 --> 00:27:28
What's the but?
452
00:27:28 --> 00:27:35
I mean somehow I've got some
wraparound to do, right?
453
00:27:35 --> 00:27:39
When I'm doing the cyclic
multiplication and I'm doing
454
00:27:39 --> 00:27:47
the wraparound because w^n,
the wraparound comes
455
00:27:47 --> 00:27:49
from that, right?
456
00:27:49 --> 00:27:52
That's why I never get as high
as n, because when I get to n I
457
00:27:52 --> 00:27:55
go back to the zeroth power.
458
00:27:55 --> 00:28:02
OK, so what's the relation
of k and n and l here?
459
00:28:02 --> 00:28:07
We just need the right
word to express it.
460
00:28:07 --> 00:28:08
What's the word?
461
00:28:08 --> 00:28:10
Mod.
462
00:28:10 --> 00:28:17
So that's the word I'm
looking for. k+n is l.
463
00:28:17 --> 00:28:23
With wraparound and wraparound
means that the nice notation
464
00:28:23 --> 00:28:31
that people use is a mod m.
465
00:28:31 --> 00:28:35
Let's practice.
466
00:28:35 --> 00:28:45
What is two plus two mod seven?
467
00:28:45 --> 00:28:47
Four.
468
00:28:47 --> 00:28:49
Two plus two is four,
even in 18.085.
469
00:28:49 --> 00:28:51
Right, OK.
470
00:28:51 --> 00:29:04
But two plus two mod three is,
two plus two mod three is? one.
471
00:29:04 --> 00:29:06
Everybody sees it?
472
00:29:06 --> 00:29:14
I'm taking z^2 times z^2, z^4,
but I'm doing with N=3, so z^3
473
00:29:14 --> 00:29:20
is on3, so that z^4 is really
just z to the first power.
474
00:29:20 --> 00:29:27
So this is the little
nifty notation that
475
00:29:27 --> 00:29:29
says make it cyclic.
476
00:29:29 --> 00:29:34
Bring it back so that
l only has the values
477
00:29:34 --> 00:29:36
here, zero up to N-1.
478
00:29:38 --> 00:29:41
And then stops.
479
00:29:41 --> 00:29:45
OK.
480
00:29:45 --> 00:29:52
OK, so we'll have more practice
with examples when we
481
00:29:52 --> 00:29:56
do some filtering.
482
00:29:56 --> 00:30:00
Have you got that fundamental,
so we've talked about this rule
483
00:30:00 --> 00:30:07
one here. f times g goes
to those coefficients.
484
00:30:07 --> 00:30:11
And if it's the cyclic
case then I put a circle
485
00:30:11 --> 00:30:12
around that star.
486
00:30:12 --> 00:30:16
And I do the wraparound.
487
00:30:16 --> 00:30:26
But it's just, it's the z
transform, it's polynomials in
488
00:30:26 --> 00:30:31
z or polynomials in w, and when
it's polynomials in w, you use
489
00:30:31 --> 00:30:34
that special property
that w_N is one.
490
00:30:34 --> 00:30:38
Yeah, OK.
491
00:30:38 --> 00:30:43
Now, I see I've written
another line, there.
492
00:30:43 --> 00:30:46
That I could
convolve functions.
493
00:30:46 --> 00:30:48
Let me do a couple
more examples.
494
00:30:48 --> 00:30:50
Couple of examples.
495
00:30:50 --> 00:30:59
First, before I go
to that line, OK.
496
00:30:59 --> 00:31:05
So I'm up to this line.
497
00:31:05 --> 00:31:12
A couple of examples here.
498
00:31:12 --> 00:31:18
Let's see, what example
would I want to do?
499
00:31:18 --> 00:31:28
Let's see, OK, I want to do one
example with a delta function.
500
00:31:28 --> 00:31:31
One example with a
delta function.
501
00:31:31 --> 00:31:32
One example with
the delta vector.
502
00:31:32 --> 00:31:41
Yeah, let me take the function
g(x) identically one.
503
00:31:41 --> 00:31:45
OK, constant function.
504
00:31:45 --> 00:31:48
In this rule.
505
00:31:48 --> 00:31:50
I want to see what
happens with the rule.
506
00:31:50 --> 00:32:00
OK, then f(x) g(x) is the
same as f(x), right?
507
00:32:00 --> 00:32:03
Because this function g(x) is
so simple, it's just one.
508
00:32:03 --> 00:32:09
Now, what about
the coefficient?
509
00:32:09 --> 00:32:13
So I have the coefficients
of c, what are the Fourier
510
00:32:13 --> 00:32:15
coefficients, what are the d's?
511
00:32:15 --> 00:32:18
Ah yes, what are the d's?
512
00:32:18 --> 00:32:22
So I'm testing my rule on a
really really simple case,
513
00:32:22 --> 00:32:25
g(x) identically one.
514
00:32:25 --> 00:32:29
What, you have to tell me, in
order to check the right side
515
00:32:29 --> 00:32:33
of the rule you have to tell me
the Fourier coefficients for
516
00:32:33 --> 00:32:35
that very special function.
517
00:32:35 --> 00:32:38
What would be the
Fourier coefficients?
518
00:32:38 --> 00:32:41
If I expand the function
one in a Fourier
519
00:32:41 --> 00:32:44
series, what do I see?
520
00:32:44 --> 00:32:44
I see a one.
521
00:32:44 --> 00:32:47
Yeah, that's it, I see one.
522
00:32:47 --> 00:32:54
So what are its coefficients?
d_0, right, is one?
523
00:32:54 --> 00:33:01
And the other d's are
all zero, right?
524
00:33:01 --> 00:33:08
So my vector of d, my vector of
d's is a whole lot of zeroes
525
00:33:08 --> 00:33:09
on the negative side.
526
00:33:09 --> 00:33:11
A one right there in
the center, and then
527
00:33:11 --> 00:33:12
a lot of zeroes.
528
00:33:12 --> 00:33:24
And now I want to
convolve that with c.
529
00:33:24 --> 00:33:28
I'm practicing the convolution
rule on a case that's so
530
00:33:28 --> 00:33:34
simple it's confusing, right?
531
00:33:34 --> 00:33:45
I mean, it's a big mess,
this multiplication.
532
00:33:45 --> 00:33:47
What do I get out of this?
533
00:33:47 --> 00:33:58
If d is this vector, if d has
this property that d_0 is one
534
00:33:58 --> 00:34:04
and others are zero, all others
are zero, so this is my little
535
00:34:04 --> 00:34:10
example, what does this
sum boil down to?
536
00:34:10 --> 00:34:17
Well, I only get something
when l=k, right?
537
00:34:17 --> 00:34:22
I only got something when l=k,
because then I have d_0 and
538
00:34:22 --> 00:34:25
that's the only d
that's around.
539
00:34:25 --> 00:34:30
So in this sum, something
happens only when k
540
00:34:30 --> 00:34:31
and l are the same.
541
00:34:31 --> 00:34:33
And then what happens?
542
00:34:33 --> 00:34:37
Then I have a one, I have c_l,
and that's h_l, so that's
543
00:34:37 --> 00:34:39
all I'm concluding then.
544
00:34:39 --> 00:34:43
That this h is the same as c.
545
00:34:43 --> 00:34:48
I'm sorry, it's so dumb.
546
00:34:48 --> 00:34:51
My point is that in
convolution, this is the
547
00:34:51 --> 00:34:54
thing that acts like one.
548
00:34:54 --> 00:34:57
Because in multiplication,
that's the thing, that's the
549
00:34:57 --> 00:34:59
function that acts like one.
550
00:34:59 --> 00:35:01
That's the function
that is one.
551
00:35:01 --> 00:35:11
So this is the one in, oh,
would you allow me to do this?
552
00:35:11 --> 00:35:18
I'm going to create a
matrix with these d's.
553
00:35:18 --> 00:35:24
There's another way
to see convolution.
554
00:35:24 --> 00:35:26
Yeah, there's another way
to see convolution and
555
00:35:26 --> 00:35:28
discrete convolution.
556
00:35:28 --> 00:35:31
Maybe the discrete
one's the better.
557
00:35:31 --> 00:35:36
Yeah can you stand one more
way to write the formula?
558
00:35:36 --> 00:35:41
One more way to write, now I'm
going to do, I'm going to
559
00:35:41 --> 00:35:47
do discrete convolution.
560
00:35:47 --> 00:35:48
Discrete cyclic.
561
00:35:48 --> 00:35:53
Just So how am I
going to write it?
562
00:35:53 --> 00:35:57
I'm going to write it by
a matrix multiplication.
563
00:35:57 --> 00:35:59
Because you know that in
this course a matrix
564
00:35:59 --> 00:36:01
was going to show up.
565
00:36:01 --> 00:36:06
So it's going to be a
matrix multiplication.
566
00:36:06 --> 00:36:09
So I just have to tell you
the matrix, so this is
567
00:36:09 --> 00:36:12
going to be some matrix.
568
00:36:12 --> 00:36:15
Let me take N to be four.
569
00:36:15 --> 00:36:20
So then I have, you watch.
570
00:36:20 --> 00:36:32
So I have four d's, and the
output is the four h's.
571
00:36:32 --> 00:36:37
And the rule I'm
following is this rule.
572
00:36:37 --> 00:36:43
Is this, the same old rule
but with the cyclic part.
573
00:36:43 --> 00:36:47
And now I want to show you the
matrix that'll just do this.
574
00:36:47 --> 00:36:54
Look, I've put the c's
in the first column.
575
00:36:54 --> 00:37:00
And then I go, yeah,
here's another.
576
00:37:00 --> 00:37:07
So it's a cyclic matrix.
577
00:37:07 --> 00:37:09
So let me finish it up.
578
00:37:09 --> 00:37:13
It's going to be four by four,
it's going to be cyclic.
579
00:37:13 --> 00:37:18
So I have a c_0, c_0, c_0,
c_0 on the diagonal.
580
00:37:18 --> 00:37:22
That's fine, that's because z
to the zeroes multiplying all
581
00:37:22 --> 00:37:24
the d's and leaving
them in place.
582
00:37:24 --> 00:37:29
And then I have c_1's, and
then I think I come around
583
00:37:29 --> 00:37:30
again here for a c_1.
584
00:37:31 --> 00:37:37
And I have c_2's, see
where see c_3, c_2.
585
00:37:38 --> 00:37:43
And I come around again,
to a c_2 and a c_2.
586
00:37:43 --> 00:37:48
And c_3 comes around to
a c_3, a c_3 and a c_3.
587
00:37:49 --> 00:37:59
Well, can you, I hope you can
see, this is cyclic matrix.
588
00:37:59 --> 00:38:06
It's only got one, it starts
with a vector c, and those
589
00:38:06 --> 00:38:10
are on the diagonal and
the diagonals wrap around.
590
00:38:10 --> 00:38:13
That's the other word that you
often see when you see the
591
00:38:13 --> 00:38:16
word cyclic, wraparound.
592
00:38:16 --> 00:38:23
It's because you
think of a circle.
593
00:38:23 --> 00:38:26
If you go the second time
around, it's wrapped
594
00:38:26 --> 00:38:28
around the first time.
595
00:38:28 --> 00:38:32
OK, just can you look and see
that this is the right formula
596
00:38:32 --> 00:38:34
for h_0? h_0 is c_0*d_0.
597
00:38:36 --> 00:38:37
Where does that come from?
598
00:38:37 --> 00:38:41
Remember, h_0 is the
coefficient of z^0
599
00:38:41 --> 00:38:43
in the answer.
600
00:38:43 --> 00:38:47
So it comes from c_0*d_0 to the
zeroth power in the input.
601
00:38:47 --> 00:38:49
And then why is
there is a c_3*d_1?
602
00:38:49 --> 00:38:52
603
00:38:52 --> 00:38:57
Why is there a c_3*d_1, and
then a c_2*d_2 and then a
604
00:38:57 --> 00:39:00
c_1*d_3 all piling up into h_0?
605
00:39:02 --> 00:39:04
Tell me now, why is
there a c_3*d_1?
606
00:39:04 --> 00:39:07
607
00:39:07 --> 00:39:10
Because we're doing mod
four is one way to say it.
608
00:39:10 --> 00:39:12
Three and one add to four.
609
00:39:12 --> 00:39:21
Because c_3 is the w cubed guy,
and d_1 is the coefficient of
610
00:39:21 --> 00:39:30
w^1 t and w^3 times w^1 piles
back into the constant.
611
00:39:30 --> 00:39:33
And you see the pattern
of that matrix?
612
00:39:33 --> 00:39:38
So these matrices
are very important.
613
00:39:38 --> 00:39:40
So they circle around.
614
00:39:40 --> 00:39:45
Oh, we've actually met a
matrix of this type, the
615
00:39:45 --> 00:39:49
first day of 18.085.
616
00:39:49 --> 00:39:51
What was that matrix?
617
00:39:51 --> 00:39:53
It was one of our
four great matrices.
618
00:39:53 --> 00:39:56
And now here it is back again.
619
00:39:56 --> 00:40:00
Which one was it?
620
00:40:00 --> 00:40:03
Well, you remember
the letter for it.
621
00:40:03 --> 00:40:05
Which isn't going to change.
622
00:40:05 --> 00:40:11
And do you remember the
particular matrix?
623
00:40:11 --> 00:40:14
Well, everybody does remember
that matrix, right?
624
00:40:14 --> 00:40:18
Twos were on the diagonal,
minus ones were on the
625
00:40:18 --> 00:40:22
diagonal, and the diagonal
curve continued.
626
00:40:22 --> 00:40:26
Minus one was on this diagonal
and that, continued and
627
00:40:26 --> 00:40:28
zeroes was on this diagonal.
628
00:40:28 --> 00:40:36
So this is cyclic convolution,
the circulant matrix, cyclic
629
00:40:36 --> 00:40:43
convolution by c, what's the
c that produces that
630
00:40:43 --> 00:40:46
convolution matrix?
631
00:40:46 --> 00:40:50
It's just, it's got
- well there it is.
632
00:40:50 --> 00:40:56
The first column is it.
633
00:40:56 --> 00:40:57
Right, right.
634
00:40:57 --> 00:41:09
And somehow I would say that
that's an even vector.
635
00:41:09 --> 00:41:12
It's sort of, I associate
it with cosine.
636
00:41:12 --> 00:41:13
It's an even vector.
637
00:41:13 --> 00:41:19
Here is the zero term, and
then these are the same, not
638
00:41:19 --> 00:41:21
to worry about that part.
639
00:41:21 --> 00:41:25
Do you see that we've
seen that matrix before?
640
00:41:25 --> 00:41:29
And the cyclic convolution
means you take its second
641
00:41:29 --> 00:41:32
differences, of course.
642
00:41:32 --> 00:41:36
We're taking second
differences, but our everything
643
00:41:36 --> 00:41:40
in our world is cyclic.
644
00:41:40 --> 00:41:46
So the result, the x_4 is x_0.
645
00:41:47 --> 00:41:50
So we're taking second
differences - well, maybe I
646
00:41:50 --> 00:41:55
should say d - we're taking
second differences d_i,
647
00:41:55 --> 00:42:04
2d_i's, -d_(i-1), -d(i+1),
I don't know if this is.
648
00:42:04 --> 00:42:07
So there's a minus
one, two, minus one.
649
00:42:07 --> 00:42:10
And we're cycling around
so that d_0 is d_4.
650
00:42:11 --> 00:42:20
And d_1 is d_5, and
d_-1 is d_3, whatever.
651
00:42:20 --> 00:42:25
OK, I'm just reminding you,
we've seen these before.
652
00:42:25 --> 00:42:33
OK, so this is another way
to remember the formula.
653
00:42:33 --> 00:42:37
OK now can I ask you a
practical question?
654
00:42:37 --> 00:42:39
A practical question.
655
00:42:39 --> 00:42:47
Let me bring back this second
grade multiplication.
656
00:42:47 --> 00:42:50
Well, I have a granddaughter
named Elizabeth, I'll have to
657
00:42:50 --> 00:42:52
admit I didn't think about
mentioning Elizabeth.
658
00:42:52 --> 00:42:54
She's six.
659
00:42:54 --> 00:42:59
And she delights in sending
me long multiplications.
660
00:42:59 --> 00:43:02
I mean, really long.
661
00:43:02 --> 00:43:05
And then every time I talk to
her on the phone, she says
662
00:43:05 --> 00:43:06
have you done that one yet?
663
00:43:06 --> 00:43:11
And I say, I'm working on it
I've got MATLAB at work.
664
00:43:11 --> 00:43:15
Because they're ridiculous
and I haven't figured
665
00:43:15 --> 00:43:15
out how to tell her.
666
00:43:15 --> 00:43:20
I mean, she just writes
page after page.
667
00:43:20 --> 00:43:24
Times 100, plus three,
minus seven, just
668
00:43:24 --> 00:43:25
whatever she things of.
669
00:43:25 --> 00:43:30
OK, now I need help from
the convolution groupal,
670
00:43:30 --> 00:43:31
here, actually.
671
00:43:31 --> 00:43:35
So let's suppose that Elizabeth
has given me a multiplication
672
00:43:35 --> 00:43:41
in which I have a thousand
digits times a thousand, right.
673
00:43:41 --> 00:43:44
Which Mathematica is prepared
to do exactly, right?
674
00:43:44 --> 00:43:51
MATLAB will mess up, but
Mathematica and Maple and
675
00:43:51 --> 00:43:55
symbolic packages will
do exact computations.
676
00:43:55 --> 00:43:59
So what would be the right way,
well let may make it 1,024.
677
00:43:59 --> 00:44:08
1,024 digits times
1,024 digits.
678
00:44:08 --> 00:44:11
Let's do the cyclic
version first.
679
00:44:11 --> 00:44:13
Elizabeth doesn't
know about cyclic.
680
00:44:13 --> 00:44:14
Maybe I could teach her that.
681
00:44:14 --> 00:44:18
That'll keep her busy while
I'm doing the multiplication.
682
00:44:18 --> 00:44:23
OK, right.
683
00:44:23 --> 00:44:25
Only her older brother
would explain it to her,
684
00:44:25 --> 00:44:26
that's the trouble.
685
00:44:26 --> 00:44:30
OK, so how am I going to do, or
how are you going to do on the
686
00:44:30 --> 00:44:37
quiz, multiplication of a
1,024 digits times 1,024?
687
00:44:37 --> 00:44:40
And I'll make it easy by making
it cyclic, so I just want
688
00:44:40 --> 00:44:43
1,024 digits in the answer.
689
00:44:43 --> 00:44:45
OK.
690
00:44:45 --> 00:44:48
How would you do it?
691
00:44:48 --> 00:44:55
Well, before today, you would
have just multiplied, right?
692
00:44:55 --> 00:44:59
You would have written
down 1,024 two, lines of
693
00:44:59 --> 00:45:03
1,024, done an addition.
694
00:45:03 --> 00:45:08
And you would have had a
million multiplications to do.
695
00:45:08 --> 00:45:10
But how would you do it now?
696
00:45:10 --> 00:45:14
Apart from giving
it to Mathematica.
697
00:45:14 --> 00:45:17
What's a faster way to do it?
698
00:45:17 --> 00:45:21
What's a faster way
to do a convolution?
699
00:45:21 --> 00:45:24
The fast way to do a
convolution is to use the
700
00:45:24 --> 00:45:27
convolution rule, go this way.
701
00:45:27 --> 00:45:32
So take these numbers, these
1,024 numbers, in c and these
702
00:45:32 --> 00:45:40
1,024 numbers in d, and,
well what do I have to do?
703
00:45:40 --> 00:45:42
I want to use the
convolution rule, because
704
00:45:42 --> 00:45:47
multiplying is fast.
705
00:45:47 --> 00:45:50
Now I've got functions.
706
00:45:50 --> 00:45:52
But I'm in the cyclic case.
707
00:45:52 --> 00:45:58
So I'm in the cyclic case,
so what should I do?
708
00:45:58 --> 00:46:03
How can I change this
to be the cyclic case?
709
00:46:03 --> 00:46:08
This is like f, j, g, g, j.
710
00:46:08 --> 00:46:16
So multiplication of components
of things in function space is
711
00:46:16 --> 00:46:18
convolution of coefficients.
712
00:46:18 --> 00:46:22
So now, this is the cyclic.
713
00:46:22 --> 00:46:25
So let me make it cyclic.
714
00:46:25 --> 00:46:28
So again, what's your problem?
715
00:46:28 --> 00:46:31
The problem is to do this
cyclic multiplication.
716
00:46:31 --> 00:46:33
What's the idea?
717
00:46:33 --> 00:46:41
The idea is to transform c back
to f, to transform d back to g.
718
00:46:41 --> 00:46:44
Do the multiplications,
now I have only 1,024
719
00:46:44 --> 00:46:45
multiplications.
720
00:46:45 --> 00:46:47
Not 1,024 squared.
721
00:46:47 --> 00:46:49
That's the point.
722
00:46:49 --> 00:46:53
And if I do this directly,
I've got 1,024 squared
723
00:46:53 --> 00:46:55
multiplications to do.
724
00:46:55 --> 00:46:57
Much better.
725
00:46:57 --> 00:47:02
Transform back to here, do just
1,024 - what's the MATLAB
726
00:47:02 --> 00:47:05
command for that, when
you're multiplying each
727
00:47:05 --> 00:47:06
component by itself?
728
00:47:06 --> 00:47:09
It's not the dot
product, notice.
729
00:47:09 --> 00:47:13
It's not the dot product
because I'm not summing.
730
00:47:13 --> 00:47:18
Do you know the MATLAB command,
if I have a sequence of numbers
731
00:47:18 --> 00:47:25
of vector f, of length 1,024
and I want to get that result?
732
00:47:25 --> 00:47:26
What's the result?
733
00:47:26 --> 00:47:32
It's a vector of length 1,024
that takes each f times its g.
734
00:47:32 --> 00:47:34
But doesn't do any adds.
735
00:47:34 --> 00:47:36
That's what's there.
736
00:47:36 --> 00:47:38
What's the MATLAB
command for that?
737
00:47:38 --> 00:47:40
Dot, yeah.
738
00:47:40 --> 00:47:42
Dot star, right.
739
00:47:42 --> 00:47:45
So that dot says
component by component.
740
00:47:45 --> 00:47:48
OK, so what's the plan here?
741
00:47:48 --> 00:47:53
I do c's back to F.
742
00:47:53 --> 00:48:00
By the Fourier matrix. d back
to g, by the Fourier matrix,
743
00:48:00 --> 00:48:04
then I do a very quick
multiplication.
744
00:48:04 --> 00:48:05
And then what?
745
00:48:05 --> 00:48:07
Then I mustn't forget.
746
00:48:07 --> 00:48:11
That I'm in frequency space,
and what do I have to do?
747
00:48:11 --> 00:48:16
I've got to get back
into coefficient space.
748
00:48:16 --> 00:48:24
So I do the inverse transform
- here's the formula, then.
749
00:48:24 --> 00:48:32
I'm doing the inverse transform
of F, so the transform of c
750
00:48:32 --> 00:48:39
dot star, the transform of d.
751
00:48:39 --> 00:48:44
To get c, d.
752
00:48:44 --> 00:48:47
Is that right?
753
00:48:47 --> 00:48:51
So I took c, and I got
back into the function.
754
00:48:51 --> 00:48:54
I took d, and got back
to its function, with
755
00:48:54 --> 00:48:55
the Fourier matrix.
756
00:48:55 --> 00:48:59
OK, I'm in the Fourier and
now I'm in this space.
757
00:48:59 --> 00:49:03
I've added up coefficients
to get in this space.
758
00:49:03 --> 00:49:06
Now I do the dot
star, the fast one.
759
00:49:06 --> 00:49:08
And then I transform back.
760
00:49:08 --> 00:49:11
So why is that faster?
761
00:49:11 --> 00:49:14
Than just doing it?
762
00:49:14 --> 00:49:20
Because what's the
cost of F times c?
763
00:49:20 --> 00:49:23
And how am I going to do that?
764
00:49:23 --> 00:49:25
I'm going to do with the fast
Fourier transform, right.
765
00:49:25 --> 00:49:26
That's the point.
766
00:49:26 --> 00:49:32
I can multiply by F, or by
F inverse faster so I have
767
00:49:32 --> 00:49:36
three of these transforms.
768
00:49:36 --> 00:49:39
I've got to get two guys
into the other space,
769
00:49:39 --> 00:49:42
and the answer back out.
770
00:49:42 --> 00:49:49
So I have sort of three of
these n log n, but that
771
00:49:49 --> 00:49:53
will easily be n squared.
772
00:49:53 --> 00:49:53
Right?
773
00:49:53 --> 00:50:02
So if you have a convolution to
do, and it's possible to do
774
00:50:02 --> 00:50:07
this, get into the other space
where it's just an element
775
00:50:07 --> 00:50:09
by element multiplication.
776
00:50:09 --> 00:50:11
And that would apply
in either direction.
777
00:50:11 --> 00:50:13
Because the rule
goes both ways.
778
00:50:13 --> 00:50:18
If I have this convolution
to do, I would find the
779
00:50:18 --> 00:50:24
coefficients here, the c's, the
coefficients of d of the g's.
780
00:50:24 --> 00:50:28
I would do this one by one
multiplication, and then I
781
00:50:28 --> 00:50:32
have the Fourier coefficients
of the convolution.
782
00:50:32 --> 00:50:35
Right, OK?
783
00:50:35 --> 00:50:37
Do I have a moment?
784
00:50:37 --> 00:50:37
Well, hardly.
785
00:50:37 --> 00:50:41
Just can I write down what the
formula for a convolution of
786
00:50:41 --> 00:50:44
two functions would look like?
787
00:50:44 --> 00:50:46
Sorry, f(x) convolved
with g(x).
788
00:50:48 --> 00:50:50
Let me make it cyclic.
789
00:50:50 --> 00:50:53
Just to see what it
would look like.
790
00:50:53 --> 00:50:56
What am I expecting
for that convolution?
791
00:50:56 --> 00:50:59
I'm expecting a function, and
somehow there's going to be an
792
00:50:59 --> 00:51:02
integral instead of a sum,
where I had sums, but
793
00:51:02 --> 00:51:04
now I have integrals.
794
00:51:04 --> 00:51:05
And here's the point.
795
00:51:05 --> 00:51:10
I'll have f(t) times g(x-t)dt.
796
00:51:10 --> 00:51:16
797
00:51:16 --> 00:51:19
All I'm asking you to look
at is the fact that the
798
00:51:19 --> 00:51:21
way here I had k, and l-k.
799
00:51:23 --> 00:51:27
For functions, your eye
sees that right away as
800
00:51:27 --> 00:51:28
a t, and an (x-t)dt.
801
00:51:30 --> 00:51:33
These add to the answer.
802
00:51:33 --> 00:51:36
These add to the
result, that x.
803
00:51:36 --> 00:51:39
That would be the
cyclical one, yeah.
804
00:51:39 --> 00:51:45
So I could go zero- these are
all periodic functions, so all
805
00:51:45 --> 00:51:46
2pi periods are the same.
806
00:51:46 --> 00:51:50
The book will do that properly.
807
00:51:50 --> 00:51:54
OK, we've got the filtering
to discuss on Monday.
808
00:51:54 --> 00:51:59
You can see that this
convolution stuff just takes
809
00:51:59 --> 00:52:03
a little new thinking,
but it comes out nicely.