1
00:00:00,000 --> 00:00:00,499
2
00:00:00,499 --> 00:00:03,144
The following content is
provided under a Creative
3
00:00:03,144 --> 00:00:03,810
Commons license.
4
00:00:03,810 --> 00:00:05,518
Your support will help
MIT OpenCourseWare
5
00:00:05,518 --> 00:00:10,405
continue to offer high-quality
educational resources for free.
6
00:00:10,405 --> 00:00:12,530
To make a donation, or to
view additional materials
7
00:00:12,530 --> 00:00:17,080
from hundreds of MIT courses,
visit MIT OpenCourseWare
8
00:00:17,080 --> 00:00:20,550
at ocw.mit.edu.
9
00:00:20,550 --> 00:00:22,950
PROFESSOR STRANG: So,
thanks for coming today.
10
00:00:22,950 --> 00:00:30,370
This is a key lecture in
the application of Fourier,
11
00:00:30,370 --> 00:00:33,090
you could say.
12
00:00:33,090 --> 00:00:35,640
So convolution is the big word.
13
00:00:35,640 --> 00:00:39,250
And a major application
of convolution
14
00:00:39,250 --> 00:00:43,200
is filtering, signal processing.
15
00:00:43,200 --> 00:00:46,530
So we'll develop
that application.
16
00:00:46,530 --> 00:00:48,940
But it's nothing
but convolution.
17
00:00:48,940 --> 00:00:53,860
So the key idea is
these convolution rules,
18
00:00:53,860 --> 00:00:55,340
where they come from.
19
00:00:55,340 --> 00:00:58,510
And what these new
symbols, that's
20
00:00:58,510 --> 00:01:03,400
the symbol for the
convolution of two functions.
21
00:01:03,400 --> 00:01:05,390
They could be functions here.
22
00:01:05,390 --> 00:01:09,240
Here they're long
vectors of coefficients,
23
00:01:09,240 --> 00:01:14,240
and so these are the rules.
24
00:01:14,240 --> 00:01:16,240
So it's just a little
bit of algebra.
25
00:01:16,240 --> 00:01:21,110
But it just is so central
to all this subject.
26
00:01:21,110 --> 00:01:26,290
Signal processing is certainly
the most important little thing
27
00:01:26,290 --> 00:01:27,310
to know.
28
00:01:27,310 --> 00:01:29,230
That if I multiply
two functions,
29
00:01:29,230 --> 00:01:31,650
so that's where we
started last time.
30
00:01:31,650 --> 00:01:33,790
If I have a function
f with Fourier
31
00:01:33,790 --> 00:01:39,900
coefficients c, and a function
g with coefficients d, then,
32
00:01:39,900 --> 00:01:44,060
oh, wrong.
33
00:01:44,060 --> 00:01:48,590
I convolve the
coefficients, right.
34
00:01:48,590 --> 00:01:53,900
So f has coefficients
c, g has coefficients d.
35
00:01:53,900 --> 00:01:57,920
So if I multiply the functions
I definitely do not just
36
00:01:57,920 --> 00:02:00,580
multiply each c times the d.
37
00:02:00,580 --> 00:02:04,640
I do this convolution operation,
which we have to remember.
38
00:02:04,640 --> 00:02:09,790
That's our main first step is
to remember what that was about.
39
00:02:09,790 --> 00:02:13,550
And then this is
the other direction.
40
00:02:13,550 --> 00:02:15,910
So we didn't see this before.
41
00:02:15,910 --> 00:02:20,980
That I mean, there's always
this fantastic symmetry
42
00:02:20,980 --> 00:02:26,360
between physical space
and frequency space.
43
00:02:26,360 --> 00:02:30,810
And convolution in one is
multiplication in the other.
44
00:02:30,810 --> 00:02:33,240
It's so easy to remember.
45
00:02:33,240 --> 00:02:36,890
If I multiply in one
space, I do a convolution
46
00:02:36,890 --> 00:02:38,240
in the other space.
47
00:02:38,240 --> 00:02:41,490
If I do a convolution
of functions,
48
00:02:41,490 --> 00:02:45,500
I do a multiplication
of coefficients.
49
00:02:45,500 --> 00:02:49,510
So that's the rule to know.
50
00:02:49,510 --> 00:02:52,440
And now if we expand
on it, by sort
51
00:02:52,440 --> 00:02:54,750
of seeing again what it means.
52
00:02:54,750 --> 00:02:59,780
So let me do a quick
repeat of this first step
53
00:02:59,780 --> 00:03:04,840
to remember what this symbol,
convolution symbol, means.
54
00:03:04,840 --> 00:03:07,790
And then another
thing I have to do.
55
00:03:07,790 --> 00:03:11,380
Here, I'm talking about the
infinite case, functions,
56
00:03:11,380 --> 00:03:15,810
and with a whole infinite
sequence of coefficients.
57
00:03:15,810 --> 00:03:20,530
I've got to do the
cyclic case, too.
58
00:03:20,530 --> 00:03:23,590
Which goes with the
discrete transform.
59
00:03:23,590 --> 00:03:26,630
OK, but let's start
with the infinite case
60
00:03:26,630 --> 00:03:29,160
as we did last time.
61
00:03:29,160 --> 00:03:31,500
Maybe, here's something.
62
00:03:31,500 --> 00:03:32,780
Here's a suggestion.
63
00:03:32,780 --> 00:03:36,480
We had f(x).
64
00:03:36,480 --> 00:03:40,840
We started with f(x), as
the sum -- and remember,
65
00:03:40,840 --> 00:03:45,050
to have nice formulas, we're
doing the complex version --
66
00:03:45,050 --> 00:03:47,470
c_k*e^(ikx).
67
00:03:47,470 --> 00:03:49,350
Let me suggest something.
68
00:03:49,350 --> 00:03:52,880
Let me write z for e^(ikx).
69
00:03:52,880 --> 00:04:00,060
So I'm going to just write
that at the sum of c_k*z^k.
70
00:04:00,060 --> 00:04:01,600
So z is e^(ix).
71
00:04:01,600 --> 00:04:05,970
72
00:04:05,970 --> 00:04:09,300
This is like a good
point to make anyway.
73
00:04:09,300 --> 00:04:14,220
When we have this e^(ikx),
it's natural to think of that
74
00:04:14,220 --> 00:04:18,520
as a complex number
on the unit circle.
75
00:04:18,520 --> 00:04:19,020
Right?
76
00:04:19,020 --> 00:04:21,180
That's always the
message, e to the i
77
00:04:21,180 --> 00:04:23,390
real is on the unit circle.
78
00:04:23,390 --> 00:04:25,100
Absolute value one.
79
00:04:25,100 --> 00:04:29,760
And this is great for
periodic functions.
80
00:04:29,760 --> 00:04:32,870
Because if these
functions have period 2pi,
81
00:04:32,870 --> 00:04:39,750
and if we think of our function
as being on the circle,
82
00:04:39,750 --> 00:04:42,290
it obviously has period 2pi.
83
00:04:42,290 --> 00:04:44,770
I mean, the picture says, yes.
84
00:04:44,770 --> 00:04:47,030
If I go 2pi, I come back.
85
00:04:47,030 --> 00:04:52,625
So x is the angle, right?
x is the-- In this picture,
86
00:04:52,625 --> 00:04:59,050
a little bit unusual maybe,
x there would be the angle.
87
00:04:59,050 --> 00:05:03,320
And it's just a little
bit easier to write.
88
00:05:03,320 --> 00:05:06,630
And maybe it even has an
official name, the Z-transform.
89
00:05:06,630 --> 00:05:07,660
How do you like that?
90
00:05:07,660 --> 00:05:13,791
You learn a transform
just, in fourteen seconds.
91
00:05:13,791 --> 00:05:14,290
Z-transform.
92
00:05:14,290 --> 00:05:17,750
93
00:05:17,750 --> 00:05:20,980
It'll make it easier
to multiply by g.
94
00:05:20,980 --> 00:05:26,070
So g is going to be
the sum of d_k*z^k.
95
00:05:26,070 --> 00:05:30,100
So just think of these
as long, well I'm
96
00:05:30,100 --> 00:05:32,400
tempted to say long polynomials.
97
00:05:32,400 --> 00:05:37,140
I mean, very long, because
they can be infinite series.
98
00:05:37,140 --> 00:05:40,460
Infinite negative powers
and positive powers.
99
00:05:40,460 --> 00:05:44,410
But just think of it as a
bunch of powers of z times
100
00:05:44,410 --> 00:05:46,670
a bunch of powers of z.
101
00:05:46,670 --> 00:05:51,090
And if you multiply a
couple of polynomials,
102
00:05:51,090 --> 00:05:54,230
you're doing convolution.
103
00:05:54,230 --> 00:05:56,890
I guess my message is,
you've been doing convolution
104
00:05:56,890 --> 00:05:59,740
since the second grade.
105
00:05:59,740 --> 00:06:01,380
That's the real message.
106
00:06:01,380 --> 00:06:04,020
Here, let me show you.
107
00:06:04,020 --> 00:06:06,530
This is convolution too.
108
00:06:06,530 --> 00:06:14,550
Suppose I have to
multiply a 123 times 456?
109
00:06:14,550 --> 00:06:17,910
OK, so what do you do?
110
00:06:17,910 --> 00:06:22,640
Remember back, it's a long
way back but we can do this.
111
00:06:22,640 --> 00:06:25,470
One, two, three times
four, five, six.
112
00:06:25,470 --> 00:06:30,460
So I multiply the three,
oh there's a little point.
113
00:06:30,460 --> 00:06:33,170
Where that second-grade
teacher's going to panic.
114
00:06:33,170 --> 00:06:37,010
I'm going to write that as 18.
115
00:06:37,010 --> 00:06:41,040
And that's 15, and that's 12.
116
00:06:41,040 --> 00:06:43,830
Sorry about that, yeah.
117
00:06:43,830 --> 00:06:48,930
And 12, 10 and 8, right?
118
00:06:48,930 --> 00:06:50,140
Four, five and six.
119
00:06:50,140 --> 00:06:53,970
So you see the nine
multiplications
120
00:06:53,970 --> 00:06:55,740
that you have to do?
121
00:06:55,740 --> 00:06:58,650
Nine multiplications,
three times three.
122
00:06:58,650 --> 00:07:04,994
OK, right now
imagine those were,
123
00:07:04,994 --> 00:07:06,160
they could have been longer.
124
00:07:06,160 --> 00:07:13,810
But they were finite length
filters, we could say.
125
00:07:13,810 --> 00:07:17,200
And now, what does
convolution do?
126
00:07:17,200 --> 00:07:19,990
Just what you did
in multiplication.
127
00:07:19,990 --> 00:07:23,690
When I add 12, 10 and
6, what am I doing?
128
00:07:23,690 --> 00:07:26,580
I'm putting together the
three times the four,
129
00:07:26,580 --> 00:07:29,340
the two times the five,
and the one times the six.
130
00:07:29,340 --> 00:07:31,510
That's what convolution,
those are the things
131
00:07:31,510 --> 00:07:33,220
that convolution puts together.
132
00:07:33,220 --> 00:07:40,710
So we got an 18, a 27, 28.
133
00:07:40,710 --> 00:07:45,020
13, and 4.
134
00:07:45,020 --> 00:07:49,750
So I guess I'm saying
that, alright, here's
135
00:07:49,750 --> 00:07:51,110
what I really want to say.
136
00:07:51,110 --> 00:07:56,600
I want to say that the
convolution of those,
137
00:07:56,600 --> 00:08:04,010
with these, four, five, six,
is this sequence here-- Oh.
138
00:08:04,010 --> 00:08:04,980
Yeah, that's right.
139
00:08:04,980 --> 00:08:09,430
4, 13, 28.
140
00:08:09,430 --> 00:08:13,020
27, and 18.
141
00:08:13,020 --> 00:08:15,060
If you just look at that.
142
00:08:15,060 --> 00:08:17,690
Where did that 13 come from?
143
00:08:17,690 --> 00:08:21,360
Let's just remember, where
does that 13 come from?
144
00:08:21,360 --> 00:08:25,160
That came from, this was z^0.
145
00:08:25,160 --> 00:08:29,890
This 13 is 13 z to the
first power, right?
146
00:08:29,890 --> 00:08:33,100
We're just checking
all the powers here.
147
00:08:33,100 --> 00:08:35,520
There's 13 z to the first power.
148
00:08:35,520 --> 00:08:37,220
Where do we get a first power?
149
00:08:37,220 --> 00:08:41,620
We get a z^0 times 5z^1.
150
00:08:41,620 --> 00:08:43,460
So that's 5z^1.
151
00:08:43,460 --> 00:08:49,520
And we also have
2z^1, times 4z^0.
152
00:08:49,520 --> 00:08:50,040
Right?
153
00:08:50,040 --> 00:08:52,170
Two times four gave the eight.
154
00:08:52,170 --> 00:08:55,070
So the eight from there
and the five from there
155
00:08:55,070 --> 00:08:56,650
produce that 13.
156
00:08:56,650 --> 00:09:00,120
And that's just what you
did in multiplication.
157
00:09:00,120 --> 00:09:01,630
Right?
158
00:09:01,630 --> 00:09:06,620
So that's multiplication
of two series.
159
00:09:06,620 --> 00:09:08,490
That's not cyclic.
160
00:09:08,490 --> 00:09:11,010
This is definitely
not yet cyclic,
161
00:09:11,010 --> 00:09:13,750
but we'll make it
cyclic in a minute.
162
00:09:13,750 --> 00:09:20,720
This is the infinite one, except
that we had all zeroes beyond.
163
00:09:20,720 --> 00:09:24,500
OK, so that if you, in
non-cyclic convolution like
164
00:09:24,500 --> 00:09:29,280
this, if I have
length m and length n,
165
00:09:29,280 --> 00:09:34,790
then I get length
m+n, maybe m+n-1.
166
00:09:34,790 --> 00:09:37,830
OK, so that's convolution.
167
00:09:37,830 --> 00:09:40,590
Without carrying numbers.
168
00:09:40,590 --> 00:09:42,190
Without doing it right.
169
00:09:42,190 --> 00:09:46,360
OK, so and what does
that correspond to?
170
00:09:46,360 --> 00:09:48,770
Let me just, so you
see it every way.
171
00:09:48,770 --> 00:09:57,410
That corresponds to 1+2z+3z^2,
multiplying 4+5z+6z^2.
172
00:09:57,410 --> 00:10:00,160
173
00:10:00,160 --> 00:10:02,830
And it gave-- That's times.
174
00:10:02,830 --> 00:10:10,160
And it gave this
thing up to 18z^4.
175
00:10:10,160 --> 00:10:14,400
Just, exactly the multiplication
that you've always done.
176
00:10:14,400 --> 00:10:17,380
OK, so that's the idea.
177
00:10:17,380 --> 00:10:19,510
Over on that board I'm
going to put a formula
178
00:10:19,510 --> 00:10:22,230
for this convolution operation.
179
00:10:22,230 --> 00:10:27,740
But my point on this board
is, you've done it always.
180
00:10:27,740 --> 00:10:29,900
When you multiply a
couple of polynomials,
181
00:10:29,900 --> 00:10:31,930
you collect powers.
182
00:10:31,930 --> 00:10:34,760
And that's all convolution
is doing, collecting
183
00:10:34,760 --> 00:10:36,820
each power separately.
184
00:10:36,820 --> 00:10:38,120
OK, let's do it.
185
00:10:38,120 --> 00:10:46,950
So then f(x)g(x) is, when
I multiply that polynomial
186
00:10:46,950 --> 00:10:53,330
or series by that polynomial,
I get some polynomial in, with
187
00:10:53,330 --> 00:10:56,890
coefficients, oh I
was changed to l,
188
00:10:56,890 --> 00:11:00,640
just to have a
different symbol there.
189
00:11:00,640 --> 00:11:03,270
And what was the formula for l?
190
00:11:03,270 --> 00:11:05,620
For h_l?
191
00:11:05,620 --> 00:11:10,780
What is the coefficient of z^l?
192
00:11:10,780 --> 00:11:15,230
If I multiply that by that, do
you remember the story there?
193
00:11:15,230 --> 00:11:21,100
When I multiply that by that
and I looked for the terms that
194
00:11:21,100 --> 00:11:23,580
gave me z^l?
195
00:11:23,580 --> 00:11:27,350
OK, well that means that this
power times this power is going
196
00:11:27,350 --> 00:11:29,220
to be z^l.
197
00:11:29,220 --> 00:11:33,030
So that the index, do you
remember what happened?
198
00:11:33,030 --> 00:11:38,160
It was a lot of different, just
the way-- This h_l is here.
199
00:11:38,160 --> 00:11:41,750
Here's h_2 or something.
200
00:11:41,750 --> 00:11:43,990
I've got to do an addition.
201
00:11:43,990 --> 00:11:49,730
Because a bunch of c's come in
with different d's, and what's
202
00:11:49,730 --> 00:11:53,820
the deal then? c_k
comes in with which d?
203
00:11:53,820 --> 00:11:56,820
This is the magic number there.
204
00:11:56,820 --> 00:12:04,420
What's the subscript that if I
look at the coefficient of z^l,
205
00:12:04,420 --> 00:12:07,760
I look at each of these.
206
00:12:07,760 --> 00:12:13,240
And then I pick out the one of
these that will give me a z^l.
207
00:12:13,240 --> 00:12:16,690
And which one is it?
208
00:12:16,690 --> 00:12:18,550
d_(l-k).
209
00:12:18,550 --> 00:12:22,690
It's that magic quantity
that the eye spots perfectly.
210
00:12:22,690 --> 00:12:32,850
k and l-k, adding to l simply
because z^k z^(l-k) multiplies
211
00:12:32,850 --> 00:12:35,600
to z^l.
212
00:12:35,600 --> 00:12:36,880
Same thing.
213
00:12:36,880 --> 00:12:37,540
Right?
214
00:12:37,540 --> 00:12:43,930
OK, this is, now I'll
use that notation.
215
00:12:43,930 --> 00:12:47,800
This is c convolved
with the d. c
216
00:12:47,800 --> 00:12:50,640
convolved with d is h.
c convolved with d is h.
217
00:12:50,640 --> 00:12:55,810
So this is the l-th component.
c convolved with d is my symbol.
218
00:12:55,810 --> 00:12:58,450
This h is the convolution.
219
00:12:58,450 --> 00:13:01,660
And that's the convolution rule.
220
00:13:01,660 --> 00:13:04,720
It's just whatever
operation you have
221
00:13:04,720 --> 00:13:06,990
to do to get the right answer.
222
00:13:06,990 --> 00:13:11,460
The right answer
when you multiply.
223
00:13:11,460 --> 00:13:13,540
So that one.
224
00:13:13,540 --> 00:13:16,640
Ready for the discrete case?
225
00:13:16,640 --> 00:13:18,200
The finite case?
226
00:13:18,200 --> 00:13:25,270
The case when, you have
power, when z becomes w.
227
00:13:25,270 --> 00:13:27,040
The discrete case.
228
00:13:27,040 --> 00:13:28,850
The cyclic case, sorry.
229
00:13:28,850 --> 00:13:33,950
Maybe emphasize the cyclic
case, meaning it circles around,
230
00:13:33,950 --> 00:13:39,730
is the case when z becomes this
very special z, on the unit
231
00:13:39,730 --> 00:13:42,390
circle that we know as w.
232
00:13:42,390 --> 00:13:47,510
OK, and we have to say,
and of course it's w_N,
233
00:13:47,510 --> 00:13:54,730
I have to tell you in the cyclic
case how long the cycle is.
234
00:13:54,730 --> 00:13:57,840
So this would be a case.
235
00:13:57,840 --> 00:14:02,590
Watch what you do here
to make this cyclic.
236
00:14:02,590 --> 00:14:09,990
OK, I have three inputs,
so N is three here.
237
00:14:09,990 --> 00:14:13,150
Now I'm going to do the cyclic.
238
00:14:13,150 --> 00:14:16,190
So instead of z's, I
should be putting w's.
239
00:14:16,190 --> 00:14:18,420
I will.
240
00:14:18,420 --> 00:14:23,240
Just to emphasize, it's good to
think of it with the w there,
241
00:14:23,240 --> 00:14:27,510
because the w has this special
property that's critical
242
00:14:27,510 --> 00:14:31,750
to everything, OK so now I'm
going to, I think of this
243
00:14:31,750 --> 00:14:38,230
as 1w^0, 2w^1, and 3 w squared.
244
00:14:38,230 --> 00:14:40,800
I'm thinking of the same
multiplication here,
245
00:14:40,800 --> 00:14:45,570
but now w's.
246
00:14:45,570 --> 00:14:51,540
OK, so I'll end up with 18w^4,
and four was the constant
247
00:14:51,540 --> 00:14:55,700
and 13 w's and so on.
248
00:14:55,700 --> 00:14:59,910
All these numbers.
249
00:14:59,910 --> 00:15:01,940
What's the difference?
250
00:15:01,940 --> 00:15:04,070
Ready for the key point?
251
00:15:04,070 --> 00:15:07,340
Now, what's happened
in this cyclic case?
252
00:15:07,340 --> 00:15:12,010
Well, the difference
is what is w^4?
253
00:15:12,010 --> 00:15:15,740
If we're in the cyclic
case, N is three now,
254
00:15:15,740 --> 00:15:19,420
our guys have length
three, our circle
255
00:15:19,420 --> 00:15:24,270
is, w now is 1/3
of the way around.
256
00:15:24,270 --> 00:15:27,860
So that's my w,
here's my w squared.
257
00:15:27,860 --> 00:15:29,640
Here is my one.
258
00:15:29,640 --> 00:15:32,850
But here it is also w cubed.
259
00:15:32,850 --> 00:15:39,900
So w is the same as w^4,
w^2 is the same as w^5.
260
00:15:39,900 --> 00:15:44,320
So, what's the difference?
261
00:15:44,320 --> 00:15:48,440
What can I do now?
262
00:15:48,440 --> 00:15:54,300
If I'm in this discrete
case, then my inputs
263
00:15:54,300 --> 00:15:58,030
are a vector of length N, three.
264
00:15:58,030 --> 00:16:00,380
A vector of length N, three.
265
00:16:00,380 --> 00:16:06,180
And I want to get out to
a vector of length three.
266
00:16:06,180 --> 00:16:10,180
I'm not happy with that
in the cyclic case,
267
00:16:10,180 --> 00:16:13,920
because I'm not happy with w^4.
268
00:16:13,920 --> 00:16:17,790
So now tell me again that last,
when I do the multiplication
269
00:16:17,790 --> 00:16:21,920
and I just do it, there's
no difference except in how
270
00:16:21,920 --> 00:16:23,710
I write the answer.
271
00:16:23,710 --> 00:16:35,010
18w^4 is the same as? w. w^4 is
the same as w, when N is three.
272
00:16:35,010 --> 00:16:41,290
So that 18w^4 cycles
back in, with this 13.
273
00:16:41,290 --> 00:16:45,650
So now if I do the, can
I show you the symbol?
274
00:16:45,650 --> 00:16:47,730
I'll just do a
little circle there.
275
00:16:47,730 --> 00:16:52,130
To say this is now the
cyclic convolution.
276
00:16:52,130 --> 00:16:54,620
Then this isn't the
answer any more.
277
00:16:54,620 --> 00:16:57,090
The answer now for
cyclic convolution,
278
00:16:57,090 --> 00:16:59,190
I only want three numbers.
279
00:16:59,190 --> 00:17:01,600
And if you can tell me what
those three numbers are,
280
00:17:01,600 --> 00:17:04,250
we've got it.
281
00:17:04,250 --> 00:17:07,240
Move those over a little to
make room for the three numbers.
282
00:17:07,240 --> 00:17:11,940
So there's the answer,
the space for the answer.
283
00:17:11,940 --> 00:17:14,430
What do I write in?
284
00:17:14,430 --> 00:17:19,750
How many, what's
the constant term?
285
00:17:19,750 --> 00:17:21,460
It's 31, right.
286
00:17:21,460 --> 00:17:22,410
It's 31.
287
00:17:22,410 --> 00:17:25,290
Where did 31 come from?
288
00:17:25,290 --> 00:17:31,540
It came from, you could say,
cycling that 27w^3 back with
289
00:17:31,540 --> 00:17:33,000
the four.
290
00:17:33,000 --> 00:17:36,250
Because there's no,
w^3 is the same as one.
291
00:17:36,250 --> 00:17:40,730
So I've gone around the circle
when I come around to w^3.
292
00:17:40,730 --> 00:17:45,820
So that 27 and 4
combined into that 31.
293
00:17:45,820 --> 00:17:47,930
And let's see.
294
00:17:47,930 --> 00:17:50,130
So what multiplications
am I doing here?
295
00:17:50,130 --> 00:17:53,280
One times four gives
me the constant.
296
00:17:53,280 --> 00:18:00,520
Two times six, that's two
w's and six w squareds,
297
00:18:00,520 --> 00:18:03,870
that's 12w^3, that's 12.
298
00:18:03,870 --> 00:18:10,880
And then 3 w squareds
and five w's is 15w^3.
299
00:18:10,880 --> 00:18:13,330
But that's the same as 15.
300
00:18:13,330 --> 00:18:15,330
So that's why we get 31.
301
00:18:15,330 --> 00:18:20,670
We got 4, we've got
12 and we've got 15.
302
00:18:20,670 --> 00:18:25,350
OK, now what's the second
component, the w component
303
00:18:25,350 --> 00:18:32,510
of the cyclic convolution?
304
00:18:32,510 --> 00:18:36,120
Tell me what number do I write
in, in that middle position?
305
00:18:36,120 --> 00:18:39,100
How many w's do I have?
306
00:18:39,100 --> 00:18:41,300
31 again.
307
00:18:41,300 --> 00:18:47,050
This 28, this 18 is
coming back by three.
308
00:18:47,050 --> 00:18:48,920
To 13.
309
00:18:48,920 --> 00:18:53,010
You see, I could have
done that multiplication.
310
00:18:53,010 --> 00:18:57,540
So coming back to 31, am
I going to get another 31?
311
00:18:57,540 --> 00:19:01,030
No, what's the w^2 guy?
312
00:19:01,030 --> 00:19:01,830
28.
313
00:19:01,830 --> 00:19:02,330
Yeah.
314
00:19:02,330 --> 00:19:09,730
28, because there's no w
to the fifth to come back.
315
00:19:09,730 --> 00:19:15,890
So 28 uses three
multiplications.
316
00:19:15,890 --> 00:19:21,320
The 31 there used the
four and these two.
317
00:19:21,320 --> 00:19:24,630
This came back over to here.
318
00:19:24,630 --> 00:19:28,640
And this 31 used,
this 18 came back.
319
00:19:28,640 --> 00:19:30,920
I could have put the 18 here.
320
00:19:30,920 --> 00:19:33,450
You know, I could have
lined it up just three.
321
00:19:33,450 --> 00:19:35,470
So I'll write a formula for it.
322
00:19:35,470 --> 00:19:37,790
So that's the answer.
323
00:19:37,790 --> 00:19:41,610
31, 31, 28.
324
00:19:41,610 --> 00:19:44,740
Could I just suggest a
little check on that?
325
00:19:44,740 --> 00:19:46,460
Just to check on the numbers.
326
00:19:46,460 --> 00:19:52,590
I think that if I add up
these numbers, I get six.
327
00:19:52,590 --> 00:19:57,770
And if I add up those
numbers I get 15.
328
00:19:57,770 --> 00:20:01,180
And if I multiply
that, I get 90.
329
00:20:01,180 --> 00:20:04,410
And if I add those
numbers I get 90, right?
330
00:20:04,410 --> 00:20:08,440
So those add to 90.
331
00:20:08,440 --> 00:20:12,820
So I'm just saying the
miracle check is add these,
332
00:20:12,820 --> 00:20:15,040
multiply by the sum of those.
333
00:20:15,040 --> 00:20:17,720
And you get the sum of those.
334
00:20:17,720 --> 00:20:20,700
Why is that?
335
00:20:20,700 --> 00:20:23,480
Somehow seems right, doesn't it?
336
00:20:23,480 --> 00:20:29,790
Because somehow I've taken
all nine products here, and so
337
00:20:29,790 --> 00:20:33,460
when I add all the results,
I'll have the sum of all nine
338
00:20:33,460 --> 00:20:35,220
of these possible products.
339
00:20:35,220 --> 00:20:37,590
So I'll have 6 times 15.
340
00:20:37,590 --> 00:20:41,310
Actually, here's a
good way to look at it.
341
00:20:41,310 --> 00:20:43,730
In doing this, I just set w=1.
342
00:20:43,730 --> 00:20:48,620
343
00:20:48,620 --> 00:20:51,840
I just set w=1 in
the polynomial.
344
00:20:51,840 --> 00:20:56,220
When I set w to one, this
becomes six, this becomes 15
345
00:20:56,220 --> 00:20:59,540
and the answer becomes
90, when w is one.
346
00:20:59,540 --> 00:21:00,040
Yeah.
347
00:21:00,040 --> 00:21:02,910
So that's another way to see.
348
00:21:02,910 --> 00:21:07,420
And actually, this
multiplication,
349
00:21:07,420 --> 00:21:13,400
the second grade
version, had w, well,
350
00:21:13,400 --> 00:21:19,600
I'm almost going to say
w=10, but not quite that.
351
00:21:19,600 --> 00:21:22,540
Because it's written in
the opposite order, right?
352
00:21:22,540 --> 00:21:25,290
If w was ten, this would be one.
353
00:21:25,290 --> 00:21:28,270
Well, anyway, w can't
be ten, it's got
354
00:21:28,270 --> 00:21:31,580
to stay on the unit circle, so.
355
00:21:31,580 --> 00:21:35,760
So somewhere in the non-cyclic
case, it's something like w=10,
356
00:21:35,760 --> 00:21:38,090
or w=1/10, maybe.
357
00:21:38,090 --> 00:21:39,040
Whatever.
358
00:21:39,040 --> 00:21:40,190
OK.
359
00:21:40,190 --> 00:21:42,670
Could you take the
convolution now,
360
00:21:42,670 --> 00:21:45,270
let me give you just
another example.
361
00:21:45,270 --> 00:21:49,700
Do it mentally.
362
00:21:49,700 --> 00:21:55,560
What's the convolution
of [0, 1, 0, 0],
363
00:21:55,560 --> 00:21:57,990
let me make a little longer.
364
00:21:57,990 --> 00:22:02,200
So I take the, first of all,
the non-cyclic convolution
365
00:22:02,200 --> 00:22:05,810
of [0, 0, 1, 0].
366
00:22:05,810 --> 00:22:07,010
OK.
367
00:22:07,010 --> 00:22:11,780
What's the ordinary-- how
long is the answer now?
368
00:22:11,780 --> 00:22:13,320
This is just practice.
369
00:22:13,320 --> 00:22:15,590
How many components
am I going to have
370
00:22:15,590 --> 00:22:22,650
in the ordinary convolution of
those two guys of length four?
371
00:22:22,650 --> 00:22:24,930
I think it's seven.
372
00:22:24,930 --> 00:22:26,610
I think it'll be seven.
373
00:22:26,610 --> 00:22:34,250
Because we'll have here one, we
have no, we have z^0, z^1, z^2,
374
00:22:34,250 --> 00:22:35,910
z^3.
375
00:22:35,910 --> 00:22:39,450
And here we'll have again the
same, it would go up to z^6,
376
00:22:39,450 --> 00:22:43,020
but remember there's a z^0,
so that's why we have seven.
377
00:22:43,020 --> 00:22:47,440
And what will it be?
378
00:22:47,440 --> 00:22:49,600
What will it be?
379
00:22:49,600 --> 00:22:53,020
I guess, actually, this wasn't
a brilliant example, was it?
380
00:22:53,020 --> 00:22:55,970
But let's finish it.
381
00:22:55,970 --> 00:23:01,330
So I'm just multiplying
z by z squared, so
382
00:23:01,330 --> 00:23:02,650
what do you get for an answer?
383
00:23:02,650 --> 00:23:13,600
I think the one shows
up in the z cubed.
384
00:23:13,600 --> 00:23:14,970
Is that right?
385
00:23:14,970 --> 00:23:22,000
Yeah. z to the first power here,
z squared here, z cubed here,
386
00:23:22,000 --> 00:23:26,490
and we always have to remember
everything in Chapter 4 starts
387
00:23:26,490 --> 00:23:27,640
at zero.
388
00:23:27,640 --> 00:23:30,340
Zero, z^0's the first one.
389
00:23:30,340 --> 00:23:33,030
OK, that's not too
great an example,
390
00:23:33,030 --> 00:23:40,580
because what happens if I do
the circular cyclic convolution?
391
00:23:40,580 --> 00:23:42,500
What would be the
cyclic convolution?
392
00:23:42,500 --> 00:23:47,150
Now I'm expecting
four guys only, right?
393
00:23:47,150 --> 00:23:50,250
The cyclic keeps
the same length.
394
00:23:50,250 --> 00:23:55,150
And what would be the answer?
395
00:23:55,150 --> 00:23:58,330
Well, there's nobody to fold
back, so it would be just
396
00:23:58,330 --> 00:23:59,910
[0, 0, 0, 1].
397
00:23:59,910 --> 00:24:06,210
So let me update this a
little bit with a one here.
398
00:24:06,210 --> 00:24:08,090
OK, just to practice.
399
00:24:08,090 --> 00:24:12,000
So suppose I do
that convolution.
400
00:24:12,000 --> 00:24:15,850
Un-cyclic, first.
401
00:24:15,850 --> 00:24:18,220
What do I change here now?
402
00:24:18,220 --> 00:24:21,780
I've now got a z squared
and I've also got a z cubed,
403
00:24:21,780 --> 00:24:25,170
but I only have a
single one there.
404
00:24:25,170 --> 00:24:27,330
Let's make it a little
more interesting.
405
00:24:27,330 --> 00:24:30,930
OK, make it like so.
406
00:24:30,930 --> 00:24:36,530
Alright, z+z^2 is what
we're looking at, z+z^2 is
407
00:24:36,530 --> 00:24:39,820
multiplying z^2+z^3.
408
00:24:39,820 --> 00:24:43,870
And in the long
form, what do I get?
409
00:24:43,870 --> 00:24:47,920
Let me make space, tell
me what numbers to put in.
410
00:24:47,920 --> 00:24:52,140
If I multiply z+z^2 times
z^2+z^3, I get what?
411
00:24:52,140 --> 00:24:54,420
1z^3, how many z^4?
412
00:24:54,420 --> 00:24:57,070
413
00:24:57,070 --> 00:24:58,220
Two of them.
414
00:24:58,220 --> 00:25:00,240
How many z^5?
415
00:25:00,240 --> 00:25:01,760
One, and nobody there.
416
00:25:01,760 --> 00:25:02,890
OK.
417
00:25:02,890 --> 00:25:09,600
And now the cyclic
version would be what?
418
00:25:09,600 --> 00:25:12,540
What's my answer now
for the cyclic version?
419
00:25:12,540 --> 00:25:15,850
Let me take those out.
420
00:25:15,850 --> 00:25:21,680
So the cyclic version would
bring the two back to the zero.
421
00:25:21,680 --> 00:25:24,320
Would bring that
one back so there.
422
00:25:24,320 --> 00:25:27,440
That zero will still be zero.
423
00:25:27,440 --> 00:25:32,070
And I checked that I
haven't missed anything
424
00:25:32,070 --> 00:25:35,760
by adding those up to get
four, and adding this up
425
00:25:35,760 --> 00:25:37,610
to get two times two.
426
00:25:37,610 --> 00:25:38,660
Yeah.
427
00:25:38,660 --> 00:25:42,700
OK, so that's the rule.
428
00:25:42,700 --> 00:25:47,330
And it's a lot cleaner
to see these answers
429
00:25:47,330 --> 00:25:52,770
than to see this formula.
430
00:25:52,770 --> 00:25:57,390
And I need, actually of course,
now mentioning that formula,
431
00:25:57,390 --> 00:25:59,640
I need a cyclic formula.
432
00:25:59,640 --> 00:26:03,080
So can I write above
it the cyclic formula?
433
00:26:03,080 --> 00:26:12,580
What do I get when I'm,
instead of this sum, which
434
00:26:12,580 --> 00:26:17,390
went from k equal minus
infinity to infinity,
435
00:26:17,390 --> 00:26:25,000
in the cyclic case, h_k is just
going to be a sum from zero
436
00:26:25,000 --> 00:26:33,720
to N-1, and there'll be
a c_k, and a d something.
437
00:26:33,720 --> 00:26:42,620
And now this is the cyclic
case, so I guess this makes us,
438
00:26:42,620 --> 00:26:45,460
I think what our situation
now is we understand
439
00:26:45,460 --> 00:26:47,620
the cyclic case from examples.
440
00:26:47,620 --> 00:26:52,590
And now we just have the job of
how do I put it into algebra.
441
00:26:52,590 --> 00:26:55,270
How do I put it into symbols?
442
00:26:55,270 --> 00:27:02,280
What's the point?
c_k d_n, let's say.
443
00:27:02,280 --> 00:27:08,780
But oh no, I'm
looking for h_l, yeah.
444
00:27:08,780 --> 00:27:10,260
So what's the deal?
445
00:27:10,260 --> 00:27:14,780
Here l was k.
446
00:27:14,780 --> 00:27:17,900
That one is the sum of
that one and that one.
447
00:27:17,900 --> 00:27:21,630
So here, l is the
sum of k and n, but.
448
00:27:21,630 --> 00:27:28,720
What's the but?
449
00:27:28,720 --> 00:27:35,360
I mean somehow I've got some
wraparound to do, right?
450
00:27:35,360 --> 00:27:39,200
When I'm doing the cyclic
multiplication and I'm doing
451
00:27:39,200 --> 00:27:47,890
the wraparound because w^N,
the wraparound comes from that,
452
00:27:47,890 --> 00:27:49,490
right?
453
00:27:49,490 --> 00:27:52,710
That's why I never get as high
as N, because when I get to N
454
00:27:52,710 --> 00:27:55,460
I go back to the zeroth power.
455
00:27:55,460 --> 00:28:02,960
OK, so what's the relation
of k and n and l here?
456
00:28:02,960 --> 00:28:07,440
We just need the right
word to express it.
457
00:28:07,440 --> 00:28:08,740
What's the word?
458
00:28:08,740 --> 00:28:10,770
Mod.
459
00:28:10,770 --> 00:28:17,360
So that's the word I'm
looking for. k+n is l.
460
00:28:17,360 --> 00:28:21,320
With wraparound and
wraparound means,
461
00:28:21,320 --> 00:28:35,590
the nice notation that people
use is mod N. Let's practice.
462
00:28:35,590 --> 00:28:45,670
What is two plus two mod seven?
463
00:28:45,670 --> 00:28:47,400
Four.
464
00:28:47,400 --> 00:28:49,950
Two plus two is
four, even in 18.085.
465
00:28:49,950 --> 00:28:51,360
Right, OK.
466
00:28:51,360 --> 00:29:03,570
But two plus two mod three
is, two plus two mod three is?
467
00:29:03,570 --> 00:29:04,120
One.
468
00:29:04,120 --> 00:29:06,520
Everybody sees it?
469
00:29:06,520 --> 00:29:13,690
I'm taking z^2 times z^2,
z^4, but I'm doing with N=3,
470
00:29:13,690 --> 00:29:18,020
so z^3 is one, so that z^4
is really just z to the first
471
00:29:18,020 --> 00:29:20,080
power.
472
00:29:20,080 --> 00:29:27,300
So this is the
little nifty notation
473
00:29:27,300 --> 00:29:29,450
that says make it cyclic.
474
00:29:29,450 --> 00:29:34,860
Bring it back so that l
only has the values here,
475
00:29:34,860 --> 00:29:38,750
zero up to N-1.
476
00:29:38,750 --> 00:29:41,410
And then stops.
477
00:29:41,410 --> 00:29:45,070
OK.
478
00:29:45,070 --> 00:29:50,830
OK, so we'll have more
practice with examples
479
00:29:50,830 --> 00:29:56,410
when we do some filtering.
480
00:29:56,410 --> 00:29:58,770
Have you got that
fundamental-- So we've
481
00:29:58,770 --> 00:30:03,490
talked about this rule
one here. f times g
482
00:30:03,490 --> 00:30:07,530
goes to those coefficients.
483
00:30:07,530 --> 00:30:10,160
And if it's the
cyclic case then I
484
00:30:10,160 --> 00:30:12,460
put a circle around that star.
485
00:30:12,460 --> 00:30:16,450
And I do the wraparound.
486
00:30:16,450 --> 00:30:23,750
But it's just, it's
the Z-transform,
487
00:30:23,750 --> 00:30:28,590
it's polynomials in z
or polynomials in w,
488
00:30:28,590 --> 00:30:32,360
and when it's polynomials in w,
you use that special property
489
00:30:32,360 --> 00:30:34,700
that w^N is one.
490
00:30:34,700 --> 00:30:38,150
Yeah, OK.
491
00:30:38,150 --> 00:30:43,420
Now, I see I've written
another line, there.
492
00:30:43,420 --> 00:30:46,390
That I could convolve functions.
493
00:30:46,390 --> 00:30:48,520
Let me do a couple
more examples.
494
00:30:48,520 --> 00:30:50,390
Couple of examples.
495
00:30:50,390 --> 00:30:59,200
First, before I go
to that line, OK.
496
00:30:59,200 --> 00:31:05,520
So I'm up to this line.
497
00:31:05,520 --> 00:31:12,640
A couple of examples here.
498
00:31:12,640 --> 00:31:18,840
Let's see, what example
would I want to do?
499
00:31:18,840 --> 00:31:28,270
Let's see, OK, I want to do one
example with a delta function.
500
00:31:28,270 --> 00:31:31,080
One example with
a delta function.
501
00:31:31,080 --> 00:31:32,870
One example with
the delta vector.
502
00:31:32,870 --> 00:31:41,890
Yeah, let me take the
function g(x) identically one.
503
00:31:41,890 --> 00:31:45,820
OK, constant function.
504
00:31:45,820 --> 00:31:48,000
In this rule.
505
00:31:48,000 --> 00:31:50,300
I want to see what
happens with the rule.
506
00:31:50,300 --> 00:32:00,140
OK, then f(x) g(x) is
the same as f(x), right?
507
00:32:00,140 --> 00:32:03,770
Because this function g(x)
is so simple, it's just one.
508
00:32:03,770 --> 00:32:09,520
Now, what about the coefficient?
509
00:32:09,520 --> 00:32:13,480
So I have the coefficients
of c, what are the Fourier
510
00:32:13,480 --> 00:32:15,370
coefficients, what are the d's?
511
00:32:15,370 --> 00:32:18,900
Ah yes, what are the d's?
512
00:32:18,900 --> 00:32:22,840
So I'm testing my rule on a
really really simple case,
513
00:32:22,840 --> 00:32:25,600
g(x) identically one.
514
00:32:25,600 --> 00:32:27,970
What, you have to
tell me, in order
515
00:32:27,970 --> 00:32:30,490
to check the right
side of the rule
516
00:32:30,490 --> 00:32:34,440
you have to tell me the Fourier
coefficients for that very
517
00:32:34,440 --> 00:32:35,530
special function.
518
00:32:35,530 --> 00:32:38,270
What would be the
Fourier coefficients?
519
00:32:38,270 --> 00:32:42,260
If I expand the function
one in a Fourier series,
520
00:32:42,260 --> 00:32:44,000
what do I see?
521
00:32:44,000 --> 00:32:44,890
I see a one.
522
00:32:44,890 --> 00:32:47,240
Yeah, that's it, I see one.
523
00:32:47,240 --> 00:32:54,570
So what are its coefficients?
d_0, right, is one?
524
00:32:54,570 --> 00:33:01,610
And the other d's
are all zero, right?
525
00:33:01,610 --> 00:33:07,580
So my vector of d, my
vector of d's is a whole lot
526
00:33:07,580 --> 00:33:09,600
of zeroes on the negative side.
527
00:33:09,600 --> 00:33:12,780
A one right there in the center,
and then a lot of zeroes.
528
00:33:12,780 --> 00:33:24,770
And now I want to
convolve that with c.
529
00:33:24,770 --> 00:33:27,450
I'm practicing the
convolution rule
530
00:33:27,450 --> 00:33:34,590
on a case that's so simple
it's confusing, right?
531
00:33:34,590 --> 00:33:45,570
I mean, it's a big mess,
this multiplication.
532
00:33:45,570 --> 00:33:47,070
What do I get out of this?
533
00:33:47,070 --> 00:33:58,280
If d is this vector, if d has
this property that d_0 is one
534
00:33:58,280 --> 00:34:03,450
and others are zero,
all others are zero,
535
00:34:03,450 --> 00:34:06,250
so this is my
little example, what
536
00:34:06,250 --> 00:34:10,940
does this sum boil down to?
537
00:34:10,940 --> 00:34:17,900
Well, I only get
something when l=k, right?
538
00:34:17,900 --> 00:34:22,760
I only got something when
l=k, because then I have d_0
539
00:34:22,760 --> 00:34:25,560
and that's the only
d that's around.
540
00:34:25,560 --> 00:34:29,430
So in this sum,
something happens only
541
00:34:29,430 --> 00:34:31,970
when k and l are the same.
542
00:34:31,970 --> 00:34:33,160
And then what happens?
543
00:34:33,160 --> 00:34:37,190
Then I have a one, I
have c_l, and that's h_l,
544
00:34:37,190 --> 00:34:39,280
so that's all I'm
concluding then.
545
00:34:39,280 --> 00:34:43,210
That this h is the same as c.
546
00:34:43,210 --> 00:34:48,210
I'm sorry, it's so dumb.
547
00:34:48,210 --> 00:34:51,570
My point is that in
convolution, this is
548
00:34:51,570 --> 00:34:54,450
the thing that acts like one.
549
00:34:54,450 --> 00:34:56,480
Because in
multiplication, that's
550
00:34:56,480 --> 00:34:59,380
the thing, that's the
function that acts like one.
551
00:34:59,380 --> 00:35:01,890
That's the function that is one.
552
00:35:01,890 --> 00:35:11,640
So this is the one in-- Oh,
would you allow me to do this?
553
00:35:11,640 --> 00:35:18,820
I'm going to create a
matrix with these d's.
554
00:35:18,820 --> 00:35:24,560
There's another way
to see convolution.
555
00:35:24,560 --> 00:35:26,680
Yeah, there's another
way to see convolution
556
00:35:26,680 --> 00:35:28,490
and discrete convolution.
557
00:35:28,490 --> 00:35:31,090
Maybe the discrete
one's the better.
558
00:35:31,090 --> 00:35:36,860
Yeah can you stand one more
way to write the formula?
559
00:35:36,860 --> 00:35:40,720
One more way to write,
now I'm going to do,
560
00:35:40,720 --> 00:35:47,940
I'm going to do
discrete convolution.
561
00:35:47,940 --> 00:35:51,720
Discrete cyclic.
562
00:35:51,720 --> 00:35:53,010
So how am I going to write it?
563
00:35:53,010 --> 00:35:57,210
I'm going to write it by
a matrix multiplication.
564
00:35:57,210 --> 00:35:58,905
Because you know
that in this course
565
00:35:58,905 --> 00:36:01,190
a matrix was going to show up.
566
00:36:01,190 --> 00:36:06,290
So it's going to be a
matrix multiplication.
567
00:36:06,290 --> 00:36:08,780
So I just have to
tell you the matrix,
568
00:36:08,780 --> 00:36:12,300
so this is going
to be some matrix.
569
00:36:12,300 --> 00:36:15,020
Let me take N to be four.
570
00:36:15,020 --> 00:36:20,240
So then I have, you watch.
571
00:36:20,240 --> 00:36:32,080
So I have four d's, and
the output is the four h's.
572
00:36:32,080 --> 00:36:37,350
And the rule I'm
following is this rule.
573
00:36:37,350 --> 00:36:43,660
Is this, the same old rule
but with the cyclic part.
574
00:36:43,660 --> 00:36:47,070
And now I want to show you the
matrix that'll just do this.
575
00:36:47,070 --> 00:36:54,120
Look, I've put the c's
in the first column.
576
00:36:54,120 --> 00:37:00,340
And then I go, yeah,
here's another.
577
00:37:00,340 --> 00:37:07,380
So it's a cyclic matrix.
578
00:37:07,380 --> 00:37:09,230
So let me finish it up.
579
00:37:09,230 --> 00:37:13,240
It's going to be four by
four, it's going to be cyclic.
580
00:37:13,240 --> 00:37:18,340
So I have a c_0, c_0,
c_0, c_0 on the diagonal.
581
00:37:18,340 --> 00:37:21,315
That's fine, that's
because z to the zero
582
00:37:21,315 --> 00:37:24,740
is multiplying all the d's
and leaving them in place.
583
00:37:24,740 --> 00:37:28,200
And then I have
c_1's, and then I
584
00:37:28,200 --> 00:37:31,520
think I come around
again here for a c_1.
585
00:37:31,520 --> 00:37:38,750
And I have c_2's, see
where see c_3, c_2.
586
00:37:38,750 --> 00:37:43,510
And I come around again,
to a c_2 and a c_2.
587
00:37:43,510 --> 00:37:49,490
And c_3 comes around to
a c_3, a c_3 and a c_3.
588
00:37:49,490 --> 00:37:59,400
Well, can you, I hope you can
see, this is cyclic matrix.
589
00:37:59,400 --> 00:38:05,150
It's only got one, it
starts with a vector c,
590
00:38:05,150 --> 00:38:10,640
and those are on the diagonal
and the diagonals wrap around.
591
00:38:10,640 --> 00:38:12,350
That's the other
word that you often
592
00:38:12,350 --> 00:38:16,140
see when you see the
word cyclic, wraparound.
593
00:38:16,140 --> 00:38:23,760
It's because you
think of a circle.
594
00:38:23,760 --> 00:38:25,640
If you go a second
time around, it's
595
00:38:25,640 --> 00:38:28,530
wrapped around the first time.
596
00:38:28,530 --> 00:38:32,000
OK, just can you look and see
that this is the right formula
597
00:38:32,000 --> 00:38:36,300
for h_0? h_0 is c_0*d_0.
598
00:38:36,300 --> 00:38:37,950
Where does that come from?
599
00:38:37,950 --> 00:38:43,230
Remember, h_0 is the coefficient
of z^0 in the answer.
600
00:38:43,230 --> 00:38:47,980
So it comes from c_0*d_0 to
the zeroth power in the input.
601
00:38:47,980 --> 00:38:49,930
And then why is
there is a c_3*d_1?
602
00:38:49,930 --> 00:38:52,960
603
00:38:52,960 --> 00:38:58,450
Why is there a c_3*d_1, and then
a c_2*d_2 and then a c_1*d_3
604
00:38:58,450 --> 00:39:02,630
all piling up into h_0?
605
00:39:02,630 --> 00:39:04,380
Tell me now, why
is there a c_3*d_1?
606
00:39:04,380 --> 00:39:07,800
607
00:39:07,800 --> 00:39:10,370
Because we're doing mod
four is one way to say it.
608
00:39:10,370 --> 00:39:12,840
Three and one add to four.
609
00:39:12,840 --> 00:39:20,750
Because c_3 is the w cubed
guy, and d_1 is the coefficient
610
00:39:20,750 --> 00:39:30,790
of w^1 and w^3 times w^1
piles back into the constant.
611
00:39:30,790 --> 00:39:33,840
And you see the
pattern of that matrix?
612
00:39:33,840 --> 00:39:38,420
So these matrices
are very important.
613
00:39:38,420 --> 00:39:40,620
So they circle around.
614
00:39:40,620 --> 00:39:45,570
Oh, we've actually met
a matrix of this type,
615
00:39:45,570 --> 00:39:49,920
the first day of 18.085.
616
00:39:49,920 --> 00:39:51,160
What was that matrix?
617
00:39:51,160 --> 00:39:53,760
It was one of our
four great matrices.
618
00:39:53,760 --> 00:39:56,650
And now here it is back again.
619
00:39:56,650 --> 00:40:00,700
Which one was it?
620
00:40:00,700 --> 00:40:03,430
Well, you remember
the letter for it.
621
00:40:03,430 --> 00:40:05,170
Which isn't going to change.
622
00:40:05,170 --> 00:40:11,170
And do you remember
the particular matrix?
623
00:40:11,170 --> 00:40:14,320
Well, everybody does
remember that matrix, right?
624
00:40:14,320 --> 00:40:19,510
Twos were on the diagonal,
minus ones were on the diagonal,
625
00:40:19,510 --> 00:40:22,180
and the diagonal
curve continued.
626
00:40:22,180 --> 00:40:26,510
Minus one was on this diagonal
and that continued and zeroes
627
00:40:26,510 --> 00:40:28,120
was on this diagonal.
628
00:40:28,120 --> 00:40:35,610
So this is cyclic convolution,
the circulant matrix,
629
00:40:35,610 --> 00:40:42,050
cyclic convolution
by c, what's the c
630
00:40:42,050 --> 00:40:46,920
that produces that
convolution matrix?
631
00:40:46,920 --> 00:40:50,720
It's just, it's got--
well there it is.
632
00:40:50,720 --> 00:40:56,380
The first column is it.
633
00:40:56,380 --> 00:40:57,350
Right, right.
634
00:40:57,350 --> 00:41:09,530
And somehow I would say
that that's an even vector.
635
00:41:09,530 --> 00:41:12,190
It's sort of, I
associate it with cosine.
636
00:41:12,190 --> 00:41:13,300
It's an even vector.
637
00:41:13,300 --> 00:41:18,810
Here is the zero term, and
then these are the same,
638
00:41:18,810 --> 00:41:21,030
not to worry about that part.
639
00:41:21,030 --> 00:41:25,210
Do you see that we've
seen that matrix before?
640
00:41:25,210 --> 00:41:28,440
And the cyclic
convolution means you
641
00:41:28,440 --> 00:41:32,400
take its second
differences, of course.
642
00:41:32,400 --> 00:41:37,750
We're taking second differences,
but everything in our world
643
00:41:37,750 --> 00:41:40,260
is cyclic.
644
00:41:40,260 --> 00:41:47,210
So the result, the x_4 is x_0.
645
00:41:47,210 --> 00:41:49,990
So we're taking second
differences -- well,
646
00:41:49,990 --> 00:41:55,990
maybe I should say d -- we're
taking second differences d_i,
647
00:41:55,990 --> 00:42:02,260
2d_i's, minus d_(i-1)
minus d_(i+1).
648
00:42:02,260 --> 00:42:05,720
I don't know if this is--
So there's a minus one,
649
00:42:05,720 --> 00:42:07,070
two, minus one.
650
00:42:07,070 --> 00:42:11,760
And we're cycling around
so that d_0 is d_4.
651
00:42:11,760 --> 00:42:20,180
And d_1 is d_5, and
d_(-1) is d_3, whatever.
652
00:42:20,180 --> 00:42:25,140
OK, I'm just reminding you,
we've seen these before.
653
00:42:25,140 --> 00:42:33,280
OK, so this is another way
to remember the formula.
654
00:42:33,280 --> 00:42:37,950
OK now, can I ask you
a practical question?
655
00:42:37,950 --> 00:42:39,910
A practical question.
656
00:42:39,910 --> 00:42:47,610
Let me bring back this
second grade multiplication.
657
00:42:47,610 --> 00:42:49,810
Well, I have a granddaughter
named Elizabeth,
658
00:42:49,810 --> 00:42:51,230
I'll have to admit
I didn't think
659
00:42:51,230 --> 00:42:52,590
about mentioning Elizabeth.
660
00:42:52,590 --> 00:42:54,270
She's six.
661
00:42:54,270 --> 00:42:59,710
And she delights in sending
me long multiplications.
662
00:42:59,710 --> 00:43:02,570
I mean, really long.
663
00:43:02,570 --> 00:43:04,570
And then every time I
talk to her on the phone,
664
00:43:04,570 --> 00:43:06,330
she says have you
done that one yet?
665
00:43:06,330 --> 00:43:07,880
And I say, I'm working on it.
666
00:43:07,880 --> 00:43:11,340
I've got MATLAB at work.
667
00:43:11,340 --> 00:43:15,260
Because they're ridiculous
and I haven't figured out
668
00:43:15,260 --> 00:43:15,970
how to tell her.
669
00:43:15,970 --> 00:43:20,230
I mean, she just
writes page after page.
670
00:43:20,230 --> 00:43:23,240
Times 100, plus
three, minus seven,
671
00:43:23,240 --> 00:43:25,510
just whatever she things of.
672
00:43:25,510 --> 00:43:31,130
OK, now I need help from
the convolution rule. here,
673
00:43:31,130 --> 00:43:31,960
actually.
674
00:43:31,960 --> 00:43:34,360
So let's suppose that
Elizabeth has given me
675
00:43:34,360 --> 00:43:38,070
a multiplication in
which I have a thousand
676
00:43:38,070 --> 00:43:41,470
digits times a thousand, right.
677
00:43:41,470 --> 00:43:44,880
Which Mathematica is prepared
to do exactly, right?
678
00:43:44,880 --> 00:43:51,100
MATLAB will mess up, but
Mathematica and Maple
679
00:43:51,100 --> 00:43:55,360
and symbolic packages will
do exact computations.
680
00:43:55,360 --> 00:43:59,770
So what would be the right way,
well let may make it 1,024.
681
00:43:59,770 --> 00:44:08,220
1,024 digits times 1,024 digits.
682
00:44:08,220 --> 00:44:11,260
Let's do the cyclic
version first.
683
00:44:11,260 --> 00:44:13,340
Elizabeth doesn't
know about cyclic.
684
00:44:13,340 --> 00:44:14,740
Maybe I could teach her that.
685
00:44:14,740 --> 00:44:18,650
That'll keep her busy while
I'm doing the multiplication.
686
00:44:18,650 --> 00:44:23,040
OK, right.
687
00:44:23,040 --> 00:44:25,060
Only, her older brother
would explain it
688
00:44:25,060 --> 00:44:26,320
to her, that's the trouble.
689
00:44:26,320 --> 00:44:30,080
OK, so how am I going
to do, or how are you
690
00:44:30,080 --> 00:44:32,810
going to do on the
quiz, multiplication
691
00:44:32,810 --> 00:44:37,070
of a 1,024 digits times 1,024?
692
00:44:37,070 --> 00:44:39,880
And I'll make it easy
by making it cyclic,
693
00:44:39,880 --> 00:44:43,310
so I just want 1,024
digits in the answer.
694
00:44:43,310 --> 00:44:45,420
OK.
695
00:44:45,420 --> 00:44:48,390
How would you do it?
696
00:44:48,390 --> 00:44:55,930
Well, before today, you would
have just multiplied, right?
697
00:44:55,930 --> 00:45:00,730
You would have written down
1,024, two lines of 1,024,
698
00:45:00,730 --> 00:45:03,490
done an addition.
699
00:45:03,490 --> 00:45:08,090
And you would have had a
million multiplications to do.
700
00:45:08,090 --> 00:45:10,360
But how would you do it now?
701
00:45:10,360 --> 00:45:14,680
Apart from giving
it to Mathematica.
702
00:45:14,680 --> 00:45:17,760
What's a faster way to do it?
703
00:45:17,760 --> 00:45:21,450
What's a faster way
to do a convolution?
704
00:45:21,450 --> 00:45:23,460
The fast way to do
a convolution is
705
00:45:23,460 --> 00:45:27,300
to use the convolution
rule, go this way.
706
00:45:27,300 --> 00:45:32,000
So take these numbers,
these 1,024 numbers,
707
00:45:32,000 --> 00:45:35,930
in c and these
1,024 numbers in d,
708
00:45:35,930 --> 00:45:40,250
and, well what do I have to do?
709
00:45:40,250 --> 00:45:41,820
I want to use the
convolution rule,
710
00:45:41,820 --> 00:45:47,190
because multiplying is fast.
711
00:45:47,190 --> 00:45:50,200
Now I've got functions.
712
00:45:50,200 --> 00:45:52,110
But I'm in the cyclic case.
713
00:45:52,110 --> 00:45:58,520
So I'm in the cyclic
case, so what should I do?
714
00:45:58,520 --> 00:46:03,020
How can I change this
to be the cyclic case?
715
00:46:03,020 --> 00:46:08,830
This is like f_j g_j.
716
00:46:08,830 --> 00:46:15,690
So multiplication of components
of things in function space
717
00:46:15,690 --> 00:46:18,170
is convolution of coefficients.
718
00:46:18,170 --> 00:46:22,130
So now, this is the cyclic.
719
00:46:22,130 --> 00:46:25,820
So let me make it cyclic.
720
00:46:25,820 --> 00:46:28,320
So again, what's your problem?
721
00:46:28,320 --> 00:46:31,740
The problem is to do this
cyclic multiplication.
722
00:46:31,740 --> 00:46:33,440
What's the idea?
723
00:46:33,440 --> 00:46:41,270
The idea is to transform c back
to f, to transform d back to g.
724
00:46:41,270 --> 00:46:43,510
Do the multiplications,
now I have
725
00:46:43,510 --> 00:46:45,610
only 1,024 multiplications.
726
00:46:45,610 --> 00:46:47,690
Not 1,024 squared.
727
00:46:47,690 --> 00:46:49,350
That's the point.
728
00:46:49,350 --> 00:46:52,710
And if I do this
directly, I've got 1,024
729
00:46:52,710 --> 00:46:55,380
squared multiplications to do.
730
00:46:55,380 --> 00:46:57,230
Much better.
731
00:46:57,230 --> 00:47:01,160
Transform back to
here, do just 1,024--
732
00:47:01,160 --> 00:47:04,430
what's the MATLAB command for
that, when you're multiplying
733
00:47:04,430 --> 00:47:06,630
each component by itself?
734
00:47:06,630 --> 00:47:09,730
It's not the dot
product, notice.
735
00:47:09,730 --> 00:47:13,300
It's not the dot product
because I'm not summing.
736
00:47:13,300 --> 00:47:14,920
Do you know the
MATLAB command, if I
737
00:47:14,920 --> 00:47:18,950
have a sequence of numbers
of numbers, a vector
738
00:47:18,950 --> 00:47:25,520
f of length 1,024, and I
want to get that result?
739
00:47:25,520 --> 00:47:26,940
What's the result?
740
00:47:26,940 --> 00:47:32,450
It's a vector of length 1,024
that takes each f times its g.
741
00:47:32,450 --> 00:47:34,600
But doesn't do any adds.
742
00:47:34,600 --> 00:47:36,120
That's what's there.
743
00:47:36,120 --> 00:47:38,880
What's the MATLAB
command for that?
744
00:47:38,880 --> 00:47:40,370
Dot, yeah.
745
00:47:40,370 --> 00:47:42,620
Dot star, right.
746
00:47:42,620 --> 00:47:45,680
So that dot says
component by component.
747
00:47:45,680 --> 00:47:48,980
OK, so what's the plan here?
748
00:47:48,980 --> 00:47:53,100
I do c's back to f.
749
00:47:53,100 --> 00:48:00,740
By the Fourier matrix. d back
to g, by the Fourier matrix,
750
00:48:00,740 --> 00:48:04,470
then I do a very
quick multiplication.
751
00:48:04,470 --> 00:48:05,520
And then what?
752
00:48:05,520 --> 00:48:07,440
Then I mustn't forget.
753
00:48:07,440 --> 00:48:11,830
That I'm in frequency space,
and what do I have to do?
754
00:48:11,830 --> 00:48:16,680
I've got to get back
into coefficient space.
755
00:48:16,680 --> 00:48:24,160
So I do the inverse transform
of-- Here's the formula, then.
756
00:48:24,160 --> 00:48:32,550
I'm doing the inverse transform
of, so the transform of c
757
00:48:32,550 --> 00:48:39,930
dot star, the transform of d.
758
00:48:39,930 --> 00:48:44,690
To get c, d.
759
00:48:44,690 --> 00:48:47,560
Is that right?
760
00:48:47,560 --> 00:48:51,630
So I took c, and I got
back into the function.
761
00:48:51,630 --> 00:48:54,210
I took d, and got
back to its function,
762
00:48:54,210 --> 00:48:55,920
with the Fourier matrix.
763
00:48:55,920 --> 00:48:59,740
OK, I'm in the Fourier
and now I'm in this space.
764
00:48:59,740 --> 00:49:03,000
I've added up coefficients
to get in this space.
765
00:49:03,000 --> 00:49:06,050
Now I do the dot
star, the fast one.
766
00:49:06,050 --> 00:49:08,390
And then I transform back.
767
00:49:08,390 --> 00:49:11,850
So why is that faster?
768
00:49:11,850 --> 00:49:14,960
Than just doing it?
769
00:49:14,960 --> 00:49:20,850
Because what's the
cost of F times c?
770
00:49:20,850 --> 00:49:23,180
And how am I going to do that?
771
00:49:23,180 --> 00:49:25,730
I'm going to do with the fast
Fourier transform, right.
772
00:49:25,730 --> 00:49:26,650
That's the point.
773
00:49:26,650 --> 00:49:32,300
I can multiply by F, or
by F inverse, faster.
774
00:49:32,300 --> 00:49:36,120
So I have three of
these transforms.
775
00:49:36,120 --> 00:49:39,800
I've got to get two guys
into the other space,
776
00:49:39,800 --> 00:49:42,480
and the answer back out.
777
00:49:42,480 --> 00:49:45,460
So I have sort of
three of these N
778
00:49:45,460 --> 00:49:53,190
log N's but that will
easily beat N squared.
779
00:49:53,190 --> 00:49:53,690
Right?
780
00:49:53,690 --> 00:50:01,560
So if you have a
convolution to do,
781
00:50:01,560 --> 00:50:05,950
and it's possible to do this,
get into the other space
782
00:50:05,950 --> 00:50:09,380
where it's just an element
by element multiplication.
783
00:50:09,380 --> 00:50:11,710
And that would apply
in either direction.
784
00:50:11,710 --> 00:50:13,910
Because the rule goes both ways.
785
00:50:13,910 --> 00:50:17,240
If I have this
convolution to do,
786
00:50:17,240 --> 00:50:19,980
I would find the
coefficients here,
787
00:50:19,980 --> 00:50:24,200
the c's, the coefficients
of d of the g's.
788
00:50:24,200 --> 00:50:27,470
I would do this one
by one multiplication,
789
00:50:27,470 --> 00:50:31,200
and then I have the
Fourier coefficients
790
00:50:31,200 --> 00:50:32,860
of the convolution.
791
00:50:32,860 --> 00:50:35,770
Right, OK?
792
00:50:35,770 --> 00:50:37,110
Do I have a moment?
793
00:50:37,110 --> 00:50:37,870
Well, hardly.
794
00:50:37,870 --> 00:50:39,830
Just, can I write
down what the formula
795
00:50:39,830 --> 00:50:44,380
for a convolution of two
functions would look like?
796
00:50:44,380 --> 00:50:48,380
Sorry, f(x) convolved with g(x).
797
00:50:48,380 --> 00:50:50,610
Let me make it cyclic.
798
00:50:50,610 --> 00:50:53,300
Just to see what
it would look like.
799
00:50:53,300 --> 00:50:56,140
What am I expecting
for that convolution?
800
00:50:56,140 --> 00:50:58,570
I'm expecting a function,
and somehow there's
801
00:50:58,570 --> 00:51:02,450
going to be an integral instead
of a sum, where I had sums,
802
00:51:02,450 --> 00:51:04,140
but now I have integrals.
803
00:51:04,140 --> 00:51:05,890
And here's the point.
804
00:51:05,890 --> 00:51:13,000
I'll have f(t) times g(x-t) dt.
805
00:51:13,000 --> 00:51:16,180
806
00:51:16,180 --> 00:51:20,160
All I'm asking you to look at
is the fact that the way here I
807
00:51:20,160 --> 00:51:23,480
had k, and l-k.
808
00:51:23,480 --> 00:51:28,190
For functions, your eye
sees that right away as a t,
809
00:51:28,190 --> 00:51:30,420
and an (x-t), dt.
810
00:51:30,420 --> 00:51:33,290
These add to the answer.
811
00:51:33,290 --> 00:51:36,880
These add to the result, that x.
812
00:51:36,880 --> 00:51:39,670
That would be the
cyclical one, yeah.
813
00:51:39,670 --> 00:51:41,930
So I could go
zero-- These are all
814
00:51:41,930 --> 00:51:46,780
periodic functions, so all
2pi periods are the same.
815
00:51:46,780 --> 00:51:50,250
The book will do that properly.
816
00:51:50,250 --> 00:51:54,440
OK, we've got the filtering
to discuss on Monday.
817
00:51:54,440 --> 00:51:59,030
You can see that this
convolution stuff just
818
00:51:59,030 --> 00:52:03,220
takes a little new thinking,
but it comes out nicely.