1
00:00:00 --> 00:00:01
 
 

2
00:00:01 --> 00:00:03
The following content is
provided under a Creative

3
00:00:03 --> 00:00:03
Commons license.
 

4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:10
offer high-quality educational
resources for free.

6
00:00:10 --> 00:00:12
To make a donation, or to view
additional materials from

7
00:00:12 --> 00:00:17
hundreds of MIT courses, visit
MIT OpenCourseWare

8
00:00:17 --> 00:00:20
at ocw.mit.edu.
 

9
00:00:20 --> 00:00:22
PROFESSOR STRANG: So,
thanks for coming today.

10
00:00:22 --> 00:00:30
This is a key lecture in the
application of Fourier,

11
00:00:30 --> 00:00:33
you could say.
 

12
00:00:33 --> 00:00:35
So convolution is the big word.
 

13
00:00:35 --> 00:00:40
And a major application of
convolution is filtering

14
00:00:40 --> 00:00:43
signal processing.
 

15
00:00:43 --> 00:00:46
So we'll develop
that application.

16
00:00:46 --> 00:00:48
But it's nothing
but convolution.

17
00:00:48 --> 00:00:53
So the key idea is these
convolution rules,

18
00:00:53 --> 00:00:55
where they come from.
 

19
00:00:55 --> 00:00:59
And what these new symbols,
that's the symbol for the

20
00:00:59 --> 00:01:03
convolution of two functions.
 

21
00:01:03 --> 00:01:05
They could be functions here.
 

22
00:01:05 --> 00:01:11
Here they're long vectors
of coefficients, and so

23
00:01:11 --> 00:01:14
these are the rules.
 

24
00:01:14 --> 00:01:16
So it's just a little
bit of algebra.

25
00:01:16 --> 00:01:21
But it just is so central
to all this subject.

26
00:01:21 --> 00:01:25
Signal processing is
certainly the most important

27
00:01:25 --> 00:01:27
little thing to know.
 

28
00:01:27 --> 00:01:29
That if I multiply two
functions, so that's where

29
00:01:29 --> 00:01:31
we started last time.
 

30
00:01:31 --> 00:01:36
If I have a function f with
Fourier coefficient c, and a

31
00:01:36 --> 00:01:44
function g with coefficients
d, then, oh, wrong.

32
00:01:44 --> 00:01:48
I convolve the
coefficients, right.

33
00:01:48 --> 00:01:53
So f has coefficient c,
g has coefficient d.

34
00:01:53 --> 00:01:57
So if I multiply the functions
I definitely do not just

35
00:01:57 --> 00:02:00
multiply each c times the d.
 

36
00:02:00 --> 00:02:03
I do this convolution
operation, which we

37
00:02:03 --> 00:02:04
have to remember.
 

38
00:02:04 --> 00:02:07
That's our main first
step is to remember

39
00:02:07 --> 00:02:09
what that was about.
 

40
00:02:09 --> 00:02:13
And then this is the
other direction.

41
00:02:13 --> 00:02:15
So we didn't see this before.
 

42
00:02:15 --> 00:02:20
That I mean, there's always
this fantastic symmetry

43
00:02:20 --> 00:02:26
between physical space
and frequency space.

44
00:02:26 --> 00:02:30
And convolution in one is
multiplication in the other.

45
00:02:30 --> 00:02:33
It's so easy to remember.
 

46
00:02:33 --> 00:02:36
If I multiply in one
space, I do a convolution

47
00:02:36 --> 00:02:38
in the other space.
 

48
00:02:38 --> 00:02:42
If I do a convolution
of functions, I do a

49
00:02:42 --> 00:02:45
multiplication of coefficients.
 

50
00:02:45 --> 00:02:49
So that's the rule to know.
 

51
00:02:49 --> 00:02:53
And now if we expand on
it, by sort of seeing

52
00:02:53 --> 00:02:54
again what it means.
 

53
00:02:54 --> 00:02:59
So let me do a quick repeat
of this first step to

54
00:02:59 --> 00:03:04
remember what this symbol,
convolution symbol, means.

55
00:03:04 --> 00:03:07
And then another
thing I have to do.

56
00:03:07 --> 00:03:11
Here, I'm talking about the
infinite case, functions

57
00:03:11 --> 00:03:15
and with a whole infinite
sequence of coefficients.

58
00:03:15 --> 00:03:20
I've got to do the
cyclic case, too.

59
00:03:20 --> 00:03:23
Which goes with the
discrete transform.

60
00:03:23 --> 00:03:26
OK, but let's start with
the infinite case as

61
00:03:26 --> 00:03:29
we did last time.
 

62
00:03:29 --> 00:03:31
Maybe, here's something.
 

63
00:03:31 --> 00:03:32
Here's a suggestion.
 

64
00:03:32 --> 00:03:34
We had f(x).
 

65
00:03:36 --> 00:03:39
We started with
f(x), as the sum.

66
00:03:39 --> 00:03:43
And remember to have nice
formulas, we're doing the

67
00:03:43 --> 00:03:45
complex version. c_k*e^(ikx).
 

68
00:03:47 --> 00:03:49
Let me suggest something.
 

69
00:03:49 --> 00:03:51
Let me write z for e^(ikx).
 

70
00:03:52 --> 00:03:56
So I'm going to just write
that at the sum of c_k*z^k.

71
00:03:57 --> 00:04:01
z to So z is e^(ix).
 

72
00:04:01 --> 00:04:05
 
 

73
00:04:05 --> 00:04:09
This is like a good
point to make anyway.

74
00:04:09 --> 00:04:14
When we have this e^(ikx), it's
natural to think of that as a

75
00:04:14 --> 00:04:18
complex number on
the unit circle.

76
00:04:18 --> 00:04:19
Right?
 

77
00:04:19 --> 00:04:21
That's always the message,
e to the i real is

78
00:04:21 --> 00:04:23
on the unit circle.
 

79
00:04:23 --> 00:04:25
Absolute value one.
 

80
00:04:25 --> 00:04:29
And this is great for
periodic function.

81
00:04:29 --> 00:04:33
Because if these functions have
period 2pi, and if we think of

82
00:04:33 --> 00:04:40
our function as being on the
circle, it obviously

83
00:04:40 --> 00:04:41
has period 2pi.
 

84
00:04:42 --> 00:04:44
I mean, the picture says, yes.
 

85
00:04:44 --> 00:04:47
If I go 2pi, I come back.
 

86
00:04:47 --> 00:04:53
So x is the angle, right? x is
the in this picture, a little

87
00:04:53 --> 00:04:59
bit unusual maybe, x there
would be the angle.

88
00:04:59 --> 00:05:03
And it's just a little
bit easier to write.

89
00:05:03 --> 00:05:06
And maybe it even has an
official name, the z transform.

90
00:05:06 --> 00:05:07
How do you like that?
 

91
00:05:07 --> 00:05:17
You learn a transform just, in
fourteen seconds. z transform.

92
00:05:17 --> 00:05:21
It'll make it easier to
multiply by g, so g is going

93
00:05:21 --> 00:05:23
to be the sum of d_k*z^k.
 

94
00:05:26 --> 00:05:30
So just think of these as
long, well I'm tempted to

95
00:05:30 --> 00:05:32
say, long polynomials.
 

96
00:05:32 --> 00:05:37
I mean, very long, because
they can be infinite series.

97
00:05:37 --> 00:05:40
Infinite negative powers
and positive powers.

98
00:05:40 --> 00:05:44
But just think of it as a
bunch of powers of z times

99
00:05:44 --> 00:05:46
a bunch of powers of z.
 

100
00:05:46 --> 00:05:51
And if you multiply a couple
of polynomials, you're

101
00:05:51 --> 00:05:54
doing convolution.
 

102
00:05:54 --> 00:05:56
I guess my message is, you've
been doing convolution

103
00:05:56 --> 00:05:59
since the second grade.
 

104
00:05:59 --> 00:06:01
That's the real message.
 

105
00:06:01 --> 00:06:04
Here, let me show you.
 

106
00:06:04 --> 00:06:06
This is convolution, too.
 

107
00:06:06 --> 00:06:14
Suppose I have to multiply
a 123 times 456?

108
00:06:14 --> 00:06:17
OK, so what do you do?
 

109
00:06:17 --> 00:06:22
Remember back, it's a long
way back but we can do this.

110
00:06:22 --> 00:06:25
One, two, three times
four, five, six.

111
00:06:25 --> 00:06:30
So I multiply the three, oh
there's a little point.

112
00:06:30 --> 00:06:33
Where that second-grade
teacher's going to panic.

113
00:06:33 --> 00:06:37
I'm going to write that as 18.
 

114
00:06:37 --> 00:06:41
And that's 15, and that's 12.
 

115
00:06:41 --> 00:06:43
Sorry about that, yeah.
 

116
00:06:43 --> 00:06:48
And 12, ten and eight, right?
 

117
00:06:48 --> 00:06:50
Four, or five and six.
 

118
00:06:50 --> 00:06:54
So you see the nine
multiplications that

119
00:06:54 --> 00:06:55
you have to do?
 

120
00:06:55 --> 00:06:58
Nine multiplications,
three times three.

121
00:06:58 --> 00:07:05
OK, right now imagine
those were, they could

122
00:07:05 --> 00:07:06
have been longer.
 

123
00:07:06 --> 00:07:13
But they were finite length
filters, we could say.

124
00:07:13 --> 00:07:17
And now, what does
convolution do?

125
00:07:17 --> 00:07:19
Just what you did
in multiplication.

126
00:07:19 --> 00:07:23
When I add 12, 10 and
6, what am I doing?

127
00:07:23 --> 00:07:26
I'm putting together the three
times the four, the two

128
00:07:26 --> 00:07:29
times the five, and
the one times the six.

129
00:07:29 --> 00:07:31
That's what convolution,
those are the things that

130
00:07:31 --> 00:07:33
convolution puts together.
 

131
00:07:33 --> 00:07:40
So we got an 18, a 27, 28.
 

132
00:07:40 --> 00:07:45
13, and four.
 

133
00:07:45 --> 00:07:49
So I guess I'm saying that,
alright, here's what

134
00:07:49 --> 00:07:51
I really want to say.
 

135
00:07:51 --> 00:07:57
I want to say that the
convolution of those, with

136
00:07:57 --> 00:08:04
these, four, five, six, is
this sequence here - oh.

137
00:08:04 --> 00:08:04
Yeah, that's right.
 

138
00:08:04 --> 00:08:09
Four, 13, 28.
 

139
00:08:09 --> 00:08:13
27, and 18.
 

140
00:08:13 --> 00:08:15
If you just look at that.
 

141
00:08:15 --> 00:08:17
Where did that 13 come from?
 

142
00:08:17 --> 00:08:21
Let's just remember, where
does that 13 come from?

143
00:08:21 --> 00:08:23
That came from, this was z^0.
 

144
00:08:25 --> 00:08:29
This 13 is 13z to the
first power, right?

145
00:08:29 --> 00:08:33
We're just checking
all the powers here.

146
00:08:33 --> 00:08:35
There's 13z to the first power.
 

147
00:08:35 --> 00:08:37
Where do we get a first power?
 

148
00:08:37 --> 00:08:40
We get a z^0 times 5z^1.
 

149
00:08:41 --> 00:08:42
So that's 5z^1.
 

150
00:08:43 --> 00:08:47
And we also have
2z^1, times 4z^0.

151
00:08:49 --> 00:08:50
Right?
 

152
00:08:50 --> 00:08:52
Two times four gave the eight.
 

153
00:08:52 --> 00:08:54
So the eight from there
and the five from

154
00:08:54 --> 00:08:56
there produce that 13.
 

155
00:08:56 --> 00:09:00
And that's just what you
did in multiplication.

156
00:09:00 --> 00:09:01
Right?
 

157
00:09:01 --> 00:09:06
So that's multiplication
of two series.

158
00:09:06 --> 00:09:08
That's not cyclic.
 

159
00:09:08 --> 00:09:11
This is definitely not yet
cyclic, and but we'll make

160
00:09:11 --> 00:09:13
it cyclic in a minute.
 

161
00:09:13 --> 00:09:16
This is the infinite
one, except that we

162
00:09:16 --> 00:09:20
had all zeroes beyond.
 

163
00:09:20 --> 00:09:24
OK, so that if you in
non-cyclic convolution like

164
00:09:24 --> 00:09:30
this, if I have length m
and length n, then I get

165
00:09:30 --> 00:09:32
length m+n, maybe m+n-1.
 

166
00:09:34 --> 00:09:37
OK, so that's convolution.
 

167
00:09:37 --> 00:09:40
Without carrying numbers.
 

168
00:09:40 --> 00:09:42
Without doing it right.
 

169
00:09:42 --> 00:09:46
OK, so and what does
that correspond to?

170
00:09:46 --> 00:09:48
Let me just, so you
see it every way.

171
00:09:48 --> 00:09:55
That corresponds to 1+2z+3z^2,
multiplying 4+5z+6z^2.

172
00:10:00 --> 00:10:02
And it gave, that's times.
 

173
00:10:02 --> 00:10:08
And it gave this
thing up to 18z^4.

174
00:10:10 --> 00:10:12
Just, exactly the
multiplication that

175
00:10:12 --> 00:10:14
you've always done.
 

176
00:10:14 --> 00:10:17
OK, so that's the idea.
 

177
00:10:17 --> 00:10:20
Over on that board I'm going
to put a formula for this

178
00:10:20 --> 00:10:22
convolution operation.
 

179
00:10:22 --> 00:10:27
But my point on this board
is, you've done it always.

180
00:10:27 --> 00:10:30
When you multiply a couple
of polynomials, you

181
00:10:30 --> 00:10:31
collect powers.
 

182
00:10:31 --> 00:10:34
And that's all convolution
is doing, collecting

183
00:10:34 --> 00:10:36
each power separately.
 

184
00:10:36 --> 00:10:38
OK, let's do it.
 

185
00:10:38 --> 00:10:47
So then f(x), g(x), is, when I
multiply that polynomial or

186
00:10:47 --> 00:10:53
series by that polynomial, I
get some polynomial in, with

187
00:10:53 --> 00:10:58
coefficients, oh I was
changed to l, just to have a

188
00:10:58 --> 00:11:00
different symbol there.
 

189
00:11:00 --> 00:11:03
And what was the formula for l?
 

190
00:11:03 --> 00:11:03
For h_l?
 

191
00:11:05 --> 00:11:08
What is the coefficient of z^l?
 

192
00:11:08 --> 00:11:10
 
 

193
00:11:10 --> 00:11:15
If I multiply that by that, do
you remember the story there?

194
00:11:15 --> 00:11:20
When I multiply that by
that and I looked for the

195
00:11:20 --> 00:11:22
terms that gave me z^l?
 

196
00:11:23 --> 00:11:26
OK, well that means that
this power times this

197
00:11:26 --> 00:11:27
power is going to be z^l.
 

198
00:11:29 --> 00:11:33
So that the index, do you
remember what happened?

199
00:11:33 --> 00:11:38
It was a lot of different, just
the way this h_l is here.

200
00:11:38 --> 00:11:41
Here's h_2 or something.
 

201
00:11:41 --> 00:11:43
I've got to do an addition.
 

202
00:11:43 --> 00:11:49
Because a bunch of c's come
in with different d's, and

203
00:11:49 --> 00:11:53
what's the deal then? c_k
comes in with which d?

204
00:11:53 --> 00:11:56
This is the magic number there.
 

205
00:11:56 --> 00:12:02
What's the subscript that if
I look at the coefficient of

206
00:12:02 --> 00:12:07
z^l, I look at each of these.
 

207
00:12:07 --> 00:12:11
And then I pick out the one of
these that will give me a z^l.

208
00:12:13 --> 00:12:14
And which one is it?
 

209
00:12:14 --> 00:12:14
d_(l-k).
 

210
00:12:14 --> 00:12:18
 
 

211
00:12:18 --> 00:12:23
It's that magic quantity that
the eye spots perfectly. k and

212
00:12:23 --> 00:12:32
l-k, adding to l simply because
z to the k is z^(l-k),

213
00:12:32 --> 00:12:33
multiplies to z^l.
 

214
00:12:35 --> 00:12:36
Same thing.
 

215
00:12:36 --> 00:12:37
Right?
 

216
00:12:37 --> 00:12:43
OK, this is, now I'll
use that notation.

217
00:12:43 --> 00:12:49
This is c convolved with the
d. c convolved with d is h.

218
00:12:49 --> 00:12:50
c convolved with d is h.
 

219
00:12:50 --> 00:12:54
So this is the l'th
component. c convolved

220
00:12:54 --> 00:12:55
with d is my symbol.
 

221
00:12:55 --> 00:12:58
This h is the convolution.
 

222
00:12:58 --> 00:13:01
And that's the
convolution rule.

223
00:13:01 --> 00:13:05
It's just whatever operation
you have to do to get

224
00:13:05 --> 00:13:06
the right answer.
 

225
00:13:06 --> 00:13:11
The right answer
when you multiply.

226
00:13:11 --> 00:13:13
So that one.
 

227
00:13:13 --> 00:13:16
Ready for the discrete case?
 

228
00:13:16 --> 00:13:18
The finite case?
 

229
00:13:18 --> 00:13:25
The case when, you have
power, when z becomes w.

230
00:13:25 --> 00:13:27
The discrete case.
 

231
00:13:27 --> 00:13:28
The cyclic case, sorry.
 

232
00:13:28 --> 00:13:33
Maybe emphasize the cyclic
case, meaning it circles

233
00:13:33 --> 00:13:38
around, is the case when z
becomes this very special

234
00:13:38 --> 00:13:42
z, on the unit circle
that we know as w.

235
00:13:42 --> 00:13:48
OK, and we have to say, and of
course it's w_N, I have to tell

236
00:13:48 --> 00:13:54
you in the cyclic case
how long the cycle is.

237
00:13:54 --> 00:13:57
So this would be a case.
 

238
00:13:57 --> 00:14:02
Watch what you do here
to make this cyclic.

239
00:14:02 --> 00:14:09
OK, I have three inputs,
so N is three here.

240
00:14:09 --> 00:14:13
Now I'm going to do the cyclic.
 

241
00:14:13 --> 00:14:16
So instead of z's, I
should be putting w's.

242
00:14:16 --> 00:14:18
I will.
 

243
00:14:18 --> 00:14:23
Just to emphasize, it's good to
think of it with the w there,

244
00:14:23 --> 00:14:27
because the w has this special
property that's critical to

245
00:14:27 --> 00:14:32
everything, OK so now I'm going
to, I think of this is as

246
00:14:32 --> 00:14:32
1w^0, 2w^1, and 3w^2.
 

247
00:14:32 --> 00:14:38
 
 

248
00:14:38 --> 00:14:40
I'm thinking of the
same multiplication

249
00:14:40 --> 00:14:45
here, but now w's.
 

250
00:14:45 --> 00:14:51
OK, so I'll end up with 18w^4,
and four was the constant

251
00:14:51 --> 00:14:55
and 3w's and so on.
 

252
00:14:55 --> 00:14:59
All these numbers.
 

253
00:14:59 --> 00:15:01
What's the difference?
 

254
00:15:01 --> 00:15:04
Ready for the key point?
 

255
00:15:04 --> 00:15:07
Now, what's happened
in this cyclic case?

256
00:15:07 --> 00:15:10
Well, the difference
is what is w^4?

257
00:15:12 --> 00:15:16
If we're in the cyclic case, N
is three now, our guys have

258
00:15:16 --> 00:15:24
length three, our circle is w
now is 1/3 of the way around.

259
00:15:24 --> 00:15:27
So that's my w,
here's my w squared.

260
00:15:27 --> 00:15:29
Here is my one.
 

261
00:15:29 --> 00:15:32
But here it is also w cubed.
 

262
00:15:32 --> 00:15:38
So w is the same as w^4,
w^2 is the same as w^5.

263
00:15:39 --> 00:15:44
So, what's the difference?
 

264
00:15:44 --> 00:15:48
What can I do now?
 

265
00:15:48 --> 00:15:55
If I'm in this discrete
case, then my inputs are a

266
00:15:55 --> 00:15:58
vector of length N, three.
 

267
00:15:58 --> 00:16:00
A vector of length N, three.
 

268
00:16:00 --> 00:16:06
And I want to get out to a
vector of length three.

269
00:16:06 --> 00:16:10
I'm not happy with that in
the cyclic case, because

270
00:16:10 --> 00:16:12
I'm not happy with w^4.
 

271
00:16:13 --> 00:16:17
So now tell me again that last,
when I do the multiplication

272
00:16:17 --> 00:16:21
and I just do it, there's no
difference except in how

273
00:16:21 --> 00:16:23
I write the answer.
 

274
00:16:23 --> 00:16:35
18w^4 is the same as? w. w^4 is
the same as w, when N is three.

275
00:16:35 --> 00:16:41
So that 18w^4 cycles
back in, with this 13.

276
00:16:41 --> 00:16:45
So now if I do the, can
I show you the symbol?

277
00:16:45 --> 00:16:47
I'll just do a little
circle there.

278
00:16:47 --> 00:16:52
To say this is now the
cyclic convolution.

279
00:16:52 --> 00:16:54
Then this isn't the
answer any more.

280
00:16:54 --> 00:16:57
The answer now for cyclic
convolution, I only

281
00:16:57 --> 00:16:59
want three numbers.
 

282
00:16:59 --> 00:17:01
And if you can tell me
what those three numbers

283
00:17:01 --> 00:17:04
are, we've got it.
 

284
00:17:04 --> 00:17:06
Move those over a little
to make room for

285
00:17:06 --> 00:17:07
the three numbers.
 

286
00:17:07 --> 00:17:11
So there's the answer, the
space for the answer.

287
00:17:11 --> 00:17:14
What do I write in?
 

288
00:17:14 --> 00:17:19
How many, what's
the constant term?

289
00:17:19 --> 00:17:21
It's 31, right.
 

290
00:17:21 --> 00:17:22
It's 31.
 

291
00:17:22 --> 00:17:25
Where did 31 come from?
 

292
00:17:25 --> 00:17:31
It came from, you could
say, cycling that 27w^3

293
00:17:31 --> 00:17:33
back with the four.
 

294
00:17:33 --> 00:17:36
Because there's no w^3
is the same as one.

295
00:17:36 --> 00:17:40
So I've gone around the circle
when I come around to w^3.

296
00:17:40 --> 00:17:45
So that 27 and four
combined into that 31.

297
00:17:45 --> 00:17:49
And let's see so so
what multiplications

298
00:17:49 --> 00:17:50
am I doing here?
 

299
00:17:50 --> 00:17:53
One times four gives
me the constant.

300
00:17:53 --> 00:18:00
Two times six, that's
2w's, and six w squareds,

301
00:18:00 --> 00:18:03
that's 12w^3, that's 12.
 

302
00:18:03 --> 00:18:09
And then 3w^2, and
five w's is 15w^3.

303
00:18:10 --> 00:18:13
But that's the same as 15.
 

304
00:18:13 --> 00:18:15
So that's why we get 31.
 

305
00:18:15 --> 00:18:20
We got four, we've got
12 and we've got 15.

306
00:18:20 --> 00:18:25
OK, now what's the second
component, the w component

307
00:18:25 --> 00:18:32
of the cyclic convolution?
 

308
00:18:32 --> 00:18:36
Tell me what number do I write
in, in that middle position?

309
00:18:36 --> 00:18:39
How many w's do I have?
 

310
00:18:39 --> 00:18:41
31 again.
 

311
00:18:41 --> 00:18:47
This 28, this 18 is
coming back by three.

312
00:18:47 --> 00:18:48
To 13.
 

313
00:18:48 --> 00:18:53
You see, I could have done
that multiplication.

314
00:18:53 --> 00:18:57
So coming back to 31, am I
going to get another 31?

315
00:18:57 --> 00:19:01
No, what's the w^2 guy?
 

316
00:19:01 --> 00:19:01
28.
 

317
00:19:01 --> 00:19:02
Yeah.
 

318
00:19:02 --> 00:19:09
28, because there's no to
the fifth to come back.

319
00:19:09 --> 00:19:15
So 28 uses three
multiplications.

320
00:19:15 --> 00:19:21
The 31 there used the
four and these two.

321
00:19:21 --> 00:19:24
This came back over to here.
 

322
00:19:24 --> 00:19:28
And this 31 used,
this 18 came back.

323
00:19:28 --> 00:19:30
I could have put the 18 here.
 

324
00:19:30 --> 00:19:33
You know, I could have
lined it up just three.

325
00:19:33 --> 00:19:35
So I'll write a formula for it.
 

326
00:19:35 --> 00:19:37
So that's the answer.
 

327
00:19:37 --> 00:19:41
31, 31, 28.
 

328
00:19:41 --> 00:19:44
Could I just suggest a
little check on that?

329
00:19:44 --> 00:19:46
Just to check on the numbers.
 

330
00:19:46 --> 00:19:52
I think that if I add up
these numbers, I get six.

331
00:19:52 --> 00:19:57
And if I add up those
numbers I get 15.

332
00:19:57 --> 00:20:01
And if I multiply
that, I get 90.

333
00:20:01 --> 00:20:04
And if I add those
numbers I get 90, right?

334
00:20:04 --> 00:20:08
So those add to 90.
 

335
00:20:08 --> 00:20:13
So I'm just saying the miracle
check is add these, multiply

336
00:20:13 --> 00:20:15
by the sum of those.
 

337
00:20:15 --> 00:20:17
And you get the sum of those.
 

338
00:20:17 --> 00:20:20
Why is that?
 

339
00:20:20 --> 00:20:23
Somehow seems
right, doesn't it?

340
00:20:23 --> 00:20:29
Because somehow I've taken all
nine products here, and so when

341
00:20:29 --> 00:20:33
I add all the results, I'll
have the sum of all nine of

342
00:20:33 --> 00:20:35
these possible products.
 

343
00:20:35 --> 00:20:37
So I'll have six times 15.
 

344
00:20:37 --> 00:20:41
Actually, here's a good
way to look at it.

345
00:20:41 --> 00:20:43
In doing this, I just set w=1.
 

346
00:20:43 --> 00:20:48
 
 

347
00:20:48 --> 00:20:51
I just set w=1 in
the polynomial.

348
00:20:51 --> 00:20:55
When I set w to one, this
becomes six, this becomes

349
00:20:55 --> 00:20:59
15 and the answer becomes
90, when w is one.

350
00:20:59 --> 00:21:00
Yeah.
 

351
00:21:00 --> 00:21:02
So that's another way to see.
 

352
00:21:02 --> 00:21:08
And actually, this
multiplication, the second

353
00:21:08 --> 00:21:14
grade version, had w, well,
I'm almost going to say

354
00:21:14 --> 00:21:19
w=10, but not quite that.
 

355
00:21:19 --> 00:21:22
Because it's written in the
opposite order, right?

356
00:21:22 --> 00:21:25
If w was ten, this
would be one.

357
00:21:25 --> 00:21:28
Well, anyway, w can't be
ten, it's got to stay

358
00:21:28 --> 00:21:31
on the unit circle, so.
 

359
00:21:31 --> 00:21:34
So somewhere in the non-cyclic
case, it's something like

360
00:21:34 --> 00:21:38
w=10, or w=1/10, maybe.
 

361
00:21:38 --> 00:21:39
Whatever.
 

362
00:21:39 --> 00:21:40
OK.
 

363
00:21:40 --> 00:21:43
Could you take the convolution
now, let me give you

364
00:21:43 --> 00:21:45
just another example.
 

365
00:21:45 --> 00:21:49
Do it mentally.
 

366
00:21:49 --> 00:21:55
What's the convolution
of , let me

367
00:21:55 --> 00:21:57
make a little longer.
 

368
00:21:57 --> 00:22:02
So I take the, first of all,
the non-cyclic convolution

369
00:22:02 --> 00:22:05
of .
 

370
00:22:05 --> 00:22:07
OK.
 

371
00:22:07 --> 00:22:11
What's the ordinary - how
long is the answer now?

372
00:22:11 --> 00:22:13
This is just practice.
 

373
00:22:13 --> 00:22:16
How many components am I going
to have in the ordinary

374
00:22:16 --> 00:22:22
convolution of those two
guys of length four?

375
00:22:22 --> 00:22:24
I think it's seven.
 

376
00:22:24 --> 00:22:26
I think it'll be seven.
 

377
00:22:26 --> 00:22:30
Because we'll have here one,
we have no, we have it

378
00:22:30 --> 00:22:34
z^0, z^1, z^2, z^3.
 

379
00:22:35 --> 00:22:39
And here we'll have again the
same, it would go up to z^6 but

380
00:22:39 --> 00:22:43
remember there's a z^0, so
that's why we have seven.

381
00:22:43 --> 00:22:47
And what will it be?
 

382
00:22:47 --> 00:22:49
What will it be?
 

383
00:22:49 --> 00:22:53
I guess, actually, this wasn't
a brilliant example, was it?

384
00:22:53 --> 00:22:55
But let's finish it.
 

385
00:22:55 --> 00:23:01
So I'm just multiplying z
by z squared, so what do

386
00:23:01 --> 00:23:02
you get for an answer?
 

387
00:23:02 --> 00:23:07
I think the one shows
up in the z^3.

388
00:23:07 --> 00:23:13
 
 

389
00:23:13 --> 00:23:14
Is that right?
 

390
00:23:14 --> 00:23:22
Yeah. z^1, here z^2 here, z^3
here, and we always have

391
00:23:22 --> 00:23:27
to remember everything in
Chapter 4 starts at zero.

392
00:23:27 --> 00:23:30
Zero, z^0's the first one.
 

393
00:23:30 --> 00:23:34
OK, that's not too great an
example, because what happens

394
00:23:34 --> 00:23:40
if I do the circular
cyclic convolution?

395
00:23:40 --> 00:23:42
What would be the
cyclic convolution?

396
00:23:42 --> 00:23:47
Now I'm expecting four
guys only, right?

397
00:23:47 --> 00:23:50
A cyclic keeps the same length.
 

398
00:23:50 --> 00:23:55
And what would be the answer?
 

399
00:23:55 --> 00:23:57
Well, there's nobody to
fold back, so it would

400
00:23:57 --> 00:23:59
be just .
 

401
00:23:59 --> 00:24:06
So let me update this a
little bit with one here.

402
00:24:06 --> 00:24:08
OK, just to practice.
 

403
00:24:08 --> 00:24:12
So suppose I do
that convolution.

404
00:24:12 --> 00:24:15
Un-cyclic, first.
 

405
00:24:15 --> 00:24:18
What do I change here now?
 

406
00:24:18 --> 00:24:22
I've now got a z^2 and I've
also got a z^3, but I only

407
00:24:22 --> 00:24:25
have a single one there.
 

408
00:24:25 --> 00:24:27
Let's make it a little
more interesting.

409
00:24:27 --> 00:24:30
OK, make it like so.
 

410
00:24:30 --> 00:24:36
Alright, z+z^2 is what
we're looking at, z+z^2

411
00:24:36 --> 00:24:37
is multiplying z^2+z^3.
 

412
00:24:39 --> 00:24:43
And in the long form,
what do I get?

413
00:24:43 --> 00:24:47
Let me make space, tell me
what numbers to put in.

414
00:24:47 --> 00:24:52
If I multiply
z+z^2+z^3, I get what?

415
00:24:52 --> 00:24:54
1z^3, how many z^4?
 

416
00:24:54 --> 00:24:57
 
 

417
00:24:57 --> 00:24:58
Two of them.
 

418
00:24:58 --> 00:24:58
How many z^5?
 

419
00:25:00 --> 00:25:01
One, and nobody there.
 

420
00:25:01 --> 00:25:02
OK.
 

421
00:25:02 --> 00:25:09
And now the cyclic
version would be what?

422
00:25:09 --> 00:25:12
What's what's my answer now
for the cyclic version?

423
00:25:12 --> 00:25:15
Let me take those out.
 

424
00:25:15 --> 00:25:21
So the cyclic version would
bring the two back to the zero.

425
00:25:21 --> 00:25:24
Would bring that
one back so there.

426
00:25:24 --> 00:25:27
That zero will still be zero.
 

427
00:25:27 --> 00:25:32
And I checked that I haven't
missed anything by adding those

428
00:25:32 --> 00:25:37
up to get four, and adding
this up to get two times two.

429
00:25:37 --> 00:25:38
Yeah.
 

430
00:25:38 --> 00:25:42
OK, so that's the rule.
 

431
00:25:42 --> 00:25:47
And it's a lot cleaner to
see these answers than

432
00:25:47 --> 00:25:52
to see this formula.
 

433
00:25:52 --> 00:25:57
And I need, actually of course,
now mentioning that formula,

434
00:25:57 --> 00:25:59
I need a cyclic formula.
 

435
00:25:59 --> 00:26:03
So can I write above it
the cyclic formula?

436
00:26:03 --> 00:26:13
What do I get when I'm, instead
of this sum, which went from

437
00:26:13 --> 00:26:21
k equal minus infinity to
infinity, in the cyclic case,

438
00:26:21 --> 00:26:29
h_k is just going to be a sum
from zero to N-1, and there'll

439
00:26:29 --> 00:26:33
be a c_k, and a d something.
 

440
00:26:33 --> 00:26:42
And now this is the cyclic
case, so I guess this makes us,

441
00:26:42 --> 00:26:46
I think what our situation now
is we understand the cyclic

442
00:26:46 --> 00:26:47
case from examples.
 

443
00:26:47 --> 00:26:52
And now we just have the job of
how do I put it into algebra.

444
00:26:52 --> 00:26:55
How do I put it into symbols?
 

445
00:26:55 --> 00:27:02
What's the point?
c_k d_n, let's say.

446
00:27:02 --> 00:27:08
But oh no, I'm looking
for h_l, yeah.

447
00:27:08 --> 00:27:10
So what's the deal?
 

448
00:27:10 --> 00:27:14
Here l was k.
 

449
00:27:14 --> 00:27:17
That one is the sum of
that one and that one.

450
00:27:17 --> 00:27:21
So here, l is the sum
of k and n, but.

451
00:27:21 --> 00:27:28
What's the but?
 

452
00:27:28 --> 00:27:35
I mean somehow I've got some
wraparound to do, right?

453
00:27:35 --> 00:27:39
When I'm doing the cyclic
multiplication and I'm doing

454
00:27:39 --> 00:27:47
the wraparound because w^n,
the wraparound comes

455
00:27:47 --> 00:27:49
from that, right?
 

456
00:27:49 --> 00:27:52
That's why I never get as high
as n, because when I get to n I

457
00:27:52 --> 00:27:55
go back to the zeroth power.
 

458
00:27:55 --> 00:28:02
OK, so what's the relation
of k and n and l here?

459
00:28:02 --> 00:28:07
We just need the right
word to express it.

460
00:28:07 --> 00:28:08
What's the word?
 

461
00:28:08 --> 00:28:10
Mod.
 

462
00:28:10 --> 00:28:17
So that's the word I'm
looking for. k+n is l.

463
00:28:17 --> 00:28:23
With wraparound and wraparound
means that the nice notation

464
00:28:23 --> 00:28:31
that people use is a mod m.
 

465
00:28:31 --> 00:28:35
Let's practice.
 

466
00:28:35 --> 00:28:45
What is two plus two mod seven?
 

467
00:28:45 --> 00:28:47
Four.
 

468
00:28:47 --> 00:28:49
Two plus two is four,
even in 18.085.

469
00:28:49 --> 00:28:51
Right, OK.
 

470
00:28:51 --> 00:29:04
But two plus two mod three is,
two plus two mod three is? one.

471
00:29:04 --> 00:29:06
Everybody sees it?
 

472
00:29:06 --> 00:29:14
I'm taking z^2 times z^2, z^4,
but I'm doing with N=3, so z^3

473
00:29:14 --> 00:29:20
is on3, so that z^4 is really
just z to the first power.

474
00:29:20 --> 00:29:27
So this is the little
nifty notation that

475
00:29:27 --> 00:29:29
says make it cyclic.
 

476
00:29:29 --> 00:29:34
Bring it back so that
l only has the values

477
00:29:34 --> 00:29:36
here, zero up to N-1.
 

478
00:29:38 --> 00:29:41
And then stops.
 

479
00:29:41 --> 00:29:45
OK.
 

480
00:29:45 --> 00:29:52
OK, so we'll have more practice
with examples when we

481
00:29:52 --> 00:29:56
do some filtering.
 

482
00:29:56 --> 00:30:00
Have you got that fundamental,
so we've talked about this rule

483
00:30:00 --> 00:30:07
one here. f times g goes
to those coefficients.

484
00:30:07 --> 00:30:11
And if it's the cyclic
case then I put a circle

485
00:30:11 --> 00:30:12
around that star.
 

486
00:30:12 --> 00:30:16
And I do the wraparound.
 

487
00:30:16 --> 00:30:26
But it's just, it's the z
transform, it's polynomials in

488
00:30:26 --> 00:30:31
z or polynomials in w, and when
it's polynomials in w, you use

489
00:30:31 --> 00:30:34
that special property
that w_N is one.

490
00:30:34 --> 00:30:38
Yeah, OK.
 

491
00:30:38 --> 00:30:43
Now, I see I've written
another line, there.

492
00:30:43 --> 00:30:46
That I could
convolve functions.

493
00:30:46 --> 00:30:48
Let me do a couple
more examples.

494
00:30:48 --> 00:30:50
Couple of examples.
 

495
00:30:50 --> 00:30:59
First, before I go
to that line, OK.

496
00:30:59 --> 00:31:05
So I'm up to this line.
 

497
00:31:05 --> 00:31:12
A couple of examples here.
 

498
00:31:12 --> 00:31:18
Let's see, what example
would I want to do?

499
00:31:18 --> 00:31:28
Let's see, OK, I want to do one
example with a delta function.

500
00:31:28 --> 00:31:31
One example with a
delta function.

501
00:31:31 --> 00:31:32
One example with
the delta vector.

502
00:31:32 --> 00:31:41
Yeah, let me take the function
g(x) identically one.

503
00:31:41 --> 00:31:45
OK, constant function.
 

504
00:31:45 --> 00:31:48
In this rule.
 

505
00:31:48 --> 00:31:50
I want to see what
happens with the rule.

506
00:31:50 --> 00:32:00
OK, then f(x) g(x) is the
same as f(x), right?

507
00:32:00 --> 00:32:03
Because this function g(x) is
so simple, it's just one.

508
00:32:03 --> 00:32:09
Now, what about
the coefficient?

509
00:32:09 --> 00:32:13
So I have the coefficients
of c, what are the Fourier

510
00:32:13 --> 00:32:15
coefficients, what are the d's?
 

511
00:32:15 --> 00:32:18
Ah yes, what are the d's?
 

512
00:32:18 --> 00:32:22
So I'm testing my rule on a
really really simple case,

513
00:32:22 --> 00:32:25
g(x) identically one.
 

514
00:32:25 --> 00:32:29
What, you have to tell me, in
order to check the right side

515
00:32:29 --> 00:32:33
of the rule you have to tell me
the Fourier coefficients for

516
00:32:33 --> 00:32:35
that very special function.
 

517
00:32:35 --> 00:32:38
What would be the
Fourier coefficients?

518
00:32:38 --> 00:32:41
If I expand the function
one in a Fourier

519
00:32:41 --> 00:32:44
series, what do I see?
 

520
00:32:44 --> 00:32:44
I see a one.
 

521
00:32:44 --> 00:32:47
Yeah, that's it, I see one.
 

522
00:32:47 --> 00:32:54
So what are its coefficients?
d_0, right, is one?

523
00:32:54 --> 00:33:01
And the other d's are
all zero, right?

524
00:33:01 --> 00:33:08
So my vector of d, my vector of
d's is a whole lot of zeroes

525
00:33:08 --> 00:33:09
on the negative side.
 

526
00:33:09 --> 00:33:11
A one right there in
the center, and then

527
00:33:11 --> 00:33:12
a lot of zeroes.
 

528
00:33:12 --> 00:33:24
And now I want to
convolve that with c.

529
00:33:24 --> 00:33:28
I'm practicing the convolution
rule on a case that's so

530
00:33:28 --> 00:33:34
simple it's confusing, right?
 

531
00:33:34 --> 00:33:45
I mean, it's a big mess,
this multiplication.

532
00:33:45 --> 00:33:47
What do I get out of this?
 

533
00:33:47 --> 00:33:58
If d is this vector, if d has
this property that d_0 is one

534
00:33:58 --> 00:34:04
and others are zero, all others
are zero, so this is my little

535
00:34:04 --> 00:34:10
example, what does this
sum boil down to?

536
00:34:10 --> 00:34:17
Well, I only get something
when l=k, right?

537
00:34:17 --> 00:34:22
I only got something when l=k,
because then I have d_0 and

538
00:34:22 --> 00:34:25
that's the only d
that's around.

539
00:34:25 --> 00:34:30
So in this sum, something
happens only when k

540
00:34:30 --> 00:34:31
and l are the same.
 

541
00:34:31 --> 00:34:33
And then what happens?
 

542
00:34:33 --> 00:34:37
Then I have a one, I have c_l,
and that's h_l, so that's

543
00:34:37 --> 00:34:39
all I'm concluding then.
 

544
00:34:39 --> 00:34:43
That this h is the same as c.
 

545
00:34:43 --> 00:34:48
I'm sorry, it's so dumb.
 

546
00:34:48 --> 00:34:51
My point is that in
convolution, this is the

547
00:34:51 --> 00:34:54
thing that acts like one.
 

548
00:34:54 --> 00:34:57
Because in multiplication,
that's the thing, that's the

549
00:34:57 --> 00:34:59
function that acts like one.
 

550
00:34:59 --> 00:35:01
That's the function
that is one.

551
00:35:01 --> 00:35:11
So this is the one in, oh,
would you allow me to do this?

552
00:35:11 --> 00:35:18
I'm going to create a
matrix with these d's.

553
00:35:18 --> 00:35:24
There's another way
to see convolution.

554
00:35:24 --> 00:35:26
Yeah, there's another way
to see convolution and

555
00:35:26 --> 00:35:28
discrete convolution.
 

556
00:35:28 --> 00:35:31
Maybe the discrete
one's the better.

557
00:35:31 --> 00:35:36
Yeah can you stand one more
way to write the formula?

558
00:35:36 --> 00:35:41
One more way to write, now I'm
going to do, I'm going to

559
00:35:41 --> 00:35:47
do discrete convolution.
 

560
00:35:47 --> 00:35:48
Discrete cyclic.
 

561
00:35:48 --> 00:35:53
Just So how am I
going to write it?

562
00:35:53 --> 00:35:57
I'm going to write it by
a matrix multiplication.

563
00:35:57 --> 00:35:59
Because you know that in
this course a matrix

564
00:35:59 --> 00:36:01
was going to show up.
 

565
00:36:01 --> 00:36:06
So it's going to be a
matrix multiplication.

566
00:36:06 --> 00:36:09
So I just have to tell you
the matrix, so this is

567
00:36:09 --> 00:36:12
going to be some matrix.
 

568
00:36:12 --> 00:36:15
Let me take N to be four.
 

569
00:36:15 --> 00:36:20
So then I have, you watch.
 

570
00:36:20 --> 00:36:32
So I have four d's, and the
output is the four h's.

571
00:36:32 --> 00:36:37
And the rule I'm
following is this rule.

572
00:36:37 --> 00:36:43
Is this, the same old rule
but with the cyclic part.

573
00:36:43 --> 00:36:47
And now I want to show you the
matrix that'll just do this.

574
00:36:47 --> 00:36:54
Look, I've put the c's
in the first column.

575
00:36:54 --> 00:37:00
And then I go, yeah,
here's another.

576
00:37:00 --> 00:37:07
So it's a cyclic matrix.
 

577
00:37:07 --> 00:37:09
So let me finish it up.
 

578
00:37:09 --> 00:37:13
It's going to be four by four,
it's going to be cyclic.

579
00:37:13 --> 00:37:18
So I have a c_0, c_0, c_0,
c_0 on the diagonal.

580
00:37:18 --> 00:37:22
That's fine, that's because z
to the zeroes multiplying all

581
00:37:22 --> 00:37:24
the d's and leaving
them in place.

582
00:37:24 --> 00:37:29
And then I have c_1's, and
then I think I come around

583
00:37:29 --> 00:37:30
again here for a c_1.
 

584
00:37:31 --> 00:37:37
And I have c_2's, see
where see c_3, c_2.

585
00:37:38 --> 00:37:43
And I come around again,
to a c_2 and a c_2.

586
00:37:43 --> 00:37:48
And c_3 comes around to
a c_3, a c_3 and a c_3.

587
00:37:49 --> 00:37:59
Well, can you, I hope you can
see, this is cyclic matrix.

588
00:37:59 --> 00:38:06
It's only got one, it starts
with a vector c, and those

589
00:38:06 --> 00:38:10
are on the diagonal and
the diagonals wrap around.

590
00:38:10 --> 00:38:13
That's the other word that you
often see when you see the

591
00:38:13 --> 00:38:16
word cyclic, wraparound.
 

592
00:38:16 --> 00:38:23
It's because you
think of a circle.

593
00:38:23 --> 00:38:26
If you go the second time
around, it's wrapped

594
00:38:26 --> 00:38:28
around the first time.
 

595
00:38:28 --> 00:38:32
OK, just can you look and see
that this is the right formula

596
00:38:32 --> 00:38:34
for h_0? h_0 is c_0*d_0.
 

597
00:38:36 --> 00:38:37
Where does that come from?
 

598
00:38:37 --> 00:38:41
Remember, h_0 is the
coefficient of z^0

599
00:38:41 --> 00:38:43
in the answer.
 

600
00:38:43 --> 00:38:47
So it comes from c_0*d_0 to the
zeroth power in the input.

601
00:38:47 --> 00:38:49
And then why is
there is a c_3*d_1?

602
00:38:49 --> 00:38:52
 
 

603
00:38:52 --> 00:38:57
Why is there a c_3*d_1, and
then a c_2*d_2 and then a

604
00:38:57 --> 00:39:00
c_1*d_3 all piling up into h_0?
 

605
00:39:02 --> 00:39:04
Tell me now, why is
there a c_3*d_1?

606
00:39:04 --> 00:39:07
 
 

607
00:39:07 --> 00:39:10
Because we're doing mod
four is one way to say it.

608
00:39:10 --> 00:39:12
Three and one add to four.
 

609
00:39:12 --> 00:39:21
Because c_3 is the w cubed guy,
and d_1 is the coefficient of

610
00:39:21 --> 00:39:30
w^1 t and w^3 times w^1 piles
back into the constant.

611
00:39:30 --> 00:39:33
And you see the pattern
of that matrix?

612
00:39:33 --> 00:39:38
So these matrices
are very important.

613
00:39:38 --> 00:39:40
So they circle around.
 

614
00:39:40 --> 00:39:45
Oh, we've actually met a
matrix of this type, the

615
00:39:45 --> 00:39:49
first day of 18.085.
 

616
00:39:49 --> 00:39:51
What was that matrix?
 

617
00:39:51 --> 00:39:53
It was one of our
four great matrices.

618
00:39:53 --> 00:39:56
And now here it is back again.
 

619
00:39:56 --> 00:40:00
Which one was it?
 

620
00:40:00 --> 00:40:03
Well, you remember
the letter for it.

621
00:40:03 --> 00:40:05
Which isn't going to change.
 

622
00:40:05 --> 00:40:11
And do you remember the
particular matrix?

623
00:40:11 --> 00:40:14
Well, everybody does remember
that matrix, right?

624
00:40:14 --> 00:40:18
Twos were on the diagonal,
minus ones were on the

625
00:40:18 --> 00:40:22
diagonal, and the diagonal
curve continued.

626
00:40:22 --> 00:40:26
Minus one was on this diagonal
and that, continued and

627
00:40:26 --> 00:40:28
zeroes was on this diagonal.
 

628
00:40:28 --> 00:40:36
So this is cyclic convolution,
the circulant matrix, cyclic

629
00:40:36 --> 00:40:43
convolution by c, what's the
c that produces that

630
00:40:43 --> 00:40:46
convolution matrix?
 

631
00:40:46 --> 00:40:50
It's just, it's got
- well there it is.

632
00:40:50 --> 00:40:56
The first column is it.
 

633
00:40:56 --> 00:40:57
Right, right.
 

634
00:40:57 --> 00:41:09
And somehow I would say that
that's an even vector.

635
00:41:09 --> 00:41:12
It's sort of, I associate
it with cosine.

636
00:41:12 --> 00:41:13
It's an even vector.
 

637
00:41:13 --> 00:41:19
Here is the zero term, and
then these are the same, not

638
00:41:19 --> 00:41:21
to worry about that part.
 

639
00:41:21 --> 00:41:25
Do you see that we've
seen that matrix before?

640
00:41:25 --> 00:41:29
And the cyclic convolution
means you take its second

641
00:41:29 --> 00:41:32
differences, of course.
 

642
00:41:32 --> 00:41:36
We're taking second
differences, but our everything

643
00:41:36 --> 00:41:40
in our world is cyclic.
 

644
00:41:40 --> 00:41:46
So the result, the x_4 is x_0.
 

645
00:41:47 --> 00:41:50
So we're taking second
differences - well, maybe I

646
00:41:50 --> 00:41:55
should say d - we're taking
second differences d_i,

647
00:41:55 --> 00:42:04
2d_i's, -d_(i-1), -d(i+1),
I don't know if this is.

648
00:42:04 --> 00:42:07
So there's a minus
one, two, minus one.

649
00:42:07 --> 00:42:10
And we're cycling around
so that d_0 is d_4.

650
00:42:11 --> 00:42:20
And d_1 is d_5, and
d_-1 is d_3, whatever.

651
00:42:20 --> 00:42:25
OK, I'm just reminding you,
we've seen these before.

652
00:42:25 --> 00:42:33
OK, so this is another way
to remember the formula.

653
00:42:33 --> 00:42:37
OK now can I ask you a
practical question?

654
00:42:37 --> 00:42:39
A practical question.
 

655
00:42:39 --> 00:42:47
Let me bring back this second
grade multiplication.

656
00:42:47 --> 00:42:50
Well, I have a granddaughter
named Elizabeth, I'll have to

657
00:42:50 --> 00:42:52
admit I didn't think about
mentioning Elizabeth.

658
00:42:52 --> 00:42:54
She's six.
 

659
00:42:54 --> 00:42:59
And she delights in sending
me long multiplications.

660
00:42:59 --> 00:43:02
I mean, really long.
 

661
00:43:02 --> 00:43:05
And then every time I talk to
her on the phone, she says

662
00:43:05 --> 00:43:06
have you done that one yet?
 

663
00:43:06 --> 00:43:11
And I say, I'm working on it
I've got MATLAB at work.

664
00:43:11 --> 00:43:15
Because they're ridiculous
and I haven't figured

665
00:43:15 --> 00:43:15
out how to tell her.
 

666
00:43:15 --> 00:43:20
I mean, she just writes
page after page.

667
00:43:20 --> 00:43:24
Times 100, plus three,
minus seven, just

668
00:43:24 --> 00:43:25
whatever she things of.
 

669
00:43:25 --> 00:43:30
OK, now I need help from
the convolution groupal,

670
00:43:30 --> 00:43:31
here, actually.
 

671
00:43:31 --> 00:43:35
So let's suppose that Elizabeth
has given me a multiplication

672
00:43:35 --> 00:43:41
in which I have a thousand
digits times a thousand, right.

673
00:43:41 --> 00:43:44
Which Mathematica is prepared
to do exactly, right?

674
00:43:44 --> 00:43:51
MATLAB will mess up, but
Mathematica and Maple and

675
00:43:51 --> 00:43:55
symbolic packages will
do exact computations.

676
00:43:55 --> 00:43:59
So what would be the right way,
well let may make it 1,024.

677
00:43:59 --> 00:44:08
1,024 digits times
1,024 digits.

678
00:44:08 --> 00:44:11
Let's do the cyclic
version first.

679
00:44:11 --> 00:44:13
Elizabeth doesn't
know about cyclic.

680
00:44:13 --> 00:44:14
Maybe I could teach her that.
 

681
00:44:14 --> 00:44:18
That'll keep her busy while
I'm doing the multiplication.

682
00:44:18 --> 00:44:23
OK, right.
 

683
00:44:23 --> 00:44:25
Only her older brother
would explain it to her,

684
00:44:25 --> 00:44:26
that's the trouble.
 

685
00:44:26 --> 00:44:30
OK, so how am I going to do, or
how are you going to do on the

686
00:44:30 --> 00:44:37
quiz, multiplication of a
1,024 digits times 1,024?

687
00:44:37 --> 00:44:40
And I'll make it easy by making
it cyclic, so I just want

688
00:44:40 --> 00:44:43
1,024 digits in the answer.
 

689
00:44:43 --> 00:44:45
OK.
 

690
00:44:45 --> 00:44:48
How would you do it?
 

691
00:44:48 --> 00:44:55
Well, before today, you would
have just multiplied, right?

692
00:44:55 --> 00:44:59
You would have written
down 1,024 two, lines of

693
00:44:59 --> 00:45:03
1,024, done an addition.
 

694
00:45:03 --> 00:45:08
And you would have had a
million multiplications to do.

695
00:45:08 --> 00:45:10
But how would you do it now?
 

696
00:45:10 --> 00:45:14
Apart from giving
it to Mathematica.

697
00:45:14 --> 00:45:17
What's a faster way to do it?
 

698
00:45:17 --> 00:45:21
What's a faster way
to do a convolution?

699
00:45:21 --> 00:45:24
The fast way to do a
convolution is to use the

700
00:45:24 --> 00:45:27
convolution rule, go this way.
 

701
00:45:27 --> 00:45:32
So take these numbers, these
1,024 numbers, in c and these

702
00:45:32 --> 00:45:40
1,024 numbers in d, and,
well what do I have to do?

703
00:45:40 --> 00:45:42
I want to use the
convolution rule, because

704
00:45:42 --> 00:45:47
multiplying is fast.
 

705
00:45:47 --> 00:45:50
Now I've got functions.
 

706
00:45:50 --> 00:45:52
But I'm in the cyclic case.
 

707
00:45:52 --> 00:45:58
So I'm in the cyclic case,
so what should I do?

708
00:45:58 --> 00:46:03
How can I change this
to be the cyclic case?

709
00:46:03 --> 00:46:08
This is like f, j, g, g, j.
 

710
00:46:08 --> 00:46:16
So multiplication of components
of things in function space is

711
00:46:16 --> 00:46:18
convolution of coefficients.
 

712
00:46:18 --> 00:46:22
So now, this is the cyclic.
 

713
00:46:22 --> 00:46:25
So let me make it cyclic.
 

714
00:46:25 --> 00:46:28
So again, what's your problem?
 

715
00:46:28 --> 00:46:31
The problem is to do this
cyclic multiplication.

716
00:46:31 --> 00:46:33
What's the idea?
 

717
00:46:33 --> 00:46:41
The idea is to transform c back
to f, to transform d back to g.

718
00:46:41 --> 00:46:44
Do the multiplications,
now I have only 1,024

719
00:46:44 --> 00:46:45
multiplications.
 

720
00:46:45 --> 00:46:47
Not 1,024 squared.
 

721
00:46:47 --> 00:46:49
That's the point.
 

722
00:46:49 --> 00:46:53
And if I do this directly,
I've got 1,024 squared

723
00:46:53 --> 00:46:55
multiplications to do.
 

724
00:46:55 --> 00:46:57
Much better.
 

725
00:46:57 --> 00:47:02
Transform back to here, do just
1,024 - what's the MATLAB

726
00:47:02 --> 00:47:05
command for that, when
you're multiplying each

727
00:47:05 --> 00:47:06
component by itself?
 

728
00:47:06 --> 00:47:09
It's not the dot
product, notice.

729
00:47:09 --> 00:47:13
It's not the dot product
because I'm not summing.

730
00:47:13 --> 00:47:18
Do you know the MATLAB command,
if I have a sequence of numbers

731
00:47:18 --> 00:47:25
of vector f, of length 1,024
and I want to get that result?

732
00:47:25 --> 00:47:26
What's the result?
 

733
00:47:26 --> 00:47:32
It's a vector of length 1,024
that takes each f times its g.

734
00:47:32 --> 00:47:34
But doesn't do any adds.
 

735
00:47:34 --> 00:47:36
That's what's there.
 

736
00:47:36 --> 00:47:38
What's the MATLAB
command for that?

737
00:47:38 --> 00:47:40
Dot, yeah.
 

738
00:47:40 --> 00:47:42
Dot star, right.
 

739
00:47:42 --> 00:47:45
So that dot says
component by component.

740
00:47:45 --> 00:47:48
OK, so what's the plan here?
 

741
00:47:48 --> 00:47:53
I do c's back to F.
 

742
00:47:53 --> 00:48:00
By the Fourier matrix. d back
to g, by the Fourier matrix,

743
00:48:00 --> 00:48:04
then I do a very quick
multiplication.

744
00:48:04 --> 00:48:05
And then what?
 

745
00:48:05 --> 00:48:07
Then I mustn't forget.
 

746
00:48:07 --> 00:48:11
That I'm in frequency space,
and what do I have to do?

747
00:48:11 --> 00:48:16
I've got to get back
into coefficient space.

748
00:48:16 --> 00:48:24
So I do the inverse transform
- here's the formula, then.

749
00:48:24 --> 00:48:32
I'm doing the inverse transform
of F, so the transform of c

750
00:48:32 --> 00:48:39
dot star, the transform of d.
 

751
00:48:39 --> 00:48:44
To get c, d.
 

752
00:48:44 --> 00:48:47
Is that right?
 

753
00:48:47 --> 00:48:51
So I took c, and I got
back into the function.

754
00:48:51 --> 00:48:54
I took d, and got back
to its function, with

755
00:48:54 --> 00:48:55
the Fourier matrix.
 

756
00:48:55 --> 00:48:59
OK, I'm in the Fourier and
now I'm in this space.

757
00:48:59 --> 00:49:03
I've added up coefficients
to get in this space.

758
00:49:03 --> 00:49:06
Now I do the dot
star, the fast one.

759
00:49:06 --> 00:49:08
And then I transform back.
 

760
00:49:08 --> 00:49:11
So why is that faster?
 

761
00:49:11 --> 00:49:14
Than just doing it?
 

762
00:49:14 --> 00:49:20
Because what's the
cost of F times c?

763
00:49:20 --> 00:49:23
And how am I going to do that?
 

764
00:49:23 --> 00:49:25
I'm going to do with the fast
Fourier transform, right.

765
00:49:25 --> 00:49:26
That's the point.
 

766
00:49:26 --> 00:49:32
I can multiply by F, or by
F inverse faster so I have

767
00:49:32 --> 00:49:36
three of these transforms.
 

768
00:49:36 --> 00:49:39
I've got to get two guys
into the other space,

769
00:49:39 --> 00:49:42
and the answer back out.
 

770
00:49:42 --> 00:49:49
So I have sort of three of
these n log n, but that

771
00:49:49 --> 00:49:53
will easily be n squared.
 

772
00:49:53 --> 00:49:53
Right?
 

773
00:49:53 --> 00:50:02
So if you have a convolution to
do, and it's possible to do

774
00:50:02 --> 00:50:07
this, get into the other space
where it's just an element

775
00:50:07 --> 00:50:09
by element multiplication.
 

776
00:50:09 --> 00:50:11
And that would apply
in either direction.

777
00:50:11 --> 00:50:13
Because the rule
goes both ways.

778
00:50:13 --> 00:50:18
If I have this convolution
to do, I would find the

779
00:50:18 --> 00:50:24
coefficients here, the c's, the
coefficients of d of the g's.

780
00:50:24 --> 00:50:28
I would do this one by one
multiplication, and then I

781
00:50:28 --> 00:50:32
have the Fourier coefficients
of the convolution.

782
00:50:32 --> 00:50:35
Right, OK?
 

783
00:50:35 --> 00:50:37
Do I have a moment?
 

784
00:50:37 --> 00:50:37
Well, hardly.
 

785
00:50:37 --> 00:50:41
Just can I write down what the
formula for a convolution of

786
00:50:41 --> 00:50:44
two functions would look like?
 

787
00:50:44 --> 00:50:46
Sorry, f(x) convolved
with g(x).

788
00:50:48 --> 00:50:50
Let me make it cyclic.
 

789
00:50:50 --> 00:50:53
Just to see what it
would look like.

790
00:50:53 --> 00:50:56
What am I expecting
for that convolution?

791
00:50:56 --> 00:50:59
I'm expecting a function, and
somehow there's going to be an

792
00:50:59 --> 00:51:02
integral instead of a sum,
where I had sums, but

793
00:51:02 --> 00:51:04
now I have integrals.
 

794
00:51:04 --> 00:51:05
And here's the point.
 

795
00:51:05 --> 00:51:10
I'll have f(t) times g(x-t)dt.
 

796
00:51:10 --> 00:51:16
 
 

797
00:51:16 --> 00:51:19
All I'm asking you to look
at is the fact that the

798
00:51:19 --> 00:51:21
way here I had k, and l-k.
 

799
00:51:23 --> 00:51:27
For functions, your eye
sees that right away as

800
00:51:27 --> 00:51:28
a t, and an (x-t)dt.
 

801
00:51:30 --> 00:51:33
These add to the answer.
 

802
00:51:33 --> 00:51:36
These add to the
result, that x.

803
00:51:36 --> 00:51:39
That would be the
cyclical one, yeah.

804
00:51:39 --> 00:51:45
So I could go zero- these are
all periodic functions, so all

805
00:51:45 --> 00:51:46
2pi periods are the same.
 

806
00:51:46 --> 00:51:50
The book will do that properly.
 

807
00:51:50 --> 00:51:54
OK, we've got the filtering
to discuss on Monday.

808
00:51:54 --> 00:51:59
You can see that this
convolution stuff just takes

809
00:51:59 --> 00:52:03
a little new thinking,
but it comes out nicely.