WEBVTT 00:00:00.499 --> 00:00:03.144 The following content is provided under a Creative 00:00:03.144 --> 00:00:03.810 Commons license. 00:00:03.810 --> 00:00:05.518 Your support will help MIT OpenCourseWare 00:00:05.518 --> 00:00:10.405 continue to offer high-quality educational resources for free. 00:00:10.405 --> 00:00:12.530 To make a donation, or to view additional materials 00:00:12.530 --> 00:00:17.080 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:17.080 --> 00:00:20.550 at ocw.mit.edu. 00:00:20.550 --> 00:00:22.950 PROFESSOR STRANG: So, thanks for coming today. 00:00:22.950 --> 00:00:30.370 This is a key lecture in the application of Fourier, 00:00:30.370 --> 00:00:33.090 you could say. 00:00:33.090 --> 00:00:35.640 So convolution is the big word. 00:00:35.640 --> 00:00:39.250 And a major application of convolution 00:00:39.250 --> 00:00:43.200 is filtering, signal processing. 00:00:43.200 --> 00:00:46.530 So we'll develop that application. 00:00:46.530 --> 00:00:48.940 But it's nothing but convolution. 00:00:48.940 --> 00:00:53.860 So the key idea is these convolution rules, 00:00:53.860 --> 00:00:55.340 where they come from. 00:00:55.340 --> 00:00:58.510 And what these new symbols, that's 00:00:58.510 --> 00:01:03.400 the symbol for the convolution of two functions. 00:01:03.400 --> 00:01:05.390 They could be functions here. 00:01:05.390 --> 00:01:09.240 Here they're long vectors of coefficients, 00:01:09.240 --> 00:01:14.240 and so these are the rules. 00:01:14.240 --> 00:01:16.240 So it's just a little bit of algebra. 00:01:16.240 --> 00:01:21.110 But it just is so central to all this subject. 00:01:21.110 --> 00:01:26.290 Signal processing is certainly the most important little thing 00:01:26.290 --> 00:01:27.310 to know. 00:01:27.310 --> 00:01:29.230 That if I multiply two functions, 00:01:29.230 --> 00:01:31.650 so that's where we started last time. 00:01:31.650 --> 00:01:33.790 If I have a function f with Fourier 00:01:33.790 --> 00:01:39.900 coefficients c, and a function g with coefficients d, then, 00:01:39.900 --> 00:01:44.060 oh, wrong. 00:01:44.060 --> 00:01:48.590 I convolve the coefficients, right. 00:01:48.590 --> 00:01:53.900 So f has coefficients c, g has coefficients d. 00:01:53.900 --> 00:01:57.920 So if I multiply the functions I definitely do not just 00:01:57.920 --> 00:02:00.580 multiply each c times the d. 00:02:00.580 --> 00:02:04.640 I do this convolution operation, which we have to remember. 00:02:04.640 --> 00:02:09.790 That's our main first step is to remember what that was about. 00:02:09.790 --> 00:02:13.550 And then this is the other direction. 00:02:13.550 --> 00:02:15.910 So we didn't see this before. 00:02:15.910 --> 00:02:20.980 That I mean, there's always this fantastic symmetry 00:02:20.980 --> 00:02:26.360 between physical space and frequency space. 00:02:26.360 --> 00:02:30.810 And convolution in one is multiplication in the other. 00:02:30.810 --> 00:02:33.240 It's so easy to remember. 00:02:33.240 --> 00:02:36.890 If I multiply in one space, I do a convolution 00:02:36.890 --> 00:02:38.240 in the other space. 00:02:38.240 --> 00:02:41.490 If I do a convolution of functions, 00:02:41.490 --> 00:02:45.500 I do a multiplication of coefficients. 00:02:45.500 --> 00:02:49.510 So that's the rule to know. 00:02:49.510 --> 00:02:52.440 And now if we expand on it, by sort 00:02:52.440 --> 00:02:54.750 of seeing again what it means. 00:02:54.750 --> 00:02:59.780 So let me do a quick repeat of this first step 00:02:59.780 --> 00:03:04.840 to remember what this symbol, convolution symbol, means. 00:03:04.840 --> 00:03:07.790 And then another thing I have to do. 00:03:07.790 --> 00:03:11.380 Here, I'm talking about the infinite case, functions, 00:03:11.380 --> 00:03:15.810 and with a whole infinite sequence of coefficients. 00:03:15.810 --> 00:03:20.530 I've got to do the cyclic case, too. 00:03:20.530 --> 00:03:23.590 Which goes with the discrete transform. 00:03:23.590 --> 00:03:26.630 OK, but let's start with the infinite case 00:03:26.630 --> 00:03:29.160 as we did last time. 00:03:29.160 --> 00:03:31.500 Maybe, here's something. 00:03:31.500 --> 00:03:32.780 Here's a suggestion. 00:03:32.780 --> 00:03:36.480 We had f(x). 00:03:36.480 --> 00:03:40.840 We started with f(x), as the sum -- and remember, 00:03:40.840 --> 00:03:45.050 to have nice formulas, we're doing the complex version -- 00:03:45.050 --> 00:03:47.470 c_k*e^(ikx). 00:03:47.470 --> 00:03:49.350 Let me suggest something. 00:03:49.350 --> 00:03:52.880 Let me write z for e^(ikx). 00:03:52.880 --> 00:04:00.060 So I'm going to just write that at the sum of c_k*z^k. 00:04:00.060 --> 00:04:01.600 So z is e^(ix). 00:04:05.970 --> 00:04:09.300 This is like a good point to make anyway. 00:04:09.300 --> 00:04:14.220 When we have this e^(ikx), it's natural to think of that 00:04:14.220 --> 00:04:18.520 as a complex number on the unit circle. 00:04:18.520 --> 00:04:19.020 Right? 00:04:19.020 --> 00:04:21.180 That's always the message, e to the i 00:04:21.180 --> 00:04:23.390 real is on the unit circle. 00:04:23.390 --> 00:04:25.100 Absolute value one. 00:04:25.100 --> 00:04:29.760 And this is great for periodic functions. 00:04:29.760 --> 00:04:32.870 Because if these functions have period 2pi, 00:04:32.870 --> 00:04:39.750 and if we think of our function as being on the circle, 00:04:39.750 --> 00:04:42.290 it obviously has period 2pi. 00:04:42.290 --> 00:04:44.770 I mean, the picture says, yes. 00:04:44.770 --> 00:04:47.030 If I go 2pi, I come back. 00:04:47.030 --> 00:04:52.625 So x is the angle, right? x is the-- In this picture, 00:04:52.625 --> 00:04:59.050 a little bit unusual maybe, x there would be the angle. 00:04:59.050 --> 00:05:03.320 And it's just a little bit easier to write. 00:05:03.320 --> 00:05:06.630 And maybe it even has an official name, the Z-transform. 00:05:06.630 --> 00:05:07.660 How do you like that? 00:05:07.660 --> 00:05:13.791 You learn a transform just, in fourteen seconds. 00:05:13.791 --> 00:05:14.290 Z-transform. 00:05:17.750 --> 00:05:20.980 It'll make it easier to multiply by g. 00:05:20.980 --> 00:05:26.070 So g is going to be the sum of d_k*z^k. 00:05:26.070 --> 00:05:30.100 So just think of these as long, well I'm 00:05:30.100 --> 00:05:32.400 tempted to say long polynomials. 00:05:32.400 --> 00:05:37.140 I mean, very long, because they can be infinite series. 00:05:37.140 --> 00:05:40.460 Infinite negative powers and positive powers. 00:05:40.460 --> 00:05:44.410 But just think of it as a bunch of powers of z times 00:05:44.410 --> 00:05:46.670 a bunch of powers of z. 00:05:46.670 --> 00:05:51.090 And if you multiply a couple of polynomials, 00:05:51.090 --> 00:05:54.230 you're doing convolution. 00:05:54.230 --> 00:05:56.890 I guess my message is, you've been doing convolution 00:05:56.890 --> 00:05:59.740 since the second grade. 00:05:59.740 --> 00:06:01.380 That's the real message. 00:06:01.380 --> 00:06:04.020 Here, let me show you. 00:06:04.020 --> 00:06:06.530 This is convolution too. 00:06:06.530 --> 00:06:14.550 Suppose I have to multiply a 123 times 456? 00:06:14.550 --> 00:06:17.910 OK, so what do you do? 00:06:17.910 --> 00:06:22.640 Remember back, it's a long way back but we can do this. 00:06:22.640 --> 00:06:25.470 One, two, three times four, five, six. 00:06:25.470 --> 00:06:30.460 So I multiply the three, oh there's a little point. 00:06:30.460 --> 00:06:33.170 Where that second-grade teacher's going to panic. 00:06:33.170 --> 00:06:37.010 I'm going to write that as 18. 00:06:37.010 --> 00:06:41.040 And that's 15, and that's 12. 00:06:41.040 --> 00:06:43.830 Sorry about that, yeah. 00:06:43.830 --> 00:06:48.930 And 12, 10 and 8, right? 00:06:48.930 --> 00:06:50.140 Four, five and six. 00:06:50.140 --> 00:06:53.970 So you see the nine multiplications 00:06:53.970 --> 00:06:55.740 that you have to do? 00:06:55.740 --> 00:06:58.650 Nine multiplications, three times three. 00:06:58.650 --> 00:07:04.994 OK, right now imagine those were, 00:07:04.994 --> 00:07:06.160 they could have been longer. 00:07:06.160 --> 00:07:13.810 But they were finite length filters, we could say. 00:07:13.810 --> 00:07:17.200 And now, what does convolution do? 00:07:17.200 --> 00:07:19.990 Just what you did in multiplication. 00:07:19.990 --> 00:07:23.690 When I add 12, 10 and 6, what am I doing? 00:07:23.690 --> 00:07:26.580 I'm putting together the three times the four, 00:07:26.580 --> 00:07:29.340 the two times the five, and the one times the six. 00:07:29.340 --> 00:07:31.510 That's what convolution, those are the things 00:07:31.510 --> 00:07:33.220 that convolution puts together. 00:07:33.220 --> 00:07:40.710 So we got an 18, a 27, 28. 00:07:40.710 --> 00:07:45.020 13, and 4. 00:07:45.020 --> 00:07:49.750 So I guess I'm saying that, alright, here's 00:07:49.750 --> 00:07:51.110 what I really want to say. 00:07:51.110 --> 00:07:56.600 I want to say that the convolution of those, 00:07:56.600 --> 00:08:04.010 with these, four, five, six, is this sequence here-- Oh. 00:08:04.010 --> 00:08:04.980 Yeah, that's right. 00:08:04.980 --> 00:08:09.430 4, 13, 28. 00:08:09.430 --> 00:08:13.020 27, and 18. 00:08:13.020 --> 00:08:15.060 If you just look at that. 00:08:15.060 --> 00:08:17.690 Where did that 13 come from? 00:08:17.690 --> 00:08:21.360 Let's just remember, where does that 13 come from? 00:08:21.360 --> 00:08:25.160 That came from, this was z^0. 00:08:25.160 --> 00:08:29.890 This 13 is 13 z to the first power, right? 00:08:29.890 --> 00:08:33.100 We're just checking all the powers here. 00:08:33.100 --> 00:08:35.520 There's 13 z to the first power. 00:08:35.520 --> 00:08:37.220 Where do we get a first power? 00:08:37.220 --> 00:08:41.620 We get a z^0 times 5z^1. 00:08:41.620 --> 00:08:43.460 So that's 5z^1. 00:08:43.460 --> 00:08:49.520 And we also have 2z^1, times 4z^0. 00:08:49.520 --> 00:08:50.040 Right? 00:08:50.040 --> 00:08:52.170 Two times four gave the eight. 00:08:52.170 --> 00:08:55.070 So the eight from there and the five from there 00:08:55.070 --> 00:08:56.650 produce that 13. 00:08:56.650 --> 00:09:00.120 And that's just what you did in multiplication. 00:09:00.120 --> 00:09:01.630 Right? 00:09:01.630 --> 00:09:06.620 So that's multiplication of two series. 00:09:06.620 --> 00:09:08.490 That's not cyclic. 00:09:08.490 --> 00:09:11.010 This is definitely not yet cyclic, 00:09:11.010 --> 00:09:13.750 but we'll make it cyclic in a minute. 00:09:13.750 --> 00:09:20.720 This is the infinite one, except that we had all zeroes beyond. 00:09:20.720 --> 00:09:24.500 OK, so that if you, in non-cyclic convolution like 00:09:24.500 --> 00:09:29.280 this, if I have length m and length n, 00:09:29.280 --> 00:09:34.790 then I get length m+n, maybe m+n-1. 00:09:34.790 --> 00:09:37.830 OK, so that's convolution. 00:09:37.830 --> 00:09:40.590 Without carrying numbers. 00:09:40.590 --> 00:09:42.190 Without doing it right. 00:09:42.190 --> 00:09:46.360 OK, so and what does that correspond to? 00:09:46.360 --> 00:09:48.770 Let me just, so you see it every way. 00:09:48.770 --> 00:09:57.410 That corresponds to 1+2z+3z^2, multiplying 4+5z+6z^2. 00:10:00.160 --> 00:10:02.830 And it gave-- That's times. 00:10:02.830 --> 00:10:10.160 And it gave this thing up to 18z^4. 00:10:10.160 --> 00:10:14.400 Just, exactly the multiplication that you've always done. 00:10:14.400 --> 00:10:17.380 OK, so that's the idea. 00:10:17.380 --> 00:10:19.510 Over on that board I'm going to put a formula 00:10:19.510 --> 00:10:22.230 for this convolution operation. 00:10:22.230 --> 00:10:27.740 But my point on this board is, you've done it always. 00:10:27.740 --> 00:10:29.900 When you multiply a couple of polynomials, 00:10:29.900 --> 00:10:31.930 you collect powers. 00:10:31.930 --> 00:10:34.760 And that's all convolution is doing, collecting 00:10:34.760 --> 00:10:36.820 each power separately. 00:10:36.820 --> 00:10:38.120 OK, let's do it. 00:10:38.120 --> 00:10:46.950 So then f(x)g(x) is, when I multiply that polynomial 00:10:46.950 --> 00:10:53.330 or series by that polynomial, I get some polynomial in, with 00:10:53.330 --> 00:10:56.890 coefficients, oh I was changed to l, 00:10:56.890 --> 00:11:00.640 just to have a different symbol there. 00:11:00.640 --> 00:11:03.270 And what was the formula for l? 00:11:03.270 --> 00:11:05.620 For h_l? 00:11:05.620 --> 00:11:10.780 What is the coefficient of z^l? 00:11:10.780 --> 00:11:15.230 If I multiply that by that, do you remember the story there? 00:11:15.230 --> 00:11:21.100 When I multiply that by that and I looked for the terms that 00:11:21.100 --> 00:11:23.580 gave me z^l? 00:11:23.580 --> 00:11:27.350 OK, well that means that this power times this power is going 00:11:27.350 --> 00:11:29.220 to be z^l. 00:11:29.220 --> 00:11:33.030 So that the index, do you remember what happened? 00:11:33.030 --> 00:11:38.160 It was a lot of different, just the way-- This h_l is here. 00:11:38.160 --> 00:11:41.750 Here's h_2 or something. 00:11:41.750 --> 00:11:43.990 I've got to do an addition. 00:11:43.990 --> 00:11:49.730 Because a bunch of c's come in with different d's, and what's 00:11:49.730 --> 00:11:53.820 the deal then? c_k comes in with which d? 00:11:53.820 --> 00:11:56.820 This is the magic number there. 00:11:56.820 --> 00:12:04.420 What's the subscript that if I look at the coefficient of z^l, 00:12:04.420 --> 00:12:07.760 I look at each of these. 00:12:07.760 --> 00:12:13.240 And then I pick out the one of these that will give me a z^l. 00:12:13.240 --> 00:12:16.690 And which one is it? 00:12:16.690 --> 00:12:18.550 d_(l-k). 00:12:18.550 --> 00:12:22.690 It's that magic quantity that the eye spots perfectly. 00:12:22.690 --> 00:12:32.850 k and l-k, adding to l simply because z^k z^(l-k) multiplies 00:12:32.850 --> 00:12:35.600 to z^l. 00:12:35.600 --> 00:12:36.880 Same thing. 00:12:36.880 --> 00:12:37.540 Right? 00:12:37.540 --> 00:12:43.930 OK, this is, now I'll use that notation. 00:12:43.930 --> 00:12:47.800 This is c convolved with the d. c 00:12:47.800 --> 00:12:50.640 convolved with d is h. c convolved with d is h. 00:12:50.640 --> 00:12:55.810 So this is the l-th component. c convolved with d is my symbol. 00:12:55.810 --> 00:12:58.450 This h is the convolution. 00:12:58.450 --> 00:13:01.660 And that's the convolution rule. 00:13:01.660 --> 00:13:04.720 It's just whatever operation you have 00:13:04.720 --> 00:13:06.990 to do to get the right answer. 00:13:06.990 --> 00:13:11.460 The right answer when you multiply. 00:13:11.460 --> 00:13:13.540 So that one. 00:13:13.540 --> 00:13:16.640 Ready for the discrete case? 00:13:16.640 --> 00:13:18.200 The finite case? 00:13:18.200 --> 00:13:25.270 The case when, you have power, when z becomes w. 00:13:25.270 --> 00:13:27.040 The discrete case. 00:13:27.040 --> 00:13:28.850 The cyclic case, sorry. 00:13:28.850 --> 00:13:33.950 Maybe emphasize the cyclic case, meaning it circles around, 00:13:33.950 --> 00:13:39.730 is the case when z becomes this very special z, on the unit 00:13:39.730 --> 00:13:42.390 circle that we know as w. 00:13:42.390 --> 00:13:47.510 OK, and we have to say, and of course it's w_N, 00:13:47.510 --> 00:13:54.730 I have to tell you in the cyclic case how long the cycle is. 00:13:54.730 --> 00:13:57.840 So this would be a case. 00:13:57.840 --> 00:14:02.590 Watch what you do here to make this cyclic. 00:14:02.590 --> 00:14:09.990 OK, I have three inputs, so N is three here. 00:14:09.990 --> 00:14:13.150 Now I'm going to do the cyclic. 00:14:13.150 --> 00:14:16.190 So instead of z's, I should be putting w's. 00:14:16.190 --> 00:14:18.420 I will. 00:14:18.420 --> 00:14:23.240 Just to emphasize, it's good to think of it with the w there, 00:14:23.240 --> 00:14:27.510 because the w has this special property that's critical 00:14:27.510 --> 00:14:31.750 to everything, OK so now I'm going to, I think of this 00:14:31.750 --> 00:14:38.230 as 1w^0, 2w^1, and 3 w squared. 00:14:38.230 --> 00:14:40.800 I'm thinking of the same multiplication here, 00:14:40.800 --> 00:14:45.570 but now w's. 00:14:45.570 --> 00:14:51.540 OK, so I'll end up with 18w^4, and four was the constant 00:14:51.540 --> 00:14:55.700 and 13 w's and so on. 00:14:55.700 --> 00:14:59.910 All these numbers. 00:14:59.910 --> 00:15:01.940 What's the difference? 00:15:01.940 --> 00:15:04.070 Ready for the key point? 00:15:04.070 --> 00:15:07.340 Now, what's happened in this cyclic case? 00:15:07.340 --> 00:15:12.010 Well, the difference is what is w^4? 00:15:12.010 --> 00:15:15.740 If we're in the cyclic case, N is three now, 00:15:15.740 --> 00:15:19.420 our guys have length three, our circle 00:15:19.420 --> 00:15:24.270 is, w now is 1/3 of the way around. 00:15:24.270 --> 00:15:27.860 So that's my w, here's my w squared. 00:15:27.860 --> 00:15:29.640 Here is my one. 00:15:29.640 --> 00:15:32.850 But here it is also w cubed. 00:15:32.850 --> 00:15:39.900 So w is the same as w^4, w^2 is the same as w^5. 00:15:39.900 --> 00:15:44.320 So, what's the difference? 00:15:44.320 --> 00:15:48.440 What can I do now? 00:15:48.440 --> 00:15:54.300 If I'm in this discrete case, then my inputs 00:15:54.300 --> 00:15:58.030 are a vector of length N, three. 00:15:58.030 --> 00:16:00.380 A vector of length N, three. 00:16:00.380 --> 00:16:06.180 And I want to get out to a vector of length three. 00:16:06.180 --> 00:16:10.180 I'm not happy with that in the cyclic case, 00:16:10.180 --> 00:16:13.920 because I'm not happy with w^4. 00:16:13.920 --> 00:16:17.790 So now tell me again that last, when I do the multiplication 00:16:17.790 --> 00:16:21.920 and I just do it, there's no difference except in how 00:16:21.920 --> 00:16:23.710 I write the answer. 00:16:23.710 --> 00:16:35.010 18w^4 is the same as? w. w^4 is the same as w, when N is three. 00:16:35.010 --> 00:16:41.290 So that 18w^4 cycles back in, with this 13. 00:16:41.290 --> 00:16:45.650 So now if I do the, can I show you the symbol? 00:16:45.650 --> 00:16:47.730 I'll just do a little circle there. 00:16:47.730 --> 00:16:52.130 To say this is now the cyclic convolution. 00:16:52.130 --> 00:16:54.620 Then this isn't the answer any more. 00:16:54.620 --> 00:16:57.090 The answer now for cyclic convolution, 00:16:57.090 --> 00:16:59.190 I only want three numbers. 00:16:59.190 --> 00:17:01.600 And if you can tell me what those three numbers are, 00:17:01.600 --> 00:17:04.250 we've got it. 00:17:04.250 --> 00:17:07.240 Move those over a little to make room for the three numbers. 00:17:07.240 --> 00:17:11.940 So there's the answer, the space for the answer. 00:17:11.940 --> 00:17:14.430 What do I write in? 00:17:14.430 --> 00:17:19.750 How many, what's the constant term? 00:17:19.750 --> 00:17:21.460 It's 31, right. 00:17:21.460 --> 00:17:22.410 It's 31. 00:17:22.410 --> 00:17:25.290 Where did 31 come from? 00:17:25.290 --> 00:17:31.540 It came from, you could say, cycling that 27w^3 back with 00:17:31.540 --> 00:17:33.000 the four. 00:17:33.000 --> 00:17:36.250 Because there's no, w^3 is the same as one. 00:17:36.250 --> 00:17:40.730 So I've gone around the circle when I come around to w^3. 00:17:40.730 --> 00:17:45.820 So that 27 and 4 combined into that 31. 00:17:45.820 --> 00:17:47.930 And let's see. 00:17:47.930 --> 00:17:50.130 So what multiplications am I doing here? 00:17:50.130 --> 00:17:53.280 One times four gives me the constant. 00:17:53.280 --> 00:18:00.520 Two times six, that's two w's and six w squareds, 00:18:00.520 --> 00:18:03.870 that's 12w^3, that's 12. 00:18:03.870 --> 00:18:10.880 And then 3 w squareds and five w's is 15w^3. 00:18:10.880 --> 00:18:13.330 But that's the same as 15. 00:18:13.330 --> 00:18:15.330 So that's why we get 31. 00:18:15.330 --> 00:18:20.670 We got 4, we've got 12 and we've got 15. 00:18:20.670 --> 00:18:25.350 OK, now what's the second component, the w component 00:18:25.350 --> 00:18:32.510 of the cyclic convolution? 00:18:32.510 --> 00:18:36.120 Tell me what number do I write in, in that middle position? 00:18:36.120 --> 00:18:39.100 How many w's do I have? 00:18:39.100 --> 00:18:41.300 31 again. 00:18:41.300 --> 00:18:47.050 This 28, this 18 is coming back by three. 00:18:47.050 --> 00:18:48.920 To 13. 00:18:48.920 --> 00:18:53.010 You see, I could have done that multiplication. 00:18:53.010 --> 00:18:57.540 So coming back to 31, am I going to get another 31? 00:18:57.540 --> 00:19:01.030 No, what's the w^2 guy? 00:19:01.030 --> 00:19:01.830 28. 00:19:01.830 --> 00:19:02.330 Yeah. 00:19:02.330 --> 00:19:09.730 28, because there's no w to the fifth to come back. 00:19:09.730 --> 00:19:15.890 So 28 uses three multiplications. 00:19:15.890 --> 00:19:21.320 The 31 there used the four and these two. 00:19:21.320 --> 00:19:24.630 This came back over to here. 00:19:24.630 --> 00:19:28.640 And this 31 used, this 18 came back. 00:19:28.640 --> 00:19:30.920 I could have put the 18 here. 00:19:30.920 --> 00:19:33.450 You know, I could have lined it up just three. 00:19:33.450 --> 00:19:35.470 So I'll write a formula for it. 00:19:35.470 --> 00:19:37.790 So that's the answer. 00:19:37.790 --> 00:19:41.610 31, 31, 28. 00:19:41.610 --> 00:19:44.740 Could I just suggest a little check on that? 00:19:44.740 --> 00:19:46.460 Just to check on the numbers. 00:19:46.460 --> 00:19:52.590 I think that if I add up these numbers, I get six. 00:19:52.590 --> 00:19:57.770 And if I add up those numbers I get 15. 00:19:57.770 --> 00:20:01.180 And if I multiply that, I get 90. 00:20:01.180 --> 00:20:04.410 And if I add those numbers I get 90, right? 00:20:04.410 --> 00:20:08.440 So those add to 90. 00:20:08.440 --> 00:20:12.820 So I'm just saying the miracle check is add these, 00:20:12.820 --> 00:20:15.040 multiply by the sum of those. 00:20:15.040 --> 00:20:17.720 And you get the sum of those. 00:20:17.720 --> 00:20:20.700 Why is that? 00:20:20.700 --> 00:20:23.480 Somehow seems right, doesn't it? 00:20:23.480 --> 00:20:29.790 Because somehow I've taken all nine products here, and so 00:20:29.790 --> 00:20:33.460 when I add all the results, I'll have the sum of all nine 00:20:33.460 --> 00:20:35.220 of these possible products. 00:20:35.220 --> 00:20:37.590 So I'll have 6 times 15. 00:20:37.590 --> 00:20:41.310 Actually, here's a good way to look at it. 00:20:41.310 --> 00:20:43.730 In doing this, I just set w=1. 00:20:48.620 --> 00:20:51.840 I just set w=1 in the polynomial. 00:20:51.840 --> 00:20:56.220 When I set w to one, this becomes six, this becomes 15 00:20:56.220 --> 00:20:59.540 and the answer becomes 90, when w is one. 00:20:59.540 --> 00:21:00.040 Yeah. 00:21:00.040 --> 00:21:02.910 So that's another way to see. 00:21:02.910 --> 00:21:07.420 And actually, this multiplication, 00:21:07.420 --> 00:21:13.400 the second grade version, had w, well, 00:21:13.400 --> 00:21:19.600 I'm almost going to say w=10, but not quite that. 00:21:19.600 --> 00:21:22.540 Because it's written in the opposite order, right? 00:21:22.540 --> 00:21:25.290 If w was ten, this would be one. 00:21:25.290 --> 00:21:28.270 Well, anyway, w can't be ten, it's got 00:21:28.270 --> 00:21:31.580 to stay on the unit circle, so. 00:21:31.580 --> 00:21:35.760 So somewhere in the non-cyclic case, it's something like w=10, 00:21:35.760 --> 00:21:38.090 or w=1/10, maybe. 00:21:38.090 --> 00:21:39.040 Whatever. 00:21:39.040 --> 00:21:40.190 OK. 00:21:40.190 --> 00:21:42.670 Could you take the convolution now, 00:21:42.670 --> 00:21:45.270 let me give you just another example. 00:21:45.270 --> 00:21:49.700 Do it mentally. 00:21:49.700 --> 00:21:55.560 What's the convolution of [0, 1, 0, 0], 00:21:55.560 --> 00:21:57.990 let me make a little longer. 00:21:57.990 --> 00:22:02.200 So I take the, first of all, the non-cyclic convolution 00:22:02.200 --> 00:22:05.810 of [0, 0, 1, 0]. 00:22:05.810 --> 00:22:07.010 OK. 00:22:07.010 --> 00:22:11.780 What's the ordinary-- how long is the answer now? 00:22:11.780 --> 00:22:13.320 This is just practice. 00:22:13.320 --> 00:22:15.590 How many components am I going to have 00:22:15.590 --> 00:22:22.650 in the ordinary convolution of those two guys of length four? 00:22:22.650 --> 00:22:24.930 I think it's seven. 00:22:24.930 --> 00:22:26.610 I think it'll be seven. 00:22:26.610 --> 00:22:34.250 Because we'll have here one, we have no, we have z^0, z^1, z^2, 00:22:34.250 --> 00:22:35.910 z^3. 00:22:35.910 --> 00:22:39.450 And here we'll have again the same, it would go up to z^6, 00:22:39.450 --> 00:22:43.020 but remember there's a z^0, so that's why we have seven. 00:22:43.020 --> 00:22:47.440 And what will it be? 00:22:47.440 --> 00:22:49.600 What will it be? 00:22:49.600 --> 00:22:53.020 I guess, actually, this wasn't a brilliant example, was it? 00:22:53.020 --> 00:22:55.970 But let's finish it. 00:22:55.970 --> 00:23:01.330 So I'm just multiplying z by z squared, so 00:23:01.330 --> 00:23:02.650 what do you get for an answer? 00:23:02.650 --> 00:23:13.600 I think the one shows up in the z cubed. 00:23:13.600 --> 00:23:14.970 Is that right? 00:23:14.970 --> 00:23:22.000 Yeah. z to the first power here, z squared here, z cubed here, 00:23:22.000 --> 00:23:26.490 and we always have to remember everything in Chapter 4 starts 00:23:26.490 --> 00:23:27.640 at zero. 00:23:27.640 --> 00:23:30.340 Zero, z^0's the first one. 00:23:30.340 --> 00:23:33.030 OK, that's not too great an example, 00:23:33.030 --> 00:23:40.580 because what happens if I do the circular cyclic convolution? 00:23:40.580 --> 00:23:42.500 What would be the cyclic convolution? 00:23:42.500 --> 00:23:47.150 Now I'm expecting four guys only, right? 00:23:47.150 --> 00:23:50.250 The cyclic keeps the same length. 00:23:50.250 --> 00:23:55.150 And what would be the answer? 00:23:55.150 --> 00:23:58.330 Well, there's nobody to fold back, so it would be just 00:23:58.330 --> 00:23:59.910 [0, 0, 0, 1]. 00:23:59.910 --> 00:24:06.210 So let me update this a little bit with a one here. 00:24:06.210 --> 00:24:08.090 OK, just to practice. 00:24:08.090 --> 00:24:12.000 So suppose I do that convolution. 00:24:12.000 --> 00:24:15.850 Un-cyclic, first. 00:24:15.850 --> 00:24:18.220 What do I change here now? 00:24:18.220 --> 00:24:21.780 I've now got a z squared and I've also got a z cubed, 00:24:21.780 --> 00:24:25.170 but I only have a single one there. 00:24:25.170 --> 00:24:27.330 Let's make it a little more interesting. 00:24:27.330 --> 00:24:30.930 OK, make it like so. 00:24:30.930 --> 00:24:36.530 Alright, z+z^2 is what we're looking at, z+z^2 is 00:24:36.530 --> 00:24:39.820 multiplying z^2+z^3. 00:24:39.820 --> 00:24:43.870 And in the long form, what do I get? 00:24:43.870 --> 00:24:47.920 Let me make space, tell me what numbers to put in. 00:24:47.920 --> 00:24:52.140 If I multiply z+z^2 times z^2+z^3, I get what? 00:24:52.140 --> 00:24:54.420 1z^3, how many z^4? 00:24:57.070 --> 00:24:58.220 Two of them. 00:24:58.220 --> 00:25:00.240 How many z^5? 00:25:00.240 --> 00:25:01.760 One, and nobody there. 00:25:01.760 --> 00:25:02.890 OK. 00:25:02.890 --> 00:25:09.600 And now the cyclic version would be what? 00:25:09.600 --> 00:25:12.540 What's my answer now for the cyclic version? 00:25:12.540 --> 00:25:15.850 Let me take those out. 00:25:15.850 --> 00:25:21.680 So the cyclic version would bring the two back to the zero. 00:25:21.680 --> 00:25:24.320 Would bring that one back so there. 00:25:24.320 --> 00:25:27.440 That zero will still be zero. 00:25:27.440 --> 00:25:32.070 And I checked that I haven't missed anything 00:25:32.070 --> 00:25:35.760 by adding those up to get four, and adding this up 00:25:35.760 --> 00:25:37.610 to get two times two. 00:25:37.610 --> 00:25:38.660 Yeah. 00:25:38.660 --> 00:25:42.700 OK, so that's the rule. 00:25:42.700 --> 00:25:47.330 And it's a lot cleaner to see these answers 00:25:47.330 --> 00:25:52.770 than to see this formula. 00:25:52.770 --> 00:25:57.390 And I need, actually of course, now mentioning that formula, 00:25:57.390 --> 00:25:59.640 I need a cyclic formula. 00:25:59.640 --> 00:26:03.080 So can I write above it the cyclic formula? 00:26:03.080 --> 00:26:12.580 What do I get when I'm, instead of this sum, which 00:26:12.580 --> 00:26:17.390 went from k equal minus infinity to infinity, 00:26:17.390 --> 00:26:25.000 in the cyclic case, h_k is just going to be a sum from zero 00:26:25.000 --> 00:26:33.720 to N-1, and there'll be a c_k, and a d something. 00:26:33.720 --> 00:26:42.620 And now this is the cyclic case, so I guess this makes us, 00:26:42.620 --> 00:26:45.460 I think what our situation now is we understand 00:26:45.460 --> 00:26:47.620 the cyclic case from examples. 00:26:47.620 --> 00:26:52.590 And now we just have the job of how do I put it into algebra. 00:26:52.590 --> 00:26:55.270 How do I put it into symbols? 00:26:55.270 --> 00:27:02.280 What's the point? c_k d_n, let's say. 00:27:02.280 --> 00:27:08.780 But oh no, I'm looking for h_l, yeah. 00:27:08.780 --> 00:27:10.260 So what's the deal? 00:27:10.260 --> 00:27:14.780 Here l was k. 00:27:14.780 --> 00:27:17.900 That one is the sum of that one and that one. 00:27:17.900 --> 00:27:21.630 So here, l is the sum of k and n, but. 00:27:21.630 --> 00:27:28.720 What's the but? 00:27:28.720 --> 00:27:35.360 I mean somehow I've got some wraparound to do, right? 00:27:35.360 --> 00:27:39.200 When I'm doing the cyclic multiplication and I'm doing 00:27:39.200 --> 00:27:47.890 the wraparound because w^N, the wraparound comes from that, 00:27:47.890 --> 00:27:49.490 right? 00:27:49.490 --> 00:27:52.710 That's why I never get as high as N, because when I get to N 00:27:52.710 --> 00:27:55.460 I go back to the zeroth power. 00:27:55.460 --> 00:28:02.960 OK, so what's the relation of k and n and l here? 00:28:02.960 --> 00:28:07.440 We just need the right word to express it. 00:28:07.440 --> 00:28:08.740 What's the word? 00:28:08.740 --> 00:28:10.770 Mod. 00:28:10.770 --> 00:28:17.360 So that's the word I'm looking for. k+n is l. 00:28:17.360 --> 00:28:21.320 With wraparound and wraparound means, 00:28:21.320 --> 00:28:35.590 the nice notation that people use is mod N. Let's practice. 00:28:35.590 --> 00:28:45.670 What is two plus two mod seven? 00:28:45.670 --> 00:28:47.400 Four. 00:28:47.400 --> 00:28:49.950 Two plus two is four, even in 18.085. 00:28:49.950 --> 00:28:51.360 Right, OK. 00:28:51.360 --> 00:29:03.570 But two plus two mod three is, two plus two mod three is? 00:29:03.570 --> 00:29:04.120 One. 00:29:04.120 --> 00:29:06.520 Everybody sees it? 00:29:06.520 --> 00:29:13.690 I'm taking z^2 times z^2, z^4, but I'm doing with N=3, 00:29:13.690 --> 00:29:18.020 so z^3 is one, so that z^4 is really just z to the first 00:29:18.020 --> 00:29:20.080 power. 00:29:20.080 --> 00:29:27.300 So this is the little nifty notation 00:29:27.300 --> 00:29:29.450 that says make it cyclic. 00:29:29.450 --> 00:29:34.860 Bring it back so that l only has the values here, 00:29:34.860 --> 00:29:38.750 zero up to N-1. 00:29:38.750 --> 00:29:41.410 And then stops. 00:29:41.410 --> 00:29:45.070 OK. 00:29:45.070 --> 00:29:50.830 OK, so we'll have more practice with examples 00:29:50.830 --> 00:29:56.410 when we do some filtering. 00:29:56.410 --> 00:29:58.770 Have you got that fundamental-- So we've 00:29:58.770 --> 00:30:03.490 talked about this rule one here. f times g 00:30:03.490 --> 00:30:07.530 goes to those coefficients. 00:30:07.530 --> 00:30:10.160 And if it's the cyclic case then I 00:30:10.160 --> 00:30:12.460 put a circle around that star. 00:30:12.460 --> 00:30:16.450 And I do the wraparound. 00:30:16.450 --> 00:30:23.750 But it's just, it's the Z-transform, 00:30:23.750 --> 00:30:28.590 it's polynomials in z or polynomials in w, 00:30:28.590 --> 00:30:32.360 and when it's polynomials in w, you use that special property 00:30:32.360 --> 00:30:34.700 that w^N is one. 00:30:34.700 --> 00:30:38.150 Yeah, OK. 00:30:38.150 --> 00:30:43.420 Now, I see I've written another line, there. 00:30:43.420 --> 00:30:46.390 That I could convolve functions. 00:30:46.390 --> 00:30:48.520 Let me do a couple more examples. 00:30:48.520 --> 00:30:50.390 Couple of examples. 00:30:50.390 --> 00:30:59.200 First, before I go to that line, OK. 00:30:59.200 --> 00:31:05.520 So I'm up to this line. 00:31:05.520 --> 00:31:12.640 A couple of examples here. 00:31:12.640 --> 00:31:18.840 Let's see, what example would I want to do? 00:31:18.840 --> 00:31:28.270 Let's see, OK, I want to do one example with a delta function. 00:31:28.270 --> 00:31:31.080 One example with a delta function. 00:31:31.080 --> 00:31:32.870 One example with the delta vector. 00:31:32.870 --> 00:31:41.890 Yeah, let me take the function g(x) identically one. 00:31:41.890 --> 00:31:45.820 OK, constant function. 00:31:45.820 --> 00:31:48.000 In this rule. 00:31:48.000 --> 00:31:50.300 I want to see what happens with the rule. 00:31:50.300 --> 00:32:00.140 OK, then f(x) g(x) is the same as f(x), right? 00:32:00.140 --> 00:32:03.770 Because this function g(x) is so simple, it's just one. 00:32:03.770 --> 00:32:09.520 Now, what about the coefficient? 00:32:09.520 --> 00:32:13.480 So I have the coefficients of c, what are the Fourier 00:32:13.480 --> 00:32:15.370 coefficients, what are the d's? 00:32:15.370 --> 00:32:18.900 Ah yes, what are the d's? 00:32:18.900 --> 00:32:22.840 So I'm testing my rule on a really really simple case, 00:32:22.840 --> 00:32:25.600 g(x) identically one. 00:32:25.600 --> 00:32:27.970 What, you have to tell me, in order 00:32:27.970 --> 00:32:30.490 to check the right side of the rule 00:32:30.490 --> 00:32:34.440 you have to tell me the Fourier coefficients for that very 00:32:34.440 --> 00:32:35.530 special function. 00:32:35.530 --> 00:32:38.270 What would be the Fourier coefficients? 00:32:38.270 --> 00:32:42.260 If I expand the function one in a Fourier series, 00:32:42.260 --> 00:32:44.000 what do I see? 00:32:44.000 --> 00:32:44.890 I see a one. 00:32:44.890 --> 00:32:47.240 Yeah, that's it, I see one. 00:32:47.240 --> 00:32:54.570 So what are its coefficients? d_0, right, is one? 00:32:54.570 --> 00:33:01.610 And the other d's are all zero, right? 00:33:01.610 --> 00:33:07.580 So my vector of d, my vector of d's is a whole lot 00:33:07.580 --> 00:33:09.600 of zeroes on the negative side. 00:33:09.600 --> 00:33:12.780 A one right there in the center, and then a lot of zeroes. 00:33:12.780 --> 00:33:24.770 And now I want to convolve that with c. 00:33:24.770 --> 00:33:27.450 I'm practicing the convolution rule 00:33:27.450 --> 00:33:34.590 on a case that's so simple it's confusing, right? 00:33:34.590 --> 00:33:45.570 I mean, it's a big mess, this multiplication. 00:33:45.570 --> 00:33:47.070 What do I get out of this? 00:33:47.070 --> 00:33:58.280 If d is this vector, if d has this property that d_0 is one 00:33:58.280 --> 00:34:03.450 and others are zero, all others are zero, 00:34:03.450 --> 00:34:06.250 so this is my little example, what 00:34:06.250 --> 00:34:10.940 does this sum boil down to? 00:34:10.940 --> 00:34:17.900 Well, I only get something when l=k, right? 00:34:17.900 --> 00:34:22.760 I only got something when l=k, because then I have d_0 00:34:22.760 --> 00:34:25.560 and that's the only d that's around. 00:34:25.560 --> 00:34:29.430 So in this sum, something happens only 00:34:29.430 --> 00:34:31.970 when k and l are the same. 00:34:31.970 --> 00:34:33.160 And then what happens? 00:34:33.160 --> 00:34:37.190 Then I have a one, I have c_l, and that's h_l, 00:34:37.190 --> 00:34:39.280 so that's all I'm concluding then. 00:34:39.280 --> 00:34:43.210 That this h is the same as c. 00:34:43.210 --> 00:34:48.210 I'm sorry, it's so dumb. 00:34:48.210 --> 00:34:51.570 My point is that in convolution, this is 00:34:51.570 --> 00:34:54.450 the thing that acts like one. 00:34:54.450 --> 00:34:56.480 Because in multiplication, that's 00:34:56.480 --> 00:34:59.380 the thing, that's the function that acts like one. 00:34:59.380 --> 00:35:01.890 That's the function that is one. 00:35:01.890 --> 00:35:11.640 So this is the one in-- Oh, would you allow me to do this? 00:35:11.640 --> 00:35:18.820 I'm going to create a matrix with these d's. 00:35:18.820 --> 00:35:24.560 There's another way to see convolution. 00:35:24.560 --> 00:35:26.680 Yeah, there's another way to see convolution 00:35:26.680 --> 00:35:28.490 and discrete convolution. 00:35:28.490 --> 00:35:31.090 Maybe the discrete one's the better. 00:35:31.090 --> 00:35:36.860 Yeah can you stand one more way to write the formula? 00:35:36.860 --> 00:35:40.720 One more way to write, now I'm going to do, 00:35:40.720 --> 00:35:47.940 I'm going to do discrete convolution. 00:35:47.940 --> 00:35:51.720 Discrete cyclic. 00:35:51.720 --> 00:35:53.010 So how am I going to write it? 00:35:53.010 --> 00:35:57.210 I'm going to write it by a matrix multiplication. 00:35:57.210 --> 00:35:58.905 Because you know that in this course 00:35:58.905 --> 00:36:01.190 a matrix was going to show up. 00:36:01.190 --> 00:36:06.290 So it's going to be a matrix multiplication. 00:36:06.290 --> 00:36:08.780 So I just have to tell you the matrix, 00:36:08.780 --> 00:36:12.300 so this is going to be some matrix. 00:36:12.300 --> 00:36:15.020 Let me take N to be four. 00:36:15.020 --> 00:36:20.240 So then I have, you watch. 00:36:20.240 --> 00:36:32.080 So I have four d's, and the output is the four h's. 00:36:32.080 --> 00:36:37.350 And the rule I'm following is this rule. 00:36:37.350 --> 00:36:43.660 Is this, the same old rule but with the cyclic part. 00:36:43.660 --> 00:36:47.070 And now I want to show you the matrix that'll just do this. 00:36:47.070 --> 00:36:54.120 Look, I've put the c's in the first column. 00:36:54.120 --> 00:37:00.340 And then I go, yeah, here's another. 00:37:00.340 --> 00:37:07.380 So it's a cyclic matrix. 00:37:07.380 --> 00:37:09.230 So let me finish it up. 00:37:09.230 --> 00:37:13.240 It's going to be four by four, it's going to be cyclic. 00:37:13.240 --> 00:37:18.340 So I have a c_0, c_0, c_0, c_0 on the diagonal. 00:37:18.340 --> 00:37:21.315 That's fine, that's because z to the zero 00:37:21.315 --> 00:37:24.740 is multiplying all the d's and leaving them in place. 00:37:24.740 --> 00:37:28.200 And then I have c_1's, and then I 00:37:28.200 --> 00:37:31.520 think I come around again here for a c_1. 00:37:31.520 --> 00:37:38.750 And I have c_2's, see where see c_3, c_2. 00:37:38.750 --> 00:37:43.510 And I come around again, to a c_2 and a c_2. 00:37:43.510 --> 00:37:49.490 And c_3 comes around to a c_3, a c_3 and a c_3. 00:37:49.490 --> 00:37:59.400 Well, can you, I hope you can see, this is cyclic matrix. 00:37:59.400 --> 00:38:05.150 It's only got one, it starts with a vector c, 00:38:05.150 --> 00:38:10.640 and those are on the diagonal and the diagonals wrap around. 00:38:10.640 --> 00:38:12.350 That's the other word that you often 00:38:12.350 --> 00:38:16.140 see when you see the word cyclic, wraparound. 00:38:16.140 --> 00:38:23.760 It's because you think of a circle. 00:38:23.760 --> 00:38:25.640 If you go a second time around, it's 00:38:25.640 --> 00:38:28.530 wrapped around the first time. 00:38:28.530 --> 00:38:32.000 OK, just can you look and see that this is the right formula 00:38:32.000 --> 00:38:36.300 for h_0? h_0 is c_0*d_0. 00:38:36.300 --> 00:38:37.950 Where does that come from? 00:38:37.950 --> 00:38:43.230 Remember, h_0 is the coefficient of z^0 in the answer. 00:38:43.230 --> 00:38:47.980 So it comes from c_0*d_0 to the zeroth power in the input. 00:38:47.980 --> 00:38:49.930 And then why is there is a c_3*d_1? 00:38:52.960 --> 00:38:58.450 Why is there a c_3*d_1, and then a c_2*d_2 and then a c_1*d_3 00:38:58.450 --> 00:39:02.630 all piling up into h_0? 00:39:02.630 --> 00:39:04.380 Tell me now, why is there a c_3*d_1? 00:39:07.800 --> 00:39:10.370 Because we're doing mod four is one way to say it. 00:39:10.370 --> 00:39:12.840 Three and one add to four. 00:39:12.840 --> 00:39:20.750 Because c_3 is the w cubed guy, and d_1 is the coefficient 00:39:20.750 --> 00:39:30.790 of w^1 and w^3 times w^1 piles back into the constant. 00:39:30.790 --> 00:39:33.840 And you see the pattern of that matrix? 00:39:33.840 --> 00:39:38.420 So these matrices are very important. 00:39:38.420 --> 00:39:40.620 So they circle around. 00:39:40.620 --> 00:39:45.570 Oh, we've actually met a matrix of this type, 00:39:45.570 --> 00:39:49.920 the first day of 18.085. 00:39:49.920 --> 00:39:51.160 What was that matrix? 00:39:51.160 --> 00:39:53.760 It was one of our four great matrices. 00:39:53.760 --> 00:39:56.650 And now here it is back again. 00:39:56.650 --> 00:40:00.700 Which one was it? 00:40:00.700 --> 00:40:03.430 Well, you remember the letter for it. 00:40:03.430 --> 00:40:05.170 Which isn't going to change. 00:40:05.170 --> 00:40:11.170 And do you remember the particular matrix? 00:40:11.170 --> 00:40:14.320 Well, everybody does remember that matrix, right? 00:40:14.320 --> 00:40:19.510 Twos were on the diagonal, minus ones were on the diagonal, 00:40:19.510 --> 00:40:22.180 and the diagonal curve continued. 00:40:22.180 --> 00:40:26.510 Minus one was on this diagonal and that continued and zeroes 00:40:26.510 --> 00:40:28.120 was on this diagonal. 00:40:28.120 --> 00:40:35.610 So this is cyclic convolution, the circulant matrix, 00:40:35.610 --> 00:40:42.050 cyclic convolution by c, what's the c 00:40:42.050 --> 00:40:46.920 that produces that convolution matrix? 00:40:46.920 --> 00:40:50.720 It's just, it's got-- well there it is. 00:40:50.720 --> 00:40:56.380 The first column is it. 00:40:56.380 --> 00:40:57.350 Right, right. 00:40:57.350 --> 00:41:09.530 And somehow I would say that that's an even vector. 00:41:09.530 --> 00:41:12.190 It's sort of, I associate it with cosine. 00:41:12.190 --> 00:41:13.300 It's an even vector. 00:41:13.300 --> 00:41:18.810 Here is the zero term, and then these are the same, 00:41:18.810 --> 00:41:21.030 not to worry about that part. 00:41:21.030 --> 00:41:25.210 Do you see that we've seen that matrix before? 00:41:25.210 --> 00:41:28.440 And the cyclic convolution means you 00:41:28.440 --> 00:41:32.400 take its second differences, of course. 00:41:32.400 --> 00:41:37.750 We're taking second differences, but everything in our world 00:41:37.750 --> 00:41:40.260 is cyclic. 00:41:40.260 --> 00:41:47.210 So the result, the x_4 is x_0. 00:41:47.210 --> 00:41:49.990 So we're taking second differences -- well, 00:41:49.990 --> 00:41:55.990 maybe I should say d -- we're taking second differences d_i, 00:41:55.990 --> 00:42:02.260 2d_i's, minus d_(i-1) minus d_(i+1). 00:42:02.260 --> 00:42:05.720 I don't know if this is-- So there's a minus one, 00:42:05.720 --> 00:42:07.070 two, minus one. 00:42:07.070 --> 00:42:11.760 And we're cycling around so that d_0 is d_4. 00:42:11.760 --> 00:42:20.180 And d_1 is d_5, and d_(-1) is d_3, whatever. 00:42:20.180 --> 00:42:25.140 OK, I'm just reminding you, we've seen these before. 00:42:25.140 --> 00:42:33.280 OK, so this is another way to remember the formula. 00:42:33.280 --> 00:42:37.950 OK now, can I ask you a practical question? 00:42:37.950 --> 00:42:39.910 A practical question. 00:42:39.910 --> 00:42:47.610 Let me bring back this second grade multiplication. 00:42:47.610 --> 00:42:49.810 Well, I have a granddaughter named Elizabeth, 00:42:49.810 --> 00:42:51.230 I'll have to admit I didn't think 00:42:51.230 --> 00:42:52.590 about mentioning Elizabeth. 00:42:52.590 --> 00:42:54.270 She's six. 00:42:54.270 --> 00:42:59.710 And she delights in sending me long multiplications. 00:42:59.710 --> 00:43:02.570 I mean, really long. 00:43:02.570 --> 00:43:04.570 And then every time I talk to her on the phone, 00:43:04.570 --> 00:43:06.330 she says have you done that one yet? 00:43:06.330 --> 00:43:07.880 And I say, I'm working on it. 00:43:07.880 --> 00:43:11.340 I've got MATLAB at work. 00:43:11.340 --> 00:43:15.260 Because they're ridiculous and I haven't figured out 00:43:15.260 --> 00:43:15.970 how to tell her. 00:43:15.970 --> 00:43:20.230 I mean, she just writes page after page. 00:43:20.230 --> 00:43:23.240 Times 100, plus three, minus seven, 00:43:23.240 --> 00:43:25.510 just whatever she things of. 00:43:25.510 --> 00:43:31.130 OK, now I need help from the convolution rule. here, 00:43:31.130 --> 00:43:31.960 actually. 00:43:31.960 --> 00:43:34.360 So let's suppose that Elizabeth has given me 00:43:34.360 --> 00:43:38.070 a multiplication in which I have a thousand 00:43:38.070 --> 00:43:41.470 digits times a thousand, right. 00:43:41.470 --> 00:43:44.880 Which Mathematica is prepared to do exactly, right? 00:43:44.880 --> 00:43:51.100 MATLAB will mess up, but Mathematica and Maple 00:43:51.100 --> 00:43:55.360 and symbolic packages will do exact computations. 00:43:55.360 --> 00:43:59.770 So what would be the right way, well let may make it 1,024. 00:43:59.770 --> 00:44:08.220 1,024 digits times 1,024 digits. 00:44:08.220 --> 00:44:11.260 Let's do the cyclic version first. 00:44:11.260 --> 00:44:13.340 Elizabeth doesn't know about cyclic. 00:44:13.340 --> 00:44:14.740 Maybe I could teach her that. 00:44:14.740 --> 00:44:18.650 That'll keep her busy while I'm doing the multiplication. 00:44:18.650 --> 00:44:23.040 OK, right. 00:44:23.040 --> 00:44:25.060 Only, her older brother would explain it 00:44:25.060 --> 00:44:26.320 to her, that's the trouble. 00:44:26.320 --> 00:44:30.080 OK, so how am I going to do, or how are you 00:44:30.080 --> 00:44:32.810 going to do on the quiz, multiplication 00:44:32.810 --> 00:44:37.070 of a 1,024 digits times 1,024? 00:44:37.070 --> 00:44:39.880 And I'll make it easy by making it cyclic, 00:44:39.880 --> 00:44:43.310 so I just want 1,024 digits in the answer. 00:44:43.310 --> 00:44:45.420 OK. 00:44:45.420 --> 00:44:48.390 How would you do it? 00:44:48.390 --> 00:44:55.930 Well, before today, you would have just multiplied, right? 00:44:55.930 --> 00:45:00.730 You would have written down 1,024, two lines of 1,024, 00:45:00.730 --> 00:45:03.490 done an addition. 00:45:03.490 --> 00:45:08.090 And you would have had a million multiplications to do. 00:45:08.090 --> 00:45:10.360 But how would you do it now? 00:45:10.360 --> 00:45:14.680 Apart from giving it to Mathematica. 00:45:14.680 --> 00:45:17.760 What's a faster way to do it? 00:45:17.760 --> 00:45:21.450 What's a faster way to do a convolution? 00:45:21.450 --> 00:45:23.460 The fast way to do a convolution is 00:45:23.460 --> 00:45:27.300 to use the convolution rule, go this way. 00:45:27.300 --> 00:45:32.000 So take these numbers, these 1,024 numbers, 00:45:32.000 --> 00:45:35.930 in c and these 1,024 numbers in d, 00:45:35.930 --> 00:45:40.250 and, well what do I have to do? 00:45:40.250 --> 00:45:41.820 I want to use the convolution rule, 00:45:41.820 --> 00:45:47.190 because multiplying is fast. 00:45:47.190 --> 00:45:50.200 Now I've got functions. 00:45:50.200 --> 00:45:52.110 But I'm in the cyclic case. 00:45:52.110 --> 00:45:58.520 So I'm in the cyclic case, so what should I do? 00:45:58.520 --> 00:46:03.020 How can I change this to be the cyclic case? 00:46:03.020 --> 00:46:08.830 This is like f_j g_j. 00:46:08.830 --> 00:46:15.690 So multiplication of components of things in function space 00:46:15.690 --> 00:46:18.170 is convolution of coefficients. 00:46:18.170 --> 00:46:22.130 So now, this is the cyclic. 00:46:22.130 --> 00:46:25.820 So let me make it cyclic. 00:46:25.820 --> 00:46:28.320 So again, what's your problem? 00:46:28.320 --> 00:46:31.740 The problem is to do this cyclic multiplication. 00:46:31.740 --> 00:46:33.440 What's the idea? 00:46:33.440 --> 00:46:41.270 The idea is to transform c back to f, to transform d back to g. 00:46:41.270 --> 00:46:43.510 Do the multiplications, now I have 00:46:43.510 --> 00:46:45.610 only 1,024 multiplications. 00:46:45.610 --> 00:46:47.690 Not 1,024 squared. 00:46:47.690 --> 00:46:49.350 That's the point. 00:46:49.350 --> 00:46:52.710 And if I do this directly, I've got 1,024 00:46:52.710 --> 00:46:55.380 squared multiplications to do. 00:46:55.380 --> 00:46:57.230 Much better. 00:46:57.230 --> 00:47:01.160 Transform back to here, do just 1,024-- 00:47:01.160 --> 00:47:04.430 what's the MATLAB command for that, when you're multiplying 00:47:04.430 --> 00:47:06.630 each component by itself? 00:47:06.630 --> 00:47:09.730 It's not the dot product, notice. 00:47:09.730 --> 00:47:13.300 It's not the dot product because I'm not summing. 00:47:13.300 --> 00:47:14.920 Do you know the MATLAB command, if I 00:47:14.920 --> 00:47:18.950 have a sequence of numbers of numbers, a vector 00:47:18.950 --> 00:47:25.520 f of length 1,024, and I want to get that result? 00:47:25.520 --> 00:47:26.940 What's the result? 00:47:26.940 --> 00:47:32.450 It's a vector of length 1,024 that takes each f times its g. 00:47:32.450 --> 00:47:34.600 But doesn't do any adds. 00:47:34.600 --> 00:47:36.120 That's what's there. 00:47:36.120 --> 00:47:38.880 What's the MATLAB command for that? 00:47:38.880 --> 00:47:40.370 Dot, yeah. 00:47:40.370 --> 00:47:42.620 Dot star, right. 00:47:42.620 --> 00:47:45.680 So that dot says component by component. 00:47:45.680 --> 00:47:48.980 OK, so what's the plan here? 00:47:48.980 --> 00:47:53.100 I do c's back to f. 00:47:53.100 --> 00:48:00.740 By the Fourier matrix. d back to g, by the Fourier matrix, 00:48:00.740 --> 00:48:04.470 then I do a very quick multiplication. 00:48:04.470 --> 00:48:05.520 And then what? 00:48:05.520 --> 00:48:07.440 Then I mustn't forget. 00:48:07.440 --> 00:48:11.830 That I'm in frequency space, and what do I have to do? 00:48:11.830 --> 00:48:16.680 I've got to get back into coefficient space. 00:48:16.680 --> 00:48:24.160 So I do the inverse transform of-- Here's the formula, then. 00:48:24.160 --> 00:48:32.550 I'm doing the inverse transform of, so the transform of c 00:48:32.550 --> 00:48:39.930 dot star, the transform of d. 00:48:39.930 --> 00:48:44.690 To get c, d. 00:48:44.690 --> 00:48:47.560 Is that right? 00:48:47.560 --> 00:48:51.630 So I took c, and I got back into the function. 00:48:51.630 --> 00:48:54.210 I took d, and got back to its function, 00:48:54.210 --> 00:48:55.920 with the Fourier matrix. 00:48:55.920 --> 00:48:59.740 OK, I'm in the Fourier and now I'm in this space. 00:48:59.740 --> 00:49:03.000 I've added up coefficients to get in this space. 00:49:03.000 --> 00:49:06.050 Now I do the dot star, the fast one. 00:49:06.050 --> 00:49:08.390 And then I transform back. 00:49:08.390 --> 00:49:11.850 So why is that faster? 00:49:11.850 --> 00:49:14.960 Than just doing it? 00:49:14.960 --> 00:49:20.850 Because what's the cost of F times c? 00:49:20.850 --> 00:49:23.180 And how am I going to do that? 00:49:23.180 --> 00:49:25.730 I'm going to do with the fast Fourier transform, right. 00:49:25.730 --> 00:49:26.650 That's the point. 00:49:26.650 --> 00:49:32.300 I can multiply by F, or by F inverse, faster. 00:49:32.300 --> 00:49:36.120 So I have three of these transforms. 00:49:36.120 --> 00:49:39.800 I've got to get two guys into the other space, 00:49:39.800 --> 00:49:42.480 and the answer back out. 00:49:42.480 --> 00:49:45.460 So I have sort of three of these N 00:49:45.460 --> 00:49:53.190 log N's but that will easily beat N squared. 00:49:53.190 --> 00:49:53.690 Right? 00:49:53.690 --> 00:50:01.560 So if you have a convolution to do, 00:50:01.560 --> 00:50:05.950 and it's possible to do this, get into the other space 00:50:05.950 --> 00:50:09.380 where it's just an element by element multiplication. 00:50:09.380 --> 00:50:11.710 And that would apply in either direction. 00:50:11.710 --> 00:50:13.910 Because the rule goes both ways. 00:50:13.910 --> 00:50:17.240 If I have this convolution to do, 00:50:17.240 --> 00:50:19.980 I would find the coefficients here, 00:50:19.980 --> 00:50:24.200 the c's, the coefficients of d of the g's. 00:50:24.200 --> 00:50:27.470 I would do this one by one multiplication, 00:50:27.470 --> 00:50:31.200 and then I have the Fourier coefficients 00:50:31.200 --> 00:50:32.860 of the convolution. 00:50:32.860 --> 00:50:35.770 Right, OK? 00:50:35.770 --> 00:50:37.110 Do I have a moment? 00:50:37.110 --> 00:50:37.870 Well, hardly. 00:50:37.870 --> 00:50:39.830 Just, can I write down what the formula 00:50:39.830 --> 00:50:44.380 for a convolution of two functions would look like? 00:50:44.380 --> 00:50:48.380 Sorry, f(x) convolved with g(x). 00:50:48.380 --> 00:50:50.610 Let me make it cyclic. 00:50:50.610 --> 00:50:53.300 Just to see what it would look like. 00:50:53.300 --> 00:50:56.140 What am I expecting for that convolution? 00:50:56.140 --> 00:50:58.570 I'm expecting a function, and somehow there's 00:50:58.570 --> 00:51:02.450 going to be an integral instead of a sum, where I had sums, 00:51:02.450 --> 00:51:04.140 but now I have integrals. 00:51:04.140 --> 00:51:05.890 And here's the point. 00:51:05.890 --> 00:51:13.000 I'll have f(t) times g(x-t) dt. 00:51:16.180 --> 00:51:20.160 All I'm asking you to look at is the fact that the way here I 00:51:20.160 --> 00:51:23.480 had k, and l-k. 00:51:23.480 --> 00:51:28.190 For functions, your eye sees that right away as a t, 00:51:28.190 --> 00:51:30.420 and an (x-t), dt. 00:51:30.420 --> 00:51:33.290 These add to the answer. 00:51:33.290 --> 00:51:36.880 These add to the result, that x. 00:51:36.880 --> 00:51:39.670 That would be the cyclical one, yeah. 00:51:39.670 --> 00:51:41.930 So I could go zero-- These are all 00:51:41.930 --> 00:51:46.780 periodic functions, so all 2pi periods are the same. 00:51:46.780 --> 00:51:50.250 The book will do that properly. 00:51:50.250 --> 00:51:54.440 OK, we've got the filtering to discuss on Monday. 00:51:54.440 --> 00:51:59.030 You can see that this convolution stuff just 00:51:59.030 --> 00:52:03.220 takes a little new thinking, but it comes out nicely.