Instructor: Prof. Gilbert Strang
The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free. To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.
PROFESSOR STRANG: OK, so I feel I should have brought pizza instead of math problems. I mean, but this is our final, final, hour. Open for any questions, any discussion about Fourier topics. Looking toward tomorrow evening's exam. And, well, hoping you'll find the whole course useful at many times in the future. So, somebody emailed me that the posted solution, so I just posted a quick solution to the un-collected homework ten, and so I haven't even looked to see what did it say for this question. But what is the answer? If I convolve the delta function with the delta function, what do I get? Delta, right. If I convolve anything with delta I get delta because I'm over in the other space. I'm multiplying by one. Yeah, so if I use the convolution rule to go over to the other space, where this becomes just one times one I'm getting one, and transform back and I've got delta. Well that question won't be on tomorrow's exam. But I thought we could deal with that fast. Now I'm open to questions about any topic. Homework or not. Yeah, thanks.
PROFESSOR STRANG: I could try. Did I make up the exam? Probably. Yeah, I'll recognize it. OK, alright, I'll read it out. OK, so this an exam from 2005, not that far back. OK, so it has some questions. Shall I tell you what the whole question is but it looks like this Part D is sort of separate from the others. But why not, this gives us some questions to think about. So if I just read out the questions. One was, find the coefficients of, the function is e^(-x). That would naturally occur to me as a function that you could, so from zero to 2pi. And then periodic. Always good to graph it. So will e to the minus x look like? It'll come down to, much steeper than that, but that would do it. So that's e^(-x). Here it's one, and here it's e^(-2pi). And then repeat, repeat, repeat. So find its Fourier coefficients. I'll just say what the question is and you can ask me. And then what's the decay rate. Oh, maybe you can tell me the decay rate. Even before you tell me the coefficient. What would the decay rate be if I expressed this as a combination of Fourier terms, harmonics. How quickly will those coefficients drop off? So 1/k, only. That's right, 1/k, because this function has a jump at zero. Well, at zero and again at 2pi at every one of those points. It has a jump, I guess, of about this distance, between one and that number. Which would be the jump that we would see of course every time. Yeah, so the coefficients would decay like 1/k.
Then it asked to compute the sum of those coefficients squared. OK, so how can we find the sum of those coefficients squared even before I know what they are? Well, you know the answer, right? How am I going to do this problem? Yeah, so I'm looking for the sum of the squares of coefficients and then there's this magic trick. What is it that helps me to find that sum of squares? How do I go about that? I connected to the integral of the function square. It can go zero to 2pi of the function square. And you remember the reason, and I've left a little space for the fudge factor. 2pi, or 1/2pi, we can figure out what it is by taking a simple case. Yeah, why don't we take a simple case to figure out should there be a 2pi or a 1/2pi or what? What's a simple case here? Suppose I take f(x)=1. Suppose I take the function f(x)=1, then the right side. So if f(x) is one, the right side would be 2pi, I guess. The integral of one, and the left side would be what? What would be the Fourier coefficients of the constant function? Well just c_0, the only one we would have would be c_0, the constant term. The c_0*e^0. And it would be one. So I'd have a one there, when k is zero, and otherwise I wouldn't have anything. So this would be one squared, this would be a one. So I do need to divide by 2pi. Do you agree with that? To make it correct I'd better have a 1/2pi to get that. To be right. OK, and then this gives me the answer then, I just do this integral, e^(-2x), then f is e^(-x). So f squared would be e^(-2x), which I can certainly integrate and get some expression. Yeah, OK. So that's a straightforward part.
And now no to the Part D, which you asked about which, is oh wow. OK, so now I'm asking about an equation u, du/dx, plus u(x) equals a train of deltas. So delta(x) made periodic. Can I write it that way? So this is the periodic case. So I'm looking for a periodic answer. So I'm in the Fourier series world when I take, I'll be looking for coefficients, not the integral transform. OK. So alright, so I'm making it clear. So how do I proceed now, with such a problem? So I've mentioned this morning that a problem like this could appear. So I'm happy to discuss it. What do I do? Transform. Absolutely. Take Fourier transform. OK, so what happens when I take Fourier transforms of u? Well, I guess so we're doing periodic, so we're going to go to coefficients, right? I'll call them, shall I call, shall I think of u(x), shall I just call its coefficient c? It's the only function and I have around so I might as well use c for it. So here's the function, and Fourier takes this to the set of coefficients, OK. And the point is that we can separate - I mean this is the whole reason why Fourier's so great. That we can do each frequency separately. So let's see then. So this has coefficient c_k, and what are the coefficients, what's the k'th coefficient, so this is all the e^(ikx) term. What's the k'th coefficient of the derivative? ikc_k, right? And what's the k'th coefficient of the delta function? One, right? Is that right or is there 1/2pi? Is it 1/2pi? OK, 1/2pi. Yeah, I guess that's right. Because the coefficient, I would have to take the integral times, yeah, the integral with the delta function will give me the one, and then there's a 1/2pi.
Right, OK. So now I've got the answer. Now I have this equation at each k. Following each eigenvector, following each separate frequency. And it's easy to do. It's just 1/2pi. And there's one plus ik, I think. OK, and now those are the c_k's. So I think maybe this is it. Sum from minus infinity, I know what the c_k is. 2pi(1+ik), times e^(ikx). That's my u(x), I think. Had to be, right? So I just did the sort of automatic steps. When I made up this exam, I don't know whether I was thinking I could find out can I actually do that sum to produce some function u(x) that I am familiar with. I'm not sure. Anybody reading the question, so the question says solve this differential equation. Doesn't say what it means by solve. Anyway, I've done it. I guess. Shall we agree that I passed on this question? If we knew some tricky way to do this addition, but I don't immediately see it. So that's good. OK, you're welcome. Thanks. OK, so that was a very good question to ask. Has it got, it's got the method of how do you deal with a differential equation. By the way, if this could be a difference equation, too. That could be u(x+h)-u(x-h). Shall I just emphasize that that wouldn't have been any harder? u(x+h)-u(x-h)/2h, let me take, for example, plus u equals delta. This is like I've taken a center difference there, instead of a derivative. How would you write a solution to that? Same method. So what would the method be? I'd take transforms. So here this would have coefficient c_k, this would have coefficients 1/2pi.
And what would be the coefficients of these guys? Well, there's a 1/2h. So what are the Fourier coefficients of u(x+h)? Just to see that all these things, I've got constant coefficients here. Why shouldn't Fourier work? So if it's u(x+h), so there's an e to the something factor, right? When I've shifted a function then in the transform I see an e to the i thing. So let's just see. If u(x), so if u(x) is the sum of c_k*e^(ikx), then u(x+h) would be the sum of c_k*e^ik(x+h), right? No. Nothing surprising there. So now I'm seeing the factor. There's c_k, it's multiplied by the factor e^(ikh). So this is the same c_k, but an e^(ikh), from that one. And this would be the minus, and this would shift the other way. So it would be the same c_k, and probably e^(-ikh). If I did that right. Yeah. Anyway, once again we have still linear problem. This is c_k times something equals 1/2pi. By the way, this thing is what I would call the transfer function. You see that language? We haven't used that language much. Or at all, it's sort of more of a systems theory, control theory word. But hey, we're always doing the same correct thing. That's the thing that transfers the input to the output. So maybe the transfer function is one over. Maybe I include the division in the transfer function. So finally I get c_k is this 1/2pi guy, divided by whatever that was, 1/2h times that number, minus 1/2h of that plus one. Whatever is multiplying c_k there. It's a transfer function. Yeah. So there's another word which we could have, and should have, used before today. But here it is.
OK, so that's some thoughts about that question. Any other direction to go?
PROFESSOR STRANG: Yes. go ahead. Yeah.
PROFESSOR STRANG: Lab problem 12, OK. 4.5, Fourier integrals, number 12. Oh yeah, maybe that was also. A question was raised about was it right on the posted solution. So maybe not. Let's see what we want to do. Yeah I just thought that was pretty - I could remember the day I thought of this equation. Integral of u minus derivative of u equal delta(x). I thought that's kind of a cool looking equation. But I don't know that I've ever solved it, before. So integral of u, actually yeah, I look cheerful, but so I'm finishing the fourth edition of the linear algebra book. Introduction to Linear Algebra. So I love writing. I've done all the writing and now comes the only horrible part, solving all those stupid exercises. So that, you may think of me in sunny Singapore or somewhere, but what I'll be doing is not on the beach. It's in an office somewhere, doing Problem 1.1.1, and two, three, it's not life. I mean, it's. But, OK. Anyway, that looks like a pretty good equation. And certainly I'm going to transform it. So I can see a little question coming up. So am in the Fourier integral world, yeah. So I should you use this notation. u hat of k, right? Over ik, is that the right thing when I've integrated? Yeah. That's made it smoother. That's dragging the function down. And I guess the one point that's a little tricky there is always, are you dividing by zero. Because zero is one of the frequencies. And so I sort of need u hat of zero to be zero for this. So it's looking slightly dangerous at k=0, but otherwise it's certainly improving things by speeding up the decay rate. And then this would be minus ik u hat of k, as we just said. And this would be the one, maybe it's a one in the Fourier integral world, where the 2pi went in a different point. So I think that's, yeah. So I would now just, the usual thing was the transfer function. 1/(ik)-ik, so that's my transfer function. That's multiplying u hat. Maybe I can simplify that by putting it all over ik, one minus ik. Is it plus k?
Does that look good, or does that look bad? k squared? Does that look better? So I'm wondering, so over here is k squared over ik, is that the same as the minus ik? Yeah. Yeah, bring it up and the minus i squared is one and I've got k squared. Yeah, so this looks good. So that's then dividing by it gives me because, I'm dividing the one. So u hat k is one over this. So it's the transfer function. One plus k squared. OK. I don't know whether I had in mind to go any further than this. Did you get this far, or what?
PROFESSOR STRANG: It said that? Oh jeez. I bitterly regret saying these things. Yeah, OK. I don't know u(x). Did anybody have any luck? You did. Did I say it with a derivative? Oh one, over one plus k squared, oh yeah. One over one plus k squared, we know what to do. And then i k will take its derivative. So what do we do if it's one over one plus k squared? That's the one where, that was the guy which was the e^(-x), right? Oh, it's just one side. No, two sides, isn't it? Two sides, yeah. Yeah, we just got a corner. Yeah, OK. That's looking good. And now maybe I need to divide by two. a is one here but there is a division by two. Yeah, that looks good. OK, and now I take - This function, its transform has that. And now if I want to multiply by ik I have to take the derivative. So I believe with your help here, that the derivative of that is the same. Can I just take the derivative while we're looking at it? I mean e^x is the most - you know, that was created to take its derivative, right? So its derivative is itself. The derivative of this is a minus. So I think that maybe that's the answer. e^x over two, coming up to 1/2, I guess. And then - oh, well the picture won't look, because of that minus sign. Yeah. Huh. Good, because that minus sign it's minus e^(-x) over two. So it starts at minus 1/2. So there's a jump of one. Or drop of one. Is that right, there should be a drop of one in u hat? Probably. Yeah, yeah. The integral will be smooth, at zero where the delta function's hitting us. I have a minus sign and a derivative. But yeah, doesn't that look good? If the derivative has a delta, the function has a jump. And with that minus sign, and with one delta the jump should be a drop of one. Which is that. Yeah. And the integral minus the derivative should be otherwise zero.
So again this is not unrelated to exam questions. I did all this stuff, and I really should check, did I get it right? Does that solve the differential equation? I mean, you could say of course it does if you took all these steps right. But it's wise to check. Plug it in. OK, it has the drop of one. So it deals correctly with the delta because, the derivative has a delta. And things match. And now what else do I have to check? I have to check out all the other points, right? I've just checked that yep, this answer is good at the jump. The tricky point. But now what about all the other points? So where the delta is zero, I should just check that the integral of u, so what is the integral of u? Let me just check for the points left of the origin. So I'm just going to look at this part. Yeah, x negative. So its integral, what's the integral of e^x over two? I guess e^x over two, right? Minus? And what's the derivative of e^x? Oh, the derivative is also e^x over two and I get zero, or x negative. As I want to. Because the delta is zero in that left side. And similarly on the right side. It should be OK. And then the key point was that the jump was right. Yeah, so thank you. That's good. Yes please.
PROFESSOR STRANG: What, sorry? Why did I take the derivative? What will be the derivative of this? Yes, there is. That's right. What would be the derivative of this? Huh, yeah, good question. What's the derivative of this answer. So it's that, on the left. and then there's a delta function, a minus delta, right, because it's dropped down. And then the derivative of that is e^(-x). Yeah, so if I graph the derivative, cool. I graph the derivative, it looks like the function. It has a spike going down there. At the origin, and then the derivative here is positive. Right? This function's coming up. It's e(-x) over two. It's positive, it's there. Yeah. Oh wow, that's a nice graph. So that's the derivative, this is the graph of du/dx. Yeah, thanks. And similarly I could graph the integral, the integral - well, would you want to see the integral? Probably not. Maybe, yeah. The integral would probably look pretty much just like that but without the delta, yeah. Cool, isn't that nice? It's artistic. That's the derivative, and the integral is the same thing without the delta. And when you subtract, you get the delta. OK, you sure you guys don't want to come to Singapore and help with these problem solutions? There's plenty for everybody. OK.
Alright, so another question. Yeah. 2005.
PROFESSOR STRANG: OK, alright. Yeah, which one?
PROFESSOR STRANG: OK, I'll just read out the whole question. Yeah. So this is 2005, Problem 3. OK it's a half hat function. This is Fourier integral, because it's on the whole line. And it's half a hat for between zero and one. The function is coming down. And then otherwise it's zero. So its graph, done. Graph its derivative. OK, let's graph the derivative of that function. Derivative is certainly zero along there. Then the derivative is minus one here. And then the derivative is zero. So the derivative looks to me like it's that, OK. So yeah, that function - is that right? No. It's not right. There's a delta. That was the tricky part of the problem. Right, there's a delta in the derivative, right here because of that jump. There's a delta. OK. Good. Now, is that better? That's good. OK. What's its transform? The transform of this derivative. What's the Fourier transform? Well, I guess we got two parts. So this is my function u'(x). The derivative. Now, what's u' transform? So it should be a function of k, now. Alright, what do you think? So it's the sum of two things. That delta, which Fourier transforms to one. And this guy down below between zero and one, which was like the one we did today, we've got the integral from zero to one. Now it happens to be a minus one e^(-ikx)dx. And that, of course, we can do. OK Oh, there's solutions here.
PROFESSOR STRANG: Yeah. So OK, so I think we're doing alright. This expression is familiar. It's (1-e^(-ik))/ik. Is that right? We have a minus. Is the minus, yeah, I think the minus looks good. And yeah, at least, that's here and it's probably got the minus signs correct. So why was it? I plugged in x=1, and that's why I got an e^(-ik). And then I plugged in x=0 and that's where that one came from. OK, so but your question was about - that was it? Oh, I see. OK. Right, and then the next part asked about the transform of the original guy. The transform of the original guy. Now, what would be the transform of the original guy? The original u. Right, OK, yeah. The reason I'm sort of stuttering is that if I'm going to integrate - I mean, what are you going to tell me? What's a quick way to find the transform of the original u now? Divide by ik. Divide by ik. And then I'm all ready to do that except I'm worried that k=0. But it should come out alright, now. So I have to hope that this thing comes out to be zero at k=0. Can you see that it does? What is, yeah?
This is a good point. What does u hat at zero represent? Have you thought about that? If I look at the Fourier transform, which I've got here. So let me write what I've got here. I've got here that Fourier transform of this function. And if I take the Fourier transform of any function, and I look at zero frequency. What am I seeing? The average, right. I'm saying the average. What's the average value of that function? We never thought about that. But that's fun. What's the average of this guy? If I integrate over the whole line. Do I get zero? Yes. Because I get the integral of the delta part gives me a one. But the integral of this box part gives me a minus one. So the integral, so this does equal zero at k=0, and I can safely do that division and get, so now u hat of k is this expression divided by ik. So it's one minus this thing. I'm just copying. Minus ik over ik, all that divided by ik. And I can, of course, maneuver that some more to make it look better. OK, these are good questions, because it's giving me a few functions to do the standard rules. Shifting, integrating, differentiating.
Oh, here's a Christmas present. What does that mean? Is the convolution of this function with itself the whole hat? What is that, a Christmas present? Is the convolution, the convolution of that with itself. So first point, we don't want to compute a convolution. You may have noticed, you have not computed convolutions or some things like this. It's not a lot of fun. Because you have that integral to do. And you'd have to separate out the parts where it's this and the part where it's this and the parts where it's that. It's not nice. Much better to multiply in the transform domain. Much better. OK, so if I multiply in the transform domain, I don't think I get the Fourier transform of the hat. Of the whole hat. But can you give me a convincing argument for why - this is here I'm in the transform domain and I'm going to convolve with itself, so I'm just going to square it. With the decay rate, what's the decay rate as it is now? The decay rate is, yes. The decay rate is a good key. So what is the decay rate right now? Here's a function with a jump. So I know that even though has a slightly messy looking transform, I know the decay rate. It's what, with a jump is 1/k. OK, so this must be, and I sort of do see a k down here, so that's sensible. OK, now, if I convolve, then I multiply in this domain. So that would give me the k squared, as you said. So could that be the transform of the whole hat?
Does the whole hat have a - ooh, could be. I'm pretty sure the answer's no. Yeah, no way. Yeah, I think that that's the best answer. I've lost the reason. Because I'm getting the k squared to k rate. If I convolve something with itself, that would give me a k squared, and the decay rate for the transform of the hat is a k squared. But they wouldn't be the same, yeah. OK, yeah, I won't. Yes, please.
PROFESSOR STRANG: Right. It's just convention. The difference was that that was the Fourier integral problem. Where we put, you might have noticed that the 2pi goes on the other - and this was the Fourier series problem. That was its only difference. So it's just a convention. And maybe other people would choose a different convention. So that is a slight wiggle in the presentation that the 2pi in the Fourier integral section got moved to the transform in the other direction. Yeah, good. Yes, thanks.
PROFESSOR STRANG: Oh, OK. I'm really just going by this rule that if there is a jump in the function, and nothing worse, then the decay rate is 1/k, and if there's a jump in the derivative, as there would be here. So that has a jump in slope, then the decay rate is one over k squared and so on. But those are the main cases. Yeah, so I'm not using anything deep there. And by the way, I realize here that one MATLAB problem that I didn't assign this semester but it's quite fun, is to actually see the Gibbs phenomenon. We know the terms in the Fourier series, with a jump. And if you computed - make it the periodic case, if you've computed the terms in the Fourier series they would stay near here and then I'd see the famous Gibbs stuff here. So that - you know, it's quite pleasant to see the printout of the Gibbs phenomenon. Not getting any shorter, just moving closer to the jump as you increase the number of terms. Yeah, it's quite interesting. But couldn't do everything. Yeah, ready for more. Any questions? Well, these are good. Yes, thanks.
PROFESSOR STRANG: With cyclic convolutions. OK, what could I do with cyclic convolutions? Let's see. What should I do with cyclic convolutions? I better make a little space. Let me make a little space and think. Anybody got a suggested problem for a cyclic convolution? I mean, one type of problem would certainly be if I gave you a cyclic convolution and asked for - I mean, the direct way would be to say OK, what's the cyclic convolution of (1, 4, 2), cyclically with (2, 1, 3) or something. But you could - that's just a calculation, so you would get that. I won't discuss that further. Now suppose I ask it with the unknown, the thing that I don't know here? So let me do that. Shall I say three, yeah. (3, 1, 1). Cyclically convolved with some unknown (x_0, x_1, x_2) equals some right-hand side, what should we take for the right-hand side? (1, 1, 1), for example? OK, I'll erase the top one.
OK, how would you go with that? So that's three by three, three unknowns. It's linear, so somehow I could write that as three equations in three unknowns. The question is how do I get some insight into this. And when I'm seeing a convolution, what's my immediate thought? Transform. That's what this month is all about. November. So I'll take the DFT of everything. So what's the discrete Fourier transform of (1, 1, 1)? Up there is going to be the transform. So let me call it x_0 hat, x_1 hat, x_2 hat. Just to emphasize. So n is three here, in this problem. Or maybe c_0, c_1, c_2 you might prefer. Whatever. And now this is going to be a multiplication, so it's in MATLAB notation will be a dot star, right? And so now I have to - these are the known ones, so what's the discrete Fourier transform of (1, 1, 1), the cyclic convolution of that? It's (1, 0, 0) is it? Or it's (3, 0, 0)? Is it three - I guess when I asked that question, you're open to two answers. You could either be going from x space to frequency space or frequency to the other. Because I haven't told you what space we're in. OK, let's go with that one. How did you get to there? You multiplied by the three by three Fourier matrix, is that what you did? Aright, so now can you do that here? So separately I have to do a little multiplication of the three by three Fourier matrix. One, one - sorry, let me get it right.
So I'm going to transform this. Make a little space here. Transform that. So it's 1, 1, 1, 1, 1 and this is w, w squared, w squared and w to the fourth, which is the same as w. OK, and now I want to multiply that by (3, 1, 1). And get the answer there. OK, so what do I have? Five. And what is that other one? Oh, boy. What is three plus w plus w squared? Anybody know that one? Well, I do know what one plus w plus w squared is. It is zero. Why is one plus w plus w squared zero? So I claim that if I have three plus, I think the answer there is two. And here I think I've also got three, w, w squared, I think if it was a one the answer would be a zero. But it's a three, so I still have two there. I think that's probably right. Can I just remind you why? There's one, there's w, and there's w squared and they add to zero. Yeah, yeah. For many reasons. That's a very, very, handy property. That the sum of the nth roots of one add to zero, OK, so now where I am I? So I got (5, 2, 2). So I've transformed. Now what do I do next?
I divide. So this is telling me that this is three separate equations. There's three, two, x_1 hat is zero. Right? No, x_2 hat. And 2x_2 hat. Sorry, one was right. And 2x_2 hat is zero. I'm just doing the multiplication as I'm supposed to. In the other space it's at each, for each component separately. So now I know x_0 hat is 3/5, and the others are zero. So now I know what these numbers are. Oh, yeah. (3/5, 0, 0). And now what? I mean, that statement's clearly true, right? Point star means five times this gives me three. Two times zero gives me zero, two times zero gives zero. So I'm golden, but I'm not finished. So that's not x, that's its transform. So what do I have to do? Go back. OK, so now can you tell me what this is? Up to a factor of 1/n, or n, which we may have to figure out separately. So apart from this possibly appearing, and probably appearing, give me an idea of what's the inverse transform of that? (3/5, 3/5, 3/5), good. 3/5, all three times.
OK, and now I can check to see if it's right. And again, I have not attempted to get the 1/n's right, I've just waited to the end. So now I'll just do this convolution and see if I really do get (1, 1, 1). So can we do that convolution? What's the typical, say, this guy. Where does that come from in the convolution? It comes from this, times what? This. That's the constant, right? And that's the constant. Contributes to the constant. And what other products contribute to the constant? So we're really seeing - thank you for this question. So this one multiplies which one? The answer is, this one multiplies this guy. You may say don't worry me about it because they're the same. But it's nice to know. So this one multiplies this one. And this guy multiplies this guy. And we'll remember in a minute why. And then you add them up, that's what convolution is. So I'm getting 9/5+3/5+3/5 is 15/5, so that's three. So what's your conclusion here? I should divide by three, right? So I should divide by three. So that's easy to do. Those will all be one. Yeah, OK. So that's the right answer, right? And then I check that it's right.
So let me just close by remembering why was it - so I'm doing up just a forward convolution here. Solving the equation was a deconvolution, figuring out that x. But now that I've got it and I'm just checking, that's just do it forward and see if you got the right answer. And so we're just remembering what it means to do a convolution. So you remember the point about convolutions. That if I'm convolving (a, b, c) cyclically with (d, e, f), that's a constant term. Just tell me again, now, what's the constant terms in this? It's ad plus b times what? This is the first term in the convolution. You remember that, the day we brought convolutions into the course? What does b multiply? f, why f. Because b is the coefficient of w, if I'm taking polynomials, this is a plus bw plus cw squared, and this corresponds to d plus ew plus fw squared. And I'm looking for the constant. So I get an a times ad gives me a constant. bw times what? times f w squared gives me the w cubed. Which is the one so that bf here and then there's a ce. Because c w squared multiplying ew gives me the w cubed, the one which folds in. Yeah, OK, so anyway. That's a quick reminder of cyclic convolutions. That was a good question to ask, yeah. You know, I fully realize this especially this third topic, lots of things like this. Straightforward if you have lots of time, but we had to keep moving. So remembering what cyclic convolution was is a good chance to do it.