1
00:00:00,000 --> 00:00:00,030
2
00:00:00,030 --> 00:00:02,285
The following content is
provided under a Creative
3
00:00:02,285 --> 00:00:03,610
Commons license.
4
00:00:03,610 --> 00:00:05,460
Your support will help
MIT OpenCourseWare
5
00:00:05,460 --> 00:00:09,940
continue to offer high-quality
educational resources for free.
6
00:00:09,940 --> 00:00:12,530
To make a donation, or to
view additional materials
7
00:00:12,530 --> 00:00:16,130
from hundreds of MIT courses,
visit MIT OpenCourseWare
8
00:00:16,130 --> 00:00:21,070
at ocw.mit.edu.
9
00:00:21,070 --> 00:00:24,930
PROFESSOR STRANG: Well, hope
you had a good Thanksgiving.
10
00:00:24,930 --> 00:00:30,190
So this is partly
review today, even.
11
00:00:30,190 --> 00:00:31,880
Wednesday even more review.
12
00:00:31,880 --> 00:00:36,880
Wednesday evening,
or Wednesday at 4
13
00:00:36,880 --> 00:00:38,900
I'll be here for any questions.
14
00:00:38,900 --> 00:00:44,220
And then the exam is
Thursday at 7:30 in Walker.
15
00:00:44,220 --> 00:00:49,380
Top floor of Walker this
time, not the same 54-100.
16
00:00:49,380 --> 00:00:54,760
OK, and then, no
lectures after that.
17
00:00:54,760 --> 00:00:56,300
Holiday, whatever.
18
00:00:56,300 --> 00:00:57,360
Yes.
19
00:00:57,360 --> 00:01:00,270
Right, you get a
chance to do something.
20
00:01:00,270 --> 00:01:02,520
Catch up with all
those other courses
21
00:01:02,520 --> 00:01:05,510
that are being neglected
in favor of 18.085.
22
00:01:05,510 --> 00:01:06,960
Right.
23
00:01:06,960 --> 00:01:11,240
OK, so here's a bit
of review right away.
24
00:01:11,240 --> 00:01:14,770
We really had four cases.
25
00:01:14,770 --> 00:01:19,370
We started with Fourier series,
that was periodic functions.
26
00:01:19,370 --> 00:01:23,930
And then discrete Fourier
series, also periodic in a way.
27
00:01:23,930 --> 00:01:26,600
Because w^N was one.
28
00:01:26,600 --> 00:01:31,340
So that we have N
numbers and then
29
00:01:31,340 --> 00:01:33,510
we could repeat
them if we wanted.
30
00:01:33,510 --> 00:01:37,500
So those are the
two that repeat.
31
00:01:37,500 --> 00:01:41,020
This is the f(x),
this is all x, so that
32
00:01:41,020 --> 00:01:45,310
would be the Fourier integral
that we did just last week.
33
00:01:45,310 --> 00:01:47,130
Fourier integral transform.
34
00:01:47,130 --> 00:01:51,220
And this was the--
Well, these are all,
35
00:01:51,220 --> 00:01:53,840
this is the discrete
all the way.
36
00:01:53,840 --> 00:02:00,810
So that's-- oh, you can
see these pair off, right?
37
00:02:00,810 --> 00:02:07,260
The periodic function,
the 2pi periodic function
38
00:02:07,260 --> 00:02:10,930
has Fourier
coefficients for all k,
39
00:02:10,930 --> 00:02:14,630
so that's the pair
that we started with.
40
00:02:14,630 --> 00:02:16,010
Section 4.1.
41
00:02:16,010 --> 00:02:20,850
This sort of pairs, I don't
know whether to say with itself.
42
00:02:20,850 --> 00:02:26,990
I mean, we start with N numbers
and we end with N numbers.
43
00:02:26,990 --> 00:02:29,670
We have N numbers
in physical space.
44
00:02:29,670 --> 00:02:33,810
And we have N numbers
in frequency space.
45
00:02:33,810 --> 00:02:39,650
Right, so we call those,
so those went to c_0 up
46
00:02:39,650 --> 00:02:42,630
to c_(N-1).
47
00:02:42,630 --> 00:02:47,360
And this, all x, pair--
for the function,
48
00:02:47,360 --> 00:02:48,820
paired off with itself.
49
00:02:48,820 --> 00:02:55,600
Or with this went to F--
well maybe I used small f.
50
00:02:55,600 --> 00:02:58,650
I guess I did in last week.
51
00:02:58,650 --> 00:03:03,520
So that, and I called its
Fourier transform f hat of k,
52
00:03:03,520 --> 00:03:06,820
all k.
53
00:03:06,820 --> 00:03:10,970
So that's the pairing kind of
inside n-dimensional space,
54
00:03:10,970 --> 00:03:12,670
with the Fourier matrix.
55
00:03:12,670 --> 00:03:16,220
This is the pairing
of the formula for f,
56
00:03:16,220 --> 00:03:19,310
and its similar
formula for f hat,
57
00:03:19,310 --> 00:03:22,390
and these are the guys that
connect with each other.
58
00:03:22,390 --> 00:03:25,580
OK, so that's what we know.
59
00:03:25,580 --> 00:03:31,330
What we haven't done is
anything in two dimensions.
60
00:03:31,330 --> 00:03:34,300
So I would like to
include that today.
61
00:03:34,300 --> 00:03:36,750
I think my real
message about 2-D,
62
00:03:36,750 --> 00:03:39,310
and I'm not going to
include it on the exam,
63
00:03:39,310 --> 00:03:45,870
but you might wonder, OK, can
I have a function of x and y?
64
00:03:45,870 --> 00:03:48,040
And will the whole setup work?
65
00:03:48,040 --> 00:03:49,510
And the answer is yes.
66
00:03:49,510 --> 00:03:54,670
So really, my message is not
to be afraid in any way of 2-D.
67
00:03:54,670 --> 00:04:00,590
It's just the same formulas
with x,y or two indices, k,l.
68
00:04:00,590 --> 00:04:01,090
Yeah.
69
00:04:01,090 --> 00:04:02,190
You'll see that.
70
00:04:02,190 --> 00:04:05,630
OK, now for the new part.
71
00:04:05,630 --> 00:04:09,380
What's a convolution equation?
72
00:04:09,380 --> 00:04:12,710
That's my word for
an equation where
73
00:04:12,710 --> 00:04:16,230
instead of doing a
convolution and finding
74
00:04:16,230 --> 00:04:19,360
the right-hand
side, instead we're
75
00:04:19,360 --> 00:04:21,760
given the right-hand side.
76
00:04:21,760 --> 00:04:26,560
And the unknown is
in the convolution.
77
00:04:26,560 --> 00:04:30,200
So let me write examples
of convolution equation.
78
00:04:30,200 --> 00:04:34,010
Every one of these would
allow a convolution.
79
00:04:34,010 --> 00:04:35,600
So the convolution
equation would
80
00:04:35,600 --> 00:04:42,650
be something the integral
of F(t) u, for the unknown,
81
00:04:42,650 --> 00:04:48,550
at x-t, is-- Oh no, sorry.
82
00:04:48,550 --> 00:04:50,400
F will be the right-hand side.
83
00:04:50,400 --> 00:04:53,430
F of, well, can I--
Yeah, better if I put it
84
00:04:53,430 --> 00:04:55,090
on the right-hand side.
85
00:04:55,090 --> 00:04:59,840
Wouldn't want to call
it the right-hand side.
86
00:04:59,840 --> 00:05:05,500
So this would be some, shall
I call it often K for kernel
87
00:05:05,500 --> 00:05:08,190
is sometimes the word.
88
00:05:08,190 --> 00:05:13,350
So what I'm saying is
equations come this way.
89
00:05:13,350 --> 00:05:19,130
This is really K
convolved with u.
90
00:05:19,130 --> 00:05:24,260
Equals F. You see, the only
novelty is the unknown is here.
91
00:05:24,260 --> 00:05:28,574
So that's why the word
deconvolution is up there.
92
00:05:28,574 --> 00:05:29,990
Because that's
what we have to do.
93
00:05:29,990 --> 00:05:34,120
We have to undo the convolution,
this unknown function
94
00:05:34,120 --> 00:05:39,390
is convolved with a known, K is
known, some known kernel that
95
00:05:39,390 --> 00:05:42,810
tells us the point
spread of the telescope
96
00:05:42,810 --> 00:05:44,060
or whatever we're doing.
97
00:05:44,060 --> 00:05:47,490
And gives us the output
that we're looking at.
98
00:05:47,490 --> 00:05:50,120
And then we have
to find the input.
99
00:05:50,120 --> 00:05:54,760
OK, can I write down the similar
equations for the other three
100
00:05:54,760 --> 00:05:55,360
here?
101
00:05:55,360 --> 00:05:58,770
And then we'll just
think how would we find
102
00:05:58,770 --> 00:06:00,340
u, how would we solve them?
103
00:06:00,340 --> 00:06:07,260
So the equation here
might be that some kernel
104
00:06:07,260 --> 00:06:13,500
circle convolved with
the unknown u is some y.
105
00:06:13,500 --> 00:06:16,940
These are now vectors.
106
00:06:16,940 --> 00:06:19,220
This is known.
107
00:06:19,220 --> 00:06:20,380
This is known.
108
00:06:20,380 --> 00:06:23,710
And those, the N components
of u, are unknown.
109
00:06:23,710 --> 00:06:26,710
OK, so that would be
the same problem here.
110
00:06:26,710 --> 00:06:27,680
What would be here?
111
00:06:27,680 --> 00:06:28,660
Same thing.
112
00:06:28,660 --> 00:06:31,520
Now the integral will go
from-- The only difference is
113
00:06:31,520 --> 00:06:36,420
the integral will go from
minus infinity to infinity,
114
00:06:36,420 --> 00:06:39,990
K(t)u(x-t) dt equal f(x).
115
00:06:39,990 --> 00:06:43,570
116
00:06:43,570 --> 00:06:48,970
And finally regular convolution.
117
00:06:48,970 --> 00:06:50,210
What am I going to call it?
118
00:06:50,210 --> 00:06:55,720
K would be a sequence,
maybe I should call it a,
119
00:06:55,720 --> 00:07:04,141
known, convolved with u,
unknown, is some c, known.
120
00:07:04,141 --> 00:07:04,640
Yeah.
121
00:07:04,640 --> 00:07:07,570
So those would be
four equations.
122
00:07:07,570 --> 00:07:09,210
You might say, wait
a minute, where
123
00:07:09,210 --> 00:07:11,220
is Professor Strang come
up with these problems
124
00:07:11,220 --> 00:07:13,450
at the last week of the course.
125
00:07:13,450 --> 00:07:18,720
But, these are exactly
the type of problems
126
00:07:18,720 --> 00:07:20,520
that we know and love.
127
00:07:20,520 --> 00:07:25,370
These come from
constant-coefficient,
128
00:07:25,370 --> 00:07:31,170
time-invariant, shift-invariant
linear problems.
129
00:07:31,170 --> 00:07:33,620
LTI, linear time-invariant.
130
00:07:33,620 --> 00:07:40,370
And my lecture Wednesday,
just before Thanksgiving, took
131
00:07:40,370 --> 00:07:45,300
a differential equation
for u and found,
132
00:07:45,300 --> 00:07:46,670
and put it in this form.
133
00:07:46,670 --> 00:07:48,640
I'll come back to that.
134
00:07:48,640 --> 00:07:51,750
So suddenly we're
seeing, I mean,
135
00:07:51,750 --> 00:07:53,800
we're actually seeing
some new things
136
00:07:53,800 --> 00:07:56,640
but also it includes
all the old ones.
137
00:07:56,640 --> 00:08:00,170
These are all of the best
problems in the world.
138
00:08:00,170 --> 00:08:02,920
These linear
constant-coefficient problems.
139
00:08:02,920 --> 00:08:05,900
Time invariant, of
any of these types.
140
00:08:05,900 --> 00:08:08,700
This one was an
integral from minus pi
141
00:08:08,700 --> 00:08:12,400
to pi, where this
one went all the way.
142
00:08:12,400 --> 00:08:16,540
So this is not brand new stuff.
143
00:08:16,540 --> 00:08:19,810
But it sort of looks new.
144
00:08:19,810 --> 00:08:26,890
And now the question is, so my
immediate question is, before
145
00:08:26,890 --> 00:08:31,150
doing any example, how would
you solve such an equation.
146
00:08:31,150 --> 00:08:34,820
And I saw on old exams,
some of this sort
147
00:08:34,820 --> 00:08:39,690
for example, let me
focus on this one.
148
00:08:39,690 --> 00:08:42,280
Let me, instead of
K there, I'm not
149
00:08:42,280 --> 00:08:46,080
used to using K for a
vector, I'm used to,
150
00:08:46,080 --> 00:08:48,860
well maybe I'll use c.
151
00:08:48,860 --> 00:08:50,220
For the vector there.
152
00:08:50,220 --> 00:08:59,060
So this is n
equations, n unknowns.
153
00:08:59,060 --> 00:09:01,610
Oops, capital N
is our usual here.
154
00:09:01,610 --> 00:09:03,160
For the number.
155
00:09:03,160 --> 00:09:06,740
N u, N unknown u's.
156
00:09:06,740 --> 00:09:10,670
It's a matrix equation
with a circulant matrix.
157
00:09:10,670 --> 00:09:17,510
So all these equations are
sort of the special best kind.
158
00:09:17,510 --> 00:09:19,540
Because they're convolutions.
159
00:09:19,540 --> 00:09:22,140
And now tell me the main point.
160
00:09:22,140 --> 00:09:24,820
How do we solve
equations like this?
161
00:09:24,820 --> 00:09:30,680
How do we do a deconvolution,
so the unknown is convolved
162
00:09:30,680 --> 00:09:35,330
with c here, it's convolved
with K, it's convolved with a,
163
00:09:35,330 --> 00:09:40,780
how do we deconvolve
it to get u by itself?
164
00:09:40,780 --> 00:09:43,840
So what's the central idea here?
165
00:09:43,840 --> 00:09:47,950
Central idea: go
into frequency space.
166
00:09:47,950 --> 00:09:50,160
Use the convolution rule.
167
00:09:50,160 --> 00:09:57,730
In frequency space,
where these transform,
168
00:09:57,730 --> 00:10:01,190
we're looking at multiplication.
169
00:10:01,190 --> 00:10:04,160
And multiplication, we can undo.
170
00:10:04,160 --> 00:10:06,680
We can de-multiply.
171
00:10:06,680 --> 00:10:10,930
De-multiply is just a big
word for divide, right?
172
00:10:10,930 --> 00:10:12,580
So that's the point.
173
00:10:12,580 --> 00:10:14,030
Get into that space.
174
00:10:14,030 --> 00:10:15,870
That's what we've been
doing all the time.
175
00:10:15,870 --> 00:10:21,490
I better get one example,
the example from the problem
176
00:10:21,490 --> 00:10:23,620
Wednesday, just up here.
177
00:10:23,620 --> 00:10:24,810
Just so you see it.
178
00:10:24,810 --> 00:10:27,280
This won't look like a
convolution equation,
179
00:10:27,280 --> 00:10:33,570
but do you remember that it
was -u'' plus a squared u equal
180
00:10:33,570 --> 00:10:37,410
some f(x)?
181
00:10:37,410 --> 00:10:40,500
So that's a constant, it's
certainly constant-coefficient,
182
00:10:40,500 --> 00:10:42,270
linear, time-invariant.
183
00:10:42,270 --> 00:10:43,100
Right, OK.
184
00:10:43,100 --> 00:10:44,600
And how did we solve that?
185
00:10:44,600 --> 00:10:46,960
We took Fourier transforms.
186
00:10:46,960 --> 00:10:51,770
So this was the
second derivative,
187
00:10:51,770 --> 00:10:52,930
the Fourier transform.
188
00:10:52,930 --> 00:10:56,870
What is the rule for the
Fourier transform of derivative?
189
00:10:56,870 --> 00:11:01,130
Every derivative brings
down an ik in the transform.
190
00:11:01,130 --> 00:11:03,030
So we get ik twice.
191
00:11:03,030 --> 00:11:07,200
So it's k squared. i squared
cancels the minus one.
192
00:11:07,200 --> 00:11:12,380
So that's the transform of -u''.
193
00:11:12,380 --> 00:11:17,010
This is the transform of
ordinary a squared u, just
194
00:11:17,010 --> 00:11:17,840
a squared.
195
00:11:17,840 --> 00:11:19,880
And this is f hat.
196
00:11:19,880 --> 00:11:23,420
So we've got into
frequency space.
197
00:11:23,420 --> 00:11:25,880
Where we are just
seeing a multiplication,
198
00:11:25,880 --> 00:11:34,620
k squared plus a squared, u
hat, of-- this is u hat of k,
199
00:11:34,620 --> 00:11:39,000
equals f hat of k, right?
200
00:11:39,000 --> 00:11:43,190
Oh well, sorry we were-- Yeah,
that's right. f hat of k,
201
00:11:43,190 --> 00:11:45,230
right.
202
00:11:45,230 --> 00:11:48,090
So we're in frequency
space, where we just
203
00:11:48,090 --> 00:11:49,220
see a multiplication.
204
00:11:49,220 --> 00:11:58,010
So again, this is now we
just demultiply, just divide.
205
00:11:58,010 --> 00:12:03,000
And then we have the answer,
but we have its transform.
206
00:12:03,000 --> 00:12:05,550
And then we have
to transform back.
207
00:12:05,550 --> 00:12:10,560
So we have to do the Fourier
transform to get to the,
208
00:12:10,560 --> 00:12:15,460
I'll say the inverse
Fourier transform.
209
00:12:15,460 --> 00:12:19,655
To get back to u(x), the answer.
210
00:12:19,655 --> 00:12:21,160
That's the model.
211
00:12:21,160 --> 00:12:22,110
That's the model.
212
00:12:22,110 --> 00:12:25,410
And that's maybe the
one that we've seen,
213
00:12:25,410 --> 00:12:32,710
now we're able to think
about all these four topics.
214
00:12:32,710 --> 00:12:34,660
Right.
215
00:12:34,660 --> 00:12:38,090
OK, so what was the key idea?
216
00:12:38,090 --> 00:12:40,520
Get into frequency
space and then it's
217
00:12:40,520 --> 00:12:44,160
just a, the equation is
just a multiplication,
218
00:12:44,160 --> 00:12:46,980
so the solution is
just a division.
219
00:12:46,980 --> 00:12:52,050
So can I do that now with
these four examples, just see.
220
00:12:52,050 --> 00:12:54,480
So this is like bring
the pieces together.
221
00:12:54,480 --> 00:12:59,140
OK, and deconvolution is
a very key thing to do.
222
00:12:59,140 --> 00:13:01,370
OK, so I'll take
all four of those
223
00:13:01,370 --> 00:13:04,000
and bring them into
frequency space.
224
00:13:04,000 --> 00:13:07,100
So this will be
maybe, you'll let
225
00:13:07,100 --> 00:13:14,080
me use K hat of, oh, K hat
of k that's not too good.
226
00:13:14,080 --> 00:13:27,390
Well, stuck with it.
227
00:13:27,390 --> 00:13:28,680
What am I doing here?
228
00:13:28,680 --> 00:13:31,040
In this 2pi periodic one?
229
00:13:31,040 --> 00:13:35,250
That's the one I started
with, but now I've got,
230
00:13:35,250 --> 00:13:37,030
I'm using hats and so on.
231
00:13:37,030 --> 00:13:42,180
I didn't do that in Section 4.1.
232
00:13:42,180 --> 00:13:46,980
What the heck am I going to,
what notation am I going to do?
233
00:13:46,980 --> 00:13:51,740
And I really didn't do
convolution that much,
234
00:13:51,740 --> 00:13:52,760
for functions.
235
00:13:52,760 --> 00:13:55,300
So let me jump to here.
236
00:13:55,300 --> 00:13:57,030
I'll come back.
237
00:13:57,030 --> 00:13:59,750
It follows exactly
the same pattern.
238
00:13:59,750 --> 00:14:01,350
So let me jump to this one.
239
00:14:01,350 --> 00:14:05,920
OK, so I have a
convolution equation now.
240
00:14:05,920 --> 00:14:10,700
This is one where you
could do this one.
241
00:14:10,700 --> 00:14:16,185
This could appear on the quiz
because I can do all of it.
242
00:14:16,185 --> 00:14:18,690
So what is this convolution?
243
00:14:18,690 --> 00:14:19,870
OK.
244
00:14:19,870 --> 00:14:21,920
I've got N equations,
N unknowns.
245
00:14:21,920 --> 00:14:23,800
Let me write them
in matrix form,
246
00:14:23,800 --> 00:14:28,880
just so you see it that way
too. c_0, c_1, c_2, c_3.
247
00:14:28,880 --> 00:14:30,970
I'll make N equal four.
248
00:14:30,970 --> 00:14:39,790
And then these are, this
convolution has c_0, c_2, c_1.
249
00:14:39,790 --> 00:14:44,510
c_3, c_2, c_1, c_0.
250
00:14:44,510 --> 00:14:49,960
This'll be c_3, c_3,
c_3, I'm writing down
251
00:14:49,960 --> 00:14:53,830
all the right numbers
in the right places.
252
00:14:53,830 --> 00:14:56,020
So that when I do
that multiplication
253
00:14:56,020 --> 00:15:04,740
with the unknown,
[u 0, u 1, u 2, u 3],
254
00:15:04,740 --> 00:15:10,530
I get the right-hand, the
known right-hand side.
255
00:15:10,530 --> 00:15:12,350
Maybe b would be
a little better.
256
00:15:12,350 --> 00:15:15,740
Because we're more used
to b as as a known.
257
00:15:15,740 --> 00:15:22,490
It's just an Ax=b problem,
or an Au=b problem.
258
00:15:22,490 --> 00:15:24,550
It looks like a
convolution but now
259
00:15:24,550 --> 00:15:26,520
it's just a matrix
multiplication.
260
00:15:26,520 --> 00:15:30,340
So this is just
[b 0, b 1, b 2, b 3].
261
00:15:30,340 --> 00:15:32,800
OK.
262
00:15:32,800 --> 00:15:34,920
That's our equation.
263
00:15:34,920 --> 00:15:36,950
Special type of matrix.
264
00:15:36,950 --> 00:15:38,470
Circulant matrix.
265
00:15:38,470 --> 00:15:40,730
So this is just
literally the same
266
00:15:40,730 --> 00:15:46,630
as c circularly convolved
with u equals b.
267
00:15:46,630 --> 00:15:50,640
I just wrote it out
in matrix language.
268
00:15:50,640 --> 00:15:59,640
So you could call MATLAB with
that matrix, and so one way
269
00:15:59,640 --> 00:16:04,960
to answer it would be get
the inverse of the matrix.
270
00:16:04,960 --> 00:16:11,190
But if it was
large, a better way
271
00:16:11,190 --> 00:16:16,190
would be switch over
to frequency space.
272
00:16:16,190 --> 00:16:16,970
Think, now.
273
00:16:16,970 --> 00:16:23,680
What happens when I switch these
vectors to frequency space?
274
00:16:23,680 --> 00:16:25,540
It becomes a multiplication.
275
00:16:25,540 --> 00:16:29,570
So this becomes
a multiplication.
276
00:16:29,570 --> 00:16:36,990
Now so c, I want the Fourier,
from the c's, what am I
277
00:16:36,990 --> 00:16:45,580
going to-- So these are
all in the space where
278
00:16:45,580 --> 00:16:46,820
it's a convolution.
279
00:16:46,820 --> 00:16:49,480
What am I going to call it
where it's in the space where
280
00:16:49,480 --> 00:16:51,270
it's a multiplication?
281
00:16:51,270 --> 00:16:54,050
I just need three new names.
282
00:16:54,050 --> 00:16:59,190
Maybe I'll use c
hat, u hat, and b hat
283
00:16:59,190 --> 00:17:02,950
just because there's no
doubt in anybody's mind
284
00:17:02,950 --> 00:17:05,460
that when you see
that hat, you've
285
00:17:05,460 --> 00:17:07,690
gone into frequency space.
286
00:17:07,690 --> 00:17:11,800
Now, what's the equation
in frequency space?
287
00:17:11,800 --> 00:17:15,760
And then I'll do an example.
288
00:17:15,760 --> 00:17:20,360
It's a multiplication,
but I don't usually
289
00:17:20,360 --> 00:17:27,160
see a vector and nothing there.
290
00:17:27,160 --> 00:17:31,270
What's the multiplication
in frequency space?
291
00:17:31,270 --> 00:17:37,310
It's component by component.
292
00:17:37,310 --> 00:17:48,100
c_0*u_0 equals b_0. c hat
1 u hat 1 equals b hat 1.
293
00:17:48,100 --> 00:17:53,760
c hat 2 u hat 2 equals b hat 2.
294
00:17:53,760 --> 00:18:00,650
And finally, c hat 3 u
hat 3 equals b hat 3.
295
00:18:00,650 --> 00:18:05,040
And there might be, I don't
swear that there isn't, a 1/4
296
00:18:05,040 --> 00:18:06,210
somewhere.
297
00:18:06,210 --> 00:18:07,780
Right?
298
00:18:07,780 --> 00:18:12,830
But the point is, we're
in frequency space now.
299
00:18:12,830 --> 00:18:15,190
We just have a
component by component,
300
00:18:15,190 --> 00:18:21,550
each component of c hat
times each component of u hat
301
00:18:21,550 --> 00:18:23,910
gives us a component
of b hat; now we're
302
00:18:23,910 --> 00:18:26,560
ready for a deconvolution;
just divide.
303
00:18:26,560 --> 00:18:29,860
So now u hat,
obviously I don't have
304
00:18:29,860 --> 00:18:34,940
to write all these,
b hat 0 over c hat 0.
305
00:18:34,940 --> 00:18:35,720
Right?
306
00:18:35,720 --> 00:18:36,850
I just do a division.
307
00:18:36,850 --> 00:18:44,520
So on down to u hat 3 is b hat,
is the third component of b,
308
00:18:44,520 --> 00:18:47,840
divided by the third
component of c.
309
00:18:47,840 --> 00:18:50,860
OK, now don't forget here.
310
00:18:50,860 --> 00:18:53,800
That in going from
here to here, I
311
00:18:53,800 --> 00:18:57,730
had to figure out what
the c hats were, right?
312
00:18:57,730 --> 00:19:03,330
I had to do the Fourier matrix,
or the inverse Fourier matrix
313
00:19:03,330 --> 00:19:07,620
to go from c to c
hat, from b to b hat,
314
00:19:07,620 --> 00:19:10,870
so everything got
Fourier transformed.
315
00:19:10,870 --> 00:19:17,500
But the object was to
make the equation easy.
316
00:19:17,500 --> 00:19:21,720
And of course, now we've
got four trivial equations
317
00:19:21,720 --> 00:19:24,160
that we just solved that way.
318
00:19:24,160 --> 00:19:27,460
Alright, let me see if I
can just pull this down
319
00:19:27,460 --> 00:19:32,730
with some questions.
320
00:19:32,730 --> 00:19:34,000
Here's a good question.
321
00:19:34,000 --> 00:19:39,300
When is a circulant
matrix invertible?
322
00:19:39,300 --> 00:19:42,900
When will this method work?
323
00:19:42,900 --> 00:19:45,520
The circulant matrix could
fail to be invertible.
324
00:19:45,520 --> 00:19:50,090
How would I know that?
325
00:19:50,090 --> 00:19:54,730
If it's singular, and how
would I, if I proceed this way,
326
00:19:54,730 --> 00:19:57,600
here I've got an answer.
327
00:19:57,600 --> 00:19:59,290
But if it's singular
I'm not really
328
00:19:59,290 --> 00:20:00,540
expecting to get an answer.
329
00:20:00,540 --> 00:20:03,260
Let me lift the board a little.
330
00:20:03,260 --> 00:20:14,100
So where would I get, oops,
have to stop this method.
331
00:20:14,100 --> 00:20:16,460
In solving those four equations.
332
00:20:16,460 --> 00:20:20,140
Where would I learn
that it's is singular?
333
00:20:20,140 --> 00:20:22,820
What could go wrong in this?
334
00:20:22,820 --> 00:20:23,330
Yes.
335
00:20:23,330 --> 00:20:25,010
AUDIENCE: [INAUDIBLE]
336
00:20:25,010 --> 00:20:26,380
PROFESSOR STRANG: That's right.
337
00:20:26,380 --> 00:20:32,600
Always in math, the question
is are you dividing by zero.
338
00:20:32,600 --> 00:20:36,210
So the question of whether
the matrix is singular,
339
00:20:36,210 --> 00:20:39,420
is the same as the question
of whether c_0 hat,
340
00:20:39,420 --> 00:20:46,860
c_01, c_02, and c_03-- sorry,
c_0 hat, c_1 hat, c_2 hat,
341
00:20:46,860 --> 00:20:49,740
and c_3 hat, can't be zero.
342
00:20:49,740 --> 00:20:55,780
That's, in fact, even better
those four numbers, those four
343
00:20:55,780 --> 00:21:00,190
c hats, are actually the
eigenvalues of the matrix.
344
00:21:00,190 --> 00:21:06,090
We've switched, what the Fourier
transform did, was switch over
345
00:21:06,090 --> 00:21:08,780
to the eigenvalues
and eigenvectors.
346
00:21:08,780 --> 00:21:12,050
And there, that's the whole
message of those guys is,
347
00:21:12,050 --> 00:21:14,010
you follow each one separately.
348
00:21:14,010 --> 00:21:15,880
Just the way we're doing here.
349
00:21:15,880 --> 00:21:20,150
So this is the component of
the b in the four eigenvector
350
00:21:20,150 --> 00:21:21,200
directions.
351
00:21:21,200 --> 00:21:26,020
Those are the four eigenvalues,
and I have to divide by them.
352
00:21:26,020 --> 00:21:30,620
You see, the idea is, like,
we've diagonalized the matrix.
353
00:21:30,620 --> 00:21:35,570
We've had that
matrix, which is full.
354
00:21:35,570 --> 00:21:41,810
And we take-- By taking the
Fourier transforms, that's
355
00:21:41,810 --> 00:21:46,440
the same thing as as
putting in the eigenvectors,
356
00:21:46,440 --> 00:21:51,660
switching the matrix to
this diagonal matrix, right?
357
00:21:51,660 --> 00:21:56,590
Our problem has become,
like, the diagonalized form
358
00:21:56,590 --> 00:22:01,670
is c_0 hat down to c_3 hat,
sitting on the diagonal.
359
00:22:01,670 --> 00:22:03,420
All zeroes elsewhere.
360
00:22:03,420 --> 00:22:06,920
That's when we switched, when
we did Fourier transform we
361
00:22:06,920 --> 00:22:08,780
were switching to eigenvectors.
362
00:22:08,780 --> 00:22:10,830
OK, so that's the message.
363
00:22:10,830 --> 00:22:18,780
That the test for singularity is
the Fourier, the transform of c
364
00:22:18,780 --> 00:22:21,560
hits zero.
365
00:22:21,560 --> 00:22:22,930
Then we're in trouble.
366
00:22:22,930 --> 00:22:25,350
Let me do an example you know.
367
00:22:25,350 --> 00:22:27,580
Let me do an example you know.
368
00:22:27,580 --> 00:22:33,250
OK here's, so finally now
we get a numerical example.
369
00:22:33,250 --> 00:22:38,900
The example we really
know is this one, right?
370
00:22:38,900 --> 00:22:41,620
As I start writing that,
you may say in your mind,
371
00:22:41,620 --> 00:22:43,280
oh no not again.
372
00:22:43,280 --> 00:22:47,840
But give it to me, one more
week with these matrices.
373
00:22:47,840 --> 00:22:54,710
But it'll be the C matrix, so
it's going to be the circulant.
374
00:22:54,710 --> 00:22:57,900
Recognize this?
375
00:22:57,900 --> 00:23:02,320
And it's got those minus
ones in the corners, too.
376
00:23:02,320 --> 00:23:05,750
OK, let's go back to day one.
377
00:23:05,750 --> 00:23:09,390
Is that matrix invertible?
378
00:23:09,390 --> 00:23:12,310
Yes or no.
379
00:23:12,310 --> 00:23:14,340
Please, no.
380
00:23:14,340 --> 00:23:17,230
Everybody knows that
matrix is not invertible.
381
00:23:17,230 --> 00:23:21,400
And do you remember
what's in the null space?
382
00:23:21,400 --> 00:23:25,440
Yes, what's the vector in the
null space of that matrix?
383
00:23:25,440 --> 00:23:27,350
All ones.
384
00:23:27,350 --> 00:23:30,620
Now, when I take,
just think now.
385
00:23:30,620 --> 00:23:35,250
When I take Fourier
transform, that all ones is
386
00:23:35,250 --> 00:23:38,640
going to transform to what?
387
00:23:38,640 --> 00:23:40,300
It's going to
transform to the delta.
388
00:23:40,300 --> 00:23:45,370
It'll transform to the one
that is like [1, 0, 0, 0].
389
00:23:45,370 --> 00:23:47,400
Or maybe it's [4, 0, 0, 0].
390
00:23:47,400 --> 00:24:01,370
But it's that, well, OK, now I'm
ready to take, so here's my c.
391
00:24:01,370 --> 00:24:02,810
So what's my method now?
392
00:24:02,810 --> 00:24:07,190
I'm going to do this method,
and I'm going to run into,
393
00:24:07,190 --> 00:24:09,520
this thing is going to be zero.
394
00:24:09,520 --> 00:24:11,470
Because that's the
eigenvalue that
395
00:24:11,470 --> 00:24:16,320
goes with the [1, 1, 1, 1, 1]
column, the constant, the zero
396
00:24:16,320 --> 00:24:19,140
frequency in frequency space.
397
00:24:19,140 --> 00:24:20,590
You'll see it happen.
398
00:24:20,590 --> 00:24:24,511
So let's take the Fourier
transform of that.
399
00:24:24,511 --> 00:24:26,260
And then we would have
to take the Fourier
400
00:24:26,260 --> 00:24:28,450
transform of the
right-hand side b, whatever
401
00:24:28,450 --> 00:24:29,380
that happened to be.
402
00:24:29,380 --> 00:24:31,480
But it's always the left side.
403
00:24:31,480 --> 00:24:33,680
The singular or not matrix.
404
00:24:33,680 --> 00:24:35,760
I believe we'll
be singular here.
405
00:24:35,760 --> 00:24:42,470
So, OK, just remind me how do
I take transforms of this guy?
406
00:24:42,470 --> 00:24:44,600
Gosh, we have to
be able to do that.
407
00:24:44,600 --> 00:24:48,430
That's Section 4 point,
well, 4.3 isn't, yeah.
408
00:24:48,430 --> 00:24:53,170
The DFT of that vector.
409
00:24:53,170 --> 00:24:56,960
What do I get?
410
00:24:56,960 --> 00:24:58,570
Yes.
411
00:24:58,570 --> 00:25:02,240
How do I take the
DFT of a vector?
412
00:25:02,240 --> 00:25:05,420
I multiply by the
Fourier matrix, right?
413
00:25:05,420 --> 00:25:06,800
Yes.
414
00:25:06,800 --> 00:25:10,770
So I have to multiply that
thing by the Fourier matrix.
415
00:25:10,770 --> 00:25:17,090
So to get c hat, this was
big C for the matrix, little
416
00:25:17,090 --> 00:25:21,790
c for the vector that goes
into it, into column zero.
417
00:25:21,790 --> 00:25:24,290
And c hat for its transform.
418
00:25:24,290 --> 00:25:29,640
OK, so now here comes the
Fourier matrix that we know, 1,
419
00:25:29,640 --> 00:25:39,770
i, i^2, i^3; 1, i^2, i^4,
i^6; 1, i^3, i^6, and i^9.
420
00:25:39,770 --> 00:25:46,340
So I want to transform that
c to get, to find out c hat.
421
00:25:46,340 --> 00:25:52,300
OK, and what do I get up there?
422
00:25:52,300 --> 00:25:54,840
What's the first component,
the zeroth component
423
00:25:54,840 --> 00:26:02,820
I should say, when I take this
guy, this four, this vector
424
00:26:02,820 --> 00:26:05,070
with four components
and I get back
425
00:26:05,070 --> 00:26:08,910
four components, the
frequency components, what's
426
00:26:08,910 --> 00:26:10,640
the first one?
427
00:26:10,640 --> 00:26:13,570
Ones times this,
what am I getting?
428
00:26:13,570 --> 00:26:14,220
Zero.
429
00:26:14,220 --> 00:26:16,230
That's what I expected.
430
00:26:16,230 --> 00:26:20,100
That tells me the matrix is
not going to be invertible.
431
00:26:20,100 --> 00:26:28,140
Because in a different language,
I'm finding the eigenvalues
432
00:26:28,140 --> 00:26:30,100
and that's one of them.
433
00:26:30,100 --> 00:26:32,610
And if an eigenvalue
is zero, that
434
00:26:32,610 --> 00:26:36,270
means the eigenvector is
getting knocked out completely.
435
00:26:36,270 --> 00:26:40,590
And there's no way a c
inverse could recover
436
00:26:40,590 --> 00:26:42,370
when that eigenvector is gone.
437
00:26:42,370 --> 00:26:43,860
OK, let's do the other ones.
438
00:26:43,860 --> 00:26:49,510
Two minus i, nothing.
439
00:26:49,510 --> 00:26:51,880
What's the other one here?
440
00:26:51,880 --> 00:26:56,560
Two, this is minus i, and
that's plus i, I think.
441
00:26:56,560 --> 00:26:59,040
So I think it's just two.
442
00:26:59,040 --> 00:27:02,910
Alright, this is 2 i squared,
can I write in some of these
443
00:27:02,910 --> 00:27:07,190
just so I have a little,
i squared is minus one,
444
00:27:07,190 --> 00:27:10,170
and i^4 is one, and
that's minus one.
445
00:27:10,170 --> 00:27:15,660
So that's two plus one,
plus one, I think is four.
446
00:27:15,660 --> 00:27:20,100
And then i^3 is the
same as minus i.
447
00:27:20,100 --> 00:27:22,800
And i^9 is the same as plus i.
448
00:27:22,800 --> 00:27:34,900
So I think I'm getting two
plus i, nothing, minus i: two.
449
00:27:34,900 --> 00:27:36,770
So what's my claim?
450
00:27:36,770 --> 00:27:39,280
My claim is that these
are the four eigenvalues,
451
00:27:39,280 --> 00:27:43,080
that the Fourier-- Fourier
diagonalizes these problems.
452
00:27:43,080 --> 00:27:44,800
That's what it comes to.
453
00:27:44,800 --> 00:27:48,700
Fourier diagonalizes all
constant-coefficient,
454
00:27:48,700 --> 00:27:51,850
shift-invariant,
linear problems.
455
00:27:51,850 --> 00:27:55,820
And tells us here
are eigenvalues.
456
00:27:55,820 --> 00:27:57,530
So [0, 2, 4, 2].
457
00:27:57,530 --> 00:27:59,180
Would you like
to, how do I check
458
00:27:59,180 --> 00:28:00,710
the eigenvalues of a matrix?
459
00:28:00,710 --> 00:28:02,940
Let's just remember.
460
00:28:02,940 --> 00:28:06,870
If I give you four numbers and
I say those are the eigenvalues,
461
00:28:06,870 --> 00:28:09,730
and you look at that
matrix, what quick check
462
00:28:09,730 --> 00:28:12,020
does everybody do?
463
00:28:12,020 --> 00:28:14,650
Compute the?
464
00:28:14,650 --> 00:28:15,425
The trace.
465
00:28:15,425 --> 00:28:17,880
Add up the diagonal
of the matrix,
466
00:28:17,880 --> 00:28:21,560
add up the proposed eigenvalues,
they had better be the same.
467
00:28:21,560 --> 00:28:22,330
And they are.
468
00:28:22,330 --> 00:28:23,725
I get eight both ways.
469
00:28:23,725 --> 00:28:26,100
That doesn't mean, of course,
that these four numbers are
470
00:28:26,100 --> 00:28:28,430
right, but I think they are.
471
00:28:28,430 --> 00:28:29,200
Yeah, yeah.
472
00:28:29,200 --> 00:28:31,810
So those added up to
eight, those numbers
473
00:28:31,810 --> 00:28:33,240
added up to eight.
474
00:28:33,240 --> 00:28:36,520
And yep.
475
00:28:36,520 --> 00:28:38,270
And these are real.
476
00:28:38,270 --> 00:28:39,970
They came out
real, and how did I
477
00:28:39,970 --> 00:28:44,530
know that would happen
from the matrix?
478
00:28:44,530 --> 00:28:51,030
What matrices am I certain
to get real eigenvalues for?
479
00:28:51,030 --> 00:28:51,900
Symmetric.
480
00:28:51,900 --> 00:28:52,690
Right.
481
00:28:52,690 --> 00:28:55,300
Now, what about, I
heard the word positive.
482
00:28:55,300 --> 00:28:58,070
Of course, that's the other
question I have to ask.
483
00:28:58,070 --> 00:29:03,760
Is this matrix
positive definite?
484
00:29:03,760 --> 00:29:05,590
OK, everybody, this
is the language
485
00:29:05,590 --> 00:29:06,770
we've learned in 18.085.
486
00:29:06,770 --> 00:29:09,980
Is that matrix positive
definite, yes or no?
487
00:29:09,980 --> 00:29:11,150
No.
488
00:29:11,150 --> 00:29:12,020
What is it?
489
00:29:12,020 --> 00:29:15,260
It's positive semi-definite.
490
00:29:15,260 --> 00:29:18,460
What does that tell
me about eigenvalues?
491
00:29:18,460 --> 00:29:22,100
There, one is zero, that's why
it's not positive definite.
492
00:29:22,100 --> 00:29:24,270
But the others are positive.
493
00:29:24,270 --> 00:29:27,010
So that sure enough,
in other words,
494
00:29:27,010 --> 00:29:32,260
what we've done here, for that
matrix that came on day one
495
00:29:32,260 --> 00:29:38,040
and now we're seeing
it on day N-1 here.
496
00:29:38,040 --> 00:29:42,590
We're we're seeing sort of in
a new way, because at that time
497
00:29:42,590 --> 00:29:47,230
we didn't know these four
were the eigenvectors
498
00:29:47,230 --> 00:29:49,000
of that matrix.
499
00:29:49,000 --> 00:29:50,300
But they are.
500
00:29:50,300 --> 00:29:56,080
And we're coming to
the same conclusion
501
00:29:56,080 --> 00:29:58,010
we came to on day one.
502
00:29:58,010 --> 00:30:03,270
That the matrix is
positive semi-definite
503
00:30:03,270 --> 00:30:05,470
and that we know
its eigenvalues.
504
00:30:05,470 --> 00:30:08,850
And we can,
actually, let me even
505
00:30:08,850 --> 00:30:11,350
take it one more step, just
because this example is
506
00:30:11,350 --> 00:30:13,200
so perfect.
507
00:30:13,200 --> 00:30:17,300
Some right-hand sides we
could solve for, right?
508
00:30:17,300 --> 00:30:21,070
If I have a matrix
that's singular.
509
00:30:21,070 --> 00:30:24,380
Way, way back, even, I think
it was like a worked example
510
00:30:24,380 --> 00:30:32,330
in Section 1.1, I could ask the
question when is Cx=b solvable.
511
00:30:32,330 --> 00:30:34,950
Because there are some
right-hand sides that'll work.
512
00:30:34,950 --> 00:30:37,280
Because if I just take
an x and multiply by C,
513
00:30:37,280 --> 00:30:40,280
I'll get a right-hand
side that works.
514
00:30:40,280 --> 00:30:48,770
But for which vectors,
right-hand sides b,
515
00:30:48,770 --> 00:30:52,000
will my method work?
516
00:30:52,000 --> 00:30:54,470
The ones that have...?
517
00:30:54,470 --> 00:30:56,180
Yeah, the ones that have which?
518
00:30:56,180 --> 00:31:01,830
What do I need with these
c's, c_0, c_1, c_2, and c_3,
519
00:31:01,830 --> 00:31:06,630
for my solution to be possible.
520
00:31:06,630 --> 00:31:09,710
I need b_0 hat.
521
00:31:09,710 --> 00:31:11,540
Equals zero.
522
00:31:11,540 --> 00:31:13,380
I need b_0 hat equals zero.
523
00:31:13,380 --> 00:31:15,180
And then what does that say?
524
00:31:15,180 --> 00:31:18,500
That means that the
b, the vector b,
525
00:31:18,500 --> 00:31:22,790
has no constant term
in the Fourier series.
526
00:31:22,790 --> 00:31:27,950
It means that the vector b is
orthogonal to the [1, 1, 1, 1],
527
00:31:27,950 --> 00:31:29,290
eigenvector.
528
00:31:29,290 --> 00:31:34,050
So this is like a subtle point
but just driving home the point
529
00:31:34,050 --> 00:31:39,120
that what Fourier does is
diagonalize everything.
530
00:31:39,120 --> 00:31:42,990
It diagonalizes all the
important problems of,
531
00:31:42,990 --> 00:31:47,330
all the simplest problems,
of differential equations.
532
00:31:47,330 --> 00:31:52,730
You know, I mean this is like
18.03 looked at from Fourier's
533
00:31:52,730 --> 00:31:53,840
point of view.
534
00:31:53,840 --> 00:31:57,060
OK, what more could I
do with that equation?
535
00:31:57,060 --> 00:32:00,680
I think you really are seeing
all the good stuff here.
536
00:32:00,680 --> 00:32:02,570
You're seeing the matrix.
537
00:32:02,570 --> 00:32:05,050
We're recognizing
it as a circulant.
538
00:32:05,050 --> 00:32:08,860
We're realizing that we could
take its Fourier transform.
539
00:32:08,860 --> 00:32:11,700
We get the eigenvalues.
540
00:32:11,700 --> 00:32:15,270
We're diagonalizing the
matrix, the convolution
541
00:32:15,270 --> 00:32:18,530
becomes a multiplication,
and the solution becomes,
542
00:32:18,530 --> 00:32:22,040
inversion becomes division.
543
00:32:22,040 --> 00:32:26,990
I hope you see that.
544
00:32:26,990 --> 00:32:30,820
That's really a model
problem for this course.
545
00:32:30,820 --> 00:32:33,160
OK. yeah.
546
00:32:33,160 --> 00:32:33,710
Questions.
547
00:32:33,710 --> 00:32:41,417
Good.
548
00:32:41,417 --> 00:32:44,000
AUDIENCE: [INAUDIBLE] PROFESSOR
STRANG: Would I give you a six
549
00:32:44,000 --> 00:32:45,930
by six Fourier matrix on a test?
550
00:32:45,930 --> 00:32:46,730
Probably not.
551
00:32:46,730 --> 00:32:48,080
No.
552
00:32:48,080 --> 00:32:49,630
I just about could.
553
00:32:49,630 --> 00:32:55,220
I mean, it's, six by six, those
are pretty decent numbers.
554
00:32:55,220 --> 00:32:56,730
Right.
555
00:32:56,730 --> 00:32:59,850
Those six roots of
unity, but not quite.
556
00:32:59,850 --> 00:33:00,580
Right, yeah.
557
00:33:00,580 --> 00:33:01,080
Yeah.
558
00:33:01,080 --> 00:33:01,590
Yeah.
559
00:33:01,590 --> 00:33:05,720
So four by four is, five by five
would not be nice, certainly.
560
00:33:05,720 --> 00:33:09,570
Who knows the cosine
of 72 degrees?
561
00:33:09,570 --> 00:33:10,300
Crazy.
562
00:33:10,300 --> 00:33:13,620
But, 60 degrees we could do.
563
00:33:13,620 --> 00:33:17,260
So the Fourier matrix would
be full of square roots
564
00:33:17,260 --> 00:33:20,720
of three over two, and one
over two, and i's, and so on.
565
00:33:20,720 --> 00:33:24,960
But it wouldn't be as nice,
so really four by four
566
00:33:24,960 --> 00:33:26,910
is sort of the model.
567
00:33:26,910 --> 00:33:28,310
Yeah, yeah.
568
00:33:28,310 --> 00:33:31,890
So four by four is that model.
569
00:33:31,890 --> 00:33:33,230
Other questions?
570
00:33:33,230 --> 00:33:35,180
Because this is
really a key example.
571
00:33:35,180 --> 00:33:37,240
Yeah.
572
00:33:37,240 --> 00:33:41,860
When I calculated the
eigenvalues, yeah.
573
00:33:41,860 --> 00:33:42,420
Ah.
574
00:33:42,420 --> 00:33:46,330
Because this matrix, I know
everything about that matrix
575
00:33:46,330 --> 00:33:47,800
when I know its first vector.
576
00:33:47,800 --> 00:33:49,232
AUDIENCE: [INAUDIBLE]
577
00:33:49,232 --> 00:33:50,940
PROFESSOR STRANG:
Yeah, it's because it's
578
00:33:50,940 --> 00:33:52,120
a circulant matrix.
579
00:33:52,120 --> 00:33:57,010
It's because that matrix
is expressing convolution
580
00:33:57,010 --> 00:34:03,170
with this vector.
[2, -1, 0, -1].
581
00:34:03,170 --> 00:34:06,400
That circulant
matrix essentially
582
00:34:06,400 --> 00:34:08,820
is built from four
numbers, right.
583
00:34:08,820 --> 00:34:09,320
Yeah.
584
00:34:09,320 --> 00:34:11,640
Yeah, and they go in
the zeroth column.
585
00:34:11,640 --> 00:34:12,700
Right, yeah.
586
00:34:12,700 --> 00:34:14,580
Yeah.
587
00:34:14,580 --> 00:34:20,580
Right, so there is an
example where we could like
588
00:34:20,580 --> 00:34:21,780
do everything.
589
00:34:21,780 --> 00:34:27,660
Now, and let me just remember
that with this example,
590
00:34:27,660 --> 00:34:29,440
we could do everything.
591
00:34:29,440 --> 00:34:37,410
So this is an example of, you
could say this type of problem.
592
00:34:37,410 --> 00:34:42,310
But with a very special kernel
there, so it turned out to be,
593
00:34:42,310 --> 00:34:44,470
it looks like an
integral equation here
594
00:34:44,470 --> 00:34:49,010
but if that kernel involves
delta functions and so on then
595
00:34:49,010 --> 00:34:51,530
it can be just a
differential equation.
596
00:34:51,530 --> 00:34:53,420
And then that's
what we got there.
597
00:34:53,420 --> 00:34:59,590
So we took all-- The same
steps we did here, we did here.
598
00:34:59,590 --> 00:35:06,110
We took the Fourier transform,
and I emphasize there,
599
00:35:06,110 --> 00:35:09,850
just to remember Wednesday,
this was a delta function.
600
00:35:09,850 --> 00:35:13,660
When I took the Fourier
transform I got a one,
601
00:35:13,660 --> 00:35:16,020
so this was a one over this.
602
00:35:16,020 --> 00:35:22,190
And I did the inverse transform
and I got back to the function
603
00:35:22,190 --> 00:35:24,260
that I drew.
604
00:35:24,260 --> 00:35:29,170
Which was e^(-ax) over 2a.
605
00:35:29,170 --> 00:35:31,240
And even.
606
00:35:31,240 --> 00:35:38,650
So, yeah. this was
the answer u(x).
607
00:35:38,650 --> 00:35:41,950
So I was able to do that,
I mean this step was easy,
608
00:35:41,950 --> 00:35:43,110
that step is easy.
609
00:35:43,110 --> 00:35:45,960
That step is easy,
the division is easy.
610
00:35:45,960 --> 00:35:55,270
And then I just recognize this
as the transform of this one,
611
00:35:55,270 --> 00:35:57,680
this example that we had done.
612
00:35:57,680 --> 00:35:59,570
Once I divided by 2a.
613
00:35:59,570 --> 00:36:01,420
So you should be
able to do this.
614
00:36:01,420 --> 00:36:05,230
So those are two that you
should really be able to do.
615
00:36:05,230 --> 00:36:07,510
I'm not going to,
obviously I'm not going to,
616
00:36:07,510 --> 00:36:13,600
ask you a 2-D problem on the
exam or even on a homework.
617
00:36:13,600 --> 00:36:16,970
But now if you'll
allow me, I'd like
618
00:36:16,970 --> 00:36:22,360
to spend a few minutes to
get into 2-D. Because really,
619
00:36:22,360 --> 00:36:25,540
you've got the
main thoughts here.
620
00:36:25,540 --> 00:36:28,340
That Fourier is
the same as finding
621
00:36:28,340 --> 00:36:30,910
eigenvectors and eigenvalues.
622
00:36:30,910 --> 00:36:35,520
That's the main thought
for these LTI problems.
623
00:36:35,520 --> 00:36:40,420
OK, now suppose I have, let's
just get the formalities
624
00:36:40,420 --> 00:36:41,560
straight here.
625
00:36:41,560 --> 00:36:45,170
Suppose I have a
function of x and y.
626
00:36:45,170 --> 00:36:48,440
2pi periodic in x, and in y.
627
00:36:48,440 --> 00:36:54,530
So if I bump x by 2pi, or
if I bump y by 2pi -- oh,
628
00:36:54,530 --> 00:36:59,190
I'm using capital F
for the periodic guys.
629
00:36:59,190 --> 00:37:04,410
So let me stay with
capital F -- x, y+2pi.
630
00:37:04,410 --> 00:37:08,470
OK, so I have a function.
631
00:37:08,470 --> 00:37:10,440
This is given.
632
00:37:10,440 --> 00:37:13,610
This is, it's in 2-D now.
633
00:37:13,610 --> 00:37:16,300
And I want to write
its Fourier series.
634
00:37:16,300 --> 00:37:18,850
So I'm just asking
the question what
635
00:37:18,850 --> 00:37:22,770
does the Fourier series look
like for a function of two
636
00:37:22,770 --> 00:37:25,210
variables.
637
00:37:25,210 --> 00:37:28,930
The point is, it's going
to be a nice answer.
638
00:37:28,930 --> 00:37:34,100
And so everything, what you
know how to do in 1-D you can do
639
00:37:34,100 --> 00:37:41,090
in 2-D. So let me write the
complex form, the e^(ik) stuff.
640
00:37:41,090 --> 00:37:44,410
So what would I write,
how would I write this?
641
00:37:44,410 --> 00:37:48,390
I would write that as
a sum, but it'll have,
642
00:37:48,390 --> 00:37:51,630
I'll make it a double sum,
I'll write two sigmas just
643
00:37:51,630 --> 00:37:57,100
to emphasize that we're summing
from k equal minus infinity
644
00:37:57,100 --> 00:38:02,285
to infinity, and from l equal
minus infinity to infinity.
645
00:38:02,285 --> 00:38:07,390
We have coefficients c_kl.
646
00:38:07,390 --> 00:38:11,270
They depend on two indices,
this is the pattern to know.
647
00:38:11,270 --> 00:38:19,130
Multiplying our e^(ikx),
and our e^(ily).
648
00:38:19,130 --> 00:38:23,760
Right, good.
649
00:38:23,760 --> 00:38:25,990
So, alright.
650
00:38:25,990 --> 00:38:27,070
Let me ask you.
651
00:38:27,070 --> 00:38:30,540
How would I find c_23?
652
00:38:30,540 --> 00:38:35,040
Just to know that-- We could
find all these coefficients,
653
00:38:35,040 --> 00:38:36,720
find formulas for them.
654
00:38:36,720 --> 00:38:39,210
We could do examples.
655
00:38:39,210 --> 00:38:41,380
How would I find c_23?
656
00:38:41,380 --> 00:38:49,240
So this is my F. I know
F. I want to find c_23.
657
00:38:49,240 --> 00:38:52,000
What's the magic trick?
658
00:38:52,000 --> 00:38:56,800
And I'm 2pi periodic, so all
integrals, all the integrals--
659
00:38:56,800 --> 00:38:58,480
And I'm giving you
a hint, of course.
660
00:38:58,480 --> 00:39:01,520
I'm going to integrate.
661
00:39:01,520 --> 00:39:04,840
And the integrals will
all go from minus pi to pi
662
00:39:04,840 --> 00:39:06,290
in x and in y.
663
00:39:06,290 --> 00:39:09,650
They'll integrate over
the period square.
664
00:39:09,650 --> 00:39:12,590
Here's the period
square from, there's
665
00:39:12,590 --> 00:39:18,615
the center. x direction, y
direction, goes out to pi
666
00:39:18,615 --> 00:39:21,270
and goes up to pi.
667
00:39:21,270 --> 00:39:26,360
So all integrals
will be over dxdy.
668
00:39:26,360 --> 00:39:28,350
But what do I integrate?
669
00:39:28,350 --> 00:39:33,070
To find c_23.
670
00:39:33,070 --> 00:39:36,770
Well, these guys are orthogonal.
671
00:39:36,770 --> 00:39:38,460
That's what's making
everything work,
672
00:39:38,460 --> 00:39:40,830
they're orthogonal
and very special.
673
00:39:40,830 --> 00:39:43,940
So that by use
orthogonality, what do I do?
674
00:39:43,940 --> 00:39:46,320
I multiply by?
675
00:39:46,320 --> 00:39:50,140
Just tell me what
to multiply by.
676
00:39:50,140 --> 00:39:57,890
By this and integrate.
677
00:39:57,890 --> 00:40:04,260
OK, what is it that I multiply
by if I'm shooting for c_23,
678
00:40:04,260 --> 00:40:15,580
for example? e^(i2x),
is it e^(i2x)?
679
00:40:15,580 --> 00:40:17,150
Minus, right.
680
00:40:17,150 --> 00:40:27,800
I multiply by e^(-i2x),
e^(-i3y), and integrate.
681
00:40:27,800 --> 00:40:28,420
Yeah.
682
00:40:28,420 --> 00:40:31,980
So when I multiply by
that and integrate,
683
00:40:31,980 --> 00:40:35,830
everything will go
except the c_23 term.
684
00:40:35,830 --> 00:40:38,430
Which will be
multiplied by what?
685
00:40:38,430 --> 00:40:43,290
So I'll just have c_23
times probably 2pi squared.
686
00:40:43,290 --> 00:40:46,130
I guess 2pi will come
in from both integrals,
687
00:40:46,130 --> 00:40:51,230
so the formula will be
c_kl-- c_kl will be,
688
00:40:51,230 --> 00:40:52,790
do you want me to
write this formula?
689
00:40:52,790 --> 00:40:55,740
I'll write it here and
then forget it right away.
690
00:40:55,740 --> 00:41:00,570
c_kl will be one
over 2pi squared,
691
00:41:00,570 --> 00:41:07,020
the integral of my
function times my e^(-ikx),
692
00:41:07,020 --> 00:41:10,350
times my e^(-ily) dxdy.
693
00:41:10,350 --> 00:41:15,020
694
00:41:15,020 --> 00:41:17,510
So that just makes the point.
695
00:41:17,510 --> 00:41:22,750
That there's nothing new here,
it's just up a dimension.
696
00:41:22,750 --> 00:41:24,700
But the formulas
all look the same,
697
00:41:24,700 --> 00:41:31,620
and if F was a-- Well, if F is
a delta function, if F is now
698
00:41:31,620 --> 00:41:34,690
a 2-D delta function.
699
00:41:34,690 --> 00:41:38,500
We haven't done delta
functions in 2-D, why don't we?
700
00:41:38,500 --> 00:41:45,340
Suppose F is the delta
function in 2-D. Then
701
00:41:45,340 --> 00:41:49,110
what are the coefficients?
702
00:41:49,110 --> 00:41:50,860
What do you think
you this means,
703
00:41:50,860 --> 00:41:53,230
this delta function in 2-D?
704
00:41:53,230 --> 00:41:57,250
So if I put in the delta
here, and I integrate.
705
00:41:57,250 --> 00:42:01,350
And what do I get then?
706
00:42:01,350 --> 00:42:06,280
So if this guy is a delta, a
two-dimensional delta function,
707
00:42:06,280 --> 00:42:10,990
the rule is that
when I integrate over
708
00:42:10,990 --> 00:42:15,230
a region that includes
the spike, so it's a spike
709
00:42:15,230 --> 00:42:19,610
sitting up above a plane now,
instead of sitting above a line
710
00:42:19,610 --> 00:42:21,250
it's sitting above a plane.
711
00:42:21,250 --> 00:42:25,700
Then I get the value, so
this is the delta function
712
00:42:25,700 --> 00:42:27,030
at the origin.
713
00:42:27,030 --> 00:42:29,690
So I get the value of
this at the origin.
714
00:42:29,690 --> 00:42:32,100
So what answer do I get?
715
00:42:32,100 --> 00:42:34,110
I get one out of the
integral and then
716
00:42:34,110 --> 00:42:35,360
I just have this constant.
717
00:42:35,360 --> 00:42:38,970
So it's constant again.
718
00:42:38,970 --> 00:42:41,460
And it's just one.
719
00:42:41,460 --> 00:42:45,950
So the Fourier coefficients
of the delta function
720
00:42:45,950 --> 00:42:46,600
are constant.
721
00:42:46,600 --> 00:42:49,700
All frequencies
there are the same.
722
00:42:49,700 --> 00:42:53,020
What about a line
of delta functions?
723
00:42:53,020 --> 00:42:54,460
And what does that mean?
724
00:42:54,460 --> 00:43:01,730
What about, yeah let me
try to draw delta(x).
725
00:43:01,730 --> 00:43:06,770
Suppose I have a
function of x and y --
726
00:43:06,770 --> 00:43:12,290
it's just worth imagining
a line of delta functions.
727
00:43:12,290 --> 00:43:21,490
So I'm in the xy-- Let me
look again at this thing.
728
00:43:21,490 --> 00:43:25,480
I have delta functions
all along this line.
729
00:43:25,480 --> 00:43:26,530
Now.
730
00:43:26,530 --> 00:43:29,520
Here is a crazy example,
just to say well there
731
00:43:29,520 --> 00:43:34,140
is something new in 2-D. So
previously my delta function
732
00:43:34,140 --> 00:43:36,210
was just at that point.
733
00:43:36,210 --> 00:43:38,130
And the old integrals
just picked out
734
00:43:38,130 --> 00:43:40,210
the value at that point.
735
00:43:40,210 --> 00:43:43,490
But now think of
a delta function's
736
00:43:43,490 --> 00:43:46,400
a sort of line of spikes.
737
00:43:46,400 --> 00:43:48,820
Going up here, and then
of course it's periodic.
738
00:43:48,820 --> 00:43:51,970
Everything's periodic
so that line continues
739
00:43:51,970 --> 00:43:55,280
and this line appears here,
and this line appears here.
740
00:43:55,280 --> 00:44:02,010
But I only have to focus
on one period square.
741
00:44:02,010 --> 00:44:04,750
What's my answer now?
742
00:44:04,750 --> 00:44:10,930
If this function suddenly
changes from a one-point delta
743
00:44:10,930 --> 00:44:13,210
function to a line
of delta functions?
744
00:44:13,210 --> 00:44:17,390
Now tell me what the
coefficients are.
745
00:44:17,390 --> 00:44:19,240
What are the
Fourier coefficients
746
00:44:19,240 --> 00:44:22,920
in 2-D for a line
of delta functions?
747
00:44:22,920 --> 00:44:28,760
A straight line of delta
functions going up the y axis?
748
00:44:28,760 --> 00:44:32,260
It'll be-- Let's see.
749
00:44:32,260 --> 00:44:34,850
What do I do?
750
00:44:34,850 --> 00:44:39,060
I'm go to integrate--
Oh yeah, what is it?
751
00:44:39,060 --> 00:44:40,600
Good question.
752
00:44:40,600 --> 00:44:42,450
OK.
753
00:44:42,450 --> 00:44:47,260
So what is the--
When do I get zero
754
00:44:47,260 --> 00:44:49,570
and when do I not
get zero out of this?
755
00:44:49,570 --> 00:44:54,600
Yeah, tell me first when do I
get zero out of this integral?
756
00:44:54,600 --> 00:44:56,250
And when do I not?
757
00:44:56,250 --> 00:45:00,170
What am I doing here?
758
00:45:00,170 --> 00:45:01,690
Help me.
759
00:45:01,690 --> 00:45:06,350
I said 2-D was easy and I've
got in over my head here.
760
00:45:06,350 --> 00:45:10,070
So look.
761
00:45:10,070 --> 00:45:12,700
I can do the x integral, right?
762
00:45:12,700 --> 00:45:15,950
We all know how to
do the x integral.
763
00:45:15,950 --> 00:45:18,440
Yes, is that right?
764
00:45:18,440 --> 00:45:22,142
If I integrate with respect
to x, what do I get?
765
00:45:22,142 --> 00:45:24,100
Let's see, I'll keep that
one over 2pi squared.
766
00:45:24,100 --> 00:45:28,680
Now I'm trying to
do this integral.
767
00:45:28,680 --> 00:45:34,710
Do I get a one?
768
00:45:34,710 --> 00:45:37,840
If I get a one from
the x integral.
769
00:45:37,840 --> 00:45:40,410
So then I'm down
to one integral,
770
00:45:40,410 --> 00:45:47,160
just the y integral is
left. e^(-ily) dy, right?
771
00:45:47,160 --> 00:45:52,720
I did the x part, which said,
OK, take the value at x=0,
772
00:45:52,720 --> 00:45:53,990
which was one.
773
00:45:53,990 --> 00:45:57,850
So the x integral was one, good.
774
00:45:57,850 --> 00:45:59,900
And now I've got
down to this part.
775
00:45:59,900 --> 00:46:03,950
Now what is that integral?
776
00:46:03,950 --> 00:46:10,230
It's two-- wait a minute.
777
00:46:10,230 --> 00:46:16,570
Depends on l, doesn't it?
778
00:46:16,570 --> 00:46:21,940
When l-- Yeah, so it's going
to depend on whether l is zero
779
00:46:21,940 --> 00:46:23,080
or not.
780
00:46:23,080 --> 00:46:26,080
Is that right?
781
00:46:26,080 --> 00:46:28,480
Yeah, that's sort
of interesting.
782
00:46:28,480 --> 00:46:33,670
If l is zero, then I'm
getting -- then this is a 2pi.
783
00:46:33,670 --> 00:46:38,820
So the answer, so
I'm getting c_k0,
784
00:46:38,820 --> 00:46:42,530
when l is zero I'm
getting a 2pi out of that.
785
00:46:42,530 --> 00:46:44,820
If l is zero, I'm
integrating one,
786
00:46:44,820 --> 00:46:46,940
I get a 2pi, cancels
one of those.
787
00:46:46,940 --> 00:46:48,020
I get a one over 2pi.
788
00:46:48,020 --> 00:46:51,760
789
00:46:51,760 --> 00:46:59,280
And otherwise the other c_kl's,
when l is not zero, are what?
790
00:46:59,280 --> 00:47:00,270
Just zero, I think.
791
00:47:00,270 --> 00:47:03,120
The integral of this thing,
this is a periodic guy,
792
00:47:03,120 --> 00:47:07,290
if I integrate it from
minus pi to pi it's zero.
793
00:47:07,290 --> 00:47:11,670
What am I-- I'm making a
big deal out of something
794
00:47:11,670 --> 00:47:14,070
that shouldn't be a big deal.
795
00:47:14,070 --> 00:47:20,020
The delta(x) function,
this is just,
796
00:47:20,020 --> 00:47:23,470
its Fourier series is
just the one we know.
797
00:47:23,470 --> 00:47:25,830
Sum of e^(ikx)'s.
798
00:47:25,830 --> 00:47:31,300
799
00:47:31,300 --> 00:47:33,210
Do you see what's happened here?
800
00:47:33,210 --> 00:47:36,850
It was supposed to
be a double sum.
801
00:47:36,850 --> 00:47:41,060
But the ones, when l
wasn't zero, aren't there.
802
00:47:41,060 --> 00:47:42,270
The only ones-- Yeah.
803
00:47:42,270 --> 00:47:48,020
So I'm back to the, for a line
of spikes, a line of deltas,
804
00:47:48,020 --> 00:47:50,410
I'm back to-- So
it only depended
805
00:47:50,410 --> 00:47:54,020
on x, so the Fourier series is
just the one I already know.
806
00:47:54,020 --> 00:47:56,660
All ones.
807
00:47:56,660 --> 00:47:59,470
When there's no--
When l is zero,
808
00:47:59,470 --> 00:48:05,060
all ones or 1/(2pi)'s, all
constants when l is zero,
809
00:48:05,060 --> 00:48:11,340
but there's no y-- There's no
oscillation in the y direction.
810
00:48:11,340 --> 00:48:17,810
OK, I don't know why I
got into that example,
811
00:48:17,810 --> 00:48:21,560
because the conclusion was
just it's the Fourier series
812
00:48:21,560 --> 00:48:25,330
that we already know and
it doesn't depend on l,
813
00:48:25,330 --> 00:48:28,940
because the function
didn't depend on y.
814
00:48:28,940 --> 00:48:32,840
OK, then we could
imagine delta functions
815
00:48:32,840 --> 00:48:36,630
in other positions,
or a general function.
816
00:48:36,630 --> 00:48:44,220
OK, so that's 2-D. Would I
want to tackle a 2-D-- Ha,
817
00:48:44,220 --> 00:48:48,370
we've got two minutes. that's
one dimension a minute.
818
00:48:48,370 --> 00:48:50,500
Right, OK.
819
00:48:50,500 --> 00:48:57,400
What happens, what's a
2-D discrete convolution?
820
00:48:57,400 --> 00:49:00,480
What's a 2-D
discrete convolution?
821
00:49:00,480 --> 00:49:03,180
Now, you might say OK, why
is Professor Strang inventing
822
00:49:03,180 --> 00:49:04,380
these problems?
823
00:49:04,380 --> 00:49:07,240
Because a 2-D
discrete convolution
824
00:49:07,240 --> 00:49:10,170
is the core idea of
image processing.
825
00:49:10,170 --> 00:49:14,440
If I have an image, what
does image processing do?
826
00:49:14,440 --> 00:49:18,360
Image processing
takes my image, it
827
00:49:18,360 --> 00:49:20,860
separates it into
pixels, right, that's
828
00:49:20,860 --> 00:49:23,980
all the image is,
bunch of pixels.
829
00:49:23,980 --> 00:49:28,720
Then many 2-D image
processing algorithms,
830
00:49:28,720 --> 00:49:32,100
will take-- JPEG,
old JPEG for example,
831
00:49:32,100 --> 00:49:36,720
would take an eight by
eight, eight by eight 2-D,
832
00:49:36,720 --> 00:49:39,130
in other words--
Eight by eight, 2-D
833
00:49:39,130 --> 00:49:42,600
is the main point,
set of pixels.
834
00:49:42,600 --> 00:49:43,740
And transform it.
835
00:49:43,740 --> 00:49:46,020
Do a 2-D transform.
836
00:49:46,020 --> 00:49:48,610
So what is a 2-D transform?
837
00:49:48,610 --> 00:49:56,730
What would be the 2-D transform
that would correspond to this?
838
00:49:56,730 --> 00:49:59,420
First of all how
big's the matrix?
839
00:49:59,420 --> 00:50:01,470
Just so we get an idea.
840
00:50:01,470 --> 00:50:04,250
I probably won't get to
the end of this example.
841
00:50:04,250 --> 00:50:10,290
But just, so in 1-D, my
matrix was four by four.
842
00:50:10,290 --> 00:50:15,740
Now I've got, so that was
for four points on a line.
843
00:50:15,740 --> 00:50:22,085
Now I've got a square of points.
844
00:50:22,085 --> 00:50:25,560
So how big is my matrix?
845
00:50:25,560 --> 00:50:26,830
16, right?
846
00:50:26,830 --> 00:50:32,590
16 by 16, because it's
operating on 16 pixels.
847
00:50:32,590 --> 00:50:37,370
It's operating on 16
pixels, where in 1-D it only
848
00:50:37,370 --> 00:50:38,790
had four to act on.
849
00:50:38,790 --> 00:50:42,960
So I'm going to end up with
a 16 by 16 matrix here.
850
00:50:42,960 --> 00:50:48,150
And I think-- Let me
see, what do I need?
851
00:50:48,150 --> 00:50:49,630
Oh, wait a minute.
852
00:50:49,630 --> 00:50:51,330
Uh-oh.
853
00:50:51,330 --> 00:50:54,620
Yeah, I think the
time's up here.
854
00:50:54,620 --> 00:50:59,080
Yeah, because my C, has my
C got to have 16 components?
855
00:50:59,080 --> 00:51:00,140
Yes.
856
00:51:00,140 --> 00:51:03,010
My u has to have 16,
my right-hand side
857
00:51:03,010 --> 00:51:05,060
has got these 16 components.
858
00:51:05,060 --> 00:51:05,600
Yeah.
859
00:51:05,600 --> 00:51:09,900
So I'm up to 16 but a
very special circulant
860
00:51:09,900 --> 00:51:11,610
of a circulant.
861
00:51:11,610 --> 00:51:13,750
It'll be a circulant
of a circulant somehow.
862
00:51:13,750 --> 00:51:17,150
OK, enough for 2-D.
I'll see you Wednesday
863
00:51:17,150 --> 00:51:18,430
and we're back to reality.
864
00:51:18,430 --> 00:51:19,581
OK.
865
00:51:19,581 --> 00:51:20,080