1
00:00:00 --> 00:00:01
2
00:00:01 --> 00:00:02
The following content is
provided under a Creative
3
00:00:02 --> 00:00:03
Commons license.
4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to
5
00:00:06 --> 00:00:09
offer high-quality educational
resources for free.
6
00:00:09 --> 00:00:13
To make a donation, or to view
additional materials from
7
00:00:13 --> 00:00:15
hundreds of MIT courses, visit
MIT OpenCourseWare
8
00:00:15 --> 00:00:20
at ocw.mit.edu.
9
00:00:20 --> 00:00:25
PROFESSOR STRANG: OK, so I
thought I'd quickly list the
10
00:00:25 --> 00:00:29
four central topics for the
Fourier part of the course, and
11
00:00:29 --> 00:00:32
I would guess that there'll be
a question on each of those
12
00:00:32 --> 00:00:36
four in the final quiz.
13
00:00:36 --> 00:00:39
Final exam.
14
00:00:39 --> 00:00:43
And we covered 4.1,
Fourier series.
15
00:00:43 --> 00:00:50
I don't plan to do discussion
in 4.2 of other series like
16
00:00:50 --> 00:00:53
Bessel and Legendre and so on.
17
00:00:53 --> 00:00:56
If I can, I'll come
back to those in the
18
00:00:56 --> 00:00:58
very last lectures.
19
00:00:58 --> 00:01:06
But I want to pick up now on
the second key example, which
20
00:01:06 --> 00:01:10
is the discrete Fourier series
that has only N terms.
21
00:01:10 --> 00:01:13
Instead this series, as
you're looking at it, has
22
00:01:13 --> 00:01:15
infinitely many terms.
23
00:01:15 --> 00:01:20
But I'm going to cut
back to n terms.
24
00:01:20 --> 00:01:25
So this K, well I can go
one to N, but the best
25
00:01:25 --> 00:01:27
way is zero to N-1.
26
00:01:29 --> 00:01:34
So that's the discrete series.
27
00:01:34 --> 00:01:38
We're going to be dealing with
a vector, c_0 up to c_(n-1).
28
00:01:40 --> 00:01:44
And a vector of
function values.
29
00:01:44 --> 00:01:46
Vector to vector.
30
00:01:46 --> 00:01:49
So a matrix is going to be
the key to everything here.
31
00:01:49 --> 00:01:51
An n by n matrix.
32
00:01:51 --> 00:01:52
So that's what's coming.
33
00:01:52 --> 00:01:58
Then, after that probably
by Friday will be the
34
00:01:58 --> 00:01:59
integral transform.
35
00:01:59 --> 00:02:04
And early next week, and a
little after Thanksgiving
36
00:02:04 --> 00:02:08
would be this key idea of
convolution, which is really
37
00:02:08 --> 00:02:12
important in so many
applications and needs
38
00:02:12 --> 00:02:14
to be separated out.
39
00:02:14 --> 00:02:17
Convolution will apply
to all of these.
40
00:02:17 --> 00:02:20
And you'll see the point there.
41
00:02:20 --> 00:02:24
OK, ready for the idea
of the discrete one.
42
00:02:24 --> 00:02:28
OK, so the key idea is we only
have N terms, so I can't expect
43
00:02:28 --> 00:02:30
to reproduce a whole function.
44
00:02:30 --> 00:02:35
I can get n function value,
so I'll get this at
45
00:02:35 --> 00:02:37
n different points.
46
00:02:37 --> 00:02:42
And if I'm thinking of my F(x),
on, let me take zero to 2pi
47
00:02:42 --> 00:02:47
now, and I'm thinking again
of an F(x) that's periodic.
48
00:02:47 --> 00:02:51
So it goes, does whatever.
49
00:02:51 --> 00:02:57
If it was like that, if that
was my F(x), just remember
50
00:02:57 --> 00:03:00
about Fourier series, what
could you tell me about the
51
00:03:00 --> 00:03:02
Fourier series for that F(x)?
52
00:03:03 --> 00:03:07
Well, when I continue,
periodically am I
53
00:03:07 --> 00:03:09
going to see a jump?
54
00:03:09 --> 00:03:10
Yes.
55
00:03:10 --> 00:03:18
There'll be a jump at every
2pi, so the underlying function
56
00:03:18 --> 00:03:28
is not continuous, has a jump,
so that our special examples
57
00:03:28 --> 00:03:34
with jumps and slow decay, 1/k
decay rate for the Fourier
58
00:03:34 --> 00:03:36
coefficients would apply here.
59
00:03:36 --> 00:03:39
OK, that's remembering
Fourier series.
60
00:03:39 --> 00:03:42
Now, Fourier integrals,
I'm just going to take,
61
00:03:42 --> 00:03:43
let me just take N=4.
62
00:03:43 --> 00:03:46
63
00:03:46 --> 00:03:49
So I'll take this point,
you know what four points
64
00:03:49 --> 00:03:50
am I going to take?
65
00:03:50 --> 00:03:54
I'm not going to repeat, ah.
66
00:03:54 --> 00:04:00
Just for the heck of it,
let's make it look periodic.
67
00:04:00 --> 00:04:02
Yeah, let's make
it look periodic.
68
00:04:02 --> 00:04:04
I never should have
done it otherwise.
69
00:04:04 --> 00:04:08
OK, so there's one point.
70
00:04:08 --> 00:04:12
I'm just going to take four
equally spaced, equally
71
00:04:12 --> 00:04:17
spaced being key,
zero, one, two, three.
72
00:04:17 --> 00:04:21
So N-1 is three.
73
00:04:21 --> 00:04:28
And these four values
I can get right.
74
00:04:28 --> 00:04:30
So those are my four x's.
75
00:04:30 --> 00:04:33
The four x's that
I'm going to take.
76
00:04:33 --> 00:04:36
So this is x=0, of course.
77
00:04:36 --> 00:04:41
And this one is, what's
the x there? x is
78
00:04:41 --> 00:04:43
2pi, the whole deal.
79
00:04:43 --> 00:04:46
The whole interval, but
it's divided into N
80
00:04:46 --> 00:04:47
pieces, so 2pi/N.
81
00:04:47 --> 00:04:50
82
00:04:50 --> 00:04:54
So we are going to see a lot
of fractions involving pi/N.
83
00:04:55 --> 00:04:56
Or 2pi/N.
84
00:04:57 --> 00:04:59
They're just going to
come up everywhere.
85
00:04:59 --> 00:05:04
Because that's the delta
x, the delta h, the step
86
00:05:04 --> 00:05:08
in the discrete form.
87
00:05:08 --> 00:05:08
OK.
88
00:05:08 --> 00:05:11
So those are the x's.
89
00:05:11 --> 00:05:22
So I'm only going to get this
at x=0, 2pi/N, 4pi/N, up to
90
00:05:22 --> 00:05:24
whatever it comes out
to, (N-1)*2pi/N.
91
00:05:24 --> 00:05:28
92
00:05:28 --> 00:05:31
That doesn't look elegant
so we probably will try to
93
00:05:31 --> 00:05:32
avoid writing that again.
94
00:05:32 --> 00:05:39
But this was the three times
2pi/4 that got us to there.
95
00:05:39 --> 00:05:45
So this is 2pi/8, pi/4.
96
00:05:46 --> 00:05:50
Sorry, 2pi/4, 4pi/4,
6pi - oh, ok.
97
00:05:50 --> 00:05:54
So we're only going to get
equality at those x's.
98
00:05:54 --> 00:05:58
So it's those x's that
I should plug in here.
99
00:05:58 --> 00:06:00
OK, so let me do that.
100
00:06:00 --> 00:06:05
Let me plus in, what does
this series look like
101
00:06:05 --> 00:06:09
at these four points?
102
00:06:09 --> 00:06:10
Let me see.
103
00:06:10 --> 00:06:17
So writing out all four
equations, F(0) is going
104
00:06:17 --> 00:06:18
to match the right side.
105
00:06:18 --> 00:06:21
At zero, because zero
is one of my points.
106
00:06:21 --> 00:06:23
So I get c_0+c_1+c_2+c_3.
107
00:06:30 --> 00:06:35
At x=0, right, all the e to
the whatevers were one.
108
00:06:35 --> 00:06:44
Now at the next x, 2pi/N, now
I'm matching the series with a
109
00:06:44 --> 00:06:46
function at that value, there.
110
00:06:46 --> 00:06:46
2pi/N.
111
00:06:48 --> 00:06:54
So now I have k=0, so
what does k=0 give me?
112
00:06:54 --> 00:06:58
When k is zero, what's
my term look like?
113
00:06:58 --> 00:07:03
It's just c_0 times one.
114
00:07:03 --> 00:07:07
Because if k is zero, e to
the whatever is again one.
115
00:07:07 --> 00:07:09
So this is just c_0.
116
00:07:09 --> 00:07:11
Now, here's the key.
117
00:07:11 --> 00:07:17
What is that business
there at x=2pi/N?
118
00:07:19 --> 00:07:22
Here comes the important number
in this whole business.
119
00:07:22 --> 00:07:25
The important number this
whole business is, I'll
120
00:07:25 --> 00:07:28
call it w, is e^(2pi*i)/N.
121
00:07:28 --> 00:07:33
122
00:07:33 --> 00:07:40
That's a complex number,
and we'll place it in
123
00:07:40 --> 00:07:42
the complex plane.
124
00:07:42 --> 00:07:46
So you must get to
know that number. w.
125
00:07:46 --> 00:07:48
Because when I plug in x equal
126
00:07:48 --> 00:07:51
- k is now one here,
so I have c_1.
127
00:07:53 --> 00:08:01
Times this e to the i one,
2pi/N, what is that? w.
128
00:08:01 --> 00:08:02
So it's w*c_1.
129
00:08:04 --> 00:08:08
And what do I get from the
next term in this series?
130
00:08:08 --> 00:08:08
I have c_2*e^(i2x).
131
00:08:08 --> 00:08:14
132
00:08:14 --> 00:08:20
x is 2pi/N, do you see
what's happening? w
133
00:08:20 --> 00:08:22
squared is e^(4pi*i/N).
134
00:08:22 --> 00:08:26
135
00:08:26 --> 00:08:29
We're using the beauty
of exponentials, that's
136
00:08:29 --> 00:08:30
making everything go.
137
00:08:30 --> 00:08:32
It's w squared.
138
00:08:32 --> 00:08:35
And this guy would
be w cubed, c_3.
139
00:08:35 --> 00:08:38
140
00:08:38 --> 00:08:41
That's the series.
141
00:08:41 --> 00:08:45
At our points.
142
00:08:45 --> 00:08:48
So this, like, we'll
just see this once.
143
00:08:48 --> 00:08:49
What were we doing?
144
00:08:49 --> 00:08:51
But this is what we're
going to live with here.
145
00:08:51 --> 00:08:57
So I'll rewrite it all in terms
of w, so you just, you've got
146
00:08:57 --> 00:08:58
the idea of what are we doing?
147
00:08:58 --> 00:09:04
We're matching at N points and
now we get the good notation w.
148
00:09:04 --> 00:09:07
OK, and at the next
point, 4pi/N.
149
00:09:09 --> 00:09:12
Well, again, k=0
term, it's c_0.
150
00:09:13 --> 00:09:19
The k=1 term, can you tell me
what I get when k is one, I
151
00:09:19 --> 00:09:30
have something times c_1, it's
e^(i*4pi/N), what's that? w
152
00:09:30 --> 00:09:35
squared. e^(i*4pi/N)
is w squared.
153
00:09:35 --> 00:09:39
Look, this matrix is
coming out symmetric.
154
00:09:39 --> 00:09:45
And this will be w^4, and
that'll turn out to be w^6, and
155
00:09:45 --> 00:09:50
then the last row of the matrix
is going to come from matching
156
00:09:50 --> 00:09:56
at 6pi/N, and it will turn
out to be c_0, and it'll
157
00:09:56 --> 00:10:02
be w^3, w^6, and w^9.
158
00:10:02 --> 00:10:06
159
00:10:06 --> 00:10:10
So I did the last row
or two a bit fast.
160
00:10:10 --> 00:10:15
Just because the patterns
there, and you'll spot it.
161
00:10:15 --> 00:10:22
Now I want to pull
out of the matrix.
162
00:10:22 --> 00:10:26
The Fourier matrix, the four
by four Fourier matrix
163
00:10:26 --> 00:10:29
that multiplies the c's
and gives me the F's.
164
00:10:29 --> 00:10:36
Let me call these numbers are
y_0, y_1, y_2, and y_3, just
165
00:10:36 --> 00:10:41
to have a, so it's a vector
of values of the function.
166
00:10:41 --> 00:10:43
And this is a vector
of coefficients.
167
00:10:43 --> 00:10:47
So it's wise to - the Fourier
transform goes between
168
00:10:47 --> 00:10:56
y's and c's, and y's.
169
00:10:56 --> 00:11:02
Connects the vector, and this
is N values, N function
170
00:11:02 --> 00:11:06
values in physical space.
171
00:11:06 --> 00:11:11
These are N coefficients in
frequency space, and one way is
172
00:11:11 --> 00:11:14
the discrete Fourier transform
and the other way is
173
00:11:14 --> 00:11:17
the inverse discrete
Fourier transform.
174
00:11:17 --> 00:11:20
So, and it's a little
bit confused, which
175
00:11:20 --> 00:11:22
is which, actually.
176
00:11:22 --> 00:11:25
There was no question with
Fourier series, it was easy to
177
00:11:25 --> 00:11:28
tell the function from the
coefficients because the
178
00:11:28 --> 00:11:30
function was f x and the
coefficients were a
179
00:11:30 --> 00:11:31
bunch of numbers.
180
00:11:31 --> 00:11:34
Here we only have N numbers.
181
00:11:34 --> 00:11:40
N y's and N c's and we're
transforming back and forth.
182
00:11:40 --> 00:11:45
So I'm finding right now the
matrix, this Fourier matrix
183
00:11:45 --> 00:11:48
is going to take the c's
and give me the y's.
184
00:11:48 --> 00:11:52
So I'm talking here
about the matrix F.
185
00:11:52 --> 00:11:54
And let's see what F is.
186
00:11:54 --> 00:12:04
Can you just see what, that was
y_0, to y_3, equals this matrix
187
00:12:04 --> 00:12:06
F, the four by four matrix.
188
00:12:06 --> 00:12:08
Times c_0 to c_3.
189
00:12:10 --> 00:12:14
This is really the inverse.
190
00:12:14 --> 00:12:14
DFT.
191
00:12:14 --> 00:12:20
192
00:12:20 --> 00:12:27
It reconstructs the y's from
the c's, or an easier way to
193
00:12:27 --> 00:12:32
say that is add up the series.
194
00:12:32 --> 00:12:39
So that's the usual DFT is
taking us, starting with the
195
00:12:39 --> 00:12:41
y's and giving us the c's.
196
00:12:41 --> 00:12:46
So this would be the
discrete Fourier transform.
197
00:12:46 --> 00:12:50
Take a function, produce
its coefficients.
198
00:12:50 --> 00:12:56
The reverse step is take the
coefficients, add back, add up
199
00:12:56 --> 00:12:59
the series, get the function.
200
00:12:59 --> 00:13:02
And that's what F does.
201
00:13:02 --> 00:13:05
So I'm going to focus on F.
202
00:13:05 --> 00:13:09
Which is the ones that
involves this w guy.
203
00:13:09 --> 00:13:13
OK, can you read off, peel off
from here what the matrix
204
00:13:13 --> 00:13:15
F is, the Fourier matrix?
205
00:13:15 --> 00:13:20
Its first row is all ones.
206
00:13:20 --> 00:13:24
Its first column is all ones.
207
00:13:24 --> 00:13:31
And then it's one, w,
w^2, w^3, w^2, w^4,
208
00:13:31 --> 00:13:35
w^6, w^3, w^6, and w^9.
209
00:13:37 --> 00:13:39
To the ninth power.
210
00:13:39 --> 00:13:43
OK, so it's made up of
these powers of w.
211
00:13:43 --> 00:13:49
Zero first, second, and third
power and then higher powers.
212
00:13:49 --> 00:13:52
But higher powers are -
let's draw a picture.
213
00:13:52 --> 00:13:54
Where is w?
214
00:13:54 --> 00:13:57
So this is the complex plane.
215
00:13:57 --> 00:14:00
Here's the real, and here's
the imaginary direction.
216
00:14:00 --> 00:14:05
And let me take N to
be, here N was four.
217
00:14:05 --> 00:14:10
Let me take N to be eight in my
picture, just to have some,
218
00:14:10 --> 00:14:13
then I'll be able to spot
the fours and the eights.
219
00:14:13 --> 00:14:16
If N is eight, where
would w^8 be?
220
00:14:16 --> 00:14:32
So w^8 is meant to be
the e^(2pi*i/8), now.
221
00:14:32 --> 00:14:38
Where's that number
in that picture?
222
00:14:38 --> 00:14:43
Is it on the circle?
223
00:14:43 --> 00:14:46
Absolutely.
224
00:14:46 --> 00:14:51
This discrete Fourier transform
never gets off the circle.
225
00:14:51 --> 00:14:55
And in fact, it never gets off
eight points on the circle.
226
00:14:55 --> 00:14:58
Which are the eight
equally spaced points.
227
00:14:58 --> 00:15:02
This says go 1/8 of the way
of the whole way around,
228
00:15:02 --> 00:15:04
which will be here.
229
00:15:04 --> 00:15:05
So there's w^8.
230
00:15:05 --> 00:15:08
231
00:15:08 --> 00:15:11
When I square it, what's
the square of w^8?
232
00:15:12 --> 00:15:13
The angle.
233
00:15:13 --> 00:15:17
What happens to the angle
when I multiply numbers?
234
00:15:17 --> 00:15:19
I add the exponents.
235
00:15:19 --> 00:15:22
Here, if I'm multiplying
by itself, it'll
236
00:15:22 --> 00:15:23
double the exponent.
237
00:15:23 --> 00:15:24
Double the angle.
238
00:15:24 --> 00:15:34
It'll be, there's w^8 squared,
so that point is w^8 squared.
239
00:15:34 --> 00:15:37
And it's the same as
w^4 because it's 1/4
240
00:15:37 --> 00:15:39
of the way around.
241
00:15:39 --> 00:15:45
And what is that number
in ordinary language? i.
242
00:15:45 --> 00:15:47
It's i, right?
243
00:15:47 --> 00:15:53
It's on the unit
circle at 90 degrees.
244
00:15:53 --> 00:15:56
And on the imaginary
axis it's just i.
245
00:15:56 --> 00:15:59
So all this here
was i, i^2 i^3.
246
00:16:00 --> 00:16:01
i^2, i^4, i^6.
247
00:16:02 --> 00:16:02
i^3, i^6, i^9.
248
00:16:04 --> 00:16:09
The four by four Fourier
matrix is just powers of i.
249
00:16:09 --> 00:16:12
And the N by N, the eight
by eight Fourier matrix
250
00:16:12 --> 00:16:13
is powers of w^8.
251
00:16:15 --> 00:16:16
And let's find all the
rest of the powers.
252
00:16:16 --> 00:16:21
So there's w^8, w^8 squared,
where's w^8 cubed?
253
00:16:21 --> 00:16:22
Here.
254
00:16:22 --> 00:16:25
I multiply those guys, I add
the angle, I'm out to here.
255
00:16:25 --> 00:16:28
Here's the fourth power, here's
the fifth power, sixth power,
256
00:16:28 --> 00:16:34
seventh power and finally we
get to w^8, to the eighth
257
00:16:34 --> 00:16:37
power, which is one.
258
00:16:37 --> 00:16:40
Because if I take the eighth
power here, I've got
259
00:16:40 --> 00:16:45
e^(2pi*8), and that's one.
260
00:16:45 --> 00:16:50
So that's the picture
to remember.
261
00:16:50 --> 00:16:57
And while looking at that
picture, could you tell me the
262
00:16:57 --> 00:17:00
sum of those eight numbers?
263
00:17:00 --> 00:17:03
What do those eight
members add up to?
264
00:17:03 --> 00:17:13
So I just want to ask what do
w^1 plus w up to w^7, when I'm
265
00:17:13 --> 00:17:17
doing the eighth
guy, adds up to?
266
00:17:17 --> 00:17:21
So the sum of those eight
numbers, well I wouldn't ask
267
00:17:21 --> 00:17:24
the question if it didn't
have a nice answer.
268
00:17:24 --> 00:17:24
Zero.
269
00:17:24 --> 00:17:29
That's of course the
usual answer in math.
270
00:17:29 --> 00:17:34
We do all this work
and we get zero.
271
00:17:34 --> 00:17:37
But now, of course the
real math is why.
272
00:17:37 --> 00:17:43
I mean, so math is, you get
equations but then always
273
00:17:43 --> 00:17:45
there's that kicker, why.
274
00:17:45 --> 00:17:48
And why do they add up to zero?
275
00:17:48 --> 00:17:51
Here.
276
00:17:51 --> 00:17:54
Can you see that?
277
00:17:54 --> 00:18:00
Why those eight numbers
would add to zero?
278
00:18:00 --> 00:18:02
Well, yeah.
279
00:18:02 --> 00:18:04
So what do these add up to?
280
00:18:04 --> 00:18:06
Just those two guys?
281
00:18:06 --> 00:18:09
One and minus one add to zero.
282
00:18:09 --> 00:18:13
That and that add to zero,
I can pair them off.
283
00:18:13 --> 00:18:15
This and this add to zero.
284
00:18:15 --> 00:18:17
That and that add to zero.
285
00:18:17 --> 00:18:19
Yeah, so that's one
way of seeing it.
286
00:18:19 --> 00:18:24
But actually, even if N was
three or five or some odd
287
00:18:24 --> 00:18:27
number, and I had - so should
I do the picture for N=3?
288
00:18:28 --> 00:18:31
Just a little, doesn't need
a very big picture for N=3.
289
00:18:32 --> 00:18:34
So where would the
roots be for N=3?
290
00:18:35 --> 00:18:37
One of them is always one.
291
00:18:37 --> 00:18:41
These are the three
cube roots of one.
292
00:18:41 --> 00:18:44
That I'm going to draw.
293
00:18:44 --> 00:18:48
Those were the eight
eighth roots of one.
294
00:18:48 --> 00:18:51
Because if I take any one of
those numbers its eighth
295
00:18:51 --> 00:18:55
power brings me back to one.
296
00:18:55 --> 00:18:59
So where are these guys, so
this is three numbers equally
297
00:18:59 --> 00:19:00
spaced around the circle.
298
00:19:00 --> 00:19:02
Where do you think they are?
299
00:19:02 --> 00:19:10
Well, it's gotta be at 120
degrees, 240 degrees and 360.
300
00:19:10 --> 00:19:14
And again, those
will add to zero.
301
00:19:14 --> 00:19:18
You can't pair them off because
we got an odd number, but it's
302
00:19:18 --> 00:19:23
safely zero and we
could see why.
303
00:19:23 --> 00:19:33
OK, so this section, this topic
is all about that matrix.
304
00:19:33 --> 00:19:37
It's all about that four by
four, or N by N matrix.
305
00:19:37 --> 00:19:39
Which has powers of w.
306
00:19:39 --> 00:19:43
Notice that the matrix is,
it hasn't got any zeroes.
307
00:19:43 --> 00:19:46
Hasn't even got anything
close to zero.
308
00:19:46 --> 00:19:50
All those numbers are
on the unit circle.
309
00:19:50 --> 00:19:58
So multiplying by that matrix,
which is the same as adding up
310
00:19:58 --> 00:20:06
the Fourier series at the four
points, doing this calculation,
311
00:20:06 --> 00:20:10
finding these y's from the c's,
reconstructing the y's, given
312
00:20:10 --> 00:20:16
the coefficients, has 16 terms.
313
00:20:16 --> 00:20:18
Right?
314
00:20:18 --> 00:20:22
The matrix has got 16 entries,
all the entries in the matrix
315
00:20:22 --> 00:20:25
are showing up here
in powers of w.
316
00:20:25 --> 00:20:28
We've got 16 multiplications.
317
00:20:28 --> 00:20:33
Well, maybe a couple of them
were easy, but basically 16
318
00:20:33 --> 00:20:36
multiplications and additions.
319
00:20:36 --> 00:20:39
Pretty much N squared work.
320
00:20:39 --> 00:20:50
And that's not bad.
321
00:20:50 --> 00:20:54
Except if you want to do it
a million times, right?
322
00:20:54 --> 00:20:58
Suppose we have a typical
N might be 1024.
323
00:20:58 --> 00:21:03
So the matrix has then got,
is 1024 squared numbers,
324
00:21:03 --> 00:21:05
which is about a million.
325
00:21:05 --> 00:21:09
So if we have a matrix with a
million entries, a million
326
00:21:09 --> 00:21:16
operations to do, and then we
do it a million times then it's
327
00:21:16 --> 00:21:21
getting up to money that
Congress would give
328
00:21:21 --> 00:21:23
to the banks.
329
00:21:23 --> 00:21:24
Whatever.
330
00:21:24 --> 00:21:25
Anyway.
331
00:21:25 --> 00:21:27
Serious money.
332
00:21:27 --> 00:21:31
Serious computing time.
333
00:21:31 --> 00:21:39
And the wonderful thing is
that there is a faster
334
00:21:39 --> 00:21:43
way than n squared.
335
00:21:43 --> 00:21:48
There's a faster way
to do these 16 terms
336
00:21:48 --> 00:21:50
than the obvious way.
337
00:21:50 --> 00:21:51
You might say, well,
they're pretty nice.
338
00:21:51 --> 00:21:53
And they are.
339
00:21:53 --> 00:21:55
But they're nicer
than you can know.
340
00:21:55 --> 00:21:58
I mean, nicer than you
can see immediately.
341
00:21:58 --> 00:22:08
This matrix will break up into
simple steps with lots of
342
00:22:08 --> 00:22:15
zeroes in each step, and the
final result is that instead of
343
00:22:15 --> 00:22:19
n squared, capital N squared,
which is done the old
344
00:22:19 --> 00:22:22
fashioned way, let's say.
345
00:22:22 --> 00:22:26
That that there's a fast
Fourier transform.
346
00:22:26 --> 00:22:33
So the FFT, I should maybe find
a better space for the most
347
00:22:33 --> 00:22:42
important algorithm in the last
hundred years, and here it is.
348
00:22:42 --> 00:22:45
Right there.
349
00:22:45 --> 00:22:49
So my one goal for Wednesday
will be to give you some
350
00:22:49 --> 00:22:52
insight into the FFT.
351
00:22:52 --> 00:22:55
But here I'm just
looking at the result.
352
00:22:55 --> 00:22:59
The number of computations
is, instead of N squared,
353
00:22:59 --> 00:23:03
it's N times log N.
354
00:23:03 --> 00:23:08
It's log to the base two,
or maybe 1/2 N log N.
355
00:23:08 --> 00:23:11
So I'll erase some of the
other things close by
356
00:23:11 --> 00:23:14
so you can see it.
357
00:23:14 --> 00:23:20
So that, where n squared was,
if N was a thousand, N squared
358
00:23:20 --> 00:23:26
was a million, If N is a
thousand, this is a thousand,
359
00:23:26 --> 00:23:30
and what's the logarithm of a
thousand, to the base two?
360
00:23:30 --> 00:23:33
That's the ten, right?
361
00:23:33 --> 00:23:36
The 2^10 gave us that 1,024.
362
00:23:36 --> 00:23:37
So this is 10^4.
363
00:23:39 --> 00:23:45
So N is 10^3, the logarithm
of 1,024 more exactly, the
364
00:23:45 --> 00:23:48
logarithm is ten, so we're
10^4 instead of 10^6.
365
00:23:51 --> 00:23:54
So that's a saving of a
factor of 100 that's true.
366
00:23:54 --> 00:24:00
I mean it just comes from doing
the addition and multiplication
367
00:24:00 --> 00:24:02
in the right order.
368
00:24:02 --> 00:24:05
You get this incredible saving.
369
00:24:05 --> 00:24:08
And it's sort of makes whole
calculations that were
370
00:24:08 --> 00:24:16
previously impossible are now
possible because instead of
371
00:24:16 --> 00:24:24
where it might have been 100
minutes, it's now one minute.
372
00:24:24 --> 00:24:29
So everything focuses on this
Fourier matrix and its inverse,
373
00:24:29 --> 00:24:35
which we still have to find but
it'll come out beautifully.
374
00:24:35 --> 00:24:38
In fact, I could tell you
what F inverse, just
375
00:24:38 --> 00:24:41
so you see it coming.
376
00:24:41 --> 00:24:45
F inverse will, so what
happened to w, that
377
00:24:45 --> 00:24:48
all-important number w?
378
00:24:48 --> 00:24:59
Well, let me repeat what w is.
379
00:24:59 --> 00:25:02
I'll put it on this board.
380
00:25:02 --> 00:25:06
F has powers of w.
w is e^(2pi*i/N).
381
00:25:06 --> 00:25:12
382
00:25:12 --> 00:25:14
And what power has it got?
383
00:25:14 --> 00:25:19
In the j,k position?
384
00:25:19 --> 00:25:25
So now you can see the
formula for these entries.
385
00:25:25 --> 00:25:29
Only, you have to let me
start the count at j
386
00:25:29 --> 00:25:31
and k equals zero.
387
00:25:31 --> 00:25:35
This is zero based, where
MATLAB is one based.
388
00:25:35 --> 00:25:40
And to do this stuff you always
have to shift things by one.
389
00:25:40 --> 00:25:44
Some other, more recent,
software like Python on
390
00:25:44 --> 00:25:50
is zero based, because
this is so common.
391
00:25:50 --> 00:25:54
So here's the formula.
392
00:25:54 --> 00:25:59
It's w to the power j times k.
393
00:25:59 --> 00:26:01
That's neat.
394
00:26:01 --> 00:26:06
That's what goes
into the matrix F.
395
00:26:06 --> 00:26:09
This here is row three.
396
00:26:09 --> 00:26:11
It looks like row four but
I'm starting with zero,
397
00:26:11 --> 00:26:13
so that's row three.
398
00:26:13 --> 00:26:14
That's column three.
399
00:26:14 --> 00:26:21
This is w^3 times three.
w^9, which by the way
400
00:26:21 --> 00:26:23
is the same as what?
401
00:26:23 --> 00:26:32
If w is i in this four by four,
it is. w is i, what's w^9,
402
00:26:32 --> 00:26:36
what's i to the 9th power?
403
00:26:36 --> 00:26:38
Same as?
404
00:26:38 --> 00:26:41
Can you do i^9 in your head?
405
00:26:41 --> 00:26:42
Sure.
406
00:26:42 --> 00:26:45
Because you're starting here.
407
00:26:45 --> 00:26:50
You're taking its ninth power,
so one, two, three, four,
408
00:26:50 --> 00:26:53
you're back to one. i^4 is one.
409
00:26:53 --> 00:26:57
One, two, three, four
more, i^8 is one. i^9.
410
00:26:57 --> 00:27:02
That will actually
be just i. i^9.
411
00:27:02 --> 00:27:04
So they're all powers of i.
412
00:27:04 --> 00:27:06
But this is the good
way to see it.
413
00:27:06 --> 00:27:09
It's three times three,
because it's in.
414
00:27:09 --> 00:27:12
And this, of course, is three
times two, and two times
415
00:27:12 --> 00:27:14
two, and three times one.
416
00:27:14 --> 00:27:15
You see the entries of F?
417
00:27:15 --> 00:27:16
Yeah.
418
00:27:16 --> 00:27:19
So now I'd better write down,
I promised to write this
419
00:27:19 --> 00:27:22
formula in a better form.
420
00:27:22 --> 00:27:27
All those formulas in a better
- nobody is going to write this
421
00:27:27 --> 00:27:31
out for matrices of
order 1000, right?
422
00:27:31 --> 00:27:36
So we've got to write
the decent formula.
423
00:27:36 --> 00:27:42
So the decent formula is that
the jth y, because it's going
424
00:27:42 --> 00:27:48
to come in equation number j,
will be the sum on k=0 to
425
00:27:48 --> 00:27:55
N-1, of c_k times w^jk.
426
00:27:55 --> 00:27:58
427
00:27:58 --> 00:28:01
Those are the entries,
those are the c's that it
428
00:28:01 --> 00:28:03
multiplies and the y's.
429
00:28:03 --> 00:28:06
This is just vector
y's, this is y=Fc.
430
00:28:06 --> 00:28:09
431
00:28:09 --> 00:28:12
Now we've finally got
a good notation.
432
00:28:12 --> 00:28:16
So this was horrible notation.
433
00:28:16 --> 00:28:19
I mean, that's exhausting.
434
00:28:19 --> 00:28:22
This shows you the algebra, but
you have to be sort of prepared
435
00:28:22 --> 00:28:25
- well, the information is here
because this is telling you
436
00:28:25 --> 00:28:30
what the entries of F, of the
matrix, are But this is with
437
00:28:30 --> 00:28:35
indices, and this is
with whole vectors.
438
00:28:35 --> 00:28:40
So that would be y, in MATLAB
that would be the inverse
439
00:28:40 --> 00:28:49
fast Fourier transform of c.
440
00:28:49 --> 00:28:52
So you see that
that's our matrix.
441
00:28:52 --> 00:28:58
Now, I was going to say before
I show why, I was going to
442
00:28:58 --> 00:29:01
say what about the inverse?
443
00:29:01 --> 00:29:08
Because if the FFT gave us a
fast way to multiply by F, we
444
00:29:08 --> 00:29:11
want it also to give us a
fast way to do the inverse.
445
00:29:11 --> 00:29:14
We have to go both ways here.
446
00:29:14 --> 00:29:19
We get our data, we put it into
frequency space, we look at
447
00:29:19 --> 00:29:22
it in frequency space, we
understand what's going on.
448
00:29:22 --> 00:29:24
By separating out
the frequencies.
449
00:29:24 --> 00:29:28
We maybe smooth it, we maybe
convolve it, whatever we do.
450
00:29:28 --> 00:29:29
Compress it.
451
00:29:29 --> 00:29:34
And then we've got to go
back to physical space.
452
00:29:34 --> 00:29:36
So we have to go both ways.
453
00:29:36 --> 00:29:41
And the question is
what about F inverse?
454
00:29:41 --> 00:29:52
And let me just say, what
goes into F inverse?
455
00:29:52 --> 00:29:54
The point is that the
number that goes into F
456
00:29:54 --> 00:29:59
inverse is just like w.
457
00:29:59 --> 00:30:04
But it's the complex conjugate,
and I'll call it w bar.
458
00:30:04 --> 00:30:05
And that's e^(-2pi*i/N).
459
00:30:05 --> 00:30:10
460
00:30:10 --> 00:30:13
So the thing that's going to
go into the inverse matrix
461
00:30:13 --> 00:30:16
is the powers of w bar.
462
00:30:16 --> 00:30:21
The complex conjugate. jk.
463
00:30:21 --> 00:30:28
Let me just say right away,
that just as there were 2pi's
464
00:30:28 --> 00:30:34
in the Fourier series world,
that got in our way,
465
00:30:34 --> 00:30:38
and physicists can't
stand those 2pi's.
466
00:30:39 --> 00:30:41
They try to get rid of
them but course they
467
00:30:41 --> 00:30:43
can't, they're there.
468
00:30:43 --> 00:30:47
So they put it up
into the exponent.
469
00:30:47 --> 00:30:50
Well, I guess, I'm blaming
physicists, I've done it too.
470
00:30:50 --> 00:30:52
I've put the 2pi up there.
471
00:30:52 --> 00:30:53
For the Fourier series.
472
00:30:53 --> 00:30:57
Anyway whatever, 2pi's
are all in your hair
473
00:30:57 --> 00:30:59
in the Fourier series.
474
00:30:59 --> 00:31:03
And in this, for the discrete
one, the corresponding
475
00:31:03 --> 00:31:05
thing is an N.
476
00:31:05 --> 00:31:08
There's a factor N that
I have to deal with.
477
00:31:08 --> 00:31:13
Because look, here's
what I'm saying.
478
00:31:13 --> 00:31:16
Let me multiply F by F inverse.
479
00:31:16 --> 00:31:17
OK.
480
00:31:17 --> 00:31:23
Say, 1, 1, 1, 1, 1 i, i^2,
i^3, this is my F, right?
481
00:31:23 --> 00:31:24
Whoops.
482
00:31:24 --> 00:31:31
Don't ever let me do that. i^2,
fourth, sixth. i^3, i^6, i^9.
483
00:31:33 --> 00:31:39
Now, my claim is that F inverse
is going to involve the complex
484
00:31:39 --> 00:31:41
conjugate, w bar. i bar.
485
00:31:41 --> 00:31:44
What is the complex
conjugate of i?
486
00:31:44 --> 00:31:48
You see that we really have to
use complex numbers here, but
487
00:31:48 --> 00:31:50
they're on the unit circle.
488
00:31:50 --> 00:31:51
You couldn't ask
for a better line.
489
00:31:51 --> 00:31:54
What's the complex conjugate
that's going to go
490
00:31:54 --> 00:31:56
into F inverse?
491
00:31:56 --> 00:32:03
If I take i and I take its
conjugate, what does that mean?
492
00:32:03 --> 00:32:07
So the conjugate of that is
just, flip it across the
493
00:32:07 --> 00:32:10
real axis to the other one.
494
00:32:10 --> 00:32:12
You see, it's great?
495
00:32:12 --> 00:32:16
The complex conjugate
of i is minus i.
496
00:32:16 --> 00:32:19
Here's w^8, here's w bar^8.
497
00:32:19 --> 00:32:22
498
00:32:22 --> 00:32:24
It's just across the axis.
499
00:32:24 --> 00:32:29
So it still has size one.
500
00:32:29 --> 00:32:31
And, of course, it's e to the?
501
00:32:31 --> 00:32:34
This angle, which is
just minus this angle.
502
00:32:34 --> 00:32:37
So that's why we
get this minus.
503
00:32:37 --> 00:32:42
Because when I flip it across,
i changes to minus i, angles
504
00:32:42 --> 00:32:44
change it to minus angle.
505
00:32:44 --> 00:32:46
And finally I was going to
do this multiplication
506
00:32:46 --> 00:32:47
just to see.
507
00:32:47 --> 00:32:50
So now I'm claiming that
this is 1, 1, 1 1, 1
508
00:32:50 --> 00:32:55
1, 1, -i, -i^2, -i^3.
509
00:32:55 --> 00:33:08
So on; -i^2, -i^3, fourth,
sixth, sixth, and ninth.
510
00:33:08 --> 00:33:12
And now, if you do that
multiplication, you will
511
00:33:12 --> 00:33:13
get the identity matrix.
512
00:33:13 --> 00:33:15
That's the great thing.
513
00:33:15 --> 00:33:16
You get the identity.
514
00:33:16 --> 00:33:20
1, 1, 1, 1.
515
00:33:20 --> 00:33:23
Except for a factor N.
516
00:33:23 --> 00:33:27
And do you see that coming?
517
00:33:27 --> 00:33:31
When I multiply this by
this, what do I get?
518
00:33:31 --> 00:33:34
Four.
519
00:33:34 --> 00:33:39
If they were N by N, I'd have
N ones against N ones, so I'd
520
00:33:39 --> 00:33:45
have to expect an N there.
521
00:33:45 --> 00:33:51
So F times F inverse, so, in
other words if I want to get
522
00:33:51 --> 00:33:57
the identity, I'd better
divide this by N.
523
00:33:57 --> 00:34:01
So now I have F F
inverse equal I.
524
00:34:01 --> 00:34:07
So F inverse is just like F,
except complex conjugate
525
00:34:07 --> 00:34:09
and divide by N.
526
00:34:09 --> 00:34:12
So if I want this formula
to be really correct,
527
00:34:12 --> 00:34:16
I have to divide by N.
528
00:34:16 --> 00:34:17
OK?
529
00:34:17 --> 00:34:21
So and that's what
w and w bar is.
530
00:34:21 --> 00:34:25
And another letter that
often gets used in
531
00:34:25 --> 00:34:27
this topic is omega.
532
00:34:27 --> 00:34:34
And I figure that's a good way
to separate out the complex
533
00:34:34 --> 00:34:39
conjugates, let's call this guy
w and call this guy omega.
534
00:34:39 --> 00:34:42
So this is w and here's omega.
535
00:34:42 --> 00:34:44
I don't know if that helps.
536
00:34:44 --> 00:34:48
But you do have to keep
straight whether you're
537
00:34:48 --> 00:34:50
talking about this
number or that number.
538
00:34:50 --> 00:34:55
And this number could be called
w, because omega wouldn't
539
00:34:55 --> 00:34:56
be easy to type.
540
00:34:56 --> 00:35:01
So you have to pay attention,
which one you're doing
541
00:35:01 --> 00:35:05
and when, somewhere you
have to divide by N.
542
00:35:05 --> 00:35:08
But otherwise it's fantastic.
543
00:35:08 --> 00:35:17
It's just beautiful.
544
00:35:17 --> 00:35:21
Can we do that multiplication,
just to see that
545
00:35:21 --> 00:35:23
it really works?
546
00:35:23 --> 00:35:25
Well, the ones worked fine.
547
00:35:25 --> 00:35:32
What about the ones times,
or what about the ones
548
00:35:32 --> 00:35:36
times that column.
549
00:35:36 --> 00:35:42
So I'm looking at the (1,2)
entry in this multiplication.
550
00:35:42 --> 00:35:45
Or rather, the (0,1),
entry I should say.
551
00:35:45 --> 00:35:48
This is the zeroth row, and
this is the row number one, and
552
00:35:48 --> 00:35:51
what do I get out of that?
553
00:35:51 --> 00:35:54
Zero, right.
554
00:35:54 --> 00:35:55
That was my point.
555
00:35:55 --> 00:36:00
That this is adding up,
oh well, going the
556
00:36:00 --> 00:36:01
other way around.
557
00:36:01 --> 00:36:04
Because it's the complex
conjugate, but it's adding up
558
00:36:04 --> 00:36:12
the four, same four numbers
and giving me that zero.
559
00:36:12 --> 00:36:15
And now how about the
ones on the diagonal?
560
00:36:15 --> 00:36:20
When I do this times this,
so this is the same,
561
00:36:20 --> 00:36:23
the row times itself.
562
00:36:23 --> 00:36:26
What am I seeing there?
563
00:36:26 --> 00:36:28
What are these terms?
564
00:36:28 --> 00:36:33
That's certainly one, what
is i times minus i? i times
565
00:36:33 --> 00:36:35
minus i is one again.
566
00:36:35 --> 00:36:38
A number's getting multiplied
by its conjugate.
567
00:36:38 --> 00:36:41
That gives you something real
and positive when you multiply
568
00:36:41 --> 00:36:44
a number by its conjugate.
569
00:36:44 --> 00:36:47
In fact, we get one every time
so we get four ones, we divide
570
00:36:47 --> 00:36:57
by four and we get that.
571
00:36:57 --> 00:37:03
Maybe you actually see here,
just so you remember, so F
572
00:37:03 --> 00:37:11
inverse is, I'm concluding
that F inverse, I'm looking
573
00:37:11 --> 00:37:13
now, what is F inverse?
574
00:37:13 --> 00:37:20
It's F conjugate divided by
this nuisance number N.
575
00:37:20 --> 00:37:23
I could put square root of N
with F, and then square root of
576
00:37:23 --> 00:37:27
N would show up with F inverse,
just the way I could have done,
577
00:37:27 --> 00:37:30
square the 2pi with one, and
square the 2pi with the other.
578
00:37:30 --> 00:37:37
But let's leave it like so.
579
00:37:37 --> 00:37:40
So that's not proved in a way.
580
00:37:40 --> 00:37:48
Just, checked for a couple
of cases with N=4,
581
00:37:48 --> 00:37:51
but it always works.
582
00:37:51 --> 00:37:52
That's the beauty.
583
00:37:52 --> 00:37:58
In other words, the two
directions, the y to c and the
584
00:37:58 --> 00:38:04
c to y, are both going to
be speeded up by the FFT.
585
00:38:04 --> 00:38:07
The FFT, whatever it did,
whatever it could do with the
586
00:38:07 --> 00:38:12
w and powers of w, it can
do the same with w bar.
587
00:38:12 --> 00:38:14
And powers of w bar.
588
00:38:14 --> 00:38:18
If the FFT is a fast way to to
multiply by that, it'll also
589
00:38:18 --> 00:38:24
give me a fast way to multiply,
to do the other transform.
590
00:38:24 --> 00:38:29
So the FFT is what makes every
- this would all be important
591
00:38:29 --> 00:38:32
if the FFT didn't exist.
592
00:38:32 --> 00:38:34
And of course actually,
Fourier didn't notice it.
593
00:38:34 --> 00:38:42
Gauss again had the idea, but
he had so many ideas it wasn't
594
00:38:42 --> 00:38:46
very clear in his notes.
595
00:38:46 --> 00:38:48
The FFT idea.
596
00:38:48 --> 00:38:56
But then Cooley and Tukey,
guys at Bell Labs in the
597
00:38:56 --> 00:38:59
1950s discovered it.
598
00:38:59 --> 00:39:04
In a small paper, I don't
think they had any idea
599
00:39:04 --> 00:39:06
how important it would be.
600
00:39:06 --> 00:39:10
But they published their paper
and people saw what was
601
00:39:10 --> 00:39:13
happening, and realized that
they should, that that gave the
602
00:39:13 --> 00:39:16
right way to do the transforms.
603
00:39:16 --> 00:39:24
OK, I was going to say here, do
you notice that this one, when
604
00:39:24 --> 00:39:30
I take a vector, that vectors
and its complex conjugate,
605
00:39:30 --> 00:39:33
what am I getting?
606
00:39:33 --> 00:39:35
I'm getting the length squared.
607
00:39:35 --> 00:39:37
I'm getting the right
inner product.
608
00:39:37 --> 00:39:41
This is the right inner product
when things are complex.
609
00:39:41 --> 00:39:42
Something times its conjugate.
610
00:39:42 --> 00:39:47
I'm just reminding you that the
inner product of two vectors,
611
00:39:47 --> 00:39:54
uv, u dot v or u comma v,
should now be the sum
612
00:39:54 --> 00:39:57
- it used to be the
sum of u_i*v_i.
613
00:39:59 --> 00:40:02
But now we're in the complex
case, so we should take
614
00:40:02 --> 00:40:04
the conjugate one of them.
615
00:40:04 --> 00:40:08
And here I guess that's the
v, and this is the u, and
616
00:40:08 --> 00:40:15
so the point is that we're
taking the right, yeah.
617
00:40:15 --> 00:40:22
We have, oh yeah. let me
just say it this way.
618
00:40:22 --> 00:40:26
These columns are orthogonal.
619
00:40:26 --> 00:40:29
Those four columns, the columns
of the Fourier matrix,
620
00:40:29 --> 00:40:31
are orthogonal.
621
00:40:31 --> 00:40:36
In other words, we have
an orthogonal basis.
622
00:40:36 --> 00:40:38
And it's those four columns.
623
00:40:38 --> 00:40:43
I almost missed saying that.
624
00:40:43 --> 00:40:47
But you remember from last
week, the whole point about
625
00:40:47 --> 00:40:51
sines and cosines and
exponentials in the function
626
00:40:51 --> 00:40:53
case, was orthogonal.
627
00:40:53 --> 00:40:55
That's what made everything
work, and it makes
628
00:40:55 --> 00:40:58
everything work here.
629
00:40:58 --> 00:41:01
Here we have vectors,
not functions.
630
00:41:01 --> 00:41:04
Orthogonal just means
dot products are zero.
631
00:41:04 --> 00:41:09
But they're complex, so that in
the dot product, one factor or
632
00:41:09 --> 00:41:11
the other has to
get conjugated.
633
00:41:11 --> 00:41:15
Anyway, that's an
orthogonal matrix.
634
00:41:15 --> 00:41:20
Apart from this N,
this capital N thing.
635
00:41:20 --> 00:41:24
So if I normalize it, if I take
out one over square root of
636
00:41:24 --> 00:41:28
N, now the length of
every column is one.
637
00:41:28 --> 00:41:31
So now I have
orthonormal columns.
638
00:41:31 --> 00:41:34
One squared, one squared, one
squared, one squared is four,
639
00:41:34 --> 00:41:38
divided by four is one.
640
00:41:38 --> 00:41:45
And every column has length is
a vector in four-dimensional
641
00:41:45 --> 00:41:47
complex space.
642
00:41:47 --> 00:41:49
Of length one.
643
00:41:49 --> 00:41:51
And orthogonal.
644
00:41:51 --> 00:41:52
So that means orthonormal.
645
00:41:52 --> 00:41:55
It's a fantastic matrix.
646
00:41:55 --> 00:41:59
That's the best basis we'll
ever have for complex
647
00:41:59 --> 00:42:00
four-dimensional space.
648
00:42:00 --> 00:42:06
That basis.
649
00:42:06 --> 00:42:09
Well, actually when I say
the best basis we'll ever
650
00:42:09 --> 00:42:13
have, I realize I wrote
a book about wavelets.
651
00:42:13 --> 00:42:16
And the idea of wavelets
is, there's another basis.
652
00:42:16 --> 00:42:18
A wavelet basis.
653
00:42:18 --> 00:42:21
And I hope to tell you
about that at the very
654
00:42:21 --> 00:42:24
end of the semester.
655
00:42:24 --> 00:42:26
So wavelets would be another
basis, would you like to
656
00:42:26 --> 00:42:29
know the wavelet basis?
657
00:42:29 --> 00:42:32
Have we got?
658
00:42:32 --> 00:42:37
Just in case the end of the
semester doesn't come,
659
00:42:37 --> 00:42:40
can I write down here
the wavelet basis?
660
00:42:40 --> 00:42:41
OK.
661
00:42:41 --> 00:42:45
And you can decide you
maybe like it better.
662
00:42:45 --> 00:42:49
So the four by four
wavelet basis, alright.
663
00:42:49 --> 00:42:53
So this will be the
wavelet matrix.
664
00:42:53 --> 00:42:56
Well, this was too
good to lose, right?
665
00:42:56 --> 00:42:59
That's a terrific vector.
666
00:42:59 --> 00:43:02
All ones, constant vector.
667
00:43:02 --> 00:43:06
It's the thing that gives us
the average, this c_0 is
668
00:43:06 --> 00:43:13
going to again be the
average of the y's.
669
00:43:13 --> 00:43:17
How do I get that average?
670
00:43:17 --> 00:43:21
Somehow I'm doing, the c_0, the
constant term is again I'm
671
00:43:21 --> 00:43:24
going to take one of everything
and I'll divide by four and
672
00:43:24 --> 00:43:25
I'll have the average.
673
00:43:25 --> 00:43:27
So I like that vector.
674
00:43:27 --> 00:43:30
That's the DC vector,
if I'm talking about
675
00:43:30 --> 00:43:31
direct current vector.
676
00:43:31 --> 00:43:36
If I'm talking about networks.
677
00:43:36 --> 00:43:39
In an image, it would be the
whole, you could call it
678
00:43:39 --> 00:43:41
the blackboard vector.
679
00:43:41 --> 00:43:46
The whole image of a empty
blackboard would be constant.
680
00:43:46 --> 00:43:49
OK, so that's too good to lose.
681
00:43:49 --> 00:43:52
Now, the obvious thing
orthogonal to that would
682
00:43:52 --> 00:43:56
be .
683
00:43:56 --> 00:43:57
Right?
684
00:43:57 --> 00:44:00
Now I've got two
orthogonal columns.
685
00:44:00 --> 00:44:03
And probably that's pretty
close to something
686
00:44:03 --> 00:44:05
I have over here.
687
00:44:05 --> 00:44:08
Now, here's the point
about wavelets.
688
00:44:08 --> 00:44:10
You'll see it with
the next guy.
689
00:44:10 --> 00:44:14
The next one, you saw that
those two are orthogonal.
690
00:44:14 --> 00:44:17
And real, and simple,
nice numbers.
691
00:44:17 --> 00:44:24
Now, the next one will
be .
692
00:44:24 --> 00:44:28
Do you see that
that's orthogonal?
693
00:44:28 --> 00:44:30
Obviously, because its
dot product with that is
694
00:44:30 --> 00:44:33
zero and that is zero.
695
00:44:33 --> 00:44:35
And you want to tell me the
last column of the wavelet, of
696
00:44:35 --> 00:44:37
the simple wavelet matrix?
697
00:44:37 --> 00:44:47
This would be the Haar, named
after Haar, the Haar wavelets.
698
00:44:47 --> 00:44:50
The whole point about
wavelets was a new basis.
699
00:44:50 --> 00:44:53
Somebody realized, well
actually, Haar lived a long
700
00:44:53 --> 00:44:57
time ago, but they got improved
and improved, but here's
701
00:44:57 --> 00:44:57
the simplest one.
702
00:44:57 --> 00:45:00
What's the good fourth column?
703
00:45:00 --> 00:45:05
Fourth orthogonal vector here?
704
00:45:05 --> 00:45:06
What shall I start with?
705
00:45:06 --> 00:45:17
Zeroes, and then I'll
put one minus one.
706
00:45:17 --> 00:45:23
So the beauty of wavelets is,
I've got this averaging vector.
707
00:45:23 --> 00:45:28
And then I've got this
difference, sort of on a big
708
00:45:28 --> 00:45:31
scale, and then I've got
differences on a little scale
709
00:45:31 --> 00:45:36
with zero, differences here
with zeroes so as I keep going
710
00:45:36 --> 00:45:42
I could do eight by eight, 16
by 16, it would - I get
711
00:45:42 --> 00:45:44
the averaging guy.
712
00:45:44 --> 00:45:47
And then I get differencing, I
get large scale differences and
713
00:45:47 --> 00:45:49
then I get finer differences.
714
00:45:49 --> 00:45:53
So coarse differences,
finer differences, finer
715
00:45:53 --> 00:45:54
finer differences.
716
00:45:54 --> 00:46:03
And that's a really good way
to - if your function, if
717
00:46:03 --> 00:46:06
your signal has jumps.
718
00:46:06 --> 00:46:11
We saw that Fourier
wasn't perfect, right?
719
00:46:11 --> 00:46:13
And a lot of signals
have jumps.
720
00:46:13 --> 00:46:13
Right?
721
00:46:13 --> 00:46:18
Your shocks, whatever you're
trying to represent.
722
00:46:18 --> 00:46:22
This is a better basis
to represent a jump, a
723
00:46:22 --> 00:46:25
discontinuity, than Fourier.
724
00:46:25 --> 00:46:29
Because Fourier, at a
discontinuity, does its best
725
00:46:29 --> 00:46:32
but it's not a local basis.
726
00:46:32 --> 00:46:37
It can't, like, especially
focus on that jump point.
727
00:46:37 --> 00:46:42
And and the result is the
coefficients decay very slowly
728
00:46:42 --> 00:46:44
and it's hard to compute with.
729
00:46:44 --> 00:46:49
Whereas the wavelet is great
for signals that have jumps.
730
00:46:49 --> 00:46:52
If something happens
suddenly at a certain time.
731
00:46:52 --> 00:46:56
Anyway, that would be another,
and of course I haven't made
732
00:46:56 --> 00:47:00
these unit vectors, I'd have to
divide that by the square root
733
00:47:00 --> 00:47:03
of four and divide these guys
by the square root of two
734
00:47:03 --> 00:47:05
just to normalize them.
735
00:47:05 --> 00:47:08
But normalizing
isn't important.
736
00:47:08 --> 00:47:11
Orthogonal is what
is important.
737
00:47:11 --> 00:47:20
OK, so that's competition,
you could say.
738
00:47:20 --> 00:47:22
That's the competition.
739
00:47:22 --> 00:47:29
OK, back to main guy, so this
is the famous formula that
740
00:47:29 --> 00:47:34
gives the y's from the c's, and
what's the other formula that
741
00:47:34 --> 00:47:40
gives the c's from the y's?
742
00:47:40 --> 00:47:44
So now this is the transform
in the other direction.
743
00:47:44 --> 00:47:49
So it's going to be a
sum from j=0 to N-1.
744
00:47:49 --> 00:47:53
745
00:47:53 --> 00:47:57
And what's going
to go in there?
746
00:47:57 --> 00:48:01
Have you got the idea
of the two transforms?
747
00:48:01 --> 00:48:04
In one direction this is e^i.
748
00:48:05 --> 00:48:07
You see why we don't
want to write it.
749
00:48:07 --> 00:48:10
That is, that number
is e^(2pi*i*j*k/N).
750
00:48:10 --> 00:48:17
751
00:48:17 --> 00:48:23
That's what that number is,
in the Fourier matrix.
752
00:48:23 --> 00:48:28
So you see using w
makes it much cleaner.
753
00:48:28 --> 00:48:34
OK, what goes there? w bar.
754
00:48:34 --> 00:48:37
The complex conjugate, e to
the minus all that stuff. w
755
00:48:37 --> 00:48:41
bar to the same power, jk.
756
00:48:42 --> 00:48:45
And then I have to remember
to divide by N somewhere.
757
00:48:45 --> 00:48:48
I can put it here, or
I can put it there.
758
00:48:48 --> 00:48:55
But the all ones vector has
length squared equal to N
759
00:48:55 --> 00:48:59
and I've got to divide
by an N somewhere.
760
00:48:59 --> 00:49:01
OK, those are the two
formulas that many people
761
00:49:01 --> 00:49:02
would just remember.
762
00:49:02 --> 00:49:06
For them that's the discrete
Fourier transform and the
763
00:49:06 --> 00:49:08
inverse, and maybe they
would put the N over there.
764
00:49:08 --> 00:49:11
Probably that would
be more common.
765
00:49:11 --> 00:49:16
I got started with the Fourier
matrix with this side, and
766
00:49:16 --> 00:49:24
then the DFT matrix comes
here, and I needed the N.
767
00:49:24 --> 00:49:26
So those are the formulas.
768
00:49:26 --> 00:49:29
I don't know how you'd
prefer to remember these.
769
00:49:29 --> 00:49:31
Somehow you have to commit
something to memory here,
770
00:49:31 --> 00:49:33
certainly what w is.
771
00:49:33 --> 00:49:37
And the fact that
it's w and w bar?
772
00:49:37 --> 00:49:40
And the fact that the
sums start at zero.
773
00:49:40 --> 00:49:43
And the fact that you have just
these simple expressions for
774
00:49:43 --> 00:49:46
the entries in the matrix.
775
00:49:46 --> 00:49:48
And this, then would
be c=F inverse*y.
776
00:49:48 --> 00:49:53
777
00:49:53 --> 00:49:59
That's what I've written
down there in detail.
778
00:49:59 --> 00:50:05
OK, so that's the discrete
Fourier transform, and I
779
00:50:05 --> 00:50:06
think we're out of time.
780
00:50:06 --> 00:50:09
I've got more to say in
a little bit about the
781
00:50:09 --> 00:50:12
FFT, how that works.