1
00:00:08,750 --> 00:00:09,310
Okay.
2
00:00:09,310 --> 00:00:13,070
This is a lecture where
complex numbers come in.
3
00:00:13,070 --> 00:00:17,160
It's a -- complex numbers
have slipped into this course
4
00:00:17,160 --> 00:00:23,480
because even a real matrix
can have complex eigenvalues.
5
00:00:23,480 --> 00:00:27,420
So we met complex numbers
there as the eigenvalues
6
00:00:27,420 --> 00:00:30,030
and complex eigenvectors.
7
00:00:30,030 --> 00:00:33,560
And we -- or --
8
00:00:33,560 --> 00:00:36,410
this is probably the last --
9
00:00:36,410 --> 00:00:38,870
we have a lot of other things
to do about eigenvalues
10
00:00:38,870 --> 00:00:40,580
and eigenvectors.
11
00:00:40,580 --> 00:00:43,610
And that will be mostly real.
12
00:00:43,610 --> 00:00:46,090
But at one point
somewhere, we have
13
00:00:46,090 --> 00:00:51,140
to see what you do when the
numbers become complex numbers.
14
00:00:51,140 --> 00:00:54,500
What happens when the
vectors are complex,
15
00:00:54,500 --> 00:00:57,360
when the matrixes are
complex, when the --
16
00:00:57,360 --> 00:01:01,640
what's the inner product of two,
the dot product of two complex
17
00:01:01,640 --> 00:01:03,730
vectors --
18
00:01:03,730 --> 00:01:06,050
we just have to make
the change, just see --
19
00:01:06,050 --> 00:01:09,340
what is the change when
numbers become complex?
20
00:01:09,340 --> 00:01:14,360
Then, can I tell you about
the most important example
21
00:01:14,360 --> 00:01:17,270
of complex matrixes?
22
00:01:17,270 --> 00:01:20,460
It comes in the Fourier matrix.
23
00:01:20,460 --> 00:01:23,480
So the Fourier matrix,
which I'll describe,
24
00:01:23,480 --> 00:01:25,400
is a complex matrix.
25
00:01:25,400 --> 00:01:28,850
It's certainly the most
important complex matrix.
26
00:01:28,850 --> 00:01:34,770
It's the matrix that we
need in Fourier transform.
27
00:01:34,770 --> 00:01:37,930
And the -- really, the special
thing that I want to tell you
28
00:01:37,930 --> 00:01:42,340
about is what's called the
fast Fourier transform,
29
00:01:42,340 --> 00:01:48,000
and everybody refers to it
as the FFT and it's in all
30
00:01:48,000 --> 00:01:50,500
computer and it's used --
31
00:01:50,500 --> 00:01:54,010
it's being used as we
speak in a thousand places,
32
00:01:54,010 --> 00:01:59,510
because it has, like,
transformed whole industries
33
00:01:59,510 --> 00:02:03,090
to be able to do the
Fourier transform fast,
34
00:02:03,090 --> 00:02:09,270
which means multiplying --
how do I multiply fast by that
35
00:02:09,270 --> 00:02:11,730
matrix -- by that n by n matrix?
36
00:02:11,730 --> 00:02:16,380
Normally, multiplications
by an n by n matrix --
37
00:02:16,380 --> 00:02:21,440
would normally be n
squared multiplications,
38
00:02:21,440 --> 00:02:24,460
because I've got n squared
entries and none of them is
39
00:02:24,460 --> 00:02:24,960
zero.
40
00:02:24,960 --> 00:02:27,780
This is a full matrix.
41
00:02:27,780 --> 00:02:30,700
And it's a matrix with
orthogonal columns.
42
00:02:30,700 --> 00:02:34,140
I mean, it's just,
like, the best matrix.
43
00:02:34,140 --> 00:02:37,700
And this fast Fourier
transform idea
44
00:02:37,700 --> 00:02:43,680
reduces this n squared,
which was slowing up
45
00:02:43,680 --> 00:02:47,815
the calculation of Fourier
transforms down to n log(n).
46
00:02:51,000 --> 00:02:54,480
n log(n), log to the
base two, actually.
47
00:02:54,480 --> 00:02:57,940
And it's this --
when that hit --
48
00:02:57,940 --> 00:03:01,930
when that possibility hit,
it made a big difference.
49
00:03:01,930 --> 00:03:08,430
Everybody realized
gradually what, --
50
00:03:08,430 --> 00:03:11,800
that this simple idea -- you'll
see it's just a simple matrix
51
00:03:11,800 --> 00:03:15,060
factorization -- but
it changed everything.
52
00:03:15,060 --> 00:03:15,630
Okay.
53
00:03:15,630 --> 00:03:19,250
So I want to talk about
complex vectors and matrixes
54
00:03:19,250 --> 00:03:22,610
in general, recap a
little bit from last time,
55
00:03:22,610 --> 00:03:27,450
and the Fourier
matrix in particular.
56
00:03:27,450 --> 00:03:28,940
Okay.
57
00:03:28,940 --> 00:03:31,360
So what's the deal?
58
00:03:31,360 --> 00:03:32,210
All right.
59
00:03:32,210 --> 00:03:36,500
The main point is,
what about length?
60
00:03:36,500 --> 00:03:40,560
I'm given a vector,
I have a vector x.
61
00:03:40,560 --> 00:03:44,310
Or let me call it z as a
reminder that it's complex,
62
00:03:44,310 --> 00:03:45,260
for the moment.
63
00:03:45,260 --> 00:03:48,950
But I can -- later I'll
call the components x.
64
00:03:48,950 --> 00:03:50,740
They'll be complex numbers.
65
00:03:50,740 --> 00:03:53,090
But it's a vector --
66
00:03:53,090 --> 00:03:56,500
z1, z2 down to zn.
67
00:03:56,500 --> 00:04:01,590
So the only novelty is
it's not in R^n anymore.
68
00:04:01,590 --> 00:04:04,370
It's in complex n
dimensional space.
69
00:04:04,370 --> 00:04:07,510
Each of those numbers
is a complex number.
70
00:04:07,510 --> 00:04:14,750
So this z,z1 is in C^n, n
dimensional complex space
71
00:04:14,750 --> 00:04:17,700
instead of R^n.
72
00:04:17,700 --> 00:04:20,750
So just a different letter
there, but now the point
73
00:04:20,750 --> 00:04:23,200
about its length is what?
74
00:04:23,200 --> 00:04:29,190
The point about its length is
that z transpose z is no good.
75
00:04:34,800 --> 00:04:38,530
z transpose z -- if I just
put down z transpose here,
76
00:04:38,530 --> 00:04:42,090
it would be z1, z2, to zn.
77
00:04:45,580 --> 00:04:47,990
Doing that multiplication
doesn't give me
78
00:04:47,990 --> 00:04:50,090
the right thing.
79
00:04:50,090 --> 00:04:51,300
W-Why not?
80
00:04:51,300 --> 00:04:57,390
Because the length squared
should be positive.
81
00:04:57,390 --> 00:05:01,930
And if I multiply -- suppose
this is, like, 1 and i.
82
00:05:01,930 --> 00:05:06,560
What's the length of the
vector with components 1 and i?
83
00:05:06,560 --> 00:05:08,550
What if I do this,
so n is just two.
84
00:05:08,550 --> 00:05:11,380
I'm in C^2, two
dimensional space,
85
00:05:11,380 --> 00:05:16,510
complex space with the vector
whose components are 1 and i.
86
00:05:16,510 --> 00:05:17,310
All right.
87
00:05:17,310 --> 00:05:23,320
So if I took one times one
and i times i and added,
88
00:05:23,320 --> 00:05:27,150
z transpose z would be zero.
89
00:05:27,150 --> 00:05:29,310
But I don't -- that
vector is not --
90
00:05:29,310 --> 00:05:32,030
doesn't have length zero -- the
vector with the components 1
91
00:05:32,030 --> 00:05:33,590
and i --
92
00:05:33,590 --> 00:05:41,710
this multiplication -- what I
really want is z1 conjugate z1.
93
00:05:41,710 --> 00:05:46,510
You remember that z1
conjugate z1 is --
94
00:05:46,510 --> 00:05:50,370
so you see that first step
will be z1 conjugate z1,
95
00:05:50,370 --> 00:05:56,820
which is the magnitude of z1
squared, which is what I want.
96
00:05:56,820 --> 00:06:00,550
That's, like, three
squared or five squared.
97
00:06:00,550 --> 00:06:11,170
Now, if it's -- if z1 is i, then
I multiplied by minus i gives
98
00:06:11,170 --> 00:06:19,290
one plus one, so the component
of length -- the component i,
99
00:06:19,290 --> 00:06:21,731
its modulus squared is plus
100
00:06:21,731 --> 00:06:22,230
one.
101
00:06:22,230 --> 00:06:22,990
That's great.
102
00:06:22,990 --> 00:06:25,560
So what I want to do
then is do that --
103
00:06:25,560 --> 00:06:30,680
I want z1 bar z1, z2
bar z2, zn bar zn.
104
00:06:30,680 --> 00:06:33,210
And remember that -- you
remember this complex
105
00:06:33,210 --> 00:06:33,900
conjugate.
106
00:06:33,900 --> 00:06:38,270
So -- so there's the point.
107
00:06:38,270 --> 00:06:41,030
Now I can erase
the no good and put
108
00:06:41,030 --> 00:06:48,800
is good, because that now gives
the answer zero for the zero
109
00:06:48,800 --> 00:06:52,590
vector, of course, but it
gives a positive length
110
00:06:52,590 --> 00:06:55,780
squared for any other vector.
111
00:06:55,780 --> 00:06:58,890
So it's a -- it's the
right definition of length,
112
00:06:58,890 --> 00:07:04,370
and essentially the message is
that we're always going to be
113
00:07:04,370 --> 00:07:06,750
taking --
114
00:07:06,750 --> 00:07:09,400
when we transpose, we also
take complex conjugate.
115
00:07:09,400 --> 00:07:13,670
So let's -- let's find
the length of one --
116
00:07:13,670 --> 00:07:19,890
so the vector one i, that's
z, that's that vector z.
117
00:07:19,890 --> 00:07:23,490
Now I take the conjugate of one
is one, the conjugate of i is
118
00:07:23,490 --> 00:07:24,650
minus i.
119
00:07:24,650 --> 00:07:29,340
I take this vector,
I get one plus one --
120
00:07:29,340 --> 00:07:29,840
I get
121
00:07:29,840 --> 00:07:30,339
two.
122
00:07:33,050 --> 00:07:36,810
So that's a vector and that's a
vector of length -- square root
123
00:07:36,810 --> 00:07:37,740
of two.
124
00:07:37,740 --> 00:07:40,560
Square root of two is the
length and not the zero
125
00:07:40,560 --> 00:07:43,830
that we would have got
from one minus i squared.
126
00:07:43,830 --> 00:07:46,210
Okay.
127
00:07:46,210 --> 00:07:49,250
So the message really is
whenever we transpose,
128
00:07:49,250 --> 00:07:51,660
we also take conjugates.
129
00:07:51,660 --> 00:07:57,300
So here's a symbol --
one symbol to do both.
130
00:07:57,300 --> 00:08:03,890
So that symbol H, it stands
for a guy named Hermite,
131
00:08:03,890 --> 00:08:06,710
who didn't actually
pronounce the H,
132
00:08:06,710 --> 00:08:09,630
but let's pronounce it --
133
00:08:09,630 --> 00:08:14,040
so I would call
that z Hermitian z.
134
00:08:14,040 --> 00:08:20,940
I'll -- let me write that word,
Herm- so his name was Hermite,
135
00:08:20,940 --> 00:08:27,240
and then we make it into
an adjective, Hermitian.
136
00:08:27,240 --> 00:08:32,000
So z Hermitian z. z H z.
137
00:08:32,000 --> 00:08:34,360
Okay.
138
00:08:34,360 --> 00:08:37,510
So, that's the --
139
00:08:37,510 --> 00:08:40,730
that's the, length squared.
140
00:08:40,730 --> 00:08:43,409
Now what's the inner product?
141
00:08:43,409 --> 00:08:45,000
Well, it should match.
142
00:08:45,000 --> 00:08:49,850
The inner product
of two vectors --
143
00:08:49,850 --> 00:08:56,060
so inner product is no longer --
144
00:08:56,060 --> 00:08:59,230
used to be y transpose x.
145
00:08:59,230 --> 00:09:01,680
That's for real vectors.
146
00:09:01,680 --> 00:09:04,970
For complex vectors,
whenever we transpose,
147
00:09:04,970 --> 00:09:07,220
we also take the conjugate.
148
00:09:07,220 --> 00:09:12,110
So it's y Hermitian x.
149
00:09:12,110 --> 00:09:14,160
Of course it's not
real anymore, usually.
150
00:09:14,160 --> 00:09:19,610
That -- the inner product will
usually be complex number.
151
00:09:19,610 --> 00:09:24,100
But if y and x are the
same, if they're the same z,
152
00:09:24,100 --> 00:09:29,240
then we have z -- z H z,
we have the length squared,
153
00:09:29,240 --> 00:09:31,950
and that's what we want, the
inner product of a vector with
154
00:09:31,950 --> 00:09:35,490
itself should be
its length squared.
155
00:09:35,490 --> 00:09:40,230
So this is, like, forced on us
because this is forced on us.
156
00:09:40,230 --> 00:09:43,750
So -- so this z -- this --
157
00:09:43,750 --> 00:09:45,660
everybody's picking
up what this equals.
158
00:09:45,660 --> 00:09:52,170
This is z1 squared
plus zn squared.
159
00:09:52,170 --> 00:09:53,820
That's the length squared.
160
00:09:53,820 --> 00:09:57,080
And that's the inner product
that we have to go with.
161
00:09:57,080 --> 00:10:00,560
So it could be a
complex number now.
162
00:10:00,560 --> 00:10:02,790
One more change.
163
00:10:02,790 --> 00:10:04,750
Well, two more changes.
164
00:10:04,750 --> 00:10:08,780
We've got to change the
idea of a symmetric matrix.
165
00:10:08,780 --> 00:10:11,870
So I'll just recap on
symmetric matrixes.
166
00:10:11,870 --> 00:10:21,610
Symmetric means A transpose
equals A, but not --
167
00:10:21,610 --> 00:10:30,430
no good if A is complex.
168
00:10:30,430 --> 00:10:33,890
So what do we instead --
169
00:10:37,490 --> 00:10:40,360
that applies perfectly
to real matrixes.
170
00:10:40,360 --> 00:10:42,760
But now if my
matrixes were complex,
171
00:10:42,760 --> 00:10:51,910
I want to take the transpose
and the conjugate to equal A.
172
00:10:51,910 --> 00:10:55,430
So there's -- that's the --
the right complex version
173
00:10:55,430 --> 00:10:56,780
of symmetry.
174
00:10:56,780 --> 00:10:59,470
The com- the symmetry
now means when
175
00:10:59,470 --> 00:11:03,180
I transpose it, flip across the
diagonal and take conjugates.
176
00:11:03,180 --> 00:11:07,250
So, for example -- here
would be an example.
177
00:11:07,250 --> 00:11:09,480
On the diagonal, it
had better be real,
178
00:11:09,480 --> 00:11:15,760
because when I flip it, the
diagonal is still there and it
179
00:11:15,760 --> 00:11:18,410
has to -- and then when I take
the complex conjugate it has
180
00:11:18,410 --> 00:11:21,240
to be still there, so it
better be a real number,
181
00:11:21,240 --> 00:11:24,580
let me say two and five.
182
00:11:24,580 --> 00:11:27,790
What about entries
off the diagonal?
183
00:11:27,790 --> 00:11:32,200
If this entry is,
say, three plus i,
184
00:11:32,200 --> 00:11:36,630
then this entry had better be --
185
00:11:36,630 --> 00:11:40,070
because I want whatever
this -- when I transpose,
186
00:11:40,070 --> 00:11:42,790
it'll show up here
and i conjugate.
187
00:11:42,790 --> 00:11:47,000
So I need three minus I there.
188
00:11:47,000 --> 00:11:51,400
So there's a matrix with --
189
00:11:51,400 --> 00:11:55,640
that corresponds to
symmetry, but it's complex.
190
00:11:55,640 --> 00:12:01,550
And those matrixes are
called Hermitian matrixes.
191
00:12:01,550 --> 00:12:03,170
Hermitian matrixes.
192
00:12:03,170 --> 00:12:06,210
A H equals A.
193
00:12:06,210 --> 00:12:08,790
Fine.
194
00:12:08,790 --> 00:12:13,140
Okay, that's -- and those
matrixes have real eigenvalues
195
00:12:13,140 --> 00:12:15,950
and they have
perpendicular eigenvectors.
196
00:12:15,950 --> 00:12:18,760
What does perpendicular mean?
197
00:12:18,760 --> 00:12:21,760
Perpendicular means
the inner product --
198
00:12:21,760 --> 00:12:23,585
so let's go on to perpendicular.
199
00:12:28,180 --> 00:12:31,410
Well, when I had perpendicular
vectors, for example,
200
00:12:31,410 --> 00:12:37,820
they were like q1, q2 up to qn.
201
00:12:37,820 --> 00:12:41,630
That's my -- q is my letter
that I use for perpendicular.
202
00:12:41,630 --> 00:12:43,190
Actually, I usually --
203
00:12:43,190 --> 00:12:45,840
I also mean unit length.
204
00:12:45,840 --> 00:12:49,010
So those are perpendicular
unit vectors.
205
00:12:49,010 --> 00:12:51,030
But now what does
-- so it's a --
206
00:12:51,030 --> 00:12:54,730
orthonormal basis, I'll
still use those words,
207
00:12:54,730 --> 00:12:58,560
but how do I compute
perpendicular?
208
00:12:58,560 --> 00:13:00,770
How do I check perpendicular?
209
00:13:00,770 --> 00:13:07,370
This means that the inner
product of qi with qj --
210
00:13:07,370 --> 00:13:11,510
but now I not only transpose,
I must conjugate, right,
211
00:13:11,510 --> 00:13:20,210
to get zero if i is not
j and one if i is j.
212
00:13:20,210 --> 00:13:25,950
So it's a unit vector, meaning
unit length, orthogonal --
213
00:13:25,950 --> 00:13:28,930
all the angles are right
angles, but these are angles
214
00:13:28,930 --> 00:13:33,020
in complex n dimensional space.
215
00:13:33,020 --> 00:13:37,155
So it's q1, q on-
qi bar transpose.
216
00:13:39,790 --> 00:13:44,310
Or, for short, qi H qj.
217
00:13:44,310 --> 00:13:46,650
So it will still be
true -- so let me --
218
00:13:46,650 --> 00:13:51,230
again I'll create a
matrix out of those guys.
219
00:13:51,230 --> 00:13:56,090
The matrix will have these
q-s in its columns, q2 to qn.
220
00:14:00,320 --> 00:14:03,400
And I want to turn that
into matrix language,
221
00:14:03,400 --> 00:14:05,130
just like before.
222
00:14:05,130 --> 00:14:06,690
What does that mean?
223
00:14:06,690 --> 00:14:08,800
That means I want all
these inner products,
224
00:14:08,800 --> 00:14:13,920
so I take these columns of
Q, multiply by their rows --
225
00:14:13,920 --> 00:14:17,250
so it was -- it used to be Q
-- it used to be Q transpose Q
226
00:14:17,250 --> 00:14:18,160
equals I, right?
227
00:14:20,840 --> 00:14:22,535
This was an orthogonal matrix.
228
00:14:27,100 --> 00:14:30,490
But what's changed?
229
00:14:30,490 --> 00:14:33,290
These are now complex vectors.
230
00:14:33,290 --> 00:14:37,730
Their inner products are --
involve conjugating the first
231
00:14:37,730 --> 00:14:39,920
factor.
232
00:14:39,920 --> 00:14:44,240
So it's not -- it's the
conjugate of Q transpose.
233
00:14:44,240 --> 00:14:47,050
It's Q bar transpose Q.
234
00:14:47,050 --> 00:14:48,590
Q H.
235
00:14:48,590 --> 00:14:55,240
So can I call this -- let me
call it Q H Q, which is I.
236
00:14:55,240 --> 00:14:57,260
So that's our new --
237
00:14:57,260 --> 00:14:59,550
you -- you see I'm just
translating, and the --
238
00:14:59,550 --> 00:15:07,300
the book h- on one page gives a
little dictionary of the right
239
00:15:07,300 --> 00:15:12,900
words in the real case, R^n,
and the corresponding words
240
00:15:12,900 --> 00:15:17,120
in the complex case for
the vector space C^n.
241
00:15:17,120 --> 00:15:19,600
Of course, C^n is
a vector space,
242
00:15:19,600 --> 00:15:22,650
the numbers we multiply
are now complex numbers --
243
00:15:22,650 --> 00:15:27,770
we're just moving into
complex n dimensional space.
244
00:15:27,770 --> 00:15:29,700
Okay.
245
00:15:29,700 --> 00:15:34,450
Now -- actually, I have to say
we changed the word symmet-
246
00:15:34,450 --> 00:15:38,500
symmetric to Hermitian
for those matrixes.
247
00:15:38,500 --> 00:15:43,300
People also change this word
orthogonal into another word
248
00:15:43,300 --> 00:15:52,730
that happens to be unitary,
as a word that applies --
249
00:15:52,730 --> 00:15:56,540
that signals that we might be
dealing with a complex matrix
250
00:15:56,540 --> 00:15:57,320
here.
251
00:15:57,320 --> 00:15:59,940
So what's a unitary matrix?
252
00:15:59,940 --> 00:16:02,960
It's a -- it's just like
an orthogonal matrix.
253
00:16:02,960 --> 00:16:10,800
It's a square, n by n matrix
with orthonormal columns,
254
00:16:10,800 --> 00:16:13,970
perpendicular columns,
unit vectors --
255
00:16:13,970 --> 00:16:20,010
unit vectors computed by --
and perpendicularity computed
256
00:16:20,010 --> 00:16:23,820
by remembering that
there's a conjugate as well
257
00:16:23,820 --> 00:16:25,250
as a transpose.
258
00:16:25,250 --> 00:16:26,270
Okay.
259
00:16:26,270 --> 00:16:28,520
So those are the words.
260
00:16:28,520 --> 00:16:32,880
Now I'm ready to get into the
substance of the lecture which
261
00:16:32,880 --> 00:16:38,290
is the most famous complex
matrix, which happens
262
00:16:38,290 --> 00:16:41,500
to be one of these guys.
263
00:16:41,500 --> 00:16:50,240
It has orthogonal columns,
and it's named after Fourier
264
00:16:50,240 --> 00:16:52,800
because it comes into
the Fourier transform,
265
00:16:52,800 --> 00:16:55,380
so it's the matrix
that's all around us.
266
00:16:55,380 --> 00:16:55,880
Okay.
267
00:16:58,450 --> 00:17:05,660
Let me tell you what it is
first of all in the n by n case.
268
00:17:05,660 --> 00:17:09,750
Then often I'll let n be four
because four is a good size
269
00:17:09,750 --> 00:17:11,150
to work with.
270
00:17:11,150 --> 00:17:13,369
But here's the n by
n Fourier matrix.
271
00:17:17,790 --> 00:17:20,319
Its first column is
the vector of ones.
272
00:17:24,310 --> 00:17:26,319
It's n by n, of course.
273
00:17:26,319 --> 00:17:30,420
Its second column is
the powers, the --
274
00:17:30,420 --> 00:17:35,370
actually, better if I move
from the math department
275
00:17:35,370 --> 00:17:39,910
to EE for this one half
hour and then, please,
276
00:17:39,910 --> 00:17:42,100
let me move back again.
277
00:17:42,100 --> 00:17:43,870
Okay.
278
00:17:43,870 --> 00:17:46,170
What's the difference between
those two departments?
279
00:17:46,170 --> 00:17:50,640
It's just math starts
counting with one
280
00:17:50,640 --> 00:17:56,180
and electrical engineers
start counting at zero.
281
00:17:56,180 --> 00:17:57,840
Actually, they're
probably right.
282
00:17:57,840 --> 00:18:00,720
So anyway, we'll give
them -- humor them.
283
00:18:00,720 --> 00:18:05,090
So this is really
the zeroes column.
284
00:18:05,090 --> 00:18:08,030
And the first column
up to the n-1, that's
285
00:18:08,030 --> 00:18:11,780
the one inconvenient spot
in electrical engineering.
286
00:18:11,780 --> 00:18:14,760
All these expressions
start at zero, no problem,
287
00:18:14,760 --> 00:18:16,720
but they end at n-1.
288
00:18:16,720 --> 00:18:24,000
Well, that's -- that's the
difficulty of Course 6.
289
00:18:24,000 --> 00:18:27,750
So what's -- they're the powers
of a number that I'm going
290
00:18:27,750 --> 00:18:28,660
to call W --
291
00:18:28,660 --> 00:18:34,840
W squared, W cubed, W to the
-- now what is the W here?
292
00:18:34,840 --> 00:18:36,060
What's the power?
293
00:18:36,060 --> 00:18:39,000
This was the zeroes power,
first power, second power,
294
00:18:39,000 --> 00:18:43,290
this will be n
minus first power.
295
00:18:43,290 --> 00:18:45,130
That's the column.
296
00:18:45,130 --> 00:18:46,500
What's the next column?
297
00:18:46,500 --> 00:18:51,920
It's the powers of W squared, W
to the fourth, W to the sixth,
298
00:18:51,920 --> 00:18:55,870
W to the two n-1.
299
00:18:55,870 --> 00:18:58,010
And then more columns
and more columns
300
00:18:58,010 --> 00:19:00,060
and more columns and
what's the last column?
301
00:19:02,790 --> 00:19:07,361
It's the powers of --
302
00:19:07,361 --> 00:19:07,860
let's see.
303
00:19:07,860 --> 00:19:10,860
We -- actually, if we
look around rows, w-
304
00:19:10,860 --> 00:19:12,520
this matrix is symmetric.
305
00:19:12,520 --> 00:19:17,220
It's symmetric in the old
not quite perfect way,
306
00:19:17,220 --> 00:19:21,100
not perfect because these
numbers are complex.
307
00:19:21,100 --> 00:19:25,370
And so it's -- that
first row is all ones.
308
00:19:25,370 --> 00:19:28,740
One W W squared up
to W to the n-1.
309
00:19:28,740 --> 00:19:34,500
That's the last column is
the powers of W to the n-1,
310
00:19:34,500 --> 00:19:38,841
so this guy matches that, and
finally we get W to something
311
00:19:38,841 --> 00:19:39,340
here.
312
00:19:43,060 --> 00:19:46,670
I guess we could actually figure
out what that something is.
313
00:19:46,670 --> 00:19:48,610
What are the entries
of this matrix?
314
00:19:48,610 --> 00:19:57,490
The i j entry of
this matrix are --
315
00:19:57,490 --> 00:20:00,420
I going to -- are you going to
allow me to let i go from zero
316
00:20:00,420 --> 00:20:01,890
to n minus one?
317
00:20:01,890 --> 00:20:07,180
So i and g go from zero to n-1.
318
00:20:07,180 --> 00:20:10,870
So the one -- the zero
zero entry is a one --
319
00:20:10,870 --> 00:20:15,670
it's just this same W guy
to the power i times j.
320
00:20:21,960 --> 00:20:22,460
Let's see.
321
00:20:22,460 --> 00:20:24,690
I'm jumping into formulas
here and I have to tell you
322
00:20:24,690 --> 00:20:27,903
what W is and then you know
everything about this matrix.
323
00:20:30,960 --> 00:20:34,520
So W is the -- well,
shall we finish here?
324
00:20:34,520 --> 00:20:37,890
What was this -- this is
the (n-1) (n-1) entry.
325
00:20:37,890 --> 00:20:41,380
This is W to the n-1 squared.
326
00:20:41,380 --> 00:20:46,140
Everything's looking like a
mess here, because we have --
327
00:20:46,140 --> 00:20:51,890
not too bad, because all
the entries are powers of W.
328
00:20:51,890 --> 00:20:53,830
There -- none of them are zero.
329
00:20:53,830 --> 00:20:56,600
This is a full matrix.
330
00:20:56,600 --> 00:20:59,840
But W is a very special number.
331
00:20:59,840 --> 00:21:08,720
W is the special number
whose n-th power is one.
332
00:21:08,720 --> 00:21:12,780
In fact -- well, actually,
there are n numbers like that.
333
00:21:12,780 --> 00:21:14,620
One of them is one, of course.
334
00:21:14,620 --> 00:21:19,850
But the one we --
the W we want is --
335
00:21:19,850 --> 00:21:23,470
the angle is two pi over n.
336
00:21:28,060 --> 00:21:29,130
Is that what I mean?
337
00:21:29,130 --> 00:21:32,740
n over two pi.
338
00:21:32,740 --> 00:21:34,310
No, two pi over n.
339
00:21:34,310 --> 00:21:42,540
W is E to the I and the
angle is two pi over n.
340
00:21:42,540 --> 00:21:43,550
Right.
341
00:21:43,550 --> 00:21:47,590
Where is this W in
the complex plane?
342
00:21:47,590 --> 00:21:49,870
It's -- it's on the
unit circle, right?
343
00:21:49,870 --> 00:21:56,160
This is -- it's the cosine
of two pi over n plus I times
344
00:21:56,160 --> 00:21:59,620
the sine of two pi over n.
345
00:21:59,620 --> 00:22:02,840
But actually, forget this.
346
00:22:02,840 --> 00:22:09,050
It's never good to work with
the real and imaginary parts,
347
00:22:09,050 --> 00:22:14,130
the rectangular coordinates,
when we're taking powers.
348
00:22:14,130 --> 00:22:17,770
To take that to the tenth power,
we can't see what we're doing.
349
00:22:17,770 --> 00:22:19,890
To take this form
to the tenth power,
350
00:22:19,890 --> 00:22:22,350
we see immediately
what we're doing.
351
00:22:22,350 --> 00:22:26,480
It would be e to
the i 20 pi over n.
352
00:22:26,480 --> 00:22:30,570
So when our matrix is full of
powers -- so it's this formula,
353
00:22:30,570 --> 00:22:32,690
and where is this on
the complex plain?
354
00:22:32,690 --> 00:22:36,830
Here are the real numbers,
here's the imaginary axis,
355
00:22:36,830 --> 00:22:40,410
here's the unit
circle of radius one,
356
00:22:40,410 --> 00:22:44,240
and this number is
on the unit circle
357
00:22:44,240 --> 00:22:48,860
at this angle, which is one
n-th of the full way round.
358
00:22:48,860 --> 00:22:52,700
So if I drew, for
example, n equals six,
359
00:22:52,700 --> 00:22:56,490
this would be e to the
two pi, two pi over six,
360
00:22:56,490 --> 00:22:59,160
it would be one sixth of the way
around, it would be 60 degrees.
361
00:23:03,250 --> 00:23:06,090
And where is W squared?
362
00:23:06,090 --> 00:23:13,120
So I -- my W is e to the two
pi I over six in this case,
363
00:23:13,120 --> 00:23:16,760
in this six by -- for the
six by six Fourier transform,
364
00:23:16,760 --> 00:23:22,800
it's totally constructed out
of this number and its powers.
365
00:23:22,800 --> 00:23:24,480
So what are its powers?
366
00:23:24,480 --> 00:23:27,470
Well, its powers are on
the unit circle, right?
367
00:23:27,470 --> 00:23:32,680
Because when I square a
number, a complex number,
368
00:23:32,680 --> 00:23:37,740
I square its absolute value,
which gives me one again.
369
00:23:37,740 --> 00:23:40,500
All the powers have --
are on the unit circle.
370
00:23:40,500 --> 00:23:43,250
And the -- the angle
gets doubled to a hundred
371
00:23:43,250 --> 00:23:46,180
and twenty, so there's W
squared, there's W cubed,
372
00:23:46,180 --> 00:23:50,660
there's W to the fourth, there's
W to the fifth and there is W
373
00:23:50,660 --> 00:23:57,070
to the sixth, as we hoped, W to
the sixth coming back to one.
374
00:23:57,070 --> 00:24:01,080
So those are the six --
375
00:24:01,080 --> 00:24:02,620
can I say this on TV?
376
00:24:02,620 --> 00:24:07,130
The six sixth roots
of one, and it's
377
00:24:07,130 --> 00:24:14,560
this one, the primitive one we
say, the first one, which is W.
378
00:24:14,560 --> 00:24:17,840
Okay, so what -- let
me change -- let me --
379
00:24:17,840 --> 00:24:21,440
I said I would probably
switch to n equal four.
380
00:24:21,440 --> 00:24:24,130
What's W for that?
381
00:24:24,130 --> 00:24:25,970
It's the fourth root of one.
382
00:24:25,970 --> 00:24:28,590
W to the fourth will be one.
383
00:24:28,590 --> 00:24:33,410
W will be e to the two
pi i over four now.
384
00:24:38,820 --> 00:24:41,040
What's that?
385
00:24:41,040 --> 00:24:43,270
This is e to the i pi over two.
386
00:24:43,270 --> 00:24:46,180
This is a quarter of the
way around the unit circle,
387
00:24:46,180 --> 00:24:53,930
and that's exactly i, a
quarter of the way around.
388
00:24:53,930 --> 00:24:58,660
And sure enough, the
powers are i, i squared,
389
00:24:58,660 --> 00:25:01,670
which is minus
one, i cubed, which
390
00:25:01,670 --> 00:25:08,780
is minus i and finally i to
the fourth which is one, right.
391
00:25:08,780 --> 00:25:12,760
So there's W, W squared, W
cubed, W to the fourth --
392
00:25:12,760 --> 00:25:15,050
I'm really ready to
write down this Fourier
393
00:25:15,050 --> 00:25:18,780
matrix for the
four by four case,
394
00:25:18,780 --> 00:25:21,790
just so we see that clearly.
395
00:25:21,790 --> 00:25:22,890
Let me do it here.
396
00:25:22,890 --> 00:25:34,760
F4 is -- all right, one one one
one one one one W -- it's I.
397
00:25:34,760 --> 00:25:35,590
I squared.
398
00:25:35,590 --> 00:25:36,770
That's minus one.
399
00:25:36,770 --> 00:25:38,520
i cubed is minus i.
400
00:25:41,350 --> 00:25:43,490
I'll -- I could write
i squared and i cubed.
401
00:25:43,490 --> 00:25:46,140
Why don't I, just so we
see the pattern for sure.
402
00:25:46,140 --> 00:25:54,040
i squared, i cubed, i squared,
i cubed, i fourth, i sixth --
403
00:25:54,040 --> 00:25:58,690
i fourth, i sixth and i ninth.
404
00:25:58,690 --> 00:26:01,970
You see the exponents
fall in this nice --
405
00:26:01,970 --> 00:26:05,760
the exponent is the row number
times the column number,
406
00:26:05,760 --> 00:26:08,770
always starting at zero.
407
00:26:08,770 --> 00:26:09,560
Okay.
408
00:26:09,560 --> 00:26:12,910
And now I can put in those
numbers if you like --
409
00:26:12,910 --> 00:26:20,520
one one one one, one i minus
one minus i, one minus one,
410
00:26:20,520 --> 00:26:28,710
one minus one and one
minus i minus one i.
411
00:26:28,710 --> 00:26:29,410
No.
412
00:26:29,410 --> 00:26:31,111
Yes.
413
00:26:31,111 --> 00:26:31,610
Right.
414
00:26:38,260 --> 00:26:40,965
What's -- why do I think
that matrix is so remarkable?
415
00:26:44,520 --> 00:26:49,030
It's the four by four matrix
that comes into the four point
416
00:26:49,030 --> 00:26:52,020
Fourier transform.
417
00:26:52,020 --> 00:26:55,700
When we want to find the Fourier
transform, the four point
418
00:26:55,700 --> 00:27:01,380
Fourier transform of a
vector with four components,
419
00:27:01,380 --> 00:27:05,540
we want to multiply
by this F4 or we
420
00:27:05,540 --> 00:27:08,920
want to multiply by F4 inverse.
421
00:27:08,920 --> 00:27:10,840
One way we're taking
the transform,
422
00:27:10,840 --> 00:27:13,820
one way we're taking
the inverse transform.
423
00:27:13,820 --> 00:27:17,950
Actually, they're so close that
it's easy to confuse the two.
424
00:27:17,950 --> 00:27:21,990
The inverse of this matrix
will be a nice matrix also.
425
00:27:25,250 --> 00:27:29,190
So -- and that's, of
course, what makes it --
426
00:27:29,190 --> 00:27:32,370
that -- I guess
Fourier knew that.
427
00:27:32,370 --> 00:27:34,200
He knew the inverse
of this matrix.
428
00:27:36,970 --> 00:27:39,890
A- as you'll see, it just comes
from the fact that the columns
429
00:27:39,890 --> 00:27:41,640
are orthogonal --
430
00:27:41,640 --> 00:27:44,050
from the fact that the
columns are orthogonal,
431
00:27:44,050 --> 00:27:51,220
we will quickly figure
out what is the inverse.
432
00:27:51,220 --> 00:27:53,910
What Fourier didn't
know -- didn't notice --
433
00:27:53,910 --> 00:27:58,180
I think Gauss noticed it but
didn't make a point of it
434
00:27:58,180 --> 00:28:00,020
and then it turned
out to be really
435
00:28:00,020 --> 00:28:04,770
important was the fact that this
matrix is so special that you
436
00:28:04,770 --> 00:28:10,340
can break it up into nice pieces
with lots of zeroes, factors
437
00:28:10,340 --> 00:28:13,360
that have lots of
zeroes and multiply
438
00:28:13,360 --> 00:28:16,650
by it or by its inverse
very, very fast.
439
00:28:16,650 --> 00:28:17,150
Okay.
440
00:28:19,740 --> 00:28:22,850
But how did it get into
this lecture first?
441
00:28:22,850 --> 00:28:26,340
Because the columns
are orthogonal.
442
00:28:26,340 --> 00:28:29,130
Can I just check that the
columns of this matrix
443
00:28:29,130 --> 00:28:30,265
are orthogonal?
444
00:28:33,480 --> 00:28:38,360
So the inner product of
that column with that column
445
00:28:38,360 --> 00:28:41,260
is zero.
446
00:28:41,260 --> 00:28:48,250
The inner product of column
one with column three is zero.
447
00:28:48,250 --> 00:28:55,850
How about the inner
product of two and four?
448
00:28:55,850 --> 00:29:01,280
Can I take the inner product
of column two with column four?
449
00:29:01,280 --> 00:29:04,860
Or even the inner product of two
with three, let's -- let's see,
450
00:29:04,860 --> 00:29:06,820
does that --
451
00:29:06,820 --> 00:29:07,751
let me do two and
452
00:29:07,751 --> 00:29:08,250
four.
453
00:29:15,330 --> 00:29:17,260
Okay.
454
00:29:17,260 --> 00:29:19,110
What -- oh, I see, yes, hmm.
455
00:29:19,110 --> 00:29:19,610
Hmm.
456
00:29:19,610 --> 00:29:30,060
Let's see, I believe that those
two columns are orthogonal.
457
00:29:30,060 --> 00:29:33,520
So let me take their inner
product and hope to get zero.
458
00:29:33,520 --> 00:29:36,310
Okay, now if you hadn't
listened to the first half
459
00:29:36,310 --> 00:29:39,890
of this lecture, when you
took the inner product of that
460
00:29:39,890 --> 00:29:42,760
with that, you would have
multiplied one by one,
461
00:29:42,760 --> 00:29:48,090
i by minus i, and that
would have given you one,
462
00:29:48,090 --> 00:29:51,370
minus one by minus one
would have given you
463
00:29:51,370 --> 00:29:55,940
another one minus I by I would
have been minus I squared,
464
00:29:55,940 --> 00:29:56,810
that's another one.
465
00:30:00,870 --> 00:30:04,240
So do I conclude that the
inner product of columns --
466
00:30:04,240 --> 00:30:05,920
I said columns two
and four, that's
467
00:30:05,920 --> 00:30:08,550
because I forgot those
are columns one and three.
468
00:30:12,110 --> 00:30:13,670
I'm interested in
their inner product
469
00:30:13,670 --> 00:30:17,570
and I'm hoping it's zero, but
it doesn't look like zero.
470
00:30:17,570 --> 00:30:19,090
Nevertheless, it is zero.
471
00:30:19,090 --> 00:30:20,580
Those columns are perpendicular.
472
00:30:20,580 --> 00:30:21,500
Why?
473
00:30:21,500 --> 00:30:25,390
Because the inner product --
474
00:30:25,390 --> 00:30:26,520
we conjugate.
475
00:30:26,520 --> 00:30:29,150
Do you remember that the --
one of the vectors in the inner
476
00:30:29,150 --> 00:30:31,510
product has to get conjugated.
477
00:30:31,510 --> 00:30:34,750
So when I conjugated, it
changes that i to a minus i,
478
00:30:34,750 --> 00:30:37,900
changes this to a plus
i, changes those --
479
00:30:37,900 --> 00:30:44,060
that second sine and that
fourth sine and I do get zero.
480
00:30:44,060 --> 00:30:46,490
So those columns are orthogonal.
481
00:30:46,490 --> 00:30:49,130
So columns are orthogonal.
482
00:30:51,780 --> 00:30:55,630
They're not quite orthonormal.
483
00:30:55,630 --> 00:30:58,020
But I could fix that easily.
484
00:30:58,020 --> 00:31:02,710
They -- all those
columns have length two.
485
00:31:02,710 --> 00:31:07,450
Length squared is four, like
this -- the four I had there --
486
00:31:07,450 --> 00:31:10,260
this length squared, one plus
-- one squared one squared one
487
00:31:10,260 --> 00:31:13,910
squared one squared is
four, square root is two --
488
00:31:13,910 --> 00:31:17,420
so if I really wanted them --
suppose I really wanted to fix
489
00:31:17,420 --> 00:31:21,300
life perfectly, I
could divide by two,
490
00:31:21,300 --> 00:31:26,653
and now I have columns that
are actually orthonormal.
491
00:31:33,420 --> 00:31:35,560
So what?
492
00:31:35,560 --> 00:31:38,040
So I can invert
right away, right?
493
00:31:38,040 --> 00:31:41,970
O- orthonormal columns means --
now I'm keeping this one half
494
00:31:41,970 --> 00:31:43,820
in here for the moment --
495
00:31:43,820 --> 00:31:48,060
c- means F4 Hermitian,
can I use that,
496
00:31:48,060 --> 00:31:51,620
conjugate transpose times F4 is
497
00:31:51,620 --> 00:31:52,120
i.
498
00:31:56,230 --> 00:31:59,150
So I see what the inverse is.
499
00:31:59,150 --> 00:32:01,840
The inverse of F4 is
-- it's just like an --
500
00:32:01,840 --> 00:32:03,180
an orthogonal matrix.
501
00:32:03,180 --> 00:32:06,230
The inverse is the transpose
-- here the inverse is
502
00:32:06,230 --> 00:32:08,990
the conjugate transpose.
503
00:32:08,990 --> 00:32:10,250
So, fine.
504
00:32:10,250 --> 00:32:15,000
That -- that tells me that
anything good that I learn
505
00:32:15,000 --> 00:32:19,610
about F4 I'll know the same --
506
00:32:19,610 --> 00:32:21,710
I'll know a similar
fact about its inverse,
507
00:32:21,710 --> 00:32:25,620
because its inverse is just
its conjugate transpose.
508
00:32:25,620 --> 00:32:28,080
Okay, now -- so what's good?
509
00:32:28,080 --> 00:32:30,430
Well, first, the
columns are orthogonal.
510
00:32:30,430 --> 00:32:33,870
That's a key fact, then.
511
00:32:33,870 --> 00:32:36,670
That's the thing that
makes the inverse easy.
512
00:32:36,670 --> 00:32:40,740
But what property is it that
leads to the fast Fourier
513
00:32:40,740 --> 00:32:41,460
transform?
514
00:32:41,460 --> 00:32:44,000
So now I'm going to talk,
in these last minutes,
515
00:32:44,000 --> 00:32:48,010
about the fast
Fourier transform.
516
00:32:48,010 --> 00:32:49,290
What -- here's the idea.
517
00:32:51,980 --> 00:32:56,030
F6, our six by six
matrix, will c-
518
00:32:56,030 --> 00:33:02,250
there's a neat connection
to F3, half as big.
519
00:33:02,250 --> 00:33:05,200
There's a connection
of F8 to F4.
520
00:33:05,200 --> 00:33:09,690
There's a connection
of F(64) to F(32).
521
00:33:09,690 --> 00:33:12,070
Shall I write down what
that connection is?
522
00:33:12,070 --> 00:33:15,190
What's the connection
of F(64) to F(32)?
523
00:33:15,190 --> 00:33:22,950
So F(64) is a 64 by
64 matrix whose W
524
00:33:22,950 --> 00:33:26,500
is the 64th root of one.
525
00:33:26,500 --> 00:33:31,870
So it's one 64th of
the way round in F(64).
526
00:33:31,870 --> 00:33:35,959
And it -- do- and F(32)
is a 32 by 32 matrix.
527
00:33:35,959 --> 00:33:37,375
Remember, they're
different sizes.
528
00:33:40,450 --> 00:33:45,680
And the W in that 32 by 32
matrix is the 32nd root of one,
529
00:33:45,680 --> 00:33:50,220
which is twice as far -- that
-- you sh- see that key point --
530
00:33:50,220 --> 00:33:55,850
that's the -- that's how 32
and 64 are connected in the Ws.
531
00:33:55,850 --> 00:33:59,380
The W for 64 is one
64th of the way --
532
00:33:59,380 --> 00:34:05,610
so all I'm saying is
that if I square the W --
533
00:34:05,610 --> 00:34:09,219
W(64), that's what I'm using
for the one over -- the --
534
00:34:09,219 --> 00:34:15,070
W sixty f- this Wn is either
the i two pi over n --
535
00:34:15,070 --> 00:34:18,690
so W(64) is one 64th
of the way around it.
536
00:34:18,690 --> 00:34:24,139
When I square that,
what do I get but W(32)?
537
00:34:24,139 --> 00:34:25,969
Right?
538
00:34:25,969 --> 00:34:31,060
If I square this matrix,
I double the angle --
539
00:34:31,060 --> 00:34:36,300
if I square this number, I
double the angle, I get, the --
540
00:34:36,300 --> 00:34:40,120
the W(32).
541
00:34:40,120 --> 00:34:43,239
So somehow there's
a little hope here
542
00:34:43,239 --> 00:34:46,440
to connect F(64) with F(32).
543
00:34:46,440 --> 00:34:49,400
And here's the connection.
544
00:34:49,400 --> 00:34:49,980
Okay.
545
00:34:49,980 --> 00:34:53,530
Let me -- let me go back, --
546
00:34:53,530 --> 00:34:54,170
yes, let me --
547
00:34:54,170 --> 00:34:58,780
I'll do it here.
548
00:34:58,780 --> 00:35:00,520
Here's the connection.
549
00:35:00,520 --> 00:35:01,780
F(64).
550
00:35:01,780 --> 00:35:06,530
The 64 by 64 Fourier
matrix is connected
551
00:35:06,530 --> 00:35:10,070
to two copies of F(32).
552
00:35:10,070 --> 00:35:13,120
Let me leave a little
space for the connection.
553
00:35:13,120 --> 00:35:15,990
So this is 64 by 64.
554
00:35:15,990 --> 00:35:18,980
Here's a matrix
of that same size,
555
00:35:18,980 --> 00:35:25,640
because it's got two copies of
F(32) and two zero matrixes.
556
00:35:25,640 --> 00:35:31,500
Those zero matrixes are the key,
because when I multiply by this
557
00:35:31,500 --> 00:35:35,570
matrix, just as it is, regular
multiplication, I would take --
558
00:35:35,570 --> 00:35:37,160
need 64 --
559
00:35:37,160 --> 00:35:40,320
I would -- I would have 64
squared little multiplications
560
00:35:40,320 --> 00:35:41,470
to do.
561
00:35:41,470 --> 00:35:45,000
But this matrix is half zero.
562
00:35:45,000 --> 00:35:46,600
Well, of course, the
two aren't equal.
563
00:35:46,600 --> 00:35:52,010
I'm going to put an equals sign,
but there has to be some fix up
564
00:35:52,010 --> 00:35:53,570
factors --
565
00:35:53,570 --> 00:35:57,640
one there and one there --
566
00:35:57,640 --> 00:36:00,620
to make it true.
567
00:36:00,620 --> 00:36:05,260
The beauty is that these fix
up factors will be really --
568
00:36:05,260 --> 00:36:07,550
almost all zeroes.
569
00:36:07,550 --> 00:36:11,470
So that as soon as we
get this formula right,
570
00:36:11,470 --> 00:36:15,670
we've got a great idea for
how to get from the sixty-
571
00:36:15,670 --> 00:36:21,180
from the 64 squared calculations
-- so this original --
572
00:36:21,180 --> 00:36:25,110
originally we have 64 squared
calculations from there,
573
00:36:25,110 --> 00:36:29,570
but this one will give us
-- this is -- this will --
574
00:36:29,570 --> 00:36:34,660
we don't need that many -- we
only need two times 32 squared,
575
00:36:34,660 --> 00:36:38,120
because we've got that twice.
576
00:36:38,120 --> 00:36:42,440
And -- plus the fix-up.
577
00:36:42,440 --> 00:36:47,570
So I have to tell you what's
in this fix-up matrix.
578
00:36:47,570 --> 00:36:50,370
The one on the right is
actually a permutation matrix,
579
00:36:50,370 --> 00:36:55,820
a very simple odds and evens
permutation matrix, the --
580
00:36:55,820 --> 00:36:58,260
ones show up --
581
00:36:58,260 --> 00:37:00,820
I haven't put enough
ones, I really need a --
582
00:37:00,820 --> 00:37:05,106
32 of these guys at --
double space and then --
583
00:37:09,210 --> 00:37:12,400
you see it's -- it's
a permutation matrix.
584
00:37:12,400 --> 00:37:14,070
What it does --
585
00:37:14,070 --> 00:37:18,430
shall I call it P for
permutation matrix?
586
00:37:18,430 --> 00:37:23,920
So what that P does when
it multiplies a vector,
587
00:37:23,920 --> 00:37:28,770
it takes the odd -- the even
numbered components first
588
00:37:28,770 --> 00:37:30,580
and then the odds.
589
00:37:30,580 --> 00:37:33,640
You see this -- this one
skipping every time is going
590
00:37:33,640 --> 00:37:40,260
to pick out x0, x2, x4, x6 and
then below that will come --
591
00:37:40,260 --> 00:37:42,335
will pick out x1, x3, x5.
592
00:37:45,850 --> 00:37:49,650
And of course, that can be
hard wired in the computer
593
00:37:49,650 --> 00:37:52,350
to be instantaneous.
594
00:37:52,350 --> 00:37:56,570
So that says -- so
far, what have we said?
595
00:37:56,570 --> 00:38:00,960
We're saying that the 64 by
64 Fourier matrix is really
596
00:38:00,960 --> 00:38:04,540
separated into -- separate
your vector into the odd --
597
00:38:04,540 --> 00:38:07,430
into the even components
and the odd components,
598
00:38:07,430 --> 00:38:12,820
then do a 32 size Fourier
transform onto those
599
00:38:12,820 --> 00:38:17,420
separately, and then put
the pieces together again.
600
00:38:17,420 --> 00:38:21,890
So the pieces -- putting them
together turns them out to be I
601
00:38:21,890 --> 00:38:25,350
and a diagonal matrix
and I and a minus,
602
00:38:25,350 --> 00:38:26,615
that same diagonal matrix.
603
00:38:29,330 --> 00:38:33,540
So the fix-up cost is really
the cost of multiplying by D,
604
00:38:33,540 --> 00:38:38,120
this diagonal matrix, because
there's essentially no cost
605
00:38:38,120 --> 00:38:39,020
in --
606
00:38:39,020 --> 00:38:42,630
in the I part or in
the permutation part,
607
00:38:42,630 --> 00:38:50,720
so really it's -- the fix-up
cost is essentially because D
608
00:38:50,720 --> 00:38:52,950
is diagonal --
609
00:38:52,950 --> 00:38:54,160
is 32 multiplications.
610
00:38:58,930 --> 00:39:00,430
That's the -- there
you're seeing --
611
00:39:00,430 --> 00:39:02,888
of course we didn't check the
formula or we didn't even say
612
00:39:02,888 --> 00:39:05,030
what D is yet, but I will --
613
00:39:05,030 --> 00:39:08,900
this diagonal matrix
D is powers of W --
614
00:39:08,900 --> 00:39:13,345
one W W squared down
to W to the 31st.
615
00:39:19,720 --> 00:39:22,630
So you see that when I --
to do a multiplication by D,
616
00:39:22,630 --> 00:39:25,920
I need to do 32 multiplications.
617
00:39:25,920 --> 00:39:27,780
There they are.
618
00:39:27,780 --> 00:39:33,030
Then -- but the other, the more
serious work is to do the F(32)
619
00:39:33,030 --> 00:39:34,910
twice on the --
620
00:39:34,910 --> 00:39:37,300
separately on the
even numbered and odd
621
00:39:37,300 --> 00:39:41,600
numbered components,
so twice 32 squared.
622
00:39:41,600 --> 00:39:45,970
So 64 squared is gone now.
623
00:39:45,970 --> 00:39:47,990
And that's the new count.
624
00:39:50,580 --> 00:39:54,500
Okay, great, but what next?
625
00:39:54,500 --> 00:39:58,730
So that's -- I -- we
now have the key idea --
626
00:39:58,730 --> 00:40:01,110
we would have to
check the algebra,
627
00:40:01,110 --> 00:40:06,770
but it's just checking a lot of
sums that come out correctly.
628
00:40:06,770 --> 00:40:10,290
This is right -- the right
way to see the fast Fourier
629
00:40:10,290 --> 00:40:15,010
transform, or one
right way to see it.
630
00:40:15,010 --> 00:40:17,240
Then you've got to see
what's the next idea.
631
00:40:17,240 --> 00:40:21,390
The next idea is to
break the 32s down.
632
00:40:21,390 --> 00:40:23,450
Break those 32s down.
633
00:40:23,450 --> 00:40:27,430
So we have this factor,
and now we have the F(32),
634
00:40:27,430 --> 00:40:32,630
but that breaks into
some guy here --
635
00:40:32,630 --> 00:40:36,510
F thirty- F six- F(16) --
636
00:40:36,510 --> 00:40:37,610
F(16).
637
00:40:37,610 --> 00:40:43,320
Each -- each F(32) is breaking
into two copies of F(16),
638
00:40:43,320 --> 00:40:47,950
and then we have a
permutation and then the --
639
00:40:47,950 --> 00:40:52,210
so this is a -- like, this
was a 64 size permutation,
640
00:40:52,210 --> 00:40:55,410
this is a 32 size permutation --
641
00:40:55,410 --> 00:40:57,252
I guess I've got it twice.
642
00:40:57,252 --> 00:40:57,960
So it's -- I'm --
643
00:40:57,960 --> 00:41:01,120
I'm just using the same
idea recursively --
644
00:41:01,120 --> 00:41:05,270
recursion is the key word --
that on each of those F(32)s --
645
00:41:05,270 --> 00:41:09,465
so here's zero zero -- it's
just -- to get F(32) --
646
00:41:13,810 --> 00:41:16,980
this is the odd
even permutations --
647
00:41:16,980 --> 00:41:20,580
so you see, we're --
the combination of those
648
00:41:20,580 --> 00:41:24,210
permutations, what's it doing?
649
00:41:24,210 --> 00:41:28,070
This guy separates into odds
-- in -- into evens and odds,
650
00:41:28,070 --> 00:41:34,160
and then this guy separates
the evens into the ones --
651
00:41:34,160 --> 00:41:37,430
the numbers that are
mult- the even evens,
652
00:41:37,430 --> 00:41:41,080
which means zero
four eight sixteen --
653
00:41:41,080 --> 00:41:48,580
and even odds, which means
two, six, ten, fourteen --
654
00:41:48,580 --> 00:41:52,240
and then odd evens and odd odds.
655
00:41:52,240 --> 00:41:55,930
You see, together these
permutations then break it --
656
00:41:55,930 --> 00:42:00,630
break our vector down into
x, even even and three other
657
00:42:00,630 --> 00:42:02,090
pieces.
658
00:42:02,090 --> 00:42:05,070
Those are the four pieces
that separately get multiplied
659
00:42:05,070 --> 00:42:06,650
by F(16) --
660
00:42:06,650 --> 00:42:12,808
separately fixed up by these Is
and Ds and Is and minus Ds --
661
00:42:15,680 --> 00:42:19,340
so this count is now reduced.
662
00:42:19,340 --> 00:42:23,220
This count is now --
what's it reduced to?
663
00:42:23,220 --> 00:42:28,490
So that's going to be gone,
because 32 squared -- that's --
664
00:42:28,490 --> 00:42:30,680
that's the change
I'm making, right?
665
00:42:30,680 --> 00:42:35,130
The 32 squared -- w- so -- so
it's this that's now reduced.
666
00:42:35,130 --> 00:42:40,150
So I still have two times it,
but now what's 32 squared?
667
00:42:40,150 --> 00:42:47,710
It's gone in favor of two
sixteen squareds plus sixteen.
668
00:42:47,710 --> 00:42:54,055
That's -- and then the
original 32 to fix.
669
00:43:00,170 --> 00:43:01,910
Maybe you see what's happening.
670
00:43:01,910 --> 00:43:04,580
Even easier than this
formula is w- what's --
671
00:43:04,580 --> 00:43:09,410
when I do the recursion
more and more times,
672
00:43:09,410 --> 00:43:12,670
I get simpler and simpler
factors in the middle.
673
00:43:12,670 --> 00:43:15,330
Eventually I'll be down
to two point or one point
674
00:43:15,330 --> 00:43:18,320
Fourier transforms.
675
00:43:18,320 --> 00:43:21,170
But I get more and more
factors piling up on the right
676
00:43:21,170 --> 00:43:21,720
and left.
677
00:43:21,720 --> 00:43:23,650
On the right, I'm just
getting permutation
678
00:43:23,650 --> 00:43:24,450
matrixes.
679
00:43:24,450 --> 00:43:29,300
On the left, I'm getting
these guys, these Is and Ds,
680
00:43:29,300 --> 00:43:32,490
so that there was
a 32 there and --
681
00:43:32,490 --> 00:43:35,320
each one of these is costing 32.
682
00:43:35,320 --> 00:43:39,330
Each one of those is costing
And how many will there be?
683
00:43:42,450 --> 00:43:45,080
So you see the 32
for this original fix
684
00:43:45,080 --> 00:43:49,850
up, because D had 32
numbers, 32 for this next fix
685
00:43:49,850 --> 00:43:54,750
up, because D has
16 and 16 more.
686
00:43:54,750 --> 00:43:56,920
I keep going.
687
00:43:56,920 --> 00:44:00,320
So the count in the
middle goes down to zip,
688
00:44:00,320 --> 00:44:04,080
but these fix up counts
are all that I'm left with,
689
00:44:04,080 --> 00:44:10,820
and how many factors -- how many
fix-ups have I got -- log in --
690
00:44:10,820 --> 00:44:15,740
from 64, one step to 32, one
step to 16, one step to eight,
691
00:44:15,740 --> 00:44:17,660
four, two and one.
692
00:44:17,660 --> 00:44:19,030
Six steps.
693
00:44:19,030 --> 00:44:20,550
So I have six fix-up --
694
00:44:20,550 --> 00:44:23,540
six fix up factors.
695
00:44:23,540 --> 00:44:33,380
Finally I get to six times
the That's my final count.
696
00:44:33,380 --> 00:44:41,430
Instead of 64 squared, this is
log to the base two of 64 times
697
00:44:41,430 --> 00:44:42,960
64 --
698
00:44:42,960 --> 00:44:45,190
actually, half of 64.
699
00:44:45,190 --> 00:44:51,600
So actually, the final count is
n log to the base two of n --
700
00:44:51,600 --> 00:44:52,815
that's the 32 --
701
00:44:55,380 --> 00:44:57,430
a half.
702
00:44:57,430 --> 00:45:02,160
So can I put a box
around that wonderful,
703
00:45:02,160 --> 00:45:07,210
extremely important and
satisfying conclusion --
704
00:45:07,210 --> 00:45:12,400
that the fast Fourier transform
multiplies by an n by n matrix,
705
00:45:12,400 --> 00:45:16,110
but it does it not in n
squared steps, but in one
706
00:45:16,110 --> 00:45:19,100
half n log n steps.
707
00:45:19,100 --> 00:45:21,130
And if we just --
708
00:45:21,130 --> 00:45:25,070
complete by doing a
count, let's suppose --
709
00:45:25,070 --> 00:45:31,680
suppose -- a typical
case would be two to the
710
00:45:31,680 --> 00:45:32,180
tenth.
711
00:45:35,540 --> 00:45:39,760
Now n squared is
bigger than a million.
712
00:45:44,560 --> 00:45:47,910
So it's a thousand twenty four
times a thousand twenty four.
713
00:45:47,910 --> 00:45:51,390
But what is n --
what is one half --
714
00:45:51,390 --> 00:45:55,690
what is the new count,
done the right way?
715
00:45:55,690 --> 00:46:02,550
It's n -- the thousand
twenty four times one half,
716
00:46:02,550 --> 00:46:04,640
and what's the logarithm?
717
00:46:04,640 --> 00:46:06,180
It's ten.
718
00:46:06,180 --> 00:46:08,540
So times ten over two.
719
00:46:08,540 --> 00:46:12,880
So it's five times -- it's five
times a thousand twenty four,
720
00:46:12,880 --> 00:46:16,690
where this one was a thousand
twenty four times a thousand
721
00:46:16,690 --> 00:46:17,460
twenty four.
722
00:46:21,450 --> 00:46:25,470
We've reduced the calculation
by a factor of 200 just
723
00:46:25,470 --> 00:46:28,830
by factoring the
matrix properly.
724
00:46:28,830 --> 00:46:34,580
This was a thousand times n,
we're now down to five times n.
725
00:46:34,580 --> 00:46:38,570
So we can do 200
Fourier transforms,
726
00:46:38,570 --> 00:46:42,160
where before we
could do one, and
727
00:46:42,160 --> 00:46:45,720
in real scientific calculations
where Fourier transforms are
728
00:46:45,720 --> 00:46:48,200
happening all the
time, we're saving
729
00:46:48,200 --> 00:46:52,120
a factor of in one
of the major steps
730
00:46:52,120 --> 00:46:55,960
of modern scientific computing.
731
00:46:55,960 --> 00:46:58,280
So that's the idea of the
fast Fourier transform,
732
00:46:58,280 --> 00:47:00,020
and you see the
whole thing hinged
733
00:47:00,020 --> 00:47:07,360
on being a special matrix
with orthonormal columns.
734
00:47:07,360 --> 00:47:13,020
Okay, that's actually
it for complex numbers.
735
00:47:13,020 --> 00:47:16,960
I'm back next time really to --
736
00:47:16,960 --> 00:47:22,560
to real numbers,
eigenvalues and eigenvectors
737
00:47:22,560 --> 00:47:27,190
and the key idea of
positive definite matrixes
738
00:47:27,190 --> 00:47:28,790
is going to show up.
739
00:47:28,790 --> 00:47:30,660
What's a positive
definite matrix?
740
00:47:30,660 --> 00:47:32,550
And it's terrific
that this course
741
00:47:32,550 --> 00:47:35,520
is going to reach
positive definiteness,
742
00:47:35,520 --> 00:47:37,840
because those are
the matrixes that you
743
00:47:37,840 --> 00:47:40,320
see the most in applications.
744
00:47:40,320 --> 00:47:41,965
Okay, see you next time.
745
00:47:46,240 --> 00:47:47,790
Thanks.