WEBVTT
00:00:08.750 --> 00:00:09.310
Okay.
00:00:09.310 --> 00:00:13.070
This is a lecture where
complex numbers come in.
00:00:13.070 --> 00:00:17.160
It's a -- complex numbers
have slipped into this course
00:00:17.160 --> 00:00:23.480
because even a real matrix
can have complex eigenvalues.
00:00:23.480 --> 00:00:27.420
So we met complex numbers
there as the eigenvalues
00:00:27.420 --> 00:00:30.030
and complex eigenvectors.
00:00:30.030 --> 00:00:33.560
And we -- or --
00:00:33.560 --> 00:00:36.410
this is probably the last --
00:00:36.410 --> 00:00:38.870
we have a lot of other things
to do about eigenvalues
00:00:38.870 --> 00:00:40.580
and eigenvectors.
00:00:40.580 --> 00:00:43.610
And that will be mostly real.
00:00:43.610 --> 00:00:46.090
But at one point
somewhere, we have
00:00:46.090 --> 00:00:51.140
to see what you do when the
numbers become complex numbers.
00:00:51.140 --> 00:00:54.500
What happens when the
vectors are complex,
00:00:54.500 --> 00:00:57.360
when the matrixes are
complex, when the --
00:00:57.360 --> 00:01:01.640
what's the inner product of two,
the dot product of two complex
00:01:01.640 --> 00:01:03.730
vectors --
00:01:03.730 --> 00:01:06.050
we just have to make
the change, just see --
00:01:06.050 --> 00:01:09.340
what is the change when
numbers become complex?
00:01:09.340 --> 00:01:14.360
Then, can I tell you about
the most important example
00:01:14.360 --> 00:01:17.270
of complex matrixes?
00:01:17.270 --> 00:01:20.460
It comes in the Fourier matrix.
00:01:20.460 --> 00:01:23.480
So the Fourier matrix,
which I'll describe,
00:01:23.480 --> 00:01:25.400
is a complex matrix.
00:01:25.400 --> 00:01:28.850
It's certainly the most
important complex matrix.
00:01:28.850 --> 00:01:34.770
It's the matrix that we
need in Fourier transform.
00:01:34.770 --> 00:01:37.930
And the -- really, the special
thing that I want to tell you
00:01:37.930 --> 00:01:42.340
about is what's called the
fast Fourier transform,
00:01:42.340 --> 00:01:48.000
and everybody refers to it
as the FFT and it's in all
00:01:48.000 --> 00:01:50.500
computer and it's used --
00:01:50.500 --> 00:01:54.010
it's being used as we
speak in a thousand places,
00:01:54.010 --> 00:01:59.510
because it has, like,
transformed whole industries
00:01:59.510 --> 00:02:03.090
to be able to do the
Fourier transform fast,
00:02:03.090 --> 00:02:09.270
which means multiplying --
how do I multiply fast by that
00:02:09.270 --> 00:02:11.730
matrix -- by that n by n matrix?
00:02:11.730 --> 00:02:16.380
Normally, multiplications
by an n by n matrix --
00:02:16.380 --> 00:02:21.440
would normally be n
squared multiplications,
00:02:21.440 --> 00:02:24.460
because I've got n squared
entries and none of them is
00:02:24.460 --> 00:02:24.960
zero.
00:02:24.960 --> 00:02:27.780
This is a full matrix.
00:02:27.780 --> 00:02:30.700
And it's a matrix with
orthogonal columns.
00:02:30.700 --> 00:02:34.140
I mean, it's just,
like, the best matrix.
00:02:34.140 --> 00:02:37.700
And this fast Fourier
transform idea
00:02:37.700 --> 00:02:43.680
reduces this n squared,
which was slowing up
00:02:43.680 --> 00:02:47.815
the calculation of Fourier
transforms down to n log(n).
00:02:51.000 --> 00:02:54.480
n log(n), log to the
base two, actually.
00:02:54.480 --> 00:02:57.940
And it's this --
when that hit --
00:02:57.940 --> 00:03:01.930
when that possibility hit,
it made a big difference.
00:03:01.930 --> 00:03:08.430
Everybody realized
gradually what, --
00:03:08.430 --> 00:03:11.800
that this simple idea -- you'll
see it's just a simple matrix
00:03:11.800 --> 00:03:15.060
factorization -- but
it changed everything.
00:03:15.060 --> 00:03:15.630
Okay.
00:03:15.630 --> 00:03:19.250
So I want to talk about
complex vectors and matrixes
00:03:19.250 --> 00:03:22.610
in general, recap a
little bit from last time,
00:03:22.610 --> 00:03:27.450
and the Fourier
matrix in particular.
00:03:27.450 --> 00:03:28.940
Okay.
00:03:28.940 --> 00:03:31.360
So what's the deal?
00:03:31.360 --> 00:03:32.210
All right.
00:03:32.210 --> 00:03:36.500
The main point is,
what about length?
00:03:36.500 --> 00:03:40.560
I'm given a vector,
I have a vector x.
00:03:40.560 --> 00:03:44.310
Or let me call it z as a
reminder that it's complex,
00:03:44.310 --> 00:03:45.260
for the moment.
00:03:45.260 --> 00:03:48.950
But I can -- later I'll
call the components x.
00:03:48.950 --> 00:03:50.740
They'll be complex numbers.
00:03:50.740 --> 00:03:53.090
But it's a vector --
00:03:53.090 --> 00:03:56.500
z1, z2 down to zn.
00:03:56.500 --> 00:04:01.590
So the only novelty is
it's not in R^n anymore.
00:04:01.590 --> 00:04:04.370
It's in complex n
dimensional space.
00:04:04.370 --> 00:04:07.510
Each of those numbers
is a complex number.
00:04:07.510 --> 00:04:14.750
So this z,z1 is in C^n, n
dimensional complex space
00:04:14.750 --> 00:04:17.700
instead of R^n.
00:04:17.700 --> 00:04:20.750
So just a different letter
there, but now the point
00:04:20.750 --> 00:04:23.200
about its length is what?
00:04:23.200 --> 00:04:29.190
The point about its length is
that z transpose z is no good.
00:04:34.800 --> 00:04:38.530
z transpose z -- if I just
put down z transpose here,
00:04:38.530 --> 00:04:42.090
it would be z1, z2, to zn.
00:04:45.580 --> 00:04:47.990
Doing that multiplication
doesn't give me
00:04:47.990 --> 00:04:50.090
the right thing.
00:04:50.090 --> 00:04:51.300
W-Why not?
00:04:51.300 --> 00:04:57.390
Because the length squared
should be positive.
00:04:57.390 --> 00:05:01.930
And if I multiply -- suppose
this is, like, 1 and i.
00:05:01.930 --> 00:05:06.560
What's the length of the
vector with components 1 and i?
00:05:06.560 --> 00:05:08.550
What if I do this,
so n is just two.
00:05:08.550 --> 00:05:11.380
I'm in C^2, two
dimensional space,
00:05:11.380 --> 00:05:16.510
complex space with the vector
whose components are 1 and i.
00:05:16.510 --> 00:05:17.310
All right.
00:05:17.310 --> 00:05:23.320
So if I took one times one
and i times i and added,
00:05:23.320 --> 00:05:27.150
z transpose z would be zero.
00:05:27.150 --> 00:05:29.310
But I don't -- that
vector is not --
00:05:29.310 --> 00:05:32.030
doesn't have length zero -- the
vector with the components 1
00:05:32.030 --> 00:05:33.590
and i --
00:05:33.590 --> 00:05:41.710
this multiplication -- what I
really want is z1 conjugate z1.
00:05:41.710 --> 00:05:46.510
You remember that z1
conjugate z1 is --
00:05:46.510 --> 00:05:50.370
so you see that first step
will be z1 conjugate z1,
00:05:50.370 --> 00:05:56.820
which is the magnitude of z1
squared, which is what I want.
00:05:56.820 --> 00:06:00.550
That's, like, three
squared or five squared.
00:06:00.550 --> 00:06:11.170
Now, if it's -- if z1 is i, then
I multiplied by minus i gives
00:06:11.170 --> 00:06:19.290
one plus one, so the component
of length -- the component i,
00:06:19.290 --> 00:06:21.731
its modulus squared is plus
00:06:21.731 --> 00:06:22.230
one.
00:06:22.230 --> 00:06:22.990
That's great.
00:06:22.990 --> 00:06:25.560
So what I want to do
then is do that --
00:06:25.560 --> 00:06:30.680
I want z1 bar z1, z2
bar z2, zn bar zn.
00:06:30.680 --> 00:06:33.210
And remember that -- you
remember this complex
00:06:33.210 --> 00:06:33.900
conjugate.
00:06:33.900 --> 00:06:38.270
So -- so there's the point.
00:06:38.270 --> 00:06:41.030
Now I can erase
the no good and put
00:06:41.030 --> 00:06:48.800
is good, because that now gives
the answer zero for the zero
00:06:48.800 --> 00:06:52.590
vector, of course, but it
gives a positive length
00:06:52.590 --> 00:06:55.780
squared for any other vector.
00:06:55.780 --> 00:06:58.890
So it's a -- it's the
right definition of length,
00:06:58.890 --> 00:07:04.370
and essentially the message is
that we're always going to be
00:07:04.370 --> 00:07:06.750
taking --
00:07:06.750 --> 00:07:09.400
when we transpose, we also
take complex conjugate.
00:07:09.400 --> 00:07:13.670
So let's -- let's find
the length of one --
00:07:13.670 --> 00:07:19.890
so the vector one i, that's
z, that's that vector z.
00:07:19.890 --> 00:07:23.490
Now I take the conjugate of one
is one, the conjugate of i is
00:07:23.490 --> 00:07:24.650
minus i.
00:07:24.650 --> 00:07:29.340
I take this vector,
I get one plus one --
00:07:29.340 --> 00:07:29.840
I get
00:07:29.840 --> 00:07:30.339
two.
00:07:33.050 --> 00:07:36.810
So that's a vector and that's a
vector of length -- square root
00:07:36.810 --> 00:07:37.740
of two.
00:07:37.740 --> 00:07:40.560
Square root of two is the
length and not the zero
00:07:40.560 --> 00:07:43.830
that we would have got
from one minus i squared.
00:07:43.830 --> 00:07:46.210
Okay.
00:07:46.210 --> 00:07:49.250
So the message really is
whenever we transpose,
00:07:49.250 --> 00:07:51.660
we also take conjugates.
00:07:51.660 --> 00:07:57.300
So here's a symbol --
one symbol to do both.
00:07:57.300 --> 00:08:03.890
So that symbol H, it stands
for a guy named Hermite,
00:08:03.890 --> 00:08:06.710
who didn't actually
pronounce the H,
00:08:06.710 --> 00:08:09.630
but let's pronounce it --
00:08:09.630 --> 00:08:14.040
so I would call
that z Hermitian z.
00:08:14.040 --> 00:08:20.940
I'll -- let me write that word,
Herm- so his name was Hermite,
00:08:20.940 --> 00:08:27.240
and then we make it into
an adjective, Hermitian.
00:08:27.240 --> 00:08:32.000
So z Hermitian z. z H z.
00:08:32.000 --> 00:08:34.360
Okay.
00:08:34.360 --> 00:08:37.510
So, that's the --
00:08:37.510 --> 00:08:40.730
that's the, length squared.
00:08:40.730 --> 00:08:43.409
Now what's the inner product?
00:08:43.409 --> 00:08:45.000
Well, it should match.
00:08:45.000 --> 00:08:49.850
The inner product
of two vectors --
00:08:49.850 --> 00:08:56.060
so inner product is no longer --
00:08:56.060 --> 00:08:59.230
used to be y transpose x.
00:08:59.230 --> 00:09:01.680
That's for real vectors.
00:09:01.680 --> 00:09:04.970
For complex vectors,
whenever we transpose,
00:09:04.970 --> 00:09:07.220
we also take the conjugate.
00:09:07.220 --> 00:09:12.110
So it's y Hermitian x.
00:09:12.110 --> 00:09:14.160
Of course it's not
real anymore, usually.
00:09:14.160 --> 00:09:19.610
That -- the inner product will
usually be complex number.
00:09:19.610 --> 00:09:24.100
But if y and x are the
same, if they're the same z,
00:09:24.100 --> 00:09:29.240
then we have z -- z H z,
we have the length squared,
00:09:29.240 --> 00:09:31.950
and that's what we want, the
inner product of a vector with
00:09:31.950 --> 00:09:35.490
itself should be
its length squared.
00:09:35.490 --> 00:09:40.230
So this is, like, forced on us
because this is forced on us.
00:09:40.230 --> 00:09:43.750
So -- so this z -- this --
00:09:43.750 --> 00:09:45.660
everybody's picking
up what this equals.
00:09:45.660 --> 00:09:52.170
This is z1 squared
plus zn squared.
00:09:52.170 --> 00:09:53.820
That's the length squared.
00:09:53.820 --> 00:09:57.080
And that's the inner product
that we have to go with.
00:09:57.080 --> 00:10:00.560
So it could be a
complex number now.
00:10:00.560 --> 00:10:02.790
One more change.
00:10:02.790 --> 00:10:04.750
Well, two more changes.
00:10:04.750 --> 00:10:08.780
We've got to change the
idea of a symmetric matrix.
00:10:08.780 --> 00:10:11.870
So I'll just recap on
symmetric matrixes.
00:10:11.870 --> 00:10:21.610
Symmetric means A transpose
equals A, but not --
00:10:21.610 --> 00:10:30.430
no good if A is complex.
00:10:30.430 --> 00:10:33.890
So what do we instead --
00:10:37.490 --> 00:10:40.360
that applies perfectly
to real matrixes.
00:10:40.360 --> 00:10:42.760
But now if my
matrixes were complex,
00:10:42.760 --> 00:10:51.910
I want to take the transpose
and the conjugate to equal A.
00:10:51.910 --> 00:10:55.430
So there's -- that's the --
the right complex version
00:10:55.430 --> 00:10:56.780
of symmetry.
00:10:56.780 --> 00:10:59.470
The com- the symmetry
now means when
00:10:59.470 --> 00:11:03.180
I transpose it, flip across the
diagonal and take conjugates.
00:11:03.180 --> 00:11:07.250
So, for example -- here
would be an example.
00:11:07.250 --> 00:11:09.480
On the diagonal, it
had better be real,
00:11:09.480 --> 00:11:15.760
because when I flip it, the
diagonal is still there and it
00:11:15.760 --> 00:11:18.410
has to -- and then when I take
the complex conjugate it has
00:11:18.410 --> 00:11:21.240
to be still there, so it
better be a real number,
00:11:21.240 --> 00:11:24.580
let me say two and five.
00:11:24.580 --> 00:11:27.790
What about entries
off the diagonal?
00:11:27.790 --> 00:11:32.200
If this entry is,
say, three plus i,
00:11:32.200 --> 00:11:36.630
then this entry had better be --
00:11:36.630 --> 00:11:40.070
because I want whatever
this -- when I transpose,
00:11:40.070 --> 00:11:42.790
it'll show up here
and i conjugate.
00:11:42.790 --> 00:11:47.000
So I need three minus I there.
00:11:47.000 --> 00:11:51.400
So there's a matrix with --
00:11:51.400 --> 00:11:55.640
that corresponds to
symmetry, but it's complex.
00:11:55.640 --> 00:12:01.550
And those matrixes are
called Hermitian matrixes.
00:12:01.550 --> 00:12:03.170
Hermitian matrixes.
00:12:03.170 --> 00:12:06.210
A H equals A.
00:12:06.210 --> 00:12:08.790
Fine.
00:12:08.790 --> 00:12:13.140
Okay, that's -- and those
matrixes have real eigenvalues
00:12:13.140 --> 00:12:15.950
and they have
perpendicular eigenvectors.
00:12:15.950 --> 00:12:18.760
What does perpendicular mean?
00:12:18.760 --> 00:12:21.760
Perpendicular means
the inner product --
00:12:21.760 --> 00:12:23.585
so let's go on to perpendicular.
00:12:28.180 --> 00:12:31.410
Well, when I had perpendicular
vectors, for example,
00:12:31.410 --> 00:12:37.820
they were like q1, q2 up to qn.
00:12:37.820 --> 00:12:41.630
That's my -- q is my letter
that I use for perpendicular.
00:12:41.630 --> 00:12:43.190
Actually, I usually --
00:12:43.190 --> 00:12:45.840
I also mean unit length.
00:12:45.840 --> 00:12:49.010
So those are perpendicular
unit vectors.
00:12:49.010 --> 00:12:51.030
But now what does
-- so it's a --
00:12:51.030 --> 00:12:54.730
orthonormal basis, I'll
still use those words,
00:12:54.730 --> 00:12:58.560
but how do I compute
perpendicular?
00:12:58.560 --> 00:13:00.770
How do I check perpendicular?
00:13:00.770 --> 00:13:07.370
This means that the inner
product of qi with qj --
00:13:07.370 --> 00:13:11.510
but now I not only transpose,
I must conjugate, right,
00:13:11.510 --> 00:13:20.210
to get zero if i is not
j and one if i is j.
00:13:20.210 --> 00:13:25.950
So it's a unit vector, meaning
unit length, orthogonal --
00:13:25.950 --> 00:13:28.930
all the angles are right
angles, but these are angles
00:13:28.930 --> 00:13:33.020
in complex n dimensional space.
00:13:33.020 --> 00:13:37.155
So it's q1, q on-
qi bar transpose.
00:13:39.790 --> 00:13:44.310
Or, for short, qi H qj.
00:13:44.310 --> 00:13:46.650
So it will still be
true -- so let me --
00:13:46.650 --> 00:13:51.230
again I'll create a
matrix out of those guys.
00:13:51.230 --> 00:13:56.090
The matrix will have these
q-s in its columns, q2 to qn.
00:14:00.320 --> 00:14:03.400
And I want to turn that
into matrix language,
00:14:03.400 --> 00:14:05.130
just like before.
00:14:05.130 --> 00:14:06.690
What does that mean?
00:14:06.690 --> 00:14:08.800
That means I want all
these inner products,
00:14:08.800 --> 00:14:13.920
so I take these columns of
Q, multiply by their rows --
00:14:13.920 --> 00:14:17.250
so it was -- it used to be Q
-- it used to be Q transpose Q
00:14:17.250 --> 00:14:18.160
equals I, right?
00:14:20.840 --> 00:14:22.535
This was an orthogonal matrix.
00:14:27.100 --> 00:14:30.490
But what's changed?
00:14:30.490 --> 00:14:33.290
These are now complex vectors.
00:14:33.290 --> 00:14:37.730
Their inner products are --
involve conjugating the first
00:14:37.730 --> 00:14:39.920
factor.
00:14:39.920 --> 00:14:44.240
So it's not -- it's the
conjugate of Q transpose.
00:14:44.240 --> 00:14:47.050
It's Q bar transpose Q.
00:14:47.050 --> 00:14:48.590
Q H.
00:14:48.590 --> 00:14:55.240
So can I call this -- let me
call it Q H Q, which is I.
00:14:55.240 --> 00:14:57.260
So that's our new --
00:14:57.260 --> 00:14:59.550
you -- you see I'm just
translating, and the --
00:14:59.550 --> 00:15:07.300
the book h- on one page gives a
little dictionary of the right
00:15:07.300 --> 00:15:12.900
words in the real case, R^n,
and the corresponding words
00:15:12.900 --> 00:15:17.120
in the complex case for
the vector space C^n.
00:15:17.120 --> 00:15:19.600
Of course, C^n is
a vector space,
00:15:19.600 --> 00:15:22.650
the numbers we multiply
are now complex numbers --
00:15:22.650 --> 00:15:27.770
we're just moving into
complex n dimensional space.
00:15:27.770 --> 00:15:29.700
Okay.
00:15:29.700 --> 00:15:34.450
Now -- actually, I have to say
we changed the word symmet-
00:15:34.450 --> 00:15:38.500
symmetric to Hermitian
for those matrixes.
00:15:38.500 --> 00:15:43.300
People also change this word
orthogonal into another word
00:15:43.300 --> 00:15:52.730
that happens to be unitary,
as a word that applies --
00:15:52.730 --> 00:15:56.540
that signals that we might be
dealing with a complex matrix
00:15:56.540 --> 00:15:57.320
here.
00:15:57.320 --> 00:15:59.940
So what's a unitary matrix?
00:15:59.940 --> 00:16:02.960
It's a -- it's just like
an orthogonal matrix.
00:16:02.960 --> 00:16:10.800
It's a square, n by n matrix
with orthonormal columns,
00:16:10.800 --> 00:16:13.970
perpendicular columns,
unit vectors --
00:16:13.970 --> 00:16:20.010
unit vectors computed by --
and perpendicularity computed
00:16:20.010 --> 00:16:23.820
by remembering that
there's a conjugate as well
00:16:23.820 --> 00:16:25.250
as a transpose.
00:16:25.250 --> 00:16:26.270
Okay.
00:16:26.270 --> 00:16:28.520
So those are the words.
00:16:28.520 --> 00:16:32.880
Now I'm ready to get into the
substance of the lecture which
00:16:32.880 --> 00:16:38.290
is the most famous complex
matrix, which happens
00:16:38.290 --> 00:16:41.500
to be one of these guys.
00:16:41.500 --> 00:16:50.240
It has orthogonal columns,
and it's named after Fourier
00:16:50.240 --> 00:16:52.800
because it comes into
the Fourier transform,
00:16:52.800 --> 00:16:55.380
so it's the matrix
that's all around us.
00:16:55.380 --> 00:16:55.880
Okay.
00:16:58.450 --> 00:17:05.660
Let me tell you what it is
first of all in the n by n case.
00:17:05.660 --> 00:17:09.750
Then often I'll let n be four
because four is a good size
00:17:09.750 --> 00:17:11.150
to work with.
00:17:11.150 --> 00:17:13.369
But here's the n by
n Fourier matrix.
00:17:17.790 --> 00:17:20.319
Its first column is
the vector of ones.
00:17:24.310 --> 00:17:26.319
It's n by n, of course.
00:17:26.319 --> 00:17:30.420
Its second column is
the powers, the --
00:17:30.420 --> 00:17:35.370
actually, better if I move
from the math department
00:17:35.370 --> 00:17:39.910
to EE for this one half
hour and then, please,
00:17:39.910 --> 00:17:42.100
let me move back again.
00:17:42.100 --> 00:17:43.870
Okay.
00:17:43.870 --> 00:17:46.170
What's the difference between
those two departments?
00:17:46.170 --> 00:17:50.640
It's just math starts
counting with one
00:17:50.640 --> 00:17:56.180
and electrical engineers
start counting at zero.
00:17:56.180 --> 00:17:57.840
Actually, they're
probably right.
00:17:57.840 --> 00:18:00.720
So anyway, we'll give
them -- humor them.
00:18:00.720 --> 00:18:05.090
So this is really
the zeroes column.
00:18:05.090 --> 00:18:08.030
And the first column
up to the n-1, that's
00:18:08.030 --> 00:18:11.780
the one inconvenient spot
in electrical engineering.
00:18:11.780 --> 00:18:14.760
All these expressions
start at zero, no problem,
00:18:14.760 --> 00:18:16.720
but they end at n-1.
00:18:16.720 --> 00:18:24.000
Well, that's -- that's the
difficulty of Course 6.
00:18:24.000 --> 00:18:27.750
So what's -- they're the powers
of a number that I'm going
00:18:27.750 --> 00:18:28.660
to call W --
00:18:28.660 --> 00:18:34.840
W squared, W cubed, W to the
-- now what is the W here?
00:18:34.840 --> 00:18:36.060
What's the power?
00:18:36.060 --> 00:18:39.000
This was the zeroes power,
first power, second power,
00:18:39.000 --> 00:18:43.290
this will be n
minus first power.
00:18:43.290 --> 00:18:45.130
That's the column.
00:18:45.130 --> 00:18:46.500
What's the next column?
00:18:46.500 --> 00:18:51.920
It's the powers of W squared, W
to the fourth, W to the sixth,
00:18:51.920 --> 00:18:55.870
W to the two n-1.
00:18:55.870 --> 00:18:58.010
And then more columns
and more columns
00:18:58.010 --> 00:19:00.060
and more columns and
what's the last column?
00:19:02.790 --> 00:19:07.361
It's the powers of --
00:19:07.361 --> 00:19:07.860
let's see.
00:19:07.860 --> 00:19:10.860
We -- actually, if we
look around rows, w-
00:19:10.860 --> 00:19:12.520
this matrix is symmetric.
00:19:12.520 --> 00:19:17.220
It's symmetric in the old
not quite perfect way,
00:19:17.220 --> 00:19:21.100
not perfect because these
numbers are complex.
00:19:21.100 --> 00:19:25.370
And so it's -- that
first row is all ones.
00:19:25.370 --> 00:19:28.740
One W W squared up
to W to the n-1.
00:19:28.740 --> 00:19:34.500
That's the last column is
the powers of W to the n-1,
00:19:34.500 --> 00:19:38.841
so this guy matches that, and
finally we get W to something
00:19:38.841 --> 00:19:39.340
here.
00:19:43.060 --> 00:19:46.670
I guess we could actually figure
out what that something is.
00:19:46.670 --> 00:19:48.610
What are the entries
of this matrix?
00:19:48.610 --> 00:19:57.490
The i j entry of
this matrix are --
00:19:57.490 --> 00:20:00.420
I going to -- are you going to
allow me to let i go from zero
00:20:00.420 --> 00:20:01.890
to n minus one?
00:20:01.890 --> 00:20:07.180
So i and g go from zero to n-1.
00:20:07.180 --> 00:20:10.870
So the one -- the zero
zero entry is a one --
00:20:10.870 --> 00:20:15.670
it's just this same W guy
to the power i times j.
00:20:21.960 --> 00:20:22.460
Let's see.
00:20:22.460 --> 00:20:24.690
I'm jumping into formulas
here and I have to tell you
00:20:24.690 --> 00:20:27.903
what W is and then you know
everything about this matrix.
00:20:30.960 --> 00:20:34.520
So W is the -- well,
shall we finish here?
00:20:34.520 --> 00:20:37.890
What was this -- this is
the (n-1) (n-1) entry.
00:20:37.890 --> 00:20:41.380
This is W to the n-1 squared.
00:20:41.380 --> 00:20:46.140
Everything's looking like a
mess here, because we have --
00:20:46.140 --> 00:20:51.890
not too bad, because all
the entries are powers of W.
00:20:51.890 --> 00:20:53.830
There -- none of them are zero.
00:20:53.830 --> 00:20:56.600
This is a full matrix.
00:20:56.600 --> 00:20:59.840
But W is a very special number.
00:20:59.840 --> 00:21:08.720
W is the special number
whose n-th power is one.
00:21:08.720 --> 00:21:12.780
In fact -- well, actually,
there are n numbers like that.
00:21:12.780 --> 00:21:14.620
One of them is one, of course.
00:21:14.620 --> 00:21:19.850
But the one we --
the W we want is --
00:21:19.850 --> 00:21:23.470
the angle is two pi over n.
00:21:28.060 --> 00:21:29.130
Is that what I mean?
00:21:29.130 --> 00:21:32.740
n over two pi.
00:21:32.740 --> 00:21:34.310
No, two pi over n.
00:21:34.310 --> 00:21:42.540
W is E to the I and the
angle is two pi over n.
00:21:42.540 --> 00:21:43.550
Right.
00:21:43.550 --> 00:21:47.590
Where is this W in
the complex plane?
00:21:47.590 --> 00:21:49.870
It's -- it's on the
unit circle, right?
00:21:49.870 --> 00:21:56.160
This is -- it's the cosine
of two pi over n plus I times
00:21:56.160 --> 00:21:59.620
the sine of two pi over n.
00:21:59.620 --> 00:22:02.840
But actually, forget this.
00:22:02.840 --> 00:22:09.050
It's never good to work with
the real and imaginary parts,
00:22:09.050 --> 00:22:14.130
the rectangular coordinates,
when we're taking powers.
00:22:14.130 --> 00:22:17.770
To take that to the tenth power,
we can't see what we're doing.
00:22:17.770 --> 00:22:19.890
To take this form
to the tenth power,
00:22:19.890 --> 00:22:22.350
we see immediately
what we're doing.
00:22:22.350 --> 00:22:26.480
It would be e to
the i 20 pi over n.
00:22:26.480 --> 00:22:30.570
So when our matrix is full of
powers -- so it's this formula,
00:22:30.570 --> 00:22:32.690
and where is this on
the complex plain?
00:22:32.690 --> 00:22:36.830
Here are the real numbers,
here's the imaginary axis,
00:22:36.830 --> 00:22:40.410
here's the unit
circle of radius one,
00:22:40.410 --> 00:22:44.240
and this number is
on the unit circle
00:22:44.240 --> 00:22:48.860
at this angle, which is one
n-th of the full way round.
00:22:48.860 --> 00:22:52.700
So if I drew, for
example, n equals six,
00:22:52.700 --> 00:22:56.490
this would be e to the
two pi, two pi over six,
00:22:56.490 --> 00:22:59.160
it would be one sixth of the way
around, it would be 60 degrees.
00:23:03.250 --> 00:23:06.090
And where is W squared?
00:23:06.090 --> 00:23:13.120
So I -- my W is e to the two
pi I over six in this case,
00:23:13.120 --> 00:23:16.760
in this six by -- for the
six by six Fourier transform,
00:23:16.760 --> 00:23:22.800
it's totally constructed out
of this number and its powers.
00:23:22.800 --> 00:23:24.480
So what are its powers?
00:23:24.480 --> 00:23:27.470
Well, its powers are on
the unit circle, right?
00:23:27.470 --> 00:23:32.680
Because when I square a
number, a complex number,
00:23:32.680 --> 00:23:37.740
I square its absolute value,
which gives me one again.
00:23:37.740 --> 00:23:40.500
All the powers have --
are on the unit circle.
00:23:40.500 --> 00:23:43.250
And the -- the angle
gets doubled to a hundred
00:23:43.250 --> 00:23:46.180
and twenty, so there's W
squared, there's W cubed,
00:23:46.180 --> 00:23:50.660
there's W to the fourth, there's
W to the fifth and there is W
00:23:50.660 --> 00:23:57.070
to the sixth, as we hoped, W to
the sixth coming back to one.
00:23:57.070 --> 00:24:01.080
So those are the six --
00:24:01.080 --> 00:24:02.620
can I say this on TV?
00:24:02.620 --> 00:24:07.130
The six sixth roots
of one, and it's
00:24:07.130 --> 00:24:14.560
this one, the primitive one we
say, the first one, which is W.
00:24:14.560 --> 00:24:17.840
Okay, so what -- let
me change -- let me --
00:24:17.840 --> 00:24:21.440
I said I would probably
switch to n equal four.
00:24:21.440 --> 00:24:24.130
What's W for that?
00:24:24.130 --> 00:24:25.970
It's the fourth root of one.
00:24:25.970 --> 00:24:28.590
W to the fourth will be one.
00:24:28.590 --> 00:24:33.410
W will be e to the two
pi i over four now.
00:24:38.820 --> 00:24:41.040
What's that?
00:24:41.040 --> 00:24:43.270
This is e to the i pi over two.
00:24:43.270 --> 00:24:46.180
This is a quarter of the
way around the unit circle,
00:24:46.180 --> 00:24:53.930
and that's exactly i, a
quarter of the way around.
00:24:53.930 --> 00:24:58.660
And sure enough, the
powers are i, i squared,
00:24:58.660 --> 00:25:01.670
which is minus
one, i cubed, which
00:25:01.670 --> 00:25:08.780
is minus i and finally i to
the fourth which is one, right.
00:25:08.780 --> 00:25:12.760
So there's W, W squared, W
cubed, W to the fourth --
00:25:12.760 --> 00:25:15.050
I'm really ready to
write down this Fourier
00:25:15.050 --> 00:25:18.780
matrix for the
four by four case,
00:25:18.780 --> 00:25:21.790
just so we see that clearly.
00:25:21.790 --> 00:25:22.890
Let me do it here.
00:25:22.890 --> 00:25:34.760
F4 is -- all right, one one one
one one one one W -- it's I.
00:25:34.760 --> 00:25:35.590
I squared.
00:25:35.590 --> 00:25:36.770
That's minus one.
00:25:36.770 --> 00:25:38.520
i cubed is minus i.
00:25:41.350 --> 00:25:43.490
I'll -- I could write
i squared and i cubed.
00:25:43.490 --> 00:25:46.140
Why don't I, just so we
see the pattern for sure.
00:25:46.140 --> 00:25:54.040
i squared, i cubed, i squared,
i cubed, i fourth, i sixth --
00:25:54.040 --> 00:25:58.690
i fourth, i sixth and i ninth.
00:25:58.690 --> 00:26:01.970
You see the exponents
fall in this nice --
00:26:01.970 --> 00:26:05.760
the exponent is the row number
times the column number,
00:26:05.760 --> 00:26:08.770
always starting at zero.
00:26:08.770 --> 00:26:09.560
Okay.
00:26:09.560 --> 00:26:12.910
And now I can put in those
numbers if you like --
00:26:12.910 --> 00:26:20.520
one one one one, one i minus
one minus i, one minus one,
00:26:20.520 --> 00:26:28.710
one minus one and one
minus i minus one i.
00:26:28.710 --> 00:26:29.410
No.
00:26:29.410 --> 00:26:31.111
Yes.
00:26:31.111 --> 00:26:31.610
Right.
00:26:38.260 --> 00:26:40.965
What's -- why do I think
that matrix is so remarkable?
00:26:44.520 --> 00:26:49.030
It's the four by four matrix
that comes into the four point
00:26:49.030 --> 00:26:52.020
Fourier transform.
00:26:52.020 --> 00:26:55.700
When we want to find the Fourier
transform, the four point
00:26:55.700 --> 00:27:01.380
Fourier transform of a
vector with four components,
00:27:01.380 --> 00:27:05.540
we want to multiply
by this F4 or we
00:27:05.540 --> 00:27:08.920
want to multiply by F4 inverse.
00:27:08.920 --> 00:27:10.840
One way we're taking
the transform,
00:27:10.840 --> 00:27:13.820
one way we're taking
the inverse transform.
00:27:13.820 --> 00:27:17.950
Actually, they're so close that
it's easy to confuse the two.
00:27:17.950 --> 00:27:21.990
The inverse of this matrix
will be a nice matrix also.
00:27:25.250 --> 00:27:29.190
So -- and that's, of
course, what makes it --
00:27:29.190 --> 00:27:32.370
that -- I guess
Fourier knew that.
00:27:32.370 --> 00:27:34.200
He knew the inverse
of this matrix.
00:27:36.970 --> 00:27:39.890
A- as you'll see, it just comes
from the fact that the columns
00:27:39.890 --> 00:27:41.640
are orthogonal --
00:27:41.640 --> 00:27:44.050
from the fact that the
columns are orthogonal,
00:27:44.050 --> 00:27:51.220
we will quickly figure
out what is the inverse.
00:27:51.220 --> 00:27:53.910
What Fourier didn't
know -- didn't notice --
00:27:53.910 --> 00:27:58.180
I think Gauss noticed it but
didn't make a point of it
00:27:58.180 --> 00:28:00.020
and then it turned
out to be really
00:28:00.020 --> 00:28:04.770
important was the fact that this
matrix is so special that you
00:28:04.770 --> 00:28:10.340
can break it up into nice pieces
with lots of zeroes, factors
00:28:10.340 --> 00:28:13.360
that have lots of
zeroes and multiply
00:28:13.360 --> 00:28:16.650
by it or by its inverse
very, very fast.
00:28:16.650 --> 00:28:17.150
Okay.
00:28:19.740 --> 00:28:22.850
But how did it get into
this lecture first?
00:28:22.850 --> 00:28:26.340
Because the columns
are orthogonal.
00:28:26.340 --> 00:28:29.130
Can I just check that the
columns of this matrix
00:28:29.130 --> 00:28:30.265
are orthogonal?
00:28:33.480 --> 00:28:38.360
So the inner product of
that column with that column
00:28:38.360 --> 00:28:41.260
is zero.
00:28:41.260 --> 00:28:48.250
The inner product of column
one with column three is zero.
00:28:48.250 --> 00:28:55.850
How about the inner
product of two and four?
00:28:55.850 --> 00:29:01.280
Can I take the inner product
of column two with column four?
00:29:01.280 --> 00:29:04.860
Or even the inner product of two
with three, let's -- let's see,
00:29:04.860 --> 00:29:06.820
does that --
00:29:06.820 --> 00:29:07.751
let me do two and
00:29:07.751 --> 00:29:08.250
four.
00:29:15.330 --> 00:29:17.260
Okay.
00:29:17.260 --> 00:29:19.110
What -- oh, I see, yes, hmm.
00:29:19.110 --> 00:29:19.610
Hmm.
00:29:19.610 --> 00:29:30.060
Let's see, I believe that those
two columns are orthogonal.
00:29:30.060 --> 00:29:33.520
So let me take their inner
product and hope to get zero.
00:29:33.520 --> 00:29:36.310
Okay, now if you hadn't
listened to the first half
00:29:36.310 --> 00:29:39.890
of this lecture, when you
took the inner product of that
00:29:39.890 --> 00:29:42.760
with that, you would have
multiplied one by one,
00:29:42.760 --> 00:29:48.090
i by minus i, and that
would have given you one,
00:29:48.090 --> 00:29:51.370
minus one by minus one
would have given you
00:29:51.370 --> 00:29:55.940
another one minus I by I would
have been minus I squared,
00:29:55.940 --> 00:29:56.810
that's another one.
00:30:00.870 --> 00:30:04.240
So do I conclude that the
inner product of columns --
00:30:04.240 --> 00:30:05.920
I said columns two
and four, that's
00:30:05.920 --> 00:30:08.550
because I forgot those
are columns one and three.
00:30:12.110 --> 00:30:13.670
I'm interested in
their inner product
00:30:13.670 --> 00:30:17.570
and I'm hoping it's zero, but
it doesn't look like zero.
00:30:17.570 --> 00:30:19.090
Nevertheless, it is zero.
00:30:19.090 --> 00:30:20.580
Those columns are perpendicular.
00:30:20.580 --> 00:30:21.500
Why?
00:30:21.500 --> 00:30:25.390
Because the inner product --
00:30:25.390 --> 00:30:26.520
we conjugate.
00:30:26.520 --> 00:30:29.150
Do you remember that the --
one of the vectors in the inner
00:30:29.150 --> 00:30:31.510
product has to get conjugated.
00:30:31.510 --> 00:30:34.750
So when I conjugated, it
changes that i to a minus i,
00:30:34.750 --> 00:30:37.900
changes this to a plus
i, changes those --
00:30:37.900 --> 00:30:44.060
that second sine and that
fourth sine and I do get zero.
00:30:44.060 --> 00:30:46.490
So those columns are orthogonal.
00:30:46.490 --> 00:30:49.130
So columns are orthogonal.
00:30:51.780 --> 00:30:55.630
They're not quite orthonormal.
00:30:55.630 --> 00:30:58.020
But I could fix that easily.
00:30:58.020 --> 00:31:02.710
They -- all those
columns have length two.
00:31:02.710 --> 00:31:07.450
Length squared is four, like
this -- the four I had there --
00:31:07.450 --> 00:31:10.260
this length squared, one plus
-- one squared one squared one
00:31:10.260 --> 00:31:13.910
squared one squared is
four, square root is two --
00:31:13.910 --> 00:31:17.420
so if I really wanted them --
suppose I really wanted to fix
00:31:17.420 --> 00:31:21.300
life perfectly, I
could divide by two,
00:31:21.300 --> 00:31:26.653
and now I have columns that
are actually orthonormal.
00:31:33.420 --> 00:31:35.560
So what?
00:31:35.560 --> 00:31:38.040
So I can invert
right away, right?
00:31:38.040 --> 00:31:41.970
O- orthonormal columns means --
now I'm keeping this one half
00:31:41.970 --> 00:31:43.820
in here for the moment --
00:31:43.820 --> 00:31:48.060
c- means F4 Hermitian,
can I use that,
00:31:48.060 --> 00:31:51.620
conjugate transpose times F4 is
00:31:51.620 --> 00:31:52.120
i.
00:31:56.230 --> 00:31:59.150
So I see what the inverse is.
00:31:59.150 --> 00:32:01.840
The inverse of F4 is
-- it's just like an --
00:32:01.840 --> 00:32:03.180
an orthogonal matrix.
00:32:03.180 --> 00:32:06.230
The inverse is the transpose
-- here the inverse is
00:32:06.230 --> 00:32:08.990
the conjugate transpose.
00:32:08.990 --> 00:32:10.250
So, fine.
00:32:10.250 --> 00:32:15.000
That -- that tells me that
anything good that I learn
00:32:15.000 --> 00:32:19.610
about F4 I'll know the same --
00:32:19.610 --> 00:32:21.710
I'll know a similar
fact about its inverse,
00:32:21.710 --> 00:32:25.620
because its inverse is just
its conjugate transpose.
00:32:25.620 --> 00:32:28.080
Okay, now -- so what's good?
00:32:28.080 --> 00:32:30.430
Well, first, the
columns are orthogonal.
00:32:30.430 --> 00:32:33.870
That's a key fact, then.
00:32:33.870 --> 00:32:36.670
That's the thing that
makes the inverse easy.
00:32:36.670 --> 00:32:40.740
But what property is it that
leads to the fast Fourier
00:32:40.740 --> 00:32:41.460
transform?
00:32:41.460 --> 00:32:44.000
So now I'm going to talk,
in these last minutes,
00:32:44.000 --> 00:32:48.010
about the fast
Fourier transform.
00:32:48.010 --> 00:32:49.290
What -- here's the idea.
00:32:51.980 --> 00:32:56.030
F6, our six by six
matrix, will c-
00:32:56.030 --> 00:33:02.250
there's a neat connection
to F3, half as big.
00:33:02.250 --> 00:33:05.200
There's a connection
of F8 to F4.
00:33:05.200 --> 00:33:09.690
There's a connection
of F(64) to F(32).
00:33:09.690 --> 00:33:12.070
Shall I write down what
that connection is?
00:33:12.070 --> 00:33:15.190
What's the connection
of F(64) to F(32)?
00:33:15.190 --> 00:33:22.950
So F(64) is a 64 by
64 matrix whose W
00:33:22.950 --> 00:33:26.500
is the 64th root of one.
00:33:26.500 --> 00:33:31.870
So it's one 64th of
the way round in F(64).
00:33:31.870 --> 00:33:35.959
And it -- do- and F(32)
is a 32 by 32 matrix.
00:33:35.959 --> 00:33:37.375
Remember, they're
different sizes.
00:33:40.450 --> 00:33:45.680
And the W in that 32 by 32
matrix is the 32nd root of one,
00:33:45.680 --> 00:33:50.220
which is twice as far -- that
-- you sh- see that key point --
00:33:50.220 --> 00:33:55.850
that's the -- that's how 32
and 64 are connected in the Ws.
00:33:55.850 --> 00:33:59.380
The W for 64 is one
64th of the way --
00:33:59.380 --> 00:34:05.610
so all I'm saying is
that if I square the W --
00:34:05.610 --> 00:34:09.219
W(64), that's what I'm using
for the one over -- the --
00:34:09.219 --> 00:34:15.070
W sixty f- this Wn is either
the i two pi over n --
00:34:15.070 --> 00:34:18.690
so W(64) is one 64th
of the way around it.
00:34:18.690 --> 00:34:24.139
When I square that,
what do I get but W(32)?
00:34:24.139 --> 00:34:25.969
Right?
00:34:25.969 --> 00:34:31.060
If I square this matrix,
I double the angle --
00:34:31.060 --> 00:34:36.300
if I square this number, I
double the angle, I get, the --
00:34:36.300 --> 00:34:40.120
the W(32).
00:34:40.120 --> 00:34:43.239
So somehow there's
a little hope here
00:34:43.239 --> 00:34:46.440
to connect F(64) with F(32).
00:34:46.440 --> 00:34:49.400
And here's the connection.
00:34:49.400 --> 00:34:49.980
Okay.
00:34:49.980 --> 00:34:53.530
Let me -- let me go back, --
00:34:53.530 --> 00:34:54.170
yes, let me --
00:34:54.170 --> 00:34:58.780
I'll do it here.
00:34:58.780 --> 00:35:00.520
Here's the connection.
00:35:00.520 --> 00:35:01.780
F(64).
00:35:01.780 --> 00:35:06.530
The 64 by 64 Fourier
matrix is connected
00:35:06.530 --> 00:35:10.070
to two copies of F(32).
00:35:10.070 --> 00:35:13.120
Let me leave a little
space for the connection.
00:35:13.120 --> 00:35:15.990
So this is 64 by 64.
00:35:15.990 --> 00:35:18.980
Here's a matrix
of that same size,
00:35:18.980 --> 00:35:25.640
because it's got two copies of
F(32) and two zero matrixes.
00:35:25.640 --> 00:35:31.500
Those zero matrixes are the key,
because when I multiply by this
00:35:31.500 --> 00:35:35.570
matrix, just as it is, regular
multiplication, I would take --
00:35:35.570 --> 00:35:37.160
need 64 --
00:35:37.160 --> 00:35:40.320
I would -- I would have 64
squared little multiplications
00:35:40.320 --> 00:35:41.470
to do.
00:35:41.470 --> 00:35:45.000
But this matrix is half zero.
00:35:45.000 --> 00:35:46.600
Well, of course, the
two aren't equal.
00:35:46.600 --> 00:35:52.010
I'm going to put an equals sign,
but there has to be some fix up
00:35:52.010 --> 00:35:53.570
factors --
00:35:53.570 --> 00:35:57.640
one there and one there --
00:35:57.640 --> 00:36:00.620
to make it true.
00:36:00.620 --> 00:36:05.260
The beauty is that these fix
up factors will be really --
00:36:05.260 --> 00:36:07.550
almost all zeroes.
00:36:07.550 --> 00:36:11.470
So that as soon as we
get this formula right,
00:36:11.470 --> 00:36:15.670
we've got a great idea for
how to get from the sixty-
00:36:15.670 --> 00:36:21.180
from the 64 squared calculations
-- so this original --
00:36:21.180 --> 00:36:25.110
originally we have 64 squared
calculations from there,
00:36:25.110 --> 00:36:29.570
but this one will give us
-- this is -- this will --
00:36:29.570 --> 00:36:34.660
we don't need that many -- we
only need two times 32 squared,
00:36:34.660 --> 00:36:38.120
because we've got that twice.
00:36:38.120 --> 00:36:42.440
And -- plus the fix-up.
00:36:42.440 --> 00:36:47.570
So I have to tell you what's
in this fix-up matrix.
00:36:47.570 --> 00:36:50.370
The one on the right is
actually a permutation matrix,
00:36:50.370 --> 00:36:55.820
a very simple odds and evens
permutation matrix, the --
00:36:55.820 --> 00:36:58.260
ones show up --
00:36:58.260 --> 00:37:00.820
I haven't put enough
ones, I really need a --
00:37:00.820 --> 00:37:05.106
32 of these guys at --
double space and then --
00:37:09.210 --> 00:37:12.400
you see it's -- it's
a permutation matrix.
00:37:12.400 --> 00:37:14.070
What it does --
00:37:14.070 --> 00:37:18.430
shall I call it P for
permutation matrix?
00:37:18.430 --> 00:37:23.920
So what that P does when
it multiplies a vector,
00:37:23.920 --> 00:37:28.770
it takes the odd -- the even
numbered components first
00:37:28.770 --> 00:37:30.580
and then the odds.
00:37:30.580 --> 00:37:33.640
You see this -- this one
skipping every time is going
00:37:33.640 --> 00:37:40.260
to pick out x0, x2, x4, x6 and
then below that will come --
00:37:40.260 --> 00:37:42.335
will pick out x1, x3, x5.
00:37:45.850 --> 00:37:49.650
And of course, that can be
hard wired in the computer
00:37:49.650 --> 00:37:52.350
to be instantaneous.
00:37:52.350 --> 00:37:56.570
So that says -- so
far, what have we said?
00:37:56.570 --> 00:38:00.960
We're saying that the 64 by
64 Fourier matrix is really
00:38:00.960 --> 00:38:04.540
separated into -- separate
your vector into the odd --
00:38:04.540 --> 00:38:07.430
into the even components
and the odd components,
00:38:07.430 --> 00:38:12.820
then do a 32 size Fourier
transform onto those
00:38:12.820 --> 00:38:17.420
separately, and then put
the pieces together again.
00:38:17.420 --> 00:38:21.890
So the pieces -- putting them
together turns them out to be I
00:38:21.890 --> 00:38:25.350
and a diagonal matrix
and I and a minus,
00:38:25.350 --> 00:38:26.615
that same diagonal matrix.
00:38:29.330 --> 00:38:33.540
So the fix-up cost is really
the cost of multiplying by D,
00:38:33.540 --> 00:38:38.120
this diagonal matrix, because
there's essentially no cost
00:38:38.120 --> 00:38:39.020
in --
00:38:39.020 --> 00:38:42.630
in the I part or in
the permutation part,
00:38:42.630 --> 00:38:50.720
so really it's -- the fix-up
cost is essentially because D
00:38:50.720 --> 00:38:52.950
is diagonal --
00:38:52.950 --> 00:38:54.160
is 32 multiplications.
00:38:58.930 --> 00:39:00.430
That's the -- there
you're seeing --
00:39:00.430 --> 00:39:02.888
of course we didn't check the
formula or we didn't even say
00:39:02.888 --> 00:39:05.030
what D is yet, but I will --
00:39:05.030 --> 00:39:08.900
this diagonal matrix
D is powers of W --
00:39:08.900 --> 00:39:13.345
one W W squared down
to W to the 31st.
00:39:19.720 --> 00:39:22.630
So you see that when I --
to do a multiplication by D,
00:39:22.630 --> 00:39:25.920
I need to do 32 multiplications.
00:39:25.920 --> 00:39:27.780
There they are.
00:39:27.780 --> 00:39:33.030
Then -- but the other, the more
serious work is to do the F(32)
00:39:33.030 --> 00:39:34.910
twice on the --
00:39:34.910 --> 00:39:37.300
separately on the
even numbered and odd
00:39:37.300 --> 00:39:41.600
numbered components,
so twice 32 squared.
00:39:41.600 --> 00:39:45.970
So 64 squared is gone now.
00:39:45.970 --> 00:39:47.990
And that's the new count.
00:39:50.580 --> 00:39:54.500
Okay, great, but what next?
00:39:54.500 --> 00:39:58.730
So that's -- I -- we
now have the key idea --
00:39:58.730 --> 00:40:01.110
we would have to
check the algebra,
00:40:01.110 --> 00:40:06.770
but it's just checking a lot of
sums that come out correctly.
00:40:06.770 --> 00:40:10.290
This is right -- the right
way to see the fast Fourier
00:40:10.290 --> 00:40:15.010
transform, or one
right way to see it.
00:40:15.010 --> 00:40:17.240
Then you've got to see
what's the next idea.
00:40:17.240 --> 00:40:21.390
The next idea is to
break the 32s down.
00:40:21.390 --> 00:40:23.450
Break those 32s down.
00:40:23.450 --> 00:40:27.430
So we have this factor,
and now we have the F(32),
00:40:27.430 --> 00:40:32.630
but that breaks into
some guy here --
00:40:32.630 --> 00:40:36.510
F thirty- F six- F(16) --
00:40:36.510 --> 00:40:37.610
F(16).
00:40:37.610 --> 00:40:43.320
Each -- each F(32) is breaking
into two copies of F(16),
00:40:43.320 --> 00:40:47.950
and then we have a
permutation and then the --
00:40:47.950 --> 00:40:52.210
so this is a -- like, this
was a 64 size permutation,
00:40:52.210 --> 00:40:55.410
this is a 32 size permutation --
00:40:55.410 --> 00:40:57.252
I guess I've got it twice.
00:40:57.252 --> 00:40:57.960
So it's -- I'm --
00:40:57.960 --> 00:41:01.120
I'm just using the same
idea recursively --
00:41:01.120 --> 00:41:05.270
recursion is the key word --
that on each of those F(32)s --
00:41:05.270 --> 00:41:09.465
so here's zero zero -- it's
just -- to get F(32) --
00:41:13.810 --> 00:41:16.980
this is the odd
even permutations --
00:41:16.980 --> 00:41:20.580
so you see, we're --
the combination of those
00:41:20.580 --> 00:41:24.210
permutations, what's it doing?
00:41:24.210 --> 00:41:28.070
This guy separates into odds
-- in -- into evens and odds,
00:41:28.070 --> 00:41:34.160
and then this guy separates
the evens into the ones --
00:41:34.160 --> 00:41:37.430
the numbers that are
mult- the even evens,
00:41:37.430 --> 00:41:41.080
which means zero
four eight sixteen --
00:41:41.080 --> 00:41:48.580
and even odds, which means
two, six, ten, fourteen --
00:41:48.580 --> 00:41:52.240
and then odd evens and odd odds.
00:41:52.240 --> 00:41:55.930
You see, together these
permutations then break it --
00:41:55.930 --> 00:42:00.630
break our vector down into
x, even even and three other
00:42:00.630 --> 00:42:02.090
pieces.
00:42:02.090 --> 00:42:05.070
Those are the four pieces
that separately get multiplied
00:42:05.070 --> 00:42:06.650
by F(16) --
00:42:06.650 --> 00:42:12.808
separately fixed up by these Is
and Ds and Is and minus Ds --
00:42:15.680 --> 00:42:19.340
so this count is now reduced.
00:42:19.340 --> 00:42:23.220
This count is now --
what's it reduced to?
00:42:23.220 --> 00:42:28.490
So that's going to be gone,
because 32 squared -- that's --
00:42:28.490 --> 00:42:30.680
that's the change
I'm making, right?
00:42:30.680 --> 00:42:35.130
The 32 squared -- w- so -- so
it's this that's now reduced.
00:42:35.130 --> 00:42:40.150
So I still have two times it,
but now what's 32 squared?
00:42:40.150 --> 00:42:47.710
It's gone in favor of two
sixteen squareds plus sixteen.
00:42:47.710 --> 00:42:54.055
That's -- and then the
original 32 to fix.
00:43:00.170 --> 00:43:01.910
Maybe you see what's happening.
00:43:01.910 --> 00:43:04.580
Even easier than this
formula is w- what's --
00:43:04.580 --> 00:43:09.410
when I do the recursion
more and more times,
00:43:09.410 --> 00:43:12.670
I get simpler and simpler
factors in the middle.
00:43:12.670 --> 00:43:15.330
Eventually I'll be down
to two point or one point
00:43:15.330 --> 00:43:18.320
Fourier transforms.
00:43:18.320 --> 00:43:21.170
But I get more and more
factors piling up on the right
00:43:21.170 --> 00:43:21.720
and left.
00:43:21.720 --> 00:43:23.650
On the right, I'm just
getting permutation
00:43:23.650 --> 00:43:24.450
matrixes.
00:43:24.450 --> 00:43:29.300
On the left, I'm getting
these guys, these Is and Ds,
00:43:29.300 --> 00:43:32.490
so that there was
a 32 there and --
00:43:32.490 --> 00:43:35.320
each one of these is costing 32.
00:43:35.320 --> 00:43:39.330
Each one of those is costing
And how many will there be?
00:43:42.450 --> 00:43:45.080
So you see the 32
for this original fix
00:43:45.080 --> 00:43:49.850
up, because D had 32
numbers, 32 for this next fix
00:43:49.850 --> 00:43:54.750
up, because D has
16 and 16 more.
00:43:54.750 --> 00:43:56.920
I keep going.
00:43:56.920 --> 00:44:00.320
So the count in the
middle goes down to zip,
00:44:00.320 --> 00:44:04.080
but these fix up counts
are all that I'm left with,
00:44:04.080 --> 00:44:10.820
and how many factors -- how many
fix-ups have I got -- log in --
00:44:10.820 --> 00:44:15.740
from 64, one step to 32, one
step to 16, one step to eight,
00:44:15.740 --> 00:44:17.660
four, two and one.
00:44:17.660 --> 00:44:19.030
Six steps.
00:44:19.030 --> 00:44:20.550
So I have six fix-up --
00:44:20.550 --> 00:44:23.540
six fix up factors.
00:44:23.540 --> 00:44:33.380
Finally I get to six times
the That's my final count.
00:44:33.380 --> 00:44:41.430
Instead of 64 squared, this is
log to the base two of 64 times
00:44:41.430 --> 00:44:42.960
64 --
00:44:42.960 --> 00:44:45.190
actually, half of 64.
00:44:45.190 --> 00:44:51.600
So actually, the final count is
n log to the base two of n --
00:44:51.600 --> 00:44:52.815
that's the 32 --
00:44:55.380 --> 00:44:57.430
a half.
00:44:57.430 --> 00:45:02.160
So can I put a box
around that wonderful,
00:45:02.160 --> 00:45:07.210
extremely important and
satisfying conclusion --
00:45:07.210 --> 00:45:12.400
that the fast Fourier transform
multiplies by an n by n matrix,
00:45:12.400 --> 00:45:16.110
but it does it not in n
squared steps, but in one
00:45:16.110 --> 00:45:19.100
half n log n steps.
00:45:19.100 --> 00:45:21.130
And if we just --
00:45:21.130 --> 00:45:25.070
complete by doing a
count, let's suppose --
00:45:25.070 --> 00:45:31.680
suppose -- a typical
case would be two to the
00:45:31.680 --> 00:45:32.180
tenth.
00:45:35.540 --> 00:45:39.760
Now n squared is
bigger than a million.
00:45:44.560 --> 00:45:47.910
So it's a thousand twenty four
times a thousand twenty four.
00:45:47.910 --> 00:45:51.390
But what is n --
what is one half --
00:45:51.390 --> 00:45:55.690
what is the new count,
done the right way?
00:45:55.690 --> 00:46:02.550
It's n -- the thousand
twenty four times one half,
00:46:02.550 --> 00:46:04.640
and what's the logarithm?
00:46:04.640 --> 00:46:06.180
It's ten.
00:46:06.180 --> 00:46:08.540
So times ten over two.
00:46:08.540 --> 00:46:12.880
So it's five times -- it's five
times a thousand twenty four,
00:46:12.880 --> 00:46:16.690
where this one was a thousand
twenty four times a thousand
00:46:16.690 --> 00:46:17.460
twenty four.
00:46:21.450 --> 00:46:25.470
We've reduced the calculation
by a factor of 200 just
00:46:25.470 --> 00:46:28.830
by factoring the
matrix properly.
00:46:28.830 --> 00:46:34.580
This was a thousand times n,
we're now down to five times n.
00:46:34.580 --> 00:46:38.570
So we can do 200
Fourier transforms,
00:46:38.570 --> 00:46:42.160
where before we
could do one, and
00:46:42.160 --> 00:46:45.720
in real scientific calculations
where Fourier transforms are
00:46:45.720 --> 00:46:48.200
happening all the
time, we're saving
00:46:48.200 --> 00:46:52.120
a factor of in one
of the major steps
00:46:52.120 --> 00:46:55.960
of modern scientific computing.
00:46:55.960 --> 00:46:58.280
So that's the idea of the
fast Fourier transform,
00:46:58.280 --> 00:47:00.020
and you see the
whole thing hinged
00:47:00.020 --> 00:47:07.360
on being a special matrix
with orthonormal columns.
00:47:07.360 --> 00:47:13.020
Okay, that's actually
it for complex numbers.
00:47:13.020 --> 00:47:16.960
I'm back next time really to --
00:47:16.960 --> 00:47:22.560
to real numbers,
eigenvalues and eigenvectors
00:47:22.560 --> 00:47:27.190
and the key idea of
positive definite matrixes
00:47:27.190 --> 00:47:28.790
is going to show up.
00:47:28.790 --> 00:47:30.660
What's a positive
definite matrix?
00:47:30.660 --> 00:47:32.550
And it's terrific
that this course
00:47:32.550 --> 00:47:35.520
is going to reach
positive definiteness,
00:47:35.520 --> 00:47:37.840
because those are
the matrixes that you
00:47:37.840 --> 00:47:40.320
see the most in applications.
00:47:40.320 --> 00:47:41.965
Okay, see you next time.
00:47:46.240 --> 00:47:47.790
Thanks.