WEBVTT
00:00:00.090 --> 00:00:02.500
The following content is
provided under a Creative
00:00:02.500 --> 00:00:04.019
Commons license.
00:00:04.019 --> 00:00:06.360
Your support will help
MIT OpenCourseWare
00:00:06.360 --> 00:00:10.730
continue to offer high-quality
educational resources for free.
00:00:10.730 --> 00:00:13.330
To make a donation or
view additional materials
00:00:13.330 --> 00:00:17.217
from hundreds of MIT courses,
visit MIT OpenCourseWare
00:00:17.217 --> 00:00:17.842
at ocw.mit.edu.
00:00:21.240 --> 00:00:24.360
ERIK DEMAINE: All right,
let's get started.
00:00:24.360 --> 00:00:25.510
I am Erik Demaine.
00:00:25.510 --> 00:00:27.130
You can call me Erik.
00:00:27.130 --> 00:00:29.020
Today we're going
to do another divide
00:00:29.020 --> 00:00:31.970
and conquer algorithm called
the fast Fourier transform.
00:00:31.970 --> 00:00:35.870
It's probably the most
taught algorithm at MIT.
00:00:35.870 --> 00:00:39.300
It's used in all
sorts of contexts,
00:00:39.300 --> 00:00:41.440
especially digital
signal processing
00:00:41.440 --> 00:00:45.046
like MP3 compression,
and all sorts of things.
00:00:45.046 --> 00:00:46.670
But we're going to
think about it today
00:00:46.670 --> 00:00:51.560
in the context of divide
and conquer and polynomials.
00:00:51.560 --> 00:00:55.540
So let me remind you-- I
mean, this class is all
00:00:55.540 --> 00:00:59.160
about polynomial time, but
usually with polynomial time
00:00:59.160 --> 00:01:01.250
we only care about
the lead term.
00:01:01.250 --> 00:01:04.250
Today-- and today
only, pretty much--
00:01:04.250 --> 00:01:09.670
we're going to be thinking about
all the terms in a polynomial.
00:01:09.670 --> 00:01:13.080
So I'll talk about
polynomials mostly a and b.
00:01:13.080 --> 00:01:15.480
You have a constant
term, and then
00:01:15.480 --> 00:01:22.170
a linear term, and a quadratic
term, and so on up to-- I
00:01:22.170 --> 00:01:24.160
will say that there are
n terms-- which means
00:01:24.160 --> 00:01:28.150
the last one is a n minus 1.
00:01:28.150 --> 00:01:32.320
Normally the degree the
polynomial here is n minus 1.
00:01:32.320 --> 00:01:35.819
I wish the degree was defined
to be n here, but whatever.
00:01:35.819 --> 00:01:37.860
That's-- I'm not-- I can't
change the definitions
00:01:37.860 --> 00:01:39.110
in algebra.
00:01:39.110 --> 00:01:42.060
So this is the
traditional algebraic way
00:01:42.060 --> 00:01:44.680
of thinking about a polynomial.
00:01:44.680 --> 00:01:48.640
Of course, you can write
it with summation notation.
00:01:48.640 --> 00:01:52.990
akx to the k, k
equals 0 to n minus 1.
00:01:52.990 --> 00:01:54.790
We'll jump back and
forth between them.
00:01:54.790 --> 00:01:59.350
I'm also going to introduce a
vector notation for polynomials
00:01:59.350 --> 00:02:06.770
because-- so the ai's are
real numbers, typically.
00:02:06.770 --> 00:02:09.620
We might change
that at some point,
00:02:09.620 --> 00:02:11.904
but usually-- a common
reason-- maybe you
00:02:11.904 --> 00:02:13.320
don't care much
about polynomials,
00:02:13.320 --> 00:02:15.260
but you definitely
care about vectors.
00:02:15.260 --> 00:02:17.180
Any kind of
one-dimensional data set
00:02:17.180 --> 00:02:20.780
is a string of real
numbers, like if you're
00:02:20.780 --> 00:02:22.530
sampling audio--
like right now we're
00:02:22.530 --> 00:02:25.240
recording this
microphone-- you're
00:02:25.240 --> 00:02:29.800
seeing lots of different--
the movement of the membrane
00:02:29.800 --> 00:02:32.120
in this microphone over time.
00:02:32.120 --> 00:02:35.875
You're sampling-- whatever--
40,000 times the second.
00:02:35.875 --> 00:02:37.500
Each one you're
measuring a real number
00:02:37.500 --> 00:02:38.860
about where that thing is.
00:02:38.860 --> 00:02:40.910
That is a sequence
of real numbers.
00:02:40.910 --> 00:02:43.610
Now, you can convert it into
a polynomial if you want.
00:02:43.610 --> 00:02:44.740
They're the same thing.
00:02:44.740 --> 00:02:46.990
x is not necessarily
meaningful here.
00:02:46.990 --> 00:02:49.770
We really care about
other coefficients.
00:02:49.770 --> 00:02:52.000
Now, given such a
polynomial there
00:02:52.000 --> 00:02:55.330
are three typical
things we want to do.
00:02:55.330 --> 00:02:58.670
So these are the
operations on polynomials.
00:03:04.730 --> 00:03:08.800
I'll say why we want
to do them in a second.
00:03:08.800 --> 00:03:11.740
So the obvious one, if
you do care about x,
00:03:11.740 --> 00:03:14.650
is some kind of evaluation.
00:03:14.650 --> 00:03:18.140
So maybe I give you
a polynomial-- a
00:03:18.140 --> 00:03:23.390
of x-- and I give you a
number-- let's call it x0--
00:03:23.390 --> 00:03:26.660
and I want to compute
what is a of x0.
00:03:26.660 --> 00:03:29.600
So if I plug in-- x here
is a general variable,
00:03:29.600 --> 00:03:31.490
but if I give you an
actual real number,
00:03:31.490 --> 00:03:36.060
say, for the x's, what
does that add up to?
00:03:39.070 --> 00:03:41.390
So, how would you
solve evaluation
00:03:41.390 --> 00:03:43.220
before we go to the
other operations?
00:03:48.230 --> 00:03:50.300
There's an obvious way.
00:03:50.300 --> 00:03:53.580
We, you know-- you compute
all the terms and add them up,
00:03:53.580 --> 00:03:59.190
but if you do that
naively, computing
00:03:59.190 --> 00:04:03.700
x to the k-- in this form,
maybe-- computing x to the k
00:04:03.700 --> 00:04:06.540
maybe takes k
multiplications, and so
00:04:06.540 --> 00:04:08.139
the total runtime
of the quadratic,
00:04:08.139 --> 00:04:09.430
but we can do better than that.
00:04:15.307 --> 00:04:17.640
I know it's 11 o'clock-- too
early in the morning think.
00:04:25.850 --> 00:04:26.654
Yeah.
00:04:26.654 --> 00:04:28.820
AUDIENCE: When you've already
calculated x to the i,
00:04:28.820 --> 00:04:31.030
you can calculate x to the
i plus 1 by multiplying it.
00:04:31.030 --> 00:04:31.390
ERIK DEMAINE: Good.
00:04:31.390 --> 00:04:33.230
Once you've computed
x to the k, you
00:04:33.230 --> 00:04:36.920
can compute x to the k plus
1 with one multiplication,
00:04:36.920 --> 00:04:42.150
and so you can compute all
x to n's in linear time
00:04:42.150 --> 00:04:48.300
and then you're basically
doing a dot product between x
00:04:48.300 --> 00:04:51.330
to the k's and the a vector.
00:04:51.330 --> 00:04:52.785
Cool, my first Frisbee.
00:04:56.510 --> 00:04:59.550
So, that's one way to do it.
00:04:59.550 --> 00:05:01.360
There's a slightly
slicker way to write
00:05:01.360 --> 00:05:02.850
it called Horner's
Rule, but it's
00:05:02.850 --> 00:05:04.641
doing exactly the same
thing that you said.
00:05:08.410 --> 00:05:11.950
It's a-- this is just a nice
algebraic way of writing it.
00:05:11.950 --> 00:05:18.580
a of x is the same as a0 plus
x times a1 plus x times a2
00:05:18.580 --> 00:05:21.340
plus, and so on.
00:05:21.340 --> 00:05:28.250
x times a n minus 1, and then
lots of-- close parentheses.
00:05:28.250 --> 00:05:31.380
So this is, of course,
equivalent to that expression
00:05:31.380 --> 00:05:36.440
by the law of distribution,
and this is essentially doing
00:05:36.440 --> 00:05:38.940
the x-products one at a time.
00:05:38.940 --> 00:05:41.890
So, this is clearly order n
additions and multiplication,
00:05:41.890 --> 00:05:45.530
so we get order n time.
00:05:45.530 --> 00:05:48.260
In this lecture,
time is the number
00:05:48.260 --> 00:05:50.065
of arithmetic operations.
00:05:50.065 --> 00:05:50.950
That's our model.
00:05:50.950 --> 00:05:54.850
Assume it takes constant
time to multiply or add
00:05:54.850 --> 00:05:55.675
two real numbers.
00:06:00.971 --> 00:06:01.470
OK.
00:06:04.550 --> 00:06:05.050
Cool.
00:06:05.050 --> 00:06:07.850
So, evaluation-- that's easy.
00:06:07.850 --> 00:06:10.330
Linear time, for today, is good.
00:06:10.330 --> 00:06:12.260
Quadratic is bad.
00:06:12.260 --> 00:06:14.200
We want to beat quadratic.
00:06:14.200 --> 00:06:16.690
OK, second thing you might
want to do with polynomials
00:06:16.690 --> 00:06:18.430
is add them.
00:06:18.430 --> 00:06:21.280
The third thing
is multiply them.
00:06:21.280 --> 00:06:25.220
So, we're given two
polynomials-- a of x and b
00:06:25.220 --> 00:06:29.950
of x-- and we want to compute
a new polynomial-- c of x--
00:06:29.950 --> 00:06:31.730
that is the summation.
00:06:31.730 --> 00:06:33.900
How do you define the summation?
00:06:33.900 --> 00:06:36.570
Well, you would like
c of x to equal a of x
00:06:36.570 --> 00:06:39.390
plus b of x for all x.
00:06:39.390 --> 00:06:41.010
That's the definition.
00:06:41.010 --> 00:06:43.570
Of course, we can do it
algebraically as well
00:06:43.570 --> 00:06:47.110
because these are
numbers in the end--
00:06:47.110 --> 00:06:49.610
for any x this
evaluates to a number--
00:06:49.610 --> 00:06:51.970
so if we add two
polynomials of this form--
00:06:51.970 --> 00:06:56.660
one with ai's, one with the b
i's-- all we're doing is adding
00:06:56.660 --> 00:06:59.470
corresponding ai's and b i's.
00:06:59.470 --> 00:07:02.180
So, this is easy.
00:07:02.180 --> 00:07:09.860
We just need ck to equal
ak plus bk for all k.
00:07:09.860 --> 00:07:14.050
So, again, linear
time-- no problem.
00:07:14.050 --> 00:07:22.450
Third operation is the exciting
one, the hard one to get good,
00:07:22.450 --> 00:07:26.840
otherwise this lecture would be
over in a couple more minutes.
00:07:26.840 --> 00:07:28.930
So multiplication-- same deal.
00:07:28.930 --> 00:07:31.730
We're given a of x
and b of x, and we
00:07:31.730 --> 00:07:36.640
want to convert that into
some c of x that, for all x,
00:07:36.640 --> 00:07:41.225
evaluates to the product
of those two polynomials.
00:07:44.720 --> 00:07:47.750
How do we do this?
00:07:47.750 --> 00:07:53.040
We can't just multiply
corresponding ak's and bk's.
00:07:53.040 --> 00:07:54.830
In fact, if you take
a big thing like this
00:07:54.830 --> 00:07:57.520
and you multiply it by
corresponding big thing-- eh,
00:07:57.520 --> 00:07:58.030
let's do it.
00:08:11.328 --> 00:08:12.453
This doesn't look like fun.
00:08:17.800 --> 00:08:19.020
We get-- let's see.
00:08:19.020 --> 00:08:22.560
So, the constant term is just
the product-- that's easy,
00:08:22.560 --> 00:08:25.070
the constant terms--
but then, if I
00:08:25.070 --> 00:08:28.360
take this product or this
product, I get linear terms.
00:08:28.360 --> 00:08:36.330
So it's going to be
a1b0 plus a0b1 times x,
00:08:36.330 --> 00:08:38.370
and then there's
a quadratic term
00:08:38.370 --> 00:08:44.960
which I get from-- switch
colors-- this and this
00:08:44.960 --> 00:08:46.170
and this.
00:08:46.170 --> 00:08:49.330
So there's three
things times x squared,
00:08:49.330 --> 00:08:51.280
and that's where I get tired.
00:08:51.280 --> 00:08:55.875
I'm going to switch to
the summation notation.
00:08:55.875 --> 00:08:58.500
I didn't go to high school, but
I assume in high school algebra
00:08:58.500 --> 00:09:00.720
you learn this.
00:09:00.720 --> 00:09:10.180
ck is the sum of
j equals 0 to k.
00:09:10.180 --> 00:09:13.770
ajbk minus j.
00:09:13.770 --> 00:09:17.180
That's the general form
because aj came from an x
00:09:17.180 --> 00:09:21.290
to the j term, bk minus j came
from x to the k minus j term.
00:09:21.290 --> 00:09:23.970
When you multiply those
together, you get x to the k,
00:09:23.970 --> 00:09:26.980
so this is the
coefficient of x to the k.
00:09:26.980 --> 00:09:28.830
Cool?
00:09:28.830 --> 00:09:31.060
So, that's what we'd
like to compute.
00:09:31.060 --> 00:09:34.340
Given a and b we want to
compute this polynomial c.
00:09:34.340 --> 00:09:35.280
How long does it take?
00:09:38.816 --> 00:09:41.780
We have to do this for all k.
00:09:41.780 --> 00:09:45.210
So, to compute the k-th
term takes order k time,
00:09:45.210 --> 00:09:46.660
so the total time is n squared.
00:09:52.370 --> 00:09:55.620
So, that's not good
for this lecture.
00:09:55.620 --> 00:09:57.240
We want to do better.
00:09:57.240 --> 00:10:03.920
In fact, today we
will achieve n log n.
00:10:06.560 --> 00:10:11.230
That's our goal-- polynomial
multiplication in n log n.
00:10:11.230 --> 00:10:14.120
Why do we care about
polynomial multiplication?
00:10:14.120 --> 00:10:16.770
Because it's equivalent
to another operation which
00:10:16.770 --> 00:10:20.900
we use all the time in
digital signal processing,
00:10:20.900 --> 00:10:23.530
image editing, all sorts
of different things,
00:10:23.530 --> 00:10:24.790
which is convolution.
00:10:32.910 --> 00:10:35.640
Convolution is usually thought
of as an operation on vectors.
00:10:44.450 --> 00:10:48.079
So, remember this
vector notation,
00:10:48.079 --> 00:10:50.120
where we're just thinking
about the coefficients.
00:10:50.120 --> 00:10:51.420
x's are kind of irrelevant.
00:10:51.420 --> 00:10:54.380
We're just thinking about
a sequence of real numbers.
00:10:54.380 --> 00:10:58.790
So, maybe that sequence of
real numbers for a represents
00:10:58.790 --> 00:10:59.930
waveform.
00:10:59.930 --> 00:11:03.080
Maybe this is the
audio I'm speaking now.
00:11:03.080 --> 00:11:05.940
And then I take
some other waveform.
00:11:05.940 --> 00:11:09.890
Here I have a Gaussian
function-- e to the minus
00:11:09.890 --> 00:11:15.080
x squared-- and I want to
take-- for all possible shifts
00:11:15.080 --> 00:11:17.290
of this Gaussian
I want to compute
00:11:17.290 --> 00:11:19.650
the dot product
between the blackboard
00:11:19.650 --> 00:11:21.325
and the piece of paper.
00:11:21.325 --> 00:11:22.950
That's some kind of
smoothing function,
00:11:22.950 --> 00:11:26.342
if I wanted to clean up
noise or something like that.
00:11:26.342 --> 00:11:28.550
You can do the same thing
on a two-dimensional image.
00:11:28.550 --> 00:11:30.060
It's a little harder
to think about,
00:11:30.060 --> 00:11:31.370
but you can map a
two-dimensional image
00:11:31.370 --> 00:11:32.750
to a one-dimensional vector.
00:11:32.750 --> 00:11:34.458
And you have a
two-dimensional Gaussian--
00:11:34.458 --> 00:11:36.920
if you ever do Gaussian
blur in Photoshop
00:11:36.920 --> 00:11:39.530
this is what you're
doing-- a convolution--
00:11:39.530 --> 00:11:43.910
and it's used to pretend that
your lens is out of focus
00:11:43.910 --> 00:11:45.640
when you do that.
00:11:45.640 --> 00:11:50.090
It's done in audio processing,
and all sorts of things.
00:11:50.090 --> 00:11:54.210
So, formally, you're
given two vectors
00:11:54.210 --> 00:11:57.480
and you want to take all
possible shifts of one vector
00:11:57.480 --> 00:12:00.200
and take the dot product
with the other one.
00:12:00.200 --> 00:12:01.880
I have that written down.
00:12:01.880 --> 00:12:07.140
I just-- dot product, same
thing as inner product,
00:12:07.140 --> 00:12:11.244
which just means multiply
corresponding positions
00:12:11.244 --> 00:12:11.910
and add them up.
00:12:22.260 --> 00:12:24.544
And, if you ignore
this minus sign,
00:12:24.544 --> 00:12:25.960
that's exactly
what this is doing.
00:12:25.960 --> 00:12:30.810
This is taking aj versus
bk, pretend it's plus j.
00:12:30.810 --> 00:12:34.650
So, that's the bj vector, but
with all possible shifts k.
00:12:34.650 --> 00:12:38.120
We compute this for all k.
00:12:38.120 --> 00:12:39.780
That's really cool.
00:12:39.780 --> 00:12:42.120
We're going to compute
it in n log n time.
00:12:42.120 --> 00:12:45.309
All different n shifts of b will
take the dot product with a.
00:12:45.309 --> 00:12:46.850
It's kind of magical
because it looks
00:12:46.850 --> 00:12:48.280
like you're doing
n squared work,
00:12:48.280 --> 00:12:50.030
but we will do it
in n log n time.
00:12:53.160 --> 00:12:56.210
The only issue is we
have to reverse b.
00:12:56.210 --> 00:12:58.809
Then the minus signs
turn into plus signs.
00:12:58.809 --> 00:13:00.350
And there's some
boundary conditions,
00:13:00.350 --> 00:13:01.710
but it's basically the same.
00:13:01.710 --> 00:13:03.860
If we can solve
polynomial multiplication,
00:13:03.860 --> 00:13:05.253
we can solve convolution.
00:13:05.253 --> 00:13:07.040
Cool?
00:13:07.040 --> 00:13:09.430
So, that's why we care
about multiplication.
00:13:12.040 --> 00:13:15.190
So, how are we
going to solve this?
00:13:15.190 --> 00:13:18.760
What I'd like to do is talk
about alternate representations
00:13:18.760 --> 00:13:19.650
of polynomials.
00:13:19.650 --> 00:13:21.270
That's the next thing here.
00:13:21.270 --> 00:13:22.730
We just did operations.
00:13:22.730 --> 00:13:25.140
Let's talk about
different representations.
00:13:25.140 --> 00:13:27.460
So, we talked about this
one representation--
00:13:27.460 --> 00:13:30.177
it's one way to
represent polynomial--
00:13:30.177 --> 00:13:31.260
but it's not the only one.
00:13:31.260 --> 00:13:32.310
You probably know others.
00:13:35.490 --> 00:13:43.890
So, on the one hand we
have representation a
00:13:43.890 --> 00:13:45.920
is a coefficient vector.
00:13:49.530 --> 00:13:52.280
So we can write down the ai's.
00:13:52.280 --> 00:13:56.900
That was just one way to
represent a polynomial.
00:13:56.900 --> 00:13:59.730
Can anyone give me another
way to represent a polynomial?
00:13:59.730 --> 00:14:01.420
I have two ways in mind.
00:14:01.420 --> 00:14:01.920
Yeah.
00:14:01.920 --> 00:14:03.174
AUDIENCE: Generating function?
00:14:03.174 --> 00:14:04.590
ERIK DEMAINE:
Generating function.
00:14:04.590 --> 00:14:10.150
Isn't that the same-- oh,
I guess in principle you
00:14:10.150 --> 00:14:12.900
could imagine writing a
recurrence on the generating
00:14:12.900 --> 00:14:14.320
function or something.
00:14:14.320 --> 00:14:14.970
It's plausible.
00:14:14.970 --> 00:14:16.480
In general, generating
functions are
00:14:16.480 --> 00:14:19.580
polynomials if you know
what that-- they are cool.
00:14:19.580 --> 00:14:21.430
So it doesn't quite
answer the question.
00:14:21.430 --> 00:14:21.930
Yeah?
00:14:21.930 --> 00:14:22.829
AUDIENCE: [INAUDIBLE]
00:14:22.829 --> 00:14:23.620
ERIK DEMAINE: Sure.
00:14:23.620 --> 00:14:24.690
AUDIENCE: Representation.
00:14:24.690 --> 00:14:25.523
ERIK DEMAINE: Sorry?
00:14:25.523 --> 00:14:26.272
AUDIENCE: Points?
00:14:26.272 --> 00:14:27.980
ERIK DEMAINE: Point
representation, yeah.
00:14:27.980 --> 00:14:30.860
I call them samples.
00:14:30.860 --> 00:14:36.740
I'm going to put that under
C. A bunch of samples, a bunch
00:14:36.740 --> 00:14:39.240
of points on the polynomial.
00:14:39.240 --> 00:14:52.050
So like xk, yk for-- how many
do we need-- I think n minus 1
00:14:52.050 --> 00:14:53.500
should do it.
00:14:53.500 --> 00:14:54.550
We'll check.
00:14:54.550 --> 00:14:55.780
Yep.
00:14:55.780 --> 00:15:00.700
And if we are told
that a of xk equals yk,
00:15:00.700 --> 00:15:07.450
and we're told that all
the xk's are distinct,
00:15:07.450 --> 00:15:10.860
then this uniquely
determines the polynomial.
00:15:10.860 --> 00:15:13.780
You have a degree n
minus one polynomial,
00:15:13.780 --> 00:15:15.680
and you have n samples.
00:15:15.680 --> 00:15:17.410
There's only one
polynomial that passes
00:15:17.410 --> 00:15:20.620
through all those points.
00:15:20.620 --> 00:15:24.890
It's a consequence of the
fundamental theorem of algebra
00:15:24.890 --> 00:15:26.490
that gives you uniqueness.
00:15:26.490 --> 00:15:30.097
Existence, I think, was
proved by Legendre in 1700s,
00:15:30.097 --> 00:15:33.059
1800s-- a long time ago.
00:15:33.059 --> 00:15:33.600
This is good.
00:15:33.600 --> 00:15:35.310
This is what we're going to use.
00:15:35.310 --> 00:15:36.603
There's another answer.
00:15:36.603 --> 00:15:39.970
I should give you
a Frisbee first.
00:15:39.970 --> 00:15:42.330
You look so excited.
00:15:42.330 --> 00:15:46.140
Just wait till I hit you,
then you'll be less excited.
00:15:46.140 --> 00:15:47.420
OK, so.
00:15:47.420 --> 00:15:50.780
Samples, coefficients,
anything else?
00:15:50.780 --> 00:15:51.280
Yeah.
00:15:51.280 --> 00:15:52.043
AUDIENCE: Roots?
00:15:52.043 --> 00:15:53.126
ERIK DEMAINE: Roots, yeah.
00:15:56.320 --> 00:15:58.790
Roots is the other
answer I was looking for,
00:15:58.790 --> 00:16:01.951
but it's not going to be so good
algorithmically, as we'll see.
00:16:01.951 --> 00:16:02.450
Sorry.
00:16:05.076 --> 00:16:06.700
So, I can give you
a sequence of roots.
00:16:06.700 --> 00:16:08.450
This is the fundamental
theorem of algebra
00:16:08.450 --> 00:16:11.590
that every polynomial
is uniquely determined
00:16:11.590 --> 00:16:12.980
by its set of roots.
00:16:12.980 --> 00:16:15.680
If you allow roots
with multiplicity then
00:16:15.680 --> 00:16:19.420
every polynomial of degree
n has exactly n roots.
00:16:19.420 --> 00:16:27.686
So, this would be some sequence
of r1 up to rn minus 1,
00:16:27.686 --> 00:16:29.810
and the polynomial would
be given as-- you actually
00:16:29.810 --> 00:16:36.920
need a constant multiplier--
but x minus r0, x minus r1.
00:16:36.920 --> 00:16:39.800
That would be polynomial.
00:16:39.800 --> 00:16:41.990
The trouble with roots
is that if I give you
00:16:41.990 --> 00:16:45.290
a coefficient vector and I
want to compute the roots,
00:16:45.290 --> 00:16:48.620
not only is it hard to do, it's
impossible to do in our model.
00:16:48.620 --> 00:16:51.020
If you're only allowed to
add, subtract, multiply,
00:16:51.020 --> 00:16:54.340
divide, take square
roots, take k-th roots,
00:16:54.340 --> 00:16:57.770
there is no way to solve
a polynomial of degree
00:16:57.770 --> 00:16:58.930
5 or larger.
00:16:58.930 --> 00:17:01.690
There's the quadratic formula,
cubic formula, quartic formula.
00:17:01.690 --> 00:17:03.060
There is no quintic formula.
00:17:03.060 --> 00:17:05.250
That's an old result. 1800s.
00:17:05.250 --> 00:17:08.420
So, going from coefficient
vector to roots
00:17:08.420 --> 00:17:10.119
takes infinite time.
00:17:10.119 --> 00:17:11.530
It's not so good.
00:17:11.530 --> 00:17:14.859
And, in particular, if we
think about our operations,
00:17:14.859 --> 00:17:18.000
addition becomes
really difficult.
00:17:18.000 --> 00:17:22.621
Multiplication is easy if I
have two polynomials represented
00:17:22.621 --> 00:17:23.579
as a sequence of roots.
00:17:23.579 --> 00:17:24.579
I want to multiply them.
00:17:24.579 --> 00:17:27.540
That's just concatenating
the vectors--
00:17:27.540 --> 00:17:31.676
taking union the vectors
of their root lists.
00:17:31.676 --> 00:17:32.300
So that's cool.
00:17:32.300 --> 00:17:34.150
I'm multiplying
the c's, I guess.
00:17:34.150 --> 00:17:36.190
But addition is really
hard because addition
00:17:36.190 --> 00:17:39.670
is sort of fundamentally
about coefficient vectors.
00:17:39.670 --> 00:17:42.455
And then, once
you go there-- you
00:17:42.455 --> 00:17:44.830
can go from roots to coefficient
vectors and add them up,
00:17:44.830 --> 00:17:46.746
but then there's no
relation between the roots
00:17:46.746 --> 00:17:49.193
of the sum of the
polynomials versus the roots
00:17:49.193 --> 00:17:51.975
of the original.
00:17:51.975 --> 00:17:53.850
I don't know for sure
that that's impossible.
00:17:53.850 --> 00:17:56.505
It's definitely very, very
hard, probably impossible.
00:18:02.200 --> 00:18:04.940
Let me draw a little table.
00:18:04.940 --> 00:18:08.170
Each of these representations
has some advantages
00:18:08.170 --> 00:18:12.700
and a disadvantage in terms
of these three operations.
00:18:12.700 --> 00:18:18.670
So, on the one hand,
we have the algorithms
00:18:18.670 --> 00:18:38.790
we care about-- evaluation,
addition, and multiplication--
00:18:38.790 --> 00:18:42.400
and on the other axis we have
our representations, which
00:18:42.400 --> 00:18:52.074
are coefficient vectors,
roots, and samples.
00:18:52.074 --> 00:18:53.990
You'll see why I chose
this order in a moment.
00:18:53.990 --> 00:18:55.448
It makes for a
nice, pretty matrix.
00:19:03.950 --> 00:19:07.260
We've talked about almost
every cell in this matrix,
00:19:07.260 --> 00:19:08.950
but let me just summarize.
00:19:08.950 --> 00:19:12.310
We started out just thinking
about coefficient vector,
00:19:12.310 --> 00:19:16.320
and evaluation was linear time.
00:19:16.320 --> 00:19:18.600
Addition was linear time.
00:19:18.600 --> 00:19:20.760
Multiplication, so
far, was quadratic,
00:19:20.760 --> 00:19:23.820
although our goal
is to make n log n.
00:19:23.820 --> 00:19:28.670
For roots, I just said
multiplication is easy.
00:19:28.670 --> 00:19:34.381
That's linear time,
addition is really hard--
00:19:34.381 --> 00:19:38.920
like, infinite time--
and evaluation,
00:19:38.920 --> 00:19:41.910
I guess that's linear time.
00:19:41.910 --> 00:19:43.560
In fact, the way it's
written, you only
00:19:43.560 --> 00:19:45.070
have a linear number
of subtractions
00:19:45.070 --> 00:19:49.330
and multiplications, so it's
really easy to evaluate.
00:19:49.330 --> 00:19:55.100
And then sample vectors-- we
haven't talked much about that.
00:19:55.100 --> 00:19:58.160
The idea is, suppose you're
given two polynomials with
00:19:58.160 --> 00:19:59.840
the same xk's.
00:19:59.840 --> 00:20:00.980
We're going to fix xk's.
00:20:00.980 --> 00:20:02.521
All we need is that
they're distinct.
00:20:02.521 --> 00:20:04.650
So xk could equal
k, for example,
00:20:04.650 --> 00:20:06.230
just a bunch of integers.
00:20:06.230 --> 00:20:10.270
And then we are told what
polynomial a evaluates
00:20:10.270 --> 00:20:12.330
to at every xk, and we're
told what polynomial
00:20:12.330 --> 00:20:14.940
b evaluates to at every xk.
00:20:14.940 --> 00:20:18.790
So, we're given some yk's
and some zk's, and then we
00:20:18.790 --> 00:20:23.970
want to compute, say, the sum
or the product of those two
00:20:23.970 --> 00:20:24.850
vectors.
00:20:24.850 --> 00:20:26.580
What do we do?
00:20:26.580 --> 00:20:32.030
Just add or multiply the
corresponding yk and zk's,
00:20:32.030 --> 00:20:36.290
because if we're
told we want c of x
00:20:36.290 --> 00:20:39.390
to equal a of x times b of
x, or c of x to equal a of x
00:20:39.390 --> 00:20:42.760
plus b of x for all x, Well, now
we know what x's we care about.
00:20:42.760 --> 00:20:44.600
We just do it at the xk's.
00:20:44.600 --> 00:20:46.590
That's what we're
told for a and for b,
00:20:46.590 --> 00:20:50.275
and so to compute c of
xk it's just the sum
00:20:50.275 --> 00:20:53.290
or the product of yk and zk.
00:20:53.290 --> 00:20:56.180
So multiplication is really
easy in the sample view,
00:20:56.180 --> 00:20:59.460
and this is why we are
going to use this view.
00:20:59.460 --> 00:21:02.270
We're also going to use this
view because, as we'll see,
00:21:02.270 --> 00:21:04.270
there's a problem.
00:21:04.270 --> 00:21:08.800
Addition is easy,
multiplication is easy,
00:21:08.800 --> 00:21:13.130
evaluation is annoying.
00:21:13.130 --> 00:21:19.010
I can evaluate a of
x at xk for any k,
00:21:19.010 --> 00:21:23.880
but I can't evaluate it at
some arbitrary value of x.
00:21:23.880 --> 00:21:25.500
That's annoying.
00:21:25.500 --> 00:21:27.210
I'm told at these
finite sample points,
00:21:27.210 --> 00:21:29.100
but now I have to
somehow interpolate.
00:21:29.100 --> 00:21:31.170
This is called
polynomial interpolation,
00:21:31.170 --> 00:21:33.620
well studied in numerical
analysis and so on.
00:21:33.620 --> 00:21:38.960
You can do it, but it takes
quadratic time, in general.
00:21:38.960 --> 00:21:40.800
The best known
algorithms are quadratic.
00:21:43.890 --> 00:21:52.020
So, this is bad, this
is bad, and this is bad,
00:21:52.020 --> 00:21:56.380
so no representation is perfect.
00:21:56.380 --> 00:21:59.100
Life sucks.
00:21:59.100 --> 00:22:00.789
What we'd like is to
get the best-- now
00:22:00.789 --> 00:22:03.330
this one is really hard to work
with because converting inter
00:22:03.330 --> 00:22:08.210
roots is impossible in
an arithmetic model,
00:22:08.210 --> 00:22:11.530
so we're going to focus
on column A and column
00:22:11.530 --> 00:22:15.860
C. We kind of like to take
the min of those two columns.
00:22:15.860 --> 00:22:17.440
We won't quite get that.
00:22:17.440 --> 00:22:20.620
What we will get is an
algorithm for converting
00:22:20.620 --> 00:22:23.720
between these two
representations in n log n
00:22:23.720 --> 00:22:28.110
time-- it's not quite
linear, but close--
00:22:28.110 --> 00:22:31.080
and once we can do that,
if we want to multiply two
00:22:31.080 --> 00:22:35.090
things in the coefficient land
we can convert to sample land,
00:22:35.090 --> 00:22:38.070
do it in linear time,
and then convert back.
00:22:38.070 --> 00:22:41.230
So that's the magical
transformation
00:22:41.230 --> 00:22:43.861
we're going to cover, and it
is called the fast Fourier
00:22:43.861 --> 00:22:44.360
transfer.
00:22:44.360 --> 00:22:46.390
Fast Fourier transform
is the algorithm.
00:22:46.390 --> 00:22:49.691
Discrete Fourier transform
is that transformation
00:22:49.691 --> 00:22:50.316
mathematically.
00:22:53.172 --> 00:22:54.150
Cool.
00:22:54.150 --> 00:22:56.990
So the whole name of
the game is converting
00:22:56.990 --> 00:23:00.720
from coefficient representation
to samples, or vice versa.
00:23:00.720 --> 00:23:02.180
Turns out they're
almost the same,
00:23:02.180 --> 00:23:04.150
though that won't be
obvious for a long time--
00:23:04.150 --> 00:23:05.920
till the end of the class.
00:23:05.920 --> 00:23:08.834
Any questions before we proceed?
00:23:08.834 --> 00:23:09.808
Yeah.
00:23:09.808 --> 00:23:11.680
AUDIENCE: [INAUDIBLE]
multiply repetitions,
00:23:11.680 --> 00:23:15.564
why not we evaluate a and
b first then multiply?
00:23:15.564 --> 00:23:16.550
ERIK DEMAINE: Ah.
00:23:16.550 --> 00:23:20.700
So, OK, the question
is, if I want
00:23:20.700 --> 00:23:22.730
to-- we'll get
there in a second--
00:23:22.730 --> 00:23:25.110
but if I want to
multiply, and it's
00:23:25.110 --> 00:23:28.480
so easy to do in sample land,
why don't I just sample a and b
00:23:28.480 --> 00:23:29.550
and then multiply them?
00:23:29.550 --> 00:23:32.690
That's right, but [? effect ?]
sampling is not so easy.
00:23:32.690 --> 00:23:34.520
It takes quadratic time.
00:23:34.520 --> 00:23:35.770
Let's go there now.
00:23:43.290 --> 00:23:47.950
Because we have n samples to do,
each one will cost linear time.
00:23:47.950 --> 00:23:51.160
Remember, to evaluate a
polynomial takes linear time.
00:23:51.160 --> 00:23:53.600
If you want to think
of it in a matrix--
00:23:53.600 --> 00:24:00.725
let's enter the matrix--
then we get a big matrix.
00:24:13.920 --> 00:24:16.510
So we're given the
xi's and we just
00:24:16.510 --> 00:24:20.120
want to evaluate a given
polynomial whose coefficients
00:24:20.120 --> 00:24:27.060
are given by a0,
a1, a2, and minus 1.
00:24:27.060 --> 00:24:33.310
Our goal is to compute
the yi's-- y0, y1, y2,
00:24:33.310 --> 00:24:38.880
to yn minus 1-- and if you
know a matrix-vector product,
00:24:38.880 --> 00:24:40.974
you take this row
with that column.
00:24:40.974 --> 00:24:42.640
You take the dot
product that multiplies
00:24:42.640 --> 00:24:44.110
corresponding entries.
00:24:44.110 --> 00:24:45.130
You get y0.
00:24:45.130 --> 00:24:47.960
That is the definition of
the polynomial evaluation.
00:24:47.960 --> 00:24:51.342
I'm just going to write
a bunch of these rows
00:24:51.342 --> 00:24:52.300
so you get the pattern.
00:24:58.527 --> 00:25:01.200
Pretty simple.
00:25:09.580 --> 00:25:13.270
This is called the
Vandermonde matrix.
00:25:20.560 --> 00:25:26.698
I'll call it V. And in general--
I don't have room for it,
00:25:26.698 --> 00:25:27.281
so let me go--
00:25:40.560 --> 00:25:47.260
In general, if we look at
V ij, it's just-- sorry.
00:25:47.260 --> 00:25:49.800
You may notice I'm not
using the letter i.
00:25:49.800 --> 00:25:52.090
We will get to why in a moment.
00:25:52.090 --> 00:25:53.860
V jk.
00:25:53.860 --> 00:25:55.960
Row j, column k.
00:25:55.960 --> 00:25:59.670
That's going to be x
sub j to the power k.
00:25:59.670 --> 00:26:02.710
That's the Vandermonde matrix.
00:26:02.710 --> 00:26:04.599
We can compute it
in quadratic time.
00:26:04.599 --> 00:26:05.640
It has quadratic entries.
00:26:05.640 --> 00:26:07.730
We can use the trick
we suggested earlier--
00:26:07.730 --> 00:26:10.590
compute each term
from the previous one
00:26:10.590 --> 00:26:14.470
by multiplying by
xj-- and then we
00:26:14.470 --> 00:26:17.910
want to compute this
matrix-vector product,
00:26:17.910 --> 00:26:20.820
and you can clearly do
it in quadratic time--
00:26:20.820 --> 00:26:23.800
I'm just computing each
thing correspondingly--
00:26:23.800 --> 00:26:25.310
and that's sort of
the best you can
00:26:25.310 --> 00:26:26.790
do without any
further assumptions.
00:26:26.790 --> 00:26:30.090
So this takes-- if
I want to compute
00:26:30.090 --> 00:26:35.770
this product, that's the
coefficients to samples
00:26:35.770 --> 00:26:38.210
problem.
00:26:38.210 --> 00:26:45.860
This is the same thing as
computing V times the A vector,
00:26:45.860 --> 00:26:48.440
so this is a matrix-vector
multiplication,
00:26:48.440 --> 00:26:50.630
which takes n squared time.
00:26:54.100 --> 00:26:55.000
OK?
00:26:55.000 --> 00:26:58.110
On the other hand--
so, that's a problem
00:26:58.110 --> 00:27:00.730
because we're trying to beat
quadratic multiplication,
00:27:00.730 --> 00:27:02.880
so if we spend quadratic
time to convert over here
00:27:02.880 --> 00:27:04.740
it doesn't matter if
this is linear time.
00:27:04.740 --> 00:27:05.450
There are two problems.
00:27:05.450 --> 00:27:07.033
One is that conversion
costs too much.
00:27:07.033 --> 00:27:09.970
The other is we don't yet
know how to convert backwards.
00:27:09.970 --> 00:27:11.860
But this matrix
field gives us also
00:27:11.860 --> 00:27:13.620
the reverse transformation.
00:27:13.620 --> 00:27:21.310
If we want to convert
samples to coefficients,
00:27:21.310 --> 00:27:26.190
this is-- the best notation
I know is from MATLAB.
00:27:26.190 --> 00:27:28.036
How many people know MATLAB?
00:27:28.036 --> 00:27:28.820
A bunch.
00:27:28.820 --> 00:27:32.780
So for you, it's V
backslash A, but usually
00:27:32.780 --> 00:27:34.760
in linear algebra
like 18.06 you see
00:27:34.760 --> 00:27:40.540
you have some matrix V
times some unknown vector--
00:27:40.540 --> 00:27:42.525
usually it's called x,
here it's called a--
00:27:42.525 --> 00:27:44.380
and you know the
right-hand side.
00:27:44.380 --> 00:27:45.710
You want to solve for this.
00:27:45.710 --> 00:27:46.420
How do you do it?
00:27:49.619 --> 00:27:50.990
AUDIENCE: [INAUDIBLE]
00:27:50.990 --> 00:27:52.250
ERIK DEMAINE: Sorry?
00:27:52.250 --> 00:27:53.930
AUDIENCE: Multiply
by the inverse?
00:27:53.930 --> 00:27:55.050
ERIK DEMAINE: Multiply
by the inverse.
00:27:55.050 --> 00:27:55.549
Yeah.
00:27:55.549 --> 00:27:57.350
How do you do it in
computer science?
00:27:57.350 --> 00:27:58.130
AUDIENCE: Gaussian elimination.
00:27:58.130 --> 00:27:59.250
ERIK DEMAINE:
Gaussian elimination.
00:27:59.250 --> 00:28:01.215
Turns out inverse is
the right answer here,
00:28:01.215 --> 00:28:04.020
but Gaussian elimination
would be the standard way
00:28:04.020 --> 00:28:06.360
to solve a linear
system like that.
00:28:06.360 --> 00:28:09.250
The trouble with
Gaussian elimination
00:28:09.250 --> 00:28:16.080
is it takes cubic time
in its normal form.
00:28:16.080 --> 00:28:17.780
In this case it's a
little bit special
00:28:17.780 --> 00:28:19.830
because this matrix
is essentially fixed.
00:28:19.830 --> 00:28:22.130
The xi's don't need to change.
00:28:22.130 --> 00:28:26.720
It could be xi is just i
or something, so we can--
00:28:26.720 --> 00:28:28.160
in this case it's
a little better
00:28:28.160 --> 00:28:30.320
to compute the inverse first.
00:28:30.320 --> 00:28:35.451
So we could also just
do v inverse times a.
00:28:35.451 --> 00:28:37.700
From a numerical analysis
standpoint this is very bad,
00:28:37.700 --> 00:28:39.120
but don't worry
about it for now.
00:28:39.120 --> 00:28:41.070
We're going to get a
better algorithm today.
00:28:41.070 --> 00:28:43.780
Anyway, it doesn't
involve matrices at all,
00:28:43.780 --> 00:28:45.980
but the nice thing
is, if computing
00:28:45.980 --> 00:28:47.657
the inverse, that
takes n cubed time,
00:28:47.657 --> 00:28:48.990
but you only have to do it once.
00:28:48.990 --> 00:28:51.150
So if you have to
do this many times
00:28:51.150 --> 00:28:53.840
you can do this product
in n squared time.
00:28:53.840 --> 00:28:59.300
You just have to maintain that
v inverse once and for all.
00:28:59.300 --> 00:29:00.250
OK, great.
00:29:00.250 --> 00:29:01.820
So, we've got
quadratic algorithms
00:29:01.820 --> 00:29:03.960
to go back and forth between
these representations.
00:29:03.960 --> 00:29:06.110
That, at least,
tells us it's doable,
00:29:06.110 --> 00:29:09.600
but we, of course,
need better than
00:29:09.600 --> 00:29:14.526
quadratic to improve on the
naive multiplication algorithm,
00:29:14.526 --> 00:29:15.900
so that's what
we're going to do.
00:29:18.740 --> 00:29:20.730
In general, we
can't do any better,
00:29:20.730 --> 00:29:24.960
but we have one freedom,
which is we have said nothing
00:29:24.960 --> 00:29:26.430
about the x case.
00:29:26.430 --> 00:29:29.770
We can choose them to be
whatever we want them to be.
00:29:29.770 --> 00:29:31.800
I keep saying xk equals k.
00:29:31.800 --> 00:29:32.930
That seems fine.
00:29:32.930 --> 00:29:36.200
It's actually really bad choice
for a reason we will get to,
00:29:36.200 --> 00:29:39.210
but there is a choice where,
magically, this transformation
00:29:39.210 --> 00:29:43.910
becomes easy and you can
do it in n log n time.
00:29:43.910 --> 00:29:48.310
Before we get there,
I want to give you
00:29:48.310 --> 00:29:54.160
some motivation for how
this could possibly work.
00:30:15.140 --> 00:30:18.680
As you might expect, even just
from that n log n running time,
00:30:18.680 --> 00:30:23.930
we're going to be using
divide and conquer,
00:30:23.930 --> 00:30:28.780
so let's just think about how
divide and conquer could work.
00:30:28.780 --> 00:30:31.190
I'm going to show
you an idea and then
00:30:31.190 --> 00:30:33.760
we'll figure out how that
idea could possibly work.
00:30:33.760 --> 00:30:36.190
It doesn't work at
the moment, but we
00:30:36.190 --> 00:30:40.210
will be able to choose
the xk so that it works.
00:30:40.210 --> 00:30:45.490
So, let's say the goal-- I mean,
what we want to do is compute
00:30:45.490 --> 00:30:47.880
this v times a.
00:30:47.880 --> 00:30:54.110
I'm going to convert-- think of
that back into polynomial land.
00:30:54.110 --> 00:31:01.270
So our goal is to compute a
of x for all x in some set x.
00:31:05.050 --> 00:31:10.420
This is taking a
bunch of samples.
00:31:10.420 --> 00:31:13.580
Set x is just a set
of the xk's, but I'm
00:31:13.580 --> 00:31:19.410
going to change that set in
a moment using recursion.
00:31:19.410 --> 00:31:22.390
So the input to this algorithm
is a polynomial a of x,
00:31:22.390 --> 00:31:26.520
and it's a set
capital X of positions
00:31:26.520 --> 00:31:28.420
that I'd like to evaluate
that polynomial at.
00:31:28.420 --> 00:31:30.295
This is clearly more
general than the problem
00:31:30.295 --> 00:31:33.450
we're trying to solve, and I'm
going to solve it with divide
00:31:33.450 --> 00:31:33.950
and conquer.
00:31:33.950 --> 00:31:35.783
In divide and conquer
there are three steps.
00:31:35.783 --> 00:31:39.440
Divide, conquer, and combine.
00:31:39.440 --> 00:31:40.640
Let's start with divide.
00:31:43.660 --> 00:31:44.610
Here's the big idea.
00:31:47.997 --> 00:31:49.580
I would say there
are two natural ways
00:31:49.580 --> 00:31:50.500
to divide a vector.
00:31:50.500 --> 00:31:52.470
One is in the middle,
that's what we've always
00:31:52.470 --> 00:31:58.390
seen with merge sort and
convex hull from last time,
00:31:58.390 --> 00:32:01.000
but there's another way
which will work better here,
00:32:01.000 --> 00:32:05.480
which is the even entries
and the odd entries.
00:32:05.480 --> 00:32:08.820
So I'm going to divide into
even and odd coefficients.
00:32:15.410 --> 00:32:18.150
Let me write that down.
00:32:18.150 --> 00:32:21.860
One of them is called a sub
even of x-- that's a polynomial.
00:32:21.860 --> 00:32:24.440
It's going to have half
the degree, so it's
00:32:24.440 --> 00:32:31.750
going to be sum from k
equals 0 to-- I wrote n here,
00:32:31.750 --> 00:32:36.150
but I think I want something
like n over 2 minus 1,
00:32:36.150 --> 00:32:45.910
n minus 1 over 2, one of those
things-- of a to k x to the k.
00:32:45.910 --> 00:32:49.470
So, really, what I want--
which is easier to write-- is,
00:32:49.470 --> 00:32:54.762
in the vector notation, I
want all the even entries.
00:32:54.762 --> 00:32:56.720
I won't try to figure
out what the last one is,
00:32:56.720 --> 00:32:58.120
but it's roughly n over 2-ish.
00:33:00.750 --> 00:33:02.220
I'll just write that.
00:33:02.220 --> 00:33:03.470
You can go a little bit extra.
00:33:03.470 --> 00:33:07.210
It's fine if you define a sub
n to be 0, and a sub n plus 1
00:33:07.210 --> 00:33:10.700
to be zero, and all those to be
0, those terms will disappear.
00:33:10.700 --> 00:33:13.170
So the key thing here is
I'm taking the even entries,
00:33:13.170 --> 00:33:18.040
but I don't have 2k up here.
00:33:18.040 --> 00:33:20.640
This is the x to the 0 term.
00:33:20.640 --> 00:33:21.930
This is the x to the 1 term.
00:33:21.930 --> 00:33:24.020
This is the x to the 2 term.
00:33:24.020 --> 00:33:29.601
So there's a difference between
x to the k's and the sub 2k,
00:33:29.601 --> 00:33:31.600
but, I mean, just think
about it in vector form.
00:33:31.600 --> 00:33:33.620
Don't worry about
the algebra for now.
00:33:33.620 --> 00:33:35.703
We're going to have to
worry about it in a second,
00:33:35.703 --> 00:33:38.050
but, intuitively,
what I want to do
00:33:38.050 --> 00:33:42.620
is extract from the vector
of all the ai's these two
00:33:42.620 --> 00:33:48.090
vectors-- the odd
coefficients in order
00:33:48.090 --> 00:33:50.830
and the even
coefficients in order--
00:33:50.830 --> 00:33:53.500
but I'm going to need the
algebraic form in a moment
00:33:53.500 --> 00:33:54.750
for the combined step.
00:33:58.260 --> 00:34:01.950
It should be 2k plus 1 x to k.
00:34:06.684 --> 00:34:07.350
That's step one.
00:34:10.290 --> 00:34:10.812
Easy to do.
00:34:10.812 --> 00:34:11.770
Linear time, of course.
00:34:14.510 --> 00:34:16.510
Let's jump ahead to
step three, combine.
00:34:20.739 --> 00:34:24.560
In order to compute
a of x from-- what
00:34:24.560 --> 00:34:26.690
I'd like to do is
recursively compute a even
00:34:26.690 --> 00:34:29.082
of x and a odd of x
for some values x.
00:34:29.082 --> 00:34:30.290
It's not going to be x and x.
00:34:30.290 --> 00:34:32.469
It's going to be some other set.
00:34:32.469 --> 00:34:35.300
Let's think about
how I can compute
00:34:35.300 --> 00:34:40.739
a of x given some solutions
to a even of x and a odd of x.
00:34:40.739 --> 00:34:49.604
So this is step three, combine.
00:34:52.780 --> 00:35:00.040
So I would like
a of x over here,
00:35:00.040 --> 00:35:10.880
and I want a even of something
and a odd of something
00:35:10.880 --> 00:35:13.330
and something in here.
00:35:13.330 --> 00:35:15.680
Anyone see the algebra?
00:35:19.260 --> 00:35:20.190
Maybe start with this?
00:35:25.130 --> 00:35:26.366
I hear a mumble.
00:35:26.366 --> 00:35:26.866
Yeah.
00:35:26.866 --> 00:35:27.360
AUDIENCE: x squared.
00:35:27.360 --> 00:35:28.250
ERIK DEMAINE: x squared.
00:35:28.250 --> 00:35:28.750
Exactly.
00:35:32.138 --> 00:35:33.086
Time for a purple one.
00:35:36.404 --> 00:35:38.070
I'm getting better.
00:35:38.070 --> 00:35:38.970
Why x squared?
00:35:38.970 --> 00:35:41.740
Because we have
this mismatch here.
00:35:41.740 --> 00:35:43.290
We have a sub 2k.
00:35:43.290 --> 00:35:45.446
We want x to the 2k.
00:35:45.446 --> 00:35:46.320
How could we do that?
00:35:46.320 --> 00:35:54.190
Well, we could put x squared to
the 2k, and x squared to the k
00:35:54.190 --> 00:35:58.670
is the same thing
as x to the 2k.
00:35:58.670 --> 00:36:03.760
And so magically this
transforms into the even entries
00:36:03.760 --> 00:36:04.790
of a of x.
00:36:04.790 --> 00:36:06.110
That's half of them.
00:36:06.110 --> 00:36:11.260
We do the same thing for the
odd ones and we're almost there.
00:36:11.260 --> 00:36:18.180
Now we have a sub 2k plus 1
times x to the 2k, no plus 1.
00:36:18.180 --> 00:36:20.617
So how can I add a plus 1?
00:36:20.617 --> 00:36:21.900
AUDIENCE: Multiply by x.
00:36:21.900 --> 00:36:25.320
ERIK DEMAINE:
Multiply by x here.
00:36:25.320 --> 00:36:27.700
Take the whole
thing, multiply by x,
00:36:27.700 --> 00:36:31.010
then I get all of the
odd terms of a of x.
00:36:31.010 --> 00:36:32.880
I add these together.
00:36:32.880 --> 00:36:35.727
I get a of x.
00:36:35.727 --> 00:36:37.560
I mean, you could prove
this more carefully,
00:36:37.560 --> 00:36:42.980
but that's just algebra to
see that this is correct.
00:36:42.980 --> 00:36:45.700
Once you have this, it
tells you what I need to do
00:36:45.700 --> 00:36:50.940
is compute a even of x
squared for all x in x,
00:36:50.940 --> 00:36:54.080
so this is 4x and x.
00:36:54.080 --> 00:36:55.620
There's a for loop for you.
00:36:55.620 --> 00:36:57.120
So that's going to
take linear time.
00:36:57.120 --> 00:37:00.350
If I already know this value
and I already know this value,
00:37:00.350 --> 00:37:03.140
I do one multiplication,
one addition, and boom.
00:37:03.140 --> 00:37:05.760
I get a of x.
00:37:05.760 --> 00:37:20.580
So in the conquer step I
want to recursively compute--
00:37:20.580 --> 00:37:33.460
I think I'll call it-- a even
of y, and a odd of y for y
00:37:33.460 --> 00:37:35.440
in x squared.
00:37:35.440 --> 00:37:39.695
x squared is the set of
squares of all numbers in x.
00:37:43.220 --> 00:37:44.700
So I'm changing my set x.
00:37:44.700 --> 00:37:48.240
I started with a
polynomial a and a set x.
00:37:48.240 --> 00:37:51.310
Recursively, I'm doing
a different polynomial
00:37:51.310 --> 00:37:57.000
of half the degree-- half
the number of terms--
00:37:57.000 --> 00:38:02.650
but with a different
set of the same size.
00:38:02.650 --> 00:38:04.500
I started with x.
00:38:04.500 --> 00:38:08.300
x squared has the
same size as x, right?
00:38:08.300 --> 00:38:12.995
So let's try to figure out how
fast or slow this algorithm is.
00:38:15.504 --> 00:38:16.920
But that isn't
divide and conquer.
00:38:16.920 --> 00:38:20.290
That is going to be
our golden ticket.
00:38:20.290 --> 00:38:26.070
It's pretty simple, but we're
going to need another trick.
00:38:26.070 --> 00:38:29.540
So I'm going to
write a recurrence.
00:38:29.540 --> 00:38:32.720
Now, this recurrence
depends on two things.
00:38:32.720 --> 00:38:37.610
One is how many terms are there
in a that we've been calling n?
00:38:37.610 --> 00:38:40.170
And the other is how many
numbers are there in x?
00:38:40.170 --> 00:38:43.150
How many different places do I
have to evaluate my polynomial?
00:38:43.150 --> 00:38:47.450
So we've got t of n, and I
always call it size of x.
00:38:51.680 --> 00:38:55.620
So divide and conquer goes
hand-in-hand with recurrences.
00:38:55.620 --> 00:38:58.310
Generally you've got
the recursive part,
00:38:58.310 --> 00:39:00.440
so that's just how big
are the subproblems.
00:39:00.440 --> 00:39:01.280
How many are there?
00:39:01.280 --> 00:39:03.100
There are two subproblems.
00:39:03.100 --> 00:39:05.910
They have half the
size in terms of n,
00:39:05.910 --> 00:39:08.920
but they have the same
size in terms of x.
00:39:08.920 --> 00:39:14.190
So, all right, 2
times-- because there's
00:39:14.190 --> 00:39:24.610
two subproblems-- each with
size n over 2 and size x, plus
00:39:24.610 --> 00:39:28.240
what goes here is however much
it costs to do the divide step,
00:39:28.240 --> 00:39:30.690
plus however much it costs
to do the combined step-- all
00:39:30.690 --> 00:39:32.620
the non-recursive parts.
00:39:32.620 --> 00:39:35.640
So this is just partitioning
the vectors-- linear scan,
00:39:35.640 --> 00:39:37.290
linear time.
00:39:37.290 --> 00:39:39.480
This is-- we talked
about it-- it's
00:39:39.480 --> 00:39:43.560
a constant number of arithmetic
operations for each x.
00:39:43.560 --> 00:39:48.390
So this cause-order x time,
this cause-order n time.
00:39:48.390 --> 00:39:55.560
So, in general, we get n plus x.
00:39:55.560 --> 00:39:58.252
Now, this is, again,
not a recurrent solvable
00:39:58.252 --> 00:40:00.335
by the master method because
it has two variables.
00:40:03.130 --> 00:40:05.500
So usually when you're faced
with this sort of thing,
00:40:05.500 --> 00:40:08.600
you want to do back of
the envelope picture.
00:40:08.600 --> 00:40:10.110
Draw a recursion tree.
00:40:10.110 --> 00:40:11.600
That's a good way to go.
00:40:11.600 --> 00:40:16.720
So at the root-- now, I know
that initially x equals n.
00:40:16.720 --> 00:40:18.540
When I start out I
have n coefficients,
00:40:18.540 --> 00:40:21.270
I have n different positions
I want to evaluate them
00:40:21.270 --> 00:40:23.270
at because that's
what I want to do
00:40:23.270 --> 00:40:26.490
to do this conversion from
coefficients to samples.
00:40:26.490 --> 00:40:29.330
So at the root of the recursion
tree I'm just going to write n.
00:40:29.330 --> 00:40:33.360
Order n work to get started
and to do the recursions.
00:40:33.360 --> 00:40:35.610
There are two recursive calls.
00:40:35.610 --> 00:40:39.242
One has som-- both have
size n over 2 in terms of a,
00:40:39.242 --> 00:40:42.670
and they have the same
x-- which is also known
00:40:42.670 --> 00:40:45.340
as n-- in those two recursions.
00:40:45.340 --> 00:40:48.740
So in fact, the linear
work here will be n and n,
00:40:48.740 --> 00:40:50.680
and then we'll get n and n.
00:40:50.680 --> 00:40:53.100
x never goes down,
so x always remains
00:40:53.100 --> 00:40:56.120
n-- the original value of n.
00:40:56.120 --> 00:40:59.240
This is a bad recurrence.
00:40:59.240 --> 00:41:01.855
There are n-- sorry,
there are log n levels.
00:41:01.855 --> 00:41:02.730
That's the good news.
00:41:02.730 --> 00:41:06.300
Once we get down to constant
size we can kind of stop.
00:41:06.300 --> 00:41:08.020
When there's only
one coefficient
00:41:08.020 --> 00:41:11.740
I know how to evaluate the
polynomial with just a 0.
00:41:11.740 --> 00:41:13.090
That's easy.
00:41:13.090 --> 00:41:17.010
So down at the bottom
here, at the last level--
00:41:17.010 --> 00:41:20.550
this is the height log n--
the last level is just, again,
00:41:20.550 --> 00:41:23.440
going to be a whole bunch
of n's-- all the same n.
00:41:23.440 --> 00:41:25.740
Here n is the original
value of x because we
00:41:25.740 --> 00:41:27.246
haven't changed that.
00:41:27.246 --> 00:41:28.620
How many n's are
there down here?
00:41:33.629 --> 00:41:34.670
AUDIENCE: 2 to the log n.
00:41:34.670 --> 00:41:36.586
ERIK DEMAINE: 2 to the
log n, also known as n.
00:41:40.681 --> 00:41:41.180
Good.
00:41:41.180 --> 00:41:44.410
So we had-- because
we had log n levels,
00:41:44.410 --> 00:41:47.830
we had binary branching, so
it's 2 to the number of levels,
00:41:47.830 --> 00:41:49.100
which is just n.
00:41:49.100 --> 00:41:54.180
So this is n squared.
00:41:54.180 --> 00:41:56.095
All this work, still n squared.
00:42:01.540 --> 00:42:06.500
Clearly what we need is
for x to get smaller, too.
00:42:06.500 --> 00:42:09.750
If x-- if, in this
recursion-- let
00:42:09.750 --> 00:42:12.365
me, in red, draw the recursion
I would like to have.
00:42:15.980 --> 00:42:18.480
If x became x over
2 here, that's
00:42:18.480 --> 00:42:20.740
the only change we'd need.
00:42:20.740 --> 00:42:24.640
Then n and x change in
exactly the same way,
00:42:24.640 --> 00:42:26.700
and so then we can
just forget about x--
00:42:26.700 --> 00:42:28.310
it's going to be the same as n.
00:42:28.310 --> 00:42:33.040
Then we get 2 times n
over 2 plus order n.
00:42:33.040 --> 00:42:34.440
Look familiar?
00:42:34.440 --> 00:42:36.140
It is our bread and
butter recurrence--
00:42:36.140 --> 00:42:37.500
merge sort recurrence.
00:42:37.500 --> 00:42:39.247
That's n log n.
00:42:39.247 --> 00:42:40.330
That's what we need to do.
00:42:40.330 --> 00:42:48.050
Somehow, when we convert
our set x to x squared,
00:42:48.050 --> 00:42:49.520
I want x to get smaller.
00:42:52.100 --> 00:42:53.595
Is that at all plausible?
00:42:53.595 --> 00:42:54.470
Let's think about it.
00:43:04.150 --> 00:43:06.680
What's the base case?
00:43:06.680 --> 00:43:13.210
To keep things simple let's say
the base case when x equals 1,
00:43:13.210 --> 00:43:19.231
I'll just let x be-- let's say
I want to compute my a at 1.
00:43:19.231 --> 00:43:19.855
Keep it simple.
00:43:22.810 --> 00:43:26.007
What if I want two values in x?
00:43:26.007 --> 00:43:27.590
I'd like to have the
feature that when
00:43:27.590 --> 00:43:32.080
I square all the values in
x-- so I want two values,
00:43:32.080 --> 00:43:34.820
but when I square them
I only have one value.
00:43:38.240 --> 00:43:39.110
Solve for x.
00:43:44.346 --> 00:43:44.846
Yeah.
00:43:44.846 --> 00:43:45.971
AUDIENCE: Negative 1 and 1?
00:43:45.971 --> 00:43:49.320
ERIK DEMAINE: Negative 1 and 1.
00:43:49.320 --> 00:43:51.264
Whew, tough one.
00:43:51.264 --> 00:43:52.240
Ehh.
00:43:52.240 --> 00:43:53.470
Not bad.
00:43:53.470 --> 00:43:55.720
Good catch.
00:43:55.720 --> 00:43:56.710
Negative 1 and 1.
00:43:56.710 --> 00:43:58.977
That could work.
00:43:58.977 --> 00:44:00.060
What are negative 1 and 1?
00:44:00.060 --> 00:44:02.750
They're the square roots of 1.
00:44:02.750 --> 00:44:05.870
Negative 1 squared
is 1, 1 squared is 1.
00:44:05.870 --> 00:44:07.920
There's two square
roots for every number.
00:44:07.920 --> 00:44:09.750
Two square roots
for every number.
00:44:09.750 --> 00:44:11.480
Interesting.
00:44:11.480 --> 00:44:14.300
So that means, if I just
keep taking square roots,
00:44:14.300 --> 00:44:19.190
when I square them it
collapses by a factor of 2.
00:44:19.190 --> 00:44:28.100
Let me go to another board
and define something.
00:44:34.942 --> 00:44:36.900
You're all anticipating
what's going to happen,
00:44:36.900 --> 00:44:41.840
but I'm going to
say, a collapsing
00:44:41.840 --> 00:44:52.720
set x-- or, a set is collapsing
if either the size of x squared
00:44:52.720 --> 00:44:58.265
is the size of x divided
by 2, and, recursively,
00:44:58.265 --> 00:44:59.265
x squared is collapsing.
00:45:04.810 --> 00:45:07.310
So I need this to work all
the way down the recursion.
00:45:07.310 --> 00:45:11.020
Or I need a base case,
which is just x equals 1.
00:45:11.020 --> 00:45:12.750
There's a single item in x.
00:45:12.750 --> 00:45:15.610
So I happened to start
with x equals the item 1.
00:45:15.610 --> 00:45:16.720
It didn't have to be 1.
00:45:16.720 --> 00:45:18.980
It could have been 7.
00:45:18.980 --> 00:45:22.930
It couldn't be 0 because 0--
you won't get two numbers.
00:45:22.930 --> 00:45:24.780
There's only one
square root of 0.
00:45:24.780 --> 00:45:26.310
OK, so I lied a little bit.
00:45:26.310 --> 00:45:30.850
Other than 0, every number
has exactly two square roots,
00:45:30.850 --> 00:45:35.063
so what's the square
root of negative 1?
00:45:35.063 --> 00:45:35.563
AUDIENCE: i.
00:45:35.563 --> 00:45:37.450
ERIK DEMAINE: i.
00:45:37.450 --> 00:45:41.150
So complex numbers.
00:45:41.150 --> 00:45:47.040
If I take square roots of these
guys I get i and negative i,
00:45:47.040 --> 00:45:51.640
and again I get minus 1 and 1.
00:45:51.640 --> 00:45:53.880
That's when x equals 4.
00:45:53.880 --> 00:45:56.550
Turns out this is only going
to work for powers of 2,
00:45:56.550 --> 00:45:58.260
but, hey, if n isn't
a power of 2 just
00:45:58.260 --> 00:45:59.680
round up to the next power of 2.
00:45:59.680 --> 00:46:03.370
That only hurts me
by a factor of 2.
00:46:03.370 --> 00:46:05.100
Complex numbers.
00:46:05.100 --> 00:46:07.740
Every time I said real
number in the past,
00:46:07.740 --> 00:46:09.587
pretend I said complex number.
00:46:09.587 --> 00:46:10.920
Everything I said is still true.
00:46:10.920 --> 00:46:13.440
Actually, the root
thing is only true
00:46:13.440 --> 00:46:14.820
when you allow complex numbers.
00:46:14.820 --> 00:46:17.420
Some polynomials
have complex roots,
00:46:17.420 --> 00:46:19.780
so pretend I said complex.
00:46:19.780 --> 00:46:21.280
We're going to need
complex numbers.
00:46:21.280 --> 00:46:22.100
This is why.
00:46:22.100 --> 00:46:24.330
Because, when we'd start
taking square roots,
00:46:24.330 --> 00:46:27.890
we immediately get
complex numbers.
00:46:27.890 --> 00:46:29.512
Next would be x equals 8.
00:46:34.400 --> 00:46:37.116
Square root of i.
00:46:37.116 --> 00:46:38.990
Square root of i-- let's
see if I can do it--
00:46:38.990 --> 00:46:42.430
should be root 2 over
2 times 1 plus i.
00:46:45.420 --> 00:46:51.470
And then this one is just
the-- square root of negative i
00:46:51.470 --> 00:46:56.890
is going to be root 2
over 2 times 1 minus i,
00:46:56.890 --> 00:46:58.610
and then we have
all our old guys.
00:47:03.584 --> 00:47:05.250
Oh, and then I could
write plus or minus
00:47:05.250 --> 00:47:06.610
in front of each of these.
00:47:06.610 --> 00:47:08.485
I was like, there weren't
enough terms there.
00:47:08.485 --> 00:47:11.190
Now I've got eight-- sorry,
I've got four numbers here.
00:47:11.190 --> 00:47:12.465
I've got four numbers there.
00:47:12.465 --> 00:47:15.020
How in the world
could I remember this?
00:47:15.020 --> 00:47:16.050
Maybe I memorized it.
00:47:16.050 --> 00:47:17.530
No, I didn't memorize it.
00:47:17.530 --> 00:47:21.330
It's actually really
easy to figure this out
00:47:21.330 --> 00:47:23.715
if you know geometry.
00:47:23.715 --> 00:47:25.500
Geometry.
00:47:25.500 --> 00:47:27.710
Let's do geometry over here.
00:47:36.090 --> 00:47:36.861
It's convenient.
00:47:36.861 --> 00:47:37.860
I'm actually a geometer.
00:47:43.180 --> 00:47:45.380
You know, complex numbers
have two parts, right?
00:47:45.380 --> 00:47:47.570
The real part and
the complex part.
00:47:47.570 --> 00:47:50.560
I'm going to draw that in what's
called the complex plane, where
00:47:50.560 --> 00:47:53.690
we draw the real
part here, and I
00:47:53.690 --> 00:47:57.260
guess it's usually called the
imaginary part on the y-axis.
00:47:57.260 --> 00:48:00.190
Every point in this plane is
a complex number, and vice
00:48:00.190 --> 00:48:01.190
versa-- the same thing.
00:48:04.320 --> 00:48:05.870
So what did we start with?
00:48:05.870 --> 00:48:08.160
We started with the number 1.
00:48:08.160 --> 00:48:12.510
Number 1 will be here.
00:48:12.510 --> 00:48:14.240
It has no imaginary
part, so it's
00:48:14.240 --> 00:48:17.061
on the x-axis-- this is
the real line down here--
00:48:17.061 --> 00:48:19.310
and it's at position 1, which
I'm going to just define
00:48:19.310 --> 00:48:21.547
to be right there.
00:48:21.547 --> 00:48:22.630
Then we've got negative 1.
00:48:22.630 --> 00:48:24.650
That's over here.
00:48:24.650 --> 00:48:26.860
Then we got i.
00:48:26.860 --> 00:48:28.940
That's here, 1 times i.
00:48:28.940 --> 00:48:30.440
Then we've got negative i.
00:48:30.440 --> 00:48:33.380
That's here.
00:48:33.380 --> 00:48:36.660
Then we've got root 2
over 2 times i plus 1.
00:48:36.660 --> 00:48:40.470
That's here.
00:48:40.470 --> 00:48:43.060
Root 2 over 2 by root 2 over 2.
00:48:43.060 --> 00:48:46.920
What is a property of root 2
over 2 comma root 2 over 2?
00:48:46.920 --> 00:48:50.850
It has distance exactly
1 to the origin.
00:48:50.850 --> 00:48:57.012
If I draw this triangle,
root 2 over 2, root 2 over 2.
00:48:57.012 --> 00:48:58.290
I square this.
00:48:58.290 --> 00:48:58.990
I get a half.
00:48:58.990 --> 00:49:00.720
I square this I get a half.
00:49:00.720 --> 00:49:02.900
I add them together, I get what?
00:49:02.900 --> 00:49:06.870
Take the square root, I still
get 1, so this distance is 1.
00:49:06.870 --> 00:49:08.270
Interesting.
00:49:08.270 --> 00:49:11.070
And then I got the negative
of that, which is over here,
00:49:11.070 --> 00:49:12.670
and a negative of that.
00:49:12.670 --> 00:49:17.095
Then this is the-- with a
negative-- did I get it wrong?
00:49:17.095 --> 00:49:18.870
Yep, sorry.
00:49:18.870 --> 00:49:25.082
It doesn't matter, but I'll
think of it as i minus-- killed
00:49:25.082 --> 00:49:30.201
a chalk-- i minus 1.
00:49:30.201 --> 00:49:32.200
It's the same because I
have the plus and minus,
00:49:32.200 --> 00:49:34.150
but I like the geometry.
00:49:34.150 --> 00:49:40.380
So this point is negative root
2 over 2 by root 2 over 2,
00:49:40.380 --> 00:49:43.724
and then there's the
negative over here.
00:49:43.724 --> 00:49:45.415
What property do
these points have?
00:49:50.770 --> 00:49:51.574
I heard a word.
00:49:51.574 --> 00:49:52.490
AUDIENCE: Unit circle?
00:49:52.490 --> 00:49:53.573
ERIK DEMAINE: Unit circle.
00:49:57.690 --> 00:49:58.650
Good.
00:49:58.650 --> 00:50:00.660
Who said unit circle?
00:50:00.660 --> 00:50:02.160
Nice.
00:50:02.160 --> 00:50:03.890
It's a unit circle, right?
00:50:03.890 --> 00:50:05.390
Clearly that deserves
a Frisbee end.
00:50:09.970 --> 00:50:11.200
Unit circle.
00:50:11.200 --> 00:50:13.150
Hm.
00:50:13.150 --> 00:50:14.980
Circle.
00:50:14.980 --> 00:50:17.020
It seem good.
00:50:17.020 --> 00:50:24.100
What's going on here
is I took this number.
00:50:24.100 --> 00:50:28.000
I claimed it was
the square root of i
00:50:28.000 --> 00:50:30.470
because it turns out, if you
take points on the unit circle
00:50:30.470 --> 00:50:32.360
in the complex plane,
when you square
00:50:32.360 --> 00:50:36.260
a number it's like doubling the
angle relative to the x-axis.
00:50:36.260 --> 00:50:37.560
This is angle 0.
00:50:37.560 --> 00:50:40.900
This is angle-- what do
you call it-- 45 degrees.
00:50:40.900 --> 00:50:42.550
This is angle 90 degrees.
00:50:42.550 --> 00:50:45.360
So when I square this
number I get 90 degrees.
00:50:45.360 --> 00:50:49.774
That's why this number
is a square root of i.
00:50:49.774 --> 00:50:51.648
I probably should have
labeled some of these.
00:50:51.648 --> 00:50:53.022
This is i.
00:50:53.022 --> 00:50:54.770
This is minus i.
00:50:54.770 --> 00:51:00.320
This is minus 1, and this is 1.
00:51:00.320 --> 00:51:12.520
In general, we get something
called-- so these are
00:51:12.520 --> 00:51:15.040
called the n-th roots of unity.
00:51:19.040 --> 00:51:23.330
Unity is just a
fancy word for 1.
00:51:23.330 --> 00:51:27.380
1 is here, and first we
computed the square roots of 1.
00:51:27.380 --> 00:51:28.920
They were minus 1 and 1.
00:51:28.920 --> 00:51:32.280
Then we computed the
fourth roots of 1.
00:51:32.280 --> 00:51:37.910
All of these numbers, if
you take the fourth power,
00:51:37.910 --> 00:51:39.590
you get 1.
00:51:39.590 --> 00:51:41.970
Then we computed the
eighth roots of 1.
00:51:41.970 --> 00:51:45.160
All of these numbers, if
you take the 8th power,
00:51:45.160 --> 00:51:46.140
you get 1 again.
00:51:46.140 --> 00:51:49.074
So, in general,
n-th roots of unity.
00:51:49.074 --> 00:51:50.740
We're going to assume
n is a power of 2,
00:51:50.740 --> 00:51:53.730
but this notion actually
makes sense for any n.
00:51:53.730 --> 00:51:56.000
And they're just uniformly
spaced around the circle,
00:51:56.000 --> 00:51:59.640
and if you know some geometry
and how it relates to trig,
00:51:59.640 --> 00:52:01.820
you know that a general
point on the circle
00:52:01.820 --> 00:52:04.610
is cos theta comma sine theta.
00:52:04.610 --> 00:52:06.480
x-coordinate is cos theta.
00:52:06.480 --> 00:52:07.650
y-coordinate is sine theta.
00:52:07.650 --> 00:52:09.990
This is also a funny notation
for complex numbers--
00:52:09.990 --> 00:52:13.020
not so funny, this is the
geometric interpretation
00:52:13.020 --> 00:52:21.510
of cos theta plus i sine theta.
00:52:21.510 --> 00:52:23.810
And if I want them uniformly
spaced around the circle,
00:52:23.810 --> 00:52:27.060
and I want to include
this point-- also known
00:52:27.060 --> 00:52:30.510
as theta equals 0-- because
when theta equals 0,
00:52:30.510 --> 00:52:32.425
cos theta is 1, sine theta 0.
00:52:35.070 --> 00:52:40.380
So I want to say 4
theta equals to 0,
00:52:40.380 --> 00:52:46.630
and then-- here I'm going
to get fancy-- tau over n
00:52:46.630 --> 00:52:54.280
to tau over n up to n
minus 1 over n times tau.
00:52:54.280 --> 00:52:55.637
What's tau?
00:52:55.637 --> 00:52:56.220
AUDIENCE: 2pi.
00:52:56.220 --> 00:52:57.720
ERIK DEMAINE: 2pi, thank you.
00:53:00.590 --> 00:53:04.390
This is modern notation just
over the last couple years.
00:53:04.390 --> 00:53:09.790
I believe in tau so much I
got it tattooed on my arm.
00:53:09.790 --> 00:53:12.260
Tau is the fundamental constant.
00:53:12.260 --> 00:53:14.310
Screw pi.
00:53:14.310 --> 00:53:16.000
None of that 3 and change.
00:53:16.000 --> 00:53:19.460
Six and change is where it's at.
00:53:19.460 --> 00:53:21.196
So tau.
00:53:21.196 --> 00:53:22.320
Clearly this is much nicer.
00:53:22.320 --> 00:53:24.840
Tau over n, not 2pi over n.
00:53:24.840 --> 00:53:28.870
Tau is a whole circle.
00:53:28.870 --> 00:53:32.062
This is one n-th of a circle,
2n-ths of a circle, n minus 1
00:53:32.062 --> 00:53:33.020
over n-ths of a circle.
00:53:33.020 --> 00:53:35.340
I didn't do n n-ths of a
circle because that's also
00:53:35.340 --> 00:53:36.240
the same as 0.
00:53:39.020 --> 00:53:47.290
Now, why did I
introduce this notation?
00:53:47.290 --> 00:53:49.540
Because there's this
other great thing
00:53:49.540 --> 00:53:55.460
called Euler's formula,
which is that this
00:53:55.460 --> 00:54:02.810
equals e to the i theta.
00:54:07.130 --> 00:54:09.380
Double check.
00:54:09.380 --> 00:54:14.050
It's so rare that I get
to do real calculus.
00:54:14.050 --> 00:54:16.460
This is Euler's
formula-- e for Euler--
00:54:16.460 --> 00:54:18.840
another number-- 2 and change.
00:54:18.840 --> 00:54:21.810
e to the i-- this is funny
because it's complex-- times
00:54:21.810 --> 00:54:24.780
theta is equal to cos
theta plus i sine theta.
00:54:24.780 --> 00:54:28.980
This is the relation between
exponentials and trigonometry.
00:54:28.980 --> 00:54:31.670
That's a big thing Euler did.
00:54:31.670 --> 00:54:33.080
Cool.
00:54:33.080 --> 00:54:33.660
So what?
00:54:33.660 --> 00:54:36.080
Because this lets
us understand how
00:54:36.080 --> 00:54:42.640
squares work-- not squares the
shape, squares the operation.
00:54:50.580 --> 00:54:57.060
When I take squares-- so
if I take e to the i theta,
00:54:57.060 --> 00:55:00.020
this is one of my
roots of unity.
00:55:00.020 --> 00:55:02.380
Let me expand out what theta is.
00:55:02.380 --> 00:55:07.460
So theta is some k
times tau over n.
00:55:07.460 --> 00:55:09.010
Let's do it this way first.
00:55:09.010 --> 00:55:17.070
So in reality we have k tau
over n, but when I square it,
00:55:17.070 --> 00:55:19.790
that's the same thing as
putting the 2 right here.
00:55:19.790 --> 00:55:24.390
This is the same thing
as e to i times 2 theta.
00:55:24.390 --> 00:55:24.890
Bingo.
00:55:24.890 --> 00:55:28.200
I get what I was claiming, that
if I start at some angle theta
00:55:28.200 --> 00:55:31.450
relative to the x-axis,
when I square the number
00:55:31.450 --> 00:55:32.950
I just double the angle.
00:55:32.950 --> 00:55:35.370
This is why.
00:55:35.370 --> 00:55:39.770
This is obvious just
from regular algebra.
00:55:39.770 --> 00:55:41.820
And then this thing,
Euler's formula,
00:55:41.820 --> 00:55:45.680
tells me that corresponds to
doubling the angle on a circle.
00:55:45.680 --> 00:55:47.970
So it only works with
points on a circle.
00:55:47.970 --> 00:55:58.610
So when I go here I get e to
the i2k times tau over n--
00:55:58.610 --> 00:56:02.080
twice as far around circle.
00:56:02.080 --> 00:56:03.850
All right.
00:56:03.850 --> 00:56:06.530
Fine.
00:56:06.530 --> 00:56:10.520
What happens if I take this
number and I square it?
00:56:10.520 --> 00:56:12.030
This has a really big angle.
00:56:12.030 --> 00:56:20.500
This is 1/2 plus 1/8, whatever
that is-- 5/8 times tau.
00:56:20.500 --> 00:56:28.230
When I double that
angle I go to here.
00:56:28.230 --> 00:56:31.910
Now, this, you might
call it, 10/8, or you
00:56:31.910 --> 00:56:36.540
might also call it 1/4 because
when you go around the circle
00:56:36.540 --> 00:56:37.684
you stay on the circle.
00:56:37.684 --> 00:56:39.100
So there's another
thing going on,
00:56:39.100 --> 00:56:47.580
which is really-- this is e to
the i times 2 theta mod tau.
00:56:47.580 --> 00:56:49.720
Usually we think of mods
relative to integers,
00:56:49.720 --> 00:56:54.040
but what I mean is every time
I add a multiple of tau nothing
00:56:54.040 --> 00:56:54.540
changes.
00:56:54.540 --> 00:56:56.998
If I go around the circle five
times and then do something,
00:56:56.998 --> 00:56:59.290
it's the same as just
doing the something.
00:56:59.290 --> 00:57:01.620
So I kind of need that.
00:57:01.620 --> 00:57:08.340
And this is true because
e to the i tau equals 1.
00:57:08.340 --> 00:57:12.500
You may know it as e to
the i pi equals negative 1,
00:57:12.500 --> 00:57:15.990
but clearly this is a superior
formula-- so superior I got
00:57:15.990 --> 00:57:17.220
it tattooed on my other arm.
00:57:17.220 --> 00:57:20.549
[LAUGHTER]
00:57:20.549 --> 00:57:22.590
It's amazing what you can
do with a laser printer
00:57:22.590 --> 00:57:24.260
and a temporary tattoo kit.
00:57:24.260 --> 00:57:31.440
Sadly, these won't last, but
definitely try it at home.
00:57:31.440 --> 00:57:32.090
Cool.
00:57:32.090 --> 00:57:36.390
So e to the i tau equals
1, so going around
00:57:36.390 --> 00:57:38.550
in circles-- same thing as not.
00:57:38.550 --> 00:57:39.860
All right.
00:57:39.860 --> 00:57:42.580
So you can draw on this
picture for every number, what
00:57:42.580 --> 00:57:44.440
is its square?
00:57:44.440 --> 00:57:47.957
And, in general, if you
look at these four guys,
00:57:47.957 --> 00:57:50.040
their squares are just
going to be these two guys.
00:57:50.040 --> 00:57:52.510
If you look at these
four guys, their squares
00:57:52.510 --> 00:57:55.000
are going to be among
these four guys.
00:57:55.000 --> 00:57:56.440
So I started with 8 guys.
00:57:56.440 --> 00:57:57.530
I square them, I get 4.
00:57:57.530 --> 00:57:58.500
I square them, I get 2.
00:57:58.500 --> 00:57:59.510
I square them, I get 1.
00:57:59.510 --> 00:58:01.640
That's how we
constructed it, but you
00:58:01.640 --> 00:58:05.100
can see it works not
only for this 8-point set
00:58:05.100 --> 00:58:07.800
that we constructed
sort of by hand,
00:58:07.800 --> 00:58:09.770
but it works for the
n-th roots of unity.
00:58:09.770 --> 00:58:14.260
As long as n is a power
of 2 this set of points
00:58:14.260 --> 00:58:15.055
will be collapsing.
00:58:17.690 --> 00:58:23.240
So if n is the power of
2 for some integer k,
00:58:23.240 --> 00:58:42.210
then n-th roots of
unity are collapsing
00:58:42.210 --> 00:58:44.700
according to this definition.
00:58:44.700 --> 00:58:46.480
And that's what we want.
00:58:46.480 --> 00:58:50.140
Then this divide and conquer
algorithm runs in n log n time
00:58:50.140 --> 00:58:53.000
because every time
we square the set x,
00:58:53.000 --> 00:58:55.510
we reduce its size
by a factor of 2.
00:58:55.510 --> 00:58:58.990
We get t of n equals 2
times t of n over 2 plus n.
00:58:58.990 --> 00:59:01.190
Plus order n and
that's order n log n.
00:59:01.190 --> 00:59:05.000
And so this whole thing we
compute-- in other words,
00:59:05.000 --> 00:59:13.280
we set xk to be e to
the ik tau over n.
00:59:13.280 --> 00:59:15.780
Now, I guess I should
say how to compute that,
00:59:15.780 --> 00:59:18.500
but let's just say that's
given to you for free.
00:59:18.500 --> 00:59:21.586
It takes constant time to
compute each roots of unity.
00:59:21.586 --> 00:59:23.210
In fact, again, we
only need to do this
00:59:23.210 --> 00:59:27.000
once and for all
for each value of n,
00:59:27.000 --> 00:59:29.680
so you can think of it as just
being part of the algorithm.
00:59:29.680 --> 00:59:30.990
These are the xk's we use.
00:59:30.990 --> 00:59:33.550
I said xk could be anything
we want as long as they're all
00:59:33.550 --> 00:59:35.290
different, so I'm
going to choose them
00:59:35.290 --> 00:59:41.830
to be n uniformly spaced points
around the complex unit circle.
00:59:41.830 --> 00:59:46.610
And then magically this
algorithm runs in n log n time.
00:59:46.610 --> 00:59:49.109
It's pretty cool.
00:59:49.109 --> 00:59:51.150
Intuitively, you think of
real numbers x squared,
00:59:51.150 --> 00:59:54.360
of course it has
the same sides as x.
00:59:54.360 --> 00:59:56.580
But once you go
to complex numbers
00:59:56.580 --> 00:59:58.420
there's this nifty
trick where you
00:59:58.420 --> 01:00:00.920
start with one
n-th of the circle
01:00:00.920 --> 01:00:03.345
and you'd uniformly space
points after the first level
01:00:03.345 --> 01:00:03.920
of recursion.
01:00:03.920 --> 01:00:06.000
You only have to be
dealing with n over two
01:00:06.000 --> 01:00:08.580
of those points,
namely the even ones,
01:00:08.580 --> 01:00:11.179
also known as the n over
[? 2th ?] roots of unity.
01:00:11.179 --> 01:00:12.720
And after the next
level of recursion
01:00:12.720 --> 01:00:14.220
is the n over fourth
roots of unity.
01:00:14.220 --> 01:00:14.870
And so on.
01:00:14.870 --> 01:00:16.930
So this recursion
is well defined.
01:00:16.930 --> 01:00:19.030
When you have a vector
of size n you just
01:00:19.030 --> 01:00:21.670
have to deal with the n-th roots
of unity, and you're happy.
01:00:25.710 --> 01:00:28.090
That is fast Fourier transform.
01:00:28.090 --> 01:00:33.750
That algorithm with
these xi's is FFT.
01:00:33.750 --> 01:00:35.230
So let me write this somewhere.
01:00:44.590 --> 01:00:46.040
It kind of snuck up on us.
01:00:46.040 --> 01:00:50.690
This is the algorithm
we were aiming to find.
01:00:50.690 --> 01:01:05.549
Fast Fourier transform is that
divide and conquer algorithm
01:01:05.549 --> 01:01:06.090
on the right.
01:01:12.570 --> 01:01:14.860
I'll write it abstractly.
01:01:14.860 --> 01:01:18.080
For something called the
DFT-- the discrete Fourier
01:01:18.080 --> 01:01:29.570
transform-- is the corresponding
mathematical transformation.
01:01:29.570 --> 01:01:31.310
The fast is about an algorithm.
01:01:31.310 --> 01:01:34.840
Discrete is about discrete.
01:01:34.840 --> 01:01:40.040
So DFT is this thing that
we wanted to compute,
01:01:40.040 --> 01:01:45.660
which was basically
the product v times a.
01:01:45.660 --> 01:01:48.310
Remember, v was the
Vandermonde matrix,
01:01:48.310 --> 01:01:52.430
and it depended
on all these xk's.
01:01:52.430 --> 01:02:00.240
And we're going to set xk to
be this e to the ik tau over n.
01:02:03.400 --> 01:02:14.810
So if you remember the vjk
here was xj to the k-th power,
01:02:14.810 --> 01:02:23.650
so this just becomes e
to the ijk tau over n.
01:02:23.650 --> 01:02:26.120
It's a little funny
these are all consecutive
01:02:26.120 --> 01:02:27.640
because this is a
totally different,
01:02:27.640 --> 01:02:31.620
but-- that's the matrix.
01:02:31.620 --> 01:02:35.044
If you take that
matrix times a vector,
01:02:35.044 --> 01:02:36.835
that is called a discrete
Fourier transform
01:02:36.835 --> 01:02:41.780
of the vector, and FFT is a way
to compute it in n log n time.
01:02:41.780 --> 01:02:44.680
You just run this algorithm
and for those xk's it'll
01:02:44.680 --> 01:02:46.104
just work in n log n time.
01:02:48.880 --> 01:02:50.480
Cool.
01:02:50.480 --> 01:02:53.560
But if you remember way back to
the beginning, what we needed
01:02:53.560 --> 01:02:56.300
is a way-- this converts
a coefficient vector
01:02:56.300 --> 01:02:58.390
into a sample vector.
01:02:58.390 --> 01:03:04.890
Then we can multiply
the polynomials,
01:03:04.890 --> 01:03:08.150
then we need to transform the
sample vector that results back
01:03:08.150 --> 01:03:09.820
into a coefficient vector.
01:03:09.820 --> 01:03:11.750
So we're only half done.
01:03:11.750 --> 01:03:13.840
I only have 15 minutes.
01:03:13.840 --> 01:03:17.040
Luckily, the other half is
almost identical to this half,
01:03:17.040 --> 01:03:19.420
so let's do that next.
01:03:19.420 --> 01:03:20.390
Maybe over here.
01:03:28.160 --> 01:03:31.130
So we've got our great
divide and conquer algorithm,
01:03:31.130 --> 01:03:33.290
what we need now--
let me give you
01:03:33.290 --> 01:03:35.630
the polynomial
multiplication algorithm.
01:03:38.520 --> 01:03:42.660
Let's call this-- sound more
exotic-- fast polynomial
01:03:42.660 --> 01:03:45.565
multiplication, fast
meaning n log n.
01:03:48.830 --> 01:03:53.400
Let's say that we're
given two polynomials, a
01:03:53.400 --> 01:03:56.460
and b, represented
in coefficient form.
01:04:00.770 --> 01:04:06.280
What we need is-- I'll
call it a star, which is
01:04:06.280 --> 01:04:10.230
the result of running FFT on a.
01:04:10.230 --> 01:04:15.320
It's the discrete Fourier
transform of a. b star,
01:04:15.320 --> 01:04:17.430
do the same thing.
01:04:17.430 --> 01:04:20.250
So we know what this
means is convert
01:04:20.250 --> 01:04:23.040
a from a coefficient vector
into a sample vector.
01:04:23.040 --> 01:04:25.970
Convert b from a coefficient
vector into a sample vector.
01:04:25.970 --> 01:04:31.880
Now I have the samples of a star
at the n-th roots of unity--
01:04:31.880 --> 01:04:36.400
these guys-- and
I have the samples
01:04:36.400 --> 01:04:40.250
of b at the exact same points--
the n-th roots of unity--
01:04:40.250 --> 01:04:45.900
so I can compute c star, the
transform version-- the DFT
01:04:45.900 --> 01:04:51.920
of c-- is just the,
I mean, c star k
01:04:51.920 --> 01:04:55.900
equals a star k times b star k.
01:04:55.900 --> 01:05:01.160
This is the multiplication
algorithm for sample vectors.
01:05:01.160 --> 01:05:02.440
We started with that.
01:05:02.440 --> 01:05:03.990
That's linear time.
01:05:03.990 --> 01:05:06.420
And then the missing
piece is I need
01:05:06.420 --> 01:05:10.640
to re-compute c, which is
the inverse fast Fourier
01:05:10.640 --> 01:05:14.005
transform of c star.
01:05:17.940 --> 01:05:21.030
So this is the missing
link, so to speak.
01:05:21.030 --> 01:05:22.477
We need to be able
to go backwards
01:05:22.477 --> 01:05:23.435
in this transformation.
01:05:32.910 --> 01:05:36.630
Good news is the algorithm
is not going to change.
01:05:36.630 --> 01:05:40.104
What we're computing
isn't going to change.
01:05:40.104 --> 01:05:41.770
All that's going to
change are the xk's.
01:05:45.230 --> 01:05:46.740
Why?
01:05:46.740 --> 01:05:51.210
Because, remember
from the top-- right
01:05:51.210 --> 01:05:53.940
now we know how to compute
v times a, but what we now
01:05:53.940 --> 01:05:57.230
need to compute is
the inverse times a.
01:05:57.230 --> 01:06:00.470
So the only question
is, what is the inverse?
01:06:00.470 --> 01:06:05.660
This matrix has a super special
structure-- it's symmetric,
01:06:05.660 --> 01:06:08.340
lots of points on a
circle-- maybe the inverse
01:06:08.340 --> 01:06:10.560
has a nice structure.
01:06:10.560 --> 01:06:13.070
And it does.
01:06:13.070 --> 01:06:25.220
Claim is the inverse is v
complex conjugate divided by n.
01:06:25.220 --> 01:06:27.340
What's complex conjugate?
01:06:27.340 --> 01:06:30.220
For a geometer, it's
reflection through the x-axis.
01:06:30.220 --> 01:06:35.130
For an algebraic
person, a plus ib,
01:06:35.130 --> 01:06:40.000
the complex conjugate
is a minus ib.
01:06:40.000 --> 01:06:43.560
So just apply that to
every entry in the matrix,
01:06:43.560 --> 01:06:45.647
and then divide all
the entries by n.
01:06:45.647 --> 01:06:46.480
You get the inverse.
01:06:50.130 --> 01:06:51.490
Cool.
01:06:51.490 --> 01:06:55.610
Very cool because what
this tells us is we
01:06:55.610 --> 01:06:58.850
run exactly the same
algorithm and do exactly
01:06:58.850 --> 01:07:01.320
the same transformation.
01:07:01.320 --> 01:07:04.410
If we want to do the
inverse we can actually
01:07:04.410 --> 01:07:08.310
just use v, but with a
different choice of xk.
01:07:08.310 --> 01:07:13.315
Namely, for the inverse,
we'll call it xk inverse.
01:07:16.280 --> 01:07:19.230
We just take the
complex conjugate
01:07:19.230 --> 01:07:24.040
of this thing, which
turns out to be
01:07:24.040 --> 01:07:32.534
e to the minus ijk
tau over n, and then
01:07:32.534 --> 01:07:33.700
divide the whole thing by n.
01:07:36.610 --> 01:07:39.550
I'm using a fact here, which
is that the complex conjugate
01:07:39.550 --> 01:07:42.990
of this number is
actually, just,
01:07:42.990 --> 01:07:45.090
you put a minus sign here.
01:07:45.090 --> 01:07:45.940
Why does that hold?
01:07:45.940 --> 01:07:46.815
Because geometry.
01:07:49.350 --> 01:07:53.040
Theta is usually measuring
the counterclockwise angle
01:07:53.040 --> 01:07:54.990
from the x-axis.
01:07:54.990 --> 01:07:57.890
If you take the complex
conjugate you go from up here
01:07:57.890 --> 01:08:00.395
to down here-- the
reflection to the x-axis.
01:08:00.395 --> 01:08:02.270
That's the same thing
as if you measure theta
01:08:02.270 --> 01:08:05.380
as a clockwise angle
from the x-axis.
01:08:05.380 --> 01:08:07.790
That's the same thing
as negating the angle.
01:08:07.790 --> 01:08:09.361
So that's just a
little geometry.
01:08:09.361 --> 01:08:11.110
You can prove it
algebraically, although I
01:08:11.110 --> 01:08:13.850
don't know how off-hand.
01:08:13.850 --> 01:08:15.830
It's not hard.
01:08:15.830 --> 01:08:18.370
So if I want to take
some angle and then
01:08:18.370 --> 01:08:21.189
flip it through the x-axis,
it's the same as the negative
01:08:21.189 --> 01:08:21.750
of the angle.
01:08:21.750 --> 01:08:24.439
So the complex
conjugate of this number
01:08:24.439 --> 01:08:26.997
is the same thing
with a minus sign.
01:08:26.997 --> 01:08:28.330
And then we have to divide by n.
01:08:28.330 --> 01:08:31.570
If I just-- did I lie?
01:08:34.979 --> 01:08:37.279
I can't divide by n, sorry.
01:08:37.279 --> 01:08:38.640
It's in the wrong spot.
01:08:38.640 --> 01:08:43.670
I use these xk's and then
I apply the transform.
01:08:43.670 --> 01:08:45.670
I take this thing.
01:08:45.670 --> 01:08:49.939
I get not quite the inverse, but
I end up getting n the inverse.
01:08:49.939 --> 01:08:53.649
I multiply that by my a star.
01:08:53.649 --> 01:08:55.702
That's going to
give me n times n.
01:09:02.000 --> 01:09:05.289
So then I just take that
vector and divided by n.
01:09:05.289 --> 01:09:06.830
Boom, I've got the
inverse transform.
01:09:06.830 --> 01:09:13.200
So this is how you do inverse
fast Fourier transform.
01:09:13.200 --> 01:09:17.000
You just flip the sign and
the exponent in the xk's,
01:09:17.000 --> 01:09:20.270
then you do the same
matrix-vector product,
01:09:20.270 --> 01:09:21.495
and then you divide by n.
01:09:21.495 --> 01:09:22.870
And then you've
done the inverse.
01:09:22.870 --> 01:09:25.757
So that's how you do-- if you
believe this claim-- that's
01:09:25.757 --> 01:09:26.840
how you do this last step.
01:09:26.840 --> 01:09:27.340
Question?
01:09:27.340 --> 01:09:30.180
AUDIENCE: What's the j
in the x j [INAUDIBLE].
01:09:30.180 --> 01:09:33.330
ERIK DEMAINE:
Whoops, no j, sorry.
01:09:33.330 --> 01:09:37.180
I was imagining these guys.
01:09:37.180 --> 01:09:38.680
Just k.
01:09:38.680 --> 01:09:40.700
Thank you.
01:09:40.700 --> 01:09:43.720
Other questions?
01:09:43.720 --> 01:09:48.390
So what remains is
to prove this claim.
01:09:48.390 --> 01:09:54.420
I'd be very happy if I-- I
should probably have better--
01:09:54.420 --> 01:09:57.570
anyway, that's not the best.
01:09:57.570 --> 01:10:01.600
I'm going to call this xk prime.
01:10:01.600 --> 01:10:06.850
Then when I plug that in, I get
a different matrix, v prime.
01:10:06.850 --> 01:10:12.040
And what this claim
says is that v prime is
01:10:12.040 --> 01:10:16.330
equal to n times the inverse.
01:10:16.330 --> 01:10:18.810
So I apply the same
f of t algorithm,
01:10:18.810 --> 01:10:21.510
but with v prime
instead of v, then I
01:10:21.510 --> 01:10:23.390
get not quite the
product I want,
01:10:23.390 --> 01:10:25.230
but just n times
the product I want.
01:10:25.230 --> 01:10:29.165
Divide every term by n
and we get the inverse.
01:10:29.165 --> 01:10:31.040
This is the cool thing
about complex numbers.
01:10:31.040 --> 01:10:33.580
Here we get another cool thing.
01:10:33.580 --> 01:10:36.120
All right, but we have
to prove this claim.
01:10:36.120 --> 01:10:38.420
So let's do a little
bit of algebra.
01:10:38.420 --> 01:10:39.240
No pain, no gain.
01:10:41.851 --> 01:10:42.350
Good.
01:10:45.460 --> 01:10:49.885
So let's look at vjk prime.
01:10:55.240 --> 01:10:55.740
Sorry.
01:11:04.690 --> 01:11:12.460
So let's look at p,
the product of v,
01:11:12.460 --> 01:11:18.550
and v complex conjugate-- what
I was calling v prime up there.
01:11:18.550 --> 01:11:22.500
I claim that this thing
is n times the identity
01:11:22.500 --> 01:11:24.355
matrix, with 1's
down the diagonal
01:11:24.355 --> 01:11:26.860
and 0's everywhere else.
01:11:26.860 --> 01:11:28.990
So let's look at this product.
01:11:28.990 --> 01:11:35.370
In general, let's look at the
the jk-th item in the product.
01:11:35.370 --> 01:11:43.670
That's going to come from row
j of V, dot product with column
01:11:43.670 --> 01:11:47.620
k of the complex conjugate.
01:11:47.620 --> 01:11:49.120
Now the matrices
here are symmetric,
01:11:49.120 --> 01:11:50.890
so actually rows and
columns are the same,
01:11:50.890 --> 01:11:53.267
but that's the general
definition of the cell
01:11:53.267 --> 01:11:54.600
and the product of two matrices.
01:11:57.330 --> 01:12:01.170
So let's write it
out and a summation.
01:12:01.170 --> 01:12:05.560
We have-- from m equals
0-- it's so hard not
01:12:05.560 --> 01:12:07.450
to use i for my summations.
01:12:07.450 --> 01:12:10.270
It's the only class I have to do
it because i is already taken,
01:12:10.270 --> 01:12:12.040
but I guess I can
still use capital I,
01:12:12.040 --> 01:12:18.120
but we'll use m because i
is complex number today.
01:12:18.120 --> 01:12:30.570
So we have e to the i
tau jm over n times e
01:12:30.570 --> 01:12:36.010
to the minus i tau mk k over n.
01:12:36.010 --> 01:12:37.924
I don't know why I
changed the order.
01:12:37.924 --> 01:12:40.090
I put the tau here instead
of there, but same thing.
01:12:42.720 --> 01:12:46.040
This is just the
for every position
01:12:46.040 --> 01:12:47.980
m in the cell and
corresponding position
01:12:47.980 --> 01:12:51.020
m in-- sorry, m in the
row, corresponding position
01:12:51.020 --> 01:12:55.929
m in the column, I have
jm and I have mk-- again,
01:12:55.929 --> 01:12:56.970
the order doesn't matter.
01:12:56.970 --> 01:12:59.600
It's symmetric, but I'm
getting it right here.
01:12:59.600 --> 01:13:03.520
This is-- we're
using this formula.
01:13:03.520 --> 01:13:09.600
I put a minus sign here because
this is the complex conjugate.
01:13:09.600 --> 01:13:11.440
So now I just do some algebra.
01:13:11.440 --> 01:13:12.230
These share a lot.
01:13:12.230 --> 01:13:13.520
They share i.
01:13:13.520 --> 01:13:20.500
They share m, and they
share the divided by n.
01:13:20.500 --> 01:13:28.910
So this is sum m equals 0,
n minus 1 e to the i-- oh,
01:13:28.910 --> 01:13:40.060
they also share tau-- tau
m over n times j minus k.
01:13:40.060 --> 01:13:43.450
Please correct me if
I make any mistakes.
01:13:43.450 --> 01:13:43.950
Cool.
01:13:43.950 --> 01:13:48.660
So it depends how and
k relate, of course.
01:13:48.660 --> 01:13:54.410
If j equals k, that's also known
as something on the diagonal.
01:13:54.410 --> 01:13:59.090
I want the matrix
n, n, n, n, 0, 0.
01:13:59.090 --> 01:14:00.590
So j equals k.
01:14:00.590 --> 01:14:01.510
That's the diagonal.
01:14:01.510 --> 01:14:04.240
That's where I want to get n,
and, indeed, if j equals k,
01:14:04.240 --> 01:14:05.320
this becomes 0.
01:14:05.320 --> 01:14:09.410
So all this becomes
0. e to the 0 is 1,
01:14:09.410 --> 01:14:11.920
and so I'm summing up 1 n times.
01:14:11.920 --> 01:14:14.260
I get n.
01:14:14.260 --> 01:14:15.060
Cool.
01:14:15.060 --> 01:14:15.810
That's one case.
01:14:15.810 --> 01:14:21.110
This is n if j equals k, and
somehow I claim that everywhere
01:14:21.110 --> 01:14:23.300
else I get 0's.
01:14:23.300 --> 01:14:24.750
So let's prove that.
01:14:43.810 --> 01:14:51.030
So if j does not
equal k-- I'm going
01:14:51.030 --> 01:14:54.070
to rewrite this a little bit.
01:14:54.070 --> 01:14:57.010
j and k are fixed.
01:14:57.010 --> 01:15:01.500
m is changing in the sum,
so I want to write this
01:15:01.500 --> 01:15:17.880
as sum m equals 0 to n minus
1, e to the i tau j minus k,
01:15:17.880 --> 01:15:22.510
over m, times m.
01:15:22.510 --> 01:15:27.430
In other words, this thing
raised to the n-th power.
01:15:27.430 --> 01:15:28.569
What is this series?
01:15:33.460 --> 01:15:34.836
One word.
01:15:34.836 --> 01:15:35.760
AUDIENCE: Geometric.
01:15:35.760 --> 01:15:36.760
ERIK DEMAINE: Geometric.
01:15:36.760 --> 01:15:37.435
Thank you.
01:15:41.100 --> 01:15:44.820
This is a geometric
series, and I
01:15:44.820 --> 01:15:48.010
guess I should have waited to
give you the Frisbee until you
01:15:48.010 --> 01:15:50.380
tell me how do you
solve a geometric series
01:15:50.380 --> 01:15:51.660
with a finite term?
01:15:51.660 --> 01:15:55.050
So think of this as z to the m.
01:15:55.050 --> 01:15:56.080
Do you know the formula?
01:15:56.080 --> 01:15:59.315
AUDIENCE: That's
the e to the i tau--
01:15:59.315 --> 01:16:00.190
ERIK DEMAINE: Just z.
01:16:00.190 --> 01:16:03.295
ERIK DEMAINE: Oh,
z to the n minus 1.
01:16:03.295 --> 01:16:04.670
ERIK DEMAINE: Z
to the n minus 1.
01:16:04.670 --> 01:16:05.290
Almost.
01:16:05.290 --> 01:16:06.332
AUDIENCE: Over z minus 1.
01:16:06.332 --> 01:16:07.540
ERIK DEMAINE: Over z minus 1.
01:16:07.540 --> 01:16:08.250
Yep.
01:16:08.250 --> 01:16:14.210
That's-- If you have sum of z to
the k, k equals 0 to n minus 1.
01:16:14.210 --> 01:16:14.830
That's that.
01:16:14.830 --> 01:16:17.290
It's in the appendix
of your textbook.
01:16:17.290 --> 01:16:21.340
So we just plug
that in and we get--
01:16:21.340 --> 01:16:29.380
oh boy-- e to the-- better
chalk-- e to the i tau
01:16:29.380 --> 01:16:39.224
j minus k over n to the n-th
power minus 1 over e to the i.
01:16:39.224 --> 01:16:40.890
Actually, the denominator
doesn't really
01:16:40.890 --> 01:16:43.660
matter because this
is supposed to be 0,
01:16:43.660 --> 01:16:48.360
so it's all about the numerator,
but it's the same thing.
01:16:48.360 --> 01:16:49.130
Minus 1.
01:16:53.130 --> 01:16:54.306
Where's my red?
01:16:54.306 --> 01:16:56.590
Red.
01:16:56.590 --> 01:17:00.280
The n cancels with the n.
01:17:00.280 --> 01:17:07.090
This is an integer not equal to
zero, so we have e to the i tau
01:17:07.090 --> 01:17:09.020
to an integer power.
01:17:09.020 --> 01:17:12.270
What's e to the i tau?
01:17:12.270 --> 01:17:12.770
1.
01:17:15.430 --> 01:17:17.170
Convenient that I
have that there.
01:17:17.170 --> 01:17:21.011
1 minus 1 is 0, so we get 0.
01:17:21.011 --> 01:17:21.510
Satisfying.
01:17:24.670 --> 01:17:26.700
So that proves this claim.
01:17:26.700 --> 01:17:28.780
We prove that, on the
diagonal, we get n
01:17:28.780 --> 01:17:30.370
because we had n copies of 1.
01:17:30.370 --> 01:17:34.110
Off the diagonal we
get 0, like this.
01:17:34.110 --> 01:17:40.530
Therefore, v complex conjugate
times n is the v inverse.
01:17:40.530 --> 01:17:42.320
Therefore, this
algorithm actually
01:17:42.320 --> 01:17:46.520
computes the inverse
fast Fourier transform.
01:17:46.520 --> 01:17:49.460
So that's it for
algorithms today,
01:17:49.460 --> 01:17:52.730
but let me quickly tell you
about some applications.
01:17:52.730 --> 01:17:54.560
You've probably
taken other classes
01:17:54.560 --> 01:17:56.560
that use applications of
fast Fourier transform,
01:17:56.560 --> 01:17:58.510
so I will just summarize.
01:17:58.510 --> 01:18:02.380
If you've ever edited
audio, you probably
01:18:02.380 --> 01:18:08.490
did it in-- unless you're just
pasting audio clips together--
01:18:08.490 --> 01:18:11.160
you probably did it in what's
called frequency space.
01:18:11.160 --> 01:18:13.910
So you know that-- as I talked
about in the beginning--
01:18:13.910 --> 01:18:16.480
when we're measuring where the
membrane on this microphone
01:18:16.480 --> 01:18:21.550
goes over time, that is in
the time domain for every time
01:18:21.550 --> 01:18:23.850
we sample where,
physically, this thing is.
01:18:23.850 --> 01:18:26.277
If you apply-- I think
the way I defined it here
01:18:26.277 --> 01:18:27.860
is the inverse fast
Fourier transform,
01:18:27.860 --> 01:18:32.430
usually it's called Fourier
transform-- to that time domain
01:18:32.430 --> 01:18:34.950
vector, you get a new vector.
01:18:34.950 --> 01:18:36.826
Now it's a complex vector.
01:18:36.826 --> 01:18:38.450
You may have started
with real numbers.
01:18:38.450 --> 01:18:40.000
You get complex numbers.
01:18:40.000 --> 01:18:43.850
So for every position
in the vector--
01:18:43.850 --> 01:18:47.060
what it corresponds to-- the
x-axis is no longer time.
01:18:47.060 --> 01:18:48.360
Now it's frequency.
01:18:48.360 --> 01:18:51.390
For every frequency
you're measuring,
01:18:51.390 --> 01:18:53.510
essentially, you're
viewing this vector--
01:18:53.510 --> 01:18:57.130
the waveform-- as a bunch
of trigonometric functions--
01:18:57.130 --> 01:19:01.201
say, sine of
something times theta.
01:19:01.201 --> 01:19:03.200
If you look at one of the
entries in the vector,
01:19:03.200 --> 01:19:04.810
and it's a complex
number-- if you compute
01:19:04.810 --> 01:19:06.351
the magnitude of
the complex number--
01:19:06.351 --> 01:19:08.640
the length of the
vector, of the length
01:19:08.640 --> 01:19:13.695
of that two-coordinate vector--
that is how much stuff--
01:19:13.695 --> 01:19:15.900
of that frequency-- you have.
01:19:15.900 --> 01:19:18.630
And then the angle
of the vector, in 2D,
01:19:18.630 --> 01:19:23.670
is how that trigonometric
function shifted in time.
01:19:23.670 --> 01:19:27.900
So if you take a pure note,
like if I was playing a bell
01:19:27.900 --> 01:19:32.980
and it's exactly C major,
it looks really wavy.
01:19:32.980 --> 01:19:35.780
It's actually a
nice perfect sine
01:19:35.780 --> 01:19:39.127
curve-- some offset
depending on when I hit it--
01:19:39.127 --> 01:19:41.460
and then if you apply the
Fourier transform what you get
01:19:41.460 --> 01:19:44.790
is 0's everywhere except for the
one frequency that's appearing,
01:19:44.790 --> 01:19:48.540
and there you get 1, and
everywhere else you get 0.
01:19:48.540 --> 01:19:52.650
Well, 1 possibly rotated,
depending on the phase.
01:19:52.650 --> 01:19:55.410
And you can take
any audio stream,
01:19:55.410 --> 01:19:57.292
convert it by a
Fourier transform,
01:19:57.292 --> 01:19:58.250
do manipulations there.
01:19:58.250 --> 01:20:01.095
For example, you've probably
heard of high-pass filters that
01:20:01.095 --> 01:20:02.470
removes all the
high frequencies,
01:20:02.470 --> 01:20:04.400
or low-pass filters remove
all of the low frequencies.
01:20:04.400 --> 01:20:06.225
You just convert to
this space and zero out
01:20:06.225 --> 01:20:08.100
the parts you want, and
then you convert back
01:20:08.100 --> 01:20:10.230
with inverse Fourier transform.
01:20:10.230 --> 01:20:11.980
If you've used Adobe
Audition or Audacity,
01:20:11.980 --> 01:20:14.140
they can all do these things.
01:20:14.140 --> 01:20:15.720
And there are tons
of contexts where
01:20:15.720 --> 01:20:18.500
converting to Fourier
space makes life easy.
01:20:18.500 --> 01:20:20.790
And, in general, if you
have any time-based signal
01:20:20.790 --> 01:20:23.220
you should always think about
what do you get with FFT
01:20:23.220 --> 01:20:26.714
when you transform to
frequency-based signal,
01:20:26.714 --> 01:20:28.130
and you can do
lots of cool things
01:20:28.130 --> 01:20:29.213
you couldn't do otherwise.
01:20:29.213 --> 01:20:30.790
And it only takes n log n time.
01:20:30.790 --> 01:20:32.360
Plus people do it in hardware.
01:20:32.360 --> 01:20:33.860
There's a fast
implementation called
01:20:33.860 --> 01:20:37.640
FFTW-- the fastest Fourier
transform in the west-- which
01:20:37.640 --> 01:20:40.360
is made here at MIT
a bunch of years ago,
01:20:40.360 --> 01:20:43.390
and is still the best software
implementation of FFT.
01:20:43.390 --> 01:20:46.150
So people use it everywhere.
01:20:46.150 --> 01:20:49.720
Your noise-cancelling headsets
probably use it, MP3 uses it.
01:20:49.720 --> 01:20:51.146
It's a cool algorithm.