WEBVTT
00:00:00.090 --> 00:00:02.490
The following content is
provided under a Creative
00:00:02.490 --> 00:00:04.030
Commons license.
00:00:04.030 --> 00:00:06.330
Your support will help
MIT OpenCourseWare
00:00:06.330 --> 00:00:10.720
continue to offer high quality
educational resources for free.
00:00:10.720 --> 00:00:13.320
To make a donation or
view additional materials
00:00:13.320 --> 00:00:17.280
from hundreds of MIT courses,
visit MIT OpenCourseWare
00:00:17.280 --> 00:00:18.450
at ocw.mit.edu.
00:00:21.980 --> 00:00:25.022
[MUSIC PLAYING]
00:00:45.340 --> 00:00:46.880
ALAN OPPENHEIM: In
this lecture, we
00:00:46.880 --> 00:00:49.790
will begin the discussion
of the computation
00:00:49.790 --> 00:00:53.160
of the discrete
Fourier transform.
00:00:53.160 --> 00:00:55.940
This is an extremely
important topic
00:00:55.940 --> 00:00:59.490
in this set of lectures
for a number of reasons.
00:00:59.490 --> 00:01:03.800
One of the reasons is that the
principal algorithm that we'll
00:01:03.800 --> 00:01:06.890
be discussing, the
algorithm, which
00:01:06.890 --> 00:01:10.520
we'll refer to as the fast
Fourier transform algorithm,
00:01:10.520 --> 00:01:12.980
relies, for its
derivation, on many
00:01:12.980 --> 00:01:16.040
of the properties of the
discrete Fourier transform.
00:01:16.040 --> 00:01:19.610
And so, in fact,
developing this algorithm,
00:01:19.610 --> 00:01:23.240
provides us with an
opportunity to exercise
00:01:23.240 --> 00:01:27.260
some of the intuition and
some of the results related
00:01:27.260 --> 00:01:30.750
to discrete Fourier transforms.
00:01:30.750 --> 00:01:33.080
Of much more
practical importance
00:01:33.080 --> 00:01:36.600
is the fact that the
fast Fourier transform
00:01:36.600 --> 00:01:39.180
algorithm, which
we'll be presenting,
00:01:39.180 --> 00:01:43.350
is a highly efficient
algorithm for computing
00:01:43.350 --> 00:01:45.660
the discrete Fourier transform.
00:01:45.660 --> 00:01:49.470
And in fact, as we'll
see, it is orders
00:01:49.470 --> 00:01:53.970
of magnitude more efficient
than a straightforward direct
00:01:53.970 --> 00:01:57.500
computation of the
discrete Fourier transform.
00:01:57.500 --> 00:02:01.980
Now, We've indicated
in previous lectures,
00:02:01.980 --> 00:02:05.880
and you can well imagine
that this is true,
00:02:05.880 --> 00:02:09.120
that there are many
contexts in which we
00:02:09.120 --> 00:02:13.470
would want to explicitly compute
the discrete Fourier transform.
00:02:13.470 --> 00:02:16.710
For example, obviously,
in spectral analysis
00:02:16.710 --> 00:02:19.260
of discrete time
data, we would want
00:02:19.260 --> 00:02:23.400
to use the computation of the
discrete Fourier transform.
00:02:23.400 --> 00:02:28.200
And we've seen also that the
discrete Fourier transform can
00:02:28.200 --> 00:02:32.430
be used for the implementation
of finite impulse response
00:02:32.430 --> 00:02:35.160
digital filters, or more
generally, can be used
00:02:35.160 --> 00:02:38.440
to implement a convolution.
00:02:38.440 --> 00:02:44.650
It will turn out that because of
the efficiency of the algorithm
00:02:44.650 --> 00:02:47.870
that we'll be discussing
in these lectures, it,
00:02:47.870 --> 00:02:54.040
in fact, in many cases is
more efficient to implement
00:02:54.040 --> 00:02:58.510
the convolution by explicitly
computing a discrete Fourier
00:02:58.510 --> 00:03:01.870
transform, multiplying
the transforms together,
00:03:01.870 --> 00:03:05.210
and then Inverse transforming
the result. Of course,
00:03:05.210 --> 00:03:07.540
keeping track of the
fact that that implements
00:03:07.540 --> 00:03:10.600
a circular convolution and
doing the necessary things that
00:03:10.600 --> 00:03:12.700
need to be done if
we're interested
00:03:12.700 --> 00:03:15.840
in a linear convolution.
00:03:15.840 --> 00:03:19.190
Now, I'd like, just
briefly, to remark
00:03:19.190 --> 00:03:22.490
that this algorithm,
this-- actually, it's
00:03:22.490 --> 00:03:24.260
a class of
algorithms, which I'll
00:03:24.260 --> 00:03:28.800
refer to as the fast Fourier
transform algorithms,
00:03:28.800 --> 00:03:33.150
were a major landmark
in the development
00:03:33.150 --> 00:03:37.770
of the field of digital signal
processing in its present form.
00:03:37.770 --> 00:03:41.400
The algorithm, one of the
forms of the algorithm,
00:03:41.400 --> 00:03:47.230
was disclosed in a paper by
Cooley and Tukey in 1965.
00:03:47.230 --> 00:03:52.380
And it turns out, though,
that the algorithm
00:03:52.380 --> 00:03:56.160
has, in a variety of forms,
been reinvented several times
00:03:56.160 --> 00:03:58.870
over the past 20 or 30 years.
00:03:58.870 --> 00:04:01.860
So it is mainly this
class of algorithms
00:04:01.860 --> 00:04:05.640
that we'll be talking about
in the next several lectures.
00:04:05.640 --> 00:04:12.700
And I stress again that one
of the primary reasons why
00:04:12.700 --> 00:04:17.200
this is so important is
because of the rather dramatic
00:04:17.200 --> 00:04:19.480
improvements in
efficiency that it
00:04:19.480 --> 00:04:21.850
offers over just
a straightforward
00:04:21.850 --> 00:04:25.630
computation of the
discrete Fourier transform.
00:04:25.630 --> 00:04:28.390
Well, let's look
specifically at what's
00:04:28.390 --> 00:04:34.070
involved in the computation of
the discrete Fourier transform.
00:04:34.070 --> 00:04:36.880
The discrete Fourier
transform, as we know,
00:04:36.880 --> 00:04:41.980
is given by the sum of
x sub n times W sub n
00:04:41.980 --> 00:04:47.470
to the nk, where W sub n is
the complex exponential, e
00:04:47.470 --> 00:04:51.400
to the minus j, 2
pi over capital n.
00:04:51.400 --> 00:04:56.620
So it is this expression
that we want to compute.
00:04:56.620 --> 00:05:00.340
Or equivalently, there is
the inverse discrete Fourier
00:05:00.340 --> 00:05:04.390
transform given by
1 over capital N
00:05:04.390 --> 00:05:07.930
times the sum of
x of k, the DFT,
00:05:07.930 --> 00:05:11.320
times W sub N to the minus nk.
00:05:11.320 --> 00:05:14.680
Now, hopefully you recall
from the discussion
00:05:14.680 --> 00:05:16.960
of the discrete
Fourier transform,
00:05:16.960 --> 00:05:21.100
that the inverse DFT
is really the same
00:05:21.100 --> 00:05:24.280
as the discrete Fourier
transform if we reverse
00:05:24.280 --> 00:05:26.200
the result in N, et cetera.
00:05:26.200 --> 00:05:31.420
In other words, computation
of the inverse DFT
00:05:31.420 --> 00:05:38.020
is really essentially the same
as the computation of the DFT.
00:05:38.020 --> 00:05:40.690
So whereas we'll be
referring the discussion
00:05:40.690 --> 00:05:44.380
to the computation of the
discrete Fourier transform,
00:05:44.380 --> 00:05:46.690
essentially
everything that we say
00:05:46.690 --> 00:05:53.300
applies also to a computation
of the inverse DFT.
00:05:53.300 --> 00:05:56.360
Well, let's look,
first of all, at what
00:05:56.360 --> 00:05:59.810
would be involved in
just a straightforward
00:05:59.810 --> 00:06:03.200
computation of the
discrete Fourier transform.
00:06:03.200 --> 00:06:08.570
Basically what this expression
says is to take x of n,
00:06:08.570 --> 00:06:11.810
multiply it by the
appropriate powers of W,
00:06:11.810 --> 00:06:16.810
and then sum the result
from 0 to capital N minus 1.
00:06:16.810 --> 00:06:19.010
And so, we basically
have that set
00:06:19.010 --> 00:06:24.920
of operations to carry out for
each value of k for which we're
00:06:24.920 --> 00:06:28.190
computing the discrete
Fourier transform.
00:06:28.190 --> 00:06:31.040
Well, if we think of x
of n as being complex,
00:06:31.040 --> 00:06:34.190
which in the most
general case it will be,
00:06:34.190 --> 00:06:42.440
then we have in the direct
computation of the DFT,
00:06:42.440 --> 00:06:48.440
we have to form the product, x
of n times W sub N to the nk.
00:06:48.440 --> 00:06:54.860
And what that involves is
one complex multiplication.
00:06:54.860 --> 00:06:56.190
This is a complex number.
00:06:56.190 --> 00:06:57.800
That's a complex number.
00:06:57.800 --> 00:07:01.400
Multiplying the two is
then one complex multiply,
00:07:01.400 --> 00:07:06.980
or equivalently, four real
multiplies and two real adds.
00:07:09.680 --> 00:07:15.410
OK, well now that is to
form one term in the sum.
00:07:15.410 --> 00:07:22.680
We have n terms altogether
in the sum, or capital N
00:07:22.680 --> 00:07:25.800
minus one additions,
complex additions,
00:07:25.800 --> 00:07:27.990
to implement in the sum.
00:07:27.990 --> 00:07:35.680
So to compute the DFT
for one value of k
00:07:35.680 --> 00:07:41.350
will require capital N complex
multiplications and capital
00:07:41.350 --> 00:07:45.100
N minus 1 complex additions.
00:07:45.100 --> 00:07:48.130
Well to compute the
discrete Fourier transform
00:07:48.130 --> 00:07:53.990
for all capital N values
of k requires then
00:07:53.990 --> 00:07:58.810
n squared complex
multiplies and n times
00:07:58.810 --> 00:08:01.940
n minus 1 complex additions.
00:08:01.940 --> 00:08:08.200
Or basically if capital N
is any reasonable number,
00:08:08.200 --> 00:08:11.380
we can approximate
this as N squared.
00:08:11.380 --> 00:08:15.730
And so, approximately,
the computation involved
00:08:15.730 --> 00:08:21.460
is on the order of capital
N squared complex multiplies
00:08:21.460 --> 00:08:23.680
and adds.
00:08:23.680 --> 00:08:25.530
Now, a few remarks here.
00:08:25.530 --> 00:08:29.590
One, first of all, is
that in counting up
00:08:29.590 --> 00:08:32.650
the number of complex
multiplies and adds,
00:08:32.650 --> 00:08:38.140
we didn't particularly take
account of multiplies by unity.
00:08:38.140 --> 00:08:42.390
For example, when k is equal
to 0, W sub N to the nk
00:08:42.390 --> 00:08:44.800
is equal to unity,
and we, in fact,
00:08:44.800 --> 00:08:48.400
wouldn't have to carry
out a multiplication.
00:08:48.400 --> 00:08:52.660
But there are a
small enough number
00:08:52.660 --> 00:08:56.920
of those cases in comparison to
all of the multiplies involved
00:08:56.920 --> 00:09:00.760
so that this is generally
an indication of what
00:09:00.760 --> 00:09:04.990
the dependence of the
computation is on N.
00:09:04.990 --> 00:09:09.610
Second of all, and this will
occur throughout this next set
00:09:09.610 --> 00:09:13.810
of lectures, we are
tying a discussion
00:09:13.810 --> 00:09:19.030
of the computational complexity
or efficiency to counting
00:09:19.030 --> 00:09:22.390
the number of complex
multiplies and adds
00:09:22.390 --> 00:09:26.020
that are involved in the
computation of x of k,
00:09:26.020 --> 00:09:29.680
and not particularly focusing on
other issues, such as indexing,
00:09:29.680 --> 00:09:31.270
bookkeeping, et cetera.
00:09:31.270 --> 00:09:34.060
So in discussing the
computational complexity
00:09:34.060 --> 00:09:37.510
or efficiency,
we'll tend to refer
00:09:37.510 --> 00:09:41.050
to the number of
multiplications and adds
00:09:41.050 --> 00:09:45.320
that are involved in
implementing the transform.
00:09:45.320 --> 00:09:48.980
Well, we have then a
direct computation,
00:09:48.980 --> 00:09:53.180
which involves on the order of
n squared complex multiplies
00:09:53.180 --> 00:09:54.590
and adds.
00:09:54.590 --> 00:09:59.000
And the important point
then is its dependency on N,
00:09:59.000 --> 00:10:01.190
namely N squared.
00:10:01.190 --> 00:10:03.830
There are a variety
of algorithms
00:10:03.830 --> 00:10:07.340
that have been known
for some time, in fact,
00:10:07.340 --> 00:10:10.040
and ones that you
can generate yourself
00:10:10.040 --> 00:10:13.430
by taking advantage of
various symmetry properties
00:10:13.430 --> 00:10:16.220
of these complex exponentials.
00:10:16.220 --> 00:10:21.200
To reduce the constant
of proportionality
00:10:21.200 --> 00:10:26.960
on the computation, that is,
to make the computation perhaps
00:10:26.960 --> 00:10:31.820
not involve N squared multiplies
and adds but only 1/2 N squared
00:10:31.820 --> 00:10:33.650
or 1/4 N squared.
00:10:33.650 --> 00:10:37.940
But still, in cases like
that, the algorithms
00:10:37.940 --> 00:10:40.460
involved basically
fooling around
00:10:40.460 --> 00:10:45.530
with a constant of
proportionality out in front,
00:10:45.530 --> 00:10:50.360
whereas the main dependence on
N, the size of the transform,
00:10:50.360 --> 00:10:53.430
remains the same.
00:10:53.430 --> 00:10:57.180
And you can, I think, convince
yourself rather quickly
00:10:57.180 --> 00:11:03.000
with some fairly
standard computers
00:11:03.000 --> 00:11:07.650
that if, in fact, the
computation to be done
00:11:07.650 --> 00:11:12.690
is proportional to N squared,
then very large transforms
00:11:12.690 --> 00:11:15.810
certainly aren't possible
in very reasonable amounts
00:11:15.810 --> 00:11:17.230
of time.
00:11:17.230 --> 00:11:20.370
For example, if N is
on the order of 1,000,
00:11:20.370 --> 00:11:23.340
then we're talking about
on the order of a million
00:11:23.340 --> 00:11:25.980
complex multiplies and adds.
00:11:25.980 --> 00:11:29.310
And if we assume that
one of those took,
00:11:29.310 --> 00:11:33.120
say, two microseconds or three
microseconds to implement,
00:11:33.120 --> 00:11:35.820
or even one microsecond,
then clearly we're
00:11:35.820 --> 00:11:38.790
talking about computation
on the order of seconds.
00:11:38.790 --> 00:11:41.220
And you can convince
yourself rather quickly
00:11:41.220 --> 00:11:46.920
that the computation grows very
fast as N increases from 1,000
00:11:46.920 --> 00:11:50.220
to 2,000 to 4,000, et cetera.
00:11:50.220 --> 00:11:52.920
All right, so direct
computation, the dependency
00:11:52.920 --> 00:11:56.190
is an n squared
type of dependency.
00:11:56.190 --> 00:12:00.450
And this is in contrast
to the primary algorithm
00:12:00.450 --> 00:12:03.960
that we want to discuss, which
is the fast Fourier transform
00:12:03.960 --> 00:12:06.480
algorithm.
00:12:06.480 --> 00:12:11.940
Basically, the fast
Fourier transform algorithm
00:12:11.940 --> 00:12:18.450
is directed at computing the
discrete Fourier transform
00:12:18.450 --> 00:12:24.630
for a choice of N, the
size of the transform,
00:12:24.630 --> 00:12:26.100
highly composite.
00:12:26.100 --> 00:12:29.940
That is, N is, in general--
00:12:29.940 --> 00:12:32.100
of course, any N
can be represented
00:12:32.100 --> 00:12:34.110
as a product of primes.
00:12:34.110 --> 00:12:38.370
And as we'll see, the
larger the number of primes
00:12:38.370 --> 00:12:41.430
involved in the
decomposition of N,
00:12:41.430 --> 00:12:44.940
the greater the efficiency
of the fast Fourier
00:12:44.940 --> 00:12:48.840
transform as an algorithm for
computing the discrete Fourier
00:12:48.840 --> 00:12:50.810
transform.
00:12:50.810 --> 00:12:56.930
In particular, we'll see that
for N, a product of primes,
00:12:56.930 --> 00:13:00.860
using the fast Fourier
transform algorithm,
00:13:00.860 --> 00:13:04.760
the number of complex
multiplies and additions
00:13:04.760 --> 00:13:10.100
involved will turn out to
be proportional not to N
00:13:10.100 --> 00:13:13.620
squared as it is in
the direct computation,
00:13:13.620 --> 00:13:18.500
but proportional to N
times a sum of the primes
00:13:18.500 --> 00:13:24.530
involved in that decomposition
of N. So the greater the number
00:13:24.530 --> 00:13:28.640
of primes that we can decompose
N in and the smaller each
00:13:28.640 --> 00:13:32.930
of those primes, the higher the
efficiency of the fast Fourier
00:13:32.930 --> 00:13:36.691
transform algorithm will be.
00:13:36.691 --> 00:13:37.190
Well,
00:13:37.190 --> 00:13:42.350
The most highly composite that
we can make N, and at the same
00:13:42.350 --> 00:13:46.700
time keeping the primes,
p, as small as possible,
00:13:46.700 --> 00:13:50.760
is to choose N to
be a power of 2.
00:13:50.760 --> 00:13:54.550
So that N is of the
form 2 the nu, where
00:13:54.550 --> 00:13:58.520
nu is the log to the base 2
of N. And it will turn out,
00:13:58.520 --> 00:14:02.570
in that case, that the complex
multiplies and additions
00:14:02.570 --> 00:14:07.280
is proportional to N
times log to the base 2 N,
00:14:07.280 --> 00:14:09.530
rather than, in the
direct computation,
00:14:09.530 --> 00:14:11.900
proportional to N squared.
00:14:11.900 --> 00:14:14.990
Let me incidentally
comment that although I've
00:14:14.990 --> 00:14:19.010
referred to these p's as
primes, they, in fact,
00:14:19.010 --> 00:14:20.750
don't have to be primes.
00:14:20.750 --> 00:14:23.280
The important
issue, as we'll see,
00:14:23.280 --> 00:14:27.050
is that N be representable
as a product of numbers.
00:14:27.050 --> 00:14:28.550
And it's not
particularly important
00:14:28.550 --> 00:14:32.600
that those numbers
turn out to be primes.
00:14:32.600 --> 00:14:39.160
Well, OK, so this then
is the kind of efficiency
00:14:39.160 --> 00:14:43.420
that we will get out of the fast
Fourier transform algorithm.
00:14:43.420 --> 00:14:47.530
And let me begin the
discussion of the algorithm
00:14:47.530 --> 00:14:52.560
by concentrating on the case
in which N is a power of 2.
00:14:52.560 --> 00:14:59.540
Well, basically, the idea behind
the fast Fourier transform
00:14:59.540 --> 00:15:04.460
algorithm is to
decompose the computation
00:15:04.460 --> 00:15:08.200
of the discrete
Fourier transform
00:15:08.200 --> 00:15:12.740
into a set of subsequences.
00:15:12.740 --> 00:15:18.950
And this is-- this
generates a class of forms
00:15:18.950 --> 00:15:21.110
for the algorithm,
which are referred
00:15:21.110 --> 00:15:25.280
to as decimation-in-time
forms of the algorithm.
00:15:25.280 --> 00:15:29.210
In particular, suppose
that we consider
00:15:29.210 --> 00:15:33.170
decomposing the
original sequence x of n
00:15:33.170 --> 00:15:35.330
into two subsequences.
00:15:35.330 --> 00:15:38.060
One sequence,
subsequence, consists
00:15:38.060 --> 00:15:41.690
of the even-numbered
points of x of n.
00:15:41.690 --> 00:15:44.450
And the second
subsequence consists
00:15:44.450 --> 00:15:47.390
of the odd-numbered
points of x of n.
00:15:47.390 --> 00:15:51.020
Well, let's just do that
and see where it gets us.
00:15:51.020 --> 00:15:55.610
We can then rewrite the
discrete Fourier transform
00:15:55.610 --> 00:16:01.070
as a sum over n even
of x of n times W sub N
00:16:01.070 --> 00:16:05.950
to the nk plus the
sum over n odd of x
00:16:05.950 --> 00:16:08.750
of n W sub N to the nk.
00:16:08.750 --> 00:16:13.280
Simply pulling out from this
the even-numbered values
00:16:13.280 --> 00:16:16.976
and the odd-numbered values.
00:16:16.976 --> 00:16:19.980
All right, now by a
substitution of variables,
00:16:19.980 --> 00:16:28.260
we can substitute for n 2 r
in the even-numbered terms.
00:16:28.260 --> 00:16:31.950
And we can substitute
n equals 2 r plus 1
00:16:31.950 --> 00:16:35.170
for the odd-number
terms, in which case,
00:16:35.170 --> 00:16:39.780
when we convert these two
sums over r, the sum on r
00:16:39.780 --> 00:16:44.700
will now range from 0 to
capital N over 2 minus 1.
00:16:44.700 --> 00:16:48.300
As r runs from 0 to
capital N over 2 minus 1,
00:16:48.300 --> 00:16:52.740
2 r ranges over the
even-numbered terms, and 2 r
00:16:52.740 --> 00:16:57.250
plus 1 ranges over the
odd-numbered terms.
00:16:57.250 --> 00:17:01.650
All right, so making the
substitution of variables then,
00:17:01.650 --> 00:17:08.770
this sum is converted
into a sum from r equals 0
00:17:08.770 --> 00:17:11.186
to N over 2 minus 1.
00:17:11.186 --> 00:17:16.400
x of 2r W sub N to
the 2rk plus a sum--
00:17:16.400 --> 00:17:18.640
those are the
even-numbered terms--
00:17:18.640 --> 00:17:22.510
plus a sum over the
odd-numbered terms from 0
00:17:22.510 --> 00:17:26.109
to n over 2 minus
1. x of 2r plus 1,
00:17:26.109 --> 00:17:33.760
the odd-numbered points, times W
sub N to the 2r plus 1 times k.
00:17:33.760 --> 00:17:35.820
Well, let's look at this factor.
00:17:35.820 --> 00:17:39.220
W sub N to the 2r
plus 1 times k,
00:17:39.220 --> 00:17:46.170
that's equal to W sub N to
the k, because of the 1,
00:17:46.170 --> 00:17:50.080
and W sub N to the 2rk.
00:17:50.080 --> 00:17:53.530
So we can, first of
all, break this term up
00:17:53.530 --> 00:17:56.560
into a product of
these two terms.
00:17:56.560 --> 00:18:01.420
And now, let's look at this
one, W sub N to the 2rk.
00:18:01.420 --> 00:18:07.055
Well, W sub N squared is
equal to e to the minus j
00:18:07.055 --> 00:18:10.410
2 pi over capital N squared.
00:18:10.410 --> 00:18:12.960
So the factor of 2.
00:18:12.960 --> 00:18:14.960
If we take the 2 down
in the denominator
00:18:14.960 --> 00:18:18.440
and associate it with the
N, we can rewrite this as e
00:18:18.440 --> 00:18:23.820
to the minus j 2 pi over N/2.
00:18:23.820 --> 00:18:31.540
So what W sub N squared is equal
to is W sub capital N over 2.
00:18:31.540 --> 00:18:34.680
In other words, the W that we
would associate with an N/2
00:18:34.680 --> 00:18:38.580
point discrete
Fourier transform.
00:18:38.580 --> 00:18:43.470
Finally, substituting
this together
00:18:43.470 --> 00:18:47.470
with this statement back
into this expression,
00:18:47.470 --> 00:18:55.240
we have, for x of k, the
sum from 0 to N/2 minus 1,
00:18:55.240 --> 00:19:01.125
x of 2r W sub N over
2 to the rk plus W sub
00:19:01.125 --> 00:19:07.050
N to the k times the sum of
x of 2r plus 1 W sub N over 2
00:19:07.050 --> 00:19:09.630
to the rk.
00:19:09.630 --> 00:19:14.610
Well, what this looks like is
an N/2 point discrete Fourier
00:19:14.610 --> 00:19:16.420
transform.
00:19:16.420 --> 00:19:18.850
And what this looks
like is an N/2
00:19:18.850 --> 00:19:20.320
point discrete
Fourier transform.
00:19:20.320 --> 00:19:22.550
And in fact, that's
what they are.
00:19:22.550 --> 00:19:27.880
This is an N/2 point DFT
of the even-numbered points
00:19:27.880 --> 00:19:30.100
in a sequence x of n.
00:19:30.100 --> 00:19:35.560
This is an N/2 point of
the odd-numbered points
00:19:35.560 --> 00:19:37.870
in the sequence x of n.
00:19:37.870 --> 00:19:41.110
So this says we
can compute x of k
00:19:41.110 --> 00:19:47.680
by computing an N/2 point
DFT, another N/2 point DFT,
00:19:47.680 --> 00:19:51.010
and combining those two
together with this factor
00:19:51.010 --> 00:19:54.520
out in front, W sub N to the k.
00:19:54.520 --> 00:19:56.350
Now, it's not
particularly obvious
00:19:56.350 --> 00:19:58.450
yet that, in fact,
in doing this,
00:19:58.450 --> 00:20:00.980
we've saved something
in computation.
00:20:00.980 --> 00:20:04.600
But let's look at
what this implies
00:20:04.600 --> 00:20:08.650
in terms of the computational
steps in a little more detail.
00:20:11.210 --> 00:20:20.567
So here we have, again,
X of k as an N/2 point
00:20:20.567 --> 00:20:28.020
DFT involving the
even-numbered points in x of n
00:20:28.020 --> 00:20:33.110
plus an N/2 point DFT involving
the odd-numbered points in x
00:20:33.110 --> 00:20:34.590
of n.
00:20:34.590 --> 00:20:37.620
Those are both N/2 point DFTs.
00:20:37.620 --> 00:20:41.250
This one, if we
refer to as G of k,
00:20:41.250 --> 00:20:44.670
this one, if we
refer to as H of k,
00:20:44.670 --> 00:20:48.240
then the computation
of the DFT, X of k
00:20:48.240 --> 00:20:54.790
is G of k plus W sub N
to the k times H of k.
00:20:54.790 --> 00:20:56.410
All right, let's
see what this means
00:20:56.410 --> 00:21:01.180
in terms of computation,
computational complexity.
00:21:01.180 --> 00:21:05.910
We have to compute
an N/2 point DFT.
00:21:05.910 --> 00:21:09.240
We just argued that
an endpoint DFT
00:21:09.240 --> 00:21:14.950
involves N squared complex
multiplies and adds.
00:21:14.950 --> 00:21:18.260
So the computation
of G of k involves
00:21:18.260 --> 00:21:23.010
N/2 squared complex
multiplies and adds.
00:21:23.010 --> 00:21:27.900
The computation of H of
k involves, likewise,
00:21:27.900 --> 00:21:32.010
N/2 squared complex
multiplies and adds.
00:21:32.010 --> 00:21:36.710
So we have then 2 times
N/2 squared multiplies
00:21:36.710 --> 00:21:44.400
and adds due to the computation
of these two N/2 point DFT.
00:21:44.400 --> 00:21:47.970
In addition, we have to
combine these together
00:21:47.970 --> 00:21:51.810
by multiplying by
W sub N to the k.
00:21:51.810 --> 00:21:56.010
That involves capital
N complex multiplies
00:21:56.010 --> 00:22:00.430
and adds, because k ranges
from 0 to N minus 1.
00:22:00.430 --> 00:22:04.570
In other words, k ranges
over N different values.
00:22:04.570 --> 00:22:08.580
So there are N complex
multiplies involved in this,
00:22:08.580 --> 00:22:12.610
and then N complex adds in
combining these together.
00:22:12.610 --> 00:22:16.200
So in addition to the
computation of the two N/2
00:22:16.200 --> 00:22:20.430
point DFTs, we have the
combination of them.
00:22:20.430 --> 00:22:26.880
And the resulting computation
involves N plus N squared over
00:22:26.880 --> 00:22:30.720
2 complex multiplies and adds.
00:22:30.720 --> 00:22:33.540
And you can convince yourself
on the back of an envelope
00:22:33.540 --> 00:22:37.350
very quickly that this
number will always turn out
00:22:37.350 --> 00:22:39.810
to be less than N
squared, except, I
00:22:39.810 --> 00:22:46.410
think, for some trivial cases
like N equals 1 or N equals 2.
00:22:46.410 --> 00:22:51.490
Now, one minor
point to be made--
00:22:51.490 --> 00:22:53.320
in combining these
two together, I
00:22:53.320 --> 00:22:58.080
commented that k has to run
from 0 to capital N minus 1.
00:22:58.080 --> 00:23:02.890
That is, there are N different
values that k ranges over,
00:23:02.890 --> 00:23:08.050
so doesn't that say that
in computing G of k,
00:23:08.050 --> 00:23:13.120
I can't just compute it from
k equals 0 to n over 2 minus 1
00:23:13.120 --> 00:23:16.050
as I normally would
in computing a DFT?
00:23:16.050 --> 00:23:19.870
Don't I have to compute it
from 0 all the way up to N
00:23:19.870 --> 00:23:22.050
or all the way up to N minus 1?
00:23:22.050 --> 00:23:23.620
Well, sure I do.
00:23:23.620 --> 00:23:26.440
I do, because those
are the values that I
00:23:26.440 --> 00:23:29.110
need in here and in here.
00:23:29.110 --> 00:23:32.380
But what does that imply
in terms of computation?
00:23:32.380 --> 00:23:41.550
Well, we know that an N/2
point DFT is periodic in k,
00:23:41.550 --> 00:23:47.070
so that if we compute
only the first N/2 points,
00:23:47.070 --> 00:23:49.230
we know that the
next N/2 points are
00:23:49.230 --> 00:23:51.000
going to look exactly the same.
00:23:51.000 --> 00:23:53.100
And so, there's no
point in computing them.
00:23:53.100 --> 00:23:57.330
So the point then is
that in computing G of k
00:23:57.330 --> 00:24:00.450
for this computation,
even though we want k
00:24:00.450 --> 00:24:04.410
to run from 0 to
N minus 1, we only
00:24:04.410 --> 00:24:08.610
need to explicitly do
this computation for k
00:24:08.610 --> 00:24:11.170
from 0 to N over 2 minus 1.
00:24:11.170 --> 00:24:14.490
And then we simply use
those values over again
00:24:14.490 --> 00:24:19.320
for the second range from
k from N/2 to N minus 1.
00:24:19.320 --> 00:24:23.820
And that, by the way,
is the essential key
00:24:23.820 --> 00:24:27.300
in the fast Fourier
transform algorithm.
00:24:27.300 --> 00:24:31.080
It's basically that periodicity
in the discrete Fourier
00:24:31.080 --> 00:24:33.270
transform that will
lead to the kind
00:24:33.270 --> 00:24:36.630
of computational efficiency
that's about to result.
00:24:36.630 --> 00:24:40.260
All right, so let's look
then at how we would actually
00:24:40.260 --> 00:24:44.970
do the computation of the
discrete Fourier transform.
00:24:44.970 --> 00:24:50.430
We know then that we will
break the computation
00:24:50.430 --> 00:24:57.830
into a computation involving
the even-numbered points and x
00:24:57.830 --> 00:25:02.880
of n and the odd-numbered
points and x of n.
00:25:02.880 --> 00:25:06.750
Compute an N/2 point
discrete Fourier transform
00:25:06.750 --> 00:25:10.785
to obtain the sequence
G of k, another N/2
00:25:10.785 --> 00:25:15.090
point discrete Fourier transform
to compute the sequence H of k,
00:25:15.090 --> 00:25:19.740
and then combine the results
together by multiplying H of k
00:25:19.740 --> 00:25:24.510
by W sub N to the k and
adding that to G of k.
00:25:24.510 --> 00:25:28.140
I'll illustrate the
computation, incidentally,
00:25:28.140 --> 00:25:31.920
with choosing N
equal to 8, simply
00:25:31.920 --> 00:25:34.450
because I have to choose
N equal to something.
00:25:34.450 --> 00:25:36.690
And it turns out,
interestingly enough,
00:25:36.690 --> 00:25:41.880
that N equals 8 is
sufficiently general,
00:25:41.880 --> 00:25:45.300
so that you can infer
the generality for N
00:25:45.300 --> 00:25:48.120
equal to an arbitrary
power of 2 just
00:25:48.120 --> 00:25:51.420
from the computational
issues that we'll
00:25:51.420 --> 00:25:53.760
raise for this particular case.
00:25:53.760 --> 00:25:56.850
All right, so we
have N/2 point DFT
00:25:56.850 --> 00:25:59.670
of the even-numbered
points, the N/2
00:25:59.670 --> 00:26:03.210
point DFT of the
odd-numbered points.
00:26:03.210 --> 00:26:08.280
And now, to compute the output
points, the discrete Fourier
00:26:08.280 --> 00:26:18.450
transform, X of k, we
then want to take G of 0
00:26:18.450 --> 00:26:23.790
and add it to H of 0
multiplied by W sub N to the 0.
00:26:23.790 --> 00:26:27.300
So using flow graph
notation, we have
00:26:27.300 --> 00:26:36.370
G of 0 added to H of 0
multiplied by W sub N so the 0.
00:26:36.370 --> 00:26:41.950
For x of 1, we want G
of 1 added to H of 1
00:26:41.950 --> 00:26:45.830
multiplied by W sub N to the 1.
00:26:45.830 --> 00:26:51.050
For x of 2, we want
G of 2 plus H of 2
00:26:51.050 --> 00:26:53.630
multiplied by W sub N squared.
00:26:53.630 --> 00:26:58.010
And for x of 3,
likewise, G of 3 added
00:26:58.010 --> 00:27:01.970
to H of 3 multiplied
by W sub N cubed.
00:27:01.970 --> 00:27:09.650
So that computation, then,
gives us x of 0 through x of 3.
00:27:09.650 --> 00:27:17.420
For x of 4, we want G of 4 plus
W sub N to the 4 times H of 4.
00:27:17.420 --> 00:27:19.190
But what's G of 4?
00:27:19.190 --> 00:27:24.950
G of 4 is equal to G of 0,
because of the fact that G of k
00:27:24.950 --> 00:27:27.560
is periodic with period 4.
00:27:27.560 --> 00:27:30.190
This is a four-point DFT
that we're computing.
00:27:30.190 --> 00:27:33.350
So G of k is periodic
with period 4.
00:27:33.350 --> 00:27:40.100
And consequently, H of 4 and
G of 4 are equal to H of 0
00:27:40.100 --> 00:27:41.870
and G of 0.
00:27:41.870 --> 00:27:47.000
So to compute the
next four points,
00:27:47.000 --> 00:28:05.560
we can likewise use the outputs
of these N/2 point DFTs.
00:28:05.560 --> 00:28:10.240
To compute x of 4, we want G
of 4, which is equal to G of 0,
00:28:10.240 --> 00:28:15.520
because of the periodicity,
plus W sub N to 4 times H of 4.
00:28:15.520 --> 00:28:17.830
H of 4 is equal to H of 0.
00:28:17.830 --> 00:28:22.750
So we have then for x of
4, G of 0 plus W sub N
00:28:22.750 --> 00:28:25.730
to 4 times H of 0.
00:28:25.730 --> 00:28:32.440
Likewise, for x of 5, we
have G of 5, or G of 1,
00:28:32.440 --> 00:28:37.390
plus W sub N to the 5th
times H of 5, or H of 1
00:28:37.390 --> 00:28:40.390
because of the periodicity,
and likewise, down
00:28:40.390 --> 00:28:42.560
through the remaining
four points.
00:28:42.560 --> 00:28:46.060
So we showed previous--
on the previous view
00:28:46.060 --> 00:28:49.310
graph, the computation
for the first four points.
00:28:49.310 --> 00:28:53.140
And here we have the computation
for the last four points.
00:28:53.140 --> 00:28:56.920
Putting those two together
then, the computation
00:28:56.920 --> 00:29:06.040
for the eight-point DFT
is, as I've indicated here,
00:29:06.040 --> 00:29:08.080
the even-numbered
points into an N/2 point
00:29:08.080 --> 00:29:13.630
DFT, the odd-numbered points
into an N/2 point DFT,
00:29:13.630 --> 00:29:16.870
and the results
combined together
00:29:16.870 --> 00:29:20.600
according to the computation
that we've shown here.
00:29:20.600 --> 00:29:22.810
Now, a couple of
things that I'd like
00:29:22.810 --> 00:29:26.070
to draw your attention to
about the flow graph as it's
00:29:26.070 --> 00:29:27.430
developing.
00:29:27.430 --> 00:29:30.790
First of all, note
that the flow graph,
00:29:30.790 --> 00:29:35.500
as we're looking at it here,
involves a computation, which
00:29:35.500 --> 00:29:39.790
if we were to just pull out
the computation of these two
00:29:39.790 --> 00:29:42.550
points, or these two
points, et cetera,
00:29:42.550 --> 00:29:49.450
it has the structure of a set of
butterfly-looking flow graphs.
00:29:49.450 --> 00:29:52.390
And, in fact, we'll find
it convenient to refer
00:29:52.390 --> 00:29:57.680
to these as butterflies
in the computation.
00:29:57.680 --> 00:30:00.620
The second thing that I'd
like to draw your attention to
00:30:00.620 --> 00:30:07.110
is that the multiplications
by powers of W
00:30:07.110 --> 00:30:13.020
always are applied to
the branches which come
00:30:13.020 --> 00:30:14.970
from this bottom transform.
00:30:14.970 --> 00:30:18.390
And that's because we had
the expression that X of k
00:30:18.390 --> 00:30:23.460
was equal to G of k plus W
sub N to the k times H of k,
00:30:23.460 --> 00:30:25.560
and this was the H of k.
00:30:25.560 --> 00:30:30.390
So it's in the bottom
branch of these butterflies
00:30:30.390 --> 00:30:34.410
that we see always
the multiplication
00:30:34.410 --> 00:30:37.440
by these powers of
W. And I say that, I
00:30:37.440 --> 00:30:41.220
want to point that out hit
now, because it's somewhat
00:30:41.220 --> 00:30:44.730
difficult in drawing these
flow graphs to always get
00:30:44.730 --> 00:30:48.610
these powers of W close
enough to the arrow
00:30:48.610 --> 00:30:50.970
so that it's clear where
the computation, where
00:30:50.970 --> 00:30:54.110
the multiplication
needs to be carried out.
00:30:54.110 --> 00:30:56.330
All right, so here's
where we're at.
00:30:56.330 --> 00:30:59.150
We had counted out
previously the number
00:30:59.150 --> 00:31:01.850
of complex multiplies
and adds involved,
00:31:01.850 --> 00:31:07.280
and we found that it was on
the order of N plus 2 times
00:31:07.280 --> 00:31:12.080
and over 2 squared, which
is less than N squared.
00:31:12.080 --> 00:31:14.960
And consequently, this
is a more efficient way
00:31:14.960 --> 00:31:18.380
to compute the discrete
Fourier transform than simply
00:31:18.380 --> 00:31:20.540
a direct computation.
00:31:20.540 --> 00:31:26.610
However, like this
assumes that we're
00:31:26.610 --> 00:31:33.130
going to compute these N/2 point
DFTs as a direct computation.
00:31:33.130 --> 00:31:36.990
But in fact, if N/2
is a composite number,
00:31:36.990 --> 00:31:39.420
in particular a
power of 2, which
00:31:39.420 --> 00:31:42.630
it is if N was a
power of 2, then we
00:31:42.630 --> 00:31:47.580
can apply the same strategy to
the computation of these N/2
00:31:47.580 --> 00:31:49.530
point DFTs.
00:31:49.530 --> 00:31:55.920
So we can break these N/2
point DFT computations into--
00:31:55.920 --> 00:31:59.160
each one of these--
into two N/4 point
00:31:59.160 --> 00:32:05.040
DFTs plus some
additional arithmetic
00:32:05.040 --> 00:32:09.250
in order to form
the N/2 point DFTs.
00:32:09.250 --> 00:32:13.210
So each one of
the N/2 point DFTs
00:32:13.210 --> 00:32:22.690
can then be decomposed into
an N/4 point DFT computation,
00:32:22.690 --> 00:32:26.440
into two N/4 point
DFT computations.
00:32:26.440 --> 00:32:32.380
In particular, we had,
recall, for the top half
00:32:32.380 --> 00:32:36.820
of the previous view graph,
the even-numbered points
00:32:36.820 --> 00:32:38.890
in the original sequence.
00:32:38.890 --> 00:32:41.050
All right, now we want
to compute the DFT
00:32:41.050 --> 00:32:43.100
of those even-numbered points.
00:32:43.100 --> 00:32:45.670
So we can go through
exactly the same steps.
00:32:45.670 --> 00:32:49.360
We can break that into
its even-numbered points
00:32:49.360 --> 00:32:52.550
and its odd numbered points.
00:32:52.550 --> 00:32:56.590
So we have the even numbered
of the even-numbered points
00:32:56.590 --> 00:33:01.330
into an N/4 point
DFT, the odd numbered
00:33:01.330 --> 00:33:07.060
of the even-numbered points
into a second N/4 point DFT,
00:33:07.060 --> 00:33:09.910
and then those
results are combined
00:33:09.910 --> 00:33:15.400
in exactly the same way as the
N/2 point DFTs were combined
00:33:15.400 --> 00:33:21.010
in the previous view
graph, but now with N/2
00:33:21.010 --> 00:33:24.160
as the subscript rather
than N as the subscript,
00:33:24.160 --> 00:33:27.807
because here what we're
computing is an N/2 point DFT.
00:33:30.310 --> 00:33:32.020
However, in drawing
the flow graph,
00:33:32.020 --> 00:33:36.820
we can re-express the
powers of W sub N over 2
00:33:36.820 --> 00:33:40.960
in terms of powers of W
sub N, because of the fact
00:33:40.960 --> 00:33:47.120
that W sub N squared is equal
to W sub N over 2 or vice versa.
00:33:47.120 --> 00:33:49.930
All right, so we
have this then apply
00:33:49.930 --> 00:33:53.710
to the even-numbered points
and to the odd-numbered points.
00:33:53.710 --> 00:33:56.290
And when we do
that, then the flow
00:33:56.290 --> 00:34:04.910
graph that results is
as I've indicated here.
00:34:04.910 --> 00:34:08.040
We have the even numbered
of the even-numbered points
00:34:08.040 --> 00:34:10.969
and the odd numbered of
the even-numbered points.
00:34:10.969 --> 00:34:13.760
We have the even numbered
of the odd-numbered points
00:34:13.760 --> 00:34:17.480
and the odd numbered of
the odd-numbered points.
00:34:17.480 --> 00:34:20.570
Those are each going
into N/4 point DFTs.
00:34:23.429 --> 00:34:26.704
Here is then the
sequence G of k.
00:34:29.350 --> 00:34:29.850
I'm sorry.
00:34:29.850 --> 00:34:33.030
Here is the sequence G of k.
00:34:33.030 --> 00:34:35.699
Here is the sequence H of k.
00:34:35.699 --> 00:34:40.409
This then implements the
computation of the N/2 point
00:34:40.409 --> 00:34:43.110
DFTs that we had before.
00:34:43.110 --> 00:34:48.900
And then we have the output DFT
obtained by combining those.
00:34:48.900 --> 00:34:53.260
So this flow graph results
from simply carrying
00:34:53.260 --> 00:34:58.030
the previous strategy back one
step to implementing the N/2
00:34:58.030 --> 00:35:00.300
point DFTs.
00:35:00.300 --> 00:35:06.040
All right, well, now we have the
computation of N/4 point DFTs.
00:35:06.040 --> 00:35:13.090
And we can do exactly the same
thing with the N/4 point DFTs,
00:35:13.090 --> 00:35:17.680
except that for this
particular example, N/4--
00:35:17.680 --> 00:35:20.950
if N is equal to 8,
N/4 is equal to 2.
00:35:20.950 --> 00:35:24.490
The computation of
the two-point DFTs
00:35:24.490 --> 00:35:28.130
is a relatively
straightforward thing.
00:35:28.130 --> 00:35:30.940
But in general, if
N wasn't equal to 8,
00:35:30.940 --> 00:35:33.580
if it was a more
general power of 2,
00:35:33.580 --> 00:35:35.530
we could likewise
apply this strategy
00:35:35.530 --> 00:35:40.140
to the computation of the N/4
point DFTs, the N/8 point DFTs,
00:35:40.140 --> 00:35:42.760
and over 16 point
DFTs, et cetera,
00:35:42.760 --> 00:35:48.950
until we got to the
computation of two-point DFTs.
00:35:48.950 --> 00:35:51.980
All right, so just simply
inserting the computation
00:35:51.980 --> 00:35:57.290
for these two-point DFTs,
which involves simply adding--
00:35:57.290 --> 00:36:01.970
well, adding x of 4 to x
of 0, multiplying x of 4
00:36:01.970 --> 00:36:05.690
by W sub N to the 4 and
adding it to x of 0.
00:36:05.690 --> 00:36:12.230
The resulting flow graph
for the entire computation
00:36:12.230 --> 00:36:20.990
is as I show here, where we
have now the input points.
00:36:20.990 --> 00:36:23.690
We have the output points.
00:36:23.690 --> 00:36:32.410
Here we're computing
the two-point DFTs,
00:36:32.410 --> 00:36:36.880
combining those together
to get the four-point DFTs,
00:36:36.880 --> 00:36:41.090
combining those together to
get the eight-point DFTs,
00:36:41.090 --> 00:36:42.190
et cetera.
00:36:42.190 --> 00:36:46.210
And I'll remind you again that
the basic computation that's
00:36:46.210 --> 00:36:51.970
involved throughout these stages
is the butterfly computation.
00:36:51.970 --> 00:36:59.560
We see it here, we see it
here, and we see it here.
00:36:59.560 --> 00:37:03.250
And always, the
multiplication by powers of W
00:37:03.250 --> 00:37:09.020
is applied to the branches
coming from the bottom node
00:37:09.020 --> 00:37:10.130
of the butterfly.
00:37:10.130 --> 00:37:14.720
And that's because of
the fact that X of k
00:37:14.720 --> 00:37:19.280
is expressed as G of k plus W
sub N to the k times H of k.
00:37:19.280 --> 00:37:21.950
And it's always the
H of Ks that are
00:37:21.950 --> 00:37:24.500
what are being computed
in the bottom half
00:37:24.500 --> 00:37:26.900
as we've been developing
this flow graph.
00:37:26.900 --> 00:37:29.849
It will turn out that that's
not true for other forms
00:37:29.849 --> 00:37:32.390
of the algorithm, but for the
form that we're developing here
00:37:32.390 --> 00:37:34.140
it is.
00:37:34.140 --> 00:37:37.700
All right, well,
having carried this
00:37:37.700 --> 00:37:42.110
all the way back to
the two-point DFTs,
00:37:42.110 --> 00:37:46.580
let's see what this has
accomplished for us in terms
00:37:46.580 --> 00:37:50.620
of computational efficiency.
00:37:50.620 --> 00:37:55.410
We began with N equal
to a power of 2.
00:37:55.410 --> 00:37:59.340
We thought about computing
a direct discrete Fourier
00:37:59.340 --> 00:38:04.320
transform, but that involved
N squared complex multiplies
00:38:04.320 --> 00:38:06.720
and adds.
00:38:06.720 --> 00:38:09.510
Instead of that, we
broke the computation
00:38:09.510 --> 00:38:14.970
into two N/2 point
DFTs plus some diddling
00:38:14.970 --> 00:38:17.460
to fit those together.
00:38:17.460 --> 00:38:21.720
The computation of
the two N/2 point DFTs
00:38:21.720 --> 00:38:27.960
involved complex multiplies
and adds equal to 2,
00:38:27.960 --> 00:38:31.680
because there were two of
them, times N/2 squared
00:38:31.680 --> 00:38:34.470
plus N for the diddles--
00:38:34.470 --> 00:38:37.920
the diddles being involved in
fitting those two N/2 point
00:38:37.920 --> 00:38:40.240
DFTs together.
00:38:40.240 --> 00:38:42.820
And this, for N
greater than two,
00:38:42.820 --> 00:38:46.010
is a bigger number than that.
00:38:46.010 --> 00:38:48.510
All right, then we said,
well, why stop there?
00:38:48.510 --> 00:38:56.670
Let's break these two N/2 point
DFTs into four N/4 point DFTs.
00:38:56.670 --> 00:39:00.420
That meant then that
this N/2 squared, which
00:39:00.420 --> 00:39:05.340
came from a direct
computation of the DFT,
00:39:05.340 --> 00:39:12.330
gets replaced instead
by two N/4 point DFTs.
00:39:12.330 --> 00:39:19.710
So we have 2 times N/4
squared plus N/2, which
00:39:19.710 --> 00:39:24.760
are the diddles involved in,
again, fitting those together.
00:39:24.760 --> 00:39:29.250
So the N/2 squared, which
involved computing one N/2
00:39:29.250 --> 00:39:32.520
point DFT gets
replaced by this when
00:39:32.520 --> 00:39:35.730
we go to the N/4 point DFTs.
00:39:35.730 --> 00:39:37.970
And consequently,
substituting this in there,
00:39:37.970 --> 00:39:40.170
the computation is--
00:39:40.170 --> 00:39:43.200
instead of that,
it's N/4 squared
00:39:43.200 --> 00:39:49.140
times 4 plus N plus N. This
N coming from that 2 times
00:39:49.140 --> 00:39:50.580
this N/2.
00:39:50.580 --> 00:39:53.670
Well, we can go
through this again.
00:39:53.670 --> 00:39:59.730
We broke the N/4 point
DFTs into N/8 point DFTs.
00:39:59.730 --> 00:40:03.420
Therefore, the N/4 squared
got replaced by N/8
00:40:03.420 --> 00:40:09.290
squared twice, plus N/4
to fit those together.
00:40:09.290 --> 00:40:12.350
That gets substituted
in for this N/4 squared.
00:40:12.350 --> 00:40:15.200
We get 8 times N/8 squared.
00:40:15.200 --> 00:40:20.820
We pick up another N, which
is this 4 times that N/4.
00:40:20.820 --> 00:40:23.900
And you can see that
each time we do this,
00:40:23.900 --> 00:40:28.850
this number gets smaller,
and we keep picking up
00:40:28.850 --> 00:40:32.450
another factor of N added in.
00:40:32.450 --> 00:40:36.140
Well, if you track this through,
what you find is that for N
00:40:36.140 --> 00:40:41.180
equal to a power of 2, in
fact, the computation then
00:40:41.180 --> 00:40:48.860
comes out proportional to
N plus N plus N nu times,
00:40:48.860 --> 00:40:52.160
where nu is log for
the base 2 of N.
00:40:52.160 --> 00:40:57.620
And so, the number of complex
multiplies and adds involved
00:40:57.620 --> 00:41:01.550
in implementing the
discrete Fourier transform
00:41:01.550 --> 00:41:07.430
using the fast Fourier transform
algorithm is proportional to N
00:41:07.430 --> 00:41:11.200
times nu rather than
proportional to N squared.
00:41:11.200 --> 00:41:14.840
A rather dramatic savings.
00:41:14.840 --> 00:41:18.000
OK, well there is one--
00:41:18.000 --> 00:41:19.700
there are a number
of modifications
00:41:19.700 --> 00:41:23.660
that we're going to want to
make to this basic structure
00:41:23.660 --> 00:41:27.230
for the computation as we
go through this lecture
00:41:27.230 --> 00:41:29.660
and the next several lectures.
00:41:29.660 --> 00:41:36.200
The first is to introduce
a fairly straightforward
00:41:36.200 --> 00:41:42.290
simplification by,
first of all, again,
00:41:42.290 --> 00:41:48.220
reminding you that the basic
computation involved always,
00:41:48.220 --> 00:41:51.760
from one stage to
the next, is always
00:41:51.760 --> 00:41:57.790
a butterfly type of computation
with multiplications
00:41:57.790 --> 00:42:03.280
by powers of W applied to the
signal from the bottom node.
00:42:03.280 --> 00:42:08.260
And you should notice, and
it's rather straightforward
00:42:08.260 --> 00:42:11.770
to verify why this
is true, that in each
00:42:11.770 --> 00:42:17.320
of these butterflies,
the separation in powers
00:42:17.320 --> 00:42:22.780
of W between the factor going
to the top node at the end
00:42:22.780 --> 00:42:24.550
of the butterfly
and the factor going
00:42:24.550 --> 00:42:28.090
to the bottom node
in the butterfly,
00:42:28.090 --> 00:42:31.780
they are always
separated by N/2.
00:42:31.780 --> 00:42:39.850
So that means that the basic
computation in the computation
00:42:39.850 --> 00:42:45.250
of the DFT, as I illustrated
on the previous flow graph,
00:42:45.250 --> 00:42:49.090
is a butterfly, where
the characteristic of it
00:42:49.090 --> 00:42:53.770
is that it's multiplication
by unity from the top node,
00:42:53.770 --> 00:42:58.420
and multiplication by some
power of W from the bottom
00:42:58.420 --> 00:43:04.360
to the top node, and a power
of W displaced from this one
00:43:04.360 --> 00:43:09.400
by N/2 from the bottom
node to the bottom node.
00:43:09.400 --> 00:43:13.480
Well, let's decompose
this power of W
00:43:13.480 --> 00:43:20.520
into W sub N to the r
times W sub N to the N/2.
00:43:20.520 --> 00:43:21.810
W sub N to the N/2.
00:43:21.810 --> 00:43:22.920
Well, what is that?
00:43:22.920 --> 00:43:25.590
That's e to the minus j
2 pi over capitalized N--
00:43:25.590 --> 00:43:27.420
that's W sub N--
00:43:27.420 --> 00:43:29.820
raised to the N/2 power.
00:43:29.820 --> 00:43:31.440
The capital N's cancel out.
00:43:31.440 --> 00:43:32.700
The 2's cancel out.
00:43:32.700 --> 00:43:36.820
We're left with e to the minus
j pi, which is equal to minus 1.
00:43:36.820 --> 00:43:41.730
So then, in fact, this factor,
or equivalently, this factor,
00:43:41.730 --> 00:43:48.450
is equal to minus 1 or
minus W sub N to the r.
00:43:48.450 --> 00:43:53.360
Consequently, we could
redraw the butterfly
00:43:53.360 --> 00:44:00.290
so that rather than implementing
that multiplication in both
00:44:00.290 --> 00:44:02.660
the branch going to the
top node and the branch
00:44:02.660 --> 00:44:05.510
going to the bottom
node, we simply
00:44:05.510 --> 00:44:09.620
multiply the value
coming into the butterfly
00:44:09.620 --> 00:44:13.880
by W sub N to the
r, and then add it
00:44:13.880 --> 00:44:18.260
to this value at the top
node, and subtract it
00:44:18.260 --> 00:44:21.320
from this value at
the bottom node.
00:44:21.320 --> 00:44:25.960
So we could redraw our
original flow graph
00:44:25.960 --> 00:44:30.080
by replacing the
butterfly computation
00:44:30.080 --> 00:44:34.400
in the previous flow graph by
this butterfly computation,
00:44:34.400 --> 00:44:37.400
where the multiplication
by the power of W
00:44:37.400 --> 00:44:40.580
is extracted from
inside the butterfly,
00:44:40.580 --> 00:44:45.230
so that basically we reduce the
number of multiplications by W
00:44:45.230 --> 00:44:49.430
that we have to implement
by a factor of 2.
00:44:49.430 --> 00:44:56.500
The flow graph that results when
we do that we have here then,
00:44:56.500 --> 00:45:02.680
where the powers of W are
now slid out of the butterfly
00:45:02.680 --> 00:45:06.500
to the entering node
to the butterfly.
00:45:06.500 --> 00:45:09.550
And we now have, rather
than multiplication
00:45:09.550 --> 00:45:14.740
by powers of W in the butterfly,
each butterfly computation
00:45:14.740 --> 00:45:19.670
consists of an addition
and a subtraction.
00:45:19.670 --> 00:45:22.990
So the basic computation
now is a multiplication
00:45:22.990 --> 00:45:29.020
by a power of W followed by
an addition and a subtraction.
00:45:29.020 --> 00:45:31.990
And this follows all the
way through a multiplication
00:45:31.990 --> 00:45:38.380
by a power of W and then an
addition and a subtraction,
00:45:38.380 --> 00:45:42.280
this minus 1 being
associated with this line.
00:45:42.280 --> 00:45:45.340
The minus 1's, of
course, always occur
00:45:45.340 --> 00:45:50.330
on the branches coming from the
bottom node of the butterfly.
00:45:50.330 --> 00:45:52.300
That's consistent
with everything
00:45:52.300 --> 00:45:56.700
that we've been saying
throughout this discussion.
00:45:56.700 --> 00:46:03.410
All right, so this then is
one form of an algorithm,
00:46:03.410 --> 00:46:06.620
referred to as the fast
Fourier transform algorithm,
00:46:06.620 --> 00:46:10.550
for implementing the computation
of the discrete Fourier
00:46:10.550 --> 00:46:12.110
transform.
00:46:12.110 --> 00:46:16.670
As the algorithm developed, it
developed somewhat naturally
00:46:16.670 --> 00:46:19.490
in a flow graph form.
00:46:19.490 --> 00:46:23.360
And in fact, we can
interpret this flow graph
00:46:23.360 --> 00:46:26.600
as a computational
recipe for computing
00:46:26.600 --> 00:46:29.900
the discrete Fourier transform.
00:46:29.900 --> 00:46:33.650
There are a few
things that I would
00:46:33.650 --> 00:46:35.450
like to draw your attention to.
00:46:35.450 --> 00:46:39.170
One primary one in particular
at this point, that
00:46:39.170 --> 00:46:45.500
is that we notice that the
input sequence values, because
00:46:45.500 --> 00:46:49.340
of the way that we derive this
algorithm, the input sequence
00:46:49.340 --> 00:46:54.830
values got rearranged in
a somewhat funny order
00:46:54.830 --> 00:46:57.410
at the input to the flow graph.
00:46:57.410 --> 00:46:59.120
Although, they come
out of the flow
00:46:59.120 --> 00:47:04.850
graph in a sequential order,
0, 1, 2, 3, et cetera.
00:47:04.850 --> 00:47:10.745
Over here, it's an order
of 0, 4, 2, 6 1, 5, 3, 7.
00:47:10.745 --> 00:47:15.410
It will turn out that that
order has a certain amount
00:47:15.410 --> 00:47:16.310
of significance.
00:47:16.310 --> 00:47:20.180
In fact, as we'll
see, as we continue
00:47:20.180 --> 00:47:25.550
this discussion in the next
lecture, that this is in fact
00:47:25.550 --> 00:47:27.260
a bit reversed order.
00:47:27.260 --> 00:47:31.100
It corresponds to rearranging
the order of the bits
00:47:31.100 --> 00:47:36.320
in the binary representation
of the indices.
00:47:36.320 --> 00:47:40.130
And furthermore, it's
possible to interpret
00:47:40.130 --> 00:47:43.580
this flow graph
in terms, not just
00:47:43.580 --> 00:47:49.860
of the computation of the
discrete Fourier transform,
00:47:49.860 --> 00:47:54.410
but it also suggests
how to arrange
00:47:54.410 --> 00:48:00.020
the memory, that is, the
storage of the input sequence
00:48:00.020 --> 00:48:03.690
and the resulting storage
of the output sequence.
00:48:03.690 --> 00:48:07.200
There are some efficiencies
that result when we do that.
00:48:07.200 --> 00:48:10.010
And that's one of
the first topics
00:48:10.010 --> 00:48:12.590
that we'll begin with next time.
00:48:12.590 --> 00:48:17.210
So in the next
lecture then, we will
00:48:17.210 --> 00:48:22.220
continue on the development
of the fast Fourier transform
00:48:22.220 --> 00:48:25.550
algorithm, picking
up from the point
00:48:25.550 --> 00:48:29.150
that we've reached
so far, which is--
00:48:29.150 --> 00:48:31.970
the point that we've
reached is to have
00:48:31.970 --> 00:48:37.010
exposed one form of the fast
Fourier transform algorithm.
00:48:37.010 --> 00:48:40.250
There are some important points
to be made about that form.
00:48:40.250 --> 00:48:44.930
And then a variety of
modifications of that form
00:48:44.930 --> 00:48:47.480
that have some advantages,
and, as it will turn out,
00:48:47.480 --> 00:48:49.510
also some disadvantages.
00:48:49.510 --> 00:48:52.350
Thank you.
00:48:52.350 --> 00:48:55.824
[MUSIC PLAYING]