WEBVTT
00:00:00.405 --> 00:00:04.200
PROFESSOR: Chapter six, problem
set three, problem set
00:00:04.200 --> 00:00:06.140
two solutions.
00:03:27.510 --> 00:03:28.100
OK.
00:03:28.100 --> 00:03:29.460
Good morning.
00:03:29.460 --> 00:03:32.690
Nice to see you all again
on this mild day.
00:03:32.690 --> 00:03:37.070
I want to start off by thanking
Ashish greatly for
00:03:37.070 --> 00:03:39.610
giving the last three
lectures.
00:03:39.610 --> 00:03:42.420
I have very high confidence in
Ashish, and I'm sure he give
00:03:42.420 --> 00:03:45.090
you excellent lectures and was
able to answer all your
00:03:45.090 --> 00:03:46.340
questions, right?
00:03:48.900 --> 00:03:52.000
But if you feel dissatisfied at
any point, you'll have an
00:03:52.000 --> 00:03:54.295
opportunity to ask the
question again today.
00:03:57.020 --> 00:03:59.930
We will continue now
in chapter six.
00:03:59.930 --> 00:04:04.010
I understand Ashish got up to
section six point one already,
00:04:04.010 --> 00:04:07.440
so that makes my life
a little easier.
00:04:07.440 --> 00:04:09.870
And we'll just continue to plow
ahead through the notes.
00:04:09.870 --> 00:04:16.600
You have as handouts today
chapter six, the outgoing
00:04:16.600 --> 00:04:21.769
problem set, the solutions to
the incoming problem set.
00:04:21.769 --> 00:04:26.690
So does anyone have any comments
or questions or
00:04:26.690 --> 00:04:30.120
suggestions as we go forward
at this point?
00:04:30.120 --> 00:04:34.360
Let's say I'll do a very quick
review of everything so far,
00:04:34.360 --> 00:04:37.440
at least what we need to
proceed, so you'll have a
00:04:37.440 --> 00:04:40.870
chance to ask questions along
there if you like.
00:04:40.870 --> 00:04:43.040
Anyone have anything to say?
00:04:43.040 --> 00:04:44.570
Anyone want to say
thanks to Ashish?
00:04:48.830 --> 00:04:49.280
All right.
00:04:49.280 --> 00:04:51.970
Great.
00:04:51.970 --> 00:04:52.550
OK.
00:04:52.550 --> 00:04:57.190
So a very quick review.
00:04:57.190 --> 00:05:02.300
We first said we were going to
work with a continuous time
00:05:02.300 --> 00:05:05.260
Additive White Gaussian noise
channel, but we quickly
00:05:05.260 --> 00:05:08.780
reduced it to a discrete
time additive white
00:05:08.780 --> 00:05:10.110
Gaussian noise model.
00:05:10.110 --> 00:05:15.030
Vector model, Y equals X plus N,
where the vectors may go on
00:05:15.030 --> 00:05:16.690
as long as you like.
00:05:16.690 --> 00:05:19.390
And implicit in here is we
have some kind of power
00:05:19.390 --> 00:05:21.980
limitation expressed
by various --
00:05:21.980 --> 00:05:24.930
I know I've probably introduced
too many parameters
00:05:24.930 --> 00:05:27.710
in the early part
of the course.
00:05:27.710 --> 00:05:31.820
But you'll see them all in the
literature, and I thought you
00:05:31.820 --> 00:05:36.180
might as well see the relation
between them all at once.
00:05:36.180 --> 00:05:36.850
The noise.
00:05:36.850 --> 00:05:40.350
The main thing about this
is it's an independent,
00:05:40.350 --> 00:05:44.640
identically distributed noise,
also independent of X, and
00:05:44.640 --> 00:05:49.020
it's characterized simply by
its variance per symbol, or
00:05:49.020 --> 00:05:49.960
average power.
00:05:49.960 --> 00:05:54.360
Again, there are various
notations that we
00:05:54.360 --> 00:05:57.310
can use for its power.
00:05:57.310 --> 00:05:58.310
OK.
00:05:58.310 --> 00:06:03.310
However you express the power
of the signal of the noise,
00:06:03.310 --> 00:06:05.290
you're going to get some
signal-to-noise ratio out of
00:06:05.290 --> 00:06:09.160
that, as long as you use
consistent the same units for
00:06:09.160 --> 00:06:10.960
signal and noise.
00:06:10.960 --> 00:06:13.860
And so the key parameters
of this channel turn
00:06:13.860 --> 00:06:16.160
out to be just two.
00:06:16.160 --> 00:06:19.200
One is the signal-to-noise
ratio.
00:06:19.200 --> 00:06:23.000
And the other is something which
in the discrete time
00:06:23.000 --> 00:06:26.260
domain we might think of as a
data rate, rho, the number of
00:06:26.260 --> 00:06:28.880
bits per two dimensions.
00:06:28.880 --> 00:06:32.360
But we were at pains
to show --
00:06:32.360 --> 00:06:35.380
and the reason we measure it in
bits per two dimensions is
00:06:35.380 --> 00:06:40.180
that it converts directly to the
spectral efficiency, or we
00:06:40.180 --> 00:06:42.990
sometimes say, the nominal
spectral efficiency, or the
00:06:42.990 --> 00:06:45.340
Nyquist spectral efficiency.
00:06:45.340 --> 00:06:48.330
Basically, this is the spectral
efficiency if you
00:06:48.330 --> 00:06:52.620
assume that you only use the
nominal Nyquist bandwidth.
00:06:52.620 --> 00:06:57.630
And both in 450 and here it was
shown that you can really
00:06:57.630 --> 00:07:03.710
get as close to the nominal
bandwidth as you like, its
00:07:03.710 --> 00:07:08.530
sharp or roll off as you
like, by using filters.
00:07:08.530 --> 00:07:15.830
Very sharp filters or very sharp
digital filters that
00:07:15.830 --> 00:07:20.660
today we can really
realistically get effectively
00:07:20.660 --> 00:07:23.700
to the Nyquist bandwidth.
00:07:23.700 --> 00:07:27.150
So this is a good measure of
nominal spectral efficiency,
00:07:27.150 --> 00:07:29.780
which is an extremely important
parameter, the
00:07:29.780 --> 00:07:33.920
spectral efficiency in
continuous time.
00:07:33.920 --> 00:07:36.820
And then the Shannon limit is
expressed in this very simple
00:07:36.820 --> 00:07:44.020
form that we can never do better
in data rate or nominal
00:07:44.020 --> 00:07:48.920
spectral efficiency than
log of 1 plus SNR.
00:07:48.920 --> 00:07:51.860
And the units are bits
per two dimensions.
00:07:51.860 --> 00:07:55.410
So there's an awful lot about
parameters and units in the
00:07:55.410 --> 00:07:57.850
first part of the course, and
sometimes it's a little
00:07:57.850 --> 00:07:59.100
overwhelming and confusing.
00:08:01.730 --> 00:08:05.670
But in my experience in
engineering is, it's extremely
00:08:05.670 --> 00:08:08.630
important to get the
units right.
00:08:08.630 --> 00:08:11.470
It helps you in thinking
clearly.
00:08:11.470 --> 00:08:14.040
It helps you to focus
on the right things.
00:08:14.040 --> 00:08:16.530
And so in the first couple of
chapters, there's a lot of
00:08:16.530 --> 00:08:18.950
emphasis on getting
the units right.
00:08:18.950 --> 00:08:21.490
For instance, bits per
two dimensions.
00:08:21.490 --> 00:08:25.590
I could simply assert to you
that things are always to come
00:08:25.590 --> 00:08:28.060
out better if you do things
per two dimensions.
00:08:28.060 --> 00:08:34.039
This basically has to do with
Gaussian distributions being
00:08:34.039 --> 00:08:37.374
simpler in two dimensions than
they are in one dimension.
00:08:37.374 --> 00:08:41.630
So you know you can get closed
form integrals in two and four
00:08:41.630 --> 00:08:44.420
and so forth dimensions that you
can't get in one and three
00:08:44.420 --> 00:08:45.610
and so forth dimensions.
00:08:45.610 --> 00:08:50.350
So you know, the geometrical
formulas for the volume of the
00:08:50.350 --> 00:08:53.060
sphere and so forth are much
simpler in even dimensions
00:08:53.060 --> 00:08:55.550
than they are in
odd dimensions.
00:08:55.550 --> 00:09:00.610
This all, I think, has to do
with the fact that really a
00:09:00.610 --> 00:09:03.750
one-dimensional complex Gaussian
variable, or in
00:09:03.750 --> 00:09:06.570
general, complex Gaussian
variables, are in some sense
00:09:06.570 --> 00:09:10.780
mathematically simpler than
real Gaussian variables.
00:09:10.780 --> 00:09:14.270
For instance, the Q function
is a very ugly thing.
00:09:14.270 --> 00:09:16.670
In two dimensions, you can
closed-form integrals with
00:09:16.670 --> 00:09:18.520
probability of error.
00:09:18.520 --> 00:09:22.210
So there's a lot of clues that
we really want to think in
00:09:22.210 --> 00:09:26.293
terms of pairs of
real dimensions.
00:09:29.580 --> 00:09:29.640
All right.
00:09:29.640 --> 00:09:32.280
And then we introduced a
couple more parameters.
00:09:32.280 --> 00:09:35.580
And I understand Ashish
got the questions
00:09:35.580 --> 00:09:36.460
that I get every year.
00:09:36.460 --> 00:09:39.040
Why do we introduce both
of these things?
00:09:39.040 --> 00:09:43.130
SNR norm, which is SNR
normalized by 2 to the rho
00:09:43.130 --> 00:09:45.690
minus 1, that's motivated
directly by the
00:09:45.690 --> 00:09:47.540
Shannon limit formula.
00:09:47.540 --> 00:09:52.720
Or Eb over N_0, which is
SNR divided by rho.
00:09:52.720 --> 00:09:54.170
You know, these are
both more or less
00:09:54.170 --> 00:09:55.940
equivalent to each other.
00:09:55.940 --> 00:10:00.240
From this Eb over N_0 is just 2
to the rho minus 1 over rho
00:10:00.240 --> 00:10:05.010
times SNR norm, so why do we
introduce both of them?
00:10:05.010 --> 00:10:11.900
And there's not an
intellectually very strong
00:10:11.900 --> 00:10:15.760
argument to introduce
both of them.
00:10:15.760 --> 00:10:20.530
Because one is easily translated
into the other.
00:10:20.530 --> 00:10:26.060
For fixed nominal spectral
efficiency rho, there clearly
00:10:26.060 --> 00:10:28.060
is just a one-to-one
translation.
00:10:28.060 --> 00:10:29.940
We could use either one.
00:10:29.940 --> 00:10:33.900
If you graph something versus
Eb over N_0 or versus SNR
00:10:33.900 --> 00:10:38.260
norm, it's just a matter of
shifting it by this factor.
00:10:38.260 --> 00:10:42.230
They're going to be exactly the
same graph with the 0 dB
00:10:42.230 --> 00:10:47.650
point in a different place,
according to where 0 dB is.
00:10:47.650 --> 00:10:52.460
Philosophically, of course, SNR
norm, which is sometimes
00:10:52.460 --> 00:10:58.960
called the gap to capacity, is
exactly a measure of how many
00:10:58.960 --> 00:11:02.070
dBs away are we from
the Shannon limit.
00:11:02.070 --> 00:11:06.570
Measuring things on
a log scale in dB.
00:11:06.570 --> 00:11:10.400
And so 0 dB is always the
Shannon limit point with SNR
00:11:10.400 --> 00:11:15.700
norm, because this statement
translates into SNR norm is
00:11:15.700 --> 00:11:18.900
greater than 1, which is 0 dB.
00:11:18.900 --> 00:11:23.040
So here the focus is always, how
far are you from capacity?
00:11:23.040 --> 00:11:25.270
How far are you from
the Shannon limit?
00:11:25.270 --> 00:11:29.200
And that really is
the modern view.
00:11:29.200 --> 00:11:32.410
In the early days of coding, the
view was, well, how much
00:11:32.410 --> 00:11:35.160
coding gain can we get
over no coding?
00:11:35.160 --> 00:11:42.960
And as we'll see, Eb over N_0
is often a very convenient
00:11:42.960 --> 00:11:46.370
parameter to use when we're
focusing on coding gain.
00:11:46.370 --> 00:11:49.040
In fact, for binary linear block
codes, which is what
00:11:49.040 --> 00:11:51.960
we're talking about in chapter
six, we get an extremely
00:11:51.960 --> 00:11:57.700
simple expression that Eb over
N_0 it's just kd over N, where
00:11:57.700 --> 00:12:02.590
N, k, d are the basic parameters
of a block code.
00:12:02.590 --> 00:12:04.150
If you don't know those
yet, you will by
00:12:04.150 --> 00:12:06.910
the end of this lecture.
00:12:06.910 --> 00:12:14.210
And Eb over N_0 is what was put
forward in the early days,
00:12:14.210 --> 00:12:17.370
when the principal coding
application was coding for
00:12:17.370 --> 00:12:19.180
deep space communications.
00:12:19.180 --> 00:12:22.320
It makes sense in the
power-limited regime.
00:12:22.320 --> 00:12:27.180
In the power-limited regime,
we showed that this
00:12:27.180 --> 00:12:31.540
essentially goes -- this is rho
log2 over rho, so up to a
00:12:31.540 --> 00:12:36.190
factor of natural logarithm of
two, these are the same,
00:12:36.190 --> 00:12:39.610
almost independent of the
spectral efficiency, as well.
00:12:39.610 --> 00:12:44.440
And so this is a very natural
thing to use, especially in
00:12:44.440 --> 00:12:46.240
the power-limited regime.
00:12:46.240 --> 00:12:50.180
In the bandwidth-limited regime,
as rho gets large,
00:12:50.180 --> 00:12:52.910
then these things become
very different.
00:12:52.910 --> 00:12:56.050
SNR norm always keeps us in
the neighborhood of 0 dB.
00:12:56.050 --> 00:13:01.210
This thing goes up to 10 dB,
20 dB, 30 dB, and so forth.
00:13:01.210 --> 00:13:03.940
Nonetheless, you'll see in
some literature, people
00:13:03.940 --> 00:13:09.910
continue to measure against Eb
over N_0 maybe I would say,
00:13:09.910 --> 00:13:11.290
just because they don't
know any better.
00:13:11.290 --> 00:13:16.000
Anyway, since I started writing
these notes about
00:13:16.000 --> 00:13:19.090
eight or nine years ago, I've
been advocating SNR norm.
00:13:19.090 --> 00:13:23.830
It's more and more widely
used in our business,
00:13:23.830 --> 00:13:24.930
in the coding business.
00:13:24.930 --> 00:13:30.680
Or equivalently, one always
shows nowadays how far are you
00:13:30.680 --> 00:13:35.370
from capacity, and that's
what SNR norm is about.
00:13:35.370 --> 00:13:38.690
And you can always translate
this into this and this into
00:13:38.690 --> 00:13:40.830
this by this simple formula.
00:13:40.830 --> 00:13:42.190
So that's why we introduce
them both.
00:13:42.190 --> 00:13:44.630
Eb over N_0 is traditional.
00:13:44.630 --> 00:13:49.821
SNR norm is more of the modern
gap to capacity viewpoint.
00:13:49.821 --> 00:13:51.430
Any questions about that?
00:13:51.430 --> 00:13:54.830
Because I understood that
Ashish got a number of
00:13:54.830 --> 00:13:56.410
questions about that.
00:13:56.410 --> 00:13:56.875
Yes?
00:13:56.875 --> 00:13:59.725
AUDIENCE: And the probability
of error you mentioned
00:13:59.725 --> 00:14:03.550
[UNINTELLIGIBLE] is natural
with SNR model.
00:14:03.550 --> 00:14:04.610
PROFESSOR: Well, this
is a slightly
00:14:04.610 --> 00:14:05.860
different point, actually.
00:14:08.010 --> 00:14:11.060
So we go on from this to talk
about the power-limited
00:14:11.060 --> 00:14:14.580
regime, which we defined more
or less arbitrarily as the
00:14:14.580 --> 00:14:17.860
regime where rho is less than
or equal to two bits per two
00:14:17.860 --> 00:14:20.620
dimensions, and the
bandwidth-limited regime,
00:14:20.620 --> 00:14:23.800
which is where rho is larger.
00:14:23.800 --> 00:14:30.310
And at this point, I simply
assert that it's better to do
00:14:30.310 --> 00:14:32.940
everything per two dimensions
in the bandwidth-limited
00:14:32.940 --> 00:14:39.120
regime and to do everything
per bit in the
00:14:39.120 --> 00:14:41.270
power-limited regime.
00:14:41.270 --> 00:14:47.410
And the reason for this is
basically long practice and
00:14:47.410 --> 00:14:51.090
intuition and experience that
things do work out better, and
00:14:51.090 --> 00:14:53.980
this is the proper
normalization.
00:14:53.980 --> 00:14:55.920
But I think at this point
in the course, with your
00:14:55.920 --> 00:15:00.350
background, this is only an
assertion, all right?
00:15:00.350 --> 00:15:05.590
So I simply say, this is the way
we're going to do things
00:15:05.590 --> 00:15:08.140
in bandwidth-limited
and power-limited.
00:15:08.140 --> 00:15:13.220
For most of the rest of the
course, we're going to be in
00:15:13.220 --> 00:15:14.400
the power-limited regime.
00:15:14.400 --> 00:15:18.820
Then we'll come back to the
bandwidth-limited very late in
00:15:18.820 --> 00:15:19.810
the course.
00:15:19.810 --> 00:15:26.400
So you can now forget this
assertion for a while.
00:15:26.400 --> 00:15:26.850
Yes?
00:15:26.850 --> 00:15:28.100
AUDIENCE: [INAUDIBLE]
00:15:31.270 --> 00:15:33.430
PROFESSOR: Well, I'm trying
to focus on that here.
00:15:33.430 --> 00:15:38.390
That if two systems have
different rhos, then you
00:15:38.390 --> 00:15:42.330
should take that into account
in their comparison.
00:15:42.330 --> 00:15:59.200
And when you see charts of a
rate 7/8 system, or a rho 7/4
00:15:59.200 --> 00:16:03.650
system versus one that's rate
1/8, those are very different
00:16:03.650 --> 00:16:10.490
regimes, and it probably isn't
fair to compare two systems,
00:16:10.490 --> 00:16:13.380
just in terms of their
probability of error versus Eb
00:16:13.380 --> 00:16:16.480
over N_0 at two different
spectral efficiencies.
00:16:16.480 --> 00:16:21.310
What is more fair is to compare
them in terms of their
00:16:21.310 --> 00:16:22.560
gap to capacity.
00:16:27.210 --> 00:16:30.640
You can get more powerful
codes with more
00:16:30.640 --> 00:16:34.570
error-correcting capability the
lower in rate that you go.
00:16:34.570 --> 00:16:39.960
And so you should start from
the point of view that two
00:16:39.960 --> 00:16:45.330
systems with different rho are
incomparable, and then using
00:16:45.330 --> 00:16:48.730
this you can say, but, you know,
if we can get within 1
00:16:48.730 --> 00:16:51.440
dB of the Shannon limit with
both of them, then they're
00:16:51.440 --> 00:16:53.890
both approximately
equally powerful.
00:16:53.890 --> 00:16:56.640
That would be the modern
point of view.
00:16:56.640 --> 00:17:00.400
But it's really apples and
oranges if they have different
00:17:00.400 --> 00:17:03.560
rates, different rhos.
00:17:03.560 --> 00:17:03.990
Yes?
00:17:03.990 --> 00:17:05.240
AUDIENCE: [INAUDIBLE]
00:17:07.550 --> 00:17:10.115
If [UNINTELLIGIBLE]
00:17:10.115 --> 00:17:13.349
systems modulation
[INAUDIBLE].
00:17:13.349 --> 00:17:16.910
Then is it fair --
00:17:16.910 --> 00:17:21.079
because if one of them
has coding --
00:17:21.079 --> 00:17:24.710
PROFESSOR: If none of them
have coding, well --
00:17:24.710 --> 00:17:26.054
AUDIENCE: No, just modulation.
00:17:28.960 --> 00:17:31.870
PROFESSOR: So what shall
I say to that.
00:17:31.870 --> 00:17:35.500
In the power-limited regime,
an uncoded system is simply
00:17:35.500 --> 00:17:41.900
binary modulation or 4-QAM,
and there aren't different
00:17:41.900 --> 00:17:43.170
systems to compare, really.
00:17:43.170 --> 00:17:46.850
There's only one baseline
system.
00:17:46.850 --> 00:17:51.105
As you go up into the
bandwidth-limited regime, then
00:17:51.105 --> 00:17:55.290
it is fair to compare
systems of the same
00:17:55.290 --> 00:17:58.650
class, like M by M QAM.
00:17:58.650 --> 00:18:01.180
That's a simple uncoded
system.
00:18:01.180 --> 00:18:04.060
Now there, rho keeps changing.
00:18:04.060 --> 00:18:08.260
rho equals 2, rho equals 4, rho
equals 6, rho equals 8.
00:18:08.260 --> 00:18:11.210
Or you can easily find
intermediate
00:18:11.210 --> 00:18:13.610
rates that are uncoded.
00:18:13.610 --> 00:18:17.740
And there you find that this
normalization makes all these
00:18:17.740 --> 00:18:19.540
systems comparable.
00:18:19.540 --> 00:18:24.160
In fact, we saw that we got a
universal baseline curve for
00:18:24.160 --> 00:18:28.100
the bandwidth-limited regime,
which was independent of rho,
00:18:28.100 --> 00:18:33.520
for, say, the class of 4 by
4, by the class of M by M
00:18:33.520 --> 00:18:37.900
bandwidth-limited QAM systems.
00:18:37.900 --> 00:18:38.300
I'm sorry.
00:18:38.300 --> 00:18:41.600
I got off a plane at nine
o'clock last night, so I may
00:18:41.600 --> 00:18:45.440
not be totally coherent today.
00:18:45.440 --> 00:18:49.750
But all of these systems have
exactly the same baseline
00:18:49.750 --> 00:18:55.210
curve up to the approximations
we've made that their exactly
00:18:55.210 --> 00:18:57.130
four nearest neighbors
and so forth.
00:18:57.130 --> 00:19:02.040
But on a curve of probability
of error per two dimensions,
00:19:02.040 --> 00:19:06.930
per QAM symbol, versus
SNR norm, we
00:19:06.930 --> 00:19:08.290
get a universal curve.
00:19:08.290 --> 00:19:11.560
So that indicates they really
are comparable.
00:19:11.560 --> 00:19:15.460
They form a class where the
performance of the class is
00:19:15.460 --> 00:19:16.820
independent of rho.
00:19:16.820 --> 00:19:20.730
And that's sort of typical of
the bandwidth-limited regime.
00:19:20.730 --> 00:19:24.010
Two systems that differ, that
basically use the same code
00:19:24.010 --> 00:19:28.220
with smaller constellation
or a larger-based uncoded
00:19:28.220 --> 00:19:30.870
constellation are going to be
directly comparable, and are
00:19:30.870 --> 00:19:35.370
probably going to have the
same gap to capacity.
00:19:35.370 --> 00:19:37.370
Going to be just as far away
from the Shannon limit.
00:19:37.370 --> 00:19:39.410
But we don't have that
phenomenon in the
00:19:39.410 --> 00:19:40.660
power-limited case.
00:19:45.080 --> 00:19:46.780
Other questions, or
shall I proceed?
00:19:49.810 --> 00:19:51.060
OK.
00:19:52.880 --> 00:19:58.490
Again continuing what I hope
will be a quick review, but I
00:19:58.490 --> 00:20:00.720
don't want to go any faster than
you're comfortable with.
00:20:05.960 --> 00:20:12.140
We get this baseline curve
for 2-PAM or 2
00:20:12.140 --> 00:20:14.330
by 2 QAM, same thing.
00:20:18.400 --> 00:20:26.520
So this is the baseline of Pb of
E on a log scale, where we
00:20:26.520 --> 00:20:29.460
typically go down to ten to
the minus six, ten to the
00:20:29.460 --> 00:20:31.050
minus five.
00:20:31.050 --> 00:20:33.320
Here's 0 dB.
00:20:33.320 --> 00:20:37.670
This goes down through
about 9.6 dB here or
00:20:37.670 --> 00:20:40.710
about 10.5 dB here.
00:20:40.710 --> 00:20:47.010
And the ultimate Shannon
limit is over here.
00:20:47.010 --> 00:20:52.150
The ultimate Shannon limit for
very low rho, as rho goes to
00:20:52.150 --> 00:20:55.840
0, is about minus 1.6 dB.
00:20:55.840 --> 00:20:57.630
And we get expressions.
00:20:57.630 --> 00:21:00.780
0 dB is the Shannon limit
at rho equals 1.
00:21:00.780 --> 00:21:05.570
At rho equals 2, it's
up about 1.8 dB.
00:21:05.570 --> 00:21:08.250
Shannon limit for rho equals
2, and so forth.
00:21:08.250 --> 00:21:12.390
And anyway, so we see as a
function of rho, we can
00:21:12.390 --> 00:21:16.210
measure the gap to capacity at
different probabilities of
00:21:16.210 --> 00:21:19.780
error and see how much
coding gain is,
00:21:19.780 --> 00:21:20.780
in principle, possible.
00:21:20.780 --> 00:21:26.610
And then for a coded system,
we can put that on here.
00:21:26.610 --> 00:21:29.980
The effective coding gain
is the distance
00:21:29.980 --> 00:21:32.760
between here and here.
00:21:32.760 --> 00:21:36.130
At some target probability
of error.
00:21:36.130 --> 00:21:39.550
It's going to differ according
to probably of error.
00:21:39.550 --> 00:21:46.740
You found a way of getting a
good, rough-cut estimate, at
00:21:46.740 --> 00:21:50.630
least for not very complicated
codes called the union bound
00:21:50.630 --> 00:21:57.040
estimate for any signal
constellation in any number of
00:21:57.040 --> 00:21:58.290
dimensions.
00:21:58.290 --> 00:22:04.980
So if we have a signal
constellation A, which
00:22:04.980 --> 00:22:10.180
consists of M points in N
dimensions, so forth,
00:22:10.180 --> 00:22:16.630
basically found that we could,
in the power-limited regime,
00:22:16.630 --> 00:22:22.190
get an approximate expression
from considering pairwise
00:22:22.190 --> 00:22:25.400
error probabilities that's
very simple.
00:22:25.400 --> 00:22:27.140
And I used this notation.
00:22:27.140 --> 00:22:34.300
Q of the square root of some
coding gain of the
00:22:34.300 --> 00:22:40.330
constellation times
2 Eb over N_0.
00:22:40.330 --> 00:22:45.150
If it's just 2-PAM then the
coding gain becomes 1.
00:22:45.150 --> 00:22:52.430
This is the average number
of nearest neighbors per
00:22:52.430 --> 00:22:55.810
transmitted bit.
00:22:55.810 --> 00:23:00.470
And so the whole thing reduces
to this expression in terms of
00:23:00.470 --> 00:23:01.610
couple of parameters.
00:23:01.610 --> 00:23:06.610
the principal parameter is the
nominal coding gain, which is
00:23:06.610 --> 00:23:13.840
the minimum squared distance of
A over 4 times the energy
00:23:13.840 --> 00:23:19.170
per bit of this constellation
A. So we really only need to
00:23:19.170 --> 00:23:24.950
know this kind of normalized
measure of goodness of the
00:23:24.950 --> 00:23:27.410
constellation.
00:23:27.410 --> 00:23:35.810
And K_b of A Is the average
number of nearest neighbors.
00:23:35.810 --> 00:23:38.070
I forget what we call it.
00:23:38.070 --> 00:23:43.250
What is the numerator here?
00:23:43.250 --> 00:23:50.430
K_min of A, which itself is an
average, over the number of
00:23:50.430 --> 00:23:56.430
bits that we're actually
sending, which is log2 of the
00:23:56.430 --> 00:23:59.690
size of A.
00:23:59.690 --> 00:24:03.110
So basically we only need to
know a couple of parameters of
00:24:03.110 --> 00:24:04.610
this signal constellation.
00:24:04.610 --> 00:24:08.480
Its minimum square distance
is very important.
00:24:08.480 --> 00:24:11.890
Energy parameter, which we
choose to make the energy per
00:24:11.890 --> 00:24:15.020
bit, or so that we get
this expression.
00:24:15.020 --> 00:24:19.600
And the average number of
nearest neighbors per bit that
00:24:19.600 --> 00:24:19.877
we transmit.
00:24:19.877 --> 00:24:21.127
OK.
00:24:23.920 --> 00:24:33.840
And our best example so far is
the tetrahedral constellation,
00:24:33.840 --> 00:24:37.760
where we basically
pick every other
00:24:37.760 --> 00:24:44.245
point from the 4 simplex.
00:24:48.160 --> 00:24:50.690
Maybe I should do that
in a different color.
00:24:50.690 --> 00:24:52.636
I don't think that's
a different color.
00:24:52.636 --> 00:24:54.750
Nope.
00:24:54.750 --> 00:24:55.260
Anyway.
00:24:55.260 --> 00:24:57.710
You know this quite
well by now.
00:24:57.710 --> 00:25:01.330
And if we do that, we find that
normalizing everything,
00:25:01.330 --> 00:25:05.730
this is 4/3 or 1.25 dB.
00:25:05.730 --> 00:25:10.400
And every point has three
nearest neighbors, and we're
00:25:10.400 --> 00:25:12.330
sending 2 bits, so
this is 3/2.
00:25:15.710 --> 00:25:20.020
So then we get this handy dandy
engineering rule to make
00:25:20.020 --> 00:25:25.000
a quick plot of the union
bound estimate.
00:25:25.000 --> 00:25:27.860
Given that we decided to put
this on a log-log scale to
00:25:27.860 --> 00:25:33.340
start with, all we have to do
to plot this expression --
00:25:33.340 --> 00:25:42.570
if we're given, as we always
are, the baseline curve, which
00:25:42.570 --> 00:25:49.010
is simply Q to the square
root of 2 Eb over N_0.
00:25:49.010 --> 00:25:52.150
How do you convert this
curve to this curve
00:25:52.150 --> 00:25:53.380
on a log-log plot?
00:25:53.380 --> 00:25:57.070
Well, you simply move it to the
left by the coding gain.
00:25:57.070 --> 00:26:03.920
And you move it up by the
log of whatever K_b is.
00:26:03.920 --> 00:26:08.460
So if the dominant effect is
moving it left, in this case,
00:26:08.460 --> 00:26:12.450
by 1.25 dB --
00:26:12.450 --> 00:26:15.110
I'm going to wind up getting
about this curve, so I won't
00:26:15.110 --> 00:26:15.960
draw it again --
00:26:15.960 --> 00:26:22.330
then we move it up by a factor
of 3/2, we developed a rule of
00:26:22.330 --> 00:26:26.480
thumb that said a factor of two
is basically going to cost
00:26:26.480 --> 00:26:30.230
you about 0.2 dB.
00:26:30.230 --> 00:26:31.770
Around ten to the minus five.
00:26:31.770 --> 00:26:35.390
This is just based on the slope
of this baseline curve,
00:26:35.390 --> 00:26:38.040
as long as we're somewhat
in that region.
00:26:38.040 --> 00:26:43.540
So this will cost us about 1/10
of a dB, so we'll get an
00:26:43.540 --> 00:26:49.250
effective coding gain
of about 1.15 dB.
00:26:49.250 --> 00:26:52.250
So graphically, we take
this curve bodily.
00:26:52.250 --> 00:26:58.410
We move it over 1.25 dB,
and we move it up
00:26:58.410 --> 00:27:00.420
by a factor of 3/2.
00:27:00.420 --> 00:27:02.890
And what we'll find is the
effective coding gain is
00:27:02.890 --> 00:27:07.730
thereby reduced to, I estimate,
about 1.15 dB.
00:27:07.730 --> 00:27:11.110
And this is all just engineering
rules of thumb.
00:27:11.110 --> 00:27:12.830
Nice template.
00:27:12.830 --> 00:27:15.570
I sometimes say you should
fill out a copy of this
00:27:15.570 --> 00:27:18.100
baseline curve, cut it out, put
it in your wallet for the
00:27:18.100 --> 00:27:20.650
duration of this course,
because you'll have the
00:27:20.650 --> 00:27:22.220
opportunity to make
this kind of
00:27:22.220 --> 00:27:25.150
calculation again and again.
00:27:25.150 --> 00:27:25.540
All right.
00:27:25.540 --> 00:27:28.780
So that's union bound SNR.
00:27:28.780 --> 00:27:31.150
For simple constellations,
the union bound
00:27:31.150 --> 00:27:34.480
estimate is very accurate.
00:27:34.480 --> 00:27:37.280
So this is a very good way to
proceed, from an engineering
00:27:37.280 --> 00:27:38.560
point of view.
00:27:38.560 --> 00:27:41.212
We want to write a paper?
00:27:41.212 --> 00:27:43.330
You know, it's a little late
to write a paper on the
00:27:43.330 --> 00:27:46.270
performance of the four simplex
signal constellations,
00:27:46.270 --> 00:27:49.430
but if you wanted to, and you
wanted to have one graph in
00:27:49.430 --> 00:27:51.790
that paper, you would
have the graph of --
00:27:51.790 --> 00:27:57.290
well, you would actually
probably compute the exact
00:27:57.290 --> 00:28:01.020
error of probability, either by
analysis or by Monte Carlo
00:28:01.020 --> 00:28:03.150
simulation, you would
put that in there.
00:28:03.150 --> 00:28:05.590
But in your first draft of the
paper, you would put in union
00:28:05.590 --> 00:28:09.100
bound estimate and you would
find that wasn't far off.
00:28:11.630 --> 00:28:12.680
OK.
00:28:12.680 --> 00:28:13.830
Any questions on that?
00:28:13.830 --> 00:28:19.030
This has basically got
us up to chapter six.
00:28:26.090 --> 00:28:35.240
So in chapter six now, which
Ashish got into last time.
00:28:35.240 --> 00:28:40.840
This is basically about binary
linear block codes.
00:28:44.510 --> 00:28:48.820
And well, we first just
start talking about
00:28:48.820 --> 00:28:51.330
binary block codes.
00:28:51.330 --> 00:28:55.750
That's what Ashish
did last time.
00:28:58.350 --> 00:29:03.310
We basically take 0 and 1
as our binary alphabet.
00:29:03.310 --> 00:29:08.000
We take a blocks of length n.
00:29:08.000 --> 00:29:08.170
Sorry.
00:29:08.170 --> 00:29:12.020
Are we using little
n or big N?
00:29:12.020 --> 00:29:15.170
Little n, where n is called
the block plank.
00:29:15.170 --> 00:29:23.760
So we take the set of all binary
symbols of length n,
00:29:23.760 --> 00:29:31.730
and we're going to convert
this to real n space.
00:29:31.730 --> 00:29:33.950
How do we convert it?
00:29:33.950 --> 00:29:36.770
Component-wise, by the
standard 2-PAM map.
00:29:36.770 --> 00:29:41.660
We map 0 into plus alpha,
1 into minus alpha.
00:29:41.660 --> 00:29:46.740
So this maps into plus or
minus alpha to the n.
00:29:46.740 --> 00:29:50.680
So the entire universe of
possible code words that we
00:29:50.680 --> 00:29:58.000
have is this set of 2 to the
n real n-tuples, which are
00:29:58.000 --> 00:30:01.340
simply the vertices of an
n cube of size 2 alpha,
00:30:01.340 --> 00:30:02.840
obviously, right?
00:30:02.840 --> 00:30:04.390
Just like this.
00:30:04.390 --> 00:30:10.320
We have eight possible
vertices in 3-space.
00:30:10.320 --> 00:30:12.300
They obviously all have
the same power.
00:30:12.300 --> 00:30:17.810
They all lie on the surface of
a sphere of squared radius n
00:30:17.810 --> 00:30:20.010
alpha squared.
00:30:20.010 --> 00:30:22.310
So they're not only the vertices
of a cube, they're
00:30:22.310 --> 00:30:26.810
vertices that are spread on
an equal energy sphere.
00:30:26.810 --> 00:30:32.480
And the whole idea is that our
code is going to be some
00:30:32.480 --> 00:30:36.870
subset of this, and the code
will map under the same map,
00:30:36.870 --> 00:30:43.060
which we call S, into
some subset of the
00:30:43.060 --> 00:30:45.610
vertices of the n-cube.
00:30:45.610 --> 00:30:45.990
OK?
00:30:45.990 --> 00:30:50.820
And again, our favorite example
is the tetrahedron.
00:30:50.820 --> 00:30:57.950
For instance, if we take the
code as being 0, 0, 0, 0, 1,
00:30:57.950 --> 00:31:06.150
1, 1, 0, 1 1, 1, 0, those for
binary three-tuples, this maps
00:31:06.150 --> 00:31:10.310
into T, the tetrahedron.
00:31:10.310 --> 00:31:11.656
OK?
00:31:11.656 --> 00:31:15.230
Well, we get this signal
structure, which we've already
00:31:15.230 --> 00:31:16.230
found the coding gain.
00:31:16.230 --> 00:31:18.180
It has a little bit
of coding gain.
00:31:18.180 --> 00:31:22.090
We actually accomplished
something.
00:31:22.090 --> 00:31:24.160
All right.
00:31:24.160 --> 00:31:30.780
So that's the basic idea that
we go through in 6.1.
00:31:30.780 --> 00:31:33.230
That's our favorite example.
00:31:33.230 --> 00:31:37.440
And Ashish also shows you that,
you know, you might
00:31:37.440 --> 00:31:41.040
think this is an awfully
restricted way of designing
00:31:41.040 --> 00:31:43.260
constellations.
00:31:43.260 --> 00:31:47.120
But when we're talking about low
spectral efficiencies, by
00:31:47.120 --> 00:31:50.960
going through the capacity
calculation, we can assure
00:31:50.960 --> 00:31:55.310
ourselves that in principle,
this is not going to be very
00:31:55.310 --> 00:31:57.070
sub-optimal for it.
00:31:57.070 --> 00:32:00.930
Capacity is about the regime
where n gets very large for
00:32:00.930 --> 00:32:02.970
very long block codes.
00:32:02.970 --> 00:32:06.500
We don't lose much in potential
coding gain, or in
00:32:06.500 --> 00:32:10.975
the Shannon limit, equivalently,
by restricting
00:32:10.975 --> 00:32:15.170
the input output input alphabet
to be just these two
00:32:15.170 --> 00:32:18.230
numbers, plus or minus alpha,
I should say, rather than
00:32:18.230 --> 00:32:24.310
using the whole real line as
an input, as long as the
00:32:24.310 --> 00:32:35.100
nominal spectral efficiency is
less than or equal to one bit
00:32:35.100 --> 00:32:37.840
per two dimensions.
00:32:37.840 --> 00:32:41.180
And the exact numbers --
00:32:41.180 --> 00:32:47.780
at rho equals 1, you lose about
0.2 dB, in principle,
00:32:47.780 --> 00:32:51.720
using the Shannon limit
as your guide.
00:32:51.720 --> 00:32:56.290
And for lower rho, it
just goes to 0.
00:32:56.290 --> 00:32:59.260
And you did this last time.
00:32:59.260 --> 00:32:59.650
OK.
00:32:59.650 --> 00:33:04.610
So it's a very reasonable, and
obviously attractive from an
00:33:04.610 --> 00:33:08.170
implementation point of view,
way of designing signal
00:33:08.170 --> 00:33:09.790
constellations.
00:33:09.790 --> 00:33:12.060
And again, this is basically all
we're going to be talking
00:33:12.060 --> 00:33:15.060
about for the majority of
the course, is signal
00:33:15.060 --> 00:33:16.440
constellations like this.
00:33:16.440 --> 00:33:19.530
But of course as we get to codes
that are thousands of
00:33:19.530 --> 00:33:22.960
bits long, or perhaps even
infinitely long, as in the
00:33:22.960 --> 00:33:28.820
case of convolutional codes and
trellis codes, sometimes
00:33:28.820 --> 00:33:31.390
you forget that we're really
talking about just putting
00:33:31.390 --> 00:33:35.830
points in Euclidian
signal space.
00:33:35.830 --> 00:33:39.500
Because we're going to be very
abstracted back into the
00:33:39.500 --> 00:33:42.320
binary domain.
00:33:42.320 --> 00:33:46.680
But in this course, we're always
thinking of codes as
00:33:46.680 --> 00:33:49.103
means of designing signal
constellations.
00:33:53.630 --> 00:33:57.940
Of course, codes are used far
more widely than just for
00:33:57.940 --> 00:34:01.370
signaling over the additive
white Gaussian noise channel.
00:34:01.370 --> 00:34:08.540
I've sort of packaged this
course as a search to get to
00:34:08.540 --> 00:34:10.070
capacity, we have added
the additive white
00:34:10.070 --> 00:34:11.150
Gaussian noise channel.
00:34:11.150 --> 00:34:13.580
First of all because
this corresponds
00:34:13.580 --> 00:34:16.449
very closely to history.
00:34:16.449 --> 00:34:19.830
Within this package, we can
talk about all of the
00:34:19.830 --> 00:34:22.380
principal classes of codes that
have been developed to
00:34:22.380 --> 00:34:26.355
date, and we can compare them
in some performance terms.
00:34:29.090 --> 00:34:31.860
How close do they get to
capacity on the additive white
00:34:31.860 --> 00:34:33.330
Gaussian noise channel.
00:34:33.330 --> 00:34:38.100
So you get most of what you
would get if you took a
00:34:38.100 --> 00:34:42.449
communications-free view
of coding theory.
00:34:42.449 --> 00:34:44.370
You know, people are interested
in codes for
00:34:44.370 --> 00:34:50.020
computer memories, for
God knows what,
00:34:50.020 --> 00:34:51.190
lots of other things.
00:34:51.190 --> 00:34:57.500
And for many of these other
applications, you are going to
00:34:57.500 --> 00:34:59.320
be interested in the same
class of codes.
00:34:59.320 --> 00:35:03.120
Basically, you're going to want
to maximize the distance
00:35:03.120 --> 00:35:07.030
between code words for a given
rate, which has to do with the
00:35:07.030 --> 00:35:09.300
size of the code.
00:35:09.300 --> 00:35:14.270
And so you're going to be
interested in the same codes.
00:35:14.270 --> 00:35:16.930
Putting it in the framework of
getting to capacity on the
00:35:16.930 --> 00:35:21.060
additive white Gaussian noise
channel gives a motivation,
00:35:21.060 --> 00:35:24.060
gives a very nice story, because
over 50 years, we were
00:35:24.060 --> 00:35:28.600
able to get to capacity,
and gives it a real
00:35:28.600 --> 00:35:30.240
communications flavor.
00:35:30.240 --> 00:35:33.360
Some of you, I'm sure, are here
without any interest in
00:35:33.360 --> 00:35:34.560
communications whatsoever.
00:35:34.560 --> 00:35:36.360
You simply want to know
about coding.
00:35:36.360 --> 00:35:38.830
You will still get that story,
but you'll get it
00:35:38.830 --> 00:35:41.020
in this nice package.
00:35:41.020 --> 00:35:44.840
That's why I do it this way.
00:35:44.840 --> 00:35:45.820
OK.
00:35:45.820 --> 00:35:47.490
I'm about to get
into new stuff.
00:35:47.490 --> 00:35:49.660
End of review.
00:35:49.660 --> 00:35:52.820
Any questions?
00:35:52.820 --> 00:35:56.000
So you must have done
a great job.
00:35:56.000 --> 00:35:59.120
Everyone understands
perfectly.
00:35:59.120 --> 00:35:59.740
All right.
00:35:59.740 --> 00:36:03.350
So now let's talk about,
as I said, binary
00:36:03.350 --> 00:36:06.060
linear block codes.
00:36:06.060 --> 00:36:11.220
When you see the word "linear,"
that's a signal that
00:36:11.220 --> 00:36:12.680
there's some algebra ahead.
00:36:12.680 --> 00:36:18.250
And so this is the very first
point at which we get into
00:36:18.250 --> 00:36:22.170
what's called algebraic
coding theory.
00:36:22.170 --> 00:36:24.350
The algebra will be extremely
simple at this
00:36:24.350 --> 00:36:25.660
point, not to worry.
00:36:28.250 --> 00:36:32.730
And what are we doing here?
00:36:36.520 --> 00:36:40.740
First thing we do is to identify
0 and 1 with the
00:36:40.740 --> 00:36:45.810
binary field, which I'm always
going to write as F2.
00:36:45.810 --> 00:36:51.260
The older way of writing this
is GF(2), for Galois field
00:36:51.260 --> 00:36:52.520
with two elements.
00:36:52.520 --> 00:36:54.850
They mean exactly
the same thing.
00:36:54.850 --> 00:36:59.430
Nowadays most people
write just F2.
00:36:59.430 --> 00:37:03.570
Now we have [UNINTELLIGIBLE],
we can write this blackboard
00:37:03.570 --> 00:37:06.320
F.
00:37:06.320 --> 00:37:09.230
And OK.
00:37:09.230 --> 00:37:09.960
Step one.
00:37:09.960 --> 00:37:12.540
We've identified our alphabet
with a field.
00:37:16.840 --> 00:37:20.450
So algebraically,
this is a field.
00:37:20.450 --> 00:37:23.380
Some of you know exactly
what I mean by that.
00:37:23.380 --> 00:37:24.540
Others don't.
00:37:24.540 --> 00:37:26.060
We'll come back to this again.
00:37:26.060 --> 00:37:29.380
Informally, I would say a field
is simply something
00:37:29.380 --> 00:37:32.970
where you can add, subtract,
multiply, or divide.
00:37:32.970 --> 00:37:36.825
Our best examples of that before
now have been the real
00:37:36.825 --> 00:37:39.710
and complex fields.
00:37:39.710 --> 00:37:42.560
There is a more formal
definition of that.
00:37:42.560 --> 00:37:45.740
In this case, we have
only two elements.
00:37:45.740 --> 00:37:52.840
So let me just write down the
tables, which you all know,
00:37:52.840 --> 00:37:55.500
regardless of your background.
00:37:55.500 --> 00:37:57.690
What's the addition table
for this field?
00:37:57.690 --> 00:38:01.610
Well, 0 is the additive
identity.
00:38:01.610 --> 00:38:09.100
So we know that 0 added to
anything gives itself, so that
00:38:09.100 --> 00:38:11.615
gives us three of the entries
of this table.
00:38:14.540 --> 00:38:16.550
What do we put down here?
00:38:16.550 --> 00:38:20.970
There's really only one choice
to satisfy one of the axioms
00:38:20.970 --> 00:38:27.260
of the field, which is that
under addition, the field must
00:38:27.260 --> 00:38:29.230
form a group.
00:38:29.230 --> 00:38:32.140
In particular, that means that
all elements are invertible.
00:38:32.140 --> 00:38:36.320
That means that each row or
column has to be a permutation
00:38:36.320 --> 00:38:37.540
of the group elements.
00:38:37.540 --> 00:38:40.980
And so the only possibility
here is to put in a 0.
00:38:40.980 --> 00:38:46.270
And well, we are forced, if
we're going to make 0 and 1
00:38:46.270 --> 00:38:50.600
into a field to find an addition
table which is the
00:38:50.600 --> 00:38:52.790
table of mod2 addition.
00:38:52.790 --> 00:38:55.690
You've had this stated
as an axiom before.
00:38:55.690 --> 00:39:00.640
You can derive it just from the
fact that 0 needs to be
00:39:00.640 --> 00:39:05.000
the additive identity, and then
we need to put a 0 in
00:39:05.000 --> 00:39:06.885
here in order to get
invertibility.
00:39:12.490 --> 00:39:14.120
Under multiplication.
00:39:14.120 --> 00:39:16.340
What is the multiplication
table?
00:39:16.340 --> 00:39:21.180
Well, the additive identity
is also a nullifier under
00:39:21.180 --> 00:39:22.820
multiplication in any field.
00:39:22.820 --> 00:39:26.150
So 0 times anything
is equal to 0.
00:39:26.150 --> 00:39:29.410
1 times anything is
equal to itself.
00:39:29.410 --> 00:39:34.000
So that completely determines
the multiplication table.
00:39:34.000 --> 00:39:37.870
Again, just the table of
mod2 multiplication.
00:39:37.870 --> 00:39:41.800
So if I'd done this
axiomatically, I would have
00:39:41.800 --> 00:39:43.410
given you the axioms of field.
00:39:43.410 --> 00:39:47.410
Then I would show that under
this binary addition
00:39:47.410 --> 00:39:50.800
operation, this multiplication
operation,
00:39:50.800 --> 00:39:52.780
we satisfy the axioms.
00:39:52.780 --> 00:39:54.900
I'll do that as we get
into chapter seven.
00:39:57.630 --> 00:39:58.880
OK.
00:40:00.550 --> 00:40:12.300
So now we have 0 1 to the n,
n-tuples, binary n-tuples, we
00:40:12.300 --> 00:40:17.570
will now regard as n-tuples
of field elements, F2.
00:40:20.890 --> 00:40:29.648
And these will be regarded as
vectors in a vector space.
00:40:29.648 --> 00:40:35.330
Well, I'll just say that F2 to
the n is a vector space, which
00:40:35.330 --> 00:40:37.990
clearly has 2 to
the n elements.
00:40:37.990 --> 00:40:39.290
Now again, informally.
00:40:39.290 --> 00:40:41.320
What's a vector space?
00:40:41.320 --> 00:40:46.450
A vector space is always
over a given field.
00:40:46.450 --> 00:40:47.890
In this case, it's going
to be over the
00:40:47.890 --> 00:40:49.150
binary field, of course.
00:40:49.150 --> 00:40:51.740
F2.
00:40:51.740 --> 00:40:54.720
The given field is
called a scalar.
00:40:54.720 --> 00:40:57.670
Just like vector space over
the reals, the scalars are
00:40:57.670 --> 00:40:58.760
real numbers.
00:40:58.760 --> 00:41:02.910
Here the scalars are
elements of F2.
00:41:02.910 --> 00:41:07.450
And what do we have to have to
make something algebraically a
00:41:07.450 --> 00:41:08.400
vector space?
00:41:08.400 --> 00:41:12.930
We have to have that the
addition of two vectors is
00:41:12.930 --> 00:41:19.310
well-defined and gives another
vector, and that the
00:41:19.310 --> 00:41:23.420
multiplication of a vector by
a scalar is well-defined and
00:41:23.420 --> 00:41:27.120
gives another vector in
our vector space.
00:41:27.120 --> 00:41:28.370
All right?
00:41:30.830 --> 00:41:31.500
OK.
00:41:31.500 --> 00:41:35.130
So how are we going to define
that to addition?
00:41:35.130 --> 00:41:38.600
If we want to add
two n-tuples --
00:41:38.600 --> 00:41:40.940
again, you aren't going to see
anything here that you haven't
00:41:40.940 --> 00:41:43.400
already seen in some
other context.
00:41:43.400 --> 00:41:43.860
What do we do?
00:41:43.860 --> 00:41:49.030
We add them component-wise,
using the component-wise rules
00:41:49.030 --> 00:41:51.630
of field addition.
00:41:51.630 --> 00:41:51.730
OK.
00:41:51.730 --> 00:41:55.380
So we just add two
vector n-tuples,
00:41:55.380 --> 00:41:56.910
component by component.
00:41:56.910 --> 00:42:00.570
We obviously get some result,
which is itself a binary
00:42:00.570 --> 00:42:03.290
vector or an F2 vector.
00:42:03.290 --> 00:42:05.220
No problem there.
00:42:05.220 --> 00:42:08.050
Except that --
00:42:08.050 --> 00:42:09.620
well, all right.
00:42:09.620 --> 00:42:11.980
As long as we're talking about
all possible n-tuples, the
00:42:11.980 --> 00:42:14.160
result is certainly in the
vector space, right?
00:42:14.160 --> 00:42:17.530
We add two binary n-tuples,
we get a binary n-tuple.
00:42:17.530 --> 00:42:20.320
So the result is
in F2 to the n.
00:42:20.320 --> 00:42:24.450
This is going to be the key test
when we get to subspaces
00:42:24.450 --> 00:42:27.990
of F2 to the n.
00:42:27.990 --> 00:42:28.035
OK.
00:42:28.035 --> 00:42:31.470
And multiplication
is even easier.
00:42:31.470 --> 00:42:34.100
How do we define multiplication
by scalars?
00:42:34.100 --> 00:42:35.560
Well, we only have
two scalars --
00:42:35.560 --> 00:42:37.570
0 and 1.
00:42:37.570 --> 00:42:45.530
So 0 times any vector is going
to equal the all 0 vector.
00:42:45.530 --> 00:42:47.770
Again, you can view that
as just component-wise
00:42:47.770 --> 00:42:51.480
multiplication of everything
by 0, and since 0 times
00:42:51.480 --> 00:42:53.270
anything equals 0, we're
always going
00:42:53.270 --> 00:42:55.400
to get the 0 vector.
00:42:55.400 --> 00:42:57.150
All right?
00:42:57.150 --> 00:43:02.210
So is the 0 vector in
the vector space?
00:43:02.210 --> 00:43:02.800
Well, yes.
00:43:02.800 --> 00:43:05.130
If we're talking about several
n-tuples, it of
00:43:05.130 --> 00:43:06.770
course always is.
00:43:06.770 --> 00:43:12.610
And one times anything, again,
we can just do this
00:43:12.610 --> 00:43:14.970
component-wise, and it
just gives itself.
00:43:17.480 --> 00:43:18.560
Trivially.
00:43:18.560 --> 00:43:21.950
So since this was a vector in
the vector space, this is
00:43:21.950 --> 00:43:25.660
certainly a vector in
the vector space.
00:43:25.660 --> 00:43:26.080
OK.
00:43:26.080 --> 00:43:29.745
This seems pretty
trivial so far.
00:43:29.745 --> 00:43:35.750
But what's a binary
linear block code?
00:43:35.750 --> 00:43:40.620
Again, focusing on the
linear, is a --
00:43:40.620 --> 00:43:43.400
now I'll give a formal
definition.
00:43:43.400 --> 00:43:51.660
it's a subspace of F2
to the n for some n
00:43:51.660 --> 00:43:52.910
called the block length.
00:43:55.430 --> 00:43:56.630
All right.
00:43:56.630 --> 00:43:58.700
What do I mean when
I say a subspace?
00:43:58.700 --> 00:44:03.080
I mean subsets of the elements
of a vector space that itself
00:44:03.080 --> 00:44:05.010
forms a vector space.
00:44:05.010 --> 00:44:11.110
So in this case when I
say subset, I mean a
00:44:11.110 --> 00:44:14.270
set of binary n-tuples.
00:44:14.270 --> 00:44:16.280
OK?
00:44:16.280 --> 00:44:18.460
That itself forms
a vector space.
00:44:18.460 --> 00:44:18.810
OK.
00:44:18.810 --> 00:44:20.570
What are the components
of that?
00:44:20.570 --> 00:44:24.540
To check that it forms a vector
space, let's see.
00:44:24.540 --> 00:44:30.032
Multiplication by scalars is,
again, easy to check.
00:44:30.032 --> 00:44:34.110
If it's going to be a subspace,
then the all 0 --
00:44:34.110 --> 00:44:48.140
so it has to contain the all 0
vector in order that when I
00:44:48.140 --> 00:44:50.860
multiply by the scalar
0, I get another
00:44:50.860 --> 00:44:52.110
element of this subspace.
00:44:55.260 --> 00:44:59.070
Multiplication by 1 is always
trivially satisfied.
00:44:59.070 --> 00:45:01.830
If I start off with a set, I
multiply by 1, I'm going to
00:45:01.830 --> 00:45:04.960
get the same set.
00:45:04.960 --> 00:45:14.120
So let's check whether the
elements of a subspace are
00:45:14.120 --> 00:45:17.760
closed under vector addition.
00:45:17.760 --> 00:45:19.140
What do I mean by that?
00:45:19.140 --> 00:45:24.270
I mean if you add two elements
of the of the subspace
00:45:24.270 --> 00:45:28.440
together, you get another
element of the subspace.
00:45:28.440 --> 00:45:32.255
That's the key property
to check.
00:45:36.650 --> 00:45:40.670
Key property -- we can
write that as closure
00:45:40.670 --> 00:45:42.995
under vector addition.
00:45:47.900 --> 00:45:52.280
Which is also called
the group property.
00:45:52.280 --> 00:45:56.460
It means that just under
addition, under vector
00:45:56.460 --> 00:46:00.090
addition, the set of elements
that you have forms a group.
00:46:00.090 --> 00:46:01.850
You add any two elements,
you get another
00:46:01.850 --> 00:46:03.100
element of the subset.
00:46:06.350 --> 00:46:12.030
The example is our favorite
example so far.
00:46:12.030 --> 00:46:18.420
Let's take this little code,
and I'm going to ask.
00:46:18.420 --> 00:46:24.150
Is that a subspace of
F2 to the three?
00:46:24.150 --> 00:46:35.580
So does this equal subspace
of the set of all binary
00:46:35.580 --> 00:46:36.830
three-tuples?
00:46:40.620 --> 00:46:46.880
Anyone care to hazard a guess
whether it is or isn't?
00:46:46.880 --> 00:46:47.810
AUDIENCE: It is.
00:46:47.810 --> 00:46:48.790
PROFESSOR: It is?
00:46:48.790 --> 00:46:50.040
Why?
00:46:53.012 --> 00:46:54.380
AUDIENCE: It has the
all 0 vector.
00:46:54.380 --> 00:46:58.200
PROFESSOR: It has the all
0 vector, first of all.
00:46:58.200 --> 00:47:00.480
Good.
00:47:00.480 --> 00:47:01.730
AUDIENCE: [INAUDIBLE]
00:47:04.170 --> 00:47:07.588
PROFESSOR: And it's closed
under addition.
00:47:07.588 --> 00:47:11.470
Now how might we see that?
00:47:11.470 --> 00:47:14.820
You could, of course, just
take all pair-wise --
00:47:14.820 --> 00:47:17.400
you could form the addition
table of these four elements,
00:47:17.400 --> 00:47:20.130
and you would find that you
always get another one of
00:47:20.130 --> 00:47:20.690
these elements.
00:47:20.690 --> 00:47:26.240
For instance, 0, 1, 1 plus 1,
0, 1, is equal to 1, 1, 0.
00:47:26.240 --> 00:47:31.150
In fact, you easily see that you
take any two of these, add
00:47:31.150 --> 00:47:33.155
them together, you get
the third one.
00:47:33.155 --> 00:47:37.840
If you call this a, b, and c,
a plus b plus c equals 0,
00:47:37.840 --> 00:47:40.460
addition is the same as
subtraction, because we're in
00:47:40.460 --> 00:47:42.460
a binary field.
00:47:42.460 --> 00:47:46.620
So that means that a plus b
equals c, a equals b plus c, c
00:47:46.620 --> 00:47:50.820
equals b plus a, whatever
you like.
00:47:50.820 --> 00:47:55.000
And of course, if you add 0 to
anything, it's trivially
00:47:55.000 --> 00:47:56.250
closed under that.
00:47:59.090 --> 00:47:59.580
All right.
00:47:59.580 --> 00:48:01.060
So it is.
00:48:01.060 --> 00:48:02.370
It satisfies --
00:48:02.370 --> 00:48:04.320
it's all you've got to check.
00:48:09.415 --> 00:48:13.430
A more abstract proof of this
would be, this is the set of
00:48:13.430 --> 00:48:16.360
all even-weight three-tuples.
00:48:16.360 --> 00:48:19.370
If I add even to even, I'm
going to get even.
00:48:19.370 --> 00:48:21.440
So of course my result is
going to be another
00:48:21.440 --> 00:48:24.540
even-weight three-tuple,
therefore in the set.
00:48:24.540 --> 00:48:26.130
That's the more algebraic
proof.
00:48:29.560 --> 00:48:31.760
All right.
00:48:31.760 --> 00:48:36.140
Suppose I just add 0, 0, 1
to all of these things.
00:48:36.140 --> 00:48:38.470
I'll get the set of all
odd-weight n-tuples.
00:48:41.520 --> 00:48:43.680
Add any odd-weight
n-tuple to this.
00:48:43.680 --> 00:48:46.800
Let me take the C prime,
which is the set of
00:48:46.800 --> 00:48:48.050
all odd-weight n-tuples.
00:48:50.140 --> 00:48:52.195
Is that a vector space?
00:48:54.950 --> 00:48:57.360
It doesn't have 0, and in fact,
it's not even closed
00:48:57.360 --> 00:48:59.920
under vector addition.
00:48:59.920 --> 00:49:00.230
All right.
00:49:00.230 --> 00:49:06.060
If I take in C double prime is
equal to 0, 0, 0 0, 1, 1, 1,
00:49:06.060 --> 00:49:10.135
0, 1 and I stop there,
is that a subspace?
00:49:13.120 --> 00:49:13.390
No.
00:49:13.390 --> 00:49:15.850
Because?
00:49:15.850 --> 00:49:17.530
Not closed.
00:49:17.530 --> 00:49:21.000
For instance, if I add these
two together, I would get
00:49:21.000 --> 00:49:23.670
that, and that's missing.
00:49:23.670 --> 00:49:24.040
OK.
00:49:24.040 --> 00:49:31.080
So actually, everything is
much simpler when we're
00:49:31.080 --> 00:49:32.640
talking about finite fields.
00:49:32.640 --> 00:49:35.820
All the finite dimensional
vector spaces consist of a
00:49:35.820 --> 00:49:38.140
finite number of elements.
00:49:38.140 --> 00:49:42.310
It's easier than real and
complex vector spaces.
00:49:42.310 --> 00:49:46.856
There's no analysis involved.
00:49:46.856 --> 00:49:47.430
So forth.
00:49:47.430 --> 00:49:47.960
All right.
00:49:47.960 --> 00:49:52.960
So a binary linear block
code is in a subspace
00:49:52.960 --> 00:49:55.630
of F2 to the n.
00:49:55.630 --> 00:49:59.430
What are some of the key
algebraic facts we know about
00:49:59.430 --> 00:50:06.460
vector spaces from our study
of linear algebra, which I
00:50:06.460 --> 00:50:11.380
assume all of you have had
in some form or another?
00:50:11.380 --> 00:50:14.210
What's a key algebraic property
of a vector space?
00:50:14.210 --> 00:50:18.250
What's the very first
property?
00:50:18.250 --> 00:50:20.894
All vector spaces have a --
00:50:20.894 --> 00:50:23.820
AUDIENCE: [INAUDIBLE]
00:50:23.820 --> 00:50:25.080
PROFESSOR: Norm?
00:50:25.080 --> 00:50:26.410
No.
00:50:26.410 --> 00:50:27.250
Dimension.
00:50:27.250 --> 00:50:30.710
Dimension, that's
where I'm going.
00:50:30.710 --> 00:50:33.484
What's the significance
of dimension?
00:50:33.484 --> 00:50:34.734
AUDIENCE: [INAUDIBLE]
00:50:36.860 --> 00:50:40.410
PROFESSOR: The definition of
dimension is the number of
00:50:40.410 --> 00:50:42.750
generators in any basis.
00:50:42.750 --> 00:50:45.010
So we're talking about
generators of the vector
00:50:45.010 --> 00:50:49.840
space, or a set of generators
which form a basis.
00:50:49.840 --> 00:50:53.600
If you think we have the same
properties here in the case of
00:50:53.600 --> 00:51:01.310
vector spaces over finite
fields, well, we probably do.
00:51:01.310 --> 00:51:02.970
But let's see how
it works out.
00:51:02.970 --> 00:51:05.620
So we're talking about
things like
00:51:05.620 --> 00:51:10.930
generators, basis, dimension.
00:51:10.930 --> 00:51:16.840
These are all closely
interlinked properties.
00:51:16.840 --> 00:51:20.660
Again, the fact that everything
is finite here
00:51:20.660 --> 00:51:24.840
gives us very elementary
ways of addressing
00:51:24.840 --> 00:51:27.510
all of these concepts.
00:51:27.510 --> 00:51:29.300
We don't have any
geometry yet.
00:51:29.300 --> 00:51:30.290
We don't have norms.
00:51:30.290 --> 00:51:31.310
We don't have distances.
00:51:31.310 --> 00:51:32.260
We don't have angles.
00:51:32.260 --> 00:51:34.300
We'll talk about that
in a minute.
00:51:34.300 --> 00:51:38.940
Here we just have a set that
basically has these two
00:51:38.940 --> 00:51:39.530
properties.
00:51:39.530 --> 00:51:41.890
It contains the all-0 vector,
and it's closed
00:51:41.890 --> 00:51:42.745
as the group property.
00:51:42.745 --> 00:51:46.280
It's closed under
vector addition.
00:51:46.280 --> 00:51:46.860
All right.
00:51:46.860 --> 00:51:56.530
So the first property is that
the all-0 vector is always in
00:51:56.530 --> 00:52:02.080
the subspace, which I'll
represent by C, meaning code.
00:52:02.080 --> 00:52:06.340
So when I talk about a code
now, I'm talking about a
00:52:06.340 --> 00:52:09.410
binary linear block code,
which by definition is a
00:52:09.410 --> 00:52:13.360
vector space, a subspace
of F2 to the n, where
00:52:13.360 --> 00:52:16.050
n is the code length.
00:52:16.050 --> 00:52:16.430
All right.
00:52:16.430 --> 00:52:19.700
Let me try to find a set of
generators for the code.
00:52:22.320 --> 00:52:22.546
All right?
00:52:22.546 --> 00:52:28.040
If somebody gives me a code,
they say, this is a binary
00:52:28.040 --> 00:52:29.930
linear block code.
00:52:29.930 --> 00:52:33.530
Let me see if I can find a
set of generators for it.
00:52:33.530 --> 00:52:37.180
If I find three generators,
then I'll know the
00:52:37.180 --> 00:52:39.280
dimension is three.
00:52:39.280 --> 00:52:40.700
That's basically where
I'm going.
00:52:40.700 --> 00:52:44.440
I'll state this a little bit
more formally as we go ahead.
00:52:44.440 --> 00:52:46.090
So suppose I'm given a code.
00:52:46.090 --> 00:52:49.710
I know it's a binary linear
block code, so I know it has
00:52:49.710 --> 00:52:52.970
the all-0 element.
00:52:52.970 --> 00:52:56.500
So how might I go about finding
a set of generators?
00:52:59.410 --> 00:53:01.170
Let's just take a greedy
algorithm.
00:53:01.170 --> 00:53:02.420
All right?
00:53:05.320 --> 00:53:08.060
Suppose the code contains
only the all-0 vector.
00:53:08.060 --> 00:53:09.575
Is that a subspace?
00:53:12.250 --> 00:53:15.410
What's its dimension?
00:53:15.410 --> 00:53:15.540
0.
00:53:15.540 --> 00:53:16.790
All right?
00:53:18.900 --> 00:53:22.260
So that's a very trivial vector
space, but it's a
00:53:22.260 --> 00:53:23.210
vector space.
00:53:23.210 --> 00:53:27.120
Satisfies the axioms.
00:53:27.120 --> 00:53:29.890
Suppose it is not the
trivial code.
00:53:29.890 --> 00:53:32.560
That means it has more than
the all-0 vector.
00:53:32.560 --> 00:53:38.330
So I take as my first generator
any non-zero vector.
00:53:42.940 --> 00:53:43.410
OK?
00:53:43.410 --> 00:53:46.940
I can always do that.
00:53:46.940 --> 00:53:49.510
Don't have the axiom of choice
involved here, because
00:53:49.510 --> 00:53:51.390
everything is finite.
00:53:51.390 --> 00:53:54.930
So I'm going to take g1 to
be any non-zero vector.
00:53:54.930 --> 00:53:57.020
Now I've got a generator.
00:53:57.020 --> 00:54:06.295
And how many code words
does it generate?
00:54:10.510 --> 00:54:14.150
I want to take the set of all
binary linear combinations of
00:54:14.150 --> 00:54:16.980
all the generators that
I have so far.
00:54:16.980 --> 00:54:19.990
At this point, the binary linear
combinations are 0
00:54:19.990 --> 00:54:25.400
times g1 and 1 times g1 And that
just gives me this word
00:54:25.400 --> 00:54:26.650
and this word.
00:54:26.650 --> 00:54:29.620
So now I have counted for two
words with one generator.
00:54:32.480 --> 00:54:34.040
Could it be that that's
the whole code?
00:54:36.990 --> 00:54:39.100
Sure.
00:54:39.100 --> 00:54:43.460
The all-0 vector and any
non-zero vector together form
00:54:43.460 --> 00:54:46.225
a one-dimensional subspace.
00:54:49.670 --> 00:54:51.050
That's all you can get
from one dimension.
00:54:51.050 --> 00:54:53.420
So I could be finished
here now.
00:54:53.420 --> 00:54:55.870
But if I'm not finished, there's
still more code words
00:54:55.870 --> 00:54:57.670
that I haven't accounted for.
00:54:57.670 --> 00:55:01.580
Then I greedily pick
a second generator.
00:55:01.580 --> 00:55:15.180
So this is now, let's say, any
vector not generated by g1.
00:55:20.740 --> 00:55:22.700
So I have a branch here.
00:55:22.700 --> 00:55:27.860
Either I've finished or I can
pick another generator g2.
00:55:27.860 --> 00:55:33.580
Now with g1 and g-two, how many
vectors can I generate?
00:55:33.580 --> 00:55:36.930
Let me take all binary
linear combinations.
00:55:41.350 --> 00:55:47.600
So a binary linear combination
is any vector of the form
00:55:47.600 --> 00:55:54.865
alpha1 g1 plus alpha2 g2 where
these are both scalars.
00:55:57.470 --> 00:56:02.830
And therefore this can be 0 or
1, this could be 0 or 1.
00:56:02.830 --> 00:56:03.880
So what have I got now?
00:56:03.880 --> 00:56:09.830
I've got 0, g1 g2 and g1 plus g2
I've got four binary linear
00:56:09.830 --> 00:56:11.380
combinations of two
generators.
00:56:15.340 --> 00:56:17.520
Could that be the whole code?
00:56:17.520 --> 00:56:19.040
Certainly.
00:56:19.040 --> 00:56:23.613
At this point, again, consider
our standing example.
00:56:26.730 --> 00:56:29.430
I start out what I take
as my first generator.
00:56:29.430 --> 00:56:38.480
Let me take g1 equal 1 0, 1, g2
equals whatever, 0, 1, 1.
00:56:38.480 --> 00:56:42.180
Then if I take all binary linear
combinations of these
00:56:42.180 --> 00:56:45.030
two generators, I'm going to
get the whole code, right?
00:56:45.030 --> 00:56:46.690
These four code words.
00:56:46.690 --> 00:56:48.733
They can all be expressed
in this form.
00:56:51.410 --> 00:56:56.060
Or I'm not done, and then
I have to pick g3.
00:56:56.060 --> 00:56:59.940
And how many binary linear
combinations are there of g1,
00:56:59.940 --> 00:57:01.190
g2, and g3?
00:57:03.210 --> 00:57:04.400
Eight.
00:57:04.400 --> 00:57:06.235
Are they all necessarily
in my subspace?
00:57:09.150 --> 00:57:14.300
Yes, by the fact that the
subspace is closed under
00:57:14.300 --> 00:57:16.040
scalar multiplication.
00:57:16.040 --> 00:57:17.830
Alpha g1.
00:57:17.830 --> 00:57:22.860
Alpha1 g1, alpha2 g2 alpha3 g3
are all in the subspace.
00:57:22.860 --> 00:57:29.450
Any vector addition of any of
these scalar multiples is in
00:57:29.450 --> 00:57:30.280
the subspace.
00:57:30.280 --> 00:57:36.010
So I get now eight possible
elements of the subspace, and
00:57:36.010 --> 00:57:37.285
I either may be done or not.
00:57:40.820 --> 00:57:48.760
Continuing in this way, I get
some number gk of generators,
00:57:48.760 --> 00:57:53.410
just by picking greedily the
next one until I'm done.
00:57:53.410 --> 00:57:55.140
All right?
00:57:55.140 --> 00:57:57.500
When I'm done --
00:57:57.500 --> 00:57:58.510
so I have to stop.
00:57:58.510 --> 00:57:59.760
Why do I have to stop?
00:58:02.990 --> 00:58:03.370
All right.
00:58:03.370 --> 00:58:04.660
Let's look at the size.
00:58:04.660 --> 00:58:07.750
At each point here, this
accounts for two code words,
00:58:07.750 --> 00:58:11.360
this for four, this for eight,
this for 2 to the k.
00:58:11.360 --> 00:58:15.870
How many binary n-tuples
are there?
00:58:15.870 --> 00:58:16.416
2 to the n.
00:58:16.416 --> 00:58:17.890
All right?
00:58:17.890 --> 00:58:21.300
So I clearly can't find more
than n generators.
00:58:21.300 --> 00:58:25.110
More more than n independent
generators, in the sense that
00:58:25.110 --> 00:58:28.840
the set of all the
binary linear
00:58:28.840 --> 00:58:32.050
combinations are distinct.
00:58:32.050 --> 00:58:32.460
All right.
00:58:32.460 --> 00:58:36.740
So k is, at most,
going to be n.
00:58:36.740 --> 00:58:39.310
So I will stop in a finite
number of steps.
00:58:39.310 --> 00:58:42.020
I'll stop at some number k.
00:58:42.020 --> 00:58:46.690
When I've stopped, that's
because the code consists of
00:58:46.690 --> 00:58:50.100
all binary linear combinations
of these k generators, and
00:58:50.100 --> 00:58:54.480
therefore has size 2 to the k.
00:58:54.480 --> 00:59:02.640
So the only possible size of a
subspace is 2 to the k for k
00:59:02.640 --> 00:59:04.220
less than n.
00:59:04.220 --> 00:59:10.160
A power of 2 where the
power is, at most, n.
00:59:10.160 --> 00:59:13.670
So any subspace besides
the subspace --
00:59:13.670 --> 00:59:14.920
I'm repeating myself.
00:59:17.910 --> 00:59:18.680
OK.
00:59:18.680 --> 00:59:25.370
So this means I found a basis.
00:59:25.370 --> 00:59:28.760
Does this mean that all possible
bases of any subspace
00:59:28.760 --> 00:59:30.010
have the same size?
00:59:33.284 --> 00:59:34.490
Well yeah, it must.
00:59:34.490 --> 00:59:40.650
I mean, I've proved now that any
subspace has to have this
00:59:40.650 --> 00:59:45.000
size 2 to the k for some k.
00:59:45.000 --> 00:59:47.660
So obviously, if I go through
this process, no matter how I
00:59:47.660 --> 00:59:54.330
choose my generators, I could
choose any pair of these
00:59:54.330 --> 00:59:57.550
non-zero n-tuples as
my generators.
00:59:57.550 --> 00:59:59.250
So that would be a
legitimate basis.
00:59:59.250 --> 01:00:01.320
Take any two out
of these three.
01:00:01.320 --> 01:00:04.010
But it's always going to take
exactly two of them, right?
01:00:04.010 --> 01:00:05.060
Why?
01:00:05.060 --> 01:00:08.510
Because the size of this
subspace is four.
01:00:08.510 --> 01:00:10.140
Two to the two.
01:00:10.140 --> 01:00:13.060
So if I go through this process,
I'm always going to
01:00:13.060 --> 01:00:17.590
come up with k generators, where
k is the log to the base
01:00:17.590 --> 01:00:21.340
2 of the size of this code that
I was given, which I was
01:00:21.340 --> 01:00:27.810
told was a linear code, meaning
it's a subspace.
01:00:27.810 --> 01:00:33.330
So I'm somewhat free to choose
the generators, but I'm always
01:00:33.330 --> 01:00:35.820
going to come up with k of
them if the code has
01:00:35.820 --> 01:00:37.070
size 2 to the k.
01:00:39.900 --> 01:00:50.160
So a basis is a set of k
linearly independent k-tuples,
01:00:50.160 --> 01:00:52.750
where linear independence has
the same meaning here as
01:00:52.750 --> 01:00:56.580
you're accustomed to, meaning
that all linear combinations
01:00:56.580 --> 01:01:00.310
of the k generators
are distinct.
01:01:00.310 --> 01:01:05.020
So I get 2 to the k distinct
elements of the code.
01:01:05.020 --> 01:01:10.210
And I say the dimension
of the code is k.
01:01:10.210 --> 01:01:13.040
In this case, basically it's the
size of the code is 2 to
01:01:13.040 --> 01:01:16.710
the k, then its dimension
is k.
01:01:16.710 --> 01:01:18.580
Has to be.
01:01:18.580 --> 01:01:24.610
And all basis have k
generators in them.
01:01:24.610 --> 01:01:27.800
And there are, in general,
many ways to pick them.
01:01:27.800 --> 01:01:29.300
All right?
01:01:29.300 --> 01:01:33.820
So just by considering this
greedy basis construction
01:01:33.820 --> 01:01:43.440
algorithm, I basically find the
size of any subspace is a
01:01:43.440 --> 01:01:46.850
power of two, and the power
is equal to the dimension.
01:01:46.850 --> 01:01:49.870
And any basis is going to
have that cardinality.
01:01:52.730 --> 01:01:53.580
Are you with me?
01:01:53.580 --> 01:01:57.150
This is a pretty simple proof.
01:01:57.150 --> 01:02:02.070
And I call this an n, k binary
linear block code.
01:02:02.070 --> 01:02:04.930
n being the length.
01:02:04.930 --> 01:02:09.950
That just means that every code
word is an n-tuple over
01:02:09.950 --> 01:02:11.660
the binary field.
01:02:11.660 --> 01:02:13.670
And k is the dimension.
01:02:13.670 --> 01:02:20.510
So an n, k binary linear block
code has size 2 to the k.
01:02:24.410 --> 01:02:29.170
That's 2 to the k distinct
code words.
01:02:29.170 --> 01:02:31.000
Main example is this guy.
01:02:35.070 --> 01:02:35.640
Easy?
01:02:35.640 --> 01:02:36.620
Any questions?
01:02:36.620 --> 01:02:37.910
I think this is clear.
01:02:37.910 --> 01:02:41.155
AUDIENCE: Can n use the number
of code word, I take
01:02:41.155 --> 01:02:44.190
like 2 to the k?
01:02:44.190 --> 01:02:48.060
PROFESSOR: n is something I
specify a priori as the length
01:02:48.060 --> 01:02:50.660
of every vector in the code.
01:02:50.660 --> 01:02:56.260
In other words, it has size 2 to
the k, and it's a subset of
01:02:56.260 --> 01:02:58.320
the set of all binary
n-tuples, which
01:02:58.320 --> 01:02:59.550
I write like that.
01:02:59.550 --> 01:03:01.050
In other words, the
elements of the
01:03:01.050 --> 01:03:03.662
code are binary n-tuples.
01:03:03.662 --> 01:03:05.580
If I write them out, each
one has length n.
01:03:09.610 --> 01:03:10.130
OK?
01:03:10.130 --> 01:03:12.240
We're good?
01:03:12.240 --> 01:03:13.490
What are some other examples?
01:03:18.810 --> 01:03:23.720
The simplest codes you can
think of is, first of
01:03:23.720 --> 01:03:26.600
all, an n, 0 code.
01:03:26.600 --> 01:03:29.800
That means a code with dimension
zero, has size what?
01:03:32.380 --> 01:03:33.510
1.
01:03:33.510 --> 01:03:35.350
And what does it consist of?
01:03:35.350 --> 01:03:37.230
AUDIENCE: [INAUDIBLE]
01:03:37.230 --> 01:03:37.700
PROFESSOR: Yeah.
01:03:37.700 --> 01:03:41.870
So this is the so-called trivial
code, just containing
01:03:41.870 --> 01:03:45.450
the all-0 word.
01:03:45.450 --> 01:03:46.780
Has to mention 0.
01:03:46.780 --> 01:03:49.790
It is a binary linear block
code, but it's not much use
01:03:49.790 --> 01:03:52.490
for communication.
01:03:52.490 --> 01:03:55.720
So but nonetheless, we include
that in this family.
01:03:58.420 --> 01:04:01.760
Another trivial one is n, n.
01:04:01.760 --> 01:04:03.068
What's that?
01:04:03.068 --> 01:04:06.000
AUDIENCE: [INAUDIBLE]
01:04:06.000 --> 01:04:07.480
PROFESSOR: F2 to the n, right.
01:04:07.480 --> 01:04:08.730
What's its size?
01:04:08.730 --> 01:04:09.310
2 to the n.
01:04:09.310 --> 01:04:15.370
That means it has to contain all
distinct binary n-tuples.
01:04:15.370 --> 01:04:18.340
So this is called the
trivial code.
01:04:18.340 --> 01:04:21.210
This is called the
universe code.
01:04:21.210 --> 01:04:24.470
Contains the entire universe
of binary n-tuples.
01:04:27.160 --> 01:04:29.920
Let's get some slightly
less trivial ones.
01:04:29.920 --> 01:04:32.850
n, 1.
01:04:32.850 --> 01:04:36.330
What would that be?
01:04:36.330 --> 01:04:38.280
An n, 1 code.
01:04:38.280 --> 01:04:40.640
What's its size?
01:04:40.640 --> 01:04:41.310
2.
01:04:41.310 --> 01:04:42.560
What does it consist of?
01:04:44.940 --> 01:04:48.020
The 0 word, and?
01:04:48.020 --> 01:04:50.760
Any other non-zero generator.
01:04:50.760 --> 01:04:52.250
And that's correct.
01:04:52.250 --> 01:04:59.160
This can be 0 and any generator,
two words.
01:04:59.160 --> 01:05:03.440
In communications, where we want
to maximize the distance,
01:05:03.440 --> 01:05:10.330
in some sense, between the
two code words, what is g
01:05:10.330 --> 01:05:12.650
always taken as?
01:05:12.650 --> 01:05:14.430
0,1's, right.
01:05:14.430 --> 01:05:20.160
So if it's in particular the
all-0 and the all-1 word,
01:05:20.160 --> 01:05:23.410
which I might write as a vector
of 0's and a vector of
01:05:23.410 --> 01:05:28.846
1's, this is called the
repetition code.
01:05:28.846 --> 01:05:32.600
The binary repetition
code of length n.
01:05:32.600 --> 01:05:36.310
It either gives me a 0 and I
repeat it n times, or gives me
01:05:36.310 --> 01:05:39.010
a 1 and I repeat it n times.
01:05:39.010 --> 01:05:41.860
So whenever you see n,1, you
can pretty well assume it's
01:05:41.860 --> 01:05:48.280
the repetition code, though
it might be any pair 0, g.
01:05:48.280 --> 01:05:49.490
And n, minus 1.
01:05:49.490 --> 01:05:50.860
This is an interesting one.
01:05:58.400 --> 01:06:07.930
Again, while this could
be a lot of things, in
01:06:07.930 --> 01:06:12.410
communications, whenever you see
this, this will always be
01:06:12.410 --> 01:06:16.320
the set of all even-weight
n-tuples.
01:06:22.040 --> 01:06:28.590
In other words, the set of all
n-tuples with even parity such
01:06:28.590 --> 01:06:31.310
that if you sum up all of
the components of any
01:06:31.310 --> 01:06:35.340
vector, mod2 equals 0.
01:06:35.340 --> 01:06:36.340
OK?
01:06:36.340 --> 01:06:47.110
So this I will call the single
parity check, or more briefly,
01:06:47.110 --> 01:06:51.410
the SPC code, or the
even-weight code.
01:06:51.410 --> 01:06:52.660
That's equally good.
01:06:55.260 --> 01:07:01.200
And here we maybe should do
a little bit more work.
01:07:01.200 --> 01:07:03.350
Say, is this in fact
a subspace?
01:07:03.350 --> 01:07:06.080
Does it include the all-zero
code word?
01:07:06.080 --> 01:07:08.990
Yes, all-zero has even weight.
01:07:08.990 --> 01:07:12.350
The sum of any two even-weight
code words, an even-weight
01:07:12.350 --> 01:07:15.810
code word, an even-weight
n-tuple.
01:07:15.810 --> 01:07:17.360
Yes.
01:07:17.360 --> 01:07:18.040
As here.
01:07:18.040 --> 01:07:19.250
This is an example.
01:07:19.250 --> 01:07:22.916
This is the three,
two SPC code.
01:07:27.290 --> 01:07:28.070
OK.
01:07:28.070 --> 01:07:32.242
Why is this dimension
n minus one?
01:07:32.242 --> 01:07:34.690
AUDIENCE: [INAUDIBLE]
01:07:34.690 --> 01:07:37.840
Is every code word orthogonal
to the one-vector?
01:07:37.840 --> 01:07:38.800
PROFESSOR: OK.
01:07:38.800 --> 01:07:41.620
That's an excellent answer.
01:07:41.620 --> 01:07:43.590
It's a little advanced
for us right now.
01:07:43.590 --> 01:07:45.993
I'm looking for an elementary
argument.
01:07:45.993 --> 01:07:47.243
AUDIENCE: [INAUDIBLE]
01:07:57.600 --> 01:07:57.990
PROFESSOR: OK.
01:07:57.990 --> 01:08:00.980
So you're saying we have
a set of generators
01:08:00.980 --> 01:08:03.200
that looks like this.
01:08:03.200 --> 01:08:05.210
Is that what you're saying?
01:08:05.210 --> 01:08:07.180
You are correct.
01:08:07.180 --> 01:08:12.980
And how many such generators
are there?
01:08:12.980 --> 01:08:16.229
There are n minus 1 of them.
01:08:16.229 --> 01:08:19.760
I always like to find the most
elementary argument possible.
01:08:19.760 --> 01:08:23.910
I think the most elementary
argument here is that the
01:08:23.910 --> 01:08:26.279
number of even-weight n-tuples
is equal to the number of
01:08:26.279 --> 01:08:29.160
odd-weight n-tuples, and
together they form the set of
01:08:29.160 --> 01:08:30.819
all n-tuples.
01:08:30.819 --> 01:08:34.279
So exactly half of the n-tuples
are even weight.
01:08:34.279 --> 01:08:38.029
That means there are 2 to the n
minus 1 of them, 2 to the n
01:08:38.029 --> 01:08:42.109
over 2, and therefore, the
dimension must be n minus 1.
01:08:42.109 --> 01:08:48.575
But perhaps this is just
as elementary a proof.
01:08:48.575 --> 01:08:51.124
Well, however you do it, you'll
find that there are 2
01:08:51.124 --> 01:08:56.529
to the n minus 1 of them, or
that here is clearly a set of
01:08:56.529 --> 01:08:57.260
generators.
01:08:57.260 --> 01:09:00.580
It might take a few more lines
to show that every even-weight
01:09:00.580 --> 01:09:05.720
code word is a linear
combination of this particular
01:09:05.720 --> 01:09:08.760
set of generators, but
it's certainly true.
01:09:08.760 --> 01:09:09.149
All right.
01:09:09.149 --> 01:09:13.800
So these four classes of codes,
these two entirely
01:09:13.800 --> 01:09:17.660
trivial ones, these two which
are actually somewhat more
01:09:17.660 --> 01:09:19.859
interesting for coding
purposes --
01:09:19.859 --> 01:09:23.710
we've already seen, we can get a
coding game with this length
01:09:23.710 --> 01:09:25.924
three, dimension two code --
01:09:28.450 --> 01:09:33.840
are basically the simplest
codes we can think of.
01:09:33.840 --> 01:09:36.210
The simplest binary linear
block codes.
01:09:36.210 --> 01:09:38.529
Now we'll see them
again and again.
01:09:38.529 --> 01:09:41.220
They turn up.
01:09:41.220 --> 01:09:41.649
All right?
01:09:41.649 --> 01:09:43.750
So the whole course is
going to be about
01:09:43.750 --> 01:09:45.510
finding 1's in between.
01:09:45.510 --> 01:09:47.045
More complicated ones.
01:09:47.045 --> 01:09:49.260
There's clearly more
room to play.
01:09:49.260 --> 01:09:55.650
For instance, if k is equal to
half of n, which means that
01:09:55.650 --> 01:09:59.670
rho is equal to one bit per two
dimensions, there's a lot
01:09:59.670 --> 01:10:03.360
of possibilities.
01:10:03.360 --> 01:10:06.335
And so we're going to explore
those possibilities.
01:10:12.444 --> 01:10:13.694
AUDIENCE: [INAUDIBLE]
01:10:16.276 --> 01:10:17.526
[UNINTELLIGIBLE]
01:10:27.660 --> 01:10:28.910
PROFESSOR: Sure.
01:10:31.410 --> 01:10:37.790
Let's take the 6,5 code
generated by these five
01:10:37.790 --> 01:10:39.040
generators.
01:10:40.730 --> 01:10:42.200
Well, it contains some
01:10:42.200 --> 01:10:43.525
odd-weight code words, obviously.
01:10:46.980 --> 01:10:50.750
But it's not as interesting from
a coding point of view.
01:10:54.320 --> 01:10:56.100
It's not the only one,
but it's the only one
01:10:56.100 --> 01:10:57.350
you'll ever see here.
01:11:01.220 --> 01:11:01.700
All right.
01:11:01.700 --> 01:11:02.950
Let's see.
01:11:07.580 --> 01:11:11.020
One thing I didn't point out
in the notes but probably
01:11:11.020 --> 01:11:15.510
should have here is, what is rho
for an n, k binary linear
01:11:15.510 --> 01:11:16.760
block code?
01:11:19.530 --> 01:11:21.620
How many bits can we send?
01:11:21.620 --> 01:11:24.400
Suppose we take the Euclidean
image of this.
01:11:24.400 --> 01:11:29.670
It is going to have 2 to the k
points, 2 to the k vertices of
01:11:29.670 --> 01:11:35.140
the n-cube, and so
what is rho?
01:11:35.140 --> 01:11:37.210
What is the rate in bits
per two dimensions?
01:11:37.210 --> 01:11:38.460
AUDIENCE: [INAUDIBLE]
01:11:40.526 --> 01:11:43.160
PROFESSOR: Right.
01:11:43.160 --> 01:11:49.580
The rate is basically k over n
over 2, if you like, or 2 k
01:11:49.580 --> 01:11:54.480
over n bits per two
dimensions.
01:11:54.480 --> 01:12:00.010
You can send k bits in n
dimensions, or 2 k over n bits
01:12:00.010 --> 01:12:01.870
per two dimensions.
01:12:01.870 --> 01:12:08.070
And since k can't be larger than
n, this is going to be
01:12:08.070 --> 01:12:10.900
less than or equal to two
bits per two dimensions.
01:12:10.900 --> 01:12:14.200
So again, we see we're
definitely in the
01:12:14.200 --> 01:12:20.790
power-limited regime, and that
we can really get any nominal
01:12:20.790 --> 01:12:27.910
spectral efficiency between 0
and 2 by choosing k and n
01:12:27.910 --> 01:12:30.750
appropriately.
01:12:30.750 --> 01:12:31.080
All right.
01:12:31.080 --> 01:12:34.420
So n and k determine the rate,
determine the nominal spectral
01:12:34.420 --> 01:12:35.670
efficiency.
01:12:37.930 --> 01:12:38.170
All right.
01:12:38.170 --> 01:12:42.970
Let's now talk about things that
I think are also very old
01:12:42.970 --> 01:12:44.120
hat to you.
01:12:44.120 --> 01:12:45.370
I mean weight and distance.
01:12:49.160 --> 01:12:57.840
But we begin to get into areas
that show us that these vector
01:12:57.840 --> 01:13:00.820
spaces are very different from
the real and complex vector
01:13:00.820 --> 01:13:03.000
spaces that we're
accustomed to.
01:13:03.000 --> 01:13:04.870
So what are we doing now?
01:13:04.870 --> 01:13:08.140
We're starting to get into the
geometry of this vector space.
01:13:08.140 --> 01:13:11.510
The geometry is not Euclidean
geometry, but
01:13:11.510 --> 01:13:15.090
it's Hamming geometry.
01:13:15.090 --> 01:13:20.440
We define the Hamming weight
of a vector as simply the
01:13:20.440 --> 01:13:29.000
number of 1's in v. So the
Hamming weight of the all-0
01:13:29.000 --> 01:13:31.870
vector is 0, the Hamming weight
of the all-1 vector is
01:13:31.870 --> 01:13:34.800
n, and in general, the Hamming
weight is somewhere
01:13:34.800 --> 01:13:36.020
between 0 and n.
01:13:36.020 --> 01:13:37.910
Just the number of 1's.
01:13:37.910 --> 01:13:39.980
Pretty simple.
01:13:39.980 --> 01:13:49.686
And given two vectors x and
y, what is their distance?
01:13:53.750 --> 01:13:58.145
The Hamming distance between x
and y is equal to the Hamming
01:13:58.145 --> 01:14:00.450
weight of x minus y.
01:14:00.450 --> 01:14:04.270
This is the standard way of
converting a weight metric
01:14:04.270 --> 01:14:06.480
into a distance metric.
01:14:06.480 --> 01:14:11.270
Or because addition is the
same as subtraction in a
01:14:11.270 --> 01:14:15.710
binary vector space, we might
equally well write this is as
01:14:15.710 --> 01:14:17.490
the Hamming weight
of x plus y.
01:14:20.500 --> 01:14:28.970
And more informally, this is
simply the number of places in
01:14:28.970 --> 01:14:30.220
which they differ.
01:14:37.260 --> 01:14:37.790
OK.
01:14:37.790 --> 01:14:42.350
So if x and y are identical,
then x plus y is equal to 0
01:14:42.350 --> 01:14:44.400
and the distance is 0.
01:14:44.400 --> 01:14:49.850
If they are complementary, y is
the complement of x, then
01:14:49.850 --> 01:14:51.400
they differ in every place.
01:14:51.400 --> 01:14:57.150
The sum will then be the all-1
vector, and the weight, the
01:14:57.150 --> 01:14:59.730
Hamming distance, will be n.
01:14:59.730 --> 01:15:02.060
And so again, the Hamming
distance is somewhere between
01:15:02.060 --> 01:15:05.220
0 and n, measures how
different they are.
01:15:05.220 --> 01:15:07.860
Clearly going to be important
for coding.
01:15:07.860 --> 01:15:10.510
It's going to translate directly
into Euclidean
01:15:10.510 --> 01:15:13.695
distance under this standard
2-PAM map.
01:15:13.695 --> 01:15:14.945
OK.
01:15:17.290 --> 01:15:24.570
Again, let's check that it
satisfies the distance axioms.
01:15:24.570 --> 01:15:31.150
I don't know how many of you
have seen this, but let's see.
01:15:31.150 --> 01:15:34.710
What are the distance axioms?
01:15:34.710 --> 01:15:35.960
Strict non-negativity.
01:15:40.700 --> 01:15:46.820
In other words, the Hamming
distance between x and y --
01:15:46.820 --> 01:15:48.480
that's a single comma --
01:15:48.480 --> 01:15:53.140
is greater than or equal to
0, and equality if and
01:15:53.140 --> 01:15:55.750
only if x equals y.
01:15:55.750 --> 01:15:58.280
That's what strict means.
01:15:58.280 --> 01:16:00.830
So if we find the Hamming
distance is 0, we can assert
01:16:00.830 --> 01:16:02.080
that x equals y.
01:16:04.340 --> 01:16:05.785
We have, of course, symmetry.
01:16:09.680 --> 01:16:14.430
The Hamming distance between
x and y is the same as the
01:16:14.430 --> 01:16:17.020
Hamming distance between
y and x.
01:16:17.020 --> 01:16:18.970
Order doesn't matter.
01:16:18.970 --> 01:16:26.190
And finally we have the triangle
inequality, that the
01:16:26.190 --> 01:16:31.800
Hamming distance between x and z
certainly can't be more than
01:16:31.800 --> 01:16:36.790
the Hamming distance between
x and y plus the Hamming
01:16:36.790 --> 01:16:38.800
distance between y and z.
01:16:41.490 --> 01:16:48.540
If x differs from y in only n1
places, and y differs from z
01:16:48.540 --> 01:16:52.400
in only n2 places, then clearly
z can't differ from x
01:16:52.400 --> 01:16:54.305
in more than n1 plus
n2 places.
01:16:57.740 --> 01:16:59.360
So check, check, check.
01:16:59.360 --> 01:17:04.820
This is a legitimate metric for
defining a geometry on the
01:17:04.820 --> 01:17:10.000
space, and this is the one that
we use on the space of
01:17:10.000 --> 01:17:12.140
all n-tuples.
01:17:12.140 --> 01:17:15.760
But notice it's not all like
the Euclidean Distance.
01:17:18.700 --> 01:17:22.110
Now when we have a
linear code --
01:17:22.110 --> 01:17:23.430
let's combine these things.
01:17:23.430 --> 01:17:39.010
When we have a linear code, we
have a group property which
01:17:39.010 --> 01:17:40.700
is, let me write it this way.
01:17:40.700 --> 01:17:47.430
If we take any code word and add
it to any other code word,
01:17:47.430 --> 01:17:50.940
that's in the code.
01:17:50.940 --> 01:17:58.990
And furthermore, c plus c prime
is not equal to c prime
01:17:58.990 --> 01:18:03.220
plus c single prime, c
plus c single prime,
01:18:03.220 --> 01:18:04.670
because we can --
01:18:04.670 --> 01:18:06.300
well, I'll finish it.
01:18:06.300 --> 01:18:13.290
Unless c prime equals
c double prime.
01:18:13.290 --> 01:18:14.040
Why is that?
01:18:14.040 --> 01:18:17.240
We can do subtraction,
cancellation.
01:18:17.240 --> 01:18:19.470
Cancel c out from each side.
01:18:19.470 --> 01:18:25.210
So if we add c to c double
prime, we're going to get a
01:18:25.210 --> 01:18:30.160
different result from adding c
to c prime, if c prime and c
01:18:30.160 --> 01:18:33.490
double prime are different.
01:18:33.490 --> 01:18:39.200
So this implies that
c plus C --
01:18:39.200 --> 01:18:43.010
I write that as an abbreviation
for the set of
01:18:43.010 --> 01:18:50.840
all c plus c prime, as a c prime
runs through the code C.
01:18:50.840 --> 01:18:59.130
So this is the 2 to the k sums
of the code plus any code word
01:18:59.130 --> 01:19:00.380
in the code.
01:19:03.450 --> 01:19:05.350
Sorry if I don't write
down all steps.
01:19:05.350 --> 01:19:08.950
What is that going to be?
01:19:08.950 --> 01:19:16.038
C. How did we conclude that?
01:19:16.038 --> 01:19:17.288
AUDIENCE: [INAUDIBLE]
01:19:19.510 --> 01:19:20.760
[UNINTELLIGIBLE]
01:19:24.280 --> 01:19:25.920
PROFESSOR: Perfect.
01:19:25.920 --> 01:19:28.100
Did you all hear that?
01:19:28.100 --> 01:19:30.550
By the group property,
each one of these
01:19:30.550 --> 01:19:32.190
things is in the code.
01:19:32.190 --> 01:19:36.000
By this argument, no two
of them are the same.
01:19:36.000 --> 01:19:39.760
That means I get 2 to the k
distinct elements all in the
01:19:39.760 --> 01:19:42.170
code, that's got to be the code,
because the code only
01:19:42.170 --> 01:19:43.420
has 2 to the k elements.
01:19:46.850 --> 01:19:48.630
All right.
01:19:48.630 --> 01:19:51.740
So if I add a code word --
01:19:51.740 --> 01:19:54.360
in other words, if I write
down the code --
01:19:54.360 --> 01:20:00.190
0, 0, 0, 0, 1, 1, 1 0 1, 1 1 0-
and I add any code word to
01:20:00.190 --> 01:20:04.560
it -- say I add 0
1 1 to the code.
01:20:04.560 --> 01:20:08.930
So let me just do one column
of the addition table.
01:20:08.930 --> 01:20:14.530
I get 0, 1, 1, 0, 0, 0,
1, 1, 0, 1, 0, 1.
01:20:14.530 --> 01:20:15.890
I'll get the code
itself again.
01:20:21.070 --> 01:20:24.210
This has a very important
property.
01:20:26.910 --> 01:20:38.460
The set of Hamming distances C
and c prime, as c prime runs
01:20:38.460 --> 01:20:51.660
through c, from any given code
word C, it is independent of
01:20:51.660 --> 01:20:59.920
C. So I can start from any code
word, measure the Hamming
01:20:59.920 --> 01:21:02.960
distances, the 2 to the k
Hamming distances to all other
01:21:02.960 --> 01:21:05.930
code words, including
C itself --
01:21:05.930 --> 01:21:08.390
let c prime run through the
entire code, I'm going to get
01:21:08.390 --> 01:21:11.570
a set of 2 to the k distances
called the distance
01:21:11.570 --> 01:21:13.700
profile of the code.
01:21:16.320 --> 01:21:19.670
And I claim that it doesn't
matter which code
01:21:19.670 --> 01:21:21.870
word I start from.
01:21:21.870 --> 01:21:24.510
I'm going to get the same
distance profile regardless of
01:21:24.510 --> 01:21:26.130
where I start.
01:21:26.130 --> 01:21:29.140
In other words, the set of all
distances from the all-0 word
01:21:29.140 --> 01:21:35.430
here, which is 0, 2, 2, 2, is
the same as the set of all
01:21:35.430 --> 01:21:38.270
distances from the 0,
1, 1, code word,
01:21:38.270 --> 01:21:43.480
which is 2, 0, 2, 2.
01:21:43.480 --> 01:21:47.540
And the proof is basically
that this is simply the
01:21:47.540 --> 01:21:51.940
Hamming weight of C plus c prime
as c prime runs through
01:21:51.940 --> 01:21:55.580
C. What is this equal to?
01:21:55.580 --> 01:22:01.610
So the proof is that this is
equal to the Hamming weight of
01:22:01.610 --> 01:22:07.020
c prime as c prime
runs through C.
01:22:07.020 --> 01:22:10.900
So the distance profile from any
code word is simply equal
01:22:10.900 --> 01:22:14.970
to the weight profile
of the code itself.
01:22:14.970 --> 01:22:18.820
The weight profile of this code
is 0, 2, 2, 2, Start from
01:22:18.820 --> 01:22:22.700
any code word, measure the
distances to other code words,
01:22:22.700 --> 01:22:25.980
I'm always going to get
0, 2, 2, 2, 0, to
01:22:25.980 --> 01:22:29.680
itself, and the others.
01:22:29.680 --> 01:22:32.630
Sounds sort of like the
tetrahedron, doesn't it?
01:22:32.630 --> 01:22:36.570
It's zero distance from a code
word to itself and equal
01:22:36.570 --> 01:22:39.520
distance to all the other
code words, in
01:22:39.520 --> 01:22:41.540
that particular case.
01:22:41.540 --> 01:22:41.850
OK.
01:22:41.850 --> 01:22:44.480
So again, everything is
very elementary here.
01:22:44.480 --> 01:22:50.200
The distance profile is
independent of C and equal to
01:22:50.200 --> 01:22:51.450
the weight profile.
01:23:06.330 --> 01:23:10.720
So this has an extremely
important corollary.
01:23:10.720 --> 01:23:14.440
What's the minimum Hamming
distance of the code?
01:23:14.440 --> 01:23:16.110
You might expect this
is going to be an
01:23:16.110 --> 01:23:17.450
important code parameter.
01:23:17.450 --> 01:23:18.700
AUDIENCE: [INAUDIBLE]
01:23:23.230 --> 01:23:25.900
PROFESSOR: The minimum Hamming
distance between any two code
01:23:25.900 --> 01:23:28.620
words is going to
be equal to --
01:23:28.620 --> 01:23:29.980
I think you said it.
01:23:29.980 --> 01:23:31.860
AUDIENCE: The 0
[UNINTELLIGIBLE].
01:23:31.860 --> 01:23:35.470
PROFESSOR: Non-zero
is important here.
01:23:35.470 --> 01:23:37.910
If the distance profile is equal
to the weight profile,
01:23:37.910 --> 01:23:40.660
one of the weights is always
going to be zero.
01:23:40.660 --> 01:23:43.920
And that corresponds to the
distance between a code word
01:23:43.920 --> 01:23:45.780
and itself.
01:23:45.780 --> 01:23:46.070
All right.
01:23:46.070 --> 01:23:50.170
If I go through all the other 2
to the k minus 1 distances,
01:23:50.170 --> 01:23:52.840
they're going to be weights.
01:23:52.840 --> 01:23:54.910
They're going to be the
distances from a code to all
01:23:54.910 --> 01:23:56.760
other code words.
01:23:56.760 --> 01:24:01.580
And the minimum distance is
simply going to be the minimum
01:24:01.580 --> 01:24:04.370
non-zero weight of the code.
01:24:04.370 --> 01:24:08.310
For example, in this code, the
minimum Hamming distance
01:24:08.310 --> 01:24:12.990
between any two distinct code
words is going to be equal to
01:24:12.990 --> 01:24:16.190
the minimum distance from
the 0 code word --
01:24:16.190 --> 01:24:17.320
that's another way
of doing it.
01:24:17.320 --> 01:24:20.950
Since it's independent of C, we
may as well take the base
01:24:20.950 --> 01:24:23.490
code word C to be zero.
01:24:23.490 --> 01:24:26.690
And then what's the minimum
distance to 0?
01:24:26.690 --> 01:24:27.840
To the 0-code word?
01:24:27.840 --> 01:24:29.650
It's the minimum weight.
01:24:29.650 --> 01:24:33.380
From the 0 code word, distance
equals weight.
01:24:33.380 --> 01:24:36.710
So the minimum distance is the
minimum weight of any non-zero
01:24:36.710 --> 01:24:39.300
code word, which for
this code is two.
01:24:43.500 --> 01:24:48.550
Now here the weight profile
is 0, 2, 2, 2, 0.
01:24:48.550 --> 01:24:52.490
The distance profile from any
code word to all the others is
01:24:52.490 --> 01:24:54.150
0, 2, 2,2 2.
01:24:54.150 --> 01:24:56.100
This is always the distance
to itself.
01:24:56.100 --> 01:24:59.950
So minimum distance to
other code words is
01:24:59.950 --> 01:25:03.130
always going to be two.
01:25:03.130 --> 01:25:08.560
Furthermore, the number of
nearest neighbors --
01:25:08.560 --> 01:25:12.640
to go back and use chapter
five terminology --
01:25:12.640 --> 01:25:16.530
the number of nearest neighbors
is going to be the
01:25:16.530 --> 01:25:20.230
number of code words that have
that minimum weight --
01:25:20.230 --> 01:25:22.760
in this case, three.
01:25:22.760 --> 01:25:26.800
Still sounding a lot like
a tetrahedron, isn't it?
01:25:26.800 --> 01:25:30.830
This easy map between Hamming
distance and Euclidean
01:25:30.830 --> 01:25:37.840
distance for this case and in
general for all of our cases.
01:25:37.840 --> 01:25:43.300
So corollary.
01:25:43.300 --> 01:25:48.960
The minimum Hamming distance,
which implicitly means between
01:25:48.960 --> 01:25:55.390
two distinct code words of C,
is equal to the minimum
01:25:55.390 --> 01:26:06.120
non-zero weight of C, and the
number of minimum weight code
01:26:06.120 --> 01:26:08.340
words is independent --
01:26:11.400 --> 01:26:13.830
I'm doing this backwards.
01:26:13.830 --> 01:26:18.670
From any code word, the number
of code words that distance,
01:26:18.670 --> 01:26:25.350
let's call this d, is equal
to the number of
01:26:25.350 --> 01:26:28.910
weight d code words.
01:26:28.910 --> 01:26:31.110
Sorry, you probably
can't see that.
01:26:36.360 --> 01:26:36.720
All right.
01:26:36.720 --> 01:26:39.940
So we get this symmetry property
for codes that
01:26:39.940 --> 01:26:45.900
follows from the group property
of the code that if
01:26:45.900 --> 01:26:48.510
we stand on any code word and
look out, we're always going
01:26:48.510 --> 01:26:51.590
to see the same thing.
01:26:51.590 --> 01:26:53.410
We have this constant.
01:26:53.410 --> 01:26:57.710
It's actually easiest to see
this when we make the map from
01:26:57.710 --> 01:27:01.010
the code to the Euclidean
image of the code.
01:27:08.740 --> 01:27:16.350
So the Euclidean image S of C
of the code word is going to
01:27:16.350 --> 01:27:24.760
be some set of 2 to the
k vertices of an
01:27:24.760 --> 01:27:29.295
n-cube of side alpha.
01:27:35.270 --> 01:27:41.360
Let's talk about the Euclidean
image of these properties.
01:27:41.360 --> 01:27:48.210
If the minimum Hamming distance
of the code is d,
01:27:48.210 --> 01:27:52.670
what's the minimum squared
Euclidean distance between
01:27:52.670 --> 01:27:54.470
elements of S of
C going to be?
01:27:58.450 --> 01:28:00.975
Well, let's do it coordinate
by coordinates.
01:28:03.630 --> 01:28:11.800
Let's take two code words,
C and c prime, let's say.
01:28:11.800 --> 01:28:14.560
And let's suppose we have
some Hamming distance
01:28:14.560 --> 01:28:15.910
between C and c prime.
01:28:18.740 --> 01:28:23.940
That means that c and c prime
differ in the Hamming
01:28:23.940 --> 01:28:26.450
distance, number of places.
01:28:26.450 --> 01:28:35.300
So if we map this into the
corresponding pair of vertices
01:28:35.300 --> 01:28:38.980
of the n-cube in Euclidean
space, S of C and S of c
01:28:38.980 --> 01:28:43.500
prime, how many coordinates are
these going to differ in?
01:28:46.720 --> 01:28:51.150
It's going to differ in same
number of coordinates, D_h.
01:28:51.150 --> 01:28:55.030
If they don't differ, what's
Euclidean squared distance in
01:28:55.030 --> 01:28:56.810
those coordinates?
01:28:56.810 --> 01:28:57.190
0.
01:28:57.190 --> 01:29:01.300
If they do differ, the Euclidean
squared distance is
01:29:01.300 --> 01:29:03.080
4 alpha squared.
01:29:03.080 --> 01:29:11.320
So the Euclidean distance D_e
between S of C and S of c
01:29:11.320 --> 01:29:15.000
prime is simply going to be
4 alpha squared times the
01:29:15.000 --> 01:29:18.710
Hamming distance between
C and c prime, yes?
01:29:21.490 --> 01:29:25.230
So I should say this is the
squared Euclidean distance.
01:29:25.230 --> 01:29:26.430
Why do we always talk about the
01:29:26.430 --> 01:29:27.600
squared Euclidean distance?
01:29:27.600 --> 01:29:30.730
Because it's additive,
coordinate-wise.
01:29:30.730 --> 01:29:33.900
And the Hamming distance is
additive, coordinate-wise.
01:29:33.900 --> 01:29:36.260
So there's a nice
easy map here.
01:29:38.990 --> 01:29:42.020
So what does this mean d_min
squared is going to be?
01:29:44.620 --> 01:29:50.910
d_min squared of, let's say, S
of C. This constellation that
01:29:50.910 --> 01:29:54.942
we've formed by taking the
Euclidean image of c.
01:29:54.942 --> 01:30:00.050
The minimum square distance
between points in S of C is
01:30:00.050 --> 01:30:05.140
just going to be 4 alpha squared
times d, where I don't
01:30:05.140 --> 01:30:06.810
think I ever --
01:30:06.810 --> 01:30:10.585
d equals min Hamming distance.
01:30:13.720 --> 01:30:18.980
And we're always going to talk
about n, k, d as the three key
01:30:18.980 --> 01:30:22.630
parameters of a binary
linear block code.
01:30:22.630 --> 01:30:26.920
n is the code length, F2 to the
n, k is the dimension, d
01:30:26.920 --> 01:30:29.085
is the minimum Hamming
distance.
01:30:29.085 --> 01:30:32.930
So by going into this Hamming
geometry, we've got a third
01:30:32.930 --> 01:30:34.730
key property of the code.
01:30:34.730 --> 01:30:37.300
And we see it's key, because we
can get the minimum squared
01:30:37.300 --> 01:30:40.160
distance between this Euclidean
image constellation,
01:30:40.160 --> 01:30:43.488
just 4 alpha squared d.
01:30:43.488 --> 01:30:45.868
AUDIENCE: nd makes
a probability of
01:30:45.868 --> 01:30:48.248
[UNINTELLIGIBLE] that is
dependent on [UNINTELLIGIBLE]
01:30:48.248 --> 01:30:49.210
then.
01:30:49.210 --> 01:30:49.750
PROFESSOR: Correct.
01:30:49.750 --> 01:30:53.780
This is all we need to know to
get the union bound estimate.
01:30:53.780 --> 01:30:55.660
Well, a few more things.
01:30:55.660 --> 01:31:05.590
We need to know what K_min
average of S of C. And what is
01:31:05.590 --> 01:31:08.560
that going to be?
01:31:08.560 --> 01:31:13.730
This is simply going to be the
number of words in the code.
01:31:13.730 --> 01:31:17.780
To get this minimum squared
distance, we need a Hamming
01:31:17.780 --> 01:31:20.570
distance of d.
01:31:20.570 --> 01:31:24.700
So the number of words in the
code of distance d, which is
01:31:24.700 --> 01:31:29.380
given by the parameter n sub d,
is simply going to be the
01:31:29.380 --> 01:31:31.010
number of nearest neighbors.
01:31:31.010 --> 01:31:35.210
Not just the average distance,
but I want to emphasize this
01:31:35.210 --> 01:31:36.320
symmetry property.
01:31:36.320 --> 01:31:41.700
If we stand on any point, on
any vertex of this cube in
01:31:41.700 --> 01:31:44.610
n-space, which is the code
vertex, and we look at all the
01:31:44.610 --> 01:31:48.030
other points in the
constellation, no matter which
01:31:48.030 --> 01:31:50.280
point we stand on, we
will always see the
01:31:50.280 --> 01:31:53.020
same profile of distances.
01:31:53.020 --> 01:31:57.580
We'll see precisely nd code
words at Euclidean distance 4
01:31:57.580 --> 01:31:58.990
alpha squared d.
01:31:58.990 --> 01:32:03.410
We'll see nd plus 1 at Euclidean
squared distance 4
01:32:03.410 --> 01:32:08.050
alpha squared d plus 1, and so
forth, right up the profile.
01:32:08.050 --> 01:32:12.020
So there's complete symmetry
in the constellation.
01:32:12.020 --> 01:32:15.500
In that universe, you don't know
which code point you're
01:32:15.500 --> 01:32:17.910
standing on just by looking out,
because the world looks
01:32:17.910 --> 01:32:20.330
the same to you.
01:32:20.330 --> 01:32:21.580
Is that clear?
01:32:23.820 --> 01:32:23.945
OK.
01:32:23.945 --> 01:32:27.330
So from a communications point
of view, this is important,
01:32:27.330 --> 01:32:30.720
because it means it doesn't
matter what code word we send.
01:32:30.720 --> 01:32:34.010
The probability of error from
any code word is going to be
01:32:34.010 --> 01:32:36.250
the same as the probability of
error from any other code
01:32:36.250 --> 01:32:40.150
word, because the geometry
is exactly the same.
01:32:40.150 --> 01:32:43.370
The Voronoi regions are
all the same shape.
01:32:43.370 --> 01:32:46.210
So given the exact probability
of error, not just the
01:32:46.210 --> 01:32:48.860
union-bound estimate, is going
to be independent of which
01:32:48.860 --> 01:32:50.580
code word was sent.
01:32:50.580 --> 01:32:54.310
This all follows from the fact
that it's a linear code and
01:32:54.310 --> 01:32:58.820
therefore has the group
property, which translates
01:32:58.820 --> 01:33:01.920
into this very strong
geometrical uniformity
01:33:01.920 --> 01:33:04.400
property in Euclidean space.
01:33:04.400 --> 01:33:08.020
Or actually in Hamming space
too, but it's more striking in
01:33:08.020 --> 01:33:09.270
Euclidean space.
01:33:11.740 --> 01:33:13.700
OK?
01:33:13.700 --> 01:33:17.250
So we have everything we
need to write down
01:33:17.250 --> 01:33:18.500
the union bound estimate.
01:33:22.620 --> 01:33:30.030
Union bound estimate was just
the probably of error per bit
01:33:30.030 --> 01:33:38.945
is well approximated by K_b of
constellation, in this case, S
01:33:38.945 --> 01:33:48.870
of C, times Q of the square root
of the coding gain of the
01:33:48.870 --> 01:33:52.340
constellation times
2 Eb over N_0.