WEBVTT
00:00:01.120 --> 00:00:02.900
PROFESSOR: --we'll start off
with a little review, so you
00:00:02.900 --> 00:00:06.090
might chance it.
00:00:06.090 --> 00:00:08.170
We're in the middle of
the chapter nine on
00:00:08.170 --> 00:00:09.750
convolutional codes.
00:00:09.750 --> 00:00:12.840
Today, I intend to hand
out problem set six.
00:00:12.840 --> 00:00:15.750
I believe Ashish will have
it when he gets here.
00:00:15.750 --> 00:00:19.430
And we'll hand it
out at the end.
00:00:19.430 --> 00:00:25.010
Last time, I introduced you
to a lot of things about
00:00:25.010 --> 00:00:28.770
convolutional codes and
their encoders.
00:00:28.770 --> 00:00:32.560
I stressed that a convolutional
encoder had two
00:00:32.560 --> 00:00:33.910
distinct characters.
00:00:33.910 --> 00:00:37.140
One is it's a linear
time-invariant system.
00:00:37.140 --> 00:00:41.170
So we can use the linearity
and time-invariance and
00:00:41.170 --> 00:00:45.500
algebraic analysis of
convolutional codes.
00:00:45.500 --> 00:00:49.270
Secondly, I stressed that's it's
a finite state system,
00:00:49.270 --> 00:00:51.800
because it's a finite memory
system where each memory
00:00:51.800 --> 00:00:55.630
element has a finite number of
possible values, over F2 or
00:00:55.630 --> 00:00:57.800
indeed over any finite field.
00:00:57.800 --> 00:01:00.210
And therefore, we
can use that for
00:01:00.210 --> 00:01:01.920
different kinds of analysis.
00:01:01.920 --> 00:01:05.200
We use that, for instance, in
conjunction with linear
00:01:05.200 --> 00:01:08.990
property to find the minimum
distance to a particular code.
00:01:08.990 --> 00:01:12.580
And we'll see that that's the
key to getting an efficient
00:01:12.580 --> 00:01:16.300
optimal maximum likelihood
decoder --
00:01:16.300 --> 00:01:18.820
is the fact that we only have
a finite number of states in
00:01:18.820 --> 00:01:20.400
this system.
00:01:20.400 --> 00:01:27.970
All right, so along the way I
kind of went all over the map,
00:01:27.970 --> 00:01:33.050
as far as the introducing
various algebraic quantities.
00:01:33.050 --> 00:01:36.970
I focused particularly on
formal Laurent series.
00:01:36.970 --> 00:01:40.940
And I want to just give you a
little table to make sure you
00:01:40.940 --> 00:01:43.820
have fixed in your mind what all
these different kinds of
00:01:43.820 --> 00:01:46.380
sequences we're talking
about are.
00:01:46.380 --> 00:01:49.870
On the left side, I have
Laurent sequences.
00:01:49.870 --> 00:01:51.930
These are semi-infinite
sequences that must
00:01:51.930 --> 00:01:53.140
start at some time.
00:01:53.140 --> 00:01:57.410
Their delay is not equal to
minus infinity, which would be
00:01:57.410 --> 00:01:59.830
if it were all 1's forever.
00:01:59.830 --> 00:02:01.740
But their delay is
some finite time.
00:02:01.740 --> 00:02:04.020
It could be negative,
could be positive.
00:02:04.020 --> 00:02:05.750
That's their starting time.
00:02:05.750 --> 00:02:07.930
And then they could
go on, possibly,
00:02:07.930 --> 00:02:09.299
infinitely into the future.
00:02:12.160 --> 00:02:18.500
And then, among those, we can
look particularly at the ones
00:02:18.500 --> 00:02:22.660
that start at time 0 or later,
whose delay is 0, or whose
00:02:22.660 --> 00:02:24.720
delay is non-negative.
00:02:24.720 --> 00:02:26.980
These are called the formal
power series, and you've
00:02:26.980 --> 00:02:29.250
probably encountered
them earlier.
00:02:29.250 --> 00:02:31.550
They're a lot more frequently
encountered
00:02:31.550 --> 00:02:36.170
in mathematics algebra.
00:02:36.170 --> 00:02:39.270
The rational sequences are
simply those that can be
00:02:39.270 --> 00:02:43.140
written in a very finite way as
a numerator polynomial over
00:02:43.140 --> 00:02:44.650
a denominator polynomial.
00:02:44.650 --> 00:02:50.240
Because the formal Laurent
series form a field, every
00:02:50.240 --> 00:02:54.390
element is invertible, and, in
particular, every polynomial
00:02:54.390 --> 00:02:59.520
has an inverse, which, unless
the polynomial is a monomial,
00:02:59.520 --> 00:03:02.480
is going to run on for an
infinite length of time.
00:03:02.480 --> 00:03:05.680
But nonetheless, there is a
subset, a countable subset of
00:03:05.680 --> 00:03:08.650
these sequences that we
can write in this way.
00:03:08.650 --> 00:03:11.800
And we call these the rational
sequences, and they're
00:03:11.800 --> 00:03:16.580
analogous to the rational
numbers in the real field.
00:03:16.580 --> 00:03:19.780
And their particular
characteristic, as we showed,
00:03:19.780 --> 00:03:21.940
is that they're eventually
periodic.
00:03:21.940 --> 00:03:26.460
And it's easy to see that these,
too, form a field.
00:03:26.460 --> 00:03:29.375
Obviously, the inverse of
N of d over D of d is D
00:03:29.375 --> 00:03:30.560
of d over N of d.
00:03:30.560 --> 00:03:35.220
Provided that n of d is not
equal to 0, the denominator is
00:03:35.220 --> 00:03:40.580
always required to be non-zero
for a rational sequence.
00:03:40.580 --> 00:03:44.450
So that's a very nice subset.
00:03:44.450 --> 00:03:49.220
It may or may not have
operational meaning.
00:03:49.220 --> 00:03:51.090
Certainly, when we're
talking about
00:03:51.090 --> 00:03:52.650
realizations, it has meaning.
00:03:52.650 --> 00:03:57.950
And then the finite sequences,
by eventually periodic, I will
00:03:57.950 --> 00:04:00.350
from now on include
the finite case.
00:04:00.350 --> 00:04:05.590
If we have a finite sequence,
then its end is all 0's, and
00:04:05.590 --> 00:04:08.090
that's certainly a periodic
sequence.
00:04:08.090 --> 00:04:09.830
So I would say a finite
sequence is
00:04:09.830 --> 00:04:12.420
also eventually periodic.
00:04:12.420 --> 00:04:15.100
But among the rational
sequences we have,
00:04:15.100 --> 00:04:18.540
particularly the finite
sequence, these do not form a
00:04:18.540 --> 00:04:22.520
field, because the non-monomial
polynomials
00:04:22.520 --> 00:04:23.570
cannot be inverted.
00:04:23.570 --> 00:04:26.420
So these are merely a ring.
00:04:26.420 --> 00:04:28.520
However, D has an inverse.
00:04:28.520 --> 00:04:32.280
Over here on this side, D minus
1 has an element of all
00:04:32.280 --> 00:04:33.090
these things.
00:04:33.090 --> 00:04:35.880
D has an inverse.
00:04:35.880 --> 00:04:39.820
Over here, on the causal side,
these are the causal analogs
00:04:39.820 --> 00:04:42.540
to all of these.
00:04:42.540 --> 00:04:46.220
In the case of causal rational,
we don't have
00:04:46.220 --> 00:04:47.940
particularly good notation.
00:04:47.940 --> 00:04:51.220
We just say these are the ones
that are both causal, formal
00:04:51.220 --> 00:04:54.060
power series and rational.
00:04:54.060 --> 00:04:57.600
But they are important, because
we've seen that it's
00:04:57.600 --> 00:05:01.700
this kind of sequence that you
can get as the impulse
00:05:01.700 --> 00:05:08.620
response of a time-invariant
linear system that is
00:05:08.620 --> 00:05:09.740
realizable.
00:05:09.740 --> 00:05:11.020
Realizable has two parts.
00:05:11.020 --> 00:05:15.070
One is the impulse response must
start at time 0 or later,
00:05:15.070 --> 00:05:17.520
by physical causality.
00:05:17.520 --> 00:05:22.740
Two, the impulse response must
be eventually periodic,
00:05:22.740 --> 00:05:26.370
therefore rational, if it's
going to be realizable with a
00:05:26.370 --> 00:05:27.850
finite number of memory
elements.
00:05:27.850 --> 00:05:31.200
So we include that in our
definition of realizability.
00:05:31.200 --> 00:05:34.300
So these are sometimes called
the realizable sequences, or
00:05:34.300 --> 00:05:37.030
the realizable D transforms.
00:05:37.030 --> 00:05:40.730
And these, of course, the causal
finite sequences are
00:05:40.730 --> 00:05:41.700
the polynomials.
00:05:41.700 --> 00:05:46.260
These include the polynomials,
which are trivially easy to
00:05:46.260 --> 00:05:53.460
realize with shift registers,
as we saw in our example.
00:05:53.460 --> 00:05:57.450
So that's just a review of
all these quantities.
00:05:57.450 --> 00:06:00.720
Does anyone feel the need for
further elaboration or
00:06:00.720 --> 00:06:01.390
explanation?
00:06:01.390 --> 00:06:04.290
Or can we go forward
with this?
00:06:04.290 --> 00:06:08.070
We tend to use all of them as we
go along, so I want you to
00:06:08.070 --> 00:06:10.000
have them clearly in mind.
00:06:10.000 --> 00:06:10.230
Yeah.
00:06:10.230 --> 00:06:12.580
AUDIENCE: Why was
the [INAUDIBLE]
00:06:12.580 --> 00:06:13.050
uncountable?
00:06:13.050 --> 00:06:14.930
What was the [INAUDIBLE]?
00:06:14.930 --> 00:06:18.020
PROFESSOR: Why is that
uncountable?
00:06:18.020 --> 00:06:20.980
Try to count it.
00:06:20.980 --> 00:06:23.610
I could put it in one-to-one
correspondence with a set of
00:06:23.610 --> 00:06:26.290
all binary expansions of the
real numbers, and that's an
00:06:26.290 --> 00:06:27.540
uncountable set.
00:06:35.610 --> 00:06:42.930
Some of these side comments are
just to trigger analogies
00:06:42.930 --> 00:06:48.010
in your mind, maybe make it more
reasonable, what we're
00:06:48.010 --> 00:06:50.400
talking about.
00:06:50.400 --> 00:06:53.130
OK, so next.
00:06:53.130 --> 00:07:02.150
Just again, to continue a quick
review, we talked about
00:07:02.150 --> 00:07:07.895
realizable linear time-invariant
systems.
00:07:10.900 --> 00:07:16.805
First, we talked about single
input, single output.
00:07:16.805 --> 00:07:20.470
I'm imposing an order that
didn't necessarily exist.
00:07:23.650 --> 00:07:28.150
In any case, these are
characterized by impulse
00:07:28.150 --> 00:07:44.900
response, g of D, which, for
realizability, that implies
00:07:44.900 --> 00:07:48.385
that g of D is causal
rational.
00:07:53.300 --> 00:07:57.840
The fact that it's rational
means that we can write g of D
00:07:57.840 --> 00:08:04.730
as n of D over d of D. And the
fact that it's causal, well,
00:08:04.730 --> 00:08:08.460
we would always reduce this to
lowest terms in the first
00:08:08.460 --> 00:08:11.540
place, so there'd be no point
carrying along common factors
00:08:11.540 --> 00:08:14.980
in the numerator and
denominator.
00:08:14.980 --> 00:08:18.270
By inserting enough powers of D,
we can make sure that both
00:08:18.270 --> 00:08:20.250
of these are polynomials.
00:08:20.250 --> 00:08:25.810
So if n of D, d of D are --
00:08:25.810 --> 00:08:28.880
we can ensure that they're
actually polynomials, not just
00:08:28.880 --> 00:08:31.230
finite sequences.
00:08:31.230 --> 00:08:35.720
And if it's going to be causal,
that means that
00:08:35.720 --> 00:08:37.020
we can always --
00:08:37.020 --> 00:08:42.909
if we remove common factors
of D, that means that the
00:08:42.909 --> 00:08:45.590
numerator might still have
factors of D in it, but the
00:08:45.590 --> 00:08:48.610
denominator can't have
factors of D in it.
00:08:48.610 --> 00:08:50.390
You see that?
00:08:50.390 --> 00:08:54.360
Because, once we shift all these
over as close to 0 as we
00:08:54.360 --> 00:08:58.110
can, the denominator has
to be closer to 0.
00:08:58.110 --> 00:09:03.420
If D0 equals 1, that means 1
over d of D is something that
00:09:03.420 --> 00:09:05.890
starts at times 0.
00:09:05.890 --> 00:09:07.560
We multiply times
the numerator.
00:09:07.560 --> 00:09:10.240
In order to have the whole thing
be causal, the numerator
00:09:10.240 --> 00:09:13.890
has to be causal, because a
point I didn't mention, when
00:09:13.890 --> 00:09:16.750
you multiply formal Laurent
sequences, their delays add.
00:09:19.290 --> 00:09:23.400
If a of D and b of D are
Laurent, then the delay of a
00:09:23.400 --> 00:09:29.410
of D times b of D, defined as a
convolution, is going to be
00:09:29.410 --> 00:09:31.740
the sum of the delays of each
of the component sequence.
00:09:31.740 --> 00:09:34.250
You just take the lower order
term on both and that's what
00:09:34.250 --> 00:09:37.020
determines the lower order term
on the convolution, just
00:09:37.020 --> 00:09:40.780
like in polynomials and
analogous to degrees.
00:09:40.780 --> 00:09:46.350
So from this, we find that we
can always write a causal
00:09:46.350 --> 00:09:50.300
rational sequence in this
form, if it isn't 0.
00:09:50.300 --> 00:09:54.630
If it is 0, we just take n of D
to be 0, and d of D to be 1.
00:09:54.630 --> 00:09:56.930
And that satisfies
this form, too.
00:09:59.950 --> 00:10:03.670
I guess I didn't even have
to make that comment.
00:10:03.670 --> 00:10:04.920
We're already in that form.
00:10:07.650 --> 00:10:11.700
You can convince yourself this
is a unique way to write a
00:10:11.700 --> 00:10:15.090
causal rational sequence.
00:10:15.090 --> 00:10:18.460
And every other way is just
multiplying the numerator and
00:10:18.460 --> 00:10:28.030
the denominator by some common
polynomial or finite sequence.
00:10:28.030 --> 00:10:36.100
So now we had a little theorem
that says that, if g of D is
00:10:36.100 --> 00:10:45.190
causal rational, and therefore
equal to n of D over d of D in
00:10:45.190 --> 00:10:57.242
this form, then realizable with
how many memory elements?
00:10:57.242 --> 00:10:59.620
Does anyone remember?
00:10:59.620 --> 00:11:04.165
nu equals the maximum
of the degrees.
00:11:10.080 --> 00:11:14.130
And I debated whether to
actually do this in class, and
00:11:14.130 --> 00:11:17.960
I think it's worth taking a
few minutes to actually do
00:11:17.960 --> 00:11:18.740
this in class.
00:11:18.740 --> 00:11:23.050
It'll also appear as the
first homework problem.
00:11:23.050 --> 00:11:26.090
So this'll be a little head
start on the homework.
00:11:26.090 --> 00:11:31.290
So anybody have any ideas how
we can do this realization?
00:11:31.290 --> 00:11:36.230
What we want is a circuit with
an input, u of D. Eventually,
00:11:36.230 --> 00:11:43.220
an output, y of D equals u of
D times n of D over d of D.
00:11:43.220 --> 00:11:46.410
For these are both polynomials
and D0 equals 1.
00:11:51.630 --> 00:11:55.210
A lot of you have probably seen
this in discrete-time
00:11:55.210 --> 00:11:57.050
linear filters.
00:11:57.050 --> 00:11:58.720
It's going to be just
the same technique.
00:12:01.290 --> 00:12:03.380
But if we don't have volunteers,
I don't want to
00:12:03.380 --> 00:12:05.570
waste time.
00:12:05.570 --> 00:12:06.770
The key is to use --
00:12:06.770 --> 00:12:09.190
we're of course going to need
feedback in general.
00:12:09.190 --> 00:12:13.350
To have a d of D, in general,
will not be 1.
00:12:13.350 --> 00:12:17.600
If it's not 1, then we're
going to get an infinite
00:12:17.600 --> 00:12:18.650
impulse response.
00:12:18.650 --> 00:12:22.100
To get an infinite impulse
response, we need feedback.
00:12:22.100 --> 00:12:25.020
So we're going to need some
feedback term in here.
00:12:25.020 --> 00:12:28.720
And here is the motivating
idea of the construction.
00:12:28.720 --> 00:12:32.540
We want to create something
here, v of D, which is
00:12:32.540 --> 00:12:38.030
basically equal to u of D over
d of D. If we can do that,
00:12:38.030 --> 00:12:40.330
we'll be done.
00:12:40.330 --> 00:12:43.370
I'll show you why.
00:12:43.370 --> 00:12:45.740
So we created a different
sequence, which is u of D
00:12:45.740 --> 00:12:49.965
divided by d of D. And then we
pass that sequence through a
00:12:49.965 --> 00:12:51.215
shift register.
00:12:58.660 --> 00:13:00.950
And what do we get at
the various stages
00:13:00.950 --> 00:13:03.080
of the shift register?
00:13:03.080 --> 00:13:08.807
Here we would get v of D delayed
by one time unit, v of
00:13:08.807 --> 00:13:13.540
D delayed by two time units,
and so forth, up to --
00:13:13.540 --> 00:13:17.210
let's make this nu.
00:13:17.210 --> 00:13:22.600
I want to realize it in nu
memory elements, where nu is
00:13:22.600 --> 00:13:25.750
the maximum degree of either of
these defining polynomials.
00:13:25.750 --> 00:13:35.700
So finally, out here, I
get d nu of D, v of D.
00:13:35.700 --> 00:13:38.164
Now, what's the trick?
00:13:38.164 --> 00:13:40.060
AUDIENCE: [INAUDIBLE].
00:13:40.060 --> 00:13:41.560
PROFESSOR: Excuse me?
00:13:41.560 --> 00:13:42.810
AUDIENCE: Because
[UNINTELLIGIBLE PHRASE].
00:13:48.890 --> 00:13:49.850
PROFESSOR: That is the idea.
00:13:49.850 --> 00:13:54.420
What do we get if we take out
these various lines, which are
00:13:54.420 --> 00:13:58.530
v of D, through d to the nu v
of D, and we make a linear
00:13:58.530 --> 00:14:01.640
combination of them?
00:14:01.640 --> 00:14:04.280
We're going to do this twice,
actually, once to form the
00:14:04.280 --> 00:14:13.420
feedback, and once to
form the output.
00:14:13.420 --> 00:14:17.660
For the feedback, let me not
take this into the linear
00:14:17.660 --> 00:14:18.910
combination.
00:14:21.450 --> 00:14:25.960
Otherwise, I'd get a loop
without a delay element in
00:14:25.960 --> 00:14:27.430
that, and that tends not
to be well-defined.
00:14:30.060 --> 00:14:32.560
So I want to take a linear
combination.
00:14:32.560 --> 00:14:35.300
In general, by taking a linear
combination of these things, I
00:14:35.300 --> 00:14:42.860
can get the multiple of v of D
times any polynomial degree,
00:14:42.860 --> 00:14:49.260
nu or less, just by taking
the coefficients of that
00:14:49.260 --> 00:14:52.816
polynomial as my linear
combination.
00:14:52.816 --> 00:14:56.155
Do you see that?
00:14:56.155 --> 00:14:57.590
I'm not seeing people's --
00:14:57.590 --> 00:14:58.840
AUDIENCE: [INAUDIBLE].
00:15:01.270 --> 00:15:04.035
PROFESSOR: Excuse me?
00:15:04.035 --> 00:15:05.410
AUDIENCE: In the linear
combination
00:15:05.410 --> 00:15:06.730
[UNINTELLIGIBLE PHRASE]
00:15:06.730 --> 00:15:09.920
you also take the one with
zero derivative.
00:15:09.920 --> 00:15:16.290
PROFESSOR: You're getting ahead
of me, which is fine,
00:15:16.290 --> 00:15:20.180
but I'll ask for that comment
in just a second.
00:15:20.180 --> 00:15:22.640
Let me first create
the output.
00:15:22.640 --> 00:15:26.870
Suppose I had v of D equals u of
D over d of D. Then to get
00:15:26.870 --> 00:15:30.440
the output, I simply want a
linear combination, which will
00:15:30.440 --> 00:15:40.485
give me n of D times u of D over
d of D. And that would be
00:15:40.485 --> 00:15:41.420
the correct output.
00:15:41.420 --> 00:15:44.170
And since n of D is a polynomial
degree less than
00:15:44.170 --> 00:15:49.630
nu, I can do that by simply
taking n nu times this, n nu
00:15:49.630 --> 00:15:54.270
minus 1 times that, n2 times
that, n1 times that, and 0
00:15:54.270 --> 00:15:55.450
times that.
00:15:55.450 --> 00:15:59.940
And that will give me what I
want as an output, if I'm
00:15:59.940 --> 00:16:02.510
successful in getting v of
D of this form here.
00:16:05.950 --> 00:16:10.580
So let me try a similar
trick up here.
00:16:10.580 --> 00:16:16.760
The trick is, I force d of D to
start off with a 1, to have
00:16:16.760 --> 00:16:18.830
D0 equal 1.
00:16:18.830 --> 00:16:21.320
So what I'm going to create
up here is the linear
00:16:21.320 --> 00:16:29.680
combination, d of D minus
1 times v of D.
00:16:29.680 --> 00:16:34.050
First of all, let's verify
that I can do that.
00:16:34.050 --> 00:16:35.560
d of D minus 1 is what?
00:16:35.560 --> 00:16:36.900
It's a polynomial.
00:16:36.900 --> 00:16:42.755
Its degree is the same as the
degree of d of D. Now, if d of
00:16:42.755 --> 00:16:45.190
D is equal to 1, if the
denominator is just 1, this
00:16:45.190 --> 00:16:46.610
whole thing falls out.
00:16:46.610 --> 00:16:48.620
I don't need anything
coming in here.
00:16:48.620 --> 00:16:53.050
d of D is 1. v of D is equal
to u of D, so that's the
00:16:53.050 --> 00:16:55.850
polynomial case where I
don't need feedback.
00:16:55.850 --> 00:16:58.870
So let's assume d of D is not
equal 1, therefore it's some
00:16:58.870 --> 00:17:05.000
polynomial of degree nu, degree
of d of D, which is
00:17:05.000 --> 00:17:10.790
less than or equal to
nu, and so is this.
00:17:10.790 --> 00:17:15.740
But by subtracting out the 1,
I have no constant term.
00:17:15.740 --> 00:17:18.440
So it's going to be a multiple
of D. That's another way of
00:17:18.440 --> 00:17:20.180
saying that.
00:17:20.180 --> 00:17:22.599
It's going to have no constant
terms, so I only need to form
00:17:22.599 --> 00:17:25.230
a linear combination
of these terms.
00:17:25.230 --> 00:17:26.480
So I can do it.
00:17:29.486 --> 00:17:33.940
Now, if I do that, let's just
solve this equation.
00:17:33.940 --> 00:17:37.580
I create x of D. This is
supposed to be a plus.
00:17:37.580 --> 00:17:40.890
This is supposed to be a minus,
to work over any field.
00:17:40.890 --> 00:17:44.820
This trick works over real or
complex, or what you'd like.
00:17:44.820 --> 00:17:58.260
x of D is u of d minus d of D
minus 1, times v of d, times x
00:17:58.260 --> 00:18:01.830
of D. Maybe I should just
call this v of d.
00:18:01.830 --> 00:18:08.600
Now I'm going to solve
for v of D.
00:18:08.600 --> 00:18:13.390
So I actually have a v of D on
both sides of the equation.
00:18:13.390 --> 00:18:22.280
This results in u of D equal
v of D times d of D. And
00:18:22.280 --> 00:18:25.230
dividing both sides by d of D,
I see that I succeeded in
00:18:25.230 --> 00:18:26.810
getting what I wanted.
00:18:26.810 --> 00:18:32.240
So this is a typical trick of
realizing a rational impulse
00:18:32.240 --> 00:18:37.570
response by including a feedback
loop and negative
00:18:37.570 --> 00:18:39.360
feedback in your system.
00:18:42.950 --> 00:18:45.490
This is done in the notes.
00:18:45.490 --> 00:18:47.510
I'm surprised if you haven't
seen this before.
00:18:47.510 --> 00:18:49.610
Maybe it's that people in
communications don't take
00:18:49.610 --> 00:18:52.300
circuits courses, or they don't
take digital signal
00:18:52.300 --> 00:18:53.630
processing courses.
00:18:53.630 --> 00:18:55.906
Most people look puzzled.
00:18:55.906 --> 00:18:58.340
In any case, can you agree that,
formally, I've shown
00:18:58.340 --> 00:19:00.500
that we get what we want here?
00:19:00.500 --> 00:19:09.270
So I've given a way of
realizing, given a causal
00:19:09.270 --> 00:19:13.330
rational function, where nu is
defined is the maximum degree
00:19:13.330 --> 00:19:15.510
of these two polynomials.
00:19:15.510 --> 00:19:18.960
There's a realization with
nu memory elements.
00:19:18.960 --> 00:19:21.590
A little bit more work, we could
prove that this is the
00:19:21.590 --> 00:19:23.370
minimum possible number
of memory
00:19:23.370 --> 00:19:27.600
elements for this system.
00:19:27.600 --> 00:19:29.375
That gets into linear
system theory.
00:19:29.375 --> 00:19:30.625
AUDIENCE: [INAUDIBLE].
00:19:36.020 --> 00:19:40.120
PROFESSOR: I mean, what I want
is down here, I want the sum
00:19:40.120 --> 00:19:51.510
of n_i times d to the i, v of D.
So the linear coefficients
00:19:51.510 --> 00:19:52.760
are going to be these n_i.
00:19:58.150 --> 00:20:02.640
If I drew it out, I have
n0 here, n1 here.
00:20:02.640 --> 00:20:06.240
So it's scalar linear
combination, over F2.
00:20:13.216 --> 00:20:15.000
And what is that?
00:20:15.000 --> 00:20:16.820
v of D is common.
00:20:16.820 --> 00:20:21.720
That's just n of D times v
of D. Same trick up here.
00:20:24.660 --> 00:20:27.970
Is that what was puzzling
everybody?
00:20:27.970 --> 00:20:28.540
Think about it.
00:20:28.540 --> 00:20:31.920
What is a linear combination?
00:20:31.920 --> 00:20:33.170
Something that looks
like that.
00:20:38.880 --> 00:20:46.885
The second part, let's
now talk about a
00:20:46.885 --> 00:20:48.135
convolutional encoder.
00:20:55.350 --> 00:21:00.990
And we're just going to talk
about rate 1/n in this course.
00:21:00.990 --> 00:21:02.945
So this is a single input.
00:21:07.350 --> 00:21:13.485
n output linear time
invariant system.
00:21:16.560 --> 00:21:21.550
That's my definition, I guess,
of what I mean by a
00:21:21.550 --> 00:21:22.890
convolutional encoder.
00:21:22.890 --> 00:21:27.445
And when I say linear
time-invariant, I also want it
00:21:27.445 --> 00:21:31.760
to be a realizable, in the two
senses that its impulse
00:21:31.760 --> 00:21:33.380
response is causal rational.
00:21:35.960 --> 00:21:39.292
So [INAUDIBLE]
00:21:39.292 --> 00:21:40.610
down.
00:21:40.610 --> 00:21:55.550
Now it's characterized by
n-tuple of impulse responses.
00:22:01.790 --> 00:22:11.300
So it's going to be sum g of D
equal to g1 of D, up to gn of
00:22:11.300 --> 00:22:19.350
D, where clearly all these have
to be causal and rational
00:22:19.350 --> 00:22:21.010
in order for it to
be realizable.
00:22:21.010 --> 00:22:23.740
If any one of them was not
causal or not rational, then
00:22:23.740 --> 00:22:26.110
that individual impulse
response wouldn't be
00:22:26.110 --> 00:22:30.080
realizable and the whole thing
wouldn't be realizable.
00:22:30.080 --> 00:22:35.330
So where we have the gj of D
are all causal rational.
00:22:43.160 --> 00:22:48.760
And now, again, I'm going to put
this into a standard form.
00:22:48.760 --> 00:22:56.480
In general, this is going to
be n1 of D over d1 of D, so
00:22:56.480 --> 00:23:02.730
forth, up to n_n of D over d_n
of D. So I'm going to have
00:23:02.730 --> 00:23:08.070
different denominators for
each of the numerators.
00:23:08.070 --> 00:23:16.360
But I can always put it into the
form, n1 prime of D over a
00:23:16.360 --> 00:23:25.495
common numerator, d to D, and up
to n_n prime of D over d of
00:23:25.495 --> 00:23:29.730
D. Let d of D be the least
common multiple
00:23:29.730 --> 00:23:31.890
of all these d_i's.
00:23:31.890 --> 00:23:33.450
These are all polynomials.
00:23:33.450 --> 00:23:35.650
Least common multiple is
well-defined from our
00:23:35.650 --> 00:23:37.630
discussion of factorization.
00:23:37.630 --> 00:23:40.630
So I can always put in the least
common multiple here,
00:23:40.630 --> 00:23:44.410
and then whatever d1 lacks out
of the least common multiple,
00:23:44.410 --> 00:23:46.760
I multiply top and bottom
by both of that.
00:23:46.760 --> 00:23:50.300
I have the same rational
function, now with a common
00:23:50.300 --> 00:23:52.530
denominator.
00:23:52.530 --> 00:23:56.670
So I'm always going to put
it in that standard form.
00:23:56.670 --> 00:24:13.520
And then I will say this is
always realizable, with nu
00:24:13.520 --> 00:24:17.180
equal now the max of
any of the degrees
00:24:17.180 --> 00:24:18.310
that appears in here.
00:24:18.310 --> 00:24:24.870
So let me just abbreviate that
by degree of the numerators,
00:24:24.870 --> 00:24:28.370
which I'll just write as vector
n prime of D, or the
00:24:28.370 --> 00:24:31.505
degree of the denominator d, of
the common denominator d of
00:24:31.505 --> 00:24:36.540
D. Now, that's very
easy to see.
00:24:39.350 --> 00:24:40.600
Why is that?
00:24:43.300 --> 00:24:45.970
Can certainly realize
the first one here
00:24:45.970 --> 00:24:47.220
just by doing that.
00:24:50.300 --> 00:24:52.045
Yes?
00:24:52.045 --> 00:24:56.540
If I want to just generate the
first output, I build a
00:24:56.540 --> 00:24:58.985
circuit like this, and
I don't need more of
00:24:58.985 --> 00:25:01.212
the new memory elements.
00:25:01.212 --> 00:25:02.990
There might even be some
redundancy in that.
00:25:05.980 --> 00:25:09.095
All right, so how would I
then realize the second?
00:25:15.550 --> 00:25:16.400
Is it just me?
00:25:16.400 --> 00:25:17.680
Nobody is volunteering
anything.
00:25:23.400 --> 00:25:24.460
I've realized this.
00:25:24.460 --> 00:25:29.260
Now I'd like to realize
a second output.
00:25:29.260 --> 00:25:31.440
Now I want to realize this.
00:25:31.440 --> 00:25:36.760
Make this n1 of D. I want to
realize n2 of D over d of D,
00:25:36.760 --> 00:25:39.151
times u of D.
00:25:39.151 --> 00:25:40.401
AUDIENCE: [INAUDIBLE].
00:25:43.960 --> 00:25:44.842
PROFESSOR: Thank you so much.
00:25:44.842 --> 00:25:46.165
AUDIENCE: Just take out
[UNINTELLIGIBLE].
00:25:46.165 --> 00:25:47.240
PROFESSOR: All right.
00:25:47.240 --> 00:25:51.870
So all I need is the second
linear combination.
00:25:51.870 --> 00:25:57.090
So let me just make this
n linear combinations.
00:25:57.090 --> 00:26:00.640
Then we're going to
have n outputs,
00:26:00.640 --> 00:26:02.140
an n-tuple of outputs.
00:26:02.140 --> 00:26:04.440
Then I can form each
one as a linear
00:26:04.440 --> 00:26:05.990
combination of those up there.
00:26:11.150 --> 00:26:13.140
I really like more of you
to be speaking up.
00:26:13.140 --> 00:26:13.920
Yes, thank you.
00:26:13.920 --> 00:26:15.342
AUDIENCE: [INAUDIBLE].
00:26:15.342 --> 00:26:16.290
PROFESSOR: Excuse me?
00:26:16.290 --> 00:26:18.186
AUDIENCE: There's a second
combination --
00:26:18.186 --> 00:26:21.030
I mean the inquiry should
come [UNINTELLIGIBLE].
00:26:21.030 --> 00:26:24.680
PROFESSOR: Yeah, but again, what
I want to realize, I have
00:26:24.680 --> 00:26:27.520
all the delays of
v of D up here.
00:26:27.520 --> 00:26:32.750
What I want to realize is n2
of D times v of D, which is
00:26:32.750 --> 00:26:34.340
equal to that.
00:26:34.340 --> 00:26:37.200
n2 of D is the polynomial
of degree less
00:26:37.200 --> 00:26:38.640
than or equal to nu.
00:26:38.640 --> 00:26:39.420
So I can do it.
00:26:39.420 --> 00:26:42.760
AUDIENCE: Yeah, but
[UNINTELLIGIBLE PHRASE]
00:26:42.760 --> 00:26:44.630
parallel to that n1 D over d.
00:26:44.630 --> 00:26:45.110
The --
00:26:45.110 --> 00:26:46.070
[INTERPOSING VOICES]
00:26:46.070 --> 00:26:48.490
PROFESSOR: Yeah, OK.
00:26:48.490 --> 00:26:50.800
I'm confusing you because
I put all these here.
00:26:50.800 --> 00:26:54.530
I actually wrote out the
linear combination.
00:26:54.530 --> 00:26:58.160
To say what I just said, I
really need to go back to my
00:26:58.160 --> 00:27:01.570
original form, forget these
multipliers, do the
00:27:01.570 --> 00:27:02.660
multiplications in here.
00:27:02.660 --> 00:27:06.110
That's what I mean by the
linear combinations.
00:27:06.110 --> 00:27:07.990
Now I'm OK.
00:27:07.990 --> 00:27:10.080
The inputs are just
the shifts.
00:27:10.080 --> 00:27:12.289
Make a linear combination of
them, that's the output.
00:27:17.450 --> 00:27:20.600
So easy proof.
00:27:20.600 --> 00:27:24.090
What's significantly harder to
prove in this case is you
00:27:24.090 --> 00:27:25.410
can't do any better than.
00:27:25.410 --> 00:27:29.890
There aren't any realizations
with fewer than nu, where nu
00:27:29.890 --> 00:27:33.970
is computed by first reducing
to standard form and then
00:27:33.970 --> 00:27:35.600
evaluating this.
00:27:35.600 --> 00:27:38.550
That's the best you can do.
00:27:38.550 --> 00:27:42.390
But we aren't going to go into
minimal system realizations,
00:27:42.390 --> 00:27:44.330
linear system realizations
in this course.
00:27:44.330 --> 00:27:45.667
I'll just insert it.
00:27:48.190 --> 00:27:53.200
So this is how we can always
build a convolutional encoder,
00:27:53.200 --> 00:27:55.780
whether it has feedback
or not.
00:27:55.780 --> 00:28:00.360
And so whatever we come up with
as nu, this'll basically
00:28:00.360 --> 00:28:03.400
determine the state complexity
of our decoder.
00:28:03.400 --> 00:28:08.670
The state space is dimension
nu, is finite dimension.
00:28:08.670 --> 00:28:13.490
And because we're over a finite
field, the actual
00:28:13.490 --> 00:28:16.920
number of states is
only 2 to the nu.
00:28:16.920 --> 00:28:20.085
Still can't get more than
2 to the nu states for
00:28:20.085 --> 00:28:22.310
that system up there.
00:28:22.310 --> 00:28:25.120
So we're still in finite
state world.
00:28:25.120 --> 00:28:30.940
So in particular, it's finite
state realization.
00:28:30.940 --> 00:28:33.860
And again, this is
if and only if.
00:28:33.860 --> 00:28:36.120
If we have if and only if, we
have an n-tuple of causal
00:28:36.120 --> 00:28:40.790
rational, or even a matrix
of causal rationals.
00:28:40.790 --> 00:28:43.030
We can make a realization
like this.
00:28:46.470 --> 00:28:48.575
So that's convolutional
encoders.
00:28:48.575 --> 00:28:52.630
At least write 1/n over F2.
00:28:52.630 --> 00:28:57.010
And now the next step was what's
a convolutional code.
00:29:02.510 --> 00:29:08.330
And a convolutional code,
we defined as, given a
00:29:08.330 --> 00:29:13.630
convolutional encoder, which
is just the set of impulse
00:29:13.630 --> 00:29:19.330
response, given g of D, the
corresponding convolutional
00:29:19.330 --> 00:29:25.180
code is just u of D, which is
single sequence times this
00:29:25.180 --> 00:29:30.410
n-tuple of sequences, as u of
D ranges through all the
00:29:30.410 --> 00:29:32.093
formal Laurent sequences.
00:29:37.550 --> 00:29:41.090
Sequences that start at some
time in the all-zero state and
00:29:41.090 --> 00:29:45.060
then continue perhaps forever.
00:29:45.060 --> 00:29:50.660
It's very quick to
show that C, its
00:29:50.660 --> 00:29:53.200
properties is it's linear.
00:29:53.200 --> 00:29:57.440
Obviously, it's a vector
space over F2.
00:29:57.440 --> 00:29:59.990
Multiplication by scalars
is trivial.
00:29:59.990 --> 00:30:04.300
Addition is trivial,
so it's linear.
00:30:04.300 --> 00:30:12.700
And its time-invariant, meaning
the shift of any code
00:30:12.700 --> 00:30:14.250
sequence is another
code sequence.
00:30:14.250 --> 00:30:22.040
If I have one particular code
sequence, and I want to see if
00:30:22.040 --> 00:30:25.220
the shift of that is in the
code, well, that code sequence
00:30:25.220 --> 00:30:28.610
must have been generated by some
use, so if I just shift
00:30:28.610 --> 00:30:32.370
the u by the amount of time
that I want to shift the
00:30:32.370 --> 00:30:36.800
output, y, then I'll get
the shifted output.
00:30:36.800 --> 00:30:43.980
So it's easy to show that D to
the k of C is simply equal to
00:30:43.980 --> 00:30:51.510
C for any integer k.
00:30:51.510 --> 00:30:55.080
Actually, it just suffices to
show that the single time unit
00:30:55.080 --> 00:30:57.717
shift is in the code, and that
implies all the rest of this.
00:31:01.390 --> 00:31:01.890
Yes?
00:31:01.890 --> 00:31:03.140
AUDIENCE: [UNINTELLIGIBLE].
00:31:05.740 --> 00:31:10.276
We would want at least one of
the g_i of D's to start at 0.
00:31:10.276 --> 00:31:13.380
Otherwise, they don't
cancel out, do they?
00:31:13.380 --> 00:31:14.630
PROFESSOR: Right.
00:31:17.530 --> 00:31:21.340
So what you're saying is we
don't want d to be a common
00:31:21.340 --> 00:31:24.890
factor of all the n of D's.
00:31:24.890 --> 00:31:25.680
And that's true.
00:31:25.680 --> 00:31:28.780
In fact, we don't want to have
any common factors of
00:31:28.780 --> 00:31:30.030
all the n of D's.
00:31:33.270 --> 00:31:34.520
AUDIENCE: [INAUDIBLE].
00:31:37.540 --> 00:31:40.180
PROFESSOR: The u we've
defined, so it can
00:31:40.180 --> 00:31:41.290
start at any time.
00:31:41.290 --> 00:31:43.790
AUDIENCE: [INAUDIBLE].
00:31:43.790 --> 00:31:47.190
PROFESSOR: So notice that these
have separate algebraic
00:31:47.190 --> 00:31:48.660
characters.
00:31:48.660 --> 00:31:51.800
This is a Laurent sequence.
00:31:51.800 --> 00:31:54.660
These are called causal
rational sequences.
00:31:54.660 --> 00:31:58.040
So they have different
restrictions on them, but they
00:31:58.040 --> 00:32:00.240
play together that way.
00:32:04.940 --> 00:32:09.680
So where you're getting to is
this idea of code equivalence.
00:32:12.850 --> 00:32:14.210
I can just keep going.
00:32:17.660 --> 00:32:33.600
So code equivalence, let's
abbreviate it this way.
00:32:33.600 --> 00:32:40.520
g of D and some other n-tuple,
g prime of D,
00:32:40.520 --> 00:32:42.040
generate the same code.
00:32:42.040 --> 00:32:45.865
We say that g of D generates
C up here.
00:32:49.480 --> 00:32:57.790
So two different n-tuples
generate the same code, C,
00:32:57.790 --> 00:32:59.405
which is our notion
of equivalence.
00:33:03.690 --> 00:33:10.180
And we more or less proved last
time it is true that g of
00:33:10.180 --> 00:33:14.120
D is some --
00:33:14.120 --> 00:33:22.270
this is just a single sequence
multiple of g prime of D. So
00:33:22.270 --> 00:33:24.780
we have to have a of
D not equal to 0.
00:33:24.780 --> 00:33:29.810
Otherwise, I think there could
be any sequence there.
00:33:29.810 --> 00:33:34.830
So what you were saying is that,
if all of these n's had
00:33:34.830 --> 00:33:39.205
a common factor of d, then why
not just shift them all over?
00:33:39.205 --> 00:33:41.800
And that would be the same
as multiplying --
00:33:41.800 --> 00:33:44.060
suppose we had a g prime of
D where they all had d's.
00:33:44.060 --> 00:33:46.800
Suppose we just multiply
them with d minus 1.
00:33:46.800 --> 00:33:49.370
We're still going to get the
same code, but we're going to
00:33:49.370 --> 00:33:52.580
reduce the degrees of
all the numerators.
00:33:52.580 --> 00:33:56.970
And therefore, we're likely to
get a simpler realization.
00:33:56.970 --> 00:33:59.290
And that's the track we're
going down right now.
00:33:59.290 --> 00:34:00.540
AUDIENCE: [INAUDIBLE].
00:34:05.090 --> 00:34:07.250
PROFESSOR: This would
only shift the n's.
00:34:10.650 --> 00:34:15.570
I'm assuming that all the n
of D's are multiples of d.
00:34:15.570 --> 00:34:17.590
AUDIENCE: [INAUDIBLE]
00:34:17.590 --> 00:34:18.670
the root of d, right?
00:34:18.670 --> 00:34:23.900
So if you can reduce the d of
D you can't save anything.
00:34:23.900 --> 00:34:26.370
PROFESSOR: You might not save
anything if you've got a
00:34:26.370 --> 00:34:29.245
denominator, d of D, that
has a larger degree.
00:34:29.245 --> 00:34:31.250
AUDIENCE: And that's
always the case.
00:34:31.250 --> 00:34:33.739
PROFESSOR: No, it's not
always the case.
00:34:33.739 --> 00:34:35.250
In the general case,
either one of
00:34:35.250 --> 00:34:39.530
these things can dominate.
00:34:39.530 --> 00:34:44.469
In that picture we had, if we
have a low degree, d of D,
00:34:44.469 --> 00:34:47.510
then it's only picking off of
these first elements up here.
00:34:47.510 --> 00:34:50.532
AUDIENCE: But that definition
states [UNINTELLIGIBLE]
00:34:50.532 --> 00:34:51.696
is the maximum d and also
the [UNINTELLIGIBLE].
00:34:51.696 --> 00:34:52.610
PROFESSOR: Right.
00:34:52.610 --> 00:34:56.870
So we could have a big one here
making the outputs, and a
00:34:56.870 --> 00:34:58.850
small one there going back.
00:34:58.850 --> 00:35:01.780
In fact, we could have
d of D equal to 1.
00:35:01.780 --> 00:35:04.650
Then we'd have no feedback
whatsoever.
00:35:04.650 --> 00:35:08.400
So it could be either way.
00:35:08.400 --> 00:35:12.250
But then given this code
equivalence concept, suppose
00:35:12.250 --> 00:35:13.950
we have a d of D that
is very big.
00:35:13.950 --> 00:35:17.800
Suppose it's dominated by
d of D. What would be a
00:35:17.800 --> 00:35:19.050
good thing to do?
00:35:24.610 --> 00:35:28.660
We have a certain code we want
to keep, but the nu, the
00:35:28.660 --> 00:35:31.540
numbers for the dimension of the
state space is dominated
00:35:31.540 --> 00:35:40.860
by the degree of d of D. So
suppose we have g of D equal
00:35:40.860 --> 00:35:50.660
to n of D over d of D. And we
have degree of d of D is
00:35:50.660 --> 00:35:55.750
greater than degree of n
of D. What would be a
00:35:55.750 --> 00:35:57.000
good thing to do?
00:36:02.120 --> 00:36:09.720
Why don't we just let g prime of
D be equal to d of D times
00:36:09.720 --> 00:36:19.950
g of D, and that would
be just n of D.
00:36:19.950 --> 00:36:27.240
So I'm going to convert from
the code generated by these
00:36:27.240 --> 00:36:31.060
rational generators,
which are infinite.
00:36:31.060 --> 00:36:34.880
I can easily just multiply out
the denominator, and now I
00:36:34.880 --> 00:36:39.900
have a code generated just
by the numerator terms.
00:36:39.900 --> 00:36:41.150
So it's feedback-free.
00:36:44.180 --> 00:36:46.950
That may or may not be
important to me.
00:36:46.950 --> 00:36:52.410
Actually, it turns out it's very
hard to make a case for
00:36:52.410 --> 00:36:56.420
not having feedback, but it's
simpler, in any case.
00:36:56.420 --> 00:36:59.810
We don't have this feedback
term in the encoders.
00:36:59.810 --> 00:37:02.430
So we get a feedback-free
encoder.
00:37:02.430 --> 00:37:06.525
And if this is true,
we may reduce --
00:37:09.430 --> 00:37:11.130
we can certainly never
increase it.
00:37:13.890 --> 00:37:17.950
But if this were true, then
we would reduce it.
00:37:17.950 --> 00:37:20.280
If on the other hand, this were
less than or equal, then
00:37:20.280 --> 00:37:22.510
nu would remain the same.
00:37:22.510 --> 00:37:24.370
So that seems like that would
be a good thing to do.
00:37:28.540 --> 00:37:34.910
So in fact, as I would advocate
as a first step, that
00:37:34.910 --> 00:37:37.960
you clear the denominators,
and just come up with a
00:37:37.960 --> 00:37:43.410
polynomial generator sequence
which generates the same code.
00:37:43.410 --> 00:37:45.330
We're still going to have the
same minimum distance, the
00:37:45.330 --> 00:37:48.300
same code sequence, the problem
from the decoder's
00:37:48.300 --> 00:37:50.035
point of view is going to
be absolutely unchanged.
00:37:50.035 --> 00:37:52.850
The decoder only cares what
the consequences are.
00:37:52.850 --> 00:37:54.830
It wants to find the most
likely one that was
00:37:54.830 --> 00:37:55.710
transmitted.
00:37:55.710 --> 00:37:58.970
So let's not make the encoder
any more complicated
00:37:58.970 --> 00:38:01.596
than we have to.
00:38:01.596 --> 00:38:02.846
AUDIENCE:
[UNINTELLIGIBLE PHRASE]
00:38:13.060 --> 00:38:16.590
PROFESSOR: nu doesn't change
in this condition.
00:38:16.590 --> 00:38:18.805
nu does change in
this condition.
00:38:18.805 --> 00:38:22.392
AUDIENCE: So we are always
better off multiplying by g of
00:38:22.392 --> 00:38:22.490
D.
00:38:22.490 --> 00:38:23.300
PROFESSOR: We can't
be worse off.
00:38:23.300 --> 00:38:23.610
[INTERPOSING VOICES]
00:38:23.610 --> 00:38:25.040
AUDIENCE: --or the same.
00:38:25.040 --> 00:38:27.530
PROFESSOR: Exactly.
00:38:27.530 --> 00:38:29.810
Got it.
00:38:29.810 --> 00:38:34.630
All right, so let's now talk
about whether there's anything
00:38:34.630 --> 00:38:39.300
we can do to n of D. We already
suggested that if all
00:38:39.300 --> 00:38:42.610
the n of D had a factor of
d in them, why don't
00:38:42.610 --> 00:38:43.860
just take it out?
00:38:47.870 --> 00:38:49.150
Let's just write that down.
00:38:49.150 --> 00:39:03.510
If all n of D have a factor of
d, then let's just transform n
00:39:03.510 --> 00:39:09.680
of D to d minus 1 n of D.
00:39:09.680 --> 00:39:12.455
So that reduces nu also.
00:39:15.360 --> 00:39:19.130
And let's, of course, do that
as many times as we can.
00:39:19.130 --> 00:39:23.060
So it's a good idea to take
out common factors of d.
00:39:23.060 --> 00:39:24.490
We get a simpler encoder.
00:39:24.490 --> 00:39:26.125
We're have a simpler
state diagram.
00:39:30.010 --> 00:39:38.710
This leads us to a topic called
catastrophicity, which
00:39:38.710 --> 00:39:41.990
is a misleading title because
you have no idea where I'm
00:39:41.990 --> 00:39:43.540
going right now.
00:39:43.540 --> 00:39:45.340
So I'm actually going
to try to fool you
00:39:45.340 --> 00:39:47.140
for a little while.
00:39:47.140 --> 00:39:47.500
Yes?
00:39:47.500 --> 00:39:48.726
AUDIENCE: Quick question.
00:39:48.726 --> 00:39:51.320
In the definition of code
equivalence, is there a
00:39:51.320 --> 00:39:53.340
restriction [UNINTELLIGIBLE]
00:39:53.340 --> 00:39:57.460
that shouldn't be rational,
shouldn't be causal?
00:39:57.460 --> 00:40:01.470
PROFESSOR: Should there be
a restriction on this?
00:40:01.470 --> 00:40:02.480
Yes, of course.
00:40:02.480 --> 00:40:05.940
It has to be rational
in order --
00:40:08.470 --> 00:40:10.570
But that's a consequence.
00:40:10.570 --> 00:40:13.930
If these are both causal
rational, then what does a of
00:40:13.930 --> 00:40:14.620
D have to be?
00:40:14.620 --> 00:40:16.770
It has to be rational.
00:40:16.770 --> 00:40:19.320
It has to be non-zero.
00:40:19.320 --> 00:40:22.300
And does it have to be causal?
00:40:22.300 --> 00:40:25.620
No, in this case it's
not causal.
00:40:25.620 --> 00:40:26.870
AUDIENCE: [INAUDIBLE].
00:40:31.310 --> 00:40:33.460
PROFESSOR: That's right.
00:40:33.460 --> 00:40:37.960
If there's a delay buried in g
of D, if everything starts at
00:40:37.960 --> 00:40:41.870
time 1 or later, then a of D,
in fact, can be non-causal.
00:40:41.870 --> 00:40:45.190
So the key restrictions are that
these two guys have to be
00:40:45.190 --> 00:40:46.360
causal rational.
00:40:46.360 --> 00:40:52.450
And that will tell you
restrictions on a.
00:40:52.450 --> 00:40:54.240
Let me introduce this
by example.
00:40:54.240 --> 00:40:56.860
And I'll warn you that
I'm going to try to
00:40:56.860 --> 00:40:58.940
fool you for a while.
00:40:58.940 --> 00:41:02.150
Let's just take a rate
1/1 encoder.
00:41:02.150 --> 00:41:09.060
This may seem a little peculiar
to you, because it
00:41:09.060 --> 00:41:13.910
has no redundancy, one
input, one output.
00:41:13.910 --> 00:41:18.300
And I'm going to let g of
D just be 1 plus d.
00:41:22.550 --> 00:41:23.380
So what do I mean?
00:41:23.380 --> 00:41:25.780
I'm going to try to get some
mileage out of this encoder.
00:41:29.146 --> 00:41:43.710
Here's u of D. Here's y of D,
equals 1 plus d times u of D.
00:41:43.710 --> 00:41:48.295
Now let's draw the state diagram
for this encoder.
00:41:48.295 --> 00:41:50.730
It's only got two states.
00:41:50.730 --> 00:41:51.580
One is 0.
00:41:51.580 --> 00:41:56.420
If we get a 0 in, we of
course put a 0 out.
00:41:56.420 --> 00:41:57.740
We get 0, 0.
00:41:57.740 --> 00:42:00.105
Or we could get a 1 in.
00:42:00.105 --> 00:42:03.390
In which case, we would
get a 1 out.
00:42:03.390 --> 00:42:06.230
Sorry, that was just
a single output.
00:42:06.230 --> 00:42:08.346
And go to 1.
00:42:08.346 --> 00:42:17.000
If we're in the 1 state,
and we get a 0 in,
00:42:17.000 --> 00:42:18.790
then we get a 1 out.
00:42:18.790 --> 00:42:22.990
Whereas if we get a 1 in, then
they're both 0, and we get the
00:42:22.990 --> 00:42:24.350
complement.
00:42:24.350 --> 00:42:27.140
So with the 1, we get a 0 out.
00:42:27.140 --> 00:42:30.180
So that's the state transition
diagram of this encoder.
00:42:36.500 --> 00:42:39.040
Now, let's go through a similar
kind of argument to
00:42:39.040 --> 00:42:41.470
what we went through to find
the minimum distance was 5.
00:42:41.470 --> 00:42:45.170
This is a linear system, so we
want to find the minimum
00:42:45.170 --> 00:42:48.350
non-zero weight of
any code word.
00:42:48.350 --> 00:42:51.720
And here's my argument.
00:42:51.720 --> 00:42:54.050
There's a gold star to the first
person who punches a
00:42:54.050 --> 00:42:56.380
hole in it.
00:42:56.380 --> 00:43:01.560
Any code word, you have to
start in the 0 state, so
00:43:01.560 --> 00:43:03.700
you're always going to get at
least one unit of distance
00:43:03.700 --> 00:43:07.120
when you go to the 1 state.
00:43:07.120 --> 00:43:10.550
You can stay out here as long
as you like, but eventually
00:43:10.550 --> 00:43:14.550
you have to come back
to the 0 state.
00:43:14.550 --> 00:43:16.430
And at that point, you're
going to get
00:43:16.430 --> 00:43:18.600
another unit of distance.
00:43:18.600 --> 00:43:25.360
So I claim the minimum
non-zero weight is 2.
00:43:33.328 --> 00:43:34.578
Is that correct?
00:43:39.870 --> 00:43:41.870
Let me try to create
some doubt.
00:43:41.870 --> 00:43:44.420
What is the code here?
00:43:44.420 --> 00:43:51.620
The code is the set of all
multiples of 1 plus d times u
00:43:51.620 --> 00:43:58.335
of D, where u of D is a formal
Laurent sequence.
00:44:01.240 --> 00:44:04.420
What is that?
00:44:04.420 --> 00:44:07.490
I can write this briefly as
just the set of all formal
00:44:07.490 --> 00:44:10.330
Laurent sequences
times 1 plus d.
00:44:10.330 --> 00:44:10.514
AUDIENCE: [INAUDIBLE]
00:44:10.514 --> 00:44:19.160
1 minus d minus d squared.
00:44:19.160 --> 00:44:20.410
PROFESSOR: Good.
00:44:22.300 --> 00:44:24.260
That's correct.
00:44:24.260 --> 00:44:26.952
That's what's wrong
with my claim.
00:44:26.952 --> 00:44:31.100
Let me detour around here.
00:44:31.100 --> 00:44:33.450
Let me get an answer
to this question.
00:44:33.450 --> 00:44:35.170
What is the code in this case?
00:44:35.170 --> 00:44:39.180
It's the set of all possible
sequences you can get by
00:44:39.180 --> 00:44:43.010
multiplying 1 plus d times the
formal Laurent sequence.
00:44:43.010 --> 00:44:43.685
What is that?
00:44:43.685 --> 00:44:44.904
AUDIENCE: [INAUDIBLE].
00:44:44.904 --> 00:44:45.351
PROFESSOR: What?
00:44:45.351 --> 00:44:46.480
AUDIENCE: Isn't it
[UNINTELLIGIBLE]?
00:44:46.480 --> 00:44:47.840
PROFESSOR: It's just
the set of all
00:44:47.840 --> 00:44:49.090
formal Laurent sequences.
00:44:51.430 --> 00:44:57.190
Certainly for every formal
Laurent sequence, I can get
00:44:57.190 --> 00:44:59.420
such a sequence this way.
00:44:59.420 --> 00:45:05.300
In particular, the
code includes 1.
00:45:05.300 --> 00:45:10.730
What do I need as an input to
get the output sequence 1?
00:45:10.730 --> 00:45:12.530
AUDIENCE: String of 1's.
00:45:12.530 --> 00:45:15.710
PROFESSOR: String
of 1's, right.
00:45:15.710 --> 00:45:24.760
So an example, if u of D is
equal to 1/1 plus d, which is
00:45:24.760 --> 00:45:37.910
a string of 1's, then y of D is
equal to 1 plus d times u
00:45:37.910 --> 00:45:42.010
of D, which is 1.
00:45:42.010 --> 00:45:44.190
1 plus d has an inverse,
and this is it.
00:45:47.840 --> 00:45:54.170
So now we're coming back to
Mr. Aggarwal's point.
00:45:54.170 --> 00:45:59.180
Suppose we put in u of D equals
this sequence, all 0's
00:45:59.180 --> 00:46:03.510
before times 0, all
1's after time 0.
00:46:03.510 --> 00:46:05.530
Then where will we go?
00:46:05.530 --> 00:46:08.200
We'll go 1, and then
we'll just stay
00:46:08.200 --> 00:46:11.950
in this 0 loop forever.
00:46:11.950 --> 00:46:15.090
Anything wrong with that?
00:46:15.090 --> 00:46:17.490
Is that a code word?
00:46:17.490 --> 00:46:21.190
That is a code word, the way
we've defined the code.
00:46:21.190 --> 00:46:24.310
If we had defined the code so
that the u of D were only
00:46:24.310 --> 00:46:30.220
finite input sequences, then
this claim would be correct.
00:46:30.220 --> 00:46:32.640
For any finite input sequence,
we eventually have to come
00:46:32.640 --> 00:46:33.930
back to the 0 state.
00:46:36.620 --> 00:46:48.210
So this is true for finite
inputs, not true
00:46:48.210 --> 00:46:49.460
for infinite inputs.
00:46:57.315 --> 00:46:59.175
So this is wrong.
00:46:59.175 --> 00:47:06.540
The fact is d3 equals 1.
00:47:06.540 --> 00:47:08.790
The minimum weight of
this code is 1.
00:47:11.690 --> 00:47:18.220
But you see this is, first
of all, it's a confusing
00:47:18.220 --> 00:47:22.460
situation for analytical
purposes.
00:47:22.460 --> 00:47:26.620
Let me further indicate why
it might be bad to have an
00:47:26.620 --> 00:47:30.060
encoder like this in general.
00:47:30.060 --> 00:47:44.270
The key catastrophicity, in
general, will be defined as
00:47:44.270 --> 00:47:57.540
there are finite outputs
generated by
00:47:57.540 --> 00:48:01.390
infinite inputs, as above.
00:48:07.990 --> 00:48:18.190
Or equivalently, there
are 0 loops in the
00:48:18.190 --> 00:48:19.440
state transition diagram.
00:48:25.620 --> 00:48:28.250
So by choosing a proper input,
you can just keep driving
00:48:28.250 --> 00:48:31.088
around a 0 loop forever.
00:48:31.088 --> 00:48:33.956
AUDIENCE: [INAUDIBLE].
00:48:33.956 --> 00:48:35.040
PROFESSOR: Excuse me?
00:48:35.040 --> 00:48:36.290
AUDIENCE: [INAUDIBLE]
00:48:38.315 --> 00:48:43.059
the output is becoming finite,
so it's actually --
00:48:43.059 --> 00:48:46.540
I'm just commenting on the
whole catastrophicity.
00:48:46.540 --> 00:48:48.280
PROFESSOR: OK, well, I want
to next tell you where
00:48:48.280 --> 00:48:49.720
catastrophicity comes from.
00:48:49.720 --> 00:48:54.450
The opposite of this property
is non-catastrophicity.
00:48:54.450 --> 00:48:57.752
What we want is non-catastrophic
encoders.
00:48:57.752 --> 00:49:03.880
We want encoders where, to get
a finite output, you get a
00:49:03.880 --> 00:49:07.340
finite output only from
finite inputs.
00:49:13.330 --> 00:49:16.350
That's better analytically and
it's better in practice.
00:49:16.350 --> 00:49:18.580
Here, let me draw you the
practical argument.
00:49:18.580 --> 00:49:23.490
We have, for this particular
case, u of D comes in, gets
00:49:23.490 --> 00:49:26.322
multiplied by 1 plus d.
00:49:26.322 --> 00:49:30.470
That gives us y of D, which --
00:49:30.470 --> 00:49:31.940
let me just abbreviate it.
00:49:31.940 --> 00:49:34.780
We transmit that
over a channel,
00:49:34.780 --> 00:49:38.470
and we get our decoder.
00:49:38.470 --> 00:49:42.730
Actually, decoder doesn't have
much to do here, because all
00:49:42.730 --> 00:49:46.240
possible sequences are allowed
at the output.
00:49:46.240 --> 00:49:49.980
So if this were a binary
symmetric channel, the best
00:49:49.980 --> 00:49:52.590
the decoder could do would just
be to put out whatever it
00:49:52.590 --> 00:49:56.930
sees as its guess, y-hat
of D. So this
00:49:56.930 --> 00:49:58.180
is the decoder estimate.
00:50:01.030 --> 00:50:07.140
So in any case, this
can differ by
00:50:07.140 --> 00:50:08.740
code words from that.
00:50:08.740 --> 00:50:12.070
All possible code words
are possible.
00:50:12.070 --> 00:50:19.330
And given y-hat of D, to get
back to u of D, what do we
00:50:19.330 --> 00:50:19.830
have to do?
00:50:19.830 --> 00:50:23.690
We have to divide by 1/1
plus d, in effect,
00:50:23.690 --> 00:50:25.260
to invert this equation.
00:50:25.260 --> 00:50:29.710
For y of D is equal to 1 plus d
times u of D, then the u of
00:50:29.710 --> 00:50:32.330
D that corresponds to a
particular decoded code
00:50:32.330 --> 00:50:36.630
sequence, y-hat of D, we get
by dividing by this.
00:50:36.630 --> 00:50:38.060
But this is infinite sequence.
00:50:38.060 --> 00:50:40.350
Suppose this makes
just one error.
00:50:40.350 --> 00:50:44.420
It's possible for this to
transmit all 0's, and for this
00:50:44.420 --> 00:50:45.670
to make just one error.
00:50:48.270 --> 00:50:49.600
That's a code word.
00:50:49.600 --> 00:50:52.770
That's a legitimate
code sequence.
00:50:52.770 --> 00:50:56.690
So it's something the decoder
might come up with.
00:50:56.690 --> 00:51:01.500
If it does that, then what's
this going to look like?
00:51:01.500 --> 00:51:06.440
This is going to look like an
infinite number of 1's,
00:51:06.440 --> 00:51:10.110
because this is the infinite
input that causes this code
00:51:10.110 --> 00:51:14.010
sequence with a finite
number of outputs.
00:51:14.010 --> 00:51:17.750
So it's precisely this
condition, whenever we make an
00:51:17.750 --> 00:51:22.250
error that corresponds to a
finite output sequence, and we
00:51:22.250 --> 00:51:26.110
translate it back to the encoder
input, that's an
00:51:26.110 --> 00:51:27.630
infinite encoder input.
00:51:27.630 --> 00:51:29.600
There's a one-to-one
correspondence between inputs
00:51:29.600 --> 00:51:30.420
and outputs.
00:51:30.420 --> 00:51:33.350
So it's got to happen whenever
we make that type of error.
00:51:33.350 --> 00:51:35.100
And this is called catastrophic
error
00:51:35.100 --> 00:51:36.320
propagation.
00:51:36.320 --> 00:51:37.670
The decoder hasn't done
anything bad.
00:51:37.670 --> 00:51:43.710
It made one error, as it's going
to do once in a while.
00:51:43.710 --> 00:51:47.200
But the consequences
are catastrophic.
00:51:47.200 --> 00:51:50.250
We make an infinite number of
errors in the bits that we
00:51:50.250 --> 00:51:53.408
actually deliver to the user.
00:51:53.408 --> 00:51:55.540
Now, of course, what's really
going to happen is that
00:51:55.540 --> 00:51:58.530
sometime later the decoder
is going to make
00:51:58.530 --> 00:52:00.740
another little error.
00:52:00.740 --> 00:52:04.330
And at that point, this will
switch state again.
00:52:04.330 --> 00:52:07.750
So that will terminate the error
burst when the decoder
00:52:07.750 --> 00:52:08.740
makes another error.
00:52:08.740 --> 00:52:10.130
But this is completely random.
00:52:10.130 --> 00:52:12.330
It may take a long time
for this to happen,
00:52:12.330 --> 00:52:14.450
and it's not good.
00:52:14.450 --> 00:52:16.835
So for practical reasons, we
don't want catastrophicity.
00:52:24.720 --> 00:52:27.990
I was going to give you another
example, but in the
00:52:27.990 --> 00:52:32.190
interest of time, I'll
just write it down.
00:52:32.190 --> 00:52:36.360
Here's another catastrophic
example.
00:52:36.360 --> 00:52:42.260
Let me just do a rate 1/2
encoder so it looks more
00:52:42.260 --> 00:52:44.720
reasonable.
00:52:44.720 --> 00:52:47.250
And we'll make it look
very much like the
00:52:47.250 --> 00:52:50.230
encoders we've had.
00:52:50.230 --> 00:52:55.170
Our first example looks
something like this.
00:52:55.170 --> 00:52:56.570
1 plus d squared.
00:52:56.570 --> 00:52:58.340
I'll say 1 plus d.
00:52:58.340 --> 00:52:59.730
So it's a feet-forward
encoder.
00:52:59.730 --> 00:53:01.755
It looks perfectly reasonable.
00:53:07.340 --> 00:53:10.290
The top channel is impulse
response, 1 plus d squared.
00:53:10.290 --> 00:53:14.030
The second output has impulse
response, 1 plus d.
00:53:14.030 --> 00:53:16.400
If you look at the minimum
weight for all finite
00:53:16.400 --> 00:53:18.890
sequences, it's 4, not quite
as good as the other
00:53:18.890 --> 00:53:21.550
one which had 5.
00:53:21.550 --> 00:53:23.005
But is this catastrophic?
00:53:26.430 --> 00:53:27.020
Yes?
00:53:27.020 --> 00:53:28.020
Who said yes?
00:53:28.020 --> 00:53:29.270
AUDIENCE: [INAUDIBLE].
00:53:38.390 --> 00:53:39.480
PROFESSOR: Great.
00:53:39.480 --> 00:53:46.480
So the same observation, if u
of D is equal to 1/1 plus d,
00:53:46.480 --> 00:53:53.280
which is, again, this infinite
sequence, then the output, y
00:53:53.280 --> 00:54:01.040
of D equals u of D, g
of D, is equal to --
00:54:01.040 --> 00:54:05.410
1 plus d divides both of these,
so we get 1 plus d1.
00:54:05.410 --> 00:54:06.915
We get a finite output
sequence.
00:54:11.260 --> 00:54:14.880
You saw that immediately,
because we know that
00:54:14.880 --> 00:54:18.180
polynomials are divisible by 1
plus d if and only if they
00:54:18.180 --> 00:54:21.105
have an even number
of non-zero terms.
00:54:24.960 --> 00:54:27.750
So that's a more elaborate
example of where we can get
00:54:27.750 --> 00:54:28.970
catastrophic behavior.
00:54:28.970 --> 00:54:32.400
So what should we do here?
00:54:32.400 --> 00:54:35.080
Clearly, we should divide out
the common factor from here.
00:54:35.080 --> 00:54:36.480
We can do that.
00:54:36.480 --> 00:54:41.790
If all of these polynomials
have a common factor, we
00:54:41.790 --> 00:54:43.930
should divide it out.
00:54:43.930 --> 00:54:49.360
And we know that the code
generated by -- if this is g
00:54:49.360 --> 00:54:54.940
of D, we're going to say g prime
of D is equal to 1/1
00:54:54.940 --> 00:55:00.490
plus d times 1 plus d
squared, 1 plus d.
00:55:00.490 --> 00:55:02.700
And that is -- we've already
calculated --
00:55:02.700 --> 00:55:06.350
1 plus d1.
00:55:06.350 --> 00:55:10.170
So by doing that, first of
all, we got rid of the
00:55:10.170 --> 00:55:11.870
catastrophicity.
00:55:11.870 --> 00:55:13.880
Second of all, again,
we reduced nu.
00:55:16.690 --> 00:55:20.760
So this is clearly a very
good thing to do.
00:55:20.760 --> 00:55:28.600
So more generally, going back
here, if all n of D have any
00:55:28.600 --> 00:55:29.850
common factor --
00:55:35.880 --> 00:55:37.130
in other words, let's call --
00:55:40.094 --> 00:55:43.540
I've used d, so what'll I use?
00:55:43.540 --> 00:55:52.220
b of D equals the greatest
common divisor of all of the n
00:55:52.220 --> 00:55:55.065
of D, 1 plus --
00:55:55.065 --> 00:55:56.315
AUDIENCE: [INAUDIBLE].
00:56:00.200 --> 00:56:01.055
PROFESSOR: Who --
00:56:01.055 --> 00:56:03.960
AUDIENCE: Do you think
it could be 1?
00:56:03.960 --> 00:56:08.190
PROFESSOR: You're, once again,
30 seconds ahead of me.
00:56:08.190 --> 00:56:10.830
So yes, that's what
I'm going to say.
00:56:10.830 --> 00:56:13.660
If they all have a common
factor, then they have a GCD
00:56:13.660 --> 00:56:15.150
that is not equal to 1.
00:56:15.150 --> 00:56:18.260
That's if and only if.
00:56:18.260 --> 00:56:27.613
So we're going to transform n
of D to 1 over b of D. We're
00:56:27.613 --> 00:56:31.700
going to divide out the common
factor, n of D. That will be
00:56:31.700 --> 00:56:34.760
our n prime.
00:56:34.760 --> 00:56:36.870
And now what have we achieved?
00:56:36.870 --> 00:56:40.040
We've achieved that there is
no common factor of these n
00:56:40.040 --> 00:56:50.630
prime of D. So no common factor,
which does turn out.
00:56:50.630 --> 00:56:58.410
So that will mean it's
non-catastrophic, and,
00:56:58.410 --> 00:57:08.750
furthermore, will reduce nu by
whatever the degree of this
00:57:08.750 --> 00:57:13.735
common factor was, degree
of b of D. Let me
00:57:13.735 --> 00:57:15.010
write degree of GCD.
00:57:20.740 --> 00:57:22.320
We want them to have
no common factor.
00:57:22.320 --> 00:57:28.810
We want the greatest common
divisor to be 1.
00:57:28.810 --> 00:57:31.550
Another way of saying this is
we want the ni of d to be
00:57:31.550 --> 00:57:36.950
relatively prime, the nj of
d to be relatively prime.
00:57:36.950 --> 00:57:43.130
And if that's true, then
can we prove that it's
00:57:43.130 --> 00:57:45.740
non-catastrophic, that if we
have a finite output, that it
00:57:45.740 --> 00:57:49.290
must've been generated
by a finite input?
00:57:52.188 --> 00:57:56.102
AUDIENCE: No, because u of D
times n_i of D would have to
00:57:56.102 --> 00:58:03.070
be a polynomial to be
catastrophic, and to get a
00:58:03.070 --> 00:58:04.320
polynomial.
00:58:05.916 --> 00:58:09.260
And u of D is rational.
00:58:09.260 --> 00:58:10.730
It has something in
the denominator,
00:58:10.730 --> 00:58:12.230
because u of D is infinite.
00:58:14.800 --> 00:58:17.970
That denominator should
then get cancelled
00:58:17.970 --> 00:58:19.850
out to one of these.
00:58:19.850 --> 00:58:24.500
PROFESSOR: Yeah, you have the
argument in your head.
00:58:24.500 --> 00:58:26.930
Let me leave that as an exercise
for the student,
00:58:26.930 --> 00:58:28.910
since at least a couple
of you see it.
00:58:31.840 --> 00:58:34.950
The only way that you can get
this behavior, if you have an
00:58:34.950 --> 00:58:39.620
infinite input which has in
its denominator a common
00:58:39.620 --> 00:58:42.170
factor of all the n's.
00:58:42.170 --> 00:58:45.700
So if the n's have
no common factor,
00:58:45.700 --> 00:58:46.820
then this can't happen.
00:58:46.820 --> 00:58:50.680
That's the outline
of the argument.
00:58:50.680 --> 00:58:56.970
So what's our conclusion now?
00:58:56.970 --> 00:59:01.030
Since we have multiple encoders
that generate the
00:59:01.030 --> 00:59:06.440
same code, we'd like to get the
nicest one, in some sense,
00:59:06.440 --> 00:59:10.470
as our canonical encoder
for a given code.
00:59:10.470 --> 00:59:11.540
So we have a given code.
00:59:11.540 --> 00:59:16.780
We can specify it by any causal
00:59:16.780 --> 00:59:19.630
rational transfer function.
00:59:19.630 --> 00:59:24.740
But then, having done that, we
should clean this generator
00:59:24.740 --> 00:59:26.130
n-tuple up.
00:59:26.130 --> 00:59:31.540
We should multiply out by the
least common multiple of all
00:59:31.540 --> 00:59:36.680
the divisors to make it
polynomial, feedback-free,
00:59:36.680 --> 00:59:39.650
possibly reduce nu.
00:59:39.650 --> 00:59:42.430
And then we should check the
numerators and see if they
00:59:42.430 --> 00:59:46.310
have any common factor, which
could be d or it could be any
00:59:46.310 --> 00:59:47.160
polynomial.
00:59:47.160 --> 00:59:50.120
And whatever it is, we should
divide that out.
00:59:50.120 --> 00:59:54.060
And that will certainly
reduce nu.
00:59:54.060 --> 01:00:04.190
So at the end of the day, we
have a canonical encoder,
01:00:04.190 --> 01:00:07.940
which is equal to
g of D times --
01:00:07.940 --> 01:00:10.930
we want to multiply by the least
common multiple of the
01:00:10.930 --> 01:00:13.800
denominators.
01:00:13.800 --> 01:00:17.700
We want to divide by the
greatest common divisor of the
01:00:17.700 --> 01:00:18.950
numerators.
01:00:21.770 --> 01:00:29.080
And this will be some n prime
of D, which will be
01:00:29.080 --> 01:00:31.990
polynomial.
01:00:31.990 --> 01:00:34.040
It will be delay-free.
01:00:34.040 --> 01:00:37.760
That means it doesn't have
d as a common factor.
01:00:37.760 --> 01:00:40.090
It will be non-catastrophic.
01:00:40.090 --> 01:00:41.240
That means it doesn't
have anything
01:00:41.240 --> 01:00:44.350
else as a common factor.
01:00:44.350 --> 01:00:49.890
And it will have minimal nu,
for any encoder that can
01:00:49.890 --> 01:00:52.250
generate that same code.
01:00:52.250 --> 01:00:55.350
I haven't quite proved all those
properties, but I think
01:00:55.350 --> 01:00:57.310
we're well along the way.
01:00:57.310 --> 01:00:58.365
So this is really --
01:00:58.365 --> 01:00:59.615
AUDIENCE: [INAUDIBLE].
01:01:03.430 --> 01:01:06.063
There should be at
least n1 and n2.
01:01:06.063 --> 01:01:07.165
[UNINTELLIGIBLE]
01:01:07.165 --> 01:01:11.500
PROFESSOR: Well, if we have a
rate 1/1 code, we just have
01:01:11.500 --> 01:01:14.660
one of them, then what's the
greatest common divisor?
01:01:14.660 --> 01:01:18.620
Suppose it's n of D over d of D.
The greatest common divisor
01:01:18.620 --> 01:01:22.820
is n of D. The least common
multiple is d of D. We get 1.
01:01:22.820 --> 01:01:26.250
So going through this exercise
for any rate 1/1 code, we'll
01:01:26.250 --> 01:01:30.790
find that the canonical
encoder is 1.
01:01:30.790 --> 01:01:34.800
And that is clearly what we want
for a rate 1/1 encoder,
01:01:34.800 --> 01:01:38.570
because a rate 1/1 is always
going to have an out.
01:01:38.570 --> 01:01:41.250
The code is just going to be the
universe code of all the
01:01:41.250 --> 01:01:43.770
possible formal Laurent
sequences.
01:01:43.770 --> 01:01:47.470
So we get down to the right
encoder for that code, not
01:01:47.470 --> 01:01:48.720
some stupid encoder.
01:01:50.880 --> 01:01:55.690
And in a similar way, going
through this process, we come
01:01:55.690 --> 01:01:56.590
up with the right encoder.
01:01:56.590 --> 01:01:59.230
Is this process unique?
01:01:59.230 --> 01:02:04.200
Is there only one encoder
we could come up with?
01:02:07.570 --> 01:02:11.360
Think about it a little, and
you could convince yourself
01:02:11.360 --> 01:02:15.960
there's only encoder that you
can get from doing this.
01:02:19.220 --> 01:02:20.620
This is over F2.
01:02:20.620 --> 01:02:25.000
Over a larger field, it's unique
up to units again, up
01:02:25.000 --> 01:02:28.990
to a non-zero scalar multiple,
which of course you can define
01:02:28.990 --> 01:02:34.390
to be 1 and somehow
make this unique.
01:02:34.390 --> 01:02:41.495
So it's basically a unique
canonical encoder that has all
01:02:41.495 --> 01:02:42.550
of these properties.
01:02:42.550 --> 01:02:44.880
You get it in this way.
01:02:44.880 --> 01:02:48.400
So this is what we're
always going to use.
01:02:48.400 --> 01:02:55.135
For instance, for our original
encoder, 1 plus --
01:02:55.135 --> 01:02:55.850
what was it? --
01:02:55.850 --> 01:02:59.800
1 plus d squared, 1 plus
d plus d squared.
01:02:59.800 --> 01:03:01.500
Is that already in
canonical form?
01:03:07.470 --> 01:03:09.280
This is irreducible.
01:03:09.280 --> 01:03:13.990
This is divisible by 1 plus D,
so these are relatively prime.
01:03:13.990 --> 01:03:15.510
It's feed-forward.
01:03:15.510 --> 01:03:19.330
It doesn't have any denominator
terms, and it's
01:03:19.330 --> 01:03:20.230
delay-free.
01:03:20.230 --> 01:03:25.500
So it satisfies all of these,
and clearly you can't
01:03:25.500 --> 01:03:26.400
manipulate it.
01:03:26.400 --> 01:03:28.580
The only ways you're allowed
to manipulate it are by
01:03:28.580 --> 01:03:35.530
multiplying by a of D over b of
D. And it's clear that if
01:03:35.530 --> 01:03:38.350
you have a non-trivial numerator
or denominator, the
01:03:38.350 --> 01:03:40.290
numerator will make it
catastrophic, the denominator
01:03:40.290 --> 01:03:44.800
will make it to have feedback.
01:03:44.800 --> 01:03:46.340
Now, as I was saying,
there's really
01:03:46.340 --> 01:03:47.940
nothing wrong with feedback.
01:03:47.940 --> 01:03:51.560
And so some people prefer --
01:03:51.560 --> 01:03:54.160
the one thing that this doesn't
have, in general, is
01:03:54.160 --> 01:03:57.030
it's not systematic.
01:03:57.030 --> 01:03:59.990
That means the input stream is
not one of the output streams.
01:04:06.060 --> 01:04:10.910
So given a canonical encoder
like this, can we always make
01:04:10.910 --> 01:04:12.160
it systematic?
01:04:15.450 --> 01:04:20.910
Yeah, we just divide by any
of these polynomials.
01:04:20.910 --> 01:04:25.450
So here's an equivalent
systematic encoder.
01:04:31.420 --> 01:04:35.380
For instance, if we divide
both of these by 1 plus D
01:04:35.380 --> 01:04:40.310
squared, we get 1 plus
D plus D squared,
01:04:40.310 --> 01:04:41.590
over 1 plus d squared.
01:04:45.580 --> 01:04:48.390
This now is not polynomial.
01:04:48.390 --> 01:04:49.810
It is delay-free.
01:04:54.190 --> 01:04:55.440
It's non-catastrophic.
01:04:58.570 --> 01:05:01.210
There's no 1 over something that
we can put in to get a
01:05:01.210 --> 01:05:03.140
finite thing out.
01:05:03.140 --> 01:05:07.560
It turns out it's still
minimal nu.
01:05:07.560 --> 01:05:10.090
We know how to realize this with
two memory elements now.
01:05:13.240 --> 01:05:17.580
And in general, if you're
not going to introduce a
01:05:17.580 --> 01:05:20.870
denominator term that has a
greater degree than the
01:05:20.870 --> 01:05:23.260
maximum degree of the numerator
terms, so it's still
01:05:23.260 --> 01:05:25.460
realizable with minimal nu.
01:05:25.460 --> 01:05:33.380
And so now it's systematic,
but now it's not
01:05:33.380 --> 01:05:34.630
feedback-free.
01:05:41.270 --> 01:05:44.280
So I consider this one
certainly the nicest.
01:05:44.280 --> 01:05:50.060
The canonical feedback-free
encoder is the nicest one for
01:05:50.060 --> 01:05:53.370
analytical purposes, for
instance, for trying to find
01:05:53.370 --> 01:05:56.510
minumum-weight finite
code words.
01:05:56.510 --> 01:05:59.670
Because of the finite property,
if we're looking for
01:05:59.670 --> 01:06:01.790
the non-catastrophic property,
if we're looking for
01:06:01.790 --> 01:06:03.850
minimum-weight code words,
we only have to
01:06:03.850 --> 01:06:05.720
look at finite inputs.
01:06:05.720 --> 01:06:08.990
We can now use the kind of
analysis method that we use
01:06:08.990 --> 01:06:12.930
that says you always have to
come back to the 0 state,
01:06:12.930 --> 01:06:19.830
because only finite inputs can
produce finite outputs here.
01:06:19.830 --> 01:06:22.970
This also has that property.
01:06:22.970 --> 01:06:30.580
And people tended not to use
these for a long time, but it
01:06:30.580 --> 01:06:35.400
was founded that in turbo codes,
you want to have this,
01:06:35.400 --> 01:06:38.460
one of the generators
be infinite.
01:06:38.460 --> 01:06:41.300
And it was also nice to have one
of them be systematic, so
01:06:41.300 --> 01:06:44.030
that led to a revival of
interest in systematic
01:06:44.030 --> 01:06:45.090
convolutional encoders.
01:06:45.090 --> 01:06:47.150
And this, again, it's
a unique form.
01:06:47.150 --> 01:06:50.770
If you say the first one has to
be the systematic one, then
01:06:50.770 --> 01:06:53.710
there's a unique equivalent
systematic encoder that
01:06:53.710 --> 01:06:55.330
generates any code.
01:06:55.330 --> 01:06:58.190
So it's in alternative
canonical form.
01:06:58.190 --> 01:06:59.440
Take your pick.
01:07:03.480 --> 01:07:04.390
Yeah?
01:07:04.390 --> 01:07:07.120
AUDIENCE: What's systematic?
01:07:07.120 --> 01:07:08.870
PROFESSOR: What's systematic?
01:07:08.870 --> 01:07:12.630
Systematic means that the input
that's appear unchanged
01:07:12.630 --> 01:07:15.560
as a subset of the
output bits.
01:07:15.560 --> 01:07:19.720
For convolutional codes, it
means that one of the output
01:07:19.720 --> 01:07:23.400
sequences, if you have a rate
1/n encoder, one of the output
01:07:23.400 --> 01:07:25.600
sequences is simply equal
to the input sequence.
01:07:25.600 --> 01:07:26.900
In other words, one
of the transfer
01:07:26.900 --> 01:07:29.908
functions is equal to 1.
01:07:29.908 --> 01:07:31.158
AUDIENCE: [INAUDIBLE].
01:07:34.450 --> 01:07:36.570
PROFESSOR: Yes, very good.
01:07:36.570 --> 01:07:38.998
Excellent comment.
01:07:38.998 --> 01:07:44.890
Because now, if you have a
finite output sequence, in
01:07:44.890 --> 01:07:48.110
particular you can just read
off the input sequence, the
01:07:48.110 --> 01:07:51.640
information sequence, from this
first term, and if the
01:07:51.640 --> 01:07:53.880
whole thing was finite,
it's finite.
01:07:53.880 --> 01:07:59.360
So systematic directly ensures
non-catastrophicity.
01:07:59.360 --> 01:08:00.630
Excellent, excellent comment.
01:08:05.410 --> 01:08:08.370
Which you might try to prove
by the more elaborate
01:08:08.370 --> 01:08:11.410
techniques that we had.
01:08:11.410 --> 01:08:16.590
So that gets us pretty much
through the algebra that I
01:08:16.590 --> 01:08:19.920
wanted to do.
01:08:19.920 --> 01:08:23.069
Now I want to --
01:08:23.069 --> 01:08:26.890
OK, good.
01:08:26.890 --> 01:08:29.649
Now let's go more to the
finite state picture.
01:08:33.689 --> 01:08:51.160
And our objective here is mostly
going to be to get to
01:08:51.160 --> 01:08:54.220
Viterbi decoding algorithm.
01:08:54.220 --> 01:08:57.550
But along the way, we'll see
that we can use something like
01:08:57.550 --> 01:09:00.090
the Viterbi algorithm to compute
minimum distances,
01:09:00.090 --> 01:09:04.349
too, once we have a finite state
picture to the code.
01:09:07.620 --> 01:09:17.770
Again, take our example one,
where g of D is 1 plus D
01:09:17.770 --> 01:09:21.270
squared, 1 plus D
plus D squared.
01:09:21.270 --> 01:09:27.569
We have a state transition
diagram,
01:09:27.569 --> 01:09:34.469
which looks like this.
01:09:43.890 --> 01:09:48.074
And I won't bother
to label it.
01:09:52.060 --> 01:09:55.350
And that's a complete
description of the encoder
01:09:55.350 --> 01:10:00.260
operation, if I put labels of
inputs and outputs on all
01:10:00.260 --> 01:10:01.695
these states.
01:10:01.695 --> 01:10:08.110
So 0 slash 0,0, so forth,
1 slash 1, 1.
01:10:08.110 --> 01:10:12.400
Just to draw the impulse
response, 0, 0, 1,
01:10:12.400 --> 01:10:15.380
and 0 slash 1, 1.
01:10:15.380 --> 01:10:18.110
There's the basic impulse
response going
01:10:18.110 --> 01:10:19.360
around, like that.
01:10:22.940 --> 01:10:27.900
Trellis diagram is
a very redundant
01:10:27.900 --> 01:10:29.390
picture of the same thing.
01:10:35.070 --> 01:10:38.600
We can start out in the
0 state that we like.
01:10:38.600 --> 01:10:44.950
Let's say we start off in the 0
state at some time k, as we
01:10:44.950 --> 01:10:45.530
always are.
01:10:45.530 --> 01:10:47.010
We just don't know what k is.
01:10:50.260 --> 01:10:52.600
Then let's spread the
state transition
01:10:52.600 --> 01:10:54.070
diagram out in time.
01:10:54.070 --> 01:10:57.610
Let's draw all the states
again at time k plus 1.
01:11:00.380 --> 01:11:02.490
Actually, there are only
two it can get to,
01:11:02.490 --> 01:11:03.950
at time k plus 1.
01:11:03.950 --> 01:11:09.870
So we can either stay in this
state or we can go down here
01:11:09.870 --> 01:11:15.450
to this state, at the
next unit of time.
01:11:15.450 --> 01:11:19.780
I'm just going to keep drawing
all of the possible state
01:11:19.780 --> 01:11:22.740
trajectories or paths.
01:11:22.740 --> 01:11:27.080
At time 2, I can reach
all possible states.
01:11:27.080 --> 01:11:32.730
0, 0, 1, 0, 0, 1, 1, 1.
01:11:32.730 --> 01:11:38.240
Here again, I already
know what these are.
01:11:38.240 --> 01:11:43.741
From here, I can
get 1, 0, 2, 0.
01:11:43.741 --> 01:11:46.660
It goes with a 0, 1 out.
01:11:46.660 --> 01:11:51.170
So 1 goes with a 1, 0.
01:11:51.170 --> 01:11:59.380
And k plus 3, I now have all
possible transitions.
01:11:59.380 --> 01:12:02.120
It's going to look like this.
01:12:02.120 --> 01:12:04.110
Where can I go from 0, 1?
01:12:04.110 --> 01:12:07.970
I can go back to here, or
I could go to here.
01:12:07.970 --> 01:12:09.950
And I could label these.
01:12:09.950 --> 01:12:11.625
I should label these.
01:12:11.625 --> 01:12:12.620
But I won't.
01:12:12.620 --> 01:12:15.770
Here, I can go to these two.
01:12:15.770 --> 01:12:18.210
There are eight possible
state transitions.
01:12:18.210 --> 01:12:20.440
k plus 4.
01:12:20.440 --> 01:12:22.470
It's completely repetitive,
since it's a
01:12:22.470 --> 01:12:24.940
time-invariant system.
01:12:24.940 --> 01:12:27.210
I just keep going like this.
01:12:34.610 --> 01:12:36.720
And why have I gone to
all this trouble?
01:12:36.720 --> 01:12:40.555
This clearly contains the same
information as this diagram,
01:12:40.555 --> 01:12:42.340
but it's just spread
out in time.
01:12:45.200 --> 01:12:54.210
The basic reason for doing it is
that now there is a unique
01:12:54.210 --> 01:12:58.040
trellis path corresponding to
every code word that can start
01:12:58.040 --> 01:12:59.380
at time k or later.
01:13:02.010 --> 01:13:06.840
So for instance, if I just have
an impulse at time k and
01:13:06.840 --> 01:13:12.450
nothing else, then the unique
trellis path is this.
01:13:12.450 --> 01:13:16.700
It starts at 0, goes through
two non-zero states, comes
01:13:16.700 --> 01:13:21.710
back to 0, 0, and then stays
in 0, 0 forever more.
01:13:25.860 --> 01:13:27.351
AUDIENCE: [INAUDIBLE].
01:13:27.351 --> 01:13:30.890
PROFESSOR: I've just taken an
arbitrary starting time, k.
01:13:30.890 --> 01:13:33.110
After a while, I'll
just look at it --
01:13:33.110 --> 01:13:34.100
well, it's entrained.
01:13:34.100 --> 01:13:36.710
This is kind of the steady state
version of the trellis.
01:13:36.710 --> 01:13:40.880
I've included a little
starting part here.
01:13:40.880 --> 01:13:45.670
And if I terminated the trellis,
then I would also
01:13:45.670 --> 01:13:46.360
have an ending time.
01:13:46.360 --> 01:13:49.460
We'll talk about that.
01:13:49.460 --> 01:13:52.250
So at this point, I just want
you to see what the trellis
01:13:52.250 --> 01:13:53.090
diagram is.
01:13:53.090 --> 01:13:55.740
It's basically just an unrolling
of the state
01:13:55.740 --> 01:13:58.130
transition diagram
out in time.
01:13:58.130 --> 01:14:00.310
And it allows me
to associate --
01:14:00.310 --> 01:14:02.580
there's a one-to-one
correspondence now between
01:14:02.580 --> 01:14:07.180
code words that start at time k
or later, and trellis paths.
01:14:07.180 --> 01:14:11.510
So every one corresponds
to a unique, distinct
01:14:11.510 --> 01:14:13.170
path through the code.
01:14:13.170 --> 01:14:16.170
That's what we're going
to use in decoding.
01:14:16.170 --> 01:14:20.980
Each of these paths represents
one possible decision that the
01:14:20.980 --> 01:14:24.160
decoder could do about what
was actually set.
01:14:24.160 --> 01:14:29.230
The decoder is actually going
to try to find the best of
01:14:29.230 --> 01:14:32.290
these paths, in some sense, the
one that's closest to the
01:14:32.290 --> 01:14:34.810
received sequence, in the
Hamming distance or Euclidean
01:14:34.810 --> 01:14:39.360
distance, or likelihood sense.
01:14:39.360 --> 01:14:42.810
Let's pause for a second, and
see how this could be used to
01:14:42.810 --> 01:14:48.890
compute the minimum distance
of the code.
01:14:48.890 --> 01:14:55.810
Let's assume that I have a
non-catastrophic encoder.
01:14:55.810 --> 01:14:58.900
Then I know the finite sequences
are going to be
01:14:58.900 --> 01:15:03.510
those that start in the 0
state, go through some
01:15:03.510 --> 01:15:08.170
trajectory through the state
transition diagram, and then
01:15:08.170 --> 01:15:11.620
ultimately come back
to the 0 state.
01:15:11.620 --> 01:15:15.040
If I have canonical encoder,
I agreed what I'm
01:15:15.040 --> 01:15:17.270
going to use now.
01:15:17.270 --> 01:15:21.380
So one way of computing the
minimum distance would just be
01:15:21.380 --> 01:15:28.780
to perform a search through
here, and try to find the
01:15:28.780 --> 01:15:34.496
minimum detour, the minimum
weight detour that starts in
01:15:34.496 --> 01:15:36.990
the 0 state, eventually gets
back to the 0 state.
01:15:36.990 --> 01:15:40.080
It accumulates Hamming distance
along the way.
01:15:40.080 --> 01:15:41.770
And that is the Hamming
distance of
01:15:41.770 --> 01:15:45.480
some valid code sequence.
01:15:45.480 --> 01:15:49.490
So here, it's not very tough.
01:15:49.490 --> 01:15:52.220
And I've already gone
through an argument.
01:15:52.220 --> 01:15:55.030
If you didn't know, you'd
say, OK, here's the
01:15:55.030 --> 01:15:56.090
first one to look at.
01:15:56.090 --> 01:16:01.740
This is the only one that gets
back in three time units.
01:16:01.740 --> 01:16:06.230
Let's look for the finite path
that gets back to the 0 state
01:16:06.230 --> 01:16:07.630
in four time units.
01:16:07.630 --> 01:16:09.560
Again, there's only
one of them.
01:16:09.560 --> 01:16:13.235
It looks like that, and it turns
out it has weight 6.
01:16:15.900 --> 01:16:20.190
So this is, again, it always
ends up with a 1:1 when it
01:16:20.190 --> 01:16:22.280
merges back in here.
01:16:22.280 --> 01:16:26.210
And this, I think, is 0, 1
or something like that.
01:16:26.210 --> 01:16:30.950
The only 0, 0 path in
here is this one.
01:16:30.950 --> 01:16:34.510
But you can easily see the 0,
0 path doesn't form a loop.
01:16:34.510 --> 01:16:38.160
You can't proceed down
0, 0 forever.
01:16:38.160 --> 01:16:41.280
So I go through here, and I've
already got four units of
01:16:41.280 --> 01:16:42.290
distance up to here.
01:16:42.290 --> 01:16:44.440
In fact, I need to follow.
01:16:44.440 --> 01:16:48.900
Next time I explore out here,
1, 1, 0, I've already got up
01:16:48.900 --> 01:16:51.140
to four units of distance.
01:16:51.140 --> 01:16:54.320
Now I know I'm always going
to have two at the end.
01:16:54.320 --> 01:16:59.120
So any time I get to four
or more, I can stop.
01:16:59.120 --> 01:17:01.830
And by that argument, I'm
already done when
01:17:01.830 --> 01:17:03.410
I get to this one.
01:17:03.410 --> 01:17:06.350
Or if I didn't want to use
that, I'd just explore --
01:17:06.350 --> 01:17:09.320
I already know that there's one
of weight 5, the impulse
01:17:09.320 --> 01:17:13.480
response, so I'd explore all
the possibilities out until
01:17:13.480 --> 01:17:16.450
they accumulated at least weight
5, and assure myself
01:17:16.450 --> 01:17:19.130
that there was no way of getting
back here with weight
01:17:19.130 --> 01:17:20.600
less than 5.
01:17:20.600 --> 01:17:24.030
And that would be a very
systematic way of evaluating
01:17:24.030 --> 01:17:26.020
the minimum distance
to the code.
01:17:26.020 --> 01:17:29.481
You can see you could do that
for any generators.
01:17:29.481 --> 01:17:31.020
You with me?
01:17:31.020 --> 01:17:37.620
Just explore this little graph,
picking up weight as
01:17:37.620 --> 01:17:38.870
you go along.
01:17:42.480 --> 01:17:45.210
That's basically the way the
Viterbi algorithm goes as
01:17:45.210 --> 01:17:54.940
well, except what we do is we
have some received pair at
01:17:54.940 --> 01:18:08.730
time k, r1 k, r2 k, r1 k
plus 1, r2 k plus 1.
01:18:08.730 --> 01:18:11.920
We get two received symbols at
every time, whatever channel
01:18:11.920 --> 01:18:14.300
we're going over.
01:18:14.300 --> 01:18:19.020
Assuming that the channel is
memory-less, I get a some kind
01:18:19.020 --> 01:18:21.870
of distance or weight or
likelihood weight at each
01:18:21.870 --> 01:18:26.720
time, and maximum likelihood
sequence decoding amounts to
01:18:26.720 --> 01:18:30.830
finding the sequence that's
closest, the code sequence
01:18:30.830 --> 01:18:33.880
that's closest in Hamming
distance or Euclidean distance
01:18:33.880 --> 01:18:36.350
or likelihood distance to the
transmitted sequence.
01:18:39.800 --> 01:18:43.180
So I can simply put a metric
on each of these branches
01:18:43.180 --> 01:18:45.650
corresponding to
what this was.
01:18:45.650 --> 01:18:47.590
Now, if it was a binary
symmetric channel and I
01:18:47.590 --> 01:18:51.310
received 0, 0, then I'd have
0 weight here, but weight 2
01:18:51.310 --> 01:18:53.660
here, in terms of Hamming
distance from
01:18:53.660 --> 01:18:56.090
the received sequence.
01:18:56.090 --> 01:18:58.470
And similarly, as I go along,
I'd have another set of
01:18:58.470 --> 01:18:59.130
metrics here.
01:18:59.130 --> 01:19:02.020
If I received 1, 1, then this
one would have weight 2, this
01:19:02.020 --> 01:19:05.160
one would be good, no distance
from the received sequence.
01:19:05.160 --> 01:19:06.640
Each of these would have
distance 1 from
01:19:06.640 --> 01:19:08.440
the received sequence.
01:19:08.440 --> 01:19:14.680
So I get some kind of cost
for every path in here.
01:19:14.680 --> 01:19:16.830
And then the decoding algorithm
is simply a matter
01:19:16.830 --> 01:19:20.560
of finding the minimum
cost path.
01:19:20.560 --> 01:19:23.380
You could all do that.
01:19:23.380 --> 01:19:29.165
We just had a celebration for
Andy Viterbi out in USC, where
01:19:29.165 --> 01:19:33.720
he's given them $50 million, so
they were very happy, and
01:19:33.720 --> 01:19:35.060
had a great celebration.
01:19:35.060 --> 01:19:37.970
And the thing he's best known
for is the Viterbi algorithm.
01:19:37.970 --> 01:19:41.180
But what everybody knows is
that, once you reduce the
01:19:41.180 --> 01:19:43.410
problem to this point, anybody
could have come up with the
01:19:43.410 --> 01:19:45.260
Viterbi algorithm.
01:19:45.260 --> 01:19:47.960
The Viterbi algorithm is just a
systematic way for trying to
01:19:47.960 --> 01:19:53.620
find the lowest-cost path as
you go through a trellis.
01:19:53.620 --> 01:19:56.570
And if you know dynamic
programming and it's obvious
01:19:56.570 --> 01:20:02.650
how to do it, or even if you
don't, the next idea --
01:20:02.650 --> 01:20:03.900
do we have a minute?
01:20:06.500 --> 01:20:09.220
There's only one idea in the
Viterbi algorithm, which is,
01:20:09.220 --> 01:20:12.280
when you get to this point,
you can already make a
01:20:12.280 --> 01:20:15.040
decision about what the best
path is to get to each of
01:20:15.040 --> 01:20:18.070
these states.
01:20:18.070 --> 01:20:22.130
If the closest path over the
first three intervals is this
01:20:22.130 --> 01:20:28.040
one, not this one, say, then you
can forget about this path
01:20:28.040 --> 01:20:30.270
because it's never going to be
the first part of the ultimate
01:20:30.270 --> 01:20:33.370
closest path, because we're
simply adding up independent
01:20:33.370 --> 01:20:34.620
cost, independent increments.
01:20:37.230 --> 01:20:41.870
So you can make a subdecision at
each of these states, what
01:20:41.870 --> 01:20:47.090
the best path is up to that
state, the closest path.
01:20:47.090 --> 01:20:49.290
And you only need to
remember that.
01:20:49.290 --> 01:20:52.000
That's called a survivor.
01:20:52.000 --> 01:20:56.030
And this was basically, Viterbi
was the first one to
01:20:56.030 --> 01:20:59.010
put down this algorithm
with the survivor.
01:20:59.010 --> 01:21:01.340
You can make interim
decisions.
01:21:01.340 --> 01:21:03.250
You can only keep
the survivor.
01:21:03.250 --> 01:21:06.290
Then all you have to do is
extend the survivor's one unit
01:21:06.290 --> 01:21:07.280
of time out here.
01:21:07.280 --> 01:21:10.130
Again, you have to add
the new metric.
01:21:10.130 --> 01:21:11.840
You have to compare
the metrics to see
01:21:11.840 --> 01:21:12.590
which one is better.
01:21:12.590 --> 01:21:15.450
You select the survivor, and
once again, you've only got
01:21:15.450 --> 01:21:17.630
four survivors going forward.
01:21:17.630 --> 01:21:20.500
So you get an iterative
recursive algorithm that just
01:21:20.500 --> 01:21:26.830
proceeds forward one unit of
time at a time, and keeps
01:21:26.830 --> 01:21:29.590
finding the four best survivors,
or in general, the
01:21:29.590 --> 01:21:34.630
2 to the nu best survivors at
time k plus 3, k plus 4, k
01:21:34.630 --> 01:21:37.450
plus 5, ad infinitum.
01:21:37.450 --> 01:21:40.960
And that's the Viterbi
algorithm.
01:21:40.960 --> 01:21:43.480
So once you've drawn this
trellis picture, it's very
01:21:43.480 --> 01:21:47.030
clear how you should decode, and
you get a very sufficient
01:21:47.030 --> 01:21:49.400
decoding algorithm.
01:21:49.400 --> 01:21:51.490
We'll come back next time.
01:21:51.490 --> 01:21:53.320
I'll do that a little
bit more carefully.
01:21:53.320 --> 01:21:56.040
I'll talk about terminated
codes.
01:21:56.040 --> 01:21:58.590
I'll talk about how it works
when the code isn't
01:21:58.590 --> 01:22:01.880
terminated, and I'll talk about
performance analysis.
01:22:01.880 --> 01:22:04.780
But we're actually pretty close
to the end of chapter
01:22:04.780 --> 01:22:06.030
nine at this point.