1
00:00:00,000 --> 00:00:00,280
2
00:00:00,280 --> 00:00:00,560
PROFESSOR: OK.
3
00:00:00,560 --> 00:00:01,360
Welcome back.
4
00:00:01,360 --> 00:00:04,830
I hope you had a nice break.
5
00:00:04,830 --> 00:00:10,920
The midterms will be handed out
at the end of the class.
6
00:00:10,920 --> 00:00:15,730
Ashish has just gone to
get them in my office.
7
00:00:15,730 --> 00:00:17,580
With solutions.
8
00:00:17,580 --> 00:00:19,430
I hope you look at
the solutions.
9
00:00:19,430 --> 00:00:23,910
There's more in the solutions
than is strictly necessary.
10
00:00:23,910 --> 00:00:27,070
The midterm is partly intended
as a learning exercise, and I
11
00:00:27,070 --> 00:00:30,340
hope you learned something
from it.
12
00:00:30,340 --> 00:00:34,070
I didn't learn too much from
it about the performance of
13
00:00:34,070 --> 00:00:37,770
the class, because some of the
questions were easy enough
14
00:00:37,770 --> 00:00:39,640
that practically everybody
got them, and some of the
15
00:00:39,640 --> 00:00:42,210
questions were too hard
for anybody to get.
16
00:00:42,210 --> 00:00:45,290
And so the distribution --
17
00:00:45,290 --> 00:00:48,720
you know, what I would really
like to get is a nice
18
00:00:48,720 --> 00:00:52,910
bell-shaped curve going from 0
to 100 with a median at 50 and
19
00:00:52,910 --> 00:00:55,020
the standard deviation
about 25.
20
00:00:55,020 --> 00:00:57,740
And I got a much more compressed
curve than that.
21
00:00:57,740 --> 00:01:04,650
So frankly, I didn't learn
that much about relative
22
00:01:04,650 --> 00:01:06,030
performance.
23
00:01:06,030 --> 00:01:08,880
And I gather from Ashish that
you've all been doing the
24
00:01:08,880 --> 00:01:12,230
homeworks, and have been doing
a fairly steady job on that.
25
00:01:12,230 --> 00:01:16,760
So I think everybody is
close to being in
26
00:01:16,760 --> 00:01:17,670
the same boat here.
27
00:01:17,670 --> 00:01:21,140
If anybody has concerns about
their performance, of course
28
00:01:21,140 --> 00:01:23,380
they're welcome to talk to me.
29
00:01:23,380 --> 00:01:26,560
Drop date is a little
while off.
30
00:01:26,560 --> 00:01:29,995
If you're concerned about what
your grade might be, I'll give
31
00:01:29,995 --> 00:01:31,660
you the best idea I can.
32
00:01:31,660 --> 00:01:35,620
But really, it's going
to come down to the
33
00:01:35,620 --> 00:01:37,330
final for most of you.
34
00:01:37,330 --> 00:01:39,500
So it's not too much
to say here.
35
00:01:39,500 --> 00:01:43,950
I give the grade distribution
and the midterm solutions, and
36
00:01:43,950 --> 00:01:47,180
from that, you could figure out
where you are relatively.
37
00:01:47,180 --> 00:01:54,440
And my grading philosophy is at
MIT, in graduate classes,
38
00:01:54,440 --> 00:01:57,230
roughly they're half
As and half Bs.
39
00:01:57,230 --> 00:01:59,950
I have that as a guideline
in mind.
40
00:01:59,950 --> 00:02:02,670
I try to look for where the
natural breaks are.
41
00:02:02,670 --> 00:02:06,440
I decorate the As and Bs with
pluses and minuses, but I
42
00:02:06,440 --> 00:02:09,259
don't think anybody pays
attention to those.
43
00:02:09,259 --> 00:02:12,680
And in other words, I generally
try to follow the
44
00:02:12,680 --> 00:02:17,770
MIT grading philosophy
as I understand it.
45
00:02:17,770 --> 00:02:23,330
So are there any questions about
the midterm or anything
46
00:02:23,330 --> 00:02:28,220
about the class as we go
into the second half?
47
00:02:28,220 --> 00:02:28,710
No?
48
00:02:28,710 --> 00:02:30,860
I see some people
aren't back yet.
49
00:02:30,860 --> 00:02:32,600
It's a somewhat reduced group.
50
00:02:32,600 --> 00:02:36,010
Maybe a couple people decided
to drop on the
51
00:02:36,010 --> 00:02:37,210
basis of the midterm.
52
00:02:37,210 --> 00:02:38,580
That sometimes happens.
53
00:02:38,580 --> 00:02:42,730
I don't think it necessarily
had to happen this year.
54
00:02:42,730 --> 00:02:46,420
Or maybe people are just having
an extended break.
55
00:02:46,420 --> 00:02:47,910
OK.
56
00:02:47,910 --> 00:02:53,370
We're now going into
conceptually a break, the
57
00:02:53,370 --> 00:02:55,310
second half of the course.
58
00:02:55,310 --> 00:02:59,510
The first half of the course we
were talking about trying
59
00:02:59,510 --> 00:03:02,150
to get to capacity, specifically
on the additive
60
00:03:02,150 --> 00:03:04,200
white Gaussian noise channel.
61
00:03:04,200 --> 00:03:09,280
So far all the schemes that
we've talked about have been
62
00:03:09,280 --> 00:03:14,210
block coding schemes, which
their Euclidean images wind up
63
00:03:14,210 --> 00:03:20,800
to be constellations in a finite
dimensional space.
64
00:03:20,800 --> 00:03:25,580
This is the kind of picture that
Shannon had when he wrote
65
00:03:25,580 --> 00:03:32,290
his original paper on capacity
and information, to start off
66
00:03:32,290 --> 00:03:33,280
information theory.
67
00:03:33,280 --> 00:03:38,640
Basically he proved that you
could get to capacity on any
68
00:03:38,640 --> 00:03:45,390
channel with a randomly
chosen code.
69
00:03:45,390 --> 00:03:47,400
In the setting where we are,
that would be a randomly
70
00:03:47,400 --> 00:03:51,250
chosen binary code where you
just pick all the bits of all
71
00:03:51,250 --> 00:03:54,430
the code words at random
by flipping a coin.
72
00:03:54,430 --> 00:03:55,150
OK?
73
00:03:55,150 --> 00:03:58,790
Just construct a code book at
random, and as the code gets
74
00:03:58,790 --> 00:04:00,460
long enough, that's
going to be good
75
00:04:00,460 --> 00:04:01,860
enough to get to capacity.
76
00:04:01,860 --> 00:04:03,280
It's a block code.
77
00:04:03,280 --> 00:04:08,320
2 to the nr bits, and length
n code words where r is the
78
00:04:08,320 --> 00:04:11,350
regular code.
79
00:04:11,350 --> 00:04:14,440
Well, that was a brilliant
achievement, to show that you
80
00:04:14,440 --> 00:04:16,250
could get to capacity
by doing that.
81
00:04:16,250 --> 00:04:18,310
If you can do it by choosing
codes at random, then there
82
00:04:18,310 --> 00:04:20,660
must be some particular
codes that are good.
83
00:04:20,660 --> 00:04:23,960
Otherwise the average overall
codes wouldn't be good.
84
00:04:23,960 --> 00:04:28,790
So people started to go out
to find good codes.
85
00:04:28,790 --> 00:04:34,630
And there was a fairly quick
realization that Shannon
86
00:04:34,630 --> 00:04:38,550
hadn't addressed some key
parts of the problem.
87
00:04:38,550 --> 00:04:44,470
Particularly the complexity
of encoding and decoding.
88
00:04:44,470 --> 00:04:47,720
The complexity of encoding was
pretty quickly recognized to
89
00:04:47,720 --> 00:04:53,570
be not much of an issue, because
Peter Elias and others
90
00:04:53,570 --> 00:04:56,340
proved that, at least for the
kinds of channels we're
91
00:04:56,340 --> 00:04:59,290
talking about, which are
symmetric, linear codes are
92
00:04:59,290 --> 00:05:00,270
just as good.
93
00:05:00,270 --> 00:05:03,970
A random linear code will
get to capacity.
94
00:05:03,970 --> 00:05:07,820
In other words, say, a block
code with a randomly chosen
95
00:05:07,820 --> 00:05:08,790
generator matrix.
96
00:05:08,790 --> 00:05:12,740
You pick k generators of length
n, again by flipping a
97
00:05:12,740 --> 00:05:16,590
coin, that's good enough
to get to capacity.
98
00:05:16,590 --> 00:05:21,510
So we know we can encode linear
codes with polynomial
99
00:05:21,510 --> 00:05:22,520
complexity.
100
00:05:22,520 --> 00:05:24,730
It's just the matrix
multiplication to encode a
101
00:05:24,730 --> 00:05:26,930
linear block code.
102
00:05:26,930 --> 00:05:27,300
All right?
103
00:05:27,300 --> 00:05:29,920
So encoding isn't the problem.
104
00:05:29,920 --> 00:05:31,520
But decoding.
105
00:05:31,520 --> 00:05:35,250
What kind of decoding have
we talked about so far?
106
00:05:35,250 --> 00:05:38,310
I guess particularly with Reed
Solomon codes, we've talked
107
00:05:38,310 --> 00:05:39,980
about algebraic methods.
108
00:05:39,980 --> 00:05:43,000
But our generic method is
to do maximum likelihood
109
00:05:43,000 --> 00:05:47,850
decoding, which basically
involves computing a distance
110
00:05:47,850 --> 00:05:51,350
or a metric to each
of the code words.
111
00:05:51,350 --> 00:05:52,950
Exponential number
of code words.
112
00:05:52,950 --> 00:05:54,100
2 to the nr code words.
113
00:05:54,100 --> 00:05:59,660
So we have exponential
complexity if we do maximum
114
00:05:59,660 --> 00:06:02,730
likelihood or minimum
distance decoding.
115
00:06:02,730 --> 00:06:05,330
And that, of course, was the
motivation for doing some of
116
00:06:05,330 --> 00:06:09,690
these algebraic decoding
methods, which were the
117
00:06:09,690 --> 00:06:14,820
primary focus coding theory
for several decades.
118
00:06:14,820 --> 00:06:18,980
A great deal of very good effort
went into finding good
119
00:06:18,980 --> 00:06:23,490
classes of codes that were
algebraically decodable, like
120
00:06:23,490 --> 00:06:27,930
Reed-Solomon codes, BCH codes,
Reed-Muller codes.
121
00:06:27,930 --> 00:06:28,710
[UNINTELLIGIBLE]
122
00:06:28,710 --> 00:06:30,050
don't have such --
123
00:06:30,050 --> 00:06:32,430
well, they do, we now realize.
124
00:06:32,430 --> 00:06:35,800
But that was the focus.
125
00:06:35,800 --> 00:06:38,940
And with the algebraic coding
methods, there was polynomial
126
00:06:38,940 --> 00:06:40,040
complexity.
127
00:06:40,040 --> 00:06:43,530
But it began to be realized that
these were never going to
128
00:06:43,530 --> 00:06:46,830
get you to capacity.
129
00:06:46,830 --> 00:06:51,320
Because of their difficulty in
using reliability information,
130
00:06:51,320 --> 00:06:55,740
because most of them are bounded
distance character,
131
00:06:55,740 --> 00:06:57,100
and in fact, that's
still true.
132
00:06:57,100 --> 00:07:01,850
You can't get to capacity with
algebraic decoding methods.
133
00:07:01,850 --> 00:07:05,610
So in parallel with this work
on block codes, there was a
134
00:07:05,610 --> 00:07:12,370
smaller stream of work that has
broadened, I would say, as
135
00:07:12,370 --> 00:07:15,810
the ancestor of the
capacity-approaching codes of
136
00:07:15,810 --> 00:07:20,450
today, on codes with other
kinds of structure.
137
00:07:20,450 --> 00:07:24,520
And specifically dynamical
structure, which is what we're
138
00:07:24,520 --> 00:07:25,820
going to talk about
when we talk about
139
00:07:25,820 --> 00:07:27,520
convolutional codes.
140
00:07:27,520 --> 00:07:31,250
And this was broadened into the
field of codes on graphs.
141
00:07:31,250 --> 00:07:34,750
And codes on graphs are nowadays
the way we get to
142
00:07:34,750 --> 00:07:36,040
channel capacity.
143
00:07:36,040 --> 00:07:36,790
All right?
144
00:07:36,790 --> 00:07:39,030
So that's really where we're
going in the second half of
145
00:07:39,030 --> 00:07:40,520
the course.
146
00:07:40,520 --> 00:07:42,250
We're going to preserve
linear structure.
147
00:07:42,250 --> 00:07:45,450
Linear is always good for the
symmetric channels we're
148
00:07:45,450 --> 00:07:46,580
talking about.
149
00:07:46,580 --> 00:07:48,790
But in addition, we're going
to look for other kinds of
150
00:07:48,790 --> 00:07:54,570
structure which are, in
particular, suitable for
151
00:07:54,570 --> 00:07:59,920
lower-complexity decoding
methods, are suitable for
152
00:07:59,920 --> 00:08:03,480
maximum likelihood or
quasi-maximum likelihood
153
00:08:03,480 --> 00:08:04,610
decoding methods.
154
00:08:04,610 --> 00:08:05,290
All right?
155
00:08:05,290 --> 00:08:08,940
So that's our motivation.
156
00:08:08,940 --> 00:08:09,470
OK.
157
00:08:09,470 --> 00:08:11,356
Convolutional codes.
158
00:08:11,356 --> 00:08:14,480
Convolutional codes were
actually invented by Peter
159
00:08:14,480 --> 00:08:17,645
Elias in an effort to find --
160
00:08:17,645 --> 00:08:20,640
161
00:08:20,640 --> 00:08:24,510
just as you can put a linear
structure on codes without
162
00:08:24,510 --> 00:08:28,420
harming their performance, he
was trying to find more and
163
00:08:28,420 --> 00:08:31,980
more structure but you could put
on codes while still being
164
00:08:31,980 --> 00:08:35,500
able to achieve capacity
with a random ensemble.
165
00:08:35,500 --> 00:08:38,340
And he invented something called
sliding parity-check
166
00:08:38,340 --> 00:08:42,289
codes, which, if you let the
block link go to infinity,
167
00:08:42,289 --> 00:08:46,470
become convolutional codes.
168
00:08:46,470 --> 00:08:54,630
And they say there was a trickle
of effort basically
169
00:08:54,630 --> 00:08:58,060
motivated by the fact that
a number of people did a
170
00:08:58,060 --> 00:09:01,410
practical comparison of
performance versus complexity.
171
00:09:01,410 --> 00:09:05,640
Convolutional codes always
beat block codes.
172
00:09:05,640 --> 00:09:09,880
And so, for instance, I was at
a small company was trying to
173
00:09:09,880 --> 00:09:11,920
apply coding theory.
174
00:09:11,920 --> 00:09:17,470
We quickly gravitated to
convolutional codes, because
175
00:09:17,470 --> 00:09:20,190
they always gave us a better
performance complexity
176
00:09:20,190 --> 00:09:22,200
straight off than block codes.
177
00:09:22,200 --> 00:09:26,640
And I'm going to try to indicate
why that's so.
178
00:09:26,640 --> 00:09:27,120
All right.
179
00:09:27,120 --> 00:09:34,045
So I think the easiest way to
understand convolutional codes
180
00:09:34,045 --> 00:09:38,910
is by example, by putting
down an encoder.
181
00:09:38,910 --> 00:09:43,150
And the canonical example
everybody always uses is this
182
00:09:43,150 --> 00:09:45,060
one, so why should I
be any different?
183
00:09:45,060 --> 00:09:53,660
184
00:09:53,660 --> 00:09:58,780
Here is a rate 1/2 constraint
length 2
185
00:09:58,780 --> 00:10:00,030
convolutional encoder.
186
00:10:00,030 --> 00:10:04,850
187
00:10:04,850 --> 00:10:11,140
And it operates on a stream
of bits coming in.
188
00:10:11,140 --> 00:10:16,570
So what we have here is a time
index stream of bits, uk.
189
00:10:16,570 --> 00:10:19,920
190
00:10:19,920 --> 00:10:21,430
And you can think of
this stream as
191
00:10:21,430 --> 00:10:23,690
being infinite in length.
192
00:10:23,690 --> 00:10:28,320
We'll talk about how to make
this a little bit more precise
193
00:10:28,320 --> 00:10:30,320
as we go along.
194
00:10:30,320 --> 00:10:35,630
And we have a shift register
structure here.
195
00:10:35,630 --> 00:10:39,160
The incoming bit uk is
D means a memory
196
00:10:39,160 --> 00:10:41,050
element or a delay element.
197
00:10:41,050 --> 00:10:43,030
It goes into a flip-flop.
198
00:10:43,030 --> 00:10:45,610
It's delayed for 1 time unit.
199
00:10:45,610 --> 00:10:50,560
There's some discrete time base
going on here, which is
200
00:10:50,560 --> 00:10:51,770
indexed by k.
201
00:10:51,770 --> 00:10:55,570
So k is simply the set
of all integers.
202
00:10:55,570 --> 00:10:57,630
Comes from the index set.
203
00:10:57,630 --> 00:10:58,530
Here is u k.
204
00:10:58,530 --> 00:10:59,930
Here's u k minus 1.
205
00:10:59,930 --> 00:11:03,520
Here is u k minus 2.
206
00:11:03,520 --> 00:11:03,980
All right?
207
00:11:03,980 --> 00:11:07,470
And constraint length 2
means that we save --
208
00:11:07,470 --> 00:11:10,230
we have the present
information bit.
209
00:11:10,230 --> 00:11:12,510
This is called the input
or the information bit.
210
00:11:12,510 --> 00:11:15,820
211
00:11:15,820 --> 00:11:20,300
And the two past information
bits are saved.
212
00:11:20,300 --> 00:11:24,600
And from that, we generate
two output bits.
213
00:11:24,600 --> 00:11:35,470
y1 at time k is given by u k
plus u k minus 2, where this
214
00:11:35,470 --> 00:11:36,600
is a mod 2 sum.
215
00:11:36,600 --> 00:11:38,960
Everything is in f2.
216
00:11:38,960 --> 00:11:45,510
y2k is the sum of u k plus u
k minus 1 plus u k minus 2.
217
00:11:45,510 --> 00:11:49,210
218
00:11:49,210 --> 00:11:49,760
All right?
219
00:11:49,760 --> 00:11:52,610
So this is where we get
the redundancy.
220
00:11:52,610 --> 00:11:54,750
What are we generally
doing with coding?
221
00:11:54,750 --> 00:11:59,300
We're adding redundant bits in
order to get more distance
222
00:11:59,300 --> 00:12:03,320
between the sequences that we
might send, and thereby
223
00:12:03,320 --> 00:12:06,940
hopefully to get a coding gain
on the additive white Gaussian
224
00:12:06,940 --> 00:12:09,650
noise channel.
225
00:12:09,650 --> 00:12:13,350
Here the redundancy comes from
the fact that you get 2 bits
226
00:12:13,350 --> 00:12:16,630
out for every bit
that you put in.
227
00:12:16,630 --> 00:12:20,780
So there ought to be the
possibility of getting some
228
00:12:20,780 --> 00:12:24,310
distance between possible
code sequences.
229
00:12:24,310 --> 00:12:28,390
The code sequences here are
infinite or semi-infinite, but
230
00:12:28,390 --> 00:12:31,880
nonetheless, we can establish
some minimum distance, or as
231
00:12:31,880 --> 00:12:36,260
it's called in this field,
the free distance.
232
00:12:36,260 --> 00:12:37,510
OK.
233
00:12:37,510 --> 00:12:39,870
234
00:12:39,870 --> 00:12:43,280
There are two kinds
of structure here.
235
00:12:43,280 --> 00:12:49,805
First of all, this is a linear
time invariant system.
236
00:12:49,805 --> 00:12:58,350
237
00:12:58,350 --> 00:13:01,640
If we were talking about the
real or complex field, we
238
00:13:01,640 --> 00:13:04,330
might call this a filter.
239
00:13:04,330 --> 00:13:10,010
Or it's actually a single input,
two output filter.
240
00:13:10,010 --> 00:13:16,620
So each of the y sequence
is a filtered
241
00:13:16,620 --> 00:13:18,240
version of the u sequence.
242
00:13:18,240 --> 00:13:21,020
y2 is a filtered version
of the u sequence.
243
00:13:21,020 --> 00:13:24,760
244
00:13:24,760 --> 00:13:28,820
And it's linear basically
because it's made up of linear
245
00:13:28,820 --> 00:13:32,470
elements, lay elements,
and mod 2 adders.
246
00:13:32,470 --> 00:13:40,220
Over the binary field, it's
time-invariant, because if I
247
00:13:40,220 --> 00:13:42,010
change the time index,
it doesn't
248
00:13:42,010 --> 00:13:43,110
really change the equation.
249
00:13:43,110 --> 00:13:45,750
It's very hand-waving
rough here.
250
00:13:45,750 --> 00:13:48,640
But so it's a linear
time-invariant system, and we
251
00:13:48,640 --> 00:13:57,770
can analyze it the way
we analyze a filter.
252
00:13:57,770 --> 00:14:04,410
It's redundant, so it has more
outputs than inputs, which we
253
00:14:04,410 --> 00:14:05,665
want in a coding system.
254
00:14:05,665 --> 00:14:08,630
255
00:14:08,630 --> 00:14:11,240
But other than that, we're going
to find that the same
256
00:14:11,240 --> 00:14:14,700
kind of techniques that we use
for analyzing discrete time
257
00:14:14,700 --> 00:14:17,865
linear filters can be used
to analyze the system.
258
00:14:17,865 --> 00:14:20,710
It's just over a different
field, F2, rather
259
00:14:20,710 --> 00:14:24,190
than R or C. All right?
260
00:14:24,190 --> 00:14:28,380
So that's one kind
of structure.
261
00:14:28,380 --> 00:14:36,600
And secondly, it's a finite
state system.
262
00:14:36,600 --> 00:14:38,100
Let's forget about
all the algebra.
263
00:14:38,100 --> 00:14:41,120
264
00:14:41,120 --> 00:14:45,960
There's a discrete time system
which has two memory elements
265
00:14:45,960 --> 00:14:49,560
in it, each capable
of storing 1 bit.
266
00:14:49,560 --> 00:14:54,460
So how many states are
there in the system?
267
00:14:54,460 --> 00:14:54,920
Four.
268
00:14:54,920 --> 00:14:58,760
There are four possible states
that reflect everything that
269
00:14:58,760 --> 00:14:59,770
went on in the past.
270
00:14:59,770 --> 00:15:03,860
That's all you need to know
about the future, is what the
271
00:15:03,860 --> 00:15:08,600
state is at time k, to see
what's going to happen from
272
00:15:08,600 --> 00:15:10,420
time k onwards.
273
00:15:10,420 --> 00:15:10,770
All right?
274
00:15:10,770 --> 00:15:14,860
So this is a simple four-state
system, in this case.
275
00:15:14,860 --> 00:15:17,910
In general, if we build similar
encoders out of shift
276
00:15:17,910 --> 00:15:21,320
registers, even if we put
feedback in them, it's going
277
00:15:21,320 --> 00:15:24,190
to be a finite state system.
278
00:15:24,190 --> 00:15:27,520
So we're going to restrict
ourselves to finite state
279
00:15:27,520 --> 00:15:31,200
realizations, finite
state encoders.
280
00:15:31,200 --> 00:15:35,330
And that's going to ultimately
be the basis for the decoding
281
00:15:35,330 --> 00:15:39,180
algorithm, which is going to be
Viterbi algorithm, which is
282
00:15:39,180 --> 00:15:44,770
a very efficient decoder for
any finite state system
283
00:15:44,770 --> 00:15:48,140
observed in memoryless noise,
which is what we have here.
284
00:15:48,140 --> 00:15:51,810
Hidden Markov model, as
it's sometimes called.
285
00:15:51,810 --> 00:15:52,310
All right.
286
00:15:52,310 --> 00:15:58,110
So we're going to interplay
these two types of structure.
287
00:15:58,110 --> 00:16:09,360
And I feel it's really the fact
that convolutional codes
288
00:16:09,360 --> 00:16:12,610
have these two kinds of
structure that makes them
289
00:16:12,610 --> 00:16:13,750
better than block codes.
290
00:16:13,750 --> 00:16:18,710
They have just the amount of
algebraic structure that we
291
00:16:18,710 --> 00:16:20,650
want, namely, the linear
structure.
292
00:16:20,650 --> 00:16:22,440
They don't have too
much algebraic
293
00:16:22,440 --> 00:16:24,750
structure beyond that.
294
00:16:24,750 --> 00:16:29,240
We don't have elaborate roots
of polynomials and so forth,
295
00:16:29,240 --> 00:16:31,290
as we do with Reed-Solomon
codes.
296
00:16:31,290 --> 00:16:34,310
We have a pretty modest
algebraic structure here.
297
00:16:34,310 --> 00:16:39,970
And it's sort of different in
character than the algebraic
298
00:16:39,970 --> 00:16:42,870
structure Reed-Solomon codes.
299
00:16:42,870 --> 00:16:45,810
And it's married to this
finite state dynamical
300
00:16:45,810 --> 00:16:48,740
structure, which is what
allows us to have
301
00:16:48,740 --> 00:16:51,580
low-complexity decoding.
302
00:16:51,580 --> 00:16:54,690
So that's kind of the magic
of convolutional codes.
303
00:16:54,690 --> 00:16:57,400
Not that they're very magic.
304
00:16:57,400 --> 00:16:57,680
All right.
305
00:16:57,680 --> 00:16:59,235
This is a convolutional
encoder.
306
00:16:59,235 --> 00:17:04,470
307
00:17:04,470 --> 00:17:07,530
What is a convolutional code?
308
00:17:07,530 --> 00:17:20,050
The convolutional code is
roughly the set of all
309
00:17:20,050 --> 00:17:21,465
possible output sequences.
310
00:17:21,465 --> 00:17:27,160
311
00:17:27,160 --> 00:17:29,680
And this is where we're going
to measure the minimum
312
00:17:29,680 --> 00:17:32,010
distance of the code, minimum
free distance.
313
00:17:32,010 --> 00:17:34,970
What's the minimum Hamming
distance between any two
314
00:17:34,970 --> 00:17:36,547
possible output sequences?
315
00:17:36,547 --> 00:17:39,290
316
00:17:39,290 --> 00:17:46,810
And in the notes, I go into
considerable detail to explain
317
00:17:46,810 --> 00:17:51,240
how we choose the particular
definition of all.
318
00:17:51,240 --> 00:17:52,060
What do we mean?
319
00:17:52,060 --> 00:17:55,300
Do we mean the set of all
possible output sequences if
320
00:17:55,300 --> 00:17:57,750
we put in a finite
input sequence?
321
00:17:57,750 --> 00:18:00,870
An input sequences that starts
at time 0, and this
322
00:18:00,870 --> 00:18:04,410
polynomial, let's say, that
ends at some finite time?
323
00:18:04,410 --> 00:18:08,800
Are we going to allow
semi-infinite, bi-infinite
324
00:18:08,800 --> 00:18:10,900
input sequences?
325
00:18:10,900 --> 00:18:14,650
So you see, there's some choices
to be made here.
326
00:18:14,650 --> 00:18:19,240
And actually, it's of some
importance that we make the
327
00:18:19,240 --> 00:18:25,234
right choice to get proper
insight into these codes.
328
00:18:25,234 --> 00:18:26,484
AUDIENCE: [INAUDIBLE]
329
00:18:26,484 --> 00:18:30,440
330
00:18:30,440 --> 00:18:33,610
You have to choose the
initial state, then?
331
00:18:33,610 --> 00:18:34,330
PROFESSOR: Yeah, OK.
332
00:18:34,330 --> 00:18:38,530
You know from linear systems
that the output isn't even
333
00:18:38,530 --> 00:18:42,350
well-defined unless you say
what the initial state is.
334
00:18:42,350 --> 00:18:44,800
And in general, in linear
systems, we say, well, we'll
335
00:18:44,800 --> 00:18:47,880
assume it starts out in
the all-zero state.
336
00:18:47,880 --> 00:18:51,620
And that's basically the choice
I'm going to make,
337
00:18:51,620 --> 00:18:54,731
except I'm going to do
it in a couple steps.
338
00:18:54,731 --> 00:18:57,370
But that's the right question.
339
00:18:57,370 --> 00:18:59,300
How do we initialize
the system?
340
00:18:59,300 --> 00:19:03,260
That's a part of the
specification of this encoder.
341
00:19:03,260 --> 00:19:07,180
And of course we initialize
it in the all-zero state.
342
00:19:07,180 --> 00:19:11,640
But for that reason, in a
general system, we can't allow
343
00:19:11,640 --> 00:19:15,240
bi-infinite inputs.
344
00:19:15,240 --> 00:19:18,960
Because if we don't know what
the starting time is, we don't
345
00:19:18,960 --> 00:19:23,730
know how to define the
starting state.
346
00:19:23,730 --> 00:19:27,770
Again, I'm speaking
very roughly.
347
00:19:27,770 --> 00:19:29,560
So let me speak a little
bit more precisely.
348
00:19:29,560 --> 00:19:32,390
349
00:19:32,390 --> 00:19:38,720
We're going to define the
set of all Laurent.
350
00:19:38,720 --> 00:19:42,240
351
00:19:42,240 --> 00:19:53,630
Which means semi-infinite
sequences
352
00:19:53,630 --> 00:19:59,820
over the binary field.
353
00:19:59,820 --> 00:20:12,670
And this is called F2 of D. And
this means a sequence --
354
00:20:12,670 --> 00:20:14,160
U, say --
355
00:20:14,160 --> 00:20:29,930
which is only a finite
number of non-zero
356
00:20:29,930 --> 00:20:34,990
terms before time 0.
357
00:20:34,990 --> 00:20:40,130
358
00:20:40,130 --> 00:20:43,360
In other words, it can start at
any negative time, but it
359
00:20:43,360 --> 00:20:47,260
has to have a definite
starting time.
360
00:20:47,260 --> 00:20:52,790
I'm not talking about the set of
all power series or the set
361
00:20:52,790 --> 00:20:56,650
of all sequences that start
at time 0 or later.
362
00:20:56,650 --> 00:20:58,840
The sequence can start
at any time.
363
00:20:58,840 --> 00:21:01,020
But it does have to have a
definite starting time, and
364
00:21:01,020 --> 00:21:03,250
before that, it has
to be all 0.
365
00:21:03,250 --> 00:21:05,960
366
00:21:05,960 --> 00:21:14,130
So that for instance,
the sequence that --
367
00:21:14,130 --> 00:21:16,390
I haven't introduced
D transforms yet.
368
00:21:16,390 --> 00:21:18,520
Maybe I should do that
immediately.
369
00:21:18,520 --> 00:21:23,780
But a sequence which, at minus
5, minus 4, minus 3, minus 2,
370
00:21:23,780 --> 00:21:25,130
minus 1, 0 --
371
00:21:25,130 --> 00:21:26,870
these are time indices --
372
00:21:26,870 --> 00:21:34,660
looks like 1 0 1 0 0 1 0, and
goes on indefinitely in the
373
00:21:34,660 --> 00:21:38,140
future, but has all
zeros back here --
374
00:21:38,140 --> 00:21:41,500
that is a legitimate
Laurent sequence.
375
00:21:41,500 --> 00:21:46,600
Whereas a sequence that's dot
dot dot 1 1 1 1 1 1 1 1 1 and
376
00:21:46,600 --> 00:21:50,560
so forth, all ones forever, is
not, because it doesn't have a
377
00:21:50,560 --> 00:21:51,811
definite starting time.
378
00:21:51,811 --> 00:21:59,960
379
00:21:59,960 --> 00:22:00,830
And I sort of --
380
00:22:00,830 --> 00:22:02,120
all right in this notation --
381
00:22:02,120 --> 00:22:05,790
I need to introduce
D transforms.
382
00:22:05,790 --> 00:22:15,370
The D transform of a sequence U
which has components Uk for
383
00:22:15,370 --> 00:22:23,480
k and z it is simply the
sum of Uk D to the k.
384
00:22:23,480 --> 00:22:29,310
Let me write that as U of
D. For all k and z.
385
00:22:29,310 --> 00:22:33,590
386
00:22:33,590 --> 00:22:35,640
This is simply a generating
function.
387
00:22:35,640 --> 00:22:38,290
388
00:22:38,290 --> 00:22:42,500
It looks very much like the Z
transform that you see in
389
00:22:42,500 --> 00:22:46,110
discrete time linear
filter theory.
390
00:22:46,110 --> 00:22:52,070
We would write for something
up here, well --
391
00:22:52,070 --> 00:22:57,180
So a sequence that is 1
at time 0, 0 at time
392
00:22:57,180 --> 00:22:59,280
1, 1 at time 2 --
393
00:22:59,280 --> 00:23:03,210
if U is equal to that, and 0 at
all other times, we would
394
00:23:03,210 --> 00:23:08,550
just write U of D equals
1 plus D squared.
395
00:23:08,550 --> 00:23:12,430
So it's a generating function.
396
00:23:12,430 --> 00:23:15,560
The D just gives us a location
of where the ones are.
397
00:23:15,560 --> 00:23:19,256
It's easier to write this
than this, frankly.
398
00:23:19,256 --> 00:23:22,750
It's actually hard to write
the sequence which is 1 at
399
00:23:22,750 --> 00:23:27,750
time k and 0 at all other times
in this notation, but
400
00:23:27,750 --> 00:23:30,215
it's simply D to the
k in this notation.
401
00:23:30,215 --> 00:23:32,880
402
00:23:32,880 --> 00:23:36,340
D here is algebraically simply
an indeterminate, or a
403
00:23:36,340 --> 00:23:37,860
placeholder.
404
00:23:37,860 --> 00:23:40,800
And this is the subtle
difference from Z transforms.
405
00:23:40,800 --> 00:23:45,070
The Z transforms, we would write
U of Z minus one equals
406
00:23:45,070 --> 00:23:48,960
1 plus Z minus 2 for something
like this.
407
00:23:48,960 --> 00:23:52,860
There's this Z to the minus 1
convention in Z transforms.
408
00:23:52,860 --> 00:23:54,490
That's not the important
difference.
409
00:23:54,490 --> 00:23:57,200
We could clearly use D to the
minus 1 rather than D. The
410
00:23:57,200 --> 00:24:00,382
important difference here is
that Z is usually imagined to
411
00:24:00,382 --> 00:24:03,610
have values in the
complex plane.
412
00:24:03,610 --> 00:24:06,340
We look for it's poles and
zeros in the complex
413
00:24:06,340 --> 00:24:08,380
plane and so forth.
414
00:24:08,380 --> 00:24:11,960
And so the Z transform really
is a transform.
415
00:24:11,960 --> 00:24:14,510
It takes you into the
frequency domain.
416
00:24:14,510 --> 00:24:20,540
If you make z equals e to the
j omega t or something, then
417
00:24:20,540 --> 00:24:22,660
you're in the frequency
domain.
418
00:24:22,660 --> 00:24:25,950
Here, this is still a time
domain expression.
419
00:24:25,950 --> 00:24:29,250
Just as a convenient, compact,
generating function notation
420
00:24:29,250 --> 00:24:32,300
for a particular sequence.
421
00:24:32,300 --> 00:24:36,770
So that's when I say the set
of all semi-infinite
422
00:24:36,770 --> 00:24:39,290
sequences, now I can write --
423
00:24:39,290 --> 00:24:43,410
a Laurent sequence always
looks like this.
424
00:24:43,410 --> 00:24:49,680
It starts off at some
particular time.
425
00:24:49,680 --> 00:24:57,165
Let's say the first term is Ud D
to the d plus Ud plus 1 D to
426
00:24:57,165 --> 00:24:59,840
the d plus 1 plus so forth.
427
00:24:59,840 --> 00:25:01,520
It always looks like that.
428
00:25:01,520 --> 00:25:04,980
So it has a definite starting
time d, which
429
00:25:04,980 --> 00:25:08,950
is called the delay.
430
00:25:08,950 --> 00:25:12,995
The delay of u of D is equal
to d, in this case.
431
00:25:12,995 --> 00:25:15,590
432
00:25:15,590 --> 00:25:20,070
And d can be any integer.
433
00:25:20,070 --> 00:25:24,160
It can start at a negative time
or at a positive time.
434
00:25:24,160 --> 00:25:28,120
We don't require it to start
at time 0 or later.
435
00:25:28,120 --> 00:25:31,210
But nonetheless, the sequence
has to look like this.
436
00:25:31,210 --> 00:25:35,880
437
00:25:35,880 --> 00:25:42,190
And we indicate the set of
all such sequences here.
438
00:25:42,190 --> 00:25:45,270
One of the immediate
observations that you can make
439
00:25:45,270 --> 00:25:49,490
is that this is a time
invariant set.
440
00:25:49,490 --> 00:25:52,440
Which the set of all sequences
that start at
441
00:25:52,440 --> 00:25:55,700
0 or later is not.
442
00:25:55,700 --> 00:25:58,030
Set of all sequences that start
at 0 or later has a
443
00:25:58,030 --> 00:26:04,470
definite time in it, and a set
is time-invariant if D times
444
00:26:04,470 --> 00:26:06,300
the set is equal to the set.
445
00:26:06,300 --> 00:26:12,410
So what is D times F2 of D?
446
00:26:12,410 --> 00:26:15,432
I claim it's just F2 of
D all over again.
447
00:26:15,432 --> 00:26:20,610
And in fact, D to the k for
any k is equal to F2 of D.
448
00:26:20,610 --> 00:26:24,706
So the set of all Laurent
sequences is time-invariant.
449
00:26:24,706 --> 00:26:28,400
Whereas the set of all power
series in D, that's sequences
450
00:26:28,400 --> 00:26:31,770
that have delay 0 or greater,
is not time invariant.
451
00:26:31,770 --> 00:26:34,390
It does not satisfy this.
452
00:26:34,390 --> 00:26:38,570
453
00:26:38,570 --> 00:26:41,520
Chew on that.
454
00:26:41,520 --> 00:26:44,900
An important consequence of this
is that the set of all
455
00:26:44,900 --> 00:26:47,035
Laurent sequences
forms a field.
456
00:26:47,035 --> 00:26:54,150
457
00:26:54,150 --> 00:26:57,510
Let's say what we need
to prove for a field.
458
00:26:57,510 --> 00:27:01,290
Can we add Laurent sequences?
459
00:27:01,290 --> 00:27:04,175
Yes, we simply add them in the
usual way, component-wise.
460
00:27:04,175 --> 00:27:07,130
461
00:27:07,130 --> 00:27:10,540
Same way as you do in
polynomial addition.
462
00:27:10,540 --> 00:27:14,900
You just gather together
compounds at time k.
463
00:27:14,900 --> 00:27:16,775
Can you multiply Laurent
sequences?
464
00:27:16,775 --> 00:27:20,780
465
00:27:20,780 --> 00:27:23,970
You can certainly multiply
polynomials.
466
00:27:23,970 --> 00:27:29,330
1 plus D times one plus D is
1 plus D squared over F2.
467
00:27:29,330 --> 00:27:31,055
Can you multiply Laurent
sequences?
468
00:27:31,055 --> 00:27:33,830
469
00:27:33,830 --> 00:27:35,220
Well, yes.
470
00:27:35,220 --> 00:27:43,750
We can define multiplication by
convolution as we do with
471
00:27:43,750 --> 00:27:45,000
polynomials.
472
00:27:45,000 --> 00:27:51,290
473
00:27:51,290 --> 00:28:01,220
In other words, U of D times V
of D is the sequence W of D,
474
00:28:01,220 --> 00:28:10,750
where Wk is the sum over k
prime in Z of Uk prime Vk
475
00:28:10,750 --> 00:28:12,000
minus k prime.
476
00:28:12,000 --> 00:28:14,560
477
00:28:14,560 --> 00:28:17,690
Now what's the potential problem
that can arise when
478
00:28:17,690 --> 00:28:20,650
you define multiplication
by convolution?
479
00:28:20,650 --> 00:28:25,210
If any of these terms are
infinite, we have no notion of
480
00:28:25,210 --> 00:28:27,670
convergence over F2.
481
00:28:27,670 --> 00:28:31,065
So we really have no way of
defining an infinite sum.
482
00:28:31,065 --> 00:28:34,340
An infinite sum of elements
in F2 is, in general, not
483
00:28:34,340 --> 00:28:35,590
well-defined.
484
00:28:35,590 --> 00:28:37,130
485
00:28:37,130 --> 00:28:41,870
But from the fact that each of
these starts at one time --
486
00:28:41,870 --> 00:28:43,360
this is basically multiplying
487
00:28:43,360 --> 00:28:45,390
something by the time reversal.
488
00:28:45,390 --> 00:28:48,890
We only have a finite number of
elements in any such sum.
489
00:28:48,890 --> 00:28:54,240
So that's the real reason
for using the set
490
00:28:54,240 --> 00:28:55,620
of all Laurent sequences.
491
00:28:55,620 --> 00:28:58,120
If we had the set of all
bi-infinite sequences, then
492
00:28:58,120 --> 00:29:00,650
multiplication would not
be well-defined.
493
00:29:00,650 --> 00:29:03,580
But by insisting that they
have a starting time, we
494
00:29:03,580 --> 00:29:06,390
ensure that convolution is
always well-defined.
495
00:29:06,390 --> 00:29:09,300
Therefore, multiplication
of Laurent sequences is
496
00:29:09,300 --> 00:29:10,550
well-defined.
497
00:29:10,550 --> 00:29:12,830
498
00:29:12,830 --> 00:29:13,550
OK.
499
00:29:13,550 --> 00:29:16,386
So we have addition, we
have multiplication.
500
00:29:16,386 --> 00:29:19,390
501
00:29:19,390 --> 00:29:23,660
Maybe I'll go over here.
502
00:29:23,660 --> 00:29:25,110
What else do we need
for a field?
503
00:29:25,110 --> 00:29:27,650
504
00:29:27,650 --> 00:29:31,310
So far we just have a ring.
505
00:29:31,310 --> 00:29:32,400
Inverses, yeah.
506
00:29:32,400 --> 00:29:34,600
Let's directly check inverses.
507
00:29:34,600 --> 00:29:37,780
508
00:29:37,780 --> 00:29:43,391
So suppose I have some Laurent
sequence U of D equals U(d), D
509
00:29:43,391 --> 00:29:46,635
to the d, plus so forth.
510
00:29:46,635 --> 00:29:48,085
Does that always have
an inverse?
511
00:29:48,085 --> 00:30:00,370
512
00:30:00,370 --> 00:30:04,810
In other words, we want to find
something 1 over U of D
513
00:30:04,810 --> 00:30:07,880
equals what?
514
00:30:07,880 --> 00:30:14,720
Well, let's just divide U of D
into 1 by long division, and
515
00:30:14,720 --> 00:30:16,990
we can get an inverse.
516
00:30:16,990 --> 00:30:18,520
Let me give you an example.
517
00:30:18,520 --> 00:30:22,250
Suppose U of D is equal
to 1 plus D. What's
518
00:30:22,250 --> 00:30:24,810
1 over 1 plus D?
519
00:30:24,810 --> 00:30:30,100
We take 1 plus D, we divide
it into 1, we get 1.
520
00:30:30,100 --> 00:30:34,610
1 plus D, D plus D,
D plus D squared.
521
00:30:34,610 --> 00:30:37,330
522
00:30:37,330 --> 00:30:38,580
And so forth.
523
00:30:38,580 --> 00:30:42,100
524
00:30:42,100 --> 00:30:46,170
So we get a semi-infinite
sequence.
525
00:30:46,170 --> 00:30:47,910
Not a polynomial.
526
00:30:47,910 --> 00:30:52,150
But nonetheless, it's a
legitimate Laurent sequence.
527
00:30:52,150 --> 00:30:55,440
It's actually a very simple
periodic sequence of period 1.
528
00:30:55,440 --> 00:30:58,340
529
00:30:58,340 --> 00:31:01,170
Are you with me on this?
530
00:31:01,170 --> 00:31:05,870
Could you find the inverse of
any sequence that starts at
531
00:31:05,870 --> 00:31:09,170
time 0, let's say?
532
00:31:09,170 --> 00:31:11,600
If you can do that, you can
certainly find the inverse of
533
00:31:11,600 --> 00:31:14,790
any sequence that starts at time
D. You just multiply by D
534
00:31:14,790 --> 00:31:16,040
to the minus d.
535
00:31:16,040 --> 00:31:19,680
536
00:31:19,680 --> 00:31:42,010
So every non-zero Laurent
sequence has an inverse.
537
00:31:42,010 --> 00:31:44,710
538
00:31:44,710 --> 00:31:48,240
1 over U of D, let's say.
539
00:31:48,240 --> 00:31:50,040
Which is also a Laurent
sequence.
540
00:31:50,040 --> 00:31:53,080
541
00:31:53,080 --> 00:31:55,075
Of course, as always, we
can't divide by 0.
542
00:31:55,075 --> 00:31:59,950
543
00:31:59,950 --> 00:32:02,320
But that's the definition
of a field.
544
00:32:02,320 --> 00:32:02,960
That doesn't matter.
545
00:32:02,960 --> 00:32:05,400
We have inverses.
546
00:32:05,400 --> 00:32:09,270
We checked the associative,
547
00:32:09,270 --> 00:32:11,510
distributive, commutative laws.
548
00:32:11,510 --> 00:32:12,220
They all work.
549
00:32:12,220 --> 00:32:16,270
So this is actually a field.
550
00:32:16,270 --> 00:32:16,700
Yes?
551
00:32:16,700 --> 00:32:17,686
AUDIENCE: [INAUDIBLE]
552
00:32:17,686 --> 00:32:18,936
multiplication [INAUDIBLE].
553
00:32:18,936 --> 00:32:22,616
554
00:32:22,616 --> 00:32:24,500
PROFESSOR: Yeah.
555
00:32:24,500 --> 00:32:28,200
The Laurent sequences certainly
include polynomials,
556
00:32:28,200 --> 00:32:33,150
and this is consistent with
polynomial multiplication.
557
00:32:33,150 --> 00:32:36,720
So it's just extending
polynomial multiplication
558
00:32:36,720 --> 00:32:41,000
really as far as we can to these
semi-infinite sequences
559
00:32:41,000 --> 00:32:44,620
that are semi-infinite in the
future, but not in the past.
560
00:32:44,620 --> 00:32:48,640
They're infinite in the future,
not in the past.
561
00:32:48,640 --> 00:32:50,244
AUDIENCE: I think that's
[INAUDIBLE] infinite sequence
562
00:32:50,244 --> 00:32:53,175
of [INAUDIBLE].
563
00:32:53,175 --> 00:32:54,425
[UNINTELLIGIBLE].
564
00:32:54,425 --> 00:32:59,140
565
00:32:59,140 --> 00:33:00,390
I mean suppose you --
566
00:33:00,390 --> 00:33:04,600
567
00:33:04,600 --> 00:33:06,860
To find the inverse we just have
to keep solving the set
568
00:33:06,860 --> 00:33:09,860
of linear equations, right?
569
00:33:09,860 --> 00:33:12,916
From the first equation, we get
the first one, from the
570
00:33:12,916 --> 00:33:14,166
second one, by substitution
we get the next one.
571
00:33:14,166 --> 00:33:16,650
572
00:33:16,650 --> 00:33:17,790
PROFESSOR: Yeah.
573
00:33:17,790 --> 00:33:20,470
It's long division, which is
the same as the Euclidean
574
00:33:20,470 --> 00:33:22,980
division algorithm.
575
00:33:22,980 --> 00:33:29,170
We just want to find a
coefficient here that reduces
576
00:33:29,170 --> 00:33:35,610
the remainder to start at time
1 or later, to have delay 1.
577
00:33:35,610 --> 00:33:38,610
We want to find the next
coefficient to make the
578
00:33:38,610 --> 00:33:39,890
remainder have delay 2.
579
00:33:39,890 --> 00:33:41,540
And we can always continue
this, ad infinitum.
580
00:33:41,540 --> 00:33:44,970
581
00:33:44,970 --> 00:33:46,220
AUDIENCE: [INAUDIBLE]
582
00:33:46,220 --> 00:33:52,020
583
00:33:52,020 --> 00:33:53,840
PROFESSOR: These are
semi-infinite sequences.
584
00:33:53,840 --> 00:33:56,470
Do you mean sequences that
start at time 0 or later,
585
00:33:56,470 --> 00:33:59,110
which are called formal
power series?
586
00:33:59,110 --> 00:33:59,990
Again, there are some --
587
00:33:59,990 --> 00:34:01,240
AUDIENCE: [INAUDIBLE]
588
00:34:01,240 --> 00:34:03,660
589
00:34:03,660 --> 00:34:04,130
PROFESSOR: Oh!
590
00:34:04,130 --> 00:34:05,030
Sorry, yes.
591
00:34:05,030 --> 00:34:07,940
Of course, we need
to check that.
592
00:34:07,940 --> 00:34:10,870
But you can see, it
doesn't matter.
593
00:34:10,870 --> 00:34:16,280
We could divide by 1 plus D plus
D squared plus so forth,
594
00:34:16,280 --> 00:34:18,389
and the algorithm is the same.
595
00:34:18,389 --> 00:34:22,682
In that case, of course, if we
did that, we'd get 1 plus D
596
00:34:22,682 --> 00:34:25,250
plus D squared, plus so forth.
597
00:34:25,250 --> 00:34:27,880
D plus D squared,
plus so forth.
598
00:34:27,880 --> 00:34:28,780
And what do you know?
599
00:34:28,780 --> 00:34:32,710
D times that is that,
and we get 0, and it
600
00:34:32,710 --> 00:34:35,170
terminates after 1.
601
00:34:35,170 --> 00:34:36,120
So yeah.
602
00:34:36,120 --> 00:34:36,610
Thank you.
603
00:34:36,610 --> 00:34:38,530
Long division also works.
604
00:34:38,530 --> 00:34:46,159
If we divide by any Laurent
sequence, it's the same.
605
00:34:46,159 --> 00:34:49,060
The same algorithm.
606
00:34:49,060 --> 00:34:50,310
Excellent.
607
00:34:50,310 --> 00:34:52,610
608
00:34:52,610 --> 00:34:53,300
OK.
609
00:34:53,300 --> 00:35:03,270
So every Laurent sequence
has an inverse.
610
00:35:03,270 --> 00:35:06,010
In general, what do the
inverses of polynomial
611
00:35:06,010 --> 00:35:07,260
sequences look like?
612
00:35:07,260 --> 00:35:15,100
613
00:35:15,100 --> 00:35:19,750
Inverse of a polynomial
sequence.
614
00:35:19,750 --> 00:35:25,750
615
00:35:25,750 --> 00:35:30,060
Can anyone guess what its
special property is?
616
00:35:30,060 --> 00:35:33,430
What is a polynomial sequence,
first of all?
617
00:35:33,430 --> 00:35:36,290
I say a sequence is finite if it
only has a finite number of
618
00:35:36,290 --> 00:35:37,890
non-zero terms.
619
00:35:37,890 --> 00:35:41,000
I'm going to say it's polynomial
if it's finite and
620
00:35:41,000 --> 00:35:42,410
also causal.
621
00:35:42,410 --> 00:35:45,736
Causal means it starts
at time 0 or later.
622
00:35:45,736 --> 00:35:47,990
Its delay is non-negative.
623
00:35:47,990 --> 00:35:50,840
624
00:35:50,840 --> 00:35:51,210
All right.
625
00:35:51,210 --> 00:35:55,120
So that's the polynomials that
we're already familiar with.
626
00:35:55,120 --> 00:35:58,460
What does the inverse of a
polynomial sequence look like?
627
00:35:58,460 --> 00:35:59,710
AUDIENCE: [INAUDIBLE]
628
00:35:59,710 --> 00:36:02,230
629
00:36:02,230 --> 00:36:05,230
PROFESSOR: It's always going
to be infinite, right?
630
00:36:05,230 --> 00:36:07,510
Unless it's 1 or something.
631
00:36:07,510 --> 00:36:12,970
So unless it's an invertible
unit in the polynomials, it's
632
00:36:12,970 --> 00:36:17,740
going to be an infinite
sequence.
633
00:36:17,740 --> 00:36:18,990
But it's going to
have a property.
634
00:36:18,990 --> 00:36:22,730
635
00:36:22,730 --> 00:36:25,740
Nobody can guess what the
property's going to be?
636
00:36:25,740 --> 00:36:28,070
I already mentioned it once.
637
00:36:28,070 --> 00:36:28,430
Periodic!
638
00:36:28,430 --> 00:36:29,680
Right.
639
00:36:29,680 --> 00:36:35,560
640
00:36:35,560 --> 00:36:37,920
And this is an if and
only if statement.
641
00:36:37,920 --> 00:36:46,220
642
00:36:46,220 --> 00:36:49,520
I guess I have to elaborate
a little bit, but roughly,
643
00:36:49,520 --> 00:36:52,090
that's correct.
644
00:36:52,090 --> 00:36:54,455
This is going to be one of
your homework problems.
645
00:36:54,455 --> 00:36:57,590
646
00:36:57,590 --> 00:37:00,780
Maybe I'm jumping ahead a
little, but let me --
647
00:37:00,780 --> 00:37:04,270
there are a couple of
ways of seeing this.
648
00:37:04,270 --> 00:37:06,640
One is to use some of
the algebra we did
649
00:37:06,640 --> 00:37:07,750
back in chapter seven.
650
00:37:07,750 --> 00:37:10,020
But since we didn't
really complete
651
00:37:10,020 --> 00:37:11,380
that, I won't do that.
652
00:37:11,380 --> 00:37:16,390
A second one is to see within
this long division operation,
653
00:37:16,390 --> 00:37:20,480
again, if I'm dividing by a
polynomial, there's only a
654
00:37:20,480 --> 00:37:25,540
finite set of possible
remainders so I can get up to
655
00:37:25,540 --> 00:37:31,030
shifts by multiples of D. So
if I see the same remainder
656
00:37:31,030 --> 00:37:35,900
again, if I see 1, D, and then
D squared, of course the
657
00:37:35,900 --> 00:37:38,280
series is going to have to
continue in the same way
658
00:37:38,280 --> 00:37:38,560
necessarily.
659
00:37:38,560 --> 00:37:41,010
Since there are only a finite
number of remainders, it's got
660
00:37:41,010 --> 00:37:43,200
to repeat at some point.
661
00:37:43,200 --> 00:37:45,250
Well, a way that I can do
it which is more in
662
00:37:45,250 --> 00:37:48,110
the spirit of this.
663
00:37:48,110 --> 00:38:01,290
Suppose I consider the impulse
response of a system with
664
00:38:01,290 --> 00:38:09,500
feedback, which is set up so the
impulse response is equal
665
00:38:09,500 --> 00:38:13,080
to, in this case, say, 1
over D plus D squared.
666
00:38:13,080 --> 00:38:16,850
667
00:38:16,850 --> 00:38:19,130
You see that the impulse
response of this is going to
668
00:38:19,130 --> 00:38:21,520
be 1 over 1 plus D
plus D squared.
669
00:38:21,520 --> 00:38:29,460
If I put in a 1, time 1, then
next time, I'm going to get a
670
00:38:29,460 --> 00:38:33,170
1 in here, going to get a
1 feeding back there.
671
00:38:33,170 --> 00:38:35,630
Well anyway, I believe
that's right.
672
00:38:35,630 --> 00:38:37,820
This gives an impulse response
of 1 over --
673
00:38:37,820 --> 00:38:41,930
if it doesn't, then readjust
it so it does.
674
00:38:41,930 --> 00:38:47,250
Now, after time 0, the
input is always 0.
675
00:38:47,250 --> 00:38:53,470
So I just have an autonomous
system which is producing the
676
00:38:53,470 --> 00:38:58,180
terms at time 1 and later of 1
over 1 plus D plus D squared.
677
00:38:58,180 --> 00:39:02,270
And again, it's a finite
state system.
678
00:39:02,270 --> 00:39:08,220
Because for over F2, there are
only 4 states in the system.
679
00:39:08,220 --> 00:39:11,100
So what could its state
transition diagram
680
00:39:11,100 --> 00:39:12,110
possibly look like?
681
00:39:12,110 --> 00:39:14,760
First of all, the 0
state always goes
682
00:39:14,760 --> 00:39:17,590
around to the 0 state.
683
00:39:17,590 --> 00:39:22,200
So the other states are
interconnected in some way.
684
00:39:22,200 --> 00:39:25,245
And at best, they're always
going to cycle.
685
00:39:25,245 --> 00:39:28,400
686
00:39:28,400 --> 00:39:35,380
So this is a quick proof
that 1 over 1 plus D
687
00:39:35,380 --> 00:39:36,340
plus D squared --
688
00:39:36,340 --> 00:39:39,970
in fact, any polynomial
inverse --
689
00:39:39,970 --> 00:39:41,690
is going to give you a
periodic response.
690
00:39:41,690 --> 00:39:45,730
691
00:39:45,730 --> 00:39:48,190
What I should really say is --
692
00:39:48,190 --> 00:39:53,310
let me define a rational
Laurent sequence.
693
00:39:53,310 --> 00:39:56,320
I'll leave out the Laurent.
694
00:39:56,320 --> 00:40:01,470
A rational sequence is something
of the form n of D
695
00:40:01,470 --> 00:40:09,960
over d of D where these are both
polynomial, or they're
696
00:40:09,960 --> 00:40:11,440
actually finite.
697
00:40:11,440 --> 00:40:15,386
Let's say they're
both polynomial.
698
00:40:15,386 --> 00:40:22,050
699
00:40:22,050 --> 00:40:24,950
You can reduce it to lowest
terms, if you like.
700
00:40:24,950 --> 00:40:30,780
Actually, I should make this
finite up here, so that it can
701
00:40:30,780 --> 00:40:34,380
start at time before time 0.
702
00:40:34,380 --> 00:40:42,830
And I have to have d of
D not equal to 0.
703
00:40:42,830 --> 00:40:47,330
And in fact, I generally require
d0, the first term
704
00:40:47,330 --> 00:40:48,300
down here, to be 1.
705
00:40:48,300 --> 00:40:54,010
So this is a polynomial with
time 0 term equal to 1.
706
00:40:54,010 --> 00:40:57,230
Like 1 plus D plus D squared.
707
00:40:57,230 --> 00:41:02,605
So a rational sequence is one
that looks like that.
708
00:41:02,605 --> 00:41:05,940
709
00:41:05,940 --> 00:41:12,830
We can generate a rational
sequence by feeding n of D
710
00:41:12,830 --> 00:41:17,520
into a system that has response
1 over d of D. And
711
00:41:17,520 --> 00:41:19,888
the output --
712
00:41:19,888 --> 00:41:22,150
I forget where you take
the output off here.
713
00:41:22,150 --> 00:41:24,290
Probably here.
714
00:41:24,290 --> 00:41:25,820
This will be n of D
715
00:41:25,820 --> 00:41:30,980
over d of D. OK.
716
00:41:30,980 --> 00:41:34,150
717
00:41:34,150 --> 00:41:35,470
There's a linear system.
718
00:41:35,470 --> 00:41:37,150
Single input, single output.
719
00:41:37,150 --> 00:41:43,060
If it has impulse response, 1
over d of D, then if I put in
720
00:41:43,060 --> 00:41:48,970
the sequence n of D, I'm going
to get out n of D over d of D.
721
00:41:48,970 --> 00:41:52,540
And so by the same finite memory
argument, n of D over d
722
00:41:52,540 --> 00:41:57,110
of D is going to be eventually
periodic.
723
00:41:57,110 --> 00:42:03,150
So a sequence is rational
if and only if
724
00:42:03,150 --> 00:42:06,250
it's eventually periodic.
725
00:42:06,250 --> 00:42:09,080
726
00:42:09,080 --> 00:42:11,290
And again, on the homework, I'm
going to ask you to prove
727
00:42:11,290 --> 00:42:12,960
this more carefully.
728
00:42:12,960 --> 00:42:15,710
You can use this finite memory
argument if you like.
729
00:42:15,710 --> 00:42:18,940
730
00:42:18,940 --> 00:42:21,020
So this should remind
you of something.
731
00:42:21,020 --> 00:42:26,950
This should remind you of real
numbers, integers, ratios of
732
00:42:26,950 --> 00:42:30,490
integers, which are rational
real numbers, all
733
00:42:30,490 --> 00:42:33,650
that sort of thing.
734
00:42:33,650 --> 00:42:35,380
What are the analogies here?
735
00:42:35,380 --> 00:42:42,710
736
00:42:42,710 --> 00:42:44,244
First of all, how big is it?
737
00:42:44,244 --> 00:42:47,210
738
00:42:47,210 --> 00:42:50,190
How many real numbers
are there?
739
00:42:50,190 --> 00:42:51,440
It's uncountably infinite,
right?
740
00:42:51,440 --> 00:42:54,460
741
00:42:54,460 --> 00:42:59,610
How many Laurent sequences
are there?
742
00:42:59,610 --> 00:43:02,450
743
00:43:02,450 --> 00:43:06,980
You can I think quickly convince
yourself that it's
744
00:43:06,980 --> 00:43:08,230
uncountably infinite.
745
00:43:08,230 --> 00:43:10,590
746
00:43:10,590 --> 00:43:15,820
Now, we have a special
subset of the real
747
00:43:15,820 --> 00:43:18,270
numbers called the integers.
748
00:43:18,270 --> 00:43:19,650
Oh.
749
00:43:19,650 --> 00:43:22,920
Let's think of this as a decimal
expansion, by the way.
750
00:43:22,920 --> 00:43:32,720
So we have a d, a d minus 1,
and so forth, down to a0
751
00:43:32,720 --> 00:43:35,440
decimal point a minus 1.
752
00:43:35,440 --> 00:43:38,255
These are the coefficients
of decimal expansion.
753
00:43:38,255 --> 00:43:40,970
754
00:43:40,970 --> 00:43:43,840
There's something interesting
here.
755
00:43:43,840 --> 00:43:47,610
Implicitly we always assume
there are only a finite number
756
00:43:47,610 --> 00:43:53,770
of coefficients in the decimal
expansion above the decimal
757
00:43:53,770 --> 00:43:54,920
point, right?
758
00:43:54,920 --> 00:43:58,130
There can be an infinite
number going this way.
759
00:43:58,130 --> 00:44:03,140
So that's completely analogous
to what we have here.
760
00:44:03,140 --> 00:44:04,880
What are the integers?
761
00:44:04,880 --> 00:44:09,820
The integers are the 1's
that stop at a0.
762
00:44:09,820 --> 00:44:12,540
763
00:44:12,540 --> 00:44:17,150
That are all zero to the right
side of the decimal point.
764
00:44:17,150 --> 00:44:20,770
Over here we have something that
looks like u d, D to the
765
00:44:20,770 --> 00:44:29,660
d plus u d plus 1, D to the
d plus 1, and so forth.
766
00:44:29,660 --> 00:44:34,430
And in here we have a special
subclass called the
767
00:44:34,430 --> 00:44:39,820
polynomials in D. And
what are these?
768
00:44:39,820 --> 00:44:44,400
It's not precisely analogous,
because I should really do it
769
00:44:44,400 --> 00:44:47,180
with Z minus 1, to make it
analogous and expand it in the
770
00:44:47,180 --> 00:44:48,390
other direction.
771
00:44:48,390 --> 00:44:52,010
But these are the ones that have
only a finite number of
772
00:44:52,010 --> 00:45:00,120
coefficients to the right of
the time 0 point, again.
773
00:45:00,120 --> 00:45:04,790
We noticed before that there is
a very close relationship
774
00:45:04,790 --> 00:45:07,830
between the factorization
properties of Z and the
775
00:45:07,830 --> 00:45:10,270
factorization properties
of the polynomials.
776
00:45:10,270 --> 00:45:12,390
They're both principal
ideal domains.
777
00:45:12,390 --> 00:45:15,190
They're both unique
factorization domains.
778
00:45:15,190 --> 00:45:18,505
They're both rings of the
very nicest type.
779
00:45:18,505 --> 00:45:23,680
780
00:45:23,680 --> 00:45:28,890
Then once we have in here
the rational numbers --
781
00:45:28,890 --> 00:45:34,560
that's certainly an important
subset of the reals --
782
00:45:34,560 --> 00:45:37,410
how many rationals are there?
783
00:45:37,410 --> 00:45:42,490
The rationals, this is basically
a ratio of integers.
784
00:45:42,490 --> 00:45:44,750
And there's only a countably
infinite number
785
00:45:44,750 --> 00:45:47,570
of rationals, right?
786
00:45:47,570 --> 00:45:50,490
And what's the distinguishing
characteristic of the
787
00:45:50,490 --> 00:45:55,220
rationals in terms of their
decimal expansion?
788
00:45:55,220 --> 00:45:58,500
It's eventually periodic.
789
00:45:58,500 --> 00:46:05,640
So these correspond to the
rational functions, which are
790
00:46:05,640 --> 00:46:11,520
denoted by this regular
parentheses bracket.
791
00:46:11,520 --> 00:46:14,830
So this is the polynomials.
792
00:46:14,830 --> 00:46:16,725
This is the rational
functions.
793
00:46:16,725 --> 00:46:20,450
794
00:46:20,450 --> 00:46:23,750
And because these are eventually
periodic, or
795
00:46:23,750 --> 00:46:28,870
because all of these can be
written as n of D over d of D,
796
00:46:28,870 --> 00:46:32,990
and both of these are
polynomials, or these are
797
00:46:32,990 --> 00:46:36,440
clearly countably infinite
sets, this is a countably
798
00:46:36,440 --> 00:46:37,210
infinite set.
799
00:46:37,210 --> 00:46:40,821
The set of all rational
Laurentian polynomials.
800
00:46:40,821 --> 00:46:45,540
801
00:46:45,540 --> 00:46:52,100
So again, this is
just mnemonics.
802
00:46:52,100 --> 00:46:53,930
Haven't proved anything.
803
00:46:53,930 --> 00:46:59,580
But the behavior of these things
very closely reminds
804
00:46:59,580 --> 00:47:02,260
you of the behavior of these
things over here.
805
00:47:02,260 --> 00:47:04,500
It's good to keep
this in mind.
806
00:47:04,500 --> 00:47:05,950
One other point.
807
00:47:05,950 --> 00:47:10,270
Suppose I'd allow bi-infinite
sequences.
808
00:47:10,270 --> 00:47:19,202
Then 1 over 1 plus D doesn't
have a definite expansion.
809
00:47:19,202 --> 00:47:21,552
It has two expansions.
810
00:47:21,552 --> 00:47:26,670
1 over 1 plus D, if we require
that the expansion be Laurent,
811
00:47:26,670 --> 00:47:28,175
then we only have this
possibility.
812
00:47:28,175 --> 00:47:30,920
813
00:47:30,920 --> 00:47:33,660
But if it could be bi-infinite
in the other direction, then
814
00:47:33,660 --> 00:47:37,320
you can see that this is also
equal to D minus 1 plus D
815
00:47:37,320 --> 00:47:42,840
minus 2 plus D minus 3 plus
so forth, semi-infinitely.
816
00:47:42,840 --> 00:47:48,910
So another reason we want to
rule out non-Laurent sequences
817
00:47:48,910 --> 00:47:51,020
is to have a unique inverse.
818
00:47:51,020 --> 00:47:53,980
This is just another way of
saying that the Laurent
819
00:47:53,980 --> 00:47:56,230
sequences form a field.
820
00:47:56,230 --> 00:48:02,030
That the bi-infinite sequences
don't have the
821
00:48:02,030 --> 00:48:03,220
multiplicative property.
822
00:48:03,220 --> 00:48:05,890
They're simply a group.
823
00:48:05,890 --> 00:48:09,560
And this is all written
up in the notes.
824
00:48:09,560 --> 00:48:10,110
All right.
825
00:48:10,110 --> 00:48:14,940
So where I want to get
to eventually here --
826
00:48:14,940 --> 00:48:16,190
come back over here.
827
00:48:16,190 --> 00:48:21,900
828
00:48:21,900 --> 00:48:24,490
Let's now analyze this
convolutional encoder.
829
00:48:24,490 --> 00:48:32,420
830
00:48:32,420 --> 00:48:38,730
It's a linear time invariant
circuit.
831
00:48:38,730 --> 00:48:43,080
That means it's characterized
by its impulse response.
832
00:48:43,080 --> 00:48:53,140
833
00:48:53,140 --> 00:48:57,955
Or responses, if you like,
because it has two outputs.
834
00:48:57,955 --> 00:49:00,860
835
00:49:00,860 --> 00:49:07,290
What does y1 of D, if I write
the D transform of y1, what is
836
00:49:07,290 --> 00:49:09,152
that going to equal to?
837
00:49:09,152 --> 00:49:14,060
If I just put in a single
impulse, single 1 at time 0
838
00:49:14,060 --> 00:49:18,690
for u k, what am I going
to get out up there?
839
00:49:18,690 --> 00:49:23,530
I'm going to get out 1 0
1 and then all zeros.
840
00:49:23,530 --> 00:49:26,430
841
00:49:26,430 --> 00:49:36,820
So y1 of D, the impulse response
to steps d1 of D,
842
00:49:36,820 --> 00:49:40,650
equals 1 plus D squared.
843
00:49:40,650 --> 00:49:45,780
That's the impulse response
of the first output.
844
00:49:45,780 --> 00:49:54,020
And y1 of D is simply u of
D times 1 plus D squared,
845
00:49:54,020 --> 00:49:55,270
bi-linearity and time
invariance.
846
00:49:55,270 --> 00:49:58,580
847
00:49:58,580 --> 00:50:02,570
So if I put in 1 plus
D squared, I'll get
848
00:50:02,570 --> 00:50:04,240
out 1 plus D fourth.
849
00:50:04,240 --> 00:50:07,320
850
00:50:07,320 --> 00:50:09,050
Is this too quick?
851
00:50:09,050 --> 00:50:10,500
This is just linear
system theory.
852
00:50:10,500 --> 00:50:13,100
You've all seen this
in Digital Signal
853
00:50:13,100 --> 00:50:16,080
Processing or something.
854
00:50:16,080 --> 00:50:20,390
This is undergraduate stuff,
except for over F2.
855
00:50:20,390 --> 00:50:20,920
OK?
856
00:50:20,920 --> 00:50:23,570
You've just got to check
that it's linear, it's
857
00:50:23,570 --> 00:50:24,700
time-invariant.
858
00:50:24,700 --> 00:50:27,787
Therefore it's characterized
by its impulse response.
859
00:50:27,787 --> 00:50:32,560
And so once you know the impulse
response, you want to
860
00:50:32,560 --> 00:50:35,260
know the response to any
sequence, you convolve that
861
00:50:35,260 --> 00:50:37,490
sequence with the impulse
response.
862
00:50:37,490 --> 00:50:39,636
This is that statement in
D transform notation.
863
00:50:39,636 --> 00:50:42,280
864
00:50:42,280 --> 00:50:42,800
OK?
865
00:50:42,800 --> 00:50:44,050
People happy with that?
866
00:50:44,050 --> 00:50:46,750
867
00:50:46,750 --> 00:50:49,180
I will spend time on it if it's
mysterious, but I don't
868
00:50:49,180 --> 00:50:52,420
think it should be mysterious.
869
00:50:52,420 --> 00:50:52,920
All right.
870
00:50:52,920 --> 00:50:56,040
And let's complete
the picture.
871
00:50:56,040 --> 00:50:59,150
What's the impulse response
for the second
872
00:50:59,150 --> 00:51:02,740
output in that picture?
873
00:51:02,740 --> 00:51:07,250
1 plus D plus D squared,
thank you.
874
00:51:07,250 --> 00:51:14,100
So we have y2 of D is equal
to u of D times 1
875
00:51:14,100 --> 00:51:15,560
plus D plus D squared.
876
00:51:15,560 --> 00:51:21,390
877
00:51:21,390 --> 00:51:26,870
We can aggregate this by simply
saying that a little 2
878
00:51:26,870 --> 00:51:34,690
vector, y of D, is equal to a
1 vector, u of D, times a
879
00:51:34,690 --> 00:51:45,970
little 2 vector, g of D, where
this means y1 of D, y2 of D,
880
00:51:45,970 --> 00:51:52,690
and this means g1 of
D, g2 of D. OK?
881
00:51:52,690 --> 00:51:55,845
So this is a very simple
little matrix equation.
882
00:51:55,845 --> 00:52:05,690
883
00:52:05,690 --> 00:52:09,720
And now the convolutional
code.
884
00:52:09,720 --> 00:52:14,190
This is the output sequence in
response to a particular input
885
00:52:14,190 --> 00:52:17,980
sequence u of D. So we're going
to define the code as
886
00:52:17,980 --> 00:52:23,590
the set of all possible output
sequences when the input
887
00:52:23,590 --> 00:52:25,020
sequences run through what?
888
00:52:25,020 --> 00:52:29,810
889
00:52:29,810 --> 00:52:32,160
Now, with all this elaborate
set up --
890
00:52:32,160 --> 00:52:36,950
so it's the set of all output
sequences y of D equals u of D
891
00:52:36,950 --> 00:52:41,070
g of D as u of D runs
through the set
892
00:52:41,070 --> 00:52:46,100
of all Laurent sequences.
893
00:52:46,100 --> 00:52:49,560
Sequences that start at
some definite time.
894
00:52:49,560 --> 00:52:52,310
And now I've said much more
precisely what I said back at
895
00:52:52,310 --> 00:52:53,330
the beginning.
896
00:52:53,330 --> 00:52:56,140
Because all sequences have a
definite starting time, we
897
00:52:56,140 --> 00:53:01,430
know what the starting state is
for any of these sequences.
898
00:53:01,430 --> 00:53:08,600
Always when the sequence starts
at time D, the system
899
00:53:08,600 --> 00:53:11,830
is quiet, it's in the all-zero
state, and it's driven from
900
00:53:11,830 --> 00:53:13,460
time D onwards.
901
00:53:13,460 --> 00:53:15,890
So this convolution
is well-defined.
902
00:53:15,890 --> 00:53:18,680
If we had an undefined starting
state, then this
903
00:53:18,680 --> 00:53:19,930
would not be well-defined.
904
00:53:19,930 --> 00:53:23,350
905
00:53:23,350 --> 00:53:23,770
OK.
906
00:53:23,770 --> 00:53:29,190
So a long, longer
than I attended
907
00:53:29,190 --> 00:53:31,440
lecture on Laurent sequences.
908
00:53:31,440 --> 00:53:34,770
But it's turned out, in the
theory of convolutional codes,
909
00:53:34,770 --> 00:53:36,970
it's very important
to get this right.
910
00:53:36,970 --> 00:53:40,182
People have used various
definitions.
911
00:53:40,182 --> 00:53:43,820
They've let this run through
finite sequences, polynomial
912
00:53:43,820 --> 00:53:46,910
sequences, formal
power series.
913
00:53:46,910 --> 00:53:50,910
And trust me, this is the
right way to do it.
914
00:53:50,910 --> 00:53:53,820
And I don't think anyone
is better qualified
915
00:53:53,820 --> 00:53:56,300
to make that statement.
916
00:53:56,300 --> 00:54:00,310
So that is the way we define
a convolutional code.
917
00:54:00,310 --> 00:54:02,930
918
00:54:02,930 --> 00:54:10,990
Now which direction shall
I go from here?
919
00:54:10,990 --> 00:54:14,270
I guess I ought to ask,
what are we going to
920
00:54:14,270 --> 00:54:15,740
allow g of D to be?
921
00:54:15,740 --> 00:54:18,980
922
00:54:18,980 --> 00:54:24,280
In the picture up there, we
have g1 of D and g2 of D
923
00:54:24,280 --> 00:54:34,690
polynomial, which means both
causal and finite.
924
00:54:34,690 --> 00:54:37,580
Causal means it starts
at time 0 or later.
925
00:54:37,580 --> 00:54:39,390
Finite means that it
has only a finite
926
00:54:39,390 --> 00:54:41,160
number of non-zero terms.
927
00:54:41,160 --> 00:54:43,880
928
00:54:43,880 --> 00:54:44,500
OK.
929
00:54:44,500 --> 00:54:50,080
I want to allow perhaps more
general types of encoders.
930
00:54:50,080 --> 00:54:53,630
I do want the encoder
to be finite state.
931
00:54:53,630 --> 00:54:57,680
So let me impose that
I want the encoder
932
00:54:57,680 --> 00:55:00,930
to be finite state.
933
00:55:00,930 --> 00:55:02,395
But I'm going to
allow feedback.
934
00:55:02,395 --> 00:55:08,280
935
00:55:08,280 --> 00:55:11,460
Which I don't have there.
936
00:55:11,460 --> 00:55:16,430
If I allow feedback, then I can
get infinite responses,
937
00:55:16,430 --> 00:55:19,140
infinite impulse responses.
938
00:55:19,140 --> 00:55:23,920
But by the same argument that
I've already made, it's going
939
00:55:23,920 --> 00:55:25,990
to have to eventually
be periodic, right?
940
00:55:25,990 --> 00:55:29,260
If the encoder has
finite state.
941
00:55:29,260 --> 00:55:37,660
So we have to have g1 of D
and g2 of D are general.
942
00:55:37,660 --> 00:55:40,040
We're going to allow up
to n of these for
943
00:55:40,040 --> 00:55:42,560
rate 1 over n encoders.
944
00:55:42,560 --> 00:55:45,460
If we want this to be the
impulse response of a finite
945
00:55:45,460 --> 00:55:48,820
state encoder --
946
00:55:48,820 --> 00:55:52,150
they also have to be
causal, don't they.
947
00:55:52,150 --> 00:55:56,020
In order to be realizable, has
to start at time 0 or later,
948
00:55:56,020 --> 00:55:59,320
the impulse response.
949
00:55:59,320 --> 00:56:01,620
Then I'm going to have to
require that these all be
950
00:56:01,620 --> 00:56:05,725
rational and causal.
951
00:56:05,725 --> 00:56:11,050
952
00:56:11,050 --> 00:56:14,770
And that's the only limitations
I'm going to put
953
00:56:14,770 --> 00:56:18,170
on the g of D.
954
00:56:18,170 --> 00:56:22,710
So in order to get a finite
state system, I need to have
955
00:56:22,710 --> 00:56:24,900
rational impulse responses.
956
00:56:24,900 --> 00:56:26,500
In order to have a realizable
system,
957
00:56:26,500 --> 00:56:27,750
I need causal responses.
958
00:56:27,750 --> 00:56:30,430
959
00:56:30,430 --> 00:56:34,250
So these are the requirements
on the impulse responses.
960
00:56:34,250 --> 00:56:39,770
Otherwise I'm not going to force
them to be anything in
961
00:56:39,770 --> 00:56:41,020
particular.
962
00:56:41,020 --> 00:56:42,790
963
00:56:42,790 --> 00:56:44,050
Now let's see.
964
00:56:44,050 --> 00:56:50,550
I go into realization theory
a little bit in the notes.
965
00:56:50,550 --> 00:56:53,890
Since I've spent so much time on
Laurent sequences, I don't
966
00:56:53,890 --> 00:56:56,880
think I'm going to
do that here.
967
00:56:56,880 --> 00:56:57,736
Yeah?
968
00:56:57,736 --> 00:57:00,170
AUDIENCE: Does this mean that
the denominator is also
969
00:57:00,170 --> 00:57:01,420
polynomial?
970
00:57:01,420 --> 00:57:03,480
971
00:57:03,480 --> 00:57:06,030
PROFESSOR: Correct.
972
00:57:06,030 --> 00:57:11,860
So in fact, let me always
write these.
973
00:57:11,860 --> 00:57:14,160
Let me take the least common
multiple of all the
974
00:57:14,160 --> 00:57:15,690
denominators.
975
00:57:15,690 --> 00:57:17,800
I'm going to write these then.
976
00:57:17,800 --> 00:57:26,280
g of D is going to be some
vector of polynomials up here.
977
00:57:26,280 --> 00:57:29,580
I can always write the Least
Common Multiple of all the
978
00:57:29,580 --> 00:57:32,050
denominators down here and have
a single denominator.
979
00:57:32,050 --> 00:57:34,700
980
00:57:34,700 --> 00:57:37,190
OK?
981
00:57:37,190 --> 00:57:44,720
So this is going to be my
general form for a causal,
982
00:57:44,720 --> 00:57:49,390
rational, single input and
output impulse response.
983
00:57:49,390 --> 00:57:52,670
It's going to consist of
something like this, where
984
00:57:52,670 --> 00:57:55,285
these are all polynomial.
985
00:57:55,285 --> 00:57:58,280
986
00:57:58,280 --> 00:58:06,460
This is a single polynomial
with d0 equals 1.
987
00:58:06,460 --> 00:58:08,650
Turns out that's necessary
for realizability.
988
00:58:08,650 --> 00:58:11,490
989
00:58:11,490 --> 00:58:14,770
And I'll just tell you the fact,
which you can read about
990
00:58:14,770 --> 00:58:17,050
in the notes.
991
00:58:17,050 --> 00:58:21,340
In fact, I will leave this
mostly as an exercise for the
992
00:58:21,340 --> 00:58:22,950
student, anyway.
993
00:58:22,950 --> 00:58:37,900
Realizable with nu equals max of
the degree of the n_i of D,
994
00:58:37,900 --> 00:58:39,900
or the degree of the
denominator.
995
00:58:39,900 --> 00:58:42,720
In other words, I take all
these polynomials --
996
00:58:42,720 --> 00:58:46,950
the maximum degree
of any of them --
997
00:58:46,950 --> 00:58:48,820
and I call that nu.
998
00:58:48,820 --> 00:58:50,600
And I claim there's
a realization
999
00:58:50,600 --> 00:58:51,875
with nu memory elements.
1000
00:58:51,875 --> 00:58:56,616
1001
00:58:56,616 --> 00:58:59,430
And of course, I'm going to
assume here that I've reduced
1002
00:58:59,430 --> 00:59:04,270
this to lowest terms in order
to keep that down.
1003
00:59:04,270 --> 00:59:06,270
If I didn't have the
denominator term,
1004
00:59:06,270 --> 00:59:07,880
this would be clear.
1005
00:59:07,880 --> 00:59:12,860
If I just had g of D equal
to a set of a vector of n
1006
00:59:12,860 --> 00:59:18,260
polynomials of maximum degree
nu, then it's clear that just
1007
00:59:18,260 --> 00:59:23,280
by this kind of a shift register
realization of here
1008
00:59:23,280 --> 00:59:28,870
we call the constraint length
equal to the number of memory
1009
00:59:28,870 --> 00:59:35,100
elements, I can realize any n
impulse responses which all
1010
00:59:35,100 --> 00:59:38,240
have degree nu or less, right?
1011
00:59:38,240 --> 00:59:42,910
So the only trick is showing how
to put in a feedback loop
1012
00:59:42,910 --> 00:59:47,190
which basically implements this
denominator polynomial d
1013
00:59:47,190 --> 00:59:52,000
of D. If I start off by
realizing 1 over d of D, then
1014
00:59:52,000 --> 00:59:55,100
I can basically just realize
this in the same way.
1015
00:59:55,100 --> 00:59:57,710
1016
00:59:57,710 --> 01:00:00,405
And nu is called the
constraint length.
1017
01:00:00,405 --> 01:00:05,960
1018
01:00:05,960 --> 01:00:10,703
And I have 2 to the nu states.
1019
01:00:10,703 --> 01:00:15,820
1020
01:00:15,820 --> 01:00:22,400
So by constraining this to be
of this form, I've now gone
1021
01:00:22,400 --> 01:00:23,830
full circle.
1022
01:00:23,830 --> 01:00:27,230
The number of states is in
fact 2 to the nu, and in
1023
01:00:27,230 --> 01:00:28,480
particular, it's finite.
1024
01:00:28,480 --> 01:00:31,960
1025
01:00:31,960 --> 01:00:43,600
When I constrain the impulses
responses to be like that,
1026
01:00:43,600 --> 01:00:45,630
then I guarantee that I'm going
to have a finite state
1027
01:00:45,630 --> 01:00:46,880
realization.
1028
01:00:46,880 --> 01:00:49,990
1029
01:00:49,990 --> 01:00:52,760
So from now on, that's what
my impulse response is
1030
01:00:52,760 --> 01:00:54,010
going to look like.
1031
01:00:54,010 --> 01:00:57,020
1032
01:00:57,020 --> 01:00:57,650
All right.
1033
01:00:57,650 --> 01:01:05,820
Now let's talk about code
equivalence, or encoder
1034
01:01:05,820 --> 01:01:07,070
equivalence.
1035
01:01:07,070 --> 01:01:11,550
1036
01:01:11,550 --> 01:01:17,800
I've defined the code
generated --
1037
01:01:17,800 --> 01:01:23,380
now I'm going to characterize
an encoder by its impulse
1038
01:01:23,380 --> 01:01:32,320
responses, g of D. The code
generated by g of D is u of D
1039
01:01:32,320 --> 01:01:38,260
g of D as u of D goes
through the set
1040
01:01:38,260 --> 01:01:39,555
of all Laurent sequences.
1041
01:01:39,555 --> 01:01:43,460
1042
01:01:43,460 --> 01:01:56,585
Two encoders are equivalent if
they generate the same code.
1043
01:01:56,585 --> 01:02:05,280
1044
01:02:05,280 --> 01:02:07,280
Seems reasonable.
1045
01:02:07,280 --> 01:02:09,320
What we're really ultimately
interested in in
1046
01:02:09,320 --> 01:02:12,330
communications is the behavior
of the code.
1047
01:02:12,330 --> 01:02:16,150
In particular, the minimum
distance of the code, how far
1048
01:02:16,150 --> 01:02:18,300
the sequence is separated.
1049
01:02:18,300 --> 01:02:21,720
We're not particularly
interested in the encoder.
1050
01:02:21,720 --> 01:02:24,880
At the decoder, we're just
simply going to try to tell
1051
01:02:24,880 --> 01:02:27,270
which code sequence was sent.
1052
01:02:27,270 --> 01:02:31,750
So for most purposes,
probability of decoding error,
1053
01:02:31,750 --> 01:02:33,790
performance of the decoder
and so forth, we're only
1054
01:02:33,790 --> 01:02:35,250
interested in the code itself.
1055
01:02:35,250 --> 01:02:38,210
We're not interested in this
little one-to-one map between
1056
01:02:38,210 --> 01:02:42,110
information bits and
the code sequences.
1057
01:02:42,110 --> 01:02:45,130
So this is a reasonable
definition of encoder
1058
01:02:45,130 --> 01:02:46,380
equivalence.
1059
01:02:46,380 --> 01:02:48,590
1060
01:02:48,590 --> 01:02:54,260
Now, for the case of rate 1/n
codes, which is all I'm
1061
01:02:54,260 --> 01:02:55,330
talking about here --
1062
01:02:55,330 --> 01:02:59,675
1 input, n output, so the code
generator looks like this --
1063
01:02:59,675 --> 01:03:03,970
1064
01:03:03,970 --> 01:03:06,230
it's very simple.
1065
01:03:06,230 --> 01:03:19,670
g of D and g prime of D are
equivalent if and only if g of
1066
01:03:19,670 --> 01:03:29,740
D and g prime of D differ by
some multiple a of D, where I
1067
01:03:29,740 --> 01:03:32,615
could let a of D be any
Laurent sequence.
1068
01:03:32,615 --> 01:03:37,351
But then to keep everything in
the same ballpark, I think I'd
1069
01:03:37,351 --> 01:03:41,820
have to keep a of D rational
and causal.
1070
01:03:41,820 --> 01:03:45,160
But even forgetting this -- in
other words, if one is a
1071
01:03:45,160 --> 01:03:46,510
multiple of the other --
1072
01:03:46,510 --> 01:03:49,610
then the two codes
are equivalent.
1073
01:03:49,610 --> 01:04:01,580
That's clear because I can
invert a of D. So this is just
1074
01:04:01,580 --> 01:04:03,680
a single Laurent sequence,
has an inverse.
1075
01:04:03,680 --> 01:04:08,510
This is the same thing as
saying g of D times
1076
01:04:08,510 --> 01:04:10,180
1 over a of D --
1077
01:04:10,180 --> 01:04:15,450
which is another rational,
causal sequence, is g prime of
1078
01:04:15,450 --> 01:04:17,650
D.
1079
01:04:17,650 --> 01:04:27,050
If this is true, then the
sequence generated by u of D
1080
01:04:27,050 --> 01:04:30,780
when the encoder is g of D is
the same as the sequence
1081
01:04:30,780 --> 01:04:35,135
generated by u of D a of D when
the encoder is g prime of
1082
01:04:35,135 --> 01:04:38,400
D. And this is another
rational sequence.
1083
01:04:38,400 --> 01:04:40,820
And vice versa.
1084
01:04:40,820 --> 01:04:44,450
the code sequence generated by u
of D, the encoder is g prime
1085
01:04:44,450 --> 01:04:47,740
of D, is the sequence generated
by u of D over a of
1086
01:04:47,740 --> 01:04:52,530
D if the encoder
is g of D. OK?
1087
01:04:52,530 --> 01:04:54,120
So that's really the proof.
1088
01:04:54,120 --> 01:04:57,070
1089
01:04:57,070 --> 01:05:00,520
So this is very, very
simple theorem.
1090
01:05:00,520 --> 01:05:08,810
In other words, I can multiply
any encoder n-tuple g of D by
1091
01:05:08,810 --> 01:05:14,550
any rational causal sequence out
front, in particular by a
1092
01:05:14,550 --> 01:05:17,400
polynomial or 1 over a
polynomial or something, and
1093
01:05:17,400 --> 01:05:19,950
I'm going to generate
the same code.
1094
01:05:19,950 --> 01:05:21,420
This is not going to matter.
1095
01:05:21,420 --> 01:05:24,290
1096
01:05:24,290 --> 01:05:28,400
Now, to see why we might want
to take make use of this --
1097
01:05:28,400 --> 01:05:32,452
1098
01:05:32,452 --> 01:05:35,590
let's see.
1099
01:05:35,590 --> 01:05:39,770
Well, given this, we might want
to pick some particularly
1100
01:05:39,770 --> 01:05:45,920
nice encoder from this
equivalence class of encoders,
1101
01:05:45,920 --> 01:05:49,074
all of which generate
the same code.
1102
01:05:49,074 --> 01:05:55,090
And basically, I'm going to
suggest that the nicest
1103
01:05:55,090 --> 01:06:02,040
encoder is the one we get if we
multiply by d of D. And if
1104
01:06:02,040 --> 01:06:04,710
there's any common factor
of these n of D's,
1105
01:06:04,710 --> 01:06:06,630
we divide it out.
1106
01:06:06,630 --> 01:06:11,230
That will give us the least
degree polynomial encoder,
1107
01:06:11,230 --> 01:06:13,710
which is equivalent to
the given encoder.
1108
01:06:13,710 --> 01:06:17,520
If I start off with one like
this, and I want to multiply
1109
01:06:17,520 --> 01:06:20,810
through by the denominator to
make it a polynomial, and if
1110
01:06:20,810 --> 01:06:23,200
there's any common factor to
the numerator, I want to
1111
01:06:23,200 --> 01:06:24,540
divide it out.
1112
01:06:24,540 --> 01:06:27,330
And that's what I'm going to
take as my canonical encoder.
1113
01:06:27,330 --> 01:06:30,310
1114
01:06:30,310 --> 01:06:33,924
Now let me give you a little
motivation for that.
1115
01:06:33,924 --> 01:06:36,786
AUDIENCE: [INAUDIBLE]
1116
01:06:36,786 --> 01:06:38,550
PROFESSOR: That's
one motivation.
1117
01:06:38,550 --> 01:06:41,180
It will have the simplest
realization.
1118
01:06:41,180 --> 01:06:45,260
It will have a feedback-free
realization by getting rid of
1119
01:06:45,260 --> 01:06:47,000
the denominator term.
1120
01:06:47,000 --> 01:06:49,780
And it will have the least
constraint length of all
1121
01:06:49,780 --> 01:06:52,920
equivalent encoders, and
that's a theorem too.
1122
01:06:52,920 --> 01:06:56,810
But that's not the primary
motivation.
1123
01:06:56,810 --> 01:07:00,240
Let me talk a little bit about
distance at this point.
1124
01:07:00,240 --> 01:07:05,630
1125
01:07:05,630 --> 01:07:13,050
Let me take my example
encoder again.
1126
01:07:13,050 --> 01:07:16,690
And let me ask, what's
the known distance
1127
01:07:16,690 --> 01:07:18,030
between code sequences?
1128
01:07:18,030 --> 01:07:22,320
This is clearly going to be an
important performance metric
1129
01:07:22,320 --> 01:07:25,550
of this code, right?
1130
01:07:25,550 --> 01:07:28,080
How would you go about answering
that question, or at
1131
01:07:28,080 --> 01:07:30,060
least two reasonable
ways to go?
1132
01:07:30,060 --> 01:07:37,560
1133
01:07:37,560 --> 01:07:39,810
Do you think the known distance
is greater than 1?
1134
01:07:39,810 --> 01:07:42,980
1135
01:07:42,980 --> 01:07:45,090
You think it's 1?
1136
01:07:45,090 --> 01:07:46,730
OK.
1137
01:07:46,730 --> 01:07:49,350
So how would you get
two sequences that
1138
01:07:49,350 --> 01:07:52,013
differ only in one place?
1139
01:07:52,013 --> 01:07:55,320
AUDIENCE: I mean, if you take
the impulse response.
1140
01:07:55,320 --> 01:07:56,580
Or you take the --
1141
01:07:56,580 --> 01:07:59,980
PROFESSOR: Well, the impulse
response, if I put in 1, u of
1142
01:07:59,980 --> 01:08:06,530
D equals 1, then I get out g of
D, which in this case is 1
1143
01:08:06,530 --> 01:08:11,360
plus D squared, 1 plus D plus
D squared, I actually get a
1144
01:08:11,360 --> 01:08:12,800
weight 5 sequence out.
1145
01:08:12,800 --> 01:08:16,359
1146
01:08:16,359 --> 01:08:19,225
You see that?
1147
01:08:19,225 --> 01:08:25,170
AUDIENCE: [INAUDIBLE] you get
1 and 1 0 1, and [INAUDIBLE]
1148
01:08:25,170 --> 01:08:27,010
1 and 1 and 1 --
1149
01:08:27,010 --> 01:08:31,593
PROFESSOR: So I get this out,
in D transform notation.
1150
01:08:31,593 --> 01:08:37,850
Or I get 1 0 1 1 1 1 in
terms of that times
1151
01:08:37,850 --> 01:08:41,116
0 1 1 and so forth.
1152
01:08:41,116 --> 01:08:44,569
1153
01:08:44,569 --> 01:08:47,916
y1 equals this, y2 equals that.
1154
01:08:47,916 --> 01:08:49,810
Right?
1155
01:08:49,810 --> 01:08:50,200
OK.
1156
01:08:50,200 --> 01:08:54,220
So that's a weight 5 sequence.
1157
01:08:54,220 --> 01:08:57,040
So if I put in the all-zero
sequence, I get out the
1158
01:08:57,040 --> 01:08:57,740
all-zero sequence.
1159
01:08:57,740 --> 01:09:00,990
So as always, the all-zero
sequence is an
1160
01:09:00,990 --> 01:09:02,420
element of the code.
1161
01:09:02,420 --> 01:09:06,439
This differs from the all-zero
sequence in five places.
1162
01:09:06,439 --> 01:09:10,470
So all you've shown me so far is
the minimum distance is not
1163
01:09:10,470 --> 01:09:11,720
greater than five.
1164
01:09:11,720 --> 01:09:14,550
1165
01:09:14,550 --> 01:09:17,920
I've found two sequences that
differ in five places.
1166
01:09:17,920 --> 01:09:20,806
Are there any that differ in
fewer places than that?
1167
01:09:20,806 --> 01:09:27,149
1168
01:09:27,149 --> 01:09:29,590
The answer is no.
1169
01:09:29,590 --> 01:09:33,090
So in fact, it's called
the free distance.
1170
01:09:33,090 --> 01:09:38,340
For this code, the so-called
free distance is five.
1171
01:09:38,340 --> 01:09:42,569
1172
01:09:42,569 --> 01:09:44,750
Let's think of the two different
kinds of structure
1173
01:09:44,750 --> 01:09:46,240
for this code.
1174
01:09:46,240 --> 01:09:51,670
And they lead to different but
complementary arguments that
1175
01:09:51,670 --> 01:09:53,115
both get us to this
conclusion.
1176
01:09:53,115 --> 01:09:55,820
1177
01:09:55,820 --> 01:10:00,060
First of all, the code
is linear, right?
1178
01:10:00,060 --> 01:10:01,390
So C is linear.
1179
01:10:01,390 --> 01:10:05,750
1180
01:10:05,750 --> 01:10:08,300
Is it not?
1181
01:10:08,300 --> 01:10:17,680
If I have y of D equals u of D g
of D, and y prime of D equal
1182
01:10:17,680 --> 01:10:20,270
to, say, some other sequence.
1183
01:10:20,270 --> 01:10:24,460
u prime of D times g of D. Then
let me check the group
1184
01:10:24,460 --> 01:10:29,020
property, which is all I have
to check for a binary code.
1185
01:10:29,020 --> 01:10:33,540
And I find that y of D the
sequence, the sum of these two
1186
01:10:33,540 --> 01:10:38,020
sequences, is the sequence
that's generated by the sum of
1187
01:10:38,020 --> 01:10:40,640
the two input sequences that
led to these two sequences.
1188
01:10:40,640 --> 01:10:45,860
So the binary sum of any
two sequences is
1189
01:10:45,860 --> 01:10:47,426
another code word.
1190
01:10:47,426 --> 01:10:47,910
All right?
1191
01:10:47,910 --> 01:10:49,255
Ergo it has the group
property.
1192
01:10:49,255 --> 01:10:51,820
1193
01:10:51,820 --> 01:10:55,650
Ergo C is a group.
1194
01:10:55,650 --> 01:10:56,990
And that's all we
need to check.
1195
01:10:56,990 --> 01:11:01,170
It's actually linear over F2,
as a vector space of F2.
1196
01:11:01,170 --> 01:11:03,880
1197
01:11:03,880 --> 01:11:13,410
0 is in the code, and 1 times
any code word in the code.
1198
01:11:13,410 --> 01:11:18,960
So we've checked closure under
vector addition and scalar
1199
01:11:18,960 --> 01:11:20,900
multiplication.
1200
01:11:20,900 --> 01:11:23,040
So it's linear.
1201
01:11:23,040 --> 01:11:26,080
What is the main conclusion
we get from this?
1202
01:11:26,080 --> 01:11:29,700
1203
01:11:29,700 --> 01:11:35,270
Therefore, the minimum
distance between code
1204
01:11:35,270 --> 01:11:45,780
sequences is equal to --
1205
01:11:45,780 --> 01:11:47,030
anybody?
1206
01:11:47,030 --> 01:11:48,900
1207
01:11:48,900 --> 01:12:00,280
The minimum non-zero weight
of any y of D in the code.
1208
01:12:00,280 --> 01:12:04,870
1209
01:12:04,870 --> 01:12:05,940
Exactly the same argument.
1210
01:12:05,940 --> 01:12:10,090
We're talking infinite sequences
here, but there's
1211
01:12:10,090 --> 01:12:13,150
nothing that changes
in the argument.
1212
01:12:13,150 --> 01:12:15,470
And so that really simplifies
things.
1213
01:12:15,470 --> 01:12:17,250
We simply have to ask --
1214
01:12:17,250 --> 01:12:24,430
here's a non-zero sequence
of Hamming weight 5.
1215
01:12:24,430 --> 01:12:28,950
You can add this to any code
sequence and get another
1216
01:12:28,950 --> 01:12:30,410
legitimate code sequence.
1217
01:12:30,410 --> 01:12:32,260
This is the code sequence.
1218
01:12:32,260 --> 01:12:33,580
Add it to any other,
and you get a
1219
01:12:33,580 --> 01:12:34,790
legitimate code sequence.
1220
01:12:34,790 --> 01:12:37,690
So from any other code sequence,
there's going to be
1221
01:12:37,690 --> 01:12:39,960
one of distance 5.
1222
01:12:39,960 --> 01:12:42,370
Are there any lower weight
sequences in this code?
1223
01:12:42,370 --> 01:12:49,340
1224
01:12:49,340 --> 01:12:50,250
Well, what would you --
1225
01:12:50,250 --> 01:12:51,500
AUDIENCE: [INAUDIBLE]
1226
01:12:51,500 --> 01:12:54,050
1227
01:12:54,050 --> 01:12:55,140
PROFESSOR: Good.
1228
01:12:55,140 --> 01:12:57,020
So here's where we need
to get into much
1229
01:12:57,020 --> 01:12:58,620
more concrete arguments.
1230
01:12:58,620 --> 01:13:02,340
You can see that first of all,
we're only interested in
1231
01:13:02,340 --> 01:13:06,070
looking at finite sequences
in the code.
1232
01:13:06,070 --> 01:13:08,450
And we might as well have them
start at time 0, because if
1233
01:13:08,450 --> 01:13:11,200
they start later, we could just
shift them over by time
1234
01:13:11,200 --> 01:13:12,480
and variance.
1235
01:13:12,480 --> 01:13:15,420
So we're only interested in
finite polynomial sequences
1236
01:13:15,420 --> 01:13:17,620
that start at time 0.
1237
01:13:17,620 --> 01:13:18,320
OK.
1238
01:13:18,320 --> 01:13:21,180
So just as you said, in the
first place, they're always
1239
01:13:21,180 --> 01:13:22,660
going to have bing bing!
1240
01:13:22,660 --> 01:13:24,240
Two 1's.
1241
01:13:24,240 --> 01:13:25,620
When the first 1 comes
in, we're going
1242
01:13:25,620 --> 01:13:27,800
to get two 1's out.
1243
01:13:27,800 --> 01:13:30,270
For sure.
1244
01:13:30,270 --> 01:13:33,040
Time 2.
1245
01:13:33,040 --> 01:13:37,200
This is what we get out if the
second bit in, in the input
1246
01:13:37,200 --> 01:13:39,990
sequence, is a 0.
1247
01:13:39,990 --> 01:13:45,590
What happens if the second
bit in is a 1?
1248
01:13:45,590 --> 01:13:52,450
Then we add this to this,
shift it over 1.
1249
01:13:52,450 --> 01:13:54,520
We have 1 1 in particular
at this place.
1250
01:13:54,520 --> 01:13:56,610
We get a 1 0 out.
1251
01:13:56,610 --> 01:13:57,430
OK?
1252
01:13:57,430 --> 01:14:01,320
So we conclude at the second
time, we're always going to
1253
01:14:01,320 --> 01:14:05,670
get at least another
unit of distance.
1254
01:14:05,670 --> 01:14:07,160
Now jump ahead.
1255
01:14:07,160 --> 01:14:08,810
However long this is, this
is going to have
1256
01:14:08,810 --> 01:14:10,060
to end at some time.
1257
01:14:10,060 --> 01:14:12,800
1258
01:14:12,800 --> 01:14:16,650
At the time when the last bit,
the last non-zero bit, is
1259
01:14:16,650 --> 01:14:20,080
shifting out of the shift
register, we're also going to
1260
01:14:20,080 --> 01:14:21,330
get 1 1 out.
1261
01:14:21,330 --> 01:14:24,390
1262
01:14:24,390 --> 01:14:30,565
So any finite code word has
to end with a 1 1 at
1263
01:14:30,565 --> 01:14:31,815
the last time out.
1264
01:14:31,815 --> 01:14:34,270
1265
01:14:34,270 --> 01:14:36,890
People don't seem to be totally
comfortable with this,
1266
01:14:36,890 --> 01:14:38,280
so I'll do it in a
more elaborate
1267
01:14:38,280 --> 01:14:39,880
way in just a second.
1268
01:14:39,880 --> 01:14:41,410
But this is the basis
of the argument.
1269
01:14:41,410 --> 01:14:42,956
AUDIENCE: [INAUDIBLE]
1270
01:14:42,956 --> 01:14:44,960
this?
1271
01:14:44,960 --> 01:14:45,920
PROFESSOR: At the end?
1272
01:14:45,920 --> 01:14:47,380
AUDIENCE: No, I mean
by [UNINTELLIGIBLE]
1273
01:14:47,380 --> 01:14:50,040
at the same time.
1274
01:14:50,040 --> 01:14:51,870
PROFESSOR: At the end of
a finite code word?
1275
01:14:51,870 --> 01:14:52,750
AUDIENCE: Yes.
1276
01:14:52,750 --> 01:14:53,040
PROFESSOR: All right.
1277
01:14:53,040 --> 01:14:54,290
What does a finite code
word consist of?
1278
01:14:54,290 --> 01:14:56,880
1279
01:14:56,880 --> 01:15:00,060
I'm going to claim that we only
get a finite code word
1280
01:15:00,060 --> 01:15:04,680
out when we put a finite
input sequence in.
1281
01:15:04,680 --> 01:15:05,240
OK?
1282
01:15:05,240 --> 01:15:07,670
There's no feedback here.
1283
01:15:07,670 --> 01:15:11,170
So there's going to be some last
1 in the input sequence
1284
01:15:11,170 --> 01:15:12,980
as that shifts through here.
1285
01:15:12,980 --> 01:15:15,130
At the very last time,
the last state, is
1286
01:15:15,130 --> 01:15:17,540
going to be 0 1.
1287
01:15:17,540 --> 01:15:17,920
All right?
1288
01:15:17,920 --> 01:15:20,400
Just before that comes
out, there's a 0
1289
01:15:20,400 --> 01:15:22,695
coming in, forever after.
1290
01:15:22,695 --> 01:15:26,660
So the last time we get a 1 at
u k minus 2, and that forces
1291
01:15:26,660 --> 01:15:30,780
these two bits out to be 1.
1292
01:15:30,780 --> 01:15:32,680
Or you can do it
by polynomials.
1293
01:15:32,680 --> 01:15:36,730
You can always show that you
multiply this by any finite
1294
01:15:36,730 --> 01:15:40,010
polynomial, you're going to get
highest degree terms, both
1295
01:15:40,010 --> 01:15:41,260
equal to 1.
1296
01:15:41,260 --> 01:15:43,260
1297
01:15:43,260 --> 01:15:44,260
OK.
1298
01:15:44,260 --> 01:15:48,010
I'll do this by explicitly
drawing out the state diagram.
1299
01:15:48,010 --> 01:15:56,460
So we can conclude here that we
always have 2 here, 1 here,
1300
01:15:56,460 --> 01:15:59,530
dot dot dot dot, and finally
at the end, we have to
1301
01:15:59,530 --> 01:16:00,850
have at least 2.
1302
01:16:00,850 --> 01:16:04,020
Therefore, every nonzero finite
sequence has to have
1303
01:16:04,020 --> 01:16:05,870
weight at least 5.
1304
01:16:05,870 --> 01:16:08,750
Since we've seen one of that
has weight 5, that is the
1305
01:16:08,750 --> 01:16:11,420
minimum distance.
1306
01:16:11,420 --> 01:16:13,100
OK?
1307
01:16:13,100 --> 01:16:16,260
Now let's do this by drawing
the state diagram.
1308
01:16:16,260 --> 01:16:20,730
1309
01:16:20,730 --> 01:16:23,331
Which is where we're going.
1310
01:16:23,331 --> 01:16:24,624
AUDIENCE: So I know
that [INAUDIBLE]
1311
01:16:24,624 --> 01:16:31,040
1312
01:16:31,040 --> 01:16:31,350
PROFESSOR: No.
1313
01:16:31,350 --> 01:16:35,410
We had to construct a little
argument here, and in effect,
1314
01:16:35,410 --> 01:16:36,300
make a little search.
1315
01:16:36,300 --> 01:16:38,570
And this simple argument
wouldn't work for more
1316
01:16:38,570 --> 01:16:39,890
complicated cases.
1317
01:16:39,890 --> 01:16:43,460
So let me show you how to attack
more complicated cases.
1318
01:16:43,460 --> 01:16:48,610
We want to draw a finite
state machine.
1319
01:16:48,610 --> 01:16:50,485
How do you analyze finite
state machines?
1320
01:16:50,485 --> 01:16:53,920
1321
01:16:53,920 --> 01:16:56,380
Well, a good way to start
is to draw the
1322
01:16:56,380 --> 01:16:57,730
state transition diagram.
1323
01:16:57,730 --> 01:17:04,460
1324
01:17:04,460 --> 01:17:08,690
And I don't know what the
curriculum consists of
1325
01:17:08,690 --> 01:17:11,810
nowadays, but I assume
you've all done this.
1326
01:17:11,810 --> 01:17:14,120
We draw the four possible
states.
1327
01:17:14,120 --> 01:17:15,370
0 0.
1328
01:17:15,370 --> 01:17:19,130
1329
01:17:19,130 --> 01:17:22,450
1 0, we can get to from 0 0.
1330
01:17:22,450 --> 01:17:28,850
0 0, if I get an input of 0,
then I'm going to put out 0 0,
1331
01:17:28,850 --> 01:17:31,060
then I'm going to stay
in the 0 state.
1332
01:17:31,060 --> 01:17:32,170
All right?
1333
01:17:32,170 --> 01:17:36,266
Because this is linear, if the
0 comes in, that's 0 0, stay
1334
01:17:36,266 --> 01:17:38,110
in the 0 state.
1335
01:17:38,110 --> 01:17:42,780
Or a 1 could come in.
1336
01:17:42,780 --> 01:17:46,310
And in that case, as we've
noted, we'll put out two 1's
1337
01:17:46,310 --> 01:17:50,320
as our output, and we'll
transition to the state 1 0.
1338
01:17:50,320 --> 01:17:54,040
1339
01:17:54,040 --> 01:17:57,190
Now from 1 0, where can we go?
1340
01:17:57,190 --> 01:18:09,630
For 1 0, if a 0 comes in, then
we'll put out 0 1 and we'll
1341
01:18:09,630 --> 01:18:11,745
transition to 0 1.
1342
01:18:11,745 --> 01:18:14,910
1343
01:18:14,910 --> 01:18:19,650
Or we'll get another 1 in.
1344
01:18:19,650 --> 01:18:23,182
We would put out the complement
of that, 1 0.
1345
01:18:23,182 --> 01:18:27,830
So we get another 1 1 in
this linear thing.
1346
01:18:27,830 --> 01:18:28,405
Work it out.
1347
01:18:28,405 --> 01:18:31,320
I'll just assert that.
1348
01:18:31,320 --> 01:18:33,120
And we go to state 1 1.
1349
01:18:33,120 --> 01:18:36,350
1350
01:18:36,350 --> 01:18:43,760
From 0 1, if we get a 0 in, we
return to the 0 0 state and we
1351
01:18:43,760 --> 01:18:45,260
put out 1 1.
1352
01:18:45,260 --> 01:18:49,385
So here's our basic impulse
response, right down here.
1353
01:18:49,385 --> 01:18:52,080
We put in a 1, we go through
this little cycle, we come
1354
01:18:52,080 --> 01:18:54,960
back to 0 0.
1355
01:18:54,960 --> 01:19:01,910
Or if we get a 1 in, we go
back up here, and at that
1356
01:19:01,910 --> 01:19:05,460
point, we actually only
put out two 0's.
1357
01:19:05,460 --> 01:19:06,426
Is that right?
1358
01:19:06,426 --> 01:19:08,590
Yeah, because it has to be
the complement of this.
1359
01:19:08,590 --> 01:19:12,380
1360
01:19:12,380 --> 01:19:15,980
And if we're in the 1 1 state
and another 1 comes in, we
1361
01:19:15,980 --> 01:19:19,620
stay in the 1 1 state.
1362
01:19:19,620 --> 01:19:21,340
And we put out what?
1363
01:19:21,340 --> 01:19:25,890
1364
01:19:25,890 --> 01:19:29,730
Let me do the 0 1 first.
1365
01:19:29,730 --> 01:19:34,260
0, and we're in 1 1 state,
then what do we get out?
1366
01:19:34,260 --> 01:19:37,725
We get out a 0 down here and
we get out a 1 up there.
1367
01:19:37,725 --> 01:19:38,975
We got 1 0.
1368
01:19:38,975 --> 01:19:41,750
1369
01:19:41,750 --> 01:19:46,970
Or if we get in a 1,
we put out 0 1.
1370
01:19:46,970 --> 01:19:47,330
All right.
1371
01:19:47,330 --> 01:19:49,025
So that's the state transition
diagram.
1372
01:19:49,025 --> 01:19:53,140
1373
01:19:53,140 --> 01:19:58,990
Now again, let me make the
argument that I made before,
1374
01:19:58,990 --> 01:20:01,652
now a little bit more concisely,
because we're going
1375
01:20:01,652 --> 01:20:02,902
to have to stop.
1376
01:20:02,902 --> 01:20:06,760
1377
01:20:06,760 --> 01:20:10,710
So every path through this
state transition diagram
1378
01:20:10,710 --> 01:20:13,960
corresponds to a code
sequence, right?
1379
01:20:13,960 --> 01:20:18,460
Every finite or semi-infinite
path always
1380
01:20:18,460 --> 01:20:21,650
start in the 0 state.
1381
01:20:21,650 --> 01:20:26,120
For a finite code word, I have
to come back to the 0 state.
1382
01:20:26,120 --> 01:20:27,670
Correct?
1383
01:20:27,670 --> 01:20:32,590
So I'm always going to get
two 1's when I start out.
1384
01:20:32,590 --> 01:20:35,440
Then the next time, I'm either
going to go on this transition
1385
01:20:35,440 --> 01:20:38,680
or this transition, but either
way, I'm going to get at least
1386
01:20:38,680 --> 01:20:41,100
another unit of weight.
1387
01:20:41,100 --> 01:20:43,400
And then I can do whatever I
want through here for a while,
1388
01:20:43,400 --> 01:20:46,860
but eventually I'm going to have
to come back out here.
1389
01:20:46,860 --> 01:20:48,800
And when I get back to the
0 state, I'm going to
1390
01:20:48,800 --> 01:20:50,050
get two more 1's.
1391
01:20:50,050 --> 01:20:52,170
1392
01:20:52,170 --> 01:20:55,883
So that's maybe a lot easier way
to say the minimum weight
1393
01:20:55,883 --> 01:20:57,133
has to be 5.
1394
01:20:57,133 --> 01:20:59,290
1395
01:20:59,290 --> 01:21:02,840
Again, if it were more
complicated, I'd have to make
1396
01:21:02,840 --> 01:21:04,880
more of an argument.
1397
01:21:04,880 --> 01:21:05,200
All right.
1398
01:21:05,200 --> 01:21:13,530
Next time, when we come back,
we'll talk about turning this
1399
01:21:13,530 --> 01:21:16,850
into a trellis diagram.
1400
01:21:16,850 --> 01:21:19,370
We'll talk about
catastrophicity, which is
1401
01:21:19,370 --> 01:21:22,370
where I was going with this
canonical generator.
1402
01:21:22,370 --> 01:21:25,100
It's just a minor algebraic
point, but one you need to
1403
01:21:25,100 --> 01:21:26,160
know about.
1404
01:21:26,160 --> 01:21:27,710
We'll talk about the
Viterbi algorithm.
1405
01:21:27,710 --> 01:21:29,520
I think we can probably
get through
1406
01:21:29,520 --> 01:21:32,570
this chapter on Wednesday.
1407
01:21:32,570 --> 01:21:34,510
OK?
1408
01:21:34,510 --> 01:21:37,650
Ashish has your midterms and
the midterm solutions.
1409
01:21:37,650 --> 01:21:48,287