1
00:00:00,000 --> 00:00:02,150
PROFESSOR: Good moRning.
2
00:00:02,150 --> 00:00:04,019
Again, we're left with
a little stub of
3
00:00:04,019 --> 00:00:07,020
chapter six to finish.
4
00:00:07,020 --> 00:00:09,640
This was our first chapter on
binary linear block codes.
5
00:00:09,640 --> 00:00:12,200
6
00:00:12,200 --> 00:00:16,810
Very briefly, last time we
developed the family of RM,
7
00:00:16,810 --> 00:00:20,890
Reed-Muller codes, parameterized
by R and M.
8
00:00:20,890 --> 00:00:23,300
These are codes of length two
to the M, and distance 2 to
9
00:00:23,300 --> 00:00:27,735
the M minus R for all reasonable
combinations of M
10
00:00:27,735 --> 00:00:29,460
and R, integers.
11
00:00:29,460 --> 00:00:32,570
And the reason I do these first
is because it gives us a
12
00:00:32,570 --> 00:00:34,240
nice big -- in fact,
infinite --
13
00:00:34,240 --> 00:00:37,050
family of codes that kind
of cover the waterfront.
14
00:00:37,050 --> 00:00:40,130
Let us see what we can
reasonably get in terms of the
15
00:00:40,130 --> 00:00:42,860
parameters n k d in codes.
16
00:00:42,860 --> 00:00:47,510
And, of course, n k t d tell us
most of what we know, want
17
00:00:47,510 --> 00:00:51,530
to know about the performance
of moderate complexity codes
18
00:00:51,530 --> 00:00:54,820
in terms of coding gain.
19
00:00:54,820 --> 00:00:58,290
And it's easy to see that among
these codes, we have
20
00:00:58,290 --> 00:01:01,390
sequences in which the coding
gain goes -- the nominal
21
00:01:01,390 --> 00:01:03,450
coding gain goes to infinity.
22
00:01:03,450 --> 00:01:06,570
And we also know that there's
at least one sequence, the
23
00:01:06,570 --> 00:01:10,660
bi-orthogonal sequence, for
which the effective coding
24
00:01:10,660 --> 00:01:14,630
gain completely closes the
gap to the Shannon limit.
25
00:01:14,630 --> 00:01:17,910
And in all likelihood, there
are many such sequences,
26
00:01:17,910 --> 00:01:22,140
though I'm not aware of the
actual result there.
27
00:01:22,140 --> 00:01:27,960
So it seems like we might be
almost finished here, except I
28
00:01:27,960 --> 00:01:31,000
asked about what possible fly
there could be in this
29
00:01:31,000 --> 00:01:32,300
ointment last time.
30
00:01:32,300 --> 00:01:36,710
And one of you alertly reported
that we still really
31
00:01:36,710 --> 00:01:38,990
hadn't worried about decoding.
32
00:01:38,990 --> 00:01:42,290
And the implicit decoding
algorithm here is minimum
33
00:01:42,290 --> 00:01:43,830
distance decoding.
34
00:01:43,830 --> 00:01:45,640
Exhaustive minimum distance
decoding.
35
00:01:45,640 --> 00:01:48,980
We basically have to compute the
minimum distance to each
36
00:01:48,980 --> 00:01:51,250
of the 2 to the k code words.
37
00:01:51,250 --> 00:01:54,100
And that's going to get a little
tedious as soon as k
38
00:01:54,100 --> 00:01:58,210
gets up to 16 or 107
or whatever.
39
00:01:58,210 --> 00:02:03,220
So it was recognized pretty
early in the development of
40
00:02:03,220 --> 00:02:06,170
coding theory that this was
the essential problem.
41
00:02:06,170 --> 00:02:09,340
The problem is not to construct
codes whose
42
00:02:09,340 --> 00:02:11,920
performance under maximum
likelihood decoding is near
43
00:02:11,920 --> 00:02:13,450
that of the Shannon limit
-- you can use
44
00:02:13,450 --> 00:02:16,030
random codes for that.
45
00:02:16,030 --> 00:02:19,950
But in constructing these more
algebraically structured
46
00:02:19,950 --> 00:02:23,610
codes, what we're really trying
to do is to facilitate
47
00:02:23,610 --> 00:02:26,640
the decoding problem, which
is the central problem.
48
00:02:26,640 --> 00:02:30,170
We need to find a
decoding method
49
00:02:30,170 --> 00:02:31,820
that is actually feasible.
50
00:02:31,820 --> 00:02:35,010
And, of course, Shannon didn't
address that at all.
51
00:02:35,010 --> 00:02:38,540
He just addressed the ultimate
potential of these codes with
52
00:02:38,540 --> 00:02:41,740
the optimal decoding technique
and didn't worry about its
53
00:02:41,740 --> 00:02:44,200
complexity.
54
00:02:44,200 --> 00:02:44,640
All right.
55
00:02:44,640 --> 00:02:49,560
So I'd say the fundamental
problem here is decoding
56
00:02:49,560 --> 00:02:50,810
complexity.
57
00:02:50,810 --> 00:02:52,770
58
00:02:52,770 --> 00:02:55,270
Now, one of the --
59
00:02:55,270 --> 00:03:00,080
an awful lot of the early work
on coding was about binary
60
00:03:00,080 --> 00:03:01,490
codes, of course.
61
00:03:01,490 --> 00:03:10,070
But it also considered them in
the context of binary decoding
62
00:03:10,070 --> 00:03:10,730
algorithms.
63
00:03:10,730 --> 00:03:13,450
In other words, what's assumed
is what you send as a series
64
00:03:13,450 --> 00:03:16,890
of bits and what you receive
as a series of bits.
65
00:03:16,890 --> 00:03:19,700
That's not the picture we have
here in this course, where
66
00:03:19,700 --> 00:03:24,050
what we receive is a series
of real numbers.
67
00:03:24,050 --> 00:03:26,400
Because we're going over a
continuous additive white
68
00:03:26,400 --> 00:03:28,115
Gaussian noise channel.
69
00:03:28,115 --> 00:03:31,750
Of course, there are channels
which are inherently digital,
70
00:03:31,750 --> 00:03:34,610
at least by the time the coding
engineer can get his
71
00:03:34,610 --> 00:03:35,610
hands on them.
72
00:03:35,610 --> 00:03:38,400
What he's presented with
is a string of bits.
73
00:03:38,400 --> 00:03:43,390
So it makes sense to consider
codes for bit error
74
00:03:43,390 --> 00:03:45,120
correction.
75
00:03:45,120 --> 00:03:47,980
Classic codes of this type
are, first of all, the
76
00:03:47,980 --> 00:03:49,670
repetition code.
77
00:03:49,670 --> 00:03:51,680
If I send this --
78
00:03:51,680 --> 00:03:56,190
if I use the 7 1 7 code, the
repetition code of length 7,
79
00:03:56,190 --> 00:03:59,370
I'm going to be able to correct
up to three errors.
80
00:03:59,370 --> 00:04:01,450
Three bit errors.
81
00:04:01,450 --> 00:04:03,680
Because four bits are still
going to be correct.
82
00:04:03,680 --> 00:04:06,480
In a majority vote, we'll
do the decoding.
83
00:04:06,480 --> 00:04:08,580
Most of you have probably
seen Hamming codes.
84
00:04:08,580 --> 00:04:12,860
These are codes with minimum
Hamming distance 3.
85
00:04:12,860 --> 00:04:16,649
That means if you make one
error, you're still within the
86
00:04:16,649 --> 00:04:20,240
Hamming sphere of radius 1
around the original code word.
87
00:04:20,240 --> 00:04:25,070
That's disjoined from all the
other Hamming spheres, and so
88
00:04:25,070 --> 00:04:27,670
in principle you ought to be
able to do single error
89
00:04:27,670 --> 00:04:30,560
correction with Hamming codes.
90
00:04:30,560 --> 00:04:34,180
And there are very simple
algorithms for doing that.
91
00:04:34,180 --> 00:04:34,590
OK.
92
00:04:34,590 --> 00:04:38,600
But that may be what
you have to do.
93
00:04:38,600 --> 00:04:42,450
The final point of chapter six
is that in the additive white
94
00:04:42,450 --> 00:04:47,090
Gaussian noise context,
that's not what you
95
00:04:47,090 --> 00:04:48,520
want to do at all.
96
00:04:48,520 --> 00:04:51,120
And I just want to say a few
words about that because this
97
00:04:51,120 --> 00:04:55,140
is one of the Achille's heels
of a lot of classical coding
98
00:04:55,140 --> 00:04:57,140
theory, which was addressed
to the binary
99
00:04:57,140 --> 00:05:01,340
in, binary out situation.
100
00:05:01,340 --> 00:05:05,040
So let's talk about the penalty
101
00:05:05,040 --> 00:05:06,455
for making hard decisions.
102
00:05:06,455 --> 00:05:12,260
103
00:05:12,260 --> 00:05:15,880
That is, by hard decision, we
mean a binary decision.
104
00:05:15,880 --> 00:05:17,130
0, 1, plus or minus 1.
105
00:05:17,130 --> 00:05:21,030
106
00:05:21,030 --> 00:05:25,390
And I'll do this fairly quickly,
but in a way that I
107
00:05:25,390 --> 00:05:28,390
hope will stick.
108
00:05:28,390 --> 00:05:33,800
Let me draw our block diagram
of our coding scheme.
109
00:05:33,800 --> 00:05:35,870
We've got a code.
110
00:05:35,870 --> 00:05:42,820
We've got an encoder,
which basically
111
00:05:42,820 --> 00:05:45,850
puts out a code word.
112
00:05:45,850 --> 00:05:47,490
An n-tuple in the code.
113
00:05:47,490 --> 00:05:51,060
I'm going to consider
the individual
114
00:05:51,060 --> 00:05:53,230
elements of this n-tuple.
115
00:05:53,230 --> 00:05:58,810
They're just elements of the
binary field F2 And we go
116
00:05:58,810 --> 00:06:05,680
through our standard 2-PAM map,
which maps them to s of
117
00:06:05,680 --> 00:06:10,210
xk, which is going
to be in plus or
118
00:06:10,210 --> 00:06:11,750
minus alpha in general.
119
00:06:11,750 --> 00:06:13,950
Just plus or minus 1.
120
00:06:13,950 --> 00:06:16,240
Doesn't matter up to scaling.
121
00:06:16,240 --> 00:06:19,110
In order to send them, we're
going to send this sequence of
122
00:06:19,110 --> 00:06:25,540
real numbers over the white
Gaussian noise channel.
123
00:06:25,540 --> 00:06:29,610
And as a result, we're
going to get out s --
124
00:06:29,610 --> 00:06:31,780
let's call it a received
symbol --
125
00:06:31,780 --> 00:06:37,950
rk, which is the transmitted
symbol plus some Gaussian
126
00:06:37,950 --> 00:06:40,270
noise symbol, nk.
127
00:06:40,270 --> 00:06:43,045
This is a real number.
128
00:06:43,045 --> 00:06:46,330
129
00:06:46,330 --> 00:06:48,450
And then --
130
00:06:48,450 --> 00:06:49,240
then what we do?
131
00:06:49,240 --> 00:06:50,950
Now we're into the decoder.
132
00:06:50,950 --> 00:06:54,610
This is what we received, so
it's free to us to choose what
133
00:06:54,610 --> 00:06:55,860
to do next.
134
00:06:55,860 --> 00:06:58,510
135
00:06:58,510 --> 00:07:01,590
I'm going to say observe, first
of all, that without
136
00:07:01,590 --> 00:07:07,630
loss of generality, we
can decompose rk
137
00:07:07,630 --> 00:07:11,040
into a sign and magnitude.
138
00:07:11,040 --> 00:07:17,910
So this out here is going
to be a sign of rk.
139
00:07:17,910 --> 00:07:23,160
And this is going to be
the magnitude of rk.
140
00:07:23,160 --> 00:07:25,300
And let me just give
this a name.
141
00:07:25,300 --> 00:07:31,120
Let me call this yk, and let
me call this beta k.
142
00:07:31,120 --> 00:07:31,600
OK.
143
00:07:31,600 --> 00:07:32,610
So what is this?
144
00:07:32,610 --> 00:07:40,000
This is in plus or minus 1, and
this is in the positive
145
00:07:40,000 --> 00:07:41,906
reals, so the non-negative
reals.
146
00:07:41,906 --> 00:07:44,460
147
00:07:44,460 --> 00:07:44,580
OK.
148
00:07:44,580 --> 00:07:48,020
And clearly, if I keep both of
these, I haven't lost any
149
00:07:48,020 --> 00:07:48,740
information.
150
00:07:48,740 --> 00:07:51,770
This is just a way of
representing rk in sign and
151
00:07:51,770 --> 00:07:54,290
magnitude form.
152
00:07:54,290 --> 00:07:57,735
Now, this is what's called
the hard decision.
153
00:07:57,735 --> 00:08:01,160
154
00:08:01,160 --> 00:08:02,855
Actually, let me do
this differently.
155
00:08:02,855 --> 00:08:11,130
Let me pass this through the
inverse to a 2-PAM map.
156
00:08:11,130 --> 00:08:13,610
So out of this, I get a yk,
157
00:08:13,610 --> 00:08:17,540
which is in F2 OK.
158
00:08:17,540 --> 00:08:18,580
Either of these --
159
00:08:18,580 --> 00:08:24,680
the real sign or the binary
hard decision --
160
00:08:24,680 --> 00:08:28,240
could be considered to
be the hard decision.
161
00:08:28,240 --> 00:08:29,240
Now, what am I doing here?
162
00:08:29,240 --> 00:08:32,720
I'm saying, well, if you force
me to make a guess on whether
163
00:08:32,720 --> 00:08:36,710
this individual transmission was
sending a 0 or 1, this is
164
00:08:36,710 --> 00:08:39,050
the most obvious guess.
165
00:08:39,050 --> 00:08:42,839
If I send --
166
00:08:42,839 --> 00:08:50,100
here's minus 1, here's our minus
alpha, here's plus 1 --
167
00:08:50,100 --> 00:08:55,400
if I sent a plus 1, then my
maximum likelihood per bit
168
00:08:55,400 --> 00:08:58,710
decision would be to decide
the plus 1 was sent.
169
00:08:58,710 --> 00:09:02,940
If I get a positive sign in --
170
00:09:02,940 --> 00:09:07,360
what I'm really mapping here is
rk, and minus 1 if I get a
171
00:09:07,360 --> 00:09:08,100
negative side.
172
00:09:08,100 --> 00:09:09,550
So that's what's called
a hard decision.
173
00:09:09,550 --> 00:09:12,530
I'm going to decide right here
what I think it was, what it
174
00:09:12,530 --> 00:09:13,570
most likely was.
175
00:09:13,570 --> 00:09:15,260
And this is the best
way to do it.
176
00:09:15,260 --> 00:09:17,410
Just take the sign.
177
00:09:17,410 --> 00:09:22,740
This here is a real valued
weight, often called the
178
00:09:22,740 --> 00:09:30,235
reliability, which is
helpful to retain.
179
00:09:30,235 --> 00:09:32,830
That's the main point
of what I'm going
180
00:09:32,830 --> 00:09:35,550
to talk to you about.
181
00:09:35,550 --> 00:09:36,990
If this is 0 --
182
00:09:36,990 --> 00:09:40,900
that means what we received
was right on this boundary
183
00:09:40,900 --> 00:09:44,390
between the positive and
negative side --
184
00:09:44,390 --> 00:09:46,980
then the reliability is 0.
185
00:09:46,980 --> 00:09:49,565
In likelihood terms, that means
it's equally likely that
186
00:09:49,565 --> 00:09:52,540
what was sent was a plus
1 or a minus 1.
187
00:09:52,540 --> 00:09:55,520
So in some sense, we get
no information when the
188
00:09:55,520 --> 00:09:57,900
reliability is 0.
189
00:09:57,900 --> 00:10:01,080
Fails to discriminate between
plus 1 and minus 1.
190
00:10:01,080 --> 00:10:03,880
The larger this is, the further
we get out here.
191
00:10:03,880 --> 00:10:07,710
Or symmetrically, the further
we get out here, the more
192
00:10:07,710 --> 00:10:12,000
certain we are that in that
particular symbol, neither a
193
00:10:12,000 --> 00:10:15,830
plus or a minus was sent.
194
00:10:15,830 --> 00:10:18,500
OK And it actually tuRns out,
if you work out the Gaussian
195
00:10:18,500 --> 00:10:24,750
numbers, that beta k is the
log likelihood ratio up to
196
00:10:24,750 --> 00:10:28,830
scale of the more likely
versus the log of the
197
00:10:28,830 --> 00:10:32,920
likelihood, or the more likely
versus the less likely symbol.
198
00:10:32,920 --> 00:10:35,010
So it has a minimum of 0.
199
00:10:35,010 --> 00:10:36,810
The bigger it is, the
more reliable.
200
00:10:36,810 --> 00:10:38,630
So it's natural to call
it the reliability.
201
00:10:38,630 --> 00:10:41,280
202
00:10:41,280 --> 00:10:42,770
OK.
203
00:10:42,770 --> 00:10:48,470
An awful lot of traditional
algebraic decoding neglects
204
00:10:48,470 --> 00:10:50,910
this channel down here.
205
00:10:50,910 --> 00:10:53,880
It just says, OK, let's take
these hard decisions and try
206
00:10:53,880 --> 00:10:55,430
to decode them.
207
00:10:55,430 --> 00:10:58,510
If you've previously had a
coding class, then that's
208
00:10:58,510 --> 00:11:00,000
probably what they did.
209
00:11:00,000 --> 00:11:03,020
Or they assumed a context in
which this was never available
210
00:11:03,020 --> 00:11:05,900
in the first place.
211
00:11:05,900 --> 00:11:10,110
And my only point here, my main
point, is that that's a
212
00:11:10,110 --> 00:11:13,150
very bad thing to do.
213
00:11:13,150 --> 00:11:16,420
How can we evaluate that?
214
00:11:16,420 --> 00:11:20,960
Suppose we just take this top
channel and don't look at the
215
00:11:20,960 --> 00:11:21,720
reliability.
216
00:11:21,720 --> 00:11:25,570
Then basically we have the
channel model becomes a binary
217
00:11:25,570 --> 00:11:26,550
symmetric channel.
218
00:11:26,550 --> 00:11:35,640
We have bits in, bits out, and
we have probably one minus p
219
00:11:35,640 --> 00:11:38,610
that if we send a
0 we get a 0.
220
00:11:38,610 --> 00:11:41,830
And since it's symmetric,
the same probability --
221
00:11:41,830 --> 00:11:43,690
if we send a 1, we get a 1 --
222
00:11:43,690 --> 00:11:45,610
and a probability p of
making an error.
223
00:11:45,610 --> 00:11:48,880
So we get the traditional
memoryless binary symmetric
224
00:11:48,880 --> 00:11:50,580
channel model.
225
00:11:50,580 --> 00:11:54,290
We can compute the capacity of
this model and compare it to
226
00:11:54,290 --> 00:11:59,590
the capacity of the additive
white Gaussian channel --
227
00:11:59,590 --> 00:12:01,960
this model.
228
00:12:01,960 --> 00:12:05,930
And depending on the signal to
noise ratio, from a capacity
229
00:12:05,930 --> 00:12:10,880
calculation, we find that
there is of the order
230
00:12:10,880 --> 00:12:14,880
of 2 or 3 dB loss.
231
00:12:14,880 --> 00:12:17,640
Well, as we go on in this
course, we're going to find
232
00:12:17,640 --> 00:12:21,250
that 1 dB is a very worthwhile
coding game.
233
00:12:21,250 --> 00:12:24,600
So to throw away 2 or 3 dB just
by throwing away this
234
00:12:24,600 --> 00:12:28,810
information, is a
bad thing to do.
235
00:12:28,810 --> 00:12:33,280
236
00:12:33,280 --> 00:12:39,970
I can make this point in a
different way by simply
237
00:12:39,970 --> 00:12:45,650
looking at what's the optimum
decision rule.
238
00:12:45,650 --> 00:12:50,170
For say, let me take very simple
code, just the 2-1-2
239
00:12:50,170 --> 00:12:51,170
repetition code.
240
00:12:51,170 --> 00:12:54,010
That's just 0,0,1,1.
241
00:12:54,010 --> 00:12:58,720
This is the code where you
either send plus, plus, plus
242
00:12:58,720 --> 00:13:04,910
alpha, plus alpha, or minus
alpha, minus alpha.
243
00:13:04,910 --> 00:13:09,045
And what's the decision
region?
244
00:13:09,045 --> 00:13:12,570
It's obviously this
45 degree line.
245
00:13:12,570 --> 00:13:15,800
And what is this squared
distance to
246
00:13:15,800 --> 00:13:17,330
any decision region?
247
00:13:17,330 --> 00:13:19,935
It's 2 alpha squared, right?
248
00:13:19,935 --> 00:13:23,190
249
00:13:23,190 --> 00:13:26,560
So basically, the probability
of making an error is the
250
00:13:26,560 --> 00:13:31,140
probability that the noise
variable has a magnitude in
251
00:13:31,140 --> 00:13:34,980
this dimension, and in this
direction, greater than the
252
00:13:34,980 --> 00:13:38,170
square root of 2
alpha squared.
253
00:13:38,170 --> 00:13:42,320
You've been through all those
calculation several times now.
254
00:13:42,320 --> 00:13:42,710
All right.
255
00:13:42,710 --> 00:13:47,710
So this is -- if I keep
reliability info -- in other
256
00:13:47,710 --> 00:13:53,150
words, if I keep the full
received signal -- if I
257
00:13:53,150 --> 00:14:02,200
discard reliability, then
basically what can I get out?
258
00:14:02,200 --> 00:14:07,760
Then my y, as I say, is in
0, 0, 0, 1, 1, 0, 1, 1.
259
00:14:07,760 --> 00:14:10,590
260
00:14:10,590 --> 00:14:14,000
There are only four possible
things I can see in two
261
00:14:14,000 --> 00:14:15,910
transmissions through
this channel.
262
00:14:15,910 --> 00:14:19,650
They're all binary two-tuples.
263
00:14:19,650 --> 00:14:22,420
And what does that mean?
264
00:14:22,420 --> 00:14:29,870
That means if my actual r is
in this quadrant, then I'm
265
00:14:29,870 --> 00:14:34,510
going to make a hard decision
of 0 and 0 both times.
266
00:14:34,510 --> 00:14:37,280
So I'm going to say I'm
in this quadrant.
267
00:14:37,280 --> 00:14:41,480
And likewise, I'm basically
going to decide which of these
268
00:14:41,480 --> 00:14:44,230
four quadrants I'm in, and
that's all the information the
269
00:14:44,230 --> 00:14:45,480
decoder is going to have.
270
00:14:45,480 --> 00:14:48,010
271
00:14:48,010 --> 00:14:51,370
So here I am at this
point here.
272
00:14:51,370 --> 00:14:55,200
Now the decoder simply knows,
via these two bits, which
273
00:14:55,200 --> 00:14:57,980
quadrant you're in.
274
00:14:57,980 --> 00:14:59,800
Now, it has to decide
which of these two
275
00:14:59,800 --> 00:15:01,960
code words were sent.
276
00:15:01,960 --> 00:15:06,040
What's its maximum likelihood
decision rule given just this
277
00:15:06,040 --> 00:15:07,290
information?
278
00:15:07,290 --> 00:15:11,710
279
00:15:11,710 --> 00:15:13,646
AUDIENCE: 0, 0, [INAUDIBLE]
280
00:15:13,646 --> 00:15:15,098
0, 0, [INAUDIBLE]
281
00:15:15,098 --> 00:15:17,276
1, 1, 1, 1, 1.
282
00:15:17,276 --> 00:15:17,518
And in the other
case, any one.
283
00:15:17,518 --> 00:15:18,680
PROFESSOR: OK.
284
00:15:18,680 --> 00:15:20,310
That's correct.
285
00:15:20,310 --> 00:15:22,550
So clearly, if you land
in this quadrant,
286
00:15:22,550 --> 00:15:28,200
you decide 0, 0.
287
00:15:28,200 --> 00:15:30,570
Now, if you land down
here, you decide
288
00:15:30,570 --> 00:15:34,190
1, 1 in this quadrant.
289
00:15:34,190 --> 00:15:37,750
And what am I going to
do here or here?
290
00:15:37,750 --> 00:15:40,360
Actually, here the evidence
is totally balanced.
291
00:15:40,360 --> 00:15:42,370
I have nothing that tells
me whether to
292
00:15:42,370 --> 00:15:44,530
go one way or another.
293
00:15:44,530 --> 00:15:46,990
I could make an arbitrary
decision in the hope of
294
00:15:46,990 --> 00:15:48,315
minimizing my error
probability.
295
00:15:48,315 --> 00:15:50,970
Flip a coin.
296
00:15:50,970 --> 00:15:56,496
But whichever decision I make,
what I'm going to find is that
297
00:15:56,496 --> 00:16:02,620
there's a probability of error
that -- there's a certain
298
00:16:02,620 --> 00:16:06,320
noise, namely the noise that
takes me from here to this
299
00:16:06,320 --> 00:16:09,130
decision boundary that is
going to cause an error.
300
00:16:09,130 --> 00:16:11,630
Or if I decide the other way, it
would be the one that goes
301
00:16:11,630 --> 00:16:13,780
from here to here.
302
00:16:13,780 --> 00:16:14,920
So I'm going to be stuck.
303
00:16:14,920 --> 00:16:20,440
There are going to be certain
noise variables of length,
304
00:16:20,440 --> 00:16:26,790
now, merely alpha, or square
distance alpha squared, that
305
00:16:26,790 --> 00:16:30,290
are going to cause me to make a
decision error, ultimately.
306
00:16:30,290 --> 00:16:36,710
Regardless of how I set up
this final block here.
307
00:16:36,710 --> 00:16:38,871
Decoder.
308
00:16:38,871 --> 00:16:42,650
Whatever rule I give to this
decoder, it's only going to
309
00:16:42,650 --> 00:16:48,730
take, worst case, a noise of
squared magnitude alpha
310
00:16:48,730 --> 00:16:51,340
squared to cause an error.
311
00:16:51,340 --> 00:16:52,590
AUDIENCE:
[UNINTELLIGIBLE PHRASE]
312
00:16:52,590 --> 00:16:55,561
313
00:16:55,561 --> 00:16:56,830
PROFESSOR: That's right.
314
00:16:56,830 --> 00:16:59,420
I shouldn't have thrown
this away.
315
00:16:59,420 --> 00:17:03,250
That's the elementary point
I'm trying to make here.
316
00:17:03,250 --> 00:17:03,840
All right.
317
00:17:03,840 --> 00:17:06,640
AUDIENCE:
[UNINTELLIGIBLE PHRASE]
318
00:17:06,640 --> 00:17:10,432
the distance, d_min is
different, essentially.
319
00:17:10,432 --> 00:17:11,869
I mean, we can put
it that way.
320
00:17:11,869 --> 00:17:16,020
PROFESSOR: That's where I'm
going, is that the effective
321
00:17:16,020 --> 00:17:18,690
minimum squared distance here
is 2 alpha squared.
322
00:17:18,690 --> 00:17:21,530
And this is what shows up in
our union-bound estimate in
323
00:17:21,530 --> 00:17:24,250
all of our --
324
00:17:24,250 --> 00:17:26,410
we do this kind of minimum
distance decoding.
325
00:17:26,410 --> 00:17:30,080
But if I throw away this
important information, then my
326
00:17:30,080 --> 00:17:31,745
effective minimum
squared distance
327
00:17:31,745 --> 00:17:32,980
is only alpha squared.
328
00:17:32,980 --> 00:17:35,490
That is the bottom line here.
329
00:17:35,490 --> 00:17:40,490
In dB terms, how much
of a cost is that?
330
00:17:40,490 --> 00:17:41,630
3 dB, right?
331
00:17:41,630 --> 00:17:46,000
So I've cost myself a factor
of 2 in noise margin.
332
00:17:46,000 --> 00:17:48,200
So 3 dB loss.
333
00:17:48,200 --> 00:17:52,490
334
00:17:52,490 --> 00:18:00,720
Because of lack of time, I won't
go through the argument
335
00:18:00,720 --> 00:18:05,810
in the notes, which shows that,
in fact, exactly the
336
00:18:05,810 --> 00:18:08,160
same thing occurs for any --
337
00:18:08,160 --> 00:18:11,220
whenever the minimum Hamming
distance is even here in this
338
00:18:11,220 --> 00:18:15,770
code that I started from,
you lose precisely 3 dB.
339
00:18:15,770 --> 00:18:18,850
It's not quite as clean an
argument when the minimum
340
00:18:18,850 --> 00:18:20,460
Hamming distance is odd.
341
00:18:20,460 --> 00:18:24,790
Then you lose up to 3 dB, and
it goes to 3 dB as the
342
00:18:24,790 --> 00:18:26,580
distance increases.
343
00:18:26,580 --> 00:18:31,030
But there's a very, pretty
elementary geometric argument
344
00:18:31,030 --> 00:18:35,010
that hard decisions, again, cost
you 3 dB loss, which is
345
00:18:35,010 --> 00:18:36,810
consistent with what
the capacity
346
00:18:36,810 --> 00:18:38,410
calculation gives you.
347
00:18:38,410 --> 00:18:41,780
348
00:18:41,780 --> 00:18:42,320
All right.
349
00:18:42,320 --> 00:18:47,440
Well, obviously the reason we
did this is we were trying to
350
00:18:47,440 --> 00:18:49,690
simplify things.
351
00:18:49,690 --> 00:18:54,840
What would be the first step to
unsimplify things and get
352
00:18:54,840 --> 00:18:56,170
some of this loss back?
353
00:18:56,170 --> 00:19:03,200
354
00:19:03,200 --> 00:19:07,060
Let me suggest that what this
amounts to is a two-level
355
00:19:07,060 --> 00:19:12,240
quantization of the received
real number rk.
356
00:19:12,240 --> 00:19:15,440
What's the next number
higher than 2?
357
00:19:15,440 --> 00:19:15,920
Integer.
358
00:19:15,920 --> 00:19:17,340
AUDIENCE: 3.
359
00:19:17,340 --> 00:19:18,550
PROFESSOR: 3.
360
00:19:18,550 --> 00:19:19,260
All right.
361
00:19:19,260 --> 00:19:23,860
How about a 3 level
quantization here?
362
00:19:23,860 --> 00:19:25,735
OK.
363
00:19:25,735 --> 00:19:31,880
So what we're talking about is
a highly quantized magnitude
364
00:19:31,880 --> 00:19:41,480
where instead of just making a
decision boundary here, which
365
00:19:41,480 --> 00:19:45,350
is effectively what we did for
2 level quantization, let's
366
00:19:45,350 --> 00:19:53,620
make some null zone here between
some threshold plus t
367
00:19:53,620 --> 00:19:57,550
and some threshold minus t.
368
00:19:57,550 --> 00:20:02,920
And we'll say, in this region,
the received symbol is
369
00:20:02,920 --> 00:20:05,370
unreliable.
370
00:20:05,370 --> 00:20:05,810
OK.
371
00:20:05,810 --> 00:20:07,870
That's called an erasure.
372
00:20:07,870 --> 00:20:11,640
So we make a quantized
magnitude where rk --
373
00:20:11,640 --> 00:20:18,700
374
00:20:18,700 --> 00:20:21,490
doesn't fit so neatly here.
375
00:20:21,490 --> 00:20:26,020
Let me just say this quantized
magnitude is going to give me
376
00:20:26,020 --> 00:20:29,350
out something which is either
going to be minus 1,
377
00:20:29,350 --> 00:20:31,300
a 0, or plus 1.
378
00:20:31,300 --> 00:20:37,440
It's going to be three levels,
this is minus 1, 0, plus 1.
379
00:20:37,440 --> 00:20:40,850
Or I don't want the
minus 1 there.
380
00:20:40,850 --> 00:20:44,817
The reliability is either a
fixed reliability, or it's 0.
381
00:20:44,817 --> 00:20:46,067
OK.
382
00:20:46,067 --> 00:20:51,870
383
00:20:51,870 --> 00:20:56,660
So now I've got a channel model
that looks like this.
384
00:20:56,660 --> 00:21:02,690
If you work it out, it's called
the binary erasure
385
00:21:02,690 --> 00:21:04,380
channel with errors.
386
00:21:04,380 --> 00:21:06,210
All transitions are possible.
387
00:21:06,210 --> 00:21:08,010
I can send a 0,1.
388
00:21:08,010 --> 00:21:13,500
I can receive, let's call it,
a 0, a 1, or a question mark
389
00:21:13,500 --> 00:21:15,465
for -- this is called
an erasure.
390
00:21:15,465 --> 00:21:18,650
391
00:21:18,650 --> 00:21:23,540
And this is 1 minus p minus q,
and this is p and this is q
392
00:21:23,540 --> 00:21:30,080
symmetrically, for some p and
q, which you can evaluate.
393
00:21:30,080 --> 00:21:35,440
And let's choose this threshold
t to optimize things
394
00:21:35,440 --> 00:21:37,920
and get the maximum capacity.
395
00:21:37,920 --> 00:21:40,530
Now if you do the capacity
calculation, you'll find
396
00:21:40,530 --> 00:21:44,510
there's only 1 to 1.5 dB loss.
397
00:21:44,510 --> 00:21:47,530
Again, depending on the
signal to noise ratio.
398
00:21:47,530 --> 00:21:50,320
So in effect, you've already
bought back, just with this
399
00:21:50,320 --> 00:21:58,970
very simple method, half of the
loss that you inflicted on
400
00:21:58,970 --> 00:22:01,840
yourself by making
hard decisions.
401
00:22:01,840 --> 00:22:08,930
So making erasures is a good
first step towards correcting
402
00:22:08,930 --> 00:22:11,770
this problem.
403
00:22:11,770 --> 00:22:15,520
And let's think about it again
for this simple code.
404
00:22:15,520 --> 00:22:22,780
Suppose I establish these
thresholds at
405
00:22:22,780 --> 00:22:24,380
plus t and minus t.
406
00:22:24,380 --> 00:22:27,440
Plus t and minus t.
407
00:22:27,440 --> 00:22:33,890
So now when I consider the
problem that the decoder has,
408
00:22:33,890 --> 00:22:38,300
I have nine possible regions,
all right, each with three
409
00:22:38,300 --> 00:22:44,370
possible decisions in
two dimensions.
410
00:22:44,370 --> 00:22:47,150
And so what's my decision
rule going to be now?
411
00:22:47,150 --> 00:22:50,820
412
00:22:50,820 --> 00:22:55,050
For this 0, 0 decision, I'm
going to include, of course,
413
00:22:55,050 --> 00:22:56,290
this region.
414
00:22:56,290 --> 00:23:01,750
But now if I land in this
region, that means one of the
415
00:23:01,750 --> 00:23:04,210
received symbols was erased,
but the other one gave me a
416
00:23:04,210 --> 00:23:05,690
definite indication.
417
00:23:05,690 --> 00:23:09,020
So the weight of evidence
still goes up here.
418
00:23:09,020 --> 00:23:13,300
And similarly, like so.
419
00:23:13,300 --> 00:23:16,380
So my decision region for 0 --
420
00:23:16,380 --> 00:23:20,740
definitely for 0, 0
is this region.
421
00:23:20,740 --> 00:23:26,900
And my decision region
definitely for 1, 1 is the
422
00:23:26,900 --> 00:23:29,750
symmetric region.
423
00:23:29,750 --> 00:23:32,590
And, of course, I still have
three regions where I really
424
00:23:32,590 --> 00:23:33,520
can't say anything.
425
00:23:33,520 --> 00:23:35,360
This is two erasures.
426
00:23:35,360 --> 00:23:36,720
This is 0, 1 --
427
00:23:36,720 --> 00:23:38,560
two conflicting pieces
of evidence.
428
00:23:38,560 --> 00:23:39,660
This is 1, 0 -- two
429
00:23:39,660 --> 00:23:42,360
conflicting pieces of evidence.
430
00:23:42,360 --> 00:23:48,130
So I still have to flip
a coin out in here.
431
00:23:48,130 --> 00:23:53,550
But how I improve things, that's
measured by what's the
432
00:23:53,550 --> 00:23:54,880
minimum distance to --
433
00:23:54,880 --> 00:23:57,820
434
00:23:57,820 --> 00:24:01,770
what's the minimum size of
error it takes to make a
435
00:24:01,770 --> 00:24:03,450
decision error.
436
00:24:03,450 --> 00:24:08,550
And that's going to be either
this length or this length.
437
00:24:08,550 --> 00:24:09,400
You see that?
438
00:24:09,400 --> 00:24:11,125
Just pretty clear,
intuitively?
439
00:24:11,125 --> 00:24:14,220
440
00:24:14,220 --> 00:24:18,040
So the game here is we first of
all choose t so that these
441
00:24:18,040 --> 00:24:21,110
two lengths are the same.
442
00:24:21,110 --> 00:24:23,750
That's done by squeezing t.
443
00:24:23,750 --> 00:24:25,935
You see these go in opposite
directions as t
444
00:24:25,935 --> 00:24:27,540
is raised or lowered.
445
00:24:27,540 --> 00:24:31,100
So we find the t such that
these two are the same.
446
00:24:31,100 --> 00:24:37,050
Having equalized t, we find some
value for the effective
447
00:24:37,050 --> 00:24:40,000
minimum squared distance, which
is between alpha squared
448
00:24:40,000 --> 00:24:41,800
and 2 alpha squared.
449
00:24:41,800 --> 00:24:50,310
As I remember, this somehow
gains about 1.5 dB.
450
00:24:50,310 --> 00:24:51,390
The moral is about the same.
451
00:24:51,390 --> 00:24:55,880
You can get about half of your
loss back by using erasures.
452
00:24:55,880 --> 00:24:59,990
And again, this holds for any
code which has an even minimum
453
00:24:59,990 --> 00:25:02,060
Hamming distance, if
you can accept
454
00:25:02,060 --> 00:25:04,640
this method of analysis.
455
00:25:04,640 --> 00:25:05,190
All right.
456
00:25:05,190 --> 00:25:10,870
So a first step, even if you
want to stay in the binary
457
00:25:10,870 --> 00:25:14,830
world, is to allow yourself
to use erasures as well.
458
00:25:14,830 --> 00:25:20,170
That will claw back half of the
loss that you might have
459
00:25:20,170 --> 00:25:22,240
incurred by using
hard decisions.
460
00:25:22,240 --> 00:25:25,520
And, of course, even better
would be to use a more highly
461
00:25:25,520 --> 00:25:29,500
quantized reliability and
somehow have a decoding
462
00:25:29,500 --> 00:25:33,420
algorithm that can use
soft decisions.
463
00:25:33,420 --> 00:25:37,180
Soft decisions are decisions
that have reliability metrics
464
00:25:37,180 --> 00:25:39,900
attached to them.
465
00:25:39,900 --> 00:25:43,770
And in the early
days of coding,
466
00:25:43,770 --> 00:25:46,190
people evaluated capacity.
467
00:25:46,190 --> 00:25:48,420
They evaluated this
sort of thing.
468
00:25:48,420 --> 00:25:55,780
And it was pretty generally
agreed that eight level
469
00:25:55,780 --> 00:25:59,090
quantization was going to be
practically good enough.
470
00:25:59,090 --> 00:26:00,950
16 levels, certainly
good enough.
471
00:26:00,950 --> 00:26:05,070
Nowadays you typically
go up to 64 levels.
472
00:26:05,070 --> 00:26:07,130
You might go up to much higher
numbers if you have little
473
00:26:07,130 --> 00:26:09,050
other things to worry
about like timing
474
00:26:09,050 --> 00:26:10,760
recovery and so forth.
475
00:26:10,760 --> 00:26:14,620
But if you're purely in a
synchronized, symbol by symbol
476
00:26:14,620 --> 00:26:19,150
transmission, then three or
four bits of reliability
477
00:26:19,150 --> 00:26:21,210
information are going
to be enough.
478
00:26:21,210 --> 00:26:24,510
So from an engineering
point of view, that's
479
00:26:24,510 --> 00:26:25,430
a good way to go.
480
00:26:25,430 --> 00:26:28,820
Now, of course, you're going to
need a decoding algorithm
481
00:26:28,820 --> 00:26:32,440
that can use soft decisions.
482
00:26:32,440 --> 00:26:36,030
So as we go along, I'm going to
talk about error correcting
483
00:26:36,030 --> 00:26:41,200
decoding algorithms that are not
much, because correcting
484
00:26:41,200 --> 00:26:43,830
errors only is a very bad
thing to do here.
485
00:26:43,830 --> 00:26:44,990
Errors and erasure.
486
00:26:44,990 --> 00:26:48,420
Correcting decoding algorithms,
which are fairly
487
00:26:48,420 --> 00:26:52,850
easy in algebraic
code context.
488
00:26:52,850 --> 00:26:57,170
And then finally soft decision
decoding algorithms, which is
489
00:26:57,170 --> 00:26:59,290
the kind we really want
on this channel.
490
00:26:59,290 --> 00:27:01,210
We really want to use
the soft decisions.
491
00:27:01,210 --> 00:27:04,070
We can't afford these
huge losses.
492
00:27:04,070 --> 00:27:08,710
We can't even afford 1 to
1 and a 1/2 dB loss.
493
00:27:08,710 --> 00:27:14,690
OK, so there's more set on
this in the last part of
494
00:27:14,690 --> 00:27:16,670
chapter six, but that's
all the time I want to
495
00:27:16,670 --> 00:27:18,650
spend on it in class.
496
00:27:18,650 --> 00:27:20,100
Does anyone have
any questions?
497
00:27:20,100 --> 00:27:21,080
yes?
498
00:27:21,080 --> 00:27:22,255
AUDIENCE: [INAUDIBLE]
499
00:27:22,255 --> 00:27:24,626
transformation before we do
the quantization, can we
500
00:27:24,626 --> 00:27:26,754
change the shape of the
[UNINTELLIGIBLE]
501
00:27:26,754 --> 00:27:30,602
regions such that we can
further improve the
502
00:27:30,602 --> 00:27:33,007
[INAUDIBLE PHRASE]
503
00:27:33,007 --> 00:27:35,412
improve the decoding?
504
00:27:35,412 --> 00:27:38,500
PROFESSOR: Well, that's
an interesting idea.
505
00:27:38,500 --> 00:27:41,880
But think about it from
an information
506
00:27:41,880 --> 00:27:43,570
theoretic point of view.
507
00:27:43,570 --> 00:27:48,560
Can you actually get more
information by introducing a
508
00:27:48,560 --> 00:27:51,140
one-to-one transformation,
whether it's linear or
509
00:27:51,140 --> 00:27:53,011
nonlinear, whatever?
510
00:27:53,011 --> 00:27:58,810
AUDIENCE: The point is, the
original quantization is based
511
00:27:58,810 --> 00:28:02,926
on a straight line,
[INAUDIBLE PHRASE].
512
00:28:02,926 --> 00:28:04,176
[UNINTELLIGIBLE PHRASE].
513
00:28:04,176 --> 00:28:06,200
514
00:28:06,200 --> 00:28:08,640
PROFESSOR: You want to put a
curved line on this plane?
515
00:28:08,640 --> 00:28:09,842
AUDIENCE: [INAUDIBLE].
516
00:28:09,842 --> 00:28:12,190
PROFESSOR: OK.
517
00:28:12,190 --> 00:28:14,850
That's going to imply decisions
that are not symbol
518
00:28:14,850 --> 00:28:17,050
by symbol decisions.
519
00:28:17,050 --> 00:28:22,200
That's going to imply that you
need to save r1 and r 2 in
520
00:28:22,200 --> 00:28:26,220
order even to just decide where
you're in this and then
521
00:28:26,220 --> 00:28:30,250
go through some kind of
quantization on the plane.
522
00:28:30,250 --> 00:28:33,170
And once you've got r1 and r2
and you're trying to draw
523
00:28:33,170 --> 00:28:35,610
crazy boundaries, I suggest it's
going to be simpler to
524
00:28:35,610 --> 00:28:38,990
just compute the Euclidean
distance to each of these
525
00:28:38,990 --> 00:28:48,922
points, which results in
this decision region.
526
00:28:48,922 --> 00:28:50,172
AUDIENCE: [INAUDIBLE PHRASE].
527
00:28:50,172 --> 00:28:55,880
528
00:28:55,880 --> 00:28:56,770
PROFESSOR: OK.
529
00:28:56,770 --> 00:28:59,660
I don't see the rationale for
it yet, but there's been a
530
00:28:59,660 --> 00:29:01,570
million innovations in this
business, and you
531
00:29:01,570 --> 00:29:02,820
might have a good idea.
532
00:29:02,820 --> 00:29:09,060
533
00:29:09,060 --> 00:29:11,170
Any other comments?
534
00:29:11,170 --> 00:29:15,242
I really appreciate comments
that go anywhere.
535
00:29:15,242 --> 00:29:17,590
I like to address them now
when they're ripe.
536
00:29:17,590 --> 00:29:21,950
537
00:29:21,950 --> 00:29:23,200
OK.
538
00:29:23,200 --> 00:29:25,570
539
00:29:25,570 --> 00:29:28,520
So let's go on to chapter
seven and eight.
540
00:29:28,520 --> 00:29:29,770
[UNINTELLIGIBLE] here.
541
00:29:29,770 --> 00:29:35,660
542
00:29:35,660 --> 00:29:41,770
Chapters seven and eight
are closely related.
543
00:29:41,770 --> 00:29:45,565
This is really my bow to
algebraic coding theory.
544
00:29:45,565 --> 00:29:49,580
545
00:29:49,580 --> 00:29:58,800
I expect that if any of you
have had a close course in
546
00:29:58,800 --> 00:30:01,370
coding before, that it
was primarily on
547
00:30:01,370 --> 00:30:04,336
algebraic coding theory.
548
00:30:04,336 --> 00:30:06,770
How many of you have
had a course in
549
00:30:06,770 --> 00:30:09,210
coding theory before?
550
00:30:09,210 --> 00:30:10,980
One, two, three.
551
00:30:10,980 --> 00:30:11,960
Not so many.
552
00:30:11,960 --> 00:30:14,680
Was it on algebraic
coding theory?
553
00:30:14,680 --> 00:30:15,120
Yes?
554
00:30:15,120 --> 00:30:16,390
AUDIENCE: I [UNINTELLIGIBLE]
two, one
555
00:30:16,390 --> 00:30:17,731
engineering and one algebraic.
556
00:30:17,731 --> 00:30:18,450
PROFESSOR: Excuse me?
557
00:30:18,450 --> 00:30:21,235
AUDIENCE: Two -- one for --
actually, two algebraic and
558
00:30:21,235 --> 00:30:23,420
one out of kind of this stuff.
559
00:30:23,420 --> 00:30:24,440
PROFESSOR: OK.
560
00:30:24,440 --> 00:30:26,140
Well, you're a ringer then.
561
00:30:26,140 --> 00:30:27,510
What was the one
on this stuff?
562
00:30:27,510 --> 00:30:29,106
AUDIENCE: It was out
of Wicker's book.
563
00:30:29,106 --> 00:30:29,760
PROFESSOR: Out of?
564
00:30:29,760 --> 00:30:30,620
AUDIENCE: Wicker's book.
565
00:30:30,620 --> 00:30:32,764
PROFESSOR: Wicker's book
on turbo codes.
566
00:30:32,764 --> 00:30:34,862
AUDIENCE: No, the data
storage one.
567
00:30:34,862 --> 00:30:36,180
PROFESSOR: Data storage.
568
00:30:36,180 --> 00:30:37,150
OK.
569
00:30:37,150 --> 00:30:39,800
I don't know that book.
570
00:30:39,800 --> 00:30:43,640
But it talks about sophisticated
soft decision
571
00:30:43,640 --> 00:30:45,210
type coding, and so forth.
572
00:30:45,210 --> 00:30:46,460
What was the course you took?
573
00:30:46,460 --> 00:30:47,310
AUDIENCE: Algebra.
574
00:30:47,310 --> 00:30:49,330
PROFESSOR: Algebraic.
575
00:30:49,330 --> 00:30:50,580
OK.
576
00:30:50,580 --> 00:30:52,160
577
00:30:52,160 --> 00:30:52,500
All right.
578
00:30:52,500 --> 00:30:58,910
Well, let me just make a broad
brush comment, which may or
579
00:30:58,910 --> 00:31:00,910
may not be supported
by your previous
580
00:31:00,910 --> 00:31:02,130
exposure to coding theory.
581
00:31:02,130 --> 00:31:08,280
Which is that for many, many
years, when people said coding
582
00:31:08,280 --> 00:31:11,910
theory, they meant algebraic
coding theory, coding theory
583
00:31:11,910 --> 00:31:14,800
of block codes that are
constructed by algebraic
584
00:31:14,800 --> 00:31:18,100
techniques like Reed-Muller
codes, like Reed-Solomon
585
00:31:18,100 --> 00:31:22,320
codes, BCH codes,
cyclic codes.
586
00:31:22,320 --> 00:31:25,900
And if you said, I want to
leaRn some coding theory,
587
00:31:25,900 --> 00:31:29,210
asked your graduate student to
go buy a textbook for you,
588
00:31:29,210 --> 00:31:30,450
it's probably going to
be a textbook on
589
00:31:30,450 --> 00:31:33,170
algebraic coding theory.
590
00:31:33,170 --> 00:31:38,520
Meanwhile, there was a bunch
of us who were off actually
591
00:31:38,520 --> 00:31:42,830
trying to construct codes
for real channels.
592
00:31:42,830 --> 00:31:48,130
And conceRning with really how
complex this is to implement,
593
00:31:48,130 --> 00:31:51,540
and what kind of performance
can we really get.
594
00:31:51,540 --> 00:31:56,640
And we hardly ever used
algebraic codes.
595
00:31:56,640 --> 00:31:59,580
Initially, we used convolutional
codes with
596
00:31:59,580 --> 00:32:02,970
various kinds of decoding
algorithms, threshold
597
00:32:02,970 --> 00:32:08,000
decoding, sequential decoding,
the Viterbi algorithm.
598
00:32:08,000 --> 00:32:12,520
And these get more and more
elaborate, and then finally
599
00:32:12,520 --> 00:32:17,530
the big step, well, the big
step was to capacity
600
00:32:17,530 --> 00:32:18,630
approaching codes.
601
00:32:18,630 --> 00:32:24,110
Which now people are starting to
call modeRn coding theory.
602
00:32:24,110 --> 00:32:26,710
Long, random-like codes
that have an
603
00:32:26,710 --> 00:32:28,820
iterative decoding algorithm.
604
00:32:28,820 --> 00:32:30,180
And that's what we'll
get to towards
605
00:32:30,180 --> 00:32:31,740
the end of this course.
606
00:32:31,740 --> 00:32:38,050
So from an engineers point of
view, to some extent, of all
607
00:32:38,050 --> 00:32:42,810
this work on n k d and algebraic
decoding algorithms
608
00:32:42,810 --> 00:32:46,675
and so forth was a massive
distraction.
609
00:32:46,675 --> 00:32:49,530
It really missed the point.
610
00:32:49,530 --> 00:32:53,510
Sometimes because it assumed
hard decisions.
611
00:32:53,510 --> 00:32:56,830
Or after assuming hard
decisions, it assumed bounded
612
00:32:56,830 --> 00:32:59,140
distance decoding algorithms.
613
00:32:59,140 --> 00:33:01,170
But it was all --
614
00:33:01,170 --> 00:33:04,630
from Shannon's perspective, it
was too deterministic, too
615
00:33:04,630 --> 00:33:06,530
constructed, too structured.
616
00:33:06,530 --> 00:33:11,260
You want more random elements
in your coding scheme.
617
00:33:11,260 --> 00:33:17,700
However, two things: one, this
theory is a beautiful theory,
618
00:33:17,700 --> 00:33:21,480
both the mathematical theory
of finite fields and the
619
00:33:21,480 --> 00:33:27,770
coding theory, particularly of
Reed-Solomon codes, which are
620
00:33:27,770 --> 00:33:30,280
uniquely the greatest
accomplishment of algebraic
621
00:33:30,280 --> 00:33:35,220
coding theory, and which have
proved to be very useful.
622
00:33:35,220 --> 00:33:44,140
And B, Reed-Solomon codes are
something that as engineers,
623
00:33:44,140 --> 00:33:47,210
ought to be part of your tool
kit, and you will find very
624
00:33:47,210 --> 00:33:49,550
useful a variety
of situations.
625
00:33:49,550 --> 00:33:55,360
So the objective of this part of
the course is, within some
626
00:33:55,360 --> 00:33:58,680
proportion within the overall
scheme of the course, to give
627
00:33:58,680 --> 00:34:02,430
you exposure to this
lovely theory.
628
00:34:02,430 --> 00:34:07,470
You can leaRn all about it just
by reading a book or two.
629
00:34:07,470 --> 00:34:10,750
Probably less than one book.
630
00:34:10,750 --> 00:34:13,310
And to give you some exposure
to Reed-Solomon codes, which
631
00:34:13,310 --> 00:34:15,170
have been used in practice.
632
00:34:15,170 --> 00:34:19,909
633
00:34:19,909 --> 00:34:24,120
For more than 20 years, the
deep space standard was a
634
00:34:24,120 --> 00:34:25,370
concatenated code.
635
00:34:25,370 --> 00:34:29,770
That means a sequence of two
codes, of which the inner code
636
00:34:29,770 --> 00:34:32,520
was a convolutional code
decoded by the Viterbi
637
00:34:32,520 --> 00:34:35,320
algorithm, which we'll talk
about after this.
638
00:34:35,320 --> 00:34:38,989
And then the outer code, the
code that cleans up errors
639
00:34:38,989 --> 00:34:44,530
made by the inner code, was a
Reed-Solomon code of length
640
00:34:44,530 --> 00:34:49,219
255 over the finite field
with 256 elements.
641
00:34:49,219 --> 00:34:54,610
And that's the dynamite
combination that dominated the
642
00:34:54,610 --> 00:34:57,760
power-limited coding world
for at least 20 years.
643
00:34:57,760 --> 00:34:59,840
So you ought to know,
as a minimum, about
644
00:34:59,840 --> 00:35:01,500
Reed-Solomon codes.
645
00:35:01,500 --> 00:35:05,040
So that's my objective
in going through
646
00:35:05,040 --> 00:35:06,460
these next two chapters.
647
00:35:06,460 --> 00:35:10,660
However, because these have
not basically been on the
648
00:35:10,660 --> 00:35:16,050
winning path to get to the
Shannon limit, I'm trying to
649
00:35:16,050 --> 00:35:19,400
limit it to three weeks max.
650
00:35:19,400 --> 00:35:23,190
And therefore, that means that
the presentation is going to
651
00:35:23,190 --> 00:35:27,040
be a lot faster than what
you're used to.
652
00:35:27,040 --> 00:35:30,390
I'll probably do as much in
these three weeks as most
653
00:35:30,390 --> 00:35:32,390
people do in a term in
an entire algebraic
654
00:35:32,390 --> 00:35:34,750
coding theory course.
655
00:35:34,750 --> 00:35:36,990
That's an exaggeration --
656
00:35:36,990 --> 00:35:39,810
people do a lot more -- but I
try to at least hit all the
657
00:35:39,810 --> 00:35:41,190
main points that you
need to know.
658
00:35:41,190 --> 00:35:46,000
So I apologize in advance if it
seems that now we're on a
659
00:35:46,000 --> 00:35:48,790
very fast moving train.
660
00:35:48,790 --> 00:35:50,450
But you ought to know
this stuff.
661
00:35:50,450 --> 00:35:52,600
You ought not to spend
too much time on it.
662
00:35:52,600 --> 00:35:56,670
I'm trying to reconcile those
two points of view.
663
00:35:56,670 --> 00:35:59,930
Any questions or comments?
664
00:35:59,930 --> 00:36:00,290
All right.
665
00:36:00,290 --> 00:36:01,540
Let's go.
666
00:36:01,540 --> 00:36:03,910
667
00:36:03,910 --> 00:36:07,240
Chapter seven.
668
00:36:07,240 --> 00:36:13,865
We go through a series
of algebraic objects.
669
00:36:13,865 --> 00:36:16,480
670
00:36:16,480 --> 00:36:21,100
First, the integers,
then groups.
671
00:36:21,100 --> 00:36:23,850
You certainly ought to know --
you do know about integers.
672
00:36:23,850 --> 00:36:26,460
You ought to know about groups,
at least a little bit.
673
00:36:26,460 --> 00:36:28,960
674
00:36:28,960 --> 00:36:29,580
Let's see.
675
00:36:29,580 --> 00:36:30,510
What's next?
676
00:36:30,510 --> 00:36:35,415
Then fields, particularly
finite.
677
00:36:35,415 --> 00:36:39,860
678
00:36:39,860 --> 00:36:47,670
Then polynomials over fields,
particularly finite.
679
00:36:47,670 --> 00:36:54,130
And then finally, from this we
actually get to construct
680
00:36:54,130 --> 00:36:55,380
finite fields.
681
00:36:55,380 --> 00:36:58,180
682
00:36:58,180 --> 00:37:03,180
And these are all things that
you'll encounter again and
683
00:37:03,180 --> 00:37:05,590
again, have encountered.
684
00:37:05,590 --> 00:37:07,660
I think you've probably
encountered everything except
685
00:37:07,660 --> 00:37:10,810
possibly for groups
and finite fields.
686
00:37:10,810 --> 00:37:13,340
You certainly encountered
polynomials over real and
687
00:37:13,340 --> 00:37:16,410
complex fields.
688
00:37:16,410 --> 00:37:19,890
And so in some sense, this is
a broadening of what you
689
00:37:19,890 --> 00:37:21,140
already know.
690
00:37:21,140 --> 00:37:26,720
691
00:37:26,720 --> 00:37:32,600
So I assume we start with the
integers for two reasons.
692
00:37:32,600 --> 00:37:36,020
693
00:37:36,020 --> 00:37:39,820
One is you already know about
the factorization properties
694
00:37:39,820 --> 00:37:42,250
of the integers.
695
00:37:42,250 --> 00:37:43,020
We're going to --
696
00:37:43,020 --> 00:37:46,990
that leads to a little bit of
number theory, which we're
697
00:37:46,990 --> 00:37:52,250
going to use when we talk about
groups, finite groups,
698
00:37:52,250 --> 00:37:54,640
cyclic groups.
699
00:37:54,640 --> 00:37:56,600
So this is just to remind
you, but it's
700
00:37:56,600 --> 00:37:59,440
also to put up a template.
701
00:37:59,440 --> 00:38:03,030
And we're going to find the
algebraic properties of
702
00:38:03,030 --> 00:38:05,830
polynomials are very,
very similar to
703
00:38:05,830 --> 00:38:06,890
those of the integers.
704
00:38:06,890 --> 00:38:08,820
This is because they're
both rings.
705
00:38:08,820 --> 00:38:11,420
They're both in fact principal
ideal domains.
706
00:38:11,420 --> 00:38:12,690
They're Euclidean domains.
707
00:38:12,690 --> 00:38:15,670
They both have a Euclidean
division algorithm as their
708
00:38:15,670 --> 00:38:18,230
key mathematical property.
709
00:38:18,230 --> 00:38:21,650
And so pretty much any time you
want to develop a result
710
00:38:21,650 --> 00:38:26,680
about the polynomials, you can
start from the comparable
711
00:38:26,680 --> 00:38:29,200
results that you know about the
integers and just change
712
00:38:29,200 --> 00:38:30,460
the notation.
713
00:38:30,460 --> 00:38:36,320
And you'll have a sketch of
the proof for polynomials.
714
00:38:36,320 --> 00:38:40,990
So it's important to remind
ourselves about
715
00:38:40,990 --> 00:38:43,590
the integers first.
716
00:38:43,590 --> 00:38:46,050
Now, when I say the
factorization properties of
717
00:38:46,050 --> 00:38:48,310
the integers, I'm not talking
about anything more
718
00:38:48,310 --> 00:38:51,400
complicated than what you
leaRned in grade school.
719
00:38:51,400 --> 00:38:55,120
I just want to introduce
a few terms.
720
00:38:55,120 --> 00:39:00,670
We have the notion
of divisors.
721
00:39:00,670 --> 00:39:09,685
a divides b means that a times
some quotient equals b.
722
00:39:09,685 --> 00:39:16,400
I could be talking about plus or
minus, positive or negative
723
00:39:16,400 --> 00:39:17,340
integers here.
724
00:39:17,340 --> 00:39:18,590
Everything divides 0.
725
00:39:18,590 --> 00:39:21,300
726
00:39:21,300 --> 00:39:22,120
OK.
727
00:39:22,120 --> 00:39:29,620
We have the idea of units,
which are the invertible
728
00:39:29,620 --> 00:39:33,140
integers under multiplication.
729
00:39:33,140 --> 00:39:34,390
And what are those?
730
00:39:34,390 --> 00:39:38,460
731
00:39:38,460 --> 00:39:41,600
Which integers have a
multiplicative inverse?
732
00:39:41,600 --> 00:39:44,980
733
00:39:44,980 --> 00:39:45,340
[UNINTELLIGIBLE]
734
00:39:45,340 --> 00:39:45,970
any of them?
735
00:39:45,970 --> 00:39:47,802
AUDIENCE: [INAUDIBLE PHRASE].
736
00:39:47,802 --> 00:39:49,170
PROFESSOR: [INAUDIBLE].
737
00:39:49,170 --> 00:39:50,420
All right.
738
00:39:50,420 --> 00:39:52,220
739
00:39:52,220 --> 00:39:57,530
That's a little exercise that
shows you that, in general,
740
00:39:57,530 --> 00:40:01,870
the invertible elements of any
set that has an operation like
741
00:40:01,870 --> 00:40:06,190
multiplication form a group.
742
00:40:06,190 --> 00:40:09,750
[INAUDIBLE] group thing, these
clearly form a group under
743
00:40:09,750 --> 00:40:11,320
multiplication that's
isomorphic
744
00:40:11,320 --> 00:40:12,725
to the binary group.
745
00:40:12,725 --> 00:40:14,780
Well, in fact, we've already
been using it --
746
00:40:14,780 --> 00:40:16,380
F2.
747
00:40:16,380 --> 00:40:16,830
OK.
748
00:40:16,830 --> 00:40:21,620
So we have units, and whenever
we talk about the divisors,
749
00:40:21,620 --> 00:40:24,420
factorization, primes,
so forth,
750
00:40:24,420 --> 00:40:25,700
the units are a nuisance.
751
00:40:25,700 --> 00:40:29,790
Because we can always multiply
things by units and it doesn't
752
00:40:29,790 --> 00:40:33,370
affect factorization
properties.
753
00:40:33,370 --> 00:40:35,810
This is [UNINTELLIGIBLE] a
little bit more out there.
754
00:40:35,810 --> 00:40:41,515
755
00:40:41,515 --> 00:40:45,530
What I eventually want
to get to is --
756
00:40:45,530 --> 00:40:49,510
skipping a few things that
are said in the notes --
757
00:40:49,510 --> 00:40:50,866
is unique factorization.
758
00:40:50,866 --> 00:40:55,900
759
00:40:55,900 --> 00:41:05,520
Maybe before that I should
define primes:
760
00:41:05,520 --> 00:41:07,380
are positive integers.
761
00:41:07,380 --> 00:41:10,862
The reason we stay positive
is, again --
762
00:41:10,862 --> 00:41:14,610
of course, we could have a
negative prime integer, but
763
00:41:14,610 --> 00:41:17,230
let's pick a unique
representative of two units.
764
00:41:17,230 --> 00:41:20,290
765
00:41:20,290 --> 00:41:21,860
Positive integers --
766
00:41:21,860 --> 00:41:24,330
I should've said in the notes
greater than 1, we don't
767
00:41:24,330 --> 00:41:25,625
consider 1 to be a prime --
768
00:41:25,625 --> 00:41:28,170
769
00:41:28,170 --> 00:41:34,515
that have no non-trivial
divisors.
770
00:41:34,515 --> 00:41:41,940
771
00:41:41,940 --> 00:41:44,780
And, of course, every integer
has trivial divisors.
772
00:41:44,780 --> 00:41:49,790
773
00:41:49,790 --> 00:41:55,460
Namely, plus or minus 1 and
plus or minus itself.
774
00:41:55,460 --> 00:41:57,205
Let me give this a name.
775
00:41:57,205 --> 00:41:57,580
i.
776
00:41:57,580 --> 00:42:00,140
Not a very good name.
777
00:42:00,140 --> 00:42:01,390
How about n?
778
00:42:01,390 --> 00:42:03,700
779
00:42:03,700 --> 00:42:06,300
So if we have any integer n,
it's divisible by plus or
780
00:42:06,300 --> 00:42:08,940
minus one, plus or minus n --
781
00:42:08,940 --> 00:42:11,940
that doesn't count.
782
00:42:11,940 --> 00:42:15,120
So a prime is an integer that
doesn't have any other
783
00:42:15,120 --> 00:42:18,300
divisors than these
trivial ones.
784
00:42:18,300 --> 00:42:22,610
And then I want to get to unique
factorization, which
785
00:42:22,610 --> 00:42:24,080
I'll just state, not prove.
786
00:42:24,080 --> 00:42:28,240
787
00:42:28,240 --> 00:42:41,760
Which is that every integer
is equal to a
788
00:42:41,760 --> 00:42:43,310
unique product of primes.
789
00:42:43,310 --> 00:42:50,366
790
00:42:50,366 --> 00:42:53,840
And we always have to
add up to units.
791
00:42:53,840 --> 00:42:57,860
We can add as many plus or minus
1's to the product as we
792
00:42:57,860 --> 00:43:01,990
want, as long as we have
an even number of them.
793
00:43:01,990 --> 00:43:04,200
And we'll have another
factorization, so here, we
794
00:43:04,200 --> 00:43:06,960
want to exclude the units.
795
00:43:06,960 --> 00:43:11,730
And then we have unique
factorization.
796
00:43:11,730 --> 00:43:12,260
OK.
797
00:43:12,260 --> 00:43:15,770
This is all grade school
stuff, high
798
00:43:15,770 --> 00:43:17,020
school stuff maybe.
799
00:43:17,020 --> 00:43:20,910
800
00:43:20,910 --> 00:43:23,750
Again, when I go through
polynomials, you'll see there
801
00:43:23,750 --> 00:43:28,610
is a set of concept
exactly like this.
802
00:43:28,610 --> 00:43:35,540
Now, where we want to get to
is mod-n arithmetic, which
803
00:43:35,540 --> 00:43:39,820
again, you all know
about, I assume.
804
00:43:39,820 --> 00:43:42,300
This is based on the fact --
805
00:43:42,300 --> 00:43:45,860
we can think of it as being
based on Euclidean division
806
00:43:45,860 --> 00:43:51,940
algorithm, which basically
proves that given any two
807
00:43:51,940 --> 00:44:06,280
integers m and n, we can write m
as some qn plus r where 0 is
808
00:44:06,280 --> 00:44:09,765
less than or equal to r, less
than or equal to n minus 1,
809
00:44:09,765 --> 00:44:13,470
and is called the remainder
of m, modulo n.
810
00:44:13,470 --> 00:44:16,370
811
00:44:16,370 --> 00:44:17,790
And how is this proved?
812
00:44:17,790 --> 00:44:20,330
Let's say these are both
positive integers.
813
00:44:20,330 --> 00:44:24,200
Say m is a big integer,
n is a small integer.
814
00:44:24,200 --> 00:44:29,600
We just keep subtracting integer
multiples of n from m
815
00:44:29,600 --> 00:44:30,555
until we --
816
00:44:30,555 --> 00:44:33,470
you know, the remainder will
start decreasing until we get
817
00:44:33,470 --> 00:44:36,780
the remainder in this range, and
it's obvious that we can
818
00:44:36,780 --> 00:44:39,710
always get a remainder
that's in that range.
819
00:44:39,710 --> 00:44:43,840
And further, that this
decomposition is unique under
820
00:44:43,840 --> 00:44:45,090
this restriction on r.
821
00:44:45,090 --> 00:44:49,870
822
00:44:49,870 --> 00:44:50,330
All right.
823
00:44:50,330 --> 00:45:03,080
So we write the m is congruent
or equivalent to r mod n.
824
00:45:03,080 --> 00:45:10,260
That means it differs from r by
a multiple of n where r is
825
00:45:10,260 --> 00:45:18,000
unique and r is in the set 0, 1,
up through n minus 1, which
826
00:45:18,000 --> 00:45:20,350
we say are the residue
classes mod n.
827
00:45:20,350 --> 00:45:23,970
It has to be one of
those n things.
828
00:45:23,970 --> 00:45:25,220
OK?
829
00:45:25,220 --> 00:45:26,910
830
00:45:26,910 --> 00:45:28,840
No one is having the slightest
trouble with this, right?
831
00:45:28,840 --> 00:45:29,285
Boring.
832
00:45:29,285 --> 00:45:31,180
You've seen it all before.
833
00:45:31,180 --> 00:45:32,430
I hope.
834
00:45:32,430 --> 00:45:34,700
835
00:45:34,700 --> 00:45:35,950
OK.
836
00:45:35,950 --> 00:45:37,660
837
00:45:37,660 --> 00:45:52,290
So now you ask, suppose m equals
r mod-n, and some other
838
00:45:52,290 --> 00:45:57,070
w equals s mod-n.
839
00:45:57,070 --> 00:46:07,230
What is m plus w mod-n, and you
quickly prove that it's
840
00:46:07,230 --> 00:46:10,690
equal to r plus s mod-n.
841
00:46:10,690 --> 00:46:13,530
842
00:46:13,530 --> 00:46:17,970
By transferring, by just
checking the properties of
843
00:46:17,970 --> 00:46:21,140
ordinary arithmetic, you can use
this kind of expression: m
844
00:46:21,140 --> 00:46:25,030
must equal something
times n plus r.
845
00:46:25,030 --> 00:46:28,670
w must equal something
times n plus s.
846
00:46:28,670 --> 00:46:32,850
So if we add m plus w, we're
going to get something times n
847
00:46:32,850 --> 00:46:34,860
plus r plus s.
848
00:46:34,860 --> 00:46:41,070
And that is, by definition,
going to be r plus s mod-n.
849
00:46:41,070 --> 00:46:43,900
So we get a well-defined
addition rule for these
850
00:46:43,900 --> 00:46:50,590
residues, for the elements of
this set of n residues Rn, or
851
00:46:50,590 --> 00:46:52,630
remainders.
852
00:46:52,630 --> 00:46:55,980
OK, so addition is
well-defined.
853
00:46:55,980 --> 00:47:00,790
And similarly, multiplication
is well-defined.
854
00:47:00,790 --> 00:47:04,950
mw mod-n, again, you can prove
-- and this is done
855
00:47:04,950 --> 00:47:06,200
in the notes --
856
00:47:06,200 --> 00:47:09,180
857
00:47:09,180 --> 00:47:14,930
that if mw mod-n can be
found by taking rs and
858
00:47:14,930 --> 00:47:17,984
reducing it, mod-n.
859
00:47:17,984 --> 00:47:20,650
Maybe I should have congruent
signs here, but I
860
00:47:20,650 --> 00:47:22,920
won't worry about it.
861
00:47:22,920 --> 00:47:34,150
So this is mod-n, addition
and multiplication.
862
00:47:34,150 --> 00:47:39,930
So what this says is that Rn,
combined with this mod-n
863
00:47:39,930 --> 00:47:44,460
addition operation, which in the
notes I write in that way,
864
00:47:44,460 --> 00:47:47,885
and this mod-n multiplication
operation --
865
00:47:47,885 --> 00:47:52,200
866
00:47:52,200 --> 00:47:56,190
this is a well-defined set of n
elements with a well-defined
867
00:47:56,190 --> 00:47:59,040
addition operation and
a well-defined
868
00:47:59,040 --> 00:48:01,464
multiplication operation.
869
00:48:01,464 --> 00:48:04,520
And I don't prove any properties
about it here, but
870
00:48:04,520 --> 00:48:10,160
I come back to prove that this
has almost all of the
871
00:48:10,160 --> 00:48:14,720
properties that we
want of a field.
872
00:48:14,720 --> 00:48:17,850
In fact, if n is a prime, it
does have the properties that
873
00:48:17,850 --> 00:48:19,510
we want of a field.
874
00:48:19,510 --> 00:48:25,910
So we'll later prove that the
finite field with p elements
875
00:48:25,910 --> 00:48:35,720
is simply rp with mod-p addition
and multiplication.
876
00:48:35,720 --> 00:48:39,490
And, of course, for the
particular case p equals 2, we
877
00:48:39,490 --> 00:48:41,510
already have a lot of experience
with this.
878
00:48:41,510 --> 00:48:43,000
That's how we get the
binary field.
879
00:48:43,000 --> 00:48:48,340
We just take the 0 and 1,
considered as residues mod-2.
880
00:48:48,340 --> 00:48:55,630
And then the field addition and
multiplication operations
881
00:48:55,630 --> 00:48:59,320
give us something that
satisfies the
882
00:48:59,320 --> 00:49:01,960
axioms of the field.
883
00:49:01,960 --> 00:49:05,240
We'll find that works for every
prime, and it doesn't
884
00:49:05,240 --> 00:49:07,085
work for non-primes.
885
00:49:07,085 --> 00:49:10,360
Can anyone quickly see why this
isn't going to work for
886
00:49:10,360 --> 00:49:11,610
non-primes?
887
00:49:11,610 --> 00:49:15,010
888
00:49:15,010 --> 00:49:16,500
Suppose n equals 6.
889
00:49:16,500 --> 00:49:21,270
890
00:49:21,270 --> 00:49:28,400
Anyone quickly see why this set
under these two operations
891
00:49:28,400 --> 00:49:30,290
isn't going to be a field?
892
00:49:30,290 --> 00:49:32,950
This, of course, assumes that
you know what a field is,
893
00:49:32,950 --> 00:49:36,280
which we haven't actually
said yet.
894
00:49:36,280 --> 00:49:37,765
AUDIENCE: [INAUDIBLE].
895
00:49:37,765 --> 00:49:38,525
PROFESSOR: Excuse me?
896
00:49:38,525 --> 00:49:41,000
AUDIENCE: [INAUDIBLE].
897
00:49:41,000 --> 00:49:43,200
PROFESSOR: There's no inverse.
898
00:49:43,200 --> 00:49:47,810
In a field, every non-zero
element has to have an
899
00:49:47,810 --> 00:49:52,740
inverse, just like in the
real and complex field.
900
00:49:52,740 --> 00:50:00,290
But in the integers mod-6, 3
times 2 is equal to 6, which
901
00:50:00,290 --> 00:50:01,890
is equal to 0.
902
00:50:01,890 --> 00:50:06,080
So there are two non-zero
elements that multiplied
903
00:50:06,080 --> 00:50:07,645
together give 0.
904
00:50:07,645 --> 00:50:11,170
And that means that neither 3
nor 2 can possibly have a
905
00:50:11,170 --> 00:50:14,330
multiplicative inverse.
906
00:50:14,330 --> 00:50:17,910
So that's why you have
to have a prime.
907
00:50:17,910 --> 00:50:23,490
This doesn't happen, of course,
if n were a prime.
908
00:50:23,490 --> 00:50:23,900
OK.
909
00:50:23,900 --> 00:50:28,360
So that's just a little warm
up to remind you what mod-n
910
00:50:28,360 --> 00:50:29,780
arithmetic is like.
911
00:50:29,780 --> 00:50:34,780
And in particular, this is going
to be the key element of
912
00:50:34,780 --> 00:50:37,350
a lot of proofs --
913
00:50:37,350 --> 00:50:39,960
the Euclidean division
algorithm --
914
00:50:39,960 --> 00:50:43,140
both for integers and
for polynomials.
915
00:50:43,140 --> 00:50:49,000
916
00:50:49,000 --> 00:50:49,660
OK.
917
00:50:49,660 --> 00:50:52,185
So that, I hope, was review.
918
00:50:52,185 --> 00:50:57,040
919
00:50:57,040 --> 00:50:58,530
Now we're going to talk
about groups.
920
00:50:58,530 --> 00:51:02,280
921
00:51:02,280 --> 00:51:03,600
So let's do it.
922
00:51:03,600 --> 00:51:08,220
923
00:51:08,220 --> 00:51:10,250
How many of you feel you
know what a group is?
924
00:51:10,250 --> 00:51:14,690
925
00:51:14,690 --> 00:51:16,960
Less than half the class.
926
00:51:16,960 --> 00:51:18,250
OK.
927
00:51:18,250 --> 00:51:26,020
Well, we've been talking about
groups so far here, but let's
928
00:51:26,020 --> 00:51:29,450
take a little bit more of
an axiomatic approach.
929
00:51:29,450 --> 00:51:33,270
In the notes, I actually give
two possible sets of axioms
930
00:51:33,270 --> 00:51:34,020
for a group.
931
00:51:34,020 --> 00:51:36,920
Let me start with
a standard one.
932
00:51:36,920 --> 00:51:39,710
You've heard of something called
the group property in
933
00:51:39,710 --> 00:51:40,960
this course.
934
00:51:40,960 --> 00:51:45,450
935
00:51:45,450 --> 00:51:49,490
Well, first of all, what am I
talking about when I'm talking
936
00:51:49,490 --> 00:51:50,080
about a group?
937
00:51:50,080 --> 00:51:54,670
I'm talking about a set, g,
and I'm also talking about
938
00:51:54,670 --> 00:51:58,750
some operation on that set,
which I'll use the mod
939
00:51:58,750 --> 00:52:04,370
addition circle-plus as
the group property.
940
00:52:04,370 --> 00:52:07,160
941
00:52:07,160 --> 00:52:12,650
Even if it's multiplication or
matrix inversion or -- no, it
942
00:52:12,650 --> 00:52:15,690
couldn't be matrix inversion.
943
00:52:15,690 --> 00:52:21,800
What this means is that if we
take any two elements of the
944
00:52:21,800 --> 00:52:27,430
group, then a plus b
is well-defined.
945
00:52:27,430 --> 00:52:31,440
946
00:52:31,440 --> 00:52:34,850
So you can think there's
a table.
947
00:52:34,850 --> 00:52:36,840
Here are the elements
of the group.
948
00:52:36,840 --> 00:52:39,385
a, b, c, d, e.
949
00:52:39,385 --> 00:52:41,860
a, b, c, d, e.
950
00:52:41,860 --> 00:52:46,390
And at this point, all we
know is that we can
951
00:52:46,390 --> 00:52:48,580
fill out the table.
952
00:52:48,580 --> 00:52:50,540
We get something that
goes in all here.
953
00:52:50,540 --> 00:52:53,400
954
00:52:53,400 --> 00:52:54,200
All right?
955
00:52:54,200 --> 00:52:56,137
So that defines the group.
956
00:52:56,137 --> 00:52:58,999
AUDIENCE: [INAUDIBLE PHRASE].
957
00:52:58,999 --> 00:53:02,500
PROFESSOR: That means if I
give a and b, then the
958
00:53:02,500 --> 00:53:05,140
quantity a plus b is --
959
00:53:05,140 --> 00:53:07,380
I can tell you what c is.
960
00:53:07,380 --> 00:53:10,060
There's a unique element in
each of the cells of this
961
00:53:10,060 --> 00:53:13,072
addition table, or
the group table.
962
00:53:13,072 --> 00:53:15,970
963
00:53:15,970 --> 00:53:16,780
OK.
964
00:53:16,780 --> 00:53:25,540
Group property, also called
closure, is the obvious one.
965
00:53:25,540 --> 00:53:28,760
966
00:53:28,760 --> 00:53:31,770
What's an elementary condition
of what all these sums have to
967
00:53:31,770 --> 00:53:33,715
be for this to be a group?
968
00:53:33,715 --> 00:53:42,460
969
00:53:42,460 --> 00:53:43,710
Have to be all in the group.
970
00:53:43,710 --> 00:53:49,350
971
00:53:49,350 --> 00:53:52,826
If I say I have a group that
consists of 1,2,3,4,5,
972
00:53:52,826 --> 00:53:59,820
1,2,3,4,5 and my group
operation is ordinary
973
00:53:59,820 --> 00:54:03,445
addition, then I can certainly
fill out the table.
974
00:54:03,445 --> 00:54:06,820
And I get 2,3,4,5,6 --
975
00:54:06,820 --> 00:54:08,990
whoops.
976
00:54:08,990 --> 00:54:11,450
Yeah, that's the element that's
not in the group.
977
00:54:11,450 --> 00:54:18,790
So in order to have a
group, I want a --
978
00:54:18,790 --> 00:54:23,870
I mean, it's a plus b is in
the group, all a and b.
979
00:54:23,870 --> 00:54:26,430
980
00:54:26,430 --> 00:54:30,640
So the group is closed
under addition.
981
00:54:30,640 --> 00:54:35,200
This was the property we used to
define linearity for binary
982
00:54:35,200 --> 00:54:37,150
linear block codes.
983
00:54:37,150 --> 00:54:39,500
The sum of any two n-tuples
had to be another
984
00:54:39,500 --> 00:54:40,750
n-tuple in the code.
985
00:54:40,750 --> 00:54:47,260
986
00:54:47,260 --> 00:54:53,920
We have the associativity
operation.
987
00:54:53,920 --> 00:55:03,745
This is a requirement on the
binary operation, really.
988
00:55:03,745 --> 00:55:10,190
a plus b plus c has got to be
equal to a plus b plus c.
989
00:55:10,190 --> 00:55:15,870
In other words, you can get
rid of the parentheses.
990
00:55:15,870 --> 00:55:20,560
And this, an expression like
that, is meaningful.
991
00:55:20,560 --> 00:55:23,980
It doesn't matter how you group,
whether you add these
992
00:55:23,980 --> 00:55:27,480
two things first and then that,
or add these two things
993
00:55:27,480 --> 00:55:28,730
first and then that.
994
00:55:28,730 --> 00:55:32,390
995
00:55:32,390 --> 00:55:38,300
Maybe now is a good time to
talk about the special
996
00:55:38,300 --> 00:55:41,870
property, which is
not an axiom.
997
00:55:41,870 --> 00:55:47,036
So let me call it x abelian, or
the commutative property.
998
00:55:47,036 --> 00:55:52,700
999
00:55:52,700 --> 00:55:59,590
This is true if a plus
b is equal to b
1000
00:55:59,590 --> 00:56:05,692
plus a, all a, b group.
1001
00:56:05,692 --> 00:56:09,200
Now this is, of course, true
for ordinary addition and
1002
00:56:09,200 --> 00:56:11,340
ordinary multiplication.
1003
00:56:11,340 --> 00:56:13,840
It's not true for matrix
multiplication.
1004
00:56:13,840 --> 00:56:17,960
Say if I have a group of
matrices, then in general, ab
1005
00:56:17,960 --> 00:56:19,460
is not equal to ba.
1006
00:56:19,460 --> 00:56:22,100
So for some groups this
holds, for some
1007
00:56:22,100 --> 00:56:23,350
groups it doesn't hold.
1008
00:56:23,350 --> 00:56:25,840
1009
00:56:25,840 --> 00:56:31,130
In this course, almost without
exception, every group we're
1010
00:56:31,130 --> 00:56:33,490
going to talk about is
an abelian group.
1011
00:56:33,490 --> 00:56:36,450
That means that in adding
things up, the
1012
00:56:36,450 --> 00:56:38,670
order doesn't matter.
1013
00:56:38,670 --> 00:56:43,170
So if that were true, then it in
a plus b plus c, that means
1014
00:56:43,170 --> 00:56:45,650
that's the same as c plus
b plus a, or c plus
1015
00:56:45,650 --> 00:56:46,960
a plus b, or whatever.
1016
00:56:46,960 --> 00:56:48,910
All the properties that
you're accustomed to
1017
00:56:48,910 --> 00:56:51,110
from ordinary addition.
1018
00:56:51,110 --> 00:56:53,940
But this is not one of the group
axioms, and, of course,
1019
00:56:53,940 --> 00:56:56,770
there are non-abelian groups.
1020
00:56:56,770 --> 00:57:00,520
The smallest non-abelian group
has size 6, and is the
1021
00:57:00,520 --> 00:57:04,630
permutation group on
three elements.
1022
00:57:04,630 --> 00:57:07,550
So small groups are all --
1023
00:57:07,550 --> 00:57:10,015
size 5 or less are
all abelian.
1024
00:57:10,015 --> 00:57:14,020
Because that's the only way
we can do the group table.
1025
00:57:14,020 --> 00:57:14,410
All right.
1026
00:57:14,410 --> 00:57:16,260
So that's terminology
at this point.
1027
00:57:16,260 --> 00:57:17,930
It's not part of the axiom.
1028
00:57:17,930 --> 00:57:22,040
But that's the reason that I use
a symbol that looks like a
1029
00:57:22,040 --> 00:57:25,100
plus, is I'm pretty much always
thinking an abelian
1030
00:57:25,100 --> 00:57:27,700
group, and I don't want to
trouble you with thinking
1031
00:57:27,700 --> 00:57:30,830
about non-abelian groups.
1032
00:57:30,830 --> 00:57:32,610
OK.
1033
00:57:32,610 --> 00:57:35,230
Then we have the identity
property.
1034
00:57:35,230 --> 00:57:38,610
1035
00:57:38,610 --> 00:57:42,770
There is an identity, which,
in an abelian group, is
1036
00:57:42,770 --> 00:57:44,500
always called --
1037
00:57:44,500 --> 00:57:48,300
ordinarily called 0.
1038
00:57:48,300 --> 00:57:52,040
In an additive group it's
always called 0.
1039
00:57:52,040 --> 00:57:53,270
So I'll call it 0.
1040
00:57:53,270 --> 00:57:57,930
In more abstract treatments,
it's called 1 or E for --
1041
00:57:57,930 --> 00:57:59,590
what does E stand for?
1042
00:57:59,590 --> 00:58:01,990
Eigen something.
1043
00:58:01,990 --> 00:58:10,020
There's a 0 in g such that
0 plus a equals --
1044
00:58:10,020 --> 00:58:12,640
or a plus 0 --
1045
00:58:12,640 --> 00:58:14,660
is always equal to a.
1046
00:58:14,660 --> 00:58:19,382
The 0 is the null operator on
any element of the group.
1047
00:58:19,382 --> 00:58:23,140
If you add 0 to anything, you
get what you started with.
1048
00:58:23,140 --> 00:58:26,890
1049
00:58:26,890 --> 00:58:31,330
Clearly, we have such a thing
in ordinary addition.
1050
00:58:31,330 --> 00:58:34,470
Suppose I was talking in
the real numbers under
1051
00:58:34,470 --> 00:58:38,320
multiplication, is that
a group, you guess?
1052
00:58:38,320 --> 00:58:42,875
1053
00:58:42,875 --> 00:58:46,380
It's actually not.
1054
00:58:46,380 --> 00:58:48,860
Suppose I talk about the
non-zero real numbers under
1055
00:58:48,860 --> 00:58:49,600
multiplication.
1056
00:58:49,600 --> 00:58:50,850
Is that a group?
1057
00:58:50,850 --> 00:58:53,860
1058
00:58:53,860 --> 00:58:57,700
Yes, and its identity is 1.
1059
00:58:57,700 --> 00:59:01,680
1 as the thing that if you
multiply it by anything, it
1060
00:59:01,680 --> 00:59:04,810
leaves what you started with.
1061
00:59:04,810 --> 00:59:09,540
There's the inverse, which we
don't have in the reals under
1062
00:59:09,540 --> 00:59:12,190
multiplication.
1063
00:59:12,190 --> 00:59:21,620
For all a and g, there exists
something called minus a --
1064
00:59:21,620 --> 00:59:25,700
or a to the minus 1 in
multiplicative notation --
1065
00:59:25,700 --> 00:59:37,034
in g such that a plus
minus a equals 0.
1066
00:59:37,034 --> 00:59:40,260
That's an additive inverse.
1067
00:59:40,260 --> 00:59:45,636
So in the real numbers, the
inverse of a is minus a.
1068
00:59:45,636 --> 00:59:47,740
In the integers, the
inverse of a is
1069
00:59:47,740 --> 00:59:49,660
minus a under addition.
1070
00:59:49,660 --> 00:59:54,900
In the non-zero real numbers
under multiplication, every
1071
00:59:54,900 --> 00:59:59,510
non-zero real number does have
a multiplicative inverse,
1072
00:59:59,510 --> 01:00:04,410
which is called 1 over
a, or a minus 1.
1073
01:00:04,410 --> 01:00:05,990
That's why we have
to exclude 0.
1074
01:00:05,990 --> 01:00:07,405
0 does not an inverse.
1075
01:00:07,405 --> 01:00:10,850
1076
01:00:10,850 --> 01:00:16,740
So a good example to think
about is r-star, which is
1077
01:00:16,740 --> 01:00:28,310
non-zero reals under ordinary
multiplication, is a group.
1078
01:00:28,310 --> 01:00:31,740
Is it an abelian group?
1079
01:00:31,740 --> 01:00:34,746
Is ab equal to ba?
1080
01:00:34,746 --> 01:00:35,170
Yeah.
1081
01:00:35,170 --> 01:00:37,866
So this is an infinite
abelian group.
1082
01:00:37,866 --> 01:00:53,878
1083
01:00:53,878 --> 01:00:56,900
All right.
1084
01:00:56,900 --> 01:00:59,755
OK.
1085
01:00:59,755 --> 01:01:05,440
I want to show that
the axioms lead to
1086
01:01:05,440 --> 01:01:07,465
a permutation property.
1087
01:01:07,465 --> 01:01:15,640
1088
01:01:15,640 --> 01:01:17,470
Because this is something
that's going to come
1089
01:01:17,470 --> 01:01:18,720
up again and again.
1090
01:01:18,720 --> 01:01:20,840
1091
01:01:20,840 --> 01:01:28,430
One way of saying this is if I
have, if a plus b equals c --
1092
01:01:28,430 --> 01:01:32,030
suppose I write down that
symbolic equation --
1093
01:01:32,030 --> 01:01:45,670
then I can solve for any
one of the three
1094
01:01:45,670 --> 01:01:46,920
given the other two.
1095
01:01:46,920 --> 01:01:51,686
1096
01:01:51,686 --> 01:01:52,970
How would I do that?
1097
01:01:52,970 --> 01:01:55,510
If I'm given a and b, obviously,
that tells me c
1098
01:01:55,510 --> 01:01:56,230
from the table.
1099
01:01:56,230 --> 01:02:01,890
If I'm given a and c, then a
is equal to c minus b --
1100
01:02:01,890 --> 01:02:04,660
I'm sorry -- b is equal
to c minus a.
1101
01:02:04,660 --> 01:02:07,410
1102
01:02:07,410 --> 01:02:11,360
I should say here, from this
inverse, we're going to write
1103
01:02:11,360 --> 01:02:17,730
b plus minus a in
a simpler form.
1104
01:02:17,730 --> 01:02:19,300
It's just b minus a.
1105
01:02:19,300 --> 01:02:24,700
So the fact that we have an
inverse sort of defines
1106
01:02:24,700 --> 01:02:30,590
another operation, which
we call subtraction,
1107
01:02:30,590 --> 01:02:32,825
which is not abelian.
1108
01:02:32,825 --> 01:02:33,770
Not commutative.
1109
01:02:33,770 --> 01:02:37,660
a minus b is not equal
to b minus a.
1110
01:02:37,660 --> 01:02:40,420
But it's basically implicit
in this that we
1111
01:02:40,420 --> 01:02:42,100
subtract group elements.
1112
01:02:42,100 --> 01:02:43,760
We can cancel group elements.
1113
01:02:43,760 --> 01:02:46,990
If you have a plus c equal
to b plus c, then a
1114
01:02:46,990 --> 01:02:48,030
must be equal to b.
1115
01:02:48,030 --> 01:02:50,455
Because you can subtract
c from both sides.
1116
01:02:50,455 --> 01:02:56,160
1117
01:02:56,160 --> 01:02:56,386
All right.
1118
01:02:56,386 --> 01:02:57,636
OK.
1119
01:02:57,636 --> 01:03:01,940
1120
01:03:01,940 --> 01:03:10,130
What does this mean that this
gives us a constraint on the
1121
01:03:10,130 --> 01:03:11,430
addition table of a group?
1122
01:03:11,430 --> 01:03:22,280
Suppose I take a four element
group, g, and one of the
1123
01:03:22,280 --> 01:03:25,640
elements is going to be the zero
element, and the other
1124
01:03:25,640 --> 01:03:27,980
let's just call a, b, c.
1125
01:03:27,980 --> 01:03:31,220
0, a, b, c.
1126
01:03:31,220 --> 01:03:34,000
And I have a well-defined
addition table, which
1127
01:03:34,000 --> 01:03:38,290
includes, in here, only
a 0, a, b, c.
1128
01:03:38,290 --> 01:03:42,250
From the property of the
identity, I can fill in some
1129
01:03:42,250 --> 01:03:43,500
of these things.
1130
01:03:43,500 --> 01:03:47,190
1131
01:03:47,190 --> 01:03:50,210
And now what freedom do I have
to fill in the rest?
1132
01:03:50,210 --> 01:03:52,780
How many group tables are
there of size four?
1133
01:03:52,780 --> 01:03:58,490
1134
01:03:58,490 --> 01:04:05,490
Well, the main point that I want
to make with permutation
1135
01:04:05,490 --> 01:04:09,860
property is that every row and
every column has to be a
1136
01:04:09,860 --> 01:04:12,570
permutation of the four
elements 0, a, b, c.
1137
01:04:12,570 --> 01:04:15,340
1138
01:04:15,340 --> 01:04:16,630
Why is that?
1139
01:04:16,630 --> 01:04:22,980
Suppose I had another a
in this column here.
1140
01:04:22,980 --> 01:04:27,670
Then I would have a plus 0 is
equal to a, and also a plus c
1141
01:04:27,670 --> 01:04:30,210
is equal to a.
1142
01:04:30,210 --> 01:04:35,840
But that would imply that c
equals 0, which it isn't.
1143
01:04:35,840 --> 01:04:36,026
All right.
1144
01:04:36,026 --> 01:04:40,370
So in order to be able to solve
this equation, this has
1145
01:04:40,370 --> 01:04:44,850
to be something different
from a, and likewise.
1146
01:04:44,850 --> 01:04:48,950
Since I have, in this case, a
finite group, I basically just
1147
01:04:48,950 --> 01:04:51,990
have to write the group elements
down here again in
1148
01:04:51,990 --> 01:04:53,240
some order.
1149
01:04:53,240 --> 01:04:55,480
1150
01:04:55,480 --> 01:04:58,780
It tuRns out that for four
elements, there are exactly
1151
01:04:58,780 --> 01:05:04,170
two ways of doing that,
generically.
1152
01:05:04,170 --> 01:05:07,860
One is sort of the
cyclic group.
1153
01:05:07,860 --> 01:05:09,070
I'm sorry --
1154
01:05:09,070 --> 01:05:16,690
a, b, c, 0, a, c, 0, a, b.
1155
01:05:16,690 --> 01:05:21,810
That's one way of filling out
the table such that each row
1156
01:05:21,810 --> 01:05:24,750
is a permutation
of 0, a, b, c.
1157
01:05:24,750 --> 01:05:28,060
Each column is a permutation
of 0, a, b, c.
1158
01:05:28,060 --> 01:05:31,860
So that's a legitimate
group table.
1159
01:05:31,860 --> 01:05:34,920
Let me just, for your interest,
show you that that's
1160
01:05:34,920 --> 01:05:38,120
not the only one.
1161
01:05:38,120 --> 01:05:42,950
We could also do 0, a,
b, c, 0, a, b, c.
1162
01:05:42,950 --> 01:05:45,290
If I write 0 here, then
I've got to write
1163
01:05:45,290 --> 01:05:47,480
c there and b there.
1164
01:05:47,480 --> 01:05:51,790
Got to write c here
and b there.
1165
01:05:51,790 --> 01:05:57,350
And then down here, the only
choice I have is this.
1166
01:05:57,350 --> 01:06:02,400
This is called the cyclic
group of four elements.
1167
01:06:02,400 --> 01:06:05,880
This is called the
Klein four group.
1168
01:06:05,880 --> 01:06:07,500
And this is all there is.
1169
01:06:07,500 --> 01:06:11,830
Both of them tuRn out to be
abelian if you check it out.
1170
01:06:11,830 --> 01:06:16,780
And basically, this permutation
property restricts
1171
01:06:16,780 --> 01:06:20,930
these to be the possible groups
of size four, abelian
1172
01:06:20,930 --> 01:06:23,800
or non-abelian, just to
satisfy the axiom.
1173
01:06:23,800 --> 01:06:28,270
1174
01:06:28,270 --> 01:06:33,840
I actually prove in the notes
that if you replace axioms a
1175
01:06:33,840 --> 01:06:43,020
and b with the permutation
property, you get another
1176
01:06:43,020 --> 01:06:50,830
equivalent set of axioms
for a group.
1177
01:06:50,830 --> 01:06:57,340
So the permutation of properties
is very basic.
1178
01:06:57,340 --> 01:06:59,280
All you need is --
1179
01:06:59,280 --> 01:07:03,070
for a set and a binary operation
to form a group, all
1180
01:07:03,070 --> 01:07:06,600
you need is identity,
associativity, and the
1181
01:07:06,600 --> 01:07:10,800
permutation property for
the group table,
1182
01:07:10,800 --> 01:07:13,740
the addition table.
1183
01:07:13,740 --> 01:07:15,600
Simple proof of that
in the notes.
1184
01:07:15,600 --> 01:07:18,190
1185
01:07:18,190 --> 01:07:22,050
But this observation
is going to be
1186
01:07:22,050 --> 01:07:24,720
important as we go forward.
1187
01:07:24,720 --> 01:07:27,310
It's a fundamental thing
we're going to use.
1188
01:07:27,310 --> 01:07:29,720
So you see a group can't
just be anything.
1189
01:07:29,720 --> 01:07:33,610
It already has very definite
restrictions on it.
1190
01:07:33,610 --> 01:07:38,053
1191
01:07:38,053 --> 01:07:38,840
All right.
1192
01:07:38,840 --> 01:07:43,144
Anything else I want
to say right here?
1193
01:07:43,144 --> 01:07:45,026
OK, subtraction, cancellation.
1194
01:07:45,026 --> 01:07:48,210
1195
01:07:48,210 --> 01:08:01,400
OK, I guess we can go into
the next property,
1196
01:08:01,400 --> 01:08:03,980
which is cyclic groups.
1197
01:08:03,980 --> 01:08:09,830
And I'm only going to talk about
finite cyclic groups, or
1198
01:08:09,830 --> 01:08:12,090
finite groups in general.
1199
01:08:12,090 --> 01:08:14,640
Now, a cyclic group
is probably the
1200
01:08:14,640 --> 01:08:15,890
simplest kind of group.
1201
01:08:15,890 --> 01:08:22,140
1202
01:08:22,140 --> 01:08:38,590
Let me first put up the
canonical cyclic group, which
1203
01:08:38,590 --> 01:08:52,910
is the integers mod-n, which
means the set of residues
1204
01:08:52,910 --> 01:08:56,854
mod-n and the mod-n addition
operation.
1205
01:08:56,854 --> 01:09:00,439
1206
01:09:00,439 --> 01:09:05,120
Which is written just as Zn --
1207
01:09:05,120 --> 01:09:07,029
there are various notations
for it.
1208
01:09:07,029 --> 01:09:13,029
But this, of course, we'll
just write it as Zn.
1209
01:09:13,029 --> 01:09:14,279
The integers mod-n.
1210
01:09:14,279 --> 01:09:17,109
1211
01:09:17,109 --> 01:09:20,350
Does this form a group?
1212
01:09:20,350 --> 01:09:21,835
Let's check the axioms.
1213
01:09:21,835 --> 01:09:25,350
1214
01:09:25,350 --> 01:09:29,439
Certainly, let's take
the original axioms.
1215
01:09:29,439 --> 01:09:33,060
So the Rn is closed under
addition mod-n.
1216
01:09:33,060 --> 01:09:37,830
If we add two elements, we're
going to get another element.
1217
01:09:37,830 --> 01:09:40,325
The mod-n addition operation
is associative.
1218
01:09:40,325 --> 01:09:42,560
In fact, it's abelian.
1219
01:09:42,560 --> 01:09:45,510
a plus b plus c is b plus c plus
a -- it doesn't matter
1220
01:09:45,510 --> 01:09:47,310
what order, from the
same property
1221
01:09:47,310 --> 01:09:49,510
for ordinary addition.
1222
01:09:49,510 --> 01:09:51,970
Is there an identity in here?
1223
01:09:51,970 --> 01:09:52,930
Yes.
1224
01:09:52,930 --> 01:09:56,630
The 0 is one of the elements
here and acts as the identity.
1225
01:09:56,630 --> 01:10:01,120
0 plus anything mod-n is equal
to the same thing mod-n.
1226
01:10:01,120 --> 01:10:04,660
And is there an inverse?
1227
01:10:04,660 --> 01:10:05,910
OK.
1228
01:10:05,910 --> 01:10:08,430
1229
01:10:08,430 --> 01:10:15,155
Let's explicitly write out Rn
as 0, 1 up to n minus 1.
1230
01:10:15,155 --> 01:10:18,120
1231
01:10:18,120 --> 01:10:26,080
And then I claim that if I take
any of these i, then n
1232
01:10:26,080 --> 01:10:29,130
minus i is its inverse.
1233
01:10:29,130 --> 01:10:35,270
n minus i is in the right
range if not zero.
1234
01:10:35,270 --> 01:10:36,770
The inverse of 0 is 0.
1235
01:10:36,770 --> 01:10:41,800
The inverse of i between 1 and
n minus 1 is going to be n
1236
01:10:41,800 --> 01:10:46,270
minus i because if I add
i to n minus i --
1237
01:10:46,270 --> 01:10:49,730
is equal to n, take mod-n,
that's equal to 0.
1238
01:10:49,730 --> 01:10:55,050
So every element has the
additive inverse.
1239
01:10:55,050 --> 01:11:01,460
And so the group table has a
very boring form, which does
1240
01:11:01,460 --> 01:11:04,080
satisfy the permutation
property.
1241
01:11:04,080 --> 01:11:06,910
If I take Z5, for instance.
1242
01:11:06,910 --> 01:11:09,620
1243
01:11:09,620 --> 01:11:09,845
0,1,2,3,4.
1244
01:11:09,845 --> 01:11:11,860
0 plus anything looks
like that.
1245
01:11:11,860 --> 01:11:16,250
1 times anything is
just 1,2,3,4 and 1
1246
01:11:16,250 --> 01:11:19,030
plus 4 is 0 mod 5.
1247
01:11:19,030 --> 01:11:31,400
2,3,4,0,1, and so forth.
1248
01:11:31,400 --> 01:11:31,650
3, 4, 0, 1, 2, 4, 0, 1, 2, 3.
1249
01:11:31,650 --> 01:11:35,930
Each row or column is just a
cyclic shift to the previous
1250
01:11:35,930 --> 01:11:36,455
row or column.
1251
01:11:36,455 --> 01:11:37,770
Just fill it out.
1252
01:11:37,770 --> 01:11:40,400
And does it satisfy the
permutation property?
1253
01:11:40,400 --> 01:11:43,070
Yes, it does.
1254
01:11:43,070 --> 01:11:49,400
This one here, for instance, you
can recognize as a cyclic
1255
01:11:49,400 --> 01:11:50,360
group table.
1256
01:11:50,360 --> 01:11:55,930
And so, apart from relabeling,
this is the group table of Z4.
1257
01:11:55,930 --> 01:12:01,840
Or we take a, b, c
equal to 1, 2, 3.
1258
01:12:01,840 --> 01:12:06,050
If two groups have the same
group table up to relabelling,
1259
01:12:06,050 --> 01:12:08,720
they're said to be isomorphic.
1260
01:12:08,720 --> 01:12:09,460
OK?
1261
01:12:09,460 --> 01:12:12,610
That's an important
concept in math.
1262
01:12:12,610 --> 01:12:16,680
And all it means is the two
groups act the same, except
1263
01:12:16,680 --> 01:12:18,840
that we've changed the names
of the group elements.
1264
01:12:18,840 --> 01:12:23,690
1265
01:12:23,690 --> 01:12:25,580
OK.
1266
01:12:25,580 --> 01:12:31,710
So what I'm going to show
is that all cyclic --
1267
01:12:31,710 --> 01:12:37,020
this group is sort of cyclic
because, as you see in the
1268
01:12:37,020 --> 01:12:42,830
notes, we can think of adding
1 as just being
1269
01:12:42,830 --> 01:12:45,200
cycling around a circle.
1270
01:12:45,200 --> 01:12:48,140
1271
01:12:48,140 --> 01:12:51,130
In this case, with
5 points on it.
1272
01:12:51,130 --> 01:12:55,280
And adding 2 is going two
steps around the circle.
1273
01:12:55,280 --> 01:12:57,880
And the elements on the
circle are the group
1274
01:12:57,880 --> 01:12:59,130
elements 0, 1, 2, 3, 4.
1275
01:12:59,130 --> 01:13:02,160
1276
01:13:02,160 --> 01:13:04,620
And then we get to 5.
1277
01:13:04,620 --> 01:13:06,860
We identify that with 0.
1278
01:13:06,860 --> 01:13:08,140
So we close the loop.
1279
01:13:08,140 --> 01:13:10,600
One we identify with 6.
1280
01:13:10,600 --> 01:13:11,645
This with 7.
1281
01:13:11,645 --> 01:13:13,270
This with 8.
1282
01:13:13,270 --> 01:13:14,890
Up to as many as you like.
1283
01:13:14,890 --> 01:13:17,500
So that's the intuition
why these are
1284
01:13:17,500 --> 01:13:19,000
called cyclic groups.
1285
01:13:19,000 --> 01:13:24,640
The addition operation cycles
rather than being on a line,
1286
01:13:24,640 --> 01:13:26,760
as it would be with the ordinary
integers, where we go
1287
01:13:26,760 --> 01:13:28,530
infinitely in either
direction.
1288
01:13:28,530 --> 01:13:31,880
1289
01:13:31,880 --> 01:13:49,150
So the formal definition of a
cyclic group is that g has a
1290
01:13:49,150 --> 01:13:50,400
single generator.
1291
01:13:50,400 --> 01:14:02,570
1292
01:14:02,570 --> 01:14:06,250
Call it a little g.
1293
01:14:06,250 --> 01:14:09,120
And -- am I running over?
1294
01:14:09,120 --> 01:14:11,850
Close to.
1295
01:14:11,850 --> 01:14:12,530
Is that really right?
1296
01:14:12,530 --> 01:14:14,480
I only have five minutes left.
1297
01:14:14,480 --> 01:14:18,700
We're not going to go at
the speed I had hoped.
1298
01:14:18,700 --> 01:14:19,810
g is --
1299
01:14:19,810 --> 01:14:21,680
what does that mean?
1300
01:14:21,680 --> 01:14:26,210
If we know that g contains g,
and this is never going to be
1301
01:14:26,210 --> 01:14:30,740
equal to 0, then we know that
g contains two elements at
1302
01:14:30,740 --> 01:14:33,480
least: 0 and g.
1303
01:14:33,480 --> 01:14:45,090
And by the group axioms, g
contains g, g plus g, g
1304
01:14:45,090 --> 01:14:46,970
plus g plus g --
1305
01:14:46,970 --> 01:14:49,140
we can keep adding them.
1306
01:14:49,140 --> 01:14:53,340
I'm going to call these the
integer multiples of g.
1307
01:14:53,340 --> 01:14:58,940
So this will be 1g, 2g, 3g.
1308
01:14:58,940 --> 01:15:00,800
That's what I mean when
I write an integer
1309
01:15:00,800 --> 01:15:01,580
in front of g --
1310
01:15:01,580 --> 01:15:06,560
I mean g plus itself, the
integer number of times.
1311
01:15:06,560 --> 01:15:09,020
And all those things, by the
group properties, have to be
1312
01:15:09,020 --> 01:15:11,430
in the group.
1313
01:15:11,430 --> 01:15:12,680
All right?
1314
01:15:12,680 --> 01:15:14,640
1315
01:15:14,640 --> 01:15:17,242
Again, restrict this
to finite.
1316
01:15:17,242 --> 01:15:18,985
It's a finite cyclic group.
1317
01:15:18,985 --> 01:15:22,150
1318
01:15:22,150 --> 01:15:24,960
Then at some point,
I have to --
1319
01:15:24,960 --> 01:15:26,610
this is all the elements
of the group.
1320
01:15:26,610 --> 01:15:29,300
1321
01:15:29,300 --> 01:15:31,500
So at some point, I have
to come along and
1322
01:15:31,500 --> 01:15:36,220
say that ng is 0.
1323
01:15:36,220 --> 01:15:37,760
If just by --
1324
01:15:37,760 --> 01:15:41,570
when I say generator, the
set generated by g is
1325
01:15:41,570 --> 01:15:44,110
just 1g, 2g, 3g.
1326
01:15:44,110 --> 01:15:46,190
And that's got to be all the
elements of the group, so n
1327
01:15:46,190 --> 01:15:47,780
times g has got to be 0.
1328
01:15:47,780 --> 01:15:52,560
1329
01:15:52,560 --> 01:15:59,130
And that means, without putting
too fine a point on
1330
01:15:59,130 --> 01:16:03,400
it, that if n were 5, for
instance, the group has to
1331
01:16:03,400 --> 01:16:11,720
consist of 0g, 1g, 2g, 3g, 4g,
and then 5g is equal to 0.
1332
01:16:11,720 --> 01:16:14,860
So we call this 0g or 5g.
1333
01:16:14,860 --> 01:16:17,820
1334
01:16:17,820 --> 01:16:22,976
And furthermore, the addition
table of the group, we can,
1335
01:16:22,976 --> 01:16:24,380
you know --
1336
01:16:24,380 --> 01:16:28,380
however, if we add 2 g's to 3
g's, we're going to 5 g's,
1337
01:16:28,380 --> 01:16:29,920
which is 0.
1338
01:16:29,920 --> 01:16:30,340
All right?
1339
01:16:30,340 --> 01:16:35,150
So 2g plus 3g has got to
be 5 g's, which is 0g.
1340
01:16:35,150 --> 01:16:37,783
1341
01:16:37,783 --> 01:16:41,570
So this is what the addition
table of the
1342
01:16:41,570 --> 01:16:44,400
group has to look like.
1343
01:16:44,400 --> 01:16:47,510
It's exactly the same, except
we just make a one-to-one
1344
01:16:47,510 --> 01:16:53,640
correspondence like that.
1345
01:16:53,640 --> 01:17:11,250
So the conclusion is a
finite cyclic group
1346
01:17:11,250 --> 01:17:15,420
with n elements --
1347
01:17:15,420 --> 01:17:18,425
I'll write the size of g by this
absolute value thing --
1348
01:17:18,425 --> 01:17:24,020
1349
01:17:24,020 --> 01:17:31,770
is isomorphic to the
integers mod-n --
1350
01:17:31,770 --> 01:17:35,260
this canonical cyclic group
that we started with --
1351
01:17:35,260 --> 01:17:43,340
must be isomorphic to Zn with
the isomorphism being given by
1352
01:17:43,340 --> 01:17:44,633
the one-to-one correspondence.
1353
01:17:44,633 --> 01:17:53,090
1354
01:17:53,090 --> 01:18:03,140
i_g in the group corresponds
to simply i in Zn, or 0 is
1355
01:18:03,140 --> 01:18:06,120
less than or equal to
i is less than or
1356
01:18:06,120 --> 01:18:07,530
equal to n minus 1.
1357
01:18:07,530 --> 01:18:10,390
1358
01:18:10,390 --> 01:18:14,230
The addition rule is
exactly the same.
1359
01:18:14,230 --> 01:18:18,920
OK, so this is a pretty
sweeping conclusion.
1360
01:18:18,920 --> 01:18:22,330
It says that really the only
finite cyclic group up to
1361
01:18:22,330 --> 01:18:25,990
relabeling is Zn.
1362
01:18:25,990 --> 01:18:28,560
OK, so if you understand Zn,
you understand all finite
1363
01:18:28,560 --> 01:18:30,120
cyclic groups.
1364
01:18:30,120 --> 01:18:34,120
Just in a general group, we're
going to have some element
1365
01:18:34,120 --> 01:18:38,490
which basically represents 1,
called g, and its integer
1366
01:18:38,490 --> 01:18:43,470
multiples, which represent
2, 3, 4, and so forth.
1367
01:18:43,470 --> 01:18:47,650
But we can operate with
a group just as
1368
01:18:47,650 --> 01:18:50,140
though it was Zn.
1369
01:18:50,140 --> 01:18:53,400
So it's easy to understand
finite cyclic groups.
1370
01:18:53,400 --> 01:18:56,470
They all just look like this.
1371
01:18:56,470 --> 01:18:57,070
Question?
1372
01:18:57,070 --> 01:18:57,430
Yeah.
1373
01:18:57,430 --> 01:19:00,930
AUDIENCE: Is the generator
unique?
1374
01:19:00,930 --> 01:19:03,955
PROFESSOR: No, it's not.
1375
01:19:03,955 --> 01:19:06,380
No, it's not.
1376
01:19:06,380 --> 01:19:08,950
Next time we'll start
to talk about
1377
01:19:08,950 --> 01:19:12,370
subgroups of Zn, for instance.
1378
01:19:12,370 --> 01:19:15,600
1379
01:19:15,600 --> 01:19:21,690
In the set of Z5 --
1380
01:19:21,690 --> 01:19:24,720
well, let me take this.
1381
01:19:24,720 --> 01:19:26,310
All right.
1382
01:19:26,310 --> 01:19:27,610
In Z5 --
1383
01:19:27,610 --> 01:19:28,960
let me not even do that.
1384
01:19:28,960 --> 01:19:35,160
In Z5, 1 is the generator, but
because 5 is a prime, 2 is a
1385
01:19:35,160 --> 01:19:36,880
generator, too.
1386
01:19:36,880 --> 01:19:38,770
All right?
1387
01:19:38,770 --> 01:19:40,610
Another way of writing.
1388
01:19:40,610 --> 01:19:45,305
Let's take a generator
equals 2 in Z5.
1389
01:19:45,305 --> 01:19:50,370
Then g equals 2, 2g equals 4.
1390
01:19:50,370 --> 01:19:53,820
3g equals 6, equals 1 mod-5.
1391
01:19:53,820 --> 01:19:56,360
4g equals 3.
1392
01:19:56,360 --> 01:20:01,950
And 5g, of course, equals
5, which equals zero.
1393
01:20:01,950 --> 01:20:04,610
So 2 is also a generator.
1394
01:20:04,610 --> 01:20:07,990
And, in fact, 3 and 4
are also generators.
1395
01:20:07,990 --> 01:20:09,160
There are four generators.
1396
01:20:09,160 --> 01:20:11,340
This is getting us right
into the number theory.
1397
01:20:11,340 --> 01:20:13,940
1398
01:20:13,940 --> 01:20:14,230
OK.
1399
01:20:14,230 --> 01:20:18,780
So no, g does not
have to be 1.
1400
01:20:18,780 --> 01:20:21,870
g is just something that if
you take all its integer
1401
01:20:21,870 --> 01:20:25,870
multiples, you generate
the group.
1402
01:20:25,870 --> 01:20:31,630
OK, I guess we stop there, and
we'll resume next time.
1403
01:20:31,630 --> 01:20:43,635