1
00:00:00,405 --> 00:00:04,200
PROFESSOR: Chapter six, problem
set three, problem set

2
00:00:04,200 --> 00:00:06,140
two solutions.

3
00:03:27,510 --> 00:03:28,100
OK.

4
00:03:28,100 --> 00:03:29,460
Good morning.

5
00:03:29,460 --> 00:03:32,690
Nice to see you all again
on this mild day.

6
00:03:32,690 --> 00:03:37,070
I want to start off by thanking
Ashish greatly for

7
00:03:37,070 --> 00:03:39,610
giving the last three
lectures.

8
00:03:39,610 --> 00:03:42,420
I have very high confidence in
Ashish, and I'm sure he give

9
00:03:42,420 --> 00:03:45,090
you excellent lectures and was
able to answer all your

10
00:03:45,090 --> 00:03:46,340
questions, right?

11
00:03:48,900 --> 00:03:52,000
But if you feel dissatisfied at
any point, you'll have an

12
00:03:52,000 --> 00:03:54,295
opportunity to ask the
question again today.

13
00:03:57,020 --> 00:03:59,930
We will continue now
in chapter six.

14
00:03:59,930 --> 00:04:04,010
I understand Ashish got up to
section six point one already,

15
00:04:04,010 --> 00:04:07,440
so that makes my life
a little easier.

16
00:04:07,440 --> 00:04:09,870
And we'll just continue to plow
ahead through the notes.

17
00:04:09,870 --> 00:04:16,600
You have as handouts today
chapter six, the outgoing

18
00:04:16,600 --> 00:04:21,769
problem set, the solutions to
the incoming problem set.

19
00:04:21,769 --> 00:04:26,690
So does anyone have any comments
or questions or

20
00:04:26,690 --> 00:04:30,120
suggestions as we go forward
at this point?

21
00:04:30,120 --> 00:04:34,360
Let's say I'll do a very quick
review of everything so far,

22
00:04:34,360 --> 00:04:37,440
at least what we need to
proceed, so you'll have a

23
00:04:37,440 --> 00:04:40,870
chance to ask questions along
there if you like.

24
00:04:40,870 --> 00:04:43,040
Anyone have anything to say?

25
00:04:43,040 --> 00:04:44,570
Anyone want to say
thanks to Ashish?

26
00:04:48,830 --> 00:04:49,280
All right.

27
00:04:49,280 --> 00:04:51,970
Great.

28
00:04:51,970 --> 00:04:52,550
OK.

29
00:04:52,550 --> 00:04:57,190
So a very quick review.

30
00:04:57,190 --> 00:05:02,300
We first said we were going to
work with a continuous time

31
00:05:02,300 --> 00:05:05,260
Additive White Gaussian noise
channel, but we quickly

32
00:05:05,260 --> 00:05:08,780
reduced it to a discrete
time additive white

33
00:05:08,780 --> 00:05:10,110
Gaussian noise model.

34
00:05:10,110 --> 00:05:15,030
Vector model, Y equals X plus N,
where the vectors may go on

35
00:05:15,030 --> 00:05:16,690
as long as you like.

36
00:05:16,690 --> 00:05:19,390
And implicit in here is we
have some kind of power

37
00:05:19,390 --> 00:05:21,980
limitation expressed
by various --

38
00:05:21,980 --> 00:05:24,930
I know I've probably introduced
too many parameters

39
00:05:24,930 --> 00:05:27,710
in the early part
of the course.

40
00:05:27,710 --> 00:05:31,820
But you'll see them all in the
literature, and I thought you

41
00:05:31,820 --> 00:05:36,180
might as well see the relation
between them all at once.

42
00:05:36,180 --> 00:05:36,850
The noise.

43
00:05:36,850 --> 00:05:40,350
The main thing about this
is it's an independent,

44
00:05:40,350 --> 00:05:44,640
identically distributed noise,
also independent of X, and

45
00:05:44,640 --> 00:05:49,020
it's characterized simply by
its variance per symbol, or

46
00:05:49,020 --> 00:05:49,960
average power.

47
00:05:49,960 --> 00:05:54,360
Again, there are various
notations that we

48
00:05:54,360 --> 00:05:57,310
can use for its power.

49
00:05:57,310 --> 00:05:58,310
OK.

50
00:05:58,310 --> 00:06:03,310
However you express the power
of the signal of the noise,

51
00:06:03,310 --> 00:06:05,290
you're going to get some
signal-to-noise ratio out of

52
00:06:05,290 --> 00:06:09,160
that, as long as you use
consistent the same units for

53
00:06:09,160 --> 00:06:10,960
signal and noise.

54
00:06:10,960 --> 00:06:13,860
And so the key parameters
of this channel turn

55
00:06:13,860 --> 00:06:16,160
out to be just two.

56
00:06:16,160 --> 00:06:19,200
One is the signal-to-noise
ratio.

57
00:06:19,200 --> 00:06:23,000
And the other is something which
in the discrete time

58
00:06:23,000 --> 00:06:26,260
domain we might think of as a
data rate, rho, the number of

59
00:06:26,260 --> 00:06:28,880
bits per two dimensions.

60
00:06:28,880 --> 00:06:32,360
But we were at pains
to show --

61
00:06:32,360 --> 00:06:35,380
and the reason we measure it in
bits per two dimensions is

62
00:06:35,380 --> 00:06:40,180
that it converts directly to the
spectral efficiency, or we

63
00:06:40,180 --> 00:06:42,990
sometimes say, the nominal
spectral efficiency, or the

64
00:06:42,990 --> 00:06:45,340
Nyquist spectral efficiency.

65
00:06:45,340 --> 00:06:48,330
Basically, this is the spectral
efficiency if you

66
00:06:48,330 --> 00:06:52,620
assume that you only use the
nominal Nyquist bandwidth.

67
00:06:52,620 --> 00:06:57,630
And both in 450 and here it was
shown that you can really

68
00:06:57,630 --> 00:07:03,710
get as close to the nominal
bandwidth as you like, its

69
00:07:03,710 --> 00:07:08,530
sharp or roll off as you
like, by using filters.

70
00:07:08,530 --> 00:07:15,830
Very sharp filters or very sharp
digital filters that

71
00:07:15,830 --> 00:07:20,660
today we can really
realistically get effectively

72
00:07:20,660 --> 00:07:23,700
to the Nyquist bandwidth.

73
00:07:23,700 --> 00:07:27,150
So this is a good measure of
nominal spectral efficiency,

74
00:07:27,150 --> 00:07:29,780
which is an extremely important
parameter, the

75
00:07:29,780 --> 00:07:33,920
spectral efficiency in
continuous time.

76
00:07:33,920 --> 00:07:36,820
And then the Shannon limit is
expressed in this very simple

77
00:07:36,820 --> 00:07:44,020
form that we can never do better
in data rate or nominal

78
00:07:44,020 --> 00:07:48,920
spectral efficiency than
log of 1 plus SNR.

79
00:07:48,920 --> 00:07:51,860
And the units are bits
per two dimensions.

80
00:07:51,860 --> 00:07:55,410
So there's an awful lot about
parameters and units in the

81
00:07:55,410 --> 00:07:57,850
first part of the course, and
sometimes it's a little

82
00:07:57,850 --> 00:07:59,100
overwhelming and confusing.

83
00:08:01,730 --> 00:08:05,670
But in my experience in
engineering is, it's extremely

84
00:08:05,670 --> 00:08:08,630
important to get the
units right.

85
00:08:08,630 --> 00:08:11,470
It helps you in thinking
clearly.

86
00:08:11,470 --> 00:08:14,040
It helps you to focus
on the right things.

87
00:08:14,040 --> 00:08:16,530
And so in the first couple of
chapters, there's a lot of

88
00:08:16,530 --> 00:08:18,950
emphasis on getting
the units right.

89
00:08:18,950 --> 00:08:21,490
For instance, bits per
two dimensions.

90
00:08:21,490 --> 00:08:25,590
I could simply assert to you
that things are always to come

91
00:08:25,590 --> 00:08:28,060
out better if you do things
per two dimensions.

92
00:08:28,060 --> 00:08:34,039
This basically has to do with
Gaussian distributions being

93
00:08:34,039 --> 00:08:37,374
simpler in two dimensions than
they are in one dimension.

94
00:08:37,374 --> 00:08:41,630
So you know you can get closed
form integrals in two and four

95
00:08:41,630 --> 00:08:44,420
and so forth dimensions that you
can't get in one and three

96
00:08:44,420 --> 00:08:45,610
and so forth dimensions.

97
00:08:45,610 --> 00:08:50,350
So you know, the geometrical
formulas for the volume of the

98
00:08:50,350 --> 00:08:53,060
sphere and so forth are much
simpler in even dimensions

99
00:08:53,060 --> 00:08:55,550
than they are in
odd dimensions.

100
00:08:55,550 --> 00:09:00,610
This all, I think, has to do
with the fact that really a

101
00:09:00,610 --> 00:09:03,750
one-dimensional complex Gaussian
variable, or in

102
00:09:03,750 --> 00:09:06,570
general, complex Gaussian
variables, are in some sense

103
00:09:06,570 --> 00:09:10,780
mathematically simpler than
real Gaussian variables.

104
00:09:10,780 --> 00:09:14,270
For instance, the Q function
is a very ugly thing.

105
00:09:14,270 --> 00:09:16,670
In two dimensions, you can
closed-form integrals with

106
00:09:16,670 --> 00:09:18,520
probability of error.

107
00:09:18,520 --> 00:09:22,210
So there's a lot of clues that
we really want to think in

108
00:09:22,210 --> 00:09:26,293
terms of pairs of
real dimensions.

109
00:09:29,580 --> 00:09:29,640
All right.

110
00:09:29,640 --> 00:09:32,280
And then we introduced a
couple more parameters.

111
00:09:32,280 --> 00:09:35,580
And I understand Ashish
got the questions

112
00:09:35,580 --> 00:09:36,460
that I get every year.

113
00:09:36,460 --> 00:09:39,040
Why do we introduce both
of these things?

114
00:09:39,040 --> 00:09:43,130
SNR norm, which is SNR
normalized by 2 to the rho

115
00:09:43,130 --> 00:09:45,690
minus 1, that's motivated
directly by the

116
00:09:45,690 --> 00:09:47,540
Shannon limit formula.

117
00:09:47,540 --> 00:09:52,720
Or Eb over N_0, which is
SNR divided by rho.

118
00:09:52,720 --> 00:09:54,170
You know, these are
both more or less

119
00:09:54,170 --> 00:09:55,940
equivalent to each other.

120
00:09:55,940 --> 00:10:00,240
From this Eb over N_0 is just 2
to the rho minus 1 over rho

121
00:10:00,240 --> 00:10:05,010
times SNR norm, so why do we
introduce both of them?

122
00:10:05,010 --> 00:10:11,900
And there's not an
intellectually very strong

123
00:10:11,900 --> 00:10:15,760
argument to introduce
both of them.

124
00:10:15,760 --> 00:10:20,530
Because one is easily translated
into the other.

125
00:10:20,530 --> 00:10:26,060
For fixed nominal spectral
efficiency rho, there clearly

126
00:10:26,060 --> 00:10:28,060
is just a one-to-one
translation.

127
00:10:28,060 --> 00:10:29,940
We could use either one.

128
00:10:29,940 --> 00:10:33,900
If you graph something versus
Eb over N_0 or versus SNR

129
00:10:33,900 --> 00:10:38,260
norm, it's just a matter of
shifting it by this factor.

130
00:10:38,260 --> 00:10:42,230
They're going to be exactly the
same graph with the 0 dB

131
00:10:42,230 --> 00:10:47,650
point in a different place,
according to where 0 dB is.

132
00:10:47,650 --> 00:10:52,460
Philosophically, of course, SNR
norm, which is sometimes

133
00:10:52,460 --> 00:10:58,960
called the gap to capacity, is
exactly a measure of how many

134
00:10:58,960 --> 00:11:02,070
dBs away are we from
the Shannon limit.

135
00:11:02,070 --> 00:11:06,570
Measuring things on
a log scale in dB.

136
00:11:06,570 --> 00:11:10,400
And so 0 dB is always the
Shannon limit point with SNR

137
00:11:10,400 --> 00:11:15,700
norm, because this statement
translates into SNR norm is

138
00:11:15,700 --> 00:11:18,900
greater than 1, which is 0 dB.

139
00:11:18,900 --> 00:11:23,040
So here the focus is always, how
far are you from capacity?

140
00:11:23,040 --> 00:11:25,270
How far are you from
the Shannon limit?

141
00:11:25,270 --> 00:11:29,200
And that really is
the modern view.

142
00:11:29,200 --> 00:11:32,410
In the early days of coding, the
view was, well, how much

143
00:11:32,410 --> 00:11:35,160
coding gain can we get
over no coding?

144
00:11:35,160 --> 00:11:42,960
And as we'll see, Eb over N_0
is often a very convenient

145
00:11:42,960 --> 00:11:46,370
parameter to use when we're
focusing on coding gain.

146
00:11:46,370 --> 00:11:49,040
In fact, for binary linear block
codes, which is what

147
00:11:49,040 --> 00:11:51,960
we're talking about in chapter
six, we get an extremely

148
00:11:51,960 --> 00:11:57,700
simple expression that Eb over
N_0 it's just kd over N, where

149
00:11:57,700 --> 00:12:02,590
N, k, d are the basic parameters
of a block code.

150
00:12:02,590 --> 00:12:04,150
If you don't know those
yet, you will by

151
00:12:04,150 --> 00:12:06,910
the end of this lecture.

152
00:12:06,910 --> 00:12:14,210
And Eb over N_0 is what was put
forward in the early days,

153
00:12:14,210 --> 00:12:17,370
when the principal coding
application was coding for

154
00:12:17,370 --> 00:12:19,180
deep space communications.

155
00:12:19,180 --> 00:12:22,320
It makes sense in the
power-limited regime.

156
00:12:22,320 --> 00:12:27,180
In the power-limited regime,
we showed that this

157
00:12:27,180 --> 00:12:31,540
essentially goes -- this is rho
log2 over rho, so up to a

158
00:12:31,540 --> 00:12:36,190
factor of natural logarithm of
two, these are the same,

159
00:12:36,190 --> 00:12:39,610
almost independent of the
spectral efficiency, as well.

160
00:12:39,610 --> 00:12:44,440
And so this is a very natural
thing to use, especially in

161
00:12:44,440 --> 00:12:46,240
the power-limited regime.

162
00:12:46,240 --> 00:12:50,180
In the bandwidth-limited regime,
as rho gets large,

163
00:12:50,180 --> 00:12:52,910
then these things become
very different.

164
00:12:52,910 --> 00:12:56,050
SNR norm always keeps us in
the neighborhood of 0 dB.

165
00:12:56,050 --> 00:13:01,210
This thing goes up to 10 dB,
20 dB, 30 dB, and so forth.

166
00:13:01,210 --> 00:13:03,940
Nonetheless, you'll see in
some literature, people

167
00:13:03,940 --> 00:13:09,910
continue to measure against Eb
over N_0 maybe I would say,

168
00:13:09,910 --> 00:13:11,290
just because they don't
know any better.

169
00:13:11,290 --> 00:13:16,000
Anyway, since I started writing
these notes about

170
00:13:16,000 --> 00:13:19,090
eight or nine years ago, I've
been advocating SNR norm.

171
00:13:19,090 --> 00:13:23,830
It's more and more widely
used in our business,

172
00:13:23,830 --> 00:13:24,930
in the coding business.

173
00:13:24,930 --> 00:13:30,680
Or equivalently, one always
shows nowadays how far are you

174
00:13:30,680 --> 00:13:35,370
from capacity, and that's
what SNR norm is about.

175
00:13:35,370 --> 00:13:38,690
And you can always translate
this into this and this into

176
00:13:38,690 --> 00:13:40,830
this by this simple formula.

177
00:13:40,830 --> 00:13:42,190
So that's why we introduce
them both.

178
00:13:42,190 --> 00:13:44,630
Eb over N_0 is traditional.

179
00:13:44,630 --> 00:13:49,821
SNR norm is more of the modern
gap to capacity viewpoint.

180
00:13:49,821 --> 00:13:51,430
Any questions about that?

181
00:13:51,430 --> 00:13:54,830
Because I understood that
Ashish got a number of

182
00:13:54,830 --> 00:13:56,410
questions about that.

183
00:13:56,410 --> 00:13:56,875
Yes?

184
00:13:56,875 --> 00:13:59,725
AUDIENCE: And the probability
of error you mentioned

185
00:13:59,725 --> 00:14:03,550
[UNINTELLIGIBLE] is natural
with SNR model.

186
00:14:03,550 --> 00:14:04,610
PROFESSOR: Well, this
is a slightly

187
00:14:04,610 --> 00:14:05,860
different point, actually.

188
00:14:08,010 --> 00:14:11,060
So we go on from this to talk
about the power-limited

189
00:14:11,060 --> 00:14:14,580
regime, which we defined more
or less arbitrarily as the

190
00:14:14,580 --> 00:14:17,860
regime where rho is less than
or equal to two bits per two

191
00:14:17,860 --> 00:14:20,620
dimensions, and the
bandwidth-limited regime,

192
00:14:20,620 --> 00:14:23,800
which is where rho is larger.

193
00:14:23,800 --> 00:14:30,310
And at this point, I simply
assert that it's better to do

194
00:14:30,310 --> 00:14:32,940
everything per two dimensions
in the bandwidth-limited

195
00:14:32,940 --> 00:14:39,120
regime and to do everything
per bit in the

196
00:14:39,120 --> 00:14:41,270
power-limited regime.

197
00:14:41,270 --> 00:14:47,410
And the reason for this is
basically long practice and

198
00:14:47,410 --> 00:14:51,090
intuition and experience that
things do work out better, and

199
00:14:51,090 --> 00:14:53,980
this is the proper
normalization.

200
00:14:53,980 --> 00:14:55,920
But I think at this point
in the course, with your

201
00:14:55,920 --> 00:15:00,350
background, this is only an
assertion, all right?

202
00:15:00,350 --> 00:15:05,590
So I simply say, this is the way
we're going to do things

203
00:15:05,590 --> 00:15:08,140
in bandwidth-limited
and power-limited.

204
00:15:08,140 --> 00:15:13,220
For most of the rest of the
course, we're going to be in

205
00:15:13,220 --> 00:15:14,400
the power-limited regime.

206
00:15:14,400 --> 00:15:18,820
Then we'll come back to the
bandwidth-limited very late in

207
00:15:18,820 --> 00:15:19,810
the course.

208
00:15:19,810 --> 00:15:26,400
So you can now forget this
assertion for a while.

209
00:15:26,400 --> 00:15:26,850
Yes?

210
00:15:26,850 --> 00:15:28,100
AUDIENCE: [INAUDIBLE]

211
00:15:31,270 --> 00:15:33,430
PROFESSOR: Well, I'm trying
to focus on that here.

212
00:15:33,430 --> 00:15:38,390
That if two systems have
different rhos, then you

213
00:15:38,390 --> 00:15:42,330
should take that into account
in their comparison.

214
00:15:42,330 --> 00:15:59,200
And when you see charts of a
rate 7/8 system, or a rho 7/4

215
00:15:59,200 --> 00:16:03,650
system versus one that's rate
1/8, those are very different

216
00:16:03,650 --> 00:16:10,490
regimes, and it probably isn't
fair to compare two systems,

217
00:16:10,490 --> 00:16:13,380
just in terms of their
probability of error versus Eb

218
00:16:13,380 --> 00:16:16,480
over N_0 at two different
spectral efficiencies.

219
00:16:16,480 --> 00:16:21,310
What is more fair is to compare
them in terms of their

220
00:16:21,310 --> 00:16:22,560
gap to capacity.

221
00:16:27,210 --> 00:16:30,640
You can get more powerful
codes with more

222
00:16:30,640 --> 00:16:34,570
error-correcting capability the
lower in rate that you go.

223
00:16:34,570 --> 00:16:39,960
And so you should start from
the point of view that two

224
00:16:39,960 --> 00:16:45,330
systems with different rho are
incomparable, and then using

225
00:16:45,330 --> 00:16:48,730
this you can say, but, you know,
if we can get within 1

226
00:16:48,730 --> 00:16:51,440
dB of the Shannon limit with
both of them, then they're

227
00:16:51,440 --> 00:16:53,890
both approximately
equally powerful.

228
00:16:53,890 --> 00:16:56,640
That would be the modern
point of view.

229
00:16:56,640 --> 00:17:00,400
But it's really apples and
oranges if they have different

230
00:17:00,400 --> 00:17:03,560
rates, different rhos.

231
00:17:03,560 --> 00:17:03,990
Yes?

232
00:17:03,990 --> 00:17:05,240
AUDIENCE: [INAUDIBLE]

233
00:17:07,550 --> 00:17:10,115
If [UNINTELLIGIBLE]

234
00:17:10,115 --> 00:17:13,349
systems modulation
[INAUDIBLE].

235
00:17:13,349 --> 00:17:16,910
Then is it fair --

236
00:17:16,910 --> 00:17:21,079
because if one of them
has coding --

237
00:17:21,079 --> 00:17:24,710
PROFESSOR: If none of them
have coding, well --

238
00:17:24,710 --> 00:17:26,054
AUDIENCE: No, just modulation.

239
00:17:28,960 --> 00:17:31,870
PROFESSOR: So what shall
I say to that.

240
00:17:31,870 --> 00:17:35,500
In the power-limited regime,
an uncoded system is simply

241
00:17:35,500 --> 00:17:41,900
binary modulation or 4-QAM,
and there aren't different

242
00:17:41,900 --> 00:17:43,170
systems to compare, really.

243
00:17:43,170 --> 00:17:46,850
There's only one baseline
system.

244
00:17:46,850 --> 00:17:51,105
As you go up into the
bandwidth-limited regime, then

245
00:17:51,105 --> 00:17:55,290
it is fair to compare
systems of the same

246
00:17:55,290 --> 00:17:58,650
class, like M by M QAM.

247
00:17:58,650 --> 00:18:01,180
That's a simple uncoded
system.

248
00:18:01,180 --> 00:18:04,060
Now there, rho keeps changing.

249
00:18:04,060 --> 00:18:08,260
rho equals 2, rho equals 4, rho
equals 6, rho equals 8.

250
00:18:08,260 --> 00:18:11,210
Or you can easily find
intermediate

251
00:18:11,210 --> 00:18:13,610
rates that are uncoded.

252
00:18:13,610 --> 00:18:17,740
And there you find that this
normalization makes all these

253
00:18:17,740 --> 00:18:19,540
systems comparable.

254
00:18:19,540 --> 00:18:24,160
In fact, we saw that we got a
universal baseline curve for

255
00:18:24,160 --> 00:18:28,100
the bandwidth-limited regime,
which was independent of rho,

256
00:18:28,100 --> 00:18:33,520
for, say, the class of 4 by
4, by the class of M by M

257
00:18:33,520 --> 00:18:37,900
bandwidth-limited QAM systems.

258
00:18:37,900 --> 00:18:38,300
I'm sorry.

259
00:18:38,300 --> 00:18:41,600
I got off a plane at nine
o'clock last night, so I may

260
00:18:41,600 --> 00:18:45,440
not be totally coherent today.

261
00:18:45,440 --> 00:18:49,750
But all of these systems have
exactly the same baseline

262
00:18:49,750 --> 00:18:55,210
curve up to the approximations
we've made that their exactly

263
00:18:55,210 --> 00:18:57,130
four nearest neighbors
and so forth.

264
00:18:57,130 --> 00:19:02,040
But on a curve of probability
of error per two dimensions,

265
00:19:02,040 --> 00:19:06,930
per QAM symbol, versus
SNR norm, we

266
00:19:06,930 --> 00:19:08,290
get a universal curve.

267
00:19:08,290 --> 00:19:11,560
So that indicates they really
are comparable.

268
00:19:11,560 --> 00:19:15,460
They form a class where the
performance of the class is

269
00:19:15,460 --> 00:19:16,820
independent of rho.

270
00:19:16,820 --> 00:19:20,730
And that's sort of typical of
the bandwidth-limited regime.

271
00:19:20,730 --> 00:19:24,010
Two systems that differ, that
basically use the same code

272
00:19:24,010 --> 00:19:28,220
with smaller constellation
or a larger-based uncoded

273
00:19:28,220 --> 00:19:30,870
constellation are going to be
directly comparable, and are

274
00:19:30,870 --> 00:19:35,370
probably going to have the
same gap to capacity.

275
00:19:35,370 --> 00:19:37,370
Going to be just as far away
from the Shannon limit.

276
00:19:37,370 --> 00:19:39,410
But we don't have that
phenomenon in the

277
00:19:39,410 --> 00:19:40,660
power-limited case.

278
00:19:45,080 --> 00:19:46,780
Other questions, or
shall I proceed?

279
00:19:49,810 --> 00:19:51,060
OK.

280
00:19:52,880 --> 00:19:58,490
Again continuing what I hope
will be a quick review, but I

281
00:19:58,490 --> 00:20:00,720
don't want to go any faster than
you're comfortable with.

282
00:20:05,960 --> 00:20:12,140
We get this baseline curve
for 2-PAM or 2

283
00:20:12,140 --> 00:20:14,330
by 2 QAM, same thing.

284
00:20:18,400 --> 00:20:26,520
So this is the baseline of Pb of
E on a log scale, where we

285
00:20:26,520 --> 00:20:29,460
typically go down to ten to
the minus six, ten to the

286
00:20:29,460 --> 00:20:31,050
minus five.

287
00:20:31,050 --> 00:20:33,320
Here's 0 dB.

288
00:20:33,320 --> 00:20:37,670
This goes down through
about 9.6 dB here or

289
00:20:37,670 --> 00:20:40,710
about 10.5 dB here.

290
00:20:40,710 --> 00:20:47,010
And the ultimate Shannon
limit is over here.

291
00:20:47,010 --> 00:20:52,150
The ultimate Shannon limit for
very low rho, as rho goes to

292
00:20:52,150 --> 00:20:55,840
0, is about minus 1.6 dB.

293
00:20:55,840 --> 00:20:57,630
And we get expressions.

294
00:20:57,630 --> 00:21:00,780
0 dB is the Shannon limit
at rho equals 1.

295
00:21:00,780 --> 00:21:05,570
At rho equals 2, it's
up about 1.8 dB.

296
00:21:05,570 --> 00:21:08,250
Shannon limit for rho equals
2, and so forth.

297
00:21:08,250 --> 00:21:12,390
And anyway, so we see as a
function of rho, we can

298
00:21:12,390 --> 00:21:16,210
measure the gap to capacity at
different probabilities of

299
00:21:16,210 --> 00:21:19,780
error and see how much
coding gain is,

300
00:21:19,780 --> 00:21:20,780
in principle, possible.

301
00:21:20,780 --> 00:21:26,610
And then for a coded system,
we can put that on here.

302
00:21:26,610 --> 00:21:29,980
The effective coding gain
is the distance

303
00:21:29,980 --> 00:21:32,760
between here and here.

304
00:21:32,760 --> 00:21:36,130
At some target probability
of error.

305
00:21:36,130 --> 00:21:39,550
It's going to differ according
to probably of error.

306
00:21:39,550 --> 00:21:46,740
You found a way of getting a
good, rough-cut estimate, at

307
00:21:46,740 --> 00:21:50,630
least for not very complicated
codes called the union bound

308
00:21:50,630 --> 00:21:57,040
estimate for any signal
constellation in any number of

309
00:21:57,040 --> 00:21:58,290
dimensions.

310
00:21:58,290 --> 00:22:04,980
So if we have a signal
constellation A, which

311
00:22:04,980 --> 00:22:10,180
consists of M points in N
dimensions, so forth,

312
00:22:10,180 --> 00:22:16,630
basically found that we could,
in the power-limited regime,

313
00:22:16,630 --> 00:22:22,190
get an approximate expression
from considering pairwise

314
00:22:22,190 --> 00:22:25,400
error probabilities that's
very simple.

315
00:22:25,400 --> 00:22:27,140
And I used this notation.

316
00:22:27,140 --> 00:22:34,300
Q of the square root of some
coding gain of the

317
00:22:34,300 --> 00:22:40,330
constellation times
2 Eb over N_0.

318
00:22:40,330 --> 00:22:45,150
If it's just 2-PAM then the
coding gain becomes 1.

319
00:22:45,150 --> 00:22:52,430
This is the average number
of nearest neighbors per

320
00:22:52,430 --> 00:22:55,810
transmitted bit.

321
00:22:55,810 --> 00:23:00,470
And so the whole thing reduces
to this expression in terms of

322
00:23:00,470 --> 00:23:01,610
couple of parameters.

323
00:23:01,610 --> 00:23:06,610
the principal parameter is the
nominal coding gain, which is

324
00:23:06,610 --> 00:23:13,840
the minimum squared distance of
A over 4 times the energy

325
00:23:13,840 --> 00:23:19,170
per bit of this constellation
A. So we really only need to

326
00:23:19,170 --> 00:23:24,950
know this kind of normalized
measure of goodness of the

327
00:23:24,950 --> 00:23:27,410
constellation.

328
00:23:27,410 --> 00:23:35,810
And K_b of A Is the average
number of nearest neighbors.

329
00:23:35,810 --> 00:23:38,070
I forget what we call it.

330
00:23:38,070 --> 00:23:43,250
What is the numerator here?

331
00:23:43,250 --> 00:23:50,430
K_min of A, which itself is an
average, over the number of

332
00:23:50,430 --> 00:23:56,430
bits that we're actually
sending, which is log2 of the

333
00:23:56,430 --> 00:23:59,690
size of A.

334
00:23:59,690 --> 00:24:03,110
So basically we only need to
know a couple of parameters of

335
00:24:03,110 --> 00:24:04,610
this signal constellation.

336
00:24:04,610 --> 00:24:08,480
Its minimum square distance
is very important.

337
00:24:08,480 --> 00:24:11,890
Energy parameter, which we
choose to make the energy per

338
00:24:11,890 --> 00:24:15,020
bit, or so that we get
this expression.

339
00:24:15,020 --> 00:24:19,600
And the average number of
nearest neighbors per bit that

340
00:24:19,600 --> 00:24:19,877
we transmit.

341
00:24:19,877 --> 00:24:21,127
OK.

342
00:24:23,920 --> 00:24:33,840
And our best example so far is
the tetrahedral constellation,

343
00:24:33,840 --> 00:24:37,760
where we basically
pick every other

344
00:24:37,760 --> 00:24:44,245
point from the 4 simplex.

345
00:24:48,160 --> 00:24:50,690
Maybe I should do that
in a different color.

346
00:24:50,690 --> 00:24:52,636
I don't think that's
a different color.

347
00:24:52,636 --> 00:24:54,750
Nope.

348
00:24:54,750 --> 00:24:55,260
Anyway.

349
00:24:55,260 --> 00:24:57,710
You know this quite
well by now.

350
00:24:57,710 --> 00:25:01,330
And if we do that, we find that
normalizing everything,

351
00:25:01,330 --> 00:25:05,730
this is 4/3 or 1.25 dB.

352
00:25:05,730 --> 00:25:10,400
And every point has three
nearest neighbors, and we're

353
00:25:10,400 --> 00:25:12,330
sending 2 bits, so
this is 3/2.

354
00:25:15,710 --> 00:25:20,020
So then we get this handy dandy
engineering rule to make

355
00:25:20,020 --> 00:25:25,000
a quick plot of the union
bound estimate.

356
00:25:25,000 --> 00:25:27,860
Given that we decided to put
this on a log-log scale to

357
00:25:27,860 --> 00:25:33,340
start with, all we have to do
to plot this expression --

358
00:25:33,340 --> 00:25:42,570
if we're given, as we always
are, the baseline curve, which

359
00:25:42,570 --> 00:25:49,010
is simply Q to the square
root of 2 Eb over N_0.

360
00:25:49,010 --> 00:25:52,150
How do you convert this
curve to this curve

361
00:25:52,150 --> 00:25:53,380
on a log-log plot?

362
00:25:53,380 --> 00:25:57,070
Well, you simply move it to the
left by the coding gain.

363
00:25:57,070 --> 00:26:03,920
And you move it up by the
log of whatever K_b is.

364
00:26:03,920 --> 00:26:08,460
So if the dominant effect is
moving it left, in this case,

365
00:26:08,460 --> 00:26:12,450
by 1.25 dB --

366
00:26:12,450 --> 00:26:15,110
I'm going to wind up getting
about this curve, so I won't

367
00:26:15,110 --> 00:26:15,960
draw it again --

368
00:26:15,960 --> 00:26:22,330
then we move it up by a factor
of 3/2, we developed a rule of

369
00:26:22,330 --> 00:26:26,480
thumb that said a factor of two
is basically going to cost

370
00:26:26,480 --> 00:26:30,230
you about 0.2 dB.

371
00:26:30,230 --> 00:26:31,770
Around ten to the minus five.

372
00:26:31,770 --> 00:26:35,390
This is just based on the slope
of this baseline curve,

373
00:26:35,390 --> 00:26:38,040
as long as we're somewhat
in that region.

374
00:26:38,040 --> 00:26:43,540
So this will cost us about 1/10
of a dB, so we'll get an

375
00:26:43,540 --> 00:26:49,250
effective coding gain
of about 1.15 dB.

376
00:26:49,250 --> 00:26:52,250
So graphically, we take
this curve bodily.

377
00:26:52,250 --> 00:26:58,410
We move it over 1.25 dB,
and we move it up

378
00:26:58,410 --> 00:27:00,420
by a factor of 3/2.

379
00:27:00,420 --> 00:27:02,890
And what we'll find is the
effective coding gain is

380
00:27:02,890 --> 00:27:07,730
thereby reduced to, I estimate,
about 1.15 dB.

381
00:27:07,730 --> 00:27:11,110
And this is all just engineering
rules of thumb.

382
00:27:11,110 --> 00:27:12,830
Nice template.

383
00:27:12,830 --> 00:27:15,570
I sometimes say you should
fill out a copy of this

384
00:27:15,570 --> 00:27:18,100
baseline curve, cut it out, put
it in your wallet for the

385
00:27:18,100 --> 00:27:20,650
duration of this course,
because you'll have the

386
00:27:20,650 --> 00:27:22,220
opportunity to make
this kind of

387
00:27:22,220 --> 00:27:25,150
calculation again and again.

388
00:27:25,150 --> 00:27:25,540
All right.

389
00:27:25,540 --> 00:27:28,780
So that's union bound SNR.

390
00:27:28,780 --> 00:27:31,150
For simple constellations,
the union bound

391
00:27:31,150 --> 00:27:34,480
estimate is very accurate.

392
00:27:34,480 --> 00:27:37,280
So this is a very good way to
proceed, from an engineering

393
00:27:37,280 --> 00:27:38,560
point of view.

394
00:27:38,560 --> 00:27:41,212
We want to write a paper?

395
00:27:41,212 --> 00:27:43,330
You know, it's a little late
to write a paper on the

396
00:27:43,330 --> 00:27:46,270
performance of the four simplex
signal constellations,

397
00:27:46,270 --> 00:27:49,430
but if you wanted to, and you
wanted to have one graph in

398
00:27:49,430 --> 00:27:51,790
that paper, you would
have the graph of --

399
00:27:51,790 --> 00:27:57,290
well, you would actually
probably compute the exact

400
00:27:57,290 --> 00:28:01,020
error of probability, either by
analysis or by Monte Carlo

401
00:28:01,020 --> 00:28:03,150
simulation, you would
put that in there.

402
00:28:03,150 --> 00:28:05,590
But in your first draft of the
paper, you would put in union

403
00:28:05,590 --> 00:28:09,100
bound estimate and you would
find that wasn't far off.

404
00:28:11,630 --> 00:28:12,680
OK.

405
00:28:12,680 --> 00:28:13,830
Any questions on that?

406
00:28:13,830 --> 00:28:19,030
This has basically got
us up to chapter six.

407
00:28:26,090 --> 00:28:35,240
So in chapter six now, which
Ashish got into last time.

408
00:28:35,240 --> 00:28:40,840
This is basically about binary
linear block codes.

409
00:28:44,510 --> 00:28:48,820
And well, we first just
start talking about

410
00:28:48,820 --> 00:28:51,330
binary block codes.

411
00:28:51,330 --> 00:28:55,750
That's what Ashish
did last time.

412
00:28:58,350 --> 00:29:03,310
We basically take 0 and 1
as our binary alphabet.

413
00:29:03,310 --> 00:29:08,000
We take a blocks of length n.

414
00:29:08,000 --> 00:29:08,170
Sorry.

415
00:29:08,170 --> 00:29:12,020
Are we using little
n or big N?

416
00:29:12,020 --> 00:29:15,170
Little n, where n is called
the block plank.

417
00:29:15,170 --> 00:29:23,760
So we take the set of all binary
symbols of length n,

418
00:29:23,760 --> 00:29:31,730
and we're going to convert
this to real n space.

419
00:29:31,730 --> 00:29:33,950
How do we convert it?

420
00:29:33,950 --> 00:29:36,770
Component-wise, by the
standard 2-PAM map.

421
00:29:36,770 --> 00:29:41,660
We map 0 into plus alpha,
1 into minus alpha.

422
00:29:41,660 --> 00:29:46,740
So this maps into plus or
minus alpha to the n.

423
00:29:46,740 --> 00:29:50,680
So the entire universe of
possible code words that we

424
00:29:50,680 --> 00:29:58,000
have is this set of 2 to the
n real n-tuples, which are

425
00:29:58,000 --> 00:30:01,340
simply the vertices of an
n cube of size 2 alpha,

426
00:30:01,340 --> 00:30:02,840
obviously, right?

427
00:30:02,840 --> 00:30:04,390
Just like this.

428
00:30:04,390 --> 00:30:10,320
We have eight possible
vertices in 3-space.

429
00:30:10,320 --> 00:30:12,300
They obviously all have
the same power.

430
00:30:12,300 --> 00:30:17,810
They all lie on the surface of
a sphere of squared radius n

431
00:30:17,810 --> 00:30:20,010
alpha squared.

432
00:30:20,010 --> 00:30:22,310
So they're not only the vertices
of a cube, they're

433
00:30:22,310 --> 00:30:26,810
vertices that are spread on
an equal energy sphere.

434
00:30:26,810 --> 00:30:32,480
And the whole idea is that our
code is going to be some

435
00:30:32,480 --> 00:30:36,870
subset of this, and the code
will map under the same map,

436
00:30:36,870 --> 00:30:43,060
which we call S, into
some subset of the

437
00:30:43,060 --> 00:30:45,610
vertices of the n-cube.

438
00:30:45,610 --> 00:30:45,990
OK?

439
00:30:45,990 --> 00:30:50,820
And again, our favorite example
is the tetrahedron.

440
00:30:50,820 --> 00:30:57,950
For instance, if we take the
code as being 0, 0, 0, 0, 1,

441
00:30:57,950 --> 00:31:06,150
1, 1, 0, 1 1, 1, 0, those for
binary three-tuples, this maps

442
00:31:06,150 --> 00:31:10,310
into T, the tetrahedron.

443
00:31:10,310 --> 00:31:11,656
OK?

444
00:31:11,656 --> 00:31:15,230
Well, we get this signal
structure, which we've already

445
00:31:15,230 --> 00:31:16,230
found the coding gain.

446
00:31:16,230 --> 00:31:18,180
It has a little bit
of coding gain.

447
00:31:18,180 --> 00:31:22,090
We actually accomplished
something.

448
00:31:22,090 --> 00:31:24,160
All right.

449
00:31:24,160 --> 00:31:30,780
So that's the basic idea that
we go through in 6.1.

450
00:31:30,780 --> 00:31:33,230
That's our favorite example.

451
00:31:33,230 --> 00:31:37,440
And Ashish also shows you that,
you know, you might

452
00:31:37,440 --> 00:31:41,040
think this is an awfully
restricted way of designing

453
00:31:41,040 --> 00:31:43,260
constellations.

454
00:31:43,260 --> 00:31:47,120
But when we're talking about low
spectral efficiencies, by

455
00:31:47,120 --> 00:31:50,960
going through the capacity
calculation, we can assure

456
00:31:50,960 --> 00:31:55,310
ourselves that in principle,
this is not going to be very

457
00:31:55,310 --> 00:31:57,070
sub-optimal for it.

458
00:31:57,070 --> 00:32:00,930
Capacity is about the regime
where n gets very large for

459
00:32:00,930 --> 00:32:02,970
very long block codes.

460
00:32:02,970 --> 00:32:06,500
We don't lose much in potential
coding gain, or in

461
00:32:06,500 --> 00:32:10,975
the Shannon limit, equivalently,
by restricting

462
00:32:10,975 --> 00:32:15,170
the input output input alphabet
to be just these two

463
00:32:15,170 --> 00:32:18,230
numbers, plus or minus alpha,
I should say, rather than

464
00:32:18,230 --> 00:32:24,310
using the whole real line as
an input, as long as the

465
00:32:24,310 --> 00:32:35,100
nominal spectral efficiency is
less than or equal to one bit

466
00:32:35,100 --> 00:32:37,840
per two dimensions.

467
00:32:37,840 --> 00:32:41,180
And the exact numbers --

468
00:32:41,180 --> 00:32:47,780
at rho equals 1, you lose about
0.2 dB, in principle,

469
00:32:47,780 --> 00:32:51,720
using the Shannon limit
as your guide.

470
00:32:51,720 --> 00:32:56,290
And for lower rho, it
just goes to 0.

471
00:32:56,290 --> 00:32:59,260
And you did this last time.

472
00:32:59,260 --> 00:32:59,650
OK.

473
00:32:59,650 --> 00:33:04,610
So it's a very reasonable, and
obviously attractive from an

474
00:33:04,610 --> 00:33:08,170
implementation point of view,
way of designing signal

475
00:33:08,170 --> 00:33:09,790
constellations.

476
00:33:09,790 --> 00:33:12,060
And again, this is basically all
we're going to be talking

477
00:33:12,060 --> 00:33:15,060
about for the majority of
the course, is signal

478
00:33:15,060 --> 00:33:16,440
constellations like this.

479
00:33:16,440 --> 00:33:19,530
But of course as we get to codes
that are thousands of

480
00:33:19,530 --> 00:33:22,960
bits long, or perhaps even
infinitely long, as in the

481
00:33:22,960 --> 00:33:28,820
case of convolutional codes and
trellis codes, sometimes

482
00:33:28,820 --> 00:33:31,390
you forget that we're really
talking about just putting

483
00:33:31,390 --> 00:33:35,830
points in Euclidian
signal space.

484
00:33:35,830 --> 00:33:39,500
Because we're going to be very
abstracted back into the

485
00:33:39,500 --> 00:33:42,320
binary domain.

486
00:33:42,320 --> 00:33:46,680
But in this course, we're always
thinking of codes as

487
00:33:46,680 --> 00:33:49,103
means of designing signal
constellations.

488
00:33:53,630 --> 00:33:57,940
Of course, codes are used far
more widely than just for

489
00:33:57,940 --> 00:34:01,370
signaling over the additive
white Gaussian noise channel.

490
00:34:01,370 --> 00:34:08,540
I've sort of packaged this
course as a search to get to

491
00:34:08,540 --> 00:34:10,070
capacity, we have added
the additive white

492
00:34:10,070 --> 00:34:11,150
Gaussian noise channel.

493
00:34:11,150 --> 00:34:13,580
First of all because
this corresponds

494
00:34:13,580 --> 00:34:16,449
very closely to history.

495
00:34:16,449 --> 00:34:19,830
Within this package, we can
talk about all of the

496
00:34:19,830 --> 00:34:22,380
principal classes of codes that
have been developed to

497
00:34:22,380 --> 00:34:26,355
date, and we can compare them
in some performance terms.

498
00:34:29,090 --> 00:34:31,860
How close do they get to
capacity on the additive white

499
00:34:31,860 --> 00:34:33,330
Gaussian noise channel.

500
00:34:33,330 --> 00:34:38,100
So you get most of what you
would get if you took a

501
00:34:38,100 --> 00:34:42,449
communications-free view
of coding theory.

502
00:34:42,449 --> 00:34:44,370
You know, people are interested
in codes for

503
00:34:44,370 --> 00:34:50,020
computer memories, for
God knows what,

504
00:34:50,020 --> 00:34:51,190
lots of other things.

505
00:34:51,190 --> 00:34:57,500
And for many of these other
applications, you are going to

506
00:34:57,500 --> 00:34:59,320
be interested in the same
class of codes.

507
00:34:59,320 --> 00:35:03,120
Basically, you're going to want
to maximize the distance

508
00:35:03,120 --> 00:35:07,030
between code words for a given
rate, which has to do with the

509
00:35:07,030 --> 00:35:09,300
size of the code.

510
00:35:09,300 --> 00:35:14,270
And so you're going to be
interested in the same codes.

511
00:35:14,270 --> 00:35:16,930
Putting it in the framework of
getting to capacity on the

512
00:35:16,930 --> 00:35:21,060
additive white Gaussian noise
channel gives a motivation,

513
00:35:21,060 --> 00:35:24,060
gives a very nice story, because
over 50 years, we were

514
00:35:24,060 --> 00:35:28,600
able to get to capacity,
and gives it a real

515
00:35:28,600 --> 00:35:30,240
communications flavor.

516
00:35:30,240 --> 00:35:33,360
Some of you, I'm sure, are here
without any interest in

517
00:35:33,360 --> 00:35:34,560
communications whatsoever.

518
00:35:34,560 --> 00:35:36,360
You simply want to know
about coding.

519
00:35:36,360 --> 00:35:38,830
You will still get that story,
but you'll get it

520
00:35:38,830 --> 00:35:41,020
in this nice package.

521
00:35:41,020 --> 00:35:44,840
That's why I do it this way.

522
00:35:44,840 --> 00:35:45,820
OK.

523
00:35:45,820 --> 00:35:47,490
I'm about to get
into new stuff.

524
00:35:47,490 --> 00:35:49,660
End of review.

525
00:35:49,660 --> 00:35:52,820
Any questions?

526
00:35:52,820 --> 00:35:56,000
So you must have done
a great job.

527
00:35:56,000 --> 00:35:59,120
Everyone understands
perfectly.

528
00:35:59,120 --> 00:35:59,740
All right.

529
00:35:59,740 --> 00:36:03,350
So now let's talk about,
as I said, binary

530
00:36:03,350 --> 00:36:06,060
linear block codes.

531
00:36:06,060 --> 00:36:11,220
When you see the word "linear,"
that's a signal that

532
00:36:11,220 --> 00:36:12,680
there's some algebra ahead.

533
00:36:12,680 --> 00:36:18,250
And so this is the very first
point at which we get into

534
00:36:18,250 --> 00:36:22,170
what's called algebraic
coding theory.

535
00:36:22,170 --> 00:36:24,350
The algebra will be extremely
simple at this

536
00:36:24,350 --> 00:36:25,660
point, not to worry.

537
00:36:28,250 --> 00:36:32,730
And what are we doing here?

538
00:36:36,520 --> 00:36:40,740
First thing we do is to identify
0 and 1 with the

539
00:36:40,740 --> 00:36:45,810
binary field, which I'm always
going to write as F2.

540
00:36:45,810 --> 00:36:51,260
The older way of writing this
is GF(2), for Galois field

541
00:36:51,260 --> 00:36:52,520
with two elements.

542
00:36:52,520 --> 00:36:54,850
They mean exactly
the same thing.

543
00:36:54,850 --> 00:36:59,430
Nowadays most people
write just F2.

544
00:36:59,430 --> 00:37:03,570
Now we have [UNINTELLIGIBLE],
we can write this blackboard

545
00:37:03,570 --> 00:37:06,320
F.

546
00:37:06,320 --> 00:37:09,230
And OK.

547
00:37:09,230 --> 00:37:09,960
Step one.

548
00:37:09,960 --> 00:37:12,540
We've identified our alphabet
with a field.

549
00:37:16,840 --> 00:37:20,450
So algebraically,
this is a field.

550
00:37:20,450 --> 00:37:23,380
Some of you know exactly
what I mean by that.

551
00:37:23,380 --> 00:37:24,540
Others don't.

552
00:37:24,540 --> 00:37:26,060
We'll come back to this again.

553
00:37:26,060 --> 00:37:29,380
Informally, I would say a field
is simply something

554
00:37:29,380 --> 00:37:32,970
where you can add, subtract,
multiply, or divide.

555
00:37:32,970 --> 00:37:36,825
Our best examples of that before
now have been the real

556
00:37:36,825 --> 00:37:39,710
and complex fields.

557
00:37:39,710 --> 00:37:42,560
There is a more formal
definition of that.

558
00:37:42,560 --> 00:37:45,740
In this case, we have
only two elements.

559
00:37:45,740 --> 00:37:52,840
So let me just write down the
tables, which you all know,

560
00:37:52,840 --> 00:37:55,500
regardless of your background.

561
00:37:55,500 --> 00:37:57,690
What's the addition table
for this field?

562
00:37:57,690 --> 00:38:01,610
Well, 0 is the additive
identity.

563
00:38:01,610 --> 00:38:09,100
So we know that 0 added to
anything gives itself, so that

564
00:38:09,100 --> 00:38:11,615
gives us three of the entries
of this table.

565
00:38:14,540 --> 00:38:16,550
What do we put down here?

566
00:38:16,550 --> 00:38:20,970
There's really only one choice
to satisfy one of the axioms

567
00:38:20,970 --> 00:38:27,260
of the field, which is that
under addition, the field must

568
00:38:27,260 --> 00:38:29,230
form a group.

569
00:38:29,230 --> 00:38:32,140
In particular, that means that
all elements are invertible.

570
00:38:32,140 --> 00:38:36,320
That means that each row or
column has to be a permutation

571
00:38:36,320 --> 00:38:37,540
of the group elements.

572
00:38:37,540 --> 00:38:40,980
And so the only possibility
here is to put in a 0.

573
00:38:40,980 --> 00:38:46,270
And well, we are forced, if
we're going to make 0 and 1

574
00:38:46,270 --> 00:38:50,600
into a field to find an addition
table which is the

575
00:38:50,600 --> 00:38:52,790
table of mod2 addition.

576
00:38:52,790 --> 00:38:55,690
You've had this stated
as an axiom before.

577
00:38:55,690 --> 00:39:00,640
You can derive it just from the
fact that 0 needs to be

578
00:39:00,640 --> 00:39:05,000
the additive identity, and then
we need to put a 0 in

579
00:39:05,000 --> 00:39:06,885
here in order to get
invertibility.

580
00:39:12,490 --> 00:39:14,120
Under multiplication.

581
00:39:14,120 --> 00:39:16,340
What is the multiplication
table?

582
00:39:16,340 --> 00:39:21,180
Well, the additive identity
is also a nullifier under

583
00:39:21,180 --> 00:39:22,820
multiplication in any field.

584
00:39:22,820 --> 00:39:26,150
So 0 times anything
is equal to 0.

585
00:39:26,150 --> 00:39:29,410
1 times anything is
equal to itself.

586
00:39:29,410 --> 00:39:34,000
So that completely determines
the multiplication table.

587
00:39:34,000 --> 00:39:37,870
Again, just the table of
mod2 multiplication.

588
00:39:37,870 --> 00:39:41,800
So if I'd done this
axiomatically, I would have

589
00:39:41,800 --> 00:39:43,410
given you the axioms of field.

590
00:39:43,410 --> 00:39:47,410
Then I would show that under
this binary addition

591
00:39:47,410 --> 00:39:50,800
operation, this multiplication
operation,

592
00:39:50,800 --> 00:39:52,780
we satisfy the axioms.

593
00:39:52,780 --> 00:39:54,900
I'll do that as we get
into chapter seven.

594
00:39:57,630 --> 00:39:58,880
OK.

595
00:40:00,550 --> 00:40:12,300
So now we have 0 1 to the n,
n-tuples, binary n-tuples, we

596
00:40:12,300 --> 00:40:17,570
will now regard as n-tuples
of field elements, F2.

597
00:40:20,890 --> 00:40:29,648
And these will be regarded as
vectors in a vector space.

598
00:40:29,648 --> 00:40:35,330
Well, I'll just say that F2 to
the n is a vector space, which

599
00:40:35,330 --> 00:40:37,990
clearly has 2 to
the n elements.

600
00:40:37,990 --> 00:40:39,290
Now again, informally.

601
00:40:39,290 --> 00:40:41,320
What's a vector space?

602
00:40:41,320 --> 00:40:46,450
A vector space is always
over a given field.

603
00:40:46,450 --> 00:40:47,890
In this case, it's going
to be over the

604
00:40:47,890 --> 00:40:49,150
binary field, of course.

605
00:40:49,150 --> 00:40:51,740
F2.

606
00:40:51,740 --> 00:40:54,720
The given field is
called a scalar.

607
00:40:54,720 --> 00:40:57,670
Just like vector space over
the reals, the scalars are

608
00:40:57,670 --> 00:40:58,760
real numbers.

609
00:40:58,760 --> 00:41:02,910
Here the scalars are
elements of F2.

610
00:41:02,910 --> 00:41:07,450
And what do we have to have to
make something algebraically a

611
00:41:07,450 --> 00:41:08,400
vector space?

612
00:41:08,400 --> 00:41:12,930
We have to have that the
addition of two vectors is

613
00:41:12,930 --> 00:41:19,310
well-defined and gives another
vector, and that the

614
00:41:19,310 --> 00:41:23,420
multiplication of a vector by
a scalar is well-defined and

615
00:41:23,420 --> 00:41:27,120
gives another vector in
our vector space.

616
00:41:27,120 --> 00:41:28,370
All right?

617
00:41:30,830 --> 00:41:31,500
OK.

618
00:41:31,500 --> 00:41:35,130
So how are we going to define
that to addition?

619
00:41:35,130 --> 00:41:38,600
If we want to add
two n-tuples --

620
00:41:38,600 --> 00:41:40,940
again, you aren't going to see
anything here that you haven't

621
00:41:40,940 --> 00:41:43,400
already seen in some
other context.

622
00:41:43,400 --> 00:41:43,860
What do we do?

623
00:41:43,860 --> 00:41:49,030
We add them component-wise,
using the component-wise rules

624
00:41:49,030 --> 00:41:51,630
of field addition.

625
00:41:51,630 --> 00:41:51,730
OK.

626
00:41:51,730 --> 00:41:55,380
So we just add two
vector n-tuples,

627
00:41:55,380 --> 00:41:56,910
component by component.

628
00:41:56,910 --> 00:42:00,570
We obviously get some result,
which is itself a binary

629
00:42:00,570 --> 00:42:03,290
vector or an F2 vector.

630
00:42:03,290 --> 00:42:05,220
No problem there.

631
00:42:05,220 --> 00:42:08,050
Except that --

632
00:42:08,050 --> 00:42:09,620
well, all right.

633
00:42:09,620 --> 00:42:11,980
As long as we're talking about
all possible n-tuples, the

634
00:42:11,980 --> 00:42:14,160
result is certainly in the
vector space, right?

635
00:42:14,160 --> 00:42:17,530
We add two binary n-tuples,
we get a binary n-tuple.

636
00:42:17,530 --> 00:42:20,320
So the result is
in F2 to the n.

637
00:42:20,320 --> 00:42:24,450
This is going to be the key test
when we get to subspaces

638
00:42:24,450 --> 00:42:27,990
of F2 to the n.

639
00:42:27,990 --> 00:42:28,035
OK.

640
00:42:28,035 --> 00:42:31,470
And multiplication
is even easier.

641
00:42:31,470 --> 00:42:34,100
How do we define multiplication
by scalars?

642
00:42:34,100 --> 00:42:35,560
Well, we only have
two scalars --

643
00:42:35,560 --> 00:42:37,570
0 and 1.

644
00:42:37,570 --> 00:42:45,530
So 0 times any vector is going
to equal the all 0 vector.

645
00:42:45,530 --> 00:42:47,770
Again, you can view that
as just component-wise

646
00:42:47,770 --> 00:42:51,480
multiplication of everything
by 0, and since 0 times

647
00:42:51,480 --> 00:42:53,270
anything equals 0, we're
always going

648
00:42:53,270 --> 00:42:55,400
to get the 0 vector.

649
00:42:55,400 --> 00:42:57,150
All right?

650
00:42:57,150 --> 00:43:02,210
So is the 0 vector in
the vector space?

651
00:43:02,210 --> 00:43:02,800
Well, yes.

652
00:43:02,800 --> 00:43:05,130
If we're talking about several
n-tuples, it of

653
00:43:05,130 --> 00:43:06,770
course always is.

654
00:43:06,770 --> 00:43:12,610
And one times anything, again,
we can just do this

655
00:43:12,610 --> 00:43:14,970
component-wise, and it
just gives itself.

656
00:43:17,480 --> 00:43:18,560
Trivially.

657
00:43:18,560 --> 00:43:21,950
So since this was a vector in
the vector space, this is

658
00:43:21,950 --> 00:43:25,660
certainly a vector in
the vector space.

659
00:43:25,660 --> 00:43:26,080
OK.

660
00:43:26,080 --> 00:43:29,745
This seems pretty
trivial so far.

661
00:43:29,745 --> 00:43:35,750
But what's a binary
linear block code?

662
00:43:35,750 --> 00:43:40,620
Again, focusing on the
linear, is a --

663
00:43:40,620 --> 00:43:43,400
now I'll give a formal
definition.

664
00:43:43,400 --> 00:43:51,660
it's a subspace of F2
to the n for some n

665
00:43:51,660 --> 00:43:52,910
called the block length.

666
00:43:55,430 --> 00:43:56,630
All right.

667
00:43:56,630 --> 00:43:58,700
What do I mean when
I say a subspace?

668
00:43:58,700 --> 00:44:03,080
I mean subsets of the elements
of a vector space that itself

669
00:44:03,080 --> 00:44:05,010
forms a vector space.

670
00:44:05,010 --> 00:44:11,110
So in this case when I
say subset, I mean a

671
00:44:11,110 --> 00:44:14,270
set of binary n-tuples.

672
00:44:14,270 --> 00:44:16,280
OK?

673
00:44:16,280 --> 00:44:18,460
That itself forms
a vector space.

674
00:44:18,460 --> 00:44:18,810
OK.

675
00:44:18,810 --> 00:44:20,570
What are the components
of that?

676
00:44:20,570 --> 00:44:24,540
To check that it forms a vector
space, let's see.

677
00:44:24,540 --> 00:44:30,032
Multiplication by scalars is,
again, easy to check.

678
00:44:30,032 --> 00:44:34,110
If it's going to be a subspace,
then the all 0 --

679
00:44:34,110 --> 00:44:48,140
so it has to contain the all 0
vector in order that when I

680
00:44:48,140 --> 00:44:50,860
multiply by the scalar
0, I get another

681
00:44:50,860 --> 00:44:52,110
element of this subspace.

682
00:44:55,260 --> 00:44:59,070
Multiplication by 1 is always
trivially satisfied.

683
00:44:59,070 --> 00:45:01,830
If I start off with a set, I
multiply by 1, I'm going to

684
00:45:01,830 --> 00:45:04,960
get the same set.

685
00:45:04,960 --> 00:45:14,120
So let's check whether the
elements of a subspace are

686
00:45:14,120 --> 00:45:17,760
closed under vector addition.

687
00:45:17,760 --> 00:45:19,140
What do I mean by that?

688
00:45:19,140 --> 00:45:24,270
I mean if you add two elements
of the of the subspace

689
00:45:24,270 --> 00:45:28,440
together, you get another
element of the subspace.

690
00:45:28,440 --> 00:45:32,255
That's the key property
to check.

691
00:45:36,650 --> 00:45:40,670
Key property -- we can
write that as closure

692
00:45:40,670 --> 00:45:42,995
under vector addition.

693
00:45:47,900 --> 00:45:52,280
Which is also called
the group property.

694
00:45:52,280 --> 00:45:56,460
It means that just under
addition, under vector

695
00:45:56,460 --> 00:46:00,090
addition, the set of elements
that you have forms a group.

696
00:46:00,090 --> 00:46:01,850
You add any two elements,
you get another

697
00:46:01,850 --> 00:46:03,100
element of the subset.

698
00:46:06,350 --> 00:46:12,030
The example is our favorite
example so far.

699
00:46:12,030 --> 00:46:18,420
Let's take this little code,
and I'm going to ask.

700
00:46:18,420 --> 00:46:24,150
Is that a subspace of
F2 to the three?

701
00:46:24,150 --> 00:46:35,580
So does this equal subspace
of the set of all binary

702
00:46:35,580 --> 00:46:36,830
three-tuples?

703
00:46:40,620 --> 00:46:46,880
Anyone care to hazard a guess
whether it is or isn't?

704
00:46:46,880 --> 00:46:47,810
AUDIENCE: It is.

705
00:46:47,810 --> 00:46:48,790
PROFESSOR: It is?

706
00:46:48,790 --> 00:46:50,040
Why?

707
00:46:53,012 --> 00:46:54,380
AUDIENCE: It has the
all 0 vector.

708
00:46:54,380 --> 00:46:58,200
PROFESSOR: It has the all
0 vector, first of all.

709
00:46:58,200 --> 00:47:00,480
Good.

710
00:47:00,480 --> 00:47:01,730
AUDIENCE: [INAUDIBLE]

711
00:47:04,170 --> 00:47:07,588
PROFESSOR: And it's closed
under addition.

712
00:47:07,588 --> 00:47:11,470
Now how might we see that?

713
00:47:11,470 --> 00:47:14,820
You could, of course, just
take all pair-wise --

714
00:47:14,820 --> 00:47:17,400
you could form the addition
table of these four elements,

715
00:47:17,400 --> 00:47:20,130
and you would find that you
always get another one of

716
00:47:20,130 --> 00:47:20,690
these elements.

717
00:47:20,690 --> 00:47:26,240
For instance, 0, 1, 1 plus 1,
0, 1, is equal to 1, 1, 0.

718
00:47:26,240 --> 00:47:31,150
In fact, you easily see that you
take any two of these, add

719
00:47:31,150 --> 00:47:33,155
them together, you get
the third one.

720
00:47:33,155 --> 00:47:37,840
If you call this a, b, and c,
a plus b plus c equals 0,

721
00:47:37,840 --> 00:47:40,460
addition is the same as
subtraction, because we're in

722
00:47:40,460 --> 00:47:42,460
a binary field.

723
00:47:42,460 --> 00:47:46,620
So that means that a plus b
equals c, a equals b plus c, c

724
00:47:46,620 --> 00:47:50,820
equals b plus a, whatever
you like.

725
00:47:50,820 --> 00:47:55,000
And of course, if you add 0 to
anything, it's trivially

726
00:47:55,000 --> 00:47:56,250
closed under that.

727
00:47:59,090 --> 00:47:59,580
All right.

728
00:47:59,580 --> 00:48:01,060
So it is.

729
00:48:01,060 --> 00:48:02,370
It satisfies --

730
00:48:02,370 --> 00:48:04,320
it's all you've got to check.

731
00:48:09,415 --> 00:48:13,430
A more abstract proof of this
would be, this is the set of

732
00:48:13,430 --> 00:48:16,360
all even-weight three-tuples.

733
00:48:16,360 --> 00:48:19,370
If I add even to even, I'm
going to get even.

734
00:48:19,370 --> 00:48:21,440
So of course my result is
going to be another

735
00:48:21,440 --> 00:48:24,540
even-weight three-tuple,
therefore in the set.

736
00:48:24,540 --> 00:48:26,130
That's the more algebraic
proof.

737
00:48:29,560 --> 00:48:31,760
All right.

738
00:48:31,760 --> 00:48:36,140
Suppose I just add 0, 0, 1
to all of these things.

739
00:48:36,140 --> 00:48:38,470
I'll get the set of all
odd-weight n-tuples.

740
00:48:41,520 --> 00:48:43,680
Add any odd-weight
n-tuple to this.

741
00:48:43,680 --> 00:48:46,800
Let me take the C prime,
which is the set of

742
00:48:46,800 --> 00:48:48,050
all odd-weight n-tuples.

743
00:48:50,140 --> 00:48:52,195
Is that a vector space?

744
00:48:54,950 --> 00:48:57,360
It doesn't have 0, and in fact,
it's not even closed

745
00:48:57,360 --> 00:48:59,920
under vector addition.

746
00:48:59,920 --> 00:49:00,230
All right.

747
00:49:00,230 --> 00:49:06,060
If I take in C double prime is
equal to 0, 0, 0 0, 1, 1, 1,

748
00:49:06,060 --> 00:49:10,135
0, 1 and I stop there,
is that a subspace?

749
00:49:13,120 --> 00:49:13,390
No.

750
00:49:13,390 --> 00:49:15,850
Because?

751
00:49:15,850 --> 00:49:17,530
Not closed.

752
00:49:17,530 --> 00:49:21,000
For instance, if I add these
two together, I would get

753
00:49:21,000 --> 00:49:23,670
that, and that's missing.

754
00:49:23,670 --> 00:49:24,040
OK.

755
00:49:24,040 --> 00:49:31,080
So actually, everything is
much simpler when we're

756
00:49:31,080 --> 00:49:32,640
talking about finite fields.

757
00:49:32,640 --> 00:49:35,820
All the finite dimensional
vector spaces consist of a

758
00:49:35,820 --> 00:49:38,140
finite number of elements.

759
00:49:38,140 --> 00:49:42,310
It's easier than real and
complex vector spaces.

760
00:49:42,310 --> 00:49:46,856
There's no analysis involved.

761
00:49:46,856 --> 00:49:47,430
So forth.

762
00:49:47,430 --> 00:49:47,960
All right.

763
00:49:47,960 --> 00:49:52,960
So a binary linear block
code is in a subspace

764
00:49:52,960 --> 00:49:55,630
of F2 to the n.

765
00:49:55,630 --> 00:49:59,430
What are some of the key
algebraic facts we know about

766
00:49:59,430 --> 00:50:06,460
vector spaces from our study
of linear algebra, which I

767
00:50:06,460 --> 00:50:11,380
assume all of you have had
in some form or another?

768
00:50:11,380 --> 00:50:14,210
What's a key algebraic property
of a vector space?

769
00:50:14,210 --> 00:50:18,250
What's the very first
property?

770
00:50:18,250 --> 00:50:20,894
All vector spaces have a --

771
00:50:20,894 --> 00:50:23,820
AUDIENCE: [INAUDIBLE]

772
00:50:23,820 --> 00:50:25,080
PROFESSOR: Norm?

773
00:50:25,080 --> 00:50:26,410
No.

774
00:50:26,410 --> 00:50:27,250
Dimension.

775
00:50:27,250 --> 00:50:30,710
Dimension, that's
where I'm going.

776
00:50:30,710 --> 00:50:33,484
What's the significance
of dimension?

777
00:50:33,484 --> 00:50:34,734
AUDIENCE: [INAUDIBLE]

778
00:50:36,860 --> 00:50:40,410
PROFESSOR: The definition of
dimension is the number of

779
00:50:40,410 --> 00:50:42,750
generators in any basis.

780
00:50:42,750 --> 00:50:45,010
So we're talking about
generators of the vector

781
00:50:45,010 --> 00:50:49,840
space, or a set of generators
which form a basis.

782
00:50:49,840 --> 00:50:53,600
If you think we have the same
properties here in the case of

783
00:50:53,600 --> 00:51:01,310
vector spaces over finite
fields, well, we probably do.

784
00:51:01,310 --> 00:51:02,970
But let's see how
it works out.

785
00:51:02,970 --> 00:51:05,620
So we're talking about
things like

786
00:51:05,620 --> 00:51:10,930
generators, basis, dimension.

787
00:51:10,930 --> 00:51:16,840
These are all closely
interlinked properties.

788
00:51:16,840 --> 00:51:20,660
Again, the fact that everything
is finite here

789
00:51:20,660 --> 00:51:24,840
gives us very elementary
ways of addressing

790
00:51:24,840 --> 00:51:27,510
all of these concepts.

791
00:51:27,510 --> 00:51:29,300
We don't have any
geometry yet.

792
00:51:29,300 --> 00:51:30,290
We don't have norms.

793
00:51:30,290 --> 00:51:31,310
We don't have distances.

794
00:51:31,310 --> 00:51:32,260
We don't have angles.

795
00:51:32,260 --> 00:51:34,300
We'll talk about that
in a minute.

796
00:51:34,300 --> 00:51:38,940
Here we just have a set that
basically has these two

797
00:51:38,940 --> 00:51:39,530
properties.

798
00:51:39,530 --> 00:51:41,890
It contains the all-0 vector,
and it's closed

799
00:51:41,890 --> 00:51:42,745
as the group property.

800
00:51:42,745 --> 00:51:46,280
It's closed under
vector addition.

801
00:51:46,280 --> 00:51:46,860
All right.

802
00:51:46,860 --> 00:51:56,530
So the first property is that
the all-0 vector is always in

803
00:51:56,530 --> 00:52:02,080
the subspace, which I'll
represent by C, meaning code.

804
00:52:02,080 --> 00:52:06,340
So when I talk about a code
now, I'm talking about a

805
00:52:06,340 --> 00:52:09,410
binary linear block code,
which by definition is a

806
00:52:09,410 --> 00:52:13,360
vector space, a subspace
of F2 to the n, where

807
00:52:13,360 --> 00:52:16,050
n is the code length.

808
00:52:16,050 --> 00:52:16,430
All right.

809
00:52:16,430 --> 00:52:19,700
Let me try to find a set of
generators for the code.

810
00:52:22,320 --> 00:52:22,546
All right?

811
00:52:22,546 --> 00:52:28,040
If somebody gives me a code,
they say, this is a binary

812
00:52:28,040 --> 00:52:29,930
linear block code.

813
00:52:29,930 --> 00:52:33,530
Let me see if I can find a
set of generators for it.

814
00:52:33,530 --> 00:52:37,180
If I find three generators,
then I'll know the

815
00:52:37,180 --> 00:52:39,280
dimension is three.

816
00:52:39,280 --> 00:52:40,700
That's basically where
I'm going.

817
00:52:40,700 --> 00:52:44,440
I'll state this a little bit
more formally as we go ahead.

818
00:52:44,440 --> 00:52:46,090
So suppose I'm given a code.

819
00:52:46,090 --> 00:52:49,710
I know it's a binary linear
block code, so I know it has

820
00:52:49,710 --> 00:52:52,970
the all-0 element.

821
00:52:52,970 --> 00:52:56,500
So how might I go about finding
a set of generators?

822
00:52:59,410 --> 00:53:01,170
Let's just take a greedy
algorithm.

823
00:53:01,170 --> 00:53:02,420
All right?

824
00:53:05,320 --> 00:53:08,060
Suppose the code contains
only the all-0 vector.

825
00:53:08,060 --> 00:53:09,575
Is that a subspace?

826
00:53:12,250 --> 00:53:15,410
What's its dimension?

827
00:53:15,410 --> 00:53:15,540
0.

828
00:53:15,540 --> 00:53:16,790
All right?

829
00:53:18,900 --> 00:53:22,260
So that's a very trivial vector
space, but it's a

830
00:53:22,260 --> 00:53:23,210
vector space.

831
00:53:23,210 --> 00:53:27,120
Satisfies the axioms.

832
00:53:27,120 --> 00:53:29,890
Suppose it is not the
trivial code.

833
00:53:29,890 --> 00:53:32,560
That means it has more than
the all-0 vector.

834
00:53:32,560 --> 00:53:38,330
So I take as my first generator
any non-zero vector.

835
00:53:42,940 --> 00:53:43,410
OK?

836
00:53:43,410 --> 00:53:46,940
I can always do that.

837
00:53:46,940 --> 00:53:49,510
Don't have the axiom of choice
involved here, because

838
00:53:49,510 --> 00:53:51,390
everything is finite.

839
00:53:51,390 --> 00:53:54,930
So I'm going to take g1 to
be any non-zero vector.

840
00:53:54,930 --> 00:53:57,020
Now I've got a generator.

841
00:53:57,020 --> 00:54:06,295
And how many code words
does it generate?

842
00:54:10,510 --> 00:54:14,150
I want to take the set of all
binary linear combinations of

843
00:54:14,150 --> 00:54:16,980
all the generators that
I have so far.

844
00:54:16,980 --> 00:54:19,990
At this point, the binary linear
combinations are 0

845
00:54:19,990 --> 00:54:25,400
times g1 and 1 times g1 And that
just gives me this word

846
00:54:25,400 --> 00:54:26,650
and this word.

847
00:54:26,650 --> 00:54:29,620
So now I have counted for two
words with one generator.

848
00:54:32,480 --> 00:54:34,040
Could it be that that's
the whole code?

849
00:54:36,990 --> 00:54:39,100
Sure.

850
00:54:39,100 --> 00:54:43,460
The all-0 vector and any
non-zero vector together form

851
00:54:43,460 --> 00:54:46,225
a one-dimensional subspace.

852
00:54:49,670 --> 00:54:51,050
That's all you can get
from one dimension.

853
00:54:51,050 --> 00:54:53,420
So I could be finished
here now.

854
00:54:53,420 --> 00:54:55,870
But if I'm not finished, there's
still more code words

855
00:54:55,870 --> 00:54:57,670
that I haven't accounted for.

856
00:54:57,670 --> 00:55:01,580
Then I greedily pick
a second generator.

857
00:55:01,580 --> 00:55:15,180
So this is now, let's say, any
vector not generated by g1.

858
00:55:20,740 --> 00:55:22,700
So I have a branch here.

859
00:55:22,700 --> 00:55:27,860
Either I've finished or I can
pick another generator g2.

860
00:55:27,860 --> 00:55:33,580
Now with g1 and g-two, how many
vectors can I generate?

861
00:55:33,580 --> 00:55:36,930
Let me take all binary
linear combinations.

862
00:55:41,350 --> 00:55:47,600
So a binary linear combination
is any vector of the form

863
00:55:47,600 --> 00:55:54,865
alpha1 g1 plus alpha2 g2 where
these are both scalars.

864
00:55:57,470 --> 00:56:02,830
And therefore this can be 0 or
1, this could be 0 or 1.

865
00:56:02,830 --> 00:56:03,880
So what have I got now?

866
00:56:03,880 --> 00:56:09,830
I've got 0, g1 g2 and g1 plus g2
I've got four binary linear

867
00:56:09,830 --> 00:56:11,380
combinations of two
generators.

868
00:56:15,340 --> 00:56:17,520
Could that be the whole code?

869
00:56:17,520 --> 00:56:19,040
Certainly.

870
00:56:19,040 --> 00:56:23,613
At this point, again, consider
our standing example.

871
00:56:26,730 --> 00:56:29,430
I start out what I take
as my first generator.

872
00:56:29,430 --> 00:56:38,480
Let me take g1 equal 1 0, 1, g2
equals whatever, 0, 1, 1.

873
00:56:38,480 --> 00:56:42,180
Then if I take all binary linear
combinations of these

874
00:56:42,180 --> 00:56:45,030
two generators, I'm going to
get the whole code, right?

875
00:56:45,030 --> 00:56:46,690
These four code words.

876
00:56:46,690 --> 00:56:48,733
They can all be expressed
in this form.

877
00:56:51,410 --> 00:56:56,060
Or I'm not done, and then
I have to pick g3.

878
00:56:56,060 --> 00:56:59,940
And how many binary linear
combinations are there of g1,

879
00:56:59,940 --> 00:57:01,190
g2, and g3?

880
00:57:03,210 --> 00:57:04,400
Eight.

881
00:57:04,400 --> 00:57:06,235
Are they all necessarily
in my subspace?

882
00:57:09,150 --> 00:57:14,300
Yes, by the fact that the
subspace is closed under

883
00:57:14,300 --> 00:57:16,040
scalar multiplication.

884
00:57:16,040 --> 00:57:17,830
Alpha g1.

885
00:57:17,830 --> 00:57:22,860
Alpha1 g1, alpha2 g2 alpha3 g3
are all in the subspace.

886
00:57:22,860 --> 00:57:29,450
Any vector addition of any of
these scalar multiples is in

887
00:57:29,450 --> 00:57:30,280
the subspace.

888
00:57:30,280 --> 00:57:36,010
So I get now eight possible
elements of the subspace, and

889
00:57:36,010 --> 00:57:37,285
I either may be done or not.

890
00:57:40,820 --> 00:57:48,760
Continuing in this way, I get
some number gk of generators,

891
00:57:48,760 --> 00:57:53,410
just by picking greedily the
next one until I'm done.

892
00:57:53,410 --> 00:57:55,140
All right?

893
00:57:55,140 --> 00:57:57,500
When I'm done --

894
00:57:57,500 --> 00:57:58,510
so I have to stop.

895
00:57:58,510 --> 00:57:59,760
Why do I have to stop?

896
00:58:02,990 --> 00:58:03,370
All right.

897
00:58:03,370 --> 00:58:04,660
Let's look at the size.

898
00:58:04,660 --> 00:58:07,750
At each point here, this
accounts for two code words,

899
00:58:07,750 --> 00:58:11,360
this for four, this for eight,
this for 2 to the k.

900
00:58:11,360 --> 00:58:15,870
How many binary n-tuples
are there?

901
00:58:15,870 --> 00:58:16,416
2 to the n.

902
00:58:16,416 --> 00:58:17,890
All right?

903
00:58:17,890 --> 00:58:21,300
So I clearly can't find more
than n generators.

904
00:58:21,300 --> 00:58:25,110
More more than n independent
generators, in the sense that

905
00:58:25,110 --> 00:58:28,840
the set of all the
binary linear

906
00:58:28,840 --> 00:58:32,050
combinations are distinct.

907
00:58:32,050 --> 00:58:32,460
All right.

908
00:58:32,460 --> 00:58:36,740
So k is, at most,
going to be n.

909
00:58:36,740 --> 00:58:39,310
So I will stop in a finite
number of steps.

910
00:58:39,310 --> 00:58:42,020
I'll stop at some number k.

911
00:58:42,020 --> 00:58:46,690
When I've stopped, that's
because the code consists of

912
00:58:46,690 --> 00:58:50,100
all binary linear combinations
of these k generators, and

913
00:58:50,100 --> 00:58:54,480
therefore has size 2 to the k.

914
00:58:54,480 --> 00:59:02,640
So the only possible size of a
subspace is 2 to the k for k

915
00:59:02,640 --> 00:59:04,220
less than n.

916
00:59:04,220 --> 00:59:10,160
A power of 2 where the
power is, at most, n.

917
00:59:10,160 --> 00:59:13,670
So any subspace besides
the subspace --

918
00:59:13,670 --> 00:59:14,920
I'm repeating myself.

919
00:59:17,910 --> 00:59:18,680
OK.

920
00:59:18,680 --> 00:59:25,370
So this means I found a basis.

921
00:59:25,370 --> 00:59:28,760
Does this mean that all possible
bases of any subspace

922
00:59:28,760 --> 00:59:30,010
have the same size?

923
00:59:33,284 --> 00:59:34,490
Well yeah, it must.

924
00:59:34,490 --> 00:59:40,650
I mean, I've proved now that any
subspace has to have this

925
00:59:40,650 --> 00:59:45,000
size 2 to the k for some k.

926
00:59:45,000 --> 00:59:47,660
So obviously, if I go through
this process, no matter how I

927
00:59:47,660 --> 00:59:54,330
choose my generators, I could
choose any pair of these

928
00:59:54,330 --> 00:59:57,550
non-zero n-tuples as
my generators.

929
00:59:57,550 --> 00:59:59,250
So that would be a
legitimate basis.

930
00:59:59,250 --> 01:00:01,320
Take any two out
of these three.

931
01:00:01,320 --> 01:00:04,010
But it's always going to take
exactly two of them, right?

932
01:00:04,010 --> 01:00:05,060
Why?

933
01:00:05,060 --> 01:00:08,510
Because the size of this
subspace is four.

934
01:00:08,510 --> 01:00:10,140
Two to the two.

935
01:00:10,140 --> 01:00:13,060
So if I go through this process,
I'm always going to

936
01:00:13,060 --> 01:00:17,590
come up with k generators, where
k is the log to the base

937
01:00:17,590 --> 01:00:21,340
2 of the size of this code that
I was given, which I was

938
01:00:21,340 --> 01:00:27,810
told was a linear code, meaning
it's a subspace.

939
01:00:27,810 --> 01:00:33,330
So I'm somewhat free to choose
the generators, but I'm always

940
01:00:33,330 --> 01:00:35,820
going to come up with k of
them if the code has

941
01:00:35,820 --> 01:00:37,070
size 2 to the k.

942
01:00:39,900 --> 01:00:50,160
So a basis is a set of k
linearly independent k-tuples,

943
01:00:50,160 --> 01:00:52,750
where linear independence has
the same meaning here as

944
01:00:52,750 --> 01:00:56,580
you're accustomed to, meaning
that all linear combinations

945
01:00:56,580 --> 01:01:00,310
of the k generators
are distinct.

946
01:01:00,310 --> 01:01:05,020
So I get 2 to the k distinct
elements of the code.

947
01:01:05,020 --> 01:01:10,210
And I say the dimension
of the code is k.

948
01:01:10,210 --> 01:01:13,040
In this case, basically it's the
size of the code is 2 to

949
01:01:13,040 --> 01:01:16,710
the k, then its dimension
is k.

950
01:01:16,710 --> 01:01:18,580
Has to be.

951
01:01:18,580 --> 01:01:24,610
And all basis have k
generators in them.

952
01:01:24,610 --> 01:01:27,800
And there are, in general,
many ways to pick them.

953
01:01:27,800 --> 01:01:29,300
All right?

954
01:01:29,300 --> 01:01:33,820
So just by considering this
greedy basis construction

955
01:01:33,820 --> 01:01:43,440
algorithm, I basically find the
size of any subspace is a

956
01:01:43,440 --> 01:01:46,850
power of two, and the power
is equal to the dimension.

957
01:01:46,850 --> 01:01:49,870
And any basis is going to
have that cardinality.

958
01:01:52,730 --> 01:01:53,580
Are you with me?

959
01:01:53,580 --> 01:01:57,150
This is a pretty simple proof.

960
01:01:57,150 --> 01:02:02,070
And I call this an n, k binary
linear block code.

961
01:02:02,070 --> 01:02:04,930
n being the length.

962
01:02:04,930 --> 01:02:09,950
That just means that every code
word is an n-tuple over

963
01:02:09,950 --> 01:02:11,660
the binary field.

964
01:02:11,660 --> 01:02:13,670
And k is the dimension.

965
01:02:13,670 --> 01:02:20,510
So an n, k binary linear block
code has size 2 to the k.

966
01:02:24,410 --> 01:02:29,170
That's 2 to the k distinct
code words.

967
01:02:29,170 --> 01:02:31,000
Main example is this guy.

968
01:02:35,070 --> 01:02:35,640
Easy?

969
01:02:35,640 --> 01:02:36,620
Any questions?

970
01:02:36,620 --> 01:02:37,910
I think this is clear.

971
01:02:37,910 --> 01:02:41,155
AUDIENCE: Can n use the number
of code word, I take

972
01:02:41,155 --> 01:02:44,190
like 2 to the k?

973
01:02:44,190 --> 01:02:48,060
PROFESSOR: n is something I
specify a priori as the length

974
01:02:48,060 --> 01:02:50,660
of every vector in the code.

975
01:02:50,660 --> 01:02:56,260
In other words, it has size 2 to
the k, and it's a subset of

976
01:02:56,260 --> 01:02:58,320
the set of all binary
n-tuples, which

977
01:02:58,320 --> 01:02:59,550
I write like that.

978
01:02:59,550 --> 01:03:01,050
In other words, the
elements of the

979
01:03:01,050 --> 01:03:03,662
code are binary n-tuples.

980
01:03:03,662 --> 01:03:05,580
If I write them out, each
one has length n.

981
01:03:09,610 --> 01:03:10,130
OK?

982
01:03:10,130 --> 01:03:12,240
We're good?

983
01:03:12,240 --> 01:03:13,490
What are some other examples?

984
01:03:18,810 --> 01:03:23,720
The simplest codes you can
think of is, first of

985
01:03:23,720 --> 01:03:26,600
all, an n, 0 code.

986
01:03:26,600 --> 01:03:29,800
That means a code with dimension
zero, has size what?

987
01:03:32,380 --> 01:03:33,510
1.

988
01:03:33,510 --> 01:03:35,350
And what does it consist of?

989
01:03:35,350 --> 01:03:37,230
AUDIENCE: [INAUDIBLE]

990
01:03:37,230 --> 01:03:37,700
PROFESSOR: Yeah.

991
01:03:37,700 --> 01:03:41,870
So this is the so-called trivial
code, just containing

992
01:03:41,870 --> 01:03:45,450
the all-0 word.

993
01:03:45,450 --> 01:03:46,780
Has to mention 0.

994
01:03:46,780 --> 01:03:49,790
It is a binary linear block
code, but it's not much use

995
01:03:49,790 --> 01:03:52,490
for communication.

996
01:03:52,490 --> 01:03:55,720
So but nonetheless, we include
that in this family.

997
01:03:58,420 --> 01:04:01,760
Another trivial one is n, n.

998
01:04:01,760 --> 01:04:03,068
What's that?

999
01:04:03,068 --> 01:04:06,000
AUDIENCE: [INAUDIBLE]

1000
01:04:06,000 --> 01:04:07,480
PROFESSOR: F2 to the n, right.

1001
01:04:07,480 --> 01:04:08,730
What's its size?

1002
01:04:08,730 --> 01:04:09,310
2 to the n.

1003
01:04:09,310 --> 01:04:15,370
That means it has to contain all
distinct binary n-tuples.

1004
01:04:15,370 --> 01:04:18,340
So this is called the
trivial code.

1005
01:04:18,340 --> 01:04:21,210
This is called the
universe code.

1006
01:04:21,210 --> 01:04:24,470
Contains the entire universe
of binary n-tuples.

1007
01:04:27,160 --> 01:04:29,920
Let's get some slightly
less trivial ones.

1008
01:04:29,920 --> 01:04:32,850
n, 1.

1009
01:04:32,850 --> 01:04:36,330
What would that be?

1010
01:04:36,330 --> 01:04:38,280
An n, 1 code.

1011
01:04:38,280 --> 01:04:40,640
What's its size?

1012
01:04:40,640 --> 01:04:41,310
2.

1013
01:04:41,310 --> 01:04:42,560
What does it consist of?

1014
01:04:44,940 --> 01:04:48,020
The 0 word, and?

1015
01:04:48,020 --> 01:04:50,760
Any other non-zero generator.

1016
01:04:50,760 --> 01:04:52,250
And that's correct.

1017
01:04:52,250 --> 01:04:59,160
This can be 0 and any generator,
two words.

1018
01:04:59,160 --> 01:05:03,440
In communications, where we want
to maximize the distance,

1019
01:05:03,440 --> 01:05:10,330
in some sense, between the
two code words, what is g

1020
01:05:10,330 --> 01:05:12,650
always taken as?

1021
01:05:12,650 --> 01:05:14,430
0,1's, right.

1022
01:05:14,430 --> 01:05:20,160
So if it's in particular the
all-0 and the all-1 word,

1023
01:05:20,160 --> 01:05:23,410
which I might write as a vector
of 0's and a vector of

1024
01:05:23,410 --> 01:05:28,846
1's, this is called the
repetition code.

1025
01:05:28,846 --> 01:05:32,600
The binary repetition
code of length n.

1026
01:05:32,600 --> 01:05:36,310
It either gives me a 0 and I
repeat it n times, or gives me

1027
01:05:36,310 --> 01:05:39,010
a 1 and I repeat it n times.

1028
01:05:39,010 --> 01:05:41,860
So whenever you see n,1, you
can pretty well assume it's

1029
01:05:41,860 --> 01:05:48,280
the repetition code, though
it might be any pair 0, g.

1030
01:05:48,280 --> 01:05:49,490
And n, minus 1.

1031
01:05:49,490 --> 01:05:50,860
This is an interesting one.

1032
01:05:58,400 --> 01:06:07,930
Again, while this could
be a lot of things, in

1033
01:06:07,930 --> 01:06:12,410
communications, whenever you see
this, this will always be

1034
01:06:12,410 --> 01:06:16,320
the set of all even-weight
n-tuples.

1035
01:06:22,040 --> 01:06:28,590
In other words, the set of all
n-tuples with even parity such

1036
01:06:28,590 --> 01:06:31,310
that if you sum up all of
the components of any

1037
01:06:31,310 --> 01:06:35,340
vector, mod2 equals 0.

1038
01:06:35,340 --> 01:06:36,340
OK?

1039
01:06:36,340 --> 01:06:47,110
So this I will call the single
parity check, or more briefly,

1040
01:06:47,110 --> 01:06:51,410
the SPC code, or the
even-weight code.

1041
01:06:51,410 --> 01:06:52,660
That's equally good.

1042
01:06:55,260 --> 01:07:01,200
And here we maybe should do
a little bit more work.

1043
01:07:01,200 --> 01:07:03,350
Say, is this in fact
a subspace?

1044
01:07:03,350 --> 01:07:06,080
Does it include the all-zero
code word?

1045
01:07:06,080 --> 01:07:08,990
Yes, all-zero has even weight.

1046
01:07:08,990 --> 01:07:12,350
The sum of any two even-weight
code words, an even-weight

1047
01:07:12,350 --> 01:07:15,810
code word, an even-weight
n-tuple.

1048
01:07:15,810 --> 01:07:17,360
Yes.

1049
01:07:17,360 --> 01:07:18,040
As here.

1050
01:07:18,040 --> 01:07:19,250
This is an example.

1051
01:07:19,250 --> 01:07:22,916
This is the three,
two SPC code.

1052
01:07:27,290 --> 01:07:28,070
OK.

1053
01:07:28,070 --> 01:07:32,242
Why is this dimension
n minus one?

1054
01:07:32,242 --> 01:07:34,690
AUDIENCE: [INAUDIBLE]

1055
01:07:34,690 --> 01:07:37,840
Is every code word orthogonal
to the one-vector?

1056
01:07:37,840 --> 01:07:38,800
PROFESSOR: OK.

1057
01:07:38,800 --> 01:07:41,620
That's an excellent answer.

1058
01:07:41,620 --> 01:07:43,590
It's a little advanced
for us right now.

1059
01:07:43,590 --> 01:07:45,993
I'm looking for an elementary
argument.

1060
01:07:45,993 --> 01:07:47,243
AUDIENCE: [INAUDIBLE]

1061
01:07:57,600 --> 01:07:57,990
PROFESSOR: OK.

1062
01:07:57,990 --> 01:08:00,980
So you're saying we have
a set of generators

1063
01:08:00,980 --> 01:08:03,200
that looks like this.

1064
01:08:03,200 --> 01:08:05,210
Is that what you're saying?

1065
01:08:05,210 --> 01:08:07,180
You are correct.

1066
01:08:07,180 --> 01:08:12,980
And how many such generators
are there?

1067
01:08:12,980 --> 01:08:16,229
There are n minus 1 of them.

1068
01:08:16,229 --> 01:08:19,760
I always like to find the most
elementary argument possible.

1069
01:08:19,760 --> 01:08:23,910
I think the most elementary
argument here is that the

1070
01:08:23,910 --> 01:08:26,279
number of even-weight n-tuples
is equal to the number of

1071
01:08:26,279 --> 01:08:29,160
odd-weight n-tuples, and
together they form the set of

1072
01:08:29,160 --> 01:08:30,819
all n-tuples.

1073
01:08:30,819 --> 01:08:34,279
So exactly half of the n-tuples
are even weight.

1074
01:08:34,279 --> 01:08:38,029
That means there are 2 to the n
minus 1 of them, 2 to the n

1075
01:08:38,029 --> 01:08:42,109
over 2, and therefore, the
dimension must be n minus 1.

1076
01:08:42,109 --> 01:08:48,575
But perhaps this is just
as elementary a proof.

1077
01:08:48,575 --> 01:08:51,124
Well, however you do it, you'll
find that there are 2

1078
01:08:51,124 --> 01:08:56,529
to the n minus 1 of them, or
that here is clearly a set of

1079
01:08:56,529 --> 01:08:57,260
generators.

1080
01:08:57,260 --> 01:09:00,580
It might take a few more lines
to show that every even-weight

1081
01:09:00,580 --> 01:09:05,720
code word is a linear
combination of this particular

1082
01:09:05,720 --> 01:09:08,760
set of generators, but
it's certainly true.

1083
01:09:08,760 --> 01:09:09,149
All right.

1084
01:09:09,149 --> 01:09:13,800
So these four classes of codes,
these two entirely

1085
01:09:13,800 --> 01:09:17,660
trivial ones, these two which
are actually somewhat more

1086
01:09:17,660 --> 01:09:19,859
interesting for coding
purposes --

1087
01:09:19,859 --> 01:09:23,710
we've already seen, we can get a
coding game with this length

1088
01:09:23,710 --> 01:09:25,924
three, dimension two code --

1089
01:09:28,450 --> 01:09:33,840
are basically the simplest
codes we can think of.

1090
01:09:33,840 --> 01:09:36,210
The simplest binary linear
block codes.

1091
01:09:36,210 --> 01:09:38,529
Now we'll see them
again and again.

1092
01:09:38,529 --> 01:09:41,220
They turn up.

1093
01:09:41,220 --> 01:09:41,649
All right?

1094
01:09:41,649 --> 01:09:43,750
So the whole course is
going to be about

1095
01:09:43,750 --> 01:09:45,510
finding 1's in between.

1096
01:09:45,510 --> 01:09:47,045
More complicated ones.

1097
01:09:47,045 --> 01:09:49,260
There's clearly more
room to play.

1098
01:09:49,260 --> 01:09:55,650
For instance, if k is equal to
half of n, which means that

1099
01:09:55,650 --> 01:09:59,670
rho is equal to one bit per two
dimensions, there's a lot

1100
01:09:59,670 --> 01:10:03,360
of possibilities.

1101
01:10:03,360 --> 01:10:06,335
And so we're going to explore
those possibilities.

1102
01:10:12,444 --> 01:10:13,694
AUDIENCE: [INAUDIBLE]

1103
01:10:16,276 --> 01:10:17,526
[UNINTELLIGIBLE]

1104
01:10:27,660 --> 01:10:28,910
PROFESSOR: Sure.

1105
01:10:31,410 --> 01:10:37,790
Let's take the 6,5 code
generated by these five

1106
01:10:37,790 --> 01:10:39,040
generators.

1107
01:10:40,730 --> 01:10:42,200
Well, it contains some

1108
01:10:42,200 --> 01:10:43,525
odd-weight code words, obviously.

1109
01:10:46,980 --> 01:10:50,750
But it's not as interesting from
a coding point of view.

1110
01:10:54,320 --> 01:10:56,100
It's not the only one,
but it's the only one

1111
01:10:56,100 --> 01:10:57,350
you'll ever see here.

1112
01:11:01,220 --> 01:11:01,700
All right.

1113
01:11:01,700 --> 01:11:02,950
Let's see.

1114
01:11:07,580 --> 01:11:11,020
One thing I didn't point out
in the notes but probably

1115
01:11:11,020 --> 01:11:15,510
should have here is, what is rho
for an n, k binary linear

1116
01:11:15,510 --> 01:11:16,760
block code?

1117
01:11:19,530 --> 01:11:21,620
How many bits can we send?

1118
01:11:21,620 --> 01:11:24,400
Suppose we take the Euclidean
image of this.

1119
01:11:24,400 --> 01:11:29,670
It is going to have 2 to the k
points, 2 to the k vertices of

1120
01:11:29,670 --> 01:11:35,140
the n-cube, and so
what is rho?

1121
01:11:35,140 --> 01:11:37,210
What is the rate in bits
per two dimensions?

1122
01:11:37,210 --> 01:11:38,460
AUDIENCE: [INAUDIBLE]

1123
01:11:40,526 --> 01:11:43,160
PROFESSOR: Right.

1124
01:11:43,160 --> 01:11:49,580
The rate is basically k over n
over 2, if you like, or 2 k

1125
01:11:49,580 --> 01:11:54,480
over n bits per two
dimensions.

1126
01:11:54,480 --> 01:12:00,010
You can send k bits in n
dimensions, or 2 k over n bits

1127
01:12:00,010 --> 01:12:01,870
per two dimensions.

1128
01:12:01,870 --> 01:12:08,070
And since k can't be larger than
n, this is going to be

1129
01:12:08,070 --> 01:12:10,900
less than or equal to two
bits per two dimensions.

1130
01:12:10,900 --> 01:12:14,200
So again, we see we're
definitely in the

1131
01:12:14,200 --> 01:12:20,790
power-limited regime, and that
we can really get any nominal

1132
01:12:20,790 --> 01:12:27,910
spectral efficiency between 0
and 2 by choosing k and n

1133
01:12:27,910 --> 01:12:30,750
appropriately.

1134
01:12:30,750 --> 01:12:31,080
All right.

1135
01:12:31,080 --> 01:12:34,420
So n and k determine the rate,
determine the nominal spectral

1136
01:12:34,420 --> 01:12:35,670
efficiency.

1137
01:12:37,930 --> 01:12:38,170
All right.

1138
01:12:38,170 --> 01:12:42,970
Let's now talk about things that
I think are also very old

1139
01:12:42,970 --> 01:12:44,120
hat to you.

1140
01:12:44,120 --> 01:12:45,370
I mean weight and distance.

1141
01:12:49,160 --> 01:12:57,840
But we begin to get into areas
that show us that these vector

1142
01:12:57,840 --> 01:13:00,820
spaces are very different from
the real and complex vector

1143
01:13:00,820 --> 01:13:03,000
spaces that we're
accustomed to.

1144
01:13:03,000 --> 01:13:04,870
So what are we doing now?

1145
01:13:04,870 --> 01:13:08,140
We're starting to get into the
geometry of this vector space.

1146
01:13:08,140 --> 01:13:11,510
The geometry is not Euclidean
geometry, but

1147
01:13:11,510 --> 01:13:15,090
it's Hamming geometry.

1148
01:13:15,090 --> 01:13:20,440
We define the Hamming weight
of a vector as simply the

1149
01:13:20,440 --> 01:13:29,000
number of 1's in v. So the
Hamming weight of the all-0

1150
01:13:29,000 --> 01:13:31,870
vector is 0, the Hamming weight
of the all-1 vector is

1151
01:13:31,870 --> 01:13:34,800
n, and in general, the Hamming
weight is somewhere

1152
01:13:34,800 --> 01:13:36,020
between 0 and n.

1153
01:13:36,020 --> 01:13:37,910
Just the number of 1's.

1154
01:13:37,910 --> 01:13:39,980
Pretty simple.

1155
01:13:39,980 --> 01:13:49,686
And given two vectors x and
y, what is their distance?

1156
01:13:53,750 --> 01:13:58,145
The Hamming distance between x
and y is equal to the Hamming

1157
01:13:58,145 --> 01:14:00,450
weight of x minus y.

1158
01:14:00,450 --> 01:14:04,270
This is the standard way of
converting a weight metric

1159
01:14:04,270 --> 01:14:06,480
into a distance metric.

1160
01:14:06,480 --> 01:14:11,270
Or because addition is the
same as subtraction in a

1161
01:14:11,270 --> 01:14:15,710
binary vector space, we might
equally well write this is as

1162
01:14:15,710 --> 01:14:17,490
the Hamming weight
of x plus y.

1163
01:14:20,500 --> 01:14:28,970
And more informally, this is
simply the number of places in

1164
01:14:28,970 --> 01:14:30,220
which they differ.

1165
01:14:37,260 --> 01:14:37,790
OK.

1166
01:14:37,790 --> 01:14:42,350
So if x and y are identical,
then x plus y is equal to 0

1167
01:14:42,350 --> 01:14:44,400
and the distance is 0.

1168
01:14:44,400 --> 01:14:49,850
If they are complementary, y is
the complement of x, then

1169
01:14:49,850 --> 01:14:51,400
they differ in every place.

1170
01:14:51,400 --> 01:14:57,150
The sum will then be the all-1
vector, and the weight, the

1171
01:14:57,150 --> 01:14:59,730
Hamming distance, will be n.

1172
01:14:59,730 --> 01:15:02,060
And so again, the Hamming
distance is somewhere between

1173
01:15:02,060 --> 01:15:05,220
0 and n, measures how
different they are.

1174
01:15:05,220 --> 01:15:07,860
Clearly going to be important
for coding.

1175
01:15:07,860 --> 01:15:10,510
It's going to translate directly
into Euclidean

1176
01:15:10,510 --> 01:15:13,695
distance under this standard
2-PAM map.

1177
01:15:13,695 --> 01:15:14,945
OK.

1178
01:15:17,290 --> 01:15:24,570
Again, let's check that it
satisfies the distance axioms.

1179
01:15:24,570 --> 01:15:31,150
I don't know how many of you
have seen this, but let's see.

1180
01:15:31,150 --> 01:15:34,710
What are the distance axioms?

1181
01:15:34,710 --> 01:15:35,960
Strict non-negativity.

1182
01:15:40,700 --> 01:15:46,820
In other words, the Hamming
distance between x and y --

1183
01:15:46,820 --> 01:15:48,480
that's a single comma --

1184
01:15:48,480 --> 01:15:53,140
is greater than or equal to
0, and equality if and

1185
01:15:53,140 --> 01:15:55,750
only if x equals y.

1186
01:15:55,750 --> 01:15:58,280
That's what strict means.

1187
01:15:58,280 --> 01:16:00,830
So if we find the Hamming
distance is 0, we can assert

1188
01:16:00,830 --> 01:16:02,080
that x equals y.

1189
01:16:04,340 --> 01:16:05,785
We have, of course, symmetry.

1190
01:16:09,680 --> 01:16:14,430
The Hamming distance between
x and y is the same as the

1191
01:16:14,430 --> 01:16:17,020
Hamming distance between
y and x.

1192
01:16:17,020 --> 01:16:18,970
Order doesn't matter.

1193
01:16:18,970 --> 01:16:26,190
And finally we have the triangle
inequality, that the

1194
01:16:26,190 --> 01:16:31,800
Hamming distance between x and z
certainly can't be more than

1195
01:16:31,800 --> 01:16:36,790
the Hamming distance between
x and y plus the Hamming

1196
01:16:36,790 --> 01:16:38,800
distance between y and z.

1197
01:16:41,490 --> 01:16:48,540
If x differs from y in only n1
places, and y differs from z

1198
01:16:48,540 --> 01:16:52,400
in only n2 places, then clearly
z can't differ from x

1199
01:16:52,400 --> 01:16:54,305
in more than n1 plus
n2 places.

1200
01:16:57,740 --> 01:16:59,360
So check, check, check.

1201
01:16:59,360 --> 01:17:04,820
This is a legitimate metric for
defining a geometry on the

1202
01:17:04,820 --> 01:17:10,000
space, and this is the one that
we use on the space of

1203
01:17:10,000 --> 01:17:12,140
all n-tuples.

1204
01:17:12,140 --> 01:17:15,760
But notice it's not all like
the Euclidean Distance.

1205
01:17:18,700 --> 01:17:22,110
Now when we have a
linear code --

1206
01:17:22,110 --> 01:17:23,430
let's combine these things.

1207
01:17:23,430 --> 01:17:39,010
When we have a linear code, we
have a group property which

1208
01:17:39,010 --> 01:17:40,700
is, let me write it this way.

1209
01:17:40,700 --> 01:17:47,430
If we take any code word and add
it to any other code word,

1210
01:17:47,430 --> 01:17:50,940
that's in the code.

1211
01:17:50,940 --> 01:17:58,990
And furthermore, c plus c prime
is not equal to c prime

1212
01:17:58,990 --> 01:18:03,220
plus c single prime, c
plus c single prime,

1213
01:18:03,220 --> 01:18:04,670
because we can --

1214
01:18:04,670 --> 01:18:06,300
well, I'll finish it.

1215
01:18:06,300 --> 01:18:13,290
Unless c prime equals
c double prime.

1216
01:18:13,290 --> 01:18:14,040
Why is that?

1217
01:18:14,040 --> 01:18:17,240
We can do subtraction,
cancellation.

1218
01:18:17,240 --> 01:18:19,470
Cancel c out from each side.

1219
01:18:19,470 --> 01:18:25,210
So if we add c to c double
prime, we're going to get a

1220
01:18:25,210 --> 01:18:30,160
different result from adding c
to c prime, if c prime and c

1221
01:18:30,160 --> 01:18:33,490
double prime are different.

1222
01:18:33,490 --> 01:18:39,200
So this implies that
c plus C --

1223
01:18:39,200 --> 01:18:43,010
I write that as an abbreviation
for the set of

1224
01:18:43,010 --> 01:18:50,840
all c plus c prime, as a c prime
runs through the code C.

1225
01:18:50,840 --> 01:18:59,130
So this is the 2 to the k sums
of the code plus any code word

1226
01:18:59,130 --> 01:19:00,380
in the code.

1227
01:19:03,450 --> 01:19:05,350
Sorry if I don't write
down all steps.

1228
01:19:05,350 --> 01:19:08,950
What is that going to be?

1229
01:19:08,950 --> 01:19:16,038
C. How did we conclude that?

1230
01:19:16,038 --> 01:19:17,288
AUDIENCE: [INAUDIBLE]

1231
01:19:19,510 --> 01:19:20,760
[UNINTELLIGIBLE]

1232
01:19:24,280 --> 01:19:25,920
PROFESSOR: Perfect.

1233
01:19:25,920 --> 01:19:28,100
Did you all hear that?

1234
01:19:28,100 --> 01:19:30,550
By the group property,
each one of these

1235
01:19:30,550 --> 01:19:32,190
things is in the code.

1236
01:19:32,190 --> 01:19:36,000
By this argument, no two
of them are the same.

1237
01:19:36,000 --> 01:19:39,760
That means I get 2 to the k
distinct elements all in the

1238
01:19:39,760 --> 01:19:42,170
code, that's got to be the code,
because the code only

1239
01:19:42,170 --> 01:19:43,420
has 2 to the k elements.

1240
01:19:46,850 --> 01:19:48,630
All right.

1241
01:19:48,630 --> 01:19:51,740
So if I add a code word --

1242
01:19:51,740 --> 01:19:54,360
in other words, if I write
down the code --

1243
01:19:54,360 --> 01:20:00,190
0, 0, 0, 0, 1, 1, 1 0 1, 1 1 0-
and I add any code word to

1244
01:20:00,190 --> 01:20:04,560
it -- say I add 0
1 1 to the code.

1245
01:20:04,560 --> 01:20:08,930
So let me just do one column
of the addition table.

1246
01:20:08,930 --> 01:20:14,530
I get 0, 1, 1, 0, 0, 0,
1, 1, 0, 1, 0, 1.

1247
01:20:14,530 --> 01:20:15,890
I'll get the code
itself again.

1248
01:20:21,070 --> 01:20:24,210
This has a very important
property.

1249
01:20:26,910 --> 01:20:38,460
The set of Hamming distances C
and c prime, as c prime runs

1250
01:20:38,460 --> 01:20:51,660
through c, from any given code
word C, it is independent of

1251
01:20:51,660 --> 01:20:59,920
C. So I can start from any code
word, measure the Hamming

1252
01:20:59,920 --> 01:21:02,960
distances, the 2 to the k
Hamming distances to all other

1253
01:21:02,960 --> 01:21:05,930
code words, including
C itself --

1254
01:21:05,930 --> 01:21:08,390
let c prime run through the
entire code, I'm going to get

1255
01:21:08,390 --> 01:21:11,570
a set of 2 to the k distances
called the distance

1256
01:21:11,570 --> 01:21:13,700
profile of the code.

1257
01:21:16,320 --> 01:21:19,670
And I claim that it doesn't
matter which code

1258
01:21:19,670 --> 01:21:21,870
word I start from.

1259
01:21:21,870 --> 01:21:24,510
I'm going to get the same
distance profile regardless of

1260
01:21:24,510 --> 01:21:26,130
where I start.

1261
01:21:26,130 --> 01:21:29,140
In other words, the set of all
distances from the all-0 word

1262
01:21:29,140 --> 01:21:35,430
here, which is 0, 2, 2, 2, is
the same as the set of all

1263
01:21:35,430 --> 01:21:38,270
distances from the 0,
1, 1, code word,

1264
01:21:38,270 --> 01:21:43,480
which is 2, 0, 2, 2.

1265
01:21:43,480 --> 01:21:47,540
And the proof is basically
that this is simply the

1266
01:21:47,540 --> 01:21:51,940
Hamming weight of C plus c prime
as c prime runs through

1267
01:21:51,940 --> 01:21:55,580
C. What is this equal to?

1268
01:21:55,580 --> 01:22:01,610
So the proof is that this is
equal to the Hamming weight of

1269
01:22:01,610 --> 01:22:07,020
c prime as c prime
runs through C.

1270
01:22:07,020 --> 01:22:10,900
So the distance profile from any
code word is simply equal

1271
01:22:10,900 --> 01:22:14,970
to the weight profile
of the code itself.

1272
01:22:14,970 --> 01:22:18,820
The weight profile of this code
is 0, 2, 2, 2, Start from

1273
01:22:18,820 --> 01:22:22,700
any code word, measure the
distances to other code words,

1274
01:22:22,700 --> 01:22:25,980
I'm always going to get
0, 2, 2, 2, 0, to

1275
01:22:25,980 --> 01:22:29,680
itself, and the others.

1276
01:22:29,680 --> 01:22:32,630
Sounds sort of like the
tetrahedron, doesn't it?

1277
01:22:32,630 --> 01:22:36,570
It's zero distance from a code
word to itself and equal

1278
01:22:36,570 --> 01:22:39,520
distance to all the other
code words, in

1279
01:22:39,520 --> 01:22:41,540
that particular case.

1280
01:22:41,540 --> 01:22:41,850
OK.

1281
01:22:41,850 --> 01:22:44,480
So again, everything is
very elementary here.

1282
01:22:44,480 --> 01:22:50,200
The distance profile is
independent of C and equal to

1283
01:22:50,200 --> 01:22:51,450
the weight profile.

1284
01:23:06,330 --> 01:23:10,720
So this has an extremely
important corollary.

1285
01:23:10,720 --> 01:23:14,440
What's the minimum Hamming
distance of the code?

1286
01:23:14,440 --> 01:23:16,110
You might expect this
is going to be an

1287
01:23:16,110 --> 01:23:17,450
important code parameter.

1288
01:23:17,450 --> 01:23:18,700
AUDIENCE: [INAUDIBLE]

1289
01:23:23,230 --> 01:23:25,900
PROFESSOR: The minimum Hamming
distance between any two code

1290
01:23:25,900 --> 01:23:28,620
words is going to
be equal to --

1291
01:23:28,620 --> 01:23:29,980
I think you said it.

1292
01:23:29,980 --> 01:23:31,860
AUDIENCE: The 0
[UNINTELLIGIBLE].

1293
01:23:31,860 --> 01:23:35,470
PROFESSOR: Non-zero
is important here.

1294
01:23:35,470 --> 01:23:37,910
If the distance profile is equal
to the weight profile,

1295
01:23:37,910 --> 01:23:40,660
one of the weights is always
going to be zero.

1296
01:23:40,660 --> 01:23:43,920
And that corresponds to the
distance between a code word

1297
01:23:43,920 --> 01:23:45,780
and itself.

1298
01:23:45,780 --> 01:23:46,070
All right.

1299
01:23:46,070 --> 01:23:50,170
If I go through all the other 2
to the k minus 1 distances,

1300
01:23:50,170 --> 01:23:52,840
they're going to be weights.

1301
01:23:52,840 --> 01:23:54,910
They're going to be the
distances from a code to all

1302
01:23:54,910 --> 01:23:56,760
other code words.

1303
01:23:56,760 --> 01:24:01,580
And the minimum distance is
simply going to be the minimum

1304
01:24:01,580 --> 01:24:04,370
non-zero weight of the code.

1305
01:24:04,370 --> 01:24:08,310
For example, in this code, the
minimum Hamming distance

1306
01:24:08,310 --> 01:24:12,990
between any two distinct code
words is going to be equal to

1307
01:24:12,990 --> 01:24:16,190
the minimum distance from
the 0 code word --

1308
01:24:16,190 --> 01:24:17,320
that's another way
of doing it.

1309
01:24:17,320 --> 01:24:20,950
Since it's independent of C, we
may as well take the base

1310
01:24:20,950 --> 01:24:23,490
code word C to be zero.

1311
01:24:23,490 --> 01:24:26,690
And then what's the minimum
distance to 0?

1312
01:24:26,690 --> 01:24:27,840
To the 0-code word?

1313
01:24:27,840 --> 01:24:29,650
It's the minimum weight.

1314
01:24:29,650 --> 01:24:33,380
From the 0 code word, distance
equals weight.

1315
01:24:33,380 --> 01:24:36,710
So the minimum distance is the
minimum weight of any non-zero

1316
01:24:36,710 --> 01:24:39,300
code word, which for
this code is two.

1317
01:24:43,500 --> 01:24:48,550
Now here the weight profile
is 0, 2, 2, 2, 0.

1318
01:24:48,550 --> 01:24:52,490
The distance profile from any
code word to all the others is

1319
01:24:52,490 --> 01:24:54,150
0, 2, 2,2 2.

1320
01:24:54,150 --> 01:24:56,100
This is always the distance
to itself.

1321
01:24:56,100 --> 01:24:59,950
So minimum distance to
other code words is

1322
01:24:59,950 --> 01:25:03,130
always going to be two.

1323
01:25:03,130 --> 01:25:08,560
Furthermore, the number of
nearest neighbors --

1324
01:25:08,560 --> 01:25:12,640
to go back and use chapter
five terminology --

1325
01:25:12,640 --> 01:25:16,530
the number of nearest neighbors
is going to be the

1326
01:25:16,530 --> 01:25:20,230
number of code words that have
that minimum weight --

1327
01:25:20,230 --> 01:25:22,760
in this case, three.

1328
01:25:22,760 --> 01:25:26,800
Still sounding a lot like
a tetrahedron, isn't it?

1329
01:25:26,800 --> 01:25:30,830
This easy map between Hamming
distance and Euclidean

1330
01:25:30,830 --> 01:25:37,840
distance for this case and in
general for all of our cases.

1331
01:25:37,840 --> 01:25:43,300
So corollary.

1332
01:25:43,300 --> 01:25:48,960
The minimum Hamming distance,
which implicitly means between

1333
01:25:48,960 --> 01:25:55,390
two distinct code words of C,
is equal to the minimum

1334
01:25:55,390 --> 01:26:06,120
non-zero weight of C, and the
number of minimum weight code

1335
01:26:06,120 --> 01:26:08,340
words is independent --

1336
01:26:11,400 --> 01:26:13,830
I'm doing this backwards.

1337
01:26:13,830 --> 01:26:18,670
From any code word, the number
of code words that distance,

1338
01:26:18,670 --> 01:26:25,350
let's call this d, is equal
to the number of

1339
01:26:25,350 --> 01:26:28,910
weight d code words.

1340
01:26:28,910 --> 01:26:31,110
Sorry, you probably
can't see that.

1341
01:26:36,360 --> 01:26:36,720
All right.

1342
01:26:36,720 --> 01:26:39,940
So we get this symmetry property
for codes that

1343
01:26:39,940 --> 01:26:45,900
follows from the group property
of the code that if

1344
01:26:45,900 --> 01:26:48,510
we stand on any code word and
look out, we're always going

1345
01:26:48,510 --> 01:26:51,590
to see the same thing.

1346
01:26:51,590 --> 01:26:53,410
We have this constant.

1347
01:26:53,410 --> 01:26:57,710
It's actually easiest to see
this when we make the map from

1348
01:26:57,710 --> 01:27:01,010
the code to the Euclidean
image of the code.

1349
01:27:08,740 --> 01:27:16,350
So the Euclidean image S of C
of the code word is going to

1350
01:27:16,350 --> 01:27:24,760
be some set of 2 to the
k vertices of an

1351
01:27:24,760 --> 01:27:29,295
n-cube of side alpha.

1352
01:27:35,270 --> 01:27:41,360
Let's talk about the Euclidean
image of these properties.

1353
01:27:41,360 --> 01:27:48,210
If the minimum Hamming distance
of the code is d,

1354
01:27:48,210 --> 01:27:52,670
what's the minimum squared
Euclidean distance between

1355
01:27:52,670 --> 01:27:54,470
elements of S of
C going to be?

1356
01:27:58,450 --> 01:28:00,975
Well, let's do it coordinate
by coordinates.

1357
01:28:03,630 --> 01:28:11,800
Let's take two code words,
C and c prime, let's say.

1358
01:28:11,800 --> 01:28:14,560
And let's suppose we have
some Hamming distance

1359
01:28:14,560 --> 01:28:15,910
between C and c prime.

1360
01:28:18,740 --> 01:28:23,940
That means that c and c prime
differ in the Hamming

1361
01:28:23,940 --> 01:28:26,450
distance, number of places.

1362
01:28:26,450 --> 01:28:35,300
So if we map this into the
corresponding pair of vertices

1363
01:28:35,300 --> 01:28:38,980
of the n-cube in Euclidean
space, S of C and S of c

1364
01:28:38,980 --> 01:28:43,500
prime, how many coordinates are
these going to differ in?

1365
01:28:46,720 --> 01:28:51,150
It's going to differ in same
number of coordinates, D_h.

1366
01:28:51,150 --> 01:28:55,030
If they don't differ, what's
Euclidean squared distance in

1367
01:28:55,030 --> 01:28:56,810
those coordinates?

1368
01:28:56,810 --> 01:28:57,190
0.

1369
01:28:57,190 --> 01:29:01,300
If they do differ, the Euclidean
squared distance is

1370
01:29:01,300 --> 01:29:03,080
4 alpha squared.

1371
01:29:03,080 --> 01:29:11,320
So the Euclidean distance D_e
between S of C and S of c

1372
01:29:11,320 --> 01:29:15,000
prime is simply going to be
4 alpha squared times the

1373
01:29:15,000 --> 01:29:18,710
Hamming distance between
C and c prime, yes?

1374
01:29:21,490 --> 01:29:25,230
So I should say this is the
squared Euclidean distance.

1375
01:29:25,230 --> 01:29:26,430
Why do we always talk about the

1376
01:29:26,430 --> 01:29:27,600
squared Euclidean distance?

1377
01:29:27,600 --> 01:29:30,730
Because it's additive,
coordinate-wise.

1378
01:29:30,730 --> 01:29:33,900
And the Hamming distance is
additive, coordinate-wise.

1379
01:29:33,900 --> 01:29:36,260
So there's a nice
easy map here.

1380
01:29:38,990 --> 01:29:42,020
So what does this mean d_min
squared is going to be?

1381
01:29:44,620 --> 01:29:50,910
d_min squared of, let's say, S
of C. This constellation that

1382
01:29:50,910 --> 01:29:54,942
we've formed by taking the
Euclidean image of c.

1383
01:29:54,942 --> 01:30:00,050
The minimum square distance
between points in S of C is

1384
01:30:00,050 --> 01:30:05,140
just going to be 4 alpha squared
times d, where I don't

1385
01:30:05,140 --> 01:30:06,810
think I ever --

1386
01:30:06,810 --> 01:30:10,585
d equals min Hamming distance.

1387
01:30:13,720 --> 01:30:18,980
And we're always going to talk
about n, k, d as the three key

1388
01:30:18,980 --> 01:30:22,630
parameters of a binary
linear block code.

1389
01:30:22,630 --> 01:30:26,920
n is the code length, F2 to the
n, k is the dimension, d

1390
01:30:26,920 --> 01:30:29,085
is the minimum Hamming
distance.

1391
01:30:29,085 --> 01:30:32,930
So by going into this Hamming
geometry, we've got a third

1392
01:30:32,930 --> 01:30:34,730
key property of the code.

1393
01:30:34,730 --> 01:30:37,300
And we see it's key, because we
can get the minimum squared

1394
01:30:37,300 --> 01:30:40,160
distance between this Euclidean
image constellation,

1395
01:30:40,160 --> 01:30:43,488
just 4 alpha squared d.

1396
01:30:43,488 --> 01:30:45,868
AUDIENCE: nd makes
a probability of

1397
01:30:45,868 --> 01:30:48,248
[UNINTELLIGIBLE] that is
dependent on [UNINTELLIGIBLE]

1398
01:30:48,248 --> 01:30:49,210
then.

1399
01:30:49,210 --> 01:30:49,750
PROFESSOR: Correct.

1400
01:30:49,750 --> 01:30:53,780
This is all we need to know to
get the union bound estimate.

1401
01:30:53,780 --> 01:30:55,660
Well, a few more things.

1402
01:30:55,660 --> 01:31:05,590
We need to know what K_min
average of S of C. And what is

1403
01:31:05,590 --> 01:31:08,560
that going to be?

1404
01:31:08,560 --> 01:31:13,730
This is simply going to be the
number of words in the code.

1405
01:31:13,730 --> 01:31:17,780
To get this minimum squared
distance, we need a Hamming

1406
01:31:17,780 --> 01:31:20,570
distance of d.

1407
01:31:20,570 --> 01:31:24,700
So the number of words in the
code of distance d, which is

1408
01:31:24,700 --> 01:31:29,380
given by the parameter n sub d,
is simply going to be the

1409
01:31:29,380 --> 01:31:31,010
number of nearest neighbors.

1410
01:31:31,010 --> 01:31:35,210
Not just the average distance,
but I want to emphasize this

1411
01:31:35,210 --> 01:31:36,320
symmetry property.

1412
01:31:36,320 --> 01:31:41,700
If we stand on any point, on
any vertex of this cube in

1413
01:31:41,700 --> 01:31:44,610
n-space, which is the code
vertex, and we look at all the

1414
01:31:44,610 --> 01:31:48,030
other points in the
constellation, no matter which

1415
01:31:48,030 --> 01:31:50,280
point we stand on, we
will always see the

1416
01:31:50,280 --> 01:31:53,020
same profile of distances.

1417
01:31:53,020 --> 01:31:57,580
We'll see precisely nd code
words at Euclidean distance 4

1418
01:31:57,580 --> 01:31:58,990
alpha squared d.

1419
01:31:58,990 --> 01:32:03,410
We'll see nd plus 1 at Euclidean
squared distance 4

1420
01:32:03,410 --> 01:32:08,050
alpha squared d plus 1, and so
forth, right up the profile.

1421
01:32:08,050 --> 01:32:12,020
So there's complete symmetry
in the constellation.

1422
01:32:12,020 --> 01:32:15,500
In that universe, you don't know
which code point you're

1423
01:32:15,500 --> 01:32:17,910
standing on just by looking out,
because the world looks

1424
01:32:17,910 --> 01:32:20,330
the same to you.

1425
01:32:20,330 --> 01:32:21,580
Is that clear?

1426
01:32:23,820 --> 01:32:23,945
OK.

1427
01:32:23,945 --> 01:32:27,330
So from a communications point
of view, this is important,

1428
01:32:27,330 --> 01:32:30,720
because it means it doesn't
matter what code word we send.

1429
01:32:30,720 --> 01:32:34,010
The probability of error from
any code word is going to be

1430
01:32:34,010 --> 01:32:36,250
the same as the probability of
error from any other code

1431
01:32:36,250 --> 01:32:40,150
word, because the geometry
is exactly the same.

1432
01:32:40,150 --> 01:32:43,370
The Voronoi regions are
all the same shape.

1433
01:32:43,370 --> 01:32:46,210
So given the exact probability
of error, not just the

1434
01:32:46,210 --> 01:32:48,860
union-bound estimate, is going
to be independent of which

1435
01:32:48,860 --> 01:32:50,580
code word was sent.

1436
01:32:50,580 --> 01:32:54,310
This all follows from the fact
that it's a linear code and

1437
01:32:54,310 --> 01:32:58,820
therefore has the group
property, which translates

1438
01:32:58,820 --> 01:33:01,920
into this very strong
geometrical uniformity

1439
01:33:01,920 --> 01:33:04,400
property in Euclidean space.

1440
01:33:04,400 --> 01:33:08,020
Or actually in Hamming space
too, but it's more striking in

1441
01:33:08,020 --> 01:33:09,270
Euclidean space.

1442
01:33:11,740 --> 01:33:13,700
OK?

1443
01:33:13,700 --> 01:33:17,250
So we have everything we
need to write down

1444
01:33:17,250 --> 01:33:18,500
the union bound estimate.

1445
01:33:22,620 --> 01:33:30,030
Union bound estimate was just
the probably of error per bit

1446
01:33:30,030 --> 01:33:38,945
is well approximated by K_b of
constellation, in this case, S

1447
01:33:38,945 --> 01:33:48,870
of C, times Q of the square root
of the coding gain of the

1448
01:33:48,870 --> 01:33:52,340
constellation times
2 Eb over N_0.