1 00:00:00,000 --> 00:00:02,150 PROFESSOR: Good moRning. 2 00:00:02,150 --> 00:00:04,019 Again, we're left with a little stub of 3 00:00:04,019 --> 00:00:07,020 chapter six to finish. 4 00:00:07,020 --> 00:00:09,640 This was our first chapter on binary linear block codes. 5 00:00:12,200 --> 00:00:16,810 Very briefly, last time we developed the family of RM, 6 00:00:16,810 --> 00:00:20,890 Reed-Muller codes, parameterized by R and M. 7 00:00:20,890 --> 00:00:23,300 These are codes of length two to the M, and distance 2 to 8 00:00:23,300 --> 00:00:27,735 the M minus R for all reasonable combinations of M 9 00:00:27,735 --> 00:00:29,460 and R, integers. 10 00:00:29,460 --> 00:00:32,570 And the reason I do these first is because it gives us a 11 00:00:32,570 --> 00:00:34,240 nice big -- in fact, infinite -- 12 00:00:34,240 --> 00:00:37,050 family of codes that kind of cover the waterfront. 13 00:00:37,050 --> 00:00:40,130 Let us see what we can reasonably get in terms of the 14 00:00:40,130 --> 00:00:42,860 parameters n k d in codes. 15 00:00:42,860 --> 00:00:47,510 And, of course, n k t d tell us most of what we know, want 16 00:00:47,510 --> 00:00:51,530 to know about the performance of moderate complexity codes 17 00:00:51,530 --> 00:00:54,820 in terms of coding gain. 18 00:00:54,820 --> 00:00:58,290 And it's easy to see that among these codes, we have 19 00:00:58,290 --> 00:01:01,390 sequences in which the coding gain goes -- the nominal 20 00:01:01,390 --> 00:01:03,450 coding gain goes to infinity. 21 00:01:03,450 --> 00:01:06,570 And we also know that there's at least one sequence, the 22 00:01:06,570 --> 00:01:10,660 bi-orthogonal sequence, for which the effective coding 23 00:01:10,660 --> 00:01:14,630 gain completely closes the gap to the Shannon limit. 24 00:01:14,630 --> 00:01:17,910 And in all likelihood, there are many such sequences, 25 00:01:17,910 --> 00:01:22,140 though I'm not aware of the actual result there. 26 00:01:22,140 --> 00:01:27,960 So it seems like we might be almost finished here, except I 27 00:01:27,960 --> 00:01:31,000 asked about what possible fly there could be in this 28 00:01:31,000 --> 00:01:32,300 ointment last time. 29 00:01:32,300 --> 00:01:36,710 And one of you alertly reported that we still really 30 00:01:36,710 --> 00:01:38,990 hadn't worried about decoding. 31 00:01:38,990 --> 00:01:42,290 And the implicit decoding algorithm here is minimum 32 00:01:42,290 --> 00:01:43,830 distance decoding. 33 00:01:43,830 --> 00:01:45,640 Exhaustive minimum distance decoding. 34 00:01:45,640 --> 00:01:48,980 We basically have to compute the minimum distance to each 35 00:01:48,980 --> 00:01:51,250 of the 2 to the k code words. 36 00:01:51,250 --> 00:01:54,100 And that's going to get a little tedious as soon as k 37 00:01:54,100 --> 00:01:58,210 gets up to 16 or 107 or whatever. 38 00:01:58,210 --> 00:02:03,220 So it was recognized pretty early in the development of 39 00:02:03,220 --> 00:02:06,170 coding theory that this was the essential problem. 40 00:02:06,170 --> 00:02:09,340 The problem is not to construct codes whose 41 00:02:09,340 --> 00:02:11,920 performance under maximum likelihood decoding is near 42 00:02:11,920 --> 00:02:13,450 that of the Shannon limit -- you can use 43 00:02:13,450 --> 00:02:16,030 random codes for that. 44 00:02:16,030 --> 00:02:19,950 But in constructing these more algebraically structured 45 00:02:19,950 --> 00:02:23,610 codes, what we're really trying to do is to facilitate 46 00:02:23,610 --> 00:02:26,640 the decoding problem, which is the central problem. 47 00:02:26,640 --> 00:02:30,170 We need to find a decoding method 48 00:02:30,170 --> 00:02:31,820 that is actually feasible. 49 00:02:31,820 --> 00:02:35,010 And, of course, Shannon didn't address that at all. 50 00:02:35,010 --> 00:02:38,540 He just addressed the ultimate potential of these codes with 51 00:02:38,540 --> 00:02:41,740 the optimal decoding technique and didn't worry about its 52 00:02:41,740 --> 00:02:44,200 complexity. 53 00:02:44,200 --> 00:02:44,640 All right. 54 00:02:44,640 --> 00:02:49,560 So I'd say the fundamental problem here is decoding 55 00:02:49,560 --> 00:02:50,810 complexity. 56 00:02:52,770 --> 00:02:55,270 Now, one of the -- 57 00:02:55,270 --> 00:03:00,080 an awful lot of the early work on coding was about binary 58 00:03:00,080 --> 00:03:01,490 codes, of course. 59 00:03:01,490 --> 00:03:10,070 But it also considered them in the context of binary decoding 60 00:03:10,070 --> 00:03:10,730 algorithms. 61 00:03:10,730 --> 00:03:13,450 In other words, what's assumed is what you send as a series 62 00:03:13,450 --> 00:03:16,890 of bits and what you receive as a series of bits. 63 00:03:16,890 --> 00:03:19,700 That's not the picture we have here in this course, where 64 00:03:19,700 --> 00:03:24,050 what we receive is a series of real numbers. 65 00:03:24,050 --> 00:03:26,400 Because we're going over a continuous additive white 66 00:03:26,400 --> 00:03:28,115 Gaussian noise channel. 67 00:03:28,115 --> 00:03:31,750 Of course, there are channels which are inherently digital, 68 00:03:31,750 --> 00:03:34,610 at least by the time the coding engineer can get his 69 00:03:34,610 --> 00:03:35,610 hands on them. 70 00:03:35,610 --> 00:03:38,400 What he's presented with is a string of bits. 71 00:03:38,400 --> 00:03:43,390 So it makes sense to consider codes for bit error 72 00:03:43,390 --> 00:03:45,120 correction. 73 00:03:45,120 --> 00:03:47,980 Classic codes of this type are, first of all, the 74 00:03:47,980 --> 00:03:49,670 repetition code. 75 00:03:49,670 --> 00:03:51,680 If I send this -- 76 00:03:51,680 --> 00:03:56,190 if I use the 7 1 7 code, the repetition code of length 7, 77 00:03:56,190 --> 00:03:59,370 I'm going to be able to correct up to three errors. 78 00:03:59,370 --> 00:04:01,450 Three bit errors. 79 00:04:01,450 --> 00:04:03,680 Because four bits are still going to be correct. 80 00:04:03,680 --> 00:04:06,480 In a majority vote, we'll do the decoding. 81 00:04:06,480 --> 00:04:08,580 Most of you have probably seen Hamming codes. 82 00:04:08,580 --> 00:04:12,860 These are codes with minimum Hamming distance 3. 83 00:04:12,860 --> 00:04:16,649 That means if you make one error, you're still within the 84 00:04:16,649 --> 00:04:20,240 Hamming sphere of radius 1 around the original code word. 85 00:04:20,240 --> 00:04:25,070 That's disjoined from all the other Hamming spheres, and so 86 00:04:25,070 --> 00:04:27,670 in principle you ought to be able to do single error 87 00:04:27,670 --> 00:04:30,560 correction with Hamming codes. 88 00:04:30,560 --> 00:04:34,180 And there are very simple algorithms for doing that. 89 00:04:34,180 --> 00:04:34,590 OK. 90 00:04:34,590 --> 00:04:38,600 But that may be what you have to do. 91 00:04:38,600 --> 00:04:42,450 The final point of chapter six is that in the additive white 92 00:04:42,450 --> 00:04:47,090 Gaussian noise context, that's not what you 93 00:04:47,090 --> 00:04:48,520 want to do at all. 94 00:04:48,520 --> 00:04:51,120 And I just want to say a few words about that because this 95 00:04:51,120 --> 00:04:55,140 is one of the Achille's heels of a lot of classical coding 96 00:04:55,140 --> 00:04:57,140 theory, which was addressed to the binary 97 00:04:57,140 --> 00:05:01,340 in, binary out situation. 98 00:05:01,340 --> 00:05:05,040 So let's talk about the penalty 99 00:05:05,040 --> 00:05:06,455 for making hard decisions. 100 00:05:12,260 --> 00:05:15,880 That is, by hard decision, we mean a binary decision. 101 00:05:15,880 --> 00:05:17,130 0, 1, plus or minus 1. 102 00:05:21,030 --> 00:05:25,390 And I'll do this fairly quickly, but in a way that I 103 00:05:25,390 --> 00:05:28,390 hope will stick. 104 00:05:28,390 --> 00:05:33,800 Let me draw our block diagram of our coding scheme. 105 00:05:33,800 --> 00:05:35,870 We've got a code. 106 00:05:35,870 --> 00:05:42,820 We've got an encoder, which basically 107 00:05:42,820 --> 00:05:45,850 puts out a code word. 108 00:05:45,850 --> 00:05:47,490 An n-tuple in the code. 109 00:05:47,490 --> 00:05:51,060 I'm going to consider the individual 110 00:05:51,060 --> 00:05:53,230 elements of this n-tuple. 111 00:05:53,230 --> 00:05:58,810 They're just elements of the binary field F2 And we go 112 00:05:58,810 --> 00:06:05,680 through our standard 2-PAM map, which maps them to s of 113 00:06:05,680 --> 00:06:10,210 xk, which is going to be in plus or 114 00:06:10,210 --> 00:06:11,750 minus alpha in general. 115 00:06:11,750 --> 00:06:13,950 Just plus or minus 1. 116 00:06:13,950 --> 00:06:16,240 Doesn't matter up to scaling. 117 00:06:16,240 --> 00:06:19,110 In order to send them, we're going to send this sequence of 118 00:06:19,110 --> 00:06:25,540 real numbers over the white Gaussian noise channel. 119 00:06:25,540 --> 00:06:29,610 And as a result, we're going to get out s -- 120 00:06:29,610 --> 00:06:31,780 let's call it a received symbol -- 121 00:06:31,780 --> 00:06:37,950 rk, which is the transmitted symbol plus some Gaussian 122 00:06:37,950 --> 00:06:40,270 noise symbol, nk. 123 00:06:40,270 --> 00:06:43,045 This is a real number. 124 00:06:46,330 --> 00:06:48,450 And then -- 125 00:06:48,450 --> 00:06:49,240 then what we do? 126 00:06:49,240 --> 00:06:50,950 Now we're into the decoder. 127 00:06:50,950 --> 00:06:54,610 This is what we received, so it's free to us to choose what 128 00:06:54,610 --> 00:06:55,860 to do next. 129 00:06:58,510 --> 00:07:01,590 I'm going to say observe, first of all, that without 130 00:07:01,590 --> 00:07:07,630 loss of generality, we can decompose rk 131 00:07:07,630 --> 00:07:11,040 into a sign and magnitude. 132 00:07:11,040 --> 00:07:17,910 So this out here is going to be a sign of rk. 133 00:07:17,910 --> 00:07:23,160 And this is going to be the magnitude of rk. 134 00:07:23,160 --> 00:07:25,300 And let me just give this a name. 135 00:07:25,300 --> 00:07:31,120 Let me call this yk, and let me call this beta k. 136 00:07:31,120 --> 00:07:31,600 OK. 137 00:07:31,600 --> 00:07:32,610 So what is this? 138 00:07:32,610 --> 00:07:40,000 This is in plus or minus 1, and this is in the positive 139 00:07:40,000 --> 00:07:41,906 reals, so the non-negative reals. 140 00:07:44,460 --> 00:07:44,580 OK. 141 00:07:44,580 --> 00:07:48,020 And clearly, if I keep both of these, I haven't lost any 142 00:07:48,020 --> 00:07:48,740 information. 143 00:07:48,740 --> 00:07:51,770 This is just a way of representing rk in sign and 144 00:07:51,770 --> 00:07:54,290 magnitude form. 145 00:07:54,290 --> 00:07:57,735 Now, this is what's called the hard decision. 146 00:08:01,160 --> 00:08:02,855 Actually, let me do this differently. 147 00:08:02,855 --> 00:08:11,130 Let me pass this through the inverse to a 2-PAM map. 148 00:08:11,130 --> 00:08:13,610 So out of this, I get a yk, 149 00:08:13,610 --> 00:08:17,540 which is in F2 OK. 150 00:08:17,540 --> 00:08:18,580 Either of these -- 151 00:08:18,580 --> 00:08:24,680 the real sign or the binary hard decision -- 152 00:08:24,680 --> 00:08:28,240 could be considered to be the hard decision. 153 00:08:28,240 --> 00:08:29,240 Now, what am I doing here? 154 00:08:29,240 --> 00:08:32,720 I'm saying, well, if you force me to make a guess on whether 155 00:08:32,720 --> 00:08:36,710 this individual transmission was sending a 0 or 1, this is 156 00:08:36,710 --> 00:08:39,050 the most obvious guess. 157 00:08:39,050 --> 00:08:42,839 If I send -- 158 00:08:42,839 --> 00:08:50,100 here's minus 1, here's our minus alpha, here's plus 1 -- 159 00:08:50,100 --> 00:08:55,400 if I sent a plus 1, then my maximum likelihood per bit 160 00:08:55,400 --> 00:08:58,710 decision would be to decide the plus 1 was sent. 161 00:08:58,710 --> 00:09:02,940 If I get a positive sign in -- 162 00:09:02,940 --> 00:09:07,360 what I'm really mapping here is rk, and minus 1 if I get a 163 00:09:07,360 --> 00:09:08,100 negative side. 164 00:09:08,100 --> 00:09:09,550 So that's what's called a hard decision. 165 00:09:09,550 --> 00:09:12,530 I'm going to decide right here what I think it was, what it 166 00:09:12,530 --> 00:09:13,570 most likely was. 167 00:09:13,570 --> 00:09:15,260 And this is the best way to do it. 168 00:09:15,260 --> 00:09:17,410 Just take the sign. 169 00:09:17,410 --> 00:09:22,740 This here is a real valued weight, often called the 170 00:09:22,740 --> 00:09:30,235 reliability, which is helpful to retain. 171 00:09:30,235 --> 00:09:32,830 That's the main point of what I'm going 172 00:09:32,830 --> 00:09:35,550 to talk to you about. 173 00:09:35,550 --> 00:09:36,990 If this is 0 -- 174 00:09:36,990 --> 00:09:40,900 that means what we received was right on this boundary 175 00:09:40,900 --> 00:09:44,390 between the positive and negative side -- 176 00:09:44,390 --> 00:09:46,980 then the reliability is 0. 177 00:09:46,980 --> 00:09:49,565 In likelihood terms, that means it's equally likely that 178 00:09:49,565 --> 00:09:52,540 what was sent was a plus 1 or a minus 1. 179 00:09:52,540 --> 00:09:55,520 So in some sense, we get no information when the 180 00:09:55,520 --> 00:09:57,900 reliability is 0. 181 00:09:57,900 --> 00:10:01,080 Fails to discriminate between plus 1 and minus 1. 182 00:10:01,080 --> 00:10:03,880 The larger this is, the further we get out here. 183 00:10:03,880 --> 00:10:07,710 Or symmetrically, the further we get out here, the more 184 00:10:07,710 --> 00:10:12,000 certain we are that in that particular symbol, neither a 185 00:10:12,000 --> 00:10:15,830 plus or a minus was sent. 186 00:10:15,830 --> 00:10:18,500 OK And it actually tuRns out, if you work out the Gaussian 187 00:10:18,500 --> 00:10:24,750 numbers, that beta k is the log likelihood ratio up to 188 00:10:24,750 --> 00:10:28,830 scale of the more likely versus the log of the 189 00:10:28,830 --> 00:10:32,920 likelihood, or the more likely versus the less likely symbol. 190 00:10:32,920 --> 00:10:35,010 So it has a minimum of 0. 191 00:10:35,010 --> 00:10:36,810 The bigger it is, the more reliable. 192 00:10:36,810 --> 00:10:38,630 So it's natural to call it the reliability. 193 00:10:41,280 --> 00:10:42,770 OK. 194 00:10:42,770 --> 00:10:48,470 An awful lot of traditional algebraic decoding neglects 195 00:10:48,470 --> 00:10:50,910 this channel down here. 196 00:10:50,910 --> 00:10:53,880 It just says, OK, let's take these hard decisions and try 197 00:10:53,880 --> 00:10:55,430 to decode them. 198 00:10:55,430 --> 00:10:58,510 If you've previously had a coding class, then that's 199 00:10:58,510 --> 00:11:00,000 probably what they did. 200 00:11:00,000 --> 00:11:03,020 Or they assumed a context in which this was never available 201 00:11:03,020 --> 00:11:05,900 in the first place. 202 00:11:05,900 --> 00:11:10,110 And my only point here, my main point, is that that's a 203 00:11:10,110 --> 00:11:13,150 very bad thing to do. 204 00:11:13,150 --> 00:11:16,420 How can we evaluate that? 205 00:11:16,420 --> 00:11:20,960 Suppose we just take this top channel and don't look at the 206 00:11:20,960 --> 00:11:21,720 reliability. 207 00:11:21,720 --> 00:11:25,570 Then basically we have the channel model becomes a binary 208 00:11:25,570 --> 00:11:26,550 symmetric channel. 209 00:11:26,550 --> 00:11:35,640 We have bits in, bits out, and we have probably one minus p 210 00:11:35,640 --> 00:11:38,610 that if we send a 0 we get a 0. 211 00:11:38,610 --> 00:11:41,830 And since it's symmetric, the same probability -- 212 00:11:41,830 --> 00:11:43,690 if we send a 1, we get a 1 -- 213 00:11:43,690 --> 00:11:45,610 and a probability p of making an error. 214 00:11:45,610 --> 00:11:48,880 So we get the traditional memoryless binary symmetric 215 00:11:48,880 --> 00:11:50,580 channel model. 216 00:11:50,580 --> 00:11:54,290 We can compute the capacity of this model and compare it to 217 00:11:54,290 --> 00:11:59,590 the capacity of the additive white Gaussian channel -- 218 00:11:59,590 --> 00:12:01,960 this model. 219 00:12:01,960 --> 00:12:05,930 And depending on the signal to noise ratio, from a capacity 220 00:12:05,930 --> 00:12:10,880 calculation, we find that there is of the order 221 00:12:10,880 --> 00:12:14,880 of 2 or 3 dB loss. 222 00:12:14,880 --> 00:12:17,640 Well, as we go on in this course, we're going to find 223 00:12:17,640 --> 00:12:21,250 that 1 dB is a very worthwhile coding game. 224 00:12:21,250 --> 00:12:24,600 So to throw away 2 or 3 dB just by throwing away this 225 00:12:24,600 --> 00:12:28,810 information, is a bad thing to do. 226 00:12:33,280 --> 00:12:39,970 I can make this point in a different way by simply 227 00:12:39,970 --> 00:12:45,650 looking at what's the optimum decision rule. 228 00:12:45,650 --> 00:12:50,170 For say, let me take very simple code, just the 2-1-2 229 00:12:50,170 --> 00:12:51,170 repetition code. 230 00:12:51,170 --> 00:12:54,010 That's just 0,0,1,1. 231 00:12:54,010 --> 00:12:58,720 This is the code where you either send plus, plus, plus 232 00:12:58,720 --> 00:13:04,910 alpha, plus alpha, or minus alpha, minus alpha. 233 00:13:04,910 --> 00:13:09,045 And what's the decision region? 234 00:13:09,045 --> 00:13:12,570 It's obviously this 45 degree line. 235 00:13:12,570 --> 00:13:15,800 And what is this squared distance to 236 00:13:15,800 --> 00:13:17,330 any decision region? 237 00:13:17,330 --> 00:13:19,935 It's 2 alpha squared, right? 238 00:13:23,190 --> 00:13:26,560 So basically, the probability of making an error is the 239 00:13:26,560 --> 00:13:31,140 probability that the noise variable has a magnitude in 240 00:13:31,140 --> 00:13:34,980 this dimension, and in this direction, greater than the 241 00:13:34,980 --> 00:13:38,170 square root of 2 alpha squared. 242 00:13:38,170 --> 00:13:42,320 You've been through all those calculation several times now. 243 00:13:42,320 --> 00:13:42,710 All right. 244 00:13:42,710 --> 00:13:47,710 So this is -- if I keep reliability info -- in other 245 00:13:47,710 --> 00:13:53,150 words, if I keep the full received signal -- if I 246 00:13:53,150 --> 00:14:02,200 discard reliability, then basically what can I get out? 247 00:14:02,200 --> 00:14:07,760 Then my y, as I say, is in 0, 0, 0, 1, 1, 0, 1, 1. 248 00:14:10,590 --> 00:14:14,000 There are only four possible things I can see in two 249 00:14:14,000 --> 00:14:15,910 transmissions through this channel. 250 00:14:15,910 --> 00:14:19,650 They're all binary two-tuples. 251 00:14:19,650 --> 00:14:22,420 And what does that mean? 252 00:14:22,420 --> 00:14:29,870 That means if my actual r is in this quadrant, then I'm 253 00:14:29,870 --> 00:14:34,510 going to make a hard decision of 0 and 0 both times. 254 00:14:34,510 --> 00:14:37,280 So I'm going to say I'm in this quadrant. 255 00:14:37,280 --> 00:14:41,480 And likewise, I'm basically going to decide which of these 256 00:14:41,480 --> 00:14:44,230 four quadrants I'm in, and that's all the information the 257 00:14:44,230 --> 00:14:45,480 decoder is going to have. 258 00:14:48,010 --> 00:14:51,370 So here I am at this point here. 259 00:14:51,370 --> 00:14:55,200 Now the decoder simply knows, via these two bits, which 260 00:14:55,200 --> 00:14:57,980 quadrant you're in. 261 00:14:57,980 --> 00:14:59,800 Now, it has to decide which of these two 262 00:14:59,800 --> 00:15:01,960 code words were sent. 263 00:15:01,960 --> 00:15:06,040 What's its maximum likelihood decision rule given just this 264 00:15:06,040 --> 00:15:07,290 information? 265 00:15:11,710 --> 00:15:13,646 AUDIENCE: 0, 0, [INAUDIBLE] 266 00:15:13,646 --> 00:15:15,098 0, 0, [INAUDIBLE] 267 00:15:15,098 --> 00:15:17,276 1, 1, 1, 1, 1. 268 00:15:17,276 --> 00:15:17,518 And in the other case, any one. 269 00:15:17,518 --> 00:15:18,680 PROFESSOR: OK. 270 00:15:18,680 --> 00:15:20,310 That's correct. 271 00:15:20,310 --> 00:15:22,550 So clearly, if you land in this quadrant, 272 00:15:22,550 --> 00:15:28,200 you decide 0, 0. 273 00:15:28,200 --> 00:15:30,570 Now, if you land down here, you decide 274 00:15:30,570 --> 00:15:34,190 1, 1 in this quadrant. 275 00:15:34,190 --> 00:15:37,750 And what am I going to do here or here? 276 00:15:37,750 --> 00:15:40,360 Actually, here the evidence is totally balanced. 277 00:15:40,360 --> 00:15:42,370 I have nothing that tells me whether to 278 00:15:42,370 --> 00:15:44,530 go one way or another. 279 00:15:44,530 --> 00:15:46,990 I could make an arbitrary decision in the hope of 280 00:15:46,990 --> 00:15:48,315 minimizing my error probability. 281 00:15:48,315 --> 00:15:50,970 Flip a coin. 282 00:15:50,970 --> 00:15:56,496 But whichever decision I make, what I'm going to find is that 283 00:15:56,496 --> 00:16:02,620 there's a probability of error that -- there's a certain 284 00:16:02,620 --> 00:16:06,320 noise, namely the noise that takes me from here to this 285 00:16:06,320 --> 00:16:09,130 decision boundary that is going to cause an error. 286 00:16:09,130 --> 00:16:11,630 Or if I decide the other way, it would be the one that goes 287 00:16:11,630 --> 00:16:13,780 from here to here. 288 00:16:13,780 --> 00:16:14,920 So I'm going to be stuck. 289 00:16:14,920 --> 00:16:20,440 There are going to be certain noise variables of length, 290 00:16:20,440 --> 00:16:26,790 now, merely alpha, or square distance alpha squared, that 291 00:16:26,790 --> 00:16:30,290 are going to cause me to make a decision error, ultimately. 292 00:16:30,290 --> 00:16:36,710 Regardless of how I set up this final block here. 293 00:16:36,710 --> 00:16:38,871 Decoder. 294 00:16:38,871 --> 00:16:42,650 Whatever rule I give to this decoder, it's only going to 295 00:16:42,650 --> 00:16:48,730 take, worst case, a noise of squared magnitude alpha 296 00:16:48,730 --> 00:16:51,340 squared to cause an error. 297 00:16:51,340 --> 00:16:52,590 AUDIENCE: [UNINTELLIGIBLE PHRASE] 298 00:16:55,561 --> 00:16:56,830 PROFESSOR: That's right. 299 00:16:56,830 --> 00:16:59,420 I shouldn't have thrown this away. 300 00:16:59,420 --> 00:17:03,250 That's the elementary point I'm trying to make here. 301 00:17:03,250 --> 00:17:03,840 All right. 302 00:17:03,840 --> 00:17:06,640 AUDIENCE: [UNINTELLIGIBLE PHRASE] 303 00:17:06,640 --> 00:17:10,432 the distance, d_min is different, essentially. 304 00:17:10,432 --> 00:17:11,869 I mean, we can put it that way. 305 00:17:11,869 --> 00:17:16,020 PROFESSOR: That's where I'm going, is that the effective 306 00:17:16,020 --> 00:17:18,690 minimum squared distance here is 2 alpha squared. 307 00:17:18,690 --> 00:17:21,530 And this is what shows up in our union-bound estimate in 308 00:17:21,530 --> 00:17:24,250 all of our -- 309 00:17:24,250 --> 00:17:26,410 we do this kind of minimum distance decoding. 310 00:17:26,410 --> 00:17:30,080 But if I throw away this important information, then my 311 00:17:30,080 --> 00:17:31,745 effective minimum squared distance 312 00:17:31,745 --> 00:17:32,980 is only alpha squared. 313 00:17:32,980 --> 00:17:35,490 That is the bottom line here. 314 00:17:35,490 --> 00:17:40,490 In dB terms, how much of a cost is that? 315 00:17:40,490 --> 00:17:41,630 3 dB, right? 316 00:17:41,630 --> 00:17:46,000 So I've cost myself a factor of 2 in noise margin. 317 00:17:46,000 --> 00:17:48,200 So 3 dB loss. 318 00:17:52,490 --> 00:18:00,720 Because of lack of time, I won't go through the argument 319 00:18:00,720 --> 00:18:05,810 in the notes, which shows that, in fact, exactly the 320 00:18:05,810 --> 00:18:08,160 same thing occurs for any -- 321 00:18:08,160 --> 00:18:11,220 whenever the minimum Hamming distance is even here in this 322 00:18:11,220 --> 00:18:15,770 code that I started from, you lose precisely 3 dB. 323 00:18:15,770 --> 00:18:18,850 It's not quite as clean an argument when the minimum 324 00:18:18,850 --> 00:18:20,460 Hamming distance is odd. 325 00:18:20,460 --> 00:18:24,790 Then you lose up to 3 dB, and it goes to 3 dB as the 326 00:18:24,790 --> 00:18:26,580 distance increases. 327 00:18:26,580 --> 00:18:31,030 But there's a very, pretty elementary geometric argument 328 00:18:31,030 --> 00:18:35,010 that hard decisions, again, cost you 3 dB loss, which is 329 00:18:35,010 --> 00:18:36,810 consistent with what the capacity 330 00:18:36,810 --> 00:18:38,410 calculation gives you. 331 00:18:41,780 --> 00:18:42,320 All right. 332 00:18:42,320 --> 00:18:47,440 Well, obviously the reason we did this is we were trying to 333 00:18:47,440 --> 00:18:49,690 simplify things. 334 00:18:49,690 --> 00:18:54,840 What would be the first step to unsimplify things and get 335 00:18:54,840 --> 00:18:56,170 some of this loss back? 336 00:19:03,200 --> 00:19:07,060 Let me suggest that what this amounts to is a two-level 337 00:19:07,060 --> 00:19:12,240 quantization of the received real number rk. 338 00:19:12,240 --> 00:19:15,440 What's the next number higher than 2? 339 00:19:15,440 --> 00:19:15,920 Integer. 340 00:19:15,920 --> 00:19:17,340 AUDIENCE: 3. 341 00:19:17,340 --> 00:19:18,550 PROFESSOR: 3. 342 00:19:18,550 --> 00:19:19,260 All right. 343 00:19:19,260 --> 00:19:23,860 How about a 3 level quantization here? 344 00:19:23,860 --> 00:19:25,735 OK. 345 00:19:25,735 --> 00:19:31,880 So what we're talking about is a highly quantized magnitude 346 00:19:31,880 --> 00:19:41,480 where instead of just making a decision boundary here, which 347 00:19:41,480 --> 00:19:45,350 is effectively what we did for 2 level quantization, let's 348 00:19:45,350 --> 00:19:53,620 make some null zone here between some threshold plus t 349 00:19:53,620 --> 00:19:57,550 and some threshold minus t. 350 00:19:57,550 --> 00:20:02,920 And we'll say, in this region, the received symbol is 351 00:20:02,920 --> 00:20:05,370 unreliable. 352 00:20:05,370 --> 00:20:05,810 OK. 353 00:20:05,810 --> 00:20:07,870 That's called an erasure. 354 00:20:07,870 --> 00:20:11,640 So we make a quantized magnitude where rk -- 355 00:20:18,700 --> 00:20:21,490 doesn't fit so neatly here. 356 00:20:21,490 --> 00:20:26,020 Let me just say this quantized magnitude is going to give me 357 00:20:26,020 --> 00:20:29,350 out something which is either going to be minus 1, 358 00:20:29,350 --> 00:20:31,300 a 0, or plus 1. 359 00:20:31,300 --> 00:20:37,440 It's going to be three levels, this is minus 1, 0, plus 1. 360 00:20:37,440 --> 00:20:40,850 Or I don't want the minus 1 there. 361 00:20:40,850 --> 00:20:44,817 The reliability is either a fixed reliability, or it's 0. 362 00:20:44,817 --> 00:20:46,067 OK. 363 00:20:51,870 --> 00:20:56,660 So now I've got a channel model that looks like this. 364 00:20:56,660 --> 00:21:02,690 If you work it out, it's called the binary erasure 365 00:21:02,690 --> 00:21:04,380 channel with errors. 366 00:21:04,380 --> 00:21:06,210 All transitions are possible. 367 00:21:06,210 --> 00:21:08,010 I can send a 0,1. 368 00:21:08,010 --> 00:21:13,500 I can receive, let's call it, a 0, a 1, or a question mark 369 00:21:13,500 --> 00:21:15,465 for -- this is called an erasure. 370 00:21:18,650 --> 00:21:23,540 And this is 1 minus p minus q, and this is p and this is q 371 00:21:23,540 --> 00:21:30,080 symmetrically, for some p and q, which you can evaluate. 372 00:21:30,080 --> 00:21:35,440 And let's choose this threshold t to optimize things 373 00:21:35,440 --> 00:21:37,920 and get the maximum capacity. 374 00:21:37,920 --> 00:21:40,530 Now if you do the capacity calculation, you'll find 375 00:21:40,530 --> 00:21:44,510 there's only 1 to 1.5 dB loss. 376 00:21:44,510 --> 00:21:47,530 Again, depending on the signal to noise ratio. 377 00:21:47,530 --> 00:21:50,320 So in effect, you've already bought back, just with this 378 00:21:50,320 --> 00:21:58,970 very simple method, half of the loss that you inflicted on 379 00:21:58,970 --> 00:22:01,840 yourself by making hard decisions. 380 00:22:01,840 --> 00:22:08,930 So making erasures is a good first step towards correcting 381 00:22:08,930 --> 00:22:11,770 this problem. 382 00:22:11,770 --> 00:22:15,520 And let's think about it again for this simple code. 383 00:22:15,520 --> 00:22:22,780 Suppose I establish these thresholds at 384 00:22:22,780 --> 00:22:24,380 plus t and minus t. 385 00:22:24,380 --> 00:22:27,440 Plus t and minus t. 386 00:22:27,440 --> 00:22:33,890 So now when I consider the problem that the decoder has, 387 00:22:33,890 --> 00:22:38,300 I have nine possible regions, all right, each with three 388 00:22:38,300 --> 00:22:44,370 possible decisions in two dimensions. 389 00:22:44,370 --> 00:22:47,150 And so what's my decision rule going to be now? 390 00:22:50,820 --> 00:22:55,050 For this 0, 0 decision, I'm going to include, of course, 391 00:22:55,050 --> 00:22:56,290 this region. 392 00:22:56,290 --> 00:23:01,750 But now if I land in this region, that means one of the 393 00:23:01,750 --> 00:23:04,210 received symbols was erased, but the other one gave me a 394 00:23:04,210 --> 00:23:05,690 definite indication. 395 00:23:05,690 --> 00:23:09,020 So the weight of evidence still goes up here. 396 00:23:09,020 --> 00:23:13,300 And similarly, like so. 397 00:23:13,300 --> 00:23:16,380 So my decision region for 0 -- 398 00:23:16,380 --> 00:23:20,740 definitely for 0, 0 is this region. 399 00:23:20,740 --> 00:23:26,900 And my decision region definitely for 1, 1 is the 400 00:23:26,900 --> 00:23:29,750 symmetric region. 401 00:23:29,750 --> 00:23:32,590 And, of course, I still have three regions where I really 402 00:23:32,590 --> 00:23:33,520 can't say anything. 403 00:23:33,520 --> 00:23:35,360 This is two erasures. 404 00:23:35,360 --> 00:23:36,720 This is 0, 1 -- 405 00:23:36,720 --> 00:23:38,560 two conflicting pieces of evidence. 406 00:23:38,560 --> 00:23:39,660 This is 1, 0 -- two 407 00:23:39,660 --> 00:23:42,360 conflicting pieces of evidence. 408 00:23:42,360 --> 00:23:48,130 So I still have to flip a coin out in here. 409 00:23:48,130 --> 00:23:53,550 But how I improve things, that's measured by what's the 410 00:23:53,550 --> 00:23:54,880 minimum distance to -- 411 00:23:57,820 --> 00:24:01,770 what's the minimum size of error it takes to make a 412 00:24:01,770 --> 00:24:03,450 decision error. 413 00:24:03,450 --> 00:24:08,550 And that's going to be either this length or this length. 414 00:24:08,550 --> 00:24:09,400 You see that? 415 00:24:09,400 --> 00:24:11,125 Just pretty clear, intuitively? 416 00:24:14,220 --> 00:24:18,040 So the game here is we first of all choose t so that these 417 00:24:18,040 --> 00:24:21,110 two lengths are the same. 418 00:24:21,110 --> 00:24:23,750 That's done by squeezing t. 419 00:24:23,750 --> 00:24:25,935 You see these go in opposite directions as t 420 00:24:25,935 --> 00:24:27,540 is raised or lowered. 421 00:24:27,540 --> 00:24:31,100 So we find the t such that these two are the same. 422 00:24:31,100 --> 00:24:37,050 Having equalized t, we find some value for the effective 423 00:24:37,050 --> 00:24:40,000 minimum squared distance, which is between alpha squared 424 00:24:40,000 --> 00:24:41,800 and 2 alpha squared. 425 00:24:41,800 --> 00:24:50,310 As I remember, this somehow gains about 1.5 dB. 426 00:24:50,310 --> 00:24:51,390 The moral is about the same. 427 00:24:51,390 --> 00:24:55,880 You can get about half of your loss back by using erasures. 428 00:24:55,880 --> 00:24:59,990 And again, this holds for any code which has an even minimum 429 00:24:59,990 --> 00:25:02,060 Hamming distance, if you can accept 430 00:25:02,060 --> 00:25:04,640 this method of analysis. 431 00:25:04,640 --> 00:25:05,190 All right. 432 00:25:05,190 --> 00:25:10,870 So a first step, even if you want to stay in the binary 433 00:25:10,870 --> 00:25:14,830 world, is to allow yourself to use erasures as well. 434 00:25:14,830 --> 00:25:20,170 That will claw back half of the loss that you might have 435 00:25:20,170 --> 00:25:22,240 incurred by using hard decisions. 436 00:25:22,240 --> 00:25:25,520 And, of course, even better would be to use a more highly 437 00:25:25,520 --> 00:25:29,500 quantized reliability and somehow have a decoding 438 00:25:29,500 --> 00:25:33,420 algorithm that can use soft decisions. 439 00:25:33,420 --> 00:25:37,180 Soft decisions are decisions that have reliability metrics 440 00:25:37,180 --> 00:25:39,900 attached to them. 441 00:25:39,900 --> 00:25:43,770 And in the early days of coding, 442 00:25:43,770 --> 00:25:46,190 people evaluated capacity. 443 00:25:46,190 --> 00:25:48,420 They evaluated this sort of thing. 444 00:25:48,420 --> 00:25:55,780 And it was pretty generally agreed that eight level 445 00:25:55,780 --> 00:25:59,090 quantization was going to be practically good enough. 446 00:25:59,090 --> 00:26:00,950 16 levels, certainly good enough. 447 00:26:00,950 --> 00:26:05,070 Nowadays you typically go up to 64 levels. 448 00:26:05,070 --> 00:26:07,130 You might go up to much higher numbers if you have little 449 00:26:07,130 --> 00:26:09,050 other things to worry about like timing 450 00:26:09,050 --> 00:26:10,760 recovery and so forth. 451 00:26:10,760 --> 00:26:14,620 But if you're purely in a synchronized, symbol by symbol 452 00:26:14,620 --> 00:26:19,150 transmission, then three or four bits of reliability 453 00:26:19,150 --> 00:26:21,210 information are going to be enough. 454 00:26:21,210 --> 00:26:24,510 So from an engineering point of view, that's 455 00:26:24,510 --> 00:26:25,430 a good way to go. 456 00:26:25,430 --> 00:26:28,820 Now, of course, you're going to need a decoding algorithm 457 00:26:28,820 --> 00:26:32,440 that can use soft decisions. 458 00:26:32,440 --> 00:26:36,030 So as we go along, I'm going to talk about error correcting 459 00:26:36,030 --> 00:26:41,200 decoding algorithms that are not much, because correcting 460 00:26:41,200 --> 00:26:43,830 errors only is a very bad thing to do here. 461 00:26:43,830 --> 00:26:44,990 Errors and erasure. 462 00:26:44,990 --> 00:26:48,420 Correcting decoding algorithms, which are fairly 463 00:26:48,420 --> 00:26:52,850 easy in algebraic code context. 464 00:26:52,850 --> 00:26:57,170 And then finally soft decision decoding algorithms, which is 465 00:26:57,170 --> 00:26:59,290 the kind we really want on this channel. 466 00:26:59,290 --> 00:27:01,210 We really want to use the soft decisions. 467 00:27:01,210 --> 00:27:04,070 We can't afford these huge losses. 468 00:27:04,070 --> 00:27:08,710 We can't even afford 1 to 1 and a 1/2 dB loss. 469 00:27:08,710 --> 00:27:14,690 OK, so there's more set on this in the last part of 470 00:27:14,690 --> 00:27:16,670 chapter six, but that's all the time I want to 471 00:27:16,670 --> 00:27:18,650 spend on it in class. 472 00:27:18,650 --> 00:27:20,100 Does anyone have any questions? 473 00:27:20,100 --> 00:27:21,080 yes? 474 00:27:21,080 --> 00:27:22,255 AUDIENCE: [INAUDIBLE] 475 00:27:22,255 --> 00:27:24,626 transformation before we do the quantization, can we 476 00:27:24,626 --> 00:27:26,754 change the shape of the [UNINTELLIGIBLE] 477 00:27:26,754 --> 00:27:30,602 regions such that we can further improve the 478 00:27:30,602 --> 00:27:33,007 [INAUDIBLE PHRASE] 479 00:27:33,007 --> 00:27:35,412 improve the decoding? 480 00:27:35,412 --> 00:27:38,500 PROFESSOR: Well, that's an interesting idea. 481 00:27:38,500 --> 00:27:41,880 But think about it from an information 482 00:27:41,880 --> 00:27:43,570 theoretic point of view. 483 00:27:43,570 --> 00:27:48,560 Can you actually get more information by introducing a 484 00:27:48,560 --> 00:27:51,140 one-to-one transformation, whether it's linear or 485 00:27:51,140 --> 00:27:53,011 nonlinear, whatever? 486 00:27:53,011 --> 00:27:58,810 AUDIENCE: The point is, the original quantization is based 487 00:27:58,810 --> 00:28:02,926 on a straight line, [INAUDIBLE PHRASE]. 488 00:28:02,926 --> 00:28:04,176 [UNINTELLIGIBLE PHRASE]. 489 00:28:06,200 --> 00:28:08,640 PROFESSOR: You want to put a curved line on this plane? 490 00:28:08,640 --> 00:28:09,842 AUDIENCE: [INAUDIBLE]. 491 00:28:09,842 --> 00:28:12,190 PROFESSOR: OK. 492 00:28:12,190 --> 00:28:14,850 That's going to imply decisions that are not symbol 493 00:28:14,850 --> 00:28:17,050 by symbol decisions. 494 00:28:17,050 --> 00:28:22,200 That's going to imply that you need to save r1 and r 2 in 495 00:28:22,200 --> 00:28:26,220 order even to just decide where you're in this and then 496 00:28:26,220 --> 00:28:30,250 go through some kind of quantization on the plane. 497 00:28:30,250 --> 00:28:33,170 And once you've got r1 and r2 and you're trying to draw 498 00:28:33,170 --> 00:28:35,610 crazy boundaries, I suggest it's going to be simpler to 499 00:28:35,610 --> 00:28:38,990 just compute the Euclidean distance to each of these 500 00:28:38,990 --> 00:28:48,922 points, which results in this decision region. 501 00:28:48,922 --> 00:28:50,172 AUDIENCE: [INAUDIBLE PHRASE]. 502 00:28:55,880 --> 00:28:56,770 PROFESSOR: OK. 503 00:28:56,770 --> 00:28:59,660 I don't see the rationale for it yet, but there's been a 504 00:28:59,660 --> 00:29:01,570 million innovations in this business, and you 505 00:29:01,570 --> 00:29:02,820 might have a good idea. 506 00:29:09,060 --> 00:29:11,170 Any other comments? 507 00:29:11,170 --> 00:29:15,242 I really appreciate comments that go anywhere. 508 00:29:15,242 --> 00:29:17,590 I like to address them now when they're ripe. 509 00:29:21,950 --> 00:29:23,200 OK. 510 00:29:25,570 --> 00:29:28,520 So let's go on to chapter seven and eight. 511 00:29:28,520 --> 00:29:29,770 [UNINTELLIGIBLE] here. 512 00:29:35,660 --> 00:29:41,770 Chapters seven and eight are closely related. 513 00:29:41,770 --> 00:29:45,565 This is really my bow to algebraic coding theory. 514 00:29:49,580 --> 00:29:58,800 I expect that if any of you have had a close course in 515 00:29:58,800 --> 00:30:01,370 coding before, that it was primarily on 516 00:30:01,370 --> 00:30:04,336 algebraic coding theory. 517 00:30:04,336 --> 00:30:06,770 How many of you have had a course in 518 00:30:06,770 --> 00:30:09,210 coding theory before? 519 00:30:09,210 --> 00:30:10,980 One, two, three. 520 00:30:10,980 --> 00:30:11,960 Not so many. 521 00:30:11,960 --> 00:30:14,680 Was it on algebraic coding theory? 522 00:30:14,680 --> 00:30:15,120 Yes? 523 00:30:15,120 --> 00:30:16,390 AUDIENCE: I [UNINTELLIGIBLE] two, one 524 00:30:16,390 --> 00:30:17,731 engineering and one algebraic. 525 00:30:17,731 --> 00:30:18,450 PROFESSOR: Excuse me? 526 00:30:18,450 --> 00:30:21,235 AUDIENCE: Two -- one for -- actually, two algebraic and 527 00:30:21,235 --> 00:30:23,420 one out of kind of this stuff. 528 00:30:23,420 --> 00:30:24,440 PROFESSOR: OK. 529 00:30:24,440 --> 00:30:26,140 Well, you're a ringer then. 530 00:30:26,140 --> 00:30:27,510 What was the one on this stuff? 531 00:30:27,510 --> 00:30:29,106 AUDIENCE: It was out of Wicker's book. 532 00:30:29,106 --> 00:30:29,760 PROFESSOR: Out of? 533 00:30:29,760 --> 00:30:30,620 AUDIENCE: Wicker's book. 534 00:30:30,620 --> 00:30:32,764 PROFESSOR: Wicker's book on turbo codes. 535 00:30:32,764 --> 00:30:34,862 AUDIENCE: No, the data storage one. 536 00:30:34,862 --> 00:30:36,180 PROFESSOR: Data storage. 537 00:30:36,180 --> 00:30:37,150 OK. 538 00:30:37,150 --> 00:30:39,800 I don't know that book. 539 00:30:39,800 --> 00:30:43,640 But it talks about sophisticated soft decision 540 00:30:43,640 --> 00:30:45,210 type coding, and so forth. 541 00:30:45,210 --> 00:30:46,460 What was the course you took? 542 00:30:46,460 --> 00:30:47,310 AUDIENCE: Algebra. 543 00:30:47,310 --> 00:30:49,330 PROFESSOR: Algebraic. 544 00:30:49,330 --> 00:30:50,580 OK. 545 00:30:52,160 --> 00:30:52,500 All right. 546 00:30:52,500 --> 00:30:58,910 Well, let me just make a broad brush comment, which may or 547 00:30:58,910 --> 00:31:00,910 may not be supported by your previous 548 00:31:00,910 --> 00:31:02,130 exposure to coding theory. 549 00:31:02,130 --> 00:31:08,280 Which is that for many, many years, when people said coding 550 00:31:08,280 --> 00:31:11,910 theory, they meant algebraic coding theory, coding theory 551 00:31:11,910 --> 00:31:14,800 of block codes that are constructed by algebraic 552 00:31:14,800 --> 00:31:18,100 techniques like Reed-Muller codes, like Reed-Solomon 553 00:31:18,100 --> 00:31:22,320 codes, BCH codes, cyclic codes. 554 00:31:22,320 --> 00:31:25,900 And if you said, I want to leaRn some coding theory, 555 00:31:25,900 --> 00:31:29,210 asked your graduate student to go buy a textbook for you, 556 00:31:29,210 --> 00:31:30,450 it's probably going to be a textbook on 557 00:31:30,450 --> 00:31:33,170 algebraic coding theory. 558 00:31:33,170 --> 00:31:38,520 Meanwhile, there was a bunch of us who were off actually 559 00:31:38,520 --> 00:31:42,830 trying to construct codes for real channels. 560 00:31:42,830 --> 00:31:48,130 And conceRning with really how complex this is to implement, 561 00:31:48,130 --> 00:31:51,540 and what kind of performance can we really get. 562 00:31:51,540 --> 00:31:56,640 And we hardly ever used algebraic codes. 563 00:31:56,640 --> 00:31:59,580 Initially, we used convolutional codes with 564 00:31:59,580 --> 00:32:02,970 various kinds of decoding algorithms, threshold 565 00:32:02,970 --> 00:32:08,000 decoding, sequential decoding, the Viterbi algorithm. 566 00:32:08,000 --> 00:32:12,520 And these get more and more elaborate, and then finally 567 00:32:12,520 --> 00:32:17,530 the big step, well, the big step was to capacity 568 00:32:17,530 --> 00:32:18,630 approaching codes. 569 00:32:18,630 --> 00:32:24,110 Which now people are starting to call modeRn coding theory. 570 00:32:24,110 --> 00:32:26,710 Long, random-like codes that have an 571 00:32:26,710 --> 00:32:28,820 iterative decoding algorithm. 572 00:32:28,820 --> 00:32:30,180 And that's what we'll get to towards 573 00:32:30,180 --> 00:32:31,740 the end of this course. 574 00:32:31,740 --> 00:32:38,050 So from an engineers point of view, to some extent, of all 575 00:32:38,050 --> 00:32:42,810 this work on n k d and algebraic decoding algorithms 576 00:32:42,810 --> 00:32:46,675 and so forth was a massive distraction. 577 00:32:46,675 --> 00:32:49,530 It really missed the point. 578 00:32:49,530 --> 00:32:53,510 Sometimes because it assumed hard decisions. 579 00:32:53,510 --> 00:32:56,830 Or after assuming hard decisions, it assumed bounded 580 00:32:56,830 --> 00:32:59,140 distance decoding algorithms. 581 00:32:59,140 --> 00:33:01,170 But it was all -- 582 00:33:01,170 --> 00:33:04,630 from Shannon's perspective, it was too deterministic, too 583 00:33:04,630 --> 00:33:06,530 constructed, too structured. 584 00:33:06,530 --> 00:33:11,260 You want more random elements in your coding scheme. 585 00:33:11,260 --> 00:33:17,700 However, two things: one, this theory is a beautiful theory, 586 00:33:17,700 --> 00:33:21,480 both the mathematical theory of finite fields and the 587 00:33:21,480 --> 00:33:27,770 coding theory, particularly of Reed-Solomon codes, which are 588 00:33:27,770 --> 00:33:30,280 uniquely the greatest accomplishment of algebraic 589 00:33:30,280 --> 00:33:35,220 coding theory, and which have proved to be very useful. 590 00:33:35,220 --> 00:33:44,140 And B, Reed-Solomon codes are something that as engineers, 591 00:33:44,140 --> 00:33:47,210 ought to be part of your tool kit, and you will find very 592 00:33:47,210 --> 00:33:49,550 useful a variety of situations. 593 00:33:49,550 --> 00:33:55,360 So the objective of this part of the course is, within some 594 00:33:55,360 --> 00:33:58,680 proportion within the overall scheme of the course, to give 595 00:33:58,680 --> 00:34:02,430 you exposure to this lovely theory. 596 00:34:02,430 --> 00:34:07,470 You can leaRn all about it just by reading a book or two. 597 00:34:07,470 --> 00:34:10,750 Probably less than one book. 598 00:34:10,750 --> 00:34:13,310 And to give you some exposure to Reed-Solomon codes, which 599 00:34:13,310 --> 00:34:15,170 have been used in practice. 600 00:34:19,909 --> 00:34:24,120 For more than 20 years, the deep space standard was a 601 00:34:24,120 --> 00:34:25,370 concatenated code. 602 00:34:25,370 --> 00:34:29,770 That means a sequence of two codes, of which the inner code 603 00:34:29,770 --> 00:34:32,520 was a convolutional code decoded by the Viterbi 604 00:34:32,520 --> 00:34:35,320 algorithm, which we'll talk about after this. 605 00:34:35,320 --> 00:34:38,989 And then the outer code, the code that cleans up errors 606 00:34:38,989 --> 00:34:44,530 made by the inner code, was a Reed-Solomon code of length 607 00:34:44,530 --> 00:34:49,219 255 over the finite field with 256 elements. 608 00:34:49,219 --> 00:34:54,610 And that's the dynamite combination that dominated the 609 00:34:54,610 --> 00:34:57,760 power-limited coding world for at least 20 years. 610 00:34:57,760 --> 00:34:59,840 So you ought to know, as a minimum, about 611 00:34:59,840 --> 00:35:01,500 Reed-Solomon codes. 612 00:35:01,500 --> 00:35:05,040 So that's my objective in going through 613 00:35:05,040 --> 00:35:06,460 these next two chapters. 614 00:35:06,460 --> 00:35:10,660 However, because these have not basically been on the 615 00:35:10,660 --> 00:35:16,050 winning path to get to the Shannon limit, I'm trying to 616 00:35:16,050 --> 00:35:19,400 limit it to three weeks max. 617 00:35:19,400 --> 00:35:23,190 And therefore, that means that the presentation is going to 618 00:35:23,190 --> 00:35:27,040 be a lot faster than what you're used to. 619 00:35:27,040 --> 00:35:30,390 I'll probably do as much in these three weeks as most 620 00:35:30,390 --> 00:35:32,390 people do in a term in an entire algebraic 621 00:35:32,390 --> 00:35:34,750 coding theory course. 622 00:35:34,750 --> 00:35:36,990 That's an exaggeration -- 623 00:35:36,990 --> 00:35:39,810 people do a lot more -- but I try to at least hit all the 624 00:35:39,810 --> 00:35:41,190 main points that you need to know. 625 00:35:41,190 --> 00:35:46,000 So I apologize in advance if it seems that now we're on a 626 00:35:46,000 --> 00:35:48,790 very fast moving train. 627 00:35:48,790 --> 00:35:50,450 But you ought to know this stuff. 628 00:35:50,450 --> 00:35:52,600 You ought not to spend too much time on it. 629 00:35:52,600 --> 00:35:56,670 I'm trying to reconcile those two points of view. 630 00:35:56,670 --> 00:35:59,930 Any questions or comments? 631 00:35:59,930 --> 00:36:00,290 All right. 632 00:36:00,290 --> 00:36:01,540 Let's go. 633 00:36:03,910 --> 00:36:07,240 Chapter seven. 634 00:36:07,240 --> 00:36:13,865 We go through a series of algebraic objects. 635 00:36:16,480 --> 00:36:21,100 First, the integers, then groups. 636 00:36:21,100 --> 00:36:23,850 You certainly ought to know -- you do know about integers. 637 00:36:23,850 --> 00:36:26,460 You ought to know about groups, at least a little bit. 638 00:36:28,960 --> 00:36:29,580 Let's see. 639 00:36:29,580 --> 00:36:30,510 What's next? 640 00:36:30,510 --> 00:36:35,415 Then fields, particularly finite. 641 00:36:39,860 --> 00:36:47,670 Then polynomials over fields, particularly finite. 642 00:36:47,670 --> 00:36:54,130 And then finally, from this we actually get to construct 643 00:36:54,130 --> 00:36:55,380 finite fields. 644 00:36:58,180 --> 00:37:03,180 And these are all things that you'll encounter again and 645 00:37:03,180 --> 00:37:05,590 again, have encountered. 646 00:37:05,590 --> 00:37:07,660 I think you've probably encountered everything except 647 00:37:07,660 --> 00:37:10,810 possibly for groups and finite fields. 648 00:37:10,810 --> 00:37:13,340 You certainly encountered polynomials over real and 649 00:37:13,340 --> 00:37:16,410 complex fields. 650 00:37:16,410 --> 00:37:19,890 And so in some sense, this is a broadening of what you 651 00:37:19,890 --> 00:37:21,140 already know. 652 00:37:26,720 --> 00:37:32,600 So I assume we start with the integers for two reasons. 653 00:37:36,020 --> 00:37:39,820 One is you already know about the factorization properties 654 00:37:39,820 --> 00:37:42,250 of the integers. 655 00:37:42,250 --> 00:37:43,020 We're going to -- 656 00:37:43,020 --> 00:37:46,990 that leads to a little bit of number theory, which we're 657 00:37:46,990 --> 00:37:52,250 going to use when we talk about groups, finite groups, 658 00:37:52,250 --> 00:37:54,640 cyclic groups. 659 00:37:54,640 --> 00:37:56,600 So this is just to remind you, but it's 660 00:37:56,600 --> 00:37:59,440 also to put up a template. 661 00:37:59,440 --> 00:38:03,030 And we're going to find the algebraic properties of 662 00:38:03,030 --> 00:38:05,830 polynomials are very, very similar to 663 00:38:05,830 --> 00:38:06,890 those of the integers. 664 00:38:06,890 --> 00:38:08,820 This is because they're both rings. 665 00:38:08,820 --> 00:38:11,420 They're both in fact principal ideal domains. 666 00:38:11,420 --> 00:38:12,690 They're Euclidean domains. 667 00:38:12,690 --> 00:38:15,670 They both have a Euclidean division algorithm as their 668 00:38:15,670 --> 00:38:18,230 key mathematical property. 669 00:38:18,230 --> 00:38:21,650 And so pretty much any time you want to develop a result 670 00:38:21,650 --> 00:38:26,680 about the polynomials, you can start from the comparable 671 00:38:26,680 --> 00:38:29,200 results that you know about the integers and just change 672 00:38:29,200 --> 00:38:30,460 the notation. 673 00:38:30,460 --> 00:38:36,320 And you'll have a sketch of the proof for polynomials. 674 00:38:36,320 --> 00:38:40,990 So it's important to remind ourselves about 675 00:38:40,990 --> 00:38:43,590 the integers first. 676 00:38:43,590 --> 00:38:46,050 Now, when I say the factorization properties of 677 00:38:46,050 --> 00:38:48,310 the integers, I'm not talking about anything more 678 00:38:48,310 --> 00:38:51,400 complicated than what you leaRned in grade school. 679 00:38:51,400 --> 00:38:55,120 I just want to introduce a few terms. 680 00:38:55,120 --> 00:39:00,670 We have the notion of divisors. 681 00:39:00,670 --> 00:39:09,685 a divides b means that a times some quotient equals b. 682 00:39:09,685 --> 00:39:16,400 I could be talking about plus or minus, positive or negative 683 00:39:16,400 --> 00:39:17,340 integers here. 684 00:39:17,340 --> 00:39:18,590 Everything divides 0. 685 00:39:21,300 --> 00:39:22,120 OK. 686 00:39:22,120 --> 00:39:29,620 We have the idea of units, which are the invertible 687 00:39:29,620 --> 00:39:33,140 integers under multiplication. 688 00:39:33,140 --> 00:39:34,390 And what are those? 689 00:39:38,460 --> 00:39:41,600 Which integers have a multiplicative inverse? 690 00:39:44,980 --> 00:39:45,340 [UNINTELLIGIBLE] 691 00:39:45,340 --> 00:39:45,970 any of them? 692 00:39:45,970 --> 00:39:47,802 AUDIENCE: [INAUDIBLE PHRASE]. 693 00:39:47,802 --> 00:39:49,170 PROFESSOR: [INAUDIBLE]. 694 00:39:49,170 --> 00:39:50,420 All right. 695 00:39:52,220 --> 00:39:57,530 That's a little exercise that shows you that, in general, 696 00:39:57,530 --> 00:40:01,870 the invertible elements of any set that has an operation like 697 00:40:01,870 --> 00:40:06,190 multiplication form a group. 698 00:40:06,190 --> 00:40:09,750 [INAUDIBLE] group thing, these clearly form a group under 699 00:40:09,750 --> 00:40:11,320 multiplication that's isomorphic 700 00:40:11,320 --> 00:40:12,725 to the binary group. 701 00:40:12,725 --> 00:40:14,780 Well, in fact, we've already been using it -- 702 00:40:14,780 --> 00:40:16,380 F2. 703 00:40:16,380 --> 00:40:16,830 OK. 704 00:40:16,830 --> 00:40:21,620 So we have units, and whenever we talk about the divisors, 705 00:40:21,620 --> 00:40:24,420 factorization, primes, so forth, 706 00:40:24,420 --> 00:40:25,700 the units are a nuisance. 707 00:40:25,700 --> 00:40:29,790 Because we can always multiply things by units and it doesn't 708 00:40:29,790 --> 00:40:33,370 affect factorization properties. 709 00:40:33,370 --> 00:40:35,810 This is [UNINTELLIGIBLE] a little bit more out there. 710 00:40:41,515 --> 00:40:45,530 What I eventually want to get to is -- 711 00:40:45,530 --> 00:40:49,510 skipping a few things that are said in the notes -- 712 00:40:49,510 --> 00:40:50,866 is unique factorization. 713 00:40:55,900 --> 00:41:05,520 Maybe before that I should define primes: 714 00:41:05,520 --> 00:41:07,380 are positive integers. 715 00:41:07,380 --> 00:41:10,862 The reason we stay positive is, again -- 716 00:41:10,862 --> 00:41:14,610 of course, we could have a negative prime integer, but 717 00:41:14,610 --> 00:41:17,230 let's pick a unique representative of two units. 718 00:41:20,290 --> 00:41:21,860 Positive integers -- 719 00:41:21,860 --> 00:41:24,330 I should've said in the notes greater than 1, we don't 720 00:41:24,330 --> 00:41:25,625 consider 1 to be a prime -- 721 00:41:28,170 --> 00:41:34,515 that have no non-trivial divisors. 722 00:41:41,940 --> 00:41:44,780 And, of course, every integer has trivial divisors. 723 00:41:49,790 --> 00:41:55,460 Namely, plus or minus 1 and plus or minus itself. 724 00:41:55,460 --> 00:41:57,205 Let me give this a name. 725 00:41:57,205 --> 00:41:57,580 i. 726 00:41:57,580 --> 00:42:00,140 Not a very good name. 727 00:42:00,140 --> 00:42:01,390 How about n? 728 00:42:03,700 --> 00:42:06,300 So if we have any integer n, it's divisible by plus or 729 00:42:06,300 --> 00:42:08,940 minus one, plus or minus n -- 730 00:42:08,940 --> 00:42:11,940 that doesn't count. 731 00:42:11,940 --> 00:42:15,120 So a prime is an integer that doesn't have any other 732 00:42:15,120 --> 00:42:18,300 divisors than these trivial ones. 733 00:42:18,300 --> 00:42:22,610 And then I want to get to unique factorization, which 734 00:42:22,610 --> 00:42:24,080 I'll just state, not prove. 735 00:42:28,240 --> 00:42:41,760 Which is that every integer is equal to a 736 00:42:41,760 --> 00:42:43,310 unique product of primes. 737 00:42:50,366 --> 00:42:53,840 And we always have to add up to units. 738 00:42:53,840 --> 00:42:57,860 We can add as many plus or minus 1's to the product as we 739 00:42:57,860 --> 00:43:01,990 want, as long as we have an even number of them. 740 00:43:01,990 --> 00:43:04,200 And we'll have another factorization, so here, we 741 00:43:04,200 --> 00:43:06,960 want to exclude the units. 742 00:43:06,960 --> 00:43:11,730 And then we have unique factorization. 743 00:43:11,730 --> 00:43:12,260 OK. 744 00:43:12,260 --> 00:43:15,770 This is all grade school stuff, high 745 00:43:15,770 --> 00:43:17,020 school stuff maybe. 746 00:43:20,910 --> 00:43:23,750 Again, when I go through polynomials, you'll see there 747 00:43:23,750 --> 00:43:28,610 is a set of concept exactly like this. 748 00:43:28,610 --> 00:43:35,540 Now, where we want to get to is mod-n arithmetic, which 749 00:43:35,540 --> 00:43:39,820 again, you all know about, I assume. 750 00:43:39,820 --> 00:43:42,300 This is based on the fact -- 751 00:43:42,300 --> 00:43:45,860 we can think of it as being based on Euclidean division 752 00:43:45,860 --> 00:43:51,940 algorithm, which basically proves that given any two 753 00:43:51,940 --> 00:44:06,280 integers m and n, we can write m as some qn plus r where 0 is 754 00:44:06,280 --> 00:44:09,765 less than or equal to r, less than or equal to n minus 1, 755 00:44:09,765 --> 00:44:13,470 and is called the remainder of m, modulo n. 756 00:44:16,370 --> 00:44:17,790 And how is this proved? 757 00:44:17,790 --> 00:44:20,330 Let's say these are both positive integers. 758 00:44:20,330 --> 00:44:24,200 Say m is a big integer, n is a small integer. 759 00:44:24,200 --> 00:44:29,600 We just keep subtracting integer multiples of n from m 760 00:44:29,600 --> 00:44:30,555 until we -- 761 00:44:30,555 --> 00:44:33,470 you know, the remainder will start decreasing until we get 762 00:44:33,470 --> 00:44:36,780 the remainder in this range, and it's obvious that we can 763 00:44:36,780 --> 00:44:39,710 always get a remainder that's in that range. 764 00:44:39,710 --> 00:44:43,840 And further, that this decomposition is unique under 765 00:44:43,840 --> 00:44:45,090 this restriction on r. 766 00:44:49,870 --> 00:44:50,330 All right. 767 00:44:50,330 --> 00:45:03,080 So we write the m is congruent or equivalent to r mod n. 768 00:45:03,080 --> 00:45:10,260 That means it differs from r by a multiple of n where r is 769 00:45:10,260 --> 00:45:18,000 unique and r is in the set 0, 1, up through n minus 1, which 770 00:45:18,000 --> 00:45:20,350 we say are the residue classes mod n. 771 00:45:20,350 --> 00:45:23,970 It has to be one of those n things. 772 00:45:23,970 --> 00:45:25,220 OK? 773 00:45:26,910 --> 00:45:28,840 No one is having the slightest trouble with this, right? 774 00:45:28,840 --> 00:45:29,285 Boring. 775 00:45:29,285 --> 00:45:31,180 You've seen it all before. 776 00:45:31,180 --> 00:45:32,430 I hope. 777 00:45:34,700 --> 00:45:35,950 OK. 778 00:45:37,660 --> 00:45:52,290 So now you ask, suppose m equals r mod-n, and some other 779 00:45:52,290 --> 00:45:57,070 w equals s mod-n. 780 00:45:57,070 --> 00:46:07,230 What is m plus w mod-n, and you quickly prove that it's 781 00:46:07,230 --> 00:46:10,690 equal to r plus s mod-n. 782 00:46:13,530 --> 00:46:17,970 By transferring, by just checking the properties of 783 00:46:17,970 --> 00:46:21,140 ordinary arithmetic, you can use this kind of expression: m 784 00:46:21,140 --> 00:46:25,030 must equal something times n plus r. 785 00:46:25,030 --> 00:46:28,670 w must equal something times n plus s. 786 00:46:28,670 --> 00:46:32,850 So if we add m plus w, we're going to get something times n 787 00:46:32,850 --> 00:46:34,860 plus r plus s. 788 00:46:34,860 --> 00:46:41,070 And that is, by definition, going to be r plus s mod-n. 789 00:46:41,070 --> 00:46:43,900 So we get a well-defined addition rule for these 790 00:46:43,900 --> 00:46:50,590 residues, for the elements of this set of n residues Rn, or 791 00:46:50,590 --> 00:46:52,630 remainders. 792 00:46:52,630 --> 00:46:55,980 OK, so addition is well-defined. 793 00:46:55,980 --> 00:47:00,790 And similarly, multiplication is well-defined. 794 00:47:00,790 --> 00:47:04,950 mw mod-n, again, you can prove -- and this is done 795 00:47:04,950 --> 00:47:06,200 in the notes -- 796 00:47:09,180 --> 00:47:14,930 that if mw mod-n can be found by taking rs and 797 00:47:14,930 --> 00:47:17,984 reducing it, mod-n. 798 00:47:17,984 --> 00:47:20,650 Maybe I should have congruent signs here, but I 799 00:47:20,650 --> 00:47:22,920 won't worry about it. 800 00:47:22,920 --> 00:47:34,150 So this is mod-n, addition and multiplication. 801 00:47:34,150 --> 00:47:39,930 So what this says is that Rn, combined with this mod-n 802 00:47:39,930 --> 00:47:44,460 addition operation, which in the notes I write in that way, 803 00:47:44,460 --> 00:47:47,885 and this mod-n multiplication operation -- 804 00:47:52,200 --> 00:47:56,190 this is a well-defined set of n elements with a well-defined 805 00:47:56,190 --> 00:47:59,040 addition operation and a well-defined 806 00:47:59,040 --> 00:48:01,464 multiplication operation. 807 00:48:01,464 --> 00:48:04,520 And I don't prove any properties about it here, but 808 00:48:04,520 --> 00:48:10,160 I come back to prove that this has almost all of the 809 00:48:10,160 --> 00:48:14,720 properties that we want of a field. 810 00:48:14,720 --> 00:48:17,850 In fact, if n is a prime, it does have the properties that 811 00:48:17,850 --> 00:48:19,510 we want of a field. 812 00:48:19,510 --> 00:48:25,910 So we'll later prove that the finite field with p elements 813 00:48:25,910 --> 00:48:35,720 is simply rp with mod-p addition and multiplication. 814 00:48:35,720 --> 00:48:39,490 And, of course, for the particular case p equals 2, we 815 00:48:39,490 --> 00:48:41,510 already have a lot of experience with this. 816 00:48:41,510 --> 00:48:43,000 That's how we get the binary field. 817 00:48:43,000 --> 00:48:48,340 We just take the 0 and 1, considered as residues mod-2. 818 00:48:48,340 --> 00:48:55,630 And then the field addition and multiplication operations 819 00:48:55,630 --> 00:48:59,320 give us something that satisfies the 820 00:48:59,320 --> 00:49:01,960 axioms of the field. 821 00:49:01,960 --> 00:49:05,240 We'll find that works for every prime, and it doesn't 822 00:49:05,240 --> 00:49:07,085 work for non-primes. 823 00:49:07,085 --> 00:49:10,360 Can anyone quickly see why this isn't going to work for 824 00:49:10,360 --> 00:49:11,610 non-primes? 825 00:49:15,010 --> 00:49:16,500 Suppose n equals 6. 826 00:49:21,270 --> 00:49:28,400 Anyone quickly see why this set under these two operations 827 00:49:28,400 --> 00:49:30,290 isn't going to be a field? 828 00:49:30,290 --> 00:49:32,950 This, of course, assumes that you know what a field is, 829 00:49:32,950 --> 00:49:36,280 which we haven't actually said yet. 830 00:49:36,280 --> 00:49:37,765 AUDIENCE: [INAUDIBLE]. 831 00:49:37,765 --> 00:49:38,525 PROFESSOR: Excuse me? 832 00:49:38,525 --> 00:49:41,000 AUDIENCE: [INAUDIBLE]. 833 00:49:41,000 --> 00:49:43,200 PROFESSOR: There's no inverse. 834 00:49:43,200 --> 00:49:47,810 In a field, every non-zero element has to have an 835 00:49:47,810 --> 00:49:52,740 inverse, just like in the real and complex field. 836 00:49:52,740 --> 00:50:00,290 But in the integers mod-6, 3 times 2 is equal to 6, which 837 00:50:00,290 --> 00:50:01,890 is equal to 0. 838 00:50:01,890 --> 00:50:06,080 So there are two non-zero elements that multiplied 839 00:50:06,080 --> 00:50:07,645 together give 0. 840 00:50:07,645 --> 00:50:11,170 And that means that neither 3 nor 2 can possibly have a 841 00:50:11,170 --> 00:50:14,330 multiplicative inverse. 842 00:50:14,330 --> 00:50:17,910 So that's why you have to have a prime. 843 00:50:17,910 --> 00:50:23,490 This doesn't happen, of course, if n were a prime. 844 00:50:23,490 --> 00:50:23,900 OK. 845 00:50:23,900 --> 00:50:28,360 So that's just a little warm up to remind you what mod-n 846 00:50:28,360 --> 00:50:29,780 arithmetic is like. 847 00:50:29,780 --> 00:50:34,780 And in particular, this is going to be the key element of 848 00:50:34,780 --> 00:50:37,350 a lot of proofs -- 849 00:50:37,350 --> 00:50:39,960 the Euclidean division algorithm -- 850 00:50:39,960 --> 00:50:43,140 both for integers and for polynomials. 851 00:50:49,000 --> 00:50:49,660 OK. 852 00:50:49,660 --> 00:50:52,185 So that, I hope, was review. 853 00:50:57,040 --> 00:50:58,530 Now we're going to talk about groups. 854 00:51:02,280 --> 00:51:03,600 So let's do it. 855 00:51:08,220 --> 00:51:10,250 How many of you feel you know what a group is? 856 00:51:14,690 --> 00:51:16,960 Less than half the class. 857 00:51:16,960 --> 00:51:18,250 OK. 858 00:51:18,250 --> 00:51:26,020 Well, we've been talking about groups so far here, but let's 859 00:51:26,020 --> 00:51:29,450 take a little bit more of an axiomatic approach. 860 00:51:29,450 --> 00:51:33,270 In the notes, I actually give two possible sets of axioms 861 00:51:33,270 --> 00:51:34,020 for a group. 862 00:51:34,020 --> 00:51:36,920 Let me start with a standard one. 863 00:51:36,920 --> 00:51:39,710 You've heard of something called the group property in 864 00:51:39,710 --> 00:51:40,960 this course. 865 00:51:45,450 --> 00:51:49,490 Well, first of all, what am I talking about when I'm talking 866 00:51:49,490 --> 00:51:50,080 about a group? 867 00:51:50,080 --> 00:51:54,670 I'm talking about a set, g, and I'm also talking about 868 00:51:54,670 --> 00:51:58,750 some operation on that set, which I'll use the mod 869 00:51:58,750 --> 00:52:04,370 addition circle-plus as the group property. 870 00:52:07,160 --> 00:52:12,650 Even if it's multiplication or matrix inversion or -- no, it 871 00:52:12,650 --> 00:52:15,690 couldn't be matrix inversion. 872 00:52:15,690 --> 00:52:21,800 What this means is that if we take any two elements of the 873 00:52:21,800 --> 00:52:27,430 group, then a plus b is well-defined. 874 00:52:31,440 --> 00:52:34,850 So you can think there's a table. 875 00:52:34,850 --> 00:52:36,840 Here are the elements of the group. 876 00:52:36,840 --> 00:52:39,385 a, b, c, d, e. 877 00:52:39,385 --> 00:52:41,860 a, b, c, d, e. 878 00:52:41,860 --> 00:52:46,390 And at this point, all we know is that we can 879 00:52:46,390 --> 00:52:48,580 fill out the table. 880 00:52:48,580 --> 00:52:50,540 We get something that goes in all here. 881 00:52:53,400 --> 00:52:54,200 All right? 882 00:52:54,200 --> 00:52:56,137 So that defines the group. 883 00:52:56,137 --> 00:52:58,999 AUDIENCE: [INAUDIBLE PHRASE]. 884 00:52:58,999 --> 00:53:02,500 PROFESSOR: That means if I give a and b, then the 885 00:53:02,500 --> 00:53:05,140 quantity a plus b is -- 886 00:53:05,140 --> 00:53:07,380 I can tell you what c is. 887 00:53:07,380 --> 00:53:10,060 There's a unique element in each of the cells of this 888 00:53:10,060 --> 00:53:13,072 addition table, or the group table. 889 00:53:15,970 --> 00:53:16,780 OK. 890 00:53:16,780 --> 00:53:25,540 Group property, also called closure, is the obvious one. 891 00:53:28,760 --> 00:53:31,770 What's an elementary condition of what all these sums have to 892 00:53:31,770 --> 00:53:33,715 be for this to be a group? 893 00:53:42,460 --> 00:53:43,710 Have to be all in the group. 894 00:53:49,350 --> 00:53:52,826 If I say I have a group that consists of 1,2,3,4,5, 895 00:53:52,826 --> 00:53:59,820 1,2,3,4,5 and my group operation is ordinary 896 00:53:59,820 --> 00:54:03,445 addition, then I can certainly fill out the table. 897 00:54:03,445 --> 00:54:06,820 And I get 2,3,4,5,6 -- 898 00:54:06,820 --> 00:54:08,990 whoops. 899 00:54:08,990 --> 00:54:11,450 Yeah, that's the element that's not in the group. 900 00:54:11,450 --> 00:54:18,790 So in order to have a group, I want a -- 901 00:54:18,790 --> 00:54:23,870 I mean, it's a plus b is in the group, all a and b. 902 00:54:26,430 --> 00:54:30,640 So the group is closed under addition. 903 00:54:30,640 --> 00:54:35,200 This was the property we used to define linearity for binary 904 00:54:35,200 --> 00:54:37,150 linear block codes. 905 00:54:37,150 --> 00:54:39,500 The sum of any two n-tuples had to be another 906 00:54:39,500 --> 00:54:40,750 n-tuple in the code. 907 00:54:47,260 --> 00:54:53,920 We have the associativity operation. 908 00:54:53,920 --> 00:55:03,745 This is a requirement on the binary operation, really. 909 00:55:03,745 --> 00:55:10,190 a plus b plus c has got to be equal to a plus b plus c. 910 00:55:10,190 --> 00:55:15,870 In other words, you can get rid of the parentheses. 911 00:55:15,870 --> 00:55:20,560 And this, an expression like that, is meaningful. 912 00:55:20,560 --> 00:55:23,980 It doesn't matter how you group, whether you add these 913 00:55:23,980 --> 00:55:27,480 two things first and then that, or add these two things 914 00:55:27,480 --> 00:55:28,730 first and then that. 915 00:55:32,390 --> 00:55:38,300 Maybe now is a good time to talk about the special 916 00:55:38,300 --> 00:55:41,870 property, which is not an axiom. 917 00:55:41,870 --> 00:55:47,036 So let me call it x abelian, or the commutative property. 918 00:55:52,700 --> 00:55:59,590 This is true if a plus b is equal to b 919 00:55:59,590 --> 00:56:05,692 plus a, all a, b group. 920 00:56:05,692 --> 00:56:09,200 Now this is, of course, true for ordinary addition and 921 00:56:09,200 --> 00:56:11,340 ordinary multiplication. 922 00:56:11,340 --> 00:56:13,840 It's not true for matrix multiplication. 923 00:56:13,840 --> 00:56:17,960 Say if I have a group of matrices, then in general, ab 924 00:56:17,960 --> 00:56:19,460 is not equal to ba. 925 00:56:19,460 --> 00:56:22,100 So for some groups this holds, for some 926 00:56:22,100 --> 00:56:23,350 groups it doesn't hold. 927 00:56:25,840 --> 00:56:31,130 In this course, almost without exception, every group we're 928 00:56:31,130 --> 00:56:33,490 going to talk about is an abelian group. 929 00:56:33,490 --> 00:56:36,450 That means that in adding things up, the 930 00:56:36,450 --> 00:56:38,670 order doesn't matter. 931 00:56:38,670 --> 00:56:43,170 So if that were true, then it in a plus b plus c, that means 932 00:56:43,170 --> 00:56:45,650 that's the same as c plus b plus a, or c plus 933 00:56:45,650 --> 00:56:46,960 a plus b, or whatever. 934 00:56:46,960 --> 00:56:48,910 All the properties that you're accustomed to 935 00:56:48,910 --> 00:56:51,110 from ordinary addition. 936 00:56:51,110 --> 00:56:53,940 But this is not one of the group axioms, and, of course, 937 00:56:53,940 --> 00:56:56,770 there are non-abelian groups. 938 00:56:56,770 --> 00:57:00,520 The smallest non-abelian group has size 6, and is the 939 00:57:00,520 --> 00:57:04,630 permutation group on three elements. 940 00:57:04,630 --> 00:57:07,550 So small groups are all -- 941 00:57:07,550 --> 00:57:10,015 size 5 or less are all abelian. 942 00:57:10,015 --> 00:57:14,020 Because that's the only way we can do the group table. 943 00:57:14,020 --> 00:57:14,410 All right. 944 00:57:14,410 --> 00:57:16,260 So that's terminology at this point. 945 00:57:16,260 --> 00:57:17,930 It's not part of the axiom. 946 00:57:17,930 --> 00:57:22,040 But that's the reason that I use a symbol that looks like a 947 00:57:22,040 --> 00:57:25,100 plus, is I'm pretty much always thinking an abelian 948 00:57:25,100 --> 00:57:27,700 group, and I don't want to trouble you with thinking 949 00:57:27,700 --> 00:57:30,830 about non-abelian groups. 950 00:57:30,830 --> 00:57:32,610 OK. 951 00:57:32,610 --> 00:57:35,230 Then we have the identity property. 952 00:57:38,610 --> 00:57:42,770 There is an identity, which, in an abelian group, is 953 00:57:42,770 --> 00:57:44,500 always called -- 954 00:57:44,500 --> 00:57:48,300 ordinarily called 0. 955 00:57:48,300 --> 00:57:52,040 In an additive group it's always called 0. 956 00:57:52,040 --> 00:57:53,270 So I'll call it 0. 957 00:57:53,270 --> 00:57:57,930 In more abstract treatments, it's called 1 or E for -- 958 00:57:57,930 --> 00:57:59,590 what does E stand for? 959 00:57:59,590 --> 00:58:01,990 Eigen something. 960 00:58:01,990 --> 00:58:10,020 There's a 0 in g such that 0 plus a equals -- 961 00:58:10,020 --> 00:58:12,640 or a plus 0 -- 962 00:58:12,640 --> 00:58:14,660 is always equal to a. 963 00:58:14,660 --> 00:58:19,382 The 0 is the null operator on any element of the group. 964 00:58:19,382 --> 00:58:23,140 If you add 0 to anything, you get what you started with. 965 00:58:26,890 --> 00:58:31,330 Clearly, we have such a thing in ordinary addition. 966 00:58:31,330 --> 00:58:34,470 Suppose I was talking in the real numbers under 967 00:58:34,470 --> 00:58:38,320 multiplication, is that a group, you guess? 968 00:58:42,875 --> 00:58:46,380 It's actually not. 969 00:58:46,380 --> 00:58:48,860 Suppose I talk about the non-zero real numbers under 970 00:58:48,860 --> 00:58:49,600 multiplication. 971 00:58:49,600 --> 00:58:50,850 Is that a group? 972 00:58:53,860 --> 00:58:57,700 Yes, and its identity is 1. 973 00:58:57,700 --> 00:59:01,680 1 as the thing that if you multiply it by anything, it 974 00:59:01,680 --> 00:59:04,810 leaves what you started with. 975 00:59:04,810 --> 00:59:09,540 There's the inverse, which we don't have in the reals under 976 00:59:09,540 --> 00:59:12,190 multiplication. 977 00:59:12,190 --> 00:59:21,620 For all a and g, there exists something called minus a -- 978 00:59:21,620 --> 00:59:25,700 or a to the minus 1 in multiplicative notation -- 979 00:59:25,700 --> 00:59:37,034 in g such that a plus minus a equals 0. 980 00:59:37,034 --> 00:59:40,260 That's an additive inverse. 981 00:59:40,260 --> 00:59:45,636 So in the real numbers, the inverse of a is minus a. 982 00:59:45,636 --> 00:59:47,740 In the integers, the inverse of a is 983 00:59:47,740 --> 00:59:49,660 minus a under addition. 984 00:59:49,660 --> 00:59:54,900 In the non-zero real numbers under multiplication, every 985 00:59:54,900 --> 00:59:59,510 non-zero real number does have a multiplicative inverse, 986 00:59:59,510 --> 01:00:04,410 which is called 1 over a, or a minus 1. 987 01:00:04,410 --> 01:00:05,990 That's why we have to exclude 0. 988 01:00:05,990 --> 01:00:07,405 0 does not an inverse. 989 01:00:10,850 --> 01:00:16,740 So a good example to think about is r-star, which is 990 01:00:16,740 --> 01:00:28,310 non-zero reals under ordinary multiplication, is a group. 991 01:00:28,310 --> 01:00:31,740 Is it an abelian group? 992 01:00:31,740 --> 01:00:34,746 Is ab equal to ba? 993 01:00:34,746 --> 01:00:35,170 Yeah. 994 01:00:35,170 --> 01:00:37,866 So this is an infinite abelian group. 995 01:00:53,878 --> 01:00:56,900 All right. 996 01:00:56,900 --> 01:00:59,755 OK. 997 01:00:59,755 --> 01:01:05,440 I want to show that the axioms lead to 998 01:01:05,440 --> 01:01:07,465 a permutation property. 999 01:01:15,640 --> 01:01:17,470 Because this is something that's going to come 1000 01:01:17,470 --> 01:01:18,720 up again and again. 1001 01:01:20,840 --> 01:01:28,430 One way of saying this is if I have, if a plus b equals c -- 1002 01:01:28,430 --> 01:01:32,030 suppose I write down that symbolic equation -- 1003 01:01:32,030 --> 01:01:45,670 then I can solve for any one of the three 1004 01:01:45,670 --> 01:01:46,920 given the other two. 1005 01:01:51,686 --> 01:01:52,970 How would I do that? 1006 01:01:52,970 --> 01:01:55,510 If I'm given a and b, obviously, that tells me c 1007 01:01:55,510 --> 01:01:56,230 from the table. 1008 01:01:56,230 --> 01:02:01,890 If I'm given a and c, then a is equal to c minus b -- 1009 01:02:01,890 --> 01:02:04,660 I'm sorry -- b is equal to c minus a. 1010 01:02:07,410 --> 01:02:11,360 I should say here, from this inverse, we're going to write 1011 01:02:11,360 --> 01:02:17,730 b plus minus a in a simpler form. 1012 01:02:17,730 --> 01:02:19,300 It's just b minus a. 1013 01:02:19,300 --> 01:02:24,700 So the fact that we have an inverse sort of defines 1014 01:02:24,700 --> 01:02:30,590 another operation, which we call subtraction, 1015 01:02:30,590 --> 01:02:32,825 which is not abelian. 1016 01:02:32,825 --> 01:02:33,770 Not commutative. 1017 01:02:33,770 --> 01:02:37,660 a minus b is not equal to b minus a. 1018 01:02:37,660 --> 01:02:40,420 But it's basically implicit in this that we 1019 01:02:40,420 --> 01:02:42,100 subtract group elements. 1020 01:02:42,100 --> 01:02:43,760 We can cancel group elements. 1021 01:02:43,760 --> 01:02:46,990 If you have a plus c equal to b plus c, then a 1022 01:02:46,990 --> 01:02:48,030 must be equal to b. 1023 01:02:48,030 --> 01:02:50,455 Because you can subtract c from both sides. 1024 01:02:56,160 --> 01:02:56,386 All right. 1025 01:02:56,386 --> 01:02:57,636 OK. 1026 01:03:01,940 --> 01:03:10,130 What does this mean that this gives us a constraint on the 1027 01:03:10,130 --> 01:03:11,430 addition table of a group? 1028 01:03:11,430 --> 01:03:22,280 Suppose I take a four element group, g, and one of the 1029 01:03:22,280 --> 01:03:25,640 elements is going to be the zero element, and the other 1030 01:03:25,640 --> 01:03:27,980 let's just call a, b, c. 1031 01:03:27,980 --> 01:03:31,220 0, a, b, c. 1032 01:03:31,220 --> 01:03:34,000 And I have a well-defined addition table, which 1033 01:03:34,000 --> 01:03:38,290 includes, in here, only a 0, a, b, c. 1034 01:03:38,290 --> 01:03:42,250 From the property of the identity, I can fill in some 1035 01:03:42,250 --> 01:03:43,500 of these things. 1036 01:03:47,190 --> 01:03:50,210 And now what freedom do I have to fill in the rest? 1037 01:03:50,210 --> 01:03:52,780 How many group tables are there of size four? 1038 01:03:58,490 --> 01:04:05,490 Well, the main point that I want to make with permutation 1039 01:04:05,490 --> 01:04:09,860 property is that every row and every column has to be a 1040 01:04:09,860 --> 01:04:12,570 permutation of the four elements 0, a, b, c. 1041 01:04:15,340 --> 01:04:16,630 Why is that? 1042 01:04:16,630 --> 01:04:22,980 Suppose I had another a in this column here. 1043 01:04:22,980 --> 01:04:27,670 Then I would have a plus 0 is equal to a, and also a plus c 1044 01:04:27,670 --> 01:04:30,210 is equal to a. 1045 01:04:30,210 --> 01:04:35,840 But that would imply that c equals 0, which it isn't. 1046 01:04:35,840 --> 01:04:36,026 All right. 1047 01:04:36,026 --> 01:04:40,370 So in order to be able to solve this equation, this has 1048 01:04:40,370 --> 01:04:44,850 to be something different from a, and likewise. 1049 01:04:44,850 --> 01:04:48,950 Since I have, in this case, a finite group, I basically just 1050 01:04:48,950 --> 01:04:51,990 have to write the group elements down here again in 1051 01:04:51,990 --> 01:04:53,240 some order. 1052 01:04:55,480 --> 01:04:58,780 It tuRns out that for four elements, there are exactly 1053 01:04:58,780 --> 01:05:04,170 two ways of doing that, generically. 1054 01:05:04,170 --> 01:05:07,860 One is sort of the cyclic group. 1055 01:05:07,860 --> 01:05:09,070 I'm sorry -- 1056 01:05:09,070 --> 01:05:16,690 a, b, c, 0, a, c, 0, a, b. 1057 01:05:16,690 --> 01:05:21,810 That's one way of filling out the table such that each row 1058 01:05:21,810 --> 01:05:24,750 is a permutation of 0, a, b, c. 1059 01:05:24,750 --> 01:05:28,060 Each column is a permutation of 0, a, b, c. 1060 01:05:28,060 --> 01:05:31,860 So that's a legitimate group table. 1061 01:05:31,860 --> 01:05:34,920 Let me just, for your interest, show you that that's 1062 01:05:34,920 --> 01:05:38,120 not the only one. 1063 01:05:38,120 --> 01:05:42,950 We could also do 0, a, b, c, 0, a, b, c. 1064 01:05:42,950 --> 01:05:45,290 If I write 0 here, then I've got to write 1065 01:05:45,290 --> 01:05:47,480 c there and b there. 1066 01:05:47,480 --> 01:05:51,790 Got to write c here and b there. 1067 01:05:51,790 --> 01:05:57,350 And then down here, the only choice I have is this. 1068 01:05:57,350 --> 01:06:02,400 This is called the cyclic group of four elements. 1069 01:06:02,400 --> 01:06:05,880 This is called the Klein four group. 1070 01:06:05,880 --> 01:06:07,500 And this is all there is. 1071 01:06:07,500 --> 01:06:11,830 Both of them tuRn out to be abelian if you check it out. 1072 01:06:11,830 --> 01:06:16,780 And basically, this permutation property restricts 1073 01:06:16,780 --> 01:06:20,930 these to be the possible groups of size four, abelian 1074 01:06:20,930 --> 01:06:23,800 or non-abelian, just to satisfy the axiom. 1075 01:06:28,270 --> 01:06:33,840 I actually prove in the notes that if you replace axioms a 1076 01:06:33,840 --> 01:06:43,020 and b with the permutation property, you get another 1077 01:06:43,020 --> 01:06:50,830 equivalent set of axioms for a group. 1078 01:06:50,830 --> 01:06:57,340 So the permutation of properties is very basic. 1079 01:06:57,340 --> 01:06:59,280 All you need is -- 1080 01:06:59,280 --> 01:07:03,070 for a set and a binary operation to form a group, all 1081 01:07:03,070 --> 01:07:06,600 you need is identity, associativity, and the 1082 01:07:06,600 --> 01:07:10,800 permutation property for the group table, 1083 01:07:10,800 --> 01:07:13,740 the addition table. 1084 01:07:13,740 --> 01:07:15,600 Simple proof of that in the notes. 1085 01:07:18,190 --> 01:07:22,050 But this observation is going to be 1086 01:07:22,050 --> 01:07:24,720 important as we go forward. 1087 01:07:24,720 --> 01:07:27,310 It's a fundamental thing we're going to use. 1088 01:07:27,310 --> 01:07:29,720 So you see a group can't just be anything. 1089 01:07:29,720 --> 01:07:33,610 It already has very definite restrictions on it. 1090 01:07:38,053 --> 01:07:38,840 All right. 1091 01:07:38,840 --> 01:07:43,144 Anything else I want to say right here? 1092 01:07:43,144 --> 01:07:45,026 OK, subtraction, cancellation. 1093 01:07:48,210 --> 01:08:01,400 OK, I guess we can go into the next property, 1094 01:08:01,400 --> 01:08:03,980 which is cyclic groups. 1095 01:08:03,980 --> 01:08:09,830 And I'm only going to talk about finite cyclic groups, or 1096 01:08:09,830 --> 01:08:12,090 finite groups in general. 1097 01:08:12,090 --> 01:08:14,640 Now, a cyclic group is probably the 1098 01:08:14,640 --> 01:08:15,890 simplest kind of group. 1099 01:08:22,140 --> 01:08:38,590 Let me first put up the canonical cyclic group, which 1100 01:08:38,590 --> 01:08:52,910 is the integers mod-n, which means the set of residues 1101 01:08:52,910 --> 01:08:56,854 mod-n and the mod-n addition operation. 1102 01:09:00,439 --> 01:09:05,120 Which is written just as Zn -- 1103 01:09:05,120 --> 01:09:07,029 there are various notations for it. 1104 01:09:07,029 --> 01:09:13,029 But this, of course, we'll just write it as Zn. 1105 01:09:13,029 --> 01:09:14,279 The integers mod-n. 1106 01:09:17,109 --> 01:09:20,350 Does this form a group? 1107 01:09:20,350 --> 01:09:21,835 Let's check the axioms. 1108 01:09:25,350 --> 01:09:29,439 Certainly, let's take the original axioms. 1109 01:09:29,439 --> 01:09:33,060 So the Rn is closed under addition mod-n. 1110 01:09:33,060 --> 01:09:37,830 If we add two elements, we're going to get another element. 1111 01:09:37,830 --> 01:09:40,325 The mod-n addition operation is associative. 1112 01:09:40,325 --> 01:09:42,560 In fact, it's abelian. 1113 01:09:42,560 --> 01:09:45,510 a plus b plus c is b plus c plus a -- it doesn't matter 1114 01:09:45,510 --> 01:09:47,310 what order, from the same property 1115 01:09:47,310 --> 01:09:49,510 for ordinary addition. 1116 01:09:49,510 --> 01:09:51,970 Is there an identity in here? 1117 01:09:51,970 --> 01:09:52,930 Yes. 1118 01:09:52,930 --> 01:09:56,630 The 0 is one of the elements here and acts as the identity. 1119 01:09:56,630 --> 01:10:01,120 0 plus anything mod-n is equal to the same thing mod-n. 1120 01:10:01,120 --> 01:10:04,660 And is there an inverse? 1121 01:10:04,660 --> 01:10:05,910 OK. 1122 01:10:08,430 --> 01:10:15,155 Let's explicitly write out Rn as 0, 1 up to n minus 1. 1123 01:10:18,120 --> 01:10:26,080 And then I claim that if I take any of these i, then n 1124 01:10:26,080 --> 01:10:29,130 minus i is its inverse. 1125 01:10:29,130 --> 01:10:35,270 n minus i is in the right range if not zero. 1126 01:10:35,270 --> 01:10:36,770 The inverse of 0 is 0. 1127 01:10:36,770 --> 01:10:41,800 The inverse of i between 1 and n minus 1 is going to be n 1128 01:10:41,800 --> 01:10:46,270 minus i because if I add i to n minus i -- 1129 01:10:46,270 --> 01:10:49,730 is equal to n, take mod-n, that's equal to 0. 1130 01:10:49,730 --> 01:10:55,050 So every element has the additive inverse. 1131 01:10:55,050 --> 01:11:01,460 And so the group table has a very boring form, which does 1132 01:11:01,460 --> 01:11:04,080 satisfy the permutation property. 1133 01:11:04,080 --> 01:11:06,910 If I take Z5, for instance. 1134 01:11:09,620 --> 01:11:09,845 0,1,2,3,4. 1135 01:11:09,845 --> 01:11:11,860 0 plus anything looks like that. 1136 01:11:11,860 --> 01:11:16,250 1 times anything is just 1,2,3,4 and 1 1137 01:11:16,250 --> 01:11:19,030 plus 4 is 0 mod 5. 1138 01:11:19,030 --> 01:11:31,400 2,3,4,0,1, and so forth. 1139 01:11:31,400 --> 01:11:31,650 3, 4, 0, 1, 2, 4, 0, 1, 2, 3. 1140 01:11:31,650 --> 01:11:35,930 Each row or column is just a cyclic shift to the previous 1141 01:11:35,930 --> 01:11:36,455 row or column. 1142 01:11:36,455 --> 01:11:37,770 Just fill it out. 1143 01:11:37,770 --> 01:11:40,400 And does it satisfy the permutation property? 1144 01:11:40,400 --> 01:11:43,070 Yes, it does. 1145 01:11:43,070 --> 01:11:49,400 This one here, for instance, you can recognize as a cyclic 1146 01:11:49,400 --> 01:11:50,360 group table. 1147 01:11:50,360 --> 01:11:55,930 And so, apart from relabeling, this is the group table of Z4. 1148 01:11:55,930 --> 01:12:01,840 Or we take a, b, c equal to 1, 2, 3. 1149 01:12:01,840 --> 01:12:06,050 If two groups have the same group table up to relabelling, 1150 01:12:06,050 --> 01:12:08,720 they're said to be isomorphic. 1151 01:12:08,720 --> 01:12:09,460 OK? 1152 01:12:09,460 --> 01:12:12,610 That's an important concept in math. 1153 01:12:12,610 --> 01:12:16,680 And all it means is the two groups act the same, except 1154 01:12:16,680 --> 01:12:18,840 that we've changed the names of the group elements. 1155 01:12:23,690 --> 01:12:25,580 OK. 1156 01:12:25,580 --> 01:12:31,710 So what I'm going to show is that all cyclic -- 1157 01:12:31,710 --> 01:12:37,020 this group is sort of cyclic because, as you see in the 1158 01:12:37,020 --> 01:12:42,830 notes, we can think of adding 1 as just being 1159 01:12:42,830 --> 01:12:45,200 cycling around a circle. 1160 01:12:48,140 --> 01:12:51,130 In this case, with 5 points on it. 1161 01:12:51,130 --> 01:12:55,280 And adding 2 is going two steps around the circle. 1162 01:12:55,280 --> 01:12:57,880 And the elements on the circle are the group 1163 01:12:57,880 --> 01:12:59,130 elements 0, 1, 2, 3, 4. 1164 01:13:02,160 --> 01:13:04,620 And then we get to 5. 1165 01:13:04,620 --> 01:13:06,860 We identify that with 0. 1166 01:13:06,860 --> 01:13:08,140 So we close the loop. 1167 01:13:08,140 --> 01:13:10,600 One we identify with 6. 1168 01:13:10,600 --> 01:13:11,645 This with 7. 1169 01:13:11,645 --> 01:13:13,270 This with 8. 1170 01:13:13,270 --> 01:13:14,890 Up to as many as you like. 1171 01:13:14,890 --> 01:13:17,500 So that's the intuition why these are 1172 01:13:17,500 --> 01:13:19,000 called cyclic groups. 1173 01:13:19,000 --> 01:13:24,640 The addition operation cycles rather than being on a line, 1174 01:13:24,640 --> 01:13:26,760 as it would be with the ordinary integers, where we go 1175 01:13:26,760 --> 01:13:28,530 infinitely in either direction. 1176 01:13:31,880 --> 01:13:49,150 So the formal definition of a cyclic group is that g has a 1177 01:13:49,150 --> 01:13:50,400 single generator. 1178 01:14:02,570 --> 01:14:06,250 Call it a little g. 1179 01:14:06,250 --> 01:14:09,120 And -- am I running over? 1180 01:14:09,120 --> 01:14:11,850 Close to. 1181 01:14:11,850 --> 01:14:12,530 Is that really right? 1182 01:14:12,530 --> 01:14:14,480 I only have five minutes left. 1183 01:14:14,480 --> 01:14:18,700 We're not going to go at the speed I had hoped. 1184 01:14:18,700 --> 01:14:19,810 g is -- 1185 01:14:19,810 --> 01:14:21,680 what does that mean? 1186 01:14:21,680 --> 01:14:26,210 If we know that g contains g, and this is never going to be 1187 01:14:26,210 --> 01:14:30,740 equal to 0, then we know that g contains two elements at 1188 01:14:30,740 --> 01:14:33,480 least: 0 and g. 1189 01:14:33,480 --> 01:14:45,090 And by the group axioms, g contains g, g plus g, g 1190 01:14:45,090 --> 01:14:46,970 plus g plus g -- 1191 01:14:46,970 --> 01:14:49,140 we can keep adding them. 1192 01:14:49,140 --> 01:14:53,340 I'm going to call these the integer multiples of g. 1193 01:14:53,340 --> 01:14:58,940 So this will be 1g, 2g, 3g. 1194 01:14:58,940 --> 01:15:00,800 That's what I mean when I write an integer 1195 01:15:00,800 --> 01:15:01,580 in front of g -- 1196 01:15:01,580 --> 01:15:06,560 I mean g plus itself, the integer number of times. 1197 01:15:06,560 --> 01:15:09,020 And all those things, by the group properties, have to be 1198 01:15:09,020 --> 01:15:11,430 in the group. 1199 01:15:11,430 --> 01:15:12,680 All right? 1200 01:15:14,640 --> 01:15:17,242 Again, restrict this to finite. 1201 01:15:17,242 --> 01:15:18,985 It's a finite cyclic group. 1202 01:15:22,150 --> 01:15:24,960 Then at some point, I have to -- 1203 01:15:24,960 --> 01:15:26,610 this is all the elements of the group. 1204 01:15:29,300 --> 01:15:31,500 So at some point, I have to come along and 1205 01:15:31,500 --> 01:15:36,220 say that ng is 0. 1206 01:15:36,220 --> 01:15:37,760 If just by -- 1207 01:15:37,760 --> 01:15:41,570 when I say generator, the set generated by g is 1208 01:15:41,570 --> 01:15:44,110 just 1g, 2g, 3g. 1209 01:15:44,110 --> 01:15:46,190 And that's got to be all the elements of the group, so n 1210 01:15:46,190 --> 01:15:47,780 times g has got to be 0. 1211 01:15:52,560 --> 01:15:59,130 And that means, without putting too fine a point on 1212 01:15:59,130 --> 01:16:03,400 it, that if n were 5, for instance, the group has to 1213 01:16:03,400 --> 01:16:11,720 consist of 0g, 1g, 2g, 3g, 4g, and then 5g is equal to 0. 1214 01:16:11,720 --> 01:16:14,860 So we call this 0g or 5g. 1215 01:16:17,820 --> 01:16:22,976 And furthermore, the addition table of the group, we can, 1216 01:16:22,976 --> 01:16:24,380 you know -- 1217 01:16:24,380 --> 01:16:28,380 however, if we add 2 g's to 3 g's, we're going to 5 g's, 1218 01:16:28,380 --> 01:16:29,920 which is 0. 1219 01:16:29,920 --> 01:16:30,340 All right? 1220 01:16:30,340 --> 01:16:35,150 So 2g plus 3g has got to be 5 g's, which is 0g. 1221 01:16:37,783 --> 01:16:41,570 So this is what the addition table of the 1222 01:16:41,570 --> 01:16:44,400 group has to look like. 1223 01:16:44,400 --> 01:16:47,510 It's exactly the same, except we just make a one-to-one 1224 01:16:47,510 --> 01:16:53,640 correspondence like that. 1225 01:16:53,640 --> 01:17:11,250 So the conclusion is a finite cyclic group 1226 01:17:11,250 --> 01:17:15,420 with n elements -- 1227 01:17:15,420 --> 01:17:18,425 I'll write the size of g by this absolute value thing -- 1228 01:17:24,020 --> 01:17:31,770 is isomorphic to the integers mod-n -- 1229 01:17:31,770 --> 01:17:35,260 this canonical cyclic group that we started with -- 1230 01:17:35,260 --> 01:17:43,340 must be isomorphic to Zn with the isomorphism being given by 1231 01:17:43,340 --> 01:17:44,633 the one-to-one correspondence. 1232 01:17:53,090 --> 01:18:03,140 i_g in the group corresponds to simply i in Zn, or 0 is 1233 01:18:03,140 --> 01:18:06,120 less than or equal to i is less than or 1234 01:18:06,120 --> 01:18:07,530 equal to n minus 1. 1235 01:18:10,390 --> 01:18:14,230 The addition rule is exactly the same. 1236 01:18:14,230 --> 01:18:18,920 OK, so this is a pretty sweeping conclusion. 1237 01:18:18,920 --> 01:18:22,330 It says that really the only finite cyclic group up to 1238 01:18:22,330 --> 01:18:25,990 relabeling is Zn. 1239 01:18:25,990 --> 01:18:28,560 OK, so if you understand Zn, you understand all finite 1240 01:18:28,560 --> 01:18:30,120 cyclic groups. 1241 01:18:30,120 --> 01:18:34,120 Just in a general group, we're going to have some element 1242 01:18:34,120 --> 01:18:38,490 which basically represents 1, called g, and its integer 1243 01:18:38,490 --> 01:18:43,470 multiples, which represent 2, 3, 4, and so forth. 1244 01:18:43,470 --> 01:18:47,650 But we can operate with a group just as 1245 01:18:47,650 --> 01:18:50,140 though it was Zn. 1246 01:18:50,140 --> 01:18:53,400 So it's easy to understand finite cyclic groups. 1247 01:18:53,400 --> 01:18:56,470 They all just look like this. 1248 01:18:56,470 --> 01:18:57,070 Question? 1249 01:18:57,070 --> 01:18:57,430 Yeah. 1250 01:18:57,430 --> 01:19:00,930 AUDIENCE: Is the generator unique? 1251 01:19:00,930 --> 01:19:03,955 PROFESSOR: No, it's not. 1252 01:19:03,955 --> 01:19:06,380 No, it's not. 1253 01:19:06,380 --> 01:19:08,950 Next time we'll start to talk about 1254 01:19:08,950 --> 01:19:12,370 subgroups of Zn, for instance. 1255 01:19:15,600 --> 01:19:21,690 In the set of Z5 -- 1256 01:19:21,690 --> 01:19:24,720 well, let me take this. 1257 01:19:24,720 --> 01:19:26,310 All right. 1258 01:19:26,310 --> 01:19:27,610 In Z5 -- 1259 01:19:27,610 --> 01:19:28,960 let me not even do that. 1260 01:19:28,960 --> 01:19:35,160 In Z5, 1 is the generator, but because 5 is a prime, 2 is a 1261 01:19:35,160 --> 01:19:36,880 generator, too. 1262 01:19:36,880 --> 01:19:38,770 All right? 1263 01:19:38,770 --> 01:19:40,610 Another way of writing. 1264 01:19:40,610 --> 01:19:45,305 Let's take a generator equals 2 in Z5. 1265 01:19:45,305 --> 01:19:50,370 Then g equals 2, 2g equals 4. 1266 01:19:50,370 --> 01:19:53,820 3g equals 6, equals 1 mod-5. 1267 01:19:53,820 --> 01:19:56,360 4g equals 3. 1268 01:19:56,360 --> 01:20:01,950 And 5g, of course, equals 5, which equals zero. 1269 01:20:01,950 --> 01:20:04,610 So 2 is also a generator. 1270 01:20:04,610 --> 01:20:07,990 And, in fact, 3 and 4 are also generators. 1271 01:20:07,990 --> 01:20:09,160 There are four generators. 1272 01:20:09,160 --> 01:20:11,340 This is getting us right into the number theory. 1273 01:20:13,940 --> 01:20:14,230 OK. 1274 01:20:14,230 --> 01:20:18,780 So no, g does not have to be 1. 1275 01:20:18,780 --> 01:20:21,870 g is just something that if you take all its integer 1276 01:20:21,870 --> 01:20:25,870 multiples, you generate the group. 1277 01:20:25,870 --> 01:20:31,630 OK, I guess we stop there, and we'll resume next time.