1 00:00:00,000 --> 00:00:04,180 PROFESSOR: OK, well I'm going to start now. 2 00:00:04,180 --> 00:00:07,735 As I say, I know that the wireless course has its final 3 00:00:07,735 --> 00:00:11,720 at 11:00, so we might not see the people who are taking it. 4 00:00:11,720 --> 00:00:14,770 Is anyone here in the wireless course? 5 00:00:14,770 --> 00:00:16,510 OK. 6 00:00:16,510 --> 00:00:21,860 Q.E.D. Ipso facto ergo sum. 7 00:00:21,860 --> 00:00:25,590 OK, just to remind you, I think we have Problem Set 10 8 00:00:25,590 --> 00:00:26,270 solutions today. 9 00:00:26,270 --> 00:00:29,510 In any case, they'll be available on the web. 10 00:00:29,510 --> 00:00:33,300 The final is next Tuesday. 11 00:00:33,300 --> 00:00:36,540 Do bring calculators if you have them. 12 00:00:36,540 --> 00:00:39,170 They won't be absolutely essential, but 13 00:00:39,170 --> 00:00:40,420 they might be useful. 14 00:00:43,220 --> 00:00:47,340 You may bring five sheets of notes. 15 00:00:47,340 --> 00:00:51,680 It's a good exercise to make these up. 16 00:00:51,680 --> 00:00:54,480 And otherwise, it's closed-book. 17 00:00:54,480 --> 00:00:57,860 Ashish will hold a review session tomorrow, 4:00 to 18 00:00:57,860 --> 00:01:04,084 6:00, 26-328, which I'm sure will be excellent as usual. 19 00:01:07,450 --> 00:01:11,120 All right, just to remind you, we're in Chapter 14. 20 00:01:11,120 --> 00:01:15,590 Most of the course we talked about coding for power-limited 21 00:01:15,590 --> 00:01:18,070 channels, where basically you could get away with binary 22 00:01:18,070 --> 00:01:21,010 codes with very little loss of optimality. 23 00:01:21,010 --> 00:01:24,070 For bandwidth-limited, additive white Gaussian noise 24 00:01:24,070 --> 00:01:26,890 channels, we've got to consider non-binary, 25 00:01:26,890 --> 00:01:29,680 multi-level codes, we've really got to go out into 26 00:01:29,680 --> 00:01:32,820 Euclidian space and use Euclidian distance as our 27 00:01:32,820 --> 00:01:36,390 metric rather than Hamming distance. 28 00:01:36,390 --> 00:01:42,750 So we started out with a simple idea, build a lattice 29 00:01:42,750 --> 00:01:44,610 constellation. 30 00:01:44,610 --> 00:01:49,440 A lattice is an infinite set of points in end space that 31 00:01:49,440 --> 00:01:51,030 has a group property. 32 00:01:51,030 --> 00:01:53,640 That's it, a discrete set of points. 33 00:01:53,640 --> 00:01:58,450 We take a finite set of them by chopping out a region R, a 34 00:01:58,450 --> 00:02:03,620 compact region, which chops out a finite number of points 35 00:02:03,620 --> 00:02:07,220 in this lattice or in a translate of the lattice. 36 00:02:07,220 --> 00:02:12,060 And we went through this exercise of computing the 37 00:02:12,060 --> 00:02:17,470 union-bound estimate, which I remind you is a reasonable 38 00:02:17,470 --> 00:02:20,280 estimate for codes that are not too complex. 39 00:02:20,280 --> 00:02:21,430 We're not going to use this for 40 00:02:21,430 --> 00:02:23,450 capacity-approaching codes. 41 00:02:23,450 --> 00:02:28,840 But for codes, where..., basically dominated by minimum 42 00:02:28,840 --> 00:02:33,080 distance properties, how many nearest neighbors are there, 43 00:02:33,080 --> 00:02:36,580 the likeliest error events are all to the nearest neighbors, 44 00:02:36,580 --> 00:02:39,980 we can use the union-bound estimate, which basically just 45 00:02:39,980 --> 00:02:43,940 tallies the probability of error to the nearest neighbor 46 00:02:43,940 --> 00:02:47,320 events by the union-bound, which itself is inaccurate if 47 00:02:47,320 --> 00:02:50,830 you get too many near neighbor events. 48 00:02:50,830 --> 00:02:55,320 And we've got this nice approximate expression that 49 00:02:55,320 --> 00:02:59,520 the probability of error per n dimensions, if we're signaling 50 00:02:59,520 --> 00:03:02,640 in n dimensions, is the number of nearest neighbors in the 51 00:03:02,640 --> 00:03:09,310 lattice times the Q to the square root of a function 52 00:03:09,310 --> 00:03:13,760 times an expression, which involves a coding gain of the 53 00:03:13,760 --> 00:03:19,110 lattice, and separably a shaping gain of the region 54 00:03:19,110 --> 00:03:22,490 times 3 times SNR norm. 55 00:03:22,490 --> 00:03:25,910 So this embodies all kinds of simplifications. 56 00:03:25,910 --> 00:03:29,540 It separates the effects of the lattice and the region. 57 00:03:29,540 --> 00:03:32,710 Basically, we can optimize each of these separately to 58 00:03:32,710 --> 00:03:34,720 optimize this expression. 59 00:03:34,720 --> 00:03:39,200 And also, by the way, by talking about SNR norm, which 60 00:03:39,200 --> 00:03:45,520 is SNR normalized by 2 to the spectral efficiency, it makes 61 00:03:45,520 --> 00:03:48,520 this whole expression independent of rate or 62 00:03:48,520 --> 00:03:51,420 equivalently independent of spectral efficiency. 63 00:03:51,420 --> 00:03:55,690 We saw this back when we talked about m by m QAM, we 64 00:03:55,690 --> 00:03:58,285 got a universal expression. 65 00:03:58,285 --> 00:04:02,210 The probability of error was simply this expression with a 66 00:04:02,210 --> 00:04:06,550 3SNR norm in here - regardless of what m was (regardless of 67 00:04:06,550 --> 00:04:09,730 how big the QAM constellation was), we got the same 68 00:04:09,730 --> 00:04:10,920 approximate expression. 69 00:04:10,920 --> 00:04:14,550 So this is also independent of the size of the constellations 70 00:04:14,550 --> 00:04:19,200 in this large constellation, high SNR approximation. 71 00:04:19,200 --> 00:04:23,450 Once you get to enough points, everything sort of washes out. 72 00:04:23,450 --> 00:04:26,760 So it's very useful. 73 00:04:26,760 --> 00:04:31,170 And I remind you that the coding gain basically depends 74 00:04:31,170 --> 00:04:36,650 on the minimum square distance normalized by the density of 75 00:04:36,650 --> 00:04:39,270 the lattice, if you like, normalized to two dimensions. 76 00:04:39,270 --> 00:04:45,700 The shaping gain is basically the shaping gain of an n-cube 77 00:04:45,700 --> 00:04:46,780 divided by-- 78 00:04:46,780 --> 00:04:49,310 this is the normalized second moment of an n-cube, this is 79 00:04:49,310 --> 00:04:52,140 the normalized second moment of your region, which better 80 00:04:52,140 --> 00:04:55,730 be better than that of an n-cube, meaning smaller. 81 00:04:55,730 --> 00:04:59,920 And writing it out, it's equal to this, where this is the 82 00:04:59,920 --> 00:05:02,970 second moment of your region. 83 00:05:02,970 --> 00:05:06,840 And this is the volume, again, normalized to two dimensions. 84 00:05:06,840 --> 00:05:09,520 So the baseline constellations, we showed 85 00:05:09,520 --> 00:05:12,840 these were invariant to Cartesian products. 86 00:05:12,840 --> 00:05:15,520 In any number of dimensions, n, we can take the integer 87 00:05:15,520 --> 00:05:21,520 lattice or any translate of the integer lattice and we can 88 00:05:21,520 --> 00:05:24,300 take a region, which is the n-cube -- 89 00:05:24,300 --> 00:05:28,250 which will chop out basically a square or cubic 90 00:05:28,250 --> 00:05:30,590 constellation, a regular constellation in n 91 00:05:30,590 --> 00:05:31,660 dimensions-- 92 00:05:31,660 --> 00:05:34,980 and we'll get that the coding gain is the shaping gain is 1. 93 00:05:34,980 --> 00:05:42,860 So we'll get just our baseline expression for any integer 94 00:05:42,860 --> 00:05:47,880 lattice shaped by an n-cube centered on the origin. 95 00:05:47,880 --> 00:05:49,100 And this is what it looks like. 96 00:05:49,100 --> 00:05:53,170 This is back from Chapter Four or whatever. 97 00:05:53,170 --> 00:05:56,720 We get this universal curve, the baseline curve, which 98 00:05:56,720 --> 00:06:01,930 holds for m-PAM, m by m QAM, or you can Cartesian product 99 00:06:01,930 --> 00:06:04,870 these up into as high a dimension as you like. 100 00:06:04,870 --> 00:06:08,610 The constellation we're talking about is really m-PAM 101 00:06:08,610 --> 00:06:11,190 Cartesian product to the nth. 102 00:06:11,190 --> 00:06:13,080 For any of those constellations, if we 103 00:06:13,080 --> 00:06:16,330 normalize the probability of error to two dimensions-- 104 00:06:16,330 --> 00:06:19,340 which is what I'm recommending to you, normalize everything 105 00:06:19,340 --> 00:06:23,480 per two dimensions in the high SNR regime-- 106 00:06:23,480 --> 00:06:26,240 then we basically get the number of near neighbors in 107 00:06:26,240 --> 00:06:31,775 Z-squared, the QAM lattice, which is 4 times Q to the 108 00:06:31,775 --> 00:06:33,480 square root of 3SNR norm. 109 00:06:33,480 --> 00:06:35,980 And it looks like this. 110 00:06:35,980 --> 00:06:38,260 We plot probability of error per two 111 00:06:38,260 --> 00:06:40,940 dimensions over SNR norm. 112 00:06:40,940 --> 00:06:44,160 I think at 10 to the minus 5, it's 8.4 dB. 113 00:06:44,160 --> 00:06:46,770 Does anybody remember? 114 00:06:46,770 --> 00:06:49,000 But it's somewhere around 9 dB. 115 00:06:49,000 --> 00:06:51,540 And the Shannon limit, as we recall, is for SNR 116 00:06:51,540 --> 00:06:53,700 norm, it's at 0 dB. 117 00:06:53,700 --> 00:06:56,420 So this is where we can play. 118 00:06:56,420 --> 00:07:00,090 Any code will fall somewhere between these two curves, and 119 00:07:00,090 --> 00:07:04,370 the name of the game is to get a code the gets as close to 120 00:07:04,370 --> 00:07:06,560 the Shannon limit as possible. 121 00:07:06,560 --> 00:07:10,300 And at least in this moderate complexity regime, the way we 122 00:07:10,300 --> 00:07:14,020 play the game is we try to optimize the shaping region, 123 00:07:14,020 --> 00:07:18,190 we try to optimize the lattice, we're going to get a 124 00:07:18,190 --> 00:07:21,680 certain gain from shaping, and a certain gain from coding, 125 00:07:21,680 --> 00:07:27,230 de-rated by the number of nearest neighbors, which is 126 00:07:27,230 --> 00:07:29,840 going to turn out to be an important effect for some 127 00:07:29,840 --> 00:07:31,630 well-known lattices because they have huge 128 00:07:31,630 --> 00:07:33,210 numbers of near neighbors. 129 00:07:33,210 --> 00:07:36,870 And that will give us some curve somewhere in here, and 130 00:07:36,870 --> 00:07:38,570 we plot that and we're done. 131 00:07:38,570 --> 00:07:41,290 We've written a paper for that particular lattice 132 00:07:41,290 --> 00:07:43,245 constellation, for its performance. 133 00:07:49,358 --> 00:07:53,840 Today I'm going to talk quickly about shaping, because 134 00:07:53,840 --> 00:08:00,390 that's a very easy story, and obvious, and 135 00:08:00,390 --> 00:08:01,900 it won't take long. 136 00:08:01,900 --> 00:08:05,540 Then I'm going to talk about lattices, which is really a 137 00:08:05,540 --> 00:08:10,430 subject in mathematics, what's known about lattices. 138 00:08:10,430 --> 00:08:13,630 I guess the decoding part of that story is not mathematics. 139 00:08:13,630 --> 00:08:16,550 That's engineering. 140 00:08:16,550 --> 00:08:21,410 And then I'm going to talk about trellis codes, which are 141 00:08:21,410 --> 00:08:25,600 the analog of convolutional codes, and which, similarly to 142 00:08:25,600 --> 00:08:28,090 the way convolutional codes beat block codes for the 143 00:08:28,090 --> 00:08:30,780 bandwidth for the power-limited regime, trellis 144 00:08:30,780 --> 00:08:35,140 codes tend to beat lattice codes in terms of performance 145 00:08:35,140 --> 00:08:40,950 versus complexity for the bandwidth-limited regime. 146 00:08:40,950 --> 00:08:42,010 So, shaping. 147 00:08:42,010 --> 00:08:45,410 Shaping is really an almost trivial story. 148 00:08:45,410 --> 00:08:52,200 This really wasn't recognized as a separable issue until 149 00:08:52,200 --> 00:08:57,650 perhaps sometime in the '80s, when trellis-coded modulation 150 00:08:57,650 --> 00:08:58,280 was invented. 151 00:08:58,280 --> 00:09:03,020 In '82 was Ungerboeck's paper. 152 00:09:03,020 --> 00:09:04,900 That was really the breakthrough in 153 00:09:04,900 --> 00:09:10,240 bandwidth-limited coding, that trellis codes were the first 154 00:09:10,240 --> 00:09:13,000 bandwidth-limited codes that were widely implemented. 155 00:09:13,000 --> 00:09:15,950 And after people analyzed them for a while, they realized 156 00:09:15,950 --> 00:09:19,590 that you could separate out the shaping effect from the 157 00:09:19,590 --> 00:09:23,370 coding effect, which is what I've shown by this expression. 158 00:09:23,370 --> 00:09:28,580 So anyone care to guess what the optimum region is in n 159 00:09:28,580 --> 00:09:29,980 dimensions? 160 00:09:29,980 --> 00:09:33,480 I guess I've already mentioned it last time. 161 00:09:33,480 --> 00:09:35,540 The optimum region, what do we want to do? 162 00:09:35,540 --> 00:09:39,440 We want to minimize the normalized second moment-- 163 00:09:42,150 --> 00:09:44,540 well, a second moment with appropriate normalization, 164 00:09:44,540 --> 00:09:45,810 what would you do? 165 00:09:45,810 --> 00:09:47,870 Well, the optimum region for just about anything is an 166 00:09:47,870 --> 00:09:51,660 n-sphere, and it certainly is here. 167 00:09:51,660 --> 00:09:54,650 So we take Rn equals an n-sphere. 168 00:09:57,800 --> 00:10:04,400 If we have a two-dimensional sphere, what are its 169 00:10:04,400 --> 00:10:05,650 parameters? 170 00:10:08,510 --> 00:10:13,810 We certainly know that the volume of this region, if this 171 00:10:13,810 --> 00:10:15,405 is R, it's pi times R squared. 172 00:10:19,430 --> 00:10:28,500 And the second moment of this region, how do we do that? 173 00:10:28,500 --> 00:10:31,670 We integrate ... go to polar coordinates, 0 to 2pi, 174 00:10:31,670 --> 00:10:38,190 d pi, 0 to R, rdr. 175 00:10:38,190 --> 00:10:41,590 And then what are we measuring? 176 00:10:41,590 --> 00:10:45,400 The second moment is r squared, but we normalize it 177 00:10:45,400 --> 00:10:47,020 per dimension, which is 2. 178 00:10:49,740 --> 00:10:55,820 Oh, and we've got to take the uniform distribution over v of 179 00:10:55,820 --> 00:10:59,220 R, and that's one over pi R squared. 180 00:10:59,220 --> 00:11:02,130 So have I got it now? 181 00:11:02,130 --> 00:11:04,970 This is the second moment of a uniform distribution, a 182 00:11:04,970 --> 00:11:09,100 probability distribution over the 2-sphere. 183 00:11:09,100 --> 00:11:14,230 And so this will be 1 over 2pi R squared times 184 00:11:14,230 --> 00:11:19,690 R 4th over 4, right? 185 00:11:23,520 --> 00:11:24,500 Is that all correct? 186 00:11:24,500 --> 00:11:32,510 And so p of R over v of R is already not normalized to two 187 00:11:32,510 --> 00:11:35,870 dimensions, so we don't have to re-normalize it. 188 00:11:44,620 --> 00:11:46,290 I didn't get the 2pi in here. 189 00:11:50,950 --> 00:11:51,970 I need some help, class. 190 00:11:51,970 --> 00:11:53,790 When it comes down to integration, I'm not 191 00:11:53,790 --> 00:11:56,680 necessarily the world's best. 192 00:11:56,680 --> 00:12:00,350 So R squared over 4... 193 00:12:00,350 --> 00:12:09,510 R squared over 4pi R squared, so we get 1 over 4pi. 194 00:12:09,510 --> 00:12:10,050 Is that right? 195 00:12:10,050 --> 00:12:11,670 That looks about right because it's fairly 196 00:12:11,670 --> 00:12:12,920 close to 1 over 12. 197 00:12:15,370 --> 00:12:18,390 So what is the shaping gain of a 2-sphere? 198 00:12:22,790 --> 00:12:25,680 And I can just say 2-sphere, because I know it's invariant 199 00:12:25,680 --> 00:12:28,250 to scaling. 200 00:12:28,250 --> 00:12:31,780 Rotation isn't going to do much to it. 201 00:12:31,780 --> 00:12:39,220 It's 1/12 over this, the normalized second moment, 1 202 00:12:39,220 --> 00:12:47,220 over 4pi, which is pi over 3, which is about 2/10 of a dB. 203 00:12:47,220 --> 00:12:51,530 OK, so I've improved the second moment by 2/10 of a dB, 204 00:12:51,530 --> 00:12:53,920 reduced it to 2/10 of a dB. 205 00:12:53,920 --> 00:12:57,730 This is not very impressive. 206 00:12:57,730 --> 00:13:00,680 So of course, a lot of the early work was on shaping 207 00:13:00,680 --> 00:13:02,930 two-dimensional constellations. 208 00:13:02,930 --> 00:13:06,270 And this basically says if you get a large constellation, you 209 00:13:06,270 --> 00:13:10,250 don't even see this effect to, say, 64-QAM. 210 00:13:10,250 --> 00:13:13,955 If you have a 64-QAM constellation, 8 by 8-- 211 00:13:23,870 --> 00:13:26,370 Somebody at Paradyne noticed that you could improve it 212 00:13:26,370 --> 00:13:30,390 slightly by taking the four corner points out and putting 213 00:13:30,390 --> 00:13:34,686 them up here, without reducing the minimum distance. 214 00:13:34,686 --> 00:13:40,860 OK, so there is an improved, more spherical 64-point 215 00:13:40,860 --> 00:13:41,690 constellation. 216 00:13:41,690 --> 00:13:44,840 It's improved by about 0.1 dB, as I remember. 217 00:13:44,840 --> 00:13:49,710 And what this calculation says is if you go up to 256, 1024, 218 00:13:49,710 --> 00:13:55,350 and you simply take all the points in the QAM grid that 219 00:13:55,350 --> 00:13:58,830 are surrounded by a sphere of appropriate size to pick out 2 220 00:13:58,830 --> 00:14:02,860 to the n points, for high, large constellations you get a 221 00:14:02,860 --> 00:14:04,790 gain of about 0.2 dB. 222 00:14:04,790 --> 00:14:09,030 So this is nothing to write home about. 223 00:14:09,030 --> 00:14:11,850 What happens as n goes to infinity? 224 00:14:15,370 --> 00:14:19,570 We look up in a math book what the expressions are for the 225 00:14:19,570 --> 00:14:23,070 normalized second moment-- actually, you can do it just 226 00:14:23,070 --> 00:14:26,360 by this kind of calculation. 227 00:14:26,360 --> 00:14:28,010 It's easy enough. 228 00:14:28,010 --> 00:14:39,475 And just by scaling, you'll find that the shaping gain of 229 00:14:39,475 --> 00:14:49,460 the n-sphere goes to pi times e over 6, which is 1.53 dB. 230 00:14:49,460 --> 00:14:52,630 A famous number not only now in shaping, but also in 231 00:14:52,630 --> 00:14:55,850 quantization, where you're also interested in minimizing 232 00:14:55,850 --> 00:14:59,560 the second moment of a quantization cell. 233 00:15:02,210 --> 00:15:06,450 Or it's something like 0.229 bits per two dimensions, is 234 00:15:06,450 --> 00:15:08,600 another way of expressing it. 235 00:15:08,600 --> 00:15:12,100 All right, so what is this? 236 00:15:12,100 --> 00:15:15,090 This is the maximum shaping gain you could get by the 237 00:15:15,090 --> 00:15:19,470 optimum region in the optimum number of dimensions, which is 238 00:15:19,470 --> 00:15:21,040 as large as you can go. 239 00:15:21,040 --> 00:15:24,090 So this says this is the ultimate shaping gain. 240 00:15:24,090 --> 00:15:26,120 You can never get a shaping gain of more 241 00:15:26,120 --> 00:15:29,650 than 1 and 1/2 dB. 242 00:15:29,650 --> 00:15:34,490 So it says when we play this game, we've got about 9 dB 243 00:15:34,490 --> 00:15:37,255 down here that we want to make up at 10 to the minus 5, 10 to 244 00:15:37,255 --> 00:15:42,820 the minus 6, we can get only 1 and a 1/2 dB of it by doing 245 00:15:42,820 --> 00:15:45,260 good shaping in large dimensions. 246 00:15:45,260 --> 00:15:47,620 That's the best we can ever do. 247 00:15:47,620 --> 00:15:49,390 So that's a little bit discouraging. 248 00:15:49,390 --> 00:15:53,230 It says shaping is never going to give you that much. 249 00:15:53,230 --> 00:15:57,400 On the other hand, you can't get this 1 and a 1/2 dB 250 00:15:57,400 --> 00:15:59,040 in any other way. 251 00:15:59,040 --> 00:16:01,380 If you have n-cubed constellations, you will 252 00:16:01,380 --> 00:16:04,870 always be limited to be 1 and a 1/2 dB away 253 00:16:04,870 --> 00:16:07,570 from the Shannon limit. 254 00:16:07,570 --> 00:16:09,990 So if you want to approach the Shannon limit, you do have to 255 00:16:09,990 --> 00:16:12,230 address the shaping problem. 256 00:16:12,230 --> 00:16:13,840 You can't ignore it. 257 00:16:13,840 --> 00:16:21,020 1 and a 1/2 dB is enough to be noticed in most applications, 258 00:16:21,020 --> 00:16:24,750 so you should do some moderate shaping. 259 00:16:24,750 --> 00:16:27,240 Now, of course we can plot the curve. 260 00:16:27,240 --> 00:16:31,280 It's done in the notes for any n whatsoever, and you'll see 261 00:16:31,280 --> 00:16:34,380 that it goes up fairly steeply at first. 262 00:16:34,380 --> 00:16:41,450 It gets to about 1 dB at 16 dimensions, so if you shape in 263 00:16:41,450 --> 00:16:45,820 a 16 dimensional sphere, you can get 1 dB out of this 264 00:16:45,820 --> 00:16:48,950 maximum of 1 and a 1/2 dBs you can ever get. 265 00:16:48,950 --> 00:16:53,170 If you go to 32, it's 1.2 dB or something like that, it 266 00:16:53,170 --> 00:16:55,840 begins to slope out. 267 00:16:55,840 --> 00:17:02,260 So that gives you a good idea of what you 268 00:17:02,260 --> 00:17:04,109 can actually attain. 269 00:17:04,109 --> 00:17:09,200 In the V.34 modem, for instance, a rather simple 270 00:17:09,200 --> 00:17:13,740 shaping scheme is used that gets up to about 0.8 dB of 271 00:17:13,740 --> 00:17:14,609 shaping gain. 272 00:17:14,609 --> 00:17:16,840 It's called shaping on regions. 273 00:17:20,630 --> 00:17:25,700 I just mentioned some techniques for shaping. 274 00:17:25,700 --> 00:17:29,480 As with a lot of things, once shaping was realized to be a 275 00:17:29,480 --> 00:17:33,220 separate subject from coding, there was a flurry 276 00:17:33,220 --> 00:17:34,270 of activity on it. 277 00:17:34,270 --> 00:17:37,670 People basically proposed a number of different schemes, 278 00:17:37,670 --> 00:17:41,480 particularly in the context of the V.34 modem standards 279 00:17:41,480 --> 00:17:42,952 development. 280 00:17:42,952 --> 00:17:50,010 Of which trellis shaping, I would 281 00:17:50,010 --> 00:17:53,000 say, is the most elegant. 282 00:17:53,000 --> 00:17:58,210 This is kind of a dual to trellis-coded modulation where 283 00:17:58,210 --> 00:18:01,310 we use trellis codes for shaping rather than coding. 284 00:18:01,310 --> 00:18:03,290 So that's elegant. 285 00:18:03,290 --> 00:18:08,440 But a cruder and quite effective, just fine 286 00:18:08,440 --> 00:18:16,960 technique, shell mapping, was used in the V.34 modem. 287 00:18:16,960 --> 00:18:21,700 The basic idea here is that you take a two-dimensional 288 00:18:21,700 --> 00:18:27,000 constellation, you just divide it into shells like this. 289 00:18:27,000 --> 00:18:30,610 It turns out in two dimensions, if you take a 290 00:18:30,610 --> 00:18:36,540 constant delta here, that the average energy goes up, I 291 00:18:36,540 --> 00:18:42,960 think, linearly with delta or some way nicely with delta. 292 00:18:42,960 --> 00:18:48,980 You take a constellation bigger than you really need, 293 00:18:48,980 --> 00:18:50,970 so you have the possibility of some redundancy. 294 00:18:56,090 --> 00:18:59,750 So you identify each point in the constellation by which 295 00:18:59,750 --> 00:19:03,900 region it lands in. 296 00:19:03,900 --> 00:19:06,220 You then go up to 16 dimensions or something-- 297 00:19:06,220 --> 00:19:09,770 I think it is 16 dimensions in the V.34 modem-- 298 00:19:09,770 --> 00:19:17,640 and you assign each series of points a score depending on 299 00:19:17,640 --> 00:19:20,690 which shells its points land in. 300 00:19:20,690 --> 00:19:27,150 And you basically have a big table that chooses the lowest 301 00:19:27,150 --> 00:19:34,100 energy combinations from all the possible series of eight 302 00:19:34,100 --> 00:19:37,860 shells, which can be done fairly elegantly using 303 00:19:37,860 --> 00:19:39,290 generating function techniques. 304 00:19:39,290 --> 00:19:43,010 I realize this is just hand-waving and you can't get 305 00:19:43,010 --> 00:19:50,100 it, but the effect is that you use this shell with very high 306 00:19:50,100 --> 00:19:50,730 probability. 307 00:19:50,730 --> 00:19:53,560 You sort of get a Gaussian-like probability on 308 00:19:53,560 --> 00:19:55,650 the shells. 309 00:19:55,650 --> 00:19:57,860 Let me just draw that like this. 310 00:19:57,860 --> 00:19:59,750 You don't use the points out in this 311 00:19:59,750 --> 00:20:01,410 outermost shell very much. 312 00:20:01,410 --> 00:20:04,880 You use this a lot, and they sort of have an overall 313 00:20:04,880 --> 00:20:07,290 Gaussian distribution. 314 00:20:07,290 --> 00:20:09,720 And I should mention that this is an equivalent way of 315 00:20:09,720 --> 00:20:10,970 getting shaping gain. 316 00:20:17,010 --> 00:20:21,270 If we're shaping uniformly over an n-sphere in a high 317 00:20:21,270 --> 00:20:26,870 number n of dimensions and we basically have constellations 318 00:20:26,870 --> 00:20:32,880 which consist of two-dimensional QAM points, 319 00:20:32,880 --> 00:20:36,805 what is the projection of a uniform distribution over an 320 00:20:36,805 --> 00:20:39,990 n-sphere down onto two dimensions? 321 00:20:39,990 --> 00:20:43,560 If we look at each two-dimensional component of 322 00:20:43,560 --> 00:20:47,266 these n-dimensional vectors, what distribution will it have 323 00:20:47,266 --> 00:20:52,180 if we impose a uniform spherical distribution on the 324 00:20:52,180 --> 00:20:53,830 high dimensional constellation? 325 00:20:53,830 --> 00:20:59,040 It will turn out that it becomes Gaussian in two 326 00:20:59,040 --> 00:21:00,560 dimensions. 327 00:21:00,560 --> 00:21:01,750 Think of the projection of the 328 00:21:01,750 --> 00:21:04,110 2-sphere down on one dimension. 329 00:21:04,110 --> 00:21:07,580 The projection of a 2-sphere on one dimension is 330 00:21:07,580 --> 00:21:09,730 something like this. 331 00:21:09,730 --> 00:21:14,570 It's not Gaussian, but there's more probability that one of 332 00:21:14,570 --> 00:21:18,040 the coordinates will be near 0 than it will be out at one of 333 00:21:18,040 --> 00:21:19,990 these maxima here. 334 00:21:19,990 --> 00:21:24,230 So this is just the same thing raised up to n dimensions. 335 00:21:24,230 --> 00:21:27,020 So an equivalent way of looking at this problem is we 336 00:21:27,020 --> 00:21:30,620 want somehow to get a Gaussian-like distribution on 337 00:21:30,620 --> 00:21:33,880 a one- or two-dimensional constellation, and this is one 338 00:21:33,880 --> 00:21:36,390 way of doing it. 339 00:21:36,390 --> 00:21:40,700 And you can get the same pi times e over 6 factor by just 340 00:21:40,700 --> 00:21:44,680 looking at it from an entropy point of view, the entropy of 341 00:21:44,680 --> 00:21:47,420 a two-dimensional Gaussian distribution relative to the 342 00:21:47,420 --> 00:21:51,750 entropy of a uniform distribution over a square. 343 00:21:51,750 --> 00:21:54,992 That will give you the same factor if you 344 00:21:54,992 --> 00:21:57,300 do everything right. 345 00:21:57,300 --> 00:22:01,120 OK so basically, shaping is easy. 346 00:22:01,120 --> 00:22:04,970 These techniques are not difficult to implement. 347 00:22:04,970 --> 00:22:09,320 You can get one dB of shaping gain quite easily through a 348 00:22:09,320 --> 00:22:12,230 variety of techniques. 349 00:22:12,230 --> 00:22:14,550 It's worth getting that one dB, you're not going to get it 350 00:22:14,550 --> 00:22:17,050 by any other means. 351 00:22:17,050 --> 00:22:19,660 But having done that, that's all you're ever going to get, 352 00:22:19,660 --> 00:22:22,460 and so end of story on shaping. 353 00:22:22,460 --> 00:22:29,260 So as I said, it was a flurry of activity in the '90s, and 354 00:22:29,260 --> 00:22:30,510 no one has looked at shaping since then. 355 00:22:38,270 --> 00:22:39,870 So that's part of the story. 356 00:22:42,730 --> 00:22:47,180 Let's continue now with lattices, an old 357 00:22:47,180 --> 00:22:48,430 mathematical subject. 358 00:22:51,580 --> 00:22:55,170 So what we're interested in is, what's the densest lattice 359 00:22:55,170 --> 00:22:57,096 packing in n dimensions? 360 00:22:59,970 --> 00:23:03,380 This appeared in mathematics, long before it appeared in 361 00:23:03,380 --> 00:23:05,330 engineering, as Hermite's parameter. 362 00:23:05,330 --> 00:23:07,440 And Hermite said this was the right way to measure the 363 00:23:07,440 --> 00:23:10,540 density of a lattice, normalized in all the 364 00:23:10,540 --> 00:23:11,380 appropriate ways. 365 00:23:11,380 --> 00:23:15,290 So any version of a lattice is equivalent. 366 00:23:15,290 --> 00:23:18,960 A version is something that you get by scaling or rotating 367 00:23:18,960 --> 00:23:24,020 or even by Cartesian products, which I'm not sure how 368 00:23:24,020 --> 00:23:26,860 widespread that idea is. 369 00:23:26,860 --> 00:23:30,642 So what's the densest lattice in two dimensions again? 370 00:23:30,642 --> 00:23:35,760 We want to find the densest lattice, in terms of coding 371 00:23:35,760 --> 00:23:38,700 gain or Hermite's parameter in n dimensions. 372 00:23:43,392 --> 00:23:46,230 Anyone care to guess what the densest lattice in two 373 00:23:46,230 --> 00:23:49,500 dimensions is? 374 00:23:49,500 --> 00:23:52,000 It's the hexagonal lattice. 375 00:23:52,000 --> 00:24:00,660 So in two dimensions, hexagonal, which reasons of 376 00:24:00,660 --> 00:24:04,690 mathematical history is denoted A_2. 377 00:24:04,690 --> 00:24:12,190 And its coding gain, I believe we computed it last time, and 378 00:24:12,190 --> 00:24:13,620 it was 0.6 dB. 379 00:24:18,870 --> 00:24:25,280 So what's the most we could ever do by fooling around with 380 00:24:25,280 --> 00:24:26,810 two-dimensional constellations? 381 00:24:26,810 --> 00:24:33,040 Suppose I want the best possible 256 point QAM 382 00:24:33,040 --> 00:24:35,350 constellation? 383 00:24:35,350 --> 00:24:39,990 Well, according to these large constellation approximations, 384 00:24:39,990 --> 00:24:47,540 the best I could do would be to take the 256 lowest energy 385 00:24:47,540 --> 00:24:50,260 points in some translate of the hexagonal lattice. 386 00:24:50,260 --> 00:24:53,210 So I might fool around with the translation vector a 387 00:24:53,210 --> 00:24:57,960 little bit to try to find the absolute lowest energy point, 388 00:24:57,960 --> 00:24:59,580 but it really wouldn't matter very much 389 00:24:59,580 --> 00:25:02,020 regardless of what I did. 390 00:25:02,020 --> 00:25:06,240 I could expect 0.6 dB of coding gain and 0.2 dB of 391 00:25:06,240 --> 00:25:14,360 shaping gain, or 0.8 dB better than the 8 by 8 - sorry the 16 392 00:25:14,360 --> 00:25:19,100 by 16 QAM constellation with the same number of points and 393 00:25:19,100 --> 00:25:20,350 the same minimum distance. 394 00:25:24,140 --> 00:25:28,340 This basically tells you what your options are. 395 00:25:28,340 --> 00:25:30,670 2D, there's not much to do. 396 00:25:30,670 --> 00:25:33,300 You remember way back in Chapter One, I gave you this 397 00:25:33,300 --> 00:25:36,060 problem about the 16 point, 2D constellation. 398 00:25:36,060 --> 00:25:41,690 That is precisely 16 points from a certain translate of 399 00:25:41,690 --> 00:25:45,600 the hexagonal lattice that fall within a circle that's 400 00:25:45,600 --> 00:25:48,370 just big enough to enclose 16 points. 401 00:25:48,370 --> 00:25:54,810 And it has, I forget, something like a 0.7 dB gain 402 00:25:54,810 --> 00:25:59,210 over the 4 by 4 square constellation. 403 00:25:59,210 --> 00:26:02,810 We computed that in the very first homework. 404 00:26:02,810 --> 00:26:05,840 So again, staying to two dimensions, 405 00:26:05,840 --> 00:26:07,120 things aren't very exciting. 406 00:26:10,340 --> 00:26:13,270 Turns out, it's most interesting to go 407 00:26:13,270 --> 00:26:14,710 in powers of two. 408 00:26:14,710 --> 00:26:21,320 In four dimensions, the densest lattice is something 409 00:26:21,320 --> 00:26:26,700 called D4, Schlafly's lattice, 1850-something. 410 00:26:26,700 --> 00:26:34,260 And its coding gain is precisely the square root of 411 00:26:34,260 --> 00:26:36,530 2, which is 1.5 dB. 412 00:26:42,965 --> 00:26:53,610 8 dimensions, it's E8, Gosset's lattice, about 1900. 413 00:26:53,610 --> 00:26:57,660 The D4 and E8 come, again, from the mathematical 414 00:26:57,660 --> 00:26:59,800 literature and have something to do with their relationship 415 00:26:59,800 --> 00:27:02,380 to certain groups. 416 00:27:02,380 --> 00:27:07,230 Lattices and finite groups are very closely connected. 417 00:27:07,230 --> 00:27:12,420 And this has a coding gain of 2 or 3 dB. 418 00:27:12,420 --> 00:27:18,100 16 dimensions, it's something just called lambda 16. 419 00:27:18,100 --> 00:27:23,010 It's a Barnes-Wall lattice, as all of these are, actually. 420 00:27:23,010 --> 00:27:26,630 And it's 2 to the 3/2, or 4.5 dB. 421 00:27:26,630 --> 00:27:28,380 You see a certain pattern here? 422 00:27:31,080 --> 00:27:36,910 In 32 dimensions, we have two lattices that I'll mention. 423 00:27:36,910 --> 00:27:43,850 There's the 32-dimensional Barnes-Wall lattice, which has 424 00:27:43,850 --> 00:27:45,100 a coding gain of 4.6 dB. 425 00:27:48,500 --> 00:27:54,330 Or there's actually Quebbeman in the '60s, I think, came up 426 00:27:54,330 --> 00:27:55,470 with a denser lattice. 427 00:27:55,470 --> 00:28:00,000 So this is still a subject of active investigation, 428 00:28:00,000 --> 00:28:02,760 particularly in dimensions like 19, where it's not so 429 00:28:02,760 --> 00:28:05,290 easy to find the densest lattice. 430 00:28:05,290 --> 00:28:08,555 And this, I forget, is maybe 6.2 dB. 431 00:28:08,555 --> 00:28:11,510 It's definitely denser. 432 00:28:11,510 --> 00:28:12,810 But it's something like that. 433 00:28:19,930 --> 00:28:22,660 This basically goes to infinity. 434 00:28:22,660 --> 00:28:25,340 You can get denser and denser lattices just by following 435 00:28:25,340 --> 00:28:28,460 this chain here which is a chain of Barnes-Wall lattices. 436 00:28:31,630 --> 00:28:35,920 The nominal coding gain can be taken to infinity, just as we 437 00:28:35,920 --> 00:28:41,100 saw with kd over n. 438 00:28:41,100 --> 00:28:44,570 That goes to infinity for rate 1/2 Reed-Muller codes. 439 00:28:47,390 --> 00:28:51,870 But of course, Shannon says we can't ever get more than 9 dB 440 00:28:51,870 --> 00:28:53,900 of effective coding gain here. 441 00:28:53,900 --> 00:28:56,810 Or actually, we can't get more than 7 and a 1/2 dB if we take 442 00:28:56,810 --> 00:28:59,150 into account that 1 and a 1/2 is going to be given by 443 00:28:59,150 --> 00:29:01,020 shaping gain. 444 00:29:01,020 --> 00:29:04,230 So what must be happening here? 445 00:29:04,230 --> 00:29:08,200 We must be having a huge number of near neighbors. 446 00:29:08,200 --> 00:29:12,250 The number of nearest neighbors in the hexagonal 447 00:29:12,250 --> 00:29:14,960 lattice is six. 448 00:29:14,960 --> 00:29:25,830 In D4 it's 24, in E8 it's 240, in lambda 16 449 00:29:25,830 --> 00:29:27,980 I think it's 4320-- 450 00:29:27,980 --> 00:29:30,840 the correct numbers are in the notes. 451 00:29:30,840 --> 00:29:37,100 By the time we get up here, we get something like a 146,680 452 00:29:37,100 --> 00:29:39,360 or something like that. 453 00:29:39,360 --> 00:29:44,640 So these lattices are designed to have a very 454 00:29:44,640 --> 00:29:46,580 high degree of symmetry. 455 00:29:46,580 --> 00:29:51,100 That means if you stand on any point, you look around you, 456 00:29:51,100 --> 00:29:53,790 they've pushed the neighboring points as far away 457 00:29:53,790 --> 00:29:55,710 from you as they can. 458 00:29:55,710 --> 00:29:58,140 But now there are a great many of them, so you get a uniform 459 00:29:58,140 --> 00:30:02,190 sky just filled with points around you, a number of 460 00:30:02,190 --> 00:30:05,100 dimensions. 461 00:30:05,100 --> 00:30:10,880 So that's what the problem is and that means that the 462 00:30:10,880 --> 00:30:12,250 effective coding gain-- 463 00:30:14,790 --> 00:30:16,630 I again do the calculation. 464 00:30:16,630 --> 00:30:20,340 I know here we lose at least two dB. 465 00:30:20,340 --> 00:30:21,590 This is something like 3.8 dB. 466 00:30:24,440 --> 00:30:30,030 OK, so the effective coding gain actually sags and begins 467 00:30:30,030 --> 00:30:31,760 to top out as you get out here. 468 00:30:35,690 --> 00:30:38,060 Maybe nowadays you could think of implementing a larger 469 00:30:38,060 --> 00:30:41,930 lattice, but I think this is the largest anyone has ever 470 00:30:41,930 --> 00:30:43,850 thought about implementing. 471 00:30:43,850 --> 00:30:47,900 And what do we get? 472 00:30:47,900 --> 00:30:49,560 A coding gain of less than 4 dB -- 473 00:30:49,560 --> 00:30:51,500 an effective coding gain. 474 00:30:51,500 --> 00:30:57,140 So if you use that, you'd be, say, 475 00:30:57,140 --> 00:30:58,870 somewhere around 5 db here. 476 00:31:01,845 --> 00:31:09,660 We go over 6 dB, but then we go up by this normalized per 477 00:31:09,660 --> 00:31:12,470 two dimensions, so we divide by 16. 478 00:31:12,470 --> 00:31:20,980 That's still 10,000, which is still 13 factors of 2, 479 00:31:20,980 --> 00:31:23,120 which is 2.6 dB. 480 00:31:23,120 --> 00:31:25,780 You can see where the problem is. 481 00:31:25,780 --> 00:31:29,980 So anyway, we get something that's really this moved way 482 00:31:29,980 --> 00:31:35,575 up a lot more, and as a result we get a much steeper curve. 483 00:31:38,300 --> 00:31:43,480 This method doesn't work up here, but this might be lambda 484 00:31:43,480 --> 00:31:47,500 16 or lambda 32. 485 00:31:47,500 --> 00:31:50,040 And then we can get another 1 and a 1/2 dB of shaping gain. 486 00:31:50,040 --> 00:31:52,500 So maybe 4 dB, optimistically. 487 00:31:59,740 --> 00:32:00,990 OK, not really. 488 00:32:04,500 --> 00:32:06,770 So that's a little disappointing. 489 00:32:06,770 --> 00:32:14,130 Just by looking up in the math textbooks, we don't do very 490 00:32:14,130 --> 00:32:17,310 striking things. 491 00:32:17,310 --> 00:32:22,470 I mention in particular the Barnes-Wall lattices, and if I 492 00:32:22,470 --> 00:32:27,080 had more time I would develop them because they're very much 493 00:32:27,080 --> 00:32:30,100 analogous to the Reed-Muller codes. 494 00:32:30,100 --> 00:32:35,060 They are constructed by a u, u plus v construction in the 495 00:32:35,060 --> 00:32:36,740 same way as Reed-Muller codes. 496 00:32:36,740 --> 00:32:41,830 They are length-doubling constructions that also double 497 00:32:41,830 --> 00:32:46,480 the distance, just as the Reed-Muller codes do now 498 00:32:46,480 --> 00:32:48,580 Euclidian distance. 499 00:32:48,580 --> 00:32:52,140 But let me just give you the first couple of sentences in a 500 00:32:52,140 --> 00:32:57,010 discussion of the Barnes-Wall lattices, which were only 501 00:32:57,010 --> 00:32:59,357 discovered in '59, about the same time as 502 00:32:59,357 --> 00:33:00,607 the Reed-Muller codes. 503 00:33:03,300 --> 00:33:06,225 Suppose we take the QAM grid like this. 504 00:33:09,770 --> 00:33:13,570 An important observation is that if we take every other 505 00:33:13,570 --> 00:33:18,570 point in sort-of checkerboard fashion, what do we get? 506 00:33:24,520 --> 00:33:26,990 Imagine it's going to infinity in both directions. 507 00:33:31,270 --> 00:33:36,940 So this is a subset of half the points, basically. 508 00:33:36,940 --> 00:33:46,490 Half the density, twice the volume of points that, itself 509 00:33:46,490 --> 00:33:48,890 is a rotated and scaled version 510 00:33:48,890 --> 00:33:50,140 of the integer lattice. 511 00:33:53,150 --> 00:33:55,470 Take the centers on a checkerboard and they form a 512 00:33:55,470 --> 00:33:59,660 45 degree rotated and square root of 2 scaled version of 513 00:33:59,660 --> 00:34:01,830 the integer lattice. 514 00:34:01,830 --> 00:34:05,320 If this had minimum squared distance 1 between points, 515 00:34:05,320 --> 00:34:07,859 then this has minimum squared distance 2 between points. 516 00:34:10,710 --> 00:34:13,639 So we could label the points red, black, 517 00:34:13,639 --> 00:34:16,770 red, black, red, black. 518 00:34:16,770 --> 00:34:21,340 If we did that, the red points would be a co-set of this 519 00:34:21,340 --> 00:34:25,020 sub-lattice and the black points would be a co-set-- 520 00:34:25,020 --> 00:34:26,429 meaning a translate-- 521 00:34:26,429 --> 00:34:29,110 of this sub-lattice. 522 00:34:29,110 --> 00:34:31,420 So let me write that as follows. 523 00:34:31,420 --> 00:34:36,620 If we take Z-squared, it has a sub-lattice which we call RZ 524 00:34:36,620 --> 00:34:39,560 squared, r for rotation. 525 00:34:39,560 --> 00:34:43,690 Rotation by the Hadamard matrix, 1, 1, 1 minus 1. 526 00:34:48,820 --> 00:34:54,230 If I write d min squared, this has d min squared at 1, this 527 00:34:54,230 --> 00:34:58,030 has twice the d min squared. 528 00:34:58,030 --> 00:35:02,230 So I partition my lattice, or any translate of this lattice, 529 00:35:02,230 --> 00:35:06,450 into two co-sets of the sub-lattice with twice the 530 00:35:06,450 --> 00:35:08,270 minimum squared distance. 531 00:35:08,270 --> 00:35:11,880 This is the first steps in what Ungerboeck called set 532 00:35:11,880 --> 00:35:14,370 partitioning. 533 00:35:14,370 --> 00:35:18,840 So it's an obvious way to partition any QAM-type signal 534 00:35:18,840 --> 00:35:21,270 set, red and black points. 535 00:35:21,270 --> 00:35:24,170 Having done that, I can do it again. 536 00:35:24,170 --> 00:35:26,190 Alright, do it again, what do I get? 537 00:35:26,190 --> 00:35:31,330 So now I'm going to take every other one of these points, 538 00:35:31,330 --> 00:35:33,760 which will leave me, say, these 4 points. 539 00:35:33,760 --> 00:35:39,410 Let me call these A, A, A, A. These points might be B, B, B, 540 00:35:39,410 --> 00:35:41,975 B, the other ones that I cut out. 541 00:35:41,975 --> 00:35:52,320 I could call these C, C, C, C and D, D, D, D. Now I've 542 00:35:52,320 --> 00:35:59,150 partitioned each of these subsets into two subsets. 543 00:35:59,150 --> 00:36:04,140 I've partitioned one into A, B and the other into C, D, such 544 00:36:04,140 --> 00:36:06,130 that - what does A look like? 545 00:36:10,200 --> 00:36:14,800 A is another translate of a lattice. 546 00:36:14,800 --> 00:36:18,110 This happens to be Z-squared scaled by 2. 547 00:36:18,110 --> 00:36:21,850 It's not rotated, it's just Z-squared scaled by 2. 548 00:36:21,850 --> 00:36:24,350 So we get down to 2 times Z-squared. 549 00:36:24,350 --> 00:36:26,020 And what's its minimum squared distance? 550 00:36:26,020 --> 00:36:26,835 AUDIENCE: 4. 551 00:36:26,835 --> 00:36:28,300 PROFESSOR: 4. 552 00:36:28,300 --> 00:36:29,705 So I've doubled the distance again. 553 00:36:39,010 --> 00:36:41,790 This is an elementary but important observation. 554 00:36:41,790 --> 00:36:44,120 I can continually go through these, 555 00:36:44,120 --> 00:36:45,840 partitioning into two subsets. 556 00:36:45,840 --> 00:36:48,910 Every time I do, I double the distance. 557 00:36:48,910 --> 00:36:54,270 I get another version of the integer lattice at a higher 558 00:36:54,270 --> 00:36:56,550 scale and, perhaps, rotated. 559 00:36:56,550 --> 00:37:02,410 So if I did it again, I would get eight subsets with minimum 560 00:37:02,410 --> 00:37:08,220 squared distance 8 between points in each subset. 561 00:37:08,220 --> 00:37:13,510 So it's just a repeated divide-by-2 scheme. 562 00:37:13,510 --> 00:37:15,650 All right, now to get the Barnes-Wall lattices, you 563 00:37:15,650 --> 00:37:18,290 remember how we got the Reed-Muller codes. 564 00:37:18,290 --> 00:37:24,210 We took, say, two codes of length 2 and we put them 565 00:37:24,210 --> 00:37:27,570 together with the u, u plus v construction and we got codes 566 00:37:27,570 --> 00:37:32,190 of length 4 that had the minimum square distance of the 567 00:37:32,190 --> 00:37:36,380 higher distance code, if we chose them properly. 568 00:37:36,380 --> 00:37:38,640 That's the same thing we do here. 569 00:37:38,640 --> 00:37:45,330 If we put together these two lattices, if we choose them-- 570 00:37:45,330 --> 00:37:50,390 well, just use the u, u plus v construction where u is from 571 00:37:50,390 --> 00:37:55,350 Z-squared and v is from RZ-squared. 572 00:37:55,350 --> 00:37:57,290 So in words, that's what you do. 573 00:37:57,290 --> 00:37:59,475 If you're interested in looking it up, look 574 00:37:59,475 --> 00:38:00,710 it up in the notes. 575 00:38:00,710 --> 00:38:07,030 What you get is a lattice, D4, that has normalized density 576 00:38:07,030 --> 00:38:10,080 halfway between that of Z-squared and RZ-squared, it 577 00:38:10,080 --> 00:38:11,340 turns out-- 578 00:38:11,340 --> 00:38:14,340 sort of analogous to the code-- and its minimum square 579 00:38:14,340 --> 00:38:16,990 distance is 2. 580 00:38:16,990 --> 00:38:20,190 Or if you put these two together, you get something 581 00:38:20,190 --> 00:38:24,635 called RD4, whose minimum square distance is 4. 582 00:38:27,690 --> 00:38:32,510 So you get something whose normalized volume is halfway 583 00:38:32,510 --> 00:38:35,650 between that of these two normalized volumes, but it has 584 00:38:35,650 --> 00:38:39,180 the same minimum distance as this guy, and that's where you 585 00:38:39,180 --> 00:38:41,460 get the coding gain from. 586 00:38:41,460 --> 00:38:44,650 This is denser but still has the same minimum distance. 587 00:38:47,730 --> 00:38:51,140 If we do this again, go out to eight dimensions, we get, it 588 00:38:51,140 --> 00:38:53,670 turns out, E8. 589 00:38:53,670 --> 00:38:58,190 And this has minimum square distance 4. 590 00:38:58,190 --> 00:39:01,860 But it has the same normalized volume normalized back to two 591 00:39:01,860 --> 00:39:05,610 dimensions as this guy, which only had minimum squared 592 00:39:05,610 --> 00:39:07,300 distance 2. 593 00:39:07,300 --> 00:39:10,360 So that's precisely why this has a nominal 594 00:39:10,360 --> 00:39:12,150 coding gain of 2. 595 00:39:12,150 --> 00:39:15,520 We've doubled the minimum square distance without 596 00:39:15,520 --> 00:39:18,700 changing its density, normalized back to two 597 00:39:18,700 --> 00:39:19,950 dimensions. 598 00:39:22,140 --> 00:39:25,530 All right, so you get all the Barnes-Wall lattices just by 599 00:39:25,530 --> 00:39:26,030 doing this. 600 00:39:26,030 --> 00:39:28,730 This turns out to be a sub-lattice of this guy. 601 00:39:28,730 --> 00:39:32,480 Not surprisingly, just as these lattices are nested, 602 00:39:32,480 --> 00:39:35,160 just as the Reed-Muller codes are nested, so you get a 603 00:39:35,160 --> 00:39:36,270 similar tableau. 604 00:39:36,270 --> 00:39:40,800 You get a similar construction that you can take out to 16 605 00:39:40,800 --> 00:39:43,190 dimensions, 32 dimensions, however many 606 00:39:43,190 --> 00:39:44,660 dimensions you want. 607 00:39:44,660 --> 00:39:48,410 And every time you do, you increase the coding gain by a 608 00:39:48,410 --> 00:39:51,810 factor of square root of 2, so the coding gain eventually 609 00:39:51,810 --> 00:39:54,500 gets driven to infinity, and that's the story on 610 00:39:54,500 --> 00:39:55,550 Barnes-Wall lattices. 611 00:39:55,550 --> 00:40:00,410 So similarly to Reed-Muller codes, these are lattices that 612 00:40:00,410 --> 00:40:04,440 are the densest known in certain small dimensions like 613 00:40:04,440 --> 00:40:06,220 4, 8, and 16. 614 00:40:06,220 --> 00:40:08,560 We don't know any denser lattices. 615 00:40:08,560 --> 00:40:12,270 You start to get up to 32, 64 dimensions, then we do know of 616 00:40:12,270 --> 00:40:14,000 denser lattices. 617 00:40:14,000 --> 00:40:16,750 But they are not nearly as nicely structured, and maybe 618 00:40:16,750 --> 00:40:22,020 these are still the best from a complexity point of view. 619 00:40:22,020 --> 00:40:27,090 You can play similar games on building trellis 620 00:40:27,090 --> 00:40:29,462 representations of these lattices, as we did for 621 00:40:29,462 --> 00:40:29,860 Reed-Muller codes. 622 00:40:29,860 --> 00:40:33,010 They have similar kinds of trellises because they're 623 00:40:33,010 --> 00:40:34,260 based on the same construction. 624 00:40:36,920 --> 00:40:39,580 So that's the story of Barnes-Wall lattices. 625 00:40:39,580 --> 00:40:44,480 The most famous lattice is the Leech lattice, which is a 626 00:40:44,480 --> 00:40:51,000 super-dense lattice in 24 dimensions. 627 00:40:51,000 --> 00:40:53,790 It has a coding gain of 6 dB. 628 00:40:58,630 --> 00:41:04,620 And unfortunately, it has 195,560 near neighbors or 629 00:41:04,620 --> 00:41:05,870 something like that. 630 00:41:09,870 --> 00:41:13,990 So you can see, it turns out it's a very close cousin of 631 00:41:13,990 --> 00:41:18,160 this 32-dimensional Barnes-Wall lattice. 632 00:41:18,160 --> 00:41:29,960 It's related to the 24-12-8 Golay code in the same way as 633 00:41:29,960 --> 00:41:34,710 the Barnes-Wall lattices are to the Reed-Muller codes. 634 00:41:34,710 --> 00:41:37,980 So the fact that the Golay code is so exceptional can be 635 00:41:37,980 --> 00:41:41,850 seen just as a projection of the fact that the Leech 636 00:41:41,850 --> 00:41:43,470 lattice is so exceptional. 637 00:41:43,470 --> 00:41:47,440 This somehow has the Golay code buried in it, if you can 638 00:41:47,440 --> 00:41:49,660 find it in here. 639 00:41:49,660 --> 00:41:54,310 And the Leech lattice is particularly notable not just 640 00:41:54,310 --> 00:41:57,890 because it has remarkably high coding gain for its 641 00:41:57,890 --> 00:42:00,810 dimension-- it's remarkably dense for its dimension, 642 00:42:00,810 --> 00:42:03,110 exceptional-- 643 00:42:03,110 --> 00:42:07,400 but also because it has remarkable symmetries, so it's 644 00:42:07,400 --> 00:42:11,060 connected to some of the largest exceptional groups, 645 00:42:11,060 --> 00:42:13,760 the monster exceptional groups. 646 00:42:13,760 --> 00:42:16,950 And it was a key step in the classification of all finite 647 00:42:16,950 --> 00:42:23,040 groups to find the symmetry group of this lattice. 648 00:42:23,040 --> 00:42:27,480 Just a little liberal arts for you. 649 00:42:27,480 --> 00:42:29,250 So that's what we can get with lattices. 650 00:42:29,250 --> 00:42:32,080 By going out to the Leech lattice or the Barnes-Wall 651 00:42:32,080 --> 00:42:35,690 lattice-- this could equally well be the Leech lattice-- 652 00:42:35,690 --> 00:42:39,540 again, effective coding gain is only about 4 dB or maybe 653 00:42:39,540 --> 00:42:42,210 even less because of the exceptional number of near 654 00:42:42,210 --> 00:42:43,460 neighbors here. 655 00:42:48,390 --> 00:42:52,440 To one digit of significance, that's approximately its 656 00:42:52,440 --> 00:42:53,690 coding gain. 657 00:42:56,720 --> 00:43:01,958 So how can we go beyond this? 658 00:43:01,958 --> 00:43:09,280 As I said already, the next great breakthrough was trellis 659 00:43:09,280 --> 00:43:12,495 codes for which Gottfried Ungerboeck became famous. 660 00:43:19,490 --> 00:43:24,480 Trellis codes, as I say in the notes, are basically-- 661 00:43:24,480 --> 00:43:27,200 first of all, Ungerboeck thought about the problem from 662 00:43:27,200 --> 00:43:29,450 a Euclidian distance point of view. 663 00:43:29,450 --> 00:43:32,860 So what you had to do to break out of 664 00:43:32,860 --> 00:43:34,822 Hamming distance thinking. 665 00:43:34,822 --> 00:43:37,490 He had the idea that in the bandwidth-limited regime, what 666 00:43:37,490 --> 00:43:40,000 you want to do is expand the constellation rather than 667 00:43:40,000 --> 00:43:41,730 expand the bandwidth. 668 00:43:41,730 --> 00:43:48,690 So to send four bits per second per Hertz. 669 00:43:48,690 --> 00:43:54,740 You could do that using uncoded 16-QAM constellation. 670 00:43:54,740 --> 00:43:59,350 Or Ungerboeck saw, well, let's make it into a 32-point 671 00:43:59,350 --> 00:44:00,430 constellation. 672 00:44:00,430 --> 00:44:03,970 Now we have some redundancy, now we can do some coding on 673 00:44:03,970 --> 00:44:05,220 this constellation. 674 00:44:07,440 --> 00:44:11,760 He used the set partitioning idea, which I've just 675 00:44:11,760 --> 00:44:14,360 explained to you. 676 00:44:14,360 --> 00:44:18,200 Initially, he just looked at a QAM constellation and he said, 677 00:44:18,200 --> 00:44:27,960 let's divide it up into A, B, A, B, C, D, C, D, A, B, A, B, 678 00:44:27,960 --> 00:44:37,350 C, D, C, D. So let's partition it from A, B, C, D into A and 679 00:44:37,350 --> 00:44:41,870 D as I've done it here, and B and C are these two subsets. 680 00:44:41,870 --> 00:44:48,320 And we'll break this up into A and D and B and C. And then 681 00:44:48,320 --> 00:44:51,340 some theoretical people came along and said, well, this is 682 00:44:51,340 --> 00:44:54,810 just Z-squared, this is RZ-squared, and this is 683 00:44:54,810 --> 00:44:56,060 2Z-squared. 684 00:45:01,330 --> 00:45:04,386 Let's, first of all, partition our enlarged constellation. 685 00:45:07,100 --> 00:45:08,905 Now here's the setup we're going to use for coding. 686 00:45:13,040 --> 00:45:15,240 We are going to bring in our ...If we're trying to send-- 687 00:45:15,240 --> 00:45:20,120 I forget what notation I used in the notes-- b bits per two 688 00:45:20,120 --> 00:45:21,370 dimensions. 689 00:45:28,570 --> 00:45:31,250 Let's take one bit per two dimensions and put it into a 690 00:45:31,250 --> 00:45:32,740 rate 1/2 convolutional code. 691 00:45:36,760 --> 00:45:38,010 So we're going to get out two bits. 692 00:45:41,450 --> 00:45:44,130 Everything here is going to be per two dimensions because 693 00:45:44,130 --> 00:45:46,610 we're going to use a two-dimensional signal set. 694 00:45:46,610 --> 00:45:50,420 And we'll bring in our other b minus 1 bits down here. 695 00:45:50,420 --> 00:45:51,890 These are called uncoded bits. 696 00:45:55,770 --> 00:45:58,370 For these two bits, we're going to go 697 00:45:58,370 --> 00:46:00,295 into a subset selector. 698 00:46:04,480 --> 00:46:07,040 Two bits are going to select among-- 699 00:46:07,040 --> 00:46:12,855 this going to be something from A, B, C, or D. It's going 700 00:46:12,855 --> 00:46:14,725 to tell us which subset to use. 701 00:46:18,780 --> 00:46:23,710 All right, then we'll use that in combination with these b 702 00:46:23,710 --> 00:46:37,990 minus 1 bits to select a point from a 2 to the b plus 1 bit 703 00:46:37,990 --> 00:46:39,240 QAM constellation. 704 00:46:41,530 --> 00:46:42,750 Two dimensions. 705 00:46:42,750 --> 00:46:47,650 So this is actually our signal point y, our two-dimensional 706 00:46:47,650 --> 00:46:50,220 signal point. 707 00:46:50,220 --> 00:46:54,590 Do you see how the numbers work out? 708 00:46:54,590 --> 00:47:00,280 We want to send b bits per QAM point, per two dimensions. 709 00:47:00,280 --> 00:47:06,410 We're going to use a 2 to the b plus 1 point constellation, 710 00:47:06,410 --> 00:47:08,050 so 32 instead of 16, say. 711 00:47:10,890 --> 00:47:20,280 Take one bit in here, generate two streams, one redundant. 712 00:47:20,280 --> 00:47:24,120 This will tell us A, B, C, D, so that's two of the bits you 713 00:47:24,120 --> 00:47:25,930 need to select one of these points. 714 00:47:29,720 --> 00:47:34,640 Say, in a 16-point constellation, we'll provide a 715 00:47:34,640 --> 00:47:36,080 bit label that looks like this. 716 00:47:36,080 --> 00:47:41,220 For A, it will be 0, 0. b is 0, 1, c is 1, 0, D is 1, 1. 717 00:47:41,220 --> 00:47:44,530 That's always the low-order bits in any selection for A, 718 00:47:44,530 --> 00:47:48,220 B, C, D. And then we'll take two high-order bits, which 719 00:47:48,220 --> 00:47:51,950 then tell us which of these four groups it's in. 720 00:47:51,950 --> 00:47:56,470 So we'll have another 0, 0, 0, 1, 1, 0, 1, 1. 721 00:47:59,140 --> 00:48:02,000 Now I'm talking about b equals 3. 722 00:48:02,000 --> 00:48:05,360 We have two bits that come in here and select one of these 723 00:48:05,360 --> 00:48:09,690 blocks, and then two bits that come out of here and select A, 724 00:48:09,690 --> 00:48:13,380 B, C, D within the block. 725 00:48:13,380 --> 00:48:15,330 There's two parts to the label. 726 00:48:15,330 --> 00:48:17,900 One of them determines A, B, C, D, the other one just 727 00:48:17,900 --> 00:48:20,880 determines some larger group. 728 00:48:26,080 --> 00:48:29,910 These bits are called uncoded because there's no protection. 729 00:48:29,910 --> 00:48:37,370 If you sent point A over here and received point A over 730 00:48:37,370 --> 00:48:39,980 here, that would look like a perfectly 731 00:48:39,980 --> 00:48:42,250 legitimate code word. 732 00:48:42,250 --> 00:48:48,490 If we make an error of that far, which is d min squared 733 00:48:48,490 --> 00:48:50,740 equals 4, say. 734 00:48:50,740 --> 00:48:56,300 We make an error of size d min equal 2 all the way over here, 735 00:48:56,300 --> 00:49:00,230 then there's no further protection from this code 736 00:49:00,230 --> 00:49:02,670 because this code only determines the sequence of As, 737 00:49:02,670 --> 00:49:03,920 Bs, Cs, and Ds. 738 00:49:09,720 --> 00:49:11,880 This is called a within-subset error. 739 00:49:11,880 --> 00:49:16,380 There's no protection on within subset errors. 740 00:49:16,380 --> 00:49:19,490 What the code does is it protects against smaller 741 00:49:19,490 --> 00:49:26,840 errors that are between subset errors like A to B, which has 742 00:49:26,840 --> 00:49:30,890 squared distance of 1 or A to D has a squared distance of 743 00:49:30,890 --> 00:49:32,380 two, and so forth. 744 00:49:39,250 --> 00:49:40,640 Are you with me? 745 00:49:40,640 --> 00:49:41,920 Nobody's asking questions. 746 00:49:41,920 --> 00:49:43,170 Maybe I haven't permitted it. 747 00:49:48,550 --> 00:49:53,140 Do you follow what the overall setup is? 748 00:49:53,140 --> 00:49:57,880 I've done everything now, except choose an exact 749 00:49:57,880 --> 00:49:58,790 constellation. 750 00:49:58,790 --> 00:50:03,500 For the exact constellation, I'll just choose one of the 751 00:50:03,500 --> 00:50:05,110 best QAM signal sets. 752 00:50:05,110 --> 00:50:08,590 If it's large enough, I'll shape it like a circle. 753 00:50:08,590 --> 00:50:13,860 Or I might even try to put Gaussian shaping on it by some 754 00:50:13,860 --> 00:50:15,900 completely extraneous thing that works 755 00:50:15,900 --> 00:50:17,175 only on these clumps. 756 00:50:19,750 --> 00:50:24,450 So shaping takes place on the higher order bits. 757 00:50:24,450 --> 00:50:27,630 You can see that, given a large enough constellation 758 00:50:27,630 --> 00:50:31,340 just by choosing these clumps, I can make the constellation 759 00:50:31,340 --> 00:50:33,085 sort of have the shape of a circle. 760 00:50:42,535 --> 00:50:45,950 The subsets of A, B, C, and D points have to all be the same 761 00:50:45,950 --> 00:50:48,460 size in order for this scheme to work. 762 00:50:48,460 --> 00:50:50,630 But I can do that within a sort of circular constellation 763 00:50:50,630 --> 00:50:53,250 if the constellation is large enough. 764 00:50:53,250 --> 00:50:58,650 Or I can put some Gaussian distribution on these clumps. 765 00:50:58,650 --> 00:51:00,340 So I do shaping down here. 766 00:51:03,490 --> 00:51:05,140 Initially, of course, Ungerboeck 767 00:51:05,140 --> 00:51:06,190 didn't worry about shaping. 768 00:51:06,190 --> 00:51:10,940 But if I try to do shaping, it's down on this index, which 769 00:51:10,940 --> 00:51:14,440 is considered to be the higher order bits that determine the 770 00:51:14,440 --> 00:51:16,950 growth structure of the constellation. 771 00:51:16,950 --> 00:51:20,030 These are the lower order bits, these two bits, and they 772 00:51:20,030 --> 00:51:22,250 determine the fine structure which is 773 00:51:22,250 --> 00:51:23,680 determined by this code. 774 00:51:23,680 --> 00:51:28,580 So the last thing I have to select is the code, and it 775 00:51:28,580 --> 00:51:33,260 turns out that the same code we used in the example is a 776 00:51:33,260 --> 00:51:34,530 good code to use here. 777 00:51:34,530 --> 00:51:42,290 Suppose we use the four-state rate 1/2 example code that 778 00:51:42,290 --> 00:51:51,290 I've used before that is everybody's favorite example 779 00:51:51,290 --> 00:51:56,010 because it has such excellent distance properties. 780 00:51:56,010 --> 00:51:57,680 So it looks like this. 781 00:52:00,290 --> 00:52:02,740 So that's my rate 1/2 encoder. 782 00:52:06,090 --> 00:52:12,660 You remember as a binary code, it had minimum Hamming 783 00:52:12,660 --> 00:52:14,020 distance 5. 784 00:52:14,020 --> 00:52:17,800 And the only minimum distance code word 785 00:52:17,800 --> 00:52:21,430 was the impulse response. 786 00:52:21,430 --> 00:52:23,290 Let me write the impulse response as this 787 00:52:23,290 --> 00:52:24,540 way: 11, 01, 11. 788 00:52:26,380 --> 00:52:30,200 This is what we get out at time 0, time 1, and time 2. 789 00:52:38,510 --> 00:52:42,340 Let me assign the following labels to these subsets. 790 00:52:42,340 --> 00:52:46,370 We'll make that 00, this one 11, this one 01, 791 00:52:46,370 --> 00:52:47,620 and that one 10. 792 00:52:49,640 --> 00:52:53,185 In other words, this is the complement of that. 793 00:52:53,185 --> 00:52:54,545 This is the complement of that. 794 00:53:05,690 --> 00:53:10,680 What is the nearest neighbor point in Euclidian space for 795 00:53:10,680 --> 00:53:13,300 this sequence? 796 00:53:13,300 --> 00:53:16,080 If I complement the first two bits coming out, then I'm 797 00:53:16,080 --> 00:53:24,020 going to go from one of these subsets to another, but within 798 00:53:24,020 --> 00:53:27,590 one of these RZ-squared subsets, which has minimum 799 00:53:27,590 --> 00:53:30,650 squared distance 2. 800 00:53:30,650 --> 00:53:33,460 So let's remind ourselves that the minimum squared distance 801 00:53:33,460 --> 00:53:38,480 within A, B, C, D is 1, within A, D or B, C is 2, within 802 00:53:38,480 --> 00:53:42,470 2Z-squared is 4. 803 00:53:42,470 --> 00:53:43,920 So I'm claiming-- 804 00:53:43,920 --> 00:53:47,200 there's actually a problem on this on Problem Set 10 if you 805 00:53:47,200 --> 00:53:49,900 tried to do it-- 806 00:53:49,900 --> 00:54:00,780 that any error of this type, 11, is going to cause me a 807 00:54:00,780 --> 00:54:05,620 Euclidian error of minimum squared distance 2, at least. 808 00:54:05,620 --> 00:54:09,830 It can only cause me to go from A to D, which is always 809 00:54:09,830 --> 00:54:15,850 at least squared distance 2, or from B to C, or vice versa. 810 00:54:15,850 --> 00:54:20,720 So in terms of Euclidian distance, d min squared, I'm 811 00:54:20,720 --> 00:54:25,200 always going to get a squared distance of 2 at this point. 812 00:54:25,200 --> 00:54:28,200 And actually, any place you diverged in the trellis, 813 00:54:28,200 --> 00:54:29,810 because of the structure of the code, this 814 00:54:29,810 --> 00:54:32,210 is going to be true. 815 00:54:32,210 --> 00:54:37,800 I'm always going to get a squared distance of 2 when two 816 00:54:37,800 --> 00:54:42,110 trellis paths come together in the trellis. 817 00:54:42,110 --> 00:54:45,770 I always have a binary difference of 11, so by the 818 00:54:45,770 --> 00:54:48,140 same argument I'm always going to get a Euclidian squared 819 00:54:48,140 --> 00:54:52,250 distance of 2 when two trellis paths come together. 820 00:54:52,250 --> 00:54:55,070 And furthermore, I'm going to have at least one difference 821 00:54:55,070 --> 00:54:58,790 somewhere in between, which is going to, again, in terms of 822 00:54:58,790 --> 00:55:00,730 Euclidian distance, lead to a Euclidian 823 00:55:00,730 --> 00:55:05,250 distance of at least 1. 824 00:55:05,250 --> 00:55:12,080 So the conclusion is that between any two sequences of 825 00:55:12,080 --> 00:55:16,920 constellation points that are labeled by two distinct code 826 00:55:16,920 --> 00:55:21,720 words here, I'm going to have a cumulative Euclidian squared 827 00:55:21,720 --> 00:55:25,520 distance of at least 5, by the same argument as I had for the 828 00:55:25,520 --> 00:55:26,440 binary code. 829 00:55:26,440 --> 00:55:32,740 But now it turns out that I'm still in a regime where binary 830 00:55:32,740 --> 00:55:37,140 distance, Hamming distance translates directly to 831 00:55:37,140 --> 00:55:38,450 Euclidian distance. 832 00:55:38,450 --> 00:55:41,370 This is what you saw. 833 00:55:41,370 --> 00:55:44,760 Basically, what we're using here is this is Z-squared, you 834 00:55:44,760 --> 00:55:49,420 can think of as the image of the (2, 2) binary code. 835 00:55:49,420 --> 00:55:54,360 RZ-squared is the image of the (2, 1) binary code, and this 836 00:55:54,360 --> 00:55:58,330 2Z-squared is the image of the (2, 0) binary code. 837 00:55:58,330 --> 00:56:02,370 You get the same partition structure for these codes and 838 00:56:02,370 --> 00:56:04,390 their co-sets. 839 00:56:04,390 --> 00:56:07,830 This is the set of all four two-tuples. 840 00:56:07,830 --> 00:56:10,050 We divided it into two subsets, each of minimum 841 00:56:10,050 --> 00:56:13,530 distance 2 here, and here we divide it into four subsets, 842 00:56:13,530 --> 00:56:14,785 each of minimum distance infinity. 843 00:56:17,330 --> 00:56:21,240 I'm not going to take the time to make this precise, but 844 00:56:21,240 --> 00:56:24,540 think of how that relation goes. 845 00:56:24,540 --> 00:56:28,200 So I've assured myself of a Euclidian squared distance at 846 00:56:28,200 --> 00:56:32,300 least 5 between signal point sequences that correspond to 847 00:56:32,300 --> 00:56:33,550 two different code words. 848 00:56:35,900 --> 00:56:37,890 That's excellent. 849 00:56:37,890 --> 00:56:39,910 Is the minimum squared distance of the code then 850 00:56:39,910 --> 00:56:43,190 equal to 5, as it was in the binary case of 851 00:56:43,190 --> 00:56:44,400 this trellis code? 852 00:56:44,400 --> 00:56:46,200 No, it's not. 853 00:56:46,200 --> 00:56:49,910 And the problem is I could make this kind of error, which 854 00:56:49,910 --> 00:56:52,660 is completely invisible to the code. 855 00:56:52,660 --> 00:56:55,470 I can make a within subset error, and this only has 856 00:56:55,470 --> 00:56:59,220 minimum square distance of 4. 857 00:56:59,220 --> 00:57:04,500 So the conclusion of this analysis is that this is 858 00:57:04,500 --> 00:57:08,930 what's called a four-state two-dimensional trellis code. 859 00:57:08,930 --> 00:57:11,620 This four-state two-dimensional trellis code 860 00:57:11,620 --> 00:57:15,430 has a minimum squared distance of 4, and it has a number of 861 00:57:15,430 --> 00:57:18,430 nearest neighbors per two dimension of 4. 862 00:57:18,430 --> 00:57:21,600 That's the number of nearest neighbors within 2Z-squared, 863 00:57:21,600 --> 00:57:25,320 which is one of these subsets. 864 00:57:25,320 --> 00:57:30,580 So the conclusion is that already, this has a legitimate 865 00:57:30,580 --> 00:57:32,300 3 dB coding gain. 866 00:57:32,300 --> 00:57:37,020 Its coefficient is still just the QAM coefficient, and this 867 00:57:37,020 --> 00:57:42,170 very simple code has this performance curve moved over. 868 00:57:46,490 --> 00:57:48,695 So it's 3 dB all the way up the line. 869 00:57:52,440 --> 00:57:54,600 So that's pretty good. 870 00:57:54,600 --> 00:57:57,800 This is a trivial code that could be decoding-- 871 00:57:57,800 --> 00:58:00,120 by the way, it was, again, done by the Viterbi algorithm 872 00:58:00,120 --> 00:58:02,960 of a four-state trellis. 873 00:58:02,960 --> 00:58:09,380 You just take the sequences and it's exactly the same. 874 00:58:09,380 --> 00:58:14,960 You measure distance in terms of minimum squared distance. 875 00:58:14,960 --> 00:58:25,640 The only difference is when you decode, you first find 876 00:58:25,640 --> 00:58:27,520 what the closest point is within the 877 00:58:27,520 --> 00:58:30,500 A, B, C and D subsets. 878 00:58:30,500 --> 00:58:32,910 That is what you're going to take as your representative 879 00:58:32,910 --> 00:58:33,920 for that subset. 880 00:58:33,920 --> 00:58:38,250 So you first make basically an uncoded decision, which is the 881 00:58:38,250 --> 00:58:39,190 closest point in A? 882 00:58:39,190 --> 00:58:41,750 Which is the closest point in B, which is the closest point 883 00:58:41,750 --> 00:58:45,090 in C, which is the closest point in D? 884 00:58:45,090 --> 00:58:46,750 You're going to get a metric for each of those. 885 00:58:46,750 --> 00:58:49,470 How far is each of those from the receive point? 886 00:58:49,470 --> 00:58:52,810 So then in trellis decoding, what the trellis is going to 887 00:58:52,810 --> 00:58:55,880 look like is something like this. 888 00:58:55,880 --> 00:59:03,380 A, D, A, D, B, C, and so forth. 889 00:59:09,490 --> 00:59:13,860 The trellis is going to look exactly the same, but here you 890 00:59:13,860 --> 00:59:18,840 get A, D, D, A, B, C, C, B. 891 00:59:18,840 --> 00:59:21,290 So what metric are we going to use for 892 00:59:21,290 --> 00:59:25,090 each of these branches? 893 00:59:25,090 --> 00:59:28,210 First, we find the closest point within each subset, and 894 00:59:28,210 --> 00:59:30,670 then we use the metric for that point. 895 00:59:30,670 --> 00:59:34,540 Clearly, if we're going to choose A for this time 896 00:59:34,540 --> 00:59:40,020 interval, then we should use the closest point in A as our 897 00:59:40,020 --> 00:59:41,980 receive decision. 898 00:59:41,980 --> 00:59:46,380 We can get that survivor without any memory whatsoever 899 00:59:46,380 --> 00:59:51,630 when we make that decision, and then we take the sequences 900 00:59:51,630 --> 00:59:55,180 of representatives and find the best of those 901 00:59:55,180 --> 00:59:57,570 representatives as our best trellis-coded sequence. 902 01:00:00,880 --> 01:00:03,830 Any sequence of points that has labels that correspond to 903 01:00:03,830 --> 01:00:06,210 some path through this trellis is a valid 904 01:00:06,210 --> 01:00:07,560 point and vice versa. 905 01:00:07,560 --> 01:00:10,660 If and only if the sequence of labels is according to this 906 01:00:10,660 --> 01:00:15,920 trellis is it a legitimate trellis-coded point. 907 01:00:15,920 --> 01:00:17,170 So it's a nice amalgamation. 908 01:00:22,640 --> 01:00:23,220 Great. 909 01:00:23,220 --> 01:00:27,580 With a simple four-state code, basically we've doubled the 910 01:00:27,580 --> 01:00:28,920 size the constellation. 911 01:00:28,920 --> 01:00:32,850 The constellation expansion is 2, the constellation expansion 912 01:00:32,850 --> 01:00:36,230 factor with twice as large a constellation, we've been able 913 01:00:36,230 --> 01:00:40,420 to achieve a coding gain of 3 dB. 914 01:00:40,420 --> 01:00:43,110 Not bad. 915 01:00:43,110 --> 01:00:46,730 So how do we extend this idea? 916 01:00:46,730 --> 01:00:50,710 First, extension would be to-- 917 01:00:50,710 --> 01:00:53,060 all right, let's go up to an 8 state, 16 918 01:00:53,060 --> 01:00:54,340 state, whatever code. 919 01:00:54,340 --> 01:00:56,960 But is that going to improve our minimum distance using 920 01:00:56,960 --> 01:01:00,310 this block diagram? 921 01:01:00,310 --> 01:01:02,710 It's not, because we're still stuck with this 922 01:01:02,710 --> 01:01:03,950 within subset distance. 923 01:01:03,950 --> 01:01:08,550 So we can never get past d min squared equals 4 with a 924 01:01:08,550 --> 01:01:10,920 four-way partition. 925 01:01:10,920 --> 01:01:13,490 What we have to do is make one more level of partition and we 926 01:01:13,490 --> 01:01:16,830 need to make an eight way partition, at which point 927 01:01:16,830 --> 01:01:19,280 we're going to get a within subset distance of 8. 928 01:01:29,960 --> 01:01:31,810 I left out one step, I'm sorry. 929 01:01:31,810 --> 01:01:33,060 I'll come back to it. 930 01:01:36,130 --> 01:01:40,520 We can get a code minimum squared distance up to 8, if 931 01:01:40,520 --> 01:01:43,260 we extend the minimum squared distance up to 8. 932 01:01:43,260 --> 01:01:47,830 The thing I didn't use, I didn't tell you before, was in 933 01:01:47,830 --> 01:01:51,570 computing the coding gain of this code-- 934 01:01:51,570 --> 01:01:54,190 let's just call it c-- 935 01:01:54,190 --> 01:01:55,100 what should it be? 936 01:01:55,100 --> 01:01:57,950 It should be the minimum squared distance of the code 937 01:01:57,950 --> 01:02:01,970 over the volume of the code per two dimensions. 938 01:02:01,970 --> 01:02:04,990 In this case, n is 2 so I'm just going to do it over the 939 01:02:04,990 --> 01:02:06,310 volume of the code. 940 01:02:06,310 --> 01:02:10,420 Now what is the volume of a convolutional code? 941 01:02:10,420 --> 01:02:17,030 This is actually an interesting problem, because 942 01:02:17,030 --> 01:02:20,480 the Voronoi region of a trellis code is actually an 943 01:02:20,480 --> 01:02:23,030 infinite dimensional object. 944 01:02:23,030 --> 01:02:25,400 A trellis code is really an infinite dimensional lattice. 945 01:02:25,400 --> 01:02:28,490 It's a lattice in infinite dimensional space. 946 01:02:28,490 --> 01:02:33,040 And if you look at the Voronoi region, it extends out into 947 01:02:33,040 --> 01:02:34,820 all these dimensions. 948 01:02:34,820 --> 01:02:36,890 It might not for such a simple code, but in 949 01:02:36,890 --> 01:02:39,930 principle it does. 950 01:02:39,930 --> 01:02:42,590 However, you can find other fundamental regions. 951 01:02:42,590 --> 01:02:47,040 The way to think about this is instead of 952 01:02:47,040 --> 01:02:50,040 putting the volume here-- 953 01:02:50,040 --> 01:02:52,840 well, what have we cost ourselves in volume? 954 01:02:52,840 --> 01:02:55,650 We've doubled the size of the constellation. 955 01:02:55,650 --> 01:02:57,580 And in two dimensions, that means 956 01:02:57,580 --> 01:02:58,830 we've doubled the volume. 957 01:03:02,510 --> 01:03:05,910 So we've quadrupled the minimum distance at the cost 958 01:03:05,910 --> 01:03:10,230 of doubling the volume. 959 01:03:10,230 --> 01:03:13,160 We've covered twice as much space now because we have 960 01:03:13,160 --> 01:03:15,610 twice as many points. 961 01:03:15,610 --> 01:03:18,050 And so, that's where we get the-- 962 01:03:18,050 --> 01:03:21,710 this is equal to, for this code, that's 4. 963 01:03:21,710 --> 01:03:25,750 We have a factor of 2 loss in volume, and so that's where we 964 01:03:25,750 --> 01:03:29,795 get what I said was a coding gain of 2 or 3 dB. 965 01:03:34,270 --> 01:03:37,280 Just to get you to the bottom line, we should replace the 966 01:03:37,280 --> 01:03:40,230 volume by two times-- 967 01:03:40,230 --> 01:03:44,720 I call it eta of the code-- where eta of the code is its 968 01:03:44,720 --> 01:03:51,620 redundancy per two dimensions. 969 01:03:51,620 --> 01:03:54,390 In this case, we introduce one redundant bit per two 970 01:03:54,390 --> 01:03:57,400 dimensions, and that's why the constellation has to expand by 971 01:03:57,400 --> 01:03:59,970 a factor of 2. 972 01:03:59,970 --> 01:04:03,840 You see, we get the right answer. 973 01:04:03,840 --> 01:04:08,050 The redundancy has increased by one bit, so this has to be 974 01:04:08,050 --> 01:04:10,790 2 to the b plus 1, so the constellation has to be twice 975 01:04:10,790 --> 01:04:14,220 as large, so the volume is twice as large. 976 01:04:14,220 --> 01:04:17,360 I'll just hand-wave and tell you that's the factor you want 977 01:04:17,360 --> 01:04:19,670 down here representing volume -- 978 01:04:19,670 --> 01:04:21,680 2 to the redundancy. 979 01:04:21,680 --> 01:04:27,992 And you can prove this by finding a region which you can 980 01:04:27,992 --> 01:04:30,870 show is a fundamental region of the trellis code and has 981 01:04:30,870 --> 01:04:32,030 this volume. 982 01:04:32,030 --> 01:04:35,730 But I won't go into that. 983 01:04:35,730 --> 01:04:39,000 OK, so that's the calculation which I should've mentioned 984 01:04:39,000 --> 01:04:41,090 before, I skipped right over it. 985 01:04:41,090 --> 01:04:42,220 Back to what I was saying. 986 01:04:42,220 --> 01:04:46,450 To improve trellis codes, the first thing you can do is to, 987 01:04:46,450 --> 01:04:49,760 say, go to an eight-way partition. 988 01:04:49,760 --> 01:04:53,270 So now we're going to have eight subsets. 989 01:04:53,270 --> 01:04:56,140 So we're going to need three bits here. 990 01:04:56,140 --> 01:05:01,180 So let's use two bits here and use a rate 2/3 trellis code. 991 01:05:01,180 --> 01:05:04,450 We're going to have three bits coming out here to select 992 01:05:04,450 --> 01:05:08,610 subsets, and we're going to have b minus 2 bits down here 993 01:05:08,610 --> 01:05:14,620 to select a point still a 2b plus 1 point QAM 994 01:05:14,620 --> 01:05:16,060 constellation. 995 01:05:16,060 --> 01:05:19,220 So our redundancy per two dimensions is still one bit 996 01:05:19,220 --> 01:05:20,120 per two dimensions. 997 01:05:20,120 --> 01:05:23,540 We're still going to lose a factor of 2, but now we can go 998 01:05:23,540 --> 01:05:27,570 up to d min squared equals 8 and we can go up to a coding 999 01:05:27,570 --> 01:05:29,690 gain of 6 dB. 1000 01:05:29,690 --> 01:05:33,910 So in his original paper, Ungerboeck did this. 1001 01:05:33,910 --> 01:05:39,860 He showed codes going up to 256 states in two dimensions 1002 01:05:39,860 --> 01:05:44,320 that gets you up to a nominal 6 dB coding gain. 1003 01:05:44,320 --> 01:05:48,390 And because they have low coefficients here, they get 1004 01:05:48,390 --> 01:05:51,350 very close to 6 dB of effective coding gain. 1005 01:05:54,610 --> 01:05:59,050 Already in Ungerboeck, we have codes-- 1006 01:05:59,050 --> 01:06:03,510 I mean, eventually you can get back to just the within-subset 1007 01:06:03,510 --> 01:06:07,930 coefficient again, which is always 4. 1008 01:06:07,930 --> 01:06:14,860 You can get up to, say, 6 dB with maybe 512 states of 1009 01:06:14,860 --> 01:06:21,160 effective coding gain, which is considerably better than 1010 01:06:21,160 --> 01:06:27,250 the densest lattice we would think of using, and also quite 1011 01:06:27,250 --> 01:06:28,220 reasonable to implement. 1012 01:06:28,220 --> 01:06:31,500 Now we're going to need a 256, 512-state 1013 01:06:31,500 --> 01:06:33,250 Viterbi algorithm decoder. 1014 01:06:38,510 --> 01:06:43,040 This was a stunning breakthrough. 1015 01:06:43,040 --> 01:06:48,360 Immediately in 1993, everybody started building turbo codes. 1016 01:06:48,360 --> 01:06:50,360 Back in 1982, everybody started 1017 01:06:50,360 --> 01:06:51,970 building trellis codes. 1018 01:06:51,970 --> 01:06:56,820 It happened to be the time of the V.32 modem, and everybody 1019 01:06:56,820 --> 01:07:00,680 said boy, this is just what we need to get an additional 3 or 1020 01:07:00,680 --> 01:07:04,115 4 dB coding gain in the V.32 modem. 1021 01:07:04,115 --> 01:07:07,680 What was actually done there is this was simply an 1022 01:07:07,680 --> 01:07:13,690 eight-state convolutional code, so it was rate 2/3. 1023 01:07:13,690 --> 01:07:15,230 And there was tweaking of this. 1024 01:07:15,230 --> 01:07:19,810 It was made rotationally invariant, made nonlinear. 1025 01:07:19,810 --> 01:07:23,840 And a lot of activity, which was very nice, stimulated by 1026 01:07:23,840 --> 01:07:25,130 the standards effort. 1027 01:07:25,130 --> 01:07:27,640 But that was basically the state of the art in the '80s. 1028 01:07:30,270 --> 01:07:36,070 The one other improvement that was made was 1029 01:07:36,070 --> 01:07:38,650 the following idea. 1030 01:07:38,650 --> 01:07:41,610 We really don't like doubling the size of the constellation, 1031 01:07:41,610 --> 01:07:45,060 particularly for a large one, mostly for 1032 01:07:45,060 --> 01:07:48,160 implementation reasons. 1033 01:07:48,160 --> 01:07:51,460 There were some feelings against nonlinearities a 1034 01:07:51,460 --> 01:07:53,920 larger constellation would cost you. 1035 01:07:53,920 --> 01:07:58,190 You wanted to minimize the constellation expansion ratio. 1036 01:07:58,190 --> 01:08:02,730 So how were we ever going to do that? 1037 01:08:02,730 --> 01:08:06,610 You can't do it this way in two dimensions. 1038 01:08:06,610 --> 01:08:09,255 The way to do that is to go up to four dimensions. 1039 01:08:12,270 --> 01:08:15,140 So let's do subset partitioning in four 1040 01:08:15,140 --> 01:08:16,390 dimensions. 1041 01:08:18,270 --> 01:08:20,760 So now think of four dimensional constellations. 1042 01:08:20,760 --> 01:08:24,270 In practice, these are just built from the Cartesian 1043 01:08:24,270 --> 01:08:27,790 product of two two-dimensional constellations. 1044 01:08:27,790 --> 01:08:29,830 We start with Z4. 1045 01:08:29,830 --> 01:08:32,939 The next one down is D4, which has a minimum 1046 01:08:32,939 --> 01:08:34,079 square distance of 2. 1047 01:08:34,079 --> 01:08:37,390 It's sometimes called a checkerboard constellation in 1048 01:08:37,390 --> 01:08:38,180 four dimensions. 1049 01:08:38,180 --> 01:08:41,729 It's just the red-black idea again. 1050 01:08:41,729 --> 01:08:46,010 Then below that, we have a four-way partition into RZ4, 1051 01:08:46,010 --> 01:08:51,300 which still has minimum distance 2. 1052 01:08:51,300 --> 01:08:56,279 And one more level of partitioning here into 1053 01:08:56,279 --> 01:09:02,559 something into RD4, which has minimum squared distance of 4. 1054 01:09:06,950 --> 01:09:10,160 So this is an eight-way partition of a 1055 01:09:10,160 --> 01:09:11,930 four-dimensional constellation. 1056 01:09:11,930 --> 01:09:14,189 This idea was introduced by Lee-Fang Wei. 1057 01:09:17,290 --> 01:09:22,140 Again, this is an interesting column over here. 1058 01:09:22,140 --> 01:09:25,200 You can think of this as being the (4, 4, 1) code. 1059 01:09:25,200 --> 01:09:30,376 This is the Euclidian image of the (4, 3, 2) code. 1060 01:09:30,376 --> 01:09:32,930 It kind of expanded in the lattice. 1061 01:09:32,930 --> 01:09:35,870 We know we can't do any better than (4, 2, 2). 1062 01:09:35,870 --> 01:09:39,120 Down here, this is the (4, 1, 4) code. 1063 01:09:39,120 --> 01:09:42,870 So again, this is the same partition as we get for binary 1064 01:09:42,870 --> 01:09:44,993 codes with the distances-- 1065 01:09:47,810 --> 01:09:52,609 again, for binary codes and this level of partitioning, 1066 01:09:52,609 --> 01:09:56,440 Hamming distance translates to Euclidian distance. 1067 01:09:56,440 --> 01:10:01,480 It's really just the image of these codes as you go up to 1068 01:10:01,480 --> 01:10:02,730 the lattices. 1069 01:10:05,600 --> 01:10:10,220 Now in this case, what's the best we can ever do? 1070 01:10:10,220 --> 01:10:10,910 Coding gain. 1071 01:10:10,910 --> 01:10:16,800 We still can't go up to a squared distance of greater 1072 01:10:16,800 --> 01:10:22,830 than 4 because our within-subset distance is 1073 01:10:22,830 --> 01:10:25,410 going to be within RD4. 1074 01:10:25,410 --> 01:10:28,240 So this is still limited to 4. 1075 01:10:28,240 --> 01:10:31,850 But now, what's our redundancy per two dimensions? 1076 01:10:31,850 --> 01:10:37,210 Here, say, we're going to use still a rate 2/3 1077 01:10:37,210 --> 01:10:38,490 convolutional code. 1078 01:10:38,490 --> 01:10:42,570 We'll now have an eight-subset selector in 4D, so everything 1079 01:10:42,570 --> 01:10:45,800 is per 4D here. 1080 01:10:45,800 --> 01:10:55,380 So this is going to be 2b minus two bits and this is 1081 01:10:55,380 --> 01:11:00,450 going to be, it turns out, a constellation that is twice as 1082 01:11:00,450 --> 01:11:02,010 large in four dimensions. 1083 01:11:02,010 --> 01:11:05,260 We still have one redundant bit per four dimensions, so we 1084 01:11:05,260 --> 01:11:08,050 need twice the size of the constellation in four 1085 01:11:08,050 --> 01:11:09,660 dimensions. 1086 01:11:09,660 --> 01:11:14,670 But that's only 1/2 a bit of redundancy per two dimensions. 1087 01:11:14,670 --> 01:11:18,620 So this denominator becomes 2 to the 1/2. 1088 01:11:18,620 --> 01:11:21,940 Waving hands furiously here. 1089 01:11:21,940 --> 01:11:26,990 And so this could go up to 2 to the 3/2, which is 4.5 dB. 1090 01:11:31,360 --> 01:11:35,800 So there's actually a particularly nice eight-state 1091 01:11:35,800 --> 01:11:41,620 code here that Wei came up with which, really, in terms 1092 01:11:41,620 --> 01:11:45,530 of performance versus complexity is the only code 1093 01:11:45,530 --> 01:11:48,510 that anyone has ever come up with that is distinctly better 1094 01:11:48,510 --> 01:11:50,855 than any of Ungerboeck's original codes. 1095 01:11:50,855 --> 01:11:54,040 All of Ungerboeck's original codes were in one dimension, 1096 01:11:54,040 --> 01:11:56,280 two dimensions, or PSK codes. 1097 01:11:56,280 --> 01:11:58,870 You could also do this on phase shift key 1098 01:11:58,870 --> 01:12:00,120 constellations. 1099 01:12:02,140 --> 01:12:09,260 So with the Wei code, it gives you a very simple way 1100 01:12:09,260 --> 01:12:10,840 to get 4 1/2 dB. 1101 01:12:13,750 --> 01:12:18,770 Actually, you have to go up to 16 states to get 4 1/2 dB. 1102 01:12:18,770 --> 01:12:21,290 The Wei code is only 4.2 dB. 1103 01:12:21,290 --> 01:12:24,900 And this is basically what was adopted for standards in the 1104 01:12:24,900 --> 01:12:25,810 early '90s. 1105 01:12:25,810 --> 01:12:33,470 The V.34 modem standard, also the ADSL standard has the same 1106 01:12:33,470 --> 01:12:34,020 code in it. 1107 01:12:34,020 --> 01:12:38,230 If you have DSL at home, that's what it's using in it. 1108 01:12:38,230 --> 01:12:43,460 Again, with a very simple code, the eight-state Viterbi 1109 01:12:43,460 --> 01:12:47,680 algorithm, you can get 4.2 dB of coding gain. 1110 01:12:47,680 --> 01:12:53,530 The error coefficient in this case, this has 24 nearest 1111 01:12:53,530 --> 01:12:57,110 neighbors in D4, or 12 per two dimensions. 1112 01:12:57,110 --> 01:13:05,460 So you lose 0.15 dB or something. 1113 01:13:05,460 --> 01:13:06,735 I'm sorry, 0.3 dB. 1114 01:13:11,800 --> 01:13:14,860 Anyway, this is a very nice code. 1115 01:13:21,460 --> 01:13:24,640 There are 32 and 64-state codes in V.34. 1116 01:13:24,640 --> 01:13:26,610 They don't gain you very much. 1117 01:13:26,610 --> 01:13:29,820 They each double the complexity and only gain you 1118 01:13:29,820 --> 01:13:33,710 about 0.2 dB for each one. 1119 01:13:33,710 --> 01:13:35,430 All these things tend to saturate. 1120 01:13:35,430 --> 01:13:39,760 You can't get more than about 5 to 6 dB of coding gain with 1121 01:13:39,760 --> 01:13:41,700 a reasonable size trellis code. 1122 01:13:45,790 --> 01:13:48,500 So where are we now? 1123 01:13:48,500 --> 01:13:55,610 We can get 5 or 6 dB of coding gain with, say, a 256-state 1124 01:13:55,610 --> 01:13:57,420 trellis code. 1125 01:13:57,420 --> 01:14:03,360 We can get another 1 to 1 and a 1/2 dB of shaping gain. 1126 01:14:03,360 --> 01:14:09,590 So all together, we're up to 6 to 7.5 dB. 1127 01:14:09,590 --> 01:14:15,580 From here, we're up somewhere into the 2 to 3 dB range. 1128 01:14:15,580 --> 01:14:23,980 In other words, combining trellis code with, say, 256 1129 01:14:23,980 --> 01:14:33,300 states with some shaping scheme with maybe 1 dB of 1130 01:14:33,300 --> 01:14:41,520 shaping gain, this gets you maybe 5 dB or 5.5 dB. 1131 01:14:41,520 --> 01:14:51,870 We get 6 dB total effective gain, and we're still 3 dB or 1132 01:14:51,870 --> 01:14:55,140 maybe 2 or 3 dB from Shannon limit. 1133 01:15:05,440 --> 01:15:09,840 So, close but not quite there yet. 1134 01:15:09,840 --> 01:15:13,250 Five minutes to go. 1135 01:15:13,250 --> 01:15:16,210 So in the notes, there's one page on 1136 01:15:16,210 --> 01:15:17,590 higher performance schemes. 1137 01:15:17,590 --> 01:15:20,340 There are higher performance schemes. 1138 01:15:20,340 --> 01:15:23,680 There are turbo codes for the bandwidth-limited regime that 1139 01:15:23,680 --> 01:15:27,130 get you to within 1 dB of the Shannon limit. 1140 01:15:27,130 --> 01:15:30,700 Sequential decoding gets you to within about 1 and a 1/2 dB 1141 01:15:30,700 --> 01:15:33,260 of the Shannon limit. 1142 01:15:33,260 --> 01:15:34,510 The idea of multi-level codes -- 1143 01:15:37,450 --> 01:15:42,260 you can regard each of these two-way partitions as being 1144 01:15:42,260 --> 01:15:45,340 specified by one bit. 1145 01:15:45,340 --> 01:15:48,570 So with three bits of labels, you can specify 1146 01:15:48,570 --> 01:15:51,150 an eight-way partition. 1147 01:15:51,150 --> 01:15:54,100 Another idea is to code on each of these levels 1148 01:15:54,100 --> 01:15:55,500 separately. 1149 01:15:55,500 --> 01:15:58,300 Use a binary code on each of these levels separately. 1150 01:15:58,300 --> 01:16:04,080 Each of these corresponds to an effective channel that has 1151 01:16:04,080 --> 01:16:05,410 different characteristics. 1152 01:16:05,410 --> 01:16:09,350 You've basically got to take the Gaussian distribution and 1153 01:16:09,350 --> 01:16:13,450 alias it around in accord with whatever this partition 1154 01:16:13,450 --> 01:16:14,300 actually is. 1155 01:16:14,300 --> 01:16:19,030 But you can come up with an effective binary channel at 1156 01:16:19,030 --> 01:16:20,090 each of these levels. 1157 01:16:20,090 --> 01:16:24,240 It turns out to be a binary symmetric input channel, 1158 01:16:24,240 --> 01:16:27,490 always in these cases. 1159 01:16:27,490 --> 01:16:30,960 So you could think of taking a binary-- 1160 01:16:30,960 --> 01:16:33,690 now, each one of these channels is going to have its 1161 01:16:33,690 --> 01:16:36,130 own capacity. 1162 01:16:36,130 --> 01:16:41,660 It turns out that information theoretically, the capacity of 1163 01:16:41,660 --> 01:16:46,600 multi-level coding on each of these levels independently is 1164 01:16:46,600 --> 01:16:49,460 the same as the aggregate capacity of coding it on the 1165 01:16:49,460 --> 01:16:50,645 whole channel. 1166 01:16:50,645 --> 01:16:55,450 It's a lovely separability result. 1167 01:16:55,450 --> 01:17:00,220 So in principle, if you could code at capacity on each of 1168 01:17:00,220 --> 01:17:04,150 these binary channels, then you could get to the capacity 1169 01:17:04,150 --> 01:17:08,330 of the aggregate channel, with a large enough constellation. 1170 01:17:08,330 --> 01:17:09,400 So that's a good idea. 1171 01:17:09,400 --> 01:17:14,470 We now know a lot of binary capacity-approaching code, so 1172 01:17:14,470 --> 01:17:17,860 let's use a turbo code or a low-density parity-check code 1173 01:17:17,860 --> 01:17:22,330 at each level, an appropriate code. 1174 01:17:22,330 --> 01:17:24,950 It turns out, as you might expect, that the first level 1175 01:17:24,950 --> 01:17:26,910 is the really hard level. 1176 01:17:26,910 --> 01:17:31,470 In this case, you might have a rate 1/3 or 1/2 code and 1177 01:17:31,470 --> 01:17:33,750 really work hard with your code. 1178 01:17:33,750 --> 01:17:38,030 At the next level, the effective error probability is 1179 01:17:38,030 --> 01:17:40,680 much better. 1180 01:17:40,680 --> 01:17:43,530 Making hard decisions, you might have a probability of 1181 01:17:43,530 --> 01:17:47,480 1/10 or lower of making an error. 1182 01:17:47,480 --> 01:17:49,265 So these are really much easier problems. 1183 01:17:49,265 --> 01:17:52,230 It might be a better idea at these levels just to use 1184 01:17:52,230 --> 01:17:55,940 algebraic coding or something that works very well. 1185 01:17:55,940 --> 01:17:59,450 The capacity is nearly one per bit at each level, so you want 1186 01:17:59,450 --> 01:18:02,780 to code with very little redundancy at these next 1187 01:18:02,780 --> 01:18:06,410 levels, but it's still capable of approaching capacity. 1188 01:18:06,410 --> 01:18:09,690 Well, it turns out this is a good scenario for Reed-Solomon 1189 01:18:09,690 --> 01:18:12,980 codes or some kind of algebraic code. 1190 01:18:12,980 --> 01:18:16,880 So the moral is you only really need your 1191 01:18:16,880 --> 01:18:19,390 capacity-approaching code at maybe the first or first and 1192 01:18:19,390 --> 01:18:20,930 second levels. 1193 01:18:20,930 --> 01:18:24,820 Down here, just use a very high rate-- 1194 01:18:24,820 --> 01:18:34,170 1,023, 1,003 Reed-Solomon code over gf of 2 to the 10th-- 1195 01:18:34,170 --> 01:18:36,836 very little redundancy, and drive the error rate down as 1196 01:18:36,836 --> 01:18:40,840 low as you like at each of these levels. 1197 01:18:40,840 --> 01:18:44,380 So multi-level coding is a good way in principle. 1198 01:18:44,380 --> 01:18:46,830 I'm not sure if it's been done in practice. 1199 01:18:46,830 --> 01:18:50,940 In fact, I don't know of any capacity-approaching code that 1200 01:18:50,940 --> 01:18:54,380 has been done in practice, except there's a very crude 1201 01:18:54,380 --> 01:19:02,450 technique, I would say, called Bit-Interleaved Coded 1202 01:19:02,450 --> 01:19:08,070 Modulation, BICM, which is the following simple idea. 1203 01:19:12,910 --> 01:19:16,470 Let's just take a single low-density parity-check code 1204 01:19:16,470 --> 01:19:20,290 or some capacity-approaching code. 1205 01:19:20,290 --> 01:19:23,620 It's going to have to be designed for a BICM channel. 1206 01:19:23,620 --> 01:19:26,650 Let's totally interleave the bits and then let's use them 1207 01:19:26,650 --> 01:19:29,640 as the bits that control each of these levels here. 1208 01:19:29,640 --> 01:19:32,070 At the output of the channel, we know which level the bits 1209 01:19:32,070 --> 01:19:33,410 came out of. 1210 01:19:33,410 --> 01:19:36,120 So at the output of the channel, we know what the 1211 01:19:36,120 --> 01:19:38,820 appropriate a posteriori probabilities 1212 01:19:38,820 --> 01:19:40,500 are for each bit. 1213 01:19:40,500 --> 01:19:41,850 They're going to be wildly different. 1214 01:19:41,850 --> 01:19:43,690 A bit sent through here is going to be sent through a 1215 01:19:43,690 --> 01:19:46,130 very highly reliable channel, a bit sent through here is 1216 01:19:46,130 --> 01:19:48,780 going to be sent through a very low reliability channel. 1217 01:19:48,780 --> 01:19:53,500 But with all this interleaving and the independence 1218 01:19:53,500 --> 01:19:56,380 approximation in the analysis, we're going to be able to 1219 01:19:56,380 --> 01:20:00,380 design a code for this mixture channel, a mixture of high 1220 01:20:00,380 --> 01:20:03,560 reliability and low reliability binary channels, 1221 01:20:03,560 --> 01:20:06,290 that's going to get to the capacity of this mixture 1222 01:20:06,290 --> 01:20:10,590 channel, which as I've already told you is equal to the 1223 01:20:10,590 --> 01:20:12,440 capacity of the aggregate channel. 1224 01:20:16,260 --> 01:20:19,480 I guess there's some elegant features of it. 1225 01:20:19,480 --> 01:20:21,740 But that's, in practice, what people use. 1226 01:20:21,740 --> 01:20:23,790 They use it more because they've already developed a 1227 01:20:23,790 --> 01:20:28,220 chip for binary channels, and they want to reuse that chip 1228 01:20:28,220 --> 01:20:32,310 for some large constellation channel, some 1229 01:20:32,310 --> 01:20:34,520 bandwidth-limited channel. 1230 01:20:34,520 --> 01:20:38,120 So this was originally advocated by Qualcomm as a, 1231 01:20:38,120 --> 01:20:40,740 quote, pragmatic approach. 1232 01:20:40,740 --> 01:20:46,030 OK, we've got a chip that works for binary channels, so 1233 01:20:46,030 --> 01:20:51,640 let's just interleave outputs and use it over this 1234 01:20:51,640 --> 01:20:54,750 multi-level channel and it'll probably work well enough. 1235 01:20:54,750 --> 01:20:56,930 And, in fact, that's true. 1236 01:20:56,930 --> 01:20:59,390 And the information support for it, again, is the fact 1237 01:20:59,390 --> 01:21:06,180 that you don't lose anything by coding bit by bit from a 1238 01:21:06,180 --> 01:21:07,636 capacity point of view. 1239 01:21:12,820 --> 01:21:17,030 I think that's the only one that's been used in practice. 1240 01:21:17,030 --> 01:21:22,800 And on wireless cellular channels for instance, that 1241 01:21:22,800 --> 01:21:24,300 tends to be what's used. 1242 01:21:27,340 --> 01:21:32,770 And of course, I know that perhaps a couple of you have 1243 01:21:32,770 --> 01:21:35,550 an exam at 11:00, so I'll make a point of 1244 01:21:35,550 --> 01:21:37,790 letting you out of here. 1245 01:21:37,790 --> 01:21:40,150 I've really enjoyed teaching the course this term. 1246 01:21:40,150 --> 01:21:43,110 It was the first time I was able to get to the analysis of 1247 01:21:43,110 --> 01:21:48,240 capacity-approaching codes, so I enjoyed doing that. 1248 01:21:48,240 --> 01:21:55,000 At this point, I think the subject of coding for the 1249 01:21:55,000 --> 01:21:58,080 additive white Gaussian noise channel, the traditional 1250 01:21:58,080 --> 01:21:59,780 channel, is pretty well wrapped up. 1251 01:21:59,780 --> 01:22:02,360 I don't think we're going to see any great improvements. 1252 01:22:02,360 --> 01:22:05,340 There's still some very interesting theoretical 1253 01:22:05,340 --> 01:22:08,160 questions about the convergence of these iterative 1254 01:22:08,160 --> 01:22:09,280 decoding algorithms. 1255 01:22:09,280 --> 01:22:10,980 What do they converge to? 1256 01:22:10,980 --> 01:22:12,650 How fast do they converge? 1257 01:22:12,650 --> 01:22:15,510 There are interesting connections to statistical 1258 01:22:15,510 --> 01:22:17,960 physics and inference and so forth that 1259 01:22:17,960 --> 01:22:19,365 people are still exploring. 1260 01:22:19,365 --> 01:22:22,190 They're interested in graph theoretical properties. 1261 01:22:22,190 --> 01:22:25,220 There's attempts to algebraically construct these 1262 01:22:25,220 --> 01:22:28,120 capacity-approaching codes, so we can say a lot more about 1263 01:22:28,120 --> 01:22:31,460 their parameters and their performance, particularly for 1264 01:22:31,460 --> 01:22:34,360 lower probabilities like 10 to the minus 15. 1265 01:22:34,360 --> 01:22:37,360 So you go to the information theory symposium, you'll still 1266 01:22:37,360 --> 01:22:40,750 hear a lot of papers about coding effectively for 1267 01:22:40,750 --> 01:22:44,830 additive white Gaussian noise or memoryless binary symmetric 1268 01:22:44,830 --> 01:22:47,110 input channels. 1269 01:22:47,110 --> 01:22:53,800 But people ask me, is coding dead now? 1270 01:22:53,800 --> 01:22:56,020 Are Fourier transforms dead? 1271 01:22:56,020 --> 01:22:57,150 No, they're not dead. 1272 01:22:57,150 --> 01:23:01,220 But they've become sort of part of the standard core of 1273 01:23:01,220 --> 01:23:03,570 what you should know. 1274 01:23:03,570 --> 01:23:07,240 I think this is a subject that anyone in communication should 1275 01:23:07,240 --> 01:23:10,130 certainly learn, but is it an area where people are going to 1276 01:23:10,130 --> 01:23:14,210 make further advances that are important from a practical 1277 01:23:14,210 --> 01:23:14,750 point of view? 1278 01:23:14,750 --> 01:23:17,990 Obviously not very much, in view of the Shannon limit. 1279 01:23:17,990 --> 01:23:21,030 We are very close to the Shannon limit for these 1280 01:23:21,030 --> 01:23:24,290 channels, and have been now for a few years. 1281 01:23:24,290 --> 01:23:27,350 So it's important to know about these things. 1282 01:23:27,350 --> 01:23:29,850 It's important to know when you're in the wireless course, 1283 01:23:29,850 --> 01:23:31,150 for instance. 1284 01:23:31,150 --> 01:23:33,980 You basically calculate mutual information and assume that 1285 01:23:33,980 --> 01:23:36,810 you can go at the mutual information. 1286 01:23:36,810 --> 01:23:38,690 This is how you do it. 1287 01:23:38,690 --> 01:23:43,770 So you should know how it's done, and that's why I think 1288 01:23:43,770 --> 01:23:46,820 this course continues to be worth teaching. 1289 01:23:46,820 --> 01:23:50,740 But from a point of view of research, it's become a rather 1290 01:23:50,740 --> 01:23:53,790 mature area, and the action has gone off to 1291 01:23:53,790 --> 01:23:55,290 other kinds of channels. 1292 01:23:55,290 --> 01:23:57,120 Multi this, multi that, so forth. 1293 01:23:57,120 --> 01:24:00,830 Anything to do with networks or multi-user is where the 1294 01:24:00,830 --> 01:24:03,410 action in coding is. 1295 01:24:03,410 --> 01:24:05,740 Data compression, of course, is a subject that we haven't 1296 01:24:05,740 --> 01:24:09,180 talked about at all, which is also coding. 1297 01:24:09,180 --> 01:24:12,630 So anyway, I think it's good for you to have all this under 1298 01:24:12,630 --> 01:24:14,460 your belts. 1299 01:24:14,460 --> 01:24:18,570 I hope that you find it useful in your future lives, and I 1300 01:24:18,570 --> 01:24:19,970 wish you all well on the final exam. 1301 01:24:19,970 --> 01:24:21,220 Thank you.