1 00:00:00,000 --> 00:00:02,520 The following content is provided under a Creative 2 00:00:02,520 --> 00:00:03,970 Commons license. 3 00:00:03,970 --> 00:00:06,360 Your support will help MIT OpenCourseWare 4 00:00:06,360 --> 00:00:10,660 continue to offer high quality educational resources for free. 5 00:00:10,660 --> 00:00:13,350 To make a donation or view additional materials 6 00:00:13,350 --> 00:00:17,190 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,190 --> 00:00:18,318 at ocw.mit.edu. 8 00:00:25,720 --> 00:00:27,820 PROFESSOR: I wanted to tell you a little bit 9 00:00:27,820 --> 00:00:30,280 about the use of digital communication 10 00:00:30,280 --> 00:00:32,330 schemes in the space program. 11 00:00:32,330 --> 00:00:34,780 And part of that is it wasn't just the use. 12 00:00:34,780 --> 00:00:37,330 A lot of coding theory was developed 13 00:00:37,330 --> 00:00:38,870 for use in this program. 14 00:00:38,870 --> 00:00:43,960 So in the early days, there was no error control coding. 15 00:00:43,960 --> 00:00:47,200 So they had very slow transmission rates 16 00:00:47,200 --> 00:00:48,850 and tried to compensate for not having 17 00:00:48,850 --> 00:00:54,430 error control coding by taking a long time to send a bit over. 18 00:00:54,430 --> 00:01:00,190 But in later years, with the Mariner and Viking probes, 19 00:01:00,190 --> 00:01:02,140 they started to use error control codes. 20 00:01:02,140 --> 00:01:06,820 And linear block codes are what we're talking about. 21 00:01:06,820 --> 00:01:09,820 This would be the typical parameters for such a code. 22 00:01:09,820 --> 00:01:11,560 So we know how to read this. 23 00:01:11,560 --> 00:01:14,920 This is 32 bits per block. 24 00:01:14,920 --> 00:01:16,360 Six data bits. 25 00:01:16,360 --> 00:01:19,150 And minimum timing distance of 16. 26 00:01:19,150 --> 00:01:21,640 A particular kind of code called a bar orthogonal code 27 00:01:21,640 --> 00:01:26,050 or Hadamard code, which had specific characteristics 28 00:01:26,050 --> 00:01:27,760 and specific symmetries that actually 29 00:01:27,760 --> 00:01:32,020 helped with the decoding. 30 00:01:32,020 --> 00:01:38,150 So for instance, on Mariner Nine, it's 1971. 31 00:01:38,150 --> 00:01:40,180 This went into Mars' orbit. 32 00:01:40,180 --> 00:01:45,430 And the code was used to encode the picture transmissions. 33 00:01:45,430 --> 00:01:50,920 So each data word was six bits to encode 34 00:01:50,920 --> 00:01:53,710 64 gray levels and a picture. 35 00:01:53,710 --> 00:01:57,310 It turned out that, because of transmission issues, 36 00:01:57,310 --> 00:02:00,820 the safe number of bits for a block was 30 bits. 37 00:02:00,820 --> 00:02:04,090 And after that, you had to do a little bit of realigning 38 00:02:04,090 --> 00:02:04,690 or tweaking. 39 00:02:04,690 --> 00:02:08,720 So you could send 30 bits at a time safely. 40 00:02:08,720 --> 00:02:11,200 And so, that was a choice of n in that 41 00:02:11,200 --> 00:02:13,760 vicinity was a natural choice. 42 00:02:13,760 --> 00:02:15,640 One thing you could have thought to do 43 00:02:15,640 --> 00:02:18,880 would be take the six bits and repeat them 44 00:02:18,880 --> 00:02:20,800 five times in the 30 bit window. 45 00:02:20,800 --> 00:02:23,280 And that would be a repetition code. 46 00:02:23,280 --> 00:02:26,740 It turned out that, with this particular Hadamard code, 47 00:02:26,740 --> 00:02:31,180 you could actually get the same data rates or comparable data 48 00:02:31,180 --> 00:02:31,850 rates. 49 00:02:31,850 --> 00:02:32,350 Let's see. 50 00:02:32,350 --> 00:02:33,850 What would the data rate be? 51 00:02:33,850 --> 00:02:35,290 It would be k over n, right? 52 00:02:35,290 --> 00:02:37,630 6 over 32. 53 00:02:37,630 --> 00:02:41,828 But with much better error correction properties. 54 00:02:41,828 --> 00:02:42,370 So let's see. 55 00:02:42,370 --> 00:02:46,610 How many errors could you correct in this code per block? 56 00:02:49,980 --> 00:02:52,440 Somebody? 57 00:02:52,440 --> 00:02:52,940 Seven. 58 00:02:52,940 --> 00:02:54,470 Yeah. 59 00:02:54,470 --> 00:02:58,800 Because you've got a minimum Hamming distance of 16. 60 00:02:58,800 --> 00:03:02,060 So you want D minus 1 over 2, the floor of that. 61 00:03:02,060 --> 00:03:05,210 So you could correct up to seven errors per block. 62 00:03:05,210 --> 00:03:08,060 And this code was actually used on space probes 63 00:03:08,060 --> 00:03:10,623 right into the 80s. 64 00:03:10,623 --> 00:03:12,290 And as I mentioned, this particular code 65 00:03:12,290 --> 00:03:14,748 has various symmetries that allow actually something called 66 00:03:14,748 --> 00:03:17,150 the Fast Fourier Transform to be used in the decoding. 67 00:03:17,150 --> 00:03:23,660 And so, that's really what drove this. 68 00:03:23,660 --> 00:03:27,140 As you read about these probes, it's actually staggering 69 00:03:27,140 --> 00:03:31,230 how much they did with so little. 70 00:03:31,230 --> 00:03:31,940 Let's see. 71 00:03:31,940 --> 00:03:35,840 This thing went half a billion miles almost. 72 00:03:35,840 --> 00:03:40,000 It had an onboard computer with a memory of 512 words. 73 00:03:40,000 --> 00:03:41,750 So you can imagine the kind of engineering 74 00:03:41,750 --> 00:03:45,860 that went into organizing all of this. 75 00:03:45,860 --> 00:03:50,570 The transmitters-- and this is typical of these space probes-- 76 00:03:50,570 --> 00:03:54,860 you don't have a lot of energy generated 77 00:03:54,860 --> 00:03:57,560 from your solar panels necessarily. 78 00:03:57,560 --> 00:03:59,210 So 20 watt transmitters. 79 00:03:59,210 --> 00:04:03,710 So these have to transmit over this kind of distance 80 00:04:03,710 --> 00:04:06,920 the data that you want to send in the presence of noise 81 00:04:06,920 --> 00:04:09,570 and various other errors. 82 00:04:09,570 --> 00:04:10,070 OK? 83 00:04:10,070 --> 00:04:13,160 So quite an engineering feat. 84 00:04:13,160 --> 00:04:15,130 Now the kinds of pictures that you would get? 85 00:04:15,130 --> 00:04:17,870 Well, these are pretty amazing actually, 86 00:04:17,870 --> 00:04:20,630 considering what the probes had to do. 87 00:04:20,630 --> 00:04:25,190 So over the lifetime, it sent over 7,000 images. 88 00:04:25,190 --> 00:04:28,113 Mariner 9 is still orbiting Mars, from what I understand. 89 00:04:28,113 --> 00:04:29,030 It's not sending back. 90 00:04:29,030 --> 00:04:31,910 It stopped sending back transmissions 91 00:04:31,910 --> 00:04:33,200 one or two years after this. 92 00:04:33,200 --> 00:04:40,080 But it's still in orbit until it slows down enough to crash in. 93 00:04:40,080 --> 00:04:40,580 OK. 94 00:04:40,580 --> 00:04:47,900 So as I said, you're typically talking about low power. 95 00:04:47,900 --> 00:04:49,760 20 watts. 96 00:04:49,760 --> 00:04:54,920 WNBR, what's a typical radio station 97 00:04:54,920 --> 00:04:57,758 power on a college campus? 98 00:04:57,758 --> 00:04:59,300 They advertise something on the order 99 00:04:59,300 --> 00:05:02,030 of 700 watts for their transmitter. 100 00:05:02,030 --> 00:05:07,780 So we're talking about doing a lot with a little here. 101 00:05:07,780 --> 00:05:11,060 A lot of the art is in the antenna. 102 00:05:11,060 --> 00:05:14,720 So you have an antenna that directs this power very sharply 103 00:05:14,720 --> 00:05:16,367 towards the intended receiver. 104 00:05:16,367 --> 00:05:18,200 But the more sharply you try to direct that, 105 00:05:18,200 --> 00:05:19,908 the bigger of a control problem you have. 106 00:05:19,908 --> 00:05:22,490 Because you've got to point that antenna all that more 107 00:05:22,490 --> 00:05:23,040 carefully. 108 00:05:23,040 --> 00:05:25,160 So all of these are coupled issues. 109 00:05:25,160 --> 00:05:26,660 And then at the receiver end, you've 110 00:05:26,660 --> 00:05:31,830 got very high quality amplifiers and signal processing. 111 00:05:31,830 --> 00:05:34,400 But the data coding and error correction schemes 112 00:05:34,400 --> 00:05:36,480 are a key part of that. 113 00:05:36,480 --> 00:05:38,965 And it turns out that, as you got more ambitious 114 00:05:38,965 --> 00:05:40,340 with these transmissions, you had 115 00:05:40,340 --> 00:05:41,810 to go to more complicated codes. 116 00:05:41,810 --> 00:05:43,268 And these are the codes we're going 117 00:05:43,268 --> 00:05:46,070 to talk about today, what are called convolutional codes. 118 00:05:46,070 --> 00:05:47,660 We'll talk about the coding today. 119 00:05:47,660 --> 00:05:50,120 And then we'll talk about the decoding with what's 120 00:05:50,120 --> 00:05:54,330 called the Viterbi algorithm next lecture. 121 00:05:54,330 --> 00:05:56,990 So this has been used extensively 122 00:05:56,990 --> 00:06:00,080 from late 1970s onwards. 123 00:06:00,080 --> 00:06:04,100 More recently, you have codes that are actually 124 00:06:04,100 --> 00:06:06,080 combinations of convolutional codes, what 125 00:06:06,080 --> 00:06:07,550 are called Turbo codes. 126 00:06:07,550 --> 00:06:11,630 And another family of codes, low density parity chat codes, 127 00:06:11,630 --> 00:06:16,340 which were developed in Bob Gallagher's PhD thesis here. 128 00:06:16,340 --> 00:06:17,930 Bob Gallagher's on our faculty. 129 00:06:20,730 --> 00:06:24,330 But convolutional codes were really a workhorse 130 00:06:24,330 --> 00:06:27,310 of the whole system. 131 00:06:27,310 --> 00:06:27,810 OK. 132 00:06:27,810 --> 00:06:31,560 So an example is now Cassini, which 133 00:06:31,560 --> 00:06:33,390 is in orbit around Saturn. 134 00:06:33,390 --> 00:06:35,640 It's actively sending pictures. 135 00:06:35,640 --> 00:06:37,860 This, if I read the website correctly, 136 00:06:37,860 --> 00:06:40,230 is a picture from August 29. 137 00:06:40,230 --> 00:06:43,360 And I saw other pictures posted from June and July. 138 00:06:43,360 --> 00:06:45,720 So this is a picture of one of Saturn's moons. 139 00:06:45,720 --> 00:06:47,970 And you can see the rings and the shadows of the rings 140 00:06:47,970 --> 00:06:49,230 and so on. 141 00:06:49,230 --> 00:06:51,960 This is actually recreated in natural color 142 00:06:51,960 --> 00:06:53,460 from multiple images. 143 00:06:57,550 --> 00:07:00,940 This, I guess, is part picture and part artist's rendition. 144 00:07:00,940 --> 00:07:03,562 But that shows you what Cassini looks like. 145 00:07:03,562 --> 00:07:05,020 There's only one of them out there. 146 00:07:05,020 --> 00:07:08,830 I don't think there's something else to photograph Cassini. 147 00:07:08,830 --> 00:07:14,650 So the kind of code that's used is a convolutional code. 148 00:07:14,650 --> 00:07:18,310 We'll learn what these parameters mean, 149 00:07:18,310 --> 00:07:20,560 how they enter into the definition of the code. 150 00:07:20,560 --> 00:07:22,990 And here is a typical code rate. 151 00:07:22,990 --> 00:07:25,090 You're talking about something on the order 152 00:07:25,090 --> 00:07:31,600 of 83,000 bits per second as the code rate here. 153 00:07:31,600 --> 00:07:32,710 Sorry, not the code rate. 154 00:07:32,710 --> 00:07:33,740 This is the data rate. 155 00:07:33,740 --> 00:07:34,240 OK? 156 00:07:34,240 --> 00:07:37,195 So the messages are coming. 157 00:07:39,820 --> 00:07:40,630 Let's see. 158 00:07:40,630 --> 00:07:44,180 You're sending six times this amount per second. 159 00:07:44,180 --> 00:07:46,360 But this is the rate at which the data is coming in. 160 00:07:46,360 --> 00:07:46,860 OK? 161 00:07:49,740 --> 00:07:52,020 So convolutional codes. 162 00:07:52,020 --> 00:07:54,690 And again, I keep coming back to MIT names. 163 00:07:54,690 --> 00:07:57,660 Peter Elias was on our faculty here. 164 00:07:57,660 --> 00:07:59,370 He was a department head for a while. 165 00:07:59,370 --> 00:08:05,220 And in a short paper in 1955, he invented the idea 166 00:08:05,220 --> 00:08:07,290 of convolutional codes. 167 00:08:07,290 --> 00:08:12,600 So the idea here is not to divide up your data into blocks 168 00:08:12,600 --> 00:08:15,510 but to actually work on the streaming data. 169 00:08:15,510 --> 00:08:19,290 And as the data goes past, you generate parity bets 170 00:08:19,290 --> 00:08:20,490 at a regular rate. 171 00:08:20,490 --> 00:08:22,860 And what you transmit in most typical schemes 172 00:08:22,860 --> 00:08:24,000 are just the parity bits. 173 00:08:24,000 --> 00:08:25,540 You don't send the message bits. 174 00:08:25,540 --> 00:08:27,450 So this would be a non-systematic code, 175 00:08:27,450 --> 00:08:28,230 if you like. 176 00:08:28,230 --> 00:08:30,840 So there's no part of that message that's directly 177 00:08:30,840 --> 00:08:32,880 observing the message bits. 178 00:08:35,460 --> 00:08:44,300 Now, you will actually generate and send multiple parity bids. 179 00:08:44,300 --> 00:08:51,920 So you'll have a message sequence, x zero xn. 180 00:08:51,920 --> 00:08:53,150 Sorry, x zero, x one. 181 00:09:04,430 --> 00:09:08,960 And from this, you derive parity bits. 182 00:09:08,960 --> 00:09:12,413 And you do that using the standard sorts of equations 183 00:09:12,413 --> 00:09:13,580 we've seen with block codes. 184 00:09:13,580 --> 00:09:16,010 Each parity bid here, for instance, 185 00:09:16,010 --> 00:09:22,070 parity bit zero at a time n will be some linear combination 186 00:09:22,070 --> 00:09:23,450 of message bits. 187 00:09:23,450 --> 00:09:25,650 But it's the message bits as they're streaming by. 188 00:09:25,650 --> 00:09:30,680 So you might have, for instance, this as your choice 189 00:09:30,680 --> 00:09:33,290 for parity but number zero. 190 00:09:37,840 --> 00:09:38,340 OK. 191 00:09:38,340 --> 00:09:44,640 And then parity bit number one could be some other combination 192 00:09:44,640 --> 00:09:45,580 here. 193 00:09:45,580 --> 00:09:54,490 So for instance, xn plus x n minus 2, for instance. 194 00:09:54,490 --> 00:09:54,990 OK. 195 00:09:54,990 --> 00:09:58,860 So it's a linear combination of some set of messages, just 196 00:09:58,860 --> 00:10:01,410 the way we've been generating parity bits all along. 197 00:10:01,410 --> 00:10:04,410 The plus here, of course, is we're 198 00:10:04,410 --> 00:10:06,670 talking about binary messages. 199 00:10:06,670 --> 00:10:09,570 So this is addition in GF2. 200 00:10:09,570 --> 00:10:16,710 So it's exclusive or Modulo 2 addition. 201 00:10:16,710 --> 00:10:20,040 And you can imagine a whole bunch of such parity bits. 202 00:10:20,040 --> 00:10:24,990 So in general, you would have R such parity bits computed off 203 00:10:24,990 --> 00:10:28,590 some set of message bits and transmitted instead 204 00:10:28,590 --> 00:10:30,010 of the message bits. 205 00:10:30,010 --> 00:10:34,410 So you might have, for each message been coming in, 206 00:10:34,410 --> 00:10:37,530 you might actually be sending out R parity bits. 207 00:10:37,530 --> 00:10:39,990 So what you do is just send these out in sequence. 208 00:10:39,990 --> 00:10:43,830 You'd send out the P zero value, the P1 value, time n. 209 00:10:43,830 --> 00:10:47,420 Then recompute a time n plus 1 and keep going. 210 00:10:47,420 --> 00:10:48,037 All right? 211 00:10:48,037 --> 00:10:49,370 Well, actually I have them here. 212 00:10:49,370 --> 00:10:51,640 I didn't see that. 213 00:10:51,640 --> 00:10:54,320 So all this happens on a sliding window. 214 00:10:54,320 --> 00:10:56,147 This happens for a particular choice of n. 215 00:10:56,147 --> 00:10:57,980 And then it happens for the next choice of n 216 00:10:57,980 --> 00:10:59,140 and the next choice of n. 217 00:10:59,140 --> 00:11:02,575 So you're doing this on the fly with a streaming sequence. 218 00:11:08,350 --> 00:11:12,580 So let me just put up an equation 219 00:11:12,580 --> 00:11:17,620 that explains why this is called a convolutional code. 220 00:11:17,620 --> 00:11:21,250 It turns out the expressions of this type where 221 00:11:21,250 --> 00:11:27,550 you take a data stream coming in and generate new data 222 00:11:27,550 --> 00:11:29,110 streams of this form, it turns out 223 00:11:29,110 --> 00:11:31,990 that the operation that's being carried out here 224 00:11:31,990 --> 00:11:33,910 is something referred to as convolution. 225 00:11:33,910 --> 00:11:37,420 So in general, what has P zero then? 226 00:11:37,420 --> 00:11:42,970 It's some weighted combination of x at the current time, 227 00:11:42,970 --> 00:11:46,240 x1 times step back, x2 time steps back. 228 00:11:46,240 --> 00:11:50,750 In general, k different values involved. 229 00:11:50,750 --> 00:11:59,210 So what I have is a P zero n being a summation from, 230 00:11:59,210 --> 00:12:00,530 let's say, J equals zero. 231 00:12:07,860 --> 00:12:15,460 So G zero J x of n minus J. All right? 232 00:12:15,460 --> 00:12:17,250 So this is just some set of numbers. 233 00:12:17,250 --> 00:12:19,140 Zero, one. 234 00:12:19,140 --> 00:12:23,190 Just as these bits are, these are zero, one. 235 00:12:23,190 --> 00:12:24,510 But this is the general form. 236 00:12:30,790 --> 00:12:32,920 This particular kind of combination 237 00:12:32,920 --> 00:12:36,910 is referred to as a convolution operation on the input stream. 238 00:12:36,910 --> 00:12:38,410 And we'll see much more of this when 239 00:12:38,410 --> 00:12:42,560 we come later to modeling channels, 240 00:12:42,560 --> 00:12:43,570 the physical channels. 241 00:12:43,570 --> 00:12:45,920 We'll talk about convolution type models. 242 00:12:45,920 --> 00:12:48,460 So here, it's not so important that you 243 00:12:48,460 --> 00:12:49,823 must have this expression. 244 00:12:49,823 --> 00:12:51,490 We'll have plenty of opportunity to work 245 00:12:51,490 --> 00:12:52,630 with expressions like this. 246 00:12:52,630 --> 00:12:55,240 This is just for you to know that an expression 247 00:12:55,240 --> 00:12:58,510 of this type, wherever you see a summation with indices 248 00:12:58,510 --> 00:13:02,390 that are in this form, this is referred to as a convolution. 249 00:13:02,390 --> 00:13:02,890 OK? 250 00:13:08,240 --> 00:13:12,320 So it's convolution of the message stream 251 00:13:12,320 --> 00:13:17,527 with some set of weights. 252 00:13:17,527 --> 00:13:18,360 AUDIENCE: Professor? 253 00:13:18,360 --> 00:13:19,205 PROFESSOR: Yeah? 254 00:13:19,205 --> 00:13:20,602 AUDIENCE: What does G stand for? 255 00:13:20,602 --> 00:13:22,560 PROFESSOR: The G is just a set of weights here. 256 00:13:22,560 --> 00:13:27,800 So in this particular case, for parity expression zero, 257 00:13:27,800 --> 00:13:30,995 G zero of zero would be 1. 258 00:13:30,995 --> 00:13:32,756 G zero of 1 would be 1. 259 00:13:32,756 --> 00:13:34,128 G zero of 2 would be 1. 260 00:13:34,128 --> 00:13:34,670 AUDIENCE: OK. 261 00:13:34,670 --> 00:13:36,253 PROFESSOR: It's just a set of weights. 262 00:13:40,600 --> 00:13:41,100 So yeah. 263 00:13:41,100 --> 00:13:44,790 This expression is a bit of overkill for the kind of use 264 00:13:44,790 --> 00:13:45,960 we're making of it. 265 00:13:45,960 --> 00:13:50,130 But it's just to explain the origin of the name. 266 00:13:50,130 --> 00:13:52,980 It turns out later, when we use it for channel modeling, 267 00:13:52,980 --> 00:13:54,990 the x's will not just be zeros or ones. 268 00:13:54,990 --> 00:13:56,820 They could take arbitrary real values. 269 00:13:56,820 --> 00:13:59,170 And the Gs could take arbitrary real values. 270 00:13:59,170 --> 00:14:01,560 So we'll be working with much more elaborate versions 271 00:14:01,560 --> 00:14:02,448 of this. 272 00:14:02,448 --> 00:14:02,948 OK? 273 00:14:09,440 --> 00:14:14,240 The number k is referred to as the constraint length. 274 00:14:14,240 --> 00:14:17,540 And it's the maximum number of message bits involved 275 00:14:17,540 --> 00:14:20,370 when you look over all your parity expressions. 276 00:14:20,370 --> 00:14:23,340 So in this particular instance, k would be equal to 3. 277 00:14:23,340 --> 00:14:23,840 Right? 278 00:14:23,840 --> 00:14:28,430 It's the maximum window of data that you're 279 00:14:28,430 --> 00:14:31,670 using in a non-trivial way to generate the bits. 280 00:14:31,670 --> 00:14:35,168 So here, you are using up to 3 to generate this. 281 00:14:35,168 --> 00:14:36,710 Well, in this case also, you're using 282 00:14:36,710 --> 00:14:38,540 a window of three message bits. 283 00:14:38,540 --> 00:14:40,790 It happens that you're ignoring the one in the center. 284 00:14:40,790 --> 00:14:43,970 But the constraint length is the length of message 285 00:14:43,970 --> 00:14:47,360 that you're actually looking at. 286 00:14:47,360 --> 00:14:47,860 OK. 287 00:14:50,810 --> 00:14:54,200 So in some sense, if you want to think of it this way, 288 00:14:54,200 --> 00:14:56,962 the number of parity expressions that you use? 289 00:14:56,962 --> 00:14:58,170 Well, that's straightforward. 290 00:14:58,170 --> 00:15:00,820 That's just telling you how much redundancy 291 00:15:00,820 --> 00:15:02,890 you're willing to put in. 292 00:15:02,890 --> 00:15:05,860 Whereas the constraint length is telling you 293 00:15:05,860 --> 00:15:08,560 how deeply you're folding that redundancy into the message. 294 00:15:08,560 --> 00:15:10,820 So the bigger the constraint length, 295 00:15:10,820 --> 00:15:12,460 the more message bits are involved 296 00:15:12,460 --> 00:15:14,530 in generating a parity bit. 297 00:15:14,530 --> 00:15:17,110 And so, the more you're scrambling up the message 298 00:15:17,110 --> 00:15:21,435 and spreading it over a large section of what's transmitted. 299 00:15:21,435 --> 00:15:22,810 And so, you might expect that you 300 00:15:22,810 --> 00:15:26,470 get a better error correction properties with larger 301 00:15:26,470 --> 00:15:28,020 constraint links. 302 00:15:28,020 --> 00:15:28,520 OK? 303 00:15:37,810 --> 00:15:38,310 OK. 304 00:15:43,100 --> 00:15:45,740 This is not saying anything new. 305 00:15:52,890 --> 00:15:55,470 So how do we come to actually transmitting? 306 00:15:55,470 --> 00:15:57,240 Well, we generate the parity bits. 307 00:15:57,240 --> 00:15:59,490 And then, as I said, you send all the parity bits 308 00:15:59,490 --> 00:16:01,687 associated with your computation at time zero. 309 00:16:01,687 --> 00:16:04,020 Then all the parity bits associated with the computation 310 00:16:04,020 --> 00:16:09,210 at time 1, time 2, and so on. 311 00:16:09,210 --> 00:16:13,670 So in the case of the code used on the Cassini probe, 312 00:16:13,670 --> 00:16:16,070 that's 1 over 6 rate code. 313 00:16:16,070 --> 00:16:19,500 It's actually computing six parity expressions. 314 00:16:19,500 --> 00:16:21,830 So it's transmitting six parity bits 315 00:16:21,830 --> 00:16:24,530 for each message bit that comes in. 316 00:16:24,530 --> 00:16:26,360 What happens then at the next time instant 317 00:16:26,360 --> 00:16:28,940 is that you shift everything up by one. 318 00:16:28,940 --> 00:16:30,590 And we do the whole thing. 319 00:16:34,530 --> 00:16:35,030 OK? 320 00:16:39,440 --> 00:16:43,080 Now, you can actually-- and I'll have this up on the slides-- 321 00:16:43,080 --> 00:16:45,350 you can actually crank through the equations. 322 00:16:45,350 --> 00:16:47,390 But it's not the most illuminating way 323 00:16:47,390 --> 00:16:49,750 to think of things. 324 00:16:49,750 --> 00:16:54,140 It's much easier to think of it visually through a block 325 00:16:54,140 --> 00:16:56,870 diagram of this type and using the idea of what's 326 00:16:56,870 --> 00:16:59,000 called a shift register. 327 00:16:59,000 --> 00:17:01,010 So what is a shift register? 328 00:17:01,010 --> 00:17:02,930 You may have encountered it in other places. 329 00:17:13,050 --> 00:17:19,310 So we think of a shift register as, it's basically a box that 330 00:17:19,310 --> 00:17:20,310 can remember something. 331 00:17:20,310 --> 00:17:20,810 OK? 332 00:17:20,810 --> 00:17:22,550 That's the register part of it. 333 00:17:22,550 --> 00:17:25,890 A register is something that remembers a number. 334 00:17:25,890 --> 00:17:28,550 You've got some input stream that comes in 335 00:17:28,550 --> 00:17:31,760 and some output stream emerging. 336 00:17:31,760 --> 00:17:34,610 At any time, this stores a particular number which 337 00:17:34,610 --> 00:17:36,180 is available to the output. 338 00:17:36,180 --> 00:17:38,000 So whatever is stored in the register 339 00:17:38,000 --> 00:17:40,070 is available to the output. 340 00:17:40,070 --> 00:17:41,630 The shift part of this description 341 00:17:41,630 --> 00:17:43,490 is that whatever is of the input will 342 00:17:43,490 --> 00:17:46,250 get shifted into at the next clock cycle or the next time 343 00:17:46,250 --> 00:17:46,830 instance. 344 00:17:46,830 --> 00:17:47,330 OK? 345 00:17:47,330 --> 00:17:50,360 So the input gets shifted in at the next clock cycle. 346 00:17:50,360 --> 00:17:54,050 Whatever is in here is remembered for that one clock 347 00:17:54,050 --> 00:17:56,480 cycle and is available at the output. 348 00:17:56,480 --> 00:17:57,050 Right? 349 00:17:57,050 --> 00:18:03,590 So if I have a sequence x n being fed in four n zero, one, 350 00:18:03,590 --> 00:18:09,050 two, three, and so on, if I'm seeing x n here, 351 00:18:09,050 --> 00:18:12,110 what must have gone into the previous time? 352 00:18:12,110 --> 00:18:16,280 If I see x n here at time n, if I'm seeing a particular input 353 00:18:16,280 --> 00:18:19,790 at time n, what must have gone in the previous time 354 00:18:19,790 --> 00:18:20,930 as x n minus 1? 355 00:18:20,930 --> 00:18:24,950 So what's sitting here is x minus 1. 356 00:18:24,950 --> 00:18:26,030 Right? 357 00:18:26,030 --> 00:18:29,680 And x n minus 1 is available to me at the output. 358 00:18:29,680 --> 00:18:32,700 The next clock cycle, the next input comes along. 359 00:18:32,700 --> 00:18:34,260 The xn goes in here. 360 00:18:34,260 --> 00:18:37,200 And the whole thing shifts. 361 00:18:37,200 --> 00:18:37,710 All right? 362 00:18:37,710 --> 00:18:41,940 Now what you have up there is a cascade of shift registers. 363 00:18:41,940 --> 00:18:44,130 You've got some shift registers. 364 00:18:44,130 --> 00:18:47,040 So keep in mind, the operation that I described-- 365 00:18:47,040 --> 00:18:50,280 if this is xn, if I'm looking at this time at time n, 366 00:18:50,280 --> 00:18:51,480 xn sitting here-- 367 00:18:51,480 --> 00:18:53,280 what must be in this shift register 368 00:18:53,280 --> 00:18:55,240 is the input of the previous time. 369 00:18:55,240 --> 00:18:58,650 So that's x n minus 1. 370 00:18:58,650 --> 00:18:59,850 These are shown adjacent. 371 00:18:59,850 --> 00:19:01,965 What we really mean is that one shift register 372 00:19:01,965 --> 00:19:03,808 is feeding into the next one. 373 00:19:03,808 --> 00:19:05,100 They're just shown as adjacent. 374 00:19:05,100 --> 00:19:10,050 But what must be sitting here then is x n minus 2. 375 00:19:10,050 --> 00:19:15,180 And if I read off something from here, what I'm looking at 376 00:19:15,180 --> 00:19:16,680 is x n minus 1. 377 00:19:16,680 --> 00:19:18,990 Namely, what's sitting in the register. 378 00:19:18,990 --> 00:19:21,374 What I'm looking at here is x n minus 2. 379 00:19:25,820 --> 00:19:26,900 All right. 380 00:19:26,900 --> 00:19:30,600 So do you see how this is working now? 381 00:19:30,600 --> 00:19:32,010 This is actually the same example 382 00:19:32,010 --> 00:19:34,560 that I had written up earlier, I guess, 383 00:19:34,560 --> 00:19:35,940 for the computational parity bit. 384 00:19:35,940 --> 00:19:37,720 So here's-- except it's-- 385 00:19:37,720 --> 00:19:38,220 yeah. 386 00:19:38,220 --> 00:19:39,550 It's the same one. 387 00:19:39,550 --> 00:19:41,340 P zero n. 388 00:19:41,340 --> 00:19:42,600 Maybe I have the equations. 389 00:19:42,600 --> 00:19:44,267 Let's see if I can display them for you. 390 00:19:46,610 --> 00:19:47,150 No, I can't. 391 00:19:47,150 --> 00:19:47,650 OK. 392 00:19:47,650 --> 00:19:50,600 So what's P zero n? 393 00:19:50,600 --> 00:19:55,040 P zero n is xn, that's connecting from here, 394 00:19:55,040 --> 00:19:58,340 plus x n minus 1 plus x n minus 2. 395 00:19:58,340 --> 00:20:00,920 Again, by the way, in this diagram, what 396 00:20:00,920 --> 00:20:03,680 I showed as an arrow coming from the output of the shift 397 00:20:03,680 --> 00:20:06,047 register is just a shorthand here 398 00:20:06,047 --> 00:20:08,630 that shows the arrow coming out from the body of the register. 399 00:20:08,630 --> 00:20:10,297 It's the same thing we're talking about. 400 00:20:10,297 --> 00:20:11,570 OK? 401 00:20:11,570 --> 00:20:17,240 So P zero of n is the sum of these three message bits. 402 00:20:17,240 --> 00:20:19,460 So we're talking one constraint length three here. 403 00:20:19,460 --> 00:20:21,350 And what about P1 n? 404 00:20:21,350 --> 00:20:26,720 It's xn plus x n minus 2 with nothing of xn minus 1. 405 00:20:26,720 --> 00:20:27,220 All right? 406 00:20:27,220 --> 00:20:30,010 So imagine this being the picture for every n. 407 00:20:30,010 --> 00:20:33,440 So you start off at time zero and keep going. 408 00:20:33,440 --> 00:20:35,410 Right? 409 00:20:35,410 --> 00:20:37,060 We refer to the state of the shift 410 00:20:37,060 --> 00:20:41,360 registers as the pair of numbers that we find in here. 411 00:20:41,360 --> 00:20:45,430 So if we're talking about x's that can be zeros or ones, 412 00:20:45,430 --> 00:20:47,260 the shift register combination here 413 00:20:47,260 --> 00:20:49,240 can be in one of four states. 414 00:20:49,240 --> 00:20:50,320 Right? 415 00:20:50,320 --> 00:20:51,160 Zero, zero. 416 00:20:51,160 --> 00:20:51,860 Zero, 1. 417 00:20:51,860 --> 00:20:52,360 1, zero. 418 00:20:52,360 --> 00:20:53,320 Or 1, 1. 419 00:20:53,320 --> 00:20:54,510 So four states. 420 00:20:54,510 --> 00:20:58,000 So here's a four state shift register into which 421 00:20:58,000 --> 00:20:59,530 we're feeding in the stream. 422 00:20:59,530 --> 00:21:01,540 And what gets put out on the channel 423 00:21:01,540 --> 00:21:06,130 are these parity bits interleaved. 424 00:21:06,130 --> 00:21:08,570 That clear enough? 425 00:21:08,570 --> 00:21:09,070 OK. 426 00:21:22,090 --> 00:21:23,590 Nothing I haven't said here. 427 00:21:23,590 --> 00:21:24,090 Right? 428 00:21:27,630 --> 00:21:31,620 So let's actually work through an example step by step. 429 00:21:31,620 --> 00:21:32,700 This is clear enough. 430 00:21:32,700 --> 00:21:35,580 But let's just see it concretely. 431 00:21:35,580 --> 00:21:39,990 Let's assume that I'm starting out with the shift 432 00:21:39,990 --> 00:21:42,300 registers in the zero state. 433 00:21:42,300 --> 00:21:44,640 And now, I've got this message sequence coming in 434 00:21:44,640 --> 00:21:45,670 that I want to send out. 435 00:21:45,670 --> 00:21:46,170 OK? 436 00:21:46,170 --> 00:21:49,380 So the sequence is 1, zero, 1, 1. 437 00:21:49,380 --> 00:21:53,010 So the first bit that appears here is the 1. 438 00:21:53,010 --> 00:21:55,290 And I've got to generate P zero and P1. 439 00:21:55,290 --> 00:21:58,270 Well, P zero is the exclusive OR of these three things. 440 00:21:58,270 --> 00:21:59,550 So it's 1. 441 00:21:59,550 --> 00:22:02,020 P1 is the exclusive OR of the first and the last. 442 00:22:02,020 --> 00:22:03,330 So it's again 1. 443 00:22:03,330 --> 00:22:07,980 So that defines P zero and P1 at time n. 444 00:22:07,980 --> 00:22:12,660 The same way at the next time instant, the next message input 445 00:22:12,660 --> 00:22:14,230 bit comes in. 446 00:22:14,230 --> 00:22:15,630 So we had 1, zero, 1, 1. 447 00:22:15,630 --> 00:22:17,220 We took care of the 1 here. 448 00:22:17,220 --> 00:22:19,350 Now comes a zero. 449 00:22:19,350 --> 00:22:20,470 We do the same thing. 450 00:22:20,470 --> 00:22:24,300 So the exclusive OR of all three of them appears here. 451 00:22:24,300 --> 00:22:25,380 That's the 1. 452 00:22:25,380 --> 00:22:28,030 The exclusive OR of the first and the last appears there. 453 00:22:28,030 --> 00:22:29,940 And that's the zero. 454 00:22:29,940 --> 00:22:33,060 So you can see how things are getting folded together 455 00:22:33,060 --> 00:22:36,780 because the input that was here before is now sitting in here 456 00:22:36,780 --> 00:22:39,420 and plays a role in generation of the parity 457 00:22:39,420 --> 00:22:41,550 bit for the next step. 458 00:22:41,550 --> 00:22:44,108 In fact, the word convolve means to fold together. 459 00:22:44,108 --> 00:22:46,150 And this is what it's actually trying to capture. 460 00:22:46,150 --> 00:22:50,250 You're folding together these two sets 461 00:22:50,250 --> 00:22:53,280 of weights, the weights on the top tier and the input sequence 462 00:22:53,280 --> 00:22:55,330 weight. 463 00:22:55,330 --> 00:23:00,710 And then the next two cases, similarly. 464 00:23:00,710 --> 00:23:01,220 OK. 465 00:23:01,220 --> 00:23:06,320 And that's what gets sent out at the bottom. 466 00:23:06,320 --> 00:23:08,270 So this is the transmitted sequence. 467 00:23:08,270 --> 00:23:14,530 So it's the 11100001. 468 00:23:14,530 --> 00:23:15,580 Right? 469 00:23:15,580 --> 00:23:17,660 That's all there is to it. 470 00:23:17,660 --> 00:23:21,320 The implementation of the shift register is very easy. 471 00:23:21,320 --> 00:23:25,660 And so, this is actually a very straightforward thing 472 00:23:25,660 --> 00:23:27,410 to implement. 473 00:23:27,410 --> 00:23:33,810 Now there's another viewpoint that's also very useful here. 474 00:23:33,810 --> 00:23:36,450 Another way to look at what's going on, 475 00:23:36,450 --> 00:23:40,220 which is thinking in terms of the state of the register 476 00:23:40,220 --> 00:23:43,190 and how you move between the states. 477 00:23:43,190 --> 00:23:46,357 I guess, how many here are 004? 478 00:23:46,357 --> 00:23:48,440 Are those are the ones with smiles on their faces? 479 00:23:48,440 --> 00:23:49,760 OK. 480 00:23:49,760 --> 00:23:53,130 You see a lot of this there, I imagine. 481 00:23:53,130 --> 00:23:53,630 OK. 482 00:23:53,630 --> 00:23:57,800 So how do I read a diagram like this? 483 00:23:57,800 --> 00:24:00,740 I've got a circle for each state that the shift register 484 00:24:00,740 --> 00:24:01,400 can be in. 485 00:24:01,400 --> 00:24:04,238 So the shift register can be in zero, zero. 486 00:24:04,238 --> 00:24:05,696 Zero, 1. 487 00:24:05,696 --> 00:24:07,640 1, zero. 488 00:24:07,640 --> 00:24:08,510 1, 1. 489 00:24:08,510 --> 00:24:11,000 Right? 490 00:24:11,000 --> 00:24:13,910 Each of these arcs represents a transition 491 00:24:13,910 --> 00:24:16,050 from one state to another. 492 00:24:16,050 --> 00:24:17,123 So let me ask you this. 493 00:24:17,123 --> 00:24:17,915 What does it take-- 494 00:24:20,700 --> 00:24:24,717 if I'm in the zero, zero state with my shift 495 00:24:24,717 --> 00:24:26,550 register-- so what you've got in the picture 496 00:24:26,550 --> 00:24:30,150 is your shift register sitting there with zero, zero. 497 00:24:30,150 --> 00:24:38,104 What does it take for me to get to the 1, zero state? 498 00:24:44,400 --> 00:24:46,988 What must my input have been to get to the 1, zero state? 499 00:24:50,410 --> 00:24:52,280 Imagine how these shift registers operate. 500 00:24:52,280 --> 00:24:52,900 Right? 501 00:24:52,900 --> 00:24:55,690 If I'm going to get from zero, zero to 1, zero, 502 00:24:55,690 --> 00:24:58,930 I must have fed in a 1 at the previous time instance. 503 00:24:58,930 --> 00:25:03,970 So it takes an input of 1 to go from zero, zero to the 1, zero. 504 00:25:03,970 --> 00:25:08,170 So to go from zero, zero to 1, zero, use an input of 1. 505 00:25:08,170 --> 00:25:10,990 That's the number that we write before the slash. 506 00:25:10,990 --> 00:25:14,200 That's our labeling convention for the arcs. 507 00:25:14,200 --> 00:25:17,510 We put the input that it takes to make that transition. 508 00:25:17,510 --> 00:25:20,170 And then after the slash, we put the parity bits 509 00:25:20,170 --> 00:25:21,340 that are omitted. 510 00:25:21,340 --> 00:25:24,010 So what we've got for the 1, 1 is 511 00:25:24,010 --> 00:25:29,320 the parity bits that are omitted when you've got input 1 sitting 512 00:25:29,320 --> 00:25:32,230 here, zero, zero here, and you're using the parity 513 00:25:32,230 --> 00:25:35,500 computation that I had before. 514 00:25:35,500 --> 00:25:38,140 Let's see here. 515 00:25:38,140 --> 00:25:42,310 So P zero is going to be x n plus x n minus 1 516 00:25:42,310 --> 00:25:43,570 plus x n minus 2. 517 00:25:43,570 --> 00:25:45,070 So that gives you 1. 518 00:25:45,070 --> 00:25:46,930 And what about P1? 519 00:25:46,930 --> 00:25:49,540 P1 is x n plus xn minus 2. 520 00:25:49,540 --> 00:25:51,090 So that gives you another one. 521 00:25:51,090 --> 00:25:51,590 OK? 522 00:25:51,590 --> 00:25:54,672 So if you're in state zero-- 523 00:25:54,672 --> 00:25:56,800 the zero, zero state-- 524 00:25:56,800 --> 00:25:59,020 and you get an input of 1, you're 525 00:25:59,020 --> 00:26:00,940 going to transition to 1, zero. 526 00:26:00,940 --> 00:26:03,230 And you're going to omit 1, 1. 527 00:26:03,230 --> 00:26:03,730 OK. 528 00:26:03,730 --> 00:26:06,610 So the state diagram captures all that. 529 00:26:06,610 --> 00:26:08,242 And similarly, all the way around. 530 00:26:08,242 --> 00:26:09,700 So I haven't checked each of these. 531 00:26:09,700 --> 00:26:12,350 But I hope there are no mistakes in it. 532 00:26:12,350 --> 00:26:13,630 But if you're in 1, zero? 533 00:26:13,630 --> 00:26:15,100 Oh well. 534 00:26:15,100 --> 00:26:16,600 By the way, if you're in zero, zero, 535 00:26:16,600 --> 00:26:18,190 there's no way to get to zero, 1. 536 00:26:18,190 --> 00:26:18,690 Right? 537 00:26:18,690 --> 00:26:22,360 So you don't see any arc from zero, zero to zero, 1. 538 00:26:22,360 --> 00:26:24,880 If you're in 1, zero, you can get to 1, 1. 539 00:26:24,880 --> 00:26:29,360 Or you can get to zero, 1 depending on what you feed in. 540 00:26:29,360 --> 00:26:30,680 OK? 541 00:26:30,680 --> 00:26:33,650 So it's very straightforward, then, 542 00:26:33,650 --> 00:26:35,360 to actually build out this diagram. 543 00:26:35,360 --> 00:26:39,790 Why don't we do a little bit more on here? 544 00:26:47,310 --> 00:26:47,810 OK. 545 00:26:47,810 --> 00:26:55,840 So if I'm actually abstracting from the shift register picture 546 00:26:55,840 --> 00:26:58,330 to something that's more like the state picture, 547 00:26:58,330 --> 00:27:00,790 I'm going to say, here are my four states. 548 00:27:00,790 --> 00:27:02,917 I've just drawn it a little differently than I 549 00:27:02,917 --> 00:27:04,000 have in the upper picture. 550 00:27:04,000 --> 00:27:06,520 Instead of circles with these states in them, 551 00:27:06,520 --> 00:27:08,470 I prefer to think of them this way. 552 00:27:08,470 --> 00:27:15,100 So what we said is, if you get an input of 1, you limit 1, 1. 553 00:27:15,100 --> 00:27:17,890 And you'll get to that state. 554 00:27:17,890 --> 00:27:20,770 What does it take to get to the state? 555 00:27:23,400 --> 00:27:24,355 Somebody? 556 00:27:27,680 --> 00:27:30,420 Can I have a hand and a loud voice? 557 00:27:30,420 --> 00:27:30,920 Yeah? 558 00:27:30,920 --> 00:27:32,090 AUDIENCE: Input zero. 559 00:27:32,090 --> 00:27:33,410 PROFESSOR: OK. 560 00:27:33,410 --> 00:27:37,820 And then I guess you've got to go back to this to think 561 00:27:37,820 --> 00:27:38,970 about what's happening. 562 00:27:38,970 --> 00:27:42,870 So I'll allow you to think of a zero sitting at the input here. 563 00:27:42,870 --> 00:27:45,200 So what would the parity bits be? 564 00:27:50,790 --> 00:27:54,440 So the first parity bit will be the exclusive OR 565 00:27:54,440 --> 00:27:57,200 of the zero, 1, and zero. 566 00:27:57,200 --> 00:27:58,880 So it's going to give you a 1. 567 00:27:58,880 --> 00:27:59,900 Right? 568 00:27:59,900 --> 00:28:02,060 And then the next parity bit is going 569 00:28:02,060 --> 00:28:05,360 to be exclusive OR of what's here and there. 570 00:28:05,360 --> 00:28:07,100 So that's going to be a zero. 571 00:28:07,100 --> 00:28:09,950 I hope that matches with what I have upstairs. 572 00:28:09,950 --> 00:28:13,400 We're talking about going from 1, zero to zero, 1. 573 00:28:13,400 --> 00:28:15,290 It takes a zero input to do that. 574 00:28:15,290 --> 00:28:17,200 And what you omit is 1, zero. 575 00:28:17,200 --> 00:28:17,700 Right? 576 00:28:20,560 --> 00:28:22,750 So you can fill in all of these. 577 00:28:22,750 --> 00:28:26,530 This is the state transition diagram. 578 00:28:26,530 --> 00:28:27,030 OK. 579 00:28:31,330 --> 00:28:32,260 Let's see. 580 00:28:32,260 --> 00:28:35,860 We say that, if you've got a constraint length of three 581 00:28:35,860 --> 00:28:38,080 then, of k equals three, for instance, 582 00:28:38,080 --> 00:28:41,350 or let's say if you've got a constraint length of k, 583 00:28:41,350 --> 00:28:44,510 you've got 2 to the k minus 1 states. 584 00:28:44,510 --> 00:28:48,430 well, that's because, in that constraint length, 585 00:28:48,430 --> 00:28:51,430 one of the bits involved is the input bit. 586 00:28:51,430 --> 00:28:53,350 That's not sitting in the shift registers. 587 00:28:53,350 --> 00:28:55,540 So you've got k minus 1 bits left over. 588 00:28:55,540 --> 00:28:59,230 So your shift register is k minus 1 stages long. 589 00:28:59,230 --> 00:29:03,760 And so, you've got 2 to the k minus 1 states. 590 00:29:03,760 --> 00:29:04,360 All right. 591 00:29:04,360 --> 00:29:06,040 So you could imagine generalizing this 592 00:29:06,040 --> 00:29:08,650 to more complicated sorts of situations. 593 00:29:13,740 --> 00:29:14,240 Let's see. 594 00:29:14,240 --> 00:29:17,850 Just going back to the Cassini example, 595 00:29:17,850 --> 00:29:23,070 if you let me jump back a bit, there was a k there. 596 00:29:23,070 --> 00:29:23,700 What was it? 597 00:29:29,035 --> 00:29:30,930 k of 15. 598 00:29:30,930 --> 00:29:31,440 OK. 599 00:29:31,440 --> 00:29:37,080 So for Cassini, you're using one input bit and 14 more 600 00:29:37,080 --> 00:29:38,740 bits in your register. 601 00:29:38,740 --> 00:29:39,240 OK? 602 00:29:39,240 --> 00:29:44,573 So you've got 2 to the 14 possible states there. 603 00:29:44,573 --> 00:29:45,990 So in these codes, you're actually 604 00:29:45,990 --> 00:29:48,438 using very large constraint lengths. 605 00:29:57,915 --> 00:29:58,415 OK. 606 00:30:02,150 --> 00:30:02,650 All right. 607 00:30:02,650 --> 00:30:06,653 I want to go from the state machine view to another view 608 00:30:06,653 --> 00:30:09,070 now, which is what's called-- so this is the state machine 609 00:30:09,070 --> 00:30:09,570 view-- 610 00:30:16,648 --> 00:30:18,190 to something called the trellis view. 611 00:30:25,268 --> 00:30:26,810 This is something-- by the way, there 612 00:30:26,810 --> 00:30:29,540 was a way of looking at things that 613 00:30:29,540 --> 00:30:34,530 was developed by someone else who was on our faculty, David 614 00:30:34,530 --> 00:30:35,030 Forni. 615 00:30:37,180 --> 00:30:39,430 In fact, if you visit his home, you'll see his garden. 616 00:30:39,430 --> 00:30:41,360 There's a nice trellis around it. 617 00:30:41,360 --> 00:30:43,220 And you'll see why when we draw this. 618 00:30:45,870 --> 00:30:46,370 OK. 619 00:30:46,370 --> 00:30:47,495 So what's the trellis view? 620 00:30:47,495 --> 00:30:50,060 The trellis view says take the state machine 621 00:30:50,060 --> 00:30:53,990 but unfold it in time so that all your transitions over time 622 00:30:53,990 --> 00:30:55,070 are not happening here. 623 00:30:55,070 --> 00:30:57,440 At every time step, you draw the picture again 624 00:30:57,440 --> 00:30:58,920 and look to see where you get to. 625 00:30:58,920 --> 00:30:59,850 So let's do this. 626 00:30:59,850 --> 00:31:03,860 This is the one I want to be most careful with 627 00:31:03,860 --> 00:31:10,960 and where I'll introduce a few notational conventions so 628 00:31:10,960 --> 00:31:12,610 that our later life is simplified. 629 00:31:18,970 --> 00:31:19,470 OK. 630 00:31:19,470 --> 00:31:22,290 So we've got state zero, zero. 631 00:31:22,290 --> 00:31:24,570 State zero, 1. 632 00:31:24,570 --> 00:31:26,810 State 1, zero. 633 00:31:26,810 --> 00:31:30,500 And state 1, 1. 634 00:31:30,500 --> 00:31:31,302 OK. 635 00:31:31,302 --> 00:31:33,260 Except, though, this is going to be the picture 636 00:31:33,260 --> 00:31:34,440 that I have at time-- 637 00:31:34,440 --> 00:31:35,870 let's say-- at time n equals zero. 638 00:31:39,570 --> 00:31:45,388 At time n equals 1, well, I've got the same shift registers. 639 00:31:45,388 --> 00:31:46,930 I'm going to draw this picture again. 640 00:31:57,640 --> 00:31:59,790 The easiest way to learn this is to just follow 641 00:31:59,790 --> 00:32:01,630 through one example. 642 00:32:01,630 --> 00:32:03,930 So please keep your attention here. 643 00:32:03,930 --> 00:32:05,520 And you'll have it sorted out. 644 00:32:05,520 --> 00:32:07,590 And then you won't have to worry about it again. 645 00:32:07,590 --> 00:32:10,110 It's the same thing as with LZW. 646 00:32:10,110 --> 00:32:11,190 All right. 647 00:32:11,190 --> 00:32:13,650 So it looks kind of detailed. 648 00:32:13,650 --> 00:32:14,520 Maybe tedious. 649 00:32:14,520 --> 00:32:16,600 But it's actually very simple. 650 00:32:16,600 --> 00:32:20,550 Just hang in there and follow through one example. 651 00:32:20,550 --> 00:32:21,210 OK. 652 00:32:21,210 --> 00:32:22,680 So what does this say? 653 00:32:22,680 --> 00:32:27,120 At time n equals zero, I'm in zero, zero. 654 00:32:27,120 --> 00:32:29,280 Suppose I get the input zero. 655 00:32:29,280 --> 00:32:30,430 Suppose the input is zero. 656 00:32:30,430 --> 00:32:31,680 What state do I transition to? 657 00:32:34,220 --> 00:32:34,720 Here. 658 00:32:34,720 --> 00:32:35,220 Right? 659 00:32:37,600 --> 00:32:40,370 So if I have an input zero, I'm going to transition here. 660 00:32:40,370 --> 00:32:42,670 So this is with an input of zero. 661 00:32:42,670 --> 00:32:44,440 And what are my parity bits going to be? 662 00:32:48,790 --> 00:32:49,480 Both zeros. 663 00:32:49,480 --> 00:32:49,980 Right? 664 00:32:54,490 --> 00:32:56,530 What about if I get an input of one? 665 00:32:56,530 --> 00:32:58,360 Where do I transition to? 666 00:32:58,360 --> 00:32:59,920 Well, we've already seen that here. 667 00:32:59,920 --> 00:33:03,370 If I get an input of 1, I'm going to transition to here. 668 00:33:07,760 --> 00:33:10,310 And what am I going to omit? 669 00:33:10,310 --> 00:33:11,900 Well, we've already calculated that. 670 00:33:11,900 --> 00:33:13,100 We're going to omit a 1, 1. 671 00:33:16,120 --> 00:33:16,620 Right? 672 00:33:20,240 --> 00:33:21,930 Let's do it for one more case. 673 00:33:21,930 --> 00:33:23,660 We're in zero, 1. 674 00:33:23,660 --> 00:33:25,520 What states can I transition to? 675 00:33:28,150 --> 00:33:29,560 I could go to zero, zero. 676 00:33:34,000 --> 00:33:36,310 And I would do that if my input was zero. 677 00:33:36,310 --> 00:33:36,810 Right? 678 00:33:39,380 --> 00:33:42,740 And what would my parity bits be? 679 00:33:42,740 --> 00:33:46,070 Well, that's another case for us to look at. 680 00:33:46,070 --> 00:33:48,860 If our input is zero and we're in state zero, 1, 681 00:33:48,860 --> 00:33:50,330 what would the parity bits be? 682 00:33:53,940 --> 00:33:56,820 For this choice of parity bits, depends on what specific choice 683 00:33:56,820 --> 00:34:00,010 you made, of course. 684 00:34:00,010 --> 00:34:00,520 1, 1? 685 00:34:03,460 --> 00:34:04,430 Do you agree? 686 00:34:07,210 --> 00:34:10,440 And if I get an input of 1 instead, where do I go? 687 00:34:10,440 --> 00:34:12,900 If I get an input of 1, I'm going 688 00:34:12,900 --> 00:34:16,880 to go to 1, zero, which is here. 689 00:34:19,900 --> 00:34:20,830 OK? 690 00:34:20,830 --> 00:34:23,980 So if I had an input of 1, I would go to 1, zero. 691 00:34:23,980 --> 00:34:26,278 My parity bits would be-- 692 00:34:26,278 --> 00:34:27,070 what would they be? 693 00:34:31,100 --> 00:34:33,610 Can I have a hand and a voice? 694 00:34:33,610 --> 00:34:34,650 Yeah. 695 00:34:34,650 --> 00:34:35,199 Zero, zero. 696 00:34:35,199 --> 00:34:35,699 Right. 697 00:34:41,380 --> 00:34:41,880 OK. 698 00:34:41,880 --> 00:34:42,810 So it's that simple. 699 00:34:42,810 --> 00:34:44,070 That's all you have to do. 700 00:34:44,070 --> 00:34:45,870 Fill out this picture and you're seeing 701 00:34:45,870 --> 00:34:48,659 what this picture translates to at the next time instant. 702 00:34:48,659 --> 00:34:50,250 We're not using anything more than is 703 00:34:50,250 --> 00:34:52,260 in the state transition-- 704 00:34:52,260 --> 00:34:55,750 sorry-- in the state machine diagram. 705 00:34:55,750 --> 00:34:57,418 But we're unfolding things in time, 706 00:34:57,418 --> 00:34:58,710 which is actually very helpful. 707 00:35:01,570 --> 00:35:05,160 Now, this is a simplification we'll make in drawing this. 708 00:35:05,160 --> 00:35:08,970 Because I've arranged the states in natural binary 709 00:35:08,970 --> 00:35:10,260 counting order-- 710 00:35:10,260 --> 00:35:12,664 zero zero, zero 1, 1 zero, and 1, 711 00:35:12,664 --> 00:35:15,840 1-- it's always the case that the upper arrow that 712 00:35:15,840 --> 00:35:18,840 emanates from a state corresponds 713 00:35:18,840 --> 00:35:20,190 to the input of zero. 714 00:35:20,190 --> 00:35:23,340 And the lower corresponds to an input of one. 715 00:35:23,340 --> 00:35:24,120 OK? 716 00:35:24,120 --> 00:35:27,550 So I don't really need that first thing before the dash. 717 00:35:27,550 --> 00:35:29,290 I'm just going to dispense with it. 718 00:35:29,290 --> 00:35:32,580 So if you're going up, of the two choices that you 719 00:35:32,580 --> 00:35:37,470 have when you come out of a box, if you're going up, it's zero. 720 00:35:37,470 --> 00:35:38,580 The input is zero. 721 00:35:38,580 --> 00:35:41,250 And if you're going down, the input is 1. 722 00:35:41,250 --> 00:35:43,720 So I'm just going to label that as zero, zero. 723 00:35:43,720 --> 00:35:44,220 OK? 724 00:35:44,220 --> 00:35:46,800 I'm going to label this as 1, 1. 725 00:35:50,010 --> 00:35:54,150 And I guess I've forgotten already what some of these are. 726 00:36:00,850 --> 00:36:05,570 But you can see what the whole picture starts to look like. 727 00:36:05,570 --> 00:36:06,070 OK? 728 00:36:08,650 --> 00:36:14,600 So let me actually, I'm not going to do these in detail. 729 00:36:14,600 --> 00:36:22,520 But let's just see how the next stage would differ, if at all. 730 00:36:25,680 --> 00:36:30,990 When I come to n equals 2, well, it's 731 00:36:30,990 --> 00:36:32,570 the same story all over again. 732 00:36:36,150 --> 00:36:39,840 So whatever pattern of arrows I had coming out of here, 733 00:36:39,840 --> 00:36:42,510 I have the same pattern at the next stage 734 00:36:42,510 --> 00:36:44,290 with the same labels. 735 00:36:44,290 --> 00:36:44,790 Right? 736 00:36:44,790 --> 00:36:46,207 Because there's nothing different. 737 00:36:48,540 --> 00:36:50,370 So if you'll allow me, let me actually 738 00:36:50,370 --> 00:36:52,635 fill out a few of these. 739 00:37:20,530 --> 00:37:22,330 And you'll get practice drawing one 740 00:37:22,330 --> 00:37:25,910 of these when you do recitation maybe for another example. 741 00:37:25,910 --> 00:37:28,285 So I can keep going with these. 742 00:37:28,285 --> 00:37:28,910 Let's see here. 743 00:37:28,910 --> 00:37:34,710 This is going to be 1, 1. 744 00:37:34,710 --> 00:37:37,160 If I haven't found two arrows coming out of each box, 745 00:37:37,160 --> 00:37:37,910 then I'm not done. 746 00:37:47,870 --> 00:37:48,620 Oh, this is wrong. 747 00:37:48,620 --> 00:37:51,050 Right? 748 00:37:51,050 --> 00:37:53,610 Thank you. 749 00:37:53,610 --> 00:37:55,320 From zero, 1, I can go to 1, zero. 750 00:38:03,000 --> 00:38:03,500 OK. 751 00:38:03,500 --> 00:38:07,370 So the two arrows coming out of each one, the upper arrow 752 00:38:07,370 --> 00:38:09,140 corresponds to the input having been zero. 753 00:38:09,140 --> 00:38:15,060 The lower arrow corresponds to an input having been 1. 754 00:38:15,060 --> 00:38:17,700 And there are two arrows going into each box, 755 00:38:17,700 --> 00:38:20,190 as well, corresponding to whether the bit that's 756 00:38:20,190 --> 00:38:22,840 going to get dropped off is a zero or a 1. 757 00:38:22,840 --> 00:38:23,340 All right? 758 00:38:23,340 --> 00:38:25,110 So there's a real symmetry to this. 759 00:38:29,420 --> 00:38:31,580 I'll draw one more stage just a little bit 760 00:38:31,580 --> 00:38:32,840 to make a point here. 761 00:38:47,250 --> 00:38:47,750 OK. 762 00:38:47,750 --> 00:38:48,740 So you can keep going. 763 00:38:51,310 --> 00:38:57,000 So how do you generate a code word from a trellis diagram? 764 00:38:57,000 --> 00:38:58,530 You're starting in some state. 765 00:38:58,530 --> 00:39:00,150 Typically, it's the all zero state. 766 00:39:00,150 --> 00:39:02,250 In fact, what you'll usually do is 767 00:39:02,250 --> 00:39:04,740 have a header for your message stream which is all zero. 768 00:39:04,740 --> 00:39:07,380 So you force the shift register to be in the zero state 769 00:39:07,380 --> 00:39:09,930 once the real message bits come in. 770 00:39:09,930 --> 00:39:11,610 And then you move from here. 771 00:39:11,610 --> 00:39:13,067 So you're typically starting here. 772 00:39:13,067 --> 00:39:15,150 And then you navigate, depending on whether you've 773 00:39:15,150 --> 00:39:16,890 got a zero or a 1. 774 00:39:16,890 --> 00:39:18,960 So if the first message bit is a 1, 775 00:39:18,960 --> 00:39:20,550 you're going to go down here. 776 00:39:20,550 --> 00:39:22,200 If the next message bit is a zero, 777 00:39:22,200 --> 00:39:23,760 you're going to go up here. 778 00:39:23,760 --> 00:39:25,200 If the next message bit is a zero, 779 00:39:25,200 --> 00:39:26,430 you're going to go up there. 780 00:39:26,430 --> 00:39:28,830 And the code word that you omit is going to be, 781 00:39:28,830 --> 00:39:33,670 in that case, 1, 1, 1, zero, 1, 1. 782 00:39:33,670 --> 00:39:34,650 And then all zeros. 783 00:39:34,650 --> 00:39:35,460 Right? 784 00:39:35,460 --> 00:39:37,960 Assuming you're staying at zero from then on. 785 00:39:37,960 --> 00:39:41,490 So depending on what the message sequence is, 786 00:39:41,490 --> 00:39:43,170 you can actually go through the trellis. 787 00:39:43,170 --> 00:39:46,830 It's infinitely long, or as long as your message sequence is. 788 00:39:46,830 --> 00:39:49,440 And figure out what the code word is that's omitted. 789 00:39:49,440 --> 00:39:51,180 So this is actually just a graphical way 790 00:39:51,180 --> 00:39:52,380 of displaying code words. 791 00:39:56,400 --> 00:39:58,380 So the set of code words that I get, 792 00:39:58,380 --> 00:40:00,810 does that correspond to a linear code? 793 00:40:00,810 --> 00:40:04,140 Let's assume that, somewhere downstream, all these things 794 00:40:04,140 --> 00:40:06,250 come back down to zero, zero. 795 00:40:06,250 --> 00:40:06,750 OK? 796 00:40:06,750 --> 00:40:13,720 So I'm only considering a finite window of things. 797 00:40:13,720 --> 00:40:15,100 It's not going to go on forever. 798 00:40:15,100 --> 00:40:18,990 So suppose I'm going to end my input messages with zero, 799 00:40:18,990 --> 00:40:21,130 zero at the end and come back down to that state. 800 00:40:21,130 --> 00:40:23,760 So my messages will always start with zero, zero 801 00:40:23,760 --> 00:40:26,430 to force the register to the zero, zero state. 802 00:40:26,430 --> 00:40:28,230 And they'll end with zero, zero. 803 00:40:28,230 --> 00:40:30,240 OK? 804 00:40:30,240 --> 00:40:33,150 The set of possible code words is a set of parity bits 805 00:40:33,150 --> 00:40:36,510 I omit along the way as I navigate through the trellis. 806 00:40:36,510 --> 00:40:40,170 Is the set of code words, does the set of code words 807 00:40:40,170 --> 00:40:41,370 constitute a linear code? 808 00:40:41,370 --> 00:40:42,203 That's the question. 809 00:40:47,320 --> 00:40:48,310 Maybe not obvious. 810 00:40:48,310 --> 00:40:50,140 Right? 811 00:40:50,140 --> 00:40:51,640 The way you answer that is actually 812 00:40:51,640 --> 00:40:54,660 thinking back to this setting. 813 00:40:54,660 --> 00:40:58,840 So one particular code word would 814 00:40:58,840 --> 00:41:02,950 correspond to a particular input sequence that generated it. 815 00:41:02,950 --> 00:41:06,010 A particular data message sequence that generated it. 816 00:41:06,010 --> 00:41:09,520 Another code would correspond to another message sequence. 817 00:41:09,520 --> 00:41:12,010 And the question is, is there a message sequence 818 00:41:12,010 --> 00:41:14,290 that would generate the sum of these two code words 819 00:41:14,290 --> 00:41:15,460 that you have? 820 00:41:15,460 --> 00:41:18,220 And actually, it turns out that the answer is yes 821 00:41:18,220 --> 00:41:21,040 because these parity relationships are based 822 00:41:21,040 --> 00:41:24,190 on a nice linear operation. 823 00:41:24,190 --> 00:41:24,850 OK? 824 00:41:24,850 --> 00:41:29,350 So it turns out that the set of code words that you generate 825 00:41:29,350 --> 00:41:30,550 constitutes a linear code. 826 00:41:33,140 --> 00:41:38,370 So if you were going to think of a minimum Hamming 827 00:41:38,370 --> 00:41:41,160 distance for this code, what would you 828 00:41:41,160 --> 00:41:42,180 want to be thinking of? 829 00:41:46,030 --> 00:41:49,610 I don't know if I've actually drawn this correctly right now. 830 00:41:49,610 --> 00:41:51,650 Has anyone spotted any errors along the way? 831 00:41:51,650 --> 00:41:52,670 Or do I have it right? 832 00:42:00,300 --> 00:42:01,070 Seems to be OK. 833 00:42:07,940 --> 00:42:10,280 What would, how would you look for a minimum Hamming 834 00:42:10,280 --> 00:42:12,440 distance in the set of code words generated 835 00:42:12,440 --> 00:42:16,750 over this window? 836 00:42:16,750 --> 00:42:18,550 AUDIENCE: [INAUDIBLE] 837 00:42:18,550 --> 00:42:19,960 PROFESSOR: Sorry? 838 00:42:19,960 --> 00:42:21,978 I didn't hear where that came from. 839 00:42:21,978 --> 00:42:22,478 Yeah. 840 00:42:26,780 --> 00:42:29,720 Can you speak up? 841 00:42:29,720 --> 00:42:32,300 The minimum number of ones in a non-zero code word. 842 00:42:32,300 --> 00:42:33,070 Right? 843 00:42:33,070 --> 00:42:35,270 So it would be the weight, the minimum weight code 844 00:42:35,270 --> 00:42:37,850 word you'd find among all the number of code words. 845 00:42:37,850 --> 00:42:41,000 So you would have to find a path starting here. 846 00:42:41,000 --> 00:42:43,910 All my code words are going to start here and there. 847 00:42:43,910 --> 00:42:46,880 You have to find a path through this that picks up 848 00:42:46,880 --> 00:42:48,680 the minimum number of ones. 849 00:42:48,680 --> 00:42:51,670 A path that's different from the all zeros path. 850 00:42:51,670 --> 00:42:52,190 OK? 851 00:42:52,190 --> 00:42:55,460 Find a path through there which has the minimum number of ones 852 00:42:55,460 --> 00:42:57,530 in it and the code word. 853 00:42:57,530 --> 00:42:59,030 So what would that be? 854 00:43:03,940 --> 00:43:09,590 In this particular case, maybe this path 855 00:43:09,590 --> 00:43:11,300 we highlighted in another color. 856 00:43:20,742 --> 00:43:22,200 I don't have-- this is not a proof. 857 00:43:22,200 --> 00:43:24,900 This is just a suggestion that this might be it. 858 00:43:27,660 --> 00:43:29,880 And you would have to draw in all the paths, 859 00:43:29,880 --> 00:43:31,420 explore all the other paths. 860 00:43:31,420 --> 00:43:33,970 But what would be the minimum weight along this one? 861 00:43:33,970 --> 00:43:34,470 Let's see. 862 00:43:34,470 --> 00:43:36,300 I've got 1, 1 there. 863 00:43:36,300 --> 00:43:37,620 1, zero here. 864 00:43:37,620 --> 00:43:38,400 1, 1 there. 865 00:43:38,400 --> 00:43:42,408 So I would get a weight of 5 on this path. 866 00:43:42,408 --> 00:43:44,700 Now the question is whether you could find another path 867 00:43:44,700 --> 00:43:49,360 with a smaller number of ones attached to the code word. 868 00:43:49,360 --> 00:43:52,290 And I think if you work this out in detail on [INAUDIBLE],, 869 00:43:52,290 --> 00:43:54,690 that you'll find that you're actually stuck with 5. 870 00:43:54,690 --> 00:43:56,940 OK? 871 00:43:56,940 --> 00:44:00,030 Now it turns out that the interpretation of this number 872 00:44:00,030 --> 00:44:02,400 is not quite as straightforward as the interpretation 873 00:44:02,400 --> 00:44:05,080 of the minimum Hamming distance in block codes. 874 00:44:05,080 --> 00:44:08,880 And the reason is that actually this 875 00:44:08,880 --> 00:44:11,820 is a more complicated kind of picture 876 00:44:11,820 --> 00:44:14,220 because it continues on with the structure. 877 00:44:17,660 --> 00:44:20,230 So we don't actually call it the minimum Hamming distance. 878 00:44:20,230 --> 00:44:21,803 We call it the free distance. 879 00:44:25,430 --> 00:44:29,180 So I'm just trying to evoke this. 880 00:44:29,180 --> 00:44:31,850 So the minimum weight code word you 881 00:44:31,850 --> 00:44:33,320 find among the non-zero code words 882 00:44:33,320 --> 00:44:36,770 will indeed be a code word of eight 5. 883 00:44:36,770 --> 00:44:39,260 But the interpretation of that number 884 00:44:39,260 --> 00:44:43,680 may not be directly as simple as in the case of the Hamming 885 00:44:43,680 --> 00:44:44,180 distance. 886 00:44:48,900 --> 00:44:49,680 But it's close. 887 00:44:49,680 --> 00:44:50,180 OK? 888 00:44:50,180 --> 00:44:52,440 So what it really tells you is that, 889 00:44:52,440 --> 00:44:58,500 over a data length that is maybe not 890 00:44:58,500 --> 00:45:01,530 much longer than this, the code words 891 00:45:01,530 --> 00:45:06,250 that you have are separated by this distance minimum. 892 00:45:06,250 --> 00:45:12,450 So you might expect that you could correct two bit errors 893 00:45:12,450 --> 00:45:14,640 over data lengths that correspond 894 00:45:14,640 --> 00:45:18,030 to code words that are somewhat longer than this perhaps. 895 00:45:18,030 --> 00:45:19,650 OK? 896 00:45:19,650 --> 00:45:21,750 So that's all very hand-wavy. 897 00:45:21,750 --> 00:45:23,850 But that's all we're going to do with the notion 898 00:45:23,850 --> 00:45:24,660 of free distance. 899 00:45:24,660 --> 00:45:28,290 So this is more complicated to deal with than a block code. 900 00:45:28,290 --> 00:45:31,680 But the free distance actually has that kind of intuition. 901 00:45:31,680 --> 00:45:35,790 It has the intuition of minimum Hamming distance 902 00:45:35,790 --> 00:45:39,930 locally over this window of data. 903 00:45:39,930 --> 00:45:42,300 Even if this went on for thousands of bits, 904 00:45:42,300 --> 00:45:45,420 if you got a burst of errors in this stretch that 905 00:45:45,420 --> 00:45:48,300 had up to two errors, you could correct them. 906 00:45:48,300 --> 00:45:49,890 Now we haven't talked about decoding. 907 00:45:49,890 --> 00:45:51,790 We're going to talk about that the next time. 908 00:45:51,790 --> 00:45:52,290 OK. 909 00:45:56,890 --> 00:45:58,210 So that answers this piece. 910 00:46:03,790 --> 00:46:06,160 Now let me say one thing about decoding, just 911 00:46:06,160 --> 00:46:09,830 to set us up for next time. 912 00:46:09,830 --> 00:46:13,600 If I didn't have any noise in my channel, 913 00:46:13,600 --> 00:46:19,490 it actually turns out that decoding is pretty trivial. 914 00:46:19,490 --> 00:46:20,240 How is that? 915 00:46:20,240 --> 00:46:24,200 If I gave you the sequence of parity bits, 916 00:46:24,200 --> 00:46:26,640 can you think of a way that you could recover the input 917 00:46:26,640 --> 00:46:27,140 sequence? 918 00:46:37,460 --> 00:46:39,230 Yeah? 919 00:46:39,230 --> 00:46:40,823 AUDIENCE: [INAUDIBLE] 920 00:46:40,823 --> 00:46:41,490 PROFESSOR: Good. 921 00:46:41,490 --> 00:46:42,190 Yeah. 922 00:46:42,190 --> 00:46:47,830 You see, if I add these two, I get x n minus 1 equals P zero 923 00:46:47,830 --> 00:46:52,450 n plus P1 n. 924 00:46:52,450 --> 00:46:54,790 So if you give me the parity bit stream, 925 00:46:54,790 --> 00:46:57,310 I can reconstruct for you exactly the input 926 00:46:57,310 --> 00:46:59,590 with a one step delay. 927 00:46:59,590 --> 00:47:00,420 That's pretty good. 928 00:47:00,420 --> 00:47:01,253 I'm happy with that. 929 00:47:01,253 --> 00:47:06,640 If it's taken me minutes for the signal to reach me from Saturn, 930 00:47:06,640 --> 00:47:09,850 I'm happy with a one step delay here in decoding. 931 00:47:09,850 --> 00:47:10,510 All right? 932 00:47:10,510 --> 00:47:13,630 So in the absence of noise, the inversion is simple. 933 00:47:13,630 --> 00:47:17,410 The inversion meaning deducing the input message bits 934 00:47:17,410 --> 00:47:19,210 from the output, the parity bits. 935 00:47:19,210 --> 00:47:22,450 And this is a theme you'll see in many other settings. 936 00:47:22,450 --> 00:47:24,467 If there's no noise, inversion is easy. 937 00:47:24,467 --> 00:47:26,050 You can look at the output of a system 938 00:47:26,050 --> 00:47:28,630 and figure out what the input was. 939 00:47:28,630 --> 00:47:31,210 If you know exactly how the system was creating 940 00:47:31,210 --> 00:47:32,445 the output from the input. 941 00:47:32,445 --> 00:47:34,570 But in the presence of noise, you've got a problem. 942 00:47:34,570 --> 00:47:37,300 Because you see, if you have these parity 943 00:47:37,300 --> 00:47:39,130 bits corrupted at some rate-- 944 00:47:39,130 --> 00:47:41,410 every few bits, you've got errors-- 945 00:47:41,410 --> 00:47:43,210 well, you're interpreted message is 946 00:47:43,210 --> 00:47:46,040 going to have that same kind of error rate. 947 00:47:46,040 --> 00:47:46,540 OK? 948 00:47:46,540 --> 00:47:48,248 So it's really, in the presence of noise, 949 00:47:48,248 --> 00:47:50,630 it's an unsatisfactory way to do it. 950 00:47:50,630 --> 00:47:53,840 So this doesn't work. 951 00:47:53,840 --> 00:47:57,080 We'll be looking at something more careful next time. 952 00:47:57,080 --> 00:47:57,580 OK? 953 00:47:57,580 --> 00:48:02,770 So well actually, since I have you here, 954 00:48:02,770 --> 00:48:05,110 let me put up the spot quiz. 955 00:48:05,110 --> 00:48:06,820 We haven't quite hit the mark. 956 00:48:06,820 --> 00:48:11,260 So can you answer these for me? 957 00:48:11,260 --> 00:48:13,030 What's the constraint length of this code? 958 00:48:17,880 --> 00:48:19,827 Anyone? 959 00:48:19,827 --> 00:48:20,660 Who hasn't answered? 960 00:48:20,660 --> 00:48:21,360 Yeah? 961 00:48:21,360 --> 00:48:21,860 4. 962 00:48:21,860 --> 00:48:22,360 Right? 963 00:48:22,360 --> 00:48:24,610 Because you've got xn and xn minus 1, xn minus 2, 964 00:48:24,610 --> 00:48:25,850 xn minus 3. 965 00:48:25,850 --> 00:48:28,760 That's the largest window over which you're picking things. 966 00:48:28,760 --> 00:48:29,810 What about the code rate? 967 00:48:34,370 --> 00:48:35,940 1 over 3? 968 00:48:35,940 --> 00:48:36,440 Right? 969 00:48:36,440 --> 00:48:37,982 Because for every message bit, you're 970 00:48:37,982 --> 00:48:39,737 generating three parity bits. 971 00:48:39,737 --> 00:48:41,570 You're going to shift out those three parity 972 00:48:41,570 --> 00:48:44,300 bits before you do the shifting on the shift register. 973 00:48:44,300 --> 00:48:46,322 So the code rate is 1 over 3. 974 00:48:46,322 --> 00:48:47,780 The coefficients of the generators, 975 00:48:47,780 --> 00:48:49,470 of course, you can read up there. 976 00:48:49,470 --> 00:48:52,400 What about the number of states in the state machine here? 977 00:49:00,240 --> 00:49:01,290 What? 978 00:49:01,290 --> 00:49:01,920 8. 979 00:49:01,920 --> 00:49:02,700 Right? 980 00:49:02,700 --> 00:49:05,790 Because constraint length is 4. 981 00:49:05,790 --> 00:49:07,830 But one of those is the input. 982 00:49:07,830 --> 00:49:10,380 So you've got three bits that you're storing in memory. 983 00:49:10,380 --> 00:49:11,460 2 to the 3 is 8. 984 00:49:11,460 --> 00:49:14,560 So the number of states in the machine is 8. 985 00:49:14,560 --> 00:49:15,180 OK? 986 00:49:15,180 --> 00:49:16,870 So more complicated picture than this. 987 00:49:16,870 --> 00:49:18,190 But the same principle. 988 00:49:18,190 --> 00:49:18,690 All right. 989 00:49:18,690 --> 00:49:20,990 We'll continue next time.