1 00:00:00,000 --> 00:00:02,490 The following content is provided under a Creative 2 00:00:02,490 --> 00:00:04,059 Commons license. 3 00:00:04,059 --> 00:00:06,360 Your support will help MIT OpenCourseWare 4 00:00:06,360 --> 00:00:10,720 continue to offer high-quality educational resources for free. 5 00:00:10,720 --> 00:00:13,350 To make a donation or view additional materials 6 00:00:13,350 --> 00:00:17,300 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,300 --> 00:00:18,300 at ocw.mit.edu. 8 00:00:25,305 --> 00:00:26,680 PROFESSOR: All right, we're going 9 00:00:26,680 --> 00:00:29,310 to continue talking about convolutional code. 10 00:00:29,310 --> 00:00:32,640 So I want to give you a quick reminder of how coding works 11 00:00:32,640 --> 00:00:35,190 and then talk to you about decoding. 12 00:00:35,190 --> 00:00:37,330 Can you hear me OK now? 13 00:00:37,330 --> 00:00:37,830 All right? 14 00:00:37,830 --> 00:00:38,330 OK. 15 00:00:42,410 --> 00:00:48,000 So we talked in terms of a state diagram, 16 00:00:48,000 --> 00:00:51,470 but let me remind you of the shift register picture was. 17 00:00:51,470 --> 00:00:55,970 So we had a two-stage shift register. 18 00:00:55,970 --> 00:01:00,860 For this particular example, we had xn, a stream 19 00:01:00,860 --> 00:01:03,210 of data being fed in here. 20 00:01:03,210 --> 00:01:04,760 So since this is a shift register, 21 00:01:04,760 --> 00:01:09,830 what sits in here at time n is the previous input. 22 00:01:09,830 --> 00:01:15,680 What sits here is the input from 2 times ago. 23 00:01:15,680 --> 00:01:19,370 And you can then feed these off and get your parity check. 24 00:01:19,370 --> 00:01:27,940 So take these in particular combinations 25 00:01:27,940 --> 00:01:29,670 and make your parity checks. 26 00:01:29,670 --> 00:01:37,725 So you can have one box spitting out a p0 of n. 27 00:01:37,725 --> 00:01:39,100 And then you can have another box 28 00:01:39,100 --> 00:01:44,350 that takes these same outputs from the shift register 29 00:01:44,350 --> 00:01:47,320 and puts out-- 30 00:01:47,320 --> 00:01:50,590 let me just show them. 31 00:01:50,590 --> 00:01:55,270 Actually, why don't I just put it here? 32 00:01:55,270 --> 00:01:58,510 You generate a bunch of party check bits. 33 00:01:58,510 --> 00:02:00,820 And I've shown an example on top where-- 34 00:02:00,820 --> 00:02:03,630 this is the same one I used last time. 35 00:02:03,630 --> 00:02:05,080 p0 at time n-- 36 00:02:05,080 --> 00:02:06,610 I should have had an n there-- 37 00:02:06,610 --> 00:02:10,419 is xn plus xn minus 1 plus xn minus 2. 38 00:02:10,419 --> 00:02:13,660 And p1, xn, and at xn minus 2. 39 00:02:13,660 --> 00:02:16,630 And we skip the xn minus 1. 40 00:02:16,630 --> 00:02:21,940 But you can choose different coefficients there. 41 00:02:21,940 --> 00:02:23,500 Different coefficients will give you 42 00:02:23,500 --> 00:02:25,100 codes that have different properties. 43 00:02:25,100 --> 00:02:30,880 So the choices in the code are how many shift registers do 44 00:02:30,880 --> 00:02:32,810 you have, so how much memory. 45 00:02:32,810 --> 00:02:45,400 The constraint length here is equal to what? 46 00:02:45,400 --> 00:02:48,080 Constraint length in this particular example 47 00:02:48,080 --> 00:02:49,210 on the slide? 48 00:02:49,210 --> 00:02:50,483 AUDIENCE: 3. 49 00:02:50,483 --> 00:02:51,400 PROFESSOR: k equals 3? 50 00:02:51,400 --> 00:02:54,010 Oh, you can see it. 51 00:02:54,010 --> 00:02:56,560 It's the number of message bits that 52 00:02:56,560 --> 00:03:00,692 are involved in generating a parity bit at the maximum. 53 00:03:00,692 --> 00:03:01,900 It's actually not the number. 54 00:03:01,900 --> 00:03:04,720 It's the window over which you're 55 00:03:04,720 --> 00:03:08,170 taking message bits to combine to make the parity bits. 56 00:03:11,720 --> 00:03:15,660 All right, so for instance, if you just had p1, 57 00:03:15,660 --> 00:03:18,270 you would still say that your constraint length is 3, 58 00:03:18,270 --> 00:03:21,610 because you're involving a window of three message bits. 59 00:03:21,610 --> 00:03:24,850 It's the span over which you're extending. 60 00:03:24,850 --> 00:03:27,130 All right, now, in terms of interpreting this, 61 00:03:27,130 --> 00:03:31,380 we've got the possible states of the shift register combination. 62 00:03:31,380 --> 00:03:37,900 So 0, 0; 0, 1' 1, 0' 1, 1, these are the four possible states. 63 00:03:37,900 --> 00:03:46,060 So in general, what you have is for a constraint length k, 64 00:03:46,060 --> 00:03:53,680 you've got 2 to the k minus 1 states, because one of these 65 00:03:53,680 --> 00:03:54,615 is the input. 66 00:03:54,615 --> 00:03:56,240 And then the other is stored in memory. 67 00:03:56,240 --> 00:03:58,280 So you've got k minus 1 stored in memory. 68 00:03:58,280 --> 00:04:01,030 So that's the number of states that you have. 69 00:04:01,030 --> 00:04:03,880 And that's how these circles are labeled here. 70 00:04:03,880 --> 00:04:06,610 And then on each of the arcs, what you have 71 00:04:06,610 --> 00:04:12,040 is the message bit that's coming in at that time and the parity 72 00:04:12,040 --> 00:04:14,020 bits that are emitted. 73 00:04:14,020 --> 00:04:19,930 So for instance, from 0, 0, if you've got 0 here 74 00:04:19,930 --> 00:04:23,980 and 0 here, the only places you can go to at the next step 75 00:04:23,980 --> 00:04:27,220 are 0, 0, and 0, 1, because you can either-- 76 00:04:27,220 --> 00:04:30,910 sorry, 0, 0 and 1, 0, because you can either 77 00:04:30,910 --> 00:04:33,660 feet in a 0 or a 1 from here. 78 00:04:33,660 --> 00:04:36,580 If you feed in a 0, then at the next state, you're still in 0. 79 00:04:36,580 --> 00:04:40,660 If you feed in a 1, then at the next state, you're 1, 0. 80 00:04:40,660 --> 00:04:43,550 So those are the only possibilities from 0, 0. 81 00:04:43,550 --> 00:04:48,280 And if you had a 1 in, you would go from 0, 0 to 1, 0. 82 00:04:48,280 --> 00:04:50,780 And what would be your parity check bits? 83 00:04:50,780 --> 00:04:54,040 So if you had 1 at the input and you 84 00:04:54,040 --> 00:04:56,640 have the parity check expressions 85 00:04:56,640 --> 00:04:59,950 that I have up here, you see that what you would be emitting 86 00:04:59,950 --> 00:05:03,066 would be a 1 and a 1. 87 00:05:03,066 --> 00:05:05,910 Is that right? 88 00:05:05,910 --> 00:05:09,510 If you had a 1 in the input, 0, 0, 89 00:05:09,510 --> 00:05:12,648 and you use these two parity bits, you generate a 1 and a 1. 90 00:05:12,648 --> 00:05:14,190 So that's how these arcs are labeled. 91 00:05:22,560 --> 00:05:24,240 Now we said, to actually understand 92 00:05:24,240 --> 00:05:27,600 the convolutional code well, what you really 93 00:05:27,600 --> 00:05:29,970 want to do is translate the state transition 94 00:05:29,970 --> 00:05:32,700 diagram to a trellis diagram. 95 00:05:32,700 --> 00:05:35,730 This is just showing two stages of the trellis. 96 00:05:35,730 --> 00:05:37,980 In general, the trellis would be the state diagram 97 00:05:37,980 --> 00:05:42,700 unfolded over the whole time interval of interest. 98 00:05:42,700 --> 00:05:43,720 So it's the same thing. 99 00:05:43,720 --> 00:05:44,650 It's just that we've-- 100 00:05:44,650 --> 00:05:45,150 let's see. 101 00:05:45,150 --> 00:05:50,910 We've chosen to write the states in binary counting order, 102 00:05:50,910 --> 00:05:53,540 so 0, 0; 0, 1; 1, 0; 1, 1. 103 00:05:53,540 --> 00:05:57,040 It was arranged slightly differently here. 104 00:05:57,040 --> 00:05:58,690 Apart from that, it's the same thing. 105 00:05:58,690 --> 00:06:00,900 So we're drawing the state diagram here. 106 00:06:00,900 --> 00:06:02,760 We're drawing the same state diagram 107 00:06:02,760 --> 00:06:06,530 here except this is representing the state possibilities 108 00:06:06,530 --> 00:06:07,740 at time-- 109 00:06:07,740 --> 00:06:08,772 at one particular time. 110 00:06:08,772 --> 00:06:10,230 And this is the state possibilities 111 00:06:10,230 --> 00:06:11,672 of the next particular time. 112 00:06:11,672 --> 00:06:13,380 So the state transition arrows are always 113 00:06:13,380 --> 00:06:16,170 going from one stage to the next, all right? 114 00:06:16,170 --> 00:06:19,290 So the arrow that we saw here, which takes us 115 00:06:19,290 --> 00:06:22,710 from 0, 0 to 1, 0 is going to take us 116 00:06:22,710 --> 00:06:25,620 from this box to this box. 117 00:06:25,620 --> 00:06:28,670 And what it emits on the way is the 1 and the-- 118 00:06:28,670 --> 00:06:30,690 sorry, the 1, 1. 119 00:06:30,690 --> 00:06:32,490 What it emits is the 1, 1. 120 00:06:32,490 --> 00:06:34,380 So each arc is labeled in the same way. 121 00:06:34,380 --> 00:06:37,830 This is just a rearrangement. 122 00:06:37,830 --> 00:06:40,630 Now, the nice thing I mentioned last time, the nice thing 123 00:06:40,630 --> 00:06:44,020 about this is, when you have this in binary counting order, 124 00:06:44,020 --> 00:06:47,380 then the upper arc of the two that emanate from each box 125 00:06:47,380 --> 00:06:49,540 corresponds to an input of 0. 126 00:06:49,540 --> 00:06:51,850 The lower arc corresponds to an input of 1. 127 00:06:51,850 --> 00:06:54,970 So you can actually dispense with the index 128 00:06:54,970 --> 00:06:57,610 or with the bit that's in front of the stroke there. 129 00:06:57,610 --> 00:07:02,530 So you can just make do with labeling by the parity bits. 130 00:07:02,530 --> 00:07:03,860 So you'll get used to that. 131 00:07:03,860 --> 00:07:05,580 0 is the upward movement. 132 00:07:05,580 --> 00:07:08,630 And 1 is a downward movement. 133 00:07:08,630 --> 00:07:13,070 So if you're thinking at the transmitter-- 134 00:07:13,070 --> 00:07:15,070 by the way, I hope I've made these changes well. 135 00:07:15,070 --> 00:07:17,650 I had an earlier version of this slide, 136 00:07:17,650 --> 00:07:20,560 but I changed it to work for a new set of parity bits, which 137 00:07:20,560 --> 00:07:23,440 meant I had to go in and change every one of these transitions. 138 00:07:23,440 --> 00:07:26,080 So you might see the odd bug here and there. 139 00:07:26,080 --> 00:07:28,030 But hopefully this is correct and consistent 140 00:07:28,030 --> 00:07:31,180 with the state transition diagram I showed you. 141 00:07:31,180 --> 00:07:33,040 So what we're saying is now suppose you're 142 00:07:33,040 --> 00:07:35,950 starting off in the 0, 0 state. 143 00:07:35,950 --> 00:07:37,990 And you get the sequence of message bits. 144 00:07:37,990 --> 00:07:40,880 So 0, 1, 1, 1 is your message. 145 00:07:40,880 --> 00:07:44,260 And then you bring it back to the 0 state again by appending 146 00:07:44,260 --> 00:07:45,760 two 0's. 147 00:07:45,760 --> 00:07:49,720 What's the path that you traverse through the trellis? 148 00:07:49,720 --> 00:07:52,750 Well, you're starting off in the 0 state. 149 00:07:52,750 --> 00:07:56,140 Every time you have a 0 in the message, 150 00:07:56,140 --> 00:07:57,520 you take the upper branch. 151 00:07:57,520 --> 00:08:00,460 Whenever you have a 1, you take the lower branch of the two 152 00:08:00,460 --> 00:08:01,733 that are available to you. 153 00:08:01,733 --> 00:08:03,400 So you can see very quickly how to steer 154 00:08:03,400 --> 00:08:07,480 through this trellis for any particular message sequence. 155 00:08:07,480 --> 00:08:10,120 So this is the upper one of the two here, 156 00:08:10,120 --> 00:08:13,390 and then the lower one of the two here because it's a 1, 157 00:08:13,390 --> 00:08:16,420 then the lower one of the two here because it's a 1, 158 00:08:16,420 --> 00:08:19,780 and the lower one of the two here because it's a 1, and then 159 00:08:19,780 --> 00:08:23,140 the upper one because it's a 0, and the upper one 160 00:08:23,140 --> 00:08:24,563 because it's a 0. 161 00:08:24,563 --> 00:08:26,230 So that's your path through the trellis. 162 00:08:26,230 --> 00:08:27,790 It's told to you by the message bits. 163 00:08:31,520 --> 00:08:33,710 You should also remember, by the way, this diagram 164 00:08:33,710 --> 00:08:38,539 hides a little bit, because I have just a box here 165 00:08:38,539 --> 00:08:44,450 for something that's actually a pair of registers. 166 00:08:44,450 --> 00:08:50,210 So when I just show the box, let's say, at this point, 167 00:08:50,210 --> 00:08:52,490 this actually has the 1 and the 0 sitting in it. 168 00:08:57,330 --> 00:09:00,290 So if you were just looking at this box and what was in it, 169 00:09:00,290 --> 00:09:03,210 if I just gave you the contents of that box, 170 00:09:03,210 --> 00:09:05,790 could you tell me what the input was of the previous time? 171 00:09:05,790 --> 00:09:09,450 If I just told you that the contents of that box 172 00:09:09,450 --> 00:09:11,820 are 1 and 0, can you tell me what the input 173 00:09:11,820 --> 00:09:14,840 was at the previous time? 174 00:09:14,840 --> 00:09:15,655 Yes? 175 00:09:15,655 --> 00:09:17,030 It's just what got fed in, right? 176 00:09:17,030 --> 00:09:18,980 It's just the one that got fed in. 177 00:09:18,980 --> 00:09:21,680 So this diagram is fine, but we've 178 00:09:21,680 --> 00:09:23,105 suppressed a little bit there. 179 00:09:23,105 --> 00:09:26,870 There are occasions, especially on homework and quiz problems, 180 00:09:26,870 --> 00:09:32,090 where you're given the contents of the shift registers. 181 00:09:32,090 --> 00:09:35,080 And you're asked to figure out what happened at the last time 182 00:09:35,080 --> 00:09:37,123 step, what message bit came in. 183 00:09:37,123 --> 00:09:38,540 So really, don't forget that there 184 00:09:38,540 --> 00:09:41,480 is a link between the two. 185 00:09:41,480 --> 00:09:43,830 OK, so the steering is straightforward. 186 00:09:43,830 --> 00:09:46,760 Now, what's the code word that's emitted? 187 00:09:46,760 --> 00:09:50,510 Well, it's the parity bits that you encounter on the arcs. 188 00:09:50,510 --> 00:09:52,730 So on this upper arc here, you've 189 00:09:52,730 --> 00:09:54,130 got a 0, 0 that's emitted. 190 00:09:54,130 --> 00:09:56,180 So that's what you're going to emit. 191 00:09:56,180 --> 00:10:00,510 That's the part of the code word generated by that message bit. 192 00:10:00,510 --> 00:10:02,960 And then on the lower arc, you emit 1, 1. 193 00:10:02,960 --> 00:10:06,170 On the lower arc, you emit 0, 1, lower arc, 194 00:10:06,170 --> 00:10:11,510 you emit 1, 0, and 0, 1, and then 1, 1. 195 00:10:11,510 --> 00:10:14,680 So that's the code word. 196 00:10:14,680 --> 00:10:16,630 So the set of all possible code words 197 00:10:16,630 --> 00:10:18,880 that you can get with this convolutional code 198 00:10:18,880 --> 00:10:20,620 corresponds to the set of all paths you 199 00:10:20,620 --> 00:10:23,080 can take through the trellis. 200 00:10:23,080 --> 00:10:27,040 If you're starting at 0, 0, 0, then it's 201 00:10:27,040 --> 00:10:31,930 the set of all paths starting at 0, 0, 0. 202 00:10:31,930 --> 00:10:32,800 So let's see. 203 00:10:32,800 --> 00:10:39,610 Roughly speaking, can you tell me, if I've got l stages-- 204 00:10:39,610 --> 00:10:43,142 when I say stages, I mean time, if you 205 00:10:43,142 --> 00:10:45,100 want to think of these as happening on a clock. 206 00:10:45,100 --> 00:10:49,540 If I've got l stages here and I'm starting off with the 0 207 00:10:49,540 --> 00:10:53,650 state there, for a large l, roughly how many possible paths 208 00:10:53,650 --> 00:10:54,190 do I have? 209 00:11:01,560 --> 00:11:02,150 Any thoughts? 210 00:11:04,880 --> 00:11:05,880 2 to the l? 211 00:11:05,880 --> 00:11:06,870 Yeah. 212 00:11:06,870 --> 00:11:10,350 Because you see here, coming out of a box here on each stage, 213 00:11:10,350 --> 00:11:12,300 you've got two choices. 214 00:11:12,300 --> 00:11:14,740 And you've got those two choices for l stages. 215 00:11:14,740 --> 00:11:21,320 So you've got approximately 2 to the l possible paths. 216 00:11:21,320 --> 00:11:25,330 Now, I say approximately, because well, in this case, 217 00:11:25,330 --> 00:11:25,830 it's fine. 218 00:11:25,830 --> 00:11:29,520 But now if you're a lot to start from other starting states, 219 00:11:29,520 --> 00:11:31,560 then you will have to take account of that. 220 00:11:31,560 --> 00:11:32,520 But it's of that order. 221 00:11:32,520 --> 00:11:33,982 It's exponential. 222 00:11:33,982 --> 00:11:35,940 The number of possible paths that you can have, 223 00:11:35,940 --> 00:11:38,370 the number of code words is exponential in the length 224 00:11:38,370 --> 00:11:41,648 of the trellis, right? 225 00:11:41,648 --> 00:11:43,440 OK, so that's a large number of code words. 226 00:11:47,570 --> 00:11:52,520 Our focus, though, is going to be on decoding today. 227 00:11:52,520 --> 00:11:56,420 What I did so far was just review what we saw for coding. 228 00:11:56,420 --> 00:11:58,940 We're interested in decoding now. 229 00:11:58,940 --> 00:12:04,610 So at the receiver, what you have is a knowledge of what 230 00:12:04,610 --> 00:12:05,170 the code is. 231 00:12:05,170 --> 00:12:06,170 So you have the trellis. 232 00:12:06,170 --> 00:12:07,500 You know what the labels are. 233 00:12:07,500 --> 00:12:11,000 You know that things are going to start in the zero state. 234 00:12:11,000 --> 00:12:14,637 And then you get your received signal. 235 00:12:14,637 --> 00:12:16,470 Now, what I've shown here is that, actually, 236 00:12:16,470 --> 00:12:20,300 your received signal is not necessarily going to be 237 00:12:20,300 --> 00:12:22,280 0's and 1's. 238 00:12:22,280 --> 00:12:25,160 It's probably going to be samples of some voltage, where 239 00:12:25,160 --> 00:12:26,270 you've got some waveform. 240 00:12:26,270 --> 00:12:27,050 You process it. 241 00:12:27,050 --> 00:12:28,670 And then you take a sample. 242 00:12:28,670 --> 00:12:30,680 And what you've got is a sample of some voltage. 243 00:12:30,680 --> 00:12:33,320 So you're typically looking at real numbers 244 00:12:33,320 --> 00:12:38,530 that you then have to decide whether to call as a 0 or a 1. 245 00:12:38,530 --> 00:12:46,910 OK, so yeah, maybe this is 0, 0, maybe 0, 1; 0, 1, probably 1, 246 00:12:46,910 --> 00:12:53,390 0; 0, 1; 1, 0, yeah? 247 00:12:53,390 --> 00:12:56,690 So if you were forced to choose, if you had a threshold of 0.5, 248 00:12:56,690 --> 00:12:59,690 for instance, and this was the range, if nominally 249 00:12:59,690 --> 00:13:04,200 these were supposed to be at 0 and 1, 250 00:13:04,200 --> 00:13:07,880 then you might actually be willing to call 251 00:13:07,880 --> 00:13:09,090 this one way or another. 252 00:13:09,090 --> 00:13:12,410 So if I was to draw this on the real axis 253 00:13:12,410 --> 00:13:16,910 thinking of a voltage, so we've got 0 volts that we're 254 00:13:16,910 --> 00:13:20,900 expecting, 1 volt or something proportional to 1 255 00:13:20,900 --> 00:13:22,073 volt that we're expecting. 256 00:13:22,073 --> 00:13:23,990 These are the two possible values depending on 257 00:13:23,990 --> 00:13:26,660 whether a 0 is sent or a 1 is sent. 258 00:13:26,660 --> 00:13:28,190 This is because we've coded the bits 259 00:13:28,190 --> 00:13:31,400 at the transmitter for physical transmission 260 00:13:31,400 --> 00:13:33,393 on a continuous time channel. 261 00:13:33,393 --> 00:13:35,810 And then at the receiving end, we're doing some processing 262 00:13:35,810 --> 00:13:38,180 and extracting samples, right? 263 00:13:38,180 --> 00:13:40,190 But because of noise, what might happen 264 00:13:40,190 --> 00:13:44,450 is that you get samples anywhere around the 0 265 00:13:44,450 --> 00:13:48,170 or anywhere around the 1, depending 266 00:13:48,170 --> 00:13:50,330 on the particular transmission instance. 267 00:13:50,330 --> 00:13:52,670 It'll vary from one instant to the next. 268 00:13:52,670 --> 00:13:56,160 And if the noise is really bad, then of course, 269 00:13:56,160 --> 00:13:58,280 what started off as a 0 here with the noise 270 00:13:58,280 --> 00:14:00,020 added to it, by the time you sample it, 271 00:14:00,020 --> 00:14:03,740 might fall in a region where you call it a 1. 272 00:14:03,740 --> 00:14:05,390 So there is an intermediate step. 273 00:14:05,390 --> 00:14:07,160 And very often, you have access to that. 274 00:14:10,450 --> 00:14:16,060 And then you've got to figure out how to do your decoding. 275 00:14:16,060 --> 00:14:20,283 All right, is this the same slide? 276 00:14:20,283 --> 00:14:21,700 Or does it say anything different? 277 00:14:21,700 --> 00:14:24,168 OK. 278 00:14:24,168 --> 00:14:25,460 So what are we going to do now? 279 00:14:25,460 --> 00:14:28,880 We're going to, of all the paths available to us, 280 00:14:28,880 --> 00:14:32,690 we're going to try and find the path along which 281 00:14:32,690 --> 00:14:36,890 the emitted parity bits come closest, in some sense, 282 00:14:36,890 --> 00:14:40,830 to the sequence of samples here. 283 00:14:40,830 --> 00:14:43,190 That's, if you were doing minimum distance, 284 00:14:43,190 --> 00:14:45,530 in some sense, that's what you'd want to do. 285 00:14:45,530 --> 00:14:49,910 If you believed that errors further away from 0 286 00:14:49,910 --> 00:14:53,180 are less likely than errors close to 0, 287 00:14:53,180 --> 00:14:57,710 then you would want to have a reconstructed set of parity 288 00:14:57,710 --> 00:14:59,480 bits along whatever path you choose 289 00:14:59,480 --> 00:15:01,970 to come close to the values there. 290 00:15:06,380 --> 00:15:08,480 Now, it turns out that it's actually simpler 291 00:15:08,480 --> 00:15:12,710 initially to think of first making a decision 292 00:15:12,710 --> 00:15:15,530 to call these 0's or 1's and then finding 293 00:15:15,530 --> 00:15:18,950 a path through this that comes closest to the 0, 1 294 00:15:18,950 --> 00:15:21,890 sequence that approximates the voltage samples 295 00:15:21,890 --> 00:15:23,655 that you've actually got. 296 00:15:23,655 --> 00:15:25,280 So we make a distinction between what's 297 00:15:25,280 --> 00:15:38,450 called hard decision decoding and soft decision decoding. 298 00:15:41,200 --> 00:15:45,770 So in soft decision decoding, which we'll talk about later, 299 00:15:45,770 --> 00:15:47,262 you preserve those voltage samples. 300 00:15:47,262 --> 00:15:48,470 And you don't mess with them. 301 00:15:48,470 --> 00:15:50,660 But in hard decision decoding, at each stage, 302 00:15:50,660 --> 00:15:53,150 you just make a decision, on each sample, 303 00:15:53,150 --> 00:15:56,270 decide to call it a 0 or a 1 and proceed from there. 304 00:15:56,270 --> 00:16:00,320 OK, so which do you think is likely to get you 305 00:16:00,320 --> 00:16:02,540 better performance if you're doing 306 00:16:02,540 --> 00:16:05,288 the optimal thing after that? 307 00:16:05,288 --> 00:16:06,260 AUDIENCE: Soft. 308 00:16:06,260 --> 00:16:06,630 PROFESSOR: The soft? 309 00:16:06,630 --> 00:16:07,190 Yeah. 310 00:16:07,190 --> 00:16:09,260 Because when you make the decision at one stage, 311 00:16:09,260 --> 00:16:11,720 you're throwing away some information. 312 00:16:11,720 --> 00:16:14,505 You're not taking account of how these samples might 313 00:16:14,505 --> 00:16:15,380 relate to each other. 314 00:16:15,380 --> 00:16:18,648 You're treating that sample in isolation. 315 00:16:18,648 --> 00:16:20,690 If you know that what you're going to end up with 316 00:16:20,690 --> 00:16:24,170 is a code word that corresponds to a path through here, 317 00:16:24,170 --> 00:16:25,730 then there is additional information 318 00:16:25,730 --> 00:16:27,813 that actually couples the different numbers you're 319 00:16:27,813 --> 00:16:29,120 getting across there. 320 00:16:29,120 --> 00:16:31,430 And so you have a hope of doing better 321 00:16:31,430 --> 00:16:33,100 with soft decision decoding. 322 00:16:33,100 --> 00:16:36,480 So postpone the decision until later. 323 00:16:36,480 --> 00:16:37,350 But you pay a cost. 324 00:16:37,350 --> 00:16:39,350 Or you could pay a cost for that, because you've 325 00:16:39,350 --> 00:16:41,683 got to deal, for instance, with the real numbers and all 326 00:16:41,683 --> 00:16:43,230 of that. 327 00:16:43,230 --> 00:16:47,400 So hard decision decoding can simplify your processing. 328 00:16:47,400 --> 00:16:49,800 So what you'll say is, I'll just make a choice here. 329 00:16:49,800 --> 00:16:53,810 So I'll call this 0, 0; 0, 1; 0, 1, and so on, 330 00:16:53,810 --> 00:16:58,820 and then look for a path through the Trellis along which 331 00:16:58,820 --> 00:17:01,340 the emitted party bits come closest 332 00:17:01,340 --> 00:17:07,280 to what I've approximated that sequence of samples by. 333 00:17:07,280 --> 00:17:10,280 So we're talking about having distance again. 334 00:17:10,280 --> 00:17:13,609 And minimum timing distance is going to give us the most 335 00:17:13,609 --> 00:17:16,910 likely path, given that you've already committed 336 00:17:16,910 --> 00:17:22,020 to interpreting the received samples as 0's or 1's. 337 00:17:22,020 --> 00:17:23,750 So what you might imagine is, OK, you've 338 00:17:23,750 --> 00:17:26,960 got this received sequence. 339 00:17:26,960 --> 00:17:31,067 You've got a tabulation of all the possible paths 340 00:17:31,067 --> 00:17:32,900 through the trellis and the parity bits that 341 00:17:32,900 --> 00:17:35,300 are emitted along those paths. 342 00:17:35,300 --> 00:17:37,940 Each path corresponds to a different message. 343 00:17:37,940 --> 00:17:40,040 What you actually have here-- let's see. 344 00:17:40,040 --> 00:17:43,640 We've got 12 bits here, because in addition to the message, 345 00:17:43,640 --> 00:17:48,080 I'm appending a 0, 0 to each one, which forces the trellis 346 00:17:48,080 --> 00:17:49,430 back down to the 0 state. 347 00:17:49,430 --> 00:17:52,310 So what I'm actually doing here is, 348 00:17:52,310 --> 00:17:55,517 I actually have a message that's this followed by two 0's. 349 00:17:55,517 --> 00:17:57,350 And so if you're trying to connect these two 350 00:17:57,350 --> 00:18:00,050 columns with the trellis that I had on the previous page, 351 00:18:00,050 --> 00:18:01,880 that's how you should think about it. 352 00:18:01,880 --> 00:18:04,850 But with any particular message, you navigate up and down 353 00:18:04,850 --> 00:18:06,380 on the trellis. 354 00:18:06,380 --> 00:18:09,710 This particular one, you navigate up, up, down, up. 355 00:18:09,710 --> 00:18:12,380 And that's the sequence that's generated. 356 00:18:12,380 --> 00:18:14,000 That's the code word that you would 357 00:18:14,000 --> 00:18:16,340 expect if this was the message. 358 00:18:16,340 --> 00:18:19,230 What you'll do is you'll search over all possibilities. 359 00:18:19,230 --> 00:18:21,410 At least that's one way to do this, in principle, 360 00:18:21,410 --> 00:18:23,990 search over all possibilities for the code word 361 00:18:23,990 --> 00:18:26,270 here that's closest to what you received. 362 00:18:28,800 --> 00:18:30,855 The trouble is, that's a lot of code words. 363 00:18:30,855 --> 00:18:31,980 That's a lot of code words. 364 00:18:31,980 --> 00:18:34,350 So this can quickly get out of hand. 365 00:18:34,350 --> 00:18:36,180 If you've got long sequences, which 366 00:18:36,180 --> 00:18:39,480 is exactly where you want to do convolutional coding, 367 00:18:39,480 --> 00:18:43,568 you've got a very long table. 368 00:18:43,568 --> 00:18:45,360 So you really want to find an efficient way 369 00:18:45,360 --> 00:18:46,650 to do this matching. 370 00:18:49,377 --> 00:18:50,960 I just wrote down the Hamming distance 371 00:18:50,960 --> 00:18:58,160 that happens to hold for what is the message that was actually 372 00:18:58,160 --> 00:19:02,240 sent, which also will be the message that you will recover 373 00:19:02,240 --> 00:19:04,160 at the receiver if you do the optimal thing 374 00:19:04,160 --> 00:19:06,360 and you don't get fooled by the errors. 375 00:19:06,360 --> 00:19:09,020 So what I've got here as a 2 is the Hamming distance 376 00:19:09,020 --> 00:19:14,540 between the code word here and the received message. 377 00:19:23,670 --> 00:19:26,630 So if that 2 was the smallest one in that whole stack-- 378 00:19:26,630 --> 00:19:29,343 I haven't filled them all out-- 379 00:19:29,343 --> 00:19:31,010 then that's the one that you would call. 380 00:19:34,080 --> 00:19:39,050 OK, so a much cleverer way of doing 381 00:19:39,050 --> 00:19:41,000 this was invented by the Viterbi, 382 00:19:41,000 --> 00:19:44,090 who did his bachelor's degree here, 383 00:19:44,090 --> 00:19:45,320 then moved to the West Coast. 384 00:19:45,320 --> 00:19:49,530 He was very involved in the JPL program. 385 00:19:49,530 --> 00:19:52,070 But he was also a founder of, well, 386 00:19:52,070 --> 00:19:55,443 a succession of companies, but most recently, Qualcomm. 387 00:19:55,443 --> 00:19:57,110 And he's a big friend of the department. 388 00:19:57,110 --> 00:19:58,700 He's on our visiting committee. 389 00:19:58,700 --> 00:20:02,270 Or he has served time with the visiting committee. 390 00:20:02,270 --> 00:20:06,990 So this is an algorithm that he developed in the early days. 391 00:20:06,990 --> 00:20:08,440 And we're going to talk about it. 392 00:20:12,500 --> 00:20:14,330 I think I'll put it all up on the slide. 393 00:20:14,330 --> 00:20:16,220 And then let's talk. 394 00:20:16,220 --> 00:20:18,510 All right, there is a lot there. 395 00:20:18,510 --> 00:20:20,690 I don't want you to struggle through that. 396 00:20:20,690 --> 00:20:24,092 Let's talk about it here. 397 00:20:24,092 --> 00:20:25,550 And when we're done, I think what's 398 00:20:25,550 --> 00:20:26,592 up there will make sense. 399 00:20:26,592 --> 00:20:32,960 That's for you to refer to from the slides later. 400 00:20:32,960 --> 00:20:34,933 And it's my little checklist to know 401 00:20:34,933 --> 00:20:36,350 that I've spoken about everything, 402 00:20:36,350 --> 00:20:41,060 but don't try and navigate that just yet. 403 00:20:41,060 --> 00:20:43,730 So here is what Viterbi says. 404 00:20:43,730 --> 00:20:50,010 He says, we're starting off from some initial state. 405 00:20:50,010 --> 00:20:52,062 This is the zero state. 406 00:20:52,062 --> 00:20:56,062 At an intermediate state, intermediate time-- 407 00:20:56,062 --> 00:20:57,270 sorry, I shouldn't say state. 408 00:20:57,270 --> 00:20:59,640 I meant stage or time. 409 00:20:59,640 --> 00:21:03,390 At an intermediate stage, we have these four possibilities. 410 00:21:07,840 --> 00:21:10,495 What I'm going to do for a given received sequence-- 411 00:21:10,495 --> 00:21:12,370 and let me actually put the received sequence 412 00:21:12,370 --> 00:21:14,495 I'm going to use in this example. 413 00:21:14,495 --> 00:21:15,745 We've got a received sequence. 414 00:21:25,620 --> 00:21:29,790 Let's say it's 0, 0 on the first stage, and then 415 00:21:29,790 --> 00:21:37,650 0, 1 on the second stage, and 0, 1; 1, 0-- 416 00:21:37,650 --> 00:21:39,060 AUDIENCE: [INAUDIBLE] 417 00:21:39,060 --> 00:21:41,010 PROFESSOR: Yeah? 418 00:21:41,010 --> 00:21:44,670 Did someone say something? 419 00:21:44,670 --> 00:21:45,328 No? 420 00:21:45,328 --> 00:21:46,120 I thought I heard-- 421 00:21:46,120 --> 00:21:48,000 OK. 422 00:21:48,000 --> 00:21:50,790 We'll park the question for now and check in again later. 423 00:21:53,810 --> 00:21:57,690 OK, here is the received sequence. 424 00:21:57,690 --> 00:22:01,770 What we're trying to do is find a path through the trellis 425 00:22:01,770 --> 00:22:03,420 where the emitted bits come closest 426 00:22:03,420 --> 00:22:04,650 to this in Hamming distance. 427 00:22:09,240 --> 00:22:11,400 Here is what Viterbi proposes to do. 428 00:22:11,400 --> 00:22:14,070 He says, from the starting state, 429 00:22:14,070 --> 00:22:16,920 let's find the optimum path to each 430 00:22:16,920 --> 00:22:19,990 of these states at any particular time, 431 00:22:19,990 --> 00:22:21,810 let's say a time i here. 432 00:22:21,810 --> 00:22:25,050 Here is time i. 433 00:22:25,050 --> 00:22:28,200 Let's find the optimum path to these 434 00:22:28,200 --> 00:22:29,830 with the associated minimum cost. 435 00:22:29,830 --> 00:22:32,080 So let's assume that I have that. 436 00:22:32,080 --> 00:22:34,270 So what I'm going to do is, for each of these, 437 00:22:34,270 --> 00:22:36,090 I'm going to put in some number. 438 00:22:36,090 --> 00:22:37,560 This won't be exactly the notation 439 00:22:37,560 --> 00:22:42,698 that we have on this slide, but it's streamlined. 440 00:22:45,910 --> 00:22:50,005 These p's correspond to what we call path metrics. 441 00:22:58,280 --> 00:23:00,430 And I should actually have an index i here 442 00:23:00,430 --> 00:23:02,740 to tell you that I'm doing this at time i, 443 00:23:02,740 --> 00:23:05,190 but I'll just leave that off. 444 00:23:05,190 --> 00:23:19,800 p sub 1 is cost along optimal path to state one. 445 00:23:23,220 --> 00:23:25,170 OK, so assume that, magically, you've 446 00:23:25,170 --> 00:23:28,230 computed the optimal path to this 447 00:23:28,230 --> 00:23:29,620 starting from the initial time. 448 00:23:29,620 --> 00:23:31,710 So what that means is maybe you've gone down 449 00:23:31,710 --> 00:23:33,120 to a particular stage here. 450 00:23:33,120 --> 00:23:34,512 You've gone down further. 451 00:23:34,512 --> 00:23:35,970 And then maybe you've come up here. 452 00:23:35,970 --> 00:23:38,693 Maybe that's the optimum path. 453 00:23:38,693 --> 00:23:40,110 So what you're going to keep track 454 00:23:40,110 --> 00:23:44,340 of is, for each of these times, for each 455 00:23:44,340 --> 00:23:49,280 of these states, what's the cost, the optimal cost? 456 00:23:49,280 --> 00:23:52,530 Or what's the cost along an optimal path there? 457 00:23:52,530 --> 00:23:53,850 OK, now what do I mean by cost? 458 00:23:53,850 --> 00:23:56,700 I just mean Hamming distance between what 459 00:23:56,700 --> 00:23:59,568 I received and the parity bits emitted along 460 00:23:59,568 --> 00:24:00,610 the way up to that point. 461 00:24:05,370 --> 00:24:10,120 So Viterbi is going to keep track of this for every stage 462 00:24:10,120 --> 00:24:12,380 as you step along and for every one of these states. 463 00:24:16,250 --> 00:24:20,170 Now, let's take this particular one. 464 00:24:20,170 --> 00:24:25,420 If I'm transitioning to state one here, let's see. 465 00:24:25,420 --> 00:24:29,300 This emits a 0, 0 if I go from-- 466 00:24:29,300 --> 00:24:31,525 let's go back to our trellis. 467 00:24:35,557 --> 00:24:36,890 I should draw this up, actually. 468 00:24:36,890 --> 00:24:44,490 But if I go along the top, I am at 0, 0. 469 00:24:44,490 --> 00:24:46,720 What other state comes into the top one? 470 00:24:50,000 --> 00:24:52,628 This comes in. 471 00:24:52,628 --> 00:24:54,120 And this emits 1, 1. 472 00:24:58,820 --> 00:25:01,540 So what's the cost I incurred if I take the upper path? 473 00:25:04,360 --> 00:25:08,230 The cost is just the Hamming distance between the 1, 0 474 00:25:08,230 --> 00:25:10,340 that I received and the 0, 0 that I have here. 475 00:25:10,340 --> 00:25:11,890 So there is a cost of 1. 476 00:25:11,890 --> 00:25:13,840 Let me, again, use colors for costs here. 477 00:25:17,260 --> 00:25:19,300 What's the cost I incur if I instead 478 00:25:19,300 --> 00:25:24,420 come to this point from p2, again cost of 1? 479 00:25:31,690 --> 00:25:33,070 So this is the generic picture. 480 00:25:33,070 --> 00:25:35,070 What you're going to do is, you're having this-- 481 00:25:35,070 --> 00:25:36,580 you have this at any stage. 482 00:25:36,580 --> 00:25:39,830 You compute the branch costs and continues. 483 00:25:39,830 --> 00:25:52,870 And now suppose p1 was equal to 3 and p2 was equal to 4, 484 00:25:52,870 --> 00:25:55,870 and you wanted to figure out what's the shortest way, what's 485 00:25:55,870 --> 00:25:59,500 the minimum-cost way to get from the origin to this point, 486 00:25:59,500 --> 00:26:01,150 to p1 at the next time instant? 487 00:26:04,510 --> 00:26:05,740 What's the minimum cost? 488 00:26:05,740 --> 00:26:07,494 And what's the root? 489 00:26:07,494 --> 00:26:11,470 AUDIENCE: [INAUDIBLE] 490 00:26:11,470 --> 00:26:13,630 PROFESSOR: If you came from here, 491 00:26:13,630 --> 00:26:17,080 you've incurred a cost of 3 up to this point. 492 00:26:17,080 --> 00:26:19,270 And you're adding an additional cost of 1. 493 00:26:19,270 --> 00:26:22,060 You'll end up with a cost of 4 to get to here. 494 00:26:22,060 --> 00:26:25,360 If you get to hear from p2, well, you've 495 00:26:25,360 --> 00:26:29,030 incurred a cost of, let's say, 4 up to this point. 496 00:26:29,030 --> 00:26:31,180 And now you're going to incur an additional cost 497 00:26:31,180 --> 00:26:32,350 to bring it to 5. 498 00:26:32,350 --> 00:26:36,040 So your best route to p1 at this time 499 00:26:36,040 --> 00:26:38,680 is to come from p1 at this time using this arc. 500 00:26:41,240 --> 00:26:43,610 So if you've built it up at a particular stage, 501 00:26:43,610 --> 00:26:45,665 then it's actually very straightforward 502 00:26:45,665 --> 00:26:47,790 to figure out what you should do at the next stage. 503 00:26:47,790 --> 00:26:52,151 So let me now start putting some time indices on this. 504 00:26:52,151 --> 00:27:01,631 This would be p1 at time i is equal to 3. 505 00:27:01,631 --> 00:27:08,610 p1 at time i plus 1 is equal to 4. 506 00:27:08,610 --> 00:27:10,020 This is p2 at time i. 507 00:27:20,020 --> 00:27:22,030 So you can actually forget about this arrow, 508 00:27:22,030 --> 00:27:24,322 because there is no way you're going to use that arrow. 509 00:27:24,322 --> 00:27:26,440 Whenever you come to this stage at this time, 510 00:27:26,440 --> 00:27:30,558 you're going to come via the upper branch. 511 00:27:30,558 --> 00:27:32,350 So at every stage, you're going to do this. 512 00:27:32,350 --> 00:27:34,503 And it's a very simple calculation. 513 00:27:34,503 --> 00:27:36,670 So now we've got slightly more elaborate notation up 514 00:27:36,670 --> 00:27:38,795 on the board, but I hope you have the general idea. 515 00:27:43,380 --> 00:27:45,890 This is an instance, by the way, and a way 516 00:27:45,890 --> 00:27:49,310 of thinking about such problems that's referred 517 00:27:49,310 --> 00:27:51,168 to as dynamic programming. 518 00:27:59,780 --> 00:28:02,000 It works for these sorts of rooting problems. 519 00:28:02,000 --> 00:28:04,850 We're rooting ourselves along a trellis where 520 00:28:04,850 --> 00:28:07,790 the total cost of taking a path is the sum 521 00:28:07,790 --> 00:28:10,010 of the costs at every stage. 522 00:28:10,010 --> 00:28:13,250 So the total Hamming distance between the bits 523 00:28:13,250 --> 00:28:16,310 you emit along the way and the bits that you've received 524 00:28:16,310 --> 00:28:19,550 is made up of the Hamming distance between the branch 525 00:28:19,550 --> 00:28:21,590 here and the piece you've received here 526 00:28:21,590 --> 00:28:24,920 plus the Hamming distance on the branch here plus the piece you 527 00:28:24,920 --> 00:28:27,110 between the branch here and the branch-- 528 00:28:27,110 --> 00:28:29,850 sorry, the received segment over there, and so on. 529 00:28:29,850 --> 00:28:31,520 So the total Hamming distance is made up 530 00:28:31,520 --> 00:28:34,190 of the sum of the Hamming distances along the way. 531 00:28:34,190 --> 00:28:38,510 In all such situations where you've got a total cost that's 532 00:28:38,510 --> 00:28:41,210 additive over the path and you've 533 00:28:41,210 --> 00:28:43,310 got to do an optimization, dynamic programming 534 00:28:43,310 --> 00:28:45,840 is something you can think of. 535 00:28:45,840 --> 00:28:49,100 And the idea we've used here is actually one 536 00:28:49,100 --> 00:28:51,770 that you might come at naturally. 537 00:28:51,770 --> 00:28:55,190 If you found the best way from here to the Student Center, 538 00:28:55,190 --> 00:28:59,240 and it happens to go through Lobby 7, 539 00:28:59,240 --> 00:29:02,443 what's your best way from here to Lobby 7? 540 00:29:02,443 --> 00:29:04,610 Presumably, it's going to be the section of the path 541 00:29:04,610 --> 00:29:06,830 that you would take to the Student Center that 542 00:29:06,830 --> 00:29:09,590 passes through Lobby 7, because if you had a better way 543 00:29:09,590 --> 00:29:10,970 to get to Lobby 7, you would have 544 00:29:10,970 --> 00:29:14,920 used it to get to the Student Center via Lobby 7. 545 00:29:14,920 --> 00:29:16,490 It's just that idea. 546 00:29:16,490 --> 00:29:21,140 So on an optimum path where the costs are additive, 547 00:29:21,140 --> 00:29:24,710 it must be the case that the optimum path 548 00:29:24,710 --> 00:29:26,957 to an intermediate point is exactly 549 00:29:26,957 --> 00:29:28,790 the section of the optimum path to the point 550 00:29:28,790 --> 00:29:34,560 that you're looking at, a simple idea. 551 00:29:34,560 --> 00:29:38,578 OK, so let's go back to the more formal way 552 00:29:38,578 --> 00:29:39,995 it's written up here on the slide. 553 00:29:46,150 --> 00:29:48,100 So we talk about the branch metric. 554 00:29:48,100 --> 00:29:51,670 That's just the Hamming distance that we computed here 555 00:29:51,670 --> 00:29:52,840 for the branch. 556 00:29:52,840 --> 00:29:56,860 It's the difference between what we received 557 00:29:56,860 --> 00:30:02,740 and what would be transmitted if you moved along that arc. 558 00:30:02,740 --> 00:30:04,030 So that's the branch metric. 559 00:30:04,030 --> 00:30:06,850 It's the piece contributed by the branch. 560 00:30:06,850 --> 00:30:09,100 This is the notation we've used. 561 00:30:09,100 --> 00:30:12,550 We've already talked about this, that you could either do a hard 562 00:30:12,550 --> 00:30:15,280 decision kind of rule where you've already set these 563 00:30:15,280 --> 00:30:16,090 to 1's and 0's. 564 00:30:16,090 --> 00:30:19,150 Or you could stick with the original samples. 565 00:30:19,150 --> 00:30:21,520 If you've already converted them to 1's and 0's, there 566 00:30:21,520 --> 00:30:25,380 is a natural notion of distance, which is the Hamming distance. 567 00:30:25,380 --> 00:30:26,860 And there is a probabilistic reason 568 00:30:26,860 --> 00:30:28,670 why you would want to do that. 569 00:30:28,670 --> 00:30:30,550 So we're sticking to the Hamming distance 570 00:30:30,550 --> 00:30:34,660 setting right now, so hard decision decoding. 571 00:30:34,660 --> 00:30:38,080 And the path metric, this is a more elaborate notation 572 00:30:38,080 --> 00:30:39,770 than what I have here. 573 00:30:39,770 --> 00:30:43,390 So instead of a subscript to denote the state, 574 00:30:43,390 --> 00:30:47,690 this has got the state index here, and the time index here, 575 00:30:47,690 --> 00:30:50,980 and pm for path metric instead of just p, 576 00:30:50,980 --> 00:30:52,550 but it's the same thing. 577 00:30:52,550 --> 00:30:55,220 So for each state and at each stage, 578 00:30:55,220 --> 00:31:01,150 so for each of the four states and for each of the stages, 579 00:31:01,150 --> 00:31:03,230 you're going to compute this. 580 00:31:03,230 --> 00:31:06,300 And you can the path metric up to time 581 00:31:06,300 --> 00:31:08,110 i is the smallest sum of the branch 582 00:31:08,110 --> 00:31:10,270 metrics over all the sequences that 583 00:31:10,270 --> 00:31:12,220 will get you to that place. 584 00:31:12,220 --> 00:31:14,600 And if you assume you have that at any stage, 585 00:31:14,600 --> 00:31:17,440 then the computation that takes you to the next stage 586 00:31:17,440 --> 00:31:18,130 is an easy one. 587 00:31:22,920 --> 00:31:24,830 I think I've said all this. 588 00:31:24,830 --> 00:31:26,220 You can come back to it later. 589 00:31:31,730 --> 00:31:33,690 So let's actually just step through this. 590 00:31:36,790 --> 00:31:39,005 So we're at some intermediate stage. 591 00:31:39,005 --> 00:31:41,130 We're just doing the same thing I had on the board. 592 00:31:41,130 --> 00:31:43,140 I'm doing it again in pictures here 593 00:31:43,140 --> 00:31:45,750 so you get to think about it one more time. 594 00:31:45,750 --> 00:31:50,250 Suppose we've received 0, 0. 595 00:31:50,250 --> 00:31:54,660 We first label each of the arcs here by the Hamming distance 596 00:31:54,660 --> 00:31:58,080 between the bits we'd emit along the arc and the bits 597 00:31:58,080 --> 00:32:01,260 we've actually received, so Hamming distant 0 here 598 00:32:01,260 --> 00:32:04,390 between what we would emit and what we received, 599 00:32:04,390 --> 00:32:07,800 Hamming distance 2 here on this arc, Hamming distance 600 00:32:07,800 --> 00:32:09,520 2 on this arc, and so on. 601 00:32:09,520 --> 00:32:13,170 So the red numbers here are below the top two 602 00:32:13,170 --> 00:32:14,730 just are the costs on the arcs. 603 00:32:25,170 --> 00:32:27,400 Actually, I don't like the last line of that slide. 604 00:32:27,400 --> 00:32:29,070 So you may want to strike that. 605 00:32:29,070 --> 00:32:31,950 We're not going to really be talking about the most 606 00:32:31,950 --> 00:32:33,000 likely branch metric. 607 00:32:33,000 --> 00:32:34,875 We're only going to make decisions once we're 608 00:32:34,875 --> 00:32:36,220 done with the whole path. 609 00:32:41,290 --> 00:32:45,250 So we assume at some stage, that we have the path metrics 610 00:32:45,250 --> 00:32:48,220 up to that point. 611 00:32:48,220 --> 00:32:50,860 And then we do the computation that I just talked about. 612 00:33:05,538 --> 00:33:06,080 So let's see. 613 00:33:06,080 --> 00:33:10,157 In this particular case, what would be the path metric value 614 00:33:10,157 --> 00:33:10,865 in this position? 615 00:33:15,365 --> 00:33:16,880 It's the same thing we did already, 616 00:33:16,880 --> 00:33:19,470 but just another chance to look at it. 617 00:33:19,470 --> 00:33:23,840 What would be the value of the path metric there? 618 00:33:23,840 --> 00:33:26,130 3? 619 00:33:26,130 --> 00:33:29,400 Because you can either do 3 plus 1 on that arc. 620 00:33:29,400 --> 00:33:33,330 Or you can do 2 plus 1 on this-- sorry, 3 plus 1 621 00:33:33,330 --> 00:33:36,180 on this arc for a cost of 4 or 2 plus 1 622 00:33:36,180 --> 00:33:38,640 on this arc for a cost of 3. 623 00:33:38,640 --> 00:33:39,870 So it should be a 3 there. 624 00:33:39,870 --> 00:33:41,537 And this is the arc that you would pick. 625 00:33:45,790 --> 00:33:49,780 And similarly, you can do it for all of them. 626 00:33:49,780 --> 00:33:52,990 So once you have one stage, you can fill out the next stage 627 00:33:52,990 --> 00:33:56,055 completely and then keep track of the arcs that 628 00:33:56,055 --> 00:33:56,680 lead you there. 629 00:34:03,833 --> 00:34:06,000 And at some point, you'll-- at each stage, actually, 630 00:34:06,000 --> 00:34:08,949 you can prune away things that you're not going to be using. 631 00:34:08,949 --> 00:34:11,070 So you're never going to use that edge. 632 00:34:11,070 --> 00:34:13,080 So you don't have to worry about it anymore. 633 00:34:13,080 --> 00:34:15,060 You're never going to use this edge. 634 00:34:15,060 --> 00:34:16,889 There are also stages where you might 635 00:34:16,889 --> 00:34:19,139 have two different ways of getting to a box 636 00:34:19,139 --> 00:34:20,656 and incurring the same cost. 637 00:34:20,656 --> 00:34:22,739 And then it doesn't matter which of them you pick. 638 00:34:22,739 --> 00:34:24,120 You can pick one or the other. 639 00:34:24,120 --> 00:34:26,850 In terms of the overall cost, it's not going to matter. 640 00:34:30,674 --> 00:34:34,546 AUDIENCE: [INAUDIBLE] 641 00:34:38,487 --> 00:34:40,070 PROFESSOR: Yeah, so what you're saying 642 00:34:40,070 --> 00:34:43,510 is that, if there isn't a unique way back, 643 00:34:43,510 --> 00:34:44,510 then you're not certain. 644 00:34:44,510 --> 00:34:46,552 Well, you're never certain here in this business. 645 00:34:46,552 --> 00:34:48,060 You're just doing your best guess. 646 00:34:48,060 --> 00:34:50,929 So what you would do when you commit to one particular choice 647 00:34:50,929 --> 00:34:53,485 when there are two equally likely costs is you're saying, 648 00:34:53,485 --> 00:34:54,860 the probability of error is going 649 00:34:54,860 --> 00:34:56,402 to be the same with this choice as it 650 00:34:56,402 --> 00:34:57,830 will be with the other choice. 651 00:34:57,830 --> 00:35:01,010 And in the end, that's what we have for the metric. 652 00:35:01,010 --> 00:35:03,073 It is unsatisfying, perhaps. 653 00:35:03,073 --> 00:35:04,490 Now, there are schemes where you'd 654 00:35:04,490 --> 00:35:06,865 keep a list of the possibilities and try and do something 655 00:35:06,865 --> 00:35:09,790 with that, because maybe there is some higher level thing that 656 00:35:09,790 --> 00:35:13,370 would help you disambiguate between possibilities, 657 00:35:13,370 --> 00:35:16,160 but that would complicate the processing. 658 00:35:16,160 --> 00:35:19,003 But as far as this goes, you make a choice. 659 00:35:19,003 --> 00:35:19,670 And you move on. 660 00:35:26,840 --> 00:35:28,430 So you can imagine actually working 661 00:35:28,430 --> 00:35:30,380 through this whole thing. 662 00:35:36,027 --> 00:35:38,110 If you knew you were starting from the zero state, 663 00:35:38,110 --> 00:35:40,260 you'd start off with a zero cost there. 664 00:35:40,260 --> 00:35:42,210 So you're at infinity here, which 665 00:35:42,210 --> 00:35:46,290 is going to force all the optimal paths to come from 0. 666 00:35:46,290 --> 00:35:49,360 And then you'll continue from there. 667 00:35:49,360 --> 00:35:57,840 So I just wanted to show you a few things that come with this. 668 00:36:04,150 --> 00:36:13,280 Actually, I might have shown you everything I want on that. 669 00:36:13,280 --> 00:36:17,460 So let's just go back to the soft decision decoding. 670 00:36:17,460 --> 00:36:20,730 So how might things differ if you go back 671 00:36:20,730 --> 00:36:22,260 to soft decision decoding? 672 00:36:22,260 --> 00:36:23,460 So let me find that slide. 673 00:36:33,140 --> 00:36:35,800 The Viterbi algorithm doesn't care 674 00:36:35,800 --> 00:36:37,630 how you come at these costs. 675 00:36:37,630 --> 00:36:39,913 The Viterbi algorithm is just dynamic programming 676 00:36:39,913 --> 00:36:40,580 on this trellis. 677 00:36:40,580 --> 00:36:42,550 It finds you the minimum-cost path. 678 00:36:42,550 --> 00:36:44,320 It's up to you how you decide what 679 00:36:44,320 --> 00:36:47,170 cost to attribute to an edge. 680 00:36:47,170 --> 00:36:49,600 So the question is, are there other costs that you 681 00:36:49,600 --> 00:36:51,100 might have come up with? 682 00:36:51,100 --> 00:36:53,650 Well, if the received sequence has been translated already 683 00:36:53,650 --> 00:36:58,270 to 0's and 1's, then Hamming distance is the natural thing. 684 00:36:58,270 --> 00:37:01,240 But if you're keeping particular numbers there, 685 00:37:01,240 --> 00:37:04,820 then it turns out that you might want to do things differently. 686 00:37:04,820 --> 00:37:15,320 So suppose at a particular stage, what you got was not-- 687 00:37:15,320 --> 00:37:15,920 let's see. 688 00:37:15,920 --> 00:37:17,680 Did I put numbers up there? 689 00:37:17,680 --> 00:37:27,360 Suppose it wasn't 0 and 1, but it was some particular numbers, 690 00:37:27,360 --> 00:37:33,870 let's say, 0.3 and 0.7. 691 00:37:33,870 --> 00:37:38,190 And you had actually translated them to a 0 and a 1 692 00:37:38,190 --> 00:37:41,700 in your hard decision decoding. 693 00:37:41,700 --> 00:37:43,530 If you decide not to do that, but to stick 694 00:37:43,530 --> 00:37:44,970 with these numbers, then what you 695 00:37:44,970 --> 00:37:48,420 have is the task for any particular edge 696 00:37:48,420 --> 00:37:52,140 that you're traveling on of finding 697 00:37:52,140 --> 00:37:54,300 the distance between the parity bits 698 00:37:54,300 --> 00:37:58,800 you would emit on that edge and the samples that you have here. 699 00:37:58,800 --> 00:38:02,910 It turns out that a very widely used cost for soft decision 700 00:38:02,910 --> 00:38:06,250 decoding is the sum of squared differences. 701 00:38:06,250 --> 00:38:09,900 So what you would have is 1 minus 0.3 702 00:38:09,900 --> 00:38:15,390 squared plus 1 minus 0.7 squared. 703 00:38:18,540 --> 00:38:21,060 So it would be the first bit that you 704 00:38:21,060 --> 00:38:23,610 emit on this arc minus the first sample 705 00:38:23,610 --> 00:38:29,423 that you got squared error plus the second sample, 706 00:38:29,423 --> 00:38:30,840 the second bit that you would emit 707 00:38:30,840 --> 00:38:34,590 on this arc minus the second sample, whole thing squared. 708 00:38:34,590 --> 00:38:39,450 If there was another arc that was a 1 and 0 arc, 709 00:38:39,450 --> 00:38:43,500 then what you would computers 1 minus 0.3 710 00:38:43,500 --> 00:38:51,630 squared plus 0 minus 0.7 squared. 711 00:38:51,630 --> 00:38:54,300 So it's just a different way of coming up with the cost. 712 00:38:54,300 --> 00:38:56,760 The rest of the Viterbi algorithm is exactly the same. 713 00:38:56,760 --> 00:39:00,880 the navigating through the trellis is exactly the same. 714 00:39:00,880 --> 00:39:05,610 It turns out that there is a logic and a reason 715 00:39:05,610 --> 00:39:11,670 behind this particular metric for situations where 716 00:39:11,670 --> 00:39:16,850 your voltage samples are distributed 717 00:39:16,850 --> 00:39:22,930 in the familiar bell-shaped fashion here, 718 00:39:22,930 --> 00:39:24,542 what's called a Gaussian distribution. 719 00:39:24,542 --> 00:39:26,000 We'll talk about it more next time. 720 00:39:33,710 --> 00:39:38,830 So what we're saying is that if you send a 1, 721 00:39:38,830 --> 00:39:41,320 you get a spread of possible values. 722 00:39:41,320 --> 00:39:43,360 The probability of your values falling 723 00:39:43,360 --> 00:39:47,230 in some particular range here can be computed by the area 724 00:39:47,230 --> 00:39:48,920 under this particular curve. 725 00:39:48,920 --> 00:39:53,000 It's got an analytical expression. 726 00:39:53,000 --> 00:39:54,400 So this is the most likely spot. 727 00:39:54,400 --> 00:39:55,900 But there is certainly probabilities 728 00:39:55,900 --> 00:39:58,180 of falling in any particular interval here. 729 00:39:58,180 --> 00:40:01,580 Well, what does the Gaussian distribution look like? 730 00:40:01,580 --> 00:40:03,960 We'll talk more about it. 731 00:40:03,960 --> 00:40:06,920 the essential part of it is e to the minus-- 732 00:40:06,920 --> 00:40:07,420 let's see. 733 00:40:07,420 --> 00:40:10,060 Let me put some labels here. 734 00:40:10,060 --> 00:40:11,260 This is where it's centered. 735 00:40:11,260 --> 00:40:13,120 Let me call it mu. 736 00:40:13,120 --> 00:40:17,840 And let's say x is the value along the axis. 737 00:40:17,840 --> 00:40:21,580 So we'll have e to the x minus mu 738 00:40:21,580 --> 00:40:27,530 all squared divided by some normalizing parameter. 739 00:40:27,530 --> 00:40:30,610 Well actually, let's just call it 740 00:40:30,610 --> 00:40:36,780 capital N. Think of capital N as a noise variance. 741 00:40:36,780 --> 00:40:38,820 Actually, let me just call it N sub 0 742 00:40:38,820 --> 00:40:42,340 so you don't think it's a counting number. 743 00:40:42,340 --> 00:40:45,060 Think of it as a noise variance. 744 00:40:45,060 --> 00:40:50,000 So the larger that N is, would you 745 00:40:50,000 --> 00:40:52,020 spread out more or less here? 746 00:40:55,165 --> 00:40:57,290 Well, just from the fact that I call it a variance, 747 00:40:57,290 --> 00:40:59,167 maybe you would guess that, if N is larger, 748 00:40:59,167 --> 00:41:00,500 you're going to spread out more. 749 00:41:03,080 --> 00:41:06,710 Well, in this kind of setting, when you take log likelihoods-- 750 00:41:06,710 --> 00:41:09,080 you've seen that computation in the chapters-- 751 00:41:09,080 --> 00:41:11,300 what ends up appearing in your cost criterion 752 00:41:11,300 --> 00:41:13,910 is x minus mu squared. 753 00:41:13,910 --> 00:41:16,010 So it's the squared difference from the mean 754 00:41:16,010 --> 00:41:17,830 that you want to be looking at. 755 00:41:17,830 --> 00:41:20,942 And that's exactly why, in that kind of setting, 756 00:41:20,942 --> 00:41:23,150 this is what you end up choosing as your cost metric. 757 00:41:27,860 --> 00:41:37,013 But once you're done computing those metrics, 758 00:41:37,013 --> 00:41:38,930 the rest of the Viterbi algorithm is the same. 759 00:41:42,220 --> 00:41:47,990 So once you have the convolutional coding in hand, 760 00:41:47,990 --> 00:41:50,750 you know how to decode, you can start 761 00:41:50,750 --> 00:41:54,920 to do some comparisons of how these different codes perform. 762 00:41:54,920 --> 00:41:57,920 There is an extensive discussion in the chapter. 763 00:41:57,920 --> 00:42:01,240 Let me just give you some highlights here. 764 00:42:01,240 --> 00:42:03,080 OK, so what are we plotting here? 765 00:42:05,660 --> 00:42:10,430 What we're saying is we send a whole bunch of message bits 766 00:42:10,430 --> 00:42:11,690 through the channel. 767 00:42:11,690 --> 00:42:14,210 And then we decode at the other end. 768 00:42:14,210 --> 00:42:16,680 And what we're talking about is-- 769 00:42:16,680 --> 00:42:17,180 let's see. 770 00:42:17,180 --> 00:42:18,830 Here is the binary symmetric channel. 771 00:42:18,830 --> 00:42:20,840 Here is the error probability on the channel. 772 00:42:20,840 --> 00:42:24,100 You can see to well what this is, but it's the-- 773 00:42:24,100 --> 00:42:26,500 why is that chopped off? 774 00:42:26,500 --> 00:42:30,850 It's the probability of error overall end to end, 775 00:42:30,850 --> 00:42:32,350 not of the channel, but after you've 776 00:42:32,350 --> 00:42:35,570 done your coding and decoding. 777 00:42:35,570 --> 00:42:36,070 Let's see. 778 00:42:36,070 --> 00:42:37,750 Do we recognize any of these codes? 779 00:42:37,750 --> 00:42:42,220 Here is the uncoded case where basically you're 780 00:42:42,220 --> 00:42:46,750 exposing the stream directly to the error on the binary 781 00:42:46,750 --> 00:42:49,330 symmetric channel. 782 00:42:49,330 --> 00:42:51,280 We expect higher errors when we have 783 00:42:51,280 --> 00:42:55,120 higher probabilities of flipping a bit on the channel. 784 00:42:55,120 --> 00:42:57,520 So this is the uncoded case. 785 00:42:57,520 --> 00:42:59,630 What does the Hamming code do? 786 00:42:59,630 --> 00:43:04,270 That's a Hamming code probability, 0.74. 787 00:43:04,270 --> 00:43:08,080 So the Hamming code performance, you can see here 788 00:43:08,080 --> 00:43:10,150 end to end what it looks like. 789 00:43:10,150 --> 00:43:11,650 What's the rate that goes with that? 790 00:43:15,588 --> 00:43:17,130 What's the rate of that Hamming code? 791 00:43:20,370 --> 00:43:22,380 4 over 7, right? 792 00:43:22,380 --> 00:43:28,370 Because n is the number of bits in the message-- 793 00:43:28,370 --> 00:43:29,370 sorry, in the code word. 794 00:43:29,370 --> 00:43:33,450 And 4 is the number of bits in the message, so 4 over 7, 795 00:43:33,450 --> 00:43:36,240 something over 1/2. 796 00:43:36,240 --> 00:43:36,780 Let's see. 797 00:43:36,780 --> 00:43:40,080 Do we know what that code might be? 798 00:43:42,980 --> 00:43:45,260 Any codes you know about that take 4 message bits 799 00:43:45,260 --> 00:43:49,400 and pad them to 8 code word bits? 800 00:43:52,090 --> 00:43:53,705 You've seen at least one such code. 801 00:43:53,705 --> 00:43:54,580 AUDIENCE: [INAUDIBLE] 802 00:43:54,580 --> 00:43:54,850 PROFESSOR: Sorry? 803 00:43:54,850 --> 00:43:56,200 AUDIENCE: Rectangular parity? 804 00:43:56,200 --> 00:43:58,320 PROFESSOR: Rectangular parity, right? 805 00:43:58,320 --> 00:44:00,070 If you didn't have that corner parity bit, 806 00:44:00,070 --> 00:44:01,960 but you just did the rows and columns, 807 00:44:01,960 --> 00:44:05,720 then you'd arrange 4 bits in a 2 by 2 pattern 808 00:44:05,720 --> 00:44:07,448 and then have 4 parity bits. 809 00:44:07,448 --> 00:44:08,740 So that's a rectangular parity. 810 00:44:08,740 --> 00:44:11,710 That's rate 1/2. 811 00:44:11,710 --> 00:44:14,650 What this denotes is a convolutional code. 812 00:44:14,650 --> 00:44:17,320 It's actually the code we've been-- 813 00:44:17,320 --> 00:44:19,820 sorry, no, this one is the code we've been looking at. 814 00:44:19,820 --> 00:44:23,980 So let me explain to you what that notation means when 815 00:44:23,980 --> 00:44:25,225 you're reading the chapter. 816 00:44:31,530 --> 00:44:34,620 This code is represented as-- 817 00:44:34,620 --> 00:44:38,130 the one we've been talking about as represented as this. 818 00:44:38,130 --> 00:44:42,030 So what this is is the constraint length. 819 00:44:42,030 --> 00:44:44,220 And what this is is just to tell me 820 00:44:44,220 --> 00:44:49,500 that the generator bits I used for my party generation 821 00:44:49,500 --> 00:44:52,830 correspond to the binary representation of 7 822 00:44:52,830 --> 00:44:56,250 and the binary representation of 5. 823 00:44:56,250 --> 00:44:59,190 So remember that for my first parody bit, 824 00:44:59,190 --> 00:45:03,630 I chose xn plus xn minus 1 plus xn minus 2. 825 00:45:03,630 --> 00:45:05,190 I picked all three of them. 826 00:45:05,190 --> 00:45:09,060 For my second parity bit, I took xn plus xn minus 2. 827 00:45:09,060 --> 00:45:10,470 I skipped the middle one. 828 00:45:10,470 --> 00:45:13,830 So the notation that's used to denote a convolutional code 829 00:45:13,830 --> 00:45:20,480 with these two generators is, just for compactness, the 7, 830 00:45:20,480 --> 00:45:22,870 5 there. 831 00:45:22,870 --> 00:45:23,370 Let's see. 832 00:45:23,370 --> 00:45:26,040 Is this redundant, the k, the value of k? 833 00:45:26,040 --> 00:45:30,300 Could you have figured out what the constraint length is? 834 00:45:30,300 --> 00:45:31,120 AUDIENCE: Yeah. 835 00:45:31,120 --> 00:45:31,787 PROFESSOR: Yeah. 836 00:45:31,787 --> 00:45:35,290 It's already staring at you here what the constraints length. 837 00:45:35,290 --> 00:45:36,780 So this is a little redundant. 838 00:45:36,780 --> 00:45:38,490 It's just that it's a convenient way 839 00:45:38,490 --> 00:45:42,150 to distinguish a convolutional code from a Hamming code. 840 00:45:46,333 --> 00:45:48,750 Now, we have to be a little careful comparing these codes, 841 00:45:48,750 --> 00:45:51,000 because the rates are all a little different. 842 00:45:51,000 --> 00:45:53,520 Here the rate is 1/2. 843 00:45:53,520 --> 00:45:55,860 What's the rate for this convolution 844 00:45:55,860 --> 00:46:01,763 for the two convolution codes here, the 3, 7, 6 845 00:46:01,763 --> 00:46:04,040 and the 3, 7, 5? 846 00:46:04,040 --> 00:46:06,040 AUDIENCE: [INAUDIBLE] 847 00:46:07,040 --> 00:46:08,147 PROFESSOR: Sorry, what's-- 848 00:46:08,147 --> 00:46:09,230 AUDIENCE: 1/2 [INAUDIBLE]. 849 00:46:09,230 --> 00:46:10,750 PROFESSOR: 1/2, right? 850 00:46:10,750 --> 00:46:17,360 So the rate is 1 over the number of parity bits 851 00:46:17,360 --> 00:46:18,610 you're generating per message. 852 00:46:23,300 --> 00:46:27,380 One message bit, r parity bits, therefore, a rate of 1 over r. 853 00:46:27,380 --> 00:46:30,140 So these are rate one-off codes, just like the rectangular case. 854 00:46:34,100 --> 00:46:37,070 This is constraint length 4. 855 00:46:37,070 --> 00:46:42,470 And you can actually write out what that would be there. 856 00:46:42,470 --> 00:46:45,350 How big is the trellis for the constraint length 4 case? 857 00:46:47,940 --> 00:46:48,700 How many stages? 858 00:46:53,308 --> 00:46:54,350 This last one down there? 859 00:46:58,260 --> 00:46:58,780 8, right? 860 00:46:58,780 --> 00:47:03,130 Constraint length 4, that means k equals 4, 2 to the k minus 1, 861 00:47:03,130 --> 00:47:04,080 so 2 cubed states. 862 00:47:04,080 --> 00:47:07,020 So there is 8 states that we're talking about. 863 00:47:07,020 --> 00:47:11,340 The Cassini convolution code that I showed you last time 864 00:47:11,340 --> 00:47:14,370 had a constraint length of 15. 865 00:47:14,370 --> 00:47:18,170 So how many states there on the trellis? 866 00:47:18,170 --> 00:47:22,740 2 to the 14, that's a lot of states. 867 00:47:22,740 --> 00:47:25,120 So that's a lot of computation there happening. 868 00:47:25,120 --> 00:47:26,800 And actually, there is no hope of that 869 00:47:26,800 --> 00:47:30,010 having been done if it wasn't for the Viterbi algorithm. 870 00:47:30,010 --> 00:47:33,340 All right, we'll talk more about comparison between these codes 871 00:47:33,340 --> 00:47:35,230 next time. 872 00:47:35,230 --> 00:47:37,600 Sorry, I should do one more thing here. 873 00:47:37,600 --> 00:47:40,015 I did talk last time about this notion of free distance. 874 00:47:43,800 --> 00:47:48,090 Let's just stare at this a second. 875 00:47:48,090 --> 00:47:51,750 We said the free distance was the weight of the smallest 876 00:47:51,750 --> 00:47:54,510 non-zero codeword. 877 00:47:54,510 --> 00:47:56,940 And it gave you a handle on the performance of the code. 878 00:47:56,940 --> 00:48:00,810 It was the Hamming distance for the set of code words 879 00:48:00,810 --> 00:48:04,450 you could generate between 0, 0 here and 0, 0 there. 880 00:48:04,450 --> 00:48:10,940 Can you see by inspection here what might be a candidate 881 00:48:10,940 --> 00:48:13,768 free distance here? 882 00:48:13,768 --> 00:48:15,560 I think what we had last time was 5, right? 883 00:48:15,560 --> 00:48:18,470 This is the 1, 1; 0, 1; 1, 1. 884 00:48:18,470 --> 00:48:23,138 And we'll pick up 1, 2, 3, 4, 5, a weight of 5. 885 00:48:23,138 --> 00:48:25,430 And it turns out there is no other path that's smaller. 886 00:48:25,430 --> 00:48:28,550 So the performance of this particular code 887 00:48:28,550 --> 00:48:31,520 is indicated by that number 5. 888 00:48:31,520 --> 00:48:35,300 It tells you that you can correct two bits. 889 00:48:35,300 --> 00:48:37,400 But actually, it tells you much more 890 00:48:37,400 --> 00:48:42,830 than you would typically try and extract from a typical block 891 00:48:42,830 --> 00:48:45,768 code where you would say, if there is Hamming distance 5, 892 00:48:45,768 --> 00:48:47,060 you only can correct two codes. 893 00:48:47,060 --> 00:48:49,250 Here you've got message bits that 894 00:48:49,250 --> 00:48:52,010 go on for a long time, thousands of bits. 895 00:48:52,010 --> 00:48:56,210 So what this is telling you is that in a duration that's 896 00:48:56,210 --> 00:48:59,900 of the order of five or six message bits, 897 00:48:59,900 --> 00:49:03,080 you can, with this scheme, correct up to two bits. 898 00:49:03,080 --> 00:49:05,870 You can have bursts of errors that are very frequent 899 00:49:05,870 --> 00:49:08,810 and correct them with this Viterbi decoding. 900 00:49:08,810 --> 00:49:11,000 So the free distance is an important notion. 901 00:49:11,000 --> 00:49:13,700 So when you do the examples in recitation tomorrow, 902 00:49:13,700 --> 00:49:17,120 please look out for what the free distance is for your codes 903 00:49:17,120 --> 00:49:19,960 and compare with what we have here.