1 00:00:00,000 --> 00:01:36,980 [WRITING ON CHALKBOARD] 2 00:01:36,980 --> 00:01:38,442 PROFESSOR: Good morning. 3 00:01:38,442 --> 00:01:44,780 I think I'll start even though people are not quite all here, 4 00:01:44,780 --> 00:01:46,030 because it's time. 5 00:01:49,340 --> 00:01:51,990 Sorry, I'd hoped to start Chapter 10 last time, but 6 00:01:51,990 --> 00:01:54,550 we'll start it this time. 7 00:01:54,550 --> 00:01:58,600 When Ashish comes, we'll have handouts for Chapter 10, the 8 00:01:58,600 --> 00:02:00,280 new problem set and the old problem set 9 00:02:00,280 --> 00:02:03,800 solutions as usual. 10 00:02:03,800 --> 00:02:10,630 This chapter is about trellis diagrams for block codes. 11 00:02:10,630 --> 00:02:13,330 Binary linear block codes of the kind that we've seen 12 00:02:13,330 --> 00:02:17,800 before, it's actually a much more general notion that 13 00:02:17,800 --> 00:02:19,990 applies to -- 14 00:02:19,990 --> 00:02:24,360 well, linearity is important, although we don't see it very 15 00:02:24,360 --> 00:02:26,090 specifically in the trellis diagram. 16 00:02:26,090 --> 00:02:29,280 But we'll see that for constructing a unique minimal 17 00:02:29,280 --> 00:02:33,650 trellis, linearity is an important aspect. 18 00:02:33,650 --> 00:02:37,580 Or more generally, it's just the group property of codes 19 00:02:37,580 --> 00:02:40,470 that allows us to construct unique 20 00:02:40,470 --> 00:02:43,170 minimal trellis diagrams. 21 00:02:43,170 --> 00:02:45,930 And I think it's best to motivate this 22 00:02:45,930 --> 00:02:48,820 chapter by an example. 23 00:02:48,820 --> 00:02:52,580 Here's a nice example of a code that we know very well. 24 00:02:52,580 --> 00:02:54,620 It has a nice trellis diagram. 25 00:02:54,620 --> 00:02:58,075 It's the 8, 4, 4 Reed-Muller code. 26 00:03:00,720 --> 00:03:03,890 Just to remind you of how we construct this code, or one of 27 00:03:03,890 --> 00:03:04,450 the methods. 28 00:03:04,450 --> 00:03:06,160 We have many methods. 29 00:03:06,160 --> 00:03:09,920 We take this kind of universal eight by eight -- 30 00:03:09,920 --> 00:03:14,010 well, it's a list of eight tuples. 31 00:03:14,010 --> 00:03:17,420 And it's the tensor product of this little 32 00:03:17,420 --> 00:03:19,710 matrix three times. 33 00:03:19,710 --> 00:03:24,770 And if we create this, we get a matrix that looks like that. 34 00:03:24,770 --> 00:03:29,310 And we just pick the rows that have weight four or greater, 35 00:03:29,310 --> 00:03:32,780 and they are the generators of the 8, 4, 4 Reed-Muller code. 36 00:03:32,780 --> 00:03:35,520 So we'll take those as our set of generators. 37 00:03:35,520 --> 00:03:40,040 It's the 16 code words generated by these four 38 00:03:40,040 --> 00:03:42,980 generators here. 39 00:03:42,980 --> 00:03:46,780 This goes back to the first part of the term, but I hope 40 00:03:46,780 --> 00:03:50,790 you recall that fairly quickly. 41 00:03:50,790 --> 00:03:54,200 Now I'd like to draw a trellis diagram for this code. 42 00:03:54,200 --> 00:03:56,600 I don't particularly know why yet. 43 00:03:56,600 --> 00:03:59,490 One idea I might have in mind is to do a Viterbi algorithm 44 00:03:59,490 --> 00:04:02,080 decoding as a maximum likelihood decoding algorithm 45 00:04:02,080 --> 00:04:03,290 for this code. 46 00:04:03,290 --> 00:04:04,920 So let's see if I can come up with an 47 00:04:04,920 --> 00:04:06,130 efficient trellis diagram. 48 00:04:06,130 --> 00:04:10,190 That means one that first of all has really 49 00:04:10,190 --> 00:04:11,190 few states in it. 50 00:04:11,190 --> 00:04:15,450 Relatively few branches would be efficient as a decoding map 51 00:04:15,450 --> 00:04:16,709 for the Viterbi algorithm. 52 00:04:16,709 --> 00:04:20,180 And here's a very nice one. 53 00:04:20,180 --> 00:04:23,920 This looks sort of like a rate 1/2 54 00:04:23,920 --> 00:04:26,540 four-state trellis diagram. 55 00:04:26,540 --> 00:04:30,710 Of course, it's not time-invariant, doesn't go on 56 00:04:30,710 --> 00:04:32,730 forever like for a convolutional code, for a 57 00:04:32,730 --> 00:04:33,950 block code. 58 00:04:33,950 --> 00:04:36,950 Block code only exists over a finite time. 59 00:04:36,950 --> 00:04:40,530 So we're going to get a block trellis that starts at a 60 00:04:40,530 --> 00:04:42,600 particular time, that ends at a particular 61 00:04:42,600 --> 00:04:45,520 time in a single state. 62 00:04:45,520 --> 00:04:49,460 But we want the basic property that the -- 63 00:04:49,460 --> 00:04:53,320 the paths in this trellis are in one-to-one correspondence 64 00:04:53,320 --> 00:04:55,350 with the words in the code. 65 00:04:55,350 --> 00:04:57,460 So, how many words are there in the code? 66 00:04:57,460 --> 00:04:59,620 There are 16 in this code. 67 00:04:59,620 --> 00:05:01,690 How many paths are there through this trellis? 68 00:05:01,690 --> 00:05:04,270 Well, again it has a very regular structure. 69 00:05:04,270 --> 00:05:06,320 There is a four-way branch here. 70 00:05:06,320 --> 00:05:08,710 Wherever you get to, you have a two-way branch here. 71 00:05:08,710 --> 00:05:10,530 You have a two-way branch here. 72 00:05:10,530 --> 00:05:12,000 You have no choice here. 73 00:05:12,000 --> 00:05:16,330 So there are 16 possible ways to get through this trellis, 74 00:05:16,330 --> 00:05:19,320 16 trajectories if you like. 75 00:05:19,320 --> 00:05:23,690 And next let's check if they correspond to all the possible 76 00:05:23,690 --> 00:05:25,510 code words. 77 00:05:25,510 --> 00:05:30,590 Here is the all 0 code word up here. 78 00:05:30,590 --> 00:05:31,580 What's next? 79 00:05:31,580 --> 00:05:33,140 Let's check for the generators. 80 00:05:33,140 --> 00:05:38,300 Here is one of the generators, 1, 1, 1, 1, 0, 0, 0, 0. 81 00:05:38,300 --> 00:05:41,440 And it's a 1, 1, 0, 0. 82 00:05:41,440 --> 00:05:44,580 This is 1, 1 also. 83 00:05:44,580 --> 00:05:45,830 1, 1 -- 84 00:05:48,090 --> 00:05:50,430 no, I'm sorry. 85 00:05:50,430 --> 00:05:53,390 1, 1, 0, 0, 1, 1, 0, 0. 86 00:05:53,390 --> 00:05:53,960 That's right. 87 00:05:53,960 --> 00:05:56,590 That's this one going through here. 88 00:05:56,590 --> 00:06:00,080 And 1, 0, 1, 0, 1, 0, 1, 0. 89 00:06:00,080 --> 00:06:02,290 That's here. 90 00:06:02,290 --> 00:06:04,230 And what's the last one? 91 00:06:04,230 --> 00:06:05,060 All ones. 92 00:06:05,060 --> 00:06:08,050 Well, that's up here too. 93 00:06:08,050 --> 00:06:10,190 The all one path. 94 00:06:10,190 --> 00:06:10,520 Yeah. 95 00:06:10,520 --> 00:06:12,755 AUDIENCE: How did you know whether to make the 96 00:06:12,755 --> 00:06:14,580 [INAUDIBLE] procedure? 97 00:06:14,580 --> 00:06:15,980 PROFESSOR: I had side information. 98 00:06:15,980 --> 00:06:19,390 A genie told me. 99 00:06:19,390 --> 00:06:21,500 I know that this is an efficient trellis. 100 00:06:21,500 --> 00:06:28,200 Where we'll get to, and I hope today in this lecture, is a 101 00:06:28,200 --> 00:06:31,460 turn-the-crank method of constructing not only a 102 00:06:31,460 --> 00:06:34,570 trellis, but the minimal possible trellis. 103 00:06:34,570 --> 00:06:37,690 So you'll be able to see how starting from a set of four 104 00:06:37,690 --> 00:06:41,020 generators like this, turn the crank, and I'll produce a 105 00:06:41,020 --> 00:06:43,500 trellis for you. 106 00:06:43,500 --> 00:06:45,170 But of course you don't see that yet. 107 00:06:45,170 --> 00:06:46,800 I just know that this is a trellis, 108 00:06:46,800 --> 00:06:48,420 and it's a nice trellis. 109 00:06:48,420 --> 00:06:50,890 It has nice, regular structure. 110 00:06:50,890 --> 00:06:53,680 You could decode this with a Viterbi algorithm, and you 111 00:06:53,680 --> 00:06:57,750 could see the complexity would be comparable to the 112 00:06:57,750 --> 00:07:01,700 complexity of decoding a rate 1/2 four-state 113 00:07:01,700 --> 00:07:03,210 convolutional code. 114 00:07:03,210 --> 00:07:07,860 And in fact, you remember I compared our example rate 1/2 115 00:07:07,860 --> 00:07:10,900 four-state convolutional code which had a coding gain of 4 116 00:07:10,900 --> 00:07:15,270 dB with this code saying they roughly have the same 117 00:07:15,270 --> 00:07:15,970 complexity. 118 00:07:15,970 --> 00:07:19,360 And when I said that, this is what I had in mind, that I 119 00:07:19,360 --> 00:07:21,700 could decode each of them with a Viterbi algorithm with 120 00:07:21,700 --> 00:07:25,570 approximately the same amount of complexity. 121 00:07:25,570 --> 00:07:29,000 And unfortunately this only has a nominal coding gain of 3 122 00:07:29,000 --> 00:07:31,710 dB, and an effective coding gain of 123 00:07:31,710 --> 00:07:33,890 slightly less than that. 124 00:07:33,890 --> 00:07:37,410 So it's not quite as good as the convolutional code. 125 00:07:37,410 --> 00:07:39,340 And at the end of the day, we're going to find out that 126 00:07:39,340 --> 00:07:42,190 that's kind of typical of a comparison between block and 127 00:07:42,190 --> 00:07:46,080 convolutional codes of the same trellis complexity. 128 00:07:46,080 --> 00:07:49,870 But on the other hand, this is certainly a better way to 129 00:07:49,870 --> 00:07:55,260 decode this code, recursive by the Viterbi algorithm, than to 130 00:07:55,260 --> 00:07:59,360 do a full maximum likelihood decoding of all 16 code words, 131 00:07:59,360 --> 00:08:02,490 which involves just brute force computing the distance 132 00:08:02,490 --> 00:08:03,660 to 16 code words. 133 00:08:03,660 --> 00:08:07,590 This is if you like a more organized way of performing 134 00:08:07,590 --> 00:08:11,540 the computation of how far is the receive sequence from each 135 00:08:11,540 --> 00:08:12,790 of the 16 code words. 136 00:08:15,600 --> 00:08:21,310 So the reason we look for trellises of block codes is 137 00:08:21,310 --> 00:08:23,980 first of all, it's better than exhaustive 138 00:08:23,980 --> 00:08:25,480 maximum likelihood decoding. 139 00:08:25,480 --> 00:08:29,030 It is a maximum likelihood decoding algorithm since we're 140 00:08:29,030 --> 00:08:33,070 going to find that we can construct a trellis for any 141 00:08:33,070 --> 00:08:40,159 linear block code, which at worst, it's never going to be 142 00:08:40,159 --> 00:08:44,000 more complicated than just enumerating all the code words 143 00:08:44,000 --> 00:08:46,940 and doing exhaustive maximum likelihood decoding. 144 00:08:46,940 --> 00:08:49,760 And usually, as in this case, it's going to be a more 145 00:08:49,760 --> 00:08:53,440 efficient method of doing maximum likelihood decoding. 146 00:08:53,440 --> 00:08:56,310 So we will have a general method of maximum likelihood 147 00:08:56,310 --> 00:08:58,310 decoding that is more efficient than 148 00:08:58,310 --> 00:09:01,150 the exhaustive method. 149 00:09:01,150 --> 00:09:02,400 Secondly, there's an interesting 150 00:09:02,400 --> 00:09:05,120 link to system theory. 151 00:09:05,120 --> 00:09:07,670 I won't make a great deal of this in this course. 152 00:09:07,670 --> 00:09:09,000 But what are we really doing here? 153 00:09:09,000 --> 00:09:16,280 We're representing a block code as a finite state system. 154 00:09:16,280 --> 00:09:19,030 In fact, a linear block code is a finite state linear 155 00:09:19,030 --> 00:09:21,980 system, although again the linearity is not terribly 156 00:09:21,980 --> 00:09:25,150 transparent from this picture. 157 00:09:25,150 --> 00:09:29,510 So we have a linear finite state system, just as we did 158 00:09:29,510 --> 00:09:31,300 for the convolutional code. 159 00:09:31,300 --> 00:09:34,045 However, it's necessarily time varying. 160 00:09:34,045 --> 00:09:37,360 It couldn't possibly be shift invariant because we only have 161 00:09:37,360 --> 00:09:39,600 a finite time axis, if you like. 162 00:09:39,600 --> 00:09:44,220 Time axis only goes over four time units in this particular 163 00:09:44,220 --> 00:09:47,180 picture, where I've grouped two outputs at a time. 164 00:09:49,870 --> 00:09:57,190 So that's interesting, and for our purposes an interesting 165 00:09:57,190 --> 00:10:02,940 aspect of this is that we now get a notion of how complex is 166 00:10:02,940 --> 00:10:04,920 a block code. 167 00:10:04,920 --> 00:10:08,710 So far we've been focused on parameters like n, k, d, which 168 00:10:08,710 --> 00:10:13,670 are all algebraic parameters of the code itself. 169 00:10:13,670 --> 00:10:16,310 But we really want to know about performance versus 170 00:10:16,310 --> 00:10:16,940 complexity. 171 00:10:16,940 --> 00:10:19,540 So how complex is the 8 4 4 code? 172 00:10:19,540 --> 00:10:21,400 Well, now we have a little bit of a handle on it. 173 00:10:21,400 --> 00:10:27,730 We can say its complexity is that of a four-state machine. 174 00:10:27,730 --> 00:10:31,900 So trellis complexity gives us a measure of the complexity of 175 00:10:31,900 --> 00:10:35,330 a block code so that we can, for instance, say the 176 00:10:35,330 --> 00:10:38,955 Reed-Muller codes tend to be less complex than PCH codes. 177 00:10:38,955 --> 00:10:41,120 And I'll try to give some substance to 178 00:10:41,120 --> 00:10:43,450 that as we go along. 179 00:10:43,450 --> 00:10:46,110 So we get additional parameters that have more to 180 00:10:46,110 --> 00:10:48,460 do with what we really care about, which is decoding 181 00:10:48,460 --> 00:10:51,050 complexity. 182 00:10:51,050 --> 00:10:54,030 And finally, I introduce this subject because it's a 183 00:10:54,030 --> 00:10:58,270 stepping stone to where we're really going, is this notion 184 00:10:58,270 --> 00:11:02,990 of codes on graphs, which is the underlying concept for the 185 00:11:02,990 --> 00:11:07,510 capacity-approaching codes that is our goal point. 186 00:11:07,510 --> 00:11:11,650 Our whole goal in this course is to get to capacity, 187 00:11:11,650 --> 00:11:14,800 specifically on the additive white Gaussian noise channel. 188 00:11:14,800 --> 00:11:17,300 And the way that people have found they get to capacity 189 00:11:17,300 --> 00:11:20,690 turbo codes, low-density parity check codes, the 190 00:11:20,690 --> 00:11:24,160 underlying concept that I'm going to be framing that in is 191 00:11:24,160 --> 00:11:25,680 codes on graphs. 192 00:11:25,680 --> 00:11:30,950 And this is kind of a code on a very elementary graph. 193 00:11:30,950 --> 00:11:34,910 So it's a good way from here to there. 194 00:11:34,910 --> 00:11:37,760 So that's why we're taking a little time to 195 00:11:37,760 --> 00:11:38,750 look at this subject. 196 00:11:38,750 --> 00:11:42,600 Even though as I say, at the end of the day, even though 197 00:11:42,600 --> 00:11:44,780 there's a better way of decoding block codes than the 198 00:11:44,780 --> 00:11:49,670 ways we've had previously, and a better non-algebraic way, it 199 00:11:49,670 --> 00:11:52,480 still is not going to turn out to be as good as convolutional 200 00:11:52,480 --> 00:11:54,226 codes, which we already know about. 201 00:11:57,020 --> 00:11:58,930 Any questions on what we're doing, 202 00:11:58,930 --> 00:12:00,840 motivation and so forth? 203 00:12:00,840 --> 00:12:01,788 Yeah. 204 00:12:01,788 --> 00:12:06,054 AUDIENCE: [INAUDIBLE] convolutional codes is that 205 00:12:06,054 --> 00:12:08,424 each node doesn't have two things coming on to it? 206 00:12:08,424 --> 00:12:11,820 PROFESSOR: That's right, it's a little bit more irregular, 207 00:12:11,820 --> 00:12:15,270 and this trellis section doesn't look the same. 208 00:12:15,270 --> 00:12:18,210 We had a completely mixing trellis section for the 209 00:12:18,210 --> 00:12:19,400 convolutional code. 210 00:12:19,400 --> 00:12:24,160 This kind of divides it into two halves which don't meet. 211 00:12:24,160 --> 00:12:29,300 This basically expresses the code as a certain subcode, and 212 00:12:29,300 --> 00:12:31,450 its coset is down here. 213 00:12:31,450 --> 00:12:33,200 So, it's not quite the same. 214 00:12:33,200 --> 00:12:35,080 Looks different. 215 00:12:35,080 --> 00:12:37,801 Maybe this will turn out to be interesting. 216 00:12:37,801 --> 00:12:39,051 Good observation. 217 00:12:43,430 --> 00:12:44,560 Anything else at this point? 218 00:12:44,560 --> 00:12:46,352 AUDIENCE: You just said that this is not a 219 00:12:46,352 --> 00:12:47,696 convolutional code. 220 00:12:47,696 --> 00:12:51,760 Or maybe we can have more than one states, and it is a 221 00:12:51,760 --> 00:12:54,030 convolutional code. 222 00:12:54,030 --> 00:12:56,020 PROFESSOR: Let's see, it's not a convolutional code. 223 00:12:56,020 --> 00:13:01,800 It's not even a terminated convolutional code, because it 224 00:13:01,800 --> 00:13:03,670 doesn't have the simple shift register 225 00:13:03,670 --> 00:13:05,150 type of trellis diagram. 226 00:13:05,150 --> 00:13:07,060 At least it's not the kind of convolutional 227 00:13:07,060 --> 00:13:09,380 code we know about. 228 00:13:09,380 --> 00:13:10,780 But why should it be? 229 00:13:10,780 --> 00:13:14,565 Let's open up our minds to more possibilities. 230 00:13:17,490 --> 00:13:20,810 We don't really care. 231 00:13:20,810 --> 00:13:25,300 The objective that we have in mind here is we're just going 232 00:13:25,300 --> 00:13:29,950 to try to come up with the most efficient trellis picture 233 00:13:29,950 --> 00:13:32,560 that we can for block code. 234 00:13:32,560 --> 00:13:37,020 And we're going to use the linearity, and we're going to 235 00:13:37,020 --> 00:13:39,820 basically try to find as few states as 236 00:13:39,820 --> 00:13:41,670 possible at each time. 237 00:13:41,670 --> 00:13:44,370 That's going to be our measure of efficiency. 238 00:13:44,370 --> 00:13:46,550 That's not the only one you could think of, but it'll turn 239 00:13:46,550 --> 00:13:51,330 out that any notion of efficient representation comes 240 00:13:51,330 --> 00:13:52,820 down to the same thing. 241 00:13:52,820 --> 00:13:55,750 So we'll focus on a minimal state representation of a 242 00:13:55,750 --> 00:14:00,100 block code, and you can think of this as an exercise in 243 00:14:00,100 --> 00:14:01,570 system theory and minimal 244 00:14:01,570 --> 00:14:04,400 realizations of a linear system. 245 00:14:04,400 --> 00:14:06,860 This you can view. 246 00:14:06,860 --> 00:14:09,730 A set of all 16 code words you can view is the set of 247 00:14:09,730 --> 00:14:14,160 possible trajectories of a linear system on a time axis 248 00:14:14,160 --> 00:14:18,610 of length 8, and we want to find a minimal realization, 249 00:14:18,610 --> 00:14:22,100 minimal state realization of that linear system. 250 00:14:22,100 --> 00:14:25,660 So that'll make sense to some of you, and it won't make an 251 00:14:25,660 --> 00:14:27,040 awful lot of sense to the rest of you. 252 00:14:29,810 --> 00:14:36,610 So we're after a minimal, in the state sense, minimal state 253 00:14:36,610 --> 00:14:38,098 complexity, let's say. 254 00:14:42,940 --> 00:14:47,180 And let's continue to focus on this example, and let's focus 255 00:14:47,180 --> 00:14:48,745 on a particular time. 256 00:14:48,745 --> 00:14:53,790 Let's focus on the halfway point here. 257 00:14:53,790 --> 00:14:56,330 Where do the state times occur, by the way? 258 00:14:56,330 --> 00:14:59,100 They occur between the symbol tags. 259 00:14:59,100 --> 00:15:02,960 You can think of a state as being associated with a cut 260 00:15:02,960 --> 00:15:07,730 between a certain set of symbols which we call the past 261 00:15:07,730 --> 00:15:11,170 and another set of symbols which we call the future, 262 00:15:11,170 --> 00:15:17,090 again using just some theoretic temporal language. 263 00:15:17,090 --> 00:15:18,400 So if we make a cut -- 264 00:15:18,400 --> 00:15:22,420 let's say I do it over here. 265 00:15:22,420 --> 00:15:25,920 States are based on cuts between a past and a future, 266 00:15:25,920 --> 00:15:29,470 so the state actually occurs between the fourth and the 267 00:15:29,470 --> 00:15:33,030 fifth symbol here, not at either of them, if you want to 268 00:15:33,030 --> 00:15:37,330 draw where is the stated time. 269 00:15:37,330 --> 00:15:39,960 And let's start with this example. 270 00:15:39,960 --> 00:15:45,040 And let's ask if we could find any simpler trellis for this. 271 00:15:45,040 --> 00:15:49,680 In other words, let's see if we can have fewer than four 272 00:15:49,680 --> 00:15:51,155 states at this midpoint. 273 00:15:54,640 --> 00:15:56,330 Could we possibly have fewer than four 274 00:15:56,330 --> 00:15:57,580 states at the midpoint? 275 00:16:04,360 --> 00:16:07,420 What's the key property of states? 276 00:16:07,420 --> 00:16:11,840 The key property of states is that once you get to a state, 277 00:16:11,840 --> 00:16:15,390 that then becomes a summary of all the history that you know 278 00:16:15,390 --> 00:16:23,940 about the past, and any past sequence or partial sequence 279 00:16:23,940 --> 00:16:28,910 that gets you to this state has to have the same set of 280 00:16:28,910 --> 00:16:33,440 possible continuations over here in the future in order 281 00:16:33,440 --> 00:16:37,740 for this to be a valid trellis representation. 282 00:16:37,740 --> 00:16:41,750 So maybe it will help if I draw this just focusing on the 283 00:16:41,750 --> 00:16:42,690 central time. 284 00:16:42,690 --> 00:16:46,010 There are two ways to get to this central time. 285 00:16:46,010 --> 00:16:49,450 One is to get to what we call the 0 state. 286 00:16:49,450 --> 00:16:53,900 We can get there by 0, 0, 0, 0 or 1, 1, 1, 1. 287 00:16:53,900 --> 00:16:58,510 There's a second state which we can get to again by two 288 00:16:58,510 --> 00:17:07,050 paths, which are 0, 0, 1, 1 or 1, 1, 0, 0. 289 00:17:07,050 --> 00:17:13,839 There's a third one, which we can get to by 1, 0, 1, 0 or 290 00:17:13,839 --> 00:17:17,190 its complement, 0, 1, 0, 1. 291 00:17:17,190 --> 00:17:22,750 We'll see that each of these involves a four-tuple and its 292 00:17:22,750 --> 00:17:23,422 complement. 293 00:17:23,422 --> 00:17:29,070 And this is 1, 0, 0, 1 or 0, 1, 1, 0. 294 00:17:29,070 --> 00:17:34,590 Those are the set of possible past sequences that can get to 295 00:17:34,590 --> 00:17:36,620 any of these four states. 296 00:17:36,620 --> 00:17:39,410 There are two of them. 297 00:17:39,410 --> 00:17:44,910 Now what this trellis says is that from this state, 298 00:17:44,910 --> 00:17:49,250 regardless of how we got here, either of these two things, 299 00:17:49,250 --> 00:17:52,140 there are two possible continuations, which happen to 300 00:17:52,140 --> 00:17:53,390 be the same thing. 301 00:17:57,000 --> 00:17:59,640 And the property state has to be that either of these 302 00:17:59,640 --> 00:18:05,580 continuations is a legitimate continuation of either of the 303 00:18:05,580 --> 00:18:07,240 paths that it takes to get there. 304 00:18:07,240 --> 00:18:10,900 So we now have four possible code words that go through 305 00:18:10,900 --> 00:18:15,470 this particular state, that pass through this state. 306 00:18:15,470 --> 00:18:20,560 And down here, similarly, I think what we have is the same 307 00:18:20,560 --> 00:18:23,370 set of code words. 308 00:18:23,370 --> 00:18:29,480 There's complete symmetry and so forth. 309 00:18:29,480 --> 00:18:34,550 Let's ask, could we combine these two states, somehow 310 00:18:34,550 --> 00:18:36,985 smush them together into a single state? 311 00:18:36,985 --> 00:18:41,880 And the answer is obviously no, because 0, 0, 1, 1 is not 312 00:18:41,880 --> 00:18:45,040 a continuation of the all 0 sequence, or 313 00:18:45,040 --> 00:18:46,530 of the all 1 sequence. 314 00:18:46,530 --> 00:18:50,320 And 0, 0, 0, 0 is not a continuation of the 0, 0, 1, 1 315 00:18:50,320 --> 00:18:53,605 sequence or the 1, 1, 0, 0 sequence. 316 00:18:56,240 --> 00:18:59,210 The property of the state is the Markov property. 317 00:19:02,560 --> 00:19:07,950 And there are many ways of phrasing this, but this is the 318 00:19:07,950 --> 00:19:31,880 defining property of states, is that if two past paths have 319 00:19:31,880 --> 00:19:33,130 a future continuation -- 320 00:19:35,800 --> 00:19:40,040 I hope you understand this language as I introduce it -- 321 00:19:40,040 --> 00:19:54,640 in common, then all their future 322 00:19:54,640 --> 00:19:56,150 continuations are in common. 323 00:20:12,500 --> 00:20:15,890 So I have two past paths, let's say. 324 00:20:15,890 --> 00:20:17,800 This is the definition of states. 325 00:20:17,800 --> 00:20:22,030 If we can find such a situation, then we can say 326 00:20:22,030 --> 00:20:27,210 that these two past paths go through the same state at the 327 00:20:27,210 --> 00:20:32,350 cut time, because then we can smush them together. 328 00:20:32,350 --> 00:20:36,270 You can think of this as our starting from, but suppose we 329 00:20:36,270 --> 00:20:41,080 drew all 16 code words, and we drew a -- 330 00:20:41,080 --> 00:20:45,350 here would be a 16 state trellis, and the first two 331 00:20:45,350 --> 00:20:50,440 elements of it might be 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1. 332 00:20:50,440 --> 00:20:53,810 And we'd ask, can we smush these two states together? 333 00:20:53,810 --> 00:20:58,870 And the answer is yes because of this property, because 0, 334 00:20:58,870 --> 00:21:01,840 0, 0 can also be followed by 1, 1, 1, 1. 335 00:21:01,840 --> 00:21:03,960 There are actually four states that we're 336 00:21:03,960 --> 00:21:05,360 smushing together here. 337 00:21:09,720 --> 00:21:11,850 1, 1, 1, 1 and 0, 0, 0. 338 00:21:11,850 --> 00:21:14,360 These are all legitimate code words, and because they're all 339 00:21:14,360 --> 00:21:18,340 legitimate, we can basically break this up into a Cartesian 340 00:21:18,340 --> 00:21:23,240 product of two paths with two futures, and represent all 341 00:21:23,240 --> 00:21:24,710 four of them by a single state. 342 00:21:24,710 --> 00:21:27,300 So that's what states do. 343 00:21:27,300 --> 00:21:30,850 If we want a minimal state diagram, we should do this as 344 00:21:30,850 --> 00:21:31,840 much as possible. 345 00:21:31,840 --> 00:21:35,180 It's not very well-defined right now, but we should merge 346 00:21:35,180 --> 00:21:38,870 states wherever this property holds. 347 00:21:38,870 --> 00:21:45,330 So this implies they go through a common state, and 348 00:21:45,330 --> 00:21:46,580 it's if and only if. 349 00:21:52,080 --> 00:21:54,730 So you could see it very concretely 350 00:21:54,730 --> 00:21:55,920 from the trellis diagram. 351 00:21:55,920 --> 00:21:58,890 This is the property that we need. 352 00:21:58,890 --> 00:22:03,510 You can see that for this particular code, we do have 353 00:22:03,510 --> 00:22:08,630 the possibility of combining all 16 possible states here 354 00:22:08,630 --> 00:22:12,360 into 4, because we can combine them all 355 00:22:12,360 --> 00:22:14,380 pair-wise in this way. 356 00:22:14,380 --> 00:22:16,770 This is going to come from the linear property of the code. 357 00:22:16,770 --> 00:22:20,208 That's why everything is so symmetric. 358 00:22:20,208 --> 00:22:23,030 But we can't go any further than that. 359 00:22:23,030 --> 00:22:30,282 So we can conclude from this is that for the 8, 4, 4 code, 360 00:22:30,282 --> 00:22:40,620 the minimal state space at the center has four 361 00:22:40,620 --> 00:22:47,900 states, has size four. 362 00:22:53,440 --> 00:22:55,920 For simple code like this, you can simply see what all the 363 00:22:55,920 --> 00:23:00,340 possibilities are, and this is the best we can do. 364 00:23:00,340 --> 00:23:02,190 So do you get that? 365 00:23:02,190 --> 00:23:05,490 This is fundamental system theory 366 00:23:05,490 --> 00:23:07,720 state realization theory. 367 00:23:07,720 --> 00:23:09,620 I don't know where in the curriculum 368 00:23:09,620 --> 00:23:11,650 one gets this nowadays. 369 00:23:11,650 --> 00:23:16,540 Probably somewhere over towards the control side, 370 00:23:16,540 --> 00:23:21,280 maybe in digital signal processing, but maybe not. 371 00:23:21,280 --> 00:23:23,240 But we're doing system theory here. 372 00:23:26,910 --> 00:23:28,768 We all get that? 373 00:23:28,768 --> 00:23:29,716 Yeah. 374 00:23:29,716 --> 00:23:30,664 AUDIENCE: [INAUDIBLE] 375 00:23:30,664 --> 00:23:35,605 properties basically defined, I have two possible paths, and 376 00:23:35,605 --> 00:23:37,887 they have one common [INAUDIBLE] 377 00:23:37,887 --> 00:23:42,780 data, they have [INAUDIBLE] 378 00:23:42,780 --> 00:23:46,040 PROFESSOR: Yeah, if they go through a common state, that 379 00:23:46,040 --> 00:23:48,840 implies this statement. 380 00:23:48,840 --> 00:23:55,160 Conversely, if we have a set of past paths and future paths 381 00:23:55,160 --> 00:23:58,902 that have this property, then we can define a state. 382 00:23:58,902 --> 00:24:01,530 In other words, states are always going to look something 383 00:24:01,530 --> 00:24:04,840 like this, with a set of past paths. 384 00:24:04,840 --> 00:24:11,520 We have a set of future paths, and the state is simply a node 385 00:24:11,520 --> 00:24:14,050 that they go through such that you can combine any of these 386 00:24:14,050 --> 00:24:15,710 paths with any of these futures. 387 00:24:15,710 --> 00:24:18,100 If it's symmetric, you could say the same thing. 388 00:24:18,100 --> 00:24:22,050 If two futures have a past in common, then they have all 389 00:24:22,050 --> 00:24:23,740 their paths in common. 390 00:24:23,740 --> 00:24:28,430 But whenever you have a state representation -- 391 00:24:28,430 --> 00:24:31,770 let's suppose we have three states there. 392 00:24:31,770 --> 00:24:35,530 Then this is what, just looking at those states and 393 00:24:35,530 --> 00:24:39,000 the initial and final states, this is what the code is going 394 00:24:39,000 --> 00:24:40,250 to have to look like. 395 00:24:44,850 --> 00:24:50,180 Because property of state is that however you got there, 396 00:24:50,180 --> 00:24:52,520 you have to be able to take any of these over here. 397 00:24:52,520 --> 00:24:54,620 Otherwise the trellis is invalid. 398 00:24:54,620 --> 00:24:57,720 If not all of these nine sequences are code words, then 399 00:24:57,720 --> 00:25:00,370 it's not a valid trellis representation, or 400 00:25:00,370 --> 00:25:04,590 equivalently, it's not a valid state representation. 401 00:25:04,590 --> 00:25:08,690 So if, and only if, we can draw the thing in 402 00:25:08,690 --> 00:25:11,090 this way, we get it. 403 00:25:11,090 --> 00:25:16,390 So for a small code, you can just see how much combining 404 00:25:16,390 --> 00:25:18,400 you can do. 405 00:25:18,400 --> 00:25:22,120 In particular, you can ask, what can the 0, 0 state -- 406 00:25:22,120 --> 00:25:25,210 0, 0 path sequence -- what can that be combined with? 407 00:25:28,370 --> 00:25:38,160 Well, it can only be combined with sequences that have a 0 408 00:25:38,160 --> 00:25:39,070 first part. 409 00:25:39,070 --> 00:25:43,180 So let me introduce now the idea of subcodes. 410 00:25:50,960 --> 00:26:07,610 We have a linear code C. Let's define an interval on the time 411 00:26:07,610 --> 00:26:13,540 axis: k, k prime, whatever. 412 00:26:13,540 --> 00:26:21,440 The subcode C on that interval is the set of all code words. 413 00:26:29,100 --> 00:26:35,010 Elegant way to say it is whose support is in k, k prime. 414 00:26:39,080 --> 00:26:42,940 In other words, which are all 0 outside, which have all 0 415 00:26:42,940 --> 00:26:45,130 symbols outside of this interval that we've 416 00:26:45,130 --> 00:26:46,410 identified. 417 00:26:46,410 --> 00:26:49,390 So that's just notation. 418 00:26:49,390 --> 00:26:55,350 So specifically, let's say the past code with respect to a 419 00:26:55,350 --> 00:26:58,630 certain time, like the midpoint there. 420 00:26:58,630 --> 00:27:04,450 Well, we can define that as the set of all code words that 421 00:27:04,450 --> 00:27:08,020 are 0 outside the past, the first four symbols, and we 422 00:27:08,020 --> 00:27:12,800 define the future subcode as the set of all code words that 423 00:27:12,800 --> 00:27:14,400 are 0 outside the future. 424 00:27:14,400 --> 00:27:16,620 For this code, what is that? 425 00:27:16,620 --> 00:27:18,630 We list the 16 code words. 426 00:27:18,630 --> 00:27:20,070 We make this the past. 427 00:27:20,070 --> 00:27:22,680 We make this the future. 428 00:27:22,680 --> 00:27:28,700 What is the past subcode here? 429 00:27:28,700 --> 00:27:33,430 In this particular case, the past subcode is, of course, 430 00:27:33,430 --> 00:27:34,380 the all 0 sequence. 431 00:27:34,380 --> 00:27:39,250 It was always in this code or this sequence, 432 00:27:39,250 --> 00:27:41,110 1, 1, 1, 0, 0, 0. 433 00:27:41,110 --> 00:27:44,690 It's a linear code of dimension one, and it consists 434 00:27:44,690 --> 00:27:45,940 of these two code words. 435 00:27:51,030 --> 00:27:56,120 It is precisely the set of all code words that can be 436 00:27:56,120 --> 00:27:59,450 followed by the all 0 sequence. 437 00:27:59,450 --> 00:28:03,060 So I can read it directly off of this trellis picture here. 438 00:28:06,620 --> 00:28:07,510 So everybody with me? 439 00:28:07,510 --> 00:28:11,510 The past subcode just consists of those two words. 440 00:28:11,510 --> 00:28:16,460 The future subcode consists similarly. 441 00:28:16,460 --> 00:28:23,370 Looking at the trellis, it's the 1's that are all 0 outside 442 00:28:23,370 --> 00:28:24,310 the future. 443 00:28:24,310 --> 00:28:28,040 In other words, can follow a past which is all 0. 444 00:28:28,040 --> 00:28:29,860 In other words, it's these two code words. 445 00:28:34,230 --> 00:28:36,640 So here are two little subcodes with a code. 446 00:28:39,210 --> 00:28:48,940 So by definition, this is subcode of C. If C is linear, 447 00:28:48,940 --> 00:28:51,940 you can quickly convince yourself this is a linear 448 00:28:51,940 --> 00:28:55,090 subcode, so it's going to have a certain dimension. 449 00:28:55,090 --> 00:28:57,980 It's going to have size equal to a power of two. 450 00:28:57,980 --> 00:29:00,950 It's even going to have a minimum distance that is at 451 00:29:00,950 --> 00:29:03,750 least as great as the minimum distance of C, because it 452 00:29:03,750 --> 00:29:07,560 consists of code words of C, so its minimum non-zero weight 453 00:29:07,560 --> 00:29:12,040 is going to be at least as great as that of C. So it has 454 00:29:12,040 --> 00:29:13,290 some properties immediately. 455 00:29:18,840 --> 00:29:23,450 These past and future codes seem to have a lot to do with 456 00:29:23,450 --> 00:29:26,230 the structure of the trellis up here. 457 00:29:26,230 --> 00:29:33,400 In fact, the zero state consists of precisely -- 458 00:29:33,400 --> 00:29:40,440 the sequences that go through the zero state here are not 459 00:29:40,440 --> 00:29:46,310 coincidentally the set of all past code words in Cp followed 460 00:29:46,310 --> 00:29:49,740 by the set of all future code words in Cf. 461 00:29:56,000 --> 00:29:57,100 And why is that? 462 00:29:57,100 --> 00:30:05,350 It's because the set of all ways of getting to here, 463 00:30:05,350 --> 00:30:08,090 getting to this state, have to be the set of all code 464 00:30:08,090 --> 00:30:12,770 sequences that can be followed by the all 0 sequence. 465 00:30:12,770 --> 00:30:16,200 Similarly, the set of all continuations from this state 466 00:30:16,200 --> 00:30:18,190 have to be the set of all continuations 467 00:30:18,190 --> 00:30:20,900 of the all 0 sequence. 468 00:30:20,900 --> 00:30:24,690 So the zero state is always going to be a little 469 00:30:24,690 --> 00:30:28,660 sub-trellis that is going to represent in effect -- 470 00:30:28,660 --> 00:30:32,930 you can draw it as a sum, or a product, of the past and 471 00:30:32,930 --> 00:30:34,000 future subcodes. 472 00:30:34,000 --> 00:30:38,690 In other words, there's a two-dimensional code that has 473 00:30:38,690 --> 00:30:40,450 these two generators. 474 00:30:40,450 --> 00:30:44,600 g1, g2 generates a little two-dimensional code. 475 00:30:44,600 --> 00:30:49,040 Here's the fourth code word in it, the all 1 code sequence. 476 00:30:49,040 --> 00:30:52,970 And every element in that two-dimensional code goes 477 00:30:52,970 --> 00:30:54,220 through the zero state. 478 00:30:56,922 --> 00:30:58,890 AUDIENCE: [INAUDIBLE] 479 00:30:58,890 --> 00:30:59,874 code word. 480 00:30:59,874 --> 00:31:02,050 So then you know -- 481 00:31:02,050 --> 00:31:03,890 PROFESSOR: It's whatever it happens to be. 482 00:31:03,890 --> 00:31:07,110 AUDIENCE: Then you have a separate state just for -- 483 00:31:11,910 --> 00:31:14,150 you could combine them. 484 00:31:14,150 --> 00:31:15,930 PROFESSOR: It doesn't matter. 485 00:31:15,930 --> 00:31:20,780 Suppose that Cp has dimension one. 486 00:31:20,780 --> 00:31:22,720 Cf has dimension one. 487 00:31:22,720 --> 00:31:27,515 That means Cp is going to be something like this. 488 00:31:30,810 --> 00:31:34,830 Cf is going to be something like this. 489 00:31:34,830 --> 00:31:37,358 Whatever, what way. 490 00:31:37,358 --> 00:31:39,500 AUDIENCE: [INAUDIBLE] 491 00:31:39,500 --> 00:31:44,110 PROFESSOR: It just follows from linearity. 492 00:31:44,110 --> 00:31:47,650 And clearly, since C itself was linear, we're allowed to 493 00:31:47,650 --> 00:31:51,010 add x, x, x, x all 0's to y, y, y, y. 494 00:31:51,010 --> 00:31:53,800 That's a code word, because these were both code words in 495 00:31:53,800 --> 00:32:00,720 C. So the linearity allows you to fill in this fourth corner 496 00:32:00,720 --> 00:32:04,410 of the rectangle, if you like. 497 00:32:04,410 --> 00:32:09,160 So Cp plus Cf is always going to look like this. 498 00:32:09,160 --> 00:32:12,930 And we'll always call this the zero state, the state that you 499 00:32:12,930 --> 00:32:15,330 get to by the all 0 sequence. 500 00:32:15,330 --> 00:32:18,580 And that can be followed by the all 0 sequence. 501 00:32:18,580 --> 00:32:22,220 That's always going to be called the zero state. 502 00:32:22,220 --> 00:32:24,620 By linearity, it's always going to look like that. 503 00:32:24,620 --> 00:32:26,370 It might not be dimension one. 504 00:32:26,370 --> 00:32:27,650 It could have any dimension here. 505 00:32:32,330 --> 00:32:36,920 So we're really beginning to get somewhere now. 506 00:32:36,920 --> 00:32:42,170 What do all these other states look like, grouped 507 00:32:42,170 --> 00:32:43,420 theoretically? 508 00:32:46,310 --> 00:32:47,130 Cosets. 509 00:32:47,130 --> 00:32:49,730 Somebody -- who said cosets? 510 00:32:49,730 --> 00:32:53,010 Good, that's right. 511 00:32:53,010 --> 00:32:57,870 So Cp plus Cf is itself a subcode of C, a 512 00:32:57,870 --> 00:33:00,530 two-dimensional subcode. 513 00:33:00,530 --> 00:33:07,050 So there are four cosets of Cp plus Cf in C, and we'll see if 514 00:33:07,050 --> 00:33:09,385 they correspond to the four states. 515 00:33:14,000 --> 00:33:16,080 So how do we do this algebraically? 516 00:33:19,130 --> 00:33:24,645 Let's draw a generator matrix for C in a certain form. 517 00:33:29,280 --> 00:33:33,670 Again, I'll draw the past, the future. 518 00:33:33,670 --> 00:33:37,690 So that's the only division I'm concerned with right now. 519 00:33:37,690 --> 00:33:42,990 And let me draw it in general form first. 520 00:33:42,990 --> 00:33:47,860 I've defined this past subcode, Cp, at a certain 521 00:33:47,860 --> 00:33:51,140 dimension, so it has a certain number of generators. 522 00:33:51,140 --> 00:33:56,420 So we're going to put up here a set of generators for Cp, g 523 00:33:56,420 --> 00:34:01,390 of Cp, however many there happen to be. 524 00:34:01,390 --> 00:34:03,810 And what's their common characteristic? 525 00:34:03,810 --> 00:34:08,935 They all are all 0 in the future. 526 00:34:08,935 --> 00:34:10,614 So they're all going to look like that. 527 00:34:16,460 --> 00:34:22,690 Then I'm going to take a set of generators of Cf, and 528 00:34:22,690 --> 00:34:27,480 similarly I'm going to use them up over there. 529 00:34:27,480 --> 00:34:30,739 These are all code words, and they're clearly linear and 530 00:34:30,739 --> 00:34:32,400 independent of those. 531 00:34:32,400 --> 00:34:34,630 So I'm on my way to constructing a generator 532 00:34:34,630 --> 00:34:36,920 matrix for C. 533 00:34:36,920 --> 00:34:40,750 But obviously I'm going to need some more. 534 00:34:40,750 --> 00:34:44,600 We can call this the number of generators in the past, the 535 00:34:44,600 --> 00:34:47,199 number of generators in the future. 536 00:34:47,199 --> 00:34:49,699 This is the dimension of Cp. 537 00:34:49,699 --> 00:34:53,699 This is the dimension of Cf. 538 00:34:53,699 --> 00:34:57,490 And now I need some more generators, which by 539 00:34:57,490 --> 00:35:00,435 definition are going to have to span both past and future. 540 00:35:03,790 --> 00:35:06,240 Let's see. 541 00:35:06,240 --> 00:35:09,880 Since I've already introduced the coset language, let me 542 00:35:09,880 --> 00:35:16,400 just mysteriously put that as the generators 543 00:35:16,400 --> 00:35:18,350 of a quotient group. 544 00:35:18,350 --> 00:35:25,150 C mod Cp and Cf is what that means, and it's a quotient 545 00:35:25,150 --> 00:35:30,442 group, if you know or recall what a quotient group is. 546 00:35:30,442 --> 00:35:35,790 But anyway, there are k minus kp minus kf of these. 547 00:35:35,790 --> 00:35:38,440 And I don't particularly care what I put down in here. 548 00:35:38,440 --> 00:35:40,480 Their property is that they have to span 549 00:35:40,480 --> 00:35:44,070 both past and future. 550 00:35:44,070 --> 00:35:48,830 So for our particular example, here's the generator for the 551 00:35:48,830 --> 00:35:52,250 8, 4, 4 code. 552 00:35:52,250 --> 00:35:55,590 What we're talking about is a generator that looks 553 00:35:55,590 --> 00:35:58,390 like 1, 1, 1, 1. 554 00:35:58,390 --> 00:36:02,410 That's k past is 1. 555 00:36:02,410 --> 00:36:05,470 0, 0, 0, 1, 1, 1, 1. 556 00:36:05,470 --> 00:36:07,220 k future is 1. 557 00:36:07,220 --> 00:36:09,700 They're both one-dimensional codes. 558 00:36:09,700 --> 00:36:14,150 Now I need two other generators, which must span 559 00:36:14,150 --> 00:36:15,400 past and future. 560 00:36:17,690 --> 00:36:22,180 So what'll I take? 561 00:36:22,180 --> 00:36:26,020 Let me just take two more. 562 00:36:26,020 --> 00:36:32,675 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0. 563 00:36:32,675 --> 00:36:35,400 That would be one possible choice. 564 00:36:35,400 --> 00:36:41,350 And so this is k minus kp minus kf, which is 2. 565 00:36:53,250 --> 00:36:57,740 What I'm going to claim is that I'm going to need a state 566 00:36:57,740 --> 00:37:00,510 for every possible linear combination of these 567 00:37:00,510 --> 00:37:01,760 generators. 568 00:37:06,330 --> 00:37:07,580 So claim -- 569 00:37:11,330 --> 00:37:16,880 well, let me break it down into more logical sequence. 570 00:37:16,880 --> 00:37:27,020 Every code word in C can be written as some linear 571 00:37:27,020 --> 00:37:30,020 combination of the generators. 572 00:37:30,020 --> 00:37:32,060 What does that amount to? 573 00:37:32,060 --> 00:37:40,010 It amounts to a certain code word in the past code plus a 574 00:37:40,010 --> 00:37:43,540 certain component in the future code, either all 0 or 575 00:37:43,540 --> 00:37:46,080 they're all one, plus -- 576 00:37:46,080 --> 00:37:52,430 let me call this the state code, S, as I do in the notes. 577 00:37:52,430 --> 00:37:55,410 So in this case we have a two-dimensional state code. 578 00:37:55,410 --> 00:37:56,890 Its elements are eight-tuples. 579 00:37:56,890 --> 00:38:00,770 It's just a subcode of the code we started with, so it's 580 00:38:00,770 --> 00:38:06,440 not quite clear how it corresponds to states yet. 581 00:38:06,440 --> 00:38:10,340 Well, let me just call that S for state sequence. 582 00:38:17,250 --> 00:38:22,730 And I claim I can break this up because the proof is that 583 00:38:22,730 --> 00:38:29,240 for every c in C is equal to a sum of -- 584 00:38:29,240 --> 00:38:31,150 it's a linear combination of the generators. 585 00:38:31,150 --> 00:38:34,480 So I just break that up into the three possible parts. 586 00:38:34,480 --> 00:38:38,200 So I get one part that's in the past subcode, one part's 587 00:38:38,200 --> 00:38:41,030 in the future subcode, and one is in the state. 588 00:38:41,030 --> 00:38:45,260 So this is the past, this is the future, this is the state. 589 00:38:45,260 --> 00:38:48,700 So I just break it up according to this. 590 00:38:48,700 --> 00:38:51,760 So the 16 code words can all be written in this way in our 591 00:38:51,760 --> 00:38:53,010 particular example. 592 00:38:57,610 --> 00:39:03,346 So let's project this code word on the past. 593 00:39:03,346 --> 00:39:04,955 What does projection mean? 594 00:39:04,955 --> 00:39:06,905 It means you just don't look at the part 595 00:39:06,905 --> 00:39:08,470 that's in the future. 596 00:39:08,470 --> 00:39:10,350 So we're looking at four-tuples now. 597 00:39:14,210 --> 00:39:19,970 So C projected on to the past, this is the part 598 00:39:19,970 --> 00:39:22,964 that lives over here. 599 00:39:22,964 --> 00:39:26,840 It is equal to -- well, if I project the past code word on 600 00:39:26,840 --> 00:39:32,840 the past, I basically get the past code word again. 601 00:39:32,840 --> 00:39:35,385 Let me write it like that, but it's a little bit redundant. 602 00:39:39,950 --> 00:39:45,160 Incidental comment: I can regard this past subcode as 603 00:39:45,160 --> 00:39:51,940 either an 8 1 4 code, but what is it really? 604 00:39:51,940 --> 00:39:53,230 It doesn't live out here. 605 00:39:53,230 --> 00:39:54,860 It's support is on these four. 606 00:39:54,860 --> 00:40:03,660 It's really a 4 1 4 repetition code that lives on the past. 607 00:40:03,660 --> 00:40:06,070 It's support is the past. 608 00:40:06,070 --> 00:40:11,090 Similarly, the support here is the future, and it's 609 00:40:11,090 --> 00:40:14,710 effectively a 4 1 4 repetition code for our example. 610 00:40:17,960 --> 00:40:22,840 But formally, if we project this on the past, it's a 611 00:40:22,840 --> 00:40:24,590 one-to-one projection. 612 00:40:24,590 --> 00:40:27,050 What happens to the future part? 613 00:40:27,050 --> 00:40:30,480 This disappears, because the projection of anything in the 614 00:40:30,480 --> 00:40:36,350 future code on the past is all 0 by definition. 615 00:40:36,350 --> 00:40:39,360 Plus, it's the projection of the state 616 00:40:39,360 --> 00:40:40,480 sequence on the past. 617 00:40:40,480 --> 00:40:42,340 Could the projection of the state sequence on 618 00:40:42,340 --> 00:40:43,670 the past be all 0? 619 00:40:48,450 --> 00:40:50,000 This is actually an important point. 620 00:40:53,120 --> 00:40:55,570 Could the the projection of any linear combination of 621 00:40:55,570 --> 00:40:59,880 these generators down here be all 0 in the past? 622 00:40:59,880 --> 00:41:01,920 The answer is no by the definition 623 00:41:01,920 --> 00:41:06,050 of the future subcode. 624 00:41:06,050 --> 00:41:08,180 If I found a linear combination of this that was 625 00:41:08,180 --> 00:41:11,090 all 0 in the past, I should add that to the generators of 626 00:41:11,090 --> 00:41:12,670 the future subcode. 627 00:41:12,670 --> 00:41:16,820 So by defining this in this way, I've forced this to be 628 00:41:16,820 --> 00:41:21,260 nonzero, unless, of course, the state 629 00:41:21,260 --> 00:41:22,840 sequence itself was 0. 630 00:41:22,840 --> 00:41:25,890 The all 0 sequence is in the state code. 631 00:41:25,890 --> 00:41:28,940 So this is nonzero if S is nonzero. 632 00:41:33,570 --> 00:41:36,530 And similarly, if I project a code word on the 633 00:41:36,530 --> 00:41:38,480 future, I get -- 634 00:41:42,390 --> 00:41:44,070 this becomes all 0. 635 00:41:44,070 --> 00:41:48,350 I get the element of the future code projected on the 636 00:41:48,350 --> 00:41:53,190 future, plus the state on the future, the state projected on 637 00:41:53,190 --> 00:41:54,440 the future. 638 00:42:00,569 --> 00:42:08,910 And now I claim further that this means I can draw a 639 00:42:08,910 --> 00:42:10,230 trellis as follows. 640 00:42:16,220 --> 00:42:17,940 Start from here. 641 00:42:17,940 --> 00:42:22,860 For the all 0 sequence, I'll have a bunch of parallel paths 642 00:42:22,860 --> 00:42:27,530 going to the zero state that together add 643 00:42:27,530 --> 00:42:28,780 up to the past subcode. 644 00:42:31,610 --> 00:42:32,870 These are precise. 645 00:42:32,870 --> 00:42:37,450 One of these is going to be all 0, and these are precisely 646 00:42:37,450 --> 00:42:42,665 the ones over here that can be followed by all 0, as I've 647 00:42:42,665 --> 00:42:43,915 already claimed. 648 00:42:46,480 --> 00:42:50,910 So these are the ones that can be followed by 0, and 649 00:42:50,910 --> 00:42:53,930 similarly over here I'm going to put all the future subcode. 650 00:42:59,140 --> 00:43:04,200 I will say these correspond to the state sequence 0. 651 00:43:04,200 --> 00:43:08,450 These are the ones where if I put 0 coefficients down here, 652 00:43:08,450 --> 00:43:11,890 I just get linear combinations of the past subcode and the 653 00:43:11,890 --> 00:43:12,790 future subcode. 654 00:43:12,790 --> 00:43:14,810 I'm going to write those as going through 655 00:43:14,810 --> 00:43:16,080 all one trellis state. 656 00:43:16,080 --> 00:43:18,620 And I believe I've already made the argument that this 657 00:43:18,620 --> 00:43:25,010 state at least is legitimate, that every combination of 658 00:43:25,010 --> 00:43:27,270 something in the past subcode with something in the future 659 00:43:27,270 --> 00:43:29,360 subcode is a code word. 660 00:43:29,360 --> 00:43:34,340 This is simply just Cp plus Cf, what I wrote before. 661 00:43:34,340 --> 00:43:37,320 So I've got one state that represents all of Cp plus Cf. 662 00:43:42,080 --> 00:43:46,830 And now let's take another typical state down here. 663 00:43:46,830 --> 00:43:49,130 We'll say this state corresponds to the state 664 00:43:49,130 --> 00:43:54,970 sequence S, or S is a general state sequence. 665 00:43:54,970 --> 00:43:57,320 Think of it as being nonzero. 666 00:43:57,320 --> 00:44:00,410 And what am I going to put on that? 667 00:44:00,410 --> 00:44:04,820 I'm going to put Cp plus this state sequence 668 00:44:04,820 --> 00:44:07,200 projected on the past. 669 00:44:07,200 --> 00:44:10,900 And that way I'll get all past projections that are of this 670 00:44:10,900 --> 00:44:14,790 form for a specific S projected on the past, 671 00:44:14,790 --> 00:44:17,820 allowing this to vary through the past subcode. 672 00:44:21,600 --> 00:44:27,320 And similarly over here, I will let the set of all these 673 00:44:27,320 --> 00:44:30,630 trellis branches be Cf plus the state 674 00:44:30,630 --> 00:44:31,880 projected on the future. 675 00:44:37,220 --> 00:44:40,050 So all the things that go through this state here will 676 00:44:40,050 --> 00:44:47,800 be Cp plus Cf plus this particular state sequence. 677 00:44:47,800 --> 00:44:50,530 So for each of the state sequences, I claim I can 678 00:44:50,530 --> 00:44:52,260 define a state. 679 00:44:52,260 --> 00:44:56,810 So it's the set of all past continuations of anything that 680 00:44:56,810 --> 00:45:00,040 projects on the past as anything in the past subcode 681 00:45:00,040 --> 00:45:04,750 plus a past projection of the state sequence can be combined 682 00:45:04,750 --> 00:45:08,540 with anything in the future, Cf plus the future 683 00:45:08,540 --> 00:45:11,730 continuation of the state sequence. 684 00:45:11,730 --> 00:45:15,330 One of these is simply the past projection of the state 685 00:45:15,330 --> 00:45:17,730 sequence plus the future projection 686 00:45:17,730 --> 00:45:20,630 of the state sequence. 687 00:45:20,630 --> 00:45:25,210 And this is S, and by construction this is a code 688 00:45:25,210 --> 00:45:27,630 word in the code. 689 00:45:27,630 --> 00:45:33,540 Now by linearity I can add any past code word to this past 690 00:45:33,540 --> 00:45:35,920 projection. 691 00:45:35,920 --> 00:45:41,450 And that's a code word, so any of these elements plus Sf, 692 00:45:41,450 --> 00:45:45,790 continued by Sf, the future projection is a code word. 693 00:45:45,790 --> 00:45:49,540 See, there we're taking something generated by Cp and 694 00:45:49,540 --> 00:45:52,400 the state code. 695 00:45:52,400 --> 00:45:55,500 And similarly, anything by here comes out here. 696 00:45:55,500 --> 00:46:00,900 So I think I've left one or two details undone, but are 697 00:46:00,900 --> 00:46:05,460 you convinced that I can define a state in this way 698 00:46:05,460 --> 00:46:09,510 such that all of these pasts can be followed by all these 699 00:46:09,510 --> 00:46:11,730 futures, and they're code words? 700 00:46:11,730 --> 00:46:17,600 In fact, they correspond to this subset of 701 00:46:17,600 --> 00:46:20,550 elements of the code. 702 00:46:20,550 --> 00:46:24,430 We have examples of it up here. 703 00:46:24,430 --> 00:46:28,890 For instance, 0, 0, 1, 1. 704 00:46:28,890 --> 00:46:29,900 Or let's see. 705 00:46:29,900 --> 00:46:34,590 Let's take a past protection, 1, 1, 0, 0. 706 00:46:34,590 --> 00:46:36,650 Here's a typical state sequence, 1, 1, 707 00:46:36,650 --> 00:46:39,370 0, 0, 1, 1, 0, 0. 708 00:46:39,370 --> 00:46:46,920 And I can add to this anything in the past, so I get 0, 0, 1, 709 00:46:46,920 --> 00:46:50,740 1, 1, 1, 0, 0. 710 00:46:50,740 --> 00:46:54,490 I can add to it any of these two, anything in the future. 711 00:46:54,490 --> 00:47:01,570 So I get 1, 1, 0, 0, 0, 0, 1, 1 and 0, 0, 1, 1, 0, 0, 1, 1. 712 00:47:01,570 --> 00:47:05,260 And I claim that all four of these have the state projected 713 00:47:05,260 --> 00:47:07,730 on the past equals 1, 1, 0, 0. 714 00:47:07,730 --> 00:47:13,270 The state projected on the future is 1, 1, 0, 0. 715 00:47:13,270 --> 00:47:16,210 And that any of these pasts can be followed by any of 716 00:47:16,210 --> 00:47:16,870 these futures. 717 00:47:16,870 --> 00:47:23,450 So this is C past plus C future plus 1, 1, 718 00:47:23,450 --> 00:47:27,080 0, 0, 1, 1, 0, 0. 719 00:47:27,080 --> 00:47:28,330 That is the claim. 720 00:47:36,520 --> 00:47:41,510 And furthermore, if I go through this, dot dot dot, I'm 721 00:47:41,510 --> 00:47:44,540 going to generate everything in C. So this is the partition 722 00:47:44,540 --> 00:47:52,970 of C -- into what? 723 00:47:52,970 --> 00:48:05,520 S cosets of Cp plus Cf, subcode of C. So 724 00:48:05,520 --> 00:48:07,280 algebraically, that's what's going on. 725 00:48:09,990 --> 00:48:13,340 But having done this partition, at least with 726 00:48:13,340 --> 00:48:20,230 respect to this state space, I can create something which has 727 00:48:20,230 --> 00:48:24,960 S states the size of S, which is -- 728 00:48:24,960 --> 00:48:25,690 what is this? 729 00:48:25,690 --> 00:48:31,830 The dimension of S is the dimension of C minus the 730 00:48:31,830 --> 00:48:36,290 dimension of Cp minus the dimension of Cf. 731 00:48:41,940 --> 00:48:45,420 So I've argued that I can get a state 732 00:48:45,420 --> 00:48:50,120 space with this dimension. 733 00:48:50,120 --> 00:48:55,590 It's a linear state space, and it has a certain dimension, 734 00:48:55,590 --> 00:48:58,046 which is just the dimension of C minus the dimension of the 735 00:48:58,046 --> 00:48:59,184 past subcode minus the dimension 736 00:48:59,184 --> 00:49:00,450 of the future subcode. 737 00:49:00,450 --> 00:49:06,180 It's the number of generators that I need for S. So in the 738 00:49:06,180 --> 00:49:10,210 example, I need two generators for S, so I get a state space 739 00:49:10,210 --> 00:49:12,860 of dimension two, or size four. 740 00:49:15,730 --> 00:49:20,015 It's just a vector space over F2 in this case. 741 00:49:23,240 --> 00:49:24,400 You all with me? 742 00:49:24,400 --> 00:49:25,600 I think you are. 743 00:49:25,600 --> 00:49:28,560 I don't see any great puzzles. 744 00:49:28,560 --> 00:49:33,380 So I can get at least these few states. 745 00:49:33,380 --> 00:49:34,630 Could I get any fewer? 746 00:49:39,750 --> 00:49:43,800 Can I possibly merge any two of these states? 747 00:49:43,800 --> 00:49:50,490 Could I draw a trellis in which, say, I've mushed the 748 00:49:50,490 --> 00:49:55,938 zero state together with one of these nonzero states? 749 00:49:55,938 --> 00:50:00,610 And the answer to that is clearly no by the definition 750 00:50:00,610 --> 00:50:02,490 of the past and the future subcodes. 751 00:50:02,490 --> 00:50:08,410 These are all of the future sequences that can follow the 752 00:50:08,410 --> 00:50:10,320 all 0 past sequence. 753 00:50:10,320 --> 00:50:13,010 There can't be any more down here, so we can't possibly 754 00:50:13,010 --> 00:50:14,840 mush them together. 755 00:50:14,840 --> 00:50:21,280 And similarly, just by going through this, if you can't 756 00:50:21,280 --> 00:50:26,070 take the past part of one state and combine it with a 757 00:50:26,070 --> 00:50:31,170 future part of another state sequence, because that clearly 758 00:50:31,170 --> 00:50:33,320 is not in the code. 759 00:50:33,320 --> 00:50:39,570 And I don't have a slick proof of that in mind at the moment, 760 00:50:39,570 --> 00:50:40,900 but there is one in the notes. 761 00:50:51,770 --> 00:50:55,800 The conclusion from this is what I call 762 00:50:55,800 --> 00:50:58,150 the state space theorem. 763 00:50:58,150 --> 00:51:09,015 Given code C, any partition of the time axis, the total index 764 00:51:09,015 --> 00:51:18,060 set, into past and future, I can do this 765 00:51:18,060 --> 00:51:19,210 not just at the midpoint. 766 00:51:19,210 --> 00:51:24,610 I can do this any point along the code that I want to, any 767 00:51:24,610 --> 00:51:28,910 partition into past and future. 768 00:51:32,470 --> 00:51:33,720 The minimal -- 769 00:51:36,040 --> 00:51:49,170 we get a linear state space, in other words a vector space, 770 00:51:49,170 --> 00:51:56,590 S. And the dimension of S is equal to the dimension of C 771 00:51:56,590 --> 00:52:02,020 minus the dimension of this past subcode minus the 772 00:52:02,020 --> 00:52:03,800 dimension of the future subcode. 773 00:52:06,310 --> 00:52:09,590 So we can simply calculate what the minimal dimension of 774 00:52:09,590 --> 00:52:11,990 a state space is at any point along here. 775 00:52:16,770 --> 00:52:22,110 So for instance, at this point here, what's the minimal size 776 00:52:22,110 --> 00:52:24,930 of a state space? 777 00:52:24,930 --> 00:52:26,620 What's the past subcode here? 778 00:52:26,620 --> 00:52:31,960 What's the set of all code sequences that are all 0 in 779 00:52:31,960 --> 00:52:35,970 the future, as we have all 0's out here? 780 00:52:35,970 --> 00:52:41,090 This point, Cp, is just 0, 0. 781 00:52:41,090 --> 00:52:44,400 It just has dimension 0. 782 00:52:44,400 --> 00:52:46,040 Yeah, 0 783 00:52:46,040 --> 00:52:47,810 What's the future subcode? 784 00:52:47,810 --> 00:52:54,240 This is the subcode which has support all out here. 785 00:52:54,240 --> 00:52:58,550 This is anything that starts with 0, 0. 786 00:52:58,550 --> 00:53:07,290 This is 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1. 787 00:53:07,290 --> 00:53:09,700 Let me just write down generators. 788 00:53:09,700 --> 00:53:12,500 1, 1, 1, 1, 0, 0. 789 00:53:12,500 --> 00:53:17,590 So there is -- this is dimension Cp equals 0. 790 00:53:17,590 --> 00:53:20,920 Dimension Cf equals 2. 791 00:53:25,290 --> 00:53:35,480 And so the minimum size of the state space is 4 minus 2. 792 00:53:35,480 --> 00:53:36,730 That's again 2. 793 00:53:40,226 --> 00:53:42,975 AUDIENCE: Does this mean that [INAUDIBLE] 794 00:53:42,975 --> 00:53:46,610 each state's the number of states is the same? 795 00:53:46,610 --> 00:53:48,010 PROFESSOR: Certainly not true in general. 796 00:53:50,560 --> 00:53:54,800 Let me ask about time three. 797 00:53:54,800 --> 00:53:56,050 Can certainly do that. 798 00:53:59,650 --> 00:54:03,790 Time three, what's the past subcode? 799 00:54:03,790 --> 00:54:08,940 This is the code words that have support on the first 800 00:54:08,940 --> 00:54:10,490 three symbols. 801 00:54:10,490 --> 00:54:12,450 Again, that has to have dimension 0, because the 802 00:54:12,450 --> 00:54:15,660 minimum weight of this code is 4. 803 00:54:15,660 --> 00:54:18,695 So there can't be anything that just looks like 1,1,1 and 804 00:54:18,695 --> 00:54:22,390 then five 0's 805 00:54:22,390 --> 00:54:26,670 What is the future subcode for that? 806 00:54:26,670 --> 00:54:29,610 Now the only generator of the future subcode 807 00:54:29,610 --> 00:54:32,580 is 0, 1, 1, 1, 1. 808 00:54:32,580 --> 00:54:35,830 We have to go this far, and then it's what futures can we 809 00:54:35,830 --> 00:54:37,350 get from here? 810 00:54:37,350 --> 00:54:41,720 There are only two words that are in the future. 811 00:54:41,720 --> 00:54:45,300 So in that case we have dimension past is 0. 812 00:54:45,300 --> 00:54:47,480 Dimension future is 1. 813 00:54:47,480 --> 00:54:53,720 So if we drew a state space here, we would have to have 814 00:54:53,720 --> 00:54:54,970 eight states. 815 00:54:57,850 --> 00:55:02,230 In fact, I can do that if I just draw a little state in 816 00:55:02,230 --> 00:55:06,460 the middle of each of these lines. 817 00:55:09,700 --> 00:55:13,050 I get now a trellis picture which has an explicit state at 818 00:55:13,050 --> 00:55:18,820 this third time here, but it has eight states. 819 00:55:18,820 --> 00:55:21,550 Oo ugly, so I've masked that. 820 00:55:21,550 --> 00:55:23,205 I've suppressed that by this nice 821 00:55:23,205 --> 00:55:24,950 little four-state trellis. 822 00:55:24,950 --> 00:55:28,720 But yeah, if you insisted on drawing a state space at time 823 00:55:28,720 --> 00:55:30,220 three, it would have eight states in it. 824 00:55:36,400 --> 00:55:39,900 We'll get back to that when I get to this turn-the-crank 825 00:55:39,900 --> 00:55:45,760 procedure, if I do, of getting to minimal trellises. 826 00:55:45,760 --> 00:55:46,610 So what have we done? 827 00:55:46,610 --> 00:55:53,360 We've established that there is a uniquely defined minimal 828 00:55:53,360 --> 00:55:56,270 state space size for any partition 829 00:55:56,270 --> 00:55:57,590 between past and future. 830 00:55:57,590 --> 00:55:59,800 In other words, if there's any time where you want to make a 831 00:55:59,800 --> 00:56:03,960 cut in the time axis between past and future, you can 832 00:56:03,960 --> 00:56:06,800 define what the minimal state space size is. 833 00:56:10,320 --> 00:56:19,070 And now you might ask the question, can we 834 00:56:19,070 --> 00:56:33,290 simultaneously achieve minimal state spaces at all times? 835 00:56:40,200 --> 00:56:46,750 In other words, can we draw a single trellis which gets the 836 00:56:46,750 --> 00:56:48,850 minimal state space at each time? 837 00:56:54,400 --> 00:56:56,040 We haven't proved that there is. 838 00:56:56,040 --> 00:56:59,770 It might be that if you push in the state space at one 839 00:56:59,770 --> 00:57:02,470 time, it forces the state space to balloon out at 840 00:57:02,470 --> 00:57:05,200 another time. 841 00:57:05,200 --> 00:57:07,590 And in fact, for nonlinear codes, that is 842 00:57:07,590 --> 00:57:08,710 typically what happens. 843 00:57:08,710 --> 00:57:11,900 But for linear codes, a minor miracle occurs. 844 00:57:11,900 --> 00:57:13,760 And the answer is yes. 845 00:57:17,710 --> 00:57:25,190 So how am I going to prove that? 846 00:57:25,190 --> 00:57:30,040 I'm going to prove it via this very handy tool, which is also 847 00:57:30,040 --> 00:57:37,230 a construction tool which is called a trellis-oriented, or 848 00:57:37,230 --> 00:57:44,085 a little bit more formally, a minimum span generator matrix. 849 00:57:56,350 --> 00:58:07,830 The idea here is, given a generator matrix, reduce to 850 00:58:07,830 --> 00:58:10,940 trellis-oriented form. 851 00:58:10,940 --> 00:58:16,600 We're going to prove that this is unique, or effectively 852 00:58:16,600 --> 00:58:27,690 unique, and specifies minimal trellis at all times in a 853 00:58:27,690 --> 00:58:28,940 certain way. 854 00:58:36,235 --> 00:58:37,510 Let's start again. 855 00:58:51,190 --> 00:58:55,320 So loosely, what is a 856 00:58:55,320 --> 00:58:58,522 trellis-oriented generator matrix? 857 00:58:58,522 --> 00:59:08,090 Well, if we have a generator, say 0, 0, 1, 1, 1, 1, 0, 0, 858 00:59:08,090 --> 00:59:18,470 its span is this interval from its starting time to its 859 00:59:18,470 --> 00:59:19,160 ending time. 860 00:59:19,160 --> 00:59:21,200 It's this. 861 00:59:21,200 --> 00:59:27,110 We say it starts at time 3, it ends at time 6, the span is 862 00:59:27,110 --> 00:59:30,510 the interval 3 to 6 in this case. 863 00:59:33,060 --> 00:59:38,190 So we think of this as not being active before time 3. 864 00:59:38,190 --> 00:59:42,980 At time 3 we get the first interesting thing happens. 865 00:59:42,980 --> 00:59:45,640 A 1 comes up, but then goes through something. 866 00:59:45,640 --> 00:59:47,820 These don't have to all be ones. 867 00:59:47,820 --> 00:59:49,880 It could be 1, 1, 0, 1. 868 00:59:49,880 --> 00:59:53,330 Then there's a last 1 at some point, and then it quiets 869 00:59:53,330 --> 00:59:55,083 down, and it's dead again. 870 00:59:55,083 --> 00:59:56,333 AUDIENCE: [INAUDIBLE] 871 00:59:59,250 --> 01:00:01,220 PROFESSOR: Yes. 872 01:00:01,220 --> 01:00:03,310 Except I'm not sure. 873 01:00:03,310 --> 01:00:06,160 Would you say the support of this was 3, 4, and 6, or would 874 01:00:06,160 --> 01:00:10,620 you say it was the interval from 3 to 6? 875 01:00:10,620 --> 01:00:11,150 I don't know. 876 01:00:11,150 --> 01:00:13,490 What's the definition of support? 877 01:00:13,490 --> 01:00:15,160 AUDIENCE: I don't know it. 878 01:00:15,160 --> 01:00:17,200 PROFESSOR: I'm not sure I know. 879 01:00:17,200 --> 01:00:20,180 It's probably been defined both ways in different 880 01:00:20,180 --> 01:00:21,410 literatures. 881 01:00:21,410 --> 01:00:26,780 So I'll call it a span, but it's roughly the support of 882 01:00:26,780 --> 01:00:28,030 the interesting part. 883 01:00:32,400 --> 01:00:37,340 So that's the definition of span. 884 01:00:37,340 --> 01:00:41,750 And what a trellis-oriented general matrix is, a 885 01:00:41,750 --> 01:00:52,400 minimal-span matrix, meaning all generators have a short a 886 01:00:52,400 --> 01:00:53,650 span as possible. 887 01:00:56,980 --> 01:00:58,615 I'll just say, are as short as possible. 888 01:01:03,353 --> 01:01:04,690 This is a lecture. 889 01:01:04,690 --> 01:01:07,690 I can be loose. 890 01:01:07,690 --> 01:01:10,210 Now it's not even clear that's well-defined yet. 891 01:01:12,760 --> 01:01:21,430 But let me again give you an example, and show you how to 892 01:01:21,430 --> 01:01:27,480 find a trellis-oriented generating 893 01:01:27,480 --> 01:01:29,115 matrix just by example. 894 01:01:33,670 --> 01:01:40,010 Let me take our example here, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 895 01:01:40,010 --> 01:01:46,890 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1. 896 01:01:46,890 --> 01:01:49,690 So there is a generator matrix that's not in 897 01:01:49,690 --> 01:01:52,350 trellis-oriented form. 898 01:01:52,350 --> 01:01:57,190 In particular, I can see that by adding this to this, I can 899 01:01:57,190 --> 01:02:00,090 make this generator have shorter span. 900 01:02:00,090 --> 01:02:04,450 I can replace it by one, which is four 0's and then four 1's, 901 01:02:04,450 --> 01:02:06,090 which we've already found useful in 902 01:02:06,090 --> 01:02:09,340 constructing a trellis. 903 01:02:09,340 --> 01:02:12,920 But let me proceed through it more systematically. 904 01:02:12,920 --> 01:02:16,050 How can I make -- 905 01:02:16,050 --> 01:02:19,380 this generator generates all words in the code. 906 01:02:19,380 --> 01:02:22,200 So how can I find a set of linearly independent 907 01:02:22,200 --> 01:02:25,670 generators that has shorter span? 908 01:02:25,670 --> 01:02:28,770 There's the sort of greedy way of proceeding. 909 01:02:28,770 --> 01:02:31,380 Let's take the first two generators. 910 01:02:31,380 --> 01:02:34,390 Can I combine these to find anything that has a shorter 911 01:02:34,390 --> 01:02:39,560 span, that I can replace one of the generators with? 912 01:02:39,560 --> 01:02:40,546 Anybody? 913 01:02:40,546 --> 01:02:42,120 AUDIENCE: No. 914 01:02:42,120 --> 01:02:43,515 PROFESSOR: No, I can't. 915 01:02:43,515 --> 01:02:45,190 That's not the answer I expected to hear. 916 01:02:48,142 --> 01:02:49,392 AUDIENCE: [INAUDIBLE] 917 01:02:52,570 --> 01:02:55,220 PROFESSOR: Let's identify what the spans of these are. 918 01:02:55,220 --> 01:02:56,500 That has a span of four. 919 01:02:56,500 --> 01:02:57,990 That has a span of six. 920 01:02:57,990 --> 01:02:59,840 That has a span of seven. 921 01:02:59,840 --> 01:03:01,470 This has a span of eight. 922 01:03:01,470 --> 01:03:04,388 What I'm trying to do is to reduce that. 923 01:03:04,388 --> 01:03:07,150 AUDIENCE: [INAUDIBLE] 924 01:03:07,150 --> 01:03:09,470 PROFESSOR: So if I add the first two, that seems like a 925 01:03:09,470 --> 01:03:16,920 good thing to do, because then I get 0, 0, 1, 1, and I've 926 01:03:16,920 --> 01:03:19,050 reduced the span of that generator. 927 01:03:25,830 --> 01:03:30,075 Any way I can combine these two to reduce the span? 928 01:03:34,830 --> 01:03:38,060 Obviously not, because the span of this is completely 929 01:03:38,060 --> 01:03:40,270 included in the span of this. 930 01:03:40,270 --> 01:03:43,760 If I add these two together, then I'll get something which 931 01:03:43,760 --> 01:03:47,280 still starts here and ends here. 932 01:03:47,280 --> 01:03:50,860 Why was I able to reduce the span here? 933 01:03:50,860 --> 01:03:54,970 Because the spans had the same starting time, so I got a 934 01:03:54,970 --> 01:03:59,075 cancellation of the first bit. 935 01:03:59,075 --> 01:04:00,470 I got a 0 up here. 936 01:04:00,470 --> 01:04:03,540 That's what I wanted. 937 01:04:03,540 --> 01:04:05,860 So that's the key to proceeding. 938 01:04:05,860 --> 01:04:09,280 If I see any place, any two that have the same starting 939 01:04:09,280 --> 01:04:15,750 time, then I can add them together, and the result will 940 01:04:15,750 --> 01:04:17,610 be something that has a shorter span than at 941 01:04:17,610 --> 01:04:18,860 least one of them. 942 01:04:22,280 --> 01:04:23,840 So let me do that. 943 01:04:23,840 --> 01:04:27,730 I see two here that have the same starting time, and so I 944 01:04:27,730 --> 01:04:31,820 add that to that, and I get 0 1 0 1. 945 01:04:31,820 --> 01:04:35,520 So I've got the span down to here. 946 01:04:35,520 --> 01:04:37,020 I can do that down here. 947 01:04:39,640 --> 01:04:46,140 And I get this, which is a very nice reduction in span. 948 01:04:46,140 --> 01:04:49,880 Can I go any further? 949 01:04:49,880 --> 01:04:52,890 I can clearly look for the same thing in ending times, if 950 01:04:52,890 --> 01:04:56,960 I could find two ending times that were the same. 951 01:04:56,960 --> 01:05:00,640 I could do the same trick, add those two generators together, 952 01:05:00,640 --> 01:05:03,960 and the result would be something that was shorter 953 01:05:03,960 --> 01:05:07,680 than at least one of the component generators. 954 01:05:07,680 --> 01:05:10,330 But here are my starting times, here, 955 01:05:10,330 --> 01:05:12,680 here, here, and here. 956 01:05:12,680 --> 01:05:13,780 They're all different. 957 01:05:13,780 --> 01:05:19,640 So I have no possibility for combining two and getting a 958 01:05:19,640 --> 01:05:21,240 better starting time. 959 01:05:21,240 --> 01:05:24,550 Similarly, the ending times are here, here, and here. 960 01:05:24,550 --> 01:05:26,590 They're all different. 961 01:05:26,590 --> 01:05:27,840 So I'm done. 962 01:05:33,910 --> 01:05:35,190 So, that's how you find a 963 01:05:35,190 --> 01:05:39,340 trellis-oriented generator matrix. 964 01:05:39,340 --> 01:05:46,450 And there's a closely associated theorem, which is 965 01:05:46,450 --> 01:05:57,040 that a generator matrix g is trellis-oriented if, and only 966 01:05:57,040 --> 01:06:11,522 if, all starting times differ, all ending terms differ. 967 01:06:24,320 --> 01:06:24,800 Yeah? 968 01:06:24,800 --> 01:06:27,596 AUDIENCE: What's the definition of all generators 969 01:06:27,596 --> 01:06:31,410 as short as possible? 970 01:06:31,410 --> 01:06:34,150 PROFESSOR: I'm now making it. 971 01:06:34,150 --> 01:06:35,400 This is really just motivation. 972 01:06:40,280 --> 01:06:43,130 So this can be my actual definition of a 973 01:06:43,130 --> 01:06:45,030 trellis-oriented generator matrix, which 974 01:06:45,030 --> 01:06:46,910 I've proved this theorem. 975 01:06:46,910 --> 01:06:49,550 It's a matrix in which all the starting times are different, 976 01:06:49,550 --> 01:06:51,310 and all the ending times are different. 977 01:06:51,310 --> 01:06:54,240 And what I've achieved is that all generators 978 01:06:54,240 --> 01:06:55,827 are as short as possible. 979 01:07:01,940 --> 01:07:05,285 How would I go about proving the theorem? 980 01:07:09,160 --> 01:07:15,170 Suppose I have a matrix that's not in this form. 981 01:07:15,170 --> 01:07:17,405 And I've already proved that I can shorten the generators. 982 01:07:20,120 --> 01:07:23,140 So if it's not in this form, it's not trellis-oriented. 983 01:07:23,140 --> 01:07:26,840 The generators are not as short as possible. 984 01:07:26,840 --> 01:07:28,810 So all I need to do is prove the other side. 985 01:07:28,810 --> 01:07:30,510 Suppose it is in this form. 986 01:07:33,170 --> 01:07:38,510 Then could I possibly find shorter generators? 987 01:07:41,850 --> 01:07:48,810 And the proof of that is basically the following lemma, 988 01:07:48,810 --> 01:08:02,863 that the subcode C, take any interval kk prime. 989 01:08:10,520 --> 01:08:19,355 So given a trellis-oriented generating matrix, namely one 990 01:08:19,355 --> 01:08:23,100 that satisfies these conditions, then the subcode 991 01:08:23,100 --> 01:08:26,640 consisting of all the code words who have support on this 992 01:08:26,640 --> 01:08:37,910 interval is generated by the gj in G that have 993 01:08:37,910 --> 01:08:44,700 support in kk prime. 994 01:08:49,410 --> 01:08:53,090 And that's kind of intuitive and obvious. 995 01:08:53,090 --> 01:08:56,479 On the one hand, it's clear that any linear combination of 996 01:08:56,479 --> 01:09:01,979 these generators is a code word in the subcode. 997 01:09:01,979 --> 01:09:04,689 Obviously they all have support in this interval. 998 01:09:04,689 --> 01:09:06,899 I can't generate anything that has support outside that 999 01:09:06,899 --> 01:09:10,244 interval by linear combinations. 1000 01:09:14,319 --> 01:09:21,590 So the only question is, can I possibly get anything that's 1001 01:09:21,590 --> 01:09:28,210 in the subcode by combining with a generator that is 1002 01:09:28,210 --> 01:09:30,300 outside this interval? 1003 01:09:30,300 --> 01:09:39,510 And again, it's clear that say I take kk prime to be here, if 1004 01:09:39,510 --> 01:09:43,569 all the starting times are different, and all the ending 1005 01:09:43,569 --> 01:09:47,200 times are different, and I start to add generators that 1006 01:09:47,200 --> 01:09:50,850 are outside here, I can't possibly get cancellation of 1007 01:09:50,850 --> 01:09:54,420 the starting time, or I can't possibly get cancellation of 1008 01:09:54,420 --> 01:09:55,690 the ending time. 1009 01:09:55,690 --> 01:09:59,470 So I'm going to wind up with a code word which has support 1010 01:09:59,470 --> 01:10:00,720 outside the interval. 1011 01:10:03,160 --> 01:10:06,812 So, it really is a simple lemma. 1012 01:10:06,812 --> 01:10:07,804 AUDIENCE: [INAUDIBLE] 1013 01:10:07,804 --> 01:10:10,794 have the all 0 code word? 1014 01:10:10,794 --> 01:10:11,690 PROFESSOR: Excuse me? 1015 01:10:11,690 --> 01:10:14,430 AUDIENCE: The all 0 word will be part of the subcode. 1016 01:10:14,430 --> 01:10:16,683 PROFESSOR: The all 0 word is always part of the subcode. 1017 01:10:16,683 --> 01:10:18,036 AUDIENCE: [INAUDIBLE] 1018 01:10:18,036 --> 01:10:20,300 who has something outside this interval. 1019 01:10:20,300 --> 01:10:23,585 Then when you add that with the all 0 one, you would get 1020 01:10:23,585 --> 01:10:26,258 something that does not belong to the subcode, [INAUDIBLE] 1021 01:10:29,180 --> 01:10:32,350 PROFESSOR: Right, but I have to ask if 1022 01:10:32,350 --> 01:10:35,880 there's any code word. 1023 01:10:35,880 --> 01:10:38,480 So I have to consider all the code words, really. 1024 01:10:38,480 --> 01:10:44,120 But I'm saying if I have a code word C that is sum of the 1025 01:10:44,120 --> 01:10:48,010 generators, and I have a nonzero coefficient here on 1026 01:10:48,010 --> 01:10:53,390 any generator that has starting time outside this 1027 01:10:53,390 --> 01:10:56,920 interval or ending time outside this interval, then I 1028 01:10:56,920 --> 01:10:59,320 can't possibly get cancellation of 1029 01:10:59,320 --> 01:11:00,460 that nonzero thing. 1030 01:11:00,460 --> 01:11:06,090 And so if it really requires this to be nonzero, and then 1031 01:11:06,090 --> 01:11:07,570 I'm going to say that I get a code word 1032 01:11:07,570 --> 01:11:08,820 that's not in this subcode. 1033 01:11:11,340 --> 01:11:13,690 So that's very quick, intuitive 1034 01:11:13,690 --> 01:11:15,380 sketch of that proof. 1035 01:11:15,380 --> 01:11:24,750 Again, for writing it out, see the notes. 1036 01:11:24,750 --> 01:11:34,510 But this lemma then proves this theorem, because it now 1037 01:11:34,510 --> 01:11:35,760 says I can't -- 1038 01:11:40,260 --> 01:11:43,000 if all the starting times are different, and all the ending 1039 01:11:43,000 --> 01:11:47,210 times are different, then I can't possibly get shorter 1040 01:11:47,210 --> 01:11:49,043 support for any of the generators. 1041 01:11:52,700 --> 01:11:55,510 And I don't think so in a totally convincing way, but 1042 01:11:55,510 --> 01:11:59,650 that basically is the point here. 1043 01:11:59,650 --> 01:12:00,902 Yeah? 1044 01:12:00,902 --> 01:12:02,152 AUDIENCE: [INAUDIBLE] 1045 01:12:06,030 --> 01:12:07,750 PROFESSOR: Well, now I'm defining this. 1046 01:12:07,750 --> 01:12:11,610 So, meaning that all the starting times are different, 1047 01:12:11,610 --> 01:12:12,540 and all the ending times are different. 1048 01:12:12,540 --> 01:12:15,130 From now on that's going to be my definition of a 1049 01:12:15,130 --> 01:12:17,350 trellis-oriented generator matrix. 1050 01:12:17,350 --> 01:12:20,647 So, saying that this property implies this property. 1051 01:12:25,790 --> 01:12:30,350 But now this lemma has an interesting consequence. 1052 01:12:30,350 --> 01:12:54,190 This implies for any past and future, Cp is generated by the 1053 01:12:54,190 --> 01:13:02,820 generators in a trellis-oriented generating 1054 01:13:02,820 --> 01:13:09,090 matrix with support in p. 1055 01:13:09,090 --> 01:13:16,880 And similarly, Cf is generated by the generators 1056 01:13:16,880 --> 01:13:20,690 with support in f. 1057 01:13:20,690 --> 01:13:34,510 So for any p and f, I get a picture that looks like this. 1058 01:13:34,510 --> 01:13:37,350 I can draw now this boundary anywhere I want, and if the 1059 01:13:37,350 --> 01:13:40,700 matrix was trellis-oriented, then I'm going to get a 1060 01:13:40,700 --> 01:13:43,830 certain subset of generators that live in the past, and 1061 01:13:43,830 --> 01:13:45,880 they'll be the generators in fact of Cp. 1062 01:13:49,120 --> 01:13:52,190 The dimension of them will simply be the number of 1063 01:13:52,190 --> 01:13:53,980 generators that live in the past for that 1064 01:13:53,980 --> 01:13:55,750 definition of the past. 1065 01:13:55,750 --> 01:13:56,980 There will be some other set that live 1066 01:13:56,980 --> 01:13:58,240 entirely in the future. 1067 01:13:58,240 --> 01:14:00,930 Those will be the ones that generate the future. 1068 01:14:00,930 --> 01:14:03,110 And the ones that are neither wholly in the past or in the 1069 01:14:03,110 --> 01:14:07,140 future will generate the state code. 1070 01:14:07,140 --> 01:14:14,750 So that says I can read off the dimensions of the state 1071 01:14:14,750 --> 01:14:18,010 spaces just by looking at this single trellis-oriented 1072 01:14:18,010 --> 01:14:20,910 generator matrix. 1073 01:14:20,910 --> 01:14:23,940 So now let me very quickly go through the calculation that I 1074 01:14:23,940 --> 01:14:27,880 went through a little bit laboriously over here. 1075 01:14:27,880 --> 01:14:31,970 Suppose I make the whole thing the future. 1076 01:14:31,970 --> 01:14:35,960 In other words, I draw a cut before time 0. 1077 01:14:35,960 --> 01:14:42,340 Then the state space has dimension 0. 1078 01:14:42,340 --> 01:14:44,290 All four generators live on the future. 1079 01:14:44,290 --> 01:14:50,440 If I make the cut here, the state space has dimension 1, 1080 01:14:50,440 --> 01:14:53,530 because three of the generators live on the future. 1081 01:14:53,530 --> 01:14:56,630 One is active at time 1. 1082 01:14:56,630 --> 01:15:02,050 So that means I'm going to get a two-state trellis at time 1. 1083 01:15:02,050 --> 01:15:06,250 If I make the cut here between past and future, I see two 1084 01:15:06,250 --> 01:15:10,670 generators live on the future and are inactive, haven't 1085 01:15:10,670 --> 01:15:12,210 started yet. 1086 01:15:12,210 --> 01:15:15,870 But two of the generators have already started. 1087 01:15:15,870 --> 01:15:19,380 At this time we have three active generators. 1088 01:15:19,380 --> 01:15:22,210 Only one is still completely on the future. 1089 01:15:22,210 --> 01:15:28,170 At this time in the middle I have two generators which -- 1090 01:15:28,170 --> 01:15:32,660 past subcode, future subcode, and two that aren't. 1091 01:15:32,660 --> 01:15:35,470 So I get this form that we looked at over there. 1092 01:15:35,470 --> 01:15:38,610 And I get a state space here generated by these two 1093 01:15:38,610 --> 01:15:41,600 generators of size two. 1094 01:15:41,600 --> 01:15:47,950 So just by looking at this I go right through the picture, 1095 01:15:47,950 --> 01:15:51,740 and I find the size of the state space at each time. 1096 01:15:55,990 --> 01:16:00,310 So this is an algebraic way of finding the minimal state. 1097 01:16:00,310 --> 01:16:07,860 So now I can draw from this a trellis just by writing down 1098 01:16:07,860 --> 01:16:09,110 these state spaces. 1099 01:16:13,796 --> 01:16:17,040 Let me leave these trellises here. 1100 01:16:17,040 --> 01:16:19,720 Let me draw now a full trellis, or an 1101 01:16:19,720 --> 01:16:21,000 eight-section trellis. 1102 01:16:21,000 --> 01:16:25,840 I'm only going to put one bit on each trellis. 1103 01:16:25,840 --> 01:16:29,300 Here is the starting state, dimension 0. 1104 01:16:29,300 --> 01:16:30,190 One state. 1105 01:16:30,190 --> 01:16:34,806 From that I can go out to 0 or 1. 1106 01:16:34,806 --> 01:16:38,750 I won't label the state spaces. 1107 01:16:38,750 --> 01:16:40,956 Next time, I'm still just branching. 1108 01:16:47,400 --> 01:16:49,750 Actually, to make it look like that, I'm going to want to 1109 01:16:49,750 --> 01:16:56,990 come down here, 0, 1, 1, 1, 1, 0. 1110 01:17:02,610 --> 01:17:06,040 I basically just have four states at time two 1111 01:17:06,040 --> 01:17:08,880 corresponding to whatever the first two bits are. 1112 01:17:08,880 --> 01:17:11,570 They all go to different states, because all these have 1113 01:17:11,570 --> 01:17:14,500 possible future continuations. 1114 01:17:14,500 --> 01:17:16,680 Then I have eight states at this 1115 01:17:16,680 --> 01:17:19,215 time, still just branching. 1116 01:17:36,000 --> 01:17:40,966 Sorry, this is going to take some time to do correctly. 1117 01:17:48,250 --> 01:17:53,075 Add this time, then I come in here. 1118 01:17:53,075 --> 01:17:54,325 0, 1. 1119 01:18:17,880 --> 01:18:23,620 Anyway, and it's symmetrical on the other side. 1120 01:18:23,620 --> 01:18:27,580 So that's what a full trellis will look like. 1121 01:18:27,580 --> 01:18:30,620 And how did I do this trellis in principle? 1122 01:18:30,620 --> 01:18:35,130 I wrote down all 16 code words. 1123 01:18:35,130 --> 01:18:37,810 I wrote down what linear combinations they were up 1124 01:18:37,810 --> 01:18:41,970 here, and I therefore found what states they went through 1125 01:18:41,970 --> 01:18:43,350 at each time. 1126 01:18:43,350 --> 01:18:48,020 And then I just drew the graph that goes through that 1127 01:18:48,020 --> 01:18:50,340 describes those trajectories. 1128 01:18:50,340 --> 01:18:52,550 So I wrote down all the code words. 1129 01:18:52,550 --> 01:18:56,060 I wrote down all these state codes at all these times. 1130 01:18:56,060 --> 01:18:57,530 I could have written them down in any order. 1131 01:18:57,530 --> 01:19:01,640 And then I just connected the dots according 1132 01:19:01,640 --> 01:19:03,620 to which state -- 1133 01:19:03,620 --> 01:19:07,850 I make this calculation as to which state they go through. 1134 01:19:15,590 --> 01:19:17,530 That's a good place to stop. 1135 01:19:17,530 --> 01:19:18,780 Let me summarize. 1136 01:19:21,800 --> 01:19:28,590 We now have a method, given a generator matrix for a binary 1137 01:19:28,590 --> 01:19:32,740 linear block code, of reducing that generator matrix to 1138 01:19:32,740 --> 01:19:36,160 trellis-oriented form. 1139 01:19:36,160 --> 01:19:39,840 Just by inspection of the trellis-oriented generator 1140 01:19:39,840 --> 01:19:43,740 matrix, we can determine what the state space dimensions are 1141 01:19:43,740 --> 01:19:48,580 at each instant of time, at each possible state cut. 1142 01:19:51,660 --> 01:19:58,110 And we can then draw a trellis which achieves that minimal 1143 01:19:58,110 --> 01:20:02,910 state space dimension, or minimal state space size, for 1144 01:20:02,910 --> 01:20:04,400 every moment of time simultaneously. 1145 01:20:07,290 --> 01:20:13,580 And I assert that this trellis is the minimal trellis in 1146 01:20:13,580 --> 01:20:16,360 every respect, whether you're trying to minimize state 1147 01:20:16,360 --> 01:20:21,350 complexity, or if you're using the Viterbi algorithm. 1148 01:20:21,350 --> 01:20:23,960 Really what you want to minimize is the number of 1149 01:20:23,960 --> 01:20:27,370 branches, but there's a calculation about branch 1150 01:20:27,370 --> 01:20:31,650 spaces in the notes that shows that this also achieves the 1151 01:20:31,650 --> 01:20:35,190 minimal branch space size at every time, regardless of how 1152 01:20:35,190 --> 01:20:38,260 you draw the trellis. 1153 01:20:38,260 --> 01:20:42,960 More elaborate calculations with a more refined notion of 1154 01:20:42,960 --> 01:20:45,260 Viterbi algorithm complexities still come up 1155 01:20:45,260 --> 01:20:47,160 with the same result. 1156 01:20:47,160 --> 01:20:51,730 The result is there is essentially a unique minimal 1157 01:20:51,730 --> 01:20:54,450 trellis, where it's minimal in every way. 1158 01:20:57,775 --> 01:21:05,950 And this comes from the linear, or more broadly, the 1159 01:21:05,950 --> 01:21:09,510 group property of the code. 1160 01:21:09,510 --> 01:21:14,600 So for any linear group code, there's a well-defined, 1161 01:21:14,600 --> 01:21:21,670 essentially unique up to re-labeling, minimal trellis 1162 01:21:21,670 --> 01:21:26,430 that achieves the minimum of whatever complexity quantity 1163 01:21:26,430 --> 01:21:27,340 you want to define. 1164 01:21:27,340 --> 01:21:35,170 So in this sense, the trellis complexity of a group or 1165 01:21:35,170 --> 01:21:38,090 linear code is very well-defined. 1166 01:21:38,090 --> 01:21:43,150 It's defined by this minimal trellis, or by its parameters, 1167 01:21:43,150 --> 01:21:46,170 and we can use that as a measure of the complexity of 1168 01:21:46,170 --> 01:21:49,540 the corresponding group code. 1169 01:21:49,540 --> 01:21:52,790 And from an engineering point of view, once we've got this 1170 01:21:52,790 --> 01:21:55,620 minimal trellis, we can use the Viterbi algorithm to do 1171 01:21:55,620 --> 01:21:58,540 maximum likelihood decoding. 1172 01:21:58,540 --> 01:22:01,650 Just in the same way we did for convolutional codes, sweep 1173 01:22:01,650 --> 01:22:04,242 from the starting node to the ending node. 1174 01:22:06,830 --> 01:22:10,510 Digest that, and we'll come back and talk about it again. 1175 01:22:10,510 --> 01:22:12,470 Do a few more details on Monday, but 1176 01:22:12,470 --> 01:22:13,720 that's the basic idea.