WEBVTT 00:00:01.120 --> 00:00:02.900 PROFESSOR: --we'll start off with a little review, so you 00:00:02.900 --> 00:00:06.090 might chance it. 00:00:06.090 --> 00:00:08.170 We're in the middle of the chapter nine on 00:00:08.170 --> 00:00:09.750 convolutional codes. 00:00:09.750 --> 00:00:12.840 Today, I intend to hand out problem set six. 00:00:12.840 --> 00:00:15.750 I believe Ashish will have it when he gets here. 00:00:15.750 --> 00:00:19.430 And we'll hand it out at the end. 00:00:19.430 --> 00:00:25.010 Last time, I introduced you to a lot of things about 00:00:25.010 --> 00:00:28.770 convolutional codes and their encoders. 00:00:28.770 --> 00:00:32.560 I stressed that a convolutional encoder had two 00:00:32.560 --> 00:00:33.910 distinct characters. 00:00:33.910 --> 00:00:37.140 One is it's a linear time-invariant system. 00:00:37.140 --> 00:00:41.170 So we can use the linearity and time-invariance and 00:00:41.170 --> 00:00:45.500 algebraic analysis of convolutional codes. 00:00:45.500 --> 00:00:49.270 Secondly, I stressed that's it's a finite state system, 00:00:49.270 --> 00:00:51.800 because it's a finite memory system where each memory 00:00:51.800 --> 00:00:55.630 element has a finite number of possible values, over F2 or 00:00:55.630 --> 00:00:57.800 indeed over any finite field. 00:00:57.800 --> 00:01:00.210 And therefore, we can use that for 00:01:00.210 --> 00:01:01.920 different kinds of analysis. 00:01:01.920 --> 00:01:05.200 We use that, for instance, in conjunction with linear 00:01:05.200 --> 00:01:08.990 property to find the minimum distance to a particular code. 00:01:08.990 --> 00:01:12.580 And we'll see that that's the key to getting an efficient 00:01:12.580 --> 00:01:16.300 optimal maximum likelihood decoder -- 00:01:16.300 --> 00:01:18.820 is the fact that we only have a finite number of states in 00:01:18.820 --> 00:01:20.400 this system. 00:01:20.400 --> 00:01:27.970 All right, so along the way I kind of went all over the map, 00:01:27.970 --> 00:01:33.050 as far as the introducing various algebraic quantities. 00:01:33.050 --> 00:01:36.970 I focused particularly on formal Laurent series. 00:01:36.970 --> 00:01:40.940 And I want to just give you a little table to make sure you 00:01:40.940 --> 00:01:43.820 have fixed in your mind what all these different kinds of 00:01:43.820 --> 00:01:46.380 sequences we're talking about are. 00:01:46.380 --> 00:01:49.870 On the left side, I have Laurent sequences. 00:01:49.870 --> 00:01:51.930 These are semi-infinite sequences that must 00:01:51.930 --> 00:01:53.140 start at some time. 00:01:53.140 --> 00:01:57.410 Their delay is not equal to minus infinity, which would be 00:01:57.410 --> 00:01:59.830 if it were all 1's forever. 00:01:59.830 --> 00:02:01.740 But their delay is some finite time. 00:02:01.740 --> 00:02:04.020 It could be negative, could be positive. 00:02:04.020 --> 00:02:05.750 That's their starting time. 00:02:05.750 --> 00:02:07.930 And then they could go on, possibly, 00:02:07.930 --> 00:02:09.299 infinitely into the future. 00:02:12.160 --> 00:02:18.500 And then, among those, we can look particularly at the ones 00:02:18.500 --> 00:02:22.660 that start at time 0 or later, whose delay is 0, or whose 00:02:22.660 --> 00:02:24.720 delay is non-negative. 00:02:24.720 --> 00:02:26.980 These are called the formal power series, and you've 00:02:26.980 --> 00:02:29.250 probably encountered them earlier. 00:02:29.250 --> 00:02:31.550 They're a lot more frequently encountered 00:02:31.550 --> 00:02:36.170 in mathematics algebra. 00:02:36.170 --> 00:02:39.270 The rational sequences are simply those that can be 00:02:39.270 --> 00:02:43.140 written in a very finite way as a numerator polynomial over 00:02:43.140 --> 00:02:44.650 a denominator polynomial. 00:02:44.650 --> 00:02:50.240 Because the formal Laurent series form a field, every 00:02:50.240 --> 00:02:54.390 element is invertible, and, in particular, every polynomial 00:02:54.390 --> 00:02:59.520 has an inverse, which, unless the polynomial is a monomial, 00:02:59.520 --> 00:03:02.480 is going to run on for an infinite length of time. 00:03:02.480 --> 00:03:05.680 But nonetheless, there is a subset, a countable subset of 00:03:05.680 --> 00:03:08.650 these sequences that we can write in this way. 00:03:08.650 --> 00:03:11.800 And we call these the rational sequences, and they're 00:03:11.800 --> 00:03:16.580 analogous to the rational numbers in the real field. 00:03:16.580 --> 00:03:19.780 And their particular characteristic, as we showed, 00:03:19.780 --> 00:03:21.940 is that they're eventually periodic. 00:03:21.940 --> 00:03:26.460 And it's easy to see that these, too, form a field. 00:03:26.460 --> 00:03:29.375 Obviously, the inverse of N of d over D of d is D 00:03:29.375 --> 00:03:30.560 of d over N of d. 00:03:30.560 --> 00:03:35.220 Provided that n of d is not equal to 0, the denominator is 00:03:35.220 --> 00:03:40.580 always required to be non-zero for a rational sequence. 00:03:40.580 --> 00:03:44.450 So that's a very nice subset. 00:03:44.450 --> 00:03:49.220 It may or may not have operational meaning. 00:03:49.220 --> 00:03:51.090 Certainly, when we're talking about 00:03:51.090 --> 00:03:52.650 realizations, it has meaning. 00:03:52.650 --> 00:03:57.950 And then the finite sequences, by eventually periodic, I will 00:03:57.950 --> 00:04:00.350 from now on include the finite case. 00:04:00.350 --> 00:04:05.590 If we have a finite sequence, then its end is all 0's, and 00:04:05.590 --> 00:04:08.090 that's certainly a periodic sequence. 00:04:08.090 --> 00:04:09.830 So I would say a finite sequence is 00:04:09.830 --> 00:04:12.420 also eventually periodic. 00:04:12.420 --> 00:04:15.100 But among the rational sequences we have, 00:04:15.100 --> 00:04:18.540 particularly the finite sequence, these do not form a 00:04:18.540 --> 00:04:22.520 field, because the non-monomial polynomials 00:04:22.520 --> 00:04:23.570 cannot be inverted. 00:04:23.570 --> 00:04:26.420 So these are merely a ring. 00:04:26.420 --> 00:04:28.520 However, D has an inverse. 00:04:28.520 --> 00:04:32.280 Over here on this side, D minus 1 has an element of all 00:04:32.280 --> 00:04:33.090 these things. 00:04:33.090 --> 00:04:35.880 D has an inverse. 00:04:35.880 --> 00:04:39.820 Over here, on the causal side, these are the causal analogs 00:04:39.820 --> 00:04:42.540 to all of these. 00:04:42.540 --> 00:04:46.220 In the case of causal rational, we don't have 00:04:46.220 --> 00:04:47.940 particularly good notation. 00:04:47.940 --> 00:04:51.220 We just say these are the ones that are both causal, formal 00:04:51.220 --> 00:04:54.060 power series and rational. 00:04:54.060 --> 00:04:57.600 But they are important, because we've seen that it's 00:04:57.600 --> 00:05:01.700 this kind of sequence that you can get as the impulse 00:05:01.700 --> 00:05:08.620 response of a time-invariant linear system that is 00:05:08.620 --> 00:05:09.740 realizable. 00:05:09.740 --> 00:05:11.020 Realizable has two parts. 00:05:11.020 --> 00:05:15.070 One is the impulse response must start at time 0 or later, 00:05:15.070 --> 00:05:17.520 by physical causality. 00:05:17.520 --> 00:05:22.740 Two, the impulse response must be eventually periodic, 00:05:22.740 --> 00:05:26.370 therefore rational, if it's going to be realizable with a 00:05:26.370 --> 00:05:27.850 finite number of memory elements. 00:05:27.850 --> 00:05:31.200 So we include that in our definition of realizability. 00:05:31.200 --> 00:05:34.300 So these are sometimes called the realizable sequences, or 00:05:34.300 --> 00:05:37.030 the realizable D transforms. 00:05:37.030 --> 00:05:40.730 And these, of course, the causal finite sequences are 00:05:40.730 --> 00:05:41.700 the polynomials. 00:05:41.700 --> 00:05:46.260 These include the polynomials, which are trivially easy to 00:05:46.260 --> 00:05:53.460 realize with shift registers, as we saw in our example. 00:05:53.460 --> 00:05:57.450 So that's just a review of all these quantities. 00:05:57.450 --> 00:06:00.720 Does anyone feel the need for further elaboration or 00:06:00.720 --> 00:06:01.390 explanation? 00:06:01.390 --> 00:06:04.290 Or can we go forward with this? 00:06:04.290 --> 00:06:08.070 We tend to use all of them as we go along, so I want you to 00:06:08.070 --> 00:06:10.000 have them clearly in mind. 00:06:10.000 --> 00:06:10.230 Yeah. 00:06:10.230 --> 00:06:12.580 AUDIENCE: Why was the [INAUDIBLE] 00:06:12.580 --> 00:06:13.050 uncountable? 00:06:13.050 --> 00:06:14.930 What was the [INAUDIBLE]? 00:06:14.930 --> 00:06:18.020 PROFESSOR: Why is that uncountable? 00:06:18.020 --> 00:06:20.980 Try to count it. 00:06:20.980 --> 00:06:23.610 I could put it in one-to-one correspondence with a set of 00:06:23.610 --> 00:06:26.290 all binary expansions of the real numbers, and that's an 00:06:26.290 --> 00:06:27.540 uncountable set. 00:06:35.610 --> 00:06:42.930 Some of these side comments are just to trigger analogies 00:06:42.930 --> 00:06:48.010 in your mind, maybe make it more reasonable, what we're 00:06:48.010 --> 00:06:50.400 talking about. 00:06:50.400 --> 00:06:53.130 OK, so next. 00:06:53.130 --> 00:07:02.150 Just again, to continue a quick review, we talked about 00:07:02.150 --> 00:07:07.895 realizable linear time-invariant systems. 00:07:10.900 --> 00:07:16.805 First, we talked about single input, single output. 00:07:16.805 --> 00:07:20.470 I'm imposing an order that didn't necessarily exist. 00:07:23.650 --> 00:07:28.150 In any case, these are characterized by impulse 00:07:28.150 --> 00:07:44.900 response, g of D, which, for realizability, that implies 00:07:44.900 --> 00:07:48.385 that g of D is causal rational. 00:07:53.300 --> 00:07:57.840 The fact that it's rational means that we can write g of D 00:07:57.840 --> 00:08:04.730 as n of D over d of D. And the fact that it's causal, well, 00:08:04.730 --> 00:08:08.460 we would always reduce this to lowest terms in the first 00:08:08.460 --> 00:08:11.540 place, so there'd be no point carrying along common factors 00:08:11.540 --> 00:08:14.980 in the numerator and denominator. 00:08:14.980 --> 00:08:18.270 By inserting enough powers of D, we can make sure that both 00:08:18.270 --> 00:08:20.250 of these are polynomials. 00:08:20.250 --> 00:08:25.810 So if n of D, d of D are -- 00:08:25.810 --> 00:08:28.880 we can ensure that they're actually polynomials, not just 00:08:28.880 --> 00:08:31.230 finite sequences. 00:08:31.230 --> 00:08:35.720 And if it's going to be causal, that means that 00:08:35.720 --> 00:08:37.020 we can always -- 00:08:37.020 --> 00:08:42.909 if we remove common factors of D, that means that the 00:08:42.909 --> 00:08:45.590 numerator might still have factors of D in it, but the 00:08:45.590 --> 00:08:48.610 denominator can't have factors of D in it. 00:08:48.610 --> 00:08:50.390 You see that? 00:08:50.390 --> 00:08:54.360 Because, once we shift all these over as close to 0 as we 00:08:54.360 --> 00:08:58.110 can, the denominator has to be closer to 0. 00:08:58.110 --> 00:09:03.420 If D0 equals 1, that means 1 over d of D is something that 00:09:03.420 --> 00:09:05.890 starts at times 0. 00:09:05.890 --> 00:09:07.560 We multiply times the numerator. 00:09:07.560 --> 00:09:10.240 In order to have the whole thing be causal, the numerator 00:09:10.240 --> 00:09:13.890 has to be causal, because a point I didn't mention, when 00:09:13.890 --> 00:09:16.750 you multiply formal Laurent sequences, their delays add. 00:09:19.290 --> 00:09:23.400 If a of D and b of D are Laurent, then the delay of a 00:09:23.400 --> 00:09:29.410 of D times b of D, defined as a convolution, is going to be 00:09:29.410 --> 00:09:31.740 the sum of the delays of each of the component sequence. 00:09:31.740 --> 00:09:34.250 You just take the lower order term on both and that's what 00:09:34.250 --> 00:09:37.020 determines the lower order term on the convolution, just 00:09:37.020 --> 00:09:40.780 like in polynomials and analogous to degrees. 00:09:40.780 --> 00:09:46.350 So from this, we find that we can always write a causal 00:09:46.350 --> 00:09:50.300 rational sequence in this form, if it isn't 0. 00:09:50.300 --> 00:09:54.630 If it is 0, we just take n of D to be 0, and d of D to be 1. 00:09:54.630 --> 00:09:56.930 And that satisfies this form, too. 00:09:59.950 --> 00:10:03.670 I guess I didn't even have to make that comment. 00:10:03.670 --> 00:10:04.920 We're already in that form. 00:10:07.650 --> 00:10:11.700 You can convince yourself this is a unique way to write a 00:10:11.700 --> 00:10:15.090 causal rational sequence. 00:10:15.090 --> 00:10:18.460 And every other way is just multiplying the numerator and 00:10:18.460 --> 00:10:28.030 the denominator by some common polynomial or finite sequence. 00:10:28.030 --> 00:10:36.100 So now we had a little theorem that says that, if g of D is 00:10:36.100 --> 00:10:45.190 causal rational, and therefore equal to n of D over d of D in 00:10:45.190 --> 00:10:57.242 this form, then realizable with how many memory elements? 00:10:57.242 --> 00:10:59.620 Does anyone remember? 00:10:59.620 --> 00:11:04.165 nu equals the maximum of the degrees. 00:11:10.080 --> 00:11:14.130 And I debated whether to actually do this in class, and 00:11:14.130 --> 00:11:17.960 I think it's worth taking a few minutes to actually do 00:11:17.960 --> 00:11:18.740 this in class. 00:11:18.740 --> 00:11:23.050 It'll also appear as the first homework problem. 00:11:23.050 --> 00:11:26.090 So this'll be a little head start on the homework. 00:11:26.090 --> 00:11:31.290 So anybody have any ideas how we can do this realization? 00:11:31.290 --> 00:11:36.230 What we want is a circuit with an input, u of D. Eventually, 00:11:36.230 --> 00:11:43.220 an output, y of D equals u of D times n of D over d of D. 00:11:43.220 --> 00:11:46.410 For these are both polynomials and D0 equals 1. 00:11:51.630 --> 00:11:55.210 A lot of you have probably seen this in discrete-time 00:11:55.210 --> 00:11:57.050 linear filters. 00:11:57.050 --> 00:11:58.720 It's going to be just the same technique. 00:12:01.290 --> 00:12:03.380 But if we don't have volunteers, I don't want to 00:12:03.380 --> 00:12:05.570 waste time. 00:12:05.570 --> 00:12:06.770 The key is to use -- 00:12:06.770 --> 00:12:09.190 we're of course going to need feedback in general. 00:12:09.190 --> 00:12:13.350 To have a d of D, in general, will not be 1. 00:12:13.350 --> 00:12:17.600 If it's not 1, then we're going to get an infinite 00:12:17.600 --> 00:12:18.650 impulse response. 00:12:18.650 --> 00:12:22.100 To get an infinite impulse response, we need feedback. 00:12:22.100 --> 00:12:25.020 So we're going to need some feedback term in here. 00:12:25.020 --> 00:12:28.720 And here is the motivating idea of the construction. 00:12:28.720 --> 00:12:32.540 We want to create something here, v of D, which is 00:12:32.540 --> 00:12:38.030 basically equal to u of D over d of D. If we can do that, 00:12:38.030 --> 00:12:40.330 we'll be done. 00:12:40.330 --> 00:12:43.370 I'll show you why. 00:12:43.370 --> 00:12:45.740 So we created a different sequence, which is u of D 00:12:45.740 --> 00:12:49.965 divided by d of D. And then we pass that sequence through a 00:12:49.965 --> 00:12:51.215 shift register. 00:12:58.660 --> 00:13:00.950 And what do we get at the various stages 00:13:00.950 --> 00:13:03.080 of the shift register? 00:13:03.080 --> 00:13:08.807 Here we would get v of D delayed by one time unit, v of 00:13:08.807 --> 00:13:13.540 D delayed by two time units, and so forth, up to -- 00:13:13.540 --> 00:13:17.210 let's make this nu. 00:13:17.210 --> 00:13:22.600 I want to realize it in nu memory elements, where nu is 00:13:22.600 --> 00:13:25.750 the maximum degree of either of these defining polynomials. 00:13:25.750 --> 00:13:35.700 So finally, out here, I get d nu of D, v of D. 00:13:35.700 --> 00:13:38.164 Now, what's the trick? 00:13:38.164 --> 00:13:40.060 AUDIENCE: [INAUDIBLE]. 00:13:40.060 --> 00:13:41.560 PROFESSOR: Excuse me? 00:13:41.560 --> 00:13:42.810 AUDIENCE: Because [UNINTELLIGIBLE PHRASE]. 00:13:48.890 --> 00:13:49.850 PROFESSOR: That is the idea. 00:13:49.850 --> 00:13:54.420 What do we get if we take out these various lines, which are 00:13:54.420 --> 00:13:58.530 v of D, through d to the nu v of D, and we make a linear 00:13:58.530 --> 00:14:01.640 combination of them? 00:14:01.640 --> 00:14:04.280 We're going to do this twice, actually, once to form the 00:14:04.280 --> 00:14:13.420 feedback, and once to form the output. 00:14:13.420 --> 00:14:17.660 For the feedback, let me not take this into the linear 00:14:17.660 --> 00:14:18.910 combination. 00:14:21.450 --> 00:14:25.960 Otherwise, I'd get a loop without a delay element in 00:14:25.960 --> 00:14:27.430 that, and that tends not to be well-defined. 00:14:30.060 --> 00:14:32.560 So I want to take a linear combination. 00:14:32.560 --> 00:14:35.300 In general, by taking a linear combination of these things, I 00:14:35.300 --> 00:14:42.860 can get the multiple of v of D times any polynomial degree, 00:14:42.860 --> 00:14:49.260 nu or less, just by taking the coefficients of that 00:14:49.260 --> 00:14:52.816 polynomial as my linear combination. 00:14:52.816 --> 00:14:56.155 Do you see that? 00:14:56.155 --> 00:14:57.590 I'm not seeing people's -- 00:14:57.590 --> 00:14:58.840 AUDIENCE: [INAUDIBLE]. 00:15:01.270 --> 00:15:04.035 PROFESSOR: Excuse me? 00:15:04.035 --> 00:15:05.410 AUDIENCE: In the linear combination 00:15:05.410 --> 00:15:06.730 [UNINTELLIGIBLE PHRASE] 00:15:06.730 --> 00:15:09.920 you also take the one with zero derivative. 00:15:09.920 --> 00:15:16.290 PROFESSOR: You're getting ahead of me, which is fine, 00:15:16.290 --> 00:15:20.180 but I'll ask for that comment in just a second. 00:15:20.180 --> 00:15:22.640 Let me first create the output. 00:15:22.640 --> 00:15:26.870 Suppose I had v of D equals u of D over d of D. Then to get 00:15:26.870 --> 00:15:30.440 the output, I simply want a linear combination, which will 00:15:30.440 --> 00:15:40.485 give me n of D times u of D over d of D. And that would be 00:15:40.485 --> 00:15:41.420 the correct output. 00:15:41.420 --> 00:15:44.170 And since n of D is a polynomial degree less than 00:15:44.170 --> 00:15:49.630 nu, I can do that by simply taking n nu times this, n nu 00:15:49.630 --> 00:15:54.270 minus 1 times that, n2 times that, n1 times that, and 0 00:15:54.270 --> 00:15:55.450 times that. 00:15:55.450 --> 00:15:59.940 And that will give me what I want as an output, if I'm 00:15:59.940 --> 00:16:02.510 successful in getting v of D of this form here. 00:16:05.950 --> 00:16:10.580 So let me try a similar trick up here. 00:16:10.580 --> 00:16:16.760 The trick is, I force d of D to start off with a 1, to have 00:16:16.760 --> 00:16:18.830 D0 equal 1. 00:16:18.830 --> 00:16:21.320 So what I'm going to create up here is the linear 00:16:21.320 --> 00:16:29.680 combination, d of D minus 1 times v of D. 00:16:29.680 --> 00:16:34.050 First of all, let's verify that I can do that. 00:16:34.050 --> 00:16:35.560 d of D minus 1 is what? 00:16:35.560 --> 00:16:36.900 It's a polynomial. 00:16:36.900 --> 00:16:42.755 Its degree is the same as the degree of d of D. Now, if d of 00:16:42.755 --> 00:16:45.190 D is equal to 1, if the denominator is just 1, this 00:16:45.190 --> 00:16:46.610 whole thing falls out. 00:16:46.610 --> 00:16:48.620 I don't need anything coming in here. 00:16:48.620 --> 00:16:53.050 d of D is 1. v of D is equal to u of D, so that's the 00:16:53.050 --> 00:16:55.850 polynomial case where I don't need feedback. 00:16:55.850 --> 00:16:58.870 So let's assume d of D is not equal 1, therefore it's some 00:16:58.870 --> 00:17:05.000 polynomial of degree nu, degree of d of D, which is 00:17:05.000 --> 00:17:10.790 less than or equal to nu, and so is this. 00:17:10.790 --> 00:17:15.740 But by subtracting out the 1, I have no constant term. 00:17:15.740 --> 00:17:18.440 So it's going to be a multiple of D. That's another way of 00:17:18.440 --> 00:17:20.180 saying that. 00:17:20.180 --> 00:17:22.599 It's going to have no constant terms, so I only need to form 00:17:22.599 --> 00:17:25.230 a linear combination of these terms. 00:17:25.230 --> 00:17:26.480 So I can do it. 00:17:29.486 --> 00:17:33.940 Now, if I do that, let's just solve this equation. 00:17:33.940 --> 00:17:37.580 I create x of D. This is supposed to be a plus. 00:17:37.580 --> 00:17:40.890 This is supposed to be a minus, to work over any field. 00:17:40.890 --> 00:17:44.820 This trick works over real or complex, or what you'd like. 00:17:44.820 --> 00:17:58.260 x of D is u of d minus d of D minus 1, times v of d, times x 00:17:58.260 --> 00:18:01.830 of D. Maybe I should just call this v of d. 00:18:01.830 --> 00:18:08.600 Now I'm going to solve for v of D. 00:18:08.600 --> 00:18:13.390 So I actually have a v of D on both sides of the equation. 00:18:13.390 --> 00:18:22.280 This results in u of D equal v of D times d of D. And 00:18:22.280 --> 00:18:25.230 dividing both sides by d of D, I see that I succeeded in 00:18:25.230 --> 00:18:26.810 getting what I wanted. 00:18:26.810 --> 00:18:32.240 So this is a typical trick of realizing a rational impulse 00:18:32.240 --> 00:18:37.570 response by including a feedback loop and negative 00:18:37.570 --> 00:18:39.360 feedback in your system. 00:18:42.950 --> 00:18:45.490 This is done in the notes. 00:18:45.490 --> 00:18:47.510 I'm surprised if you haven't seen this before. 00:18:47.510 --> 00:18:49.610 Maybe it's that people in communications don't take 00:18:49.610 --> 00:18:52.300 circuits courses, or they don't take digital signal 00:18:52.300 --> 00:18:53.630 processing courses. 00:18:53.630 --> 00:18:55.906 Most people look puzzled. 00:18:55.906 --> 00:18:58.340 In any case, can you agree that, formally, I've shown 00:18:58.340 --> 00:19:00.500 that we get what we want here? 00:19:00.500 --> 00:19:09.270 So I've given a way of realizing, given a causal 00:19:09.270 --> 00:19:13.330 rational function, where nu is defined is the maximum degree 00:19:13.330 --> 00:19:15.510 of these two polynomials. 00:19:15.510 --> 00:19:18.960 There's a realization with nu memory elements. 00:19:18.960 --> 00:19:21.590 A little bit more work, we could prove that this is the 00:19:21.590 --> 00:19:23.370 minimum possible number of memory 00:19:23.370 --> 00:19:27.600 elements for this system. 00:19:27.600 --> 00:19:29.375 That gets into linear system theory. 00:19:29.375 --> 00:19:30.625 AUDIENCE: [INAUDIBLE]. 00:19:36.020 --> 00:19:40.120 PROFESSOR: I mean, what I want is down here, I want the sum 00:19:40.120 --> 00:19:51.510 of n_i times d to the i, v of D. So the linear coefficients 00:19:51.510 --> 00:19:52.760 are going to be these n_i. 00:19:58.150 --> 00:20:02.640 If I drew it out, I have n0 here, n1 here. 00:20:02.640 --> 00:20:06.240 So it's scalar linear combination, over F2. 00:20:13.216 --> 00:20:15.000 And what is that? 00:20:15.000 --> 00:20:16.820 v of D is common. 00:20:16.820 --> 00:20:21.720 That's just n of D times v of D. Same trick up here. 00:20:24.660 --> 00:20:27.970 Is that what was puzzling everybody? 00:20:27.970 --> 00:20:28.540 Think about it. 00:20:28.540 --> 00:20:31.920 What is a linear combination? 00:20:31.920 --> 00:20:33.170 Something that looks like that. 00:20:38.880 --> 00:20:46.885 The second part, let's now talk about a 00:20:46.885 --> 00:20:48.135 convolutional encoder. 00:20:55.350 --> 00:21:00.990 And we're just going to talk about rate 1/n in this course. 00:21:00.990 --> 00:21:02.945 So this is a single input. 00:21:07.350 --> 00:21:13.485 n output linear time invariant system. 00:21:16.560 --> 00:21:21.550 That's my definition, I guess, of what I mean by a 00:21:21.550 --> 00:21:22.890 convolutional encoder. 00:21:22.890 --> 00:21:27.445 And when I say linear time-invariant, I also want it 00:21:27.445 --> 00:21:31.760 to be a realizable, in the two senses that its impulse 00:21:31.760 --> 00:21:33.380 response is causal rational. 00:21:35.960 --> 00:21:39.292 So [INAUDIBLE] 00:21:39.292 --> 00:21:40.610 down. 00:21:40.610 --> 00:21:55.550 Now it's characterized by n-tuple of impulse responses. 00:22:01.790 --> 00:22:11.300 So it's going to be sum g of D equal to g1 of D, up to gn of 00:22:11.300 --> 00:22:19.350 D, where clearly all these have to be causal and rational 00:22:19.350 --> 00:22:21.010 in order for it to be realizable. 00:22:21.010 --> 00:22:23.740 If any one of them was not causal or not rational, then 00:22:23.740 --> 00:22:26.110 that individual impulse response wouldn't be 00:22:26.110 --> 00:22:30.080 realizable and the whole thing wouldn't be realizable. 00:22:30.080 --> 00:22:35.330 So where we have the gj of D are all causal rational. 00:22:43.160 --> 00:22:48.760 And now, again, I'm going to put this into a standard form. 00:22:48.760 --> 00:22:56.480 In general, this is going to be n1 of D over d1 of D, so 00:22:56.480 --> 00:23:02.730 forth, up to n_n of D over d_n of D. So I'm going to have 00:23:02.730 --> 00:23:08.070 different denominators for each of the numerators. 00:23:08.070 --> 00:23:16.360 But I can always put it into the form, n1 prime of D over a 00:23:16.360 --> 00:23:25.495 common numerator, d to D, and up to n_n prime of D over d of 00:23:25.495 --> 00:23:29.730 D. Let d of D be the least common multiple 00:23:29.730 --> 00:23:31.890 of all these d_i's. 00:23:31.890 --> 00:23:33.450 These are all polynomials. 00:23:33.450 --> 00:23:35.650 Least common multiple is well-defined from our 00:23:35.650 --> 00:23:37.630 discussion of factorization. 00:23:37.630 --> 00:23:40.630 So I can always put in the least common multiple here, 00:23:40.630 --> 00:23:44.410 and then whatever d1 lacks out of the least common multiple, 00:23:44.410 --> 00:23:46.760 I multiply top and bottom by both of that. 00:23:46.760 --> 00:23:50.300 I have the same rational function, now with a common 00:23:50.300 --> 00:23:52.530 denominator. 00:23:52.530 --> 00:23:56.670 So I'm always going to put it in that standard form. 00:23:56.670 --> 00:24:13.520 And then I will say this is always realizable, with nu 00:24:13.520 --> 00:24:17.180 equal now the max of any of the degrees 00:24:17.180 --> 00:24:18.310 that appears in here. 00:24:18.310 --> 00:24:24.870 So let me just abbreviate that by degree of the numerators, 00:24:24.870 --> 00:24:28.370 which I'll just write as vector n prime of D, or the 00:24:28.370 --> 00:24:31.505 degree of the denominator d, of the common denominator d of 00:24:31.505 --> 00:24:36.540 D. Now, that's very easy to see. 00:24:39.350 --> 00:24:40.600 Why is that? 00:24:43.300 --> 00:24:45.970 Can certainly realize the first one here 00:24:45.970 --> 00:24:47.220 just by doing that. 00:24:50.300 --> 00:24:52.045 Yes? 00:24:52.045 --> 00:24:56.540 If I want to just generate the first output, I build a 00:24:56.540 --> 00:24:58.985 circuit like this, and I don't need more of 00:24:58.985 --> 00:25:01.212 the new memory elements. 00:25:01.212 --> 00:25:02.990 There might even be some redundancy in that. 00:25:05.980 --> 00:25:09.095 All right, so how would I then realize the second? 00:25:15.550 --> 00:25:16.400 Is it just me? 00:25:16.400 --> 00:25:17.680 Nobody is volunteering anything. 00:25:23.400 --> 00:25:24.460 I've realized this. 00:25:24.460 --> 00:25:29.260 Now I'd like to realize a second output. 00:25:29.260 --> 00:25:31.440 Now I want to realize this. 00:25:31.440 --> 00:25:36.760 Make this n1 of D. I want to realize n2 of D over d of D, 00:25:36.760 --> 00:25:39.151 times u of D. 00:25:39.151 --> 00:25:40.401 AUDIENCE: [INAUDIBLE]. 00:25:43.960 --> 00:25:44.842 PROFESSOR: Thank you so much. 00:25:44.842 --> 00:25:46.165 AUDIENCE: Just take out [UNINTELLIGIBLE]. 00:25:46.165 --> 00:25:47.240 PROFESSOR: All right. 00:25:47.240 --> 00:25:51.870 So all I need is the second linear combination. 00:25:51.870 --> 00:25:57.090 So let me just make this n linear combinations. 00:25:57.090 --> 00:26:00.640 Then we're going to have n outputs, 00:26:00.640 --> 00:26:02.140 an n-tuple of outputs. 00:26:02.140 --> 00:26:04.440 Then I can form each one as a linear 00:26:04.440 --> 00:26:05.990 combination of those up there. 00:26:11.150 --> 00:26:13.140 I really like more of you to be speaking up. 00:26:13.140 --> 00:26:13.920 Yes, thank you. 00:26:13.920 --> 00:26:15.342 AUDIENCE: [INAUDIBLE]. 00:26:15.342 --> 00:26:16.290 PROFESSOR: Excuse me? 00:26:16.290 --> 00:26:18.186 AUDIENCE: There's a second combination -- 00:26:18.186 --> 00:26:21.030 I mean the inquiry should come [UNINTELLIGIBLE]. 00:26:21.030 --> 00:26:24.680 PROFESSOR: Yeah, but again, what I want to realize, I have 00:26:24.680 --> 00:26:27.520 all the delays of v of D up here. 00:26:27.520 --> 00:26:32.750 What I want to realize is n2 of D times v of D, which is 00:26:32.750 --> 00:26:34.340 equal to that. 00:26:34.340 --> 00:26:37.200 n2 of D is the polynomial of degree less 00:26:37.200 --> 00:26:38.640 than or equal to nu. 00:26:38.640 --> 00:26:39.420 So I can do it. 00:26:39.420 --> 00:26:42.760 AUDIENCE: Yeah, but [UNINTELLIGIBLE PHRASE] 00:26:42.760 --> 00:26:44.630 parallel to that n1 D over d. 00:26:44.630 --> 00:26:45.110 The -- 00:26:45.110 --> 00:26:46.070 [INTERPOSING VOICES] 00:26:46.070 --> 00:26:48.490 PROFESSOR: Yeah, OK. 00:26:48.490 --> 00:26:50.800 I'm confusing you because I put all these here. 00:26:50.800 --> 00:26:54.530 I actually wrote out the linear combination. 00:26:54.530 --> 00:26:58.160 To say what I just said, I really need to go back to my 00:26:58.160 --> 00:27:01.570 original form, forget these multipliers, do the 00:27:01.570 --> 00:27:02.660 multiplications in here. 00:27:02.660 --> 00:27:06.110 That's what I mean by the linear combinations. 00:27:06.110 --> 00:27:07.990 Now I'm OK. 00:27:07.990 --> 00:27:10.080 The inputs are just the shifts. 00:27:10.080 --> 00:27:12.289 Make a linear combination of them, that's the output. 00:27:17.450 --> 00:27:20.600 So easy proof. 00:27:20.600 --> 00:27:24.090 What's significantly harder to prove in this case is you 00:27:24.090 --> 00:27:25.410 can't do any better than. 00:27:25.410 --> 00:27:29.890 There aren't any realizations with fewer than nu, where nu 00:27:29.890 --> 00:27:33.970 is computed by first reducing to standard form and then 00:27:33.970 --> 00:27:35.600 evaluating this. 00:27:35.600 --> 00:27:38.550 That's the best you can do. 00:27:38.550 --> 00:27:42.390 But we aren't going to go into minimal system realizations, 00:27:42.390 --> 00:27:44.330 linear system realizations in this course. 00:27:44.330 --> 00:27:45.667 I'll just insert it. 00:27:48.190 --> 00:27:53.200 So this is how we can always build a convolutional encoder, 00:27:53.200 --> 00:27:55.780 whether it has feedback or not. 00:27:55.780 --> 00:28:00.360 And so whatever we come up with as nu, this'll basically 00:28:00.360 --> 00:28:03.400 determine the state complexity of our decoder. 00:28:03.400 --> 00:28:08.670 The state space is dimension nu, is finite dimension. 00:28:08.670 --> 00:28:13.490 And because we're over a finite field, the actual 00:28:13.490 --> 00:28:16.920 number of states is only 2 to the nu. 00:28:16.920 --> 00:28:20.085 Still can't get more than 2 to the nu states for 00:28:20.085 --> 00:28:22.310 that system up there. 00:28:22.310 --> 00:28:25.120 So we're still in finite state world. 00:28:25.120 --> 00:28:30.940 So in particular, it's finite state realization. 00:28:30.940 --> 00:28:33.860 And again, this is if and only if. 00:28:33.860 --> 00:28:36.120 If we have if and only if, we have an n-tuple of causal 00:28:36.120 --> 00:28:40.790 rational, or even a matrix of causal rationals. 00:28:40.790 --> 00:28:43.030 We can make a realization like this. 00:28:46.470 --> 00:28:48.575 So that's convolutional encoders. 00:28:48.575 --> 00:28:52.630 At least write 1/n over F2. 00:28:52.630 --> 00:28:57.010 And now the next step was what's a convolutional code. 00:29:02.510 --> 00:29:08.330 And a convolutional code, we defined as, given a 00:29:08.330 --> 00:29:13.630 convolutional encoder, which is just the set of impulse 00:29:13.630 --> 00:29:19.330 response, given g of D, the corresponding convolutional 00:29:19.330 --> 00:29:25.180 code is just u of D, which is single sequence times this 00:29:25.180 --> 00:29:30.410 n-tuple of sequences, as u of D ranges through all the 00:29:30.410 --> 00:29:32.093 formal Laurent sequences. 00:29:37.550 --> 00:29:41.090 Sequences that start at some time in the all-zero state and 00:29:41.090 --> 00:29:45.060 then continue perhaps forever. 00:29:45.060 --> 00:29:50.660 It's very quick to show that C, its 00:29:50.660 --> 00:29:53.200 properties is it's linear. 00:29:53.200 --> 00:29:57.440 Obviously, it's a vector space over F2. 00:29:57.440 --> 00:29:59.990 Multiplication by scalars is trivial. 00:29:59.990 --> 00:30:04.300 Addition is trivial, so it's linear. 00:30:04.300 --> 00:30:12.700 And its time-invariant, meaning the shift of any code 00:30:12.700 --> 00:30:14.250 sequence is another code sequence. 00:30:14.250 --> 00:30:22.040 If I have one particular code sequence, and I want to see if 00:30:22.040 --> 00:30:25.220 the shift of that is in the code, well, that code sequence 00:30:25.220 --> 00:30:28.610 must have been generated by some use, so if I just shift 00:30:28.610 --> 00:30:32.370 the u by the amount of time that I want to shift the 00:30:32.370 --> 00:30:36.800 output, y, then I'll get the shifted output. 00:30:36.800 --> 00:30:43.980 So it's easy to show that D to the k of C is simply equal to 00:30:43.980 --> 00:30:51.510 C for any integer k. 00:30:51.510 --> 00:30:55.080 Actually, it just suffices to show that the single time unit 00:30:55.080 --> 00:30:57.717 shift is in the code, and that implies all the rest of this. 00:31:01.390 --> 00:31:01.890 Yes? 00:31:01.890 --> 00:31:03.140 AUDIENCE: [UNINTELLIGIBLE]. 00:31:05.740 --> 00:31:10.276 We would want at least one of the g_i of D's to start at 0. 00:31:10.276 --> 00:31:13.380 Otherwise, they don't cancel out, do they? 00:31:13.380 --> 00:31:14.630 PROFESSOR: Right. 00:31:17.530 --> 00:31:21.340 So what you're saying is we don't want d to be a common 00:31:21.340 --> 00:31:24.890 factor of all the n of D's. 00:31:24.890 --> 00:31:25.680 And that's true. 00:31:25.680 --> 00:31:28.780 In fact, we don't want to have any common factors of 00:31:28.780 --> 00:31:30.030 all the n of D's. 00:31:33.270 --> 00:31:34.520 AUDIENCE: [INAUDIBLE]. 00:31:37.540 --> 00:31:40.180 PROFESSOR: The u we've defined, so it can 00:31:40.180 --> 00:31:41.290 start at any time. 00:31:41.290 --> 00:31:43.790 AUDIENCE: [INAUDIBLE]. 00:31:43.790 --> 00:31:47.190 PROFESSOR: So notice that these have separate algebraic 00:31:47.190 --> 00:31:48.660 characters. 00:31:48.660 --> 00:31:51.800 This is a Laurent sequence. 00:31:51.800 --> 00:31:54.660 These are called causal rational sequences. 00:31:54.660 --> 00:31:58.040 So they have different restrictions on them, but they 00:31:58.040 --> 00:32:00.240 play together that way. 00:32:04.940 --> 00:32:09.680 So where you're getting to is this idea of code equivalence. 00:32:12.850 --> 00:32:14.210 I can just keep going. 00:32:17.660 --> 00:32:33.600 So code equivalence, let's abbreviate it this way. 00:32:33.600 --> 00:32:40.520 g of D and some other n-tuple, g prime of D, 00:32:40.520 --> 00:32:42.040 generate the same code. 00:32:42.040 --> 00:32:45.865 We say that g of D generates C up here. 00:32:49.480 --> 00:32:57.790 So two different n-tuples generate the same code, C, 00:32:57.790 --> 00:32:59.405 which is our notion of equivalence. 00:33:03.690 --> 00:33:10.180 And we more or less proved last time it is true that g of 00:33:10.180 --> 00:33:14.120 D is some -- 00:33:14.120 --> 00:33:22.270 this is just a single sequence multiple of g prime of D. So 00:33:22.270 --> 00:33:24.780 we have to have a of D not equal to 0. 00:33:24.780 --> 00:33:29.810 Otherwise, I think there could be any sequence there. 00:33:29.810 --> 00:33:34.830 So what you were saying is that, if all of these n's had 00:33:34.830 --> 00:33:39.205 a common factor of d, then why not just shift them all over? 00:33:39.205 --> 00:33:41.800 And that would be the same as multiplying -- 00:33:41.800 --> 00:33:44.060 suppose we had a g prime of D where they all had d's. 00:33:44.060 --> 00:33:46.800 Suppose we just multiply them with d minus 1. 00:33:46.800 --> 00:33:49.370 We're still going to get the same code, but we're going to 00:33:49.370 --> 00:33:52.580 reduce the degrees of all the numerators. 00:33:52.580 --> 00:33:56.970 And therefore, we're likely to get a simpler realization. 00:33:56.970 --> 00:33:59.290 And that's the track we're going down right now. 00:33:59.290 --> 00:34:00.540 AUDIENCE: [INAUDIBLE]. 00:34:05.090 --> 00:34:07.250 PROFESSOR: This would only shift the n's. 00:34:10.650 --> 00:34:15.570 I'm assuming that all the n of D's are multiples of d. 00:34:15.570 --> 00:34:17.590 AUDIENCE: [INAUDIBLE] 00:34:17.590 --> 00:34:18.670 the root of d, right? 00:34:18.670 --> 00:34:23.900 So if you can reduce the d of D you can't save anything. 00:34:23.900 --> 00:34:26.370 PROFESSOR: You might not save anything if you've got a 00:34:26.370 --> 00:34:29.245 denominator, d of D, that has a larger degree. 00:34:29.245 --> 00:34:31.250 AUDIENCE: And that's always the case. 00:34:31.250 --> 00:34:33.739 PROFESSOR: No, it's not always the case. 00:34:33.739 --> 00:34:35.250 In the general case, either one of 00:34:35.250 --> 00:34:39.530 these things can dominate. 00:34:39.530 --> 00:34:44.469 In that picture we had, if we have a low degree, d of D, 00:34:44.469 --> 00:34:47.510 then it's only picking off of these first elements up here. 00:34:47.510 --> 00:34:50.532 AUDIENCE: But that definition states [UNINTELLIGIBLE] 00:34:50.532 --> 00:34:51.696 is the maximum d and also the [UNINTELLIGIBLE]. 00:34:51.696 --> 00:34:52.610 PROFESSOR: Right. 00:34:52.610 --> 00:34:56.870 So we could have a big one here making the outputs, and a 00:34:56.870 --> 00:34:58.850 small one there going back. 00:34:58.850 --> 00:35:01.780 In fact, we could have d of D equal to 1. 00:35:01.780 --> 00:35:04.650 Then we'd have no feedback whatsoever. 00:35:04.650 --> 00:35:08.400 So it could be either way. 00:35:08.400 --> 00:35:12.250 But then given this code equivalence concept, suppose 00:35:12.250 --> 00:35:13.950 we have a d of D that is very big. 00:35:13.950 --> 00:35:17.800 Suppose it's dominated by d of D. What would be a 00:35:17.800 --> 00:35:19.050 good thing to do? 00:35:24.610 --> 00:35:28.660 We have a certain code we want to keep, but the nu, the 00:35:28.660 --> 00:35:31.540 numbers for the dimension of the state space is dominated 00:35:31.540 --> 00:35:40.860 by the degree of d of D. So suppose we have g of D equal 00:35:40.860 --> 00:35:50.660 to n of D over d of D. And we have degree of d of D is 00:35:50.660 --> 00:35:55.750 greater than degree of n of D. What would be a 00:35:55.750 --> 00:35:57.000 good thing to do? 00:36:02.120 --> 00:36:09.720 Why don't we just let g prime of D be equal to d of D times 00:36:09.720 --> 00:36:19.950 g of D, and that would be just n of D. 00:36:19.950 --> 00:36:27.240 So I'm going to convert from the code generated by these 00:36:27.240 --> 00:36:31.060 rational generators, which are infinite. 00:36:31.060 --> 00:36:34.880 I can easily just multiply out the denominator, and now I 00:36:34.880 --> 00:36:39.900 have a code generated just by the numerator terms. 00:36:39.900 --> 00:36:41.150 So it's feedback-free. 00:36:44.180 --> 00:36:46.950 That may or may not be important to me. 00:36:46.950 --> 00:36:52.410 Actually, it turns out it's very hard to make a case for 00:36:52.410 --> 00:36:56.420 not having feedback, but it's simpler, in any case. 00:36:56.420 --> 00:36:59.810 We don't have this feedback term in the encoders. 00:36:59.810 --> 00:37:02.430 So we get a feedback-free encoder. 00:37:02.430 --> 00:37:06.525 And if this is true, we may reduce -- 00:37:09.430 --> 00:37:11.130 we can certainly never increase it. 00:37:13.890 --> 00:37:17.950 But if this were true, then we would reduce it. 00:37:17.950 --> 00:37:20.280 If on the other hand, this were less than or equal, then 00:37:20.280 --> 00:37:22.510 nu would remain the same. 00:37:22.510 --> 00:37:24.370 So that seems like that would be a good thing to do. 00:37:28.540 --> 00:37:34.910 So in fact, as I would advocate as a first step, that 00:37:34.910 --> 00:37:37.960 you clear the denominators, and just come up with a 00:37:37.960 --> 00:37:43.410 polynomial generator sequence which generates the same code. 00:37:43.410 --> 00:37:45.330 We're still going to have the same minimum distance, the 00:37:45.330 --> 00:37:48.300 same code sequence, the problem from the decoder's 00:37:48.300 --> 00:37:50.035 point of view is going to be absolutely unchanged. 00:37:50.035 --> 00:37:52.850 The decoder only cares what the consequences are. 00:37:52.850 --> 00:37:54.830 It wants to find the most likely one that was 00:37:54.830 --> 00:37:55.710 transmitted. 00:37:55.710 --> 00:37:58.970 So let's not make the encoder any more complicated 00:37:58.970 --> 00:38:01.596 than we have to. 00:38:01.596 --> 00:38:02.846 AUDIENCE: [UNINTELLIGIBLE PHRASE] 00:38:13.060 --> 00:38:16.590 PROFESSOR: nu doesn't change in this condition. 00:38:16.590 --> 00:38:18.805 nu does change in this condition. 00:38:18.805 --> 00:38:22.392 AUDIENCE: So we are always better off multiplying by g of 00:38:22.392 --> 00:38:22.490 D. 00:38:22.490 --> 00:38:23.300 PROFESSOR: We can't be worse off. 00:38:23.300 --> 00:38:23.610 [INTERPOSING VOICES] 00:38:23.610 --> 00:38:25.040 AUDIENCE: --or the same. 00:38:25.040 --> 00:38:27.530 PROFESSOR: Exactly. 00:38:27.530 --> 00:38:29.810 Got it. 00:38:29.810 --> 00:38:34.630 All right, so let's now talk about whether there's anything 00:38:34.630 --> 00:38:39.300 we can do to n of D. We already suggested that if all 00:38:39.300 --> 00:38:42.610 the n of D had a factor of d in them, why don't 00:38:42.610 --> 00:38:43.860 just take it out? 00:38:47.870 --> 00:38:49.150 Let's just write that down. 00:38:49.150 --> 00:39:03.510 If all n of D have a factor of d, then let's just transform n 00:39:03.510 --> 00:39:09.680 of D to d minus 1 n of D. 00:39:09.680 --> 00:39:12.455 So that reduces nu also. 00:39:15.360 --> 00:39:19.130 And let's, of course, do that as many times as we can. 00:39:19.130 --> 00:39:23.060 So it's a good idea to take out common factors of d. 00:39:23.060 --> 00:39:24.490 We get a simpler encoder. 00:39:24.490 --> 00:39:26.125 We're have a simpler state diagram. 00:39:30.010 --> 00:39:38.710 This leads us to a topic called catastrophicity, which 00:39:38.710 --> 00:39:41.990 is a misleading title because you have no idea where I'm 00:39:41.990 --> 00:39:43.540 going right now. 00:39:43.540 --> 00:39:45.340 So I'm actually going to try to fool you 00:39:45.340 --> 00:39:47.140 for a little while. 00:39:47.140 --> 00:39:47.500 Yes? 00:39:47.500 --> 00:39:48.726 AUDIENCE: Quick question. 00:39:48.726 --> 00:39:51.320 In the definition of code equivalence, is there a 00:39:51.320 --> 00:39:53.340 restriction [UNINTELLIGIBLE] 00:39:53.340 --> 00:39:57.460 that shouldn't be rational, shouldn't be causal? 00:39:57.460 --> 00:40:01.470 PROFESSOR: Should there be a restriction on this? 00:40:01.470 --> 00:40:02.480 Yes, of course. 00:40:02.480 --> 00:40:05.940 It has to be rational in order -- 00:40:08.470 --> 00:40:10.570 But that's a consequence. 00:40:10.570 --> 00:40:13.930 If these are both causal rational, then what does a of 00:40:13.930 --> 00:40:14.620 D have to be? 00:40:14.620 --> 00:40:16.770 It has to be rational. 00:40:16.770 --> 00:40:19.320 It has to be non-zero. 00:40:19.320 --> 00:40:22.300 And does it have to be causal? 00:40:22.300 --> 00:40:25.620 No, in this case it's not causal. 00:40:25.620 --> 00:40:26.870 AUDIENCE: [INAUDIBLE]. 00:40:31.310 --> 00:40:33.460 PROFESSOR: That's right. 00:40:33.460 --> 00:40:37.960 If there's a delay buried in g of D, if everything starts at 00:40:37.960 --> 00:40:41.870 time 1 or later, then a of D, in fact, can be non-causal. 00:40:41.870 --> 00:40:45.190 So the key restrictions are that these two guys have to be 00:40:45.190 --> 00:40:46.360 causal rational. 00:40:46.360 --> 00:40:52.450 And that will tell you restrictions on a. 00:40:52.450 --> 00:40:54.240 Let me introduce this by example. 00:40:54.240 --> 00:40:56.860 And I'll warn you that I'm going to try to 00:40:56.860 --> 00:40:58.940 fool you for a while. 00:40:58.940 --> 00:41:02.150 Let's just take a rate 1/1 encoder. 00:41:02.150 --> 00:41:09.060 This may seem a little peculiar to you, because it 00:41:09.060 --> 00:41:13.910 has no redundancy, one input, one output. 00:41:13.910 --> 00:41:18.300 And I'm going to let g of D just be 1 plus d. 00:41:22.550 --> 00:41:23.380 So what do I mean? 00:41:23.380 --> 00:41:25.780 I'm going to try to get some mileage out of this encoder. 00:41:29.146 --> 00:41:43.710 Here's u of D. Here's y of D, equals 1 plus d times u of D. 00:41:43.710 --> 00:41:48.295 Now let's draw the state diagram for this encoder. 00:41:48.295 --> 00:41:50.730 It's only got two states. 00:41:50.730 --> 00:41:51.580 One is 0. 00:41:51.580 --> 00:41:56.420 If we get a 0 in, we of course put a 0 out. 00:41:56.420 --> 00:41:57.740 We get 0, 0. 00:41:57.740 --> 00:42:00.105 Or we could get a 1 in. 00:42:00.105 --> 00:42:03.390 In which case, we would get a 1 out. 00:42:03.390 --> 00:42:06.230 Sorry, that was just a single output. 00:42:06.230 --> 00:42:08.346 And go to 1. 00:42:08.346 --> 00:42:17.000 If we're in the 1 state, and we get a 0 in, 00:42:17.000 --> 00:42:18.790 then we get a 1 out. 00:42:18.790 --> 00:42:22.990 Whereas if we get a 1 in, then they're both 0, and we get the 00:42:22.990 --> 00:42:24.350 complement. 00:42:24.350 --> 00:42:27.140 So with the 1, we get a 0 out. 00:42:27.140 --> 00:42:30.180 So that's the state transition diagram of this encoder. 00:42:36.500 --> 00:42:39.040 Now, let's go through a similar kind of argument to 00:42:39.040 --> 00:42:41.470 what we went through to find the minimum distance was 5. 00:42:41.470 --> 00:42:45.170 This is a linear system, so we want to find the minimum 00:42:45.170 --> 00:42:48.350 non-zero weight of any code word. 00:42:48.350 --> 00:42:51.720 And here's my argument. 00:42:51.720 --> 00:42:54.050 There's a gold star to the first person who punches a 00:42:54.050 --> 00:42:56.380 hole in it. 00:42:56.380 --> 00:43:01.560 Any code word, you have to start in the 0 state, so 00:43:01.560 --> 00:43:03.700 you're always going to get at least one unit of distance 00:43:03.700 --> 00:43:07.120 when you go to the 1 state. 00:43:07.120 --> 00:43:10.550 You can stay out here as long as you like, but eventually 00:43:10.550 --> 00:43:14.550 you have to come back to the 0 state. 00:43:14.550 --> 00:43:16.430 And at that point, you're going to get 00:43:16.430 --> 00:43:18.600 another unit of distance. 00:43:18.600 --> 00:43:25.360 So I claim the minimum non-zero weight is 2. 00:43:33.328 --> 00:43:34.578 Is that correct? 00:43:39.870 --> 00:43:41.870 Let me try to create some doubt. 00:43:41.870 --> 00:43:44.420 What is the code here? 00:43:44.420 --> 00:43:51.620 The code is the set of all multiples of 1 plus d times u 00:43:51.620 --> 00:43:58.335 of D, where u of D is a formal Laurent sequence. 00:44:01.240 --> 00:44:04.420 What is that? 00:44:04.420 --> 00:44:07.490 I can write this briefly as just the set of all formal 00:44:07.490 --> 00:44:10.330 Laurent sequences times 1 plus d. 00:44:10.330 --> 00:44:10.514 AUDIENCE: [INAUDIBLE] 00:44:10.514 --> 00:44:19.160 1 minus d minus d squared. 00:44:19.160 --> 00:44:20.410 PROFESSOR: Good. 00:44:22.300 --> 00:44:24.260 That's correct. 00:44:24.260 --> 00:44:26.952 That's what's wrong with my claim. 00:44:26.952 --> 00:44:31.100 Let me detour around here. 00:44:31.100 --> 00:44:33.450 Let me get an answer to this question. 00:44:33.450 --> 00:44:35.170 What is the code in this case? 00:44:35.170 --> 00:44:39.180 It's the set of all possible sequences you can get by 00:44:39.180 --> 00:44:43.010 multiplying 1 plus d times the formal Laurent sequence. 00:44:43.010 --> 00:44:43.685 What is that? 00:44:43.685 --> 00:44:44.904 AUDIENCE: [INAUDIBLE]. 00:44:44.904 --> 00:44:45.351 PROFESSOR: What? 00:44:45.351 --> 00:44:46.480 AUDIENCE: Isn't it [UNINTELLIGIBLE]? 00:44:46.480 --> 00:44:47.840 PROFESSOR: It's just the set of all 00:44:47.840 --> 00:44:49.090 formal Laurent sequences. 00:44:51.430 --> 00:44:57.190 Certainly for every formal Laurent sequence, I can get 00:44:57.190 --> 00:44:59.420 such a sequence this way. 00:44:59.420 --> 00:45:05.300 In particular, the code includes 1. 00:45:05.300 --> 00:45:10.730 What do I need as an input to get the output sequence 1? 00:45:10.730 --> 00:45:12.530 AUDIENCE: String of 1's. 00:45:12.530 --> 00:45:15.710 PROFESSOR: String of 1's, right. 00:45:15.710 --> 00:45:24.760 So an example, if u of D is equal to 1/1 plus d, which is 00:45:24.760 --> 00:45:37.910 a string of 1's, then y of D is equal to 1 plus d times u 00:45:37.910 --> 00:45:42.010 of D, which is 1. 00:45:42.010 --> 00:45:44.190 1 plus d has an inverse, and this is it. 00:45:47.840 --> 00:45:54.170 So now we're coming back to Mr. Aggarwal's point. 00:45:54.170 --> 00:45:59.180 Suppose we put in u of D equals this sequence, all 0's 00:45:59.180 --> 00:46:03.510 before times 0, all 1's after time 0. 00:46:03.510 --> 00:46:05.530 Then where will we go? 00:46:05.530 --> 00:46:08.200 We'll go 1, and then we'll just stay 00:46:08.200 --> 00:46:11.950 in this 0 loop forever. 00:46:11.950 --> 00:46:15.090 Anything wrong with that? 00:46:15.090 --> 00:46:17.490 Is that a code word? 00:46:17.490 --> 00:46:21.190 That is a code word, the way we've defined the code. 00:46:21.190 --> 00:46:24.310 If we had defined the code so that the u of D were only 00:46:24.310 --> 00:46:30.220 finite input sequences, then this claim would be correct. 00:46:30.220 --> 00:46:32.640 For any finite input sequence, we eventually have to come 00:46:32.640 --> 00:46:33.930 back to the 0 state. 00:46:36.620 --> 00:46:48.210 So this is true for finite inputs, not true 00:46:48.210 --> 00:46:49.460 for infinite inputs. 00:46:57.315 --> 00:46:59.175 So this is wrong. 00:46:59.175 --> 00:47:06.540 The fact is d3 equals 1. 00:47:06.540 --> 00:47:08.790 The minimum weight of this code is 1. 00:47:11.690 --> 00:47:18.220 But you see this is, first of all, it's a confusing 00:47:18.220 --> 00:47:22.460 situation for analytical purposes. 00:47:22.460 --> 00:47:26.620 Let me further indicate why it might be bad to have an 00:47:26.620 --> 00:47:30.060 encoder like this in general. 00:47:30.060 --> 00:47:44.270 The key catastrophicity, in general, will be defined as 00:47:44.270 --> 00:47:57.540 there are finite outputs generated by 00:47:57.540 --> 00:48:01.390 infinite inputs, as above. 00:48:07.990 --> 00:48:18.190 Or equivalently, there are 0 loops in the 00:48:18.190 --> 00:48:19.440 state transition diagram. 00:48:25.620 --> 00:48:28.250 So by choosing a proper input, you can just keep driving 00:48:28.250 --> 00:48:31.088 around a 0 loop forever. 00:48:31.088 --> 00:48:33.956 AUDIENCE: [INAUDIBLE]. 00:48:33.956 --> 00:48:35.040 PROFESSOR: Excuse me? 00:48:35.040 --> 00:48:36.290 AUDIENCE: [INAUDIBLE] 00:48:38.315 --> 00:48:43.059 the output is becoming finite, so it's actually -- 00:48:43.059 --> 00:48:46.540 I'm just commenting on the whole catastrophicity. 00:48:46.540 --> 00:48:48.280 PROFESSOR: OK, well, I want to next tell you where 00:48:48.280 --> 00:48:49.720 catastrophicity comes from. 00:48:49.720 --> 00:48:54.450 The opposite of this property is non-catastrophicity. 00:48:54.450 --> 00:48:57.752 What we want is non-catastrophic encoders. 00:48:57.752 --> 00:49:03.880 We want encoders where, to get a finite output, you get a 00:49:03.880 --> 00:49:07.340 finite output only from finite inputs. 00:49:13.330 --> 00:49:16.350 That's better analytically and it's better in practice. 00:49:16.350 --> 00:49:18.580 Here, let me draw you the practical argument. 00:49:18.580 --> 00:49:23.490 We have, for this particular case, u of D comes in, gets 00:49:23.490 --> 00:49:26.322 multiplied by 1 plus d. 00:49:26.322 --> 00:49:30.470 That gives us y of D, which -- 00:49:30.470 --> 00:49:31.940 let me just abbreviate it. 00:49:31.940 --> 00:49:34.780 We transmit that over a channel, 00:49:34.780 --> 00:49:38.470 and we get our decoder. 00:49:38.470 --> 00:49:42.730 Actually, decoder doesn't have much to do here, because all 00:49:42.730 --> 00:49:46.240 possible sequences are allowed at the output. 00:49:46.240 --> 00:49:49.980 So if this were a binary symmetric channel, the best 00:49:49.980 --> 00:49:52.590 the decoder could do would just be to put out whatever it 00:49:52.590 --> 00:49:56.930 sees as its guess, y-hat of D. So this 00:49:56.930 --> 00:49:58.180 is the decoder estimate. 00:50:01.030 --> 00:50:07.140 So in any case, this can differ by 00:50:07.140 --> 00:50:08.740 code words from that. 00:50:08.740 --> 00:50:12.070 All possible code words are possible. 00:50:12.070 --> 00:50:19.330 And given y-hat of D, to get back to u of D, what do we 00:50:19.330 --> 00:50:19.830 have to do? 00:50:19.830 --> 00:50:23.690 We have to divide by 1/1 plus d, in effect, 00:50:23.690 --> 00:50:25.260 to invert this equation. 00:50:25.260 --> 00:50:29.710 For y of D is equal to 1 plus d times u of D, then the u of 00:50:29.710 --> 00:50:32.330 D that corresponds to a particular decoded code 00:50:32.330 --> 00:50:36.630 sequence, y-hat of D, we get by dividing by this. 00:50:36.630 --> 00:50:38.060 But this is infinite sequence. 00:50:38.060 --> 00:50:40.350 Suppose this makes just one error. 00:50:40.350 --> 00:50:44.420 It's possible for this to transmit all 0's, and for this 00:50:44.420 --> 00:50:45.670 to make just one error. 00:50:48.270 --> 00:50:49.600 That's a code word. 00:50:49.600 --> 00:50:52.770 That's a legitimate code sequence. 00:50:52.770 --> 00:50:56.690 So it's something the decoder might come up with. 00:50:56.690 --> 00:51:01.500 If it does that, then what's this going to look like? 00:51:01.500 --> 00:51:06.440 This is going to look like an infinite number of 1's, 00:51:06.440 --> 00:51:10.110 because this is the infinite input that causes this code 00:51:10.110 --> 00:51:14.010 sequence with a finite number of outputs. 00:51:14.010 --> 00:51:17.750 So it's precisely this condition, whenever we make an 00:51:17.750 --> 00:51:22.250 error that corresponds to a finite output sequence, and we 00:51:22.250 --> 00:51:26.110 translate it back to the encoder input, that's an 00:51:26.110 --> 00:51:27.630 infinite encoder input. 00:51:27.630 --> 00:51:29.600 There's a one-to-one correspondence between inputs 00:51:29.600 --> 00:51:30.420 and outputs. 00:51:30.420 --> 00:51:33.350 So it's got to happen whenever we make that type of error. 00:51:33.350 --> 00:51:35.100 And this is called catastrophic error 00:51:35.100 --> 00:51:36.320 propagation. 00:51:36.320 --> 00:51:37.670 The decoder hasn't done anything bad. 00:51:37.670 --> 00:51:43.710 It made one error, as it's going to do once in a while. 00:51:43.710 --> 00:51:47.200 But the consequences are catastrophic. 00:51:47.200 --> 00:51:50.250 We make an infinite number of errors in the bits that we 00:51:50.250 --> 00:51:53.408 actually deliver to the user. 00:51:53.408 --> 00:51:55.540 Now, of course, what's really going to happen is that 00:51:55.540 --> 00:51:58.530 sometime later the decoder is going to make 00:51:58.530 --> 00:52:00.740 another little error. 00:52:00.740 --> 00:52:04.330 And at that point, this will switch state again. 00:52:04.330 --> 00:52:07.750 So that will terminate the error burst when the decoder 00:52:07.750 --> 00:52:08.740 makes another error. 00:52:08.740 --> 00:52:10.130 But this is completely random. 00:52:10.130 --> 00:52:12.330 It may take a long time for this to happen, 00:52:12.330 --> 00:52:14.450 and it's not good. 00:52:14.450 --> 00:52:16.835 So for practical reasons, we don't want catastrophicity. 00:52:24.720 --> 00:52:27.990 I was going to give you another example, but in the 00:52:27.990 --> 00:52:32.190 interest of time, I'll just write it down. 00:52:32.190 --> 00:52:36.360 Here's another catastrophic example. 00:52:36.360 --> 00:52:42.260 Let me just do a rate 1/2 encoder so it looks more 00:52:42.260 --> 00:52:44.720 reasonable. 00:52:44.720 --> 00:52:47.250 And we'll make it look very much like the 00:52:47.250 --> 00:52:50.230 encoders we've had. 00:52:50.230 --> 00:52:55.170 Our first example looks something like this. 00:52:55.170 --> 00:52:56.570 1 plus d squared. 00:52:56.570 --> 00:52:58.340 I'll say 1 plus d. 00:52:58.340 --> 00:52:59.730 So it's a feet-forward encoder. 00:52:59.730 --> 00:53:01.755 It looks perfectly reasonable. 00:53:07.340 --> 00:53:10.290 The top channel is impulse response, 1 plus d squared. 00:53:10.290 --> 00:53:14.030 The second output has impulse response, 1 plus d. 00:53:14.030 --> 00:53:16.400 If you look at the minimum weight for all finite 00:53:16.400 --> 00:53:18.890 sequences, it's 4, not quite as good as the other 00:53:18.890 --> 00:53:21.550 one which had 5. 00:53:21.550 --> 00:53:23.005 But is this catastrophic? 00:53:26.430 --> 00:53:27.020 Yes? 00:53:27.020 --> 00:53:28.020 Who said yes? 00:53:28.020 --> 00:53:29.270 AUDIENCE: [INAUDIBLE]. 00:53:38.390 --> 00:53:39.480 PROFESSOR: Great. 00:53:39.480 --> 00:53:46.480 So the same observation, if u of D is equal to 1/1 plus d, 00:53:46.480 --> 00:53:53.280 which is, again, this infinite sequence, then the output, y 00:53:53.280 --> 00:54:01.040 of D equals u of D, g of D, is equal to -- 00:54:01.040 --> 00:54:05.410 1 plus d divides both of these, so we get 1 plus d1. 00:54:05.410 --> 00:54:06.915 We get a finite output sequence. 00:54:11.260 --> 00:54:14.880 You saw that immediately, because we know that 00:54:14.880 --> 00:54:18.180 polynomials are divisible by 1 plus d if and only if they 00:54:18.180 --> 00:54:21.105 have an even number of non-zero terms. 00:54:24.960 --> 00:54:27.750 So that's a more elaborate example of where we can get 00:54:27.750 --> 00:54:28.970 catastrophic behavior. 00:54:28.970 --> 00:54:32.400 So what should we do here? 00:54:32.400 --> 00:54:35.080 Clearly, we should divide out the common factor from here. 00:54:35.080 --> 00:54:36.480 We can do that. 00:54:36.480 --> 00:54:41.790 If all of these polynomials have a common factor, we 00:54:41.790 --> 00:54:43.930 should divide it out. 00:54:43.930 --> 00:54:49.360 And we know that the code generated by -- if this is g 00:54:49.360 --> 00:54:54.940 of D, we're going to say g prime of D is equal to 1/1 00:54:54.940 --> 00:55:00.490 plus d times 1 plus d squared, 1 plus d. 00:55:00.490 --> 00:55:02.700 And that is -- we've already calculated -- 00:55:02.700 --> 00:55:06.350 1 plus d1. 00:55:06.350 --> 00:55:10.170 So by doing that, first of all, we got rid of the 00:55:10.170 --> 00:55:11.870 catastrophicity. 00:55:11.870 --> 00:55:13.880 Second of all, again, we reduced nu. 00:55:16.690 --> 00:55:20.760 So this is clearly a very good thing to do. 00:55:20.760 --> 00:55:28.600 So more generally, going back here, if all n of D have any 00:55:28.600 --> 00:55:29.850 common factor -- 00:55:35.880 --> 00:55:37.130 in other words, let's call -- 00:55:40.094 --> 00:55:43.540 I've used d, so what'll I use? 00:55:43.540 --> 00:55:52.220 b of D equals the greatest common divisor of all of the n 00:55:52.220 --> 00:55:55.065 of D, 1 plus -- 00:55:55.065 --> 00:55:56.315 AUDIENCE: [INAUDIBLE]. 00:56:00.200 --> 00:56:01.055 PROFESSOR: Who -- 00:56:01.055 --> 00:56:03.960 AUDIENCE: Do you think it could be 1? 00:56:03.960 --> 00:56:08.190 PROFESSOR: You're, once again, 30 seconds ahead of me. 00:56:08.190 --> 00:56:10.830 So yes, that's what I'm going to say. 00:56:10.830 --> 00:56:13.660 If they all have a common factor, then they have a GCD 00:56:13.660 --> 00:56:15.150 that is not equal to 1. 00:56:15.150 --> 00:56:18.260 That's if and only if. 00:56:18.260 --> 00:56:27.613 So we're going to transform n of D to 1 over b of D. We're 00:56:27.613 --> 00:56:31.700 going to divide out the common factor, n of D. That will be 00:56:31.700 --> 00:56:34.760 our n prime. 00:56:34.760 --> 00:56:36.870 And now what have we achieved? 00:56:36.870 --> 00:56:40.040 We've achieved that there is no common factor of these n 00:56:40.040 --> 00:56:50.630 prime of D. So no common factor, which does turn out. 00:56:50.630 --> 00:56:58.410 So that will mean it's non-catastrophic, and, 00:56:58.410 --> 00:57:08.750 furthermore, will reduce nu by whatever the degree of this 00:57:08.750 --> 00:57:13.735 common factor was, degree of b of D. Let me 00:57:13.735 --> 00:57:15.010 write degree of GCD. 00:57:20.740 --> 00:57:22.320 We want them to have no common factor. 00:57:22.320 --> 00:57:28.810 We want the greatest common divisor to be 1. 00:57:28.810 --> 00:57:31.550 Another way of saying this is we want the ni of d to be 00:57:31.550 --> 00:57:36.950 relatively prime, the nj of d to be relatively prime. 00:57:36.950 --> 00:57:43.130 And if that's true, then can we prove that it's 00:57:43.130 --> 00:57:45.740 non-catastrophic, that if we have a finite output, that it 00:57:45.740 --> 00:57:49.290 must've been generated by a finite input? 00:57:52.188 --> 00:57:56.102 AUDIENCE: No, because u of D times n_i of D would have to 00:57:56.102 --> 00:58:03.070 be a polynomial to be catastrophic, and to get a 00:58:03.070 --> 00:58:04.320 polynomial. 00:58:05.916 --> 00:58:09.260 And u of D is rational. 00:58:09.260 --> 00:58:10.730 It has something in the denominator, 00:58:10.730 --> 00:58:12.230 because u of D is infinite. 00:58:14.800 --> 00:58:17.970 That denominator should then get cancelled 00:58:17.970 --> 00:58:19.850 out to one of these. 00:58:19.850 --> 00:58:24.500 PROFESSOR: Yeah, you have the argument in your head. 00:58:24.500 --> 00:58:26.930 Let me leave that as an exercise for the student, 00:58:26.930 --> 00:58:28.910 since at least a couple of you see it. 00:58:31.840 --> 00:58:34.950 The only way that you can get this behavior, if you have an 00:58:34.950 --> 00:58:39.620 infinite input which has in its denominator a common 00:58:39.620 --> 00:58:42.170 factor of all the n's. 00:58:42.170 --> 00:58:45.700 So if the n's have no common factor, 00:58:45.700 --> 00:58:46.820 then this can't happen. 00:58:46.820 --> 00:58:50.680 That's the outline of the argument. 00:58:50.680 --> 00:58:56.970 So what's our conclusion now? 00:58:56.970 --> 00:59:01.030 Since we have multiple encoders that generate the 00:59:01.030 --> 00:59:06.440 same code, we'd like to get the nicest one, in some sense, 00:59:06.440 --> 00:59:10.470 as our canonical encoder for a given code. 00:59:10.470 --> 00:59:11.540 So we have a given code. 00:59:11.540 --> 00:59:16.780 We can specify it by any causal 00:59:16.780 --> 00:59:19.630 rational transfer function. 00:59:19.630 --> 00:59:24.740 But then, having done that, we should clean this generator 00:59:24.740 --> 00:59:26.130 n-tuple up. 00:59:26.130 --> 00:59:31.540 We should multiply out by the least common multiple of all 00:59:31.540 --> 00:59:36.680 the divisors to make it polynomial, feedback-free, 00:59:36.680 --> 00:59:39.650 possibly reduce nu. 00:59:39.650 --> 00:59:42.430 And then we should check the numerators and see if they 00:59:42.430 --> 00:59:46.310 have any common factor, which could be d or it could be any 00:59:46.310 --> 00:59:47.160 polynomial. 00:59:47.160 --> 00:59:50.120 And whatever it is, we should divide that out. 00:59:50.120 --> 00:59:54.060 And that will certainly reduce nu. 00:59:54.060 --> 01:00:04.190 So at the end of the day, we have a canonical encoder, 01:00:04.190 --> 01:00:07.940 which is equal to g of D times -- 01:00:07.940 --> 01:00:10.930 we want to multiply by the least common multiple of the 01:00:10.930 --> 01:00:13.800 denominators. 01:00:13.800 --> 01:00:17.700 We want to divide by the greatest common divisor of the 01:00:17.700 --> 01:00:18.950 numerators. 01:00:21.770 --> 01:00:29.080 And this will be some n prime of D, which will be 01:00:29.080 --> 01:00:31.990 polynomial. 01:00:31.990 --> 01:00:34.040 It will be delay-free. 01:00:34.040 --> 01:00:37.760 That means it doesn't have d as a common factor. 01:00:37.760 --> 01:00:40.090 It will be non-catastrophic. 01:00:40.090 --> 01:00:41.240 That means it doesn't have anything 01:00:41.240 --> 01:00:44.350 else as a common factor. 01:00:44.350 --> 01:00:49.890 And it will have minimal nu, for any encoder that can 01:00:49.890 --> 01:00:52.250 generate that same code. 01:00:52.250 --> 01:00:55.350 I haven't quite proved all those properties, but I think 01:00:55.350 --> 01:00:57.310 we're well along the way. 01:00:57.310 --> 01:00:58.365 So this is really -- 01:00:58.365 --> 01:00:59.615 AUDIENCE: [INAUDIBLE]. 01:01:03.430 --> 01:01:06.063 There should be at least n1 and n2. 01:01:06.063 --> 01:01:07.165 [UNINTELLIGIBLE] 01:01:07.165 --> 01:01:11.500 PROFESSOR: Well, if we have a rate 1/1 code, we just have 01:01:11.500 --> 01:01:14.660 one of them, then what's the greatest common divisor? 01:01:14.660 --> 01:01:18.620 Suppose it's n of D over d of D. The greatest common divisor 01:01:18.620 --> 01:01:22.820 is n of D. The least common multiple is d of D. We get 1. 01:01:22.820 --> 01:01:26.250 So going through this exercise for any rate 1/1 code, we'll 01:01:26.250 --> 01:01:30.790 find that the canonical encoder is 1. 01:01:30.790 --> 01:01:34.800 And that is clearly what we want for a rate 1/1 encoder, 01:01:34.800 --> 01:01:38.570 because a rate 1/1 is always going to have an out. 01:01:38.570 --> 01:01:41.250 The code is just going to be the universe code of all the 01:01:41.250 --> 01:01:43.770 possible formal Laurent sequences. 01:01:43.770 --> 01:01:47.470 So we get down to the right encoder for that code, not 01:01:47.470 --> 01:01:48.720 some stupid encoder. 01:01:50.880 --> 01:01:55.690 And in a similar way, going through this process, we come 01:01:55.690 --> 01:01:56.590 up with the right encoder. 01:01:56.590 --> 01:01:59.230 Is this process unique? 01:01:59.230 --> 01:02:04.200 Is there only one encoder we could come up with? 01:02:07.570 --> 01:02:11.360 Think about it a little, and you could convince yourself 01:02:11.360 --> 01:02:15.960 there's only encoder that you can get from doing this. 01:02:19.220 --> 01:02:20.620 This is over F2. 01:02:20.620 --> 01:02:25.000 Over a larger field, it's unique up to units again, up 01:02:25.000 --> 01:02:28.990 to a non-zero scalar multiple, which of course you can define 01:02:28.990 --> 01:02:34.390 to be 1 and somehow make this unique. 01:02:34.390 --> 01:02:41.495 So it's basically a unique canonical encoder that has all 01:02:41.495 --> 01:02:42.550 of these properties. 01:02:42.550 --> 01:02:44.880 You get it in this way. 01:02:44.880 --> 01:02:48.400 So this is what we're always going to use. 01:02:48.400 --> 01:02:55.135 For instance, for our original encoder, 1 plus -- 01:02:55.135 --> 01:02:55.850 what was it? -- 01:02:55.850 --> 01:02:59.800 1 plus d squared, 1 plus d plus d squared. 01:02:59.800 --> 01:03:01.500 Is that already in canonical form? 01:03:07.470 --> 01:03:09.280 This is irreducible. 01:03:09.280 --> 01:03:13.990 This is divisible by 1 plus D, so these are relatively prime. 01:03:13.990 --> 01:03:15.510 It's feed-forward. 01:03:15.510 --> 01:03:19.330 It doesn't have any denominator terms, and it's 01:03:19.330 --> 01:03:20.230 delay-free. 01:03:20.230 --> 01:03:25.500 So it satisfies all of these, and clearly you can't 01:03:25.500 --> 01:03:26.400 manipulate it. 01:03:26.400 --> 01:03:28.580 The only ways you're allowed to manipulate it are by 01:03:28.580 --> 01:03:35.530 multiplying by a of D over b of D. And it's clear that if 01:03:35.530 --> 01:03:38.350 you have a non-trivial numerator or denominator, the 01:03:38.350 --> 01:03:40.290 numerator will make it catastrophic, the denominator 01:03:40.290 --> 01:03:44.800 will make it to have feedback. 01:03:44.800 --> 01:03:46.340 Now, as I was saying, there's really 01:03:46.340 --> 01:03:47.940 nothing wrong with feedback. 01:03:47.940 --> 01:03:51.560 And so some people prefer -- 01:03:51.560 --> 01:03:54.160 the one thing that this doesn't have, in general, is 01:03:54.160 --> 01:03:57.030 it's not systematic. 01:03:57.030 --> 01:03:59.990 That means the input stream is not one of the output streams. 01:04:06.060 --> 01:04:10.910 So given a canonical encoder like this, can we always make 01:04:10.910 --> 01:04:12.160 it systematic? 01:04:15.450 --> 01:04:20.910 Yeah, we just divide by any of these polynomials. 01:04:20.910 --> 01:04:25.450 So here's an equivalent systematic encoder. 01:04:31.420 --> 01:04:35.380 For instance, if we divide both of these by 1 plus D 01:04:35.380 --> 01:04:40.310 squared, we get 1 plus D plus D squared, 01:04:40.310 --> 01:04:41.590 over 1 plus d squared. 01:04:45.580 --> 01:04:48.390 This now is not polynomial. 01:04:48.390 --> 01:04:49.810 It is delay-free. 01:04:54.190 --> 01:04:55.440 It's non-catastrophic. 01:04:58.570 --> 01:05:01.210 There's no 1 over something that we can put in to get a 01:05:01.210 --> 01:05:03.140 finite thing out. 01:05:03.140 --> 01:05:07.560 It turns out it's still minimal nu. 01:05:07.560 --> 01:05:10.090 We know how to realize this with two memory elements now. 01:05:13.240 --> 01:05:17.580 And in general, if you're not going to introduce a 01:05:17.580 --> 01:05:20.870 denominator term that has a greater degree than the 01:05:20.870 --> 01:05:23.260 maximum degree of the numerator terms, so it's still 01:05:23.260 --> 01:05:25.460 realizable with minimal nu. 01:05:25.460 --> 01:05:33.380 And so now it's systematic, but now it's not 01:05:33.380 --> 01:05:34.630 feedback-free. 01:05:41.270 --> 01:05:44.280 So I consider this one certainly the nicest. 01:05:44.280 --> 01:05:50.060 The canonical feedback-free encoder is the nicest one for 01:05:50.060 --> 01:05:53.370 analytical purposes, for instance, for trying to find 01:05:53.370 --> 01:05:56.510 minumum-weight finite code words. 01:05:56.510 --> 01:05:59.670 Because of the finite property, if we're looking for 01:05:59.670 --> 01:06:01.790 the non-catastrophic property, if we're looking for 01:06:01.790 --> 01:06:03.850 minimum-weight code words, we only have to 01:06:03.850 --> 01:06:05.720 look at finite inputs. 01:06:05.720 --> 01:06:08.990 We can now use the kind of analysis method that we use 01:06:08.990 --> 01:06:12.930 that says you always have to come back to the 0 state, 01:06:12.930 --> 01:06:19.830 because only finite inputs can produce finite outputs here. 01:06:19.830 --> 01:06:22.970 This also has that property. 01:06:22.970 --> 01:06:30.580 And people tended not to use these for a long time, but it 01:06:30.580 --> 01:06:35.400 was founded that in turbo codes, you want to have this, 01:06:35.400 --> 01:06:38.460 one of the generators be infinite. 01:06:38.460 --> 01:06:41.300 And it was also nice to have one of them be systematic, so 01:06:41.300 --> 01:06:44.030 that led to a revival of interest in systematic 01:06:44.030 --> 01:06:45.090 convolutional encoders. 01:06:45.090 --> 01:06:47.150 And this, again, it's a unique form. 01:06:47.150 --> 01:06:50.770 If you say the first one has to be the systematic one, then 01:06:50.770 --> 01:06:53.710 there's a unique equivalent systematic encoder that 01:06:53.710 --> 01:06:55.330 generates any code. 01:06:55.330 --> 01:06:58.190 So it's in alternative canonical form. 01:06:58.190 --> 01:06:59.440 Take your pick. 01:07:03.480 --> 01:07:04.390 Yeah? 01:07:04.390 --> 01:07:07.120 AUDIENCE: What's systematic? 01:07:07.120 --> 01:07:08.870 PROFESSOR: What's systematic? 01:07:08.870 --> 01:07:12.630 Systematic means that the input that's appear unchanged 01:07:12.630 --> 01:07:15.560 as a subset of the output bits. 01:07:15.560 --> 01:07:19.720 For convolutional codes, it means that one of the output 01:07:19.720 --> 01:07:23.400 sequences, if you have a rate 1/n encoder, one of the output 01:07:23.400 --> 01:07:25.600 sequences is simply equal to the input sequence. 01:07:25.600 --> 01:07:26.900 In other words, one of the transfer 01:07:26.900 --> 01:07:29.908 functions is equal to 1. 01:07:29.908 --> 01:07:31.158 AUDIENCE: [INAUDIBLE]. 01:07:34.450 --> 01:07:36.570 PROFESSOR: Yes, very good. 01:07:36.570 --> 01:07:38.998 Excellent comment. 01:07:38.998 --> 01:07:44.890 Because now, if you have a finite output sequence, in 01:07:44.890 --> 01:07:48.110 particular you can just read off the input sequence, the 01:07:48.110 --> 01:07:51.640 information sequence, from this first term, and if the 01:07:51.640 --> 01:07:53.880 whole thing was finite, it's finite. 01:07:53.880 --> 01:07:59.360 So systematic directly ensures non-catastrophicity. 01:07:59.360 --> 01:08:00.630 Excellent, excellent comment. 01:08:05.410 --> 01:08:08.370 Which you might try to prove by the more elaborate 01:08:08.370 --> 01:08:11.410 techniques that we had. 01:08:11.410 --> 01:08:16.590 So that gets us pretty much through the algebra that I 01:08:16.590 --> 01:08:19.920 wanted to do. 01:08:19.920 --> 01:08:23.069 Now I want to -- 01:08:23.069 --> 01:08:26.890 OK, good. 01:08:26.890 --> 01:08:29.649 Now let's go more to the finite state picture. 01:08:33.689 --> 01:08:51.160 And our objective here is mostly going to be to get to 01:08:51.160 --> 01:08:54.220 Viterbi decoding algorithm. 01:08:54.220 --> 01:08:57.550 But along the way, we'll see that we can use something like 01:08:57.550 --> 01:09:00.090 the Viterbi algorithm to compute minimum distances, 01:09:00.090 --> 01:09:04.349 too, once we have a finite state picture to the code. 01:09:07.620 --> 01:09:17.770 Again, take our example one, where g of D is 1 plus D 01:09:17.770 --> 01:09:21.270 squared, 1 plus D plus D squared. 01:09:21.270 --> 01:09:27.569 We have a state transition diagram, 01:09:27.569 --> 01:09:34.469 which looks like this. 01:09:43.890 --> 01:09:48.074 And I won't bother to label it. 01:09:52.060 --> 01:09:55.350 And that's a complete description of the encoder 01:09:55.350 --> 01:10:00.260 operation, if I put labels of inputs and outputs on all 01:10:00.260 --> 01:10:01.695 these states. 01:10:01.695 --> 01:10:08.110 So 0 slash 0,0, so forth, 1 slash 1, 1. 01:10:08.110 --> 01:10:12.400 Just to draw the impulse response, 0, 0, 1, 01:10:12.400 --> 01:10:15.380 and 0 slash 1, 1. 01:10:15.380 --> 01:10:18.110 There's the basic impulse response going 01:10:18.110 --> 01:10:19.360 around, like that. 01:10:22.940 --> 01:10:27.900 Trellis diagram is a very redundant 01:10:27.900 --> 01:10:29.390 picture of the same thing. 01:10:35.070 --> 01:10:38.600 We can start out in the 0 state that we like. 01:10:38.600 --> 01:10:44.950 Let's say we start off in the 0 state at some time k, as we 01:10:44.950 --> 01:10:45.530 always are. 01:10:45.530 --> 01:10:47.010 We just don't know what k is. 01:10:50.260 --> 01:10:52.600 Then let's spread the state transition 01:10:52.600 --> 01:10:54.070 diagram out in time. 01:10:54.070 --> 01:10:57.610 Let's draw all the states again at time k plus 1. 01:11:00.380 --> 01:11:02.490 Actually, there are only two it can get to, 01:11:02.490 --> 01:11:03.950 at time k plus 1. 01:11:03.950 --> 01:11:09.870 So we can either stay in this state or we can go down here 01:11:09.870 --> 01:11:15.450 to this state, at the next unit of time. 01:11:15.450 --> 01:11:19.780 I'm just going to keep drawing all of the possible state 01:11:19.780 --> 01:11:22.740 trajectories or paths. 01:11:22.740 --> 01:11:27.080 At time 2, I can reach all possible states. 01:11:27.080 --> 01:11:32.730 0, 0, 1, 0, 0, 1, 1, 1. 01:11:32.730 --> 01:11:38.240 Here again, I already know what these are. 01:11:38.240 --> 01:11:43.741 From here, I can get 1, 0, 2, 0. 01:11:43.741 --> 01:11:46.660 It goes with a 0, 1 out. 01:11:46.660 --> 01:11:51.170 So 1 goes with a 1, 0. 01:11:51.170 --> 01:11:59.380 And k plus 3, I now have all possible transitions. 01:11:59.380 --> 01:12:02.120 It's going to look like this. 01:12:02.120 --> 01:12:04.110 Where can I go from 0, 1? 01:12:04.110 --> 01:12:07.970 I can go back to here, or I could go to here. 01:12:07.970 --> 01:12:09.950 And I could label these. 01:12:09.950 --> 01:12:11.625 I should label these. 01:12:11.625 --> 01:12:12.620 But I won't. 01:12:12.620 --> 01:12:15.770 Here, I can go to these two. 01:12:15.770 --> 01:12:18.210 There are eight possible state transitions. 01:12:18.210 --> 01:12:20.440 k plus 4. 01:12:20.440 --> 01:12:22.470 It's completely repetitive, since it's a 01:12:22.470 --> 01:12:24.940 time-invariant system. 01:12:24.940 --> 01:12:27.210 I just keep going like this. 01:12:34.610 --> 01:12:36.720 And why have I gone to all this trouble? 01:12:36.720 --> 01:12:40.555 This clearly contains the same information as this diagram, 01:12:40.555 --> 01:12:42.340 but it's just spread out in time. 01:12:45.200 --> 01:12:54.210 The basic reason for doing it is that now there is a unique 01:12:54.210 --> 01:12:58.040 trellis path corresponding to every code word that can start 01:12:58.040 --> 01:12:59.380 at time k or later. 01:13:02.010 --> 01:13:06.840 So for instance, if I just have an impulse at time k and 01:13:06.840 --> 01:13:12.450 nothing else, then the unique trellis path is this. 01:13:12.450 --> 01:13:16.700 It starts at 0, goes through two non-zero states, comes 01:13:16.700 --> 01:13:21.710 back to 0, 0, and then stays in 0, 0 forever more. 01:13:25.860 --> 01:13:27.351 AUDIENCE: [INAUDIBLE]. 01:13:27.351 --> 01:13:30.890 PROFESSOR: I've just taken an arbitrary starting time, k. 01:13:30.890 --> 01:13:33.110 After a while, I'll just look at it -- 01:13:33.110 --> 01:13:34.100 well, it's entrained. 01:13:34.100 --> 01:13:36.710 This is kind of the steady state version of the trellis. 01:13:36.710 --> 01:13:40.880 I've included a little starting part here. 01:13:40.880 --> 01:13:45.670 And if I terminated the trellis, then I would also 01:13:45.670 --> 01:13:46.360 have an ending time. 01:13:46.360 --> 01:13:49.460 We'll talk about that. 01:13:49.460 --> 01:13:52.250 So at this point, I just want you to see what the trellis 01:13:52.250 --> 01:13:53.090 diagram is. 01:13:53.090 --> 01:13:55.740 It's basically just an unrolling of the state 01:13:55.740 --> 01:13:58.130 transition diagram out in time. 01:13:58.130 --> 01:14:00.310 And it allows me to associate -- 01:14:00.310 --> 01:14:02.580 there's a one-to-one correspondence now between 01:14:02.580 --> 01:14:07.180 code words that start at time k or later, and trellis paths. 01:14:07.180 --> 01:14:11.510 So every one corresponds to a unique, distinct 01:14:11.510 --> 01:14:13.170 path through the code. 01:14:13.170 --> 01:14:16.170 That's what we're going to use in decoding. 01:14:16.170 --> 01:14:20.980 Each of these paths represents one possible decision that the 01:14:20.980 --> 01:14:24.160 decoder could do about what was actually set. 01:14:24.160 --> 01:14:29.230 The decoder is actually going to try to find the best of 01:14:29.230 --> 01:14:32.290 these paths, in some sense, the one that's closest to the 01:14:32.290 --> 01:14:34.810 received sequence, in the Hamming distance or Euclidean 01:14:34.810 --> 01:14:39.360 distance, or likelihood sense. 01:14:39.360 --> 01:14:42.810 Let's pause for a second, and see how this could be used to 01:14:42.810 --> 01:14:48.890 compute the minimum distance of the code. 01:14:48.890 --> 01:14:55.810 Let's assume that I have a non-catastrophic encoder. 01:14:55.810 --> 01:14:58.900 Then I know the finite sequences are going to be 01:14:58.900 --> 01:15:03.510 those that start in the 0 state, go through some 01:15:03.510 --> 01:15:08.170 trajectory through the state transition diagram, and then 01:15:08.170 --> 01:15:11.620 ultimately come back to the 0 state. 01:15:11.620 --> 01:15:15.040 If I have canonical encoder, I agreed what I'm 01:15:15.040 --> 01:15:17.270 going to use now. 01:15:17.270 --> 01:15:21.380 So one way of computing the minimum distance would just be 01:15:21.380 --> 01:15:28.780 to perform a search through here, and try to find the 01:15:28.780 --> 01:15:34.496 minimum detour, the minimum weight detour that starts in 01:15:34.496 --> 01:15:36.990 the 0 state, eventually gets back to the 0 state. 01:15:36.990 --> 01:15:40.080 It accumulates Hamming distance along the way. 01:15:40.080 --> 01:15:41.770 And that is the Hamming distance of 01:15:41.770 --> 01:15:45.480 some valid code sequence. 01:15:45.480 --> 01:15:49.490 So here, it's not very tough. 01:15:49.490 --> 01:15:52.220 And I've already gone through an argument. 01:15:52.220 --> 01:15:55.030 If you didn't know, you'd say, OK, here's the 01:15:55.030 --> 01:15:56.090 first one to look at. 01:15:56.090 --> 01:16:01.740 This is the only one that gets back in three time units. 01:16:01.740 --> 01:16:06.230 Let's look for the finite path that gets back to the 0 state 01:16:06.230 --> 01:16:07.630 in four time units. 01:16:07.630 --> 01:16:09.560 Again, there's only one of them. 01:16:09.560 --> 01:16:13.235 It looks like that, and it turns out it has weight 6. 01:16:15.900 --> 01:16:20.190 So this is, again, it always ends up with a 1:1 when it 01:16:20.190 --> 01:16:22.280 merges back in here. 01:16:22.280 --> 01:16:26.210 And this, I think, is 0, 1 or something like that. 01:16:26.210 --> 01:16:30.950 The only 0, 0 path in here is this one. 01:16:30.950 --> 01:16:34.510 But you can easily see the 0, 0 path doesn't form a loop. 01:16:34.510 --> 01:16:38.160 You can't proceed down 0, 0 forever. 01:16:38.160 --> 01:16:41.280 So I go through here, and I've already got four units of 01:16:41.280 --> 01:16:42.290 distance up to here. 01:16:42.290 --> 01:16:44.440 In fact, I need to follow. 01:16:44.440 --> 01:16:48.900 Next time I explore out here, 1, 1, 0, I've already got up 01:16:48.900 --> 01:16:51.140 to four units of distance. 01:16:51.140 --> 01:16:54.320 Now I know I'm always going to have two at the end. 01:16:54.320 --> 01:16:59.120 So any time I get to four or more, I can stop. 01:16:59.120 --> 01:17:01.830 And by that argument, I'm already done when 01:17:01.830 --> 01:17:03.410 I get to this one. 01:17:03.410 --> 01:17:06.350 Or if I didn't want to use that, I'd just explore -- 01:17:06.350 --> 01:17:09.320 I already know that there's one of weight 5, the impulse 01:17:09.320 --> 01:17:13.480 response, so I'd explore all the possibilities out until 01:17:13.480 --> 01:17:16.450 they accumulated at least weight 5, and assure myself 01:17:16.450 --> 01:17:19.130 that there was no way of getting back here with weight 01:17:19.130 --> 01:17:20.600 less than 5. 01:17:20.600 --> 01:17:24.030 And that would be a very systematic way of evaluating 01:17:24.030 --> 01:17:26.020 the minimum distance to the code. 01:17:26.020 --> 01:17:29.481 You can see you could do that for any generators. 01:17:29.481 --> 01:17:31.020 You with me? 01:17:31.020 --> 01:17:37.620 Just explore this little graph, picking up weight as 01:17:37.620 --> 01:17:38.870 you go along. 01:17:42.480 --> 01:17:45.210 That's basically the way the Viterbi algorithm goes as 01:17:45.210 --> 01:17:54.940 well, except what we do is we have some received pair at 01:17:54.940 --> 01:18:08.730 time k, r1 k, r2 k, r1 k plus 1, r2 k plus 1. 01:18:08.730 --> 01:18:11.920 We get two received symbols at every time, whatever channel 01:18:11.920 --> 01:18:14.300 we're going over. 01:18:14.300 --> 01:18:19.020 Assuming that the channel is memory-less, I get a some kind 01:18:19.020 --> 01:18:21.870 of distance or weight or likelihood weight at each 01:18:21.870 --> 01:18:26.720 time, and maximum likelihood sequence decoding amounts to 01:18:26.720 --> 01:18:30.830 finding the sequence that's closest, the code sequence 01:18:30.830 --> 01:18:33.880 that's closest in Hamming distance or Euclidean distance 01:18:33.880 --> 01:18:36.350 or likelihood distance to the transmitted sequence. 01:18:39.800 --> 01:18:43.180 So I can simply put a metric on each of these branches 01:18:43.180 --> 01:18:45.650 corresponding to what this was. 01:18:45.650 --> 01:18:47.590 Now, if it was a binary symmetric channel and I 01:18:47.590 --> 01:18:51.310 received 0, 0, then I'd have 0 weight here, but weight 2 01:18:51.310 --> 01:18:53.660 here, in terms of Hamming distance from 01:18:53.660 --> 01:18:56.090 the received sequence. 01:18:56.090 --> 01:18:58.470 And similarly, as I go along, I'd have another set of 01:18:58.470 --> 01:18:59.130 metrics here. 01:18:59.130 --> 01:19:02.020 If I received 1, 1, then this one would have weight 2, this 01:19:02.020 --> 01:19:05.160 one would be good, no distance from the received sequence. 01:19:05.160 --> 01:19:06.640 Each of these would have distance 1 from 01:19:06.640 --> 01:19:08.440 the received sequence. 01:19:08.440 --> 01:19:14.680 So I get some kind of cost for every path in here. 01:19:14.680 --> 01:19:16.830 And then the decoding algorithm is simply a matter 01:19:16.830 --> 01:19:20.560 of finding the minimum cost path. 01:19:20.560 --> 01:19:23.380 You could all do that. 01:19:23.380 --> 01:19:29.165 We just had a celebration for Andy Viterbi out in USC, where 01:19:29.165 --> 01:19:33.720 he's given them $50 million, so they were very happy, and 01:19:33.720 --> 01:19:35.060 had a great celebration. 01:19:35.060 --> 01:19:37.970 And the thing he's best known for is the Viterbi algorithm. 01:19:37.970 --> 01:19:41.180 But what everybody knows is that, once you reduce the 01:19:41.180 --> 01:19:43.410 problem to this point, anybody could have come up with the 01:19:43.410 --> 01:19:45.260 Viterbi algorithm. 01:19:45.260 --> 01:19:47.960 The Viterbi algorithm is just a systematic way for trying to 01:19:47.960 --> 01:19:53.620 find the lowest-cost path as you go through a trellis. 01:19:53.620 --> 01:19:56.570 And if you know dynamic programming and it's obvious 01:19:56.570 --> 01:20:02.650 how to do it, or even if you don't, the next idea -- 01:20:02.650 --> 01:20:03.900 do we have a minute? 01:20:06.500 --> 01:20:09.220 There's only one idea in the Viterbi algorithm, which is, 01:20:09.220 --> 01:20:12.280 when you get to this point, you can already make a 01:20:12.280 --> 01:20:15.040 decision about what the best path is to get to each of 01:20:15.040 --> 01:20:18.070 these states. 01:20:18.070 --> 01:20:22.130 If the closest path over the first three intervals is this 01:20:22.130 --> 01:20:28.040 one, not this one, say, then you can forget about this path 01:20:28.040 --> 01:20:30.270 because it's never going to be the first part of the ultimate 01:20:30.270 --> 01:20:33.370 closest path, because we're simply adding up independent 01:20:33.370 --> 01:20:34.620 cost, independent increments. 01:20:37.230 --> 01:20:41.870 So you can make a subdecision at each of these states, what 01:20:41.870 --> 01:20:47.090 the best path is up to that state, the closest path. 01:20:47.090 --> 01:20:49.290 And you only need to remember that. 01:20:49.290 --> 01:20:52.000 That's called a survivor. 01:20:52.000 --> 01:20:56.030 And this was basically, Viterbi was the first one to 01:20:56.030 --> 01:20:59.010 put down this algorithm with the survivor. 01:20:59.010 --> 01:21:01.340 You can make interim decisions. 01:21:01.340 --> 01:21:03.250 You can only keep the survivor. 01:21:03.250 --> 01:21:06.290 Then all you have to do is extend the survivor's one unit 01:21:06.290 --> 01:21:07.280 of time out here. 01:21:07.280 --> 01:21:10.130 Again, you have to add the new metric. 01:21:10.130 --> 01:21:11.840 You have to compare the metrics to see 01:21:11.840 --> 01:21:12.590 which one is better. 01:21:12.590 --> 01:21:15.450 You select the survivor, and once again, you've only got 01:21:15.450 --> 01:21:17.630 four survivors going forward. 01:21:17.630 --> 01:21:20.500 So you get an iterative recursive algorithm that just 01:21:20.500 --> 01:21:26.830 proceeds forward one unit of time at a time, and keeps 01:21:26.830 --> 01:21:29.590 finding the four best survivors, or in general, the 01:21:29.590 --> 01:21:34.630 2 to the nu best survivors at time k plus 3, k plus 4, k 01:21:34.630 --> 01:21:37.450 plus 5, ad infinitum. 01:21:37.450 --> 01:21:40.960 And that's the Viterbi algorithm. 01:21:40.960 --> 01:21:43.480 So once you've drawn this trellis picture, it's very 01:21:43.480 --> 01:21:47.030 clear how you should decode, and you get a very sufficient 01:21:47.030 --> 01:21:49.400 decoding algorithm. 01:21:49.400 --> 01:21:51.490 We'll come back next time. 01:21:51.490 --> 01:21:53.320 I'll do that a little bit more carefully. 01:21:53.320 --> 01:21:56.040 I'll talk about terminated codes. 01:21:56.040 --> 01:21:58.590 I'll talk about how it works when the code isn't 01:21:58.590 --> 01:22:01.880 terminated, and I'll talk about performance analysis. 01:22:01.880 --> 01:22:04.780 But we're actually pretty close to the end of chapter 01:22:04.780 --> 01:22:06.030 nine at this point.