WEBVTT 00:00:00.000 --> 00:00:01.988 align:middle line:90% [SQUEAKING] 00:00:01.988 --> 00:00:02.982 align:middle line:90% [RUSTLING] 00:00:02.982 --> 00:00:04.970 align:middle line:90% [CLICKING] 00:00:04.970 --> 00:00:12.740 align:middle line:90% 00:00:12.740 --> 00:00:17.240 align:middle line:84% PETER SHOR: OK, so I'm going to start with a timeline. 00:00:17.240 --> 00:00:29.050 align:middle line:84% So Shannon, 1948, discovered the noisy coding theorem. 00:00:29.050 --> 00:00:38.280 align:middle line:90% 00:00:38.280 --> 00:00:54.200 align:middle line:84% Hamming discovered Hamming codes in 1950. 00:00:54.200 --> 00:01:05.930 align:middle line:84% Reed and Solomon discovered Reed-Solomon codes in 1960. 00:01:05.930 --> 00:01:17.890 align:middle line:84% Concatenated codes were discovered in 1966. 00:01:17.890 --> 00:01:35.530 align:middle line:84% And Berlekamp-Massey decoding algorithm for Reed-Solomon codes 00:01:35.530 --> 00:01:41.130 align:middle line:90% was discovered in 1969. 00:01:41.130 --> 00:01:44.490 align:middle line:90% So this is a lot of steps. 00:01:44.490 --> 00:01:47.090 align:middle line:84% And it turns out that the Berlekamp-Massey decoding 00:01:47.090 --> 00:01:48.870 align:middle line:84% algorithm for Reed-Solomon codes, 00:01:48.870 --> 00:01:51.570 align:middle line:84% along with concatenated codes, was 00:01:51.570 --> 00:02:02.700 align:middle line:84% what made using error-correcting codes practical. 00:02:02.700 --> 00:02:17.620 align:middle line:84% And in Voyager, deep space was 1977. 00:02:17.620 --> 00:02:20.780 align:middle line:84% And one of the Voyager aircrafts is still 00:02:20.780 --> 00:02:24.340 align:middle line:90% transmitting signals to Earth. 00:02:24.340 --> 00:02:28.900 align:middle line:84% And they're doing that using Reed-Solomon codes, which 00:02:28.900 --> 00:02:34.900 align:middle line:84% is what I'm going to tell you about later in the lecture. 00:02:34.900 --> 00:02:39.260 align:middle line:84% And after all this stuff, Reed-Solomon codes were still 00:02:39.260 --> 00:02:40.890 align:middle line:90% used until-- 00:02:40.890 --> 00:02:44.380 align:middle line:90% 00:02:44.380 --> 00:02:49.380 align:middle line:84% oh, I don't know-- around mid-1990s. 00:02:49.380 --> 00:02:52.680 align:middle line:84% Actually, they were still used until probably around 2000, 00:02:52.680 --> 00:02:57.560 align:middle line:84% but in the mid-1990s, these three French engineers-- 00:02:57.560 --> 00:03:00.520 align:middle line:84% so Reed-Solomon codes do not actually 00:03:00.520 --> 00:03:04.880 align:middle line:84% meet Shannon's bound from the noisy coding theorem. 00:03:04.880 --> 00:03:09.640 align:middle line:84% Reed-Solomon codes come within a constant factor of it, 00:03:09.640 --> 00:03:16.400 align:middle line:84% but between 1948, for the next 40-something years, 00:03:16.400 --> 00:03:22.600 align:middle line:84% coding theorists have struggled to find a technique for actually 00:03:22.600 --> 00:03:25.400 align:middle line:90% meeting Shannon's bound. 00:03:25.400 --> 00:03:32.560 align:middle line:84% And in 1990-something, this paper appeared by three French 00:03:32.560 --> 00:03:35.300 align:middle line:90% engineers. 00:03:35.300 --> 00:03:39.740 align:middle line:84% They weren't even coding theorists. 00:03:39.740 --> 00:03:47.320 align:middle line:84% They were three people at a French telecom company. 00:03:47.320 --> 00:03:50.880 align:middle line:84% And they claimed to meet Shannon's bound 00:03:50.880 --> 00:03:53.690 align:middle line:84% with a completely different technique. 00:03:53.690 --> 00:03:56.890 align:middle line:84% And none of the coding theorist researchers 00:03:56.890 --> 00:04:01.370 align:middle line:84% believed it until they programmed up, and it worked. 00:04:01.370 --> 00:04:05.570 align:middle line:84% And even that-- it took many years for them 00:04:05.570 --> 00:04:07.650 align:middle line:90% to explain why it worked. 00:04:07.650 --> 00:04:09.810 align:middle line:84% And in fact, they still don't understand why it 00:04:09.810 --> 00:04:11.910 align:middle line:90% worked, why turbo codes work. 00:04:11.910 --> 00:04:16.529 align:middle line:84% They do understand why a similar type of codes, called 00:04:16.529 --> 00:04:20.010 align:middle line:90% LDPC codes, work really well. 00:04:20.010 --> 00:04:28.970 align:middle line:84% But so now, after this discovery, 00:04:28.970 --> 00:04:31.690 align:middle line:84% Reed-Solomon codes gradually faded out of use 00:04:31.690 --> 00:04:32.810 align:middle line:90% in new applications. 00:04:32.810 --> 00:04:38.370 align:middle line:84% And turbo codes and LDPC codes and polar codes, 00:04:38.370 --> 00:04:54.134 align:middle line:84% which were discovered in the 2000s, took over. 00:04:54.134 --> 00:04:58.660 align:middle line:84% But I find it amazing that you needed something 00:04:58.660 --> 00:05:04.260 align:middle line:84% like three discoveries to actually make 00:05:04.260 --> 00:05:08.100 align:middle line:90% Shannon's theorem practical. 00:05:08.100 --> 00:05:09.790 align:middle line:90% So what is a Reed-Solomon code? 00:05:09.790 --> 00:05:17.140 align:middle line:90% 00:05:17.140 --> 00:05:21.420 align:middle line:84% OK, well, it's based on an ancient theorem 00:05:21.420 --> 00:05:46.451 align:middle line:84% from mathematics, a polynomial of degree d over a finite field 00:05:46.451 --> 00:05:53.940 align:middle line:90% that has, at most, d roots. 00:05:53.940 --> 00:06:02.350 align:middle line:90% 00:06:02.350 --> 00:06:04.860 align:middle line:90% OK, so a Reed-Solomon code-- 00:06:04.860 --> 00:06:09.710 align:middle line:90% 00:06:09.710 --> 00:06:15.660 align:middle line:84% you have a message, m0 through mk. 00:06:15.660 --> 00:06:22.230 align:middle line:90% 00:06:22.230 --> 00:06:27.870 align:middle line:84% Well, mk minus 1 because we're starting with 0. 00:06:27.870 --> 00:06:43.470 align:middle line:84% Associate the polynomial p of x equals m0 plus m1 x 00:06:43.470 --> 00:06:51.220 align:middle line:84% plus m2 x squared plus mk minus 1 x to the k minus 1. 00:06:51.220 --> 00:06:58.480 align:middle line:90% 00:06:58.480 --> 00:07:01.960 align:middle line:90% So this is the message. 00:07:01.960 --> 00:07:07.380 align:middle line:84% And once you have the message, you construct this polynomial, 00:07:07.380 --> 00:07:16.000 align:middle line:84% although we're not going to actually transmit 00:07:16.000 --> 00:07:16.820 align:middle line:90% the polynomial. 00:07:16.820 --> 00:07:33.256 align:middle line:84% What we're going to transmit is the code word, which is p of 0, 00:07:33.256 --> 00:07:42.600 align:middle line:84% p of 1, p of 2, all the way up through p of n minus 1. 00:07:42.600 --> 00:07:46.736 align:middle line:84% Now, you note, to do this, we need 00:07:46.736 --> 00:07:49.280 align:middle line:90% n is less than or equal to-- 00:07:49.280 --> 00:07:54.060 align:middle line:90% 00:07:54.060 --> 00:07:57.700 align:middle line:84% OK, so this is over a finite field. 00:07:57.700 --> 00:08:01.300 align:middle line:84% And for this lecture, we'll assume the finite field 00:08:01.300 --> 00:08:03.340 align:middle line:90% is the integers mod p. 00:08:03.340 --> 00:08:13.060 align:middle line:84% In real life, Reed-Solomon codes-- 00:08:13.060 --> 00:08:17.100 align:middle line:84% they actually use the polynomial z over the finite field with 2 00:08:17.100 --> 00:08:22.420 align:middle line:84% to the k elements, but that's because they're using binary. 00:08:22.420 --> 00:08:27.780 align:middle line:84% And 2 to the k is very convenient for binary. 00:08:27.780 --> 00:08:31.380 align:middle line:84% But the theory is exactly the same if we take z sub p. 00:08:31.380 --> 00:08:37.700 align:middle line:84% And so for this, well, we need n minus 1 different elements 00:08:37.700 --> 00:08:40.538 align:middle line:90% in the field-- 00:08:40.538 --> 00:08:42.080 align:middle line:84% or n different elements in the field. 00:08:42.080 --> 00:08:49.810 align:middle line:84% And for that, we need n is, at most, p. 00:08:49.810 --> 00:08:56.150 align:middle line:90% 00:08:56.150 --> 00:09:01.950 align:middle line:84% And in fact, as an aside, you don't have to use 0, 1, 2 00:09:01.950 --> 00:09:10.470 align:middle line:84% through n minus 1 for these points. 00:09:10.470 --> 00:09:13.910 align:middle line:84% You could just use any n distinct points, 00:09:13.910 --> 00:09:20.030 align:middle line:84% but the lecture notes start out by saying choose n points. 00:09:20.030 --> 00:09:23.910 align:middle line:84% And then they say later, we'll use the point 0 00:09:23.910 --> 00:09:24.930 align:middle line:90% through n minus 1. 00:09:24.930 --> 00:09:30.270 align:middle line:84% So I'm just going to start out with 0 through n minus 1. 00:09:30.270 --> 00:09:30.770 align:middle line:90% OK. 00:09:30.770 --> 00:09:40.898 align:middle line:90% 00:09:40.898 --> 00:09:56.610 align:middle line:84% And note-- Reed-Solomon's codes are linear. 00:09:56.610 --> 00:09:59.600 align:middle line:90% 00:09:59.600 --> 00:10:05.251 align:middle line:84% And the generator matrix G is equal to 1, 00:10:05.251 --> 00:10:21.160 align:middle line:84% 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 2, 4, 8, et cetera, 00:10:21.160 --> 00:10:30.960 align:middle line:84% 2 to the k minus 1, 1, 3, 9, 27, et cetera, 00:10:30.960 --> 00:10:37.160 align:middle line:84% and 3 to the k minus 1, all the way up to-- 00:10:37.160 --> 00:10:47.170 align:middle line:84% well, how many values of the polynomial are there? 00:10:47.170 --> 00:10:55.710 align:middle line:84% There are-- how many numbers do you plug into the polynomial? 00:10:55.710 --> 00:11:00.850 align:middle line:84% There are n minus 1, so this last column is 1 n minus 1, 00:11:00.850 --> 00:11:08.850 align:middle line:84% n minus 1 squared through n minus 1 to the k minus 1. 00:11:08.850 --> 00:11:14.870 align:middle line:84% And I should note that these numbers are all mod z sub p. 00:11:14.870 --> 00:11:22.070 align:middle line:84% So if p was 13, this entry would not be 27, but 1. 00:11:22.070 --> 00:11:22.570 align:middle line:90% Oh. 00:11:22.570 --> 00:11:28.970 align:middle line:84% Well, I mean 27 is 1 mod 13, but yeah. 00:11:28.970 --> 00:11:35.590 align:middle line:84% So this is the generator matrix G. And if you put in m0, 00:11:35.590 --> 00:11:47.300 align:middle line:84% m1 through mk, the i-th entry and the code word is just 1 plus 00:11:47.300 --> 00:11:52.860 align:middle line:84% m1 times 3 to the 1st plus m2 times 3 to the 2nd plus m3 times 00:11:52.860 --> 00:11:57.610 align:middle line:84% 3 cubed, et cetera, which is the value of p of x at 3. 00:11:57.610 --> 00:12:02.340 align:middle line:90% 00:12:02.340 --> 00:12:07.660 align:middle line:84% So they're linear codes, which means 00:12:07.660 --> 00:12:14.890 align:middle line:84% that we want to find how many errors they can correct. 00:12:14.890 --> 00:12:36.590 align:middle line:90% 00:12:36.590 --> 00:12:39.890 align:middle line:90% So remember-- OK. 00:12:39.890 --> 00:12:53.010 align:middle line:84% So need to find minimum weight nonzero code word. 00:12:53.010 --> 00:12:57.190 align:middle line:90% 00:12:57.190 --> 00:13:04.710 align:middle line:84% If weight-- let's give this a name, c. 00:13:04.710 --> 00:13:11.430 align:middle line:84% If weight of c is equal to d, we can correct 00:13:11.430 --> 00:13:14.560 align:middle line:90% d minus 1 over 2 errors. 00:13:14.560 --> 00:13:28.110 align:middle line:90% 00:13:28.110 --> 00:13:36.290 align:middle line:84% Theorem-- and that's because the minimum distance between any two 00:13:36.290 --> 00:13:39.750 align:middle line:90% code words is d minus 1 over 2. 00:13:39.750 --> 00:13:43.130 align:middle line:90% 00:13:43.130 --> 00:13:59.410 align:middle line:84% Minimum weight non-zero code word 00:13:59.410 --> 00:14:09.230 align:middle line:84% has Hamming weight n minus k plus 1. 00:14:09.230 --> 00:14:19.850 align:middle line:90% 00:14:19.850 --> 00:14:26.450 align:middle line:90% Proof-- the code word-- 00:14:26.450 --> 00:14:28.770 align:middle line:84% well, we know what the code word is. 00:14:28.770 --> 00:14:50.940 align:middle line:84% It's just p of 0, p of 1 through p of n minus 1 00:14:50.940 --> 00:15:01.100 align:middle line:84% where p equals m0 plus m1 x plus plus mk minus 1 x 00:15:01.100 --> 00:15:11.000 align:middle line:84% to the k minus 1 is a polynomial of degree k. 00:15:11.000 --> 00:15:14.900 align:middle line:90% 00:15:14.900 --> 00:15:18.320 align:middle line:84% And remember that this polynomial-- 00:15:18.320 --> 00:15:20.780 align:middle line:84% actually, the coefficients are the message. 00:15:20.780 --> 00:15:25.440 align:middle line:84% So it could be any polynomial of degree k minus 1. 00:15:25.440 --> 00:15:29.310 align:middle line:90% 00:15:29.310 --> 00:15:34.430 align:middle line:84% And now, we use this theorem that I reminded you a little 00:15:34.430 --> 00:15:38.990 align:middle line:84% while ago-- a polynomial of degree d has, at most, d roots. 00:15:38.990 --> 00:15:51.100 align:middle line:84% So p of x has, at most, k minus 1 0s. 00:15:51.100 --> 00:15:56.670 align:middle line:90% 00:15:56.670 --> 00:15:59.470 align:middle line:84% So at most, k minus 1 of these entries 00:15:59.470 --> 00:16:21.110 align:middle line:84% are 0, which means at least n minus k plus 1 values of p sub 00:16:21.110 --> 00:16:24.263 align:middle line:90% i are nonzero. 00:16:24.263 --> 00:16:30.320 align:middle line:90% 00:16:30.320 --> 00:16:36.320 align:middle line:84% Because you have n values in the code word, 00:16:36.320 --> 00:16:39.460 align:middle line:84% k minus 1 of them or fewer are 0, 00:16:39.460 --> 00:16:43.160 align:middle line:84% which means this many have to be nonzero. 00:16:43.160 --> 00:16:57.590 align:middle line:84% So the minimum Hamming weight is n minus k plus 1. 00:16:57.590 --> 00:17:09.640 align:middle line:90% 00:17:09.640 --> 00:17:21.650 align:middle line:84% Suppose we want to correct e errors. 00:17:21.650 --> 00:17:24.200 align:middle line:90% 00:17:24.200 --> 00:17:32.610 align:middle line:84% Well, we need minimum distance to be greater than or equal 00:17:32.610 --> 00:17:36.890 align:middle line:90% to 2e plus 1. 00:17:36.890 --> 00:17:45.970 align:middle line:84% And that means we need n minus k plus 1 greater than or equal 00:17:45.970 --> 00:17:51.910 align:middle line:84% to 2e plus 1 or n minus k is greater than or equal to 2e. 00:17:51.910 --> 00:17:55.930 align:middle line:90% 00:17:55.930 --> 00:17:56.622 align:middle line:90% OK. 00:17:56.622 --> 00:18:00.220 align:middle line:84% So that are our Reed-Solomon codes. 00:18:00.220 --> 00:18:03.010 align:middle line:90% 00:18:03.010 --> 00:18:14.650 align:middle line:84% Now, remember that to have a usable code, 00:18:14.650 --> 00:18:20.410 align:middle line:84% we not only need to be able to encode, 00:18:20.410 --> 00:18:25.110 align:middle line:84% but we also need to be able to correct errors and decode. 00:18:25.110 --> 00:18:31.070 align:middle line:84% So the question is how do you decode? 00:18:31.070 --> 00:18:33.110 align:middle line:84% So this is what I will spend probably 00:18:33.110 --> 00:18:37.030 align:middle line:90% the next half hour telling you. 00:18:37.030 --> 00:18:37.530 align:middle line:90% OK. 00:18:37.530 --> 00:18:43.302 align:middle line:90% 00:18:43.302 --> 00:18:52.430 align:middle line:84% And I am not going to give you the Berlekamp-Massey decoding 00:18:52.430 --> 00:18:58.750 align:middle line:84% algorithm because that is conceptually very-- 00:18:58.750 --> 00:19:01.830 align:middle line:84% or I shouldn't say "very"-- that's conceptually fairly 00:19:01.830 --> 00:19:07.150 align:middle line:84% difficult. And I'm going to give you a much easier decoding 00:19:07.150 --> 00:19:09.710 align:middle line:84% algorithm to explain, which is not as efficient 00:19:09.710 --> 00:19:12.830 align:middle line:84% as the Berlekamp-Massey decoding algorithm. 00:19:12.830 --> 00:19:16.590 align:middle line:84% And there's another decoding algorithm, which is probably 00:19:16.590 --> 00:19:21.600 align:middle line:84% equally efficient, which is based on the Euclidean algorithm 00:19:21.600 --> 00:19:25.397 align:middle line:84% for finding greatest common divisors, 00:19:25.397 --> 00:19:27.980 align:middle line:84% but I'm not going to give you that one, either, because again, 00:19:27.980 --> 00:19:31.020 align:middle line:84% that's fairly difficult, conceptually. 00:19:31.020 --> 00:19:51.640 align:middle line:90% 00:19:51.640 --> 00:19:55.530 align:middle line:90% How do we decode? 00:19:55.530 --> 00:19:58.080 align:middle line:90% 00:19:58.080 --> 00:20:01.410 align:middle line:84% So this decoding algorithm is based on a theorem. 00:20:01.410 --> 00:20:16.400 align:middle line:90% 00:20:16.400 --> 00:20:32.770 align:middle line:84% So theorem-- suppose we have a code word, 00:20:32.770 --> 00:20:39.550 align:middle line:84% c equals p of 0, p of 1 through p of n minus 1. 00:20:39.550 --> 00:20:42.850 align:middle line:90% 00:20:42.850 --> 00:20:57.330 align:middle line:84% We get the transmission, or rather, we receive the code word 00:20:57.330 --> 00:21:00.410 align:middle line:90% with some errors in it. 00:21:00.410 --> 00:21:08.250 align:middle line:84% c tilde equals r0, r1, r2 through rn minus 1. 00:21:08.250 --> 00:21:14.490 align:middle line:84% And these guys are not going to have a polynomial where 00:21:14.490 --> 00:21:16.650 align:middle line:90% ri equals p of i. 00:21:16.650 --> 00:21:25.300 align:middle line:84% For some of these, we have ri equals p of i, 00:21:25.300 --> 00:21:27.150 align:middle line:90% but for some others, we won't. 00:21:27.150 --> 00:21:32.620 align:middle line:90% 00:21:32.620 --> 00:21:40.610 align:middle line:84% Assume there are i less than e errors. 00:21:40.610 --> 00:21:48.180 align:middle line:90% 00:21:48.180 --> 00:22:05.900 align:middle line:84% Then there are positions i1, i2 through il, where p of i 00:22:05.900 --> 00:22:09.700 align:middle line:90% sub l is not equal to r-- 00:22:09.700 --> 00:22:13.840 align:middle line:90% 00:22:13.840 --> 00:22:24.270 align:middle line:84% p sub i sub k is not equal to r sub i sub k. 00:22:24.270 --> 00:22:26.430 align:middle line:90% So we have, at most, i errors. 00:22:26.430 --> 00:22:30.620 align:middle line:84% And we'll assume they're in positions i1 through il. 00:22:30.620 --> 00:22:33.630 align:middle line:90% 00:22:33.630 --> 00:22:37.230 align:middle line:84% And l is less than or equal to e because we're assuming 00:22:37.230 --> 00:22:41.820 align:middle line:84% there are fewer errors than the code is able to decode. 00:22:41.820 --> 00:22:44.790 align:middle line:90% 00:22:44.790 --> 00:22:47.430 align:middle line:84% So this is the hypothesis of the theorem. 00:22:47.430 --> 00:22:50.300 align:middle line:84% And I need to put the theorem itself on a different board. 00:22:50.300 --> 00:22:59.190 align:middle line:90% 00:22:59.190 --> 00:23:35.010 align:middle line:84% Then-- continued-- there are nonzero polynomials f and q 00:23:35.010 --> 00:23:51.650 align:middle line:84% and q of x of degree less than or equal to k plus e minus 1 00:23:51.650 --> 00:24:02.030 align:middle line:84% such that q of i is equal to r sub i times f of i. 00:24:02.030 --> 00:24:06.040 align:middle line:90% 00:24:06.040 --> 00:24:06.540 align:middle line:90% OK. 00:24:06.540 --> 00:24:09.240 align:middle line:90% 00:24:09.240 --> 00:24:12.560 align:middle line:84% So this looks like a totally crazy theorem. 00:24:12.560 --> 00:24:15.380 align:middle line:84% What does it have to do with decoding? 00:24:15.380 --> 00:24:21.140 align:middle line:84% Well, you will see what it has to do 00:24:21.140 --> 00:24:23.552 align:middle line:84% with decoding when I give you the decoding algorithm, 00:24:23.552 --> 00:24:25.010 align:middle line:84% but first, let's prove the theorem. 00:24:25.010 --> 00:24:31.940 align:middle line:90% 00:24:31.940 --> 00:24:46.900 align:middle line:84% Let f of x equal x minus i1 x minus i2 through x minus i sub l 00:24:46.900 --> 00:24:51.680 align:middle line:84% is equal to the product of x minus i sub k, 00:24:51.680 --> 00:24:55.060 align:middle line:90% k equals 1 through l. 00:24:55.060 --> 00:25:00.460 align:middle line:84% So this is-- you just take all the positions 00:25:00.460 --> 00:25:02.260 align:middle line:90% which are in error. 00:25:02.260 --> 00:25:05.380 align:middle line:84% Of course, you don't know which positions are in error when 00:25:05.380 --> 00:25:13.990 align:middle line:84% you receive this word, which is the code word with the most e 00:25:13.990 --> 00:25:19.630 align:middle line:84% errors, but we'll let f of x be that. 00:25:19.630 --> 00:25:30.010 align:middle line:84% And we will let q of x equals p of x times f of x. 00:25:30.010 --> 00:25:58.490 align:middle line:90% 00:25:58.490 --> 00:25:58.990 align:middle line:90% OK. 00:25:58.990 --> 00:26:02.790 align:middle line:90% 00:26:02.790 --> 00:26:03.750 align:middle line:90% Oh, I think we-- 00:26:03.750 --> 00:26:15.360 align:middle line:90% 00:26:15.360 --> 00:26:15.860 align:middle line:90% ah. 00:26:15.860 --> 00:26:19.080 align:middle line:90% 00:26:19.080 --> 00:26:21.480 align:middle line:90% I should have put this down. 00:26:21.480 --> 00:26:30.480 align:middle line:84% q of i equals r sub i, f sub i for i equals 0, 1 00:26:30.480 --> 00:26:32.680 align:middle line:90% through n minus 1. 00:26:32.680 --> 00:26:39.880 align:middle line:84% So this is only true at the positions which 00:26:39.880 --> 00:26:43.240 align:middle line:90% give you values in the code. 00:26:43.240 --> 00:26:48.080 align:middle line:84% And of course, this r sub i is not 00:26:48.080 --> 00:26:54.760 align:middle line:84% defined for any position other than 0, 1 through n minus 1. 00:26:54.760 --> 00:27:02.180 align:middle line:84% So this condition is probably a little bit unnecessary, 00:27:02.180 --> 00:27:04.810 align:middle line:84% but still, I'm going to write it. 00:27:04.810 --> 00:27:26.570 align:middle line:84% So if i is in i0 i1 through i sub l, then f of i 00:27:26.570 --> 00:27:37.130 align:middle line:84% equals 0 because f of x is just the product of x minus i sub 00:27:37.130 --> 00:27:44.490 align:middle line:90% k, where i is in this set. 00:27:44.490 --> 00:27:57.410 align:middle line:84% And q of x equals p of x, f of x equals 0, 00:27:57.410 --> 00:28:03.180 align:middle line:84% or q of i equals p of I times f of i equals 0. 00:28:03.180 --> 00:28:06.950 align:middle line:90% 00:28:06.950 --> 00:28:13.070 align:middle line:84% So we wanted to show that q of i equals r sub i times f of i. 00:28:13.070 --> 00:28:25.270 align:middle line:84% q of i equals r sub i times f of i equals 0. 00:28:25.270 --> 00:28:30.170 align:middle line:84% Because remember-- q of x was p of x times f of x. 00:28:30.170 --> 00:28:30.670 align:middle line:90% OK? 00:28:30.670 --> 00:28:36.670 align:middle line:84% So the other alternative is if i is not 00:28:36.670 --> 00:28:48.110 align:middle line:84% in the set of positions where things are in error, then, 00:28:48.110 --> 00:28:56.370 align:middle line:84% well, if there are no errors, then r sub i equals p of i. 00:28:56.370 --> 00:28:59.750 align:middle line:90% 00:28:59.750 --> 00:29:09.240 align:middle line:84% And q of i is equal to p of i times r sub i. 00:29:09.240 --> 00:29:26.104 align:middle line:90% 00:29:26.104 --> 00:29:33.920 align:middle line:84% q of i equals p of i times f of i, which is equal to-- 00:29:33.920 --> 00:29:40.060 align:middle line:84% since ri equals p of i, it's r sub i times f of i. 00:29:40.060 --> 00:29:44.440 align:middle line:90% 00:29:44.440 --> 00:29:49.120 align:middle line:84% And that's what we wanted-- q of i equals r sub i times f of i. 00:29:49.120 --> 00:29:53.120 align:middle line:84% So whether or not i is in the position containing the errors, 00:29:53.120 --> 00:29:57.960 align:middle line:84% q of i equals p of i times f of i. 00:29:57.960 --> 00:30:00.410 align:middle line:90% OK, so that's the theorem. 00:30:00.410 --> 00:30:07.370 align:middle line:84% We have these two polynomials, f of x 00:30:07.370 --> 00:30:11.590 align:middle line:84% and q of x, which have this relation between them. 00:30:11.590 --> 00:30:16.450 align:middle line:90% 00:30:16.450 --> 00:30:19.160 align:middle line:90% How do we decode? 00:30:19.160 --> 00:30:23.090 align:middle line:90% 00:30:23.090 --> 00:30:24.200 align:middle line:90% Linear algebra. 00:30:24.200 --> 00:30:28.250 align:middle line:90% 00:30:28.250 --> 00:30:32.210 align:middle line:84% So the lecture notes give you an explicit example 00:30:32.210 --> 00:30:35.690 align:middle line:84% of how to decode, but I am not going 00:30:35.690 --> 00:30:41.930 align:middle line:84% to go through it in class because it's really 00:30:41.930 --> 00:30:45.350 align:middle line:90% fairly tedious to go through. 00:30:45.350 --> 00:30:48.560 align:middle line:84% And I will just explain the theory. 00:30:48.560 --> 00:30:52.850 align:middle line:90% 00:30:52.850 --> 00:30:57.460 align:middle line:84% So we have q of i equals r sub i times f of i. 00:30:57.460 --> 00:31:01.900 align:middle line:84% So this is a whole bunch of equations. q of 0 equals r 00:31:01.900 --> 00:31:04.260 align:middle line:90% 0 times f of 0. 00:31:04.260 --> 00:31:08.660 align:middle line:90% q of 1 equals r1 times f of 1. 00:31:08.660 --> 00:31:14.140 align:middle line:84% q of 2 equals r2 times f of 2 all the way down 00:31:14.140 --> 00:31:25.780 align:middle line:84% to q of n minus 1 equals r sub n minus 1 times f of n minus 1. 00:31:25.780 --> 00:31:33.220 align:middle line:84% So you can think of these equations as linear equations 00:31:33.220 --> 00:31:35.440 align:middle line:84% in the coefficients of the polynomials. 00:31:35.440 --> 00:31:46.460 align:middle line:90% 00:31:46.460 --> 00:31:52.380 align:middle line:90% And OK. 00:31:52.380 --> 00:31:58.030 align:middle line:84% So what do they look like in terms of the linear equations? 00:31:58.030 --> 00:32:01.190 align:middle line:90% Well, q of 0-- 00:32:01.190 --> 00:32:04.950 align:middle line:84% well, I should probably say what the coefficients are. 00:32:04.950 --> 00:32:22.070 align:middle line:84% q of x equals q0 plus q1x plus plus Q sub k plus e minus 1 x 00:32:22.070 --> 00:32:25.630 align:middle line:90% to the k plus e minus 1. 00:32:25.630 --> 00:32:33.990 align:middle line:84% And f of x equals f0 plus f1 x plus plus. 00:32:33.990 --> 00:32:35.490 align:middle line:90% So this has degree e. 00:32:35.490 --> 00:32:39.990 align:middle line:90% 00:32:39.990 --> 00:32:45.530 align:middle line:90% So it's f sub e x to the e. 00:32:45.530 --> 00:32:48.350 align:middle line:90% 00:32:48.350 --> 00:32:48.850 align:middle line:90% OK. 00:32:48.850 --> 00:32:57.158 align:middle line:90% 00:32:57.158 --> 00:32:59.600 align:middle line:84% So what do these equations look like? 00:32:59.600 --> 00:33:03.080 align:middle line:84% Well, the first one, q of 0, is just q0. 00:33:03.080 --> 00:33:06.570 align:middle line:90% So it's q0 equals r0 f0. 00:33:06.570 --> 00:33:09.440 align:middle line:90% 00:33:09.440 --> 00:33:11.520 align:middle line:90% The next one is q of 1. 00:33:11.520 --> 00:33:18.720 align:middle line:84% So it's just q0 plus q1 plus q2 plus dot, dot, dot plus q-- 00:33:18.720 --> 00:33:19.920 align:middle line:90% and what was the degree? 00:33:19.920 --> 00:33:30.760 align:middle line:84% k plus e minus 1 is equal to r1 f0 plus f1 plus dot, dot, dot 00:33:30.760 --> 00:33:32.840 align:middle line:90% plus f sub e. 00:33:32.840 --> 00:33:36.340 align:middle line:90% And the next one is f of 2. 00:33:36.340 --> 00:33:44.480 align:middle line:84% So it's q0 plus 4 plus 2 q1 plus 4 q2 plus dot, dot, dot 00:33:44.480 --> 00:33:52.800 align:middle line:84% plus 2 to the k plus e minus 1 q sub k plus e minus 1. 00:33:52.800 --> 00:34:02.300 align:middle line:84% Of course, this is mod p, so the numbers 00:34:02.300 --> 00:34:04.820 align:middle line:90% don't actually grow huge. 00:34:04.820 --> 00:34:20.860 align:middle line:84% And this equals r sub 2 f of 0 plus 2 f1 plus dot, dot, dot 00:34:20.860 --> 00:34:24.159 align:middle line:90% plus 2 to the e f sub e. 00:34:24.159 --> 00:34:31.820 align:middle line:90% 00:34:31.820 --> 00:34:33.179 align:middle line:90% OK. 00:34:33.179 --> 00:34:43.060 align:middle line:84% I'm going to claim this is a linear equation in coefficients 00:34:43.060 --> 00:34:46.580 align:middle line:90% of q and r. 00:34:46.580 --> 00:34:48.940 align:middle line:84% Well, what are the coefficients of q and r? 00:34:48.940 --> 00:34:55.230 align:middle line:84% Well, there's q0, q1, q2, 2k plus e minus 1, and f0, 00:34:55.230 --> 00:34:59.950 align:middle line:90% f1 through f sub e. 00:34:59.950 --> 00:35:03.850 align:middle line:84% And it's clearly a linear equation in the coefficients. 00:35:03.850 --> 00:35:41.510 align:middle line:90% 00:35:41.510 --> 00:35:45.640 align:middle line:90% So linear algebra-- 00:35:45.640 --> 00:35:51.000 align:middle line:90% 00:35:51.000 --> 00:35:51.932 align:middle line:90% OK. 00:35:51.932 --> 00:35:55.200 align:middle line:84% We have a bunch of linear equations 00:35:55.200 --> 00:35:57.560 align:middle line:90% and a bunch of variables. 00:35:57.560 --> 00:35:59.090 align:middle line:90% How many solutions do they have? 00:35:59.090 --> 00:36:18.080 align:middle line:90% 00:36:18.080 --> 00:36:28.800 align:middle line:90% Well, let's see. 00:36:28.800 --> 00:36:30.220 align:middle line:90% How many variables? 00:36:30.220 --> 00:36:40.400 align:middle line:90% 00:36:40.400 --> 00:36:43.720 align:middle line:84% Well, there are e plus 1 variables 00:36:43.720 --> 00:36:47.610 align:middle line:84% in f of x because there are e plus 1 coefficients. 00:36:47.610 --> 00:36:52.690 align:middle line:84% And there are k plus e minus 1 plus 1 variables in q 00:36:52.690 --> 00:36:58.210 align:middle line:84% of x because there are k plus e coefficients. 00:36:58.210 --> 00:37:01.290 align:middle line:90% So how many variables? 00:37:01.290 --> 00:37:12.870 align:middle line:84% There are k plus e plus E plus 1. 00:37:12.870 --> 00:37:18.810 align:middle line:90% 00:37:18.810 --> 00:37:21.836 align:middle line:90% How many equations? 00:37:21.836 --> 00:37:25.570 align:middle line:90% 00:37:25.570 --> 00:37:31.010 align:middle line:84% Well, there are n equations because we have one 00:37:31.010 --> 00:37:34.010 align:middle line:84% for q0, q1, all the way up to q of n minus 1. 00:37:34.010 --> 00:37:37.330 align:middle line:90% 00:37:37.330 --> 00:37:49.580 align:middle line:84% n equations, but somewhere, we have n minus k equals 2e. 00:37:49.580 --> 00:38:00.460 align:middle line:90% 00:38:00.460 --> 00:38:05.500 align:middle line:84% And up there we said we needed n minus k greater than or equal 00:38:05.500 --> 00:38:09.500 align:middle line:84% to 2e to correct e errors, but let's assume that n minus k 00:38:09.500 --> 00:38:17.060 align:middle line:84% equals 2e because that's the minimum we need to correct e 00:38:17.060 --> 00:38:18.820 align:middle line:90% errors. 00:38:18.820 --> 00:38:21.860 align:middle line:90% So n minus k equals 2e. 00:38:21.860 --> 00:38:32.700 align:middle line:84% So we have k plus 2e plus 1 variables and k plus 2e 00:38:32.700 --> 00:38:35.380 align:middle line:90% equations. 00:38:35.380 --> 00:38:39.540 align:middle line:84% And these equations are all linearly independent. 00:38:39.540 --> 00:38:40.830 align:middle line:90% And how do you see that? 00:38:40.830 --> 00:38:45.480 align:middle line:90% 00:38:45.480 --> 00:38:50.330 align:middle line:84% OK, I'm not going to really explain this in full detail. 00:38:50.330 --> 00:39:00.560 align:middle line:90% 00:39:00.560 --> 00:39:09.091 align:middle line:84% But remember G is equal to 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 2, 00:39:09.091 --> 00:39:20.880 align:middle line:84% 4, 8, 2 to the n minus 1, 1, 3, 9, 00:39:20.880 --> 00:39:27.000 align:middle line:84% 27 through 3 to the n minus 1, et cetera. 00:39:27.000 --> 00:39:29.510 align:middle line:90% So this is a Vandermonde matrix. 00:39:29.510 --> 00:39:37.720 align:middle line:90% 00:39:37.720 --> 00:39:50.400 align:middle line:84% And Vandermonde matrices have full rank. 00:39:50.400 --> 00:39:55.770 align:middle line:90% 00:39:55.770 --> 00:39:59.730 align:middle line:84% And this is a theorem which we could easily have proved at some 00:39:59.730 --> 00:40:04.130 align:middle line:84% point in our course, but we didn't. 00:40:04.130 --> 00:40:06.770 align:middle line:84% And I'm not going to prove it today, 00:40:06.770 --> 00:40:15.890 align:middle line:84% but this is the generating matrix for our code. 00:40:15.890 --> 00:40:17.930 align:middle line:90% And it has full rank. 00:40:17.930 --> 00:40:23.570 align:middle line:84% And by using that fact and a similar fact on the right-hand 00:40:23.570 --> 00:40:30.690 align:middle line:84% side for the f's, you can show that these linear equations have 00:40:30.690 --> 00:40:40.180 align:middle line:84% full rank, which means that we have n minus k equals-- 00:40:40.180 --> 00:40:46.420 align:middle line:84% we have k plus 2e linearly independent equations and k plus 00:40:46.420 --> 00:40:48.980 align:middle line:90% 2e plus 1 variables. 00:40:48.980 --> 00:41:02.280 align:middle line:84% And that means there exists a nonzero solution. 00:41:02.280 --> 00:41:08.300 align:middle line:90% 00:41:08.300 --> 00:41:17.700 align:middle line:84% And in fact, you can use the fact that s sub 00:41:17.700 --> 00:41:23.340 align:middle line:90% e equals 1 because f was-- 00:41:23.340 --> 00:41:26.330 align:middle line:90% yeah, f was this. 00:41:26.330 --> 00:41:29.300 align:middle line:90% 00:41:29.300 --> 00:41:31.840 align:middle line:90% And so f sub e equals 1. 00:41:31.840 --> 00:41:34.620 align:middle line:90% 00:41:34.620 --> 00:41:54.350 align:middle line:84% Use f sub e equals 1 to get a unique solution, assuming l 00:41:54.350 --> 00:41:55.210 align:middle line:90% equals e. 00:41:55.210 --> 00:41:58.310 align:middle line:90% 00:41:58.310 --> 00:42:00.270 align:middle line:84% OK, I don't know what happens when 00:42:00.270 --> 00:42:03.230 align:middle line:84% l is-- the number of actual errors 00:42:03.230 --> 00:42:05.330 align:middle line:84% is smaller than e, the maximum number of errors 00:42:05.330 --> 00:42:08.150 align:middle line:90% you can correct. 00:42:08.150 --> 00:42:10.830 align:middle line:84% In other error correction algorithms 00:42:10.830 --> 00:42:15.030 align:middle line:84% for Reed-Solomon codes, you basically 00:42:15.030 --> 00:42:21.270 align:middle line:84% have to go through all possibilities of how 00:42:21.270 --> 00:42:25.270 align:middle line:84% big l is and check each one out, except there are usually 00:42:25.270 --> 00:42:28.390 align:middle line:84% much simpler ways to do that than going through all 00:42:28.390 --> 00:42:32.960 align:middle line:90% of these possibilities. 00:42:32.960 --> 00:42:33.732 align:middle line:90% OK. 00:42:33.732 --> 00:42:43.280 align:middle line:84% So we have one solution to all of these equations. 00:42:43.280 --> 00:42:49.400 align:middle line:84% And that means we can figure out what q of x and f 00:42:49.400 --> 00:42:56.026 align:middle line:84% of x are because we know that q of x and-- 00:42:56.026 --> 00:42:58.560 align:middle line:84% sorry, q of x and p of x are because we 00:42:58.560 --> 00:43:02.670 align:middle line:84% know that q of x and p of x satisfy these-- 00:43:02.670 --> 00:43:08.560 align:middle line:90% 00:43:08.560 --> 00:43:09.100 align:middle line:90% I'm sorry. 00:43:09.100 --> 00:43:12.600 align:middle line:84% We can figure out what f of x and q of x 00:43:12.600 --> 00:43:21.320 align:middle line:84% are because we had linear equations for the coefficients 00:43:21.320 --> 00:43:23.560 align:middle line:90% of q of x and f of x. 00:43:23.560 --> 00:43:26.100 align:middle line:84% So we can deduce q of x and f of x. 00:43:26.100 --> 00:43:40.060 align:middle line:90% 00:43:40.060 --> 00:43:51.060 align:middle line:84% So linear algebra gives q of x and f of x. 00:43:51.060 --> 00:43:54.420 align:middle line:90% And we had p of x-- 00:43:54.420 --> 00:43:57.750 align:middle line:90% this was the message-- 00:43:57.750 --> 00:44:03.340 align:middle line:90% times f of x equals q of x. 00:44:03.340 --> 00:44:12.260 align:middle line:84% So p of x equals q of x over f of x. 00:44:12.260 --> 00:44:14.230 align:middle line:84% And we have gotten the message back. 00:44:14.230 --> 00:44:19.991 align:middle line:90% 00:44:19.991 --> 00:44:20.491 align:middle line:90% OK. 00:44:20.491 --> 00:44:23.788 align:middle line:90% 00:44:23.788 --> 00:44:29.100 align:middle line:84% So that's how to decode Reed-Solomon codes. 00:44:29.100 --> 00:44:31.870 align:middle line:84% The last thing I want to do is tell you 00:44:31.870 --> 00:44:41.910 align:middle line:84% about concatenated codes and why these make Reed-Solomon codes 00:44:41.910 --> 00:44:43.130 align:middle line:90% truly practical. 00:44:43.130 --> 00:45:02.310 align:middle line:90% 00:45:02.310 --> 00:45:14.480 align:middle line:84% So we have n equals k plus 2e for Reed-Solomon codes. 00:45:14.480 --> 00:45:21.830 align:middle line:90% 00:45:21.830 --> 00:45:35.400 align:middle line:84% And let's assume the Reed-Solomon codes over 00:45:35.400 --> 00:45:41.400 align:middle line:84% the finite field with 2 to the 8th elements. 00:45:41.400 --> 00:45:46.560 align:middle line:84% So I don't know whether Professor Moitra mentioned it 00:45:46.560 --> 00:45:48.320 align:middle line:84% before, but there's a finite field 00:45:48.320 --> 00:45:51.240 align:middle line:90% for every power of a prime. 00:45:51.240 --> 00:45:55.680 align:middle line:84% And one of the very common finite fields used 00:45:55.680 --> 00:45:58.390 align:middle line:84% for Reed-Solomon codes in practice is 2 to the 8th. 00:45:58.390 --> 00:46:02.000 align:middle line:90% 00:46:02.000 --> 00:46:09.700 align:middle line:90% So let's say n equals 256. 00:46:09.700 --> 00:46:13.240 align:middle line:90% That's 2 to the 8th. 00:46:13.240 --> 00:46:17.840 align:middle line:90% k equals 150. 00:46:17.840 --> 00:46:30.010 align:middle line:84% And that implies that e equals n minus k over 2 because we had n 00:46:30.010 --> 00:46:32.370 align:middle line:90% minus k equals 2e. 00:46:32.370 --> 00:46:37.150 align:middle line:84% And 256 negative 150 over 2 is 53. 00:46:37.150 --> 00:46:41.210 align:middle line:90% 00:46:41.210 --> 00:46:49.890 align:middle line:84% Can encode-- let's see, k was 150-- 00:46:49.890 --> 00:47:04.410 align:middle line:84% 150 bytes, if we assume a byte is 8 bits, into 256 bytes, 00:47:04.410 --> 00:47:13.730 align:middle line:84% correct 53 errors, which means 53 is the number of bytes 00:47:13.730 --> 00:47:15.230 align:middle line:90% that can have errors in them. 00:47:15.230 --> 00:47:19.890 align:middle line:90% 00:47:19.890 --> 00:47:30.500 align:middle line:84% And if you think about this, in general, 00:47:30.500 --> 00:47:43.460 align:middle line:84% cannot correct an error rate greater than n over 2 00:47:43.460 --> 00:47:47.020 align:middle line:84% because the number of errors you can correct is n minus k over 2. 00:47:47.020 --> 00:47:49.620 align:middle line:84% And even if you use the smallest possible code 00:47:49.620 --> 00:47:53.020 align:middle line:90% which encodes a single byte-- 00:47:53.020 --> 00:47:58.630 align:middle line:84% that's 8 bits-- you cannot correct more than n over 2 00:47:58.630 --> 00:47:59.130 align:middle line:90% errors. 00:47:59.130 --> 00:48:01.980 align:middle line:90% 00:48:01.980 --> 00:48:02.480 align:middle line:90% OK. 00:48:02.480 --> 00:48:22.600 align:middle line:90% 00:48:22.600 --> 00:48:49.560 align:middle line:84% Let's assume the bit error rate is 10%. 00:48:49.560 --> 00:48:52.160 align:middle line:84% So you're transmitting 0s and 1s. 00:48:52.160 --> 00:48:57.920 align:middle line:84% And you're grouping them into blocks of 8 bits, 00:48:57.920 --> 00:49:07.630 align:middle line:84% but each of your bit is likely to have a 10% error. 00:49:07.630 --> 00:49:10.640 align:middle line:90% 00:49:10.640 --> 00:49:18.600 align:middle line:84% Probability of byte-- that's a group of 8 bits-- 00:49:18.600 --> 00:49:41.930 align:middle line:84% is in error-- is 1 minus 1 minus 1/10 to the 8th, OK, 00:49:41.930 --> 00:49:43.815 align:middle line:90% which equals 57%. 00:49:43.815 --> 00:49:48.990 align:middle line:90% 00:49:48.990 --> 00:49:51.570 align:middle line:84% This is the probability that each of your bits 00:49:51.570 --> 00:49:53.690 align:middle line:90% is not an error. 00:49:53.690 --> 00:49:58.070 align:middle line:90% So you raise it to the 8th. 00:49:58.070 --> 00:50:00.770 align:middle line:84% That's the probability that all of your bits 00:50:00.770 --> 00:50:02.250 align:middle line:90% do not have an error. 00:50:02.250 --> 00:50:04.750 align:middle line:84% And the probability of an error is 1 minus this. 00:50:04.750 --> 00:50:06.490 align:middle line:90% That's 57%. 00:50:06.490 --> 00:50:10.450 align:middle line:84% So if you're trying to use a Reed-Solomon code with blocks 00:50:10.450 --> 00:50:18.260 align:middle line:84% of size 8, you cannot find parameters that will let it 00:50:18.260 --> 00:50:19.280 align:middle line:90% correct the error. 00:50:19.280 --> 00:50:27.746 align:middle line:90% 00:50:27.746 --> 00:50:29.920 align:middle line:84% And I want to say what do you do? 00:50:29.920 --> 00:50:31.490 align:middle line:90% You use concatenated codes. 00:50:31.490 --> 00:50:56.260 align:middle line:90% 00:50:56.260 --> 00:51:00.850 align:middle line:90% You can look in code tables. 00:51:00.850 --> 00:51:04.380 align:middle line:90% 00:51:04.380 --> 00:51:07.980 align:middle line:84% And these are available on the web. 00:51:07.980 --> 00:51:23.560 align:middle line:84% Find that there is a 19, 8, 7 code. 00:51:23.560 --> 00:51:29.510 align:middle line:90% 00:51:29.510 --> 00:51:40.150 align:middle line:84% This encodes 8 bits into 19 bits. 00:51:40.150 --> 00:51:46.110 align:middle line:84% And the distance is equal to 7, which 00:51:46.110 --> 00:51:47.720 align:middle line:90% means it corrects 3 errors. 00:51:47.720 --> 00:51:57.750 align:middle line:90% 00:51:57.750 --> 00:51:59.220 align:middle line:90% OK. 00:51:59.220 --> 00:52:01.330 align:middle line:84% So we should call this code something. 00:52:01.330 --> 00:52:04.070 align:middle line:90% 00:52:04.070 --> 00:52:07.700 align:middle line:90% So why don't we call it C inner? 00:52:07.700 --> 00:52:12.150 align:middle line:90% 00:52:12.150 --> 00:52:13.503 align:middle line:90% OK. 00:52:13.503 --> 00:52:21.190 align:middle line:90% And use Reed-Solomon code. 00:52:21.190 --> 00:52:26.240 align:middle line:90% 00:52:26.240 --> 00:52:34.880 align:middle line:84% And somewhere, I had this example of 150 bytes 00:52:34.880 --> 00:52:39.920 align:middle line:84% into 256 bytes and correct 53 errors. 00:52:39.920 --> 00:52:45.880 align:middle line:90% So I guess that is the 250. 00:52:45.880 --> 00:52:51.800 align:middle line:90% That's n and 150. 00:52:51.800 --> 00:53:08.840 align:middle line:84% And corrects 53 errors because the distance is 107. 00:53:08.840 --> 00:53:11.170 align:middle line:84% So how do you concatenate two codes? 00:53:11.170 --> 00:53:25.860 align:middle line:90% 00:53:25.860 --> 00:53:38.140 align:middle line:84% So the first thing you do is apply the outer code 00:53:38.140 --> 00:53:38.810 align:middle line:90% to your message. 00:53:38.810 --> 00:53:43.500 align:middle line:90% 00:53:43.500 --> 00:53:44.000 align:middle line:90% Wait. 00:53:44.000 --> 00:53:48.570 align:middle line:90% 00:53:48.570 --> 00:53:49.070 align:middle line:90% Yeah. 00:53:49.070 --> 00:53:59.660 align:middle line:90% 00:53:59.660 --> 00:54:00.720 align:middle line:90% What does that look like? 00:54:00.720 --> 00:54:07.120 align:middle line:84% Well, your message, you break into blocks of 8 bits. 00:54:07.120 --> 00:54:13.030 align:middle line:84% And 8 bits can be represented as numbers between 0 and 256. 00:54:13.030 --> 00:54:26.110 align:middle line:84% So your message might look like 317, 251, 109, 113, 00:54:26.110 --> 00:54:29.350 align:middle line:90% 75, et cetera. 00:54:29.350 --> 00:54:44.070 align:middle line:84% And it has 150 of these, 150 of these things. 00:54:44.070 --> 00:54:51.730 align:middle line:84% So now, you encode it using your Reed-Solomon code. 00:54:51.730 --> 00:54:55.430 align:middle line:90% And you get 250. 00:54:55.430 --> 00:55:01.550 align:middle line:90% code word length, 250. 00:55:01.550 --> 00:55:03.270 align:middle line:90% Looks pretty much the same-- 00:55:03.270 --> 00:55:12.640 align:middle line:90% 113, 11, 17, 257, et cetera. 00:55:12.640 --> 00:55:14.770 align:middle line:84% And there are 250 of these numbers. 00:55:14.770 --> 00:55:17.400 align:middle line:90% 00:55:17.400 --> 00:55:38.360 align:middle line:84% Take each of these numbers and encode using C inner. 00:55:38.360 --> 00:55:42.520 align:middle line:84% And now, to decode, you first decode C inner. 00:55:42.520 --> 00:55:45.920 align:middle line:90% And you get this code word. 00:55:45.920 --> 00:55:53.720 align:middle line:84% And now, you decode it using the decoding 00:55:53.720 --> 00:55:57.250 align:middle line:84% algorithm for Reed-Solomon codes, and you get your message. 00:55:57.250 --> 00:56:11.890 align:middle line:90% 00:56:11.890 --> 00:56:15.440 align:middle line:90% And you have taken your message. 00:56:15.440 --> 00:56:21.250 align:middle line:90% 00:56:21.250 --> 00:56:31.370 align:middle line:84% Take, I guess, 8 times 150 is equal to, I think, 00:56:31.370 --> 00:56:45.916 align:middle line:84% 1,200 bits into what was our 19, 8, 00:56:45.916 --> 00:56:53.930 align:middle line:90% 7 bit codes into 19 times 256. 00:56:53.930 --> 00:57:01.230 align:middle line:84% OK, I did not do this multiplication before I class. 00:57:01.230 --> 00:57:15.220 align:middle line:84% So 250 times 20 is around 5,000 bits. 00:57:15.220 --> 00:57:26.580 align:middle line:84% So you've taken 1,200 bits to 5,000 bits. 00:57:26.580 --> 00:57:34.340 align:middle line:84% And you can ask what is the probability 00:57:34.340 --> 00:57:48.870 align:middle line:84% you decode successfully with a 10% bit error rate? 00:57:48.870 --> 00:57:58.620 align:middle line:90% 00:57:58.620 --> 00:58:02.990 align:middle line:84% OK, so I used Mathematica for this. 00:58:02.990 --> 00:58:28.630 align:middle line:84% But at a 10% error rate, if you have 8 bits, 10% error rate, 00:58:28.630 --> 00:58:35.716 align:middle line:84% probability of greater than or equal to 3 errors is 11%. 00:58:35.716 --> 00:58:44.070 align:middle line:90% 00:58:44.070 --> 00:58:46.710 align:middle line:90% And now, the chance-- 00:58:46.710 --> 00:58:48.770 align:middle line:90% and what do we have? 00:58:48.770 --> 00:58:54.070 align:middle line:84% We have an 11% error rate in the bytes. 00:58:54.070 --> 00:58:59.150 align:middle line:90% We have 256 bytes. 00:58:59.150 --> 00:59:03.010 align:middle line:84% And if there are fewer than 53 of them in error, 00:59:03.010 --> 00:59:07.250 align:middle line:84% you can decode with high probability. 00:59:07.250 --> 00:59:10.650 align:middle line:84% And from probability theory, you know how to figure out 00:59:10.650 --> 00:59:14.010 align:middle line:84% the probability that you cannot decode, 00:59:14.010 --> 00:59:29.530 align:middle line:84% but the probability you cannot decode is approximately 10 00:59:29.530 --> 00:59:38.820 align:middle line:84% to the minus 60th, which is really incredibly small. 00:59:38.820 --> 00:59:41.490 align:middle line:90% 00:59:41.490 --> 00:59:49.930 align:middle line:84% So that means that if you take concatenated code 00:59:49.930 --> 00:59:54.130 align:middle line:84% and use this code for the outer code 00:59:54.130 --> 00:59:56.330 align:middle line:84% and this code for the inner code, 00:59:56.330 --> 01:00:00.500 align:middle line:84% and you have a 10% error rate on your bits, 01:00:00.500 --> 01:00:05.650 align:middle line:84% your information almost always gets through completely intact. 01:00:05.650 --> 01:00:09.620 align:middle line:90% 01:00:09.620 --> 01:00:15.030 align:middle line:84% And actually, so this 10% error rate-- 01:00:15.030 --> 01:00:18.340 align:middle line:90% 01:00:18.340 --> 01:00:21.680 align:middle line:84% we've taken 1,200 bits to 5,000 bits. 01:00:21.680 --> 01:00:25.580 align:middle line:84% We've multiplied the number of bits by 4. 01:00:25.580 --> 01:00:28.820 align:middle line:84% Now, if you look at Shannon's theorem, 01:00:28.820 --> 01:00:33.500 align:middle line:84% if you have 1,200 bits and a 10% error rate, 01:00:33.500 --> 01:00:37.260 align:middle line:84% you should be able to get by with multiplying the number 01:00:37.260 --> 01:00:38.940 align:middle line:90% of bits by 2. 01:00:38.940 --> 01:00:43.740 align:middle line:84% So the ideal code would give us 2,400 bits. 01:00:43.740 --> 01:00:45.500 align:middle line:90% And we've used twice that many-- 01:00:45.500 --> 01:00:46.760 align:middle line:90% 5,000 bits. 01:00:46.760 --> 01:00:48.010 align:middle line:90% So that's not that horrible. 01:00:48.010 --> 01:00:50.700 align:middle line:90% 01:00:50.700 --> 01:00:53.500 align:middle line:84% If we use turbo codes, or LDPC codes, 01:00:53.500 --> 01:00:57.950 align:middle line:84% or Polar codes, which were invented 01:00:57.950 --> 01:01:00.910 align:middle line:84% much later than Reed-Solomon codes, 01:01:00.910 --> 01:01:02.470 align:middle line:84% we would reduce that to something 01:01:02.470 --> 01:01:08.190 align:middle line:90% very close to Shannon's bound. 01:01:08.190 --> 01:01:13.765 align:middle line:90% But this was good enough for-- 01:01:13.765 --> 01:01:15.090 align:middle line:90% let's see. 01:01:15.090 --> 01:01:20.710 align:middle line:90% 01:01:20.710 --> 01:01:23.460 align:middle line:84% Yeah, so this was good enough for 30 years. 01:01:23.460 --> 01:01:28.414 align:middle line:90% 01:01:28.414 --> 01:01:31.590 align:middle line:84% All the machinery for using Reed-Solomon codes had been 01:01:31.590 --> 01:01:36.350 align:middle line:84% discovered by 1970, and they remained in widespread use-- 01:01:36.350 --> 01:01:38.570 align:middle line:84% well, they remain in widespread use today. 01:01:38.570 --> 01:01:41.710 align:middle line:90% Anytime you play a CD-- 01:01:41.710 --> 01:01:44.210 align:middle line:84% of course, very few people are playing CDs anymore, 01:01:44.210 --> 01:01:47.380 align:middle line:84% but anytime you play a CD, you're using Reed-Solomon codes. 01:01:47.380 --> 01:01:51.430 align:middle line:90% 01:01:51.430 --> 01:01:53.840 align:middle line:90% OK. 01:01:53.840 --> 01:01:57.800 align:middle line:84% There's one last thing I want to say. 01:01:57.800 --> 01:02:00.840 align:middle line:84% Well, I've told you how to decode the outer code. 01:02:00.840 --> 01:02:02.690 align:middle line:84% How can you decode the inner code? 01:02:02.690 --> 01:02:41.620 align:middle line:90% 01:02:41.620 --> 01:02:46.290 align:middle line:90% It's a 19, 8, 7 code. 01:02:46.290 --> 01:02:50.600 align:middle line:90% 01:02:50.600 --> 01:03:01.130 align:middle line:84% There are 2 to the 19th, which is approximately 500,000, 01:03:01.130 --> 01:03:01.740 align:middle line:90% code words. 01:03:01.740 --> 01:03:07.970 align:middle line:90% 01:03:07.970 --> 01:03:14.210 align:middle line:84% And can use table lookup for these. 01:03:14.210 --> 01:03:21.410 align:middle line:84% You have, in your computer, a complete table of all-- 01:03:21.410 --> 01:03:31.370 align:middle line:90% there are 500,000 possible. 01:03:31.370 --> 01:03:34.550 align:middle line:84% Yeah, there are only 256 code words, 01:03:34.550 --> 01:03:42.930 align:middle line:84% but those get mapped into 19-bit words. 01:03:42.930 --> 01:03:47.050 align:middle line:84% And there are 500,000 19-bit words. 01:03:47.050 --> 01:03:49.310 align:middle line:84% And you can use table lookup to decode. 01:03:49.310 --> 01:03:56.390 align:middle line:84% You can just list all possible 500,000 words you might receive. 01:03:56.390 --> 01:03:59.590 align:middle line:84% And to decode them, you look up in the table 01:03:59.590 --> 01:04:01.590 align:middle line:90% to find the nearest code word. 01:04:01.590 --> 01:04:05.030 align:middle line:90% And that's the decoding. 01:04:05.030 --> 01:04:09.190 align:middle line:84% And nowadays, this is incredibly easy. 01:04:09.190 --> 01:04:14.150 align:middle line:84% In 1977, when Voyager did it, I expect that the Voyager 01:04:14.150 --> 01:04:19.790 align:middle line:84% spacecraft did not have enough memory for 500,000 table lookup 01:04:19.790 --> 01:04:20.290 align:middle line:90% words. 01:04:20.290 --> 01:04:22.390 align:middle line:84% So they must have done something cleverer. 01:04:22.390 --> 01:04:24.630 align:middle line:84% Well, one thing you could do is, instead 01:04:24.630 --> 01:04:35.050 align:middle line:84% of using Reed-Solomon codes with 8 bits in each block, 01:04:35.050 --> 01:04:37.550 align:middle line:84% use ones with smaller number of bits in each block. 01:04:37.550 --> 01:04:43.162 align:middle line:84% And then you will get a smaller code word. 01:04:43.162 --> 01:04:45.370 align:middle line:84% Or another thing you could do is, for the inner code, 01:04:45.370 --> 01:04:47.190 align:middle line:84% you could use a Hamming code word. 01:04:47.190 --> 01:04:51.560 align:middle line:84% And Hamming codes are very easy to decode, 01:04:51.560 --> 01:04:57.160 align:middle line:84% although they really do not get quite as good performance 01:04:57.160 --> 01:05:02.080 align:middle line:84% as you know tree error correcting codes. 01:05:02.080 --> 01:05:05.040 align:middle line:84% And there are other things you can do, too. 01:05:05.040 --> 01:05:10.880 align:middle line:84% So I don't actually know what they did for Voyager, 01:05:10.880 --> 01:05:15.640 align:middle line:84% but it's probably somewhere on the line. 01:05:15.640 --> 01:05:17.160 align:middle line:90% And I could look it up. 01:05:17.160 --> 01:05:22.160 align:middle line:84% OK, so that's all I have on error correcting codes. 01:05:22.160 --> 01:05:29.240 align:middle line:84% And in fact, this is my last lecture of the course 01:05:29.240 --> 01:05:31.480 align:middle line:84% because Professor Moitra will be giving 01:05:31.480 --> 01:05:37.420 align:middle line:90% the next two non-exam lectures. 01:05:37.420 --> 01:05:39.600 align:middle line:90% And we have one exam for you. 01:05:39.600 --> 01:05:41.150 align:middle line:90% OK. 01:05:41.150 --> 01:05:47.000 align:middle line:90%