WEBVTT 00:00:00.000 --> 00:00:01.924 align:middle line:90% [SQUEAKING] 00:00:01.924 --> 00:00:04.329 align:middle line:90% [RUSTLING] 00:00:04.329 --> 00:00:10.582 align:middle line:90% [CLICKING] 00:00:10.582 --> 00:00:12.640 align:middle line:84% PETER SHOR: OK, so today we're going 00:00:12.640 --> 00:00:19.360 align:middle line:84% to be talking about error correcting codes. 00:00:19.360 --> 00:00:24.400 align:middle line:84% So on Thursday, Professor Moitra told you 00:00:24.400 --> 00:00:28.200 align:middle line:84% about Shannon's noisy coding theorem. 00:00:28.200 --> 00:00:31.080 align:middle line:84% And I don't know if he drew this picture or not. 00:00:31.080 --> 00:00:33.765 align:middle line:90% It's not in the notes. 00:00:33.765 --> 00:00:35.720 align:middle line:84% And it's a picture everybody draws 00:00:35.720 --> 00:00:38.460 align:middle line:84% when they talk about Shannon's noisy coding theorem. 00:00:38.460 --> 00:00:51.960 align:middle line:84% So you have some message, message drawn from m, 00:00:51.960 --> 00:00:54.800 align:middle line:90% which is some set of messages. 00:00:54.800 --> 00:00:57.900 align:middle line:90% And now you encode it. 00:00:57.900 --> 00:01:05.690 align:middle line:90% 00:01:05.690 --> 00:01:07.860 align:middle line:84% And now you have a noisy channel. 00:01:07.860 --> 00:01:16.370 align:middle line:90% 00:01:16.370 --> 00:01:26.770 align:middle line:84% And you get c tilde, or you get the received code word, which 00:01:26.770 --> 00:01:31.850 align:middle line:90% maybe I should call it c tilde. 00:01:31.850 --> 00:01:39.010 align:middle line:84% And now you decode and you get something 00:01:39.010 --> 00:01:41.570 align:middle line:90% which I'll call m tilde. 00:01:41.570 --> 00:02:09.660 align:middle line:84% And Shannon, if the noise at the rate which I want to say is 00:02:09.660 --> 00:02:11.850 align:middle line:90% log k over-- 00:02:11.850 --> 00:02:14.900 align:middle line:90% 00:02:14.900 --> 00:02:16.200 align:middle line:90% what is the rate? 00:02:16.200 --> 00:02:17.600 align:middle line:90% It's k over n, right-- 00:02:17.600 --> 00:02:22.780 align:middle line:90% 00:02:22.780 --> 00:02:29.260 align:middle line:84% so these messages are k bits, and this code is n bits-- 00:02:29.260 --> 00:02:32.170 align:middle line:90% 00:02:32.170 --> 00:02:42.980 align:middle line:84% is less than the channel capacity, 00:02:42.980 --> 00:02:52.740 align:middle line:84% there exists a code such that as n goes to infinity and k over n 00:02:52.740 --> 00:02:57.110 align:middle line:84% remains the same, the probability 00:02:57.110 --> 00:03:04.630 align:middle line:90% that m tilde equals m goes to 1. 00:03:04.630 --> 00:03:08.340 align:middle line:84% So in the limit of long block lengths-- 00:03:08.340 --> 00:03:10.950 align:middle line:90% 00:03:10.950 --> 00:03:16.570 align:middle line:84% I don't know how much coding theory terminology Ankur used, 00:03:16.570 --> 00:03:20.200 align:middle line:84% but n is called the block length-- 00:03:20.200 --> 00:03:27.670 align:middle line:90% 00:03:27.670 --> 00:03:32.760 align:middle line:90% then the error rate goes to 0. 00:03:32.760 --> 00:03:35.390 align:middle line:90% 00:03:35.390 --> 00:03:38.430 align:middle line:84% So that is the noisy-channel coding theorem. 00:03:38.430 --> 00:03:41.890 align:middle line:84% So did Professor Moitra put this diagram on the board? 00:03:41.890 --> 00:03:46.710 align:middle line:90% 00:03:46.710 --> 00:03:47.870 align:middle line:90% OK. 00:03:47.870 --> 00:03:48.530 align:middle line:90% Oh, well. 00:03:48.530 --> 00:03:52.710 align:middle line:90% 00:03:52.710 --> 00:04:06.630 align:middle line:84% And Shannon used random codes to prove his theorem. 00:04:06.630 --> 00:04:19.680 align:middle line:90% 00:04:19.680 --> 00:04:23.220 align:middle line:84% And these are totally impractical for large n. 00:04:23.220 --> 00:04:37.960 align:middle line:90% 00:04:37.960 --> 00:04:40.440 align:middle line:84% Because what you have is you have 00:04:40.440 --> 00:04:44.800 align:middle line:84% a list of code words, which is exponential in n, 00:04:44.800 --> 00:04:50.160 align:middle line:90% and you get a received word. 00:04:50.160 --> 00:04:52.330 align:middle line:84% And you want to find which of these code words 00:04:52.330 --> 00:04:53.330 align:middle line:90% it's closest to. 00:04:53.330 --> 00:04:57.930 align:middle line:84% And we don't know any way of doing 00:04:57.930 --> 00:05:01.290 align:middle line:84% this that is substantially better than going 00:05:01.290 --> 00:05:04.270 align:middle line:84% through the entire list of code words, computing the distance, 00:05:04.270 --> 00:05:08.450 align:middle line:90% and finding the shortest one. 00:05:08.450 --> 00:05:10.210 align:middle line:84% And in fact, you can show that this 00:05:10.210 --> 00:05:16.210 align:middle line:84% is an NP-hard problem, which means that there probably 00:05:16.210 --> 00:05:18.770 align:middle line:90% isn't such a way. 00:05:18.770 --> 00:05:24.890 align:middle line:84% So what you need is you need to find 00:05:24.890 --> 00:05:36.350 align:middle line:84% specific codes with practical decoding algorithms. 00:05:36.350 --> 00:05:46.050 align:middle line:90% 00:05:46.050 --> 00:05:51.180 align:middle line:84% And today and Thursday, I will tell you about two 00:05:51.180 --> 00:05:52.660 align:middle line:90% of these codes. 00:05:52.660 --> 00:06:00.580 align:middle line:84% And they will basically stand in for the whole class 00:06:00.580 --> 00:06:01.530 align:middle line:90% of linear codes. 00:06:01.530 --> 00:06:08.500 align:middle line:90% 00:06:08.500 --> 00:06:23.700 align:middle line:84% So, Shannon proved his theorem in 1948. 00:06:23.700 --> 00:06:33.700 align:middle line:84% And two years later, Hamming found 00:06:33.700 --> 00:06:38.190 align:middle line:90% the first error-correcting code. 00:06:38.190 --> 00:06:41.180 align:middle line:90% 00:06:41.180 --> 00:06:47.660 align:middle line:84% Hamming, 1950, actually found a whole class 00:06:47.660 --> 00:06:48.890 align:middle line:90% of error-correcting codes. 00:06:48.890 --> 00:06:50.430 align:middle line:84% But the first part of the talk, I'll 00:06:50.430 --> 00:06:55.030 align:middle line:84% just talk about the simplest one in this class-- 00:06:55.030 --> 00:07:07.220 align:middle line:84% can encode four bits into seven bits and correct one error. 00:07:07.220 --> 00:07:15.550 align:middle line:90% 00:07:15.550 --> 00:07:20.910 align:middle line:84% So if you have, I don't know, a million bits that you 00:07:20.910 --> 00:07:27.150 align:middle line:84% want to send, and their error rate is 1 in 100, 00:07:27.150 --> 00:07:30.950 align:middle line:84% then this is not going to work because you 00:07:30.950 --> 00:07:37.070 align:middle line:84% can break the million bits into blocks of four 00:07:37.070 --> 00:07:40.790 align:middle line:84% and transmit each of them in seven bits. 00:07:40.790 --> 00:07:46.240 align:middle line:84% But chances are very high that in these million bits 00:07:46.240 --> 00:07:49.920 align:middle line:84% you will find one block of four which 00:07:49.920 --> 00:07:54.880 align:middle line:84% has two errors in it, which means it will not be corrected. 00:07:54.880 --> 00:07:57.400 align:middle line:84% So what you really would like is you would like a way 00:07:57.400 --> 00:08:04.680 align:middle line:84% of encoding, say, 1,000 bits into 1,500 bits that correct 10 00:08:04.680 --> 00:08:06.200 align:middle line:90% errors. 00:08:06.200 --> 00:08:08.040 align:middle line:84% And we will get to something like that 00:08:08.040 --> 00:08:16.680 align:middle line:84% with Reed-Solomon codes, which we'll talk about on Thursday. 00:08:16.680 --> 00:08:20.280 align:middle line:90% But what is the Hamming code? 00:08:20.280 --> 00:08:24.040 align:middle line:84% So I'm going to give you, first, a very simple description 00:08:24.040 --> 00:08:29.400 align:middle line:84% of the Hamming code, and then a more complicated one. 00:08:29.400 --> 00:08:32.559 align:middle line:84% And the simple description, we're 00:08:32.559 --> 00:08:49.010 align:middle line:84% going to start with a Venn diagram with four bits-- 00:08:49.010 --> 00:08:59.650 align:middle line:90% well, with three regions. 00:08:59.650 --> 00:09:01.290 align:middle line:84% And these regions are actually going 00:09:01.290 --> 00:09:03.900 align:middle line:84% to correspond to specific bits of this code. 00:09:03.900 --> 00:09:08.610 align:middle line:90% 00:09:08.610 --> 00:09:10.274 align:middle line:90% OK. 00:09:10.274 --> 00:09:13.950 align:middle line:84% Now, let me see if I can do this. 00:09:13.950 --> 00:09:22.410 align:middle line:90% 00:09:22.410 --> 00:09:23.550 align:middle line:90% I want to do this. 00:09:23.550 --> 00:09:29.090 align:middle line:84% So let's take the four bits to be bit a, bit 00:09:29.090 --> 00:09:34.050 align:middle line:90% b, bit c, and bit d. 00:09:34.050 --> 00:09:35.540 align:middle line:90% So these are the message bits. 00:09:35.540 --> 00:09:44.420 align:middle line:90% 00:09:44.420 --> 00:09:48.150 align:middle line:84% And now what we're going to do is add three parity check bits. 00:09:48.150 --> 00:09:57.180 align:middle line:90% 00:09:57.180 --> 00:10:02.820 align:middle line:84% Add these-- 4 plus 3 is 7, which is why we're encoding 00:10:02.820 --> 00:10:04.820 align:middle line:90% four bits into seven bits. 00:10:04.820 --> 00:10:11.020 align:middle line:84% And these bits are going to give you 00:10:11.020 --> 00:10:13.340 align:middle line:90% the error-correcting ability. 00:10:13.340 --> 00:10:22.580 align:middle line:90% So let's say this is a plus b. 00:10:22.580 --> 00:10:28.160 align:middle line:90% This is a plus b plus d. 00:10:28.160 --> 00:10:30.780 align:middle line:90% 00:10:30.780 --> 00:10:33.960 align:middle line:90% This is b plus c plus d. 00:10:33.960 --> 00:10:36.780 align:middle line:90% 00:10:36.780 --> 00:10:44.110 align:middle line:84% And this is a plus c plus d, where 00:10:44.110 --> 00:10:47.340 align:middle line:84% the plus is exclusive r, so binary addition. 00:10:47.340 --> 00:10:57.750 align:middle line:90% 00:10:57.750 --> 00:11:12.520 align:middle line:84% Example, let's take 1, 0, 1, 1 to be the message. 00:11:12.520 --> 00:11:20.950 align:middle line:90% 00:11:20.950 --> 00:11:24.790 align:middle line:90% So, 1, 0, 1, 1. 00:11:24.790 --> 00:11:28.686 align:middle line:90% I'm sorry, 1, 0, 1, 1. 00:11:28.686 --> 00:11:33.470 align:middle line:90% 00:11:33.470 --> 00:11:38.280 align:middle line:84% So now we want to compute the parity-check bits. 00:11:38.280 --> 00:11:43.660 align:middle line:84% So what we do is we put a bit in here 00:11:43.660 --> 00:11:49.000 align:middle line:84% so that the sum of the bits in this circle is even. 00:11:49.000 --> 00:11:54.200 align:middle line:84% So that would be 0, because 1 plus 1 is 0. 00:11:54.200 --> 00:11:58.220 align:middle line:84% And similarly, this is 0, because 1 plus 1 is 0. 00:11:58.220 --> 00:12:03.400 align:middle line:84% And this is 1 because 1 plus 1 plus 1 is 1. 00:12:03.400 --> 00:12:07.520 align:middle line:84% And we know to add a 1 to make it 0. 00:12:07.520 --> 00:12:10.292 align:middle line:90% OK. 00:12:10.292 --> 00:12:23.700 align:middle line:84% So our code is 1, 0, 1, 1, and now 0, 0, 1. 00:12:23.700 --> 00:12:27.280 align:middle line:90% 00:12:27.280 --> 00:12:28.240 align:middle line:90% OK. 00:12:28.240 --> 00:12:30.880 align:middle line:84% Now let's send it through the channel, 00:12:30.880 --> 00:12:33.520 align:middle line:84% and we will change one of the bits. 00:12:33.520 --> 00:12:36.150 align:middle line:84% Does anybody have a suggestion for which bit we want to change? 00:12:36.150 --> 00:12:41.850 align:middle line:90% 00:12:41.850 --> 00:12:44.730 align:middle line:84% Who wants to change the second bit? 00:12:44.730 --> 00:12:46.500 align:middle line:90% OK, we'll do that. 00:12:46.500 --> 00:12:50.290 align:middle line:90% So it's 1, 1, 1, 1, 0, 0, 1. 00:12:50.290 --> 00:12:57.770 align:middle line:90% So what we do now is we compute. 00:12:57.770 --> 00:13:03.130 align:middle line:84% Let's call these a, b, c, d, e, f, g. 00:13:03.130 --> 00:13:15.560 align:middle line:84% And this was a, b, c, d, e, f, g, I believe. 00:13:15.560 --> 00:13:19.330 align:middle line:90% 00:13:19.330 --> 00:13:20.170 align:middle line:90% Yeah. 00:13:20.170 --> 00:13:26.290 align:middle line:84% So now we compute a plus d plus c plus g, 00:13:26.290 --> 00:13:28.870 align:middle line:90% a plus c plus d plus g. 00:13:28.870 --> 00:13:32.570 align:middle line:90% 00:13:32.570 --> 00:13:41.700 align:middle line:84% That is 0, so this circle is correct. 00:13:41.700 --> 00:13:47.840 align:middle line:84% Now we compute a plus e plus b plus d, a plus e plus b plus d, 00:13:47.840 --> 00:13:49.180 align:middle line:90% that's 1. 00:13:49.180 --> 00:13:53.220 align:middle line:90% So this circle is incorrect. 00:13:53.220 --> 00:14:00.472 align:middle line:84% And b plus c plus d plus f, b plus c plus d plus f, that-- 00:14:00.472 --> 00:14:02.200 align:middle line:90% b, c, d, f-- 00:14:02.200 --> 00:14:04.180 align:middle line:90% That's also equal to 1. 00:14:04.180 --> 00:14:06.980 align:middle line:84% So this circle is also incorrect. 00:14:06.980 --> 00:14:12.740 align:middle line:84% And so now we know the incorrect bit is not in this region, 00:14:12.740 --> 00:14:16.380 align:middle line:84% but is in the intersection of this region and this region. 00:14:16.380 --> 00:14:18.690 align:middle line:90% And that means it is b. 00:14:18.690 --> 00:14:24.740 align:middle line:90% 00:14:24.740 --> 00:14:32.040 align:middle line:84% So the received word is not incorrect. 00:14:32.040 --> 00:14:34.710 align:middle line:90% 00:14:34.710 --> 00:14:37.870 align:middle line:90% Bit is b. 00:14:37.870 --> 00:14:46.680 align:middle line:84% And now we change b, so we get 1, 0, 1, 1, 0, 0, 1. 00:14:46.680 --> 00:14:47.180 align:middle line:90% Right. 00:14:47.180 --> 00:14:51.910 align:middle line:90% 00:14:51.910 --> 00:14:56.750 align:middle line:84% And while you might have noticed that the message was 00:14:56.750 --> 00:14:59.780 align:middle line:84% the first four bits, so this is the message. 00:14:59.780 --> 00:15:09.270 align:middle line:90% 00:15:09.270 --> 00:15:09.770 align:middle line:90% OK. 00:15:09.770 --> 00:15:31.010 align:middle line:90% 00:15:31.010 --> 00:15:34.400 align:middle line:84% So now I'm going to tell you about general linear codes, 00:15:34.400 --> 00:15:37.810 align:middle line:90% and then general Hamming codes. 00:15:37.810 --> 00:15:48.640 align:middle line:90% 00:15:48.640 --> 00:16:08.080 align:middle line:84% A linear code over a field F is a collection 00:16:08.080 --> 00:16:30.570 align:middle line:84% of length n strings in F. So, if c1, c2 are in the code-- 00:16:30.570 --> 00:16:32.670 align:middle line:84% I should probably call this a code C-- 00:16:32.670 --> 00:16:35.290 align:middle line:90% 00:16:35.290 --> 00:16:41.730 align:middle line:90% then c1 plus c2 is in the code. 00:16:41.730 --> 00:16:49.810 align:middle line:84% And if c1 is in the code and alpha is in F, 00:16:49.810 --> 00:16:52.420 align:middle line:90% alpha times C1 is in the code. 00:16:52.420 --> 00:16:55.490 align:middle line:90% 00:16:55.490 --> 00:17:00.160 align:middle line:84% So any linear combination of code words is also a code word. 00:17:00.160 --> 00:17:04.290 align:middle line:90% 00:17:04.290 --> 00:17:06.730 align:middle line:84% And it's fairly easy to see that this 00:17:06.730 --> 00:17:12.310 align:middle line:90% holds for this Hamming code. 00:17:12.310 --> 00:17:21.790 align:middle line:84% Because if you have a set of bits that satisfies this, 00:17:21.790 --> 00:17:24.849 align:middle line:84% so the sum of these, sum of these, and sum of these 00:17:24.849 --> 00:17:28.720 align:middle line:84% is all 0, and you have another set of bits that satisfies this, 00:17:28.720 --> 00:17:34.100 align:middle line:84% then when you take the sum of these sets of bits, 00:17:34.100 --> 00:17:36.440 align:middle line:84% the sum of things within this circle 00:17:36.440 --> 00:17:39.205 align:middle line:84% is the sum of the things from the first code 00:17:39.205 --> 00:17:40.580 align:middle line:84% word in this circle, plus the sum 00:17:40.580 --> 00:17:43.180 align:middle line:84% of things in the second code word in this circle, 00:17:43.180 --> 00:17:45.960 align:middle line:90% and 0 plus 0 equals 0. 00:17:45.960 --> 00:17:47.430 align:middle line:90% So it's also a code. 00:17:47.430 --> 00:17:50.540 align:middle line:90% 00:17:50.540 --> 00:17:53.610 align:middle line:84% So we have this is a linear code. 00:17:53.610 --> 00:18:10.140 align:middle line:90% 00:18:10.140 --> 00:18:25.150 align:middle line:84% By linear algebra, every code C see 00:18:25.150 --> 00:19:00.630 align:middle line:84% has a generating matrix G such that C is the row sums of G. 00:19:00.630 --> 00:19:02.790 align:middle line:84% So now I'm going to write down the Hamming 00:19:02.790 --> 00:19:04.510 align:middle line:90% code in a different way. 00:19:04.510 --> 00:19:08.300 align:middle line:84% And actually, you're not going to get quite the same-- 00:19:08.300 --> 00:19:13.310 align:middle line:90% 00:19:13.310 --> 00:19:22.670 align:middle line:84% well, I mean, I'm going to write down the Hamming 00:19:22.670 --> 00:19:23.960 align:middle line:90% code in a different way. 00:19:23.960 --> 00:19:29.904 align:middle line:84% And let's see, we have 1, 0, 0, 0. 00:19:29.904 --> 00:19:37.360 align:middle line:90% 00:19:37.360 --> 00:19:40.760 align:middle line:90% So this is a first bit. 00:19:40.760 --> 00:19:45.400 align:middle line:84% So now the first two bits of the parity check matrix 00:19:45.400 --> 00:19:53.480 align:middle line:84% are both 1 here, so 1, 0, 0, 0, 1, 1, 0. 00:19:53.480 --> 00:20:04.560 align:middle line:84% So when you put these bits in the Venn diagram, they will be-- 00:20:04.560 --> 00:20:06.760 align:middle line:90% assuming I've done this right-- 00:20:06.760 --> 00:20:10.160 align:middle line:84% they will satisfy the parity checks. 00:20:10.160 --> 00:20:13.240 align:middle line:90% And then there's 0, 1, 0, 0. 00:20:13.240 --> 00:20:21.040 align:middle line:84% And that's just 0, 1, 1 and 0, 0, 1, 0. 00:20:21.040 --> 00:20:29.578 align:middle line:84% And this is going to be 1, 0, 1 and 0, 0, 0, 1, 1, 1, 00:20:29.578 --> 00:20:37.210 align:middle line:84% 1, because this fourth bit is the center region of the Venn 00:20:37.210 --> 00:20:38.250 align:middle line:90% diagram. 00:20:38.250 --> 00:20:44.570 align:middle line:84% And that means all three parity checks need to be 1 for them 00:20:44.570 --> 00:20:45.670 align:middle line:90% to satisfy that. 00:20:45.670 --> 00:20:48.580 align:middle line:90% So G is equal to this. 00:20:48.580 --> 00:20:56.730 align:middle line:90% 00:20:56.730 --> 00:21:01.190 align:middle line:84% And there are 16 code words in C-- 00:21:01.190 --> 00:21:06.330 align:middle line:90% 00:21:06.330 --> 00:21:12.510 align:middle line:90% in C. And what do I want to say? 00:21:12.510 --> 00:21:17.658 align:middle line:90% 00:21:17.658 --> 00:21:28.340 align:middle line:84% And then the row sums of G and 0, 0, 0, 0, 0, 0 is in C. 00:21:28.340 --> 00:21:30.460 align:middle line:84% And actually, the all-zeros vector 00:21:30.460 --> 00:21:35.700 align:middle line:84% is a code word of every linear code, 00:21:35.700 --> 00:21:43.540 align:middle line:84% because we know that if c1 is in C and alpha 00:21:43.540 --> 00:21:47.440 align:middle line:84% is in F, then alpha c1 is in C. And if alpha is 0, 00:21:47.440 --> 00:21:50.620 align:middle line:84% then alpha c1 is the all-zeros vector. 00:21:50.620 --> 00:21:53.520 align:middle line:84% So every linear code has the all-zeros vector in it. 00:21:53.520 --> 00:21:57.620 align:middle line:90% 00:21:57.620 --> 00:22:11.750 align:middle line:90% OK, correct. 00:22:11.750 --> 00:22:14.740 align:middle line:90% 00:22:14.740 --> 00:22:16.870 align:middle line:90% Oh, OK. 00:22:16.870 --> 00:22:32.030 align:middle line:90% 00:22:32.030 --> 00:22:36.740 align:middle line:84% We need some more definitions before I can continue. 00:22:36.740 --> 00:22:41.430 align:middle line:90% 00:22:41.430 --> 00:22:49.660 align:middle line:90% The Hamming weight of a vector-- 00:22:49.660 --> 00:22:57.510 align:middle line:90% 00:22:57.510 --> 00:23:00.640 align:middle line:84% and let me see if I can find what-- 00:23:00.640 --> 00:23:12.830 align:middle line:90% 00:23:12.830 --> 00:23:27.200 align:middle line:90% w of v is the number of non-- 00:23:27.200 --> 00:23:31.120 align:middle line:90% wait, I did this wrong. 00:23:31.120 --> 00:23:41.440 align:middle line:84% The Hamming weight w of v of a vector v 00:23:41.440 --> 00:23:47.530 align:middle line:84% is the number of non-zeros v contains. 00:23:47.530 --> 00:23:53.600 align:middle line:90% 00:23:53.600 --> 00:24:06.140 align:middle line:84% Let v equal 0, 1, 3, 2, 0, 4, 1, say. 00:24:06.140 --> 00:24:09.000 align:middle line:90% 00:24:09.000 --> 00:24:14.695 align:middle line:84% The Hamming weight is the number of non-zeros-- with 1, 2, 3, 4, 00:24:14.695 --> 00:24:15.850 align:middle line:90% 5-- 00:24:15.850 --> 00:24:16.930 align:middle line:90% is equal to 5. 00:24:16.930 --> 00:24:25.330 align:middle line:90% 00:24:25.330 --> 00:24:41.850 align:middle line:84% The Hamming distance of v and w is the number 00:24:41.850 --> 00:24:43.560 align:middle line:90% of places where v and w differ. 00:24:43.560 --> 00:25:03.850 align:middle line:90% 00:25:03.850 --> 00:25:07.010 align:middle line:84% So let's pick an easier example-- 00:25:07.010 --> 00:25:10.020 align:middle line:90% 0, 1, 3 and-- 00:25:10.020 --> 00:25:14.880 align:middle line:90% 00:25:14.880 --> 00:25:15.380 align:middle line:90% OK. 00:25:15.380 --> 00:25:21.600 align:middle line:90% 00:25:21.600 --> 00:25:26.340 align:middle line:84% I want to use the same notation for Hamming distances 00:25:26.340 --> 00:25:26.910 align:middle line:90% in the notes. 00:25:26.910 --> 00:25:38.860 align:middle line:90% 00:25:38.860 --> 00:25:45.060 align:middle line:90% So this is d sub H of v, w. 00:25:45.060 --> 00:25:50.740 align:middle line:90% And 1, 1, 2 is-- 00:25:50.740 --> 00:25:52.480 align:middle line:84% well, how many places do they differ? 00:25:52.480 --> 00:25:54.359 align:middle line:90% Anybody? 00:25:54.359 --> 00:25:55.225 align:middle line:90% AUDIENCE: 2. 00:25:55.225 --> 00:25:55.950 align:middle line:90% PETER SHOR: 2. 00:25:55.950 --> 00:25:58.900 align:middle line:90% 00:25:58.900 --> 00:26:02.350 align:middle line:90% So call this c1 and c2. 00:26:02.350 --> 00:26:04.940 align:middle line:90% 00:26:04.940 --> 00:26:11.670 align:middle line:84% So the Hamming distance between c1, c2 is equal to 2. 00:26:11.670 --> 00:26:18.630 align:middle line:84% And it's easy to see that the Hamming weight of c1 minus c2 00:26:18.630 --> 00:26:24.310 align:middle line:84% is equal to the Hamming distance between c1 and c2. 00:26:24.310 --> 00:26:29.205 align:middle line:84% Because if two bits are the same, 00:26:29.205 --> 00:26:30.330 align:middle line:90% then their difference is 0. 00:26:30.330 --> 00:26:33.270 align:middle line:84% And if two bits are different, then their difference 00:26:33.270 --> 00:26:35.030 align:middle line:90% is non-zero. 00:26:35.030 --> 00:26:44.590 align:middle line:84% So here, c1 minus c2 is equal to minus 1, 0, 00:26:44.590 --> 00:26:51.500 align:middle line:90% 1 has Hamming weight 2. 00:26:51.500 --> 00:26:55.430 align:middle line:90% 00:26:55.430 --> 00:27:05.220 align:middle line:90% So theorem, a code can correct. 00:27:05.220 --> 00:27:14.400 align:middle line:90% 00:27:14.400 --> 00:27:33.760 align:middle line:84% Let d of a code be minimum over c1, c2 in the code c1 not equal 00:27:33.760 --> 00:27:38.770 align:middle line:84% to c2, the Hamming distance between c1 and c2. 00:27:38.770 --> 00:27:42.000 align:middle line:90% 00:27:42.000 --> 00:27:44.680 align:middle line:84% So it's the smallest Hamming distance 00:27:44.680 --> 00:27:47.950 align:middle line:90% between two non-equal vectors. 00:27:47.950 --> 00:27:50.960 align:middle line:90% 00:27:50.960 --> 00:27:57.620 align:middle line:90% Then the code can correct. 00:27:57.620 --> 00:28:03.400 align:middle line:90% 00:28:03.400 --> 00:28:17.440 align:middle line:84% t is equal to dH of c over 2 floor errors. 00:28:17.440 --> 00:28:37.405 align:middle line:90% 00:28:37.405 --> 00:28:38.280 align:middle line:90% And what's the proof? 00:28:38.280 --> 00:28:44.610 align:middle line:90% 00:28:44.610 --> 00:28:46.840 align:middle line:90% Well, let's assume-- 00:28:46.840 --> 00:28:53.450 align:middle line:90% 00:28:53.450 --> 00:28:59.850 align:middle line:84% I'm sorry, I wrote this formula down wrong-- 00:28:59.850 --> 00:29:02.520 align:middle line:90% d minus 1 over 2 errors. 00:29:02.520 --> 00:29:09.020 align:middle line:90% 00:29:09.020 --> 00:29:32.220 align:middle line:84% Suppose it cannot, then, well, there's a received word, 00:29:32.220 --> 00:30:04.290 align:middle line:84% a received word c tilde such that, well, 00:30:04.290 --> 00:30:07.380 align:middle line:84% the two code words could have resulted in c tilde. 00:30:07.380 --> 00:30:16.030 align:middle line:90% 00:30:16.030 --> 00:30:25.870 align:middle line:84% So I mean, this is just saying that if you 00:30:25.870 --> 00:30:30.710 align:middle line:84% can correct every word with errors to a unique code word, 00:30:30.710 --> 00:30:35.270 align:middle line:84% assuming there are fewer than t errors, 00:30:35.270 --> 00:30:39.630 align:middle line:84% then the code can correct t errors. 00:30:39.630 --> 00:30:41.330 align:middle line:84% So if it cannot correct t errors, 00:30:41.330 --> 00:30:45.710 align:middle line:84% there must be some word c tilde so that there are two code words 00:30:45.710 --> 00:30:53.910 align:middle line:84% which that are in Hamming distance t of C tilde. 00:30:53.910 --> 00:31:01.770 align:middle line:90% So t is equal to c tilde. 00:31:01.770 --> 00:31:05.154 align:middle line:90% 00:31:05.154 --> 00:31:10.320 align:middle line:84% Or actually, t is greater than or equal to the Hamming distance 00:31:10.320 --> 00:31:18.360 align:middle line:84% between c tilde and c1, and t is greater than 00:31:18.360 --> 00:31:25.190 align:middle line:84% or equal to the Hamming distance between c tilde and c2. 00:31:25.190 --> 00:31:28.520 align:middle line:90% 00:31:28.520 --> 00:31:38.960 align:middle line:84% Then the distance between c1 and c2 must be less than 00:31:38.960 --> 00:31:52.400 align:middle line:84% or equal to the distance from c1 to c tilde, plus the distance 00:31:52.400 --> 00:31:57.560 align:middle line:90% between c tilde and c2. 00:31:57.560 --> 00:32:01.690 align:middle line:84% And this is just because Hamming distance is a metric, 00:32:01.690 --> 00:32:07.650 align:middle line:84% which means if the distance between c1 and c tilde is, say, 00:32:07.650 --> 00:32:11.770 align:middle line:84% t, and the distance between c tilde and c2 is t, 00:32:11.770 --> 00:32:15.560 align:middle line:84% then the distance between c1 and c2 is at most 2t. 00:32:15.560 --> 00:32:19.170 align:middle line:90% 00:32:19.170 --> 00:32:32.730 align:middle line:84% But this is less than or equal to 2t by these inequalities. 00:32:32.730 --> 00:32:37.770 align:middle line:84% And that is less than or equal to, 00:32:37.770 --> 00:32:40.680 align:middle line:84% I guess, the Hamming distance for the code. 00:32:40.680 --> 00:32:44.010 align:middle line:90% 00:32:44.010 --> 00:32:46.690 align:middle line:84% But we know by the definition of the Hamming distance 00:32:46.690 --> 00:32:52.250 align:middle line:84% from the code that it's the minimum distance 00:32:52.250 --> 00:32:55.490 align:middle line:90% between two code words. 00:32:55.490 --> 00:32:57.190 align:middle line:90% So this is a contradiction. 00:32:57.190 --> 00:33:00.660 align:middle line:90% 00:33:00.660 --> 00:33:06.140 align:middle line:84% I should say less than here, because t 00:33:06.140 --> 00:33:12.660 align:middle line:84% was 1 less than the distance of code divided by 2. 00:33:12.660 --> 00:33:13.160 align:middle line:90% OK. 00:33:13.160 --> 00:33:49.860 align:middle line:90% 00:33:49.860 --> 00:34:05.270 align:middle line:84% For a linear code, let d equal the minimum over all code words 00:34:05.270 --> 00:34:13.920 align:middle line:84% in the code of the Hamming weight. 00:34:13.920 --> 00:34:18.590 align:middle line:90% 00:34:18.590 --> 00:34:20.159 align:middle line:84% What did we use for Hamming weight? 00:34:20.159 --> 00:34:24.630 align:middle line:90% 00:34:24.630 --> 00:34:26.810 align:middle line:90% w-- OK, so w of c. 00:34:26.810 --> 00:34:30.070 align:middle line:90% 00:34:30.070 --> 00:34:33.469 align:middle line:84% And we have to specify that c is not equal to 0 00:34:33.469 --> 00:34:38.250 align:middle line:84% here, so the minimum weight of any non-zero code word. 00:34:38.250 --> 00:34:43.989 align:middle line:90% 00:34:43.989 --> 00:34:55.190 align:middle line:84% Then c can correct d minus 1 over 2 errors. 00:34:55.190 --> 00:35:02.960 align:middle line:90% 00:35:02.960 --> 00:35:05.037 align:middle line:90% Proof. 00:35:05.037 --> 00:35:06.620 align:middle line:84% Well, we're going to use this theorem. 00:35:06.620 --> 00:35:13.832 align:middle line:90% 00:35:13.832 --> 00:35:16.330 align:middle line:84% Might as well call this theorem 2. 00:35:16.330 --> 00:35:16.830 align:middle line:90% Proof. 00:35:16.830 --> 00:35:29.920 align:middle line:90% 00:35:29.920 --> 00:35:39.800 align:middle line:84% So the minimum over c1 and c2 in C of the distance 00:35:39.800 --> 00:35:50.520 align:middle line:84% between the two code words is equal to the minimum 00:35:50.520 --> 00:35:55.370 align:middle line:90% c in C, the weight of c. 00:35:55.370 --> 00:35:59.870 align:middle line:90% Because, well, why? 00:35:59.870 --> 00:36:05.530 align:middle line:84% Because the weight of c1 minus c2 00:36:05.530 --> 00:36:10.180 align:middle line:84% is equal to the Hamming distance between c1 and c2. 00:36:10.180 --> 00:36:13.250 align:middle line:90% 00:36:13.250 --> 00:36:16.610 align:middle line:84% So for linear code c, you want c to be 00:36:16.610 --> 00:36:21.160 align:middle line:84% able to correct d minus 1 over 2 errors. 00:36:21.160 --> 00:37:09.228 align:middle line:90% 00:37:09.228 --> 00:37:19.300 align:middle line:84% So the generating G for Hamming code 00:37:19.300 --> 00:37:32.006 align:middle line:84% was G equal 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 00:37:32.006 --> 00:37:38.680 align:middle line:84% 0, 1, 0, 1, and 0, 0, 0, 1, 1, 1, 1. 00:37:38.680 --> 00:37:43.260 align:middle line:90% 00:37:43.260 --> 00:37:47.790 align:middle line:84% OK, I want to claim you can define a Hamming code with block 00:37:47.790 --> 00:37:50.700 align:middle line:90% length 2 to the k minus 1. 00:37:50.700 --> 00:38:26.488 align:middle line:90% 00:38:26.488 --> 00:38:37.510 align:middle line:84% n equals 2 to the n minus k minus 1. 00:38:37.510 --> 00:38:40.310 align:middle line:84% So here for this Hamming code k, which 00:38:40.310 --> 00:38:47.000 align:middle line:84% is the number of bits it encodes is 4. 00:38:47.000 --> 00:38:51.920 align:middle line:84% And the block length is 7, which is 2 to the 7 minus 4. 00:38:51.920 --> 00:38:55.480 align:middle line:90% That's 8 minus 1 is 7. 00:38:55.480 --> 00:38:56.750 align:middle line:90% And how do we do that? 00:38:56.750 --> 00:39:22.520 align:middle line:90% 00:39:22.520 --> 00:39:25.800 align:middle line:84% Well, we're going to do the same thing as we did before-- 00:39:25.800 --> 00:39:34.440 align:middle line:90% 1, 1, 1, 1, 1, 1, 1, 1. 00:39:34.440 --> 00:39:36.550 align:middle line:90% So that you see is-- 00:39:36.550 --> 00:39:56.780 align:middle line:90% 00:39:56.780 --> 00:40:01.380 align:middle line:84% and then, OK, so we put the identity matrix here. 00:40:01.380 --> 00:40:24.460 align:middle line:84% And now we're going to put all binary strings of length 00:40:24.460 --> 00:40:42.090 align:middle line:84% s with at least two 1's, so 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 00:40:42.090 --> 00:40:50.190 align:middle line:84% 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1. 00:40:50.190 --> 00:40:58.450 align:middle line:84% So I guess there are six here, then four, 1, 1, 0, 1, 1, 0, 1, 00:40:58.450 --> 00:41:00.750 align:middle line:90% 1, 1, 1, 1. 00:41:00.750 --> 00:41:06.080 align:middle line:90% So this is 11, and this is 15. 00:41:06.080 --> 00:41:13.470 align:middle line:90% 00:41:13.470 --> 00:41:19.190 align:middle line:84% So here, k equals 11, n equals 15. 00:41:19.190 --> 00:41:28.150 align:middle line:84% k equals-- well, there are 2 to the s strings of length s total. 00:41:28.150 --> 00:41:33.150 align:middle line:84% And if we remove the ones with 1, 1 and the ones with 0, 1's, 00:41:33.150 --> 00:41:36.770 align:middle line:84% you get 2 to the s minus s minus 1. 00:41:36.770 --> 00:41:43.760 align:middle line:90% 00:41:43.760 --> 00:41:51.720 align:middle line:90% And n has s more columns than-- 00:41:51.720 --> 00:41:54.760 align:middle line:84% I mean, so this identity matrix, has an equal number 00:41:54.760 --> 00:41:56.080 align:middle line:90% of rows and columns. 00:41:56.080 --> 00:42:06.720 align:middle line:84% So we have s more columns than k. 00:42:06.720 --> 00:42:13.920 align:middle line:84% So that's k plus s equals 2 to the s minus 1. 00:42:13.920 --> 00:42:17.070 align:middle line:84% So this is the 15, 11 Hamming code. 00:42:17.070 --> 00:42:34.640 align:middle line:90% 00:42:34.640 --> 00:43:12.090 align:middle line:84% And now we have to explain why does it correct 00:43:12.090 --> 00:43:19.800 align:middle line:90% Why does it correct one error? 00:43:19.800 --> 00:43:26.170 align:middle line:90% 00:43:26.170 --> 00:43:42.020 align:middle line:84% Need to show all row sums, minimum weight 3. 00:43:42.020 --> 00:43:47.180 align:middle line:90% 00:43:47.180 --> 00:43:50.010 align:middle line:84% Are all non-zero row sums have minimum weight 3. 00:43:50.010 --> 00:43:54.020 align:middle line:90% 00:43:54.020 --> 00:43:56.410 align:middle line:84% So let's start, start with easy case. 00:43:56.410 --> 00:44:02.900 align:middle line:90% 00:44:02.900 --> 00:44:10.960 align:middle line:90% All rows have minimum weight 3. 00:44:10.960 --> 00:44:14.020 align:middle line:84% Or rather, all rows have weight at least 3. 00:44:14.020 --> 00:44:16.140 align:middle line:90% Why is that? 00:44:16.140 --> 00:44:19.420 align:middle line:90% Well, let's pick any row. 00:44:19.420 --> 00:44:22.040 align:middle line:90% Let's say this one. 00:44:22.040 --> 00:44:34.670 align:middle line:90% 00:44:34.670 --> 00:44:43.790 align:middle line:84% The identity matrix has 1 bit in it, 00:44:43.790 --> 00:44:46.850 align:middle line:84% and the parity check bits have at least 2 bits in them. 00:44:46.850 --> 00:44:59.110 align:middle line:90% 00:44:59.110 --> 00:45:08.780 align:middle line:84% 1 bit, parity check bits, 2 bits. 00:45:08.780 --> 00:45:16.250 align:middle line:90% 00:45:16.250 --> 00:45:17.560 align:middle line:90% Any sum of two rows-- 00:45:17.560 --> 00:45:25.710 align:middle line:90% 00:45:25.710 --> 00:45:38.960 align:middle line:84% well, let's take two rows and look at their sum. 00:45:38.960 --> 00:45:44.560 align:middle line:84% There are two different ones in the identity matrix part. 00:45:44.560 --> 00:45:48.400 align:middle line:84% So we need at least one 1 in the parity check bits. 00:45:48.400 --> 00:45:52.120 align:middle line:84% But we arrange these parity check bits so that they were all 00:45:52.120 --> 00:45:53.000 align:middle line:90% different. 00:45:53.000 --> 00:45:56.680 align:middle line:84% And when you take two non-different strings 00:45:56.680 --> 00:46:00.160 align:middle line:84% and you sum them mod 2, you don't get 0. 00:46:00.160 --> 00:46:02.560 align:middle line:90% You get at least 1. 00:46:02.560 --> 00:46:08.120 align:middle line:84% So here, the identity matrix part 00:46:08.120 --> 00:46:18.920 align:middle line:84% has two bits, and the parity check, at least 1 bit. 00:46:18.920 --> 00:46:20.700 align:middle line:84% I should have said at least 2 bits there. 00:46:20.700 --> 00:46:23.560 align:middle line:90% 00:46:23.560 --> 00:46:38.170 align:middle line:84% And any sum of three rows or more, the identity part 00:46:38.170 --> 00:46:39.500 align:middle line:90% has at least three bits. 00:46:39.500 --> 00:46:53.650 align:middle line:90% 00:46:53.650 --> 00:46:55.370 align:middle line:84% So the last thing I want to do today 00:46:55.370 --> 00:47:06.250 align:middle line:84% is tell you how to correct errors in linear codes, because. 00:47:06.250 --> 00:47:16.038 align:middle line:90% 00:47:16.038 --> 00:47:17.792 align:middle line:90% AUDIENCE: I have a question. 00:47:17.792 --> 00:47:18.500 align:middle line:90% PETER SHOR: Yeah. 00:47:18.500 --> 00:47:24.998 align:middle line:84% AUDIENCE: Why is that k 2 to the third minus y minus 1? 00:47:24.998 --> 00:47:26.250 align:middle line:90% PETER SHOR: OK. 00:47:26.250 --> 00:47:32.020 align:middle line:84% So we're looking at all binary strings of length s with 00:47:32.020 --> 00:47:34.340 align:middle line:90% at least two 1's. 00:47:34.340 --> 00:47:37.420 align:middle line:84% So how many binary strings of length s are there? 00:47:37.420 --> 00:47:42.060 align:middle line:84% There are 2 to the s binary strings of length s. 00:47:42.060 --> 00:47:45.780 align:middle line:84% And now we have to subtract out the ones that have 1, 1 or 0, 00:47:45.780 --> 00:47:46.580 align:middle line:90% 1's. 00:47:46.580 --> 00:47:49.220 align:middle line:84% There are s 1's with 1, 1, because the 1 can 00:47:49.220 --> 00:47:51.180 align:middle line:90% be in any of the s positions. 00:47:51.180 --> 00:47:55.140 align:middle line:84% And there's one string with 0, 1's, which is all-zero strings. 00:47:55.140 --> 00:47:58.860 align:middle line:84% So the number of binary strings with at least two 1's 00:47:58.860 --> 00:48:03.340 align:middle line:90% is 2 to the s minus s minus 1. 00:48:03.340 --> 00:48:09.220 align:middle line:84% And then that's the number of rows in this matrix. 00:48:09.220 --> 00:48:11.940 align:middle line:84% To get the number of columns in this matrix, 00:48:11.940 --> 00:48:15.860 align:middle line:84% well, we just take the number of rows, because the identity 00:48:15.860 --> 00:48:19.340 align:middle line:84% matrix has an equal number of rows and columns, and add s. 00:48:19.340 --> 00:48:24.820 align:middle line:90% And that's 2 to the s minus 1. 00:48:24.820 --> 00:48:28.080 align:middle line:84% So this is the class of Hamming codes. 00:48:28.080 --> 00:48:36.770 align:middle line:90% 00:48:36.770 --> 00:48:39.130 align:middle line:84% I should say something about them later. 00:48:39.130 --> 00:48:48.760 align:middle line:84% But let me first tell you how to correct errors in linear codes. 00:48:48.760 --> 00:49:11.350 align:middle line:90% 00:49:11.350 --> 00:49:11.850 align:middle line:90% OK. 00:49:11.850 --> 00:49:17.630 align:middle line:90% 00:49:17.630 --> 00:49:20.330 align:middle line:84% Oh, wait, I want to pull this down. 00:49:20.330 --> 00:49:31.760 align:middle line:90% 00:49:31.760 --> 00:49:55.360 align:middle line:84% Well, to work with the code, need to encode a message, 00:49:55.360 --> 00:49:59.660 align:middle line:90% identify positions with error. 00:49:59.660 --> 00:50:05.125 align:middle line:90% 00:50:05.125 --> 00:50:07.500 align:middle line:84% And once you've identified the positions with the errors, 00:50:07.500 --> 00:50:10.070 align:middle line:84% you can fix them and then decode. 00:50:10.070 --> 00:50:19.560 align:middle line:90% 00:50:19.560 --> 00:50:43.810 align:middle line:84% And we defined generator matrices, 00:50:43.810 --> 00:51:00.730 align:middle line:84% generator matrix for code G equals code 00:51:00.730 --> 00:51:04.340 align:middle line:90% is set of all row sums. 00:51:04.340 --> 00:51:15.970 align:middle line:90% 00:51:15.970 --> 00:51:27.740 align:middle line:84% To encode, use C equals m times G. So m is the message, 00:51:27.740 --> 00:51:32.290 align:middle line:84% c is the code word, and G is a generator matrix. 00:51:32.290 --> 00:51:42.260 align:middle line:90% 00:51:42.260 --> 00:51:56.820 align:middle line:84% So if m equals 1, 0, 0, 1, mG is the sum 00:51:56.820 --> 00:51:59.240 align:middle line:84% of the first row and the last row, 00:51:59.240 --> 00:52:03.160 align:middle line:84% because this is the first one and the last one. 00:52:03.160 --> 00:52:06.954 align:middle line:90% So it's 1, 0, 0, 1, 0, 0, 1. 00:52:06.954 --> 00:52:13.940 align:middle line:90% 00:52:13.940 --> 00:52:15.800 align:middle line:90% So encoding is easy. 00:52:15.800 --> 00:52:20.670 align:middle line:90% 00:52:20.670 --> 00:52:23.030 align:middle line:84% To decode, we need the parity check matrix. 00:52:23.030 --> 00:52:40.790 align:middle line:90% 00:52:40.790 --> 00:52:48.230 align:middle line:90% H is defined as-- 00:52:48.230 --> 00:52:55.430 align:middle line:90% well, we want GH equals 0. 00:52:55.430 --> 00:53:09.544 align:middle line:84% And we want H to have maximum rank so that GH, H maximum rank 00:53:09.544 --> 00:53:13.770 align:middle line:90% with this condition. 00:53:13.770 --> 00:53:24.400 align:middle line:90% 00:53:24.400 --> 00:53:39.380 align:middle line:90% OK, let's leave this up. 00:53:39.380 --> 00:53:51.080 align:middle line:90% 00:53:51.080 --> 00:53:52.780 align:middle line:90% And I will say something. 00:53:52.780 --> 00:53:57.080 align:middle line:90% 00:53:57.080 --> 00:54:08.520 align:middle line:84% For all codes, all linear codes, we 00:54:08.520 --> 00:54:30.650 align:middle line:84% can find a generator matrix G, G of the form 1, 1, 1, 1, 1, 1. 00:54:30.650 --> 00:54:35.210 align:middle line:84% So this is the identity matrix and some matrix here s. 00:54:35.210 --> 00:54:46.310 align:middle line:84% So you see, here the second part is s 00:54:46.310 --> 00:54:50.690 align:middle line:84% and this first part is the identity. 00:54:50.690 --> 00:54:57.970 align:middle line:84% And by general linear algebra configurations proof, 00:54:57.970 --> 00:55:00.590 align:middle line:84% proof is really Gaussian elimination. 00:55:00.590 --> 00:55:10.450 align:middle line:90% 00:55:10.450 --> 00:55:14.050 align:middle line:84% You take the code and you do Gaussian elimination on it, 00:55:14.050 --> 00:55:17.140 align:middle line:84% and then you rearrange the order of the coordinates 00:55:17.140 --> 00:55:20.780 align:middle line:90% to get this identity. 00:55:20.780 --> 00:55:24.820 align:middle line:84% And I'm not going to go into further details, 00:55:24.820 --> 00:55:31.860 align:middle line:84% because they're not that important for this, 00:55:31.860 --> 00:55:33.890 align:middle line:84% and they're not in the notes either. 00:55:33.890 --> 00:55:43.380 align:middle line:90% 00:55:43.380 --> 00:55:53.425 align:middle line:84% And I want to claim that here H is equal to minus s I, 00:55:53.425 --> 00:55:59.980 align:middle line:84% where you have a minus s here and an I of the right form. 00:55:59.980 --> 00:56:13.550 align:middle line:84% So H is equal to minus 1 minus 1, 0. 00:56:13.550 --> 00:56:17.230 align:middle line:84% I feel stupid putting these minus 1's on because minus 1 00:56:17.230 --> 00:56:18.880 align:middle line:90% equals 1-- 00:56:18.880 --> 00:56:24.350 align:middle line:90% 00:56:24.350 --> 00:56:30.950 align:middle line:84% there's another row here, minus 1, minus 1, minus 1, 1, 1, 1-- 00:56:30.950 --> 00:56:34.530 align:middle line:84% because minus 1 equals 1 in binary. 00:56:34.530 --> 00:56:41.430 align:middle line:84% So this really could be simplified to this. 00:56:41.430 --> 00:56:44.970 align:middle line:84% But for non-binary codes, you really need the minus s. 00:56:44.970 --> 00:56:48.230 align:middle line:90% 00:56:48.230 --> 00:56:49.420 align:middle line:90% And you can see that-- 00:56:49.420 --> 00:56:52.190 align:middle line:90% 00:56:52.190 --> 00:56:54.780 align:middle line:90% oh, well, I will explain. 00:56:54.780 --> 00:57:02.590 align:middle line:90% 00:57:02.590 --> 00:57:06.870 align:middle line:90% And G equals I s. 00:57:06.870 --> 00:57:10.400 align:middle line:84% And note this I is a different size identity 00:57:10.400 --> 00:57:12.560 align:middle line:90% matrix than this I. 00:57:12.560 --> 00:57:24.280 align:middle line:84% But anyway, GH is equal to I s minus s I. 00:57:24.280 --> 00:57:26.760 align:middle line:84% And when you do matrix multiplication here, 00:57:26.760 --> 00:57:40.160 align:middle line:84% you get I times minus s plus s times I equals I times minus s, 00:57:40.160 --> 00:57:43.230 align:middle line:90% plus s times I equals 0. 00:57:43.230 --> 00:57:53.040 align:middle line:90% 00:57:53.040 --> 00:58:02.120 align:middle line:84% So GH is 0, which was our condition for parity check 00:58:02.120 --> 00:58:03.240 align:middle line:90% matrix. 00:58:03.240 --> 00:58:10.050 align:middle line:84% And you can also see that H is the maximum rank matrix 00:58:10.050 --> 00:58:17.935 align:middle line:84% such that GH equals 0, because, well, that's linear algebra 2. 00:58:17.935 --> 00:58:35.730 align:middle line:90% 00:58:35.730 --> 00:58:50.050 align:middle line:84% I mean, I want to say H transpose generates 00:58:50.050 --> 00:59:07.900 align:middle line:84% the perpendicular space to G. And the dimension of H transpose 00:59:07.900 --> 00:59:13.020 align:middle line:84% plus the dimension of G is equal to n, the dimension 00:59:13.020 --> 00:59:15.420 align:middle line:90% of the entire space. 00:59:15.420 --> 00:59:24.740 align:middle line:84% And this is true for any linear spaces, 00:59:24.740 --> 00:59:32.260 align:middle line:84% even those over finite fields, even though G intersect H 00:59:32.260 --> 00:59:36.112 align:middle line:90% transpose might be non-empty. 00:59:36.112 --> 00:59:38.620 align:middle line:90% 00:59:38.620 --> 00:59:43.860 align:middle line:84% So I mean, you guys are used to thinking of linear algebra 00:59:43.860 --> 00:59:45.420 align:middle line:90% over the reals. 00:59:45.420 --> 00:59:52.380 align:middle line:84% And then you have a space s and a perpendicular space s perp. 00:59:52.380 --> 00:59:55.020 align:middle line:84% And s intersect s perp is just 0. 00:59:55.020 --> 00:59:56.350 align:middle line:90% This isn't true here. 00:59:56.350 --> 01:00:01.100 align:middle line:90% 01:00:01.100 --> 01:00:02.000 align:middle line:90% Let's see. 01:00:02.000 --> 01:00:14.270 align:middle line:90% 01:00:14.270 --> 01:00:19.470 align:middle line:84% Look at the last row in G. It's perpendicular to itself. 01:00:19.470 --> 01:00:28.030 align:middle line:84% So the last row in G is also in H, 01:00:28.030 --> 01:00:30.170 align:middle line:90% in the row space of H transpose. 01:00:30.170 --> 01:00:37.630 align:middle line:84% Because everything that's perpendicular to all the guys 01:00:37.630 --> 01:00:41.650 align:middle line:90% in G is in H transpose. 01:00:41.650 --> 01:00:43.350 align:middle line:84% And you can see that the last row 01:00:43.350 --> 01:00:48.350 align:middle line:84% is perpendicular to all the other rows in G. 01:00:48.350 --> 01:00:57.790 align:middle line:84% So there's some linear combination of columns in H 01:00:57.790 --> 01:00:58.770 align:middle line:90% that gives you that. 01:00:58.770 --> 01:01:00.950 align:middle line:84% And you can see it's the sum of all three columns, 01:01:00.950 --> 01:01:05.920 align:middle line:84% because you need 1, 1, 1 in the last three entries 01:01:05.920 --> 01:01:07.040 align:middle line:90% in the column. 01:01:07.040 --> 01:01:17.680 align:middle line:84% And then the sum of the first three 01:01:17.680 --> 01:01:25.640 align:middle line:84% is the sum of these four columns is 0, 0, 0, 1. 01:01:25.640 --> 01:01:34.420 align:middle line:84% So a space and its perpendicular space can intersect, 01:01:34.420 --> 01:01:35.680 align:middle line:90% but that doesn't matter. 01:01:35.680 --> 01:01:38.730 align:middle line:84% So this is the parity check matrix. 01:01:38.730 --> 01:01:43.400 align:middle line:90% 01:01:43.400 --> 01:02:02.330 align:middle line:84% And now suppose we have a linear code with G 01:02:02.330 --> 01:02:10.320 align:middle line:84% and H are the generators matrix and the parity check matrix. 01:02:10.320 --> 01:02:15.410 align:middle line:90% 01:02:15.410 --> 01:02:21.040 align:middle line:90% Encode, we get mG error. 01:02:21.040 --> 01:02:23.570 align:middle line:90% 01:02:23.570 --> 01:02:28.850 align:middle line:90% We have mG plus e. 01:02:28.850 --> 01:02:39.930 align:middle line:84% And the next step is compute the syndrome of c tilde, 01:02:39.930 --> 01:02:47.370 align:middle line:84% and that's c tilde times H. This is 01:02:47.370 --> 01:03:00.900 align:middle line:84% mG plus e times H is equal to mGH plus eH, which is just eH. 01:03:00.900 --> 01:03:03.530 align:middle line:84% Because remember, G times H was 0. 01:03:03.530 --> 01:03:08.300 align:middle line:90% 01:03:08.300 --> 01:03:10.110 align:middle line:90% And this is called syndrome. 01:03:10.110 --> 01:03:35.220 align:middle line:90% 01:03:35.220 --> 01:03:39.060 align:middle line:84% And it's called the syndrome because it's what 01:03:39.060 --> 01:03:41.010 align:middle line:90% you use to diagnose the error. 01:03:41.010 --> 01:03:44.260 align:middle line:90% 01:03:44.260 --> 01:03:58.790 align:middle line:84% So syndrome depends only on the error 01:03:58.790 --> 01:04:10.300 align:middle line:84% and not on the encoded code word, not on the message. 01:04:10.300 --> 01:04:13.330 align:middle line:90% 01:04:13.330 --> 01:04:13.830 align:middle line:90% OK. 01:04:13.830 --> 01:04:18.710 align:middle line:90% 01:04:18.710 --> 01:04:30.410 align:middle line:90% How do you go from s to e? 01:04:30.410 --> 01:04:33.550 align:middle line:90% 01:04:33.550 --> 01:04:39.500 align:middle line:90% Well, it's not easy. 01:04:39.500 --> 01:04:42.190 align:middle line:90% 01:04:42.190 --> 01:04:48.390 align:middle line:90% Depends on the code. 01:04:48.390 --> 01:04:52.030 align:middle line:84% There is no efficient algorithm for taking 01:04:52.030 --> 01:04:55.560 align:middle line:84% a syndrome for an arbitrary code and computing the error. 01:04:55.560 --> 01:04:59.400 align:middle line:84% You have to do something that is codependent, 01:04:59.400 --> 01:05:03.320 align:middle line:90% and we're not going to-- 01:05:03.320 --> 01:05:05.680 align:middle line:84% I mean, there could be lots and lots 01:05:05.680 --> 01:05:08.200 align:middle line:84% of different ways going from the syndrome to the error, 01:05:08.200 --> 01:05:09.620 align:middle line:90% depending on the code. 01:05:09.620 --> 01:05:11.580 align:middle line:84% And some of them can be quite complicated. 01:05:11.580 --> 01:05:13.040 align:middle line:90% Some of them are easy. 01:05:13.040 --> 01:05:14.890 align:middle line:84% For Hamming codes, it's very easy. 01:05:14.890 --> 01:05:27.000 align:middle line:90% 01:05:27.000 --> 01:05:39.310 align:middle line:84% Code, assume error has 1 bit, or Hamming weight 1. 01:05:39.310 --> 01:05:42.120 align:middle line:90% 01:05:42.120 --> 01:05:46.988 align:middle line:84% So 1 bit equal 1, and all the others are 0-- 01:05:46.988 --> 01:05:52.240 align:middle line:90% 0, 0, 0, 1, 0, 0, 0. 01:05:52.240 --> 01:06:01.970 align:middle line:84% Let's make this the kth bit times the Hamming matrix 01:06:01.970 --> 01:06:05.210 align:middle line:90% is kth-- 01:06:05.210 --> 01:06:11.250 align:middle line:84% not Hamming matrix-- times the parity check matrix. 01:06:11.250 --> 01:06:15.330 align:middle line:84% So if you have 1, if you have a vector with 1 01:06:15.330 --> 01:06:17.110 align:middle line:84% and you multiply it by the matrix, 01:06:17.110 --> 01:06:18.900 align:middle line:84% you just get the kth row of the matrix. 01:06:18.900 --> 01:06:42.850 align:middle line:90% 01:06:42.850 --> 01:06:48.930 align:middle line:84% So because the Hamming code can only correct 1 bit errors, 01:06:48.930 --> 01:06:51.500 align:middle line:84% we only need to worry about the case 01:06:51.500 --> 01:06:59.720 align:middle line:84% where the error e has 1 bit in it. 01:06:59.720 --> 01:07:01.980 align:middle line:90% And that case is easy to decode. 01:07:01.980 --> 01:07:07.760 align:middle line:84% So let's do an example, because I think we have just enough-- 01:07:07.760 --> 01:07:11.020 align:middle line:84% well, we probably have more than enough time for an example, 01:07:11.020 --> 01:07:18.220 align:middle line:90% but we should do an example. 01:07:18.220 --> 01:07:30.830 align:middle line:84% Encode 1, 1, 1, 0 in our 7-bit Hamming code. 01:07:30.830 --> 01:07:36.940 align:middle line:90% 01:07:36.940 --> 01:07:39.580 align:middle line:84% So there is our 7-bit Hamming code up there. 01:07:39.580 --> 01:07:46.340 align:middle line:84% We sum the first three rows, because we have the first three 01:07:46.340 --> 01:07:50.950 align:middle line:90% entries of our message are 1. 01:07:50.950 --> 01:08:03.070 align:middle line:84% m equals 1, 1, 1, so that's, well, it's 1, 1, 1, 0, 0, 0, 0. 01:08:03.070 --> 01:08:13.550 align:middle line:90% 01:08:13.550 --> 01:08:26.330 align:middle line:84% Add error, and you get, let's say, e equals 0, 0, 0, 0, 0 1, 01:08:26.330 --> 01:08:26.830 align:middle line:90% 0. 01:08:26.830 --> 01:08:31.531 align:middle line:90% We get 1, 1, 1, 0, 0, 1, 0. 01:08:31.531 --> 01:08:36.229 align:middle line:90% 01:08:36.229 --> 01:08:42.430 align:middle line:84% So now we have mG plus e times H. 01:08:42.430 --> 01:08:46.590 align:middle line:84% So that's the sum of the first three rows of H. 01:08:46.590 --> 01:08:48.720 align:middle line:90% And that's zeros. 01:08:48.720 --> 01:08:57.028 align:middle line:84% And then add the fifth row of H, that's 0, 1, 0 equals 0, 1, 0. 01:08:57.028 --> 01:08:58.070 align:middle line:90% And this is the syndrome. 01:08:58.070 --> 01:09:01.960 align:middle line:90% 01:09:01.960 --> 01:09:13.779 align:middle line:84% 0, 1, 0 is the sixth row of H. I think I said fifth here, 01:09:13.779 --> 01:09:15.119 align:middle line:90% but I meant sixth-- 01:09:15.119 --> 01:09:34.200 align:middle line:84% the sixth row of H, error in position 6, 0, 0, 0, 0, 0, 1, 0. 01:09:34.200 --> 01:09:39.359 align:middle line:90% And this was c tilde. 01:09:39.359 --> 01:09:49.010 align:middle line:84% So c tilde plus e is equal to 1, 1, 1, 0, 0, 0, 0. 01:09:49.010 --> 01:09:51.759 align:middle line:84% And this first bit is the message. 01:09:51.759 --> 01:10:11.530 align:middle line:90% 01:10:11.530 --> 01:10:27.410 align:middle line:84% Now there's one more section in the notes about Hamming codes. 01:10:27.410 --> 01:10:29.890 align:middle line:84% I don't want to get into Reed-Solomon codes 01:10:29.890 --> 01:10:33.410 align:middle line:84% this time, because I want to do them all next time. 01:10:33.410 --> 01:10:35.240 align:middle line:90% And that's about perfect codes. 01:10:35.240 --> 01:11:12.220 align:middle line:90% 01:11:12.220 --> 01:11:49.190 align:middle line:84% A perfect code is one all of whose syndromes correspond 01:11:49.190 --> 01:12:00.830 align:middle line:90% to an error of t bits or fewer. 01:12:00.830 --> 01:12:04.030 align:middle line:84% If it's a t error-correcting perfect code, 01:12:04.030 --> 01:12:06.190 align:middle line:90% all of the syndromes correspond. 01:12:06.190 --> 01:12:10.180 align:middle line:84% So the binary, the Hamming codes are perfect codes. 01:12:10.180 --> 01:12:22.430 align:middle line:90% 01:12:22.430 --> 01:12:24.730 align:middle line:84% So how many syndromes do they have? 01:12:24.730 --> 01:12:36.030 align:middle line:90% 01:12:36.030 --> 01:12:54.540 align:middle line:84% Well, their syndromes are length s bits. 01:12:54.540 --> 01:13:08.120 align:middle line:90% 01:13:08.120 --> 01:13:11.660 align:middle line:90% So there's 2 to the s syndromes. 01:13:11.660 --> 01:13:16.120 align:middle line:90% 01:13:16.120 --> 01:13:20.410 align:middle line:84% And the code has 2 to the s minus 1 bits. 01:13:20.410 --> 01:13:34.520 align:middle line:90% 01:13:34.520 --> 01:13:37.840 align:middle line:84% So I want to say, first, the zero syndrome 01:13:37.840 --> 01:13:40.210 align:middle line:90% corresponds to no error. 01:13:40.210 --> 01:13:42.170 align:middle line:84% And every other one of these syndromes 01:13:42.170 --> 01:13:45.730 align:middle line:90% corresponds to exactly 1 bit. 01:13:45.730 --> 01:13:48.320 align:middle line:84% So there's exactly as many syndromes as bits. 01:13:48.320 --> 01:13:55.846 align:middle line:90% 01:13:55.846 --> 01:14:00.860 align:middle line:84% And you can ask, are there any other perfect codes? 01:14:00.860 --> 01:14:46.258 align:middle line:90% 01:14:46.258 --> 01:15:02.380 align:middle line:84% So to have a perfect code, you need all the possible errors. 01:15:02.380 --> 01:15:08.800 align:middle line:84% So let's say L equals 0 through t. 01:15:08.800 --> 01:15:12.980 align:middle line:90% 01:15:12.980 --> 01:15:16.980 align:middle line:84% How many ways are there of getting 01:15:16.980 --> 01:15:19.660 align:middle line:90% an error of exactly L bits? 01:15:19.660 --> 01:15:29.980 align:middle line:84% It's n choose L is equal to 2 to the something to the number 01:15:29.980 --> 01:15:32.820 align:middle line:90% of bits in the syndrome. 01:15:32.820 --> 01:15:35.680 align:middle line:84% So this is a condition you need for perfect codes. 01:15:35.680 --> 01:15:39.990 align:middle line:90% 01:15:39.990 --> 01:15:41.450 align:middle line:90% And it's satisfied. 01:15:41.450 --> 01:15:46.070 align:middle line:90% 01:15:46.070 --> 01:15:55.430 align:middle line:84% Or this is a solution you need for binary perfect codes, 01:15:55.430 --> 01:15:57.700 align:middle line:90% for Hamming codes. 01:15:57.700 --> 01:16:02.230 align:middle line:90% 01:16:02.230 --> 01:16:05.360 align:middle line:84% And it's also satisfied, if I'm remembering these numbers right, 01:16:05.360 --> 01:16:11.070 align:middle line:90% n equals 23 and t equals 3. 01:16:11.070 --> 01:16:16.230 align:middle line:84% So there's exactly one solution, other than the one 01:16:16.230 --> 01:16:18.590 align:middle line:90% you get for Hamming codes. 01:16:18.590 --> 01:16:23.350 align:middle line:84% And remarkably, maybe it's remarkably, 01:16:23.350 --> 01:16:28.220 align:middle line:84% this is something called the Golay code. 01:16:28.220 --> 01:16:30.830 align:middle line:90% 01:16:30.830 --> 01:16:37.340 align:middle line:84% And I should have looked this up before coming to class, 01:16:37.340 --> 01:16:40.160 align:middle line:90% but I didn't get around to it. 01:16:40.160 --> 01:16:48.640 align:middle line:84% This was discovered by Golay sometime after Hamming's paper. 01:16:48.640 --> 01:16:57.680 align:middle line:84% But the construction had already been found by a football-- 01:16:57.680 --> 01:17:00.440 align:middle line:84% although you should call that soccer-- 01:17:00.440 --> 01:17:02.800 align:middle line:84% enthusiast who I think was finished 01:17:02.800 --> 01:17:06.880 align:middle line:84% in coming up with a betting system for soccer games-- 01:17:06.880 --> 01:17:08.380 align:middle line:90% football games, I should say. 01:17:08.380 --> 01:17:11.560 align:middle line:90% 01:17:11.560 --> 01:17:21.280 align:middle line:84% So this is actually a remarkable mathematical construction, 01:17:21.280 --> 01:17:24.160 align:middle line:84% which is related to finite simple groups. 01:17:24.160 --> 01:17:31.682 align:middle line:84% But I don't have time, I mean, we don't have the mathematics 01:17:31.682 --> 01:17:34.390 align:middle line:84% to go into that, and I certainly don't have time to go into that. 01:17:34.390 --> 01:17:40.730 align:middle line:84% But this is the one other binary perfect code. 01:17:40.730 --> 01:17:45.330 align:middle line:84% There's also a code over z sub 3 called 01:17:45.330 --> 01:17:53.170 align:middle line:84% the ternary-Golay code, where you have a 3 to the s here. 01:17:53.170 --> 01:18:02.610 align:middle line:84% And, I guess, a 3 to the something on this one 01:18:02.610 --> 01:18:05.130 align:middle line:90% as well, 3 to the L, I think. 01:18:05.130 --> 01:18:16.710 align:middle line:84% But anyway, if you want to use perfect code-- 01:18:16.710 --> 01:18:19.273 align:middle line:84% I mean, so at first, after discovering the Hamming code, 01:18:19.273 --> 01:18:20.690 align:middle line:84% you would say, well, perfect codes 01:18:20.690 --> 01:18:25.520 align:middle line:84% are exactly what we want to use, but there aren't enough of them. 01:18:25.520 --> 01:18:31.000 align:middle line:90%