WEBVTT 00:00:15.370 --> 00:00:17.450 PROFESSOR: And what we saw was the performance was quite 00:00:17.450 --> 00:00:20.910 different in the two regimes. 00:00:20.910 --> 00:00:23.900 In power-limited regime, typically our SNR is much 00:00:23.900 --> 00:00:28.930 smaller than one, whereas in the bandwidth-limited regime, 00:00:28.930 --> 00:00:33.360 the SNR is large. 00:00:33.360 --> 00:00:36.370 And because of this behavior of SNR, what we saw is that 00:00:36.370 --> 00:00:38.650 the Shannon spectral efficiency in the 00:00:38.650 --> 00:00:42.010 bandwidth-limited regime, it doubles for every -- 00:00:42.010 --> 00:00:43.810 if we double our SNR. 00:00:43.810 --> 00:00:47.670 For every 3 dB increase in SNR, the spectral efficiency 00:00:47.670 --> 00:00:50.790 increases by a factor of 2. 00:00:50.790 --> 00:00:53.470 In the bandwidth limited regime, if we have a 3 dB 00:00:53.470 --> 00:00:57.070 increase in SNR, the spectral efficiency increases by one 00:00:57.070 --> 00:00:59.080 bit per two dimension. 00:00:59.080 --> 00:01:02.470 On the other hand, if we double our bandwidth, then the 00:01:02.470 --> 00:01:05.110 capacity in bits per second is not affected in the 00:01:05.110 --> 00:01:06.830 power-limited regime. 00:01:06.830 --> 00:01:09.540 But if we double our bandwidth, then in the 00:01:09.540 --> 00:01:12.970 bandwidth-limited regime, the capacity increases 00:01:12.970 --> 00:01:15.200 approximately by a factor of 2. 00:01:15.200 --> 00:01:17.010 And that's really what motivated the name 00:01:17.010 --> 00:01:19.680 bandwidth-limited and power-limited regime. 00:01:19.680 --> 00:01:23.260 If we want a more operational definition, then we say that 00:01:23.260 --> 00:01:26.390 the flow is less than two bits per two dimensions, we will 00:01:26.390 --> 00:01:29.330 have this bandwidth-limited regime. 00:01:29.330 --> 00:01:32.590 And in this case, rho is greater than two bits per two 00:01:32.590 --> 00:01:34.430 dimensions. 00:01:34.430 --> 00:01:37.430 The number two bits per two dimensions was chosen because 00:01:37.430 --> 00:01:40.360 it is like the largest spectral efficiency we can get 00:01:40.360 --> 00:01:42.210 through binary transmission. 00:01:42.210 --> 00:01:45.100 If it's an uncoded two-PAM over a channel, then we get 00:01:45.100 --> 00:01:46.990 two bits per two dimensions. 00:01:46.990 --> 00:01:49.220 If we are coding, the only thing we can do is reduce 00:01:49.220 --> 00:01:51.270 spectral efficiency. 00:01:51.270 --> 00:01:53.770 So typically, if we are going to operate in power-limited 00:01:53.770 --> 00:01:56.940 regime, the operational meaning is we can get away 00:01:56.940 --> 00:01:58.380 with binary transmission. 00:01:58.380 --> 00:02:00.900 In the bandwidth-limited regime, we have to resort to 00:02:00.900 --> 00:02:03.160 non-binary transmission. 00:02:03.160 --> 00:02:09.180 So in other words, binary modulation is done in 00:02:09.180 --> 00:02:17.460 power-limited regime, whereas we need multi-level modulation 00:02:17.460 --> 00:02:19.240 in the bandwidth-limited regime. 00:02:28.440 --> 00:02:32.290 The baseline system here is 2-PAM. 00:02:32.290 --> 00:02:35.190 The uncoded performance was of 2-PAM. 00:02:35.190 --> 00:02:38.920 In the bandwidth limited regime, it is M-PAM. 00:02:38.920 --> 00:02:40.680 And the way we measure performance in the 00:02:40.680 --> 00:02:44.950 power-limited regime is the probability of bit error as a 00:02:44.950 --> 00:02:46.200 function of EbN0. 00:02:50.406 --> 00:02:51.360 OK? 00:02:51.360 --> 00:02:54.260 In the bandwidth-limited regime, the performance 00:02:54.260 --> 00:02:59.080 measure is done by probability of error per two dimensions as 00:02:59.080 --> 00:03:00.340 a function of SNR norm. 00:03:07.600 --> 00:03:11.560 And in this case, we saw that the gap to capacity -- 00:03:11.560 --> 00:03:18.670 or rather, to put it in other words, the ultimate limit on 00:03:18.670 --> 00:03:31.090 EbN0 is minus 1.59 dB. 00:03:31.090 --> 00:03:47.810 And here, the ultimate limit on SNR norm is 0 dB. 00:03:59.440 --> 00:04:00.520 OK? 00:04:00.520 --> 00:04:04.100 Any questions on this? 00:04:04.100 --> 00:04:04.570 Yes. 00:04:04.570 --> 00:04:09.770 AUDIENCE: Why do we use Eb over N_0 SNR norm for 00:04:09.770 --> 00:04:10.460 [INAUDIBLE] 00:04:10.460 --> 00:04:11.226 bandwidth limited? 00:04:11.226 --> 00:04:13.050 PROFESSOR: That's a good question. 00:04:13.050 --> 00:04:16.839 AUDIENCE: Why do we use Eb over N_0 for both regimes? 00:04:16.839 --> 00:04:18.890 PROFESSOR: Or why don't use SNR norm for both regimes? 00:04:18.890 --> 00:04:19.545 AUDIENCE: Yeah. 00:04:19.545 --> 00:04:21.480 PROFESSOR: Now if we think about the bandwidth limited 00:04:21.480 --> 00:04:23.070 regime, what we really care about is 00:04:23.070 --> 00:04:25.930 spectral efficiency, right? 00:04:25.930 --> 00:04:29.080 What SNR norm does, if you remember the definition, is 00:04:29.080 --> 00:04:32.150 that it compass the amount of SNR we require for a practical 00:04:32.150 --> 00:04:35.400 system to that of the best possible system. 00:04:35.400 --> 00:04:37.310 So in other words, if you do care about spectral 00:04:37.310 --> 00:04:42.890 efficiency, SNR norm is the right measure to look for. 00:04:42.890 --> 00:04:46.230 OK, now what happens in the power-limited regime? 00:04:46.230 --> 00:04:49.310 It turns out, probably more for historic reasons, people 00:04:49.310 --> 00:04:52.120 started with EbN0 in the power-limited regime. 00:04:52.120 --> 00:05:00.985 And if you look at the definition of EbN0, it is SNR 00:05:00.985 --> 00:05:02.330 over rho, right? 00:05:02.330 --> 00:05:07.070 So in the power limited regime, our rho is small, the 00:05:07.070 --> 00:05:10.490 SNR is going to be small, but if you look at -- because we 00:05:10.490 --> 00:05:13.840 are in the power limited regime so we have lots of 00:05:13.840 --> 00:05:17.730 bandwidth, so our SNR is going to be small and the spectral 00:05:17.730 --> 00:05:19.450 efficiency is going to be small. 00:05:19.450 --> 00:05:23.240 But if you look at the ratio between the two, it's going to 00:05:23.240 --> 00:05:29.480 be greater than minus 1.59 dB. 00:05:29.480 --> 00:05:29.800 OK. 00:05:29.800 --> 00:05:33.610 so it turns out that the kind of limit we do take, our EbN0 00:05:33.610 --> 00:05:37.810 remains constant as minus 1.59 dB, and that's probably one of 00:05:37.810 --> 00:05:40.590 the reasons that motivated to use EbN0 in the 00:05:40.590 --> 00:05:44.180 power limited regime. 00:05:44.180 --> 00:05:46.590 On the other hand, one could also argue is that what really 00:05:46.590 --> 00:05:48.820 happens in the power limited regime is that our bandwidth 00:05:48.820 --> 00:05:50.370 becomes really large. 00:05:50.370 --> 00:05:51.330 So if this [UNINTELLIGIBLE] 00:05:51.330 --> 00:05:54.520 stick with 2-PAM system, then we do get a spectral 00:05:54.520 --> 00:05:57.720 efficiency of two bits per two dimensions, but that's just 00:05:57.720 --> 00:06:00.270 because we are using a particular modulation scheme. 00:06:00.270 --> 00:06:03.100 If our bandwidth is really large, we are not really going 00:06:03.100 --> 00:06:05.690 to care about what spectral efficiency we use. 00:06:05.690 --> 00:06:08.790 What really matters is this energy per bit, and that's why 00:06:08.790 --> 00:06:10.150 this is a reasonable assumption. 00:06:12.870 --> 00:06:15.710 Does that answer your question? 00:06:15.710 --> 00:06:20.010 Right, it's not completely clear as to why this EbN0 is 00:06:20.010 --> 00:06:23.370 the best here, and SNR norm is here if you don't take the 00:06:23.370 --> 00:06:26.410 limit rho going to zero here, but again, you can think of it 00:06:26.410 --> 00:06:28.970 more as a convention. 00:06:28.970 --> 00:06:29.360 OK. 00:06:29.360 --> 00:06:30.610 AUDIENCE: [INAUDIBLE] 00:06:33.170 --> 00:06:35.790 PROFESSOR: So if you look in the power-limited regime, you 00:06:35.790 --> 00:06:38.530 are saying rho is less than two bits per two dimensions. 00:06:38.530 --> 00:06:41.070 If you use an uncoded 2-PAM, what's your spectral 00:06:41.070 --> 00:06:42.800 efficiency? 00:06:42.800 --> 00:06:44.740 It's two bits per two dimension, right? 00:06:44.740 --> 00:06:47.900 Now the idea is, suppose we want to design a system with 00:06:47.900 --> 00:06:50.050 spectral efficiency greater than two bits per two 00:06:50.050 --> 00:06:51.150 dimensions? 00:06:51.150 --> 00:06:54.020 We cannot really use a 2-PAM system. 00:06:54.020 --> 00:06:56.685 Because if you put coding on top of it, all we are going to 00:06:56.685 --> 00:06:58.860 do is simply reduce the spectral efficiency below two 00:06:58.860 --> 00:07:00.730 bits per two dimension. 00:07:00.730 --> 00:07:05.150 So we have to start with a non-binary modulation, right? 00:07:05.150 --> 00:07:07.180 So that's how we distinguish between power-limited and 00:07:07.180 --> 00:07:09.426 bandwidth-limited 00:07:09.426 --> 00:07:12.157 AUDIENCE: That's just because you're using the 2-PAM as your 00:07:12.157 --> 00:07:14.728 baseline [INAUDIBLE]? 00:07:14.728 --> 00:07:17.620 PROFESSOR: Right. 00:07:17.620 --> 00:07:19.080 OK? 00:07:19.080 --> 00:07:21.610 All right. 00:07:21.610 --> 00:07:24.655 So let us do an example to finish off this analysis. 00:07:39.920 --> 00:07:41.730 Now suppose -- 00:07:41.730 --> 00:07:44.430 say you are at a summer project, and you're assigned 00:07:44.430 --> 00:07:46.410 to design some system. 00:07:46.410 --> 00:07:49.410 Your boss gives you some specifications, like 00:07:49.410 --> 00:07:51.740 continuous time specifications. 00:07:51.740 --> 00:07:54.665 In particular, you have a baseband system. 00:08:03.320 --> 00:08:06.575 The baseband system has a bandwidth of one Megahertz. 00:08:10.510 --> 00:08:18.590 You have a power, P, which is one unit, so that's another 00:08:18.590 --> 00:08:20.090 resource you have. 00:08:20.090 --> 00:08:22.280 And if you measure your channel, it can be reasonably 00:08:22.280 --> 00:08:26.070 approximated as an AWGN channel, so there is no ISI or 00:08:26.070 --> 00:08:29.450 any filtering going on, just Additive White Gaussian Noise. 00:08:29.450 --> 00:08:34.159 And your noise a single sided spectral density of ten to the 00:08:34.159 --> 00:08:39.566 minus six units per Hertz of the bandwidth. 00:08:42.400 --> 00:08:43.650 And what is your goal? 00:08:49.070 --> 00:08:50.650 So you have the following goal. 00:08:54.070 --> 00:09:05.240 Design a 2-PAM system, with a specified 00:09:05.240 --> 00:09:06.490 probability of bit error. 00:09:11.380 --> 00:09:24.470 And what you want to do is compare this to the ultimate 00:09:24.470 --> 00:09:25.720 Shannon limit. 00:09:33.080 --> 00:09:36.770 So that is your objective. 00:09:36.770 --> 00:09:40.540 So since we have to compare it with Shannon limit, and we 00:09:40.540 --> 00:09:42.740 have already a formula for the Shannon limit, let's just 00:09:42.740 --> 00:09:43.990 start with that. 00:09:50.320 --> 00:09:54.230 So for this problem, we have the Shannon limit. 00:09:57.060 --> 00:10:02.870 You have rho is less than log base 2 of 1 plus SNR. 00:10:02.870 --> 00:10:05.900 For SNR, I can talk in terms of this continuous time 00:10:05.900 --> 00:10:13.240 parameters, it's P over N_0 W. My P is one unit, and nought 00:10:13.240 --> 00:10:16.730 is ten to the minus six, and my bandwidth, W, is one 00:10:16.730 --> 00:10:19.570 Megahertz here, which is ten to the six. 00:10:19.570 --> 00:10:22.250 So my P over N_0 W is basically one. 00:10:22.250 --> 00:10:24.700 So I have 1 plus 1, which is 2. 00:10:24.700 --> 00:10:27.100 This is log base 2 of 2, or it's 00:10:27.100 --> 00:10:29.820 one bit per two dimension. 00:10:29.820 --> 00:10:37.100 So my capacity in bits per second is rho W, and rho is 00:10:37.100 --> 00:10:40.610 one bit per two dimension, the bandwidth is one Megahertz. 00:10:40.610 --> 00:10:44.090 So I get ten to the six bits per second. 00:10:44.090 --> 00:10:46.230 So this is my Shannon capacity for this 00:10:46.230 --> 00:10:47.870 particular AWGN system. 00:10:51.490 --> 00:10:52.740 OK. 00:10:54.320 --> 00:10:58.030 The next thing we want to do is compare this with a 00:10:58.030 --> 00:11:00.700 practical system, and see how close we get 00:11:00.700 --> 00:11:02.550 to the Shannon limit. 00:11:02.550 --> 00:11:06.080 And since you only have to work with 2-PAM, the generic 00:11:06.080 --> 00:11:10.080 architecture is something we saw last time. 00:11:10.080 --> 00:11:15.610 You have input bits coming in, let's call them X sub k, where 00:11:15.610 --> 00:11:17.230 k is the kth bit. 00:11:17.230 --> 00:11:20.550 And they belong to a certain constellation, 00:11:20.550 --> 00:11:21.280 let's call the -- 00:11:21.280 --> 00:11:24.920 the constellation points are just -- it's a 2-PAM system, 00:11:24.920 --> 00:11:27.690 so we have minus alpha and alpha. 00:11:27.690 --> 00:11:32.080 And this goes through a PAM modulator, and one parameter 00:11:32.080 --> 00:11:33.920 to specify for the PAM modulator 00:11:33.920 --> 00:11:36.891 is the symbol interval. 00:11:36.891 --> 00:11:40.780 Right, the time between sending consecutive signals 00:11:40.780 --> 00:11:42.490 over the channel. 00:11:42.490 --> 00:11:47.260 What you get out is X of t, this is the 00:11:47.260 --> 00:11:48.950 channel model, N of t. 00:11:48.950 --> 00:11:51.460 There's already noise over the channel, and what you 00:11:51.460 --> 00:11:54.880 get out is Y of t. 00:11:54.880 --> 00:11:57.470 So this is the generic architecture. 00:11:57.470 --> 00:12:05.230 And now, your goal for the design problem is to select 00:12:05.230 --> 00:12:12.630 alpha and t in the right way, so that they satisfy this 00:12:12.630 --> 00:12:15.930 continuous time constraints, and at the same time, you have 00:12:15.930 --> 00:12:18.050 your probability of error of ten to the minus five. 00:12:23.970 --> 00:12:26.520 So what would be an obvious choice for T? 00:12:30.312 --> 00:12:31.562 AUDIENCE: [INAUDIBLE] 00:12:34.950 --> 00:12:35.580 PROFESSOR: Right. 00:12:35.580 --> 00:12:39.790 So the first idea is you are given a certain amount of 00:12:39.790 --> 00:12:43.890 bandwidth, and you clearly want send your signals as fast 00:12:43.890 --> 00:12:47.550 as possible in order to get excellent data rate. 00:12:47.550 --> 00:12:50.100 Now because you have a certain amount of bandwidth, what 00:12:50.100 --> 00:12:52.510 Nyquist's criteria tells you is that you want 00:12:52.510 --> 00:12:54.730 to have zero ISI. 00:12:54.730 --> 00:12:59.450 And if you want to have zero ISI, what you do know is that 00:12:59.450 --> 00:13:02.710 the symbol interval should be greater than or 00:13:02.710 --> 00:13:04.670 equal to 1 over 2W. 00:13:04.670 --> 00:13:08.640 You cannot signal at a rate faster than one over T, and so 00:13:08.640 --> 00:13:14.590 if we look at this, it's 1 over 2 times 10 to the 6. 00:13:14.590 --> 00:13:18.280 Now it I do use this particular value of T, then 00:13:18.280 --> 00:13:21.710 what's my alpha going to be? 00:13:21.710 --> 00:13:25.320 Well, alpha is simply the energy per symbol. 00:13:25.320 --> 00:13:30.370 So I know alpha squared is the power that I have times the 00:13:30.370 --> 00:13:34.030 symbol interval, T. It's just a definition. 00:13:34.030 --> 00:13:36.490 This comes from orthonormality of the PAM 00:13:36.490 --> 00:13:38.100 system that we have. 00:13:38.100 --> 00:13:43.590 Now P is one, because that's what I specified as a system 00:13:43.590 --> 00:13:48.225 specification, so this is just T, which is one over times ten 00:13:48.225 --> 00:13:49.970 to the six. 00:13:49.970 --> 00:13:54.022 So I can select this value of alpha and this value of T. 00:13:54.022 --> 00:13:55.272 AUDIENCE: [INAUDIBLE] 00:13:57.830 --> 00:14:02.110 PROFESSOR: Well, alpha squared is energy per symbol. 00:14:02.110 --> 00:14:04.300 So what will that be? 00:14:04.300 --> 00:14:06.910 What's the energy per symbol, if you're sending every T 00:14:06.910 --> 00:14:09.740 seconds, and if you have a power of P? 00:14:09.740 --> 00:14:12.720 Es, that I mentioned last time, or energy per two 00:14:12.720 --> 00:14:14.230 dimensions. 00:14:14.230 --> 00:14:16.950 So in PAM, it will be energy per symbols. 00:14:16.950 --> 00:14:18.200 In that case, it will be 2P. 00:14:23.120 --> 00:14:23.540 OK. 00:14:23.540 --> 00:14:26.400 But now if I select these values of alpha and T, will my 00:14:26.400 --> 00:14:27.950 system work? 00:14:27.950 --> 00:14:30.900 Is this a reasonable design, or is there 00:14:30.900 --> 00:14:32.150 something wrong here? 00:14:37.690 --> 00:14:39.790 I'm clearly satisfying my -- 00:14:39.790 --> 00:14:41.690 AUDIENCE: [INAUDIBLE] 00:14:41.690 --> 00:14:44.190 PROFESSOR: The probability of error, right, exactly. 00:14:44.190 --> 00:14:47.870 So in fact, I do know how to calculate it, right? 00:14:47.870 --> 00:14:50.460 What's the probability of bit error? 00:14:50.460 --> 00:14:56.560 Well, we saw that last time it was Q of root 2 Eb over N_0. 00:14:59.850 --> 00:15:02.480 Eb is same as alpha squared, because we 00:15:02.480 --> 00:15:04.440 have one bit per symbol. 00:15:04.440 --> 00:15:08.420 So alpha squared is this quantity here, 1 over 2 times 00:15:08.420 --> 00:15:10.020 10 to the 6. 00:15:10.020 --> 00:15:13.580 So this is Q of square root of -- 00:15:13.580 --> 00:15:19.810 so 2 alpha squared is 10 to the 6, 1 over 10 to the 6. 00:15:19.810 --> 00:15:23.410 N_0, I know, is ten to the minus six, so this is 00:15:23.410 --> 00:15:25.960 actually Q of 1. 00:15:25.960 --> 00:15:34.430 And if I do calculate that, it's like 17 percent, which is 00:15:34.430 --> 00:15:36.170 nowhere close to ten to the minus five. 00:15:44.030 --> 00:15:46.140 So any suggestions on how I can improve my system? 00:15:48.680 --> 00:15:49.580 AUDIENCE: Increase T [INAUDIBLE] 00:15:49.580 --> 00:15:51.570 PROFESSOR: Increase T, right? 00:15:51.570 --> 00:15:53.290 What's happening right now is -- 00:15:53.290 --> 00:15:56.960 the reason we selected this value of T in the first place 00:15:56.960 --> 00:15:59.940 is because we wanted to send our signals as fast as 00:15:59.940 --> 00:16:03.590 possible avoid ISI, but that's just one of criteria in my 00:16:03.590 --> 00:16:04.860 system, right? 00:16:04.860 --> 00:16:08.810 I have to also satisfy this probability of error criteria, 00:16:08.810 --> 00:16:11.650 so I want to make sure my probability of error is going 00:16:11.650 --> 00:16:12.930 to be small. 00:16:12.930 --> 00:16:15.960 If I look at the expression for probability of error, it 00:16:15.960 --> 00:16:19.170 doesn't really look at T. All it looks at is this ratio of 00:16:19.170 --> 00:16:20.360 Eb/N0, right? 00:16:20.360 --> 00:16:23.735 So if I want to reduce my probability of error, I have 00:16:23.735 --> 00:16:25.880 to increase my energy per bit. 00:16:25.880 --> 00:16:29.460 Now my energy per bit is P times T, so the only hope of 00:16:29.460 --> 00:16:32.780 increasing my energy per bit will be to increase T, which 00:16:32.780 --> 00:16:36.798 means I have to signal at a slower rate. 00:16:36.798 --> 00:16:38.190 OK? 00:16:38.190 --> 00:16:39.750 So we have probability of -- 00:16:39.750 --> 00:16:41.440 let's write the calculation down. 00:16:48.200 --> 00:16:50.120 It's ten to the minus five. 00:16:50.120 --> 00:16:53.150 Last time, we saw that the best way to solve this is to 00:16:53.150 --> 00:16:59.510 look at the waterfall curve, and EbN_0 in this case is 00:16:59.510 --> 00:17:03.410 approximately 9.6 dB. 00:17:03.410 --> 00:17:05.550 I will say that that's approximately ten on the 00:17:05.550 --> 00:17:06.800 linear scale. 00:17:12.440 --> 00:17:17.690 So this implies that energy per bit is ten 00:17:17.690 --> 00:17:19.890 to the minus five. 00:17:19.890 --> 00:17:24.660 So energy per bit is P times T, in this case, it's ten to 00:17:24.660 --> 00:17:29.890 the minus five. p is one, so this implies that t is ten to 00:17:29.890 --> 00:17:31.420 the minus five. 00:17:31.420 --> 00:17:35.320 So I can send one bit every ten to the minus five seconds. 00:17:35.320 --> 00:17:38.100 So my rate that I achieve -- 00:17:38.100 --> 00:17:39.350 just write it here -- 00:17:43.130 --> 00:17:49.187 which is ten to the five bits per second. 00:17:49.187 --> 00:17:50.437 OK? 00:17:54.520 --> 00:17:59.050 If you compare this to the Shannon limit, the Shannon 00:17:59.050 --> 00:18:00.970 limit is right here, you have ten to the 00:18:00.970 --> 00:18:02.770 six bits per second. 00:18:02.770 --> 00:18:05.980 So you lose by a factor of ten in your data rate if you're 00:18:05.980 --> 00:18:08.730 going to use an uncoded 2-PAM system. 00:18:08.730 --> 00:18:12.610 So what this example tells you is that if you're going to do 00:18:12.610 --> 00:18:15.830 more sophisticated cording, you can gain up to a factor of 00:18:15.830 --> 00:18:18.920 ten in your data rate. 00:18:18.920 --> 00:18:22.340 So if the 10 dB did not really impress you last time, 00:18:22.340 --> 00:18:25.110 hopefully this example throws more light on 00:18:25.110 --> 00:18:26.360 the value of coding. 00:18:28.650 --> 00:18:30.015 Are there any questions on this example? 00:18:35.675 --> 00:18:37.615 AUDIENCE: Since the -- 00:18:37.615 --> 00:18:41.510 since we are signaling at a faster rate now, instead of 00:18:41.510 --> 00:18:43.610 using sink process, we can use something better. 00:18:43.610 --> 00:18:45.220 PROFESSOR: That's a very good point, yes. 00:18:45.220 --> 00:18:51.480 Well, if you look at the nominal bandwidth here, it's 1 00:18:51.480 --> 00:18:53.240 over 2T, right? 00:18:53.240 --> 00:18:58.920 T is 10 to the minus 5 seconds, so this says 1 over 2 00:18:58.920 --> 00:19:01.830 times 10 to the minus 5. 00:19:01.830 --> 00:19:06.065 So it's going to be 5 times 10 to the 4, or 50 KHz. 00:19:08.800 --> 00:19:09.270 OK? 00:19:09.270 --> 00:19:13.060 The available bandwidth you have, the system bandwidth, if 00:19:13.060 --> 00:19:15.340 you will, is 1 Megahertz. 00:19:15.340 --> 00:19:20.440 But if you're going to do Nyquist's ideal sinks pulses, 00:19:20.440 --> 00:19:23.745 then you only need 50 KHz of bandwidth in 00:19:23.745 --> 00:19:25.240 your system, right? 00:19:25.240 --> 00:19:29.900 So one advantage of this system, if you will, is that 00:19:29.900 --> 00:19:32.610 you're not required to do the complicated sink pulses. 00:19:32.610 --> 00:19:34.750 Do not need to send those pulses. 00:19:34.750 --> 00:19:38.480 You could simply send, for example, square pulses and, 00:19:38.480 --> 00:19:40.950 because your bandwidth is such low, you have a very low 00:19:40.950 --> 00:19:43.060 complexity system. 00:19:43.060 --> 00:19:47.720 Of course, the price you pay is you reduce the data rate by 00:19:47.720 --> 00:19:48.970 a factor of ten. 00:19:51.368 --> 00:19:52.920 OK, it's a good point. 00:19:52.920 --> 00:19:54.990 In fact, there are many points that will come up in this 00:19:54.990 --> 00:19:59.080 example if you think about it later on, so feel free to ask 00:19:59.080 --> 00:20:03.050 me questions if you think about some issues later on. 00:20:07.590 --> 00:20:08.850 Ok. 00:20:08.850 --> 00:20:12.930 So I think we have motivated the need for coding enough 00:20:12.930 --> 00:20:15.435 now, so let's look at our encoder design. 00:20:27.760 --> 00:20:38.360 So a typical encoder design takes bits in -- 00:20:38.360 --> 00:20:41.110 we saw this last time in the context of 00:20:41.110 --> 00:20:42.440 spectral efficiency -- 00:20:42.440 --> 00:20:43.885 and produces symbols out. 00:20:47.840 --> 00:20:52.880 So I can represent my bits by, say a vector b, and I can 00:20:52.880 --> 00:20:56.760 represent my symbols by a vector x. 00:20:56.760 --> 00:20:59.880 So every sequence of b bits gets mapped to a 00:20:59.880 --> 00:21:02.290 sequence of N symbols. 00:21:02.290 --> 00:21:05.860 Now this output sequence of symbols is not any arbitrary 00:21:05.860 --> 00:21:11.160 sequence, but it lies in a set of all possible sequences, 00:21:11.160 --> 00:21:16.860 which I denote by C. And this set is essentially a set of 00:21:16.860 --> 00:21:21.230 permissible output symbol sequences, which I will write 00:21:21.230 --> 00:21:27.590 by C sub j, which is a vector in Rn, because there are N 00:21:27.590 --> 00:21:29.870 symbols being produced. 00:21:29.870 --> 00:21:34.550 And we can have set up to j, j goes from 1 to M. So we can 00:21:34.550 --> 00:21:38.390 have up to M symbols. 00:21:38.390 --> 00:21:43.990 And note that here, M has to be equal to 2 to the b, in 00:21:43.990 --> 00:21:46.220 order to be able to map every sequence 00:21:46.220 --> 00:21:48.900 of b bits to M symbols. 00:21:48.900 --> 00:22:01.500 So this C is known as a codebook, and each C sub j is 00:22:01.500 --> 00:22:02.750 called a codeword. 00:22:07.280 --> 00:22:07.437 OK? 00:22:07.437 --> 00:22:10.870 The standard definition of an encoder. 00:22:10.870 --> 00:22:15.480 Now in today's lecture and half of next week's lecture, 00:22:15.480 --> 00:22:19.160 we will be seeing at a very specific case when N 00:22:19.160 --> 00:22:22.130 equals 1 and 2. 00:22:22.130 --> 00:22:26.010 In that case, instead of using the letter C, we will be using 00:22:26.010 --> 00:22:30.970 a different letter, A. So C, in that case, we'll call it 00:22:30.970 --> 00:22:32.220 actually a constellation. 00:22:34.650 --> 00:22:38.080 So in particular if C is one, it's a PAM constellation. 00:22:38.080 --> 00:22:40.500 If N is 2, it's a QAM constellation. 00:22:40.500 --> 00:22:44.180 And we'll be denoting it by a letter A instead of C. 00:22:44.180 --> 00:22:50.090 So A is again, a sequence of symbols a_j, which belongs to 00:22:50.090 --> 00:22:58.410 Rn, where one is less than j is less than M. OK, in this 00:22:58.410 --> 00:22:59.660 case a is a constellation. 00:23:08.040 --> 00:23:13.810 a_j's are known as symbols, or sometimes they're also known 00:23:13.810 --> 00:23:15.576 as signal points in the constellation. 00:23:22.840 --> 00:23:24.610 There are a number of definitions that 00:23:24.610 --> 00:23:26.140 follow from this -- 00:23:26.140 --> 00:23:28.170 number of properties of the constellation, 00:23:28.170 --> 00:23:29.620 rather, that follow. 00:23:29.620 --> 00:23:34.260 So in particular N is known as the dimension of your 00:23:34.260 --> 00:23:36.900 constellation. 00:23:36.900 --> 00:23:40.160 The number N is this, and here, it's the number of 00:23:40.160 --> 00:23:43.870 symbol sequences you output for a sequence of b 00:23:43.870 --> 00:23:47.090 bits that are in. 00:23:47.090 --> 00:23:49.135 M is the size of your constellation. 00:23:59.130 --> 00:24:06.430 The energy per constellation is given by 1 over M times the 00:24:06.430 --> 00:24:14.210 summation the norm of a_j squared, where j goes from one 00:24:14.210 --> 00:24:24.460 to M. The minimum distance of your constellation is simply 00:24:24.460 --> 00:24:27.710 the Euclidean minimum distance between two points in the 00:24:27.710 --> 00:24:30.380 constellation. 00:24:30.380 --> 00:24:33.290 So if you take the norm of a_i minus a_j, and minimize it 00:24:33.290 --> 00:24:35.270 over all possible values i and j. 00:24:38.330 --> 00:24:42.960 The number of nearest neighbors, of the average 00:24:42.960 --> 00:24:44.460 number of nearest -- 00:24:44.460 --> 00:24:59.360 K_min of A is the average number of nearest neighbors in 00:24:59.360 --> 00:25:04.480 A. 00:25:04.480 --> 00:25:06.220 In addition to this, there are some orthonormalized 00:25:06.220 --> 00:25:08.700 parameters that you saw last time. 00:25:17.460 --> 00:25:22.017 The spectral efficiency, which is in units of bits per two 00:25:22.017 --> 00:25:29.630 dimensions is 2b over N. And if you want to eliminate b, we 00:25:29.630 --> 00:25:33.700 use the relation that b is log M to the base 2 here. 00:25:33.700 --> 00:25:41.540 And so we have 2 log M to the base 2 over N. 00:25:41.540 --> 00:25:47.170 The energy per two dimensions, denoted by Es, is simply 2 00:25:47.170 --> 00:25:49.920 over N E(A). 00:25:49.920 --> 00:25:52.920 So E(A) is the average energy of your constellation. 00:25:52.920 --> 00:25:55.900 If you divide it by the number of dimensions you have, you 00:25:55.900 --> 00:26:00.330 get energy per dimension, and you multiply it by 2. 00:26:00.330 --> 00:26:07.080 And finally, the energy per bit is Es over rho, or it can 00:26:07.080 --> 00:26:12.060 also be expressed as E(A), which is the energy per symbol 00:26:12.060 --> 00:26:15.430 over the number of bits per symbol, which is log 00:26:15.430 --> 00:26:16.943 M to the base 2. 00:26:26.120 --> 00:26:29.120 It might seem like a lot of definitions, but you will see 00:26:29.120 --> 00:26:33.110 very soon that they have a very tight interplay among one 00:26:33.110 --> 00:26:38.030 another, so it's not nearly as overwhelming as it might seem 00:26:38.030 --> 00:26:38.980 at the first point. 00:26:38.980 --> 00:26:40.450 AUDIENCE: [INAUDIBLE] 00:26:40.450 --> 00:26:41.430 PROFESSOR: Yes. 00:26:41.430 --> 00:26:42.680 AUDIENCE: Why is [UNINTELLIGIBLE]? 00:26:46.780 --> 00:26:49.010 PROFESSOR: Because you have two [UNINTELLIGIBLE] b bits 00:26:49.010 --> 00:26:54.522 coming in, right, which you map to each -- 00:26:54.522 --> 00:26:57.486 AUDIENCE: [INAUDIBLE] 00:26:57.486 --> 00:26:59.860 There are two [INAUDIBLE] possible sequences, but all of 00:26:59.860 --> 00:27:02.670 them need not be used, right? 00:27:02.670 --> 00:27:06.340 PROFESSOR: Well, we assume that there is no coding going 00:27:06.340 --> 00:27:09.050 on before the encoder. 00:27:09.050 --> 00:27:11.690 So you have the source code, for which there's a sequence 00:27:11.690 --> 00:27:16.021 of IID bits, and then they mapped to 00:27:16.021 --> 00:27:19.180 a sequence of symbols. 00:27:19.180 --> 00:27:22.700 So we'll see all of our possible input bits coming in 00:27:22.700 --> 00:27:25.190 here, because it's produced by a source code, 00:27:25.190 --> 00:27:27.620 like a Huffman code. 00:27:27.620 --> 00:27:27.745 Right? 00:27:27.745 --> 00:27:30.920 And the idea here is, perhaps what you're asking is -- 00:27:30.920 --> 00:27:33.250 this did not span the entire space of Rn. 00:27:35.780 --> 00:27:38.400 We want to select these sequences carefully here. 00:27:46.590 --> 00:27:49.230 Maybe we'll come back to that later on in the course. 00:27:53.200 --> 00:27:54.470 OK, so let's do an example. 00:28:03.290 --> 00:28:16.040 So the example is, say we have A, which is a 2-PAM system, 00:28:16.040 --> 00:28:21.340 and you want to look at this constellation, B, which is 00:28:21.340 --> 00:28:26.720 denoted by A raised to K. The definition of A raised to k is 00:28:26.720 --> 00:28:36.800 it's a sequence of K symbols where each x_i belongs to A. 00:28:36.800 --> 00:28:54.049 So this is also known as the K-fold Cartesian product of A. 00:28:54.049 --> 00:28:55.023 AUDIENCE: Another question. 00:28:55.023 --> 00:28:58.380 So it has been pre-decided that b bits will be encoded 00:28:58.380 --> 00:28:58.810 [UNINTELLIGIBLE]? 00:28:58.810 --> 00:29:00.240 PROFESSOR: Right. 00:29:00.240 --> 00:29:02.450 So this is a specific structure we are imposing on 00:29:02.450 --> 00:29:03.700 the encoder. 00:29:07.830 --> 00:29:10.590 So this is the constellation, and you'll want to study the 00:29:10.590 --> 00:29:12.740 properties for this constellation. 00:29:12.740 --> 00:29:18.230 For this constellation, what's N going to be? 00:29:18.230 --> 00:29:20.648 What's the dimension going to be? 00:29:20.648 --> 00:29:22.130 AUDIENCE: [INAUDIBLE] 00:29:22.130 --> 00:29:28.700 PROFESSOR: For B, not A. It's going to be K. Well, the 00:29:28.700 --> 00:29:32.050 number of points in this constellation, how 00:29:32.050 --> 00:29:33.910 many points are there? 00:29:33.910 --> 00:29:35.500 There are K coordinates. 00:29:35.500 --> 00:29:38.520 Each coordinate can be plus or minus alpha. 00:29:38.520 --> 00:29:44.670 So we have 2 to the K possible points in this constellation. 00:29:44.670 --> 00:29:45.920 OK? 00:29:47.560 --> 00:29:49.840 What's E of A going to be? 00:30:00.554 --> 00:30:02.990 AUDIENCE: [INAUDIBLE] 00:30:02.990 --> 00:30:05.890 PROFESSOR: K alpha squared. right? 00:30:05.890 --> 00:30:07.570 Basically, we have K coordinates. 00:30:07.570 --> 00:30:11.170 The energy for each coordinate will simply add up. 00:30:11.170 --> 00:30:14.060 The energy across each coordinate is always going to 00:30:14.060 --> 00:30:15.570 be alpha squared. 00:30:15.570 --> 00:30:18.480 So each point in this constellation has an energy of 00:30:18.480 --> 00:30:19.960 K alpha squared. 00:30:19.960 --> 00:30:22.290 So regardless of how many points we have, the average 00:30:22.290 --> 00:30:24.930 energy is always going to be K alpha squared. 00:30:28.260 --> 00:30:29.510 Does everybody see this? 00:30:29.510 --> 00:30:30.418 AUDIENCE: [INAUDIBLE] 00:30:30.418 --> 00:30:32.690 PROFESSOR: You're right. 00:30:32.690 --> 00:30:33.940 Maybe that was the confusion. 00:30:36.260 --> 00:30:38.090 It's a good thing. 00:30:38.090 --> 00:30:42.560 Just getting too used to writing E of A. OK. 00:30:42.560 --> 00:30:44.390 What's d_min of b going to be? 00:30:55.324 --> 00:30:56.318 AUDIENCE: [INAUDIBLE] 00:30:56.318 --> 00:30:57.312 2 alpha. 00:30:57.312 --> 00:30:58.562 PROFESSOR: 2 alpha. 00:31:01.090 --> 00:31:03.140 I think everybody had the right idea. 00:31:03.140 --> 00:31:07.990 So the minimum distance here is two alpha for A. If we look 00:31:07.990 --> 00:31:12.780 at this point B, we can fix K minus 1 coordinates for two 00:31:12.780 --> 00:31:15.890 points to be the same, they will only differ in one point. 00:31:15.890 --> 00:31:18.930 And so the minimum distance is across that 00:31:18.930 --> 00:31:21.430 point, which is 2 alpha. 00:31:21.430 --> 00:31:23.690 What is K_min of B going to be? 00:31:29.910 --> 00:31:34.920 It's going to be K. For each point -- let's say the point 00:31:34.920 --> 00:31:39.890 which has all alphas, we can fix K minus 1 coordinate and 00:31:39.890 --> 00:31:42.710 find another point which is different in only one of the 00:31:42.710 --> 00:31:44.580 coordinates, say the first coordinate. 00:31:44.580 --> 00:31:47.580 We can do it for all K different coordinates, so 00:31:47.580 --> 00:31:53.830 K_min is going to be K for each point, and hence the 00:31:53.830 --> 00:31:56.830 average number of nearest neighbors is also K. 00:31:56.830 --> 00:32:00.870 OK, so now in this case, let's first start with the 00:32:00.870 --> 00:32:02.360 normalized parameters. 00:32:02.360 --> 00:32:05.940 That's always good to start with spectral efficiency. 00:32:05.940 --> 00:32:14.710 That's 2 log M over N. Well, log of M is going to be K, so 00:32:14.710 --> 00:32:18.440 N is going to be K. So this is going to be two bits per two 00:32:18.440 --> 00:32:24.300 dimensions, and this is the same as that of the original 00:32:24.300 --> 00:32:30.370 constellation, A. Your energy per two dimensions is going to 00:32:30.370 --> 00:32:35.200 be 2 over N E(B). 00:32:35.200 --> 00:32:39.080 E(B) is K alpha squared, N equals K, so this is 2 alpha 00:32:39.080 --> 00:32:41.980 squared, and that is the same as the original constellation, 00:32:41.980 --> 00:32:45.120 A. 00:32:45.120 --> 00:32:46.910 Finally. 00:32:46.910 --> 00:32:51.350 energy per bit is Es over rho, so it's 2 alpha 00:32:51.350 --> 00:32:53.190 squared over 2. 00:32:53.190 --> 00:32:55.880 So that's alpha squared, and that's same as the 2-PAM 00:32:55.880 --> 00:32:58.000 constellation. 00:32:58.000 --> 00:33:00.470 So why did I go through all of these calculations? 00:33:03.120 --> 00:33:06.560 What we see is that the normalized parameters, rho, 00:33:06.560 --> 00:33:10.920 Es, and Eb, are the same for the Cartesian product as for 00:33:10.920 --> 00:33:12.900 the original constellation. 00:33:12.900 --> 00:33:16.080 And at some level, that should not be too surprising, right? 00:33:16.080 --> 00:33:19.340 Because what I'll be doing in this Cartesian product, we are 00:33:19.340 --> 00:33:21.510 not really doing any coding, right? 00:33:21.510 --> 00:33:24.270 In this original constellation, we had one bit 00:33:24.270 --> 00:33:27.080 coming in, and we are mapping it to one symbol. 00:33:27.080 --> 00:33:29.640 All we are doing in the Cartesian product is we are 00:33:29.640 --> 00:33:32.520 taking K bits in and mapping them to K symbols. 00:33:32.520 --> 00:33:34.880 So we still have one bit per symbol. 00:33:34.880 --> 00:33:39.770 The noise is IID, so it's optimal to two decisions for 00:33:39.770 --> 00:33:42.280 each of the coordinates independently, and decide 00:33:42.280 --> 00:33:44.460 whether that coordinate corresponds to plus alpha or 00:33:44.460 --> 00:33:45.730 minus alpha. 00:33:45.730 --> 00:33:47.750 So in other words, there's nothing gained by doing this 00:33:47.750 --> 00:33:49.650 Cartesian product. 00:33:49.650 --> 00:33:51.280 And we will see, the probability of error 00:33:51.280 --> 00:33:54.750 expression depends on these normalized parameters, if we 00:33:54.750 --> 00:33:58.140 want to look at Pb of E, and so we do not gain anything in 00:33:58.140 --> 00:34:01.980 terms of the probability of error, versus EbN_0, trade-off 00:34:01.980 --> 00:34:03.730 through Cartesian product. 00:34:03.730 --> 00:34:07.390 So I'm making the note here because that's the 00:34:07.390 --> 00:34:09.520 only space I have. 00:34:09.520 --> 00:34:14.020 So the note is if I look at probability of bit error 00:34:14.020 --> 00:34:21.060 versus EbN_0, the curve we saw last time, it is the same for 00:34:21.060 --> 00:34:28.560 B and A. You should be able to convince yourself about this, 00:34:28.560 --> 00:34:30.630 and so there is really no coding going on here. 00:34:35.040 --> 00:34:36.460 Are there any questions on this? 00:34:39.969 --> 00:34:43.510 Let's look at this problem a bit more carefully now. 00:35:05.875 --> 00:35:06.869 AUDIENCE: [INAUDIBLE] 00:35:06.869 --> 00:35:07.863 PROFESSOR: Yeah. 00:35:07.863 --> 00:35:09.113 AUDIENCE: Why [INAUDIBLE] 00:35:12.750 --> 00:35:14.020 PROFESSOR: Right. 00:35:14.020 --> 00:35:15.770 AUDIENCE: What does he use? 00:35:15.770 --> 00:35:19.350 PROFESSOR: Energy per two dimensions. 00:35:19.350 --> 00:35:21.960 Es will always be energy per two dimensions. 00:35:21.960 --> 00:35:23.800 Throughout the course, we'll be using these notations. 00:35:23.800 --> 00:35:26.040 Eb is the energy per bit, Es is the energy per two 00:35:26.040 --> 00:35:27.560 dimensions. 00:35:27.560 --> 00:35:30.600 And if you want to say energy per symbol, we'll be using 00:35:30.600 --> 00:35:34.068 this notation E sub the constellation. 00:35:34.068 --> 00:35:34.512 AUDIENCE: Oh. 00:35:34.512 --> 00:35:36.290 It's not energy per bit. 00:35:36.290 --> 00:35:37.220 PROFESSOR: No, this is energy -- 00:35:37.220 --> 00:35:39.060 B is my constellation. 00:35:39.060 --> 00:35:42.210 So that's why it's energy of that constellation, average 00:35:42.210 --> 00:35:43.990 energy per symbol in that constellation. 00:36:14.320 --> 00:36:19.930 OK, so let us consider the special case when K equals 3. 00:36:19.930 --> 00:36:24.040 So in that case, B is A^q. 00:36:24.040 --> 00:36:27.420 So if I look at all possible points in B, they are going to 00:36:27.420 --> 00:36:30.950 lie on the vertices of a three-dimensional cube. 00:36:30.950 --> 00:36:33.365 That's a Cartesian product in three dimensions. 00:36:43.760 --> 00:36:46.580 And all my constellations points are basically on the 00:36:46.580 --> 00:36:47.830 vertices of this cube. 00:36:54.070 --> 00:36:57.680 The distance here is going to be 2 alpha. 00:36:57.680 --> 00:37:01.960 That's the length of each edge in my cube, and that's what B 00:37:01.960 --> 00:37:03.025 is going to be. 00:37:03.025 --> 00:37:06.930 Clearly, the minimum distances is 2 alpha, as we saw before. 00:37:06.930 --> 00:37:12.260 Now let me define a different constellation, B prime, and 00:37:12.260 --> 00:37:15.120 only going to take four vertices from these possible 00:37:15.120 --> 00:37:16.660 eight vertices. 00:37:16.660 --> 00:37:21.560 I'm going to take this vertex here, I'm going to take this 00:37:21.560 --> 00:37:27.120 vertex here, this one, and this one. 00:37:27.120 --> 00:37:29.140 I'm only taking four vertices. 00:37:29.140 --> 00:37:32.540 If I want to tell you explicitly what the points 00:37:32.540 --> 00:37:36.310 are, I need to draw an axis, so I'm simply drawing the x, 00:37:36.310 --> 00:37:38.680 y, and z axis here. 00:37:38.680 --> 00:37:42.740 This is x-axis, this is y-axis, and z-axis. 00:37:42.740 --> 00:37:46.180 And B prime is a subset of the points in this 00:37:46.180 --> 00:37:49.460 three-dimensional Cartesian product. 00:37:49.460 --> 00:37:55.280 They will be alpha, alpha, alpha; minus alpha, minus 00:37:55.280 --> 00:38:03.950 alpha, alpha; alpha, minus alpha, minus alpha; and let's 00:38:03.950 --> 00:38:09.050 see, minus alpha, alpha, minus alpha. 00:38:09.050 --> 00:38:12.300 So two of the coordinates will be minus alpha here, in these 00:38:12.300 --> 00:38:16.260 three points, and we have one coordinate all alphas. 00:38:16.260 --> 00:38:18.000 So this is my B prime. 00:38:18.000 --> 00:38:20.490 What is the minimum distance going to be for B prime? 00:38:26.226 --> 00:38:28.150 AUDIENCE: [INAUDIBLE] 00:38:28.150 --> 00:38:29.540 PROFESSOR: 2 over 2 alpha, right? 00:38:29.540 --> 00:38:32.200 It's basically the length of this edge here. 00:38:32.200 --> 00:38:34.740 This is 2 alpha, this is 2 alpha. 00:38:34.740 --> 00:38:35.990 So it's 2 over 2 alpha. 00:38:45.300 --> 00:38:48.760 So in other words, by simply selecting a subset of points, 00:38:48.760 --> 00:38:51.660 I have been able to increase my minimum distance. 00:38:51.660 --> 00:38:54.220 Because my minimum distance is larger, I hope that the 00:38:54.220 --> 00:38:56.810 probability of error will be smaller as opposed to the 00:38:56.810 --> 00:38:58.420 original constellation. 00:38:58.420 --> 00:39:00.330 But this comes at the price, right? 00:39:00.330 --> 00:39:01.580 And what's the price? 00:39:05.400 --> 00:39:07.540 The spectral efficiency is smaller, right? 00:39:07.540 --> 00:39:10.450 What if I look at my spectral efficiency? 00:39:10.450 --> 00:39:13.070 Well, I'm only sending out two points, two 00:39:13.070 --> 00:39:15.090 bits per each point. 00:39:15.090 --> 00:39:18.040 So two bits, each point takes three dimensions, so my 00:39:18.040 --> 00:39:21.450 spectral efficiency is 2 times 2 over 3 bits per two 00:39:21.450 --> 00:39:26.480 dimensions, or it's 4 over 3 bits per two dimensions. 00:39:26.480 --> 00:39:29.110 And this is in contrast to the two bits per two dimensions we 00:39:29.110 --> 00:39:30.610 had for B. 00:39:30.610 --> 00:39:33.690 So in other words, there is a trade-off between your 00:39:33.690 --> 00:39:37.180 spectral efficiency and the minimum distance. 00:39:37.180 --> 00:39:40.200 We'll start with a K-dimensional Cartesian 00:39:40.200 --> 00:39:42.720 product of A, which has all points. 00:39:42.720 --> 00:39:46.690 We took a subset of points, and if we chose them smartly, 00:39:46.690 --> 00:39:49.490 we were able to increase the minimum distance, but the 00:39:49.490 --> 00:39:52.490 price we had to pay was to reduce the spectral 00:39:52.490 --> 00:39:53.740 efficiency. 00:39:57.610 --> 00:40:00.040 AUDIENCE: Where did this two / three come from? 00:40:00.040 --> 00:40:00.840 PROFESSOR: This two here? 00:40:00.840 --> 00:40:02.090 AUDIENCE: 2/3, yes. 00:40:02.090 --> 00:40:05.920 PROFESSOR: 2/3, I am sending two bits per symbol, right? 00:40:05.920 --> 00:40:07.275 Each symbol has three dimensions. 00:40:10.370 --> 00:40:13.780 So it's 2/3 bit per dimension, or 4/3 bit per two dimension. 00:40:16.490 --> 00:40:25.590 OK, so the point was it seems like there is a trade-off 00:40:25.590 --> 00:40:33.236 between minimum distance and spectral efficiency. 00:40:37.840 --> 00:40:40.300 And indeed, this might seem like a reasonable trade-off, 00:40:40.300 --> 00:40:42.710 and a lot of coding here that we will be seeing in the early 00:40:42.710 --> 00:40:46.850 part of the course is indeed motivated by this trade-off. 00:40:46.850 --> 00:40:49.250 You want to reduce your spectral efficiency in order 00:40:49.250 --> 00:40:50.520 to increase your probability of error. 00:40:53.250 --> 00:40:55.240 And this has in fact been quite a dominant design 00:40:55.240 --> 00:40:58.570 principle for a large number of codes that have come up in 00:40:58.570 --> 00:40:59.870 coding theory. 00:40:59.870 --> 00:41:04.910 However, if you look at what Shannon says, Shannon says 00:41:04.910 --> 00:41:07.320 something quite different. 00:41:07.320 --> 00:41:10.530 In Shannon's theorem, all they say is, you have -- 00:41:10.530 --> 00:41:16.390 if your spectral efficiency is below a certain amount, then 00:41:16.390 --> 00:41:24.180 your probability of bit error can be made arbitrarily small. 00:41:24.180 --> 00:41:27.590 OK, so what Shannon is saying, it's something much stronger 00:41:27.590 --> 00:41:28.490 than this trade-off. 00:41:28.490 --> 00:41:31.190 It's saying if you reduce your spectral efficiency below a 00:41:31.190 --> 00:41:34.520 certain quantity which is finite, then the probability 00:41:34.520 --> 00:41:37.030 of error can be made arbitrarily small. 00:41:37.030 --> 00:41:40.150 There is no statement of minimum distance in this 00:41:40.150 --> 00:41:41.710 theorem here. 00:41:41.710 --> 00:41:44.280 And indeed, if you look at the most modern codes which are 00:41:44.280 --> 00:41:48.640 capacity approaching, they are not designed to maximize the 00:41:48.640 --> 00:41:49.900 minimum distance. 00:41:49.900 --> 00:41:52.770 They are designed to work well with some practical decoding 00:41:52.770 --> 00:41:55.120 algorithms, like the belief propagation of 00:41:55.120 --> 00:41:56.640 algorithms and so on. 00:41:56.640 --> 00:41:58.890 So they are designed on a somewhat different principle 00:41:58.890 --> 00:42:00.620 than minimum distance. 00:42:00.620 --> 00:42:04.140 But nevertheless, this is quite a powerful tool that we 00:42:04.140 --> 00:42:06.910 will be using in the early part of this course. 00:42:06.910 --> 00:42:09.800 We start with a K-dimensional Cartesian product, select a 00:42:09.800 --> 00:42:12.580 subset of points, and we want to increase the minimum 00:42:12.580 --> 00:42:14.395 distance at the cost of spectral efficiency. 00:42:18.460 --> 00:42:21.160 OK. 00:42:21.160 --> 00:42:23.247 Now are there any questions? 00:42:23.247 --> 00:42:24.497 AUDIENCE: [INAUDIBLE] 00:42:29.620 --> 00:42:31.390 PROFESSOR: That's a good question. 00:42:31.390 --> 00:42:34.430 Suppose I have a 2-PAM constellation, then I can 00:42:34.430 --> 00:42:38.060 easily write the probability of bit error as a function of 00:42:38.060 --> 00:42:39.190 Q function. 00:42:39.190 --> 00:42:42.310 If it is a more complicated expression, I have to 00:42:42.310 --> 00:42:45.990 integrate over the decision regions, which we'll be seeing 00:42:45.990 --> 00:42:47.850 later on in this lecture. 00:42:47.850 --> 00:42:50.150 And it's not usually possible to get an exact probability of 00:42:50.150 --> 00:42:51.200 error expression. 00:42:51.200 --> 00:42:54.480 We usually use an in-union bound, to bound it by a pair 00:42:54.480 --> 00:42:55.580 of [UNINTELLIGIBLE] error probability. 00:42:55.580 --> 00:42:57.300 We'll be doing all that just now. 00:43:05.430 --> 00:43:06.070 OK. 00:43:06.070 --> 00:43:08.090 So now let us -- 00:43:08.090 --> 00:43:09.740 I have talked now enough now about encoder, and we'll be 00:43:09.740 --> 00:43:12.840 visiting it very soon, but let us switch gears and talk about 00:43:12.840 --> 00:43:15.380 the decoder now. 00:43:15.380 --> 00:43:17.820 OK, what does a decoder do? 00:43:17.820 --> 00:43:19.850 So the goal of a decoder is the following. 00:43:23.360 --> 00:43:29.270 You get your received vector Y, which is X plus N, and from 00:43:29.270 --> 00:43:36.400 Y, you want to estimate X-hat as a point in your signal 00:43:36.400 --> 00:43:37.620 constellation. 00:43:37.620 --> 00:43:40.530 So you receive a noisy version of X, and you want to estimate 00:43:40.530 --> 00:43:45.560 X-hat at the decoder. 00:43:45.560 --> 00:43:48.000 So this is the architecture of your decoder. 00:43:48.000 --> 00:43:55.830 And the goal here is you want to minimize the 00:43:55.830 --> 00:43:58.890 probability of error. 00:43:58.890 --> 00:44:01.490 And what's the probability of error? 00:44:01.490 --> 00:44:07.420 It's basically probability that X is not equal to X-hat. 00:44:07.420 --> 00:44:11.780 So that is your general criteria at the decoder. 00:44:11.780 --> 00:44:14.290 Now what we'll doing next is basically going through this 00:44:14.290 --> 00:44:18.210 exercise to show that this minimum probability of error 00:44:18.210 --> 00:44:22.590 criteria is equivalent to a bunch of other criteria. 00:44:22.590 --> 00:44:26.340 So the first criteria is the MAP criteria: Maximum 00:44:26.340 --> 00:44:27.590 A-Posteriori Rule. 00:45:28.930 --> 00:45:35.110 So our probability of error is basically -- 00:45:35.110 --> 00:45:39.890 I can track it as an integral of probability of error given 00:45:39.890 --> 00:45:49.030 Y times the density function of Y. So if I want to minimize 00:45:49.030 --> 00:45:51.970 my probability of error, I want to minimize each term in 00:45:51.970 --> 00:45:53.190 this integral. 00:45:53.190 --> 00:46:00.070 So this implies I want to minimize probability of error 00:46:00.070 --> 00:46:02.860 given Y for each possible value of Y. OK? 00:46:05.370 --> 00:46:06.910 Now what's that going to be? 00:46:06.910 --> 00:46:10.240 Well, in order to look at what this term is, 00:46:10.240 --> 00:46:12.340 suppose I make a decision. 00:46:12.340 --> 00:46:15.520 I receive Y, and I decide a symbol a_j is sent. 00:46:36.610 --> 00:46:38.880 Then what's the probability of error going to be? 00:46:41.480 --> 00:46:50.070 My probability of error given Y is going to be 1 minus the 00:46:50.070 --> 00:46:51.890 probability that I was correct. 00:46:51.890 --> 00:46:55.570 Probability that I was correct is probability X equals a_j, 00:46:55.570 --> 00:47:01.580 given Y. This follows from the definition. 00:47:01.580 --> 00:47:04.720 So if I want to minimize my probability of error given Y, 00:47:04.720 --> 00:47:07.300 I want to actually choose an a_j that maximizes the 00:47:07.300 --> 00:47:14.890 probability of a_j given Y. So this implies, choose a_j. 00:47:25.860 --> 00:47:28.706 And this is known as the MAP rule. 00:47:32.110 --> 00:47:35.800 So the idea behind the MAP rule is to choose the symbol 00:47:35.800 --> 00:47:38.690 in the constellation that maximizes the posterior 00:47:38.690 --> 00:47:42.720 probability, given the received symbol. 00:47:42.720 --> 00:47:45.410 Now, this MAP rule is equivalent to the maximum 00:47:45.410 --> 00:47:49.180 likelihood rule, under the assumption that all the signal 00:47:49.180 --> 00:47:51.460 points a_j are equally likely. 00:47:51.460 --> 00:47:53.510 The proof is not hard, you just use 00:47:53.510 --> 00:47:55.520 Bayes Theorem for that. 00:47:55.520 --> 00:48:05.990 So suppose all a_j's are equally likely. 00:48:08.940 --> 00:48:15.600 Then probability of a_j given Y, which by Bayes Theorem is 00:48:15.600 --> 00:48:20.400 the density of Y given a_j, times the probability of a_j. 00:48:20.400 --> 00:48:22.940 But since all a_j's are equally likely, I will just 00:48:22.940 --> 00:48:31.850 write it as 1 over M, over the density of Y. Now because Y is 00:48:31.850 --> 00:48:35.330 fixed, the density of Y is fixed, so this quantity is 00:48:35.330 --> 00:48:37.840 just proportional to -- 00:48:37.840 --> 00:48:40.260 the proportionality symbol -- 00:48:40.260 --> 00:48:42.110 to the density of Y given a_j. 00:48:46.430 --> 00:48:48.560 I won't be writing all the vectors, I 00:48:48.560 --> 00:48:49.320 might be missing some. 00:48:49.320 --> 00:48:50.990 But please bear with me. 00:48:50.990 --> 00:49:04.590 So this implies we want to choose a_j that maximizes the 00:49:04.590 --> 00:49:10.450 density of Y given a_j, and this is known as the maximum 00:49:10.450 --> 00:49:12.800 likelihood rule. 00:49:12.800 --> 00:49:14.820 And there is one final rule. 00:49:14.820 --> 00:49:18.580 Basically if the noise is additive Gaussian, then the 00:49:18.580 --> 00:49:24.510 density of Y given a_j is simply proportional to E 00:49:24.510 --> 00:49:29.450 raised to minus the norm of Y minus a_j squared. 00:49:29.450 --> 00:49:33.400 So if we want to maximize this quantity, we want to minimize 00:49:33.400 --> 00:49:37.170 Y minus a_j squared. 00:49:37.170 --> 00:49:48.170 So we want to choose a_j that minimizes the Euclidean 00:49:48.170 --> 00:49:51.330 distance between Y minus a_j. 00:49:51.330 --> 00:49:54.770 And this is known as the minimum distance decision 00:49:54.770 --> 00:49:56.310 rule, MDD rule. 00:49:56.310 --> 00:49:56.970 Yes? 00:49:56.970 --> 00:49:59.430 AUDIENCE: We are ignoring [UNINTELLIGIBLE]? 00:49:59.430 --> 00:50:00.840 PROFESSOR: We are ignoring P value. 00:50:00.840 --> 00:50:03.660 Because for a given Y, Py is going to be fixed for all 00:50:03.660 --> 00:50:05.790 possible choices of a_j. 00:50:05.790 --> 00:50:08.440 The goal is I'm given Y, and I want to decide which signal 00:50:08.440 --> 00:50:12.730 point was set, because that's the probability of error given 00:50:12.730 --> 00:50:16.360 Y. This is my criteria now. 00:50:16.360 --> 00:50:19.746 So Y is fixed, so the density of Y is fixed. 00:50:19.746 --> 00:50:22.040 AUDIENCE: [INAUDIBLE] 00:50:22.040 --> 00:50:24.340 PROFESSOR: It's basically given by -- 00:50:24.340 --> 00:50:26.720 in order to find this density, we'll just condition it on 00:50:26.720 --> 00:50:31.416 a_j, and sum up over all possible values of a_j. 00:50:31.416 --> 00:50:33.610 Just take the marginal of Y, right? 00:50:33.610 --> 00:50:36.040 I mean, to write this explicitly. 00:50:36.040 --> 00:50:37.850 I'm writing it here. 00:50:37.850 --> 00:50:42.750 It's going to be sigma P of Y given a_j that's the 00:50:42.750 --> 00:50:44.400 probability of a_j. 00:50:44.400 --> 00:50:47.306 And this you can find by the Gaussian. 00:50:47.306 --> 00:50:51.500 AUDIENCE: But then you have [INAUDIBLE]. 00:50:51.500 --> 00:50:53.470 PROFESSOR: But this is a summation over 00:50:53.470 --> 00:50:54.416 all possible a_j's. 00:50:54.416 --> 00:50:57.500 I should write, sorry -- a_k's. 00:50:57.500 --> 00:50:59.630 This is just a summation over all k's, right? 00:50:59.630 --> 00:51:02.240 So basically, Py is going to be a mixture of several 00:51:02.240 --> 00:51:04.180 Gaussians, OK? 00:51:04.180 --> 00:51:05.160 And it's fixed. 00:51:05.160 --> 00:51:07.120 AUDIENCE: [INAUDIBLE] 00:51:07.120 --> 00:51:08.370 PROFESSOR: Right. 00:51:10.510 --> 00:51:11.360 OK. 00:51:11.360 --> 00:51:14.700 So we want to choose the Minimum Distance Decision 00:51:14.700 --> 00:51:23.170 rule, and I should have the variance of noise here. 00:51:23.170 --> 00:51:26.310 OK, so what we have so far is we started with the Minimum 00:51:26.310 --> 00:51:28.600 Probability of Error rule, and that's the 00:51:28.600 --> 00:51:30.335 criteria of your decoder. 00:51:30.335 --> 00:51:35.510 Be sure this is equivalent to MAP rule, and that basically 00:51:35.510 --> 00:51:38.950 comes just from the definition. 00:51:38.950 --> 00:51:42.560 This integral here, we want to minimize each term in the 00:51:42.560 --> 00:51:45.210 integral, and that basically implies that the Maximum 00:51:45.210 --> 00:51:47.280 A-Posteriori rule is the best. 00:51:47.280 --> 00:51:52.280 This implied Maximum Likelihood rule, and Maximum 00:51:52.280 --> 00:51:56.030 Likelihood rule comes from the fact that all the signal 00:51:56.030 --> 00:51:58.250 points are equally likely. 00:51:58.250 --> 00:52:02.510 And this implies, then, the Minimum Distance Decision 00:52:02.510 --> 00:52:05.200 rule, and that comes from the fact that your noise is 00:52:05.200 --> 00:52:06.280 Additive Gaussian. 00:52:06.280 --> 00:52:09.270 So you have an exponential in the -- 00:52:09.270 --> 00:52:12.290 you have the Euclidean distance as an exponent, and 00:52:12.290 --> 00:52:14.850 you want to minimize the Euclidean distance. 00:52:14.850 --> 00:52:17.470 So this is the story we have so far. 00:52:17.470 --> 00:52:20.745 And it turns out that the Minimum Distance Decision rule 00:52:20.745 --> 00:52:23.130 is actually quite nice, because it gives you a lot of 00:52:23.130 --> 00:52:25.950 geometrical insights. 00:52:25.950 --> 00:52:28.160 So say I have three points. 00:52:28.160 --> 00:52:30.410 We not even draw the coordinates. 00:52:30.410 --> 00:52:37.180 My constellation, A, has three points, and let me write them 00:52:37.180 --> 00:52:38.430 as a1, a2, a3. 00:52:40.620 --> 00:52:43.070 This is my constellation, for example. 00:52:43.070 --> 00:52:48.040 And say I receive a symbol Y. Then the job of the decoder is 00:52:48.040 --> 00:52:51.310 to figure out whether I sent a1, a2 or a3. 00:52:51.310 --> 00:52:52.820 How will the decoder do that? 00:52:52.820 --> 00:52:55.680 Well, it will measure the distance from all the three 00:52:55.680 --> 00:52:58.230 constellation points and select the one with the 00:52:58.230 --> 00:53:00.660 smallest Euclidean distance. 00:53:00.660 --> 00:53:03.130 More generally, what we want to do is we want to look at 00:53:03.130 --> 00:53:06.930 the space of all received Y symbols, and partition it into 00:53:06.930 --> 00:53:08.240 decision regions. 00:53:08.240 --> 00:53:11.190 So that if the point falls in a certain decision region, we 00:53:11.190 --> 00:53:13.490 say that the constellation point corresponding to that 00:53:13.490 --> 00:53:15.760 decision region was sent. 00:53:15.760 --> 00:53:18.115 And how do I find the decision region? 00:53:18.115 --> 00:53:20.680 Well, I start drawing hyperplanes between every two 00:53:20.680 --> 00:53:22.660 pair of constellation points. 00:53:22.660 --> 00:53:25.500 Say I want to find the decision region of point a1. 00:53:25.500 --> 00:53:28.060 I draw a hyperplane between a1 and a3, which is 00:53:28.060 --> 00:53:29.720 given by this line. 00:53:29.720 --> 00:53:32.580 I draw a hyperplane between a1 and a2 which is 00:53:32.580 --> 00:53:34.130 given by this point. 00:53:34.130 --> 00:53:37.860 And so the region -- the set of points which are closer to 00:53:37.860 --> 00:53:43.740 a1 than a2 and a3 is basically bounded by this region here. 00:53:43.740 --> 00:53:45.450 So this is my R1. 00:53:45.450 --> 00:53:48.230 Similarly, if I want to find a decision region for a2 and a3, 00:53:48.230 --> 00:53:50.640 I will draw this line here. 00:53:50.640 --> 00:53:53.550 This will be R2 and this will be R3. 00:53:53.550 --> 00:53:55.110 So these are my decision regions. 00:53:59.210 --> 00:54:03.160 So if I want to write that formally, Rj 00:54:03.160 --> 00:54:04.710 is my decision region. 00:54:04.710 --> 00:54:10.110 And it is the set of points Y belong to Rn, such that the 00:54:10.110 --> 00:54:15.540 norm of Y minus a_j squared is going to be less than or equal 00:54:15.540 --> 00:54:17.800 to -- it doesn't matter if you have less than or equal to, 00:54:17.800 --> 00:54:20.550 because the point [UNINTELLIGIBLE] bound 00:54:20.550 --> 00:54:21.800 [UNINTELLIGIBLE] probability zero -- 00:54:24.022 --> 00:54:26.070 is radius squared. 00:54:26.070 --> 00:54:28.950 That's just the definition of Rj. 00:54:28.950 --> 00:54:32.360 Now the way to construct Rj was to look at all the half 00:54:32.360 --> 00:54:35.480 planes which are closer to this point than any other 00:54:35.480 --> 00:54:37.020 point, and take the intersection of 00:54:37.020 --> 00:54:38.780 all the half planes. 00:54:38.780 --> 00:54:41.970 So in other words, I can also write Rj to be the 00:54:41.970 --> 00:54:44.230 intersection of these half planes -- 00:54:44.230 --> 00:54:47.620 the intersections over all points, j prime not equal to j 00:54:47.620 --> 00:54:49.240 -- of Rj, j prime. 00:54:49.240 --> 00:54:52.508 So Rj, j prime is your half plane where -- 00:55:06.460 --> 00:55:10.100 so norm of Y minus a_j prime squared is greater than or 00:55:10.100 --> 00:55:14.650 equal to norm of Y minus a_j squared. 00:55:14.650 --> 00:55:18.960 Note that this is a_j prime, and this is a_j here. 00:55:18.960 --> 00:55:21.180 OK. 00:55:21.180 --> 00:55:23.750 So it turns out that this decision region has a somewhat 00:55:23.750 --> 00:55:26.200 nice structure, because they're intersection of a 00:55:26.200 --> 00:55:29.245 bunch of half planes, their shape is the convex polytope. 00:55:43.290 --> 00:55:46.495 And they're also known by the name Voronoi regions. 00:55:53.680 --> 00:55:57.820 OK, so these regions are known as Voronoi regions here. 00:55:57.820 --> 00:56:02.160 Now the set of points whose hyperplanes are active in a 00:56:02.160 --> 00:56:05.550 certain decision region has a special name, too, and it's 00:56:05.550 --> 00:56:07.550 called the relevant subset. 00:56:07.550 --> 00:56:10.760 So in this case, the relevant subset of a1 is a2 and a3, 00:56:10.760 --> 00:56:13.970 because both of them have hyperplanes that are active in 00:56:13.970 --> 00:56:16.590 the decision region of a1. 00:56:16.590 --> 00:56:18.300 So let me write that down. 00:56:53.160 --> 00:56:56.470 So the relevant subset is the set of points, a_j prime, 00:56:56.470 --> 00:57:01.410 whose hyperplanes are active in this decision region Rj. 00:57:01.410 --> 00:57:04.450 There's a theorem which says that the nearest neighbors are 00:57:04.450 --> 00:57:06.740 always included in the relevant subset. 00:57:06.740 --> 00:57:07.990 It's asserted in your notes. 00:57:10.560 --> 00:57:11.810 OK? 00:57:21.490 --> 00:57:24.040 So now that we have this Minimum Distance Decision 00:57:24.040 --> 00:57:27.010 rule, let us see if we can get a hang with the 00:57:27.010 --> 00:57:28.260 probability of error. 00:58:23.550 --> 00:58:27.340 Let me see the probability of error, given that I sent a 00:58:27.340 --> 00:58:29.340 symbol, a_j. 00:58:29.340 --> 00:58:32.090 I want the value that probability of error. 00:58:32.090 --> 00:58:38.970 That's simply the probability that Y does not belong to Rj, 00:58:38.970 --> 00:58:42.380 given that I sent the symbol a_j. 00:58:42.380 --> 00:58:45.100 That's when an error happens. 00:58:45.100 --> 00:58:48.720 That's same as probability that the noise vector -- 00:58:48.720 --> 00:58:51.360 because Y is a_j plus N now -- 00:58:51.360 --> 00:58:56.130 does not belong to the Rj minus a_j, and the noise is 00:58:56.130 --> 00:59:00.570 independent of a_j so I remove the conditioning. 00:59:00.570 --> 00:59:04.490 And that is 1 minus the probability that the noise 00:59:04.490 --> 00:59:08.180 does belong to Rj minus a_j. 00:59:08.180 --> 00:59:11.300 If I want to find this integral, find this 00:59:11.300 --> 00:59:18.320 expression, I will integrate over the region Rj minus a_j, 00:59:18.320 --> 00:59:24.760 of the density of the noise, dN. 00:59:24.760 --> 00:59:27.690 No note that the noise has a spherical symmetry, but 00:59:27.690 --> 00:59:30.650 unfortunately despite that, the integral is not a 00:59:30.650 --> 00:59:33.740 straightforward integral, because this region here has 00:59:33.740 --> 00:59:35.055 sharp edges. 00:59:35.055 --> 00:59:37.750 The decision region is a convex polytope, so it's 00:59:37.750 --> 00:59:40.540 typically something like this, and if this was your point, 00:59:40.540 --> 00:59:44.450 a_j, your noise does have a spherical symmetry about these 00:59:44.450 --> 00:59:48.060 spheres, but when it intersects the decision 00:59:48.060 --> 00:59:50.010 boundary, things get ugly. 00:59:50.010 --> 00:59:52.800 And so this decision -- this is not a 00:59:52.800 --> 00:59:55.740 nice integral in general. 00:59:59.420 --> 01:00:02.510 And so unfortunately, there is not much progress we can make 01:00:02.510 --> 01:00:05.690 beyond this point for the exact probability of error 01:00:05.690 --> 01:00:09.430 expression, but we can say some nice geometrical 01:00:09.430 --> 01:00:12.400 properties about the probability of error. 01:00:12.400 --> 01:00:18.400 The first property is that probability of error is 01:00:18.400 --> 01:00:23.170 invariant to translations. 01:00:28.890 --> 01:00:30.900 And this should be quite obvious. 01:00:30.900 --> 01:00:34.750 You have, say, a constellation with two points here, and say 01:00:34.750 --> 01:00:36.480 I subtract off the mean. 01:00:36.480 --> 01:00:39.520 So I get a different constellation whose 01:00:39.520 --> 01:00:41.240 points are like this. 01:00:41.240 --> 01:00:43.040 This is my constellation, A, and this is my 01:00:43.040 --> 01:00:44.840 constellation, A prime. 01:00:44.840 --> 01:00:47.000 The probability of error will be same for the two 01:00:47.000 --> 01:00:50.610 constellations because the decision regions will have the 01:00:50.610 --> 01:00:53.940 same distance from both the points. 01:00:53.940 --> 01:00:55.760 This should be quite obvious. 01:00:55.760 --> 01:00:59.120 And basically, what this really says is if I have any 01:00:59.120 --> 01:01:02.150 constellation, I can always subtract off the mean, and get 01:01:02.150 --> 01:01:04.740 another constellation with the same probability of error, but 01:01:04.740 --> 01:01:06.940 with smaller average energy. 01:01:06.940 --> 01:01:21.400 And so this implies that any optimal constellation will 01:01:21.400 --> 01:01:23.455 have zero mean. 01:01:29.710 --> 01:01:34.850 The second point is, the probability of error is 01:01:34.850 --> 01:01:44.570 invariant to orthonormal rotations. 01:01:49.100 --> 01:01:53.540 So if I had, say, one point, one constellation, with these 01:01:53.540 --> 01:01:58.530 four points, and I rotate it by 45 degrees, what I get is 01:01:58.530 --> 01:02:01.180 another constellation with these four points. 01:02:01.180 --> 01:02:04.035 And both these constellations are simply rotations of one 01:02:04.035 --> 01:02:07.060 another, and they have the same probability of error. 01:02:07.060 --> 01:02:09.110 And the easiest way to see that is the decision regions 01:02:09.110 --> 01:02:12.290 here are simply the four quadrants. 01:02:12.290 --> 01:02:14.770 And if I want to integrate my probability of error, I will 01:02:14.770 --> 01:02:17.380 be integrating it over this region. 01:02:17.380 --> 01:02:21.490 Here, my decision regions will be these 45 degree lines. 01:02:21.490 --> 01:02:24.730 And if I want to integrate the probability of error for this 01:02:24.730 --> 01:02:30.250 point here, it will be given by noise, which is symmetric 01:02:30.250 --> 01:02:31.880 about these circles. 01:02:31.880 --> 01:02:34.590 Basically, since the noise is invariant to orthonormal 01:02:34.590 --> 01:02:37.360 rotations, it should be quite obvious that the probability 01:02:37.360 --> 01:02:39.264 of error is invariant to rotations. 01:02:39.264 --> 01:02:41.120 AUDIENCE: [INAUDIBLE] 01:02:41.120 --> 01:02:42.260 PROFESSOR: Any rotation, right. 01:02:42.260 --> 01:02:45.230 AUDIENCE: So what do you mean by [UNINTELLIGIBLE]? 01:02:45.230 --> 01:02:47.882 PROFESSOR: Basically, you're preserving the distance. 01:02:47.882 --> 01:02:50.440 So you're just rotating the point, not scaling it. 01:02:50.440 --> 01:02:51.380 AUDIENCE: [INAUDIBLE] 01:02:51.380 --> 01:02:53.680 PROFESSOR: Unitarily. 01:02:53.680 --> 01:02:54.360 OK? 01:02:54.360 --> 01:02:57.260 I mean you will be proving these properties in the next 01:02:57.260 --> 01:02:59.434 homework, which is just handed out. 01:02:59.434 --> 01:03:01.210 AUDIENCE: So those [UNINTELLIGIBLE] 01:03:01.210 --> 01:03:06.228 hold, because Minimum Distance rule is orthonormal, right? 01:03:06.228 --> 01:03:09.005 And that's because you assume Gaussian -- 01:03:09.005 --> 01:03:10.982 PROFESSOR: Because you assume Gaussian noise. 01:03:10.982 --> 01:03:12.430 AUDIENCE: So that's the only assumption we make? 01:03:12.430 --> 01:03:13.260 PROFESSOR: Right. 01:03:13.260 --> 01:03:15.525 AUDIENCE: -- for those [INAUDIBLE], right? 01:03:15.525 --> 01:03:16.775 PROFESSOR: I think so. 01:03:37.470 --> 01:03:39.430 AUDIENCE: Why [INAUDIBLE] constellation 01:03:39.430 --> 01:03:41.910 must have zero mean? 01:03:41.910 --> 01:03:45.082 PROFESSOR: Because if I have any constellation, there's a 01:03:45.082 --> 01:03:46.810 certain probability of error, right? 01:03:46.810 --> 01:03:50.240 Can always subtract out the mean from the constellation, I 01:03:50.240 --> 01:03:53.470 get a new constellation with a smaller average energy with 01:03:53.470 --> 01:03:55.808 the same probability of error. 01:03:55.808 --> 01:03:58.253 AUDIENCE: Oh, so in terms of [INAUDIBLE] 01:03:58.253 --> 01:03:59.231 PROFESSOR: Right. 01:03:59.231 --> 01:04:01.431 If you're looking at a trade-off of probability of 01:04:01.431 --> 01:04:04.121 error versus energy, usually what we look at. 01:04:07.070 --> 01:04:08.360 So maybe that's a good point. 01:04:08.360 --> 01:04:09.774 I should just mention it. 01:04:13.646 --> 01:04:16.066 For probability of error versus -- 01:04:18.970 --> 01:04:20.570 we're looking at this trade-off here. 01:04:57.970 --> 01:04:58.110 Ok. 01:04:58.110 --> 01:05:01.820 The next idea is to basically bound the probability of error 01:05:01.820 --> 01:05:05.330 by a union bound, because we cannot compute an exact 01:05:05.330 --> 01:05:07.760 expression for the probability of error, so we might as well 01:05:07.760 --> 01:05:10.540 compute a bound which is tractable. 01:05:10.540 --> 01:05:16.410 So we'll look at what is known as the pairwise error 01:05:16.410 --> 01:05:17.660 probability. 01:05:25.450 --> 01:05:30.370 So the idea behind pairwise error probability is suppose I 01:05:30.370 --> 01:05:35.380 send a point, a_j, what is the probability that instead of 01:05:35.380 --> 01:05:40.910 a_j at the receiver, I decide that a_j prime was sent. 01:05:40.910 --> 01:05:44.230 This is the pairwise error probability. 01:05:44.230 --> 01:05:50.020 So geometrically, say a_j and a_j prime are two points here. 01:05:50.020 --> 01:05:52.620 Let me draw some coordinate axis here. 01:05:52.620 --> 01:05:56.910 And say I sent point a_j, and there is noise on the channel, 01:05:56.910 --> 01:05:58.100 that takes me to this point. 01:05:58.100 --> 01:06:02.670 So this is my Y, and this is the noise vector. 01:06:02.670 --> 01:06:04.310 OK? 01:06:04.310 --> 01:06:09.050 And now what I want to know is under what conditions will I 01:06:09.050 --> 01:06:11.450 decide a_j prime over a_j. 01:06:11.450 --> 01:06:15.440 What is the probability of deciding a_j prime over a_j? 01:06:15.440 --> 01:06:18.840 So let's draw a line joining a_j prime and a_j. 01:06:18.840 --> 01:06:21.370 So how would I decide -- 01:06:21.370 --> 01:06:24.360 suppose I receive this point, Y, and I wanted to decide 01:06:24.360 --> 01:06:26.550 between a_j and a_j prime. 01:06:26.550 --> 01:06:29.718 What would be my decision rule? 01:06:29.718 --> 01:06:31.610 AUDIENCE: [INAUDIBLE] 01:06:31.610 --> 01:06:31.865 PROFESSOR: Uh-huh. 01:06:31.865 --> 01:06:33.115 AUDIENCE: [INAUDIBLE] 01:06:37.980 --> 01:06:38.235 PROFESSOR: You select -- 01:06:38.235 --> 01:06:39.440 AUDIENCE: [INAUDIBLE] a_j prime. 01:06:39.440 --> 01:06:40.370 PROFESSOR: Exactly. 01:06:40.370 --> 01:06:43.710 An equivalent way of saying it is to project Y onto this 01:06:43.710 --> 01:06:46.790 line, a_j prime minus a_j. 01:06:46.790 --> 01:06:49.390 We take two projections, one orthonormal to the line, one 01:06:49.390 --> 01:06:51.830 along on the line, and receive this projection. 01:06:56.100 --> 01:06:58.290 This is a straight line like this. 01:06:58.290 --> 01:06:59.890 Let's call it n tilde. 01:06:59.890 --> 01:07:03.098 I should change my chalk, it's getting too blunt now. 01:07:06.220 --> 01:07:10.270 This projection here is closer to a_j prime or a_j. 01:07:10.270 --> 01:07:14.670 So in other words, this probability of error is same 01:07:14.670 --> 01:07:18.980 as the probability that this n tilde, which is the projection 01:07:18.980 --> 01:07:26.380 of Y onto a_j prime minus a_j, is greater than or equal to 01:07:26.380 --> 01:07:31.010 the norm of a_j prime minus a_j over 2. 01:07:33.900 --> 01:07:37.130 OK, now why did I use the notation n tilde here? 01:07:37.130 --> 01:07:40.650 Because the projection Y onto a_j, which is this. 01:07:40.650 --> 01:07:43.920 n tilde is same as the projection of the noise onto a 01:07:43.920 --> 01:07:47.300 line joining a_j prime minus a_j. 01:07:47.300 --> 01:07:56.140 So n tilde, I can write it as projection of N onto a_j prime 01:07:56.140 --> 01:07:59.510 minus a_j over the norm. 01:08:03.920 --> 01:08:05.390 AUDIENCE: [INAUDIBLE] 01:08:05.390 --> 01:08:05.880 PROFESSOR: Sorry? 01:08:05.880 --> 01:08:07.092 AUDIENCE: Why, exactly? 01:08:07.092 --> 01:08:09.170 PROFESSOR: You can just see geometrically, right? 01:08:09.170 --> 01:08:12.110 This is a 90 degree here. 01:08:12.110 --> 01:08:13.590 This is the noise. 01:08:13.590 --> 01:08:16.660 If I project the noise, it will be this component here. 01:08:23.660 --> 01:08:26.160 AUDIENCE: [INAUDIBLE] 01:08:26.160 --> 01:08:26.430 PROFESSOR: All right. 01:08:26.430 --> 01:08:27.750 This should be 90, I'm sorry. 01:08:27.750 --> 01:08:28.670 This is 90. 01:08:28.670 --> 01:08:29.870 I'm messing things up. 01:08:29.870 --> 01:08:31.410 OK, this is the noise here. 01:08:31.410 --> 01:08:34.830 This noise, if I project it onto a_j prime minus a_j, it's 01:08:34.830 --> 01:08:36.460 going to be this component. 01:08:36.460 --> 01:08:40.800 This is Y, if I project it, it's the same component. 01:08:40.800 --> 01:08:45.740 Now, if the noise is IID with variance sigma squared in each 01:08:45.740 --> 01:08:49.200 coordinate, we are simply projecting the noise onto one 01:08:49.200 --> 01:08:51.670 orthonormal vector. 01:08:51.670 --> 01:08:57.979 So n tilde is also Gaussian, with zero mean 01:08:57.979 --> 01:09:00.220 variance sigma squared. 01:09:00.220 --> 01:09:05.330 So we can use that to find this probability of error. 01:09:11.590 --> 01:09:14.979 So in that case, the probability of error -- 01:09:18.241 --> 01:09:21.040 I should write this -- 01:09:21.040 --> 01:09:26.890 probability of a_j prime going to a_j is simply probability 01:09:26.890 --> 01:09:30.460 that this Gaussian is greater than some distance, and that's 01:09:30.460 --> 01:09:39.520 Q of norm of a_j prime minus a_j over two sigma. 01:09:39.520 --> 01:09:39.990 Yes? 01:09:39.990 --> 01:09:42.560 AUDIENCE: What is sigma? 01:09:42.560 --> 01:09:47.100 PROFESSOR: So the noise vector is IID, in each of the 01:09:47.100 --> 01:09:50.510 components, and has a variance of sigma squared. 01:09:50.510 --> 01:09:55.100 Sigma is basically N_0 over 2, if your noise is flat with -- 01:09:55.100 --> 01:09:58.120 so let me just write that down, sigma 01:09:58.120 --> 01:09:59.190 squared is N_0 over 2. 01:09:59.190 --> 01:10:01.700 If you have an AWGN channel, and you project it on each 01:10:01.700 --> 01:10:04.920 orthonormal signal, that's what you get. 01:10:04.920 --> 01:10:07.435 AUDIENCE: What if you project the noise vector on 01:10:07.435 --> 01:10:09.910 [INAUDIBLE] 01:10:09.910 --> 01:10:13.380 why is [INAUDIBLE] 01:10:13.380 --> 01:10:14.260 you don't have -- 01:10:14.260 --> 01:10:15.150 PROFESSOR: So you have a noise vector. 01:10:15.150 --> 01:10:18.230 If you have a Gaussian vector, and you project it onto an 01:10:18.230 --> 01:10:19.250 orthonormal basis -- 01:10:19.250 --> 01:10:19.600 AUDIENCE: Yes. 01:10:19.600 --> 01:10:22.010 But [INAUDIBLE] normal? 01:10:22.010 --> 01:10:22.180 PROFESSOR: Right. 01:10:22.180 --> 01:10:24.650 [INAUDIBLE] 01:10:24.650 --> 01:10:27.465 We are only projecting out on one vector which you need now. 01:10:30.754 --> 01:10:32.710 AUDIENCE: So then you're saying -- 01:10:32.710 --> 01:10:33.688 OK, yeah. 01:10:33.688 --> 01:10:37.111 The assumption is the noise is symmetric in all dimensions? 01:10:37.111 --> 01:10:38.361 PROFESSOR: Right. 01:10:42.500 --> 01:10:45.420 Let's do this algebraically, so you're convinced that there 01:10:45.420 --> 01:10:50.910 is no magic I'm doing here. 01:10:50.910 --> 01:10:54.860 So we can write this as you said, as the probability that 01:10:54.860 --> 01:11:00.370 the norm of Y minus a_j squared is greater than norm 01:11:00.370 --> 01:11:06.260 of Y minus a_j prime squared, given that Y [UNINTELLIGIBLE] 01:11:06.260 --> 01:11:10.280 a_j, so Y is a_j plus the noise vector. 01:11:10.280 --> 01:11:17.180 So I sub in for Y. What I get is probability that Y is a_j 01:11:17.180 --> 01:11:22.050 plus N. So here I have norm of N squared is greater than or 01:11:22.050 --> 01:11:27.790 equal to norm of a_j plus N minus a_j prime squared. 01:11:30.850 --> 01:11:34.110 And since the only random variable here is this noise, 01:11:34.110 --> 01:11:37.860 N, I can remove the conditioning [UNINTELLIGIBLE] 01:11:37.860 --> 01:11:39.250 down there. 01:11:39.250 --> 01:11:42.940 Let me expand this second norm term there. 01:11:42.940 --> 01:11:49.080 That's basically the norm of a_j minus a_j prime squared 01:11:49.080 --> 01:11:54.340 plus the norm of N squared minus two times the 01:11:54.340 --> 01:11:57.430 projection of N -- 01:11:57.430 --> 01:11:59.660 or rather, the inner product of N -- 01:11:59.660 --> 01:12:01.410 and a_j prime minus a_j. 01:12:04.440 --> 01:12:08.560 So this is the probability that the inner product of N 01:12:08.560 --> 01:12:16.980 and a_j prime minus a_j is greater than or equal to norm 01:12:16.980 --> 01:12:22.860 of a_j prime minus a_j squared over 2. 01:12:22.860 --> 01:12:26.730 If you divide by the norm of a_j prime minus a_j, you get 01:12:26.730 --> 01:12:30.886 the same expression as we had. 01:12:38.770 --> 01:12:39.690 So it's the same thing. 01:12:39.690 --> 01:12:42.910 This was done geometrically, this is done algebraically. 01:12:42.910 --> 01:12:45.690 So this is the expression of the probability of error, and 01:12:45.690 --> 01:12:51.006 this is Q of the norm of a_j prime minus a_j over 2 sigma. 01:13:53.860 --> 01:13:57.160 So now that we have the pairwise error probability, we 01:13:57.160 --> 01:13:59.200 can use it to bound the probability 01:13:59.200 --> 01:14:01.520 of error given a_j. 01:14:01.520 --> 01:14:05.090 Well by definition, the probably of error given a_j is 01:14:05.090 --> 01:14:09.240 simply the probability of the union of all the possible 01:14:09.240 --> 01:14:15.910 error events of the a_j goes to a_j prime over all possible 01:14:15.910 --> 01:14:19.120 j prime, not equal to j. 01:14:19.120 --> 01:14:22.630 This by the union bound is less than or equal to the 01:14:22.630 --> 01:14:28.570 summations of the probability that a_j goes to a_j prime. 01:14:28.570 --> 01:14:30.230 That's just using union bound. 01:14:30.230 --> 01:14:33.150 And now I can sub that expression over from there. 01:14:40.010 --> 01:14:41.800 This is the same as..So the summation is over j prime not 01:14:41.800 --> 01:14:48.710 equal to j times Q of the norm of a_j prime 01:14:48.710 --> 01:14:54.380 minus a_j over 2 sigma. 01:14:54.380 --> 01:14:57.170 Now let me write the summation in a different way. 01:14:57.170 --> 01:14:59.530 I'm going to write the summation over all possible 01:14:59.530 --> 01:15:05.820 distances which belong to the set of distance, times K_D of 01:15:05.820 --> 01:15:09.550 a_j, times Q of D over 2 sigma. 01:15:12.620 --> 01:15:23.590 Where the set D is the set of all possible 01:15:23.590 --> 01:15:30.160 distances from a_j. 01:15:30.160 --> 01:15:31.410 Ok? 01:15:33.170 --> 01:15:45.270 And K_D of a_j is the number of points at 01:15:45.270 --> 01:15:49.040 distance D from a_j. 01:15:51.780 --> 01:15:53.625 That's just a straightforward change of variables. 01:15:56.210 --> 01:15:59.930 Now if you look at this expression, then Q of B over 2 01:15:59.930 --> 01:16:03.070 sigma basically behaves like an exponential, for an algebra 01:16:03.070 --> 01:16:04.950 use of the argument. 01:16:04.950 --> 01:16:12.470 So recall that Q of X is like half E to the minus X squared 01:16:12.470 --> 01:16:16.770 over 2, for X much larger than 1. 01:16:16.770 --> 01:16:21.260 So what you are really seeing here is that you have a sum of 01:16:21.260 --> 01:16:24.830 a bunch of exponentials, each written by this 01:16:24.830 --> 01:16:26.560 term, K_D of a_j. 01:16:26.560 --> 01:16:30.370 Now if you think about the argument being large, then 01:16:30.370 --> 01:16:33.740 when you have a sum of exponentials, the term with 01:16:33.740 --> 01:16:37.070 the smallest exponent will dominate, because they are all 01:16:37.070 --> 01:16:39.440 decreasing exponentials. 01:16:39.440 --> 01:16:48.260 So this term can be written as approximately K_min of a_j 01:16:48.260 --> 01:16:53.310 times Q of d_min over 2 sigma. 01:16:56.470 --> 01:16:58.770 So what I am doing is I'm only picking up one 01:16:58.770 --> 01:17:02.630 term from this summation. 01:17:02.630 --> 01:17:05.825 So far, we have a strict upper bound here, so this summation 01:17:05.825 --> 01:17:08.520 is a strict upper bound on the probability of error given 01:17:08.520 --> 01:17:11.910 a_j, But now what I am doing is I'm only going to keep one 01:17:11.910 --> 01:17:15.770 term in the summation, the term which has the smallest 01:17:15.770 --> 01:17:17.260 exponent here. 01:17:17.260 --> 01:17:24.860 So I'm looking at the smallest value of D in this set of 01:17:24.860 --> 01:17:27.780 possible distances from a_j. 01:17:27.780 --> 01:17:28.073 AUDIENCE: So you're just looking 01:17:28.073 --> 01:17:29.100 at the nearest neighbor. 01:17:29.100 --> 01:17:30.490 PROFESSOR: You're looking at essentially the nearest 01:17:30.490 --> 01:17:32.510 neighbor, geometrically speaking. 01:17:32.510 --> 01:17:36.140 And this approximation actually works 01:17:36.140 --> 01:17:38.150 quite well in practice. 01:17:38.150 --> 01:17:40.530 It's not a bound on the probability of error given 01:17:40.530 --> 01:17:44.120 a_j, but it's an approximation. 01:17:44.120 --> 01:17:45.520 And why did I do this? 01:17:45.520 --> 01:17:49.812 Well, if I want to look at the probability over all error, 01:17:49.812 --> 01:17:52.340 what's that going to be? 01:17:52.340 --> 01:17:58.060 It's going to be the average over all possible a_j's of 01:17:58.060 --> 01:18:00.910 probability of error given a_j. 01:18:00.910 --> 01:18:04.880 Now, so I want to take an average of this quantity. 01:18:04.880 --> 01:18:06.270 So this is a constant. 01:18:06.270 --> 01:18:09.130 So I will just take the average over this, and that's 01:18:09.130 --> 01:18:15.720 going to be K_min of the constellation, which is the 01:18:15.720 --> 01:18:19.890 average number of nearest neighbors, times Q of D_min 01:18:19.890 --> 01:18:21.140 over 2 sigma. 01:18:24.680 --> 01:18:27.240 This is approximate here. 01:18:27.240 --> 01:18:31.270 So this is an approximation that will be used, and it's a 01:18:31.270 --> 01:18:33.930 very useful approximation, and it is known as 01:18:33.930 --> 01:18:35.180 the Union Bound Estimate. 01:18:49.140 --> 01:18:51.790 It's no longer a bound on the probability of 01:18:51.790 --> 01:18:53.910 error, it's an estimate. 01:18:53.910 --> 01:18:56.405 And in fact, there is a homework problem where you 01:18:56.405 --> 01:18:59.140 will be showing that the Union Bound Estimate is in fact 01:18:59.140 --> 01:19:00.800 exact for an M-PAM constellation. 01:19:03.500 --> 01:19:05.910 And I will let you think why that is the case. 01:19:05.910 --> 01:19:08.170 I was going to do it, but then I realized it's a homework 01:19:08.170 --> 01:19:12.610 problem, so you might as well spend some time on it. 01:19:12.610 --> 01:19:15.030 So the last thing that I wanted to do today is find a 01:19:15.030 --> 01:19:16.630 lower bound on the probability of error. 01:19:29.460 --> 01:19:32.260 So if I look at probability of error, it's a union of bunch 01:19:32.260 --> 01:19:33.210 of the events. 01:19:33.210 --> 01:19:34.190 AUDIENCE: [INAUDIBLE] 01:19:34.190 --> 01:19:34.860 PROFESSOR: Yes. 01:19:34.860 --> 01:19:36.530 AUDIENCE: [INAUDIBLE] 01:19:36.530 --> 01:19:41.400 That union should be with a_j prime. 01:19:41.400 --> 01:19:44.180 PROFESSOR: The union should be with a_j -- 01:19:44.180 --> 01:19:45.510 yeah. 01:19:45.510 --> 01:19:48.430 It's not what I have? 01:19:48.430 --> 01:19:52.550 So I'm taking a union over all possible events, but a_j's 01:19:52.550 --> 01:19:54.128 confused with a_j prime. 01:20:03.590 --> 01:20:04.290 AUDIENCE: [INAUDIBLE] 01:20:04.290 --> 01:20:08.638 a_j going to union j prime not equal to j, a_j prime. 01:20:13.430 --> 01:20:14.350 PROFESSOR: Oh, I see. 01:20:14.350 --> 01:20:16.750 AUDIENCE: I think you need parentheses around the -- 01:20:16.750 --> 01:20:18.392 AUDIENCE: Brackets around the -- 01:20:18.392 --> 01:20:20.210 AUDIENCE: [INAUDIBLE] 01:20:20.210 --> 01:20:23.232 another set of parentheses behind the event a_j 01:20:23.232 --> 01:20:24.500 going to a_j prime. 01:20:24.500 --> 01:20:27.184 Because that's the event you were talking about there. 01:20:27.184 --> 01:20:28.642 At least that's [INAUDIBLE] 01:20:34.000 --> 01:20:35.720 PROFESSOR: So you are saying that -- 01:20:35.720 --> 01:20:37.754 AUDIENCE: Put parentheses after the u. 01:20:37.754 --> 01:20:38.620 PROFESSOR: After the u. 01:20:38.620 --> 01:20:39.780 Like this? 01:20:39.780 --> 01:20:41.235 AUDIENCE: Yeah, right. 01:20:41.235 --> 01:20:43.175 That's the event. 01:20:43.175 --> 01:20:43.660 PROFESSOR: Right. 01:20:43.660 --> 01:20:45.600 That's what I meant. 01:20:45.600 --> 01:20:46.360 OK, fine. 01:20:46.360 --> 01:20:47.610 Fair enough. 01:20:53.120 --> 01:20:55.130 OK, so basically, the lower bound is 01:20:55.130 --> 01:20:56.280 actually quite simple. 01:20:56.280 --> 01:20:58.470 All I'm going to do is only take one 01:20:58.470 --> 01:21:00.180 event from that union. 01:21:00.180 --> 01:21:02.360 I'm only going to take one point, which is the minimum 01:21:02.360 --> 01:21:04.810 distance from a_j. 01:21:04.810 --> 01:21:11.580 So probability of error given a_j is greater than or equal 01:21:11.580 --> 01:21:16.890 to probability that a_j goes to a_j prime, where now a_j 01:21:16.890 --> 01:21:19.390 prime is the nearest neighbor of a_j. 01:21:19.390 --> 01:21:25.230 And this we know from PAM analysis is simply Q of d_min 01:21:25.230 --> 01:21:27.052 over 2 sigma. 01:21:27.052 --> 01:21:29.350 So this is a strict lower bound on the probability of 01:21:29.350 --> 01:21:31.960 error, and it has the same exponent as 01:21:31.960 --> 01:21:33.210 the Union Bound Estimate. 01:21:47.510 --> 01:21:49.865 Of course, if I want to find the overall probability of 01:21:49.865 --> 01:21:52.240 error, I can just take an average of this. 01:21:52.240 --> 01:21:55.165 Since this is fixed, it's going to be the same quantity. 01:21:57.670 --> 01:22:02.100 So far what we have is a strict upper bound on the 01:22:02.100 --> 01:22:06.260 probability of error, which is this quantity here, a union 01:22:06.260 --> 01:22:09.600 bound estimate, and we have a lower bound on the 01:22:09.600 --> 01:22:11.110 probability of error. 01:22:11.110 --> 01:22:14.100 In the next lecture, we will be looking at how to use these 01:22:14.100 --> 01:22:17.190 bounds to compute a probability of error for small 01:22:17.190 --> 01:22:20.530 signal constellations, and quantify the performance 01:22:20.530 --> 01:22:22.280 trade-off of the probability of error versus the 01:22:22.280 --> 01:22:24.820 EbN_0 and so on. 01:22:24.820 --> 01:22:26.450 I think this is a natural point to stop. 01:22:26.450 --> 01:22:27.700 It's almost time now.