1 00:00:00,000 --> 00:00:02,490 The following content is provided under a Creative 2 00:00:02,490 --> 00:00:04,059 Commons license. 3 00:00:04,059 --> 00:00:06,360 Your support will help MIT OpenCourseWare 4 00:00:06,360 --> 00:00:10,720 continue to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,350 To make a donation or view additional materials 6 00:00:13,350 --> 00:00:17,290 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,290 --> 00:00:18,294 at ocw.mit.edu. 8 00:00:27,440 --> 00:00:30,590 PROFESSOR: Today we're going to dig a little deeper 9 00:00:30,590 --> 00:00:32,820 into the system that we've been talking about. 10 00:00:32,820 --> 00:00:36,770 So we've already talked about source coding and source 11 00:00:36,770 --> 00:00:38,520 decoding. 12 00:00:38,520 --> 00:00:40,460 And then we talked about channel coding 13 00:00:40,460 --> 00:00:44,990 as we've just finished talking about block codes and Viterbi-- 14 00:00:44,990 --> 00:00:47,360 convolutional codes and Viterbi decoding. 15 00:00:47,360 --> 00:00:50,360 So that's the coding here and the decoding. 16 00:00:50,360 --> 00:00:53,030 And now we're going to drill down to the next level 17 00:00:53,030 --> 00:00:56,300 to start to talk about the actual signals going 18 00:00:56,300 --> 00:00:58,233 across physical channels. 19 00:00:58,233 --> 00:00:59,900 So this is going to actually extend over 20 00:00:59,900 --> 00:01:02,630 the entire next module of the course. 21 00:01:02,630 --> 00:01:05,209 I want to describe this in the context of something 22 00:01:05,209 --> 00:01:10,052 you're going to be doing in labs 4 through 6. 23 00:01:10,052 --> 00:01:11,510 You're actually going to experiment 24 00:01:11,510 --> 00:01:13,550 with a specific channel. 25 00:01:13,550 --> 00:01:18,860 What you'll have is bits coming in, code words coming in, 26 00:01:18,860 --> 00:01:22,470 being translated to signals. 27 00:01:22,470 --> 00:01:24,290 In this case, discrete time signals, 28 00:01:24,290 --> 00:01:27,110 and I'll give you an example shortly of that. 29 00:01:27,110 --> 00:01:30,380 The signals will then be adapted through the modulator 30 00:01:30,380 --> 00:01:34,670 for transmission on an analog channel. 31 00:01:34,670 --> 00:01:37,250 So there's a modulation process and there's 32 00:01:37,250 --> 00:01:40,430 a digital-to-analog conversion process. 33 00:01:40,430 --> 00:01:42,500 You'll be generating waveform that you apply 34 00:01:42,500 --> 00:01:44,570 to the speaker in your laptop. 35 00:01:44,570 --> 00:01:46,850 That's going to be a transmitter. 36 00:01:46,850 --> 00:01:52,070 The channel is going to be just the air around you 37 00:01:52,070 --> 00:01:55,250 with all the disturbances of room acoustics and noise 38 00:01:55,250 --> 00:01:57,223 and all of that, all the distortions from that. 39 00:01:57,223 --> 00:01:58,640 And then you'll pick up the signal 40 00:01:58,640 --> 00:02:01,610 on the microphone on your laptop or an external microphone 41 00:02:01,610 --> 00:02:03,080 if you want. 42 00:02:03,080 --> 00:02:07,520 Conversion from analog to digital, demodulation 43 00:02:07,520 --> 00:02:09,289 and filtering to undo the modulation, 44 00:02:09,289 --> 00:02:11,600 and we'll be talking about this in more detail 45 00:02:11,600 --> 00:02:14,690 to get another sequence of samples. 46 00:02:14,690 --> 00:02:16,190 After which you have a decision rule 47 00:02:16,190 --> 00:02:19,460 that then looks at the samples and says, did I get a 0 or a 1? 48 00:02:19,460 --> 00:02:22,470 And you spit out the bits of your code word. 49 00:02:22,470 --> 00:02:26,490 OK, so this is what we're going to be looking at. 50 00:02:26,490 --> 00:02:29,170 So here is what you might be sending 51 00:02:29,170 --> 00:02:30,830 at the transmitting end. 52 00:02:30,830 --> 00:02:32,750 You've got the bits coming in. 53 00:02:32,750 --> 00:02:35,240 You're going to convert them to signals, 54 00:02:35,240 --> 00:02:37,740 and we're going to think of discrete time signals. 55 00:02:37,740 --> 00:02:41,520 So this is a signal x of n-- n takes integer value, 56 00:02:41,520 --> 00:02:43,910 so that's my discrete time clock. 57 00:02:43,910 --> 00:02:46,620 And the typical waveform might look like this. 58 00:02:46,620 --> 00:02:53,360 I might decide just very simply to have levels held at 0.5 for, 59 00:02:53,360 --> 00:02:56,780 let's say, 16 samples per bit, and then held 60 00:02:56,780 --> 00:03:01,110 at 0 for 16 samples to denote a 1 and a 0 respectively. 61 00:03:01,110 --> 00:03:07,550 So here's a 1, a 00, 111, 0101. 62 00:03:07,550 --> 00:03:11,070 So we're converting two samples. 63 00:03:11,070 --> 00:03:13,280 This is a sample number, and then the next step 64 00:03:13,280 --> 00:03:15,020 will be to actually-- 65 00:03:15,020 --> 00:03:17,390 in your computer you'll send this to your digital 66 00:03:17,390 --> 00:03:19,100 to analog converter which will-- 67 00:03:19,100 --> 00:03:24,548 with a particular clock cycle, convert this to real time. 68 00:03:24,548 --> 00:03:26,840 What you might imagine is that the actual waveform that 69 00:03:26,840 --> 00:03:28,280 goes out on the channel is somehow 70 00:03:28,280 --> 00:03:30,710 related to the continuous waveform 71 00:03:30,710 --> 00:03:35,030 that you get by just connecting the tops of these discrete time 72 00:03:35,030 --> 00:03:37,400 values. 73 00:03:37,400 --> 00:03:40,220 The actual mechanism for transmission through the air 74 00:03:40,220 --> 00:03:41,487 we'll talk about next time. 75 00:03:41,487 --> 00:03:43,070 So right now we're just going to focus 76 00:03:43,070 --> 00:03:46,670 on the level of the discrete time signals. 77 00:03:46,670 --> 00:03:50,780 And at the other end, after you've done your transmissions 78 00:03:50,780 --> 00:03:53,150 through the channel and you've demodulated and filtered, 79 00:03:53,150 --> 00:03:55,820 you get a sequence which ideally is 80 00:03:55,820 --> 00:03:59,360 a replication of the sequence that you sent in. 81 00:03:59,360 --> 00:04:03,880 It can't have a scale factor, scale factors don't worry us. 82 00:04:03,880 --> 00:04:06,650 In this case, you see that the amplitude is divided by 2. 83 00:04:06,650 --> 00:04:09,460 But basically you see the trace of what was 84 00:04:09,460 --> 00:04:12,370 sent at the transmitting end. 85 00:04:12,370 --> 00:04:14,530 There's some distortion that's introduced 86 00:04:14,530 --> 00:04:16,360 by the dynamics of the channel, and we'll 87 00:04:16,360 --> 00:04:18,529 be talking about that in more detail later. 88 00:04:18,529 --> 00:04:21,399 So we aren't getting quite the straight edges. 89 00:04:21,399 --> 00:04:23,830 But after a brief transient period, 90 00:04:23,830 --> 00:04:26,230 the waveform seems to settle to the constant value 91 00:04:26,230 --> 00:04:28,240 that we had of the input. 92 00:04:28,240 --> 00:04:30,805 So this is our received set of samples. 93 00:04:30,805 --> 00:04:32,680 Now in this figure, I've assumed that there's 94 00:04:32,680 --> 00:04:34,630 no noise, only the distortion. 95 00:04:34,630 --> 00:04:36,760 This lecture is going to be about the noise. 96 00:04:36,760 --> 00:04:39,750 I wanted you to get the sense of what distortion does, 97 00:04:39,750 --> 00:04:42,250 and then we'll park that issue and come back to it next time 98 00:04:42,250 --> 00:04:45,030 and actually for several lectures after that. 99 00:04:45,030 --> 00:04:47,620 But this lecture we're going to focus on noise. 100 00:04:47,620 --> 00:04:51,070 Before we look at noise, this is what a noise-free 101 00:04:51,070 --> 00:04:53,800 received signal might look like with just the distortion in it. 102 00:04:56,360 --> 00:04:56,860 OK. 103 00:04:56,860 --> 00:05:01,630 And now you've got to convert to a bit sequence. 104 00:05:01,630 --> 00:05:04,660 So a simple way to do that is pick an appropriate point 105 00:05:04,660 --> 00:05:06,460 in each bit slot. 106 00:05:06,460 --> 00:05:08,580 Each slot of 16 samples long. 107 00:05:08,580 --> 00:05:10,780 Pick an appropriate point, taking 108 00:05:10,780 --> 00:05:12,280 account of these transient effects 109 00:05:12,280 --> 00:05:14,610 and so on, and then sample. 110 00:05:14,610 --> 00:05:18,100 And if the sample value is above a threshold, 111 00:05:18,100 --> 00:05:19,690 you'll declare a 1. 112 00:05:19,690 --> 00:05:23,990 If the sample values below the threshold, you'll declare a 0. 113 00:05:23,990 --> 00:05:27,380 And so you reconstruct the sequence that went in. 114 00:05:27,380 --> 00:05:31,450 So we have the sample and threshold feature here. 115 00:05:31,450 --> 00:05:34,060 So we're just taking one of the samples in the bit period, 116 00:05:34,060 --> 00:05:36,490 comparing with the threshold, and making a declaration. 117 00:05:36,490 --> 00:05:38,500 That's a very simple-minded decision rule. 118 00:05:41,210 --> 00:05:41,860 OK. 119 00:05:41,860 --> 00:05:45,880 So we'll come back to distortion. 120 00:05:45,880 --> 00:05:48,920 Today I want to talk about noise, 121 00:05:48,920 --> 00:05:51,120 and I want to then suppress distortion. 122 00:05:51,120 --> 00:05:52,880 So let's forget about distortion. 123 00:05:52,880 --> 00:05:54,530 Let's assume that the received signal 124 00:05:54,530 --> 00:05:59,840 yn is exactly what was sent except for some additive noise. 125 00:05:59,840 --> 00:06:04,430 So what we're imagining is you send 126 00:06:04,430 --> 00:06:16,750 a nice clean set of samples here into your digital-to-analog 127 00:06:16,750 --> 00:06:22,660 converter, and what comes out ideally 128 00:06:22,660 --> 00:06:28,160 would be the same set of samples, 129 00:06:28,160 --> 00:06:31,130 but actually what happens is that each of these samples 130 00:06:31,130 --> 00:06:32,930 is perturbed by noise. 131 00:06:32,930 --> 00:06:39,000 And so you get something that might look like this. 132 00:06:44,290 --> 00:06:50,185 OK, so this is y, then, and what we had before was x of n. 133 00:06:55,080 --> 00:06:55,580 OK. 134 00:06:55,580 --> 00:06:57,872 So nominally you'd get the same thing. 135 00:06:57,872 --> 00:06:59,330 The only thing that's different now 136 00:06:59,330 --> 00:07:01,850 is you've got an additive noise. 137 00:07:01,850 --> 00:07:05,630 We're going to assume that this noise sample 138 00:07:05,630 --> 00:07:10,130 wn is independent from one sample to the next. 139 00:07:10,130 --> 00:07:12,030 So when the channel and the processing 140 00:07:12,030 --> 00:07:15,710 and so on decides to put a noise sample on this, 141 00:07:15,710 --> 00:07:18,350 it doesn't pay attention to what noise sample was 142 00:07:18,350 --> 00:07:19,530 out of on either side. 143 00:07:19,530 --> 00:07:22,138 So every noise sample is picked independently. 144 00:07:22,138 --> 00:07:23,930 And it's picked from the same distribution. 145 00:07:23,930 --> 00:07:27,270 That's with the identically distributed part of this mean. 146 00:07:27,270 --> 00:07:30,560 So the characteristics of the noise 147 00:07:30,560 --> 00:07:32,310 are the same right through our signal. 148 00:07:32,310 --> 00:07:33,200 That's what we're assuming. 149 00:07:33,200 --> 00:07:34,867 That's the identically distributed part. 150 00:07:34,867 --> 00:07:37,250 It's a statement about the stationarity of the noise 151 00:07:37,250 --> 00:07:40,070 characteristics. 152 00:07:40,070 --> 00:07:42,020 All of this can be generalized, but this 153 00:07:42,020 --> 00:07:45,100 is where we're going to have our story 154 00:07:45,100 --> 00:07:48,740 and that's all we're going to consider. 155 00:07:48,740 --> 00:07:52,940 OK, a key metric, then, is what's 156 00:07:52,940 --> 00:07:54,080 the signal-to-noise ratio? 157 00:07:54,080 --> 00:07:57,530 This is something that you see all over the place, the SNR. 158 00:07:57,530 --> 00:08:01,490 Usually what people mean is signal power, 159 00:08:01,490 --> 00:08:04,060 and power is usually the square of a signal-- that's 160 00:08:04,060 --> 00:08:05,060 what you're thinking of. 161 00:08:05,060 --> 00:08:07,633 If you think of voltages, for instance, 162 00:08:07,633 --> 00:08:10,050 the square of the voltage gives you power in the resistor. 163 00:08:10,050 --> 00:08:12,290 So you think of the signal as being x, 164 00:08:12,290 --> 00:08:15,002 Its power as being x squared. 165 00:08:15,002 --> 00:08:16,460 Except you've got to decide, do you 166 00:08:16,460 --> 00:08:19,760 want to talk about the peak power or the time average power 167 00:08:19,760 --> 00:08:22,560 or some other measurement of the signal power? 168 00:08:22,560 --> 00:08:26,280 So that's the signal part of this ratio. 169 00:08:26,280 --> 00:08:31,130 And then the noise part of the ratio is the noise variance. 170 00:08:31,130 --> 00:08:33,799 So we have a noise component wn, it's 171 00:08:33,799 --> 00:08:36,350 the expected squared amplitude of that. 172 00:08:36,350 --> 00:08:38,299 Oh, by the way, I didn't-- 173 00:08:38,299 --> 00:08:40,520 this is on my slide, but I didn't say it yet. 174 00:08:40,520 --> 00:08:42,960 I'm going to assume the noise is zero mean. 175 00:08:42,960 --> 00:08:45,740 Which means that these excursions from what 176 00:08:45,740 --> 00:08:51,260 you expect on average are at 0. 177 00:08:51,260 --> 00:08:54,320 If there was a systematic bias to the noise, 178 00:08:54,320 --> 00:08:56,810 if I knew that there was a non-zero mean, 179 00:08:56,810 --> 00:08:58,700 I could just factor that into my processing 180 00:08:58,700 --> 00:09:01,800 and think of my expected received signal 181 00:09:01,800 --> 00:09:04,200 as taking account of that non-zero mean. 182 00:09:04,200 --> 00:09:06,140 So there's no loss of generality, really. 183 00:09:06,140 --> 00:09:10,010 I'm assuming a zero mean noise. 184 00:09:10,010 --> 00:09:12,700 OK. 185 00:09:12,700 --> 00:09:14,890 Now when you come to actually computing numbers, 186 00:09:14,890 --> 00:09:16,575 this is another example-- 187 00:09:16,575 --> 00:09:17,950 showing another kind of waveform, 188 00:09:17,950 --> 00:09:21,070 this is the sum of sinusoids, I assume, 189 00:09:21,070 --> 00:09:22,910 to which you're adding some noise. 190 00:09:22,910 --> 00:09:24,370 And in this particular simulation, 191 00:09:24,370 --> 00:09:28,030 by tweaking the value of A there, that's the-- 192 00:09:28,030 --> 00:09:30,040 it's a gain factor on the signal. 193 00:09:30,040 --> 00:09:32,780 You can actually vary the signal-to-noise ratio 194 00:09:32,780 --> 00:09:35,030 and get a feel for what difference 195 00:09:35,030 --> 00:09:37,210 signal-to-noise ratio is represented. 196 00:09:37,210 --> 00:09:40,600 So at high signal-to-noise ratio, 197 00:09:40,600 --> 00:09:43,460 the noise isn't perturbing what went down very much. 198 00:09:43,460 --> 00:09:46,150 But when you get the lower signal-to-noise ratios, 199 00:09:46,150 --> 00:09:48,490 the noise is actually distorting the signal 200 00:09:48,490 --> 00:09:52,190 that you started with quite substantially. 201 00:09:52,190 --> 00:09:58,930 Now the SNR here is described in dB, decibels. 202 00:09:58,930 --> 00:10:02,170 And so let me just say a word about that. 203 00:10:02,170 --> 00:10:04,338 That's a unit you'll see all the time. 204 00:10:04,338 --> 00:10:05,380 You've seen all the time. 205 00:10:08,060 --> 00:10:10,510 So we're really trying to measure 206 00:10:10,510 --> 00:10:11,810 a signal-to-noise ratio. 207 00:10:11,810 --> 00:10:13,660 So this is what you would normally think of. 208 00:10:13,660 --> 00:10:15,880 But in many applications, a logarithmic scale 209 00:10:15,880 --> 00:10:18,340 is really what you want to deal with. 210 00:10:18,340 --> 00:10:20,500 For instance, if you're measuring 211 00:10:20,500 --> 00:10:24,760 the response of the ear to noise intensities, 212 00:10:24,760 --> 00:10:27,310 it turns out there's a logarithmic feature built 213 00:10:27,310 --> 00:10:29,120 into our sensors. 214 00:10:29,120 --> 00:10:31,750 So usually want to be measuring power and power ratios 215 00:10:31,750 --> 00:10:33,940 in terms of a log scale. 216 00:10:33,940 --> 00:10:36,470 That should have had a capital B there. 217 00:10:36,470 --> 00:10:40,600 So here's the definition of what a ratio is on dB. 218 00:10:40,600 --> 00:10:43,430 It's the ratio log to the base 10 times 10. 219 00:10:46,360 --> 00:10:47,770 One caution here. 220 00:10:47,770 --> 00:10:49,750 I told you that when we talk about powers, 221 00:10:49,750 --> 00:10:52,100 that's the square of the amplitude. 222 00:10:52,100 --> 00:10:54,430 So if you're going to compare amplitudes, 223 00:10:54,430 --> 00:10:57,040 ratio of amplitudes on a log scale, then 224 00:10:57,040 --> 00:10:59,200 actually what you end up doing is taking 20 225 00:10:59,200 --> 00:11:01,560 log 10 ratio of amplitudes. 226 00:11:01,560 --> 00:11:03,910 So you'll sometimes see this definition as 20 log 227 00:11:03,910 --> 00:11:06,310 to the base 10 ratio of amplitudes, 228 00:11:06,310 --> 00:11:09,880 and what people are doing, then, is comparing amplitude ratios, 229 00:11:09,880 --> 00:11:10,630 not power ratios. 230 00:11:10,630 --> 00:11:11,505 You have a question? 231 00:11:11,505 --> 00:11:15,260 AUDIENCE: Why do we define power as amplitude squared? 232 00:11:15,260 --> 00:11:17,080 PROFESSOR: In sum-- so the question was, 233 00:11:17,080 --> 00:11:19,750 why do we define power as amplitude squared? 234 00:11:23,530 --> 00:11:29,350 If you think of an electrical circuit 235 00:11:29,350 --> 00:11:31,990 with some signal applied across it, 236 00:11:31,990 --> 00:11:34,030 a voltage, the instantaneous power 237 00:11:34,030 --> 00:11:38,560 dissipated in the resistor is given by that. 238 00:11:38,560 --> 00:11:42,700 So people start to think of square of a quantity as power. 239 00:11:42,700 --> 00:11:44,200 In the continuous time domain that's 240 00:11:44,200 --> 00:11:47,260 very natural in signals that come from physics, 241 00:11:47,260 --> 00:11:49,510 and that terminology is just being carried over 242 00:11:49,510 --> 00:11:51,190 to this kind of a discrete time setting. 243 00:11:51,190 --> 00:11:55,105 So when people say power, they mean square of the signal. 244 00:11:55,105 --> 00:11:56,730 It could've been called something else. 245 00:11:59,526 --> 00:12:01,860 OK. 246 00:12:01,860 --> 00:12:05,450 So you can actually span huge ratios in power 247 00:12:05,450 --> 00:12:10,210 on this log scale with much more better behaved numbers. 248 00:12:10,210 --> 00:12:14,540 0 dB, then, is a ratio of 1. 249 00:12:14,540 --> 00:12:16,730 3 dB, this is good to carry around in your head. 250 00:12:16,730 --> 00:12:19,830 3 dB, it's actually 3.01-something, 251 00:12:19,830 --> 00:12:23,270 but 3 dB is a factor of 2 on the power ratio, 252 00:12:23,270 --> 00:12:27,360 or square root of 2 on an amplitude ratio. 253 00:12:27,360 --> 00:12:30,050 So let's actually go back to what I showed you 254 00:12:30,050 --> 00:12:31,560 on the previous slide. 255 00:12:31,560 --> 00:12:36,470 So here, for instance, is an SNR of 0.4 dB. 256 00:12:36,470 --> 00:12:38,750 If I figure that that's close to 0 dB, 257 00:12:38,750 --> 00:12:41,660 then I should expect that the noise power and signal 258 00:12:41,660 --> 00:12:45,710 power are about equal, and the noise amplitude and signal 259 00:12:45,710 --> 00:12:47,400 amplitude are about equal. 260 00:12:47,400 --> 00:12:49,430 So what I expect to see is perturbations 261 00:12:49,430 --> 00:12:51,470 of the original signal that are comparable 262 00:12:51,470 --> 00:12:53,750 with the signal values themselves, 263 00:12:53,750 --> 00:12:55,250 and that's sort of what we see here. 264 00:12:55,250 --> 00:12:59,180 The shape of the signal is pretty distorted at this point 265 00:12:59,180 --> 00:13:02,960 because the typical amplitude of the noise sample 266 00:13:02,960 --> 00:13:06,460 is comparable with the signal sample that I'm interested in. 267 00:13:06,460 --> 00:13:08,120 OK, so when you get to 0 dB, you're 268 00:13:08,120 --> 00:13:12,530 starting to get quite disturbed-looking waveforms. 269 00:13:12,530 --> 00:13:16,290 When you have 20 dB in power, that's actually 100-- 270 00:13:16,290 --> 00:13:18,777 ratio of 100-- sorry, what is that? 271 00:13:18,777 --> 00:13:20,360 Yeah, that's a ratio of 100, isn't it? 272 00:13:20,360 --> 00:13:22,490 On par? 273 00:13:22,490 --> 00:13:24,300 So it's a ratio of 10 on amplitudes, 274 00:13:24,300 --> 00:13:25,550 and that's what you're seeing. 275 00:13:25,550 --> 00:13:29,300 The noise excursions are about a 10th of what 276 00:13:29,300 --> 00:13:31,625 the signal amplitudes are. 277 00:13:31,625 --> 00:13:32,125 All right. 278 00:13:32,125 --> 00:13:38,050 It takes a little getting used to, but it's fairly standard. 279 00:13:38,050 --> 00:13:38,550 OK. 280 00:13:38,550 --> 00:13:40,900 So now we want to figure out how to describe 281 00:13:40,900 --> 00:13:42,070 noise and work with it. 282 00:13:44,780 --> 00:13:49,420 So let's look at a typical run of a noise sequence. 283 00:13:49,420 --> 00:13:53,380 What I've done is just extracted the noise piece 284 00:13:53,380 --> 00:13:55,430 of a typical received signal. 285 00:13:55,430 --> 00:13:57,520 So it's got excursions above and below 0. 286 00:13:57,520 --> 00:14:00,550 Remember, I said it was a zero mean random variable that we're 287 00:14:00,550 --> 00:14:04,690 thinking of, zero mean noise. 288 00:14:04,690 --> 00:14:08,080 And you can describe how these values are distributed by just 289 00:14:08,080 --> 00:14:10,510 doing a simple histogram. 290 00:14:10,510 --> 00:14:13,120 And if you only take a few values like 100 samples, 291 00:14:13,120 --> 00:14:15,865 you get a pretty messy-looking histogram, 292 00:14:15,865 --> 00:14:18,167 it doesn't seem to have much structure. 293 00:14:18,167 --> 00:14:19,750 But as you take more and more samples, 294 00:14:19,750 --> 00:14:23,140 you'll typically find that the histogram actually settles out 295 00:14:23,140 --> 00:14:27,980 to a nice shape, to some subtle kind of shape. 296 00:14:27,980 --> 00:14:30,670 Normalizing this to have unit area under it 297 00:14:30,670 --> 00:14:32,950 gives you what's called the probability density 298 00:14:32,950 --> 00:14:34,430 function for the noise. 299 00:14:34,430 --> 00:14:36,130 So this is a term-- 300 00:14:36,130 --> 00:14:40,040 kind of notion that's critical in working with noise. 301 00:14:40,040 --> 00:14:42,070 So here's a step of idealization. 302 00:14:42,070 --> 00:14:44,615 We're stepping back from thinking about histograms 303 00:14:44,615 --> 00:14:46,990 to just a mathematical way of talking 304 00:14:46,990 --> 00:14:50,230 about how random quantities distribute themselves. 305 00:14:50,230 --> 00:14:51,820 So we'll talk about-- 306 00:14:51,820 --> 00:14:55,300 by the way, we've been using W for the noise 307 00:14:55,300 --> 00:14:58,510 and X for the signal, but if you look in probability books, 308 00:14:58,510 --> 00:15:01,150 the first variable that people-- the first symbol people reach 309 00:15:01,150 --> 00:15:03,597 for and they want to talk about a random variable is X, 310 00:15:03,597 --> 00:15:05,680 and I got stuck with a whole bunch of figures that 311 00:15:05,680 --> 00:15:08,830 had X in them, so I didn't want to change it to W. 312 00:15:08,830 --> 00:15:09,700 This is anything. 313 00:15:09,700 --> 00:15:11,520 We're going to apply it to our W, 314 00:15:11,520 --> 00:15:15,080 but for now it's some capital X. The other convention 315 00:15:15,080 --> 00:15:16,580 when you talk about random variables 316 00:15:16,580 --> 00:15:18,730 as you tend to use a capital letter to denote 317 00:15:18,730 --> 00:15:21,010 the random variable. 318 00:15:21,010 --> 00:15:21,510 OK. 319 00:15:21,510 --> 00:15:23,980 So we say that X is a random variable governed 320 00:15:23,980 --> 00:15:27,550 by a particular probability density function. 321 00:15:27,550 --> 00:15:29,680 If you can compute the probability 322 00:15:29,680 --> 00:15:32,050 that X lies in some particular interval 323 00:15:32,050 --> 00:15:35,530 by taking the corresponding area under that PDF. 324 00:15:35,530 --> 00:15:39,550 So the PDF is the object that gives you probabilities 325 00:15:39,550 --> 00:15:41,300 from areas under the integrals. 326 00:15:41,300 --> 00:15:43,600 So if you want the probability that the quantity X, 327 00:15:43,600 --> 00:15:47,380 take the numerical values in this range, X1 to X2, 328 00:15:47,380 --> 00:15:50,590 then you integrate the PDF from X1 to X2, 329 00:15:50,590 --> 00:15:53,320 and this area is what you call-- 330 00:15:53,320 --> 00:15:56,587 that area is the probability. 331 00:15:56,587 --> 00:15:58,420 And the total area under the PDF, of course, 332 00:15:58,420 --> 00:16:00,910 has to be 1 because the probability 333 00:16:00,910 --> 00:16:03,130 that X lies somewhere is 1. 334 00:16:03,130 --> 00:16:07,070 The probability that X takes some value is 1. 335 00:16:07,070 --> 00:16:10,180 So this is how we work with PDFs. 336 00:16:10,180 --> 00:16:13,210 Again, you'll find when people want to sketch a PDF, 337 00:16:13,210 --> 00:16:16,353 the reflex is to sketch one of these bell-shaped things. 338 00:16:16,353 --> 00:16:18,520 And it turns out there's actually a reason for that. 339 00:16:21,580 --> 00:16:24,040 This bell-shaped thing or a specific bell-shaped thing 340 00:16:24,040 --> 00:16:26,440 called the Gaussian tends to arise 341 00:16:26,440 --> 00:16:28,120 in all sorts of applications, and that's 342 00:16:28,120 --> 00:16:30,760 a consequence of something called the central limit 343 00:16:30,760 --> 00:16:32,290 theorem. 344 00:16:32,290 --> 00:16:37,823 This is considered one of the most important results 345 00:16:37,823 --> 00:16:38,740 in probability theory. 346 00:16:38,740 --> 00:16:44,290 It actually dates back to about the 1730s as a conjecture, 347 00:16:44,290 --> 00:16:46,820 but it was Laplace who-- 348 00:16:46,820 --> 00:16:52,978 in I guess the late 1700s, early 1800s who actually proved it. 349 00:16:52,978 --> 00:16:55,270 And it wasn't actually called the central limit theorem 350 00:16:55,270 --> 00:16:57,845 until much more recently, till about 1930 or so. 351 00:16:57,845 --> 00:17:00,220 And was called that because it was the limit theorem that 352 00:17:00,220 --> 00:17:01,720 was central to all of probability, 353 00:17:01,720 --> 00:17:03,178 that was the thinking. 354 00:17:03,178 --> 00:17:04,720 So here is the central limit theorem. 355 00:17:04,720 --> 00:17:08,530 It says that if you sum up a whole bunch 356 00:17:08,530 --> 00:17:13,000 of little random quantities that are not necessarily Gaussian, 357 00:17:13,000 --> 00:17:16,700 and if they each have finite mean and finite variance, 358 00:17:16,700 --> 00:17:18,950 the sum is going to have a distribution that's going 359 00:17:18,950 --> 00:17:21,319 to look increasingly Gaussian. 360 00:17:21,319 --> 00:17:22,760 So you could start, for instance, 361 00:17:22,760 --> 00:17:29,430 with a random variable that's described by this triangular 362 00:17:29,430 --> 00:17:29,930 PDF. 363 00:17:32,450 --> 00:17:34,880 Take a whole bunch of random variables generated 364 00:17:34,880 --> 00:17:36,230 according to that PDF. 365 00:17:36,230 --> 00:17:38,750 When I say generated according to that PDF, what I mean 366 00:17:38,750 --> 00:17:41,390 is that the probability that you get a value between any two 367 00:17:41,390 --> 00:17:46,640 limits here is the area under that piece of the triangle. 368 00:17:46,640 --> 00:17:49,250 Generate a whole bunch of these and sum them together, 369 00:17:49,250 --> 00:17:50,900 you find that the resulting histogram 370 00:17:50,900 --> 00:17:54,350 starts to look Gaussian. 371 00:17:54,350 --> 00:17:56,427 You can start with another kind of distribution, 372 00:17:56,427 --> 00:17:58,010 and again, it starts to look Gaussian. 373 00:17:58,010 --> 00:18:01,403 And the more of these you add, the more it looks Gaussian. 374 00:18:01,403 --> 00:18:03,320 And so this can be actually made very precise. 375 00:18:03,320 --> 00:18:05,450 There's a very precise sense in which 376 00:18:05,450 --> 00:18:08,212 the limiting distribution in a situation like this 377 00:18:08,212 --> 00:18:08,795 is a Gaussian. 378 00:18:11,420 --> 00:18:12,530 So what is a Gaussian? 379 00:18:12,530 --> 00:18:14,210 I've got to describe that for you. 380 00:18:14,210 --> 00:18:16,370 I'll do it in more detail in a second. 381 00:18:19,100 --> 00:18:22,030 First, let me tell you how we defined these two 382 00:18:22,030 --> 00:18:22,810 key parameters. 383 00:18:22,810 --> 00:18:26,298 These are things that from other sorts of contexts. 384 00:18:26,298 --> 00:18:28,465 The mean and the standard deviation of the variance, 385 00:18:28,465 --> 00:18:30,670 you know it from quiz scores at least, 386 00:18:30,670 --> 00:18:34,390 but here is the mathematical definition in terms of a PDF. 387 00:18:34,390 --> 00:18:38,350 So if you have a PDF for a random variable capital X, 388 00:18:38,350 --> 00:18:42,600 the mean value of capital X is-- 389 00:18:42,600 --> 00:18:44,620 it's basically the average value of X 390 00:18:44,620 --> 00:18:48,400 weighted by the probability, which is what you expect. 391 00:18:48,400 --> 00:18:50,910 So it's X times the PDF integrated 392 00:18:50,910 --> 00:18:52,300 over all possible values. 393 00:18:52,300 --> 00:18:55,060 That's the definition of the expected value. 394 00:18:55,060 --> 00:18:57,060 And what we do when we take the expected value 395 00:18:57,060 --> 00:19:01,240 of the mean value on a quiz is a sort of discrete time version 396 00:19:01,240 --> 00:19:02,330 of this. 397 00:19:02,330 --> 00:19:06,050 So we're seeing how many people in a particular bin 398 00:19:06,050 --> 00:19:08,510 and multiply by the score for the people in that bin 399 00:19:08,510 --> 00:19:10,390 and sum over all possible bins. 400 00:19:10,390 --> 00:19:12,730 That's one way to think of what this is doing, 401 00:19:12,730 --> 00:19:15,590 assuming you've got the right normalization of the PDF. 402 00:19:18,710 --> 00:19:22,200 And the variance is the expected squared deviation 403 00:19:22,200 --> 00:19:23,770 from the mean value. 404 00:19:23,770 --> 00:19:27,060 So here's a deviation from the mean value. 405 00:19:27,060 --> 00:19:30,070 You square it, and now you want to take its expected value, 406 00:19:30,070 --> 00:19:33,180 so you weight it by the PDF of X and that gives you 407 00:19:33,180 --> 00:19:33,970 the variance. 408 00:19:33,970 --> 00:19:37,230 So the variance is the expected squared deviation 409 00:19:37,230 --> 00:19:39,291 from the mean value. 410 00:19:39,291 --> 00:19:42,750 OK, so the PDF is valuable in getting all of this. 411 00:19:46,410 --> 00:19:51,150 And to get a sense of what means and standard deviations 412 00:19:51,150 --> 00:19:52,082 and variances do-- 413 00:19:52,082 --> 00:19:54,540 I don't know if I said from the previous slide, by the way, 414 00:19:54,540 --> 00:19:57,120 that standard deviation is the square root of the variance. 415 00:19:57,120 --> 00:19:57,930 Did I say that? 416 00:19:57,930 --> 00:20:00,330 Maybe not. 417 00:20:00,330 --> 00:20:02,470 But I have it at the bottom of the slide, right? 418 00:20:02,470 --> 00:20:02,970 OK. 419 00:20:07,280 --> 00:20:08,050 OK. 420 00:20:08,050 --> 00:20:11,170 So shifting the mean of a random variable, 421 00:20:11,170 --> 00:20:15,127 if I define a new random variable with the same PPF 422 00:20:15,127 --> 00:20:16,960 except for a different mean, what that means 423 00:20:16,960 --> 00:20:19,390 is that-- what that signifies is that the PDF has just 424 00:20:19,390 --> 00:20:21,040 shifted over by that amount. 425 00:20:21,040 --> 00:20:23,450 So changing the mean and nothing else 426 00:20:23,450 --> 00:20:28,030 will just shift the PDF over to the corresponding position. 427 00:20:28,030 --> 00:20:30,850 Changing the variance from a small value to a large value 428 00:20:30,850 --> 00:20:33,240 will spread out the PDF because you're 429 00:20:33,240 --> 00:20:35,890 the variance is capturing the expected squared deviation 430 00:20:35,890 --> 00:20:37,040 from the mean. 431 00:20:37,040 --> 00:20:42,040 So a higher variance PDF has got to have a larger spread. 432 00:20:42,040 --> 00:20:44,020 But because the areas normalized to 1, 433 00:20:44,020 --> 00:20:46,780 if it spreads out this way, it's got to come down on top, 434 00:20:46,780 --> 00:20:48,760 and that's what you're seeing here. 435 00:20:48,760 --> 00:20:50,260 All these pictures actually turn out 436 00:20:50,260 --> 00:20:52,900 to be drawn for the Gaussian, but my statements 437 00:20:52,900 --> 00:20:56,028 are more general here. 438 00:20:56,028 --> 00:20:57,320 But here's the Gaussian itself. 439 00:21:00,260 --> 00:21:02,300 So now I'm going back to my notation 440 00:21:02,300 --> 00:21:04,970 W. We're going to think of a random variable 441 00:21:04,970 --> 00:21:09,997 W which is going to be typical of all my noise samples. 442 00:21:09,997 --> 00:21:12,080 It's going to have some mean which we'll be taking 443 00:21:12,080 --> 00:21:15,050 to be 0 and our examples. 444 00:21:15,050 --> 00:21:17,370 It's got a variance sigma squared. 445 00:21:17,370 --> 00:21:21,860 So if a random variable has this particular PDF, 446 00:21:21,860 --> 00:21:23,210 we call it Gaussian. 447 00:21:23,210 --> 00:21:26,330 That's the definition of a Gaussian random variable. 448 00:21:26,330 --> 00:21:27,830 The number here, while you've got 449 00:21:27,830 --> 00:21:30,860 to remember it at some point, but all it's doing 450 00:21:30,860 --> 00:21:34,740 is normalizing to unit area. 451 00:21:34,740 --> 00:21:36,710 So the key thing about a Gaussian 452 00:21:36,710 --> 00:21:40,730 is that it's an exponential with a negative sign there 453 00:21:40,730 --> 00:21:43,880 of the squared deviation from the mean normalized 454 00:21:43,880 --> 00:21:46,490 by the variance with that extra factor 2 there. 455 00:21:49,730 --> 00:21:54,790 So different choices of variance will give you different shapes 456 00:21:54,790 --> 00:21:55,290 here. 457 00:21:55,290 --> 00:21:57,350 So the smaller variances correspond 458 00:21:57,350 --> 00:22:01,050 to the more peaked and more sharply-falling PDFs. 459 00:22:03,758 --> 00:22:04,300 So let's see. 460 00:22:04,300 --> 00:22:06,620 How many standard deviations away from the mean 461 00:22:06,620 --> 00:22:10,060 you have to go before you have very low probability 462 00:22:10,060 --> 00:22:11,410 of reaching there? 463 00:22:16,430 --> 00:22:18,380 Anyone? 464 00:22:18,380 --> 00:22:20,480 There's no unique answer to this, but yeah? 465 00:22:20,480 --> 00:22:21,180 AUDIENCE: 3? 466 00:22:21,180 --> 00:22:22,820 PROFESSOR: 3 is not about idea. 467 00:22:22,820 --> 00:22:24,320 So let's see. 468 00:22:24,320 --> 00:22:26,150 Let's take sigma squared equals 1. 469 00:22:26,150 --> 00:22:28,950 That's variance of 1, so the standard deviation is 1. 470 00:22:28,950 --> 00:22:31,580 So for the red trace, by the time 471 00:22:31,580 --> 00:22:35,990 we get out to the number 3, we expect to actually see 472 00:22:35,990 --> 00:22:37,550 a very low value for the PDF. 473 00:22:37,550 --> 00:22:39,700 So 3 sounds about right. 474 00:22:39,700 --> 00:22:42,320 Does that hold up for the blue one? 475 00:22:42,320 --> 00:22:44,840 Sigma squared is 0.25. 476 00:22:44,840 --> 00:22:47,970 So the square root of that is a standard deviation, 477 00:22:47,970 --> 00:22:50,120 which is 0.5, so 3 times that. 478 00:22:50,120 --> 00:22:54,440 So when we get out to about 1.5, we should be essentially at 0. 479 00:22:54,440 --> 00:22:56,960 So don't forget the square root. 480 00:22:56,960 --> 00:23:00,320 The other thing-- actually, I should have commented on this 481 00:23:00,320 --> 00:23:03,110 earlier, let me show it to you-- 482 00:23:03,110 --> 00:23:08,540 on this slide that I had, I labeled this arrow 483 00:23:08,540 --> 00:23:10,310 here just schematically to show you 484 00:23:10,310 --> 00:23:12,350 that it's a measure of width. 485 00:23:12,350 --> 00:23:14,973 But the tag I put on it is standard deviation. 486 00:23:14,973 --> 00:23:16,640 Standard deviation is the thing that you 487 00:23:16,640 --> 00:23:19,010 want to use when you want to measure width 488 00:23:19,010 --> 00:23:19,790 on a distribution. 489 00:23:19,790 --> 00:23:21,290 That has the right units. 490 00:23:21,290 --> 00:23:23,900 Standard deviation, the square root of variance 491 00:23:23,900 --> 00:23:27,200 has the same units as X. If X is a voltage, 492 00:23:27,200 --> 00:23:29,150 the standard deviation is units of voltage. 493 00:23:29,150 --> 00:23:32,600 It would be a mistake to label a spread here by the variance. 494 00:23:32,600 --> 00:23:34,910 You want to think in terms of standard deviation 495 00:23:34,910 --> 00:23:38,410 when you're thinking about spread. 496 00:23:38,410 --> 00:23:40,790 So you define the variance and then take the square root 497 00:23:40,790 --> 00:23:44,210 to get the standard deviation. 498 00:23:44,210 --> 00:23:45,320 OK. 499 00:23:45,320 --> 00:23:49,220 So for our noise in this kind of setting, 500 00:23:49,220 --> 00:23:50,720 in our communications setting, we're 501 00:23:50,720 --> 00:23:54,350 going to assume that every noise sample was 502 00:23:54,350 --> 00:23:56,510 drawn from a Gaussian distribution with zero mean. 503 00:23:56,510 --> 00:23:59,190 Just the same kind of distribution that I showed you. 504 00:23:59,190 --> 00:24:02,870 So the only thing that's going to change from one example 505 00:24:02,870 --> 00:24:04,640 to another will be the variance. 506 00:24:04,640 --> 00:24:07,360 But for a given case, we're talking about IID noise. 507 00:24:07,360 --> 00:24:10,162 You're going to fix the variance, have zero mean, 508 00:24:10,162 --> 00:24:11,870 and all your noise samples will be pulled 509 00:24:11,870 --> 00:24:13,430 from that same distribution. 510 00:24:16,910 --> 00:24:21,440 If you were actually looking at data here for these excursions, 511 00:24:21,440 --> 00:24:23,690 if you were actually looking at what 512 00:24:23,690 --> 00:24:27,620 the excursions from the baseline are, and you wanted 513 00:24:27,620 --> 00:24:30,530 in a numerical experiment-- in a simulation setting, 514 00:24:30,530 --> 00:24:33,050 for instance, or in a physical experiment 515 00:24:33,050 --> 00:24:35,830 to get an estimate of what the mean and variance are, 516 00:24:35,830 --> 00:24:38,150 well, we've got very familiar expressions. 517 00:24:38,150 --> 00:24:43,437 You would take the sample mean or the sample variance. 518 00:24:43,437 --> 00:24:45,020 The square root of the sample variance 519 00:24:45,020 --> 00:24:48,090 would then be your estimate of the standard deviation. 520 00:24:48,090 --> 00:24:51,030 So we can come at the same objects-- 521 00:24:51,030 --> 00:24:54,380 well, we have the PDF, which is the mathematical construct, 522 00:24:54,380 --> 00:24:57,040 but in an experimental setting, this 523 00:24:57,040 --> 00:24:59,810 is how you would go about estimating these. 524 00:24:59,810 --> 00:25:01,820 And there's a whole big theory of estimation 525 00:25:01,820 --> 00:25:04,580 that tells you whether these are good estimates or not 526 00:25:04,580 --> 00:25:06,188 and offers alternatives, and we're not 527 00:25:06,188 --> 00:25:07,230 getting into any of that. 528 00:25:07,230 --> 00:25:09,740 We're staying close to the basics 529 00:25:09,740 --> 00:25:12,470 and close to what makes sense intuitively 530 00:25:12,470 --> 00:25:15,950 and what's essentially used all over. 531 00:25:20,750 --> 00:25:26,150 So now we have the task at the receiver 532 00:25:26,150 --> 00:25:28,975 of getting a bunch of samples like this 533 00:25:28,975 --> 00:25:31,100 and then trying to decide whether what we're seeing 534 00:25:31,100 --> 00:25:34,640 is a reflection of a 1 or a 0. 535 00:25:34,640 --> 00:25:40,760 If we had 0's sent from here, what 536 00:25:40,760 --> 00:25:46,880 we're going to see after we receive the noisy signal is 537 00:25:46,880 --> 00:25:49,738 perturbed samples. 538 00:25:49,738 --> 00:25:51,780 And so we're going to look at a particular sample 539 00:25:51,780 --> 00:25:55,580 and try and decide whether in that bit slot what was sent 540 00:25:55,580 --> 00:25:56,450 was a 0 or a 1. 541 00:25:59,570 --> 00:26:04,410 I'm going to actually use a scheme for illustration here 542 00:26:04,410 --> 00:26:07,910 that's not the scheme that I've suggested here. 543 00:26:07,910 --> 00:26:09,980 Here, I suggested something that's sending 0. 544 00:26:09,980 --> 00:26:13,910 If I'm communicating a 0 and I'm sending some other voltage 545 00:26:13,910 --> 00:26:19,130 level when I want to communicate a 1, I'm going between 0 and 1. 546 00:26:19,130 --> 00:26:20,810 It turns out on the physical channel, 547 00:26:20,810 --> 00:26:25,280 if you've got a transmitter with a certain peak power, 548 00:26:25,280 --> 00:26:27,680 you're probably better off using a plus V 549 00:26:27,680 --> 00:26:30,680 to indicate a 1 and a minus V for a 0 550 00:26:30,680 --> 00:26:33,650 because you're using that transmitter at full power 551 00:26:33,650 --> 00:26:34,250 all the time. 552 00:26:34,250 --> 00:26:36,980 So you're actually trying to overcome the noise 553 00:26:36,980 --> 00:26:38,548 as strongly as possible. 554 00:26:38,548 --> 00:26:40,340 So that's the scheme I'm going to consider. 555 00:26:40,340 --> 00:26:41,798 I'm going to consider that when you 556 00:26:41,798 --> 00:26:45,650 want to signal a 1, what you're doing at the transmitting end 557 00:26:45,650 --> 00:26:51,460 is sending out L samples at plus some peak voltage Vp. 558 00:26:51,460 --> 00:26:56,420 And when you want to signal a 0, you send L samples at minus Vp. 559 00:26:56,420 --> 00:27:01,160 So this is what we refer to as a bipolar signaling scheme. 560 00:27:12,280 --> 00:27:15,520 So it would be something like this. 561 00:27:21,690 --> 00:27:23,980 This is the xn. 562 00:27:23,980 --> 00:27:28,460 And this is what I'm using to signal a 1, 563 00:27:28,460 --> 00:27:31,570 and this is what I'm using to signal a 0. 564 00:27:31,570 --> 00:27:35,440 But in terms of actual voltage levels, 565 00:27:35,440 --> 00:27:40,150 this is minus Vp and Vp here. 566 00:27:52,800 --> 00:27:55,440 And on the receiving end, what I'm 567 00:27:55,440 --> 00:27:58,140 getting at any particular samples-- 568 00:27:58,140 --> 00:28:00,990 so I pick one particular sample to look at, 569 00:28:00,990 --> 00:28:05,730 and when I look at that sample-- let's say at sample n sub j. 570 00:28:05,730 --> 00:28:08,370 So maybe I'm looking in the j-th bit slot 571 00:28:08,370 --> 00:28:12,000 and I pick one particular sample time, let we call that n sub j. 572 00:28:12,000 --> 00:28:16,620 And I have to decide, am I looking at plus Vp with noise 573 00:28:16,620 --> 00:28:19,500 or am I looking at minus Vp with noise? 574 00:28:19,500 --> 00:28:21,312 That's a decision. 575 00:28:21,312 --> 00:28:22,020 I know the Vp's-- 576 00:28:24,527 --> 00:28:26,610 assume that we've taken care of the scaling and so 577 00:28:26,610 --> 00:28:28,080 on across the channel. 578 00:28:28,080 --> 00:28:30,780 And I know the characteristics of the noise. 579 00:28:30,780 --> 00:28:33,780 I know that the noise samples are Gaussian, zero mean, 580 00:28:33,780 --> 00:28:34,530 and some variance. 581 00:28:39,460 --> 00:28:43,580 So if I draw a picture that's turned sideways 582 00:28:43,580 --> 00:28:50,510 here in terms of the received signal, let's see. 583 00:28:53,300 --> 00:28:55,820 I might get something centered around 584 00:28:55,820 --> 00:29:01,010 minus Vp or something centered around Vp. 585 00:29:01,010 --> 00:29:05,810 If a minus Vp was sent, then it's got a noise added to it. 586 00:29:05,810 --> 00:29:07,880 The noise has a Gaussian distribution. 587 00:29:11,730 --> 00:29:13,550 So this is the distribution of values 588 00:29:13,550 --> 00:29:17,270 I expect if a 0 was sent. 589 00:29:17,270 --> 00:29:20,350 So this is-- let me call it-- 590 00:29:20,350 --> 00:29:23,630 it's the distribution of Y-- 591 00:29:23,630 --> 00:29:25,730 I'm not going to put all the attachments here-- 592 00:29:25,730 --> 00:29:28,780 if a 0 was sent. 593 00:29:28,780 --> 00:29:33,050 Because my shorthand notation for the density of Y assuming 594 00:29:33,050 --> 00:29:34,760 a 0 was sent. 595 00:29:34,760 --> 00:29:38,720 I haven't drawn a very good Gaussian, but you get the idea. 596 00:29:38,720 --> 00:29:46,740 And here's the distribution of Y if a 1 was sent. 597 00:29:51,950 --> 00:29:55,800 So what I'm actually measuring is some number out here. 598 00:29:55,800 --> 00:29:56,630 I get some number. 599 00:30:00,640 --> 00:30:02,230 And I've got to decide, did that come 600 00:30:02,230 --> 00:30:08,260 from having sent a 0 and getting this much noise 601 00:30:08,260 --> 00:30:11,140 or did it come from sending a 1 and getting this much noise? 602 00:30:14,688 --> 00:30:15,480 That's the problem. 603 00:30:18,910 --> 00:30:21,000 So if 0's and 1's are equally likely, 604 00:30:21,000 --> 00:30:22,890 what do you think is a sensible rule here? 605 00:30:26,620 --> 00:30:29,010 Just pick a threshold where these two cross. 606 00:30:29,010 --> 00:30:32,350 Threshold in the middle. 607 00:30:32,350 --> 00:30:35,160 So if the sample is above the threshold, you declare a 1. 608 00:30:35,160 --> 00:30:39,310 If it's below the threshold, you declare a 0. 609 00:30:39,310 --> 00:30:42,160 What if 0's and 1's were not equally likely? 610 00:30:42,160 --> 00:30:44,590 Suppose it was much more likely that you would get a 1. 611 00:30:47,108 --> 00:30:49,650 And suppose we're still thinking in terms of threshold rules, 612 00:30:49,650 --> 00:30:51,360 what might you want to do? 613 00:30:51,360 --> 00:30:54,273 Suppose it's much more likely that we get a 1. 614 00:30:54,273 --> 00:30:55,690 AUDIENCE: Move the threshold to -- 615 00:30:55,690 --> 00:30:56,338 PROFESSOR: Sorry? 616 00:30:56,338 --> 00:30:57,970 AUDIENCE: Move the threshold to the left. 617 00:30:57,970 --> 00:30:59,262 PROFESSOR: Move it to the left. 618 00:30:59,262 --> 00:31:03,130 So you want to actually allow for the fact 619 00:31:03,130 --> 00:31:05,230 that most of the time you're getting 1's, 620 00:31:05,230 --> 00:31:08,140 and so you really have to get close to the 0 621 00:31:08,140 --> 00:31:10,050 before you going to declare a 0. 622 00:31:10,050 --> 00:31:11,950 So your bias kind of gets built in. 623 00:31:11,950 --> 00:31:15,680 Now this is just thinking as an engineer what you might do. 624 00:31:15,680 --> 00:31:18,760 It turns out that for Gaussian noise, 625 00:31:18,760 --> 00:31:20,410 the optimum decision rule in terms 626 00:31:20,410 --> 00:31:22,330 of minimizing the probability of error 627 00:31:22,330 --> 00:31:25,210 is exactly a threshold rule of this kind. 628 00:31:25,210 --> 00:31:27,910 And the analysis will tell you where that threshold should be. 629 00:31:27,910 --> 00:31:29,920 So we're not getting into proving 630 00:31:29,920 --> 00:31:32,080 that this is the optimum, but it turns out 631 00:31:32,080 --> 00:31:34,990 with Gaussian noise, the minimum probability of error decision 632 00:31:34,990 --> 00:31:39,100 rule for this kind of a hypothesis test-- 633 00:31:39,100 --> 00:31:41,380 this is a classic hypothesis test-- 634 00:31:41,380 --> 00:31:42,830 is to pick a threshold. 635 00:31:42,830 --> 00:31:44,860 Now that's not true necessarily for other sorts 636 00:31:44,860 --> 00:31:47,530 of distributions, it's not true for the settings, 637 00:31:47,530 --> 00:31:51,470 but for the Gaussian it turns out it's what you have to do. 638 00:31:51,470 --> 00:31:54,590 So let's just assume equal prior probabilities. 639 00:31:54,590 --> 00:31:57,570 So 0's and 1's come at you with equal probability, 640 00:31:57,570 --> 00:32:01,150 and we now have to figure out what the probability of error 641 00:32:01,150 --> 00:32:02,440 is. 642 00:32:02,440 --> 00:32:07,300 So there's a slide here with some computation. 643 00:32:07,300 --> 00:32:08,950 Let me just walk you through that. 644 00:32:08,950 --> 00:32:11,075 We don't have to follow all the details and you can 645 00:32:11,075 --> 00:32:12,400 study it and more-- 646 00:32:12,400 --> 00:32:14,740 I mean, you can study it at leisure, 647 00:32:14,740 --> 00:32:17,580 but it's the same picture I showed. 648 00:32:17,580 --> 00:32:18,080 OK? 649 00:32:18,080 --> 00:32:19,043 AUDIENCE: [INAUDIBLE] 650 00:32:19,043 --> 00:32:19,710 PROFESSOR: Yeah? 651 00:32:19,710 --> 00:32:20,915 AUDIENCE: [? Sorry ?] [? to ?] [? interrupt, but I ?] have 652 00:32:20,915 --> 00:32:21,415 a question-- 653 00:32:21,415 --> 00:32:22,082 PROFESSOR: Yeah. 654 00:32:22,082 --> 00:32:23,150 AUDIENCE: --the Gaussian. 655 00:32:23,150 --> 00:32:23,600 PROFESSOR: [? About ?] [INAUDIBLE]?? 656 00:32:23,600 --> 00:32:24,800 AUDIENCE: [INAUDIBLE]. 657 00:32:24,800 --> 00:32:24,970 PROFESSOR: Yeah. 658 00:32:24,970 --> 00:32:26,922 AUDIENCE: Is that true when the two Gaussians 659 00:32:26,922 --> 00:32:28,570 have different variances? 660 00:32:28,570 --> 00:32:29,770 PROFESSOR: No. 661 00:32:29,770 --> 00:32:32,140 OK, I'm assuming-- OK, the question-- the comment 662 00:32:32,140 --> 00:32:35,980 was that this rule of the threshold being the optimum 663 00:32:35,980 --> 00:32:38,650 is not necessarily true if the Gaussians have 664 00:32:38,650 --> 00:32:41,980 unequal variances. 665 00:32:41,980 --> 00:32:43,960 But I'm assuming IID noise. 666 00:32:43,960 --> 00:32:46,300 I'm assuming Independent Identically Distributed noise. 667 00:32:46,300 --> 00:32:49,000 So the noise samples are governed by the same Gaussian 668 00:32:49,000 --> 00:32:51,130 right through, and then this turns out 669 00:32:51,130 --> 00:32:52,840 to be the optimum rule. 670 00:32:52,840 --> 00:32:56,110 Thanks for catching that. 671 00:32:56,110 --> 00:33:00,910 So you can imagine the picture with-- 672 00:33:00,910 --> 00:33:03,670 suppose the noise is very sharply 673 00:33:03,670 --> 00:33:10,270 peaked for one of these cases and very shallow 674 00:33:10,270 --> 00:33:11,230 for the other one. 675 00:33:11,230 --> 00:33:15,040 So there's high variance for the 1's and there's low variance 676 00:33:15,040 --> 00:33:16,090 for the 0's. 677 00:33:16,090 --> 00:33:20,170 You might then anticipate that if you got a signal way over 678 00:33:20,170 --> 00:33:23,440 to the left here, you're going to call it a 1, not a 0. 679 00:33:23,440 --> 00:33:27,220 So each case needs to be dealt with separately. 680 00:33:27,220 --> 00:33:30,250 But assuming these are equal variance, which 681 00:33:30,250 --> 00:33:33,870 goes with the IID case, this is the optimum rule. 682 00:33:33,870 --> 00:33:35,800 OK. 683 00:33:35,800 --> 00:33:37,810 So let me just step through this. 684 00:33:37,810 --> 00:33:40,150 What we're saying now is that what's 685 00:33:40,150 --> 00:33:41,613 the probability of making an error? 686 00:33:41,613 --> 00:33:43,780 Well, let me actually write down an expression here. 687 00:33:50,050 --> 00:33:53,970 So the probability of an error-- 688 00:33:53,970 --> 00:33:56,130 this is the general expression. 689 00:33:56,130 --> 00:33:59,460 It's the probability that I send a 0-- 690 00:33:59,460 --> 00:34:04,170 let me just say that this is the probability of sending 691 00:34:04,170 --> 00:34:16,210 0 times the probability of declaring 1 692 00:34:16,210 --> 00:34:17,449 given that 0 was sent. 693 00:34:20,905 --> 00:34:22,530 And then there's the other possibility. 694 00:34:22,530 --> 00:34:26,449 The probability that I sent a 1, and here's 695 00:34:26,449 --> 00:34:32,170 the probability of declaring a 0 given 1 was sent. 696 00:34:35,517 --> 00:34:37,100 So it turns out these are the only two 697 00:34:37,100 --> 00:34:40,933 ways you can make an error, and these are mutually exclusive, 698 00:34:40,933 --> 00:34:42,350 and so what you're doing is adding 699 00:34:42,350 --> 00:34:45,199 the probabilities of the two ways of making an error. 700 00:34:45,199 --> 00:34:48,170 You can either have a 0 sent, and then the question 701 00:34:48,170 --> 00:34:51,770 is, what's the probability of declaring a 1 if a 0 was sent? 702 00:34:51,770 --> 00:34:53,540 And then you have the corresponding term 703 00:34:53,540 --> 00:34:56,560 on the other side. 704 00:34:56,560 --> 00:34:58,660 If P0 equals P1-- 705 00:34:58,660 --> 00:35:01,090 in other words, if both of them are 0.5, 706 00:35:01,090 --> 00:35:04,720 this is going to be 1 minus P0. 707 00:35:04,720 --> 00:35:07,780 If they're both 0.5, then you can pull that out, 708 00:35:07,780 --> 00:35:10,720 and what you're looking at for the probability of error 709 00:35:10,720 --> 00:35:15,982 is just the sum of the areas under these two tails. 710 00:35:15,982 --> 00:35:17,440 Oh sorry, not the sum of the areas. 711 00:35:17,440 --> 00:35:20,760 If these are both 0.5, you pull out 0.5-- 712 00:35:20,760 --> 00:35:21,260 yeah. 713 00:35:21,260 --> 00:35:24,260 It's the sum of those two areas. 714 00:35:24,260 --> 00:35:24,760 OK. 715 00:35:24,760 --> 00:35:28,170 So 0.5 times the sum of those two areas. 716 00:35:28,170 --> 00:35:32,670 Well in the symmetric case, these two areas are the same. 717 00:35:32,670 --> 00:35:36,310 The area to the right of this threshold under the Gaussian 718 00:35:36,310 --> 00:35:37,990 here is the probability of declaring 719 00:35:37,990 --> 00:35:40,540 a 1 given that a 0 was sent. 720 00:35:40,540 --> 00:35:42,010 The area under the tail to the left 721 00:35:42,010 --> 00:35:43,510 here is the probability of declaring 722 00:35:43,510 --> 00:35:45,580 a 0 given that a 1 was sent. 723 00:35:45,580 --> 00:35:47,750 Those two areas are the same. 724 00:35:47,750 --> 00:35:51,070 So you'll discover that the probability of error 725 00:35:51,070 --> 00:35:53,715 is just the area under one of these tails. 726 00:35:53,715 --> 00:35:55,340 Just the area under one of those tails. 727 00:35:55,340 --> 00:35:56,757 So that's all you have to compute. 728 00:35:59,290 --> 00:36:01,040 So how do we do that? 729 00:36:01,040 --> 00:36:03,820 Well, as the area under a Gaussian. 730 00:36:03,820 --> 00:36:05,920 We write down the Gaussian. 731 00:36:05,920 --> 00:36:09,340 Let's pretend that this was 0 and this was Vp. 732 00:36:09,340 --> 00:36:12,430 It doesn't make a difference as far as the computation of areas 733 00:36:12,430 --> 00:36:16,280 goes, but it makes the expressions easier to write. 734 00:36:16,280 --> 00:36:23,580 So I'm saying that the area under the table 735 00:36:23,580 --> 00:36:36,810 here is equal to the area under the tail there. 736 00:36:36,810 --> 00:36:38,010 I can do it either way. 737 00:36:38,010 --> 00:36:41,370 I can either center the Gaussian at minus Vp and look 738 00:36:41,370 --> 00:36:44,730 at the area to the right of 0, or I can center the Gaussian at 739 00:36:44,730 --> 00:36:48,138 0 and look at the area to the right of Vp. 740 00:36:48,138 --> 00:36:49,930 And the way the expression is written here, 741 00:36:49,930 --> 00:36:52,900 it chooses to do it the second way. 742 00:36:52,900 --> 00:36:56,790 So what we're saying is, here is the Gaussian. 743 00:36:56,790 --> 00:36:58,260 It's centered at 0, so I don't have 744 00:36:58,260 --> 00:37:02,130 to subtract any term off that term in the numerator. 745 00:37:02,130 --> 00:37:04,940 Here's the 2 sigma squared in the denominator. 746 00:37:04,940 --> 00:37:07,860 And I integrate it from Vp onwards. 747 00:37:07,860 --> 00:37:09,320 There's this notation introduced. 748 00:37:09,320 --> 00:37:11,640 Vp is square root of ES. 749 00:37:11,640 --> 00:37:13,590 The reason is that we're thinking 750 00:37:13,590 --> 00:37:17,970 in terms of the energy of a single sample-- 751 00:37:17,970 --> 00:37:19,560 or the power of a single sample, they 752 00:37:19,560 --> 00:37:21,518 turn out to be the same thing because it's just 753 00:37:21,518 --> 00:37:22,300 a single sample. 754 00:37:22,300 --> 00:37:25,060 So it's just the notation that's traditionally used. 755 00:37:25,060 --> 00:37:27,810 But what we're talking about as Vp there. 756 00:37:27,810 --> 00:37:32,250 So the area from Vp to infinity under the Gaussian 757 00:37:32,250 --> 00:37:36,430 with the normalization factor here. 758 00:37:36,430 --> 00:37:39,460 Now this is not an integral you can evaluate in closed form. 759 00:37:39,460 --> 00:37:41,530 It is a tabulated integral. 760 00:37:41,530 --> 00:37:44,080 Tabulated most conveniently in terms of something 761 00:37:44,080 --> 00:37:45,708 called the error function. 762 00:37:45,708 --> 00:37:47,500 And so when you work through the calculus-- 763 00:37:47,500 --> 00:37:49,417 and I won't show it to here, it's in the book, 764 00:37:49,417 --> 00:37:51,280 you might do it in recitation, you 765 00:37:51,280 --> 00:37:53,410 discover that the probability of error 766 00:37:53,410 --> 00:37:59,080 is this error function of the square root of ES over N0. 767 00:37:59,080 --> 00:38:01,990 N0's notation for 2 sigma squared. 768 00:38:01,990 --> 00:38:04,240 If I translate that back to notation we've been using, 769 00:38:04,240 --> 00:38:07,360 it's just Vp over sigma. 770 00:38:07,360 --> 00:38:09,670 So the error performance, the probability of error 771 00:38:09,670 --> 00:38:13,330 is a function of the ratio of the peak amplitude 772 00:38:13,330 --> 00:38:17,710 on the signal to the standard deviation of the noise. 773 00:38:17,710 --> 00:38:20,730 That's sort of the square root of the SNR. 774 00:38:20,730 --> 00:38:23,560 The SNR would be square of the amplitude 775 00:38:23,560 --> 00:38:26,450 to square of the standard deviation. 776 00:38:26,450 --> 00:38:29,110 So this is the square root of the SNR. 777 00:38:29,110 --> 00:38:32,160 And what does this function look like? 778 00:38:32,160 --> 00:38:35,440 We can plot it. 779 00:38:35,440 --> 00:38:39,730 So that's exactly that computation. 780 00:38:39,730 --> 00:38:43,270 This is a simulation on the theory overlaid on each other, 781 00:38:43,270 --> 00:38:45,100 but we have 0.5. 782 00:38:45,100 --> 00:38:47,950 This function is called the complementary error function. 783 00:38:47,950 --> 00:38:52,250 The C is for complementary, erf is for error function, 784 00:38:52,250 --> 00:38:54,700 and here's the square root of ES over N0 785 00:38:54,700 --> 00:38:57,220 which we had in the previous expression. 786 00:38:57,220 --> 00:39:01,120 So you're really thinking of signal-to-noise ratio 787 00:39:01,120 --> 00:39:06,850 along this axis in dB and the probability of error 788 00:39:06,850 --> 00:39:09,250 on a logarithmic scale down here. 789 00:39:09,250 --> 00:39:12,130 So as the signal-to-noise ratio increases, 790 00:39:12,130 --> 00:39:15,340 as a signal becomes more powerful relative to the noise, 791 00:39:15,340 --> 00:39:18,670 the probability of error decreases. 792 00:39:18,670 --> 00:39:21,080 Visually what's going on? 793 00:39:21,080 --> 00:39:22,330 Let's go back to this picture. 794 00:39:25,680 --> 00:39:29,175 When the noise decreases relative to the signal, what's 795 00:39:29,175 --> 00:39:31,050 happening is that these Gaussians are getting 796 00:39:31,050 --> 00:39:33,250 more peaked and they're pulling in more tightly, 797 00:39:33,250 --> 00:39:36,300 and so there's less chance of confusing the two cases. 798 00:39:36,300 --> 00:39:37,920 So it's as simple as that. 799 00:39:37,920 --> 00:39:41,730 It's the separation between these two levels divided 800 00:39:41,730 --> 00:39:44,130 by standard deviation of the noise that's really 801 00:39:44,130 --> 00:39:45,780 going to determine performance. 802 00:39:45,780 --> 00:39:48,070 How far apart are these two cases 803 00:39:48,070 --> 00:39:50,070 relative to the standard deviation of the noise? 804 00:39:50,070 --> 00:39:53,557 That's the square root of the signal-to-noise ratio. 805 00:39:53,557 --> 00:39:55,140 That's what determines the probability 806 00:39:55,140 --> 00:39:56,310 of error in this case. 807 00:39:59,500 --> 00:40:01,290 OK. 808 00:40:01,290 --> 00:40:07,140 So are we done or could we be doing better? 809 00:40:07,140 --> 00:40:10,980 If you think of what we did, we looked at the samples 810 00:40:10,980 --> 00:40:13,800 in a bit slice, in a bit slot. 811 00:40:13,800 --> 00:40:16,170 We took one of those samples and we carried out 812 00:40:16,170 --> 00:40:18,030 this decision rule on it. 813 00:40:18,030 --> 00:40:20,920 Could we be doing better than that? 814 00:40:20,920 --> 00:40:21,442 Yeah? 815 00:40:21,442 --> 00:40:23,234 AUDIENCE: We could look at one more sample? 816 00:40:23,234 --> 00:40:25,290 PROFESSOR: We could look at more than one sample. 817 00:40:25,290 --> 00:40:26,700 This was a little bit arbitrary. 818 00:40:26,700 --> 00:40:28,320 It was conservative. 819 00:40:28,320 --> 00:40:32,640 Why you often do that is because the number of samples in a bit 820 00:40:32,640 --> 00:40:35,010 slot is small and you don't want to get near the edges 821 00:40:35,010 --> 00:40:37,380 because you're little worried about the transience. 822 00:40:37,380 --> 00:40:39,750 You've got a long enough-- 823 00:40:39,750 --> 00:40:43,020 if you've got enough samples in a bit slot and the transience 824 00:40:43,020 --> 00:40:45,120 have died out, then maybe you can just pick out 825 00:40:45,120 --> 00:40:47,230 a bigger chunk in the middle. 826 00:40:47,230 --> 00:40:51,320 And so that's what we're going to think to do here. 827 00:40:51,320 --> 00:40:53,370 OK. 828 00:40:53,370 --> 00:40:56,470 So it's the same setting, but we're 829 00:40:56,470 --> 00:40:59,290 going to average M samples. 830 00:40:59,290 --> 00:41:03,290 We've got L samples per bit. 831 00:41:03,290 --> 00:41:05,440 We may not be confident capturing all of those 832 00:41:05,440 --> 00:41:08,060 were averaging because there's some stuff at the edges, 833 00:41:08,060 --> 00:41:09,340 so let's pick M of them. 834 00:41:09,340 --> 00:41:16,795 Maybe less than L. Take M of them and compute the average. 835 00:41:16,795 --> 00:41:18,670 And I'm doing this just for one of the cases. 836 00:41:18,670 --> 00:41:22,520 You'd have to do the same thing for the minus Vp case. 837 00:41:22,520 --> 00:41:26,200 So the question is, what does the average do? 838 00:41:26,200 --> 00:41:27,700 So why did you want to average them? 839 00:41:27,700 --> 00:41:29,034 What was your intuition? 840 00:41:29,034 --> 00:41:32,497 AUDIENCE: Because that would-- it [? would be ?] [INAUDIBLE].. 841 00:41:32,497 --> 00:41:33,080 PROFESSOR: OK. 842 00:41:33,080 --> 00:41:34,430 So here's the key thing. 843 00:41:34,430 --> 00:41:38,000 If you've got independent noise samples and you average them, 844 00:41:38,000 --> 00:41:39,730 you're going to decrease the variance. 845 00:41:39,730 --> 00:41:42,560 If you've got M independent noise samples from an IID 846 00:41:42,560 --> 00:41:45,740 process, you decrease the variance by M. 847 00:41:45,740 --> 00:41:50,180 This doesn't hold if the noise samples are not independent. 848 00:41:50,180 --> 00:41:54,050 In fact, if one noise sample equals the other, then 849 00:41:54,050 --> 00:41:55,820 when you add the two, you get something 850 00:41:55,820 --> 00:41:59,960 whose variances is four times rather than just twice. 851 00:41:59,960 --> 00:42:02,330 So it's critical that these be independent. 852 00:42:02,330 --> 00:42:05,960 So if we've got independent samples-- 853 00:42:05,960 --> 00:42:07,970 independent noise samples from one sample 854 00:42:07,970 --> 00:42:11,090 to the next and we average them-- well, let's 855 00:42:11,090 --> 00:42:12,920 just average both sides of this equation. 856 00:42:12,920 --> 00:42:14,990 We've got the average of Y going to be 857 00:42:14,990 --> 00:42:16,790 the average of these values, which is just 858 00:42:16,790 --> 00:42:20,090 going to be Vp again because it's constant at Vp, 859 00:42:20,090 --> 00:42:22,112 plus the average of W. 860 00:42:22,112 --> 00:42:23,570 Here's the other interesting thing. 861 00:42:23,570 --> 00:42:25,070 We're not going to try proving this, 862 00:42:25,070 --> 00:42:29,810 but it turns out that the average 863 00:42:29,810 --> 00:42:33,107 of a sum of independent Gaussians is, again, Gaussian. 864 00:42:33,107 --> 00:42:35,440 You might believe that if you think of the central limit 865 00:42:35,440 --> 00:42:35,860 theorem. 866 00:42:35,860 --> 00:42:37,360 You think of each of these Gaussians 867 00:42:37,360 --> 00:42:40,220 being approximated by sums of random variables. 868 00:42:40,220 --> 00:42:41,860 So the sum of these Gaussians is then 869 00:42:41,860 --> 00:42:43,960 a sum of just a larger number of random variables 870 00:42:43,960 --> 00:42:45,790 that should still be Gaussian. 871 00:42:45,790 --> 00:42:48,030 So the sum of an independent set of Gaussians 872 00:42:48,030 --> 00:42:50,530 is, again, Gaussian. 873 00:42:50,530 --> 00:42:53,710 So all I need to know for this average W 874 00:42:53,710 --> 00:42:55,540 since it's Gaussian is what is its mean 875 00:42:55,540 --> 00:42:57,310 and what is its variance? 876 00:42:57,310 --> 00:42:59,273 It turns out if you add up a bunch of zero 877 00:42:59,273 --> 00:43:00,940 mean random variables, you get something 878 00:43:00,940 --> 00:43:03,200 with zero mean, no surprise. 879 00:43:03,200 --> 00:43:05,200 And if you add-- 880 00:43:05,200 --> 00:43:08,020 if you take the average, then the variance actually 881 00:43:08,020 --> 00:43:12,353 drops by that factor M. 882 00:43:12,353 --> 00:43:13,770 So what you're going to do is take 883 00:43:13,770 --> 00:43:16,770 the average of the signal, average of the noise. 884 00:43:16,770 --> 00:43:18,995 That shrinks the noise component. 885 00:43:18,995 --> 00:43:20,370 You have the same kind of picture 886 00:43:20,370 --> 00:43:22,650 but now with a higher signal-to-noise ratio. 887 00:43:22,650 --> 00:43:26,640 Now what you've got in the numerator instead of ES 888 00:43:26,640 --> 00:43:29,240 is EB, which is M times ES. 889 00:43:29,240 --> 00:43:31,290 You're summing the energies of all the samples 890 00:43:31,290 --> 00:43:32,550 that you've taken. 891 00:43:32,550 --> 00:43:35,450 And that's what we refer to as EB, it's the energy of the bit. 892 00:43:40,730 --> 00:43:41,230 All right. 893 00:43:41,230 --> 00:43:46,390 It turns out that that has all sorts of implications. 894 00:43:46,390 --> 00:43:48,292 You certainly want to be averaging 895 00:43:48,292 --> 00:43:50,500 if you've got this kind of setting, because otherwise 896 00:43:50,500 --> 00:43:53,590 you're leaving all these samples on the table 897 00:43:53,590 --> 00:43:55,160 and not making good use of them. 898 00:43:55,160 --> 00:43:57,670 So if you're really getting ambitious, 899 00:43:57,670 --> 00:43:59,590 you really want to be extracting all of that. 900 00:44:03,280 --> 00:44:06,150 Also, if you want to maintain the same error performance 901 00:44:06,150 --> 00:44:08,248 and the noise intensity increases, 902 00:44:08,248 --> 00:44:10,540 then you're going to want to have more samples per bit. 903 00:44:10,540 --> 00:44:12,415 You may want to slow down your signaling rate 904 00:44:12,415 --> 00:44:14,230 so you can put more samples per bit. 905 00:44:14,230 --> 00:44:18,460 It turns out in the deep space probe examples 906 00:44:18,460 --> 00:44:20,830 that we've been talking about, that's 907 00:44:20,830 --> 00:44:22,130 exactly what's happening. 908 00:44:22,130 --> 00:44:25,420 If you look at Voyager 2, it was transmitting 909 00:44:25,420 --> 00:44:28,547 at 115 kilobits in 1979. 910 00:44:28,547 --> 00:44:30,130 That's the year, I joined the faculty, 911 00:44:30,130 --> 00:44:32,680 that's a long time ago. 912 00:44:32,680 --> 00:44:34,270 That was near Jupiter. 913 00:44:34,270 --> 00:44:40,492 Last month-- I mean, it's gone past Jupiter, Saturn. 914 00:44:40,492 --> 00:44:41,950 The other planet I only like to say 915 00:44:41,950 --> 00:44:44,408 the Greek name of because it comes out wrong when I say it. 916 00:44:44,408 --> 00:44:45,550 It's Ouranos. 917 00:44:45,550 --> 00:44:48,590 And then Neptune. 918 00:44:48,590 --> 00:44:51,340 So it went past all of these. 919 00:44:51,340 --> 00:44:53,200 And now it's about 9 billion miles away. 920 00:44:53,200 --> 00:44:55,817 It's twice as far away from the sun as Pluto is. 921 00:44:55,817 --> 00:44:57,400 But look at the transmission rate now. 922 00:44:57,400 --> 00:45:00,320 It's 160 bits per second. 923 00:45:00,320 --> 00:45:01,480 So it's greatly reduced. 924 00:45:01,480 --> 00:45:07,720 And the reason is that over this extended interval, 925 00:45:07,720 --> 00:45:10,150 the energy per sample that arrives at Earth 926 00:45:10,150 --> 00:45:11,530 is just minuscule. 927 00:45:11,530 --> 00:45:14,850 I mean, it was small enough to begin with from Jupiter 928 00:45:14,850 --> 00:45:16,660 and look at what it does now. 929 00:45:16,660 --> 00:45:19,600 So it's about 1,000 times less in power 930 00:45:19,600 --> 00:45:23,290 and you've gone down 1,000 times less more or less 931 00:45:23,290 --> 00:45:26,110 in your signaling rate because you're 932 00:45:26,110 --> 00:45:30,640 trying to put that much more time in the signal. 933 00:45:30,640 --> 00:45:32,650 So these trade-offs are driven by trying 934 00:45:32,650 --> 00:45:37,450 to get the same energy per bit for a given noise 935 00:45:37,450 --> 00:45:39,147 to maintain the performance. 936 00:45:41,770 --> 00:45:43,520 As I was reading up on this, there 937 00:45:43,520 --> 00:45:46,875 were little references to things that went wrong. 938 00:45:46,875 --> 00:45:48,250 The only a handful of things that 939 00:45:48,250 --> 00:45:50,333 are listed as having gone wrong, but they turn out 940 00:45:50,333 --> 00:45:53,200 to be related to decoding. 941 00:45:53,200 --> 00:45:57,610 So there was a command that was incorrectly decoded and kept 942 00:45:57,610 --> 00:46:01,450 some heaters on for very long and caused some malfunction. 943 00:46:01,450 --> 00:46:03,680 Here was a flipped bit. 944 00:46:03,680 --> 00:46:04,960 This is one of only-- 945 00:46:04,960 --> 00:46:06,970 these are a few of only a small list 946 00:46:06,970 --> 00:46:10,210 of things that are listed as having gone wrong. 947 00:46:10,210 --> 00:46:14,530 But a flipped bit here caused a problem. 948 00:46:14,530 --> 00:46:17,230 You've got very few bits in these computers to begin with. 949 00:46:17,230 --> 00:46:19,040 Remember the numbers we had last time. 950 00:46:19,040 --> 00:46:22,780 So a flipped bit can cause trouble. 951 00:46:22,780 --> 00:46:24,160 OK. 952 00:46:24,160 --> 00:46:26,245 Let's do one last piece here. 953 00:46:29,380 --> 00:46:33,040 We're going to try and be even less conservative. 954 00:46:33,040 --> 00:46:45,570 So suppose I know that when a 1 is sent, 955 00:46:45,570 --> 00:46:48,610 what I receive is a waveform of a particular type. 956 00:46:48,610 --> 00:46:52,140 So the piece of the response corresponding to this 957 00:46:52,140 --> 00:46:53,400 has some particular shape. 958 00:46:53,400 --> 00:46:54,270 Suppose I know that. 959 00:46:58,860 --> 00:46:59,360 OK. 960 00:46:59,360 --> 00:47:00,760 So nothing is constant here. 961 00:47:00,760 --> 00:47:04,690 This is the actual y of n sequence. 962 00:47:04,690 --> 00:47:06,340 And then to this, I'm adding noise. 963 00:47:11,867 --> 00:47:12,700 So here's the thing. 964 00:47:12,700 --> 00:47:17,650 I've got a yn which is no longer just a constant plus noise, 965 00:47:17,650 --> 00:47:20,250 it's some known profile plus noise. 966 00:47:20,250 --> 00:47:21,825 That known profile is actually what 967 00:47:21,825 --> 00:47:23,200 the xn is going to look like when 968 00:47:23,200 --> 00:47:24,550 it goes through the channel. 969 00:47:24,550 --> 00:47:26,350 I should perhaps have called it y0 of n, 970 00:47:26,350 --> 00:47:29,680 but let's stick to x0 of n. 971 00:47:29,680 --> 00:47:33,730 So x0 of n is known, and we've got the noise. 972 00:47:33,730 --> 00:47:37,510 The question is, do you want to just be averaging or do 973 00:47:37,510 --> 00:47:40,110 you want to try something else? 974 00:47:40,110 --> 00:47:44,620 If I've got this kind of signal received 975 00:47:44,620 --> 00:47:48,858 and I've got the same amount of noise added to each sample, 976 00:47:48,858 --> 00:47:50,650 which of these samples is more trustworthy? 977 00:47:50,650 --> 00:47:52,317 Which sample do you want to weight more? 978 00:47:55,850 --> 00:47:59,660 I've got some amount of noise adding 979 00:47:59,660 --> 00:48:01,460 into all of these samples, so there's 980 00:48:01,460 --> 00:48:04,520 some standard deviations' worth on each of these. 981 00:48:07,890 --> 00:48:11,600 Which is the most trustworthy sample here? 982 00:48:11,600 --> 00:48:12,318 Yeah? 983 00:48:12,318 --> 00:48:13,610 AUDIENCE: The one on the right? 984 00:48:13,610 --> 00:48:14,030 PROFESSOR: Yeah. 985 00:48:14,030 --> 00:48:15,290 It's the one on the right because it's 986 00:48:15,290 --> 00:48:16,400 got the largest amplitude. 987 00:48:16,400 --> 00:48:19,170 By itself it has the largest signal-to-noise ratio. 988 00:48:19,170 --> 00:48:21,260 So if you're going to combine these samples, 989 00:48:21,260 --> 00:48:22,718 you would think that you would want 990 00:48:22,718 --> 00:48:25,920 to put more weight on the sample that was larger. 991 00:48:25,920 --> 00:48:28,020 So you can actually formulate that analytically. 992 00:48:28,020 --> 00:48:30,860 So we're going to combine the received samples 993 00:48:30,860 --> 00:48:33,115 with some set of weights an. 994 00:48:33,115 --> 00:48:35,240 Here's what it's going to do on the right-hand side 995 00:48:35,240 --> 00:48:37,290 of that equation. 996 00:48:37,290 --> 00:48:39,540 Again, when you take a weighted combination 997 00:48:39,540 --> 00:48:44,370 of zero mean Gaussians, as you get a zero mean Gaussian. 998 00:48:44,370 --> 00:48:48,180 So all you need to know is what's the variance of a scaled 999 00:48:48,180 --> 00:48:49,650 Gaussian? 1000 00:48:49,650 --> 00:48:50,370 So let's see. 1001 00:48:50,370 --> 00:48:58,680 If I have a wn having variance sigma squared, 1002 00:48:58,680 --> 00:49:01,110 what do you think is the variance of 3 times wn? 1003 00:49:08,930 --> 00:49:11,660 3 times wn means the excursions are scaled by 3, 1004 00:49:11,660 --> 00:49:13,810 so what's the variance? 1005 00:49:13,810 --> 00:49:14,310 9. 1006 00:49:20,080 --> 00:49:23,260 So scaling by a particular number 1007 00:49:23,260 --> 00:49:26,510 scales the variance by the square of that number. 1008 00:49:26,510 --> 00:49:28,660 So the Gaussian you're adding in here 1009 00:49:28,660 --> 00:49:32,050 has a variance which is sigma squared 1010 00:49:32,050 --> 00:49:34,855 times the sum of the W squared. 1011 00:49:34,855 --> 00:49:35,355 Sorry. 1012 00:49:35,355 --> 00:49:38,320 The sigma squared times to sum of the A squareds. 1013 00:49:38,320 --> 00:49:40,840 That's what the variance of the Gaussian is. 1014 00:49:40,840 --> 00:49:44,950 So you can actually write a very simple optimization problem. 1015 00:49:44,950 --> 00:49:49,130 What choice of weights maximizes the signal-to-noise ratio? 1016 00:49:49,130 --> 00:49:50,890 And you discover, indeed, exactly 1017 00:49:50,890 --> 00:49:55,680 that you're going to put the largest weight on the largest 1018 00:49:55,680 --> 00:49:57,510 sample. 1019 00:49:57,510 --> 00:50:00,250 And when you do that, the resulting signal-to-noise ratio 1020 00:50:00,250 --> 00:50:04,540 is, again, energy of the signal that was transmitted 1021 00:50:04,540 --> 00:50:05,780 divided by the variance. 1022 00:50:05,780 --> 00:50:09,580 So if you do the optimum processing with this so-called 1023 00:50:09,580 --> 00:50:13,630 matched filtering, you're going to get to energy 1024 00:50:13,630 --> 00:50:14,800 of the sample-- 1025 00:50:14,800 --> 00:50:17,680 sorry, energy of the bit over the noise 1026 00:50:17,680 --> 00:50:19,590 variance governing the performance. 1027 00:50:19,590 --> 00:50:22,420 So it's the bit energy over the noise variance 1028 00:50:22,420 --> 00:50:24,220 that's going to determine performance 1029 00:50:24,220 --> 00:50:27,700 provided you milk that bit slot for everything it's 1030 00:50:27,700 --> 00:50:29,660 worth by doing the match filtering. 1031 00:50:29,660 --> 00:50:30,160 OK. 1032 00:50:30,160 --> 00:50:32,430 We'll leave it at that for today.