1 00:00:00,360 --> 00:00:03,330 PROFESSOR: So many of you have signed up this term. 2 00:00:03,330 --> 00:00:05,940 As most of you know, this course is now on an alternate 3 00:00:05,940 --> 00:00:08,460 year basis, but it really depends on how much interest 4 00:00:08,460 --> 00:00:09,620 there is in it. 5 00:00:09,620 --> 00:00:14,780 So this is certainly very satisfactory interest. 6 00:00:14,780 --> 00:00:19,420 This morning I'm just going to go through the information 7 00:00:19,420 --> 00:00:22,020 sheet logistics a little, tell you a little bit about the 8 00:00:22,020 --> 00:00:28,440 course, and then go through a very rapid path through the 9 00:00:28,440 --> 00:00:30,730 first three chapters. 10 00:00:30,730 --> 00:00:35,180 So hang onto your seats. 11 00:00:35,180 --> 00:00:39,830 But first of all, the information. 12 00:00:39,830 --> 00:00:44,530 One thing I did not mention in the information sheet is that 13 00:00:44,530 --> 00:00:47,290 due to the sponsorship of OpenCourseWare and the 14 00:00:47,290 --> 00:00:52,360 department, we are going to have this class on TV or 15 00:00:52,360 --> 00:00:57,100 videotaped for OCW in the department archives. 16 00:00:57,100 --> 00:00:59,970 And I hope that's OK with everybody. 17 00:00:59,970 --> 00:01:03,020 I think at some point we'll send out a permission sheet. 18 00:01:03,020 --> 00:01:06,940 Does anyone anticipate having any problem with the camera 19 00:01:06,940 --> 00:01:11,580 lingering on them or the back of them from time to time? 20 00:01:11,580 --> 00:01:13,230 I don't think this is going to be an issue. 21 00:01:13,230 --> 00:01:16,940 But if it is, either mention it now or come 22 00:01:16,940 --> 00:01:18,500 up and see me privately. 23 00:01:18,500 --> 00:01:20,600 As I say, I think eventually you'll have to 24 00:01:20,600 --> 00:01:22,400 sign permission sheets. 25 00:01:22,400 --> 00:01:24,790 Our cameraman is Tom White. 26 00:01:24,790 --> 00:01:26,170 Let's give a hand to Tom. 27 00:01:29,930 --> 00:01:34,760 And he promises to be as unobtrusive as possible. 28 00:01:34,760 --> 00:01:35,110 OK. 29 00:01:35,110 --> 00:01:36,750 There are four handouts this morning. 30 00:01:36,750 --> 00:01:39,010 I hope you have them all. 31 00:01:39,010 --> 00:01:40,750 First of all, the information sheet. 32 00:01:40,750 --> 00:01:45,360 Second, a little personal data sheet that I ask you to each 33 00:01:45,360 --> 00:01:48,660 fill out so we know who's in the class and a 34 00:01:48,660 --> 00:01:50,270 little bit about you. 35 00:01:50,270 --> 00:01:53,840 The first problem set, which is due a week from today. 36 00:01:53,840 --> 00:01:56,940 In general, problem sets will be due on Wednesdays and 37 00:01:56,940 --> 00:01:58,700 handed out on Wednesdays. 38 00:01:58,700 --> 00:02:01,710 And please do pass them in on time. 39 00:02:01,710 --> 00:02:04,160 We won't make any serious attempt to grade them if they 40 00:02:04,160 --> 00:02:07,490 don't come in on time. 41 00:02:07,490 --> 00:02:09,874 I'll mention our philosophy about problem 42 00:02:09,874 --> 00:02:11,680 sets in a little while. 43 00:02:11,680 --> 00:02:15,250 And then chapters one through three, which are all sort of 44 00:02:15,250 --> 00:02:16,890 warm-up chapters. 45 00:02:16,890 --> 00:02:20,740 And problem set one is this kind of warm-up exercises to 46 00:02:20,740 --> 00:02:22,840 get you into context, to get you into the 47 00:02:22,840 --> 00:02:25,456 mood for this course. 48 00:02:25,456 --> 00:02:27,746 It doesn't have to do -- 49 00:02:27,746 --> 00:02:28,480 well, it does. 50 00:02:28,480 --> 00:02:33,230 But at this point, it's more just getting up to speed with 51 00:02:33,230 --> 00:02:37,210 the language and the environment. 52 00:02:37,210 --> 00:02:40,240 The teaching assistant is Ashish Khisti. 53 00:02:40,240 --> 00:02:41,880 Ashish, why don't you stand up just so everybody 54 00:02:41,880 --> 00:02:43,600 is sure to see you. 55 00:02:43,600 --> 00:02:44,950 I'm delighted to have Ashish. 56 00:02:44,950 --> 00:02:47,720 He was a superb student this course. 57 00:02:47,720 --> 00:02:50,230 Certainly understands it very well. 58 00:02:50,230 --> 00:02:52,930 Ashish will even be a distinguished guest lecturer 59 00:02:52,930 --> 00:02:56,510 for lectures two through four, because I will be on vacation 60 00:02:56,510 --> 00:02:57,670 during that time. 61 00:02:57,670 --> 00:03:00,330 So you'll get to know Ashish very well. 62 00:03:00,330 --> 00:03:05,220 I think he will be able to help you quite a bit. 63 00:03:05,220 --> 00:03:08,680 His office hours we've tentatively scheduled for 64 00:03:08,680 --> 00:03:12,900 Tuesday from five to seven in the basement area of Building 65 00:03:12,900 --> 00:03:14,710 32, the Stata Center. 66 00:03:14,710 --> 00:03:18,120 Is that going to be OK with everybody? 67 00:03:18,120 --> 00:03:19,970 Is there anybody who's --? 68 00:03:19,970 --> 00:03:22,180 It's of course set that way because the homeworks are due 69 00:03:22,180 --> 00:03:23,430 on Wednesday. 70 00:03:25,620 --> 00:03:27,300 That's all right? 71 00:03:27,300 --> 00:03:27,830 All right. 72 00:03:27,830 --> 00:03:29,700 We aim to please. 73 00:03:29,700 --> 00:03:34,100 I've already mentioned Tom White. 74 00:03:34,100 --> 00:03:36,470 I want to mention that we've tentatively scheduled the 75 00:03:36,470 --> 00:03:40,300 midterm for Wednesday, March 16, which is the Wednesday 76 00:03:40,300 --> 00:03:44,430 before MIT vacation week. 77 00:03:44,430 --> 00:03:48,510 And normally I've run it for two hours to try to minimize 78 00:03:48,510 --> 00:03:49,860 time pressure. 79 00:03:49,860 --> 00:03:51,880 For 9:30 class, that's easy. 80 00:03:51,880 --> 00:03:54,000 We simply start it at nine. 81 00:03:54,000 --> 00:03:56,690 But again, if anyone has any personal problem with that, 82 00:03:56,690 --> 00:03:57,940 please let me know. 83 00:04:00,880 --> 00:04:02,230 You may not know about that yet. 84 00:04:02,230 --> 00:04:04,873 But I want to let you know immediately. 85 00:04:09,180 --> 00:04:11,410 Prerequisite. 86 00:04:11,410 --> 00:04:16,740 I've held to the policy of enforcing the prerequisite, 87 00:04:16,740 --> 00:04:19,860 which is 6.450, rigorously. 88 00:04:19,860 --> 00:04:24,070 Not so much because the content is 89 00:04:24,070 --> 00:04:24,990 really what's needed. 90 00:04:24,990 --> 00:04:28,370 This course is entitled Principles of Digital 91 00:04:28,370 --> 00:04:30,060 Communications two. 92 00:04:30,060 --> 00:04:35,200 But what it really is is a course on coding, modern 93 00:04:35,200 --> 00:04:39,940 coding, in a communications wrapper. 94 00:04:39,940 --> 00:04:43,555 Our motivation all along will be how to get to capacity on 95 00:04:43,555 --> 00:04:46,380 the additive white Gaussian noise channel. 96 00:04:46,380 --> 00:04:50,580 Within that story, we're going to develop all the principal 97 00:04:50,580 --> 00:04:52,620 classes of codes that you would see 98 00:04:52,620 --> 00:04:53,780 in any coding course. 99 00:04:53,780 --> 00:04:56,820 But we're always going to be measuring them by how well 100 00:04:56,820 --> 00:05:00,397 they do on a particular communications channel, the 101 00:05:00,397 --> 00:05:04,130 canonical additive white Gaussian noise channel. 102 00:05:04,130 --> 00:05:07,540 But in fact, we're not going to use very much of what you 103 00:05:07,540 --> 00:05:08,910 had in 6.450. 104 00:05:08,910 --> 00:05:13,150 The point of having taken 6.450 is simply to be up to a 105 00:05:13,150 --> 00:05:16,910 certain level of speed and sophistication in dealing with 106 00:05:16,910 --> 00:05:19,270 rather mathematical concepts. 107 00:05:19,270 --> 00:05:23,320 As you know, 450 is rather mathematical. 108 00:05:23,320 --> 00:05:26,210 Most of the Area 1 courses are rather mathematical. 109 00:05:26,210 --> 00:05:27,770 This one is rather mathematical. 110 00:05:27,770 --> 00:05:33,510 And if you simply aren't used to making logical arguments 111 00:05:33,510 --> 00:05:36,130 fairly quickly and in a snap, you're going to have trouble 112 00:05:36,130 --> 00:05:37,330 with this course. 113 00:05:37,330 --> 00:05:40,660 So that's really the reason for a prerequisite. 114 00:05:40,660 --> 00:05:46,890 And I have found it's important to enforce it, 115 00:05:46,890 --> 00:05:50,430 because we always get, each year, a couple of people who 116 00:05:50,430 --> 00:05:52,590 probably shouldn't be here. 117 00:05:52,590 --> 00:05:56,960 So is there anyone here who hasn't taken a 118 00:05:56,960 --> 00:05:58,435 version of 450 before? 119 00:06:01,770 --> 00:06:02,840 Yes? 120 00:06:02,840 --> 00:06:03,460 What's your name? 121 00:06:03,460 --> 00:06:04,710 AUDIENCE: Mesrob. 122 00:06:07,658 --> 00:06:09,900 I emailed you. 123 00:06:09,900 --> 00:06:10,310 PROFESSOR: Oh. 124 00:06:10,310 --> 00:06:11,690 So you emailed me -- 125 00:06:11,690 --> 00:06:12,650 Mesrob? 126 00:06:12,650 --> 00:06:13,710 OK. 127 00:06:13,710 --> 00:06:19,300 And you had taken a couple of other very rigorous courses, 128 00:06:19,300 --> 00:06:21,215 and so I'm happy with your prerequisites. 129 00:06:24,410 --> 00:06:33,100 The registrar told me that Mukul Agarwal and Daniel Wang, 130 00:06:33,100 --> 00:06:40,200 that's you, and Andrew Cross have not had the 131 00:06:40,200 --> 00:06:42,670 prerequisites. 132 00:06:42,670 --> 00:06:44,700 Daniel? 133 00:06:44,700 --> 00:06:46,910 What's the story? 134 00:06:46,910 --> 00:06:49,950 AUDIENCE: I audited [INAUDIBLE]. 135 00:06:49,950 --> 00:06:51,640 PROFESSOR: You audited 450? 136 00:06:51,640 --> 00:06:52,440 OK. 137 00:06:52,440 --> 00:06:55,980 Perhaps we could talk after class about, in general, what 138 00:06:55,980 --> 00:06:59,835 courses you've taken, and why you think you can keep up it. 139 00:06:59,835 --> 00:07:01,320 AUDIENCE: [INAUDIBLE]. 140 00:07:01,320 --> 00:07:03,020 PROFESSOR: OK. 141 00:07:03,020 --> 00:07:06,990 Or just let me know how you prefer to handle it. 142 00:07:06,990 --> 00:07:09,490 We can do that immediately after class in a corner of the 143 00:07:09,490 --> 00:07:12,190 classroom, if you want. 144 00:07:12,190 --> 00:07:13,890 And are the other two people here? 145 00:07:13,890 --> 00:07:15,140 Agarwal? 146 00:07:17,180 --> 00:07:17,830 No? 147 00:07:17,830 --> 00:07:20,670 Cross? 148 00:07:20,670 --> 00:07:24,090 So they seem to have selected themselves out. 149 00:07:24,090 --> 00:07:24,790 All right. 150 00:07:24,790 --> 00:07:25,330 Very good. 151 00:07:25,330 --> 00:07:30,230 And everyone else here has had 450. 152 00:07:30,230 --> 00:07:31,480 All righty. 153 00:07:34,380 --> 00:07:35,630 There's no text. 154 00:07:37,970 --> 00:07:41,660 Unfortunately there's really isn't any text that comes 155 00:07:41,660 --> 00:07:44,580 close to covering all the material we're going to cover. 156 00:07:44,580 --> 00:07:50,560 Many of the individual chapters in this course are 157 00:07:50,560 --> 00:07:53,850 covered by various textbooks, of course, in far greater 158 00:07:53,850 --> 00:07:57,130 depth than I am able to cover them here. 159 00:07:57,130 --> 00:08:00,070 So partly what you're getting from me is a selection of a 160 00:08:00,070 --> 00:08:02,060 thread of what you really need to know, at 161 00:08:02,060 --> 00:08:04,380 least first time through. 162 00:08:04,380 --> 00:08:08,650 And over the years, I've developed course notes of my 163 00:08:08,650 --> 00:08:11,430 own, which I'm now reasonably satisfied with. 164 00:08:11,430 --> 00:08:16,570 At least up through the last chapters, where we'll spend a 165 00:08:16,570 --> 00:08:18,530 little bit more time this year. 166 00:08:18,530 --> 00:08:20,950 And so I hope that that will be satisfactory. 167 00:08:20,950 --> 00:08:24,220 But I do give you some supplementary texts in here, 168 00:08:24,220 --> 00:08:25,970 and more are coming out all the time. 169 00:08:25,970 --> 00:08:29,190 And part of being a graduate student is understanding where 170 00:08:29,190 --> 00:08:32,740 to find supplementary material if you need to. 171 00:08:32,740 --> 00:08:34,770 So let me know if you ever feel -- 172 00:08:34,770 --> 00:08:36,890 if you ever want to know, where can I read more about 173 00:08:36,890 --> 00:08:40,980 something, I'd be happy to give you a suggestion. 174 00:08:40,980 --> 00:08:42,770 Office hours I already mentioned. 175 00:08:42,770 --> 00:08:44,900 Problem sets. 176 00:08:44,900 --> 00:08:49,830 They'll be weekly, except before the exam and except in 177 00:08:49,830 --> 00:08:51,090 the last week to the course. 178 00:08:54,390 --> 00:08:57,840 The purpose of the problem sets is for you to get 179 00:08:57,840 --> 00:09:00,000 practice in getting up to speed. 180 00:09:00,000 --> 00:09:01,350 If you don't use them that way, you're 181 00:09:01,350 --> 00:09:03,290 not using them correctly. 182 00:09:03,290 --> 00:09:06,120 I absolutely don't care if you do them together or whatever 183 00:09:06,120 --> 00:09:08,330 method is most effective for you to learn. 184 00:09:08,330 --> 00:09:12,230 These are intended for you to learn. 185 00:09:12,230 --> 00:09:16,120 I don't recommend going in -- many of these of have been 186 00:09:16,120 --> 00:09:20,450 offered in prior years, and the answers are to be found 187 00:09:20,450 --> 00:09:23,410 somewhere on the net or in somebody's library. 188 00:09:23,410 --> 00:09:26,970 But as you've heard in every other class, I don't recommend 189 00:09:26,970 --> 00:09:30,460 your going about them that way. 190 00:09:30,460 --> 00:09:34,320 Not much weight is put on them in the grading. 191 00:09:34,320 --> 00:09:37,500 The TA or the grader will simply put a grade of zero, 192 00:09:37,500 --> 00:09:38,130 one, or two. 193 00:09:38,130 --> 00:09:41,190 We'll be happy to discuss any of them where you feel you 194 00:09:41,190 --> 00:09:45,190 didn't really get it, or don't know how to get your arms 195 00:09:45,190 --> 00:09:48,470 around a particular problem. 196 00:09:48,470 --> 00:09:53,430 And in fact, I wouldn't put any weight on the homeworks, 197 00:09:53,430 --> 00:09:55,690 except that every year students say, well, if you put 198 00:09:55,690 --> 00:09:59,050 0% on the problem sets, then we won't do them. 199 00:09:59,050 --> 00:10:02,470 And so I put 15% on the problem sets. 200 00:10:02,470 --> 00:10:05,680 But in fact the problem sets -- 201 00:10:05,680 --> 00:10:10,060 we do get an idea of how engaged you are in the course 202 00:10:10,060 --> 00:10:13,520 and how much you're getting it from the problem sets. 203 00:10:13,520 --> 00:10:16,820 That's important feedback, both generally for the class 204 00:10:16,820 --> 00:10:20,090 and individually for each of you. 205 00:10:20,090 --> 00:10:23,450 You know, there have been cases where somebody is just 206 00:10:23,450 --> 00:10:26,500 copying the problem set from the previous year every week, 207 00:10:26,500 --> 00:10:29,660 and then we can pretty well predict that they'll bomb out 208 00:10:29,660 --> 00:10:32,810 on the midterm, and they do. 209 00:10:32,810 --> 00:10:33,660 So anyway. 210 00:10:33,660 --> 00:10:35,630 You've been at MIT a while. 211 00:10:35,630 --> 00:10:38,180 This is pretty much the standard philosophy, I think, 212 00:10:38,180 --> 00:10:40,990 at least in Area 1. 213 00:10:40,990 --> 00:10:45,032 Any questions about what I just said? 214 00:10:45,032 --> 00:10:46,282 No? 215 00:10:48,760 --> 00:10:51,990 The midterm. 216 00:10:51,990 --> 00:10:56,770 Scheduled for two hours in the Wednesday, the last class 217 00:10:56,770 --> 00:10:58,880 before the vacation. 218 00:10:58,880 --> 00:11:00,590 Counts for 1/3. 219 00:11:00,590 --> 00:11:03,420 The final is scheduled during finals week. 220 00:11:03,420 --> 00:11:05,866 I don't have control over that. 221 00:11:05,866 --> 00:11:09,120 It will count for 1/2. 222 00:11:09,120 --> 00:11:12,390 Basically, the grade comes from adding up your scores on 223 00:11:12,390 --> 00:11:13,640 those two things. 224 00:11:15,760 --> 00:11:18,130 Doing a scatter chart and trying to make some 225 00:11:18,130 --> 00:11:20,360 intelligent guess. 226 00:11:20,360 --> 00:11:24,120 Usually we get a little more than half A's, a little less 227 00:11:24,120 --> 00:11:26,340 than half B's. 228 00:11:26,340 --> 00:11:28,740 But I don't have any fixed number in mind for that. 229 00:11:28,740 --> 00:11:32,720 It's really, I try to do it based on how you do, using 230 00:11:32,720 --> 00:11:36,550 pluses and minuses for decoration. 231 00:11:36,550 --> 00:11:39,980 And I don't know. 232 00:11:39,980 --> 00:11:42,440 Again, I think it's like what you see in every other course. 233 00:11:46,980 --> 00:11:47,540 All right. 234 00:11:47,540 --> 00:11:51,090 The topics. 235 00:11:51,090 --> 00:11:56,090 What's going to be different about 6.451 this year from 236 00:11:56,090 --> 00:11:59,440 2003, which is the last time it was offered 237 00:11:59,440 --> 00:12:02,300 for previous years? 238 00:12:02,300 --> 00:12:03,470 Not too much. 239 00:12:03,470 --> 00:12:06,110 Basically, this course -- 240 00:12:06,110 --> 00:12:07,760 originally there was a single course, which 241 00:12:07,760 --> 00:12:10,760 covered 6.450 and 451. 242 00:12:10,760 --> 00:12:14,730 It evolved, got split into two courses. 243 00:12:14,730 --> 00:12:18,390 And this course has pretty much become a coding course. 244 00:12:18,390 --> 00:12:22,040 So you are here because you're interested in coding, in 245 00:12:22,040 --> 00:12:25,030 particular for communications. 246 00:12:25,030 --> 00:12:27,220 This is a good first course in coding for, I 247 00:12:27,220 --> 00:12:29,910 think, other interests. 248 00:12:29,910 --> 00:12:32,140 Are you Andrew Cross? 249 00:12:32,140 --> 00:12:32,235 Yes. 250 00:12:32,235 --> 00:12:33,860 What is your situation on the prerequisites? 251 00:12:36,695 --> 00:12:37,945 AUDIENCE: [INAUDIBLE]. 252 00:12:43,850 --> 00:12:44,100 PROFESSOR: OK. 253 00:12:44,100 --> 00:12:45,350 Let's talk about it afterwards. 254 00:12:50,780 --> 00:12:50,932 Alright. 255 00:12:50,932 --> 00:12:58,190 So as I said earlier, the course is pretty much 256 00:12:58,190 --> 00:13:02,850 stabilized at this point to be a course on coding over the 257 00:13:02,850 --> 00:13:04,300 additive white Gaussian channel. 258 00:13:04,300 --> 00:13:07,380 Those of you who are here with other interests, that's not 259 00:13:07,380 --> 00:13:10,250 the only place that coding is used, I think will be 260 00:13:10,250 --> 00:13:11,920 reasonably satisfied with the course. 261 00:13:11,920 --> 00:13:14,710 We're always putting it in a communication context, but we 262 00:13:14,710 --> 00:13:18,380 talk about all the major classes of algebraic block 263 00:13:18,380 --> 00:13:22,820 codes, convolutional codes, tail-biting codes, capacity 264 00:13:22,820 --> 00:13:26,450 approaching codes, low-density parity-check codes, lattice 265 00:13:26,450 --> 00:13:28,034 codes, trellis coded modulation, if 266 00:13:28,034 --> 00:13:30,420 you want to get ... 267 00:13:30,420 --> 00:13:33,030 and all of their decoding algorithms, which are just as 268 00:13:33,030 --> 00:13:35,130 important as the codes themselves. 269 00:13:35,130 --> 00:13:39,960 So I hope that's what you're after, is to get a view of 270 00:13:39,960 --> 00:13:45,130 which codes have proved to be most important over the years. 271 00:13:45,130 --> 00:13:49,660 My context is communication, but I believe you'll get 272 00:13:49,660 --> 00:13:52,420 exposed to most of the things you would need, regardless of 273 00:13:52,420 --> 00:13:55,800 why you happen to be interested in coding. 274 00:13:55,800 --> 00:14:00,380 What is going to be different this year is I'm going to be 275 00:14:00,380 --> 00:14:08,210 rather ruthlessly chopping out not perfectly essential topics 276 00:14:08,210 --> 00:14:09,880 wherever I can through the course. 277 00:14:09,880 --> 00:14:12,610 And in particular, I'm going to run through chapters one 278 00:14:12,610 --> 00:14:17,760 through three today, assuming that -- 279 00:14:17,760 --> 00:14:19,350 well, one is just introduction. 280 00:14:19,350 --> 00:14:24,540 Chapter two is basically how you get from continuous time 281 00:14:24,540 --> 00:14:27,420 to discrete time on additive white Gaussian noise channels. 282 00:14:27,420 --> 00:14:32,030 You did that fairly well in 6.450, so I assume all I need 283 00:14:32,030 --> 00:14:34,650 to do is basically sketch it for you. 284 00:14:34,650 --> 00:14:39,910 And chapter three is proof of the channel capacity formula, 285 00:14:39,910 --> 00:14:43,140 which you may or may not have encountered somewhere else. 286 00:14:43,140 --> 00:14:45,980 But even if you haven't encountered it, you're 287 00:14:45,980 --> 00:14:48,820 probably reasonably willing to take it on faith. 288 00:14:48,820 --> 00:14:52,610 And so that allows us to get immediately into chapter four 289 00:14:52,610 --> 00:14:56,300 in the next lecture, where Ashish will be starting to 290 00:14:56,300 --> 00:15:00,000 work up from the smallest, simplest little constellations 291 00:15:00,000 --> 00:15:05,260 in Euclidean space, which is where the story begins. 292 00:15:05,260 --> 00:15:08,530 This hopefully will give us more time at the end of the 293 00:15:08,530 --> 00:15:14,100 course for more on what's been happening in the last ten 294 00:15:14,100 --> 00:15:15,420 years or so. 295 00:15:15,420 --> 00:15:19,000 First of all, a much more analytical treatment of 296 00:15:19,000 --> 00:15:21,480 capacity approaching codes than I've been able 297 00:15:21,480 --> 00:15:23,880 to give in the past. 298 00:15:23,880 --> 00:15:24,830 The real theory. 299 00:15:24,830 --> 00:15:26,520 How do you design them? 300 00:15:26,520 --> 00:15:28,770 How do they perform? 301 00:15:28,770 --> 00:15:29,800 Some more details. 302 00:15:29,800 --> 00:15:32,460 Enough so that you could go ahead and implement them, I 303 00:15:32,460 --> 00:15:37,540 believe, and analyze them, and optimize them. 304 00:15:37,540 --> 00:15:40,070 And that's, I think, really important in this course. 305 00:15:40,070 --> 00:15:42,840 Not just because that's the way people are doing coding 306 00:15:42,840 --> 00:15:47,380 these days, but also because that's the end of the story. 307 00:15:47,380 --> 00:15:50,830 After 50 years, we finally did get to channel capacity, and 308 00:15:50,830 --> 00:15:54,120 this was the way we got there. 309 00:15:54,120 --> 00:15:59,290 In addition, if we really do well, we'll be able to do at 310 00:15:59,290 --> 00:16:03,620 least a week, maybe two weeks, on codes for band limited 311 00:16:03,620 --> 00:16:06,740 channels, where you need to go beyond binary. 312 00:16:06,740 --> 00:16:11,260 You need to send codes, at least symbols, that have many 313 00:16:11,260 --> 00:16:14,630 levels on them, not binary levels, in order to send it at 314 00:16:14,630 --> 00:16:18,260 high spec coefficiencies, many bits per second per Hertz, 315 00:16:18,260 --> 00:16:22,910 which is, of course, what you want for most wireless or 316 00:16:22,910 --> 00:16:25,170 wireline channels. 317 00:16:25,170 --> 00:16:28,170 You are able to send lots of bits per second per Hertz. 318 00:16:28,170 --> 00:16:29,510 How do we do that? 319 00:16:29,510 --> 00:16:32,760 Again, hopefully getting to capacity. 320 00:16:32,760 --> 00:16:38,230 And finally, I always offer the teaser, and for at least 321 00:16:38,230 --> 00:16:40,460 five years, I haven't been able to get there. 322 00:16:40,460 --> 00:16:44,270 We could talk about linear Gaussian channels. 323 00:16:44,270 --> 00:16:47,420 Where you get into equalization, precoding. 324 00:16:47,420 --> 00:16:50,540 Other topics that are very important. 325 00:16:50,540 --> 00:16:54,310 But I'm almost certain we won't get there. 326 00:16:54,310 --> 00:16:57,780 And in addition, this subject, this kind of a scalar version 327 00:16:57,780 --> 00:17:00,150 of what's done to a fare-thee-well in the matrix 328 00:17:00,150 --> 00:17:04,660 way, in the wireless course, 6.452, which is taught in the 329 00:17:04,660 --> 00:17:07,180 next classroom in the next hour and a half. 330 00:17:07,180 --> 00:17:10,150 So I anticipate in future the whole equalization of 331 00:17:10,150 --> 00:17:11,180 precoding -- 332 00:17:11,180 --> 00:17:14,819 intellectually, it should be merged with Wireless. 333 00:17:14,819 --> 00:17:18,829 Because they do matrix versions of all of that. 334 00:17:18,829 --> 00:17:21,260 On the other hand, it forms a part of this story, which is 335 00:17:21,260 --> 00:17:24,329 just basically point-to-point transmission over the additive 336 00:17:24,329 --> 00:17:27,390 white Gaussian noise channel. 337 00:17:27,390 --> 00:17:27,760 All right. 338 00:17:27,760 --> 00:17:31,170 I very much encourage questions and feedback, and 339 00:17:31,170 --> 00:17:35,370 I'm always willing to be distracted, tell stories, 340 00:17:35,370 --> 00:17:37,200 respond to just what's on your mind. 341 00:17:37,200 --> 00:17:40,380 It's much more fun for me, and probably helps you, too. 342 00:17:40,380 --> 00:17:45,060 Does anybody have any questions at this point, or 343 00:17:45,060 --> 00:17:46,310 observations? 344 00:17:51,290 --> 00:17:54,230 No? 345 00:17:54,230 --> 00:17:54,950 All right. 346 00:17:54,950 --> 00:17:57,050 We should just get into it. 347 00:18:07,690 --> 00:18:10,160 OK. 348 00:18:10,160 --> 00:18:14,700 So I say I always like to be thinking of this from a 349 00:18:14,700 --> 00:18:19,110 communications design point of view. 350 00:18:19,110 --> 00:18:20,590 We are engineers. 351 00:18:20,590 --> 00:18:26,020 I am a communications engineer and theorist. 352 00:18:26,020 --> 00:18:29,270 And what's the design problem that we're really trying to 353 00:18:29,270 --> 00:18:31,650 solve in this course? 354 00:18:31,650 --> 00:18:35,160 The design problem is that somebody -- 355 00:18:35,160 --> 00:18:39,790 your boss, or the FCC, or somebody -- gives you a 356 00:18:39,790 --> 00:18:44,640 channel, which usually involves some set of 357 00:18:44,640 --> 00:18:45,890 frequencies and ... 358 00:18:48,890 --> 00:18:52,410 over some physical medium and says, OK. 359 00:18:52,410 --> 00:18:55,790 I want you to design a communications device, a 360 00:18:55,790 --> 00:19:01,550 modem, to get as much digital information over that channel 361 00:19:01,550 --> 00:19:04,160 as you can. 362 00:19:04,160 --> 00:19:08,180 So in chapter one, we talk about a couple of channels 363 00:19:08,180 --> 00:19:14,480 which have been very important for the development of coding. 364 00:19:14,480 --> 00:19:17,526 One, for instance, the deep space channel. 365 00:19:17,526 --> 00:19:20,340 The deep space channel -- what's the problem? 366 00:19:20,340 --> 00:19:23,270 You have a highly power-limited 367 00:19:23,270 --> 00:19:26,050 satellite way out there. 368 00:19:26,050 --> 00:19:29,120 It's got a little tiny antenna. 369 00:19:29,120 --> 00:19:33,700 It can use whatever frequencies it wants. 370 00:19:33,700 --> 00:19:38,840 It's talking to a huge 140-foot dish or whatever 371 00:19:38,840 --> 00:19:41,330 somewhere in Pasadena or somewhere 372 00:19:41,330 --> 00:19:42,960 else around the world. 373 00:19:42,960 --> 00:19:46,840 And that's basically the channel. 374 00:19:46,840 --> 00:19:50,240 And you can send any wave form you'd like, subject, really, 375 00:19:50,240 --> 00:19:51,840 to power limitation. 376 00:19:51,840 --> 00:19:55,450 This is an extreme example of a power-limited channel. 377 00:19:55,450 --> 00:19:57,210 How do we characterize such a channel? 378 00:19:57,210 --> 00:20:03,140 Well, frequencies are unlimited, but the 379 00:20:03,140 --> 00:20:04,930 signal-to-noise ratio is very poor. 380 00:20:04,930 --> 00:20:09,820 You get a very tiny signal, and you get noise in the 381 00:20:09,820 --> 00:20:13,390 antenna front end, which they've done everything they 382 00:20:13,390 --> 00:20:17,520 can reduce it, they've cooled it to a few degrees above zero 383 00:20:17,520 --> 00:20:18,450 and so forth. 384 00:20:18,450 --> 00:20:23,450 Nonetheless, the front end noise -- in the very first 385 00:20:23,450 --> 00:20:26,390 stages of amplification are what is the noise. 386 00:20:26,390 --> 00:20:30,970 And you've basically got to transmit through that. 387 00:20:30,970 --> 00:20:39,810 So you have a pure additive white Gaussian noise channel, 388 00:20:39,810 --> 00:20:46,250 which we we'll always write as the output of the channel Y of 389 00:20:46,250 --> 00:20:52,690 t is the input, X of t, plus the noise, N of t. 390 00:20:52,690 --> 00:20:58,680 Which is additive, independent of the signal, and white. 391 00:20:58,680 --> 00:21:00,795 The spectrum can be taken to be flat. 392 00:21:05,930 --> 00:21:07,280 All right. 393 00:21:07,280 --> 00:21:08,660 And what else do we have? 394 00:21:08,660 --> 00:21:20,730 We have that this has some average power P, and this has 395 00:21:20,730 --> 00:21:33,620 some noise power spectral density which is characterized 396 00:21:33,620 --> 00:21:37,520 by a parameter N_0, which is basically the 397 00:21:37,520 --> 00:21:40,150 noise power per Hertz. 398 00:21:40,150 --> 00:21:44,250 Per positive frequently Hertz, as it turns out, because these 399 00:21:44,250 --> 00:21:46,425 things were all defined back in the dark ages. 400 00:21:49,470 --> 00:21:50,830 OK. 401 00:21:50,830 --> 00:21:55,460 And we also, either because of the channel specification or 402 00:21:55,460 --> 00:22:01,260 because ultimately, we decide to signal in a particular 403 00:22:01,260 --> 00:22:12,050 bandwidth, we have a bandwidth W. Again, that's just measured 404 00:22:12,050 --> 00:22:14,210 over the positive frequencies. 405 00:22:14,210 --> 00:22:18,210 So if it goes from zero to W, that's a baseband channel. 406 00:22:18,210 --> 00:22:23,350 If it goes from W_0 to W_0 plus W, 407 00:22:23,350 --> 00:22:26,790 that's a passband channel. 408 00:22:26,790 --> 00:22:28,930 But we're going to find out it doesn't really matter where 409 00:22:28,930 --> 00:22:32,760 the passband is, or the baseband. 410 00:22:32,760 --> 00:22:38,150 So those are three parameters that specify an additive white 411 00:22:38,150 --> 00:22:40,420 Gaussian noise channel. 412 00:22:40,420 --> 00:22:45,010 And we can aggregate them into a signal noise ratio. 413 00:22:45,010 --> 00:22:49,680 A signal-to-noise ratio is the ratio of a signal power to a 414 00:22:49,680 --> 00:22:50,490 noise power. 415 00:22:50,490 --> 00:22:54,910 In this case, the signal power, by definition, is P. 416 00:22:54,910 --> 00:22:58,470 The noise power, it's N_0 per Hertz. 417 00:22:58,470 --> 00:23:00,680 We're going across W Hertz. 418 00:23:00,680 --> 00:23:03,940 So by the way we define N_0, the signal-to-noise ratio is 419 00:23:03,940 --> 00:23:07,050 just P over N_0 W. That's the amount of noise 420 00:23:07,050 --> 00:23:09,230 power in the band. 421 00:23:09,230 --> 00:23:12,270 And there's an implicit assumption here that only the 422 00:23:12,270 --> 00:23:20,180 noise in the band concerns you, and it's one of the key 423 00:23:20,180 --> 00:23:24,730 results of 6.450 or equivalent courses that out-of-band noise 424 00:23:24,730 --> 00:23:27,210 is irrelevant, because you know information about what's 425 00:23:27,210 --> 00:23:29,530 in-band, so you may as well filter it out. 426 00:23:29,530 --> 00:23:32,940 And therefore we don't have to count any noise power that's 427 00:23:32,940 --> 00:23:36,280 outside the signal transmission band or the 428 00:23:36,280 --> 00:23:39,440 signal space, as we would say. 429 00:23:39,440 --> 00:23:44,980 So two key things. 430 00:23:44,980 --> 00:23:50,080 It's easier to keep in mind the signal-to-noise ratio than 431 00:23:50,080 --> 00:23:51,870 both P and N_0. 432 00:23:51,870 --> 00:23:55,580 By an amplifier, you could scale both P and N_0 so really 433 00:23:55,580 --> 00:23:58,850 all you're interested in is the ratio anyway. 434 00:23:58,850 --> 00:24:03,520 So we have these two key parameters for a additive 435 00:24:03,520 --> 00:24:05,080 white Gaussian noise channel. 436 00:24:07,590 --> 00:24:07,950 All right. 437 00:24:07,950 --> 00:24:12,570 A more typical kind of design problem is, say, a -- 438 00:24:12,570 --> 00:24:18,300 well, the telephone line modem, which is something on 439 00:24:18,300 --> 00:24:21,840 which I've spent a good part of my career. 440 00:24:21,840 --> 00:24:24,110 Your boss says, OK. 441 00:24:24,110 --> 00:24:27,370 I want you to build me a modem that sends as much data as 442 00:24:27,370 --> 00:24:30,830 possible over telephone lines. 443 00:24:30,830 --> 00:24:32,240 How would you approach that problem? 444 00:24:38,650 --> 00:24:38,790 OK? 445 00:24:38,790 --> 00:24:40,820 This is your summer project . 446 00:24:40,820 --> 00:24:42,550 What would you do on the first day? 447 00:24:46,190 --> 00:24:48,140 I really would like a little feedback, folks. 448 00:24:48,140 --> 00:24:49,840 Makes it so much more fun for me. 449 00:24:49,840 --> 00:24:50,328 Yeah? 450 00:24:50,328 --> 00:24:53,045 AUDIENCE: Can we just tell him he's [INAUDIBLE] 451 00:24:53,045 --> 00:24:54,320 frequency -- 452 00:24:54,320 --> 00:24:55,360 we have [INAUDIBLE] 453 00:24:55,360 --> 00:24:59,320 frequency, so it's not like this time it would 454 00:24:59,320 --> 00:25:02,036 [INAUDIBLE]. 455 00:25:02,036 --> 00:25:02,470 PROFESSOR: All right. 456 00:25:02,470 --> 00:25:06,930 So you have in mind a certain channel model. 457 00:25:06,930 --> 00:25:11,766 How would you verify that channel model? 458 00:25:11,766 --> 00:25:14,650 AUDIENCE: Well, you put the channel frequency response, 459 00:25:14,650 --> 00:25:16,390 you could measure it somehow. 460 00:25:16,390 --> 00:25:19,750 You could estimate it. 461 00:25:19,750 --> 00:25:20,080 PROFESSOR: Yeah. 462 00:25:20,080 --> 00:25:25,620 You could take a sine wave generator and run it across 463 00:25:25,620 --> 00:25:27,580 the frequencies, and see how much gets 464 00:25:27,580 --> 00:25:28,690 through at the receiver. 465 00:25:28,690 --> 00:25:35,190 And you could plot some kind of a channel response, which 466 00:25:35,190 --> 00:25:40,160 for a particular telephone channel might go 467 00:25:40,160 --> 00:25:41,920 something like that. 468 00:25:41,920 --> 00:25:45,700 Turns out telephone channels are designed so that they have 469 00:25:45,700 --> 00:25:50,170 no transmission above 4 kilohertz, because you sample 470 00:25:50,170 --> 00:25:53,370 them 8000 times a second inside the network. 471 00:25:53,370 --> 00:25:56,140 They're are also designed to have no DC response. 472 00:25:56,140 --> 00:25:59,400 So this might go from -- this is -- 473 00:25:59,400 --> 00:26:01,760 well, I'm just drawing some generic response. 474 00:26:01,760 --> 00:26:05,120 And dB, I'll give you my dB lecture later. 475 00:26:08,220 --> 00:26:12,300 300 to 3800 Hertz might be something like that. 476 00:26:12,300 --> 00:26:14,310 And of course, if you measure another channel, would it be 477 00:26:14,310 --> 00:26:16,350 exactly the same? 478 00:26:16,350 --> 00:26:17,900 No, but it might be roughly the same. 479 00:26:17,900 --> 00:26:20,080 I mean, all these channels are engineered 480 00:26:20,080 --> 00:26:22,560 to pass human voice. 481 00:26:22,560 --> 00:26:22,740 Alright? 482 00:26:22,740 --> 00:26:25,580 In particular, there are now international standards that 483 00:26:25,580 --> 00:26:30,260 say it's got to pass at least between here and here with 484 00:26:30,260 --> 00:26:32,680 pretty good fidelity. 485 00:26:32,680 --> 00:26:33,010 All right. 486 00:26:33,010 --> 00:26:34,240 So that's the first thing you do. 487 00:26:34,240 --> 00:26:37,930 You develop a model of the channel. 488 00:26:37,930 --> 00:26:47,090 And the key things about it, first order are, it's nearly a 489 00:26:47,090 --> 00:26:48,680 linear channel. 490 00:26:48,680 --> 00:26:50,720 What you put in will nearly come out. 491 00:26:53,330 --> 00:26:59,330 It has a certain bandwidth, which is somewhere between 492 00:26:59,330 --> 00:27:02,400 2400, we used to say, way back in the '60s. 493 00:27:02,400 --> 00:27:06,900 Nowadays we might say we can actually get through 3600, 494 00:27:06,900 --> 00:27:08,160 3700 Hertz. 495 00:27:08,160 --> 00:27:12,880 But that's what's the order of the channel bandwidth is. 496 00:27:12,880 --> 00:27:20,080 So we have W equals 3700 Hertz. 497 00:27:20,080 --> 00:27:24,560 And then, of course, we have some noise. 498 00:27:24,560 --> 00:27:26,790 If we didn't have any noise, how many bits per second could 499 00:27:26,790 --> 00:27:28,040 we get through here? 500 00:27:31,660 --> 00:27:33,230 Hello? 501 00:27:33,230 --> 00:27:35,090 Infinite, right. 502 00:27:35,090 --> 00:27:39,040 Send one number with infinite precision. 503 00:27:39,040 --> 00:27:42,710 And that would communicate the entire file that 504 00:27:42,710 --> 00:27:43,380 we wanted to send. 505 00:27:43,380 --> 00:27:47,545 And so we have a signal-to-noise ratio. 506 00:27:47,545 --> 00:27:50,120 It's questionable whether the noise is totally Gaussian. 507 00:27:50,120 --> 00:27:53,310 You've heard a telephone channel. 508 00:27:53,310 --> 00:27:57,580 You know, it's some weird stuff. 509 00:27:57,580 --> 00:28:01,510 But this is what our kind of normal telephone 510 00:28:01,510 --> 00:28:03,570 channel would look like. 511 00:28:03,570 --> 00:28:06,100 I should say, greater than this. 512 00:28:09,170 --> 00:28:10,350 Nowadays. 513 00:28:10,350 --> 00:28:11,690 Used to be not nearly as good... 514 00:28:11,690 --> 00:28:12,960 Or I shouldn't say nowadays. 515 00:28:12,960 --> 00:28:14,920 This was true about ten years ago. 516 00:28:14,920 --> 00:28:17,200 Nowadays everybody is using cell phones, and these things 517 00:28:17,200 --> 00:28:19,026 have all gotten terrible again. 518 00:28:19,026 --> 00:28:19,380 All right? 519 00:28:19,380 --> 00:28:21,775 So please do not design your modem for a cell phone. 520 00:28:21,775 --> 00:28:26,580 Let's plug a nice wired connection into the wall. 521 00:28:26,580 --> 00:28:27,130 OK. 522 00:28:27,130 --> 00:28:28,780 So that's a first gross 523 00:28:28,780 --> 00:28:30,950 characterization of this channel. 524 00:28:30,950 --> 00:28:34,550 What would you then do on the second day 525 00:28:34,550 --> 00:28:35,855 of your summer project? 526 00:28:41,450 --> 00:28:42,190 Yeah? 527 00:28:42,190 --> 00:28:43,440 AUDIENCE: [INAUDIBLE]. 528 00:28:45,270 --> 00:28:46,230 PROFESSOR: OK. 529 00:28:46,230 --> 00:28:48,320 That's a good idea. 530 00:28:48,320 --> 00:28:51,808 What do you have in mind in designing a modulations key? 531 00:28:51,808 --> 00:28:53,058 AUDIENCE: [INAUDIBLE]. 532 00:28:56,500 --> 00:28:59,430 PROFESSOR: A signal constellation. 533 00:28:59,430 --> 00:28:59,970 All right. 534 00:28:59,970 --> 00:29:01,850 What signal constellation are you going to use? 535 00:29:04,778 --> 00:29:06,780 PAM? 536 00:29:06,780 --> 00:29:07,570 QAM? 537 00:29:07,570 --> 00:29:08,560 Yeah? 538 00:29:08,560 --> 00:29:10,920 What would control your choice? 539 00:29:14,360 --> 00:29:15,440 ISI might. 540 00:29:15,440 --> 00:29:19,680 This channel doesn't have a flat response. 541 00:29:19,680 --> 00:29:20,460 We might -- 542 00:29:20,460 --> 00:29:27,470 let's say we've actually developed a model where Y of t 543 00:29:27,470 --> 00:29:32,900 is X of t going through a linear filter, convolved with 544 00:29:32,900 --> 00:29:36,580 some response h of t plus N of t. 545 00:29:36,580 --> 00:29:42,530 So what we transmit gets filtered. 546 00:29:42,530 --> 00:29:44,970 We add noise to it. 547 00:29:44,970 --> 00:29:48,430 And that's what we see at the output. 548 00:29:48,430 --> 00:29:50,730 That's just a first order model. 549 00:29:50,730 --> 00:29:54,230 This is basically the way modems were designed for 550 00:29:54,230 --> 00:29:57,380 telephone channels for many, many years. 551 00:29:57,380 --> 00:29:59,230 Just with that simple model. 552 00:29:59,230 --> 00:29:59,610 All right. 553 00:29:59,610 --> 00:30:03,070 So if we have a filter here, we might have to worry about 554 00:30:03,070 --> 00:30:04,320 intersymbol interference. 555 00:30:06,970 --> 00:30:09,540 But you're not quite back at the basic level. 556 00:30:09,540 --> 00:30:14,580 Let's suppose we decide that the channel is reliable just 557 00:30:14,580 --> 00:30:18,550 within some particular frequency band w, and within 558 00:30:18,550 --> 00:30:22,740 that band, it's more or less flat. 559 00:30:22,740 --> 00:30:23,060 All right? 560 00:30:23,060 --> 00:30:26,540 When I talk about white Gaussian noise, I'm always 561 00:30:26,540 --> 00:30:30,830 going to mean that the channel, the noise spectral 562 00:30:30,830 --> 00:30:34,900 density, and the channel response are flat within the 563 00:30:34,900 --> 00:30:36,150 given bandwidth. 564 00:30:38,590 --> 00:30:38,890 All right? 565 00:30:38,890 --> 00:30:41,840 So I'm not going to have to worry about ISI at this level. 566 00:30:41,840 --> 00:30:44,540 Of course I will in my telephone line modem. 567 00:30:44,540 --> 00:30:45,358 Yeah? 568 00:30:45,358 --> 00:30:48,046 AUDIENCE: So even if we want high speed, [INAUDIBLE] 569 00:30:54,420 --> 00:30:58,350 so we want a rate that would be greater than [INAUDIBLE]. 570 00:31:01,260 --> 00:31:02,860 PROFESSOR: I suppose. 571 00:31:02,860 --> 00:31:05,150 I mean, do you have enough information to say? 572 00:31:09,070 --> 00:31:14,310 AUDIENCE: Well, for example, if we [INAUDIBLE] 573 00:31:17,060 --> 00:31:23,910 64 kilobits per second, can we travel 3,700 [INAUDIBLE] of 574 00:31:23,910 --> 00:31:28,004 bandwidth that we need use the bandwidth [INAUDIBLE]. 575 00:31:30,870 --> 00:31:31,076 PROFESSOR: OK. 576 00:31:31,076 --> 00:31:33,570 Is there any chance of sending 64 kilobits 577 00:31:33,570 --> 00:31:34,820 through this channel? 578 00:31:37,300 --> 00:31:38,260 AUDIENCE: Yes. 579 00:31:38,260 --> 00:31:39,610 PROFESSOR: There is? 580 00:31:39,610 --> 00:31:40,860 How do you know that? 581 00:31:46,264 --> 00:31:48,160 AUDIENCE: If you use [INAUDIBLE PHRASE]. 582 00:32:05,180 --> 00:32:05,620 PROFESSOR: All right. 583 00:32:05,620 --> 00:32:08,740 So that would be 32 level complement -- 584 00:32:08,740 --> 00:32:14,280 that would be five bits per Hertz, basically, talking in 585 00:32:14,280 --> 00:32:17,120 terms of spectral efficiency. 586 00:32:17,120 --> 00:32:20,960 Times even 4000 kilohertz, that only gives you 20 -- 587 00:32:20,960 --> 00:32:21,872 AUDIENCE: Then a higher level. 588 00:32:21,872 --> 00:32:23,050 PROFESSOR: Then a higher level. 589 00:32:23,050 --> 00:32:27,300 1024 level QAM. 590 00:32:27,300 --> 00:32:29,880 But what's going to limit you? 591 00:32:29,880 --> 00:32:32,135 I mean, why not make it a million level? 592 00:32:32,135 --> 00:32:33,385 AUDIENCE: [INAUDIBLE]. 593 00:32:35,430 --> 00:32:38,030 PROFESSOR: The signal-to-noise ratio, all right? 594 00:32:38,030 --> 00:32:39,280 OK. 595 00:32:41,560 --> 00:32:42,070 Well. 596 00:32:42,070 --> 00:32:46,270 I suggest what you do on your second day is you start 597 00:32:46,270 --> 00:32:48,120 thinking about, on the one hand, 598 00:32:48,120 --> 00:32:50,370 simple modulation schemes. 599 00:32:50,370 --> 00:32:53,130 And I would say 32 -bit QAM is simple in the 600 00:32:53,130 --> 00:32:54,720 context of this course. 601 00:32:54,720 --> 00:32:56,910 It has no coding. 602 00:32:56,910 --> 00:33:02,600 But just kind of give yourself a baseline of what might work. 603 00:33:02,600 --> 00:33:05,090 But another very good thing to do is to 604 00:33:05,090 --> 00:33:08,580 establish an upper limit. 605 00:33:08,580 --> 00:33:12,520 Now, it's kind of amazing that we can do this. 606 00:33:12,520 --> 00:33:15,840 Up to 1948, no one would've thought that it was a 607 00:33:15,840 --> 00:33:18,810 reasonable to say, well, there's a certain upper limit 608 00:33:18,810 --> 00:33:21,260 on what we could ever transmit through this channel. 609 00:33:21,260 --> 00:33:24,690 The idea was if you transmitted fast, you'd make a 610 00:33:24,690 --> 00:33:25,450 lot of errors. 611 00:33:25,450 --> 00:33:27,920 If you slowed down, you'd make fewer errors. 612 00:33:27,920 --> 00:33:30,570 And you know, there's no hard and fast limit. 613 00:33:30,570 --> 00:33:39,220 But of course, what Shannon did was to show that for 614 00:33:39,220 --> 00:33:42,400 mathematically well-specified channels, of which this is 615 00:33:42,400 --> 00:33:47,860 one, there is a certain limiting data rate called the 616 00:33:47,860 --> 00:33:49,110 channel capacity. 617 00:33:55,680 --> 00:34:01,240 And what the capacity is for an additive white Gaussian 618 00:34:01,240 --> 00:34:03,840 noise channel, this is the most famous formula in 619 00:34:03,840 --> 00:34:05,325 information theory. 620 00:34:05,325 --> 00:34:07,710 If we have time, I'll derive it for you. 621 00:34:07,710 --> 00:34:11,159 Otherwise I encourage you to look it up. 622 00:34:11,159 --> 00:34:18,389 The capacity of this channel is entirely specified by these 623 00:34:18,389 --> 00:34:22,100 two parameters, bandwidth and signal-to-noise ratio. 624 00:34:22,100 --> 00:34:25,949 And it's simply W times the binary logarithm of one plus 625 00:34:25,949 --> 00:34:28,969 SNR in bits per second. 626 00:34:28,969 --> 00:34:29,449 All right? 627 00:34:29,449 --> 00:34:34,920 And the Shannon proved a strong capacity 628 00:34:34,920 --> 00:34:36,370 theorem and a converse. 629 00:34:36,370 --> 00:34:39,880 The capacity theorem says, there does exist a 630 00:34:39,880 --> 00:34:41,210 coding scheme -- 631 00:34:41,210 --> 00:34:43,909 perhaps involving QAM, perhaps involving lots 632 00:34:43,909 --> 00:34:45,670 of other stuff -- 633 00:34:45,670 --> 00:34:50,110 that can get you as close as you want in rate, for any rate 634 00:34:50,110 --> 00:34:55,100 less than capacity, there exists a -- 635 00:34:55,100 --> 00:35:02,060 let's say a coding slash modulation scheme, slash 636 00:35:02,060 --> 00:35:10,660 modulation, slash detection, slash decoding scheme -- 637 00:35:10,660 --> 00:35:15,370 such that the probability of error is less than or equal to 638 00:35:15,370 --> 00:35:19,930 epsilon, where epsilon can be chosen as in any arbitrarily 639 00:35:19,930 --> 00:35:23,520 small number. 640 00:35:23,520 --> 00:35:23,830 All right? 641 00:35:23,830 --> 00:35:28,230 So if you want a probability of error of ten to the minus 642 00:35:28,230 --> 00:35:32,680 five or ten to the minus ten, however you measure it -- 643 00:35:32,680 --> 00:35:33,640 can't be zero. 644 00:35:33,640 --> 00:35:36,380 Very big difference between zero and ten to the minus ten. 645 00:35:36,380 --> 00:35:38,320 But if you want probability of error of less than ten to the 646 00:35:38,320 --> 00:35:42,620 minus ten, then Shannon says there exists a scheme that can 647 00:35:42,620 --> 00:35:47,030 get there, as long as your rate is less than this 648 00:35:47,030 --> 00:35:50,610 capacity or Shannon limit. 649 00:35:50,610 --> 00:35:51,100 All right. 650 00:35:51,100 --> 00:35:54,900 So I suggest it would be a good idea to calculate what 651 00:35:54,900 --> 00:35:56,150 that is for this. 652 00:35:58,670 --> 00:36:02,000 Now, signal-to-noise ratio is 37 dB. 653 00:36:04,830 --> 00:36:07,990 Then one plus SNR is about the same as SNR. 654 00:36:10,660 --> 00:36:16,770 Log2 of 37 dB, factor of 2 is 3 dB, so that's 655 00:36:16,770 --> 00:36:19,610 about 12 and a third. 656 00:36:19,610 --> 00:36:23,210 This is our first example of why calculating in dB makes 657 00:36:23,210 --> 00:36:24,750 things very easy. 658 00:36:24,750 --> 00:36:28,660 So this is about 12 and a third times whatever the 659 00:36:28,660 --> 00:36:30,380 bandwidth is. 660 00:36:30,380 --> 00:36:35,320 3700 Hertz, and we go through that calculation, and we get 661 00:36:35,320 --> 00:36:40,490 something over 4,2000 bits per second. 662 00:36:40,490 --> 00:36:41,000 All right? 663 00:36:41,000 --> 00:36:47,820 So you have no hope of going more than 42000 bits per 664 00:36:47,820 --> 00:36:51,100 second, if this is, in fact, an accurate channel model. 665 00:36:51,100 --> 00:36:54,640 I mean, I'm assuming that these two numbers are correct. 666 00:36:54,640 --> 00:36:57,950 If this turns out to be 50 dB, then you 667 00:36:57,950 --> 00:37:00,297 can go a little further. 668 00:37:00,297 --> 00:37:01,460 All right? 669 00:37:01,460 --> 00:37:01,790 Yes? 670 00:37:01,790 --> 00:37:03,040 AUDIENCE: [INAUDIBLE]. 671 00:37:06,010 --> 00:37:07,080 PROFESSOR: Yes. 672 00:37:07,080 --> 00:37:14,110 And it's really only true for a perfectly flat channel that 673 00:37:14,110 --> 00:37:17,740 satisfies the complete mathematical definition of a 674 00:37:17,740 --> 00:37:23,840 white Gaussian noise channel within the band. 675 00:37:23,840 --> 00:37:28,300 I'm shocked to find that this is -- 676 00:37:28,300 --> 00:37:31,080 lots of people get through MIT Communications course and 677 00:37:31,080 --> 00:37:34,840 never see a proof of this formula. 678 00:37:34,840 --> 00:37:37,260 This is about as fundamental as you get. 679 00:37:37,260 --> 00:37:40,880 That's why I include one in this course. 680 00:37:40,880 --> 00:37:44,800 Or perhaps you've never even seen the formula before. 681 00:37:44,800 --> 00:37:47,100 Certainly don't understand where it's valid and where 682 00:37:47,100 --> 00:37:49,470 it's not valid. 683 00:37:49,470 --> 00:37:50,040 OK. 684 00:37:50,040 --> 00:38:00,420 So on the other hand, we might try something like 32 QAM. 685 00:38:00,420 --> 00:38:04,060 If we know something about that, we make it so that it 686 00:38:04,060 --> 00:38:05,910 fits in here. 687 00:38:05,910 --> 00:38:09,830 We find that we can get maybe five bits 688 00:38:09,830 --> 00:38:12,800 per second per Hertz. 689 00:38:12,800 --> 00:38:15,510 That's something called spectral efficiency, which in 690 00:38:15,510 --> 00:38:21,490 this course, we'll always designate by rho. 691 00:38:21,490 --> 00:38:26,230 So we can get maybe 3000 Hertz without having to worry about 692 00:38:26,230 --> 00:38:28,190 filtering too much. 693 00:38:28,190 --> 00:38:34,090 So we're going to have a system that sends 3000 QAM 694 00:38:34,090 --> 00:38:35,940 symbols per second -- 695 00:38:35,940 --> 00:38:40,270 3000 two-dimensional signals per second, each one 696 00:38:40,270 --> 00:38:43,080 carrying five bits. 697 00:38:43,080 --> 00:38:48,130 And so with that, we'll get 15000 bits per second. 698 00:38:48,130 --> 00:38:52,340 So these kind of establish the alpha and omega, the baseline 699 00:38:52,340 --> 00:38:54,820 and the ultimate limit, of what we could 700 00:38:54,820 --> 00:38:55,960 do over this channel. 701 00:38:55,960 --> 00:39:00,690 With a simple uncoded scheme, we would check that we can, in 702 00:39:00,690 --> 00:39:03,780 fact, get our desired error rate at the signal-to-noise 703 00:39:03,780 --> 00:39:05,350 ratio that we have here. 704 00:39:05,350 --> 00:39:10,260 So we know, without sweating hard, we could get 15000 bits 705 00:39:10,260 --> 00:39:10,800 per second. 706 00:39:10,800 --> 00:39:14,710 Shannon says we can get 42000 bits per second. 707 00:39:14,710 --> 00:39:17,680 Maybe we want to put a little coding in there. 708 00:39:17,680 --> 00:39:22,090 How does Shannon say you could get to this marvelous rate? 709 00:39:22,090 --> 00:39:25,490 He says, well, what you need to do is to choose a very, 710 00:39:25,490 --> 00:39:26,740 very long code. 711 00:39:29,230 --> 00:39:34,570 And if you choose it the right way and decode it in the right 712 00:39:34,570 --> 00:39:38,800 way, then I can prove to you that your probability of error 713 00:39:38,800 --> 00:39:40,820 will be very small. 714 00:39:40,820 --> 00:39:44,480 Of course, Shannon's proof is not constructive at all. 715 00:39:44,480 --> 00:39:47,450 If you've ever seen it, it involves just choosing a code 716 00:39:47,450 --> 00:39:53,260 at random, decoding it exhaustively by simply looking 717 00:39:53,260 --> 00:39:56,740 at all the two the nr code words, and finding which one 718 00:39:56,740 --> 00:39:59,020 is closest to the received signal, and 719 00:39:59,020 --> 00:40:00,650 that's not very practical. 720 00:40:00,650 --> 00:40:03,480 So there's nothing, in that sense, practical about 721 00:40:03,480 --> 00:40:05,290 Shannon's theorem. 722 00:40:08,100 --> 00:40:11,150 But Shannon says, the way you get there is you code. 723 00:40:11,150 --> 00:40:12,300 What is coding? 724 00:40:12,300 --> 00:40:15,200 Coding is introducing constraints into your 725 00:40:15,200 --> 00:40:16,030 transmission. 726 00:40:16,030 --> 00:40:20,020 Uncoded is where you just send an independent five bits in 727 00:40:20,020 --> 00:40:21,030 every symbol. 728 00:40:21,030 --> 00:40:24,190 And the next symbol is newly, you choose it new. 729 00:40:24,190 --> 00:40:26,830 So there's no dependencies between the symbols. 730 00:40:26,830 --> 00:40:28,850 It's just one shot communication, again 731 00:40:28,850 --> 00:40:31,540 and again and again. 732 00:40:31,540 --> 00:40:34,360 Coding is introducing constraints. 733 00:40:34,360 --> 00:40:37,720 From Shannon, it's clear you need to introduce constraints 734 00:40:37,720 --> 00:40:41,170 over very, very long sequences. 735 00:40:41,170 --> 00:40:44,210 Basically what we do is we take some alphabet of all the 736 00:40:44,210 --> 00:40:46,420 sequences that we could possibly send -- maybe the 737 00:40:46,420 --> 00:40:50,810 alphabet is all 1024 QAM sequences that we could 738 00:40:50,810 --> 00:40:52,180 possibly send. 739 00:40:52,180 --> 00:40:56,830 And then the code, by the code, we say we weed out a lot 740 00:40:56,830 --> 00:40:58,340 of those sequences. 741 00:40:58,340 --> 00:41:03,110 So we only send a small subset of all sequences that 742 00:41:03,110 --> 00:41:04,710 we might have sent. 743 00:41:04,710 --> 00:41:07,920 As a result, we make sure that there is a big distance 744 00:41:07,920 --> 00:41:10,780 between the sequences that are actually in the code. 745 00:41:10,780 --> 00:41:13,430 The fact that there's the much bigger distance than there is 746 00:41:13,430 --> 00:41:16,740 between the signal points in any particular small 747 00:41:16,740 --> 00:41:21,910 constellation means that if we do truly exhaustive maximum 748 00:41:21,910 --> 00:41:25,050 likelihood decoding, it's extremely unlikely we'll 749 00:41:25,050 --> 00:41:27,440 confuse one sequence for another. 750 00:41:27,440 --> 00:41:31,060 And therefore, we can get a very low error probability. 751 00:41:31,060 --> 00:41:37,580 And the whole course is about the history of people's 752 00:41:37,580 --> 00:41:45,730 efforts to realize what Shannon said on the particular 753 00:41:45,730 --> 00:41:46,860 channel that we're talking about. 754 00:41:46,860 --> 00:41:49,490 The additive white Gaussian noise channel, which, as you 755 00:41:49,490 --> 00:41:51,980 might imagine, is the canonical channel. 756 00:41:51,980 --> 00:41:55,460 But it's also proved to be a very good model for things 757 00:41:55,460 --> 00:41:58,810 like the deep space channel, the telephone line channel. 758 00:41:58,810 --> 00:42:01,260 In chapter one, I talk about the history and you can get 759 00:42:01,260 --> 00:42:03,980 some sense for the time flow. 760 00:42:03,980 --> 00:42:06,960 And it really took until -- 761 00:42:06,960 --> 00:42:12,040 well, depending on how you count it, up to about 1995 to 762 00:42:12,040 --> 00:42:17,900 when people could get effectively within tenths of a 763 00:42:17,900 --> 00:42:22,860 dB of the Shannon limit and you could say 764 00:42:22,860 --> 00:42:24,310 the problem was cracked. 765 00:42:24,310 --> 00:42:26,310 And research has continued for a little -- 766 00:42:26,310 --> 00:42:29,200 the past ten years, and this has been made more practical, 767 00:42:29,200 --> 00:42:33,410 and they've spent more channels refined in many, many 768 00:42:33,410 --> 00:42:34,660 directions. 769 00:42:34,660 --> 00:42:38,260 But really, you could say it was a 50 year project to take 770 00:42:38,260 --> 00:42:40,920 this existence theorem and make it real in a practical 771 00:42:40,920 --> 00:42:42,570 engineering sense. 772 00:42:42,570 --> 00:42:44,430 And that's the story of this course. 773 00:42:44,430 --> 00:42:46,070 I think it's a terrific story. 774 00:42:46,070 --> 00:42:50,120 So that's the way I like to package the course. 775 00:42:50,120 --> 00:42:54,730 How are we going to get to what Shannon said we could do? 776 00:42:54,730 --> 00:42:57,990 Alright, so that's the whole story. 777 00:42:57,990 --> 00:43:01,460 You having a problem hearing me? 778 00:43:01,460 --> 00:43:03,860 AUDIENCE: It's just the noise in the system. 779 00:43:03,860 --> 00:43:04,420 PROFESSOR: OK. 780 00:43:04,420 --> 00:43:06,780 Well, it's probably not an additive white 781 00:43:06,780 --> 00:43:08,030 Gaussian noise channel. 782 00:43:14,380 --> 00:43:16,180 One of the things about an additive white Gaussian noise 783 00:43:16,180 --> 00:43:19,090 channel is that there's no dependence in it. 784 00:43:19,090 --> 00:43:20,470 Time dependence in it. 785 00:43:20,470 --> 00:43:24,320 So a third channel of very very practical interest these 786 00:43:24,320 --> 00:43:30,370 days is a wireless channel, radio channel. 787 00:43:30,370 --> 00:43:34,830 Let's imagine it's a single user channel. 788 00:43:34,830 --> 00:43:36,040 Everybody in this -- everything in this course is 789 00:43:36,040 --> 00:43:38,830 going to be single user, point-to-point. 790 00:43:38,830 --> 00:43:41,360 But nonetheless, the FCC has said we 791 00:43:41,360 --> 00:43:43,990 can have five Megahertz. 792 00:43:43,990 --> 00:43:45,330 All right? 793 00:43:45,330 --> 00:43:48,430 And we've got to stay within that five Megahertz up to some 794 00:43:48,430 --> 00:43:53,740 little slopover that won't bother anybody. 795 00:43:53,740 --> 00:43:56,410 And again, your design problem could be, well, how do you 796 00:43:56,410 --> 00:44:00,240 send as much data as possible through a five megahertz 797 00:44:00,240 --> 00:44:02,430 wireless channel? 798 00:44:02,430 --> 00:44:05,470 If you take the Wireless course, and I encourage you to 799 00:44:05,470 --> 00:44:08,760 do that, or even if you listen to the last couple of 800 00:44:08,760 --> 00:44:12,200 lectures, I think, in 450, you know that there's a big 801 00:44:12,200 --> 00:44:15,960 difference in the wireless channel and it's time-varying. 802 00:44:15,960 --> 00:44:18,790 In particular, it has fading. 803 00:44:18,790 --> 00:44:21,690 So sometimes it's good and sometimes it's bad. 804 00:44:21,690 --> 00:44:23,270 You have outages. 805 00:44:23,270 --> 00:44:25,710 Not characteristic of the telephone line and the deep 806 00:44:25,710 --> 00:44:27,140 space channel. 807 00:44:27,140 --> 00:44:28,390 So that introduces -- well -- 808 00:44:30,690 --> 00:44:33,610 a whole lot, many more considerations. 809 00:44:33,610 --> 00:44:35,910 I'm not going to talk about that in this course. 810 00:44:35,910 --> 00:44:39,540 For that course, it's held in the next room immediately 811 00:44:39,540 --> 00:44:40,630 following this one. 812 00:44:40,630 --> 00:44:44,660 And I think it's an excellent course. 813 00:44:44,660 --> 00:44:45,130 OK. 814 00:44:45,130 --> 00:44:49,020 So that's the story of the course. 815 00:44:49,020 --> 00:44:53,280 We're going to go from day one knowing nothing about how to 816 00:44:53,280 --> 00:44:55,760 signal through this channel, except maybe we've taken some 817 00:44:55,760 --> 00:45:00,550 course that's introduced to us words like PAM and QAM. 818 00:45:00,550 --> 00:45:05,080 And by the end of the course, we're going to know how to go 819 00:45:05,080 --> 00:45:07,010 at the Shannon limit. 820 00:45:07,010 --> 00:45:09,030 And we're going to know a whole lot of techniques at 821 00:45:09,030 --> 00:45:09,630 that point. 822 00:45:09,630 --> 00:45:11,250 We're going to know how to code, we're going to know how 823 00:45:11,250 --> 00:45:14,650 to decode at least some representative examples, which 824 00:45:14,650 --> 00:45:20,920 I've chosen to be the ones that have been the most useful 825 00:45:20,920 --> 00:45:21,630 in practice. 826 00:45:21,630 --> 00:45:26,060 And they also tend to be the most interesting in theory. 827 00:45:26,060 --> 00:45:26,790 OK? 828 00:45:26,790 --> 00:45:28,040 Is that clear? 829 00:45:33,850 --> 00:45:36,150 OK. 830 00:45:36,150 --> 00:45:39,680 Just an aside on the first homework set. 831 00:45:39,680 --> 00:45:42,780 As I said, these are just supposed to be warm-up 832 00:45:42,780 --> 00:45:48,120 exercises that should get you comfortable with operating in 833 00:45:48,120 --> 00:45:53,710 the additive white Gaussian noise channel, which, at least 834 00:45:53,710 --> 00:46:00,210 some discrete time, turns out to be trying to code in 835 00:46:00,210 --> 00:46:01,500 Euclidean space. 836 00:46:01,500 --> 00:46:04,810 To encode and decode in Euclidean space. 837 00:46:04,810 --> 00:46:07,520 And the very first thing I do here is to 838 00:46:07,520 --> 00:46:10,080 give you my dB lecture. 839 00:46:10,080 --> 00:46:11,880 Now, how many people here would say they're 840 00:46:11,880 --> 00:46:15,350 comfortable using dB? 841 00:46:15,350 --> 00:46:17,060 Nobody. 842 00:46:17,060 --> 00:46:18,310 OK. 843 00:46:20,380 --> 00:46:25,810 Over the years, I've tried to encourage Bob Gallager to talk 844 00:46:25,810 --> 00:46:27,840 more about dB in 6.450. 845 00:46:27,840 --> 00:46:30,980 And now I believe he does give the dB lecture, but I don't 846 00:46:30,980 --> 00:46:34,550 believe he gives it with very much conviction. 847 00:46:34,550 --> 00:46:38,730 I'm convinced the dB are a very useful thing to know 848 00:46:38,730 --> 00:46:43,210 about, and I think Bob had a bad experience with dB early 849 00:46:43,210 --> 00:46:47,570 in life is really what the problem is. 850 00:46:47,570 --> 00:46:53,620 Some army sergeant said somebody was three dB taller 851 00:46:53,620 --> 00:46:56,406 than somebody else, and he was so revolted, he never wanted 852 00:46:56,406 --> 00:46:58,950 to talk about dB again. 853 00:46:58,950 --> 00:46:59,390 OK. 854 00:46:59,390 --> 00:47:04,870 dB, which stands for decibel, one tenth of a Bell -- 855 00:47:04,870 --> 00:47:09,250 the reason B is capitalized is it is after somebody's name, 856 00:47:09,250 --> 00:47:12,060 Alexander Graham Bell -- 857 00:47:12,060 --> 00:47:15,900 are very useful whenever you want to use logarithms. 858 00:47:15,900 --> 00:47:19,760 They basically are a system of logarithms. 859 00:47:19,760 --> 00:47:23,980 This gets all confused with ten log ten and 20 log ten and 860 00:47:23,980 --> 00:47:29,040 EE tests, but basically it's just a very well-designed, for 861 00:47:29,040 --> 00:47:31,875 humans, system of logarithms. 862 00:47:35,930 --> 00:47:36,670 Think about it. 863 00:47:36,670 --> 00:47:39,570 What would be the most convenient system of 864 00:47:39,570 --> 00:47:40,210 logarithms? 865 00:47:40,210 --> 00:47:42,740 We're going to use this whenever we have talking about 866 00:47:42,740 --> 00:47:48,130 factors, or the multiplication of a bunch of numbers. 867 00:47:48,130 --> 00:47:51,650 First of all, you'd want to have it be based on the base 868 00:47:51,650 --> 00:47:55,342 ten number system, because then it's -- 869 00:47:55,342 --> 00:47:58,170 you know, we have a decimal system, and we want to be able 870 00:47:58,170 --> 00:48:01,770 to very conveniently multiply by ten, 100, 871 00:48:01,770 --> 00:48:04,280 1000, and so forth. 872 00:48:04,280 --> 00:48:07,490 So a very natural logarithm to consider is 873 00:48:07,490 --> 00:48:09,550 log to the base ten. 874 00:48:09,550 --> 00:48:16,160 But so if we have log to the base ten of some factor -- 875 00:48:16,160 --> 00:48:20,860 let's say log to the base ten of 2 is 0.3010. 876 00:48:20,860 --> 00:48:23,800 And in fact, you know there are lots of tables that give 877 00:48:23,800 --> 00:48:26,000 logs to base ten. 878 00:48:26,000 --> 00:48:29,880 But it's just not so easy to remember that the log of base 879 00:48:29,880 --> 00:48:32,450 ten of 2 is 0.3. 880 00:48:32,450 --> 00:48:37,520 Basically if we take the log to the base ten of the whole 881 00:48:37,520 --> 00:48:40,180 range from, say, one to ten, that's really 882 00:48:40,180 --> 00:48:41,130 all we have to know. 883 00:48:41,130 --> 00:48:45,830 Because what we need is one factor of ten. 884 00:48:45,830 --> 00:48:50,880 This goes from zero to one. 885 00:48:50,880 --> 00:48:51,240 OK. 886 00:48:51,240 --> 00:48:55,970 What we'd really like to do is to spread this out for human 887 00:48:55,970 --> 00:48:57,890 factors engineering. 888 00:48:57,890 --> 00:49:02,250 We'd like to take ten to the base ten, which will now map 889 00:49:02,250 --> 00:49:05,770 the range from one to ten into a nice interval 890 00:49:05,770 --> 00:49:06,970 from zero to ten. 891 00:49:06,970 --> 00:49:11,850 That's very easy for people to understand. 892 00:49:11,850 --> 00:49:19,530 And ten to the log10 of 2 is 3.01 and so forth. 893 00:49:19,530 --> 00:49:19,950 OK. 894 00:49:19,950 --> 00:49:25,170 So if that's why we say a factor of 2 is equal to 3 dB. 895 00:49:25,170 --> 00:49:28,980 I find it's always helpful to say "a factor of" when talking 896 00:49:28,980 --> 00:49:31,110 about dB, just to remind ourselves, we're always 897 00:49:31,110 --> 00:49:33,100 talking about a multiplicative factor. 898 00:49:33,100 --> 00:49:36,915 So a factor of alpha is 10 log10 alpha dB. 899 00:49:39,950 --> 00:49:43,340 Now most of you are mathematically sophisticated 900 00:49:43,340 --> 00:49:46,800 enough to know this is basically the same thing as 901 00:49:46,800 --> 00:49:53,300 saying log to the base ten to the one-tenth of alpha. 902 00:49:53,300 --> 00:49:57,220 So we're really using a base here, which is ten to the 903 00:49:57,220 --> 00:50:03,250 one-tenth point 1, which is about 1.25 something or other. 904 00:50:03,250 --> 00:50:11,730 So the number whose one dB, which is beta to the one, is 905 00:50:11,730 --> 00:50:15,700 about 1.25. 906 00:50:15,700 --> 00:50:19,810 So we're just using a log to a certain base, which is very 907 00:50:19,810 --> 00:50:20,340 convenient. 908 00:50:20,340 --> 00:50:24,240 The base is ten to the one-tenth. 909 00:50:24,240 --> 00:50:28,320 And that's all it is, all right? 910 00:50:28,320 --> 00:50:31,470 Beyond that it's not very mysterious. 911 00:50:31,470 --> 00:50:33,960 So I think it's very useful just to 912 00:50:33,960 --> 00:50:40,790 remember a little table. 913 00:50:40,790 --> 00:50:44,180 alpha of one is how much in dB? 914 00:50:44,180 --> 00:50:45,730 It's zero. 915 00:50:45,730 --> 00:50:50,300 Or I write it two ways. 916 00:50:50,300 --> 00:50:54,000 Round numbers, or in any system, it's that. 917 00:50:54,000 --> 00:50:57,670 1.25 is about 1 dB. 918 00:50:57,670 --> 00:51:02,820 2 is 3 dB. 919 00:51:02,820 --> 00:51:04,910 Let's see. 920 00:51:04,910 --> 00:51:08,930 3 is about 4.8 dB. 921 00:51:08,930 --> 00:51:09,950 4 is what? 922 00:51:09,950 --> 00:51:14,260 4 is just 2 squared, so that has to be 6 dB. 923 00:51:14,260 --> 00:51:15,350 5 is what? 924 00:51:15,350 --> 00:51:18,690 5 is 10/2, so that has to be -- 925 00:51:18,690 --> 00:51:23,686 ten down here is 10 dB. 926 00:51:23,686 --> 00:51:26,530 That's exactly 10 dB. 927 00:51:26,530 --> 00:51:31,110 And so 5 has to be 7 dB. 928 00:51:31,110 --> 00:51:35,870 And 8 is 9 dB. 929 00:51:35,870 --> 00:51:37,360 And that's consistent, by the way. 930 00:51:37,360 --> 00:51:41,350 You see then ten eighth, which is 5/4, is 931 00:51:41,350 --> 00:51:44,550 1.25, which is 1 dB. 932 00:51:44,550 --> 00:51:48,090 And just, I find it useful in everyday life. 933 00:51:48,090 --> 00:51:50,460 Maybe that makes me an engineering nerd. 934 00:51:50,460 --> 00:51:53,740 But the first problem, for instance, has to do with 935 00:51:53,740 --> 00:51:55,250 compound interest. 936 00:51:55,250 --> 00:51:59,990 If you just remember dB values of things, you can do things 937 00:51:59,990 --> 00:52:03,560 like compound interest calculations in your head. 938 00:52:03,560 --> 00:52:06,570 Or I gave you a more engineering example over here. 939 00:52:06,570 --> 00:52:11,570 If you want to evaluate log2 of 37 dB, it's very easy. 940 00:52:11,570 --> 00:52:13,480 You just divide by 3. 941 00:52:13,480 --> 00:52:14,320 All right? 942 00:52:14,320 --> 00:52:19,070 So anyway. 943 00:52:19,070 --> 00:52:25,780 I encourage you to memorize this short table. 944 00:52:25,780 --> 00:52:27,500 There's some more things in it. 945 00:52:27,500 --> 00:52:30,860 And use it day by day, and become known 946 00:52:30,860 --> 00:52:32,185 as a real MIT person. 947 00:52:38,390 --> 00:52:40,510 Why schemes? 948 00:52:40,510 --> 00:52:41,760 Well, why not? 949 00:52:44,870 --> 00:52:46,290 All right. 950 00:52:46,290 --> 00:52:51,580 The other problems on the first homework set -- 951 00:52:51,580 --> 00:52:56,060 one's a quite algebraic construction, using Hadamard 952 00:52:56,060 --> 00:53:02,560 matrices of geometrical signal sets, like biorthogonal NS 953 00:53:02,560 --> 00:53:08,070 orthogonal and simplex signal sets, with a 954 00:53:08,070 --> 00:53:09,940 easy decoding algorithm. 955 00:53:09,940 --> 00:53:12,760 And that's good to know about. 956 00:53:12,760 --> 00:53:18,720 Next one is a QAM problem, asking you to look at various 957 00:53:18,720 --> 00:53:22,140 arrangements of signal points in two dimensions, and try to 958 00:53:22,140 --> 00:53:25,230 find the best ones for various criteria. 959 00:53:25,230 --> 00:53:29,890 The last one has to do with spherical shaping of large 960 00:53:29,890 --> 00:53:35,150 constellations, in many dimensions, with high spectral 961 00:53:35,150 --> 00:53:35,730 efficiency. 962 00:53:35,730 --> 00:53:38,020 Many bits per second per Hertz. 963 00:53:41,070 --> 00:53:41,670 OK. 964 00:53:41,670 --> 00:53:43,080 And that's chapter one. 965 00:53:53,130 --> 00:53:53,360 Ok. 966 00:53:53,360 --> 00:53:55,010 Chapter two we'll do very quickly. 967 00:53:55,010 --> 00:53:57,150 And then I could see we're not going to get to 968 00:53:57,150 --> 00:53:59,640 chapter three at all. 969 00:53:59,640 --> 00:54:00,760 And that's fine. 970 00:54:00,760 --> 00:54:02,010 You can read it. 971 00:54:04,720 --> 00:54:08,150 Ashish, if you want to say a word about chapter three next 972 00:54:08,150 --> 00:54:10,260 time, that would be fine. 973 00:54:12,840 --> 00:54:20,330 Chapter two is really about given a continuous time 974 00:54:20,330 --> 00:54:30,840 additive white Gaussian noise channel, with model Y of t 975 00:54:30,840 --> 00:54:37,320 equals X of t plus N of t, and here we have a certain power 976 00:54:37,320 --> 00:54:41,150 and we have a certain bandwidth limitation W, the 977 00:54:41,150 --> 00:54:44,560 same things we were using before, here we have a certain 978 00:54:44,560 --> 00:54:46,740 power spectral density N_0. 979 00:54:46,740 --> 00:54:49,165 So there are the parameters of the channel. 980 00:54:52,040 --> 00:55:02,390 Or I've collapsed them into two parameters, SNR and W. How 981 00:55:02,390 --> 00:55:09,410 can we convert this to a discrete time additive white 982 00:55:09,410 --> 00:55:15,110 Gaussian noise channel, which will be a channel 983 00:55:15,110 --> 00:55:16,970 model like this -- 984 00:55:16,970 --> 00:55:24,140 a sequence of symbols Yk is equal to a sequence of 985 00:55:24,140 --> 00:55:32,000 transmitted symbols Xk plus a sequence of noise symbols Nk. 986 00:55:32,000 --> 00:55:43,950 Where again, this is going to be IID Gaussian. 987 00:55:43,950 --> 00:55:46,630 That's what "white" is in discrete time. 988 00:55:46,630 --> 00:55:48,830 Independent Identically Distributed 989 00:55:48,830 --> 00:55:51,570 Gaussian random variables. 990 00:55:51,570 --> 00:55:56,346 This is going to have some variance S_N, it's going to 991 00:55:56,346 --> 00:55:59,830 have some power constraint S_X. 992 00:55:59,830 --> 00:56:04,820 We don't specifically see a bandwidth in here, but the 993 00:56:04,820 --> 00:56:07,240 bandwidth is essentially how many 994 00:56:07,240 --> 00:56:10,060 symbols we get per second. 995 00:56:10,060 --> 00:56:14,570 And I think all of you should have a sense that a bandwidth 996 00:56:14,570 --> 00:56:20,230 of W translates, through the sampling theorem, or through 997 00:56:20,230 --> 00:56:25,200 PAM modulation, or something else, is roughly equivalent in 998 00:56:25,200 --> 00:56:30,360 discrete time to two W real symbols per second, or to W 999 00:56:30,360 --> 00:56:34,260 complex symbols per second. 1000 00:56:34,260 --> 00:56:37,200 Now the -- sorry, did I screw you up? 1001 00:56:41,780 --> 00:56:43,030 Do I need to readjust? 1002 00:57:03,210 --> 00:57:03,960 All right. 1003 00:57:03,960 --> 00:57:05,210 Perfect. 1004 00:57:07,780 --> 00:57:14,650 There is a particular way we can -- 1005 00:57:14,650 --> 00:57:15,440 well. 1006 00:57:15,440 --> 00:57:20,650 So we're going to show that these two, the continuous time 1007 00:57:20,650 --> 00:57:23,390 and the discrete time channel -- 1008 00:57:23,390 --> 00:57:26,310 if we have a continuous time channel, we can get a discrete 1009 00:57:26,310 --> 00:57:31,690 time channel like that, where the values of the parameters, 1010 00:57:31,690 --> 00:57:34,810 signal-to-noise ratio, for instance, translate in a 1011 00:57:34,810 --> 00:57:39,240 natural way and that they're the same. 1012 00:57:39,240 --> 00:57:42,530 And furthermore, where we carry as much information on 1013 00:57:42,530 --> 00:57:44,360 the discrete time channel as we do on a 1014 00:57:44,360 --> 00:57:45,360 continuous time channel. 1015 00:57:45,360 --> 00:57:48,560 So there's no loss of optimality or generality in 1016 00:57:48,560 --> 00:57:50,140 doing this. 1017 00:57:50,140 --> 00:57:57,020 Now you've all, with a couple of exceptions, taken 6.450, so 1018 00:57:57,020 --> 00:57:59,410 you know how to do this. 1019 00:57:59,410 --> 00:58:07,070 One way of doing this in a fairly engineering sense is 1020 00:58:07,070 --> 00:58:08,320 orthonormal PAM -- 1021 00:58:11,200 --> 00:58:14,010 Pulse Amplitude Modulation. 1022 00:58:14,010 --> 00:58:18,630 And you should know how this goes. 1023 00:58:18,630 --> 00:58:23,540 We're going to take in a sequence X, which I might 1024 00:58:23,540 --> 00:58:29,340 write out as a sequence of real symbol levels Xk, perhaps 1025 00:58:29,340 --> 00:58:31,150 chosen from a PAM alphabet. 1026 00:58:31,150 --> 00:58:34,130 I'm not even going to specify. 1027 00:58:34,130 --> 00:58:35,780 And what do I need to know here? 1028 00:58:35,780 --> 00:58:40,490 This is going to be at two W symbols per second, real 1029 00:58:40,490 --> 00:58:41,740 symbols per second. 1030 00:58:44,910 --> 00:58:45,350 OK. 1031 00:58:45,350 --> 00:58:47,640 That's my transmitted sequence that I want to get 1032 00:58:47,640 --> 00:58:48,890 to the other end. 1033 00:58:51,000 --> 00:58:53,820 PAM modulator -- 1034 00:58:53,820 --> 00:58:58,280 so this is PAM modulator -- 1035 00:58:58,280 --> 00:59:04,610 is specified by a certain pulse response p of t. 1036 00:59:04,610 --> 00:59:13,220 and I'm going to require that this be orthonormal in the 1037 00:59:13,220 --> 00:59:14,230 following sense. 1038 00:59:14,230 --> 00:59:20,940 That the inner product between p of T minus kT and p of T 1039 00:59:20,940 --> 00:59:26,490 minus jT is equal to the chronic or delta. 1040 00:59:26,490 --> 00:59:27,250 Delta kj. 1041 00:59:27,250 --> 00:59:33,070 So that the time shifts of this single basic pulse by T, 1042 00:59:33,070 --> 00:59:41,710 where T equals one over two w, the symbol interval, are going 1043 00:59:41,710 --> 00:59:44,050 to be orthogonal to each other, and what's more, 1044 00:59:44,050 --> 00:59:44,770 orthonormal. 1045 00:59:44,770 --> 00:59:54,350 So in effect, I've got a signal space consisting of all 1046 00:59:54,350 --> 00:59:59,180 the linear combinations of the time shifts of p of T by 1047 00:59:59,180 --> 01:00:02,390 integer values of T, and I'm going to send something that's 1048 01:00:02,390 --> 01:00:06,550 in that signal space simply by amplitude modulating each one 1049 01:00:06,550 --> 01:00:08,770 that comes along. 1050 01:00:08,770 --> 01:00:09,410 All right. 1051 01:00:09,410 --> 01:00:13,760 So what I get out here is the continuous time signal X of T, 1052 01:00:13,760 --> 01:00:17,754 sum of Xk p of T minus kT. 1053 01:00:17,754 --> 01:00:19,004 That's what the modulator does. 1054 01:00:25,470 --> 01:00:30,290 Then the channel, continuous time channel -- so this is 1055 01:00:30,290 --> 01:00:31,900 discrete time here. 1056 01:00:31,900 --> 01:00:34,570 Now I'm into continuous time. 1057 01:00:34,570 --> 01:00:40,160 The channel is going to add white Gaussian noise N of T 1058 01:00:40,160 --> 01:00:49,500 with N_0 power spectral density to give me channel 1059 01:00:49,500 --> 01:00:55,120 output Y of T is X of T plus N of T. 1060 01:00:55,120 --> 01:01:01,800 And now I need a receiver to get back to discrete time. 1061 01:01:01,800 --> 01:01:13,180 The receiver will simply be a sampled matched filter, which 1062 01:01:13,180 --> 01:01:18,420 has many properties which you should recall. 1063 01:01:18,420 --> 01:01:20,460 Physically what does it look like? 1064 01:01:20,460 --> 01:01:26,740 We pass Y of T through p of minus T. The matched filter's 1065 01:01:26,740 --> 01:01:27,740 turnaround in time. 1066 01:01:27,740 --> 01:01:30,250 What it's doing is performing an inner product. 1067 01:01:30,250 --> 01:01:38,100 We then sample at T samples per second. 1068 01:01:38,100 --> 01:01:40,300 Perfectly phased. 1069 01:01:40,300 --> 01:01:45,960 And as a result, we get out some sequence y equal Yk. 1070 01:01:48,740 --> 01:01:55,060 And the purpose of this is so that Yk is the inner product 1071 01:01:55,060 --> 01:02:00,190 of Y of t with p of T minus kT. 1072 01:02:08,730 --> 01:02:13,190 And you should be aware that this is a realization -- 1073 01:02:13,190 --> 01:02:16,710 this is a correlator-type inner product. 1074 01:02:16,710 --> 01:02:19,490 Correlate and sample inner product. 1075 01:02:19,490 --> 01:02:20,920 All right? 1076 01:02:20,920 --> 01:02:25,140 So what are some of the properties that you developed 1077 01:02:25,140 --> 01:02:30,700 ad nauseum in 6.450? 1078 01:02:30,700 --> 01:02:37,520 First of all, if we take the signal part of this, and we 1079 01:02:37,520 --> 01:02:42,400 have the sampler phasing right, what do we get out for 1080 01:02:42,400 --> 01:02:43,820 the signal part of Yk? 1081 01:02:47,520 --> 01:02:51,470 We have X of T is this. 1082 01:02:51,470 --> 01:02:56,560 If we correlate that against, let me say, p of T minus jT 1083 01:02:56,560 --> 01:03:00,320 for all j, we're going to get zero for everything except for 1084 01:03:00,320 --> 01:03:01,510 the desired sample. 1085 01:03:01,510 --> 01:03:03,920 For the desired sample, we're just going to get the desired 1086 01:03:03,920 --> 01:03:05,150 sample out. 1087 01:03:05,150 --> 01:03:10,540 So from this, we get Yk is equal to Xk plus 1088 01:03:10,540 --> 01:03:13,910 the noise term, Nk. 1089 01:03:13,910 --> 01:03:16,330 In other words, there's no intersymbol interference. 1090 01:03:22,790 --> 01:03:25,110 The noise term -- 1091 01:03:29,840 --> 01:03:32,790 by taking these inner products, what are these? 1092 01:03:32,790 --> 01:03:39,570 These are the coefficients of yK in the signal space, under 1093 01:03:39,570 --> 01:03:47,200 -- these are an orthonormal expansion of Y of T. Those 1094 01:03:47,200 --> 01:03:51,650 components which lie in the signal space. 1095 01:03:51,650 --> 01:03:53,390 Can we ignore the components that don't lie 1096 01:03:53,390 --> 01:03:55,350 in the signal space? 1097 01:03:55,350 --> 01:03:55,610 Yes. 1098 01:03:55,610 --> 01:03:58,370 If it's out of white Gaussian noise by the theorem of 1099 01:03:58,370 --> 01:04:00,670 irrelevance or whatever it was called. 1100 01:04:00,670 --> 01:04:08,910 There is no information about the Xs contained in any part 1101 01:04:08,910 --> 01:04:12,988 of Y or T that is orthogonal to the signal space, in the 1102 01:04:12,988 --> 01:04:15,450 orthogonal space. 1103 01:04:15,450 --> 01:04:18,720 So we simply take the parts that are in the signal space. 1104 01:04:18,720 --> 01:04:24,230 And another property is that this is white Gaussian noise. 1105 01:04:24,230 --> 01:04:34,860 Then Nk is simply an IID jointly Gaussian sequence with 1106 01:04:34,860 --> 01:04:42,060 variance S_N equal to N_0 over 2 if the power of spectral 1107 01:04:42,060 --> 01:04:43,310 density here was N_0. 1108 01:04:52,350 --> 01:04:55,610 And one other property of I should have mentioned here is 1109 01:04:55,610 --> 01:04:59,650 that again by the orthonormal property, as you would think 1110 01:04:59,650 --> 01:05:03,920 intuitively, if we sent two W symbols per second, and each 1111 01:05:03,920 --> 01:05:11,242 one is limited to power S_X, then this will 1112 01:05:11,242 --> 01:05:12,492 have power 2 W S_X. 1113 01:05:17,830 --> 01:05:20,040 The powers are the same in discrete time 1114 01:05:20,040 --> 01:05:23,570 and continuous time. 1115 01:05:23,570 --> 01:05:29,940 So what do I get? 1116 01:05:29,940 --> 01:05:33,620 Let me now show the SNR. 1117 01:05:36,206 --> 01:05:41,785 Let me compute it in discrete time and in continuous time. 1118 01:05:41,785 --> 01:05:52,780 The SNR in discrete time is 2 W S_X over S_N, which 1119 01:05:52,780 --> 01:05:56,320 was N_0 over 2. 1120 01:06:04,860 --> 01:06:07,020 I'm not seeing the two's fall in the right place. 1121 01:06:07,020 --> 01:06:08,270 What am I doing wrong? 1122 01:06:12,210 --> 01:06:15,730 So this is the power. 1123 01:06:23,220 --> 01:06:26,640 Discrete time and signal-to-noise ratio is just 1124 01:06:26,640 --> 01:06:29,370 S_X over S_N. 1125 01:06:29,370 --> 01:06:34,700 Now S_X is P over 2W. 1126 01:06:34,700 --> 01:06:38,744 S_N is N_0 over 2. 1127 01:06:38,744 --> 01:06:40,046 Now I'm happy. 1128 01:06:40,046 --> 01:06:42,380 It's P over W N_0. 1129 01:06:42,380 --> 01:06:47,626 Continuous time is p over W N_0. 1130 01:06:47,626 --> 01:06:48,090 OK. 1131 01:06:48,090 --> 01:06:49,690 The point is, the signal-to-noise 1132 01:06:49,690 --> 01:06:52,194 ratio is the same. 1133 01:06:55,050 --> 01:07:06,365 In discrete time, talking about bandwidth, the bandwidth 1134 01:07:06,365 --> 01:07:10,119 is really 2W symbols per second. 1135 01:07:10,119 --> 01:07:15,990 So that's our measure of bandwidth in discrete time. 1136 01:07:15,990 --> 01:07:22,655 And that is approximately equal to W Hertz 1137 01:07:22,655 --> 01:07:23,625 in continuous time. 1138 01:07:23,625 --> 01:07:25,565 Now, I haven't proved that to you yet, have I? 1139 01:07:29,970 --> 01:07:34,890 How to prove that we can get a bandwidth as 1140 01:07:34,890 --> 01:07:38,200 small as W but no smaller. 1141 01:07:38,200 --> 01:07:43,140 Again, you've done this in 6.450, so I hope I can just 1142 01:07:43,140 --> 01:07:44,390 remind you. 1143 01:07:46,552 --> 01:07:55,350 If the shifts of p of T are orthonormal, that says 1144 01:07:55,350 --> 01:08:11,920 something about p of T star p of T, which is the correlation 1145 01:08:11,920 --> 01:08:14,270 of p of T with itself. 1146 01:08:14,270 --> 01:08:22,010 It means g of T has to be 1 to equal 0, and it has to be 0 at 1147 01:08:22,010 --> 01:08:25,589 all other times, at all other integer times. 1148 01:08:28,790 --> 01:08:35,420 Which means the Fourier transform, power spectral 1149 01:08:35,420 --> 01:08:44,500 density of p of T, it has to satisfy the Aliasing Theorem, 1150 01:08:44,500 --> 01:08:50,680 or the zero ISI, the Nyquist criterion for zero ISI, which 1151 01:08:50,680 --> 01:08:55,350 basically means the alias version of the power spectral 1152 01:08:55,350 --> 01:09:06,260 density of this has to add up to a perfect brick wall 1153 01:09:06,260 --> 01:09:11,504 response between 1 over 2T minus 1 over 2T and 1 over 2T 1154 01:09:11,504 --> 01:09:13,810 in the frequency domain. 1155 01:09:13,810 --> 01:09:16,695 And there are simple ways to make it do that. 1156 01:09:19,640 --> 01:09:25,939 Which is simply to have a roll-off, a real frequency 1157 01:09:25,939 --> 01:09:29,470 response, with a roll-off that's really sharp as you 1158 01:09:29,470 --> 01:09:32,380 like above 1 over 2T. 1159 01:09:32,380 --> 01:09:44,729 But so we can say the nominal Nyquist bandwidth equals 1 1160 01:09:44,729 --> 01:09:51,020 over 2T, or T is 1 over 2W. 1161 01:09:51,020 --> 01:09:59,510 So it's W. 1162 01:09:59,510 --> 01:10:03,920 So the moral of this is that we can 1163 01:10:03,920 --> 01:10:07,840 design orthonormal pulses. 1164 01:10:07,840 --> 01:10:12,380 If we have a similar interval of T equals 1 over 2W, we can 1165 01:10:12,380 --> 01:10:15,440 design an orthonormal signal set that satisfies this 1166 01:10:15,440 --> 01:10:24,500 condition for any bandwidth that's not much greater than 1167 01:10:24,500 --> 01:10:27,570 W. All right? 1168 01:10:27,570 --> 01:10:31,690 And it's obvious from the Aliasing theorem that we can't 1169 01:10:31,690 --> 01:10:37,520 do it if we try to choose some bandwidth that's less than W. 1170 01:10:37,520 --> 01:10:43,140 So that's what justifies our saying, if we are 2M symbols 1171 01:10:43,140 --> 01:10:46,000 per second, we're going to have to use at least W Hertz 1172 01:10:46,000 --> 01:10:47,960 of bandwidth. 1173 01:10:47,960 --> 01:10:51,760 But we don't have to use very much more than W Hertz of 1174 01:10:51,760 --> 01:10:55,530 bandwidth if we're using orthonormal PAM as our 1175 01:10:55,530 --> 01:10:58,220 signaling scheme. 1176 01:10:58,220 --> 01:11:03,400 So we call this the nominal bandwidth. 1177 01:11:03,400 --> 01:11:06,270 In real life, there will be a little roll-off -- 1178 01:11:06,270 --> 01:11:07,620 five percent, ten percent. 1179 01:11:11,080 --> 01:11:15,425 And that's a fudge factor in going from discrete time to 1180 01:11:15,425 --> 01:11:16,060 continuous time. 1181 01:11:16,060 --> 01:11:22,182 But it's fair that we can get as close to W as we like. 1182 01:11:22,182 --> 01:11:25,668 Certainly in the approaching Shannon limit theoretically, 1183 01:11:25,668 --> 01:11:29,747 we would say that we can get as close to W as we need to, I 1184 01:11:29,747 --> 01:11:32,110 should specify. 1185 01:11:32,110 --> 01:11:33,360 All right? 1186 01:11:37,740 --> 01:11:43,590 So there's one other parameter that Ashish is going to be 1187 01:11:43,590 --> 01:11:48,160 talking a lot more about in the next lectures which is 1188 01:11:48,160 --> 01:11:49,934 called spectral efficiency. 1189 01:11:54,290 --> 01:11:58,634 So far I haven't told you what information the sequence of 1190 01:11:58,634 --> 01:12:01,410 symbols is carrying. 1191 01:12:01,410 --> 01:12:05,880 But let's suppose it's carrying R bits per second. 1192 01:12:05,880 --> 01:12:06,090 All right? 1193 01:12:06,090 --> 01:12:06,910 We have a modem. 1194 01:12:06,910 --> 01:12:10,200 It's a 64000 bits per second modem or whatever. 1195 01:12:10,200 --> 01:12:12,260 R is 64000. 1196 01:12:12,260 --> 01:12:15,090 Or whatever it is. 1197 01:12:15,090 --> 01:12:18,371 My objective is to make R as high as possible subject to 1198 01:12:18,371 --> 01:12:20,060 the Shannon limit. 1199 01:12:20,060 --> 01:12:25,540 If I am sending R bits per second across a channel which 1200 01:12:25,540 --> 01:12:30,260 is W Hertz Y, in continuous time, I'm simply going to 1201 01:12:30,260 --> 01:12:33,200 define, I'm always going to write this as rho. 1202 01:12:36,952 --> 01:12:42,310 And I'm going to write it simply as a rate divided by 1203 01:12:42,310 --> 01:12:44,020 the bandwidth. 1204 01:12:44,020 --> 01:12:46,290 So my telephone line case, for instance. 1205 01:12:46,290 --> 01:12:52,078 If I was sending 40000 bits per second in 3700 Hertz 1206 01:12:52,078 --> 01:12:57,300 bandwidth, I'd be sending 12 bits per second per Hertz. 1207 01:12:57,300 --> 01:13:00,360 That's why we say that. 1208 01:13:00,360 --> 01:13:01,860 That's clearly a key thing. 1209 01:13:01,860 --> 01:13:04,360 How much data can I jam in? 1210 01:13:04,360 --> 01:13:06,890 We expect it to go linearly with the bandwidth. 1211 01:13:06,890 --> 01:13:12,350 Rho is the measure of how much data per unit of bandwidth. 1212 01:13:12,350 --> 01:13:17,500 What does that translate into in discrete time? 1213 01:13:17,500 --> 01:13:22,650 Well, I'm dividing up this rate into 1214 01:13:22,650 --> 01:13:25,680 2W symbols per second. 1215 01:13:25,680 --> 01:13:28,690 How many bits am I sending per symbol? 1216 01:13:31,470 --> 01:13:35,115 I'm sending R over 2W per symbol. 1217 01:13:39,800 --> 01:13:47,330 So simply rho over two bits per symbol, or since I'm 1218 01:13:47,330 --> 01:13:50,510 always going to think of Euclidean space, I will often 1219 01:13:50,510 --> 01:13:53,160 write that as dimension-- 1220 01:13:53,160 --> 01:13:56,495 rho over 2 bits per dimension. 1221 01:13:56,495 --> 01:13:56,990 All right. 1222 01:13:56,990 --> 01:14:01,310 Well that's not the exact translation I like. 1223 01:14:01,310 --> 01:14:04,920 I'd like to talk in terms of rho. 1224 01:14:04,920 --> 01:14:09,002 So here, rho can be evaluated at the number of bits I'm 1225 01:14:09,002 --> 01:14:12,078 sending per two symbols, or per two dimensions. 1226 01:14:16,780 --> 01:14:16,960 All right. 1227 01:14:16,960 --> 01:14:23,492 So for the discrete time system, our major spectral 1228 01:14:23,492 --> 01:14:25,862 efficiency is just going to be the number of bits I'm sending 1229 01:14:25,862 --> 01:14:27,284 every two dimensions. 1230 01:14:30,140 --> 01:14:33,320 It turns out if you get into this game, you feel that two 1231 01:14:33,320 --> 01:14:34,210 dimensions are more 1232 01:14:34,210 --> 01:14:36,980 fundamental than one dimension. 1233 01:14:36,980 --> 01:14:43,040 Which may be a manifestation of the fact that complex is 1234 01:14:43,040 --> 01:14:46,100 more fundamental than real. 1235 01:14:46,100 --> 01:14:49,080 Really we want to talk about the amount of information 1236 01:14:49,080 --> 01:14:55,654 we're sending per a single complex dimension, or we can 1237 01:14:55,654 --> 01:14:57,304 say two real dimensions. 1238 01:14:57,304 --> 01:14:59,660 Geometrically they amount to the same thing. 1239 01:14:59,660 --> 01:15:02,760 Again, in the notes, there are discussions about how you go 1240 01:15:02,760 --> 01:15:05,300 back and forth, and so forth. 1241 01:15:05,300 --> 01:15:09,840 So the bottom line here is that we can talk about 1242 01:15:09,840 --> 01:15:11,760 spectral efficiency, which is obviously a 1243 01:15:11,760 --> 01:15:13,310 continuous time concept. 1244 01:15:13,310 --> 01:15:16,470 But we can talk about it in the discrete time domain. 1245 01:15:16,470 --> 01:15:19,100 And what it's going to mean simply is the rate that we're 1246 01:15:19,100 --> 01:15:22,084 managing to send, by particular scheme, if it's per 1247 01:15:22,084 --> 01:15:23,290 two dimensions. 1248 01:15:23,290 --> 01:15:27,425 For instance, 32 QAM has a spectral efficiency of-- 1249 01:15:34,400 --> 01:15:36,745 I didn't make myself clear at all. 1250 01:15:36,745 --> 01:15:38,245 AUDIENCE: Five bits per two. 1251 01:15:38,245 --> 01:15:41,280 PROFESSOR: Five bits per two dimensions, all right? 1252 01:15:41,280 --> 01:15:45,600 So that's the basic spectral efficiency of a 32 PAM 1253 01:15:45,600 --> 01:15:46,650 modulation scheme. 1254 01:15:46,650 --> 01:15:49,760 2 PAM, plus or minus one. 1255 01:15:49,760 --> 01:15:52,384 What's the spectral efficiency of that if it's for two 1256 01:15:52,384 --> 01:15:53,634 dimensions? 1257 01:15:55,550 --> 01:15:58,850 It's a trick question. 1258 01:15:58,850 --> 01:16:01,765 One bit per one dimension. 1259 01:16:01,765 --> 01:16:05,020 It's two bits per two dimensions. 1260 01:16:05,020 --> 01:16:10,400 If it's 2 PAM, it's actually the same thing as 4 QAM, 1261 01:16:10,400 --> 01:16:14,860 because 4 QAM is just doing 2 PAM place in 1262 01:16:14,860 --> 01:16:17,800 two successive symbols. 1263 01:16:17,800 --> 01:16:23,882 So 2 PAM, you could send two bits per two dimensions. 1264 01:16:23,882 --> 01:16:26,542 4 QAM is also two bits per two dimensions Ok. 1265 01:16:30,470 --> 01:16:32,020 Similarly for any [UNINTELLIGIBLE], we'll be 1266 01:16:32,020 --> 01:16:35,170 talking about codes which cover many dimensions. 1267 01:16:35,170 --> 01:16:40,730 If you send R bits in N dimensions, then the spectral 1268 01:16:40,730 --> 01:16:46,130 efficiency is going to be 2R over N. So R over N in one 1269 01:16:46,130 --> 01:16:49,040 dimension, 2R over N in two dimensions. 1270 01:16:49,040 --> 01:16:52,670 I guess initially, this catches people up. 1271 01:16:52,670 --> 01:16:56,600 Because it's more difficult to normalize for two dimensions 1272 01:16:56,600 --> 01:16:57,750 than for one dimension. 1273 01:16:57,750 --> 01:16:59,610 But come on, suck it up. 1274 01:16:59,610 --> 01:17:01,215 You're a graduate student at MIT. 1275 01:17:01,215 --> 01:17:04,170 You can do it for two dimensions. 1276 01:17:04,170 --> 01:17:09,340 And I assert that it really is the more natural thing. 1277 01:17:09,340 --> 01:17:12,550 In particular, makes this nice correspondence between 1278 01:17:12,550 --> 01:17:15,020 discrete and noncontinuous time parameters. 1279 01:17:15,020 --> 01:17:17,232 So the SNR is the same. 1280 01:17:17,232 --> 01:17:19,982 The bandwidth has different interpretations, but we're 1281 01:17:19,982 --> 01:17:22,928 talking with W in both cases. 1282 01:17:22,928 --> 01:17:28,200 Spectral efficiency, we can measure it as either discrete 1283 01:17:28,200 --> 01:17:30,192 time or continuous time. 1284 01:17:30,192 --> 01:17:32,300 And oh, by the way. 1285 01:17:32,300 --> 01:17:36,671 What is the channel capacity formula in terms of spectral 1286 01:17:36,671 --> 01:17:38,643 efficiency? 1287 01:17:38,643 --> 01:17:45,180 The channel capacity was basically the data rate has to 1288 01:17:45,180 --> 01:17:50,210 be less than W log2 of one plus SNR. 1289 01:17:52,900 --> 01:17:54,924 Find an equivalent way. 1290 01:17:54,924 --> 01:17:58,830 So this is writing things in terms of bits per second, or 1291 01:17:58,830 --> 01:18:04,210 equivalently rho, which is R over W, has to be less than 1292 01:18:04,210 --> 01:18:08,350 log2 of one plus SNR. 1293 01:18:08,350 --> 01:18:12,450 This is in bits per second per Hertz. 1294 01:18:12,450 --> 01:18:16,893 Or in discrete time, we have the same formula, except this 1295 01:18:16,893 --> 01:18:20,060 becomes 50 bits per two dimensions. 1296 01:18:20,060 --> 01:18:23,098 And so things have been normalized so that now you 1297 01:18:23,098 --> 01:18:25,344 only have a single parameter to worry about. 1298 01:18:25,344 --> 01:18:27,340 What special efficiencies can you achieve? 1299 01:18:27,340 --> 01:18:31,560 It is purely a matter of what SNR you have. 1300 01:18:31,560 --> 01:18:35,540 Telephone line channel 37 dB, then you get spectral 1301 01:18:35,540 --> 01:18:37,651 efficiency of 12 and one-third by Shannon. 1302 01:18:40,950 --> 01:18:42,400 OK. 1303 01:18:42,400 --> 01:18:46,200 So I'm sorry not to be able to give you the very lovely 1304 01:18:46,200 --> 01:18:50,193 derivation of this formula as a little bit of culture. 1305 01:18:50,193 --> 01:18:52,445 I encourage you to read chapter three. 1306 01:18:52,445 --> 01:18:54,500 You will not be held responsible for any of these 1307 01:18:54,500 --> 01:18:55,850 first three chapters. 1308 01:18:55,850 --> 01:18:59,132 They're all more or less just for orientation. 1309 01:18:59,132 --> 01:19:02,480 Next time, Ashish will get into the real stuff. 1310 01:19:02,480 --> 01:19:03,730 Any final questions? 1311 01:19:07,640 --> 01:19:08,920 OK. 1312 01:19:08,920 --> 01:19:10,170 See you in two weeks.