1 00:00:00,000 --> 00:00:04,970 2 00:00:04,970 --> 00:00:05,285 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 3 00:00:05,285 --> 00:00:06,535 JOHANN SEBASTIAN BACH] 4 00:00:06,535 --> 00:00:18,910 5 00:00:18,910 --> 00:00:22,140 PROFESSOR: Well, there's one bit of mystery left, which I'd 6 00:00:22,140 --> 00:00:24,440 like to get rid of right now. 7 00:00:24,440 --> 00:00:28,210 And that's that we've been blithely doing things like 8 00:00:28,210 --> 00:00:33,660 cons assuming there's always another one. 9 00:00:33,660 --> 00:00:37,060 That we've been doing these things like car-ing and 10 00:00:37,060 --> 00:00:38,750 cdr-ing and assuming that we had some idea 11 00:00:38,750 --> 00:00:40,020 how this can be done. 12 00:00:40,020 --> 00:00:43,800 Now indeed we said that that's equivalent to having 13 00:00:43,800 --> 00:00:45,780 procedures. 14 00:00:45,780 --> 00:00:48,360 But that doesn't really solve the problem, because the 15 00:00:48,360 --> 00:00:50,470 procedure need all sorts of complicated mechanisms like 16 00:00:50,470 --> 00:00:53,010 environment structures and things like that to work. 17 00:00:53,010 --> 00:00:55,770 And those were ultimately made out of conses in the model 18 00:00:55,770 --> 00:00:59,380 that we had, so that really doesn't solve the problem. 19 00:00:59,380 --> 00:01:02,860 Now the problem here is the glue the data 20 00:01:02,860 --> 00:01:04,760 structure's made out of. 21 00:01:04,760 --> 00:01:07,370 What kind of possible thing could it be? 22 00:01:07,370 --> 00:01:11,060 We've been showing you things like a machine, a computer 23 00:01:11,060 --> 00:01:14,700 that has a controller, and some 24 00:01:14,700 --> 00:01:16,980 registers, and maybe a stack. 25 00:01:16,980 --> 00:01:18,270 And we haven't said anything about, for 26 00:01:18,270 --> 00:01:20,570 example, larger memory. 27 00:01:20,570 --> 00:01:23,740 And I think that's what we have to worry about right now. 28 00:01:23,740 --> 00:01:27,160 But just to make it perfectly clear that this is an 29 00:01:27,160 --> 00:01:31,320 inessential, purely implementational thing, I'd 30 00:01:31,320 --> 00:01:33,500 like to show you, for example, how you can do it 31 00:01:33,500 --> 00:01:34,800 all with the numbers. 32 00:01:34,800 --> 00:01:37,590 That's an easy one. 33 00:01:37,590 --> 00:01:45,020 Famous fellow by the name of Godel, a logician at the end 34 00:01:45,020 --> 00:01:51,050 of the 1930s, invented a very clever way of encoding the 35 00:01:51,050 --> 00:01:54,320 complicated expressions as numbers. 36 00:01:54,320 --> 00:01:55,540 For example-- 37 00:01:55,540 --> 00:01:58,250 I'm not saying exactly what Godel's scheme is, because he 38 00:01:58,250 --> 00:01:59,660 didn't use words like cons. 39 00:01:59,660 --> 00:02:03,090 He had other kinds of ways of combining to make expressions. 40 00:02:03,090 --> 00:02:05,860 But he said, I'm going to assign a number to every 41 00:02:05,860 --> 00:02:07,920 algebraic expression. 42 00:02:07,920 --> 00:02:09,970 And the way I'm going to manufacture these numbers is 43 00:02:09,970 --> 00:02:12,470 by combining the numbers of the parts. 44 00:02:12,470 --> 00:02:15,880 So for example, what we were doing our world, we could say 45 00:02:15,880 --> 00:02:34,130 that if objects are represented by numbers, then 46 00:02:34,130 --> 00:02:42,660 cons of x and y could be represented by 2 to the x 47 00:02:42,660 --> 00:02:46,130 times 2 to the y. 48 00:02:46,130 --> 00:02:49,560 Because then we could extract the parts. 49 00:02:49,560 --> 00:02:57,500 We could say, for example, that then car of, say, x is 50 00:02:57,500 --> 00:03:06,690 the number of factors of 2 in x. 51 00:03:06,690 --> 00:03:10,690 And of course cdr is the same thing. 52 00:03:10,690 --> 00:03:16,510 It's the number of factors of 3 in x. 53 00:03:16,510 --> 00:03:19,660 Now this is a perfectly reasonable scheme, except for 54 00:03:19,660 --> 00:03:22,870 the fact that the numbers rapidly get to be much larger 55 00:03:22,870 --> 00:03:25,500 in number of digits than the number of 56 00:03:25,500 --> 00:03:27,950 protons in the universe. 57 00:03:27,950 --> 00:03:30,420 So there's no easy way to use this scheme other than the 58 00:03:30,420 --> 00:03:33,430 theoretical one. 59 00:03:33,430 --> 00:03:37,010 On the other hand, there are other ways of representing 60 00:03:37,010 --> 00:03:38,450 these things. 61 00:03:38,450 --> 00:03:44,010 We have been thinking in terms of little boxes. 62 00:03:44,010 --> 00:03:47,000 We've been thinking about our cons structures as looking 63 00:03:47,000 --> 00:03:50,280 sort of like this. 64 00:03:50,280 --> 00:03:53,610 They're little pigeon holes with things in them. 65 00:03:53,610 --> 00:03:57,210 And of course we arrange them in little trees. 66 00:03:57,210 --> 00:04:00,680 I wish that the semiconductor manufacturers would supply me 67 00:04:00,680 --> 00:04:04,280 with something appropriate for this, but actually what they 68 00:04:04,280 --> 00:04:09,380 do supply me with is a linear memory. 69 00:04:09,380 --> 00:04:15,170 Memory is sort of a big pile of pigeonholes, 70 00:04:15,170 --> 00:04:17,720 pigeonholes like this. 71 00:04:17,720 --> 00:04:21,470 Each of which can hold a certain sized object, a fixed 72 00:04:21,470 --> 00:04:23,390 size object. 73 00:04:23,390 --> 00:04:25,890 So, for example, a complicated list with 25 elements won't 74 00:04:25,890 --> 00:04:28,550 fit in one of these. 75 00:04:28,550 --> 00:04:30,600 However, each of these is indexed by an address. 76 00:04:30,600 --> 00:04:33,970 77 00:04:33,970 --> 00:04:36,750 So the address might be zero here, one here, two here, 78 00:04:36,750 --> 00:04:38,060 three here, and so on. 79 00:04:38,060 --> 00:04:40,400 That we write these down as numbers is unimportant. 80 00:04:40,400 --> 00:04:42,710 What matters is that they're distinct as a way to get to 81 00:04:42,710 --> 00:04:44,970 the next one. 82 00:04:44,970 --> 00:04:48,300 And inside of each of these, we can stuff something into 83 00:04:48,300 --> 00:04:49,530 these pigeonholes. 84 00:04:49,530 --> 00:04:52,300 That's what memory is like, for those of you who haven't 85 00:04:52,300 --> 00:04:53,550 built a computer. 86 00:04:53,550 --> 00:04:56,690 87 00:04:56,690 --> 00:04:59,280 Now the problem is how are we going to impose on this type 88 00:04:59,280 --> 00:05:03,290 of structure, this nice tree structure. 89 00:05:03,290 --> 00:05:05,480 Well it's not very hard, and there have been numerous 90 00:05:05,480 --> 00:05:06,630 schemes involved in this. 91 00:05:06,630 --> 00:05:09,930 The most important one is to say, well assuming that the 92 00:05:09,930 --> 00:05:13,920 semiconductor manufacturer allows me to arrange my memory 93 00:05:13,920 --> 00:05:16,390 so that one of these pigeonholes is big enough to 94 00:05:16,390 --> 00:05:21,706 hold the address of another I haven't made. 95 00:05:21,706 --> 00:05:23,730 Now it actually has to be a little bit bigger because I 96 00:05:23,730 --> 00:05:28,215 have to also install or store some information as to a tag 97 00:05:28,215 --> 00:05:30,390 which describes the kind of thing that's there. 98 00:05:30,390 --> 00:05:32,350 And we'll see that in a second. 99 00:05:32,350 --> 00:05:34,560 And of course if the semiconductor manufacturer 100 00:05:34,560 --> 00:05:37,470 doesn't arrange it so I can do that, then of course I can, 101 00:05:37,470 --> 00:05:40,910 with some cleverness, arrange combinations of these to fit 102 00:05:40,910 --> 00:05:43,770 together in that way. 103 00:05:43,770 --> 00:05:48,510 So we're going to have to imagine imposing this 104 00:05:48,510 --> 00:05:51,740 complicated tree structure on our nice linear memory. 105 00:05:51,740 --> 00:05:57,540 If we look at the first still store, we see a classic scheme 106 00:05:57,540 --> 00:05:59,490 for doing that. 107 00:05:59,490 --> 00:06:03,910 It's a standard way of representing Lisp structures 108 00:06:03,910 --> 00:06:05,980 in a linear memory. 109 00:06:05,980 --> 00:06:12,030 What we do is we divide this memory into two parts. 110 00:06:12,030 --> 00:06:17,580 An array called the cars, and an array called the cdrs. 111 00:06:17,580 --> 00:06:20,470 Now whether those happen to be sequential addresses or 112 00:06:20,470 --> 00:06:22,560 whatever, it's not important. 113 00:06:22,560 --> 00:06:25,800 That's somebody's implementation details. 114 00:06:25,800 --> 00:06:28,960 But there are two arrays here. 115 00:06:28,960 --> 00:06:34,840 Linear arrays indexed by sequential indices like this. 116 00:06:34,840 --> 00:06:36,150 What is stored in each of these 117 00:06:36,150 --> 00:06:41,430 pigeonholes is a typed object. 118 00:06:41,430 --> 00:06:44,840 And what we have here are types which begin with letters 119 00:06:44,840 --> 00:06:47,790 like p, standing for a pair. 120 00:06:47,790 --> 00:06:50,040 Or n, standing for a number. 121 00:06:50,040 --> 00:06:57,290 Or e, standing for an empty list. The end of the list. And 122 00:06:57,290 --> 00:07:00,420 so if we wish to represent an object like this, the list 123 00:07:00,420 --> 00:07:04,310 beginning with 1, 2 and then having a 3 and a 4 as its 124 00:07:04,310 --> 00:07:06,430 second and third elements. 125 00:07:06,430 --> 00:07:10,220 A list containing a list as its first part and then two 126 00:07:10,220 --> 00:07:12,610 numbers as a second and third parts. 127 00:07:12,610 --> 00:07:15,250 Then of course we draw it sort of like this these days, in 128 00:07:15,250 --> 00:07:17,320 box-and-pointer notation. 129 00:07:17,320 --> 00:07:21,190 And you see, these are the three cells that have as their 130 00:07:21,190 --> 00:07:28,390 car pointer the object which is either 1, 2 or 3 or 4. 131 00:07:28,390 --> 00:07:30,860 And then of course the 1, 2, the car of this entire 132 00:07:30,860 --> 00:07:33,870 structure, is itself a substructure which contains a 133 00:07:33,870 --> 00:07:35,940 sublist like that. 134 00:07:35,940 --> 00:07:39,970 What I'm about to do is put down places which are-- 135 00:07:39,970 --> 00:07:41,880 I'm going to assign indices. 136 00:07:41,880 --> 00:07:45,530 Like this 1, over here, represents the 137 00:07:45,530 --> 00:07:46,850 index of this cell. 138 00:07:46,850 --> 00:07:49,850 139 00:07:49,850 --> 00:07:55,540 But that pointer that we see here is a reference to the 140 00:07:55,540 --> 00:07:57,640 pair of pigeonholes in the cars and the cdrs that are 141 00:07:57,640 --> 00:08:02,000 labeled by 1 in my linear memory down here. 142 00:08:02,000 --> 00:08:05,920 So if I wish to impose this structure on my linear memory, 143 00:08:05,920 --> 00:08:08,780 what I do is I say, oh yes, why don't we drop 144 00:08:08,780 --> 00:08:12,220 this into cell 1? 145 00:08:12,220 --> 00:08:12,660 I pick one. 146 00:08:12,660 --> 00:08:14,270 There's 1. 147 00:08:14,270 --> 00:08:16,640 And that says that its car, I'm going to 148 00:08:16,640 --> 00:08:17,950 assign it to be a pair. 149 00:08:17,950 --> 00:08:22,590 It's a pair, which is in index 5. 150 00:08:22,590 --> 00:08:26,360 And the cdr, which is this one over here, is a pair which I'm 151 00:08:26,360 --> 00:08:28,340 going to stick into place 2. 152 00:08:28,340 --> 00:08:30,890 p2. 153 00:08:30,890 --> 00:08:32,950 And take a look at p2. 154 00:08:32,950 --> 00:08:37,550 Oh yes, well p2 is a thing whose car is the number 3, so 155 00:08:37,550 --> 00:08:39,520 as you see, an n3. 156 00:08:39,520 --> 00:08:46,640 And whose cdr, over here, is a pair, which lives in place 4. 157 00:08:46,640 --> 00:08:48,650 So that's what this p4 is. 158 00:08:48,650 --> 00:08:56,200 p4 is a number whose value is 4 in its car and whose cdr is 159 00:08:56,200 --> 00:08:59,170 an empty list right there. 160 00:08:59,170 --> 00:09:00,690 And that ends it. 161 00:09:00,690 --> 00:09:05,750 So this is the traditional way of representing this kind of 162 00:09:05,750 --> 00:09:11,620 binary tree in a linear memory. 163 00:09:11,620 --> 00:09:15,770 Now the next question, of course, that we might want to 164 00:09:15,770 --> 00:09:18,440 worry about is just a little bit of implementation. 165 00:09:18,440 --> 00:09:22,690 That means that when I write procedures of the form 166 00:09:22,690 --> 00:09:24,600 assigned a, [UNINTELLIGIBLE] procedures-- 167 00:09:24,600 --> 00:09:29,000 lines of register machine code of the form assigned a, the 168 00:09:29,000 --> 00:09:30,140 car of [UNINTELLIGIBLE] 169 00:09:30,140 --> 00:09:38,740 b, what I really mean is addressing these elements. 170 00:09:38,740 --> 00:09:44,470 And so we're going to think of that as a abbreviation for it. 171 00:09:44,470 --> 00:09:46,720 Now of course in order to write that down I'm going to 172 00:09:46,720 --> 00:09:49,140 introduce some sort of a structure called a vector. 173 00:09:49,140 --> 00:09:52,120 174 00:09:52,120 --> 00:09:53,990 And we're going to have something which will reference 175 00:09:53,990 --> 00:09:58,710 a vector, just so we can write it down. 176 00:09:58,710 --> 00:10:02,240 Which takes the name of the vector, or the-- 177 00:10:02,240 --> 00:10:03,970 I don't think that name is the right word. 178 00:10:03,970 --> 00:10:12,010 Which takes the vector and the index, and I have to have a 179 00:10:12,010 --> 00:10:13,950 way of setting one of those with something called a vector 180 00:10:13,950 --> 00:10:16,280 set, I don't really care. 181 00:10:16,280 --> 00:10:19,520 But let's look, for example, at then that kind of 182 00:10:19,520 --> 00:10:26,470 implementation of car and cdr. 183 00:10:26,470 --> 00:10:31,470 So for example if I happen to have a register b, which 184 00:10:31,470 --> 00:10:37,580 contains the type index of a pair, and therefore it is the 185 00:10:37,580 --> 00:10:41,930 pointer to a pair, then I could take the car of that and 186 00:10:41,930 --> 00:10:42,760 if I-- write this down-- 187 00:10:42,760 --> 00:10:44,490 I might put that in register a. 188 00:10:44,490 --> 00:10:49,400 What that really is is a representation of the assign 189 00:10:49,400 --> 00:10:52,890 to a, the value of vector reffing-- 190 00:10:52,890 --> 00:10:54,700 or array indexing, if you will-- or 191 00:10:54,700 --> 00:10:58,490 something, the cars object-- 192 00:10:58,490 --> 00:10:59,990 whatever that is-- 193 00:10:59,990 --> 00:11:02,650 with the index, b. 194 00:11:02,650 --> 00:11:06,330 And similarly for cdr. And we can do the same thing for 195 00:11:06,330 --> 00:11:10,370 assignment to data structures, if we need to do that sort of 196 00:11:10,370 --> 00:11:11,840 thing at all. 197 00:11:11,840 --> 00:11:14,580 It's not too hard to build that. 198 00:11:14,580 --> 00:11:16,170 Well now the next question is how are we going to do 199 00:11:16,170 --> 00:11:18,010 allocation. 200 00:11:18,010 --> 00:11:21,550 And every so often I say I want a cons. 201 00:11:21,550 --> 00:11:23,790 Now conses don't grow on trees. 202 00:11:23,790 --> 00:11:25,340 Or maybe they should. 203 00:11:25,340 --> 00:11:29,980 But I have to have some way of getting the next one. 204 00:11:29,980 --> 00:11:33,920 I have to have some idea of if their memory is unused that I 205 00:11:33,920 --> 00:11:35,630 might want to allocate from. 206 00:11:35,630 --> 00:11:37,380 And there are many schemes for doing this. 207 00:11:37,380 --> 00:11:38,660 And the particular thing I'm showing you 208 00:11:38,660 --> 00:11:42,100 right now is not essential. 209 00:11:42,100 --> 00:11:44,960 However it's convenient and has been done many times. 210 00:11:44,960 --> 00:11:47,660 One scheme's was called the free list allocation scheme. 211 00:11:47,660 --> 00:11:50,570 What that means is that all of the free memory that there is 212 00:11:50,570 --> 00:11:54,700 in the world is linked together in a linked list, 213 00:11:54,700 --> 00:11:56,960 just like all the other stuff. 214 00:11:56,960 --> 00:12:01,230 And whenever you need a free cell to make a new cons, you 215 00:12:01,230 --> 00:12:04,440 grab the first, one make the free list be the cdr of it, 216 00:12:04,440 --> 00:12:06,030 and then allocate that. 217 00:12:06,030 --> 00:12:09,530 And so what that looks like is something like this. 218 00:12:09,530 --> 00:12:18,510 Here we have the free list starting in 6. 219 00:12:18,510 --> 00:12:24,860 And what that is is a pointer-off to say 8. 220 00:12:24,860 --> 00:12:27,020 So what it says is, this one is free and the 221 00:12:27,020 --> 00:12:28,870 next one is an 8. 222 00:12:28,870 --> 00:12:32,880 This one is free and the next one is in 3, the next one 223 00:12:32,880 --> 00:12:33,930 that's free. 224 00:12:33,930 --> 00:12:37,680 That one's free and the next one is in 0. 225 00:12:37,680 --> 00:12:40,940 That one's free and the next one's in 15. 226 00:12:40,940 --> 00:12:42,780 Something like that. 227 00:12:42,780 --> 00:12:46,400 We can imagine having such a structure. 228 00:12:46,400 --> 00:12:50,480 Given that we have something like that, then it's possible 229 00:12:50,480 --> 00:12:53,940 to just get one when you need it. 230 00:12:53,940 --> 00:12:57,960 And so a program for doing cons, this is what 231 00:12:57,960 --> 00:12:59,320 cons might turn into. 232 00:12:59,320 --> 00:13:05,410 To assign to a register A the result of cons-ing, a B onto 233 00:13:05,410 --> 00:13:08,250 C, the value in this containing B and the value 234 00:13:08,250 --> 00:13:11,240 containing C, what we have to do is get the current 235 00:13:11,240 --> 00:13:13,400 [? type ?] ahead of the freelist, make the free list 236 00:13:13,400 --> 00:13:19,840 be its cdr. Then we have to change the cars to be the 237 00:13:19,840 --> 00:13:25,680 thing we're making up to be in A to be the B, the thing in B. 238 00:13:25,680 --> 00:13:30,880 And we have to make change the cdrs of the thing that's in A 239 00:13:30,880 --> 00:13:36,020 to be C. And then what we have in A is the right new frob, 240 00:13:36,020 --> 00:13:36,650 whatever it is. 241 00:13:36,650 --> 00:13:40,470 The object that we want. 242 00:13:40,470 --> 00:13:43,490 Now there's a little bit of a cheat here that I haven't told 243 00:13:43,490 --> 00:13:47,155 you about, which is somewhere around here I haven't set that 244 00:13:47,155 --> 00:13:51,540 I've the type of the thing that I'm cons-ing up to be a 245 00:13:51,540 --> 00:13:53,510 pair, and I ought to. 246 00:13:53,510 --> 00:13:56,570 So there should be some sort of bits here are being set, 247 00:13:56,570 --> 00:13:59,810 and I just haven't written that down. 248 00:13:59,810 --> 00:14:01,480 We could have arranged it, of course, for the free lift to 249 00:14:01,480 --> 00:14:03,100 be made out of pairs. 250 00:14:03,100 --> 00:14:06,430 And so then there's no problem with that. 251 00:14:06,430 --> 00:14:10,160 But that sort of-- again, an inessential detail in a way 252 00:14:10,160 --> 00:14:13,500 some particular programmer or architect or whatever might 253 00:14:13,500 --> 00:14:17,540 manufacture his machine or Lisp system. 254 00:14:17,540 --> 00:14:23,930 So for example, just looking at this, to allocate given 255 00:14:23,930 --> 00:14:27,200 that I had already the structure that you saw before, 256 00:14:27,200 --> 00:14:31,900 supposing I wanted to allocate a new cell, which is going to 257 00:14:31,900 --> 00:14:38,680 be representation of list one, one, two, where already one 258 00:14:38,680 --> 00:14:43,430 two was the car of the list we were playing with before. 259 00:14:43,430 --> 00:14:44,780 Well that's not so hard. 260 00:14:44,780 --> 00:14:47,670 I stored that one and one, so p1 one is the 261 00:14:47,670 --> 00:14:49,530 representation of this. 262 00:14:49,530 --> 00:14:51,690 This is p5. 263 00:14:51,690 --> 00:14:54,070 That's going to be the cdr of this. 264 00:14:54,070 --> 00:14:55,610 Now we're going to pull something off the free list, 265 00:14:55,610 --> 00:14:57,780 but remember the free list started at six. 266 00:14:57,780 --> 00:15:01,540 The new free list after this allocation is eight, a free 267 00:15:01,540 --> 00:15:02,890 list beginning at eight. 268 00:15:02,890 --> 00:15:06,360 And of course in six now we have a number one, which is 269 00:15:06,360 --> 00:15:10,540 what we wanted, with its cdr being the pair starting in 270 00:15:10,540 --> 00:15:13,330 location five. 271 00:15:13,330 --> 00:15:16,810 And that's no big deal. 272 00:15:16,810 --> 00:15:21,480 So the only problem really remaining here is, well, I 273 00:15:21,480 --> 00:15:25,080 don't have an infinitely large memory. 274 00:15:25,080 --> 00:15:28,070 If I do this for a little while, say, for example, 275 00:15:28,070 --> 00:15:30,745 supposing it takes me a microsecond to do a cons, and 276 00:15:30,745 --> 00:15:34,570 I have a million cons memory then I'm only going to run out 277 00:15:34,570 --> 00:15:38,000 in a second, and that's pretty bad. 278 00:15:38,000 --> 00:15:41,470 So what we do to prevent that disaster, that ecological 279 00:15:41,470 --> 00:15:44,300 disaster, talk about right after questions. 280 00:15:44,300 --> 00:15:45,550 Are there any questions? 281 00:15:45,550 --> 00:15:51,500 282 00:15:51,500 --> 00:15:52,030 Yes. 283 00:15:52,030 --> 00:15:54,830 AUDIENCE: In the environment diagrams that we were drawing 284 00:15:54,830 --> 00:15:58,630 we would use the body of procedures, and you would 285 00:15:58,630 --> 00:16:02,620 eventually wind up with things that were no longer useful in 286 00:16:02,620 --> 00:16:04,930 that structure. 287 00:16:04,930 --> 00:16:06,890 How is that represented? 288 00:16:06,890 --> 00:16:09,180 PROFESSOR: There's two problems here. 289 00:16:09,180 --> 00:16:13,870 One you were asking is that material becomes useless. 290 00:16:13,870 --> 00:16:14,920 We'll talk about that in a second. 291 00:16:14,920 --> 00:16:18,100 That has to do with how to prevent ecological disasters. 292 00:16:18,100 --> 00:16:20,190 If I make a lot of garbage I have to somehow be able to 293 00:16:20,190 --> 00:16:21,820 clean up after myself. 294 00:16:21,820 --> 00:16:23,430 And we'll talk about that in a second. 295 00:16:23,430 --> 00:16:25,370 The other question you're asking is how you represent 296 00:16:25,370 --> 00:16:27,210 the environments, I think. 297 00:16:27,210 --> 00:16:27,600 AUDIENCE: Yes. 298 00:16:27,600 --> 00:16:28,190 PROFESSOR: OK. 299 00:16:28,190 --> 00:16:29,780 And the environment structures can be represented in 300 00:16:29,780 --> 00:16:30,860 arbitrary ways. 301 00:16:30,860 --> 00:16:31,780 There are lots of them. 302 00:16:31,780 --> 00:16:33,630 I mean, here I'm just telling you about list cells. 303 00:16:33,630 --> 00:16:36,400 Of course every real system has vectors of arbitrary 304 00:16:36,400 --> 00:16:39,500 length as well as the vectors of length, too, which 305 00:16:39,500 --> 00:16:41,080 represent list cells. 306 00:16:41,080 --> 00:16:45,460 And the environment structures that one uses in a 307 00:16:45,460 --> 00:16:49,890 professionally written Lisp system tend to be vectors 308 00:16:49,890 --> 00:16:52,350 which contain a number of elements approximately equal 309 00:16:52,350 --> 00:16:56,090 to the number of arguments-- a little bit more because you 310 00:16:56,090 --> 00:16:58,290 need certain glue. 311 00:16:58,290 --> 00:17:00,360 So remember, the environment [UNINTELLIGIBLE] 312 00:17:00,360 --> 00:17:00,740 frames. 313 00:17:00,740 --> 00:17:03,980 The frames are constructed by applying a procedure. 314 00:17:03,980 --> 00:17:08,849 In doing so, an allocation is made of a place which is the 315 00:17:08,849 --> 00:17:11,270 number of arguments long plus [? unglue ?] 316 00:17:11,270 --> 00:17:13,859 that gets linked into a chain. 317 00:17:13,859 --> 00:17:15,660 It's just like algol at that level. 318 00:17:15,660 --> 00:17:19,810 319 00:17:19,810 --> 00:17:21,060 There any other questions? 320 00:17:21,060 --> 00:17:23,700 321 00:17:23,700 --> 00:17:23,920 OK. 322 00:17:23,920 --> 00:17:26,106 Thank you, and let's take a short break. 323 00:17:26,106 --> 00:17:26,449 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 324 00:17:26,449 --> 00:17:27,699 JOHANN SEBASTIAN BACH] 325 00:17:27,699 --> 00:18:12,270 326 00:18:12,270 --> 00:18:15,840 PROFESSOR: Well, as I just said, computer memories 327 00:18:15,840 --> 00:18:19,420 supplied by the semiconductor manufacturers are finite. 328 00:18:19,420 --> 00:18:21,620 And that's quite a pity. 329 00:18:21,620 --> 00:18:24,030 It might not always be that way. 330 00:18:24,030 --> 00:18:27,990 Just for a quick calculation, you can see that it's possible 331 00:18:27,990 --> 00:18:28,860 that if [? memory ?] 332 00:18:28,860 --> 00:18:32,130 prices keep going at the rate they're going that if you 333 00:18:32,130 --> 00:18:34,950 still took a microsecond second to do a cons, then-- 334 00:18:34,950 --> 00:18:37,120 first of all, everybody should know that there's about pi 335 00:18:37,120 --> 00:18:39,450 times ten to the seventh seconds in a year. 336 00:18:39,450 --> 00:18:42,640 And so that would be ten to the seventh plus ten to the 337 00:18:42,640 --> 00:18:43,940 sixth is ten to the thirteenth. 338 00:18:43,940 --> 00:18:45,870 So there's maybe ten to the fourteenth conses in the life 339 00:18:45,870 --> 00:18:47,520 of a machine. 340 00:18:47,520 --> 00:18:49,900 If there was ten to the fourteenth words of memory on 341 00:18:49,900 --> 00:18:54,020 your machine, you'd never run out. 342 00:18:54,020 --> 00:18:56,310 And that's not completely unreasonable. 343 00:18:56,310 --> 00:18:58,460 Ten to the fourteenth is not a very large number. 344 00:18:58,460 --> 00:19:03,860 345 00:19:03,860 --> 00:19:05,180 I don't think it is. 346 00:19:05,180 --> 00:19:08,700 But then again I like to play with astronomy. 347 00:19:08,700 --> 00:19:11,380 It's at least ten to the eighteenth centimeters between 348 00:19:11,380 --> 00:19:12,930 us and the nearest star. 349 00:19:12,930 --> 00:19:19,620 But the thing I'm about to worry about is, at least in 350 00:19:19,620 --> 00:19:22,130 the current economic state of affairs, ten to the fourteenth 351 00:19:22,130 --> 00:19:24,200 pieces of memory is expensive. 352 00:19:24,200 --> 00:19:27,280 And so I suppose what we have to do is make 353 00:19:27,280 --> 00:19:28,120 do with much smaller. 354 00:19:28,120 --> 00:19:30,170 Memories 355 00:19:30,170 --> 00:19:35,800 Now in general we want to have an illusion of infinity. 356 00:19:35,800 --> 00:19:38,850 All we need to do is arrange it so that whenever you look, 357 00:19:38,850 --> 00:19:40,100 the thing is there. 358 00:19:40,100 --> 00:19:42,670 359 00:19:42,670 --> 00:19:45,105 That's really an important idea. 360 00:19:45,105 --> 00:19:49,540 361 00:19:49,540 --> 00:19:52,470 A person or a computer lives only a finite amount of time 362 00:19:52,470 --> 00:19:55,280 and can only take a finite number of looks at something. 363 00:19:55,280 --> 00:19:58,190 And so you really only need a finite amount of stuff. 364 00:19:58,190 --> 00:20:01,730 But you have to arrange it so no matter how much there is, 365 00:20:01,730 --> 00:20:04,000 how much you really claim there is, there's always 366 00:20:04,000 --> 00:20:06,900 enough stuff so that when you take a look, it's there. 367 00:20:06,900 --> 00:20:08,750 And so you only need a finite amount. 368 00:20:08,750 --> 00:20:11,630 But let's see. 369 00:20:11,630 --> 00:20:14,980 One problem is, as was brought up, that there are possible 370 00:20:14,980 --> 00:20:18,660 ways that there is lots of stuff that we make that we 371 00:20:18,660 --> 00:20:19,410 don't need. 372 00:20:19,410 --> 00:20:20,895 And we could recycle the material out 373 00:20:20,895 --> 00:20:22,760 of which its made. 374 00:20:22,760 --> 00:20:27,820 An example is the fact that we're building environment 375 00:20:27,820 --> 00:20:30,470 structures, and we do so every time we call a procedure. 376 00:20:30,470 --> 00:20:32,810 We have built in it a environment frame. 377 00:20:32,810 --> 00:20:34,840 That environment frame doesn't necessarily 378 00:20:34,840 --> 00:20:36,730 have a very long lifetime. 379 00:20:36,730 --> 00:20:40,330 Its lifetime, meaning its usefulness, may exist only 380 00:20:40,330 --> 00:20:42,850 over the invocation of the procedure. 381 00:20:42,850 --> 00:20:45,860 Or if the procedure exports another procedure by returning 382 00:20:45,860 --> 00:20:48,260 it as a value and that procedure is defined inside of 383 00:20:48,260 --> 00:20:52,210 it, well then the lifetime of the frame of the outer 384 00:20:52,210 --> 00:20:57,070 procedure still is only the lifetime of the procedure 385 00:20:57,070 --> 00:20:58,530 which was exported. 386 00:20:58,530 --> 00:21:01,960 And so ultimately, a lot of that is garbage. 387 00:21:01,960 --> 00:21:05,370 There are other ways of producing garbage as well. 388 00:21:05,370 --> 00:21:07,240 Users produce garbage. 389 00:21:07,240 --> 00:21:10,930 An example of user garbage is something like this. 390 00:21:10,930 --> 00:21:15,220 If we write a program to, for example, append two lists 391 00:21:15,220 --> 00:21:19,890 together, well one way to do it is to reverse the first 392 00:21:19,890 --> 00:21:23,440 list onto the empty list and reverse that onto the second 393 00:21:23,440 --> 00:21:28,160 list. Now that's not terribly bad way of doing it. 394 00:21:28,160 --> 00:21:31,070 And however, the intermediate result, which is the reversal 395 00:21:31,070 --> 00:21:37,300 of the first list as done by this program, is never going 396 00:21:37,300 --> 00:21:39,920 to be accessed ever again after it's copied back on to 397 00:21:39,920 --> 00:21:41,010 the second. 398 00:21:41,010 --> 00:21:43,580 It's an intermediate result. 399 00:21:43,580 --> 00:21:47,230 It's going to be hard to ever see how anybody would ever be 400 00:21:47,230 --> 00:21:48,600 able to access it. 401 00:21:48,600 --> 00:21:51,050 In fact, it will go away. 402 00:21:51,050 --> 00:21:53,190 Now if we make a lot of garbage like that, and we 403 00:21:53,190 --> 00:21:56,210 should be allowed to, then there's got to be some way to 404 00:21:56,210 --> 00:21:58,800 reclaim that garbage. 405 00:21:58,800 --> 00:22:03,050 Well, what I'd like to tell you about now is a very clever 406 00:22:03,050 --> 00:22:09,820 technique whereby a Lisp system can prove a small 407 00:22:09,820 --> 00:22:12,750 theorem every so often on the [? forum, ?] the following 408 00:22:12,750 --> 00:22:17,410 piece of junk will never be accessed again. 409 00:22:17,410 --> 00:22:21,400 It can have no affect on the future of the computation. 410 00:22:21,400 --> 00:22:24,920 It's actually based on a very simple idea. 411 00:22:24,920 --> 00:22:28,570 We've designed our computers to look sort of like this. 412 00:22:28,570 --> 00:22:35,280 There's some data path, which contains the registers. 413 00:22:35,280 --> 00:22:42,610 There are things like x, and env, and val, and so on. 414 00:22:42,610 --> 00:22:47,490 And there's one here called stack, some sort which points 415 00:22:47,490 --> 00:22:50,240 off to a structure somewhere, which is the stack. 416 00:22:50,240 --> 00:22:51,740 And we'll worry about that in a second. 417 00:22:51,740 --> 00:22:55,390 There's some finite controller, finite state 418 00:22:55,390 --> 00:22:56,730 machine controller. 419 00:22:56,730 --> 00:22:59,850 And there's some control signals that go this way and 420 00:22:59,850 --> 00:23:02,270 predicate results that come this way, not 421 00:23:02,270 --> 00:23:04,260 the interesting part. 422 00:23:04,260 --> 00:23:07,140 There's some sort of structured memory, which I 423 00:23:07,140 --> 00:23:10,460 just told you how to make, which may contain a stack. 424 00:23:10,460 --> 00:23:12,690 I didn't tell you how to make things of arbitrary shape, 425 00:23:12,690 --> 00:23:13,450 only pairs. 426 00:23:13,450 --> 00:23:16,280 But in fact with what I've told you can simulate a stack 427 00:23:16,280 --> 00:23:19,140 by a big list. I don't plan to do that, it's not a 428 00:23:19,140 --> 00:23:20,360 nice way to do it. 429 00:23:20,360 --> 00:23:22,990 But we could have something like that. 430 00:23:22,990 --> 00:23:25,990 We have all sorts of little data structures in here that 431 00:23:25,990 --> 00:23:27,470 are hooked together in funny ways. 432 00:23:27,470 --> 00:23:30,115 433 00:23:30,115 --> 00:23:32,560 They connect to other things. 434 00:23:32,560 --> 00:23:33,250 And so on. 435 00:23:33,250 --> 00:23:37,190 And ultimately things up there are pointers to these. 436 00:23:37,190 --> 00:23:40,730 The things that are in the registers are pointers off to 437 00:23:40,730 --> 00:23:44,910 the data structures that live in this Lisp structure memory. 438 00:23:44,910 --> 00:23:52,660 Now the truth of the matter is that the entire consciousness 439 00:23:52,660 --> 00:23:55,550 of this machine is in these registers. 440 00:23:55,550 --> 00:23:59,380 There is no possible way that the machine, if done 441 00:23:59,380 --> 00:24:02,410 correctly, if built correctly, can access anything in this 442 00:24:02,410 --> 00:24:05,580 Lisp structure memory unless the thing in that Lisp 443 00:24:05,580 --> 00:24:10,310 structure memory is connected by a sequence of data 444 00:24:10,310 --> 00:24:15,070 structures to the registers. 445 00:24:15,070 --> 00:24:17,550 If it's accessible by legitimate data structure 446 00:24:17,550 --> 00:24:20,100 selectors from the pointers that are 447 00:24:20,100 --> 00:24:22,280 stored in these registers. 448 00:24:22,280 --> 00:24:24,940 Things like array references, perhaps. 449 00:24:24,940 --> 00:24:28,790 Or cons cell references, cars and cdrs. 450 00:24:28,790 --> 00:24:30,970 But I can't just talk about a random place in this memory, 451 00:24:30,970 --> 00:24:32,740 because I can't get to it. 452 00:24:32,740 --> 00:24:34,665 These are being arbitrary names I'm not allowed to 453 00:24:34,665 --> 00:24:38,985 count, at least as I'm evaluating expressions. 454 00:24:38,985 --> 00:24:41,620 455 00:24:41,620 --> 00:24:44,600 If that's the case then there's a very simple theorem 456 00:24:44,600 --> 00:24:47,160 to be proved. 457 00:24:47,160 --> 00:24:49,730 Which is, if I start with all lead pointers that are in all 458 00:24:49,730 --> 00:24:53,570 these registers and recursively chase out, marking 459 00:24:53,570 --> 00:24:57,980 all the places I can get to by selectors, then eventually I 460 00:24:57,980 --> 00:25:00,750 mark everything they can be gotten to. 461 00:25:00,750 --> 00:25:02,140 Anything which is not so marked is 462 00:25:02,140 --> 00:25:05,560 garbage and can be recycled. 463 00:25:05,560 --> 00:25:07,200 Very simple. 464 00:25:07,200 --> 00:25:11,180 Cannot affect the future of the computation. 465 00:25:11,180 --> 00:25:16,616 So let me show you that in a particular example. 466 00:25:16,616 --> 00:25:19,800 Now that means I'm going to have to append to my 467 00:25:19,800 --> 00:25:23,640 description of the list structure a mark. 468 00:25:23,640 --> 00:25:29,080 And so here, for example, is a Lisp structured memory. 469 00:25:29,080 --> 00:25:31,360 And in this Lisp structured memory is a Lisp structure 470 00:25:31,360 --> 00:25:33,030 beginning in a place I'm going to call-- 471 00:25:33,030 --> 00:25:35,640 472 00:25:35,640 --> 00:25:38,590 this is the root. 473 00:25:38,590 --> 00:25:40,120 Now it doesn't really have to have a root. 474 00:25:40,120 --> 00:25:42,670 It could be a bunch of them, like all the registers. 475 00:25:42,670 --> 00:25:45,270 But I could cleverly arrange it so all the registers, all 476 00:25:45,270 --> 00:25:47,080 the things that are in old registers are also at the 477 00:25:47,080 --> 00:25:50,770 right moment put into this root structure, and then we've 478 00:25:50,770 --> 00:25:51,850 got one pointer to it. 479 00:25:51,850 --> 00:25:54,570 I don't really care. 480 00:25:54,570 --> 00:25:57,290 So the idea is we're going to cons up stuff until our free 481 00:25:57,290 --> 00:25:58,720 list is empty. 482 00:25:58,720 --> 00:26:00,950 We've run out of things. 483 00:26:00,950 --> 00:26:04,560 Now we're going to do this process of proving the theorem 484 00:26:04,560 --> 00:26:07,850 that a certain percentage of the memory has got crap in it. 485 00:26:07,850 --> 00:26:10,320 And then we're going to recycle that to grow new 486 00:26:10,320 --> 00:26:14,570 trees, a standard use of such garbage. 487 00:26:14,570 --> 00:26:17,090 488 00:26:17,090 --> 00:26:18,840 So in any case, what do we have here? 489 00:26:18,840 --> 00:26:21,700 Well we have some data structure which starts out 490 00:26:21,700 --> 00:26:27,502 over here one. 491 00:26:27,502 --> 00:26:33,980 And in fact it has a car in five, and its cdr is in two. 492 00:26:33,980 --> 00:26:36,700 And all the marks start out at zero. 493 00:26:36,700 --> 00:26:39,920 Well let's start marking, just to play this game. 494 00:26:39,920 --> 00:26:42,540 OK. 495 00:26:42,540 --> 00:26:47,090 So for example, since I can access one from the root I 496 00:26:47,090 --> 00:26:48,390 will mark that. 497 00:26:48,390 --> 00:26:50,960 Let me mark it. 498 00:26:50,960 --> 00:26:52,430 Bang. 499 00:26:52,430 --> 00:26:54,560 That's marked. 500 00:26:54,560 --> 00:27:00,020 Now since I have a five here I can go to five and see, well 501 00:27:00,020 --> 00:27:01,450 I'll mark that. 502 00:27:01,450 --> 00:27:01,760 Bang. 503 00:27:01,760 --> 00:27:02,900 That's useful stuff. 504 00:27:02,900 --> 00:27:05,410 But five references as a number in its car, I'm not 505 00:27:05,410 --> 00:27:08,700 interested in marking numbers but its cdr is seven. 506 00:27:08,700 --> 00:27:10,450 So I can mark that. 507 00:27:10,450 --> 00:27:12,260 Bang. 508 00:27:12,260 --> 00:27:16,000 Seven is the empty list, the only thing that references, 509 00:27:16,000 --> 00:27:17,120 and it's got a number in its car. 510 00:27:17,120 --> 00:27:19,490 Not interesting. 511 00:27:19,490 --> 00:27:20,500 Well now let's go back here. 512 00:27:20,500 --> 00:27:21,650 I forgot about something. 513 00:27:21,650 --> 00:27:22,840 Two. 514 00:27:22,840 --> 00:27:25,960 See in other words, if I'm looking at cell one, cell one 515 00:27:25,960 --> 00:27:30,370 contains a two right over here. 516 00:27:30,370 --> 00:27:31,730 A reference to two. 517 00:27:31,730 --> 00:27:35,700 That means I should go mark two. 518 00:27:35,700 --> 00:27:37,140 Bang. 519 00:27:37,140 --> 00:27:38,960 Two contains a reference to four. 520 00:27:38,960 --> 00:27:41,593 It's got a number in its car, I'm not interested in that, so 521 00:27:41,593 --> 00:27:43,780 I'm going to go mark that. 522 00:27:43,780 --> 00:27:47,840 Four refers to seven through its car, and is empty in its 523 00:27:47,840 --> 00:27:49,990 cdr, but I've already marked that one so I don't have to 524 00:27:49,990 --> 00:27:51,400 mark it again. 525 00:27:51,400 --> 00:27:55,000 This is all the accessible structure from that place. 526 00:27:55,000 --> 00:27:58,710 Simple recursive mark algorithm. 527 00:27:58,710 --> 00:28:01,160 Now there are some unhappinesses about that 528 00:28:01,160 --> 00:28:04,920 algorithm, and we can worry about that a second. 529 00:28:04,920 --> 00:28:07,280 But basically you'll see that all the things that have not 530 00:28:07,280 --> 00:28:14,220 been marked are places that are free, and I could recycle. 531 00:28:14,220 --> 00:28:16,210 So the next stage after that is going to be to scan through 532 00:28:16,210 --> 00:28:21,180 all of my memory, looking for things that are not marked. 533 00:28:21,180 --> 00:28:23,370 Every time I come across a marked thing I unmark it, and 534 00:28:23,370 --> 00:28:26,390 every time I come across an unmarked thing I'm going to 535 00:28:26,390 --> 00:28:28,770 link it together in my free list. 536 00:28:28,770 --> 00:28:32,120 Classic, very simple algorithm. 537 00:28:32,120 --> 00:28:33,840 So let's see. 538 00:28:33,840 --> 00:28:34,770 Is that very simple? 539 00:28:34,770 --> 00:28:35,570 Yes it is. 540 00:28:35,570 --> 00:28:38,340 I'm not going to go through the code in any detail, but I 541 00:28:38,340 --> 00:28:40,090 just want to show you about how long it is. 542 00:28:40,090 --> 00:28:42,490 Let's look at the mark phase. 543 00:28:42,490 --> 00:28:45,060 Here's the first part of the mark phase. 544 00:28:45,060 --> 00:28:48,280 We pick up the root. 545 00:28:48,280 --> 00:28:52,380 We're going to use that as a recursive procedure call. 546 00:28:52,380 --> 00:28:55,800 We're going to sweep from there, after when we're done 547 00:28:55,800 --> 00:28:57,380 with marking. 548 00:28:57,380 --> 00:28:59,840 And then we're going to do a little couple of instructions 549 00:28:59,840 --> 00:29:01,920 that do this checking out on the marks and changing the 550 00:29:01,920 --> 00:29:04,000 marks and things like that, according to the algorithm 551 00:29:04,000 --> 00:29:05,500 I've just shown you. 552 00:29:05,500 --> 00:29:06,470 It comes out here. 553 00:29:06,470 --> 00:29:08,800 You have to mark the cars of things and you also have to be 554 00:29:08,800 --> 00:29:10,660 able to mark the cdrs of things. 555 00:29:10,660 --> 00:29:14,370 That's the entire mark phase. 556 00:29:14,370 --> 00:29:16,590 I'll just tell you a little story about this. 557 00:29:16,590 --> 00:29:22,950 The old DEC PDP-6 computer, this was the way that the 558 00:29:22,950 --> 00:29:26,740 mark-sweep garbage collection, as it was, was written. 559 00:29:26,740 --> 00:29:31,940 The program was so small that with the data that it needed, 560 00:29:31,940 --> 00:29:34,310 with the registers that it needed to manipulate the 561 00:29:34,310 --> 00:29:38,070 memory, it fit into the fast registers of the machine, 562 00:29:38,070 --> 00:29:39,280 which were 16. 563 00:29:39,280 --> 00:29:39,800 The whole program. 564 00:29:39,800 --> 00:29:40,700 And you could execute 565 00:29:40,700 --> 00:29:43,170 instructions in the fast registers. 566 00:29:43,170 --> 00:29:46,560 So it's an extremely small program, and it could run very 567 00:29:46,560 --> 00:29:48,870 fast. 568 00:29:48,870 --> 00:29:53,130 Now unfortunately, of course, this program, because the fact 569 00:29:53,130 --> 00:29:57,630 that it's recursive in the way that you do something first 570 00:29:57,630 --> 00:29:59,690 and then you do something after that, you have to work 571 00:29:59,690 --> 00:30:03,410 on the cars and then the cdrs, it requires auxiliary memory. 572 00:30:03,410 --> 00:30:05,680 So Lisp systems-- 573 00:30:05,680 --> 00:30:08,260 those requires a stack for marking. 574 00:30:08,260 --> 00:30:12,440 Lisp systems that are built this way have a limit to the 575 00:30:12,440 --> 00:30:16,060 depth of recursion you can have in data structures in 576 00:30:16,060 --> 00:30:19,930 either the car or the cdr, and that doesn't work very nicely. 577 00:30:19,930 --> 00:30:23,180 On the other hand, you never notice it if it's big enough. 578 00:30:23,180 --> 00:30:27,650 And that's certainly been the case for most Maclisp, for 579 00:30:27,650 --> 00:30:30,760 example, which ran Macsyma where you could deal with 580 00:30:30,760 --> 00:30:33,560 expressions of thousands of elements long. 581 00:30:33,560 --> 00:30:35,490 These are algebraic expressions with thousand of 582 00:30:35,490 --> 00:30:39,490 terms. And there's no problem with that. 583 00:30:39,490 --> 00:30:42,190 Such, the garbage collector does work. 584 00:30:42,190 --> 00:30:44,750 On the other hand, there's a very clever modification to 585 00:30:44,750 --> 00:30:49,020 this algorithm, which I will not describe, by Peter Deutsch 586 00:30:49,020 --> 00:30:50,720 and Schorr and Waite-- 587 00:30:50,720 --> 00:30:55,380 Herb Schorr from IBM and Waite, who I don't know. 588 00:30:55,380 --> 00:30:58,470 That algorithm allows you to build-- you do can do this 589 00:30:58,470 --> 00:31:01,990 without auxiliary memory, by remembering as you walk the 590 00:31:01,990 --> 00:31:04,760 data structures where you came from by reversing the pointers 591 00:31:04,760 --> 00:31:06,650 as you go down and crawling up the reverse 592 00:31:06,650 --> 00:31:07,520 pointers as you go up. 593 00:31:07,520 --> 00:31:09,130 It's a rather tricky algorithm. 594 00:31:09,130 --> 00:31:11,230 The first time you write it-- or in fact, the first three 595 00:31:11,230 --> 00:31:14,350 times you write it it has a terrible bug in it. 596 00:31:14,350 --> 00:31:18,110 And it's also rather slow, because it's complicated. 597 00:31:18,110 --> 00:31:21,150 It takes about six times as many memory references to do 598 00:31:21,150 --> 00:31:24,580 the sorts of things that we're talking about. 599 00:31:24,580 --> 00:31:28,140 Well now once I've done this marking phase, and I get into 600 00:31:28,140 --> 00:31:30,920 a position where things look like this, let's look-- 601 00:31:30,920 --> 00:31:31,510 yes. 602 00:31:31,510 --> 00:31:35,590 Here we have the mark done, just as I did it. 603 00:31:35,590 --> 00:31:37,330 Now we have to perform the sweep phase. 604 00:31:37,330 --> 00:31:39,820 And I described to you what this sweep is like. 605 00:31:39,820 --> 00:31:42,130 I'm going to walk down from one end of memory or the 606 00:31:42,130 --> 00:31:45,690 other, I don't care where, scanning every cell that's in 607 00:31:45,690 --> 00:31:46,836 the memory. 608 00:31:46,836 --> 00:31:51,000 And as I scan these cells, I'm going to link them together, 609 00:31:51,000 --> 00:31:53,890 if they are free, into the free list. And if they're not 610 00:31:53,890 --> 00:31:57,500 free, I'm going to unmark them so the marks become zero. 611 00:31:57,500 --> 00:32:00,050 And in fact what I get-- well the program is not very 612 00:32:00,050 --> 00:32:00,460 complicated. 613 00:32:00,460 --> 00:32:02,780 It looks sort of like this-- it's a little longer. 614 00:32:02,780 --> 00:32:04,820 Here's the first piece of it. 615 00:32:04,820 --> 00:32:06,710 This one's coming down from the top of memory. 616 00:32:06,710 --> 00:32:09,580 I don't want you to try to understand this at this point. 617 00:32:09,580 --> 00:32:11,030 It's rather simple. 618 00:32:11,030 --> 00:32:14,260 It's a very simple algorithm, but there's pieces of it that 619 00:32:14,260 --> 00:32:15,970 just sort of look like this. 620 00:32:15,970 --> 00:32:18,600 They're all sort of obvious. 621 00:32:18,600 --> 00:32:21,060 And after we've done the sweep, we get an answer that 622 00:32:21,060 --> 00:32:22,310 looks like that. 623 00:32:22,310 --> 00:32:25,330 624 00:32:25,330 --> 00:32:27,150 Now there are some disadvantages with mark-sweep 625 00:32:27,150 --> 00:32:29,590 algorithms of this sort. 626 00:32:29,590 --> 00:32:31,940 Serious ones. 627 00:32:31,940 --> 00:32:34,250 One important disadvantage is that your memories get larger 628 00:32:34,250 --> 00:32:36,498 and larger. 629 00:32:36,498 --> 00:32:39,100 As you say, address spaces get larger and larger, you're 630 00:32:39,100 --> 00:32:43,080 willing to represent more and more stuff, then it gets very 631 00:32:43,080 --> 00:32:46,360 costly to scan all of memory. 632 00:32:46,360 --> 00:32:50,490 What you'd really like to do is only scan useful stuff. 633 00:32:50,490 --> 00:32:56,120 It would even be better if you realized that some stuff was 634 00:32:56,120 --> 00:32:59,320 known to be good and useful, and you don't have to look at 635 00:32:59,320 --> 00:33:00,370 it more than once or twice. 636 00:33:00,370 --> 00:33:01,550 Or very rarely. 637 00:33:01,550 --> 00:33:05,120 Whereas other stuff that you're not so sure about, you 638 00:33:05,120 --> 00:33:10,110 can look at more detail every time you want to do this, want 639 00:33:10,110 --> 00:33:11,910 to garbage collect. 640 00:33:11,910 --> 00:33:15,660 Well there are algorithms that are organized in this way. 641 00:33:15,660 --> 00:33:18,850 Let me tell you about a famous old algorithm which allows you 642 00:33:18,850 --> 00:33:20,370 only look at the part of memory which 643 00:33:20,370 --> 00:33:22,800 is known to be useful. 644 00:33:22,800 --> 00:33:24,690 And which happens to be the fastest known garbage 645 00:33:24,690 --> 00:33:26,310 collector algorithm. 646 00:33:26,310 --> 00:33:28,170 This is the Minsky-Feinchel-Yochelson 647 00:33:28,170 --> 00:33:30,150 garbage collector algorithm. 648 00:33:30,150 --> 00:33:36,660 It was invented by Minsky in 1961 or '60 or something, for 649 00:33:36,660 --> 00:33:45,870 the RLE PDP-1 Lisp, which had 4,096 words of list memory, 650 00:33:45,870 --> 00:33:48,480 and a drum. 651 00:33:48,480 --> 00:33:50,890 And the whole idea was to garbage collect 652 00:33:50,890 --> 00:33:53,380 this terrible memory. 653 00:33:53,380 --> 00:33:56,510 What Minsky realized was the easiest way to do this is to 654 00:33:56,510 --> 00:33:59,950 scan the memory in the same sense, walking the good 655 00:33:59,950 --> 00:34:06,350 structure, copying it out into the drum, compacted. 656 00:34:06,350 --> 00:34:09,429 And then when we were done copying it all out, then you 657 00:34:09,429 --> 00:34:12,300 swap that back into your memory. 658 00:34:12,300 --> 00:34:14,260 Now whether or you not use a drum, or another piece of 659 00:34:14,260 --> 00:34:17,030 memory, or something like that isn't important. 660 00:34:17,030 --> 00:34:18,260 In fact, I don't think people use 661 00:34:18,260 --> 00:34:20,350 drums anymore for anything. 662 00:34:20,350 --> 00:34:25,920 But this algorithm basically depends upon having about 663 00:34:25,920 --> 00:34:30,270 twice as much address space as you're actually using. 664 00:34:30,270 --> 00:34:35,370 And so what you have is some, initially, some mixture of 665 00:34:35,370 --> 00:34:37,110 useful data and garbage. 666 00:34:37,110 --> 00:34:38,560 So this is called fromspace. 667 00:34:38,560 --> 00:34:45,179 668 00:34:45,179 --> 00:34:47,800 And this is a mixture of crud. 669 00:34:47,800 --> 00:34:52,000 Some of it's important and some of it isn't. 670 00:34:52,000 --> 00:34:55,770 Now there's another place which is hopefully big enough, 671 00:34:55,770 --> 00:34:58,240 if we recall, tospace, which is where we're copying to. 672 00:34:58,240 --> 00:35:01,590 673 00:35:01,590 --> 00:35:03,260 And what happens is-- and I'm not going to go 674 00:35:03,260 --> 00:35:04,970 through this detail. 675 00:35:04,970 --> 00:35:07,590 It's in our book quite explicitly. 676 00:35:07,590 --> 00:35:11,030 There's a root point where you start from. 677 00:35:11,030 --> 00:35:14,600 And the idea is that you start with the root. 678 00:35:14,600 --> 00:35:18,610 You copy the first thing you see, the first thing that the 679 00:35:18,610 --> 00:35:22,810 root points at, to the beginning of tospace. 680 00:35:22,810 --> 00:35:24,790 The first thing is a pair or something 681 00:35:24,790 --> 00:35:27,560 like, a data structure. 682 00:35:27,560 --> 00:35:32,330 You then also leave behind a broken heart saying, I moved 683 00:35:32,330 --> 00:35:36,330 this object from here to here, giving the place 684 00:35:36,330 --> 00:35:37,800 where it moved to. 685 00:35:37,800 --> 00:35:40,270 This is called a broken heart because a friend of mine who 686 00:35:40,270 --> 00:35:44,760 implemented one of these in 1966 was a very romantic 687 00:35:44,760 --> 00:35:46,760 character and called it a broken heart. 688 00:35:46,760 --> 00:35:49,580 689 00:35:49,580 --> 00:35:53,570 But in any case, the next thing you do is now you have a 690 00:35:53,570 --> 00:35:57,840 new free pointer which is here, and you start scanning. 691 00:35:57,840 --> 00:36:00,235 You scan this data structure you just copied. 692 00:36:00,235 --> 00:36:02,710 And every time you encounter a pointer in it, you treat it as 693 00:36:02,710 --> 00:36:04,000 if it was the root pointer here. 694 00:36:04,000 --> 00:36:05,170 Oh, I'm sorry. 695 00:36:05,170 --> 00:36:06,330 The other thing you do is you now move the 696 00:36:06,330 --> 00:36:09,220 root pointer to there. 697 00:36:09,220 --> 00:36:11,310 So now you scan this, and everything you see you treat 698 00:36:11,310 --> 00:36:14,110 as it were the root pointer. 699 00:36:14,110 --> 00:36:16,360 So if you see something, well it points 700 00:36:16,360 --> 00:36:18,510 up into there somewhere. 701 00:36:18,510 --> 00:36:21,780 Is it pointing at a thing which you've not copied yet? 702 00:36:21,780 --> 00:36:23,880 Is there a broken heart there? 703 00:36:23,880 --> 00:36:25,370 If there's a broken heart there and it's something you 704 00:36:25,370 --> 00:36:27,640 have copied, you've just replaced this pointer with the 705 00:36:27,640 --> 00:36:30,620 thing a broken heart points at. 706 00:36:30,620 --> 00:36:33,030 If this thing has not been copied, you copy it to the 707 00:36:33,030 --> 00:36:34,430 next place over here. 708 00:36:34,430 --> 00:36:39,860 Move your free pointer over here, and then leave a broken 709 00:36:39,860 --> 00:36:43,670 heart behind and scan. 710 00:36:43,670 --> 00:36:46,840 And eventually when the scant pointer hits the free pointer, 711 00:36:46,840 --> 00:36:50,140 everything in memory has been copied. 712 00:36:50,140 --> 00:36:52,170 And then there's a whole bunch of empty space up here, which 713 00:36:52,170 --> 00:36:53,820 you could either make into a free list, if that's what you 714 00:36:53,820 --> 00:36:54,470 want to do. 715 00:36:54,470 --> 00:36:56,270 But generally you don't in this kind of system. 716 00:36:56,270 --> 00:36:57,470 In this system you sequentially 717 00:36:57,470 --> 00:37:00,910 allocate your memory. 718 00:37:00,910 --> 00:37:03,820 That is a very, very nice algorithm, and sort of the one 719 00:37:03,820 --> 00:37:06,790 we use in the scheme that you've been using. 720 00:37:06,790 --> 00:37:09,490 And it's expected-- 721 00:37:09,490 --> 00:37:12,400 I believe no one has found a faster algorithm than that. 722 00:37:12,400 --> 00:37:14,150 There are very simple modifications to this 723 00:37:14,150 --> 00:37:19,060 algorithm invented by Henry Baker which allow one to run 724 00:37:19,060 --> 00:37:21,210 this algorithm in real time, meaning you don't have to stop 725 00:37:21,210 --> 00:37:22,010 to garbage collect. 726 00:37:22,010 --> 00:37:25,410 But you could interleave the consing that the machine does 727 00:37:25,410 --> 00:37:27,870 when its running with steps of the garbage collection 728 00:37:27,870 --> 00:37:31,370 process, so that the garbage collector's distributed, and 729 00:37:31,370 --> 00:37:32,980 the machine doesn't have to stop, and garbage 730 00:37:32,980 --> 00:37:34,640 collecting can start. 731 00:37:34,640 --> 00:37:37,520 Of course in the case of machines with virtual memory 732 00:37:37,520 --> 00:37:41,760 where a lot of it is in inaccessible places, this 733 00:37:41,760 --> 00:37:44,460 becomes a very expensive process. 734 00:37:44,460 --> 00:37:47,690 And there have been numerous attempts to 735 00:37:47,690 --> 00:37:49,190 make this much better. 736 00:37:49,190 --> 00:37:52,080 There is a nice paper, for those of you who are 737 00:37:52,080 --> 00:37:56,210 interested, by Moon and other people which describes a 738 00:37:56,210 --> 00:37:57,940 modification to the incremental 739 00:37:57,940 --> 00:38:00,290 Minsky-Feinchel-Yochelson algorithm, and modification 740 00:38:00,290 --> 00:38:05,980 the Baker algorithm which is more efficient for virtual 741 00:38:05,980 --> 00:38:08,340 memory systems. 742 00:38:08,340 --> 00:38:12,840 Well I think now the mystery to this is sort of gone. 743 00:38:12,840 --> 00:38:14,090 And I'd like to see if there are any questions. 744 00:38:14,090 --> 00:38:19,780 745 00:38:19,780 --> 00:38:20,810 Yes. 746 00:38:20,810 --> 00:38:24,100 AUDIENCE: I saw one of you run the garbage collector on the 747 00:38:24,100 --> 00:38:27,640 systems upstairs, and it seemed to me to run extremely 748 00:38:27,640 --> 00:38:30,190 fast. Did the whole thing take-- 749 00:38:30,190 --> 00:38:31,880 does it sweep through all of memory? 750 00:38:31,880 --> 00:38:32,510 PROFESSOR: No. 751 00:38:32,510 --> 00:38:34,480 It swept through exactly what was needed to 752 00:38:34,480 --> 00:38:37,320 copy the useful structure. 753 00:38:37,320 --> 00:38:40,030 It's a copying collector. 754 00:38:40,030 --> 00:38:45,090 And it is very fast. On the whole, I suppose to copy-- 755 00:38:45,090 --> 00:38:47,000 in a Bobcat-- 756 00:38:47,000 --> 00:38:52,450 to copy, I think, a three megabyte thing or something is 757 00:38:52,450 --> 00:38:56,800 less than a second, real time. 758 00:38:56,800 --> 00:38:59,200 Really, these are very small programs. One thing you should 759 00:38:59,200 --> 00:39:05,400 realise is that garbage collectors have to be small. 760 00:39:05,400 --> 00:39:08,770 Not because they have to be fast, but because no one can 761 00:39:08,770 --> 00:39:11,340 debug a complicated garbage collector. 762 00:39:11,340 --> 00:39:15,000 A garbage collector, if it doesn't work, will trash your 763 00:39:15,000 --> 00:39:16,940 memory in such a way that you cannot figure out what the 764 00:39:16,940 --> 00:39:18,350 hell happened. 765 00:39:18,350 --> 00:39:20,660 You need an audit trail. 766 00:39:20,660 --> 00:39:22,460 Because it rearranges everything, and how do you 767 00:39:22,460 --> 00:39:23,740 know what happened there? 768 00:39:23,740 --> 00:39:27,480 So this is the only kind of program that it really, 769 00:39:27,480 --> 00:39:30,100 seriously matters if you stare at it long enough so you 770 00:39:30,100 --> 00:39:31,970 believe that it works. 771 00:39:31,970 --> 00:39:35,100 And sort of prove it to yourself. 772 00:39:35,100 --> 00:39:36,940 So there's no way to debug it. 773 00:39:36,940 --> 00:39:39,230 And that takes it being small enough so you can 774 00:39:39,230 --> 00:39:41,690 hold it in your head. 775 00:39:41,690 --> 00:39:45,020 Garbage collectors are special in this way. 776 00:39:45,020 --> 00:39:47,130 So every reasonable garbage collector has gotten small, 777 00:39:47,130 --> 00:39:52,430 and generally small programs are fast. Yes. 778 00:39:52,430 --> 00:39:53,650 AUDIENCE: Can you repeat the name of this 779 00:39:53,650 --> 00:39:54,510 technique once again? 780 00:39:54,510 --> 00:39:56,220 PROFESSOR: That's the Minsky-Feinchel-Yochelson 781 00:39:56,220 --> 00:39:58,420 garbage collector. 782 00:39:58,420 --> 00:39:59,340 AUDIENCE: You got that? 783 00:39:59,340 --> 00:40:02,210 PROFESSOR: Minsky invented it in '61 for the RLE PDP-1. 784 00:40:02,210 --> 00:40:07,410 A version of it was developed and elaborated to be used in 785 00:40:07,410 --> 00:40:12,410 Multics Maclisp by Feinchel and Yochelson in somewhere 786 00:40:12,410 --> 00:40:19,570 around 1968 or '69. 787 00:40:19,570 --> 00:40:20,650 OK. 788 00:40:20,650 --> 00:40:22,640 Let's take a break. 789 00:40:22,640 --> 00:40:22,934 [MUSIC: "JESU, JOY OF MAN'S DESIRING" BY 790 00:40:22,934 --> 00:40:24,184 JOHANN SEBASTIAN BACH] 791 00:40:24,184 --> 00:41:17,310 792 00:41:17,310 --> 00:41:21,600 PROFESSOR: Well we've come to the end of this subject, and 793 00:41:21,600 --> 00:41:24,860 we've already shown you a universal machine which is 794 00:41:24,860 --> 00:41:26,740 down to evaluator. 795 00:41:26,740 --> 00:41:28,980 It's down to the level of detail you could imagine you 796 00:41:28,980 --> 00:41:30,420 could make one. 797 00:41:30,420 --> 00:41:34,390 This is a particular implementation of Lisp, built 798 00:41:34,390 --> 00:41:37,530 on one of those scheme chips that was talked about 799 00:41:37,530 --> 00:41:39,180 yesterday, sitting over here. 800 00:41:39,180 --> 00:41:42,990 This is mostly interface to somebody's memory with a 801 00:41:42,990 --> 00:41:45,010 little bit of timing and other such stuff. 802 00:41:45,010 --> 00:41:48,760 But this fellow actually ran Lisp at a fairly reasonable 803 00:41:48,760 --> 00:41:50,610 rate, as interpretive. 804 00:41:50,610 --> 00:41:56,500 It ran Lisp as fast as a DEC PDP-10 back in 1979. 805 00:41:56,500 --> 00:41:59,870 And so it's gotten pretty hardware. 806 00:41:59,870 --> 00:42:02,470 Pretty concrete. 807 00:42:02,470 --> 00:42:05,000 We've also downed you a bit with the 808 00:42:05,000 --> 00:42:07,370 things you can compute. 809 00:42:07,370 --> 00:42:11,850 But is it the case that there are things we can't compute? 810 00:42:11,850 --> 00:42:14,690 And so I'd like to end this with showing you some things 811 00:42:14,690 --> 00:42:18,190 that you'd like be able to compute that you can't. 812 00:42:18,190 --> 00:42:22,720 The answer is yes, there are things you can't compute. 813 00:42:22,720 --> 00:42:28,200 For example, something you'd really like is-- 814 00:42:28,200 --> 00:42:30,210 if you're writing [UNINTELLIGIBLE], you'd like a 815 00:42:30,210 --> 00:42:32,800 program that would check that the thing you're 816 00:42:32,800 --> 00:42:34,630 going to do will work. 817 00:42:34,630 --> 00:42:36,080 Wouldn't that be nice? 818 00:42:36,080 --> 00:42:37,960 You'd like something that would catch infinite loops, 819 00:42:37,960 --> 00:42:43,190 for example, in programs that were written by users. 820 00:42:43,190 --> 00:42:45,890 But in general you can't write such a program that will read 821 00:42:45,890 --> 00:42:48,760 any program and determine whether or not it's an 822 00:42:48,760 --> 00:42:50,990 infinite loop. 823 00:42:50,990 --> 00:42:51,685 Let me show you that. 824 00:42:51,685 --> 00:42:53,340 It's a little bit of a minor mathematics. 825 00:42:53,340 --> 00:42:58,780 826 00:42:58,780 --> 00:43:01,360 Let's imagine that we just had a mathematical 827 00:43:01,360 --> 00:43:02,620 function before we start. 828 00:43:02,620 --> 00:43:12,980 And there is one, called s, which takes a procedure and 829 00:43:12,980 --> 00:43:14,230 its argument, a. 830 00:43:14,230 --> 00:43:19,320 831 00:43:19,320 --> 00:43:24,250 And what s does is it determines whether or not it's 832 00:43:24,250 --> 00:43:26,632 safe to run p on a. 833 00:43:26,632 --> 00:43:34,500 And what I mean by that is this: it's true if p applied 834 00:43:34,500 --> 00:43:45,330 to a will converge to a value without an error. 835 00:43:45,330 --> 00:43:52,365 836 00:43:52,365 --> 00:44:06,890 And it's false if p of a loops forever or makes an error. 837 00:44:06,890 --> 00:44:15,000 838 00:44:15,000 --> 00:44:18,780 Now that's surely a function. 839 00:44:18,780 --> 00:44:21,830 There is some for every procedure and for every 840 00:44:21,830 --> 00:44:25,900 argument you could give it that is either true or false 841 00:44:25,900 --> 00:44:28,440 that it converges without making an error. 842 00:44:28,440 --> 00:44:31,770 And you could make a giant table of them. 843 00:44:31,770 --> 00:44:34,710 But the question is, can you write a procedure that compute 844 00:44:34,710 --> 00:44:37,430 the values of this function? 845 00:44:37,430 --> 00:44:39,720 Well let's assume that we can. 846 00:44:39,720 --> 00:44:58,740 Suppose that we have a procedure called "safe" that 847 00:44:58,740 --> 00:44:59,990 computes the value of s. 848 00:44:59,990 --> 00:45:12,170 849 00:45:12,170 --> 00:45:17,620 Now I'm going to show you by several methods that 850 00:45:17,620 --> 00:45:19,760 you can't do this. 851 00:45:19,760 --> 00:45:22,300 The easiest one, or the first one, let's define a procedure 852 00:45:22,300 --> 00:45:23,810 called diag1. 853 00:45:23,810 --> 00:45:38,250 Given that we have safe, we can define diag1 to be the 854 00:45:38,250 --> 00:45:43,430 procedure of one argument, p, which has the following 855 00:45:43,430 --> 00:45:44,780 properties. 856 00:45:44,780 --> 00:45:54,620 If if it's safe to apply p to itself, then I wish to have an 857 00:45:54,620 --> 00:45:55,870 infinite loop. 858 00:45:55,870 --> 00:45:59,330 859 00:45:59,330 --> 00:46:00,715 Otherwise I'm going to return 3. 860 00:46:00,715 --> 00:46:03,680 861 00:46:03,680 --> 00:46:04,470 Remember it was 42. 862 00:46:04,470 --> 00:46:07,060 What's the answer to the big question? 863 00:46:07,060 --> 00:46:08,525 Where of course we know what an infinite loop is. 864 00:46:08,525 --> 00:46:12,050 865 00:46:12,050 --> 00:46:16,130 Infinite loop, to be a procedure of no arguments, 866 00:46:16,130 --> 00:46:18,430 which is that nice lambda calculus loop. 867 00:46:18,430 --> 00:46:24,680 Lambda of x, x of x, applied to lambda of x, x of x. 868 00:46:24,680 --> 00:46:26,550 So there's nothing left to the imagination here. 869 00:46:26,550 --> 00:46:29,830 870 00:46:29,830 --> 00:46:32,500 Well let's see what the story is. 871 00:46:32,500 --> 00:46:38,100 I'm supposing it's the case that we worry about the 872 00:46:38,100 --> 00:46:43,180 procedure called diag1 applied to diag1. 873 00:46:43,180 --> 00:46:45,860 874 00:46:45,860 --> 00:46:49,970 Well what could it possibly be? 875 00:46:49,970 --> 00:46:51,390 Well I don't know. 876 00:46:51,390 --> 00:46:57,310 We're going to substitute diag1 for p in the body here. 877 00:46:57,310 --> 00:47:00,220 Well is it safe to compute diag1 of diag1? 878 00:47:00,220 --> 00:47:00,780 I don't know. 879 00:47:00,780 --> 00:47:03,400 There are two possibilities. 880 00:47:03,400 --> 00:47:06,260 If it's safe to compute diag1 of diag1 that means it 881 00:47:06,260 --> 00:47:08,490 shouldn't loop. 882 00:47:08,490 --> 00:47:09,540 That means I go to here, but then I 883 00:47:09,540 --> 00:47:10,560 produce an infinite loop. 884 00:47:10,560 --> 00:47:12,210 So it can't be safe. 885 00:47:12,210 --> 00:47:15,055 But if it's not safe to compute diag1 of diag1 then 886 00:47:15,055 --> 00:47:16,020 the answer to this is 3. 887 00:47:16,020 --> 00:47:20,530 But that's diag1 of diag1, so it had to be safe. 888 00:47:20,530 --> 00:47:27,470 So therefore by contradiction you cannot produce safe. 889 00:47:27,470 --> 00:47:30,565 For those of you who were boggled by that one I'm going 890 00:47:30,565 --> 00:47:32,820 to say it again, in a different way. 891 00:47:32,820 --> 00:47:35,530 Listen to one more alternative. 892 00:47:35,530 --> 00:47:36,780 Let's define diag2. 893 00:47:36,780 --> 00:47:39,840 894 00:47:39,840 --> 00:47:45,260 These are named diag because of Cantor's diagonal argument. 895 00:47:45,260 --> 00:47:48,180 These are instances of a famous argument which was 896 00:47:48,180 --> 00:47:52,070 originally used by Cantor in the late part of the last 897 00:47:52,070 --> 00:47:56,420 century to prove that the real numbers were not countable, 898 00:47:56,420 --> 00:47:58,160 that there are too many real numbers to 899 00:47:58,160 --> 00:48:00,190 be counted by integers. 900 00:48:00,190 --> 00:48:02,710 That there are more points on a line, for example, than 901 00:48:02,710 --> 00:48:05,260 there are counting numbers. 902 00:48:05,260 --> 00:48:07,190 It may or may not be obvious, and I don't want to 903 00:48:07,190 --> 00:48:08,440 get into that now. 904 00:48:08,440 --> 00:48:10,900 905 00:48:10,900 --> 00:48:15,820 But diag2 is again a procedure of one argument p. 906 00:48:15,820 --> 00:48:19,220 It's almost the same as the previous one, which is, if 907 00:48:19,220 --> 00:48:26,445 it's safe to compute p on p, then I'm going to produce-- 908 00:48:26,445 --> 00:48:29,310 909 00:48:29,310 --> 00:48:34,010 then I want to compute some other things 910 00:48:34,010 --> 00:48:38,960 other than p of p. 911 00:48:38,960 --> 00:48:40,210 Otherwise I'm going to put out false. 912 00:48:40,210 --> 00:48:43,600 913 00:48:43,600 --> 00:48:46,510 Where other then it says, whatever p of p, I'm going to 914 00:48:46,510 --> 00:48:48,880 put out something else. 915 00:48:48,880 --> 00:48:51,860 I can give you an example of a definition of other than which 916 00:48:51,860 --> 00:48:53,890 I think works. 917 00:48:53,890 --> 00:48:55,640 Let's see. 918 00:48:55,640 --> 00:48:56,330 Yes. 919 00:48:56,330 --> 00:49:06,580 Where other than be a procedure of one argument x 920 00:49:06,580 --> 00:49:14,090 which says, if its eq x to, say, quote a, then the answer 921 00:49:14,090 --> 00:49:15,720 is quote b. 922 00:49:15,720 --> 00:49:16,970 Otherwise it's quote a. 923 00:49:16,970 --> 00:49:20,090 924 00:49:20,090 --> 00:49:22,770 That always produces something which is not what 925 00:49:22,770 --> 00:49:25,350 its argument is. 926 00:49:25,350 --> 00:49:26,540 That's all it is. 927 00:49:26,540 --> 00:49:28,250 That's all I wanted. 928 00:49:28,250 --> 00:49:30,640 Well now let's consider this one, diag2 of diag2. 929 00:49:30,640 --> 00:49:38,220 930 00:49:38,220 --> 00:49:39,560 Well look. 931 00:49:39,560 --> 00:49:42,950 This only does something dangerous, like calling p of 932 00:49:42,950 --> 00:49:47,470 p, if it's safe to do so. 933 00:49:47,470 --> 00:49:51,590 So if safe defined at all, if you can define such a 934 00:49:51,590 --> 00:49:55,680 procedure, safe, then this procedure is always defined 935 00:49:55,680 --> 00:49:57,225 and therefore safe on any inputs. 936 00:49:57,225 --> 00:50:01,540 937 00:50:01,540 --> 00:50:11,770 So diag2 of diag2 must reduce to other than diag2 of diag2. 938 00:50:11,770 --> 00:50:15,496 939 00:50:15,496 --> 00:50:20,020 And that doesn't make sense, so we have a contradiction, 940 00:50:20,020 --> 00:50:22,950 and therefore we can't define safe. 941 00:50:22,950 --> 00:50:27,210 I just waned to do that twice, slightly differently, so you 942 00:50:27,210 --> 00:50:32,260 wouldn't feel that the first one was a trick. 943 00:50:32,260 --> 00:50:34,275 They may be both tricks, but they're at 944 00:50:34,275 --> 00:50:37,300 least slightly different. 945 00:50:37,300 --> 00:50:40,080 So I suppose that pretty much wraps it up. 946 00:50:40,080 --> 00:50:43,540 I've just proved what we call the halting theorem, and I 947 00:50:43,540 --> 00:50:46,720 suppose with that we're going to halt. 948 00:50:46,720 --> 00:50:47,970 I hope you have a good time. 949 00:50:47,970 --> 00:50:50,900 950 00:50:50,900 --> 00:50:53,300 Are there any questions? 951 00:50:53,300 --> 00:50:53,810 Yes. 952 00:50:53,810 --> 00:50:56,940 AUDIENCE: What is the value of s of diag1? 953 00:50:56,940 --> 00:50:57,430 PROFESSOR: Of what? 954 00:50:57,430 --> 00:51:00,120 AUDIENCE: S of diag1. 955 00:51:00,120 --> 00:51:02,340 If you said s is a function and we can 956 00:51:02,340 --> 00:51:02,620 [INTERPOSING VOICES] 957 00:51:02,620 --> 00:51:03,870 PROFESSOR: Oh, I don't know. 958 00:51:03,870 --> 00:51:04,350 I don't know. 959 00:51:04,350 --> 00:51:06,850 It's a function, but I don't know how to compute it. 960 00:51:06,850 --> 00:51:08,610 I can't do it. 961 00:51:08,610 --> 00:51:11,530 I'm just a machine, too. 962 00:51:11,530 --> 00:51:12,210 Right? 963 00:51:12,210 --> 00:51:14,670 There's no machine that in principle-- 964 00:51:14,670 --> 00:51:16,690 it might be that in that particular case you just 965 00:51:16,690 --> 00:51:18,580 asked, with some thinking I could figure it out. 966 00:51:18,580 --> 00:51:21,670 But in general I can't compute the value of s any better than 967 00:51:21,670 --> 00:51:23,780 any other machine can. 968 00:51:23,780 --> 00:51:27,210 There is such a function, it's just that no machine can be 969 00:51:27,210 --> 00:51:29,580 built to compute it. 970 00:51:29,580 --> 00:51:32,980 Now there's a way of saying that that should not be 971 00:51:32,980 --> 00:51:35,350 surprising. 972 00:51:35,350 --> 00:51:36,390 Going through this-- 973 00:51:36,390 --> 00:51:41,020 I mean, I don't have time to do this here, but the number 974 00:51:41,020 --> 00:51:44,600 of functions is very large. 975 00:51:44,600 --> 00:51:48,210 If there's a certain number of answers possible and a certain 976 00:51:48,210 --> 00:51:50,520 number of inputs possible, then it's the number of 977 00:51:50,520 --> 00:51:52,290 answers raised to the number inputs is the number of 978 00:51:52,290 --> 00:51:54,720 possible functions. 979 00:51:54,720 --> 00:51:55,970 On one variable. 980 00:51:55,970 --> 00:51:58,150 981 00:51:58,150 --> 00:52:03,690 Now that's always bigger than the thing you're raising to, 982 00:52:03,690 --> 00:52:05,480 the exponent. 983 00:52:05,480 --> 00:52:12,150 The number of functions is larger than the number of 984 00:52:12,150 --> 00:52:15,340 programs that one can write, by an 985 00:52:15,340 --> 00:52:17,840 infinity counting argument. 986 00:52:17,840 --> 00:52:19,475 And it's much larger. 987 00:52:19,475 --> 00:52:22,540 So there must be a lot of functions that can't be 988 00:52:22,540 --> 00:52:26,280 computed by programs. 989 00:52:26,280 --> 00:52:27,300 AUDIENCE: A few moments ago you were talking about 990 00:52:27,300 --> 00:52:30,640 specifications and automatic generation of solutions. 991 00:52:30,640 --> 00:52:33,360 Do you see any steps between specifications and solutions? 992 00:52:33,360 --> 00:52:37,250 993 00:52:37,250 --> 00:52:38,720 PROFESSOR: Steps between. 994 00:52:38,720 --> 00:52:42,720 You mean, you're saying, how you go about constructing 995 00:52:42,720 --> 00:52:45,205 devices given that have specifications for the device? 996 00:52:45,205 --> 00:52:45,500 Sure. 997 00:52:45,500 --> 00:52:48,540 AUDIENCE: There's a lot of software engineering that goes 998 00:52:48,540 --> 00:52:51,703 through specifications through many layers of design and then 999 00:52:51,703 --> 00:52:52,430 implementation. 1000 00:52:52,430 --> 00:52:52,850 PROFESSOR: Yes? 1001 00:52:52,850 --> 00:52:55,600 AUDIENCE: I was curious if you think that's realistic. 1002 00:52:55,600 --> 00:52:57,210 PROFESSOR: Well I think that some of it's realistic and 1003 00:52:57,210 --> 00:52:58,100 some of it isn't. 1004 00:52:58,100 --> 00:53:01,680 I mean, surely if I want to build an electrical filter and 1005 00:53:01,680 --> 00:53:07,160 I have a rather interesting possibility. 1006 00:53:07,160 --> 00:53:12,180 Supposing I want to build a thing that matches some power 1007 00:53:12,180 --> 00:53:19,906 output to the radio transmitter, to some antenna. 1008 00:53:19,906 --> 00:53:21,970 And I'm really out of this power-- 1009 00:53:21,970 --> 00:53:23,230 it's output tube out here. 1010 00:53:23,230 --> 00:53:25,920 And the problem is that they have different impedances. 1011 00:53:25,920 --> 00:53:27,550 I want them to match the impedances. 1012 00:53:27,550 --> 00:53:29,920 I also want to make a filter in there which is going to get 1013 00:53:29,920 --> 00:53:32,780 rid of some harmonic radiation. 1014 00:53:32,780 --> 00:53:36,270 Well one old-fashioned technique for doing this is 1015 00:53:36,270 --> 00:53:38,860 called image impedances, or something like that. 1016 00:53:38,860 --> 00:53:40,240 And what you do is you say you have a basic 1017 00:53:40,240 --> 00:53:43,300 module called an L-section. 1018 00:53:43,300 --> 00:53:44,550 Looks like this. 1019 00:53:44,550 --> 00:53:47,080 1020 00:53:47,080 --> 00:53:50,470 If I happen to connect this to some resistance, r, and if I 1021 00:53:50,470 --> 00:53:55,150 make this impedance x, xl, and if it happens to be q times r, 1022 00:53:55,150 --> 00:53:59,710 then this produces a low pass filter with a q square plus 1023 00:53:59,710 --> 00:54:02,110 one impedance match. 1024 00:54:02,110 --> 00:54:03,120 Just what I need. 1025 00:54:03,120 --> 00:54:04,610 Because now I can take two of these, hook them 1026 00:54:04,610 --> 00:54:06,510 together like this. 1027 00:54:06,510 --> 00:54:11,660 1028 00:54:11,660 --> 00:54:16,570 OK, and I take another one and I'll hook them 1029 00:54:16,570 --> 00:54:18,290 together like that. 1030 00:54:18,290 --> 00:54:20,320 And I have two L-sections hooked together. 1031 00:54:20,320 --> 00:54:22,460 And this will step the impedance down to one that I 1032 00:54:22,460 --> 00:54:25,530 know, and this will step it up to one I know. 1033 00:54:25,530 --> 00:54:26,710 Each of these is a low pass filter 1034 00:54:26,710 --> 00:54:28,090 getting rid of some harmonics. 1035 00:54:28,090 --> 00:54:30,270 It's good filter, it's called a pie-section filter. 1036 00:54:30,270 --> 00:54:31,700 Great. 1037 00:54:31,700 --> 00:54:34,300 Except for the fact that in doing what I just did, I've 1038 00:54:34,300 --> 00:54:38,620 made a terrible inefficiency in this system. 1039 00:54:38,620 --> 00:54:41,620 I've made two coils where I should have made one. 1040 00:54:41,620 --> 00:54:45,200 And the problem with most software engineering art is 1041 00:54:45,200 --> 00:54:47,440 that there's no mechanism, other than peephole 1042 00:54:47,440 --> 00:54:50,280 optimization and compilers, for getting rid of the 1043 00:54:50,280 --> 00:54:52,810 redundant parts that are constructed when doing top 1044 00:54:52,810 --> 00:54:55,350 down design. 1045 00:54:55,350 --> 00:54:57,160 It's even worse, there are lots of very important 1046 00:54:57,160 --> 00:55:01,110 structures that you can't construct at all this way. 1047 00:55:01,110 --> 00:55:03,940 So I think that the standard top down design is a rather 1048 00:55:03,940 --> 00:55:05,710 shallow business. 1049 00:55:05,710 --> 00:55:08,315 Doesn't really capture what people want to do in design. 1050 00:55:08,315 --> 00:55:10,100 I'll give you another electrical example. 1051 00:55:10,100 --> 00:55:12,140 Electrical examples are so much clearer than 1052 00:55:12,140 --> 00:55:14,440 computational examples, because computation examples 1053 00:55:14,440 --> 00:55:17,220 require a certain degree of complexity to explain them. 1054 00:55:17,220 --> 00:55:19,650 But one of my favorite examples in the electrical 1055 00:55:19,650 --> 00:55:23,330 world is how would I ever come up with the output stage of 1056 00:55:23,330 --> 00:55:27,530 this inter-stage connection in an IF amplifier. 1057 00:55:27,530 --> 00:55:32,410 It's a little transistor here, and let's see. 1058 00:55:32,410 --> 00:55:37,560 Well I'm going to have a tank, and I'm going to hook this up 1059 00:55:37,560 --> 00:55:43,040 to, say, I'm going to link-couple that to the input 1060 00:55:43,040 --> 00:55:44,850 of the next stage. 1061 00:55:44,850 --> 00:55:48,580 Here's a perfectly plausible plan-- 1062 00:55:48,580 --> 00:55:51,070 well except for the fact that since I put that going up I 1063 00:55:51,070 --> 00:55:53,170 should make that going that way. 1064 00:55:53,170 --> 00:55:56,050 Here's a perfectly plausible plan for a-- 1065 00:55:56,050 --> 00:55:57,270 no I shouldn't. 1066 00:55:57,270 --> 00:55:57,940 I'm dumb. 1067 00:55:57,940 --> 00:55:59,690 Excuse me. 1068 00:55:59,690 --> 00:56:00,730 Doesn't matter. 1069 00:56:00,730 --> 00:56:01,540 The point is [UNINTELLIGIBLE] 1070 00:56:01,540 --> 00:56:02,560 plan for a couple [UNINTELLIGIBLE] 1071 00:56:02,560 --> 00:56:04,590 stages together. 1072 00:56:04,590 --> 00:56:07,620 Now what the problem is is what's this hierarchically? 1073 00:56:07,620 --> 00:56:09,480 It's not one thing. 1074 00:56:09,480 --> 00:56:11,990 Hierarchically it doesn't make any sense at all. 1075 00:56:11,990 --> 00:56:17,230 It's the inductance of a tuned circuit, it's the primary of a 1076 00:56:17,230 --> 00:56:22,350 transformer, and it's also the DC path by which bias 1077 00:56:22,350 --> 00:56:26,460 conditions get to the collector of that transistor. 1078 00:56:26,460 --> 00:56:29,170 And there's no simple top-down design that's going to produce 1079 00:56:29,170 --> 00:56:33,400 a structure like that with so many overlapping uses for a 1080 00:56:33,400 --> 00:56:34,530 particular thing. 1081 00:56:34,530 --> 00:56:39,015 Playing Scrabble, where you have to do triple word scores, 1082 00:56:39,015 --> 00:56:44,950 or whatever, is not so easy in top-down design strategy. 1083 00:56:44,950 --> 00:56:48,100 Yet most of real engineering is based on getting the most 1084 00:56:48,100 --> 00:56:52,140 oomph for effort. 1085 00:56:52,140 --> 00:56:54,860 And that's what you're seeing here. 1086 00:56:54,860 --> 00:56:55,550 Yeah? 1087 00:56:55,550 --> 00:56:56,810 AUDIENCE: Is this the last question? 1088 00:56:56,810 --> 00:57:00,282 1089 00:57:00,282 --> 00:57:18,640 [LAUGHTER] 1090 00:57:18,640 --> 00:57:19,890 PROFESSOR: Apparently so. 1091 00:57:19,890 --> 00:57:23,240 1092 00:57:23,240 --> 00:57:26,092 Thank you. 1093 00:57:26,092 --> 00:57:39,040 [APPLAUSE] 1094 00:57:39,040 --> 00:57:39,383 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 1095 00:57:39,383 --> 00:57:40,633 JOHANN SEBASTIAN BACH] 1096 00:57:40,633 --> 00:58:53,546