WEBVTT 00:00:04.970 --> 00:00:05.285 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 00:00:05.285 --> 00:00:06.535 JOHANN SEBASTIAN BACH] 00:00:18.910 --> 00:00:22.140 PROFESSOR: Well, there's one bit of mystery left, which I'd 00:00:22.140 --> 00:00:24.440 like to get rid of right now. 00:00:24.440 --> 00:00:28.210 And that's that we've been blithely doing things like 00:00:28.210 --> 00:00:33.660 cons assuming there's always another one. 00:00:33.660 --> 00:00:37.060 That we've been doing these things like car-ing and 00:00:37.060 --> 00:00:38.750 cdr-ing and assuming that we had some idea 00:00:38.750 --> 00:00:40.020 how this can be done. 00:00:40.020 --> 00:00:43.800 Now indeed we said that that's equivalent to having 00:00:43.800 --> 00:00:45.780 procedures. 00:00:45.780 --> 00:00:48.360 But that doesn't really solve the problem, because the 00:00:48.360 --> 00:00:50.470 procedure need all sorts of complicated mechanisms like 00:00:50.470 --> 00:00:53.010 environment structures and things like that to work. 00:00:53.010 --> 00:00:55.770 And those were ultimately made out of conses in the model 00:00:55.770 --> 00:00:59.380 that we had, so that really doesn't solve the problem. 00:00:59.380 --> 00:01:02.860 Now the problem here is the glue the data 00:01:02.860 --> 00:01:04.760 structure's made out of. 00:01:04.760 --> 00:01:07.370 What kind of possible thing could it be? 00:01:07.370 --> 00:01:11.060 We've been showing you things like a machine, a computer 00:01:11.060 --> 00:01:14.700 that has a controller, and some 00:01:14.700 --> 00:01:16.980 registers, and maybe a stack. 00:01:16.980 --> 00:01:18.270 And we haven't said anything about, for 00:01:18.270 --> 00:01:20.570 example, larger memory. 00:01:20.570 --> 00:01:23.740 And I think that's what we have to worry about right now. 00:01:23.740 --> 00:01:27.160 But just to make it perfectly clear that this is an 00:01:27.160 --> 00:01:31.320 inessential, purely implementational thing, I'd 00:01:31.320 --> 00:01:33.500 like to show you, for example, how you can do it 00:01:33.500 --> 00:01:34.800 all with the numbers. 00:01:34.800 --> 00:01:37.590 That's an easy one. 00:01:37.590 --> 00:01:45.020 Famous fellow by the name of Godel, a logician at the end 00:01:45.020 --> 00:01:51.050 of the 1930s, invented a very clever way of encoding the 00:01:51.050 --> 00:01:54.320 complicated expressions as numbers. 00:01:54.320 --> 00:01:55.540 For example-- 00:01:55.540 --> 00:01:58.250 I'm not saying exactly what Godel's scheme is, because he 00:01:58.250 --> 00:01:59.660 didn't use words like cons. 00:01:59.660 --> 00:02:03.090 He had other kinds of ways of combining to make expressions. 00:02:03.090 --> 00:02:05.860 But he said, I'm going to assign a number to every 00:02:05.860 --> 00:02:07.920 algebraic expression. 00:02:07.920 --> 00:02:09.970 And the way I'm going to manufacture these numbers is 00:02:09.970 --> 00:02:12.470 by combining the numbers of the parts. 00:02:12.470 --> 00:02:15.880 So for example, what we were doing our world, we could say 00:02:15.880 --> 00:02:34.130 that if objects are represented by numbers, then 00:02:34.130 --> 00:02:42.660 cons of x and y could be represented by 2 to the x 00:02:42.660 --> 00:02:46.130 times 2 to the y. 00:02:46.130 --> 00:02:49.560 Because then we could extract the parts. 00:02:49.560 --> 00:02:57.500 We could say, for example, that then car of, say, x is 00:02:57.500 --> 00:03:06.690 the number of factors of 2 in x. 00:03:06.690 --> 00:03:10.690 And of course cdr is the same thing. 00:03:10.690 --> 00:03:16.510 It's the number of factors of 3 in x. 00:03:16.510 --> 00:03:19.660 Now this is a perfectly reasonable scheme, except for 00:03:19.660 --> 00:03:22.870 the fact that the numbers rapidly get to be much larger 00:03:22.870 --> 00:03:25.500 in number of digits than the number of 00:03:25.500 --> 00:03:27.950 protons in the universe. 00:03:27.950 --> 00:03:30.420 So there's no easy way to use this scheme other than the 00:03:30.420 --> 00:03:33.430 theoretical one. 00:03:33.430 --> 00:03:37.010 On the other hand, there are other ways of representing 00:03:37.010 --> 00:03:38.450 these things. 00:03:38.450 --> 00:03:44.010 We have been thinking in terms of little boxes. 00:03:44.010 --> 00:03:47.000 We've been thinking about our cons structures as looking 00:03:47.000 --> 00:03:50.280 sort of like this. 00:03:50.280 --> 00:03:53.610 They're little pigeon holes with things in them. 00:03:53.610 --> 00:03:57.210 And of course we arrange them in little trees. 00:03:57.210 --> 00:04:00.680 I wish that the semiconductor manufacturers would supply me 00:04:00.680 --> 00:04:04.280 with something appropriate for this, but actually what they 00:04:04.280 --> 00:04:09.380 do supply me with is a linear memory. 00:04:09.380 --> 00:04:15.170 Memory is sort of a big pile of pigeonholes, 00:04:15.170 --> 00:04:17.720 pigeonholes like this. 00:04:17.720 --> 00:04:21.470 Each of which can hold a certain sized object, a fixed 00:04:21.470 --> 00:04:23.390 size object. 00:04:23.390 --> 00:04:25.890 So, for example, a complicated list with 25 elements won't 00:04:25.890 --> 00:04:28.550 fit in one of these. 00:04:28.550 --> 00:04:30.600 However, each of these is indexed by an address. 00:04:33.970 --> 00:04:36.750 So the address might be zero here, one here, two here, 00:04:36.750 --> 00:04:38.060 three here, and so on. 00:04:38.060 --> 00:04:40.400 That we write these down as numbers is unimportant. 00:04:40.400 --> 00:04:42.710 What matters is that they're distinct as a way to get to 00:04:42.710 --> 00:04:44.970 the next one. 00:04:44.970 --> 00:04:48.300 And inside of each of these, we can stuff something into 00:04:48.300 --> 00:04:49.530 these pigeonholes. 00:04:49.530 --> 00:04:52.300 That's what memory is like, for those of you who haven't 00:04:52.300 --> 00:04:53.550 built a computer. 00:04:56.690 --> 00:04:59.280 Now the problem is how are we going to impose on this type 00:04:59.280 --> 00:05:03.290 of structure, this nice tree structure. 00:05:03.290 --> 00:05:05.480 Well it's not very hard, and there have been numerous 00:05:05.480 --> 00:05:06.630 schemes involved in this. 00:05:06.630 --> 00:05:09.930 The most important one is to say, well assuming that the 00:05:09.930 --> 00:05:13.920 semiconductor manufacturer allows me to arrange my memory 00:05:13.920 --> 00:05:16.390 so that one of these pigeonholes is big enough to 00:05:16.390 --> 00:05:21.706 hold the address of another I haven't made. 00:05:21.706 --> 00:05:23.730 Now it actually has to be a little bit bigger because I 00:05:23.730 --> 00:05:28.215 have to also install or store some information as to a tag 00:05:28.215 --> 00:05:30.390 which describes the kind of thing that's there. 00:05:30.390 --> 00:05:32.350 And we'll see that in a second. 00:05:32.350 --> 00:05:34.560 And of course if the semiconductor manufacturer 00:05:34.560 --> 00:05:37.470 doesn't arrange it so I can do that, then of course I can, 00:05:37.470 --> 00:05:40.910 with some cleverness, arrange combinations of these to fit 00:05:40.910 --> 00:05:43.770 together in that way. 00:05:43.770 --> 00:05:48.510 So we're going to have to imagine imposing this 00:05:48.510 --> 00:05:51.740 complicated tree structure on our nice linear memory. 00:05:51.740 --> 00:05:57.540 If we look at the first still store, we see a classic scheme 00:05:57.540 --> 00:05:59.490 for doing that. 00:05:59.490 --> 00:06:03.910 It's a standard way of representing Lisp structures 00:06:03.910 --> 00:06:05.980 in a linear memory. 00:06:05.980 --> 00:06:12.030 What we do is we divide this memory into two parts. 00:06:12.030 --> 00:06:17.580 An array called the cars, and an array called the cdrs. 00:06:17.580 --> 00:06:20.470 Now whether those happen to be sequential addresses or 00:06:20.470 --> 00:06:22.560 whatever, it's not important. 00:06:22.560 --> 00:06:25.800 That's somebody's implementation details. 00:06:25.800 --> 00:06:28.960 But there are two arrays here. 00:06:28.960 --> 00:06:34.840 Linear arrays indexed by sequential indices like this. 00:06:34.840 --> 00:06:36.150 What is stored in each of these 00:06:36.150 --> 00:06:41.430 pigeonholes is a typed object. 00:06:41.430 --> 00:06:44.840 And what we have here are types which begin with letters 00:06:44.840 --> 00:06:47.790 like p, standing for a pair. 00:06:47.790 --> 00:06:50.040 Or n, standing for a number. 00:06:50.040 --> 00:06:57.290 Or e, standing for an empty list. The end of the list. And 00:06:57.290 --> 00:07:00.420 so if we wish to represent an object like this, the list 00:07:00.420 --> 00:07:04.310 beginning with 1, 2 and then having a 3 and a 4 as its 00:07:04.310 --> 00:07:06.430 second and third elements. 00:07:06.430 --> 00:07:10.220 A list containing a list as its first part and then two 00:07:10.220 --> 00:07:12.610 numbers as a second and third parts. 00:07:12.610 --> 00:07:15.250 Then of course we draw it sort of like this these days, in 00:07:15.250 --> 00:07:17.320 box-and-pointer notation. 00:07:17.320 --> 00:07:21.190 And you see, these are the three cells that have as their 00:07:21.190 --> 00:07:28.390 car pointer the object which is either 1, 2 or 3 or 4. 00:07:28.390 --> 00:07:30.860 And then of course the 1, 2, the car of this entire 00:07:30.860 --> 00:07:33.870 structure, is itself a substructure which contains a 00:07:33.870 --> 00:07:35.940 sublist like that. 00:07:35.940 --> 00:07:39.970 What I'm about to do is put down places which are-- 00:07:39.970 --> 00:07:41.880 I'm going to assign indices. 00:07:41.880 --> 00:07:45.530 Like this 1, over here, represents the 00:07:45.530 --> 00:07:46.850 index of this cell. 00:07:49.850 --> 00:07:55.540 But that pointer that we see here is a reference to the 00:07:55.540 --> 00:07:57.640 pair of pigeonholes in the cars and the cdrs that are 00:07:57.640 --> 00:08:02.000 labeled by 1 in my linear memory down here. 00:08:02.000 --> 00:08:05.920 So if I wish to impose this structure on my linear memory, 00:08:05.920 --> 00:08:08.780 what I do is I say, oh yes, why don't we drop 00:08:08.780 --> 00:08:12.220 this into cell 1? 00:08:12.220 --> 00:08:12.660 I pick one. 00:08:12.660 --> 00:08:14.270 There's 1. 00:08:14.270 --> 00:08:16.640 And that says that its car, I'm going to 00:08:16.640 --> 00:08:17.950 assign it to be a pair. 00:08:17.950 --> 00:08:22.590 It's a pair, which is in index 5. 00:08:22.590 --> 00:08:26.360 And the cdr, which is this one over here, is a pair which I'm 00:08:26.360 --> 00:08:28.340 going to stick into place 2. 00:08:28.340 --> 00:08:30.890 p2. 00:08:30.890 --> 00:08:32.950 And take a look at p2. 00:08:32.950 --> 00:08:37.550 Oh yes, well p2 is a thing whose car is the number 3, so 00:08:37.550 --> 00:08:39.520 as you see, an n3. 00:08:39.520 --> 00:08:46.640 And whose cdr, over here, is a pair, which lives in place 4. 00:08:46.640 --> 00:08:48.650 So that's what this p4 is. 00:08:48.650 --> 00:08:56.200 p4 is a number whose value is 4 in its car and whose cdr is 00:08:56.200 --> 00:08:59.170 an empty list right there. 00:08:59.170 --> 00:09:00.690 And that ends it. 00:09:00.690 --> 00:09:05.750 So this is the traditional way of representing this kind of 00:09:05.750 --> 00:09:11.620 binary tree in a linear memory. 00:09:11.620 --> 00:09:15.770 Now the next question, of course, that we might want to 00:09:15.770 --> 00:09:18.440 worry about is just a little bit of implementation. 00:09:18.440 --> 00:09:22.690 That means that when I write procedures of the form 00:09:22.690 --> 00:09:24.600 assigned a, [UNINTELLIGIBLE] procedures-- 00:09:24.600 --> 00:09:29.000 lines of register machine code of the form assigned a, the 00:09:29.000 --> 00:09:30.140 car of [UNINTELLIGIBLE] 00:09:30.140 --> 00:09:38.740 b, what I really mean is addressing these elements. 00:09:38.740 --> 00:09:44.470 And so we're going to think of that as a abbreviation for it. 00:09:44.470 --> 00:09:46.720 Now of course in order to write that down I'm going to 00:09:46.720 --> 00:09:49.140 introduce some sort of a structure called a vector. 00:09:52.120 --> 00:09:53.990 And we're going to have something which will reference 00:09:53.990 --> 00:09:58.710 a vector, just so we can write it down. 00:09:58.710 --> 00:10:02.240 Which takes the name of the vector, or the-- 00:10:02.240 --> 00:10:03.970 I don't think that name is the right word. 00:10:03.970 --> 00:10:12.010 Which takes the vector and the index, and I have to have a 00:10:12.010 --> 00:10:13.950 way of setting one of those with something called a vector 00:10:13.950 --> 00:10:16.280 set, I don't really care. 00:10:16.280 --> 00:10:19.520 But let's look, for example, at then that kind of 00:10:19.520 --> 00:10:26.470 implementation of car and cdr. 00:10:26.470 --> 00:10:31.470 So for example if I happen to have a register b, which 00:10:31.470 --> 00:10:37.580 contains the type index of a pair, and therefore it is the 00:10:37.580 --> 00:10:41.930 pointer to a pair, then I could take the car of that and 00:10:41.930 --> 00:10:42.760 if I-- write this down-- 00:10:42.760 --> 00:10:44.490 I might put that in register a. 00:10:44.490 --> 00:10:49.400 What that really is is a representation of the assign 00:10:49.400 --> 00:10:52.890 to a, the value of vector reffing-- 00:10:52.890 --> 00:10:54.700 or array indexing, if you will-- or 00:10:54.700 --> 00:10:58.490 something, the cars object-- 00:10:58.490 --> 00:10:59.990 whatever that is-- 00:10:59.990 --> 00:11:02.650 with the index, b. 00:11:02.650 --> 00:11:06.330 And similarly for cdr. And we can do the same thing for 00:11:06.330 --> 00:11:10.370 assignment to data structures, if we need to do that sort of 00:11:10.370 --> 00:11:11.840 thing at all. 00:11:11.840 --> 00:11:14.580 It's not too hard to build that. 00:11:14.580 --> 00:11:16.170 Well now the next question is how are we going to do 00:11:16.170 --> 00:11:18.010 allocation. 00:11:18.010 --> 00:11:21.550 And every so often I say I want a cons. 00:11:21.550 --> 00:11:23.790 Now conses don't grow on trees. 00:11:23.790 --> 00:11:25.340 Or maybe they should. 00:11:25.340 --> 00:11:29.980 But I have to have some way of getting the next one. 00:11:29.980 --> 00:11:33.920 I have to have some idea of if their memory is unused that I 00:11:33.920 --> 00:11:35.630 might want to allocate from. 00:11:35.630 --> 00:11:37.380 And there are many schemes for doing this. 00:11:37.380 --> 00:11:38.660 And the particular thing I'm showing you 00:11:38.660 --> 00:11:42.100 right now is not essential. 00:11:42.100 --> 00:11:44.960 However it's convenient and has been done many times. 00:11:44.960 --> 00:11:47.660 One scheme's was called the free list allocation scheme. 00:11:47.660 --> 00:11:50.570 What that means is that all of the free memory that there is 00:11:50.570 --> 00:11:54.700 in the world is linked together in a linked list, 00:11:54.700 --> 00:11:56.960 just like all the other stuff. 00:11:56.960 --> 00:12:01.230 And whenever you need a free cell to make a new cons, you 00:12:01.230 --> 00:12:04.440 grab the first, one make the free list be the cdr of it, 00:12:04.440 --> 00:12:06.030 and then allocate that. 00:12:06.030 --> 00:12:09.530 And so what that looks like is something like this. 00:12:09.530 --> 00:12:18.510 Here we have the free list starting in 6. 00:12:18.510 --> 00:12:24.860 And what that is is a pointer-off to say 8. 00:12:24.860 --> 00:12:27.020 So what it says is, this one is free and the 00:12:27.020 --> 00:12:28.870 next one is an 8. 00:12:28.870 --> 00:12:32.880 This one is free and the next one is in 3, the next one 00:12:32.880 --> 00:12:33.930 that's free. 00:12:33.930 --> 00:12:37.680 That one's free and the next one is in 0. 00:12:37.680 --> 00:12:40.940 That one's free and the next one's in 15. 00:12:40.940 --> 00:12:42.780 Something like that. 00:12:42.780 --> 00:12:46.400 We can imagine having such a structure. 00:12:46.400 --> 00:12:50.480 Given that we have something like that, then it's possible 00:12:50.480 --> 00:12:53.940 to just get one when you need it. 00:12:53.940 --> 00:12:57.960 And so a program for doing cons, this is what 00:12:57.960 --> 00:12:59.320 cons might turn into. 00:12:59.320 --> 00:13:05.410 To assign to a register A the result of cons-ing, a B onto 00:13:05.410 --> 00:13:08.250 C, the value in this containing B and the value 00:13:08.250 --> 00:13:11.240 containing C, what we have to do is get the current 00:13:11.240 --> 00:13:13.400 [? type ?] ahead of the freelist, make the free list 00:13:13.400 --> 00:13:19.840 be its cdr. Then we have to change the cars to be the 00:13:19.840 --> 00:13:25.680 thing we're making up to be in A to be the B, the thing in B. 00:13:25.680 --> 00:13:30.880 And we have to make change the cdrs of the thing that's in A 00:13:30.880 --> 00:13:36.020 to be C. And then what we have in A is the right new frob, 00:13:36.020 --> 00:13:36.650 whatever it is. 00:13:36.650 --> 00:13:40.470 The object that we want. 00:13:40.470 --> 00:13:43.490 Now there's a little bit of a cheat here that I haven't told 00:13:43.490 --> 00:13:47.155 you about, which is somewhere around here I haven't set that 00:13:47.155 --> 00:13:51.540 I've the type of the thing that I'm cons-ing up to be a 00:13:51.540 --> 00:13:53.510 pair, and I ought to. 00:13:53.510 --> 00:13:56.570 So there should be some sort of bits here are being set, 00:13:56.570 --> 00:13:59.810 and I just haven't written that down. 00:13:59.810 --> 00:14:01.480 We could have arranged it, of course, for the free lift to 00:14:01.480 --> 00:14:03.100 be made out of pairs. 00:14:03.100 --> 00:14:06.430 And so then there's no problem with that. 00:14:06.430 --> 00:14:10.160 But that sort of-- again, an inessential detail in a way 00:14:10.160 --> 00:14:13.500 some particular programmer or architect or whatever might 00:14:13.500 --> 00:14:17.540 manufacture his machine or Lisp system. 00:14:17.540 --> 00:14:23.930 So for example, just looking at this, to allocate given 00:14:23.930 --> 00:14:27.200 that I had already the structure that you saw before, 00:14:27.200 --> 00:14:31.900 supposing I wanted to allocate a new cell, which is going to 00:14:31.900 --> 00:14:38.680 be representation of list one, one, two, where already one 00:14:38.680 --> 00:14:43.430 two was the car of the list we were playing with before. 00:14:43.430 --> 00:14:44.780 Well that's not so hard. 00:14:44.780 --> 00:14:47.670 I stored that one and one, so p1 one is the 00:14:47.670 --> 00:14:49.530 representation of this. 00:14:49.530 --> 00:14:51.690 This is p5. 00:14:51.690 --> 00:14:54.070 That's going to be the cdr of this. 00:14:54.070 --> 00:14:55.610 Now we're going to pull something off the free list, 00:14:55.610 --> 00:14:57.780 but remember the free list started at six. 00:14:57.780 --> 00:15:01.540 The new free list after this allocation is eight, a free 00:15:01.540 --> 00:15:02.890 list beginning at eight. 00:15:02.890 --> 00:15:06.360 And of course in six now we have a number one, which is 00:15:06.360 --> 00:15:10.540 what we wanted, with its cdr being the pair starting in 00:15:10.540 --> 00:15:13.330 location five. 00:15:13.330 --> 00:15:16.810 And that's no big deal. 00:15:16.810 --> 00:15:21.480 So the only problem really remaining here is, well, I 00:15:21.480 --> 00:15:25.080 don't have an infinitely large memory. 00:15:25.080 --> 00:15:28.070 If I do this for a little while, say, for example, 00:15:28.070 --> 00:15:30.745 supposing it takes me a microsecond to do a cons, and 00:15:30.745 --> 00:15:34.570 I have a million cons memory then I'm only going to run out 00:15:34.570 --> 00:15:38.000 in a second, and that's pretty bad. 00:15:38.000 --> 00:15:41.470 So what we do to prevent that disaster, that ecological 00:15:41.470 --> 00:15:44.300 disaster, talk about right after questions. 00:15:44.300 --> 00:15:45.550 Are there any questions? 00:15:51.500 --> 00:15:52.030 Yes. 00:15:52.030 --> 00:15:54.830 AUDIENCE: In the environment diagrams that we were drawing 00:15:54.830 --> 00:15:58.630 we would use the body of procedures, and you would 00:15:58.630 --> 00:16:02.620 eventually wind up with things that were no longer useful in 00:16:02.620 --> 00:16:04.930 that structure. 00:16:04.930 --> 00:16:06.890 How is that represented? 00:16:06.890 --> 00:16:09.180 PROFESSOR: There's two problems here. 00:16:09.180 --> 00:16:13.870 One you were asking is that material becomes useless. 00:16:13.870 --> 00:16:14.920 We'll talk about that in a second. 00:16:14.920 --> 00:16:18.100 That has to do with how to prevent ecological disasters. 00:16:18.100 --> 00:16:20.190 If I make a lot of garbage I have to somehow be able to 00:16:20.190 --> 00:16:21.820 clean up after myself. 00:16:21.820 --> 00:16:23.430 And we'll talk about that in a second. 00:16:23.430 --> 00:16:25.370 The other question you're asking is how you represent 00:16:25.370 --> 00:16:27.210 the environments, I think. 00:16:27.210 --> 00:16:27.600 AUDIENCE: Yes. 00:16:27.600 --> 00:16:28.190 PROFESSOR: OK. 00:16:28.190 --> 00:16:29.780 And the environment structures can be represented in 00:16:29.780 --> 00:16:30.860 arbitrary ways. 00:16:30.860 --> 00:16:31.780 There are lots of them. 00:16:31.780 --> 00:16:33.630 I mean, here I'm just telling you about list cells. 00:16:33.630 --> 00:16:36.400 Of course every real system has vectors of arbitrary 00:16:36.400 --> 00:16:39.500 length as well as the vectors of length, too, which 00:16:39.500 --> 00:16:41.080 represent list cells. 00:16:41.080 --> 00:16:45.460 And the environment structures that one uses in a 00:16:45.460 --> 00:16:49.890 professionally written Lisp system tend to be vectors 00:16:49.890 --> 00:16:52.350 which contain a number of elements approximately equal 00:16:52.350 --> 00:16:56.090 to the number of arguments-- a little bit more because you 00:16:56.090 --> 00:16:58.290 need certain glue. 00:16:58.290 --> 00:17:00.360 So remember, the environment [UNINTELLIGIBLE] 00:17:00.360 --> 00:17:00.740 frames. 00:17:00.740 --> 00:17:03.980 The frames are constructed by applying a procedure. 00:17:03.980 --> 00:17:08.849 In doing so, an allocation is made of a place which is the 00:17:08.849 --> 00:17:11.270 number of arguments long plus [? unglue ?] 00:17:11.270 --> 00:17:13.859 that gets linked into a chain. 00:17:13.859 --> 00:17:15.660 It's just like algol at that level. 00:17:19.810 --> 00:17:21.060 There any other questions? 00:17:23.700 --> 00:17:23.920 OK. 00:17:23.920 --> 00:17:26.106 Thank you, and let's take a short break. 00:17:26.106 --> 00:17:26.449 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 00:17:26.449 --> 00:17:27.699 JOHANN SEBASTIAN BACH] 00:18:12.270 --> 00:18:15.840 PROFESSOR: Well, as I just said, computer memories 00:18:15.840 --> 00:18:19.420 supplied by the semiconductor manufacturers are finite. 00:18:19.420 --> 00:18:21.620 And that's quite a pity. 00:18:21.620 --> 00:18:24.030 It might not always be that way. 00:18:24.030 --> 00:18:27.990 Just for a quick calculation, you can see that it's possible 00:18:27.990 --> 00:18:28.860 that if [? memory ?] 00:18:28.860 --> 00:18:32.130 prices keep going at the rate they're going that if you 00:18:32.130 --> 00:18:34.950 still took a microsecond second to do a cons, then-- 00:18:34.950 --> 00:18:37.120 first of all, everybody should know that there's about pi 00:18:37.120 --> 00:18:39.450 times ten to the seventh seconds in a year. 00:18:39.450 --> 00:18:42.640 And so that would be ten to the seventh plus ten to the 00:18:42.640 --> 00:18:43.940 sixth is ten to the thirteenth. 00:18:43.940 --> 00:18:45.870 So there's maybe ten to the fourteenth conses in the life 00:18:45.870 --> 00:18:47.520 of a machine. 00:18:47.520 --> 00:18:49.900 If there was ten to the fourteenth words of memory on 00:18:49.900 --> 00:18:54.020 your machine, you'd never run out. 00:18:54.020 --> 00:18:56.310 And that's not completely unreasonable. 00:18:56.310 --> 00:18:58.460 Ten to the fourteenth is not a very large number. 00:19:03.860 --> 00:19:05.180 I don't think it is. 00:19:05.180 --> 00:19:08.700 But then again I like to play with astronomy. 00:19:08.700 --> 00:19:11.380 It's at least ten to the eighteenth centimeters between 00:19:11.380 --> 00:19:12.930 us and the nearest star. 00:19:12.930 --> 00:19:19.620 But the thing I'm about to worry about is, at least in 00:19:19.620 --> 00:19:22.130 the current economic state of affairs, ten to the fourteenth 00:19:22.130 --> 00:19:24.200 pieces of memory is expensive. 00:19:24.200 --> 00:19:27.280 And so I suppose what we have to do is make 00:19:27.280 --> 00:19:28.120 do with much smaller. 00:19:28.120 --> 00:19:30.170 Memories 00:19:30.170 --> 00:19:35.800 Now in general we want to have an illusion of infinity. 00:19:35.800 --> 00:19:38.850 All we need to do is arrange it so that whenever you look, 00:19:38.850 --> 00:19:40.100 the thing is there. 00:19:42.670 --> 00:19:45.105 That's really an important idea. 00:19:49.540 --> 00:19:52.470 A person or a computer lives only a finite amount of time 00:19:52.470 --> 00:19:55.280 and can only take a finite number of looks at something. 00:19:55.280 --> 00:19:58.190 And so you really only need a finite amount of stuff. 00:19:58.190 --> 00:20:01.730 But you have to arrange it so no matter how much there is, 00:20:01.730 --> 00:20:04.000 how much you really claim there is, there's always 00:20:04.000 --> 00:20:06.900 enough stuff so that when you take a look, it's there. 00:20:06.900 --> 00:20:08.750 And so you only need a finite amount. 00:20:08.750 --> 00:20:11.630 But let's see. 00:20:11.630 --> 00:20:14.980 One problem is, as was brought up, that there are possible 00:20:14.980 --> 00:20:18.660 ways that there is lots of stuff that we make that we 00:20:18.660 --> 00:20:19.410 don't need. 00:20:19.410 --> 00:20:20.895 And we could recycle the material out 00:20:20.895 --> 00:20:22.760 of which its made. 00:20:22.760 --> 00:20:27.820 An example is the fact that we're building environment 00:20:27.820 --> 00:20:30.470 structures, and we do so every time we call a procedure. 00:20:30.470 --> 00:20:32.810 We have built in it a environment frame. 00:20:32.810 --> 00:20:34.840 That environment frame doesn't necessarily 00:20:34.840 --> 00:20:36.730 have a very long lifetime. 00:20:36.730 --> 00:20:40.330 Its lifetime, meaning its usefulness, may exist only 00:20:40.330 --> 00:20:42.850 over the invocation of the procedure. 00:20:42.850 --> 00:20:45.860 Or if the procedure exports another procedure by returning 00:20:45.860 --> 00:20:48.260 it as a value and that procedure is defined inside of 00:20:48.260 --> 00:20:52.210 it, well then the lifetime of the frame of the outer 00:20:52.210 --> 00:20:57.070 procedure still is only the lifetime of the procedure 00:20:57.070 --> 00:20:58.530 which was exported. 00:20:58.530 --> 00:21:01.960 And so ultimately, a lot of that is garbage. 00:21:01.960 --> 00:21:05.370 There are other ways of producing garbage as well. 00:21:05.370 --> 00:21:07.240 Users produce garbage. 00:21:07.240 --> 00:21:10.930 An example of user garbage is something like this. 00:21:10.930 --> 00:21:15.220 If we write a program to, for example, append two lists 00:21:15.220 --> 00:21:19.890 together, well one way to do it is to reverse the first 00:21:19.890 --> 00:21:23.440 list onto the empty list and reverse that onto the second 00:21:23.440 --> 00:21:28.160 list. Now that's not terribly bad way of doing it. 00:21:28.160 --> 00:21:31.070 And however, the intermediate result, which is the reversal 00:21:31.070 --> 00:21:37.300 of the first list as done by this program, is never going 00:21:37.300 --> 00:21:39.920 to be accessed ever again after it's copied back on to 00:21:39.920 --> 00:21:41.010 the second. 00:21:41.010 --> 00:21:43.580 It's an intermediate result. 00:21:43.580 --> 00:21:47.230 It's going to be hard to ever see how anybody would ever be 00:21:47.230 --> 00:21:48.600 able to access it. 00:21:48.600 --> 00:21:51.050 In fact, it will go away. 00:21:51.050 --> 00:21:53.190 Now if we make a lot of garbage like that, and we 00:21:53.190 --> 00:21:56.210 should be allowed to, then there's got to be some way to 00:21:56.210 --> 00:21:58.800 reclaim that garbage. 00:21:58.800 --> 00:22:03.050 Well, what I'd like to tell you about now is a very clever 00:22:03.050 --> 00:22:09.820 technique whereby a Lisp system can prove a small 00:22:09.820 --> 00:22:12.750 theorem every so often on the [? forum, ?] the following 00:22:12.750 --> 00:22:17.410 piece of junk will never be accessed again. 00:22:17.410 --> 00:22:21.400 It can have no affect on the future of the computation. 00:22:21.400 --> 00:22:24.920 It's actually based on a very simple idea. 00:22:24.920 --> 00:22:28.570 We've designed our computers to look sort of like this. 00:22:28.570 --> 00:22:35.280 There's some data path, which contains the registers. 00:22:35.280 --> 00:22:42.610 There are things like x, and env, and val, and so on. 00:22:42.610 --> 00:22:47.490 And there's one here called stack, some sort which points 00:22:47.490 --> 00:22:50.240 off to a structure somewhere, which is the stack. 00:22:50.240 --> 00:22:51.740 And we'll worry about that in a second. 00:22:51.740 --> 00:22:55.390 There's some finite controller, finite state 00:22:55.390 --> 00:22:56.730 machine controller. 00:22:56.730 --> 00:22:59.850 And there's some control signals that go this way and 00:22:59.850 --> 00:23:02.270 predicate results that come this way, not 00:23:02.270 --> 00:23:04.260 the interesting part. 00:23:04.260 --> 00:23:07.140 There's some sort of structured memory, which I 00:23:07.140 --> 00:23:10.460 just told you how to make, which may contain a stack. 00:23:10.460 --> 00:23:12.690 I didn't tell you how to make things of arbitrary shape, 00:23:12.690 --> 00:23:13.450 only pairs. 00:23:13.450 --> 00:23:16.280 But in fact with what I've told you can simulate a stack 00:23:16.280 --> 00:23:19.140 by a big list. I don't plan to do that, it's not a 00:23:19.140 --> 00:23:20.360 nice way to do it. 00:23:20.360 --> 00:23:22.990 But we could have something like that. 00:23:22.990 --> 00:23:25.990 We have all sorts of little data structures in here that 00:23:25.990 --> 00:23:27.470 are hooked together in funny ways. 00:23:30.115 --> 00:23:32.560 They connect to other things. 00:23:32.560 --> 00:23:33.250 And so on. 00:23:33.250 --> 00:23:37.190 And ultimately things up there are pointers to these. 00:23:37.190 --> 00:23:40.730 The things that are in the registers are pointers off to 00:23:40.730 --> 00:23:44.910 the data structures that live in this Lisp structure memory. 00:23:44.910 --> 00:23:52.660 Now the truth of the matter is that the entire consciousness 00:23:52.660 --> 00:23:55.550 of this machine is in these registers. 00:23:55.550 --> 00:23:59.380 There is no possible way that the machine, if done 00:23:59.380 --> 00:24:02.410 correctly, if built correctly, can access anything in this 00:24:02.410 --> 00:24:05.580 Lisp structure memory unless the thing in that Lisp 00:24:05.580 --> 00:24:10.310 structure memory is connected by a sequence of data 00:24:10.310 --> 00:24:15.070 structures to the registers. 00:24:15.070 --> 00:24:17.550 If it's accessible by legitimate data structure 00:24:17.550 --> 00:24:20.100 selectors from the pointers that are 00:24:20.100 --> 00:24:22.280 stored in these registers. 00:24:22.280 --> 00:24:24.940 Things like array references, perhaps. 00:24:24.940 --> 00:24:28.790 Or cons cell references, cars and cdrs. 00:24:28.790 --> 00:24:30.970 But I can't just talk about a random place in this memory, 00:24:30.970 --> 00:24:32.740 because I can't get to it. 00:24:32.740 --> 00:24:34.665 These are being arbitrary names I'm not allowed to 00:24:34.665 --> 00:24:38.985 count, at least as I'm evaluating expressions. 00:24:41.620 --> 00:24:44.600 If that's the case then there's a very simple theorem 00:24:44.600 --> 00:24:47.160 to be proved. 00:24:47.160 --> 00:24:49.730 Which is, if I start with all lead pointers that are in all 00:24:49.730 --> 00:24:53.570 these registers and recursively chase out, marking 00:24:53.570 --> 00:24:57.980 all the places I can get to by selectors, then eventually I 00:24:57.980 --> 00:25:00.750 mark everything they can be gotten to. 00:25:00.750 --> 00:25:02.140 Anything which is not so marked is 00:25:02.140 --> 00:25:05.560 garbage and can be recycled. 00:25:05.560 --> 00:25:07.200 Very simple. 00:25:07.200 --> 00:25:11.180 Cannot affect the future of the computation. 00:25:11.180 --> 00:25:16.616 So let me show you that in a particular example. 00:25:16.616 --> 00:25:19.800 Now that means I'm going to have to append to my 00:25:19.800 --> 00:25:23.640 description of the list structure a mark. 00:25:23.640 --> 00:25:29.080 And so here, for example, is a Lisp structured memory. 00:25:29.080 --> 00:25:31.360 And in this Lisp structured memory is a Lisp structure 00:25:31.360 --> 00:25:33.030 beginning in a place I'm going to call-- 00:25:35.640 --> 00:25:38.590 this is the root. 00:25:38.590 --> 00:25:40.120 Now it doesn't really have to have a root. 00:25:40.120 --> 00:25:42.670 It could be a bunch of them, like all the registers. 00:25:42.670 --> 00:25:45.270 But I could cleverly arrange it so all the registers, all 00:25:45.270 --> 00:25:47.080 the things that are in old registers are also at the 00:25:47.080 --> 00:25:50.770 right moment put into this root structure, and then we've 00:25:50.770 --> 00:25:51.850 got one pointer to it. 00:25:51.850 --> 00:25:54.570 I don't really care. 00:25:54.570 --> 00:25:57.290 So the idea is we're going to cons up stuff until our free 00:25:57.290 --> 00:25:58.720 list is empty. 00:25:58.720 --> 00:26:00.950 We've run out of things. 00:26:00.950 --> 00:26:04.560 Now we're going to do this process of proving the theorem 00:26:04.560 --> 00:26:07.850 that a certain percentage of the memory has got crap in it. 00:26:07.850 --> 00:26:10.320 And then we're going to recycle that to grow new 00:26:10.320 --> 00:26:14.570 trees, a standard use of such garbage. 00:26:17.090 --> 00:26:18.840 So in any case, what do we have here? 00:26:18.840 --> 00:26:21.700 Well we have some data structure which starts out 00:26:21.700 --> 00:26:27.502 over here one. 00:26:27.502 --> 00:26:33.980 And in fact it has a car in five, and its cdr is in two. 00:26:33.980 --> 00:26:36.700 And all the marks start out at zero. 00:26:36.700 --> 00:26:39.920 Well let's start marking, just to play this game. 00:26:39.920 --> 00:26:42.540 OK. 00:26:42.540 --> 00:26:47.090 So for example, since I can access one from the root I 00:26:47.090 --> 00:26:48.390 will mark that. 00:26:48.390 --> 00:26:50.960 Let me mark it. 00:26:50.960 --> 00:26:52.430 Bang. 00:26:52.430 --> 00:26:54.560 That's marked. 00:26:54.560 --> 00:27:00.020 Now since I have a five here I can go to five and see, well 00:27:00.020 --> 00:27:01.450 I'll mark that. 00:27:01.450 --> 00:27:01.760 Bang. 00:27:01.760 --> 00:27:02.900 That's useful stuff. 00:27:02.900 --> 00:27:05.410 But five references as a number in its car, I'm not 00:27:05.410 --> 00:27:08.700 interested in marking numbers but its cdr is seven. 00:27:08.700 --> 00:27:10.450 So I can mark that. 00:27:10.450 --> 00:27:12.260 Bang. 00:27:12.260 --> 00:27:16.000 Seven is the empty list, the only thing that references, 00:27:16.000 --> 00:27:17.120 and it's got a number in its car. 00:27:17.120 --> 00:27:19.490 Not interesting. 00:27:19.490 --> 00:27:20.500 Well now let's go back here. 00:27:20.500 --> 00:27:21.650 I forgot about something. 00:27:21.650 --> 00:27:22.840 Two. 00:27:22.840 --> 00:27:25.960 See in other words, if I'm looking at cell one, cell one 00:27:25.960 --> 00:27:30.370 contains a two right over here. 00:27:30.370 --> 00:27:31.730 A reference to two. 00:27:31.730 --> 00:27:35.700 That means I should go mark two. 00:27:35.700 --> 00:27:37.140 Bang. 00:27:37.140 --> 00:27:38.960 Two contains a reference to four. 00:27:38.960 --> 00:27:41.593 It's got a number in its car, I'm not interested in that, so 00:27:41.593 --> 00:27:43.780 I'm going to go mark that. 00:27:43.780 --> 00:27:47.840 Four refers to seven through its car, and is empty in its 00:27:47.840 --> 00:27:49.990 cdr, but I've already marked that one so I don't have to 00:27:49.990 --> 00:27:51.400 mark it again. 00:27:51.400 --> 00:27:55.000 This is all the accessible structure from that place. 00:27:55.000 --> 00:27:58.710 Simple recursive mark algorithm. 00:27:58.710 --> 00:28:01.160 Now there are some unhappinesses about that 00:28:01.160 --> 00:28:04.920 algorithm, and we can worry about that a second. 00:28:04.920 --> 00:28:07.280 But basically you'll see that all the things that have not 00:28:07.280 --> 00:28:14.220 been marked are places that are free, and I could recycle. 00:28:14.220 --> 00:28:16.210 So the next stage after that is going to be to scan through 00:28:16.210 --> 00:28:21.180 all of my memory, looking for things that are not marked. 00:28:21.180 --> 00:28:23.370 Every time I come across a marked thing I unmark it, and 00:28:23.370 --> 00:28:26.390 every time I come across an unmarked thing I'm going to 00:28:26.390 --> 00:28:28.770 link it together in my free list. 00:28:28.770 --> 00:28:32.120 Classic, very simple algorithm. 00:28:32.120 --> 00:28:33.840 So let's see. 00:28:33.840 --> 00:28:34.770 Is that very simple? 00:28:34.770 --> 00:28:35.570 Yes it is. 00:28:35.570 --> 00:28:38.340 I'm not going to go through the code in any detail, but I 00:28:38.340 --> 00:28:40.090 just want to show you about how long it is. 00:28:40.090 --> 00:28:42.490 Let's look at the mark phase. 00:28:42.490 --> 00:28:45.060 Here's the first part of the mark phase. 00:28:45.060 --> 00:28:48.280 We pick up the root. 00:28:48.280 --> 00:28:52.380 We're going to use that as a recursive procedure call. 00:28:52.380 --> 00:28:55.800 We're going to sweep from there, after when we're done 00:28:55.800 --> 00:28:57.380 with marking. 00:28:57.380 --> 00:28:59.840 And then we're going to do a little couple of instructions 00:28:59.840 --> 00:29:01.920 that do this checking out on the marks and changing the 00:29:01.920 --> 00:29:04.000 marks and things like that, according to the algorithm 00:29:04.000 --> 00:29:05.500 I've just shown you. 00:29:05.500 --> 00:29:06.470 It comes out here. 00:29:06.470 --> 00:29:08.800 You have to mark the cars of things and you also have to be 00:29:08.800 --> 00:29:10.660 able to mark the cdrs of things. 00:29:10.660 --> 00:29:14.370 That's the entire mark phase. 00:29:14.370 --> 00:29:16.590 I'll just tell you a little story about this. 00:29:16.590 --> 00:29:22.950 The old DEC PDP-6 computer, this was the way that the 00:29:22.950 --> 00:29:26.740 mark-sweep garbage collection, as it was, was written. 00:29:26.740 --> 00:29:31.940 The program was so small that with the data that it needed, 00:29:31.940 --> 00:29:34.310 with the registers that it needed to manipulate the 00:29:34.310 --> 00:29:38.070 memory, it fit into the fast registers of the machine, 00:29:38.070 --> 00:29:39.280 which were 16. 00:29:39.280 --> 00:29:39.800 The whole program. 00:29:39.800 --> 00:29:40.700 And you could execute 00:29:40.700 --> 00:29:43.170 instructions in the fast registers. 00:29:43.170 --> 00:29:46.560 So it's an extremely small program, and it could run very 00:29:46.560 --> 00:29:48.870 fast. 00:29:48.870 --> 00:29:53.130 Now unfortunately, of course, this program, because the fact 00:29:53.130 --> 00:29:57.630 that it's recursive in the way that you do something first 00:29:57.630 --> 00:29:59.690 and then you do something after that, you have to work 00:29:59.690 --> 00:30:03.410 on the cars and then the cdrs, it requires auxiliary memory. 00:30:03.410 --> 00:30:05.680 So Lisp systems-- 00:30:05.680 --> 00:30:08.260 those requires a stack for marking. 00:30:08.260 --> 00:30:12.440 Lisp systems that are built this way have a limit to the 00:30:12.440 --> 00:30:16.060 depth of recursion you can have in data structures in 00:30:16.060 --> 00:30:19.930 either the car or the cdr, and that doesn't work very nicely. 00:30:19.930 --> 00:30:23.180 On the other hand, you never notice it if it's big enough. 00:30:23.180 --> 00:30:27.650 And that's certainly been the case for most Maclisp, for 00:30:27.650 --> 00:30:30.760 example, which ran Macsyma where you could deal with 00:30:30.760 --> 00:30:33.560 expressions of thousands of elements long. 00:30:33.560 --> 00:30:35.490 These are algebraic expressions with thousand of 00:30:35.490 --> 00:30:39.490 terms. And there's no problem with that. 00:30:39.490 --> 00:30:42.190 Such, the garbage collector does work. 00:30:42.190 --> 00:30:44.750 On the other hand, there's a very clever modification to 00:30:44.750 --> 00:30:49.020 this algorithm, which I will not describe, by Peter Deutsch 00:30:49.020 --> 00:30:50.720 and Schorr and Waite-- 00:30:50.720 --> 00:30:55.380 Herb Schorr from IBM and Waite, who I don't know. 00:30:55.380 --> 00:30:58.470 That algorithm allows you to build-- you do can do this 00:30:58.470 --> 00:31:01.990 without auxiliary memory, by remembering as you walk the 00:31:01.990 --> 00:31:04.760 data structures where you came from by reversing the pointers 00:31:04.760 --> 00:31:06.650 as you go down and crawling up the reverse 00:31:06.650 --> 00:31:07.520 pointers as you go up. 00:31:07.520 --> 00:31:09.130 It's a rather tricky algorithm. 00:31:09.130 --> 00:31:11.230 The first time you write it-- or in fact, the first three 00:31:11.230 --> 00:31:14.350 times you write it it has a terrible bug in it. 00:31:14.350 --> 00:31:18.110 And it's also rather slow, because it's complicated. 00:31:18.110 --> 00:31:21.150 It takes about six times as many memory references to do 00:31:21.150 --> 00:31:24.580 the sorts of things that we're talking about. 00:31:24.580 --> 00:31:28.140 Well now once I've done this marking phase, and I get into 00:31:28.140 --> 00:31:30.920 a position where things look like this, let's look-- 00:31:30.920 --> 00:31:31.510 yes. 00:31:31.510 --> 00:31:35.590 Here we have the mark done, just as I did it. 00:31:35.590 --> 00:31:37.330 Now we have to perform the sweep phase. 00:31:37.330 --> 00:31:39.820 And I described to you what this sweep is like. 00:31:39.820 --> 00:31:42.130 I'm going to walk down from one end of memory or the 00:31:42.130 --> 00:31:45.690 other, I don't care where, scanning every cell that's in 00:31:45.690 --> 00:31:46.836 the memory. 00:31:46.836 --> 00:31:51.000 And as I scan these cells, I'm going to link them together, 00:31:51.000 --> 00:31:53.890 if they are free, into the free list. And if they're not 00:31:53.890 --> 00:31:57.500 free, I'm going to unmark them so the marks become zero. 00:31:57.500 --> 00:32:00.050 And in fact what I get-- well the program is not very 00:32:00.050 --> 00:32:00.460 complicated. 00:32:00.460 --> 00:32:02.780 It looks sort of like this-- it's a little longer. 00:32:02.780 --> 00:32:04.820 Here's the first piece of it. 00:32:04.820 --> 00:32:06.710 This one's coming down from the top of memory. 00:32:06.710 --> 00:32:09.580 I don't want you to try to understand this at this point. 00:32:09.580 --> 00:32:11.030 It's rather simple. 00:32:11.030 --> 00:32:14.260 It's a very simple algorithm, but there's pieces of it that 00:32:14.260 --> 00:32:15.970 just sort of look like this. 00:32:15.970 --> 00:32:18.600 They're all sort of obvious. 00:32:18.600 --> 00:32:21.060 And after we've done the sweep, we get an answer that 00:32:21.060 --> 00:32:22.310 looks like that. 00:32:25.330 --> 00:32:27.150 Now there are some disadvantages with mark-sweep 00:32:27.150 --> 00:32:29.590 algorithms of this sort. 00:32:29.590 --> 00:32:31.940 Serious ones. 00:32:31.940 --> 00:32:34.250 One important disadvantage is that your memories get larger 00:32:34.250 --> 00:32:36.498 and larger. 00:32:36.498 --> 00:32:39.100 As you say, address spaces get larger and larger, you're 00:32:39.100 --> 00:32:43.080 willing to represent more and more stuff, then it gets very 00:32:43.080 --> 00:32:46.360 costly to scan all of memory. 00:32:46.360 --> 00:32:50.490 What you'd really like to do is only scan useful stuff. 00:32:50.490 --> 00:32:56.120 It would even be better if you realized that some stuff was 00:32:56.120 --> 00:32:59.320 known to be good and useful, and you don't have to look at 00:32:59.320 --> 00:33:00.370 it more than once or twice. 00:33:00.370 --> 00:33:01.550 Or very rarely. 00:33:01.550 --> 00:33:05.120 Whereas other stuff that you're not so sure about, you 00:33:05.120 --> 00:33:10.110 can look at more detail every time you want to do this, want 00:33:10.110 --> 00:33:11.910 to garbage collect. 00:33:11.910 --> 00:33:15.660 Well there are algorithms that are organized in this way. 00:33:15.660 --> 00:33:18.850 Let me tell you about a famous old algorithm which allows you 00:33:18.850 --> 00:33:20.370 only look at the part of memory which 00:33:20.370 --> 00:33:22.800 is known to be useful. 00:33:22.800 --> 00:33:24.690 And which happens to be the fastest known garbage 00:33:24.690 --> 00:33:26.310 collector algorithm. 00:33:26.310 --> 00:33:28.170 This is the Minsky-Feinchel-Yochelson 00:33:28.170 --> 00:33:30.150 garbage collector algorithm. 00:33:30.150 --> 00:33:36.660 It was invented by Minsky in 1961 or '60 or something, for 00:33:36.660 --> 00:33:45.870 the RLE PDP-1 Lisp, which had 4,096 words of list memory, 00:33:45.870 --> 00:33:48.480 and a drum. 00:33:48.480 --> 00:33:50.890 And the whole idea was to garbage collect 00:33:50.890 --> 00:33:53.380 this terrible memory. 00:33:53.380 --> 00:33:56.510 What Minsky realized was the easiest way to do this is to 00:33:56.510 --> 00:33:59.950 scan the memory in the same sense, walking the good 00:33:59.950 --> 00:34:06.350 structure, copying it out into the drum, compacted. 00:34:06.350 --> 00:34:09.429 And then when we were done copying it all out, then you 00:34:09.429 --> 00:34:12.300 swap that back into your memory. 00:34:12.300 --> 00:34:14.260 Now whether or you not use a drum, or another piece of 00:34:14.260 --> 00:34:17.030 memory, or something like that isn't important. 00:34:17.030 --> 00:34:18.260 In fact, I don't think people use 00:34:18.260 --> 00:34:20.350 drums anymore for anything. 00:34:20.350 --> 00:34:25.920 But this algorithm basically depends upon having about 00:34:25.920 --> 00:34:30.270 twice as much address space as you're actually using. 00:34:30.270 --> 00:34:35.370 And so what you have is some, initially, some mixture of 00:34:35.370 --> 00:34:37.110 useful data and garbage. 00:34:37.110 --> 00:34:38.560 So this is called fromspace. 00:34:45.179 --> 00:34:47.800 And this is a mixture of crud. 00:34:47.800 --> 00:34:52.000 Some of it's important and some of it isn't. 00:34:52.000 --> 00:34:55.770 Now there's another place which is hopefully big enough, 00:34:55.770 --> 00:34:58.240 if we recall, tospace, which is where we're copying to. 00:35:01.590 --> 00:35:03.260 And what happens is-- and I'm not going to go 00:35:03.260 --> 00:35:04.970 through this detail. 00:35:04.970 --> 00:35:07.590 It's in our book quite explicitly. 00:35:07.590 --> 00:35:11.030 There's a root point where you start from. 00:35:11.030 --> 00:35:14.600 And the idea is that you start with the root. 00:35:14.600 --> 00:35:18.610 You copy the first thing you see, the first thing that the 00:35:18.610 --> 00:35:22.810 root points at, to the beginning of tospace. 00:35:22.810 --> 00:35:24.790 The first thing is a pair or something 00:35:24.790 --> 00:35:27.560 like, a data structure. 00:35:27.560 --> 00:35:32.330 You then also leave behind a broken heart saying, I moved 00:35:32.330 --> 00:35:36.330 this object from here to here, giving the place 00:35:36.330 --> 00:35:37.800 where it moved to. 00:35:37.800 --> 00:35:40.270 This is called a broken heart because a friend of mine who 00:35:40.270 --> 00:35:44.760 implemented one of these in 1966 was a very romantic 00:35:44.760 --> 00:35:46.760 character and called it a broken heart. 00:35:49.580 --> 00:35:53.570 But in any case, the next thing you do is now you have a 00:35:53.570 --> 00:35:57.840 new free pointer which is here, and you start scanning. 00:35:57.840 --> 00:36:00.235 You scan this data structure you just copied. 00:36:00.235 --> 00:36:02.710 And every time you encounter a pointer in it, you treat it as 00:36:02.710 --> 00:36:04.000 if it was the root pointer here. 00:36:04.000 --> 00:36:05.170 Oh, I'm sorry. 00:36:05.170 --> 00:36:06.330 The other thing you do is you now move the 00:36:06.330 --> 00:36:09.220 root pointer to there. 00:36:09.220 --> 00:36:11.310 So now you scan this, and everything you see you treat 00:36:11.310 --> 00:36:14.110 as it were the root pointer. 00:36:14.110 --> 00:36:16.360 So if you see something, well it points 00:36:16.360 --> 00:36:18.510 up into there somewhere. 00:36:18.510 --> 00:36:21.780 Is it pointing at a thing which you've not copied yet? 00:36:21.780 --> 00:36:23.880 Is there a broken heart there? 00:36:23.880 --> 00:36:25.370 If there's a broken heart there and it's something you 00:36:25.370 --> 00:36:27.640 have copied, you've just replaced this pointer with the 00:36:27.640 --> 00:36:30.620 thing a broken heart points at. 00:36:30.620 --> 00:36:33.030 If this thing has not been copied, you copy it to the 00:36:33.030 --> 00:36:34.430 next place over here. 00:36:34.430 --> 00:36:39.860 Move your free pointer over here, and then leave a broken 00:36:39.860 --> 00:36:43.670 heart behind and scan. 00:36:43.670 --> 00:36:46.840 And eventually when the scant pointer hits the free pointer, 00:36:46.840 --> 00:36:50.140 everything in memory has been copied. 00:36:50.140 --> 00:36:52.170 And then there's a whole bunch of empty space up here, which 00:36:52.170 --> 00:36:53.820 you could either make into a free list, if that's what you 00:36:53.820 --> 00:36:54.470 want to do. 00:36:54.470 --> 00:36:56.270 But generally you don't in this kind of system. 00:36:56.270 --> 00:36:57.470 In this system you sequentially 00:36:57.470 --> 00:37:00.910 allocate your memory. 00:37:00.910 --> 00:37:03.820 That is a very, very nice algorithm, and sort of the one 00:37:03.820 --> 00:37:06.790 we use in the scheme that you've been using. 00:37:06.790 --> 00:37:09.490 And it's expected-- 00:37:09.490 --> 00:37:12.400 I believe no one has found a faster algorithm than that. 00:37:12.400 --> 00:37:14.150 There are very simple modifications to this 00:37:14.150 --> 00:37:19.060 algorithm invented by Henry Baker which allow one to run 00:37:19.060 --> 00:37:21.210 this algorithm in real time, meaning you don't have to stop 00:37:21.210 --> 00:37:22.010 to garbage collect. 00:37:22.010 --> 00:37:25.410 But you could interleave the consing that the machine does 00:37:25.410 --> 00:37:27.870 when its running with steps of the garbage collection 00:37:27.870 --> 00:37:31.370 process, so that the garbage collector's distributed, and 00:37:31.370 --> 00:37:32.980 the machine doesn't have to stop, and garbage 00:37:32.980 --> 00:37:34.640 collecting can start. 00:37:34.640 --> 00:37:37.520 Of course in the case of machines with virtual memory 00:37:37.520 --> 00:37:41.760 where a lot of it is in inaccessible places, this 00:37:41.760 --> 00:37:44.460 becomes a very expensive process. 00:37:44.460 --> 00:37:47.690 And there have been numerous attempts to 00:37:47.690 --> 00:37:49.190 make this much better. 00:37:49.190 --> 00:37:52.080 There is a nice paper, for those of you who are 00:37:52.080 --> 00:37:56.210 interested, by Moon and other people which describes a 00:37:56.210 --> 00:37:57.940 modification to the incremental 00:37:57.940 --> 00:38:00.290 Minsky-Feinchel-Yochelson algorithm, and modification 00:38:00.290 --> 00:38:05.980 the Baker algorithm which is more efficient for virtual 00:38:05.980 --> 00:38:08.340 memory systems. 00:38:08.340 --> 00:38:12.840 Well I think now the mystery to this is sort of gone. 00:38:12.840 --> 00:38:14.090 And I'd like to see if there are any questions. 00:38:19.780 --> 00:38:20.810 Yes. 00:38:20.810 --> 00:38:24.100 AUDIENCE: I saw one of you run the garbage collector on the 00:38:24.100 --> 00:38:27.640 systems upstairs, and it seemed to me to run extremely 00:38:27.640 --> 00:38:30.190 fast. Did the whole thing take-- 00:38:30.190 --> 00:38:31.880 does it sweep through all of memory? 00:38:31.880 --> 00:38:32.510 PROFESSOR: No. 00:38:32.510 --> 00:38:34.480 It swept through exactly what was needed to 00:38:34.480 --> 00:38:37.320 copy the useful structure. 00:38:37.320 --> 00:38:40.030 It's a copying collector. 00:38:40.030 --> 00:38:45.090 And it is very fast. On the whole, I suppose to copy-- 00:38:45.090 --> 00:38:47.000 in a Bobcat-- 00:38:47.000 --> 00:38:52.450 to copy, I think, a three megabyte thing or something is 00:38:52.450 --> 00:38:56.800 less than a second, real time. 00:38:56.800 --> 00:38:59.200 Really, these are very small programs. One thing you should 00:38:59.200 --> 00:39:05.400 realise is that garbage collectors have to be small. 00:39:05.400 --> 00:39:08.770 Not because they have to be fast, but because no one can 00:39:08.770 --> 00:39:11.340 debug a complicated garbage collector. 00:39:11.340 --> 00:39:15.000 A garbage collector, if it doesn't work, will trash your 00:39:15.000 --> 00:39:16.940 memory in such a way that you cannot figure out what the 00:39:16.940 --> 00:39:18.350 hell happened. 00:39:18.350 --> 00:39:20.660 You need an audit trail. 00:39:20.660 --> 00:39:22.460 Because it rearranges everything, and how do you 00:39:22.460 --> 00:39:23.740 know what happened there? 00:39:23.740 --> 00:39:27.480 So this is the only kind of program that it really, 00:39:27.480 --> 00:39:30.100 seriously matters if you stare at it long enough so you 00:39:30.100 --> 00:39:31.970 believe that it works. 00:39:31.970 --> 00:39:35.100 And sort of prove it to yourself. 00:39:35.100 --> 00:39:36.940 So there's no way to debug it. 00:39:36.940 --> 00:39:39.230 And that takes it being small enough so you can 00:39:39.230 --> 00:39:41.690 hold it in your head. 00:39:41.690 --> 00:39:45.020 Garbage collectors are special in this way. 00:39:45.020 --> 00:39:47.130 So every reasonable garbage collector has gotten small, 00:39:47.130 --> 00:39:52.430 and generally small programs are fast. Yes. 00:39:52.430 --> 00:39:53.650 AUDIENCE: Can you repeat the name of this 00:39:53.650 --> 00:39:54.510 technique once again? 00:39:54.510 --> 00:39:56.220 PROFESSOR: That's the Minsky-Feinchel-Yochelson 00:39:56.220 --> 00:39:58.420 garbage collector. 00:39:58.420 --> 00:39:59.340 AUDIENCE: You got that? 00:39:59.340 --> 00:40:02.210 PROFESSOR: Minsky invented it in '61 for the RLE PDP-1. 00:40:02.210 --> 00:40:07.410 A version of it was developed and elaborated to be used in 00:40:07.410 --> 00:40:12.410 Multics Maclisp by Feinchel and Yochelson in somewhere 00:40:12.410 --> 00:40:19.570 around 1968 or '69. 00:40:19.570 --> 00:40:20.650 OK. 00:40:20.650 --> 00:40:22.640 Let's take a break. 00:40:22.640 --> 00:40:22.934 [MUSIC: "JESU, JOY OF MAN'S DESIRING" BY 00:40:22.934 --> 00:40:24.184 JOHANN SEBASTIAN BACH] 00:41:17.310 --> 00:41:21.600 PROFESSOR: Well we've come to the end of this subject, and 00:41:21.600 --> 00:41:24.860 we've already shown you a universal machine which is 00:41:24.860 --> 00:41:26.740 down to evaluator. 00:41:26.740 --> 00:41:28.980 It's down to the level of detail you could imagine you 00:41:28.980 --> 00:41:30.420 could make one. 00:41:30.420 --> 00:41:34.390 This is a particular implementation of Lisp, built 00:41:34.390 --> 00:41:37.530 on one of those scheme chips that was talked about 00:41:37.530 --> 00:41:39.180 yesterday, sitting over here. 00:41:39.180 --> 00:41:42.990 This is mostly interface to somebody's memory with a 00:41:42.990 --> 00:41:45.010 little bit of timing and other such stuff. 00:41:45.010 --> 00:41:48.760 But this fellow actually ran Lisp at a fairly reasonable 00:41:48.760 --> 00:41:50.610 rate, as interpretive. 00:41:50.610 --> 00:41:56.500 It ran Lisp as fast as a DEC PDP-10 back in 1979. 00:41:56.500 --> 00:41:59.870 And so it's gotten pretty hardware. 00:41:59.870 --> 00:42:02.470 Pretty concrete. 00:42:02.470 --> 00:42:05.000 We've also downed you a bit with the 00:42:05.000 --> 00:42:07.370 things you can compute. 00:42:07.370 --> 00:42:11.850 But is it the case that there are things we can't compute? 00:42:11.850 --> 00:42:14.690 And so I'd like to end this with showing you some things 00:42:14.690 --> 00:42:18.190 that you'd like be able to compute that you can't. 00:42:18.190 --> 00:42:22.720 The answer is yes, there are things you can't compute. 00:42:22.720 --> 00:42:28.200 For example, something you'd really like is-- 00:42:28.200 --> 00:42:30.210 if you're writing [UNINTELLIGIBLE], you'd like a 00:42:30.210 --> 00:42:32.800 program that would check that the thing you're 00:42:32.800 --> 00:42:34.630 going to do will work. 00:42:34.630 --> 00:42:36.080 Wouldn't that be nice? 00:42:36.080 --> 00:42:37.960 You'd like something that would catch infinite loops, 00:42:37.960 --> 00:42:43.190 for example, in programs that were written by users. 00:42:43.190 --> 00:42:45.890 But in general you can't write such a program that will read 00:42:45.890 --> 00:42:48.760 any program and determine whether or not it's an 00:42:48.760 --> 00:42:50.990 infinite loop. 00:42:50.990 --> 00:42:51.685 Let me show you that. 00:42:51.685 --> 00:42:53.340 It's a little bit of a minor mathematics. 00:42:58.780 --> 00:43:01.360 Let's imagine that we just had a mathematical 00:43:01.360 --> 00:43:02.620 function before we start. 00:43:02.620 --> 00:43:12.980 And there is one, called s, which takes a procedure and 00:43:12.980 --> 00:43:14.230 its argument, a. 00:43:19.320 --> 00:43:24.250 And what s does is it determines whether or not it's 00:43:24.250 --> 00:43:26.632 safe to run p on a. 00:43:26.632 --> 00:43:34.500 And what I mean by that is this: it's true if p applied 00:43:34.500 --> 00:43:45.330 to a will converge to a value without an error. 00:43:52.365 --> 00:44:06.890 And it's false if p of a loops forever or makes an error. 00:44:15.000 --> 00:44:18.780 Now that's surely a function. 00:44:18.780 --> 00:44:21.830 There is some for every procedure and for every 00:44:21.830 --> 00:44:25.900 argument you could give it that is either true or false 00:44:25.900 --> 00:44:28.440 that it converges without making an error. 00:44:28.440 --> 00:44:31.770 And you could make a giant table of them. 00:44:31.770 --> 00:44:34.710 But the question is, can you write a procedure that compute 00:44:34.710 --> 00:44:37.430 the values of this function? 00:44:37.430 --> 00:44:39.720 Well let's assume that we can. 00:44:39.720 --> 00:44:58.740 Suppose that we have a procedure called "safe" that 00:44:58.740 --> 00:44:59.990 computes the value of s. 00:45:12.170 --> 00:45:17.620 Now I'm going to show you by several methods that 00:45:17.620 --> 00:45:19.760 you can't do this. 00:45:19.760 --> 00:45:22.300 The easiest one, or the first one, let's define a procedure 00:45:22.300 --> 00:45:23.810 called diag1. 00:45:23.810 --> 00:45:38.250 Given that we have safe, we can define diag1 to be the 00:45:38.250 --> 00:45:43.430 procedure of one argument, p, which has the following 00:45:43.430 --> 00:45:44.780 properties. 00:45:44.780 --> 00:45:54.620 If if it's safe to apply p to itself, then I wish to have an 00:45:54.620 --> 00:45:55.870 infinite loop. 00:45:59.330 --> 00:46:00.715 Otherwise I'm going to return 3. 00:46:03.680 --> 00:46:04.470 Remember it was 42. 00:46:04.470 --> 00:46:07.060 What's the answer to the big question? 00:46:07.060 --> 00:46:08.525 Where of course we know what an infinite loop is. 00:46:12.050 --> 00:46:16.130 Infinite loop, to be a procedure of no arguments, 00:46:16.130 --> 00:46:18.430 which is that nice lambda calculus loop. 00:46:18.430 --> 00:46:24.680 Lambda of x, x of x, applied to lambda of x, x of x. 00:46:24.680 --> 00:46:26.550 So there's nothing left to the imagination here. 00:46:29.830 --> 00:46:32.500 Well let's see what the story is. 00:46:32.500 --> 00:46:38.100 I'm supposing it's the case that we worry about the 00:46:38.100 --> 00:46:43.180 procedure called diag1 applied to diag1. 00:46:45.860 --> 00:46:49.970 Well what could it possibly be? 00:46:49.970 --> 00:46:51.390 Well I don't know. 00:46:51.390 --> 00:46:57.310 We're going to substitute diag1 for p in the body here. 00:46:57.310 --> 00:47:00.220 Well is it safe to compute diag1 of diag1? 00:47:00.220 --> 00:47:00.780 I don't know. 00:47:00.780 --> 00:47:03.400 There are two possibilities. 00:47:03.400 --> 00:47:06.260 If it's safe to compute diag1 of diag1 that means it 00:47:06.260 --> 00:47:08.490 shouldn't loop. 00:47:08.490 --> 00:47:09.540 That means I go to here, but then I 00:47:09.540 --> 00:47:10.560 produce an infinite loop. 00:47:10.560 --> 00:47:12.210 So it can't be safe. 00:47:12.210 --> 00:47:15.055 But if it's not safe to compute diag1 of diag1 then 00:47:15.055 --> 00:47:16.020 the answer to this is 3. 00:47:16.020 --> 00:47:20.530 But that's diag1 of diag1, so it had to be safe. 00:47:20.530 --> 00:47:27.470 So therefore by contradiction you cannot produce safe. 00:47:27.470 --> 00:47:30.565 For those of you who were boggled by that one I'm going 00:47:30.565 --> 00:47:32.820 to say it again, in a different way. 00:47:32.820 --> 00:47:35.530 Listen to one more alternative. 00:47:35.530 --> 00:47:36.780 Let's define diag2. 00:47:39.840 --> 00:47:45.260 These are named diag because of Cantor's diagonal argument. 00:47:45.260 --> 00:47:48.180 These are instances of a famous argument which was 00:47:48.180 --> 00:47:52.070 originally used by Cantor in the late part of the last 00:47:52.070 --> 00:47:56.420 century to prove that the real numbers were not countable, 00:47:56.420 --> 00:47:58.160 that there are too many real numbers to 00:47:58.160 --> 00:48:00.190 be counted by integers. 00:48:00.190 --> 00:48:02.710 That there are more points on a line, for example, than 00:48:02.710 --> 00:48:05.260 there are counting numbers. 00:48:05.260 --> 00:48:07.190 It may or may not be obvious, and I don't want to 00:48:07.190 --> 00:48:08.440 get into that now. 00:48:10.900 --> 00:48:15.820 But diag2 is again a procedure of one argument p. 00:48:15.820 --> 00:48:19.220 It's almost the same as the previous one, which is, if 00:48:19.220 --> 00:48:26.445 it's safe to compute p on p, then I'm going to produce-- 00:48:29.310 --> 00:48:34.010 then I want to compute some other things 00:48:34.010 --> 00:48:38.960 other than p of p. 00:48:38.960 --> 00:48:40.210 Otherwise I'm going to put out false. 00:48:43.600 --> 00:48:46.510 Where other then it says, whatever p of p, I'm going to 00:48:46.510 --> 00:48:48.880 put out something else. 00:48:48.880 --> 00:48:51.860 I can give you an example of a definition of other than which 00:48:51.860 --> 00:48:53.890 I think works. 00:48:53.890 --> 00:48:55.640 Let's see. 00:48:55.640 --> 00:48:56.330 Yes. 00:48:56.330 --> 00:49:06.580 Where other than be a procedure of one argument x 00:49:06.580 --> 00:49:14.090 which says, if its eq x to, say, quote a, then the answer 00:49:14.090 --> 00:49:15.720 is quote b. 00:49:15.720 --> 00:49:16.970 Otherwise it's quote a. 00:49:20.090 --> 00:49:22.770 That always produces something which is not what 00:49:22.770 --> 00:49:25.350 its argument is. 00:49:25.350 --> 00:49:26.540 That's all it is. 00:49:26.540 --> 00:49:28.250 That's all I wanted. 00:49:28.250 --> 00:49:30.640 Well now let's consider this one, diag2 of diag2. 00:49:38.220 --> 00:49:39.560 Well look. 00:49:39.560 --> 00:49:42.950 This only does something dangerous, like calling p of 00:49:42.950 --> 00:49:47.470 p, if it's safe to do so. 00:49:47.470 --> 00:49:51.590 So if safe defined at all, if you can define such a 00:49:51.590 --> 00:49:55.680 procedure, safe, then this procedure is always defined 00:49:55.680 --> 00:49:57.225 and therefore safe on any inputs. 00:50:01.540 --> 00:50:11.770 So diag2 of diag2 must reduce to other than diag2 of diag2. 00:50:15.496 --> 00:50:20.020 And that doesn't make sense, so we have a contradiction, 00:50:20.020 --> 00:50:22.950 and therefore we can't define safe. 00:50:22.950 --> 00:50:27.210 I just waned to do that twice, slightly differently, so you 00:50:27.210 --> 00:50:32.260 wouldn't feel that the first one was a trick. 00:50:32.260 --> 00:50:34.275 They may be both tricks, but they're at 00:50:34.275 --> 00:50:37.300 least slightly different. 00:50:37.300 --> 00:50:40.080 So I suppose that pretty much wraps it up. 00:50:40.080 --> 00:50:43.540 I've just proved what we call the halting theorem, and I 00:50:43.540 --> 00:50:46.720 suppose with that we're going to halt. 00:50:46.720 --> 00:50:47.970 I hope you have a good time. 00:50:50.900 --> 00:50:53.300 Are there any questions? 00:50:53.300 --> 00:50:53.810 Yes. 00:50:53.810 --> 00:50:56.940 AUDIENCE: What is the value of s of diag1? 00:50:56.940 --> 00:50:57.430 PROFESSOR: Of what? 00:50:57.430 --> 00:51:00.120 AUDIENCE: S of diag1. 00:51:00.120 --> 00:51:02.340 If you said s is a function and we can 00:51:02.340 --> 00:51:02.620 [INTERPOSING VOICES] 00:51:02.620 --> 00:51:03.870 PROFESSOR: Oh, I don't know. 00:51:03.870 --> 00:51:04.350 I don't know. 00:51:04.350 --> 00:51:06.850 It's a function, but I don't know how to compute it. 00:51:06.850 --> 00:51:08.610 I can't do it. 00:51:08.610 --> 00:51:11.530 I'm just a machine, too. 00:51:11.530 --> 00:51:12.210 Right? 00:51:12.210 --> 00:51:14.670 There's no machine that in principle-- 00:51:14.670 --> 00:51:16.690 it might be that in that particular case you just 00:51:16.690 --> 00:51:18.580 asked, with some thinking I could figure it out. 00:51:18.580 --> 00:51:21.670 But in general I can't compute the value of s any better than 00:51:21.670 --> 00:51:23.780 any other machine can. 00:51:23.780 --> 00:51:27.210 There is such a function, it's just that no machine can be 00:51:27.210 --> 00:51:29.580 built to compute it. 00:51:29.580 --> 00:51:32.980 Now there's a way of saying that that should not be 00:51:32.980 --> 00:51:35.350 surprising. 00:51:35.350 --> 00:51:36.390 Going through this-- 00:51:36.390 --> 00:51:41.020 I mean, I don't have time to do this here, but the number 00:51:41.020 --> 00:51:44.600 of functions is very large. 00:51:44.600 --> 00:51:48.210 If there's a certain number of answers possible and a certain 00:51:48.210 --> 00:51:50.520 number of inputs possible, then it's the number of 00:51:50.520 --> 00:51:52.290 answers raised to the number inputs is the number of 00:51:52.290 --> 00:51:54.720 possible functions. 00:51:54.720 --> 00:51:55.970 On one variable. 00:51:58.150 --> 00:52:03.690 Now that's always bigger than the thing you're raising to, 00:52:03.690 --> 00:52:05.480 the exponent. 00:52:05.480 --> 00:52:12.150 The number of functions is larger than the number of 00:52:12.150 --> 00:52:15.340 programs that one can write, by an 00:52:15.340 --> 00:52:17.840 infinity counting argument. 00:52:17.840 --> 00:52:19.475 And it's much larger. 00:52:19.475 --> 00:52:22.540 So there must be a lot of functions that can't be 00:52:22.540 --> 00:52:26.280 computed by programs. 00:52:26.280 --> 00:52:27.300 AUDIENCE: A few moments ago you were talking about 00:52:27.300 --> 00:52:30.640 specifications and automatic generation of solutions. 00:52:30.640 --> 00:52:33.360 Do you see any steps between specifications and solutions? 00:52:37.250 --> 00:52:38.720 PROFESSOR: Steps between. 00:52:38.720 --> 00:52:42.720 You mean, you're saying, how you go about constructing 00:52:42.720 --> 00:52:45.205 devices given that have specifications for the device? 00:52:45.205 --> 00:52:45.500 Sure. 00:52:45.500 --> 00:52:48.540 AUDIENCE: There's a lot of software engineering that goes 00:52:48.540 --> 00:52:51.703 through specifications through many layers of design and then 00:52:51.703 --> 00:52:52.430 implementation. 00:52:52.430 --> 00:52:52.850 PROFESSOR: Yes? 00:52:52.850 --> 00:52:55.600 AUDIENCE: I was curious if you think that's realistic. 00:52:55.600 --> 00:52:57.210 PROFESSOR: Well I think that some of it's realistic and 00:52:57.210 --> 00:52:58.100 some of it isn't. 00:52:58.100 --> 00:53:01.680 I mean, surely if I want to build an electrical filter and 00:53:01.680 --> 00:53:07.160 I have a rather interesting possibility. 00:53:07.160 --> 00:53:12.180 Supposing I want to build a thing that matches some power 00:53:12.180 --> 00:53:19.906 output to the radio transmitter, to some antenna. 00:53:19.906 --> 00:53:21.970 And I'm really out of this power-- 00:53:21.970 --> 00:53:23.230 it's output tube out here. 00:53:23.230 --> 00:53:25.920 And the problem is that they have different impedances. 00:53:25.920 --> 00:53:27.550 I want them to match the impedances. 00:53:27.550 --> 00:53:29.920 I also want to make a filter in there which is going to get 00:53:29.920 --> 00:53:32.780 rid of some harmonic radiation. 00:53:32.780 --> 00:53:36.270 Well one old-fashioned technique for doing this is 00:53:36.270 --> 00:53:38.860 called image impedances, or something like that. 00:53:38.860 --> 00:53:40.240 And what you do is you say you have a basic 00:53:40.240 --> 00:53:43.300 module called an L-section. 00:53:43.300 --> 00:53:44.550 Looks like this. 00:53:47.080 --> 00:53:50.470 If I happen to connect this to some resistance, r, and if I 00:53:50.470 --> 00:53:55.150 make this impedance x, xl, and if it happens to be q times r, 00:53:55.150 --> 00:53:59.710 then this produces a low pass filter with a q square plus 00:53:59.710 --> 00:54:02.110 one impedance match. 00:54:02.110 --> 00:54:03.120 Just what I need. 00:54:03.120 --> 00:54:04.610 Because now I can take two of these, hook them 00:54:04.610 --> 00:54:06.510 together like this. 00:54:11.660 --> 00:54:16.570 OK, and I take another one and I'll hook them 00:54:16.570 --> 00:54:18.290 together like that. 00:54:18.290 --> 00:54:20.320 And I have two L-sections hooked together. 00:54:20.320 --> 00:54:22.460 And this will step the impedance down to one that I 00:54:22.460 --> 00:54:25.530 know, and this will step it up to one I know. 00:54:25.530 --> 00:54:26.710 Each of these is a low pass filter 00:54:26.710 --> 00:54:28.090 getting rid of some harmonics. 00:54:28.090 --> 00:54:30.270 It's good filter, it's called a pie-section filter. 00:54:30.270 --> 00:54:31.700 Great. 00:54:31.700 --> 00:54:34.300 Except for the fact that in doing what I just did, I've 00:54:34.300 --> 00:54:38.620 made a terrible inefficiency in this system. 00:54:38.620 --> 00:54:41.620 I've made two coils where I should have made one. 00:54:41.620 --> 00:54:45.200 And the problem with most software engineering art is 00:54:45.200 --> 00:54:47.440 that there's no mechanism, other than peephole 00:54:47.440 --> 00:54:50.280 optimization and compilers, for getting rid of the 00:54:50.280 --> 00:54:52.810 redundant parts that are constructed when doing top 00:54:52.810 --> 00:54:55.350 down design. 00:54:55.350 --> 00:54:57.160 It's even worse, there are lots of very important 00:54:57.160 --> 00:55:01.110 structures that you can't construct at all this way. 00:55:01.110 --> 00:55:03.940 So I think that the standard top down design is a rather 00:55:03.940 --> 00:55:05.710 shallow business. 00:55:05.710 --> 00:55:08.315 Doesn't really capture what people want to do in design. 00:55:08.315 --> 00:55:10.100 I'll give you another electrical example. 00:55:10.100 --> 00:55:12.140 Electrical examples are so much clearer than 00:55:12.140 --> 00:55:14.440 computational examples, because computation examples 00:55:14.440 --> 00:55:17.220 require a certain degree of complexity to explain them. 00:55:17.220 --> 00:55:19.650 But one of my favorite examples in the electrical 00:55:19.650 --> 00:55:23.330 world is how would I ever come up with the output stage of 00:55:23.330 --> 00:55:27.530 this inter-stage connection in an IF amplifier. 00:55:27.530 --> 00:55:32.410 It's a little transistor here, and let's see. 00:55:32.410 --> 00:55:37.560 Well I'm going to have a tank, and I'm going to hook this up 00:55:37.560 --> 00:55:43.040 to, say, I'm going to link-couple that to the input 00:55:43.040 --> 00:55:44.850 of the next stage. 00:55:44.850 --> 00:55:48.580 Here's a perfectly plausible plan-- 00:55:48.580 --> 00:55:51.070 well except for the fact that since I put that going up I 00:55:51.070 --> 00:55:53.170 should make that going that way. 00:55:53.170 --> 00:55:56.050 Here's a perfectly plausible plan for a-- 00:55:56.050 --> 00:55:57.270 no I shouldn't. 00:55:57.270 --> 00:55:57.940 I'm dumb. 00:55:57.940 --> 00:55:59.690 Excuse me. 00:55:59.690 --> 00:56:00.730 Doesn't matter. 00:56:00.730 --> 00:56:01.540 The point is [UNINTELLIGIBLE] 00:56:01.540 --> 00:56:02.560 plan for a couple [UNINTELLIGIBLE] 00:56:02.560 --> 00:56:04.590 stages together. 00:56:04.590 --> 00:56:07.620 Now what the problem is is what's this hierarchically? 00:56:07.620 --> 00:56:09.480 It's not one thing. 00:56:09.480 --> 00:56:11.990 Hierarchically it doesn't make any sense at all. 00:56:11.990 --> 00:56:17.230 It's the inductance of a tuned circuit, it's the primary of a 00:56:17.230 --> 00:56:22.350 transformer, and it's also the DC path by which bias 00:56:22.350 --> 00:56:26.460 conditions get to the collector of that transistor. 00:56:26.460 --> 00:56:29.170 And there's no simple top-down design that's going to produce 00:56:29.170 --> 00:56:33.400 a structure like that with so many overlapping uses for a 00:56:33.400 --> 00:56:34.530 particular thing. 00:56:34.530 --> 00:56:39.015 Playing Scrabble, where you have to do triple word scores, 00:56:39.015 --> 00:56:44.950 or whatever, is not so easy in top-down design strategy. 00:56:44.950 --> 00:56:48.100 Yet most of real engineering is based on getting the most 00:56:48.100 --> 00:56:52.140 oomph for effort. 00:56:52.140 --> 00:56:54.860 And that's what you're seeing here. 00:56:54.860 --> 00:56:55.550 Yeah? 00:56:55.550 --> 00:56:56.810 AUDIENCE: Is this the last question? 00:57:00.282 --> 00:57:18.640 [LAUGHTER] 00:57:18.640 --> 00:57:19.890 PROFESSOR: Apparently so. 00:57:23.240 --> 00:57:26.092 Thank you. 00:57:26.092 --> 00:57:39.040 [APPLAUSE] 00:57:39.040 --> 00:57:39.383 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 00:57:39.383 --> 00:57:40.633 JOHANN SEBASTIAN BACH]