1 00:00:04,970 --> 00:00:05,285 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 2 00:00:05,285 --> 00:00:06,535 JOHANN SEBASTIAN BACH] 3 00:00:18,910 --> 00:00:22,140 PROFESSOR: Well, there's one bit of mystery left, which I'd 4 00:00:22,140 --> 00:00:24,440 like to get rid of right now. 5 00:00:24,440 --> 00:00:28,210 And that's that we've been blithely doing things like 6 00:00:28,210 --> 00:00:33,660 cons assuming there's always another one. 7 00:00:33,660 --> 00:00:37,060 That we've been doing these things like car-ing and 8 00:00:37,060 --> 00:00:38,750 cdr-ing and assuming that we had some idea 9 00:00:38,750 --> 00:00:40,020 how this can be done. 10 00:00:40,020 --> 00:00:43,800 Now indeed we said that that's equivalent to having 11 00:00:43,800 --> 00:00:45,780 procedures. 12 00:00:45,780 --> 00:00:48,360 But that doesn't really solve the problem, because the 13 00:00:48,360 --> 00:00:50,470 procedure need all sorts of complicated mechanisms like 14 00:00:50,470 --> 00:00:53,010 environment structures and things like that to work. 15 00:00:53,010 --> 00:00:55,770 And those were ultimately made out of conses in the model 16 00:00:55,770 --> 00:00:59,380 that we had, so that really doesn't solve the problem. 17 00:00:59,380 --> 00:01:02,860 Now the problem here is the glue the data 18 00:01:02,860 --> 00:01:04,760 structure's made out of. 19 00:01:04,760 --> 00:01:07,370 What kind of possible thing could it be? 20 00:01:07,370 --> 00:01:11,060 We've been showing you things like a machine, a computer 21 00:01:11,060 --> 00:01:14,700 that has a controller, and some 22 00:01:14,700 --> 00:01:16,980 registers, and maybe a stack. 23 00:01:16,980 --> 00:01:18,270 And we haven't said anything about, for 24 00:01:18,270 --> 00:01:20,570 example, larger memory. 25 00:01:20,570 --> 00:01:23,740 And I think that's what we have to worry about right now. 26 00:01:23,740 --> 00:01:27,160 But just to make it perfectly clear that this is an 27 00:01:27,160 --> 00:01:31,320 inessential, purely implementational thing, I'd 28 00:01:31,320 --> 00:01:33,500 like to show you, for example, how you can do it 29 00:01:33,500 --> 00:01:34,800 all with the numbers. 30 00:01:34,800 --> 00:01:37,590 That's an easy one. 31 00:01:37,590 --> 00:01:45,020 Famous fellow by the name of Godel, a logician at the end 32 00:01:45,020 --> 00:01:51,050 of the 1930s, invented a very clever way of encoding the 33 00:01:51,050 --> 00:01:54,320 complicated expressions as numbers. 34 00:01:54,320 --> 00:01:55,540 For example-- 35 00:01:55,540 --> 00:01:58,250 I'm not saying exactly what Godel's scheme is, because he 36 00:01:58,250 --> 00:01:59,660 didn't use words like cons. 37 00:01:59,660 --> 00:02:03,090 He had other kinds of ways of combining to make expressions. 38 00:02:03,090 --> 00:02:05,860 But he said, I'm going to assign a number to every 39 00:02:05,860 --> 00:02:07,920 algebraic expression. 40 00:02:07,920 --> 00:02:09,970 And the way I'm going to manufacture these numbers is 41 00:02:09,970 --> 00:02:12,470 by combining the numbers of the parts. 42 00:02:12,470 --> 00:02:15,880 So for example, what we were doing our world, we could say 43 00:02:15,880 --> 00:02:34,130 that if objects are represented by numbers, then 44 00:02:34,130 --> 00:02:42,660 cons of x and y could be represented by 2 to the x 45 00:02:42,660 --> 00:02:46,130 times 2 to the y. 46 00:02:46,130 --> 00:02:49,560 Because then we could extract the parts. 47 00:02:49,560 --> 00:02:57,500 We could say, for example, that then car of, say, x is 48 00:02:57,500 --> 00:03:06,690 the number of factors of 2 in x. 49 00:03:06,690 --> 00:03:10,690 And of course cdr is the same thing. 50 00:03:10,690 --> 00:03:16,510 It's the number of factors of 3 in x. 51 00:03:16,510 --> 00:03:19,660 Now this is a perfectly reasonable scheme, except for 52 00:03:19,660 --> 00:03:22,870 the fact that the numbers rapidly get to be much larger 53 00:03:22,870 --> 00:03:25,500 in number of digits than the number of 54 00:03:25,500 --> 00:03:27,950 protons in the universe. 55 00:03:27,950 --> 00:03:30,420 So there's no easy way to use this scheme other than the 56 00:03:30,420 --> 00:03:33,430 theoretical one. 57 00:03:33,430 --> 00:03:37,010 On the other hand, there are other ways of representing 58 00:03:37,010 --> 00:03:38,450 these things. 59 00:03:38,450 --> 00:03:44,010 We have been thinking in terms of little boxes. 60 00:03:44,010 --> 00:03:47,000 We've been thinking about our cons structures as looking 61 00:03:47,000 --> 00:03:50,280 sort of like this. 62 00:03:50,280 --> 00:03:53,610 They're little pigeon holes with things in them. 63 00:03:53,610 --> 00:03:57,210 And of course we arrange them in little trees. 64 00:03:57,210 --> 00:04:00,680 I wish that the semiconductor manufacturers would supply me 65 00:04:00,680 --> 00:04:04,280 with something appropriate for this, but actually what they 66 00:04:04,280 --> 00:04:09,380 do supply me with is a linear memory. 67 00:04:09,380 --> 00:04:15,170 Memory is sort of a big pile of pigeonholes, 68 00:04:15,170 --> 00:04:17,720 pigeonholes like this. 69 00:04:17,720 --> 00:04:21,470 Each of which can hold a certain sized object, a fixed 70 00:04:21,470 --> 00:04:23,390 size object. 71 00:04:23,390 --> 00:04:25,890 So, for example, a complicated list with 25 elements won't 72 00:04:25,890 --> 00:04:28,550 fit in one of these. 73 00:04:28,550 --> 00:04:30,600 However, each of these is indexed by an address. 74 00:04:33,970 --> 00:04:36,750 So the address might be zero here, one here, two here, 75 00:04:36,750 --> 00:04:38,060 three here, and so on. 76 00:04:38,060 --> 00:04:40,400 That we write these down as numbers is unimportant. 77 00:04:40,400 --> 00:04:42,710 What matters is that they're distinct as a way to get to 78 00:04:42,710 --> 00:04:44,970 the next one. 79 00:04:44,970 --> 00:04:48,300 And inside of each of these, we can stuff something into 80 00:04:48,300 --> 00:04:49,530 these pigeonholes. 81 00:04:49,530 --> 00:04:52,300 That's what memory is like, for those of you who haven't 82 00:04:52,300 --> 00:04:53,550 built a computer. 83 00:04:56,690 --> 00:04:59,280 Now the problem is how are we going to impose on this type 84 00:04:59,280 --> 00:05:03,290 of structure, this nice tree structure. 85 00:05:03,290 --> 00:05:05,480 Well it's not very hard, and there have been numerous 86 00:05:05,480 --> 00:05:06,630 schemes involved in this. 87 00:05:06,630 --> 00:05:09,930 The most important one is to say, well assuming that the 88 00:05:09,930 --> 00:05:13,920 semiconductor manufacturer allows me to arrange my memory 89 00:05:13,920 --> 00:05:16,390 so that one of these pigeonholes is big enough to 90 00:05:16,390 --> 00:05:21,706 hold the address of another I haven't made. 91 00:05:21,706 --> 00:05:23,730 Now it actually has to be a little bit bigger because I 92 00:05:23,730 --> 00:05:28,215 have to also install or store some information as to a tag 93 00:05:28,215 --> 00:05:30,390 which describes the kind of thing that's there. 94 00:05:30,390 --> 00:05:32,350 And we'll see that in a second. 95 00:05:32,350 --> 00:05:34,560 And of course if the semiconductor manufacturer 96 00:05:34,560 --> 00:05:37,470 doesn't arrange it so I can do that, then of course I can, 97 00:05:37,470 --> 00:05:40,910 with some cleverness, arrange combinations of these to fit 98 00:05:40,910 --> 00:05:43,770 together in that way. 99 00:05:43,770 --> 00:05:48,510 So we're going to have to imagine imposing this 100 00:05:48,510 --> 00:05:51,740 complicated tree structure on our nice linear memory. 101 00:05:51,740 --> 00:05:57,540 If we look at the first still store, we see a classic scheme 102 00:05:57,540 --> 00:05:59,490 for doing that. 103 00:05:59,490 --> 00:06:03,910 It's a standard way of representing Lisp structures 104 00:06:03,910 --> 00:06:05,980 in a linear memory. 105 00:06:05,980 --> 00:06:12,030 What we do is we divide this memory into two parts. 106 00:06:12,030 --> 00:06:17,580 An array called the cars, and an array called the cdrs. 107 00:06:17,580 --> 00:06:20,470 Now whether those happen to be sequential addresses or 108 00:06:20,470 --> 00:06:22,560 whatever, it's not important. 109 00:06:22,560 --> 00:06:25,800 That's somebody's implementation details. 110 00:06:25,800 --> 00:06:28,960 But there are two arrays here. 111 00:06:28,960 --> 00:06:34,840 Linear arrays indexed by sequential indices like this. 112 00:06:34,840 --> 00:06:36,150 What is stored in each of these 113 00:06:36,150 --> 00:06:41,430 pigeonholes is a typed object. 114 00:06:41,430 --> 00:06:44,840 And what we have here are types which begin with letters 115 00:06:44,840 --> 00:06:47,790 like p, standing for a pair. 116 00:06:47,790 --> 00:06:50,040 Or n, standing for a number. 117 00:06:50,040 --> 00:06:57,290 Or e, standing for an empty list. The end of the list. And 118 00:06:57,290 --> 00:07:00,420 so if we wish to represent an object like this, the list 119 00:07:00,420 --> 00:07:04,310 beginning with 1, 2 and then having a 3 and a 4 as its 120 00:07:04,310 --> 00:07:06,430 second and third elements. 121 00:07:06,430 --> 00:07:10,220 A list containing a list as its first part and then two 122 00:07:10,220 --> 00:07:12,610 numbers as a second and third parts. 123 00:07:12,610 --> 00:07:15,250 Then of course we draw it sort of like this these days, in 124 00:07:15,250 --> 00:07:17,320 box-and-pointer notation. 125 00:07:17,320 --> 00:07:21,190 And you see, these are the three cells that have as their 126 00:07:21,190 --> 00:07:28,390 car pointer the object which is either 1, 2 or 3 or 4. 127 00:07:28,390 --> 00:07:30,860 And then of course the 1, 2, the car of this entire 128 00:07:30,860 --> 00:07:33,870 structure, is itself a substructure which contains a 129 00:07:33,870 --> 00:07:35,940 sublist like that. 130 00:07:35,940 --> 00:07:39,970 What I'm about to do is put down places which are-- 131 00:07:39,970 --> 00:07:41,880 I'm going to assign indices. 132 00:07:41,880 --> 00:07:45,530 Like this 1, over here, represents the 133 00:07:45,530 --> 00:07:46,850 index of this cell. 134 00:07:49,850 --> 00:07:55,540 But that pointer that we see here is a reference to the 135 00:07:55,540 --> 00:07:57,640 pair of pigeonholes in the cars and the cdrs that are 136 00:07:57,640 --> 00:08:02,000 labeled by 1 in my linear memory down here. 137 00:08:02,000 --> 00:08:05,920 So if I wish to impose this structure on my linear memory, 138 00:08:05,920 --> 00:08:08,780 what I do is I say, oh yes, why don't we drop 139 00:08:08,780 --> 00:08:12,220 this into cell 1? 140 00:08:12,220 --> 00:08:12,660 I pick one. 141 00:08:12,660 --> 00:08:14,270 There's 1. 142 00:08:14,270 --> 00:08:16,640 And that says that its car, I'm going to 143 00:08:16,640 --> 00:08:17,950 assign it to be a pair. 144 00:08:17,950 --> 00:08:22,590 It's a pair, which is in index 5. 145 00:08:22,590 --> 00:08:26,360 And the cdr, which is this one over here, is a pair which I'm 146 00:08:26,360 --> 00:08:28,340 going to stick into place 2. 147 00:08:28,340 --> 00:08:30,890 p2. 148 00:08:30,890 --> 00:08:32,950 And take a look at p2. 149 00:08:32,950 --> 00:08:37,550 Oh yes, well p2 is a thing whose car is the number 3, so 150 00:08:37,550 --> 00:08:39,520 as you see, an n3. 151 00:08:39,520 --> 00:08:46,640 And whose cdr, over here, is a pair, which lives in place 4. 152 00:08:46,640 --> 00:08:48,650 So that's what this p4 is. 153 00:08:48,650 --> 00:08:56,200 p4 is a number whose value is 4 in its car and whose cdr is 154 00:08:56,200 --> 00:08:59,170 an empty list right there. 155 00:08:59,170 --> 00:09:00,690 And that ends it. 156 00:09:00,690 --> 00:09:05,750 So this is the traditional way of representing this kind of 157 00:09:05,750 --> 00:09:11,620 binary tree in a linear memory. 158 00:09:11,620 --> 00:09:15,770 Now the next question, of course, that we might want to 159 00:09:15,770 --> 00:09:18,440 worry about is just a little bit of implementation. 160 00:09:18,440 --> 00:09:22,690 That means that when I write procedures of the form 161 00:09:22,690 --> 00:09:24,600 assigned a, [UNINTELLIGIBLE] procedures-- 162 00:09:24,600 --> 00:09:29,000 lines of register machine code of the form assigned a, the 163 00:09:29,000 --> 00:09:30,140 car of [UNINTELLIGIBLE] 164 00:09:30,140 --> 00:09:38,740 b, what I really mean is addressing these elements. 165 00:09:38,740 --> 00:09:44,470 And so we're going to think of that as a abbreviation for it. 166 00:09:44,470 --> 00:09:46,720 Now of course in order to write that down I'm going to 167 00:09:46,720 --> 00:09:49,140 introduce some sort of a structure called a vector. 168 00:09:52,120 --> 00:09:53,990 And we're going to have something which will reference 169 00:09:53,990 --> 00:09:58,710 a vector, just so we can write it down. 170 00:09:58,710 --> 00:10:02,240 Which takes the name of the vector, or the-- 171 00:10:02,240 --> 00:10:03,970 I don't think that name is the right word. 172 00:10:03,970 --> 00:10:12,010 Which takes the vector and the index, and I have to have a 173 00:10:12,010 --> 00:10:13,950 way of setting one of those with something called a vector 174 00:10:13,950 --> 00:10:16,280 set, I don't really care. 175 00:10:16,280 --> 00:10:19,520 But let's look, for example, at then that kind of 176 00:10:19,520 --> 00:10:26,470 implementation of car and cdr. 177 00:10:26,470 --> 00:10:31,470 So for example if I happen to have a register b, which 178 00:10:31,470 --> 00:10:37,580 contains the type index of a pair, and therefore it is the 179 00:10:37,580 --> 00:10:41,930 pointer to a pair, then I could take the car of that and 180 00:10:41,930 --> 00:10:42,760 if I-- write this down-- 181 00:10:42,760 --> 00:10:44,490 I might put that in register a. 182 00:10:44,490 --> 00:10:49,400 What that really is is a representation of the assign 183 00:10:49,400 --> 00:10:52,890 to a, the value of vector reffing-- 184 00:10:52,890 --> 00:10:54,700 or array indexing, if you will-- or 185 00:10:54,700 --> 00:10:58,490 something, the cars object-- 186 00:10:58,490 --> 00:10:59,990 whatever that is-- 187 00:10:59,990 --> 00:11:02,650 with the index, b. 188 00:11:02,650 --> 00:11:06,330 And similarly for cdr. And we can do the same thing for 189 00:11:06,330 --> 00:11:10,370 assignment to data structures, if we need to do that sort of 190 00:11:10,370 --> 00:11:11,840 thing at all. 191 00:11:11,840 --> 00:11:14,580 It's not too hard to build that. 192 00:11:14,580 --> 00:11:16,170 Well now the next question is how are we going to do 193 00:11:16,170 --> 00:11:18,010 allocation. 194 00:11:18,010 --> 00:11:21,550 And every so often I say I want a cons. 195 00:11:21,550 --> 00:11:23,790 Now conses don't grow on trees. 196 00:11:23,790 --> 00:11:25,340 Or maybe they should. 197 00:11:25,340 --> 00:11:29,980 But I have to have some way of getting the next one. 198 00:11:29,980 --> 00:11:33,920 I have to have some idea of if their memory is unused that I 199 00:11:33,920 --> 00:11:35,630 might want to allocate from. 200 00:11:35,630 --> 00:11:37,380 And there are many schemes for doing this. 201 00:11:37,380 --> 00:11:38,660 And the particular thing I'm showing you 202 00:11:38,660 --> 00:11:42,100 right now is not essential. 203 00:11:42,100 --> 00:11:44,960 However it's convenient and has been done many times. 204 00:11:44,960 --> 00:11:47,660 One scheme's was called the free list allocation scheme. 205 00:11:47,660 --> 00:11:50,570 What that means is that all of the free memory that there is 206 00:11:50,570 --> 00:11:54,700 in the world is linked together in a linked list, 207 00:11:54,700 --> 00:11:56,960 just like all the other stuff. 208 00:11:56,960 --> 00:12:01,230 And whenever you need a free cell to make a new cons, you 209 00:12:01,230 --> 00:12:04,440 grab the first, one make the free list be the cdr of it, 210 00:12:04,440 --> 00:12:06,030 and then allocate that. 211 00:12:06,030 --> 00:12:09,530 And so what that looks like is something like this. 212 00:12:09,530 --> 00:12:18,510 Here we have the free list starting in 6. 213 00:12:18,510 --> 00:12:24,860 And what that is is a pointer-off to say 8. 214 00:12:24,860 --> 00:12:27,020 So what it says is, this one is free and the 215 00:12:27,020 --> 00:12:28,870 next one is an 8. 216 00:12:28,870 --> 00:12:32,880 This one is free and the next one is in 3, the next one 217 00:12:32,880 --> 00:12:33,930 that's free. 218 00:12:33,930 --> 00:12:37,680 That one's free and the next one is in 0. 219 00:12:37,680 --> 00:12:40,940 That one's free and the next one's in 15. 220 00:12:40,940 --> 00:12:42,780 Something like that. 221 00:12:42,780 --> 00:12:46,400 We can imagine having such a structure. 222 00:12:46,400 --> 00:12:50,480 Given that we have something like that, then it's possible 223 00:12:50,480 --> 00:12:53,940 to just get one when you need it. 224 00:12:53,940 --> 00:12:57,960 And so a program for doing cons, this is what 225 00:12:57,960 --> 00:12:59,320 cons might turn into. 226 00:12:59,320 --> 00:13:05,410 To assign to a register A the result of cons-ing, a B onto 227 00:13:05,410 --> 00:13:08,250 C, the value in this containing B and the value 228 00:13:08,250 --> 00:13:11,240 containing C, what we have to do is get the current 229 00:13:11,240 --> 00:13:13,400 [? type ?] ahead of the freelist, make the free list 230 00:13:13,400 --> 00:13:19,840 be its cdr. Then we have to change the cars to be the 231 00:13:19,840 --> 00:13:25,680 thing we're making up to be in A to be the B, the thing in B. 232 00:13:25,680 --> 00:13:30,880 And we have to make change the cdrs of the thing that's in A 233 00:13:30,880 --> 00:13:36,020 to be C. And then what we have in A is the right new frob, 234 00:13:36,020 --> 00:13:36,650 whatever it is. 235 00:13:36,650 --> 00:13:40,470 The object that we want. 236 00:13:40,470 --> 00:13:43,490 Now there's a little bit of a cheat here that I haven't told 237 00:13:43,490 --> 00:13:47,155 you about, which is somewhere around here I haven't set that 238 00:13:47,155 --> 00:13:51,540 I've the type of the thing that I'm cons-ing up to be a 239 00:13:51,540 --> 00:13:53,510 pair, and I ought to. 240 00:13:53,510 --> 00:13:56,570 So there should be some sort of bits here are being set, 241 00:13:56,570 --> 00:13:59,810 and I just haven't written that down. 242 00:13:59,810 --> 00:14:01,480 We could have arranged it, of course, for the free lift to 243 00:14:01,480 --> 00:14:03,100 be made out of pairs. 244 00:14:03,100 --> 00:14:06,430 And so then there's no problem with that. 245 00:14:06,430 --> 00:14:10,160 But that sort of-- again, an inessential detail in a way 246 00:14:10,160 --> 00:14:13,500 some particular programmer or architect or whatever might 247 00:14:13,500 --> 00:14:17,540 manufacture his machine or Lisp system. 248 00:14:17,540 --> 00:14:23,930 So for example, just looking at this, to allocate given 249 00:14:23,930 --> 00:14:27,200 that I had already the structure that you saw before, 250 00:14:27,200 --> 00:14:31,900 supposing I wanted to allocate a new cell, which is going to 251 00:14:31,900 --> 00:14:38,680 be representation of list one, one, two, where already one 252 00:14:38,680 --> 00:14:43,430 two was the car of the list we were playing with before. 253 00:14:43,430 --> 00:14:44,780 Well that's not so hard. 254 00:14:44,780 --> 00:14:47,670 I stored that one and one, so p1 one is the 255 00:14:47,670 --> 00:14:49,530 representation of this. 256 00:14:49,530 --> 00:14:51,690 This is p5. 257 00:14:51,690 --> 00:14:54,070 That's going to be the cdr of this. 258 00:14:54,070 --> 00:14:55,610 Now we're going to pull something off the free list, 259 00:14:55,610 --> 00:14:57,780 but remember the free list started at six. 260 00:14:57,780 --> 00:15:01,540 The new free list after this allocation is eight, a free 261 00:15:01,540 --> 00:15:02,890 list beginning at eight. 262 00:15:02,890 --> 00:15:06,360 And of course in six now we have a number one, which is 263 00:15:06,360 --> 00:15:10,540 what we wanted, with its cdr being the pair starting in 264 00:15:10,540 --> 00:15:13,330 location five. 265 00:15:13,330 --> 00:15:16,810 And that's no big deal. 266 00:15:16,810 --> 00:15:21,480 So the only problem really remaining here is, well, I 267 00:15:21,480 --> 00:15:25,080 don't have an infinitely large memory. 268 00:15:25,080 --> 00:15:28,070 If I do this for a little while, say, for example, 269 00:15:28,070 --> 00:15:30,745 supposing it takes me a microsecond to do a cons, and 270 00:15:30,745 --> 00:15:34,570 I have a million cons memory then I'm only going to run out 271 00:15:34,570 --> 00:15:38,000 in a second, and that's pretty bad. 272 00:15:38,000 --> 00:15:41,470 So what we do to prevent that disaster, that ecological 273 00:15:41,470 --> 00:15:44,300 disaster, talk about right after questions. 274 00:15:44,300 --> 00:15:45,550 Are there any questions? 275 00:15:51,500 --> 00:15:52,030 Yes. 276 00:15:52,030 --> 00:15:54,830 AUDIENCE: In the environment diagrams that we were drawing 277 00:15:54,830 --> 00:15:58,630 we would use the body of procedures, and you would 278 00:15:58,630 --> 00:16:02,620 eventually wind up with things that were no longer useful in 279 00:16:02,620 --> 00:16:04,930 that structure. 280 00:16:04,930 --> 00:16:06,890 How is that represented? 281 00:16:06,890 --> 00:16:09,180 PROFESSOR: There's two problems here. 282 00:16:09,180 --> 00:16:13,870 One you were asking is that material becomes useless. 283 00:16:13,870 --> 00:16:14,920 We'll talk about that in a second. 284 00:16:14,920 --> 00:16:18,100 That has to do with how to prevent ecological disasters. 285 00:16:18,100 --> 00:16:20,190 If I make a lot of garbage I have to somehow be able to 286 00:16:20,190 --> 00:16:21,820 clean up after myself. 287 00:16:21,820 --> 00:16:23,430 And we'll talk about that in a second. 288 00:16:23,430 --> 00:16:25,370 The other question you're asking is how you represent 289 00:16:25,370 --> 00:16:27,210 the environments, I think. 290 00:16:27,210 --> 00:16:27,600 AUDIENCE: Yes. 291 00:16:27,600 --> 00:16:28,190 PROFESSOR: OK. 292 00:16:28,190 --> 00:16:29,780 And the environment structures can be represented in 293 00:16:29,780 --> 00:16:30,860 arbitrary ways. 294 00:16:30,860 --> 00:16:31,780 There are lots of them. 295 00:16:31,780 --> 00:16:33,630 I mean, here I'm just telling you about list cells. 296 00:16:33,630 --> 00:16:36,400 Of course every real system has vectors of arbitrary 297 00:16:36,400 --> 00:16:39,500 length as well as the vectors of length, too, which 298 00:16:39,500 --> 00:16:41,080 represent list cells. 299 00:16:41,080 --> 00:16:45,460 And the environment structures that one uses in a 300 00:16:45,460 --> 00:16:49,890 professionally written Lisp system tend to be vectors 301 00:16:49,890 --> 00:16:52,350 which contain a number of elements approximately equal 302 00:16:52,350 --> 00:16:56,090 to the number of arguments-- a little bit more because you 303 00:16:56,090 --> 00:16:58,290 need certain glue. 304 00:16:58,290 --> 00:17:00,360 So remember, the environment [UNINTELLIGIBLE] 305 00:17:00,360 --> 00:17:00,740 frames. 306 00:17:00,740 --> 00:17:03,980 The frames are constructed by applying a procedure. 307 00:17:03,980 --> 00:17:08,849 In doing so, an allocation is made of a place which is the 308 00:17:08,849 --> 00:17:11,270 number of arguments long plus [? unglue ?] 309 00:17:11,270 --> 00:17:13,859 that gets linked into a chain. 310 00:17:13,859 --> 00:17:15,660 It's just like algol at that level. 311 00:17:19,810 --> 00:17:21,060 There any other questions? 312 00:17:23,700 --> 00:17:23,920 OK. 313 00:17:23,920 --> 00:17:26,106 Thank you, and let's take a short break. 314 00:17:26,106 --> 00:17:26,449 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 315 00:17:26,449 --> 00:17:27,699 JOHANN SEBASTIAN BACH] 316 00:18:12,270 --> 00:18:15,840 PROFESSOR: Well, as I just said, computer memories 317 00:18:15,840 --> 00:18:19,420 supplied by the semiconductor manufacturers are finite. 318 00:18:19,420 --> 00:18:21,620 And that's quite a pity. 319 00:18:21,620 --> 00:18:24,030 It might not always be that way. 320 00:18:24,030 --> 00:18:27,990 Just for a quick calculation, you can see that it's possible 321 00:18:27,990 --> 00:18:28,860 that if [? memory ?] 322 00:18:28,860 --> 00:18:32,130 prices keep going at the rate they're going that if you 323 00:18:32,130 --> 00:18:34,950 still took a microsecond second to do a cons, then-- 324 00:18:34,950 --> 00:18:37,120 first of all, everybody should know that there's about pi 325 00:18:37,120 --> 00:18:39,450 times ten to the seventh seconds in a year. 326 00:18:39,450 --> 00:18:42,640 And so that would be ten to the seventh plus ten to the 327 00:18:42,640 --> 00:18:43,940 sixth is ten to the thirteenth. 328 00:18:43,940 --> 00:18:45,870 So there's maybe ten to the fourteenth conses in the life 329 00:18:45,870 --> 00:18:47,520 of a machine. 330 00:18:47,520 --> 00:18:49,900 If there was ten to the fourteenth words of memory on 331 00:18:49,900 --> 00:18:54,020 your machine, you'd never run out. 332 00:18:54,020 --> 00:18:56,310 And that's not completely unreasonable. 333 00:18:56,310 --> 00:18:58,460 Ten to the fourteenth is not a very large number. 334 00:19:03,860 --> 00:19:05,180 I don't think it is. 335 00:19:05,180 --> 00:19:08,700 But then again I like to play with astronomy. 336 00:19:08,700 --> 00:19:11,380 It's at least ten to the eighteenth centimeters between 337 00:19:11,380 --> 00:19:12,930 us and the nearest star. 338 00:19:12,930 --> 00:19:19,620 But the thing I'm about to worry about is, at least in 339 00:19:19,620 --> 00:19:22,130 the current economic state of affairs, ten to the fourteenth 340 00:19:22,130 --> 00:19:24,200 pieces of memory is expensive. 341 00:19:24,200 --> 00:19:27,280 And so I suppose what we have to do is make 342 00:19:27,280 --> 00:19:28,120 do with much smaller. 343 00:19:28,120 --> 00:19:30,170 Memories 344 00:19:30,170 --> 00:19:35,800 Now in general we want to have an illusion of infinity. 345 00:19:35,800 --> 00:19:38,850 All we need to do is arrange it so that whenever you look, 346 00:19:38,850 --> 00:19:40,100 the thing is there. 347 00:19:42,670 --> 00:19:45,105 That's really an important idea. 348 00:19:49,540 --> 00:19:52,470 A person or a computer lives only a finite amount of time 349 00:19:52,470 --> 00:19:55,280 and can only take a finite number of looks at something. 350 00:19:55,280 --> 00:19:58,190 And so you really only need a finite amount of stuff. 351 00:19:58,190 --> 00:20:01,730 But you have to arrange it so no matter how much there is, 352 00:20:01,730 --> 00:20:04,000 how much you really claim there is, there's always 353 00:20:04,000 --> 00:20:06,900 enough stuff so that when you take a look, it's there. 354 00:20:06,900 --> 00:20:08,750 And so you only need a finite amount. 355 00:20:08,750 --> 00:20:11,630 But let's see. 356 00:20:11,630 --> 00:20:14,980 One problem is, as was brought up, that there are possible 357 00:20:14,980 --> 00:20:18,660 ways that there is lots of stuff that we make that we 358 00:20:18,660 --> 00:20:19,410 don't need. 359 00:20:19,410 --> 00:20:20,895 And we could recycle the material out 360 00:20:20,895 --> 00:20:22,760 of which its made. 361 00:20:22,760 --> 00:20:27,820 An example is the fact that we're building environment 362 00:20:27,820 --> 00:20:30,470 structures, and we do so every time we call a procedure. 363 00:20:30,470 --> 00:20:32,810 We have built in it a environment frame. 364 00:20:32,810 --> 00:20:34,840 That environment frame doesn't necessarily 365 00:20:34,840 --> 00:20:36,730 have a very long lifetime. 366 00:20:36,730 --> 00:20:40,330 Its lifetime, meaning its usefulness, may exist only 367 00:20:40,330 --> 00:20:42,850 over the invocation of the procedure. 368 00:20:42,850 --> 00:20:45,860 Or if the procedure exports another procedure by returning 369 00:20:45,860 --> 00:20:48,260 it as a value and that procedure is defined inside of 370 00:20:48,260 --> 00:20:52,210 it, well then the lifetime of the frame of the outer 371 00:20:52,210 --> 00:20:57,070 procedure still is only the lifetime of the procedure 372 00:20:57,070 --> 00:20:58,530 which was exported. 373 00:20:58,530 --> 00:21:01,960 And so ultimately, a lot of that is garbage. 374 00:21:01,960 --> 00:21:05,370 There are other ways of producing garbage as well. 375 00:21:05,370 --> 00:21:07,240 Users produce garbage. 376 00:21:07,240 --> 00:21:10,930 An example of user garbage is something like this. 377 00:21:10,930 --> 00:21:15,220 If we write a program to, for example, append two lists 378 00:21:15,220 --> 00:21:19,890 together, well one way to do it is to reverse the first 379 00:21:19,890 --> 00:21:23,440 list onto the empty list and reverse that onto the second 380 00:21:23,440 --> 00:21:28,160 list. Now that's not terribly bad way of doing it. 381 00:21:28,160 --> 00:21:31,070 And however, the intermediate result, which is the reversal 382 00:21:31,070 --> 00:21:37,300 of the first list as done by this program, is never going 383 00:21:37,300 --> 00:21:39,920 to be accessed ever again after it's copied back on to 384 00:21:39,920 --> 00:21:41,010 the second. 385 00:21:41,010 --> 00:21:43,580 It's an intermediate result. 386 00:21:43,580 --> 00:21:47,230 It's going to be hard to ever see how anybody would ever be 387 00:21:47,230 --> 00:21:48,600 able to access it. 388 00:21:48,600 --> 00:21:51,050 In fact, it will go away. 389 00:21:51,050 --> 00:21:53,190 Now if we make a lot of garbage like that, and we 390 00:21:53,190 --> 00:21:56,210 should be allowed to, then there's got to be some way to 391 00:21:56,210 --> 00:21:58,800 reclaim that garbage. 392 00:21:58,800 --> 00:22:03,050 Well, what I'd like to tell you about now is a very clever 393 00:22:03,050 --> 00:22:09,820 technique whereby a Lisp system can prove a small 394 00:22:09,820 --> 00:22:12,750 theorem every so often on the [? forum, ?] the following 395 00:22:12,750 --> 00:22:17,410 piece of junk will never be accessed again. 396 00:22:17,410 --> 00:22:21,400 It can have no affect on the future of the computation. 397 00:22:21,400 --> 00:22:24,920 It's actually based on a very simple idea. 398 00:22:24,920 --> 00:22:28,570 We've designed our computers to look sort of like this. 399 00:22:28,570 --> 00:22:35,280 There's some data path, which contains the registers. 400 00:22:35,280 --> 00:22:42,610 There are things like x, and env, and val, and so on. 401 00:22:42,610 --> 00:22:47,490 And there's one here called stack, some sort which points 402 00:22:47,490 --> 00:22:50,240 off to a structure somewhere, which is the stack. 403 00:22:50,240 --> 00:22:51,740 And we'll worry about that in a second. 404 00:22:51,740 --> 00:22:55,390 There's some finite controller, finite state 405 00:22:55,390 --> 00:22:56,730 machine controller. 406 00:22:56,730 --> 00:22:59,850 And there's some control signals that go this way and 407 00:22:59,850 --> 00:23:02,270 predicate results that come this way, not 408 00:23:02,270 --> 00:23:04,260 the interesting part. 409 00:23:04,260 --> 00:23:07,140 There's some sort of structured memory, which I 410 00:23:07,140 --> 00:23:10,460 just told you how to make, which may contain a stack. 411 00:23:10,460 --> 00:23:12,690 I didn't tell you how to make things of arbitrary shape, 412 00:23:12,690 --> 00:23:13,450 only pairs. 413 00:23:13,450 --> 00:23:16,280 But in fact with what I've told you can simulate a stack 414 00:23:16,280 --> 00:23:19,140 by a big list. I don't plan to do that, it's not a 415 00:23:19,140 --> 00:23:20,360 nice way to do it. 416 00:23:20,360 --> 00:23:22,990 But we could have something like that. 417 00:23:22,990 --> 00:23:25,990 We have all sorts of little data structures in here that 418 00:23:25,990 --> 00:23:27,470 are hooked together in funny ways. 419 00:23:30,115 --> 00:23:32,560 They connect to other things. 420 00:23:32,560 --> 00:23:33,250 And so on. 421 00:23:33,250 --> 00:23:37,190 And ultimately things up there are pointers to these. 422 00:23:37,190 --> 00:23:40,730 The things that are in the registers are pointers off to 423 00:23:40,730 --> 00:23:44,910 the data structures that live in this Lisp structure memory. 424 00:23:44,910 --> 00:23:52,660 Now the truth of the matter is that the entire consciousness 425 00:23:52,660 --> 00:23:55,550 of this machine is in these registers. 426 00:23:55,550 --> 00:23:59,380 There is no possible way that the machine, if done 427 00:23:59,380 --> 00:24:02,410 correctly, if built correctly, can access anything in this 428 00:24:02,410 --> 00:24:05,580 Lisp structure memory unless the thing in that Lisp 429 00:24:05,580 --> 00:24:10,310 structure memory is connected by a sequence of data 430 00:24:10,310 --> 00:24:15,070 structures to the registers. 431 00:24:15,070 --> 00:24:17,550 If it's accessible by legitimate data structure 432 00:24:17,550 --> 00:24:20,100 selectors from the pointers that are 433 00:24:20,100 --> 00:24:22,280 stored in these registers. 434 00:24:22,280 --> 00:24:24,940 Things like array references, perhaps. 435 00:24:24,940 --> 00:24:28,790 Or cons cell references, cars and cdrs. 436 00:24:28,790 --> 00:24:30,970 But I can't just talk about a random place in this memory, 437 00:24:30,970 --> 00:24:32,740 because I can't get to it. 438 00:24:32,740 --> 00:24:34,665 These are being arbitrary names I'm not allowed to 439 00:24:34,665 --> 00:24:38,985 count, at least as I'm evaluating expressions. 440 00:24:41,620 --> 00:24:44,600 If that's the case then there's a very simple theorem 441 00:24:44,600 --> 00:24:47,160 to be proved. 442 00:24:47,160 --> 00:24:49,730 Which is, if I start with all lead pointers that are in all 443 00:24:49,730 --> 00:24:53,570 these registers and recursively chase out, marking 444 00:24:53,570 --> 00:24:57,980 all the places I can get to by selectors, then eventually I 445 00:24:57,980 --> 00:25:00,750 mark everything they can be gotten to. 446 00:25:00,750 --> 00:25:02,140 Anything which is not so marked is 447 00:25:02,140 --> 00:25:05,560 garbage and can be recycled. 448 00:25:05,560 --> 00:25:07,200 Very simple. 449 00:25:07,200 --> 00:25:11,180 Cannot affect the future of the computation. 450 00:25:11,180 --> 00:25:16,616 So let me show you that in a particular example. 451 00:25:16,616 --> 00:25:19,800 Now that means I'm going to have to append to my 452 00:25:19,800 --> 00:25:23,640 description of the list structure a mark. 453 00:25:23,640 --> 00:25:29,080 And so here, for example, is a Lisp structured memory. 454 00:25:29,080 --> 00:25:31,360 And in this Lisp structured memory is a Lisp structure 455 00:25:31,360 --> 00:25:33,030 beginning in a place I'm going to call-- 456 00:25:35,640 --> 00:25:38,590 this is the root. 457 00:25:38,590 --> 00:25:40,120 Now it doesn't really have to have a root. 458 00:25:40,120 --> 00:25:42,670 It could be a bunch of them, like all the registers. 459 00:25:42,670 --> 00:25:45,270 But I could cleverly arrange it so all the registers, all 460 00:25:45,270 --> 00:25:47,080 the things that are in old registers are also at the 461 00:25:47,080 --> 00:25:50,770 right moment put into this root structure, and then we've 462 00:25:50,770 --> 00:25:51,850 got one pointer to it. 463 00:25:51,850 --> 00:25:54,570 I don't really care. 464 00:25:54,570 --> 00:25:57,290 So the idea is we're going to cons up stuff until our free 465 00:25:57,290 --> 00:25:58,720 list is empty. 466 00:25:58,720 --> 00:26:00,950 We've run out of things. 467 00:26:00,950 --> 00:26:04,560 Now we're going to do this process of proving the theorem 468 00:26:04,560 --> 00:26:07,850 that a certain percentage of the memory has got crap in it. 469 00:26:07,850 --> 00:26:10,320 And then we're going to recycle that to grow new 470 00:26:10,320 --> 00:26:14,570 trees, a standard use of such garbage. 471 00:26:17,090 --> 00:26:18,840 So in any case, what do we have here? 472 00:26:18,840 --> 00:26:21,700 Well we have some data structure which starts out 473 00:26:21,700 --> 00:26:27,502 over here one. 474 00:26:27,502 --> 00:26:33,980 And in fact it has a car in five, and its cdr is in two. 475 00:26:33,980 --> 00:26:36,700 And all the marks start out at zero. 476 00:26:36,700 --> 00:26:39,920 Well let's start marking, just to play this game. 477 00:26:39,920 --> 00:26:42,540 OK. 478 00:26:42,540 --> 00:26:47,090 So for example, since I can access one from the root I 479 00:26:47,090 --> 00:26:48,390 will mark that. 480 00:26:48,390 --> 00:26:50,960 Let me mark it. 481 00:26:50,960 --> 00:26:52,430 Bang. 482 00:26:52,430 --> 00:26:54,560 That's marked. 483 00:26:54,560 --> 00:27:00,020 Now since I have a five here I can go to five and see, well 484 00:27:00,020 --> 00:27:01,450 I'll mark that. 485 00:27:01,450 --> 00:27:01,760 Bang. 486 00:27:01,760 --> 00:27:02,900 That's useful stuff. 487 00:27:02,900 --> 00:27:05,410 But five references as a number in its car, I'm not 488 00:27:05,410 --> 00:27:08,700 interested in marking numbers but its cdr is seven. 489 00:27:08,700 --> 00:27:10,450 So I can mark that. 490 00:27:10,450 --> 00:27:12,260 Bang. 491 00:27:12,260 --> 00:27:16,000 Seven is the empty list, the only thing that references, 492 00:27:16,000 --> 00:27:17,120 and it's got a number in its car. 493 00:27:17,120 --> 00:27:19,490 Not interesting. 494 00:27:19,490 --> 00:27:20,500 Well now let's go back here. 495 00:27:20,500 --> 00:27:21,650 I forgot about something. 496 00:27:21,650 --> 00:27:22,840 Two. 497 00:27:22,840 --> 00:27:25,960 See in other words, if I'm looking at cell one, cell one 498 00:27:25,960 --> 00:27:30,370 contains a two right over here. 499 00:27:30,370 --> 00:27:31,730 A reference to two. 500 00:27:31,730 --> 00:27:35,700 That means I should go mark two. 501 00:27:35,700 --> 00:27:37,140 Bang. 502 00:27:37,140 --> 00:27:38,960 Two contains a reference to four. 503 00:27:38,960 --> 00:27:41,593 It's got a number in its car, I'm not interested in that, so 504 00:27:41,593 --> 00:27:43,780 I'm going to go mark that. 505 00:27:43,780 --> 00:27:47,840 Four refers to seven through its car, and is empty in its 506 00:27:47,840 --> 00:27:49,990 cdr, but I've already marked that one so I don't have to 507 00:27:49,990 --> 00:27:51,400 mark it again. 508 00:27:51,400 --> 00:27:55,000 This is all the accessible structure from that place. 509 00:27:55,000 --> 00:27:58,710 Simple recursive mark algorithm. 510 00:27:58,710 --> 00:28:01,160 Now there are some unhappinesses about that 511 00:28:01,160 --> 00:28:04,920 algorithm, and we can worry about that a second. 512 00:28:04,920 --> 00:28:07,280 But basically you'll see that all the things that have not 513 00:28:07,280 --> 00:28:14,220 been marked are places that are free, and I could recycle. 514 00:28:14,220 --> 00:28:16,210 So the next stage after that is going to be to scan through 515 00:28:16,210 --> 00:28:21,180 all of my memory, looking for things that are not marked. 516 00:28:21,180 --> 00:28:23,370 Every time I come across a marked thing I unmark it, and 517 00:28:23,370 --> 00:28:26,390 every time I come across an unmarked thing I'm going to 518 00:28:26,390 --> 00:28:28,770 link it together in my free list. 519 00:28:28,770 --> 00:28:32,120 Classic, very simple algorithm. 520 00:28:32,120 --> 00:28:33,840 So let's see. 521 00:28:33,840 --> 00:28:34,770 Is that very simple? 522 00:28:34,770 --> 00:28:35,570 Yes it is. 523 00:28:35,570 --> 00:28:38,340 I'm not going to go through the code in any detail, but I 524 00:28:38,340 --> 00:28:40,090 just want to show you about how long it is. 525 00:28:40,090 --> 00:28:42,490 Let's look at the mark phase. 526 00:28:42,490 --> 00:28:45,060 Here's the first part of the mark phase. 527 00:28:45,060 --> 00:28:48,280 We pick up the root. 528 00:28:48,280 --> 00:28:52,380 We're going to use that as a recursive procedure call. 529 00:28:52,380 --> 00:28:55,800 We're going to sweep from there, after when we're done 530 00:28:55,800 --> 00:28:57,380 with marking. 531 00:28:57,380 --> 00:28:59,840 And then we're going to do a little couple of instructions 532 00:28:59,840 --> 00:29:01,920 that do this checking out on the marks and changing the 533 00:29:01,920 --> 00:29:04,000 marks and things like that, according to the algorithm 534 00:29:04,000 --> 00:29:05,500 I've just shown you. 535 00:29:05,500 --> 00:29:06,470 It comes out here. 536 00:29:06,470 --> 00:29:08,800 You have to mark the cars of things and you also have to be 537 00:29:08,800 --> 00:29:10,660 able to mark the cdrs of things. 538 00:29:10,660 --> 00:29:14,370 That's the entire mark phase. 539 00:29:14,370 --> 00:29:16,590 I'll just tell you a little story about this. 540 00:29:16,590 --> 00:29:22,950 The old DEC PDP-6 computer, this was the way that the 541 00:29:22,950 --> 00:29:26,740 mark-sweep garbage collection, as it was, was written. 542 00:29:26,740 --> 00:29:31,940 The program was so small that with the data that it needed, 543 00:29:31,940 --> 00:29:34,310 with the registers that it needed to manipulate the 544 00:29:34,310 --> 00:29:38,070 memory, it fit into the fast registers of the machine, 545 00:29:38,070 --> 00:29:39,280 which were 16. 546 00:29:39,280 --> 00:29:39,800 The whole program. 547 00:29:39,800 --> 00:29:40,700 And you could execute 548 00:29:40,700 --> 00:29:43,170 instructions in the fast registers. 549 00:29:43,170 --> 00:29:46,560 So it's an extremely small program, and it could run very 550 00:29:46,560 --> 00:29:48,870 fast. 551 00:29:48,870 --> 00:29:53,130 Now unfortunately, of course, this program, because the fact 552 00:29:53,130 --> 00:29:57,630 that it's recursive in the way that you do something first 553 00:29:57,630 --> 00:29:59,690 and then you do something after that, you have to work 554 00:29:59,690 --> 00:30:03,410 on the cars and then the cdrs, it requires auxiliary memory. 555 00:30:03,410 --> 00:30:05,680 So Lisp systems-- 556 00:30:05,680 --> 00:30:08,260 those requires a stack for marking. 557 00:30:08,260 --> 00:30:12,440 Lisp systems that are built this way have a limit to the 558 00:30:12,440 --> 00:30:16,060 depth of recursion you can have in data structures in 559 00:30:16,060 --> 00:30:19,930 either the car or the cdr, and that doesn't work very nicely. 560 00:30:19,930 --> 00:30:23,180 On the other hand, you never notice it if it's big enough. 561 00:30:23,180 --> 00:30:27,650 And that's certainly been the case for most Maclisp, for 562 00:30:27,650 --> 00:30:30,760 example, which ran Macsyma where you could deal with 563 00:30:30,760 --> 00:30:33,560 expressions of thousands of elements long. 564 00:30:33,560 --> 00:30:35,490 These are algebraic expressions with thousand of 565 00:30:35,490 --> 00:30:39,490 terms. And there's no problem with that. 566 00:30:39,490 --> 00:30:42,190 Such, the garbage collector does work. 567 00:30:42,190 --> 00:30:44,750 On the other hand, there's a very clever modification to 568 00:30:44,750 --> 00:30:49,020 this algorithm, which I will not describe, by Peter Deutsch 569 00:30:49,020 --> 00:30:50,720 and Schorr and Waite-- 570 00:30:50,720 --> 00:30:55,380 Herb Schorr from IBM and Waite, who I don't know. 571 00:30:55,380 --> 00:30:58,470 That algorithm allows you to build-- you do can do this 572 00:30:58,470 --> 00:31:01,990 without auxiliary memory, by remembering as you walk the 573 00:31:01,990 --> 00:31:04,760 data structures where you came from by reversing the pointers 574 00:31:04,760 --> 00:31:06,650 as you go down and crawling up the reverse 575 00:31:06,650 --> 00:31:07,520 pointers as you go up. 576 00:31:07,520 --> 00:31:09,130 It's a rather tricky algorithm. 577 00:31:09,130 --> 00:31:11,230 The first time you write it-- or in fact, the first three 578 00:31:11,230 --> 00:31:14,350 times you write it it has a terrible bug in it. 579 00:31:14,350 --> 00:31:18,110 And it's also rather slow, because it's complicated. 580 00:31:18,110 --> 00:31:21,150 It takes about six times as many memory references to do 581 00:31:21,150 --> 00:31:24,580 the sorts of things that we're talking about. 582 00:31:24,580 --> 00:31:28,140 Well now once I've done this marking phase, and I get into 583 00:31:28,140 --> 00:31:30,920 a position where things look like this, let's look-- 584 00:31:30,920 --> 00:31:31,510 yes. 585 00:31:31,510 --> 00:31:35,590 Here we have the mark done, just as I did it. 586 00:31:35,590 --> 00:31:37,330 Now we have to perform the sweep phase. 587 00:31:37,330 --> 00:31:39,820 And I described to you what this sweep is like. 588 00:31:39,820 --> 00:31:42,130 I'm going to walk down from one end of memory or the 589 00:31:42,130 --> 00:31:45,690 other, I don't care where, scanning every cell that's in 590 00:31:45,690 --> 00:31:46,836 the memory. 591 00:31:46,836 --> 00:31:51,000 And as I scan these cells, I'm going to link them together, 592 00:31:51,000 --> 00:31:53,890 if they are free, into the free list. And if they're not 593 00:31:53,890 --> 00:31:57,500 free, I'm going to unmark them so the marks become zero. 594 00:31:57,500 --> 00:32:00,050 And in fact what I get-- well the program is not very 595 00:32:00,050 --> 00:32:00,460 complicated. 596 00:32:00,460 --> 00:32:02,780 It looks sort of like this-- it's a little longer. 597 00:32:02,780 --> 00:32:04,820 Here's the first piece of it. 598 00:32:04,820 --> 00:32:06,710 This one's coming down from the top of memory. 599 00:32:06,710 --> 00:32:09,580 I don't want you to try to understand this at this point. 600 00:32:09,580 --> 00:32:11,030 It's rather simple. 601 00:32:11,030 --> 00:32:14,260 It's a very simple algorithm, but there's pieces of it that 602 00:32:14,260 --> 00:32:15,970 just sort of look like this. 603 00:32:15,970 --> 00:32:18,600 They're all sort of obvious. 604 00:32:18,600 --> 00:32:21,060 And after we've done the sweep, we get an answer that 605 00:32:21,060 --> 00:32:22,310 looks like that. 606 00:32:25,330 --> 00:32:27,150 Now there are some disadvantages with mark-sweep 607 00:32:27,150 --> 00:32:29,590 algorithms of this sort. 608 00:32:29,590 --> 00:32:31,940 Serious ones. 609 00:32:31,940 --> 00:32:34,250 One important disadvantage is that your memories get larger 610 00:32:34,250 --> 00:32:36,498 and larger. 611 00:32:36,498 --> 00:32:39,100 As you say, address spaces get larger and larger, you're 612 00:32:39,100 --> 00:32:43,080 willing to represent more and more stuff, then it gets very 613 00:32:43,080 --> 00:32:46,360 costly to scan all of memory. 614 00:32:46,360 --> 00:32:50,490 What you'd really like to do is only scan useful stuff. 615 00:32:50,490 --> 00:32:56,120 It would even be better if you realized that some stuff was 616 00:32:56,120 --> 00:32:59,320 known to be good and useful, and you don't have to look at 617 00:32:59,320 --> 00:33:00,370 it more than once or twice. 618 00:33:00,370 --> 00:33:01,550 Or very rarely. 619 00:33:01,550 --> 00:33:05,120 Whereas other stuff that you're not so sure about, you 620 00:33:05,120 --> 00:33:10,110 can look at more detail every time you want to do this, want 621 00:33:10,110 --> 00:33:11,910 to garbage collect. 622 00:33:11,910 --> 00:33:15,660 Well there are algorithms that are organized in this way. 623 00:33:15,660 --> 00:33:18,850 Let me tell you about a famous old algorithm which allows you 624 00:33:18,850 --> 00:33:20,370 only look at the part of memory which 625 00:33:20,370 --> 00:33:22,800 is known to be useful. 626 00:33:22,800 --> 00:33:24,690 And which happens to be the fastest known garbage 627 00:33:24,690 --> 00:33:26,310 collector algorithm. 628 00:33:26,310 --> 00:33:28,170 This is the Minsky-Feinchel-Yochelson 629 00:33:28,170 --> 00:33:30,150 garbage collector algorithm. 630 00:33:30,150 --> 00:33:36,660 It was invented by Minsky in 1961 or '60 or something, for 631 00:33:36,660 --> 00:33:45,870 the RLE PDP-1 Lisp, which had 4,096 words of list memory, 632 00:33:45,870 --> 00:33:48,480 and a drum. 633 00:33:48,480 --> 00:33:50,890 And the whole idea was to garbage collect 634 00:33:50,890 --> 00:33:53,380 this terrible memory. 635 00:33:53,380 --> 00:33:56,510 What Minsky realized was the easiest way to do this is to 636 00:33:56,510 --> 00:33:59,950 scan the memory in the same sense, walking the good 637 00:33:59,950 --> 00:34:06,350 structure, copying it out into the drum, compacted. 638 00:34:06,350 --> 00:34:09,429 And then when we were done copying it all out, then you 639 00:34:09,429 --> 00:34:12,300 swap that back into your memory. 640 00:34:12,300 --> 00:34:14,260 Now whether or you not use a drum, or another piece of 641 00:34:14,260 --> 00:34:17,030 memory, or something like that isn't important. 642 00:34:17,030 --> 00:34:18,260 In fact, I don't think people use 643 00:34:18,260 --> 00:34:20,350 drums anymore for anything. 644 00:34:20,350 --> 00:34:25,920 But this algorithm basically depends upon having about 645 00:34:25,920 --> 00:34:30,270 twice as much address space as you're actually using. 646 00:34:30,270 --> 00:34:35,370 And so what you have is some, initially, some mixture of 647 00:34:35,370 --> 00:34:37,110 useful data and garbage. 648 00:34:37,110 --> 00:34:38,560 So this is called fromspace. 649 00:34:45,179 --> 00:34:47,800 And this is a mixture of crud. 650 00:34:47,800 --> 00:34:52,000 Some of it's important and some of it isn't. 651 00:34:52,000 --> 00:34:55,770 Now there's another place which is hopefully big enough, 652 00:34:55,770 --> 00:34:58,240 if we recall, tospace, which is where we're copying to. 653 00:35:01,590 --> 00:35:03,260 And what happens is-- and I'm not going to go 654 00:35:03,260 --> 00:35:04,970 through this detail. 655 00:35:04,970 --> 00:35:07,590 It's in our book quite explicitly. 656 00:35:07,590 --> 00:35:11,030 There's a root point where you start from. 657 00:35:11,030 --> 00:35:14,600 And the idea is that you start with the root. 658 00:35:14,600 --> 00:35:18,610 You copy the first thing you see, the first thing that the 659 00:35:18,610 --> 00:35:22,810 root points at, to the beginning of tospace. 660 00:35:22,810 --> 00:35:24,790 The first thing is a pair or something 661 00:35:24,790 --> 00:35:27,560 like, a data structure. 662 00:35:27,560 --> 00:35:32,330 You then also leave behind a broken heart saying, I moved 663 00:35:32,330 --> 00:35:36,330 this object from here to here, giving the place 664 00:35:36,330 --> 00:35:37,800 where it moved to. 665 00:35:37,800 --> 00:35:40,270 This is called a broken heart because a friend of mine who 666 00:35:40,270 --> 00:35:44,760 implemented one of these in 1966 was a very romantic 667 00:35:44,760 --> 00:35:46,760 character and called it a broken heart. 668 00:35:49,580 --> 00:35:53,570 But in any case, the next thing you do is now you have a 669 00:35:53,570 --> 00:35:57,840 new free pointer which is here, and you start scanning. 670 00:35:57,840 --> 00:36:00,235 You scan this data structure you just copied. 671 00:36:00,235 --> 00:36:02,710 And every time you encounter a pointer in it, you treat it as 672 00:36:02,710 --> 00:36:04,000 if it was the root pointer here. 673 00:36:04,000 --> 00:36:05,170 Oh, I'm sorry. 674 00:36:05,170 --> 00:36:06,330 The other thing you do is you now move the 675 00:36:06,330 --> 00:36:09,220 root pointer to there. 676 00:36:09,220 --> 00:36:11,310 So now you scan this, and everything you see you treat 677 00:36:11,310 --> 00:36:14,110 as it were the root pointer. 678 00:36:14,110 --> 00:36:16,360 So if you see something, well it points 679 00:36:16,360 --> 00:36:18,510 up into there somewhere. 680 00:36:18,510 --> 00:36:21,780 Is it pointing at a thing which you've not copied yet? 681 00:36:21,780 --> 00:36:23,880 Is there a broken heart there? 682 00:36:23,880 --> 00:36:25,370 If there's a broken heart there and it's something you 683 00:36:25,370 --> 00:36:27,640 have copied, you've just replaced this pointer with the 684 00:36:27,640 --> 00:36:30,620 thing a broken heart points at. 685 00:36:30,620 --> 00:36:33,030 If this thing has not been copied, you copy it to the 686 00:36:33,030 --> 00:36:34,430 next place over here. 687 00:36:34,430 --> 00:36:39,860 Move your free pointer over here, and then leave a broken 688 00:36:39,860 --> 00:36:43,670 heart behind and scan. 689 00:36:43,670 --> 00:36:46,840 And eventually when the scant pointer hits the free pointer, 690 00:36:46,840 --> 00:36:50,140 everything in memory has been copied. 691 00:36:50,140 --> 00:36:52,170 And then there's a whole bunch of empty space up here, which 692 00:36:52,170 --> 00:36:53,820 you could either make into a free list, if that's what you 693 00:36:53,820 --> 00:36:54,470 want to do. 694 00:36:54,470 --> 00:36:56,270 But generally you don't in this kind of system. 695 00:36:56,270 --> 00:36:57,470 In this system you sequentially 696 00:36:57,470 --> 00:37:00,910 allocate your memory. 697 00:37:00,910 --> 00:37:03,820 That is a very, very nice algorithm, and sort of the one 698 00:37:03,820 --> 00:37:06,790 we use in the scheme that you've been using. 699 00:37:06,790 --> 00:37:09,490 And it's expected-- 700 00:37:09,490 --> 00:37:12,400 I believe no one has found a faster algorithm than that. 701 00:37:12,400 --> 00:37:14,150 There are very simple modifications to this 702 00:37:14,150 --> 00:37:19,060 algorithm invented by Henry Baker which allow one to run 703 00:37:19,060 --> 00:37:21,210 this algorithm in real time, meaning you don't have to stop 704 00:37:21,210 --> 00:37:22,010 to garbage collect. 705 00:37:22,010 --> 00:37:25,410 But you could interleave the consing that the machine does 706 00:37:25,410 --> 00:37:27,870 when its running with steps of the garbage collection 707 00:37:27,870 --> 00:37:31,370 process, so that the garbage collector's distributed, and 708 00:37:31,370 --> 00:37:32,980 the machine doesn't have to stop, and garbage 709 00:37:32,980 --> 00:37:34,640 collecting can start. 710 00:37:34,640 --> 00:37:37,520 Of course in the case of machines with virtual memory 711 00:37:37,520 --> 00:37:41,760 where a lot of it is in inaccessible places, this 712 00:37:41,760 --> 00:37:44,460 becomes a very expensive process. 713 00:37:44,460 --> 00:37:47,690 And there have been numerous attempts to 714 00:37:47,690 --> 00:37:49,190 make this much better. 715 00:37:49,190 --> 00:37:52,080 There is a nice paper, for those of you who are 716 00:37:52,080 --> 00:37:56,210 interested, by Moon and other people which describes a 717 00:37:56,210 --> 00:37:57,940 modification to the incremental 718 00:37:57,940 --> 00:38:00,290 Minsky-Feinchel-Yochelson algorithm, and modification 719 00:38:00,290 --> 00:38:05,980 the Baker algorithm which is more efficient for virtual 720 00:38:05,980 --> 00:38:08,340 memory systems. 721 00:38:08,340 --> 00:38:12,840 Well I think now the mystery to this is sort of gone. 722 00:38:12,840 --> 00:38:14,090 And I'd like to see if there are any questions. 723 00:38:19,780 --> 00:38:20,810 Yes. 724 00:38:20,810 --> 00:38:24,100 AUDIENCE: I saw one of you run the garbage collector on the 725 00:38:24,100 --> 00:38:27,640 systems upstairs, and it seemed to me to run extremely 726 00:38:27,640 --> 00:38:30,190 fast. Did the whole thing take-- 727 00:38:30,190 --> 00:38:31,880 does it sweep through all of memory? 728 00:38:31,880 --> 00:38:32,510 PROFESSOR: No. 729 00:38:32,510 --> 00:38:34,480 It swept through exactly what was needed to 730 00:38:34,480 --> 00:38:37,320 copy the useful structure. 731 00:38:37,320 --> 00:38:40,030 It's a copying collector. 732 00:38:40,030 --> 00:38:45,090 And it is very fast. On the whole, I suppose to copy-- 733 00:38:45,090 --> 00:38:47,000 in a Bobcat-- 734 00:38:47,000 --> 00:38:52,450 to copy, I think, a three megabyte thing or something is 735 00:38:52,450 --> 00:38:56,800 less than a second, real time. 736 00:38:56,800 --> 00:38:59,200 Really, these are very small programs. One thing you should 737 00:38:59,200 --> 00:39:05,400 realise is that garbage collectors have to be small. 738 00:39:05,400 --> 00:39:08,770 Not because they have to be fast, but because no one can 739 00:39:08,770 --> 00:39:11,340 debug a complicated garbage collector. 740 00:39:11,340 --> 00:39:15,000 A garbage collector, if it doesn't work, will trash your 741 00:39:15,000 --> 00:39:16,940 memory in such a way that you cannot figure out what the 742 00:39:16,940 --> 00:39:18,350 hell happened. 743 00:39:18,350 --> 00:39:20,660 You need an audit trail. 744 00:39:20,660 --> 00:39:22,460 Because it rearranges everything, and how do you 745 00:39:22,460 --> 00:39:23,740 know what happened there? 746 00:39:23,740 --> 00:39:27,480 So this is the only kind of program that it really, 747 00:39:27,480 --> 00:39:30,100 seriously matters if you stare at it long enough so you 748 00:39:30,100 --> 00:39:31,970 believe that it works. 749 00:39:31,970 --> 00:39:35,100 And sort of prove it to yourself. 750 00:39:35,100 --> 00:39:36,940 So there's no way to debug it. 751 00:39:36,940 --> 00:39:39,230 And that takes it being small enough so you can 752 00:39:39,230 --> 00:39:41,690 hold it in your head. 753 00:39:41,690 --> 00:39:45,020 Garbage collectors are special in this way. 754 00:39:45,020 --> 00:39:47,130 So every reasonable garbage collector has gotten small, 755 00:39:47,130 --> 00:39:52,430 and generally small programs are fast. Yes. 756 00:39:52,430 --> 00:39:53,650 AUDIENCE: Can you repeat the name of this 757 00:39:53,650 --> 00:39:54,510 technique once again? 758 00:39:54,510 --> 00:39:56,220 PROFESSOR: That's the Minsky-Feinchel-Yochelson 759 00:39:56,220 --> 00:39:58,420 garbage collector. 760 00:39:58,420 --> 00:39:59,340 AUDIENCE: You got that? 761 00:39:59,340 --> 00:40:02,210 PROFESSOR: Minsky invented it in '61 for the RLE PDP-1. 762 00:40:02,210 --> 00:40:07,410 A version of it was developed and elaborated to be used in 763 00:40:07,410 --> 00:40:12,410 Multics Maclisp by Feinchel and Yochelson in somewhere 764 00:40:12,410 --> 00:40:19,570 around 1968 or '69. 765 00:40:19,570 --> 00:40:20,650 OK. 766 00:40:20,650 --> 00:40:22,640 Let's take a break. 767 00:40:22,640 --> 00:40:22,934 [MUSIC: "JESU, JOY OF MAN'S DESIRING" BY 768 00:40:22,934 --> 00:40:24,184 JOHANN SEBASTIAN BACH] 769 00:41:17,310 --> 00:41:21,600 PROFESSOR: Well we've come to the end of this subject, and 770 00:41:21,600 --> 00:41:24,860 we've already shown you a universal machine which is 771 00:41:24,860 --> 00:41:26,740 down to evaluator. 772 00:41:26,740 --> 00:41:28,980 It's down to the level of detail you could imagine you 773 00:41:28,980 --> 00:41:30,420 could make one. 774 00:41:30,420 --> 00:41:34,390 This is a particular implementation of Lisp, built 775 00:41:34,390 --> 00:41:37,530 on one of those scheme chips that was talked about 776 00:41:37,530 --> 00:41:39,180 yesterday, sitting over here. 777 00:41:39,180 --> 00:41:42,990 This is mostly interface to somebody's memory with a 778 00:41:42,990 --> 00:41:45,010 little bit of timing and other such stuff. 779 00:41:45,010 --> 00:41:48,760 But this fellow actually ran Lisp at a fairly reasonable 780 00:41:48,760 --> 00:41:50,610 rate, as interpretive. 781 00:41:50,610 --> 00:41:56,500 It ran Lisp as fast as a DEC PDP-10 back in 1979. 782 00:41:56,500 --> 00:41:59,870 And so it's gotten pretty hardware. 783 00:41:59,870 --> 00:42:02,470 Pretty concrete. 784 00:42:02,470 --> 00:42:05,000 We've also downed you a bit with the 785 00:42:05,000 --> 00:42:07,370 things you can compute. 786 00:42:07,370 --> 00:42:11,850 But is it the case that there are things we can't compute? 787 00:42:11,850 --> 00:42:14,690 And so I'd like to end this with showing you some things 788 00:42:14,690 --> 00:42:18,190 that you'd like be able to compute that you can't. 789 00:42:18,190 --> 00:42:22,720 The answer is yes, there are things you can't compute. 790 00:42:22,720 --> 00:42:28,200 For example, something you'd really like is-- 791 00:42:28,200 --> 00:42:30,210 if you're writing [UNINTELLIGIBLE], you'd like a 792 00:42:30,210 --> 00:42:32,800 program that would check that the thing you're 793 00:42:32,800 --> 00:42:34,630 going to do will work. 794 00:42:34,630 --> 00:42:36,080 Wouldn't that be nice? 795 00:42:36,080 --> 00:42:37,960 You'd like something that would catch infinite loops, 796 00:42:37,960 --> 00:42:43,190 for example, in programs that were written by users. 797 00:42:43,190 --> 00:42:45,890 But in general you can't write such a program that will read 798 00:42:45,890 --> 00:42:48,760 any program and determine whether or not it's an 799 00:42:48,760 --> 00:42:50,990 infinite loop. 800 00:42:50,990 --> 00:42:51,685 Let me show you that. 801 00:42:51,685 --> 00:42:53,340 It's a little bit of a minor mathematics. 802 00:42:58,780 --> 00:43:01,360 Let's imagine that we just had a mathematical 803 00:43:01,360 --> 00:43:02,620 function before we start. 804 00:43:02,620 --> 00:43:12,980 And there is one, called s, which takes a procedure and 805 00:43:12,980 --> 00:43:14,230 its argument, a. 806 00:43:19,320 --> 00:43:24,250 And what s does is it determines whether or not it's 807 00:43:24,250 --> 00:43:26,632 safe to run p on a. 808 00:43:26,632 --> 00:43:34,500 And what I mean by that is this: it's true if p applied 809 00:43:34,500 --> 00:43:45,330 to a will converge to a value without an error. 810 00:43:52,365 --> 00:44:06,890 And it's false if p of a loops forever or makes an error. 811 00:44:15,000 --> 00:44:18,780 Now that's surely a function. 812 00:44:18,780 --> 00:44:21,830 There is some for every procedure and for every 813 00:44:21,830 --> 00:44:25,900 argument you could give it that is either true or false 814 00:44:25,900 --> 00:44:28,440 that it converges without making an error. 815 00:44:28,440 --> 00:44:31,770 And you could make a giant table of them. 816 00:44:31,770 --> 00:44:34,710 But the question is, can you write a procedure that compute 817 00:44:34,710 --> 00:44:37,430 the values of this function? 818 00:44:37,430 --> 00:44:39,720 Well let's assume that we can. 819 00:44:39,720 --> 00:44:58,740 Suppose that we have a procedure called "safe" that 820 00:44:58,740 --> 00:44:59,990 computes the value of s. 821 00:45:12,170 --> 00:45:17,620 Now I'm going to show you by several methods that 822 00:45:17,620 --> 00:45:19,760 you can't do this. 823 00:45:19,760 --> 00:45:22,300 The easiest one, or the first one, let's define a procedure 824 00:45:22,300 --> 00:45:23,810 called diag1. 825 00:45:23,810 --> 00:45:38,250 Given that we have safe, we can define diag1 to be the 826 00:45:38,250 --> 00:45:43,430 procedure of one argument, p, which has the following 827 00:45:43,430 --> 00:45:44,780 properties. 828 00:45:44,780 --> 00:45:54,620 If if it's safe to apply p to itself, then I wish to have an 829 00:45:54,620 --> 00:45:55,870 infinite loop. 830 00:45:59,330 --> 00:46:00,715 Otherwise I'm going to return 3. 831 00:46:03,680 --> 00:46:04,470 Remember it was 42. 832 00:46:04,470 --> 00:46:07,060 What's the answer to the big question? 833 00:46:07,060 --> 00:46:08,525 Where of course we know what an infinite loop is. 834 00:46:12,050 --> 00:46:16,130 Infinite loop, to be a procedure of no arguments, 835 00:46:16,130 --> 00:46:18,430 which is that nice lambda calculus loop. 836 00:46:18,430 --> 00:46:24,680 Lambda of x, x of x, applied to lambda of x, x of x. 837 00:46:24,680 --> 00:46:26,550 So there's nothing left to the imagination here. 838 00:46:29,830 --> 00:46:32,500 Well let's see what the story is. 839 00:46:32,500 --> 00:46:38,100 I'm supposing it's the case that we worry about the 840 00:46:38,100 --> 00:46:43,180 procedure called diag1 applied to diag1. 841 00:46:45,860 --> 00:46:49,970 Well what could it possibly be? 842 00:46:49,970 --> 00:46:51,390 Well I don't know. 843 00:46:51,390 --> 00:46:57,310 We're going to substitute diag1 for p in the body here. 844 00:46:57,310 --> 00:47:00,220 Well is it safe to compute diag1 of diag1? 845 00:47:00,220 --> 00:47:00,780 I don't know. 846 00:47:00,780 --> 00:47:03,400 There are two possibilities. 847 00:47:03,400 --> 00:47:06,260 If it's safe to compute diag1 of diag1 that means it 848 00:47:06,260 --> 00:47:08,490 shouldn't loop. 849 00:47:08,490 --> 00:47:09,540 That means I go to here, but then I 850 00:47:09,540 --> 00:47:10,560 produce an infinite loop. 851 00:47:10,560 --> 00:47:12,210 So it can't be safe. 852 00:47:12,210 --> 00:47:15,055 But if it's not safe to compute diag1 of diag1 then 853 00:47:15,055 --> 00:47:16,020 the answer to this is 3. 854 00:47:16,020 --> 00:47:20,530 But that's diag1 of diag1, so it had to be safe. 855 00:47:20,530 --> 00:47:27,470 So therefore by contradiction you cannot produce safe. 856 00:47:27,470 --> 00:47:30,565 For those of you who were boggled by that one I'm going 857 00:47:30,565 --> 00:47:32,820 to say it again, in a different way. 858 00:47:32,820 --> 00:47:35,530 Listen to one more alternative. 859 00:47:35,530 --> 00:47:36,780 Let's define diag2. 860 00:47:39,840 --> 00:47:45,260 These are named diag because of Cantor's diagonal argument. 861 00:47:45,260 --> 00:47:48,180 These are instances of a famous argument which was 862 00:47:48,180 --> 00:47:52,070 originally used by Cantor in the late part of the last 863 00:47:52,070 --> 00:47:56,420 century to prove that the real numbers were not countable, 864 00:47:56,420 --> 00:47:58,160 that there are too many real numbers to 865 00:47:58,160 --> 00:48:00,190 be counted by integers. 866 00:48:00,190 --> 00:48:02,710 That there are more points on a line, for example, than 867 00:48:02,710 --> 00:48:05,260 there are counting numbers. 868 00:48:05,260 --> 00:48:07,190 It may or may not be obvious, and I don't want to 869 00:48:07,190 --> 00:48:08,440 get into that now. 870 00:48:10,900 --> 00:48:15,820 But diag2 is again a procedure of one argument p. 871 00:48:15,820 --> 00:48:19,220 It's almost the same as the previous one, which is, if 872 00:48:19,220 --> 00:48:26,445 it's safe to compute p on p, then I'm going to produce-- 873 00:48:29,310 --> 00:48:34,010 then I want to compute some other things 874 00:48:34,010 --> 00:48:38,960 other than p of p. 875 00:48:38,960 --> 00:48:40,210 Otherwise I'm going to put out false. 876 00:48:43,600 --> 00:48:46,510 Where other then it says, whatever p of p, I'm going to 877 00:48:46,510 --> 00:48:48,880 put out something else. 878 00:48:48,880 --> 00:48:51,860 I can give you an example of a definition of other than which 879 00:48:51,860 --> 00:48:53,890 I think works. 880 00:48:53,890 --> 00:48:55,640 Let's see. 881 00:48:55,640 --> 00:48:56,330 Yes. 882 00:48:56,330 --> 00:49:06,580 Where other than be a procedure of one argument x 883 00:49:06,580 --> 00:49:14,090 which says, if its eq x to, say, quote a, then the answer 884 00:49:14,090 --> 00:49:15,720 is quote b. 885 00:49:15,720 --> 00:49:16,970 Otherwise it's quote a. 886 00:49:20,090 --> 00:49:22,770 That always produces something which is not what 887 00:49:22,770 --> 00:49:25,350 its argument is. 888 00:49:25,350 --> 00:49:26,540 That's all it is. 889 00:49:26,540 --> 00:49:28,250 That's all I wanted. 890 00:49:28,250 --> 00:49:30,640 Well now let's consider this one, diag2 of diag2. 891 00:49:38,220 --> 00:49:39,560 Well look. 892 00:49:39,560 --> 00:49:42,950 This only does something dangerous, like calling p of 893 00:49:42,950 --> 00:49:47,470 p, if it's safe to do so. 894 00:49:47,470 --> 00:49:51,590 So if safe defined at all, if you can define such a 895 00:49:51,590 --> 00:49:55,680 procedure, safe, then this procedure is always defined 896 00:49:55,680 --> 00:49:57,225 and therefore safe on any inputs. 897 00:50:01,540 --> 00:50:11,770 So diag2 of diag2 must reduce to other than diag2 of diag2. 898 00:50:15,496 --> 00:50:20,020 And that doesn't make sense, so we have a contradiction, 899 00:50:20,020 --> 00:50:22,950 and therefore we can't define safe. 900 00:50:22,950 --> 00:50:27,210 I just waned to do that twice, slightly differently, so you 901 00:50:27,210 --> 00:50:32,260 wouldn't feel that the first one was a trick. 902 00:50:32,260 --> 00:50:34,275 They may be both tricks, but they're at 903 00:50:34,275 --> 00:50:37,300 least slightly different. 904 00:50:37,300 --> 00:50:40,080 So I suppose that pretty much wraps it up. 905 00:50:40,080 --> 00:50:43,540 I've just proved what we call the halting theorem, and I 906 00:50:43,540 --> 00:50:46,720 suppose with that we're going to halt. 907 00:50:46,720 --> 00:50:47,970 I hope you have a good time. 908 00:50:50,900 --> 00:50:53,300 Are there any questions? 909 00:50:53,300 --> 00:50:53,810 Yes. 910 00:50:53,810 --> 00:50:56,940 AUDIENCE: What is the value of s of diag1? 911 00:50:56,940 --> 00:50:57,430 PROFESSOR: Of what? 912 00:50:57,430 --> 00:51:00,120 AUDIENCE: S of diag1. 913 00:51:00,120 --> 00:51:02,340 If you said s is a function and we can 914 00:51:02,340 --> 00:51:02,620 [INTERPOSING VOICES] 915 00:51:02,620 --> 00:51:03,870 PROFESSOR: Oh, I don't know. 916 00:51:03,870 --> 00:51:04,350 I don't know. 917 00:51:04,350 --> 00:51:06,850 It's a function, but I don't know how to compute it. 918 00:51:06,850 --> 00:51:08,610 I can't do it. 919 00:51:08,610 --> 00:51:11,530 I'm just a machine, too. 920 00:51:11,530 --> 00:51:12,210 Right? 921 00:51:12,210 --> 00:51:14,670 There's no machine that in principle-- 922 00:51:14,670 --> 00:51:16,690 it might be that in that particular case you just 923 00:51:16,690 --> 00:51:18,580 asked, with some thinking I could figure it out. 924 00:51:18,580 --> 00:51:21,670 But in general I can't compute the value of s any better than 925 00:51:21,670 --> 00:51:23,780 any other machine can. 926 00:51:23,780 --> 00:51:27,210 There is such a function, it's just that no machine can be 927 00:51:27,210 --> 00:51:29,580 built to compute it. 928 00:51:29,580 --> 00:51:32,980 Now there's a way of saying that that should not be 929 00:51:32,980 --> 00:51:35,350 surprising. 930 00:51:35,350 --> 00:51:36,390 Going through this-- 931 00:51:36,390 --> 00:51:41,020 I mean, I don't have time to do this here, but the number 932 00:51:41,020 --> 00:51:44,600 of functions is very large. 933 00:51:44,600 --> 00:51:48,210 If there's a certain number of answers possible and a certain 934 00:51:48,210 --> 00:51:50,520 number of inputs possible, then it's the number of 935 00:51:50,520 --> 00:51:52,290 answers raised to the number inputs is the number of 936 00:51:52,290 --> 00:51:54,720 possible functions. 937 00:51:54,720 --> 00:51:55,970 On one variable. 938 00:51:58,150 --> 00:52:03,690 Now that's always bigger than the thing you're raising to, 939 00:52:03,690 --> 00:52:05,480 the exponent. 940 00:52:05,480 --> 00:52:12,150 The number of functions is larger than the number of 941 00:52:12,150 --> 00:52:15,340 programs that one can write, by an 942 00:52:15,340 --> 00:52:17,840 infinity counting argument. 943 00:52:17,840 --> 00:52:19,475 And it's much larger. 944 00:52:19,475 --> 00:52:22,540 So there must be a lot of functions that can't be 945 00:52:22,540 --> 00:52:26,280 computed by programs. 946 00:52:26,280 --> 00:52:27,300 AUDIENCE: A few moments ago you were talking about 947 00:52:27,300 --> 00:52:30,640 specifications and automatic generation of solutions. 948 00:52:30,640 --> 00:52:33,360 Do you see any steps between specifications and solutions? 949 00:52:37,250 --> 00:52:38,720 PROFESSOR: Steps between. 950 00:52:38,720 --> 00:52:42,720 You mean, you're saying, how you go about constructing 951 00:52:42,720 --> 00:52:45,205 devices given that have specifications for the device? 952 00:52:45,205 --> 00:52:45,500 Sure. 953 00:52:45,500 --> 00:52:48,540 AUDIENCE: There's a lot of software engineering that goes 954 00:52:48,540 --> 00:52:51,703 through specifications through many layers of design and then 955 00:52:51,703 --> 00:52:52,430 implementation. 956 00:52:52,430 --> 00:52:52,850 PROFESSOR: Yes? 957 00:52:52,850 --> 00:52:55,600 AUDIENCE: I was curious if you think that's realistic. 958 00:52:55,600 --> 00:52:57,210 PROFESSOR: Well I think that some of it's realistic and 959 00:52:57,210 --> 00:52:58,100 some of it isn't. 960 00:52:58,100 --> 00:53:01,680 I mean, surely if I want to build an electrical filter and 961 00:53:01,680 --> 00:53:07,160 I have a rather interesting possibility. 962 00:53:07,160 --> 00:53:12,180 Supposing I want to build a thing that matches some power 963 00:53:12,180 --> 00:53:19,906 output to the radio transmitter, to some antenna. 964 00:53:19,906 --> 00:53:21,970 And I'm really out of this power-- 965 00:53:21,970 --> 00:53:23,230 it's output tube out here. 966 00:53:23,230 --> 00:53:25,920 And the problem is that they have different impedances. 967 00:53:25,920 --> 00:53:27,550 I want them to match the impedances. 968 00:53:27,550 --> 00:53:29,920 I also want to make a filter in there which is going to get 969 00:53:29,920 --> 00:53:32,780 rid of some harmonic radiation. 970 00:53:32,780 --> 00:53:36,270 Well one old-fashioned technique for doing this is 971 00:53:36,270 --> 00:53:38,860 called image impedances, or something like that. 972 00:53:38,860 --> 00:53:40,240 And what you do is you say you have a basic 973 00:53:40,240 --> 00:53:43,300 module called an L-section. 974 00:53:43,300 --> 00:53:44,550 Looks like this. 975 00:53:47,080 --> 00:53:50,470 If I happen to connect this to some resistance, r, and if I 976 00:53:50,470 --> 00:53:55,150 make this impedance x, xl, and if it happens to be q times r, 977 00:53:55,150 --> 00:53:59,710 then this produces a low pass filter with a q square plus 978 00:53:59,710 --> 00:54:02,110 one impedance match. 979 00:54:02,110 --> 00:54:03,120 Just what I need. 980 00:54:03,120 --> 00:54:04,610 Because now I can take two of these, hook them 981 00:54:04,610 --> 00:54:06,510 together like this. 982 00:54:11,660 --> 00:54:16,570 OK, and I take another one and I'll hook them 983 00:54:16,570 --> 00:54:18,290 together like that. 984 00:54:18,290 --> 00:54:20,320 And I have two L-sections hooked together. 985 00:54:20,320 --> 00:54:22,460 And this will step the impedance down to one that I 986 00:54:22,460 --> 00:54:25,530 know, and this will step it up to one I know. 987 00:54:25,530 --> 00:54:26,710 Each of these is a low pass filter 988 00:54:26,710 --> 00:54:28,090 getting rid of some harmonics. 989 00:54:28,090 --> 00:54:30,270 It's good filter, it's called a pie-section filter. 990 00:54:30,270 --> 00:54:31,700 Great. 991 00:54:31,700 --> 00:54:34,300 Except for the fact that in doing what I just did, I've 992 00:54:34,300 --> 00:54:38,620 made a terrible inefficiency in this system. 993 00:54:38,620 --> 00:54:41,620 I've made two coils where I should have made one. 994 00:54:41,620 --> 00:54:45,200 And the problem with most software engineering art is 995 00:54:45,200 --> 00:54:47,440 that there's no mechanism, other than peephole 996 00:54:47,440 --> 00:54:50,280 optimization and compilers, for getting rid of the 997 00:54:50,280 --> 00:54:52,810 redundant parts that are constructed when doing top 998 00:54:52,810 --> 00:54:55,350 down design. 999 00:54:55,350 --> 00:54:57,160 It's even worse, there are lots of very important 1000 00:54:57,160 --> 00:55:01,110 structures that you can't construct at all this way. 1001 00:55:01,110 --> 00:55:03,940 So I think that the standard top down design is a rather 1002 00:55:03,940 --> 00:55:05,710 shallow business. 1003 00:55:05,710 --> 00:55:08,315 Doesn't really capture what people want to do in design. 1004 00:55:08,315 --> 00:55:10,100 I'll give you another electrical example. 1005 00:55:10,100 --> 00:55:12,140 Electrical examples are so much clearer than 1006 00:55:12,140 --> 00:55:14,440 computational examples, because computation examples 1007 00:55:14,440 --> 00:55:17,220 require a certain degree of complexity to explain them. 1008 00:55:17,220 --> 00:55:19,650 But one of my favorite examples in the electrical 1009 00:55:19,650 --> 00:55:23,330 world is how would I ever come up with the output stage of 1010 00:55:23,330 --> 00:55:27,530 this inter-stage connection in an IF amplifier. 1011 00:55:27,530 --> 00:55:32,410 It's a little transistor here, and let's see. 1012 00:55:32,410 --> 00:55:37,560 Well I'm going to have a tank, and I'm going to hook this up 1013 00:55:37,560 --> 00:55:43,040 to, say, I'm going to link-couple that to the input 1014 00:55:43,040 --> 00:55:44,850 of the next stage. 1015 00:55:44,850 --> 00:55:48,580 Here's a perfectly plausible plan-- 1016 00:55:48,580 --> 00:55:51,070 well except for the fact that since I put that going up I 1017 00:55:51,070 --> 00:55:53,170 should make that going that way. 1018 00:55:53,170 --> 00:55:56,050 Here's a perfectly plausible plan for a-- 1019 00:55:56,050 --> 00:55:57,270 no I shouldn't. 1020 00:55:57,270 --> 00:55:57,940 I'm dumb. 1021 00:55:57,940 --> 00:55:59,690 Excuse me. 1022 00:55:59,690 --> 00:56:00,730 Doesn't matter. 1023 00:56:00,730 --> 00:56:01,540 The point is [UNINTELLIGIBLE] 1024 00:56:01,540 --> 00:56:02,560 plan for a couple [UNINTELLIGIBLE] 1025 00:56:02,560 --> 00:56:04,590 stages together. 1026 00:56:04,590 --> 00:56:07,620 Now what the problem is is what's this hierarchically? 1027 00:56:07,620 --> 00:56:09,480 It's not one thing. 1028 00:56:09,480 --> 00:56:11,990 Hierarchically it doesn't make any sense at all. 1029 00:56:11,990 --> 00:56:17,230 It's the inductance of a tuned circuit, it's the primary of a 1030 00:56:17,230 --> 00:56:22,350 transformer, and it's also the DC path by which bias 1031 00:56:22,350 --> 00:56:26,460 conditions get to the collector of that transistor. 1032 00:56:26,460 --> 00:56:29,170 And there's no simple top-down design that's going to produce 1033 00:56:29,170 --> 00:56:33,400 a structure like that with so many overlapping uses for a 1034 00:56:33,400 --> 00:56:34,530 particular thing. 1035 00:56:34,530 --> 00:56:39,015 Playing Scrabble, where you have to do triple word scores, 1036 00:56:39,015 --> 00:56:44,950 or whatever, is not so easy in top-down design strategy. 1037 00:56:44,950 --> 00:56:48,100 Yet most of real engineering is based on getting the most 1038 00:56:48,100 --> 00:56:52,140 oomph for effort. 1039 00:56:52,140 --> 00:56:54,860 And that's what you're seeing here. 1040 00:56:54,860 --> 00:56:55,550 Yeah? 1041 00:56:55,550 --> 00:56:56,810 AUDIENCE: Is this the last question? 1042 00:57:00,282 --> 00:57:18,640 [LAUGHTER] 1043 00:57:18,640 --> 00:57:19,890 PROFESSOR: Apparently so. 1044 00:57:23,240 --> 00:57:26,092 Thank you. 1045 00:57:26,092 --> 00:57:39,040 [APPLAUSE] 1046 00:57:39,040 --> 00:57:39,383 [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY 1047 00:57:39,383 --> 00:57:40,633 JOHANN SEBASTIAN BACH]