1 00:00:13 --> 00:00:20 And this is going to use some of the techniques we learned 2 00:00:20 --> 00:00:25 last time with respect to amortized analysis. 3 00:00:25 --> 00:00:32 And, what's neat about what we're going to talk about today 4 00:00:32 --> 00:00:39 is it's a way of comparing algorithms that are so-called 5 00:00:39 --> 00:00:48 online algorithms. And we're going to introduce 6 00:00:48 --> 00:01:00 this notion with a problem which is called self organizing lists. 7 00:01:00 --> 00:01:10 OK, and so the set up for this problem is that we have a list, 8 00:01:10 --> 00:01:17 L, of n elements. And, we have an operation. 9 00:01:17 --> 00:01:26 Woops, I've got to spell things right, access of x, 10 00:01:26 --> 00:01:33 which accesses item x in the list. 11 00:01:33 --> 00:01:40 It could be by searching, or it could be however you want 12 00:01:40 --> 00:01:42 to do it. But basically, 13 00:01:42 --> 00:01:46 it goes and touches that element. 14 00:01:46 --> 00:01:54 And we were to say the cost of that operation is whatever the 15 00:01:54 --> 00:02:01 rank of x is in the list, which is just the distance of x 16 00:02:01 --> 00:02:10 from the head of the list. And the other thing that we can 17 00:02:10 --> 00:02:18 do that the algorithm can do, so this is what the user would 18 00:02:18 --> 00:02:22 do. He just simply runs a whole 19 00:02:22 --> 00:02:30 bunch of accesses on the list, OK, accessing one element after 20 00:02:30 --> 00:02:36 another in any order that he or she cares to. 21 00:02:36 --> 00:02:43 And then, L can be reordered, however, by transposing 22 00:02:43 --> 00:02:53 adjacent elements. And the cost for that is one. 23 00:02:53 --> 00:03:01 So, for example, suppose the list is the 24 00:03:01 --> 00:03:04 following. 25 00:03:04 --> 00:03:21 26 00:03:21 --> 00:03:29 OK, I missed something here. It doesn't matter. 27 00:03:29 --> 00:03:40 Well, I'll just make it be what I have so that it matches the 28 00:03:40 --> 00:03:46 online video. OK, so here we have a list. 29 00:03:46 --> 00:03:51 And so, if I do something like access element 14 here, 30 00:03:51 --> 00:03:56 the element of key 14, OK, that this costs me one, 31 00:03:56 --> 00:04:01 two, three, four. So here the cost is four to 32 00:04:01 --> 00:04:05 access. And so, we're going to have 33 00:04:05 --> 00:04:10 some sequence of accesses that the user is going to do. 34 00:04:10 --> 00:04:14 And obviously, if something is accessed more 35 00:04:14 --> 00:04:21 frequently, we'd like to move it up to the front of the list so 36 00:04:21 --> 00:04:24 that you don't have to search as far. 37 00:04:24 --> 00:04:28 OK, and to do that, if I want to transpose 38 00:04:28 --> 00:04:31 something, so, for example, 39 00:04:31 --> 00:04:38 if I transpose three and 50, that just costs me one. 40 00:04:38 --> 00:04:43 OK, so then I would make this be 50, and make this be three. 41 00:04:43 --> 00:04:49 OK, sorry, normally you just do it by swapping pointers. 42 00:04:49 --> 00:04:52 OK, so those are the two operations. 43 00:04:52 --> 00:04:59 And, we are going to do this in what's called an online fashion. 44 00:04:59 --> 00:05:06 So, let's just define online. So, a sequence, 45 00:05:06 --> 00:05:16 S, of operations is provided one at a time for each 46 00:05:16 --> 00:05:24 operation. An online algorithm must 47 00:05:24 --> 00:05:34 execute the operation immediately without getting a 48 00:05:34 --> 00:05:48 chance to look at what else is coming in the sequence. 49 00:05:48 --> 00:05:51 So, when you make your decision for the first element, 50 00:05:51 --> 00:05:54 you don't get to see ahead as to what the second, 51 00:05:54 --> 00:05:57 or third, or whatever is. And the second one you get, 52 00:05:57 --> 00:06:00 you get and you have to make your decision as to what to do 53 00:06:00 --> 00:06:06 and so forth. So, that's an online algorithm. 54 00:06:06 --> 00:06:11 Similarly, an off-line algorithm, OK, 55 00:06:11 --> 00:06:18 may see all of S in advance. OK, so you can see an off-line 56 00:06:18 --> 00:06:27 algorithm gets to see the whole sequence, and then decide what 57 00:06:27 --> 00:06:33 it wants to do about element one, element two, 58 00:06:33 --> 00:06:37 or whatever. OK, so an off-line algorithm 59 00:06:37 --> 00:06:41 can look at the whole sequence and say, OK, I can see that item 60 00:06:41 --> 00:06:45 number 17 is being accessed a lot, or early on move him up 61 00:06:45 --> 00:06:49 closer to the front of the list, and then the accesses cost less 62 00:06:49 --> 00:06:53 for the off-line algorithm. The online algorithm doesn't 63 00:06:53 --> 00:06:56 get to see any of that. OK, so this is sort of like, 64 00:06:56 --> 00:07:00 if you're familiar with the game Tetris. 65 00:07:00 --> 00:07:02 OK, and Tetris, you get one shape after another 66 00:07:02 --> 00:07:05 that starts coming down, and you have to twiddle it, 67 00:07:05 --> 00:07:08 and move it to the side, and drop it into place. 68 00:07:08 --> 00:07:11 And there, sometimes you get a one step look-ahead on some of 69 00:07:11 --> 00:07:14 them so you can see what the next shape is, 70 00:07:14 --> 00:07:17 but often it's purely online. You don't get to see that next 71 00:07:17 --> 00:07:20 shape or whatever, and you have to make a decision 72 00:07:20 --> 00:07:22 for each one. And you make a decision, 73 00:07:22 --> 00:07:26 and you realize that the next shape, ah, if you had made a 74 00:07:26 --> 00:07:28 different decision it would have been better. 75 00:07:28 --> 00:07:32 OK, so that's the kind of problem. 76 00:07:32 --> 00:07:38 Off-line Tetris would be, I get to see the whole sequence 77 00:07:38 --> 00:07:42 of shapes. And now let me decide what I'm 78 00:07:42 --> 00:07:47 going to do with this one. OK, and so, in this, 79 00:07:47 --> 00:07:53 the goal is for any of the algorithms, either online or 80 00:07:53 --> 00:08:00 off-line is to minimize the total cost, which we'll denote 81 00:08:00 --> 00:08:06 by, I forgot to name this. This is algorithm A here. 82 00:08:06 --> 00:08:09 The total cost, C_A of S, OK, 83 00:08:09 --> 00:08:13 so the cost of algorithm A on the sequence, 84 00:08:13 --> 00:08:16 S. That's just the notation we'll 85 00:08:16 --> 00:08:22 use for what the total cost is. So, any questions about the 86 00:08:22 --> 00:08:27 setup to this problem? So, we have an online problem. 87 00:08:27 --> 00:08:32 We're going to get these things one at a time, 88 00:08:32 --> 00:08:37 OK, and we have to decide what to do. 89 00:08:37 --> 00:08:41 So, let's do a worst-case analysis for this. 90 00:08:41 --> 00:08:52 91 00:08:52 --> 00:08:56 OK, so if we're doing a worst-case analysis, 92 00:08:56 --> 00:09:01 we can view that we have an adversary that we are playing 93 00:09:01 --> 00:09:06 against who's going to provide the sequence. 94 00:09:06 --> 00:09:10 The user is going to be able to see what we do. 95 00:09:10 --> 00:09:14 And so, what's the adversary strategy? 96 00:09:14 --> 00:09:18 Thwart our plots, yes, that's his idea. 97 00:09:18 --> 00:09:22 And how is he going to thwart them, or she? 98 00:09:22 --> 00:09:29 Which is what for this problem? What's he going to do? 99 00:09:29 --> 00:09:32 Yeah. No matter how we reorder 100 00:09:32 --> 00:09:39 elements using the transposes, he's going to look at every 101 00:09:39 --> 00:09:43 step and say, what's the last element? 102 00:09:43 --> 00:09:48 That's the one I'm going to access, right? 103 00:09:48 --> 00:09:53 So, the adversary always, always accesses the tail 104 00:09:53 --> 00:09:57 element of L. No matter what it is, 105 00:09:57 --> 00:10:03 no matter how we reorder things, OK, for each one, 106 00:10:03 --> 00:10:09 adversary just accesses the tail. 107 00:10:09 --> 00:10:13 So the cost of this, of any algorithm, 108 00:10:13 --> 00:10:18 then, is going to be omega size of S times n, 109 00:10:18 --> 00:10:25 OK, because you're always going to pay for every sequence. 110 00:10:25 --> 00:10:31 You're going to have to go in to pay a cost of n, 111 00:10:31 --> 00:10:36 OK, for every element in the sequence. 112 00:10:36 --> 00:10:42 OK, so not terribly in the worst-case. 113 00:10:42 --> 00:10:46 Not terribly good. So, people in studying this 114 00:10:46 --> 00:10:50 problem: question? That analysis is for the online 115 00:10:50 --> 00:10:54 algorithm, right. The off-line algorithm, 116 00:10:54 --> 00:10:59 right, if you named those things, that's right. 117 00:10:59 --> 00:11:03 OK, so we're looking at trying to solve this in an off-line 118 00:11:03 --> 00:11:06 sense, sorry, in an online sense. 119 00:11:06 --> 00:11:10 OK, and so the point is that for the online algorithm, 120 00:11:10 --> 00:11:15 the adversary can be incredibly mean, OK, and just always access 121 00:11:15 --> 00:11:17 the thing at the end, OK? 122 00:11:17 --> 00:11:21 So, what sort of the history of this problem is, 123 00:11:21 --> 00:11:24 that people said, well, if I can't do well in the 124 00:11:24 --> 00:11:29 worst-case, maybe I should be looking at average case, 125 00:11:29 --> 00:11:32 OK, and look at, say, the different elements 126 00:11:32 --> 00:11:37 having some probability distribution. 127 00:11:37 --> 00:11:44 OK, so the average case analysis, OK, 128 00:11:44 --> 00:11:56 let's suppose that element x is accessed with probability, 129 00:11:56 --> 00:12:03 P of x. OK, so suppose that we have 130 00:12:03 --> 00:12:14 some a priori distribution on the elements. 131 00:12:14 --> 00:12:20 OK, then the expected cost of the algorithm on a sequence, 132 00:12:20 --> 00:12:26 OK, so if I put all the elements into some order, 133 00:12:26 --> 00:12:33 OK, and don't try to reorder, but just simply look at, 134 00:12:33 --> 00:12:39 is there a static ordering that would work well for a 135 00:12:39 --> 00:12:44 distribution? It's just going to be, 136 00:12:44 --> 00:12:49 by definition of expectation, the probability of x times, 137 00:12:49 --> 00:12:54 in this case, the cost, which is the rank of 138 00:12:54 --> 00:13:01 x in whatever that ordering is that I decide I'm going to use. 139 00:13:01 --> 00:13:09 OK, and this is minimized when? So, this is just the definition 140 00:13:09 --> 00:13:15 of expectations: the probability that I access x 141 00:13:15 --> 00:13:21 times the cost summed over all the elements. 142 00:13:21 --> 00:13:28 And the cost is just going to be its position in the list. 143 00:13:28 --> 00:13:33 So, when is this value, this summation, 144 00:13:33 --> 00:13:41 going to be minimized? When the element is most likely 145 00:13:41 --> 00:13:44 as the lowest rank, and then what, 146 00:13:44 --> 00:13:49 what about the other element? OK, so what does that mean? 147 00:13:49 --> 00:13:54 Yeah, sort them, yeah, sort them on the basis of 148 00:13:54 --> 00:13:58 decreasing probability, OK? 149 00:13:58 --> 00:14:04 So, it's minimized when L is sorted, OK, in decreasing order 150 00:14:04 --> 00:14:10 with respect to P. OK, so just sort them with the 151 00:14:10 --> 00:14:16 most likely one at the front, and then just decreasing 152 00:14:16 --> 00:14:20 probability. That way, whenever I access 153 00:14:20 --> 00:14:27 something with some probability, OK, I'm going to access, 154 00:14:27 --> 00:14:33 it's more likely that I'm going to access. 155 00:14:33 --> 00:14:37 And that's not too difficult to actually prove. 156 00:14:37 --> 00:14:41 You just look at, suppose there were two that 157 00:14:41 --> 00:14:46 were out of order, and show that if you swap them, 158 00:14:46 --> 00:14:50 you would improve this optimization function. 159 00:14:50 --> 00:14:56 OK, so if you didn't know it, this suggests the following 160 00:14:56 --> 00:15:05 heuristic, OK, which is simply keep 161 00:15:05 --> 00:15:21 account of the number of times each element is accessed, 162 00:15:21 --> 00:15:38 and maintain the list in order of decreasing count. 163 00:15:38 --> 00:15:42 OK, so whenever something is accessed, increment its count, 164 00:15:42 --> 00:15:45 OK, and that will move it, at most, one position, 165 00:15:45 --> 00:15:48 which only costs me one transposed to move it, 166 00:15:48 --> 00:15:50 perhaps, forward. OK, actually, 167 00:15:50 --> 00:15:54 I guess it could be more if you have a whole bunch of ties, 168 00:15:54 --> 00:15:55 right? Yeah. 169 00:15:55 --> 00:15:58 So, it could cost more. But the idea is, 170 00:15:58 --> 00:16:02 over time, the law of large numbers says that this is going 171 00:16:02 --> 00:16:06 to approach the probability distribution. 172 00:16:06 --> 00:16:11 The frequency with which you access this, divided by the 173 00:16:11 --> 00:16:15 total number of accesses, will be the probability. 174 00:16:15 --> 00:16:20 And so, therefore you will get things in decreasing 175 00:16:20 --> 00:16:24 probability, OK, assuming that there is some 176 00:16:24 --> 00:16:29 distribution that all of these elements are chosen according 177 00:16:29 --> 00:16:35 to, or accessed according to. So, it doesn't seem like 178 00:16:35 --> 00:16:40 there's that much more you could really do here. 179 00:16:40 --> 00:16:46 And that's why I think this notion of competitive analysis 180 00:16:46 --> 00:16:51 is so persuasive, because it's really amazingly 181 00:16:51 --> 00:16:55 strong, OK? And it came about because of 182 00:16:55 --> 00:17:00 what people were doing in practice. 183 00:17:00 --> 00:17:04 So practice, what people implement it was a 184 00:17:04 --> 00:17:07 so-called move to front heuristic. 185 00:17:07 --> 00:17:12 OK, and the basic idea was, after you access an element, 186 00:17:12 --> 00:17:18 just move it up to the front. OK, that only doubles the cost 187 00:17:18 --> 00:17:24 of accessing the element because I go and I access it, 188 00:17:24 --> 00:17:29 chasing it down paying the rank, and then I have to do rank 189 00:17:29 --> 00:17:36 number of transposes to bring it back to the front. 190 00:17:36 --> 00:17:39 So, it only cost me a factor of two, and now, 191 00:17:39 --> 00:17:43 if it happens to be a frequently accessed elements, 192 00:17:43 --> 00:17:47 over time you would hope that the most likely elements were 193 00:17:47 --> 00:17:55 near the front of that list. So, after accessing x, 194 00:17:55 --> 00:18:05 move x, the head of the list using transposes, 195 00:18:05 --> 00:18:17 and the cost is just equal to twice the rank in L of x, 196 00:18:17 --> 00:18:25 OK, where the two here has two parts. 197 00:18:25 --> 00:18:34 One is the access, and the other is the 198 00:18:34 --> 00:18:41 transposes. OK, so that's sort of what they 199 00:18:41 --> 00:18:43 did. And one of the nice properties 200 00:18:43 --> 00:18:47 of this is that if it turns out that there is locality in the 201 00:18:47 --> 00:18:50 access pattern, if it's not just a static 202 00:18:50 --> 00:18:53 distribution, but rather once I've accessed 203 00:18:53 --> 00:18:57 something, if it's more likely I'm going to access it again, 204 00:18:57 --> 00:19:01 which tends to be the case for many input types of patterns, 205 00:19:01 --> 00:19:05 this responds well to locality because it's going to be up near 206 00:19:05 --> 00:19:10 the front if I access it very soon after I've accessed. 207 00:19:10 --> 00:19:15 So, there is what's called temporal locality, 208 00:19:15 --> 00:19:19 meaning that in time, I tend to access, 209 00:19:19 --> 00:19:26 so it may be that I access some thing's very hot for awhile; 210 00:19:26 --> 00:19:32 then it gets very cold. This type of algorithm responds 211 00:19:32 --> 00:19:37 very well to the hotness of the accessing. 212 00:19:37 --> 00:19:43 OK, so it responds well to locality in S. 213 00:19:43 --> 00:19:48 So, this is sort of what was known up to the point that a 214 00:19:48 --> 00:19:54 very famous paper was written by Danny Sleator and Bob Tarjan, 215 00:19:54 --> 00:19:59 where they took a totally different approach to looking at 216 00:19:59 --> 00:20:04 this kind of problem. OK, and it's an approach that 217 00:20:04 --> 00:20:09 matter you see everywhere from analysis of caching and 218 00:20:09 --> 00:20:14 high-performance processors to analyses of disk paging to just 219 00:20:14 --> 00:20:21 a huge number of applications of this basic technique. 220 00:20:21 --> 00:20:30 And, that's the technique of competitive analysis. 221 00:20:30 --> 00:20:45 OK, so here's the definition. So, online algorithm A is alpha 222 00:20:45 --> 00:20:55 competitive. If there exists a constant, 223 00:20:55 --> 00:21:08 k, such that for any sequence, S, of operations, 224 00:21:08 --> 00:21:23 the cost of S using algorithm A is bounded by alpha times the 225 00:21:23 --> 00:21:39 cost of opt, where opt is the optimal offline algorithm. 226 00:21:39 --> 00:21:43 OK, so the optimal off-line, the one that knows the whole 227 00:21:43 --> 00:21:47 sequence and does the absolute best it could do on that 228 00:21:47 --> 00:21:50 sequence, OK, that's this cost here. 229 00:21:50 --> 00:21:53 This is sometimes called God's algorithm, OK, 230 00:21:53 --> 00:21:58 not to bring religion into the classroom, or to offend anybody, 231 00:21:58 --> 00:22:03 but that is what people sometimes call it. 232 00:22:03 --> 00:22:07 OK, so the fully omniscient knows absolutely the best thing 233 00:22:07 --> 00:22:10 that could be done, sees into the future, 234 00:22:10 --> 00:22:12 the whole works, OK? 235 00:22:12 --> 00:22:16 It gets to apply that. That's what opts algorithm is. 236 00:22:16 --> 00:22:21 And, what we're saying is that the cost is basically whatever 237 00:22:21 --> 00:22:24 this alpha factor is. It could be a function of 238 00:22:24 --> 00:22:27 things, or it could be a constant, OK, 239 00:22:27 --> 00:22:30 times whatever the best algorithm is. 240 00:22:30 --> 00:22:36 Plus, there's a potential for a constant out here. 241 00:22:36 --> 00:22:38 OK, so for example, if alpha is two, 242 00:22:38 --> 00:22:43 and we say it's two competitive, that means you're 243 00:22:43 --> 00:22:47 going to do, at worst, twice the algorithm that has 244 00:22:47 --> 00:22:51 all the information. But you're doing it online, 245 00:22:51 --> 00:22:54 for example. OK, it's a really pretty 246 00:22:54 --> 00:22:57 powerful notion. And what's interesting about 247 00:22:57 --> 00:23:04 this, it's not even clear these things should exist to my mind. 248 00:23:04 --> 00:23:08 OK, what's pretty remarkable about this, I think, 249 00:23:08 --> 00:23:12 is that there is no assumption of distribution, 250 00:23:12 --> 00:23:16 of probability distribution or anything. 251 00:23:16 --> 00:23:20 It's whatever the sequence is that you give it. 252 00:23:20 --> 00:23:24 You are within a factor of alpha, essentially, 253 00:23:24 --> 00:23:30 of the best algorithm, OK, which is pretty remarkable. 254 00:23:30 --> 00:23:38 OK, and so, we're going to prove the following theorem, 255 00:23:38 --> 00:23:45 which is the one that Sleator and Tarjan proved. 256 00:23:45 --> 00:23:55 And that is that MTF is four competitive for self organizing 257 00:23:55 --> 00:23:58 lists. OK, so the idea here is that 258 00:23:58 --> 00:24:02 suppose the adversary says, oh, I'm always going to access 259 00:24:02 --> 00:24:06 the thing at the end of the list like we said in the beginning. 260 00:24:06 --> 00:24:10 So, the adversary says, I'm always going to access the 261 00:24:10 --> 00:24:12 thing there. I'm going to make MTF work 262 00:24:12 --> 00:24:16 really bad, because you're going to go and move that thing all 263 00:24:16 --> 00:24:20 the way up to the front. And I'm just going to access 264 00:24:20 --> 00:24:23 the thing way at the end again. OK, well it turns out, 265 00:24:23 --> 00:24:26 yeah, that's a bad sequence for move to front, 266 00:24:26 --> 00:24:30 OK, and it will take a long time. 267 00:24:30 --> 00:24:34 But it turns out God couldn't have done better, 268 00:24:34 --> 00:24:40 OK, by more than a factor of four no matter how long the list 269 00:24:40 --> 00:24:43 is. OK, that's pretty amazing. 270 00:24:43 --> 00:24:49 OK, so that's a bad sequence. But, if there's a way that the 271 00:24:49 --> 00:24:55 sequence exhibits any kind of locality or anything that can be 272 00:24:55 --> 00:25:00 taken advantage of, if you could see the whole 273 00:25:00 --> 00:25:06 thing, MTF takes advantage of it too, OK, within a factor of 274 00:25:06 --> 00:25:10 four. OK, it's a pretty remarkable 275 00:25:10 --> 00:25:14 theorem, and it's the basis of many types of analysis of online 276 00:25:14 --> 00:25:17 algorithms. Almost all online algorithms 277 00:25:17 --> 00:25:21 today are analyzed using some kind of competitive analysis. 278 00:25:21 --> 00:25:25 OK, not always. Sometimes you do probabilistic 279 00:25:25 --> 00:25:28 analysis, or whatever, but the dominant thing is too 280 00:25:28 --> 00:25:33 competitive analysis because then you don't have to make any 281 00:25:33 --> 00:25:38 statistical assumptions. OK, just prove that it works 282 00:25:38 --> 00:25:40 well no matter what. This is remarkable, 283 00:25:40 --> 00:25:42 I think. Isn't it remarkable? 284 00:25:42 --> 00:25:47 So, let's prove this theorem, we're just going to spend the 285 00:25:47 --> 00:25:49 rest of the lecture on this proof. 286 00:25:49 --> 00:25:54 OK, and the proof in some ways is not hard, but it's also not 287 00:25:54 --> 00:25:56 necessarily completely intuitive. 288 00:25:56 --> 00:26:00 So, you will have to pay attention. 289 00:26:00 --> 00:26:09 OK, so let's get some notation down. 290 00:26:09 --> 00:26:22 Let's let L_i be MTF's list after the i'th access. 291 00:26:22 --> 00:26:38 And, let's let L be opt's list after the i'th access. 292 00:26:38 --> 00:26:42 OK, so generally what I'll do is I'll put a star if we are 293 00:26:42 --> 00:26:46 talking about opt, and have nothing if we're 294 00:26:46 --> 00:26:50 talking about MTF. OK, so that's going to be the 295 00:26:50 --> 00:26:51 list. So, we can say, 296 00:26:51 --> 00:26:55 what's the list? So, we're going to set it up 297 00:26:55 --> 00:27:00 where we have one in operation that transforms list i minus one 298 00:27:00 --> 00:27:03 into list i. OK, that's what the i'th 299 00:27:03 --> 00:27:09 operation does. OK, and move to front does it 300 00:27:09 --> 00:27:16 by moving whatever the thing that was accessed to the front. 301 00:27:16 --> 00:27:23 And opt does whatever opt thinks is the best thing to do. 302 00:27:23 --> 00:27:28 We don't know. So, we're going to let c_i be 303 00:27:28 --> 00:27:33 MTF's cost for the I'th operation. 304 00:27:33 --> 00:27:40 And that's just twice the rank in L_i minus one of x if the 305 00:27:40 --> 00:27:47 operation accesses x, OK, two times the rank in L_i 306 00:27:47 --> 00:27:55 minus one because we're going to be accessing it in L_i minus one 307 00:27:55 --> 00:28:01 and transforming it into L_i. And similarly, 308 00:28:01 --> 00:28:09 we'll let c_i star be opt's cost for the i'th operation. 309 00:28:09 --> 00:28:14 And that's just equal to, well, to access it, 310 00:28:14 --> 00:28:20 it's going to be the rank in L_i minus one star, 311 00:28:20 --> 00:28:28 whatever its list is of x at that step, because it's got to 312 00:28:28 --> 00:28:34 access it. And then, some number of 313 00:28:34 --> 00:28:41 transposes, t_i if opt forms t_i transposes. 314 00:28:41 --> 00:28:51 OK, so we have the setup where we have two different lists that 315 00:28:51 --> 00:28:59 are being managed, and we have different costs in 316 00:28:59 --> 00:29:04 the list. And, we're interested in is 317 00:29:04 --> 00:29:11 comparing in some way MTF's list with opt's list at any point in 318 00:29:11 --> 00:29:14 time. And, how do you think we're 319 00:29:14 --> 00:29:19 going to do that? What technique do you think we 320 00:29:19 --> 00:29:25 should use to compare these two lists, general technique from 321 00:29:25 --> 00:29:28 last lecture? Well, it is going to be 322 00:29:28 --> 00:29:35 amortized, but what? How are we going to compare 323 00:29:35 --> 00:29:39 them? What technique did we learn 324 00:29:39 --> 00:29:43 last time? Potential function, 325 00:29:43 --> 00:29:47 good. OK, we're going to define a 326 00:29:47 --> 00:29:54 potential function, OK, that measures how far apart 327 00:29:54 --> 00:29:59 these two lists are. OK, and the idea is, 328 00:29:59 --> 00:30:08 if, let's define that and then we'll take a look at it. 329 00:30:08 --> 00:30:21 So, we're going to define the potential function phi mapping 330 00:30:21 --> 00:30:37 the set of MTF's lists into the real numbers by the following. 331 00:30:37 --> 00:30:46 phi of L_i is going to be twice the cardinality of this set. 332 00:30:46 --> 00:31:04 333 00:31:04 --> 00:31:08 OK, so this is the precedes-operation and list, 334 00:31:08 --> 00:31:10 i. So, we can define a 335 00:31:10 --> 00:31:15 relationship between any two elements that says that x 336 00:31:15 --> 00:31:20 precedes y in L_i if, as I'm accessing it from the 337 00:31:20 --> 00:31:25 head, I hit x first. OK, so what I'm interested in, 338 00:31:25 --> 00:31:31 here, are in some sense the disagreements between the two 339 00:31:31 --> 00:31:35 lists. This is where x precedes y in 340 00:31:35 --> 00:31:39 MTF's list, but y precedes x in opt's list. 341 00:31:39 --> 00:31:40 They disagree, OK? 342 00:31:40 --> 00:31:45 And, what we're interested in is the cardinality of the set. 343 00:31:45 --> 00:31:49 And we're going to multiply it by two. 344 00:31:49 --> 00:31:53 OK, so that's equal to two times; so there is a name for 345 00:31:53 --> 00:31:58 this type of thing. We saw that when we were doing 346 00:31:58 --> 00:32:03 sorting. Anybody remember the name? 347 00:32:03 --> 00:32:08 It was very briefly. I don't expect anybody to 348 00:32:08 --> 00:32:14 remember, but somebody might. Inversions: good, 349 00:32:14 --> 00:32:18 OK, twice the number of inversions. 350 00:32:18 --> 00:32:25 So, let's just do an example. So, let's say L_i is the list 351 00:32:25 --> 00:32:37 with five elements. OK, I'll use characters for the 352 00:32:37 --> 00:32:47 order just to keep things simple. 353 00:32:47 --> 00:33:01 So, in this case phi of L_i is going to be twice the 354 00:33:01 --> 00:33:12 cardinality of the set. So what we want to do is see 355 00:33:12 --> 00:33:18 which things are out of order. So here, I look at E and C are 356 00:33:18 --> 00:33:21 in this order, but C and E in that order. 357 00:33:21 --> 00:33:26 So, those are out of order. So, that counts as one of my 358 00:33:26 --> 00:33:29 elements, EC, and then, E and A, 359 00:33:29 --> 00:33:36 A and E. OK, so those are out of order, 360 00:33:36 --> 00:33:41 and then ED, DE, out of order, 361 00:33:41 --> 00:33:48 and then EB, BE, those are out of order. 362 00:33:48 --> 00:33:53 And now, I go C, A, C, A. 363 00:33:53 --> 00:34:00 Those are in order, so it doesn't count. 364 00:34:00 --> 00:34:08 CD, CD, CB, CB, so, nothing with C. 365 00:34:08 --> 00:34:12 Then, A, D, A, D, those are in order, 366 00:34:12 --> 00:34:16 A, B, A, B, those are in order. So then, DB, 367 00:34:16 --> 00:34:20 BD, so BD. And that's the last one. 368 00:34:20 --> 00:34:26 So, that's my potential function, which is equal to, 369 00:34:26 --> 00:34:31 therefore, ten, because the cardinality of the 370 00:34:31 --> 00:34:37 set is five. I have five inversions, 371 00:34:37 --> 00:34:44 OK, between the two lists. OK, so let's just check some 372 00:34:44 --> 00:34:49 properties of this potential function. 373 00:34:49 --> 00:34:57 The first one is notice that phi of L_i is greater than or 374 00:34:57 --> 00:35:04 equal to zero for all i. The number of inversions might 375 00:35:04 --> 00:35:07 be zero, but is never less than zero. 376 00:35:07 --> 00:35:11 OK, it's always at least zero. So, that's one of the 377 00:35:11 --> 00:35:16 properties that we normally have we are dealing with potential 378 00:35:16 --> 00:35:19 functions. And, the other thing is, 379 00:35:19 --> 00:35:23 well, what about phi of L0? Is that equal to zero? 380 00:35:23 --> 00:35:27 Well, it depends upon what list they start with. 381 00:35:27 --> 00:35:31 OK, so what's the initial ordering? 382 00:35:31 --> 00:35:35 So, it's zero if they start with the same list. 383 00:35:35 --> 00:35:41 Then there are no inversions. But, they might start with 384 00:35:41 --> 00:35:46 different lists. We'll talk about different 385 00:35:46 --> 00:35:50 lists later on, but let's say for now that it's 386 00:35:50 --> 00:35:55 zero because they start with the same list. 387 00:35:55 --> 00:36:00 That seems like a fair comparison. 388 00:36:00 --> 00:36:04 OK, so we have this potential function now that's counting up, 389 00:36:04 --> 00:36:07 how different are these two lists? 390 00:36:07 --> 00:36:10 Intuitively, we're going to do is the more 391 00:36:10 --> 00:36:14 differences there are in the list, the more we are going to 392 00:36:14 --> 00:36:18 be able to have more stored up work than we can pay for it. 393 00:36:18 --> 00:36:22 That's the basic idea. So, the more that opt changes 394 00:36:22 --> 00:36:27 the list, so it's not the same as ours, in some sense the more 395 00:36:27 --> 00:36:32 we are going to be in a position as MTF to take advantage of that 396 00:36:32 --> 00:36:37 difference in delivering up work for us to do. 397 00:36:37 --> 00:36:42 And we'll see how that plays out. 398 00:36:42 --> 00:36:50 So, let's first also make another observation. 399 00:36:50 --> 00:36:59 So, how much does phi change from one transpose? 400 00:36:59 --> 00:37:08 How much does phi change from one transpose? 401 00:37:08 --> 00:37:14 So, basically that's asking, if you do a transpose, 402 00:37:14 --> 00:37:18 what happens to the number of inversions? 403 00:37:18 --> 00:37:24 So, what happens when a transposing is done? 404 00:37:24 --> 00:37:31 What's going to happen to phi? What's going to happen to the 405 00:37:31 --> 00:37:37 number of inversions? So, if I change, 406 00:37:37 --> 00:37:43 it is less than n minus one, yes, if n is sufficiently 407 00:37:43 --> 00:37:47 large, yes. OK, if I change, 408 00:37:47 --> 00:37:54 so you can think about it here. Suppose I switch two of these 409 00:37:54 --> 00:37:59 elements here. How much are things going to 410 00:37:59 --> 00:38:05 change? Yeah, it's basically one or 411 00:38:05 --> 00:38:11 minus one, OK, because a transpose creates or 412 00:38:11 --> 00:38:17 destroys one inversion. So, if you think about it, 413 00:38:17 --> 00:38:21 what if I change, for example, 414 00:38:21 --> 00:38:29 C and A, the relationship of C and A to everything else in the 415 00:38:29 --> 00:38:36 list is going to stay the same. The only thing, 416 00:38:36 --> 00:38:41 possibly, that happens is that if they are in the same order 417 00:38:41 --> 00:38:46 when I transpose them, I've created an inversion. 418 00:38:46 --> 00:38:51 Or, if they were in the wrong order when I transpose them, 419 00:38:51 --> 00:38:55 now they're in the right order. So therefore, 420 00:38:55 --> 00:39:01 the change to the potential function is going to be plus or 421 00:39:01 --> 00:39:08 minus two because we're counting twice the number of inversions. 422 00:39:08 --> 00:39:14 OK, any questions about that? So, transposes don't change the 423 00:39:14 --> 00:39:17 potential very much, just by one. 424 00:39:17 --> 00:39:22 It either goes up by two or down by two, just by one 425 00:39:22 --> 00:39:26 inversion. So now, let's take a look at 426 00:39:26 --> 00:39:31 how these two algorithms operate. 427 00:39:31 --> 00:39:46 428 00:39:46 --> 00:40:03 OK, so what happens when op i accesses x in the two lists? 429 00:40:03 --> 00:40:07 What's going to be going on? To do that, let's define the 430 00:40:07 --> 00:40:09 following sets. 431 00:40:09 --> 00:40:33 432 00:40:33 --> 00:40:34 Why do I keep doing that? 433 00:40:34 --> 00:41:54 434 00:41:54 --> 00:41:57 OK, so we're going to look at the, when we access x, 435 00:41:57 --> 00:42:01 we are going to look at the two lists, and see what the 436 00:42:01 --> 00:42:05 relationship is, so, based on things that come 437 00:42:05 --> 00:42:10 before and after. So, I think a picture is very 438 00:42:10 --> 00:42:15 helpful to understand what's going on here. 439 00:42:15 --> 00:42:19 OK, so let's let, so here's L_i minus one, 440 00:42:19 --> 00:42:25 and we have our list, which I'll draw like this. 441 00:42:25 --> 00:42:30 And somewhere in there, we have x. 442 00:42:30 --> 00:42:40 OK, and then we have L_i minus one star, which is opt's list, 443 00:42:40 --> 00:42:48 OK, and he's got x somewhere else, or she. 444 00:42:48 --> 00:42:59 OK, and so, what is this set? This is the set of Y that come 445 00:42:59 --> 00:43:05 before x. So, that basically sets A and 446 00:43:05 --> 00:43:08 B. OK, those things that come 447 00:43:08 --> 00:43:13 before x in both. And, some of them, 448 00:43:13 --> 00:43:19 the A's come before it in x, but come after it in, 449 00:43:19 --> 00:43:24 come before it in A, but come after it in B. 450 00:43:24 --> 00:43:31 OK, and similarly down here, what's this set? 451 00:43:31 --> 00:43:40 452 00:43:40 --> 00:43:42 That's A union C, good. 453 00:43:42 --> 00:43:44 And this one? Duh. 454 00:43:44 --> 00:43:51 Yeah, it better be C union D because I've got A union B over 455 00:43:51 --> 00:43:57 there, and I've got x. So that better be everything 456 00:43:57 --> 00:44:02 else. OK, and here is B union D. 457 00:44:02 --> 00:44:11 OK, so those are the four sets that we're going to care about. 458 00:44:11 --> 00:44:19 We're actually mostly going to care about these two sets. 459 00:44:19 --> 00:44:26 OK, and we also know something about the r here. 460 00:44:26 --> 00:44:35 The position of x is going to be the rank in L_i minus one of 461 00:44:35 --> 00:44:39 x. And here, this is our star. 462 00:44:39 --> 00:44:44 It's just to the rank in L_i minus one star of x. 463 00:44:44 --> 00:44:47 So, we know what these ranks are. 464 00:44:47 --> 00:44:51 And what we're going to be interested in is, 465 00:44:51 --> 00:44:56 in fact, characterizing the rank in terms of the sets. 466 00:44:56 --> 00:45:01 OK, so what's the position of this? 467 00:45:01 --> 00:45:09 Well, the rank, we have that r is equal to the 468 00:45:09 --> 00:45:17 size of A. What's the size of B plus one? 469 00:45:17 --> 00:45:28 OK, and r star is equal to the size of A plus the size of C 470 00:45:28 --> 00:45:34 plus one. So, let's take a look at what 471 00:45:34 --> 00:45:41 happens when these two algorithms do their thing. 472 00:45:41 --> 00:45:48 So, when the access to x occurs, we move x to the front 473 00:45:48 --> 00:45:53 of the list. OK, it goes right up to the 474 00:45:53 --> 00:45:57 front. So, how many inversions are 475 00:45:57 --> 00:46:03 created and destroyed? So, how many are created by 476 00:46:03 --> 00:46:05 this? That's probably a, 477 00:46:05 --> 00:46:08 how many inversions are created? 478 00:46:08 --> 00:46:33 479 00:46:33 --> 00:46:35 How many inversions are created? 480 00:46:35 --> 00:46:39 So, we move x to the front. So what we are concerned about 481 00:46:39 --> 00:46:43 is that anything that was in one of these sets that came, 482 00:46:43 --> 00:46:47 where it's going to change in order versus down here. 483 00:46:47 --> 00:46:51 So, if I look in B, well, let's take a look at A. 484 00:46:51 --> 00:46:56 OK, so A, those are the things that are in the same order in 485 00:46:56 --> 00:46:58 both. So, everything that's in A, 486 00:46:58 --> 00:47:02 when I move x to the front, each thing in A is going to 487 00:47:02 --> 00:47:09 count for one more inversion. Does everybody see that? 488 00:47:09 --> 00:47:15 So, I create a cardinality of A inversions. 489 00:47:15 --> 00:47:23 And, we are going to destroy, well, everything in B came 490 00:47:23 --> 00:47:31 before x in this list, and after x in this. 491 00:47:31 --> 00:47:36 But after we move x, they're in the right order. 492 00:47:36 --> 00:47:43 So, I'm going to destroy B inversions, cardinality of B 493 00:47:43 --> 00:47:48 inversions. OK, so that's what happens we 494 00:47:48 --> 00:47:52 operate with move to front. We destroy. 495 00:47:52 --> 00:47:58 We create A inversions and destroy B inversions, 496 00:47:58 --> 00:48:06 OK, by doing this movement. OK, now, let's take a look at 497 00:48:06 --> 00:48:09 what opt does. So, each transpose, 498 00:48:09 --> 00:48:16 we don't know what opt does. He might move x this way or 499 00:48:16 --> 00:48:18 that way. We don't know. 500 00:48:18 --> 00:48:22 But each transpose, I opt, well, 501 00:48:22 --> 00:48:29 we're going to be interested in how many inversions it creates, 502 00:48:29 --> 00:48:34 and we already argued that it's going to create, 503 00:48:34 --> 00:48:40 at most, one inversion per transpose. 504 00:48:40 --> 00:48:46 So, he can go and create more inversions, OK? 505 00:48:46 --> 00:48:53 So, let me write it over here. Thus -- 506 00:48:53 --> 00:49:05 507 00:49:05 --> 00:49:16 -- the change in potential is going to be, at most, 508 00:49:16 --> 00:49:26 twice, A minus B plus t_i. OK, so t_i, remember, 509 00:49:26 --> 00:49:39 was the number of transposes that opt does on the i'th step 510 00:49:39 --> 00:49:49 for the i'th operation. OK, so we're going to create 511 00:49:49 --> 00:49:56 the change in potential is, at most, twice this function. 512 00:49:56 --> 00:50:07 So, we are now going to look to see how we use this fact, 513 00:50:07 --> 00:50:16 and these two facts, this fact and this fact, 514 00:50:16 --> 00:50:26 OK, to show that opt can't be much better than MTF. 515 00:50:26 --> 00:50:33 OK, good. The way we are going to do that 516 00:50:33 --> 00:50:38 is look at the amortized cost of the I'th operation. 517 00:50:38 --> 00:50:41 OK, what's MTF's amortized cost? 518 00:50:41 --> 00:50:47 OK, and then we'll make the argument, which is the one you 519 00:50:47 --> 00:50:53 always make that the amortized cost upper bound the true costs, 520 00:50:53 --> 00:50:57 OK? But the amortized cost is going 521 00:50:57 --> 00:51:04 to be easier to calculate. OK, so amortized cost is just C 522 00:51:04 --> 00:51:09 hat, actually, let me make sure I have lots of 523 00:51:09 --> 00:51:15 room here on the right, c_i hat, which is equal to the 524 00:51:15 --> 00:51:19 true cost plus the change in potential. 525 00:51:19 --> 00:51:25 OK, that's just the definition of amortized cost when given 526 00:51:25 --> 00:51:29 potential functions, OK? 527 00:51:29 --> 00:51:36 So, what is the cost of operation i, OK, 528 00:51:36 --> 00:51:45 in this context here? OK, we accessed x there. 529 00:51:45 --> 00:51:56 What's the cost of operation i? Two times the rank of x, 530 00:51:56 --> 00:52:05 which is 2r. OK, so 2r, that part of it. 531 00:52:05 --> 00:52:13 OK, well, we have an upper bound on the change in 532 00:52:13 --> 00:52:17 potential. That's this. 533 00:52:17 --> 00:52:25 OK, so that's two times the cardinality of A minus 534 00:52:25 --> 00:52:33 cardinality of B plus t_i. OK, everybody with me? 535 00:52:33 --> 00:52:37 Yeah? OK, I see lots of nods. 536 00:52:37 --> 00:52:42 That's good. OK, that's equal to 2r plus two 537 00:52:42 --> 00:52:48 of size of A minus, OK, I want to plug in for B, 538 00:52:48 --> 00:52:55 and it turns out very nicely. I have an equation involving A, 539 00:52:55 --> 00:53:00 B, and r. So, I get rid of the variable 540 00:53:00 --> 00:53:06 size of B by just plugging that in. 541 00:53:06 --> 00:53:11 OK, and so what do I plug in here? 542 00:53:11 --> 00:53:18 What's B equal to? Yeah, r minus size of A minus 543 00:53:18 --> 00:53:23 one. I wrote it the other way. 544 00:53:23 --> 00:53:31 OK, and then plus t_i. OK, and this is since r is A 545 00:53:31 --> 00:53:37 plus B plus one. OK, everybody with me still? 546 00:53:37 --> 00:53:44 I'm just doing algebra. We've got to make sure we do 547 00:53:44 --> 00:53:50 the algebra right. OK, so that's equal to, 548 00:53:50 --> 00:53:57 let's just multiply all this out now and get 2r plus, 549 00:53:57 --> 00:54:03 I have 2A here minus A. So, that's 4A. 550 00:54:03 --> 00:54:08 And then, two times minus r is minus 2r. 551 00:54:08 --> 00:54:12 Two times minus one is minus two. 552 00:54:12 --> 00:54:18 Oh, but it's minus-minus two, so it's plus two. 553 00:54:18 --> 00:54:24 OK, and then I have 2t_i. So, that's just algebra. 554 00:54:24 --> 00:54:31 OK, so that's not bad. We've just got rid of another 555 00:54:31 --> 00:54:37 variable. What variable did we get rid 556 00:54:37 --> 00:54:38 of? r. 557 00:54:38 --> 00:54:46 It didn't matter what the rank was as long as I knew what the 558 00:54:46 --> 00:54:54 number of inversions was here. OK, so that's now equal to 4A 559 00:54:54 --> 00:55:02 plus two plus 2t_i. And, that's less than or equal 560 00:55:02 --> 00:55:11 to, I claim, four times r star plus t_i using our other fact. 561 00:55:11 --> 00:55:19 Since r star is equal to the size of A plus the size of C, 562 00:55:19 --> 00:55:28 plus one, then that's greater than or equal to the size of A 563 00:55:28 --> 00:55:35 plus one. OK, if I look at this, 564 00:55:35 --> 00:55:42 I'm basically looking at A. The fact that A, 565 00:55:42 --> 00:55:51 what did I do here? If r star is greater than or 566 00:55:51 --> 00:55:58 equal to A plus one, right, so therefore, 567 00:55:58 --> 00:56:05 A plus one, good. Yeah, so this is basically less 568 00:56:05 --> 00:56:09 than or equal to 4A plus four, which is four times A plus one. 569 00:56:09 --> 00:56:13 I probably should have put in another algebra step here, 570 00:56:13 --> 00:56:16 OK, because if I can't verify it like this, 571 00:56:16 --> 00:56:19 then I get nervous. This is basically, 572 00:56:19 --> 00:56:23 at most, 4A plus four. That's four times A plus one, 573 00:56:23 --> 00:56:26 and A plus one is less than or equal to r star. 574 00:56:26 --> 00:56:30 And then, 2t_i is, at most, 4TI. 575 00:56:30 --> 00:56:43 So, I've got this. Does everybody see where that 576 00:56:43 --> 00:56:54 came from? But what is r star plus t_i? 577 00:56:54 --> 00:57:04 What is r star plus t_i? What is it? 578 00:57:04 --> 00:57:13 It's c_i star. That's just c_i star. 579 00:57:13 --> 00:57:24 So, the amortized cost of i'th operation is, 580 00:57:24 --> 00:57:36 at most, four times opt's cost. OK, that's pretty remarkable. 581 00:57:36 --> 00:57:44 OK, so amortized cost of the i'th operation is just four 582 00:57:44 --> 00:57:48 times opt's cost. Now, of course, 583 00:57:48 --> 00:57:55 we have to now go through and analyze the total cost. 584 00:57:55 --> 00:58:03 But this is now the routine way that we analyze things with a 585 00:58:03 --> 00:58:12 potential function. So, the costs of MTF of S is 586 00:58:12 --> 00:58:21 just the summation of the individual costs, 587 00:58:21 --> 00:58:30 OK, by definition. And that is just the sum, 588 00:58:30 --> 00:58:38 i equals one, to S of the amortized cost 589 00:58:38 --> 00:58:45 plus, minus the change in potential. 590 00:58:45 --> 00:58:55 OK, did I do this right? No, I put the parentheses in 591 00:58:55 --> 00:59:01 the wrong place. Now I've got it right. 592 00:59:01 --> 00:59:04 Good. I just missed a parenthesis. 593 00:59:04 --> 00:59:08 OK, so this is, so in the past what I did was I 594 00:59:08 --> 00:59:13 expressed the amortized cost as being equal to c_i plus the 595 00:59:13 --> 00:59:17 change in potential. I'm just throwing these two 596 00:59:17 --> 00:59:22 terms over to the other side and saying, what's the true cost in 597 00:59:22 --> 00:59:26 terms of the amortized cost? OK, so I get c hat of i plus 598 00:59:26 --> 00:59:31 phi sub L_i minus one minus phi of L_i, OK, by making that 599 00:59:31 --> 00:59:36 substitution. OK, that's less than or equal 600 00:59:36 --> 00:59:41 to since this is linear. Well, I know what the sum of 601 00:59:41 --> 00:59:44 the amortized cost is. It's, at most, 602 00:59:44 --> 00:59:47 4c_i star. So, the sum of them is, 603 00:59:47 --> 00:59:52 at most, to that sum, I equals one to S of 4c_i star. 604 00:59:52 --> 00:59:56 And then, as happens in all these things, 605 00:59:56 --> 1:00:01 you get a telescope with these terms. 606 1:00:01 --> 1:00:08.706 Every term is added in once and subtracted out once, 607 1:00:08.706 --> 1:00:13.542 except for the ones at the limit. 608 1:00:13.542 --> 1:00:22.76 So, I get plus phi of L_0 minus phi of L sub cardinality of S. 609 1:00:22.76 --> 1:00:31.222 And now, this term is zero. And this term is greater than 610 1:00:31.222 --> 1:00:39.468 or equal to zero. OK, so therefore this whole 611 1:00:39.468 --> 1:00:47.579 thing is less than or equal to, well, what's that? 612 1:00:47.579 --> 1:00:53.041 That's just four times opt's cost. 613 1:00:53.041 --> 1:01:00.591 And so, we're four competitive. OK, this is amazing, 614 1:01:00.591 --> 1:01:02.739 I think. It's not that hard, 615 1:01:02.739 --> 1:01:06.956 OK, but it's quite amazing that just by doing a simple 616 1:01:06.956 --> 1:01:11.65 heuristic, you're nearly as good as any omniscient algorithm 617 1:01:11.65 --> 1:01:15.151 could possibly be. OK, you're nearly as good. 618 1:01:15.151 --> 1:01:17.141 And, in fact, in practice, 619 1:01:17.141 --> 1:01:21.358 this is a great heuristic. So, if ever you have things 620 1:01:21.358 --> 1:01:25.893 like a hash table that you're actually seeing by chaining, 621 1:01:25.893 --> 1:01:30.667 OK, often it's the case that if when you access the elements, 622 1:01:30.667 --> 1:01:35.679 you're just bringing them up to the front of the list if it's an 623 1:01:35.679 --> 1:01:40.533 unsorted list that you've put them into, just bring them up to 624 1:01:40.533 --> 1:01:45.448 the front. You can easily save 30 to 40% 625 1:01:45.448 --> 1:01:50.767 in run time for the accessing to the hash table because you will 626 1:01:50.767 --> 1:01:54.736 be much more likely to find the elements inside. 627 1:01:54.736 --> 1:01:59.295 Of course, it depends on the distribution and so forth, 628 1:01:59.295 --> 1:02:03.601 for empirical matters, but the point is that you are 629 1:02:03.601 --> 1:02:08.667 not going to be too far off from the ordering that an optimal 630 1:02:08.667 --> 1:02:12.551 algorithm would do, optimal off-line algorithm: 631 1:02:12.551 --> 1:02:17.037 I mean, amazing. OK: optimal off-line. 632 1:02:17.037 --> 1:02:22.276 Now, it turns out that in the reading that we assigned, 633 1:02:22.276 --> 1:02:28 so, we assigned you Sleator and Tarjan's original paper. 634 1:02:28 --> 1:02:39.388 In that reading, they actually have a slightly 635 1:02:39.388 --> 1:02:55.08 different model where they count transposes that move in excess 636 1:02:55.08 --> 1:03:09 to element x towards the front of the list as free. 637 1:03:09 --> 1:03:14.555 OK, so, and this basically models, so here's the idea is if 638 1:03:14.555 --> 1:03:19.536 I actually have a linked list, and when I chase down, 639 1:03:19.536 --> 1:03:23.846 once I find x, I can actually move x up to the 640 1:03:23.846 --> 1:03:29.306 front with just a constant number of pointer operations to 641 1:03:29.306 --> 1:03:34 splice it out and put it up to the front. 642 1:03:34 --> 1:03:36.785 I don't actually have to transpose all way back down. 643 1:03:36.785 --> 1:03:39.196 OK, so that's kind of the model that they use, 644 1:03:39.196 --> 1:03:40.857 which is a more realistic model. 645 1:03:40.857 --> 1:03:43.75 OK, I presented this argument because it's a little bit 646 1:03:43.75 --> 1:03:45.732 simpler. OK, and the model is a little 647 1:03:45.732 --> 1:03:47.285 bit simpler. But in our model, 648 1:03:47.285 --> 1:03:50.339 they have, when you access something, you want to bring it 649 1:03:50.339 --> 1:03:52.803 up to the front, or anything that you happen to 650 1:03:52.803 --> 1:03:55.91 go across during that time, you could bring up to the front 651 1:03:55.91 --> 1:04:06 essentially for free. This model is the splicing in, 652 1:04:06 --> 1:04:17.466 splicing x in and out of L in constant time. 653 1:04:17.466 --> 1:04:24.4 Then, MTF is, it turns out, 654 1:04:24.4 --> 1:04:32.289 too competitive. It's within a factor of two of 655 1:04:32.289 --> 1:04:36.487 optimal, OK, if you use that. And that's actually a good 656 1:04:36.487 --> 1:04:40.761 exercise to work through. You could also go read about it 657 1:04:40.761 --> 1:04:44.958 in the reading to understand this better, to look to see 658 1:04:44.958 --> 1:04:47.401 where you would use those things. 659 1:04:47.401 --> 1:04:51.675 You have to have another term representing the number of, 660 1:04:51.675 --> 1:04:55.796 quote, "free" transposes. But it turns out that all the 661 1:04:55.796 --> 1:04:58.467 math works out pretty much the same. 662 1:04:58.467 --> 1:05:01.673 OK, let's see, another thing I promised you 663 1:05:01.673 --> 1:05:06.023 is, what if, to look at the case, what if they don't start 664 1:05:06.023 --> 1:05:14.48 with the same lists? OK, what if the two lists are 665 1:05:14.48 --> 1:05:25.251 different when they start? Then, the potential function at 666 1:05:25.251 --> 1:05:33 the beginning might be as big as what? 667 1:05:33 --> 1:05:37.997 How big are the potential function start out as if the 668 1:05:37.997 --> 1:05:42.805 lists are different? So, suppose we're starting out, 669 1:05:42.805 --> 1:05:45.539 you have a list, and opt says, 670 1:05:45.539 --> 1:05:51.008 OK, I'm going to start out by ordering my list according to 671 1:05:51.008 --> 1:05:56.948 the sequence that I want to use, OK, and MTF orders it according 672 1:05:56.948 --> 1:06:02.511 to the sequence it must use. What list is opt going to start 673 1:06:02.511 --> 1:06:09.9 out with as an adversary? Yeah, it's going to pick the 674 1:06:09.9 --> 1:06:16.447 reverse of what ever MTF starts out with, right, 675 1:06:16.447 --> 1:06:21.601 because then, if he picks the reverse, 676 1:06:21.601 --> 1:06:25.92 what's the number of inversions? 677 1:06:25.92 --> 1:06:34 It's how many inversions in a reverse ordered list? 678 1:06:34 --> 1:06:36.465 Yeah, n choose two, OK. 679 1:06:36.465 --> 1:06:41.508 Is it n choose two, or n minus one choose two? 680 1:06:41.508 --> 1:06:47.112 n minus one choose two, OK, inversions that you get 681 1:06:47.112 --> 1:06:53.163 because it's basically a triangular number when you add 682 1:06:53.163 --> 1:06:55.853 them up. But in any case, 683 1:06:55.853 --> 1:07:00 it's order n^2, worst case. 684 1:07:00 --> 1:07:07.03 So, what does that do to our analysis here? 685 1:07:07.03 --> 1:07:15.064 It says that the cost of MTF of S is going to be, 686 1:07:15.064 --> 1:07:22.596 well, this is no longer zero. This is now n^2. 687 1:07:22.596 --> 1:07:30.63 OK, so we get that costs of MTF of S is, at most, 688 1:07:30.63 --> 1:07:39 four times opt's thing plus order n^2, OK? 689 1:07:39 --> 1:07:51.329 And, if we look at the definition, did we erase it 690 1:07:51.329 --> 1:07:58.625 already? OK, this is still for 691 1:07:58.625 --> 1:08:09.696 competitive, OK, since n^2 is constant as the 692 1:08:09.696 --> 1:08:19.283 size of S goes to infinity. This is, once again, 693 1:08:19.283 --> 1:08:22.364 sort of your notion of, what does it mean to be a 694 1:08:22.364 --> 1:08:24.868 constant? OK, so as the size of the list 695 1:08:24.868 --> 1:08:28.847 gets bigger, all we're doing is accessing whatever that number, 696 1:08:28.847 --> 1:08:31.864 n, is of elements. That number doesn't grow with 697 1:08:31.864 --> 1:08:34.753 the problem size, OK, even if it starts out as 698 1:08:34.753 --> 1:08:37 some variable number, n. 699 1:08:37 --> 1:08:42.579 OK, it doesn't grow with the problem size. 700 1:08:42.579 --> 1:08:47.071 We still end up being competitive. 701 1:08:47.071 --> 1:08:53.603 This is just the k that was in that definition of 702 1:08:53.603 --> 1:08:58.23 competitiveness. OK, any questions? 703 1:08:58.23 --> 1:09:02.761 Yeah? Well, so you could change the 704 1:09:02.761 --> 1:09:05.523 cost model a little bit. Yeah. 705 1:09:05.523 --> 1:09:08.666 And that's a good one to work out. 706 1:09:08.666 --> 1:09:12.666 But if you say the cost of transposing, so, 707 1:09:12.666 --> 1:09:17.904 the cost of transposing is probably moving two pointers, 708 1:09:17.904 --> 1:09:21.523 approximately. No, one, three pointers. 709 1:09:21.523 --> 1:09:26.952 So, suppose that the cost of, wow, that's a good exercise, 710 1:09:26.952 --> 1:09:29.714 OK? Suppose the cost was three 711 1:09:29.714 --> 1:09:34.571 times to do a transpose, was three times the cost of 712 1:09:34.571 --> 1:09:40 doing an access, of following a pointer. 713 1:09:40 --> 1:09:43.741 OK, how would that change the number here? 714 1:09:43.741 --> 1:09:46.752 OK, good exercise, great exercise. 715 1:09:46.752 --> 1:09:51.863 OK, hmm, good final question. OK, yes, it will affect the 716 1:09:51.863 --> 1:09:56.517 constant here just as when we do the free transpose, 717 1:09:56.517 --> 1:10:02.266 when we move things towards the front, that we consider those as 718 1:10:02.266 --> 1:10:06.09 free, OK. Those operations end up 719 1:10:06.09 --> 1:10:11.545 reducing the constant as well. OK, but the point is that this 720 1:10:11.545 --> 1:10:17 constant is independent of the constant having to do with the 721 1:10:17 --> 1:10:22.636 number of elements in the list. So that's a different constant. 722 1:10:22.636 --> 1:10:27.181 So, this is a constant. OK, and so as with a lot of 723 1:10:27.181 --> 1:10:31 these things, there's two things. 724 1:10:31 --> 1:10:34.505 One is, there's the theory. So, theory here backs up 725 1:10:34.505 --> 1:10:37.048 practice. OK, those practitioners knew 726 1:10:37.048 --> 1:10:40.485 what they were doing, OK, without knowing what they 727 1:10:40.485 --> 1:10:43.028 were doing. OK, so that's really good. 728 1:10:43.028 --> 1:10:46.603 OK, and we have a deeper understanding that's led to, 729 1:10:46.603 --> 1:10:50.727 as I say, many algorithms for things like, the important ones 730 1:10:50.727 --> 1:10:53.682 are like paging. So, what's the comment page 731 1:10:53.682 --> 1:10:57.532 replacement policy that people study, people have at most 732 1:10:57.532 --> 1:11:02 operating systems? Who's done 6.033 or something? 733 1:11:02 --> 1:11:04.272 Yeah, it's Least Recently Used, LRU. 734 1:11:04.272 --> 1:11:06.155 People have heard of that, OK. 735 1:11:06.155 --> 1:11:09.597 So, you can analyze LRU competitive, and show that LRU 736 1:11:09.597 --> 1:11:13.363 is actually competitive with optimal page replacement under 737 1:11:13.363 --> 1:11:16.48 certain assumptions. OK, and there are also other 738 1:11:16.48 --> 1:11:18.363 things. Like, people do random 739 1:11:18.363 --> 1:11:21.805 replacement algorithms, and there are a whole bunch of 740 1:11:21.805 --> 1:11:25.831 other kinds of things that can be analyzed with the competitive 741 1:11:25.831 --> 1:11:28.883 analysis framework. OK, so it's very cool stuff. 742 1:11:28.883 --> 1:11:32.324 And, we are going to see more in recitation on Friday, 743 1:11:32.324 --> 1:11:36.09 see a couple of other really good problems that are maybe a 744 1:11:36.09 --> 1:11:40.181 little bit easier than this one, OK, definitely easier than this 745 1:11:40.181 --> 1:11:45.479 one. OK, they give you hopefully 746 1:11:45.479 --> 1:11:51.407 some more intuition about competitive analysis. 747 1:11:51.407 --> 1:11:58.237 I also want to warn you about next week's problem set. 748 1:11:58.237 --> 1:12:07 So, next week's problem set has a programming assignment on it. 749 1:12:07 --> 1:12:10.676 OK, and the programming assignment is mandatory, 750 1:12:10.676 --> 1:12:13.649 meaning, well, all the problem sets are 751 1:12:13.649 --> 1:12:17.639 mandatory as you know, but if you decide not to do a 752 1:12:17.639 --> 1:12:22.568 problem there's a little bit of a penalty and then the penalties 753 1:12:22.568 --> 1:12:26.401 scale dramatically as you stop doing problem sets. 754 1:12:26.401 --> 1:12:30 But this one is mandatory-mandatory. 755 1:12:30 --> 1:12:33.823 OK, you don't pass the class. You'll get an incomplete if you 756 1:12:33.823 --> 1:12:36.18 do not do this programming assignment. 757 1:12:36.18 --> 1:12:39.43 Now, I know that some people are less practiced with 758 1:12:39.43 --> 1:12:42.169 programming. And so, what I encourage you to 759 1:12:42.169 --> 1:12:45.865 do over the weekend is spent a few minutes and work on your 760 1:12:45.865 --> 1:12:49.624 programming skills if you're not up to snuff in programming. 761 1:12:49.624 --> 1:12:53.129 It's not going to be a long assignment, but if you don't 762 1:12:53.129 --> 1:12:55.932 know how to read a file and write out a file, 763 1:12:55.932 --> 1:12:58.608 and be able to write a dozen lines of code, 764 1:12:58.608 --> 1:13:02.176 OK, if you are weak on that, this weekend would be a good 765 1:13:02.176 --> 1:13:06 idea to practice reading in a text file. 766 1:13:06 --> 1:13:09.571 It's going to be a text file. Read it in a text file, 767 1:13:09.571 --> 1:13:12.524 decent manipulations, write out a text file, 768 1:13:12.524 --> 1:13:14.791 OK? So, I don't want people to get 769 1:13:14.791 --> 1:13:19.186 caught with this being mandatory and that not have time to finish 770 1:13:19.186 --> 1:13:23.513 it because they are busy trying to learn how to program in short 771 1:13:23.513 --> 1:13:25.848 order. I know some people take this 772 1:13:25.848 --> 1:13:31 course without quite getting all the programming prerequisites. 773 1:13:31 --> 1:13:33.964 Here's where you need it. Question? 774 1:13:33.964 --> 1:13:37.712 No language limitations. Pick your language. 775 1:13:37.712 --> 1:13:41.548 The answer will be written in, I think, Java, 776 1:13:41.548 --> 1:13:46.082 and Eric has graciously volunteered to use Python for 777 1:13:46.082 --> 1:13:51.138 his solution to this problem. We'll see whether he lives up 778 1:13:51.138 --> 1:13:53.928 to that promise. You did already? 779 1:13:53.928 --> 1:13:57.241 OK, and George wrote the Java solution. 780 1:13:57.241 --> 1:14:00.117 And so, C is fine. Matlab is fine. 781 1:14:00.117 --> 1:14:05 OK, what else is fine? Anything is fine. 782 1:14:05 --> 1:14:07.6 Scheme is fine. Scheme is fine. 783 1:14:07.6 --> 1:14:11.587 Scheme is great. OK, so any such things will be 784 1:14:11.587 --> 1:14:15.141 just fine. So, we don't care what language 785 1:14:15.141 --> 1:14:18.435 you program in, but you will have to do 786 1:14:18.435 --> 1:14:21.295 programming to solve this problem. 787 1:14:21.295 --> 1:14:24 OK, so thanks very much. See you next week.