1 00:00:00,050 --> 00:00:01,770 The following content is provided 2 00:00:01,770 --> 00:00:04,010 under a Creative Commons license. 3 00:00:04,010 --> 00:00:06,860 Your support will help MIT OpenCourseWare continue 4 00:00:06,860 --> 00:00:10,720 to offer high quality educational resources for free. 5 00:00:10,720 --> 00:00:13,330 To make a donation or view additional materials 6 00:00:13,330 --> 00:00:17,207 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:17,207 --> 00:00:17,832 at ocw.mit.edu. 8 00:00:22,537 --> 00:00:25,120 PROFESSOR: All right, today we continue our exciting adventure 9 00:00:25,120 --> 00:00:27,320 into dynamic programming. 10 00:00:27,320 --> 00:00:28,630 Are you excited? 11 00:00:28,630 --> 00:00:31,810 I'm excited, super excited. 12 00:00:31,810 --> 00:00:34,190 Dynamic programming, as you recall way back 13 00:00:34,190 --> 00:00:37,920 before Thanksgiving, is a super exciting powerful technique 14 00:00:37,920 --> 00:00:41,690 to design algorithms, especially to solve optimization problems 15 00:00:41,690 --> 00:00:44,670 where you want to maximize or minimize something. 16 00:00:44,670 --> 00:00:48,500 Last time, we saw how two algorithms we already 17 00:00:48,500 --> 00:00:52,380 knew-- namely, how to compute the nth Fibonacci number 18 00:00:52,380 --> 00:00:56,130 and how to compute shortest paths via Bellman-Ford-- 19 00:00:56,130 --> 00:00:58,780 are really dynamic programs in disguise. 20 00:00:58,780 --> 00:01:01,270 And indeed for, at least for Bellman-Ford, 21 00:01:01,270 --> 00:01:03,700 that's how they were invented, was 22 00:01:03,700 --> 00:01:05,170 to apply a general technique which 23 00:01:05,170 --> 00:01:09,160 we're going to see today in full generality, 24 00:01:09,160 --> 00:01:11,240 more or less-- most of this is generality-- 25 00:01:11,240 --> 00:01:13,230 in five easy steps. 26 00:01:13,230 --> 00:01:15,570 And we're going to see that technique applied 27 00:01:15,570 --> 00:01:17,840 to two new problems which are much more 28 00:01:17,840 --> 00:01:20,600 interesting than the ones we've already solved-- namely, 29 00:01:20,600 --> 00:01:23,170 how to make your text look nice in a paragraph, 30 00:01:23,170 --> 00:01:24,660 where to break the lines. 31 00:01:24,660 --> 00:01:26,320 That's text justification. 32 00:01:26,320 --> 00:01:29,340 And how to win and make loads of money at blackjack. 33 00:01:29,340 --> 00:01:31,950 So lots of practical stuff here, and we're 34 00:01:31,950 --> 00:01:35,980 going to see one new technique for 35 00:01:35,980 --> 00:01:37,820 general dynamic programming. 36 00:01:37,820 --> 00:01:40,470 These are some things I wrote last time. 37 00:01:40,470 --> 00:01:42,500 Actually, one of them I didn't write last time. 38 00:01:42,500 --> 00:01:44,500 In general, you can think of dynamic programming 39 00:01:44,500 --> 00:01:47,720 as a carefully executed brute force search. 40 00:01:47,720 --> 00:01:49,880 So in some sense, your algorithm is 41 00:01:49,880 --> 00:01:52,720 going to be trying all the possibilities, 42 00:01:52,720 --> 00:01:54,650 but somehow avoiding the fact that there 43 00:01:54,650 --> 00:01:56,550 are exponentially many of them. 44 00:01:56,550 --> 00:01:58,600 By thinking of it in a clever way, 45 00:01:58,600 --> 00:02:00,410 you can reduce the exponential search space 46 00:02:00,410 --> 00:02:02,700 down to a polynomial one, even though you're still 47 00:02:02,700 --> 00:02:05,490 not being very intelligent you're still blindly trying 48 00:02:05,490 --> 00:02:07,280 all possibilities. 49 00:02:07,280 --> 00:02:10,630 So that's the brute force part. 50 00:02:10,630 --> 00:02:12,970 In more detail, the three main techniques 51 00:02:12,970 --> 00:02:16,610 in dynamic programming are the idea of guessing, the idea 52 00:02:16,610 --> 00:02:20,280 that, oh, I want to find the best way to solve a problem. 53 00:02:20,280 --> 00:02:23,740 Let's pick out some feature of the solution 54 00:02:23,740 --> 00:02:25,540 that I want to know. 55 00:02:25,540 --> 00:02:28,680 I don't know it, so I'll guess the answer-- 56 00:02:28,680 --> 00:02:32,540 meaning I'll try all the possibilities for that choice 57 00:02:32,540 --> 00:02:34,920 and take the best one. 58 00:02:34,920 --> 00:02:38,020 So guessing is really central to dynamic programming. 59 00:02:38,020 --> 00:02:42,240 Then we also use a recursion, some way 60 00:02:42,240 --> 00:02:46,250 to express the solution to our problem in terms of solutions 61 00:02:46,250 --> 00:02:47,970 to sub-problems. 62 00:02:47,970 --> 00:02:50,119 So it's usually very easy to get a recursion 63 00:02:50,119 --> 00:02:51,660 for a lot of problems as long as they 64 00:02:51,660 --> 00:02:53,810 have some kind of substructure. 65 00:02:53,810 --> 00:02:56,390 Like shortest paths, we had that some paths of shortest paths 66 00:02:56,390 --> 00:02:59,310 were also shortest paths, so that was handy. 67 00:02:59,310 --> 00:03:02,190 Usually the recursion by itself is exponential time, 68 00:03:02,190 --> 00:03:04,140 like even with Fibonacci numbers. 69 00:03:04,140 --> 00:03:06,240 But we add in this technique of memoization, 70 00:03:06,240 --> 00:03:09,480 which is just once we compute an answer we've stored in a lookup 71 00:03:09,480 --> 00:03:11,180 table, if we ever need that answer again 72 00:03:11,180 --> 00:03:13,900 we reuse it instead of recomputing it. 73 00:03:13,900 --> 00:03:15,170 So we store it. 74 00:03:15,170 --> 00:03:19,610 We write down in our memo pad anything that we compute. 75 00:03:19,610 --> 00:03:22,360 Those techniques, all these techniques together 76 00:03:22,360 --> 00:03:24,610 give you, typically, a polynomial time 77 00:03:24,610 --> 00:03:26,880 dynamic program-- when they work, of course. 78 00:03:26,880 --> 00:03:29,500 Memoization makes the recursion polynomial time. 79 00:03:29,500 --> 00:03:33,260 The guessing is what is doing a brute force search. 80 00:03:33,260 --> 00:03:37,610 And magically, it all works if you're careful. 81 00:03:37,610 --> 00:03:41,397 Another perspective-- kind of an orthogonal perspective 82 00:03:41,397 --> 00:03:43,480 or another way of thinking about it, which I think 83 00:03:43,480 --> 00:03:45,146 should be comfortable for you because we 84 00:03:45,146 --> 00:03:47,430 spent a lot of time doing shortest paths 85 00:03:47,430 --> 00:03:50,730 and expressing problems that we care about in terms of shortest 86 00:03:50,730 --> 00:03:53,750 paths even if they don't look like it at first glance-- 87 00:03:53,750 --> 00:03:55,340 dynamic programming in some sense 88 00:03:55,340 --> 00:04:00,079 is always computing shortest paths in a DAG. 89 00:04:00,079 --> 00:04:01,870 So you have some problem you want to solve, 90 00:04:01,870 --> 00:04:05,020 like you have text you want to split up into lines so it looks 91 00:04:05,020 --> 00:04:11,100 nice in a paragraph, you express that problem somehow 92 00:04:11,100 --> 00:04:13,530 as a directed acyclic graph. 93 00:04:13,530 --> 00:04:15,620 And then we know how to compute shortest path 94 00:04:15,620 --> 00:04:18,160 in directed acyclic graphs in linear time. 95 00:04:18,160 --> 00:04:21,019 And that's basically what dynamic programming is doing. 96 00:04:21,019 --> 00:04:23,760 I didn't realize this until last week, 97 00:04:23,760 --> 00:04:24,990 so this is a new perspective. 98 00:04:24,990 --> 00:04:26,399 It's an experimental perspective. 99 00:04:26,399 --> 00:04:27,440 But I think it's helpful. 100 00:04:27,440 --> 00:04:30,910 It's actually-- dynamic programming is not that new. 101 00:04:30,910 --> 00:04:33,700 It's all about how to be clever in setting up that DAG. 102 00:04:33,700 --> 00:04:37,680 But in the end, the algorithm is very simple. 103 00:04:37,680 --> 00:04:41,300 And then we had this other perspective-- 104 00:04:41,300 --> 00:04:43,580 back to this perspective, I guess. 105 00:04:43,580 --> 00:04:47,020 In general, we have-- the real problem we want to solve, 106 00:04:47,020 --> 00:04:50,560 we generalize it in some sense by considering 107 00:04:50,560 --> 00:04:53,750 lots of different sub-problems that we might care about. 108 00:04:53,750 --> 00:04:56,700 Like with Fibonacci, we had the nth Fibonacci number. 109 00:04:59,082 --> 00:05:01,040 We really just wanted the nth Fibonacci number. 110 00:05:01,040 --> 00:05:04,410 But along the way, we're going to compute all f1 up to fn. 111 00:05:04,410 --> 00:05:06,430 So those are our sub-problems. 112 00:05:06,430 --> 00:05:08,180 And if we compute the amount of time 113 00:05:08,180 --> 00:05:10,389 we need to solve each sub-problem 114 00:05:10,389 --> 00:05:12,680 and multiply that by the number of sub-problems we get, 115 00:05:12,680 --> 00:05:14,900 the total time required by the algorithm. 116 00:05:14,900 --> 00:05:18,130 This is a general true fact. 117 00:05:18,130 --> 00:05:21,710 And the fun part here is we get to treat any recursive calls 118 00:05:21,710 --> 00:05:26,010 in this recursion as free, as constant time, 119 00:05:26,010 --> 00:05:29,310 because we really only pay for it first time. 120 00:05:29,310 --> 00:05:30,990 That's counted out here. 121 00:05:30,990 --> 00:05:33,360 The second time we call it, it's already memoized, 122 00:05:33,360 --> 00:05:36,300 so we don't have to pay for it. 123 00:05:36,300 --> 00:05:38,430 So this is, in some sense, an amortization, 124 00:05:38,430 --> 00:05:42,100 if you remember amortization from table doubling. 125 00:05:42,100 --> 00:05:44,030 We're just changing around when we 126 00:05:44,030 --> 00:05:46,000 count the cost of each sub-problem, 127 00:05:46,000 --> 00:05:49,390 and then this is the total running time. 128 00:05:49,390 --> 00:05:53,300 OK, so that's the spirit we saw already. 129 00:05:53,300 --> 00:05:55,180 I'm going to give you the five general steps, 130 00:05:55,180 --> 00:05:59,700 and then we're going to apply them to two new problems. 131 00:05:59,700 --> 00:06:07,590 So five easy steps to dynamic programming. 132 00:06:13,974 --> 00:06:16,390 Unfortunately, these are not necessarily sequential steps. 133 00:06:16,390 --> 00:06:17,980 They're a little bit interdependent, 134 00:06:17,980 --> 00:06:21,390 and so "easy" should be in quotes. 135 00:06:21,390 --> 00:06:26,510 This is how you would express a dynamic program, 136 00:06:26,510 --> 00:06:29,550 and in some sense how you'd invent one, 137 00:06:29,550 --> 00:06:32,740 but in particular how you would explain one. 138 00:06:32,740 --> 00:06:35,800 OK, let me get to the main steps first. 139 00:06:42,890 --> 00:06:46,230 First step is to figure out what your sub-problems are 140 00:06:46,230 --> 00:06:48,540 going to be. 141 00:06:48,540 --> 00:06:51,280 Second part is to guess something. 142 00:06:51,280 --> 00:06:59,780 Third step is to relate sub-problem solutions, usually 143 00:06:59,780 --> 00:07:02,610 with a recurrence. 144 00:07:02,610 --> 00:07:03,985 I guess always with a recurrence. 145 00:07:08,100 --> 00:07:11,595 Fourth step is to actually build an algorithm. 146 00:07:20,730 --> 00:07:22,700 And we saw two ways to do that last time. 147 00:07:22,700 --> 00:07:24,970 One is to use recursion and memoization, which 148 00:07:24,970 --> 00:07:26,470 is the way I like to think about it. 149 00:07:26,470 --> 00:07:33,150 But if you prefer, you can follow the bottom up approach. 150 00:07:33,150 --> 00:07:35,905 And usually that's called building a table. 151 00:07:40,910 --> 00:07:43,630 And that one's basically to turn our recursion and memoization, 152 00:07:43,630 --> 00:07:46,680 which is kind of fancy, into a bunch 153 00:07:46,680 --> 00:07:48,790 of for loops, which is pretty simple. 154 00:07:48,790 --> 00:07:52,550 And this is going to be more practical, faster, and so on. 155 00:07:52,550 --> 00:07:54,680 And depending on your preference, one of them 156 00:07:54,680 --> 00:07:56,290 is more intuitive than the other. 157 00:07:56,290 --> 00:07:57,060 It doesn't matter. 158 00:07:57,060 --> 00:08:02,306 They have the same running time, more or less, 159 00:08:02,306 --> 00:08:03,420 in the worst case. 160 00:08:07,860 --> 00:08:10,895 Then the fifth step is to solve the original problem. 161 00:08:18,946 --> 00:08:22,670 All right, so we've sort of seen this before. 162 00:08:22,670 --> 00:08:25,740 In fact I have, over here, a convenient table. 163 00:08:28,320 --> 00:08:30,520 It's called cheating. 164 00:08:30,520 --> 00:08:32,925 The two problems we saw last time, Fibonacci numbers 165 00:08:32,925 --> 00:08:34,929 and shortest paths. 166 00:08:34,929 --> 00:08:36,840 And I've got steps one, two, three, 167 00:08:36,840 --> 00:08:39,440 four-- I ran out of room, so I didn't write five yet. 168 00:08:39,440 --> 00:08:40,690 But we'll get there. 169 00:08:40,690 --> 00:08:42,890 So what are our sub-problems? 170 00:08:42,890 --> 00:08:47,430 Well, for Fibonacci, they were f1 through fn. 171 00:08:47,430 --> 00:08:49,300 So there were n different sub-problems. 172 00:08:49,300 --> 00:08:53,010 And in general because of this formula, 173 00:08:53,010 --> 00:08:57,360 we want to count how many sub-problems are there. 174 00:08:57,360 --> 00:09:07,830 So number of sub-problems is-- this 175 00:09:07,830 --> 00:09:10,610 is what we need to do algorithmically. 176 00:09:10,610 --> 00:09:13,130 And then for analysis, we want to counter 177 00:09:13,130 --> 00:09:15,640 number of sub-problems for step one. 178 00:09:15,640 --> 00:09:17,680 And so for Fibonacci there were n of them. 179 00:09:17,680 --> 00:09:24,490 For shortest paths, we defined this delta sub k of sv. 180 00:09:24,490 --> 00:09:26,260 This was the shortest path from s 181 00:09:26,260 --> 00:09:29,170 to v they uses at most k edges. 182 00:09:29,170 --> 00:09:31,840 That was sort of what Bellman-Ford was doing. 183 00:09:31,840 --> 00:09:34,690 And the number of different sub-problems 184 00:09:34,690 --> 00:09:37,990 here was v squared, because we had to do this for every vertex 185 00:09:37,990 --> 00:09:41,660 v and we had to do it for every value of k between 0 186 00:09:41,660 --> 00:09:44,850 and v minus 1. v minus was is the number of rounds 187 00:09:44,850 --> 00:09:46,970 we need in Bellman-Ford. 188 00:09:46,970 --> 00:09:50,720 So it's v times v, different sub-problems, 189 00:09:50,720 --> 00:09:52,460 b squared of them. 190 00:09:52,460 --> 00:09:56,540 OK, second thing was we wanted to solve our problem. 191 00:09:56,540 --> 00:10:00,990 And we do that by guessing some feature of the solution. 192 00:10:00,990 --> 00:10:03,150 In Fibonacci, there was no guessing. 193 00:10:03,150 --> 00:10:06,360 So the number of different choices for your guess is one. 194 00:10:06,360 --> 00:10:07,190 There's nothing. 195 00:10:07,190 --> 00:10:11,120 There's only one choice, which is to do nothing. 196 00:10:11,120 --> 00:10:15,280 And for shortest paths, what we guessed was-- 197 00:10:15,280 --> 00:10:18,800 we know we're looking for some path from s v. B Let's guess 198 00:10:18,800 --> 00:10:21,220 what the last edge is. 199 00:10:21,220 --> 00:10:25,020 There's some last edge from u to v, 200 00:10:25,020 --> 00:10:27,140 assuming the path has more than one edge-- or more 201 00:10:27,140 --> 00:10:29,330 than zero edges. 202 00:10:29,330 --> 00:10:31,230 When could the edge possibly be? 203 00:10:31,230 --> 00:10:33,890 Well, it's some incoming edge to v. 204 00:10:33,890 --> 00:10:36,420 So there's going to be indegree of v different choices 205 00:10:36,420 --> 00:10:37,740 for that. 206 00:10:37,740 --> 00:10:41,280 And to account for the case that that's zero, we do a plus 1. 207 00:10:41,280 --> 00:10:43,790 But that's not a big deal. 208 00:10:43,790 --> 00:10:47,560 So that was the number of different choices. 209 00:10:47,560 --> 00:10:49,640 In general if we're going to guess something, 210 00:10:49,640 --> 00:10:55,456 we need to write down the number of choices. 211 00:10:55,456 --> 00:10:59,310 For the guess, how many different possibilities 212 00:10:59,310 --> 00:11:01,250 are there? 213 00:11:01,250 --> 00:11:04,620 That's our analysis. 214 00:11:04,620 --> 00:11:09,150 OK, the next thing is the recurrence. 215 00:11:09,150 --> 00:11:10,750 That's step three. 216 00:11:10,750 --> 00:11:13,200 We want to relate all the sub-problem solutions 217 00:11:13,200 --> 00:11:14,310 to each other. 218 00:11:14,310 --> 00:11:15,865 For Fibonacci, that's the definition 219 00:11:15,865 --> 00:11:16,740 of Fibonacci numbers. 220 00:11:16,740 --> 00:11:19,180 So it's really easy. 221 00:11:19,180 --> 00:11:22,775 For shortest paths, we wrote this min. 222 00:11:22,775 --> 00:11:25,179 In general, typically it's a min or a max, 223 00:11:25,179 --> 00:11:26,720 whatever you're trying to solve here. 224 00:11:26,720 --> 00:11:28,830 We're doing shortest paths. 225 00:11:28,830 --> 00:11:31,780 You could do longest paths in the same way. 226 00:11:31,780 --> 00:11:36,550 So you compute them in of delta sub sk minus 1 of su. 227 00:11:36,550 --> 00:11:41,245 The idea is we want to compute this part of the path, the s 228 00:11:41,245 --> 00:11:42,450 to u part. 229 00:11:42,450 --> 00:11:44,970 And we know that has one fewer edge, because we just 230 00:11:44,970 --> 00:11:46,975 guessed what the last edge was. 231 00:11:46,975 --> 00:11:49,100 Except we don't really know what the last edge was, 232 00:11:49,100 --> 00:11:50,250 so we have to try them all. 233 00:11:50,250 --> 00:11:54,380 We try all the incoming edges into v-- that's this part-- 234 00:11:54,380 --> 00:11:56,810 and for each of them we compute-- 235 00:11:56,810 --> 00:11:58,770 I forgot something here. 236 00:11:58,770 --> 00:12:01,360 This is the cost of the first part of the path. 237 00:12:01,360 --> 00:12:05,470 Then I also need to do plus the weight of the uv edge. 238 00:12:05,470 --> 00:12:09,080 That will be the total cost of that path. 239 00:12:09,080 --> 00:12:12,350 You add those up, you do it for every incoming edge. 240 00:12:12,350 --> 00:12:16,250 That is, in some sense, considering all possible paths. 241 00:12:16,250 --> 00:12:18,250 Assuming you find the shortest path from s to u, 242 00:12:18,250 --> 00:12:20,360 that's going to be the best way to get there. 243 00:12:20,360 --> 00:12:25,380 And then use some edge from u to v for some choice of u. 244 00:12:25,380 --> 00:12:26,500 This will try all of them. 245 00:12:26,500 --> 00:12:28,333 So it's really trying all the possibilities. 246 00:12:28,333 --> 00:12:30,636 So it's pretty clear this is correct 247 00:12:30,636 --> 00:12:32,260 if there are no negative weight cycles. 248 00:12:32,260 --> 00:12:33,509 You have to prove some things. 249 00:12:33,509 --> 00:12:35,840 We've already proved them. 250 00:12:35,840 --> 00:12:40,390 It's just slow, but once you add memoization, it's fast. 251 00:12:40,390 --> 00:12:44,300 Now, how long does it take to evaluate this recurrence, 252 00:12:44,300 --> 00:12:46,380 constant time, if you don't count the recursive 253 00:12:46,380 --> 00:12:48,630 calls or count them as constant? 254 00:12:48,630 --> 00:12:52,770 Over here, we're taking a min over n degree of v things. 255 00:12:52,770 --> 00:12:56,210 So we have to pay n degree of v time, 256 00:12:56,210 --> 00:12:58,020 again the recursions as free. 257 00:12:58,020 --> 00:13:00,620 But for each one of them, we have to do an addition. 258 00:13:00,620 --> 00:13:04,459 So it's constant work per guess. 259 00:13:04,459 --> 00:13:05,500 And this is quite common. 260 00:13:05,500 --> 00:13:09,110 Often, the number of guesses and the running time 261 00:13:09,110 --> 00:13:11,554 per sub-problem are the same, the constant factors. 262 00:13:11,554 --> 00:13:12,720 Sometimes they're different. 263 00:13:12,720 --> 00:13:15,860 We'll see some examples today. 264 00:13:15,860 --> 00:13:17,065 OK, step four. 265 00:13:19,700 --> 00:13:20,220 Let's see. 266 00:13:20,220 --> 00:13:24,770 So here we evaluate the time per sub-problem. 267 00:13:24,770 --> 00:13:26,950 Once you have the recurrence, that becomes clear. 268 00:13:26,950 --> 00:13:29,130 You want to make sure that's polynomial. 269 00:13:29,130 --> 00:13:30,450 Often these are the same. 270 00:13:33,910 --> 00:13:38,062 And then we add the recursive memorize or build a DP table. 271 00:13:38,062 --> 00:13:39,270 I'm not going to write those. 272 00:13:39,270 --> 00:13:41,580 We did it for Fibonacci last time, shortest paths. 273 00:13:41,580 --> 00:13:43,780 Pretty easy. 274 00:13:43,780 --> 00:13:49,470 And in general, what we need to check here 275 00:13:49,470 --> 00:13:55,000 is that the sub problem recurrence is acyclic. 276 00:14:04,750 --> 00:14:08,260 In other words, that it has a topological order so 277 00:14:08,260 --> 00:14:11,400 we can use topological sort. 278 00:14:11,400 --> 00:14:14,110 We don't actually use topological algorithm usually. 279 00:14:14,110 --> 00:14:16,226 You can just think about it. 280 00:14:16,226 --> 00:14:17,600 In the case of Fibonacci numbers, 281 00:14:17,600 --> 00:14:19,430 it's clear you want to start with the smallest one 282 00:14:19,430 --> 00:14:20,430 and end up with the biggest one. 283 00:14:20,430 --> 00:14:22,310 You can't do the reverse, because then when 284 00:14:22,310 --> 00:14:24,684 you're trying to computer the nth you don't have the ones 285 00:14:24,684 --> 00:14:26,490 you need, the n minus 1 and n minus 2. 286 00:14:26,490 --> 00:14:28,240 But if you do it in this order, you always 287 00:14:28,240 --> 00:14:30,560 have the one you need by the time you get there. 288 00:14:30,560 --> 00:14:33,630 In general, there's a DAG there-- 289 00:14:33,630 --> 00:14:40,450 and for Fibonacci, it was like this. 290 00:14:40,450 --> 00:14:42,080 Every node depends on the previous 291 00:14:42,080 --> 00:14:44,670 and the second previous. 292 00:14:44,670 --> 00:14:46,570 But you just choose a topological order, 293 00:14:46,570 --> 00:14:50,740 which is here left to right, and you're golden. 294 00:14:50,740 --> 00:14:52,240 And these are actually the for loops 295 00:14:52,240 --> 00:14:54,396 you get in the bottom of DP. 296 00:14:54,396 --> 00:14:56,520 For shortest paths, you have to think a little bit. 297 00:14:56,520 --> 00:15:00,160 You have to do the for loop over k on the outside, 298 00:15:00,160 --> 00:15:01,770 the for loop over V on the inside. 299 00:15:01,770 --> 00:15:04,370 The reverse does not work. 300 00:15:04,370 --> 00:15:07,380 I won't go through that, but we drew the DAG last time. 301 00:15:07,380 --> 00:15:10,550 And that's the main thing you need to do here. 302 00:15:10,550 --> 00:15:12,710 And then, of course, you use this formula 303 00:15:12,710 --> 00:15:16,820 to compute the overall running time, which is just multiplying 304 00:15:16,820 --> 00:15:19,510 this quantity with this quantity. 305 00:15:25,350 --> 00:15:28,730 Total time. 306 00:15:28,730 --> 00:15:30,602 Then there's just one last step that 307 00:15:30,602 --> 00:15:33,060 usually isn't that big a deal, but you have think about it. 308 00:15:33,060 --> 00:15:35,185 You need to make sure that the problem you actually 309 00:15:35,185 --> 00:15:37,820 cared about solving gets solved. 310 00:15:37,820 --> 00:15:40,230 In the case of Fibonacci and shortest paths, 311 00:15:40,230 --> 00:15:41,580 this is pretty clear. 312 00:15:41,580 --> 00:15:42,330 I didn't write it. 313 00:15:46,623 --> 00:15:49,710 We can do it on here. 314 00:15:49,710 --> 00:15:50,845 Solve the original problem. 315 00:15:54,580 --> 00:15:55,880 Fibonaci, it is Fn. 316 00:15:55,880 --> 00:15:57,460 And this is one of our sub-problems, 317 00:15:57,460 --> 00:15:59,360 so if we solve all of them, we're done. 318 00:15:59,360 --> 00:16:01,260 For shortest paths, it's basically 319 00:16:01,260 --> 00:16:10,350 delta sub v minus 1 of sv for all v. That's 320 00:16:10,350 --> 00:16:12,100 single source shortest paths. 321 00:16:12,100 --> 00:16:13,990 And by our Bellman-Ford analysis, 322 00:16:13,990 --> 00:16:15,880 that gives us the right shortest paths. 323 00:16:15,880 --> 00:16:17,590 There are no negative weight cycles. 324 00:16:17,590 --> 00:16:22,420 And sometimes this requires extra time 325 00:16:22,420 --> 00:16:25,000 to combine your solutions to get the real thing. 326 00:16:25,000 --> 00:16:29,220 Here of course, we just have the answers, so writing them down 327 00:16:29,220 --> 00:16:31,810 does not take very long. 328 00:16:31,810 --> 00:16:35,100 So that's the dominant running time-- which I didn't write, 329 00:16:35,100 --> 00:16:38,780 I should have written in under for here-- this ends up 330 00:16:38,780 --> 00:16:42,601 being n, this ends up being VE. 331 00:16:45,280 --> 00:16:47,530 OK, I don't want to spend more time on those examples. 332 00:16:47,530 --> 00:16:48,488 Let's go to new things. 333 00:17:07,410 --> 00:17:11,210 So first problem we're going to look at today 334 00:17:11,210 --> 00:17:12,245 is text justification. 335 00:17:21,220 --> 00:17:23,970 And the informal statement of this problem 336 00:17:23,970 --> 00:17:26,310 is you're given some text-- which 337 00:17:26,310 --> 00:17:30,170 means a string, a whole bunch of characters. 338 00:17:30,170 --> 00:17:33,195 And we want to split them into good lines. 339 00:17:36,130 --> 00:17:39,410 The rules of the game here are we're going to, 340 00:17:39,410 --> 00:17:41,720 like in the early lectures of document distance 341 00:17:41,720 --> 00:17:44,320 where you have some definition of splitting 342 00:17:44,320 --> 00:17:48,790 a document into words separated by spaces. 343 00:17:48,790 --> 00:17:51,980 And what we want to do is cut. 344 00:17:51,980 --> 00:17:54,970 We can only cut between word boundaries. 345 00:17:54,970 --> 00:17:58,630 And we want to write some text, it's 346 00:17:58,630 --> 00:18:00,160 going to have some spaces in it. 347 00:18:00,160 --> 00:18:06,270 Then there's a new line, something like that. 348 00:18:06,270 --> 00:18:09,536 And we want to justify our text on the right here. 349 00:18:09,536 --> 00:18:11,910 And so we'd like to avoid big gaps like this because they 350 00:18:11,910 --> 00:18:15,330 look ugly, they're hard to read. 351 00:18:15,330 --> 00:18:20,070 Now, if you use Microsoft Word-- at least 352 00:18:20,070 --> 00:18:22,400 before the latest versions-- they 353 00:18:22,400 --> 00:18:24,440 follow a greedy strategy, which is very simple. 354 00:18:24,440 --> 00:18:26,710 You pack as many words as you can on the first line, 355 00:18:26,710 --> 00:18:28,710 then you go to the next line, pack as many words 356 00:18:28,710 --> 00:18:30,270 as you can on the second line. 357 00:18:30,270 --> 00:18:31,720 Keep going like that. 358 00:18:31,720 --> 00:18:34,900 And that strategy is not optimal. 359 00:18:34,900 --> 00:18:38,070 If you use LaTeX-- as some of you 360 00:18:38,070 --> 00:18:41,100 have been doing on problem sets, and I think also new versions 361 00:18:41,100 --> 00:18:43,840 of Word but I'm not sure-- then it 362 00:18:43,840 --> 00:18:46,140 uses dynamic programming to solve this problem. 363 00:18:46,140 --> 00:18:48,380 And that's what we're going to do here. 364 00:18:48,380 --> 00:18:53,560 So let me specify a little bit more about what we mean here. 365 00:18:53,560 --> 00:18:57,055 So the text we're going to think of as a list of words. 366 00:19:01,010 --> 00:19:05,330 And we're going to define a quantity badness. 367 00:19:08,590 --> 00:19:12,530 And this is an anesthetic quantity, if you will. 368 00:19:12,530 --> 00:19:15,190 I'm going to tell you what LaTeX uses. 369 00:19:15,190 --> 00:19:20,098 But this is sort of how bad it is 370 00:19:20,098 --> 00:19:31,010 to use-- or let's say, yeah, words i through j as a line. 371 00:19:36,410 --> 00:19:37,700 So this is python notation. 372 00:19:37,700 --> 00:19:40,090 So it starts at i and ends at J minus 1. 373 00:19:40,090 --> 00:19:42,090 That'll be convenient. 374 00:19:42,090 --> 00:19:43,800 So I have this list of words. 375 00:19:43,800 --> 00:19:46,440 And if I look at words i through j minus 1 376 00:19:46,440 --> 00:19:50,630 and I think of what happens if I pack them in a line, well, 377 00:19:50,630 --> 00:19:54,080 they may fit or they may not fit. 378 00:19:54,080 --> 00:19:57,000 So there are going to be two cases. 379 00:19:57,000 --> 00:19:59,930 If they don't fit, I'm going to write infinity. 380 00:19:59,930 --> 00:20:00,805 So that's really bad. 381 00:20:07,890 --> 00:20:13,360 So I have some notion of how wide my line can be. 382 00:20:13,360 --> 00:20:15,760 And if the sum of the lengths of those words plus the sum 383 00:20:15,760 --> 00:20:18,890 of the lengths of the spaces as small as possible 384 00:20:18,890 --> 00:20:23,370 is bigger than the width of my screen-- or page, 385 00:20:23,370 --> 00:20:25,296 I guess-- then I say they don't fit, 386 00:20:25,296 --> 00:20:27,420 and then I define badness to be infinity-- meaning, 387 00:20:27,420 --> 00:20:30,210 I never want to do that. 388 00:20:30,210 --> 00:20:32,530 This is actually LaTeX sloppy mode, 389 00:20:32,530 --> 00:20:34,850 if you want to be technical. 390 00:20:34,850 --> 00:20:49,120 Otherwise, it's going to be page width minus total width cubed. 391 00:20:52,500 --> 00:20:53,110 Why cubed? 392 00:20:53,110 --> 00:20:54,770 Who knows. 393 00:20:54,770 --> 00:20:57,700 This is the LaTeX rule. 394 00:20:57,700 --> 00:21:01,620 And squared would probably also be fine. 395 00:21:01,620 --> 00:21:04,280 So this is the width of the page minus the total width 396 00:21:04,280 --> 00:21:06,560 of those words, which you also have 397 00:21:06,560 --> 00:21:08,120 to include the spaces here. 398 00:21:08,120 --> 00:21:09,120 You take the difference. 399 00:21:09,120 --> 00:21:10,130 You cube it. 400 00:21:10,130 --> 00:21:12,550 And so when this is small-- I mean, 401 00:21:12,550 --> 00:21:14,510 when these are very close-- then this 402 00:21:14,510 --> 00:21:16,550 is going to be close to zero. 403 00:21:16,550 --> 00:21:17,450 That's good. 404 00:21:17,450 --> 00:21:19,231 That means you use most of the line. 405 00:21:19,231 --> 00:21:21,605 When the total width is much smaller than the page width, 406 00:21:21,605 --> 00:21:22,938 then this will be a large value. 407 00:21:22,938 --> 00:21:24,460 You cube it, it will be even larger. 408 00:21:24,460 --> 00:21:27,930 So this will highly discourage big gaps like this. 409 00:21:27,930 --> 00:21:31,120 And it will very much discourage not fitting. 410 00:21:31,120 --> 00:21:32,950 So there's a tradeoff, of course. 411 00:21:32,950 --> 00:21:35,520 And the idea is you might-- in the greedy algorithm, 412 00:21:35,520 --> 00:21:38,800 you make the first line as good as you can. 413 00:21:38,800 --> 00:21:40,869 But it might actually be better to leave out 414 00:21:40,869 --> 00:21:42,410 some of the words that would fit here 415 00:21:42,410 --> 00:21:44,360 in order to make the next line better. 416 00:21:44,360 --> 00:21:46,600 In general, it's hard to tell, where should I 417 00:21:46,600 --> 00:21:51,050 cut the lines in order to get the best overall strategy? 418 00:21:51,050 --> 00:21:52,890 What I'd like to minimize is the sum 419 00:21:52,890 --> 00:21:55,430 of the badnesses of the lines. 420 00:21:55,430 --> 00:21:57,590 So it's a sum of cubes, and that's really hard 421 00:21:57,590 --> 00:21:59,579 to think about. 422 00:21:59,579 --> 00:22:01,370 But that's what dynamic programming is for. 423 00:22:01,370 --> 00:22:02,800 You don't have to think. 424 00:22:02,800 --> 00:22:06,010 It's great because it's brute force. 425 00:22:06,010 --> 00:22:12,600 OK, so the first thing we need to do is define sub-problems. 426 00:22:12,600 --> 00:22:15,030 This is, in some sense, the hard part. 427 00:22:15,030 --> 00:22:18,220 The rest will follow easily. 428 00:22:18,220 --> 00:22:21,460 So I think actually it might be easier 429 00:22:21,460 --> 00:22:24,150 to think about, for this problem, what 430 00:22:24,150 --> 00:22:26,190 would be the brute force strategy? 431 00:22:26,190 --> 00:22:28,440 How would you try all possibilities, exponential 432 00:22:28,440 --> 00:22:29,330 time? 433 00:22:29,330 --> 00:22:29,830 Suggestions? 434 00:22:40,414 --> 00:22:40,914 Yeah? 435 00:22:40,914 --> 00:22:44,240 AUDIENCE: Try all partitions of the words that don't fit? 436 00:22:44,240 --> 00:22:45,990 PROFESSOR: Try all partitions of the word, 437 00:22:45,990 --> 00:22:48,970 so-- of the string of words. 438 00:22:48,970 --> 00:22:51,797 So I mean, it could be it all fits in on one line. 439 00:22:51,797 --> 00:22:53,380 It could be it's split into two lines. 440 00:22:53,380 --> 00:22:55,320 I try all possible splits there. 441 00:22:55,320 --> 00:22:57,460 In general, I'm guessing for every word, 442 00:22:57,460 --> 00:22:59,850 does this start a line or not? 443 00:22:59,850 --> 00:23:01,470 That would be all ways. 444 00:23:01,470 --> 00:23:02,999 And so there are 2 to the n. 445 00:23:02,999 --> 00:23:05,290 If I have n words, there's 2 to the n different splits. 446 00:23:05,290 --> 00:23:08,650 For every word I say yes or no, does this is begin a line? 447 00:23:08,650 --> 00:23:11,331 So what I'd like to figure out is where those lines begin. 448 00:23:11,331 --> 00:23:12,830 That was the point of that exercise. 449 00:23:16,010 --> 00:23:17,000 So any suggestions? 450 00:23:17,000 --> 00:23:19,290 Maybe it's actually easier to jump ahead and think, 451 00:23:19,290 --> 00:23:21,800 what would I guess in my solution 452 00:23:21,800 --> 00:23:25,450 if I have this big string of words? 453 00:23:25,450 --> 00:23:27,500 What's the natural first thing to guess? 454 00:23:34,200 --> 00:23:34,700 Yeah? 455 00:23:34,700 --> 00:23:36,620 AUDIENCE: Guess how long the first line is? 456 00:23:36,620 --> 00:23:38,740 PROFESSOR: Guess how long the first line is, yeah. 457 00:23:38,740 --> 00:23:40,490 We know that the first word begins a line. 458 00:23:40,490 --> 00:23:44,580 But where does the second line begin? 459 00:23:44,580 --> 00:23:47,120 So I'd like to guess where the second line begins. 460 00:23:47,120 --> 00:23:51,000 That's-- so you know, I have the beginning of a line here 461 00:23:51,000 --> 00:23:55,820 and then I have a beginning of a line here at the fourth word. 462 00:23:55,820 --> 00:23:57,810 Where does the second line begin? 463 00:23:57,810 --> 00:23:58,590 I don't know. 464 00:23:58,590 --> 00:23:59,180 Guess. 465 00:23:59,180 --> 00:24:01,200 So I'm going to try all the possible words 466 00:24:01,200 --> 00:24:03,010 after the first word. 467 00:24:03,010 --> 00:24:06,019 And say, well, what if I started my second line here? 468 00:24:06,019 --> 00:24:07,560 At some point I'm going to be packing 469 00:24:07,560 --> 00:24:10,600 too much into the first line, and so I abort. 470 00:24:10,600 --> 00:24:14,720 But I'll try them all. 471 00:24:14,720 --> 00:24:16,310 Why not? 472 00:24:16,310 --> 00:24:18,070 OK, that's good. 473 00:24:18,070 --> 00:24:19,600 The issue is that once I've chosen 474 00:24:19,600 --> 00:24:21,590 where the second line is, of course the next thing I want 475 00:24:21,590 --> 00:24:23,220 to guess is where the third line begins. 476 00:24:23,220 --> 00:24:25,428 And then I want I guess where the fourth line begins, 477 00:24:25,428 --> 00:24:26,230 and so on. 478 00:24:26,230 --> 00:24:28,240 In general, I need to set up my sub-problems 479 00:24:28,240 --> 00:24:31,150 so that after I do the first guess 480 00:24:31,150 --> 00:24:34,660 I have the problem of the original type. 481 00:24:34,660 --> 00:24:37,530 So originally I have all the words. 482 00:24:37,530 --> 00:24:41,210 But after I guess where the second line begins, 483 00:24:41,210 --> 00:24:43,345 I have the remaining words. 484 00:24:45,870 --> 00:24:48,930 What's a good word for the remaining words? 485 00:24:48,930 --> 00:24:52,470 If I give you a list of words and I want from here on, 486 00:24:52,470 --> 00:24:57,538 it's called-- what? 487 00:24:57,538 --> 00:24:59,050 A sub-problem, yes. 488 00:24:59,050 --> 00:25:02,380 That's what we want to define. 489 00:25:02,380 --> 00:25:04,684 It's called a suffix of the array. 490 00:25:04,684 --> 00:25:06,100 That's the word I was looking for. 491 00:25:06,100 --> 00:25:09,210 It's tough when I only have one word answers. 492 00:25:09,210 --> 00:25:11,370 So my sub-problems are going to be suffixes. 493 00:25:16,080 --> 00:25:20,570 Which is, in python notation, i colon. 494 00:25:20,570 --> 00:25:21,445 They call it splices. 495 00:25:24,990 --> 00:25:29,796 And how many sub-problems are there if I have n words? 496 00:25:33,510 --> 00:25:35,870 Two? 497 00:25:35,870 --> 00:25:36,683 Sorry? 498 00:25:36,683 --> 00:25:37,579 AUDIENCE: 2 to the n. 499 00:25:37,579 --> 00:25:38,370 PROFESSOR: 2 the n? 500 00:25:41,150 --> 00:25:43,700 That would be a problem if it's 2 to the n. 501 00:25:43,700 --> 00:25:44,690 I hope it's only n. 502 00:25:49,980 --> 00:25:52,820 Originally, we said, OK, for every word, we're going to say, 503 00:25:52,820 --> 00:25:54,152 is this in our out? 504 00:25:54,152 --> 00:25:55,360 Is this the beginning or not? 505 00:25:55,360 --> 00:25:56,810 That's 2 to the n. 506 00:25:56,810 --> 00:25:59,430 But here, the idea is we're only thinking about, well, 507 00:25:59,430 --> 00:26:00,742 what are the words that remain? 508 00:26:00,742 --> 00:26:02,950 And it could be you've dealt with the first 100 words 509 00:26:02,950 --> 00:26:04,430 and then you've got n minus 100 left, 510 00:26:04,430 --> 00:26:06,155 or it could be you've dealt with the first thousand words 511 00:26:06,155 --> 00:26:07,363 and you've got n minus 1,000. 512 00:26:07,363 --> 00:26:10,890 There's only n choices for that. 513 00:26:10,890 --> 00:26:14,540 We're only remembering one line, this is the key. 514 00:26:14,540 --> 00:26:17,290 Even though we may have already guessed several lines, 515 00:26:17,290 --> 00:26:19,110 we're just going to remember, well, OK. 516 00:26:19,110 --> 00:26:20,770 This is what we have left to do. 517 00:26:20,770 --> 00:26:22,900 So let's forget about the past. 518 00:26:22,900 --> 00:26:25,960 This is what makes dynamic programming efficient. 519 00:26:25,960 --> 00:26:29,250 And we're just going to solve it, solve these sub-problems, 520 00:26:29,250 --> 00:26:30,910 forgetting about the past. 521 00:26:30,910 --> 00:26:32,530 So the sub-problem-- I'm not going 522 00:26:32,530 --> 00:26:35,120 to write it here-- is if I give you these words, 523 00:26:35,120 --> 00:26:38,420 never mind the other words, how do I pack them optimally 524 00:26:38,420 --> 00:26:40,290 into a paragraph? 525 00:26:40,290 --> 00:26:42,920 I don't care about the other words, just these words. 526 00:26:42,920 --> 00:26:45,355 So this is a different version of the same problem. 527 00:26:45,355 --> 00:26:46,730 Initially, we have n words to do. 528 00:26:46,730 --> 00:26:48,970 Now I have n minus i words to do. 529 00:26:48,970 --> 00:26:50,980 But it's again text justification. 530 00:26:50,980 --> 00:26:54,576 I want to solve this problem on those words. 531 00:26:54,576 --> 00:26:56,200 That's just how I'm going to define it. 532 00:26:56,200 --> 00:27:00,890 This will work if I can specify a recurrence relation. 533 00:27:00,890 --> 00:27:06,190 As we said, what we guess is where 534 00:27:06,190 --> 00:27:09,450 to break the first line, where to start 535 00:27:09,450 --> 00:27:16,870 the second line for those words. 536 00:27:16,870 --> 00:27:23,690 OK, so this is-- it could be the i plus first line. 537 00:27:23,690 --> 00:27:26,930 It could be the i plus second line-- or sorry, word. 538 00:27:26,930 --> 00:27:31,740 Some word after i is where we guess the second word. 539 00:27:31,740 --> 00:27:39,450 The number of choices for the guess is at most n minus i. 540 00:27:39,450 --> 00:27:43,040 I'm just going to think of that as order n. 541 00:27:43,040 --> 00:27:45,220 It won't matter. 542 00:27:45,220 --> 00:27:47,680 The third part is we need a recurrence relation. 543 00:27:53,540 --> 00:27:56,450 I claim this is very easy. 544 00:27:56,450 --> 00:28:02,130 I'm going to-- I didn't give this problem a name, 545 00:28:02,130 --> 00:28:06,982 so I'm just going to write it as DP of i. 546 00:28:09,730 --> 00:28:13,500 So this is going to be the solution to that suffix, words 547 00:28:13,500 --> 00:28:14,500 from i onward. 548 00:28:18,030 --> 00:28:21,370 And I'd like to-- what I want to do 549 00:28:21,370 --> 00:28:24,389 is consider all possible guesses. 550 00:28:24,389 --> 00:28:26,930 So I mean this is going to be pretty formulaic at this point. 551 00:28:26,930 --> 00:28:29,290 After I've set up these ideas there's pretty much 552 00:28:29,290 --> 00:28:31,320 only one thing I can write here, which 553 00:28:31,320 --> 00:28:36,960 is I want to do a for loop. 554 00:28:48,080 --> 00:28:50,380 That would be the for loop of where 555 00:28:50,380 --> 00:28:52,680 the second line can start. 556 00:28:52,680 --> 00:28:54,140 I can't start at i, because that's 557 00:28:54,140 --> 00:28:55,580 where the first line starts. 558 00:28:55,580 --> 00:28:58,510 But it could start at i plus 1. 559 00:28:58,510 --> 00:29:02,480 And this special value of n will mean 560 00:29:02,480 --> 00:29:06,044 that there is no second line. 561 00:29:06,044 --> 00:29:09,420 OK, so DP of i-- now I want to do this for loop 562 00:29:09,420 --> 00:29:11,320 in order to try all the possible guesses. 563 00:29:11,320 --> 00:29:14,320 j will be the word where the next thing starts. 564 00:29:14,320 --> 00:29:16,200 So then what do I write up here? 565 00:29:18,890 --> 00:29:24,850 If I make this guess-- all right, so I have word 566 00:29:24,850 --> 00:29:29,730 i is the first word of the first line. 567 00:29:29,730 --> 00:29:32,630 And then word j is the first word of the second line. 568 00:29:35,105 --> 00:29:36,730 And then there's more stuff down below. 569 00:29:36,730 --> 00:29:39,000 I don't know what that is. 570 00:29:39,000 --> 00:29:44,240 But how can I use recursion to specify this? 571 00:29:49,480 --> 00:29:52,170 DP of j, exactly. 572 00:29:55,650 --> 00:29:57,540 I guess if I'm doing recursion, I 573 00:29:57,540 --> 00:30:00,580 should use parentheses instead of brackets. 574 00:30:00,580 --> 00:30:02,014 But if you're doing it bottom up, 575 00:30:02,014 --> 00:30:03,180 it would be square brackets. 576 00:30:05,760 --> 00:30:07,060 So that's just DP of j. 577 00:30:07,060 --> 00:30:12,170 That's the cost of the rest of the problem. 578 00:30:12,170 --> 00:30:15,140 And I can assume that that's free to compute. 579 00:30:15,140 --> 00:30:17,330 This is the magic of dynamic programming. 580 00:30:17,330 --> 00:30:20,490 But then I also have to think about, well, 581 00:30:20,490 --> 00:30:21,900 what about the first line? 582 00:30:21,900 --> 00:30:24,140 How much does that cost? 583 00:30:24,140 --> 00:30:26,819 Well, that's just badness of ij. 584 00:30:26,819 --> 00:30:28,110 And we've already defined that. 585 00:30:28,110 --> 00:30:29,952 We can compute it in constant time. 586 00:30:29,952 --> 00:30:32,160 Dynamic programming doesn't really care what this is. 587 00:30:32,160 --> 00:30:33,480 It could be anything. 588 00:30:33,480 --> 00:30:34,980 As long as you're trying to minimize 589 00:30:34,980 --> 00:30:37,490 the sum of the badnesses, whatever function is in here, 590 00:30:37,490 --> 00:30:40,164 we just compute it here. 591 00:30:40,164 --> 00:30:41,830 That's the power of dynamic programming. 592 00:30:41,830 --> 00:30:43,900 It works for all variations of this problem, 593 00:30:43,900 --> 00:30:45,480 however you define badness. 594 00:30:45,480 --> 00:30:47,480 So you might say, oh, that's a weird definition. 595 00:30:47,480 --> 00:30:49,050 I want to use something else instead. 596 00:30:49,050 --> 00:30:50,799 That's fine, as long as you can compute it 597 00:30:50,799 --> 00:30:55,520 in terms of just i and j and looking at those words. 598 00:30:55,520 --> 00:30:58,492 OK, now I need to do a min over the whole thing. 599 00:30:58,492 --> 00:31:00,450 So I want to minimize the sum of the badnesses. 600 00:31:03,240 --> 00:31:05,815 So I compute for every guess of j, 601 00:31:05,815 --> 00:31:07,690 I compute the cost of the rest of the problem 602 00:31:07,690 --> 00:31:10,590 plus the cost of that first line. 603 00:31:10,590 --> 00:31:13,706 And this, is in some sense, checking all possible solutions 604 00:31:13,706 --> 00:31:14,205 magically. 605 00:31:16,880 --> 00:31:18,320 OK. 606 00:31:18,320 --> 00:31:20,360 That's the recurrence. 607 00:31:20,360 --> 00:31:23,370 We need to check some things. 608 00:31:23,370 --> 00:31:26,430 I guess right now we just want to compute how much time does 609 00:31:26,430 --> 00:31:28,710 this cost, time per sub-problem. 610 00:31:35,110 --> 00:31:38,560 To do this for loop, basically I do constant work-- all of this 611 00:31:38,560 --> 00:31:40,910 is constant work-- for each choice. 612 00:31:40,910 --> 00:31:43,960 So there's order n choices, so this is order n. 613 00:31:51,766 --> 00:31:53,140 Now we have to check that there's 614 00:31:53,140 --> 00:32:00,740 a topological order for this problem 615 00:32:00,740 --> 00:32:02,920 or for these sub-problems. 616 00:32:02,920 --> 00:32:06,210 And this is easy, but a little different 617 00:32:06,210 --> 00:32:08,590 from what we've done before because we 618 00:32:08,590 --> 00:32:11,970 have to actually work from the end backwards, 619 00:32:11,970 --> 00:32:14,120 because we're expressing DP of i in terms 620 00:32:14,120 --> 00:32:16,280 of DP of larger values of i. 621 00:32:16,280 --> 00:32:18,562 j is always bigger than i. 622 00:32:18,562 --> 00:32:20,520 And so we have to do it from the right end back 623 00:32:20,520 --> 00:32:21,228 to the beginning. 624 00:32:27,209 --> 00:32:30,260 And n minus 1 down to 0. 625 00:32:30,260 --> 00:32:32,670 I didn't actually define DP of n. 626 00:32:32,670 --> 00:32:42,830 There's a base case here which is DP of n equals 0. 627 00:32:42,830 --> 00:32:45,430 Because the meaning of DP of n is I have zero words, 628 00:32:45,430 --> 00:32:46,740 the nth word onward. 629 00:32:46,740 --> 00:32:47,920 There is no nth word. 630 00:32:47,920 --> 00:32:50,890 It's 0 to n minus 1 in this notation. 631 00:32:50,890 --> 00:32:53,150 So I don't pay anything for a blank line. 632 00:32:56,030 --> 00:32:59,410 OK, so that's our top logical order. 633 00:32:59,410 --> 00:33:01,720 This one, of course, is instantaneous. 634 00:33:01,720 --> 00:33:03,240 And then we work backwards. 635 00:33:03,240 --> 00:33:05,450 And always whenever we need to compute something, 636 00:33:05,450 --> 00:33:08,760 we already have the value. 637 00:33:08,760 --> 00:33:14,634 The total time we get is going to be 638 00:33:14,634 --> 00:33:16,050 the number of sub problems-- which 639 00:33:16,050 --> 00:33:19,820 is n times the running time per sub-problem. 640 00:33:19,820 --> 00:33:23,220 which is order n, which is order n squared. 641 00:33:23,220 --> 00:33:26,889 And in the worst case, it is indeed theta n squared. 642 00:33:26,889 --> 00:33:29,430 Although in practice it's going to work better, because lines 643 00:33:29,430 --> 00:33:32,350 can't be too long. 644 00:33:32,350 --> 00:33:33,680 So that's the running time. 645 00:33:33,680 --> 00:33:36,050 Then finally we have to check that the original problem 646 00:33:36,050 --> 00:33:38,150 actually gets solved. 647 00:33:38,150 --> 00:33:40,870 And in this case, the original problem we need to solve 648 00:33:40,870 --> 00:33:47,850 is DP of 0 because DP of 0 means I take words from 0 onwards. 649 00:33:47,850 --> 00:33:49,740 That's everybody. 650 00:33:49,740 --> 00:33:52,500 So that's the actual problem I want to solve. 651 00:33:52,500 --> 00:33:53,410 So we work backwards. 652 00:33:53,410 --> 00:33:55,090 We solve all these sub-problems that we don't directly 653 00:33:55,090 --> 00:33:57,490 care about, but then the first one is the one we want. 654 00:33:57,490 --> 00:33:58,620 And we're done. 655 00:33:58,620 --> 00:34:01,330 So in quadratic time, we can find the best way 656 00:34:01,330 --> 00:34:05,140 to pack words into lines. 657 00:34:05,140 --> 00:34:06,042 Question? 658 00:34:06,042 --> 00:34:06,917 AUDIENCE: [INAUDIBLE] 659 00:34:09,739 --> 00:34:11,330 PROFESSOR: DP of j is returning. 660 00:34:11,330 --> 00:34:12,280 It's like this. 661 00:34:12,280 --> 00:34:14,570 So DP of-- this is a recursive definition. 662 00:34:14,570 --> 00:34:16,570 Imagine this is a recursive function. 663 00:34:16,570 --> 00:34:20,790 I wrote equals, which is Haskell notation, if you will. 664 00:34:20,790 --> 00:34:23,969 But normally, you think of this as like def DP of i 665 00:34:23,969 --> 00:34:26,380 is return min of this. 666 00:34:26,380 --> 00:34:28,010 This is python. 667 00:34:28,010 --> 00:34:31,880 So it's returning the cost. 668 00:34:31,880 --> 00:34:35,360 What was the best way to pack those lines from j onwards? 669 00:34:35,360 --> 00:34:36,870 That's what DP of j returns. 670 00:34:36,870 --> 00:34:38,810 So it's a number. 671 00:34:38,810 --> 00:34:41,440 It's going to be a sum of badness values. 672 00:34:41,440 --> 00:34:43,072 Then we add on one new badness value. 673 00:34:43,072 --> 00:34:44,530 It's still a sum of badness values. 674 00:34:44,530 --> 00:34:47,260 We return the best one that we find. 675 00:34:47,260 --> 00:34:49,060 Now, this does not actually pack the words. 676 00:34:49,060 --> 00:34:51,384 That's a good-- maybe your implicit question. 677 00:34:51,384 --> 00:34:53,420 It's not telling you how to pack the words. 678 00:34:53,420 --> 00:34:55,840 It's telling you how much it costs to pack the words. 679 00:34:55,840 --> 00:34:58,960 This is a lot like shortest paths where we didn't-- it was 680 00:34:58,960 --> 00:35:02,386 annoying to actually figure out what the shortest path was. 681 00:35:02,386 --> 00:35:03,760 Not that annoying, but that's not 682 00:35:03,760 --> 00:35:05,176 what we were usually aiming to do. 683 00:35:05,176 --> 00:35:08,302 We were just trying to figure out the shortest path weight. 684 00:35:08,302 --> 00:35:10,260 And then once we knew the shortest path weight, 685 00:35:10,260 --> 00:35:13,320 it was pretty easy to reconstruct the paths. 686 00:35:13,320 --> 00:35:17,740 So maybe I'll take a little diversion to that 687 00:35:17,740 --> 00:35:19,160 and talk about parent pointers. 688 00:35:26,580 --> 00:35:28,410 The idea with parent pointers is just 689 00:35:28,410 --> 00:35:34,785 remember which guess was best. 690 00:35:42,700 --> 00:35:45,540 it's a very simple idea, but it applies to all dynamic programs 691 00:35:45,540 --> 00:35:47,700 and lets you find the actual solution, not just 692 00:35:47,700 --> 00:35:49,411 the cost of the solution. 693 00:35:49,411 --> 00:35:51,160 We did the same thing with shortest paths. 694 00:35:51,160 --> 00:35:53,870 We even called them parent. 695 00:35:53,870 --> 00:35:59,070 So when we compute this min, were trying all choices of j. 696 00:35:59,070 --> 00:36:01,770 One of them-- or maybe more than one, but at least one of them 697 00:36:01,770 --> 00:36:03,260 actually gave you the min. 698 00:36:03,260 --> 00:36:05,810 That's usually called the arg min in mathematics. 699 00:36:05,810 --> 00:36:08,470 It's what was the value of j that gave you 700 00:36:08,470 --> 00:36:11,130 the minimum value of this thing. 701 00:36:11,130 --> 00:36:12,940 So I mean, when you compute the min, 702 00:36:12,940 --> 00:36:14,940 you're iterating over every single one. 703 00:36:14,940 --> 00:36:17,740 Just keep track of which one was the best. 704 00:36:17,740 --> 00:36:18,240 That's it. 705 00:36:18,240 --> 00:36:19,490 Call that the parent pointer. 706 00:36:22,440 --> 00:36:23,930 Do I need to write that? 707 00:36:23,930 --> 00:36:30,100 Here, parent-- parent of i is going 708 00:36:30,100 --> 00:36:34,900 to be arg min of that same thing. 709 00:36:34,900 --> 00:36:38,270 So it's a j value. 710 00:36:38,270 --> 00:36:41,290 It's the best j value for i. 711 00:36:41,290 --> 00:36:43,190 And so we store that for each i. 712 00:36:43,190 --> 00:36:45,150 It cost no more work, just a constant factor 713 00:36:45,150 --> 00:36:47,210 more work than computing the min. 714 00:36:47,210 --> 00:36:49,310 We also write down the arg min. 715 00:36:49,310 --> 00:36:52,110 So we're already storing the min in the DP table. 716 00:36:52,110 --> 00:36:54,700 DP of i would get sorted to be that. 717 00:36:54,700 --> 00:36:56,400 We also store parent of i. 718 00:36:56,400 --> 00:37:00,260 And then once we're done, we start with our original problem 719 00:37:00,260 --> 00:37:02,140 and we follow parent pointers to figure out 720 00:37:02,140 --> 00:37:04,040 what the best choices were. 721 00:37:04,040 --> 00:37:08,400 So we start at 0 because we know word zero begins a line. 722 00:37:08,400 --> 00:37:12,120 And then 0 will be the first line. 723 00:37:12,120 --> 00:37:15,020 Then we go to parent of 0. 724 00:37:15,020 --> 00:37:18,190 That will be where the second line begins. 725 00:37:18,190 --> 00:37:20,960 Then we go to parent of parent of 0. 726 00:37:20,960 --> 00:37:25,010 That will be where the third line begins. 727 00:37:25,010 --> 00:37:26,850 OK, because these were the best choices 728 00:37:26,850 --> 00:37:28,280 for where the second line begins, 729 00:37:28,280 --> 00:37:31,374 this is the best place where the second line begins. 730 00:37:31,374 --> 00:37:32,790 Given that this is the first line, 731 00:37:32,790 --> 00:37:34,922 this is the best line where the second line begins 732 00:37:34,922 --> 00:37:36,380 given that this was the first line. 733 00:37:36,380 --> 00:37:39,460 So that's really the third line given this was the second line. 734 00:37:39,460 --> 00:37:41,980 Little confusing, but you just a simple for loop. 735 00:37:41,980 --> 00:37:44,960 You start with 0 because that's our original problem. 736 00:37:44,960 --> 00:37:47,810 You keep calling parent of the thing you currently have. 737 00:37:47,810 --> 00:37:49,830 In linear time, you will reconstruct 738 00:37:49,830 --> 00:37:51,630 where the lines break. 739 00:37:51,630 --> 00:37:54,540 So you can use this technique in any DP. 740 00:37:54,540 --> 00:37:56,240 It's very simple. 741 00:37:56,240 --> 00:37:57,970 It's totally automatic. 742 00:37:57,970 --> 00:38:01,100 Just like memoization is a technique 743 00:38:01,100 --> 00:38:02,880 that you can apply without thinking, 744 00:38:02,880 --> 00:38:06,246 you could even write a program, given a recursive algorithm, 745 00:38:06,246 --> 00:38:08,245 would turn into a memorized recursive algorithm. 746 00:38:08,245 --> 00:38:10,550 It's totally automated. 747 00:38:10,550 --> 00:38:12,410 Same thing with the bottom up DP table. 748 00:38:12,410 --> 00:38:17,567 As long as you know what the topological order is, 749 00:38:17,567 --> 00:38:19,150 just make those for loops and then put 750 00:38:19,150 --> 00:38:21,945 exactly the recursive call but turn it into an array call. 751 00:38:21,945 --> 00:38:23,570 Boom, you've got a bottom up algorithm. 752 00:38:23,570 --> 00:38:26,150 Totally automatic, no thinking required. 753 00:38:26,150 --> 00:38:28,550 Parent pointers also, no thinking required. 754 00:38:28,550 --> 00:38:31,260 As long as you're following the structure of trial guesses 755 00:38:31,260 --> 00:38:33,020 compute some value-- just remember 756 00:38:33,020 --> 00:38:36,440 what the guess was-- you reconstruct your solution. 757 00:38:36,440 --> 00:38:38,440 That's the great thing about dynamic programming 758 00:38:38,440 --> 00:38:39,850 is how much of it is automatic. 759 00:38:39,850 --> 00:38:42,480 The hard part is figuring out what to guess 760 00:38:42,480 --> 00:38:45,700 and then what your sub-problems are, or the other order. 761 00:38:45,700 --> 00:38:48,329 Whatever works. 762 00:38:48,329 --> 00:38:49,620 Any other questions about text? 763 00:38:52,290 --> 00:38:55,680 I would like to move on to blackjack. 764 00:38:55,680 --> 00:38:58,340 OK, now I brought some cards, because some of you 765 00:38:58,340 --> 00:39:00,450 may not know the rules to blackjack. 766 00:39:00,450 --> 00:39:01,990 How many people know blackjack? 767 00:39:01,990 --> 00:39:02,655 OK. 768 00:39:02,655 --> 00:39:04,780 How many people do not and are willing to admit it? 769 00:39:04,780 --> 00:39:05,690 A few, all right. 770 00:39:05,690 --> 00:39:08,240 So this is for you and for fun, entertainment. 771 00:39:08,240 --> 00:39:11,600 So I'm going to bring Victor up to help demonstrate 772 00:39:11,600 --> 00:39:13,120 the rules of blackjack. 773 00:39:13,120 --> 00:39:15,650 We're going to play standard Casino blackjack 774 00:39:15,650 --> 00:39:18,430 as in the movie 21, or whatever. 775 00:39:18,430 --> 00:39:23,442 So I'm going to just do a random cut here so I can't sheet. 776 00:39:23,442 --> 00:39:26,140 You have a tablet, that's scary. 777 00:39:26,140 --> 00:39:27,725 You're going to look at strategy. 778 00:39:27,725 --> 00:39:29,320 VICTOR: Nothing special. 779 00:39:29,320 --> 00:39:30,250 PROFESSOR: All right. 780 00:39:30,250 --> 00:39:32,760 Hopefully you do not have x-ray vision. 781 00:39:32,760 --> 00:39:34,680 So the way it works is there's a dealer 782 00:39:34,680 --> 00:39:36,901 player and one or more players. 783 00:39:36,901 --> 00:39:39,400 We're just going to do it with one player to keep it simple. 784 00:39:39,400 --> 00:39:40,525 I'm going to be the dealer. 785 00:39:40,525 --> 00:39:42,710 So my strategy is actually totally deterministic, 786 00:39:42,710 --> 00:39:44,090 there's nothing interesting. 787 00:39:44,090 --> 00:39:46,440 Victor has the hard part of winning. 788 00:39:46,440 --> 00:39:51,130 So to start out, I believe we deal to you first, then 789 00:39:51,130 --> 00:39:54,910 to me, then to you, then to me. 790 00:39:54,910 --> 00:39:56,600 So let's hold up these cards, Victor, 791 00:39:56,600 --> 00:39:57,959 so that people can see them. 792 00:39:57,959 --> 00:39:59,500 You don't get to see one of my cards. 793 00:39:59,500 --> 00:40:02,310 That's some peculiarity of the rule. 794 00:40:02,310 --> 00:40:07,190 And if the sum of our cards goes over 21, we lose the game. 795 00:40:07,190 --> 00:40:08,420 Victor first. 796 00:40:08,420 --> 00:40:12,570 I cannot have a value more than 21 in these hands, 797 00:40:12,570 --> 00:40:14,420 because I only have two cards. 798 00:40:14,420 --> 00:40:16,550 You have a value of-- ha, ace. 799 00:40:16,550 --> 00:40:17,050 Great. 800 00:40:17,050 --> 00:40:18,480 An ace can be a 1 or an 11. 801 00:40:18,480 --> 00:40:19,540 That's the fun rule. 802 00:40:19,540 --> 00:40:22,384 So this is either an 8 or an 18. 803 00:40:22,384 --> 00:40:24,800 And so Victor has a choice of whether to take another card 804 00:40:24,800 --> 00:40:25,480 or not. 805 00:40:25,480 --> 00:40:27,284 What would you like to do? 806 00:40:27,284 --> 00:40:29,180 VICTOR: Standard strategy says stand. 807 00:40:29,180 --> 00:40:30,054 PROFESSOR: He stands. 808 00:40:30,054 --> 00:40:31,845 So he's going to stick to that. 809 00:40:31,845 --> 00:40:34,800 At this point, my cards flip over. 810 00:40:34,800 --> 00:40:37,790 I have 17, which is same you, which 811 00:40:37,790 --> 00:40:39,613 I believe means-- I forget about tie rules. 812 00:40:39,613 --> 00:40:40,460 VICTOR: I have 18. 813 00:40:40,460 --> 00:40:41,150 PROFESSOR: You have 18. 814 00:40:41,150 --> 00:40:41,220 All right. 815 00:40:41,220 --> 00:40:41,402 VICTOR: See? 816 00:40:41,402 --> 00:40:42,210 The strategy works. 817 00:40:42,210 --> 00:40:43,464 PROFESSOR: So that's good. 818 00:40:43,464 --> 00:40:45,880 I'm going to hit in the hope that I have a small card that 819 00:40:45,880 --> 00:40:47,970 will push me right above you. 820 00:40:47,970 --> 00:40:48,790 But I do not. 821 00:40:48,790 --> 00:40:49,570 I lose. 822 00:40:49,570 --> 00:40:50,129 I'm sad. 823 00:40:50,129 --> 00:40:51,670 VICTOR: It says always stand on a 17. 824 00:40:51,670 --> 00:40:52,870 PROFESSOR: Oh, always stand on 17? 825 00:40:52,870 --> 00:40:53,370 Huh. 826 00:40:53,370 --> 00:40:54,860 All right, never mind. 827 00:40:54,860 --> 00:40:55,660 Thanks. 828 00:40:55,660 --> 00:40:56,720 Yeah, I still lose. 829 00:40:56,720 --> 00:40:58,110 The game is over. 830 00:40:58,110 --> 00:41:01,800 My strategy is always stand on a value-- 831 00:41:01,800 --> 00:41:03,360 VICTOR: Stand on 17. 832 00:41:03,360 --> 00:41:05,270 PROFESSOR: 17 or higher. 833 00:41:05,270 --> 00:41:09,030 And if I have a value less than 17, I always take another card. 834 00:41:09,030 --> 00:41:11,090 So let's do it one more time to get it right. 835 00:41:11,090 --> 00:41:15,880 So I'm going to deal to you, deal to me, deal to you, 836 00:41:15,880 --> 00:41:16,410 deal to me. 837 00:41:16,410 --> 00:41:19,450 So hold up your cards. 838 00:41:19,450 --> 00:41:21,780 You have 18 again. 839 00:41:21,780 --> 00:41:22,940 Are you cheating? 840 00:41:25,600 --> 00:41:27,990 VICTOR: I still have to stand. 841 00:41:27,990 --> 00:41:31,120 PROFESSOR: You still stand, according to tablet. 842 00:41:31,120 --> 00:41:33,560 So I, in this case, have a 20. 843 00:41:33,560 --> 00:41:35,465 And so this I win. 844 00:41:35,465 --> 00:41:37,310 So you get the idea. 845 00:41:37,310 --> 00:41:39,069 Let's say in each case we're betting $1. 846 00:41:39,069 --> 00:41:40,360 So at this point, we'd be even. 847 00:41:40,360 --> 00:41:43,300 He won $1, I won $1. 848 00:41:43,300 --> 00:41:48,300 But in general, slight-- I think it's balanced. 849 00:41:48,300 --> 00:41:51,127 VICTOR: For these rules, there's a 1% advantage for the house. 850 00:41:51,127 --> 00:41:52,710 PROFESSOR: 1% advantage for the house. 851 00:41:52,710 --> 00:41:53,210 Interesting. 852 00:41:53,210 --> 00:41:56,120 All right, well, that's beyond this class. 853 00:41:56,120 --> 00:42:00,170 What we're going to see is how to cheat in blackjack. 854 00:42:00,170 --> 00:42:05,040 So this is going to be-- I encourage you to try this out 855 00:42:05,040 --> 00:42:05,750 at casinos. 856 00:42:05,750 --> 00:42:06,310 Just kidding. 857 00:42:10,790 --> 00:42:14,860 This is a little bit difficult to actually do in a casino 858 00:42:14,860 --> 00:42:17,510 unless you have an inside man. 859 00:42:17,510 --> 00:42:21,090 So if you have an inside man, go for it. 860 00:42:21,090 --> 00:42:23,500 It's guaranteed to win you lots of money 861 00:42:23,500 --> 00:42:25,240 because it's going to play optimally. 862 00:42:25,240 --> 00:42:26,830 In perfect information blackjack, 863 00:42:26,830 --> 00:42:31,260 I suppose that I already know the entire deck. 864 00:42:31,260 --> 00:42:34,270 Suppose somehow either I get to put the deck there, 865 00:42:34,270 --> 00:42:35,990 or I have some x-ray vision. 866 00:42:35,990 --> 00:42:38,205 I get to see the entire deck ahead of time. 867 00:42:38,205 --> 00:42:39,580 And then somebody's going to play 868 00:42:39,580 --> 00:42:42,970 through a game over and over with me-- or not over and over, 869 00:42:42,970 --> 00:42:45,444 but until the deck is depleted-- and I 870 00:42:45,444 --> 00:42:47,110 want to know in each case, should I hit, 871 00:42:47,110 --> 00:42:49,660 or should I stand? 872 00:42:49,660 --> 00:42:51,160 And I claim with dynamic programming 873 00:42:51,160 --> 00:42:54,545 you can figure that out-- using exactly the same strategy 874 00:42:54,545 --> 00:42:55,820 as text, actually. 875 00:42:55,820 --> 00:42:57,390 It's really for each word, should I 876 00:42:57,390 --> 00:42:59,980 start a new line or not? 877 00:42:59,980 --> 00:43:02,150 Same problem here. 878 00:43:02,150 --> 00:43:04,310 It's slightly more complicated to write down. 879 00:43:08,370 --> 00:43:11,620 So let's say the deck is a sequence of cards. 880 00:43:11,620 --> 00:43:19,030 And I'm going to call it c0, c1 up to cn minus 1, n cards. 881 00:43:19,030 --> 00:43:23,010 And you are one player. 882 00:43:23,010 --> 00:43:25,710 First is the dealer. 883 00:43:25,710 --> 00:43:27,570 I don't know how to solve this for two 884 00:43:27,570 --> 00:43:29,740 players, interesting open problem. 885 00:43:29,740 --> 00:43:34,120 But for one player I can do it. 886 00:43:34,120 --> 00:43:39,620 Let's say $1 bet per hand, I think they're called. 887 00:43:39,620 --> 00:43:40,480 I'm not sure. 888 00:43:40,480 --> 00:43:41,950 Per play? 889 00:43:41,950 --> 00:43:42,810 Per box? 890 00:43:42,810 --> 00:43:43,777 Whatever. 891 00:43:43,777 --> 00:43:44,985 You're not allowed to double. 892 00:43:44,985 --> 00:43:46,151 You're not allowed to split. 893 00:43:46,151 --> 00:43:48,530 All these fancy rules are harder to think about, 894 00:43:48,530 --> 00:43:52,480 although you might be able to solve them as well. 895 00:43:52,480 --> 00:43:55,279 So the idea is I have some cards. 896 00:43:55,279 --> 00:43:56,570 Should I hit or should I stand? 897 00:43:56,570 --> 00:43:57,310 I don't know. 898 00:43:57,310 --> 00:43:58,550 I'll guess. 899 00:43:58,550 --> 00:44:05,340 So our guessing-- let's jump ahead to the guessing part-- 900 00:44:05,340 --> 00:44:10,000 is whether we want to hit or stand given a card. 901 00:44:10,000 --> 00:44:12,470 Actually, it would be easier to think 902 00:44:12,470 --> 00:44:15,120 about an entire play, an entire hand. 903 00:44:15,120 --> 00:44:17,080 We're going to guess, how many times should I 904 00:44:17,080 --> 00:44:28,410 hit in the first play? 905 00:44:28,410 --> 00:44:30,840 So initially, four cards are dealt. 906 00:44:30,840 --> 00:44:32,502 I look at my hands. 907 00:44:32,502 --> 00:44:34,210 Actually, I don't really look at my hand. 908 00:44:34,210 --> 00:44:36,230 I'm just going to guess ahead of time. 909 00:44:36,230 --> 00:44:38,920 I think I'll hit five times this time. 910 00:44:38,920 --> 00:44:40,600 I think I'll hit zero times this time. 911 00:44:40,600 --> 00:44:42,490 I mean, I'm just going to try them all. 912 00:44:42,490 --> 00:44:46,270 So I don't really have to be intelligent here, OK? 913 00:44:46,270 --> 00:44:50,690 It's kind of crazy but it works. 914 00:44:50,690 --> 00:44:53,870 Our sub-problems, can anyone tell me 915 00:44:53,870 --> 00:44:58,829 what our sub-problems would be, In one word or less? 916 00:44:58,829 --> 00:44:59,870 Less would be impressive. 917 00:45:02,779 --> 00:45:03,279 Yeah? 918 00:45:03,279 --> 00:45:04,740 AUDIENCE: Where you start the new hand. 919 00:45:04,740 --> 00:45:05,850 PROFESSOR: Where do you start the new hand? 920 00:45:05,850 --> 00:45:06,349 Yeah. 921 00:45:06,349 --> 00:45:08,980 So it's going to be suffixes of the cards. 922 00:45:08,980 --> 00:45:12,170 So at some point we do a play, and then we get to ith card. 923 00:45:12,170 --> 00:45:15,120 And then the rest of the game will be from the ith card on. 924 00:45:15,120 --> 00:45:20,450 So it's going to be suffix ci colon, 925 00:45:20,450 --> 00:45:22,210 I guess would be the notation here. 926 00:45:22,210 --> 00:45:24,190 It's a bit awkward. 927 00:45:24,190 --> 00:45:25,640 These are the cards that remain. 928 00:45:25,640 --> 00:45:28,190 And so the sub-problem is, what is the best play? 929 00:45:28,190 --> 00:45:31,400 What's the best outcome given $1 bets? 930 00:45:31,400 --> 00:45:34,990 How much money can I make-- maximize my winning, say-- 931 00:45:34,990 --> 00:45:36,800 given these cards onward? 932 00:45:36,800 --> 00:45:39,547 Who knows what happened to their earlier cards, but just 933 00:45:39,547 --> 00:45:40,380 these are the cards. 934 00:45:40,380 --> 00:45:42,100 I'm left with. 935 00:45:42,100 --> 00:45:53,330 Number of sub-problems is-- hmm? 936 00:46:00,450 --> 00:46:01,840 n. 937 00:46:01,840 --> 00:46:05,220 How many choices of i are there? n choices. 938 00:46:05,220 --> 00:46:06,760 This really important. 939 00:46:06,760 --> 00:46:10,000 It's really useful that we're thinking about suffixes. 940 00:46:10,000 --> 00:46:12,477 It's not that some subset of the cards have been played. 941 00:46:12,477 --> 00:46:14,810 That would be really hard, because there's exponentially 942 00:46:14,810 --> 00:46:16,890 many different subsets that could be left. 943 00:46:16,890 --> 00:46:18,820 It's always a prefix that gets played, 944 00:46:18,820 --> 00:46:20,080 and therefore suffix is left. 945 00:46:20,080 --> 00:46:22,605 And there's only n suffixes, remember that. 946 00:46:22,605 --> 00:46:25,890 We're going to use it over and over in dynamic programming. 947 00:46:25,890 --> 00:46:28,310 So now we need to solve the sub-problem. 948 00:46:28,310 --> 00:46:31,000 Starting from ci, what's the best way to play? 949 00:46:31,000 --> 00:46:33,480 Well, the first four cards are fixed, 950 00:46:33,480 --> 00:46:36,020 and then we guess how many hits are left. 951 00:46:36,020 --> 00:46:38,240 So it's going to be something like n minus i 952 00:46:38,240 --> 00:46:41,274 minus four different possibilities for-- I mean, 953 00:46:41,274 --> 00:46:42,940 that would be the maximum number of hits 954 00:46:42,940 --> 00:46:45,060 I could take all the remaining cards. 955 00:46:45,060 --> 00:46:49,270 That would be the most. 956 00:46:49,270 --> 00:46:53,670 And let's see, so the number of choices-- I'll just say it's, 957 00:46:53,670 --> 00:46:54,620 at most, n. 958 00:46:54,620 --> 00:46:58,190 I don't have to be fancy here. 959 00:46:58,190 --> 00:47:02,540 OK, now we go to the recurrence. 960 00:47:08,900 --> 00:47:12,600 So I'm going to call this blackjack of i. 961 00:47:12,600 --> 00:47:14,800 It's going to be the solution. 962 00:47:14,800 --> 00:47:18,930 I want to solve this sub-problem from i onwards. 963 00:47:18,930 --> 00:47:20,850 What's the best play? 964 00:47:20,850 --> 00:47:26,850 And I guess it's going to be a max if I'm measuring winnings. 965 00:47:26,850 --> 00:47:30,470 And what's the winnings if I decide to hit this many times? 966 00:47:30,470 --> 00:47:33,530 It's a little bit hard to write down the exact formula. 967 00:47:33,530 --> 00:47:37,270 I'm going to write a rough version which 968 00:47:37,270 --> 00:47:40,450 is the outcome of that first play. 969 00:47:40,450 --> 00:47:47,400 It's going to be either I lose $1, we tie, or I win $1. 970 00:47:47,400 --> 00:47:49,300 So if we end up with the same value, 971 00:47:49,300 --> 00:47:51,450 you actually-- in most versions-- 972 00:47:51,450 --> 00:47:53,250 you get your money back, nothing changes. 973 00:47:53,250 --> 00:47:55,320 The bet is nullified. 974 00:47:55,320 --> 00:47:57,022 So that's a zero outcome. 975 00:47:57,022 --> 00:47:58,480 But if we're only betting $1, these 976 00:47:58,480 --> 00:47:59,850 are the three possible outcomes. 977 00:47:59,850 --> 00:48:01,250 You can compute this, right? 978 00:48:01,250 --> 00:48:03,470 If I told you how many times you hit, 979 00:48:03,470 --> 00:48:06,660 then you just execute through those cards 980 00:48:06,660 --> 00:48:08,650 and you compute the values of my hand, 981 00:48:08,650 --> 00:48:11,020 of your hand versus the dealer's hand. 982 00:48:11,020 --> 00:48:13,130 You see, did anyone bust? 983 00:48:13,130 --> 00:48:13,990 If so, they lose. 984 00:48:13,990 --> 00:48:16,530 Otherwise you compare the values and you 985 00:48:16,530 --> 00:48:19,360 see which is bigger or smaller. 986 00:48:19,360 --> 00:48:22,210 This is easy to do in linear time. 987 00:48:22,210 --> 00:48:22,949 No biggie. 988 00:48:22,949 --> 00:48:24,865 What's useful here is that the dealer strategy 989 00:48:24,865 --> 00:48:25,920 is deterministic. 990 00:48:25,920 --> 00:48:29,217 So after you know how many cards you take, what the dealer does 991 00:48:29,217 --> 00:48:30,550 is force, because he just looks. 992 00:48:30,550 --> 00:48:32,290 Do I have 17 or greater? 993 00:48:32,290 --> 00:48:35,840 If not, take another card and keep repeating that. 994 00:48:35,840 --> 00:48:37,220 So it's a deterministic strategy. 995 00:48:37,220 --> 00:48:39,940 In linear time, you can figure out what the outcome is. 996 00:48:39,940 --> 00:48:42,910 Then you also have to add the outcome of all the remaining 997 00:48:42,910 --> 00:48:44,703 cards, which is just BG of j. 998 00:48:44,703 --> 00:48:47,950 This is recursion, super easy. 999 00:48:47,950 --> 00:48:50,230 We do this for all choices of j. 1000 00:48:53,740 --> 00:48:59,340 It's like a range of i plus 4 up to n, I think. 1001 00:49:06,170 --> 00:49:07,640 Sure, that'll work. 1002 00:49:07,640 --> 00:49:09,920 I should probably put an if here, 1003 00:49:09,920 --> 00:49:11,420 which is if it's a valid play. 1004 00:49:14,067 --> 00:49:15,400 There are some constraints here. 1005 00:49:15,400 --> 00:49:18,532 If I've already busted, I can't hit again. 1006 00:49:18,532 --> 00:49:20,490 So in fact what you have to do in this for loop 1007 00:49:20,490 --> 00:49:22,200 is say, well, maybe I take another hit. 1008 00:49:22,200 --> 00:49:23,241 Maybe I take another hit. 1009 00:49:23,241 --> 00:49:25,330 At some point I go over 21, and then you 1010 00:49:25,330 --> 00:49:26,690 have to stop the for loop. 1011 00:49:26,690 --> 00:49:27,980 So I'm writing that as an if. 1012 00:49:27,980 --> 00:49:30,680 You can also do it with a break, however you want. 1013 00:49:30,680 --> 00:49:34,910 But that's-- you're considering all possible options, 1014 00:49:34,910 --> 00:49:36,311 all valid options of play. 1015 00:49:36,311 --> 00:49:38,060 For each of them, you see what the outcome 1016 00:49:38,060 --> 00:49:40,950 was after the dealer takes some more cards. 1017 00:49:40,950 --> 00:49:44,510 This is actually a little bit funny. 1018 00:49:44,510 --> 00:49:50,100 Sorry, this should really be the number of hits in range 1019 00:49:50,100 --> 00:49:51,880 from, let's say, 0 to n. 1020 00:49:51,880 --> 00:49:53,560 Maybe you don't hit at all. 1021 00:49:53,560 --> 00:49:55,500 And then j is a little bit tricky, 1022 00:49:55,500 --> 00:50:01,150 because this is actually i plus 4 plus the number of hits 1023 00:50:01,150 --> 00:50:02,740 plus the number of dealer hits. 1024 00:50:05,750 --> 00:50:07,700 OK, so you have to run this algorithm 1025 00:50:07,700 --> 00:50:09,880 to compute what happened, which computes 1026 00:50:09,880 --> 00:50:11,970 how many times a dealer took a card. 1027 00:50:11,970 --> 00:50:14,660 That's how many cards got consumed. 1028 00:50:14,660 --> 00:50:17,880 And so that's-- if you do i plus 4 plus that plus that, 1029 00:50:17,880 --> 00:50:21,450 that's how many cards are left, or where the cards resume. 1030 00:50:21,450 --> 00:50:23,200 And then you call BG on that. 1031 00:50:23,200 --> 00:50:25,700 So we're, in general, from BG of i-- 1032 00:50:25,700 --> 00:50:29,920 if you think of the DAG-- there's some position, 1033 00:50:29,920 --> 00:50:32,350 maybe i plus 4 happens. 1034 00:50:32,350 --> 00:50:33,350 Maybe it doesn't happen. 1035 00:50:33,350 --> 00:50:36,480 It depends on what the dealer does. 1036 00:50:36,480 --> 00:50:40,650 We're going to depend on i plus 6, i plus 5 maybe. 1037 00:50:40,650 --> 00:50:43,040 It's going to be all of these possibilities. 1038 00:50:43,040 --> 00:50:46,072 These are all different plays. 1039 00:50:46,072 --> 00:50:47,655 And then on each of these edges, we're 1040 00:50:47,655 --> 00:50:50,090 going to have plus 1, 0, or minus 1. 1041 00:50:50,090 --> 00:50:55,220 Those are the outcomes, whether I won or lost or tied. 1042 00:50:55,220 --> 00:50:59,040 And then we're just computing a shortest path in this DAG. 1043 00:50:59,040 --> 00:51:02,780 It's actually really easy if you think about it that way. 1044 00:51:02,780 --> 00:51:04,404 This is just how many cards are left. 1045 00:51:04,404 --> 00:51:07,070 From that position, you just see what are all the possibilities? 1046 00:51:07,070 --> 00:51:09,710 What are all the edges that I could go to? 1047 00:51:09,710 --> 00:51:11,340 What states could I to go to next? 1048 00:51:11,340 --> 00:51:12,730 How many cards are remaining? 1049 00:51:12,730 --> 00:51:15,330 How much did it cost me or win me? 1050 00:51:15,330 --> 00:51:18,382 And then take longest paths in that DAG. 1051 00:51:18,382 --> 00:51:20,090 That will give you the exact same answer. 1052 00:51:20,090 --> 00:51:22,580 That's what this dynamic programming is doing. 1053 00:51:22,580 --> 00:51:25,390 In the lecture notes, there's more details where I actually 1054 00:51:25,390 --> 00:51:28,000 tried to write out this function, this recurrence 1055 00:51:28,000 --> 00:51:29,390 as an algorithm. 1056 00:51:29,390 --> 00:51:31,810 You could do it, assuming I've got everything right. 1057 00:51:31,810 --> 00:51:33,700 It's not that hard. 1058 00:51:33,700 --> 00:51:36,410 The order here is just the same as the order we did before. 1059 00:51:36,410 --> 00:51:38,960 The running time is going to be cubic in the worst case, 1060 00:51:38,960 --> 00:51:41,820 because we have-- it's a little non-obvious, 1061 00:51:41,820 --> 00:51:43,530 but we have n sub-problems. 1062 00:51:43,530 --> 00:51:45,220 For each of them, we have n choices. 1063 00:51:45,220 --> 00:51:49,740 And for each choice we have to run the dealer strategy. 1064 00:51:49,740 --> 00:51:51,870 And so that conceivably could take linear time. 1065 00:51:51,870 --> 00:51:53,590 Here I'm assuming a general value of 21. 1066 00:51:53,590 --> 00:51:55,760 If 21 is actually constant, it only 1067 00:51:55,760 --> 00:51:57,940 be constant time to play out a single hand, 1068 00:51:57,940 --> 00:51:59,450 and then it's quadratic time. 1069 00:51:59,450 --> 00:52:03,380 So it depends on your model of generalized blackjack. 1070 00:52:03,380 --> 00:52:04,380 But that's it. 1071 00:52:04,380 --> 00:52:06,940 And get some flavor of the power of dynamic programming, 1072 00:52:06,940 --> 00:52:09,110 we're going to see it's even more powerful than this 1073 00:52:09,110 --> 00:52:11,430 in the next two lectures.