1 00:00:00,000 --> 00:00:03,888 [SQUEAKING][RUSTLING][CLICKING] 2 00:00:12,650 --> 00:00:15,860 ERIK DEMAINE: Today we're going to, in one lecture, 3 00:00:15,860 --> 00:00:19,390 cover an entire field, which is computational complexity. 4 00:00:19,390 --> 00:00:23,410 It's sort of-- it meets algorithms 5 00:00:23,410 --> 00:00:25,780 in an interesting way, which is, algorithms is mostly 6 00:00:25,780 --> 00:00:28,090 about showing how to solve problems well 7 00:00:28,090 --> 00:00:30,730 and showing that you can solve a problem well. 8 00:00:30,730 --> 00:00:33,250 And computational complexity is more about the lower bound 9 00:00:33,250 --> 00:00:35,260 side, proving that you can't prove-- 10 00:00:35,260 --> 00:00:37,210 you can't solve a problem very well, 11 00:00:37,210 --> 00:00:39,912 you can't find a good algorithm to solve it. 12 00:00:39,912 --> 00:00:42,370 We've seen a little bit about lower bounds several lectures 13 00:00:42,370 --> 00:00:45,810 ago, proving search and sorting lower bounds 14 00:00:45,810 --> 00:00:51,070 in a bounded branching decision tree model. 15 00:00:51,070 --> 00:00:54,460 But these are much stronger notions of badness. 16 00:00:54,460 --> 00:00:58,370 This is not about n versus n log n or constant versus log n. 17 00:00:58,370 --> 00:01:02,380 This is about polynomial versus exponential, 18 00:01:02,380 --> 00:01:05,170 which has been the sort of bread-and-butter model 19 00:01:05,170 --> 00:01:06,010 in this class. 20 00:01:06,010 --> 00:01:07,600 Polynomial is a good running time, 21 00:01:07,600 --> 00:01:09,058 and we're always striving for that. 22 00:01:09,058 --> 00:01:12,220 Exponential is usually pretty trivial to get. 23 00:01:12,220 --> 00:01:14,290 And so we're going to talk about some different-- 24 00:01:14,290 --> 00:01:16,210 they're called complexity classes 25 00:01:16,210 --> 00:01:18,460 that talk about this issue and different ways 26 00:01:18,460 --> 00:01:20,500 to prove hardness. 27 00:01:20,500 --> 00:01:22,000 This is a pretty high-level lecture, 28 00:01:22,000 --> 00:01:23,417 so you're not going to be expected 29 00:01:23,417 --> 00:01:24,850 to be able to prove hardness. 30 00:01:24,850 --> 00:01:26,600 But you'll get a flavor of what it's like, 31 00:01:26,600 --> 00:01:29,330 and this will segue nicely into other follow-on classes, 32 00:01:29,330 --> 00:01:32,290 which is-- we're at pretty much the end of 006, 33 00:01:32,290 --> 00:01:37,450 so natural to talk about what other things you might study. 34 00:01:37,450 --> 00:01:41,650 One result we'll prove today is that most problems actually 35 00:01:41,650 --> 00:01:44,770 have no algorithm, which is kind of shocking, 36 00:01:44,770 --> 00:01:46,720 and lots of other fun things. 37 00:01:46,720 --> 00:01:50,710 So let's get started with the notion of P. 38 00:01:50,710 --> 00:01:55,660 This is the set of all problems solvable in polynomial time. 39 00:02:01,110 --> 00:02:02,720 We talked about what polynomial time 40 00:02:02,720 --> 00:02:06,000 means a bunch last lecture. 41 00:02:06,000 --> 00:02:08,060 So just recall that polynomial time 42 00:02:08,060 --> 00:02:14,540 means polynomial in the problem size, which I'll denote as n 43 00:02:14,540 --> 00:02:21,924 here, the number of words in your input. 44 00:02:28,560 --> 00:02:32,970 OK, so these are the problems that are efficiently solvable. 45 00:02:32,970 --> 00:02:35,910 P is the set of all of them. 46 00:02:35,910 --> 00:02:40,200 And for contrast, EXP is the set of all problems solvable 47 00:02:40,200 --> 00:02:42,926 in exponential time. 48 00:02:42,926 --> 00:02:53,880 It's the problems solvable in exponential time. 49 00:02:53,880 --> 00:02:57,210 Exponential here means something like 2 50 00:02:57,210 --> 00:03:00,135 to the n to the constant. 51 00:03:00,135 --> 00:03:02,690 That's one reasonable definition of exponential, 52 00:03:02,690 --> 00:03:06,990 so just the exponentiation of this-- of polynomial. 53 00:03:06,990 --> 00:03:10,875 So as you might expect, most-- 54 00:03:10,875 --> 00:03:13,000 every problem that we've talked about in this class 55 00:03:13,000 --> 00:03:16,350 so far can be solved in exponential time rather easily. 56 00:03:16,350 --> 00:03:18,240 And algorithms, in some sense, is about 57 00:03:18,240 --> 00:03:20,790 distinguishing these two, which problems are in P 58 00:03:20,790 --> 00:03:24,890 versus are in say EXP minus P. 59 00:03:24,890 --> 00:03:27,570 So to formalize this a little bit, 60 00:03:27,570 --> 00:03:32,970 I'm going to draw a picture, which 61 00:03:32,970 --> 00:03:35,250 is a bit of a simplification of reality, 62 00:03:35,250 --> 00:03:38,112 but for the purposes of this class will suffice, 63 00:03:38,112 --> 00:03:39,570 and I think is a really helpful way 64 00:03:39,570 --> 00:03:43,740 to think about things, which is to have a big axis for-- 65 00:03:43,740 --> 00:03:46,680 a single axis for, how hard is your problem, 66 00:03:46,680 --> 00:03:50,190 what is the difficulty of solving your problem? 67 00:03:50,190 --> 00:03:52,990 And I want to be sure to leave-- 68 00:03:52,990 --> 00:03:55,080 so the easiest problems are over here. 69 00:03:55,080 --> 00:03:57,660 And each problem is a dot on this axis. 70 00:03:57,660 --> 00:04:01,263 Hardest problems are way down the line. 71 00:04:01,263 --> 00:04:03,930 And I want to make sure to leave enough space for all the things 72 00:04:03,930 --> 00:04:05,080 that I care about. 73 00:04:05,080 --> 00:04:10,470 So P, I'm just going to call this segment up front. 74 00:04:10,470 --> 00:04:13,940 And then I'm going to have a bigger 75 00:04:13,940 --> 00:04:17,149 thing for exponential time. 76 00:04:20,399 --> 00:04:24,385 So this is just to say that P is nested inside EXP. 77 00:04:24,385 --> 00:04:26,510 Every problem that can be solved in polynomial time 78 00:04:26,510 --> 00:04:28,190 can also be solved in exponential time 79 00:04:28,190 --> 00:04:30,740 because polynomial is less than or equal to exponential. 80 00:04:30,740 --> 00:04:33,720 These are just upper bounds. 81 00:04:33,720 --> 00:04:36,270 Being an EXP means you're somewhere from this line 82 00:04:36,270 --> 00:04:36,780 to the left. 83 00:04:36,780 --> 00:04:39,210 Being in P means you're somewhere from this line 84 00:04:39,210 --> 00:04:41,550 to the left, in terms of difficulty. 85 00:04:41,550 --> 00:04:48,120 But formally, we would write P is contained in EXP as sets. 86 00:04:48,120 --> 00:04:52,827 In fact, they're also known to be different from each other. 87 00:04:52,827 --> 00:04:55,410 There are problems that can be solved in exponential time that 88 00:04:55,410 --> 00:04:57,210 cannot be solved in polynomial time. 89 00:04:57,210 --> 00:05:02,010 For example-- I'll put that here, sure. 90 00:05:09,480 --> 00:05:18,510 For example, n by n chess is in exponential time, 91 00:05:18,510 --> 00:05:19,960 but not polynomial time. 92 00:05:19,960 --> 00:05:22,170 So what is the n by chess problem? 93 00:05:22,170 --> 00:05:24,960 This is, I give you an n by n chessboard, 94 00:05:24,960 --> 00:05:28,327 and I describe to you a position. 95 00:05:28,327 --> 00:05:29,910 Here's where all the white pieces are. 96 00:05:29,910 --> 00:05:31,530 Here's where all the black pieces are. 97 00:05:31,530 --> 00:05:34,890 You can have an arbitrary number of queens and bishops and pawns 98 00:05:34,890 --> 00:05:37,020 of each color, of course, up to n 99 00:05:37,020 --> 00:05:39,540 squared of them so they don't overlap each other. 100 00:05:39,540 --> 00:05:42,382 And I want to know, does white win from this position? 101 00:05:42,382 --> 00:05:43,590 Let's say it's white to move. 102 00:05:43,590 --> 00:05:45,000 Can white win? 103 00:05:45,000 --> 00:05:47,700 And that problem can be solved in an exponential time 104 00:05:47,700 --> 00:05:51,703 by exploring the entire tree of all possible games. 105 00:05:51,703 --> 00:05:54,120 But it cannot-- but you can prove that it cannot be solved 106 00:05:54,120 --> 00:05:56,110 in polynomial time. 107 00:05:56,110 --> 00:05:57,840 So that's a nice example. 108 00:05:57,840 --> 00:06:01,630 A more positive example, so to speak, 109 00:06:01,630 --> 00:06:04,530 is negative weight cycle detection. 110 00:06:04,530 --> 00:06:08,269 I guess it's literally negative, but it's morally positive. 111 00:06:13,465 --> 00:06:15,840 Negative weight cycle detection is the following problem. 112 00:06:15,840 --> 00:06:18,497 I give you a graph, a directed graph with weights, 113 00:06:18,497 --> 00:06:20,580 and I want to know, does it have a negative weight 114 00:06:20,580 --> 00:06:22,380 cycle, yes or no? 115 00:06:22,380 --> 00:06:24,520 And this problem is in? 116 00:06:24,520 --> 00:06:25,200 AUDIENCE: P. 117 00:06:25,200 --> 00:06:28,870 ERIK DEMAINE: P, because we saw a polynomial time algorithm 118 00:06:28,870 --> 00:06:29,370 for this. 119 00:06:29,370 --> 00:06:31,987 You run Bellman-Ford on an augmented graph. 120 00:06:31,987 --> 00:06:34,320 So this is an example of a problem we know how to solve. 121 00:06:34,320 --> 00:06:36,120 This whole class is full of examples 122 00:06:36,120 --> 00:06:38,250 that we know how to solve in polynomial time. 123 00:06:38,250 --> 00:06:44,550 But this is a nice, non-trivial and succinct one to phrase. 124 00:06:44,550 --> 00:06:46,590 It's also an example of a decision problem. 125 00:06:46,590 --> 00:06:49,890 A lot of-- basically all the problems I'll talk about today 126 00:06:49,890 --> 00:06:52,560 are decision problems, like we talked about last class, 127 00:06:52,560 --> 00:06:54,810 meaning, the answer is just yes or no. 128 00:06:54,810 --> 00:06:57,720 Can white win from this position, yes or no? 129 00:06:57,720 --> 00:07:00,000 Is there a negative weight cycle, yes or no? 130 00:07:00,000 --> 00:07:04,950 Tetris we can also formulate as a problem. 131 00:07:04,950 --> 00:07:07,170 This is a version of Tetris that we might call 132 00:07:07,170 --> 00:07:09,670 perfect information Tetris. 133 00:07:09,670 --> 00:07:12,120 Suppose I give you a Tetris board. 134 00:07:12,120 --> 00:07:16,350 It has some garbage left over from your past playing, 135 00:07:16,350 --> 00:07:17,580 or maybe it started that way. 136 00:07:17,580 --> 00:07:20,610 And I give you the sequence of n pieces that are going to come. 137 00:07:20,610 --> 00:07:22,500 And I want to know, can I survive 138 00:07:22,500 --> 00:07:23,940 this sequence of n pieces? 139 00:07:23,940 --> 00:07:25,710 Can you place each of these pieces 140 00:07:25,710 --> 00:07:29,820 as they fall such that you never overflow 141 00:07:29,820 --> 00:07:34,130 the top of the board on an n by n board? 142 00:07:34,130 --> 00:07:37,850 This problem can be solved in exponential time. 143 00:07:37,850 --> 00:07:40,820 But we don't know whether it can be solved in polynomial time. 144 00:07:48,412 --> 00:07:50,120 We will talk about that more in a moment. 145 00:07:50,120 --> 00:07:53,750 It's a problem that very likely is not in P, 146 00:07:53,750 --> 00:07:57,080 but we can't actually prove it yet. 147 00:07:57,080 --> 00:07:59,778 All right, so there's one other class 148 00:07:59,778 --> 00:08:01,070 I want to define at this point. 149 00:08:01,070 --> 00:08:03,470 And we'll get to a fourth one also. 150 00:08:03,470 --> 00:08:07,490 But R is the class of all problems that 151 00:08:07,490 --> 00:08:10,360 can be solved in finite time. 152 00:08:10,360 --> 00:08:11,335 R stands for finite. 153 00:08:23,660 --> 00:08:25,520 R stands for recursive, actually. 154 00:08:25,520 --> 00:08:27,620 This is a notion by Church way back 155 00:08:27,620 --> 00:08:29,690 in the foundations of computing. 156 00:08:29,690 --> 00:08:32,570 As we know, we write recursive algorithms to solve problems. 157 00:08:32,570 --> 00:08:34,070 In the beginning, that was the only way to do it. 158 00:08:34,070 --> 00:08:35,487 Now we have other ways with loops. 159 00:08:35,487 --> 00:08:38,210 But they're all effectively recursion in the end. 160 00:08:38,210 --> 00:08:40,669 So R is all the problems that can be solved 161 00:08:40,669 --> 00:08:43,130 in finite time on any computer. 162 00:08:43,130 --> 00:08:45,320 So very general, this should include 163 00:08:45,320 --> 00:08:47,420 everything we care about. 164 00:08:47,420 --> 00:08:49,010 And it's bigger than EXP, but includes 165 00:08:49,010 --> 00:08:53,750 problems that take doubly exponential time or whatever. 166 00:08:53,750 --> 00:09:00,330 So I will draw a region for R. So everything-- 167 00:09:00,330 --> 00:09:03,000 it includes P. It includes EXP. 168 00:09:03,000 --> 00:09:05,580 And so we also have containment but not 169 00:09:05,580 --> 00:09:10,003 equal R. There's, of course, many classes in between. 170 00:09:10,003 --> 00:09:11,420 You could talk about problems that 171 00:09:11,420 --> 00:09:14,150 take double A exponential time, and that would have a thing 172 00:09:14,150 --> 00:09:15,230 in between here. 173 00:09:15,230 --> 00:09:17,450 Or there's also-- between P and EXP 174 00:09:17,450 --> 00:09:18,920 there's a lot of different things. 175 00:09:18,920 --> 00:09:22,170 We will talk about one of them. 176 00:09:22,170 --> 00:09:25,310 But before we get to the finer side of things, 177 00:09:25,310 --> 00:09:30,305 let me talk in particular about R. So we have a nice example, 178 00:09:30,305 --> 00:09:34,580 we being computational complexity theory-- 179 00:09:34,580 --> 00:09:37,460 or I guess this is usually just called theoretical computer 180 00:09:37,460 --> 00:09:38,120 science-- 181 00:09:38,120 --> 00:09:40,250 has a problem. 182 00:09:40,250 --> 00:09:49,070 And if you're interested in this, you can take 6041, 183 00:09:49,070 --> 00:09:50,908 I think. 184 00:09:50,908 --> 00:09:51,950 That doesn't sound right. 185 00:09:51,950 --> 00:09:53,480 That's a probability. 186 00:09:53,480 --> 00:09:54,230 It'll come to me. 187 00:10:02,470 --> 00:10:07,153 We have an explicit problem that is not in R. 188 00:10:07,153 --> 00:10:09,070 So this class has been all about problems that 189 00:10:09,070 --> 00:10:11,190 are in P. You have the number? 190 00:10:11,190 --> 00:10:12,130 AUDIENCE: 6045. 191 00:10:12,130 --> 00:10:13,480 ERIK DEMAINE: 6045, thank you. 192 00:10:13,480 --> 00:10:15,340 It's so close to this class. 193 00:10:15,340 --> 00:10:18,340 Or it's so close to 6046, which is the natural successor 194 00:10:18,340 --> 00:10:19,700 to this class. 195 00:10:19,700 --> 00:10:24,040 So in 6045 we talk about this. 196 00:10:24,040 --> 00:10:25,890 So this class is all about problems that 197 00:10:25,890 --> 00:10:27,720 are in P, which is very easy. 198 00:10:27,720 --> 00:10:34,740 But in fact, there are problems way out here beyond R. 199 00:10:34,740 --> 00:10:38,310 And here is one such problem, which 200 00:10:38,310 --> 00:10:39,780 we won't prove here today. 201 00:10:39,780 --> 00:10:43,380 It takes a whole lecture to prove this. 202 00:10:43,380 --> 00:10:49,340 Given a computer program, does it ever halt? 203 00:10:59,160 --> 00:11:00,563 Does it ever terminate? 204 00:11:00,563 --> 00:11:02,730 This would be a great thing if we knew how to solve. 205 00:11:02,730 --> 00:11:04,848 It's basically an infinite loop detector. 206 00:11:04,848 --> 00:11:06,390 If your problem doesn't halt, then it 207 00:11:06,390 --> 00:11:07,920 has an infinite loop of some sort. 208 00:11:07,920 --> 00:11:10,150 And you'd like to tell your user, hey, 209 00:11:10,150 --> 00:11:11,860 you have a bug in your program. 210 00:11:11,860 --> 00:11:13,890 So this is one part of bug detection. 211 00:11:13,890 --> 00:11:14,980 And it's impossible. 212 00:11:14,980 --> 00:11:17,550 There is no algorithm that always-- 213 00:11:17,550 --> 00:11:19,800 that solves all inputs to this problem. 214 00:11:19,800 --> 00:11:23,157 Maybe given one program that, say, has 0 lines of code, 215 00:11:23,157 --> 00:11:23,990 it could solve that. 216 00:11:23,990 --> 00:11:25,950 It says, yeah, that one terminates. 217 00:11:25,950 --> 00:11:28,870 And maybe you can detect simple kinds of infinite loops. 218 00:11:28,870 --> 00:11:30,960 So there's some inputs, some computer programs 219 00:11:30,960 --> 00:11:32,040 that you could detect. 220 00:11:32,040 --> 00:11:36,970 But there's no one algorithm that solves all inputs. 221 00:11:36,970 --> 00:11:39,760 This is kind of sad news. 222 00:11:39,760 --> 00:11:42,110 We call such problems uncomputable. 223 00:11:42,110 --> 00:11:51,450 This is just another word for being not in R. 224 00:11:51,450 --> 00:11:54,300 OK, and next thing I'd like to do 225 00:11:54,300 --> 00:11:57,600 is prove to you that most decision 226 00:11:57,600 --> 00:12:00,630 problems are uncomputable, or sketcher proof. 227 00:12:08,740 --> 00:12:10,910 So remember, decision problems are problems where 228 00:12:10,910 --> 00:12:12,920 the answer is just yes or no. 229 00:12:12,920 --> 00:12:14,630 This is a very special kind of problem. 230 00:12:18,620 --> 00:12:23,660 And even those, almost all of them, cannot be solved. 231 00:12:23,660 --> 00:12:27,200 So halting is an example of a problem we want-- 232 00:12:27,200 --> 00:12:28,460 we can't solve. 233 00:12:28,460 --> 00:12:32,552 This whole class, this 006, is about problems we can solve. 234 00:12:32,552 --> 00:12:34,760 But today I'm going to show you that, actually, those 235 00:12:34,760 --> 00:12:35,640 are in the minority. 236 00:12:35,640 --> 00:12:37,400 Most problems cannot be computed. 237 00:12:37,400 --> 00:12:41,810 This is very strange and also a little depressing. 238 00:12:41,810 --> 00:12:43,620 So we'll talk more about that in a moment. 239 00:12:43,620 --> 00:12:45,840 First let me argue why this is the case. 240 00:12:45,840 --> 00:12:48,230 So I'm going to be a little informal 241 00:12:48,230 --> 00:12:51,650 about what exactly is a computer program 242 00:12:51,650 --> 00:12:53,720 and what exactly is a decision problem. 243 00:12:53,720 --> 00:12:58,370 But roughly, all I need to do, the only level 244 00:12:58,370 --> 00:13:01,670 of precision I need is just to count how many are there. 245 00:13:01,670 --> 00:13:03,440 What is a computer program? 246 00:13:03,440 --> 00:13:04,940 Well, it's usually a file. 247 00:13:04,940 --> 00:13:05,630 What's a file? 248 00:13:05,630 --> 00:13:07,290 It's like a string of characters. 249 00:13:07,290 --> 00:13:08,120 What's a character? 250 00:13:08,120 --> 00:13:09,280 It's a string of bits. 251 00:13:09,280 --> 00:13:12,200 So a program is just, in the end, a string of bits, 252 00:13:12,200 --> 00:13:14,000 finite string of bits. 253 00:13:18,112 --> 00:13:19,070 We all understand that. 254 00:13:19,070 --> 00:13:23,360 Whatever language you define, in the end, 255 00:13:23,360 --> 00:13:26,300 every program is just a string of bits. 256 00:13:26,300 --> 00:13:30,030 And a string of bits we can translate into a number. 257 00:13:30,030 --> 00:13:36,110 So we can convert between strings of bits and numbers. 258 00:13:36,110 --> 00:13:38,300 When I say number, I mean what's usually 259 00:13:38,300 --> 00:13:43,320 called a natural number or a non-negative integer. 260 00:13:43,320 --> 00:13:51,470 This is usually represented by bold board bold-- 261 00:13:51,470 --> 00:13:54,740 blackboard bold capital N. So this 262 00:13:54,740 --> 00:13:56,870 is just 0, 1, 2, and so on. 263 00:14:02,270 --> 00:14:03,905 Now, what about decision problems? 264 00:14:10,850 --> 00:14:13,820 Decision problem is a specification 265 00:14:13,820 --> 00:14:14,940 of what we want to solve. 266 00:14:14,940 --> 00:14:17,360 So we can think of it as saying, for every input, 267 00:14:17,360 --> 00:14:19,190 is the answer yes or no? 268 00:14:19,190 --> 00:14:21,710 That's literally what a decision problem is. 269 00:14:21,710 --> 00:14:23,870 The only question is, what is an input? 270 00:14:23,870 --> 00:14:26,900 And we've talked about inputs and the size of inputs. 271 00:14:26,900 --> 00:14:30,090 And there's lots of different ways to measure them. 272 00:14:30,090 --> 00:14:32,660 But in the end, we can think of an input as a string of bits 273 00:14:32,660 --> 00:14:33,170 also. 274 00:14:33,170 --> 00:14:34,930 It's just a file. 275 00:14:34,930 --> 00:14:41,980 So a decision problem is a function 276 00:14:41,980 --> 00:14:46,480 from inputs to yes or no. 277 00:14:50,410 --> 00:14:53,320 And inputs we're going to say, well, that's 278 00:14:53,320 --> 00:15:03,506 a string of bits, which we can associate with a number in N. 279 00:15:03,506 --> 00:15:05,680 So here we can start to tie things together. 280 00:15:08,230 --> 00:15:12,060 So in other words, a program is a finite string of bits, 281 00:15:12,060 --> 00:15:17,310 and a problem is, in some sense, an infinite string of bits 282 00:15:17,310 --> 00:15:20,050 because there are infinitely many possible inputs. 283 00:15:20,050 --> 00:15:23,280 And for each of them, we specify yes or no. 284 00:15:23,280 --> 00:15:31,940 So this is basically an infinite string of bits. 285 00:15:38,050 --> 00:15:46,140 So we can imagine 011010001110, infinitely. 286 00:15:46,140 --> 00:15:48,720 Just some-- for every string of bits, we can say, OK, 287 00:15:48,720 --> 00:15:52,050 if your input is the number 0, here's the answer-- 288 00:15:52,050 --> 00:15:53,070 no. 289 00:15:53,070 --> 00:15:57,110 If your input is the number 1, then the answer is yes. 290 00:15:57,110 --> 00:15:58,860 If your input is the number 2, your answer 291 00:15:58,860 --> 00:16:01,080 is yes, and so on down this line. 292 00:16:01,080 --> 00:16:02,970 Every infinite string of bits corresponds 293 00:16:02,970 --> 00:16:05,040 to exactly one decision problem, which 294 00:16:05,040 --> 00:16:07,980 specifies for every possible input integer, which 295 00:16:07,980 --> 00:16:12,100 corresponds to a string of bits, what is the answer, yes or no? 296 00:16:12,100 --> 00:16:15,010 So this may seem subtle, or it may seem like not a big deal. 297 00:16:15,010 --> 00:16:16,390 This is a finite string of bits. 298 00:16:16,390 --> 00:16:17,850 This is an infinite string of bits. 299 00:16:17,850 --> 00:16:21,030 But mathematics has well studied this problem. 300 00:16:21,030 --> 00:16:26,040 And infinite strings of bits, there are very many of them, 301 00:16:26,040 --> 00:16:26,737 infinitely many. 302 00:16:26,737 --> 00:16:27,570 It's not surprising. 303 00:16:27,570 --> 00:16:29,830 There are also infinitely many integers. 304 00:16:29,830 --> 00:16:32,190 So maybe it doesn't seem that deep. 305 00:16:32,190 --> 00:16:37,990 But there's a difference in infinitude. 306 00:16:37,990 --> 00:16:42,761 Programs and integers are countably infinite. 307 00:16:47,580 --> 00:16:50,640 And infinite strings of bits are what's called uncountable. 308 00:16:56,210 --> 00:16:58,250 I think the most intuitive way to see 309 00:16:58,250 --> 00:17:00,540 this is, an infinite string of bits, 310 00:17:00,540 --> 00:17:04,849 if I put a decimal or a binary point in front, 311 00:17:04,849 --> 00:17:09,530 this encodes a real number between 0 and 1. 312 00:17:09,530 --> 00:17:17,369 So this is roughly a real number in 01. 313 00:17:17,369 --> 00:17:19,849 And when I'm writing approximately equal here, 314 00:17:19,849 --> 00:17:21,349 this really goes in both directions. 315 00:17:21,349 --> 00:17:24,109 Given a decision problem, I can define a string of bits, 316 00:17:24,109 --> 00:17:26,030 of course giving me the answer for all inputs. 317 00:17:26,030 --> 00:17:28,447 And I can convert that into a real number between 0 and 1. 318 00:17:28,447 --> 00:17:31,070 But also the other direction, if I take any real number, 319 00:17:31,070 --> 00:17:33,030 that is a corresponding decision problem. 320 00:17:33,030 --> 00:17:36,740 These are 1 to 1 bijection between them. 321 00:17:36,740 --> 00:17:39,710 And the bad news is, real numbers are uncountable, 322 00:17:39,710 --> 00:17:42,050 and natural numbers are countable, 323 00:17:42,050 --> 00:17:47,660 which means there's a lot more of these than there are these. 324 00:17:47,660 --> 00:17:55,490 So one way you might phrase this is, informally, 325 00:17:55,490 --> 00:17:57,350 the number of natural numbers is way smaller 326 00:17:57,350 --> 00:17:59,340 than the number of real numbers. 327 00:17:59,340 --> 00:18:05,120 And so from that, we derive that most problems are unsolvable, 328 00:18:05,120 --> 00:18:10,220 because every program solves exactly one decision problem. 329 00:18:10,220 --> 00:18:12,590 We can also run a program, conceptually, 330 00:18:12,590 --> 00:18:14,810 on all possible inputs, and we will figure out 331 00:18:14,810 --> 00:18:16,290 what function at solving. 332 00:18:16,290 --> 00:18:18,980 And if we don't allow random numbers in our program, which 333 00:18:18,980 --> 00:18:23,490 I'm not here, then every program solves exactly one decision 334 00:18:23,490 --> 00:18:23,990 problem. 335 00:18:23,990 --> 00:18:27,080 Possibly, it's even worse for us because multiple programs 336 00:18:27,080 --> 00:18:29,150 probably solve the same decision problem. 337 00:18:29,150 --> 00:18:31,200 They're just-- they add irrelevant lines of code 338 00:18:31,200 --> 00:18:32,700 or they don't do anything different. 339 00:18:32,700 --> 00:18:35,690 Or you run Bellman-Ford versus running Bellman-Ford 340 00:18:35,690 --> 00:18:39,780 five times, you'll get the same result. 341 00:18:39,780 --> 00:18:41,748 And that's actually the bad direction for us. 342 00:18:41,748 --> 00:18:43,790 We'd like to know whether there is a program that 343 00:18:43,790 --> 00:18:46,290 solves every decision problem. 344 00:18:46,290 --> 00:18:50,130 And because there are only this many programs 345 00:18:50,130 --> 00:18:52,130 and this many decision problems, it just-- there 346 00:18:52,130 --> 00:18:53,930 aren't enough to go around. 347 00:18:53,930 --> 00:18:59,670 So most-- what's the phrasing? 348 00:18:59,670 --> 00:19:05,460 Not nearly enough programs for all problems, and so there's 349 00:19:05,460 --> 00:19:17,980 no assignment of programs to problems 350 00:19:17,980 --> 00:19:19,605 because there's just too many problems. 351 00:19:22,610 --> 00:19:25,880 More money, more problems, I guess. 352 00:19:25,880 --> 00:19:27,470 So when I first saw this result, I 353 00:19:27,470 --> 00:19:29,488 was shocked and dismayed that-- 354 00:19:29,488 --> 00:19:32,030 why are we even doing computer science if most problems can't 355 00:19:32,030 --> 00:19:32,780 be solved? 356 00:19:32,780 --> 00:19:35,480 Luckily, it seems like most of the problems we care about 357 00:19:35,480 --> 00:19:36,350 can be solved. 358 00:19:36,350 --> 00:19:38,020 That's what this class is all about. 359 00:19:38,020 --> 00:19:40,020 And in fact, even the problems that seem really, 360 00:19:40,020 --> 00:19:43,460 really hard for us to solve, like n by n chess, where we can 361 00:19:43,460 --> 00:19:45,380 prove it takes exponential time, there 362 00:19:45,380 --> 00:19:46,700 is an algorithm to solve chess. 363 00:19:46,700 --> 00:19:48,497 It's just really slow. 364 00:19:48,497 --> 00:19:50,330 And this is a statement about, most problems 365 00:19:50,330 --> 00:19:51,980 can even be solved in finite time 366 00:19:51,980 --> 00:19:54,830 no matter how much time you give them. 367 00:19:54,830 --> 00:19:58,310 So it's not all bad. 368 00:19:58,310 --> 00:20:13,380 Luckily, most problems we care about 369 00:20:13,380 --> 00:20:17,370 are in R. I don't know why. 370 00:20:17,370 --> 00:20:19,770 This is sort of a mystery of life. 371 00:20:19,770 --> 00:20:22,470 But it's good news. 372 00:20:22,470 --> 00:20:28,230 Or it's why we keep persevering trying to solve problems 373 00:20:28,230 --> 00:20:30,322 with algorithms. 374 00:20:30,322 --> 00:20:32,280 AUDIENCE: Is it because when we state problems, 375 00:20:32,280 --> 00:20:35,830 the statement tends to be small? 376 00:20:35,830 --> 00:20:37,030 ERIK DEMAINE: Well, this-- 377 00:20:37,030 --> 00:20:38,530 so the question was, maybe it's just 378 00:20:38,530 --> 00:20:40,950 because these short statement problems are easy. 379 00:20:40,950 --> 00:20:44,820 But this is a pretty short statement, and it's hard. 380 00:20:44,820 --> 00:20:49,310 I think-- I don't have a great reason why. 381 00:20:49,310 --> 00:20:50,750 I wish I understood. 382 00:20:50,750 --> 00:20:52,250 There's a general result that if you 383 00:20:52,250 --> 00:20:55,355 have any question about the program itself, then 384 00:20:55,355 --> 00:20:56,730 there's no algorithm to solve it. 385 00:20:56,730 --> 00:20:59,090 Basically, any non-trivial question about programs is 386 00:20:59,090 --> 00:21:04,190 hard, is not in R. And I guess if you took-- 387 00:21:04,190 --> 00:21:07,168 if you imagine taking a random statement of a problem, 388 00:21:07,168 --> 00:21:08,960 then maybe this will be in the middle of it 389 00:21:08,960 --> 00:21:10,790 with some probability. 390 00:21:10,790 --> 00:21:11,990 Maybe that's why most. 391 00:21:11,990 --> 00:21:14,150 But this is a very strong notion of most. 392 00:21:14,150 --> 00:21:16,520 There are so many more real numbers than natural numbers 393 00:21:16,520 --> 00:21:17,750 that-- 394 00:21:17,750 --> 00:21:18,380 I don't know. 395 00:21:23,780 --> 00:21:31,580 I want to add one more class to this picture, which is NP. 396 00:21:31,580 --> 00:21:35,150 It nestles in between P and EXP. 397 00:21:35,150 --> 00:21:42,980 So we know that P is contained in or equal to NP. 398 00:21:42,980 --> 00:21:46,780 And NP is contained in or equal to EXP. 399 00:21:46,780 --> 00:21:49,400 We don't know whether there's a quality here or here. 400 00:21:49,400 --> 00:21:52,370 Probably not, but we can't prove it. 401 00:21:52,370 --> 00:21:53,865 But what is this class? 402 00:21:53,865 --> 00:21:56,420 A couple of different ways to define it-- 403 00:21:56,420 --> 00:22:01,340 you might find one way or the other more intuitive. 404 00:22:01,340 --> 00:22:02,160 They're equivalent. 405 00:22:02,160 --> 00:22:07,820 So as long as you understand at least one of them, it's good. 406 00:22:07,820 --> 00:22:12,010 NP is just a class of decision problems. 407 00:22:12,010 --> 00:22:15,980 So I define P and EXP and R arbitrary. 408 00:22:15,980 --> 00:22:18,350 They can be problems with any kind of output. 409 00:22:18,350 --> 00:22:21,440 But NP only makes sense for decision problems. 410 00:22:21,440 --> 00:22:27,980 And it's going to look almost like the definition of P-- 411 00:22:32,420 --> 00:22:34,340 problem solvable in polynomial time. 412 00:22:34,340 --> 00:22:36,260 We've just restricted to decision problems. 413 00:22:36,260 --> 00:22:40,220 But we're going to allow a strange kind 414 00:22:40,220 --> 00:22:44,000 of computer or algorithm, which I 415 00:22:44,000 --> 00:22:46,659 like to call a lucky algorithm. 416 00:22:50,880 --> 00:22:53,150 And this is going to relate to the notion of guessing 417 00:22:53,150 --> 00:22:55,190 that we talked about for the last four lectures 418 00:22:55,190 --> 00:22:57,320 in dynamic programming. 419 00:22:57,320 --> 00:22:59,330 With dynamic programming, we said, oh, there 420 00:22:59,330 --> 00:23:01,210 are all these different choices I could make. 421 00:23:01,210 --> 00:23:02,210 What's the right choice? 422 00:23:02,210 --> 00:23:05,840 I don't know, so I'd like to make a guess. 423 00:23:05,840 --> 00:23:08,180 And what that meant in terms of a real algorithm is, 424 00:23:08,180 --> 00:23:09,950 we tried all of the possibilities, 425 00:23:09,950 --> 00:23:12,170 and then took the max or the OR or whatever 426 00:23:12,170 --> 00:23:14,750 over all those possibilities. 427 00:23:14,750 --> 00:23:15,620 And so we were-- 428 00:23:15,620 --> 00:23:17,690 but what we were simulating is something 429 00:23:17,690 --> 00:23:20,570 that I call a lucky algorithm, which can make guesses 430 00:23:20,570 --> 00:23:22,790 and always makes the right guess. 431 00:23:22,790 --> 00:23:25,590 This is a computer that is impossible to buy. 432 00:23:25,590 --> 00:23:28,310 It would be great if you could buy a computer that's lucky. 433 00:23:28,310 --> 00:23:30,268 But we don't know how to build such a computer. 434 00:23:32,460 --> 00:23:35,820 So what does this mean? 435 00:23:35,820 --> 00:23:37,790 So informally, it means your algorithm 436 00:23:37,790 --> 00:23:41,330 can make lucky guesses, and it always makes the right guess. 437 00:23:41,330 --> 00:23:43,790 And whereas in DP, we had to try all the options 438 00:23:43,790 --> 00:23:46,400 and spend time for all of them, the lucky algorithm only 439 00:23:46,400 --> 00:23:49,940 has to spend time on the lucky guess, on the correct guess. 440 00:23:49,940 --> 00:23:54,410 More formally, this is called a non-deterministic model 441 00:23:54,410 --> 00:23:55,778 of computation. 442 00:24:02,330 --> 00:24:04,310 And this N is the-- 443 00:24:04,310 --> 00:24:06,830 the N in non-determinism is the N for NP. 444 00:24:06,830 --> 00:24:09,110 So this is non-deterministic polynomial time. 445 00:24:11,800 --> 00:24:17,670 So algorithm can make guesses. 446 00:24:23,610 --> 00:24:27,870 And then in the end, it should output yes or no. 447 00:24:33,750 --> 00:24:37,162 Like say if you're exploring a maze, this algorithm could say, 448 00:24:37,162 --> 00:24:38,370 should I go left or go right? 449 00:24:38,370 --> 00:24:40,470 I'm going to guess whether to go left or go right. 450 00:24:40,470 --> 00:24:42,010 And let's say it guesses left. 451 00:24:42,010 --> 00:24:43,260 And so then it just goes left. 452 00:24:43,260 --> 00:24:45,232 And then it reaches another junction. 453 00:24:45,232 --> 00:24:46,690 It says, should I go left or right? 454 00:24:46,690 --> 00:24:48,190 And it'll say, I'll guess, and it'll 455 00:24:48,190 --> 00:24:49,590 say, guess right this time. 456 00:24:49,590 --> 00:24:53,393 And in the end, if I get to some dead end maybe and I say no, 457 00:24:53,393 --> 00:24:55,560 or if I get to the destination I'm trying to get to, 458 00:24:55,560 --> 00:24:56,280 I say yes. 459 00:24:56,280 --> 00:24:58,720 So that's a non-deterministic algorithm. 460 00:24:58,720 --> 00:25:02,550 And what does it mean to run that algorithm? 461 00:25:02,550 --> 00:25:06,090 What does it mean for the guesses to be lucky? 462 00:25:06,090 --> 00:25:07,920 Here's what it means. 463 00:25:07,920 --> 00:25:11,490 These guesses are guaranteed-- 464 00:25:11,490 --> 00:25:15,372 which way you end up going is guaranteed to lead you to a yes 465 00:25:15,372 --> 00:25:16,080 if there is one-- 466 00:25:25,910 --> 00:25:27,960 if possible. 467 00:25:27,960 --> 00:25:32,420 So in my maze analogy, if my destination 468 00:25:32,420 --> 00:25:34,640 is reachable from my source, then 469 00:25:34,640 --> 00:25:38,420 I'm guaranteed, whenever I guessed left or right, 470 00:25:38,420 --> 00:25:42,410 I will choose a path that leads me to my destination. 471 00:25:42,410 --> 00:25:44,870 Whereas, if the destination is in some disconnected part 472 00:25:44,870 --> 00:25:47,520 of the maze and I can't get there, 473 00:25:47,520 --> 00:25:49,168 then I don't know what the guesses do. 474 00:25:49,168 --> 00:25:50,210 It doesn't really matter. 475 00:25:50,210 --> 00:25:52,460 Because no matter what I do, I'll end up in a dead end 476 00:25:52,460 --> 00:25:54,230 and say no. 477 00:25:54,230 --> 00:25:56,610 That's the model. 478 00:25:56,610 --> 00:25:59,802 As long as you have an algorithm that always outputs yes or no 479 00:25:59,802 --> 00:26:02,010 in polynomial time-- because we're only talking about 480 00:26:02,010 --> 00:26:04,560 polynomial time, lucky algorithms-- 481 00:26:04,560 --> 00:26:06,360 if there's any way to get to a yes, 482 00:26:06,360 --> 00:26:10,220 then your machine will magically find it 483 00:26:10,220 --> 00:26:13,480 without having to spend any time to make these decisions. 484 00:26:13,480 --> 00:26:16,100 So it's a pretty magical computer, 485 00:26:16,100 --> 00:26:18,620 and it's not a computer that exists in real life. 486 00:26:18,620 --> 00:26:21,430 But it's a computer that's great to program on. 487 00:26:21,430 --> 00:26:22,345 It's very powerful. 488 00:26:22,345 --> 00:26:23,970 You could solve lots of things with it. 489 00:26:23,970 --> 00:26:24,590 Yeah. 490 00:26:24,590 --> 00:26:27,800 AUDIENCE: If you had this magical computer, 491 00:26:27,800 --> 00:26:30,080 it can guess whether it's yes or no, 492 00:26:30,080 --> 00:26:33,117 why doesn't it just answer the question? 493 00:26:33,117 --> 00:26:33,950 ERIK DEMAINE: Right. 494 00:26:33,950 --> 00:26:37,200 So what if we-- so a nice check is, 495 00:26:37,200 --> 00:26:39,745 does this make all problems trivial, all decision problems? 496 00:26:39,745 --> 00:26:41,120 Maybe I should say, well, I don't 497 00:26:41,120 --> 00:26:42,703 know whether the answer to the problem 498 00:26:42,703 --> 00:26:45,590 is yes or no, so I'll just guess yes or no. 499 00:26:45,590 --> 00:26:49,470 This is problematic because-- 500 00:26:49,470 --> 00:26:51,930 so I might say, it will guess A or B, 501 00:26:51,930 --> 00:26:54,750 and if I choose the A option, I will output yes, 502 00:26:54,750 --> 00:26:56,910 and if I choose the B option, I will output no. 503 00:26:56,910 --> 00:27:00,055 In this model, that algorithm will always output yes. 504 00:27:00,055 --> 00:27:01,680 Because what it's saying is, if there's 505 00:27:01,680 --> 00:27:05,800 any way to get to a yes answer, I will do that way. 506 00:27:05,800 --> 00:27:08,598 And so such an algorithm that tries to cheat and just 507 00:27:08,598 --> 00:27:10,140 guess the whole answer to the problem 508 00:27:10,140 --> 00:27:11,310 will actually end up always saying 509 00:27:11,310 --> 00:27:12,600 yes, which means it doesn't solve 510 00:27:12,600 --> 00:27:13,725 a very interesting problem. 511 00:27:13,725 --> 00:27:15,780 It only solves the problem, which 512 00:27:15,780 --> 00:27:20,220 is represented by the bit vector 1111111, 513 00:27:20,220 --> 00:27:21,930 where all the answers are yes. 514 00:27:21,930 --> 00:27:23,520 But good check. 515 00:27:23,520 --> 00:27:24,610 Yeah. 516 00:27:24,610 --> 00:27:26,235 AUDIENCE: Does there have to be a bound 517 00:27:26,235 --> 00:27:30,330 of a number of things it has to choose between when it 518 00:27:30,330 --> 00:27:31,075 [AUDIO OUT] 519 00:27:31,075 --> 00:27:31,825 ERIK DEMAINE: Yes. 520 00:27:31,825 --> 00:27:34,110 AUDIENCE: Does it have an exponential number of them? 521 00:27:36,750 --> 00:27:39,690 ERIK DEMAINE: Exponential number of choices is OK. 522 00:27:39,690 --> 00:27:42,240 I usually like to think of it, as you can only 523 00:27:42,240 --> 00:27:45,270 guess one bit at a time. 524 00:27:45,270 --> 00:27:46,932 But we're allowed polynomial time, 525 00:27:46,932 --> 00:27:48,390 so you're actually allowed to guess 526 00:27:48,390 --> 00:27:49,870 polynomial number of bits. 527 00:27:49,870 --> 00:27:52,710 At that point, you can guess over an exponential size space, 528 00:27:52,710 --> 00:27:55,840 but not more than exponential. 529 00:27:55,840 --> 00:27:59,070 So it's-- yeah, polynomial time let's say in the one-bit 530 00:27:59,070 --> 00:28:00,570 guessing model. 531 00:28:00,570 --> 00:28:02,430 What did I say? 532 00:28:02,430 --> 00:28:07,440 Makes guesses-- let's add binary here. 533 00:28:07,440 --> 00:28:12,210 Otherwise we get some other class, which I don't want. 534 00:28:12,210 --> 00:28:16,020 OK, let's do an example, a real example of such an algorithm 535 00:28:16,020 --> 00:28:20,550 that's useful, which is Tetris. 536 00:28:20,550 --> 00:28:27,960 So I claim Tetris is in NP because there 537 00:28:27,960 --> 00:28:31,140 is a lucky algorithm and non-deterministic polynomial 538 00:28:31,140 --> 00:28:34,260 time algorithm that can solve the Tetris game. 539 00:28:34,260 --> 00:28:36,600 So again, you're given a board, you're 540 00:28:36,600 --> 00:28:39,870 given some sequence of pieces, and you 541 00:28:39,870 --> 00:28:41,730 want to know whether there's some way 542 00:28:41,730 --> 00:28:44,920 to place the pieces that lets you survive. 543 00:28:44,920 --> 00:28:53,790 And so what I'm going to do is, for each piece, 544 00:28:53,790 --> 00:28:56,820 I'm going to guess how to place it. 545 00:28:59,915 --> 00:29:01,290 So for the first piece, I'm going 546 00:29:01,290 --> 00:29:04,440 to guess how far left or right do I move it. 547 00:29:04,440 --> 00:29:05,820 Then I let it fall one step. 548 00:29:05,820 --> 00:29:07,290 Maybe I rotate it. 549 00:29:07,290 --> 00:29:10,560 I choose a sequence of moves among left, right, down, 550 00:29:10,560 --> 00:29:13,440 rotate right, rotate left. 551 00:29:13,440 --> 00:29:17,370 And all along the way, I check, is that move valid? 552 00:29:17,370 --> 00:29:21,190 If the move is invalid at any point, I just say, return no. 553 00:29:21,190 --> 00:29:24,160 And then if the piece gets nestled into a good spot, 554 00:29:24,160 --> 00:29:25,570 I continue to the next piece. 555 00:29:25,570 --> 00:29:27,653 I do the same thing, guess all the possible things 556 00:29:27,653 --> 00:29:28,540 I could do to that. 557 00:29:28,540 --> 00:29:31,750 Again, I only need to guess one bit at a time. 558 00:29:31,750 --> 00:29:34,090 And I'll only need to do a polynomial number of guesses, 559 00:29:34,090 --> 00:29:37,060 like a linear number of guesses, for each piece about where 560 00:29:37,060 --> 00:29:39,220 it falls in, so maybe a quadratic number of guesses 561 00:29:39,220 --> 00:29:40,600 overall. 562 00:29:40,600 --> 00:29:42,955 And then at the end, if I survived-- 563 00:29:42,955 --> 00:29:45,260 oh, I also have to check if a line clears. 564 00:29:45,260 --> 00:29:47,380 Then I clear the line. 565 00:29:47,380 --> 00:29:50,260 And if in the end I survive, I return yes. 566 00:29:50,260 --> 00:29:53,800 So this is a non-deterministic algorithm. 567 00:29:53,800 --> 00:29:58,220 So I would say, check the rules of the game. 568 00:29:58,220 --> 00:30:06,460 And if we survive, return yes. 569 00:30:06,460 --> 00:30:10,270 And if at any point we violate the rules-- for example, 570 00:30:10,270 --> 00:30:12,220 we go off the top of the board-- 571 00:30:12,220 --> 00:30:13,933 we return no. 572 00:30:13,933 --> 00:30:15,850 So this is an algorithm that sometimes returns 573 00:30:15,850 --> 00:30:17,920 no and sometimes returns yes depending 574 00:30:17,920 --> 00:30:19,330 on what choices you make. 575 00:30:19,330 --> 00:30:21,070 And this model guarantees, if there's 576 00:30:21,070 --> 00:30:24,660 any way to get to a yes, it will find it. 577 00:30:24,660 --> 00:30:28,980 If I swapped these answers, if I returned yes 578 00:30:28,980 --> 00:30:32,192 when I violated the rules and returned no if I survived, 579 00:30:32,192 --> 00:30:33,900 this would be an uninteresting algorithm. 580 00:30:33,900 --> 00:30:36,460 Because it's very easy to lose in Tetris. 581 00:30:36,460 --> 00:30:38,850 The hard part is to survive. 582 00:30:38,850 --> 00:30:41,760 If I say, is there any way to play the game in such a way 583 00:30:41,760 --> 00:30:45,000 that I violate the rules, then, of course, the answer is yes. 584 00:30:45,000 --> 00:30:47,980 You can just stack pieces and go off the top. 585 00:30:47,980 --> 00:30:52,180 There's an asymmetry in this definition of yes versus no. 586 00:30:52,180 --> 00:30:55,500 It always finds yes answers if possible. 587 00:30:55,500 --> 00:30:58,090 It doesn't always find no answers if possible. 588 00:30:58,090 --> 00:31:01,230 So it's very important the way that I wrote these questions. 589 00:31:01,230 --> 00:31:03,540 It's important that I define Tetris as the problem of, 590 00:31:03,540 --> 00:31:04,890 can I survive? 591 00:31:04,890 --> 00:31:08,490 The problem of can I not survive, 592 00:31:08,490 --> 00:31:11,330 is it impossible to survive, that's a different question. 593 00:31:11,330 --> 00:31:14,130 That problem is not in NP, probably. 594 00:31:14,130 --> 00:31:17,370 OK, so slight subtlety there, yes versus no. 595 00:31:17,370 --> 00:31:19,230 Let me give you the other definition of NP. 596 00:31:19,230 --> 00:31:21,240 So if this one's confusing, which-- 597 00:31:21,240 --> 00:31:23,820 although I prefer this definition. 598 00:31:23,820 --> 00:31:26,290 Most people do not. 599 00:31:26,290 --> 00:31:28,200 So this is confusing. 600 00:31:28,200 --> 00:31:31,630 Let's do the other definition. 601 00:31:31,630 --> 00:32:04,300 So another definition is that NP is a set of decision problems 602 00:32:04,300 --> 00:32:07,120 that can be checked in polynomial time. 603 00:32:18,672 --> 00:32:20,380 This actually came up in the last lecture 604 00:32:20,380 --> 00:32:22,060 where we talked about subset sum. 605 00:32:22,060 --> 00:32:24,820 I said, here's a bunch of integers, 606 00:32:24,820 --> 00:32:26,920 here's a target integer, and I can prove to you 607 00:32:26,920 --> 00:32:30,310 that this integer can be represented as a sum of numbers 608 00:32:30,310 --> 00:32:34,210 from my subset of numbers from my set, because here they are. 609 00:32:34,210 --> 00:32:38,590 I gave you this plus this plus this equals the target sum. 610 00:32:38,590 --> 00:32:41,890 And so that is a solution, in some sense, that can 611 00:32:41,890 --> 00:32:44,720 be checked for a yes example. 612 00:32:44,720 --> 00:32:48,790 If I can represent my number as a subset sum of a given set, 613 00:32:48,790 --> 00:32:50,590 it's easy for me to prove that to you. 614 00:32:50,590 --> 00:32:53,200 And you can check it just by adding up the numbers 615 00:32:53,200 --> 00:32:56,810 and checking that each number was in the set. 616 00:32:56,810 --> 00:33:00,647 Whereas no instances, I had an example of a target sum 617 00:33:00,647 --> 00:33:01,730 that could not be reached. 618 00:33:01,730 --> 00:33:03,563 And the only reason I knew that is because I 619 00:33:03,563 --> 00:33:05,780 had brute-forced the thing. 620 00:33:05,780 --> 00:33:09,500 And there's no succinct way to prove to you that that number 621 00:33:09,500 --> 00:33:11,410 can't be represented. 622 00:33:11,410 --> 00:33:16,880 A similar thing with Tetris, what I would say is-- 623 00:33:16,880 --> 00:33:20,150 so this is version 1, version 2-- 624 00:33:20,150 --> 00:33:29,060 for Tetris is that, a certificate 625 00:33:29,060 --> 00:33:38,990 for a yes input of Tetris is a sequence 626 00:33:38,990 --> 00:33:40,475 of moves for the pieces. 627 00:33:48,520 --> 00:33:50,730 OK, if it's possible to survive in Tetris, 628 00:33:50,730 --> 00:33:51,810 I can prove it to you. 629 00:33:51,810 --> 00:33:55,710 I can just play the game and show you that I survived. 630 00:33:55,710 --> 00:33:58,230 No answers, I don't know, it's hard to prove 631 00:33:58,230 --> 00:34:00,750 to you that I can't survive a given sequence of pieces. 632 00:34:00,750 --> 00:34:02,010 But yes answers are easy. 633 00:34:02,010 --> 00:34:04,947 I just show you, here's the sequence of button presses 634 00:34:04,947 --> 00:34:06,780 I'll do for this piece, then for this piece, 635 00:34:06,780 --> 00:34:07,613 then for this piece. 636 00:34:07,613 --> 00:34:09,840 Notice it's exactly the same thing 637 00:34:09,840 --> 00:34:12,989 that I guessed in the beginning of this algorithm. 638 00:34:12,989 --> 00:34:15,328 And then I did some other work to implement the rules. 639 00:34:15,328 --> 00:34:17,370 And similarly, if I gave you a certificate, which 640 00:34:17,370 --> 00:34:20,760 is the things that I wanted to guess of how to play the game, 641 00:34:20,760 --> 00:34:23,790 I can check this certificate by just implementing 642 00:34:23,790 --> 00:34:26,409 the rules of Tetris and seeing whether I survived. 643 00:34:26,409 --> 00:34:29,219 And if you violate the rules at any point, you say no. 644 00:34:29,219 --> 00:34:32,219 And if you survive, you return yes. 645 00:34:32,219 --> 00:34:34,620 That's what's called a verification algorithm. 646 00:34:34,620 --> 00:34:38,070 So let me formalize this notion. 647 00:34:38,070 --> 00:34:48,480 Given a problem input plus a certificate, 648 00:34:48,480 --> 00:34:56,790 like that one over there, there is a polynomial time-- 649 00:34:56,790 --> 00:34:58,550 so this is yet another definition. 650 00:34:58,550 --> 00:35:01,370 This is what I mean by this definition of NP-- 651 00:35:01,370 --> 00:35:12,250 verification algorithm that satisfies two properties. 652 00:35:12,250 --> 00:35:15,670 One is, for every yes input-- 653 00:35:15,670 --> 00:35:19,770 so every input where the answer is yes to the problem-- 654 00:35:19,770 --> 00:35:28,440 there exists a certificate such that the verifier says yes. 655 00:35:33,720 --> 00:35:36,150 So this is saying, it's possible to prove to me 656 00:35:36,150 --> 00:35:38,700 that an answer is yes, because if you ever 657 00:35:38,700 --> 00:35:41,940 have an input that the answer happens to be yes, 658 00:35:41,940 --> 00:35:44,290 you can prove it to me by giving me a certificate. 659 00:35:44,290 --> 00:35:47,610 There's always some certificate that proves the answer's yes. 660 00:35:47,610 --> 00:35:50,770 Because the verifier, which runs in regular polynomial time-- 661 00:35:50,770 --> 00:35:53,100 this is a regular, old-fashioned, 662 00:35:53,100 --> 00:35:55,440 down-to-earth verification algorithm, 663 00:35:55,440 --> 00:35:57,870 polynomial time in our usual sense-- 664 00:35:57,870 --> 00:35:59,340 it will say yes. 665 00:35:59,340 --> 00:36:01,500 And furthermore, the yes answers from the verifier 666 00:36:01,500 --> 00:36:06,060 are actually meaningful, because if I ever give it a no input, 667 00:36:06,060 --> 00:36:10,230 it always says no, no matter what certificate I give it. 668 00:36:18,730 --> 00:36:21,600 So this should really formalize what all this means. 669 00:36:21,600 --> 00:36:24,030 It's equivalent to the previous definition. 670 00:36:24,030 --> 00:36:27,210 This is saying that proofs exist for yes instances. 671 00:36:27,210 --> 00:36:29,740 And this is saying that proofs don't exist for no instances, 672 00:36:29,740 --> 00:36:31,440 meaning there are no false proofs. 673 00:36:31,440 --> 00:36:33,450 So if the verifier ever outputs yes, 674 00:36:33,450 --> 00:36:36,797 you know that the answer to your problem is yes. 675 00:36:36,797 --> 00:36:38,380 But if it outputs no, you're not sure. 676 00:36:38,380 --> 00:36:40,080 Maybe you got the certificate wrong 677 00:36:40,080 --> 00:36:41,790 because we only know there's some certificate where 678 00:36:41,790 --> 00:36:42,810 the verifier will say yes. 679 00:36:42,810 --> 00:36:44,435 Or maybe it was a no input, and then it 680 00:36:44,435 --> 00:36:46,620 didn't matter what certificate you used. 681 00:36:46,620 --> 00:36:49,390 But it's nice, because it says on, say, Tetris, 682 00:36:49,390 --> 00:36:51,150 if I give you the sequence of pieces, 683 00:36:51,150 --> 00:36:53,692 it's very easy to write down a verifier which just implements 684 00:36:53,692 --> 00:36:55,740 the rules of Tetris. 685 00:36:55,740 --> 00:36:57,960 And so then you can at least check whether a solution 686 00:36:57,960 --> 00:36:59,880 is valid in the yes case. 687 00:36:59,880 --> 00:37:03,040 In the no case, we don't have anything useful. 688 00:37:03,040 --> 00:37:06,660 So NP is a structure, some additional structure 689 00:37:06,660 --> 00:37:09,810 about the yes inputs in your problem. 690 00:37:09,810 --> 00:37:12,600 And a lot of decision problems are in NP. 691 00:37:12,600 --> 00:37:14,850 A lot of the problems that we care about 692 00:37:14,850 --> 00:37:16,380 can be phrased as an NP problem. 693 00:37:16,380 --> 00:37:18,030 As long as it's a decision problem, 694 00:37:18,030 --> 00:37:25,070 usually, answering yes or no is provable, like subset sum, 695 00:37:25,070 --> 00:37:26,335 like Tetris. 696 00:37:26,335 --> 00:37:28,460 These are all problems where, if the answer is yes, 697 00:37:28,460 --> 00:37:31,805 I can give you a convincing proof why. 698 00:37:31,805 --> 00:37:33,680 And it turns out a lot-- so a lot of problems 699 00:37:33,680 --> 00:37:35,700 fall into this NP setting. 700 00:37:35,700 --> 00:37:39,350 And so we have some tools for talking about problems 701 00:37:39,350 --> 00:37:45,510 being hard with respect to NP. 702 00:37:45,510 --> 00:37:50,220 Let me first talk a little bit about P. 703 00:37:50,220 --> 00:37:55,580 Does not equal NP, question mark. 704 00:37:55,580 --> 00:37:58,470 A lot of people conjecture that P does not equal NP. 705 00:37:58,470 --> 00:38:00,750 It's sort of a standard conjecture 706 00:38:00,750 --> 00:38:02,130 in theoretical computer science. 707 00:38:02,130 --> 00:38:06,690 But we don't know how to prove whether P equals NP or does not 708 00:38:06,690 --> 00:38:08,130 equal NP. 709 00:38:08,130 --> 00:38:12,160 And so in this picture, I've drawn the hypothesis, 710 00:38:12,160 --> 00:38:16,513 which is that NP is a strictly bigger region than P is. 711 00:38:16,513 --> 00:38:18,180 But we don't actually know whether there 712 00:38:18,180 --> 00:38:20,203 are problems in this region. 713 00:38:20,203 --> 00:38:21,870 We don't know whether there are problems 714 00:38:21,870 --> 00:38:24,690 in this region between NP and EXP. 715 00:38:24,690 --> 00:38:26,340 We conjecture there are problems here 716 00:38:26,340 --> 00:38:28,170 and there are problems here. 717 00:38:28,170 --> 00:38:30,690 There's definitely problems here or problems here, 718 00:38:30,690 --> 00:38:31,998 but we don't know which one. 719 00:38:31,998 --> 00:38:33,540 Because we know P does not equal EXP, 720 00:38:33,540 --> 00:38:35,123 but we don't know whether P equals NP, 721 00:38:35,123 --> 00:38:37,440 and we don't know whether P equals EXP. 722 00:38:37,440 --> 00:38:41,250 If you could prove that P does not equal NP, or disprove it, 723 00:38:41,250 --> 00:38:44,460 you would win $1 million, which not that much money these days. 724 00:38:44,460 --> 00:38:47,880 But you would be famous to for the rest of time 725 00:38:47,880 --> 00:38:49,710 if you could ever prove this. 726 00:38:49,710 --> 00:38:53,130 Every year, there's usually a crackpot proof 727 00:38:53,130 --> 00:38:56,340 that doesn't work out. 728 00:38:56,340 --> 00:38:57,960 Some of them go to me. 729 00:38:57,960 --> 00:38:59,790 Please don't send them. 730 00:38:59,790 --> 00:39:02,490 And anyway, it's a very hard problem. 731 00:39:02,490 --> 00:39:04,740 It is sort of the core problem in theoretical computer 732 00:39:04,740 --> 00:39:07,380 science, how to prove P does not equal NP. 733 00:39:07,380 --> 00:39:11,470 But for the most part, we just assume it. 734 00:39:11,470 --> 00:39:12,970 Now, what does this conjecture mean? 735 00:39:12,970 --> 00:39:15,300 It essentially means-- the way I like to say it is, 736 00:39:15,300 --> 00:39:17,850 you cannot engineer luck. 737 00:39:17,850 --> 00:39:19,530 Because NP problems are problems you 738 00:39:19,530 --> 00:39:20,910 can solve by lucky algorithms. 739 00:39:20,910 --> 00:39:24,990 P are problems you can solve by regular old algorithms. 740 00:39:24,990 --> 00:39:27,930 And so if P equalled NP, it means 741 00:39:27,930 --> 00:39:30,810 luck doesn't buy you anything, which seems weird. 742 00:39:30,810 --> 00:39:34,650 If I can magically make these super powerful guesses, 743 00:39:34,650 --> 00:39:36,570 then I can solve the problem that that's 744 00:39:36,570 --> 00:39:39,120 NP, that seems super powerful, way 745 00:39:39,120 --> 00:39:41,055 more powerful than regular algorithms, 746 00:39:41,055 --> 00:39:42,930 where we have to actually brute-force and try 747 00:39:42,930 --> 00:39:44,970 all the choices. 748 00:39:44,970 --> 00:39:48,550 And so it seems pretty solid that P does not equal NP. 749 00:39:48,550 --> 00:39:51,090 That's my-- of course, we don't know how to prove it. 750 00:39:51,090 --> 00:39:52,680 Another phrasing is that it's harder 751 00:39:52,680 --> 00:39:56,190 to come up with proofs than it is to check them, 752 00:39:56,190 --> 00:39:57,690 from a mathematical perspective. 753 00:39:57,690 --> 00:40:00,300 This is equivalent to P does not equal NP. 754 00:40:00,300 --> 00:40:03,030 So that's why you should believe it. 755 00:40:03,030 --> 00:40:06,430 Now, let's go over here. 756 00:40:11,740 --> 00:40:16,960 The next notion is NP-hardness. 757 00:40:27,910 --> 00:40:30,730 So in particular, I want to claim-- 758 00:40:30,730 --> 00:40:33,160 this is a theorem that exists in the literature-- 759 00:40:33,160 --> 00:40:40,740 that if P does not equal NP, then Tetris is not NP. 760 00:40:44,180 --> 00:40:46,300 So I said right here, Tetris is in EXP, 761 00:40:46,300 --> 00:40:47,920 but we don't know whether it's in NP. 762 00:40:47,920 --> 00:40:50,140 But in fact, we conjecture it is not NP 763 00:40:50,140 --> 00:40:52,210 because we conjecture that P does not equal NP. 764 00:40:52,210 --> 00:40:53,710 If you could prove this conjecture-- 765 00:40:53,710 --> 00:40:56,252 and there's a lot of theorems that are conditioned assuming P 766 00:40:56,252 --> 00:40:57,190 does not equal NP-- 767 00:40:57,190 --> 00:41:00,160 then we get some nice results, like Tetris cannot be solved 768 00:41:00,160 --> 00:41:01,030 in polynomial time. 769 00:41:01,030 --> 00:41:05,320 It cannot figure out whether I can win a Tetris game 770 00:41:05,320 --> 00:41:08,380 in polynomial time in the input size. 771 00:41:08,380 --> 00:41:10,850 Why? 772 00:41:10,850 --> 00:41:15,590 This is a consequence of another theorem, which 773 00:41:15,590 --> 00:41:21,620 is that Tetris is NP-hard. 774 00:41:21,620 --> 00:41:24,410 I'm going to define NP-hard informally first, 775 00:41:24,410 --> 00:41:27,500 and then I'll define it slightly more formally in a second. 776 00:41:27,500 --> 00:41:32,420 But this means, roughly, that Tetris is 777 00:41:32,420 --> 00:41:38,390 as hard as all problems in NP. 778 00:41:41,780 --> 00:41:46,080 So let me draw this in the picture. 779 00:41:46,080 --> 00:41:52,170 So NP-hard is this part. 780 00:41:54,810 --> 00:41:56,580 Did I leave myself enough room? 781 00:41:56,580 --> 00:41:57,150 Maybe not. 782 00:42:02,310 --> 00:42:05,060 Well, we'll squeeze it in. 783 00:42:05,060 --> 00:42:07,790 There's another region here for EXP-hard. 784 00:42:14,290 --> 00:42:18,670 So your problem being in NP was a positive result. 785 00:42:18,670 --> 00:42:22,090 It says you're no more difficult than this line. 786 00:42:22,090 --> 00:42:24,880 You're either at this position or to the left. 787 00:42:24,880 --> 00:42:26,680 Being in P was also a positive statement. 788 00:42:26,680 --> 00:42:28,690 It says you're here or to the left. 789 00:42:28,690 --> 00:42:30,970 Being in P is better than being in NP 790 00:42:30,970 --> 00:42:33,940 because this is a subset of that. 791 00:42:33,940 --> 00:42:36,130 NP-hard is a lower bound. 792 00:42:36,130 --> 00:42:40,210 It says, you are at this point, at this level of difficulty, 793 00:42:40,210 --> 00:42:42,730 or to the right. 794 00:42:42,730 --> 00:42:46,540 And so it goes from here off to infinity in difficulty. 795 00:42:46,540 --> 00:42:48,250 And EXP-hard says you're at least 796 00:42:48,250 --> 00:42:53,260 as hard as the right extent of the EXP set, 797 00:42:53,260 --> 00:42:55,720 or you're harder than that, in a sense that we 798 00:42:55,720 --> 00:42:58,660 will formalize in a moment. 799 00:42:58,660 --> 00:43:01,470 And this place right here, as you might imagine, 800 00:43:01,470 --> 00:43:02,670 is kind of interesting. 801 00:43:02,670 --> 00:43:05,610 It's exactly where NP meets NP-hard. 802 00:43:05,610 --> 00:43:09,433 This thing is called NP-complete. 803 00:43:09,433 --> 00:43:11,350 You probably have heard about NP-completeness, 804 00:43:11,350 --> 00:43:12,690 a famous notion. 805 00:43:12,690 --> 00:43:14,610 And this is what it means. 806 00:43:14,610 --> 00:43:17,840 It is, the problems that are in NP-- 807 00:43:17,840 --> 00:43:19,840 so they have a lucky algorithm that solves them, 808 00:43:19,840 --> 00:43:21,673 they can be verified, there are certificates 809 00:43:21,673 --> 00:43:25,170 that can be verified-- and they are NP-hard. 810 00:43:25,170 --> 00:43:28,080 So they're in NP, and they are the hardest among problems 811 00:43:28,080 --> 00:43:29,310 in NP. 812 00:43:29,310 --> 00:43:31,200 Now, they're not the hardest problem. 813 00:43:31,200 --> 00:43:33,030 There are actually many problems right here 814 00:43:33,030 --> 00:43:36,750 at this single level of difficulty called NP-complete. 815 00:43:36,750 --> 00:43:39,130 Among them is Tetris. 816 00:43:39,130 --> 00:43:43,120 There are many others, which I will list in a moment. 817 00:43:43,120 --> 00:43:45,790 So that is NP-completeness. 818 00:43:45,790 --> 00:43:52,810 So because these problems are the hardest problems in NP, 819 00:43:52,810 --> 00:43:55,090 if there's any problems here in between-- 820 00:43:55,090 --> 00:43:59,550 in NP minus P, then these must be among them. 821 00:43:59,550 --> 00:44:01,880 And so if you assume that P does not 822 00:44:01,880 --> 00:44:03,830 equal NP, as most people do, then 823 00:44:03,830 --> 00:44:06,500 you know that all problems at this right-most extreme 824 00:44:06,500 --> 00:44:08,660 of NP, the hardest of the problems in NP, 825 00:44:08,660 --> 00:44:10,580 they must not be NP. 826 00:44:10,580 --> 00:44:12,980 And that's why I can say, if P does not equal NP, 827 00:44:12,980 --> 00:44:17,700 Tetris is not NP, and also, any NP complete problem is not NP. 828 00:44:17,700 --> 00:44:21,300 OK, what does "as hard as" mean? 829 00:44:26,110 --> 00:44:30,430 This is our good friend reductions. 830 00:44:33,010 --> 00:44:35,095 We talked about reductions a lot in this class. 831 00:44:38,260 --> 00:44:40,600 Reductions are the easy way to use algorithms. 832 00:44:40,600 --> 00:44:43,600 You just take your problem and reduce it to a problem 833 00:44:43,600 --> 00:44:45,490 you already know how to solve. 834 00:44:45,490 --> 00:44:52,150 You take the input to some problem that you want to solve, 835 00:44:52,150 --> 00:44:56,530 and you convert it into an input to some other problem, 836 00:44:56,530 --> 00:44:58,960 like single source shortest paths or something like that 837 00:44:58,960 --> 00:45:01,810 that you already have an algorithm for solving. 838 00:45:01,810 --> 00:45:05,050 So if you have an algorithm that solves problem B, 839 00:45:05,050 --> 00:45:09,760 you can convert that into a solution for B. 840 00:45:09,760 --> 00:45:11,980 And a reduction should also tell me how to-- 841 00:45:11,980 --> 00:45:14,350 given a solution to B, how to convert it back 842 00:45:14,350 --> 00:45:18,610 into a solution for A. And when I say solution here, 843 00:45:18,610 --> 00:45:21,370 I actually mean certificate from over there. 844 00:45:21,370 --> 00:45:28,280 So how-- so if I-- 845 00:45:28,280 --> 00:45:32,360 so if I have-- so reduction consists of these two pieces-- 846 00:45:32,360 --> 00:45:35,045 how to convert an input at A to an input for B, 847 00:45:35,045 --> 00:45:36,980 and given a solution to B, how to convert it 848 00:45:36,980 --> 00:45:37,853 to a solution to A. 849 00:45:37,853 --> 00:45:40,520 Let me give you some examples of reductions you've already seen. 850 00:45:40,520 --> 00:45:42,620 You've seen a lot of them. 851 00:45:42,620 --> 00:45:47,180 If I have unweighted shortest paths on the left-- 852 00:45:47,180 --> 00:45:52,580 unweighted single source shortest paths-- 853 00:45:52,580 --> 00:45:55,400 I can reduce that to weighted shortest paths. 854 00:45:59,990 --> 00:46:01,262 How? 855 00:46:01,262 --> 00:46:02,970 AUDIENCE: Set all the weights to 1. 856 00:46:02,970 --> 00:46:04,080 ERIK DEMAINE: Set all the weights to 1. 857 00:46:04,080 --> 00:46:06,060 So here I'm given a graph without weights. 858 00:46:06,060 --> 00:46:07,800 If I set all the weights to 1, that 859 00:46:07,800 --> 00:46:10,330 turns it into an input for a weighted single source shortest 860 00:46:10,330 --> 00:46:10,830 paths. 861 00:46:10,830 --> 00:46:12,497 So if you didn't know how to solve this, 862 00:46:12,497 --> 00:46:14,170 you could solve it by converting it. 863 00:46:14,170 --> 00:46:16,450 If you've already written, say, a Dijkstra algorithm, 864 00:46:16,450 --> 00:46:18,815 you could apply it to solve unweighted single source 865 00:46:18,815 --> 00:46:19,440 shortest paths. 866 00:46:19,440 --> 00:46:21,240 Now, we know a faster way to solve this, 867 00:46:21,240 --> 00:46:23,340 but it's only a log factor faster. 868 00:46:23,340 --> 00:46:26,452 And here we're talking about polynomial versus exponential. 869 00:46:26,452 --> 00:46:27,660 So this is a valid reduction. 870 00:46:27,660 --> 00:46:29,035 It's not the most interesting one 871 00:46:29,035 --> 00:46:32,280 from an algorithmic standpoint, but it is an algorithm. 872 00:46:32,280 --> 00:46:35,640 Another one we've seen is, if you 873 00:46:35,640 --> 00:46:37,470 have integer weights in the left, 874 00:46:37,470 --> 00:46:39,630 you can convert that to unweighted 875 00:46:39,630 --> 00:46:42,780 on the right, positive integer weights, 876 00:46:42,780 --> 00:46:45,510 by subdividing each edge of weight W 877 00:46:45,510 --> 00:46:50,040 into W edges of no weight. 878 00:46:50,040 --> 00:46:52,230 So that's maybe a little bit less efficient. 879 00:46:52,230 --> 00:46:55,200 It depends what the sum of the weights are. 880 00:46:55,200 --> 00:46:59,950 Another version that we've seen is longest path in a graph. 881 00:46:59,950 --> 00:47:04,470 We can-- weighted path we can reduce to shortest path 882 00:47:04,470 --> 00:47:10,290 in a graph, weighted by negating all the weights. 883 00:47:10,290 --> 00:47:12,540 We did this in some of the dynamic programming things. 884 00:47:12,540 --> 00:47:14,362 Like oh, longest path on a DAG? 885 00:47:14,362 --> 00:47:16,320 We can convert that into shortest path on a DAG 886 00:47:16,320 --> 00:47:18,120 just by negating all the weights. 887 00:47:18,120 --> 00:47:20,700 So these are all examples of converting one problem 888 00:47:20,700 --> 00:47:21,630 to another. 889 00:47:21,630 --> 00:47:23,400 Usually, you convert from-- 890 00:47:23,400 --> 00:47:25,440 for algorithms, you convert from a problem 891 00:47:25,440 --> 00:47:28,200 you want to solve into a problem that you already 892 00:47:28,200 --> 00:47:30,070 know how to solve. 893 00:47:30,070 --> 00:47:32,610 But it turns out the same tool reductions can be used 894 00:47:32,610 --> 00:47:35,510 to prove negative results too. 895 00:47:35,510 --> 00:47:38,640 And in this case, we're going to reduce from a problem that we 896 00:47:38,640 --> 00:47:42,750 think cannot be solved and reduce it to the problem that 897 00:47:42,750 --> 00:47:45,330 we're interested in solving. 898 00:47:45,330 --> 00:47:48,390 So let me write more precisely what this means. 899 00:47:48,390 --> 00:47:50,710 If you can find a reduction like this, 900 00:47:50,710 --> 00:47:53,880 it means that solving A is at least 901 00:47:53,880 --> 00:48:04,300 as easy as solving B. Because I could solve A, in particular, 902 00:48:04,300 --> 00:48:08,080 by converting it into B, solving B, and then converting it back 903 00:48:08,080 --> 00:48:11,380 to a solution to A. So in other words, if I can solve B, 904 00:48:11,380 --> 00:48:14,065 I can solve A, which I can phrase informally as, 905 00:48:14,065 --> 00:48:16,270 A is at least as easy as B. 906 00:48:16,270 --> 00:48:19,660 And now using grammar, contrapositive 907 00:48:19,660 --> 00:48:23,770 whatever, this is the same thing as saying that B is at least as 908 00:48:23,770 --> 00:48:34,430 hard as A. And this is what I mean by at least as hard as. 909 00:48:34,430 --> 00:48:37,120 So this is my definition of at least as hard, 910 00:48:37,120 --> 00:48:40,810 in this notion of NP-hardness. 911 00:48:40,810 --> 00:48:44,020 So what NP-hard means is that I'm at least as 912 00:48:44,020 --> 00:48:46,000 hard as all problems in NP. 913 00:48:46,000 --> 00:48:49,420 So what that means is, every problem in NP 914 00:48:49,420 --> 00:48:53,845 can be reduced to Tetris, which is kind of funny. 915 00:48:53,845 --> 00:48:55,720 But in particular, that means that if there's 916 00:48:55,720 --> 00:48:57,595 an algorithm for Tetris, there's an algorithm 917 00:48:57,595 --> 00:48:59,950 for all problems in NP. 918 00:48:59,950 --> 00:49:01,720 And so that's actually the contrapositive 919 00:49:01,720 --> 00:49:02,350 of this statement. 920 00:49:02,350 --> 00:49:04,160 So this is saying, if there's a polynomial-- 921 00:49:04,160 --> 00:49:06,368 if I take the contrapositive of this, this is saying, 922 00:49:06,368 --> 00:49:08,590 if there's a polynomial time algorithm for Tetris, 923 00:49:08,590 --> 00:49:10,960 then P equals NP, there's a polynomial time algorithm 924 00:49:10,960 --> 00:49:12,670 for every problem in NP. 925 00:49:12,670 --> 00:49:14,830 And the way we prove that is by reductions. 926 00:49:14,830 --> 00:49:19,000 We take an arbitrary problem in NP, and we reduce it to Tetris. 927 00:49:19,000 --> 00:49:22,090 Luckily, that's not as hard as it sounds because it's already 928 00:49:22,090 --> 00:49:22,900 been done once. 929 00:49:22,900 --> 00:49:27,250 There is already a reduction from NP to-- 930 00:49:27,250 --> 00:49:31,570 from all problems in NP to singular problems out there, 931 00:49:31,570 --> 00:49:32,710 the NP-complete problems. 932 00:49:32,710 --> 00:49:35,110 There is some first NP-complete problem, which 933 00:49:35,110 --> 00:49:38,240 I guess is the Turing machine. 934 00:49:38,240 --> 00:49:40,250 It's basically simulating a lucky algorithm, 935 00:49:40,250 --> 00:49:42,558 so it's kind of a not very interesting problem. 936 00:49:42,558 --> 00:49:44,350 But from that problem, if you can reduce it 937 00:49:44,350 --> 00:49:46,142 to any other problem, you know that problem 938 00:49:46,142 --> 00:49:48,050 is NP-hard as well. 939 00:49:48,050 --> 00:49:56,250 And so briefly, I want to show you some examples of that here. 940 00:49:56,250 --> 00:49:59,090 So I want to start out with a problem 941 00:49:59,090 --> 00:50:02,510 that I'm just going to assume is NP-complete. 942 00:50:02,510 --> 00:50:05,510 And it's called 3-partition. 943 00:50:05,510 --> 00:50:08,600 One way to phrase it is, I give you a bunch of integers-- 944 00:50:08,600 --> 00:50:14,360 I think I have it written down over here, also the board. 945 00:50:14,360 --> 00:50:17,870 I give you n integers, and I'd like to divide them up 946 00:50:17,870 --> 00:50:22,310 into n over 3 groups of size 3, such that each group of size 3 947 00:50:22,310 --> 00:50:24,740 has the same sum. 948 00:50:24,740 --> 00:50:27,668 And it's written there on the board. 949 00:50:27,668 --> 00:50:29,960 So you can also think of this as the following problem. 950 00:50:29,960 --> 00:50:33,560 I give you a bunch of rectangles that are a side length-- 951 00:50:33,560 --> 00:50:36,920 or a bunch of sticks, let's say, of varying lengths, 952 00:50:36,920 --> 00:50:41,010 and I want to group them up like on the right diagram, 953 00:50:41,010 --> 00:50:44,720 so in groups of 3, such that the total length of each group 954 00:50:44,720 --> 00:50:46,970 is exactly the same. 955 00:50:46,970 --> 00:50:48,530 This is just a problem. 956 00:50:48,530 --> 00:50:51,350 And just believe for now that it is NP-complete. 957 00:50:51,350 --> 00:50:52,490 I won't prove that. 958 00:50:52,490 --> 00:50:57,170 But what I'd like to show you is a reduction from this problem 959 00:50:57,170 --> 00:50:59,000 to another problem-- 960 00:50:59,000 --> 00:51:01,520 solving jigsaw puzzles. 961 00:51:01,520 --> 00:51:03,110 So you might think jigsaw puzzles 962 00:51:03,110 --> 00:51:09,590 are really easy, and especially easy if I lose the projector. 963 00:51:09,590 --> 00:51:11,480 But in fact, if you have a jigsaw puzzle 964 00:51:11,480 --> 00:51:14,540 where some of the matches are ambiguous, 965 00:51:14,540 --> 00:51:17,810 if there's multiple pieces that could fit against a given tab 966 00:51:17,810 --> 00:51:22,250 or pocket, then I claim I can represent 967 00:51:22,250 --> 00:51:26,030 this 3-partition problem by building little sticks, 968 00:51:26,030 --> 00:51:27,360 like here. 969 00:51:27,360 --> 00:51:30,650 So if I want to represent a stick of length ai, 970 00:51:30,650 --> 00:51:32,090 I'm just going to build an ai-- 971 00:51:32,090 --> 00:51:33,920 I didn't mention they're all integers, 972 00:51:33,920 --> 00:51:38,110 and they're polynomial-sized integers. 973 00:51:38,110 --> 00:51:43,360 I'm going to represent that by ai different pieces here. 974 00:51:43,360 --> 00:51:47,140 And the red tabs and pockets are designed 975 00:51:47,140 --> 00:51:48,850 to be unique global to the puzzle, 976 00:51:48,850 --> 00:51:50,680 like a regular jigsaw puzzle. 977 00:51:50,680 --> 00:51:56,320 Given this piece on the left and this tab on the right, 978 00:51:56,320 --> 00:51:59,320 there's a unique pocket, there's a piece with unique pocket 979 00:51:59,320 --> 00:52:01,250 that fits perfectly into that piece. 980 00:52:01,250 --> 00:52:03,760 So this joining is forced. 981 00:52:03,760 --> 00:52:05,470 And also this joining is forced. 982 00:52:05,470 --> 00:52:07,510 But the blue tabs and pockets are different. 983 00:52:07,510 --> 00:52:09,610 They're all the same. 984 00:52:09,610 --> 00:52:11,030 They're all identical. 985 00:52:11,030 --> 00:52:14,230 And so if I build this frame using 986 00:52:14,230 --> 00:52:20,110 the red unique assignments, and I build these rectangles, 987 00:52:20,110 --> 00:52:23,140 if I want to pack these rectangles into this rectangle, 988 00:52:23,140 --> 00:52:26,110 that's exactly the 3-partition problem, with some details 989 00:52:26,110 --> 00:52:27,400 that I didn't fill in. 990 00:52:27,400 --> 00:52:30,820 But it turns out you'd be forced to group these 991 00:52:30,820 --> 00:52:35,461 into groups of size 3, something like this, 992 00:52:35,461 --> 00:52:39,110 with varying lengths. 993 00:52:39,110 --> 00:52:40,860 OK, so that's an example of a reduction. 994 00:52:40,860 --> 00:52:42,680 If you believe the 3-partition is NP-hard, 995 00:52:42,680 --> 00:52:45,770 this proves to you that jigsaw puzzles are NP-hard, 996 00:52:45,770 --> 00:52:47,568 something you may not have known. 997 00:52:47,568 --> 00:52:49,110 Every time you solve a jigsaw puzzle, 998 00:52:49,110 --> 00:52:50,652 you can feel good about yourself now, 999 00:52:50,652 --> 00:52:52,950 especially if it has ambiguous mates. 1000 00:52:52,950 --> 00:52:54,890 Next is Tetris. 1001 00:52:54,890 --> 00:52:58,940 So here is a reduction from the same 3-partition problem, 1002 00:52:58,940 --> 00:53:01,220 which is one of my favorite problems, to Tetris. 1003 00:53:01,220 --> 00:53:03,740 It starts out with this strange board. 1004 00:53:03,740 --> 00:53:10,230 It has a bunch of columns here where I could put pieces. 1005 00:53:10,230 --> 00:53:13,310 So I'm not allowed to put pieces in these dark regions. 1006 00:53:13,310 --> 00:53:16,670 They all have height T. T is the target sum 1007 00:53:16,670 --> 00:53:18,770 that we want all of the numbers to-- 1008 00:53:18,770 --> 00:53:21,200 all of the triples of numbers to add up to. 1009 00:53:21,200 --> 00:53:24,260 And there's n over 3 of these slots 1010 00:53:24,260 --> 00:53:26,730 where I can try to put pieces. 1011 00:53:26,730 --> 00:53:29,470 And it's-- because of this thing over on the right, 1012 00:53:29,470 --> 00:53:32,940 there's no way to clear lines in this game. 1013 00:53:32,940 --> 00:53:35,820 And now to represent a single number ai, 1014 00:53:35,820 --> 00:53:38,250 I'm going to give you this sequence of pieces, 1015 00:53:38,250 --> 00:53:40,380 which starts with an L piece. 1016 00:53:40,380 --> 00:53:43,410 And then it has ai repetitions of this pattern, 1017 00:53:43,410 --> 00:53:45,780 and then it ends with these two pieces. 1018 00:53:45,780 --> 00:53:49,500 And so what ends up happening is that-- 1019 00:53:49,500 --> 00:53:51,300 this is in the intended solution-- 1020 00:53:51,300 --> 00:53:56,220 you first place an L at the bottom of one of these buckets, 1021 00:53:56,220 --> 00:53:58,950 and then you repeat this pattern in this nice way. 1022 00:53:58,950 --> 00:54:03,990 And it fills up the ai, roughly, height of this bucket. 1023 00:54:03,990 --> 00:54:08,260 And then at the end, you have to put the I here. 1024 00:54:08,260 --> 00:54:10,750 And what this ends up guaranteeing 1025 00:54:10,750 --> 00:54:14,290 is that all of these pieces go into a single bucket. 1026 00:54:14,290 --> 00:54:14,950 You can check. 1027 00:54:14,950 --> 00:54:15,910 It's tedious. 1028 00:54:15,910 --> 00:54:18,700 But if you tried to put some of these pieces 1029 00:54:18,700 --> 00:54:20,980 in one bucket and other pieces in a different bucket, 1030 00:54:20,980 --> 00:54:23,862 you would lose some space, and then you would die. 1031 00:54:23,862 --> 00:54:26,320 So if you want to survive, you have to put all these pieces 1032 00:54:26,320 --> 00:54:26,995 into one bucket. 1033 00:54:26,995 --> 00:54:28,870 And so again, we're just stacking rectangles. 1034 00:54:28,870 --> 00:54:31,303 We're putting a whole bunch of rectangles in one pocket 1035 00:54:31,303 --> 00:54:33,220 and then a bunch of rectangles another pocket. 1036 00:54:33,220 --> 00:54:35,300 We can switch back and forth however we want. 1037 00:54:35,300 --> 00:54:37,210 But the only way to win, it turns out, 1038 00:54:37,210 --> 00:54:40,060 is if you get all of those rectangles 1039 00:54:40,060 --> 00:54:42,040 to add up to exactly the right height. 1040 00:54:42,040 --> 00:54:43,930 Then you get a picture like this. 1041 00:54:43,930 --> 00:54:45,730 If you don't get a picture like this, 1042 00:54:45,730 --> 00:54:47,710 you can prove you end up dying. 1043 00:54:47,710 --> 00:54:49,360 Then I'll give you a bunch of Ls. 1044 00:54:49,360 --> 00:54:52,300 Then I'll finally give you this T, which clears some lines. 1045 00:54:52,300 --> 00:54:53,327 And then I'll give you-- 1046 00:54:53,327 --> 00:54:54,910 the most satisfying Tetris game ever-- 1047 00:54:54,910 --> 00:54:56,530 I'll give you a ton of I's, and you 1048 00:54:56,530 --> 00:54:59,290 get Tetris, Tetris, Tetris, and you clear the entire board. 1049 00:54:59,290 --> 00:55:01,883 And so if you can solve the 3-partition problem 1050 00:55:01,883 --> 00:55:03,550 you can clear the board and win the game 1051 00:55:03,550 --> 00:55:06,650 and be the best Tetris player ever. 1052 00:55:06,650 --> 00:55:09,340 And if there is no solution to 3-partition, 1053 00:55:09,340 --> 00:55:11,540 you're guaranteed to lose. 1054 00:55:11,540 --> 00:55:14,384 And so this proves Tetris is NP-hard. 1055 00:55:14,384 --> 00:55:16,120 Cool. 1056 00:55:16,120 --> 00:55:20,040 So what else do I want to say, briefly? 1057 00:55:20,040 --> 00:55:24,230 I think that's the main idea. 1058 00:55:24,230 --> 00:55:27,490 So another example-- so this spot 1059 00:55:27,490 --> 00:55:29,380 is called EXP-completeness. 1060 00:55:35,440 --> 00:55:38,800 And this includes problems such as n by n chess. 1061 00:55:38,800 --> 00:55:42,023 So we know that chess requires exponential time 1062 00:55:42,023 --> 00:55:43,690 because, in fact, it's among the hardest 1063 00:55:43,690 --> 00:55:45,340 problems in exponential time. 1064 00:55:45,340 --> 00:55:47,605 But most common are the-- 1065 00:55:47,605 --> 00:55:51,390 that's somehow because of the two-player nature of the game. 1066 00:55:51,390 --> 00:55:53,610 Most common are NP-complete problems. 1067 00:55:53,610 --> 00:55:56,760 And we have a bunch of example NP-complete problems 1068 00:55:56,760 --> 00:55:59,170 I'll just briefly mention here. 1069 00:55:59,170 --> 00:56:01,920 So we saw the subset sum problem, 1070 00:56:01,920 --> 00:56:04,690 which we had a polynomial time algorithm for-- 1071 00:56:04,690 --> 00:56:07,600 sorry, a pseudo polynomial time algorithm for last class-- 1072 00:56:07,600 --> 00:56:09,660 in fact has no polynomial time algorithm, 1073 00:56:09,660 --> 00:56:11,250 assuming P equals NP. 1074 00:56:11,250 --> 00:56:15,000 So pseudo poly is the best you can hope for for subset sum. 1075 00:56:15,000 --> 00:56:17,310 There's a related notion called weakly NP-hardness, 1076 00:56:17,310 --> 00:56:19,140 which I won't get into here. 1077 00:56:19,140 --> 00:56:20,460 3-partition is one we saw. 1078 00:56:20,460 --> 00:56:23,080 We saw some reductions to other problems. 1079 00:56:23,080 --> 00:56:24,930 So these are all NP complete. 1080 00:56:24,930 --> 00:56:27,450 Longest common subsequence is another dynamic programming 1081 00:56:27,450 --> 00:56:28,933 problem we saw with two sequences. 1082 00:56:28,933 --> 00:56:30,600 I mentioned you could solve it for three 1083 00:56:30,600 --> 00:56:32,320 or four or any constant number. 1084 00:56:32,320 --> 00:56:34,620 But if I give you n sequences each of length n, 1085 00:56:34,620 --> 00:56:37,830 that problem is NP-hard, NP-complete. 1086 00:56:37,830 --> 00:56:39,958 Longest simple path in a graph-- we 1087 00:56:39,958 --> 00:56:41,250 know how to solve longest path. 1088 00:56:41,250 --> 00:56:43,830 You just solve shortest path and negative weights. 1089 00:56:43,830 --> 00:56:46,500 But longest simple path, where you don't repeat vertices, 1090 00:56:46,500 --> 00:56:47,820 that's NP-complete. 1091 00:56:47,820 --> 00:56:51,270 Relatedly, one of the most famous NP-complete problems 1092 00:56:51,270 --> 00:56:53,640 is traveling salesman problem, finding 1093 00:56:53,640 --> 00:56:58,330 the shortest path that visits all vertices in a given graph. 1094 00:56:58,330 --> 00:57:00,090 So instead of just going from A to B, 1095 00:57:00,090 --> 00:57:03,420 I want to visit all the vertices in the graph. 1096 00:57:03,420 --> 00:57:05,010 A lot of these problems I'm phrasing 1097 00:57:05,010 --> 00:57:07,140 as optimization problems. 1098 00:57:07,140 --> 00:57:08,815 But when I say NP-complete, I actually 1099 00:57:08,815 --> 00:57:10,440 mean a decision version of the problem. 1100 00:57:10,440 --> 00:57:12,690 For example, with this one, the decision question 1101 00:57:12,690 --> 00:57:15,450 is, is the shortest path that visits all vertices in a graph 1102 00:57:15,450 --> 00:57:17,940 less than or equal to a given value x. 1103 00:57:17,940 --> 00:57:19,830 If you can solve this, then by binary search 1104 00:57:19,830 --> 00:57:22,500 you can solve the overall weight. 1105 00:57:22,500 --> 00:57:25,230 3-coloring a graph is hard, even though 2-coloring a graph 1106 00:57:25,230 --> 00:57:26,130 is polynomial. 1107 00:57:26,130 --> 00:57:27,720 3-coloring is NP-complete. 1108 00:57:27,720 --> 00:57:30,150 Assigning three colors to the vertices 1109 00:57:30,150 --> 00:57:32,460 so that no adjacent vertices have the same color, 1110 00:57:32,460 --> 00:57:34,750 finding the largest clique in a given graph, which 1111 00:57:34,750 --> 00:57:38,040 would be useful for analyzing social networks, whatever. 1112 00:57:38,040 --> 00:57:40,170 This is a fun one for me as a geometer. 1113 00:57:40,170 --> 00:57:43,200 If you're in a three-dimensional world, which I am, 1114 00:57:43,200 --> 00:57:47,850 and I want to find the shortest path from here to there that 1115 00:57:47,850 --> 00:57:49,680 doesn't collide with any obstacles, 1116 00:57:49,680 --> 00:57:53,250 like this desk and all the chairs and so on, in 3D, 1117 00:57:53,250 --> 00:57:53,970 this problem-- 1118 00:57:53,970 --> 00:57:56,400 if you can fly, so if you're a drone flying 1119 00:57:56,400 --> 00:57:58,170 among all these obstacles, you want 1120 00:57:58,170 --> 00:58:02,025 to find the shortest path from A to B, this is NP-complete. 1121 00:58:02,025 --> 00:58:02,970 It's quite surprising. 1122 00:58:02,970 --> 00:58:04,560 In two dimensions, it's polynomial. 1123 00:58:04,560 --> 00:58:06,400 You can reduce it to graph shortest paths. 1124 00:58:06,400 --> 00:58:08,100 But in 3D, it's NP-hard. 1125 00:58:08,100 --> 00:58:10,350 This is a formula problem that comes up a lot. 1126 00:58:10,350 --> 00:58:12,240 Given a Boolean formula with AND, OR, or NOT, 1127 00:58:12,240 --> 00:58:14,940 can you ever make it true, if it has some variables that 1128 00:58:14,940 --> 00:58:15,930 are not assigned? 1129 00:58:15,930 --> 00:58:18,810 And some more fun examples are Minesweeper or Sudoku. 1130 00:58:18,810 --> 00:58:21,443 Basically any paper and pencil puzzle you've ever played, 1131 00:58:21,443 --> 00:58:22,860 there's probably a paper out there 1132 00:58:22,860 --> 00:58:24,390 proving that it's NP-complete. 1133 00:58:24,390 --> 00:58:27,240 And on the video game side, Super Mario Brothers 1134 00:58:27,240 --> 00:58:28,180 is NP-hard. 1135 00:58:28,180 --> 00:58:29,460 Legend of Zelda is NP-hard. 1136 00:58:29,460 --> 00:58:30,780 Pokemon is NP-hard. 1137 00:58:30,780 --> 00:58:33,960 These problems are actually all a little bit harder than NP, 1138 00:58:33,960 --> 00:58:37,540 in a different class called P-space, which I won't go into. 1139 00:58:37,540 --> 00:58:39,630 But if you're interested in this stuff, 1140 00:58:39,630 --> 00:58:42,690 there is a whole class devoted to it, which has online video 1141 00:58:42,690 --> 00:58:45,900 lectures, so you can watch them whenever you want, 1142 00:58:45,900 --> 00:58:50,640 called 6.892, that gives a bunch of especially fun 1143 00:58:50,640 --> 00:58:53,850 examples of NP-hardness and other types of hardness 1144 00:58:53,850 --> 00:58:56,880 proofs from a sort of algorithm perspective for lots 1145 00:58:56,880 --> 00:59:01,200 of games and puzzles you might care about. 1146 00:59:01,200 --> 00:59:03,280 And that's it.