WEBVTT 00:00:14.000 --> 00:00:18.000 Good morning everyone. Today we are going to do some 00:00:18.000 --> 00:00:23.000 algorithms, back to algorithms, and we are going to use a lot 00:00:23.000 --> 00:00:27.000 of the, well, some of the simpler mathematics 00:00:27.000 --> 00:00:31.000 that we developed last class like the master theorem for 00:00:31.000 --> 00:00:35.000 solving recurrences. We are going to use this a lot. 00:00:35.000 --> 00:00:39.000 Because we are going to talk about recursive algorithms 00:00:39.000 --> 00:00:41.000 today. And so we will find their 00:00:41.000 --> 00:00:43.000 running time using the master theorem. 00:00:43.000 --> 00:00:46.000 This is just the same as it was last time, I hope, 00:00:46.000 --> 00:00:49.000 unless I made a mistake. A couple of reminders. 00:00:49.000 --> 00:00:52.000 You should all go to recitation on Friday. 00:00:52.000 --> 00:00:54.000 That is required. If you want to, 00:00:54.000 --> 00:00:56.000 you can go to homework lab on Sunday. 00:00:56.000 --> 00:00:59.000 That may be a good excuse for you to actually work on your 00:00:59.000 --> 00:01:04.000 problem set a few hours early. Well, actually, 00:01:04.000 --> 00:01:07.000 it's due on Wednesday so you have lots of time. 00:01:07.000 --> 00:01:10.000 And there is no class on Monday. 00:01:10.000 --> 00:01:13.000 It is the holiday known as Student Holiday, 00:01:13.000 --> 00:01:16.000 so don't come. Today we are going to cover 00:01:16.000 --> 00:01:20.000 something called Divide and Conquer. 00:01:27.000 --> 00:01:31.000 Also known as divide and rule or divide et impera for those of 00:01:31.000 --> 00:01:34.000 you who know Latin, which is a tried and tested way 00:01:34.000 --> 00:01:39.000 of conquering a land by dividing it into sections of some kind. 00:01:39.000 --> 00:01:43.000 It could be different political factions, different whatever. 00:01:43.000 --> 00:01:47.000 And then somehow making them no longer like each other. 00:01:47.000 --> 00:01:50.000 Like starting a family feud is always a good method. 00:01:50.000 --> 00:01:53.000 You should remember this on your quiz. 00:01:53.000 --> 00:01:56.000 I'm kidding. And if you can separate this 00:01:56.000 --> 00:02:00.000 big power structure into little power structures such that you 00:02:00.000 --> 00:02:03.000 dominate each little power structure then you can conquer 00:02:03.000 --> 00:02:06.000 all of them individually, as long as you make sure they 00:02:06.000 --> 00:02:09.000 don't get back together. That is divide and conquer as 00:02:09.000 --> 00:02:11.000 practiced, say, by the British. 00:02:11.000 --> 00:02:14.000 But today we are going to do divide and conquer as practiced 00:02:14.000 --> 00:02:17.000 in Cormen, Leiserson, Rivest and Stein or every other 00:02:17.000 --> 00:02:20.000 algorithm textbook. This is a very basic and very 00:02:20.000 --> 00:02:22.000 powerful algorithm design technique. 00:02:22.000 --> 00:02:27.000 So, this is our first real algorithm design experience. 00:02:27.000 --> 00:02:30.000 We are still sort of mostly in the analysis mode, 00:02:30.000 --> 00:02:33.000 but we are going to do some actual design. 00:02:33.000 --> 00:02:37.000 We're going to cover maybe only three or four major design 00:02:37.000 --> 00:02:39.000 techniques. This is one of them, 00:02:39.000 --> 00:02:43.000 so it is really important. And it will lead to all sorts 00:02:43.000 --> 00:02:45.000 of recurrences, so we will get to use 00:02:45.000 --> 00:02:49.000 everything from last class and see why it is useful. 00:02:49.000 --> 00:02:51.000 As you might expect, the first step in 00:02:51.000 --> 00:02:57.000 divide-and-conquer is divide and the second step is conquer. 00:02:57.000 --> 00:03:01.000 But you may not have guessed that there are three steps. 00:03:01.000 --> 00:03:04.000 And I am leaving some blank space here, so you should, 00:03:04.000 --> 00:03:06.000 too. Divide-and-conquer is an 00:03:06.000 --> 00:03:10.000 algorithmic technique. You are given some big problem 00:03:10.000 --> 00:03:14.000 you want to solve, you don't really know how to 00:03:14.000 --> 00:03:18.000 solve it in an efficient way, so you are going to split it up 00:03:18.000 --> 00:03:22.000 into subproblems. That is the divide. 00:03:22.000 --> 00:03:26.000 You are going to divide that problem, or more precisely the 00:03:26.000 --> 00:03:30.000 instance of that problem, the particular instance of that 00:03:30.000 --> 00:03:32.000 problem you have into subproblems. 00:03:32.000 --> 00:03:36.000 And those subproblems should be smaller in some sense. 00:03:36.000 --> 00:03:41.000 And smaller means normally that the value of N is smaller than 00:03:41.000 --> 00:03:45.000 it was in the original problem. So, you sort of made some 00:03:45.000 --> 00:03:47.000 progress. Now you have one, 00:03:47.000 --> 00:03:51.000 or more likely you have several subproblems you need to solve. 00:03:51.000 --> 00:03:55.000 Each of them is smaller. So, you recursively solve each 00:03:55.000 --> 00:04:00.000 subproblem. That is the conquer step. 00:04:00.000 --> 00:04:04.000 You conquer each subproblem recursively. 00:04:04.000 --> 00:04:11.000 And then somehow you combine those solutions into a solution 00:04:11.000 --> 00:04:14.000 for the whole problem. 00:04:23.000 --> 00:04:26.000 So, this is the general divide-and-conquer paradigm. 00:04:26.000 --> 00:04:29.000 And lots of algorithms fit it. You have already seen one 00:04:29.000 --> 00:04:33.000 algorithm that fits this paradigm, if you can remember. 00:04:33.000 --> 00:04:36.000 Merge sort, good. Wow, you are all awake. 00:04:36.000 --> 00:04:39.000 I'm impressed. So, we saw merge sort. 00:04:39.000 --> 00:04:42.000 And, if I am clever, I could fit it in this space. 00:04:42.000 --> 00:04:44.000 Sure. Let's be clever. 00:04:44.000 --> 00:04:48.000 A quick review on merge sort. Phrased in this 1, 00:04:48.000 --> 00:04:52.000 2, 3 kind of method. The first step was to divide 00:04:52.000 --> 00:04:56.000 your array into two halves. This really doesn't mean 00:04:56.000 --> 00:04:59.000 anything because you just sort of think, oh, 00:04:59.000 --> 00:05:05.000 I will pretend my array is divided into two halves. 00:05:05.000 --> 00:05:07.000 There is no work here. This is zero time. 00:05:07.000 --> 00:05:11.000 You just look at your array. Here is your array. 00:05:11.000 --> 00:05:14.000 I guess maybe you compute n/2 and take the floor. 00:05:14.000 --> 00:05:17.000 That takes constant time. And you say OK, 00:05:17.000 --> 00:05:21.000 I am pretending my array is now divided into the left part and 00:05:21.000 --> 00:05:24.000 the right part. And then the interesting part 00:05:24.000 --> 00:05:27.000 is that you recursively solve each one. 00:05:27.000 --> 00:05:32.000 That's the conquer. We recursively sort each 00:05:32.000 --> 00:05:37.000 subarray. And then the third step is to 00:05:37.000 --> 00:05:44.000 combine those solutions. And so here we really see what 00:05:44.000 --> 00:05:49.000 this means. We now have a sorted version of 00:05:49.000 --> 00:05:55.000 this array by induction. We have a sorted version of 00:05:55.000 --> 00:06:02.000 this array by induction. We now want the sorted version 00:06:02.000 --> 00:06:07.000 of the whole array. And we saw that was the merge 00:06:07.000 --> 00:06:10.000 problem, merging two sorted arrays. 00:06:10.000 --> 00:06:15.000 And that we saw how to do in linear time, order n time. 00:06:15.000 --> 00:06:20.000 I am not going to repeat that, but the point is it falls into 00:06:20.000 --> 00:06:24.000 that framework. I want to write the running 00:06:24.000 --> 00:06:29.000 time and merge sort as a recurrence. 00:06:29.000 --> 00:06:33.000 You have already seen the recurrence, you have already 00:06:33.000 --> 00:06:37.000 been told the solution, but now we actually know how to 00:06:37.000 --> 00:06:39.000 solve it. And, furthermore, 00:06:39.000 --> 00:06:43.000 every algorithm that follows the divide-and-conquer paradigm 00:06:43.000 --> 00:06:47.000 will have a recurrence of pretty much the same form, 00:06:47.000 --> 00:06:51.000 very much like our good friend the master method. 00:06:51.000 --> 00:06:56.000 Let's do it for merge sort where we sort of already know 00:06:56.000 --> 00:07:00.000 the answer and get a bit of practice. 00:07:00.000 --> 00:07:02.000 This is the merge sort recurrence. 00:07:02.000 --> 00:07:06.000 You should know and love this recurrence because it comes up 00:07:06.000 --> 00:07:09.000 all over the place. It comes from this general 00:07:09.000 --> 00:07:13.000 approach by just seeing what are the sizes of the subproblems you 00:07:13.000 --> 00:07:17.000 are solving and how many there are and how much extra work you 00:07:17.000 --> 00:07:20.000 are doing. You have here the size of the 00:07:20.000 --> 00:07:22.000 subproblems. It happens here that both 00:07:22.000 --> 00:07:25.000 subproblems have the same size roughly. 00:07:25.000 --> 00:07:29.000 There is this sloppiness that we have, which really should be 00:07:29.000 --> 00:07:34.000 T of floor of n over 2 plus T of ceiling of n over 2. 00:07:34.000 --> 00:07:37.000 And when you go to recitation on Friday you will see why that 00:07:37.000 --> 00:07:40.000 is OK, the floors and ceilings don't matter. 00:07:40.000 --> 00:07:43.000 There is a theorem you can prove that that's happy. 00:07:43.000 --> 00:07:47.000 You can assume that N is a power of 2, but we are just 00:07:47.000 --> 00:07:51.000 going to assume that for now. We just have two problems with 00:07:51.000 --> 00:07:53.000 size n over 2. This 2 is the number of 00:07:53.000 --> 00:07:55.000 subproblems. 00:08:00.000 --> 00:08:03.000 And this order n is all the extra work we are doing. 00:08:03.000 --> 00:08:07.000 Now, what is the extra work potentially? 00:08:07.000 --> 00:08:10.000 Well, the conquering is always just recursion. 00:08:10.000 --> 00:08:14.000 There is sort of no work there except this lead part. 00:08:14.000 --> 00:08:19.000 The dividing in this case is trivial, but in general it might 00:08:19.000 --> 00:08:23.000 involve some work. And the combining here involves 00:08:23.000 --> 00:08:25.000 linear work. So, this is the 00:08:25.000 --> 00:08:29.000 divide-and-conquer running times. 00:08:29.000 --> 00:08:31.000 So, this is the nonrecursive work. 00:08:31.000 --> 00:08:36.000 And that is generally how you convert a divide-and-conquer 00:08:36.000 --> 00:08:41.000 algorithm into a recurrence. It's really easy and you 00:08:41.000 --> 00:08:44.000 usually get to apply the master method. 00:08:44.000 --> 00:08:47.000 Here we are in Case 2. Very good. 00:08:47.000 --> 00:08:49.000 This is Case 2. And k is zero here. 00:08:49.000 --> 00:08:54.000 And so in the recursion tree, all of the costs are roughly 00:08:54.000 --> 00:08:58.000 the same. They are all n to the log base 00:08:58.000 --> 00:09:02.000 b of a. Here n to the log base 2 of 2 00:09:02.000 --> 00:09:04.000 is just n. So these are equal. 00:09:04.000 --> 00:09:07.000 We get an extra log factor because of the number of levels 00:09:07.000 --> 00:09:11.000 in the recursion tree. Remember the intuition behind 00:09:11.000 --> 00:09:13.000 the master method. This is n log n, 00:09:13.000 --> 00:09:16.000 and that is good. Merge sort is a fast sorting 00:09:16.000 --> 00:09:20.000 algorithm n log n. Insertion sort was n squared. 00:09:20.000 --> 00:09:23.000 In some sense, n log n is the best you can do. 00:09:23.000 --> 00:09:26.000 We will cover that in two lectures from now, 00:09:26.000 --> 00:09:30.000 but just foreshadowing. Today we are going to do 00:09:30.000 --> 00:09:32.000 different divide-and-conquer algorithms. 00:09:32.000 --> 00:09:36.000 Sorting is one problem. There are all sorts of problems 00:09:36.000 --> 00:09:39.000 we might want to solve. It turns out a lot of them you 00:09:39.000 --> 00:09:42.000 can apply divide-and-conquer to. Not every problem. 00:09:42.000 --> 00:09:45.000 Like how to wake up in the morning, it's not so easy to 00:09:45.000 --> 00:09:48.000 solve a divide-and-conquer, although maybe that's a good 00:09:48.000 --> 00:09:51.000 problem set problem. 00:09:59.000 --> 00:10:03.000 The next divide-and-conquer algorithm we are going to look 00:10:03.000 --> 00:10:09.000 at is even simpler than sorting, even simpler than merge sort, 00:10:09.000 --> 00:10:13.000 but it drives home the point of when you have only one 00:10:13.000 --> 00:10:16.000 subproblem. How many people have seen 00:10:16.000 --> 00:10:19.000 binary search before? Anyone hasn't? 00:10:19.000 --> 00:10:22.000 One, OK. I will go very quickly then. 00:10:22.000 --> 00:10:27.000 You have some element X. You want to find X in a sorted 00:10:27.000 --> 00:10:31.000 array. How many people had not seen it 00:10:31.000 --> 00:10:33.000 before they saw it in recitation? 00:10:33.000 --> 00:10:34.000 No one, OK. Good. 00:10:34.000 --> 00:10:38.000 You have seen it in another class, probably 6.001 or 00:10:38.000 --> 00:10:39.000 something. Very good. 00:10:39.000 --> 00:10:42.000 You took the prerequisites. OK. 00:10:42.000 --> 00:10:46.000 I just want to phrase it as a divide-and-conquer because you 00:10:46.000 --> 00:10:50.000 don't normally see it that way. The divide step is you compare 00:10:50.000 --> 00:10:53.000 X with the middle element in your array. 00:10:53.000 --> 00:10:56.000 Then the conquer step. Here is your array. 00:10:56.000 --> 00:11:02.000 Here is the middle element. You compare X with this thing 00:11:02.000 --> 00:11:06.000 if let's say X is smaller than the middle element in your 00:11:06.000 --> 00:11:09.000 array. You know that X is in the left 00:11:09.000 --> 00:11:13.000 half because it is sorted, a nice loop invariant there, 00:11:13.000 --> 00:11:17.000 whatever, but we are just going to think of that as recursively 00:11:17.000 --> 00:11:22.000 I am going to solve the problem of finding X in this subarray. 00:11:22.000 --> 00:11:26.000 We recurse in one subarray, unlike merge sort where we had 00:11:26.000 --> 00:11:31.000 two recursions. And then the combined step we 00:11:31.000 --> 00:11:35.000 don't do anything. I mean if you find X in here 00:11:35.000 --> 00:11:38.000 then you've found X in the whole array. 00:11:38.000 --> 00:11:42.000 There is nothing to bring it back up really. 00:11:42.000 --> 00:11:46.000 So, this is just phrasing binary search in the 00:11:46.000 --> 00:11:51.000 divide-and-conquer paradigm. It is kind of a trivial 00:11:51.000 --> 00:11:56.000 example, but there are lots of circumstances where you only 00:11:56.000 --> 00:12:02.000 need to recurse in one side. And it is important to see how 00:12:02.000 --> 00:12:06.000 much of a difference making one recursion versus making two 00:12:06.000 --> 00:12:09.000 recursions can be. This is the recurrence for 00:12:09.000 --> 00:12:13.000 binary search. We start with a problem size n. 00:12:13.000 --> 00:12:16.000 We reduce it to 1. There is an implicit 1 factor 00:12:16.000 --> 00:12:19.000 here. One subproblem of size n over 2 00:12:19.000 --> 00:12:21.000 roughly. Again, floors and ceilings 00:12:21.000 --> 00:12:24.000 don't matter. Plus a constant which is to 00:12:24.000 --> 00:12:28.000 compare X with the middle element, so it is actually like 00:12:28.000 --> 00:12:32.000 one comparison. This has a solution, 00:12:32.000 --> 00:12:35.000 log n. And you all know the running 00:12:35.000 --> 00:12:38.000 time of binary search, but here it is at solving the 00:12:38.000 --> 00:12:41.000 recurrence. I mean, there are a couple of 00:12:41.000 --> 00:12:44.000 differences here. We don't have the additive 00:12:44.000 --> 00:12:48.000 order n term. If we did, it would be linear, 00:12:48.000 --> 00:12:51.000 the running the time. Still better than n log n. 00:12:51.000 --> 00:12:55.000 So, we are getting rid of the 2, bringing it down to a 1, 00:12:55.000 --> 00:13:00.000 taking the n and bringing it down to a 1. 00:13:00.000 --> 00:13:02.000 That is making the running time a lot faster, 00:13:02.000 --> 00:13:05.000 the whole factor of n faster. No big surprise there. 00:13:05.000 --> 00:13:08.000 Let's do some more interesting algorithms. 00:13:18.000 --> 00:13:24.000 The powering a number problem is I give you a number X. 00:13:24.000 --> 00:13:31.000 I give that as like a real number or floating point number 00:13:31.000 --> 00:13:35.000 or whatever. And I give you an integer n, 00:13:35.000 --> 00:13:40.000 at least zero. I want to compute X to the 00:13:40.000 --> 00:13:44.000 power n. So, it's a very simple problem. 00:13:44.000 --> 00:13:49.000 It is, in some sense, even easier than all of these. 00:13:49.000 --> 00:13:54.000 But here it is. And divide-and-conquer is sort 00:13:54.000 --> 00:14:00.000 of the right thing to do. So, the naÔve algorithm is very 00:14:00.000 --> 00:14:04.000 simple. How do you compute X to the 00:14:04.000 --> 00:14:07.000 power n? Well, the definition of X to 00:14:07.000 --> 00:14:11.000 the power n is I take X and I multiply by X n times. 00:14:11.000 --> 00:14:16.000 So, I take X times X times X times X where there are n copies 00:14:16.000 --> 00:14:19.000 of X totally. And that's X to the n. 00:14:19.000 --> 00:14:22.000 No big surprise. That is n multiplications, 00:14:22.000 --> 00:14:27.000 or n minus 1 multiplications, theta n time. 00:14:34.000 --> 00:14:39.000 But that's not the best you can do for this problem. 00:14:39.000 --> 00:14:44.000 Any suggestions on what we might do using 00:14:44.000 --> 00:14:49.000 divide-and-conquer? Has anyone seen this algorithm 00:14:49.000 --> 00:14:51.000 before? A few, OK. 00:14:51.000 --> 00:14:56.000 For the rest? Testing on the spot creativity, 00:14:56.000 --> 00:15:04.000 which is very difficult, but I always like a challenge. 00:15:04.000 --> 00:15:08.000 I mean random ideas. What could we possibly do to 00:15:08.000 --> 00:15:12.000 solve this problem in less than linear time? 00:15:12.000 --> 00:15:16.000 How is this sort of a divide-and-conquer problem? 00:15:16.000 --> 00:15:19.000 We have to inputs, X and n. 00:15:19.000 --> 00:15:22.000 Yeah? We could try to divide on X. 00:15:22.000 --> 00:15:25.000 It seems a bit hard. It is just some number. 00:15:25.000 --> 00:15:30.000 Or, we could try to divide on n. 00:15:30.000 --> 00:15:32.000 Any guesses? Look at X to the n over 2, 00:15:32.000 --> 00:15:35.000 very good. That is exactly the idea of the 00:15:35.000 --> 00:15:38.000 divide-and-conquer algorithm. 00:15:48.000 --> 00:15:50.000 We would like to look at X to the n over 2. 00:15:50.000 --> 00:15:53.000 This is going to be a little bit tricky. 00:15:53.000 --> 00:15:56.000 Now we are going to have to pay attention to floors and 00:15:56.000 --> 00:15:59.000 ceilings. What I would like to say is 00:15:59.000 --> 00:16:04.000 while X to the n is X to the n over 2 times X to the n over 2. 00:16:04.000 --> 00:16:07.000 And this is true if n is even. If it is odd then I need to be 00:16:07.000 --> 00:16:11.000 a little bit more careful. But let's just think about the 00:16:11.000 --> 00:16:14.000 intuition why this is a good divide-and-conquer algorithm. 00:16:14.000 --> 00:16:17.000 We have a problem of size n, let's say. 00:16:17.000 --> 00:16:20.000 We convert it into, it looks like two subproblems 00:16:20.000 --> 00:16:23.000 of size n over 2, but in fact they are the same 00:16:23.000 --> 00:16:25.000 subproblems. So, I only have to solve one of 00:16:25.000 --> 00:16:28.000 them. If I compute X to the n over 2. 00:16:28.000 --> 00:16:33.000 Yeah, I know X to the n over 2. So, there is one conversive 00:16:33.000 --> 00:16:38.000 call, problem of size n over 2, then I square that number. 00:16:38.000 --> 00:16:44.000 And that is one computation. So, exactly the same recurrence 00:16:44.000 --> 00:16:48.000 as binary search, log n time much better than n. 00:16:48.000 --> 00:16:51.000 Cool. I also have to solve the odd 00:16:51.000 --> 00:16:53.000 case. So, n is odd. 00:16:53.000 --> 00:00:02.000 I will look at n minus 1 over 00:16:56.000 --> 00:17:02.000 N minus 1 better be even. And then I am missing another 00:17:02.000 --> 00:17:04.000 factor of X. If n is odd, 00:17:04.000 --> 00:17:09.000 I am going to have to do one recursive call and two 00:17:09.000 --> 00:17:12.000 multiplications. The same recurrence. 00:17:12.000 --> 00:17:17.000 One recursive problem of size n over 2, plus constant time. 00:17:17.000 --> 00:17:21.000 The dividing work here is dividing by 2 and the 00:17:21.000 --> 00:17:27.000 combination work is doing one or possibly two multiplications. 00:17:27.000 --> 00:17:32.000 And this is lg n. And if all you are allowed to 00:17:32.000 --> 00:17:37.000 do is multiply numbers, lg n is the best you can do. 00:17:37.000 --> 00:17:40.000 Good. Simple but powerful. 00:17:40.000 --> 00:17:44.000 Whenever you want to compute a power of a number, 00:17:44.000 --> 00:17:48.000 now you know what to do. 00:17:55.000 --> 00:17:58.000 Does anyone not know the definition of Fibonacci numbers 00:17:58.000 --> 00:18:01.000 and is willing to admit it? OK, so this is a good old 00:18:01.000 --> 00:18:02.000 friend. I will write down the 00:18:02.000 --> 00:18:05.000 definition as just a reminder. And, in particular, 00:18:05.000 --> 00:18:07.000 the base cases. 00:18:17.000 --> 00:18:18.000 Fibonacci numbers, I will claim, 00:18:18.000 --> 00:18:21.000 are very important because it appears throughout nature. 00:18:21.000 --> 00:18:24.000 You look at certain fruits, you see the Fibonacci sequence. 00:18:24.000 --> 00:18:28.000 If you count the number of little bumps around each ring. 00:18:28.000 --> 00:18:31.000 If you look at the sand on the beach and how the waves hit it, 00:18:31.000 --> 00:18:34.000 it's the Fibonacci sequence I am told. 00:18:34.000 --> 00:18:39.000 If you look all over the place, Fibonacci sequence is there. 00:18:39.000 --> 00:18:43.000 How does nature compute the Fibonacci sequence? 00:18:43.000 --> 00:18:46.000 Well, that is a different class. 00:18:46.000 --> 00:18:52.000 But how are we going to compute the Fibonacci sequence as fast 00:18:52.000 --> 00:18:56.000 as possible? You have probably seen two 00:18:56.000 --> 00:18:59.000 algorithms. The most naÔve algorithm is the 00:18:59.000 --> 00:19:04.000 recursive algorithm. Where you say OK, 00:19:04.000 --> 00:19:06.000 f of n. I say well, if n is zero, 00:19:06.000 --> 00:19:09.000 return zero, if n is 1, return one. 00:19:09.000 --> 00:19:13.000 Otherwise, recursively compute f of n minus 1 and f of n minus 00:19:13.000 --> 00:19:17.000 2, add them together. How much time does this 00:19:17.000 --> 00:19:20.000 algorithm take, for those who have seen it 00:19:20.000 --> 00:19:22.000 before? This is not obvious to guess. 00:19:22.000 --> 00:19:26.000 It doesn't have to be exact. 00:19:31.000 --> 00:19:33.000 OK. And how many people have seen 00:19:33.000 --> 00:19:36.000 this algorithm before and analyzed it? 00:19:36.000 --> 00:19:39.000 Half, OK. So what is the running time? 00:19:39.000 --> 00:19:41.000 Really, really long, very good. 00:19:41.000 --> 00:19:44.000 Anymore precise answers? What's that? 00:19:44.000 --> 00:19:45.000 Exponential, yes. 00:19:45.000 --> 00:19:48.000 That is also correct and more precise. 00:19:48.000 --> 00:19:52.000 I will be even more precise. Maybe you haven't seen this 00:19:52.000 --> 00:19:56.000 analysis before. It's phi to the n where phi is 00:19:56.000 --> 00:20:01.000 the Golden Ratio. Again, Golden Ratio appears 00:20:01.000 --> 00:20:03.000 throughout the world in mathematics. 00:20:03.000 --> 00:20:07.000 This is probably the only time in this class, 00:20:07.000 --> 00:20:11.000 I'm afraid, but there we go. It made its cameo so we are 00:20:11.000 --> 00:20:14.000 happy. This is called exponential 00:20:14.000 --> 00:20:16.000 time. This is bigger than one, 00:20:16.000 --> 00:20:20.000 that's all you need to know. This is exponential time. 00:20:20.000 --> 00:20:25.000 Exponential time means basically some constant to the 00:20:25.000 --> 00:20:28.000 power n. Exponential time is a very long 00:20:28.000 --> 00:20:30.000 time. It's bad. 00:20:30.000 --> 00:20:35.000 Polynomial time is good. [LAUGHTER] This is what we 00:20:35.000 --> 00:20:38.000 want, polynomial time algorithms. 00:20:38.000 --> 00:20:42.000 This class is basically entirely about polynomial time 00:20:42.000 --> 00:20:44.000 algorithms. Question? 00:20:44.000 --> 00:20:47.000 Oh, say what the algorithm does again. 00:20:47.000 --> 00:20:53.000 Define function Fibonacci of n? I check for the base cases. 00:20:53.000 --> 00:20:59.000 Otherwise, I recursively call Fibonacci of n minus 1. 00:20:59.000 --> 00:21:02.000 I recursively call Fibonacci of n minus 2 and add those two 00:21:02.000 --> 00:21:04.000 numbers together. So, you get this branching 00:21:04.000 --> 00:21:06.000 tree. You are solving two subproblems 00:21:06.000 --> 00:21:09.000 of almost the same size, just additively smaller by one 00:21:09.000 --> 00:21:10.000 or two. I mean you are almost not 00:21:10.000 --> 00:21:13.000 reducing the problem size at all, so that's intuitively why 00:21:13.000 --> 00:21:16.000 it is exponential. You can draw a recursion tree 00:21:16.000 --> 00:21:18.000 and you will see how big it gets and how quickly. 00:21:18.000 --> 00:21:21.000 I mean by n over two levels, you've only reduced on one 00:21:21.000 --> 00:21:23.000 branch the problem from n to n over 2. 00:21:23.000 --> 00:21:26.000 The other one, maybe you've gotten from n down 00:21:26.000 --> 00:21:29.000 to one, but none of the branches have stopped after n of two 00:21:29.000 --> 00:21:32.000 levels. You have at least 2 to the 00:21:32.000 --> 00:21:37.000 power n over 2 which is like square root of 2 to the power n, 00:21:37.000 --> 00:21:40.000 which is getting close to phi to the n. 00:21:40.000 --> 00:21:42.000 So, this is definitely exponential. 00:21:42.000 --> 00:21:46.000 And exponential is bad. We want polynomial. 00:21:46.000 --> 00:21:49.000 N squared, n cubed, log n would be nice. 00:21:49.000 --> 00:21:53.000 Anything that is bounded above by a polynomial is good. 00:21:53.000 --> 00:21:57.000 This is an old idea. It goes back to one of the main 00:21:57.000 --> 00:22:01.000 people who said polynomial is good, Jack Edmonds who is famous 00:22:01.000 --> 00:22:06.000 in the combinatorial optimization world. 00:22:06.000 --> 00:22:12.000 He is my academic grandfather on one side. 00:22:12.000 --> 00:22:18.000 He is a very interesting guy. 00:22:32.000 --> 00:22:34.000 OK, so that's a really bad algorithm. 00:22:34.000 --> 00:22:37.000 You have probably seen a somewhat better algorithm, 00:22:37.000 --> 00:22:41.000 which you might think of as the bottom-up implantation of that 00:22:41.000 --> 00:22:43.000 recursive algorithm. 00:22:49.000 --> 00:22:51.000 Or, another way to think of it is if you build out the 00:22:51.000 --> 00:22:54.000 recursion tree for Fibonacci of n, you will see that there are 00:22:54.000 --> 00:22:58.000 lots of common subtrees that you are just wasting time on. 00:22:58.000 --> 00:23:01.000 When you solve Fibonacci of n minus 1, it again solves 00:23:01.000 --> 00:23:04.000 Fibonacci of n minus 2. Why solve it twice? 00:23:04.000 --> 00:23:07.000 You only need to solve it once. So, it is really easy to do 00:23:07.000 --> 00:23:11.000 that if you do it bottom-up. But you could also do it 00:23:11.000 --> 00:23:14.000 recursively with some cache of things you have already 00:23:14.000 --> 00:23:16.000 computed. So, no big surprise. 00:23:16.000 --> 00:23:19.000 You compute the Fibonacci numbers in order. 00:23:19.000 --> 00:23:22.000 And each time, when you compute Fibonacci of 00:23:22.000 --> 00:23:24.000 n, let's say, you have already computed the 00:23:24.000 --> 00:23:27.000 previous two, you add them together, 00:23:27.000 --> 00:23:32.000 it takes constant time. So, the running time here is 00:23:32.000 --> 00:23:36.000 linear, linear in n, and as our input. 00:23:36.000 --> 00:23:39.000 Great. Is that the best we can do? 00:23:39.000 --> 00:23:42.000 No. Any ideas on how we could 00:23:42.000 --> 00:23:47.000 compute Fibonacci of n faster than linear time? 00:23:47.000 --> 00:23:52.000 Now we should diverge from what you have already seen, 00:23:52.000 --> 00:23:56.000 most of you. Any ideas using techniques you 00:23:56.000 --> 00:24:00.000 have already seen? Yes? 00:24:00.000 --> 00:24:02.000 Yes. We can use the mathematical 00:24:02.000 --> 00:24:05.000 trick of phi and psi to the nth powers. 00:24:05.000 --> 00:24:09.000 In fact, you can just use phi, phi, pi, pho, 00:24:09.000 --> 00:24:13.000 phum, whatever you want to call this Greek letter. 00:24:13.000 --> 00:24:16.000 Good. Here is the mathematical trick. 00:24:16.000 --> 00:24:18.000 And, indeed, this is cheating, 00:24:18.000 --> 00:24:21.000 as you have said. This is no good, 00:24:21.000 --> 00:24:24.000 but so it is. We will call it naÔve recursive 00:24:24.000 --> 00:24:30.000 squaring and say well, we know recursive squaring. 00:24:30.000 --> 00:24:32.000 Recursive squaring takes log n time. 00:24:32.000 --> 00:24:36.000 Let's use recursive squaring. And if you happen to know lots 00:24:36.000 --> 00:24:40.000 of properties of the Fibonacci numbers, you don't have to, 00:24:40.000 --> 00:24:44.000 but here is one of them. If you take phi to the n 00:24:44.000 --> 00:24:48.000 divided by root 5 and you round it to the nearest integer that 00:24:48.000 --> 00:24:51.000 is the nth Fibonacci number. 00:24:58.000 --> 00:25:01.000 This is pretty cool. Fibonacci of n is basically phi 00:25:01.000 --> 00:25:03.000 to the n. We could apply recursive 00:25:03.000 --> 00:25:07.000 squaring to compute phi to the n in log n time, 00:25:07.000 --> 00:25:11.000 divide by root 5, assume that our computer has an 00:25:11.000 --> 00:25:16.000 operation that rounds a number to its nearest integer and poof, 00:25:16.000 --> 00:25:19.000 we are done. That doesn't work for many 00:25:19.000 --> 00:25:22.000 different reasons. On a real machine, 00:25:22.000 --> 00:25:26.000 probably you would represent phi and root 5 as floating point 00:25:26.000 --> 00:25:32.000 numbers which have some fixed amount of precise bits. 00:25:32.000 --> 00:25:35.000 And if you do this computation, you will lose some of the 00:25:35.000 --> 00:25:37.000 important bits. And when you round to the 00:25:37.000 --> 00:25:39.000 nearest integer you won't get the right answer. 00:25:39.000 --> 00:25:42.000 So, floating point round off will kill you on a floating 00:25:42.000 --> 00:25:45.000 point machine. On a theoretical machine where 00:25:45.000 --> 00:25:48.000 we magically have numbers that can do crazy things like this, 00:25:48.000 --> 00:25:51.000 I mean it really takes more than constant time per 00:25:51.000 --> 00:25:53.000 multiplication. So, we are sort of in a 00:25:53.000 --> 00:25:56.000 different model. You cannot multiply phi times 00:25:56.000 --> 00:25:58.000 phi in constant time. I mean that's sort of outside 00:25:58.000 --> 00:26:03.000 the boundaries of this course, but that's the way it is. 00:26:03.000 --> 00:26:06.000 In fact, in a normal machine, some problems you can only 00:26:06.000 --> 00:26:09.000 solve in exponential time. In a machine where you can 00:26:09.000 --> 00:26:13.000 multiply real numbers and round them to the nearest integers, 00:26:13.000 --> 00:26:15.000 you can solve them in polynomial time. 00:26:15.000 --> 00:26:19.000 So, it really breaks the model. You can do crazy things if you 00:26:19.000 --> 00:26:22.000 were allowed to do this. This is not allowed. 00:26:22.000 --> 00:26:25.000 And I am foreshadowing like three classes ahead, 00:26:25.000 --> 00:26:28.000 or three courses ahead, so I shouldn't talk more about 00:26:28.000 --> 00:26:32.000 it. But it turns out we can use 00:26:32.000 --> 00:26:35.000 recursive squaring in a different way if we use a 00:26:35.000 --> 00:26:38.000 different property of Fibonacci numbers. 00:26:38.000 --> 00:26:42.000 And then we will just stick with integers and everything 00:26:42.000 --> 00:26:45.000 will be happy. Don't forget to go to 00:26:45.000 --> 00:26:48.000 recitation and if you want to homework lab. 00:26:48.000 --> 00:26:51.000 Don't come here on Monday. That is required. 00:26:51.000 --> 00:26:57.000 This is sort of the right recursive squaring algorithm. 00:26:57.000 --> 00:27:02.000 And this is a bit hard to guess if you haven't already seen it, 00:27:02.000 --> 00:27:07.000 so I will just give it to you. I will call this a theorem. 00:27:07.000 --> 00:27:12.000 It's the first time I get to use the word theorem in this 00:27:12.000 --> 00:27:15.000 class. It turns out the nth Fibonacci 00:27:15.000 --> 00:27:18.000 number is the nth power of this matrix. 00:27:18.000 --> 00:27:21.000 Cool. If you look at it a little bit 00:27:21.000 --> 00:27:24.000 you say oh, yeah, of course. 00:27:24.000 --> 00:27:29.000 And we will prove this theorem in a second. 00:27:29.000 --> 00:27:31.000 But once we have this theorem, we can compute f of n by 00:27:31.000 --> 00:27:34.000 computing the nth power of this matrix. 00:27:34.000 --> 00:27:36.000 It's a two-by-two matrix. You multiply two two-by-two 00:27:36.000 --> 00:27:39.000 matrixes together, you get a two-by-two matrix. 00:27:39.000 --> 00:27:41.000 So that is constant size, four numbers. 00:27:41.000 --> 00:27:44.000 I can handle four numbers. We don't have crazy precision 00:27:44.000 --> 00:27:46.000 problems on the floating point side. 00:27:46.000 --> 00:27:48.000 There are only four numbers to deal with. 00:27:48.000 --> 00:27:52.000 Matrixes aren't getting bigger. So, the running time of this 00:27:52.000 --> 00:27:55.000 divide-and-conquer algorithm will still be log n because it 00:27:55.000 --> 00:27:57.000 takes a constant time per two-by-two matrix 00:27:57.000 --> 00:28:00.000 multiplication. Yes? 00:28:00.000 --> 00:28:02.000 Oh, yes. Thank you. 00:28:02.000 --> 00:28:06.000 I have a type error. Sorry about that. 00:28:06.000 --> 00:28:12.000 F of n is indeed the upper left corner, I hope. 00:28:12.000 --> 00:28:17.000 I better check I don't have it off by one. 00:28:17.000 --> 00:28:21.000 I do. It's F_n upper right corner, 00:28:21.000 --> 00:28:24.000 indeed. That's what you said. 00:28:24.000 --> 00:28:27.000 F of n. I need more space. 00:28:27.000 --> 00:28:32.000 Sorry. I really ought to have a 00:28:32.000 --> 00:28:36.000 two-by-two matrix on the left-hand side there. 00:28:36.000 --> 00:28:40.000 Thank you. So, I compute this nth power of 00:28:40.000 --> 00:28:44.000 a matrix in log n time, I take the upper right corner 00:28:44.000 --> 00:28:48.000 or the lower left corner, your choice, 00:28:48.000 --> 00:28:51.000 that is the nth Fibonacci number. 00:28:51.000 --> 00:28:56.000 This implies an order log n time algorithm with the same 00:28:56.000 --> 00:29:01.000 recurrence as the last two, binary search and really the 00:29:01.000 --> 00:29:07.000 recursive squaring algorithm. It is log n plus a constant, 00:29:07.000 --> 00:29:10.000 so log n. Let's prove that theorem. 00:29:30.000 --> 00:29:34.000 Any suggestions on what techniques we might use for 00:29:34.000 --> 00:29:37.000 proving this theorem, or what technique, 00:29:37.000 --> 00:29:40.000 singular? Induction, very good. 00:29:40.000 --> 00:29:44.000 I think any time I ask that question the answer is 00:29:44.000 --> 00:29:47.000 induction. Hint for the future in this 00:29:47.000 --> 00:29:49.000 class. 00:29:54.000 --> 00:29:56.000 A friend of mine, when he took an analysis class, 00:29:56.000 --> 00:29:59.000 whenever the professor asked, and what is the answer to this 00:29:59.000 --> 00:30:01.000 question, the answer was always zero. 00:30:01.000 --> 00:30:04.000 If you have taken analysis class that is funny. 00:30:04.000 --> 00:30:10.000 [LAUGHTER] Maybe I will try to ask some questions whose answers 00:30:10.000 --> 00:30:13.000 are zero just for our own amusement. 00:30:13.000 --> 00:30:17.000 We are going to induct on n. It's pretty much the obvious 00:30:17.000 --> 00:30:20.000 thing to do. But we have to check some 00:30:20.000 --> 00:30:23.000 cases. So, the base case is we have 00:30:23.000 --> 00:30:27.000 this to the first power. And that is itself [(1, 00:30:27.000 --> 00:30:33.000 1), (1, 0)]. And I should have said n is at 00:30:33.000 --> 00:30:36.000 least 1. And you can check. 00:30:36.000 --> 00:30:40.000 This is supposed to be F_2, F_1, F_1, F_0. 00:30:40.000 --> 00:30:46.000 And you can check it is, F_0 is 0, F_1 is 1 and F_2 is 00:30:46.000 --> 00:00:01.000 Base case is correct, step case is about as exciting, 00:00:01.000 --> 00:30:46.000 Good. 00:30:52.000 --> 00:30:59.000 but you've got to prove that your algorithm works. 00:30:59.000 --> 00:31:03.000 Suppose this is what we want to compute. 00:31:03.000 --> 00:31:10.000 I am just going to sort of, well, there are many ways I can 00:31:10.000 --> 00:31:14.000 do this. I will just do it the fast way 00:31:14.000 --> 00:31:18.000 because it's really not that exciting. 00:31:18.000 --> 00:31:23.000 Which direction? Let's do this direction. 00:31:23.000 --> 00:31:29.000 I want to use induction on n. If I want to use induction on 00:31:29.000 --> 00:31:33.000 n, presumably I should use what I already know is true. 00:31:33.000 --> 00:31:37.000 If I decrease n by 1, I have this property that this 00:31:37.000 --> 00:31:40.000 thing is going to be [(1, 1), (1, 0)] to the power n 00:31:40.000 --> 00:31:42.000 minus 1. This I already know, 00:31:42.000 --> 00:31:45.000 by the induction hypothesis, [(1, 1), (1, 00:31:45.000 --> 00:31:48.000 0)] to the n minus 1. So, presumably I should use it 00:31:48.000 --> 00:31:51.000 in some way. This equality is not yet true, 00:31:51.000 --> 00:31:54.000 you may have noticed. So, I need to add something on. 00:31:54.000 --> 00:31:59.000 What could I possibly add on to be correct? 00:31:59.000 --> 00:32:01.000 Well, another factor of [(1, 1), (1, 0)]. 00:32:01.000 --> 00:32:05.000 The way I am developing this proof is the only way it could 00:32:05.000 --> 00:32:08.000 possibly be, in some sense. If you know its induction, 00:32:08.000 --> 00:32:12.000 this is all that you could do. And then you check. 00:32:12.000 --> 00:32:14.000 Indeed, this equality holds conveniently. 00:32:14.000 --> 00:32:17.000 For example, F_(n plus 1) is the product of 00:32:17.000 --> 00:32:20.000 these two things. It is this row times this 00:32:20.000 --> 00:32:22.000 column. So, it is F_n times 1 plus F_(n 00:32:22.000 --> 00:32:25.000 minus 1) times 1, which is indeed the definition 00:32:25.000 --> 00:32:30.000 of F_(n plus 1). And you could check four of the 00:32:30.000 --> 00:32:32.000 entries. This is true. 00:32:32.000 --> 00:32:36.000 Great. If that is true then I would 00:32:36.000 --> 00:32:39.000 just put these together. That is [(1, 00:32:39.000 --> 00:32:43.000 1), (1, 0)] to the n minus 1 times [(1, 1), 00:32:43.000 --> 00:32:47.000 (1, 0)], which is [(1, 1), (1, 0)] to the n, 00:32:47.000 --> 00:32:50.000 end of proof. A very simple proof, 00:32:50.000 --> 00:32:55.000 but you have to do that in order to know if this algorithm 00:32:55.000 --> 00:32:57.000 really works. Good. 00:32:57.000 --> 00:33:00.000 Question? Oh, yes. 00:33:00.000 --> 00:33:03.000 Thank you. This, in the lower right, 00:33:03.000 --> 00:33:08.000 we should have F_(n minus 1). This is why you should really 00:33:08.000 --> 00:33:12.000 check your proofs. We would have discovered that 00:33:12.000 --> 00:33:16.000 when I checked that this was that row times that column, 00:33:16.000 --> 00:33:20.000 but that is why you are here, to fix my bugs. 00:33:20.000 --> 00:33:25.000 That's the great thing about being up here instead of in a 00:33:25.000 --> 00:33:29.000 quiz. But that is a minor mistake. 00:33:29.000 --> 00:33:31.000 You wouldn't lose much for that. 00:33:31.000 --> 00:33:34.000 All right. More divide-and-conquer 00:33:34.000 --> 00:33:37.000 algorithms. Still, we have done relatively 00:33:37.000 --> 00:33:41.000 simple ones so far. In fact, the fanciest has been 00:33:41.000 --> 00:33:44.000 merge sort, which we already saw. 00:33:44.000 --> 00:33:48.000 So, that is not too exciting. The rest have all be log n 00:33:48.000 --> 00:33:51.000 time. Let's break out of the log n 00:33:51.000 --> 00:33:54.000 world. Well, you all have the master 00:33:54.000 --> 00:33:58.000 method memorized, right, so I can erase that. 00:33:58.000 --> 00:34:01.000 Good. This will be a good test. 00:34:01.000 --> 00:34:04.000 Next problem is matrix multiplication, 00:34:04.000 --> 00:34:08.000 following right up on this two-by-two matrix 00:34:08.000 --> 00:34:11.000 multiplication. Let's see how we can compute 00:34:11.000 --> 00:34:15.000 n-by-n matrix multiplications. Just for a recap, 00:34:15.000 --> 00:34:19.000 you should know how to multiply matrixes, but here is the 00:34:19.000 --> 00:34:22.000 definition so we can turn it into an algorithm. 00:34:22.000 --> 00:34:26.000 You have two matrixes, A and B, which are capital 00:34:26.000 --> 00:34:28.000 levels. The ijth entry. 00:34:28.000 --> 00:34:34.000 Ith row, jth column is called little a_ij or little b_ij. 00:34:34.000 --> 00:34:40.000 And your goal is to compute the products of those matrixes. 00:34:40.000 --> 00:34:45.000 I should probably say that i and j range from 1 to n. 00:34:45.000 --> 00:34:51.000 So, they are square matrixes. The output is to compute C 00:34:51.000 --> 00:34:57.000 which has entry c_ij which is the product of A and B. 00:34:57.000 --> 00:35:01.000 And, for a recap, the ijth entry of the product 00:35:01.000 --> 00:35:08.000 is the inner product of the ith row of A with the jth column of 00:35:08.000 --> 00:35:13.000 B. But you can write that out as a 00:35:13.000 --> 00:35:17.000 sum like so. We want to compute this thing 00:35:17.000 --> 00:35:22.000 for every i and j. What is the obvious algorithm 00:35:22.000 --> 00:35:27.000 for doing this? Well, for every i and j you 00:35:27.000 --> 00:35:32.000 compute the sum. You compute all the products. 00:35:32.000 --> 00:35:35.000 You compute the sum. So, it's like n operations here 00:35:35.000 --> 00:35:37.000 roughly. I mean like 2n minus 1, 00:35:37.000 --> 00:35:40.000 whatever. It is order n operations. 00:35:40.000 --> 00:35:43.000 There are n^2 entries of C that I need to compute, 00:35:43.000 --> 00:35:47.000 so that's n^3 time. I will write this out just for 00:35:47.000 --> 00:35:50.000 the programmers at heart. Here is the pseudocode. 00:35:50.000 --> 00:35:53.000 It's rare that I will write pseudocode. 00:35:53.000 --> 00:35:57.000 And this is a simple enough algorithm that I can write it in 00:35:57.000 --> 00:36:03.000 gory detail. But it gives you some basis for 00:36:03.000 --> 00:36:06.000 this analysis if you like to program. 00:36:06.000 --> 00:36:10.000 It is a triply nested for loop. 00:36:17.000 --> 00:36:23.000 And I made a coding error. Hopefully you haven't written 00:36:23.000 --> 00:36:27.000 that far yet. I need c_ij to be initialized 00:36:27.000 --> 00:36:31.000 to zero. And then I add to c_ij the 00:36:31.000 --> 00:36:36.000 appropriate product, a_ik, b_kj. 00:36:36.000 --> 00:36:39.000 That is the algorithm. And the point is you have a 00:36:39.000 --> 00:36:42.000 nesting of n for loops from 1 to n. 00:36:42.000 --> 00:36:46.000 That takes n^3 time because this is constant and that is 00:36:46.000 --> 00:36:48.000 constant. So, very simple, 00:36:48.000 --> 00:36:51.000 n^3. Let's do better. 00:36:57.000 --> 00:36:58.000 And, of course, we are going to use 00:36:58.000 --> 00:36:59.000 divide-and-conquer. 00:37:04.000 --> 00:37:06.000 Now, how are we going to divide a matrix? 00:37:06.000 --> 00:37:10.000 There are a lot of numbers in a matrix, n^2 of them in each one. 00:37:10.000 --> 00:37:12.000 There are all sorts of ways you could divide. 00:37:12.000 --> 00:37:15.000 So far all of the divide-and-conquers we have done 00:37:15.000 --> 00:37:19.000 have been problems of size n into some number of problems of 00:37:19.000 --> 00:37:21.000 size n over 2. I am going to say I start with 00:37:21.000 --> 00:37:25.000 some matrixes of size n-by-n. I want to convert it down to 00:37:25.000 --> 00:37:28.000 something like n/2-by-n/2. Any suggestions how I might do 00:37:28.000 --> 00:37:30.000 that? Yeah? 00:37:30.000 --> 00:37:33.000 Block form the matrix, indeed. 00:37:33.000 --> 00:37:38.000 That is the right answer. So, this is the first 00:37:38.000 --> 00:37:43.000 divide-and-conquer algorithm. This will not work, 00:37:43.000 --> 00:37:47.000 but it has the first idea that we need. 00:37:47.000 --> 00:37:51.000 We have a n-by-n matrix. We can view it, 00:37:51.000 --> 00:37:55.000 this equality is more, you can think of it as, 00:37:55.000 --> 00:38:01.000 it's really the thing, a two-by-two block matrix where 00:38:01.000 --> 00:38:06.000 each entry in this two-by-two block matrix is a block of 00:38:06.000 --> 00:38:11.000 n/2-by-n/2 submatrixes. 00:38:20.000 --> 00:38:27.000 I will think of C as being divided into three parts, 00:38:27.000 --> 00:38:32.000 r, s, t and u. Even though I write them as 00:38:32.000 --> 00:38:37.000 lower case letters they are really matrixes. 00:38:37.000 --> 00:38:42.000 Each is n/2-by-n/2. And A, I will split into a, 00:38:42.000 --> 00:38:46.000 b, c, d, times B, I will split into e, 00:38:46.000 --> 00:38:48.000 f, g, h. Why not? 00:38:48.000 --> 00:38:53.000 This is certainly true. And if you've seen some linear 00:38:53.000 --> 00:39:01.000 algebra, this is basically what you can do with matrixes. 00:39:01.000 --> 00:39:05.000 Now I can pretend these are two-by-two and sort of forget 00:39:05.000 --> 00:39:09.000 the fact that these little letters are matrixes and say 00:39:09.000 --> 00:39:14.000 well, r is the inner product of this row with this column. 00:39:14.000 --> 00:39:17.000 It is ae times bg. Let me not cheat or else it 00:39:17.000 --> 00:39:23.000 will be too easy. r=ae+bg, s=af+bh, 00:39:23.000 --> 00:39:34.000 t=ce+dh and u=cf+dg. It's nothing like making it too 00:39:34.000 --> 00:39:41.000 hard on yourself. OK, got them right. 00:39:41.000 --> 00:39:48.000 Good. I mean this is just a fact 00:39:48.000 --> 00:39:59.000 about how you would expand out this product. 00:39:59.000 --> 00:40:01.000 And so now we have a recursive algorithm. 00:40:01.000 --> 00:40:04.000 In fact, we have a divide-and-conquer algorithm. 00:40:04.000 --> 00:40:07.000 We start with our n-by-n matrix. 00:40:07.000 --> 00:40:09.000 Well, we have two of them actually. 00:40:09.000 --> 00:40:12.000 We divide it into eight little pieces, a, b, 00:40:12.000 --> 00:40:13.000 c, d, e, f, g, h. 00:40:13.000 --> 00:40:16.000 Then we compute these things and that gives us C, 00:40:16.000 --> 00:40:20.000 just by sticking them together. Now, how do we compute these 00:40:20.000 --> 00:40:23.000 things? Well, these innocent-looking 00:40:23.000 --> 00:40:27.000 little products between these two little numbers are actually 00:40:27.000 --> 00:40:31.000 recursive matrix multiplications. 00:40:31.000 --> 00:40:37.000 Because each of these little letters is an n/2-by-n/2 matrix 00:40:37.000 --> 00:40:42.000 so I have to recursively compute the product. 00:40:42.000 --> 00:40:49.000 There are like eight recursive multiplications of n/2-by-n/2 00:40:49.000 --> 00:40:52.000 matrixes. That is what bites us. 00:40:52.000 --> 00:40:59.000 And then there are like four additions, plus minor work of 00:40:59.000 --> 00:41:05.000 gluing things together. How long does it take to add 00:41:05.000 --> 00:41:06.000 two matrixes together? n^2. 00:41:06.000 --> 00:41:09.000 This is cheap. It just takes n^2. 00:41:09.000 --> 00:41:12.000 Remember, we are trying to beat n^3 for our matrix 00:41:12.000 --> 00:41:15.000 multiplication. Addition is a really easy 00:41:15.000 --> 00:41:17.000 problem. You just have to add every 00:41:17.000 --> 00:41:20.000 number. There is no way you can do 00:41:20.000 --> 00:41:23.000 better than n^2. So, that is not recursive. 00:41:23.000 --> 00:41:26.000 That is the nice thing. But the bad thing is we have 00:41:26.000 --> 00:41:31.000 eight of these recursions. We have 00:41:31.000 --> 00:41:39.000 T(n)=8T(n/2)+Theta(n^2). And I have erased the master 00:41:39.000 --> 00:41:46.000 method, but you should all have it memorized. 00:41:46.000 --> 00:41:52.000 What is the solution to this recurrence? 00:41:52.000 --> 00:41:56.000 Theta(n^3). That is annoying. 00:41:56.000 --> 00:42:01.000 All right. A is 8, b is 2, 00:42:01.000 --> 00:42:07.000 log base 2 of 8 is 3. Every computer scientist should 00:42:07.000 --> 00:42:09.000 know that. n^log_b(a)=n^3. 00:42:09.000 --> 00:42:15.000 That is polynomially larger than n^2, so we are in Case 1. 00:42:15.000 --> 00:42:19.000 Thank you. Let's get them upside down. 00:42:19.000 --> 00:42:23.000 This is n^3, no better than our previous 00:42:23.000 --> 00:42:26.000 algorithm. That kind of sucks. 00:42:26.000 --> 00:42:30.000 And now comes the divine inspiration. 00:42:30.000 --> 00:42:34.000 Let's go over here. 00:42:41.000 --> 00:42:44.000 There are some algorithms like this Fibonacci algorithm where 00:42:44.000 --> 00:42:47.000 if you sat down for a little while, it's no big deal, 00:42:47.000 --> 00:42:51.000 you would figure it out. I mean it is kind of clever to 00:42:51.000 --> 00:42:54.000 look at that matrix and then everything works happily. 00:42:54.000 --> 00:42:57.000 It is not obvious but it is not that amazingly clever. 00:42:57.000 --> 00:43:01.000 This is an algorithm that is amazingly clever. 00:43:01.000 --> 00:43:05.000 You may have seen it before which steals the thunder a 00:43:05.000 --> 00:43:10.000 little bit, but it is still really, really cool so you 00:43:10.000 --> 00:43:15.000 should be happy to see it again. And how Strassen came up with 00:43:15.000 --> 00:43:19.000 this algorithm, he must have been very clever. 00:43:19.000 --> 00:43:24.000 The idea is we've got to get rid of these multiplications. 00:43:24.000 --> 00:43:30.000 I could do a hundred additions. That only costs Theta(n^2). 00:43:30.000 --> 00:43:33.000 I have to reduce this 8 to something smaller. 00:43:33.000 --> 00:43:35.000 It turns out, if you try to split the 00:43:35.000 --> 00:43:39.000 matrices into three-by-three or something, that doesn't help 00:43:39.000 --> 00:43:41.000 you. You get the same problem 00:43:41.000 --> 00:43:44.000 because we're using fundamentally the same 00:43:44.000 --> 00:43:47.000 algorithm, just in a different order. 00:43:47.000 --> 00:43:51.000 We have got to somehow reduce the number of multiplications. 00:43:51.000 --> 00:43:55.000 We are going to reduce it to 7. The claim is that if we have 00:43:55.000 --> 00:43:59.000 two two-by-two matrices we can take their product using seven 00:43:59.000 --> 00:44:03.000 multiplications. If that were true, 00:44:03.000 --> 00:44:08.000 we would reduce the 8 to a 7 and presumably make things run 00:44:08.000 --> 00:44:11.000 faster. We will see how fast in a 00:44:11.000 --> 00:44:14.000 moment. You can compute it in your 00:44:14.000 --> 00:44:16.000 head. If you are bored and like 00:44:16.000 --> 00:44:21.000 computing logs that are non-integral logs then go ahead. 00:44:21.000 --> 00:44:23.000 All right. Here we are. 00:44:23.000 --> 00:44:28.000 This algorithm is unfortunately rather long, but it is only 00:44:28.000 --> 00:44:32.000 seven multiplications. 00:44:45.000 --> 00:44:49.000 Each of these P's is a product of two things which only 00:44:49.000 --> 00:44:54.000 involves addition or subtraction, the same thing. 00:45:38.000 --> 00:45:41.000 Those are seven multiplications. 00:45:41.000 --> 00:45:44.000 And I can compute those in 7T(n/2). 00:45:44.000 --> 00:45:47.000 Oh, indeed it is. Six was wrong. 00:45:47.000 --> 00:45:51.000 Six and seven are the same, very good. 00:45:51.000 --> 00:45:57.000 You know, you think that copying something would not be 00:45:57.000 --> 00:46:02.000 such a challenging task. But when you become an 00:46:02.000 --> 00:46:07.000 absent-minded professor like me then you will know how easy it 00:46:07.000 --> 00:46:07.000 is. OK. We have them all correct, hopefully. 00:46:10.000 --> 00:46:12.000 We continue. That wasn't enough. 00:46:12.000 --> 00:46:16.000 Of course we had seven things. Clearly we have to reduce this 00:46:16.000 --> 00:46:19.000 down to four things, the elements of C. 00:46:19.000 --> 00:46:22.000 Here they are, the elements of C, 00:46:22.000 --> 00:46:25.000 r, s, t, u. It turns out r=P_5+P_4-P_2+P_6. 00:46:25.000 --> 00:46:27.000 Of course. Didn't you all see that? 00:46:27.000 --> 00:46:33.000 [LAUGHTER] I mean this one is really easy, 00:46:33.000 --> 00:46:36.000 s=P_1+P2. t=P_3+P_4. 00:46:36.000 --> 00:46:41.000 I mean that is clearly how they were chosen. 00:46:41.000 --> 00:46:48.000 And then u is another tricky one, u=P_5+P_1-P_3-P_7. 00:46:48.000 --> 00:46:52.000 OK. Now, which one of these would 00:46:52.000 --> 00:46:57.000 you like me to check? Don't be so nice. 00:46:57.000 --> 00:47:03.000 How about s? I can show you s is right. 00:47:03.000 --> 00:47:06.000 Any preferences? u. 00:47:06.000 --> 00:47:09.000 Oh, no, sign errors. OK. 00:47:09.000 --> 00:47:14.000 Here we go. The claim that this really 00:47:14.000 --> 00:47:21.000 works is you have to check all four of them. 00:47:21.000 --> 00:47:25.000 And I did in my notes. u=P_5. 00:47:25.000 --> 00:47:31.000 P_5=(ae + ah + de + dh). That is P_5. 00:47:31.000 --> 00:47:35.000 Check me. If I screw up, 00:47:35.000 --> 00:47:40.000 I am really hosed. (af - ah) = P_1. 00:47:40.000 --> 00:47:48.000 P_3 has a minus sign in front, so that is (ce + de). 00:47:48.000 --> 00:47:55.000 And then we have minus P_7, which is a big one, 00:47:55.000 --> 00:48:00.000 (ae + af - ce - cf). OK. 00:48:00.000 --> 00:48:12.000 Now I need like the assistant that crosses off things in 00:48:12.000 --> 00:48:19.000 parallel like the movie, right? 00:48:19.000 --> 00:48:25.000 ah, de, af, ce, ae, thank you, 00:48:25.000 --> 00:48:37.000 and hopefully these survive, dh minus minus cf. 00:48:37.000 --> 00:48:40.000 And, if we are lucky, that is exactly what is written 00:48:40.000 --> 00:48:42.000 here, except in the opposite order. 00:48:42.000 --> 00:48:45.000 Magic, right? Where the hell did Strassen get 00:48:45.000 --> 00:48:47.000 this? You have to be careful. 00:48:47.000 --> 00:48:51.000 It is OK that the plus is in the wrong order because plus is 00:48:51.000 --> 00:48:54.000 commutative, but the multiplications better not be in 00:48:54.000 --> 00:48:58.000 the wrong order because multiplication over matrixes is 00:48:58.000 --> 00:49:01.000 not commutative. I check cf, OK, 00:49:01.000 --> 00:49:05.000 dh, they are in the right order. 00:49:05.000 --> 00:49:11.000 I won't check the other three. That is matrix multiplication 00:49:11.000 --> 00:49:17.000 in hopefully subcubic time. Let's write down the 00:49:17.000 --> 00:49:19.000 recurrence. T(n) is now 7. 00:49:19.000 --> 00:49:25.000 Maybe I should write down the algorithm for kicks. 00:49:25.000 --> 00:49:30.000 Why not? Assuming I have time. 00:49:30.000 --> 00:49:33.000 Lots of time. Last lecture I was ten minutes 00:49:33.000 --> 00:49:36.000 early. I ended ten minutes early. 00:49:36.000 --> 00:49:40.000 I apologize for that. I know it really upsets you. 00:49:40.000 --> 00:49:46.000 And I didn't realize exactly when the class was supposed to 00:49:46.000 --> 00:49:48.000 end. So, today, I get to go ten 00:49:48.000 --> 00:49:50.000 minutes late. OK. 00:49:50.000 --> 00:49:52.000 Good. I'm glad you all agree. 00:49:52.000 --> 00:49:55.000 [LAUGHTER] I am kidding. Don't worry. 00:49:55.000 --> 00:49:56.000 OK. Algorithm. 00:49:56.000 --> 00:50:02.000 This is Strassen. First we divide, 00:50:02.000 --> 00:50:07.000 then we conquer and then we combine. 00:50:07.000 --> 00:50:14.000 As usual, I don't have it written anywhere here. 00:50:14.000 --> 00:50:17.000 Fine. Divide A and B. 00:50:17.000 --> 00:50:27.000 This is sort of trivial. Then we compute the terms -- 00:50:36.000 --> 00:50:39.000 -- for the products. This means we get ready to 00:50:39.000 --> 00:50:42.000 compute all the P's. We compute a+b, 00:50:42.000 --> 00:50:45.000 c+d, g-e, a+d, e+h and so on. 00:50:45.000 --> 00:50:49.000 All of the terms that appear in here, we compute those. 00:50:49.000 --> 00:50:54.000 That takes n2 time because it is just a bunch of additions and 00:50:54.000 --> 00:50:56.000 subtractions. No big deal. 00:50:56.000 --> 00:51:04.000 A constant number of them. Then we conquer by recursively 00:51:04.000 --> 00:51:12.000 computing all the P_i's. That is each our product of 00:51:12.000 --> 00:51:17.000 seven of them. We have P_1, 00:51:17.000 --> 00:51:21.000 P_2 up to P_7. And, finally, 00:51:21.000 --> 00:51:30.000 we combine, which is to compute r, s, t and u. 00:51:36.000 --> 00:51:40.000 And those are just additions and subtractions again, 00:51:40.000 --> 00:51:43.000 so they take n^2 times. So, here we finally get an 00:51:43.000 --> 00:51:47.000 algorithm that is nontrivial both in dividing and in 00:51:47.000 --> 00:51:51.000 combining. Recursion is always recursion, 00:51:51.000 --> 00:51:54.000 but now we have interesting steps one and three. 00:51:54.000 --> 00:51:59.000 The recurrence is T(n) is seven recursive subproblems, 00:51:59.000 --> 00:52:03.000 each are size n/2 plus order n^2, to do all this addition 00:52:03.000 --> 00:52:07.000 work. Now we need to solve this 00:52:07.000 --> 00:52:11.000 recurrence. We compute n^log_b(a), 00:52:11.000 --> 00:52:16.000 which here is nlog_2(7). And we know log base 2 of 8 is 00:00:03.000 --> 00:52:20.000 Log base 2 of 7 is going be a 00:52:20.000 --> 00:52:26.000 little bit less than 3 but still bigger than 2 because log base 2 00:52:26.000 --> 00:52:30.000 of 4 is 2. So, it is going to be 00:52:30.000 --> 00:52:36.000 polynomially larger than n^2 but polynomially smaller than n^3. 00:52:36.000 --> 00:52:41.000 We are again in Case 1. And this is the cheating way to 00:52:41.000 --> 00:52:44.000 write n log base 2 of 7, nlg7. 00:52:44.000 --> 00:52:48.000 lg means log base 2. You should know that. 00:52:48.000 --> 00:52:53.000 It is all over the textbook and in our problem sets and what 00:52:53.000 --> 00:52:56.000 not, nlg7. And, in particular, 00:52:56.000 --> 00:53:02.000 if I have my calculator here. This is a good old-fashion 00:53:02.000 --> 00:53:04.000 calculator. No, that is wrong. 00:53:04.000 --> 00:53:07.000 Sorry. It is strictly less than 2.81. 00:53:07.000 --> 00:53:11.000 It is strictly less than 2.81. That is cool. 00:53:11.000 --> 00:53:14.000 I mean it is polynomially better than n^3. 00:53:14.000 --> 00:53:18.000 Still not as good as addition, which is n^2. 00:53:18.000 --> 00:53:23.000 It is generally believed, although we don't know whether 00:53:23.000 --> 00:53:29.000 you can multiply as fast as you can divide for matrices. 00:53:29.000 --> 00:53:32.000 We think you cannot get n^2, but who knows? 00:53:32.000 --> 00:53:36.000 It could still happen. There are no lower bounds. 00:53:36.000 --> 00:53:40.000 This is not the best algorithm for matrix multiplication. 00:53:40.000 --> 00:53:44.000 It is sort of the simplest that beats n^3. 00:53:44.000 --> 00:53:46.000 The best so far is like n^2.376. 00:53:46.000 --> 00:53:50.000 Getting closer to 2. You might think these numbers 00:53:50.000 --> 00:53:54.000 are a bit weird. Maybe the constants out here 00:53:54.000 --> 00:54:00.000 dominate the improvement you are getting in the exponent. 00:54:00.000 --> 00:54:03.000 It turns out improving the exponent is a big deal. 00:54:03.000 --> 00:54:06.000 I mean, as n gets larger exponents really come out to 00:54:06.000 --> 00:54:09.000 bite you. So, n^3 is pretty impractical 00:54:09.000 --> 00:54:13.000 for any very large values of n. And we known that Strassen will 00:54:13.000 --> 00:54:16.000 beat normal matrix multiplication if n is 00:54:16.000 --> 00:54:19.000 sufficiently large. The claim is that roughly at 00:54:19.000 --> 00:54:22.000 about 32 or so already you get an improvement, 00:54:22.000 --> 00:54:25.000 for other reasons, not just because the exponent 00:54:25.000 --> 00:54:30.000 gets better, but there you go. So, this is pretty good. 00:54:30.000 --> 00:54:33.000 This is completely impractical, so don't use whatever this 00:54:33.000 --> 00:54:36.000 algorithm is. I don't have the reference 00:54:36.000 --> 00:54:40.000 handy, but it is just trying to get a theoretical improvement. 00:54:40.000 --> 00:54:44.000 There may be others that are in between and more reasonable but 00:54:44.000 --> 00:54:46.000 that is not it. Wow, lots of time. 00:54:46.000 --> 00:54:49.000 Any questions? We're not done yet, 00:54:49.000 --> 00:54:52.000 but any questions before we move on for matrix 00:54:52.000 --> 00:54:53.000 multiplication? OK. 00:54:53.000 --> 00:54:56.000 I have one more problem. 00:55:15.000 --> 00:55:17.000 Divide-and-conquer is a pretty general idea. 00:55:17.000 --> 00:55:20.000 I mean, you can use it to dominate countries. 00:55:20.000 --> 00:55:22.000 You can use it to multiply matrices. 00:55:22.000 --> 00:55:26.000 I mean, who would have thought? Here is a very different kind 00:55:26.000 --> 00:55:30.000 of problem you can solve with divide-and-conquer. 00:55:30.000 --> 00:55:32.000 It is not exactly an algorithmic problem, 00:55:32.000 --> 00:55:35.000 although it is computer science. 00:55:35.000 --> 00:55:37.000 That is clear. This is very large-scale 00:55:37.000 --> 00:55:40.000 integration. The chips, they are very large 00:55:40.000 --> 00:55:44.000 scale integrated. Probably even more these days, 00:55:44.000 --> 00:55:47.000 but that is the catch phrase. Here is a problem, 00:55:47.000 --> 00:55:51.000 and it arises in VLSI layout. We won't get into too many 00:55:51.000 --> 00:55:54.000 details why, but you have some circuit. 00:55:54.000 --> 00:55:58.000 And here I am going to assume that the circuit is a binary 00:55:58.000 --> 00:56:02.000 tree. This is just part of a circuit. 00:56:02.000 --> 00:56:07.000 Assume for now here that it is just a complete binary tree. 00:56:07.000 --> 00:56:11.000 A complete binary tree looks like this. 00:56:11.000 --> 00:56:15.000 In all of my teachings, I have drawn this figure for 00:56:15.000 --> 00:56:18.000 sure the most. It is my favorite figure, 00:56:18.000 --> 00:56:21.000 the height four complete binary tree. 00:56:21.000 --> 00:56:25.000 OK, there it is. I have some tree like that as 00:56:25.000 --> 00:56:30.000 some height. I want to imbed it into some 00:56:30.000 --> 00:56:36.000 chip layout on a grid. Let's say it has n leaves. 00:56:36.000 --> 00:56:41.000 I want to imbed it into a grid with minimum area. 00:56:41.000 --> 00:56:47.000 This is a very cute problem and it really shows you another way 00:56:47.000 --> 00:56:53.000 in which divide-and-conquer is a useful and powerful tool. 00:56:53.000 --> 00:57:00.000 So, I have this tree. I like to draw it in this way. 00:57:00.000 --> 00:57:02.000 I want to somehow draw it on the grid. 00:57:02.000 --> 00:57:06.000 What that means is the vertices have to be imbedded onto dots on 00:57:06.000 --> 00:57:09.000 the grid, and I am talking about the square grid. 00:57:09.000 --> 00:57:12.000 It has to go to vertices of the grid. 00:57:12.000 --> 00:57:15.000 And these edges have to be routed as sort of orthogonal 00:57:15.000 --> 00:57:19.000 paths between one dot and another, so that should be an 00:57:19.000 --> 00:57:22.000 edge and they shouldn't cross and all these good things 00:57:22.000 --> 00:57:25.000 because wires do not like to cross. 00:57:25.000 --> 00:57:28.000 There is the obvious way to solve this problem and there is 00:57:28.000 --> 00:57:31.000 the right way. 00:57:36.000 --> 00:57:37.000 And let's talk about both of them. 00:57:37.000 --> 00:57:41.000 Neither of them is particularly obvious, but divide-and-conquer 00:57:41.000 --> 00:57:44.000 sort of gives you a hint in the right direction. 00:57:44.000 --> 00:57:47.000 So, the naÔve imbedding. I seem to like the word naÔve 00:57:47.000 --> 00:57:48.000 here. 00:57:54.000 --> 00:57:58.000 I am going to draw this bottom up because it is easier, 00:57:58.000 --> 00:58:02.000 so leave three grid lines and then start drawing. 00:58:02.000 --> 00:58:05.000 I don't know how big that is going to be. 00:58:05.000 --> 00:58:11.000 Here is the bottom of our tree. This is like the little three 00:58:11.000 --> 00:58:15.000 nodes there. And then I leave a blank column 00:58:15.000 --> 00:58:21.000 and then a blank column. I don't actually need to leave 00:58:21.000 --> 00:58:25.000 those blank columns, but it makes a prettier 00:58:25.000 --> 00:58:30.000 drawing. And then we work our way up. 00:58:37.000 --> 00:58:40.000 There is the tree, which should be aligned, 00:58:40.000 --> 00:58:42.000 on a grid. No crossings. 00:58:42.000 --> 00:58:45.000 Everything is happy. How much area does it take? 00:58:45.000 --> 00:58:49.000 By area, I mean sort of the area of the bounding box. 00:58:49.000 --> 00:58:54.000 So, I count this blank space even though I am not using it 00:58:54.000 --> 00:59:00.000 and I count all this blank space even though I am not using it. 00:59:00.000 --> 00:59:04.000 I want to look at the height. Let's call this H(n). 00:59:04.000 --> 00:59:08.000 And to look at the width, which I will call W(n). 00:59:08.000 --> 00:59:13.000 Now, it is probably pretty obvious that H(n) is like log n, 00:59:13.000 --> 00:59:18.000 W(n) is like n or whatever. But I want to write it as a 00:59:18.000 --> 00:59:22.000 recurrence because that will inspire us to do the right 00:59:22.000 --> 00:59:23.000 thing. H(n). 00:59:23.000 --> 00:59:27.000 Well, if you think of this as a recursion-tree, 00:59:27.000 --> 00:59:31.000 in some sense. We start with the big tree. 00:59:31.000 --> 00:59:35.000 We split it into two halves, two subtrees of size n/2 indeed 00:59:35.000 --> 00:59:40.000 because we are counting leaves. It is exactly n/2 on each side. 00:59:40.000 --> 00:59:43.000 Then for height they are in parallel so it is no big deal. 00:59:43.000 --> 00:59:46.000 The height is just the height of this thing, 00:59:46.000 --> 00:59:48.000 one of these subproblems plus one. 00:59:48.000 --> 00:59:52.000 The width, you have to add together the two widths and also 00:59:52.000 --> 00:59:55.000 add on 1. You don't have to add on 1 00:59:55.000 --> 01:00:00.000 here, but it doesn't matter. It is certainly at most 1. 01:00:00.000 --> 01:00:10.598 H(n) = H(n/2) + Theta(1), there you do have to add 1, 01:00:10.598 --> 01:00:20.178 and W(n) = 2W(n/2) + O(1). The usual base cases. 01:00:20.178 --> 01:00:32.000 I mean, these are recurrences we should know and love. 01:00:32.000 --> 01:00:36.155 This is log n, I sort of have already given 01:00:36.155 --> 01:00:40.409 away the answers, and this better be linear. 01:00:40.409 --> 01:00:45.356 This is again Case 1. And to the log base 2 of 2 is 01:00:45.356 --> 01:00:49.512 n, which is the answer, much bigger than 1. 01:00:49.512 --> 01:00:54.459 And here n to the log base 2 of 1 is n to the zero, 01:00:54.459 --> 01:01:00.000 which is 1, which is the same so we get log n. 01:01:00.000 --> 01:01:04.085 The area is n log n, but if you are making chips you 01:01:04.085 --> 01:01:08.331 want the area as small as possible so you can fit more 01:01:08.331 --> 01:01:12.256 good stuff in there. So, we would like to aim for, 01:01:12.256 --> 01:01:16.182 well, we certainly cannot do a better area than n. 01:01:16.182 --> 01:01:19.627 You've got to put the leaves down somewhere, 01:01:19.627 --> 01:01:22.110 but this is already pretty good. 01:01:22.110 --> 01:01:26.356 It is only a log factor off, but we want to aim for n. 01:01:26.356 --> 01:01:31.053 How could we get n? Any guesses on what needs to 01:01:31.053 --> 01:01:34.794 change in this layout? Not how to do it because that 01:01:34.794 --> 01:01:37.801 is not obvious, but in terms of height and 01:01:37.801 --> 01:01:41.616 width what should we do? It is pretty hard to get the 01:01:41.616 --> 01:01:44.696 height smaller than log n, I will tell you, 01:01:44.696 --> 01:01:48.584 because this is a tree. It cannot really get its width 01:01:48.584 --> 01:01:52.398 down to less than log n. What could we do to make the 01:01:52.398 --> 01:01:54.819 product linear? Just random ideas. 01:01:54.819 --> 01:01:59.000 What are two functions whose product is n? 01:02:07.000 --> 01:02:09.227 Square root of n and square root of n. 01:02:09.227 --> 01:02:12.298 That is a good choice. Were there other suggestions? 01:02:12.298 --> 01:02:15.188 n times constant. Yeah, n times constant would be 01:02:15.188 --> 01:02:17.054 nice. But I claim you cannot get 01:02:17.054 --> 01:02:19.704 either of these down to less than a constant. 01:02:19.704 --> 01:02:23.316 You could aim for n over log n by log n, that is more likely, 01:02:23.316 --> 01:02:25.544 but I think that is almost impossible. 01:02:25.544 --> 01:02:30.000 Root n by root n is the right answer, so let's go with that. 01:02:30.000 --> 01:02:36.050 So, root n by root n. We haven't seen any recurrences 01:02:36.050 --> 01:02:42.450 whose solution is root n, but surely they are out there. 01:02:42.450 --> 01:02:49.316 Let's say the goal is to get W(n) = Theta(root n) and to get 01:02:49.316 --> 01:02:54.785 H(n) = Theta(root n). If we did that we would be 01:02:54.785 --> 01:03:02.000 happy, because then the area is the product is linear. 01:03:02.000 --> 01:03:05.677 How? What is a recurrence that is in 01:03:05.677 --> 01:03:11.245 the usual master method form whose solution is root n? 01:03:11.245 --> 01:03:15.237 I mean, you could think of it that way. 01:03:15.237 --> 01:03:20.490 Recurrence is a bit tricky, but let's just think of 01:03:20.490 --> 01:03:24.377 n^log_b(a). When is log base b of a Ω? 01:03:24.377 --> 01:03:29.000 Because then n^log_b(a) is root n. 01:03:29.000 --> 01:03:33.726 And there is some hope that I could get a root n solution to 01:03:33.726 --> 01:03:36.770 recurrence. This is designed by knowing 01:03:36.770 --> 01:03:41.095 that it is divide-and-conquer, and therefore it must be 01:03:41.095 --> 01:03:44.940 something like this. It is easy once you know the 01:03:44.940 --> 01:03:49.186 approach you are supposed to take and you can try this 01:03:49.186 --> 01:03:51.989 approach. When is log base b of a Ω? 01:03:51.989 --> 01:03:54.633 Lots of solutions, shout them out. 01:03:54.633 --> 01:04:00.000 4 and 2, that is a good one. I better get this right. 01:04:00.000 --> 01:04:06.410 Log base 4 of 2 is Ω because the square root of 4 is 2. 01:04:06.410 --> 01:04:10.089 So, let's aim for this. Why not? 01:04:10.089 --> 01:04:14.007 When would we get log base 4 of 2? 01:04:14.007 --> 01:04:19.823 This is b, this is a, so it should be 2T(n/4) plus 01:04:19.823 --> 01:04:24.690 something. And if I want the n^log_b(a) to 01:04:24.690 --> 01:04:31.338 dominate, it has got to be polynomially smaller than root 01:04:31.338 --> 01:04:34.388 n. So, this should be 01:04:34.388 --> 01:04:37.320 n^1/2-epsilon. But it could be smaller. 01:04:37.320 --> 01:04:39.867 It could be 1. Zero would be nice, 01:04:39.867 --> 01:04:43.029 but that is probably too much to hope for. 01:04:43.029 --> 01:04:46.966 So, something smaller, strictly polynomially smaller 01:04:46.966 --> 01:04:49.203 than root n. That is our goal. 01:04:49.203 --> 01:04:53.293 And now comes the magic. If you played with this for a 01:04:53.293 --> 01:04:58.000 while you would find it, I think, at this point. 01:04:58.000 --> 01:05:02.560 When you know that you are somehow solve this problem of 01:05:02.560 --> 01:05:07.287 size n with two subproblems of size n/4 what could you do? 01:05:07.287 --> 01:05:11.268 Well, if you start thinking of things as squares, 01:05:11.268 --> 01:05:15.000 this is the natural thing that happens. 01:05:20.000 --> 01:05:30.000 This is called the H layout. You can imagine why. 01:05:30.000 --> 01:05:35.578 It would be much easier to draw if I had a grid board, 01:05:35.578 --> 01:05:40.000 a graph board, whatever, if that exists. 01:05:46.000 --> 01:05:48.983 This is a recursive layout. I am only going to draw a 01:05:48.983 --> 01:05:51.680 couple iterations, but hopefully you can imagine 01:05:51.680 --> 01:05:53.000 the generalization. 01:06:05.000 --> 01:06:08.578 I take four Hs, a good plan because I want 01:06:08.578 --> 01:06:12.156 problems of size n/4. This has n/4 leaves. 01:06:12.156 --> 01:06:15.385 This has n/4 leaves. This is the root, 01:06:15.385 --> 01:06:19.400 by the way, in the middle. This has n/4 leaves. 01:06:19.400 --> 01:06:23.501 This has n/4 leaves. So, I have four problems of 01:06:23.501 --> 01:06:26.904 size n/4. Somehow I have got to get that 01:06:26.904 --> 01:06:31.471 down to two. Thankfully, if I look at width 01:06:31.471 --> 01:06:35.920 or if I look at height, there are only two that matter. 01:06:35.920 --> 01:06:40.286 And these two matter and these two get along for free. 01:06:40.286 --> 01:06:44.818 They are going in parallel, just like we had with height 01:06:44.818 --> 01:06:48.031 over here. But I get that both in height 01:06:48.031 --> 01:06:50.173 and in width. If I measure, 01:06:50.173 --> 01:06:54.538 well, now they are equal, so I will just call them the 01:06:54.538 --> 01:06:58.000 length. We have L(n)-by-L(n). 01:06:58.000 --> 01:07:01.117 And if I compute well, what is L(n)? 01:07:01.117 --> 01:07:06.194 I have here L(n/4) because there are only a quarter of the 01:07:06.194 --> 01:07:11.271 leaves in this one or in that one, then I have a constant, 01:07:11.271 --> 01:07:15.813 Theta(1), no big deal, and then I have L(n/4) again. 01:07:15.813 --> 01:07:20.000 So, I get the recurrence that I wanted. 01:07:26.000 --> 01:07:35.170 L(n) = 2L(n/4) + Theta(1). And that has solution square 01:07:35.170 --> 01:07:39.976 root of n, as we claimed before. Again, we are in Case 1 of the 01:07:39.976 --> 01:07:41.759 master method. Cool, ha? 01:07:41.759 --> 01:07:44.395 This is a much more compact layout. 01:07:44.395 --> 01:07:47.108 Charles, did you invent this layout? 01:07:47.108 --> 01:07:49.589 No. But I know it appears on your 01:07:49.589 --> 01:07:53.620 PhD thesis and you extended it in various directions. 01:07:53.620 --> 01:07:58.271 So, this is sort of a classic cool layout of trees into grids 01:07:58.271 --> 01:08:03.000 and another application of divide-and-conquer. 01:08:03.000 --> 01:08:07.788 I mean this is not particularly useful for algorithms directly. 01:08:07.788 --> 01:08:10.647 It is useful for VLSI layout directly. 01:08:10.647 --> 01:08:14.663 But it gives you more flavor of how you should think. 01:08:14.663 --> 01:08:18.371 If you know what running time you are aiming for, 01:08:18.371 --> 01:08:23.237 like in problem sets in quizzes often we say here is the running 01:08:23.237 --> 01:08:27.563 time you have got to get, think about the recurrence that 01:08:27.563 --> 01:08:31.116 will get you there. And that could inspire you. 01:08:31.116 --> 01:08:33.665 And that is it. Recitation Friday. 01:08:33.665 --> 01:08:36.446 Homework lab Sunday. No class Monday. 01:08:36.446 --> 01:08:39.000 See you Wednesday.