WEBVTT 00:00:00.000 --> 00:00:01.434 align:middle line:90% [SQUEAKING] 00:00:01.434 --> 00:00:02.390 align:middle line:90% 00:00:02.390 --> 00:00:03.802 align:middle line:90% [RUSTLING] 00:00:03.802 --> 00:00:04.302 align:middle line:90% 00:00:04.302 --> 00:00:05.736 align:middle line:90% [CLICKING] 00:00:05.736 --> 00:00:10.720 align:middle line:90% 00:00:10.720 --> 00:00:13.500 align:middle line:84% ZACHARY ABEL: All right, let's get into it. 00:00:13.500 --> 00:00:16.170 align:middle line:84% Last lecture of material before the quiz. 00:00:16.170 --> 00:00:18.390 align:middle line:84% So let's make sure we make the most of it. 00:00:18.390 --> 00:00:21.350 align:middle line:84% Today we're talking about recurrences. 00:00:21.350 --> 00:00:25.400 align:middle line:84% "Recurrence" is a fancy word for saying sequence described 00:00:25.400 --> 00:00:27.530 align:middle line:90% by induction. 00:00:27.530 --> 00:00:34.146 align:middle line:84% So it's a sequence defined inductively. 00:00:34.146 --> 00:00:37.040 align:middle line:90% 00:00:37.040 --> 00:00:38.880 align:middle line:84% And we've seen multiple examples. 00:00:38.880 --> 00:00:41.420 align:middle line:84% A great example is the Fibonacci numbers. 00:00:41.420 --> 00:00:43.160 align:middle line:90% Are there two B's? 00:00:43.160 --> 00:00:45.200 align:middle line:90% "Fibonnaci." 00:00:45.200 --> 00:00:47.010 align:middle line:84% Let's pretend I spelled that right. 00:00:47.010 --> 00:00:51.530 align:middle line:84% No, it's two B's and one N isn't it? 00:00:51.530 --> 00:00:52.650 align:middle line:90% Are there two C's? 00:00:52.650 --> 00:00:53.820 align:middle line:90% I don't know. 00:00:53.820 --> 00:00:56.630 align:middle line:90% [CHUCKLES] No one's helping me. 00:00:56.630 --> 00:00:58.220 align:middle line:90% Doesn't matter. 00:00:58.220 --> 00:01:02.580 align:middle line:90% Defined by F0 is 0, F1 is 1. 00:01:02.580 --> 00:01:07.250 align:middle line:84% And the important part is that fn is defined as Fn minus 1 00:01:07.250 --> 00:01:11.280 align:middle line:84% plus Fn minus 2 when n is at least 2. 00:01:11.280 --> 00:01:13.310 align:middle line:84% And this is the part where we're describing 00:01:13.310 --> 00:01:14.820 align:middle line:90% the sequence inductively. 00:01:14.820 --> 00:01:16.530 align:middle line:84% Once you know the first two terms, 00:01:16.530 --> 00:01:18.330 align:middle line:84% the next term is the sum of those. 00:01:18.330 --> 00:01:20.580 align:middle line:90% So F2 is 0 plus 1 is 2. 00:01:20.580 --> 00:01:22.590 align:middle line:90% F3 is 1 plus 2 is 3. 00:01:22.590 --> 00:01:23.810 align:middle line:90% Then 2 plus 3 is 5. 00:01:23.810 --> 00:01:27.980 align:middle line:84% 3 plus 5 is 8, then 13, 21, 34, and so on. 00:01:27.980 --> 00:01:30.300 align:middle line:84% It's just a sequence defined inductively. 00:01:30.300 --> 00:01:32.130 align:middle line:90% That's the idea of a recurrence. 00:01:32.130 --> 00:01:35.120 align:middle line:84% And these show up all the time when analyzing algorithms 00:01:35.120 --> 00:01:38.150 align:middle line:84% for many of the same reasons that summations show up 00:01:38.150 --> 00:01:39.630 align:middle line:90% when analyzing algorithms. 00:01:39.630 --> 00:01:42.480 align:middle line:84% And we need a lot of the same things out of them. 00:01:42.480 --> 00:01:47.030 align:middle line:84% If we end up with an algorithm whose runtime we can describe 00:01:47.030 --> 00:01:50.750 align:middle line:84% as the Fibonacci numbers or as a similar recurrence, 00:01:50.750 --> 00:01:54.090 align:middle line:84% well, that's not as useful as it could be. 00:01:54.090 --> 00:01:57.860 align:middle line:84% A closed form would be a lot more helpful, right, 00:01:57.860 --> 00:02:00.840 align:middle line:84% if we could have it, if we could find it. 00:02:00.840 --> 00:02:05.160 align:middle line:84% And indeed, Fibonacci numbers do have a closed form. 00:02:05.160 --> 00:02:10.560 align:middle line:84% The n-th Fibonacci number is 1 over square root of 5 times 1 00:02:10.560 --> 00:02:14.180 align:middle line:84% plus square root of 5 over 2 to the n minus 1 00:02:14.180 --> 00:02:17.750 align:middle line:84% minus square root of 5 over 2 to the n. 00:02:17.750 --> 00:02:19.710 align:middle line:90% And there's a closed form. 00:02:19.710 --> 00:02:24.350 align:middle line:84% You just take two exponentials, subtract them, divide by root 5, 00:02:24.350 --> 00:02:27.350 align:middle line:84% and you get the n-th Fibonacci number exactly. 00:02:27.350 --> 00:02:30.300 align:middle line:84% I think this is a really cool formula, also non-obvious. 00:02:30.300 --> 00:02:34.220 align:middle line:84% How do we possibly go from this to that? 00:02:34.220 --> 00:02:37.680 align:middle line:84% Well, we have some techniques that might work sometimes, 00:02:37.680 --> 00:02:41.900 align:middle line:90% and other times we won't. 00:02:41.900 --> 00:02:44.720 align:middle line:84% But certainly, once we know this formula, 00:02:44.720 --> 00:02:46.820 align:middle line:84% if we believe this formula or want 00:02:46.820 --> 00:02:48.150 align:middle line:90% to check whether it's true-- 00:02:48.150 --> 00:02:50.930 align:middle line:84% now that we know the formula, we can just try the induction 00:02:50.930 --> 00:02:53.390 align:middle line:84% and see if the induction works out. 00:02:53.390 --> 00:02:54.998 align:middle line:90% In this case, it will work out. 00:02:54.998 --> 00:02:57.540 align:middle line:84% And then you'll not only know that this is the right formula, 00:02:57.540 --> 00:02:59.120 align:middle line:90% but you'll have a proof for it. 00:02:59.120 --> 00:03:05.900 align:middle line:84% So just like with summations, if you know a closed form, then 00:03:05.900 --> 00:03:07.950 align:middle line:84% usually it's easy to prove it by induction. 00:03:07.950 --> 00:03:10.740 align:middle line:84% So finding the closed form is the hard part, 00:03:10.740 --> 00:03:12.540 align:middle line:90% proving it is the easy part. 00:03:12.540 --> 00:03:18.320 align:middle line:84% So this guess-and-check-- guess and check-- 00:03:18.320 --> 00:03:21.260 align:middle line:84% technique that we use for sums works just as well 00:03:21.260 --> 00:03:22.346 align:middle line:90% for recurrences. 00:03:22.346 --> 00:03:25.400 align:middle line:90% 00:03:25.400 --> 00:03:29.360 align:middle line:84% Now, recurrences very often show up, especially when 00:03:29.360 --> 00:03:32.100 align:middle line:90% analyzing recursive algorithms. 00:03:32.100 --> 00:03:34.760 align:middle line:84% So let's talk about recursive algorithms 00:03:34.760 --> 00:03:36.180 align:middle line:90% and show some examples. 00:03:36.180 --> 00:03:38.150 align:middle line:84% So let's start with a very classic, 00:03:38.150 --> 00:03:40.900 align:middle line:84% fun example called the "Towers of Hanoi." 00:03:40.900 --> 00:03:49.940 align:middle line:90% 00:03:49.940 --> 00:03:53.210 align:middle line:90% So this comes with its own myth. 00:03:53.210 --> 00:03:57.820 align:middle line:84% It was described by Edouard Luca in 1883. 00:03:57.820 --> 00:04:00.420 align:middle line:84% He said there is a temple in Hanoi 00:04:00.420 --> 00:04:03.690 align:middle line:90% that has three large pillars. 00:04:03.690 --> 00:04:06.370 align:middle line:84% I have some helpers today acting as my pillars. 00:04:06.370 --> 00:04:09.450 align:middle line:84% We've got "P-Town," "Wire," and Totoro. 00:04:09.450 --> 00:04:14.190 align:middle line:84% And on the leftmost pillar are 64 golden disks 00:04:14.190 --> 00:04:17.880 align:middle line:84% stacked on top of each other, getting bigger as you go down. 00:04:17.880 --> 00:04:22.140 align:middle line:84% And the monks there are moving the disks around one at a time 00:04:22.140 --> 00:04:24.370 align:middle line:90% but according to a couple rules. 00:04:24.370 --> 00:04:27.990 align:middle line:84% You're only allowed to move one disk at a time from the top 00:04:27.990 --> 00:04:29.850 align:middle line:90% to some other peg. 00:04:29.850 --> 00:04:33.210 align:middle line:84% And you're never allowed to put a big box 00:04:33.210 --> 00:04:34.210 align:middle line:90% on top of a small box-- 00:04:34.210 --> 00:04:39.300 align:middle line:84% I mean big golden disk on top of a small golden disk. 00:04:39.300 --> 00:04:40.720 align:middle line:90% Those are the rules-- 00:04:40.720 --> 00:04:42.100 align:middle line:90% one peg at a time. 00:04:42.100 --> 00:04:45.720 align:middle line:84% So I can't pick these both up and move them elsewhere. 00:04:45.720 --> 00:04:49.800 align:middle line:84% And always small on top of big, never the opposite. 00:04:49.800 --> 00:04:55.110 align:middle line:84% The goal is if all the disks start over here on the left peg, 00:04:55.110 --> 00:04:57.220 align:middle line:84% we want to get them all to a different peg. 00:04:57.220 --> 00:04:58.762 align:middle line:84% We want to move them all to the right 00:04:58.762 --> 00:05:02.670 align:middle line:84% peg by moving one disk at a time. 00:05:02.670 --> 00:05:06.720 align:middle line:84% So let me write that down quickly. 00:05:06.720 --> 00:05:14.940 align:middle line:84% So we have, let's say, n disks on the left peg. 00:05:14.940 --> 00:05:16.223 align:middle line:90% We have two other pegs. 00:05:16.223 --> 00:05:19.230 align:middle line:90% 00:05:19.230 --> 00:05:26.850 align:middle line:84% And we're going to move one at a time, 00:05:26.850 --> 00:05:40.355 align:middle line:84% always keeping smaller disks on top of bigger ones. 00:05:40.355 --> 00:05:43.770 align:middle line:90% 00:05:43.770 --> 00:05:53.730 align:middle line:84% Goal-- all disks on the right peg. 00:05:53.730 --> 00:05:56.920 align:middle line:84% And as the myth goes, when the monks finish, 00:05:56.920 --> 00:06:01.140 align:middle line:84% when they get all 64 disks over to the other peg, 00:06:01.140 --> 00:06:03.000 align:middle line:90% the universe ends. 00:06:03.000 --> 00:06:04.060 align:middle line:90% I don't know why. 00:06:04.060 --> 00:06:05.410 align:middle line:90% That's just the story. 00:06:05.410 --> 00:06:11.305 align:middle line:84% Maybe there's a weight-sensitive detonation switch over there. 00:06:11.305 --> 00:06:13.180 align:middle line:84% And you have to get all that weight on there, 00:06:13.180 --> 00:06:15.070 align:middle line:90% and then the universe blows up. 00:06:15.070 --> 00:06:15.850 align:middle line:90% It's a myth. 00:06:15.850 --> 00:06:18.210 align:middle line:84% You don't need actual answers here. 00:06:18.210 --> 00:06:19.740 align:middle line:90% But that's the game. 00:06:19.740 --> 00:06:22.720 align:middle line:84% And the question is, how many moves will it take? 00:06:22.720 --> 00:06:23.980 align:middle line:90% Is it even possible? 00:06:23.980 --> 00:06:26.590 align:middle line:84% How long does the universe have to live? 00:06:26.590 --> 00:06:27.640 align:middle line:90% Do we have a year? 00:06:27.640 --> 00:06:30.270 align:middle line:90% Do we have a century? 00:06:30.270 --> 00:06:32.320 align:middle line:84% Let's see if we can figure it out. 00:06:32.320 --> 00:06:34.200 align:middle line:84% Let's see if we can find an algorithm 00:06:34.200 --> 00:06:37.620 align:middle line:84% to get all of the disks from one side to the other. 00:06:37.620 --> 00:06:40.800 align:middle line:90% Are we clear on the game? 00:06:40.800 --> 00:06:41.580 align:middle line:90% Nice. 00:06:41.580 --> 00:06:43.110 align:middle line:90% Let's start small. 00:06:43.110 --> 00:06:46.230 align:middle line:84% I always recommend starting small. 00:06:46.230 --> 00:06:50.280 align:middle line:84% What happens if we have just one disk and we need to move it from 00:06:50.280 --> 00:06:52.050 align:middle line:90% "P-Town" to Totoro? 00:06:52.050 --> 00:06:53.315 align:middle line:90% How are we going to do that? 00:06:53.315 --> 00:06:56.010 align:middle line:90% 00:06:56.010 --> 00:06:59.100 align:middle line:90% Is that a hand? 00:06:59.100 --> 00:07:01.020 align:middle line:84% So how do I move a disk from here 00:07:01.020 --> 00:07:03.060 align:middle line:84% to here according to the rules of the game? 00:07:03.060 --> 00:07:03.560 align:middle line:90% Yes? 00:07:03.560 --> 00:07:04.622 align:middle line:90% AUDIENCE: You just do it. 00:07:04.622 --> 00:07:05.830 align:middle line:90% ZACHARY ABEL: You just do it. 00:07:05.830 --> 00:07:08.320 align:middle line:84% Yeah, we're allowed to move one disk at a time. 00:07:08.320 --> 00:07:11.110 align:middle line:84% And there aren't any bigger or smaller disks to get in the way. 00:07:11.110 --> 00:07:13.385 align:middle line:90% So you just do it in this case. 00:07:13.385 --> 00:07:16.620 align:middle line:90% 00:07:16.620 --> 00:07:21.900 align:middle line:84% So if we're trying to move just disk number 1 from, 00:07:21.900 --> 00:07:23.890 align:middle line:90% let's say, peg A to peg C-- 00:07:23.890 --> 00:07:26.610 align:middle line:90% we'll call the pegs A, B, C-- 00:07:26.610 --> 00:07:31.000 align:middle line:84% well, you do it just by doing it. 00:07:31.000 --> 00:07:36.420 align:middle line:84% You move disk 1 from A to C. Great, OK. 00:07:36.420 --> 00:07:39.660 align:middle line:90% What if we have two disks? 00:07:39.660 --> 00:07:41.020 align:middle line:90% Here we go. 00:07:41.020 --> 00:07:44.200 align:middle line:84% How am I going to get both of these from there to there? 00:07:44.200 --> 00:07:45.490 align:middle line:90% Who can help me out? 00:07:45.490 --> 00:07:46.283 align:middle line:90% Yes, please. 00:07:46.283 --> 00:07:47.700 align:middle line:84% AUDIENCE: Small one in the middle. 00:07:47.700 --> 00:07:48.730 align:middle line:84% ZACHARY ABEL: Small one in the middle. 00:07:48.730 --> 00:07:49.647 align:middle line:90% AUDIENCE: Second one-- 00:07:49.647 --> 00:07:51.700 align:middle line:84% ZACHARY ABEL: Second one goes over here. 00:07:51.700 --> 00:07:53.140 align:middle line:90% Small one on the big one. 00:07:53.140 --> 00:07:54.940 align:middle line:90% Absolutely, perfect. 00:07:54.940 --> 00:08:01.380 align:middle line:84% So if we're trying to move disks 1 and 2, the two smallest disks, 00:08:01.380 --> 00:08:07.180 align:middle line:84% again from peg A to peg C, as your colleague pointed out, 00:08:07.180 --> 00:08:11.220 align:middle line:84% first let's move disk 1 from A to B. Then 00:08:11.220 --> 00:08:16.170 align:middle line:84% we'll move disk 2 from A to C. Then we'll move disk 1 from B 00:08:16.170 --> 00:08:19.330 align:middle line:84% to C, just as we acted out over here. 00:08:19.330 --> 00:08:24.090 align:middle line:90% Next case-- three boxes. 00:08:24.090 --> 00:08:28.740 align:middle line:90% Now, this-- there we go-- 00:08:28.740 --> 00:08:32.250 align:middle line:84% this might take a little bit of planning ahead. 00:08:32.250 --> 00:08:34.900 align:middle line:84% And the thing to remember is that, at some point, 00:08:34.900 --> 00:08:36.720 align:middle line:84% we want to be able to make this move, 00:08:36.720 --> 00:08:40.659 align:middle line:84% where we take the biggest box from the left to the right. 00:08:40.659 --> 00:08:44.130 align:middle line:84% And so to do that, we need the other boxes here in the middle. 00:08:44.130 --> 00:08:46.140 align:middle line:84% So somewhere along the way, we're 00:08:46.140 --> 00:08:49.560 align:middle line:84% going to need to go from here to here. 00:08:49.560 --> 00:08:51.190 align:middle line:84% Of course, that's not a valid move. 00:08:51.190 --> 00:08:53.520 align:middle line:90% We can't move two boxes at once. 00:08:53.520 --> 00:08:54.610 align:middle line:90% Who can help me out? 00:08:54.610 --> 00:08:57.220 align:middle line:84% How are we going to do this first part of the algorithm? 00:08:57.220 --> 00:08:58.128 align:middle line:90% Please. 00:08:58.128 --> 00:08:59.920 align:middle line:84% AUDIENCE: Move the small box to the middle. 00:08:59.920 --> 00:09:01.505 align:middle line:84% ZACHARY ABEL: Small box to the middle. 00:09:01.505 --> 00:09:04.230 align:middle line:90% 00:09:04.230 --> 00:09:07.590 align:middle line:84% We're trying to move these two to the middle. 00:09:07.590 --> 00:09:09.380 align:middle line:90% So here to here is our goal. 00:09:09.380 --> 00:09:11.950 align:middle line:90% 00:09:11.950 --> 00:09:12.660 align:middle line:90% AUDIENCE: Wait. 00:09:12.660 --> 00:09:13.350 align:middle line:90% I need to think about this. 00:09:13.350 --> 00:09:14.110 align:middle line:90% ZACHARY ABEL: Yes. 00:09:14.110 --> 00:09:16.028 align:middle line:84% AUDIENCE: Move the small box to the right. 00:09:16.028 --> 00:09:17.570 align:middle line:84% ZACHARY ABEL: Small box to the right. 00:09:17.570 --> 00:09:19.426 align:middle line:84% AUDIENCE: And then the medium box-- 00:09:19.426 --> 00:09:21.370 align:middle line:84% ZACHARY ABEL: Medium box to the middle. 00:09:21.370 --> 00:09:22.780 align:middle line:90% Small box on top of it. 00:09:22.780 --> 00:09:23.550 align:middle line:90% Perfect. 00:09:23.550 --> 00:09:26.190 align:middle line:84% Does that look familiar to anyone? 00:09:26.190 --> 00:09:28.110 align:middle line:84% That's kind of exactly what we just 00:09:28.110 --> 00:09:31.960 align:middle line:84% described for moving boxes 1 and 2 from one peg to another. 00:09:31.960 --> 00:09:34.420 align:middle line:84% Instead of moving from the left peg to the right peg, 00:09:34.420 --> 00:09:36.720 align:middle line:84% we're moving from the left peg to the middle peg. 00:09:36.720 --> 00:09:39.010 align:middle line:84% But that's the only thing that's changing. 00:09:39.010 --> 00:09:42.360 align:middle line:84% We can use the same algorithm to move these two boxes from here 00:09:42.360 --> 00:09:44.580 align:middle line:90% to here in three moves. 00:09:44.580 --> 00:09:46.420 align:middle line:90% Now we can do this. 00:09:46.420 --> 00:09:49.300 align:middle line:84% And now we can do the same algorithm again 00:09:49.300 --> 00:09:52.520 align:middle line:84% to move these two boxes from here to here. 00:09:52.520 --> 00:09:59.620 align:middle line:84% And if we think about what that algorithm says, it says 1, 2, 3. 00:09:59.620 --> 00:10:02.590 align:middle line:84% Let's write that one down, as well. 00:10:02.590 --> 00:10:08.960 align:middle line:84% So if our goal is to move the interval 1 through 3, 00:10:08.960 --> 00:10:13.717 align:middle line:84% so boxes 1, 2, and 3, from peg A to peg C, 00:10:13.717 --> 00:10:15.550 align:middle line:84% well, then the first thing we're going to do 00:10:15.550 --> 00:10:20.020 align:middle line:84% is move boxes 1 and 2 from A to B. Then 00:10:20.020 --> 00:10:23.950 align:middle line:84% we're going to move box 3 on its own from A to C, 00:10:23.950 --> 00:10:28.780 align:middle line:90% then boxes 1 and 2 from B to C. 00:10:28.780 --> 00:10:31.600 align:middle line:84% And here, when I'm saying move two boxes at a time from one 00:10:31.600 --> 00:10:35.350 align:middle line:84% peg to another, I'm really invoking this strategy. 00:10:35.350 --> 00:10:37.610 align:middle line:84% We already know how to do it with two boxes. 00:10:37.610 --> 00:10:41.330 align:middle line:84% So let's use that as the first part of our algorithm. 00:10:41.330 --> 00:10:42.170 align:middle line:90% Then make one move. 00:10:42.170 --> 00:10:43.810 align:middle line:90% Then use that again. 00:10:43.810 --> 00:10:46.210 align:middle line:90% Does that make sense? 00:10:46.210 --> 00:10:48.023 align:middle line:84% Let's do one more example together. 00:10:48.023 --> 00:10:54.040 align:middle line:90% 00:10:54.040 --> 00:10:56.450 align:middle line:90% OK, great. 00:10:56.450 --> 00:10:58.250 align:middle line:90% We have four disks. 00:10:58.250 --> 00:11:02.110 align:middle line:84% We need to move them from the left to the right. 00:11:02.110 --> 00:11:03.890 align:middle line:84% And we're going to follow-- hey, everyone. 00:11:03.890 --> 00:11:07.000 align:middle line:84% We're going to follow the same idea. 00:11:07.000 --> 00:11:09.340 align:middle line:84% Somewhere along the way, we're going 00:11:09.340 --> 00:11:12.236 align:middle line:84% to need to get to this position-- 00:11:12.236 --> 00:11:15.010 align:middle line:90% 00:11:15.010 --> 00:11:19.510 align:middle line:84% to this position-- so that we can move the bigger box over 00:11:19.510 --> 00:11:20.920 align:middle line:90% here. 00:11:20.920 --> 00:11:23.980 align:middle line:84% And then we're going to need to, again, move these three 00:11:23.980 --> 00:11:27.250 align:middle line:84% boxes together somehow, with some sequence of moves, 00:11:27.250 --> 00:11:29.350 align:middle line:90% to finish the job. 00:11:29.350 --> 00:11:30.970 align:middle line:84% And I claim we have all the tools 00:11:30.970 --> 00:11:33.760 align:middle line:90% we need to do exactly that. 00:11:33.760 --> 00:11:40.090 align:middle line:84% So step 1-- in fact, let's write this down first. 00:11:40.090 --> 00:11:44.450 align:middle line:84% If we're trying to move the interval from 1 to 4-- 00:11:44.450 --> 00:11:47.440 align:middle line:90% so boxes 1, 2, 3, and 4-- 00:11:47.440 --> 00:11:51.530 align:middle line:84% from A to C, let's follow the same strategy here. 00:11:51.530 --> 00:11:53.713 align:middle line:90% Let's put the top three boxes-- 00:11:53.713 --> 00:11:56.470 align:middle line:90% 00:11:56.470 --> 00:11:59.440 align:middle line:84% let's move those from A to B somehow. 00:11:59.440 --> 00:12:02.020 align:middle line:84% Then we'll move box 4 on its own from A 00:12:02.020 --> 00:12:09.310 align:middle line:84% to C. Then we'll move boxes 1 through 3 from B to C. 00:12:09.310 --> 00:12:12.610 align:middle line:84% And here, where I'm saying move three boxes from one 00:12:12.610 --> 00:12:16.040 align:middle line:84% peg to another, we have an algorithm for that. 00:12:16.040 --> 00:12:18.010 align:middle line:90% It's this algorithm. 00:12:18.010 --> 00:12:19.090 align:middle line:90% Yeah? 00:12:19.090 --> 00:12:21.950 align:middle line:84% So let's see what this looks like in practice. 00:12:21.950 --> 00:12:25.540 align:middle line:84% So first things first, we're trying to move these top three 00:12:25.540 --> 00:12:28.480 align:middle line:90% boxes from here to here. 00:12:28.480 --> 00:12:29.930 align:middle line:90% Well, we have a way to do that. 00:12:29.930 --> 00:12:39.730 align:middle line:90% That's 1, 2, 3, 4, 5, 6, 7. 00:12:39.730 --> 00:12:42.280 align:middle line:84% If we followed the algorithm that's listed there, 00:12:42.280 --> 00:12:45.220 align:middle line:84% those are the seven moves it says to do. 00:12:45.220 --> 00:12:47.690 align:middle line:84% And I want to point out one important thing. 00:12:47.690 --> 00:12:49.840 align:middle line:90% We put this one over here. 00:12:49.840 --> 00:12:52.880 align:middle line:84% Back when we were just working with three boxes, 00:12:52.880 --> 00:12:54.635 align:middle line:84% we were assuming that all of the pegs 00:12:54.635 --> 00:12:56.260 align:middle line:84% were empty, that there was nothing else 00:12:56.260 --> 00:12:58.160 align:middle line:84% to worry about other than those three boxes. 00:12:58.160 --> 00:13:00.400 align:middle line:84% But now there's a fourth box in the way. 00:13:00.400 --> 00:13:02.410 align:middle line:84% So can we really run our algorithm 00:13:02.410 --> 00:13:07.180 align:middle line:84% for the top three boxes even when there's a fourth box? 00:13:07.180 --> 00:13:10.300 align:middle line:84% And, yeah, we can thankfully because the fourth box 00:13:10.300 --> 00:13:11.860 align:middle line:90% will always be underneath. 00:13:11.860 --> 00:13:15.110 align:middle line:84% The fourth box doesn't affect the movement of 1 through 3. 00:13:15.110 --> 00:13:18.640 align:middle line:84% In fact, any larger boxes don't affect the movement of boxes 1 00:13:18.640 --> 00:13:19.400 align:middle line:90% through 3. 00:13:19.400 --> 00:13:22.870 align:middle line:84% So we can use our algorithm for three boxes 00:13:22.870 --> 00:13:25.580 align:middle line:84% and just temporarily ignore the fact that there's a fourth box. 00:13:25.580 --> 00:13:26.660 align:middle line:90% It doesn't matter. 00:13:26.660 --> 00:13:30.250 align:middle line:84% And that's how we were able to get here. 00:13:30.250 --> 00:13:32.140 align:middle line:90% Then we're going to do this. 00:13:32.140 --> 00:13:35.590 align:middle line:84% And then, again, we're going to follow our three-box strategy 00:13:35.590 --> 00:13:37.850 align:middle line:90% to get these three over to here. 00:13:37.850 --> 00:13:47.830 align:middle line:84% So that looks like 1, 2, 3, 4, 5, 6, 7. 00:13:47.830 --> 00:13:50.020 align:middle line:90% And we've done four boxes. 00:13:50.020 --> 00:13:51.496 align:middle line:90% Yes? 00:13:51.496 --> 00:13:53.200 align:middle line:84% AUDIENCE: How are you able to claim 00:13:53.200 --> 00:13:55.630 align:middle line:84% that that sequence of moves, especially 00:13:55.630 --> 00:13:59.390 align:middle line:84% as the number of boxes becomes bigger, is the most optimal? 00:13:59.390 --> 00:14:00.980 align:middle line:90% ZACHARY ABEL: Great question. 00:14:00.980 --> 00:14:03.860 align:middle line:84% How are we able to claim that this sequence of moves, 00:14:03.860 --> 00:14:06.620 align:middle line:84% especially with more and more boxes or disks, 00:14:06.620 --> 00:14:08.630 align:middle line:90% is the optimal algorithm? 00:14:08.630 --> 00:14:11.990 align:middle line:84% We've started describing an algorithm to do it, 00:14:11.990 --> 00:14:14.020 align:middle line:90% but is it the best? 00:14:14.020 --> 00:14:15.880 align:middle line:90% That's a great question. 00:14:15.880 --> 00:14:19.030 align:middle line:90% It turns out it is the best. 00:14:19.030 --> 00:14:20.630 align:middle line:84% We're not going to prove it right now. 00:14:20.630 --> 00:14:23.860 align:middle line:84% But it's not much harder than what we've already done. 00:14:23.860 --> 00:14:27.890 align:middle line:84% The basic idea is, at some point, 00:14:27.890 --> 00:14:30.500 align:middle line:84% you have to move the smallest-- or the largest box. 00:14:30.500 --> 00:14:34.750 align:middle line:84% So at some point, you have to get to either this position 00:14:34.750 --> 00:14:37.550 align:middle line:90% or this position. 00:14:37.550 --> 00:14:39.253 align:middle line:84% And we know how many steps that takes, 00:14:39.253 --> 00:14:41.670 align:middle line:84% assuming we're doing some proof by induction or something. 00:14:41.670 --> 00:14:45.140 align:middle line:84% So we know we have to spend some number of steps 00:14:45.140 --> 00:14:47.700 align:middle line:90% for n minus 1 boxes. 00:14:47.700 --> 00:14:49.620 align:middle line:90% Then we have to make a move. 00:14:49.620 --> 00:14:51.170 align:middle line:84% And then we have to put these back 00:14:51.170 --> 00:14:54.860 align:middle line:90% on top, which, again, takes-- 00:14:54.860 --> 00:14:56.190 align:middle line:90% a landslide. 00:14:56.190 --> 00:15:02.840 align:middle line:90% 00:15:02.840 --> 00:15:05.490 align:middle line:84% So with an induction very similar to that, 00:15:05.490 --> 00:15:08.316 align:middle line:84% you can prove that, in fact, this is the optimal algorithm. 00:15:08.316 --> 00:15:10.850 align:middle line:90% 00:15:10.850 --> 00:15:13.670 align:middle line:84% OK, but how many moves does it take? 00:15:13.670 --> 00:15:17.660 align:middle line:84% Let's see if we can figure that out. 00:15:17.660 --> 00:15:25.600 align:middle line:84% So let's let T of n be the number of moves with n disks. 00:15:25.600 --> 00:15:28.580 align:middle line:90% 00:15:28.580 --> 00:15:32.970 align:middle line:84% And we want to figure out T of n somehow. 00:15:32.970 --> 00:15:36.200 align:middle line:84% So here, T of 1, if we're just moving one disk, 00:15:36.200 --> 00:15:37.770 align:middle line:90% we make one move. 00:15:37.770 --> 00:15:39.490 align:middle line:90% So T of 1 is 1. 00:15:39.490 --> 00:15:43.790 align:middle line:84% T of 2-- well, T of 2 is written here. 00:15:43.790 --> 00:15:45.230 align:middle line:90% It's 1, 2-- three moves. 00:15:45.230 --> 00:15:49.090 align:middle line:84% If we're moving two disks, we go 1, 2, 3-- 00:15:49.090 --> 00:15:50.000 align:middle line:90% three moves. 00:15:50.000 --> 00:15:52.840 align:middle line:90% How about T of 3? 00:15:52.840 --> 00:15:57.430 align:middle line:84% Well, it says do the two-disk algorithm. 00:15:57.430 --> 00:15:59.680 align:middle line:90% So that's T of 2. 00:15:59.680 --> 00:16:02.470 align:middle line:90% Then do a single move. 00:16:02.470 --> 00:16:05.120 align:middle line:84% Then do the two-disk algorithm again. 00:16:05.120 --> 00:16:07.310 align:middle line:90% So that's plus T of 2 again. 00:16:07.310 --> 00:16:10.690 align:middle line:90% So that's 3 plus 1 plus 3 is 7. 00:16:10.690 --> 00:16:13.870 align:middle line:90% Likewise, T of 4 is-- 00:16:13.870 --> 00:16:15.890 align:middle line:84% well, you do the three-disk algorithm. 00:16:15.890 --> 00:16:17.770 align:middle line:90% That's T of 3. 00:16:17.770 --> 00:16:18.950 align:middle line:90% We do one move. 00:16:18.950 --> 00:16:20.870 align:middle line:84% Then we do the three-disk algorithm again. 00:16:20.870 --> 00:16:22.130 align:middle line:90% That's another T of 3. 00:16:22.130 --> 00:16:27.060 align:middle line:84% So we're looking at 2T of 3 plus 1, which is 14-- 00:16:27.060 --> 00:16:29.380 align:middle line:90% 15. 00:16:29.380 --> 00:16:32.050 align:middle line:84% And now we've exactly described the number of moves 00:16:32.050 --> 00:16:35.270 align:middle line:90% that this algorithm takes. 00:16:35.270 --> 00:16:37.610 align:middle line:84% And just to really be clear about it, 00:16:37.610 --> 00:16:43.270 align:middle line:84% if we're trying to move disks 1 through n, so n disks together 00:16:43.270 --> 00:16:51.970 align:middle line:84% from peg A to peg C, we're going to first move the first n 00:16:51.970 --> 00:16:54.910 align:middle line:90% minus 1 disks from A to B. 00:16:54.910 --> 00:16:59.350 align:middle line:84% Then we're going to move disk n on its own from A to C, 00:16:59.350 --> 00:17:04.660 align:middle line:84% then the remaining n minus 1 disks from B to C-- 00:17:04.660 --> 00:17:08.589 align:middle line:84% exactly the same strategy we've just described. 00:17:08.589 --> 00:17:11.050 align:middle line:84% And this is going to be our general strategy. 00:17:11.050 --> 00:17:15.579 align:middle line:84% To move n disks, we do our n minus 1 disk algorithm, then 00:17:15.579 --> 00:17:18.550 align:middle line:84% a move, then our n minus 1 disk algorithm again. 00:17:18.550 --> 00:17:23.070 align:middle line:84% So now we've described an algorithm by induction. 00:17:23.070 --> 00:17:25.730 align:middle line:84% [WHISPERING] And secretly that's all recursion is. 00:17:25.730 --> 00:17:26.886 align:middle line:90% We're just inducting. 00:17:26.886 --> 00:17:29.440 align:middle line:90% 00:17:29.440 --> 00:17:32.980 align:middle line:84% But in general, the number of moves 00:17:32.980 --> 00:17:36.490 align:middle line:84% we take is, well, however many moves 00:17:36.490 --> 00:17:40.670 align:middle line:84% we needed to take for the n minus 1 disk algorithm plus 1 00:17:40.670 --> 00:17:42.420 align:middle line:90% plus Tn minus 1 again. 00:17:42.420 --> 00:17:47.960 align:middle line:90% So it's 2T of n minus 1 plus 1. 00:17:47.960 --> 00:17:49.380 align:middle line:90% And this is our sequence. 00:17:49.380 --> 00:17:50.550 align:middle line:90% This is our recurrence. 00:17:50.550 --> 00:17:53.970 align:middle line:84% And we're getting to the place where these are in the way. 00:17:53.970 --> 00:17:55.933 align:middle line:90% So let's move those. 00:17:55.933 --> 00:17:56.433 align:middle line:90% Success. 00:17:56.433 --> 00:17:59.300 align:middle line:90% 00:17:59.300 --> 00:18:01.020 align:middle line:84% OK, we've described our algorithm. 00:18:01.020 --> 00:18:02.310 align:middle line:90% We've described its runtime. 00:18:02.310 --> 00:18:04.220 align:middle line:84% But like I said earlier, closed form 00:18:04.220 --> 00:18:06.362 align:middle line:90% would be so much nicer, right? 00:18:06.362 --> 00:18:09.690 align:middle line:84% I want to know how long the universe takes with 64 disks. 00:18:09.690 --> 00:18:13.340 align:middle line:84% I don't want to have to compute 64 terms of my sequence. 00:18:13.340 --> 00:18:15.030 align:middle line:90% So what does this look like? 00:18:15.030 --> 00:18:17.120 align:middle line:90% Well, let's see. 00:18:17.120 --> 00:18:21.470 align:middle line:90% We have our numbers 1, 3, 7, 15. 00:18:21.470 --> 00:18:27.680 align:middle line:84% 2 times that plus 1 is 31, then 63, then 127. 00:18:27.680 --> 00:18:29.335 align:middle line:84% Is anyone starting to see a pattern? 00:18:29.335 --> 00:18:32.660 align:middle line:90% 00:18:32.660 --> 00:18:33.320 align:middle line:90% Yes? 00:18:33.320 --> 00:18:35.880 align:middle line:84% AUDIENCE: Isn't it just 2 to the n minus 1. 00:18:35.880 --> 00:18:39.230 align:middle line:90% ZACHARY ABEL: We have a guess. 00:18:39.230 --> 00:18:42.660 align:middle line:90% T of n is 2 to the n minus 1. 00:18:42.660 --> 00:18:48.140 align:middle line:84% This looks suspiciously like 2, 4, 8, 16, 32, 64, 128. 00:18:48.140 --> 00:18:51.480 align:middle line:84% And especially since we really are doubling every time, 00:18:51.480 --> 00:18:56.180 align:middle line:84% it seems likely that it might be related to powers of 2 somehow. 00:18:56.180 --> 00:18:58.010 align:middle line:84% Well, the good thing about having a guess 00:18:58.010 --> 00:18:59.730 align:middle line:84% is that we know what to do with it. 00:18:59.730 --> 00:19:00.990 align:middle line:90% We check it. 00:19:00.990 --> 00:19:04.650 align:middle line:84% We try to use it to prove this formula by induction. 00:19:04.650 --> 00:19:06.750 align:middle line:84% And if it works, we have the right formula. 00:19:06.750 --> 00:19:07.900 align:middle line:90% Turns out it works. 00:19:07.900 --> 00:19:15.350 align:middle line:90% 00:19:15.350 --> 00:19:19.310 align:middle line:84% You can prove it with a pretty straightforward induction. 00:19:19.310 --> 00:19:22.310 align:middle line:84% And it turns out this is, indeed, the correct formula. 00:19:22.310 --> 00:19:25.440 align:middle line:84% So this is a clear example of what 00:19:25.440 --> 00:19:26.690 align:middle line:90% we said about guess and check. 00:19:26.690 --> 00:19:30.080 align:middle line:84% If you can guess the formula, you check it with induction. 00:19:30.080 --> 00:19:32.150 align:middle line:84% But we had to recognize that pattern 00:19:32.150 --> 00:19:36.320 align:middle line:84% to be able to have something to check. 00:19:36.320 --> 00:19:38.690 align:middle line:84% But finally, we can answer our question-- how long 00:19:38.690 --> 00:19:41.570 align:middle line:90% does the universe have left? 00:19:41.570 --> 00:19:45.030 align:middle line:90% 64 disks-- so T of 64. 00:19:45.030 --> 00:19:49.940 align:middle line:84% That's 2 to the 64 minus 1 steps. 00:19:49.940 --> 00:19:52.920 align:middle line:84% Let's assume these monks are really strong 00:19:52.920 --> 00:19:56.030 align:middle line:84% and they can move one disk per second. 00:19:56.030 --> 00:20:02.210 align:middle line:84% Turns out, if I flip to the correct page in my notes, 00:20:02.210 --> 00:20:07.015 align:middle line:84% that many seconds is about a half trillion years. 00:20:07.015 --> 00:20:11.960 align:middle line:90% 00:20:11.960 --> 00:20:14.570 align:middle line:90% Universe has got some time left. 00:20:14.570 --> 00:20:17.760 align:middle line:84% And I find it surprising that with just 64 disks, 00:20:17.760 --> 00:20:21.080 align:middle line:84% you can get all the way up into the trillions of years. 00:20:21.080 --> 00:20:22.623 align:middle line:90% That's very quick growth. 00:20:22.623 --> 00:20:25.190 align:middle line:90% 00:20:25.190 --> 00:20:27.520 align:middle line:84% All right, any questions about Towers of Hanoi? 00:20:27.520 --> 00:20:30.740 align:middle line:90% 00:20:30.740 --> 00:20:31.495 align:middle line:90% Awesome. 00:20:31.495 --> 00:20:33.870 align:middle line:84% I thought the camera in the back was a hand for a second. 00:20:33.870 --> 00:20:35.990 align:middle line:90% I apologize for that. 00:20:35.990 --> 00:20:41.090 align:middle line:84% Let's move on to our next recursive algorithm. 00:20:41.090 --> 00:20:43.620 align:middle line:84% I promised last time that we would talk about sorting. 00:20:43.620 --> 00:20:45.980 align:middle line:90% So let's talk about sorting. 00:20:45.980 --> 00:20:47.810 align:middle line:84% And before we get to merge sort, which 00:20:47.810 --> 00:20:51.532 align:middle line:84% is the fast algorithm we're going to talk about today, 00:20:51.532 --> 00:20:53.990 align:middle line:84% let's talk about a simpler algorithm called selection sort. 00:20:53.990 --> 00:20:59.810 align:middle line:90% 00:20:59.810 --> 00:21:04.800 align:middle line:84% The problem is you have a list of n numbers. 00:21:04.800 --> 00:21:07.520 align:middle line:90% 00:21:07.520 --> 00:21:16.370 align:middle line:84% And you want to arrange them in increasing order. 00:21:16.370 --> 00:21:19.140 align:middle line:84% So your list is in whatever order it's in. 00:21:19.140 --> 00:21:22.050 align:middle line:84% And you want to figure out the correct increasing order. 00:21:22.050 --> 00:21:24.950 align:middle line:90% So put them in sorted order. 00:21:24.950 --> 00:21:31.850 align:middle line:84% And the idea for selection sort is a very simple one. 00:21:31.850 --> 00:21:35.312 align:middle line:90% Find the smallest. 00:21:35.312 --> 00:21:36.250 align:middle line:90% Pull it out. 00:21:36.250 --> 00:21:42.020 align:middle line:90% 00:21:42.020 --> 00:21:43.660 align:middle line:90% Repeat until done. 00:21:43.660 --> 00:21:46.430 align:middle line:90% 00:21:46.430 --> 00:21:48.390 align:middle line:90% So let's see this in action. 00:21:48.390 --> 00:21:49.970 align:middle line:90% And I have some helpers today. 00:21:49.970 --> 00:21:53.240 align:middle line:84% I'm going to need a few more volunteers. 00:21:53.240 --> 00:21:54.550 align:middle line:90% Thankfully, I brought some. 00:21:54.550 --> 00:22:03.200 align:middle line:90% 00:22:03.200 --> 00:22:04.100 align:middle line:90% Nice. 00:22:04.100 --> 00:22:09.770 align:middle line:84% All right, so we have a bunch of helper friends. 00:22:09.770 --> 00:22:14.960 align:middle line:84% And we need to somehow put them in sorted order. 00:22:14.960 --> 00:22:17.930 align:middle line:90% We're going by height here. 00:22:17.930 --> 00:22:22.470 align:middle line:84% And for selection sort, we said find the smallest. 00:22:22.470 --> 00:22:24.930 align:middle line:84% Well, clearly that's "P-Town" here. 00:22:24.930 --> 00:22:27.240 align:middle line:90% Pull them out, and then repeat. 00:22:27.240 --> 00:22:28.457 align:middle line:90% OK, who's the smallest left? 00:22:28.457 --> 00:22:29.790 align:middle line:90% Well, that's going to be "Wire." 00:22:29.790 --> 00:22:32.810 align:middle line:90% Who's the smallest left? 00:22:32.810 --> 00:22:34.532 align:middle line:84% Well, it's maybe a little less clear. 00:22:34.532 --> 00:22:35.990 align:middle line:84% But let's talk more precisely about 00:22:35.990 --> 00:22:39.020 align:middle line:84% how we're going to find the smallest every time. 00:22:39.020 --> 00:22:42.000 align:middle line:84% And we can do this with just a bunch of comparisons. 00:22:42.000 --> 00:22:43.820 align:middle line:90% So let's compare the first two. 00:22:43.820 --> 00:22:46.510 align:middle line:84% Between Ralph and "Wire," whose shorter? 00:22:46.510 --> 00:22:47.270 align:middle line:90% AUDIENCE: Wire. 00:22:47.270 --> 00:22:48.570 align:middle line:90% ZACHARY ABEL: "Wire," yeah, OK. 00:22:48.570 --> 00:22:51.170 align:middle line:84% So "Wire" is the smallest one we've seen so far. 00:22:51.170 --> 00:22:54.080 align:middle line:84% Between "Wire" and "P-Town," whose shorter? 00:22:54.080 --> 00:22:55.280 align:middle line:90% "P-Town." 00:22:55.280 --> 00:22:57.140 align:middle line:84% Between these two, whose shorter? 00:22:57.140 --> 00:22:58.730 align:middle line:90% Still "P-Town." 00:22:58.730 --> 00:23:00.150 align:middle line:90% Still "P-Town." 00:23:00.150 --> 00:23:01.410 align:middle line:90% Still "P-Town." 00:23:01.410 --> 00:23:03.060 align:middle line:90% Still "P-Town." 00:23:03.060 --> 00:23:05.030 align:middle line:84% So we got through the end of the list. 00:23:05.030 --> 00:23:06.990 align:middle line:84% "P-Town" is the smallest one we've seen. 00:23:06.990 --> 00:23:10.700 align:middle line:84% So "P-Town" is our smallest, softest, roundest boy. 00:23:10.700 --> 00:23:13.650 align:middle line:84% And that is how we are going to find the smallest. 00:23:13.650 --> 00:23:19.010 align:middle line:84% We can do that with just n minus 1 comparisons. 00:23:19.010 --> 00:23:20.970 align:middle line:90% We compared the first two. 00:23:20.970 --> 00:23:23.723 align:middle line:84% Then whoever won that, we compared to the third, 00:23:23.723 --> 00:23:25.640 align:middle line:84% then compared that to the fourth and the fifth 00:23:25.640 --> 00:23:28.392 align:middle line:84% and so on, so n minus 1 comparisons. 00:23:28.392 --> 00:23:30.850 align:middle line:84% And then you pull them out, and we're going to do it again. 00:23:30.850 --> 00:23:33.610 align:middle line:84% So now with n minus 2 comparisons, 00:23:33.610 --> 00:23:35.750 align:middle line:84% because we have one fewer animal up front, 00:23:35.750 --> 00:23:37.750 align:middle line:84% we're going to find that "Wire" is the smallest. 00:23:37.750 --> 00:23:39.270 align:middle line:90% Then we're going to find-- 00:23:39.270 --> 00:23:43.240 align:middle line:84% I think it's Bubs, is the smallest, then Totoro, 00:23:43.240 --> 00:23:48.090 align:middle line:84% then Ralph, then Coop-- get out of here-- 00:23:48.090 --> 00:23:49.890 align:middle line:90% then Butternut. 00:23:49.890 --> 00:23:52.030 align:middle line:90% And now we have our sorted list. 00:23:52.030 --> 00:23:56.460 align:middle line:84% How many comparisons did we have to do to implement that? 00:23:56.460 --> 00:23:58.320 align:middle line:84% Well, we've already said that we did 00:23:58.320 --> 00:24:00.373 align:middle line:84% n minus 1 comparisons in the first round. 00:24:00.373 --> 00:24:03.120 align:middle line:90% 00:24:03.120 --> 00:24:07.360 align:middle line:84% And then the second round, we had n minus 2 comparisons. 00:24:07.360 --> 00:24:12.480 align:middle line:84% The third round, n minus 3 comparisons, 00:24:12.480 --> 00:24:15.400 align:middle line:84% down to the last round, where we had just two animals left. 00:24:15.400 --> 00:24:18.130 align:middle line:84% We need one comparison to figure out who's shorter. 00:24:18.130 --> 00:24:19.960 align:middle line:84% And then the last one is just the last one. 00:24:19.960 --> 00:24:23.460 align:middle line:84% We don't need more comparisons for that. 00:24:23.460 --> 00:24:25.240 align:middle line:90% And we've seen this sum before. 00:24:25.240 --> 00:24:31.016 align:middle line:84% This is n minus 1 times n divided by 2 comparisons. 00:24:31.016 --> 00:24:34.020 align:middle line:90% 00:24:34.020 --> 00:24:39.700 align:middle line:84% So this is n squared over 2 plus change 00:24:39.700 --> 00:24:41.580 align:middle line:90% comparisons for selection sort. 00:24:41.580 --> 00:24:44.710 align:middle line:84% This is not the fast sort that was promised. 00:24:44.710 --> 00:24:47.520 align:middle line:90% This is still quadratic runtime. 00:24:47.520 --> 00:24:50.033 align:middle line:84% It is the easiest thing-- like when you have a small number 00:24:50.033 --> 00:24:51.700 align:middle line:84% of things in front of you and you just-- 00:24:51.700 --> 00:24:54.210 align:middle line:84% sorry for calling you "things"-- small number of animals in front 00:24:54.210 --> 00:24:54.710 align:middle line:90% of you-- 00:24:54.710 --> 00:24:55.230 align:middle line:90% [LAUGHTER] 00:24:55.230 --> 00:24:56.638 align:middle line:84% --and you just want to sort them, 00:24:56.638 --> 00:24:58.930 align:middle line:84% it's very easy to just scan through, find the smallest, 00:24:58.930 --> 00:24:59.680 align:middle line:90% find the smallest. 00:24:59.680 --> 00:25:01.000 align:middle line:90% That's what I do by hand. 00:25:01.000 --> 00:25:04.690 align:middle line:84% But for big lists, for lots and lots of numbers, 00:25:04.690 --> 00:25:07.620 align:middle line:84% this is not the way you want to go. 00:25:07.620 --> 00:25:11.260 align:middle line:90% But a much better algorithm-- 00:25:11.260 --> 00:25:15.420 align:middle line:84% not the only better algorithm, but a much better algorithm-- 00:25:15.420 --> 00:25:20.160 align:middle line:90% is pretty similar to this. 00:25:20.160 --> 00:25:22.530 align:middle line:84% And to talk about it, first let's 00:25:22.530 --> 00:25:26.130 align:middle line:90% talk about the idea of a merge. 00:25:26.130 --> 00:25:33.510 align:middle line:84% The idea is we have two sorted lists, A and B. 00:25:33.510 --> 00:25:39.163 align:middle line:84% And we want to combine them into a single sorted list. 00:25:39.163 --> 00:25:48.210 align:middle line:90% 00:25:48.210 --> 00:25:53.600 align:middle line:84% As an example, so maybe we have these, and that goes over there. 00:25:53.600 --> 00:25:56.910 align:middle line:90% 00:25:56.910 --> 00:25:58.680 align:middle line:90% There. 00:25:58.680 --> 00:25:59.610 align:middle line:90% Totoro, stay. 00:25:59.610 --> 00:26:00.990 align:middle line:90% There we go. 00:26:00.990 --> 00:26:04.950 align:middle line:84% So for example, if we have this sorted list, which is already 00:26:04.950 --> 00:26:08.310 align:middle line:84% in increasing order, and this separate sorted list that's 00:26:08.310 --> 00:26:11.070 align:middle line:84% already in increasing order, let's 00:26:11.070 --> 00:26:13.630 align:middle line:84% try to combine them into a single sorted list. 00:26:13.630 --> 00:26:16.200 align:middle line:84% And let's just do something like selection sort. 00:26:16.200 --> 00:26:18.390 align:middle line:84% Let's find the smallest and just keep 00:26:18.390 --> 00:26:20.850 align:middle line:84% finding the smallest until we have our answer. 00:26:20.850 --> 00:26:23.260 align:middle line:84% So who's the smallest between these two lists? 00:26:23.260 --> 00:26:24.545 align:middle line:90% Where do we have to look? 00:26:24.545 --> 00:26:28.620 align:middle line:90% 00:26:28.620 --> 00:26:29.380 align:middle line:90% I saw this. 00:26:29.380 --> 00:26:31.300 align:middle line:90% Was that an indication? 00:26:31.300 --> 00:26:32.488 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:26:32.488 --> 00:26:34.530 align:middle line:84% ZACHARY ABEL: All right, so the beginning of here 00:26:34.530 --> 00:26:35.980 align:middle line:90% or the beginning of here. 00:26:35.980 --> 00:26:38.430 align:middle line:84% Yeah, the smallest on the table is either 00:26:38.430 --> 00:26:40.140 align:middle line:84% going to be the smallest on this side 00:26:40.140 --> 00:26:42.280 align:middle line:90% or the smallest on this side. 00:26:42.280 --> 00:26:45.280 align:middle line:84% And we know where the smallest is because the lists are sorted. 00:26:45.280 --> 00:26:46.960 align:middle line:84% The smallest on this side is "Wire." 00:26:46.960 --> 00:26:48.820 align:middle line:84% The smallest on this side is "P-Town." 00:26:48.820 --> 00:26:50.460 align:middle line:84% We just have to look at the first 00:26:50.460 --> 00:26:52.840 align:middle line:84% from each of these two sorted lists. 00:26:52.840 --> 00:26:55.110 align:middle line:84% And with just one comparison, we know 00:26:55.110 --> 00:26:56.800 align:middle line:90% who the shortest is overall-- 00:26:56.800 --> 00:26:59.160 align:middle line:90% still "P-Town." 00:26:59.160 --> 00:27:01.080 align:middle line:90% And then we can continue-- 00:27:01.080 --> 00:27:03.820 align:middle line:84% first over here compared to the first over here. 00:27:03.820 --> 00:27:07.680 align:middle line:84% "Wire" versus Bubs, I think "Wire" is smaller. 00:27:07.680 --> 00:27:10.740 align:middle line:90% Now it's Bubs versus Totoro. 00:27:10.740 --> 00:27:13.770 align:middle line:84% And I think Bubs is smaller this time. 00:27:13.770 --> 00:27:16.000 align:middle line:90% Now it's Coop versus Totoro-- 00:27:16.000 --> 00:27:17.250 align:middle line:90% Totoro. 00:27:17.250 --> 00:27:19.440 align:middle line:90% Coop versus Ralph-- 00:27:19.440 --> 00:27:21.780 align:middle line:90% Ralph is shorter. 00:27:21.780 --> 00:27:23.370 align:middle line:84% And now we get to an interesting point 00:27:23.370 --> 00:27:25.900 align:middle line:84% where there is no first over here anymore. 00:27:25.900 --> 00:27:28.200 align:middle line:90% This side of the table is empty. 00:27:28.200 --> 00:27:30.910 align:middle line:84% Well, that just means that we're done. 00:27:30.910 --> 00:27:33.117 align:middle line:84% We know this side is already sorted and bigger 00:27:33.117 --> 00:27:34.450 align:middle line:90% than everything else we've seen. 00:27:34.450 --> 00:27:36.283 align:middle line:84% So we don't have to do any more comparisons. 00:27:36.283 --> 00:27:38.130 align:middle line:84% We just pull them back, and we're done. 00:27:38.130 --> 00:27:42.580 align:middle line:84% So that's the idea of merging two sorted lists. 00:27:42.580 --> 00:27:53.496 align:middle line:84% The algorithm says we compare smallest in each list, 00:27:53.496 --> 00:28:07.843 align:middle line:84% pull out the smallest, continue until one list becomes empty. 00:28:07.843 --> 00:28:12.300 align:middle line:90% 00:28:12.300 --> 00:28:17.120 align:middle line:84% And we put the other list at the end. 00:28:17.120 --> 00:28:20.630 align:middle line:90% 00:28:20.630 --> 00:28:23.650 align:middle line:84% So we got it down to where this list was empty. 00:28:23.650 --> 00:28:26.040 align:middle line:90% This list had two animals left. 00:28:26.040 --> 00:28:28.650 align:middle line:84% And we didn't have to do any more work. 00:28:28.650 --> 00:28:30.400 align:middle line:90% We're just done with those. 00:28:30.400 --> 00:28:32.610 align:middle line:90% Is this algorithm clear? 00:28:32.610 --> 00:28:35.580 align:middle line:90% Does it make sense why it works? 00:28:35.580 --> 00:28:39.690 align:middle line:84% How many comparisons does it take? 00:28:39.690 --> 00:28:44.460 align:middle line:84% Well, each round, every time we identify the next smallest 00:28:44.460 --> 00:28:48.450 align:middle line:84% animal, we're doing a single comparison. 00:28:48.450 --> 00:28:49.747 align:middle line:90% Yeah? 00:28:49.747 --> 00:28:51.330 align:middle line:84% In the worst case, we're going to have 00:28:51.330 --> 00:28:55.680 align:middle line:84% to do that for all of the first n minus 1 animals. 00:28:55.680 --> 00:29:00.190 align:middle line:84% If it comes down to one on the left versus one on the right, 00:29:00.190 --> 00:29:02.230 align:middle line:84% then everyone needed a comparison. 00:29:02.230 --> 00:29:05.700 align:middle line:84% We only got one freebie at the end. 00:29:05.700 --> 00:29:09.510 align:middle line:84% So worst case, this takes less than 00:29:09.510 --> 00:29:15.000 align:middle line:84% or equal to size of A plus size of B minus 1 comparisons. 00:29:15.000 --> 00:29:19.680 align:middle line:90% 00:29:19.680 --> 00:29:22.230 align:middle line:90% Sound good? 00:29:22.230 --> 00:29:23.590 align:middle line:90% Awesome. 00:29:23.590 --> 00:29:26.730 align:middle line:84% So now that we have this merge algorithm, let's 00:29:26.730 --> 00:29:28.250 align:middle line:90% talk about merge sort. 00:29:28.250 --> 00:29:32.220 align:middle line:90% 00:29:32.220 --> 00:29:39.480 align:middle line:84% So you're given a list with n elements. 00:29:39.480 --> 00:29:42.540 align:middle line:90% And we want to sort it. 00:29:42.540 --> 00:29:44.320 align:middle line:84% This list doesn't have any promises. 00:29:44.320 --> 00:29:46.960 align:middle line:84% It's not half sorted or anything like that. 00:29:46.960 --> 00:29:49.240 align:middle line:84% It's just some random list of random data. 00:29:49.240 --> 00:29:51.660 align:middle line:84% We want to put it in sorted order. 00:29:51.660 --> 00:29:59.670 align:middle line:84% And by the way, let's assume for now that n is a power of 2 00:29:59.670 --> 00:30:02.640 align:middle line:84% so we can just keep cutting in half and everything works 00:30:02.640 --> 00:30:03.460 align:middle line:90% out nicely. 00:30:03.460 --> 00:30:05.020 align:middle line:84% We'll remove that assumption later. 00:30:05.020 --> 00:30:06.853 align:middle line:84% But for now, we'll just solve this for lists 00:30:06.853 --> 00:30:09.720 align:middle line:90% whose lengths are powers of 2. 00:30:09.720 --> 00:30:17.400 align:middle line:84% So the first step, we are going to sort the first-- 00:30:17.400 --> 00:30:19.175 align:middle line:84% oh, sorry, I forgot an important step. 00:30:19.175 --> 00:30:22.630 align:middle line:90% 00:30:22.630 --> 00:30:26.050 align:middle line:90% If n equals 1, done. 00:30:26.050 --> 00:30:28.970 align:middle line:84% If you're sorting a list with just a single element, 00:30:28.970 --> 00:30:30.170 align:middle line:90% there's nothing to do. 00:30:30.170 --> 00:30:34.150 align:middle line:84% Your list is already sorted, so we'll call it done. 00:30:34.150 --> 00:30:41.260 align:middle line:84% Otherwise, we're going to the first n over 2 elements. 00:30:41.260 --> 00:30:45.130 align:middle line:90% Somehow we'll get them sorted. 00:30:45.130 --> 00:30:54.310 align:middle line:84% Then we will sort the last n over 2 elements somehow. 00:30:54.310 --> 00:30:55.695 align:middle line:84% And then you merge them together. 00:30:55.695 --> 00:30:59.050 align:middle line:90% 00:30:59.050 --> 00:31:01.960 align:middle line:84% So as long as we're somehow able to sort the first half 00:31:01.960 --> 00:31:04.510 align:middle line:84% and somehow able to sort the second half, 00:31:04.510 --> 00:31:07.600 align:middle line:84% the merge step will take care of the rest. 00:31:07.600 --> 00:31:10.150 align:middle line:90% Does this strategy make sense? 00:31:10.150 --> 00:31:10.762 align:middle line:90% Cool. 00:31:10.762 --> 00:31:12.220 align:middle line:84% And you might ask, well, how are we 00:31:12.220 --> 00:31:15.410 align:middle line:84% going to sort the first half and the last half? 00:31:15.410 --> 00:31:17.500 align:middle line:90% Maybe we'll use selection sort. 00:31:17.500 --> 00:31:19.280 align:middle line:84% Turns out, if you do that, you're still 00:31:19.280 --> 00:31:20.640 align:middle line:90% going to get quadratic runtime. 00:31:20.640 --> 00:31:23.660 align:middle line:90% That's not the best plan. 00:31:23.660 --> 00:31:25.820 align:middle line:84% The strategy I'm going to propose, 00:31:25.820 --> 00:31:29.180 align:middle line:84% and the way that merge sort is described, 00:31:29.180 --> 00:31:34.280 align:middle line:84% we're going to sort the first n over 2 elements using 00:31:34.280 --> 00:31:35.240 align:middle line:90% merge sort. 00:31:35.240 --> 00:31:37.970 align:middle line:90% 00:31:37.970 --> 00:31:43.640 align:middle line:84% And we're going to sort the last n over 2 elements using 00:31:43.640 --> 00:31:44.540 align:middle line:90% merge sort. 00:31:44.540 --> 00:31:47.930 align:middle line:90% 00:31:47.930 --> 00:31:51.930 align:middle line:84% OK, I claim I have written an algorithm here, 00:31:51.930 --> 00:31:54.590 align:middle line:90% though it might not be obvious. 00:31:54.590 --> 00:31:57.480 align:middle line:84% Recursion in general is one of those bogeyman topics. 00:31:57.480 --> 00:32:00.920 align:middle line:84% It's seen as the bane of all computer science students. 00:32:00.920 --> 00:32:02.640 align:middle line:84% 'Woe unto me.' 'The end is nigh.' 00:32:02.640 --> 00:32:06.140 align:middle line:90% It's the hardest-ever topic. 00:32:06.140 --> 00:32:08.900 align:middle line:84% And I am here to tell you it's not 00:32:08.900 --> 00:32:12.150 align:middle line:84% some new, complicated, scary topic. 00:32:12.150 --> 00:32:13.530 align:middle line:90% It's an old scary topic. 00:32:13.530 --> 00:32:16.220 align:middle line:90% No, it's an old familiar topic. 00:32:16.220 --> 00:32:17.422 align:middle line:90% It's just induction. 00:32:17.422 --> 00:32:18.880 align:middle line:84% I claim everything we're doing here 00:32:18.880 --> 00:32:20.810 align:middle line:90% is just going to be induction. 00:32:20.810 --> 00:32:23.740 align:middle line:84% And let's talk about what we mean by that. 00:32:23.740 --> 00:32:28.070 align:middle line:84% So let's run through merge sort and see what it looks like. 00:32:28.070 --> 00:32:33.370 align:middle line:84% So we have our scrambled list, which is a power of 2-- 00:32:33.370 --> 00:32:37.063 align:middle line:90% which is not a power of 2-- 00:32:37.063 --> 00:32:41.470 align:middle line:90% 00:32:41.470 --> 00:32:43.730 align:middle line:90% which is a power of 2. 00:32:43.730 --> 00:32:44.350 align:middle line:90% Whoops. 00:32:44.350 --> 00:32:46.555 align:middle line:90% [LAUGHTER] 00:32:46.555 --> 00:32:47.440 align:middle line:90% 00:32:47.440 --> 00:32:49.280 align:middle line:84% All right, last helper for the day. 00:32:49.280 --> 00:32:51.760 align:middle line:90% Thank you, Charlie. 00:32:51.760 --> 00:32:53.530 align:middle line:84% Now let's scramble this up and see 00:32:53.530 --> 00:32:56.030 align:middle line:90% if we can sort them together. 00:32:56.030 --> 00:32:59.290 align:middle line:90% So here's Butternut over here. 00:32:59.290 --> 00:33:02.110 align:middle line:90% All right. 00:33:02.110 --> 00:33:04.970 align:middle line:84% So let's run through our algorithm 00:33:04.970 --> 00:33:07.510 align:middle line:90% and see what it tells us to do. 00:33:07.510 --> 00:33:09.613 align:middle line:90% So merge sort on eight elements. 00:33:09.613 --> 00:33:12.670 align:middle line:90% 00:33:12.670 --> 00:33:15.670 align:middle line:84% So we're calling merge sort on eight elements. 00:33:15.670 --> 00:33:17.320 align:middle line:84% I'm going to abbreviate over here just 00:33:17.320 --> 00:33:20.140 align:middle line:90% to keep track of where we are. 00:33:20.140 --> 00:33:24.010 align:middle line:84% And it says the first thing to do is merge sort the first four 00:33:24.010 --> 00:33:26.950 align:middle line:84% elements, then merge sort the last four elements, 00:33:26.950 --> 00:33:28.690 align:middle line:90% then merge them together. 00:33:28.690 --> 00:33:32.680 align:middle line:84% So we're going to merge sort 1 through 4. 00:33:32.680 --> 00:33:36.580 align:middle line:84% And then we're going to merge sort 5 through 8. 00:33:36.580 --> 00:33:37.930 align:middle line:90% And then we're going to merge. 00:33:37.930 --> 00:33:40.790 align:middle line:90% 00:33:40.790 --> 00:33:42.710 align:middle line:84% That's what the algorithm tells us to do. 00:33:42.710 --> 00:33:48.040 align:middle line:84% So it tells us, take these four-- 00:33:48.040 --> 00:33:50.480 align:middle line:84% I did not do a good job scrambling these, did I? 00:33:50.480 --> 00:33:52.155 align:middle line:90% Let's do a little more. 00:33:52.155 --> 00:33:57.010 align:middle line:90% 00:33:57.010 --> 00:34:00.040 align:middle line:90% OK, we'll pretend we did that. 00:34:00.040 --> 00:34:03.045 align:middle line:84% So it tells us to merge sort the first four elements, 00:34:03.045 --> 00:34:04.670 align:middle line:84% then merge sort the last four elements, 00:34:04.670 --> 00:34:06.100 align:middle line:90% then merge them together. 00:34:06.100 --> 00:34:11.080 align:middle line:84% OK, well, here we're merge sorting the first four elements. 00:34:11.080 --> 00:34:13.270 align:middle line:84% What does our algorithm tell us to do when we're 00:34:13.270 --> 00:34:15.380 align:middle line:90% merge sorting four elements? 00:34:15.380 --> 00:34:21.050 align:middle line:84% Well, it says merge sort elements 1 through 2, 00:34:21.050 --> 00:34:23.540 align:middle line:84% then merge sort elements 3 through 4, 00:34:23.540 --> 00:34:26.230 align:middle line:90% then merge them together. 00:34:26.230 --> 00:34:27.760 align:middle line:90% Yeah? 00:34:27.760 --> 00:34:29.386 align:middle line:90% I'm just reading the code here. 00:34:29.386 --> 00:34:31.719 align:middle line:84% And that's what it tells me to do when I call merge sort 00:34:31.719 --> 00:34:33.820 align:middle line:90% on these four elements. 00:34:33.820 --> 00:34:38.270 align:middle line:84% So it tells me to call merge sort on these two, 00:34:38.270 --> 00:34:40.550 align:middle line:84% then call merge sort on these two, 00:34:40.550 --> 00:34:42.969 align:middle line:84% then merge these together into a single list-- 00:34:42.969 --> 00:34:46.300 align:middle line:84% a single sorted list of four elements. 00:34:46.300 --> 00:34:50.380 align:middle line:84% OK, well, what does merge sort with two elements mean? 00:34:50.380 --> 00:34:56.780 align:middle line:84% OK, sort the first one element, sort the next one element, 00:34:56.780 --> 00:34:57.860 align:middle line:90% then merge. 00:34:57.860 --> 00:35:00.100 align:middle line:90% Merge sort on two elements. 00:35:00.100 --> 00:35:03.290 align:middle line:84% Well, sorting 1 and 1 on their own. 00:35:03.290 --> 00:35:04.240 align:middle line:90% There's nothing to do. 00:35:04.240 --> 00:35:06.620 align:middle line:84% A list with one element is already sorted. 00:35:06.620 --> 00:35:08.270 align:middle line:84% So all I have to do is the merge. 00:35:08.270 --> 00:35:10.240 align:middle line:90% Merge this list with that list. 00:35:10.240 --> 00:35:12.200 align:middle line:90% And who's shorter. 00:35:12.200 --> 00:35:13.820 align:middle line:90% "P-Town," then Bubs. 00:35:13.820 --> 00:35:15.650 align:middle line:90% OK, now this list is sorted. 00:35:15.650 --> 00:35:18.520 align:middle line:84% Merge sort on these two elements next. 00:35:18.520 --> 00:35:20.620 align:middle line:84% Here's our-- you're kind of heavy. 00:35:20.620 --> 00:35:22.640 align:middle line:84% This one over here, this one over here. 00:35:22.640 --> 00:35:24.170 align:middle line:90% We get to do our merge. 00:35:24.170 --> 00:35:25.220 align:middle line:90% So who's shorter? 00:35:25.220 --> 00:35:26.210 align:middle line:90% It's Coop. 00:35:26.210 --> 00:35:28.090 align:middle line:90% Then it's Charlie. 00:35:28.090 --> 00:35:29.570 align:middle line:90% So where are we again? 00:35:29.570 --> 00:35:32.440 align:middle line:90% We've done this and this. 00:35:32.440 --> 00:35:33.890 align:middle line:90% Now we have to merge them. 00:35:33.890 --> 00:35:37.750 align:middle line:84% I'm just following what the code says to do. 00:35:37.750 --> 00:35:40.370 align:middle line:84% We have to merge this list with this list. 00:35:40.370 --> 00:35:43.570 align:middle line:84% OK, between these two, who's shorter? 00:35:43.570 --> 00:35:44.350 align:middle line:90% "P-Town." 00:35:44.350 --> 00:35:46.130 align:middle line:84% Between these two, who's shorter? 00:35:46.130 --> 00:35:47.020 align:middle line:90% Bubs. 00:35:47.020 --> 00:35:50.870 align:middle line:84% Now one list is empty, so the other list just falls in place. 00:35:50.870 --> 00:35:53.470 align:middle line:84% And now we have successfully called merge sort 00:35:53.470 --> 00:35:55.810 align:middle line:90% on these four elements. 00:35:55.810 --> 00:35:58.330 align:middle line:84% Let's do the same on the other side. 00:35:58.330 --> 00:36:01.150 align:middle line:90% So split into two and two. 00:36:01.150 --> 00:36:03.520 align:middle line:90% No, Butternut. 00:36:03.520 --> 00:36:05.180 align:middle line:90% Split into one and one. 00:36:05.180 --> 00:36:06.350 align:middle line:90% Merge them together. 00:36:06.350 --> 00:36:08.290 align:middle line:90% Totoro first. 00:36:08.290 --> 00:36:10.220 align:middle line:90% Split into one and one. 00:36:10.220 --> 00:36:11.600 align:middle line:90% Then merge them back together. 00:36:11.600 --> 00:36:13.160 align:middle line:90% "Wire," then Ralph. 00:36:13.160 --> 00:36:15.680 align:middle line:84% Now we have two verses two that we're merging. 00:36:15.680 --> 00:36:17.240 align:middle line:90% We have "Wire" versus Totoro. 00:36:17.240 --> 00:36:18.050 align:middle line:90% That's "Wire." 00:36:18.050 --> 00:36:19.610 align:middle line:90% We have Totoro versus Ralph. 00:36:19.610 --> 00:36:20.300 align:middle line:90% That's Totoro. 00:36:20.300 --> 00:36:21.650 align:middle line:90% Oops, there we go. 00:36:21.650 --> 00:36:23.120 align:middle line:90% Then Ralph versus Butternut. 00:36:23.120 --> 00:36:24.740 align:middle line:90% Ralph, then butternut. 00:36:24.740 --> 00:36:29.145 align:middle line:84% And now we have sorted these next four elements as well. 00:36:29.145 --> 00:36:31.925 align:middle line:90% 00:36:31.925 --> 00:36:33.550 align:middle line:84% And the last thing we're supposed to do 00:36:33.550 --> 00:36:36.610 align:middle line:84% is merge these four with those four. 00:36:36.610 --> 00:36:39.530 align:middle line:90% OK, we've got these two. 00:36:39.530 --> 00:36:40.090 align:middle line:90% It's this. 00:36:40.090 --> 00:36:41.180 align:middle line:90% We've got these two. 00:36:41.180 --> 00:36:42.180 align:middle line:90% It's this. 00:36:42.180 --> 00:36:44.300 align:middle line:90% These two, I think it's Bubs. 00:36:44.300 --> 00:36:45.610 align:middle line:90% These two, Totoro. 00:36:45.610 --> 00:36:47.660 align:middle line:90% These two-- what was it again? 00:36:47.660 --> 00:36:49.240 align:middle line:90% I think it's Ralph. 00:36:49.240 --> 00:36:54.880 align:middle line:84% Then these two is Coop, then Butternut, then Charlie. 00:36:54.880 --> 00:36:58.750 align:middle line:84% And so we just ran through merge sort on these eight elements 00:36:58.750 --> 00:37:01.240 align:middle line:90% just by following this code. 00:37:01.240 --> 00:37:02.720 align:middle line:90% And it kind of worked, right? 00:37:02.720 --> 00:37:05.500 align:middle line:84% We got our answer in the end, and it worked. 00:37:05.500 --> 00:37:09.370 align:middle line:84% But how can we be confident that it's going to work? 00:37:09.370 --> 00:37:11.420 align:middle line:84% More than that, how can we be confident 00:37:11.420 --> 00:37:15.650 align:middle line:84% that we've even described an algorithm? 00:37:15.650 --> 00:37:20.000 align:middle line:84% Because how can we describe the merge sort algorithm 00:37:20.000 --> 00:37:24.620 align:middle line:84% using merge sort before we finished describing merge sort? 00:37:24.620 --> 00:37:28.490 align:middle line:84% How can we know that this recursive call 00:37:28.490 --> 00:37:31.670 align:middle line:84% to merge sort correctly sorts our list before we've 00:37:31.670 --> 00:37:34.760 align:middle line:84% proven that merge sort correctly sorts your lists? 00:37:34.760 --> 00:37:37.130 align:middle line:84% This is the kind of circular thinking 00:37:37.130 --> 00:37:39.950 align:middle line:84% that makes recursion hard to think about. 00:37:39.950 --> 00:37:44.010 align:middle line:84% But thankfully we don't have to think about it in that way. 00:37:44.010 --> 00:37:46.650 align:middle line:84% We can think about it as just induction. 00:37:46.650 --> 00:37:52.190 align:middle line:84% We're inducting on the size of the lists that we're sorting. 00:37:52.190 --> 00:37:55.340 align:middle line:90% Merge sort on one element. 00:37:55.340 --> 00:37:57.660 align:middle line:84% We know what it does, and we know it works. 00:37:57.660 --> 00:37:58.860 align:middle line:90% It does nothing. 00:37:58.860 --> 00:38:00.530 align:middle line:90% And it's correct. 00:38:00.530 --> 00:38:03.840 align:middle line:84% If we're sorting one element, we're good. 00:38:03.840 --> 00:38:09.380 align:middle line:84% Merge sort on two elements says we're just doing that one 00:38:09.380 --> 00:38:11.330 align:middle line:90% comparison, like we saw. 00:38:11.330 --> 00:38:12.780 align:middle line:90% This also works. 00:38:12.780 --> 00:38:15.920 align:middle line:90% Merge sort on four elements-- 00:38:15.920 --> 00:38:18.570 align:middle line:84% to describe merge sort on four elements, 00:38:18.570 --> 00:38:20.843 align:middle line:84% we call merge sort with two elements, 00:38:20.843 --> 00:38:22.260 align:middle line:84% then merge sort with two elements, 00:38:22.260 --> 00:38:23.700 align:middle line:90% then merge them together. 00:38:23.700 --> 00:38:25.550 align:middle line:84% But by the time we're here, we already 00:38:25.550 --> 00:38:29.160 align:middle line:84% know what it means to merge sort two elements. 00:38:29.160 --> 00:38:32.030 align:middle line:84% And we know that it does the job correctly. 00:38:32.030 --> 00:38:35.630 align:middle line:84% So these recursive calls, don't think of them 00:38:35.630 --> 00:38:37.670 align:middle line:84% as calling back to merge sort, which we haven't 00:38:37.670 --> 00:38:39.090 align:middle line:90% finished describing yet. 00:38:39.090 --> 00:38:41.660 align:middle line:84% Think of them as calling a previous incarnation 00:38:41.660 --> 00:38:42.680 align:middle line:90% of merge sort. 00:38:42.680 --> 00:38:45.950 align:middle line:84% We're calling a smaller size, which we've already constructed, 00:38:45.950 --> 00:38:47.880 align:middle line:84% because we're building them up inductively, 00:38:47.880 --> 00:38:50.960 align:middle line:84% starting from the smaller sizes and working bigger. 00:38:50.960 --> 00:38:55.470 align:middle line:84% So we know that merge sort 2 exists and is correct. 00:38:55.470 --> 00:38:58.400 align:middle line:84% And we use that to prove that merge sort 4 00:38:58.400 --> 00:38:59.970 align:middle line:90% exists and is correct. 00:38:59.970 --> 00:39:04.280 align:middle line:84% We use that to prove that merge sort 8, which 00:39:04.280 --> 00:39:09.140 align:middle line:84% is using merge sort 4, both its description and its correctness, 00:39:09.140 --> 00:39:14.073 align:middle line:84% to make sure that merge sort 8 works and is correct. 00:39:14.073 --> 00:39:15.990 align:middle line:84% Once we know how to merge sort eight elements, 00:39:15.990 --> 00:39:18.480 align:middle line:84% we can use that to know how to merge sort 16 elements. 00:39:18.480 --> 00:39:20.790 align:middle line:84% We can use that to merge sort 32 elements. 00:39:20.790 --> 00:39:22.900 align:middle line:84% So all this circular reasoning, we 00:39:22.900 --> 00:39:25.130 align:middle line:84% can break that cycle by remembering 00:39:25.130 --> 00:39:27.440 align:middle line:84% that the recursive calls are actually 00:39:27.440 --> 00:39:29.570 align:middle line:90% your inductive hypothesis. 00:39:29.570 --> 00:39:34.700 align:middle line:84% We're allowed to assume that it works for eight elements when 00:39:34.700 --> 00:39:38.270 align:middle line:84% we're constructing and proving it for 16 elements, 00:39:38.270 --> 00:39:39.510 align:middle line:90% and so on and so forth. 00:39:39.510 --> 00:39:41.810 align:middle line:90% Does that idea make sense? 00:39:41.810 --> 00:39:43.620 align:middle line:90% Awesome. 00:39:43.620 --> 00:39:45.590 align:middle line:90% All right. 00:39:45.590 --> 00:39:49.280 align:middle line:84% It can be helpful to see it all at once and not 00:39:49.280 --> 00:39:51.980 align:middle line:90% with cute fuzzy helper friends. 00:39:51.980 --> 00:39:54.410 align:middle line:84% So here's a quick animation that's just running 00:39:54.410 --> 00:39:56.540 align:middle line:90% through the whole algorithm. 00:39:56.540 --> 00:39:58.680 align:middle line:84% And every time it has an unsorted list, 00:39:58.680 --> 00:40:00.390 align:middle line:84% it's breaking it into two halves. 00:40:00.390 --> 00:40:02.250 align:middle line:84% And then by the time they come back sorted, 00:40:02.250 --> 00:40:04.010 align:middle line:90% it's doing the merge. 00:40:04.010 --> 00:40:06.350 align:middle line:84% So we're about to merge 5, 7 with 1, 2. 00:40:06.350 --> 00:40:08.160 align:middle line:90% Looks like 1 is smaller than 5. 00:40:08.160 --> 00:40:12.380 align:middle line:84% Then 2, then 5, and 7 have no friends, so they go back in. 00:40:12.380 --> 00:40:15.320 align:middle line:84% Now we're going to call merge sort on the right half. 00:40:15.320 --> 00:40:18.500 align:middle line:84% Here we're merging one and one to get two. 00:40:18.500 --> 00:40:19.490 align:middle line:90% Now we're splitting. 00:40:19.490 --> 00:40:22.260 align:middle line:84% And now we're merging back 0 and 6. 00:40:22.260 --> 00:40:24.930 align:middle line:90% Now let's merge 3, 4 with 0, 6. 00:40:24.930 --> 00:40:27.150 align:middle line:90% Looks like 0 won, then 3. 00:40:27.150 --> 00:40:29.460 align:middle line:84% Then 4 is the winner, and then 6. 00:40:29.460 --> 00:40:31.970 align:middle line:84% And finally, we merge the two sorted halves 00:40:31.970 --> 00:40:35.450 align:middle line:84% with our familiar merge algorithm again. 00:40:35.450 --> 00:40:37.530 align:middle line:90% Next, 3 beats 5. 00:40:37.530 --> 00:40:39.180 align:middle line:90% 4 beats 5. 00:40:39.180 --> 00:40:42.500 align:middle line:90% 5 beats 6. 00:40:42.500 --> 00:40:44.690 align:middle line:90% Then 6, then 7. 00:40:44.690 --> 00:40:48.230 align:middle line:84% And that is a quick view of merge sort with just eight 00:40:48.230 --> 00:40:49.020 align:middle line:90% elements. 00:40:49.020 --> 00:40:51.430 align:middle line:84% Are there any questions on how this algorithm works? 00:40:51.430 --> 00:40:54.080 align:middle line:90% 00:40:54.080 --> 00:40:56.830 align:middle line:84% All right, still restricting to just powers of 2 for now. 00:40:56.830 --> 00:40:59.450 align:middle line:90% 00:40:59.450 --> 00:41:01.170 align:middle line:90% Well, how long does it take? 00:41:01.170 --> 00:41:02.275 align:middle line:90% We have an algorithm. 00:41:02.275 --> 00:41:02.983 align:middle line:90% I want a runtime. 00:41:02.983 --> 00:41:06.050 align:middle line:90% 00:41:06.050 --> 00:41:09.500 align:middle line:84% And just like before, we can describe our runtime 00:41:09.500 --> 00:41:12.770 align:middle line:90% with a recurrence. 00:41:12.770 --> 00:41:21.470 align:middle line:84% Let's call it M of n, which this is an upper bound 00:41:21.470 --> 00:41:33.115 align:middle line:84% on the number of comparisons when merge sorting n elements. 00:41:33.115 --> 00:41:36.110 align:middle line:90% 00:41:36.110 --> 00:41:39.090 align:middle line:84% And we're trying to figure out m of n for now 00:41:39.090 --> 00:41:41.870 align:middle line:90% just when n is a power of 2. 00:41:41.870 --> 00:41:43.530 align:middle line:90% And we have our algorithm. 00:41:43.530 --> 00:41:45.380 align:middle line:90% It's up here. 00:41:45.380 --> 00:41:48.170 align:middle line:84% The runtime for M of n, well, we're 00:41:48.170 --> 00:41:51.330 align:middle line:84% calling the algorithm for n over 2. 00:41:51.330 --> 00:41:53.010 align:middle line:90% And we're calling it twice. 00:41:53.010 --> 00:41:56.540 align:middle line:90% So 2 times M of n over 2. 00:41:56.540 --> 00:42:01.050 align:middle line:84% And then we're merging n over 2 elements with n over 2 elements. 00:42:01.050 --> 00:42:07.430 align:middle line:84% And that merge takes at most n minus 1 more comparisons. 00:42:07.430 --> 00:42:10.770 align:middle line:84% And so M, which is described by this recurrence, 00:42:10.770 --> 00:42:14.720 align:middle line:84% is an upper bound on the number of comparisons we'll need. 00:42:14.720 --> 00:42:16.850 align:middle line:84% Does this make sense, how we were 00:42:16.850 --> 00:42:21.710 align:middle line:84% able to translate this recursive algorithm into this recurrence 00:42:21.710 --> 00:42:23.870 align:middle line:90% for a bound on the runtime? 00:42:23.870 --> 00:42:27.090 align:middle line:84% This isn't the exact runtime because, like we said, 00:42:27.090 --> 00:42:30.470 align:middle line:84% merge sometimes takes fewer comparisons. 00:42:30.470 --> 00:42:34.380 align:middle line:84% If one of the lists is entirely smaller than the other lists, 00:42:34.380 --> 00:42:36.860 align:middle line:84% then you only need n over 2 comparisons. 00:42:36.860 --> 00:42:39.600 align:middle line:84% And then the other list falls into place for free. 00:42:39.600 --> 00:42:42.650 align:middle line:84% So it's going to be somewhere between n over 2 comparisons 00:42:42.650 --> 00:42:44.280 align:middle line:90% and n minus 1 comparisons. 00:42:44.280 --> 00:42:46.120 align:middle line:84% Worst case, it's going to be n minus 1. 00:42:46.120 --> 00:42:48.120 align:middle line:84% And since we're looking for an upper bound here, 00:42:48.120 --> 00:42:49.910 align:middle line:90% we'll call it n minus 1. 00:42:49.910 --> 00:42:52.670 align:middle line:90% OK, well, we have a recurrence. 00:42:52.670 --> 00:42:56.880 align:middle line:84% We can compute some terms and aim for a closed form, 00:42:56.880 --> 00:42:59.060 align:middle line:84% see if we can get an answer there. 00:42:59.060 --> 00:43:01.350 align:middle line:84% And let me write down some terms for us. 00:43:01.350 --> 00:43:02.840 align:middle line:90% Here's n. 00:43:02.840 --> 00:43:05.030 align:middle line:90% Here's selection sort. 00:43:05.030 --> 00:43:08.560 align:middle line:84% And here's merge sort, just for comparison. 00:43:08.560 --> 00:43:12.425 align:middle line:90% 00:43:12.425 --> 00:43:18.650 align:middle line:90% n is 1, 2, 4, 8, 16, and 32. 00:43:18.650 --> 00:43:21.620 align:middle line:84% That should be enough to find a pattern, right? 00:43:21.620 --> 00:43:25.790 align:middle line:84% So for selection sort-- that was the slow quadratic one-- 00:43:25.790 --> 00:43:32.970 align:middle line:84% we had 0, 1, 6, 28, 120, and 496. 00:43:32.970 --> 00:43:34.920 align:middle line:84% And we already know the formula for this. 00:43:34.920 --> 00:43:40.760 align:middle line:84% That's n squared minus n divided by 2. 00:43:40.760 --> 00:43:51.380 align:middle line:84% For merge sort, we get 0, 1, 5, 17, 49, 129. 00:43:51.380 --> 00:43:53.840 align:middle line:84% And we don't know this formula yet. 00:43:53.840 --> 00:43:57.530 align:middle line:84% Gratifying to see that it's smaller than selection sort. 00:43:57.530 --> 00:44:00.620 align:middle line:84% So this looks like an improvement on our algorithm. 00:44:00.620 --> 00:44:02.830 align:middle line:84% Can everyone see, or is Charlie in the way? 00:44:02.830 --> 00:44:03.860 align:middle line:90% Sorry about this. 00:44:03.860 --> 00:44:08.910 align:middle line:90% 00:44:08.910 --> 00:44:12.100 align:middle line:84% But looking at that, I don't really see the pattern. 00:44:12.100 --> 00:44:15.180 align:middle line:84% I can't think of a formula that gets me those numbers 00:44:15.180 --> 00:44:17.760 align:middle line:84% just by staring at them, which brings us 00:44:17.760 --> 00:44:19.980 align:middle line:84% to our other technique for trying 00:44:19.980 --> 00:44:23.490 align:middle line:84% to find a closed form when we don't know one yet. 00:44:23.490 --> 00:44:26.820 align:middle line:84% Just like with sums, we had the perturbation method 00:44:26.820 --> 00:44:29.820 align:middle line:84% when we were trying to figure out a closed form. 00:44:29.820 --> 00:44:35.490 align:middle line:84% Well, here, with recurrences, we have what we call "plug 00:44:35.490 --> 00:44:37.860 align:middle line:90% and chug"-- 00:44:37.860 --> 00:44:40.300 align:middle line:84% not to be confused with "guess and check." 00:44:40.300 --> 00:44:43.920 align:middle line:84% Whoever named these, I don't like them-- 00:44:43.920 --> 00:44:45.340 align:middle line:90% the names, not the people. 00:44:45.340 --> 00:44:48.600 align:middle line:90% They're fine, I assume. 00:44:48.600 --> 00:44:50.740 align:middle line:84% We have the plug and chug technique, 00:44:50.740 --> 00:44:52.690 align:middle line:90% which looks like the following. 00:44:52.690 --> 00:45:07.630 align:middle line:84% So roughly speaking, substitute the recurrence into itself. 00:45:07.630 --> 00:45:12.420 align:middle line:84% So we get expanded and expanded versions of the formula. 00:45:12.420 --> 00:45:15.990 align:middle line:84% And look for a pattern in that expansion. 00:45:15.990 --> 00:45:18.250 align:middle line:84% Let's see if we can do that with merge sort. 00:45:18.250 --> 00:45:21.570 align:middle line:90% 00:45:21.570 --> 00:45:24.250 align:middle line:90% So we have M of n. 00:45:24.250 --> 00:45:27.700 align:middle line:84% We know I'm going to put the n minus 1 first. 00:45:27.700 --> 00:45:32.830 align:middle line:84% This is n minus 1 plus 2 times M of n over 2. 00:45:32.830 --> 00:45:34.950 align:middle line:84% That's just our recurrence exactly 00:45:34.950 --> 00:45:38.100 align:middle line:90% as it was described before. 00:45:38.100 --> 00:45:43.480 align:middle line:84% But this M of n over 2, M still satisfies our recurrence. 00:45:43.480 --> 00:45:46.500 align:middle line:84% So let's apply this recurrence to n over 2 00:45:46.500 --> 00:45:50.860 align:middle line:84% and replace this term with the recurrence version of that. 00:45:50.860 --> 00:45:58.260 align:middle line:84% So M over n is going to equal n minus 1 plus 2 times n 00:45:58.260 --> 00:46:04.530 align:middle line:84% over 2 minus 1 plus 2 times M of n over 4. 00:46:04.530 --> 00:46:07.860 align:middle line:84% So I applied our recurrence to the input of n over 2 00:46:07.860 --> 00:46:10.170 align:middle line:90% and substituted that in here. 00:46:10.170 --> 00:46:12.960 align:middle line:84% Are we clear on how I got from here to here? 00:46:12.960 --> 00:46:15.840 align:middle line:90% This is the key step. 00:46:15.840 --> 00:46:19.020 align:middle line:84% We can expand this out a little bit. 00:46:19.020 --> 00:46:21.250 align:middle line:90% So this is n minus 1. 00:46:21.250 --> 00:46:24.670 align:middle line:84% This is-- 2 times that is going to be n minus 2. 00:46:24.670 --> 00:46:31.020 align:middle line:84% And then we have a 2 and 2 plus 4 M of n over 4. 00:46:31.020 --> 00:46:33.430 align:middle line:84% All right, so we expanded it out one step. 00:46:33.430 --> 00:46:35.700 align:middle line:84% And now we have this recurrence, which still 00:46:35.700 --> 00:46:39.270 align:middle line:90% describes the same sequence. 00:46:39.270 --> 00:46:40.540 align:middle line:90% And we can do it again. 00:46:40.540 --> 00:46:43.200 align:middle line:90% Let's do it again. 00:46:43.200 --> 00:46:52.080 align:middle line:84% This is n minus 1 plus n minus 2 plus 4 times n over 4 minus 1 00:46:52.080 --> 00:46:57.570 align:middle line:90% plus 2 times M of n over 8. 00:46:57.570 --> 00:46:59.200 align:middle line:90% Pretty sure I did that right. 00:46:59.200 --> 00:47:02.580 align:middle line:84% We can expand that out a little bit. 00:47:02.580 --> 00:47:09.580 align:middle line:84% So M of n is still equal to n minus 1 plus n minus 2 plus-- 00:47:09.580 --> 00:47:11.910 align:middle line:84% and now we have 4 times this, which 00:47:11.910 --> 00:47:14.820 align:middle line:90% is going to give us n minus 4. 00:47:14.820 --> 00:47:17.020 align:middle line:90% Plus, there's a 4 and 2. 00:47:17.020 --> 00:47:20.030 align:middle line:90% We have 8 times M of n over 8. 00:47:20.030 --> 00:47:25.950 align:middle line:90% 00:47:25.950 --> 00:47:29.040 align:middle line:84% And if we keep doing this, we can 00:47:29.040 --> 00:47:31.770 align:middle line:84% guess at the pattern we're going to see when we expand this 00:47:31.770 --> 00:47:33.010 align:middle line:90% further and further. 00:47:33.010 --> 00:47:38.910 align:middle line:84% We can guess that M of n is going to look like n minus 1 00:47:38.910 --> 00:47:49.620 align:middle line:84% plus n minus 2 plus n minus 4 plus n minus 8 plus n minus 2 00:47:49.620 --> 00:48:02.190 align:middle line:84% to the t minus 1 plus 2 to the t times M of n over 2 to the t. 00:48:02.190 --> 00:48:05.850 align:middle line:84% If we expand this many times, it looks like we're 00:48:05.850 --> 00:48:08.460 align:middle line:90% going to end up with this. 00:48:08.460 --> 00:48:11.550 align:middle line:84% This was just me guessing at the formula we're going to get, 00:48:11.550 --> 00:48:16.710 align:middle line:84% but informed guessing by how this keeps expanding. 00:48:16.710 --> 00:48:21.030 align:middle line:84% And remember, we're already assuming that n is a power of 2. 00:48:21.030 --> 00:48:24.120 align:middle line:84% We're assuming that n is 2 to the k. 00:48:24.120 --> 00:48:29.610 align:middle line:84% So what happens if we choose t equals k? 00:48:29.610 --> 00:48:40.500 align:middle line:84% Then M of n would equal n minus 1 plus n minus 2 plus n minus 4 00:48:40.500 --> 00:48:48.300 align:middle line:84% plus n minus 2 to the k minus 1 plus 2 to the k M of n over 2 00:48:48.300 --> 00:48:51.060 align:middle line:90% to the k. 00:48:51.060 --> 00:48:54.480 align:middle line:90% OK, but 2 to the k is n. 00:48:54.480 --> 00:48:57.510 align:middle line:90% And n over 2 to the k is 1. 00:48:57.510 --> 00:49:01.520 align:middle line:84% So this term here is n times M of 1. 00:49:01.520 --> 00:49:04.680 align:middle line:90% 00:49:04.680 --> 00:49:07.200 align:middle line:90% Yeah? 00:49:07.200 --> 00:49:10.400 align:middle line:84% All right, let me make sure I'm still matching my algebra. 00:49:10.400 --> 00:49:13.680 align:middle line:90% 00:49:13.680 --> 00:49:18.330 align:middle line:84% And then, from here, we have a sum to deal with. 00:49:18.330 --> 00:49:21.760 align:middle line:84% Let's take a look at all of these n terms. 00:49:21.760 --> 00:49:23.140 align:middle line:90% How many are there? 00:49:23.140 --> 00:49:26.342 align:middle line:84% Well, it's 2 to the 0 up to 2 to the k minus 1. 00:49:26.342 --> 00:49:27.550 align:middle line:90% So there should be k of them. 00:49:27.550 --> 00:49:28.050 align:middle line:90% Yes? 00:49:28.050 --> 00:49:30.510 align:middle line:84% AUDIENCE: Why is that n times M of 1 00:49:30.510 --> 00:49:33.030 align:middle line:84% instead of erase the k [INAUDIBLE] by 1? 00:49:33.030 --> 00:49:34.425 align:middle line:90% ZACHARY ABEL: Instead of what? 00:49:34.425 --> 00:49:36.432 align:middle line:84% AUDIENCE: Erase the k [INAUDIBLE] 00:49:36.432 --> 00:49:37.510 align:middle line:90% ZACHARY ABEL: 2 to the k? 00:49:37.510 --> 00:49:40.110 align:middle line:90% So 2 to the k is n. 00:49:40.110 --> 00:49:41.940 align:middle line:84% We are assuming that n is a power of 2 00:49:41.940 --> 00:49:43.710 align:middle line:90% and k is that exponent. 00:49:43.710 --> 00:49:46.060 align:middle line:90% So these two are the same term. 00:49:46.060 --> 00:49:47.430 align:middle line:90% Good question. 00:49:47.430 --> 00:49:48.520 align:middle line:90% So let's see what we got. 00:49:48.520 --> 00:49:49.720 align:middle line:90% How many n's did we have. 00:49:49.720 --> 00:49:52.350 align:middle line:90% We have k of them. 00:49:52.350 --> 00:49:54.480 align:middle line:90% This is k times n. 00:49:54.480 --> 00:49:59.100 align:middle line:84% Now let's deal with these parts that are being subtracted. 00:49:59.100 --> 00:50:06.750 align:middle line:84% Minus 1 plus 2 plus 4 plus up to 2 to the k minus 1. 00:50:06.750 --> 00:50:09.485 align:middle line:84% And then this term here, what was M of 1 again? 00:50:09.485 --> 00:50:12.090 align:middle line:90% 00:50:12.090 --> 00:50:12.810 align:middle line:90% Yeah? 00:50:12.810 --> 00:50:14.320 align:middle line:90% I see someone going like this. 00:50:14.320 --> 00:50:14.820 align:middle line:90% Excellent. 00:50:14.820 --> 00:50:16.980 align:middle line:90% 0, yeah. 00:50:16.980 --> 00:50:19.710 align:middle line:84% There's nothing to do when we have just one element. 00:50:19.710 --> 00:50:20.590 align:middle line:90% M of 1 is 0. 00:50:20.590 --> 00:50:22.980 align:middle line:84% So that whole term, in fact, disappears. 00:50:22.980 --> 00:50:24.480 align:middle line:84% And we're just left with this, which 00:50:24.480 --> 00:50:27.270 align:middle line:90% has kn and a geometric series. 00:50:27.270 --> 00:50:29.920 align:middle line:84% And we know formulas for geometric series. 00:50:29.920 --> 00:50:32.880 align:middle line:90% I'll spare us the details. 00:50:32.880 --> 00:50:41.730 align:middle line:84% This is going to be equal to kn minus 2 to the k minus 1. 00:50:41.730 --> 00:50:46.380 align:middle line:84% And if we remember what k is, k is log 2 of n. 00:50:46.380 --> 00:50:50.200 align:middle line:90% This is n times log 2 of n. 00:50:50.200 --> 00:50:51.870 align:middle line:90% That's this term. 00:50:51.870 --> 00:50:54.680 align:middle line:84% And then this 2 to the k is just n. 00:50:54.680 --> 00:51:00.160 align:middle line:84% So minus n plus 1 is our formula. 00:51:00.160 --> 00:51:02.140 align:middle line:84% And we got that not by looking at the numbers 00:51:02.140 --> 00:51:04.382 align:middle line:84% but by expanding out the recurrence 00:51:04.382 --> 00:51:05.715 align:middle line:90% and looking for a pattern there. 00:51:05.715 --> 00:51:08.320 align:middle line:90% 00:51:08.320 --> 00:51:10.400 align:middle line:84% Of course, this is still just a guess. 00:51:10.400 --> 00:51:14.060 align:middle line:84% We didn't really explain why this thing is true. 00:51:14.060 --> 00:51:16.250 align:middle line:84% We would maybe want to prove that by induction. 00:51:16.250 --> 00:51:19.700 align:middle line:84% But easier option-- once we have our actual guess, 00:51:19.700 --> 00:51:21.440 align:middle line:84% let's just prove this by induction. 00:51:21.440 --> 00:51:23.860 align:middle line:84% Once we have our guess, we can check it. 00:51:23.860 --> 00:51:28.000 align:middle line:84% And it turns out this is the correct formula. 00:51:28.000 --> 00:51:30.550 align:middle line:90% There's your check. 00:51:30.550 --> 00:51:32.440 align:middle line:90% Questions on that? 00:51:32.440 --> 00:51:34.070 align:middle line:90% Plug and chug method-- 00:51:34.070 --> 00:51:35.380 align:middle line:90% really useful method. 00:51:35.380 --> 00:51:39.230 align:middle line:84% Doesn't always bear fruit, but often does. 00:51:39.230 --> 00:51:41.780 align:middle line:84% And it's a nice tool to have in your back pocket. 00:51:41.780 --> 00:51:44.270 align:middle line:84% One bit of advice when using plug and chug, by the way, 00:51:44.270 --> 00:51:47.300 align:middle line:84% is notice when I had this n minus 1 and n minus 2, 00:51:47.300 --> 00:51:48.590 align:middle line:90% I didn't combine them. 00:51:48.590 --> 00:51:51.300 align:middle line:84% I could have written that as 2n minus 3. 00:51:51.300 --> 00:51:55.790 align:middle line:84% And then the next one would have been, what, 3n minus 7? 00:51:55.790 --> 00:51:58.800 align:middle line:84% But maybe the pattern is less clear if we did that. 00:51:58.800 --> 00:52:02.150 align:middle line:84% I tried to keep terms from the different levels of recursion 00:52:02.150 --> 00:52:04.520 align:middle line:84% separate from each other so I can keep track of what's 00:52:04.520 --> 00:52:06.710 align:middle line:90% coming from each stage. 00:52:06.710 --> 00:52:08.790 align:middle line:84% So when you're doing plug and chug, 00:52:08.790 --> 00:52:12.410 align:middle line:84% try not to simplify everything at the beginning. 00:52:12.410 --> 00:52:16.820 align:middle line:84% It might help you see the pattern more readily. 00:52:16.820 --> 00:52:21.920 align:middle line:90% All right, back to merge sort. 00:52:21.920 --> 00:52:26.515 align:middle line:84% Finally, what if n isn't a power of 2? 00:52:26.515 --> 00:52:39.740 align:middle line:90% 00:52:39.740 --> 00:52:43.860 align:middle line:84% I claim the algorithm works pretty much as described. 00:52:43.860 --> 00:52:47.030 align:middle line:84% We don't really need to change it all that much. 00:52:47.030 --> 00:52:50.440 align:middle line:84% At least we don't have to change the idea, just the measurements. 00:52:50.440 --> 00:52:59.290 align:middle line:84% So we're going to sort the first n over 2 floor elements. 00:52:59.290 --> 00:53:04.693 align:middle line:84% We're going to sort the last ceiling of n over 2 elements. 00:53:04.693 --> 00:53:08.920 align:middle line:90% 00:53:08.920 --> 00:53:10.365 align:middle line:90% And then merge them together. 00:53:10.365 --> 00:53:14.000 align:middle line:90% 00:53:14.000 --> 00:53:17.500 align:middle line:84% So this notation, floor notation, this means n over 2 00:53:17.500 --> 00:53:19.870 align:middle line:84% rounded down to the nearest integer. 00:53:19.870 --> 00:53:24.460 align:middle line:84% So if n was 11, we would be sorting the first five elements 00:53:24.460 --> 00:53:27.230 align:middle line:84% and then sorting the last six elements. 00:53:27.230 --> 00:53:30.980 align:middle line:84% And we'd again have two sorted lists, now with different sizes. 00:53:30.980 --> 00:53:32.060 align:middle line:90% But that doesn't matter. 00:53:32.060 --> 00:53:34.310 align:middle line:84% Merge doesn't care what size the lists are. 00:53:34.310 --> 00:53:35.810 align:middle line:90% We can merge them together. 00:53:35.810 --> 00:53:38.740 align:middle line:84% And this is our actual implementation of merge sort 00:53:38.740 --> 00:53:41.650 align:middle line:90% no matter what the size is. 00:53:41.650 --> 00:53:44.890 align:middle line:84% And we can similarly bound its runtime. 00:53:44.890 --> 00:53:49.420 align:middle line:84% Now m of n, the number of comparisons 00:53:49.420 --> 00:53:51.940 align:middle line:90% when merge sorting n elements-- 00:53:51.940 --> 00:53:55.720 align:middle line:84% or at least an upper bound on the number of comparisons-- 00:53:55.720 --> 00:53:58.630 align:middle line:84% well, now we're recursively calling floor of n over 2 00:53:58.630 --> 00:54:00.550 align:middle line:84% and ceiling of n over 2, and then 00:54:00.550 --> 00:54:04.090 align:middle line:84% still doing a merge with n total elements. 00:54:04.090 --> 00:54:07.720 align:middle line:84% We can be conservative and say the number of comparisons 00:54:07.720 --> 00:54:14.380 align:middle line:84% is at most 2 times the amount of comparisons 00:54:14.380 --> 00:54:18.400 align:middle line:84% it takes to sort the bigger of the two halves. 00:54:18.400 --> 00:54:19.960 align:middle line:84% If the other half is smaller, then 00:54:19.960 --> 00:54:22.840 align:middle line:84% presumably that's even fewer comparisons-- 00:54:22.840 --> 00:54:29.740 align:middle line:84% so this recurrence and then, still, plus n minus 1 00:54:29.740 --> 00:54:32.050 align:middle line:90% for the merge at most. 00:54:32.050 --> 00:54:34.420 align:middle line:84% So this is a conservative measurement 00:54:34.420 --> 00:54:36.880 align:middle line:84% of how many comparisons we need in the worst 00:54:36.880 --> 00:54:39.160 align:middle line:90% case for n elements. 00:54:39.160 --> 00:54:41.890 align:middle line:90% Are we OK with how we got here? 00:54:41.890 --> 00:54:43.390 align:middle line:90% Nice. 00:54:43.390 --> 00:54:46.100 align:middle line:84% And now we could try the same thing. 00:54:46.100 --> 00:54:48.490 align:middle line:84% We could try to plug and chug or something 00:54:48.490 --> 00:54:49.990 align:middle line:84% or guess and check and see if we can 00:54:49.990 --> 00:54:52.690 align:middle line:84% get a closed form for this recurrence here. 00:54:52.690 --> 00:54:57.010 align:middle line:84% I don't want to, especially the floors and-- 00:54:57.010 --> 00:54:58.490 align:middle line:90% or the ceilings, rather. 00:54:58.490 --> 00:54:59.870 align:middle line:90% They get in the way. 00:54:59.870 --> 00:55:03.790 align:middle line:84% They make it a lot harder to do something like that. 00:55:03.790 --> 00:55:08.290 align:middle line:84% Thankfully, we have yet another tool. 00:55:08.290 --> 00:55:12.700 align:middle line:84% Turns out algorithms like merge sort, 00:55:12.700 --> 00:55:18.760 align:middle line:84% more generally what we'll call "divide and conquer" 00:55:18.760 --> 00:55:20.290 align:middle line:90% recurrences-- 00:55:20.290 --> 00:55:26.560 align:middle line:84% sorry, algorithms-- there are lots of divide 00:55:26.560 --> 00:55:30.100 align:middle line:84% and conquer algorithms very similar 00:55:30.100 --> 00:55:34.630 align:middle line:84% to merge sort that give runtime recurrences very 00:55:34.630 --> 00:55:36.250 align:middle line:90% similar to merge sort. 00:55:36.250 --> 00:55:40.120 align:middle line:84% "Divide and conquer" means that usually whatever set of data you 00:55:40.120 --> 00:55:42.428 align:middle line:84% have, you're dividing it into smaller groups. 00:55:42.428 --> 00:55:43.220 align:middle line:90% There might be two. 00:55:43.220 --> 00:55:45.070 align:middle line:90% There might be multiple. 00:55:45.070 --> 00:55:47.930 align:middle line:84% You're recursively calling your algorithm on each group, 00:55:47.930 --> 00:55:48.760 align:middle line:90% potentially. 00:55:48.760 --> 00:55:50.570 align:middle line:90% Possibly your group's overlap. 00:55:50.570 --> 00:55:53.920 align:middle line:84% So you need more recursive calls. 00:55:53.920 --> 00:55:56.200 align:middle line:84% And then, after you get information 00:55:56.200 --> 00:55:59.080 align:middle line:84% about each of your subsets, you somehow 00:55:59.080 --> 00:56:02.470 align:middle line:84% combine them back together to get your answer 00:56:02.470 --> 00:56:04.060 align:middle line:90% for the full list. 00:56:04.060 --> 00:56:06.320 align:middle line:84% So merge sort is a good example there. 00:56:06.320 --> 00:56:09.400 align:middle line:84% You'll see some others in recitation and problem set 00:56:09.400 --> 00:56:12.020 align:middle line:84% for other divide and conquer algorithms. 00:56:12.020 --> 00:56:16.330 align:middle line:84% We're not so concerned with algorithm design in this class. 00:56:16.330 --> 00:56:20.170 align:middle line:84% But tools that will be used for designing algorithms 00:56:20.170 --> 00:56:23.170 align:middle line:84% in the future are very appropriate. 00:56:23.170 --> 00:56:26.390 align:middle line:84% And I'm singling out divide and conquer algorithms 00:56:26.390 --> 00:56:28.930 align:middle line:84% because we have a really nice tool that 00:56:28.930 --> 00:56:32.020 align:middle line:84% helps us analyze the runtime of divide 00:56:32.020 --> 00:56:34.090 align:middle line:84% and conquer algorithms like this. 00:56:34.090 --> 00:56:35.580 align:middle line:90% It's called the master theorem. 00:56:35.580 --> 00:56:46.370 align:middle line:90% 00:56:46.370 --> 00:56:49.340 align:middle line:84% It's called the master theorem, and it looks like that. 00:56:49.340 --> 00:56:53.180 align:middle line:84% It's a lot of text, so I didn't want to write it by hand. 00:56:53.180 --> 00:56:55.910 align:middle line:90% What the heck is going on here? 00:56:55.910 --> 00:56:57.570 align:middle line:84% Well, the point is the following. 00:56:57.570 --> 00:57:00.080 align:middle line:90% Suppose we have a recurrence. 00:57:00.080 --> 00:57:08.863 align:middle line:84% So we'll call it a divide and conquer recurrence, 00:57:08.863 --> 00:57:10.530 align:middle line:84% is going to be one that looks like this. 00:57:10.530 --> 00:57:16.130 align:middle line:84% It's going to be T of n equals some constant a times T 00:57:16.130 --> 00:57:23.400 align:middle line:84% of the ceiling of n over b plus some function f of n, 00:57:23.400 --> 00:57:28.890 align:middle line:90% where a is-- 00:57:28.890 --> 00:57:31.490 align:middle line:84% let's see, what are our conditions? a is at least 1 00:57:31.490 --> 00:57:38.540 align:middle line:84% and b is greater than 1, strictly greater than 1, 00:57:38.540 --> 00:57:40.730 align:middle line:90% and f of n is some function. 00:57:40.730 --> 00:57:42.810 align:middle line:90% Merge sort certainly fits this. 00:57:42.810 --> 00:57:47.060 align:middle line:84% a is 2. b is 2. f of n is n minus 1. 00:57:47.060 --> 00:57:50.030 align:middle line:84% And the master theorem says if you have a divide and conquer 00:57:50.030 --> 00:57:52.970 align:middle line:84% recurrence that looks like this, if you 00:57:52.970 --> 00:57:56.480 align:middle line:84% check just a couple quick conditions on how quickly f 00:57:56.480 --> 00:58:02.210 align:middle line:84% grows, I can use that to tell you a theta bound on T. 00:58:02.210 --> 00:58:04.470 align:middle line:84% It's not giving me an exact closed form. 00:58:04.470 --> 00:58:06.000 align:middle line:84% It's giving me an asymptotic bound. 00:58:06.000 --> 00:58:15.120 align:middle line:84% It's going to say that T of n is in theta of something. 00:58:15.120 --> 00:58:17.840 align:middle line:84% There are three cases where the master theorem can tell you 00:58:17.840 --> 00:58:23.360 align:middle line:84% explicitly T of n is theta of this nice simple thing. 00:58:23.360 --> 00:58:26.150 align:middle line:90% All right, so that's the idea. 00:58:26.150 --> 00:58:27.900 align:middle line:90% Let's not prove that right now. 00:58:27.900 --> 00:58:30.980 align:middle line:84% But let's talk about what the three cases mean 00:58:30.980 --> 00:58:33.950 align:middle line:84% and where they come from and get some intuition 00:58:33.950 --> 00:58:35.320 align:middle line:90% for the master theorem. 00:58:35.320 --> 00:58:51.970 align:middle line:90% 00:58:51.970 --> 00:58:54.870 align:middle line:84% And if we think about a divide and conquer algorithm like this, 00:58:54.870 --> 00:59:02.390 align:middle line:84% it says, well, at the very top, we have n elements in our list. 00:59:02.390 --> 00:59:05.310 align:middle line:84% And we're going to do some work with recursive calls. 00:59:05.310 --> 00:59:08.090 align:middle line:84% That's the a times T of thing term. 00:59:08.090 --> 00:59:11.420 align:middle line:84% But then we're going to do some work with these n elements 00:59:11.420 --> 00:59:12.270 align:middle line:90% themselves. 00:59:12.270 --> 00:59:14.000 align:middle line:90% And that's f of n. 00:59:14.000 --> 00:59:19.170 align:middle line:84% So we're going to do f of n work at this node. 00:59:19.170 --> 00:59:22.080 align:middle line:84% And all the rest will be done in the children 00:59:22.080 --> 00:59:26.150 align:middle line:84% if we're thinking about this as the recursive call 00:59:26.150 --> 00:59:28.850 align:middle line:90% stack, call tree. 00:59:28.850 --> 00:59:31.830 align:middle line:84% Here's where we're dealing with T of n. 00:59:31.830 --> 00:59:36.290 align:middle line:84% Here's where we're dealing with T of n over b. 00:59:36.290 --> 00:59:39.440 align:middle line:84% And there are "a" different children. 00:59:39.440 --> 00:59:41.740 align:middle line:84% In merge sort, we're splitting it into halves. 00:59:41.740 --> 00:59:43.680 align:middle line:84% We're making two recursive calls. 00:59:43.680 --> 00:59:46.120 align:middle line:84% In this example, maybe there are three recursive calls. 00:59:46.120 --> 00:59:48.210 align:middle line:90% So a is 3. 00:59:48.210 --> 00:59:53.532 align:middle line:84% And at each of these nodes, we are calling f of n over b. 00:59:53.532 --> 00:59:57.300 align:middle line:90% 00:59:57.300 --> 01:00:01.110 align:middle line:84% And each of these is making some recursive calls. 01:00:01.110 --> 01:00:06.930 align:middle line:84% Each of these nodes is doing work f of n over b squared. 01:00:06.930 --> 01:00:11.010 align:middle line:84% Then, again, f of n over b cubed. 01:00:11.010 --> 01:00:13.380 align:middle line:90% And so on all the way down-- 01:00:13.380 --> 01:00:21.510 align:middle line:84% f of n over b to the h, however tall this tree is. 01:00:21.510 --> 01:00:25.900 align:middle line:84% So each of these nodes is doing this amount of work in itself. 01:00:25.900 --> 01:00:28.350 align:middle line:84% And the rest of the work is done recursively. 01:00:28.350 --> 01:00:30.880 align:middle line:84% How many nodes do we have at each level? 01:00:30.880 --> 01:00:36.180 align:middle line:84% Well, we have one node here, "a" nodes here, 01:00:36.180 --> 01:00:47.310 align:middle line:84% "a" squared nodes here, "a" cubed nodes here, 01:00:47.310 --> 01:00:51.770 align:middle line:84% a to the h nodes here at the bottom. 01:00:51.770 --> 01:00:55.200 align:middle line:84% So we're getting more and more subproblems. 01:00:55.200 --> 01:00:56.870 align:middle line:84% But the sizes of those subproblems 01:00:56.870 --> 01:00:59.690 align:middle line:90% are getting smaller and smaller. 01:00:59.690 --> 01:01:02.804 align:middle line:84% To put this in context, for merge sort, 01:01:02.804 --> 01:01:06.120 align:middle line:84% we called merge with n elements at the very top. 01:01:06.120 --> 01:01:08.600 align:middle line:84% We had two cases where we called merge with n 01:01:08.600 --> 01:01:11.480 align:middle line:84% over 2 elements, four times where we called merge 01:01:11.480 --> 01:01:14.150 align:middle line:84% with n over 4 elements, eight times we 01:01:14.150 --> 01:01:17.810 align:middle line:84% called merge with n over 8 elements, and so on and so on. 01:01:17.810 --> 01:01:19.230 align:middle line:90% Does this idea make sense? 01:01:19.230 --> 01:01:25.400 align:middle line:84% This is a conceptual view of all of the different recursive calls 01:01:25.400 --> 01:01:28.790 align:middle line:84% in our divide and conquer algorithm. 01:01:28.790 --> 01:01:32.180 align:middle line:84% And this sum here, fn times 1 plus f 01:01:32.180 --> 01:01:35.510 align:middle line:84% of n over b times a, this times a squared, this times a cubed, 01:01:35.510 --> 01:01:39.710 align:middle line:84% all the way down to the bottom, this is our runtime. 01:01:39.710 --> 01:01:40.965 align:middle line:90% I'm being sketchy here. 01:01:40.965 --> 01:01:42.840 align:middle line:84% I'm not dealing with the floors and ceilings. 01:01:42.840 --> 01:01:45.320 align:middle line:90% This is just intuition anyway. 01:01:45.320 --> 01:01:47.400 align:middle line:90% So I'm going to be a bit sloppy. 01:01:47.400 --> 01:01:49.940 align:middle line:90% I apologize in advance. 01:01:49.940 --> 01:01:53.450 align:middle line:84% But this is conceptually our runtime. 01:01:53.450 --> 01:01:55.320 align:middle line:84% First of all, how tall is this tree? 01:01:55.320 --> 01:01:57.860 align:middle line:90% What is this h? 01:01:57.860 --> 01:01:59.843 align:middle line:90% Well, we're going down until-- 01:01:59.843 --> 01:02:01.760 align:middle line:84% well, with merge sort, we're going all the way 01:02:01.760 --> 01:02:05.168 align:middle line:84% until we get a single element, we sort a single element. 01:02:05.168 --> 01:02:06.210 align:middle line:90% And that's our base case. 01:02:06.210 --> 01:02:08.680 align:middle line:84% And we're certainly not going beyond that. 01:02:08.680 --> 01:02:09.930 align:middle line:90% Well, what does that say here? 01:02:09.930 --> 01:02:14.030 align:middle line:84% That says n and b to the h are equal to each other. 01:02:14.030 --> 01:02:20.455 align:middle line:84% In other words, the height of the tree is log b of n. 01:02:20.455 --> 01:02:23.870 align:middle line:90% 01:02:23.870 --> 01:02:25.470 align:middle line:84% All right, and now we have the sum. 01:02:25.470 --> 01:02:27.710 align:middle line:90% And this is our runtime. 01:02:27.710 --> 01:02:31.550 align:middle line:84% The master theorem says that this sum often 01:02:31.550 --> 01:02:37.130 align:middle line:84% has three familiar forms, one of three familiar forms. 01:02:37.130 --> 01:02:41.370 align:middle line:84% In the first case, in case 1 of the master theorem, 01:02:41.370 --> 01:02:44.700 align:middle line:84% it says if fn is big O of this thing-- 01:02:44.700 --> 01:02:50.090 align:middle line:84% ignore what this thing is-- if fn is big O, if fn is small-- 01:02:50.090 --> 01:02:57.500 align:middle line:84% so if f of n is small, imagine in the extreme case fn 01:02:57.500 --> 01:02:58.970 align:middle line:90% is just 1. 01:02:58.970 --> 01:03:03.500 align:middle line:84% There's only a single step to do at each of these nodes. 01:03:03.500 --> 01:03:10.730 align:middle line:84% Well, then our runtime looks like 1 plus a plus 01:03:10.730 --> 01:03:13.370 align:middle line:90% a squared plus a cubed-- 01:03:13.370 --> 01:03:17.090 align:middle line:90% cubed-- plus a to the h. 01:03:17.090 --> 01:03:21.950 align:middle line:84% If the f's are insignificant compared to the a's, then 01:03:21.950 --> 01:03:25.430 align:middle line:90% our sum is basically this. 01:03:25.430 --> 01:03:28.790 align:middle line:84% And this is a geometric series, which is basically as big as a 01:03:28.790 --> 01:03:32.187 align:middle line:84% to the h itself, up to constant factors, lower-order terms. 01:03:32.187 --> 01:03:34.020 align:middle line:84% We have a geometric series formula for that. 01:03:34.020 --> 01:03:37.540 align:middle line:84% But basically it's saying if fn is small, 01:03:37.540 --> 01:03:43.670 align:middle line:84% then this last term dominates the whole sum. 01:03:43.670 --> 01:03:45.940 align:middle line:84% We have these two conflicting effects 01:03:45.940 --> 01:03:48.460 align:middle line:84% where the f's are getting smaller 01:03:48.460 --> 01:03:50.150 align:middle line:84% because the inputs are getting smaller. 01:03:50.150 --> 01:03:53.740 align:middle line:84% But the powers of a are getting bigger. 01:03:53.740 --> 01:03:58.510 align:middle line:84% And so case 1 is describing the case where the a's grow faster 01:03:58.510 --> 01:04:01.150 align:middle line:90% than the f's shrink. 01:04:01.150 --> 01:04:03.230 align:middle line:84% And it turns out that in that case, 01:04:03.230 --> 01:04:07.930 align:middle line:84% the runtime is basically just a to the h itself, that T to the n 01:04:07.930 --> 01:04:13.510 align:middle line:84% is in theta of a to the h, which is-- 01:04:13.510 --> 01:04:15.310 align:middle line:90% a to the h is, what? 01:04:15.310 --> 01:04:20.650 align:middle line:84% a to the log b of n, which, by log rules, 01:04:20.650 --> 01:04:24.670 align:middle line:84% is the same as n to the log b of a. 01:04:24.670 --> 01:04:28.300 align:middle line:84% And that's exactly the result from case 1. 01:04:28.300 --> 01:04:32.420 align:middle line:84% If the a's overpower the f's, then this last term wins. 01:04:32.420 --> 01:04:36.270 align:middle line:84% And that basically gives us our asymptotic runtime. 01:04:36.270 --> 01:04:45.510 align:middle line:84% On the other extreme, if f grows fast, if f is big, 01:04:45.510 --> 01:04:51.450 align:middle line:84% so it grows really quickly, then the f terms over here 01:04:51.450 --> 01:04:55.560 align:middle line:84% are going to overpower the a terms. 01:04:55.560 --> 01:04:58.680 align:middle line:84% And we end up getting the opposite effect, where 01:04:58.680 --> 01:05:01.980 align:middle line:84% this first term is the biggest, and all the rest don't really 01:05:01.980 --> 01:05:04.530 align:middle line:90% contribute much asymptotically. 01:05:04.530 --> 01:05:08.790 align:middle line:84% So if f is big, then the first term in our sum dominates. 01:05:08.790 --> 01:05:11.330 align:middle line:90% And we end up getting T of n-- 01:05:11.330 --> 01:05:14.850 align:middle line:90% T of n is in theta of f. 01:05:14.850 --> 01:05:17.710 align:middle line:90% And that is what case 3 says. 01:05:17.710 --> 01:05:20.790 align:middle line:84% So if f grows really fast, then f alone 01:05:20.790 --> 01:05:23.280 align:middle line:84% is where most of the work happens. 01:05:23.280 --> 01:05:25.410 align:middle line:84% In terms of our tree, that means that the work we 01:05:25.410 --> 01:05:29.470 align:middle line:84% had to do in this top node, fn, that's the bulk of it. 01:05:29.470 --> 01:05:34.260 align:middle line:84% And everything else we did down here is small in comparison. 01:05:34.260 --> 01:05:38.730 align:middle line:84% All right, now there's a case 3-- 01:05:38.730 --> 01:05:40.420 align:middle line:90% I want to keep that tree. 01:05:40.420 --> 01:05:42.660 align:middle line:90% So let's come over here. 01:05:42.660 --> 01:05:45.060 align:middle line:84% Sorry, there's a case 2, the middle case, 01:05:45.060 --> 01:05:47.820 align:middle line:84% of master theorem that says, what if these two 01:05:47.820 --> 01:05:50.060 align:middle line:90% effects balance each other out? 01:05:50.060 --> 01:05:55.950 align:middle line:90% 01:05:55.950 --> 01:05:59.790 align:middle line:84% And in this case, it's basically asking, 01:05:59.790 --> 01:06:05.130 align:middle line:84% what if all of these terms, so fn times 1, fn over b times a, 01:06:05.130 --> 01:06:08.072 align:middle line:84% fn over b squared times a squared, all of these terms 01:06:08.072 --> 01:06:10.655 align:middle line:84% in the sum, what if they're all basically equal to each other? 01:06:10.655 --> 01:06:13.980 align:middle line:90% 01:06:13.980 --> 01:06:23.540 align:middle line:84% So in the middle, all terms of the sum are basically equal. 01:06:23.540 --> 01:06:27.810 align:middle line:90% 01:06:27.810 --> 01:06:31.690 align:middle line:84% And so we end up getting that the sum is-- 01:06:31.690 --> 01:06:35.470 align:middle line:84% well, there are h terms here, h plus 1 terms. 01:06:35.470 --> 01:06:38.170 align:middle line:84% And each of the terms is basically the same. 01:06:38.170 --> 01:06:41.230 align:middle line:84% So I'm going to take this where it's a to the h times-- 01:06:41.230 --> 01:06:43.480 align:middle line:84% well, n over b to the h is 1, right? 01:06:43.480 --> 01:06:51.960 align:middle line:84% So this is basically, well, h plus 1 times-- 01:06:51.960 --> 01:06:53.790 align:middle line:90% what was that last term again? 01:06:53.790 --> 01:06:57.450 align:middle line:90% a to the h times f of 1. 01:06:57.450 --> 01:06:59.220 align:middle line:84% If all the terms are basically equal, 01:06:59.220 --> 01:07:03.861 align:middle line:84% then it's just the number of terms times this last one. 01:07:03.861 --> 01:07:05.100 align:middle line:90% We're doing asymptotics. 01:07:05.100 --> 01:07:07.710 align:middle line:90% I don't care about that plus 1. 01:07:07.710 --> 01:07:10.030 align:middle line:90% f of 1 is a constant. 01:07:10.030 --> 01:07:11.730 align:middle line:90% I don't care about that. 01:07:11.730 --> 01:07:16.150 align:middle line:84% So this ends up being basically a to the h times h. 01:07:16.150 --> 01:07:19.770 align:middle line:90% This is theta of. 01:07:19.770 --> 01:07:22.890 align:middle line:90% And if we remember what h was-- 01:07:22.890 --> 01:07:27.300 align:middle line:84% sorry, if we remember what a to the h was, that's n to the log 01:07:27.300 --> 01:07:28.470 align:middle line:90% b of a. 01:07:28.470 --> 01:07:30.270 align:middle line:90% And if we remember what h was-- 01:07:30.270 --> 01:07:33.910 align:middle line:90% 01:07:33.910 --> 01:07:37.660 align:middle line:90% times another factor, log b. 01:07:37.660 --> 01:07:41.200 align:middle line:90% Sorry, yeah, log b of n. 01:07:41.200 --> 01:07:43.660 align:middle line:84% So a to the h is this power of n. 01:07:43.660 --> 01:07:46.030 align:middle line:90% h is log of n. 01:07:46.030 --> 01:07:50.440 align:middle line:84% It's basically saying in, case 2, if things balance out nicely, 01:07:50.440 --> 01:07:52.960 align:middle line:84% you just incur an extra factor of log n 01:07:52.960 --> 01:07:55.305 align:middle line:84% from what you would otherwise expect from case 1. 01:07:55.305 --> 01:07:58.180 align:middle line:90% 01:07:58.180 --> 01:08:01.310 align:middle line:84% And turns out merge sort is in that middle ground. 01:08:01.310 --> 01:08:13.840 align:middle line:84% It's in case 2 because fn here is n minus 1 for merge sort. 01:08:13.840 --> 01:08:23.170 align:middle line:84% And it's asking, is this theta of n to the log base 2 of 2? 01:08:23.170 --> 01:08:25.390 align:middle line:84% That's the condition for case 2 to apply, 01:08:25.390 --> 01:08:27.069 align:middle line:90% according to master theorem. 01:08:27.069 --> 01:08:30.630 align:middle line:84% And, yeah, n minus 1 is in theta of n. 01:08:30.630 --> 01:08:32.080 align:middle line:90% Its exponent is 1. 01:08:32.080 --> 01:08:33.010 align:middle line:90% So, yes. 01:08:33.010 --> 01:08:34.175 align:middle line:90% So we're in case 2. 01:08:34.175 --> 01:08:36.689 align:middle line:90% 01:08:36.689 --> 01:08:43.200 align:middle line:84% And this says that master theorem of n is theta of n 01:08:43.200 --> 01:08:45.899 align:middle line:84% to that exponent-- that exponent is 1-- 01:08:45.899 --> 01:08:50.790 align:middle line:90% times an extra factor of log n. 01:08:50.790 --> 01:08:53.819 align:middle line:84% So that's how we can use the master theorem just 01:08:53.819 --> 01:08:56.529 align:middle line:84% by pattern matching on the shape of the recurrence. 01:08:56.529 --> 01:09:00.569 align:middle line:90% a is 2. b is 2. fn is n minus 1. 01:09:00.569 --> 01:09:03.220 align:middle line:84% Make sure it satisfies the condition for case 2. 01:09:03.220 --> 01:09:05.520 align:middle line:84% And in that case, we can conclude 01:09:05.520 --> 01:09:09.689 align:middle line:84% that Mn is theta of n log n, which 01:09:09.689 --> 01:09:13.439 align:middle line:84% is the formula we found earlier, at least when we were only 01:09:13.439 --> 01:09:14.950 align:middle line:90% dealing with powers of 2. 01:09:14.950 --> 01:09:21.270 align:middle line:84% We had n log n plus a linear term, minus a linear term. 01:09:21.270 --> 01:09:24.550 align:middle line:84% And so this matches what we had in general. 01:09:24.550 --> 01:09:25.779 align:middle line:90% It's less informative. 01:09:25.779 --> 01:09:28.540 align:middle line:84% It's only a theta bound, not an exact formula. 01:09:28.540 --> 01:09:32.449 align:middle line:84% But it still gives us what we would expect, which is nice. 01:09:32.449 --> 01:09:35.819 align:middle line:90% 01:09:35.819 --> 01:09:39.141 align:middle line:84% Last thing I want to say about this, merge sort 01:09:39.141 --> 01:09:41.580 align:middle line:90% does have gaps-- 01:09:41.580 --> 01:09:42.840 align:middle line:90% sorry, not merge sort. 01:09:42.840 --> 01:09:46.500 align:middle line:90% Master theorem does have gaps. 01:09:46.500 --> 01:09:50.609 align:middle line:84% If you have a divide and conquer recurrence like this, 01:09:50.609 --> 01:09:53.410 align:middle line:84% you can't always just apply master theorem. 01:09:53.410 --> 01:09:57.510 align:middle line:84% Sometimes you have cases that don't fall into any of the three 01:09:57.510 --> 01:09:59.080 align:middle line:90% cases of master theorem. 01:09:59.080 --> 01:10:02.010 align:middle line:90% Case 1, 2, and 3 are all false. 01:10:02.010 --> 01:10:03.520 align:middle line:90% Those conditions are not valid. 01:10:03.520 --> 01:10:05.440 align:middle line:84% So you can't apply master theorem at all. 01:10:05.440 --> 01:10:09.510 align:middle line:84% So master theorem is not snake oil 01:10:09.510 --> 01:10:11.320 align:middle line:90% that will solve every problem. 01:10:11.320 --> 01:10:12.910 align:middle line:90% It'll solve many problems. 01:10:12.910 --> 01:10:17.110 align:middle line:84% And in 6.121, you'll be using it quite a bit. 01:10:17.110 --> 01:10:20.790 align:middle line:84% And you'll see how useful it can be in practice. 01:10:20.790 --> 01:10:24.180 align:middle line:90% Final thought. 01:10:24.180 --> 01:10:28.390 align:middle line:84% Let's come back and compare some-- yeah, 01:10:28.390 --> 01:10:29.830 align:middle line:84% I'll just use this-- compare some 01:10:29.830 --> 01:10:35.300 align:middle line:84% of our recurrences and the growth rates that they describe. 01:10:35.300 --> 01:10:38.980 align:middle line:84% So we had, for example, Fibonacci numbers. 01:10:38.980 --> 01:10:43.400 align:middle line:84% Fn is Fn minus 1 plus Fn minus 2. 01:10:43.400 --> 01:10:50.630 align:middle line:84% We had Towers of Hanoi, where Hn was 2H of n minus 1 plus 1. 01:10:50.630 --> 01:10:51.950 align:middle line:90% This was Fibonacci. 01:10:51.950 --> 01:10:53.980 align:middle line:90% This was Hanoi. 01:10:53.980 --> 01:10:57.145 align:middle line:90% We have merge sort-- 01:10:57.145 --> 01:11:04.410 align:middle line:84% merge sort of n, which is 2 M of n over 2 plus n minus 1. 01:11:04.410 --> 01:11:08.350 align:middle line:90% 01:11:08.350 --> 01:11:10.870 align:middle line:84% And other divide and conquer recurrences-- 01:11:10.870 --> 01:11:13.120 align:middle line:84% I'll use "D" for divide and conquer. 01:11:13.120 --> 01:11:20.050 align:middle line:84% This is a times D of n over b plus f of n. 01:11:20.050 --> 01:11:21.700 align:middle line:84% And if we take a look at what formulas 01:11:21.700 --> 01:11:28.180 align:middle line:84% we got for these, this Fibonacci, this was theta of 1.6 01:11:28.180 --> 01:11:32.920 align:middle line:84% to the n, that 1 plus root 5 all divided by 2 to the n. 01:11:32.920 --> 01:11:35.540 align:middle line:84% The Fibonacci numbers grow exponentially. 01:11:35.540 --> 01:11:39.670 align:middle line:84% The Towers of Hanoi grow exponentially. 01:11:39.670 --> 01:11:41.440 align:middle line:90% Merge sort is very small-- 01:11:41.440 --> 01:11:42.540 align:middle line:90% n log n. 01:11:42.540 --> 01:11:46.180 align:middle line:90% 01:11:46.180 --> 01:11:49.420 align:middle line:84% And the main difference here is that, in each 01:11:49.420 --> 01:11:52.390 align:middle line:84% of these recurrences, we're making recursive calls 01:11:52.390 --> 01:11:54.160 align:middle line:90% to smaller terms. 01:11:54.160 --> 01:11:57.550 align:middle line:84% But how quickly are those smaller terms decreasing? 01:11:57.550 --> 01:12:00.220 align:middle line:84% In merge sort, every time we make a recursive call, 01:12:00.220 --> 01:12:01.760 align:middle line:90% we cut the size in half. 01:12:01.760 --> 01:12:03.820 align:middle line:90% We cut n in half. 01:12:03.820 --> 01:12:05.920 align:middle line:84% So there are only logarithmically 01:12:05.920 --> 01:12:10.270 align:middle line:84% many times that we can cut it in half before we bottom out. 01:12:10.270 --> 01:12:13.210 align:middle line:84% So that's why our call tree over there 01:12:13.210 --> 01:12:15.950 align:middle line:90% was only logarithmic height. 01:12:15.950 --> 01:12:18.880 align:middle line:84% And so the runtime ends up being pretty small because there's not 01:12:18.880 --> 01:12:20.590 align:middle line:90% much of a tree to deal with. 01:12:20.590 --> 01:12:23.240 align:middle line:84% By contrast, with Hanoi and Fibonacci, 01:12:23.240 --> 01:12:25.310 align:middle line:84% every time we make a recursive call, 01:12:25.310 --> 01:12:28.390 align:middle line:90% we're only subtracting 1 or 2. 01:12:28.390 --> 01:12:31.910 align:middle line:84% And so we have to do that linearly many times, 01:12:31.910 --> 01:12:35.240 align:middle line:84% like n times, before we get down to the bottom. 01:12:35.240 --> 01:12:37.400 align:middle line:84% And every time we make this call, 01:12:37.400 --> 01:12:39.670 align:middle line:84% the tree is getting exponentially bigger. 01:12:39.670 --> 01:12:42.490 align:middle line:84% It's doubling every time we do this, basically. 01:12:42.490 --> 01:12:43.990 align:middle line:84% And that's why, in these two cases, 01:12:43.990 --> 01:12:48.070 align:middle line:84% we're seeing exponential growth, because the tree is huge. 01:12:48.070 --> 01:12:50.690 align:middle line:84% So just thinking about this recursive call tree, 01:12:50.690 --> 01:12:53.410 align:middle line:84% which terms are calling which other terms and how far 01:12:53.410 --> 01:12:55.690 align:middle line:84% down does that go and how many nodes are there, 01:12:55.690 --> 01:12:59.680 align:middle line:84% is a great way to get intuition and a rough idea of how 01:12:59.680 --> 01:13:01.370 align:middle line:84% quickly these things are growing, 01:13:01.370 --> 01:13:03.400 align:middle line:84% and a great reason why divide and conquer 01:13:03.400 --> 01:13:07.690 align:middle line:84% algorithms in general are often really good, because they bottom 01:13:07.690 --> 01:13:10.030 align:middle line:90% out quickly. 01:13:10.030 --> 01:13:11.900 align:middle line:84% That is everything I wanted to say. 01:13:11.900 --> 01:13:13.540 align:middle line:90% Thank you so much. 01:13:13.540 --> 01:13:15.220 align:middle line:84% We'll see you in recitation tomorrow. 01:13:15.220 --> 01:13:17.970 align:middle line:84% And good luck studying for your quiz. 01:13:17.970 --> 01:13:23.000 align:middle line:90%