WEBVTT 00:00:00.000 --> 00:00:02.460 align:middle line:90% [SQUEAKING] 00:00:02.460 --> 00:00:04.428 align:middle line:90% [RUSTLING] 00:00:04.428 --> 00:00:07.380 align:middle line:90% [CLICKING] 00:00:07.380 --> 00:00:10.335 align:middle line:90% 00:00:10.335 --> 00:00:11.460 align:middle line:90% ZACHARY ABEL: Good morning. 00:00:11.460 --> 00:00:12.600 align:middle line:90% Good afternoon. 00:00:12.600 --> 00:00:14.193 align:middle line:90% Welcome to lecture 6. 00:00:14.193 --> 00:00:16.730 align:middle line:90% 00:00:16.730 --> 00:00:20.750 align:middle line:84% Today we will be finishing up our discussion of summations, 00:00:20.750 --> 00:00:22.650 align:middle line:84% that Brynmor started for us last time, 00:00:22.650 --> 00:00:26.060 align:middle line:84% and then moving on to asymptotic notation, which 00:00:26.060 --> 00:00:30.090 align:middle line:84% is one of the bread-and-butter topics of algorithm analysis. 00:00:30.090 --> 00:00:33.050 align:middle line:84% It's a really useful language and terminology 00:00:33.050 --> 00:00:34.970 align:middle line:84% that we can use to talk about algorithms 00:00:34.970 --> 00:00:36.650 align:middle line:84% and compare algorithms without getting 00:00:36.650 --> 00:00:38.430 align:middle line:90% bogged down in all the details. 00:00:38.430 --> 00:00:41.070 align:middle line:84% But before we do that, let's talk about sums. 00:00:41.070 --> 00:00:44.330 align:middle line:84% And before we do that, let's play with blocks. 00:00:44.330 --> 00:00:46.530 align:middle line:90% So here is a very cool question. 00:00:46.530 --> 00:00:47.835 align:middle line:90% I have a stack of blocks. 00:00:47.835 --> 00:00:49.710 align:middle line:84% You could imagine it's even taller than this. 00:00:49.710 --> 00:00:51.620 align:middle line:90% I just grabbed six for now. 00:00:51.620 --> 00:00:58.160 align:middle line:84% And what I want to do is lean them out over the table and-- 00:00:58.160 --> 00:01:00.020 align:middle line:90% whoop, don't want him to fall. 00:01:00.020 --> 00:01:01.820 align:middle line:90% That's criterion one. 00:01:01.820 --> 00:01:05.050 align:middle line:84% What I'm hoping for is to get this top block 00:01:05.050 --> 00:01:08.410 align:middle line:90% all the way off of this table. 00:01:08.410 --> 00:01:10.970 align:middle line:84% So none of the block is over the table. 00:01:10.970 --> 00:01:13.490 align:middle line:84% So currently, I'm not quite there. 00:01:13.490 --> 00:01:17.590 align:middle line:84% We can see there's an inch, half an inch left. 00:01:17.590 --> 00:01:22.540 align:middle line:90% So it seems possible, maybe. 00:01:22.540 --> 00:01:23.120 align:middle line:90% So. 00:01:23.120 --> 00:01:26.270 align:middle line:84% The rules are we can only have one block per layer. 00:01:26.270 --> 00:01:27.200 align:middle line:90% We can't double up. 00:01:27.200 --> 00:01:28.790 align:middle line:84% We can't do weird angles and whatnot. 00:01:28.790 --> 00:01:31.340 align:middle line:84% It's just slide these left and right until you're happy. 00:01:31.340 --> 00:01:33.850 align:middle line:90% And hopefully, it doesn't fall. 00:01:33.850 --> 00:01:35.950 align:middle line:90% Are we clear on the game? 00:01:35.950 --> 00:01:36.610 align:middle line:90% Nice. 00:01:36.610 --> 00:01:38.500 align:middle line:90% Who thinks it's possible? 00:01:38.500 --> 00:01:39.560 align:middle line:90% Raise your hand. 00:01:39.560 --> 00:01:42.310 align:middle line:90% Who thinks it's not possible? 00:01:42.310 --> 00:01:43.280 align:middle line:90% All right. 00:01:43.280 --> 00:01:44.510 align:middle line:90% Well, let's try. 00:01:44.510 --> 00:01:48.260 align:middle line:90% Hopefully, I don't hurt myself. 00:01:48.260 --> 00:01:51.722 align:middle line:84% So I'm going to follow a nice, greedy strategy. 00:01:51.722 --> 00:01:53.180 align:middle line:84% So I'm going to start from the top. 00:01:53.180 --> 00:01:55.840 align:middle line:84% And then I'm just going to keep sliding left as much as I can 00:01:55.840 --> 00:01:57.780 align:middle line:90% until it's about to fall over. 00:01:57.780 --> 00:02:00.930 align:middle line:84% So if I go past the center of mass, it's going to fall over. 00:02:00.930 --> 00:02:03.563 align:middle line:84% So I'm going to go right up until that edge and stop. 00:02:03.563 --> 00:02:04.980 align:middle line:84% And don't worry about the physics. 00:02:04.980 --> 00:02:07.460 align:middle line:84% We're not here to talk about physics. 00:02:07.460 --> 00:02:08.759 align:middle line:90% But I'm going to do the same. 00:02:08.759 --> 00:02:11.160 align:middle line:84% So let's freeze the top two layers and try again. 00:02:11.160 --> 00:02:15.000 align:middle line:84% Let's slide this over until it's just about to fall. 00:02:15.000 --> 00:02:17.160 align:middle line:90% It feels right around there. 00:02:17.160 --> 00:02:19.970 align:middle line:90% All right, let's keep going. 00:02:19.970 --> 00:02:22.610 align:middle line:90% All right, up, up. 00:02:22.610 --> 00:02:24.320 align:middle line:90% Yep, OK. 00:02:24.320 --> 00:02:25.440 align:middle line:90% It leaned a little bit. 00:02:25.440 --> 00:02:28.260 align:middle line:84% I'm getting worried, but we'll make it work. 00:02:28.260 --> 00:02:30.860 align:middle line:90% Let's see. 00:02:30.860 --> 00:02:32.720 align:middle line:90% Yeah, OK. 00:02:32.720 --> 00:02:36.140 align:middle line:84% And I think I might already be off the table, which 00:02:36.140 --> 00:02:36.900 align:middle line:90% was my goal. 00:02:36.900 --> 00:02:38.580 align:middle line:90% But let's go a little further. 00:02:38.580 --> 00:02:40.820 align:middle line:90% Why not? 00:02:40.820 --> 00:02:41.750 align:middle line:90% Yes. 00:02:41.750 --> 00:02:42.390 align:middle line:90% OK. 00:02:42.390 --> 00:02:46.610 align:middle line:90% And finally, right around. 00:02:46.610 --> 00:02:49.370 align:middle line:84% OK, that's as far as I'm willing to go 00:02:49.370 --> 00:02:53.610 align:middle line:90% I think that is off the table. 00:02:53.610 --> 00:02:54.350 align:middle line:90% AUDIENCE: Whoa. 00:02:54.350 --> 00:02:55.290 align:middle line:90% ZACHARY ABEL: It's possible. 00:02:55.290 --> 00:02:55.790 align:middle line:90% [CLAPPING] 00:02:55.790 --> 00:02:56.670 align:middle line:90% Yes, thank you. 00:02:56.670 --> 00:02:59.370 align:middle line:90% [APPLAUSE] 00:02:59.370 --> 00:03:01.330 align:middle line:90% 00:03:01.330 --> 00:03:02.090 align:middle line:90% I cheated. 00:03:02.090 --> 00:03:05.950 align:middle line:84% I put little marks on the back, so I knew how far to go. 00:03:05.950 --> 00:03:07.213 align:middle line:90% But it is possible. 00:03:07.213 --> 00:03:08.630 align:middle line:84% And now we can raise the question. 00:03:08.630 --> 00:03:09.950 align:middle line:90% How far can we go? 00:03:09.950 --> 00:03:12.020 align:middle line:90% Can we get two lengths out? 00:03:12.020 --> 00:03:14.660 align:middle line:84% Here's what that would look like, two lengths out. 00:03:14.660 --> 00:03:17.920 align:middle line:84% Can we get five lengths out, 10 lengths out? 00:03:17.920 --> 00:03:18.890 align:middle line:90% Let's stop at 10. 00:03:18.890 --> 00:03:22.460 align:middle line:84% Who thinks we can get 10 lengths out over the table? 00:03:22.460 --> 00:03:24.110 align:middle line:90% Who thinks 10 is possible? 00:03:24.110 --> 00:03:26.770 align:middle line:90% Who thinks 10 is not possible? 00:03:26.770 --> 00:03:30.610 align:middle line:90% All right, you're both right. 00:03:30.610 --> 00:03:32.860 align:middle line:90% I'll explain why in a second. 00:03:32.860 --> 00:03:35.390 align:middle line:84% Let's not have an accident during lecture. 00:03:35.390 --> 00:03:36.290 align:middle line:90% There we go. 00:03:36.290 --> 00:03:38.780 align:middle line:84% So, I'm not going to prove this now. 00:03:38.780 --> 00:03:41.470 align:middle line:84% But turns out that greedy strategy I just used 00:03:41.470 --> 00:03:43.630 align:middle line:90% is the best you can do. 00:03:43.630 --> 00:03:44.860 align:middle line:90% It's not too hard to prove. 00:03:44.860 --> 00:03:47.200 align:middle line:84% It's in the textbook, if you're curious. 00:03:47.200 --> 00:03:50.050 align:middle line:84% But let me tell you what that greedy strategy ends up looking 00:03:50.050 --> 00:03:52.790 align:middle line:90% like, if I have some chalk. 00:03:52.790 --> 00:03:54.410 align:middle line:90% Here we go. 00:03:54.410 --> 00:03:57.140 align:middle line:84% So, as you might imagine, this first block 00:03:57.140 --> 00:04:00.600 align:middle line:84% sticks out half a unit over the next block. 00:04:00.600 --> 00:04:04.430 align:middle line:84% So its center of mass is right here on the edge. 00:04:04.430 --> 00:04:09.020 align:middle line:84% By the way, I'm imagining these blocks are one unit wide. 00:04:09.020 --> 00:04:13.610 align:middle line:84% The next one, right here, that's a quarter unit. 00:04:13.610 --> 00:04:15.860 align:middle line:84% Because that's what it takes to put the center of mass 00:04:15.860 --> 00:04:18.500 align:middle line:84% of these two blocks combined right 00:04:18.500 --> 00:04:20.690 align:middle line:84% over the edge of the third block. 00:04:20.690 --> 00:04:23.850 align:middle line:84% Turns out this right here is a sixth. 00:04:23.850 --> 00:04:26.660 align:middle line:84% This right here is an eighth, and all the way 00:04:26.660 --> 00:04:30.510 align:middle line:84% down until the very end, until it is the very end. 00:04:30.510 --> 00:04:35.030 align:middle line:84% The n-th block has a gap of 1 over 2n. 00:04:35.030 --> 00:04:40.040 align:middle line:84% So this is block 1, 2, 3, 4, 5, all the way to n. 00:04:40.040 --> 00:04:42.390 align:middle line:84% Turns out this is the best you can do. 00:04:42.390 --> 00:04:50.090 align:middle line:84% Which means the distance you can get with n blocks is 1/2 plus 00:04:50.090 --> 00:04:55.690 align:middle line:84% a 1/4 plus 1/6 up to 1 over 2n, which-- 00:04:55.690 --> 00:04:58.510 align:middle line:84% let's factor out the 1/2, just because that's cleaner. 00:04:58.510 --> 00:04:59.860 align:middle line:90% That's the sum. 00:04:59.860 --> 00:05:03.070 align:middle line:90% K equals 1 to n of 1 over k. 00:05:03.070 --> 00:05:11.710 align:middle line:84% This is also known as 1/2 times the n-th harmonic number, 00:05:11.710 --> 00:05:17.020 align:middle line:90% harmonic number. 00:05:17.020 --> 00:05:19.030 align:middle line:84% This is a standard name for this sum, 00:05:19.030 --> 00:05:24.910 align:middle line:84% 1 plus 1/2 plus 1/3 plus 1/4, up to 1 over n And 1/2 of that is 00:05:24.910 --> 00:05:29.740 align:middle line:84% exactly as far as we can get with n blocks, right? 00:05:29.740 --> 00:05:33.440 align:middle line:84% So now the question is, how big does this get? 00:05:33.440 --> 00:05:35.795 align:middle line:84% If I have 1,000 blocks, how far can I get? 00:05:35.795 --> 00:05:38.170 align:middle line:84% If I have a million blocks, if I have an unlimited number 00:05:38.170 --> 00:05:40.100 align:middle line:90% of blocks, how far can I get? 00:05:40.100 --> 00:05:44.410 align:middle line:84% Which means we're asking about this sum, this harmonic number. 00:05:44.410 --> 00:05:46.670 align:middle line:90% I want to know about that sum. 00:05:46.670 --> 00:05:48.740 align:middle line:90% Do we have a closed form for it? 00:05:48.740 --> 00:05:50.670 align:middle line:90% No, not really. 00:05:50.670 --> 00:05:53.220 align:middle line:84% But as we saw last time, we don't especially 00:05:53.220 --> 00:05:56.350 align:middle line:84% need a closed form if all we care about is approximation. 00:05:56.350 --> 00:05:58.175 align:middle line:90% So let's approximate that sum. 00:05:58.175 --> 00:06:01.080 align:middle line:90% 00:06:01.080 --> 00:06:05.400 align:middle line:84% So we have our sum Hn, which is defined 00:06:05.400 --> 00:06:10.770 align:middle line:84% as the sum of the first n reciprocals. 00:06:10.770 --> 00:06:12.402 align:middle line:90% And we want to approximate it. 00:06:12.402 --> 00:06:13.860 align:middle line:84% And can someone remind me what tool 00:06:13.860 --> 00:06:17.670 align:middle line:84% we have to approximate sums like this? 00:06:17.670 --> 00:06:18.390 align:middle line:90% Yes? 00:06:18.390 --> 00:06:18.840 align:middle line:90% AUDIENCE: The integral. 00:06:18.840 --> 00:06:21.000 align:middle line:84% ZACHARY ABEL: The integral method, absolutely. 00:06:21.000 --> 00:06:25.740 align:middle line:84% So this function 1 over x is a decreasing function. 00:06:25.740 --> 00:06:28.740 align:middle line:84% And so we have our integral method, our integral bound, 00:06:28.740 --> 00:06:37.999 align:middle line:84% that says that our sum Hn is between the integral 00:06:37.999 --> 00:06:39.390 align:middle line:90% plus the last term. 00:06:39.390 --> 00:06:41.490 align:middle line:90% In this case, the last term is-- 00:06:41.490 --> 00:06:46.830 align:middle line:84% sorry, plus the first term, 1, and the integral 1 to n 00:06:46.830 --> 00:06:52.540 align:middle line:84% of 1 over x dx plus the last term, which is 1 over n. 00:06:52.540 --> 00:06:57.160 align:middle line:84% So this bound is integral plus f of n. 00:06:57.160 --> 00:07:01.000 align:middle line:84% Less equal to the sum is less equal to the integral plus 00:07:01.000 --> 00:07:03.550 align:middle line:90% f of 1. 00:07:03.550 --> 00:07:07.810 align:middle line:84% And I claim this bound is really easy to remember. 00:07:07.810 --> 00:07:10.430 align:middle line:84% I plus fn, I plus f1, are pretty simple things. 00:07:10.430 --> 00:07:15.700 align:middle line:84% And also, if you're dealing with an increasing function instead 00:07:15.700 --> 00:07:18.590 align:middle line:84% of a decreasing function, it's the same two bounds, 00:07:18.590 --> 00:07:20.200 align:middle line:90% but they're on the other side. 00:07:20.200 --> 00:07:22.360 align:middle line:84% Just because if it's increasing, then fn 00:07:22.360 --> 00:07:23.900 align:middle line:90% is going to be bigger than f1. 00:07:23.900 --> 00:07:26.530 align:middle line:84% So certainly, fn should be on the bigger side. 00:07:26.530 --> 00:07:29.320 align:middle line:84% But this is a really clean bound. 00:07:29.320 --> 00:07:32.020 align:middle line:84% And we can write out what this says. 00:07:32.020 --> 00:07:37.880 align:middle line:84% Integral from 1 over x dx from 1 to n, that's ln of n. 00:07:37.880 --> 00:07:42.790 align:middle line:84% Let's assume we can do that without writing it down. 00:07:42.790 --> 00:07:44.900 align:middle line:84% You don't want to see me integrate live. 00:07:44.900 --> 00:07:46.980 align:middle line:90% That will go poorly. 00:07:46.980 --> 00:07:52.500 align:middle line:84% But what this says is that ln of n plus 1 over n 00:07:52.500 --> 00:08:01.050 align:middle line:84% is less equal to Hn, is less equal to ln of n plus 1. 00:08:01.050 --> 00:08:02.790 align:middle line:90% All right? 00:08:02.790 --> 00:08:04.640 align:middle line:90% Which is a pretty nice bound. 00:08:04.640 --> 00:08:07.560 align:middle line:90% 00:08:07.560 --> 00:08:11.040 align:middle line:90% But we're really interested in-- 00:08:11.040 --> 00:08:13.830 align:middle line:84% well, first, let's answer a question. 00:08:13.830 --> 00:08:16.530 align:middle line:84% If we want to get 10 blocks out, then we're 00:08:16.530 --> 00:08:19.830 align:middle line:90% asking for Hn to be about 20. 00:08:19.830 --> 00:08:21.790 align:middle line:84% Which means we're asking for these bounds. 00:08:21.790 --> 00:08:24.900 align:middle line:84% These bounds are basically, ln of n. 00:08:24.900 --> 00:08:28.420 align:middle line:84% The 1 over n and the 1 don't affect things all that much. 00:08:28.420 --> 00:08:34.350 align:middle line:84% So really, Hn is basically, ln of n. 00:08:34.350 --> 00:08:36.037 align:middle line:90% I'm being imprecise here. 00:08:36.037 --> 00:08:37.870 align:middle line:84% But we'll be more precise about it in a bit. 00:08:37.870 --> 00:08:40.500 align:middle line:84% But for now, it's about ln of n, which 00:08:40.500 --> 00:08:48.010 align:middle line:84% means to get 10 blocks out, we need Hn equals 20. 00:08:48.010 --> 00:08:52.270 align:middle line:84% Because the distance was 1/2 Hn, which means we need n is 00:08:52.270 --> 00:08:55.810 align:middle line:90% approximately e to the 20. 00:08:55.810 --> 00:08:58.510 align:middle line:90% We need that many blocks. 00:08:58.510 --> 00:09:06.310 align:middle line:84% We need about 400 billion blocks to get 10 blocks away 00:09:06.310 --> 00:09:07.780 align:middle line:90% from the table. 00:09:07.780 --> 00:09:10.250 align:middle line:84% Each one is about an inch and a half thick. 00:09:10.250 --> 00:09:12.250 align:middle line:90% So that gets us to-- 00:09:12.250 --> 00:09:13.600 align:middle line:90% carry the 1-- 00:09:13.600 --> 00:09:17.620 align:middle line:84% 40 times the distance to the moon. 00:09:17.620 --> 00:09:20.440 align:middle line:84% Probably going to run into some other issues before you're 00:09:20.440 --> 00:09:23.110 align:middle line:84% able to stack something that tall. 00:09:23.110 --> 00:09:26.540 align:middle line:84% So mathematically, yes, we can get 10 blocks over the table; 00:09:26.540 --> 00:09:28.370 align:middle line:90% physically, not so much. 00:09:28.370 --> 00:09:31.120 align:middle line:90% That's why you're both right. 00:09:31.120 --> 00:09:36.030 align:middle line:84% But all of this was really an excuse to get us here. 00:09:36.030 --> 00:09:42.600 align:middle line:84% Hn is approximately ln of n plus some smaller terms. 00:09:42.600 --> 00:09:45.360 align:middle line:84% And let's try to make that a little more precise. 00:09:45.360 --> 00:09:46.840 align:middle line:90% What do we mean by that? 00:09:46.840 --> 00:09:50.080 align:middle line:84% What do we mean by Hn is approximately ln of n? 00:09:50.080 --> 00:09:51.910 align:middle line:84% Well, let me give us a definition. 00:09:51.910 --> 00:10:01.170 align:middle line:90% 00:10:01.170 --> 00:10:03.930 align:middle line:90% Let's write that f of n-- 00:10:03.930 --> 00:10:08.560 align:middle line:84% that's a tilde-- f of n is tilde of g of n. 00:10:08.560 --> 00:10:14.720 align:middle line:84% So this is asymptotically equivalent. 00:10:14.720 --> 00:10:17.520 align:middle line:90% 00:10:17.520 --> 00:10:21.660 align:middle line:84% So f and g are asymptotically equivalent 00:10:21.660 --> 00:10:28.230 align:middle line:84% if and only if the limit of fn over g of n as n 00:10:28.230 --> 00:10:32.250 align:middle line:90% goes to infinity is 1. 00:10:32.250 --> 00:10:34.660 align:middle line:84% If you divide them and get a limit of 1, 00:10:34.660 --> 00:10:37.516 align:middle line:84% then these functions are asymptotically equivalent. 00:10:37.516 --> 00:10:41.360 align:middle line:90% 00:10:41.360 --> 00:10:45.320 align:middle line:84% And in this case, well, we have these nice bounds on Hn. 00:10:45.320 --> 00:10:46.890 align:middle line:84% Well, let's take that inequality. 00:10:46.890 --> 00:10:50.528 align:middle line:84% Let's divide all three sides by ln of n. 00:10:50.528 --> 00:10:51.570 align:middle line:90% What are we going to get? 00:10:51.570 --> 00:10:55.400 align:middle line:84% We're going to get 1 plus 1 over n ln of n 00:10:55.400 --> 00:11:01.430 align:middle line:84% is less equal to Hn over ln of n, is less equal to 1 plus 1 00:11:01.430 --> 00:11:04.820 align:middle line:90% over ln of n. 00:11:04.820 --> 00:11:10.070 align:middle line:84% And as we take the limit, as n goes to infinity, 00:11:10.070 --> 00:11:12.420 align:middle line:90% the left side looks like 1. 00:11:12.420 --> 00:11:15.080 align:middle line:84% This thing in the denominator gets big, so 1 over that 00:11:15.080 --> 00:11:15.720 align:middle line:90% gets small. 00:11:15.720 --> 00:11:19.070 align:middle line:90% The left side goes to 1. 00:11:19.070 --> 00:11:24.620 align:middle line:84% Lim n goes to infinity of Hn over ln of n. 00:11:24.620 --> 00:11:29.150 align:middle line:84% And on the right side, we have 1 plus something that goes to 0. 00:11:29.150 --> 00:11:31.240 align:middle line:84% Because that denominator gets arbitrarily big. 00:11:31.240 --> 00:11:35.150 align:middle line:90% 00:11:35.150 --> 00:11:38.160 align:middle line:90% So the limit equals 1. 00:11:38.160 --> 00:11:41.200 align:middle line:84% I learned this in high school as the squeeze theorem. 00:11:41.200 --> 00:11:43.530 align:middle line:84% If you can bound your limit between two things that 00:11:43.530 --> 00:11:46.500 align:middle line:84% are the same, then the limit is that thing. 00:11:46.500 --> 00:11:48.570 align:middle line:84% But what we've concluded here is that Hn 00:11:48.570 --> 00:11:53.580 align:middle line:84% divided by ln of n limits to 1 as n goes to infinity. 00:11:53.580 --> 00:12:01.920 align:middle line:84% And therefore-- sorry, Hn over ln of n 00:12:01.920 --> 00:12:06.780 align:middle line:84% equals 1, which implies that Hn is asymptotically 00:12:06.780 --> 00:12:10.860 align:middle line:90% equivalent to ln of n. 00:12:10.860 --> 00:12:13.470 align:middle line:84% So when I wrote approximately ln of n 00:12:13.470 --> 00:12:17.100 align:middle line:84% up there, probably what I meant was this much more precise 00:12:17.100 --> 00:12:20.100 align:middle line:84% thing, that these are approximately equal in the sense 00:12:20.100 --> 00:12:22.800 align:middle line:90% that their ratio limits to 1. 00:12:22.800 --> 00:12:24.840 align:middle line:90% Does that make sense? 00:12:24.840 --> 00:12:26.640 align:middle line:90% Nice. 00:12:26.640 --> 00:12:27.305 align:middle line:90% All right. 00:12:27.305 --> 00:12:29.880 align:middle line:90% 00:12:29.880 --> 00:12:34.080 align:middle line:84% Now, before moving on to asymptotics fully, 00:12:34.080 --> 00:12:35.930 align:middle line:84% there's one other topic in and summations 00:12:35.930 --> 00:12:38.268 align:middle line:84% that I want to talk about, which is not summations 00:12:38.268 --> 00:12:39.143 align:middle line:90% at all, but products. 00:12:39.143 --> 00:12:44.900 align:middle line:90% 00:12:44.900 --> 00:12:47.420 align:middle line:84% And really, what I want to say about products 00:12:47.420 --> 00:12:52.310 align:middle line:84% is that often, you can analyze products as if they were sums, 00:12:52.310 --> 00:12:56.660 align:middle line:90% turn into sums. 00:12:56.660 --> 00:13:00.110 align:middle line:84% For example, if we care about n factorial, 00:13:00.110 --> 00:13:06.260 align:middle line:84% which is here, 1 times 2 times 3, all the way up to n-- 00:13:06.260 --> 00:13:08.380 align:middle line:84% who can tell me how to turn this into a sum? 00:13:08.380 --> 00:13:15.500 align:middle line:90% 00:13:15.500 --> 00:13:18.920 align:middle line:84% It's something written on this board. 00:13:18.920 --> 00:13:19.653 align:middle line:90% Yes? 00:13:19.653 --> 00:13:20.570 align:middle line:90% AUDIENCE: [INAUDIBLE]. 00:13:20.570 --> 00:13:22.110 align:middle line:90% ZACHARY ABEL: Yeah, take logs. 00:13:22.110 --> 00:13:22.790 align:middle line:90% Perfect. 00:13:22.790 --> 00:13:25.700 align:middle line:84% What happens if we take the log of both sides? 00:13:25.700 --> 00:13:28.460 align:middle line:90% So log of n factorial-- 00:13:28.460 --> 00:13:30.060 align:middle line:90% here, let's be precise. 00:13:30.060 --> 00:13:35.370 align:middle line:84% We'll call it ln of n factorial, the natural log, so log base e. 00:13:35.370 --> 00:13:38.320 align:middle line:84% Well, remember, when you take logs products turn to sums. 00:13:38.320 --> 00:13:44.040 align:middle line:84% In this sense, this becomes ln of 1 plus ln of 2 plus ln of 3 00:13:44.040 --> 00:13:48.150 align:middle line:90% plus ln of n. 00:13:48.150 --> 00:13:50.220 align:middle line:84% So if we want to understand n factorial, 00:13:50.220 --> 00:13:52.180 align:middle line:84% we could just as well understand its log. 00:13:52.180 --> 00:13:54.670 align:middle line:84% And its log is the sum of a bunch of other logs. 00:13:54.670 --> 00:13:55.690 align:middle line:90% And now we have a sum. 00:13:55.690 --> 00:13:58.590 align:middle line:84% And we can use all of our summation tools 00:13:58.590 --> 00:14:02.400 align:middle line:84% on this sum to get information about n factorial. 00:14:02.400 --> 00:14:06.090 align:middle line:84% For example, we could use the integral bound. 00:14:06.090 --> 00:14:08.622 align:middle line:84% ln of x is an increasing function this time, 00:14:08.622 --> 00:14:10.080 align:middle line:84% so we'll use the increasing version 00:14:10.080 --> 00:14:14.850 align:middle line:90% of the bound, which says that-- 00:14:14.850 --> 00:14:15.670 align:middle line:90% here we go. 00:14:15.670 --> 00:14:25.620 align:middle line:84% So ln of 1 plus the integral from 1 to n of ln of x dx 00:14:25.620 --> 00:14:29.895 align:middle line:84% is less equal to the sum of ln of k. 00:14:29.895 --> 00:14:33.560 align:middle line:84% k equals 1 to n, which is less equal to ln 00:14:33.560 --> 00:14:37.310 align:middle line:90% of n plus the same integral. 00:14:37.310 --> 00:14:39.373 align:middle line:84% And if we drew out that integral-- like I said, 00:14:39.373 --> 00:14:41.040 align:middle line:84% you don't want to see me integrate live. 00:14:41.040 --> 00:14:42.800 align:middle line:90% I will get it wrong. 00:14:42.800 --> 00:14:45.290 align:middle line:84% But if we trust my notes, this integral 00:14:45.290 --> 00:14:52.655 align:middle line:84% comes out to n ln of n minus n plus 1. 00:14:52.655 --> 00:14:55.280 align:middle line:90% 00:14:55.280 --> 00:14:59.970 align:middle line:84% Ln of 1 is 0, thankfully, keeps things simple. 00:14:59.970 --> 00:15:06.380 align:middle line:84% And so what we end up finding is that n ln of n minus n plus 1 00:15:06.380 --> 00:15:09.750 align:middle line:84% is less equal to this sum, which, as we know, 00:15:09.750 --> 00:15:12.670 align:middle line:84% is ln of n factorial, which is less 00:15:12.670 --> 00:15:17.470 align:middle line:84% equal to this same integral plus another ln of n, 00:15:17.470 --> 00:15:23.120 align:middle line:90% n plus 1 ln of n minus n plus 1. 00:15:23.120 --> 00:15:26.100 align:middle line:84% So we were able to translate this product into a sum 00:15:26.100 --> 00:15:30.640 align:middle line:84% and then use our summation tool to get us a bound on ln of n 00:15:30.640 --> 00:15:32.540 align:middle line:84% And if we want a bound on n factor-- 00:15:32.540 --> 00:15:33.800 align:middle line:90% sorry, ln of n factorial. 00:15:33.800 --> 00:15:34.750 align:middle line:90% I misspoke. 00:15:34.750 --> 00:15:36.790 align:middle line:90% If we want a bound on ln-- 00:15:36.790 --> 00:15:38.440 align:middle line:90% I misspoke again, apologies. 00:15:38.440 --> 00:15:41.030 align:middle line:84% If we want a bound on n factorial itself, 00:15:41.030 --> 00:15:44.830 align:middle line:84% we can just undo the log by exponentiating all three 00:15:44.830 --> 00:15:45.860 align:middle line:90% of these. 00:15:45.860 --> 00:15:48.610 align:middle line:84% Take e to this, e to this, e to this. 00:15:48.610 --> 00:15:50.590 align:middle line:90% That preserves our inequalities. 00:15:50.590 --> 00:15:52.600 align:middle line:84% And then what we'll end up getting, 00:15:52.600 --> 00:15:56.650 align:middle line:84% after we simplify it a bit, is that n to the n over e 00:15:56.650 --> 00:16:00.460 align:middle line:84% to the n minus 1 is less equal to n factorial, 00:16:00.460 --> 00:16:06.525 align:middle line:84% is less equal to n to the n plus 1 over e to the n minus 1. 00:16:06.525 --> 00:16:09.550 align:middle line:90% 00:16:09.550 --> 00:16:12.250 align:middle line:84% So same denominator on both sides, we have n to the n 00:16:12.250 --> 00:16:14.780 align:middle line:84% versus n to the n plus 1 on the two sides. 00:16:14.780 --> 00:16:17.590 align:middle line:84% So the two bounds are off by a factor of n, 00:16:17.590 --> 00:16:19.580 align:middle line:90% not as good as we had last time. 00:16:19.580 --> 00:16:24.970 align:middle line:84% We were off by an additive 1, less than that even. 00:16:24.970 --> 00:16:28.720 align:middle line:84% But a factor of n isn't bad for a number as big as n factorial. 00:16:28.720 --> 00:16:32.160 align:middle line:90% This isn't a bad bound. 00:16:32.160 --> 00:16:35.790 align:middle line:84% Is everyone clear on how we got to this bound? 00:16:35.790 --> 00:16:36.810 align:middle line:90% Wonderful. 00:16:36.810 --> 00:16:39.010 align:middle line:90% It's not the best bound we have. 00:16:39.010 --> 00:16:41.520 align:middle line:90% Ow. 00:16:41.520 --> 00:16:45.250 align:middle line:90% Let me tell you two better ones. 00:16:45.250 --> 00:16:48.330 align:middle line:84% Oh, sorry, before that, is this bound 00:16:48.330 --> 00:16:51.030 align:middle line:84% enough to say that n factorial is asymptotically 00:16:51.030 --> 00:16:55.680 align:middle line:84% equivalent to one side or the other? 00:16:55.680 --> 00:16:57.430 align:middle line:90% I see some head shakes. 00:16:57.430 --> 00:16:58.505 align:middle line:90% Can someone tell me why? 00:16:58.505 --> 00:17:07.200 align:middle line:90% 00:17:07.200 --> 00:17:07.990 align:middle line:90% That's all right. 00:17:07.990 --> 00:17:10.650 align:middle line:90% I'm tired, too. 00:17:10.650 --> 00:17:13.300 align:middle line:84% These two bounds are too far apart from each other. 00:17:13.300 --> 00:17:15.660 align:middle line:90% They're a factor of n apart. 00:17:15.660 --> 00:17:20.099 align:middle line:84% And so we can't say that this divided by this limits to 1 00:17:20.099 --> 00:17:22.359 align:middle line:84% or this divided by this limits to 1. 00:17:22.359 --> 00:17:25.030 align:middle line:84% Because we have an n range in there to account for. 00:17:25.030 --> 00:17:26.460 align:middle line:90% And n goes to infinity. 00:17:26.460 --> 00:17:30.030 align:middle line:84% So we don't know where in this range n factorial lies. 00:17:30.030 --> 00:17:34.290 align:middle line:84% And so we can't pin down that limit going to a specific value. 00:17:34.290 --> 00:17:36.080 align:middle line:84% So this isn't tight enough to say 00:17:36.080 --> 00:17:38.240 align:middle line:84% that n factorial is asymptotically 00:17:38.240 --> 00:17:42.170 align:middle line:90% equivalent to something. 00:17:42.170 --> 00:17:45.385 align:middle line:84% But that's where Stirling's approximation comes in. 00:17:45.385 --> 00:17:54.890 align:middle line:90% 00:17:54.890 --> 00:17:57.230 align:middle line:84% Stirling's approximation says that n factorial 00:17:57.230 --> 00:18:03.230 align:middle line:84% is asymptotically equivalent to the square root of 2 pi n 00:18:03.230 --> 00:18:09.020 align:middle line:90% times n over e to the n. 00:18:09.020 --> 00:18:11.370 align:middle line:84% We're not going to prove that, but it's really useful. 00:18:11.370 --> 00:18:12.412 align:middle line:90% You should write it down. 00:18:12.412 --> 00:18:16.490 align:middle line:84% You might end up using it in the problem set, hint, hint, 00:18:16.490 --> 00:18:19.010 align:middle line:90% extremely useful. 00:18:19.010 --> 00:18:21.710 align:middle line:84% But remember, this is just saying that n factorial 00:18:21.710 --> 00:18:24.300 align:middle line:90% divided by this limits to 1. 00:18:24.300 --> 00:18:27.740 align:middle line:84% So when n is really, really big, the quotient of these two things 00:18:27.740 --> 00:18:29.360 align:middle line:90% is close to 1. 00:18:29.360 --> 00:18:33.450 align:middle line:84% What it doesn't tell us is what happens for smaller values of n. 00:18:33.450 --> 00:18:35.190 align:middle line:90% What if n is just 10? 00:18:35.190 --> 00:18:39.080 align:middle line:84% Can we use this to approximate 10 factorial? 00:18:39.080 --> 00:18:40.020 align:middle line:90% How about 100? 00:18:40.020 --> 00:18:40.740 align:middle line:90% How about 1,000. 00:18:40.740 --> 00:18:41.130 align:middle line:90% I don't know. 00:18:41.130 --> 00:18:42.360 align:middle line:90% This doesn't really tell us. 00:18:42.360 --> 00:18:45.180 align:middle line:84% It doesn't tell us how good an approximation this is. 00:18:45.180 --> 00:18:49.700 align:middle line:84% All it says is that it's good in the limit. 00:18:49.700 --> 00:18:53.840 align:middle line:84% So let me tell you an even stronger bound. 00:18:53.840 --> 00:18:55.970 align:middle line:84% I don't know who to credit this bound to. 00:18:55.970 --> 00:18:59.480 align:middle line:84% But it's even more precise than this one. 00:18:59.480 --> 00:19:06.860 align:middle line:84% It says that n factorial divided by this thing, 2 pi n, n over e 00:19:06.860 --> 00:19:07.950 align:middle line:90% to the n-- 00:19:07.950 --> 00:19:12.080 align:middle line:84% so this is a thing that Stirling tells us limits to 1. 00:19:12.080 --> 00:19:15.440 align:middle line:84% And now we have some bounds on how quickly it does that, 00:19:15.440 --> 00:19:23.650 align:middle line:84% e to the 1 over 12n, e to the 1 over 12n plus 1. 00:19:23.650 --> 00:19:25.820 align:middle line:84% So not only does this limit to 1, 00:19:25.820 --> 00:19:29.560 align:middle line:84% it does so very, very fast, exponentially fast. 00:19:29.560 --> 00:19:31.990 align:middle line:84% And this, we can plug in specific values of n 00:19:31.990 --> 00:19:35.390 align:middle line:84% and see, OK, how close is this approximation to n factorial? 00:19:35.390 --> 00:19:39.000 align:middle line:84% This is an effective bound, one we can get actual approximations 00:19:39.000 --> 00:19:39.500 align:middle line:90% out of. 00:19:39.500 --> 00:19:41.560 align:middle line:90% This is just a limiting bound. 00:19:41.560 --> 00:19:43.700 align:middle line:84% But of course, this is a lot more complicated. 00:19:43.700 --> 00:19:46.033 align:middle line:84% Sometimes, we don't want to deal with all these details, 00:19:46.033 --> 00:19:47.110 align:middle line:90% and this is enough. 00:19:47.110 --> 00:19:49.880 align:middle line:84% Sometimes, even that's too much, and that's enough. 00:19:49.880 --> 00:19:53.650 align:middle line:84% So pick your level of granularity as you need it. 00:19:53.650 --> 00:19:56.560 align:middle line:84% We're also not going to prove this bound, that's 00:19:56.560 --> 00:19:58.930 align:middle line:90% even harder than Stirling. 00:19:58.930 --> 00:20:00.790 align:middle line:90% But it's true and useful. 00:20:00.790 --> 00:20:01.920 align:middle line:90% Questions on that? 00:20:01.920 --> 00:20:05.410 align:middle line:90% 00:20:05.410 --> 00:20:06.350 align:middle line:90% All right. 00:20:06.350 --> 00:20:09.290 align:middle line:84% So that's everything I wanted to say about summations. 00:20:09.290 --> 00:20:12.260 align:middle line:84% And now let's jump back to asymptotics. 00:20:12.260 --> 00:20:14.320 align:middle line:84% We already started talking about asymptotics 00:20:14.320 --> 00:20:18.160 align:middle line:90% with asymptotic equivalence. 00:20:18.160 --> 00:20:19.940 align:middle line:90% And let's do an experiment. 00:20:19.940 --> 00:20:21.920 align:middle line:90% Let's do an exercise. 00:20:21.920 --> 00:20:24.120 align:middle line:84% We've already seen a sorting algorithm. 00:20:24.120 --> 00:20:29.010 align:middle line:84% We saw swap sort in lecture 4, where you have a list. 00:20:29.010 --> 00:20:30.930 align:middle line:84% You want to put it in sorted order. 00:20:30.930 --> 00:20:32.900 align:middle line:84% And you do that by finding a pair that's 00:20:32.900 --> 00:20:36.300 align:middle line:84% out of order and next to each other and changing their order. 00:20:36.300 --> 00:20:37.730 align:middle line:90% And you keep doing that, Keep. 00:20:37.730 --> 00:20:40.610 align:middle line:84% Finding pairs that are next to each other and out of order, 00:20:40.610 --> 00:20:44.158 align:middle line:90% until the whole thing is sorted. 00:20:44.158 --> 00:20:45.450 align:middle line:90% So I'm going to call that swap. 00:20:45.450 --> 00:20:48.380 align:middle line:90% 00:20:48.380 --> 00:20:52.100 align:middle line:84% We saw that the number of swaps that you'll need to make 00:20:52.100 --> 00:20:55.100 align:middle line:90% is exactly equal to-- 00:20:55.100 --> 00:20:59.510 align:middle line:84% the number of swaps is exactly equal to the number 00:20:59.510 --> 00:21:05.750 align:middle line:84% of inverted pairs; so the number of pairs 00:21:05.750 --> 00:21:09.170 align:middle line:84% of two numbers in our list, where the big one is somewhere 00:21:09.170 --> 00:21:11.095 align:middle line:90% to the left of the small one. 00:21:11.095 --> 00:21:13.220 align:middle line:84% So they're in the wrong order, and we'll eventually 00:21:13.220 --> 00:21:16.580 align:middle line:90% need to change order. 00:21:16.580 --> 00:21:18.320 align:middle line:84% And, well, how many inverted pairs 00:21:18.320 --> 00:21:20.480 align:middle line:84% could there be in the worst case? 00:21:20.480 --> 00:21:23.720 align:middle line:84% Well, in the worst case, every pair is inverted. 00:21:23.720 --> 00:21:29.390 align:middle line:84% 1, the number 1, is inverted with 2 through n. 00:21:29.390 --> 00:21:30.817 align:middle line:90% And 2 is inverted. 00:21:30.817 --> 00:21:32.400 align:middle line:84% Well, it's already inverted through 1. 00:21:32.400 --> 00:21:33.600 align:middle line:90% We already counted that. 00:21:33.600 --> 00:21:37.790 align:middle line:84% But 2 would be inverted with 3 through n. 00:21:37.790 --> 00:21:41.420 align:middle line:84% Those are new inversions that we haven't counted yet. 00:21:41.420 --> 00:21:48.300 align:middle line:84% 3 is inverted with everything after it plus 1. 00:21:48.300 --> 00:21:52.790 align:middle line:84% So this is the worst case number of swaps we'll have to make. 00:21:52.790 --> 00:21:56.310 align:middle line:84% And as we've seen, we know how to compute this sum. 00:21:56.310 --> 00:21:59.780 align:middle line:84% This is n minus 1 times n divided by 2, 00:21:59.780 --> 00:22:03.740 align:middle line:84% also known as n squared over 2 minus n over 2. 00:22:03.740 --> 00:22:07.530 align:middle line:84% And this, I'm going to call S of n. 00:22:07.530 --> 00:22:11.750 align:middle line:84% We have our function S of n, S being reminiscent of swap sort. 00:22:11.750 --> 00:22:14.930 align:middle line:84% It's a function describing the worst case 00:22:14.930 --> 00:22:18.780 align:middle line:84% number of swaps we'll need to make in swap sort. 00:22:18.780 --> 00:22:22.260 align:middle line:84% On Thursday, we're going to see another sorting algorithm called 00:22:22.260 --> 00:22:28.480 align:middle line:84% merge sort I'll describe it then. 00:22:28.480 --> 00:22:29.880 align:middle line:84% I'm not going to describe it now. 00:22:29.880 --> 00:22:32.430 align:middle line:84% We don't need to know what it does or how. 00:22:32.430 --> 00:22:34.990 align:middle line:84% We just need to know how fast it runs. 00:22:34.990 --> 00:22:38.010 align:middle line:84% How many comparisons does it have to make? 00:22:38.010 --> 00:22:41.650 align:middle line:90% So let's define m of n. 00:22:41.650 --> 00:22:48.090 align:middle line:84% This is going to be n log 2 of n minus n plus 1. 00:22:48.090 --> 00:22:51.585 align:middle line:84% And on Thursday, we'll see that merge sort 00:22:51.585 --> 00:22:55.950 align:middle line:84% takes at most, this many steps to complete. 00:22:55.950 --> 00:22:59.080 align:middle line:84% And now I want to know, between these two functions, 00:22:59.080 --> 00:23:01.170 align:middle line:84% between these two algorithms, which 00:23:01.170 --> 00:23:05.550 align:middle line:84% one is a better choice for sorting lots of numbers? 00:23:05.550 --> 00:23:08.350 align:middle line:84% If these are the two algorithms I have to pick from, 00:23:08.350 --> 00:23:12.540 align:middle line:84% and I know their run times, they're n squared over 2 minus n 00:23:12.540 --> 00:23:19.660 align:middle line:84% over 2 versus n log 2n minus n plus 1. 00:23:19.660 --> 00:23:23.020 align:middle line:90% This is S. This is m. 00:23:23.020 --> 00:23:25.750 align:middle line:90% Which one's the better choice? 00:23:25.750 --> 00:23:28.570 align:middle line:90% And sorry, that was rhetorical. 00:23:28.570 --> 00:23:30.340 align:middle line:90% Let's talk through it. 00:23:30.340 --> 00:23:33.190 align:middle line:84% Let's do an eye tracker to see which parts 00:23:33.190 --> 00:23:36.850 align:middle line:84% of the various formulas we look at when making this comparison. 00:23:36.850 --> 00:23:40.420 align:middle line:84% And just to emphasize, I'm most worried about what 00:23:40.420 --> 00:23:42.740 align:middle line:84% happens with lots and lots and lots of numbers. 00:23:42.740 --> 00:23:46.690 align:middle line:84% I'm trying to sort a million, a billion, a trillion numbers. 00:23:46.690 --> 00:23:49.150 align:middle line:84% So I'm interested in what happens in the limit 00:23:49.150 --> 00:23:50.830 align:middle line:90% as n gets really, really big. 00:23:50.830 --> 00:23:54.753 align:middle line:84% If we're trying to 10 numbers, well, I don't care. 00:23:54.753 --> 00:23:56.170 align:middle line:84% However you want to do it is fine. 00:23:56.170 --> 00:23:58.270 align:middle line:90% 10 is small. 00:23:58.270 --> 00:24:00.640 align:middle line:90% But with big data-- 00:24:00.640 --> 00:24:02.570 align:middle line:90% no one uses that term anymore. 00:24:02.570 --> 00:24:04.906 align:middle line:90% I don't know why. 00:24:04.906 --> 00:24:09.950 align:middle line:84% With lots and lots of data, these differences matter. 00:24:09.950 --> 00:24:12.230 align:middle line:84% So let's see how we make this comparison. 00:24:12.230 --> 00:24:13.780 align:middle line:90% What are we going to look at? 00:24:13.780 --> 00:24:17.470 align:middle line:84% Well, the first thing my eye goes to 00:24:17.470 --> 00:24:20.080 align:middle line:84% is this right here, which is to say, 00:24:20.080 --> 00:24:21.890 align:middle line:84% that's the last place my eye goes. 00:24:21.890 --> 00:24:24.560 align:middle line:90% I don't care about that term. 00:24:24.560 --> 00:24:29.650 align:middle line:84% The n squared over 2 is much more important. 00:24:29.650 --> 00:24:32.110 align:middle line:84% By the time we're up into the n squareds, 00:24:32.110 --> 00:24:34.540 align:middle line:84% a little, measly n over 2 doesn't really 00:24:34.540 --> 00:24:36.310 align:middle line:90% make much of a difference. 00:24:36.310 --> 00:24:37.070 align:middle line:90% Right? 00:24:37.070 --> 00:24:39.560 align:middle line:84% Intuitively, anyway, I haven't made this precise. 00:24:39.560 --> 00:24:42.820 align:middle line:84% But the first thing that I do, when looking at this, is I just 00:24:42.820 --> 00:24:44.420 align:middle line:90% completely ignore that n over 2. 00:24:44.420 --> 00:24:45.820 align:middle line:90% I don't care. 00:24:45.820 --> 00:24:48.940 align:middle line:84% Likewise, I don't care about this n minus 1 00:24:48.940 --> 00:24:50.080 align:middle line:90% that we're subtracting. 00:24:50.080 --> 00:24:54.610 align:middle line:84% The n log n grows faster than that. 00:24:54.610 --> 00:24:56.930 align:middle line:84% So with big N, this is going to dominate. 00:24:56.930 --> 00:24:59.140 align:middle line:84% And this is just going to be a rounding error. 00:24:59.140 --> 00:25:02.680 align:middle line:84% So that's the first thing I do, ignore the lower terms. 00:25:02.680 --> 00:25:06.910 align:middle line:84% Now I'm just comparing n squared over 2 versus n log n. 00:25:06.910 --> 00:25:12.730 align:middle line:84% The next thing I ignore is this over 2. 00:25:12.730 --> 00:25:15.160 align:middle line:90% I don't really care. 00:25:15.160 --> 00:25:17.030 align:middle line:84% I don't care that we're dividing by 2. 00:25:17.030 --> 00:25:20.530 align:middle line:84% If this had been n squared divided by 1,000, 00:25:20.530 --> 00:25:24.970 align:middle line:84% if this had been a million times n log n, 00:25:24.970 --> 00:25:28.930 align:middle line:84% is that really going to change my determination? 00:25:28.930 --> 00:25:31.610 align:middle line:84% For small n values, yeah, those constants matter. 00:25:31.610 --> 00:25:34.790 align:middle line:84% Those lower terms matter, for that matter, when n is small. 00:25:34.790 --> 00:25:37.780 align:middle line:84% But in the limit, as n gets bigger and bigger, 00:25:37.780 --> 00:25:40.870 align:middle line:84% that extra factor of n is going to overtake 00:25:40.870 --> 00:25:42.820 align:middle line:84% that extra factor of log n, no matter what 00:25:42.820 --> 00:25:45.130 align:middle line:90% constants we throw in there. 00:25:45.130 --> 00:25:46.720 align:middle line:90% Right? 00:25:46.720 --> 00:25:49.430 align:middle line:84% So I don't really care about those as well. 00:25:49.430 --> 00:25:51.580 align:middle line:84% Really, all I'm focusing on is the n 00:25:51.580 --> 00:25:55.030 align:middle line:90% squared versus the n log n. 00:25:55.030 --> 00:25:57.970 align:middle line:90% Does that make sense? 00:25:57.970 --> 00:26:00.410 align:middle line:84% Everything I just did was completely informal. 00:26:00.410 --> 00:26:02.330 align:middle line:90% I haven't justified any of that. 00:26:02.330 --> 00:26:03.490 align:middle line:90% But we will. 00:26:03.490 --> 00:26:05.510 align:middle line:84% But first, we need to motivate it. 00:26:05.510 --> 00:26:09.660 align:middle line:84% And there's one other reason, one other motivation, 00:26:09.660 --> 00:26:14.910 align:middle line:84% for not really caring about this constant factor. 00:26:14.910 --> 00:26:17.770 align:middle line:84% With swap sort, what we're we counting, 00:26:17.770 --> 00:26:21.060 align:middle line:84% the number of actual swaps we make. 00:26:21.060 --> 00:26:23.290 align:middle line:84% We didn't say how you're finding those swaps. 00:26:23.290 --> 00:26:25.030 align:middle line:84% But there are clean ways to do it, 00:26:25.030 --> 00:26:28.620 align:middle line:84% so that the number of steps you take in the whole algorithm 00:26:28.620 --> 00:26:31.680 align:middle line:90% is about n squared over 2. 00:26:31.680 --> 00:26:34.740 align:middle line:84% But wait, we have to do more steps 00:26:34.740 --> 00:26:36.040 align:middle line:90% than just the swaps, right? 00:26:36.040 --> 00:26:38.710 align:middle line:84% We have to read numbers from memory. 00:26:38.710 --> 00:26:41.590 align:middle line:90% We have to compare them. 00:26:41.590 --> 00:26:43.330 align:middle line:84% We have to write numbers to memory. 00:26:43.330 --> 00:26:45.010 align:middle line:90% We have to increment our index. 00:26:45.010 --> 00:26:46.890 align:middle line:84% We have lots and lots of instructions 00:26:46.890 --> 00:26:49.120 align:middle line:84% that we have to do during this algorithm. 00:26:49.120 --> 00:26:52.080 align:middle line:84% So maybe it's just those four instructions 00:26:52.080 --> 00:26:53.260 align:middle line:90% every time we make a swap. 00:26:53.260 --> 00:27:00.000 align:middle line:84% So it's really 4 times n squared over 2 instructions. 00:27:00.000 --> 00:27:02.800 align:middle line:84% Instead of counting swaps, we're now counting instructions. 00:27:02.800 --> 00:27:05.470 align:middle line:84% And I'm being very loose about that. 00:27:05.470 --> 00:27:07.640 align:middle line:90% But it doesn't matter. 00:27:07.640 --> 00:27:08.960 align:middle line:90% OK. 00:27:08.960 --> 00:27:13.040 align:middle line:84% Well, not all instructions are created equal. 00:27:13.040 --> 00:27:15.110 align:middle line:84% Every instruction on your CPU happens 00:27:15.110 --> 00:27:17.900 align:middle line:90% on a fixed, regular clock. 00:27:17.900 --> 00:27:22.100 align:middle line:90% And maybe adds take-- 00:27:22.100 --> 00:27:23.940 align:middle line:84% when we have to increment our index, 00:27:23.940 --> 00:27:27.740 align:middle line:84% maybe that takes, I don't know, five cycles. 00:27:27.740 --> 00:27:31.700 align:middle line:90% And comparisons take 15 cycles. 00:27:31.700 --> 00:27:39.140 align:middle line:84% And read and writes to memory take, I don't know, 200 cycles. 00:27:39.140 --> 00:27:41.310 align:middle line:84% Because memory is a lot slower than arithmetic. 00:27:41.310 --> 00:27:43.140 align:middle line:84% I made up these numbers, by the way. 00:27:43.140 --> 00:27:43.980 align:middle line:90% These aren't real. 00:27:43.980 --> 00:27:46.340 align:middle line:90% Don't trust me on that. 00:27:46.340 --> 00:27:50.090 align:middle line:84% But in this fiction, if we're counting cycles instead 00:27:50.090 --> 00:27:53.090 align:middle line:84% of instructions, we're doing one of each of these 00:27:53.090 --> 00:27:55.850 align:middle line:90% per round or something. 00:27:55.850 --> 00:28:00.860 align:middle line:84% So really, we're doing, if I did my math right, about 210 times n 00:28:00.860 --> 00:28:02.437 align:middle line:90% squared cycles. 00:28:02.437 --> 00:28:06.085 align:middle line:90% 00:28:06.085 --> 00:28:07.210 align:middle line:90% So we're changing our unit. 00:28:07.210 --> 00:28:09.270 align:middle line:84% We're measuring different things. 00:28:09.270 --> 00:28:10.720 align:middle line:90% Let's measure something else. 00:28:10.720 --> 00:28:13.750 align:middle line:84% Say we're on a 2.4 gigahertz processor, 00:28:13.750 --> 00:28:17.800 align:middle line:90% 2.4 gigahertz processor. 00:28:17.800 --> 00:28:21.490 align:middle line:84% How long does this take in real life time? 00:28:21.490 --> 00:28:29.250 align:middle line:84% Well, it takes about 210 divided by 2.4 billion seconds-- 00:28:29.250 --> 00:28:31.453 align:middle line:90% sorry, times n squared seconds. 00:28:31.453 --> 00:28:35.295 align:middle line:90% 00:28:35.295 --> 00:28:37.030 align:middle line:84% So if we change what we're measuring, 00:28:37.030 --> 00:28:41.790 align:middle line:84% if we change our units, instructions, cycles, seconds, 00:28:41.790 --> 00:28:44.460 align:middle line:84% the thing that stays is the n squared. 00:28:44.460 --> 00:28:47.430 align:middle line:84% The constant changes, but it's always 00:28:47.430 --> 00:28:49.620 align:middle line:90% proportional to n squared. 00:28:49.620 --> 00:28:55.050 align:middle line:84% Get a better CPU, higher throughput, 00:28:55.050 --> 00:28:57.820 align:middle line:84% your time decreases, but only by a constant factor. 00:28:57.820 --> 00:29:00.060 align:middle line:84% It's still proportional to n squared. 00:29:00.060 --> 00:29:03.420 align:middle line:84% This is another reason that it's useful to be able to just ignore 00:29:03.420 --> 00:29:04.120 align:middle line:90% that constant. 00:29:04.120 --> 00:29:08.610 align:middle line:84% I don't care what it is, because I don't really care specifically 00:29:08.610 --> 00:29:11.550 align:middle line:90% what units we're measuring. 00:29:11.550 --> 00:29:13.290 align:middle line:84% I just care that it's proportional 00:29:13.290 --> 00:29:15.630 align:middle line:90% to the right answer. 00:29:15.630 --> 00:29:19.770 align:middle line:84% Does that make sense, why we might want to throw away 00:29:19.770 --> 00:29:21.300 align:middle line:90% information like this? 00:29:21.300 --> 00:29:23.940 align:middle line:84% Because it's easier to think about. 00:29:23.940 --> 00:29:27.450 align:middle line:84% N squared is easier to think about than all this other junk 00:29:27.450 --> 00:29:29.520 align:middle line:90% cluttering it up. 00:29:29.520 --> 00:29:35.400 align:middle line:84% One more motivating example, what's the population of the US? 00:29:35.400 --> 00:29:37.210 align:middle line:84% Does anyone know, just off the top? 00:29:37.210 --> 00:29:38.023 align:middle line:90% Yes? 00:29:38.023 --> 00:29:38.940 align:middle line:90% AUDIENCE: 330 million. 00:29:38.940 --> 00:29:44.350 align:middle line:84% ZACHARY ABEL: About 330 million, absolutely, about 335-- 00:29:44.350 --> 00:29:45.450 align:middle line:90% what is it? 00:29:45.450 --> 00:29:46.620 align:middle line:90% I wrote it down-- 00:29:46.620 --> 00:29:56.680 align:middle line:84% 893,238 is the population of the US as of January 1, 2024, 00:29:56.680 --> 00:29:59.790 align:middle line:84% according to the US Census Bureau. 00:29:59.790 --> 00:30:02.710 align:middle line:84% Is that more useful than what your colleague said, 00:30:02.710 --> 00:30:04.980 align:middle line:90% 330 million? 00:30:04.980 --> 00:30:07.320 align:middle line:84% I don't get any more information out 00:30:07.320 --> 00:30:09.610 align:middle line:84% of that in my measly, little brain. 00:30:09.610 --> 00:30:12.540 align:middle line:84% 330, I already have trouble conceiving 00:30:12.540 --> 00:30:13.810 align:middle line:90% of a number that large. 00:30:13.810 --> 00:30:17.070 align:middle line:84% I don't need all the extra detail. 00:30:17.070 --> 00:30:20.250 align:middle line:84% Same for algorithm analysis, I want to be able to just focus 00:30:20.250 --> 00:30:23.970 align:middle line:84% in on the parts that matter and have the option to strip off 00:30:23.970 --> 00:30:25.650 align:middle line:90% the parts that don't. 00:30:25.650 --> 00:30:27.100 align:middle line:90% So that's our motivation. 00:30:27.100 --> 00:30:31.890 align:middle line:84% We want to be able to communicate less information 00:30:31.890 --> 00:30:34.000 align:middle line:84% about the runtimes of our algorithms, 00:30:34.000 --> 00:30:37.320 align:middle line:84% or about other functions, while still being mathematically 00:30:37.320 --> 00:30:41.710 align:middle line:84% precise about it and making sure we know what was lost. 00:30:41.710 --> 00:30:43.230 align:middle line:84% What information are we retaining, 00:30:43.230 --> 00:30:47.200 align:middle line:84% and what information are we losing, never to get again? 00:30:47.200 --> 00:30:51.460 align:middle line:84% Because sometimes, you need that extra information. 00:30:51.460 --> 00:30:55.410 align:middle line:84% Throwing away all this stuff is a great, broad-strokes way 00:30:55.410 --> 00:30:58.120 align:middle line:90% to compare certain algorithms. 00:30:58.120 --> 00:30:59.720 align:middle line:84% But if we have two algorithms that 00:30:59.720 --> 00:31:02.360 align:middle line:84% are both proportional to n log n, oh well, now 00:31:02.360 --> 00:31:03.860 align:middle line:84% that constant factor really matters. 00:31:03.860 --> 00:31:06.090 align:middle line:84% And we better go and figure out what it is. 00:31:06.090 --> 00:31:09.410 align:middle line:84% Likewise, if we're comparing the US population 00:31:09.410 --> 00:31:12.090 align:middle line:84% between this year and last year, well, 00:31:12.090 --> 00:31:15.870 align:middle line:84% last year, it was also about 330 million and change. 00:31:15.870 --> 00:31:19.350 align:middle line:84% So if all we know is 330, 330, we can't compare them. 00:31:19.350 --> 00:31:22.430 align:middle line:84% We don't know if it went up or down or by how much. 00:31:22.430 --> 00:31:33.020 align:middle line:84% Turns out, last year, it was about 334.2 million. 00:31:33.020 --> 00:31:36.060 align:middle line:84% So now we had a reason that we wanted more information, 00:31:36.060 --> 00:31:38.390 align:middle line:84% wanted more precise measurements of the population 00:31:38.390 --> 00:31:41.270 align:middle line:90% than just 330 million. 00:31:41.270 --> 00:31:43.170 align:middle line:84% But wait until you have that reason. 00:31:43.170 --> 00:31:45.650 align:middle line:84% Let's do the broad stroke stuff first. 00:31:45.650 --> 00:31:48.410 align:middle line:90% And often, that'll be enough. 00:31:48.410 --> 00:31:51.290 align:middle line:84% Does that make sense, as a motivation, 00:31:51.290 --> 00:31:54.690 align:middle line:84% for why we want to throw away information about our functions? 00:31:54.690 --> 00:31:56.530 align:middle line:90% Nice. 00:31:56.530 --> 00:32:01.180 align:middle line:84% So let's talk about how to do that. 00:32:01.180 --> 00:32:03.550 align:middle line:84% And the main one we're going to talk about 00:32:03.550 --> 00:32:05.413 align:middle line:90% is called Big O notation. 00:32:05.413 --> 00:32:11.260 align:middle line:90% 00:32:11.260 --> 00:32:14.920 align:middle line:84% The idea-- this is not yet a definition, still an idea. 00:32:14.920 --> 00:32:18.460 align:middle line:84% We're going to say that f of n belongs to big O of g 00:32:18.460 --> 00:32:21.050 align:middle line:90% of n, weird notation. 00:32:21.050 --> 00:32:23.853 align:middle line:90% But you'll get used to it. 00:32:23.853 --> 00:32:24.353 align:middle line:90% When? 00:32:24.353 --> 00:32:27.190 align:middle line:90% 00:32:27.190 --> 00:32:32.320 align:middle line:84% Well, roughly f of n is less than or equal to g of n, 00:32:32.320 --> 00:32:33.730 align:middle line:90% roughly. 00:32:33.730 --> 00:32:37.570 align:middle line:84% If we say f is big O of g, what we're saying 00:32:37.570 --> 00:32:40.450 align:middle line:84% is that an algorithm with runtime f of n 00:32:40.450 --> 00:32:43.930 align:middle line:84% is at least as fast, maybe much faster than, 00:32:43.930 --> 00:32:49.936 align:middle line:84% an algorithm with runtime g of n, with some caveats, 00:32:49.936 --> 00:32:50.703 align:middle line:90% with caveats. 00:32:50.703 --> 00:32:53.620 align:middle line:90% 00:32:53.620 --> 00:32:56.613 align:middle line:84% First of all, let's throw away constant factors. 00:32:56.613 --> 00:33:08.060 align:middle line:90% 00:33:08.060 --> 00:33:11.520 align:middle line:84% If we have to scale g up by 10, and now g is bigger, 00:33:11.520 --> 00:33:12.210 align:middle line:90% that's fine. 00:33:12.210 --> 00:33:14.180 align:middle line:90% That's still big O. 00:33:14.180 --> 00:33:16.290 align:middle line:84% And likewise, we're going to ignore-- 00:33:16.290 --> 00:33:24.590 align:middle line:84% I don't care what happens for small numbers, ignoring small n. 00:33:24.590 --> 00:33:25.500 align:middle line:90% Don't need them. 00:33:25.500 --> 00:33:27.600 align:middle line:84% I care about what happens in the limit. 00:33:27.600 --> 00:33:30.050 align:middle line:84% What are the trends when n gets bigger and bigger? 00:33:30.050 --> 00:33:39.723 align:middle line:84% So these are my intuitions, also ignoring lower order terms. 00:33:39.723 --> 00:33:42.440 align:middle line:90% 00:33:42.440 --> 00:33:45.030 align:middle line:84% I still haven't defined that either, by the way. 00:33:45.030 --> 00:33:46.580 align:middle line:84% But we talked about them over here. 00:33:46.580 --> 00:33:48.740 align:middle line:84% The n over 2 was a lower order term 00:33:48.740 --> 00:33:50.070 align:middle line:90% compared to n squared over 2. 00:33:50.070 --> 00:33:52.550 align:middle line:84% I want to be able to ignore things like that. 00:33:52.550 --> 00:33:55.280 align:middle line:90% So this is still not precise. 00:33:55.280 --> 00:33:58.460 align:middle line:84% Let's finally, finally, make it precise, after this question. 00:33:58.460 --> 00:33:59.020 align:middle line:90% Yes? 00:33:59.020 --> 00:34:00.043 align:middle line:90% AUDIENCE: [INAUDIBLE]. 00:34:00.043 --> 00:34:02.210 align:middle line:84% ZACHARY ABEL: Oh, no, I want to give the definition. 00:34:02.210 --> 00:34:05.290 align:middle line:90% Thank you. 00:34:05.290 --> 00:34:09.580 align:middle line:84% So our definition, our actual, final definition of big O 00:34:09.580 --> 00:34:12.070 align:middle line:90% is the following. 00:34:12.070 --> 00:34:22.090 align:middle line:84% f of n is n big O of g of n means there exists a number 00:34:22.090 --> 00:34:29.170 align:middle line:84% greater than 0, and there exists a natural number n0, such 00:34:29.170 --> 00:34:35.350 align:middle line:84% that for all n greater or equal to n0, f of n 00:34:35.350 --> 00:34:39.850 align:middle line:90% is less equal to C times g of n. 00:34:39.850 --> 00:34:46.429 align:middle line:84% And if we're not restricting to positive functions, 00:34:46.429 --> 00:34:48.500 align:middle line:84% sometimes we put it in absolute value in there. 00:34:48.500 --> 00:34:49.969 align:middle line:90% That's less important. 00:34:49.969 --> 00:34:52.909 align:middle line:84% The other parts of the formula are the more important parts. 00:34:52.909 --> 00:34:56.900 align:middle line:84% There exists a constant factor C so that f 00:34:56.900 --> 00:34:59.150 align:middle line:90% is less equal to C times g. 00:34:59.150 --> 00:35:01.563 align:middle line:84% That's this, ignoring constant factors. 00:35:01.563 --> 00:35:04.640 align:middle line:90% 00:35:04.640 --> 00:35:08.780 align:middle line:84% There exists an n0 so that all n greater than n0, 00:35:08.780 --> 00:35:10.050 align:middle line:90% greater than or equal to n0-- 00:35:10.050 --> 00:35:12.800 align:middle line:84% so there's some starting point, so 00:35:12.800 --> 00:35:16.970 align:middle line:84% that after that starting point, this inequality is always true. 00:35:16.970 --> 00:35:18.890 align:middle line:84% And I don't care what happens before n0. 00:35:18.890 --> 00:35:22.070 align:middle line:90% 00:35:22.070 --> 00:35:25.760 align:middle line:84% So that's this point here, ignoring small n. 00:35:25.760 --> 00:35:29.570 align:middle line:84% Small n are the ones smaller than n0, 00:35:29.570 --> 00:35:36.230 align:middle line:84% n sub 0 is sometimes pronounced n0, N-A-U-G-H-T . 00:35:36.230 --> 00:35:37.250 align:middle line:90% Question? 00:35:37.250 --> 00:35:40.980 align:middle line:84% AUDIENCE: [INAUDIBLE] that f of n is [INAUDIBLE]? 00:35:40.980 --> 00:35:42.380 align:middle line:90% ZACHARY ABEL: Oh, gosh, yes. 00:35:42.380 --> 00:35:48.080 align:middle line:90% Thank you, g of n-- 00:35:48.080 --> 00:35:50.210 align:middle line:84% not the line I wanted to mess up today. 00:35:50.210 --> 00:35:51.500 align:middle line:90% Thank you so much. 00:35:51.500 --> 00:35:55.293 align:middle line:84% Yes, f of n is in big O of g of n, means roughly 00:35:55.293 --> 00:35:56.710 align:middle line:84% that g is bigger if you're allowed 00:35:56.710 --> 00:36:00.490 align:middle line:84% to scale it by a constant and throw away small values. 00:36:00.490 --> 00:36:01.690 align:middle line:90% All right? 00:36:01.690 --> 00:36:02.590 align:middle line:90% Yes? 00:36:02.590 --> 00:36:04.900 align:middle line:90% AUDIENCE: [INAUDIBLE]. 00:36:04.900 --> 00:36:06.130 align:middle line:90% ZACHARY ABEL: Yeah. 00:36:06.130 --> 00:36:13.960 align:middle line:84% So fn is in big O of g of n means 00:36:13.960 --> 00:36:20.380 align:middle line:84% that there's some positive value C and some starting point n0, 00:36:20.380 --> 00:36:24.460 align:middle line:84% so that for all input values that are bigger than n0 00:36:24.460 --> 00:36:29.890 align:middle line:84% or equal, fn is less equal to C times g of n. 00:36:29.890 --> 00:36:31.660 align:middle line:84% So this is extending our intuition 00:36:31.660 --> 00:36:35.200 align:middle line:90% that f is smaller than g. 00:36:35.200 --> 00:36:37.630 align:middle line:84% But you're allowed to put in a constant factor, 00:36:37.630 --> 00:36:39.410 align:middle line:84% if you need to, before that's true. 00:36:39.410 --> 00:36:40.270 align:middle line:90% Yes? 00:36:40.270 --> 00:36:41.500 align:middle line:84% AUDIENCE: Why do you need the constant factor? 00:36:41.500 --> 00:36:43.060 align:middle line:84% I thought we said beforehand that we 00:36:43.060 --> 00:36:45.668 align:middle line:90% don't need the constant factor. 00:36:45.668 --> 00:36:46.460 align:middle line:90% ZACHARY ABEL: Yeah. 00:36:46.460 --> 00:36:48.603 align:middle line:84% So the question was, why are we putting 00:36:48.603 --> 00:36:50.520 align:middle line:84% in a constant factor, when the whole point was 00:36:50.520 --> 00:36:52.390 align:middle line:90% to ignore the constant factor? 00:36:52.390 --> 00:36:57.340 align:middle line:84% We ignore it by saying that there exists a constant factor, 00:36:57.340 --> 00:36:58.950 align:middle line:90% and I don't care what it is. 00:36:58.950 --> 00:37:03.100 align:middle line:84% As long as some constant exists that makes this formula true, 00:37:03.100 --> 00:37:06.660 align:middle line:90% then f is big O of g. 00:37:06.660 --> 00:37:14.010 align:middle line:84% So we put it in so that we can not care what its value is. 00:37:14.010 --> 00:37:15.150 align:middle line:90% Yes? 00:37:15.150 --> 00:37:17.590 align:middle line:84% AUDIENCE: Can that constant be between 0 and 1? 00:37:17.590 --> 00:37:19.720 align:middle line:84% ZACHARY ABEL: Can the constant be between 0 and 1? 00:37:19.720 --> 00:37:20.380 align:middle line:90% Good question. 00:37:20.380 --> 00:37:21.840 align:middle line:90% Sure, why not? 00:37:21.840 --> 00:37:24.910 align:middle line:84% If you want to make g smaller, and it's still bigger than f, 00:37:24.910 --> 00:37:28.473 align:middle line:90% go for it, no problems there. 00:37:28.473 --> 00:37:29.890 align:middle line:84% All right, let's do some examples. 00:37:29.890 --> 00:37:31.500 align:middle line:84% Hopefully, some concrete examples 00:37:31.500 --> 00:37:34.560 align:middle line:90% will make this a bit clearer. 00:37:34.560 --> 00:37:38.550 align:middle line:90% Let's say f of n is n. 00:37:38.550 --> 00:37:41.790 align:middle line:90% g of n is n squared. 00:37:41.790 --> 00:37:44.990 align:middle line:90% Is f of n in big O of g of n? 00:37:44.990 --> 00:37:47.770 align:middle line:90% 00:37:47.770 --> 00:37:51.760 align:middle line:90% So is n in big O of n squared? 00:37:51.760 --> 00:37:54.640 align:middle line:84% Well, if this is supposed to say that the thing on the right 00:37:54.640 --> 00:37:57.340 align:middle line:84% is bigger, then hopefully, we would want 00:37:57.340 --> 00:37:59.680 align:middle line:90% n squared to be bigger than n. 00:37:59.680 --> 00:38:02.230 align:middle line:90% And thankfully, it is. 00:38:02.230 --> 00:38:05.320 align:middle line:84% In this case, we don't need the C to help us. f 00:38:05.320 --> 00:38:06.920 align:middle line:90% is already smaller than g. 00:38:06.920 --> 00:38:09.310 align:middle line:84% n is already smaller than n squared. 00:38:09.310 --> 00:38:15.850 align:middle line:84% n is less equal to 1 times n squared for all n, 00:38:15.850 --> 00:38:18.160 align:middle line:90% for all natural numbers n. 00:38:18.160 --> 00:38:22.160 align:middle line:84% So this would be our n0 value of 0. 00:38:22.160 --> 00:38:25.540 align:middle line:84% And this would be our C value of 1. 00:38:25.540 --> 00:38:32.410 align:middle line:84% All we have to do is show that some n0 exists and some C exists 00:38:32.410 --> 00:38:34.250 align:middle line:90% to make this statement true. 00:38:34.250 --> 00:38:37.640 align:middle line:84% And then we're able to conclude that n is big O of n squared. 00:38:37.640 --> 00:38:38.732 align:middle line:90% Question? 00:38:38.732 --> 00:38:44.190 align:middle line:84% AUDIENCE: [INAUDIBLE] n0 be 1 [INAUDIBLE] squared 00:38:44.190 --> 00:38:47.560 align:middle line:90% in less than n [INAUDIBLE]. 00:38:47.560 --> 00:38:49.150 align:middle line:90% ZACHARY ABEL: Ah, good question. 00:38:49.150 --> 00:38:53.070 align:middle line:84% Don't we need n0 to be 1 because this inequality is false 00:38:53.070 --> 00:38:55.830 align:middle line:90% when n is between 0 and 1? 00:38:55.830 --> 00:38:57.960 align:middle line:90% Excellent point. 00:38:57.960 --> 00:39:00.280 align:middle line:84% I'm going to give two fixes for that. 00:39:00.280 --> 00:39:03.030 align:middle line:84% First of all, we could change this to 1, absolutely. 00:39:03.030 --> 00:39:04.920 align:middle line:90% That's fine. 00:39:04.920 --> 00:39:09.450 align:middle line:84% The other change is that let's go ahead and assume 00:39:09.450 --> 00:39:13.740 align:middle line:84% that f and g are mapping the natural numbers 00:39:13.740 --> 00:39:15.960 align:middle line:90% to the real numbers. 00:39:15.960 --> 00:39:18.870 align:middle line:84% So the inputs are just 0, 1, 2, 3 and not 00:39:18.870 --> 00:39:20.500 align:middle line:90% all the real numbers in between. 00:39:20.500 --> 00:39:23.580 align:middle line:84% Often, that's what we use when we're analyzing algorithms 00:39:23.580 --> 00:39:28.170 align:middle line:84% because we don't have fractional inputs to algorithms. 00:39:28.170 --> 00:39:30.690 align:middle line:84% You could absolutely do all of this analysis and all 00:39:30.690 --> 00:39:35.910 align:middle line:84% these definitions if your domain was the reals instead. 00:39:35.910 --> 00:39:38.470 align:middle line:90% But let's deal with naturals. 00:39:38.470 --> 00:39:40.510 align:middle line:90% Great point. 00:39:40.510 --> 00:39:44.380 align:middle line:84% Also, in fact, why don't we go ahead 00:39:44.380 --> 00:39:46.823 align:middle line:84% and assume that all our functions are positive valued? 00:39:46.823 --> 00:39:49.750 align:middle line:90% 00:39:49.750 --> 00:39:51.860 align:middle line:84% Then we won't need that weird absolute value. 00:39:51.860 --> 00:39:54.340 align:middle line:84% You can still do a lot of this if you 00:39:54.340 --> 00:39:57.230 align:middle line:84% allow some of your functions to be 0 or negative. 00:39:57.230 --> 00:39:59.720 align:middle line:84% But let's not clutter our minds with that today. 00:39:59.720 --> 00:40:02.170 align:middle line:84% Let's go ahead and assume that all of our functions 00:40:02.170 --> 00:40:05.000 align:middle line:84% map the natural numbers to positive real values. 00:40:05.000 --> 00:40:06.040 align:middle line:90% Yes? 00:40:06.040 --> 00:40:08.290 align:middle line:84% AUDIENCE: Is your definition of the top the same thing 00:40:08.290 --> 00:40:11.770 align:middle line:84% as saying f of n is asymptotically equivalent 00:40:11.770 --> 00:40:14.530 align:middle line:90% some constant [INAUDIBLE]. 00:40:14.530 --> 00:40:17.350 align:middle line:84% ZACHARY ABEL: The question was, is this statement 00:40:17.350 --> 00:40:20.110 align:middle line:84% equivalent to saying that f is asymptotically 00:40:20.110 --> 00:40:23.510 align:middle line:84% equivalent to some constant multiple of g? 00:40:23.510 --> 00:40:24.650 align:middle line:90% That's a great question. 00:40:24.650 --> 00:40:25.835 align:middle line:90% The answer is no. 00:40:25.835 --> 00:40:27.585 align:middle line:84% And I'll challenge you to think about why. 00:40:27.585 --> 00:40:30.700 align:middle line:90% 00:40:30.700 --> 00:40:33.190 align:middle line:90% Let's do a few more examples. 00:40:33.190 --> 00:40:34.890 align:middle line:84% Actually, I want to come back over here. 00:40:34.890 --> 00:40:51.630 align:middle line:90% 00:40:51.630 --> 00:40:55.590 align:middle line:90% f of n is n squared. 00:40:55.590 --> 00:40:57.000 align:middle line:90% g of n is n. 00:40:57.000 --> 00:41:00.050 align:middle line:90% Is f of n in big O of g of n? 00:41:00.050 --> 00:41:03.360 align:middle line:90% 00:41:03.360 --> 00:41:06.930 align:middle line:84% Is n squared smaller than n, if you're 00:41:06.930 --> 00:41:10.530 align:middle line:90% allowed to scale by a constant? 00:41:10.530 --> 00:41:13.650 align:middle line:90% And this time, the answer is no. 00:41:13.650 --> 00:41:15.900 align:middle line:84% An algorithm that takes n squared time certainly 00:41:15.900 --> 00:41:19.050 align:middle line:84% wants to look slower, take longer, 00:41:19.050 --> 00:41:21.130 align:middle line:84% than an algorithm that takes n time. 00:41:21.130 --> 00:41:25.800 align:middle line:84% So if big O is going to be useful for algorithm comparison, 00:41:25.800 --> 00:41:28.710 align:middle line:84% we hope that this would be false. 00:41:28.710 --> 00:41:30.670 align:middle line:90% And thankfully, it is. 00:41:30.670 --> 00:41:33.150 align:middle line:90% And let's talk about why. 00:41:33.150 --> 00:41:37.870 align:middle line:84% So if n squared were in big O of n, 00:41:37.870 --> 00:41:41.680 align:middle line:84% we would need there exists a C, and there 00:41:41.680 --> 00:41:48.550 align:middle line:84% exists an n0, so that for all n greater or equal to n0-- 00:41:48.550 --> 00:41:54.890 align:middle line:84% let's see, n squared is less equal to C times n. 00:41:54.890 --> 00:41:57.082 align:middle line:90% I just wrote out the definition. 00:41:57.082 --> 00:41:58.540 align:middle line:84% So we would need to be able to pick 00:41:58.540 --> 00:42:00.410 align:middle line:84% some constant and some starting point, 00:42:00.410 --> 00:42:03.167 align:middle line:84% so that after that starting point, 00:42:03.167 --> 00:42:04.750 align:middle line:84% n squared is always less than or equal 00:42:04.750 --> 00:42:07.180 align:middle line:90% to that fixed constant times n. 00:42:07.180 --> 00:42:12.380 align:middle line:84% But no matter what constant we pick, when n gets big enough, 00:42:12.380 --> 00:42:15.010 align:middle line:90% this inequality is always false. 00:42:15.010 --> 00:42:19.210 align:middle line:84% We can't pick a constant big enough so that this remains 00:42:19.210 --> 00:42:21.470 align:middle line:90% true for all N, for all big N. 00:42:21.470 --> 00:42:27.040 align:middle line:84% If we pick C is 1,000, then from 1,0001 on, this inequality is 00:42:27.040 --> 00:42:27.950 align:middle line:90% going to be false. 00:42:27.950 --> 00:42:30.490 align:middle line:84% If C is a million, then from a million and 1 on, 00:42:30.490 --> 00:42:31.760 align:middle line:90% this inequality is false. 00:42:31.760 --> 00:42:33.880 align:middle line:84% No matter what C we pick, the left side 00:42:33.880 --> 00:42:37.410 align:middle line:90% will overtake the right side. 00:42:37.410 --> 00:42:40.950 align:middle line:84% And increasing n0 doesn't help with that. 00:42:40.950 --> 00:42:45.820 align:middle line:84% Even if we ignore more small values, all the big values fail. 00:42:45.820 --> 00:42:54.155 align:middle line:84% You can't ignore all the values, so can't choose C big enough. 00:42:54.155 --> 00:43:00.250 align:middle line:90% 00:43:00.250 --> 00:43:04.050 align:middle line:84% So thankfully, n squared is not big O of n. 00:43:04.050 --> 00:43:05.670 align:middle line:90% Let's do another example. 00:43:05.670 --> 00:43:10.320 align:middle line:90% Is n squared plus 3n plus 7, Is. 00:43:10.320 --> 00:43:13.530 align:middle line:90% That big O of n squared? 00:43:13.530 --> 00:43:17.400 align:middle line:84% We said we wanted to be able to ignore lower order terms, 00:43:17.400 --> 00:43:20.940 align:middle line:90% ignore things that look smaller. 00:43:20.940 --> 00:43:24.910 align:middle line:84% Well, this one ends up being true. 00:43:24.910 --> 00:43:26.350 align:middle line:90% And let's see why. 00:43:26.350 --> 00:43:30.840 align:middle line:84% Well, there are lots of ways to prove this. 00:43:30.840 --> 00:43:33.480 align:middle line:84% I'm going to be extremely wasteful and do the following. 00:43:33.480 --> 00:43:39.830 align:middle line:84% n squared plus 3n plus 7 is less equal to n squared plus 3n 00:43:39.830 --> 00:43:43.160 align:middle line:90% squared plus 7n squared. 00:43:43.160 --> 00:43:48.980 align:middle line:84% All I'm doing is saying this n is less equal to n squared. 00:43:48.980 --> 00:43:53.810 align:middle line:84% This 7 is less equal to 7n squared. 00:43:53.810 --> 00:43:59.120 align:middle line:90% And this equals 11n squared. 00:43:59.120 --> 00:44:03.230 align:middle line:84% And so we were able to show that this function is 00:44:03.230 --> 00:44:07.970 align:middle line:84% less equal to 11 times this function for all n. 00:44:07.970 --> 00:44:15.980 align:middle line:84% So we can take n0 equals 0, and C is 11, and we're done. 00:44:15.980 --> 00:44:19.820 align:middle line:84% So, lower order terms like this at most 00:44:19.820 --> 00:44:22.410 align:middle line:84% force us to pick a bigger C If we want. 00:44:22.410 --> 00:44:24.860 align:middle line:84% But again, we don't care about the specific value of C, 00:44:24.860 --> 00:44:27.360 align:middle line:90% as long as such a value exists. 00:44:27.360 --> 00:44:30.110 align:middle line:84% So we can always swallow up these smaller terms, 00:44:30.110 --> 00:44:32.970 align:middle line:84% just absorb them into that C constant. 00:44:32.970 --> 00:44:34.062 align:middle line:90% Yes? 00:44:34.062 --> 00:44:36.270 align:middle line:84% AUDIENCE: What [INAUDIBLE] know by looking at it that 00:44:36.270 --> 00:44:39.900 align:middle line:84% [INAUDIBLE] but obviously, n is going to be much smaller than n 00:44:39.900 --> 00:44:42.750 align:middle line:84% squared [INAUDIBLE] know what the function-- is 00:44:42.750 --> 00:44:44.803 align:middle line:84% one function bigger than the other [INAUDIBLE]? 00:44:44.803 --> 00:44:45.970 align:middle line:90% ZACHARY ABEL: Good question. 00:44:45.970 --> 00:44:47.887 align:middle line:84% What if we didn't know whether one function is 00:44:47.887 --> 00:44:50.640 align:middle line:84% bigger than another function just by looking at it? 00:44:50.640 --> 00:44:53.190 align:middle line:90% Well that is our task. 00:44:53.190 --> 00:44:54.940 align:middle line:84% We're trying to compare these functions. 00:44:54.940 --> 00:44:56.648 align:middle line:84% And if we don't know which one is bigger, 00:44:56.648 --> 00:44:59.460 align:middle line:84% then we better figure that out before we get our answer. 00:44:59.460 --> 00:45:03.030 align:middle line:84% So I don't have a general way to do that. 00:45:03.030 --> 00:45:05.556 align:middle line:84% I can just say that it's necessary. 00:45:05.556 --> 00:45:09.060 align:middle line:90% 00:45:09.060 --> 00:45:11.290 align:middle line:90% Let's do another example. 00:45:11.290 --> 00:45:15.870 align:middle line:90% 3n is that big O of n minus 4. 00:45:15.870 --> 00:45:17.670 align:middle line:84% We were also saying, we'd like to be 00:45:17.670 --> 00:45:22.540 align:middle line:84% able to ignore lower-order terms on the right-hand side. 00:45:22.540 --> 00:45:25.950 align:middle line:84% And intuitively, that n minus 4, that minus 4 term doesn't really 00:45:25.950 --> 00:45:27.400 align:middle line:90% change anything. 00:45:27.400 --> 00:45:30.415 align:middle line:84% This should still look like big O of n. 00:45:30.415 --> 00:45:32.990 align:middle line:90% 00:45:32.990 --> 00:45:35.510 align:middle line:90% And we can make that precise. 00:45:35.510 --> 00:45:46.520 align:middle line:84% n minus 4 is greater than or equal to n minus, well, n over 2 00:45:46.520 --> 00:45:49.700 align:middle line:90% when n is at least 8. 00:45:49.700 --> 00:45:51.560 align:middle line:84% All we're saying is that the n function 00:45:51.560 --> 00:45:54.980 align:middle line:90% overtakes the 4 function. 00:45:54.980 --> 00:45:55.500 align:middle line:90% Excuse me. 00:45:55.500 --> 00:45:58.460 align:middle line:84% So this 4, we can pretend it's an n over 2. 00:45:58.460 --> 00:46:00.200 align:middle line:84% We've lost a lot, but it doesn't matter. 00:46:00.200 --> 00:46:05.150 align:middle line:84% Because this is still a constant times n. 00:46:05.150 --> 00:46:10.590 align:middle line:84% So this function is bigger than a constant times n. 00:46:10.590 --> 00:46:14.600 align:middle line:84% And we can use that to do our proof Let's see, 00:46:14.600 --> 00:46:19.700 align:middle line:84% 3n is less equal to 6 times 1/2 n, 00:46:19.700 --> 00:46:24.590 align:middle line:90% which is less equal to 6 times-- 00:46:24.590 --> 00:46:32.550 align:middle line:84% that's that, so n minus 4 when n is at least 8. 00:46:32.550 --> 00:46:39.040 align:middle line:84% And so now we have our C. And we have our n0. 00:46:39.040 --> 00:46:41.980 align:middle line:84% 3n is less equal to a constant times n minus 4, 00:46:41.980 --> 00:46:44.100 align:middle line:84% which is the function we care about. 00:46:44.100 --> 00:46:47.280 align:middle line:84% So in the same way, we can ignore lower-order terms 00:46:47.280 --> 00:46:49.470 align:middle line:84% on the right, again, by just swallowing them 00:46:49.470 --> 00:46:53.820 align:middle line:84% into the constant C at the cost of a slightly higher constant C. 00:46:53.820 --> 00:46:55.830 align:middle line:90% Does that make sense? 00:46:55.830 --> 00:46:56.910 align:middle line:90% Nice. 00:46:56.910 --> 00:46:59.790 align:middle line:84% Now, some of you might recognize the definition of big O 00:46:59.790 --> 00:47:04.090 align:middle line:84% as looking a lot like epsilon delta definitions of limits. 00:47:04.090 --> 00:47:06.320 align:middle line:84% And if you haven't seen those, that's fine. 00:47:06.320 --> 00:47:06.820 align:middle line:90% Don't worry. 00:47:06.820 --> 00:47:07.900 align:middle line:90% It doesn't matter. 00:47:07.900 --> 00:47:11.230 align:middle line:90% But they are similar concepts. 00:47:11.230 --> 00:47:14.640 align:middle line:84% And we have a useful fact that compares 00:47:14.640 --> 00:47:18.840 align:middle line:84% the two, that I will write down now, theorem. 00:47:18.840 --> 00:47:23.400 align:middle line:90% 00:47:23.400 --> 00:47:31.130 align:middle line:84% So we're assuming that f and g are functions from N to R+ 00:47:31.130 --> 00:47:41.090 align:middle line:84% to the positive reals if the limit of f of n over g of n as n 00:47:41.090 --> 00:47:51.440 align:middle line:84% goes to infinity exists and equals infinity. 00:47:51.440 --> 00:47:54.980 align:middle line:84% So if f over g gets arbitrarily big, 00:47:54.980 --> 00:47:57.170 align:middle line:84% then there's no constant you can choose 00:47:57.170 --> 00:48:02.300 align:middle line:84% to bring g back up to f, because f gets bigger 00:48:02.300 --> 00:48:05.660 align:middle line:90% and bigger compared to g. 00:48:05.660 --> 00:48:11.810 align:middle line:90% Then f is not in big O of g. 00:48:11.810 --> 00:48:25.570 align:middle line:84% Likewise, if the limit exists and is less than infinity, 00:48:25.570 --> 00:48:31.060 align:middle line:90% then f is in big O of g. 00:48:31.060 --> 00:48:42.970 align:middle line:84% And finally, if the limit does not exist, 00:48:42.970 --> 00:48:48.010 align:middle line:84% inconclusive, f might be big O of g. f might not be big O of g. 00:48:48.010 --> 00:48:49.660 align:middle line:84% If the limit doesn't exist, the limit 00:48:49.660 --> 00:48:52.480 align:middle line:90% doesn't help us get an answer. 00:48:52.480 --> 00:48:56.270 align:middle line:84% But if the limit does exist, it tells us the answer. 00:48:56.270 --> 00:48:58.970 align:middle line:84% If the limit exists, it tells us yes 00:48:58.970 --> 00:49:02.020 align:middle line:90% in this case, no in that case. 00:49:02.020 --> 00:49:04.250 align:middle line:90% So this is a one-sided test. 00:49:04.250 --> 00:49:06.080 align:middle line:84% If the limit exists, you're golden. 00:49:06.080 --> 00:49:08.200 align:middle line:84% If the limit doesn't exist, you have 00:49:08.200 --> 00:49:10.476 align:middle line:84% to fall back to the more complicated definition. 00:49:10.476 --> 00:49:13.210 align:middle line:90% 00:49:13.210 --> 00:49:14.600 align:middle line:90% I'm not going to prove that. 00:49:14.600 --> 00:49:16.390 align:middle line:90% It's not difficult to prove. 00:49:16.390 --> 00:49:18.530 align:middle line:84% It's in the notes, if you want to read about it. 00:49:18.530 --> 00:49:20.560 align:middle line:90% But let's use it. 00:49:20.560 --> 00:49:21.230 align:middle line:90% Let's see. 00:49:21.230 --> 00:49:27.193 align:middle line:84% Is n squared in big O of a million n? 00:49:27.193 --> 00:49:32.300 align:middle line:90% 00:49:32.300 --> 00:49:33.840 align:middle line:90% Well, how do we test that? 00:49:33.840 --> 00:49:35.370 align:middle line:90% Let's use this limit test. 00:49:35.370 --> 00:49:39.320 align:middle line:84% We can look at the limit of f over g 00:49:39.320 --> 00:49:42.860 align:middle line:90% of n squared over a million n. 00:49:42.860 --> 00:49:47.120 align:middle line:84% As n goes to infinity, n squared over a constant times 00:49:47.120 --> 00:49:51.170 align:middle line:90% n, that's infinity. 00:49:51.170 --> 00:49:53.220 align:middle line:90% And so the limit exists. 00:49:53.220 --> 00:49:54.420 align:middle line:90% It equals infinity. 00:49:54.420 --> 00:49:57.040 align:middle line:84% And that tells us that f is not big O of g. 00:49:57.040 --> 00:50:02.570 align:middle line:90% 00:50:02.570 --> 00:50:05.240 align:middle line:84% So, very often, if you're working with actual functions, 00:50:05.240 --> 00:50:06.740 align:middle line:84% and you can actually take the limit, 00:50:06.740 --> 00:50:09.200 align:middle line:84% and the limit actually exists, you 00:50:09.200 --> 00:50:11.000 align:middle line:84% get your answer much faster than if you 00:50:11.000 --> 00:50:14.230 align:middle line:84% were trying to think through the C's and the n0's and whatnot. 00:50:14.230 --> 00:50:17.780 align:middle line:90% 00:50:17.780 --> 00:50:22.200 align:middle line:84% Another example is 1,000 in big O of n squared. 00:50:22.200 --> 00:50:24.730 align:middle line:90% 00:50:24.730 --> 00:50:27.520 align:middle line:84% What the heck do I even mean by that? 00:50:27.520 --> 00:50:29.150 align:middle line:90% 1,000 isn't a function of n. 00:50:29.150 --> 00:50:31.720 align:middle line:84% I thought we were talking about functions of n. 00:50:31.720 --> 00:50:33.850 align:middle line:90% Right? 00:50:33.850 --> 00:50:36.130 align:middle line:84% Well, it is a function of n if you 00:50:36.130 --> 00:50:37.640 align:middle line:90% want to think of it that way. 00:50:37.640 --> 00:50:39.015 align:middle line:90% Let me plot it for you. 00:50:39.015 --> 00:50:41.530 align:middle line:90% 00:50:41.530 --> 00:50:44.980 align:middle line:90% Let me plot the n squared. 00:50:44.980 --> 00:50:47.300 align:middle line:90% This is n squared. 00:50:47.300 --> 00:50:50.560 align:middle line:84% And let me plot the function that takes every n and sends it 00:50:50.560 --> 00:50:56.350 align:middle line:90% to the value 1,000. 00:50:56.350 --> 00:51:01.420 align:middle line:84% So this is f of n, which is defined as no matter what 00:51:01.420 --> 00:51:04.280 align:middle line:84% your input is, ignore it, and spit out 1,000. 00:51:04.280 --> 00:51:07.490 align:middle line:84% It's the constant function that spits out 1,000. 00:51:07.490 --> 00:51:09.080 align:middle line:84% So when you see something like this, 00:51:09.080 --> 00:51:13.160 align:middle line:84% when you see a constant in one of these asymptotic notations, 00:51:13.160 --> 00:51:16.600 align:middle line:84% it's really talking about the constant function that 00:51:16.600 --> 00:51:18.560 align:middle line:90% sends every n to that value. 00:51:18.560 --> 00:51:23.180 align:middle line:84% And in that sense, now we can ask, is this constant function 00:51:23.180 --> 00:51:25.970 align:middle line:90% big O of n squared? 00:51:25.970 --> 00:51:28.310 align:middle line:84% And we can answer it the same way as before by looking 00:51:28.310 --> 00:51:30.080 align:middle line:90% at the limit of f over g. 00:51:30.080 --> 00:51:36.170 align:middle line:84% The limit of f, which is 1,000, over g, which is n squared, 00:51:36.170 --> 00:51:40.740 align:middle line:84% as n goes to infinity, that limit is 0. 00:51:40.740 --> 00:51:46.070 align:middle line:84% So yes, yes, 1,000 is big O of n squared. 00:51:46.070 --> 00:51:47.700 align:middle line:84% When the limit exists, it's great. 00:51:47.700 --> 00:51:49.610 align:middle line:90% We're very happy. 00:51:49.610 --> 00:51:57.170 align:middle line:84% More examples-- we're trying to be permissive 00:51:57.170 --> 00:51:59.424 align:middle line:90% with constant factors. 00:51:59.424 --> 00:52:07.405 align:middle line:84% So is it true that 3 to the n is big O of 2 to the n? 00:52:07.405 --> 00:52:10.310 align:middle line:90% 00:52:10.310 --> 00:52:12.030 align:middle line:84% If constant factors don't matter, 00:52:12.030 --> 00:52:14.307 align:middle line:84% then maybe I should be able to change 00:52:14.307 --> 00:52:15.890 align:middle line:84% this constant to a different constant, 00:52:15.890 --> 00:52:18.650 align:middle line:84% and it won't really change, according to big O. 00:52:18.650 --> 00:52:21.400 align:middle line:90% Is this true? 00:52:21.400 --> 00:52:23.180 align:middle line:90% And this time it's not. 00:52:23.180 --> 00:52:25.420 align:middle line:84% Because we can try our limit test. 00:52:25.420 --> 00:52:30.130 align:middle line:84% The limit of f over g, that's 3 to the n over 2 to the n. 00:52:30.130 --> 00:52:36.610 align:middle line:84% That's the limit of 1.5 to the n, which is infinity. 00:52:36.610 --> 00:52:38.890 align:middle line:90% So the answer is no. 00:52:38.890 --> 00:52:42.530 align:middle line:84% Changing this 2 to a 3 isn't changing by a constant factor. 00:52:42.530 --> 00:52:46.120 align:middle line:84% It's changing by a constant factor n times. 00:52:46.120 --> 00:52:49.115 align:middle line:84% And that n gets bigger and bigger. 00:52:49.115 --> 00:52:50.740 align:middle line:84% So the difference between the two sides 00:52:50.740 --> 00:52:52.390 align:middle line:90% gets bigger and bigger. 00:52:52.390 --> 00:52:55.305 align:middle line:90% So this is a no. 00:52:55.305 --> 00:52:58.540 align:middle line:90% 00:52:58.540 --> 00:53:03.160 align:middle line:84% Some quick notation, I've been writing f of n 00:53:03.160 --> 00:53:05.115 align:middle line:90% is in big O of g of n. 00:53:05.115 --> 00:53:08.080 align:middle line:90% 00:53:08.080 --> 00:53:09.890 align:middle line:84% And this is my preferred notation. 00:53:09.890 --> 00:53:10.810 align:middle line:90% This is set notation. 00:53:10.810 --> 00:53:14.020 align:middle line:84% F of n is an element of this set. 00:53:14.020 --> 00:53:16.230 align:middle line:84% So big O of g of n is really a set 00:53:16.230 --> 00:53:19.800 align:middle line:84% of all the functions that have the relationship we described. 00:53:19.800 --> 00:53:21.760 align:middle line:84% This, I think, is the clearest notation. 00:53:21.760 --> 00:53:25.000 align:middle line:84% It's what I encourage you to use. 00:53:25.000 --> 00:53:27.250 align:middle line:84% But you're going to see some other notations. 00:53:27.250 --> 00:53:32.940 align:middle line:84% For example, you're going to see f of n equals big O of g of n, 00:53:32.940 --> 00:53:34.840 align:middle line:90% which is just super confusing. 00:53:34.840 --> 00:53:35.890 align:middle line:90% I don't like it. 00:53:35.890 --> 00:53:37.600 align:middle line:90% I don't recommend using it. 00:53:37.600 --> 00:53:40.015 align:middle line:84% But it is pretty standard in the math community, 00:53:40.015 --> 00:53:41.140 align:middle line:90% so you are going to see it. 00:53:41.140 --> 00:53:44.550 align:middle line:90% So I have to warn you. 00:53:44.550 --> 00:53:48.030 align:middle line:84% You're also going to see f of n is 00:53:48.030 --> 00:53:52.440 align:middle line:90% less equal to big O of g of n. 00:53:52.440 --> 00:53:54.780 align:middle line:84% This is, again, an abuse of notation. 00:53:54.780 --> 00:53:56.260 align:middle line:90% fn is a function. 00:53:56.260 --> 00:53:58.090 align:middle line:84% This thing on the right is a set. 00:53:58.090 --> 00:54:00.130 align:middle line:84% But it's communicating the same idea. 00:54:00.130 --> 00:54:02.700 align:middle line:84% In fact, it's emphasizing that same idea. 00:54:02.700 --> 00:54:05.650 align:middle line:84% Big O means that g is bigger than f, roughly. 00:54:05.650 --> 00:54:09.370 align:middle line:84% And so this less equal sign is just a reminder of that. 00:54:09.370 --> 00:54:12.030 align:middle line:84% So this is also good notation I recommend it 00:54:12.030 --> 00:54:14.440 align:middle line:90% if you want to use it. 00:54:14.440 --> 00:54:18.040 align:middle line:84% You're also going to see f of n is 00:54:18.040 --> 00:54:23.720 align:middle line:84% big O of g of n instead of is an element of big O of g of n. 00:54:23.720 --> 00:54:26.900 align:middle line:84% That's also fine, pretty clear, unambiguous. 00:54:26.900 --> 00:54:29.860 align:middle line:84% All of these mean exactly the same thing. 00:54:29.860 --> 00:54:31.720 align:middle line:90% Sound good? 00:54:31.720 --> 00:54:32.540 align:middle line:90% All right. 00:54:32.540 --> 00:54:36.190 align:middle line:90% What you should not write ever-- 00:54:36.190 --> 00:54:42.610 align:middle line:84% so we were saying, for example, we know that merge sort m of n 00:54:42.610 --> 00:54:49.520 align:middle line:84% is less equal to big O of swap sort of n, 00:54:49.520 --> 00:54:52.780 align:middle line:84% those two functions we looked at earlier, the n squared one 00:54:52.780 --> 00:54:53.990 align:middle line:90% and the n log n one. 00:54:53.990 --> 00:54:56.030 align:middle line:84% So this is a true statement, right? 00:54:56.030 --> 00:54:57.590 align:middle line:90% The n1 was smaller. 00:54:57.590 --> 00:55:00.940 align:middle line:84% If we want to say that the S is bigger, 00:55:00.940 --> 00:55:04.305 align:middle line:84% should we say that Sn is greater or equal to big O of m of n? 00:55:04.305 --> 00:55:07.690 align:middle line:90% 00:55:07.690 --> 00:55:09.710 align:middle line:84% And from the way I'm phrasing the question, 00:55:09.710 --> 00:55:11.820 align:middle line:90% the answer is probably no. 00:55:11.820 --> 00:55:13.510 align:middle line:90% No, we should not write this. 00:55:13.510 --> 00:55:16.950 align:middle line:84% That's like saying you must be at least 60 inches 00:55:16.950 --> 00:55:20.190 align:middle line:90% or less to ride this ride. 00:55:20.190 --> 00:55:22.180 align:middle line:90% How tall do I need to be? 00:55:22.180 --> 00:55:22.680 align:middle line:90% 60. 00:55:22.680 --> 00:55:24.300 align:middle line:90% What? 00:55:24.300 --> 00:55:26.640 align:middle line:90% It's not correct. 00:55:26.640 --> 00:55:30.810 align:middle line:84% Big O is functions that are smaller than m of n 00:55:30.810 --> 00:55:34.290 align:middle line:84% And if we're supposed to be bigger than some functions that 00:55:34.290 --> 00:55:37.890 align:middle line:84% are smaller than m of n, it doesn't make sense. 00:55:37.890 --> 00:55:39.010 align:middle line:90% Don't write that. 00:55:39.010 --> 00:55:41.520 align:middle line:84% However, this is still an idea that we often 00:55:41.520 --> 00:55:42.960 align:middle line:90% want to communicate. 00:55:42.960 --> 00:55:45.060 align:middle line:90% S is bigger than m. 00:55:45.060 --> 00:55:47.650 align:middle line:84% We want to emphasize that's is bigger than m 00:55:47.650 --> 00:55:50.800 align:middle line:84% instead of emphasizing that m is smaller than S. 00:55:50.800 --> 00:55:52.990 align:middle line:84% And we have different notation for that. 00:55:52.990 --> 00:56:03.240 align:middle line:84% Instead of a big O, we use S is big omega of m of n. 00:56:03.240 --> 00:56:07.500 align:middle line:84% So big omega is just the reverse of big O. 00:56:07.500 --> 00:56:09.170 align:middle line:90% We have our definition. 00:56:09.170 --> 00:56:18.290 align:middle line:84% f is in big omega of g means that g is in big O of f. 00:56:18.290 --> 00:56:21.050 align:middle line:84% Let's take that as our definition. 00:56:21.050 --> 00:56:23.250 align:middle line:84% And we're getting lots of symbols around here. 00:56:23.250 --> 00:56:25.390 align:middle line:84% So let's see if we can compile them into a table. 00:56:25.390 --> 00:56:29.660 align:middle line:90% 00:56:29.660 --> 00:56:36.080 align:middle line:84% So we started with f is asymptotically equivalent to g. 00:56:36.080 --> 00:56:41.540 align:middle line:84% The definition is that the limit as n goes to infinity of f of n 00:56:41.540 --> 00:56:45.410 align:middle line:90% over g of n equals 1. 00:56:45.410 --> 00:56:49.040 align:middle line:84% The idea, the intuition, is that f 00:56:49.040 --> 00:56:56.863 align:middle line:84% is equivalent to g up to lower order terms, lower order terms. 00:56:56.863 --> 00:56:59.780 align:middle line:90% 00:56:59.780 --> 00:57:01.850 align:middle line:90% That was our intuition. 00:57:01.850 --> 00:57:07.170 align:middle line:84% Then we had big O. f is in big O of g. 00:57:07.170 --> 00:57:10.830 align:middle line:84% The definition this time was there exists C greater 0. 00:57:10.830 --> 00:57:15.190 align:middle line:84% There exists n0 greater or equal 0, 00:57:15.190 --> 00:57:21.000 align:middle line:84% so that for all n greater or equal to n0, f of n 00:57:21.000 --> 00:57:25.110 align:middle line:90% is less equal to C times g of n. 00:57:25.110 --> 00:57:26.830 align:middle line:90% So that's our definition. 00:57:26.830 --> 00:57:32.520 align:middle line:84% Our intuition is that f is less equal to g up 00:57:32.520 --> 00:57:37.290 align:middle line:84% to constant values, up to constant factors, 00:57:37.290 --> 00:57:43.180 align:middle line:84% and I'll say lower order terms and ignoring small numbers. 00:57:43.180 --> 00:57:45.390 align:middle line:90% But that one does that, too. 00:57:45.390 --> 00:57:48.990 align:middle line:84% Please don't confuse the intuition with the definition. 00:57:48.990 --> 00:57:51.480 align:middle line:84% And please don't use the intuition language 00:57:51.480 --> 00:57:52.300 align:middle line:90% in your proofs. 00:57:52.300 --> 00:57:55.540 align:middle line:84% Use the definition language in your proofs. 00:57:55.540 --> 00:57:59.160 align:middle line:84% So think about it this way, but write it this way. 00:57:59.160 --> 00:58:05.570 align:middle line:84% And we just defined f is in big omega of g 00:58:05.570 --> 00:58:11.725 align:middle line:84% means the definition is that g is in big O of f. 00:58:11.725 --> 00:58:15.620 align:middle line:90% 00:58:15.620 --> 00:58:20.360 align:middle line:84% And the idea is that g is greater-- 00:58:20.360 --> 00:58:24.110 align:middle line:84% sorry, is that f is greater or equal 00:58:24.110 --> 00:58:30.805 align:middle line:84% to g up to constants and lower order terms. 00:58:30.805 --> 00:58:34.220 align:middle line:90% 00:58:34.220 --> 00:58:37.110 align:middle line:84% So we have a more permissive version of less equal. 00:58:37.110 --> 00:58:39.870 align:middle line:84% That's big O. A more permissive version of greater or equal, 00:58:39.870 --> 00:58:42.290 align:middle line:90% that's big omega. 00:58:42.290 --> 00:58:44.870 align:middle line:90% Questions about that? 00:58:44.870 --> 00:58:47.990 align:middle line:90% All right. 00:58:47.990 --> 00:58:50.780 align:middle line:84% What did I want to say about that? 00:58:50.780 --> 00:58:53.240 align:middle line:84% Let me define one more while I'm over here. 00:58:53.240 --> 00:58:57.120 align:middle line:90% f is in theta of g. 00:58:57.120 --> 00:58:59.930 align:middle line:90% That's a capital Greek theta. 00:58:59.930 --> 00:59:07.260 align:middle line:84% And this is going to mean, roughly, f equals g-- 00:59:07.260 --> 00:59:11.790 align:middle line:84% sorry, f is approximately equal to g up 00:59:11.790 --> 00:59:18.570 align:middle line:84% to constants and lower order terms. 00:59:18.570 --> 00:59:24.390 align:middle line:84% And the definition is just f is in big O of g, 00:59:24.390 --> 00:59:27.215 align:middle line:90% and g is in big O of f. 00:59:27.215 --> 00:59:31.380 align:middle line:90% 00:59:31.380 --> 00:59:35.460 align:middle line:84% So f and g are roughly within constant factors of each other 00:59:35.460 --> 00:59:36.876 align:middle line:90% in both directions. 00:59:36.876 --> 00:59:40.290 align:middle line:90% 00:59:40.290 --> 00:59:42.810 align:middle line:84% And I want to call attention to this first one 00:59:42.810 --> 00:59:45.330 align:middle line:90% versus this last one. 00:59:45.330 --> 00:59:49.200 align:middle line:84% Asymptotic equivalence said that f and g were roughly equal 00:59:49.200 --> 00:59:51.510 align:middle line:90% up to lower order terms. 00:59:51.510 --> 00:59:55.200 align:middle line:84% Theta says that f and g are approximately equal 00:59:55.200 --> 00:59:59.670 align:middle line:84% up to constant factors and lower order terms. 00:59:59.670 --> 01:00:03.890 align:middle line:84% Theta is more permissive than tilde. 01:00:03.890 --> 01:00:05.900 align:middle line:84% Where did I leave off on the boards? 01:00:05.900 --> 01:00:07.380 align:middle line:90% Over here? 01:00:07.380 --> 01:00:08.365 align:middle line:90% Yeah, over here. 01:00:08.365 --> 01:00:13.980 align:middle line:90% 01:00:13.980 --> 01:00:14.480 align:middle line:90% Good. 01:00:14.480 --> 01:00:19.490 align:middle line:90% 01:00:19.490 --> 01:00:24.410 align:middle line:84% So we can say that our S of n function, our simple sort 01:00:24.410 --> 01:00:32.690 align:middle line:84% function, n squared over 2 and junk, is in theta of n squared. 01:00:32.690 --> 01:00:36.620 align:middle line:84% It's both greater than n squared and less 01:00:36.620 --> 01:00:38.780 align:middle line:84% than n squared, if you're allowed 01:00:38.780 --> 01:00:39.985 align:middle line:90% to put constants in there. 01:00:39.985 --> 01:00:45.140 align:middle line:90% 01:00:45.140 --> 01:00:50.690 align:middle line:84% What's not true is Sn is asymptotically 01:00:50.690 --> 01:00:51.960 align:middle line:90% equivalent to n squared. 01:00:51.960 --> 01:00:55.220 align:middle line:90% This is false. 01:00:55.220 --> 01:00:57.740 align:middle line:84% Because if you take this and divide by that 01:00:57.740 --> 01:01:00.960 align:middle line:84% and take the limit, you get 2, not 1. 01:01:00.960 --> 01:01:03.810 align:middle line:90% That n squared over 2 matters. 01:01:03.810 --> 01:01:06.760 align:middle line:84% So this cares what that constant factor is. 01:01:06.760 --> 01:01:08.550 align:middle line:90% This doesn't. 01:01:08.550 --> 01:01:11.900 align:middle line:84% This would be a true statement, if you put the 1/2 in there. 01:01:11.900 --> 01:01:14.760 align:middle line:90% 01:01:14.760 --> 01:01:17.340 align:middle line:84% So the asymptotic equivalence does 01:01:17.340 --> 01:01:18.780 align:middle line:90% single out that constant factor. 01:01:18.780 --> 01:01:22.890 align:middle line:84% Whereas, theta ignores that constant factor. 01:01:22.890 --> 01:01:24.000 align:middle line:90% Was there a question? 01:01:24.000 --> 01:01:27.540 align:middle line:84% AUDIENCE: So [INAUDIBLE] be the same thing 01:01:27.540 --> 01:01:34.290 align:middle line:84% as saying there exists a C such that tilde [INAUDIBLE]? 01:01:34.290 --> 01:01:36.480 align:middle line:84% ZACHARY ABEL: So the question was 01:01:36.480 --> 01:01:38.610 align:middle line:84% similar to your previous question. 01:01:38.610 --> 01:01:47.910 align:middle line:84% Does f in theta of g mean that there exists a constant such 01:01:47.910 --> 01:01:51.690 align:middle line:90% that f is tilde of C times g? 01:01:51.690 --> 01:01:53.093 align:middle line:90% That's a really good question. 01:01:53.093 --> 01:01:56.670 align:middle line:90% 01:01:56.670 --> 01:02:02.990 align:middle line:84% And the answer, I will tell you, is actually the other way. 01:02:02.990 --> 01:02:05.480 align:middle line:84% If there's a constant factor between them, 01:02:05.480 --> 01:02:09.380 align:middle line:84% and then the limit goes to 1, then absolutely, 01:02:09.380 --> 01:02:11.450 align:middle line:90% they are theta of each other. 01:02:11.450 --> 01:02:14.040 align:middle line:84% But there doesn't have to be that constant factor. 01:02:14.040 --> 01:02:15.970 align:middle line:84% And let's look at an example of that. 01:02:15.970 --> 01:02:27.020 align:middle line:90% 01:02:27.020 --> 01:02:34.595 align:middle line:84% Let's see, 3 plus sine of n is that theta of 1. 01:02:34.595 --> 01:02:38.180 align:middle line:90% 01:02:38.180 --> 01:02:42.740 align:middle line:84% 3 plus sine of n, we want to know, is it theta of 1? 01:02:42.740 --> 01:02:45.650 align:middle line:84% So is it bounded below and bounded 01:02:45.650 --> 01:02:50.090 align:middle line:90% above by some constant times 1? 01:02:50.090 --> 01:02:52.250 align:middle line:90% And in this case, yes, it is. 01:02:52.250 --> 01:02:57.260 align:middle line:84% Because 2 is less equal to 3 plus sine of n, 01:02:57.260 --> 01:02:58.605 align:middle line:90% which is less equal to 4. 01:02:58.605 --> 01:03:01.450 align:middle line:90% 01:03:01.450 --> 01:03:04.220 align:middle line:84% So we can scale this by a constant. 01:03:04.220 --> 01:03:06.920 align:middle line:84% And it's always smaller than our function. 01:03:06.920 --> 01:03:08.720 align:middle line:84% We can scale this by a different constant. 01:03:08.720 --> 01:03:10.750 align:middle line:84% And it's always bigger than our function. 01:03:10.750 --> 01:03:14.350 align:middle line:84% And that's the condition we needed for big O on both sides. 01:03:14.350 --> 01:03:17.620 align:middle line:90% So yes, this is true. 01:03:17.620 --> 01:03:20.080 align:middle line:84% However, you're never going to find a constant where 01:03:20.080 --> 01:03:23.980 align:middle line:84% 3 plus sine of n is asymptotically 01:03:23.980 --> 01:03:26.890 align:middle line:84% equivalent to that constant times 1. 01:03:26.890 --> 01:03:30.650 align:middle line:84% Because that ratio isn't going to have a limit. 01:03:30.650 --> 01:03:40.180 align:middle line:84% This is an example where the limit of 3 plus sine of n over 1 01:03:40.180 --> 01:03:42.720 align:middle line:90% does not exist. 01:03:42.720 --> 01:03:44.470 align:middle line:84% The function just keeps doing this forever 01:03:44.470 --> 01:03:47.530 align:middle line:84% and never settles down and picks a single value. 01:03:47.530 --> 01:03:50.090 align:middle line:84% So this is an example where the limit between them 01:03:50.090 --> 01:03:51.940 align:middle line:90% does not exist. 01:03:51.940 --> 01:03:56.000 align:middle line:84% But still, they are big O of each other. 01:03:56.000 --> 01:03:58.670 align:middle line:90% Right? 01:03:58.670 --> 01:04:00.192 align:middle line:90% In fact, in both directions. 01:04:00.192 --> 01:04:02.150 align:middle line:84% We mentioned earlier, that if the limit doesn't 01:04:02.150 --> 01:04:06.380 align:middle line:84% exist, sometimes they are not big O of each other. 01:04:06.380 --> 01:04:08.960 align:middle line:84% Can anyone think of an example of that, 01:04:08.960 --> 01:04:14.240 align:middle line:84% two functions whose ratio does not have a limit, where f is not 01:04:14.240 --> 01:04:15.055 align:middle line:90% big O of g? 01:04:15.055 --> 01:04:31.242 align:middle line:90% 01:04:31.242 --> 01:04:37.070 align:middle line:84% A little tricky-- let me give let me give an example. 01:04:37.070 --> 01:04:47.270 align:middle line:84% Let's say that f of n is n, and g of n is 1 when n is odd 01:04:47.270 --> 01:04:49.060 align:middle line:90% and n squared when n is even. 01:04:49.060 --> 01:04:53.860 align:middle line:90% 01:04:53.860 --> 01:05:04.570 align:middle line:84% So if we plot f of n is just n, g of n is following 1 or n 01:05:04.570 --> 01:05:06.590 align:middle line:84% squared, depending where you are. 01:05:06.590 --> 01:05:09.210 align:middle line:84% And it's zigzagging between the two. 01:05:09.210 --> 01:05:12.760 align:middle line:90% 01:05:12.760 --> 01:05:15.100 align:middle line:84% If you scale g by a constant, you'll 01:05:15.100 --> 01:05:20.860 align:middle line:84% never be able to make it bigger or smaller than n, 01:05:20.860 --> 01:05:23.950 align:middle line:84% because the ratio is too much, depending 01:05:23.950 --> 01:05:26.800 align:middle line:84% on whether you're looking at the odds or the evens. 01:05:26.800 --> 01:05:28.700 align:middle line:84% You can't scale it in either direction. 01:05:28.700 --> 01:05:33.310 align:middle line:90% So f in big O of g is false. 01:05:33.310 --> 01:05:37.080 align:middle line:84% And g in big O of f is also false. 01:05:37.080 --> 01:05:40.000 align:middle line:90% 01:05:40.000 --> 01:05:42.100 align:middle line:84% And it was specially constructed so 01:05:42.100 --> 01:05:51.440 align:middle line:84% that limit of f over g as n goes to infinity does not exist. 01:05:51.440 --> 01:05:54.540 align:middle line:84% Because on the odds, f over g grows really big. 01:05:54.540 --> 01:05:59.480 align:middle line:84% And on the evens, f over g grows really small. 01:05:59.480 --> 01:06:00.945 align:middle line:90% Sound good? 01:06:00.945 --> 01:06:01.445 align:middle line:90% Nice. 01:06:01.445 --> 01:06:05.210 align:middle line:90% 01:06:05.210 --> 01:06:08.720 align:middle line:84% Speaking of getting really big and really small, 01:06:08.720 --> 01:06:10.820 align:middle line:84% there are two more asymptotic symbols 01:06:10.820 --> 01:06:13.130 align:middle line:84% that we need to add to our table. 01:06:13.130 --> 01:06:16.130 align:middle line:84% And those are the last two, I promise. 01:06:16.130 --> 01:06:25.850 align:middle line:84% The first one I want to define is called little o. 01:06:25.850 --> 01:06:30.350 align:middle line:90% So f is in little o of g. 01:06:30.350 --> 01:06:34.100 align:middle line:84% The definition is going to be that the limit as n 01:06:34.100 --> 01:06:42.440 align:middle line:84% goes to infinity of f of n over g of n equals 0. 01:06:42.440 --> 01:06:50.770 align:middle line:84% And the intuition is that f is much, much, much smaller than g. 01:06:50.770 --> 01:06:55.203 align:middle line:84% If we want to say that f is insignificant compared to g, 01:06:55.203 --> 01:06:57.370 align:middle line:84% well, that's just saying that the limit between them 01:06:57.370 --> 01:06:58.110 align:middle line:90% goes to 0. 01:06:58.110 --> 01:07:01.270 align:middle line:90% 01:07:01.270 --> 01:07:04.280 align:middle line:84% So that's going to be our definition of little o. 01:07:04.280 --> 01:07:09.070 align:middle line:84% Notation-wise, little o and big O look similar. 01:07:09.070 --> 01:07:12.520 align:middle line:84% So when you're writing those by hand, please try to exaggerate. 01:07:12.520 --> 01:07:16.930 align:middle line:84% I like to write my little o's really, really small, even 01:07:16.930 --> 01:07:19.360 align:middle line:84% smaller than other lowercase letters. 01:07:19.360 --> 01:07:21.650 align:middle line:84% And I like to write my big O really, 01:07:21.650 --> 01:07:26.060 align:middle line:84% really big, not hilariously big, but noticeably big, 01:07:26.060 --> 01:07:28.290 align:middle line:84% so no one's going to confuse it for a little o. 01:07:28.290 --> 01:07:31.060 align:middle line:90% 01:07:31.060 --> 01:07:33.530 align:middle line:84% Annoying notation that those look similar-- 01:07:33.530 --> 01:07:37.720 align:middle line:84% so just be aware, and try to correct for it. 01:07:37.720 --> 01:07:39.700 align:middle line:90% And there's the reverse. 01:07:39.700 --> 01:07:43.310 align:middle line:90% f is in little omega of g. 01:07:43.310 --> 01:07:46.670 align:middle line:84% That's the lowercase omega compared to the capital omega 01:07:46.670 --> 01:07:48.140 align:middle line:90% up there. 01:07:48.140 --> 01:07:50.690 align:middle line:84% It means the opposite, that the limit as n 01:07:50.690 --> 01:08:00.380 align:middle line:84% goes to infinity of f of n over g of n equals infinity, a.k.a. 01:08:00.380 --> 01:08:02.650 align:middle line:90% g is in little o of f. 01:08:02.650 --> 01:08:05.360 align:middle line:90% 01:08:05.360 --> 01:08:10.760 align:middle line:84% So if we want to stress that f is much, much bigger than g, 01:08:10.760 --> 01:08:14.090 align:middle line:84% we can use the little omega notation, 01:08:14.090 --> 01:08:17.604 align:middle line:84% that f is much, much bigger than g. 01:08:17.604 --> 01:08:20.540 align:middle line:90% 01:08:20.540 --> 01:08:23.913 align:middle line:84% And these are useful, I mean, for the same reasons 01:08:23.913 --> 01:08:25.080 align:middle line:90% the other things are useful. 01:08:25.080 --> 01:08:28.910 align:middle line:84% They're giving us more vocabulary to compare functions 01:08:28.910 --> 01:08:32.300 align:middle line:84% in ways that let us think about only 01:08:32.300 --> 01:08:35.810 align:middle line:84% a small amount of information about each function. 01:08:35.810 --> 01:08:46.689 align:middle line:84% But little o is exactly what we meant by lower order term, claim 01:08:46.689 --> 01:08:50.410 align:middle line:84% that f is going to be asymptotically equivalent to f 01:08:50.410 --> 01:08:59.560 align:middle line:84% plus h if and only if h is in little o of f 01:08:59.560 --> 01:09:06.670 align:middle line:84% if and only if the limit of h over f is 0, 01:09:06.670 --> 01:09:10.370 align:middle line:84% h of n over f of n as n goes to infinity. 01:09:10.370 --> 01:09:12.970 align:middle line:84% So lower order term just means the ratio 01:09:12.970 --> 01:09:15.640 align:middle line:90% goes to 0 in the limit. 01:09:15.640 --> 01:09:17.470 align:middle line:84% And that's consistent with what we've 01:09:17.470 --> 01:09:20.450 align:middle line:84% been saying with lower order terms all throughout today. 01:09:20.450 --> 01:09:23.359 align:middle line:84% Lower order term just means limit goes to 0. 01:09:23.359 --> 01:09:27.837 align:middle line:84% And we have a special notation for that little o. 01:09:27.837 --> 01:09:30.760 align:middle line:84% f is asymptotically equivalent to f plus h means 01:09:30.760 --> 01:09:33.700 align:middle line:84% that according to asymptotic equivalence, 01:09:33.700 --> 01:09:36.189 align:middle line:90% the h didn't change anything. 01:09:36.189 --> 01:09:38.330 align:middle line:90% The h was insignificant. 01:09:38.330 --> 01:09:40.090 align:middle line:84% So that's another way to think about what 01:09:40.090 --> 01:09:43.410 align:middle line:90% we mean by lower order term. 01:09:43.410 --> 01:09:44.940 align:middle line:90% Make sense? 01:09:44.940 --> 01:09:45.440 align:middle line:90% Nice. 01:09:45.440 --> 01:09:47.970 align:middle line:90% 01:09:47.970 --> 01:09:49.380 align:middle line:90% Cool. 01:09:49.380 --> 01:09:53.585 align:middle line:84% Last thing I want to do today is a big, old warning. 01:09:53.585 --> 01:10:01.320 align:middle line:90% 01:10:01.320 --> 01:10:17.945 align:middle line:84% Asymptotic notation and induction don't mix easily. 01:10:17.945 --> 01:10:22.140 align:middle line:90% 01:10:22.140 --> 01:10:24.990 align:middle line:84% If you're ever doing asymptotics and induction 01:10:24.990 --> 01:10:28.990 align:middle line:90% in the same breadth, step back. 01:10:28.990 --> 01:10:30.040 align:middle line:90% Take care. 01:10:30.040 --> 01:10:31.490 align:middle line:90% Let's see what can go wrong. 01:10:31.490 --> 01:10:41.510 align:middle line:90% 01:10:41.510 --> 01:10:52.810 align:middle line:84% So, theorem, question mark, 2 to the n is big O of 1. 01:10:52.810 --> 01:10:55.540 align:middle line:84% Now, we really, really hope this isn't true. 01:10:55.540 --> 01:10:57.550 align:middle line:84% In algorithm terms, this is an algorithm 01:10:57.550 --> 01:10:59.270 align:middle line:90% that takes exponential time. 01:10:59.270 --> 01:11:01.700 align:middle line:84% This is an algorithm that takes constant time. 01:11:01.700 --> 01:11:04.450 align:middle line:84% Certainly, this one is supposed to be the smaller one. 01:11:04.450 --> 01:11:09.430 align:middle line:84% So 2 to the n should absolutely not be in big O of 1. 01:11:09.430 --> 01:11:11.230 align:middle line:90% But let's prove it anyway. 01:11:11.230 --> 01:11:20.560 align:middle line:84% Proof by induction, let's do our base case. 01:11:20.560 --> 01:11:22.220 align:middle line:90% n equals 0. 01:11:22.220 --> 01:11:26.260 align:middle line:90% So 2 to the 0 is in big O of 1. 01:11:26.260 --> 01:11:31.610 align:middle line:84% That's saying that 1 is big O of 1, which it certainly is. 01:11:31.610 --> 01:11:44.730 align:middle line:84% Inductive step, assume n is at least 0 and that 2 to the n 01:11:44.730 --> 01:11:46.310 align:middle line:90% is in big O of 1. 01:11:46.310 --> 01:11:49.290 align:middle line:90% 01:11:49.290 --> 01:11:54.600 align:middle line:84% We want to show 2 to the n plus 1 is in big O of 1. 01:11:54.600 --> 01:12:00.810 align:middle line:84% But, well, 2 to the n plus 1, that's 2 to the n plus 2 01:12:00.810 --> 01:12:07.500 align:middle line:84% to the n, which is big O of 1 plus big O of 1 01:12:07.500 --> 01:12:09.930 align:middle line:90% by our inductive assumption. 01:12:09.930 --> 01:12:14.730 align:middle line:90% And that is big O of 1. 01:12:14.730 --> 01:12:17.050 align:middle line:84% So 2 to the n plus 1 is big O of 1. 01:12:17.050 --> 01:12:18.840 align:middle line:90% And we're done. 01:12:18.840 --> 01:12:22.500 align:middle line:90% We proved our induction. 01:12:22.500 --> 01:12:26.190 align:middle line:90% What the heck went wrong? 01:12:26.190 --> 01:12:27.240 align:middle line:90% Yes? 01:12:27.240 --> 01:12:30.120 align:middle line:84% AUDIENCE: [INAUDIBLE] 2 to the n is [INAUDIBLE] is not 01:12:30.120 --> 01:12:32.248 align:middle line:90% equal to [INAUDIBLE]. 01:12:32.248 --> 01:12:34.290 align:middle line:84% ZACHARY ABEL: 2 to the n plus 1 is not equal to 2 01:12:34.290 --> 01:12:36.370 align:middle line:90% to the n plus 2 to the n? 01:12:36.370 --> 01:12:37.120 align:middle line:90% It is. 01:12:37.120 --> 01:12:39.070 align:middle line:84% 2 to the n plus 1 is just 2 times 2 01:12:39.070 --> 01:12:43.330 align:middle line:84% to the n, which I can write as two copies of 2 to the n 01:12:43.330 --> 01:12:45.700 align:middle line:90% added together. 01:12:45.700 --> 01:12:49.400 align:middle line:84% I will tell you, the mistake isn't in this line at all. 01:12:49.400 --> 01:12:52.840 align:middle line:84% I know I did something really sketchy right here. 01:12:52.840 --> 01:12:55.640 align:middle line:84% What do I mean by O of 1 plus O of 1? 01:12:55.640 --> 01:12:57.400 align:middle line:90% That's a set, and that's a set. 01:12:57.400 --> 01:12:59.440 align:middle line:90% Why am I able to add them? 01:12:59.440 --> 01:13:01.870 align:middle line:84% Also, I accidentally used that when 01:13:01.870 --> 01:13:06.220 align:middle line:84% I meant set membership instead of equality. 01:13:06.220 --> 01:13:08.240 align:middle line:84% But actually, this is kind of fine. 01:13:08.240 --> 01:13:09.890 align:middle line:90% This is an abuse of notation. 01:13:09.890 --> 01:13:12.610 align:middle line:84% We didn't really define what this means. 01:13:12.610 --> 01:13:18.200 align:middle line:84% But all we mean by it, O of f plus O of g, 01:13:18.200 --> 01:13:22.030 align:middle line:84% this is just the set of functions a plus 01:13:22.030 --> 01:13:30.880 align:middle line:84% b, where a is in O of f, and b is in O of g. 01:13:30.880 --> 01:13:32.500 align:middle line:84% This just means the set of things 01:13:32.500 --> 01:13:35.820 align:middle line:84% you can form by taking one function from this side and one 01:13:35.820 --> 01:13:39.240 align:middle line:84% function from that side and adding them together. 01:13:39.240 --> 01:13:42.540 align:middle line:84% And it is true that in this case of 1 plus O of 1 01:13:42.540 --> 01:13:45.280 align:middle line:90% equals of O of 1 as sets. 01:13:45.280 --> 01:13:46.240 align:middle line:90% So that's fine. 01:13:46.240 --> 01:13:47.710 align:middle line:90% That's not the mistake. 01:13:47.710 --> 01:13:50.160 align:middle line:90% Yes? 01:13:50.160 --> 01:13:51.810 align:middle line:84% AUDIENCE: You proved that [INAUDIBLE] 01:13:51.810 --> 01:13:54.630 align:middle line:84% every number [INAUDIBLE] equal to the [INAUDIBLE]. 01:13:54.630 --> 01:13:57.563 align:middle line:84% But you didn't prove the function [INAUDIBLE]. 01:13:57.563 --> 01:13:58.980 align:middle line:84% ZACHARY ABEL: So let me reiterate. 01:13:58.980 --> 01:14:03.880 align:middle line:84% I proved that every number 2 to the n is big O of 1 itself. 01:14:03.880 --> 01:14:08.160 align:middle line:84% But I didn't prove the function 2 to the n is big O of 1. 01:14:08.160 --> 01:14:10.330 align:middle line:90% That's exactly the idea. 01:14:10.330 --> 01:14:12.580 align:middle line:84% So let's see what we actually proved here. 01:14:12.580 --> 01:14:21.360 align:middle line:84% This here is a correct proof by induction. 01:14:21.360 --> 01:14:23.790 align:middle line:90% But what is it proving? 01:14:23.790 --> 01:14:25.420 align:middle line:90% It's a proof of what? 01:14:25.420 --> 01:14:31.060 align:middle line:84% What we really proved is that 1 is in big O of 1. 01:14:31.060 --> 01:14:32.520 align:middle line:90% That was our base case. 01:14:32.520 --> 01:14:35.120 align:middle line:84% We proved that 2 is in big O of 1. 01:14:35.120 --> 01:14:37.560 align:middle line:90% 4 is in big O of 1. 01:14:37.560 --> 01:14:39.690 align:middle line:90% 8 is in big O of 1. 01:14:39.690 --> 01:14:42.680 align:middle line:90% 16 is in big O of 1. 01:14:42.680 --> 01:14:46.050 align:middle line:84% We proved that each individual power of 2, 01:14:46.050 --> 01:14:52.520 align:middle line:84% when considered as a constant function, is big O of 1. 01:14:52.520 --> 01:14:55.260 align:middle line:84% So every constant function is big O of 1. 01:14:55.260 --> 01:14:56.310 align:middle line:90% Yeah, that's true. 01:14:56.310 --> 01:14:58.080 align:middle line:90% We proved it by induction. 01:14:58.080 --> 01:15:02.390 align:middle line:84% That has absolutely nothing to do with the 2 to the n function. 01:15:02.390 --> 01:15:06.200 align:middle line:84% It has absolutely nothing to do with the definition of 2 01:15:06.200 --> 01:15:09.770 align:middle line:90% to the n being in big O of 1. 01:15:09.770 --> 01:15:11.460 align:middle line:84% So we didn't prove the right theorem. 01:15:11.460 --> 01:15:14.390 align:middle line:84% We proved something completely irrelevant. 01:15:14.390 --> 01:15:17.810 align:middle line:90% Does that make sense? 01:15:17.810 --> 01:15:22.110 align:middle line:84% So this is why induction and asymptotics don't mix. 01:15:22.110 --> 01:15:24.320 align:middle line:84% When you see a function of n, you 01:15:24.320 --> 01:15:29.310 align:middle line:84% can't break that function into individual constant functions, 01:15:29.310 --> 01:15:30.370 align:middle line:90% one for each n value. 01:15:30.370 --> 01:15:31.480 align:middle line:90% You can't induct on n. 01:15:31.480 --> 01:15:33.600 align:middle line:84% You have to treat that function holistically 01:15:33.600 --> 01:15:36.210 align:middle line:90% as a single function. 01:15:36.210 --> 01:15:39.120 align:middle line:90% Sound good? 01:15:39.120 --> 01:15:41.220 align:middle line:90% All right. 01:15:41.220 --> 01:15:44.470 align:middle line:84% Does anyone have any questions about anything we talked about? 01:15:44.470 --> 01:15:46.080 align:middle line:90% Yes? 01:15:46.080 --> 01:15:48.420 align:middle line:84% AUDIENCE: Is [INAUDIBLE] the set of [INAUDIBLE] over g 01:15:48.420 --> 01:15:54.450 align:middle line:84% [INAUDIBLE] smaller [INAUDIBLE] that goes to 0 [INAUDIBLE]? 01:15:54.450 --> 01:15:55.900 align:middle line:84% ZACHARY ABEL: Excellent question. 01:15:55.900 --> 01:15:59.160 align:middle line:90% Does little o imply big O? 01:15:59.160 --> 01:16:01.810 align:middle line:90% So if we know that f-- 01:16:01.810 --> 01:16:05.010 align:middle line:84% oops, sorry-- if we know that f is in little o of g, 01:16:05.010 --> 01:16:10.170 align:middle line:84% does that imply that f is in big O of g? 01:16:10.170 --> 01:16:10.860 align:middle line:90% Yeah. 01:16:10.860 --> 01:16:12.660 align:middle line:90% Yeah, absolutely. 01:16:12.660 --> 01:16:15.420 align:middle line:90% And let's think about why. 01:16:15.420 --> 01:16:20.280 align:middle line:84% So, little o is defined by, let's see, f-- 01:16:20.280 --> 01:16:25.320 align:middle line:84% sorry, f over g, that limit equals 0. 01:16:25.320 --> 01:16:28.130 align:middle line:84% But remember, we already said early on that 01:16:28.130 --> 01:16:31.640 align:middle line:84% if the limit exists, then it tells you whether or not 01:16:31.640 --> 01:16:34.560 align:middle line:84% it's big O. And in this case, the limit is 0. 01:16:34.560 --> 01:16:38.810 align:middle line:84% So it tells us yes, we are big O. 01:16:38.810 --> 01:16:42.120 align:middle line:84% So little o is absolutely stronger than big O. 01:16:42.120 --> 01:16:48.620 align:middle line:84% It implies big O. We certainly don't go the other way. 01:16:48.620 --> 01:16:54.290 align:middle line:84% Likewise, let's see, there are lots of other implications 01:16:54.290 --> 01:16:55.980 align:middle line:90% between this, between these. 01:16:55.980 --> 01:17:00.620 align:middle line:84% So theta, if f is theta of g, well, that certainly 01:17:00.620 --> 01:17:05.060 align:middle line:84% implies big O and big omega, because that's 01:17:05.060 --> 01:17:06.800 align:middle line:90% how we defined theta. 01:17:06.800 --> 01:17:14.090 align:middle line:84% And in turn, these also imply little o and little omega. 01:17:14.090 --> 01:17:16.980 align:middle line:84% Because big O implies little o, as we just saw, 01:17:16.980 --> 01:17:20.060 align:middle line:84% which means big omega implies little omega, because we're just 01:17:20.060 --> 01:17:22.100 align:middle line:90% swapping the order. 01:17:22.100 --> 01:17:25.490 align:middle line:84% So yeah, we have lots of implications like this. 01:17:25.490 --> 01:17:27.960 align:middle line:90% Let me write it this way. 01:17:27.960 --> 01:17:33.330 align:middle line:84% This implies little o, and this implies little omega. 01:17:33.330 --> 01:17:38.280 align:middle line:84% And as we already saw, asymptotic equivalence 01:17:38.280 --> 01:17:45.150 align:middle line:84% implies theta, which in turn, implies all the others. 01:17:45.150 --> 01:17:46.025 align:middle line:90% Yes? 01:17:46.025 --> 01:17:46.650 align:middle line:90% AUDIENCE: Wait. 01:17:46.650 --> 01:17:48.515 align:middle line:90% [INAUDIBLE] 01:17:48.515 --> 01:17:50.140 align:middle line:84% ZACHARY ABEL: I got these arrows wrong. 01:17:50.140 --> 01:17:51.040 align:middle line:90% Thank you. 01:17:51.040 --> 01:17:53.400 align:middle line:90% No. 01:17:53.400 --> 01:17:55.690 align:middle line:84% See, I shouldn't ad lib at the board. 01:17:55.690 --> 01:17:58.230 align:middle line:84% I'm always going to get it wrong. 01:17:58.230 --> 01:17:59.650 align:middle line:90% Yeah, I apologize. 01:17:59.650 --> 01:18:02.820 align:middle line:84% Theta implies the big ones, not the little ones. 01:18:02.820 --> 01:18:07.320 align:middle line:84% The little one implies the big one, not the other way around. 01:18:07.320 --> 01:18:09.300 align:middle line:90% Sorry for the confusion. 01:18:09.300 --> 01:18:10.280 align:middle line:90% One more question? 01:18:10.280 --> 01:18:16.140 align:middle line:90% 01:18:16.140 --> 01:18:17.500 align:middle line:90% All right, thanks, everyone. 01:18:17.500 --> 01:18:20.510 align:middle line:84% We'll see you tomorrow at recitation. 01:18:20.510 --> 01:18:25.000 align:middle line:90%