WEBVTT 00:00:00.050 --> 00:00:01.770 The following content is provided 00:00:01.770 --> 00:00:04.010 under a Creative Commons license. 00:00:04.010 --> 00:00:06.860 Your support will help MIT OpenCourseWare continue 00:00:06.860 --> 00:00:10.720 to offer high quality educational resources for free. 00:00:10.720 --> 00:00:13.330 To make a donation or view additional materials 00:00:13.330 --> 00:00:17.226 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:17.226 --> 00:00:17.851 at ocw.mit.edu. 00:00:21.830 --> 00:00:23.830 PROFESSOR: If you guys want me to cover anything 00:00:23.830 --> 00:00:25.300 in particular, is there anything you 00:00:25.300 --> 00:00:26.508 didn't understand in lecture? 00:00:29.840 --> 00:00:32.640 In the last section, I covered the recursion trees 00:00:32.640 --> 00:00:34.170 because they will be on the Pset, 00:00:34.170 --> 00:00:35.940 and people said they were a bit unclear, 00:00:35.940 --> 00:00:42.312 so we can do that and cover less of the stuff that I have here. 00:00:42.312 --> 00:00:44.770 Or if there's anything else, you can tell me what you want. 00:00:44.770 --> 00:00:48.040 So there I cover recursion trees because someone said, hey, 00:00:48.040 --> 00:00:50.000 can you go over that again? 00:00:50.000 --> 00:00:51.310 Is there any pain points? 00:00:56.620 --> 00:00:57.120 No? 00:00:57.120 --> 00:00:57.936 OK. 00:00:57.936 --> 00:00:59.810 So then I'm going to give you the same choice 00:00:59.810 --> 00:01:02.310 that I gave to people last time, and that 00:01:02.310 --> 00:01:06.320 is we can go over recursion trees again, 00:01:06.320 --> 00:01:08.630 but if I do that, then I won't have time to go over 00:01:08.630 --> 00:01:11.129 the code for deleting a node from a binary search tree. 00:01:11.129 --> 00:01:12.920 So we'll go through the theory and you guys 00:01:12.920 --> 00:01:15.070 will have to go through the code on your own. 00:01:15.070 --> 00:01:19.460 But instead, we'll go over recursion trees again 00:01:19.460 --> 00:01:21.770 and remember how you solve a recurrence using 00:01:21.770 --> 00:01:23.450 recursion trees. 00:01:23.450 --> 00:01:25.120 The alternative is we don't do that 00:01:25.120 --> 00:01:29.982 and we complete the deletions part. 00:01:29.982 --> 00:01:31.690 AUDIENCE: I feel like covering deletions, 00:01:31.690 --> 00:01:34.358 since we didn't do that in lecture, that would probably 00:01:34.358 --> 00:01:36.032 be more helpful. 00:01:36.032 --> 00:01:37.240 PROFESSOR: Let's take a vote. 00:01:37.240 --> 00:01:40.730 Who wants to do deletions in painstaking detail? 00:01:40.730 --> 00:01:45.410 So deletions and not recursion? 00:01:45.410 --> 00:01:49.452 Who wants to do recursion trees and not deletion? 00:01:49.452 --> 00:01:50.580 AUDIENCE: It's about equal. 00:01:50.580 --> 00:01:52.205 PROFESSOR: It's equal and nobody cares. 00:01:52.205 --> 00:01:52.870 I'm really sad. 00:01:58.376 --> 00:02:00.230 AUDIENCE: Let's do both in half detail. 00:02:00.230 --> 00:02:01.310 PROFESSOR: OK, sure. 00:02:01.310 --> 00:02:04.080 Who remembers merge sort? 00:02:04.080 --> 00:02:06.490 What does merge sort do really quick? 00:02:06.490 --> 00:02:10.050 AUDIENCE: It takes some sort of unsorted array, 00:02:10.050 --> 00:02:13.310 splits it in half, and then continually splits it, 00:02:13.310 --> 00:02:15.480 and then once it finally gets to the point 00:02:15.480 --> 00:02:20.540 where you have arrays of two elements, then it sorts them, 00:02:20.540 --> 00:02:23.088 and then sorts those, and then sorts those. 00:02:23.088 --> 00:02:24.669 It's a fun thing. 00:02:24.669 --> 00:02:25.960 And then it merges [INAUDIBLE]. 00:02:25.960 --> 00:02:28.680 PROFESSOR: That's so much code. 00:02:28.680 --> 00:02:31.360 I don't like to write much code because for every line of code 00:02:31.360 --> 00:02:33.151 that you write, you might have a bug in it, 00:02:33.151 --> 00:02:34.530 so I like to write less code. 00:02:34.530 --> 00:02:37.950 So the way I do it is when I get to an array size of one 00:02:37.950 --> 00:02:40.550 element, I know it's already sorted. 00:02:40.550 --> 00:02:41.640 So merge sort. 00:02:41.640 --> 00:02:43.380 You have an array, it's unsorted. 00:02:43.380 --> 00:02:47.030 Split it into two halves, call merge sort on each half, 00:02:47.030 --> 00:02:49.780 assume that magically, they're going to come back sorted, 00:02:49.780 --> 00:02:52.300 and then you merge the sorted halves. 00:02:52.300 --> 00:02:53.810 How much time does merging take? 00:02:57.590 --> 00:02:58.460 OK. 00:02:58.460 --> 00:03:03.042 So the recursion for the running time of merge sort? 00:03:03.042 --> 00:03:04.536 AUDIENCE: Why does it take n time? 00:03:04.536 --> 00:03:05.161 Just too large? 00:03:05.161 --> 00:03:06.869 AUDIENCE: Isn't it the finger thing where 00:03:06.869 --> 00:03:09.018 you take each element, and you're like, this one, 00:03:09.018 --> 00:03:12.260 is that greater or less than, then you put it in the array. 00:03:12.260 --> 00:03:12.760 So you get-- 00:03:12.760 --> 00:03:16.694 PROFESSOR: Please take my word for it that it's order n. 00:03:16.694 --> 00:03:18.860 AUDIENCE: I'll explain it and then I'll be confused. 00:03:21.670 --> 00:03:22.985 PROFESSOR: OK, so order n. 00:03:22.985 --> 00:03:24.075 What's the recursion? 00:03:28.422 --> 00:03:30.130 Don't give me the solution because then I 00:03:30.130 --> 00:03:32.530 can't do the trees anymore, so give me the recursion 00:03:32.530 --> 00:03:33.675 before it's solved. 00:03:33.675 --> 00:03:37.160 Give me the recurrence formula. 00:03:37.160 --> 00:03:39.566 So it starts with T of N, right? 00:03:39.566 --> 00:03:43.117 AUDIENCE: It starts with N over 2 plus N, I think. 00:03:46.844 --> 00:03:47.635 PROFESSOR: Perfect. 00:03:50.730 --> 00:03:52.690 So you take the array, you split it into two, 00:03:52.690 --> 00:03:56.150 you call merge sort on the two halves of the arrays. 00:03:56.150 --> 00:03:57.670 So you call merge sort twice. 00:03:57.670 --> 00:03:59.140 That's why you have a 2 here. 00:03:59.140 --> 00:03:59.800 The 2 matters. 00:03:59.800 --> 00:04:01.970 Without it, you get a different answer. 00:04:01.970 --> 00:04:04.040 And when you call it, the arrays that you 00:04:04.040 --> 00:04:06.910 give it are half the size, and then merge 00:04:06.910 --> 00:04:08.760 takes order and time. 00:04:08.760 --> 00:04:11.750 Splitting depends on what you're using to store your arrays. 00:04:11.750 --> 00:04:14.860 Can be constant time or it can be order N. So the time 00:04:14.860 --> 00:04:17.810 won't change because of split. 00:04:17.810 --> 00:04:20.845 How do we solve this recurrence? 00:04:24.162 --> 00:04:25.870 The recursion tree method says that we're 00:04:25.870 --> 00:04:27.680 going to draw a call graph. 00:04:27.680 --> 00:04:29.200 So we start out with a call to merge 00:04:29.200 --> 00:04:31.910 sort with an array of size N. Then 00:04:31.910 --> 00:04:33.610 it's going to call merge sort again, 00:04:33.610 --> 00:04:35.000 but after the array is split. 00:04:35.000 --> 00:04:38.730 So it's going to call merge sort twice, size is N over 2. 00:04:41.390 --> 00:04:45.680 This guy gets an array of N over 2, calls merge sort. 00:04:45.680 --> 00:04:50.760 Two arrays, sizes N over 4, N over 4. 00:04:50.760 --> 00:04:51.630 This does the same. 00:04:54.940 --> 00:05:00.420 So this goes on forever and ever and ever until at some point 00:05:00.420 --> 00:05:03.240 we reach our base case. 00:05:03.240 --> 00:05:05.940 So we're going to have a bunch of calls 00:05:05.940 --> 00:05:09.460 here where the array size is? 00:05:09.460 --> 00:05:10.521 What's our base case? 00:05:10.521 --> 00:05:11.020 1. 00:05:11.020 --> 00:05:11.519 Excellent. 00:05:15.870 --> 00:05:18.240 So this is the call graph for merge sort, and let's 00:05:18.240 --> 00:05:20.930 put the base case here so we know what we're talking about. 00:05:20.930 --> 00:05:23.370 T of 1 is theta 1. 00:05:25.876 --> 00:05:27.250 Now inside the nodes, we're going 00:05:27.250 --> 00:05:32.150 to put the cost for each call without counting 00:05:32.150 --> 00:05:34.770 the sub-call, so the children here. 00:05:34.770 --> 00:05:38.370 That's this guy here, except instead of order N, 00:05:38.370 --> 00:05:40.160 I will write CN. 00:05:40.160 --> 00:05:42.870 Remember how sometimes we use CN instead 00:05:42.870 --> 00:05:45.500 of the order of notation? 00:05:45.500 --> 00:05:49.150 The reason we do that is if I put in the asymptotic notation, 00:05:49.150 --> 00:05:52.930 then we're going to be tempted to sum them up. 00:05:52.930 --> 00:05:56.950 You're allowed to sum terms using asymptotic notation as 00:05:56.950 --> 00:05:59.200 long as there's a finite number of them, 00:05:59.200 --> 00:06:02.420 but here, it turns out there's an infinite number of them. 00:06:02.420 --> 00:06:05.150 Also, if you go this way, you can never go wrong. 00:06:05.150 --> 00:06:08.030 You always get the right answer, so that's 00:06:08.030 --> 00:06:11.620 why we switch from order N to CN. 00:06:11.620 --> 00:06:14.980 In order to merge sort an array of size N, 00:06:14.980 --> 00:06:18.100 we're going to merge sort two arrays of size N over 2 00:06:18.100 --> 00:06:21.690 and then spend CN time on doing the merge. 00:06:21.690 --> 00:06:23.355 What are the costs here? 00:06:26.050 --> 00:06:28.140 To sort an array of N over 2, what's 00:06:28.140 --> 00:06:31.670 the cost outside the cost to merge? 00:06:31.670 --> 00:06:33.510 AUDIENCE: C of N over 2. 00:06:33.510 --> 00:06:34.860 PROFESSOR: Perfect. 00:06:34.860 --> 00:06:37.000 C times N over 2. 00:06:37.000 --> 00:06:39.410 C times N over 2. 00:06:39.410 --> 00:06:41.046 How about here? 00:06:41.046 --> 00:06:42.978 AUDIENCE: C times N over 4. 00:06:42.978 --> 00:06:45.876 PROFESSOR: Perfect. 00:06:45.876 --> 00:06:48.850 CN over 4. 00:06:48.850 --> 00:06:50.610 My nodes are really ugly. 00:06:50.610 --> 00:06:52.860 I should have drawn them like this from the beginning. 00:06:52.860 --> 00:06:54.270 CN over 4. 00:06:54.270 --> 00:06:55.250 There you go. 00:06:55.250 --> 00:06:57.620 How about down here? 00:06:57.620 --> 00:07:01.142 AUDIENCE: C of N over 2 to the i. 00:07:01.142 --> 00:07:03.490 PROFESSOR: You're going on step ahead. 00:07:03.490 --> 00:07:05.610 We'll do that right next. 00:07:05.610 --> 00:07:08.540 AUDIENCE: C of N over log N, right? 00:07:08.540 --> 00:07:11.220 Because they're log N levels, so-- 00:07:11.220 --> 00:07:14.140 PROFESSOR: Let's not worry about the number of levels. 00:07:14.140 --> 00:07:15.980 You're ruining my steps. 00:07:15.980 --> 00:07:18.455 I was going to get to that two steps after this. 00:07:18.455 --> 00:07:19.564 AUDIENCE: Is it just C? 00:07:19.564 --> 00:07:20.320 PROFESSOR: Yep. 00:07:20.320 --> 00:07:21.840 So array size is 1, right? 00:07:21.840 --> 00:07:28.140 So the cost is C. C, C, C, C. OK, you guys got it 00:07:28.140 --> 00:07:30.179 if you're thinking of levels already. 00:07:30.179 --> 00:07:31.720 The next thing I want to do is I want 00:07:31.720 --> 00:07:34.740 to figure out how many levels I have in this tree. 00:07:34.740 --> 00:07:36.600 Why do I care about that? 00:07:36.600 --> 00:07:40.900 The answer for T of N is the sum of all these costs in here 00:07:40.900 --> 00:07:44.550 because the cost of merge sorting an array of size N 00:07:44.550 --> 00:07:49.940 is the merge sort plus the costs for sorting the two arrays. 00:07:49.940 --> 00:07:52.890 And the nodes here keep track of all the time spent 00:07:52.890 --> 00:07:56.840 in recursive sub-calls, so if we can add up everything up, 00:07:56.840 --> 00:08:01.880 we have the answer to T of N. It turns out the easiest 00:08:01.880 --> 00:08:06.050 way to do that is to sum up the cost at each level 00:08:06.050 --> 00:08:10.875 because the costs are this guy copied over here. 00:08:10.875 --> 00:08:13.800 For a level, they tend to be the same, 00:08:13.800 --> 00:08:17.360 so it's reasonably easy to add them up, 00:08:17.360 --> 00:08:19.430 except in order to be able to add those up, 00:08:19.430 --> 00:08:22.980 you have to know how many levels you have. 00:08:22.980 --> 00:08:26.920 So how do I know how many levels I have? 00:08:26.920 --> 00:08:30.470 Someone already told me log N. How do I get to that log N? 00:08:35.980 --> 00:08:40.440 So when I get to the bottommost level, 00:08:40.440 --> 00:08:43.460 the number has to be 1, the number next to the node, 00:08:43.460 --> 00:08:45.080 because that's my base case. 00:08:45.080 --> 00:08:47.490 When I have a one element array, it's sorted, I'm done. 00:08:47.490 --> 00:08:49.650 I return. 00:08:49.650 --> 00:08:53.130 So I can say that for each level, 00:08:53.130 --> 00:08:58.150 the number next to the node is something as a function of L. 00:08:58.150 --> 00:08:59.750 Here, I'm going to say that this is 00:08:59.750 --> 00:09:06.850 N over 1, which is N over 2 to the 0 power. 00:09:06.850 --> 00:09:11.480 And this is N over 2, so it's N over 2 to the first power. 00:09:11.480 --> 00:09:16.280 This is N over 2 to the second, and so on and so forth. 00:09:16.280 --> 00:09:18.476 It might not be obvious if you only have two levels. 00:09:18.476 --> 00:09:20.100 I don't want to draw a lot on the board 00:09:20.100 --> 00:09:23.180 because I don't have a lot of space and I'd get my nodes 00:09:23.180 --> 00:09:25.420 all messed into each other. 00:09:25.420 --> 00:09:28.690 If it takes more than two levels to see the pattern, go for it. 00:09:28.690 --> 00:09:31.440 Expand for three levels, four levels, five levels, 00:09:31.440 --> 00:09:35.070 whatever it takes to get it right on a Pset or on a test. 00:09:35.070 --> 00:09:37.430 So you see the pattern, then you write the formula 00:09:37.430 --> 00:09:42.140 for the node size at the level. 00:09:42.140 --> 00:09:44.770 And assuming this pattern holds, we 00:09:44.770 --> 00:09:50.270 see that the size of a node at level l, the size 00:09:50.270 --> 00:09:56.880 is 2 N over 2 to the l minus 1. 00:09:56.880 --> 00:09:58.330 Fair enough? 00:09:58.330 --> 00:10:01.770 You can say N over 2 to the l, and forget 00:10:01.770 --> 00:10:04.730 that there's a minus 1, and then the asymptotics will save you, 00:10:04.730 --> 00:10:09.680 so it's no big deal, but this is the real number. 00:10:09.680 --> 00:10:16.310 So that means that at the bottommost level, at level l, 00:10:16.310 --> 00:10:18.900 this size is going to be 1. 00:10:18.900 --> 00:10:24.110 N over 2 to the l minus 1 equals 1. 00:10:24.110 --> 00:10:26.620 So now this is an equation, so I can solve for l. 00:10:26.620 --> 00:10:31.630 I pull this on the right side, N equals 2 to the l minus 1, 00:10:31.630 --> 00:10:37.771 so l minus 1 equals-- anyone? 00:10:37.771 --> 00:10:39.020 The inverse of an exponential? 00:10:41.720 --> 00:10:43.220 AUDIENCE: I wasn't paying attention. 00:10:43.220 --> 00:10:45.390 Sorry. 00:10:45.390 --> 00:10:50.837 AUDIENCE: Log N. 00:10:50.837 --> 00:10:53.170 PROFESSOR: The inverse of an exponential is a logarithm. 00:10:53.170 --> 00:10:58.470 Keep that in mind for solving 6.006 problems. 00:10:58.470 --> 00:11:06.289 l minus 1 is log N so l is log n plus 1, roughly log n. 00:11:06.289 --> 00:11:08.330 I could use log n plus 1 and go through the math. 00:11:08.330 --> 00:11:10.580 It's a bit more painful and, because we're 00:11:10.580 --> 00:11:13.940 using asymptotics, it doesn't really matter. 00:11:13.940 --> 00:11:15.610 So now we know how many levels we have. 00:11:15.610 --> 00:11:18.050 Let's see what's the cost at the level. 00:11:18.050 --> 00:11:19.740 So all the calls at a certain level, 00:11:19.740 --> 00:11:21.640 what's the sum of the costs? 00:11:21.640 --> 00:11:25.400 For this level, what's the cost? 00:11:29.600 --> 00:11:30.100 CN. 00:11:30.100 --> 00:11:32.310 And That was the easy question. 00:11:32.310 --> 00:11:33.635 Just the root, right? 00:11:33.635 --> 00:11:34.510 How about this level? 00:11:38.770 --> 00:11:43.180 Because I have two nodes, the cost in each node is CN over 2. 00:11:43.180 --> 00:11:44.060 How about this level? 00:11:49.800 --> 00:11:52.110 Four levels, each level CN over 4. 00:11:52.110 --> 00:11:53.340 How about the bottom level? 00:11:58.752 --> 00:12:01.680 AUDIENCE: CN. 00:12:01.680 --> 00:12:04.486 PROFESSOR: Why is it CN? 00:12:04.486 --> 00:12:08.440 AUDIENCE: Because there are N arrays of size 1. 00:12:08.440 --> 00:12:09.710 PROFESSOR: N arrays of size 1. 00:12:09.710 --> 00:12:11.530 Excellent. 00:12:11.530 --> 00:12:14.510 A cute argument I heard once is you start out with N, 00:12:14.510 --> 00:12:16.910 you split it into N over 2 and N over 2. 00:12:16.910 --> 00:12:19.520 Then you split this guy in N over 4, N over 4, 00:12:19.520 --> 00:12:21.980 so this is like conservation of mass. 00:12:21.980 --> 00:12:26.380 If you start with N and here, you don't end up with N total, 00:12:26.380 --> 00:12:30.210 then you lost some element somewhere on the way. 00:12:30.210 --> 00:12:32.100 So CN. 00:12:32.100 --> 00:12:33.310 CN, CN, CN, CN. 00:12:33.310 --> 00:12:34.530 I think I see a pattern. 00:12:34.530 --> 00:12:38.540 I think it's reasonable to say that for every level, it's CN. 00:12:38.540 --> 00:12:40.950 And if you write the proof, you can prove that 00:12:40.950 --> 00:12:44.300 by using math instead of waving hands. 00:12:44.300 --> 00:12:47.420 So CN times the number of levels, right? 00:12:47.420 --> 00:12:56.040 The answer for this guy is C of N is CN times l. 00:12:56.040 --> 00:12:57.586 What's l? 00:12:57.586 --> 00:12:59.289 AUDIENCE: N log N. 00:12:59.289 --> 00:13:00.080 PROFESSOR: Roughly. 00:13:00.080 --> 00:13:10.240 OK So order of N log N. C becomes 00:13:10.240 --> 00:13:14.630 order of, l is order of log N, N stays the same. 00:13:18.320 --> 00:13:18.970 Any questions? 00:13:24.100 --> 00:13:27.837 Are people getting it or did I confuse you even more? 00:13:27.837 --> 00:13:28.670 AUDIENCE: We got it. 00:13:28.670 --> 00:13:29.897 PROFESSOR: OK, sweet. 00:13:29.897 --> 00:13:31.230 Thank you for the encouragement. 00:13:31.230 --> 00:13:36.200 So this gets you through problem one of Pset 2. 00:13:36.200 --> 00:13:38.400 So in this case, the tree is nicely balanced. 00:13:38.400 --> 00:13:40.284 The cost at each level is the same. 00:13:40.284 --> 00:13:42.700 When [INAUDIBLE] talked about recursion trees in lectures, 00:13:42.700 --> 00:13:45.230 he showed two more trees, one where 00:13:45.230 --> 00:13:49.350 pretty much all the cost was up here-- the cost of the children 00:13:49.350 --> 00:13:54.000 was negligible-- and one tree where all the cost was 00:13:54.000 --> 00:13:57.240 concentrated here, so the cost of all the inner nodes 00:13:57.240 --> 00:14:01.160 was negligible and the leaves were doing all the real work. 00:14:01.160 --> 00:14:03.804 So don't be scared if your costs aren't the same. 00:14:03.804 --> 00:14:05.970 Just sum them up and you'll get to the right answer. 00:14:11.470 --> 00:14:13.770 Now I'm going to talk about binary search trees, 00:14:13.770 --> 00:14:18.920 except I will make a five minute general talk about data 00:14:18.920 --> 00:14:20.700 structures before I do that. 00:14:20.700 --> 00:14:22.310 So we use the term "data structures." 00:14:22.310 --> 00:14:24.296 I think we covered it well, and I 00:14:24.296 --> 00:14:25.670 want to give you a couple of tips 00:14:25.670 --> 00:14:29.280 for dealing with them on Pset 1. 00:14:29.280 --> 00:14:31.390 A data structure is a bunch of algorithms 00:14:31.390 --> 00:14:35.380 that help you store and then retrieve information. 00:14:35.380 --> 00:14:36.960 You have two types of algorithms. 00:14:36.960 --> 00:14:44.795 You have queries, and you have updates. 00:14:49.850 --> 00:14:51.920 You start out with an empty data structure, 00:14:51.920 --> 00:14:55.870 like an empty binary search tree or an empty list, 00:14:55.870 --> 00:14:58.370 and then you throw some data at it. 00:14:58.370 --> 00:14:59.840 That's when you update it. 00:14:59.840 --> 00:15:03.129 Then you ask it some questions, and that's when you query it. 00:15:03.129 --> 00:15:05.670 Then maybe you throw more data at it, so you do more updates, 00:15:05.670 --> 00:15:07.794 and you ask more questions, so you do more queries. 00:15:10.190 --> 00:15:12.460 What are the queries and the updates 00:15:12.460 --> 00:15:15.200 for the binary search trees that we talked about in lecture? 00:15:18.660 --> 00:15:21.140 AUDIENCE: A query would be like, what's 00:15:21.140 --> 00:15:23.977 your right child, what's your left child? 00:15:23.977 --> 00:15:25.310 PROFESSOR: So that's for a node. 00:15:28.502 --> 00:15:29.960 AUDIENCE: What are you looking for? 00:15:29.960 --> 00:15:31.460 PROFESSOR: I'm looking for something 00:15:31.460 --> 00:15:32.812 for the entire infrastructure. 00:15:32.812 --> 00:15:34.520 So for the entire tree, what's a question 00:15:34.520 --> 00:15:36.090 that you would ask the tree? 00:15:36.090 --> 00:15:37.147 PROFESSOR: Max. 00:15:37.147 --> 00:15:37.730 PROFESSOR: OK. 00:15:46.750 --> 00:15:47.250 Min. 00:15:50.914 --> 00:15:51.830 AUDIENCE: Next larger. 00:15:51.830 --> 00:15:52.460 PROFESSOR: Next larger. 00:15:52.460 --> 00:15:53.940 Are you looking at the nodes? 00:15:58.640 --> 00:16:02.860 AUDIENCE: Is there an are you balanced question? 00:16:02.860 --> 00:16:05.665 PROFESSOR: Well, I would say that the most popular operation 00:16:05.665 --> 00:16:08.980 in a binary search tree is Search, which 00:16:08.980 --> 00:16:12.560 looks for-- we call it Find in the code 00:16:12.560 --> 00:16:16.640 because most code implementations call it Find 00:16:16.640 --> 00:16:18.359 nowadays, but binary search tree. 00:16:18.359 --> 00:16:19.650 What are you going to do in it? 00:16:19.650 --> 00:16:20.870 You search for a value. 00:16:20.870 --> 00:16:23.920 That's why it has the Search in binary search. 00:16:23.920 --> 00:16:27.580 So queries are operations where you ask questions to the data 00:16:27.580 --> 00:16:31.540 structure and it doesn't change. 00:16:31.540 --> 00:16:32.400 How about updates? 00:16:32.400 --> 00:16:33.940 What did we learn for updates? 00:16:36.712 --> 00:16:37.640 AUDIENCE: Insert. 00:16:37.640 --> 00:16:39.740 PROFESSOR: Excellent. 00:16:39.740 --> 00:16:41.630 So Insert was covered in lecture, 00:16:41.630 --> 00:16:44.400 and we're doing Delete today. 00:16:49.740 --> 00:16:53.030 So data structures have this property 00:16:53.030 --> 00:16:56.460 that's called the representation invariant, RI, 00:16:56.460 --> 00:16:57.930 or Rep Invariant. 00:17:05.819 --> 00:17:09.835 Actually, before I get there, the rep invariant 00:17:09.835 --> 00:17:12.640 says that the data in the data structures 00:17:12.640 --> 00:17:14.829 is organized in this way, and as long 00:17:14.829 --> 00:17:17.349 as it's organized in this way, the data structure functions 00:17:17.349 --> 00:17:18.490 correctly. 00:17:18.490 --> 00:17:20.810 Can someone guess for a sorted array 00:17:20.810 --> 00:17:24.829 what's the representation invariant? 00:17:24.829 --> 00:17:27.294 AUDIENCE: It can mean sorted. 00:17:27.294 --> 00:17:27.960 PROFESSOR: Yeah. 00:17:27.960 --> 00:17:29.350 A sorted array should be sorted. 00:17:29.350 --> 00:17:31.690 Sounds like a very good rep invariant. 00:17:31.690 --> 00:17:33.530 So the elements should be stored an array. 00:17:33.530 --> 00:17:38.340 Every element should be smaller than any element after it. 00:17:38.340 --> 00:17:42.370 And as long as the rep invariant holds, so as long 00:17:42.370 --> 00:17:44.770 as elements are stored in the right way in the data 00:17:44.770 --> 00:17:48.890 structure, the queries will return the right results. 00:17:48.890 --> 00:17:50.360 If the rep invariant doesn't hold, 00:17:50.360 --> 00:17:53.364 then God knows what's going to happen. 00:17:53.364 --> 00:17:54.780 What can you do in a storage array 00:17:54.780 --> 00:17:56.800 as long as the rep invariant holds? 00:18:00.970 --> 00:18:01.927 Sorted array. 00:18:01.927 --> 00:18:04.010 What's the reason why I would have a sorted array? 00:18:04.010 --> 00:18:06.390 What can I do that's fast in a sorted array? 00:18:06.390 --> 00:18:08.660 AUDIENCE: Min and Max. 00:18:08.660 --> 00:18:10.140 PROFESSOR: I can do that very fast. 00:18:10.140 --> 00:18:10.790 That's good. 00:18:10.790 --> 00:18:12.414 What's the running time? 00:18:12.414 --> 00:18:13.290 AUDIENCE: A constant. 00:18:13.290 --> 00:18:14.219 PROFESSOR: Perfect. 00:18:14.219 --> 00:18:16.510 Min you look at the beginning, Max you look at the end. 00:18:16.510 --> 00:18:17.140 Yes? 00:18:17.140 --> 00:18:18.210 AUDIENCE: Binary search. 00:18:18.210 --> 00:18:19.251 PROFESSOR: Binary search. 00:18:19.251 --> 00:18:21.130 That's the other reason for that. 00:18:21.130 --> 00:18:23.090 So binary search runs in order log 00:18:23.090 --> 00:18:26.690 N time, doesn't have to look at most of the array, 00:18:26.690 --> 00:18:28.690 tells you whether an element is there are not. 00:18:28.690 --> 00:18:31.100 Now, what if the array is unsorted? 00:18:31.100 --> 00:18:33.840 Will the algorithm work? 00:18:33.840 --> 00:18:35.270 It might say something isn't there 00:18:35.270 --> 00:18:36.353 when it actually is there. 00:18:36.353 --> 00:18:38.790 You can do binary search on a non-sorted array. 00:18:38.790 --> 00:18:40.440 So if the rep invariant doesn't hold, 00:18:40.440 --> 00:18:42.390 your queries might give you a wrong answer. 00:18:44.940 --> 00:18:46.460 How about updates? 00:18:46.460 --> 00:18:49.370 How do you search something in a sorted list? 00:18:53.223 --> 00:18:54.764 AUDIENCE: You find where it should go 00:18:54.764 --> 00:18:55.885 and you move everything. 00:18:55.885 --> 00:18:56.510 PROFESSOR: Yep. 00:18:56.510 --> 00:18:59.110 So you have to move everything, make room for it, 00:18:59.110 --> 00:19:01.030 and put it there so that the array is still 00:19:01.030 --> 00:19:02.900 sorted at the end. 00:19:02.900 --> 00:19:04.570 You can't just append things at the end, 00:19:04.570 --> 00:19:10.610 even though that would be faster and lazier and less code. 00:19:10.610 --> 00:19:12.467 When you do an update to a data structure, 00:19:12.467 --> 00:19:14.550 you have to make sure that the rep invariant still 00:19:14.550 --> 00:19:16.300 holds at the end. 00:19:16.300 --> 00:19:18.800 Sort of a correctness proof for an update algorithm 00:19:18.800 --> 00:19:21.580 says that if the rep invariant holds at the beginning, 00:19:21.580 --> 00:19:24.900 the rep invariant is guaranteed to hold at the end. 00:19:24.900 --> 00:19:27.640 Why do we care about this rep invariant stuff? 00:19:27.640 --> 00:19:32.700 Suppose you have a problem, say on the next Pset, that 00:19:32.700 --> 00:19:36.890 asks you to find the place that's slow in your code 00:19:36.890 --> 00:19:39.490 and then speed it up. 00:19:39.490 --> 00:19:42.336 And suppose you recognize the data structure there, 00:19:42.336 --> 00:19:43.960 and you say that's inefficient, and you 00:19:43.960 --> 00:19:46.430 want to implement another data structure that 00:19:46.430 --> 00:19:49.620 would be more efficient. 00:19:49.620 --> 00:19:52.000 You're going to implement it. 00:19:52.000 --> 00:19:54.390 You might have bugs in an update. 00:19:54.390 --> 00:19:56.460 How do you find the bugs? 00:19:56.460 --> 00:19:58.570 Queries give you the wrong answers. 00:19:58.570 --> 00:20:01.510 You might do queries a long time after you do updates, 00:20:01.510 --> 00:20:04.624 and you're not going to know which update failed. 00:20:04.624 --> 00:20:06.790 The right way to do this is you implement the method 00:20:06.790 --> 00:20:10.250 called Check RI-- that's what I call it-- so 00:20:10.250 --> 00:20:12.170 check the representation invariant. 00:20:12.170 --> 00:20:14.630 And that method walks through the entire data structure 00:20:14.630 --> 00:20:16.960 and make sure that the rep invariant holds, 00:20:16.960 --> 00:20:19.089 and if it doesn't, it raises an exception 00:20:19.089 --> 00:20:21.380 because you know that whatever you try to do from there 00:20:21.380 --> 00:20:25.580 is not going to work, so there's no reason to keep going. 00:20:25.580 --> 00:20:28.110 So at the end of every update, you add a call 00:20:28.110 --> 00:20:32.050 to this Check RI method until you're 00:20:32.050 --> 00:20:34.480 sure that your code is correct. 00:20:34.480 --> 00:20:36.390 And after you're done debugging your code, 00:20:36.390 --> 00:20:40.180 you remove this method and you submit the code. 00:20:40.180 --> 00:20:42.310 Why do I want to remove the method? 00:20:42.310 --> 00:20:45.290 It might be painfully slow and inefficient, much slower 00:20:45.290 --> 00:20:47.530 than the actual queries and updates. 00:20:47.530 --> 00:20:49.010 For example, let's take a heap. 00:20:49.010 --> 00:20:52.610 Do people remember heaps from lecture? 00:20:52.610 --> 00:20:54.870 What's the query for a heap? 00:20:54.870 --> 00:20:55.930 Say you have a max heap. 00:20:55.930 --> 00:20:58.567 What's a query? 00:20:58.567 --> 00:20:59.650 AUDIENCE: Where's the max? 00:20:59.650 --> 00:21:00.800 PROFESSOR: OK, cool. 00:21:00.800 --> 00:21:03.570 So for a max heap, a query would be max. 00:21:03.570 --> 00:21:04.200 Running time? 00:21:07.511 --> 00:21:08.457 AUDIENCE: Constant. 00:21:08.457 --> 00:21:09.248 PROFESSOR: Perfect. 00:21:09.248 --> 00:21:10.360 Constant. 00:21:10.360 --> 00:21:13.880 What do you do? 00:21:13.880 --> 00:21:15.122 Look at the top? 00:21:15.122 --> 00:21:16.307 AUDIENCE: Yeah, exactly. 00:21:16.307 --> 00:21:16.890 PROFESSOR: OK. 00:21:16.890 --> 00:21:17.640 Sweet. 00:21:17.640 --> 00:21:23.430 So what are the two popular updates in a max heap? 00:21:23.430 --> 00:21:25.600 AUDIENCE: There would be Insert as well. 00:21:25.600 --> 00:21:28.480 PROFESSOR: OK. 00:21:28.480 --> 00:21:29.770 Insert. 00:21:29.770 --> 00:21:32.306 And did we teach you general delete? 00:21:39.310 --> 00:21:44.817 Usually Extract Max is simpler. 00:21:44.817 --> 00:21:45.650 That's all you need. 00:21:51.570 --> 00:21:53.270 What's the running time for Insert? 00:21:56.830 --> 00:21:59.390 Do people remember heaps? 00:21:59.390 --> 00:22:03.190 AUDIENCE: I think it was per N, but I'm not completely sure. 00:22:03.190 --> 00:22:04.560 PROFESSOR: Anyone else? 00:22:04.560 --> 00:22:05.250 It's not. 00:22:05.250 --> 00:22:08.135 Life would be bad if it would be N. 00:22:08.135 --> 00:22:09.217 AUDIENCE: N squared? 00:22:09.217 --> 00:22:09.800 PROFESSOR: No. 00:22:13.835 --> 00:22:16.210 It's better than N, so you guys are doing a binary search 00:22:16.210 --> 00:22:18.660 over the few running times that I gave you earlier. 00:22:18.660 --> 00:22:20.118 AUDIENCE: [INAUDIBLE] add to the N, 00:22:20.118 --> 00:22:22.498 and then you compare your neighbor, and then you 00:22:22.498 --> 00:22:24.245 [INAUDIBLE]. 00:22:24.245 --> 00:22:26.780 AUDIENCE: If it's an array, there isn't-- 00:22:26.780 --> 00:22:29.272 PROFESSOR: So conceptually, a heap looks like this. 00:22:29.272 --> 00:22:30.980 And yeah, it becomes an array eventually, 00:22:30.980 --> 00:22:32.750 but let's look at it this way. 00:22:36.500 --> 00:22:37.950 It is a full binary tree. 00:22:37.950 --> 00:22:41.290 Binary tree means that each node has at most two children, 00:22:41.290 --> 00:22:44.160 and full means that every level except for the last level 00:22:44.160 --> 00:22:45.420 is completely populated. 00:22:45.420 --> 00:22:48.770 So every internal node has exactly two children, 00:22:48.770 --> 00:22:53.050 and in here, every node except for some nodes 00:22:53.050 --> 00:22:56.700 and then some nodes after it will not have. 00:22:56.700 --> 00:22:59.090 Everything to the left is fully populated, 00:22:59.090 --> 00:23:01.870 and then at some point, you stop having children. 00:23:01.870 --> 00:23:05.150 It turns out that this is easy to store in an array, 00:23:05.150 --> 00:23:06.660 but I will not go over that. 00:23:06.660 --> 00:23:09.660 Instead, I want to go over inserting. 00:23:09.660 --> 00:23:13.980 What's the rep invariant for a heap? 00:23:13.980 --> 00:23:15.960 AUDIENCE: The max in the top, right? 00:23:15.960 --> 00:23:18.572 Well, for max heap, and then the two children 00:23:18.572 --> 00:23:20.456 are less than the next node. 00:23:20.456 --> 00:23:21.330 PROFESSOR: All right. 00:23:21.330 --> 00:23:23.515 So the guy here has to be bigger than these guys, 00:23:23.515 --> 00:23:25.640 then the guy here has to be bigger than these guys, 00:23:25.640 --> 00:23:26.980 and so on and so forth. 00:23:26.980 --> 00:23:29.990 And if you use induction, you can 00:23:29.990 --> 00:23:32.212 prove that if this is bigger than this, 00:23:32.212 --> 00:23:34.170 it has to be bigger than these guys, and bigger 00:23:34.170 --> 00:23:36.970 than these guys, and bigger than everything, and it's a max. 00:23:36.970 --> 00:23:40.060 That's the reason why we have that rep invariant. 00:23:40.060 --> 00:23:42.480 So the way we insert a node is we add it 00:23:42.480 --> 00:23:45.280 at the bottom, the only place where we could add it. 00:23:45.280 --> 00:23:48.190 And then if this guy is bigger than this guy, 00:23:48.190 --> 00:23:51.110 the rep invariant is violated, so we swap them 00:23:51.110 --> 00:23:53.270 in order to fix that. 00:23:53.270 --> 00:23:54.270 Now the guy is here. 00:23:54.270 --> 00:23:56.890 If this is bigger than this, we do another swap. 00:23:56.890 --> 00:24:00.400 If this is bigger than this, we do another swap. 00:24:00.400 --> 00:24:03.990 So you're going to go from the bottom of the heap potentially 00:24:03.990 --> 00:24:06.210 all the way up to the root. 00:24:06.210 --> 00:24:08.820 So the running time of insert is order 00:24:08.820 --> 00:24:10.700 of the height of the heap. 00:24:13.580 --> 00:24:16.810 Now, the heap is a full binary tree. 00:24:16.810 --> 00:24:17.445 I said "full." 00:24:17.445 --> 00:24:18.320 I keep saying "full." 00:24:18.320 --> 00:24:20.765 The reason I care about full is that the full binary tree 00:24:20.765 --> 00:24:24.767 is guaranteed to have a height of log N. It's always 00:24:24.767 --> 00:24:26.350 log N, where N is the number of nodes. 00:24:30.120 --> 00:24:35.303 So inserting in a heap takes log N. 00:24:35.303 --> 00:24:36.469 AUDIENCE: I have a question. 00:24:36.469 --> 00:24:39.415 Didn't they say that because it's in an array, 00:24:39.415 --> 00:24:44.230 then to find it-- oh no, I guess because you can still 00:24:44.230 --> 00:24:44.949 do the swaps. 00:24:44.949 --> 00:24:46.490 PROFESSOR: You can still do the swaps 00:24:46.490 --> 00:24:48.720 when you have it serialized in an array. 00:24:48.720 --> 00:24:50.450 You know that given an item's index, 00:24:50.450 --> 00:24:53.894 the parent is that index divided by 2. 00:24:53.894 --> 00:24:55.810 So you add an element at the end of the array, 00:24:55.810 --> 00:24:57.100 and then you know what the parent is, 00:24:57.100 --> 00:24:58.988 and then you keep swapping and swapping and swapping 00:24:58.988 --> 00:24:59.944 towards the [INAUDIBLE]. 00:24:59.944 --> 00:25:01.300 AUDIENCE: You don't ever have to put it in 00:25:01.300 --> 00:25:02.072 and shift everything over. 00:25:02.072 --> 00:25:02.420 You're only swapping. 00:25:02.420 --> 00:25:03.045 PROFESSOR: Yep. 00:25:03.045 --> 00:25:04.200 You only swap. 00:25:04.200 --> 00:25:06.131 That's important. 00:25:06.131 --> 00:25:06.880 Thanks for asking. 00:25:06.880 --> 00:25:07.650 That's important. 00:25:07.650 --> 00:25:11.850 So log N. Extract max, take my word for it, 00:25:11.850 --> 00:25:16.380 also log N. What's the running time for checking 00:25:16.380 --> 00:25:19.760 the invariant in a heap? 00:25:19.760 --> 00:25:22.590 So to make sure that this guy is a heap, if I had numbers here, 00:25:22.590 --> 00:25:24.000 what would you have to do? 00:25:29.280 --> 00:25:31.295 AUDIENCE: You'd have to look at every node. 00:25:31.295 --> 00:25:31.920 PROFESSOR: Yep. 00:25:31.920 --> 00:25:34.304 So running time? 00:25:34.304 --> 00:25:36.547 AUDIENCE: Theta of N. 00:25:36.547 --> 00:25:37.172 PROFESSOR: Yep. 00:25:40.820 --> 00:25:43.430 So if I'm going to submit code for a heap 00:25:43.430 --> 00:25:46.410 where the operations are our order of log N, 00:25:46.410 --> 00:25:50.730 or order 1, but then each of these calls Check RI, 00:25:50.730 --> 00:25:52.640 that's going to be painfully slow because I'm 00:25:52.640 --> 00:25:55.650 making the updates be order N instead of log N. 00:25:55.650 --> 00:25:59.160 So you're putting Check RI calls in every update. 00:25:59.160 --> 00:26:00.607 You debug your code. 00:26:00.607 --> 00:26:02.690 When you make sure it's correct, you remove those, 00:26:02.690 --> 00:26:05.810 and then you submit the Pset. 00:26:05.810 --> 00:26:08.530 Make sense? 00:26:08.530 --> 00:26:09.760 Sweet. 00:26:09.760 --> 00:26:12.070 And we looked a little bit at heaps, which is good. 00:26:16.520 --> 00:26:18.320 Binary search trees. 00:26:18.320 --> 00:26:21.000 So a binary tree is a tree where every node 00:26:21.000 --> 00:26:23.830 has at most two children. 00:26:23.830 --> 00:26:28.460 When we code this up, we represent a node as a Python 00:26:28.460 --> 00:26:34.810 object, and for a node, we keep track of the left child, 00:26:34.810 --> 00:26:41.880 of the right child, parent, and then this is a hollow tree. 00:26:41.880 --> 00:26:43.100 It's not very useful. 00:26:43.100 --> 00:26:46.370 This becomes useful when you start putting keys in the nodes 00:26:46.370 --> 00:26:49.690 so that you can find them and do other things with them. 00:26:49.690 --> 00:26:50.830 So each node has a key. 00:26:53.640 --> 00:26:55.390 Let me draw a binary search tree. 00:27:11.275 --> 00:27:14.360 Can people see this? 00:27:14.360 --> 00:27:15.750 So this is a binary tree. 00:27:15.750 --> 00:27:18.360 Can someone say something a bit more specific about it? 00:27:23.050 --> 00:27:24.400 AUDIENCE: It's unbalanced. 00:27:24.400 --> 00:27:24.690 PROFESSOR: OK. 00:27:24.690 --> 00:27:25.356 It's imbalanced. 00:27:28.590 --> 00:27:31.260 So that means that finding things all the way 00:27:31.260 --> 00:27:34.051 at the bottom is going to be expensive. 00:27:34.051 --> 00:27:34.550 What else? 00:27:37.520 --> 00:27:39.140 So I said it's a binary tree. 00:27:39.140 --> 00:27:40.900 Give me something more specific. 00:27:45.830 --> 00:27:50.060 So binary tree just means that every node has two children. 00:27:50.060 --> 00:27:52.470 There's a bit more structure in this guy. 00:27:52.470 --> 00:27:55.840 So if I look at the root, if I look at 23, 00:27:55.840 --> 00:27:58.740 all the nodes to the left are smaller. 00:27:58.740 --> 00:28:01.635 All the nodes to the right are bigger. 00:28:01.635 --> 00:28:04.320 Now, if I look at 8, all the nodes to the left are smaller, 00:28:04.320 --> 00:28:06.360 all the nodes to the right are greater. 00:28:10.310 --> 00:28:15.750 This additional rep invariant defines a binary search tree. 00:28:15.750 --> 00:28:19.820 This is what we talked about in class. 00:28:19.820 --> 00:28:22.320 BST. 00:28:22.320 --> 00:28:24.945 Why would I want to have this rep invariant? 00:28:24.945 --> 00:28:27.540 It sounds like a pain to maintain nodes 00:28:27.540 --> 00:28:29.810 with all these ordering constraints. 00:28:29.810 --> 00:28:32.710 What's the advantage of doing that? 00:28:32.710 --> 00:28:33.996 AUDIENCE: Search is fast. 00:28:33.996 --> 00:28:34.620 PROFESSOR: Yep. 00:28:34.620 --> 00:28:35.310 Search is fast. 00:28:35.310 --> 00:28:37.960 How do I do search? 00:28:37.960 --> 00:28:41.220 If you're looking for 42 or for 16, 00:28:41.220 --> 00:28:43.440 you'd be like, oh, it's less than 23. 00:28:43.440 --> 00:28:45.640 I'll get on this path. 00:28:45.640 --> 00:28:48.880 PROFESSOR: So start at the root, compare my key to the root. 00:28:48.880 --> 00:28:50.089 If it's smaller, go left. 00:28:50.089 --> 00:28:51.130 If it's bigger, go right. 00:28:51.130 --> 00:28:53.600 Then keep doing that until I arrive somewhere 00:28:53.600 --> 00:28:57.590 or until I arrive at a dead end if I'm looking for 14. 00:28:57.590 --> 00:28:59.860 This is a lot like binary search. 00:28:59.860 --> 00:29:02.330 Binary search in an array, you look at the middle. 00:29:02.330 --> 00:29:03.759 If your key is smaller, go left. 00:29:03.759 --> 00:29:05.300 If your key is bigger, then go right. 00:29:09.680 --> 00:29:13.290 Let's look at the code for a little bit. 00:29:13.290 --> 00:29:16.490 Look at the BST Node Class, and you'll 00:29:16.490 --> 00:29:19.020 see that it has the fields that we have up here. 00:29:19.020 --> 00:29:21.370 And look at the Find method, and this is pretty much 00:29:21.370 --> 00:29:23.020 the binary search code. 00:29:23.020 --> 00:29:26.920 Lines 8 and 9 have the return condition when you're happy 00:29:26.920 --> 00:29:31.187 and you found the key, and then line 10 compares 00:29:31.187 --> 00:29:33.520 the key that you're looking for with the key in the node 00:29:33.520 --> 00:29:38.010 that you're at, and then lines 11, 14, 16, and 19 are pretty 00:29:38.010 --> 00:29:39.776 much copy pasted, except one of them 00:29:39.776 --> 00:29:41.400 deals with the left case, the other one 00:29:41.400 --> 00:29:44.910 deals with the right case. 00:29:44.910 --> 00:29:46.955 What is the running time for Find? 00:29:57.890 --> 00:30:00.780 AUDIENCE: Wouldn't it be log N, right? 00:30:00.780 --> 00:30:03.440 PROFESSOR: I wish. 00:30:03.440 --> 00:30:06.060 If this is all you have to do to get log N, 00:30:06.060 --> 00:30:08.360 then I would have to write a lot less code. 00:30:11.060 --> 00:30:14.540 So not quite log N. We will have to go through next lecture 00:30:14.540 --> 00:30:20.020 to get to log N. Until then, what's the running time? 00:30:20.020 --> 00:30:20.900 AUDIENCE: Order h. 00:30:20.900 --> 00:30:21.525 PROFESSOR: Yep. 00:30:24.030 --> 00:30:26.030 So you told me at the beginning it's unbalanced. 00:30:26.030 --> 00:30:27.135 AUDIENCE: Yeah. 00:30:27.135 --> 00:30:29.010 PROFESSOR: So then it's not going to be fast. 00:30:32.800 --> 00:30:34.440 OK, so order h. 00:30:34.440 --> 00:30:37.090 The reason why we care about h, and the reason 00:30:37.090 --> 00:30:39.980 we don't say order N, is because next 00:30:39.980 --> 00:30:42.415 lecture after we learn how to balance a tree, 00:30:42.415 --> 00:30:45.040 there's some magic that you can do to these binary search trees 00:30:45.040 --> 00:30:47.721 to guarantee that the height is order of log N. 00:30:47.721 --> 00:30:50.220 And then we'll go through all the running times that we have 00:30:50.220 --> 00:30:52.676 and replace h with log N. 00:30:52.676 --> 00:30:55.050 Now, it happens that in this case, if you would have told 00:30:55.050 --> 00:30:59.260 me order N, I couldn't argue with you because worst case, 00:30:59.260 --> 00:31:02.790 searches are order N. Can someone give me a binary search 00:31:02.790 --> 00:31:07.071 tree that exposes this degenerate case? 00:31:07.071 --> 00:31:07.570 Yes? 00:31:07.570 --> 00:31:09.195 AUDIENCE: If it's completely unbalanced 00:31:09.195 --> 00:31:13.150 and every node is greater than the parent nodes. 00:31:13.150 --> 00:31:17.435 PROFESSOR: So give me some inserts that create it. 00:31:17.435 --> 00:31:18.820 AUDIENCE: Insert 5. 00:31:18.820 --> 00:31:20.622 PROFESSOR: 5. 00:31:20.622 --> 00:31:22.390 AUDIENCE: Insert 10. 00:31:22.390 --> 00:31:23.222 PROFESSOR: 10. 00:31:23.222 --> 00:31:24.520 AUDIENCE: Insert 15. 00:31:24.520 --> 00:31:25.638 PROFESSOR: 15. 00:31:25.638 --> 00:31:27.042 AUDIENCE: Insert 20. 00:31:27.042 --> 00:31:27.980 PROFESSOR: Yep. 00:31:27.980 --> 00:31:28.940 And I could keep going. 00:31:28.940 --> 00:31:30.540 I could say, 1, 2, 3, 4, 5. 00:31:30.540 --> 00:31:31.740 I could say 5, 10, 15. 00:31:31.740 --> 00:31:34.000 As long as these keep growing, this is basically 00:31:34.000 --> 00:31:37.290 going to be a list, so searching is 00:31:37.290 --> 00:31:41.517 order N. This is a degenerate case. 00:31:41.517 --> 00:31:43.600 Turns out it doesn't happen too often in practice. 00:31:43.600 --> 00:31:47.340 If you have random data, the height will be roughly log N. 00:31:47.340 --> 00:31:49.530 But in order to avoid those degenerate cases, 00:31:49.530 --> 00:31:54.180 we'll be doing balanced trees later on. 00:31:54.180 --> 00:31:55.680 So we covered Find. 00:31:55.680 --> 00:31:56.854 We know it's order h. 00:31:56.854 --> 00:31:58.270 How do you insert, really quickly? 00:32:09.104 --> 00:32:10.520 AUDIENCE: Do you mean in searching 00:32:10.520 --> 00:32:13.000 when it's balanced or unbalanced? 00:32:13.000 --> 00:32:14.520 PROFESSOR: This guy. 00:32:14.520 --> 00:32:16.310 So the trees look exactly the same. 00:32:16.310 --> 00:32:18.420 If it's balanced, it's going to look more 00:32:18.420 --> 00:32:20.380 like that than like this. 00:32:20.380 --> 00:32:22.200 Actually, this is balanced. 00:32:22.200 --> 00:32:23.550 This is perfectly unbalanced. 00:32:23.550 --> 00:32:25.125 This is somewhere in the middle. 00:32:25.125 --> 00:32:27.500 If it's balanced, it's just going to look more like this, 00:32:27.500 --> 00:32:30.180 but it's still a binary search tree. 00:32:30.180 --> 00:32:33.481 How would you insert a node? 00:32:33.481 --> 00:32:33.980 Yes? 00:32:33.980 --> 00:32:35.944 AUDIENCE: Can't you start at the root 00:32:35.944 --> 00:32:38.399 and find your way down, and then the first open child 00:32:38.399 --> 00:32:43.686 that you see that's applicable to your element, state it then? 00:32:43.686 --> 00:32:44.310 PROFESSOR: Yep. 00:32:44.310 --> 00:32:49.406 So if I wanted to insert 14, which way do I go? 00:32:49.406 --> 00:32:51.280 AUDIENCE: So you'd look at 23, and you'd say, 00:32:51.280 --> 00:32:53.215 it's less than 23, go left. 00:32:53.215 --> 00:32:54.670 You'd look at 8. 00:32:54.670 --> 00:32:56.610 You'd say, it's greater than 8. 00:32:56.610 --> 00:32:57.565 You'd go right. 00:32:57.565 --> 00:32:58.065 Look at 16. 00:32:58.065 --> 00:33:00.005 You'd say it's less, so you go left. 00:33:00.005 --> 00:33:00.975 15, it's less. 00:33:00.975 --> 00:33:03.855 Then you have an open spot so you stick it there. 00:33:03.855 --> 00:33:04.730 PROFESSOR: Excellent. 00:33:04.730 --> 00:33:06.930 Thank you. 00:33:06.930 --> 00:33:07.770 Yes? 00:33:07.770 --> 00:33:08.936 AUDIENCE: I have a question. 00:33:08.936 --> 00:33:11.176 What if we want to insert 5? 00:33:11.176 --> 00:33:12.359 Then-- 00:33:12.359 --> 00:33:14.025 PROFESSOR: So if you want to insert who? 00:33:14.025 --> 00:33:14.915 AUDIENCE: 5. 00:33:14.915 --> 00:33:17.590 Or actually no, we can't. 00:33:17.590 --> 00:33:22.184 I'm thinking, is there any case in which need to move a node? 00:33:22.184 --> 00:33:23.600 PROFESSOR: How would you insert 5? 00:33:23.600 --> 00:33:24.100 Let's see. 00:33:24.100 --> 00:33:25.552 What would you do for 5? 00:33:25.552 --> 00:33:29.408 AUDIENCE: For 5, then we'd insert it to the right of 4, 00:33:29.408 --> 00:33:30.860 right? 00:33:30.860 --> 00:33:32.656 PROFESSOR: Smaller, smaller, greater, 5. 00:33:32.656 --> 00:33:33.155 Right? 00:33:36.410 --> 00:33:38.560 AUDIENCE: So there would be no case in which we'd 00:33:38.560 --> 00:33:41.360 need to swap nodes or something? 00:33:41.360 --> 00:33:42.084 PROFESSOR: No. 00:33:42.084 --> 00:33:43.000 You're thinking ahead. 00:33:43.000 --> 00:33:46.900 We'll talk about that a little later when we get to deleting. 00:33:46.900 --> 00:33:51.720 As long as you follow a path in the tree, the path that finding 00:33:51.720 --> 00:33:54.720 would get you to, as soon as you hit a dead end, 00:33:54.720 --> 00:33:56.046 that's where your node belongs. 00:33:56.046 --> 00:33:58.420 Because you know next time you're going to search for it, 00:33:58.420 --> 00:34:02.230 the search is going to follow that path and find the node. 00:34:02.230 --> 00:34:02.800 Yes? 00:34:02.800 --> 00:34:04.924 AUDIENCE: If you have values are the same, like two 00:34:04.924 --> 00:34:07.620 nodes at the same number, does it 00:34:07.620 --> 00:34:09.360 matter which side you put it on? 00:34:09.360 --> 00:34:10.991 PROFESSOR: You don't. 00:34:10.991 --> 00:34:11.880 AUDIENCE: Oh, I see. 00:34:11.880 --> 00:34:15.670 It's more like you would only have four 1's in the tree. 00:34:15.670 --> 00:34:16.440 PROFESSOR: Yes. 00:34:16.440 --> 00:34:18.805 So if you're trying to store keys and values, 00:34:18.805 --> 00:34:20.179 then what you'd have to do if you 00:34:20.179 --> 00:34:21.945 want to allow multiple values for the same key 00:34:21.945 --> 00:34:23.540 is you have a linked list going off 00:34:23.540 --> 00:34:26.770 of this, which node becomes an array of values 00:34:26.770 --> 00:34:28.305 aside from the key. 00:34:28.305 --> 00:34:28.929 Smart question. 00:34:28.929 --> 00:34:29.699 Thank you. 00:34:29.699 --> 00:34:33.800 That trips you up every time you do actual code, 00:34:33.800 --> 00:34:35.889 so that's the right question to ask yourself 00:34:35.889 --> 00:34:36.880 when you're implementing this. 00:34:36.880 --> 00:34:37.838 Will I have duplicates? 00:34:37.838 --> 00:34:39.810 How do I handle them? 00:34:39.810 --> 00:34:40.489 We don't. 00:34:40.489 --> 00:34:43.300 We take the easy way out. 00:34:43.300 --> 00:34:45.460 So if you look at Insert, on the next page, 00:34:45.460 --> 00:34:48.440 you will see that the code is pretty much the Find code 00:34:48.440 --> 00:34:53.869 copy pasted, except when Self Left is None 00:34:53.869 --> 00:34:55.820 or Self Right is None, instead of returning, 00:34:55.820 --> 00:34:56.850 it creates a new node. 00:35:02.296 --> 00:35:03.820 Does that make sense to people? 00:35:06.915 --> 00:35:07.415 All right. 00:35:10.720 --> 00:35:13.730 So Delete is going to be the hardest operation for today. 00:35:13.730 --> 00:35:15.760 Before we do that, let's do a warm up operation. 00:35:18.820 --> 00:35:29.350 Let's say I want to implement Find Next Larger, also called 00:35:29.350 --> 00:35:32.110 Successor in some implementations. 00:35:32.110 --> 00:35:33.930 So I have a node. 00:35:33.930 --> 00:35:41.130 Say I have node 8, and I want to find the next key 00:35:41.130 --> 00:35:43.710 in the tree that's strictly larger than 8 00:35:43.710 --> 00:35:46.190 but smaller than anything else. 00:35:46.190 --> 00:35:48.770 So if I would take these nodes and write them down in order, 00:35:48.770 --> 00:35:52.400 I want to find the element that would go right after it. 00:35:52.400 --> 00:35:54.026 How do I do that? 00:35:54.026 --> 00:35:54.920 Don't cheat. 00:35:54.920 --> 00:35:59.588 Don't look at the code, or make my life easier and do searches. 00:35:59.588 --> 00:36:01.079 AUDIENCE: Go down one to the right, 00:36:01.079 --> 00:36:03.297 and you try to get down left as far as you can. 00:36:03.297 --> 00:36:03.880 PROFESSOR: OK. 00:36:03.880 --> 00:36:04.870 Very good. 00:36:04.870 --> 00:36:11.310 So I have a node, and it has some subtree here, 00:36:11.310 --> 00:36:17.690 so I can go to the right here, I can go all the way left. 00:36:17.690 --> 00:36:19.370 We have an operation that does this, 00:36:19.370 --> 00:36:22.182 and it's called Min for a tree. 00:36:22.182 --> 00:36:24.390 In order to find the minimum in a binary search tree, 00:36:24.390 --> 00:36:25.870 you keep going left. 00:36:25.870 --> 00:36:28.870 For example, in this case, you get 4, which is good. 00:36:28.870 --> 00:36:31.970 So the way you would code this up is if you have Min, 00:36:31.970 --> 00:36:33.740 you go to the right if you can, and then 00:36:33.740 --> 00:36:36.390 you call Min on the subtree. 00:36:36.390 --> 00:36:41.340 And you can see that lines 3 and 4 do exactly that. 00:36:41.340 --> 00:36:43.180 Good guess. 00:36:43.180 --> 00:36:45.630 But you can line 1 says case one, 00:36:45.630 --> 00:36:50.040 so you have the right answer for one case. 00:36:50.040 --> 00:36:53.760 Now we have to handle more difficult cases. 00:36:53.760 --> 00:36:59.490 What if instead, I go down a bunch of nodes, 00:36:59.490 --> 00:37:04.780 and I want to find the successor for this guy, for example, 00:37:04.780 --> 00:37:06.420 and there's nothing here. 00:37:06.420 --> 00:37:07.050 What do I do? 00:37:12.090 --> 00:37:17.160 So if I want to find the successor for 8, what do I do? 00:37:17.160 --> 00:37:17.660 Sorry. 00:37:17.660 --> 00:37:18.368 It has an answer. 00:37:18.368 --> 00:37:24.247 What if I want to find the successor for 4? 00:37:27.229 --> 00:37:28.230 AUDIENCE: Go up one. 00:37:28.230 --> 00:37:28.813 PROFESSOR: OK. 00:37:28.813 --> 00:37:29.900 Go up one. 00:37:29.900 --> 00:37:31.522 Why does that work? 00:37:31.522 --> 00:37:33.370 AUDIENCE: You know it's going to be greater. 00:37:33.370 --> 00:37:37.780 PROFESSOR: So I'm going up right. 00:37:37.780 --> 00:37:41.000 So I know that everything here is guaranteed to be smaller, 00:37:41.000 --> 00:37:44.330 everything here is guaranteed to be greater than this guy. 00:37:44.330 --> 00:37:48.180 This guy is up right, so this is guaranteed to be greater 00:37:48.180 --> 00:37:50.040 than this, and everything here is 00:37:50.040 --> 00:37:52.550 guaranteed to be greater than this, and so on and so forth 00:37:52.550 --> 00:37:54.270 for the entire tree. 00:37:54.270 --> 00:37:58.150 So if I go up right, I'm happy. 00:37:58.150 --> 00:38:00.030 I definitely found my answer. 00:38:00.030 --> 00:38:06.040 Now, what if I have something that looks like this, 00:38:06.040 --> 00:38:08.160 and I want to find the successor for this guy? 00:38:13.040 --> 00:38:14.730 AUDIENCE: There is none. 00:38:14.730 --> 00:38:16.470 PROFESSOR: In this case, there is none 00:38:16.470 --> 00:38:18.210 if there's nothing else here. 00:38:18.210 --> 00:38:22.910 What if I have this, but then I have this? 00:38:22.910 --> 00:38:24.300 So I came down this way. 00:38:32.760 --> 00:38:35.765 AUDIENCE: Are you saying you're calling on that last node? 00:38:35.765 --> 00:38:36.390 PROFESSOR: Yep. 00:38:36.390 --> 00:38:37.473 AUDIENCE: Find the larger? 00:38:41.300 --> 00:38:44.299 I guess you'd just trace back up. 00:38:44.299 --> 00:38:45.590 PROFESSOR: And where do I stop? 00:38:51.410 --> 00:38:54.110 AUDIENCE: It affects the tree, so you go up one from there. 00:38:54.110 --> 00:38:56.320 You don't stop there. 00:38:56.320 --> 00:38:58.890 PROFESSOR: Why can't I stop here? 00:38:58.890 --> 00:39:03.911 AUDIENCE: Because you know that that-- not necessarily. 00:39:03.911 --> 00:39:06.160 AUDIENCE: You know that everything in that long branch 00:39:06.160 --> 00:39:09.429 right there is less than that node [INAUDIBLE]. 00:39:09.429 --> 00:39:11.220 PROFESSOR: This is to the left of this guy, 00:39:11.220 --> 00:39:16.620 so this guy has to be greater than everything here, 00:39:16.620 --> 00:39:20.640 and then you can repeat the argument that we had before. 00:39:20.640 --> 00:39:22.060 So here, we could stop right away 00:39:22.060 --> 00:39:23.700 because we could branch left. 00:39:23.700 --> 00:39:26.340 In this case, you have to go up until you're 00:39:26.340 --> 00:39:29.580 able to go left and up. 00:39:29.580 --> 00:39:34.310 If you get to the root, then what happened? 00:39:34.310 --> 00:39:37.025 Then we're in this case, and you have no successor. 00:39:40.730 --> 00:39:42.740 So take a look at the code. 00:39:42.740 --> 00:39:45.320 The next larger, lines 1 through 9. 00:39:45.320 --> 00:39:49.070 Case two, 6 through 8, does exactly that. 00:39:49.070 --> 00:39:52.910 If I can't go to my right and find the tree there, 00:39:52.910 --> 00:39:57.570 then I go up through my parent chain, 00:39:57.570 --> 00:40:00.942 and as long as I have to go up to the left, 00:40:00.942 --> 00:40:02.900 so as long as I'm the right child of my parent, 00:40:02.900 --> 00:40:04.120 I have to keep going. 00:40:04.120 --> 00:40:06.810 The moment I find the parent where I'm the left child, 00:40:06.810 --> 00:40:07.340 I stop. 00:40:07.340 --> 00:40:08.255 That's my successor. 00:40:11.330 --> 00:40:14.270 What if I would have to find the predecessor instead? 00:40:14.270 --> 00:40:16.810 So the element that's smaller than me 00:40:16.810 --> 00:40:19.734 but bigger than everything else in the tree. 00:40:19.734 --> 00:40:20.400 What would I do? 00:40:31.624 --> 00:40:33.640 AUDIENCE: It's just the opposite. 00:40:33.640 --> 00:40:35.422 PROFESSOR: Just the opposite. 00:40:35.422 --> 00:40:39.718 So how do I do the opposite? 00:40:39.718 --> 00:40:43.809 AUDIENCE: You can take the max of the left side tree, 00:40:43.809 --> 00:40:53.400 or traverse up, and if that's less than-- 00:40:53.400 --> 00:40:55.900 PROFESSOR: OK, so if I have a left subtree, fine. 00:40:55.900 --> 00:40:59.070 Call max on it and get the rightmost node there. 00:40:59.070 --> 00:41:04.250 If not, I go up, and when do I stop? 00:41:04.250 --> 00:41:10.598 When I go left or right? 00:41:10.598 --> 00:41:13.430 AUDIENCE: You'd have to go right. 00:41:13.430 --> 00:41:14.850 Is that right? 00:41:14.850 --> 00:41:15.630 PROFESSOR: Yep. 00:41:15.630 --> 00:41:20.300 So last time, in this case, when I was going up, 00:41:20.300 --> 00:41:22.160 if I was going left, I had to keep going, 00:41:22.160 --> 00:41:25.250 and the moment I went right, I was happy and I stopped. 00:41:25.250 --> 00:41:28.560 What if I want to find the predecessor? 00:41:28.560 --> 00:41:29.810 It's the opposite, right? 00:41:29.810 --> 00:41:34.770 So I will go this way, and the moment I can go this way, 00:41:34.770 --> 00:41:35.960 I'm done. 00:41:35.960 --> 00:41:37.624 How do you do this in code? 00:41:42.570 --> 00:41:43.765 Slightly tricky. 00:41:43.765 --> 00:41:44.940 Just slightly, I promise. 00:41:49.080 --> 00:41:51.616 AUDIENCE: [INAUDIBLE]. 00:41:51.616 --> 00:41:53.790 PROFESSOR: It's hard. 00:41:53.790 --> 00:41:56.630 What I would do is copy paste the code, 00:41:56.630 --> 00:41:59.725 replace "left" with "right" everywhere, and replace "min" 00:41:59.725 --> 00:42:03.251 with "max." 00:42:03.251 --> 00:42:05.240 You get it done. 00:42:05.240 --> 00:42:07.910 So we talked about how the tree is symmetric, right? 00:42:07.910 --> 00:42:12.670 So every time, instead of saying "left," you say "right," 00:42:12.670 --> 00:42:14.830 and instead of saying "min," you say "max." 00:42:14.830 --> 00:42:15.860 That's how you do this. 00:42:18.760 --> 00:42:19.905 How do we do deletions? 00:42:23.170 --> 00:42:25.910 So suppose I'm in this tree and I want to delete 15. 00:42:25.910 --> 00:42:28.710 What do I do? 00:42:28.710 --> 00:42:29.570 AUDIENCE: Kill it. 00:42:29.570 --> 00:42:30.510 PROFESSOR: Kill it. 00:42:30.510 --> 00:42:31.714 Very good. 00:42:31.714 --> 00:42:32.880 What if I want to delete 16? 00:42:32.880 --> 00:42:34.011 What do I do? 00:42:39.663 --> 00:42:43.797 AUDIENCE: You need to put 15 where 16 is. 00:42:43.797 --> 00:42:44.380 PROFESSOR: OK. 00:42:44.380 --> 00:42:46.715 So I would put 15 here. 00:42:51.770 --> 00:42:53.640 So I had 16. 00:42:56.520 --> 00:42:58.085 Suppose I have a big tree here. 00:43:03.087 --> 00:43:04.670 Actually, let's go for an easier case. 00:43:04.670 --> 00:43:09.920 Let's say I have this tree here. 00:43:09.920 --> 00:43:12.000 So you're here, you have a big tree here, 00:43:12.000 --> 00:43:13.070 you don't have anything here, and you 00:43:13.070 --> 00:43:14.070 want to delete this guy. 00:43:17.450 --> 00:43:19.165 AUDIENCE: You know that everything less 00:43:19.165 --> 00:43:21.370 than the top node is going to be less than it, 00:43:21.370 --> 00:43:22.830 so you can just move that up. 00:43:22.830 --> 00:43:27.769 PROFESSOR: Everything less than this guy is also 00:43:27.769 --> 00:43:29.060 going to be less than this guy. 00:43:29.060 --> 00:43:32.250 So you're saying move the whole tree up. 00:43:32.250 --> 00:43:32.992 AUDIENCE: Yep. 00:43:32.992 --> 00:43:34.450 PROFESSOR: So the way we do that is 00:43:34.450 --> 00:43:38.290 we'd take this node's left link and make it point here, 00:43:38.290 --> 00:43:42.690 and take this guy's parent link and make it point here, 00:43:42.690 --> 00:43:44.730 and this guy sort of goes away. 00:43:48.250 --> 00:43:50.090 So we have two cases for deleting. 00:43:50.090 --> 00:43:53.250 We have if you're a leaf, we'll take you out. 00:44:00.600 --> 00:44:01.100 Sorry. 00:44:01.100 --> 00:44:02.190 I got confused. 00:44:02.190 --> 00:44:04.350 If you have one child and that child 00:44:04.350 --> 00:44:07.860 is in the same direction as your parent, then you can do this. 00:44:07.860 --> 00:44:21.730 What if you have one child, but it's a zigzag like this? 00:44:21.730 --> 00:44:22.380 What do you do? 00:44:29.520 --> 00:44:32.020 AUDIENCE: It's still greater than, so you do the same thing. 00:44:32.020 --> 00:44:33.250 PROFESSOR: Exactly. 00:44:33.250 --> 00:44:33.800 Same thing. 00:44:37.200 --> 00:44:40.269 Just change this guy, change this guy, and I'm happy. 00:44:40.269 --> 00:44:42.810 So it doesn't matter if you have a zigzag or a straight line. 00:44:42.810 --> 00:44:45.290 It might help you think about it to convince yourself 00:44:45.290 --> 00:44:49.970 that the code is correct, but in the end, you do the same thing. 00:44:49.970 --> 00:44:54.840 Now, what if I want to delete node 8? 00:44:54.840 --> 00:45:10.370 So what if I have a nasty case where I want to delete this guy 00:45:10.370 --> 00:45:12.833 and it has children both on the left and on the right? 00:45:22.200 --> 00:45:25.030 AUDIENCE: You have to take 8, compare it to its parent 00:45:25.030 --> 00:45:26.605 and compare it to its right child, 00:45:26.605 --> 00:45:28.667 and see which one is greater in order 00:45:28.667 --> 00:45:33.857 to figure out which node gets replaced in its spot. 00:45:33.857 --> 00:45:34.440 PROFESSOR: OK. 00:45:34.440 --> 00:45:37.064 So there is replacing that's going to happen. 00:45:37.064 --> 00:45:38.230 The answer is really tricky. 00:45:38.230 --> 00:45:41.290 I always forget this when coding. 00:45:41.290 --> 00:45:44.130 Try to understand it, and if it doesn't work, 00:45:44.130 --> 00:45:45.300 refer to the textbook. 00:45:45.300 --> 00:45:48.270 When you forget it, because you will, refer to the textbook 00:45:48.270 --> 00:45:50.120 or to the internet. 00:45:50.120 --> 00:45:54.650 So what you do is I can't just magically replace this node 00:45:54.650 --> 00:45:57.650 with one of the subtrees, but we talked right 00:45:57.650 --> 00:46:07.030 before this about Next Greater, so finding a node's successor. 00:46:07.030 --> 00:46:10.560 If this node has both a left subtree and a right subtree, 00:46:10.560 --> 00:46:13.460 then I know that if I call Find Successor on it, 00:46:13.460 --> 00:46:17.600 I'm going to go somewhere inside here, 00:46:17.600 --> 00:46:23.000 and I'm going to find a node somewhere in here all the way 00:46:23.000 --> 00:46:25.510 to the left that is this guy's successor. 00:46:29.670 --> 00:46:32.340 So what I'm going to do is I'm going 00:46:32.340 --> 00:46:38.030 to delete this node instead, and then I'm going to take its key 00:46:38.030 --> 00:46:38.990 and put it up here. 00:46:42.880 --> 00:46:48.550 So if I want to delete 8, what I do is I find its successor, 00:46:48.550 --> 00:46:53.180 then I delete it, then I take the 15 that was here-- you 00:46:53.180 --> 00:46:53.930 can see it, right? 00:46:53.930 --> 00:46:54.638 It's still there. 00:46:57.290 --> 00:46:59.680 Put it here. 00:46:59.680 --> 00:47:05.470 So the reason this works is that everything here 00:47:05.470 --> 00:47:08.010 is greater than this guy. 00:47:08.010 --> 00:47:11.240 Everything here is smaller than this guy. 00:47:11.240 --> 00:47:14.480 This is the next node that's greater than this guy, 00:47:14.480 --> 00:47:16.760 but everything else is bigger than it, 00:47:16.760 --> 00:47:19.590 right, because we wanted it to be a successor. 00:47:19.590 --> 00:47:22.840 So if I take this value and I put it up here, 00:47:22.840 --> 00:47:25.500 everything in here is still going to be greater than it. 00:47:30.310 --> 00:47:33.000 This is a successor of this guy, so everything here 00:47:33.000 --> 00:47:35.000 is still going to be smaller than the successor. 00:47:44.310 --> 00:47:45.360 Great. 00:47:45.360 --> 00:47:48.620 In order to do a delete, I find the successor, 00:47:48.620 --> 00:47:50.455 and then I call Delete on it. 00:47:50.455 --> 00:47:51.830 How do I know that this will end? 00:47:51.830 --> 00:47:54.210 How do I know that I'm not going to go 00:47:54.210 --> 00:47:58.533 into a loop that runs forever? 00:47:58.533 --> 00:47:59.699 AUDIENCE: Because it's not-- 00:47:59.699 --> 00:48:01.947 AUDIENCE: It's acyclic, right? 00:48:01.947 --> 00:48:02.530 PROFESSOR: OK. 00:48:05.670 --> 00:48:07.540 First answer, good. 00:48:07.540 --> 00:48:11.290 Eventually, worst case, I'm going to get to the maximum, 00:48:11.290 --> 00:48:16.340 and then not going on have to delete the successor anymore. 00:48:16.340 --> 00:48:17.780 Now, another thing to note here is 00:48:17.780 --> 00:48:20.710 that if this guy is the successor of this guy, 00:48:20.710 --> 00:48:24.300 it can't have anything on the left, because if it would, 00:48:24.300 --> 00:48:27.800 then whatever is down here has to be bigger than this, 00:48:27.800 --> 00:48:29.520 and whatever's to the left of this node 00:48:29.520 --> 00:48:32.560 has to be smaller than this. 00:48:32.560 --> 00:48:34.740 But we said that this is the successor of this, 00:48:34.740 --> 00:48:36.500 so there's nothing here. 00:48:36.500 --> 00:48:40.000 So this will be one of the easy cases that we talked about. 00:48:40.000 --> 00:48:42.500 The successor either has no kids, 00:48:42.500 --> 00:48:47.940 or it has only one child, only one subtree. 00:48:47.940 --> 00:48:51.140 So then I can delete it using one of the easy cases. 00:48:51.140 --> 00:48:55.170 So in fact, worst case that happens in a delete is my node 00:48:55.170 --> 00:48:56.230 has two subtrees. 00:48:56.230 --> 00:48:59.460 Then I find the successor that's only going to have one subtree, 00:48:59.460 --> 00:49:01.130 I change my links there, and I'm done. 00:49:03.970 --> 00:49:05.660 What is the running time for Delete? 00:49:15.103 --> 00:49:17.186 AUDIENCE: Is it order h, because you should do it 00:49:17.186 --> 00:49:19.939 all the way down to the bottom of the tree, right? 00:49:19.939 --> 00:49:21.480 PROFESSOR: You have the right answer. 00:49:21.480 --> 00:49:22.684 Let's see why it's order h. 00:49:22.684 --> 00:49:23.850 It has to be order h, right? 00:49:23.850 --> 00:49:25.433 Otherwise, the tree would be too slow. 00:49:25.433 --> 00:49:29.130 If it's order N, then it's bad. 00:49:29.130 --> 00:49:32.490 So why would Delete be order h? 00:49:32.490 --> 00:49:35.350 This was a heap, right, so I can't use this. 00:49:35.350 --> 00:49:39.401 I'm going to write "delete" here again. 00:49:39.401 --> 00:49:41.900 So the first thing you do is you have to search for the key, 00:49:41.900 --> 00:49:42.630 right? 00:49:42.630 --> 00:49:43.610 That's order h. 00:49:46.320 --> 00:49:48.960 Now, if it's a happy case, if it's case one or two, 00:49:48.960 --> 00:49:50.750 you change some links and you're done. 00:49:50.750 --> 00:49:51.830 What's the time for that? 00:49:54.679 --> 00:49:55.470 AUDIENCE: Constant. 00:49:55.470 --> 00:49:56.810 PROFESSOR: Constant. 00:49:56.810 --> 00:49:59.950 So happy case, order h for sure. 00:49:59.950 --> 00:50:00.659 Now sad case. 00:50:00.659 --> 00:50:02.200 If you have two children, what do you 00:50:02.200 --> 00:50:06.580 have to do after you realize that you have two subtrees? 00:50:06.580 --> 00:50:08.047 AUDIENCE: Find the successor. 00:50:08.047 --> 00:50:08.630 PROFESSOR: OK. 00:50:08.630 --> 00:50:12.378 What's the running time for finding a successor? 00:50:12.378 --> 00:50:14.270 AUDIENCE: Order h. 00:50:14.270 --> 00:50:15.090 PROFESSOR: Order h. 00:50:19.310 --> 00:50:21.287 Once I find the successor, what do I do? 00:50:25.560 --> 00:50:29.287 Call Delete on that, and what happens? 00:50:29.287 --> 00:50:30.620 It's a happy case or a sad case? 00:50:30.620 --> 00:50:32.410 AUDIENCE: It's a happy case. 00:50:32.410 --> 00:50:35.076 PROFESSOR: Happy case, a few links get swapped, 00:50:35.076 --> 00:50:36.070 constant time. 00:50:36.070 --> 00:50:40.010 So worst case, order h plus order h. 00:50:40.010 --> 00:50:41.260 Order h. 00:50:41.260 --> 00:50:45.280 So insertions are order h, deletions are order h. 00:50:45.280 --> 00:50:46.410 AUDIENCE: The first one. 00:50:46.410 --> 00:50:47.950 Because the second one is from finding the successor. 00:50:47.950 --> 00:50:49.033 What is the first one for? 00:50:49.033 --> 00:50:52.930 PROFESSOR: Finding the node for a key in the tree. 00:50:52.930 --> 00:50:55.910 So if I say Delete 8, then you have to find 8. 00:50:55.910 --> 00:50:58.880 If I give you the node, then you don't have that. 00:50:58.880 --> 00:50:59.506 Good question. 00:50:59.506 --> 00:51:00.380 It's a good question. 00:51:00.380 --> 00:51:01.000 Thank you. 00:51:06.380 --> 00:51:07.500 So that's insertion. 00:51:07.500 --> 00:51:08.470 That's deletion. 00:51:11.169 --> 00:51:12.585 Let's look at the code for Delete. 00:51:17.430 --> 00:51:18.810 Looks kind of long. 00:51:22.320 --> 00:51:26.637 So lines through 21, happy case or sad case? 00:51:31.410 --> 00:51:33.150 Try to do it by looking at the "if" 00:51:33.150 --> 00:51:35.010 instead of looking at the comments. 00:51:38.130 --> 00:51:42.180 So lines through 21 for Delete. 00:51:45.484 --> 00:51:46.900 AUDIENCE: On this tree? 00:51:46.900 --> 00:51:49.258 Which tree, because there are two deletes? 00:51:49.258 --> 00:51:50.133 PROFESSOR: Oh really? 00:51:53.440 --> 00:51:53.940 Sorry. 00:51:53.940 --> 00:51:56.145 Why do we have two deletes? 00:51:56.145 --> 00:52:01.409 AUDIENCE: There's BST Delete and then there's BST Node Delete. 00:52:01.409 --> 00:52:02.450 PROFESSOR: So BST Delete. 00:52:06.000 --> 00:52:08.400 Finds the node, and then calls Delete on the node. 00:52:11.540 --> 00:52:13.460 And then if the node is a tree's root, 00:52:13.460 --> 00:52:16.480 then it updates the tree's root. 00:52:16.480 --> 00:52:19.410 So let's look at the nodes delete. 00:52:19.410 --> 00:52:20.295 Oh, I see. 00:52:20.295 --> 00:52:21.920 I think I was looking at the wrong one. 00:52:26.020 --> 00:52:26.580 Thank you. 00:52:26.580 --> 00:52:29.280 My Delete was much longer than yours. 00:52:29.280 --> 00:52:33.760 So lines 3 through 12, happy case or sad case? 00:52:40.180 --> 00:52:44.290 Look at the "if" on line 3 and tell me, 00:52:44.290 --> 00:52:45.650 what case is it going for? 00:52:51.208 --> 00:52:52.124 AUDIENCE: [INAUDIBLE]. 00:52:55.660 --> 00:52:57.580 PROFESSOR: If it doesn't have a left child 00:52:57.580 --> 00:52:59.280 or it doesn't have a right child, 00:52:59.280 --> 00:53:01.750 is that the happy case or the sad case? 00:53:01.750 --> 00:53:02.480 AUDIENCE: Happy. 00:53:02.480 --> 00:53:03.396 PROFESSOR: Happy case. 00:53:03.396 --> 00:53:08.300 So lines 4 through 12 handle the happy case. 00:53:08.300 --> 00:53:10.990 Lines 14 through 16 handle the sad case. 00:53:14.710 --> 00:53:17.860 Do lines 14 through 16 make sense? 00:53:17.860 --> 00:53:19.740 Find the successor, then swap the keys, 00:53:19.740 --> 00:53:21.148 then delete that successor. 00:53:26.820 --> 00:53:29.660 Now, lines 4 through 11 are pretty much what 00:53:29.660 --> 00:53:33.960 we talked about here, except I can't draw arrows on the board 00:53:33.960 --> 00:53:37.472 and instead I have to change left and right links. 00:53:37.472 --> 00:53:41.130 Line 4 has to see if we're a left child or a right child, 00:53:41.130 --> 00:53:45.240 and then lines 5 through 7 and 9 through 11 are pretty much 00:53:45.240 --> 00:53:48.600 copy paste, swap left with right. 00:53:48.600 --> 00:53:51.166 And they changed the links like we changed them here. 00:53:59.330 --> 00:54:03.150 Do we have any questions on Deletes? 00:54:03.150 --> 00:54:08.440 AUDIENCE: So if the successor had a right child, 00:54:08.440 --> 00:54:15.050 then all you do, you just do the workaround thing where 00:54:15.050 --> 00:54:15.550 you just-- 00:54:15.550 --> 00:54:17.530 PROFESSOR: Yep. 00:54:17.530 --> 00:54:19.990 So the case that it doesn't have two children. 00:54:19.990 --> 00:54:21.870 As long as it doesn't have both children, 00:54:21.870 --> 00:54:24.328 you're in the happy case and you can do some link swapping. 00:54:28.355 --> 00:54:29.870 Are you guys burned out already? 00:54:33.240 --> 00:54:33.910 Fair enough. 00:54:37.250 --> 00:54:38.540 I left a part out. 00:54:38.540 --> 00:54:43.120 What I left out is how to augment a binary tree. 00:54:43.120 --> 00:54:46.740 So binary trees by default can answer the question, 00:54:46.740 --> 00:54:49.500 what's the minimum node in a tree in order h. 00:54:49.500 --> 00:54:51.710 You go all the way to the left, you find the minimum. 00:54:51.710 --> 00:54:53.260 That's the minimum. 00:54:53.260 --> 00:54:56.920 It turns out that if you make a node a little bit fatter, 00:54:56.920 --> 00:55:03.070 so if instead of storing, say, 23 in this node, I store 23, 00:55:03.070 --> 00:55:05.630 and I store the fact that the minimum in my left subtree 00:55:05.630 --> 00:55:10.230 is 4, then it turns out that I can answer the question 00:55:10.230 --> 00:55:13.066 in constant time, what's the minimum? 00:55:13.066 --> 00:55:15.475 Oh gee, if you store the minimum here, 00:55:15.475 --> 00:55:18.550 of course you can retrieve it in constant time, right? 00:55:18.550 --> 00:55:21.940 The hard part is, how do you handle insertions and updates 00:55:21.940 --> 00:55:25.580 in the same time? 00:55:25.580 --> 00:55:27.770 So the idea is that if I have a node 00:55:27.770 --> 00:55:33.990 and I have a function here, say the minimum of everything, 00:55:33.990 --> 00:55:40.530 if I have two children, here they're 15 and 42, 00:55:40.530 --> 00:55:42.800 and say the minimum in this tree is 4 00:55:42.800 --> 00:55:45.850 and the minimum in this tree is. 00:55:45.850 --> 00:55:50.076 So if I already computed the function for these guys, 00:55:50.076 --> 00:55:51.700 how do I compute the function for this? 00:55:54.039 --> 00:55:55.580 AUDIENCE: [INAUDIBLE] and compare it? 00:55:55.580 --> 00:55:56.220 PROFESSOR: Yep. 00:55:56.220 --> 00:55:59.190 Take the minimum of these two guys, right? 00:55:59.190 --> 00:56:02.290 There are some special cases if you don't have a child. 00:56:02.290 --> 00:56:06.540 If you don't have a left child, then you're the minimum. 00:56:06.540 --> 00:56:08.440 But you write down those special cases, 00:56:08.440 --> 00:56:10.930 and you can compute this in how much time? 00:56:14.650 --> 00:56:17.020 AUDIENCE: Order h, right? 00:56:17.020 --> 00:56:19.250 PROFESSOR: What if I already computed the answer 00:56:19.250 --> 00:56:20.759 for the children? 00:56:20.759 --> 00:56:22.300 How much time does it take to compute 00:56:22.300 --> 00:56:24.432 the answer for a single node? 00:56:24.432 --> 00:56:25.304 AUDIENCE: Constant. 00:56:25.304 --> 00:56:26.137 PROFESSOR: Constant. 00:56:26.137 --> 00:56:26.720 OK. 00:56:26.720 --> 00:56:28.271 AUDIENCE: For a tree, though. 00:56:28.271 --> 00:56:29.770 PROFESSOR: For a tree, it's order h. 00:56:29.770 --> 00:56:30.270 Yeah. 00:56:30.270 --> 00:56:32.791 You're getting ahead. 00:56:32.791 --> 00:56:33.540 You're rushing me. 00:56:33.540 --> 00:56:34.748 You're not letting me finish. 00:56:34.748 --> 00:56:38.200 AUDIENCE: Are you saying that we store the minimum value? 00:56:38.200 --> 00:56:39.345 PROFESSOR: So for every-- 00:56:39.345 --> 00:56:43.425 AUDIENCE: Each node has a field that 00:56:43.425 --> 00:56:46.316 says what the minimum value is in that tree. 00:56:46.316 --> 00:56:47.550 PROFESSOR: Yep, exactly. 00:56:47.550 --> 00:56:51.340 So for each node, what's the minimum in the subtree. 00:56:51.340 --> 00:56:54.740 So if I add a node here, suppose I add three 00:56:54.740 --> 00:57:01.840 and I had my minimums, what changed? 00:57:01.840 --> 00:57:03.870 This subtree changed, this subtree changed, 00:57:03.870 --> 00:57:07.670 this subtree changed, and then this subtree changed. 00:57:07.670 --> 00:57:12.150 So I have to update the minimums here, here, here, here. 00:57:12.150 --> 00:57:13.910 Nothing else changed. 00:57:13.910 --> 00:57:16.190 Outside the path where I did the Insert, 00:57:16.190 --> 00:57:19.680 nothing changed, so I don't have to update anything. 00:57:19.680 --> 00:57:22.790 So what I do is after the Insert, I go back up 00:57:22.790 --> 00:57:25.305 and I re-compute the values. 00:57:25.305 --> 00:57:26.180 So here, I'll have 3. 00:57:26.180 --> 00:57:29.774 I go back up 3, 3, 3. 00:57:29.774 --> 00:57:32.980 AUDIENCE: You could when you're passing down, though. 00:57:32.980 --> 00:57:35.818 When you're going down that column, 00:57:35.818 --> 00:57:37.600 you can just compare it on the way down. 00:57:37.600 --> 00:57:39.100 You don't have to go back up, right? 00:57:39.100 --> 00:57:40.040 PROFESSOR: Yep. 00:57:40.040 --> 00:57:42.480 So the advantage of doing it the way I'm saying it 00:57:42.480 --> 00:57:46.400 is that you can have other functions instead of minimum. 00:57:46.400 --> 00:57:48.640 As long as you can compute the function 00:57:48.640 --> 00:57:51.480 inside the parent in constant time 00:57:51.480 --> 00:57:53.850 using the function from the children, 00:57:53.850 --> 00:57:57.379 it makes sense to compute the function on the children first. 00:57:57.379 --> 00:57:59.420 There's an obvious function that I can't tell you 00:57:59.420 --> 00:58:02.760 because that's on the Pset, but when you see the next Pset, 00:58:02.760 --> 00:58:05.810 you'll see what I mean. 00:58:05.810 --> 00:58:08.565 So if you have a function where you 00:58:08.565 --> 00:58:11.190 know the result for the children and you can compute the result 00:58:11.190 --> 00:58:15.820 for the parent in constant time, then after you do the Insert, 00:58:15.820 --> 00:58:21.550 you go up on the path and you re-compute the function. 00:58:21.550 --> 00:58:25.478 When you delete, what do you do? 00:58:25.478 --> 00:58:26.874 AUDIENCE: Same thing. 00:58:26.874 --> 00:58:27.790 PROFESSOR: Same thing. 00:58:30.640 --> 00:58:35.850 If this goes away, then this subtree changed, and then 00:58:35.850 --> 00:58:37.530 if there would be something else here, 00:58:37.530 --> 00:58:41.240 then this subtree changed, but nothing else changed. 00:58:41.240 --> 00:58:43.880 So whenever you do an Insert or a Delete, all you have to do 00:58:43.880 --> 00:58:46.210 is go back up the path to the parent 00:58:46.210 --> 00:58:52.000 and re-compute the function that you're trying to compute. 00:58:52.000 --> 00:58:55.310 And that's tree augmentation. 00:58:55.310 --> 00:58:58.370 Does this make sense somewhat? 00:58:58.370 --> 00:58:59.440 That's it. 00:58:59.440 --> 00:59:00.960 So what you'll find in lecture notes 00:59:00.960 --> 00:59:05.850 is a harder way of doing it that works for minimum, 00:59:05.850 --> 00:59:08.990 but what I told you works for everything. 00:59:08.990 --> 00:59:12.790 So don't tell people I told you how to do this for everything. 00:59:12.790 --> 00:59:15.250 Sure nobody's going to know.