WEBVTT 00:00:07.000 --> 00:00:12.000 -- valuable experience. OK, today we're going to start 00:00:12.000 --> 00:00:18.000 talking about a particular class of algorithms called greedy 00:00:18.000 --> 00:00:22.000 algorithms. But we're going to do it in the 00:00:22.000 --> 00:00:27.000 context of graphs. So, I want to review a little 00:00:27.000 --> 00:00:32.000 bit about graphs, which mostly you can find in 00:00:32.000 --> 00:00:39.000 the textbook in appendix B. And so, if you haven't reviewed 00:00:39.000 --> 00:00:45.000 in appendix B recently, please sit down and review 00:00:45.000 --> 00:00:50.000 appendix B. It will pay off especially 00:00:50.000 --> 00:00:55.000 during our take-home quiz. So, just reminder, 00:00:55.000 --> 00:01:01.000 a digraph, what's a digraph? What's that short for? 00:01:01.000 --> 00:01:04.000 Directed graph, OK? 00:01:04.000 --> 00:01:07.000 Directed graph, G equals (V,E), 00:01:07.000 --> 00:01:13.000 OK, has a set, V, of vertices. 00:01:13.000 --> 00:01:17.000 And, I always get people telling me that I have one 00:01:17.000 --> 00:01:20.000 vertice. The singular is not vertice; 00:01:20.000 --> 00:01:21.000 it is vertex, OK? 00:01:21.000 --> 00:01:25.000 The plural is vertices. The singular is vertex. 00:01:25.000 --> 00:01:29.000 It's one of those weird English words. 00:01:29.000 --> 00:01:33.000 It's probably originally like French or something, 00:01:33.000 --> 00:01:37.000 right? I don't know. 00:01:37.000 --> 00:01:47.000 OK, anyway, and we have a set, E, which is a subset of V cross 00:01:47.000 --> 00:01:52.000 V of edges. So that's a digraph. 00:01:52.000 --> 00:02:02.000 And undirected graph, E contains unordered pairs. 00:02:12.000 --> 00:02:15.000 OK, and, sorry? It's Latin, OK, 00:02:15.000 --> 00:02:21.000 so it's probably pretty old, then, in English. 00:02:21.000 --> 00:02:28.000 I guess the vertex would be a little bit of a giveaway that 00:02:28.000 --> 00:02:34.000 maybe it wasn't French. It started to be used in 1570, 00:02:34.000 --> 00:02:39.000 OK. OK, good, OK, 00:02:39.000 --> 00:02:52.000 so the number of edges is, whether it's directed or 00:02:52.000 --> 00:03:02.000 undirected, is O of what? V^2, good. 00:03:02.000 --> 00:03:05.000 OK, and one of the conventions that will have when we're 00:03:05.000 --> 00:03:09.000 dealing, once we get into graphs, we deal a lot with sets. 00:03:09.000 --> 00:03:13.000 We generally drop the vertical bar notation within O's just 00:03:13.000 --> 00:03:16.000 because it's applied. It just makes it messier. 00:03:16.000 --> 00:03:18.000 So, once again, another abuse of notation. 00:03:18.000 --> 00:03:22.000 It really should be order the size of V^2, but it just messes 00:03:22.000 --> 00:03:25.000 up, I mean, it's just more stuff to write down. 00:03:25.000 --> 00:03:29.000 And, you're multiplying these things, and all those vertical 00:03:29.000 --> 00:03:33.000 bars. Since they don't even have a 00:03:33.000 --> 00:03:36.000 sense to the vertical bar, it gets messy. 00:03:36.000 --> 00:03:40.000 So, we just drop the vertical bars there when it's in 00:03:40.000 --> 00:03:44.000 asymptotic notation. So, E is order V^2 when it's a 00:03:44.000 --> 00:03:48.000 set of pairs, because if it's a set of pairs, 00:03:48.000 --> 00:03:52.000 it's at most n choose two, which is where it's at most n^2 00:03:52.000 --> 00:03:56.000 over 2, here it could be, at most, sorry, 00:03:56.000 --> 00:04:00.000 V^2 over 2, here it's at most V^2. 00:04:00.000 --> 00:04:07.000 And then, another property that sometimes comes up is if the G 00:04:07.000 --> 00:04:11.000 is connected, we have another bound, 00:04:11.000 --> 00:04:18.000 implies that the size of E is at least the size of V minus 00:04:18.000 --> 00:04:22.000 one. OK, so if it's connected, 00:04:22.000 --> 00:04:31.000 meaning, what does it mean to have a graph that's connected? 00:04:31.000 --> 00:04:37.000 Yeah, there's a path from any vertex to any other vertex in 00:04:37.000 --> 00:04:40.000 the graph. That's what it means to be 00:04:40.000 --> 00:04:44.000 connected. So if that's the case, 00:04:44.000 --> 00:04:50.000 that a number of edges is at least the number of vertices 00:04:50.000 --> 00:04:53.000 minus one, OK? And so, what that says, 00:04:53.000 --> 00:05:00.000 so one of the things we'll get into, a fact that I just wanted 00:05:00.000 --> 00:05:04.000 to remind you, is that in that case, 00:05:04.000 --> 00:05:09.000 if I look at log E, OK, log of the number of edges, 00:05:09.000 --> 00:05:14.000 that is O of log V. And by this, 00:05:14.000 --> 00:05:18.000 is omega of log V. So, it's equal to theta of log 00:05:18.000 --> 00:05:21.000 V. OK, so basically the number of, 00:05:21.000 --> 00:05:25.000 in the case of a connected graph, the number of edges, 00:05:25.000 --> 00:05:30.000 and the number of vertices are polynomially related. 00:05:30.000 --> 00:05:36.000 So, their logs are comparable. OK, so that's helpful just to 00:05:36.000 --> 00:05:41.000 know because sometimes I just get questions later on where 00:05:41.000 --> 00:05:45.000 people will say, oh, you showed it was log E but 00:05:45.000 --> 00:05:51.000 you didn't show it was log V. And I could point out that it's 00:05:51.000 --> 00:05:55.000 the same thing. OK, so there's various ways of 00:05:55.000 --> 00:05:59.000 representing graphs in computers, and I'm just going to 00:05:59.000 --> 00:06:04.000 cover a couple of the important ones. 00:06:04.000 --> 00:06:11.000 There's actually more. We'll see some more. 00:06:11.000 --> 00:06:20.000 So, the simplest one is what's called an adjacency matrix. 00:06:20.000 --> 00:06:30.000 An adjacency matrix of the graph, G, equals (V,E), 00:06:30.000 --> 00:06:40.000 where, for simplicity, I'll let V be the set of 00:06:40.000 --> 00:06:52.000 integers from one up to n, OK, is the n by n matrix A 00:06:52.000 --> 00:07:04.000 given by the ij-th at the entry is simply one if the edge, 00:07:04.000 --> 00:07:20.000 ij, is in the edge set and zero if ij is not in the edge set. 00:07:20.000 --> 00:07:22.000 OK, so it's simply the matrix where you say, 00:07:22.000 --> 00:07:25.000 the ij entry is one if it's in the matrix. 00:07:25.000 --> 00:07:27.000 So, this is, in some sense, 00:07:27.000 --> 00:07:32.000 giving you the predicate for, is there an edge from i to j? 00:07:32.000 --> 00:07:35.000 OK, remember, predicate is Boolean formula 00:07:35.000 --> 00:07:39.000 that is either zero or one, and in this case, 00:07:39.000 --> 00:07:45.000 you're saying it's one if there is an edge from i to j and zero 00:07:45.000 --> 00:07:48.000 otherwise. OK, sometimes you have edge 00:07:48.000 --> 00:07:52.000 weighted graphs, and then sometimes what people 00:07:52.000 --> 00:07:56.000 will do is replace this by edge weights. 00:07:56.000 --> 00:08:02.000 OK, it will be the weight of the edge from i to j. 00:08:02.000 --> 00:08:12.000 So, let's just do an example of that just to make sure that our 00:08:12.000 --> 00:08:22.000 intuition corresponds to our mathematical definitions. 00:08:22.000 --> 00:08:33.000 So, here's an example graph. Let's say that's our graph. 00:08:33.000 --> 00:08:37.000 So let's just draw the adjacency the matrix. 00:08:37.000 --> 00:08:42.000 OK, so what this says: is there's an edge from one to 00:08:42.000 --> 00:08:44.000 one? And the answer is no. 00:08:44.000 --> 00:08:47.000 Is there an edge from one to two? 00:08:47.000 --> 00:08:50.000 Yes. Is there an edge from one to 00:08:50.000 --> 00:08:51.000 three here? Yep. 00:08:51.000 --> 00:08:54.000 Is there an edge for one to four? 00:08:54.000 --> 00:08:58.000 No. Is there an edge from two until 00:08:58.000 --> 00:09:00.000 one? No. 00:09:00.000 --> 00:09:02.000 Two to two? No. 00:09:02.000 --> 00:09:04.000 Two to three? Yes. 00:09:04.000 --> 00:09:06.000 Two to four? No. 00:09:06.000 --> 00:09:13.000 No edges going out of three. Edge from four to three, 00:09:13.000 --> 00:09:19.000 and that's it. That's the adjacency matrix for 00:09:19.000 --> 00:09:23.000 this particular graph, OK? 00:09:23.000 --> 00:09:33.000 And so, I can represent a graph as this adjacency matrix. 00:09:33.000 --> 00:09:43.000 OK, when I represent it in this way, how much storage do I need? 00:09:43.000 --> 00:09:52.000 OK, n^2 or V^2 because the size is the same thing for V^2 00:09:52.000 --> 00:10:03.000 storage, OK, and that's what we call a dense representation. 00:10:03.000 --> 00:10:06.000 OK, it works well when the graph is dense. 00:10:06.000 --> 00:10:11.000 So, the graph is dense if the number of edges is close to all 00:10:11.000 --> 00:10:14.000 of the edges possible. OK, then this is a good 00:10:14.000 --> 00:10:18.000 representation. But for many types of graphs, 00:10:18.000 --> 00:10:23.000 the number of edges is much less than the possible number of 00:10:23.000 --> 00:10:26.000 edges, in which case we say the graph is sparse. 00:10:26.000 --> 00:10:32.000 Can somebody give me an example of a sparse graph? 00:10:32.000 --> 00:10:35.000 A class of graphs: so, I want a class of graphs 00:10:35.000 --> 00:10:38.000 that as n grows, the number of edges in the 00:10:38.000 --> 00:10:42.000 graph doesn't grow as the square, but grows rather as 00:10:42.000 --> 00:10:45.000 something much smaller. A linked list, 00:10:45.000 --> 00:10:48.000 so, a chain, OK, if you look at it from a 00:10:48.000 --> 00:10:52.000 graph theoretically, is a perfectly good example: 00:10:52.000 --> 00:10:56.000 only n edges in the chain for a chain of length n. 00:10:56.000 --> 00:10:59.000 So therefore, the number of edges would be 00:10:59.000 --> 00:11:02.000 order V. And in particular, 00:11:02.000 --> 00:11:07.000 you'd only have one edge per row here. 00:11:07.000 --> 00:11:10.000 What other graphs are sparse? Yeah? 00:11:10.000 --> 00:11:16.000 Good, a planar graph, a graph that can be drawn in a 00:11:16.000 --> 00:11:21.000 plane turns out that if it has V vertices has, 00:11:21.000 --> 00:11:25.000 and V is at least three, then it has, 00:11:25.000 --> 00:11:30.000 at most, three V minus six edges. 00:11:30.000 --> 00:11:34.000 So, it turns out that's order V edges again. 00:11:34.000 --> 00:11:38.000 What's another example of a common graph? 00:11:38.000 --> 00:11:42.000 Yeah, binary tree, or even actually any tree, 00:11:42.000 --> 00:11:49.000 you know, what's called a free tree if you read the appendix, 00:11:49.000 --> 00:11:54.000 OK, a tree that just is a connected graph that has no 00:11:54.000 --> 00:12:00.000 cycles, OK, is another example. What's an example of a graph 00:12:00.000 --> 00:12:04.000 that's dense? A complete graph, 00:12:04.000 --> 00:12:07.000 OK: it's all ones, OK, or if you have edge 00:12:07.000 --> 00:12:11.000 weights, it would be a completely filled in matrix. 00:12:11.000 --> 00:12:14.000 OK, good. So, this is good for dense 00:12:14.000 --> 00:12:17.000 representation. But sometimes you want to have 00:12:17.000 --> 00:12:22.000 a sparse representation so we don't have to spend V^2 space to 00:12:22.000 --> 00:12:26.000 deal with all of the, where most of it's going to be 00:12:26.000 --> 00:12:28.000 zeroes. OK, it's sort of like, 00:12:28.000 --> 00:12:33.000 if we know why it's zero, why bother representing it as 00:12:33.000 --> 00:12:40.000 zero? So, one such representation is 00:12:40.000 --> 00:12:46.000 an adjacency list representation. 00:12:46.000 --> 00:12:56.000 Actually, adjacency list of a given vertex is the list, 00:12:56.000 --> 00:13:08.000 which we denote by Adj of V, of vertices adjacent to V. 00:13:08.000 --> 00:13:12.000 OK, just in terms by their terminology, vertices are 00:13:12.000 --> 00:13:17.000 adjacent, but edges are incident on vertices. 00:13:17.000 --> 00:13:22.000 OK, so the incidence is a relation between a vertex and an 00:13:22.000 --> 00:13:25.000 edge. An adjacency is a relation 00:13:25.000 --> 00:13:31.000 between two vertices. OK, that's just the language. 00:13:31.000 --> 00:13:36.000 Why they use to different terms, I don't know, 00:13:36.000 --> 00:13:40.000 but that's what they do. So, in the graph, 00:13:40.000 --> 00:13:45.000 for example, the adjacency list for vertex 00:13:45.000 --> 00:13:52.000 one is just the list or the set of two three because one has 00:13:52.000 --> 00:13:57.000 going out of one are edges to two and three. 00:13:57.000 --> 00:14:02.000 The adjacency list for two is just three, four, 00:14:02.000 --> 00:14:07.000 three. It's the empty set, 00:14:07.000 --> 00:14:10.000 and for four, it is three. 00:14:10.000 --> 00:14:13.000 OK, so that's the representation. 00:14:13.000 --> 00:14:21.000 Now, if we want to figure out how much storage is required for 00:14:21.000 --> 00:14:26.000 this representation, OK, we need to understand how 00:14:26.000 --> 00:14:35.000 long the adjacency list is. So, what is the length of an 00:14:35.000 --> 00:14:40.000 adjacency list of a vertex, V? 00:14:40.000 --> 00:14:48.000 What name do we give to that? It's the degree. 00:14:48.000 --> 00:14:57.000 So, in an undirected graph, we call it the degree of the 00:14:57.000 --> 00:15:02.000 vertex. This is undirected. 00:15:02.000 --> 00:15:07.000 OK, about here, OK. 00:15:07.000 --> 00:15:12.000 So that's an undirected case. In the directed case, 00:15:12.000 --> 00:15:18.000 OK, actually I guess the way we should do this is say this. 00:15:18.000 --> 00:15:22.000 If the degree, we call it the out degree for a 00:15:22.000 --> 00:15:25.000 digraph. OK, so in a digraph, 00:15:25.000 --> 00:15:32.000 we have an out degree and an in degree for each vertex. 00:15:32.000 --> 00:15:36.000 So here, the in degree is three. 00:15:36.000 --> 00:15:41.000 Here, the out degree is two, OK? 00:15:41.000 --> 00:15:50.000 So, one of the important lemma that comes up is what's called 00:15:50.000 --> 00:15:56.000 the handshaking lemma. OK, it's one of these 00:15:56.000 --> 00:16:01.000 mathematical lemmas. 00:16:11.000 --> 00:16:16.000 And so, it comes from a story. Go to a dinner party, 00:16:16.000 --> 00:16:21.000 and everybody at the dinner party shakes other people's 00:16:21.000 --> 00:16:24.000 hands. Some people may not shake 00:16:24.000 --> 00:16:29.000 anybody's hand. Some people may shake several 00:16:29.000 --> 00:16:32.000 people's hands. Nobody shakes hands with 00:16:32.000 --> 00:16:37.000 themselves. And at some point during the 00:16:37.000 --> 00:16:41.000 dinner party, the host goes around and counts 00:16:41.000 --> 00:16:45.000 up how many, the sum, of the number of hands that 00:16:45.000 --> 00:16:48.000 each person has shaken. OK, so he says, 00:16:48.000 --> 00:16:52.000 how many did you shake? How many did you shake? 00:16:52.000 --> 00:16:55.000 How many did you shake? He adds them up, 00:16:55.000 --> 00:17:00.000 OK, and that number is guaranteed to be even. 00:17:00.000 --> 00:17:03.000 OK, that's the handshaking lemma. 00:17:03.000 --> 00:17:08.000 Or, stated a little bit more precisely, if I take for any 00:17:08.000 --> 00:17:13.000 graph the degree of the vertex, and sum them all up, 00:17:13.000 --> 00:17:20.000 that's how many hands everybody shook, OK, that's actually equal 00:17:20.000 --> 00:17:23.000 to always twice the number of edges. 00:17:23.000 --> 00:17:26.000 So, why is that going to be true? 00:17:26.000 --> 00:17:33.000 Why is that going to be twice the number of edges? 00:17:33.000 --> 00:17:33.000 Yeah? Yeah. Every time you put in an edge, you add one to the degree of 00:17:39.000 --> 00:17:44.000 each person on each end. So, it's just two different 00:17:44.000 --> 00:17:48.000 ways of counting up the same number of edges. 00:17:48.000 --> 00:17:52.000 OK, I can go around, and if you imagine that, 00:17:52.000 --> 00:17:56.000 that every time I count the degree of the node, 00:17:56.000 --> 00:18:01.000 I put a mark on every edge. Then, when I'm done, 00:18:01.000 --> 00:18:07.000 every edge has two marks on it, one for each end. 00:18:07.000 --> 00:18:17.000 OK: a pretty simple theorem. So, what that says is that for 00:18:17.000 --> 00:18:26.000 undirected graphs, that implies that the adjacency 00:18:26.000 --> 00:18:35.000 list representation, uses how much storage? 00:18:35.000 --> 00:18:39.000 OK, at most, 2E, so order E because that's 00:18:39.000 --> 00:18:42.000 not all. Yeah, so you have to have the 00:18:42.000 --> 00:18:47.000 number of vertices plus order the number of edges, 00:18:47.000 --> 00:18:53.000 OK, whether it's directed or undirected because I may have a 00:18:53.000 --> 00:18:59.000 graph, say it has a whole bunch of vertices and no edges, 00:18:59.000 --> 00:19:05.000 that's still going to cost me order V, OK? 00:19:05.000 --> 00:19:09.000 So, it uses theta of V plus E storage. 00:19:09.000 --> 00:19:15.000 And, it's basically the same thing asymptotically. 00:19:15.000 --> 00:19:22.000 In fact, it's easier to see in some sense for digraphs because 00:19:22.000 --> 00:19:27.000 for digraphs, what I do is I just add up the 00:19:27.000 --> 00:19:33.000 out degrees, and that equal to E, OK, if I add up the out 00:19:33.000 --> 00:19:39.000 degrees as equally. In fact, this is kind of like 00:19:39.000 --> 00:19:41.000 it amortized analysis, if you will, 00:19:41.000 --> 00:19:45.000 a book keeping analysis, that if I'm adding up the total 00:19:45.000 --> 00:19:48.000 number of edges, one way of doing it is 00:19:48.000 --> 00:19:50.000 accounting for a vertex by vertex. 00:19:50.000 --> 00:19:54.000 OK, so for each vertex, I basically can take each 00:19:54.000 --> 00:19:58.000 degree, and basically each vertex, look at the degree, 00:19:58.000 --> 00:20:02.000 and that allocating of account per edge, and then ending up 00:20:02.000 --> 00:20:06.000 with twice the number of edges, that's exactly accounting type 00:20:06.000 --> 00:20:12.000 of analysis that we might do for amortized analysis. 00:20:12.000 --> 00:20:17.000 OK, so we'll see that. So, this is a sparse 00:20:17.000 --> 00:20:22.000 representation, and it's often better than an 00:20:22.000 --> 00:20:25.000 adjacency matrix. For example, 00:20:25.000 --> 00:20:32.000 you can imagine if the World Wide Web were done with an 00:20:32.000 --> 00:20:37.000 adjacency matrix as opposed to, essentially, 00:20:37.000 --> 00:20:44.000 with an adjacency list type of representation. 00:20:44.000 --> 00:20:47.000 Every link on the World Wide Web, I had to say, 00:20:47.000 --> 00:20:50.000 here are the ones that I'm connected to, 00:20:50.000 --> 00:20:53.000 and here are all the ones I'm not connected to. 00:20:53.000 --> 00:20:57.000 OK, that list of things you're not connected to for a given 00:20:57.000 --> 00:20:59.000 page would be pretty dramatically, 00:20:59.000 --> 00:21:03.000 show you that there is an advantage to sparse 00:21:03.000 --> 00:21:06.000 representation. On the other hand, 00:21:06.000 --> 00:21:13.000 one of the nice things about an adjacency matrix representation 00:21:13.000 --> 00:21:19.000 is that each edge can be represented with a single bit, 00:21:19.000 --> 00:21:24.000 whereas typical when I'm representing things with an 00:21:24.000 --> 00:21:30.000 adjacency list representation, how many bits am I going to 00:21:30.000 --> 00:21:35.000 need to represent each adjacency? 00:21:35.000 --> 00:21:39.000 You'll need order log of V to be able to name each different 00:21:39.000 --> 00:21:41.000 vertex. OK, the log of the number is 00:21:41.000 --> 00:21:46.000 the number of bits that I need. So, there are places where this 00:21:46.000 --> 00:21:50.000 is actually a far more efficient representation. 00:21:50.000 --> 00:21:53.000 In particular, if you have a very dense graph, 00:21:53.000 --> 00:21:56.000 OK, this may be a better way of representing it. 00:21:56.000 --> 00:22:01.000 OK, the other thing I want you to get, and we're going to see 00:22:01.000 --> 00:22:05.000 more of this in particular next week, is that a matrix and a 00:22:05.000 --> 00:22:11.000 graph, there are two ways of looking at the same thing. 00:22:11.000 --> 00:22:14.000 OK, and in fact, there's a lot of graph theory 00:22:14.000 --> 00:22:17.000 that when you do things like multiply the adjacency matrix, 00:22:17.000 --> 00:22:20.000 OK, and so forth. So, there's a lot of 00:22:20.000 --> 00:22:24.000 commonality between graphs and matrices, a lot of mathematics 00:22:24.000 --> 00:22:28.000 that if it applies for one, it applies to the other. 00:22:28.000 --> 00:22:31.000 Do you have a question, or just holding your finger in 00:22:31.000 --> 00:22:33.000 the air? OK, good. 00:22:33.000 --> 00:22:37.000 OK, so that's all just review. Now I want to get onto today's 00:22:37.000 --> 00:22:40.000 lecture. OK, so any questions about 00:22:40.000 --> 00:22:42.000 graphs? So, this is a good time to 00:22:42.000 --> 00:22:45.000 review appendix B. there are a lot of great 00:22:45.000 --> 00:22:48.000 properties in there, and in particular, 00:22:48.000 --> 00:22:52.000 there is a theorem that we're going to cover today that we're 00:22:52.000 --> 00:22:56.000 going to talk about today, which is properties of trees. 00:22:56.000 --> 00:23:00.000 Trees are very special kinds of graphs, so I really want you to 00:23:00.000 --> 00:23:05.000 go and look to see what the properties are. 00:23:05.000 --> 00:23:08.000 There is, I think, something like six different 00:23:08.000 --> 00:23:11.000 definitions of trees that are all equivalent, 00:23:11.000 --> 00:23:16.000 OK, and so, I think a very good idea to go through and read 00:23:16.000 --> 00:23:20.000 through that theorem. We're not going to prove it in 00:23:20.000 --> 00:23:23.000 class, but really, provides a very good basis for 00:23:23.000 --> 00:23:27.000 the thinking that we're going to be doing today. 00:23:27.000 --> 00:23:30.000 And we'll see more of that in the future. 00:23:30.000 --> 00:23:33.000 OK, so today, we're going to talk about 00:23:33.000 --> 00:23:38.000 minimum spanning trees. OK, this is one of the world's 00:23:38.000 --> 00:23:42.000 most important algorithms. OK, it is important in 00:23:42.000 --> 00:23:46.000 distributed systems. It's one of the first things 00:23:46.000 --> 00:23:50.000 that almost any distributed system tries to find is a 00:23:50.000 --> 00:23:55.000 minimum spanning tree of the nodes that happened to be alive 00:23:55.000 --> 00:23:56.000 at any point, OK? 00:23:56.000 --> 00:24:01.000 And one of the people who developed an algorithm for this, 00:24:01.000 --> 00:24:04.000 we'll talk about this a little bit later, OK, 00:24:04.000 --> 00:24:09.000 it was the basis of the billing system for AT&T for many years 00:24:09.000 --> 00:24:16.000 while it was a monopoly. OK, so very important kind of 00:24:16.000 --> 00:24:21.000 thing. It's got a huge number of 00:24:21.000 --> 00:24:25.000 applications. So the problem is the 00:24:25.000 --> 00:24:31.000 following. You have a connected undirected 00:24:31.000 --> 00:24:35.000 graph, G equals (V,E), 00:24:35.000 --> 00:24:42.000 with an edge weight function, w, which maps the edges into 00:24:42.000 --> 00:24:50.000 weights that are real numbers. And for today's lecture, 00:24:50.000 --> 00:24:56.000 we're going to make an important assumption, 00:24:56.000 --> 00:25:02.000 OK, for simplicity. The book does not make this 00:25:02.000 --> 00:25:08.000 assumption. And so, I encourage you to look 00:25:08.000 --> 00:25:16.000 at the alternative presentation or, because what they do in the 00:25:16.000 --> 00:25:22.000 book is much more general, but for simplicity and 00:25:22.000 --> 00:25:29.000 intuition, I'm going to make this a little bit easier. 00:25:29.000 --> 00:25:37.000 We're going to assume that all edge weights are distinct. 00:25:37.000 --> 00:25:39.000 OK, all edge weights are distinct. 00:25:39.000 --> 00:25:42.000 So what does that mean? What does that mean that this 00:25:42.000 --> 00:25:46.000 function, w, what property does the function, 00:25:46.000 --> 00:25:48.000 w, have if all edge weights are distinct? 00:25:48.000 --> 00:25:51.000 Who remembers their discreet math? 00:25:51.000 --> 00:25:53.000 It's injective. OK, it's one to one. 00:25:53.000 --> 00:25:56.000 OK, it's not one to one and onto necessarily. 00:25:56.000 --> 00:26:00.000 In fact, it would be kind of hard to do that because that's a 00:26:00.000 --> 00:26:05.000 pretty big set. OK, but it's one to one. 00:26:05.000 --> 00:26:09.000 It's injective. OK, so that's what we're going 00:26:09.000 --> 00:26:14.000 to assume for simplicity. OK, and the book, 00:26:14.000 --> 00:26:19.000 they don't assume that. It just means that the way you 00:26:19.000 --> 00:26:24.000 have to state things is just a little more precise. 00:26:24.000 --> 00:26:28.000 It has to be more technically precise. 00:26:28.000 --> 00:26:33.000 So, that's the input. The output is-- 00:26:33.000 --> 00:26:44.000 The output is a spanning tree, T, and by spanning tree, 00:26:44.000 --> 00:26:52.000 we mean it connects all the vertices. 00:26:52.000 --> 00:27:02.000 OK, and it's got to have minimum weight. 00:27:02.000 --> 00:27:08.000 OK, so we can write the weight of the tree is going to be, 00:27:08.000 --> 00:27:14.000 by that, we meet the sum over all edges that are in the tree 00:27:14.000 --> 00:27:18.000 of the weight of the individual edges. 00:27:18.000 --> 00:27:24.000 OK, so here I'(V,E) done a little bit of abusive notation, 00:27:24.000 --> 00:27:31.000 which is that what I should be writing is w of the edge (u,v) 00:27:31.000 --> 00:27:37.000 because this is a mapping from edges, which would give me a 00:27:37.000 --> 00:27:42.000 double parentheses. And, you know, 00:27:42.000 --> 00:27:45.000 as you know, I love to abuse notation. 00:27:45.000 --> 00:27:48.000 So, I'm going to drop that extra parentheses, 00:27:48.000 --> 00:27:54.000 because we understand that it's really the weight of the edge, 00:27:54.000 --> 00:27:57.000 OK, not the weight of the ordered pair. 00:27:57.000 --> 00:28:02.000 So, that's just a little notational convenience. 00:28:02.000 --> 00:28:05.000 OK, so one of the things, when we do the take-home exam, 00:28:05.000 --> 00:28:09.000 notational convenience can make the difference between having a 00:28:09.000 --> 00:28:12.000 horrible time writing up a problem, and an easy time. 00:28:12.000 --> 00:28:16.000 So, it's worth thinking about what kinds of notation you'll 00:28:16.000 --> 00:28:19.000 use in writing up solutions to problems, and so forth. 00:28:19.000 --> 00:28:22.000 OK, and just in general, a technical communication, 00:28:22.000 --> 00:28:25.000 you adopt good notation people understand you. 00:28:25.000 --> 00:28:29.000 You adopt a poor notation: nobody pays attention to what 00:28:29.000 --> 00:28:34.000 you're doing because they don't understand what you're saying. 00:28:34.000 --> 00:28:38.000 OK, so let's do an example. 00:28:45.000 --> 00:28:52.000 OK, so here's a graph. I think for this, 00:28:52.000 --> 00:29:02.000 somebody asked once if I was inspired by biochemistry or 00:29:02.000 --> 00:29:09.000 something, OK, but I wasn't. 00:29:09.000 --> 00:29:11.000 I was just writing these things down, OK? 00:29:11.000 --> 00:29:15.000 So, here's a graph. And let's give us some edge 00:29:15.000 --> 00:29:16.000 weights. 00:29:31.000 --> 00:29:34.000 OK, so there are some edge weights. 00:29:34.000 --> 00:29:39.000 And now, what we want is we want to find a tree. 00:29:39.000 --> 00:29:45.000 So a connected acyclic graph such that every vertex is part 00:29:45.000 --> 00:29:49.000 of the tree. But it's got to have the 00:29:49.000 --> 00:29:55.000 minimum weight possible. OK, so can somebody suggest to 00:29:55.000 --> 00:30:03.000 me some edges that have to be in this minimum spanning tree? 00:30:03.000 --> 00:30:05.000 Yeah, so nine, good. 00:30:05.000 --> 00:30:09.000 Nine has to be in there because, why? 00:30:09.000 --> 00:30:14.000 It's the only one connecting it to this vertex, 00:30:14.000 --> 00:30:16.000 OK? And likewise, 00:30:16.000 --> 00:30:22.000 15 has to be in there. So those both have to be in. 00:30:22.000 --> 00:30:26.000 What other edges have to be in? Which one? 00:30:26.000 --> 00:30:33.000 14 has to be it. Why does 14 have to be in? 00:30:33.000 --> 00:30:40.000 Well, one of 14 and three has to be in there. 00:30:40.000 --> 00:30:50.000 I want the minimum weight. The one that has the overall 00:30:50.000 --> 00:30:57.000 smallest weight. So, can somebody argue to me 00:30:57.000 --> 00:31:04.000 that three has to be in there? Yeah? 00:31:04.000 --> 00:31:09.000 That's the minimum of two, which means that if I had a, 00:31:09.000 --> 00:31:14.000 if you add something you said was a minimum spanning tree that 00:31:14.000 --> 00:31:19.000 didn't include three, right, and so therefore it had 00:31:19.000 --> 00:31:23.000 to include 14, then I could just delete this 00:31:23.000 --> 00:31:28.000 edge, 14, and put in edge three. And, I have something of lower 00:31:28.000 --> 00:31:34.000 weight, right? So, three has to be in there. 00:31:34.000 --> 00:31:37.000 What other edges have to be in there? 00:31:37.000 --> 00:31:43.000 Do a little puzzle logic. Six and five have to be in 00:31:43.000 --> 00:31:46.000 there. Why do they have to be in 00:31:46.000 --> 00:31:48.000 there? 00:32:02.000 --> 00:32:05.000 Yeah, well, I mean, it could be connected through 00:32:05.000 --> 00:32:08.000 this or something. It doesn't necessarily have to 00:32:08.000 --> 00:32:11.000 go this way. Six definitely has to be in 00:32:11.000 --> 00:32:14.000 there for the same reason that three had to be, 00:32:14.000 --> 00:32:16.000 right? Because we got two choices to 00:32:16.000 --> 00:32:19.000 connect up this guy. And so, if everything were 00:32:19.000 --> 00:32:22.000 connected but it weren't, 12, I mean, and 12 was in 00:32:22.000 --> 00:32:24.000 there. I could always, 00:32:24.000 --> 00:32:27.000 then, say, well, let's connect them up this way 00:32:27.000 --> 00:32:31.000 instead. OK, so definitely that's in 00:32:31.000 --> 00:32:35.000 there. I still don't have everything 00:32:35.000 --> 00:32:37.000 connected up. 00:32:50.000 --> 00:33:03.000 What else has to be in there for minimum spanning tree? 00:33:03.000 --> 00:33:11.000 Seven, five, and eight, why seven, 00:33:11.000 --> 00:33:22.000 five, and eight? OK, so can we argue those one 00:33:22.000 --> 00:33:32.000 at a time? Why does five have to be in 00:33:32.000 --> 00:33:37.000 there? Yeah? 00:33:37.000 --> 00:33:41.000 OK, so we have four connected components because we have this 00:33:41.000 --> 00:33:43.000 one, this one, we actually have, 00:33:43.000 --> 00:33:46.000 yeah, this one here, and this one, 00:33:46.000 --> 00:33:49.000 good. We need at least three edges to 00:33:49.000 --> 00:33:53.000 connect them because each edge is going to reduce the connected 00:33:53.000 --> 00:33:57.000 components by one. OK, so we need three edges, 00:33:57.000 --> 00:33:59.000 and those are the three cheapest ones. 00:33:59.000 --> 00:34:04.000 And they work. That works, right? 00:34:04.000 --> 00:34:11.000 Any other edges are going to be bigger, so that works. 00:34:11.000 --> 00:34:15.000 Good. OK, and so, now do we have a 00:34:15.000 --> 00:34:19.000 spanning tree? Everything is, 00:34:19.000 --> 00:34:24.000 we have one big connected graph here, right? 00:34:24.000 --> 00:34:31.000 Is that what I got? Hey, that's the same as what I 00:34:31.000 --> 00:34:35.000 got. Life is predictable. 00:34:35.000 --> 00:34:41.000 OK, so, so everybody had the idea of what a minimum spanning 00:34:41.000 --> 00:34:44.000 tree is, then, out of this, 00:34:44.000 --> 00:34:49.000 OK, what's going on there? So, let's first of all make 00:34:49.000 --> 00:34:53.000 some observations about this puzzle. 00:34:53.000 --> 00:34:59.000 And what I want to do is remind you about the optimal 00:34:59.000 --> 00:35:06.000 substructure property because it turns out minimum spanning tree 00:35:06.000 --> 00:35:12.000 has a great optimal substructure property. 00:35:12.000 --> 00:35:17.000 OK, so the setup is going to be, we're going to have some 00:35:17.000 --> 00:35:22.000 minimum spanning tree. Let's call it T. 00:35:22.000 --> 00:35:27.000 And, I'm going to show that with the other edges in the 00:35:27.000 --> 00:35:32.000 graph, are not going to be shown. 00:35:32.000 --> 00:35:37.000 OK, so here's a graph. 00:35:54.000 --> 00:35:58.000 OK, so here's a graph. It looks like the one I have my 00:35:58.000 --> 00:36:01.000 piece of paper here. OK, so the idea is, 00:36:01.000 --> 00:36:05.000 this is some minimum spanning tree. 00:36:05.000 --> 00:36:09.000 Now, we want to look at a property of optimal 00:36:09.000 --> 00:36:13.000 substructure. And the way I'm going to get 00:36:13.000 --> 00:36:17.000 that, is, I'm going to remove some edge, (u,v), 00:36:17.000 --> 00:36:22.000 move an arbitrary edge, (u,v), in the minimum spanning 00:36:22.000 --> 00:36:26.000 tree. So, let's call this u and this 00:36:26.000 --> 00:36:29.000 V. And so, we're removing this 00:36:29.000 --> 00:36:33.000 edge. OK, so when I remove an edge in 00:36:33.000 --> 00:36:36.000 a tree, what happens to the tree? 00:36:36.000 --> 00:36:39.000 What's left? I have two trees left, 00:36:39.000 --> 00:36:41.000 OK? I have two trees left. 00:36:41.000 --> 00:36:45.000 Now, proving that, that's basically one of the 00:36:45.000 --> 00:36:50.000 properties in that appendix, and the properties of trees 00:36:50.000 --> 00:36:55.000 that I want you to read, OK, because you can actually 00:36:55.000 --> 00:37:00.000 prove that kind of thing rather than it just being obvious, 00:37:00.000 --> 00:37:05.000 which is, OK? OK, so we remove that. 00:37:05.000 --> 00:37:11.000 Then, T is partitioned into two subtrees. 00:37:11.000 --> 00:37:15.000 And, we'll call them T_1 and T_2. 00:37:15.000 --> 00:37:22.000 So, here's one subtree, and here's another subtree. 00:37:22.000 --> 00:37:29.000 We'(V,E) partitioned it. No matter what edge I picked, 00:37:29.000 --> 00:37:38.000 there would be two subtrees that it's partitioned into. 00:37:38.000 --> 00:37:40.000 Even if the sub tree is a trivial subtree, 00:37:40.000 --> 00:37:43.000 for example, it just has a single node in it 00:37:43.000 --> 00:37:45.000 and no edges. 00:37:58.000 --> 00:38:11.000 So, the theorem that we'll prove demonstrates a property of 00:38:11.000 --> 00:38:24.000 optimal substructure. T_1 is a minimum spanning tree 00:38:24.000 --> 00:38:31.000 for the graph, G_1, E_1, 00:38:31.000 --> 00:38:43.000 a subgraph of G induced by the vertices in T_1. 00:38:43.000 --> 00:38:55.000 OK, that is, V_1 is just the vertices in T_1 00:38:55.000 --> 00:39:09.000 is what it means to be induced. OK, so V_1 is the vertices in 00:39:09.000 --> 00:39:12.000 T_1. So, in this picture, 00:39:12.000 --> 00:39:16.000 I didn't label it. This is T_1. 00:39:16.000 --> 00:39:20.000 This is T_2. In this picture, 00:39:20.000 --> 00:39:27.000 these are the vertices of T_1. So, that's V_1, 00:39:27.000 --> 00:39:32.000 OK? And, E_1 is the set of pairs of 00:39:32.000 --> 00:39:39.000 vertices, x and y, that are the edges that are in 00:39:39.000 --> 00:39:47.000 E_1 such that both x and y belong to V_1. 00:39:47.000 --> 00:39:49.000 OK, so I haven't shown the edges of G here. 00:39:49.000 --> 00:39:52.000 But basically, if an edge went from here to 00:39:52.000 --> 00:39:57.000 here, that would be in the E_1. If it went from here to here, 00:39:57.000 --> 00:40:00.000 it would not. And if it went from here to 00:40:00.000 --> 00:40:04.000 here, it would not. OK, so the vertices, 00:40:04.000 --> 00:40:10.000 the subgraph induced by the vertices of T_1 are just those 00:40:10.000 --> 00:40:17.000 that connect up things in T_1, and similarly for T_2. 00:40:27.000 --> 00:40:33.000 So, the theorem says that if I look at just the edges within 00:40:33.000 --> 00:40:38.000 the graph here, G_1, those that are induced by 00:40:38.000 --> 00:40:41.000 these vertices, T_1 is, in fact, 00:40:41.000 --> 00:40:46.000 a minimum spanning tree for that subgraph. 00:40:46.000 --> 00:40:51.000 That's what the theorem says. OK, if I look over here 00:40:51.000 --> 00:40:58.000 conversely, or correspondingly, if I look at the set of edges 00:40:58.000 --> 00:41:05.000 that are induced by this set of vertices, the vertices in T_2, 00:41:05.000 --> 00:41:13.000 in fact, T_2 is a minimum spanning tree on that subgraph. 00:41:13.000 --> 00:41:17.000 OK, OK, we can even do it over here. 00:41:17.000 --> 00:41:21.000 If I took a look, for example, 00:41:21.000 --> 00:41:27.000 at these, let's see, let's say we cut out five, 00:41:27.000 --> 00:41:35.000 and if I cut out edge five, that T_1 would be these four 00:41:35.000 --> 00:41:40.000 vertices here. And, the point is that if I 00:41:40.000 --> 00:41:44.000 look at the subgraph induced on that, that these edges here. 00:41:44.000 --> 00:41:48.000 In fact, the six, eight, and three are all edges 00:41:48.000 --> 00:41:52.000 in a minimum spanning tree for that subgraph. 00:41:52.000 --> 00:41:54.000 OK, so that's what the theorem says. 00:41:54.000 --> 00:41:57.000 So let's prove it. 00:42:09.000 --> 00:42:19.000 OK, and so what technique are we going to use to prove it? 00:42:19.000 --> 00:42:28.000 OK, we learned this technique last time: hint, 00:42:28.000 --> 00:42:33.000 hint. It's something you do it in 00:42:33.000 --> 00:42:39.000 your text editor all the time: cut and paste, 00:42:39.000 --> 00:42:45.000 good, cut and paste. OK, so the weight of T I can 00:42:45.000 --> 00:42:51.000 express as the weight of the edge I removed, 00:42:51.000 --> 00:42:57.000 plus the weight of T_1, plus the weight of T_2. 00:42:57.000 --> 00:43:07.000 OK, so that's the total weight. So, the argument is pretty 00:43:07.000 --> 00:43:13.000 simple. Suppose that there were some 00:43:13.000 --> 00:43:20.000 T_1 prime that was better than T_1 for G_1. 00:43:20.000 --> 00:43:31.000 Suppose I had some better way of forming a spanning tree. 00:43:31.000 --> 00:43:42.000 OK, then I would make up a T prime, which just contained the 00:43:42.000 --> 00:43:48.000 edges, (u,v), and T_1 prime, 00:43:48.000 --> 00:43:53.000 union T_2. So, I would take, 00:43:53.000 --> 00:44:05.000 if I had a better spanning tree, a spanning tree of lower 00:44:05.000 --> 00:44:12.000 weight for T_1. And I call that T_1 prime. 00:44:12.000 --> 00:44:17.000 I just substitute that and make up a spanning tree that 00:44:17.000 --> 00:44:22.000 consisted of my edge, (u,v), whatever works well for 00:44:22.000 --> 00:44:26.000 T_1 prime and whatever works well for T. 00:44:26.000 --> 00:44:30.000 And, that would be a spanning tree. 00:44:30.000 --> 00:44:36.000 And it would be better than T itself was for G, 00:44:36.000 --> 00:44:44.000 OK, because the weight of these is just as the weight for this, 00:44:44.000 --> 00:44:50.000 I now just get to use the weight of T_1 prime, 00:44:50.000 --> 00:44:54.000 and that's less. And so, therefore, 00:44:54.000 --> 00:45:02.000 the assumption that T was a minimum spanning tree would be 00:45:02.000 --> 00:45:11.000 violated if I could find a better one for the subpiece. 00:45:11.000 --> 00:45:16.000 So, we have this nice property of optimal substructure. 00:45:16.000 --> 00:45:20.000 OK, I have subproblems that exhibit optimal, 00:45:20.000 --> 00:45:25.000 if I have a globally optimal solution to the whole problem 00:45:25.000 --> 00:45:31.000 within it, I can find optimal solutions to subproblems. 00:45:31.000 --> 00:45:36.000 So, now the question is, that's one hallmark. 00:45:36.000 --> 00:45:41.000 That's one hallmark of dynamic programming. 00:45:41.000 --> 00:45:45.000 What about overlapping subproblems? 00:45:45.000 --> 00:45:51.000 Do I have that property? Do I have overlapping 00:45:51.000 --> 00:45:58.000 subproblems over here for this type of problem? 00:46:19.000 --> 00:46:20.000 So, imagine, for example, 00:46:20.000 --> 00:46:22.000 that I'm removing different edges. 00:46:22.000 --> 00:46:26.000 I look at the space of taking a given edge, and removing it. 00:46:26.000 --> 00:46:30.000 It partitions it into two pieces, and now I have another 00:46:30.000 --> 00:46:32.000 piece. And I remove it, 00:46:32.000 --> 00:46:35.000 etc. Am I going to end up getting a 00:46:35.000 --> 00:46:38.000 bunch of subproblems that are similar in there? 00:46:38.000 --> 00:46:41.000 Yeah, I am. OK, if I take out this one, 00:46:41.000 --> 00:46:43.000 then I take out, say, this one here, 00:46:43.000 --> 00:46:46.000 and then I'll have another tree here and here. 00:46:46.000 --> 00:46:51.000 OK, that would be the same as if I had originally taken this 00:46:51.000 --> 00:46:53.000 out, and then taken that one out. 00:46:53.000 --> 00:46:57.000 If I look at simple ordering of taking out the edges, 00:46:57.000 --> 00:47:00.000 I'm going to end up with a whole bunch of overlapping 00:47:00.000 --> 00:47:04.000 subproblems. Yeah, OK. 00:47:04.000 --> 00:47:14.000 So then, what does that suggest we use as an approach? 00:47:14.000 --> 00:47:18.000 Dynamic programming, good. 00:47:18.000 --> 00:47:26.000 What a surprise! Yes, OK, you could use dynamic 00:47:26.000 --> 00:47:33.000 programming. But it turns out that minimum 00:47:33.000 --> 00:47:41.000 spanning tree exhibits an even more powerful property. 00:47:41.000 --> 00:47:48.000 OK, so we'(V,E) got all the clues for dynamic programming, 00:47:48.000 --> 00:47:57.000 but it turns out that there's an even bigger clue that's going 00:47:57.000 --> 00:48:05.000 to help us to use an even more powerful technique. 00:48:05.000 --> 00:48:11.000 And that, we call, the hallmark for greedy 00:48:11.000 --> 00:48:13.000 algorithms. 00:48:32.000 --> 00:48:41.000 And that is, we have a thing called the 00:48:41.000 --> 00:48:53.000 greedy choice property, which says that a locally 00:48:53.000 --> 00:49:03.000 optimal choice is globally optimal. 00:49:03.000 --> 00:49:05.000 And, of course, as all these hallmarks is the 00:49:05.000 --> 00:49:09.000 kind of thing you want to box, OK, because these are the clues 00:49:09.000 --> 00:49:12.000 that you're going to be able to do that. 00:49:12.000 --> 00:49:15.000 So, we have this property that we call the greedy choice 00:49:15.000 --> 00:49:18.000 property. I'm going to show you how it 00:49:18.000 --> 00:49:21.000 works in this case. And when you have a greedy 00:49:21.000 --> 00:49:24.000 choice property, it turns out you can do even 00:49:24.000 --> 00:49:29.000 better that dynamic programming. OK, so when you see the two 00:49:29.000 --> 00:49:33.000 dynamic programming properties, there is a clue that says 00:49:33.000 --> 00:49:36.000 dynamic programming, yes, but also it says, 00:49:36.000 --> 00:49:41.000 let me see whether it also has this greedy property because if 00:49:41.000 --> 00:49:46.000 it does, you're going to come up with something that's even 00:49:46.000 --> 00:49:49.000 better than dynamic programming, OK? 00:49:49.000 --> 00:49:53.000 So, if you just have the two, you can usually do dynamic 00:49:53.000 --> 00:49:56.000 programming, but if you have this third one, 00:49:56.000 --> 00:50:00.000 it's like, whoa! Jackpot! 00:50:00.000 --> 00:50:04.000 OK, so here's the theorem we'll prove to illustrate this idea. 00:50:04.000 --> 00:50:08.000 Once again, these are not, all these hallmarks are not 00:50:08.000 --> 00:50:09.000 things. They are heuristics. 00:50:09.000 --> 00:50:14.000 I can't give you an algorithm to say, here's where dynamic 00:50:14.000 --> 00:50:16.000 programming works, or here's where greedy 00:50:16.000 --> 00:50:20.000 algorithms work. But I can sort of indicate when 00:50:20.000 --> 00:50:23.000 they work, the kind of structure they have. 00:50:23.000 --> 00:50:32.000 OK, so here's the theorem. So let's let T be the MST of 00:50:32.000 --> 00:50:40.000 our graph. And, let's let A be any subset 00:50:40.000 --> 00:50:49.000 of V, so, some subset of vertices. 00:50:49.000 --> 00:51:04.000 And now, let's suppose that edge, (u,v), is the least weight 00:51:04.000 --> 00:51:17.000 edge connecting our set A to A complement, that is, 00:51:17.000 --> 00:51:27.000 V minus A. Then the theorem says that 00:51:27.000 --> 00:51:39.000 (u,v) is in the minimum spanning tree. 00:51:39.000 --> 00:51:43.000 So let's just take a look at our graph over here and see if 00:51:43.000 --> 00:51:45.000 that's, in fact, the case. 00:51:45.000 --> 00:51:49.000 OK, so let's take, so one thing I could do for A 00:51:49.000 --> 00:51:53.000 is just take a singleton node. So, I take a singleton node, 00:51:53.000 --> 00:51:56.000 let's say this guy here, that can be my A, 00:51:56.000 --> 00:52:00.000 and everything else is V minus A. 00:52:00.000 --> 00:52:04.000 And I look at the least weight edge connecting this to 00:52:04.000 --> 00:52:07.000 everything else. Well, there are only two edges 00:52:07.000 --> 00:52:10.000 that connect it to everything else. 00:52:10.000 --> 00:52:15.000 And the theorem says that the lighter one is in the minimum 00:52:15.000 --> 00:52:17.000 spanning tree. Hey, I win. 00:52:17.000 --> 00:52:21.000 OK, if you take a look, every vertex that I pick, 00:52:21.000 --> 00:52:25.000 the latest edge coming out of that vertex is in the minimum 00:52:25.000 --> 00:52:29.000 spanning tree. OK, the lightest weight edge 00:52:29.000 --> 00:52:35.000 coming out, but that's not all the edges that are in here. 00:52:35.000 --> 00:52:39.000 OK, or let's just imagine, let's take a look at these 00:52:39.000 --> 00:52:43.000 three vertices connected to this set of vertices. 00:52:43.000 --> 00:52:46.000 I have three edges is going across. 00:52:46.000 --> 00:52:50.000 The least weight one is five. That's the minimum spanning 00:52:50.000 --> 00:52:53.000 tree. Or, I can cut it this way. 00:52:53.000 --> 00:52:57.000 OK, the ones above one, the edges going down are seven, 00:52:57.000 --> 00:53:02.000 eight, and 14. Seven is the least weight. 00:53:02.000 --> 00:53:04.000 It's in the minimum spanning tree. 00:53:04.000 --> 00:53:08.000 So, no matter how I choose, I could make this one in, 00:53:08.000 --> 00:53:10.000 this one out, this one in, 00:53:10.000 --> 00:53:12.000 this one out, this one in, 00:53:12.000 --> 00:53:14.000 this one out, this one in, 00:53:14.000 --> 00:53:18.000 this one out, take a look at what all the 00:53:18.000 --> 00:53:21.000 edges are. Which ever one to the least 00:53:21.000 --> 00:53:24.000 weight: it's in the minimum spanning tree. 00:53:24.000 --> 00:53:28.000 So, in some sense, that's a local property because 00:53:28.000 --> 00:53:34.000 I don't have to look at what the rest of the tree is. 00:53:34.000 --> 00:53:38.000 I'm just looking at some small set of vertices if I wish, 00:53:38.000 --> 00:53:42.000 and I say, well, if I wanted to connect that set 00:53:42.000 --> 00:53:46.000 of vertices to the rest of the world, what would I pick? 00:53:46.000 --> 00:53:50.000 I'd pick the cheapest one. That's the greedy approach. 00:53:50.000 --> 00:53:53.000 It turns out, that wins, OK, 00:53:53.000 --> 00:53:57.000 that picking that thing that's locally good for that subset, 00:53:57.000 --> 00:54:04.000 A, OK, is also globally good. OK, it optimizes the overall 00:54:04.000 --> 00:54:09.000 function. That's what the theorem says, 00:54:09.000 --> 00:54:13.000 OK? So, let's prove this theorem. 00:54:13.000 --> 00:54:20.000 Any questions about this? OK, let's prove this theorem. 00:54:20.000 --> 00:54:27.000 So, we have (u,v) is the least weight edge connecting A to D 00:54:27.000 --> 00:54:32.000 minus A. So, let's suppose that this 00:54:32.000 --> 00:54:40.000 edge, (u,v), is not in the minimum spanning tree. 00:54:40.000 --> 00:54:45.000 OK, let's suppose that somehow there is a minimum spanning tree 00:54:45.000 --> 00:54:50.000 that doesn't include this least weight edge. 00:54:50.000 --> 00:54:55.000 OK, so what technique you think will use to prove to get a 00:54:55.000 --> 00:54:58.000 contradiction here? Cut and paste, 00:54:58.000 --> 00:55:04.000 good. Yeah, we're going to cut paste. 00:55:04.000 --> 00:55:08.000 OK, we're going to cut and paste. 00:55:08.000 --> 00:55:14.000 So here, I did an example. OK, so -- 00:55:40.000 --> 00:55:42.000 OK, and so I'm going to use the notation. 00:55:42.000 --> 00:55:44.000 I'm going to color some of these in. 00:56:05.000 --> 00:56:10.000 OK, and so my notation here is this is an element of A, 00:56:10.000 --> 00:56:14.000 and color it in. It's an element of V minus A. 00:56:14.000 --> 00:56:18.000 OK, so if it's not colored it, that's an A. 00:56:18.000 --> 00:56:21.000 This is my minimum spanning tree. 00:56:21.000 --> 00:56:27.000 Once again, I'm not showing the overall edges of all the graphs, 00:56:27.000 --> 00:56:30.000 but they're there, OK? 00:56:30.000 --> 00:56:33.000 So, my edge, (u,v), which is not my minimum 00:56:33.000 --> 00:56:38.000 spanning tree I say, let's say is this edge here. 00:56:38.000 --> 00:56:42.000 It's an edge from u, u as in A, v as in V minus A. 00:56:42.000 --> 00:56:48.000 OK, so everybody see the setup? So, I want to prove that this 00:56:48.000 --> 00:56:52.000 edge should have been in the minimum spanning tree, 00:56:52.000 --> 00:56:58.000 OK, that the contention that this is a minimum spanning tree, 00:56:58.000 --> 00:57:02.000 and does include (u,v) is wrong. 00:57:02.000 --> 00:57:05.000 So, what I want to do, that, is I have a tree here, 00:57:05.000 --> 00:57:08.000 T, and I have two vertices, u and v, and in a tree, 00:57:08.000 --> 00:57:12.000 between any two vertices there is a unique, simple path: 00:57:12.000 --> 00:57:16.000 simple path meaning it doesn't go back and forth and repeat 00:57:16.000 --> 00:57:18.000 edges or vertices. OK, there's a unique, 00:57:18.000 --> 00:57:23.000 simple path from u to v. So, let's consider that path. 00:57:42.000 --> 00:57:46.000 OK, and the way that I know that that path exists is because 00:57:46.000 --> 00:57:51.000 I'(V,E) read appendix B of the textbook, section B.5.1, 00:57:51.000 --> 00:57:56.000 OK, which has this nice theorem about properties of trees. 00:57:56.000 --> 00:58:00.000 OK, so that's how I know that there exists a unique, 00:58:00.000 --> 00:58:05.000 simple path. OK, so now we're going to do is 00:58:05.000 --> 00:58:09.000 take a look at that path. So in this case, 00:58:09.000 --> 00:58:13.000 it goes from here, to here, to here, 00:58:13.000 --> 00:58:16.000 to here. And along that path, 00:58:16.000 --> 00:58:22.000 there must be a point where I connect from a vertex in A to a 00:58:22.000 --> 00:58:25.000 vertex in V minus A. Why? 00:58:25.000 --> 00:58:32.000 Well, because this is in A. This is in V minus A. 00:58:32.000 --> 00:58:42.000 So, along the path somewhere, there must be a transition. 00:58:42.000 --> 00:58:52.000 OK, they are not all in A, OK, because in particular, 00:58:52.000 --> 00:58:58.000 V isn't. OK, so we're going to do is 00:58:58.000 --> 00:59:09.000 swap (u,v) with the first edge on this path that connects a 00:59:09.000 --> 00:59:18.000 vertex in A to a vertex in V minus A. 00:59:18.000 --> 00:59:20.000 So in this case, it's this edge here. 00:59:20.000 --> 00:59:24.000 I go from A to V minus A. In general, I might be 00:59:24.000 --> 00:59:28.000 alternating many times, OK, and I just picked the first 00:59:28.000 --> 00:59:32.000 one that I encounter. OK, that this guy here. 00:59:32.000 --> 00:59:36.000 And what I do is I put this edge in. 00:59:36.000 --> 00:59:38.000 OK, so then, what happens? 00:59:38.000 --> 00:59:42.000 Well, the edge, (u,v), is the lightest thing 00:59:42.000 --> 00:59:46.000 connecting something in A to something in V minus A. 00:59:46.000 --> 00:59:49.000 So that means, in particular, 00:59:49.000 --> 00:59:53.000 it's lighter than this edge, has lower weight. 00:59:53.000 --> 00:59:57.000 So, by swapping this, I'(V,E) created a tree with 00:59:57.000 --> 01:00:02.198 lower overall weight, contradicting the assumption 01:00:02.198 --> 01:00:08.000 that this other thing was a minimum spanning tree. 01:00:08.000 --> 01:00:14.219 OK: so, a lower weight spanning tree than T results, 01:00:14.219 --> 01:00:18.000 and that's a contradiction -- 01:00:25.000 --> 01:00:33.010 -- than T results. And that's a contradiction, 01:00:33.010 --> 01:00:36.570 OK? How are we doing? 01:00:36.570 --> 01:00:44.225 Everybody with me? OK, now we get to do some 01:00:44.225 --> 01:00:46.895 algorithms. Yea! 01:00:46.895 --> 01:00:55.439 So, we are going to do an algorithm called Prim's 01:00:55.439 --> 01:01:01.853 algorithm. Prim eventually became a very 01:01:01.853 --> 01:01:07.069 high-up at AT&T because he invented this algorithm for 01:01:07.069 --> 01:01:12.187 minimum spanning trees, and it was used in all of the 01:01:12.187 --> 01:01:15.730 billing code for AT&T for many years. 01:01:15.730 --> 01:01:21.438 He was very high up at Bell Labs back in the heyday of Bell 01:01:21.438 --> 01:01:24.784 Laboratories. OK, so it just shows, 01:01:24.784 --> 01:01:30.000 all you have to do is invent an algorithm. 01:01:30.000 --> 01:01:36.702 You too can be a president of a corporate monopoly. 01:01:36.702 --> 01:01:43.807 Of course, the government can do things to monopolies, 01:01:43.807 --> 01:01:49.438 but anyway, if that's your mission in life, 01:01:49.438 --> 01:01:55.202 invent an algorithm. OK, so here's the idea. 01:01:55.202 --> 01:02:03.648 What we're going to do is we're going to maintain V minus A as a 01:02:03.648 --> 01:02:11.923 priority queue. We'll call it Q. 01:02:11.923 --> 01:02:26.076 And each vertex, we're going to key each vertex 01:02:26.076 --> 01:02:39.923 in Q with the weight of the least weight edge, 01:02:39.923 --> 01:02:53.280 connecting it to a vertex in A. So here's the code. 01:02:53.280 --> 01:03:00.000 So, we're going to start out with Q being all vertices. 01:03:00.000 --> 01:03:03.873 So, we start out with A being, if you will, 01:03:03.873 --> 01:03:07.930 the empty set. OK, and what we're going to do 01:03:07.930 --> 01:03:13.095 it is the least weight edge, therefore, for everything in 01:03:13.095 --> 01:03:18.536 the priority queue is basically going to be infinity because 01:03:18.536 --> 01:03:23.700 none of them have any edges. The least weight edge to the 01:03:23.700 --> 01:03:29.325 empty set is going to be empty. And then, we're going to start 01:03:29.325 --> 01:03:33.958 out with one guy. We'll call him S, 01:03:33.958 --> 01:03:39.489 which will set to zero for some arbitrary S in V. 01:03:39.489 --> 01:03:45.135 And then, the main part of the algorithm kicks in. 01:03:45.135 --> 01:03:51.703 So that's our initialization. OK, when we do the analysis, 01:03:51.703 --> 01:03:58.271 I'm going to write some stuff on the left hand side of the 01:03:58.271 --> 01:04:04.406 board. So if you're taking notes, 01:04:04.406 --> 01:04:14.406 you may want to also leave a little bit of space on the left 01:04:14.406 --> 01:04:22.711 hand side of your notes. So, while Q is not empty, 01:04:22.711 --> 01:04:30.000 we get the smallest element out of it. 01:04:41.000 --> 01:04:43.000 And then we do some stuff. 01:05:19.000 --> 01:05:21.970 That's it. And the only thing I should 01:05:21.970 --> 01:05:25.503 mention here is, OK, so let's just see what's 01:05:25.503 --> 01:05:28.875 going on here. And then we'll run it on the 01:05:28.875 --> 01:05:32.256 example. OK, so what we do is we take 01:05:32.256 --> 01:05:36.609 out the smallest element out of the queue at each step. 01:05:36.609 --> 01:05:40.156 And then for each step in the adjacency list, 01:05:40.156 --> 01:05:43.783 in other words, everything for which I have an 01:05:43.783 --> 01:05:46.846 edge going from v to u, we take a look, 01:05:46.846 --> 01:05:51.440 and if v is still in our set V minus A, so things we'(V,E) 01:05:51.440 --> 01:05:54.261 taken out are going to be part of A. 01:05:54.261 --> 01:05:57.163 OK, every time we take something out, 01:05:57.163 --> 01:06:02.000 that's going to be a new A that we construct. 01:06:02.000 --> 01:06:04.258 At every step, we want to find, 01:06:04.258 --> 01:06:08.400 what's the cheapest edge connecting that A to everything 01:06:08.400 --> 01:06:11.035 else? We basically are going to take 01:06:11.035 --> 01:06:15.025 whatever that cheapest thing is, OK, add that edge in, 01:06:15.025 --> 01:06:19.242 and now bring that into A and find the next cheapest one. 01:06:19.242 --> 01:06:22.103 And we just keep repeating the process. 01:06:22.103 --> 01:06:25.567 OK, we'll do it on the example. And what we do, 01:06:25.567 --> 01:06:28.955 is every time we bring it in, I keep track of, 01:06:28.955 --> 01:06:34.000 what was the vertex responsible for bringing me in. 01:06:34.000 --> 01:06:43.947 And what I claim is that at the end, if I look at the set of 01:06:43.947 --> 01:06:52.209 these pairs that I'(V,E) made here, V and pi of V, 01:06:52.209 --> 01:06:58.279 that forms the minimum spanning tree. 01:06:58.279 --> 01:07:05.441 So let's just do this. And, what's that? 01:07:05.441 --> 01:07:12.191 We're all set up. So let's get rid of these guys 01:07:12.191 --> 01:07:20.234 here because we are going to recompute them from scratch. 01:07:20.234 --> 01:07:30.000 OK, so you may want to copy the graph over again in your notes. 01:07:30.000 --> 01:07:34.840 I was going to do it, but it turned out, 01:07:34.840 --> 01:07:40.797 this is exactly the board is going to erase this. 01:07:40.797 --> 01:07:47.127 OK, well let me just modify it. OK, so we start out. 01:07:47.127 --> 01:07:54.574 We make everything be infinity. OK, so that's where I'm going 01:07:54.574 --> 01:08:01.028 to keep the key value. OK, and then what I'm going to 01:08:01.028 --> 01:08:07.952 do is find one vertex. And I'm going to call him S. 01:08:07.952 --> 01:08:11.749 And I'm going to do this vertex here. 01:08:11.749 --> 01:08:15.018 We'll call that S. So basically, 01:08:15.018 --> 01:08:19.447 I now make him be zero. And now, what I do, 01:08:19.447 --> 01:08:23.453 is I execute extract min. So basically, 01:08:23.453 --> 01:08:28.198 what I'll do is I'll just shade him like this, 01:08:28.198 --> 01:08:34.000 indicating that he has now joined the set A. 01:08:34.000 --> 01:08:40.931 So, this is going to be A. And this is element of V minus 01:08:40.931 --> 01:08:44.644 A. OK, so then what we do is we 01:08:44.644 --> 01:08:47.986 take a look. We extract him, 01:08:47.986 --> 01:08:53.433 and then for each edge in the adjacency list, 01:08:53.433 --> 01:08:59.002 OK, so for each vertex in the adjacency lists, 01:08:59.002 --> 01:09:05.315 that these guys here, OK, we're going to look to see 01:09:05.315 --> 01:09:12.000 if it's still in Q, that is, in V minus A. 01:09:12.000 --> 01:09:16.795 And if so, and its key value is less than what the value is at 01:09:16.795 --> 01:09:20.254 the edge, there, we're going to replace it by 01:09:20.254 --> 01:09:22.770 the edge value. So, in this case, 01:09:22.770 --> 01:09:25.600 we're going to replace this by seven. 01:09:25.600 --> 01:09:30.317 We're going to replace this by 15, and we're going to replace 01:09:30.317 --> 01:09:33.854 this by ten, OK, because what we're interested 01:09:33.854 --> 01:09:39.608 in is, what is the cheapest? Now, notice that everything in 01:09:39.608 --> 01:09:43.782 V minus A, that is, what's in the priority queue, 01:09:43.782 --> 01:09:48.216 everything in there, OK, now has its cheapest way of 01:09:48.216 --> 01:09:53.086 connecting it to the things that I'(V,E) already removed, 01:09:53.086 --> 01:09:57.173 the things that are in A. OK, and so now I just, 01:09:57.173 --> 01:10:01.608 OK, when I actually do that update, there's actually 01:10:01.608 --> 01:10:07.000 something implicit going on in this priority queue. 01:10:07.000 --> 01:10:10.636 And that is that I have to do a decreased key. 01:10:10.636 --> 01:10:14.111 So, there's an implicit decrease of the key. 01:10:14.111 --> 01:10:19.121 So, decreased key is a priority queue operation that lowers the 01:10:19.121 --> 01:10:22.191 value of the key in the priority queue. 01:10:22.191 --> 01:10:26.878 And so, that's implicitly going on when I look at what data 01:10:26.878 --> 01:10:31.646 structure I'm going to use to implement that priority queue. 01:10:31.646 --> 01:10:36.171 OK, so common data structures for implementing a priority 01:10:36.171 --> 01:10:41.376 queue are a min heap. OK, so I have to make sure that 01:10:41.376 --> 01:10:43.905 I'm actually doing this operation. 01:10:43.905 --> 01:10:47.355 I can't just change it and not affect my heap. 01:10:47.355 --> 01:10:51.111 So, there is an implicit operation going on there. 01:10:51.111 --> 01:10:54.407 OK, now I repeat. I find the cheapest thing, 01:10:54.407 --> 01:10:58.547 oh, and I also have to set, now, a pointer from each of 01:10:58.547 --> 01:11:02.931 these guys back to u. So here, this guy sets a 01:11:02.931 --> 01:11:07.114 pointer going this way. This guy sets a pointer going 01:11:07.114 --> 01:11:11.298 this way, and this guy sets a pointer going this way. 01:11:11.298 --> 01:11:16.206 That's my pi thing that's going to keep track of who caused me 01:11:16.206 --> 01:11:20.873 to set my value to what it is. So now, we go in and we find 01:11:20.873 --> 01:11:22.885 the cheapest thing, again. 01:11:22.885 --> 01:11:25.620 And we're going to do it fast, too. 01:11:25.620 --> 01:11:32.361 OK, this is a fast algorithm. OK, so now we're going to go do 01:11:32.361 --> 01:11:36.481 this again. So now, what's the cheapest 01:11:36.481 --> 01:11:39.843 thing to extract? This guy here, 01:11:39.843 --> 01:11:42.987 right? So, we'll take him out, 01:11:42.987 --> 01:11:47.542 OK, and now we update all of his neighbors. 01:11:47.542 --> 01:11:51.771 So this guy gets five. This guy gets 12. 01:11:51.771 --> 01:11:56.542 This guy gets nine. This guy we don't update. 01:11:56.542 --> 01:12:02.722 We don't update him because he's no longer in the priority 01:12:02.722 --> 01:12:07.464 queue. And all of these guys now, 01:12:07.464 --> 01:12:12.297 we make point to where they're supposed to point to. 01:12:12.297 --> 01:12:17.983 And, we're done with that step. Now we find the cheapest one. 01:12:17.983 --> 01:12:22.437 What's the cheapest one now? The five over here. 01:12:22.437 --> 01:12:24.807 Good. So, we take him out. 01:12:24.807 --> 01:12:30.019 OK, we update the neighbors. Here, yep, that goes to six 01:12:30.019 --> 01:12:34.000 now. And, we have that pointer. 01:12:34.000 --> 01:12:39.684 And, this guy we don't do, because he's not in there. 01:12:39.684 --> 01:12:44.604 This guy becomes 14, and this guy here becomes 01:12:44.604 --> 01:12:47.774 eight. So, we update that guy, 01:12:47.774 --> 01:12:52.803 make him be eight. Did I do this the right way? 01:12:52.803 --> 01:12:57.395 Yeah, because pi is a function of this guy. 01:12:57.395 --> 01:13:00.675 So basically, this thing, then, 01:13:00.675 --> 01:13:04.938 disappears. Yeah, did I have another one 01:13:04.938 --> 01:13:09.258 that I missed? 12, yes, good, 01:13:09.258 --> 01:13:12.584 it's removed, OK, because pi is just a 01:13:12.584 --> 01:13:14.741 function. And now I'm OK. 01:13:14.741 --> 01:13:18.516 OK, so now what do I do? OK, so now my set, 01:13:18.516 --> 01:13:23.191 A, consists of these three things, and now I want the 01:13:23.191 --> 01:13:26.786 cheapest edge. I know it's in the minimum 01:13:26.786 --> 01:13:30.561 spanning tree. So let me just greedily pick 01:13:30.561 --> 01:13:34.554 it. OK, so what's the cheapest 01:13:34.554 --> 01:13:37.108 thing now? This guy appear? 01:13:37.108 --> 01:13:39.466 Yeah, six. So we take it. 01:13:39.466 --> 01:13:44.771 We go to update these things, and nothing matters here. 01:13:44.771 --> 01:13:50.175 OK, nothing changes because these guys are already in A. 01:13:50.175 --> 01:13:54.202 OK, so now the cheapest one is eight here. 01:13:54.202 --> 01:13:56.856 Good. So, we take eight out. 01:13:56.856 --> 01:14:01.656 OK, we update this. Nothing to be done. 01:14:01.656 --> 01:14:04.970 This: nothing to be done. This: oh, no, 01:14:04.970 --> 01:14:09.242 this one, instead of 14 we can make this be three. 01:14:09.242 --> 01:14:14.212 So, we get rid of that pointer and make it point that way. 01:14:14.212 --> 01:14:16.915 Now three is the cheapest thing. 01:14:16.915 --> 01:14:21.100 So, we take it out, and of course there's nothing 01:14:21.100 --> 01:14:24.239 to be done over there. And now, last, 01:14:24.239 --> 01:14:26.506 I take nine. And it's done. 01:14:26.506 --> 01:14:32.000 And 15: it's done. And the algorithm terminates. 01:14:32.000 --> 01:14:36.972 OK, and as I look at, now, all the edges that I 01:14:36.972 --> 01:14:43.135 picked, those are exactly all the edges that we had at the 01:14:43.135 --> 01:14:48.000 beginning. OK, let's do an analysis here. 01:14:58.000 --> 01:15:06.316 OK, so let's see, this part here costs me order 01:15:06.316 --> 01:15:11.197 V, right? OK, and this part, 01:15:11.197 --> 01:15:16.983 let's see what we are doing here. 01:15:16.983 --> 01:15:27.107 Well, we're going to go through this loop how many times? 01:15:27.107 --> 01:15:32.711 V times. It's V elements we put into the 01:15:32.711 --> 01:15:35.860 queue. We are not inserting anything. 01:15:35.860 --> 01:15:39.795 We're just taking them out. This goes V times, 01:15:39.795 --> 01:15:43.818 OK, and we do a certain number of extract Mins. 01:15:43.818 --> 01:15:47.492 So, we're going to do order V extract Mins. 01:15:47.492 --> 01:15:52.915 And then we go to the adjacency list, and we have some constant 01:15:52.915 --> 01:15:55.801 things. But we have these implicit 01:15:55.801 --> 01:16:00.000 decreased keys for this stuff here. 01:16:00.000 --> 01:16:07.412 That's this thing here. OK, and so how many implicit 01:16:07.412 --> 01:16:14.389 decreased keys do we have? That's going to be the 01:16:14.389 --> 01:16:18.459 expensive thing. OK, we have, 01:16:18.459 --> 01:16:25.000 in this case, the degree of u of those. 01:16:25.000 --> 01:16:31.309 OK, so overall, how many implicit decreased 01:16:31.309 --> 01:16:38.218 keys do we have? Well, we have V times through. 01:16:38.218 --> 01:16:43.025 How big could the degree of u be? 01:16:43.025 --> 01:16:48.433 OK, it could be as big as V, order V. 01:16:48.433 --> 01:16:56.995 So, that's V^2 decreased use. But we can do a better bound 01:16:56.995 --> 01:17:04.189 than that. How many do we really have? 01:17:04.189 --> 01:17:11.948 Yeah, at most order E, OK, because what am I doing? 01:17:11.948 --> 01:17:19.086 I'm summing up the degrees of all the vertices. 01:17:19.086 --> 01:17:27.000 That's how many times I actually execute that. 01:17:27.000 --> 01:17:34.322 So, I have order E, implicit decreased keys. 01:17:34.322 --> 01:17:44.028 So the time overall is order V times time for whatever the 01:17:44.028 --> 01:17:53.224 extract Min is plus E times the time for decreased key. 01:17:53.224 --> 01:18:02.931 So now, let's look at data structures, and we can evaluate 01:18:02.931 --> 01:18:14.000 for different data structures what this formula gives us. 01:18:14.000 --> 01:18:21.492 So, we have different ways of implementing a data structure. 01:18:21.492 --> 01:18:28.222 We have the cost of extract Min, and of decreased key, 01:18:28.222 --> 01:18:32.636 and total. So, the simplest way of 01:18:32.636 --> 01:18:38.369 implementing a data structure is an unsorted array. 01:18:38.369 --> 01:18:44.904 If I have an unsorted array, how much time does it take me 01:18:44.904 --> 01:18:51.668 to extract the minimum element? If I have an unsorted array? 01:18:51.668 --> 01:18:58.433 Right, order V in this case because it's an array of size V. 01:18:58.433 --> 01:19:06.000 And, to do a decreased key, OK, I can do it in order one. 01:19:06.000 --> 01:19:14.245 So, the total is V^2, good, order V^2 algorithm. 01:19:14.245 --> 01:19:22.666 Or, as people suggested, how about a binary heap? 01:19:22.666 --> 01:19:33.017 OK, to do an extract Min in a binary heap will cost me what? 01:19:33.017 --> 01:19:38.905 O of log V. Decreased key will cost me, 01:19:38.905 --> 01:19:44.932 yeah, it turns out you can do that in order log V because 01:19:44.932 --> 01:19:49.668 basically you just have to shuffle the value, 01:19:49.668 --> 01:19:54.295 actually shuffle it up towards the root, OK? 01:19:54.295 --> 01:19:58.708 Or at log V. And, the total cost therefore 01:19:58.708 --> 01:20:01.717 is? E log V, good. 01:20:01.717 --> 01:20:06.869 Which of these is better? It depends, good. 01:20:06.869 --> 01:20:12.758 When is one better, and when is the other better? 01:20:12.758 --> 01:20:18.401 Yeah, if it's a dense graph, E is close to V^2, 01:20:18.401 --> 01:20:24.166 the array is better. But if it's a sparse graph, 01:20:24.166 --> 01:20:33.000 and E is much smaller than V^2, then the binary heap is better. 01:20:33.000 --> 01:20:37.824 So that motivated the invention of a data structure, 01:20:37.824 --> 01:20:43.216 OK, called a Fibonacci Heap. So, Fibonacci Heap is covered 01:20:43.216 --> 01:20:47.851 in Chapter 20 of CLRS. We're not going to hold you 01:20:47.851 --> 01:20:53.148 responsible for the content, but it's an interesting data 01:20:53.148 --> 01:20:57.878 structure because it's an amortized data structure. 01:20:57.878 --> 01:21:01.851 And it turns out that it is data structure, 01:21:01.851 --> 01:21:08.000 you can do extract Min in order log V amortized time. 01:21:08.000 --> 01:21:12.747 And remarkably, you can do decreased key in 01:21:12.747 --> 01:21:17.834 order one amortized. So, when I plug those in, 01:21:17.834 --> 01:21:21.000 what do I get over here? 01:21:34.000 --> 01:21:42.088 What's that going to be? Plug that it here. 01:21:42.088 --> 01:21:52.296 It's going to be V times log V plus E: E plus V log V. 01:21:52.296 --> 01:22:00.000 These are amortized, so what's this? 01:22:00.000 --> 01:22:02.317 Trick question. It's worst-case. 01:22:02.317 --> 01:22:05.979 It's not amortized over here. These are amortized, 01:22:05.979 --> 01:22:08.745 but that's the beauty of amortization. 01:22:08.745 --> 01:22:13.006 I can say it's going to be worst case: E plus V log V over 01:22:13.006 --> 01:22:17.715 here, because when I add up the amortized cost of my operations, 01:22:17.715 --> 01:22:20.480 it's an upper bound on the true costs. 01:22:20.480 --> 01:22:24.291 OK, so that's why I say, one of the beauties of this 01:22:24.291 --> 01:22:27.058 amortized analysis, and in particular, 01:22:27.058 --> 01:22:31.692 being able to assign different costs to different operations is 01:22:31.692 --> 01:22:37.000 I can just add them up and I get my worst-case costs. 01:22:37.000 --> 01:22:40.565 So this is already V log V. There are a couple other 01:22:40.565 --> 01:22:43.012 algorithms just before I let you go. 01:22:43.012 --> 01:22:47.066 Kruskal's Algorithm in the book uses another amortized data 01:22:47.066 --> 01:22:50.282 structure called a disjoint set data structure, 01:22:50.282 --> 01:22:53.498 which also runs in E log V, that is, this time: 01:22:53.498 --> 01:22:56.574 runs in this time, the same as using a binary 01:22:56.574 --> 01:23:00.000 heap. So, I'll refer you to the book. 01:23:00.000 --> 01:23:04.934 The best algorithm to date with this problem is done by our own 01:23:04.934 --> 01:23:09.233 David Karger on the faculty here with one of our former 01:23:09.233 --> 01:23:12.975 graduates, Phil Kline, who is now a professor at 01:23:12.975 --> 01:23:17.353 Brown, and Robert Tarjan, who is sort of like the master 01:23:17.353 --> 01:23:22.368 of all data structures who was a professor at Princeton in 1993. 01:23:22.368 --> 01:23:26.189 OK, it's a randomized algorithm, and it gives you 01:23:26.189 --> 01:23:32.000 order V plus E expected time. OK, so that's the best to date. 01:23:32.000 --> 01:23:36.300 It's still open as to whether there is a deterministic, 01:23:36.300 --> 01:23:40.679 there is worst-case bound, whether there is a worst-case 01:23:40.679 --> 01:23:45.059 bound that is linear time. OK, but there is a randomized 01:23:45.059 --> 01:23:47.369 to linear time, and otherwise, 01:23:47.369 --> 01:23:51.509 this is essentially the best bound without additional 01:23:51.509 --> 01:23:54.058 assumptions. OK, very cool stuff. 01:23:54.058 --> 01:23:58.675 Next, we're going to see a lot of these ideas of greedy and 01:23:58.675 --> 01:24:01.000 dynamic programming in practice.