WEBVTT 00:00:00.050 --> 00:00:01.770 The following content is provided 00:00:01.770 --> 00:00:04.010 under a Creative Commons license. 00:00:04.010 --> 00:00:06.860 Your support will help MIT OpenCourseWare continue 00:00:06.860 --> 00:00:10.720 to offer high quality educational resources for free. 00:00:10.720 --> 00:00:13.320 To make a donation or view additional materials 00:00:13.320 --> 00:00:17.207 from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:17.207 --> 00:00:17.832 at ocw.mit.edu. 00:00:22.120 --> 00:00:25.770 PROFESSOR: [INAUDIBLE] audio. 00:00:25.770 --> 00:00:32.310 We're going to have a series of lectures on shortest paths. 00:00:32.310 --> 00:00:36.380 And one of the big differences between this module 00:00:36.380 --> 00:00:39.940 and the previous one, at least from a mathematical standpoint, 00:00:39.940 --> 00:00:41.460 is that we're going to be looking 00:00:41.460 --> 00:00:44.780 at graphs that have weights on their edges. 00:00:44.780 --> 00:00:47.690 So when Eric talked about depth first search and breadth 00:00:47.690 --> 00:00:50.140 first search in the last couple of lectures, 00:00:50.140 --> 00:00:54.160 we had directed graphs and undirected graphs. 00:00:54.160 --> 00:00:57.270 But we didn't really have attributes on the edges. 00:00:57.270 --> 00:01:00.690 In particular, you have a much richer class 00:01:00.690 --> 00:01:03.190 of problems and applications if you 00:01:03.190 --> 00:01:07.700 allow for weights on graph edges. 00:01:07.700 --> 00:01:13.020 And these weights can be integers. 00:01:13.020 --> 00:01:15.270 They could be real numbers, irrationals. 00:01:15.270 --> 00:01:17.560 They could be negative, what have you. 00:01:17.560 --> 00:01:22.030 And for different classes of graphs 00:01:22.030 --> 00:01:25.790 and different restrictions on weights, 00:01:25.790 --> 00:01:29.460 there's really a set of shortest path algorithms, 00:01:29.460 --> 00:01:33.150 that we'll look at in the next few lectures, which 00:01:33.150 --> 00:01:37.050 are kind of optimized for a particular application. 00:01:37.050 --> 00:01:41.900 So we won't do specific algorithms today. 00:01:41.900 --> 00:01:43.770 But we'll set up the problem. 00:01:43.770 --> 00:01:46.050 We'll talk about the general approach 00:01:46.050 --> 00:01:48.970 that most shortest path algorithms take 00:01:48.970 --> 00:01:52.010 to solve a particular instance of a problem. 00:01:52.010 --> 00:01:56.810 And then we'll close with talking 00:01:56.810 --> 00:01:59.390 about a particular property that's pretty important. 00:01:59.390 --> 00:02:03.290 That's called the optimum or optimal substructure property. 00:02:03.290 --> 00:02:06.904 That is a technique that most shortest path algorithms, 00:02:06.904 --> 00:02:08.570 or actually all shortest path algorithms 00:02:08.570 --> 00:02:14.690 use to get efficient complexity. 00:02:14.690 --> 00:02:18.890 So asymptotic complexity is important, obviously. 00:02:18.890 --> 00:02:22.580 And we're always looking for the best algorithm with the best 00:02:22.580 --> 00:02:23.930 asymptotic complexity. 00:02:23.930 --> 00:02:26.710 And optimal substructure is a hammer 00:02:26.710 --> 00:02:30.990 that we're going to use to get that. 00:02:30.990 --> 00:02:35.850 So the canonical motivation, of course, for shortest paths 00:02:35.850 --> 00:02:38.010 is-- now, if you want to steal, or I 00:02:38.010 --> 00:02:40.640 guess borrow, a cannon from Caltech 00:02:40.640 --> 00:02:43.860 and bring it over to MIT, then you 00:02:43.860 --> 00:02:52.510 want the fastest way of getting here with your illegal goods. 00:02:52.510 --> 00:02:56.230 And you want to find the shortest way or the fastest 00:02:56.230 --> 00:02:59.080 way of getting from one location to another. 00:02:59.080 --> 00:03:03.500 So Google Maps go from point A to point 00:03:03.500 --> 00:03:09.210 B. That's a classic application of the shortest path problem. 00:03:09.210 --> 00:03:13.000 In this case, you could imagine that distance 00:03:13.000 --> 00:03:16.110 would be something that would be a very simple metric that you 00:03:16.110 --> 00:03:19.120 could use for the weights on the edges. 00:03:19.120 --> 00:03:23.360 So for this entire module, we're going 00:03:23.360 --> 00:03:31.390 to be looking at a graph G V, E, W. 00:03:31.390 --> 00:03:32.660 And you know what V and E are. 00:03:32.660 --> 00:03:38.360 They're the vertices and the edges. 00:03:38.360 --> 00:03:47.020 And W is a weight function that maps edges to weights. 00:03:47.020 --> 00:03:50.510 And so we're adding that in here. 00:03:50.510 --> 00:03:59.140 And so W would be E to R. So the set of real numbers. 00:03:59.140 --> 00:04:02.990 We're going to be looking at two different algorithms 00:04:02.990 --> 00:04:04.630 in subsequent lectures. 00:04:04.630 --> 00:04:07.085 And you'll implement one of them in your problem set. 00:04:10.380 --> 00:04:15.030 The simpler algorithm, which we'll look at first, 00:04:15.030 --> 00:04:18.510 is called Dijkstra, after Edsger Dijkstra, who 00:04:18.510 --> 00:04:21.970 did some similar work in concurrent programming, 00:04:21.970 --> 00:04:23.320 won the Turing Award. 00:04:23.320 --> 00:04:26.790 But on the side invented this cool algorithm-- 00:04:26.790 --> 00:04:30.330 or at least gets credit for it-- called Dijkstra's algorithm 00:04:30.330 --> 00:04:36.530 that assumes non-negative weight edges. 00:04:36.530 --> 00:04:38.030 So I should really say non-negative. 00:04:38.030 --> 00:04:40.690 So read that as non-negative. 00:04:40.690 --> 00:04:51.510 And that has a complexity of order V log V plus E. 00:04:51.510 --> 00:04:52.290 All right. 00:04:52.290 --> 00:04:58.330 So this is practically linear time. 00:04:58.330 --> 00:05:01.910 And typically, you're going to be dominated in many cases 00:05:01.910 --> 00:05:10.060 by E. In general, if you talk about a simple graph, what's 00:05:10.060 --> 00:05:13.690 the asymptotic relationship between E and V? 00:05:19.430 --> 00:05:21.040 Can you relate E to V? 00:05:21.040 --> 00:05:22.874 Can you give me a bound? 00:05:22.874 --> 00:05:23.942 AUDIENCE: V squared. 00:05:23.942 --> 00:05:24.650 PROFESSOR: Sorry? 00:05:24.650 --> 00:05:25.970 V squared, thanks. 00:05:25.970 --> 00:05:27.700 That's good. 00:05:27.700 --> 00:05:33.280 So you can think of E as being order V square. 00:05:33.280 --> 00:05:36.470 And you can certainly have-- that's worth recursion. 00:05:41.310 --> 00:05:44.540 So now, you can kind of imagine a complete graph. 00:05:44.540 --> 00:05:46.390 And a complete graph is something 00:05:46.390 --> 00:05:52.980 that has an edge between each pair of vertices. 00:05:52.980 --> 00:05:57.710 And that's where you'll get E being k w squared. 00:05:57.710 --> 00:05:59.370 So when you say simple graph, you're 00:05:59.370 --> 00:06:03.630 saying you have at most one edge between any pair of vertices. 00:06:03.630 --> 00:06:05.160 A multigraph is something that could 00:06:05.160 --> 00:06:08.140 have multiple edges between pairs of vertices. 00:06:08.140 --> 00:06:10.480 We won't really be talking about multigraphs 00:06:10.480 --> 00:06:12.840 in this sequence of lectures. 00:06:12.840 --> 00:06:19.420 But something to think about or keep in the back of your mind 00:06:19.420 --> 00:06:21.130 as we go through these algorithms. 00:06:21.130 --> 00:06:25.810 And so the dominating factor here, and in many cases really, 00:06:25.810 --> 00:06:30.390 is E. And Dijkstra is a nice algorithm, because it's 00:06:30.390 --> 00:06:34.080 linear in the number of edges. 00:06:34.080 --> 00:06:35.900 So that's Dijkstra. 00:06:35.900 --> 00:06:38.090 And that's the first of the algorithms 00:06:38.090 --> 00:06:40.750 that we'll look at next time. 00:06:40.750 --> 00:06:44.070 But we'll see the general structure of Dijkstra today. 00:06:44.070 --> 00:06:50.880 And then there's the Bellman-Ford algorithm 00:06:50.880 --> 00:06:56.890 that works on positive and negative edges, weight edges. 00:06:56.890 --> 00:07:03.630 And this has a complexity order V E. 00:07:03.630 --> 00:07:06.240 So you could imagine a particular implementation 00:07:06.240 --> 00:07:10.060 of Bellman-Ford running in order V cubed time. 00:07:10.060 --> 00:07:12.360 Because E could be V square. 00:07:12.360 --> 00:07:14.610 And you've got this additional E factor. 00:07:14.610 --> 00:07:17.800 So it's order V cubed versus order V log V. 00:07:17.800 --> 00:07:21.000 So when you have a chance, use Dijkstra. 00:07:21.000 --> 00:07:25.670 When you're stuck, you'd want to do Bellman-Ford, 00:07:25.670 --> 00:07:27.665 because you have these negative weight edges. 00:07:27.665 --> 00:07:29.790 And one of the challenges in negative weight edges, 00:07:29.790 --> 00:07:32.550 and I'll say a little bit more as we go along, 00:07:32.550 --> 00:07:37.050 is that you end up having to have 00:07:37.050 --> 00:07:41.720 to find cycles that are of a negative weight, 00:07:41.720 --> 00:07:44.840 because they kind of throw off your shortest path algorithm 00:07:44.840 --> 00:07:49.590 if you were just assuming that shortest path lengths are only 00:07:49.590 --> 00:07:51.180 going to decrease. 00:07:51.180 --> 00:07:55.460 But when you have negative weights, you might take a step 00:07:55.460 --> 00:07:57.900 and the overall weight might decrease. 00:07:57.900 --> 00:08:01.150 So it's kind of a longer path in terms of the number of edges. 00:08:01.150 --> 00:08:02.600 But the weight is smaller. 00:08:02.600 --> 00:08:05.970 And that kind of makes the algorithm more complicated. 00:08:05.970 --> 00:08:07.362 And it has to do more work. 00:08:07.362 --> 00:08:09.820 So that's really why there's a difference between these two 00:08:09.820 --> 00:08:10.760 complexities. 00:08:10.760 --> 00:08:14.200 And I guarantee you, you'll understand this much better 00:08:14.200 --> 00:08:16.700 after we're done with the lectures on Dijkstra 00:08:16.700 --> 00:08:20.019 and the lectures on Bellman-Ford. 00:08:20.019 --> 00:08:21.560 So that's the set up for the problem. 00:08:21.560 --> 00:08:24.030 That's what we're going to be looking at. 00:08:24.030 --> 00:08:27.830 Let's look at a couple more definitions 00:08:27.830 --> 00:08:34.549 beyond what I have here with respect to just the notation. 00:08:34.549 --> 00:08:46.883 And you can think of path p as a sequence of vertices-- V0, V1, 00:08:46.883 --> 00:08:49.880 et cetera, to Vk. 00:08:49.880 --> 00:08:57.650 And this is the path if Vi, Vi plus 1 00:08:57.650 --> 00:09:03.320 belongs to E for 0 less than or equal to i less than 00:09:03.320 --> 00:09:05.210 or equal to k. 00:09:05.210 --> 00:09:08.290 So a path is a sequence of edges. 00:09:08.290 --> 00:09:10.670 And each of those edges has to be in the graph, 00:09:10.670 --> 00:09:15.530 has to be in the set of edges E. 00:09:15.530 --> 00:09:20.090 And W of p, which is the weight of the path, 00:09:20.090 --> 00:09:22.860 we know that by the weight of edges, those are easy. 00:09:22.860 --> 00:09:24.970 They're given by the W function. 00:09:24.970 --> 00:09:27.840 The weight of the path is simply the summation 00:09:27.840 --> 00:09:29.620 of the weights of the edges. 00:09:33.100 --> 00:09:33.620 All right. 00:09:33.620 --> 00:09:35.610 So fairly obvious definitions. 00:09:35.610 --> 00:09:37.730 But obviously, we have to get these right in order 00:09:37.730 --> 00:09:40.350 to actually solve the problem correctly. 00:09:43.030 --> 00:09:49.760 And the shortest path problem is, as you can imagine, 00:09:49.760 --> 00:10:00.050 something that tries to find a path p that has minimum weight. 00:10:00.050 --> 00:10:03.840 So in general, you have some set up for the problem. 00:10:03.840 --> 00:10:07.350 But it comes down to find p with-- 00:10:13.260 --> 00:10:16.599 And there are many, many possible paths. 00:10:16.599 --> 00:10:18.640 You have to understand that there are potentially 00:10:18.640 --> 00:10:22.082 an exponential number of paths in the graphs 00:10:22.082 --> 00:10:23.040 that we would consider. 00:10:23.040 --> 00:10:26.610 And here's a real simple example where 00:10:26.610 --> 00:10:33.360 you would have an exponential number of paths. 00:10:33.360 --> 00:10:38.540 And we'll come back to this example later in the lecture. 00:10:38.540 --> 00:10:41.390 But let's assume that all the directions go this way. 00:10:44.050 --> 00:10:46.500 And it's a directed graph. 00:10:46.500 --> 00:10:49.245 Well, you could have the path that goes all the way here. 00:10:49.245 --> 00:10:52.220 But you could have the path that goes on top 00:10:52.220 --> 00:10:53.700 and all the way this way. 00:10:53.700 --> 00:10:57.510 You have basically two choices on getting to this vertex. 00:10:57.510 --> 00:10:59.190 Then you've got, given the two ways 00:10:59.190 --> 00:11:01.320 you have of getting to this vertex. 00:11:01.320 --> 00:11:03.580 You've got four ways of getting here. 00:11:03.580 --> 00:11:06.240 And then, you have eight ways of getting there. 00:11:06.240 --> 00:11:07.870 So on and so forth. 00:11:07.870 --> 00:11:10.750 So there's an exponential number of paths potentially. 00:11:10.750 --> 00:11:12.970 The other thing that's interesting here, 00:11:12.970 --> 00:11:15.900 which is important in terms of this complexity 00:11:15.900 --> 00:11:19.580 is, what's interesting about what you see here 00:11:19.580 --> 00:11:23.380 with respect to the complexity and what you see here. 00:11:23.380 --> 00:11:25.590 Anybody want to point that out? 00:11:25.590 --> 00:11:30.910 So I have this complexity here and order VE out there. 00:11:30.910 --> 00:11:34.762 What's an interesting observation 00:11:34.762 --> 00:11:37.095 if you look at this board here and the two complexities? 00:11:40.630 --> 00:11:41.810 Anybody? 00:11:41.810 --> 00:11:42.580 Yeah, back there. 00:11:42.580 --> 00:11:44.330 AUDIENCE: It's not a function of weight. 00:11:44.330 --> 00:11:45.690 PROFESSOR: It's not a function of weight. 00:11:45.690 --> 00:11:46.190 Great. 00:11:46.190 --> 00:11:48.650 That's definitely worth recursion. 00:11:48.650 --> 00:11:50.180 And I'll let you throw this one. 00:11:53.120 --> 00:11:55.990 All the way back there, all right? 00:11:55.990 --> 00:11:57.900 Right. 00:11:57.900 --> 00:11:58.580 All right, good. 00:11:58.580 --> 00:11:59.537 Good, good. 00:11:59.537 --> 00:12:00.120 That was good. 00:12:00.120 --> 00:12:01.450 That was better than what I could do. 00:12:01.450 --> 00:12:02.180 No, not really. 00:12:02.180 --> 00:12:09.390 But I would've been right in his hands right there. 00:12:09.390 --> 00:12:12.940 Anyway, so that's a great observation, actually. 00:12:12.940 --> 00:12:14.940 And I should have pointed it out right up front. 00:12:14.940 --> 00:12:17.880 But I'm glad I got it out of you. 00:12:17.880 --> 00:12:22.160 W doesn't exist in the complexity. 00:12:22.160 --> 00:12:23.650 This is pretty important. 00:12:23.650 --> 00:12:26.410 W could be a large number. 00:12:26.410 --> 00:12:29.030 I mean, it could be 2 raised to 64. 00:12:29.030 --> 00:12:34.640 The fact is that there's only E square different values 00:12:34.640 --> 00:12:37.170 possible for a weight, right. 00:12:37.170 --> 00:12:38.730 I mean, roughly speaking. 00:12:38.730 --> 00:12:41.390 If you have a complete graph, it's a simple graph, 00:12:41.390 --> 00:12:45.160 there's order E square possible weights. 00:12:45.160 --> 00:12:49.830 But the range of the weights could be exponential. 00:12:49.830 --> 00:12:54.480 I could have an edge weight of 0.0001 00:12:54.480 --> 00:12:58.090 and a different edge weight of 10 raised to 98. 00:12:58.090 --> 00:13:01.190 There's nothing that's stopping me 00:13:01.190 --> 00:13:04.040 from doing that or putting a specification like that. 00:13:04.040 --> 00:13:06.830 But the nice thing about Dijkstra, and Bellman-Ford, 00:13:06.830 --> 00:13:09.030 and virtually all of the algorithms 00:13:09.030 --> 00:13:10.960 that are useful in practice is that they 00:13:10.960 --> 00:13:15.450 don't depend on the dynamic range of the weights. 00:13:15.450 --> 00:13:17.720 And so keep that in mind as you think 00:13:17.720 --> 00:13:19.622 of shortest path algorithms. 00:13:19.622 --> 00:13:22.080 And we'll talk a little bit about this in section tomorrow, 00:13:22.080 --> 00:13:26.390 or the TAs will, as to why breadth first search and depth 00:13:26.390 --> 00:13:30.380 first search aren't directly applicable to the shortest path 00:13:30.380 --> 00:13:31.610 problem. 00:13:31.610 --> 00:13:36.480 And the hint really is the dynamic range of the weights. 00:13:36.480 --> 00:13:37.770 So keep that in mind. 00:13:37.770 --> 00:13:40.970 So a couple things why this is an interesting algorithm, 00:13:40.970 --> 00:13:44.310 or interesting problem to solve, and harder than the problems 00:13:44.310 --> 00:13:47.070 we've looked at so far like sorting and search, 00:13:47.070 --> 00:13:49.760 is that you have an exponential number of paths. 00:13:49.760 --> 00:13:51.540 And then the dynamic range of the weights 00:13:51.540 --> 00:13:53.700 can be very, very large. 00:13:53.700 --> 00:13:55.540 And it's not linear by any means. 00:13:55.540 --> 00:13:56.040 All right. 00:13:56.040 --> 00:13:59.020 So these algorithms are going to have to have some smarts. 00:13:59.020 --> 00:14:00.810 And the optimal substructure property 00:14:00.810 --> 00:14:03.670 that we'll look at towards the end of today's lecture 00:14:03.670 --> 00:14:06.070 will give you a sense of how these algorithms actually 00:14:06.070 --> 00:14:08.980 work in basically linear time. 00:14:08.980 --> 00:14:12.310 Or VE, you could think of that as being 00:14:12.310 --> 00:14:17.330 cubic time in terms of the vertices. 00:14:17.330 --> 00:14:19.530 So keep that in mind. 00:14:19.530 --> 00:14:23.470 Let's talk a little bit more about weighted graphs. 00:14:23.470 --> 00:14:24.990 I want a little bit more notation. 00:14:31.940 --> 00:14:38.170 And what I have is V0 using path p to Vk. 00:14:38.170 --> 00:14:40.470 So I'm going to write that to say that there's 00:14:40.470 --> 00:14:43.810 a particular path of V0 to Vk. 00:14:43.810 --> 00:14:48.900 Sometimes I'm searching for the path p with a minimum weight. 00:14:48.900 --> 00:14:51.190 And that's how I'm going to represent that. 00:14:51.190 --> 00:14:56.120 V0, which is a single vertex path, 00:14:56.120 --> 00:15:00.580 is the path from V0 to V0. 00:15:00.580 --> 00:15:03.680 So it's really a 0 length path. 00:15:03.680 --> 00:15:07.630 And it has weight 0. 00:15:07.630 --> 00:15:09.410 So that's one condition. 00:15:09.410 --> 00:15:12.670 The other condition that we need to look at, 00:15:12.670 --> 00:15:18.120 which is the other case, is what if there isn't a path? 00:15:18.120 --> 00:15:20.820 So I want to put those two things together, 00:15:20.820 --> 00:15:23.610 the two extremes, and of course all of the cases 00:15:23.610 --> 00:15:29.810 in between, in this definition of the shortest pathway. 00:15:29.810 --> 00:15:33.990 And so I'm going to talk about the shortest path 00:15:33.990 --> 00:15:38.990 value of the weight of the shortest path between u and v 00:15:38.990 --> 00:15:43.610 as delta, u, v. And my goal is to find delta. 00:15:43.610 --> 00:15:45.560 It's also to find the path. 00:15:45.560 --> 00:15:48.890 It doesn't help you very much if you 00:15:48.890 --> 00:15:53.270 know that there's a way of getting from here to Lexington 00:15:53.270 --> 00:15:56.830 within 14 miles if you don't know what that path is, right. 00:15:56.830 --> 00:15:59.690 So that's one aspect of it, which 00:15:59.690 --> 00:16:01.110 is you want to get the weight. 00:16:01.110 --> 00:16:02.970 But you want to get the path as well. 00:16:02.970 --> 00:16:06.230 And these algorithms will do that for you. 00:16:06.230 --> 00:16:10.150 And in particular, what we want is 00:16:10.150 --> 00:16:24.100 delta u, v to be the minimum over all the paths W p, 00:16:24.100 --> 00:16:30.730 such that p is in fact the path from u to v. 00:16:30.730 --> 00:16:40.490 And this is the case where if there exists any such path, 00:16:40.490 --> 00:16:44.920 and the last thing is you want this to be infinity, 00:16:44.920 --> 00:16:47.185 the weight to be infinity otherwise. 00:16:51.290 --> 00:16:56.490 So if you're only talking about roads going from here to Tokyo, 00:16:56.490 --> 00:16:59.540 should have length infinity. 00:16:59.540 --> 00:17:04.099 A little matter of the Pacific Ocean in between. 00:17:04.099 --> 00:17:08.970 So that's the set up in terms of the numbers 00:17:08.970 --> 00:17:11.040 that we want to see. 00:17:11.040 --> 00:17:13.800 If you're starting from a particular point, 00:17:13.800 --> 00:17:18.720 you can think of the shortest path length from your source 00:17:18.720 --> 00:17:20.240 as being a 0. 00:17:20.240 --> 00:17:22.150 Initially, everything is infinity 00:17:22.150 --> 00:17:24.790 because you haven't found any paths yet. 00:17:24.790 --> 00:17:28.450 And what you're going to do is try and reduce these infinities 00:17:28.450 --> 00:17:31.970 down for all of the vertices that 00:17:31.970 --> 00:17:34.040 are reachable from the source vertex. 00:17:34.040 --> 00:17:37.580 And it's quite possible that you may be given a graph where 00:17:37.580 --> 00:17:40.740 a particular vertices, or in your set 00:17:40.740 --> 00:17:44.030 V, that can't be reached from the particular source 00:17:44.030 --> 00:17:45.290 that you started with. 00:17:45.290 --> 00:17:50.650 And for those vertices, you're going to have your delta u, 00:17:50.650 --> 00:17:56.650 v. If v is unreachable from you, it will stay at infinity. 00:17:56.650 --> 00:17:58.390 So let's look at an example. 00:18:03.710 --> 00:18:12.980 Let's take-- it's going to be an iterative process here 00:18:12.980 --> 00:18:18.150 of finding the shortest paths. 00:18:18.150 --> 00:18:22.100 And so let's take an example that 00:18:22.100 --> 00:18:27.770 corresponds to a fairly complex graph, 00:18:27.770 --> 00:18:31.330 or at least a nontrivial one, where that's my source vertex. 00:18:31.330 --> 00:18:35.420 And I've labeled these other vertices A through F. 00:18:35.420 --> 00:18:36.725 And I have a bunch of edges. 00:18:44.780 --> 00:18:45.280 5. 00:19:06.130 --> 00:19:07.140 I got one more here. 00:19:10.230 --> 00:19:11.640 So that's what's given to me. 00:19:11.640 --> 00:19:18.720 And I want to find delta S plugged in for u. 00:19:18.720 --> 00:19:25.390 And A, B, D, et cetera plugged in for V for this graph. 00:19:25.390 --> 00:19:30.850 And let's just do this manually if you will. 00:19:30.850 --> 00:19:35.111 And just trying to do some kind of breadth first search. 00:19:35.111 --> 00:19:36.610 And we do know breadth first search. 00:19:36.610 --> 00:19:38.060 We know depth first search. 00:19:38.060 --> 00:19:45.580 You can imagine trying to use those notions to try and find 00:19:45.580 --> 00:19:47.020 the shortest paths here. 00:19:47.020 --> 00:19:48.829 So now we have to prove afterwards 00:19:48.829 --> 00:19:51.370 when we are done that these are, in fact, the shortest paths. 00:19:51.370 --> 00:19:52.720 And that's the hard part of it. 00:19:52.720 --> 00:19:56.860 But we can certainly try and fill in some numbers associated 00:19:56.860 --> 00:19:58.620 with paths that we do know about. 00:19:58.620 --> 00:20:02.635 So I'm going to say that the numbers that 00:20:02.635 --> 00:20:08.210 are inside each of these vertices-- d of u 00:20:08.210 --> 00:20:11.365 is the current weight. 00:20:14.090 --> 00:20:17.890 And so initially, I'm going to start with D of S being 0, 00:20:17.890 --> 00:20:19.699 because that's a source. 00:20:19.699 --> 00:20:21.240 And all of these other ones are going 00:20:21.240 --> 00:20:23.073 to have-- I'm not going to write this down-- 00:20:23.073 --> 00:20:26.940 but they're going to have infinity for their D of Vs. 00:20:26.940 --> 00:20:28.350 So D of A is infinity. 00:20:28.350 --> 00:20:31.210 Do of B is infinity, et cetera. 00:20:31.210 --> 00:20:34.500 And what I want to do is decrease this D number 00:20:34.500 --> 00:20:38.080 to the point where I'm confident that all of the D numbers 00:20:38.080 --> 00:20:41.230 that are inside these vertices, these are the current weights, 00:20:41.230 --> 00:20:44.310 or end up being the delta numbers. 00:20:44.310 --> 00:20:48.520 So my algorithm is done when my d numbers shrink down. 00:20:48.520 --> 00:20:52.810 And I got the delta values, the correct delta values. 00:20:52.810 --> 00:20:57.500 But if I wanted to do this, sort of a seat of the pants 00:20:57.500 --> 00:21:00.920 approach, just go off and try and iteratively 00:21:00.920 --> 00:21:03.860 reduce these numbers, you say, well, this one was infinity. 00:21:03.860 --> 00:21:08.690 But clearly, if I start from S and I follow the edges in S, 00:21:08.690 --> 00:21:11.980 I'm going to be able to mark this as a one. 00:21:11.980 --> 00:21:15.630 And similarly here, I'm going to be able to mark this as a 2. 00:21:15.630 --> 00:21:19.860 Now, I could arbitrarily pick this one 00:21:19.860 --> 00:21:23.870 here and this A vertex, and then start 00:21:23.870 --> 00:21:28.090 looking at the edges that emanate from the A vertex. 00:21:28.090 --> 00:21:36.470 And I could go off and mark this as 6, for example. 00:21:36.470 --> 00:21:42.380 And if I start from here, I'd mark this as 3. 00:21:42.380 --> 00:21:48.570 Now, is it in fact true that 6 equals 00:21:48.570 --> 00:21:54.670 delta S comma C equals 6? 00:21:54.670 --> 00:21:56.100 No. 00:21:56.100 --> 00:22:00.550 What is in fact-- is there a better way of getting to C? 00:22:00.550 --> 00:22:02.780 And what is the weight of that? 00:22:02.780 --> 00:22:06.594 What vertex do I have to go through? 00:22:06.594 --> 00:22:19.180 I mean, one way is to go from S to B to D to C, right? 00:22:19.180 --> 00:22:21.520 And that would give me 5 right? 00:22:21.520 --> 00:22:23.280 So that's 5. 00:22:23.280 --> 00:22:26.530 Can I do better than 5? 00:22:26.530 --> 00:22:27.770 Not in this graph. 00:22:27.770 --> 00:22:28.930 OK 00:22:28.930 --> 00:22:36.680 So it's not the case that the shortest length path gave you 00:22:36.680 --> 00:22:40.980 the smallest weight. 00:22:40.980 --> 00:22:42.600 I mean, that was one example of that. 00:22:42.600 --> 00:22:44.600 And I can go on an and bore you with filling 00:22:44.600 --> 00:22:45.930 in all of these numbers. 00:22:45.930 --> 00:22:47.980 But you can do that on your own. 00:22:47.980 --> 00:22:53.020 And it's really not particularly edifying to do that. 00:22:53.020 --> 00:22:55.660 But you get a sense of what you need 00:22:55.660 --> 00:22:59.430 to be able to do in order to converge on the delta. 00:22:59.430 --> 00:23:01.600 And it might take some doing. 00:23:01.600 --> 00:23:06.002 Because you have to somehow enumerate in an implicit way-- 00:23:06.002 --> 00:23:07.460 you can't do it in an explicit way, 00:23:07.460 --> 00:23:10.370 because then there'd be an exponential number of paths. 00:23:10.370 --> 00:23:12.670 But you'd have to implicitly enumerate 00:23:12.670 --> 00:23:16.480 all the different ways that you can possibly get to a vertex 00:23:16.480 --> 00:23:21.790 and discover the shortest path through that process, 00:23:21.790 --> 00:23:22.490 all right. 00:23:22.490 --> 00:23:25.080 And so we have to be able to do that in these shortest path 00:23:25.080 --> 00:23:26.510 algorithms. 00:23:26.510 --> 00:23:31.290 And this is a simple graph that has positive weights, 00:23:31.290 --> 00:23:34.160 non-negative weights with edges. 00:23:34.160 --> 00:23:38.599 It gets more complicated when you have negative weights. 00:23:38.599 --> 00:23:40.640 But before I get to that, there's one other thing 00:23:40.640 --> 00:23:43.490 that I want to talk about here with respect 00:23:43.490 --> 00:23:46.900 to discovering the actual path. 00:23:46.900 --> 00:23:49.935 So what we did here was we had delta u, 00:23:49.935 --> 00:23:55.890 v that corresponded to the weight of the shortest path. 00:23:55.890 --> 00:24:00.080 But if you want the path itself, we 00:24:00.080 --> 00:24:04.090 need to have a way of finding the sequence of vertices that 00:24:04.090 --> 00:24:08.310 corresponds to the minimum weight path. 00:24:08.310 --> 00:24:10.240 And in particular, we're going to have 00:24:10.240 --> 00:24:16.290 to define what we call the predecessor relationship. 00:24:16.290 --> 00:24:25.270 And so what I have is d of V is the value 00:24:25.270 --> 00:24:31.230 inside the circle, which is the current weight. 00:24:34.890 --> 00:24:38.400 And as d is something you're very interested in, 00:24:38.400 --> 00:24:40.270 eventually you want it to go to delta. 00:24:40.270 --> 00:24:42.330 The other thing that you're very interested in-- 00:24:42.330 --> 00:24:45.950 and this is really a fairly straightforward data structure 00:24:45.950 --> 00:24:50.050 corresponding to just the d number and this predecessor 00:24:50.050 --> 00:24:50.910 number. 00:24:50.910 --> 00:25:06.820 And pi of V is the predecessor vertex on the best path to V. 00:25:06.820 --> 00:25:10.070 And you said, pi of S equals NIL. 00:25:13.490 --> 00:25:15.950 And then you can think of this as this is eventually 00:25:15.950 --> 00:25:20.600 what we want, and this gets modified as well. 00:25:20.600 --> 00:25:24.720 So right now, when you're working and trying 00:25:24.720 --> 00:25:27.320 to find the path, you have some particular path 00:25:27.320 --> 00:25:29.500 that happens to be the current best path. 00:25:29.500 --> 00:25:31.930 And that's a sequence of vertices 00:25:31.930 --> 00:25:34.550 that you can get by following the predecessors. 00:25:34.550 --> 00:25:40.250 So once you're at a particular vertex E, you say all right, 00:25:40.250 --> 00:25:46.700 right now I can look at pi of E. And if that points me to C, 00:25:46.700 --> 00:25:49.190 then that's good. 00:25:49.190 --> 00:25:52.240 I'm going to look at pi of C. And that might point me 00:25:52.240 --> 00:25:54.320 to A, and so on and so forth. 00:25:54.320 --> 00:25:57.560 In this particular instance, pi of E 00:25:57.560 --> 00:26:02.470 is going to, when you're finally done, is going to point to A. 00:26:02.470 --> 00:26:06.150 And pi of A is going to point to S, all right, because that's 00:26:06.150 --> 00:26:10.010 the path that is the best path is this one. 00:26:10.010 --> 00:26:12.660 Like so and like that. 00:26:12.660 --> 00:26:14.380 And so those are the two data structures 00:26:14.380 --> 00:26:18.200 you need to keep in mind that you need to iterate on, 00:26:18.200 --> 00:26:24.970 this predecessor relationship and the current distance. 00:26:24.970 --> 00:26:29.340 And then this ends up being delta. 00:26:29.340 --> 00:26:31.370 You're done. 00:26:31.370 --> 00:26:33.401 And at that point, your predecessor relationship 00:26:33.401 --> 00:26:33.900 is correct. 00:26:37.140 --> 00:26:39.842 So that's the set up. 00:26:39.842 --> 00:26:41.800 The last complication I want to talk about here 00:26:41.800 --> 00:26:44.284 is negative weights. 00:26:44.284 --> 00:26:46.450 And it's, I think, appropriate to talk about it when 00:26:46.450 --> 00:26:48.370 we have Bellman-Ford up here. 00:26:53.467 --> 00:26:55.050 Which is really the general algorithm. 00:27:00.410 --> 00:27:09.190 So let's talk about-- so the first question is why. 00:27:09.190 --> 00:27:11.780 Why do these things exist, other than making 00:27:11.780 --> 00:27:14.380 our lives more difficult? 00:27:14.380 --> 00:27:15.780 So give me an example. 00:27:15.780 --> 00:27:18.855 What is the motivation for a graph with negative weights? 00:27:22.330 --> 00:27:23.990 I mean, I really would like to know. 00:27:23.990 --> 00:27:26.450 The best motivation is definitely worth recursion. 00:27:26.450 --> 00:27:28.980 Then I can use it next time. 00:27:28.980 --> 00:27:29.863 Yeah, go ahead. 00:27:29.863 --> 00:27:35.286 AUDIENCE: I'm just thinking like if your goal, 00:27:35.286 --> 00:27:37.258 if your goal [INAUDIBLE]. 00:27:40.709 --> 00:27:42.188 And some of them cost too much. 00:27:42.188 --> 00:27:45.392 Some of them get you money, and you 00:27:45.392 --> 00:27:49.090 want to know what-- you're trying to find [INAUDIBLE]. 00:27:55.499 --> 00:27:56.485 PROFESSOR: Sure. 00:27:56.485 --> 00:27:58.465 Yeah, I mean, I think that's a good motivation. 00:28:02.750 --> 00:28:04.980 I think driving, when you think about distances 00:28:04.980 --> 00:28:07.530 and so on, there's no notion of a negative distance, 00:28:07.530 --> 00:28:09.820 at least physically. 00:28:09.820 --> 00:28:13.652 But you can imagine that you could have a case where you're 00:28:13.652 --> 00:28:15.110 getting paid to drive or something, 00:28:15.110 --> 00:28:18.325 or it costs you to drive, and that would be one. 00:28:18.325 --> 00:28:18.950 Yeah, go ahead. 00:28:18.950 --> 00:28:20.366 AUDIENCE: It sounds like Monopoly. 00:28:20.366 --> 00:28:24.014 So the vertices are supposed to be [INAUDIBLE]. 00:28:24.014 --> 00:28:25.680 PROFESSOR: Oh, if you land on something, 00:28:25.680 --> 00:28:26.570 you have to pay rent. 00:28:26.570 --> 00:28:28.060 Or sometimes you land on something 00:28:28.060 --> 00:28:29.010 and you actually get money. 00:28:29.010 --> 00:28:29.926 AUDIENCE: [INAUDIBLE]. 00:28:32.792 --> 00:28:34.750 PROFESSOR: Takes you forward, backwards, right. 00:28:34.750 --> 00:28:35.458 Yeah go ahead. 00:28:35.458 --> 00:28:36.374 AUDIENCE: [INAUDIBLE]. 00:28:43.710 --> 00:28:46.670 PROFESSOR: So that is such an interesting notion. 00:28:46.670 --> 00:28:49.230 Sometimes you may want to go. 00:28:49.230 --> 00:28:52.080 And maybe in this case, you're saying 00:28:52.080 --> 00:28:55.490 it's better to take your distance metric 00:28:55.490 --> 00:28:59.940 and go further away in order to get the best way of getting 00:28:59.940 --> 00:29:01.190 there, or something like that. 00:29:01.190 --> 00:29:02.106 AUDIENCE: [INAUDIBLE]. 00:29:08.380 --> 00:29:09.160 PROFESSOR: Right. 00:29:09.160 --> 00:29:09.659 Sure. 00:29:09.659 --> 00:29:10.290 That'd be good. 00:29:10.290 --> 00:29:10.820 Right. 00:29:10.820 --> 00:29:12.103 Victor, you had your hand up. 00:29:12.103 --> 00:29:12.728 AUDIENCE: Yeah. 00:29:12.728 --> 00:29:15.223 I'm going to give [INAUDIBLE] on the highway, 00:29:15.223 --> 00:29:18.217 you can't [INAUDIBLE] distances [INAUDIBLE] negative. 00:29:18.217 --> 00:29:22.209 Well, if a government uses [INAUDIBLE] police 00:29:22.209 --> 00:29:26.201 to regulate traffic, then you might have a negative distance. 00:29:26.201 --> 00:29:28.696 Because obviously, you could go a certain way 00:29:28.696 --> 00:29:31.582 minus the [INAUDIBLE]. 00:29:31.582 --> 00:29:32.290 PROFESSOR: Right. 00:29:32.290 --> 00:29:35.120 Yeah, that's a good example. 00:29:35.120 --> 00:29:37.050 One of the things that we have to think about 00:29:37.050 --> 00:29:39.310 is-- and this is something that might come up, 00:29:39.310 --> 00:29:43.620 by the way, in a problem set or a quiz-- which is, is there 00:29:43.620 --> 00:29:48.540 a way of shifting these weights to make them all positive? 00:29:48.540 --> 00:29:52.250 So the examples we've talked about, not clear to me 00:29:52.250 --> 00:29:56.100 that in the particular settings that we talked about 00:29:56.100 --> 00:29:59.130 that you can somehow create the base 00:29:59.130 --> 00:30:03.120 case to be 0 rather than being negative. 00:30:03.120 --> 00:30:07.370 So it may not be possible in a particular scenario. 00:30:07.370 --> 00:30:09.630 But if you can do that-- and the reason 00:30:09.630 --> 00:30:12.200 I bring this up is if you can do that, suddenly 00:30:12.200 --> 00:30:14.760 instead of using an order V, E algorithm, 00:30:14.760 --> 00:30:18.120 if you can prove correctness of the final solution 00:30:18.120 --> 00:30:20.630 is exactly what you'd have gotten for the initial problem 00:30:20.630 --> 00:30:23.360 certification, you've gone from an order V, E algorithm 00:30:23.360 --> 00:30:24.894 to an order V log V algorithm. 00:30:24.894 --> 00:30:26.310 So that's a wonderful thing to do. 00:30:26.310 --> 00:30:28.160 So keep that in mind. 00:30:28.160 --> 00:30:30.720 Try and get rid of negative weight edges if you can 00:30:30.720 --> 00:30:32.730 without changing the problem certification. 00:30:32.730 --> 00:30:34.080 I saw a hand back there. 00:30:34.080 --> 00:30:35.056 AUDIENCE: Oh, no. 00:30:35.056 --> 00:30:37.008 I thought you were just asking the question, 00:30:37.008 --> 00:30:37.984 if we could do that? 00:30:37.984 --> 00:30:41.400 So I was just gettin ready to answer. 00:30:41.400 --> 00:30:47.540 PROFESSOR: OK, yeah, so that's something to keep in mind. 00:30:47.540 --> 00:30:49.956 One example that I think has come up here, which came up, 00:30:49.956 --> 00:30:51.330 I think, the last time I lectured 00:30:51.330 --> 00:30:54.620 was imagine that you're driving and there 00:30:54.620 --> 00:30:56.240 are all these advertisements. 00:30:56.240 --> 00:30:58.750 And you get paid to drive on a freeway. 00:30:58.750 --> 00:31:00.197 So the reverse toll. 00:31:00.197 --> 00:31:02.530 I mean, it's a reverse toll, because you get to go there 00:31:02.530 --> 00:31:04.940 and you have to see all these ads. 00:31:04.940 --> 00:31:09.200 And then I guess you drive pretty fast through those ads. 00:31:09.200 --> 00:31:11.380 But you have to go through. 00:31:11.380 --> 00:31:14.690 And so you get paid to go through those particular roads. 00:31:14.690 --> 00:31:16.810 And then what about social networks? 00:31:16.810 --> 00:31:19.550 I mean, there's liking people and disliking people. 00:31:19.550 --> 00:31:22.037 I mean, that sounds pretty-- that's negative and positive. 00:31:22.037 --> 00:31:23.870 One could imagine that social networks would 00:31:23.870 --> 00:31:26.190 have positive weights and negative weights. 00:31:26.190 --> 00:31:27.750 I'm surprised one of you-- I mean, 00:31:27.750 --> 00:31:29.250 I don't have an account on Facebook. 00:31:29.250 --> 00:31:32.240 But presumably, you guys do. 00:31:32.240 --> 00:31:36.154 So think of what's the-- yeah, that's right. 00:31:36.154 --> 00:31:37.570 Well, I'm not sure how this works. 00:31:37.570 --> 00:31:40.410 But you guys figure it out. 00:31:40.410 --> 00:31:41.240 So why? 00:31:41.240 --> 00:31:43.060 Reverse tolls, social networks. 00:31:47.080 --> 00:31:47.770 Lots of things. 00:31:51.040 --> 00:31:53.140 Even if you're not convinced by the motivation, 00:31:53.140 --> 00:31:56.589 I will spend a whole lecture talking about Bellman-Ford. 00:31:56.589 --> 00:31:57.880 So that's just so that's clear. 00:32:01.020 --> 00:32:04.470 So the issue with the negative weight cycles 00:32:04.470 --> 00:32:07.780 is something that is worth spending a minute on. 00:32:07.780 --> 00:32:09.780 And I talked about the fact that you 00:32:09.780 --> 00:32:13.100 had an exponential number of paths. 00:32:13.100 --> 00:32:15.820 And that causes a bit of a problem, 00:32:15.820 --> 00:32:18.610 even in the case where you have positive weights. 00:32:18.610 --> 00:32:20.420 And I will revisit that example. 00:32:20.420 --> 00:32:22.970 But here's an even worse problem that 00:32:22.970 --> 00:32:26.955 corresponds to negative cycles. 00:32:29.449 --> 00:32:30.990 So eventually, you want to terminate. 00:32:30.990 --> 00:32:32.860 The faster you terminate, and if you 00:32:32.860 --> 00:32:35.850 can talk about asymptotic complexity, 00:32:35.850 --> 00:32:38.100 obviously that means that you've terminated 00:32:38.100 --> 00:32:41.160 within a worst case bound of time. 00:32:41.160 --> 00:32:43.680 And if that's exponential, that's bad. 00:32:43.680 --> 00:32:45.040 You'd want it to be small. 00:32:45.040 --> 00:32:47.310 But what if you didn't even terminate? 00:32:47.310 --> 00:32:52.610 So suppose you have something like this 00:32:52.610 --> 00:33:02.400 where you have a graph that has negative weights on some 00:33:02.400 --> 00:33:04.140 of the edges. 00:33:04.140 --> 00:33:05.840 But others are positive. 00:33:05.840 --> 00:33:07.025 So this one has a minus 6. 00:33:17.380 --> 00:33:20.060 I think I got those right. 00:33:20.060 --> 00:33:26.520 So 2 for minus 6 over here, 3, 2, 1, and minus 2. 00:33:26.520 --> 00:33:28.500 So one thing that you notice from this graph 00:33:28.500 --> 00:33:33.700 is that you got this annoying cycle here. 00:33:33.700 --> 00:33:35.710 That's a negative weight cycle. 00:33:35.710 --> 00:33:38.130 And that's why I've picked this particular example. 00:33:38.130 --> 00:33:39.860 Minus 6 plus 2 is minus 4. 00:33:39.860 --> 00:33:41.910 Minus 4 plus 3 is minus 1. 00:33:41.910 --> 00:33:45.670 So if you had something where you can't depend on the fact 00:33:45.670 --> 00:33:50.870 that the D's are going to keep reducing. 00:33:50.870 --> 00:33:53.120 And that eventually, they'll stop reducing. 00:33:53.120 --> 00:33:53.870 Well, that's true. 00:33:53.870 --> 00:33:55.286 Eventually, they'll stop reducing. 00:33:55.286 --> 00:33:57.350 Because they're lower bounded by 0 00:33:57.350 --> 00:34:00.470 when you have positive weight edges or non-negative weight 00:34:00.470 --> 00:34:01.290 edges. 00:34:01.290 --> 00:34:05.940 But if you have a graph with a negative cycle-- and I mean, 00:34:05.940 --> 00:34:08.969 this is a recipe for an infinite loop, right-- 00:34:08.969 --> 00:34:10.980 in your program, potentially a bug. 00:34:10.980 --> 00:34:14.380 But maybe not even a bug, not a bug in implementation, 00:34:14.380 --> 00:34:15.750 but a bug in the algorithm. 00:34:15.750 --> 00:34:19.415 Because this determination condition isn't set properly. 00:34:19.415 --> 00:34:20.790 So you can imagine that you would 00:34:20.790 --> 00:34:24.100 get to B and the first time-- whoops, 00:34:24.100 --> 00:34:26.860 I'm missing a weight here. 00:34:26.860 --> 00:34:32.909 So you get to B. And you say, well, I'm done. 00:34:32.909 --> 00:34:36.739 Delta of SB is 4. 00:34:36.739 --> 00:34:37.576 But that's not true. 00:34:37.576 --> 00:34:39.659 Because you could get to B. And then you could get 00:34:39.659 --> 00:34:41.854 back to B with the weight of 3. 00:34:41.854 --> 00:34:44.020 And then you could do it again with the weight of 2, 00:34:44.020 --> 00:34:45.540 and so on and so forth. 00:34:45.540 --> 00:34:46.989 So that's a problem. 00:34:46.989 --> 00:34:49.050 So what would you like an algorithm to do? 00:34:49.050 --> 00:34:52.719 What would you like the Bellman-Ford to do here? 00:34:52.719 --> 00:34:56.855 It's not the case that all of the delta values, 00:34:56.855 --> 00:34:58.230 that is the shortest path values, 00:34:58.230 --> 00:35:00.210 are undefined for this graph. 00:35:00.210 --> 00:35:03.410 Some of them are well defined. 00:35:03.410 --> 00:35:05.440 This one, you can't ever get back to it. 00:35:05.440 --> 00:35:10.040 So clearly, delta S, S is 0. 00:35:10.040 --> 00:35:12.150 Everybody buy that? 00:35:12.150 --> 00:35:14.610 What about this one? 00:35:14.610 --> 00:35:16.640 It's 2, right. 00:35:16.640 --> 00:35:18.600 Delta S, A is 2. 00:35:21.132 --> 00:35:23.340 And everybody buys that, because there's just no way. 00:35:23.340 --> 00:35:25.560 You can't, you don't touch a negative weight cycle. 00:35:25.560 --> 00:35:27.750 You, in fact, don't touch a negative weight edge. 00:35:27.750 --> 00:35:31.400 But more importantly, you don't touch a negative weight cycle 00:35:31.400 --> 00:35:32.830 in order to get to A. And there's 00:35:32.830 --> 00:35:34.960 no way of touching that. 00:35:34.960 --> 00:35:38.970 On the other hand, anything that's in here 00:35:38.970 --> 00:35:40.870 you could run many times. 00:35:40.870 --> 00:35:44.430 And you could end up with whatever weight you wanted. 00:35:44.430 --> 00:35:46.940 There'd be a minus infinity weight. 00:35:46.940 --> 00:35:48.680 So what you want an algorithm that 00:35:48.680 --> 00:35:51.770 handles in particular negative cycles, 00:35:51.770 --> 00:35:53.400 which are the hard part here. 00:35:53.400 --> 00:35:55.590 Negative weights aren't the hard part 00:35:55.590 --> 00:35:58.960 if you can't run through these edges more than once. 00:35:58.960 --> 00:36:01.450 It's actually the negative cycles that are hard. 00:36:01.450 --> 00:36:03.500 And the negative cycles are going 00:36:03.500 --> 00:36:08.330 to make shortest path lengths indeterminate, but not 00:36:08.330 --> 00:36:12.940 necessarily for every node in the graph, 00:36:12.940 --> 00:36:14.800 like this example shows. 00:36:14.800 --> 00:36:17.600 So what you want your Bellman-Ford algorithm to do, 00:36:17.600 --> 00:36:20.730 or your shortest path algorithm that 00:36:20.730 --> 00:36:23.390 handles negative cycles to do, is 00:36:23.390 --> 00:36:25.810 to finish in reasonable amounts of time. 00:36:25.810 --> 00:36:28.930 Order V, E will take and give you 00:36:28.930 --> 00:36:33.540 the delta numbers for all of the vertices 00:36:33.540 --> 00:36:37.290 that actually have finite numbers 00:36:37.290 --> 00:36:39.150 and then mark all of these other vertices 00:36:39.150 --> 00:36:43.140 as being indeterminate, or essentially minus infinity. 00:36:43.140 --> 00:36:43.720 OK 00:36:43.720 --> 00:36:45.560 So that's your termination condition. 00:36:45.560 --> 00:36:47.770 It's different from the termination condition 00:36:47.770 --> 00:36:50.920 if you simply had non-negative edge weights. 00:36:50.920 --> 00:36:51.890 All right. 00:36:51.890 --> 00:36:55.750 So remember, it's cycles that cause a problem, not just 00:36:55.750 --> 00:36:57.230 the edges. 00:36:57.230 --> 00:37:00.250 And you have to do something about the cycles. 00:37:00.250 --> 00:37:03.210 But they may not affect the entire part of the computation. 00:37:03.210 --> 00:37:06.890 So if you don't know that you have a cycle or not, 00:37:06.890 --> 00:37:12.180 then you end up with having to use Bellman-Ford. 00:37:12.180 --> 00:37:14.750 And so that also tells you something which is interesting, 00:37:14.750 --> 00:37:18.400 which is Bellman-Ford has to detect negative cycles. 00:37:18.400 --> 00:37:20.880 If Bellman-Ford couldn't detect negative cycles, 00:37:20.880 --> 00:37:23.310 then how could it possibly be a correct algorithm 00:37:23.310 --> 00:37:25.090 for the arbitrary case? 00:37:25.090 --> 00:37:27.380 So Dijkstra doesn't have to do that. 00:37:27.380 --> 00:37:29.450 And that's why Dijkstra is simpler. 00:37:29.450 --> 00:37:30.720 All right. 00:37:30.720 --> 00:37:36.820 So let me talk about the general structure of shortest path 00:37:36.820 --> 00:37:37.600 algorithms. 00:37:37.600 --> 00:37:42.750 And the 2 important notions that I want to talk about here 00:37:42.750 --> 00:37:49.030 are the notion of relaxation, which we sort of did already 00:37:49.030 --> 00:37:51.590 when we ran through this example. 00:37:51.590 --> 00:37:53.460 But I need to formalize that. 00:37:53.460 --> 00:37:59.465 And then we'll go back and revisit this exponential graph 00:37:59.465 --> 00:37:59.965 example. 00:38:03.980 --> 00:38:06.650 So the general structural of shortest path algorithms 00:38:06.650 --> 00:38:09.260 are as follows. 00:38:09.260 --> 00:38:16.470 We're going to initialize for all u belonging to the vertex 00:38:16.470 --> 00:38:20.380 set, we set d v to be infinity. 00:38:20.380 --> 00:38:26.260 And we set the predecessor to be NIL. 00:38:26.260 --> 00:38:31.650 And then we'll set d of S to be 0. 00:38:31.650 --> 00:38:33.400 We're talking about a single source, here. 00:38:33.400 --> 00:38:35.800 We'll set that to be 0. 00:38:35.800 --> 00:38:39.740 And what we're going to do is essentially repeat. 00:38:45.110 --> 00:38:54.240 Select some edge u comma v. And I'm not specifying how. 00:38:54.240 --> 00:38:56.560 This is going to result in a different algorithm 00:38:56.560 --> 00:38:58.640 depending on the specifics of how. 00:38:58.640 --> 00:39:07.430 But the important notion is that we're going to relax edge u, 00:39:07.430 --> 00:39:11.980 v. And what the notion of relaxation is 00:39:11.980 --> 00:39:14.140 is that you're going to look at it. 00:39:14.140 --> 00:39:21.880 And you'll say, well, if d of v is greater than d of u 00:39:21.880 --> 00:39:29.000 plus w u, v, then I've discovered a better 00:39:29.000 --> 00:39:33.640 way of getting to v then I currently know. 00:39:33.640 --> 00:39:36.100 So d of v would currently be infinity, 00:39:36.100 --> 00:39:39.020 which means I haven't found a way of getting to v yet. 00:39:39.020 --> 00:39:43.110 But I know that d of u, for example, is a finite number. 00:39:43.110 --> 00:39:45.840 And I do know that this edge exists from u to v, which 00:39:45.840 --> 00:39:48.910 means that I can update the value of d of v. 00:39:48.910 --> 00:39:53.550 And that's what we call relaxation of the edge u, v. 00:39:53.550 --> 00:39:57.710 And so what you do here is if the if is true, 00:39:57.710 --> 00:40:05.770 then you set d, v to be d, u plus w u, v. 00:40:05.770 --> 00:40:09.570 And you'll also update the predecessor relationship, 00:40:09.570 --> 00:40:16.430 because the current best predecessor for v 00:40:16.430 --> 00:40:17.826 is going to be u. 00:40:17.826 --> 00:40:20.900 So that's the notion of relaxation. 00:40:20.900 --> 00:40:23.880 And I kind of ran out of room here. 00:40:23.880 --> 00:40:26.930 But you keep doing this. 00:40:26.930 --> 00:40:28.010 This repeat. 00:40:28.010 --> 00:40:30.525 So what is the repeat? 00:40:30.525 --> 00:40:42.110 Well, the repeat is until all edges 00:40:42.110 --> 00:40:56.860 have d of v less than or equal to d of u plus w u, v. 00:40:56.860 --> 00:41:01.940 And the assumption here is that you have no negative cycles. 00:41:01.940 --> 00:41:05.020 We need a different structure. 00:41:05.020 --> 00:41:07.840 The notion of relaxation is still going to be relevant. 00:41:07.840 --> 00:41:11.650 But don't think of this structure 00:41:11.650 --> 00:41:14.940 as being the structure that Bellman-Ford uses, 00:41:14.940 --> 00:41:19.580 or algorithms that can handle negative cycles use. 00:41:19.580 --> 00:41:23.370 So hopefully, you got the notion of relaxation, 00:41:23.370 --> 00:41:26.580 which is from a pictorial standpoint, 00:41:26.580 --> 00:41:29.420 it's simply something that we did 00:41:29.420 --> 00:41:35.150 when we looked at updating the value of 6 to 5, for example. 00:41:35.150 --> 00:41:39.840 So we said through this process, if I 00:41:39.840 --> 00:41:42.970 relax this particular edge and d was already set up-- 00:41:42.970 --> 00:41:47.230 let's say d, the vertex here had 3. 00:41:47.230 --> 00:41:49.450 And this was originally 6. 00:41:49.450 --> 00:41:53.620 And I look at it and I say, D of C is 6. 00:41:53.620 --> 00:41:58.530 On other hand, 6 is greater than d 00:41:58.530 --> 00:42:03.060 of the vertex D, which happens to be 3 plus 2. 00:42:03.060 --> 00:42:06.370 And since 5 is less than 6, I can relax this edge 00:42:06.370 --> 00:42:09.610 and update the value of 6 to 5. 00:42:09.610 --> 00:42:12.690 And then I update the predecessor relationship 00:42:12.690 --> 00:42:18.300 to have a pi of C to be D. That's 00:42:18.300 --> 00:42:19.460 the notion of relaxation. 00:42:19.460 --> 00:42:20.610 Fundamental notion. 00:42:20.610 --> 00:42:24.860 Going to use it in every algorithm that we talk about. 00:42:24.860 --> 00:42:26.530 When do you stop? 00:42:26.530 --> 00:42:28.480 Well, when you don't have negative cycles, 00:42:28.480 --> 00:42:32.740 there's a fairly clean termination condition, 00:42:32.740 --> 00:42:35.940 which says that you can't relax any of the edges any more. 00:42:35.940 --> 00:42:39.040 OK You get to the point where you 00:42:39.040 --> 00:42:41.430 have values that are associated with each 00:42:41.430 --> 00:42:43.170 of these vertices inside. 00:42:43.170 --> 00:42:45.150 And it doesn't matter what edge you pick, 00:42:45.150 --> 00:42:47.580 you can't improve them. 00:42:47.580 --> 00:42:50.820 So this termination condition, it 00:42:50.820 --> 00:42:54.271 could involve an order E check. 00:42:54.271 --> 00:42:55.770 So we're not talking complexity here 00:42:55.770 --> 00:42:59.190 yet in terms of being efficient. 00:42:59.190 --> 00:43:03.040 But you can imagine when I say until all edges cannot be 00:43:03.040 --> 00:43:05.720 relaxed, that you'd have to look at all the edges. 00:43:05.720 --> 00:43:07.740 And if any one of them can be relaxed, 00:43:07.740 --> 00:43:11.480 it's possible that another one can now be relaxed. 00:43:11.480 --> 00:43:13.800 So you've got to keep going until you get to the point 00:43:13.800 --> 00:43:17.610 where none of the edges can be relaxed. 00:43:17.610 --> 00:43:20.040 So this is a brute force algorithm. 00:43:20.040 --> 00:43:21.420 And it'll work. 00:43:21.420 --> 00:43:23.720 It'll just be slow. 00:43:23.720 --> 00:43:26.110 It'll work for known negative cycles. 00:43:26.110 --> 00:43:29.290 And if you just kind of randomly select these edges 00:43:29.290 --> 00:43:31.340 and just keep going, I'll give you 00:43:31.340 --> 00:43:36.880 an example where it works pretty badly in a minute. 00:43:36.880 --> 00:43:38.340 But this is an algorithm. 00:43:38.340 --> 00:43:40.030 So I guess I lied when I said we weren't 00:43:40.030 --> 00:43:42.270 going to give you an algorithm. 00:43:42.270 --> 00:43:43.150 It is an algorithm. 00:43:43.150 --> 00:43:47.150 It's just an algorithm that you never want to implement. 00:43:47.150 --> 00:43:49.940 You do want to implement the relaxation condition. 00:43:49.940 --> 00:43:55.130 But not this random way of selecting edges and having 00:43:55.130 --> 00:43:58.470 this termination condition that, in of itself, 00:43:58.470 --> 00:44:01.770 is an order E check. 00:44:01.770 --> 00:44:03.490 And one of the reasons why you don't 00:44:03.490 --> 00:44:06.080 want to implement this algorithm is coming up shortly 00:44:06.080 --> 00:44:08.710 in our exponential graph example. 00:44:08.710 --> 00:44:12.530 But let me make sure that people aren't bored. 00:44:12.530 --> 00:44:15.040 Any questions about the general structure, 00:44:15.040 --> 00:44:18.156 relaxation, anything? 00:44:18.156 --> 00:44:19.870 Are we good? 00:44:19.870 --> 00:44:20.550 OK. 00:44:20.550 --> 00:44:24.050 So you guys, I walk away from lecture thinking 00:44:24.050 --> 00:44:26.320 I've given this spectacular lecture and everybody 00:44:26.320 --> 00:44:27.170 understands. 00:44:27.170 --> 00:44:29.320 And then Victor tells me when he shows up 00:44:29.320 --> 00:44:32.260 in section in the morning, he says 00:44:32.260 --> 00:44:34.050 did people understand graphs? 00:44:34.050 --> 00:44:36.130 And everyone says no. 00:44:36.130 --> 00:44:37.810 Or did people understand x? 00:44:37.810 --> 00:44:38.850 And people say no. 00:44:38.850 --> 00:44:43.460 So at least tomorrow, tell Victor that you understood. 00:44:43.460 --> 00:44:45.840 Whether you did or not. 00:44:45.840 --> 00:44:47.090 So then I feel better. 00:44:47.090 --> 00:44:49.774 AUDIENCE: That's going to make my life real easy. 00:44:49.774 --> 00:44:51.680 PROFESSOR: Yeah, right. 00:44:51.680 --> 00:44:53.030 So good. 00:44:53.030 --> 00:44:55.540 Well, you probably like hearing stuff from Victor 00:44:55.540 --> 00:44:56.470 better than me anyway. 00:44:56.470 --> 00:44:59.100 That's the secret here, right? 00:45:02.770 --> 00:45:05.010 All right, so one of the reasons why 00:45:05.010 --> 00:45:09.380 you don't want to implement this algorithm 00:45:09.380 --> 00:45:12.850 is precisely this example that I put up. 00:45:12.850 --> 00:45:16.120 And this is a really neat example that I like a lot, 00:45:16.120 --> 00:45:20.540 because it points out two different things. 00:45:20.540 --> 00:45:24.670 It points out that exponential number 00:45:24.670 --> 00:45:27.840 of paths, an exponential number of paths in a graph, 00:45:27.840 --> 00:45:29.864 could cause a problem with this algorithm. 00:45:29.864 --> 00:45:31.280 The other thing that it points out 00:45:31.280 --> 00:45:35.590 is that we got issues with the weights of edges. 00:45:35.590 --> 00:45:39.480 One of the nice observations one of you made earlier on 00:45:39.480 --> 00:45:42.560 is that we had these neat algorithms that did not 00:45:42.560 --> 00:45:45.760 depend on the dynamic range of the weights. 00:45:45.760 --> 00:45:48.480 So let's just say that I in fact had 00:45:48.480 --> 00:45:50.970 an exponential range for the weights. 00:45:50.970 --> 00:45:55.220 I know 4 isn't exponential, but at some level, 00:45:55.220 --> 00:45:57.500 you could imagine that it's exponentially 00:45:57.500 --> 00:45:59.980 related to 1 or 2. 00:45:59.980 --> 00:46:02.390 And the point here is that if I created 00:46:02.390 --> 00:46:13.940 a graph that looked like this, where I have V4, V5, V6, V7, 00:46:13.940 --> 00:46:20.390 V8, and it had this structure, then 00:46:20.390 --> 00:46:23.270 I'm going to end up having something 00:46:23.270 --> 00:46:29.000 like 2 raised to n over 2 weight if I 00:46:29.000 --> 00:46:30.700 have n vertices in this graph. 00:46:30.700 --> 00:46:33.060 Or at least the dynamic range of these weights 00:46:33.060 --> 00:46:35.730 is going to be 2 raised to n divided by 2. 00:46:35.730 --> 00:46:36.640 Everybody buy that? 00:46:39.720 --> 00:46:43.600 So think of this graph as being a fragment of this large graph, 00:46:43.600 --> 00:46:46.380 which where n could be 100 and the weights 00:46:46.380 --> 00:46:48.380 could be 2 raised to 50. 00:46:48.380 --> 00:46:49.920 And 2 raised to 50 isn't a number 00:46:49.920 --> 00:46:51.780 that we can't handle on a computer, right? 00:46:51.780 --> 00:46:53.720 It's still less than 64 bits, right? 00:46:53.720 --> 00:46:56.250 So it's a pretty reasonable example. 00:46:56.250 --> 00:46:59.190 And we talked about multiple precision arithmetic, 00:46:59.190 --> 00:47:00.440 infinite precision arithmetic. 00:47:00.440 --> 00:47:02.520 So we can handle arbitrary numbers 00:47:02.520 --> 00:47:05.030 of an arbitrary position. 00:47:05.030 --> 00:47:07.940 So there's nothing that's stopping us 00:47:07.940 --> 00:47:11.280 from putting square root of 2 and all sorts of things. 00:47:11.280 --> 00:47:12.990 We won't do imaginary numbers. 00:47:12.990 --> 00:47:15.790 But you could imagine putting numbers 00:47:15.790 --> 00:47:20.140 with a high dynamic range as edges in a particular graph 00:47:20.140 --> 00:47:22.870 and expect the Dijkstra, assuming that all of the edges 00:47:22.870 --> 00:47:28.090 are non-negative, that Dijkstra should be able to run on it. 00:47:28.090 --> 00:47:30.760 So what happens with this example? 00:47:30.760 --> 00:47:33.090 Well, with this example, here's what happens. 00:47:33.090 --> 00:47:36.340 Let's say that I ran this algorithm. 00:47:36.340 --> 00:47:42.210 And initially, I just followed this chain here. 00:47:42.210 --> 00:47:45.320 And I get-- this starts with a 0. 00:47:45.320 --> 00:47:48.530 And this is a 4, because I get there with 4. 00:47:48.530 --> 00:47:50.280 This one is 8. 00:47:50.280 --> 00:47:51.650 And this is 10. 00:47:51.650 --> 00:47:55.380 And this is 12, 13, 14. 00:47:57.930 --> 00:48:00.740 And that's the initial pass. 00:48:00.740 --> 00:48:02.990 That's the selection. 00:48:02.990 --> 00:48:10.020 What ends up happening is that you could now relax at this-- 00:48:10.020 --> 00:48:11.430 you see 14. 00:48:11.430 --> 00:48:13.660 And let's say you relax this edge. 00:48:13.660 --> 00:48:18.940 You see that 12 and 14, you've turned that into 13. 00:48:18.940 --> 00:48:22.500 And then when you relax this edge, this turns into 12. 00:48:25.020 --> 00:48:27.640 So you go through that process. 00:48:27.640 --> 00:48:30.510 Now, this one stays 12. 00:48:30.510 --> 00:48:32.560 But now you relax this edge. 00:48:32.560 --> 00:48:35.350 And so this 12 becomes 10. 00:48:35.350 --> 00:48:37.310 And then when this changes, you need 00:48:37.310 --> 00:48:41.830 to-- if you relax this edge first, 00:48:41.830 --> 00:48:45.060 then this 13 is going to become 11. 00:48:45.060 --> 00:48:46.440 It doesn't really matter. 00:48:46.440 --> 00:48:49.450 This becomes, I guess, 11. 00:48:49.450 --> 00:48:51.281 And-- is that right? 00:48:51.281 --> 00:48:51.780 Yup. 00:48:51.780 --> 00:48:55.690 This is 11 and that's 11 as well. 00:48:55.690 --> 00:48:57.440 It might start out being 12 if you 00:48:57.440 --> 00:48:59.250 relax this edge and that edge. 00:48:59.250 --> 00:49:02.920 So you might go to 12 to 11, and so on and so forth. 00:49:02.920 --> 00:49:07.450 So for a pathological ordering, I won't belabor the point. 00:49:07.450 --> 00:49:09.700 But you see that you're going 14, 13, 12, 00:49:09.700 --> 00:49:14.480 11 with a bad ordering that corresponds 00:49:14.480 --> 00:49:16.990 to the selection of the edges. 00:49:16.990 --> 00:49:21.266 And so if the overall weight here and overall weight here, 00:49:21.266 --> 00:49:22.640 when you start out with, is going 00:49:22.640 --> 00:49:25.390 to be order 2 raised to n over 2. 00:49:25.390 --> 00:49:29.160 OK And you could be, in this particular graph, 00:49:29.160 --> 00:49:32.640 relaxing edges an exponential number of times 00:49:32.640 --> 00:49:36.940 in order to finish. 00:49:36.940 --> 00:49:39.840 And so the number of times you relax an edge 00:49:39.840 --> 00:49:42.440 could be of the order of the weights 00:49:42.440 --> 00:49:43.820 that you start out with. 00:49:43.820 --> 00:49:47.220 And that makes this algorithm an exponential time algorithm. 00:49:47.220 --> 00:49:49.050 So clearly, we have to do better than that 00:49:49.050 --> 00:49:52.540 when it comes to Dijkstra or Bellman-Ford. 00:49:52.540 --> 00:49:54.840 So how are we going to do better than that? 00:49:54.840 --> 00:49:56.260 Yeah, question back there. 00:49:56.260 --> 00:49:57.718 AUDIENCE: Is it an issue that we're 00:49:57.718 --> 00:49:59.176 starting at the [INAUDIBLE]? 00:50:02.807 --> 00:50:04.140 PROFESSOR: You're exactly right. 00:50:04.140 --> 00:50:07.410 There's an issue with the ordering that we've chosen. 00:50:07.410 --> 00:50:10.920 But what you have to show is that for any graph, 00:50:10.920 --> 00:50:12.790 the particular ordering that you choose 00:50:12.790 --> 00:50:16.910 will result in V log V plus E and so on and so forth. 00:50:16.910 --> 00:50:17.910 So you're exactly right. 00:50:17.910 --> 00:50:20.118 I mean, it's an issue with the ordering we've chosen. 00:50:20.118 --> 00:50:22.050 This is a pathological ordering. 00:50:22.050 --> 00:50:23.850 It's just meaning to say that we have 00:50:23.850 --> 00:50:27.080 to be careful about how we select. 00:50:27.080 --> 00:50:30.040 If you selected wrong, you've got problems. 00:50:30.040 --> 00:50:32.110 And so the purpose of next week is 00:50:32.110 --> 00:50:35.360 going to be how do we select these edges properly. 00:50:35.360 --> 00:50:40.070 And so I leave you with this notion of, 00:50:40.070 --> 00:50:44.200 very simple notion of, optimal substructure using 00:50:44.200 --> 00:50:46.430 two very simple terms that you can 00:50:46.430 --> 00:50:50.200 prove in literally a line of text. 00:50:50.200 --> 00:50:59.410 And the first one says as subpaths of a shortest path 00:50:59.410 --> 00:51:00.580 are shortest paths. 00:51:05.520 --> 00:51:11.920 And all that means is if I had V0, and I went to V1, 00:51:11.920 --> 00:51:15.500 and I went to V2, and these are paths here. 00:51:15.500 --> 00:51:22.930 So this could be p01, p02, p03. 00:51:22.930 --> 00:51:26.240 And so there are many vertices potentially between V0 and V1. 00:51:26.240 --> 00:51:30.010 And if you tell me that V0 through V3, 00:51:30.010 --> 00:51:34.460 the concatenation of p01, p02, and, sorry, p03 00:51:34.460 --> 00:51:35.485 are a shortest path. 00:51:40.740 --> 00:51:44.110 If this is an SP, shortest path, then that 00:51:44.110 --> 00:51:50.030 implies that each of these are shortest paths as well. 00:51:50.030 --> 00:51:52.650 And that makes sense, because if in fact there 00:51:52.650 --> 00:51:55.740 was a better way of getting from V0 to V1 00:51:55.740 --> 00:52:00.200 that was better than p01, why would you ever put p01 in here? 00:52:00.200 --> 00:52:02.390 You would use that better way. 00:52:02.390 --> 00:52:03.580 So very simple. 00:52:03.580 --> 00:52:06.810 That's what's called the optimum substructure property. 00:52:06.810 --> 00:52:12.190 And this notion of the triangle inequality 00:52:12.190 --> 00:52:14.440 is also related to that. 00:52:14.440 --> 00:52:21.700 And that simply says that if I have something like this, 00:52:21.700 --> 00:52:28.110 that I have V0, V1, and V2, then when 00:52:28.110 --> 00:52:32.810 I look at the delta value of V0, V1, 00:52:32.810 --> 00:52:38.500 and I compare that with the delta values of V0, V2, and V2, 00:52:38.500 --> 00:52:46.290 V1, then this has got to be smaller than or equal to this 00:52:46.290 --> 00:52:48.096 plus that. 00:52:48.096 --> 00:52:49.220 And that again makes sense. 00:52:49.220 --> 00:52:54.090 Because if this plus this was smaller than that, 00:52:54.090 --> 00:52:59.030 well remember I'm talking about paths here, not edges. 00:52:59.030 --> 00:53:00.890 And the better way of getting to V1 00:53:00.890 --> 00:53:03.140 would be to follow-- go through V2 rather 00:53:03.140 --> 00:53:05.640 than following this path up on top. 00:53:05.640 --> 00:53:07.480 Amazingly, these two notions are going 00:53:07.480 --> 00:53:09.670 to be enough to take this algorithm 00:53:09.670 --> 00:53:12.980 and turn it into essentially a linear time algorithm. 00:53:12.980 --> 00:53:15.530 And we'll do that next time.