WEBVTT 00:00:00.000 --> 00:00:01.960 [SQUEAKING] 00:00:01.960 --> 00:00:03.920 [RUSTLING] 00:00:03.920 --> 00:00:07.350 [CLICKING] 00:00:12.760 --> 00:00:14.540 JASON KU: OK, let's get started. 00:00:14.540 --> 00:00:19.270 Welcome to the 12th lecture of 6.006. 00:00:19.270 --> 00:00:24.100 This is our second lecture talking about weighted graphs, 00:00:24.100 --> 00:00:27.670 and in particular, weighted shortest paths, algorithms. 00:00:27.670 --> 00:00:30.740 Last time we talked about weighted graphs. 00:00:30.740 --> 00:00:32.560 This is a kind of a generalization 00:00:32.560 --> 00:00:36.610 of what we mean by distance in an unweighted graph instead 00:00:36.610 --> 00:00:40.900 of each edge having a weight of 1, essentially. 00:00:40.900 --> 00:00:44.440 We generalize that to be any integer. 00:00:44.440 --> 00:00:48.790 And last time, we showed how to solve 00:00:48.790 --> 00:00:51.790 shortest single-source shortest paths in a graph that 00:00:51.790 --> 00:00:54.130 doesn't have cycles even if it has 00:00:54.130 --> 00:00:58.540 0 or negative weights in linear time using an algorithm called 00:00:58.540 --> 00:01:00.430 DAG relaxation. 00:01:00.430 --> 00:01:04.180 We also showed in that lecture how in linear time, 00:01:04.180 --> 00:01:07.570 if we are given the shortest path weights to all the things 00:01:07.570 --> 00:01:09.940 reachable in finite-- 00:01:09.940 --> 00:01:14.380 or with shortest path distance that's finite, 00:01:14.380 --> 00:01:16.960 we can construct a shortest paths tree 00:01:16.960 --> 00:01:20.380 from those weights in linear time. 00:01:20.380 --> 00:01:23.050 So this is motivating why we're not really 00:01:23.050 --> 00:01:25.960 going to talk about parent pointers for the next couple 00:01:25.960 --> 00:01:26.800 of lectures. 00:01:26.800 --> 00:01:30.940 We're just going to concentrate on the shortest path weights. 00:01:30.940 --> 00:01:32.950 And so today, we're going to be talking 00:01:32.950 --> 00:01:35.950 about our most general algorithm we'll 00:01:35.950 --> 00:01:39.280 be showing for solving single source shortest paths, 00:01:39.280 --> 00:01:42.940 in particular in graphs that could contain cycles 00:01:42.940 --> 00:01:45.280 and could have negative weights. 00:01:45.280 --> 00:01:50.930 So just to recap our little roadmap here, 00:01:50.930 --> 00:01:53.910 single source shortest paths in linear time. 00:01:53.910 --> 00:01:56.440 Last time we discussed another linear time algorithm, 00:01:56.440 --> 00:01:58.240 DAG relaxation. 00:01:58.240 --> 00:01:59.800 And today we're going to be talking 00:01:59.800 --> 00:02:04.240 about Bellman-Ford, which isn't limited to asymptotic graphs. 00:02:04.240 --> 00:02:07.600 In particular, there could be negative weight cycles 00:02:07.600 --> 00:02:09.460 in our graph. 00:02:09.460 --> 00:02:12.970 If it has cycles, if it has negative weights, 00:02:12.970 --> 00:02:15.820 the worry is that we could have negative weight cycles, 00:02:15.820 --> 00:02:17.500 in which case there-- 00:02:17.500 --> 00:02:22.420 if a negative weight cycle is reachable from our source, 00:02:22.420 --> 00:02:24.640 then the vertices in that cycle and anything 00:02:24.640 --> 00:02:28.420 reachable from that cycle will potentially 00:02:28.420 --> 00:02:34.540 have an unbounded number of edges you need to go through. 00:02:34.540 --> 00:02:38.560 There's not a bound on the number of edges for a shortest 00:02:38.560 --> 00:02:41.470 path, because I could just keep going around 00:02:41.470 --> 00:02:44.740 that cycle as many times as I want and get a shorter path. 00:02:44.740 --> 00:02:49.558 And so we assign those distances to be minus infinity. 00:02:49.558 --> 00:02:51.850 So that's what we're going to do today in Bellman-Ford. 00:02:51.850 --> 00:02:57.400 In particular, what we're going to do is compute our shortest 00:02:57.400 --> 00:02:59.440 path distances, the shortest path 00:02:59.440 --> 00:03:03.760 waits for every vertex in our graph, 00:03:03.760 --> 00:03:07.030 setting the ones that are not reachable to infinity, 00:03:07.030 --> 00:03:10.300 and the ones that are reachable through a negative weight 00:03:10.300 --> 00:03:13.450 cycle to minus infinity, and all other ones we're going 00:03:13.450 --> 00:03:16.420 to set to a finite weight. 00:03:16.420 --> 00:03:19.630 And another thing that we might want 00:03:19.630 --> 00:03:21.340 is if there's a negative weight cycle 00:03:21.340 --> 00:03:24.130 in the graph, let's return one. 00:03:24.130 --> 00:03:25.630 So those are the two kinds of things 00:03:25.630 --> 00:03:29.350 that we're trying to solve in today's lecture. 00:03:29.350 --> 00:03:31.510 But before we do that, let's warm up 00:03:31.510 --> 00:03:36.020 with two short exercises. 00:03:36.020 --> 00:03:48.110 The first one, exercise 1, given an undirected graph, 00:03:48.110 --> 00:04:12.280 given undirected graph G, return whether G contains 00:04:12.280 --> 00:04:13.450 a negative weight cycle. 00:04:20.140 --> 00:04:23.740 Anyone have an idea of how we can solve this in linear time, 00:04:23.740 --> 00:04:25.030 actually? 00:04:25.030 --> 00:04:30.280 In fact, we can do it in order E. No-- yes, yes. 00:04:30.280 --> 00:04:34.740 Reachable from S. I guess-- 00:04:34.740 --> 00:04:40.110 let's just say a negative weight cycle at all. 00:04:40.110 --> 00:04:42.787 Not in the context of single-source shortest paths. 00:04:42.787 --> 00:04:45.120 AUDIENCE: Detect whether there's a negative weight edge? 00:04:45.120 --> 00:04:46.230 JASON KU: Ah. 00:04:46.230 --> 00:04:50.610 Your colleague has determined an interesting fact 00:04:50.610 --> 00:04:53.250 about undirected graphs. 00:04:53.250 --> 00:04:57.720 If you have a negative weight edge in an undirected graph, 00:04:57.720 --> 00:05:01.170 I can just move back and forth along that edge. 00:05:01.170 --> 00:05:03.270 That's a cycle of length 2-- 00:05:03.270 --> 00:05:07.335 or I guess three vertices back to where we came from. 00:05:07.335 --> 00:05:09.210 There is of negative weight, because I'm just 00:05:09.210 --> 00:05:13.240 traversing that weight over and over and over again. 00:05:13.240 --> 00:05:16.320 So the question of single-source shortest paths 00:05:16.320 --> 00:05:19.050 of finding negative weights is not 00:05:19.050 --> 00:05:22.780 particularly interesting in the undirected case. 00:05:22.780 --> 00:05:26.070 What I can do is just for every negative weight edge, 00:05:26.070 --> 00:05:31.050 undirected edge in my graph, I can just find the readability 00:05:31.050 --> 00:05:33.300 from the vertices-- 00:05:33.300 --> 00:05:40.200 the endpoints of that edge and label them as minus-- 00:05:40.200 --> 00:05:43.410 basically if the connected component containing S 00:05:43.410 --> 00:05:47.160 has a negative weight edge, then everything in the graph 00:05:47.160 --> 00:05:49.080 is accessible from a negative weight cycle. 00:05:49.080 --> 00:05:51.970 So this is not such an interesting problem. 00:05:51.970 --> 00:06:00.540 And so we're going to restrict our discussion today 00:06:00.540 --> 00:06:02.685 to directed graphs. 00:06:09.020 --> 00:06:20.920 So this is if and only if exists negative weight edge. 00:06:24.700 --> 00:06:32.770 OK, exercise 2, kind of a little preview for what's to come, 00:06:32.770 --> 00:06:36.460 we're actually not going to show you an algorithm directly that 00:06:36.460 --> 00:06:40.050 meets this Bellman-Ford running time, V times E. 00:06:40.050 --> 00:06:44.080 What instead we're going to show you is an algorithm that solves 00:06:44.080 --> 00:06:47.080 single-source shortest paths in-- 00:06:47.080 --> 00:06:54.070 So given an algorithm, Alg A, solves 00:06:54.070 --> 00:07:04.522 single-source shortest paths in order V times V plus E time. 00:07:04.522 --> 00:07:05.230 OK, what is that? 00:07:05.230 --> 00:07:09.430 That's V squared plus V times E. That's 00:07:09.430 --> 00:07:13.510 close to what this V times E is. 00:07:13.510 --> 00:07:15.940 That's what we're going to show you. 00:07:15.940 --> 00:07:19.870 But if I had such an algorithm, can anyone 00:07:19.870 --> 00:07:22.600 tell me a single-source shortest paths algorithm-- 00:07:22.600 --> 00:07:26.560 how we can use this algorithm to solve single-source shortest 00:07:26.560 --> 00:07:29.320 paths in just V times E time? 00:07:35.730 --> 00:07:48.990 Show how to solve SSSP in order the V times E. I 00:07:48.990 --> 00:07:52.740 guess we can put a dot there as well. 00:07:52.740 --> 00:07:55.080 So this is a little tricky. 00:07:55.080 --> 00:07:57.000 It's kind of related to the difference 00:07:57.000 --> 00:07:59.370 we had between the reachability problem 00:07:59.370 --> 00:08:01.980 and the single-source shortest paths problem 00:08:01.980 --> 00:08:05.310 that we saw last lecture. 00:08:05.310 --> 00:08:09.840 When are these asymptotically different in their upper bound 00:08:09.840 --> 00:08:17.590 is when V is asymptotically larger than E. 00:08:17.590 --> 00:08:22.350 But the connected component containing S 00:08:22.350 --> 00:08:27.090 can have at most E vertices, or order 00:08:27.090 --> 00:08:30.960 E. It can actually have at most E plus 1 vertices, 00:08:30.960 --> 00:08:37.169 because otherwise it wouldn't be connected. 00:08:37.169 --> 00:08:41.490 So, what we can do if we had such an algorithm, 00:08:41.490 --> 00:08:44.400 we could first, when we're giving our graph, explore 00:08:44.400 --> 00:08:48.120 everything in the graph using BFS or DFS, 00:08:48.120 --> 00:08:50.670 find all the things reachable from S, 00:08:50.670 --> 00:08:53.250 and then just throw away everything else. 00:08:53.250 --> 00:08:56.700 Now I have a graph for which V is asymptotically 00:08:56.700 --> 00:09:00.870 no bigger than E, and then we can use this algorithm 00:09:00.870 --> 00:09:04.530 to solve single-source shortest paths in V times E time. 00:09:04.530 --> 00:09:06.330 I'm not going to write all that down here. 00:09:06.330 --> 00:09:07.860 You can see it in the notes. 00:09:07.860 --> 00:09:08.440 Yeah? 00:09:08.440 --> 00:09:10.913 AUDIENCE: Does this work if your graph isn't simple? 00:09:10.913 --> 00:09:13.080 JASON KU: Does this work as your graph isn't simple? 00:09:13.080 --> 00:09:14.290 I haven't thought about it. 00:09:14.290 --> 00:09:16.290 We are not going to talk about non-simple graphs 00:09:16.290 --> 00:09:20.190 in this class, but probably not because you've 00:09:20.190 --> 00:09:21.780 got a lot of edges. 00:09:21.780 --> 00:09:23.940 Though in our class if we're talking 00:09:23.940 --> 00:09:26.100 about single-source shortest paths, 00:09:26.100 --> 00:09:28.830 if we have multiple edges between two vertices, 00:09:28.830 --> 00:09:30.750 we can just take the minimum weight one 00:09:30.750 --> 00:09:33.765 because it's never better to take the larger ones. 00:09:33.765 --> 00:09:35.430 Does that answer your question? 00:09:35.430 --> 00:09:36.170 Great. 00:09:36.170 --> 00:09:36.670 All right. 00:09:36.670 --> 00:09:39.687 So those are our warm-ups, that's our goal. 00:09:39.687 --> 00:09:42.270 We need to find an algorithm for single-source shortest paths. 00:09:42.270 --> 00:09:44.370 And general graphs, graphs with-- 00:09:44.370 --> 00:09:49.020 potentially graphs with cycles, and negative weights, 00:09:49.020 --> 00:09:54.270 and solve it in this V times linear kind of time. 00:09:54.270 --> 00:09:55.800 That makes sense? 00:09:55.800 --> 00:09:56.610 All right. 00:09:56.610 --> 00:10:00.150 So first, before we get to the algorithm, 00:10:00.150 --> 00:10:02.250 we're going to discuss a little bit 00:10:02.250 --> 00:10:06.670 about simple short-- about shortest paths in general. 00:10:06.670 --> 00:10:10.140 If we didn't-- the problem here is negative weights. 00:10:10.140 --> 00:10:11.650 How do we find-- 00:10:11.650 --> 00:10:14.040 if we had negative weight cycles, 00:10:14.040 --> 00:10:15.520 there seems to be these problems, 00:10:15.520 --> 00:10:20.360 because we could have minus infinities is in our deltas. 00:10:20.360 --> 00:10:22.670 But if we didn't have negative weights, 00:10:22.670 --> 00:10:26.360 I'd like to assert to you that our shortest paths, even 00:10:26.360 --> 00:10:29.580 if there are negative weights, are going to be simple. 00:10:29.580 --> 00:10:31.010 They won't repeat vertices. 00:10:31.010 --> 00:10:33.590 So that's the first thing we're going to show you. 00:10:38.520 --> 00:10:40.050 Let's see. 00:10:40.050 --> 00:10:45.235 Simple shortest paths. 00:10:49.390 --> 00:10:51.130 OK. 00:10:51.130 --> 00:10:52.740 So, claim. 00:10:55.660 --> 00:10:58.240 I'm going to give my claims numbers 00:10:58.240 --> 00:11:01.180 today just because I'm going to have a lot of them. 00:11:01.180 --> 00:11:06.580 If my shortest path distance from S to some vertex 00:11:06.580 --> 00:11:13.750 is finite, meaning it's not infinite or minus infinite-- 00:11:13.750 --> 00:11:25.315 some finite value, there exists a shortest path-- 00:11:28.110 --> 00:11:37.270 a shortest S to V path that is simple. 00:11:37.270 --> 00:11:40.030 And remember, simple means not going through a vertex 00:11:40.030 --> 00:11:41.290 more than once. 00:11:41.290 --> 00:11:42.958 All right. 00:11:42.958 --> 00:11:44.250 How are we going to prove this? 00:11:49.390 --> 00:11:55.960 Well, consider if this claim was not true. 00:11:55.960 --> 00:12:01.625 If every shortest path contained a cycle, essentially. 00:12:01.625 --> 00:12:03.830 It repeated a vertex. 00:12:03.830 --> 00:12:07.910 Then my path looks something like this. 00:12:11.350 --> 00:12:16.930 I mean, there's some vertices along here, and then I go to V. 00:12:16.930 --> 00:12:19.690 So here's S, and then there's some cycle I 00:12:19.690 --> 00:12:21.470 repeat, some vertex. 00:12:21.470 --> 00:12:26.310 I'm going to call this cycle C. 00:12:26.310 --> 00:12:28.920 Now what do I know about this path? 00:12:28.920 --> 00:12:34.080 I know that it has-- it's a shortest path 00:12:34.080 --> 00:12:36.560 and it has finite weight. 00:12:36.560 --> 00:12:39.950 So in particular, this path-- this delta distance 00:12:39.950 --> 00:12:41.820 is not minus infinity. 00:12:41.820 --> 00:12:45.010 But if this is not minus infinity, 00:12:45.010 --> 00:12:48.538 what do I know about the weight of this cycle? 00:12:48.538 --> 00:12:49.910 AUDIENCE: It's not negative. 00:12:49.910 --> 00:12:50.535 JASON KU: Yeah. 00:12:50.535 --> 00:12:51.932 It can't be negative. 00:12:51.932 --> 00:12:53.390 Because if it was negative, I could 00:12:53.390 --> 00:12:55.580 keep going around this cycle, and this 00:12:55.580 --> 00:12:58.370 would have a non-finite weight. 00:12:58.370 --> 00:13:03.350 Shortest path distance from S. So I know this is-- 00:13:03.350 --> 00:13:06.860 can't be negative, so it must be 0 or positive. 00:13:06.860 --> 00:13:10.880 But if it's 0 or positive and this is a shortest path-- 00:13:10.880 --> 00:13:16.960 went through this cycle, then I could remove it, 00:13:16.960 --> 00:13:21.360 and now I have a new path with one fewer cycle. 00:13:21.360 --> 00:13:26.130 I could just keep doing this to create a simple path. 00:13:26.130 --> 00:13:28.590 So that checks out. 00:13:28.590 --> 00:13:30.120 OK. 00:13:30.120 --> 00:13:33.060 So, that's interesting. 00:13:33.060 --> 00:13:39.000 If it's simple, what do we know about the number of edges 00:13:39.000 --> 00:13:42.570 in a simple shortest paths? 00:13:42.570 --> 00:13:45.740 How many could there possibly be? 00:13:45.740 --> 00:13:51.550 How long in number of edges could a simple shortest path 00:13:51.550 --> 00:13:52.050 be? 00:13:52.050 --> 00:13:55.610 If I can't repeat vertices, I can 00:13:55.610 --> 00:14:00.710 have at most vertices on my simple path, which 00:14:00.710 --> 00:14:04.850 means I can use at most V minus 1 edges-- 00:14:04.850 --> 00:14:06.590 fence posting. 00:14:06.590 --> 00:14:19.260 So, simple paths have at most V minus 1 edges. 00:14:25.520 --> 00:14:30.050 That's a nice little thing I'd like to box off. 00:14:30.050 --> 00:14:31.355 That's a really nice property. 00:14:33.920 --> 00:14:40.000 So while a shortest path here could have an infinite number 00:14:40.000 --> 00:14:45.310 of edges, if the shortest path distance is finite, 00:14:45.310 --> 00:14:48.560 I know I only have to check paths 00:14:48.560 --> 00:14:51.710 that use up to V minus 1 edges. 00:14:51.710 --> 00:14:56.710 In particular, this is finitely bounded in terms of the number 00:14:56.710 --> 00:14:57.950 of paths I have to consider. 00:14:57.950 --> 00:15:02.470 It's exponential, potentially, but at least it's finite. 00:15:02.470 --> 00:15:09.130 The other way, I potentially had to check every possible path 00:15:09.130 --> 00:15:12.740 of which there could be infinite if there's cycles in my graph. 00:15:12.740 --> 00:15:14.420 OK. 00:15:14.420 --> 00:15:19.930 So I have an idea. 00:15:19.930 --> 00:15:25.950 What if I could find shortest-path distances 00:15:25.950 --> 00:15:30.110 by limiting the number of edges I go through? 00:15:30.110 --> 00:15:35.525 So not the full shortest path distance from S 00:15:35.525 --> 00:15:38.870 to V, but let's limit the number of edges 00:15:38.870 --> 00:15:41.480 I'm allowed to go through, and let's 00:15:41.480 --> 00:15:43.970 talk about those shortest-path distances, 00:15:43.970 --> 00:15:46.640 just among the paths that have at most a certain number 00:15:46.640 --> 00:15:47.450 of edges. 00:15:47.450 --> 00:15:55.370 I'm going to call this k-edge distance. 00:15:55.370 --> 00:15:59.240 And I'm just going to provide a little notation here. 00:15:59.240 --> 00:16:02.340 Instead of having a delta, I'll have a delta k here. 00:16:02.340 --> 00:16:05.720 That means how many edges I'm limited by. 00:16:05.720 --> 00:16:24.335 So from S to V is shortest S to V path using at most k edges. 00:16:29.750 --> 00:16:36.800 Short-- weight-- weight of a. 00:16:36.800 --> 00:16:43.550 Shortest path, shortest S to V path using at most k edges. 00:16:43.550 --> 00:16:47.600 And these notions seem somewhat symmetric. 00:16:47.600 --> 00:16:59.700 If I were able to compute this thing for V minus 1, then-- 00:16:59.700 --> 00:17:06.430 for all the vertices, then if the distance is finite, 00:17:06.430 --> 00:17:08.910 then I'll have successfully computed 00:17:08.910 --> 00:17:12.569 the real shortest paths because of this statement. 00:17:12.569 --> 00:17:16.950 Now that doesn't mean that if this is-- 00:17:20.190 --> 00:17:21.900 it doesn't mean the other way. 00:17:21.900 --> 00:17:26.940 If this is minus infinity, if the shortest-path distance is 00:17:26.940 --> 00:17:31.820 minus infinity, it doesn't say anything about what this is. 00:17:31.820 --> 00:17:34.500 It just says the shortest path using 00:17:34.500 --> 00:17:40.200 at most V minus 1 vertices, using at most V minus 1 edges 00:17:40.200 --> 00:17:41.970 would be whatever this is. 00:17:41.970 --> 00:17:43.710 But really, the shortest path length 00:17:43.710 --> 00:17:46.620 needs to consider an infinite number of edges. 00:17:46.620 --> 00:17:49.840 So it doesn't really tell us much about that. 00:17:49.840 --> 00:17:51.460 But for the finite ones it does. 00:17:51.460 --> 00:17:53.050 It works well. 00:17:53.050 --> 00:17:56.550 And so if we are able to compute this thing for k equals 00:17:56.550 --> 00:18:01.850 V minus 1 in a graph that doesn't contain negative weight 00:18:01.850 --> 00:18:02.780 cycles, we'd be done. 00:18:06.830 --> 00:18:09.770 I claim to you a stronger statement, 00:18:09.770 --> 00:18:24.960 that if the shortest path using at most V edges from s to v Is 00:18:24.960 --> 00:18:29.940 less than-- strictly less than delta of V minus 1-- 00:18:33.660 --> 00:18:37.720 this is all in the subscript here. 00:18:37.720 --> 00:18:43.600 Basically this is the shortest-path distance 00:18:43.600 --> 00:18:49.240 of any simple path, and possibly ones that also contain cycles, 00:18:49.240 --> 00:18:53.760 but definitely it includes all the simple paths. 00:18:53.760 --> 00:18:58.590 If there's a shorter path to my vertex 00:18:58.590 --> 00:19:02.850 that goes through more than V minus 1 edges, 00:19:02.850 --> 00:19:06.810 that this path can't be simple, because it 00:19:06.810 --> 00:19:08.850 goes through a vertex more than once. 00:19:08.850 --> 00:19:13.540 Otherwise it would be included in this distance set. 00:19:13.540 --> 00:19:18.010 So if this is the case, and I found a shorter path 00:19:18.010 --> 00:19:21.790 to V that uses v edges-- 00:19:21.790 --> 00:19:23.980 yeah, that use V edges, that path 00:19:23.980 --> 00:19:28.520 can't be simple, which means that path or some path 00:19:28.520 --> 00:19:31.730 there contains a negative weight cycle. 00:19:31.730 --> 00:19:38.960 So if this is true, then I know that 00:19:38.960 --> 00:19:42.820 the real shortest-path distance from S to V 00:19:42.820 --> 00:19:45.830 must be minus infinity. 00:19:49.670 --> 00:19:53.360 I'm going to call such a vertex a witness. 00:19:53.360 --> 00:19:56.490 If we can find a vertex that has this property-- 00:19:56.490 --> 00:19:59.720 I mean, I haven't shown you how to compute these things yet, 00:19:59.720 --> 00:20:02.390 but if I were able to find a vertex V-- 00:20:02.390 --> 00:20:08.980 and these are capital V's if you're having trouble. 00:20:08.980 --> 00:20:13.150 This V is different than this V, this is cardinality. 00:20:13.150 --> 00:20:15.670 If we can find such a vertex V, that 00:20:15.670 --> 00:20:17.890 certifies that there is a negative weight 00:20:17.890 --> 00:20:21.590 cycle in our graph. 00:20:21.590 --> 00:20:25.925 So I'm going to call V is a witness. 00:20:30.630 --> 00:20:32.440 OK. 00:20:32.440 --> 00:20:38.720 So, if this property is true, it's a witness 00:20:38.720 --> 00:20:39.980 and it definitely has this. 00:20:39.980 --> 00:20:42.152 Is it possible, you think-- 00:20:42.152 --> 00:20:43.610 I'm going to claim to you that it's 00:20:43.610 --> 00:20:49.330 possible that a vertex could have minus infinite distance 00:20:49.330 --> 00:20:55.060 but not have this property halt. I could probably 00:20:55.060 --> 00:20:55.940 give you an example-- 00:20:55.940 --> 00:20:58.210 I don't have one off the top of my head right now, 00:20:58.210 --> 00:21:00.220 but that's possible. 00:21:00.220 --> 00:21:02.170 You could imagine, there might be 00:21:02.170 --> 00:21:07.630 no path going to a vertex on a negative weight cycle that 00:21:07.630 --> 00:21:10.510 goes through V exactly V edges. 00:21:10.510 --> 00:21:14.860 It might go through more edges, a shorter one. 00:21:14.860 --> 00:21:18.820 So this equation would be inequality 00:21:18.820 --> 00:21:23.530 and would not certify that this is true. 00:21:23.530 --> 00:21:28.210 But I claim to you, if a vertex has this property, 00:21:28.210 --> 00:21:33.010 if it's its shortest path distances minus infinite, 00:21:33.010 --> 00:21:35.470 then it must be reachable from a witness. 00:21:35.470 --> 00:21:36.400 So that's the claim. 00:21:41.010 --> 00:21:52.460 If delta S, V is minus infinity, then V 00:21:52.460 --> 00:22:02.560 is reachable from a witness. 00:22:02.560 --> 00:22:05.320 Reachable from a vertex that has this property-- that 00:22:05.320 --> 00:22:07.370 has this property. 00:22:07.370 --> 00:22:09.100 And if it's reachable from something 00:22:09.100 --> 00:22:13.000 that has minus infinity shortest pathway, 00:22:13.000 --> 00:22:16.940 then I can take that path go to my reachable vertex, 00:22:16.940 --> 00:22:20.470 and that's also minus infinite path. 00:22:20.470 --> 00:22:21.040 OK. 00:22:21.040 --> 00:22:22.180 So how do we prove this? 00:22:28.320 --> 00:22:37.800 Well, let's consider-- let's I'm going 00:22:37.800 --> 00:22:42.170 to state a somewhat stronger statement that we'll 00:22:42.170 --> 00:22:43.940 prove instead. 00:22:43.940 --> 00:22:47.420 It suffices to prove that every negative weight 00:22:47.420 --> 00:22:51.210 cycle contains a witness. 00:22:51.210 --> 00:22:55.860 If we are to prove that, then every vertex 00:22:55.860 --> 00:23:05.510 with this property, every vertex with this property 00:23:05.510 --> 00:23:10.840 is reachable from a negative weight cycle by definition. 00:23:10.840 --> 00:23:15.400 So, if we can prove that every-- 00:23:18.920 --> 00:23:36.510 prove every negative weight cycle contains witness. 00:23:39.620 --> 00:23:42.040 If we can prove that every negative weight cycle contains 00:23:42.040 --> 00:23:47.800 a witness, then every vertex reachable from one of those 00:23:47.800 --> 00:23:50.050 witnesses-- in particular, reachable from the negative 00:23:50.050 --> 00:23:50.650 weight cycle-- 00:23:53.850 --> 00:23:59.897 has shortest distance minus infinity, 00:23:59.897 --> 00:24:01.230 and that should prove the claim. 00:24:04.270 --> 00:24:06.900 This thing has to be reachable from a negative weight cycle. 00:24:12.700 --> 00:24:15.420 And so if we prove negative weight cycles contain 00:24:15.420 --> 00:24:19.232 witnesses, then all of these vertices 00:24:19.232 --> 00:24:20.440 are reachable from a witness. 00:24:20.440 --> 00:24:22.420 OK, great, great. 00:24:22.420 --> 00:24:24.230 Confusing myself there for a second. 00:24:24.230 --> 00:24:25.680 OK. 00:24:25.680 --> 00:24:30.820 So let's consider a negative weight cycle. 00:24:30.820 --> 00:24:32.590 NG. 00:24:32.590 --> 00:24:35.360 Here's a directed negative weight cycle. 00:24:35.360 --> 00:24:35.860 Recall. 00:24:38.425 --> 00:24:42.870 This will be my negative weight cycle C. 00:24:42.870 --> 00:24:45.760 All of the sum of the edges in this thing, 00:24:45.760 --> 00:24:48.100 the weights has negative weight. 00:24:50.920 --> 00:24:52.810 And I'm going to have a little bit notation-- 00:24:52.810 --> 00:24:56.560 if I have a vertex V here, I'm going 00:24:56.560 --> 00:25:00.220 to say that its predecessor in the cycle, I'm 00:25:00.220 --> 00:25:02.962 just going to call it V prime. 00:25:02.962 --> 00:25:04.045 That's just some notation. 00:25:06.830 --> 00:25:08.970 All right. 00:25:08.970 --> 00:25:15.480 So, if I have computed these shortest-path distances 00:25:15.480 --> 00:25:19.230 to every vertex in my graph, shortest-path distance going 00:25:19.230 --> 00:25:21.780 through at most V vertices and the shortest path 00:25:21.780 --> 00:25:25.000 distance going through at most V minus 1 vertices, 00:25:25.000 --> 00:25:27.420 then I know the following thing holds. 00:25:27.420 --> 00:25:35.600 Delta V going from S to V for any vertex in my cycle 00:25:35.600 --> 00:25:41.120 can't be bigger than delta V minus 1 00:25:41.120 --> 00:25:46.400 from S to U plus the weight-- 00:25:46.400 --> 00:25:50.520 sorry, not U-- V prime, its predecessor, 00:25:50.520 --> 00:25:56.100 plus the weight going from the predecessor to my vertex. 00:25:56.100 --> 00:25:58.210 Why is that? 00:25:58.210 --> 00:25:58.850 Why is that? 00:25:58.850 --> 00:26:02.020 Because this is the weight of some vertex-- 00:26:02.020 --> 00:26:03.940 this is the weight-- 00:26:03.940 --> 00:26:08.080 the shortest-path distance to my predecessor 00:26:08.080 --> 00:26:10.370 using one fewer edge. 00:26:10.370 --> 00:26:12.880 And so this in particular is the weight 00:26:12.880 --> 00:26:17.010 of some path that uses V edges. 00:26:17.010 --> 00:26:21.660 So if this is the shortest such path distance, 00:26:21.660 --> 00:26:24.870 this has to upper bound it at least-- 00:26:24.870 --> 00:26:25.500 at most. 00:26:25.500 --> 00:26:25.650 Yeah? 00:26:25.650 --> 00:26:27.400 AUDIENCE: Is that the triangle inequality? 00:26:27.400 --> 00:26:29.830 JASON KU: That is a statement of the triangle inequality, 00:26:29.830 --> 00:26:32.240 thank you. 00:26:32.240 --> 00:26:33.050 All right. 00:26:33.050 --> 00:26:37.460 So, yes, this is just by triangle inequality. 00:26:37.460 --> 00:26:38.720 OK. 00:26:38.720 --> 00:26:43.490 Now what we can say is, let's take this equation summed 00:26:43.490 --> 00:26:46.700 over all vertices in my cycle. 00:26:46.700 --> 00:26:51.260 So I'm just going to add summation 00:26:51.260 --> 00:26:57.350 here of all vertices in my cycle of this whole thing. 00:26:57.350 --> 00:27:01.400 I'm going to do that out a little bit neater. 00:27:01.400 --> 00:27:08.420 Summation of delta, not d. 00:27:08.420 --> 00:27:18.150 Delta V S, V. I guess I don't need this open parentheses. 00:27:18.150 --> 00:27:26.760 Equals-- or less than or equal to sum of V and C of delta 00:27:26.760 --> 00:27:30.915 V minus 1 V prime. 00:27:33.890 --> 00:27:38.690 And here, I'm summing over V and C, 00:27:38.690 --> 00:27:42.320 and this is just my notation for the predecessor. 00:27:42.320 --> 00:27:48.740 And then I'm going to sum over the weights in my cycle 00:27:48.740 --> 00:27:53.680 V and C. These are the sum of the weights in my cycle. 00:27:53.680 --> 00:27:57.090 Well, what do I know about this cycle? 00:27:57.090 --> 00:28:01.700 This is just the weight of C. The weight of C-- 00:28:01.700 --> 00:28:03.800 that's awful handwriting. 00:28:03.800 --> 00:28:06.960 C, what do I know about the weight of the cycle? 00:28:06.960 --> 00:28:09.240 It's negative. 00:28:09.240 --> 00:28:16.452 So, this is less than 0, which means that if I remove this, 00:28:16.452 --> 00:28:18.900 this needs to be a strict equality. 00:28:22.630 --> 00:28:26.740 But if the sum of all of these is strictly 00:28:26.740 --> 00:28:30.110 less than the sum of all these, we 00:28:30.110 --> 00:28:36.440 can't have none of the vertices in my graph satisfying-- 00:28:36.440 --> 00:28:38.030 not satisfying this property. 00:28:42.030 --> 00:28:45.390 If all of them are not witnesses, 00:28:45.390 --> 00:28:49.970 then this thing is bigger than this thing-- at least 00:28:49.970 --> 00:28:53.930 as big as this thing for every vertex in my cycle, which 00:28:53.930 --> 00:28:56.640 is a contradiction. 00:28:56.640 --> 00:29:02.560 So, the claim holds, if we have a negative infinite 00:29:02.560 --> 00:29:06.110 shortest-path distance, then V is reachable from a witness. 00:29:06.110 --> 00:29:09.190 So it suffices for us to find all the witnesses, 00:29:09.190 --> 00:29:12.970 find all the vertices reachable from the witnesses, 00:29:12.970 --> 00:29:16.050 and then mark them as minus infinity. 00:29:16.050 --> 00:29:18.020 Does that make sense? 00:29:18.020 --> 00:29:19.640 OK. 00:29:19.640 --> 00:29:26.610 So, now we finally are able to get to our algorithm. 00:29:26.610 --> 00:29:27.633 Bellman-Ford. 00:29:31.710 --> 00:29:34.830 And what I'm going to show you today 00:29:34.830 --> 00:29:37.020 is a little different than what is normally 00:29:37.020 --> 00:29:39.000 presented as Bellman-Ford. 00:29:39.000 --> 00:29:42.040 The original Bellman-Ford algorithm 00:29:42.040 --> 00:29:43.928 does something a little different. 00:29:43.928 --> 00:29:45.970 And because it does something a little different, 00:29:45.970 --> 00:29:48.130 which we'll talk about at the end, 00:29:48.130 --> 00:29:50.140 it's a little hairier to analyze. 00:29:50.140 --> 00:29:53.950 I'm going to show you a modification that 00:29:53.950 --> 00:29:58.270 is a little easier to analyze and has this nice property 00:29:58.270 --> 00:30:01.120 that we're going to be able to use the algorithm 00:30:01.120 --> 00:30:04.195 to give us a negative weight cycle if it exists. 00:30:06.920 --> 00:30:12.000 So, we're going to say this is maybe a modified Bellman-Ford. 00:30:12.000 --> 00:30:20.000 And the idea here is to make a vertex associate-- make 00:30:20.000 --> 00:30:23.660 many versions of a vertex. 00:30:23.660 --> 00:30:26.300 And I want this version of the vertex 00:30:26.300 --> 00:30:32.360 to correspond to whether I came here using 0 edges, 1 edge, 2 00:30:32.360 --> 00:30:34.100 edges, 3 edges-- 00:30:34.100 --> 00:30:35.600 I have a different vertex version 00:30:35.600 --> 00:30:39.530 of the vertex for each one of these-- 00:30:39.530 --> 00:30:44.660 for a path going through, at most, 00:30:44.660 --> 00:30:46.380 a certain number of edges. 00:30:46.380 --> 00:30:46.880 OK. 00:30:46.880 --> 00:30:51.080 So this is an idea called graph duplication. 00:30:51.080 --> 00:30:58.870 Idea, graph duplication. 00:30:58.870 --> 00:31:00.880 And this is a very common technique 00:31:00.880 --> 00:31:04.890 for solving graph-related problems. 00:31:04.890 --> 00:31:06.870 Because essentially what I get to do is I 00:31:06.870 --> 00:31:09.090 get to store information. 00:31:11.860 --> 00:31:15.480 If I'm having different versions of a vertex, 00:31:15.480 --> 00:31:18.270 I can have that vertex correspond 00:31:18.270 --> 00:31:22.350 to reaching that vertex in a different state. 00:31:22.350 --> 00:31:24.180 So that's what we're going to do here. 00:31:24.180 --> 00:31:36.040 The idea here is make V plus 1 levels-- 00:31:36.040 --> 00:31:38.980 basically duplicate vertices in our graph-- 00:31:44.080 --> 00:32:05.480 where vertex Vk in level k represents reaching vertex 00:32:05.480 --> 00:32:13.120 V using at most k edges. 00:32:13.120 --> 00:32:16.630 OK, so this definition seems similar to what 00:32:16.630 --> 00:32:19.840 we're doing up here. 00:32:19.840 --> 00:32:24.070 If we have vertices that have this property, then 00:32:24.070 --> 00:32:26.710 their shortest paths in this new graph 00:32:26.710 --> 00:32:30.490 might correspond to these k edge distances. 00:32:30.490 --> 00:32:32.200 And really, the name of the game here 00:32:32.200 --> 00:32:35.080 is to compute these two for every vertex, 00:32:35.080 --> 00:32:37.120 because then we can-- 00:32:37.120 --> 00:32:42.350 then if d is finite, delta is finite, 00:32:42.350 --> 00:32:45.200 then this guy will be the length of our shortest path. 00:32:45.200 --> 00:32:48.190 And if they are different, that will be a witness 00:32:48.190 --> 00:32:51.810 and we can explore from it. 00:32:51.810 --> 00:33:14.750 So-- and if we connect edges from one level 00:33:14.750 --> 00:33:26.400 to only higher levels, basically levels with a higher k, then 00:33:26.400 --> 00:33:27.840 this graph is going to be a DAG. 00:33:34.420 --> 00:33:35.800 Whoa. 00:33:35.800 --> 00:33:37.620 That's cool. 00:33:37.620 --> 00:33:38.480 Why is that cool? 00:33:38.480 --> 00:33:42.440 Because we saw how to solve single-source shortest paths 00:33:42.440 --> 00:33:44.150 in a DAG and linear time. 00:33:44.150 --> 00:33:46.160 Now this graph that we're going to construct 00:33:46.160 --> 00:33:49.100 is going to have V plus 1 levels. 00:33:49.100 --> 00:33:55.245 So could have-- our graph kind of explodes V times. 00:33:57.702 --> 00:33:59.160 We're going to do that in a second. 00:33:59.160 --> 00:34:02.560 I'm going to be more precise with what I mean there. 00:34:02.560 --> 00:34:07.740 But if we're multiplying our graph V plus 1 times, 00:34:07.740 --> 00:34:10.380 then the size of our graph is now V times larger. 00:34:10.380 --> 00:34:14.790 Now that doesn't-- that's not so hard to believe. 00:34:14.790 --> 00:34:17.969 But if we made our graph V times larger 00:34:17.969 --> 00:34:21.300 and we ran a shortest path algorithm in linear time 00:34:21.300 --> 00:34:24.969 with respect to that graph, then that graph 00:34:24.969 --> 00:34:33.670 has something like size V times V plus E size. 00:34:37.810 --> 00:34:39.100 That looks familiar, maybe? 00:34:42.929 --> 00:34:45.389 That's this running time. 00:34:45.389 --> 00:34:48.210 So if we can find an algorithm that runs in that running time, 00:34:48.210 --> 00:34:51.130 we can get down to V times E. 00:34:51.130 --> 00:34:52.159 So let's try to do that. 00:34:57.640 --> 00:34:59.830 Here's the transformation I'm going to show you. 00:34:59.830 --> 00:35:02.830 I'm going to show you first with an example. 00:35:02.830 --> 00:35:08.770 Here's an example of a directed graph that does contain 00:35:08.770 --> 00:35:10.960 a negative weight cycle. 00:35:10.960 --> 00:35:12.090 Can anyone find it for me? 00:35:16.830 --> 00:35:19.440 bcd. 00:35:19.440 --> 00:35:21.960 Has weigh minus 4 plus 3 minus 1. 00:35:21.960 --> 00:35:26.700 It has a minus 2 total weight. 00:35:26.700 --> 00:35:28.840 So that's a negative weight cycle. 00:35:28.840 --> 00:35:33.870 So in order to take shortest paths from a, 00:35:33.870 --> 00:35:39.330 I will want to say at the end of my algorithm, this better be 0, 00:35:39.330 --> 00:35:42.870 and all of these better be minus infinity. 00:35:42.870 --> 00:35:47.170 So that's what I want in my algorithm. 00:35:47.170 --> 00:35:48.700 So what's my algorithm going to be? 00:35:48.700 --> 00:35:55.250 I'm going to make V plus 1 copies of this graph, 00:35:55.250 --> 00:35:56.930 and I'm going to kind of stretch it out. 00:35:56.930 --> 00:35:57.430 OK. 00:35:57.430 --> 00:36:02.260 So here, I have V 0, 1, 2, 3, 4-- 00:36:02.260 --> 00:36:04.340 there are four vertices in my graph. 00:36:04.340 --> 00:36:08.620 So this is 1, 2, 3, 4, 5 copies of my graph. 00:36:08.620 --> 00:36:13.600 I have a version of vertex a for each one of those copies, 00:36:13.600 --> 00:36:16.390 a version of vertex b for each of those copies, c 00:36:16.390 --> 00:36:18.400 and d, et cetera. 00:36:18.400 --> 00:36:21.730 So I have this nice grid of vertices. 00:36:21.730 --> 00:36:25.930 And I'm not going to put any edges within a layer, 00:36:25.930 --> 00:36:27.900 within a level. 00:36:27.900 --> 00:36:30.660 Because then-- I mean, this graph has cycles. 00:36:30.660 --> 00:36:33.060 And I don't want cycles in my graph. 00:36:33.060 --> 00:36:36.240 What I'm going to do instead is for every edge 00:36:36.240 --> 00:36:40.140 in my original graph-- for example, the edge from a to b, 00:36:40.140 --> 00:36:43.760 I'm going to connect it to the b in the next level. 00:36:43.760 --> 00:36:49.860 So a0 is connected to b1 with an edge weight of minus 5, 00:36:49.860 --> 00:36:51.723 just like in the original. 00:36:51.723 --> 00:36:53.890 And I'm going to do that for every edge in my graph, 00:36:53.890 --> 00:36:55.807 and I'm going to repeat that down all the way. 00:36:58.100 --> 00:37:03.770 In addition, I'm going to add zero-weight edge from a0 00:37:03.770 --> 00:37:08.060 to a1 or from every vertex all the way down the line. 00:37:08.060 --> 00:37:11.510 These are all zero-weight edges corresponding to-- 00:37:11.510 --> 00:37:13.800 I'm not going to traverse an edge, 00:37:13.800 --> 00:37:17.360 I'm just going to stay at this vertex. 00:37:17.360 --> 00:37:19.040 That's going to allow us to simulate 00:37:19.040 --> 00:37:22.460 this at most k edges condition. 00:37:22.460 --> 00:37:27.060 Now if you take a look at paths in this graph from a0, 00:37:27.060 --> 00:37:32.760 our starting vertex, clearly none 00:37:32.760 --> 00:37:38.700 of the other vertices in that level are reachable from a0, 00:37:38.700 --> 00:37:40.260 just as we want. 00:37:40.260 --> 00:37:43.260 Because the shortest-path distance 00:37:43.260 --> 00:37:47.100 to any of these vertices using at most 0 edges 00:37:47.100 --> 00:37:48.340 should be infinite. 00:37:48.340 --> 00:37:50.120 I can't get there in 0 edges. 00:37:53.730 --> 00:38:03.920 But then any path in this graph using at most k edges 00:38:03.920 --> 00:38:09.530 is going to correspond to a path from a0 00:38:09.530 --> 00:38:13.050 to a vertex in that level, the corresponding level. 00:38:13.050 --> 00:38:16.610 So for example, if I had a-- 00:38:16.610 --> 00:38:26.450 if I was looking for paths 2b using 00:38:26.450 --> 00:38:34.400 at most three edges, any path-- 00:38:34.400 --> 00:38:40.670 a path from a0 to b3 in this graph 00:38:40.670 --> 00:38:43.820 would correspond to a path in this graph that 00:38:43.820 --> 00:38:47.320 uses at most three edges. 00:38:47.320 --> 00:38:50.010 so Let's find such a path. 00:38:50.010 --> 00:38:58.260 So going from a0, b1, stay at b1-- 00:38:58.260 --> 00:39:00.090 stay at b, sorry. 00:39:00.090 --> 00:39:05.020 Yeah, that's a path using fewer than three edges-- 00:39:05.020 --> 00:39:07.000 or at most three edges. 00:39:07.000 --> 00:39:09.270 But there's another path here. 00:39:09.270 --> 00:39:11.100 Where is it? 00:39:11.100 --> 00:39:14.215 Going from a, a, a to b-- 00:39:14.215 --> 00:39:15.840 OK, that's not such an interesting one. 00:39:15.840 --> 00:39:17.410 That's the same path. 00:39:17.410 --> 00:39:20.040 So I might have more than one path in here corresponding 00:39:20.040 --> 00:39:24.420 to a path in there, but my claim is that any path in here 00:39:24.420 --> 00:39:28.250 corresponds to a path in here. 00:39:28.250 --> 00:39:31.790 So what's a path of length? 00:39:31.790 --> 00:39:33.110 3, that's non-trivial. 00:39:33.110 --> 00:39:36.560 Yeah, a to c to d to b. 00:39:36.560 --> 00:39:42.620 So a to c to d to b. 00:39:42.620 --> 00:39:44.510 Yeah, that's a path. 00:39:44.510 --> 00:39:47.330 And basically, because I constructed this so 00:39:47.330 --> 00:39:49.670 that the edges always moved from level to level, 00:39:49.670 --> 00:39:54.500 as I traverse these edges, I always change levels. 00:39:54.500 --> 00:39:56.487 Yeah? 00:39:56.487 --> 00:39:58.320 AUDIENCE: But my original graph doesn't have 00:39:58.320 --> 00:40:00.587 these self-loops with 0 weight. 00:40:00.587 --> 00:40:01.170 JASON KU: Yes. 00:40:01.170 --> 00:40:06.060 My original graph doesn't have an edge from a to a. 00:40:06.060 --> 00:40:08.550 That's true. 00:40:08.550 --> 00:40:12.430 I'm using these edges to correspond to-- 00:40:12.430 --> 00:40:14.770 I'm deciding not to take an edge. 00:40:19.160 --> 00:40:22.400 It's not that I'm like doing any work here, 00:40:22.400 --> 00:40:25.350 I'm just staying there for a state. 00:40:25.350 --> 00:40:28.640 And that's what's going to allow me to get this at most edges. 00:40:28.640 --> 00:40:29.210 All right. 00:40:29.210 --> 00:40:31.040 So, this is the graph construct. 00:40:31.040 --> 00:40:37.760 Hopefully you understand that we made these V layers. 00:40:37.760 --> 00:40:44.600 This is V. And a vertex-- 00:40:44.600 --> 00:40:48.030 we made V copies of every vertex and connected 00:40:48.030 --> 00:40:49.410 them using edges in this way. 00:40:49.410 --> 00:40:50.980 OK. 00:40:50.980 --> 00:40:54.190 So, first step of Bellman-Ford is construct this graph. 00:40:58.980 --> 00:41:09.620 So, Bellman-Ford, construct G prime as described above. 00:41:09.620 --> 00:41:13.825 It has how many vertices? 00:41:16.980 --> 00:41:20.130 V times V plus 1. 00:41:20.130 --> 00:41:22.830 V times V plus 1 vertices. 00:41:22.830 --> 00:41:24.450 And how many edges? 00:41:24.450 --> 00:41:29.820 Well, I have one edge for outgoing edge for each vertex 00:41:29.820 --> 00:41:34.220 corresponding to just staying in the same place. 00:41:34.220 --> 00:41:37.880 So that's V squared vertices-- 00:41:37.880 --> 00:41:39.380 I mean edges. 00:41:39.380 --> 00:41:42.410 And then I have one edge-- 00:41:42.410 --> 00:41:45.540 for every edge in my graph, I have-- 00:41:45.540 --> 00:41:46.040 sorry. 00:41:46.040 --> 00:41:48.630 I have a V minus 1-- 00:41:48.630 --> 00:41:49.130 sorry. 00:41:49.130 --> 00:41:53.420 Just V. I have V edges for every edge in my graph. 00:41:53.420 --> 00:41:56.090 So that means-- so this is the number of vertices. 00:42:00.230 --> 00:42:11.600 And V times V plus V times E, this 00:42:11.600 --> 00:42:18.710 is V V plus E. All right, cool. 00:42:18.710 --> 00:42:22.130 So that's how many edges I have. 00:42:22.130 --> 00:42:24.900 So constructed in that way, it's a DAG. 00:42:24.900 --> 00:42:29.360 If we only have edges going to increasing levels, 00:42:29.360 --> 00:42:31.115 then this thing can't have cycles, 00:42:31.115 --> 00:42:32.990 because otherwise that would mean there would 00:42:32.990 --> 00:42:34.620 be an edge pointing backwards. 00:42:34.620 --> 00:42:37.090 And we didn't construct any of those. 00:42:37.090 --> 00:42:37.930 All right. 00:42:37.930 --> 00:42:40.750 So we construct this graph G prime. 00:42:40.750 --> 00:42:43.620 We can do that in linear time with respect to these things. 00:42:43.620 --> 00:42:46.630 I just go through all the edges, I make these edges, 00:42:46.630 --> 00:42:47.860 and I make these vertices. 00:42:47.860 --> 00:42:49.630 It doesn't take anything-- 00:42:49.630 --> 00:42:51.130 I just do it naively. 00:42:51.130 --> 00:42:59.270 Right I can do that in time V times V plus E asymptotically. 00:42:59.270 --> 00:43:01.160 OK. 00:43:01.160 --> 00:43:08.210 Now I run DAG relaxation, our nice algorithm 00:43:08.210 --> 00:43:13.625 we had last time, from-- 00:43:16.490 --> 00:43:17.840 in there was a0. 00:43:17.840 --> 00:43:22.600 I'm going to say it's S0, our source. 00:43:22.600 --> 00:43:24.490 Our source vertex. 00:43:24.490 --> 00:43:27.940 Single source shortest paths. 00:43:27.940 --> 00:43:44.670 So that I compute delta of S0 to Vk for all k and-- 00:43:44.670 --> 00:43:46.470 what is it? 00:43:46.470 --> 00:43:54.430 0 to V. That's what single source shortest paths does. 00:43:54.430 --> 00:43:58.060 It computes for me this distance from my source-- 00:43:58.060 --> 00:44:01.090 at some source to every other vertex in the graph. 00:44:01.090 --> 00:44:04.090 And so in particular I get these. 00:44:04.090 --> 00:44:05.440 Well, that is all of them. 00:44:08.350 --> 00:44:21.240 Then for each vertex V, set-- 00:44:21.240 --> 00:44:27.960 the thing I'm going to return, d-value, S to V, 00:44:27.960 --> 00:44:31.160 equal to the shortest-path distance 00:44:31.160 --> 00:44:34.340 I got from DAG relaxation to a particular vertex. 00:44:40.640 --> 00:44:45.140 V V minus 1. 00:44:45.140 --> 00:44:47.780 Why am I doing this? 00:44:47.780 --> 00:44:51.980 I'm setting it to be the shortest-path distance 00:44:51.980 --> 00:44:56.330 to the guy in the second-to-last row here 00:44:56.330 --> 00:45:02.440 or column in my modified graph. 00:45:05.930 --> 00:45:13.390 The hope is that this distance in my DAG 00:45:13.390 --> 00:45:23.020 corresponds to this distance in my original graph. 00:45:23.020 --> 00:45:27.560 The distance to V using at most V minus 1 edges. 00:45:27.560 --> 00:45:29.560 So that's the claim-- that's a claim we're going 00:45:29.560 --> 00:45:30.830 to prove in just a second. 00:45:30.830 --> 00:45:33.790 I'm going to write it down just so that we have-- 00:45:33.790 --> 00:45:36.490 just to continue our train of thought. 00:45:36.490 --> 00:45:51.670 Claim, delta S0 Vk equals delta k, the k edge distance, from S 00:45:51.670 --> 00:45:57.280 to V. That's what we want to claim. 00:45:57.280 --> 00:45:59.530 That would then-- what would that mean, then? 00:45:59.530 --> 00:46:01.930 That would mean that I'm correctly 00:46:01.930 --> 00:46:04.300 setting the shortest-path distance here 00:46:04.300 --> 00:46:09.060 for all vertices whose distances finite. 00:46:09.060 --> 00:46:09.670 Great. 00:46:09.670 --> 00:46:14.160 I mean, I set values to things where they're not finite, 00:46:14.160 --> 00:46:16.380 where they're minus infinity also, 00:46:16.380 --> 00:46:22.150 but in particular I set the ones correctly if they're finite. 00:46:22.150 --> 00:46:22.780 OK. 00:46:22.780 --> 00:46:24.280 So the last thing we need to do is 00:46:24.280 --> 00:46:28.150 deal with these minus infinity vertices. 00:46:30.690 --> 00:46:32.880 But we know how to do that. 00:46:32.880 --> 00:46:34.230 We just look at the witnesses. 00:46:34.230 --> 00:46:42.620 Because we've computed this value for k 00:46:42.620 --> 00:46:46.490 equals V equals V minus 1, and if that claim over there 00:46:46.490 --> 00:46:50.030 is true, then those shortest-path distances 00:46:50.030 --> 00:46:55.310 are the same as these k edge shortest-path distances. 00:46:55.310 --> 00:46:57.310 And we can just, for every vertex, 00:46:57.310 --> 00:46:58.690 we compare these things. 00:46:58.690 --> 00:47:02.910 If this is satisfied, we got a witness. 00:47:02.910 --> 00:47:03.490 OK. 00:47:03.490 --> 00:47:30.880 So for each witness U and V where delta S0 U 00:47:30.880 --> 00:47:39.610 V is less than, strictly, S0 U V minus 1-- 00:47:39.610 --> 00:47:42.370 that's the definition of a witness here, 00:47:42.370 --> 00:47:45.670 close the parentheses. 00:47:45.670 --> 00:47:58.390 Then for each vertex V reachable from U set-- 00:48:00.910 --> 00:48:05.410 sorry, d, is what we're returning, d of S, V 00:48:05.410 --> 00:48:07.470 equal to minus infinity. 00:48:07.470 --> 00:48:09.370 That's the end of the algorithm. 00:48:09.370 --> 00:48:12.850 Basically I'm looking for all the witnesses. 00:48:12.850 --> 00:48:16.390 For each witness, I find all of the vertices reachable from it 00:48:16.390 --> 00:48:20.150 and set it to minus infinity just as we argued before. 00:48:20.150 --> 00:48:20.740 OK. 00:48:20.740 --> 00:48:23.800 So, it remains to prove this claim. 00:48:28.420 --> 00:48:31.870 How do we prove this claim? 00:48:31.870 --> 00:48:37.020 Well, we can induct on k. 00:48:37.020 --> 00:48:39.680 Is this true for k equals 0? 00:48:39.680 --> 00:48:40.340 Yeah. 00:48:40.340 --> 00:48:42.680 We kind of already argued it over here 00:48:42.680 --> 00:48:45.740 when we are talking about our initialization step 00:48:45.740 --> 00:48:48.560 or what DAG relaxation does. 00:48:48.560 --> 00:48:51.890 It'll set this to be the shortest path from this guy 00:48:51.890 --> 00:48:53.570 to all these vertices. 00:48:53.570 --> 00:48:55.250 These aren't reachable from here, 00:48:55.250 --> 00:48:57.240 and so these are infinite. 00:48:57.240 --> 00:49:00.370 And that one's 0. 00:49:00.370 --> 00:49:03.290 So the base case-- 00:49:03.290 --> 00:49:14.970 so induct on k base case, k equals 0? 00:49:14.970 --> 00:49:16.980 Check. 00:49:16.980 --> 00:49:18.120 That's all good. 00:49:18.120 --> 00:49:23.790 Now we-- in our inductive step, let's take a look 00:49:23.790 --> 00:49:26.250 at the shortest-path distance from 0-- 00:49:26.250 --> 00:49:32.310 from S0 to V of k prime for some k prime. 00:49:32.310 --> 00:49:35.040 And the assumption is, the inductive hypothesis 00:49:35.040 --> 00:49:42.390 is that this distance is the k edge distance for all k prime 00:49:42.390 --> 00:49:43.230 less than-- 00:49:43.230 --> 00:49:47.100 I mean all k less than k prime. 00:49:47.100 --> 00:49:50.190 Well, kind of by definition of a shortest path, 00:49:50.190 --> 00:49:55.650 this is the minimum overall incoming vertices 00:49:55.650 --> 00:50:05.300 of the shortest path from S0 to U of k prime minus 1 00:50:05.300 --> 00:50:10.850 plus the weight of the edge from U of k prime 00:50:10.850 --> 00:50:21.140 minus to Vk prime for all Uk prime minus 1 00:50:21.140 --> 00:50:29.040 in the adjacencies, the incoming adjacencies of Vk prime. 00:50:29.040 --> 00:50:31.110 OK, what does this mean? 00:50:31.110 --> 00:50:34.840 I'm just saying in my graph G prime, 00:50:34.840 --> 00:50:37.870 a shortest path to this vertex needs 00:50:37.870 --> 00:50:43.040 to go first through some vertex in the layer before it, 00:50:43.040 --> 00:50:44.110 which is one of these. 00:50:44.110 --> 00:50:45.730 And in particular, I'm only connected 00:50:45.730 --> 00:50:49.700 to things adjacent to me. 00:50:49.700 --> 00:50:51.050 That's all this is saying. 00:50:51.050 --> 00:50:53.720 I have to go through that vertex and take some shortest 00:50:53.720 --> 00:50:57.610 path to one of those previous vertices. 00:50:57.610 --> 00:51:03.460 Now s actuality, these adjacencies, I constructed them 00:51:03.460 --> 00:51:07.540 to be similar to the adjacencies in my original graph 00:51:07.540 --> 00:51:11.440 in addition to one edge coming from my original vertex, 00:51:11.440 --> 00:51:17.710 from vertex V. So this is the same as the minimum of this set 00:51:17.710 --> 00:51:19.165 delta S0. 00:51:22.210 --> 00:51:22.960 Same thing. 00:51:26.730 --> 00:51:43.130 Plus W U, V for all U in the adjacent-- incoming adjacencies 00:51:43.130 --> 00:51:46.460 of my original vertex. 00:51:46.460 --> 00:51:47.890 In addition to one more term. 00:51:47.890 --> 00:51:48.890 What is that other term? 00:51:54.410 --> 00:51:56.200 These are all of the things corresponding 00:51:56.200 --> 00:51:59.200 to my incoming edges in my original thing, 00:51:59.200 --> 00:52:03.360 but I also have that one edge coming from the V before it. 00:52:03.360 --> 00:52:05.590 So this is-- I'm going to union-- 00:52:05.590 --> 00:52:16.600 union-- union this with delta S0 V of k prime minus 1. 00:52:16.600 --> 00:52:18.130 Awful. 00:52:18.130 --> 00:52:20.520 I think there's another one here. 00:52:20.520 --> 00:52:25.170 This is S0 of k prime minus 1. 00:52:25.170 --> 00:52:27.250 I"m not going to rewrite it. 00:52:27.250 --> 00:52:27.750 OK. 00:52:34.110 --> 00:52:38.040 Then by induction, this thing and this thing 00:52:38.040 --> 00:52:45.190 must be the edge shortest paths using k minus 1 vertices. 00:52:45.190 --> 00:52:49.330 And then that's just the statement of what the shortest 00:52:49.330 --> 00:52:57.730 path should be using at most k prime edges going 00:52:57.730 --> 00:53:06.000 from S to V. So, these things are the same as we claimed. 00:53:06.000 --> 00:53:08.650 Yay, check. 00:53:08.650 --> 00:53:09.220 All right. 00:53:09.220 --> 00:53:11.300 And then it's not such-- 00:53:11.300 --> 00:53:13.030 it's kind of a trivial leap, then, 00:53:13.030 --> 00:53:19.880 to say that at the end of Bellman-Ford, these guys-- 00:53:19.880 --> 00:53:23.390 sorry, the things that we return, these guys, 00:53:23.390 --> 00:53:26.360 are the shortest-path distances, because here, 00:53:26.360 --> 00:53:30.530 if they're finite, we set them to their true shortest-path 00:53:30.530 --> 00:53:34.980 distance; and if they're minus infinity, 00:53:34.980 --> 00:53:40.050 that invariant means that these things correspond to exactly 00:53:40.050 --> 00:53:41.340 this claim over here. 00:53:44.830 --> 00:53:45.565 It's a witness. 00:53:49.060 --> 00:53:51.430 And then finding all the vertices reachable 00:53:51.430 --> 00:53:56.440 from those witnesses, we set all of the infinite ones 00:53:56.440 --> 00:53:58.840 to be minus infinity as desired. 00:53:58.840 --> 00:54:02.610 OK, so what's the running time of this thing? 00:54:02.610 --> 00:54:04.720 Well, we had to construct this graph, 00:54:04.720 --> 00:54:06.930 so we had to take that time. 00:54:06.930 --> 00:54:10.770 We ran DAG relaxation, that takes the same amount of time. 00:54:10.770 --> 00:54:14.400 For every vertex, we did order V at work. 00:54:14.400 --> 00:54:17.370 And then for each witness, how many could there be? 00:54:17.370 --> 00:54:21.330 And most, V. Checking the reachability of each vertex, 00:54:21.330 --> 00:54:24.300 that can be done in how long? 00:54:24.300 --> 00:54:27.420 Order E time. 00:54:27.420 --> 00:54:30.750 Because we don't need to consider the things that 00:54:30.750 --> 00:54:32.640 aren't connected to S-- 00:54:32.640 --> 00:54:35.290 or aren't connected to the witness. 00:54:35.290 --> 00:54:41.430 So this thing takes order V times E work. 00:54:41.430 --> 00:54:44.970 So we're upper-bounded by this time 00:54:44.970 --> 00:54:47.350 it took to construct the original graph 00:54:47.350 --> 00:54:53.000 and by the claim we had before, that takes V times E time. 00:54:53.000 --> 00:54:53.930 OK. 00:54:53.930 --> 00:54:56.381 So that's Bellman-Ford. 00:54:56.381 --> 00:54:59.415 I'm just going to leave you with two nuggets. 00:55:03.910 --> 00:55:12.150 First, the shortest path, if for any witness-- 00:55:12.150 --> 00:55:13.530 let's say we have a witness here. 00:55:13.530 --> 00:55:16.020 Do I have any witnesses here? 00:55:16.020 --> 00:55:18.920 I didn't fill in all these. 00:55:18.920 --> 00:55:23.060 But is there a vertex on this cycle 00:55:23.060 --> 00:55:27.830 that goes through who has the shortest path? 00:55:27.830 --> 00:55:32.000 That goes through four vertices that's smaller than any other. 00:55:32.000 --> 00:55:32.570 OK. 00:55:32.570 --> 00:55:37.850 I can go from a to c to b to d to b to c. 00:55:40.930 --> 00:55:43.030 And you can work out this algorithm-- 00:55:43.030 --> 00:55:45.580 I have it in the notes that you can take a look at. 00:55:45.580 --> 00:55:50.450 This will actually have a shorter path for vertex-- 00:55:50.450 --> 00:55:51.160 sorry. 00:55:51.160 --> 00:55:55.070 It'll have a shorter path for vertex b. 00:56:01.010 --> 00:56:04.130 a to b to c to d to b. 00:56:04.130 --> 00:56:04.670 Thank you. 00:56:04.670 --> 00:56:09.000 That's a path of length 4, of four edges. 00:56:09.000 --> 00:56:13.640 That has shorter path than any path that has fewer edges. 00:56:13.640 --> 00:56:15.390 In particular, there's only one other path 00:56:15.390 --> 00:56:17.970 to b using fewer than four-- 00:56:17.970 --> 00:56:20.100 there's two other paths. 00:56:20.100 --> 00:56:21.900 One path of length-- 00:56:21.900 --> 00:56:24.390 that has one edge, that has weight minus 5, 00:56:24.390 --> 00:56:29.580 and one path-- this path that has weight 9 minus 1 is 8. 00:56:29.580 --> 00:56:34.590 Whereas this path, minus 5, minus 4, 3, minus 1 00:56:34.590 --> 00:56:41.610 has minus 10 plus 3 is minus 7, which is shorter than minus 5. 00:56:41.610 --> 00:56:45.060 So b and d is a witness. 00:56:45.060 --> 00:56:48.660 And if we actually take a look at that path 00:56:48.660 --> 00:57:03.400 through this graph, going from a to b to c to d back to b 00:57:03.400 --> 00:57:08.250 we see that there's a negative weight cycle in this graph. 00:57:08.250 --> 00:57:10.960 b to c to d to b. 00:57:10.960 --> 00:57:15.600 And indeed, that's always the case for our witnesses. 00:57:15.600 --> 00:57:17.820 You can see a proof of that in the notes, 00:57:17.820 --> 00:57:22.690 and you can see in recitation a little space optimization 00:57:22.690 --> 00:57:26.770 to make us not have to construct this entire graph on the fly, 00:57:26.770 --> 00:57:30.290 but actually only use order V space while they're going. 00:57:30.290 --> 00:57:30.790 OK. 00:57:30.790 --> 00:57:32.260 So that's Bellman-Ford. 00:57:32.260 --> 00:57:34.800 Sorry for running a little late.