1 00:00:00,000 --> 00:00:01,960 [SQUEAKING] 2 00:00:01,960 --> 00:00:03,920 [RUSTLING] 3 00:00:03,920 --> 00:00:07,350 [CLICKING] 4 00:00:12,760 --> 00:00:14,540 JASON KU: OK, let's get started. 5 00:00:14,540 --> 00:00:19,270 Welcome to the 12th lecture of 6.006. 6 00:00:19,270 --> 00:00:24,100 This is our second lecture talking about weighted graphs, 7 00:00:24,100 --> 00:00:27,670 and in particular, weighted shortest paths, algorithms. 8 00:00:27,670 --> 00:00:30,740 Last time we talked about weighted graphs. 9 00:00:30,740 --> 00:00:32,560 This is a kind of a generalization 10 00:00:32,560 --> 00:00:36,610 of what we mean by distance in an unweighted graph instead 11 00:00:36,610 --> 00:00:40,900 of each edge having a weight of 1, essentially. 12 00:00:40,900 --> 00:00:44,440 We generalize that to be any integer. 13 00:00:44,440 --> 00:00:48,790 And last time, we showed how to solve 14 00:00:48,790 --> 00:00:51,790 shortest single-source shortest paths in a graph that 15 00:00:51,790 --> 00:00:54,130 doesn't have cycles even if it has 16 00:00:54,130 --> 00:00:58,540 0 or negative weights in linear time using an algorithm called 17 00:00:58,540 --> 00:01:00,430 DAG relaxation. 18 00:01:00,430 --> 00:01:04,180 We also showed in that lecture how in linear time, 19 00:01:04,180 --> 00:01:07,570 if we are given the shortest path weights to all the things 20 00:01:07,570 --> 00:01:09,940 reachable in finite-- 21 00:01:09,940 --> 00:01:14,380 or with shortest path distance that's finite, 22 00:01:14,380 --> 00:01:16,960 we can construct a shortest paths tree 23 00:01:16,960 --> 00:01:20,380 from those weights in linear time. 24 00:01:20,380 --> 00:01:23,050 So this is motivating why we're not really 25 00:01:23,050 --> 00:01:25,960 going to talk about parent pointers for the next couple 26 00:01:25,960 --> 00:01:26,800 of lectures. 27 00:01:26,800 --> 00:01:30,940 We're just going to concentrate on the shortest path weights. 28 00:01:30,940 --> 00:01:32,950 And so today, we're going to be talking 29 00:01:32,950 --> 00:01:35,950 about our most general algorithm we'll 30 00:01:35,950 --> 00:01:39,280 be showing for solving single source shortest paths, 31 00:01:39,280 --> 00:01:42,940 in particular in graphs that could contain cycles 32 00:01:42,940 --> 00:01:45,280 and could have negative weights. 33 00:01:45,280 --> 00:01:50,930 So just to recap our little roadmap here, 34 00:01:50,930 --> 00:01:53,910 single source shortest paths in linear time. 35 00:01:53,910 --> 00:01:56,440 Last time we discussed another linear time algorithm, 36 00:01:56,440 --> 00:01:58,240 DAG relaxation. 37 00:01:58,240 --> 00:01:59,800 And today we're going to be talking 38 00:01:59,800 --> 00:02:04,240 about Bellman-Ford, which isn't limited to asymptotic graphs. 39 00:02:04,240 --> 00:02:07,600 In particular, there could be negative weight cycles 40 00:02:07,600 --> 00:02:09,460 in our graph. 41 00:02:09,460 --> 00:02:12,970 If it has cycles, if it has negative weights, 42 00:02:12,970 --> 00:02:15,820 the worry is that we could have negative weight cycles, 43 00:02:15,820 --> 00:02:17,500 in which case there-- 44 00:02:17,500 --> 00:02:22,420 if a negative weight cycle is reachable from our source, 45 00:02:22,420 --> 00:02:24,640 then the vertices in that cycle and anything 46 00:02:24,640 --> 00:02:28,420 reachable from that cycle will potentially 47 00:02:28,420 --> 00:02:34,540 have an unbounded number of edges you need to go through. 48 00:02:34,540 --> 00:02:38,560 There's not a bound on the number of edges for a shortest 49 00:02:38,560 --> 00:02:41,470 path, because I could just keep going around 50 00:02:41,470 --> 00:02:44,740 that cycle as many times as I want and get a shorter path. 51 00:02:44,740 --> 00:02:49,558 And so we assign those distances to be minus infinity. 52 00:02:49,558 --> 00:02:51,850 So that's what we're going to do today in Bellman-Ford. 53 00:02:51,850 --> 00:02:57,400 In particular, what we're going to do is compute our shortest 54 00:02:57,400 --> 00:02:59,440 path distances, the shortest path 55 00:02:59,440 --> 00:03:03,760 waits for every vertex in our graph, 56 00:03:03,760 --> 00:03:07,030 setting the ones that are not reachable to infinity, 57 00:03:07,030 --> 00:03:10,300 and the ones that are reachable through a negative weight 58 00:03:10,300 --> 00:03:13,450 cycle to minus infinity, and all other ones we're going 59 00:03:13,450 --> 00:03:16,420 to set to a finite weight. 60 00:03:16,420 --> 00:03:19,630 And another thing that we might want 61 00:03:19,630 --> 00:03:21,340 is if there's a negative weight cycle 62 00:03:21,340 --> 00:03:24,130 in the graph, let's return one. 63 00:03:24,130 --> 00:03:25,630 So those are the two kinds of things 64 00:03:25,630 --> 00:03:29,350 that we're trying to solve in today's lecture. 65 00:03:29,350 --> 00:03:31,510 But before we do that, let's warm up 66 00:03:31,510 --> 00:03:36,020 with two short exercises. 67 00:03:36,020 --> 00:03:48,110 The first one, exercise 1, given an undirected graph, 68 00:03:48,110 --> 00:04:12,280 given undirected graph G, return whether G contains 69 00:04:12,280 --> 00:04:13,450 a negative weight cycle. 70 00:04:20,140 --> 00:04:23,740 Anyone have an idea of how we can solve this in linear time, 71 00:04:23,740 --> 00:04:25,030 actually? 72 00:04:25,030 --> 00:04:30,280 In fact, we can do it in order E. No-- yes, yes. 73 00:04:30,280 --> 00:04:34,740 Reachable from S. I guess-- 74 00:04:34,740 --> 00:04:40,110 let's just say a negative weight cycle at all. 75 00:04:40,110 --> 00:04:42,787 Not in the context of single-source shortest paths. 76 00:04:42,787 --> 00:04:45,120 AUDIENCE: Detect whether there's a negative weight edge? 77 00:04:45,120 --> 00:04:46,230 JASON KU: Ah. 78 00:04:46,230 --> 00:04:50,610 Your colleague has determined an interesting fact 79 00:04:50,610 --> 00:04:53,250 about undirected graphs. 80 00:04:53,250 --> 00:04:57,720 If you have a negative weight edge in an undirected graph, 81 00:04:57,720 --> 00:05:01,170 I can just move back and forth along that edge. 82 00:05:01,170 --> 00:05:03,270 That's a cycle of length 2-- 83 00:05:03,270 --> 00:05:07,335 or I guess three vertices back to where we came from. 84 00:05:07,335 --> 00:05:09,210 There is of negative weight, because I'm just 85 00:05:09,210 --> 00:05:13,240 traversing that weight over and over and over again. 86 00:05:13,240 --> 00:05:16,320 So the question of single-source shortest paths 87 00:05:16,320 --> 00:05:19,050 of finding negative weights is not 88 00:05:19,050 --> 00:05:22,780 particularly interesting in the undirected case. 89 00:05:22,780 --> 00:05:26,070 What I can do is just for every negative weight edge, 90 00:05:26,070 --> 00:05:31,050 undirected edge in my graph, I can just find the readability 91 00:05:31,050 --> 00:05:33,300 from the vertices-- 92 00:05:33,300 --> 00:05:40,200 the endpoints of that edge and label them as minus-- 93 00:05:40,200 --> 00:05:43,410 basically if the connected component containing S 94 00:05:43,410 --> 00:05:47,160 has a negative weight edge, then everything in the graph 95 00:05:47,160 --> 00:05:49,080 is accessible from a negative weight cycle. 96 00:05:49,080 --> 00:05:51,970 So this is not such an interesting problem. 97 00:05:51,970 --> 00:06:00,540 And so we're going to restrict our discussion today 98 00:06:00,540 --> 00:06:02,685 to directed graphs. 99 00:06:09,020 --> 00:06:20,920 So this is if and only if exists negative weight edge. 100 00:06:24,700 --> 00:06:32,770 OK, exercise 2, kind of a little preview for what's to come, 101 00:06:32,770 --> 00:06:36,460 we're actually not going to show you an algorithm directly that 102 00:06:36,460 --> 00:06:40,050 meets this Bellman-Ford running time, V times E. 103 00:06:40,050 --> 00:06:44,080 What instead we're going to show you is an algorithm that solves 104 00:06:44,080 --> 00:06:47,080 single-source shortest paths in-- 105 00:06:47,080 --> 00:06:54,070 So given an algorithm, Alg A, solves 106 00:06:54,070 --> 00:07:04,522 single-source shortest paths in order V times V plus E time. 107 00:07:04,522 --> 00:07:05,230 OK, what is that? 108 00:07:05,230 --> 00:07:09,430 That's V squared plus V times E. That's 109 00:07:09,430 --> 00:07:13,510 close to what this V times E is. 110 00:07:13,510 --> 00:07:15,940 That's what we're going to show you. 111 00:07:15,940 --> 00:07:19,870 But if I had such an algorithm, can anyone 112 00:07:19,870 --> 00:07:22,600 tell me a single-source shortest paths algorithm-- 113 00:07:22,600 --> 00:07:26,560 how we can use this algorithm to solve single-source shortest 114 00:07:26,560 --> 00:07:29,320 paths in just V times E time? 115 00:07:35,730 --> 00:07:48,990 Show how to solve SSSP in order the V times E. I 116 00:07:48,990 --> 00:07:52,740 guess we can put a dot there as well. 117 00:07:52,740 --> 00:07:55,080 So this is a little tricky. 118 00:07:55,080 --> 00:07:57,000 It's kind of related to the difference 119 00:07:57,000 --> 00:07:59,370 we had between the reachability problem 120 00:07:59,370 --> 00:08:01,980 and the single-source shortest paths problem 121 00:08:01,980 --> 00:08:05,310 that we saw last lecture. 122 00:08:05,310 --> 00:08:09,840 When are these asymptotically different in their upper bound 123 00:08:09,840 --> 00:08:17,590 is when V is asymptotically larger than E. 124 00:08:17,590 --> 00:08:22,350 But the connected component containing S 125 00:08:22,350 --> 00:08:27,090 can have at most E vertices, or order 126 00:08:27,090 --> 00:08:30,960 E. It can actually have at most E plus 1 vertices, 127 00:08:30,960 --> 00:08:37,169 because otherwise it wouldn't be connected. 128 00:08:37,169 --> 00:08:41,490 So, what we can do if we had such an algorithm, 129 00:08:41,490 --> 00:08:44,400 we could first, when we're giving our graph, explore 130 00:08:44,400 --> 00:08:48,120 everything in the graph using BFS or DFS, 131 00:08:48,120 --> 00:08:50,670 find all the things reachable from S, 132 00:08:50,670 --> 00:08:53,250 and then just throw away everything else. 133 00:08:53,250 --> 00:08:56,700 Now I have a graph for which V is asymptotically 134 00:08:56,700 --> 00:09:00,870 no bigger than E, and then we can use this algorithm 135 00:09:00,870 --> 00:09:04,530 to solve single-source shortest paths in V times E time. 136 00:09:04,530 --> 00:09:06,330 I'm not going to write all that down here. 137 00:09:06,330 --> 00:09:07,860 You can see it in the notes. 138 00:09:07,860 --> 00:09:08,440 Yeah? 139 00:09:08,440 --> 00:09:10,913 AUDIENCE: Does this work if your graph isn't simple? 140 00:09:10,913 --> 00:09:13,080 JASON KU: Does this work as your graph isn't simple? 141 00:09:13,080 --> 00:09:14,290 I haven't thought about it. 142 00:09:14,290 --> 00:09:16,290 We are not going to talk about non-simple graphs 143 00:09:16,290 --> 00:09:20,190 in this class, but probably not because you've 144 00:09:20,190 --> 00:09:21,780 got a lot of edges. 145 00:09:21,780 --> 00:09:23,940 Though in our class if we're talking 146 00:09:23,940 --> 00:09:26,100 about single-source shortest paths, 147 00:09:26,100 --> 00:09:28,830 if we have multiple edges between two vertices, 148 00:09:28,830 --> 00:09:30,750 we can just take the minimum weight one 149 00:09:30,750 --> 00:09:33,765 because it's never better to take the larger ones. 150 00:09:33,765 --> 00:09:35,430 Does that answer your question? 151 00:09:35,430 --> 00:09:36,170 Great. 152 00:09:36,170 --> 00:09:36,670 All right. 153 00:09:36,670 --> 00:09:39,687 So those are our warm-ups, that's our goal. 154 00:09:39,687 --> 00:09:42,270 We need to find an algorithm for single-source shortest paths. 155 00:09:42,270 --> 00:09:44,370 And general graphs, graphs with-- 156 00:09:44,370 --> 00:09:49,020 potentially graphs with cycles, and negative weights, 157 00:09:49,020 --> 00:09:54,270 and solve it in this V times linear kind of time. 158 00:09:54,270 --> 00:09:55,800 That makes sense? 159 00:09:55,800 --> 00:09:56,610 All right. 160 00:09:56,610 --> 00:10:00,150 So first, before we get to the algorithm, 161 00:10:00,150 --> 00:10:02,250 we're going to discuss a little bit 162 00:10:02,250 --> 00:10:06,670 about simple short-- about shortest paths in general. 163 00:10:06,670 --> 00:10:10,140 If we didn't-- the problem here is negative weights. 164 00:10:10,140 --> 00:10:11,650 How do we find-- 165 00:10:11,650 --> 00:10:14,040 if we had negative weight cycles, 166 00:10:14,040 --> 00:10:15,520 there seems to be these problems, 167 00:10:15,520 --> 00:10:20,360 because we could have minus infinities is in our deltas. 168 00:10:20,360 --> 00:10:22,670 But if we didn't have negative weights, 169 00:10:22,670 --> 00:10:26,360 I'd like to assert to you that our shortest paths, even 170 00:10:26,360 --> 00:10:29,580 if there are negative weights, are going to be simple. 171 00:10:29,580 --> 00:10:31,010 They won't repeat vertices. 172 00:10:31,010 --> 00:10:33,590 So that's the first thing we're going to show you. 173 00:10:38,520 --> 00:10:40,050 Let's see. 174 00:10:40,050 --> 00:10:45,235 Simple shortest paths. 175 00:10:49,390 --> 00:10:51,130 OK. 176 00:10:51,130 --> 00:10:52,740 So, claim. 177 00:10:55,660 --> 00:10:58,240 I'm going to give my claims numbers 178 00:10:58,240 --> 00:11:01,180 today just because I'm going to have a lot of them. 179 00:11:01,180 --> 00:11:06,580 If my shortest path distance from S to some vertex 180 00:11:06,580 --> 00:11:13,750 is finite, meaning it's not infinite or minus infinite-- 181 00:11:13,750 --> 00:11:25,315 some finite value, there exists a shortest path-- 182 00:11:28,110 --> 00:11:37,270 a shortest S to V path that is simple. 183 00:11:37,270 --> 00:11:40,030 And remember, simple means not going through a vertex 184 00:11:40,030 --> 00:11:41,290 more than once. 185 00:11:41,290 --> 00:11:42,958 All right. 186 00:11:42,958 --> 00:11:44,250 How are we going to prove this? 187 00:11:49,390 --> 00:11:55,960 Well, consider if this claim was not true. 188 00:11:55,960 --> 00:12:01,625 If every shortest path contained a cycle, essentially. 189 00:12:01,625 --> 00:12:03,830 It repeated a vertex. 190 00:12:03,830 --> 00:12:07,910 Then my path looks something like this. 191 00:12:11,350 --> 00:12:16,930 I mean, there's some vertices along here, and then I go to V. 192 00:12:16,930 --> 00:12:19,690 So here's S, and then there's some cycle I 193 00:12:19,690 --> 00:12:21,470 repeat, some vertex. 194 00:12:21,470 --> 00:12:26,310 I'm going to call this cycle C. 195 00:12:26,310 --> 00:12:28,920 Now what do I know about this path? 196 00:12:28,920 --> 00:12:34,080 I know that it has-- it's a shortest path 197 00:12:34,080 --> 00:12:36,560 and it has finite weight. 198 00:12:36,560 --> 00:12:39,950 So in particular, this path-- this delta distance 199 00:12:39,950 --> 00:12:41,820 is not minus infinity. 200 00:12:41,820 --> 00:12:45,010 But if this is not minus infinity, 201 00:12:45,010 --> 00:12:48,538 what do I know about the weight of this cycle? 202 00:12:48,538 --> 00:12:49,910 AUDIENCE: It's not negative. 203 00:12:49,910 --> 00:12:50,535 JASON KU: Yeah. 204 00:12:50,535 --> 00:12:51,932 It can't be negative. 205 00:12:51,932 --> 00:12:53,390 Because if it was negative, I could 206 00:12:53,390 --> 00:12:55,580 keep going around this cycle, and this 207 00:12:55,580 --> 00:12:58,370 would have a non-finite weight. 208 00:12:58,370 --> 00:13:03,350 Shortest path distance from S. So I know this is-- 209 00:13:03,350 --> 00:13:06,860 can't be negative, so it must be 0 or positive. 210 00:13:06,860 --> 00:13:10,880 But if it's 0 or positive and this is a shortest path-- 211 00:13:10,880 --> 00:13:16,960 went through this cycle, then I could remove it, 212 00:13:16,960 --> 00:13:21,360 and now I have a new path with one fewer cycle. 213 00:13:21,360 --> 00:13:26,130 I could just keep doing this to create a simple path. 214 00:13:26,130 --> 00:13:28,590 So that checks out. 215 00:13:28,590 --> 00:13:30,120 OK. 216 00:13:30,120 --> 00:13:33,060 So, that's interesting. 217 00:13:33,060 --> 00:13:39,000 If it's simple, what do we know about the number of edges 218 00:13:39,000 --> 00:13:42,570 in a simple shortest paths? 219 00:13:42,570 --> 00:13:45,740 How many could there possibly be? 220 00:13:45,740 --> 00:13:51,550 How long in number of edges could a simple shortest path 221 00:13:51,550 --> 00:13:52,050 be? 222 00:13:52,050 --> 00:13:55,610 If I can't repeat vertices, I can 223 00:13:55,610 --> 00:14:00,710 have at most vertices on my simple path, which 224 00:14:00,710 --> 00:14:04,850 means I can use at most V minus 1 edges-- 225 00:14:04,850 --> 00:14:06,590 fence posting. 226 00:14:06,590 --> 00:14:19,260 So, simple paths have at most V minus 1 edges. 227 00:14:25,520 --> 00:14:30,050 That's a nice little thing I'd like to box off. 228 00:14:30,050 --> 00:14:31,355 That's a really nice property. 229 00:14:33,920 --> 00:14:40,000 So while a shortest path here could have an infinite number 230 00:14:40,000 --> 00:14:45,310 of edges, if the shortest path distance is finite, 231 00:14:45,310 --> 00:14:48,560 I know I only have to check paths 232 00:14:48,560 --> 00:14:51,710 that use up to V minus 1 edges. 233 00:14:51,710 --> 00:14:56,710 In particular, this is finitely bounded in terms of the number 234 00:14:56,710 --> 00:14:57,950 of paths I have to consider. 235 00:14:57,950 --> 00:15:02,470 It's exponential, potentially, but at least it's finite. 236 00:15:02,470 --> 00:15:09,130 The other way, I potentially had to check every possible path 237 00:15:09,130 --> 00:15:12,740 of which there could be infinite if there's cycles in my graph. 238 00:15:12,740 --> 00:15:14,420 OK. 239 00:15:14,420 --> 00:15:19,930 So I have an idea. 240 00:15:19,930 --> 00:15:25,950 What if I could find shortest-path distances 241 00:15:25,950 --> 00:15:30,110 by limiting the number of edges I go through? 242 00:15:30,110 --> 00:15:35,525 So not the full shortest path distance from S 243 00:15:35,525 --> 00:15:38,870 to V, but let's limit the number of edges 244 00:15:38,870 --> 00:15:41,480 I'm allowed to go through, and let's 245 00:15:41,480 --> 00:15:43,970 talk about those shortest-path distances, 246 00:15:43,970 --> 00:15:46,640 just among the paths that have at most a certain number 247 00:15:46,640 --> 00:15:47,450 of edges. 248 00:15:47,450 --> 00:15:55,370 I'm going to call this k-edge distance. 249 00:15:55,370 --> 00:15:59,240 And I'm just going to provide a little notation here. 250 00:15:59,240 --> 00:16:02,340 Instead of having a delta, I'll have a delta k here. 251 00:16:02,340 --> 00:16:05,720 That means how many edges I'm limited by. 252 00:16:05,720 --> 00:16:24,335 So from S to V is shortest S to V path using at most k edges. 253 00:16:29,750 --> 00:16:36,800 Short-- weight-- weight of a. 254 00:16:36,800 --> 00:16:43,550 Shortest path, shortest S to V path using at most k edges. 255 00:16:43,550 --> 00:16:47,600 And these notions seem somewhat symmetric. 256 00:16:47,600 --> 00:16:59,700 If I were able to compute this thing for V minus 1, then-- 257 00:16:59,700 --> 00:17:06,430 for all the vertices, then if the distance is finite, 258 00:17:06,430 --> 00:17:08,910 then I'll have successfully computed 259 00:17:08,910 --> 00:17:12,569 the real shortest paths because of this statement. 260 00:17:12,569 --> 00:17:16,950 Now that doesn't mean that if this is-- 261 00:17:20,190 --> 00:17:21,900 it doesn't mean the other way. 262 00:17:21,900 --> 00:17:26,940 If this is minus infinity, if the shortest-path distance is 263 00:17:26,940 --> 00:17:31,820 minus infinity, it doesn't say anything about what this is. 264 00:17:31,820 --> 00:17:34,500 It just says the shortest path using 265 00:17:34,500 --> 00:17:40,200 at most V minus 1 vertices, using at most V minus 1 edges 266 00:17:40,200 --> 00:17:41,970 would be whatever this is. 267 00:17:41,970 --> 00:17:43,710 But really, the shortest path length 268 00:17:43,710 --> 00:17:46,620 needs to consider an infinite number of edges. 269 00:17:46,620 --> 00:17:49,840 So it doesn't really tell us much about that. 270 00:17:49,840 --> 00:17:51,460 But for the finite ones it does. 271 00:17:51,460 --> 00:17:53,050 It works well. 272 00:17:53,050 --> 00:17:56,550 And so if we are able to compute this thing for k equals 273 00:17:56,550 --> 00:18:01,850 V minus 1 in a graph that doesn't contain negative weight 274 00:18:01,850 --> 00:18:02,780 cycles, we'd be done. 275 00:18:06,830 --> 00:18:09,770 I claim to you a stronger statement, 276 00:18:09,770 --> 00:18:24,960 that if the shortest path using at most V edges from s to v Is 277 00:18:24,960 --> 00:18:29,940 less than-- strictly less than delta of V minus 1-- 278 00:18:33,660 --> 00:18:37,720 this is all in the subscript here. 279 00:18:37,720 --> 00:18:43,600 Basically this is the shortest-path distance 280 00:18:43,600 --> 00:18:49,240 of any simple path, and possibly ones that also contain cycles, 281 00:18:49,240 --> 00:18:53,760 but definitely it includes all the simple paths. 282 00:18:53,760 --> 00:18:58,590 If there's a shorter path to my vertex 283 00:18:58,590 --> 00:19:02,850 that goes through more than V minus 1 edges, 284 00:19:02,850 --> 00:19:06,810 that this path can't be simple, because it 285 00:19:06,810 --> 00:19:08,850 goes through a vertex more than once. 286 00:19:08,850 --> 00:19:13,540 Otherwise it would be included in this distance set. 287 00:19:13,540 --> 00:19:18,010 So if this is the case, and I found a shorter path 288 00:19:18,010 --> 00:19:21,790 to V that uses v edges-- 289 00:19:21,790 --> 00:19:23,980 yeah, that use V edges, that path 290 00:19:23,980 --> 00:19:28,520 can't be simple, which means that path or some path 291 00:19:28,520 --> 00:19:31,730 there contains a negative weight cycle. 292 00:19:31,730 --> 00:19:38,960 So if this is true, then I know that 293 00:19:38,960 --> 00:19:42,820 the real shortest-path distance from S to V 294 00:19:42,820 --> 00:19:45,830 must be minus infinity. 295 00:19:49,670 --> 00:19:53,360 I'm going to call such a vertex a witness. 296 00:19:53,360 --> 00:19:56,490 If we can find a vertex that has this property-- 297 00:19:56,490 --> 00:19:59,720 I mean, I haven't shown you how to compute these things yet, 298 00:19:59,720 --> 00:20:02,390 but if I were able to find a vertex V-- 299 00:20:02,390 --> 00:20:08,980 and these are capital V's if you're having trouble. 300 00:20:08,980 --> 00:20:13,150 This V is different than this V, this is cardinality. 301 00:20:13,150 --> 00:20:15,670 If we can find such a vertex V, that 302 00:20:15,670 --> 00:20:17,890 certifies that there is a negative weight 303 00:20:17,890 --> 00:20:21,590 cycle in our graph. 304 00:20:21,590 --> 00:20:25,925 So I'm going to call V is a witness. 305 00:20:30,630 --> 00:20:32,440 OK. 306 00:20:32,440 --> 00:20:38,720 So, if this property is true, it's a witness 307 00:20:38,720 --> 00:20:39,980 and it definitely has this. 308 00:20:39,980 --> 00:20:42,152 Is it possible, you think-- 309 00:20:42,152 --> 00:20:43,610 I'm going to claim to you that it's 310 00:20:43,610 --> 00:20:49,330 possible that a vertex could have minus infinite distance 311 00:20:49,330 --> 00:20:55,060 but not have this property halt. I could probably 312 00:20:55,060 --> 00:20:55,940 give you an example-- 313 00:20:55,940 --> 00:20:58,210 I don't have one off the top of my head right now, 314 00:20:58,210 --> 00:21:00,220 but that's possible. 315 00:21:00,220 --> 00:21:02,170 You could imagine, there might be 316 00:21:02,170 --> 00:21:07,630 no path going to a vertex on a negative weight cycle that 317 00:21:07,630 --> 00:21:10,510 goes through V exactly V edges. 318 00:21:10,510 --> 00:21:14,860 It might go through more edges, a shorter one. 319 00:21:14,860 --> 00:21:18,820 So this equation would be inequality 320 00:21:18,820 --> 00:21:23,530 and would not certify that this is true. 321 00:21:23,530 --> 00:21:28,210 But I claim to you, if a vertex has this property, 322 00:21:28,210 --> 00:21:33,010 if it's its shortest path distances minus infinite, 323 00:21:33,010 --> 00:21:35,470 then it must be reachable from a witness. 324 00:21:35,470 --> 00:21:36,400 So that's the claim. 325 00:21:41,010 --> 00:21:52,460 If delta S, V is minus infinity, then V 326 00:21:52,460 --> 00:22:02,560 is reachable from a witness. 327 00:22:02,560 --> 00:22:05,320 Reachable from a vertex that has this property-- that 328 00:22:05,320 --> 00:22:07,370 has this property. 329 00:22:07,370 --> 00:22:09,100 And if it's reachable from something 330 00:22:09,100 --> 00:22:13,000 that has minus infinity shortest pathway, 331 00:22:13,000 --> 00:22:16,940 then I can take that path go to my reachable vertex, 332 00:22:16,940 --> 00:22:20,470 and that's also minus infinite path. 333 00:22:20,470 --> 00:22:21,040 OK. 334 00:22:21,040 --> 00:22:22,180 So how do we prove this? 335 00:22:28,320 --> 00:22:37,800 Well, let's consider-- let's I'm going 336 00:22:37,800 --> 00:22:42,170 to state a somewhat stronger statement that we'll 337 00:22:42,170 --> 00:22:43,940 prove instead. 338 00:22:43,940 --> 00:22:47,420 It suffices to prove that every negative weight 339 00:22:47,420 --> 00:22:51,210 cycle contains a witness. 340 00:22:51,210 --> 00:22:55,860 If we are to prove that, then every vertex 341 00:22:55,860 --> 00:23:05,510 with this property, every vertex with this property 342 00:23:05,510 --> 00:23:10,840 is reachable from a negative weight cycle by definition. 343 00:23:10,840 --> 00:23:15,400 So, if we can prove that every-- 344 00:23:18,920 --> 00:23:36,510 prove every negative weight cycle contains witness. 345 00:23:39,620 --> 00:23:42,040 If we can prove that every negative weight cycle contains 346 00:23:42,040 --> 00:23:47,800 a witness, then every vertex reachable from one of those 347 00:23:47,800 --> 00:23:50,050 witnesses-- in particular, reachable from the negative 348 00:23:50,050 --> 00:23:50,650 weight cycle-- 349 00:23:53,850 --> 00:23:59,897 has shortest distance minus infinity, 350 00:23:59,897 --> 00:24:01,230 and that should prove the claim. 351 00:24:04,270 --> 00:24:06,900 This thing has to be reachable from a negative weight cycle. 352 00:24:12,700 --> 00:24:15,420 And so if we prove negative weight cycles contain 353 00:24:15,420 --> 00:24:19,232 witnesses, then all of these vertices 354 00:24:19,232 --> 00:24:20,440 are reachable from a witness. 355 00:24:20,440 --> 00:24:22,420 OK, great, great. 356 00:24:22,420 --> 00:24:24,230 Confusing myself there for a second. 357 00:24:24,230 --> 00:24:25,680 OK. 358 00:24:25,680 --> 00:24:30,820 So let's consider a negative weight cycle. 359 00:24:30,820 --> 00:24:32,590 NG. 360 00:24:32,590 --> 00:24:35,360 Here's a directed negative weight cycle. 361 00:24:35,360 --> 00:24:35,860 Recall. 362 00:24:38,425 --> 00:24:42,870 This will be my negative weight cycle C. 363 00:24:42,870 --> 00:24:45,760 All of the sum of the edges in this thing, 364 00:24:45,760 --> 00:24:48,100 the weights has negative weight. 365 00:24:50,920 --> 00:24:52,810 And I'm going to have a little bit notation-- 366 00:24:52,810 --> 00:24:56,560 if I have a vertex V here, I'm going 367 00:24:56,560 --> 00:25:00,220 to say that its predecessor in the cycle, I'm 368 00:25:00,220 --> 00:25:02,962 just going to call it V prime. 369 00:25:02,962 --> 00:25:04,045 That's just some notation. 370 00:25:06,830 --> 00:25:08,970 All right. 371 00:25:08,970 --> 00:25:15,480 So, if I have computed these shortest-path distances 372 00:25:15,480 --> 00:25:19,230 to every vertex in my graph, shortest-path distance going 373 00:25:19,230 --> 00:25:21,780 through at most V vertices and the shortest path 374 00:25:21,780 --> 00:25:25,000 distance going through at most V minus 1 vertices, 375 00:25:25,000 --> 00:25:27,420 then I know the following thing holds. 376 00:25:27,420 --> 00:25:35,600 Delta V going from S to V for any vertex in my cycle 377 00:25:35,600 --> 00:25:41,120 can't be bigger than delta V minus 1 378 00:25:41,120 --> 00:25:46,400 from S to U plus the weight-- 379 00:25:46,400 --> 00:25:50,520 sorry, not U-- V prime, its predecessor, 380 00:25:50,520 --> 00:25:56,100 plus the weight going from the predecessor to my vertex. 381 00:25:56,100 --> 00:25:58,210 Why is that? 382 00:25:58,210 --> 00:25:58,850 Why is that? 383 00:25:58,850 --> 00:26:02,020 Because this is the weight of some vertex-- 384 00:26:02,020 --> 00:26:03,940 this is the weight-- 385 00:26:03,940 --> 00:26:08,080 the shortest-path distance to my predecessor 386 00:26:08,080 --> 00:26:10,370 using one fewer edge. 387 00:26:10,370 --> 00:26:12,880 And so this in particular is the weight 388 00:26:12,880 --> 00:26:17,010 of some path that uses V edges. 389 00:26:17,010 --> 00:26:21,660 So if this is the shortest such path distance, 390 00:26:21,660 --> 00:26:24,870 this has to upper bound it at least-- 391 00:26:24,870 --> 00:26:25,500 at most. 392 00:26:25,500 --> 00:26:25,650 Yeah? 393 00:26:25,650 --> 00:26:27,400 AUDIENCE: Is that the triangle inequality? 394 00:26:27,400 --> 00:26:29,830 JASON KU: That is a statement of the triangle inequality, 395 00:26:29,830 --> 00:26:32,240 thank you. 396 00:26:32,240 --> 00:26:33,050 All right. 397 00:26:33,050 --> 00:26:37,460 So, yes, this is just by triangle inequality. 398 00:26:37,460 --> 00:26:38,720 OK. 399 00:26:38,720 --> 00:26:43,490 Now what we can say is, let's take this equation summed 400 00:26:43,490 --> 00:26:46,700 over all vertices in my cycle. 401 00:26:46,700 --> 00:26:51,260 So I'm just going to add summation 402 00:26:51,260 --> 00:26:57,350 here of all vertices in my cycle of this whole thing. 403 00:26:57,350 --> 00:27:01,400 I'm going to do that out a little bit neater. 404 00:27:01,400 --> 00:27:08,420 Summation of delta, not d. 405 00:27:08,420 --> 00:27:18,150 Delta V S, V. I guess I don't need this open parentheses. 406 00:27:18,150 --> 00:27:26,760 Equals-- or less than or equal to sum of V and C of delta 407 00:27:26,760 --> 00:27:30,915 V minus 1 V prime. 408 00:27:33,890 --> 00:27:38,690 And here, I'm summing over V and C, 409 00:27:38,690 --> 00:27:42,320 and this is just my notation for the predecessor. 410 00:27:42,320 --> 00:27:48,740 And then I'm going to sum over the weights in my cycle 411 00:27:48,740 --> 00:27:53,680 V and C. These are the sum of the weights in my cycle. 412 00:27:53,680 --> 00:27:57,090 Well, what do I know about this cycle? 413 00:27:57,090 --> 00:28:01,700 This is just the weight of C. The weight of C-- 414 00:28:01,700 --> 00:28:03,800 that's awful handwriting. 415 00:28:03,800 --> 00:28:06,960 C, what do I know about the weight of the cycle? 416 00:28:06,960 --> 00:28:09,240 It's negative. 417 00:28:09,240 --> 00:28:16,452 So, this is less than 0, which means that if I remove this, 418 00:28:16,452 --> 00:28:18,900 this needs to be a strict equality. 419 00:28:22,630 --> 00:28:26,740 But if the sum of all of these is strictly 420 00:28:26,740 --> 00:28:30,110 less than the sum of all these, we 421 00:28:30,110 --> 00:28:36,440 can't have none of the vertices in my graph satisfying-- 422 00:28:36,440 --> 00:28:38,030 not satisfying this property. 423 00:28:42,030 --> 00:28:45,390 If all of them are not witnesses, 424 00:28:45,390 --> 00:28:49,970 then this thing is bigger than this thing-- at least 425 00:28:49,970 --> 00:28:53,930 as big as this thing for every vertex in my cycle, which 426 00:28:53,930 --> 00:28:56,640 is a contradiction. 427 00:28:56,640 --> 00:29:02,560 So, the claim holds, if we have a negative infinite 428 00:29:02,560 --> 00:29:06,110 shortest-path distance, then V is reachable from a witness. 429 00:29:06,110 --> 00:29:09,190 So it suffices for us to find all the witnesses, 430 00:29:09,190 --> 00:29:12,970 find all the vertices reachable from the witnesses, 431 00:29:12,970 --> 00:29:16,050 and then mark them as minus infinity. 432 00:29:16,050 --> 00:29:18,020 Does that make sense? 433 00:29:18,020 --> 00:29:19,640 OK. 434 00:29:19,640 --> 00:29:26,610 So, now we finally are able to get to our algorithm. 435 00:29:26,610 --> 00:29:27,633 Bellman-Ford. 436 00:29:31,710 --> 00:29:34,830 And what I'm going to show you today 437 00:29:34,830 --> 00:29:37,020 is a little different than what is normally 438 00:29:37,020 --> 00:29:39,000 presented as Bellman-Ford. 439 00:29:39,000 --> 00:29:42,040 The original Bellman-Ford algorithm 440 00:29:42,040 --> 00:29:43,928 does something a little different. 441 00:29:43,928 --> 00:29:45,970 And because it does something a little different, 442 00:29:45,970 --> 00:29:48,130 which we'll talk about at the end, 443 00:29:48,130 --> 00:29:50,140 it's a little hairier to analyze. 444 00:29:50,140 --> 00:29:53,950 I'm going to show you a modification that 445 00:29:53,950 --> 00:29:58,270 is a little easier to analyze and has this nice property 446 00:29:58,270 --> 00:30:01,120 that we're going to be able to use the algorithm 447 00:30:01,120 --> 00:30:04,195 to give us a negative weight cycle if it exists. 448 00:30:06,920 --> 00:30:12,000 So, we're going to say this is maybe a modified Bellman-Ford. 449 00:30:12,000 --> 00:30:20,000 And the idea here is to make a vertex associate-- make 450 00:30:20,000 --> 00:30:23,660 many versions of a vertex. 451 00:30:23,660 --> 00:30:26,300 And I want this version of the vertex 452 00:30:26,300 --> 00:30:32,360 to correspond to whether I came here using 0 edges, 1 edge, 2 453 00:30:32,360 --> 00:30:34,100 edges, 3 edges-- 454 00:30:34,100 --> 00:30:35,600 I have a different vertex version 455 00:30:35,600 --> 00:30:39,530 of the vertex for each one of these-- 456 00:30:39,530 --> 00:30:44,660 for a path going through, at most, 457 00:30:44,660 --> 00:30:46,380 a certain number of edges. 458 00:30:46,380 --> 00:30:46,880 OK. 459 00:30:46,880 --> 00:30:51,080 So this is an idea called graph duplication. 460 00:30:51,080 --> 00:30:58,870 Idea, graph duplication. 461 00:30:58,870 --> 00:31:00,880 And this is a very common technique 462 00:31:00,880 --> 00:31:04,890 for solving graph-related problems. 463 00:31:04,890 --> 00:31:06,870 Because essentially what I get to do is I 464 00:31:06,870 --> 00:31:09,090 get to store information. 465 00:31:11,860 --> 00:31:15,480 If I'm having different versions of a vertex, 466 00:31:15,480 --> 00:31:18,270 I can have that vertex correspond 467 00:31:18,270 --> 00:31:22,350 to reaching that vertex in a different state. 468 00:31:22,350 --> 00:31:24,180 So that's what we're going to do here. 469 00:31:24,180 --> 00:31:36,040 The idea here is make V plus 1 levels-- 470 00:31:36,040 --> 00:31:38,980 basically duplicate vertices in our graph-- 471 00:31:44,080 --> 00:32:05,480 where vertex Vk in level k represents reaching vertex 472 00:32:05,480 --> 00:32:13,120 V using at most k edges. 473 00:32:13,120 --> 00:32:16,630 OK, so this definition seems similar to what 474 00:32:16,630 --> 00:32:19,840 we're doing up here. 475 00:32:19,840 --> 00:32:24,070 If we have vertices that have this property, then 476 00:32:24,070 --> 00:32:26,710 their shortest paths in this new graph 477 00:32:26,710 --> 00:32:30,490 might correspond to these k edge distances. 478 00:32:30,490 --> 00:32:32,200 And really, the name of the game here 479 00:32:32,200 --> 00:32:35,080 is to compute these two for every vertex, 480 00:32:35,080 --> 00:32:37,120 because then we can-- 481 00:32:37,120 --> 00:32:42,350 then if d is finite, delta is finite, 482 00:32:42,350 --> 00:32:45,200 then this guy will be the length of our shortest path. 483 00:32:45,200 --> 00:32:48,190 And if they are different, that will be a witness 484 00:32:48,190 --> 00:32:51,810 and we can explore from it. 485 00:32:51,810 --> 00:33:14,750 So-- and if we connect edges from one level 486 00:33:14,750 --> 00:33:26,400 to only higher levels, basically levels with a higher k, then 487 00:33:26,400 --> 00:33:27,840 this graph is going to be a DAG. 488 00:33:34,420 --> 00:33:35,800 Whoa. 489 00:33:35,800 --> 00:33:37,620 That's cool. 490 00:33:37,620 --> 00:33:38,480 Why is that cool? 491 00:33:38,480 --> 00:33:42,440 Because we saw how to solve single-source shortest paths 492 00:33:42,440 --> 00:33:44,150 in a DAG and linear time. 493 00:33:44,150 --> 00:33:46,160 Now this graph that we're going to construct 494 00:33:46,160 --> 00:33:49,100 is going to have V plus 1 levels. 495 00:33:49,100 --> 00:33:55,245 So could have-- our graph kind of explodes V times. 496 00:33:57,702 --> 00:33:59,160 We're going to do that in a second. 497 00:33:59,160 --> 00:34:02,560 I'm going to be more precise with what I mean there. 498 00:34:02,560 --> 00:34:07,740 But if we're multiplying our graph V plus 1 times, 499 00:34:07,740 --> 00:34:10,380 then the size of our graph is now V times larger. 500 00:34:10,380 --> 00:34:14,790 Now that doesn't-- that's not so hard to believe. 501 00:34:14,790 --> 00:34:17,969 But if we made our graph V times larger 502 00:34:17,969 --> 00:34:21,300 and we ran a shortest path algorithm in linear time 503 00:34:21,300 --> 00:34:24,969 with respect to that graph, then that graph 504 00:34:24,969 --> 00:34:33,670 has something like size V times V plus E size. 505 00:34:37,810 --> 00:34:39,100 That looks familiar, maybe? 506 00:34:42,929 --> 00:34:45,389 That's this running time. 507 00:34:45,389 --> 00:34:48,210 So if we can find an algorithm that runs in that running time, 508 00:34:48,210 --> 00:34:51,130 we can get down to V times E. 509 00:34:51,130 --> 00:34:52,159 So let's try to do that. 510 00:34:57,640 --> 00:34:59,830 Here's the transformation I'm going to show you. 511 00:34:59,830 --> 00:35:02,830 I'm going to show you first with an example. 512 00:35:02,830 --> 00:35:08,770 Here's an example of a directed graph that does contain 513 00:35:08,770 --> 00:35:10,960 a negative weight cycle. 514 00:35:10,960 --> 00:35:12,090 Can anyone find it for me? 515 00:35:16,830 --> 00:35:19,440 bcd. 516 00:35:19,440 --> 00:35:21,960 Has weigh minus 4 plus 3 minus 1. 517 00:35:21,960 --> 00:35:26,700 It has a minus 2 total weight. 518 00:35:26,700 --> 00:35:28,840 So that's a negative weight cycle. 519 00:35:28,840 --> 00:35:33,870 So in order to take shortest paths from a, 520 00:35:33,870 --> 00:35:39,330 I will want to say at the end of my algorithm, this better be 0, 521 00:35:39,330 --> 00:35:42,870 and all of these better be minus infinity. 522 00:35:42,870 --> 00:35:47,170 So that's what I want in my algorithm. 523 00:35:47,170 --> 00:35:48,700 So what's my algorithm going to be? 524 00:35:48,700 --> 00:35:55,250 I'm going to make V plus 1 copies of this graph, 525 00:35:55,250 --> 00:35:56,930 and I'm going to kind of stretch it out. 526 00:35:56,930 --> 00:35:57,430 OK. 527 00:35:57,430 --> 00:36:02,260 So here, I have V 0, 1, 2, 3, 4-- 528 00:36:02,260 --> 00:36:04,340 there are four vertices in my graph. 529 00:36:04,340 --> 00:36:08,620 So this is 1, 2, 3, 4, 5 copies of my graph. 530 00:36:08,620 --> 00:36:13,600 I have a version of vertex a for each one of those copies, 531 00:36:13,600 --> 00:36:16,390 a version of vertex b for each of those copies, c 532 00:36:16,390 --> 00:36:18,400 and d, et cetera. 533 00:36:18,400 --> 00:36:21,730 So I have this nice grid of vertices. 534 00:36:21,730 --> 00:36:25,930 And I'm not going to put any edges within a layer, 535 00:36:25,930 --> 00:36:27,900 within a level. 536 00:36:27,900 --> 00:36:30,660 Because then-- I mean, this graph has cycles. 537 00:36:30,660 --> 00:36:33,060 And I don't want cycles in my graph. 538 00:36:33,060 --> 00:36:36,240 What I'm going to do instead is for every edge 539 00:36:36,240 --> 00:36:40,140 in my original graph-- for example, the edge from a to b, 540 00:36:40,140 --> 00:36:43,760 I'm going to connect it to the b in the next level. 541 00:36:43,760 --> 00:36:49,860 So a0 is connected to b1 with an edge weight of minus 5, 542 00:36:49,860 --> 00:36:51,723 just like in the original. 543 00:36:51,723 --> 00:36:53,890 And I'm going to do that for every edge in my graph, 544 00:36:53,890 --> 00:36:55,807 and I'm going to repeat that down all the way. 545 00:36:58,100 --> 00:37:03,770 In addition, I'm going to add zero-weight edge from a0 546 00:37:03,770 --> 00:37:08,060 to a1 or from every vertex all the way down the line. 547 00:37:08,060 --> 00:37:11,510 These are all zero-weight edges corresponding to-- 548 00:37:11,510 --> 00:37:13,800 I'm not going to traverse an edge, 549 00:37:13,800 --> 00:37:17,360 I'm just going to stay at this vertex. 550 00:37:17,360 --> 00:37:19,040 That's going to allow us to simulate 551 00:37:19,040 --> 00:37:22,460 this at most k edges condition. 552 00:37:22,460 --> 00:37:27,060 Now if you take a look at paths in this graph from a0, 553 00:37:27,060 --> 00:37:32,760 our starting vertex, clearly none 554 00:37:32,760 --> 00:37:38,700 of the other vertices in that level are reachable from a0, 555 00:37:38,700 --> 00:37:40,260 just as we want. 556 00:37:40,260 --> 00:37:43,260 Because the shortest-path distance 557 00:37:43,260 --> 00:37:47,100 to any of these vertices using at most 0 edges 558 00:37:47,100 --> 00:37:48,340 should be infinite. 559 00:37:48,340 --> 00:37:50,120 I can't get there in 0 edges. 560 00:37:53,730 --> 00:38:03,920 But then any path in this graph using at most k edges 561 00:38:03,920 --> 00:38:09,530 is going to correspond to a path from a0 562 00:38:09,530 --> 00:38:13,050 to a vertex in that level, the corresponding level. 563 00:38:13,050 --> 00:38:16,610 So for example, if I had a-- 564 00:38:16,610 --> 00:38:26,450 if I was looking for paths 2b using 565 00:38:26,450 --> 00:38:34,400 at most three edges, any path-- 566 00:38:34,400 --> 00:38:40,670 a path from a0 to b3 in this graph 567 00:38:40,670 --> 00:38:43,820 would correspond to a path in this graph that 568 00:38:43,820 --> 00:38:47,320 uses at most three edges. 569 00:38:47,320 --> 00:38:50,010 so Let's find such a path. 570 00:38:50,010 --> 00:38:58,260 So going from a0, b1, stay at b1-- 571 00:38:58,260 --> 00:39:00,090 stay at b, sorry. 572 00:39:00,090 --> 00:39:05,020 Yeah, that's a path using fewer than three edges-- 573 00:39:05,020 --> 00:39:07,000 or at most three edges. 574 00:39:07,000 --> 00:39:09,270 But there's another path here. 575 00:39:09,270 --> 00:39:11,100 Where is it? 576 00:39:11,100 --> 00:39:14,215 Going from a, a, a to b-- 577 00:39:14,215 --> 00:39:15,840 OK, that's not such an interesting one. 578 00:39:15,840 --> 00:39:17,410 That's the same path. 579 00:39:17,410 --> 00:39:20,040 So I might have more than one path in here corresponding 580 00:39:20,040 --> 00:39:24,420 to a path in there, but my claim is that any path in here 581 00:39:24,420 --> 00:39:28,250 corresponds to a path in here. 582 00:39:28,250 --> 00:39:31,790 So what's a path of length? 583 00:39:31,790 --> 00:39:33,110 3, that's non-trivial. 584 00:39:33,110 --> 00:39:36,560 Yeah, a to c to d to b. 585 00:39:36,560 --> 00:39:42,620 So a to c to d to b. 586 00:39:42,620 --> 00:39:44,510 Yeah, that's a path. 587 00:39:44,510 --> 00:39:47,330 And basically, because I constructed this so 588 00:39:47,330 --> 00:39:49,670 that the edges always moved from level to level, 589 00:39:49,670 --> 00:39:54,500 as I traverse these edges, I always change levels. 590 00:39:54,500 --> 00:39:56,487 Yeah? 591 00:39:56,487 --> 00:39:58,320 AUDIENCE: But my original graph doesn't have 592 00:39:58,320 --> 00:40:00,587 these self-loops with 0 weight. 593 00:40:00,587 --> 00:40:01,170 JASON KU: Yes. 594 00:40:01,170 --> 00:40:06,060 My original graph doesn't have an edge from a to a. 595 00:40:06,060 --> 00:40:08,550 That's true. 596 00:40:08,550 --> 00:40:12,430 I'm using these edges to correspond to-- 597 00:40:12,430 --> 00:40:14,770 I'm deciding not to take an edge. 598 00:40:19,160 --> 00:40:22,400 It's not that I'm like doing any work here, 599 00:40:22,400 --> 00:40:25,350 I'm just staying there for a state. 600 00:40:25,350 --> 00:40:28,640 And that's what's going to allow me to get this at most edges. 601 00:40:28,640 --> 00:40:29,210 All right. 602 00:40:29,210 --> 00:40:31,040 So, this is the graph construct. 603 00:40:31,040 --> 00:40:37,760 Hopefully you understand that we made these V layers. 604 00:40:37,760 --> 00:40:44,600 This is V. And a vertex-- 605 00:40:44,600 --> 00:40:48,030 we made V copies of every vertex and connected 606 00:40:48,030 --> 00:40:49,410 them using edges in this way. 607 00:40:49,410 --> 00:40:50,980 OK. 608 00:40:50,980 --> 00:40:54,190 So, first step of Bellman-Ford is construct this graph. 609 00:40:58,980 --> 00:41:09,620 So, Bellman-Ford, construct G prime as described above. 610 00:41:09,620 --> 00:41:13,825 It has how many vertices? 611 00:41:16,980 --> 00:41:20,130 V times V plus 1. 612 00:41:20,130 --> 00:41:22,830 V times V plus 1 vertices. 613 00:41:22,830 --> 00:41:24,450 And how many edges? 614 00:41:24,450 --> 00:41:29,820 Well, I have one edge for outgoing edge for each vertex 615 00:41:29,820 --> 00:41:34,220 corresponding to just staying in the same place. 616 00:41:34,220 --> 00:41:37,880 So that's V squared vertices-- 617 00:41:37,880 --> 00:41:39,380 I mean edges. 618 00:41:39,380 --> 00:41:42,410 And then I have one edge-- 619 00:41:42,410 --> 00:41:45,540 for every edge in my graph, I have-- 620 00:41:45,540 --> 00:41:46,040 sorry. 621 00:41:46,040 --> 00:41:48,630 I have a V minus 1-- 622 00:41:48,630 --> 00:41:49,130 sorry. 623 00:41:49,130 --> 00:41:53,420 Just V. I have V edges for every edge in my graph. 624 00:41:53,420 --> 00:41:56,090 So that means-- so this is the number of vertices. 625 00:42:00,230 --> 00:42:11,600 And V times V plus V times E, this 626 00:42:11,600 --> 00:42:18,710 is V V plus E. All right, cool. 627 00:42:18,710 --> 00:42:22,130 So that's how many edges I have. 628 00:42:22,130 --> 00:42:24,900 So constructed in that way, it's a DAG. 629 00:42:24,900 --> 00:42:29,360 If we only have edges going to increasing levels, 630 00:42:29,360 --> 00:42:31,115 then this thing can't have cycles, 631 00:42:31,115 --> 00:42:32,990 because otherwise that would mean there would 632 00:42:32,990 --> 00:42:34,620 be an edge pointing backwards. 633 00:42:34,620 --> 00:42:37,090 And we didn't construct any of those. 634 00:42:37,090 --> 00:42:37,930 All right. 635 00:42:37,930 --> 00:42:40,750 So we construct this graph G prime. 636 00:42:40,750 --> 00:42:43,620 We can do that in linear time with respect to these things. 637 00:42:43,620 --> 00:42:46,630 I just go through all the edges, I make these edges, 638 00:42:46,630 --> 00:42:47,860 and I make these vertices. 639 00:42:47,860 --> 00:42:49,630 It doesn't take anything-- 640 00:42:49,630 --> 00:42:51,130 I just do it naively. 641 00:42:51,130 --> 00:42:59,270 Right I can do that in time V times V plus E asymptotically. 642 00:42:59,270 --> 00:43:01,160 OK. 643 00:43:01,160 --> 00:43:08,210 Now I run DAG relaxation, our nice algorithm 644 00:43:08,210 --> 00:43:13,625 we had last time, from-- 645 00:43:16,490 --> 00:43:17,840 in there was a0. 646 00:43:17,840 --> 00:43:22,600 I'm going to say it's S0, our source. 647 00:43:22,600 --> 00:43:24,490 Our source vertex. 648 00:43:24,490 --> 00:43:27,940 Single source shortest paths. 649 00:43:27,940 --> 00:43:44,670 So that I compute delta of S0 to Vk for all k and-- 650 00:43:44,670 --> 00:43:46,470 what is it? 651 00:43:46,470 --> 00:43:54,430 0 to V. That's what single source shortest paths does. 652 00:43:54,430 --> 00:43:58,060 It computes for me this distance from my source-- 653 00:43:58,060 --> 00:44:01,090 at some source to every other vertex in the graph. 654 00:44:01,090 --> 00:44:04,090 And so in particular I get these. 655 00:44:04,090 --> 00:44:05,440 Well, that is all of them. 656 00:44:08,350 --> 00:44:21,240 Then for each vertex V, set-- 657 00:44:21,240 --> 00:44:27,960 the thing I'm going to return, d-value, S to V, 658 00:44:27,960 --> 00:44:31,160 equal to the shortest-path distance 659 00:44:31,160 --> 00:44:34,340 I got from DAG relaxation to a particular vertex. 660 00:44:40,640 --> 00:44:45,140 V V minus 1. 661 00:44:45,140 --> 00:44:47,780 Why am I doing this? 662 00:44:47,780 --> 00:44:51,980 I'm setting it to be the shortest-path distance 663 00:44:51,980 --> 00:44:56,330 to the guy in the second-to-last row here 664 00:44:56,330 --> 00:45:02,440 or column in my modified graph. 665 00:45:05,930 --> 00:45:13,390 The hope is that this distance in my DAG 666 00:45:13,390 --> 00:45:23,020 corresponds to this distance in my original graph. 667 00:45:23,020 --> 00:45:27,560 The distance to V using at most V minus 1 edges. 668 00:45:27,560 --> 00:45:29,560 So that's the claim-- that's a claim we're going 669 00:45:29,560 --> 00:45:30,830 to prove in just a second. 670 00:45:30,830 --> 00:45:33,790 I'm going to write it down just so that we have-- 671 00:45:33,790 --> 00:45:36,490 just to continue our train of thought. 672 00:45:36,490 --> 00:45:51,670 Claim, delta S0 Vk equals delta k, the k edge distance, from S 673 00:45:51,670 --> 00:45:57,280 to V. That's what we want to claim. 674 00:45:57,280 --> 00:45:59,530 That would then-- what would that mean, then? 675 00:45:59,530 --> 00:46:01,930 That would mean that I'm correctly 676 00:46:01,930 --> 00:46:04,300 setting the shortest-path distance here 677 00:46:04,300 --> 00:46:09,060 for all vertices whose distances finite. 678 00:46:09,060 --> 00:46:09,670 Great. 679 00:46:09,670 --> 00:46:14,160 I mean, I set values to things where they're not finite, 680 00:46:14,160 --> 00:46:16,380 where they're minus infinity also, 681 00:46:16,380 --> 00:46:22,150 but in particular I set the ones correctly if they're finite. 682 00:46:22,150 --> 00:46:22,780 OK. 683 00:46:22,780 --> 00:46:24,280 So the last thing we need to do is 684 00:46:24,280 --> 00:46:28,150 deal with these minus infinity vertices. 685 00:46:30,690 --> 00:46:32,880 But we know how to do that. 686 00:46:32,880 --> 00:46:34,230 We just look at the witnesses. 687 00:46:34,230 --> 00:46:42,620 Because we've computed this value for k 688 00:46:42,620 --> 00:46:46,490 equals V equals V minus 1, and if that claim over there 689 00:46:46,490 --> 00:46:50,030 is true, then those shortest-path distances 690 00:46:50,030 --> 00:46:55,310 are the same as these k edge shortest-path distances. 691 00:46:55,310 --> 00:46:57,310 And we can just, for every vertex, 692 00:46:57,310 --> 00:46:58,690 we compare these things. 693 00:46:58,690 --> 00:47:02,910 If this is satisfied, we got a witness. 694 00:47:02,910 --> 00:47:03,490 OK. 695 00:47:03,490 --> 00:47:30,880 So for each witness U and V where delta S0 U 696 00:47:30,880 --> 00:47:39,610 V is less than, strictly, S0 U V minus 1-- 697 00:47:39,610 --> 00:47:42,370 that's the definition of a witness here, 698 00:47:42,370 --> 00:47:45,670 close the parentheses. 699 00:47:45,670 --> 00:47:58,390 Then for each vertex V reachable from U set-- 700 00:48:00,910 --> 00:48:05,410 sorry, d, is what we're returning, d of S, V 701 00:48:05,410 --> 00:48:07,470 equal to minus infinity. 702 00:48:07,470 --> 00:48:09,370 That's the end of the algorithm. 703 00:48:09,370 --> 00:48:12,850 Basically I'm looking for all the witnesses. 704 00:48:12,850 --> 00:48:16,390 For each witness, I find all of the vertices reachable from it 705 00:48:16,390 --> 00:48:20,150 and set it to minus infinity just as we argued before. 706 00:48:20,150 --> 00:48:20,740 OK. 707 00:48:20,740 --> 00:48:23,800 So, it remains to prove this claim. 708 00:48:28,420 --> 00:48:31,870 How do we prove this claim? 709 00:48:31,870 --> 00:48:37,020 Well, we can induct on k. 710 00:48:37,020 --> 00:48:39,680 Is this true for k equals 0? 711 00:48:39,680 --> 00:48:40,340 Yeah. 712 00:48:40,340 --> 00:48:42,680 We kind of already argued it over here 713 00:48:42,680 --> 00:48:45,740 when we are talking about our initialization step 714 00:48:45,740 --> 00:48:48,560 or what DAG relaxation does. 715 00:48:48,560 --> 00:48:51,890 It'll set this to be the shortest path from this guy 716 00:48:51,890 --> 00:48:53,570 to all these vertices. 717 00:48:53,570 --> 00:48:55,250 These aren't reachable from here, 718 00:48:55,250 --> 00:48:57,240 and so these are infinite. 719 00:48:57,240 --> 00:49:00,370 And that one's 0. 720 00:49:00,370 --> 00:49:03,290 So the base case-- 721 00:49:03,290 --> 00:49:14,970 so induct on k base case, k equals 0? 722 00:49:14,970 --> 00:49:16,980 Check. 723 00:49:16,980 --> 00:49:18,120 That's all good. 724 00:49:18,120 --> 00:49:23,790 Now we-- in our inductive step, let's take a look 725 00:49:23,790 --> 00:49:26,250 at the shortest-path distance from 0-- 726 00:49:26,250 --> 00:49:32,310 from S0 to V of k prime for some k prime. 727 00:49:32,310 --> 00:49:35,040 And the assumption is, the inductive hypothesis 728 00:49:35,040 --> 00:49:42,390 is that this distance is the k edge distance for all k prime 729 00:49:42,390 --> 00:49:43,230 less than-- 730 00:49:43,230 --> 00:49:47,100 I mean all k less than k prime. 731 00:49:47,100 --> 00:49:50,190 Well, kind of by definition of a shortest path, 732 00:49:50,190 --> 00:49:55,650 this is the minimum overall incoming vertices 733 00:49:55,650 --> 00:50:05,300 of the shortest path from S0 to U of k prime minus 1 734 00:50:05,300 --> 00:50:10,850 plus the weight of the edge from U of k prime 735 00:50:10,850 --> 00:50:21,140 minus to Vk prime for all Uk prime minus 1 736 00:50:21,140 --> 00:50:29,040 in the adjacencies, the incoming adjacencies of Vk prime. 737 00:50:29,040 --> 00:50:31,110 OK, what does this mean? 738 00:50:31,110 --> 00:50:34,840 I'm just saying in my graph G prime, 739 00:50:34,840 --> 00:50:37,870 a shortest path to this vertex needs 740 00:50:37,870 --> 00:50:43,040 to go first through some vertex in the layer before it, 741 00:50:43,040 --> 00:50:44,110 which is one of these. 742 00:50:44,110 --> 00:50:45,730 And in particular, I'm only connected 743 00:50:45,730 --> 00:50:49,700 to things adjacent to me. 744 00:50:49,700 --> 00:50:51,050 That's all this is saying. 745 00:50:51,050 --> 00:50:53,720 I have to go through that vertex and take some shortest 746 00:50:53,720 --> 00:50:57,610 path to one of those previous vertices. 747 00:50:57,610 --> 00:51:03,460 Now s actuality, these adjacencies, I constructed them 748 00:51:03,460 --> 00:51:07,540 to be similar to the adjacencies in my original graph 749 00:51:07,540 --> 00:51:11,440 in addition to one edge coming from my original vertex, 750 00:51:11,440 --> 00:51:17,710 from vertex V. So this is the same as the minimum of this set 751 00:51:17,710 --> 00:51:19,165 delta S0. 752 00:51:22,210 --> 00:51:22,960 Same thing. 753 00:51:26,730 --> 00:51:43,130 Plus W U, V for all U in the adjacent-- incoming adjacencies 754 00:51:43,130 --> 00:51:46,460 of my original vertex. 755 00:51:46,460 --> 00:51:47,890 In addition to one more term. 756 00:51:47,890 --> 00:51:48,890 What is that other term? 757 00:51:54,410 --> 00:51:56,200 These are all of the things corresponding 758 00:51:56,200 --> 00:51:59,200 to my incoming edges in my original thing, 759 00:51:59,200 --> 00:52:03,360 but I also have that one edge coming from the V before it. 760 00:52:03,360 --> 00:52:05,590 So this is-- I'm going to union-- 761 00:52:05,590 --> 00:52:16,600 union-- union this with delta S0 V of k prime minus 1. 762 00:52:16,600 --> 00:52:18,130 Awful. 763 00:52:18,130 --> 00:52:20,520 I think there's another one here. 764 00:52:20,520 --> 00:52:25,170 This is S0 of k prime minus 1. 765 00:52:25,170 --> 00:52:27,250 I"m not going to rewrite it. 766 00:52:27,250 --> 00:52:27,750 OK. 767 00:52:34,110 --> 00:52:38,040 Then by induction, this thing and this thing 768 00:52:38,040 --> 00:52:45,190 must be the edge shortest paths using k minus 1 vertices. 769 00:52:45,190 --> 00:52:49,330 And then that's just the statement of what the shortest 770 00:52:49,330 --> 00:52:57,730 path should be using at most k prime edges going 771 00:52:57,730 --> 00:53:06,000 from S to V. So, these things are the same as we claimed. 772 00:53:06,000 --> 00:53:08,650 Yay, check. 773 00:53:08,650 --> 00:53:09,220 All right. 774 00:53:09,220 --> 00:53:11,300 And then it's not such-- 775 00:53:11,300 --> 00:53:13,030 it's kind of a trivial leap, then, 776 00:53:13,030 --> 00:53:19,880 to say that at the end of Bellman-Ford, these guys-- 777 00:53:19,880 --> 00:53:23,390 sorry, the things that we return, these guys, 778 00:53:23,390 --> 00:53:26,360 are the shortest-path distances, because here, 779 00:53:26,360 --> 00:53:30,530 if they're finite, we set them to their true shortest-path 780 00:53:30,530 --> 00:53:34,980 distance; and if they're minus infinity, 781 00:53:34,980 --> 00:53:40,050 that invariant means that these things correspond to exactly 782 00:53:40,050 --> 00:53:41,340 this claim over here. 783 00:53:44,830 --> 00:53:45,565 It's a witness. 784 00:53:49,060 --> 00:53:51,430 And then finding all the vertices reachable 785 00:53:51,430 --> 00:53:56,440 from those witnesses, we set all of the infinite ones 786 00:53:56,440 --> 00:53:58,840 to be minus infinity as desired. 787 00:53:58,840 --> 00:54:02,610 OK, so what's the running time of this thing? 788 00:54:02,610 --> 00:54:04,720 Well, we had to construct this graph, 789 00:54:04,720 --> 00:54:06,930 so we had to take that time. 790 00:54:06,930 --> 00:54:10,770 We ran DAG relaxation, that takes the same amount of time. 791 00:54:10,770 --> 00:54:14,400 For every vertex, we did order V at work. 792 00:54:14,400 --> 00:54:17,370 And then for each witness, how many could there be? 793 00:54:17,370 --> 00:54:21,330 And most, V. Checking the reachability of each vertex, 794 00:54:21,330 --> 00:54:24,300 that can be done in how long? 795 00:54:24,300 --> 00:54:27,420 Order E time. 796 00:54:27,420 --> 00:54:30,750 Because we don't need to consider the things that 797 00:54:30,750 --> 00:54:32,640 aren't connected to S-- 798 00:54:32,640 --> 00:54:35,290 or aren't connected to the witness. 799 00:54:35,290 --> 00:54:41,430 So this thing takes order V times E work. 800 00:54:41,430 --> 00:54:44,970 So we're upper-bounded by this time 801 00:54:44,970 --> 00:54:47,350 it took to construct the original graph 802 00:54:47,350 --> 00:54:53,000 and by the claim we had before, that takes V times E time. 803 00:54:53,000 --> 00:54:53,930 OK. 804 00:54:53,930 --> 00:54:56,381 So that's Bellman-Ford. 805 00:54:56,381 --> 00:54:59,415 I'm just going to leave you with two nuggets. 806 00:55:03,910 --> 00:55:12,150 First, the shortest path, if for any witness-- 807 00:55:12,150 --> 00:55:13,530 let's say we have a witness here. 808 00:55:13,530 --> 00:55:16,020 Do I have any witnesses here? 809 00:55:16,020 --> 00:55:18,920 I didn't fill in all these. 810 00:55:18,920 --> 00:55:23,060 But is there a vertex on this cycle 811 00:55:23,060 --> 00:55:27,830 that goes through who has the shortest path? 812 00:55:27,830 --> 00:55:32,000 That goes through four vertices that's smaller than any other. 813 00:55:32,000 --> 00:55:32,570 OK. 814 00:55:32,570 --> 00:55:37,850 I can go from a to c to b to d to b to c. 815 00:55:40,930 --> 00:55:43,030 And you can work out this algorithm-- 816 00:55:43,030 --> 00:55:45,580 I have it in the notes that you can take a look at. 817 00:55:45,580 --> 00:55:50,450 This will actually have a shorter path for vertex-- 818 00:55:50,450 --> 00:55:51,160 sorry. 819 00:55:51,160 --> 00:55:55,070 It'll have a shorter path for vertex b. 820 00:56:01,010 --> 00:56:04,130 a to b to c to d to b. 821 00:56:04,130 --> 00:56:04,670 Thank you. 822 00:56:04,670 --> 00:56:09,000 That's a path of length 4, of four edges. 823 00:56:09,000 --> 00:56:13,640 That has shorter path than any path that has fewer edges. 824 00:56:13,640 --> 00:56:15,390 In particular, there's only one other path 825 00:56:15,390 --> 00:56:17,970 to b using fewer than four-- 826 00:56:17,970 --> 00:56:20,100 there's two other paths. 827 00:56:20,100 --> 00:56:21,900 One path of length-- 828 00:56:21,900 --> 00:56:24,390 that has one edge, that has weight minus 5, 829 00:56:24,390 --> 00:56:29,580 and one path-- this path that has weight 9 minus 1 is 8. 830 00:56:29,580 --> 00:56:34,590 Whereas this path, minus 5, minus 4, 3, minus 1 831 00:56:34,590 --> 00:56:41,610 has minus 10 plus 3 is minus 7, which is shorter than minus 5. 832 00:56:41,610 --> 00:56:45,060 So b and d is a witness. 833 00:56:45,060 --> 00:56:48,660 And if we actually take a look at that path 834 00:56:48,660 --> 00:57:03,400 through this graph, going from a to b to c to d back to b 835 00:57:03,400 --> 00:57:08,250 we see that there's a negative weight cycle in this graph. 836 00:57:08,250 --> 00:57:10,960 b to c to d to b. 837 00:57:10,960 --> 00:57:15,600 And indeed, that's always the case for our witnesses. 838 00:57:15,600 --> 00:57:17,820 You can see a proof of that in the notes, 839 00:57:17,820 --> 00:57:22,690 and you can see in recitation a little space optimization 840 00:57:22,690 --> 00:57:26,770 to make us not have to construct this entire graph on the fly, 841 00:57:26,770 --> 00:57:30,290 but actually only use order V space while they're going. 842 00:57:30,290 --> 00:57:30,790 OK. 843 00:57:30,790 --> 00:57:32,260 So that's Bellman-Ford. 844 00:57:32,260 --> 00:57:34,800 Sorry for running a little late.