WEBVTT 00:00:00.000 --> 00:00:01.976 [SQUEAKING] 00:00:01.976 --> 00:00:03.458 [RUSTLING] 00:00:03.458 --> 00:00:04.940 [CLICKING] 00:00:12.302 --> 00:00:13.260 JASON KU: Hi, everyone. 00:00:13.260 --> 00:00:17.075 Welcome to the 11th lecture of 6.006, 00:00:17.075 --> 00:00:21.110 our first lecture on weighted shortest paths. 00:00:21.110 --> 00:00:25.940 Until now, we've only been talking about graphs that-- 00:00:25.940 --> 00:00:29.720 where we measure distance in terms of the number of edges 00:00:29.720 --> 00:00:31.390 in a path. 00:00:31.390 --> 00:00:33.407 Today, we're going to generalize that notion. 00:00:33.407 --> 00:00:35.240 But I just want to go over what we've talked 00:00:35.240 --> 00:00:37.790 about in the last two lectures. 00:00:37.790 --> 00:00:40.820 In the last two lectures, we've talked about two algorithms, 00:00:40.820 --> 00:00:43.670 breadth-first search and depth-first search 00:00:43.670 --> 00:00:45.180 to solve a range of problems. 00:00:45.180 --> 00:00:47.570 Here's some of the problems that we've been solving. 00:00:47.570 --> 00:00:51.740 Single-source shortest paths, where distances are measured 00:00:51.740 --> 00:00:54.720 in number of edges in a path. 00:00:54.720 --> 00:00:59.240 And we used BFS to solve this problem, 00:00:59.240 --> 00:01:01.590 starting from a single source, usually 00:01:01.590 --> 00:01:04.550 a vertex s that we call. 00:01:04.550 --> 00:01:06.500 And we solve that in linear time. 00:01:06.500 --> 00:01:13.490 And we solve that in order v plus e. 00:01:13.490 --> 00:01:16.580 That's what we called linear time for a graph. 00:01:16.580 --> 00:01:19.820 For the special case of single-source reachability, 00:01:19.820 --> 00:01:25.420 here we had to return a shortest path distance for every vertex. 00:01:25.420 --> 00:01:29.260 And there was, at most, E things reachable from a vertex. 00:01:29.260 --> 00:01:33.210 So this is the bound we got. 00:01:33.210 --> 00:01:35.820 But in the special case for single-source reachability, 00:01:35.820 --> 00:01:38.310 when our output only has to list the vertices that 00:01:38.310 --> 00:01:42.180 are reachable from me, the number 00:01:42.180 --> 00:01:45.660 of things reachable in basically a spanning tree 00:01:45.660 --> 00:01:48.270 of the connected component of my source 00:01:48.270 --> 00:01:52.800 can almost be of order E. And so for all the little singleton 00:01:52.800 --> 00:01:57.150 vertices in my graph, I don't really care. 00:01:57.150 --> 00:01:59.070 So I can get this in order E, but that's 00:01:59.070 --> 00:02:03.780 kind of a little optimization. 00:02:03.780 --> 00:02:05.730 The next thing we did was we talked 00:02:05.730 --> 00:02:07.510 about connected components. 00:02:07.510 --> 00:02:11.160 And we didn't just reduce to using a search algorithm 00:02:11.160 --> 00:02:13.710 like a single-source reachability 00:02:13.710 --> 00:02:15.750 algorithm like BFS or DFS. 00:02:15.750 --> 00:02:20.100 We put a for loop around that to explore the entire graph 00:02:20.100 --> 00:02:22.050 by basically saying, if I've explored 00:02:22.050 --> 00:02:26.810 one connected component, then I can 00:02:26.810 --> 00:02:29.390 look at any other vertex I haven't seen 00:02:29.390 --> 00:02:30.950 and explore the next one. 00:02:30.950 --> 00:02:33.830 And so that actually with some-- 00:02:33.830 --> 00:02:36.500 a little analysis, also got linear time, 00:02:36.500 --> 00:02:41.060 because I'm at most traversing any component of my graph once. 00:02:41.060 --> 00:02:43.670 That's kind of the idea. 00:02:43.670 --> 00:02:47.660 And we can use that using BFS or DFS really, 00:02:47.660 --> 00:02:50.630 because we're just trying to get a thing that searches 00:02:50.630 --> 00:02:53.150 an entire connected component. 00:02:53.150 --> 00:02:55.280 And then this topological sort we did 00:02:55.280 --> 00:02:57.210 at the end of the last lecture. 00:02:57.210 --> 00:03:00.470 We used full DFS to give an ordering 00:03:00.470 --> 00:03:04.710 of the vertices in a DAG-- 00:03:04.710 --> 00:03:09.960 maybe I'll specify clearly that this is only for a DAG-- 00:03:09.960 --> 00:03:14.020 where we have an ordering of the vertices 00:03:14.020 --> 00:03:18.960 so all the edges go forward in that order, for example. 00:03:18.960 --> 00:03:20.460 And that we also did in linear time. 00:03:25.710 --> 00:03:28.800 All right, in this lecture, and in actually the next four 00:03:28.800 --> 00:03:33.480 lectures, what we're going to do is 00:03:33.480 --> 00:03:36.750 instead of measuring distance in terms of the number of edges 00:03:36.750 --> 00:03:37.650 in a path-- 00:03:37.650 --> 00:03:45.810 so previously, distance equaled number of edges-- 00:03:45.810 --> 00:03:47.490 we're going to generalize that notion. 00:03:47.490 --> 00:03:50.190 So instead counting an edge, we're 00:03:50.190 --> 00:03:54.240 going to count an integer associated with that edge. 00:03:54.240 --> 00:03:56.010 It's going to be called a weight. 00:03:56.010 --> 00:04:01.140 So here's an example of a weighted graph G. 00:04:01.140 --> 00:04:03.860 And I've labeled, in red, weights 00:04:03.860 --> 00:04:04.860 for each of these edges. 00:04:04.860 --> 00:04:07.850 This is a directed graph on eight vertices. 00:04:07.850 --> 00:04:10.580 And I've got an integer associated with each edge. 00:04:10.580 --> 00:04:12.830 You'll notice, some of them are positive, some of them 00:04:12.830 --> 00:04:13.372 are negative. 00:04:13.372 --> 00:04:15.140 It's OK to be zero as well. 00:04:15.140 --> 00:04:18.649 It's just any integer edge weight here. 00:04:18.649 --> 00:04:21.339 So generally we're going to be-- along with our graph 00:04:21.339 --> 00:04:29.770 G, we're going to be given a weight function that maps 00:04:29.770 --> 00:04:38.030 the edges of G to, we're going to say, integers, in this class 00:04:38.030 --> 00:04:39.860 anyway. 00:04:39.860 --> 00:04:41.720 In other contexts, in mathematics, 00:04:41.720 --> 00:04:43.380 you might have these be real numbers. 00:04:43.380 --> 00:04:45.600 But in this class, we're going to deal with integers. 00:04:45.600 --> 00:04:48.890 So each edge, if you have an edge, 00:04:48.890 --> 00:04:53.930 we're going to say this is the edge weight-- 00:04:53.930 --> 00:04:58.100 the weight of this edge e, from e. 00:04:58.100 --> 00:05:04.910 Sometimes, if this edge e is u, v, we 00:05:04.910 --> 00:05:08.840 might sometimes say the weight from u 00:05:08.840 --> 00:05:14.060 to v, since we have a simple graph that's unambiguous. 00:05:14.060 --> 00:05:16.930 All right, so but this is just talking about our notation. 00:05:16.930 --> 00:05:21.950 So in general, for example, the weight from vertex b 00:05:21.950 --> 00:05:24.440 to f in this graph is what? 00:05:24.440 --> 00:05:26.070 Can someone tell me? 00:05:26.070 --> 00:05:26.950 AUDIENCE: Minus 4. 00:05:26.950 --> 00:05:28.030 JASON KU: Minus 4, right? 00:05:28.030 --> 00:05:28.697 It's right here. 00:05:28.697 --> 00:05:32.920 And I'll be consistent with my coloring, 00:05:32.920 --> 00:05:36.340 because I've got colored chalk today. 00:05:36.340 --> 00:05:37.480 Minus 4. 00:05:37.480 --> 00:05:38.680 Happiness. 00:05:38.680 --> 00:05:41.890 All right, so why do we care about adding weights 00:05:41.890 --> 00:05:43.330 to our graph? 00:05:43.330 --> 00:05:46.900 Well, this comes up a lot in many applications. 00:05:46.900 --> 00:05:50.110 For example, distances in a road network. 00:05:50.110 --> 00:05:54.160 if I have a road from here-- 00:05:54.160 --> 00:06:00.910 so from Mass Ave, front of MIT, to Central Square, 00:06:00.910 --> 00:06:03.340 we might think of that as one road. 00:06:03.340 --> 00:06:06.700 Maybe you've got each road is a connection 00:06:06.700 --> 00:06:10.150 between two intersections in my road network. 00:06:10.150 --> 00:06:14.390 But an edge, it takes longer to go from, 00:06:14.390 --> 00:06:17.572 say, Vassar Street to Amherst. 00:06:17.572 --> 00:06:19.030 That takes a shorter amount of time 00:06:19.030 --> 00:06:22.810 than it does to go from Memorial Drive across the river 00:06:22.810 --> 00:06:24.640 to Beacon Street. 00:06:24.640 --> 00:06:28.570 So we might want to associate a larger distance or a weight 00:06:28.570 --> 00:06:31.660 associated with that edge. 00:06:31.660 --> 00:06:35.590 Latency in a network, for example. 00:06:35.590 --> 00:06:39.070 Maybe strength of relationships in a social network. 00:06:39.070 --> 00:06:42.610 And you could imagine that it's possible maybe you're 00:06:42.610 --> 00:06:46.130 "frenemies" with someone, you don't like them, 00:06:46.130 --> 00:06:48.370 and so maybe you have a negative weight associated 00:06:48.370 --> 00:06:51.050 with an edge in a social network. 00:06:51.050 --> 00:06:52.070 I'm not sure. 00:06:52.070 --> 00:06:52.720 Maybe not. 00:06:52.720 --> 00:06:55.210 But there are lots of applications where you might 00:06:55.210 --> 00:06:59.150 want weights on your edges. 00:06:59.150 --> 00:07:02.450 So that comes to the next question of, 00:07:02.450 --> 00:07:04.130 how do I represent-- 00:07:04.130 --> 00:07:08.120 how do I give the user, or the algorithm, 00:07:08.120 --> 00:07:10.610 these weights in my graph? 00:07:10.610 --> 00:07:15.350 We had a representation for a graph. 00:07:15.350 --> 00:07:17.540 Our common way to represent a graph 00:07:17.540 --> 00:07:21.800 was store a set data structure on the vertices mapping 00:07:21.800 --> 00:07:25.490 to the adjacencies of each vertex, which 00:07:25.490 --> 00:07:28.250 we stored in what we called an adjacency list, which really 00:07:28.250 --> 00:07:32.210 could be any data structure. 00:07:32.210 --> 00:07:35.870 Commonly, it's just an array of the adjacencies. 00:07:35.870 --> 00:07:40.880 But you could also have that be a set data structure, 00:07:40.880 --> 00:07:44.520 where you can query in constant time what-- 00:07:44.520 --> 00:07:48.000 if a particular adjacency exists in that graph. 00:07:48.000 --> 00:07:53.990 So there are two common ways to store these weights. 00:07:53.990 --> 00:07:57.630 One is just, with every adjacency, 00:07:57.630 --> 00:07:58.880 I'm going to store its weight. 00:07:58.880 --> 00:08:02.030 Maybe just in a tuple. 00:08:02.030 --> 00:08:16.190 With each adjacency, also store weight of the edge 00:08:16.190 --> 00:08:20.750 that it corresponds to, just in any way. 00:08:20.750 --> 00:08:25.580 A second way, instead of trying to modify our graph structure 00:08:25.580 --> 00:08:28.040 that we gave you before, let's just 00:08:28.040 --> 00:08:31.280 have a dictionary of all the edges mapping to their weights. 00:08:31.280 --> 00:08:33.059 And we already know how to do that. 00:08:33.059 --> 00:08:43.370 Just any set data structure-- any separate set data structure 00:08:43.370 --> 00:08:51.380 mapping edges to their, I guess, weights. 00:08:51.380 --> 00:08:53.120 Bad notation, but you get the idea. 00:08:56.150 --> 00:08:58.880 And it doesn't really matter how we're doing this. 00:08:58.880 --> 00:09:01.100 The assumption that we're going to rely on here 00:09:01.100 --> 00:09:07.130 is that, given an edge, given this vertex pair, 00:09:07.130 --> 00:09:11.900 I can query what the weight of that edge is in constant time. 00:09:11.900 --> 00:09:13.280 And so if I'm going to do that, I 00:09:13.280 --> 00:09:17.210 can either store it with maybe a hash table of hash tables-- 00:09:17.210 --> 00:09:22.650 a hash table mapping the set of vertices to their adjacencies, 00:09:22.650 --> 00:09:26.900 and then each adjacency list stores its adjacencies 00:09:26.900 --> 00:09:27.740 in a hash table. 00:09:27.740 --> 00:09:29.810 And that way, in constant time, I 00:09:29.810 --> 00:09:31.610 can check what the weight is there. 00:09:31.610 --> 00:09:33.830 Or here, I'm just-- 00:09:33.830 --> 00:09:35.330 I could even have just a single hash 00:09:35.330 --> 00:09:42.960 table mapping the pair, the edge, the tuple, constant size, 00:09:42.960 --> 00:09:44.320 to its weight. 00:09:44.320 --> 00:09:46.870 So either way is fine. 00:09:46.870 --> 00:09:49.050 We're just going to assume that we can query 00:09:49.050 --> 00:09:51.030 an edge in constant time-- 00:09:51.030 --> 00:09:52.670 the weight of an edge in constant time. 00:09:52.670 --> 00:09:58.140 OK, so this is that graph example. 00:09:58.140 --> 00:09:59.400 It's a little busy here. 00:09:59.400 --> 00:10:03.040 I'm probably going to erase that in just a second. 00:10:03.040 --> 00:10:07.680 But we're going to move on to what giving these edges weights 00:10:07.680 --> 00:10:13.710 implies for these problems that we've defined 00:10:13.710 --> 00:10:15.780 in terms of unweighted graphs. 00:10:15.780 --> 00:10:18.390 In particular, we are going to be concentrating 00:10:18.390 --> 00:10:22.143 on single-source shortest paths, again, at least 00:10:22.143 --> 00:10:23.310 for the next three lectures. 00:10:23.310 --> 00:10:28.530 We'll generalize that even still in the next lecture-- 00:10:28.530 --> 00:10:30.450 I mean, in the fourth-- 00:10:30.450 --> 00:10:33.710 in three lectures from now. 00:10:33.710 --> 00:10:36.710 But what we had here was that the distance before 00:10:36.710 --> 00:10:39.920 in an unweighted graph was the number of edges in the path. 00:10:39.920 --> 00:10:46.400 Here, we're going to generalize that notion kind of obviously 00:10:46.400 --> 00:10:47.985 to weighted paths. 00:10:52.190 --> 00:10:59.330 And the weight of a path, I'm going call it pi. 00:10:59.330 --> 00:11:08.090 So some weight of path pi is just 00:11:08.090 --> 00:11:10.190 going to be the sum of the weights 00:11:10.190 --> 00:11:12.750 in the edges in the path. 00:11:12.750 --> 00:11:20.510 So edge in the path, I'm going to sum their weights. 00:11:20.510 --> 00:11:22.970 So that's all the weight of a path means. 00:11:22.970 --> 00:11:25.470 It's just I'm going to sum all the weights in a path. 00:11:25.470 --> 00:11:27.020 So if I took a look at the-- 00:11:27.020 --> 00:11:29.600 maybe there's a particular path here. 00:11:29.600 --> 00:11:35.930 The path from a to b to f to g is going to be minus 5, 00:11:35.930 --> 00:11:38.180 minus 4, 2. 00:11:38.180 --> 00:11:43.250 It's going to be minus 9 plus 2 is minus 7. 00:11:43.250 --> 00:11:44.360 So just as an example. 00:11:48.770 --> 00:11:52.460 So then what is the shortest path then? 00:11:52.460 --> 00:11:55.790 Well, kind of obviously among all paths between two 00:11:55.790 --> 00:12:00.170 vertices, it's going to be one with the minimum weight. 00:12:00.170 --> 00:12:00.920 Yeah, question. 00:12:00.920 --> 00:12:03.110 AUDIENCE: Can I use the same edge more than once? 00:12:03.110 --> 00:12:06.710 JASON KU: Can I use the same edge more than once? 00:12:06.710 --> 00:12:12.980 Right now, you're asking about the distinction in our class 00:12:12.980 --> 00:12:15.980 which we have between paths and simple paths. 00:12:15.980 --> 00:12:18.560 So here, a weighted path doesn't really 00:12:18.560 --> 00:12:21.560 care if we visit an edge more than once. 00:12:21.560 --> 00:12:24.920 So if an edge appears more than once in pi, 00:12:24.920 --> 00:12:28.730 we have to count that more than once in the edge weight-- 00:12:28.730 --> 00:12:30.110 in the weight of the path. 00:12:30.110 --> 00:12:32.630 OK, great question. 00:12:32.630 --> 00:12:39.120 But what we're going to see later on is shortest paths 00:12:39.120 --> 00:12:47.320 cannot repeat an edge more than once in certain contexts. 00:12:47.320 --> 00:12:50.418 So we're going to get to the problem there 00:12:50.418 --> 00:12:51.710 a little later in this lecture. 00:12:51.710 --> 00:12:54.410 And we're going to solve that in tomorrow's lecture. 00:12:54.410 --> 00:12:56.820 But if you have-- 00:12:56.820 --> 00:12:58.570 we're getting a little ahead of ourselves. 00:12:58.570 --> 00:13:01.330 But when we have negative weights in a graph, 00:13:01.330 --> 00:13:02.920 it's possible that things go wrong. 00:13:02.920 --> 00:13:05.020 We're going to get there in about five lines. 00:13:05.020 --> 00:13:06.850 OK, great. 00:13:06.850 --> 00:13:11.095 So a shortest path-- 00:13:17.190 --> 00:13:19.300 and in this case, I'm going to clarify that this 00:13:19.300 --> 00:13:20.680 is the weighted shortest path-- 00:13:23.850 --> 00:13:29.150 is a minimum-- min-i-mum-- 00:13:32.800 --> 00:13:47.140 sure-- is a minimum weight path from s to t. 00:13:47.140 --> 00:13:50.260 Nothing too interesting here, but there's actually 00:13:50.260 --> 00:13:52.090 some subtleties we have to deal with here. 00:13:54.790 --> 00:13:55.900 We're going to call-- 00:13:55.900 --> 00:13:58.930 just like we did with breadth-first search 00:13:58.930 --> 00:14:00.430 when we talked about shortest paths, 00:14:00.430 --> 00:14:05.620 we're going to define an expression for what 00:14:05.620 --> 00:14:10.690 the distance or the shortest path weight 00:14:10.690 --> 00:14:12.220 is between two vertices. 00:14:12.220 --> 00:14:14.830 And I'm going to represent that by a delta. 00:14:14.830 --> 00:14:23.230 A delta from a vertex s to t is going to be-- 00:14:23.230 --> 00:14:26.350 let's-- I'm going to do the wrong thing first-- 00:14:26.350 --> 00:14:36.960 the minimum over the weight of all paths for all paths pi 00:14:36.960 --> 00:14:43.030 from s to t. 00:14:43.030 --> 00:14:46.930 OK, so there's a couple things that go wrong here. 00:14:46.930 --> 00:14:50.410 First thing that goes wrong is the same thing that went wrong 00:14:50.410 --> 00:14:53.290 with breadth-first search. 00:14:53.290 --> 00:14:56.080 Anyone remember what could go wrong with breadth-first search 00:14:56.080 --> 00:14:57.357 for this delta definition? 00:14:57.357 --> 00:14:58.690 AUDIENCE: Maybe there's no path. 00:14:58.690 --> 00:15:01.450 JASON KU: Maybe there's no path, right. 00:15:01.450 --> 00:15:05.485 So except if no path. 00:15:07.990 --> 00:15:11.620 Just by convention, we're setting 00:15:11.620 --> 00:15:15.580 delta s, t, to equal infinity. 00:15:15.580 --> 00:15:20.480 But there's one additional problem 00:15:20.480 --> 00:15:26.320 with weighted shortest paths, and it's a little subtle. 00:15:26.320 --> 00:15:29.380 It's possible that a finite shortest-- 00:15:29.380 --> 00:15:33.460 finite length shortest path doesn't exist. 00:15:33.460 --> 00:15:34.850 And what do I mean by that? 00:15:34.850 --> 00:15:41.630 It means I could keep going through edges in my graph 00:15:41.630 --> 00:15:46.660 and continually getting a shorter path. 00:15:46.660 --> 00:15:51.670 So if the shortest-- the minimum weight of a path from s 00:15:51.670 --> 00:15:54.910 to t actually goes through an infinite number of edges, 00:15:54.910 --> 00:15:57.550 then this isn't really well-defined. 00:15:57.550 --> 00:16:02.410 So I'm going to change this minimum here to-- 00:16:02.410 --> 00:16:07.870 in mathematics we would, just to be specific, 00:16:07.870 --> 00:16:10.670 we call it an infimum. 00:16:10.670 --> 00:16:16.430 So if in the case where the weight of a shortest path 00:16:16.430 --> 00:16:20.570 can approach arbitrarily small, then we'll 00:16:20.570 --> 00:16:24.140 call this thing minus infinity. 00:16:24.140 --> 00:16:26.365 So when does that occur? 00:16:31.290 --> 00:16:33.320 When does that occur? 00:16:33.320 --> 00:16:36.690 When could we have our shortest path 00:16:36.690 --> 00:16:38.430 go through lots and lots of vertices? 00:16:38.430 --> 00:16:41.510 Well, let's actually take a look at this example here. 00:16:41.510 --> 00:16:43.490 Can someone tell me what the shortest path 00:16:43.490 --> 00:16:50.470 is from a to actually any vertex in this graph? 00:16:55.350 --> 00:16:57.310 AUDIENCE: b, f, g, c. 00:16:57.310 --> 00:16:58.790 JASON KU: Ah, OK. 00:16:58.790 --> 00:17:03.680 So well, we could look at this path I have to b. 00:17:03.680 --> 00:17:05.450 Let's just take a look at b. 00:17:05.450 --> 00:17:08.422 I have a path going from a to b that is minus 5. 00:17:08.422 --> 00:17:09.380 OK, that's pretty good. 00:17:09.380 --> 00:17:10.940 That's pretty small. 00:17:10.940 --> 00:17:17.329 And it seems that if I go around this graph through another way, 00:17:17.329 --> 00:17:18.630 it might be bigger. 00:17:18.630 --> 00:17:24.020 So I go 7 plus 3 plus 8-- that's 15-- 00:17:24.020 --> 00:17:26.300 minus 1-- that's 14. 00:17:26.300 --> 00:17:30.590 That's much bigger than minus 5, so it seems like minus 5 00:17:30.590 --> 00:17:32.030 should be good, right? 00:17:32.030 --> 00:17:35.150 Anyone have a problem with this path or a problem 00:17:35.150 --> 00:17:37.680 with this being the shortest path? 00:17:37.680 --> 00:17:39.980 And what your colleague just informed 00:17:39.980 --> 00:17:43.790 me was that there is something interesting happening here 00:17:43.790 --> 00:17:46.350 in this graph in particular. 00:17:46.350 --> 00:17:52.620 We have a cycle from b to f to g to c 00:17:52.620 --> 00:17:57.060 that has negative total weight back to b. 00:17:57.060 --> 00:18:03.070 This has minus 4 plus 2 plus 1 minus 1. 00:18:03.070 --> 00:18:09.420 So that total cycle has a cycle weight of minus 2, 00:18:09.420 --> 00:18:11.710 this negative weight cycle. 00:18:11.710 --> 00:18:16.740 So if I want to get to b, I could go there 00:18:16.740 --> 00:18:19.810 via this minus 5 weight edge. 00:18:19.810 --> 00:18:27.190 But every time I circled around this cycle, I incur minus 2 00:18:27.190 --> 00:18:29.410 to my path weight. 00:18:29.410 --> 00:18:32.740 So I just keep going around this cycle over and over 00:18:32.740 --> 00:18:35.540 and over and over and over and over again, 00:18:35.540 --> 00:18:41.290 and I don't have any finite length minimum weight path. 00:18:41.290 --> 00:18:46.720 And in such cases, we just say that delta is minus infinity. 00:18:46.720 --> 00:18:51.575 So the problem here is that we could have negative weight 00:18:51.575 --> 00:18:52.075 cycles-- 00:18:55.360 --> 00:18:57.340 deserves a capital letter-- 00:18:57.340 --> 00:19:03.115 Negative weight cycles. 00:19:07.390 --> 00:19:08.270 It's a problem. 00:19:11.690 --> 00:19:25.230 In particular, if there exists a path from s 00:19:25.230 --> 00:19:43.630 to some vertex v that goes through a vertex 00:19:43.630 --> 00:19:56.840 on a negative weight cycle, then I 00:19:56.840 --> 00:20:00.020 can take that path to that vertex, circle 00:20:00.020 --> 00:20:04.280 around the negative weight cycle, and then proceed to v, 00:20:04.280 --> 00:20:06.560 and I can take that cycle as many times as I want. 00:20:06.560 --> 00:20:14.360 Then this delta s,v we're going to set to minus infinity. 00:20:14.360 --> 00:20:18.260 And in such cases, in our shortest paths algorithm, 00:20:18.260 --> 00:20:23.040 we don't really care about what the shortest path is. 00:20:23.040 --> 00:20:25.640 We're not even going to deal with parent pointers 00:20:25.640 --> 00:20:30.675 here, because there is no finite length shortest path. 00:20:30.675 --> 00:20:33.050 So I'm just going to kind of throw up my hands in the air 00:20:33.050 --> 00:20:35.900 and say, you know what, I can't return you a shortest path, 00:20:35.900 --> 00:20:41.230 but I might want to return to you a negative weight cycle. 00:20:41.230 --> 00:20:45.370 If you told me that this thing has bad weight, 00:20:45.370 --> 00:20:47.650 maybe I want you to tell me what a path is 00:20:47.650 --> 00:20:52.090 that goes through a negative weight cycle to get back to s. 00:20:52.090 --> 00:20:54.340 So that's what we're going to talk about next lecture. 00:20:54.340 --> 00:20:57.375 This lecture, we are not going to talk about that. 00:20:57.375 --> 00:20:59.500 We are going to talk about weighted shortest paths, 00:20:59.500 --> 00:21:00.130 though. 00:21:00.130 --> 00:21:06.520 That's what the remainder of this unit on graphs 00:21:06.520 --> 00:21:08.980 is really about is weighted shortest paths. 00:21:08.980 --> 00:21:11.890 OK, so in weighted shortest paths, 00:21:11.890 --> 00:21:14.770 we actually know an algorithm already 00:21:14.770 --> 00:21:21.640 to solve a subset of weighted shortest parts, namely BFS, 00:21:21.640 --> 00:21:23.220 right? 00:21:23.220 --> 00:21:27.390 Now, you're like wait, Jason, BFS 00:21:27.390 --> 00:21:29.010 doesn't solve weighted shortest paths. 00:21:29.010 --> 00:21:31.050 We didn't even know about weighted graphs then. 00:21:31.050 --> 00:21:33.840 How does that solve weighted shortest paths? 00:21:33.840 --> 00:21:37.860 Well, there's a couple cases where 00:21:37.860 --> 00:21:41.850 we might be able to reduce to solving shortest paths using 00:21:41.850 --> 00:21:42.630 BFS. 00:21:42.630 --> 00:21:44.220 Can anyone think of such a scenario? 00:21:50.110 --> 00:21:55.150 So let's say, I mean, kind of what we did before was we 00:21:55.150 --> 00:21:57.910 counted the number of edges. 00:21:57.910 --> 00:22:03.680 So if we gave a weight of 1 to every edge in my graph, 00:22:03.680 --> 00:22:08.000 then just that graph, that weighted graph, 00:22:08.000 --> 00:22:12.320 corresponds to an unweighted graph using the other distance 00:22:12.320 --> 00:22:14.120 metric. 00:22:14.120 --> 00:22:17.560 So in that case, BFS just solves our problem. 00:22:17.560 --> 00:22:21.590 And in fact, we can generalize further. 00:22:21.590 --> 00:22:27.340 What if all of our weights were positive, but the same value? 00:22:27.340 --> 00:22:30.730 If it was all positive and the same value, 00:22:30.730 --> 00:22:35.680 then we could just divide by that value. 00:22:35.680 --> 00:22:40.870 Now we have an unweighted graph which we can run BFS, and then 00:22:40.870 --> 00:22:44.410 multiply shortest path distances by that value later on. 00:22:44.410 --> 00:22:48.430 And in fact, there's one further generalization we can make, 00:22:48.430 --> 00:22:51.310 which is a little bit of a tricky graph transformation 00:22:51.310 --> 00:22:53.460 problem. 00:22:53.460 --> 00:22:56.940 But we can also get this linear time algorithm 00:22:56.940 --> 00:23:00.480 for weighted single-source shortest paths 00:23:00.480 --> 00:23:06.570 in contexts where the weights aren't that large. 00:23:06.570 --> 00:23:11.100 So if I have positive edge weights-- 00:23:11.100 --> 00:23:13.380 if I have a positive edge weight, let's say-- 00:23:16.150 --> 00:23:20.170 using my weight color here-- 00:23:20.170 --> 00:23:25.720 that's, like, weight of 4, that's 00:23:25.720 --> 00:23:28.900 kind of problematic, because I don't know how to simulate 00:23:28.900 --> 00:23:30.820 that using an unweighted graph. 00:23:30.820 --> 00:23:32.350 Or do I? 00:23:32.350 --> 00:23:34.090 Anyone have an idea of how I could 00:23:34.090 --> 00:23:41.140 simulate an edge of weight 4 with an unweighted graph? 00:23:41.140 --> 00:23:41.640 Yeah. 00:23:41.640 --> 00:23:43.800 AUDIENCE: Have four edges of weight 1. 00:23:43.800 --> 00:23:45.780 JASON KU: Yeah, I can just put four edges 00:23:45.780 --> 00:23:48.960 of weight 1 in parallel here-- 00:23:48.960 --> 00:23:53.340 I'm sorry, in series, the opposite of parallel. 00:23:53.340 --> 00:24:02.880 I can just convert this here into 1, 2, 3, 4 edges. 00:24:02.880 --> 00:24:06.130 And if I do that for every edge in my graph 00:24:06.130 --> 00:24:08.340 and we have positive edge weights, 00:24:08.340 --> 00:24:10.440 then that transformation can hold. 00:24:10.440 --> 00:24:13.050 Now, that's not necessarily a good transformation to make. 00:24:13.050 --> 00:24:14.886 Why? 00:24:14.886 --> 00:24:16.740 AUDIENCE: The weight might be very big. 00:24:16.740 --> 00:24:19.170 JASON KU: Yeah, the weights might be very big 00:24:19.170 --> 00:24:22.860 compared to the number of vertices and edges in my graph. 00:24:22.860 --> 00:24:26.760 However, if the sum of all weights in my graph 00:24:26.760 --> 00:24:32.400 is asymptotically less than v plus e, 00:24:32.400 --> 00:24:34.530 we can get a linear time algorithm again 00:24:34.530 --> 00:24:35.670 by reducing to BFS. 00:24:35.670 --> 00:24:36.630 OK, so that's great. 00:24:40.650 --> 00:24:43.300 But in general, that gives us a linear time algorithm 00:24:43.300 --> 00:24:46.530 in these very special cases. 00:24:46.530 --> 00:24:48.390 And in general, it's an open problem. 00:24:48.390 --> 00:24:52.170 We don't know whether we can solve 00:24:52.170 --> 00:24:56.670 the single-source shortest paths problem 00:24:56.670 --> 00:25:00.960 in the weighted context for general graphs in linear time. 00:25:00.960 --> 00:25:03.960 We don't know how to do it. 00:25:03.960 --> 00:25:08.560 But what we do know are some algorithms that do pretty well. 00:25:08.560 --> 00:25:11.460 And that's what we use all the time. 00:25:11.460 --> 00:25:15.240 But one more special case we're going to go over today 00:25:15.240 --> 00:25:18.990 is when we have this really nice structure where 00:25:18.990 --> 00:25:22.350 we have a DAG, a Directed Acyclic 00:25:22.350 --> 00:25:28.030 Graph, like we were talking about in the last lecture. 00:25:28.030 --> 00:25:29.920 For any set of edge weights-- 00:25:29.920 --> 00:25:33.580 remember, with BFS, we needed to restrict our edge weights 00:25:33.580 --> 00:25:39.710 to be positive and maybe bounded to get this good running time? 00:25:39.710 --> 00:25:43.193 For any set of edge weights, if our graph structure is DAG-- 00:25:43.193 --> 00:25:45.110 it really has nothing to do with the weights-- 00:25:45.110 --> 00:25:47.900 if the graph structure is a DAG, then 00:25:47.900 --> 00:25:51.320 we can actually solve this single-source shortest paths 00:25:51.320 --> 00:25:56.630 problem in linear time, which is pretty awesome. 00:25:56.630 --> 00:26:02.720 Now, for general graphs, we're going to show you in the next 00:26:02.720 --> 00:26:06.980 lecture how to, for any graph-- even with cycles, 00:26:06.980 --> 00:26:08.780 even with negative weight cycles-- 00:26:08.780 --> 00:26:10.280 we're going to show you how to solve 00:26:10.280 --> 00:26:11.690 this single-source shortest paths 00:26:11.690 --> 00:26:14.910 problem in something like a quadratic running time bound. 00:26:14.910 --> 00:26:17.090 Now, this isn't the best known, but it's 00:26:17.090 --> 00:26:21.390 a really practical algorithm and people use it all the time. 00:26:21.390 --> 00:26:25.610 And we are going to show Bellman-Ford 00:26:25.610 --> 00:26:28.940 in the context of the DAG algorithm 00:26:28.940 --> 00:26:32.900 we're going to solve today. 00:26:32.900 --> 00:26:35.030 So that's the very general case in terms 00:26:35.030 --> 00:26:37.370 of restrictions on our graph. 00:26:37.370 --> 00:26:43.070 But in reality, most problems that come up in applications 00:26:43.070 --> 00:26:46.940 occur with graphs that have positive edge weights. 00:26:46.940 --> 00:26:48.380 You can think of a road network. 00:26:48.380 --> 00:26:51.110 You've got-- or non-negative ones anyway. 00:26:51.110 --> 00:26:56.000 You're traveling along, and it's not ever 00:26:56.000 --> 00:26:58.940 useful to go back to where you came from, 00:26:58.940 --> 00:27:03.470 because you want to make progress to where you're going. 00:27:03.470 --> 00:27:06.873 So in the context where you don't have negative weights, 00:27:06.873 --> 00:27:09.290 you don't have this problem where you have negative weight 00:27:09.290 --> 00:27:10.580 cycles. 00:27:10.580 --> 00:27:15.320 We can actually do a lot better by exploiting that property. 00:27:15.320 --> 00:27:17.900 And we get a bound that's a little bit-- that looks 00:27:17.900 --> 00:27:21.050 a little bit more like n log n. 00:27:21.050 --> 00:27:23.000 It's pretty close to linear. 00:27:23.000 --> 00:27:27.740 You're losing a log factor on the number of vertices. 00:27:27.740 --> 00:27:28.873 But it's pretty good. 00:27:28.873 --> 00:27:30.290 This is called Dijkstra, and we'll 00:27:30.290 --> 00:27:32.270 get to that in two lectures. 00:27:32.270 --> 00:27:35.360 OK, so that's the roadmap of what 00:27:35.360 --> 00:27:39.200 we're going to do for at least the next three lectures. 00:27:39.200 --> 00:27:41.930 But before we go on to showing you 00:27:41.930 --> 00:27:46.640 how to solve single-source shortest paths in a DAG using 00:27:46.640 --> 00:27:52.800 this algorithm that I'm calling DAG relaxation here, 00:27:52.800 --> 00:27:55.710 I'm going to go back to a thing that we talked 00:27:55.710 --> 00:27:57.908 about in breadth-first search, where 00:27:57.908 --> 00:28:00.450 in breadth-first search when we solved single-source shortest 00:28:00.450 --> 00:28:04.050 paths, we output two things. 00:28:04.050 --> 00:28:08.190 We output single-source shortest paths, these deltas, 00:28:08.190 --> 00:28:13.140 for the other definition of distance, the weights-- 00:28:13.140 --> 00:28:16.500 I mean, not the weights, the distances, the shortest 00:28:16.500 --> 00:28:18.600 distances. 00:28:18.600 --> 00:28:20.790 But we also returned parent pointers. 00:28:20.790 --> 00:28:24.990 We return parent pointers back along paths 00:28:24.990 --> 00:28:28.080 to the source along shortest paths. 00:28:28.080 --> 00:28:29.970 We call this the shortest paths tree. 00:28:29.970 --> 00:28:33.450 So I'm going to revisit this topic of shortest paths tree-- 00:28:33.450 --> 00:28:42.750 shortest path trees-- shortest path trees. 00:28:45.390 --> 00:28:47.550 And in particular, it's kind of going 00:28:47.550 --> 00:28:51.930 to be annoying to talk about both of these quantities-- 00:28:51.930 --> 00:28:56.160 distances and parent pointers-- as we go through all three 00:28:56.160 --> 00:28:57.450 of these algorithms. 00:28:57.450 --> 00:29:01.080 It's basically going to be bookkeeping to-- 00:29:01.080 --> 00:29:04.110 distances are actually sufficient for us 00:29:04.110 --> 00:29:07.560 to reconstruct parent pointers if we need them later. 00:29:07.560 --> 00:29:10.110 So what I'm going to show for you-- prove to you now 00:29:10.110 --> 00:29:12.450 is that, if I give you the shortest path 00:29:12.450 --> 00:29:19.490 distances for the subset of the graph reachable from s 00:29:19.490 --> 00:29:23.600 that doesn't go through negative weight cycles, 00:29:23.600 --> 00:29:25.340 if I'm giving you those distances, 00:29:25.340 --> 00:29:29.330 I can reconstruct parent pointers along shortest paths 00:29:29.330 --> 00:29:35.130 in linear time for any graph I might give you 00:29:35.130 --> 00:29:37.290 if I give you those shortest path distances. 00:29:37.290 --> 00:29:40.200 OK, so that's what I'm going to try to show to you now. 00:29:43.560 --> 00:29:49.290 So here's the algorithm. 00:29:49.290 --> 00:29:51.680 For weighted-- there's the caveat here 00:29:51.680 --> 00:29:52.680 I'm going to write down. 00:29:52.680 --> 00:30:03.000 For weighted shortest paths, only need parent pointers 00:30:03.000 --> 00:30:14.710 for v with finite shortest path distance-- 00:30:17.440 --> 00:30:19.250 only finite shortest path distance. 00:30:19.250 --> 00:30:21.550 We don't care about the infinite ones 00:30:21.550 --> 00:30:23.830 or the minus infinite ones, just the finite ones. 00:30:23.830 --> 00:30:27.100 OK, so here's the algorithm. 00:30:27.100 --> 00:30:36.760 I can initialize all Pv to equal-- 00:30:36.760 --> 00:30:42.730 sorry, oh, getting ahead of myself. 00:30:42.730 --> 00:30:43.676 I'm writing down DAG. 00:30:48.910 --> 00:30:54.150 Init parent pointer data structure to be empty. 00:30:57.920 --> 00:31:00.620 At first, I'm not going to sort any parent pointers. 00:31:00.620 --> 00:31:04.040 But at the beginning, I'm going to set the parent pointer 00:31:04.040 --> 00:31:07.820 of the source to be none. 00:31:07.820 --> 00:31:13.390 So that's what we kind of did in breadth-first search as well. 00:31:13.390 --> 00:31:16.600 Now, what I've given you is-- 00:31:16.600 --> 00:31:18.970 I'm trying to show that, given all the shortest path 00:31:18.970 --> 00:31:22.100 distances, I can construct these parent pointers correctly. 00:31:22.100 --> 00:31:34.110 So what I'm going to do is, for each vertex u in my graph, 00:31:34.110 --> 00:31:47.640 where my delta s of u is finite, what am I going to do? 00:31:47.640 --> 00:31:50.930 I'm going to say, well, let's take a look 00:31:50.930 --> 00:31:54.380 at all my outgoing neighbors. 00:31:54.380 --> 00:31:57.920 This is kind of what we do in every graph algorithm. 00:31:57.920 --> 00:32:10.410 For each v in the adjacency, the outgoing adjacencies of u, 00:32:10.410 --> 00:32:18.000 if there is no parent pointer assigned to this v, 00:32:18.000 --> 00:32:20.274 there's the potential that i-- u-- 00:32:20.274 --> 00:32:24.870 [CHUCKLES] I, you-- this u, this vertex u, is the parent of v. 00:32:24.870 --> 00:32:25.740 It's possible. 00:32:25.740 --> 00:32:29.940 It's some incoming edge to v. 00:32:29.940 --> 00:32:34.440 When will it be an incoming edge to v? 00:32:34.440 --> 00:32:40.860 If v not in P-- 00:32:40.860 --> 00:32:44.020 I haven't assigned it a parent pointer-- 00:32:44.020 --> 00:32:51.070 and-- so this means it could be my parent. 00:32:51.070 --> 00:32:57.270 When is it my parent along the shortest path? 00:32:57.270 --> 00:32:57.975 Sure. 00:32:57.975 --> 00:33:00.008 AUDIENCE: Sum the distance along the edge 00:33:00.008 --> 00:33:01.470 to the distance of the other. 00:33:01.470 --> 00:33:04.635 JASON KU: Yeah, so we have some edge from u 00:33:04.635 --> 00:33:08.130 to v. It has some weight. 00:33:08.130 --> 00:33:11.400 If I already know the shortest path distance to u, 00:33:11.400 --> 00:33:15.210 and I know the shortest path distance to v, 00:33:15.210 --> 00:33:20.920 if the shortest path distance from s to u-- 00:33:20.920 --> 00:33:22.370 let's draw a picture here. 00:33:22.370 --> 00:33:25.900 We've got s, we've got some path here to u, 00:33:25.900 --> 00:33:28.720 and we know we've got an edge from u 00:33:28.720 --> 00:33:37.510 to v. If this shortest path distance plus this edge weight 00:33:37.510 --> 00:33:43.570 is equal to the shortest path distance from s to v, 00:33:43.570 --> 00:33:44.290 then it better-- 00:33:44.290 --> 00:33:46.790 I mean, there may be more than one shortest path, 00:33:46.790 --> 00:33:48.640 but this is certainly a shortest path, 00:33:48.640 --> 00:33:52.240 so we can assign a parent pointer back to u. 00:33:52.240 --> 00:33:54.100 So let's write that condition down. 00:33:54.100 --> 00:33:58.630 If the shortest path distance from s to v 00:33:58.630 --> 00:34:02.620 equals the shortest path distance from s to u, 00:34:02.620 --> 00:34:09.530 and then traversing the edge from u to v, 00:34:09.530 --> 00:34:26.239 then exists shortest path that uses edge u, v, 00:34:26.239 --> 00:34:28.610 in particular this one. 00:34:31.150 --> 00:34:37.043 So set the parent of u-- 00:34:37.043 --> 00:34:40.639 of v to u. 00:34:40.639 --> 00:34:42.900 OK, so this is the algorithm. 00:34:42.900 --> 00:34:46.340 I'm not going to prove to you that this is correct. 00:34:46.340 --> 00:34:48.590 But it kind of intuitively makes sense, right? 00:34:48.590 --> 00:34:52.219 If I have these shortest path distances, 00:34:52.219 --> 00:34:55.699 you can prove by induction that not only does this parent 00:34:55.699 --> 00:35:00.830 pointer point to the right place along some shortest path here, 00:35:00.830 --> 00:35:02.480 but it also does so in linear time, 00:35:02.480 --> 00:35:04.820 because I'm looping over all the vertices 00:35:04.820 --> 00:35:07.760 and looping over its outgoing adjacencies once. 00:35:07.760 --> 00:35:14.310 Same analysis as we had for both BFS and DFS, essentially. 00:35:14.310 --> 00:35:18.770 And then, since we can do this, since we can compute parent 00:35:18.770 --> 00:35:20.990 pointers from these distances, we're 00:35:20.990 --> 00:35:25.822 going to ignore computing these parent pointers from now on. 00:35:25.822 --> 00:35:28.280 We're just going to concentrate on computing the distances, 00:35:28.280 --> 00:35:32.600 because we're going to have to take linear time anyway 00:35:32.600 --> 00:35:34.040 at least. 00:35:34.040 --> 00:35:36.270 And all these other things take more time. 00:35:36.270 --> 00:35:39.080 So we can compute the distances in more time, 00:35:39.080 --> 00:35:41.480 and then compute the parents after. 00:35:41.480 --> 00:35:43.750 OK, so that's what we're going to do. 00:35:47.090 --> 00:35:50.795 So now, with all that buildup, let's show an algorithm. 00:35:50.795 --> 00:35:56.480 [CHUCKLES] How do we compute single-source shortest paths 00:35:56.480 --> 00:36:01.150 in a DAG in linear time? 00:36:01.150 --> 00:36:02.860 Well, a DAG-- I mean, this is actually 00:36:02.860 --> 00:36:07.390 a super useful, convenient thing in algorithms in general. 00:36:07.390 --> 00:36:10.210 DAGs are just nice things. 00:36:10.210 --> 00:36:12.310 They're kind of ordered in a way. 00:36:12.310 --> 00:36:14.103 There's this topological sort order 00:36:14.103 --> 00:36:15.520 that we were talking about before. 00:36:15.520 --> 00:36:18.330 This is going to play a key role. 00:36:18.330 --> 00:36:21.490 There's a really nice structure to DAGs not having cycles, 00:36:21.490 --> 00:36:25.000 not having to deal with this negative weight cycle problem. 00:36:25.000 --> 00:36:30.820 You can only go in one direction along this graph. 00:36:30.820 --> 00:36:32.900 It's a very nice structure to exploit. 00:36:32.900 --> 00:36:35.800 And so we're going to exploit it. 00:36:35.800 --> 00:36:37.920 And here's the idea. 00:36:37.920 --> 00:36:47.210 DAG relaxation, what it's going to do 00:36:47.210 --> 00:36:50.300 is it's going to start out with some estimates of what 00:36:50.300 --> 00:36:53.180 these distances should be. 00:36:53.180 --> 00:37:01.740 So maintain distance estimates. 00:37:06.020 --> 00:37:08.870 And now I'm going to try to be careful here 00:37:08.870 --> 00:37:11.030 about how I draw my Ds. 00:37:14.510 --> 00:37:20.290 This is a d, this is a delta. 00:37:20.290 --> 00:37:22.270 This is shortest paths. 00:37:22.270 --> 00:37:24.970 This is a distance estimate. 00:37:24.970 --> 00:37:26.470 So that's what I'm going to be using 00:37:26.470 --> 00:37:29.750 for the rest of this time. 00:37:29.750 --> 00:37:35.890 So we're going to maintain these estimates of distance d, which 00:37:35.890 --> 00:37:44.430 are going to start at initially infinite. 00:37:44.430 --> 00:37:46.020 I don't know what they are. 00:37:46.020 --> 00:37:47.810 I don't know what the shortest paths are, 00:37:47.810 --> 00:37:52.040 but they better be less than infinite or else I don't care. 00:37:52.040 --> 00:37:55.160 So that's the worst case scenario. 00:37:55.160 --> 00:37:56.990 It can't be worse than this-- 00:37:56.990 --> 00:37:59.130 for every vertex. 00:37:59.130 --> 00:38:03.650 And we're going to maintain the property that 00:38:03.650 --> 00:38:08.840 estimates upper bound-- 00:38:15.100 --> 00:38:16.990 that should probably be two words-- 00:38:16.990 --> 00:38:20.950 upper bound delta s, v-- 00:38:25.337 --> 00:38:27.670 we're going to maintain that they upper-bound this thing 00:38:27.670 --> 00:38:37.100 and gradually lower until they're equal. 00:38:39.820 --> 00:38:40.820 So this is the idea. 00:38:40.820 --> 00:38:44.770 We start from an over-estimate, an upper bound on the distance 00:38:44.770 --> 00:38:45.910 estimate. 00:38:45.910 --> 00:38:49.540 And then we're repeatedly going to lower that value 00:38:49.540 --> 00:38:53.070 as we gain more information about the graph, 00:38:53.070 --> 00:38:56.680 maintaining that we're always upper-bounding the distance. 00:38:56.680 --> 00:38:58.680 And we're going to keep doing it, keep doing it, 00:38:58.680 --> 00:39:02.880 keep doing it, until, as we will try to prove to you, 00:39:02.880 --> 00:39:06.240 these estimates reach, actually reach down, all the way 00:39:06.240 --> 00:39:09.360 to our shortest path distances. 00:39:09.360 --> 00:39:10.755 So when do we lower these things? 00:39:13.810 --> 00:39:16.600 When do we lower these things? 00:39:16.600 --> 00:39:19.210 We are going to lower these distance estimates whenever 00:39:19.210 --> 00:39:21.310 the distance estimates violate what 00:39:21.310 --> 00:39:24.100 we're going to call the triangle inequality. 00:39:24.100 --> 00:39:26.038 OK, what is the triangle inequality? 00:39:37.000 --> 00:39:39.565 Triangle inequality is actually a pretty intuitive notion. 00:39:42.130 --> 00:39:44.480 It's basically saying, if I have three points-- 00:39:44.480 --> 00:39:49.410 thus, triangle-- maybe bigger so I can write a letter in them. 00:39:52.450 --> 00:39:58.210 It's basically saying that if I have a vertex u, a vertex v, 00:39:58.210 --> 00:40:04.460 vertex x, for example, the shortest path distance-- 00:40:08.950 --> 00:40:12.310 the shortest path distance delta of u, v-- 00:40:12.310 --> 00:40:15.310 that's the shortest distance from u to v-- 00:40:15.310 --> 00:40:20.810 it can't be bigger then a shortest path from u to v 00:40:20.810 --> 00:40:22.280 that also goes through x. 00:40:25.770 --> 00:40:28.670 Of my paths, I'm now restricting the paths 00:40:28.670 --> 00:40:30.960 I have to the ones that go through x. 00:40:36.030 --> 00:40:38.010 The shortest path distance from u to v 00:40:38.010 --> 00:40:41.340 can't be bigger than restricting paths that go through 00:40:41.340 --> 00:40:43.710 x and taking that shortest distance, 00:40:43.710 --> 00:40:45.870 getting the shortest path distance from here 00:40:45.870 --> 00:40:49.110 and adding it to the shortest path distance here-- 00:40:49.110 --> 00:41:01.020 delta u, x, delta x, v. That's just a statement of, 00:41:01.020 --> 00:41:04.980 I'm restricting to a subset of the paths. 00:41:04.980 --> 00:41:07.560 I can't decrease my minimum distance. 00:41:07.560 --> 00:41:10.480 So this is the statement of the triangle inequality, 00:41:10.480 --> 00:41:14.530 that the shortest path distance from u to v 00:41:14.530 --> 00:41:20.560 can't be bigger than the shortest path distance from u 00:41:20.560 --> 00:41:23.800 to x plus the shortest path distance from x 00:41:23.800 --> 00:41:29.410 to v for any x in my graph that's not u and v. 00:41:29.410 --> 00:41:32.260 So that's the triangle inequality. 00:41:32.260 --> 00:41:34.240 Pretty intuitive notion, right? 00:41:34.240 --> 00:41:36.100 Why is this useful? 00:41:36.100 --> 00:41:37.090 OK, well, if I find-- 00:41:39.980 --> 00:41:46.580 if I find an edge in my graph, if there's an edge u, v, 00:41:46.580 --> 00:41:59.560 in my graph such that this condition is violated 00:41:59.560 --> 00:42:01.840 for the estimates that I have-- 00:42:01.840 --> 00:42:05.050 it obviously can't be violated on my shortest path distances, 00:42:05.050 --> 00:42:08.230 but if it violates it on the estimates-- 00:42:08.230 --> 00:42:13.960 u, v, is bigger than u, x-- 00:42:13.960 --> 00:42:17.580 sorry, u-- how am I going to do this? 00:42:17.580 --> 00:42:20.670 I want this to be s. 00:42:20.670 --> 00:42:25.850 I'm calculating shortest path distances from s 00:42:25.850 --> 00:42:28.640 and shortest path distances from s 00:42:28.640 --> 00:42:35.330 to some incoming vertex u plus the edge weight from u 00:42:35.330 --> 00:42:39.890 to v. All right, so what is this doing? 00:42:39.890 --> 00:42:44.540 I have some edge u, v in my graph. 00:42:44.540 --> 00:42:47.840 Basically, what I've said is that I have some distance 00:42:47.840 --> 00:42:52.240 estimate to u, but going through-- 00:42:52.240 --> 00:42:55.660 making a path to v by going through u, 00:42:55.660 --> 00:42:59.380 and then the edge from u to v is better 00:42:59.380 --> 00:43:03.440 than my current estimate, my shortest path 00:43:03.440 --> 00:43:08.222 estimate to v. That's bad, right? 00:43:08.222 --> 00:43:09.930 That's violating the triangle inequality. 00:43:09.930 --> 00:43:12.150 These cannot be the right weights. 00:43:12.150 --> 00:43:14.670 These cannot be the right distances. 00:43:14.670 --> 00:43:17.820 So how we're going to do that is lower-- 00:43:17.820 --> 00:43:21.570 this is what we said, repeatedly lower these distance estimates. 00:43:21.570 --> 00:43:24.720 I'm going to lower this guy down to equal this thing. 00:43:24.720 --> 00:43:28.110 In a sense, this constraint was violated. 00:43:28.110 --> 00:43:30.630 But now we're relaxing that constraint 00:43:30.630 --> 00:43:34.140 so that this is no longer violated. 00:43:34.140 --> 00:43:38.370 So relax is a little weird word here. 00:43:38.370 --> 00:43:40.350 We're using it for historical reasons. 00:43:40.350 --> 00:43:42.750 But that's what we mean by when we say relax. 00:43:42.750 --> 00:43:45.390 This thing is a violated constraint. 00:43:45.390 --> 00:43:48.660 It's got some pressure to be resolved. 00:43:48.660 --> 00:43:50.700 And so what we're doing is, to resolve it, 00:43:50.700 --> 00:43:53.702 we're just setting this guy equal to this, 00:43:53.702 --> 00:43:55.785 so it at least resolves, locally, that constraint. 00:43:55.785 --> 00:44:00.300 Now, it may violate the triangle inequality other places now 00:44:00.300 --> 00:44:01.650 that we've done this change. 00:44:01.650 --> 00:44:04.830 But at least this constraint is now relaxed and satisfied. 00:44:04.830 --> 00:44:25.040 OK, so relax edge by lowering d of s, v, to this thing. 00:44:25.040 --> 00:44:28.310 That's what we're going to mean by relaxing an edge. 00:44:28.310 --> 00:44:35.500 And relaxing an edge is what I'll call safe. 00:44:35.500 --> 00:44:38.690 It's safe to do. 00:44:38.690 --> 00:44:40.820 What do I mean by relaxation is safe? 00:44:40.820 --> 00:44:47.210 It means that as I am computing these shortest path distances, 00:44:47.210 --> 00:44:52.300 I'm going to maintain this property that each one of these 00:44:52.300 --> 00:44:55.300 estimates-- sorry, these estimates here-- 00:44:55.300 --> 00:44:56.880 has the property that it's either 00:44:56.880 --> 00:45:05.800 infinite or it is the weight of some path to v. So 00:45:05.800 --> 00:45:07.060 that's the thing the-- 00:45:07.060 --> 00:45:08.290 relaxation is safe. 00:45:17.940 --> 00:45:27.960 OK, so each distance estimate, s, v, 00:45:27.960 --> 00:45:44.820 is weight of some path from s to v or infinite. 00:45:44.820 --> 00:45:48.810 And this is a pretty easy thing to prove. 00:45:48.810 --> 00:45:52.410 If I had the invariant that these 00:45:52.410 --> 00:45:54.220 were all weights of shortest paths, 00:45:54.220 --> 00:45:57.090 let's try to relax an edge. 00:45:57.090 --> 00:46:00.450 And we need to show that this property is maintained. 00:46:00.450 --> 00:46:04.140 Relax edge u, v, OK? 00:46:04.140 --> 00:46:08.960 Now, if I relax edge u, v, what do I do? 00:46:08.960 --> 00:46:12.490 I set this thing-- 00:46:12.490 --> 00:46:16.540 or, sorry, I set this thing, my shortest path distance 00:46:16.540 --> 00:46:19.600 to v, to be this thing plus this thing. 00:46:19.600 --> 00:46:23.110 There's a weight of an edge from u to v. 00:46:23.110 --> 00:46:27.040 Now, by my assumption that we're maintaining 00:46:27.040 --> 00:46:33.590 that this is the weight of some path in my graph, 00:46:33.590 --> 00:46:35.900 if this thing is bigger, I'm setting it 00:46:35.900 --> 00:46:38.330 to the weight of some path on my graph to u, 00:46:38.330 --> 00:46:42.970 plus an edge from u to v, and so this checks out. 00:46:42.970 --> 00:47:00.365 So assign d of s, v, to weight of some path. 00:47:04.770 --> 00:47:07.140 I'm not going to write down all the argument that I just 00:47:07.140 --> 00:47:07.770 had here. 00:47:07.770 --> 00:47:10.950 But basically, since this distance estimate 00:47:10.950 --> 00:47:15.520 was by supposition before the weight of some path to v-- 00:47:15.520 --> 00:47:20.170 to u, then this is, again, the weight of some path to v. 00:47:20.170 --> 00:47:21.210 OK, great. 00:47:21.210 --> 00:47:24.340 So now we're ready to actually go through this algorithm. 00:47:28.950 --> 00:47:35.260 So DAG relaxation, from over there, 00:47:35.260 --> 00:47:42.730 initializes all of our distance estimates to equal infinity, 00:47:42.730 --> 00:47:44.035 just like we did in BFS. 00:47:47.470 --> 00:47:57.340 Then, set my distance estimate to myself to be 0. 00:47:57.340 --> 00:48:00.340 Now, it's possible that this might be minus infinity 00:48:00.340 --> 00:48:03.480 or negative at some point. 00:48:03.480 --> 00:48:06.470 But right now, I'm just setting it to 0. 00:48:06.470 --> 00:48:11.900 And either way, 0 is going to upper-bound the distance to s. 00:48:11.900 --> 00:48:18.340 So in particular, at initialization, 00:48:18.340 --> 00:48:24.580 anything not reachable from s is set correctly. 00:48:24.580 --> 00:48:29.140 And s itself is set as an upper bound to the shortest path 00:48:29.140 --> 00:48:30.430 distance. 00:48:30.430 --> 00:48:48.730 Now we're going to process each vertex u in a topological sort 00:48:48.730 --> 00:48:50.170 order. 00:48:50.170 --> 00:48:54.250 So remember, our input to DAG relaxation is a DAG. 00:48:54.250 --> 00:48:57.220 So this thing has a topological sort order. 00:48:57.220 --> 00:49:00.700 We're going to process these vertices in that order. 00:49:00.700 --> 00:49:02.980 You can imagine we're starting at the top, 00:49:02.980 --> 00:49:05.230 and all my vertices are-- 00:49:05.230 --> 00:49:07.060 all my edges are pointed away from me. 00:49:07.060 --> 00:49:09.280 And I'm just processing each vertex 00:49:09.280 --> 00:49:13.390 down this topological sort line. 00:49:13.390 --> 00:49:15.940 And then for each of these vertices, 00:49:15.940 --> 00:49:17.210 what am I going to do? 00:49:17.210 --> 00:49:20.180 I'm going to look at all the outgoing neighbors. 00:49:20.180 --> 00:49:23.350 And if the triangle inequality is violated, 00:49:23.350 --> 00:49:26.110 I'm going to relax that edge. 00:49:26.110 --> 00:49:28.510 The algorithm is as simple as that. 00:49:28.510 --> 00:49:38.590 For each outgoing neighbor of v-- 00:49:38.590 --> 00:49:41.830 sorry, of u-- 00:49:41.830 --> 00:49:43.590 I always get u and v mixed up here. 00:49:47.780 --> 00:49:53.970 If my shortest path estimate to v 00:49:53.970 --> 00:50:03.000 violates the triangle inequality for an edge, 00:50:03.000 --> 00:50:06.120 for an incoming edge, then I'm going to set-- 00:50:06.120 --> 00:50:13.620 relax u, v, i.e. 00:50:13.620 --> 00:50:25.540 set d, s,v, equal to d, s,u, plus w, u,v. 00:50:25.540 --> 00:50:28.860 So that's the algorithm. 00:50:28.860 --> 00:50:34.990 So if I were to take a look at this example graph over here, 00:50:34.990 --> 00:50:37.180 maybe a is my start vertex. 00:50:37.180 --> 00:50:40.000 I initialize it to-- 00:50:40.000 --> 00:50:41.093 AUDIENCE: DAG. 00:50:41.093 --> 00:50:42.260 JASON KU: This is not a DAG. 00:50:42.260 --> 00:50:43.480 Thank you. 00:50:43.480 --> 00:50:44.440 Let's make it a DAG. 00:50:49.710 --> 00:50:55.740 I claim to you now this is a DAG. 00:50:55.740 --> 00:50:57.810 In particular, a topological sort order 00:50:57.810 --> 00:51:01.870 is when there's a path through all the vertices, 00:51:01.870 --> 00:51:04.350 then there's a unique topological sort order-- 00:51:04.350 --> 00:51:09.990 a, b, e, f, g, h, d, c. 00:51:09.990 --> 00:51:11.400 This is a topological order. 00:51:11.400 --> 00:51:14.470 You can check all the edges. 00:51:14.470 --> 00:51:22.300 So I'm going to start by setting the-- 00:51:22.300 --> 00:51:25.480 actually, let's use e. 00:51:25.480 --> 00:51:26.860 Let's use shortest paths from e. 00:51:26.860 --> 00:51:28.660 Why not? 00:51:28.660 --> 00:51:29.710 Shortest paths from e. 00:51:33.460 --> 00:51:38.050 Vertex a actually comes before e in the topological order. 00:51:38.050 --> 00:51:39.130 So it has no-- 00:51:39.130 --> 00:51:42.550 I mean, its shortest path distance, when I initialize, 00:51:42.550 --> 00:51:44.380 I'm going to initialize this to 0. 00:51:44.380 --> 00:51:49.290 I'm going to initialize this to infinite, infinite, infinite, 00:51:49.290 --> 00:51:52.360 infinite, all these things to infinite. 00:51:52.360 --> 00:51:54.110 These are my estimates. 00:51:54.110 --> 00:51:56.680 These are not quite the shortest paths yet-- 00:51:56.680 --> 00:51:58.900 distances. 00:51:58.900 --> 00:52:02.670 But when I get here, clearly I can't be-- 00:52:07.880 --> 00:52:11.420 distance to me being infinite can never 00:52:11.420 --> 00:52:14.150 violate the triangle inequality with something infinite 00:52:14.150 --> 00:52:14.930 or finite. 00:52:14.930 --> 00:52:16.910 It doesn't matter, right? 00:52:16.910 --> 00:52:19.760 So I don't do anything to a. 00:52:19.760 --> 00:52:23.420 Anything before my source vertex in the topological order 00:52:23.420 --> 00:52:25.928 can't be visited, because it's before 00:52:25.928 --> 00:52:26.970 in the topological order. 00:52:26.970 --> 00:52:28.310 That's kind of the point. 00:52:28.310 --> 00:52:30.620 There's no path from my source vertex 00:52:30.620 --> 00:52:35.300 to anything before it in the topological order. 00:52:35.300 --> 00:52:37.880 So same with b. 00:52:37.880 --> 00:52:41.640 b is before a in the topological order. 00:52:41.640 --> 00:52:48.150 Now, I'm at e, and it's possible we are violating triangle 00:52:48.150 --> 00:52:51.300 inequality, in particular here. 00:52:51.300 --> 00:52:55.440 I think the shortest path distance to f is infinite. 00:52:55.440 --> 00:53:02.760 But actually, if I go to e through this edge with a weight 00:53:02.760 --> 00:53:07.380 3, I know that this is violating triangle inequality. 00:53:07.380 --> 00:53:11.260 So actually, this thing is wrong, 00:53:11.260 --> 00:53:15.060 and I can set it equal to 3. 00:53:15.060 --> 00:53:20.050 Now, that might not be the shortest path distance. 00:53:20.050 --> 00:53:24.840 But right now it's a better estimate, so we've set it. 00:53:24.840 --> 00:53:26.730 Now, I'm moving on. 00:53:26.730 --> 00:53:28.170 I'm done with this guy. 00:53:28.170 --> 00:53:30.930 I move to the next vertex in my topological order. 00:53:30.930 --> 00:53:34.450 And again, I relax edges out of f. 00:53:34.450 --> 00:53:38.280 OK, so here, looking at 8, 3 plus 8 00:53:38.280 --> 00:53:41.550 is better than infinite, so we'll say that that's 11. 00:53:44.880 --> 00:53:50.460 And 3 plus 2 is better than infinite, so that's 5. 00:53:50.460 --> 00:53:52.020 Now, I keep going in the topological 00:53:52.020 --> 00:53:54.000 order. g is the next. 00:53:54.000 --> 00:53:55.860 5 plus 1 is 6. 00:53:55.860 --> 00:53:58.470 OK, so we found a better estimate here. 00:53:58.470 --> 00:54:00.930 So this 11 is not as good. 00:54:00.930 --> 00:54:04.410 6 is better, so we replace it. 00:54:04.410 --> 00:54:06.690 Here, we haven't visited before. 00:54:06.690 --> 00:54:07.530 It's still infinite. 00:54:07.530 --> 00:54:10.470 So 5 plus minus 2 is 3. 00:54:13.110 --> 00:54:14.860 This is the next in the topological order. 00:54:14.860 --> 00:54:20.610 3 plus 9 is bigger than 6, so that's not a shorter path. 00:54:20.610 --> 00:54:23.310 3 plus 4 is certainly smaller than infinite, 00:54:23.310 --> 00:54:25.740 so set that equal to 7. 00:54:25.740 --> 00:54:29.190 Then, 7 plus 5 is also bigger than 6. 00:54:29.190 --> 00:54:33.090 And actually, you can confirm that these are all the shortest 00:54:33.090 --> 00:54:36.620 path distances from e. 00:54:36.620 --> 00:54:39.830 So this algorithm seems to work. 00:54:39.830 --> 00:54:43.350 Does it actually work? 00:54:43.350 --> 00:54:45.140 Let's take a look. 00:54:45.140 --> 00:54:52.970 The claim to you is that at the end of relaxation, 00:54:52.970 --> 00:54:55.010 this algorithm, we've set-- 00:54:58.220 --> 00:55:12.530 claim at end, all the estimates equal shortest path distances. 00:55:16.340 --> 00:55:22.120 The basic idea here is that if I take 00:55:22.120 --> 00:55:26.650 the k-th vertex in the topological order, 00:55:26.650 --> 00:55:29.170 assuming that these distances are all 00:55:29.170 --> 00:55:33.100 equal for the ones before me in the topological order, 00:55:33.100 --> 00:55:35.320 I can prove by induction. 00:55:35.320 --> 00:55:41.730 We can consider a shortest path from s to v, the k-th vertex, 00:55:41.730 --> 00:55:46.040 and look at the vertex preceding me along the shortest path. 00:55:46.040 --> 00:55:52.340 That vertex better be before me in the topological order 00:55:52.340 --> 00:55:54.370 or we're not a DAG. 00:55:54.370 --> 00:55:56.370 And we've already set its shortest path distance 00:55:56.370 --> 00:56:03.040 to be equal to the correct thing by induction. 00:56:03.040 --> 00:56:05.100 So then when we processed u-- 00:56:08.160 --> 00:56:12.390 s to u to v-- 00:56:12.390 --> 00:56:16.680 when we processed u in DAG relaxation 00:56:16.680 --> 00:56:19.530 here, processed the vertex and looked at all 00:56:19.530 --> 00:56:21.960 its outgoing adjacencies, we would 00:56:21.960 --> 00:56:27.055 have relaxed this edge to be no greater than that shortest path 00:56:27.055 --> 00:56:27.555 distance. 00:56:32.320 --> 00:56:35.040 So this is correct. 00:56:35.040 --> 00:56:38.420 You can also think of it as the DAG relaxation 00:56:38.420 --> 00:56:45.360 algorithm for each vertex looks all at its incoming neighbors, 00:56:45.360 --> 00:56:47.610 assuming that their shortest path distances are 00:56:47.610 --> 00:56:49.990 computed correctly already. 00:56:49.990 --> 00:56:51.898 Any shortest path distance to me needs 00:56:51.898 --> 00:56:53.940 to be composed of a shortest path distance to one 00:56:53.940 --> 00:56:58.800 of my incoming neighbors through an edge to me, 00:56:58.800 --> 00:57:00.630 so I can just check all of them. 00:57:00.630 --> 00:57:03.780 That's what DAG relaxation does. 00:57:03.780 --> 00:57:06.340 And again, we're looping over every vertex 00:57:06.340 --> 00:57:08.880 and looking at its adjacencies doing constant work. 00:57:08.880 --> 00:57:11.160 This, again, takes linear time. 00:57:11.160 --> 00:57:15.990 OK, so that's shortest paths and a DAG. 00:57:15.990 --> 00:57:19.560 Next time, we'll look at, for general graphs, 00:57:19.560 --> 00:57:23.730 how we can use kind of the same technique in an algorithm 00:57:23.730 --> 00:57:24.840 we call Bellman-Ford. 00:57:24.840 --> 00:57:27.620 OK, that's it for today.