1 00:00:00,000 --> 00:00:01,976 [SQUEAKING] 2 00:00:01,976 --> 00:00:03,458 [RUSTLING] 3 00:00:03,458 --> 00:00:04,940 [CLICKING] 4 00:00:12,302 --> 00:00:13,260 JASON KU: Hi, everyone. 5 00:00:13,260 --> 00:00:17,075 Welcome to the 11th lecture of 6.006, 6 00:00:17,075 --> 00:00:21,110 our first lecture on weighted shortest paths. 7 00:00:21,110 --> 00:00:25,940 Until now, we've only been talking about graphs that-- 8 00:00:25,940 --> 00:00:29,720 where we measure distance in terms of the number of edges 9 00:00:29,720 --> 00:00:31,390 in a path. 10 00:00:31,390 --> 00:00:33,407 Today, we're going to generalize that notion. 11 00:00:33,407 --> 00:00:35,240 But I just want to go over what we've talked 12 00:00:35,240 --> 00:00:37,790 about in the last two lectures. 13 00:00:37,790 --> 00:00:40,820 In the last two lectures, we've talked about two algorithms, 14 00:00:40,820 --> 00:00:43,670 breadth-first search and depth-first search 15 00:00:43,670 --> 00:00:45,180 to solve a range of problems. 16 00:00:45,180 --> 00:00:47,570 Here's some of the problems that we've been solving. 17 00:00:47,570 --> 00:00:51,740 Single-source shortest paths, where distances are measured 18 00:00:51,740 --> 00:00:54,720 in number of edges in a path. 19 00:00:54,720 --> 00:00:59,240 And we used BFS to solve this problem, 20 00:00:59,240 --> 00:01:01,590 starting from a single source, usually 21 00:01:01,590 --> 00:01:04,550 a vertex s that we call. 22 00:01:04,550 --> 00:01:06,500 And we solve that in linear time. 23 00:01:06,500 --> 00:01:13,490 And we solve that in order v plus e. 24 00:01:13,490 --> 00:01:16,580 That's what we called linear time for a graph. 25 00:01:16,580 --> 00:01:19,820 For the special case of single-source reachability, 26 00:01:19,820 --> 00:01:25,420 here we had to return a shortest path distance for every vertex. 27 00:01:25,420 --> 00:01:29,260 And there was, at most, E things reachable from a vertex. 28 00:01:29,260 --> 00:01:33,210 So this is the bound we got. 29 00:01:33,210 --> 00:01:35,820 But in the special case for single-source reachability, 30 00:01:35,820 --> 00:01:38,310 when our output only has to list the vertices that 31 00:01:38,310 --> 00:01:42,180 are reachable from me, the number 32 00:01:42,180 --> 00:01:45,660 of things reachable in basically a spanning tree 33 00:01:45,660 --> 00:01:48,270 of the connected component of my source 34 00:01:48,270 --> 00:01:52,800 can almost be of order E. And so for all the little singleton 35 00:01:52,800 --> 00:01:57,150 vertices in my graph, I don't really care. 36 00:01:57,150 --> 00:01:59,070 So I can get this in order E, but that's 37 00:01:59,070 --> 00:02:03,780 kind of a little optimization. 38 00:02:03,780 --> 00:02:05,730 The next thing we did was we talked 39 00:02:05,730 --> 00:02:07,510 about connected components. 40 00:02:07,510 --> 00:02:11,160 And we didn't just reduce to using a search algorithm 41 00:02:11,160 --> 00:02:13,710 like a single-source reachability 42 00:02:13,710 --> 00:02:15,750 algorithm like BFS or DFS. 43 00:02:15,750 --> 00:02:20,100 We put a for loop around that to explore the entire graph 44 00:02:20,100 --> 00:02:22,050 by basically saying, if I've explored 45 00:02:22,050 --> 00:02:26,810 one connected component, then I can 46 00:02:26,810 --> 00:02:29,390 look at any other vertex I haven't seen 47 00:02:29,390 --> 00:02:30,950 and explore the next one. 48 00:02:30,950 --> 00:02:33,830 And so that actually with some-- 49 00:02:33,830 --> 00:02:36,500 a little analysis, also got linear time, 50 00:02:36,500 --> 00:02:41,060 because I'm at most traversing any component of my graph once. 51 00:02:41,060 --> 00:02:43,670 That's kind of the idea. 52 00:02:43,670 --> 00:02:47,660 And we can use that using BFS or DFS really, 53 00:02:47,660 --> 00:02:50,630 because we're just trying to get a thing that searches 54 00:02:50,630 --> 00:02:53,150 an entire connected component. 55 00:02:53,150 --> 00:02:55,280 And then this topological sort we did 56 00:02:55,280 --> 00:02:57,210 at the end of the last lecture. 57 00:02:57,210 --> 00:03:00,470 We used full DFS to give an ordering 58 00:03:00,470 --> 00:03:04,710 of the vertices in a DAG-- 59 00:03:04,710 --> 00:03:09,960 maybe I'll specify clearly that this is only for a DAG-- 60 00:03:09,960 --> 00:03:14,020 where we have an ordering of the vertices 61 00:03:14,020 --> 00:03:18,960 so all the edges go forward in that order, for example. 62 00:03:18,960 --> 00:03:20,460 And that we also did in linear time. 63 00:03:25,710 --> 00:03:28,800 All right, in this lecture, and in actually the next four 64 00:03:28,800 --> 00:03:33,480 lectures, what we're going to do is 65 00:03:33,480 --> 00:03:36,750 instead of measuring distance in terms of the number of edges 66 00:03:36,750 --> 00:03:37,650 in a path-- 67 00:03:37,650 --> 00:03:45,810 so previously, distance equaled number of edges-- 68 00:03:45,810 --> 00:03:47,490 we're going to generalize that notion. 69 00:03:47,490 --> 00:03:50,190 So instead counting an edge, we're 70 00:03:50,190 --> 00:03:54,240 going to count an integer associated with that edge. 71 00:03:54,240 --> 00:03:56,010 It's going to be called a weight. 72 00:03:56,010 --> 00:04:01,140 So here's an example of a weighted graph G. 73 00:04:01,140 --> 00:04:03,860 And I've labeled, in red, weights 74 00:04:03,860 --> 00:04:04,860 for each of these edges. 75 00:04:04,860 --> 00:04:07,850 This is a directed graph on eight vertices. 76 00:04:07,850 --> 00:04:10,580 And I've got an integer associated with each edge. 77 00:04:10,580 --> 00:04:12,830 You'll notice, some of them are positive, some of them 78 00:04:12,830 --> 00:04:13,372 are negative. 79 00:04:13,372 --> 00:04:15,140 It's OK to be zero as well. 80 00:04:15,140 --> 00:04:18,649 It's just any integer edge weight here. 81 00:04:18,649 --> 00:04:21,339 So generally we're going to be-- along with our graph 82 00:04:21,339 --> 00:04:29,770 G, we're going to be given a weight function that maps 83 00:04:29,770 --> 00:04:38,030 the edges of G to, we're going to say, integers, in this class 84 00:04:38,030 --> 00:04:39,860 anyway. 85 00:04:39,860 --> 00:04:41,720 In other contexts, in mathematics, 86 00:04:41,720 --> 00:04:43,380 you might have these be real numbers. 87 00:04:43,380 --> 00:04:45,600 But in this class, we're going to deal with integers. 88 00:04:45,600 --> 00:04:48,890 So each edge, if you have an edge, 89 00:04:48,890 --> 00:04:53,930 we're going to say this is the edge weight-- 90 00:04:53,930 --> 00:04:58,100 the weight of this edge e, from e. 91 00:04:58,100 --> 00:05:04,910 Sometimes, if this edge e is u, v, we 92 00:05:04,910 --> 00:05:08,840 might sometimes say the weight from u 93 00:05:08,840 --> 00:05:14,060 to v, since we have a simple graph that's unambiguous. 94 00:05:14,060 --> 00:05:16,930 All right, so but this is just talking about our notation. 95 00:05:16,930 --> 00:05:21,950 So in general, for example, the weight from vertex b 96 00:05:21,950 --> 00:05:24,440 to f in this graph is what? 97 00:05:24,440 --> 00:05:26,070 Can someone tell me? 98 00:05:26,070 --> 00:05:26,950 AUDIENCE: Minus 4. 99 00:05:26,950 --> 00:05:28,030 JASON KU: Minus 4, right? 100 00:05:28,030 --> 00:05:28,697 It's right here. 101 00:05:28,697 --> 00:05:32,920 And I'll be consistent with my coloring, 102 00:05:32,920 --> 00:05:36,340 because I've got colored chalk today. 103 00:05:36,340 --> 00:05:37,480 Minus 4. 104 00:05:37,480 --> 00:05:38,680 Happiness. 105 00:05:38,680 --> 00:05:41,890 All right, so why do we care about adding weights 106 00:05:41,890 --> 00:05:43,330 to our graph? 107 00:05:43,330 --> 00:05:46,900 Well, this comes up a lot in many applications. 108 00:05:46,900 --> 00:05:50,110 For example, distances in a road network. 109 00:05:50,110 --> 00:05:54,160 if I have a road from here-- 110 00:05:54,160 --> 00:06:00,910 so from Mass Ave, front of MIT, to Central Square, 111 00:06:00,910 --> 00:06:03,340 we might think of that as one road. 112 00:06:03,340 --> 00:06:06,700 Maybe you've got each road is a connection 113 00:06:06,700 --> 00:06:10,150 between two intersections in my road network. 114 00:06:10,150 --> 00:06:14,390 But an edge, it takes longer to go from, 115 00:06:14,390 --> 00:06:17,572 say, Vassar Street to Amherst. 116 00:06:17,572 --> 00:06:19,030 That takes a shorter amount of time 117 00:06:19,030 --> 00:06:22,810 than it does to go from Memorial Drive across the river 118 00:06:22,810 --> 00:06:24,640 to Beacon Street. 119 00:06:24,640 --> 00:06:28,570 So we might want to associate a larger distance or a weight 120 00:06:28,570 --> 00:06:31,660 associated with that edge. 121 00:06:31,660 --> 00:06:35,590 Latency in a network, for example. 122 00:06:35,590 --> 00:06:39,070 Maybe strength of relationships in a social network. 123 00:06:39,070 --> 00:06:42,610 And you could imagine that it's possible maybe you're 124 00:06:42,610 --> 00:06:46,130 "frenemies" with someone, you don't like them, 125 00:06:46,130 --> 00:06:48,370 and so maybe you have a negative weight associated 126 00:06:48,370 --> 00:06:51,050 with an edge in a social network. 127 00:06:51,050 --> 00:06:52,070 I'm not sure. 128 00:06:52,070 --> 00:06:52,720 Maybe not. 129 00:06:52,720 --> 00:06:55,210 But there are lots of applications where you might 130 00:06:55,210 --> 00:06:59,150 want weights on your edges. 131 00:06:59,150 --> 00:07:02,450 So that comes to the next question of, 132 00:07:02,450 --> 00:07:04,130 how do I represent-- 133 00:07:04,130 --> 00:07:08,120 how do I give the user, or the algorithm, 134 00:07:08,120 --> 00:07:10,610 these weights in my graph? 135 00:07:10,610 --> 00:07:15,350 We had a representation for a graph. 136 00:07:15,350 --> 00:07:17,540 Our common way to represent a graph 137 00:07:17,540 --> 00:07:21,800 was store a set data structure on the vertices mapping 138 00:07:21,800 --> 00:07:25,490 to the adjacencies of each vertex, which 139 00:07:25,490 --> 00:07:28,250 we stored in what we called an adjacency list, which really 140 00:07:28,250 --> 00:07:32,210 could be any data structure. 141 00:07:32,210 --> 00:07:35,870 Commonly, it's just an array of the adjacencies. 142 00:07:35,870 --> 00:07:40,880 But you could also have that be a set data structure, 143 00:07:40,880 --> 00:07:44,520 where you can query in constant time what-- 144 00:07:44,520 --> 00:07:48,000 if a particular adjacency exists in that graph. 145 00:07:48,000 --> 00:07:53,990 So there are two common ways to store these weights. 146 00:07:53,990 --> 00:07:57,630 One is just, with every adjacency, 147 00:07:57,630 --> 00:07:58,880 I'm going to store its weight. 148 00:07:58,880 --> 00:08:02,030 Maybe just in a tuple. 149 00:08:02,030 --> 00:08:16,190 With each adjacency, also store weight of the edge 150 00:08:16,190 --> 00:08:20,750 that it corresponds to, just in any way. 151 00:08:20,750 --> 00:08:25,580 A second way, instead of trying to modify our graph structure 152 00:08:25,580 --> 00:08:28,040 that we gave you before, let's just 153 00:08:28,040 --> 00:08:31,280 have a dictionary of all the edges mapping to their weights. 154 00:08:31,280 --> 00:08:33,059 And we already know how to do that. 155 00:08:33,059 --> 00:08:43,370 Just any set data structure-- any separate set data structure 156 00:08:43,370 --> 00:08:51,380 mapping edges to their, I guess, weights. 157 00:08:51,380 --> 00:08:53,120 Bad notation, but you get the idea. 158 00:08:56,150 --> 00:08:58,880 And it doesn't really matter how we're doing this. 159 00:08:58,880 --> 00:09:01,100 The assumption that we're going to rely on here 160 00:09:01,100 --> 00:09:07,130 is that, given an edge, given this vertex pair, 161 00:09:07,130 --> 00:09:11,900 I can query what the weight of that edge is in constant time. 162 00:09:11,900 --> 00:09:13,280 And so if I'm going to do that, I 163 00:09:13,280 --> 00:09:17,210 can either store it with maybe a hash table of hash tables-- 164 00:09:17,210 --> 00:09:22,650 a hash table mapping the set of vertices to their adjacencies, 165 00:09:22,650 --> 00:09:26,900 and then each adjacency list stores its adjacencies 166 00:09:26,900 --> 00:09:27,740 in a hash table. 167 00:09:27,740 --> 00:09:29,810 And that way, in constant time, I 168 00:09:29,810 --> 00:09:31,610 can check what the weight is there. 169 00:09:31,610 --> 00:09:33,830 Or here, I'm just-- 170 00:09:33,830 --> 00:09:35,330 I could even have just a single hash 171 00:09:35,330 --> 00:09:42,960 table mapping the pair, the edge, the tuple, constant size, 172 00:09:42,960 --> 00:09:44,320 to its weight. 173 00:09:44,320 --> 00:09:46,870 So either way is fine. 174 00:09:46,870 --> 00:09:49,050 We're just going to assume that we can query 175 00:09:49,050 --> 00:09:51,030 an edge in constant time-- 176 00:09:51,030 --> 00:09:52,670 the weight of an edge in constant time. 177 00:09:52,670 --> 00:09:58,140 OK, so this is that graph example. 178 00:09:58,140 --> 00:09:59,400 It's a little busy here. 179 00:09:59,400 --> 00:10:03,040 I'm probably going to erase that in just a second. 180 00:10:03,040 --> 00:10:07,680 But we're going to move on to what giving these edges weights 181 00:10:07,680 --> 00:10:13,710 implies for these problems that we've defined 182 00:10:13,710 --> 00:10:15,780 in terms of unweighted graphs. 183 00:10:15,780 --> 00:10:18,390 In particular, we are going to be concentrating 184 00:10:18,390 --> 00:10:22,143 on single-source shortest paths, again, at least 185 00:10:22,143 --> 00:10:23,310 for the next three lectures. 186 00:10:23,310 --> 00:10:28,530 We'll generalize that even still in the next lecture-- 187 00:10:28,530 --> 00:10:30,450 I mean, in the fourth-- 188 00:10:30,450 --> 00:10:33,710 in three lectures from now. 189 00:10:33,710 --> 00:10:36,710 But what we had here was that the distance before 190 00:10:36,710 --> 00:10:39,920 in an unweighted graph was the number of edges in the path. 191 00:10:39,920 --> 00:10:46,400 Here, we're going to generalize that notion kind of obviously 192 00:10:46,400 --> 00:10:47,985 to weighted paths. 193 00:10:52,190 --> 00:10:59,330 And the weight of a path, I'm going call it pi. 194 00:10:59,330 --> 00:11:08,090 So some weight of path pi is just 195 00:11:08,090 --> 00:11:10,190 going to be the sum of the weights 196 00:11:10,190 --> 00:11:12,750 in the edges in the path. 197 00:11:12,750 --> 00:11:20,510 So edge in the path, I'm going to sum their weights. 198 00:11:20,510 --> 00:11:22,970 So that's all the weight of a path means. 199 00:11:22,970 --> 00:11:25,470 It's just I'm going to sum all the weights in a path. 200 00:11:25,470 --> 00:11:27,020 So if I took a look at the-- 201 00:11:27,020 --> 00:11:29,600 maybe there's a particular path here. 202 00:11:29,600 --> 00:11:35,930 The path from a to b to f to g is going to be minus 5, 203 00:11:35,930 --> 00:11:38,180 minus 4, 2. 204 00:11:38,180 --> 00:11:43,250 It's going to be minus 9 plus 2 is minus 7. 205 00:11:43,250 --> 00:11:44,360 So just as an example. 206 00:11:48,770 --> 00:11:52,460 So then what is the shortest path then? 207 00:11:52,460 --> 00:11:55,790 Well, kind of obviously among all paths between two 208 00:11:55,790 --> 00:12:00,170 vertices, it's going to be one with the minimum weight. 209 00:12:00,170 --> 00:12:00,920 Yeah, question. 210 00:12:00,920 --> 00:12:03,110 AUDIENCE: Can I use the same edge more than once? 211 00:12:03,110 --> 00:12:06,710 JASON KU: Can I use the same edge more than once? 212 00:12:06,710 --> 00:12:12,980 Right now, you're asking about the distinction in our class 213 00:12:12,980 --> 00:12:15,980 which we have between paths and simple paths. 214 00:12:15,980 --> 00:12:18,560 So here, a weighted path doesn't really 215 00:12:18,560 --> 00:12:21,560 care if we visit an edge more than once. 216 00:12:21,560 --> 00:12:24,920 So if an edge appears more than once in pi, 217 00:12:24,920 --> 00:12:28,730 we have to count that more than once in the edge weight-- 218 00:12:28,730 --> 00:12:30,110 in the weight of the path. 219 00:12:30,110 --> 00:12:32,630 OK, great question. 220 00:12:32,630 --> 00:12:39,120 But what we're going to see later on is shortest paths 221 00:12:39,120 --> 00:12:47,320 cannot repeat an edge more than once in certain contexts. 222 00:12:47,320 --> 00:12:50,418 So we're going to get to the problem there 223 00:12:50,418 --> 00:12:51,710 a little later in this lecture. 224 00:12:51,710 --> 00:12:54,410 And we're going to solve that in tomorrow's lecture. 225 00:12:54,410 --> 00:12:56,820 But if you have-- 226 00:12:56,820 --> 00:12:58,570 we're getting a little ahead of ourselves. 227 00:12:58,570 --> 00:13:01,330 But when we have negative weights in a graph, 228 00:13:01,330 --> 00:13:02,920 it's possible that things go wrong. 229 00:13:02,920 --> 00:13:05,020 We're going to get there in about five lines. 230 00:13:05,020 --> 00:13:06,850 OK, great. 231 00:13:06,850 --> 00:13:11,095 So a shortest path-- 232 00:13:17,190 --> 00:13:19,300 and in this case, I'm going to clarify that this 233 00:13:19,300 --> 00:13:20,680 is the weighted shortest path-- 234 00:13:23,850 --> 00:13:29,150 is a minimum-- min-i-mum-- 235 00:13:32,800 --> 00:13:47,140 sure-- is a minimum weight path from s to t. 236 00:13:47,140 --> 00:13:50,260 Nothing too interesting here, but there's actually 237 00:13:50,260 --> 00:13:52,090 some subtleties we have to deal with here. 238 00:13:54,790 --> 00:13:55,900 We're going to call-- 239 00:13:55,900 --> 00:13:58,930 just like we did with breadth-first search 240 00:13:58,930 --> 00:14:00,430 when we talked about shortest paths, 241 00:14:00,430 --> 00:14:05,620 we're going to define an expression for what 242 00:14:05,620 --> 00:14:10,690 the distance or the shortest path weight 243 00:14:10,690 --> 00:14:12,220 is between two vertices. 244 00:14:12,220 --> 00:14:14,830 And I'm going to represent that by a delta. 245 00:14:14,830 --> 00:14:23,230 A delta from a vertex s to t is going to be-- 246 00:14:23,230 --> 00:14:26,350 let's-- I'm going to do the wrong thing first-- 247 00:14:26,350 --> 00:14:36,960 the minimum over the weight of all paths for all paths pi 248 00:14:36,960 --> 00:14:43,030 from s to t. 249 00:14:43,030 --> 00:14:46,930 OK, so there's a couple things that go wrong here. 250 00:14:46,930 --> 00:14:50,410 First thing that goes wrong is the same thing that went wrong 251 00:14:50,410 --> 00:14:53,290 with breadth-first search. 252 00:14:53,290 --> 00:14:56,080 Anyone remember what could go wrong with breadth-first search 253 00:14:56,080 --> 00:14:57,357 for this delta definition? 254 00:14:57,357 --> 00:14:58,690 AUDIENCE: Maybe there's no path. 255 00:14:58,690 --> 00:15:01,450 JASON KU: Maybe there's no path, right. 256 00:15:01,450 --> 00:15:05,485 So except if no path. 257 00:15:07,990 --> 00:15:11,620 Just by convention, we're setting 258 00:15:11,620 --> 00:15:15,580 delta s, t, to equal infinity. 259 00:15:15,580 --> 00:15:20,480 But there's one additional problem 260 00:15:20,480 --> 00:15:26,320 with weighted shortest paths, and it's a little subtle. 261 00:15:26,320 --> 00:15:29,380 It's possible that a finite shortest-- 262 00:15:29,380 --> 00:15:33,460 finite length shortest path doesn't exist. 263 00:15:33,460 --> 00:15:34,850 And what do I mean by that? 264 00:15:34,850 --> 00:15:41,630 It means I could keep going through edges in my graph 265 00:15:41,630 --> 00:15:46,660 and continually getting a shorter path. 266 00:15:46,660 --> 00:15:51,670 So if the shortest-- the minimum weight of a path from s 267 00:15:51,670 --> 00:15:54,910 to t actually goes through an infinite number of edges, 268 00:15:54,910 --> 00:15:57,550 then this isn't really well-defined. 269 00:15:57,550 --> 00:16:02,410 So I'm going to change this minimum here to-- 270 00:16:02,410 --> 00:16:07,870 in mathematics we would, just to be specific, 271 00:16:07,870 --> 00:16:10,670 we call it an infimum. 272 00:16:10,670 --> 00:16:16,430 So if in the case where the weight of a shortest path 273 00:16:16,430 --> 00:16:20,570 can approach arbitrarily small, then we'll 274 00:16:20,570 --> 00:16:24,140 call this thing minus infinity. 275 00:16:24,140 --> 00:16:26,365 So when does that occur? 276 00:16:31,290 --> 00:16:33,320 When does that occur? 277 00:16:33,320 --> 00:16:36,690 When could we have our shortest path 278 00:16:36,690 --> 00:16:38,430 go through lots and lots of vertices? 279 00:16:38,430 --> 00:16:41,510 Well, let's actually take a look at this example here. 280 00:16:41,510 --> 00:16:43,490 Can someone tell me what the shortest path 281 00:16:43,490 --> 00:16:50,470 is from a to actually any vertex in this graph? 282 00:16:55,350 --> 00:16:57,310 AUDIENCE: b, f, g, c. 283 00:16:57,310 --> 00:16:58,790 JASON KU: Ah, OK. 284 00:16:58,790 --> 00:17:03,680 So well, we could look at this path I have to b. 285 00:17:03,680 --> 00:17:05,450 Let's just take a look at b. 286 00:17:05,450 --> 00:17:08,422 I have a path going from a to b that is minus 5. 287 00:17:08,422 --> 00:17:09,380 OK, that's pretty good. 288 00:17:09,380 --> 00:17:10,940 That's pretty small. 289 00:17:10,940 --> 00:17:17,329 And it seems that if I go around this graph through another way, 290 00:17:17,329 --> 00:17:18,630 it might be bigger. 291 00:17:18,630 --> 00:17:24,020 So I go 7 plus 3 plus 8-- that's 15-- 292 00:17:24,020 --> 00:17:26,300 minus 1-- that's 14. 293 00:17:26,300 --> 00:17:30,590 That's much bigger than minus 5, so it seems like minus 5 294 00:17:30,590 --> 00:17:32,030 should be good, right? 295 00:17:32,030 --> 00:17:35,150 Anyone have a problem with this path or a problem 296 00:17:35,150 --> 00:17:37,680 with this being the shortest path? 297 00:17:37,680 --> 00:17:39,980 And what your colleague just informed 298 00:17:39,980 --> 00:17:43,790 me was that there is something interesting happening here 299 00:17:43,790 --> 00:17:46,350 in this graph in particular. 300 00:17:46,350 --> 00:17:52,620 We have a cycle from b to f to g to c 301 00:17:52,620 --> 00:17:57,060 that has negative total weight back to b. 302 00:17:57,060 --> 00:18:03,070 This has minus 4 plus 2 plus 1 minus 1. 303 00:18:03,070 --> 00:18:09,420 So that total cycle has a cycle weight of minus 2, 304 00:18:09,420 --> 00:18:11,710 this negative weight cycle. 305 00:18:11,710 --> 00:18:16,740 So if I want to get to b, I could go there 306 00:18:16,740 --> 00:18:19,810 via this minus 5 weight edge. 307 00:18:19,810 --> 00:18:27,190 But every time I circled around this cycle, I incur minus 2 308 00:18:27,190 --> 00:18:29,410 to my path weight. 309 00:18:29,410 --> 00:18:32,740 So I just keep going around this cycle over and over 310 00:18:32,740 --> 00:18:35,540 and over and over and over and over again, 311 00:18:35,540 --> 00:18:41,290 and I don't have any finite length minimum weight path. 312 00:18:41,290 --> 00:18:46,720 And in such cases, we just say that delta is minus infinity. 313 00:18:46,720 --> 00:18:51,575 So the problem here is that we could have negative weight 314 00:18:51,575 --> 00:18:52,075 cycles-- 315 00:18:55,360 --> 00:18:57,340 deserves a capital letter-- 316 00:18:57,340 --> 00:19:03,115 Negative weight cycles. 317 00:19:07,390 --> 00:19:08,270 It's a problem. 318 00:19:11,690 --> 00:19:25,230 In particular, if there exists a path from s 319 00:19:25,230 --> 00:19:43,630 to some vertex v that goes through a vertex 320 00:19:43,630 --> 00:19:56,840 on a negative weight cycle, then I 321 00:19:56,840 --> 00:20:00,020 can take that path to that vertex, circle 322 00:20:00,020 --> 00:20:04,280 around the negative weight cycle, and then proceed to v, 323 00:20:04,280 --> 00:20:06,560 and I can take that cycle as many times as I want. 324 00:20:06,560 --> 00:20:14,360 Then this delta s,v we're going to set to minus infinity. 325 00:20:14,360 --> 00:20:18,260 And in such cases, in our shortest paths algorithm, 326 00:20:18,260 --> 00:20:23,040 we don't really care about what the shortest path is. 327 00:20:23,040 --> 00:20:25,640 We're not even going to deal with parent pointers 328 00:20:25,640 --> 00:20:30,675 here, because there is no finite length shortest path. 329 00:20:30,675 --> 00:20:33,050 So I'm just going to kind of throw up my hands in the air 330 00:20:33,050 --> 00:20:35,900 and say, you know what, I can't return you a shortest path, 331 00:20:35,900 --> 00:20:41,230 but I might want to return to you a negative weight cycle. 332 00:20:41,230 --> 00:20:45,370 If you told me that this thing has bad weight, 333 00:20:45,370 --> 00:20:47,650 maybe I want you to tell me what a path is 334 00:20:47,650 --> 00:20:52,090 that goes through a negative weight cycle to get back to s. 335 00:20:52,090 --> 00:20:54,340 So that's what we're going to talk about next lecture. 336 00:20:54,340 --> 00:20:57,375 This lecture, we are not going to talk about that. 337 00:20:57,375 --> 00:20:59,500 We are going to talk about weighted shortest paths, 338 00:20:59,500 --> 00:21:00,130 though. 339 00:21:00,130 --> 00:21:06,520 That's what the remainder of this unit on graphs 340 00:21:06,520 --> 00:21:08,980 is really about is weighted shortest paths. 341 00:21:08,980 --> 00:21:11,890 OK, so in weighted shortest paths, 342 00:21:11,890 --> 00:21:14,770 we actually know an algorithm already 343 00:21:14,770 --> 00:21:21,640 to solve a subset of weighted shortest parts, namely BFS, 344 00:21:21,640 --> 00:21:23,220 right? 345 00:21:23,220 --> 00:21:27,390 Now, you're like wait, Jason, BFS 346 00:21:27,390 --> 00:21:29,010 doesn't solve weighted shortest paths. 347 00:21:29,010 --> 00:21:31,050 We didn't even know about weighted graphs then. 348 00:21:31,050 --> 00:21:33,840 How does that solve weighted shortest paths? 349 00:21:33,840 --> 00:21:37,860 Well, there's a couple cases where 350 00:21:37,860 --> 00:21:41,850 we might be able to reduce to solving shortest paths using 351 00:21:41,850 --> 00:21:42,630 BFS. 352 00:21:42,630 --> 00:21:44,220 Can anyone think of such a scenario? 353 00:21:50,110 --> 00:21:55,150 So let's say, I mean, kind of what we did before was we 354 00:21:55,150 --> 00:21:57,910 counted the number of edges. 355 00:21:57,910 --> 00:22:03,680 So if we gave a weight of 1 to every edge in my graph, 356 00:22:03,680 --> 00:22:08,000 then just that graph, that weighted graph, 357 00:22:08,000 --> 00:22:12,320 corresponds to an unweighted graph using the other distance 358 00:22:12,320 --> 00:22:14,120 metric. 359 00:22:14,120 --> 00:22:17,560 So in that case, BFS just solves our problem. 360 00:22:17,560 --> 00:22:21,590 And in fact, we can generalize further. 361 00:22:21,590 --> 00:22:27,340 What if all of our weights were positive, but the same value? 362 00:22:27,340 --> 00:22:30,730 If it was all positive and the same value, 363 00:22:30,730 --> 00:22:35,680 then we could just divide by that value. 364 00:22:35,680 --> 00:22:40,870 Now we have an unweighted graph which we can run BFS, and then 365 00:22:40,870 --> 00:22:44,410 multiply shortest path distances by that value later on. 366 00:22:44,410 --> 00:22:48,430 And in fact, there's one further generalization we can make, 367 00:22:48,430 --> 00:22:51,310 which is a little bit of a tricky graph transformation 368 00:22:51,310 --> 00:22:53,460 problem. 369 00:22:53,460 --> 00:22:56,940 But we can also get this linear time algorithm 370 00:22:56,940 --> 00:23:00,480 for weighted single-source shortest paths 371 00:23:00,480 --> 00:23:06,570 in contexts where the weights aren't that large. 372 00:23:06,570 --> 00:23:11,100 So if I have positive edge weights-- 373 00:23:11,100 --> 00:23:13,380 if I have a positive edge weight, let's say-- 374 00:23:16,150 --> 00:23:20,170 using my weight color here-- 375 00:23:20,170 --> 00:23:25,720 that's, like, weight of 4, that's 376 00:23:25,720 --> 00:23:28,900 kind of problematic, because I don't know how to simulate 377 00:23:28,900 --> 00:23:30,820 that using an unweighted graph. 378 00:23:30,820 --> 00:23:32,350 Or do I? 379 00:23:32,350 --> 00:23:34,090 Anyone have an idea of how I could 380 00:23:34,090 --> 00:23:41,140 simulate an edge of weight 4 with an unweighted graph? 381 00:23:41,140 --> 00:23:41,640 Yeah. 382 00:23:41,640 --> 00:23:43,800 AUDIENCE: Have four edges of weight 1. 383 00:23:43,800 --> 00:23:45,780 JASON KU: Yeah, I can just put four edges 384 00:23:45,780 --> 00:23:48,960 of weight 1 in parallel here-- 385 00:23:48,960 --> 00:23:53,340 I'm sorry, in series, the opposite of parallel. 386 00:23:53,340 --> 00:24:02,880 I can just convert this here into 1, 2, 3, 4 edges. 387 00:24:02,880 --> 00:24:06,130 And if I do that for every edge in my graph 388 00:24:06,130 --> 00:24:08,340 and we have positive edge weights, 389 00:24:08,340 --> 00:24:10,440 then that transformation can hold. 390 00:24:10,440 --> 00:24:13,050 Now, that's not necessarily a good transformation to make. 391 00:24:13,050 --> 00:24:14,886 Why? 392 00:24:14,886 --> 00:24:16,740 AUDIENCE: The weight might be very big. 393 00:24:16,740 --> 00:24:19,170 JASON KU: Yeah, the weights might be very big 394 00:24:19,170 --> 00:24:22,860 compared to the number of vertices and edges in my graph. 395 00:24:22,860 --> 00:24:26,760 However, if the sum of all weights in my graph 396 00:24:26,760 --> 00:24:32,400 is asymptotically less than v plus e, 397 00:24:32,400 --> 00:24:34,530 we can get a linear time algorithm again 398 00:24:34,530 --> 00:24:35,670 by reducing to BFS. 399 00:24:35,670 --> 00:24:36,630 OK, so that's great. 400 00:24:40,650 --> 00:24:43,300 But in general, that gives us a linear time algorithm 401 00:24:43,300 --> 00:24:46,530 in these very special cases. 402 00:24:46,530 --> 00:24:48,390 And in general, it's an open problem. 403 00:24:48,390 --> 00:24:52,170 We don't know whether we can solve 404 00:24:52,170 --> 00:24:56,670 the single-source shortest paths problem 405 00:24:56,670 --> 00:25:00,960 in the weighted context for general graphs in linear time. 406 00:25:00,960 --> 00:25:03,960 We don't know how to do it. 407 00:25:03,960 --> 00:25:08,560 But what we do know are some algorithms that do pretty well. 408 00:25:08,560 --> 00:25:11,460 And that's what we use all the time. 409 00:25:11,460 --> 00:25:15,240 But one more special case we're going to go over today 410 00:25:15,240 --> 00:25:18,990 is when we have this really nice structure where 411 00:25:18,990 --> 00:25:22,350 we have a DAG, a Directed Acyclic 412 00:25:22,350 --> 00:25:28,030 Graph, like we were talking about in the last lecture. 413 00:25:28,030 --> 00:25:29,920 For any set of edge weights-- 414 00:25:29,920 --> 00:25:33,580 remember, with BFS, we needed to restrict our edge weights 415 00:25:33,580 --> 00:25:39,710 to be positive and maybe bounded to get this good running time? 416 00:25:39,710 --> 00:25:43,193 For any set of edge weights, if our graph structure is DAG-- 417 00:25:43,193 --> 00:25:45,110 it really has nothing to do with the weights-- 418 00:25:45,110 --> 00:25:47,900 if the graph structure is a DAG, then 419 00:25:47,900 --> 00:25:51,320 we can actually solve this single-source shortest paths 420 00:25:51,320 --> 00:25:56,630 problem in linear time, which is pretty awesome. 421 00:25:56,630 --> 00:26:02,720 Now, for general graphs, we're going to show you in the next 422 00:26:02,720 --> 00:26:06,980 lecture how to, for any graph-- even with cycles, 423 00:26:06,980 --> 00:26:08,780 even with negative weight cycles-- 424 00:26:08,780 --> 00:26:10,280 we're going to show you how to solve 425 00:26:10,280 --> 00:26:11,690 this single-source shortest paths 426 00:26:11,690 --> 00:26:14,910 problem in something like a quadratic running time bound. 427 00:26:14,910 --> 00:26:17,090 Now, this isn't the best known, but it's 428 00:26:17,090 --> 00:26:21,390 a really practical algorithm and people use it all the time. 429 00:26:21,390 --> 00:26:25,610 And we are going to show Bellman-Ford 430 00:26:25,610 --> 00:26:28,940 in the context of the DAG algorithm 431 00:26:28,940 --> 00:26:32,900 we're going to solve today. 432 00:26:32,900 --> 00:26:35,030 So that's the very general case in terms 433 00:26:35,030 --> 00:26:37,370 of restrictions on our graph. 434 00:26:37,370 --> 00:26:43,070 But in reality, most problems that come up in applications 435 00:26:43,070 --> 00:26:46,940 occur with graphs that have positive edge weights. 436 00:26:46,940 --> 00:26:48,380 You can think of a road network. 437 00:26:48,380 --> 00:26:51,110 You've got-- or non-negative ones anyway. 438 00:26:51,110 --> 00:26:56,000 You're traveling along, and it's not ever 439 00:26:56,000 --> 00:26:58,940 useful to go back to where you came from, 440 00:26:58,940 --> 00:27:03,470 because you want to make progress to where you're going. 441 00:27:03,470 --> 00:27:06,873 So in the context where you don't have negative weights, 442 00:27:06,873 --> 00:27:09,290 you don't have this problem where you have negative weight 443 00:27:09,290 --> 00:27:10,580 cycles. 444 00:27:10,580 --> 00:27:15,320 We can actually do a lot better by exploiting that property. 445 00:27:15,320 --> 00:27:17,900 And we get a bound that's a little bit-- that looks 446 00:27:17,900 --> 00:27:21,050 a little bit more like n log n. 447 00:27:21,050 --> 00:27:23,000 It's pretty close to linear. 448 00:27:23,000 --> 00:27:27,740 You're losing a log factor on the number of vertices. 449 00:27:27,740 --> 00:27:28,873 But it's pretty good. 450 00:27:28,873 --> 00:27:30,290 This is called Dijkstra, and we'll 451 00:27:30,290 --> 00:27:32,270 get to that in two lectures. 452 00:27:32,270 --> 00:27:35,360 OK, so that's the roadmap of what 453 00:27:35,360 --> 00:27:39,200 we're going to do for at least the next three lectures. 454 00:27:39,200 --> 00:27:41,930 But before we go on to showing you 455 00:27:41,930 --> 00:27:46,640 how to solve single-source shortest paths in a DAG using 456 00:27:46,640 --> 00:27:52,800 this algorithm that I'm calling DAG relaxation here, 457 00:27:52,800 --> 00:27:55,710 I'm going to go back to a thing that we talked 458 00:27:55,710 --> 00:27:57,908 about in breadth-first search, where 459 00:27:57,908 --> 00:28:00,450 in breadth-first search when we solved single-source shortest 460 00:28:00,450 --> 00:28:04,050 paths, we output two things. 461 00:28:04,050 --> 00:28:08,190 We output single-source shortest paths, these deltas, 462 00:28:08,190 --> 00:28:13,140 for the other definition of distance, the weights-- 463 00:28:13,140 --> 00:28:16,500 I mean, not the weights, the distances, the shortest 464 00:28:16,500 --> 00:28:18,600 distances. 465 00:28:18,600 --> 00:28:20,790 But we also returned parent pointers. 466 00:28:20,790 --> 00:28:24,990 We return parent pointers back along paths 467 00:28:24,990 --> 00:28:28,080 to the source along shortest paths. 468 00:28:28,080 --> 00:28:29,970 We call this the shortest paths tree. 469 00:28:29,970 --> 00:28:33,450 So I'm going to revisit this topic of shortest paths tree-- 470 00:28:33,450 --> 00:28:42,750 shortest path trees-- shortest path trees. 471 00:28:45,390 --> 00:28:47,550 And in particular, it's kind of going 472 00:28:47,550 --> 00:28:51,930 to be annoying to talk about both of these quantities-- 473 00:28:51,930 --> 00:28:56,160 distances and parent pointers-- as we go through all three 474 00:28:56,160 --> 00:28:57,450 of these algorithms. 475 00:28:57,450 --> 00:29:01,080 It's basically going to be bookkeeping to-- 476 00:29:01,080 --> 00:29:04,110 distances are actually sufficient for us 477 00:29:04,110 --> 00:29:07,560 to reconstruct parent pointers if we need them later. 478 00:29:07,560 --> 00:29:10,110 So what I'm going to show for you-- prove to you now 479 00:29:10,110 --> 00:29:12,450 is that, if I give you the shortest path 480 00:29:12,450 --> 00:29:19,490 distances for the subset of the graph reachable from s 481 00:29:19,490 --> 00:29:23,600 that doesn't go through negative weight cycles, 482 00:29:23,600 --> 00:29:25,340 if I'm giving you those distances, 483 00:29:25,340 --> 00:29:29,330 I can reconstruct parent pointers along shortest paths 484 00:29:29,330 --> 00:29:35,130 in linear time for any graph I might give you 485 00:29:35,130 --> 00:29:37,290 if I give you those shortest path distances. 486 00:29:37,290 --> 00:29:40,200 OK, so that's what I'm going to try to show to you now. 487 00:29:43,560 --> 00:29:49,290 So here's the algorithm. 488 00:29:49,290 --> 00:29:51,680 For weighted-- there's the caveat here 489 00:29:51,680 --> 00:29:52,680 I'm going to write down. 490 00:29:52,680 --> 00:30:03,000 For weighted shortest paths, only need parent pointers 491 00:30:03,000 --> 00:30:14,710 for v with finite shortest path distance-- 492 00:30:17,440 --> 00:30:19,250 only finite shortest path distance. 493 00:30:19,250 --> 00:30:21,550 We don't care about the infinite ones 494 00:30:21,550 --> 00:30:23,830 or the minus infinite ones, just the finite ones. 495 00:30:23,830 --> 00:30:27,100 OK, so here's the algorithm. 496 00:30:27,100 --> 00:30:36,760 I can initialize all Pv to equal-- 497 00:30:36,760 --> 00:30:42,730 sorry, oh, getting ahead of myself. 498 00:30:42,730 --> 00:30:43,676 I'm writing down DAG. 499 00:30:48,910 --> 00:30:54,150 Init parent pointer data structure to be empty. 500 00:30:57,920 --> 00:31:00,620 At first, I'm not going to sort any parent pointers. 501 00:31:00,620 --> 00:31:04,040 But at the beginning, I'm going to set the parent pointer 502 00:31:04,040 --> 00:31:07,820 of the source to be none. 503 00:31:07,820 --> 00:31:13,390 So that's what we kind of did in breadth-first search as well. 504 00:31:13,390 --> 00:31:16,600 Now, what I've given you is-- 505 00:31:16,600 --> 00:31:18,970 I'm trying to show that, given all the shortest path 506 00:31:18,970 --> 00:31:22,100 distances, I can construct these parent pointers correctly. 507 00:31:22,100 --> 00:31:34,110 So what I'm going to do is, for each vertex u in my graph, 508 00:31:34,110 --> 00:31:47,640 where my delta s of u is finite, what am I going to do? 509 00:31:47,640 --> 00:31:50,930 I'm going to say, well, let's take a look 510 00:31:50,930 --> 00:31:54,380 at all my outgoing neighbors. 511 00:31:54,380 --> 00:31:57,920 This is kind of what we do in every graph algorithm. 512 00:31:57,920 --> 00:32:10,410 For each v in the adjacency, the outgoing adjacencies of u, 513 00:32:10,410 --> 00:32:18,000 if there is no parent pointer assigned to this v, 514 00:32:18,000 --> 00:32:20,274 there's the potential that i-- u-- 515 00:32:20,274 --> 00:32:24,870 [CHUCKLES] I, you-- this u, this vertex u, is the parent of v. 516 00:32:24,870 --> 00:32:25,740 It's possible. 517 00:32:25,740 --> 00:32:29,940 It's some incoming edge to v. 518 00:32:29,940 --> 00:32:34,440 When will it be an incoming edge to v? 519 00:32:34,440 --> 00:32:40,860 If v not in P-- 520 00:32:40,860 --> 00:32:44,020 I haven't assigned it a parent pointer-- 521 00:32:44,020 --> 00:32:51,070 and-- so this means it could be my parent. 522 00:32:51,070 --> 00:32:57,270 When is it my parent along the shortest path? 523 00:32:57,270 --> 00:32:57,975 Sure. 524 00:32:57,975 --> 00:33:00,008 AUDIENCE: Sum the distance along the edge 525 00:33:00,008 --> 00:33:01,470 to the distance of the other. 526 00:33:01,470 --> 00:33:04,635 JASON KU: Yeah, so we have some edge from u 527 00:33:04,635 --> 00:33:08,130 to v. It has some weight. 528 00:33:08,130 --> 00:33:11,400 If I already know the shortest path distance to u, 529 00:33:11,400 --> 00:33:15,210 and I know the shortest path distance to v, 530 00:33:15,210 --> 00:33:20,920 if the shortest path distance from s to u-- 531 00:33:20,920 --> 00:33:22,370 let's draw a picture here. 532 00:33:22,370 --> 00:33:25,900 We've got s, we've got some path here to u, 533 00:33:25,900 --> 00:33:28,720 and we know we've got an edge from u 534 00:33:28,720 --> 00:33:37,510 to v. If this shortest path distance plus this edge weight 535 00:33:37,510 --> 00:33:43,570 is equal to the shortest path distance from s to v, 536 00:33:43,570 --> 00:33:44,290 then it better-- 537 00:33:44,290 --> 00:33:46,790 I mean, there may be more than one shortest path, 538 00:33:46,790 --> 00:33:48,640 but this is certainly a shortest path, 539 00:33:48,640 --> 00:33:52,240 so we can assign a parent pointer back to u. 540 00:33:52,240 --> 00:33:54,100 So let's write that condition down. 541 00:33:54,100 --> 00:33:58,630 If the shortest path distance from s to v 542 00:33:58,630 --> 00:34:02,620 equals the shortest path distance from s to u, 543 00:34:02,620 --> 00:34:09,530 and then traversing the edge from u to v, 544 00:34:09,530 --> 00:34:26,239 then exists shortest path that uses edge u, v, 545 00:34:26,239 --> 00:34:28,610 in particular this one. 546 00:34:31,150 --> 00:34:37,043 So set the parent of u-- 547 00:34:37,043 --> 00:34:40,639 of v to u. 548 00:34:40,639 --> 00:34:42,900 OK, so this is the algorithm. 549 00:34:42,900 --> 00:34:46,340 I'm not going to prove to you that this is correct. 550 00:34:46,340 --> 00:34:48,590 But it kind of intuitively makes sense, right? 551 00:34:48,590 --> 00:34:52,219 If I have these shortest path distances, 552 00:34:52,219 --> 00:34:55,699 you can prove by induction that not only does this parent 553 00:34:55,699 --> 00:35:00,830 pointer point to the right place along some shortest path here, 554 00:35:00,830 --> 00:35:02,480 but it also does so in linear time, 555 00:35:02,480 --> 00:35:04,820 because I'm looping over all the vertices 556 00:35:04,820 --> 00:35:07,760 and looping over its outgoing adjacencies once. 557 00:35:07,760 --> 00:35:14,310 Same analysis as we had for both BFS and DFS, essentially. 558 00:35:14,310 --> 00:35:18,770 And then, since we can do this, since we can compute parent 559 00:35:18,770 --> 00:35:20,990 pointers from these distances, we're 560 00:35:20,990 --> 00:35:25,822 going to ignore computing these parent pointers from now on. 561 00:35:25,822 --> 00:35:28,280 We're just going to concentrate on computing the distances, 562 00:35:28,280 --> 00:35:32,600 because we're going to have to take linear time anyway 563 00:35:32,600 --> 00:35:34,040 at least. 564 00:35:34,040 --> 00:35:36,270 And all these other things take more time. 565 00:35:36,270 --> 00:35:39,080 So we can compute the distances in more time, 566 00:35:39,080 --> 00:35:41,480 and then compute the parents after. 567 00:35:41,480 --> 00:35:43,750 OK, so that's what we're going to do. 568 00:35:47,090 --> 00:35:50,795 So now, with all that buildup, let's show an algorithm. 569 00:35:50,795 --> 00:35:56,480 [CHUCKLES] How do we compute single-source shortest paths 570 00:35:56,480 --> 00:36:01,150 in a DAG in linear time? 571 00:36:01,150 --> 00:36:02,860 Well, a DAG-- I mean, this is actually 572 00:36:02,860 --> 00:36:07,390 a super useful, convenient thing in algorithms in general. 573 00:36:07,390 --> 00:36:10,210 DAGs are just nice things. 574 00:36:10,210 --> 00:36:12,310 They're kind of ordered in a way. 575 00:36:12,310 --> 00:36:14,103 There's this topological sort order 576 00:36:14,103 --> 00:36:15,520 that we were talking about before. 577 00:36:15,520 --> 00:36:18,330 This is going to play a key role. 578 00:36:18,330 --> 00:36:21,490 There's a really nice structure to DAGs not having cycles, 579 00:36:21,490 --> 00:36:25,000 not having to deal with this negative weight cycle problem. 580 00:36:25,000 --> 00:36:30,820 You can only go in one direction along this graph. 581 00:36:30,820 --> 00:36:32,900 It's a very nice structure to exploit. 582 00:36:32,900 --> 00:36:35,800 And so we're going to exploit it. 583 00:36:35,800 --> 00:36:37,920 And here's the idea. 584 00:36:37,920 --> 00:36:47,210 DAG relaxation, what it's going to do 585 00:36:47,210 --> 00:36:50,300 is it's going to start out with some estimates of what 586 00:36:50,300 --> 00:36:53,180 these distances should be. 587 00:36:53,180 --> 00:37:01,740 So maintain distance estimates. 588 00:37:06,020 --> 00:37:08,870 And now I'm going to try to be careful here 589 00:37:08,870 --> 00:37:11,030 about how I draw my Ds. 590 00:37:14,510 --> 00:37:20,290 This is a d, this is a delta. 591 00:37:20,290 --> 00:37:22,270 This is shortest paths. 592 00:37:22,270 --> 00:37:24,970 This is a distance estimate. 593 00:37:24,970 --> 00:37:26,470 So that's what I'm going to be using 594 00:37:26,470 --> 00:37:29,750 for the rest of this time. 595 00:37:29,750 --> 00:37:35,890 So we're going to maintain these estimates of distance d, which 596 00:37:35,890 --> 00:37:44,430 are going to start at initially infinite. 597 00:37:44,430 --> 00:37:46,020 I don't know what they are. 598 00:37:46,020 --> 00:37:47,810 I don't know what the shortest paths are, 599 00:37:47,810 --> 00:37:52,040 but they better be less than infinite or else I don't care. 600 00:37:52,040 --> 00:37:55,160 So that's the worst case scenario. 601 00:37:55,160 --> 00:37:56,990 It can't be worse than this-- 602 00:37:56,990 --> 00:37:59,130 for every vertex. 603 00:37:59,130 --> 00:38:03,650 And we're going to maintain the property that 604 00:38:03,650 --> 00:38:08,840 estimates upper bound-- 605 00:38:15,100 --> 00:38:16,990 that should probably be two words-- 606 00:38:16,990 --> 00:38:20,950 upper bound delta s, v-- 607 00:38:25,337 --> 00:38:27,670 we're going to maintain that they upper-bound this thing 608 00:38:27,670 --> 00:38:37,100 and gradually lower until they're equal. 609 00:38:39,820 --> 00:38:40,820 So this is the idea. 610 00:38:40,820 --> 00:38:44,770 We start from an over-estimate, an upper bound on the distance 611 00:38:44,770 --> 00:38:45,910 estimate. 612 00:38:45,910 --> 00:38:49,540 And then we're repeatedly going to lower that value 613 00:38:49,540 --> 00:38:53,070 as we gain more information about the graph, 614 00:38:53,070 --> 00:38:56,680 maintaining that we're always upper-bounding the distance. 615 00:38:56,680 --> 00:38:58,680 And we're going to keep doing it, keep doing it, 616 00:38:58,680 --> 00:39:02,880 keep doing it, until, as we will try to prove to you, 617 00:39:02,880 --> 00:39:06,240 these estimates reach, actually reach down, all the way 618 00:39:06,240 --> 00:39:09,360 to our shortest path distances. 619 00:39:09,360 --> 00:39:10,755 So when do we lower these things? 620 00:39:13,810 --> 00:39:16,600 When do we lower these things? 621 00:39:16,600 --> 00:39:19,210 We are going to lower these distance estimates whenever 622 00:39:19,210 --> 00:39:21,310 the distance estimates violate what 623 00:39:21,310 --> 00:39:24,100 we're going to call the triangle inequality. 624 00:39:24,100 --> 00:39:26,038 OK, what is the triangle inequality? 625 00:39:37,000 --> 00:39:39,565 Triangle inequality is actually a pretty intuitive notion. 626 00:39:42,130 --> 00:39:44,480 It's basically saying, if I have three points-- 627 00:39:44,480 --> 00:39:49,410 thus, triangle-- maybe bigger so I can write a letter in them. 628 00:39:52,450 --> 00:39:58,210 It's basically saying that if I have a vertex u, a vertex v, 629 00:39:58,210 --> 00:40:04,460 vertex x, for example, the shortest path distance-- 630 00:40:08,950 --> 00:40:12,310 the shortest path distance delta of u, v-- 631 00:40:12,310 --> 00:40:15,310 that's the shortest distance from u to v-- 632 00:40:15,310 --> 00:40:20,810 it can't be bigger then a shortest path from u to v 633 00:40:20,810 --> 00:40:22,280 that also goes through x. 634 00:40:25,770 --> 00:40:28,670 Of my paths, I'm now restricting the paths 635 00:40:28,670 --> 00:40:30,960 I have to the ones that go through x. 636 00:40:36,030 --> 00:40:38,010 The shortest path distance from u to v 637 00:40:38,010 --> 00:40:41,340 can't be bigger than restricting paths that go through 638 00:40:41,340 --> 00:40:43,710 x and taking that shortest distance, 639 00:40:43,710 --> 00:40:45,870 getting the shortest path distance from here 640 00:40:45,870 --> 00:40:49,110 and adding it to the shortest path distance here-- 641 00:40:49,110 --> 00:41:01,020 delta u, x, delta x, v. That's just a statement of, 642 00:41:01,020 --> 00:41:04,980 I'm restricting to a subset of the paths. 643 00:41:04,980 --> 00:41:07,560 I can't decrease my minimum distance. 644 00:41:07,560 --> 00:41:10,480 So this is the statement of the triangle inequality, 645 00:41:10,480 --> 00:41:14,530 that the shortest path distance from u to v 646 00:41:14,530 --> 00:41:20,560 can't be bigger than the shortest path distance from u 647 00:41:20,560 --> 00:41:23,800 to x plus the shortest path distance from x 648 00:41:23,800 --> 00:41:29,410 to v for any x in my graph that's not u and v. 649 00:41:29,410 --> 00:41:32,260 So that's the triangle inequality. 650 00:41:32,260 --> 00:41:34,240 Pretty intuitive notion, right? 651 00:41:34,240 --> 00:41:36,100 Why is this useful? 652 00:41:36,100 --> 00:41:37,090 OK, well, if I find-- 653 00:41:39,980 --> 00:41:46,580 if I find an edge in my graph, if there's an edge u, v, 654 00:41:46,580 --> 00:41:59,560 in my graph such that this condition is violated 655 00:41:59,560 --> 00:42:01,840 for the estimates that I have-- 656 00:42:01,840 --> 00:42:05,050 it obviously can't be violated on my shortest path distances, 657 00:42:05,050 --> 00:42:08,230 but if it violates it on the estimates-- 658 00:42:08,230 --> 00:42:13,960 u, v, is bigger than u, x-- 659 00:42:13,960 --> 00:42:17,580 sorry, u-- how am I going to do this? 660 00:42:17,580 --> 00:42:20,670 I want this to be s. 661 00:42:20,670 --> 00:42:25,850 I'm calculating shortest path distances from s 662 00:42:25,850 --> 00:42:28,640 and shortest path distances from s 663 00:42:28,640 --> 00:42:35,330 to some incoming vertex u plus the edge weight from u 664 00:42:35,330 --> 00:42:39,890 to v. All right, so what is this doing? 665 00:42:39,890 --> 00:42:44,540 I have some edge u, v in my graph. 666 00:42:44,540 --> 00:42:47,840 Basically, what I've said is that I have some distance 667 00:42:47,840 --> 00:42:52,240 estimate to u, but going through-- 668 00:42:52,240 --> 00:42:55,660 making a path to v by going through u, 669 00:42:55,660 --> 00:42:59,380 and then the edge from u to v is better 670 00:42:59,380 --> 00:43:03,440 than my current estimate, my shortest path 671 00:43:03,440 --> 00:43:08,222 estimate to v. That's bad, right? 672 00:43:08,222 --> 00:43:09,930 That's violating the triangle inequality. 673 00:43:09,930 --> 00:43:12,150 These cannot be the right weights. 674 00:43:12,150 --> 00:43:14,670 These cannot be the right distances. 675 00:43:14,670 --> 00:43:17,820 So how we're going to do that is lower-- 676 00:43:17,820 --> 00:43:21,570 this is what we said, repeatedly lower these distance estimates. 677 00:43:21,570 --> 00:43:24,720 I'm going to lower this guy down to equal this thing. 678 00:43:24,720 --> 00:43:28,110 In a sense, this constraint was violated. 679 00:43:28,110 --> 00:43:30,630 But now we're relaxing that constraint 680 00:43:30,630 --> 00:43:34,140 so that this is no longer violated. 681 00:43:34,140 --> 00:43:38,370 So relax is a little weird word here. 682 00:43:38,370 --> 00:43:40,350 We're using it for historical reasons. 683 00:43:40,350 --> 00:43:42,750 But that's what we mean by when we say relax. 684 00:43:42,750 --> 00:43:45,390 This thing is a violated constraint. 685 00:43:45,390 --> 00:43:48,660 It's got some pressure to be resolved. 686 00:43:48,660 --> 00:43:50,700 And so what we're doing is, to resolve it, 687 00:43:50,700 --> 00:43:53,702 we're just setting this guy equal to this, 688 00:43:53,702 --> 00:43:55,785 so it at least resolves, locally, that constraint. 689 00:43:55,785 --> 00:44:00,300 Now, it may violate the triangle inequality other places now 690 00:44:00,300 --> 00:44:01,650 that we've done this change. 691 00:44:01,650 --> 00:44:04,830 But at least this constraint is now relaxed and satisfied. 692 00:44:04,830 --> 00:44:25,040 OK, so relax edge by lowering d of s, v, to this thing. 693 00:44:25,040 --> 00:44:28,310 That's what we're going to mean by relaxing an edge. 694 00:44:28,310 --> 00:44:35,500 And relaxing an edge is what I'll call safe. 695 00:44:35,500 --> 00:44:38,690 It's safe to do. 696 00:44:38,690 --> 00:44:40,820 What do I mean by relaxation is safe? 697 00:44:40,820 --> 00:44:47,210 It means that as I am computing these shortest path distances, 698 00:44:47,210 --> 00:44:52,300 I'm going to maintain this property that each one of these 699 00:44:52,300 --> 00:44:55,300 estimates-- sorry, these estimates here-- 700 00:44:55,300 --> 00:44:56,880 has the property that it's either 701 00:44:56,880 --> 00:45:05,800 infinite or it is the weight of some path to v. So 702 00:45:05,800 --> 00:45:07,060 that's the thing the-- 703 00:45:07,060 --> 00:45:08,290 relaxation is safe. 704 00:45:17,940 --> 00:45:27,960 OK, so each distance estimate, s, v, 705 00:45:27,960 --> 00:45:44,820 is weight of some path from s to v or infinite. 706 00:45:44,820 --> 00:45:48,810 And this is a pretty easy thing to prove. 707 00:45:48,810 --> 00:45:52,410 If I had the invariant that these 708 00:45:52,410 --> 00:45:54,220 were all weights of shortest paths, 709 00:45:54,220 --> 00:45:57,090 let's try to relax an edge. 710 00:45:57,090 --> 00:46:00,450 And we need to show that this property is maintained. 711 00:46:00,450 --> 00:46:04,140 Relax edge u, v, OK? 712 00:46:04,140 --> 00:46:08,960 Now, if I relax edge u, v, what do I do? 713 00:46:08,960 --> 00:46:12,490 I set this thing-- 714 00:46:12,490 --> 00:46:16,540 or, sorry, I set this thing, my shortest path distance 715 00:46:16,540 --> 00:46:19,600 to v, to be this thing plus this thing. 716 00:46:19,600 --> 00:46:23,110 There's a weight of an edge from u to v. 717 00:46:23,110 --> 00:46:27,040 Now, by my assumption that we're maintaining 718 00:46:27,040 --> 00:46:33,590 that this is the weight of some path in my graph, 719 00:46:33,590 --> 00:46:35,900 if this thing is bigger, I'm setting it 720 00:46:35,900 --> 00:46:38,330 to the weight of some path on my graph to u, 721 00:46:38,330 --> 00:46:42,970 plus an edge from u to v, and so this checks out. 722 00:46:42,970 --> 00:47:00,365 So assign d of s, v, to weight of some path. 723 00:47:04,770 --> 00:47:07,140 I'm not going to write down all the argument that I just 724 00:47:07,140 --> 00:47:07,770 had here. 725 00:47:07,770 --> 00:47:10,950 But basically, since this distance estimate 726 00:47:10,950 --> 00:47:15,520 was by supposition before the weight of some path to v-- 727 00:47:15,520 --> 00:47:20,170 to u, then this is, again, the weight of some path to v. 728 00:47:20,170 --> 00:47:21,210 OK, great. 729 00:47:21,210 --> 00:47:24,340 So now we're ready to actually go through this algorithm. 730 00:47:28,950 --> 00:47:35,260 So DAG relaxation, from over there, 731 00:47:35,260 --> 00:47:42,730 initializes all of our distance estimates to equal infinity, 732 00:47:42,730 --> 00:47:44,035 just like we did in BFS. 733 00:47:47,470 --> 00:47:57,340 Then, set my distance estimate to myself to be 0. 734 00:47:57,340 --> 00:48:00,340 Now, it's possible that this might be minus infinity 735 00:48:00,340 --> 00:48:03,480 or negative at some point. 736 00:48:03,480 --> 00:48:06,470 But right now, I'm just setting it to 0. 737 00:48:06,470 --> 00:48:11,900 And either way, 0 is going to upper-bound the distance to s. 738 00:48:11,900 --> 00:48:18,340 So in particular, at initialization, 739 00:48:18,340 --> 00:48:24,580 anything not reachable from s is set correctly. 740 00:48:24,580 --> 00:48:29,140 And s itself is set as an upper bound to the shortest path 741 00:48:29,140 --> 00:48:30,430 distance. 742 00:48:30,430 --> 00:48:48,730 Now we're going to process each vertex u in a topological sort 743 00:48:48,730 --> 00:48:50,170 order. 744 00:48:50,170 --> 00:48:54,250 So remember, our input to DAG relaxation is a DAG. 745 00:48:54,250 --> 00:48:57,220 So this thing has a topological sort order. 746 00:48:57,220 --> 00:49:00,700 We're going to process these vertices in that order. 747 00:49:00,700 --> 00:49:02,980 You can imagine we're starting at the top, 748 00:49:02,980 --> 00:49:05,230 and all my vertices are-- 749 00:49:05,230 --> 00:49:07,060 all my edges are pointed away from me. 750 00:49:07,060 --> 00:49:09,280 And I'm just processing each vertex 751 00:49:09,280 --> 00:49:13,390 down this topological sort line. 752 00:49:13,390 --> 00:49:15,940 And then for each of these vertices, 753 00:49:15,940 --> 00:49:17,210 what am I going to do? 754 00:49:17,210 --> 00:49:20,180 I'm going to look at all the outgoing neighbors. 755 00:49:20,180 --> 00:49:23,350 And if the triangle inequality is violated, 756 00:49:23,350 --> 00:49:26,110 I'm going to relax that edge. 757 00:49:26,110 --> 00:49:28,510 The algorithm is as simple as that. 758 00:49:28,510 --> 00:49:38,590 For each outgoing neighbor of v-- 759 00:49:38,590 --> 00:49:41,830 sorry, of u-- 760 00:49:41,830 --> 00:49:43,590 I always get u and v mixed up here. 761 00:49:47,780 --> 00:49:53,970 If my shortest path estimate to v 762 00:49:53,970 --> 00:50:03,000 violates the triangle inequality for an edge, 763 00:50:03,000 --> 00:50:06,120 for an incoming edge, then I'm going to set-- 764 00:50:06,120 --> 00:50:13,620 relax u, v, i.e. 765 00:50:13,620 --> 00:50:25,540 set d, s,v, equal to d, s,u, plus w, u,v. 766 00:50:25,540 --> 00:50:28,860 So that's the algorithm. 767 00:50:28,860 --> 00:50:34,990 So if I were to take a look at this example graph over here, 768 00:50:34,990 --> 00:50:37,180 maybe a is my start vertex. 769 00:50:37,180 --> 00:50:40,000 I initialize it to-- 770 00:50:40,000 --> 00:50:41,093 AUDIENCE: DAG. 771 00:50:41,093 --> 00:50:42,260 JASON KU: This is not a DAG. 772 00:50:42,260 --> 00:50:43,480 Thank you. 773 00:50:43,480 --> 00:50:44,440 Let's make it a DAG. 774 00:50:49,710 --> 00:50:55,740 I claim to you now this is a DAG. 775 00:50:55,740 --> 00:50:57,810 In particular, a topological sort order 776 00:50:57,810 --> 00:51:01,870 is when there's a path through all the vertices, 777 00:51:01,870 --> 00:51:04,350 then there's a unique topological sort order-- 778 00:51:04,350 --> 00:51:09,990 a, b, e, f, g, h, d, c. 779 00:51:09,990 --> 00:51:11,400 This is a topological order. 780 00:51:11,400 --> 00:51:14,470 You can check all the edges. 781 00:51:14,470 --> 00:51:22,300 So I'm going to start by setting the-- 782 00:51:22,300 --> 00:51:25,480 actually, let's use e. 783 00:51:25,480 --> 00:51:26,860 Let's use shortest paths from e. 784 00:51:26,860 --> 00:51:28,660 Why not? 785 00:51:28,660 --> 00:51:29,710 Shortest paths from e. 786 00:51:33,460 --> 00:51:38,050 Vertex a actually comes before e in the topological order. 787 00:51:38,050 --> 00:51:39,130 So it has no-- 788 00:51:39,130 --> 00:51:42,550 I mean, its shortest path distance, when I initialize, 789 00:51:42,550 --> 00:51:44,380 I'm going to initialize this to 0. 790 00:51:44,380 --> 00:51:49,290 I'm going to initialize this to infinite, infinite, infinite, 791 00:51:49,290 --> 00:51:52,360 infinite, all these things to infinite. 792 00:51:52,360 --> 00:51:54,110 These are my estimates. 793 00:51:54,110 --> 00:51:56,680 These are not quite the shortest paths yet-- 794 00:51:56,680 --> 00:51:58,900 distances. 795 00:51:58,900 --> 00:52:02,670 But when I get here, clearly I can't be-- 796 00:52:07,880 --> 00:52:11,420 distance to me being infinite can never 797 00:52:11,420 --> 00:52:14,150 violate the triangle inequality with something infinite 798 00:52:14,150 --> 00:52:14,930 or finite. 799 00:52:14,930 --> 00:52:16,910 It doesn't matter, right? 800 00:52:16,910 --> 00:52:19,760 So I don't do anything to a. 801 00:52:19,760 --> 00:52:23,420 Anything before my source vertex in the topological order 802 00:52:23,420 --> 00:52:25,928 can't be visited, because it's before 803 00:52:25,928 --> 00:52:26,970 in the topological order. 804 00:52:26,970 --> 00:52:28,310 That's kind of the point. 805 00:52:28,310 --> 00:52:30,620 There's no path from my source vertex 806 00:52:30,620 --> 00:52:35,300 to anything before it in the topological order. 807 00:52:35,300 --> 00:52:37,880 So same with b. 808 00:52:37,880 --> 00:52:41,640 b is before a in the topological order. 809 00:52:41,640 --> 00:52:48,150 Now, I'm at e, and it's possible we are violating triangle 810 00:52:48,150 --> 00:52:51,300 inequality, in particular here. 811 00:52:51,300 --> 00:52:55,440 I think the shortest path distance to f is infinite. 812 00:52:55,440 --> 00:53:02,760 But actually, if I go to e through this edge with a weight 813 00:53:02,760 --> 00:53:07,380 3, I know that this is violating triangle inequality. 814 00:53:07,380 --> 00:53:11,260 So actually, this thing is wrong, 815 00:53:11,260 --> 00:53:15,060 and I can set it equal to 3. 816 00:53:15,060 --> 00:53:20,050 Now, that might not be the shortest path distance. 817 00:53:20,050 --> 00:53:24,840 But right now it's a better estimate, so we've set it. 818 00:53:24,840 --> 00:53:26,730 Now, I'm moving on. 819 00:53:26,730 --> 00:53:28,170 I'm done with this guy. 820 00:53:28,170 --> 00:53:30,930 I move to the next vertex in my topological order. 821 00:53:30,930 --> 00:53:34,450 And again, I relax edges out of f. 822 00:53:34,450 --> 00:53:38,280 OK, so here, looking at 8, 3 plus 8 823 00:53:38,280 --> 00:53:41,550 is better than infinite, so we'll say that that's 11. 824 00:53:44,880 --> 00:53:50,460 And 3 plus 2 is better than infinite, so that's 5. 825 00:53:50,460 --> 00:53:52,020 Now, I keep going in the topological 826 00:53:52,020 --> 00:53:54,000 order. g is the next. 827 00:53:54,000 --> 00:53:55,860 5 plus 1 is 6. 828 00:53:55,860 --> 00:53:58,470 OK, so we found a better estimate here. 829 00:53:58,470 --> 00:54:00,930 So this 11 is not as good. 830 00:54:00,930 --> 00:54:04,410 6 is better, so we replace it. 831 00:54:04,410 --> 00:54:06,690 Here, we haven't visited before. 832 00:54:06,690 --> 00:54:07,530 It's still infinite. 833 00:54:07,530 --> 00:54:10,470 So 5 plus minus 2 is 3. 834 00:54:13,110 --> 00:54:14,860 This is the next in the topological order. 835 00:54:14,860 --> 00:54:20,610 3 plus 9 is bigger than 6, so that's not a shorter path. 836 00:54:20,610 --> 00:54:23,310 3 plus 4 is certainly smaller than infinite, 837 00:54:23,310 --> 00:54:25,740 so set that equal to 7. 838 00:54:25,740 --> 00:54:29,190 Then, 7 plus 5 is also bigger than 6. 839 00:54:29,190 --> 00:54:33,090 And actually, you can confirm that these are all the shortest 840 00:54:33,090 --> 00:54:36,620 path distances from e. 841 00:54:36,620 --> 00:54:39,830 So this algorithm seems to work. 842 00:54:39,830 --> 00:54:43,350 Does it actually work? 843 00:54:43,350 --> 00:54:45,140 Let's take a look. 844 00:54:45,140 --> 00:54:52,970 The claim to you is that at the end of relaxation, 845 00:54:52,970 --> 00:54:55,010 this algorithm, we've set-- 846 00:54:58,220 --> 00:55:12,530 claim at end, all the estimates equal shortest path distances. 847 00:55:16,340 --> 00:55:22,120 The basic idea here is that if I take 848 00:55:22,120 --> 00:55:26,650 the k-th vertex in the topological order, 849 00:55:26,650 --> 00:55:29,170 assuming that these distances are all 850 00:55:29,170 --> 00:55:33,100 equal for the ones before me in the topological order, 851 00:55:33,100 --> 00:55:35,320 I can prove by induction. 852 00:55:35,320 --> 00:55:41,730 We can consider a shortest path from s to v, the k-th vertex, 853 00:55:41,730 --> 00:55:46,040 and look at the vertex preceding me along the shortest path. 854 00:55:46,040 --> 00:55:52,340 That vertex better be before me in the topological order 855 00:55:52,340 --> 00:55:54,370 or we're not a DAG. 856 00:55:54,370 --> 00:55:56,370 And we've already set its shortest path distance 857 00:55:56,370 --> 00:56:03,040 to be equal to the correct thing by induction. 858 00:56:03,040 --> 00:56:05,100 So then when we processed u-- 859 00:56:08,160 --> 00:56:12,390 s to u to v-- 860 00:56:12,390 --> 00:56:16,680 when we processed u in DAG relaxation 861 00:56:16,680 --> 00:56:19,530 here, processed the vertex and looked at all 862 00:56:19,530 --> 00:56:21,960 its outgoing adjacencies, we would 863 00:56:21,960 --> 00:56:27,055 have relaxed this edge to be no greater than that shortest path 864 00:56:27,055 --> 00:56:27,555 distance. 865 00:56:32,320 --> 00:56:35,040 So this is correct. 866 00:56:35,040 --> 00:56:38,420 You can also think of it as the DAG relaxation 867 00:56:38,420 --> 00:56:45,360 algorithm for each vertex looks all at its incoming neighbors, 868 00:56:45,360 --> 00:56:47,610 assuming that their shortest path distances are 869 00:56:47,610 --> 00:56:49,990 computed correctly already. 870 00:56:49,990 --> 00:56:51,898 Any shortest path distance to me needs 871 00:56:51,898 --> 00:56:53,940 to be composed of a shortest path distance to one 872 00:56:53,940 --> 00:56:58,800 of my incoming neighbors through an edge to me, 873 00:56:58,800 --> 00:57:00,630 so I can just check all of them. 874 00:57:00,630 --> 00:57:03,780 That's what DAG relaxation does. 875 00:57:03,780 --> 00:57:06,340 And again, we're looping over every vertex 876 00:57:06,340 --> 00:57:08,880 and looking at its adjacencies doing constant work. 877 00:57:08,880 --> 00:57:11,160 This, again, takes linear time. 878 00:57:11,160 --> 00:57:15,990 OK, so that's shortest paths and a DAG. 879 00:57:15,990 --> 00:57:19,560 Next time, we'll look at, for general graphs, 880 00:57:19,560 --> 00:57:23,730 how we can use kind of the same technique in an algorithm 881 00:57:23,730 --> 00:57:24,840 we call Bellman-Ford. 882 00:57:24,840 --> 00:57:27,620 OK, that's it for today.