1
00:00:00,000 --> 00:00:01,960
[SQUEAKING]
2
00:00:01,960 --> 00:00:03,920
[RUSTLING]
3
00:00:03,920 --> 00:00:07,350
[CLICKING]
4
00:00:12,760 --> 00:00:14,540
JASON KU: OK, let's get started.
5
00:00:14,540 --> 00:00:19,270
Welcome to the 12th
lecture of 6.006.
6
00:00:19,270 --> 00:00:24,100
This is our second lecture
talking about weighted graphs,
7
00:00:24,100 --> 00:00:27,670
and in particular, weighted
shortest paths, algorithms.
8
00:00:27,670 --> 00:00:30,740
Last time we talked
about weighted graphs.
9
00:00:30,740 --> 00:00:32,560
This is a kind of
a generalization
10
00:00:32,560 --> 00:00:36,610
of what we mean by distance
in an unweighted graph instead
11
00:00:36,610 --> 00:00:40,900
of each edge having a
weight of 1, essentially.
12
00:00:40,900 --> 00:00:44,440
We generalize that
to be any integer.
13
00:00:44,440 --> 00:00:48,790
And last time, we
showed how to solve
14
00:00:48,790 --> 00:00:51,790
shortest single-source
shortest paths in a graph that
15
00:00:51,790 --> 00:00:54,130
doesn't have cycles
even if it has
16
00:00:54,130 --> 00:00:58,540
0 or negative weights in linear
time using an algorithm called
17
00:00:58,540 --> 00:01:00,430
DAG relaxation.
18
00:01:00,430 --> 00:01:04,180
We also showed in that
lecture how in linear time,
19
00:01:04,180 --> 00:01:07,570
if we are given the shortest
path weights to all the things
20
00:01:07,570 --> 00:01:09,940
reachable in finite--
21
00:01:09,940 --> 00:01:14,380
or with shortest path
distance that's finite,
22
00:01:14,380 --> 00:01:16,960
we can construct a
shortest paths tree
23
00:01:16,960 --> 00:01:20,380
from those weights
in linear time.
24
00:01:20,380 --> 00:01:23,050
So this is motivating
why we're not really
25
00:01:23,050 --> 00:01:25,960
going to talk about parent
pointers for the next couple
26
00:01:25,960 --> 00:01:26,800
of lectures.
27
00:01:26,800 --> 00:01:30,940
We're just going to concentrate
on the shortest path weights.
28
00:01:30,940 --> 00:01:32,950
And so today, we're
going to be talking
29
00:01:32,950 --> 00:01:35,950
about our most general
algorithm we'll
30
00:01:35,950 --> 00:01:39,280
be showing for solving
single source shortest paths,
31
00:01:39,280 --> 00:01:42,940
in particular in graphs
that could contain cycles
32
00:01:42,940 --> 00:01:45,280
and could have negative weights.
33
00:01:45,280 --> 00:01:50,930
So just to recap our
little roadmap here,
34
00:01:50,930 --> 00:01:53,910
single source shortest
paths in linear time.
35
00:01:53,910 --> 00:01:56,440
Last time we discussed
another linear time algorithm,
36
00:01:56,440 --> 00:01:58,240
DAG relaxation.
37
00:01:58,240 --> 00:01:59,800
And today we're
going to be talking
38
00:01:59,800 --> 00:02:04,240
about Bellman-Ford, which isn't
limited to asymptotic graphs.
39
00:02:04,240 --> 00:02:07,600
In particular, there could
be negative weight cycles
40
00:02:07,600 --> 00:02:09,460
in our graph.
41
00:02:09,460 --> 00:02:12,970
If it has cycles, if it
has negative weights,
42
00:02:12,970 --> 00:02:15,820
the worry is that we could
have negative weight cycles,
43
00:02:15,820 --> 00:02:17,500
in which case there--
44
00:02:17,500 --> 00:02:22,420
if a negative weight cycle
is reachable from our source,
45
00:02:22,420 --> 00:02:24,640
then the vertices in
that cycle and anything
46
00:02:24,640 --> 00:02:28,420
reachable from that
cycle will potentially
47
00:02:28,420 --> 00:02:34,540
have an unbounded number of
edges you need to go through.
48
00:02:34,540 --> 00:02:38,560
There's not a bound on the
number of edges for a shortest
49
00:02:38,560 --> 00:02:41,470
path, because I could
just keep going around
50
00:02:41,470 --> 00:02:44,740
that cycle as many times as I
want and get a shorter path.
51
00:02:44,740 --> 00:02:49,558
And so we assign those
distances to be minus infinity.
52
00:02:49,558 --> 00:02:51,850
So that's what we're going
to do today in Bellman-Ford.
53
00:02:51,850 --> 00:02:57,400
In particular, what we're going
to do is compute our shortest
54
00:02:57,400 --> 00:02:59,440
path distances,
the shortest path
55
00:02:59,440 --> 00:03:03,760
waits for every
vertex in our graph,
56
00:03:03,760 --> 00:03:07,030
setting the ones that are
not reachable to infinity,
57
00:03:07,030 --> 00:03:10,300
and the ones that are reachable
through a negative weight
58
00:03:10,300 --> 00:03:13,450
cycle to minus infinity, and
all other ones we're going
59
00:03:13,450 --> 00:03:16,420
to set to a finite weight.
60
00:03:16,420 --> 00:03:19,630
And another thing
that we might want
61
00:03:19,630 --> 00:03:21,340
is if there's a
negative weight cycle
62
00:03:21,340 --> 00:03:24,130
in the graph, let's return one.
63
00:03:24,130 --> 00:03:25,630
So those are the
two kinds of things
64
00:03:25,630 --> 00:03:29,350
that we're trying to
solve in today's lecture.
65
00:03:29,350 --> 00:03:31,510
But before we do
that, let's warm up
66
00:03:31,510 --> 00:03:36,020
with two short exercises.
67
00:03:36,020 --> 00:03:48,110
The first one, exercise 1,
given an undirected graph,
68
00:03:48,110 --> 00:04:12,280
given undirected graph G,
return whether G contains
69
00:04:12,280 --> 00:04:13,450
a negative weight cycle.
70
00:04:20,140 --> 00:04:23,740
Anyone have an idea of how we
can solve this in linear time,
71
00:04:23,740 --> 00:04:25,030
actually?
72
00:04:25,030 --> 00:04:30,280
In fact, we can do it in
order E. No-- yes, yes.
73
00:04:30,280 --> 00:04:34,740
Reachable from S. I guess--
74
00:04:34,740 --> 00:04:40,110
let's just say a negative
weight cycle at all.
75
00:04:40,110 --> 00:04:42,787
Not in the context of
single-source shortest paths.
76
00:04:42,787 --> 00:04:45,120
AUDIENCE: Detect whether
there's a negative weight edge?
77
00:04:45,120 --> 00:04:46,230
JASON KU: Ah.
78
00:04:46,230 --> 00:04:50,610
Your colleague has determined
an interesting fact
79
00:04:50,610 --> 00:04:53,250
about undirected graphs.
80
00:04:53,250 --> 00:04:57,720
If you have a negative weight
edge in an undirected graph,
81
00:04:57,720 --> 00:05:01,170
I can just move back and
forth along that edge.
82
00:05:01,170 --> 00:05:03,270
That's a cycle of length 2--
83
00:05:03,270 --> 00:05:07,335
or I guess three vertices
back to where we came from.
84
00:05:07,335 --> 00:05:09,210
There is of negative
weight, because I'm just
85
00:05:09,210 --> 00:05:13,240
traversing that weight over
and over and over again.
86
00:05:13,240 --> 00:05:16,320
So the question of
single-source shortest paths
87
00:05:16,320 --> 00:05:19,050
of finding negative
weights is not
88
00:05:19,050 --> 00:05:22,780
particularly interesting
in the undirected case.
89
00:05:22,780 --> 00:05:26,070
What I can do is just for
every negative weight edge,
90
00:05:26,070 --> 00:05:31,050
undirected edge in my graph, I
can just find the readability
91
00:05:31,050 --> 00:05:33,300
from the vertices--
92
00:05:33,300 --> 00:05:40,200
the endpoints of that edge
and label them as minus--
93
00:05:40,200 --> 00:05:43,410
basically if the connected
component containing S
94
00:05:43,410 --> 00:05:47,160
has a negative weight edge,
then everything in the graph
95
00:05:47,160 --> 00:05:49,080
is accessible from a
negative weight cycle.
96
00:05:49,080 --> 00:05:51,970
So this is not such an
interesting problem.
97
00:05:51,970 --> 00:06:00,540
And so we're going to
restrict our discussion today
98
00:06:00,540 --> 00:06:02,685
to directed graphs.
99
00:06:09,020 --> 00:06:20,920
So this is if and only if
exists negative weight edge.
100
00:06:24,700 --> 00:06:32,770
OK, exercise 2, kind of a little
preview for what's to come,
101
00:06:32,770 --> 00:06:36,460
we're actually not going to show
you an algorithm directly that
102
00:06:36,460 --> 00:06:40,050
meets this Bellman-Ford
running time, V times E.
103
00:06:40,050 --> 00:06:44,080
What instead we're going to show
you is an algorithm that solves
104
00:06:44,080 --> 00:06:47,080
single-source
shortest paths in--
105
00:06:47,080 --> 00:06:54,070
So given an algorithm,
Alg A, solves
106
00:06:54,070 --> 00:07:04,522
single-source shortest paths
in order V times V plus E time.
107
00:07:04,522 --> 00:07:05,230
OK, what is that?
108
00:07:05,230 --> 00:07:09,430
That's V squared plus
V times E. That's
109
00:07:09,430 --> 00:07:13,510
close to what this V times E is.
110
00:07:13,510 --> 00:07:15,940
That's what we're
going to show you.
111
00:07:15,940 --> 00:07:19,870
But if I had such an
algorithm, can anyone
112
00:07:19,870 --> 00:07:22,600
tell me a single-source
shortest paths algorithm--
113
00:07:22,600 --> 00:07:26,560
how we can use this algorithm
to solve single-source shortest
114
00:07:26,560 --> 00:07:29,320
paths in just V times E time?
115
00:07:35,730 --> 00:07:48,990
Show how to solve SSSP
in order the V times E. I
116
00:07:48,990 --> 00:07:52,740
guess we can put a
dot there as well.
117
00:07:52,740 --> 00:07:55,080
So this is a little tricky.
118
00:07:55,080 --> 00:07:57,000
It's kind of related
to the difference
119
00:07:57,000 --> 00:07:59,370
we had between the
reachability problem
120
00:07:59,370 --> 00:08:01,980
and the single-source
shortest paths problem
121
00:08:01,980 --> 00:08:05,310
that we saw last lecture.
122
00:08:05,310 --> 00:08:09,840
When are these asymptotically
different in their upper bound
123
00:08:09,840 --> 00:08:17,590
is when V is asymptotically
larger than E.
124
00:08:17,590 --> 00:08:22,350
But the connected
component containing S
125
00:08:22,350 --> 00:08:27,090
can have at most E
vertices, or order
126
00:08:27,090 --> 00:08:30,960
E. It can actually have
at most E plus 1 vertices,
127
00:08:30,960 --> 00:08:37,169
because otherwise it
wouldn't be connected.
128
00:08:37,169 --> 00:08:41,490
So, what we can do if we
had such an algorithm,
129
00:08:41,490 --> 00:08:44,400
we could first, when we're
giving our graph, explore
130
00:08:44,400 --> 00:08:48,120
everything in the
graph using BFS or DFS,
131
00:08:48,120 --> 00:08:50,670
find all the things
reachable from S,
132
00:08:50,670 --> 00:08:53,250
and then just throw
away everything else.
133
00:08:53,250 --> 00:08:56,700
Now I have a graph for
which V is asymptotically
134
00:08:56,700 --> 00:09:00,870
no bigger than E, and then
we can use this algorithm
135
00:09:00,870 --> 00:09:04,530
to solve single-source shortest
paths in V times E time.
136
00:09:04,530 --> 00:09:06,330
I'm not going to write
all that down here.
137
00:09:06,330 --> 00:09:07,860
You can see it in the notes.
138
00:09:07,860 --> 00:09:08,440
Yeah?
139
00:09:08,440 --> 00:09:10,913
AUDIENCE: Does this work
if your graph isn't simple?
140
00:09:10,913 --> 00:09:13,080
JASON KU: Does this work
as your graph isn't simple?
141
00:09:13,080 --> 00:09:14,290
I haven't thought about it.
142
00:09:14,290 --> 00:09:16,290
We are not going to talk
about non-simple graphs
143
00:09:16,290 --> 00:09:20,190
in this class, but
probably not because you've
144
00:09:20,190 --> 00:09:21,780
got a lot of edges.
145
00:09:21,780 --> 00:09:23,940
Though in our class
if we're talking
146
00:09:23,940 --> 00:09:26,100
about single-source
shortest paths,
147
00:09:26,100 --> 00:09:28,830
if we have multiple edges
between two vertices,
148
00:09:28,830 --> 00:09:30,750
we can just take the
minimum weight one
149
00:09:30,750 --> 00:09:33,765
because it's never better
to take the larger ones.
150
00:09:33,765 --> 00:09:35,430
Does that answer your question?
151
00:09:35,430 --> 00:09:36,170
Great.
152
00:09:36,170 --> 00:09:36,670
All right.
153
00:09:36,670 --> 00:09:39,687
So those are our
warm-ups, that's our goal.
154
00:09:39,687 --> 00:09:42,270
We need to find an algorithm for
single-source shortest paths.
155
00:09:42,270 --> 00:09:44,370
And general graphs,
graphs with--
156
00:09:44,370 --> 00:09:49,020
potentially graphs with
cycles, and negative weights,
157
00:09:49,020 --> 00:09:54,270
and solve it in this V
times linear kind of time.
158
00:09:54,270 --> 00:09:55,800
That makes sense?
159
00:09:55,800 --> 00:09:56,610
All right.
160
00:09:56,610 --> 00:10:00,150
So first, before we
get to the algorithm,
161
00:10:00,150 --> 00:10:02,250
we're going to
discuss a little bit
162
00:10:02,250 --> 00:10:06,670
about simple short-- about
shortest paths in general.
163
00:10:06,670 --> 00:10:10,140
If we didn't-- the problem
here is negative weights.
164
00:10:10,140 --> 00:10:11,650
How do we find--
165
00:10:11,650 --> 00:10:14,040
if we had negative
weight cycles,
166
00:10:14,040 --> 00:10:15,520
there seems to be
these problems,
167
00:10:15,520 --> 00:10:20,360
because we could have minus
infinities is in our deltas.
168
00:10:20,360 --> 00:10:22,670
But if we didn't have
negative weights,
169
00:10:22,670 --> 00:10:26,360
I'd like to assert to you
that our shortest paths, even
170
00:10:26,360 --> 00:10:29,580
if there are negative weights,
are going to be simple.
171
00:10:29,580 --> 00:10:31,010
They won't repeat vertices.
172
00:10:31,010 --> 00:10:33,590
So that's the first thing
we're going to show you.
173
00:10:38,520 --> 00:10:40,050
Let's see.
174
00:10:40,050 --> 00:10:45,235
Simple shortest paths.
175
00:10:49,390 --> 00:10:51,130
OK.
176
00:10:51,130 --> 00:10:52,740
So, claim.
177
00:10:55,660 --> 00:10:58,240
I'm going to give
my claims numbers
178
00:10:58,240 --> 00:11:01,180
today just because I'm
going to have a lot of them.
179
00:11:01,180 --> 00:11:06,580
If my shortest path distance
from S to some vertex
180
00:11:06,580 --> 00:11:13,750
is finite, meaning it's not
infinite or minus infinite--
181
00:11:13,750 --> 00:11:25,315
some finite value, there
exists a shortest path--
182
00:11:28,110 --> 00:11:37,270
a shortest S to V
path that is simple.
183
00:11:37,270 --> 00:11:40,030
And remember, simple means
not going through a vertex
184
00:11:40,030 --> 00:11:41,290
more than once.
185
00:11:41,290 --> 00:11:42,958
All right.
186
00:11:42,958 --> 00:11:44,250
How are we going to prove this?
187
00:11:49,390 --> 00:11:55,960
Well, consider if this
claim was not true.
188
00:11:55,960 --> 00:12:01,625
If every shortest path
contained a cycle, essentially.
189
00:12:01,625 --> 00:12:03,830
It repeated a vertex.
190
00:12:03,830 --> 00:12:07,910
Then my path looks
something like this.
191
00:12:11,350 --> 00:12:16,930
I mean, there's some vertices
along here, and then I go to V.
192
00:12:16,930 --> 00:12:19,690
So here's S, and then
there's some cycle I
193
00:12:19,690 --> 00:12:21,470
repeat, some vertex.
194
00:12:21,470 --> 00:12:26,310
I'm going to call this cycle C.
195
00:12:26,310 --> 00:12:28,920
Now what do I know
about this path?
196
00:12:28,920 --> 00:12:34,080
I know that it has--
it's a shortest path
197
00:12:34,080 --> 00:12:36,560
and it has finite weight.
198
00:12:36,560 --> 00:12:39,950
So in particular, this
path-- this delta distance
199
00:12:39,950 --> 00:12:41,820
is not minus infinity.
200
00:12:41,820 --> 00:12:45,010
But if this is not
minus infinity,
201
00:12:45,010 --> 00:12:48,538
what do I know about the
weight of this cycle?
202
00:12:48,538 --> 00:12:49,910
AUDIENCE: It's not negative.
203
00:12:49,910 --> 00:12:50,535
JASON KU: Yeah.
204
00:12:50,535 --> 00:12:51,932
It can't be negative.
205
00:12:51,932 --> 00:12:53,390
Because if it was
negative, I could
206
00:12:53,390 --> 00:12:55,580
keep going around
this cycle, and this
207
00:12:55,580 --> 00:12:58,370
would have a non-finite weight.
208
00:12:58,370 --> 00:13:03,350
Shortest path distance
from S. So I know this is--
209
00:13:03,350 --> 00:13:06,860
can't be negative, so it
must be 0 or positive.
210
00:13:06,860 --> 00:13:10,880
But if it's 0 or positive
and this is a shortest path--
211
00:13:10,880 --> 00:13:16,960
went through this cycle,
then I could remove it,
212
00:13:16,960 --> 00:13:21,360
and now I have a new path
with one fewer cycle.
213
00:13:21,360 --> 00:13:26,130
I could just keep doing this
to create a simple path.
214
00:13:26,130 --> 00:13:28,590
So that checks out.
215
00:13:28,590 --> 00:13:30,120
OK.
216
00:13:30,120 --> 00:13:33,060
So, that's interesting.
217
00:13:33,060 --> 00:13:39,000
If it's simple, what do we
know about the number of edges
218
00:13:39,000 --> 00:13:42,570
in a simple shortest paths?
219
00:13:42,570 --> 00:13:45,740
How many could
there possibly be?
220
00:13:45,740 --> 00:13:51,550
How long in number of edges
could a simple shortest path
221
00:13:51,550 --> 00:13:52,050
be?
222
00:13:52,050 --> 00:13:55,610
If I can't repeat
vertices, I can
223
00:13:55,610 --> 00:14:00,710
have at most vertices
on my simple path, which
224
00:14:00,710 --> 00:14:04,850
means I can use at
most V minus 1 edges--
225
00:14:04,850 --> 00:14:06,590
fence posting.
226
00:14:06,590 --> 00:14:19,260
So, simple paths have
at most V minus 1 edges.
227
00:14:25,520 --> 00:14:30,050
That's a nice little
thing I'd like to box off.
228
00:14:30,050 --> 00:14:31,355
That's a really nice property.
229
00:14:33,920 --> 00:14:40,000
So while a shortest path here
could have an infinite number
230
00:14:40,000 --> 00:14:45,310
of edges, if the shortest
path distance is finite,
231
00:14:45,310 --> 00:14:48,560
I know I only have
to check paths
232
00:14:48,560 --> 00:14:51,710
that use up to V minus 1 edges.
233
00:14:51,710 --> 00:14:56,710
In particular, this is finitely
bounded in terms of the number
234
00:14:56,710 --> 00:14:57,950
of paths I have to consider.
235
00:14:57,950 --> 00:15:02,470
It's exponential, potentially,
but at least it's finite.
236
00:15:02,470 --> 00:15:09,130
The other way, I potentially
had to check every possible path
237
00:15:09,130 --> 00:15:12,740
of which there could be infinite
if there's cycles in my graph.
238
00:15:12,740 --> 00:15:14,420
OK.
239
00:15:14,420 --> 00:15:19,930
So I have an idea.
240
00:15:19,930 --> 00:15:25,950
What if I could find
shortest-path distances
241
00:15:25,950 --> 00:15:30,110
by limiting the number
of edges I go through?
242
00:15:30,110 --> 00:15:35,525
So not the full shortest
path distance from S
243
00:15:35,525 --> 00:15:38,870
to V, but let's limit
the number of edges
244
00:15:38,870 --> 00:15:41,480
I'm allowed to go
through, and let's
245
00:15:41,480 --> 00:15:43,970
talk about those
shortest-path distances,
246
00:15:43,970 --> 00:15:46,640
just among the paths that
have at most a certain number
247
00:15:46,640 --> 00:15:47,450
of edges.
248
00:15:47,450 --> 00:15:55,370
I'm going to call
this k-edge distance.
249
00:15:55,370 --> 00:15:59,240
And I'm just going to provide
a little notation here.
250
00:15:59,240 --> 00:16:02,340
Instead of having a delta,
I'll have a delta k here.
251
00:16:02,340 --> 00:16:05,720
That means how many
edges I'm limited by.
252
00:16:05,720 --> 00:16:24,335
So from S to V is shortest S to
V path using at most k edges.
253
00:16:29,750 --> 00:16:36,800
Short-- weight-- weight of a.
254
00:16:36,800 --> 00:16:43,550
Shortest path, shortest S to
V path using at most k edges.
255
00:16:43,550 --> 00:16:47,600
And these notions seem
somewhat symmetric.
256
00:16:47,600 --> 00:16:59,700
If I were able to compute this
thing for V minus 1, then--
257
00:16:59,700 --> 00:17:06,430
for all the vertices, then
if the distance is finite,
258
00:17:06,430 --> 00:17:08,910
then I'll have
successfully computed
259
00:17:08,910 --> 00:17:12,569
the real shortest paths
because of this statement.
260
00:17:12,569 --> 00:17:16,950
Now that doesn't mean
that if this is--
261
00:17:20,190 --> 00:17:21,900
it doesn't mean the other way.
262
00:17:21,900 --> 00:17:26,940
If this is minus infinity, if
the shortest-path distance is
263
00:17:26,940 --> 00:17:31,820
minus infinity, it doesn't say
anything about what this is.
264
00:17:31,820 --> 00:17:34,500
It just says the
shortest path using
265
00:17:34,500 --> 00:17:40,200
at most V minus 1 vertices,
using at most V minus 1 edges
266
00:17:40,200 --> 00:17:41,970
would be whatever this is.
267
00:17:41,970 --> 00:17:43,710
But really, the
shortest path length
268
00:17:43,710 --> 00:17:46,620
needs to consider an
infinite number of edges.
269
00:17:46,620 --> 00:17:49,840
So it doesn't really
tell us much about that.
270
00:17:49,840 --> 00:17:51,460
But for the finite ones it does.
271
00:17:51,460 --> 00:17:53,050
It works well.
272
00:17:53,050 --> 00:17:56,550
And so if we are able to
compute this thing for k equals
273
00:17:56,550 --> 00:18:01,850
V minus 1 in a graph that
doesn't contain negative weight
274
00:18:01,850 --> 00:18:02,780
cycles, we'd be done.
275
00:18:06,830 --> 00:18:09,770
I claim to you a
stronger statement,
276
00:18:09,770 --> 00:18:24,960
that if the shortest path using
at most V edges from s to v Is
277
00:18:24,960 --> 00:18:29,940
less than-- strictly less
than delta of V minus 1--
278
00:18:33,660 --> 00:18:37,720
this is all in the
subscript here.
279
00:18:37,720 --> 00:18:43,600
Basically this is the
shortest-path distance
280
00:18:43,600 --> 00:18:49,240
of any simple path, and possibly
ones that also contain cycles,
281
00:18:49,240 --> 00:18:53,760
but definitely it includes
all the simple paths.
282
00:18:53,760 --> 00:18:58,590
If there's a shorter
path to my vertex
283
00:18:58,590 --> 00:19:02,850
that goes through more
than V minus 1 edges,
284
00:19:02,850 --> 00:19:06,810
that this path can't
be simple, because it
285
00:19:06,810 --> 00:19:08,850
goes through a vertex
more than once.
286
00:19:08,850 --> 00:19:13,540
Otherwise it would be
included in this distance set.
287
00:19:13,540 --> 00:19:18,010
So if this is the case,
and I found a shorter path
288
00:19:18,010 --> 00:19:21,790
to V that uses v edges--
289
00:19:21,790 --> 00:19:23,980
yeah, that use V
edges, that path
290
00:19:23,980 --> 00:19:28,520
can't be simple, which
means that path or some path
291
00:19:28,520 --> 00:19:31,730
there contains a
negative weight cycle.
292
00:19:31,730 --> 00:19:38,960
So if this is true,
then I know that
293
00:19:38,960 --> 00:19:42,820
the real shortest-path
distance from S to V
294
00:19:42,820 --> 00:19:45,830
must be minus infinity.
295
00:19:49,670 --> 00:19:53,360
I'm going to call such
a vertex a witness.
296
00:19:53,360 --> 00:19:56,490
If we can find a vertex
that has this property--
297
00:19:56,490 --> 00:19:59,720
I mean, I haven't shown you how
to compute these things yet,
298
00:19:59,720 --> 00:20:02,390
but if I were able
to find a vertex V--
299
00:20:02,390 --> 00:20:08,980
and these are capital V's
if you're having trouble.
300
00:20:08,980 --> 00:20:13,150
This V is different than
this V, this is cardinality.
301
00:20:13,150 --> 00:20:15,670
If we can find such
a vertex V, that
302
00:20:15,670 --> 00:20:17,890
certifies that there
is a negative weight
303
00:20:17,890 --> 00:20:21,590
cycle in our graph.
304
00:20:21,590 --> 00:20:25,925
So I'm going to
call V is a witness.
305
00:20:30,630 --> 00:20:32,440
OK.
306
00:20:32,440 --> 00:20:38,720
So, if this property
is true, it's a witness
307
00:20:38,720 --> 00:20:39,980
and it definitely has this.
308
00:20:39,980 --> 00:20:42,152
Is it possible, you think--
309
00:20:42,152 --> 00:20:43,610
I'm going to claim
to you that it's
310
00:20:43,610 --> 00:20:49,330
possible that a vertex could
have minus infinite distance
311
00:20:49,330 --> 00:20:55,060
but not have this property
halt. I could probably
312
00:20:55,060 --> 00:20:55,940
give you an example--
313
00:20:55,940 --> 00:20:58,210
I don't have one off the
top of my head right now,
314
00:20:58,210 --> 00:21:00,220
but that's possible.
315
00:21:00,220 --> 00:21:02,170
You could imagine,
there might be
316
00:21:02,170 --> 00:21:07,630
no path going to a vertex on
a negative weight cycle that
317
00:21:07,630 --> 00:21:10,510
goes through V exactly V edges.
318
00:21:10,510 --> 00:21:14,860
It might go through more
edges, a shorter one.
319
00:21:14,860 --> 00:21:18,820
So this equation
would be inequality
320
00:21:18,820 --> 00:21:23,530
and would not certify
that this is true.
321
00:21:23,530 --> 00:21:28,210
But I claim to you, if a
vertex has this property,
322
00:21:28,210 --> 00:21:33,010
if it's its shortest path
distances minus infinite,
323
00:21:33,010 --> 00:21:35,470
then it must be
reachable from a witness.
324
00:21:35,470 --> 00:21:36,400
So that's the claim.
325
00:21:41,010 --> 00:21:52,460
If delta S, V is
minus infinity, then V
326
00:21:52,460 --> 00:22:02,560
is reachable from a witness.
327
00:22:02,560 --> 00:22:05,320
Reachable from a vertex that
has this property-- that
328
00:22:05,320 --> 00:22:07,370
has this property.
329
00:22:07,370 --> 00:22:09,100
And if it's reachable
from something
330
00:22:09,100 --> 00:22:13,000
that has minus infinity
shortest pathway,
331
00:22:13,000 --> 00:22:16,940
then I can take that path
go to my reachable vertex,
332
00:22:16,940 --> 00:22:20,470
and that's also
minus infinite path.
333
00:22:20,470 --> 00:22:21,040
OK.
334
00:22:21,040 --> 00:22:22,180
So how do we prove this?
335
00:22:28,320 --> 00:22:37,800
Well, let's consider--
let's I'm going
336
00:22:37,800 --> 00:22:42,170
to state a somewhat stronger
statement that we'll
337
00:22:42,170 --> 00:22:43,940
prove instead.
338
00:22:43,940 --> 00:22:47,420
It suffices to prove that
every negative weight
339
00:22:47,420 --> 00:22:51,210
cycle contains a witness.
340
00:22:51,210 --> 00:22:55,860
If we are to prove
that, then every vertex
341
00:22:55,860 --> 00:23:05,510
with this property, every
vertex with this property
342
00:23:05,510 --> 00:23:10,840
is reachable from a negative
weight cycle by definition.
343
00:23:10,840 --> 00:23:15,400
So, if we can prove that every--
344
00:23:18,920 --> 00:23:36,510
prove every negative weight
cycle contains witness.
345
00:23:39,620 --> 00:23:42,040
If we can prove that every
negative weight cycle contains
346
00:23:42,040 --> 00:23:47,800
a witness, then every vertex
reachable from one of those
347
00:23:47,800 --> 00:23:50,050
witnesses-- in particular,
reachable from the negative
348
00:23:50,050 --> 00:23:50,650
weight cycle--
349
00:23:53,850 --> 00:23:59,897
has shortest distance
minus infinity,
350
00:23:59,897 --> 00:24:01,230
and that should prove the claim.
351
00:24:04,270 --> 00:24:06,900
This thing has to be reachable
from a negative weight cycle.
352
00:24:12,700 --> 00:24:15,420
And so if we prove negative
weight cycles contain
353
00:24:15,420 --> 00:24:19,232
witnesses, then all
of these vertices
354
00:24:19,232 --> 00:24:20,440
are reachable from a witness.
355
00:24:20,440 --> 00:24:22,420
OK, great, great.
356
00:24:22,420 --> 00:24:24,230
Confusing myself
there for a second.
357
00:24:24,230 --> 00:24:25,680
OK.
358
00:24:25,680 --> 00:24:30,820
So let's consider a
negative weight cycle.
359
00:24:30,820 --> 00:24:32,590
NG.
360
00:24:32,590 --> 00:24:35,360
Here's a directed
negative weight cycle.
361
00:24:35,360 --> 00:24:35,860
Recall.
362
00:24:38,425 --> 00:24:42,870
This will be my
negative weight cycle C.
363
00:24:42,870 --> 00:24:45,760
All of the sum of the
edges in this thing,
364
00:24:45,760 --> 00:24:48,100
the weights has negative weight.
365
00:24:50,920 --> 00:24:52,810
And I'm going to have
a little bit notation--
366
00:24:52,810 --> 00:24:56,560
if I have a vertex
V here, I'm going
367
00:24:56,560 --> 00:25:00,220
to say that its predecessor
in the cycle, I'm
368
00:25:00,220 --> 00:25:02,962
just going to call it V prime.
369
00:25:02,962 --> 00:25:04,045
That's just some notation.
370
00:25:06,830 --> 00:25:08,970
All right.
371
00:25:08,970 --> 00:25:15,480
So, if I have computed these
shortest-path distances
372
00:25:15,480 --> 00:25:19,230
to every vertex in my graph,
shortest-path distance going
373
00:25:19,230 --> 00:25:21,780
through at most V vertices
and the shortest path
374
00:25:21,780 --> 00:25:25,000
distance going through at
most V minus 1 vertices,
375
00:25:25,000 --> 00:25:27,420
then I know the
following thing holds.
376
00:25:27,420 --> 00:25:35,600
Delta V going from S to V
for any vertex in my cycle
377
00:25:35,600 --> 00:25:41,120
can't be bigger
than delta V minus 1
378
00:25:41,120 --> 00:25:46,400
from S to U plus the weight--
379
00:25:46,400 --> 00:25:50,520
sorry, not U-- V
prime, its predecessor,
380
00:25:50,520 --> 00:25:56,100
plus the weight going from
the predecessor to my vertex.
381
00:25:56,100 --> 00:25:58,210
Why is that?
382
00:25:58,210 --> 00:25:58,850
Why is that?
383
00:25:58,850 --> 00:26:02,020
Because this is the
weight of some vertex--
384
00:26:02,020 --> 00:26:03,940
this is the weight--
385
00:26:03,940 --> 00:26:08,080
the shortest-path
distance to my predecessor
386
00:26:08,080 --> 00:26:10,370
using one fewer edge.
387
00:26:10,370 --> 00:26:12,880
And so this in
particular is the weight
388
00:26:12,880 --> 00:26:17,010
of some path that uses V edges.
389
00:26:17,010 --> 00:26:21,660
So if this is the shortest
such path distance,
390
00:26:21,660 --> 00:26:24,870
this has to upper
bound it at least--
391
00:26:24,870 --> 00:26:25,500
at most.
392
00:26:25,500 --> 00:26:25,650
Yeah?
393
00:26:25,650 --> 00:26:27,400
AUDIENCE: Is that the
triangle inequality?
394
00:26:27,400 --> 00:26:29,830
JASON KU: That is a statement
of the triangle inequality,
395
00:26:29,830 --> 00:26:32,240
thank you.
396
00:26:32,240 --> 00:26:33,050
All right.
397
00:26:33,050 --> 00:26:37,460
So, yes, this is just
by triangle inequality.
398
00:26:37,460 --> 00:26:38,720
OK.
399
00:26:38,720 --> 00:26:43,490
Now what we can say is, let's
take this equation summed
400
00:26:43,490 --> 00:26:46,700
over all vertices in my cycle.
401
00:26:46,700 --> 00:26:51,260
So I'm just going
to add summation
402
00:26:51,260 --> 00:26:57,350
here of all vertices in my
cycle of this whole thing.
403
00:26:57,350 --> 00:27:01,400
I'm going to do that
out a little bit neater.
404
00:27:01,400 --> 00:27:08,420
Summation of delta, not d.
405
00:27:08,420 --> 00:27:18,150
Delta V S, V. I guess I don't
need this open parentheses.
406
00:27:18,150 --> 00:27:26,760
Equals-- or less than or equal
to sum of V and C of delta
407
00:27:26,760 --> 00:27:30,915
V minus 1 V prime.
408
00:27:33,890 --> 00:27:38,690
And here, I'm
summing over V and C,
409
00:27:38,690 --> 00:27:42,320
and this is just my notation
for the predecessor.
410
00:27:42,320 --> 00:27:48,740
And then I'm going to sum
over the weights in my cycle
411
00:27:48,740 --> 00:27:53,680
V and C. These are the sum
of the weights in my cycle.
412
00:27:53,680 --> 00:27:57,090
Well, what do I know
about this cycle?
413
00:27:57,090 --> 00:28:01,700
This is just the weight
of C. The weight of C--
414
00:28:01,700 --> 00:28:03,800
that's awful handwriting.
415
00:28:03,800 --> 00:28:06,960
C, what do I know about
the weight of the cycle?
416
00:28:06,960 --> 00:28:09,240
It's negative.
417
00:28:09,240 --> 00:28:16,452
So, this is less than 0, which
means that if I remove this,
418
00:28:16,452 --> 00:28:18,900
this needs to be
a strict equality.
419
00:28:22,630 --> 00:28:26,740
But if the sum of all
of these is strictly
420
00:28:26,740 --> 00:28:30,110
less than the sum
of all these, we
421
00:28:30,110 --> 00:28:36,440
can't have none of the vertices
in my graph satisfying--
422
00:28:36,440 --> 00:28:38,030
not satisfying this property.
423
00:28:42,030 --> 00:28:45,390
If all of them
are not witnesses,
424
00:28:45,390 --> 00:28:49,970
then this thing is bigger
than this thing-- at least
425
00:28:49,970 --> 00:28:53,930
as big as this thing for every
vertex in my cycle, which
426
00:28:53,930 --> 00:28:56,640
is a contradiction.
427
00:28:56,640 --> 00:29:02,560
So, the claim holds, if we
have a negative infinite
428
00:29:02,560 --> 00:29:06,110
shortest-path distance, then
V is reachable from a witness.
429
00:29:06,110 --> 00:29:09,190
So it suffices for us to
find all the witnesses,
430
00:29:09,190 --> 00:29:12,970
find all the vertices
reachable from the witnesses,
431
00:29:12,970 --> 00:29:16,050
and then mark them
as minus infinity.
432
00:29:16,050 --> 00:29:18,020
Does that make sense?
433
00:29:18,020 --> 00:29:19,640
OK.
434
00:29:19,640 --> 00:29:26,610
So, now we finally are able
to get to our algorithm.
435
00:29:26,610 --> 00:29:27,633
Bellman-Ford.
436
00:29:31,710 --> 00:29:34,830
And what I'm going
to show you today
437
00:29:34,830 --> 00:29:37,020
is a little different
than what is normally
438
00:29:37,020 --> 00:29:39,000
presented as Bellman-Ford.
439
00:29:39,000 --> 00:29:42,040
The original
Bellman-Ford algorithm
440
00:29:42,040 --> 00:29:43,928
does something a
little different.
441
00:29:43,928 --> 00:29:45,970
And because it does
something a little different,
442
00:29:45,970 --> 00:29:48,130
which we'll talk
about at the end,
443
00:29:48,130 --> 00:29:50,140
it's a little
hairier to analyze.
444
00:29:50,140 --> 00:29:53,950
I'm going to show you
a modification that
445
00:29:53,950 --> 00:29:58,270
is a little easier to analyze
and has this nice property
446
00:29:58,270 --> 00:30:01,120
that we're going to be
able to use the algorithm
447
00:30:01,120 --> 00:30:04,195
to give us a negative
weight cycle if it exists.
448
00:30:06,920 --> 00:30:12,000
So, we're going to say this is
maybe a modified Bellman-Ford.
449
00:30:12,000 --> 00:30:20,000
And the idea here is to make
a vertex associate-- make
450
00:30:20,000 --> 00:30:23,660
many versions of a vertex.
451
00:30:23,660 --> 00:30:26,300
And I want this
version of the vertex
452
00:30:26,300 --> 00:30:32,360
to correspond to whether I came
here using 0 edges, 1 edge, 2
453
00:30:32,360 --> 00:30:34,100
edges, 3 edges--
454
00:30:34,100 --> 00:30:35,600
I have a different
vertex version
455
00:30:35,600 --> 00:30:39,530
of the vertex for
each one of these--
456
00:30:39,530 --> 00:30:44,660
for a path going
through, at most,
457
00:30:44,660 --> 00:30:46,380
a certain number of edges.
458
00:30:46,380 --> 00:30:46,880
OK.
459
00:30:46,880 --> 00:30:51,080
So this is an idea
called graph duplication.
460
00:30:51,080 --> 00:30:58,870
Idea, graph duplication.
461
00:30:58,870 --> 00:31:00,880
And this is a very
common technique
462
00:31:00,880 --> 00:31:04,890
for solving
graph-related problems.
463
00:31:04,890 --> 00:31:06,870
Because essentially
what I get to do is I
464
00:31:06,870 --> 00:31:09,090
get to store information.
465
00:31:11,860 --> 00:31:15,480
If I'm having different
versions of a vertex,
466
00:31:15,480 --> 00:31:18,270
I can have that
vertex correspond
467
00:31:18,270 --> 00:31:22,350
to reaching that vertex
in a different state.
468
00:31:22,350 --> 00:31:24,180
So that's what we're
going to do here.
469
00:31:24,180 --> 00:31:36,040
The idea here is make
V plus 1 levels--
470
00:31:36,040 --> 00:31:38,980
basically duplicate
vertices in our graph--
471
00:31:44,080 --> 00:32:05,480
where vertex Vk in level k
represents reaching vertex
472
00:32:05,480 --> 00:32:13,120
V using at most k edges.
473
00:32:13,120 --> 00:32:16,630
OK, so this definition
seems similar to what
474
00:32:16,630 --> 00:32:19,840
we're doing up here.
475
00:32:19,840 --> 00:32:24,070
If we have vertices that
have this property, then
476
00:32:24,070 --> 00:32:26,710
their shortest paths
in this new graph
477
00:32:26,710 --> 00:32:30,490
might correspond to
these k edge distances.
478
00:32:30,490 --> 00:32:32,200
And really, the name
of the game here
479
00:32:32,200 --> 00:32:35,080
is to compute these
two for every vertex,
480
00:32:35,080 --> 00:32:37,120
because then we can--
481
00:32:37,120 --> 00:32:42,350
then if d is finite,
delta is finite,
482
00:32:42,350 --> 00:32:45,200
then this guy will be the
length of our shortest path.
483
00:32:45,200 --> 00:32:48,190
And if they are different,
that will be a witness
484
00:32:48,190 --> 00:32:51,810
and we can explore from it.
485
00:32:51,810 --> 00:33:14,750
So-- and if we connect
edges from one level
486
00:33:14,750 --> 00:33:26,400
to only higher levels, basically
levels with a higher k, then
487
00:33:26,400 --> 00:33:27,840
this graph is going to be a DAG.
488
00:33:34,420 --> 00:33:35,800
Whoa.
489
00:33:35,800 --> 00:33:37,620
That's cool.
490
00:33:37,620 --> 00:33:38,480
Why is that cool?
491
00:33:38,480 --> 00:33:42,440
Because we saw how to solve
single-source shortest paths
492
00:33:42,440 --> 00:33:44,150
in a DAG and linear time.
493
00:33:44,150 --> 00:33:46,160
Now this graph that
we're going to construct
494
00:33:46,160 --> 00:33:49,100
is going to have
V plus 1 levels.
495
00:33:49,100 --> 00:33:55,245
So could have-- our graph
kind of explodes V times.
496
00:33:57,702 --> 00:33:59,160
We're going to do
that in a second.
497
00:33:59,160 --> 00:34:02,560
I'm going to be more precise
with what I mean there.
498
00:34:02,560 --> 00:34:07,740
But if we're multiplying
our graph V plus 1 times,
499
00:34:07,740 --> 00:34:10,380
then the size of our graph
is now V times larger.
500
00:34:10,380 --> 00:34:14,790
Now that doesn't-- that's
not so hard to believe.
501
00:34:14,790 --> 00:34:17,969
But if we made our
graph V times larger
502
00:34:17,969 --> 00:34:21,300
and we ran a shortest path
algorithm in linear time
503
00:34:21,300 --> 00:34:24,969
with respect to that
graph, then that graph
504
00:34:24,969 --> 00:34:33,670
has something like size
V times V plus E size.
505
00:34:37,810 --> 00:34:39,100
That looks familiar, maybe?
506
00:34:42,929 --> 00:34:45,389
That's this running time.
507
00:34:45,389 --> 00:34:48,210
So if we can find an algorithm
that runs in that running time,
508
00:34:48,210 --> 00:34:51,130
we can get down to V times E.
509
00:34:51,130 --> 00:34:52,159
So let's try to do that.
510
00:34:57,640 --> 00:34:59,830
Here's the transformation
I'm going to show you.
511
00:34:59,830 --> 00:35:02,830
I'm going to show you
first with an example.
512
00:35:02,830 --> 00:35:08,770
Here's an example of a directed
graph that does contain
513
00:35:08,770 --> 00:35:10,960
a negative weight cycle.
514
00:35:10,960 --> 00:35:12,090
Can anyone find it for me?
515
00:35:16,830 --> 00:35:19,440
bcd.
516
00:35:19,440 --> 00:35:21,960
Has weigh minus
4 plus 3 minus 1.
517
00:35:21,960 --> 00:35:26,700
It has a minus 2 total weight.
518
00:35:26,700 --> 00:35:28,840
So that's a negative
weight cycle.
519
00:35:28,840 --> 00:35:33,870
So in order to take
shortest paths from a,
520
00:35:33,870 --> 00:35:39,330
I will want to say at the end of
my algorithm, this better be 0,
521
00:35:39,330 --> 00:35:42,870
and all of these better
be minus infinity.
522
00:35:42,870 --> 00:35:47,170
So that's what I
want in my algorithm.
523
00:35:47,170 --> 00:35:48,700
So what's my
algorithm going to be?
524
00:35:48,700 --> 00:35:55,250
I'm going to make V plus
1 copies of this graph,
525
00:35:55,250 --> 00:35:56,930
and I'm going to kind
of stretch it out.
526
00:35:56,930 --> 00:35:57,430
OK.
527
00:35:57,430 --> 00:36:02,260
So here, I have
V 0, 1, 2, 3, 4--
528
00:36:02,260 --> 00:36:04,340
there are four
vertices in my graph.
529
00:36:04,340 --> 00:36:08,620
So this is 1, 2, 3, 4,
5 copies of my graph.
530
00:36:08,620 --> 00:36:13,600
I have a version of vertex a
for each one of those copies,
531
00:36:13,600 --> 00:36:16,390
a version of vertex b for
each of those copies, c
532
00:36:16,390 --> 00:36:18,400
and d, et cetera.
533
00:36:18,400 --> 00:36:21,730
So I have this nice
grid of vertices.
534
00:36:21,730 --> 00:36:25,930
And I'm not going to put
any edges within a layer,
535
00:36:25,930 --> 00:36:27,900
within a level.
536
00:36:27,900 --> 00:36:30,660
Because then-- I mean,
this graph has cycles.
537
00:36:30,660 --> 00:36:33,060
And I don't want
cycles in my graph.
538
00:36:33,060 --> 00:36:36,240
What I'm going to do
instead is for every edge
539
00:36:36,240 --> 00:36:40,140
in my original graph-- for
example, the edge from a to b,
540
00:36:40,140 --> 00:36:43,760
I'm going to connect it to
the b in the next level.
541
00:36:43,760 --> 00:36:49,860
So a0 is connected to b1 with
an edge weight of minus 5,
542
00:36:49,860 --> 00:36:51,723
just like in the original.
543
00:36:51,723 --> 00:36:53,890
And I'm going to do that
for every edge in my graph,
544
00:36:53,890 --> 00:36:55,807
and I'm going to repeat
that down all the way.
545
00:36:58,100 --> 00:37:03,770
In addition, I'm going to
add zero-weight edge from a0
546
00:37:03,770 --> 00:37:08,060
to a1 or from every vertex
all the way down the line.
547
00:37:08,060 --> 00:37:11,510
These are all zero-weight
edges corresponding to--
548
00:37:11,510 --> 00:37:13,800
I'm not going to
traverse an edge,
549
00:37:13,800 --> 00:37:17,360
I'm just going to
stay at this vertex.
550
00:37:17,360 --> 00:37:19,040
That's going to
allow us to simulate
551
00:37:19,040 --> 00:37:22,460
this at most k edges condition.
552
00:37:22,460 --> 00:37:27,060
Now if you take a look at
paths in this graph from a0,
553
00:37:27,060 --> 00:37:32,760
our starting
vertex, clearly none
554
00:37:32,760 --> 00:37:38,700
of the other vertices in that
level are reachable from a0,
555
00:37:38,700 --> 00:37:40,260
just as we want.
556
00:37:40,260 --> 00:37:43,260
Because the
shortest-path distance
557
00:37:43,260 --> 00:37:47,100
to any of these vertices
using at most 0 edges
558
00:37:47,100 --> 00:37:48,340
should be infinite.
559
00:37:48,340 --> 00:37:50,120
I can't get there in 0 edges.
560
00:37:53,730 --> 00:38:03,920
But then any path in this
graph using at most k edges
561
00:38:03,920 --> 00:38:09,530
is going to correspond
to a path from a0
562
00:38:09,530 --> 00:38:13,050
to a vertex in that level,
the corresponding level.
563
00:38:13,050 --> 00:38:16,610
So for example, if I had a--
564
00:38:16,610 --> 00:38:26,450
if I was looking
for paths 2b using
565
00:38:26,450 --> 00:38:34,400
at most three edges, any path--
566
00:38:34,400 --> 00:38:40,670
a path from a0 to
b3 in this graph
567
00:38:40,670 --> 00:38:43,820
would correspond to a
path in this graph that
568
00:38:43,820 --> 00:38:47,320
uses at most three edges.
569
00:38:47,320 --> 00:38:50,010
so Let's find such a path.
570
00:38:50,010 --> 00:38:58,260
So going from a0,
b1, stay at b1--
571
00:38:58,260 --> 00:39:00,090
stay at b, sorry.
572
00:39:00,090 --> 00:39:05,020
Yeah, that's a path using
fewer than three edges--
573
00:39:05,020 --> 00:39:07,000
or at most three edges.
574
00:39:07,000 --> 00:39:09,270
But there's another path here.
575
00:39:09,270 --> 00:39:11,100
Where is it?
576
00:39:11,100 --> 00:39:14,215
Going from a, a, a to b--
577
00:39:14,215 --> 00:39:15,840
OK, that's not such
an interesting one.
578
00:39:15,840 --> 00:39:17,410
That's the same path.
579
00:39:17,410 --> 00:39:20,040
So I might have more than one
path in here corresponding
580
00:39:20,040 --> 00:39:24,420
to a path in there, but my
claim is that any path in here
581
00:39:24,420 --> 00:39:28,250
corresponds to a path in here.
582
00:39:28,250 --> 00:39:31,790
So what's a path of length?
583
00:39:31,790 --> 00:39:33,110
3, that's non-trivial.
584
00:39:33,110 --> 00:39:36,560
Yeah, a to c to d to b.
585
00:39:36,560 --> 00:39:42,620
So a to c to d to b.
586
00:39:42,620 --> 00:39:44,510
Yeah, that's a path.
587
00:39:44,510 --> 00:39:47,330
And basically, because
I constructed this so
588
00:39:47,330 --> 00:39:49,670
that the edges always
moved from level to level,
589
00:39:49,670 --> 00:39:54,500
as I traverse these edges,
I always change levels.
590
00:39:54,500 --> 00:39:56,487
Yeah?
591
00:39:56,487 --> 00:39:58,320
AUDIENCE: But my original
graph doesn't have
592
00:39:58,320 --> 00:40:00,587
these self-loops with 0 weight.
593
00:40:00,587 --> 00:40:01,170
JASON KU: Yes.
594
00:40:01,170 --> 00:40:06,060
My original graph doesn't
have an edge from a to a.
595
00:40:06,060 --> 00:40:08,550
That's true.
596
00:40:08,550 --> 00:40:12,430
I'm using these edges
to correspond to--
597
00:40:12,430 --> 00:40:14,770
I'm deciding not
to take an edge.
598
00:40:19,160 --> 00:40:22,400
It's not that I'm like
doing any work here,
599
00:40:22,400 --> 00:40:25,350
I'm just staying
there for a state.
600
00:40:25,350 --> 00:40:28,640
And that's what's going to allow
me to get this at most edges.
601
00:40:28,640 --> 00:40:29,210
All right.
602
00:40:29,210 --> 00:40:31,040
So, this is the graph construct.
603
00:40:31,040 --> 00:40:37,760
Hopefully you understand
that we made these V layers.
604
00:40:37,760 --> 00:40:44,600
This is V. And a vertex--
605
00:40:44,600 --> 00:40:48,030
we made V copies of every
vertex and connected
606
00:40:48,030 --> 00:40:49,410
them using edges in this way.
607
00:40:49,410 --> 00:40:50,980
OK.
608
00:40:50,980 --> 00:40:54,190
So, first step of Bellman-Ford
is construct this graph.
609
00:40:58,980 --> 00:41:09,620
So, Bellman-Ford, construct
G prime as described above.
610
00:41:09,620 --> 00:41:13,825
It has how many vertices?
611
00:41:16,980 --> 00:41:20,130
V times V plus 1.
612
00:41:20,130 --> 00:41:22,830
V times V plus 1 vertices.
613
00:41:22,830 --> 00:41:24,450
And how many edges?
614
00:41:24,450 --> 00:41:29,820
Well, I have one edge for
outgoing edge for each vertex
615
00:41:29,820 --> 00:41:34,220
corresponding to just
staying in the same place.
616
00:41:34,220 --> 00:41:37,880
So that's V squared vertices--
617
00:41:37,880 --> 00:41:39,380
I mean edges.
618
00:41:39,380 --> 00:41:42,410
And then I have one edge--
619
00:41:42,410 --> 00:41:45,540
for every edge in
my graph, I have--
620
00:41:45,540 --> 00:41:46,040
sorry.
621
00:41:46,040 --> 00:41:48,630
I have a V minus 1--
622
00:41:48,630 --> 00:41:49,130
sorry.
623
00:41:49,130 --> 00:41:53,420
Just V. I have V edges for
every edge in my graph.
624
00:41:53,420 --> 00:41:56,090
So that means-- so this
is the number of vertices.
625
00:42:00,230 --> 00:42:11,600
And V times V plus
V times E, this
626
00:42:11,600 --> 00:42:18,710
is V V plus E. All right, cool.
627
00:42:18,710 --> 00:42:22,130
So that's how many edges I have.
628
00:42:22,130 --> 00:42:24,900
So constructed in
that way, it's a DAG.
629
00:42:24,900 --> 00:42:29,360
If we only have edges
going to increasing levels,
630
00:42:29,360 --> 00:42:31,115
then this thing
can't have cycles,
631
00:42:31,115 --> 00:42:32,990
because otherwise that
would mean there would
632
00:42:32,990 --> 00:42:34,620
be an edge pointing backwards.
633
00:42:34,620 --> 00:42:37,090
And we didn't
construct any of those.
634
00:42:37,090 --> 00:42:37,930
All right.
635
00:42:37,930 --> 00:42:40,750
So we construct
this graph G prime.
636
00:42:40,750 --> 00:42:43,620
We can do that in linear time
with respect to these things.
637
00:42:43,620 --> 00:42:46,630
I just go through all the
edges, I make these edges,
638
00:42:46,630 --> 00:42:47,860
and I make these vertices.
639
00:42:47,860 --> 00:42:49,630
It doesn't take anything--
640
00:42:49,630 --> 00:42:51,130
I just do it naively.
641
00:42:51,130 --> 00:42:59,270
Right I can do that in time V
times V plus E asymptotically.
642
00:42:59,270 --> 00:43:01,160
OK.
643
00:43:01,160 --> 00:43:08,210
Now I run DAG relaxation,
our nice algorithm
644
00:43:08,210 --> 00:43:13,625
we had last time, from--
645
00:43:16,490 --> 00:43:17,840
in there was a0.
646
00:43:17,840 --> 00:43:22,600
I'm going to say
it's S0, our source.
647
00:43:22,600 --> 00:43:24,490
Our source vertex.
648
00:43:24,490 --> 00:43:27,940
Single source shortest paths.
649
00:43:27,940 --> 00:43:44,670
So that I compute delta of
S0 to Vk for all k and--
650
00:43:44,670 --> 00:43:46,470
what is it?
651
00:43:46,470 --> 00:43:54,430
0 to V. That's what single
source shortest paths does.
652
00:43:54,430 --> 00:43:58,060
It computes for me this
distance from my source--
653
00:43:58,060 --> 00:44:01,090
at some source to every
other vertex in the graph.
654
00:44:01,090 --> 00:44:04,090
And so in particular
I get these.
655
00:44:04,090 --> 00:44:05,440
Well, that is all of them.
656
00:44:08,350 --> 00:44:21,240
Then for each vertex V, set--
657
00:44:21,240 --> 00:44:27,960
the thing I'm going to
return, d-value, S to V,
658
00:44:27,960 --> 00:44:31,160
equal to the
shortest-path distance
659
00:44:31,160 --> 00:44:34,340
I got from DAG relaxation
to a particular vertex.
660
00:44:40,640 --> 00:44:45,140
V V minus 1.
661
00:44:45,140 --> 00:44:47,780
Why am I doing this?
662
00:44:47,780 --> 00:44:51,980
I'm setting it to be the
shortest-path distance
663
00:44:51,980 --> 00:44:56,330
to the guy in the
second-to-last row here
664
00:44:56,330 --> 00:45:02,440
or column in my modified graph.
665
00:45:05,930 --> 00:45:13,390
The hope is that this
distance in my DAG
666
00:45:13,390 --> 00:45:23,020
corresponds to this distance
in my original graph.
667
00:45:23,020 --> 00:45:27,560
The distance to V using
at most V minus 1 edges.
668
00:45:27,560 --> 00:45:29,560
So that's the claim--
that's a claim we're going
669
00:45:29,560 --> 00:45:30,830
to prove in just a second.
670
00:45:30,830 --> 00:45:33,790
I'm going to write it down
just so that we have--
671
00:45:33,790 --> 00:45:36,490
just to continue our
train of thought.
672
00:45:36,490 --> 00:45:51,670
Claim, delta S0 Vk equals delta
k, the k edge distance, from S
673
00:45:51,670 --> 00:45:57,280
to V. That's what
we want to claim.
674
00:45:57,280 --> 00:45:59,530
That would then-- what
would that mean, then?
675
00:45:59,530 --> 00:46:01,930
That would mean
that I'm correctly
676
00:46:01,930 --> 00:46:04,300
setting the shortest-path
distance here
677
00:46:04,300 --> 00:46:09,060
for all vertices whose
distances finite.
678
00:46:09,060 --> 00:46:09,670
Great.
679
00:46:09,670 --> 00:46:14,160
I mean, I set values to things
where they're not finite,
680
00:46:14,160 --> 00:46:16,380
where they're minus
infinity also,
681
00:46:16,380 --> 00:46:22,150
but in particular I set the ones
correctly if they're finite.
682
00:46:22,150 --> 00:46:22,780
OK.
683
00:46:22,780 --> 00:46:24,280
So the last thing
we need to do is
684
00:46:24,280 --> 00:46:28,150
deal with these minus
infinity vertices.
685
00:46:30,690 --> 00:46:32,880
But we know how to do that.
686
00:46:32,880 --> 00:46:34,230
We just look at the witnesses.
687
00:46:34,230 --> 00:46:42,620
Because we've computed
this value for k
688
00:46:42,620 --> 00:46:46,490
equals V equals V minus 1,
and if that claim over there
689
00:46:46,490 --> 00:46:50,030
is true, then those
shortest-path distances
690
00:46:50,030 --> 00:46:55,310
are the same as these k edge
shortest-path distances.
691
00:46:55,310 --> 00:46:57,310
And we can just,
for every vertex,
692
00:46:57,310 --> 00:46:58,690
we compare these things.
693
00:46:58,690 --> 00:47:02,910
If this is satisfied,
we got a witness.
694
00:47:02,910 --> 00:47:03,490
OK.
695
00:47:03,490 --> 00:47:30,880
So for each witness U
and V where delta S0 U
696
00:47:30,880 --> 00:47:39,610
V is less than, strictly,
S0 U V minus 1--
697
00:47:39,610 --> 00:47:42,370
that's the definition
of a witness here,
698
00:47:42,370 --> 00:47:45,670
close the parentheses.
699
00:47:45,670 --> 00:47:58,390
Then for each vertex V
reachable from U set--
700
00:48:00,910 --> 00:48:05,410
sorry, d, is what we're
returning, d of S, V
701
00:48:05,410 --> 00:48:07,470
equal to minus infinity.
702
00:48:07,470 --> 00:48:09,370
That's the end of the algorithm.
703
00:48:09,370 --> 00:48:12,850
Basically I'm looking
for all the witnesses.
704
00:48:12,850 --> 00:48:16,390
For each witness, I find all of
the vertices reachable from it
705
00:48:16,390 --> 00:48:20,150
and set it to minus infinity
just as we argued before.
706
00:48:20,150 --> 00:48:20,740
OK.
707
00:48:20,740 --> 00:48:23,800
So, it remains to
prove this claim.
708
00:48:28,420 --> 00:48:31,870
How do we prove this claim?
709
00:48:31,870 --> 00:48:37,020
Well, we can induct on k.
710
00:48:37,020 --> 00:48:39,680
Is this true for k equals 0?
711
00:48:39,680 --> 00:48:40,340
Yeah.
712
00:48:40,340 --> 00:48:42,680
We kind of already
argued it over here
713
00:48:42,680 --> 00:48:45,740
when we are talking about
our initialization step
714
00:48:45,740 --> 00:48:48,560
or what DAG relaxation does.
715
00:48:48,560 --> 00:48:51,890
It'll set this to be the
shortest path from this guy
716
00:48:51,890 --> 00:48:53,570
to all these vertices.
717
00:48:53,570 --> 00:48:55,250
These aren't
reachable from here,
718
00:48:55,250 --> 00:48:57,240
and so these are infinite.
719
00:48:57,240 --> 00:49:00,370
And that one's 0.
720
00:49:00,370 --> 00:49:03,290
So the base case--
721
00:49:03,290 --> 00:49:14,970
so induct on k base
case, k equals 0?
722
00:49:14,970 --> 00:49:16,980
Check.
723
00:49:16,980 --> 00:49:18,120
That's all good.
724
00:49:18,120 --> 00:49:23,790
Now we-- in our inductive
step, let's take a look
725
00:49:23,790 --> 00:49:26,250
at the shortest-path
distance from 0--
726
00:49:26,250 --> 00:49:32,310
from S0 to V of k
prime for some k prime.
727
00:49:32,310 --> 00:49:35,040
And the assumption is,
the inductive hypothesis
728
00:49:35,040 --> 00:49:42,390
is that this distance is the k
edge distance for all k prime
729
00:49:42,390 --> 00:49:43,230
less than--
730
00:49:43,230 --> 00:49:47,100
I mean all k less than k prime.
731
00:49:47,100 --> 00:49:50,190
Well, kind of by definition
of a shortest path,
732
00:49:50,190 --> 00:49:55,650
this is the minimum
overall incoming vertices
733
00:49:55,650 --> 00:50:05,300
of the shortest path from
S0 to U of k prime minus 1
734
00:50:05,300 --> 00:50:10,850
plus the weight of the
edge from U of k prime
735
00:50:10,850 --> 00:50:21,140
minus to Vk prime for
all Uk prime minus 1
736
00:50:21,140 --> 00:50:29,040
in the adjacencies, the incoming
adjacencies of Vk prime.
737
00:50:29,040 --> 00:50:31,110
OK, what does this mean?
738
00:50:31,110 --> 00:50:34,840
I'm just saying in
my graph G prime,
739
00:50:34,840 --> 00:50:37,870
a shortest path to
this vertex needs
740
00:50:37,870 --> 00:50:43,040
to go first through some
vertex in the layer before it,
741
00:50:43,040 --> 00:50:44,110
which is one of these.
742
00:50:44,110 --> 00:50:45,730
And in particular,
I'm only connected
743
00:50:45,730 --> 00:50:49,700
to things adjacent to me.
744
00:50:49,700 --> 00:50:51,050
That's all this is saying.
745
00:50:51,050 --> 00:50:53,720
I have to go through that
vertex and take some shortest
746
00:50:53,720 --> 00:50:57,610
path to one of those
previous vertices.
747
00:50:57,610 --> 00:51:03,460
Now s actuality, these
adjacencies, I constructed them
748
00:51:03,460 --> 00:51:07,540
to be similar to the
adjacencies in my original graph
749
00:51:07,540 --> 00:51:11,440
in addition to one edge coming
from my original vertex,
750
00:51:11,440 --> 00:51:17,710
from vertex V. So this is the
same as the minimum of this set
751
00:51:17,710 --> 00:51:19,165
delta S0.
752
00:51:22,210 --> 00:51:22,960
Same thing.
753
00:51:26,730 --> 00:51:43,130
Plus W U, V for all U in the
adjacent-- incoming adjacencies
754
00:51:43,130 --> 00:51:46,460
of my original vertex.
755
00:51:46,460 --> 00:51:47,890
In addition to one more term.
756
00:51:47,890 --> 00:51:48,890
What is that other term?
757
00:51:54,410 --> 00:51:56,200
These are all of the
things corresponding
758
00:51:56,200 --> 00:51:59,200
to my incoming edges
in my original thing,
759
00:51:59,200 --> 00:52:03,360
but I also have that one edge
coming from the V before it.
760
00:52:03,360 --> 00:52:05,590
So this is-- I'm
going to union--
761
00:52:05,590 --> 00:52:16,600
union-- union this with delta
S0 V of k prime minus 1.
762
00:52:16,600 --> 00:52:18,130
Awful.
763
00:52:18,130 --> 00:52:20,520
I think there's
another one here.
764
00:52:20,520 --> 00:52:25,170
This is S0 of k prime minus 1.
765
00:52:25,170 --> 00:52:27,250
I"m not going to rewrite it.
766
00:52:27,250 --> 00:52:27,750
OK.
767
00:52:34,110 --> 00:52:38,040
Then by induction, this
thing and this thing
768
00:52:38,040 --> 00:52:45,190
must be the edge shortest
paths using k minus 1 vertices.
769
00:52:45,190 --> 00:52:49,330
And then that's just the
statement of what the shortest
770
00:52:49,330 --> 00:52:57,730
path should be using at
most k prime edges going
771
00:52:57,730 --> 00:53:06,000
from S to V. So, these things
are the same as we claimed.
772
00:53:06,000 --> 00:53:08,650
Yay, check.
773
00:53:08,650 --> 00:53:09,220
All right.
774
00:53:09,220 --> 00:53:11,300
And then it's not such--
775
00:53:11,300 --> 00:53:13,030
it's kind of a
trivial leap, then,
776
00:53:13,030 --> 00:53:19,880
to say that at the end of
Bellman-Ford, these guys--
777
00:53:19,880 --> 00:53:23,390
sorry, the things that
we return, these guys,
778
00:53:23,390 --> 00:53:26,360
are the shortest-path
distances, because here,
779
00:53:26,360 --> 00:53:30,530
if they're finite, we set them
to their true shortest-path
780
00:53:30,530 --> 00:53:34,980
distance; and if
they're minus infinity,
781
00:53:34,980 --> 00:53:40,050
that invariant means that these
things correspond to exactly
782
00:53:40,050 --> 00:53:41,340
this claim over here.
783
00:53:44,830 --> 00:53:45,565
It's a witness.
784
00:53:49,060 --> 00:53:51,430
And then finding all
the vertices reachable
785
00:53:51,430 --> 00:53:56,440
from those witnesses, we
set all of the infinite ones
786
00:53:56,440 --> 00:53:58,840
to be minus infinity as desired.
787
00:53:58,840 --> 00:54:02,610
OK, so what's the running
time of this thing?
788
00:54:02,610 --> 00:54:04,720
Well, we had to
construct this graph,
789
00:54:04,720 --> 00:54:06,930
so we had to take that time.
790
00:54:06,930 --> 00:54:10,770
We ran DAG relaxation, that
takes the same amount of time.
791
00:54:10,770 --> 00:54:14,400
For every vertex, we
did order V at work.
792
00:54:14,400 --> 00:54:17,370
And then for each witness,
how many could there be?
793
00:54:17,370 --> 00:54:21,330
And most, V. Checking the
reachability of each vertex,
794
00:54:21,330 --> 00:54:24,300
that can be done in how long?
795
00:54:24,300 --> 00:54:27,420
Order E time.
796
00:54:27,420 --> 00:54:30,750
Because we don't need to
consider the things that
797
00:54:30,750 --> 00:54:32,640
aren't connected to S--
798
00:54:32,640 --> 00:54:35,290
or aren't connected
to the witness.
799
00:54:35,290 --> 00:54:41,430
So this thing takes
order V times E work.
800
00:54:41,430 --> 00:54:44,970
So we're upper-bounded
by this time
801
00:54:44,970 --> 00:54:47,350
it took to construct
the original graph
802
00:54:47,350 --> 00:54:53,000
and by the claim we had before,
that takes V times E time.
803
00:54:53,000 --> 00:54:53,930
OK.
804
00:54:53,930 --> 00:54:56,381
So that's Bellman-Ford.
805
00:54:56,381 --> 00:54:59,415
I'm just going to leave
you with two nuggets.
806
00:55:03,910 --> 00:55:12,150
First, the shortest path,
if for any witness--
807
00:55:12,150 --> 00:55:13,530
let's say we have
a witness here.
808
00:55:13,530 --> 00:55:16,020
Do I have any witnesses here?
809
00:55:16,020 --> 00:55:18,920
I didn't fill in all these.
810
00:55:18,920 --> 00:55:23,060
But is there a
vertex on this cycle
811
00:55:23,060 --> 00:55:27,830
that goes through who
has the shortest path?
812
00:55:27,830 --> 00:55:32,000
That goes through four vertices
that's smaller than any other.
813
00:55:32,000 --> 00:55:32,570
OK.
814
00:55:32,570 --> 00:55:37,850
I can go from a to c
to b to d to b to c.
815
00:55:40,930 --> 00:55:43,030
And you can work
out this algorithm--
816
00:55:43,030 --> 00:55:45,580
I have it in the notes that
you can take a look at.
817
00:55:45,580 --> 00:55:50,450
This will actually have a
shorter path for vertex--
818
00:55:50,450 --> 00:55:51,160
sorry.
819
00:55:51,160 --> 00:55:55,070
It'll have a shorter
path for vertex b.
820
00:56:01,010 --> 00:56:04,130
a to b to c to d to b.
821
00:56:04,130 --> 00:56:04,670
Thank you.
822
00:56:04,670 --> 00:56:09,000
That's a path of length
4, of four edges.
823
00:56:09,000 --> 00:56:13,640
That has shorter path than
any path that has fewer edges.
824
00:56:13,640 --> 00:56:15,390
In particular, there's
only one other path
825
00:56:15,390 --> 00:56:17,970
to b using fewer than four--
826
00:56:17,970 --> 00:56:20,100
there's two other paths.
827
00:56:20,100 --> 00:56:21,900
One path of length--
828
00:56:21,900 --> 00:56:24,390
that has one edge, that
has weight minus 5,
829
00:56:24,390 --> 00:56:29,580
and one path-- this path that
has weight 9 minus 1 is 8.
830
00:56:29,580 --> 00:56:34,590
Whereas this path, minus
5, minus 4, 3, minus 1
831
00:56:34,590 --> 00:56:41,610
has minus 10 plus 3 is minus 7,
which is shorter than minus 5.
832
00:56:41,610 --> 00:56:45,060
So b and d is a witness.
833
00:56:45,060 --> 00:56:48,660
And if we actually take
a look at that path
834
00:56:48,660 --> 00:57:03,400
through this graph, going from
a to b to c to d back to b
835
00:57:03,400 --> 00:57:08,250
we see that there's a negative
weight cycle in this graph.
836
00:57:08,250 --> 00:57:10,960
b to c to d to b.
837
00:57:10,960 --> 00:57:15,600
And indeed, that's always
the case for our witnesses.
838
00:57:15,600 --> 00:57:17,820
You can see a proof
of that in the notes,
839
00:57:17,820 --> 00:57:22,690
and you can see in recitation
a little space optimization
840
00:57:22,690 --> 00:57:26,770
to make us not have to construct
this entire graph on the fly,
841
00:57:26,770 --> 00:57:30,290
but actually only use order
V space while they're going.
842
00:57:30,290 --> 00:57:30,790
OK.
843
00:57:30,790 --> 00:57:32,260
So that's Bellman-Ford.
844
00:57:32,260 --> 00:57:34,800
Sorry for running a little late.