1
00:00:07,000 --> 00:00:10,000
We're going to talk about
shortest paths,
2
00:00:10,000 --> 00:00:14,000
and we're going to talk about
shortest paths for three
3
00:00:14,000 --> 00:00:17,000
lectures.
So, this is a trilogy.
4
00:00:17,000 --> 00:00:20,000
Today will be Shortest Paths
One.
5
00:00:20,000 --> 00:00:25,000
I've been watching far too many
versions of Star Wars this
6
00:00:25,000 --> 00:00:28,000
weekend.
I saw the musical yesterday,
7
00:00:28,000 --> 00:00:31,000
matinee.
That was an MIT musical.
8
00:00:31,000 --> 00:00:35,000
That was fun,
of all three movies in about
9
00:00:35,000 --> 00:00:38,000
four hours.
That was a bit long and then I
10
00:00:38,000 --> 00:00:42,000
saw the one-man show on Friday.
One-man Star Wars:
11
00:00:42,000 --> 00:00:45,000
the original three movies in
one hour.
12
00:00:45,000 --> 00:00:48,000
That was the opposite of too
long.
13
00:00:48,000 --> 00:00:51,000
Both were fun.
So I get my trilogy fix.
14
00:00:51,000 --> 00:00:54,000
All episodes,
first we're going to start with
15
00:00:54,000 --> 00:00:58,000
The New Hope,
and we're going to talk about
16
00:00:58,000 --> 00:01:02,000
the shortest paths problem and
solve one particular problem of
17
00:01:02,000 --> 00:01:09,000
it, a very interesting version.
And then we're going to look at
18
00:01:09,000 --> 00:01:12,000
increasingly more general
versions as we go on.
19
00:01:12,000 --> 00:01:15,000
Shortest paths are sort of an
application of dynamic
20
00:01:15,000 --> 00:01:19,000
programming, which we saw last
week, and greedy algorithms,
21
00:01:19,000 --> 00:01:23,000
which we also saw last week.
So, were going to build that
22
00:01:23,000 --> 00:01:27,000
and get some pretty interesting
algorithms for an important
23
00:01:27,000 --> 00:01:30,000
problem, which is how to get
from Alderon to,
24
00:01:30,000 --> 00:01:33,000
I don't know,
Cambridge as quickly as
25
00:01:33,000 --> 00:01:37,000
possible, OK,
when you live in a graph.
26
00:01:37,000 --> 00:01:41,000
So, there's geometric shortest
paths which is a little bit
27
00:01:41,000 --> 00:01:44,000
harder.
Here, we're just going to look
28
00:01:44,000 --> 00:01:48,000
at shortest paths in graphs.
Now, hopefully you all know
29
00:01:48,000 --> 00:01:52,000
what a path in a graph is.
But, so, very quick review in
30
00:01:52,000 --> 00:01:56,000
particular because we're going
to be looking at weighted
31
00:01:56,000 --> 00:01:59,000
graphs.
So, the usual setup:
32
00:01:59,000 --> 00:02:03,000
suppose we have directed graph,
G, have some vertices,
33
00:02:03,000 --> 00:02:07,000
some edges.
We have edge weights,
34
00:02:07,000 --> 00:02:12,000
make it a little more
interesting.
35
00:02:12,000 --> 00:02:18,000
So, this is just a real number
on each edge.
36
00:02:18,000 --> 00:02:25,000
So, edge weights are usually
given by function,
37
00:02:25,000 --> 00:02:27,000
w.
For every edge,
38
00:02:27,000 --> 00:02:32,000
you get a real number.
39
00:02:40,000 --> 00:02:43,000
And then, if we look at the
paths in the graph,
40
00:02:43,000 --> 00:02:47,000
so we're going to use some
simple notation for paths called
41
00:02:47,000 --> 00:02:51,000
a path, p, starts at some
vertex, and it goes to some
42
00:02:51,000 --> 00:02:52,000
other vertex,
and so on.
43
00:02:52,000 --> 00:02:56,000
Say the last vertex is v_k,
and each of these should be a
44
00:02:56,000 --> 00:03:00,000
directed edge in the digraph.
So, this is a directed path.
45
00:03:00,000 --> 00:03:04,000
It has to respect edges in
here.
46
00:03:04,000 --> 00:03:10,000
And, we'll say that the weight
of such a path is just the sum
47
00:03:10,000 --> 00:03:14,000
of the weights of the edges
along the path.
48
00:03:14,000 --> 00:03:18,000
And, we'll call that w(p).
This is sum,
49
00:03:18,000 --> 00:03:23,000
i equals one to k minus one of
w(v_i, v_(i+1)) plus one.
50
00:03:23,000 --> 00:03:27,000
OK, so just to rub it in,
and in particular,
51
00:03:27,000 --> 00:03:32,000
how general this can be,
we have some path,
52
00:03:32,000 --> 00:03:37,000
it starts at some vertex,
there's some edge weights along
53
00:03:37,000 --> 00:03:42,000
the way.
This is some arbitrary path in
54
00:03:42,000 --> 00:03:46,000
the graph, in some hypothetical
graph.
55
00:03:58,000 --> 00:04:01,000
OK, this is mainly to point out
that some of the edge weights
56
00:04:01,000 --> 00:04:04,000
could be negative.
Some of them could be zero.
57
00:04:04,000 --> 00:04:08,000
This sum here is minus two.
So, the weight of this path is
58
00:04:08,000 --> 00:04:09,000
minus two.
And, presumably,
59
00:04:09,000 --> 00:04:12,000
the graph is much bigger than
this.
60
00:04:12,000 --> 00:04:14,000
This is just one path in the
graph.
61
00:04:14,000 --> 00:04:18,000
We're usually thinking about
simple paths that can't repeat a
62
00:04:18,000 --> 00:04:20,000
vertex.
But, sometimes we allow that.
63
00:04:20,000 --> 00:04:23,000
And then, what we care about is
the shortest path,
64
00:04:23,000 --> 00:04:26,000
or a shortest path.
Again, this may not be unique,
65
00:04:26,000 --> 00:04:31,000
but we'll still usually call it
the shortest path.
66
00:04:31,000 --> 00:04:36,000
So, we want the shortest path
from some A to some B.
67
00:04:36,000 --> 00:04:39,000
Or, we'll call the vertices u
and v.
68
00:04:39,000 --> 00:04:45,000
And we want this to be some
path of minimum possible weight,
69
00:04:45,000 --> 00:04:49,000
subject to starting at u,
and going to v.
70
00:04:49,000 --> 00:04:53,000
OK, so that's what we're
looking for.
71
00:04:53,000 --> 00:04:58,000
In general, give you a vertex,
u, give you a vertex,
72
00:04:58,000 --> 00:05:04,000
v, find a shortest path as
quickly as possible.
73
00:05:04,000 --> 00:05:06,000
What's a good algorithm for
that?
74
00:05:06,000 --> 00:05:10,000
That's the topic for the next
three lectures.
75
00:05:10,000 --> 00:05:15,000
We'll usually think about a
slightly simpler problem,
76
00:05:15,000 --> 00:05:19,000
which is just computing the
weight of that path,
77
00:05:19,000 --> 00:05:24,000
which is essentially computing
the distance from A to B.
78
00:05:24,000 --> 00:05:28,000
So, we'll call this the
shortest path weight from u to
79
00:05:28,000 --> 00:05:33,000
v.
And, we'll denote it by delta
80
00:05:33,000 --> 00:05:38,000
of (u,v), delta .
So, I mean, it's the weight of
81
00:05:38,000 --> 00:05:43,000
the shortest path,
or a weight of every shortest
82
00:05:43,000 --> 00:05:45,000
path.
Or, in other words,
83
00:05:45,000 --> 00:05:51,000
it's the Min over the weight of
each path from u to v.
84
00:05:51,000 --> 00:05:56,000
So, p here is a path.
OK, so you just consider,
85
00:05:56,000 --> 00:06:02,000
there could be a lot of
different paths.
86
00:06:02,000 --> 00:06:04,000
There could,
in principle,
87
00:06:04,000 --> 00:06:09,000
be infinitely many,
if you're allowed to repeat
88
00:06:09,000 --> 00:06:13,000
vertices.
You look at all those paths
89
00:06:13,000 --> 00:06:17,000
hypothetically.
You take the minimum weight.
90
00:06:17,000 --> 00:06:19,000
Question?
Good.
91
00:06:19,000 --> 00:06:25,000
My next question was going to
be, when do shortest paths not
92
00:06:25,000 --> 00:06:28,000
exist?
And you've hit upon one
93
00:06:28,000 --> 00:06:36,000
version, which is when you have
negative edge weights.
94
00:06:36,000 --> 00:06:40,000
So, in principle,
when you have negative edge
95
00:06:40,000 --> 00:06:45,000
weights, some shortest paths may
not exist in the sense that
96
00:06:45,000 --> 00:06:50,000
there is no shortest paths.
There are no shortest paths.
97
00:06:50,000 --> 00:06:54,000
There is no shortest path from
u to v.
98
00:06:54,000 --> 00:06:58,000
OK, in particular,
if I have two vertices,
99
00:06:58,000 --> 00:07:02,000
u and v, and I want the
shortest path between them,
100
00:07:02,000 --> 00:07:06,000
and I have negative edge
weights, well,
101
00:07:06,000 --> 00:07:12,000
this is fine.
I mean, I can still compute the
102
00:07:12,000 --> 00:07:17,000
weight of a path that has
negative weights.
103
00:07:17,000 --> 00:07:24,000
But when specifically won't I
have a single shortest path from
104
00:07:24,000 --> 00:07:26,000
u to v?
So, go ahead.
105
00:07:26,000 --> 00:07:29,000
Good.
So, if I can find the cycle
106
00:07:29,000 --> 00:07:34,000
somewhere along here whose total
weight, say, the sum of all the
107
00:07:34,000 --> 00:07:37,000
weights of these images is
negative, then I get there,
108
00:07:37,000 --> 00:07:40,000
I go around as many times as I
want.
109
00:07:40,000 --> 00:07:44,000
I keep decreasing the weight
because the weight is negative.
110
00:07:44,000 --> 00:07:48,000
I decrease it by some fixed
amount, and then I can go to v.
111
00:07:48,000 --> 00:07:52,000
So, as long as there is a
negative weights cycle reachable
112
00:07:52,000 --> 00:07:56,000
from u that can also reach v,
then there's no shortest path
113
00:07:56,000 --> 00:08:00,000
because if I take any particular
path, I can make it shorter by
114
00:08:00,000 --> 00:08:04,000
going around a couple more
times.
115
00:08:04,000 --> 00:08:07,000
So, in some sense,
this is not really a minimum.
116
00:08:07,000 --> 00:08:11,000
It's more like an infimum for
those who like to get fancy
117
00:08:11,000 --> 00:08:14,000
about such things.
But we'll just say that delta
118
00:08:14,000 --> 00:08:16,000
of (u,v) is minus infinity in
this case.
119
00:08:16,000 --> 00:08:19,000
There's a negative weights
cycle from u to v.
120
00:08:19,000 --> 00:08:23,000
So, that's one case we have to
worry about in some sense.
121
00:08:23,000 --> 00:08:27,000
But, as long as there are no
negative weight cycles,
122
00:08:27,000 --> 00:08:30,000
delta of (u,v) will be
something bigger than minus
123
00:08:30,000 --> 00:08:34,000
infinity, bounded below by some
finite value even if you could
124
00:08:34,000 --> 00:08:38,000
have negative weights,
but still no negative weights
125
00:08:38,000 --> 00:08:41,000
cycle for example,
there might not be any cycles
126
00:08:41,000 --> 00:08:45,000
in your graph.
So that's still interesting.
127
00:08:45,000 --> 00:08:50,000
And, I guess it's useful to
note that you can get from A to
128
00:08:50,000 --> 00:08:53,000
B in negative infinite time.
It's time travel,
129
00:08:53,000 --> 00:08:56,000
if the weights happen that
correspond to time.
130
00:08:56,000 --> 00:09:00,000
But when else might shortest
paths not exist?
131
00:09:00,000 --> 00:09:04,000
So, this is one case,
but there's another,
132
00:09:04,000 --> 00:09:07,000
simpler case.
It's not connected.
133
00:09:07,000 --> 00:09:12,000
There might not be any path
from u to v.
134
00:09:12,000 --> 00:09:17,000
This path might be empty.
There may be no path from u to
135
00:09:17,000 --> 00:09:21,000
v.
Here we have to define what
136
00:09:21,000 --> 00:09:25,000
happens, and here,
we'll say it's infinity if
137
00:09:25,000 --> 00:09:32,000
there's no path from u to v.
So, there are these exceptional
138
00:09:32,000 --> 00:09:35,000
cases plus infinity and minus
infinity, which are pretty
139
00:09:35,000 --> 00:09:39,000
intuitive because it takes a
really long time to get from u
140
00:09:39,000 --> 00:09:44,000
to v if there's no path there.
You can't get there from here.
141
00:09:44,000 --> 00:09:47,000
OK, but that's the definition.
Most of the time,
142
00:09:47,000 --> 00:09:50,000
this is the case we care about,
of course.
143
00:09:50,000 --> 00:09:53,000
Usually this is a finite set.
OK, good, so that's the
144
00:09:53,000 --> 00:09:56,000
definition.
We're going to get a few basic
145
00:09:56,000 --> 00:10:00,000
structural properties about
shortest paths that will allow
146
00:10:00,000 --> 00:10:04,000
us to obtain good algorithms
finding these paths when they
147
00:10:04,000 --> 00:10:07,000
exist.
And, in particular,
148
00:10:07,000 --> 00:10:10,000
we want to use ideas from
dynamic programming.
149
00:10:10,000 --> 00:10:14,000
So, if I want to use dynamic
programming to solve shortest
150
00:10:14,000 --> 00:10:17,000
paths, what do I need to
establish?
151
00:10:17,000 --> 00:10:19,000
What's the first thing I should
check?
152
00:10:19,000 --> 00:10:23,000
You've all implemented dynamic
programming by now,
153
00:10:23,000 --> 00:10:27,000
so should make complete sense
hopefully, at least more sense
154
00:10:27,000 --> 00:10:30,000
than it did a couple of weeks
ago, last week,
155
00:10:30,000 --> 00:10:34,000
when we learned it.
Dynamic programming is
156
00:10:34,000 --> 00:10:39,000
something that grows on you.
Every year I think I understand
157
00:10:39,000 --> 00:10:42,000
it better than the previous
year.
158
00:10:42,000 --> 00:10:45,000
But, in particular,
when you learned dynamic
159
00:10:45,000 --> 00:10:50,000
programming in this class,
there is this nice key property
160
00:10:50,000 --> 00:10:52,000
that you should check.
Yeah?
161
00:10:52,000 --> 00:10:54,000
Optimal substructure:
good.
162
00:10:54,000 --> 00:10:58,000
This is the phrase you should
keep in mind.
163
00:10:58,000 --> 00:11:03,000
It's not really enough for
dynamic programming to be useful
164
00:11:03,000 --> 00:11:07,000
in an efficient way,
but it at least tells you that
165
00:11:07,000 --> 00:11:12,000
you should be able to try to
apply it.
166
00:11:12,000 --> 00:11:15,000
That's a pretty weak statement,
but it's something that you
167
00:11:15,000 --> 00:11:18,000
should check.
It's definitely pretty much a
168
00:11:18,000 --> 00:11:22,000
necessary condition for dynamic
programming to make sense.
169
00:11:22,000 --> 00:11:26,000
And so, optimal some structure
here means that if I take some
170
00:11:26,000 --> 00:11:29,000
shortest path,
and I look at a subpath of that
171
00:11:29,000 --> 00:11:32,000
shortest path,
I claimed that it too is a
172
00:11:32,000 --> 00:11:34,000
shortest path,
OK, with its respective
173
00:11:34,000 --> 00:11:39,000
endpoints; obviously not between
the same endpoints.
174
00:11:39,000 --> 00:11:43,000
But if I have some shortest
path between two endpoints,
175
00:11:43,000 --> 00:11:47,000
I take any subpath and that's
also the shortest path.
176
00:11:47,000 --> 00:11:51,000
This is one version of optimal
substructure.
177
00:11:51,000 --> 00:11:55,000
This one turns out to be true
for this setup.
178
00:11:55,000 --> 00:11:59,000
And, how should I prove an
optimal substructure property?
179
00:11:59,000 --> 00:12:04,000
Cut and paste.
Yep, that works here too.
180
00:12:04,000 --> 00:12:09,000
I mean, this isn't always true.
But it's a good technique here.
181
00:12:09,000 --> 00:12:14,000
So, we're going to think about,
and I'll do essentially a proof
182
00:12:14,000 --> 00:12:17,000
by picture here.
So, suppose you have some
183
00:12:17,000 --> 00:12:22,000
subpath of some shortest path.
So, let's say the subpath is x
184
00:12:22,000 --> 00:12:25,000
to y.
And, the path goes from u to v.
185
00:12:25,000 --> 00:12:30,000
So, we assume that (u,v) is a
shortest path.
186
00:12:30,000 --> 00:12:33,000
We want to prove that (x,y) is
a shortest path.
187
00:12:33,000 --> 00:12:36,000
Well, suppose (x,y) isn't a
shortest path.
188
00:12:36,000 --> 00:12:40,000
Then there is some shorter path
that goes from x to y.
189
00:12:40,000 --> 00:12:45,000
But, if you have some shorter
path from x to y than this one.
190
00:12:45,000 --> 00:12:49,000
Then I should just erase this
part of the shortest path from u
191
00:12:49,000 --> 00:12:52,000
to v, and replace it with this
shorter one.
192
00:12:52,000 --> 00:12:56,000
So, this is some hypothetical
shorter path.
193
00:12:56,000 --> 00:12:59,000
So, suppose this existed.
If that existed,
194
00:12:59,000 --> 00:13:02,000
then I should just cut the old
path from x to y,
195
00:13:02,000 --> 00:13:07,000
and paste in this new one from
x to y.
196
00:13:07,000 --> 00:13:10,000
It's strictly shorter.
Therefore, I get a strictly
197
00:13:10,000 --> 00:13:14,000
shorter path from u to v.
But I assumed u to v was a
198
00:13:14,000 --> 00:13:16,000
shortest path:
contradiction.
199
00:13:16,000 --> 00:13:18,000
OK, so there is no shorter
path.
200
00:13:18,000 --> 00:13:21,000
And that proves the lemma that
we have this:
201
00:13:21,000 --> 00:13:25,000
subpaths of shortest paths are
shortest paths.
202
00:13:25,000 --> 00:13:29,000
OK, this should now be a pretty
familiar proof technique.
203
00:13:29,000 --> 00:13:34,000
But, there is yet another
instance of cut and paste.
204
00:13:34,000 --> 00:13:36,000
OK, so that's a good sign for
computing shortest paths.
205
00:13:36,000 --> 00:13:39,000
I mean, in terms of dynamic
programming, we won't look
206
00:13:39,000 --> 00:13:42,000
directly at dynamic programming
here because we are going to aim
207
00:13:42,000 --> 00:13:44,000
for greedy, which is even
stronger.
208
00:13:44,000 --> 00:13:47,000
But, next Monday we'll see some
dynamic programming approaches.
209
00:13:47,000 --> 00:13:49,000
Intuitively,
there are some pretty natural
210
00:13:49,000 --> 00:13:52,000
sub-problems here.
I mean, going from u to v,
211
00:13:52,000 --> 00:13:55,000
if I want to find what is the
shortest path from u to v,
212
00:13:55,000 --> 00:13:56,000
well, that's a particular
problem.
213
00:13:56,000 --> 00:13:59,000
Maybe it involves computing
shortest paths from u to some
214
00:13:59,000 --> 00:14:01,000
intermediate point,
x, and then from x to u,
215
00:14:01,000 --> 00:14:05,000
something like that.
That feels good.
216
00:14:05,000 --> 00:14:07,000
That's like,
quadratically,
217
00:14:07,000 --> 00:14:11,000
many subproblems.
And, V^2 subproblems,
218
00:14:11,000 --> 00:14:16,000
it sounds like that would lead
to a dynamic program.
219
00:14:16,000 --> 00:14:21,000
You can make it work out;
it's just a little bit trickier
220
00:14:21,000 --> 00:14:25,000
than that.
We'll see that next Monday.
221
00:14:25,000 --> 00:14:29,000
But thinking about this
intermediate point we get
222
00:14:29,000 --> 00:14:35,000
something called the triangle
inequality.
223
00:14:35,000 --> 00:14:40,000
So, you've probably heard some
form of the triangle inequality
224
00:14:40,000 --> 00:14:42,000
before.
It holds in all sorts of
225
00:14:42,000 --> 00:14:46,000
geometric spaces,
but it also holds for shortest
226
00:14:46,000 --> 00:14:51,000
paths, which is slightly less
obvious, or more obvious,
227
00:14:51,000 --> 00:14:54,000
I guess, depending on your
inclination.
228
00:14:54,000 --> 00:14:59,000
So, if you have any triple of
vertices, the shortest path from
229
00:14:59,000 --> 00:15:03,000
u to v is, at most,
the shortest path from u to x
230
00:15:03,000 --> 00:15:08,000
plus the shortest path from x to
v.
231
00:15:08,000 --> 00:15:10,000
Of course, here I need a
shortest path weight from u to
232
00:15:10,000 --> 00:15:12,000
x, and shortest path weight from
x to v.
233
00:15:12,000 --> 00:15:15,000
So, this should be pretty
natural just from the statement,
234
00:15:15,000 --> 00:15:18,000
even more natural if you draw
the picture.
235
00:15:18,000 --> 00:15:19,000
So, we have some vertex,
u.
236
00:15:19,000 --> 00:15:22,000
I'm using wiggly lines to
denote potentially long paths as
237
00:15:22,000 --> 00:15:24,000
opposed to edges.
We have some intermediate
238
00:15:24,000 --> 00:15:27,000
point, x, and we have some
target, v, and we are
239
00:15:27,000 --> 00:15:30,000
considering these three shortest
paths.
240
00:15:30,000 --> 00:15:35,000
This is the shortest path from
u to v, or this is its weights.
241
00:15:35,000 --> 00:15:38,000
This is the shortest path from
u to x.
242
00:15:38,000 --> 00:15:42,000
And here's its weight,
and the shortest path from x to
243
00:15:42,000 --> 00:15:44,000
v.
And here's its weight.
244
00:15:44,000 --> 00:15:48,000
And, the point is,
this should be the shortest
245
00:15:48,000 --> 00:15:51,000
path or a shortest path from u
to v.
246
00:15:51,000 --> 00:15:55,000
And, in particular,
one such path is you go from u
247
00:15:55,000 --> 00:16:00,000
to x, and then you go from x to
v.
248
00:16:00,000 --> 00:16:04,000
So, I mean, this sum is just
measuring the length of this
249
00:16:04,000 --> 00:16:07,000
particular path.
Take the shortest path here;
250
00:16:07,000 --> 00:16:12,000
take the shortest path here.
And, this is supposed to be the
251
00:16:12,000 --> 00:16:15,000
Min over all paths.
So, certainly this is,
252
00:16:15,000 --> 00:16:19,000
at most, this particular path,
the sum of these two values,
253
00:16:19,000 --> 00:16:22,000
OK, another proof by picture.
Clear?
254
00:16:22,000 --> 00:16:26,000
OK, this stuff is easy.
I assume we'll get into some
255
00:16:26,000 --> 00:16:31,000
more set exciting algorithms in
particular, which is always more
256
00:16:31,000 --> 00:16:36,000
exciting.
Today, we're going to look at a
257
00:16:36,000 --> 00:16:41,000
particular version of shortest
paths called,
258
00:16:41,000 --> 00:16:46,000
or the shortest paths problem
called the single source
259
00:16:46,000 --> 00:16:51,000
shortest path problem.
OK, it's a little bit more
260
00:16:51,000 --> 00:16:56,000
general than go from A to B.
The problem is,
261
00:16:56,000 --> 00:17:03,000
you're given a source vertex,
and you want to know how to get
262
00:17:03,000 --> 00:17:09,000
from that source vertex to
everywhere else.
263
00:17:09,000 --> 00:17:12,000
So, we'll call this source
vertex s.
264
00:17:12,000 --> 00:17:15,000
And from that source,
we want to find,
265
00:17:15,000 --> 00:17:21,000
let's say, the shortest path
weights from s to everyone.
266
00:17:21,000 --> 00:17:24,000
In particular,
we'd also like to know the
267
00:17:24,000 --> 00:17:30,000
shortest paths,
but that isn't too much harder.
268
00:17:30,000 --> 00:17:33,000
So, that's delta of s,
v for all vertices,
269
00:17:33,000 --> 00:17:35,000
v.
OK, so this is actually a
270
00:17:35,000 --> 00:17:39,000
little bit harder than the
problem we started with a
271
00:17:39,000 --> 00:17:41,000
getting from Alderon to
Cambridge.
272
00:17:41,000 --> 00:17:46,000
Now, we want to get from
Alderon to the entire universe.
273
00:17:46,000 --> 00:17:50,000
OK, it turns out,
this is one of the weird things
274
00:17:50,000 --> 00:17:53,000
about shortest paths,
according to the
275
00:17:53,000 --> 00:17:57,000
state-of-the-art we know today,
it seems like the following
276
00:17:57,000 --> 00:18:02,000
statement will remain true for
all time.
277
00:18:02,000 --> 00:18:05,000
But we don't know.
The best algorithms for solving
278
00:18:05,000 --> 00:18:08,000
the A to B problem,
given s, given t,
279
00:18:08,000 --> 00:18:11,000
go from s to t,
is no easier than this problem.
280
00:18:11,000 --> 00:18:15,000
It's the best ways we know how
to solve going from A to B is to
281
00:18:15,000 --> 00:18:18,000
solve how to go from A to
everywhere else.
282
00:18:18,000 --> 00:18:22,000
So, we sort of can't help
ourselves, but to solve this
283
00:18:22,000 --> 00:18:26,000
problem it turns out.
Today, we're going to look at a
284
00:18:26,000 --> 00:18:30,000
further restriction on this
problem because this is a bit
285
00:18:30,000 --> 00:18:34,000
tricky.
Will solve it next class.
286
00:18:34,000 --> 00:18:39,000
But, today we're going to get
rid of the negative weight cycle
287
00:18:39,000 --> 00:18:42,000
issue by forbidding negative
weights.
288
00:18:42,000 --> 00:18:47,000
So, we're going to assume that
all of the edge weights are
289
00:18:47,000 --> 00:18:50,000
nonnegative, so,
for all vertices,
290
00:18:50,000 --> 00:18:53,000
u and v.
So, in particular,
291
00:18:53,000 --> 00:18:58,000
shortest paths exist,
provided paths exist.
292
00:19:12,000 --> 00:19:15,000
And, we don't have to worry
about these minus infinities.
293
00:19:15,000 --> 00:19:18,000
Delta of (u,v) is always bigger
than minus infinity.
294
00:19:18,000 --> 00:19:21,000
It still might be plus infinity
if there is no path,
295
00:19:21,000 --> 00:19:23,000
but this will make life a lot
easier.
296
00:19:23,000 --> 00:19:26,000
And the algorithm we'll cover
today really requires this
297
00:19:26,000 --> 00:19:30,000
property.
You can't get away without it.
298
00:19:30,000 --> 00:19:36,000
Next class, we'll get away
without it with a fancier and
299
00:19:36,000 --> 00:19:40,000
slower algorithm.
So, as I hinted at,
300
00:19:40,000 --> 00:19:47,000
the main idea we're going to
use for the algorithm today is
301
00:19:47,000 --> 00:19:54,000
greedy, which should be faster
than dynamic programming
302
00:19:54,000 --> 00:19:58,000
generally.
And, the tricky part will be
303
00:19:58,000 --> 00:20:06,000
proving that the greedy
algorithm actually works.
304
00:20:06,000 --> 00:20:11,000
So, I think there's pretty much
only one natural way to go
305
00:20:11,000 --> 00:20:16,000
about, well, there's one way
that works to go about greedy,
306
00:20:16,000 --> 00:20:19,000
let's say.
This may be not the obvious
307
00:20:19,000 --> 00:20:22,000
one.
So, let me give you a little
308
00:20:22,000 --> 00:20:26,000
bit of setup.
The invariant we are going to
309
00:20:26,000 --> 00:20:31,000
maintain is that at all times,
we have estimates on the
310
00:20:31,000 --> 00:20:36,000
distances from the source to
every vertex.
311
00:20:36,000 --> 00:20:39,000
When I say distance,
I mean shortest path weight.
312
00:20:39,000 --> 00:20:43,000
I'm going to use weight and
distance interchangeably here
313
00:20:43,000 --> 00:20:46,000
for more intuition.
And, in particular,
314
00:20:46,000 --> 00:20:50,000
I want to maintain the set of
vertices where those estimates
315
00:20:50,000 --> 00:20:53,000
are actually the right answer.
316
00:21:10,000 --> 00:21:13,000
OK, this is little s.
This is big S.
317
00:21:13,000 --> 00:21:18,000
So, the big S will be the set
of all vertices where I know the
318
00:21:18,000 --> 00:21:21,000
answer.
What is the shortest path
319
00:21:21,000 --> 00:21:25,000
distance from little S to that
vertex in big S?
320
00:21:25,000 --> 00:21:29,000
So, for starters,
which distance do I know?
321
00:21:29,000 --> 00:21:31,000
Sorry?
s.
322
00:21:31,000 --> 00:21:34,000
I know the shortest path
distance from s to s because if
323
00:21:34,000 --> 00:21:37,000
I assume that all of my weights
are nonnegative,
324
00:21:37,000 --> 00:21:40,000
I really can't get from s to s
any faster than not doing
325
00:21:40,000 --> 00:21:43,000
anything.
OK, if I had a negative weight
326
00:21:43,000 --> 00:21:46,000
cycle, maybe the distance from s
to s is minus infinity.
327
00:21:46,000 --> 00:21:50,000
OK, but I can't have negative
weights so there's no way I can
328
00:21:50,000 --> 00:21:53,000
get from s to s any faster than
zero time.
329
00:21:53,000 --> 00:21:56,000
There might be a longer path
that still has zero cost,
330
00:21:56,000 --> 00:22:00,000
but it can't be any better than
zero.
331
00:22:00,000 --> 00:22:02,000
So, in particular,
I know that.
332
00:22:02,000 --> 00:22:05,000
So, initially,
S is certainly an s.
333
00:22:05,000 --> 00:22:09,000
OK, and the idea is we're going
to accumulate more and more
334
00:22:09,000 --> 00:22:14,000
vertices that we know.
So, at some point we know the
335
00:22:14,000 --> 00:22:16,000
distances from some of the
vertices.
336
00:22:16,000 --> 00:22:21,000
So, we have some cloud here.
This is S, and this is
337
00:22:21,000 --> 00:22:23,000
everything else.
This is the graph,
338
00:22:23,000 --> 00:22:26,000
G.
This is the subset of the
339
00:22:26,000 --> 00:22:29,000
vertices.
And, there's some edges that go
340
00:22:29,000 --> 00:22:33,000
out from there.
And, so we have estimates on
341
00:22:33,000 --> 00:22:36,000
how to get to these vertices.
Some of them,
342
00:22:36,000 --> 00:22:39,000
we may not have even seen yet.
They may not be connected to
343
00:22:39,000 --> 00:22:42,000
this portion of S.
I mean: not directly.
344
00:22:42,000 --> 00:22:44,000
They might be connected by some
longer path.
345
00:22:44,000 --> 00:22:48,000
They might be in a completely
different connected component.
346
00:22:48,000 --> 00:22:50,000
We don't know yet.
Some of them,
347
00:22:50,000 --> 00:22:53,000
we have estimates for because
we've sort of seen how to get
348
00:22:53,000 --> 00:22:55,000
there from S.
And the idea is,
349
00:22:55,000 --> 00:22:58,000
among all of these nodes where
we have estimates,
350
00:22:58,000 --> 00:23:01,000
and on to get from little S,
which is some vertex in here,
351
00:23:01,000 --> 00:23:04,000
to these vertices,
we're going to take the one for
352
00:23:04,000 --> 00:23:10,000
which the estimate is smallest.
That's the greedy choice.
353
00:23:10,000 --> 00:23:15,000
And, we're just going to add
that vertex to S.
354
00:23:15,000 --> 00:23:18,000
So, S grows one vertex per
step.
355
00:23:18,000 --> 00:23:23,000
Each step, we're going to add
to S, the vertex.
356
00:23:23,000 --> 00:23:27,000
Of course, again,
this is not a unique,
357
00:23:27,000 --> 00:23:32,000
it's a vertex,
v, in V minus S.
358
00:23:32,000 --> 00:23:39,000
So, it's something we haven't
yet computed yet whose estimated
359
00:23:39,000 --> 00:23:47,000
distance from S is minimum.
So, we look at all the vertices
360
00:23:47,000 --> 00:23:55,000
we haven't yet added to S.
Just take the one where we have
361
00:23:55,000 --> 00:24:01,000
the estimated smallest distance.
The intuition is that that
362
00:24:01,000 --> 00:24:03,000
should be a good choice.
So, if I pick the one that's
363
00:24:03,000 --> 00:24:06,000
closest to little s among all
the ones that I've seen,
364
00:24:06,000 --> 00:24:09,000
among all the paths that I've
seen, I sort of have to buy into
365
00:24:09,000 --> 00:24:11,000
that those are good paths.
But, I mean,
366
00:24:11,000 --> 00:24:13,000
maybe there's some path I
didn't see.
367
00:24:13,000 --> 00:24:16,000
Maybe you go out to here and
then you take some other path to
368
00:24:16,000 --> 00:24:18,000
some vertex, which we've already
seen.
369
00:24:18,000 --> 00:24:20,000
OK, the worry is,
well, I'd better not say that
370
00:24:20,000 --> 00:24:23,000
that's the shortest path because
there may have been some other
371
00:24:23,000 --> 00:24:25,000
way to get there.
Right, as soon as I add
372
00:24:25,000 --> 00:24:27,000
something to S,
I declare I've solved the
373
00:24:27,000 --> 00:24:32,000
problem for that vertex.
I can't change my answer later.
374
00:24:32,000 --> 00:24:35,000
OK, the estimates can change
until they get added to S.
375
00:24:35,000 --> 00:24:39,000
So, I don't want to add this
vertex to S because I haven't
376
00:24:39,000 --> 00:24:43,000
considered this path.
Well, if all my weights are
377
00:24:43,000 --> 00:24:47,000
nonnegative, and I take the
vertex here that has the
378
00:24:47,000 --> 00:24:50,000
shortest estimate from S,
so let's suppose this one is
379
00:24:50,000 --> 00:24:54,000
the shortest one,
then this can't be a shorter
380
00:24:54,000 --> 00:24:57,000
path because the distance
estimate, at least,
381
00:24:57,000 --> 00:25:02,000
from S to that vertex is larger
from S to that vertex.
382
00:25:02,000 --> 00:25:05,000
So, no way can I make the path
longer and decrease the
383
00:25:05,000 --> 00:25:07,000
distance.
That's the intuition.
384
00:25:07,000 --> 00:25:10,000
OK, it's a little bit fuzzy
here because I don't have any
385
00:25:10,000 --> 00:25:14,000
induction hypotheses set up,
and it's going to be a lot more
386
00:25:14,000 --> 00:25:17,000
work to prove that.
But that's the intuition why
387
00:25:17,000 --> 00:25:21,000
this is the right thing to do.
OK, you have to prove something
388
00:25:21,000 --> 00:25:25,000
about the distance estimates for
that to be a proof.
389
00:25:25,000 --> 00:25:27,000
But, intuitively,
it feels good.
390
00:25:27,000 --> 00:25:32,000
It was a good starting point.
OK, and then presumably we have
391
00:25:32,000 --> 00:25:34,000
to maintain these distance
estimates.
392
00:25:34,000 --> 00:25:38,000
So, the heart of the algorithm
is updating distance estimates,
393
00:25:38,000 --> 00:25:41,000
I mean, choosing the best
vertex to add to S,
394
00:25:41,000 --> 00:25:44,000
that's one step.
Then, updating the distance
395
00:25:44,000 --> 00:25:46,000
estimates is sort of where the
work is.
396
00:25:46,000 --> 00:25:50,000
And, it turns out we'll only
need to update distance
397
00:25:50,000 --> 00:25:53,000
estimates of some of the
vertices, the ones that are
398
00:25:53,000 --> 00:25:56,000
adjacent to v.
v was the vertex we just added
399
00:25:56,000 --> 00:25:59,000
to S.
So, once we add somebody to S,
400
00:25:59,000 --> 00:26:02,000
so we grow S by a little bit,
then we look at all the new
401
00:26:02,000 --> 00:26:07,000
edges that go out of S from that
vertex.
402
00:26:07,000 --> 00:26:14,000
We update something.
That's the idea.
403
00:26:14,000 --> 00:26:24,000
So, that's the idea for how
we're going to use greedy.
404
00:26:24,000 --> 00:26:32,000
Now I'll give you the
algorithm.
405
00:26:32,000 --> 00:26:40,000
So, this is called Dijkstra's
algorithm.
406
00:26:40,000 --> 00:26:47,000
Dijkstra is a famous,
recently late,
407
00:26:47,000 --> 00:26:57,000
if that makes sense,
computer scientist from the
408
00:26:57,000 --> 00:27:04,000
Netherlands.
And, this is probably the
409
00:27:04,000 --> 00:27:12,000
algorithm he is most famous for.
So, the beginning of the
410
00:27:12,000 --> 00:27:18,000
algorithm is just some
initialization,
411
00:27:18,000 --> 00:27:24,000
not too exciting.
OK, but let me tell you what
412
00:27:24,000 --> 00:27:34,000
some of the variables mean.
OK, so d is some array indexed
413
00:27:34,000 --> 00:27:42,000
by vertices, and the idea is
that d of x is the distance
414
00:27:42,000 --> 00:27:47,000
estimate for x,
so, from S to x.
415
00:27:47,000 --> 00:27:54,000
so in particular,
it's going to equal the real
416
00:27:54,000 --> 00:28:03,000
shortest path weight from S to x
when we've added x to our set
417
00:28:03,000 --> 00:28:07,000
capital, S.
OK, so this is,
418
00:28:07,000 --> 00:28:10,000
in particular,
going to be the output to the
419
00:28:10,000 --> 00:28:11,000
algorithm.
Did you have a question?
420
00:28:11,000 --> 00:28:13,000
Or were you just stretching?
Good.
421
00:28:13,000 --> 00:28:15,000
So, in d of x,
when we are done,
422
00:28:15,000 --> 00:28:17,000
d of x is the output.
For every vertex,
423
00:28:17,000 --> 00:28:20,000
it's going to give us the
shortest path weight from S to
424
00:28:20,000 --> 00:28:21,000
that vertex.
Along the way,
425
00:28:21,000 --> 00:28:24,000
it's going to be some estimated
distance from S to that vertex.
426
00:28:24,000 --> 00:28:26,000
And, we're going to improve it
over time.
427
00:28:26,000 --> 00:28:29,000
This is an infinity.
So initially,
428
00:28:29,000 --> 00:28:32,000
we know that the distance,
we know the distance from S to
429
00:28:32,000 --> 00:28:35,000
S is zero.
So, we're going to set that to
430
00:28:35,000 --> 00:28:37,000
be our estimate.
It's going to be accurate.
431
00:28:37,000 --> 00:28:40,000
Everything else we're going to
just set to infinity because we
432
00:28:40,000 --> 00:28:43,000
may not be connected.
From the beginning,
433
00:28:43,000 --> 00:28:45,000
we don't know much.
S, initially,
434
00:28:45,000 --> 00:28:47,000
is going to be infinity.
Immediately,
435
00:28:47,000 --> 00:28:49,000
we're going to add little s to
big S.
436
00:28:49,000 --> 00:28:52,000
And then, the interesting part
here is Q, which is going to
437
00:28:52,000 --> 00:28:55,000
consist of, initially all the
vertices in the graph.
438
00:28:55,000 --> 00:29:00,000
And, it's going to not just be
a queue as the letter suggests.
439
00:29:00,000 --> 00:29:03,000
It's going to be a priority
queue.
440
00:29:03,000 --> 00:29:07,000
So, it's going to maintain,
in particular,
441
00:29:07,000 --> 00:29:12,000
the vertex that has the
smallest distance estimate.
442
00:29:12,000 --> 00:29:17,000
So, this is a priority queue.
This is really an abuse of
443
00:29:17,000 --> 00:29:23,000
notation for a data structure.
OK, so this could be a heap or
444
00:29:23,000 --> 00:29:27,000
whatever.
The vertices are keyed on d,
445
00:29:27,000 --> 00:29:32,000
our distance estimate.
So, in particular,
446
00:29:32,000 --> 00:29:35,000
S will have the,
this is going to be a Min heap.
447
00:29:35,000 --> 00:29:38,000
S will be the guy who has the
minimum.
448
00:29:38,000 --> 00:29:41,000
Everyone else has the same key
initially.
449
00:29:41,000 --> 00:29:45,000
And, we're going to repeatedly
extract the minimum element from
450
00:29:45,000 --> 00:29:50,000
this queue and do other things.
OK, so this is initialization.
451
00:29:50,000 --> 00:29:53,000
OK, I'm going to call that
initialization.
452
00:29:53,000 --> 00:29:57,000
It's a pretty simple thing.
It just takes linear time,
453
00:29:57,000 --> 00:30:02,000
nothing fancy going on.
The heart of the algorithm is
454
00:30:02,000 --> 00:30:06,000
all in six lines.
And, so this is not really a
455
00:30:06,000 --> 00:30:09,000
step.
The first step here that we
456
00:30:09,000 --> 00:30:15,000
need to do is we take the vertex
whose distance estimate is
457
00:30:15,000 --> 00:30:17,000
minimum.
So that, among all the
458
00:30:17,000 --> 00:30:21,000
vertices, not yet,
and that's currently S is
459
00:30:21,000 --> 00:30:23,000
empty.
Q has everyone.
460
00:30:23,000 --> 00:30:29,000
In general, Q will have
everyone except S.
461
00:30:29,000 --> 00:30:33,000
So, we'll take the vertex,
u, that has the minimum key in
462
00:30:33,000 --> 00:30:38,000
that priority queue.
So, extract the Min from Q.
463
00:30:48,000 --> 00:31:02,000
OK.
We're going to add a little u
464
00:31:02,000 --> 00:31:07,000
to S, claim that that is now,
I mean, that's exactly what
465
00:31:07,000 --> 00:31:11,000
we're saying here.
We add to S that vertex that
466
00:31:11,000 --> 00:31:17,000
has minimum distance estimate.
And now, we need to update the
467
00:31:17,000 --> 00:31:21,000
distances.
So, we're going to look at each
468
00:31:21,000 --> 00:31:27,000
adjacent vertex for each v in
the adjacency list for u.
469
00:31:27,000 --> 00:31:30,000
We look at a few distances.
470
00:31:47,000 --> 00:31:49,000
So that's the algorithm or more
or less.
471
00:31:49,000 --> 00:31:53,000
This is the key.
I should define it a little bit
472
00:31:53,000 --> 00:31:56,000
what's going on here.
We talked mainly about
473
00:31:56,000 --> 00:32:00,000
undirected graph last time.
Here, we're thinking about
474
00:32:00,000 --> 00:32:03,000
undirected graphs.
And, the adjacency list for u
475
00:32:03,000 --> 00:32:07,000
here is just going to mean,
give me all the vertices for
476
00:32:07,000 --> 00:32:09,000
which there is an edge from u to
v.
477
00:32:09,000 --> 00:32:11,000
So, this is the outgoing
adjacency list,
478
00:32:11,000 --> 00:32:13,000
not the incoming adjacency
list.
479
00:32:13,000 --> 00:32:16,000
Undirected graphs:
you list everything.
480
00:32:16,000 --> 00:32:19,000
Directed graphs:
here, we're only going to care
481
00:32:19,000 --> 00:32:21,000
about those ones.
So, for every edge,
482
00:32:21,000 --> 00:32:24,000
(u,v), is what this is saying,
we are going to compare the
483
00:32:24,000 --> 00:32:28,000
current estimate for v,
and this candidate estimate,
484
00:32:28,000 --> 00:32:30,000
which intuitively means you go
from s to u.
485
00:32:30,000 --> 00:32:34,000
That's d of u because we now
know that that's the right
486
00:32:34,000 --> 00:32:37,000
answer.
This, in fact,
487
00:32:37,000 --> 00:32:39,000
equals, we hope,
assuming the algorithm is
488
00:32:39,000 --> 00:32:44,000
correct, this should be the
shortest path weight from s to u
489
00:32:44,000 --> 00:32:48,000
because we just added u to S.
And whenever we add something
490
00:32:48,000 --> 00:32:50,000
to S, it should have the right
value.
491
00:32:50,000 --> 00:32:54,000
So, we could say,
well, you take the shortest
492
00:32:54,000 --> 00:32:57,000
path from S to u,
and then you follow this edge
493
00:32:57,000 --> 00:32:59,000
from u to v.
That has weight,
494
00:32:59,000 --> 00:33:02,000
w, of (u,v).
That's one possible path from S
495
00:33:02,000 --> 00:33:05,000
to v.
And, if that's a shorter path
496
00:33:05,000 --> 00:33:09,000
than the one we currently have
in our estimate,
497
00:33:09,000 --> 00:33:12,000
if this is smaller than that,
then we should update the
498
00:33:12,000 --> 00:33:16,000
estimate to be that sum because
that's a better path,
499
00:33:16,000 --> 00:33:19,000
so, add it to our database of
paths, so to speak:
500
00:33:19,000 --> 00:33:23,000
OK, very intuitive operation;
clearly should not do anything
501
00:33:23,000 --> 00:33:25,000
bad.
I mean, these should be paths
502
00:33:25,000 --> 00:33:30,000
that makes sense.
We'll prove that in a moment.
503
00:33:30,000 --> 00:33:33,000
That's the first part of
correctness, that this never
504
00:33:33,000 --> 00:33:36,000
screws up.
And then, the tricky part is to
505
00:33:36,000 --> 00:33:40,000
show that it finds all the paths
that we care about.
506
00:33:40,000 --> 00:33:43,000
This step is called a
relaxation step.
507
00:33:43,000 --> 00:33:47,000
Relaxation is always a
difficult technique to teach to
508
00:33:47,000 --> 00:33:50,000
MIT students.
It doesn't come very naturally.
509
00:33:50,000 --> 00:33:55,000
But it's very simple operation.
It comes from optimization
510
00:33:55,000 --> 00:34:00,000
terminology, programming
terminology, so to speak.
511
00:34:00,000 --> 00:34:04,000
And, does this inequality look
familiar at all especially when
512
00:34:04,000 --> 00:34:08,000
you start writing it this way?
You say, the shortest path from
513
00:34:08,000 --> 00:34:12,000
S to v and the shortest path
from S to u in some edge from u
514
00:34:12,000 --> 00:34:15,000
to v, does that look like
anything we've seen?
515
00:34:15,000 --> 00:34:18,000
In fact, it was on this board
but I just erased it.
516
00:34:18,000 --> 00:34:20,000
Triangle inequality,
yeah.
517
00:34:20,000 --> 00:34:24,000
So, this is trying to make the
triangle inequality true.
518
00:34:24,000 --> 00:34:28,000
Certainly, the shortest path
from S to v should be less than
519
00:34:28,000 --> 00:34:32,000
or equal to, not greater than.
The shortest path from S to u,
520
00:34:32,000 --> 00:34:36,000
plus whatever path from u to v,
the shortest path should be,
521
00:34:36,000 --> 00:34:40,000
at most, that.
So, this is sort of a somewhat
522
00:34:40,000 --> 00:34:42,000
more general triangle
inequality.
523
00:34:42,000 --> 00:34:44,000
And, we want to,
certainly it should be true.
524
00:34:44,000 --> 00:34:46,000
So, if it's not true,
we fix it.
525
00:34:46,000 --> 00:34:48,000
If it's greater than,
we make it equal.
526
00:34:48,000 --> 00:34:51,000
But we don't want to make it
less than because that's not
527
00:34:51,000 --> 00:34:52,000
always true.
OK, but certainly,
528
00:34:52,000 --> 00:34:54,000
it should be less than or equal
to.
529
00:34:54,000 --> 00:34:56,000
So, this is fixing the triangle
inequality.
530
00:34:56,000 --> 00:35:00,000
It's trying to make that
constraint more true.
531
00:35:00,000 --> 00:35:03,000
In optimization,
that's called relaxing the
532
00:35:03,000 --> 00:35:06,000
constraint.
OK, so we're sort of relaxing
533
00:35:06,000 --> 00:35:11,000
the triangle inequality here.
In the end, we should have all
534
00:35:11,000 --> 00:35:14,000
the shortest paths.
That's a claim.
535
00:35:14,000 --> 00:35:19,000
So: a very simple algorithm.
Let's try it out on a graph,
536
00:35:19,000 --> 00:35:23,000
and that should make it more
intuitive why it's working,
537
00:35:23,000 --> 00:35:28,000
and that the rest of the
lecture will be proving that it
538
00:35:28,000 --> 00:35:32,000
works.
Yeah, this is enough room.
539
00:35:32,000 --> 00:35:36,000
So, oh, I should mention one
other thing here.
540
00:35:36,000 --> 00:35:38,000
Sorry.
Whenever we change d of v,
541
00:35:38,000 --> 00:35:43,000
this is changing the key of v
in the priority queue.
542
00:35:43,000 --> 00:35:48,000
So, implicitly what's happening
here in this assignment,
543
00:35:48,000 --> 00:35:53,000
this is getting a bit messy,
is a decreased key operation,
544
00:35:53,000 --> 00:35:58,000
OK, which we talked briefly
about last class in the context
545
00:35:58,000 --> 00:36:05,000
of minimum spanning trees where
we were also decreasing the key.
546
00:36:05,000 --> 00:36:08,000
The point is we were changing
the key of one element industry
547
00:36:08,000 --> 00:36:11,000
like station step in the
priority queue so that if it now
548
00:36:11,000 --> 00:36:14,000
becomes the minimum,
we should extract here.
549
00:36:14,000 --> 00:36:17,000
And, we are only ever
decreasing keys because we are
550
00:36:17,000 --> 00:36:20,000
always replacing larger values
with smaller values.
551
00:36:20,000 --> 00:36:23,000
So, we'll come back to that
later when we analyze the
552
00:36:23,000 --> 00:36:25,000
running time.
But, there is some data
553
00:36:25,000 --> 00:36:28,000
structure work going on here.
Again, we are abusing notation
554
00:36:28,000 --> 00:36:33,000
a bit.
OK, so here is a graph with
555
00:36:33,000 --> 00:36:36,000
edge weights.
556
00:37:06,000 --> 00:37:10,000
OK, and I want my priority
queue over here.
557
00:37:10,000 --> 00:37:14,000
And, I'm also going to draw my
estimates.
558
00:37:14,000 --> 00:37:19,000
OK, now I don't want to cheat.
So, we're going to run the
559
00:37:19,000 --> 00:37:23,000
algorithm on this graph.
s will be A,
560
00:37:23,000 --> 00:37:30,000
and I want to know the shortest
path from A to everyone else.
561
00:37:30,000 --> 00:37:32,000
So, you can check,
OK, paths exist.
562
00:37:32,000 --> 00:37:35,000
So, hopefully everything should
end up a finite value by the
563
00:37:35,000 --> 00:37:37,000
end.
All the weights are
564
00:37:37,000 --> 00:37:39,000
nonnegative, so this algorithm
should work.
565
00:37:39,000 --> 00:37:42,000
The algorithm doesn't even need
connectivity,
566
00:37:42,000 --> 00:37:45,000
but it does mean that all the
weights are nonnegative.
567
00:37:45,000 --> 00:37:48,000
So, we run the algorithm.
For the initialization,
568
00:37:48,000 --> 00:37:51,000
we set the distance estimate
for our source to be zero
569
00:37:51,000 --> 00:37:54,000
because, in fact,
there's only one path from A to
570
00:37:54,000 --> 00:37:57,000
A, and that to do nothing,
the empty path.
571
00:37:57,000 --> 00:38:00,000
So, I'm going to put the key of
zero over here.
572
00:38:00,000 --> 00:38:03,000
And, for everyone else,
we're just going to put
573
00:38:03,000 --> 00:38:08,000
infinity because we don't know
any better at this point.
574
00:38:08,000 --> 00:38:11,000
So, I'll put keys of infinity
for everyone else.
575
00:38:11,000 --> 00:38:15,000
OK, so now you can see what the
algorithm does is extract the
576
00:38:15,000 --> 00:38:18,000
minimum from the queue.
And, given our setup,
577
00:38:18,000 --> 00:38:21,000
we'll definitely choose s,
or in this case,
578
00:38:21,000 --> 00:38:23,000
A.
So, it has a weight of zero.
579
00:38:23,000 --> 00:38:26,000
Everyone else has quite a bit
larger weight.
580
00:38:26,000 --> 00:38:30,000
OK, so we look at s,
or I'll use A here.
581
00:38:30,000 --> 00:38:32,000
So, we look at A.
We add A to our set,
582
00:38:32,000 --> 00:38:33,000
S.
So, it's now removed from the
583
00:38:33,000 --> 00:38:35,000
queue.
It will never go back in
584
00:38:35,000 --> 00:38:38,000
because we never add anything to
the queue, start with all the
585
00:38:38,000 --> 00:38:40,000
vertices, and extract,
and decrease keys.
586
00:38:40,000 --> 00:38:42,000
But we never insert.
So, A is gone.
587
00:38:42,000 --> 00:38:45,000
OK, and now I want to update
the keys of all of the other
588
00:38:45,000 --> 00:38:48,000
vertices.
And the claim is I only need to
589
00:38:48,000 --> 00:38:50,000
look at the vertices that have
edges from A.
590
00:38:50,000 --> 00:38:53,000
So, there's an edge from A to
B, and that has weight ten.
591
00:38:53,000 --> 00:38:56,000
And so, I compare:
well, is it a good idea to go
592
00:38:56,000 --> 00:38:58,000
from A to A, which costs
nothing, and then to go along
593
00:38:58,000 --> 00:39:02,000
this edge, AB,
which costs ten?
594
00:39:02,000 --> 00:39:06,000
Well, it seems like a pretty
good idea because that has a
595
00:39:06,000 --> 00:39:09,000
total weight of zero plus ten,
which is ten,
596
00:39:09,000 --> 00:39:11,000
which is much smaller than
infinity.
597
00:39:11,000 --> 00:39:15,000
So, I'm going to erase this
infinity; write ten,
598
00:39:15,000 --> 00:39:19,000
and over in the queue as well.
That's the decreased key
599
00:39:19,000 --> 00:39:22,000
operation.
So now, I know a path from A to
600
00:39:22,000 --> 00:39:22,000
B.
Good.
A to C is the only other edge.
Zero plus three is less than
601
00:39:27,000 --> 00:39:28,000
infinity, so,
cool.
602
00:39:28,000 --> 00:39:33,000
I'll put three here for C,
and C is there.
603
00:39:33,000 --> 00:39:35,000
OK, the other vertices I don't
touch.
604
00:39:35,000 --> 00:39:38,000
I'm going to rewrite them here,
but the algorithm doesn't have
605
00:39:38,000 --> 00:39:41,000
to copy them.
Those keys were already there.
606
00:39:41,000 --> 00:39:44,000
It's just touching these two.
OK, that was pretty boring.
607
00:39:44,000 --> 00:39:47,000
Now we look at our queue,
and we extract the minimum
608
00:39:47,000 --> 00:39:49,000
element.
So, A is no longer in there,
609
00:39:49,000 --> 00:39:51,000
so the minimum key here is
three.
610
00:39:51,000 --> 00:39:54,000
So, the claim is that this is a
shortest path;
611
00:39:54,000 --> 00:39:56,000
from A to C,
here is the shortest path from
612
00:39:56,000 --> 00:39:58,000
A to C.
There's no other shorter way.
613
00:39:58,000 --> 00:40:03,000
You could check that,
and we'll prove it in a moment.
614
00:40:03,000 --> 00:40:05,000
Cool, so we'll remove C from
the list.
615
00:40:05,000 --> 00:40:07,000
It's gone.
Then we look at all of the
616
00:40:07,000 --> 00:40:10,000
outgoing edges from C.
So, there's one that goes up to
617
00:40:10,000 --> 00:40:13,000
B, which has weight four,
four plus three,
618
00:40:13,000 --> 00:40:15,000
which is the shortest path
weight from A to C.
619
00:40:15,000 --> 00:40:18,000
So, going from A to C,
and C to B should cost three
620
00:40:18,000 --> 00:40:21,000
plus four, which is seven,
which is less than ten.
621
00:40:21,000 --> 00:40:24,000
So, we found an even better
path to get to B.
622
00:40:24,000 --> 00:40:27,000
It's better to go like this
than it is to go like that.
623
00:40:27,000 --> 00:40:30,000
So, we write seven for B,
and there's an outgoing edge
624
00:40:30,000 --> 00:40:35,000
from C to d which costs eight.
Three plus eight is 11.
625
00:40:35,000 --> 00:40:38,000
11 is less than infinity last
time I checked.
626
00:40:38,000 --> 00:40:41,000
So, we write 11 for d.
Then we look at E.
627
00:40:41,000 --> 00:40:45,000
We have three plus two is five,
which is less than infinity.
628
00:40:45,000 --> 00:40:48,000
So, we write five for the new
key for E.
629
00:40:48,000 --> 00:40:51,000
At this point,
we have finite shortest paths
630
00:40:51,000 --> 00:40:54,000
to everywhere,
but they may not be the best
631
00:40:54,000 --> 00:40:56,000
ones.
So, we have to keep looking.
632
00:40:56,000 --> 00:40:59,000
OK, next round of the
algorithm, we extract the
633
00:40:59,000 --> 00:41:04,000
minimum key among all these.
OK, it's not B,
634
00:41:04,000 --> 00:41:07,000
which we've seen though
probably know the answer to.
635
00:41:07,000 --> 00:41:09,000
But it's E.
E has the smallest key.
636
00:41:09,000 --> 00:41:12,000
So, we now declare this to be a
shortest path.
637
00:41:12,000 --> 00:41:16,000
The way we got to E was along
this path: A to C,
638
00:41:16,000 --> 00:41:18,000
C to E, declare that to be
shortest.
639
00:41:18,000 --> 00:41:22,000
We claim we're done with E.
But we still have to update.
640
00:41:22,000 --> 00:41:25,000
What about all the outgoing
edges from E?
641
00:41:25,000 --> 00:41:28,000
There's only one here.
It costs five plus nine,
642
00:41:28,000 --> 00:41:32,000
which is 14,
which is bigger than 11.
643
00:41:32,000 --> 00:41:34,000
So, no go.
That's not an interesting path.
644
00:41:34,000 --> 00:41:38,000
Our previous path,
which went like this at a cost
645
00:41:38,000 --> 00:41:42,000
of the 11, is better than the
one we are considering now.
646
00:41:42,000 --> 00:41:45,000
I'm drawing the whole path,
but the algorithm is only
647
00:41:45,000 --> 00:41:48,000
adding these two numbers.
OK, good.
648
00:41:48,000 --> 00:41:52,000
So, I don't change anything.
Seven, 11, and five is removed,
649
00:41:52,000 --> 00:41:55,000
or E is removed.
Our new keys are seven and 11.
650
00:41:55,000 --> 00:41:57,000
So, we take the key,
seven, here,
651
00:41:57,000 --> 00:42:01,000
which is for element B,
vertex B.
652
00:42:01,000 --> 00:42:04,000
We declare the path we
currently have in our hands from
653
00:42:04,000 --> 00:42:06,000
A to B, which happens to be this
one.
654
00:42:06,000 --> 00:42:09,000
Algorithm can't actually tell
this, by the way,
655
00:42:09,000 --> 00:42:11,000
but we're drawing it anyway.
This path, A,
656
00:42:11,000 --> 00:42:14,000
C, B, is the candidate shortest
path.
657
00:42:14,000 --> 00:42:16,000
The claim is it is indeed
shortest.
658
00:42:16,000 --> 00:42:18,000
Now, we look at all the
outgoing edges.
659
00:42:18,000 --> 00:42:22,000
There's one that goes back to C
at a cost of seven plus one,
660
00:42:22,000 --> 00:42:24,000
which is eight,
which is bigger than three,
661
00:42:24,000 --> 00:42:27,000
which is good.
We already declared C to be
662
00:42:27,000 --> 00:42:29,000
done.
But the algorithm checks this
663
00:42:29,000 --> 00:42:31,000
path and says,
oh, that's no better.
664
00:42:31,000 --> 00:42:34,000
And then we look at this other
edge from B to d.
665
00:42:34,000 --> 00:42:36,000
That costs seven plus two,
which is nine,
666
00:42:36,000 --> 00:42:41,000
which is better than 11.
So, we, in fact,
667
00:42:41,000 --> 00:42:46,000
found an even shorter path.
So, the shortest path weight,
668
00:42:46,000 --> 00:42:51,000
now, for d, is nine because
there is this path that goes A,
669
00:42:51,000 --> 00:42:55,000
C, B, d for a total cost of
three plus four plus two is
670
00:42:55,000 --> 00:42:58,000
nine.
Cool, now there's only one
671
00:42:58,000 --> 00:43:02,000
element in the queue.
We remove it.
672
00:43:02,000 --> 00:43:04,000
d: we look at the outgoing
edges.
673
00:43:04,000 --> 00:43:07,000
There's one going here which
costs nine plus seven,
674
00:43:07,000 --> 00:43:09,000
which is 16,
which is way bigger than five.
675
00:43:09,000 --> 00:43:12,000
So, we're done.
Don't do anything.
676
00:43:12,000 --> 00:43:14,000
At this point,
the queue is empty.
677
00:43:14,000 --> 00:43:18,000
And the claim is that all these
numbers that are written here,
678
00:43:18,000 --> 00:43:21,000
the final values are the
shortest path weights.
679
00:43:21,000 --> 00:43:24,000
This looks an awful lot like a
five, but it's an s.
680
00:43:24,000 --> 00:43:27,000
It has a weight of zero.
I've also drawn in here all the
681
00:43:27,000 --> 00:43:31,000
shortest paths.
And, this is not hard to do.
682
00:43:31,000 --> 00:43:34,000
We're not going to talk about
it too much in this class,
683
00:43:34,000 --> 00:43:37,000
but it's mentioned in a little
bit more detail at the end of
684
00:43:37,000 --> 00:43:40,000
the textbook.
And it's something called the
685
00:43:40,000 --> 00:43:42,000
shortest path tree.
It's just something good to
686
00:43:42,000 --> 00:43:46,000
know about if you actually want
to compute shortest paths.
687
00:43:46,000 --> 00:43:48,000
In this class,
we mainly worry about the
688
00:43:48,000 --> 00:43:50,000
weights because it's pretty much
the same problem.
689
00:43:50,000 --> 00:43:55,000
The shortest path tree is the
union of all shortest paths.
690
00:43:55,000 --> 00:43:58,000
And in particular,
if you look at each vertex in
691
00:43:58,000 --> 00:44:04,000
your graph, if you consider the
last edge into that vertex that
692
00:44:04,000 --> 00:44:08,000
was relaxed among all vertices,
u, you look at the edges,
693
00:44:08,000 --> 00:44:12,000
(u,v), say, was that last one
to relax?
694
00:44:12,000 --> 00:44:15,000
So, just look at the last edges
we relaxed here.
695
00:44:15,000 --> 00:44:20,000
You put them all together:
that's called a shortest path
696
00:44:20,000 --> 00:44:23,000
tree.
And, it has the property that
697
00:44:23,000 --> 00:44:28,000
from S to everywhere else,
there is a unique path down the
698
00:44:28,000 --> 00:44:31,000
tree.
And it's the shortest path.
699
00:44:31,000 --> 00:44:34,000
It's the shortest path that we
found.
700
00:44:34,000 --> 00:44:37,000
OK, so you actually get
shortest paths out of this
701
00:44:37,000 --> 00:44:40,000
algorithm even though it's not
explicitly described.
702
00:44:40,000 --> 00:44:44,000
All we are mainly talking about
are the shortest path weights.
703
00:44:44,000 --> 00:44:48,000
Algorithm clear at this point?
Feels like it's doing the right
704
00:44:48,000 --> 00:44:50,000
thing?
You can check all those numbers
705
00:44:50,000 --> 00:44:53,000
are the best paths.
And now we're going to prove
706
00:44:53,000 --> 00:44:55,000
that.
707
00:45:12,000 --> 00:45:15,000
So: correctness.
708
00:45:25,000 --> 00:45:31,000
So the first thing I want to
prove is that relaxation never
709
00:45:31,000 --> 00:45:35,000
makes a mistake.
If it ever sets d of v to be
710
00:45:35,000 --> 00:45:42,000
something, I want to prove that
d of v is always an upper bound
711
00:45:42,000 --> 00:45:45,000
on delta.
So, we have this variant.
712
00:45:45,000 --> 00:45:51,000
It's greater than or equal to
delta of s, v for all v.
713
00:45:51,000 --> 00:45:55,000
And, this invariant holds at
all times.
714
00:45:55,000 --> 00:45:59,000
So, after initialization,
it doesn't hold before
715
00:45:59,000 --> 00:46:06,000
initialization because d isn't
defined then.
716
00:46:06,000 --> 00:46:09,000
But if you do this
initialization where you set S
717
00:46:09,000 --> 00:46:14,000
to zero, and everyone else to
infinity, and you take any
718
00:46:14,000 --> 00:46:19,000
sequence of relaxation steps,
then this variant will hold
719
00:46:19,000 --> 00:46:22,000
after each relaxation step you
apply.
720
00:46:22,000 --> 00:46:25,000
This is actually a very general
lemma.
721
00:46:25,000 --> 00:46:29,000
It's also pretty easy to prove.
It holds not only for
722
00:46:29,000 --> 00:46:33,000
Dijkstra's algorithm,
but for a lot of other
723
00:46:33,000 --> 00:46:38,000
algorithms we'll see.
Pretty much every algorithm we
724
00:46:38,000 --> 00:46:43,000
see will involve relaxation.
And, this is saying no matter
725
00:46:43,000 --> 00:46:47,000
what relaxations you do,
you always have a reasonable
726
00:46:47,000 --> 00:46:51,000
estimate in the sense that it's
greater than or equal to the
727
00:46:51,000 --> 00:46:56,000
true shortest path weight.
So, it should be converging
728
00:46:56,000 --> 00:46:58,000
from above.
So, that's the lemma.
729
00:46:58,000 --> 00:47:02,000
Let's prove it.
Any suggestions on how we
730
00:47:02,000 --> 00:47:07,000
should prove this lemma?
What technique might we use?
731
00:47:07,000 --> 00:47:09,000
What's that?
Cut and paste?
732
00:47:09,000 --> 00:47:12,000
It would be good for optimal
substructure.
733
00:47:12,000 --> 00:47:16,000
Cut and paste:
maybe sort of what's going on
734
00:47:16,000 --> 00:47:20,000
here but not exactly.
Something a little more
735
00:47:20,000 --> 00:47:22,000
general.
It's just intuition here;
736
00:47:22,000 --> 00:47:26,000
it doesn't have to be the right
answer.
737
00:47:26,000 --> 00:47:32,000
In fact, many answers are
correct, have plausible proofs.
738
00:47:32,000 --> 00:47:34,000
Induction, yeah.
So, I'm not going to write
739
00:47:34,000 --> 00:47:36,000
induction here,
but effectively we are using
740
00:47:36,000 --> 00:47:38,000
induction.
That's the answer I was
741
00:47:38,000 --> 00:47:40,000
expecting.
So, there is sort of an
742
00:47:40,000 --> 00:47:42,000
induction already in time going
on here.
743
00:47:42,000 --> 00:47:45,000
We say after initialization it
should be true.
744
00:47:45,000 --> 00:47:47,000
That's our base case.
And then, every relaxation we
745
00:47:47,000 --> 00:47:50,000
do, it should still be true.
So, we're going to assume by
746
00:47:50,000 --> 00:47:53,000
induction that all the previous
relaxations worked,
747
00:47:53,000 --> 00:47:56,000
and then we're going to prove
that the last relaxation,
748
00:47:56,000 --> 00:47:59,000
whatever it is,
works.
749
00:47:59,000 --> 00:48:01,000
So, first let's do the base
case.
750
00:48:01,000 --> 00:48:04,000
So, this is after an
initialization,
751
00:48:04,000 --> 00:48:09,000
let's say, initially.
So, initially we have d of s
752
00:48:09,000 --> 00:48:12,000
equal to zero.
And we have d of v equal to
753
00:48:12,000 --> 00:48:17,000
infinity for all other vertices,
for all vertices,
754
00:48:17,000 --> 00:48:21,000
v, not equal to little s.
OK, now we have to check that
755
00:48:21,000 --> 00:48:26,000
this inequality holds.
Well, we have delta of s,
756
00:48:26,000 --> 00:48:28,000
s.
We've already argued that
757
00:48:28,000 --> 00:48:33,000
that's zero.
You can't get negative when
758
00:48:33,000 --> 00:48:36,000
there are only nonnegative edge
weights.
759
00:48:36,000 --> 00:48:40,000
So, that's the best.
So, certainly zero is greater
760
00:48:40,000 --> 00:48:44,000
than or equal to zero.
And, we have everything else,
761
00:48:44,000 --> 00:48:47,000
well, I mean,
delta of S, v is certainly less
762
00:48:47,000 --> 00:48:50,000
than or equal to infinity.
So this holds.
763
00:48:50,000 --> 00:48:54,000
Everything is less than or
equal to infinity.
764
00:48:54,000 --> 00:48:59,000
So: base case is done.
So, now we do an induction.
765
00:48:59,000 --> 00:49:06,000
And, I'm going to write it as a
proof by contradiction.
766
00:49:06,000 --> 00:49:12,000
So, let's say,
suppose that this fails to hold
767
00:49:12,000 --> 00:49:18,000
at some point.
So, suppose for contradiction
768
00:49:18,000 --> 00:49:26,000
that the invariant is violated.
So, we'd like to sue the
769
00:49:26,000 --> 00:49:32,000
violator and find a
contradiction.
770
00:49:32,000 --> 00:49:36,000
So, it's going to be violated.
So, let's look at the first
771
00:49:36,000 --> 00:49:39,000
violation, the first time it's
violated.
772
00:49:39,000 --> 00:49:41,000
So, this is,
essentially,
773
00:49:41,000 --> 00:49:45,000
again, a proof by induction.
So, let's say we have some
774
00:49:45,000 --> 00:49:48,000
violation, d of v is less than
delta of s, v.
775
00:49:48,000 --> 00:49:53,000
That would be bad if we somehow
got an estimate smaller than the
776
00:49:53,000 --> 00:49:56,000
shortest path.
Well, then I think about
777
00:49:56,000 --> 00:50:01,000
looking at the first violation
is we know sort of by induction
778
00:50:01,000 --> 00:50:05,000
that all other values are
correct.
779
00:50:05,000 --> 00:50:09,000
OK, d of v is the first one
where we've screwed up.
780
00:50:09,000 --> 00:50:12,000
So, the invariant holds
everywhere else.
781
00:50:12,000 --> 00:50:17,000
Well, what caused this to fail,
this invariant to be violated,
782
00:50:17,000 --> 00:50:20,000
is some relaxation,
OK, on d of v.
783
00:50:20,000 --> 00:50:24,000
So, we had some d of v,
and we replaced it with some
784
00:50:24,000 --> 00:50:30,000
other d of u plus the weight of
the edge from u to v.
785
00:50:30,000 --> 00:50:34,000
And somehow,
this made it invalid.
786
00:50:34,000 --> 00:50:39,000
So, d of v is somehow less than
that.
787
00:50:39,000 --> 00:50:46,000
We just set d of v to this.
So, this must be less than
788
00:50:46,000 --> 00:50:51,000
delta of s, v.
The claim is that that's not
789
00:50:51,000 --> 00:50:59,000
possible because,
let me rewrite a little bit.
790
00:50:59,000 --> 00:51:07,000
We have d of u plus w of (u,v).
And, we have our induction
791
00:51:07,000 --> 00:51:15,000
hypothesis, which holds on u,
u of some other vertex.
792
00:51:15,000 --> 00:51:22,000
We know that d of u is at least
delta of s, u.
793
00:51:22,000 --> 00:51:30,000
So, this has to be at least
delta of s, u plus w of u,
794
00:51:30,000 --> 00:51:34,000
v.
Now, what about this w of u,
795
00:51:34,000 --> 00:51:37,000
v?
Well, that's some path from u
796
00:51:37,000 --> 00:51:40,000
to v.
So, it's got to be bigger than
797
00:51:40,000 --> 00:51:44,000
the shortest path or equal.
So certainly,
798
00:51:44,000 --> 00:51:48,000
this is greater than or equal
to delta of u,
799
00:51:48,000 --> 00:51:50,000
v.
OK, it could be larger if
800
00:51:50,000 --> 00:51:56,000
there's some multi-edged path
that has a smaller total weight,
801
00:51:56,000 --> 00:52:00,000
but it's certainly no smaller
than delta of u,
802
00:52:00,000 --> 00:52:04,000
v.
And, this looks like a good
803
00:52:04,000 --> 00:52:08,000
summation, delta of S to u,
and u to v is a triangle
804
00:52:08,000 --> 00:52:11,000
inequality, yeah.
So, that is,
805
00:52:11,000 --> 00:52:15,000
it's upside down here.
But, the triangle S,
806
00:52:15,000 --> 00:52:19,000
u, u to v, so this is only
longer than S to v.
807
00:52:19,000 --> 00:52:24,000
OK, so we have this thing,
which is simultaneously greater
808
00:52:24,000 --> 00:52:29,000
than or equal to the shortest
path weight from S to v,
809
00:52:29,000 --> 00:52:34,000
and also strictly less than the
shortest path weight from S to
810
00:52:34,000 --> 00:52:37,000
v.
So, that's a contradiction.
811
00:52:37,000 --> 00:52:41,000
Maybe contradiction is the most
intuitive way isn't the most
812
00:52:41,000 --> 00:52:43,000
intuitive way to proceed.
The intuition,
813
00:52:43,000 --> 00:52:47,000
here, is whatever you assign d
of v, you have a path in mind.
814
00:52:47,000 --> 00:52:50,000
You inductively had a path from
s to u.
815
00:52:50,000 --> 00:52:53,000
Then you added this edge.
So, that was a real path.
816
00:52:53,000 --> 00:52:57,000
We always know that every path
has weight greater than or equal
817
00:52:57,000 --> 00:53:00,000
to the shortest path.
So, it should be true,
818
00:53:00,000 --> 00:53:06,000
and here's the inductive proof.
All right, moving right along,
819
00:53:06,000 --> 00:53:14,000
so this was an easy warm-up.
We have greater than or equal
820
00:53:14,000 --> 00:53:18,000
to.
Now we have to prove less than
821
00:53:18,000 --> 00:53:23,000
or equal to at the end of the
algorithm.
822
00:53:23,000 --> 00:53:31,000
This is true all the time;
less than or equal to will only
823
00:53:31,000 --> 00:53:37,000
be true at the end.
So, we are not going to prove
824
00:53:37,000 --> 00:53:41,000
less than or equal to quite yet.
We're going to prove another
825
00:53:41,000 --> 00:53:44,000
lemma, which again,
so both of these lemmas are
826
00:53:44,000 --> 00:53:46,000
useful for other algorithms,
too.
827
00:53:46,000 --> 00:53:51,000
So, we're sort of building some
shortest path theory that we can
828
00:53:51,000 --> 00:53:54,000
apply later.
This one will give you some
829
00:53:54,000 --> 00:53:58,000
intuition about why relaxation,
not only is it not bad,
830
00:53:58,000 --> 00:54:01,000
it's actually good.
Not only does it not screw up
831
00:54:01,000 --> 00:54:07,000
anything, but it also makes
progress in the following sense.
832
00:54:07,000 --> 00:54:12,000
So, suppose you knew the
shortest path from s to some
833
00:54:12,000 --> 00:54:16,000
vertex.
OK, so you go from s to some
834
00:54:16,000 --> 00:54:19,000
other vertices.
Then you go to u.
835
00:54:19,000 --> 00:54:25,000
Then you go to v.
Suppose that is a shortest path
836
00:54:25,000 --> 00:54:29,000
from s to v.
OK, and also suppose that we
837
00:54:29,000 --> 00:54:35,000
already know in d of u the
shortest path weight from s to
838
00:54:35,000 --> 00:54:39,000
u.
So, suppose we have this
839
00:54:39,000 --> 00:54:43,000
equality.
We now know that we always have
840
00:54:43,000 --> 00:54:49,000
a greater than or equal to.
Suppose they are equal for u,
841
00:54:49,000 --> 00:54:53,000
OK, the vertex just before v in
the shortest path.
842
00:54:53,000 --> 00:54:58,000
OK, and suppose we relax that
edge, (u,v), OK,
843
00:54:58,000 --> 00:55:05,000
which is exactly this step.
This is relaxing the edge,
844
00:55:05,000 --> 00:55:08,000
(u,v).
But we'll just call it
845
00:55:08,000 --> 00:55:13,000
relaxation here.
After that relaxation,
846
00:55:13,000 --> 00:55:19,000
d of v equals delta of (s,v).
So, if we had the correct
847
00:55:19,000 --> 00:55:23,000
answer for u,
and we relax (u,v),
848
00:55:23,000 --> 00:55:29,000
then we get the correct answer
for v.
849
00:55:29,000 --> 00:55:32,000
OK, this is good news.
It means, if inductively we can
850
00:55:32,000 --> 00:55:36,000
somehow get the right answer for
u, now we know how to get the
851
00:55:36,000 --> 00:55:38,000
right answer for v.
In the algorithm,
852
00:55:38,000 --> 00:55:42,000
we don't actually know what the
vertex just before v in the
853
00:55:42,000 --> 00:55:45,000
shortest path is,
but in the analysis we can
854
00:55:45,000 --> 00:55:48,000
pretty much know that.
So, we have to prove this
855
00:55:48,000 --> 00:55:50,000
lemma.
This is actually even easier
856
00:55:50,000 --> 00:55:53,000
than the previous one:
don't even need induction
857
00:55:53,000 --> 00:55:57,000
because you just work through
what's going on in relaxation,
858
00:55:57,000 --> 00:56:01,000
and it's true.
So, here we go.
859
00:56:01,000 --> 00:56:04,000
So, we're interested in this
value, delta of Ss v.
860
00:56:04,000 --> 00:56:07,000
And we know what the shortest
path is.
861
00:56:07,000 --> 00:56:12,000
So, the shortest path weight is
the weight of this path.
862
00:56:12,000 --> 00:56:15,000
OK, so we can write down some
equality here.
863
00:56:15,000 --> 00:56:20,000
Well, I'm going to split out
the first part of the path and
864
00:56:20,000 --> 00:56:22,000
the last part of the path.
So, we have,
865
00:56:22,000 --> 00:56:27,000
I'll say, the weight from s,
so, this part of the path from
866
00:56:27,000 --> 00:56:32,000
s to u, plus the weight of this
edge, u, v.
867
00:56:32,000 --> 00:56:40,000
Remember, we could write w of a
path, and that was the total
868
00:56:40,000 --> 00:56:46,000
weight of all those edges.
So, what is this,
869
00:56:46,000 --> 00:56:51,000
the weight of this path from S
to u?
870
00:56:51,000 --> 00:56:59,000
Or, what property should I use
to figure out what that value
871
00:56:59,000 --> 00:57:02,000
is?
Yeah?
872
00:57:02,000 --> 00:57:05,000
s to v is the shortest path,
right?
873
00:57:05,000 --> 00:57:11,000
So, by optimal substructure,
from s to u is also a shortest
874
00:57:11,000 --> 00:57:13,000
path.
So, this is delta of s,
875
00:57:13,000 --> 00:57:14,000
u.
Cool.
876
00:57:14,000 --> 00:57:19,000
We'll hold on for now.
That's all we're going to say.
877
00:57:19,000 --> 00:57:24,000
On the other hand,
we know from this lemma that
878
00:57:24,000 --> 00:57:28,000
matter what we do,
d of v is greater than or equal
879
00:57:28,000 --> 00:57:32,000
to delta of s,
v.
880
00:57:32,000 --> 00:57:36,000
So, let's write that down.
So, there's a few cases,
881
00:57:36,000 --> 00:57:39,000
and this will eliminate some of
the cases.
882
00:57:39,000 --> 00:57:44,000
By that lemma correctness one,
we know that d of v is greater
883
00:57:44,000 --> 00:57:46,000
than or equal to delta of s,
v.
884
00:57:46,000 --> 00:57:50,000
So, it's either equal or
greater than at all times.
885
00:57:50,000 --> 00:57:55,000
So, I'm thinking about the time
before we do the relaxation,
886
00:57:55,000 --> 00:57:58,000
this (u,v).
So, at that point,
887
00:57:58,000 --> 00:58:02,000
this is certainly true.
So, either they're equal before
888
00:58:02,000 --> 00:58:06,000
relaxation or it's greater.
889
00:58:16,000 --> 00:58:19,000
OK, if they are equal before
relaxation, we're happy because
890
00:58:19,000 --> 00:58:23,000
relaxation only decreases values
by correctness one.
891
00:58:23,000 --> 00:58:26,000
It can't get any smaller than
this, so after relaxation it
892
00:58:26,000 --> 00:58:29,000
will also be equal.
OK, so in this case we're done.
893
00:58:29,000 --> 00:58:35,000
So, that's a trivial case.
So let's now suppose that d of
894
00:58:35,000 --> 00:58:41,000
v is greater than delta of s,
v before relaxation.
895
00:58:41,000 --> 00:58:46,000
That's perfectly valid.
Hopefully now we fix it.
896
00:58:46,000 --> 00:58:51,000
OK, well the point is,
we know this delta s,
897
00:58:51,000 --> 00:58:53,000
v.
It is this sum.
898
00:58:53,000 --> 00:58:58,000
OK, we also know this.
So, delta of s,
899
00:58:58,000 --> 00:59:03,000
u we know is d of u.
And, we have this w u,
900
00:59:03,000 --> 00:59:04,000
v.
So, delta of s,
901
00:59:04,000 --> 00:59:07,000
v is d of u plus w of (u,v)
because we are assuming we have
902
00:59:07,000 --> 00:59:11,000
this shortest path structure
where you go from s to u,
903
00:59:11,000 --> 00:59:13,000
and then you follow the edge,
(u,v).
904
00:59:13,000 --> 00:59:16,000
So, we know this.
So, we know d of v is greater
905
00:59:16,000 --> 00:59:19,000
than d of u plus w of (u,v).
By golly, that's this condition
906
00:59:19,000 --> 00:59:22,000
in relaxation.
So, we're just checking,
907
00:59:22,000 --> 00:59:24,000
relaxation actually does
something here.
908
00:59:24,000 --> 00:59:26,000
OK, if you had the wrong
distance estimate,
909
00:59:26,000 --> 00:59:33,000
this if condition is satisfied.
Therefore, we do this.
910
00:59:33,000 --> 00:59:36,000
So, in this case,
we relax.
911
00:59:36,000 --> 00:59:44,000
So, I'm just relaxing.
Then, we set d of v to d of u
912
00:59:44,000 --> 00:59:52,000
plus WUV, which is what we want.
OK, so we set d of v to d of u
913
00:59:52,000 --> 00:59:57,000
plus w of (u,v).
And, this equals,
914
00:59:57,000 --> 01:00:01,761
as we said here,
delta of S, v,
915
01:00:01,761 --> 01:00:08,000
which is what we wanted to
prove.
916
01:00:08,000 --> 01:00:12,662
Done.
OK, I'm getting more and more
917
01:00:12,662 --> 01:00:18,834
excited as we get into the meat
of this proof.
918
01:00:18,834 --> 01:00:22,400
Any questions so far?
Good.
919
01:00:22,400 --> 01:00:28,982
Now comes the hard part.
These are both very easy
920
01:00:28,982 --> 01:00:35,935
lemmas, right?
I'll use these two boards.
921
01:00:35,935 --> 01:00:41,081
We don't need these proofs
anymore.
922
01:00:41,081 --> 01:00:48,043
We just need these statements:
correctness one,
923
01:00:48,043 --> 01:00:52,583
correctness lemma;
great names.
924
01:00:52,583 --> 01:01:00,000
So, now finally we get to
correctness two.
925
01:01:00,000 --> 01:01:03,771
So, we had one and one and a
half.
926
01:01:03,771 --> 01:01:09,485
So, I guess correctness is,
itself, a mini-trilogy,
927
01:01:09,485 --> 01:01:14,400
the mini-series.
OK, so correctness two says
928
01:01:14,400 --> 01:01:20,342
when the algorithm is done,
we have the right answer.
929
01:01:20,342 --> 01:01:26,514
This is really correctness.
But, it's going to build on
930
01:01:26,514 --> 01:01:32,000
correctness one and correctness
lemma.
931
01:01:32,000 --> 01:01:35,699
So, we want d of v to equal
delta of s, v for all vertices,
932
01:01:35,699 --> 01:01:39,207
v at the end of the algorithm.
That is clearly our goal.
933
01:01:39,207 --> 01:01:42,779
Now, this theorem is assuming
that all of the weights are
934
01:01:42,779 --> 01:01:46,223
nonnegative, just to repeat.
It doesn't assume anything
935
01:01:46,223 --> 01:01:48,136
else.
So, it's going to get the
936
01:01:48,136 --> 01:01:50,687
infinities right.
But, if there are minus
937
01:01:50,687 --> 01:01:54,004
infinities, all bets are off.
OK, even if there's any
938
01:01:54,004 --> 01:01:57,831
negative weight edge anywhere,
it's not going to do the right
939
01:01:57,831 --> 01:02:02,573
thing necessarily.
But, assuming all the weights
940
01:02:02,573 --> 01:02:06,656
are nonnegative,
which is reasonable if they're
941
01:02:06,656 --> 01:02:10,207
measuring time.
Usually it costs money to
942
01:02:10,207 --> 01:02:14,378
travel along edges.
They don't pay you to do it.
943
01:02:14,378 --> 01:02:18,106
But who knows?
So, I need just to say a few
944
01:02:18,106 --> 01:02:20,769
things.
One of the things we've
945
01:02:20,769 --> 01:02:26,094
mentioned somewhere along the
way is when you add a vertex to
946
01:02:26,094 --> 01:02:32,363
S, you never change its weight.
OK, that actually requires
947
01:02:32,363 --> 01:02:35,272
proof.
I'm just going to state it
948
01:02:35,272 --> 01:02:37,636
here.
It's not hard to see.
949
01:02:37,636 --> 01:02:42,000
d of v doesn't change.
OK, this is essentially an
950
01:02:42,000 --> 01:02:47,636
induction once v is added to S.
OK, this will actually followed
951
01:02:47,636 --> 01:02:50,727
by something we'll say in a
moment.
952
01:02:50,727 --> 01:02:56,272
OK, so all I really care about
is when a vertex is added to S,
953
01:02:56,272 --> 01:03:01,000
we better have the right
estimate because after that,
954
01:03:01,000 --> 01:03:06,000
we're not going to change it,
let's say.
955
01:03:06,000 --> 01:03:10,097
OK, we could define the
algorithm that way.
956
01:03:10,097 --> 01:03:15,365
We are not, but we could.
I'll say more about this in a
957
01:03:15,365 --> 01:03:18,390
second.
So, all we care about is
958
01:03:18,390 --> 01:03:21,804
whether d of v equals delta of
s, v.
959
01:03:21,804 --> 01:03:26,780
That's what we want to prove.
So, it's clearly that.
960
01:03:26,780 --> 01:03:32,536
It should be true at the end.
But, it suffices to prove that
961
01:03:32,536 --> 01:03:38,000
it holds when v is added to S,
to capital S.
962
01:03:38,000 --> 01:03:40,327
OK, this actually implies the
first statement.
963
01:03:40,327 --> 01:03:42,086
It has sort of a funny
implication.
964
01:03:42,086 --> 01:03:44,982
But, if we can prove this,
that d of v equals delta of s,
965
01:03:44,982 --> 01:03:47,258
v, when you add to S,
we know relaxation only
966
01:03:47,258 --> 01:03:49,586
decreases value.
So, it can't get any smaller.
967
01:03:49,586 --> 01:03:51,241
It would be from correctness
one.
968
01:03:51,241 --> 01:03:54,137
Correctness one says we can't
get any smaller than delta.
969
01:03:54,137 --> 01:03:57,396
So, if we get a quality at that
point, we'll have a quality from
970
01:03:57,396 --> 01:03:59,362
then on.
So, that actually implies d of
971
01:03:59,362 --> 01:04:02,000
v never changes after that
point.
972
01:04:02,000 --> 01:04:06,085
OK, so we're going to prove
this.
973
01:04:06,085 --> 01:04:10,297
Good.
Well, suppose it isn't true.
974
01:04:10,297 --> 01:04:15,787
So this would be a proof by a
contradiction.
975
01:04:15,787 --> 01:04:22,042
Suppose for contradiction that
this fails to hold.
976
01:04:22,042 --> 01:04:26,638
And, let's look at the first
failure.
977
01:04:26,638 --> 01:04:32,000
Suppose u is the first vertex
--
978
01:04:41,000 --> 01:04:45,053
-- that's about to be added to
S.
979
01:04:45,053 --> 01:04:52,146
I want to consider the time
right before it's added to S,
980
01:04:52,146 --> 01:04:56,706
for which we don't have what we
want.
981
01:04:56,706 --> 01:05:04,166
These are not equal.
d of u does not equal delta of
982
01:05:04,166 --> 01:05:08,611
s, u.
Well, if they're not equal,
983
01:05:08,611 --> 01:05:16,805
we know from correctness one
that d of E is strictly greater
984
01:05:16,805 --> 01:05:20,971
than delta of s,
u, so, d of u.
985
01:05:20,971 --> 01:05:28,471
So, we have d of u is strictly
greater than delta of s,
986
01:05:28,471 --> 01:05:32,852
u.
OK, that's the beginning of the
987
01:05:32,852 --> 01:05:37,453
proof, nothing too exciting yet,
just some warm-up.
988
01:05:37,453 --> 01:05:41,319
OK, but this,
used already correctness one.
989
01:05:41,319 --> 01:05:46,564
I think that's the only time
that we use it in this proof.
990
01:05:46,564 --> 01:05:52,177
OK, so I sort of just want to
draw picture of what's going on.
991
01:05:52,177 --> 01:05:55,674
But I need a little bit of
description.
992
01:05:55,674 --> 01:06:00,000
So, let's look at the shortest
path.
993
01:06:00,000 --> 01:06:04,176
Somehow, d of u is greater than
the shortest path.
994
01:06:04,176 --> 01:06:08,352
So, consider the shortest path
or a shortest path.
995
01:06:08,352 --> 01:06:12,784
Let p be a shortest path,
not just any shortest path,
996
01:06:12,784 --> 01:06:15,596
but the shortest path from s to
u.
997
01:06:15,596 --> 01:06:20,028
OK, so that means that the
weight of this path is the
998
01:06:20,028 --> 01:06:24,375
shortest path weight.
So, we have some equations for
999
01:06:24,375 --> 01:06:28,636
what's going on here.
So, we care about delta of s,
1000
01:06:28,636 --> 01:06:32,791
u.
Here's a path with that weight.
1001
01:06:32,791 --> 01:06:37,473
It's got to be one because
shortest paths exist here;
1002
01:06:37,473 --> 01:06:41,796
slight exceptional cases if
it's a plus infinity,
1003
01:06:41,796 --> 01:06:45,127
but I'm not going to worry
about that.
1004
01:06:45,127 --> 01:06:49,000
So, let me draw a picture
somewhere.
1005
01:07:03,000 --> 01:07:06,283
So, we have s.
We have u.
1006
01:07:06,283 --> 01:07:11,344
Here is the shortest path from
s to u.
1007
01:07:11,344 --> 01:07:16,542
That's p.
No idea what it looks like so
1008
01:07:16,542 --> 01:07:21,056
far.
Now, what we also have is the
1009
01:07:21,056 --> 01:07:27,759
notion of capital S.
So, I'm going to draw capital
1010
01:07:27,759 --> 01:07:32,000
S.
So, this is big S.
1011
01:07:32,000 --> 01:07:34,600
We know that little s is in big
S.
1012
01:07:34,600 --> 01:07:37,279
We know that u is not yet in
big S.
1013
01:07:37,279 --> 01:07:40,195
So, I haven't screwed up
anything yet,
1014
01:07:40,195 --> 01:07:42,638
right?
This path starts in S and
1015
01:07:42,638 --> 01:07:47,445
leaves it at some point because
until we are about to add u to
1016
01:07:47,445 --> 01:07:51,070
S, so it hasn't happened yet,
so u is not in S.
1017
01:07:51,070 --> 01:07:53,671
Fine.
What I want to do is look at
1018
01:07:53,671 --> 01:07:57,375
the first place here where the
path, p, exits S.
1019
01:07:57,375 --> 01:08:02,526
So, there is some vertex here.
Let's call it x.
1020
01:08:02,526 --> 01:08:06,436
There's some vertex here.
We'll call it y.
1021
01:08:06,436 --> 01:08:10,632
OK, possibly x equals S.
Possibly y equals u.
1022
01:08:10,632 --> 01:08:16,260
But it's got to exit somewhere,
because it starts inside and
1023
01:08:16,260 --> 01:08:19,979
ends up outside.
And it's a finite path.
1024
01:08:19,979 --> 01:08:25,893
OK, so consider the first time
it happens; not the second time,
1025
01:08:25,893 --> 01:08:29,803
the first.
OK, so consider the first edge,
1026
01:08:29,803 --> 01:08:37,140
(x,y), where p exits capital S.
The shortest path from s to u
1027
01:08:37,140 --> 01:08:42,014
exits capital S.
It's got to happen somewhere.
1028
01:08:42,014 --> 01:08:46,779
Cool, now, what do we know?
Little x is in S.
1029
01:08:46,779 --> 01:08:53,060
So, it has the right answer
because u, we were about to add
1030
01:08:53,060 --> 01:08:59,776
u to S, and that was the first
violation of something in S that
1031
01:08:59,776 --> 01:09:06,206
has the wrong d of x estimate.
So, d of x equals delta of s,
1032
01:09:06,206 --> 01:09:08,565
x.
Because we are looking at the
1033
01:09:08,565 --> 01:09:11,988
first violation,
x is something that got added
1034
01:09:11,988 --> 01:09:14,423
before.
So, by induction on time,
1035
01:09:14,423 --> 01:09:18,608
or because we had the first
violation, d of x equals the
1036
01:09:18,608 --> 01:09:21,042
shortest path weight from S to
x.
1037
01:09:21,042 --> 01:09:25,000
So, that's good news.
Now we are trying to apply this
1038
01:09:25,000 --> 01:09:27,815
lemma.
It's the only thing left to do.
1039
01:09:27,815 --> 01:09:32,000
We haven't used this lemma for
anything.
1040
01:09:32,000 --> 01:09:35,358
So, we have the setup.
If we already know that one of
1041
01:09:35,358 --> 01:09:38,974
the d values is the right
answer, and we relaxed the edge
1042
01:09:38,974 --> 01:09:42,009
that goes out from it,
then we get another right
1043
01:09:42,009 --> 01:09:44,399
answer.
So that's what I want to argue
1044
01:09:44,399 --> 01:09:46,982
over here.
We know that the d of x equals
1045
01:09:46,982 --> 01:09:50,017
this weight because,
again, subpaths of shortest
1046
01:09:50,017 --> 01:09:53,504
paths are shortest paths.
We have optimal substructure,
1047
01:09:53,504 --> 01:09:56,023
so this is a shortest path,
from S to x.
1048
01:09:56,023 --> 01:09:58,800
It might not be the only one,
but it is one.
1049
01:09:58,800 --> 01:10:02,029
So we know that matches.
Now, I want to think about
1050
01:10:02,029 --> 01:10:05,000
relaxing this edge,
(x,y).
1051
01:10:05,000 --> 01:10:08,536
Well, x is in capital S.
And, the algorithm says,
1052
01:10:08,536 --> 01:10:11,778
whenever you add a vertex,
u, to the big set,
1053
01:10:11,778 --> 01:10:15,389
S, you relax all the edges that
go out from there.
1054
01:10:15,389 --> 01:10:19,368
OK, so when we added x to S,
and we now look far in the
1055
01:10:19,368 --> 01:10:22,610
future, we're about to add some
other vertex.
1056
01:10:22,610 --> 01:10:26,073
Right after we added x to S,
we relax this edge,
1057
01:10:26,073 --> 01:10:30,273
(x,y), because we relaxed every
edge that goes out from x,
1058
01:10:30,273 --> 01:10:36,443
OK, whatever they were.
Some of them went into S.
1059
01:10:36,443 --> 01:10:42,091
Some of them went out.
Here's one of them.
1060
01:10:42,091 --> 01:10:46,913
So, when we added x to S,
we got XS.
1061
01:10:46,913 --> 01:10:52,561
When we added x to S,
we relaxed the edge,
1062
01:10:52,561 --> 01:10:57,382
(x,y).
OK, so now we're going to use
1063
01:10:57,382 --> 01:11:04,000
the lemma.
So, by the correctness lemma --
1064
01:11:15,000 --> 01:11:18,863
What do you get?
Well, we add this correct
1065
01:11:18,863 --> 01:11:23,387
shortest path weight to x now.
We relax the edge,
1066
01:11:23,387 --> 01:11:26,403
(x,y).
So, now we should have the
1067
01:11:26,403 --> 01:11:29,607
correct shortest path weight for
y.
1068
01:11:29,607 --> 01:11:33,000
d of y equals delta of s,
y.
1069
01:11:33,000 --> 01:11:35,509
OK, this is sometime in the
past.
1070
01:11:35,509 --> 01:11:38,803
In particular,
now, it should still be true
1071
01:11:38,803 --> 01:11:43,588
because once you get down to the
right answer you never change
1072
01:11:43,588 --> 01:11:45,549
it.
OK, we should be done.
1073
01:11:45,549 --> 01:11:49,156
OK, why are we done?
Well, what else do we know
1074
01:11:49,156 --> 01:11:51,431
here?
We assumed something for
1075
01:11:51,431 --> 01:11:54,803
contradiction,
so we better contradict that.
1076
01:11:54,803 --> 01:11:58,647
We assume somehow,
d of u is strictly greater than
1077
01:11:58,647 --> 01:12:03,142
delta of s, u.
So, d of u here is strictly
1078
01:12:03,142 --> 01:12:06,476
greater than the length of this
whole path.
1079
01:12:06,476 --> 01:12:10,047
Well, we don't really know
whether u equals y.
1080
01:12:10,047 --> 01:12:14,015
It could, could not.
And, but what do we know about
1081
01:12:14,015 --> 01:12:18,857
this shortest path from S to y?
Well, it could only be shorter
1082
01:12:18,857 --> 01:12:21,952
than from S to u because it's a
subpath.
1083
01:12:21,952 --> 01:12:26,476
And it's the shortest path
because it's the subpath of the
1084
01:12:26,476 --> 01:12:29,888
shortest path.
The shortest path from S to y
1085
01:12:29,888 --> 01:12:36,000
has to be less than or equal to
the shortest path from S to u.
1086
01:12:36,000 --> 01:12:44,627
OK, S to y: less than or equal
to s, u, OK, just because the
1087
01:12:44,627 --> 01:12:47,405
subpath.
I'm closer.
1088
01:12:47,405 --> 01:12:51,207
I've got delta of s,
u now.
1089
01:12:51,207 --> 01:12:56,033
Somehow, I want to involve d of
u.
1090
01:12:56,033 --> 01:13:01,443
So, I want to relate d of y to
d of u.
1091
01:13:01,443 --> 01:13:07,389
What do I know about d of u?
Yeah?
1092
01:13:07,389 --> 01:13:10,814
d of u is smaller because we
have a Min heap,
1093
01:13:10,814 --> 01:13:12,449
yeah.
We always chose,
1094
01:13:12,449 --> 01:13:14,862
let's erase,
it's way down here.
1095
01:13:14,862 --> 01:13:17,664
We chose u.
This is the middle of the
1096
01:13:17,664 --> 01:13:20,778
algorithm.
It's the reason I kept this to
1097
01:13:20,778 --> 01:13:23,736
be the minimum key.
This is keyed on d.
1098
01:13:23,736 --> 01:13:28,095
So, we know that at this
moment, when we're trying to add
1099
01:13:28,095 --> 01:13:30,275
u to S, right,
y is not in S,
1100
01:13:30,275 --> 01:13:35,412
and u is not in S.
They might actually be the same
1101
01:13:35,412 --> 01:13:38,057
vertex.
But both of these vertices,
1102
01:13:38,057 --> 01:13:40,080
same or not,
are outside S.
1103
01:13:40,080 --> 01:13:44,204
We chose u because d of u has
the smallest d estimate.
1104
01:13:44,204 --> 01:13:48,250
So, d of y has to be greater
than or equal to d of u.
1105
01:13:48,250 --> 01:13:51,674
It might be equal if they're
the same vertex,
1106
01:13:51,674 --> 01:13:55,020
but it's got to be greater than
or equal to.
1107
01:13:55,020 --> 01:14:00,000
So, d of y here is greater than
or equal to d of u.
1108
01:14:00,000 --> 01:14:03,791
So, here we're using the fact
that we actually made a greedy
1109
01:14:03,791 --> 01:14:06,168
choice.
It's the one place we're using
1110
01:14:06,168 --> 01:14:09,317
the greedy choice.
Better use it somewhere because
1111
01:14:09,317 --> 01:14:13,108
you can't just take an arbitrary
vertex and declare it to be
1112
01:14:13,108 --> 01:14:15,293
done.
You've got to take the greedy
1113
01:14:15,293 --> 01:14:17,477
one.
OK, now we have d of u is less
1114
01:14:17,477 --> 01:14:20,947
than or equal to delta of s,
u, which contradicts this.
1115
01:14:20,947 --> 01:14:24,096
OK, sort of magical that that
all just worked out.
1116
01:14:24,096 --> 01:14:27,566
But sort of like the previous
proofs, you just see what
1117
01:14:27,566 --> 01:14:32,000
happens and it works.
OK, that's the approximation.
1118
01:14:32,000 --> 01:14:35,005
The only real idea here is to
look at this edge.
1119
01:14:35,005 --> 01:14:37,563
In fact, you could look at this
edge too.
1120
01:14:37,563 --> 01:14:41,209
But let's look at some edge
that comes from S and goes out
1121
01:14:41,209 --> 01:14:44,151
of S, and argue that while x has
to be correct,
1122
01:14:44,151 --> 01:14:47,156
and what we made x correct,
y had to be correct,
1123
01:14:47,156 --> 01:14:49,779
and now, why the hell are we
looking at u?
1124
01:14:49,779 --> 01:14:52,337
y is the thing you should have
looked at.
1125
01:14:52,337 --> 01:14:55,598
And, there you get a
contradiction because y had the
1126
01:14:55,598 --> 01:14:57,325
right answer.
If u equals y,
1127
01:14:57,325 --> 01:14:59,755
that's fine,
or if u and y were sort of
1128
01:14:59,755 --> 01:15:02,441
equally good,
that's also fine if all these
1129
01:15:02,441 --> 01:15:07,604
weights were zero.
So, the picture might actually
1130
01:15:07,604 --> 01:15:10,469
look like this.
But, in that case,
1131
01:15:10,469 --> 01:15:14,376
d of u is the correct answer.
It was delta SU.
1132
01:15:14,376 --> 01:15:19,064
We assumed that it wasn't.
That's where we're getting a
1133
01:15:19,064 --> 01:15:21,408
contradiction.
Pretty clear?
1134
01:15:21,408 --> 01:15:25,054
Go over this proof.
It's a bit complicated,
1135
01:15:25,054 --> 01:15:28,440
naturally.
OK, we have a little bit more
1136
01:15:28,440 --> 01:15:34,168
to cover, some easier stuff.
OK, the first thing is what's
1137
01:15:34,168 --> 01:15:36,710
the running time of this
algorithm?
1138
01:15:36,710 --> 01:15:41,196
I'll do this very quick because
we're actually seen this many
1139
01:15:41,196 --> 01:15:45,233
times before last class.
There was some initialization.
1140
01:15:45,233 --> 01:15:48,448
The initialization,
which is no longer here,
1141
01:15:48,448 --> 01:15:50,467
is linear time.
No big deal.
1142
01:15:50,467 --> 01:15:53,308
OK, extract Min.
Well, that's some data
1143
01:15:53,308 --> 01:15:56,373
structure.
So, we have something like size
1144
01:15:56,373 --> 01:15:59,065
of V.
Every vertex we extract the Min
1145
01:15:59,065 --> 01:16:01,607
once, and that's it.
So, size of V,
1146
01:16:01,607 --> 01:16:06,634
extract mins.
OK, so that's pretty simple.
1147
01:16:06,634 --> 01:16:12,466
OK, then we had this main loop.
This is a completely conceptual
1148
01:16:12,466 --> 01:16:16,135
operation.
S is not actually used in the
1149
01:16:16,135 --> 01:16:19,240
algorithm.
It's just for thinking.
1150
01:16:19,240 --> 01:16:23,285
OK, so this takes zero time.
Got to love it.
1151
01:16:23,285 --> 01:16:28,742
OK, and now the heart is here.
So, how many times does this
1152
01:16:28,742 --> 01:16:33,324
loop iterate?
That's the degree of u.
1153
01:16:33,324 --> 01:16:38,780
So, what is the total number of
times that we execute a
1154
01:16:38,780 --> 01:16:43,428
relaxation step?
It doesn't necessarily mean we
1155
01:16:43,428 --> 01:16:47,672
do this, but we at least execute
this body.
1156
01:16:47,672 --> 01:16:53,128
Over the whole algorithm,
how many times do we do this?
1157
01:16:53,128 --> 01:16:57,170
Every vertex,
we look at all the outgoing
1158
01:16:57,170 --> 01:17:02,000
edges from there.
So, the total would be?
1159
01:17:11,000 --> 01:17:12,615
Number of edges,
yeah.
1160
01:17:12,615 --> 01:17:15,307
So, this number of edges
iterations.
1161
01:17:15,307 --> 01:17:19,615
OK, this is essentially the
handshaking lemma we saw last
1162
01:17:19,615 --> 01:17:24,230
time, but for directed graphs.
And we are only looking at the
1163
01:17:24,230 --> 01:17:27,538
outgoing edges.
So, it's not a factor of two
1164
01:17:27,538 --> 01:17:32,000
here because you're only
outgoing from one side.
1165
01:17:32,000 --> 01:17:36,573
So, we have number of
reiterations.
1166
01:17:36,573 --> 01:17:42,358
In the worst case,
we do a decreased key for
1167
01:17:42,358 --> 01:17:45,183
everyone.
So, at most:
1168
01:17:45,183 --> 01:17:50,026
E decreased keys.
OK, so the time is,
1169
01:17:50,026 --> 01:17:57,695
well, we have v extract Mins,
so the time to do an extract
1170
01:17:57,695 --> 01:18:05,771
Min, whatever that is.
And we have E decreased keys,
1171
01:18:05,771 --> 01:18:12,013
whatever that is,
and this is exactly the running
1172
01:18:12,013 --> 01:18:20,336
time we had for Prim's algorithm
for a minimum spanning tree last
1173
01:18:20,336 --> 01:18:24,237
time.
And, it depends what data
1174
01:18:24,237 --> 01:18:31,000
structure you use,
what running time you get.
1175
01:18:31,000 --> 01:18:34,688
So, I'm going to skip the whole
table here.
1176
01:18:34,688 --> 01:18:39,518
But, if you use an array,
the final running time will be
1177
01:18:39,518 --> 01:18:44,436
V^2 because you have order of v
extract Min, and you have
1178
01:18:44,436 --> 01:18:49,090
constant time decreased key.
If you use a binary heap,
1179
01:18:49,090 --> 01:18:53,568
which we know and love,
then we have order log v for
1180
01:18:53,568 --> 01:18:57,609
each operation.
And so, this is V plus E log V.
1181
01:18:57,609 --> 01:19:02,000
And, so that's what we know how
to do.
1182
01:19:02,000 --> 01:19:06,830
And, if you use this fancy data
structure called a Fibonacci
1183
01:19:06,830 --> 01:19:11,005
heap, you get constant time
decreased key amortized.
1184
01:19:11,005 --> 01:19:16,000
And, you get an E plus v log v
worst case bound on the running
1185
01:19:16,000 --> 01:19:18,701
time.
So, this is the best we know
1186
01:19:18,701 --> 01:19:23,368
how to solve shortest paths
without any extra assumptions,
1187
01:19:23,368 --> 01:19:28,116
single source shortest paths
with non-negative edge weights
1188
01:19:28,116 --> 01:19:32,040
in general.
OK, this is almost as good and
1189
01:19:32,040 --> 01:19:34,353
this is sometimes better than
that.
1190
01:19:34,353 --> 01:19:38,163
But these are essentially
irrelevant except that you know
1191
01:19:38,163 --> 01:19:41,020
how to do these.
You don't know how to do a
1192
01:19:41,020 --> 01:19:45,238
Fibonacci heap unless you read
that in the chapter of the book.
1193
01:19:45,238 --> 01:19:48,434
That's why we mention the top
two running times.
1194
01:19:48,434 --> 01:19:51,632
OK, I want to talk briefly
about a simpler case,
1195
01:19:51,632 --> 01:19:55,510
which you may have seen before.
And so it's sort of fun to
1196
01:19:55,510 --> 01:20:00,000
connect this up to breadth first
search in a graph.
1197
01:20:00,000 --> 01:20:03,302
So, I mean that ends Dijkstra,
so to speak.
1198
01:20:03,302 --> 01:20:08,175
But now I want to think about a
special case where the graph is
1199
01:20:08,175 --> 01:20:12,737
unweighted, meaning w of (u,v)
equals one for all vertices,
1200
01:20:12,737 --> 01:20:15,174
u and v.
OK, suppose we had that
1201
01:20:15,174 --> 01:20:17,847
property.
Can we do any better than
1202
01:20:17,847 --> 01:20:20,599
Dijkstra?
Can we do better than this
1203
01:20:20,599 --> 01:20:23,980
running time?
Well, we probably have to look
1204
01:20:23,980 --> 01:20:26,889
at all the edges and all the
vertices.
1205
01:20:26,889 --> 01:20:32,000
So, the only thing I'm
questioning is this log v.
1206
01:20:32,000 --> 01:20:36,067
Can I avoid that?
I gave away the answer a little
1207
01:20:36,067 --> 01:20:38,779
bit.
The answer is called breadth
1208
01:20:38,779 --> 01:20:42,508
first search,
or BFS, which you have probably
1209
01:20:42,508 --> 01:20:45,813
seen before.
Next to depth first search,
1210
01:20:45,813 --> 01:20:50,050
it's one of the standard ways
to look at the graph.
1211
01:20:50,050 --> 01:20:54,627
But we can say a little bit
more than you may have seen
1212
01:20:54,627 --> 01:20:57,762
before.
Breadth for search is actually
1213
01:20:57,762 --> 01:21:02,000
Dijkstra's algorithm:
kind of nifty.
1214
01:21:02,000 --> 01:21:08,607
There are two changes.
First change is that breadth
1215
01:21:08,607 --> 01:21:13,894
for search does not use a
priority queue.
1216
01:21:13,894 --> 01:21:19,048
I'll just tell you what it uses
instead.
1217
01:21:19,048 --> 01:21:26,449
You can use a queue first in
first out honest-to-goodness
1218
01:21:26,449 --> 01:21:32,000
queue instead of a priority
queue.
1219
01:21:32,000 --> 01:21:36,728
OK, it turns out that works.
Instead of doing extract Min,
1220
01:21:36,728 --> 01:21:40,295
you just take the first thing
off the queue.
1221
01:21:40,295 --> 01:21:44,692
Instead of doing decreased key,
OK, here's a subtlety.
1222
01:21:44,692 --> 01:21:48,260
But, this if statement changes
a little bit.
1223
01:21:48,260 --> 01:21:50,831
So, here is the relaxation
step.
1224
01:21:50,831 --> 01:21:54,730
So, in order to relax,
you say this much simpler
1225
01:21:54,730 --> 01:21:57,550
thing.
If we haven't visited v yet,
1226
01:21:57,550 --> 01:22:01,782
then we declare it to have the
shortest path weight,
1227
01:22:01,782 --> 01:22:06,511
say, d of v is d of u plus one,
which is the weight of the
1228
01:22:06,511 --> 01:22:12,423
edge, (u,v).
And we add v to the end of the
1229
01:22:12,423 --> 01:22:16,076
queue.
So, now, we start with the
1230
01:22:16,076 --> 01:22:20,869
queue empty.
Actually, it will just contain
1231
01:22:20,869 --> 01:22:25,777
the vertex, S,
because that's the only thing
1232
01:22:25,777 --> 01:22:31,347
we know the shortest path for.
So, the queue is just for,
1233
01:22:31,347 --> 01:22:33,316
I know the shortest path of
this thing.
1234
01:22:33,316 --> 01:22:36,217
Just deal with it when you
can't look at all the outgoing
1235
01:22:36,217 --> 01:22:38,652
edges when you can.
So, initially that's just S.
1236
01:22:38,652 --> 01:22:40,777
You say, well,
for all the outgoing edges,
1237
01:22:40,777 --> 01:22:42,746
S has zero.
All the outgoing edges from
1238
01:22:42,746 --> 01:22:45,388
there have weight one.
The shortest path weight from
1239
01:22:45,388 --> 01:22:47,668
the source is one.
You certainly can't do any
1240
01:22:47,668 --> 01:22:49,896
better than that if all the
weights are one.
1241
01:22:49,896 --> 01:22:52,434
OK, so we add all those
vertices to the end of the
1242
01:22:52,434 --> 01:22:54,093
queue.
Then, we process things in
1243
01:22:54,093 --> 01:22:55,958
order, and we just keep
incrementing,
1244
01:22:55,958 --> 01:22:57,979
if their value is d of u,
add one to it.
1245
01:22:57,979 --> 01:23:02,543
That's d of v.
And then we are going to add v
1246
01:23:02,543 --> 01:23:05,510
to S what we get to it in the
queue.
1247
01:23:05,510 --> 01:23:09,156
OK, that is breadth for search,
very simple.
1248
01:23:09,156 --> 01:23:13,989
And, you can look at the text
for the algorithm and for an
1249
01:23:13,989 --> 01:23:17,973
example because I don't have
time to cover that.
1250
01:23:17,973 --> 01:23:21,704
But the key thing is that the
time is faster.
1251
01:23:21,704 --> 01:23:25,434
The time is order V plus E
because as before,
1252
01:23:25,434 --> 01:23:30,267
we only look at each edge once
we look at all the outgoing
1253
01:23:30,267 --> 01:23:34,930
edges from all the vertices.
As soon as we set d of v to
1254
01:23:34,930 --> 01:23:40,079
something, it will remain that.
We never touch it.
1255
01:23:40,079 --> 01:23:43,136
We are going to add it to S.
That only happens once.
1256
01:23:43,136 --> 01:23:46,193
So, this if statement,
and so on, in the in-queuing,
1257
01:23:46,193 --> 01:23:48,711
is done order E times,
or actually E times,
1258
01:23:48,711 --> 01:23:50,689
exactly.
An in-queuing to a queue,
1259
01:23:50,689 --> 01:23:54,106
and de-queuing from a queue,
that's what we use instead of
1260
01:23:54,106 --> 01:23:57,583
extract Min, take constant time,
so the total running time,
1261
01:23:57,583 --> 01:24:01,000
number of vertices plus the
number of edges.
1262
01:24:01,000 --> 01:24:04,580
OK, not so obvious that this
works, but you can prove that it
1263
01:24:04,580 --> 01:24:06,550
works using the Dijkstra
analysis.
1264
01:24:06,550 --> 01:24:09,891
All you have to do is prove
that the FIFO priority queue.
1265
01:24:09,891 --> 01:24:12,816
Once you know that,
by the correctness of Dijkstra
1266
01:24:12,816 --> 01:24:15,501
you get the correctness of
breadth for search.
1267
01:24:15,501 --> 01:24:19,022
So, not only is breadth for
search finding all the vertices,
1268
01:24:19,022 --> 01:24:21,588
which is maybe what you
normally use it for,
1269
01:24:21,588 --> 01:24:25,109
but it finds the shortest path
weights from S to every other
1270
01:24:25,109 --> 01:24:27,198
vertex when the weights are all
one.
1271
01:24:27,198 --> 01:24:30,003
So, there we go:
introduction to shortest paths.
1272
01:24:30,003 --> 01:24:33,000
Next time we'll deal with
negative weights.