1
00:00:07 --> 00:00:12
-- valuable experience.
OK, today we're going to start
2
00:00:12 --> 00:00:18
talking about a particular class
of algorithms called greedy
3
00:00:18 --> 00:00:22
algorithms.
But we're going to do it in the
4
00:00:22 --> 00:00:27
context of graphs.
So, I want to review a little
5
00:00:27 --> 00:00:32
bit about graphs,
which mostly you can find in
6
00:00:32 --> 00:00:39
the textbook in appendix B.
And so, if you haven't reviewed
7
00:00:39 --> 00:00:45
in appendix B recently,
please sit down and review
8
00:00:45 --> 00:00:50
appendix B.
It will pay off especially
9
00:00:50 --> 00:00:55
during our take-home quiz.
So, just reminder,
10
00:00:55 --> 00:01:01
a digraph, what's a digraph?
What's that short for?
11
00:01:01 --> 00:01:04
Directed graph,
OK?
12
00:01:04 --> 00:01:07
Directed graph,
G equals (V,E),
13
00:01:07 --> 00:01:13
OK, has a set,
V, of vertices.
14
00:01:13 --> 00:01:17
And, I always get people
telling me that I have one
15
00:01:17 --> 00:01:20
vertice.
The singular is not vertice;
16
00:01:20 --> 00:01:21
it is vertex,
OK?
17
00:01:21 --> 00:01:25
The plural is vertices.
The singular is vertex.
18
00:01:25 --> 00:01:29
It's one of those weird English
words.
19
00:01:29 --> 00:01:33
It's probably originally like
French or something,
20
00:01:33 --> 00:01:37
right?
I don't know.
21
00:01:37 --> 00:01:47
OK, anyway, and we have a set,
E, which is a subset of V cross
22
00:01:47 --> 00:01:52
V of edges.
So that's a digraph.
23
00:01:52 --> 00:02:02
And undirected graph,
E contains unordered pairs.
24
00:02:02 --> 00:02:12
25
00:02:12 --> 00:02:15
OK, and, sorry?
It's Latin, OK,
26
00:02:15 --> 00:02:21
so it's probably pretty old,
then, in English.
27
00:02:21 --> 00:02:28
I guess the vertex would be a
little bit of a giveaway that
28
00:02:28 --> 00:02:34
maybe it wasn't French.
It started to be used in 1570,
29
00:02:34 --> 00:02:39
OK.
OK, good, OK,
30
00:02:39 --> 00:02:52
so the number of edges is,
whether it's directed or
31
00:02:52 --> 00:03:02
undirected, is O of what?
V^2, good.
32
00:03:02 --> 00:03:05
OK, and one of the conventions
that will have when we're
33
00:03:05 --> 00:03:09
dealing, once we get into
graphs, we deal a lot with sets.
34
00:03:09 --> 00:03:13
We generally drop the vertical
bar notation within O's just
35
00:03:13 --> 00:03:16
because it's applied.
It just makes it messier.
36
00:03:16 --> 00:03:18
So, once again,
another abuse of notation.
37
00:03:18 --> 00:03:22
It really should be order the
size of V^2, but it just messes
38
00:03:22 --> 00:03:25
up, I mean, it's just more stuff
to write down.
39
00:03:25 --> 00:03:29
And, you're multiplying these
things, and all those vertical
40
00:03:29 --> 00:03:33
bars.
Since they don't even have a
41
00:03:33 --> 00:03:36
sense to the vertical bar,
it gets messy.
42
00:03:36 --> 00:03:40
So, we just drop the vertical
bars there when it's in
43
00:03:40 --> 00:03:44
asymptotic notation.
So, E is order V^2 when it's a
44
00:03:44 --> 00:03:48
set of pairs,
because if it's a set of pairs,
45
00:03:48 --> 00:03:52
it's at most n choose two,
which is where it's at most n^2
46
00:03:52 --> 00:03:56
over 2, here it could be,
at most, sorry,
47
00:03:56 --> 00:04:00
V^2 over 2, here it's at most
V^2.
48
00:04:00 --> 00:04:07
And then, another property that
sometimes comes up is if the G
49
00:04:07 --> 00:04:11
is connected,
we have another bound,
50
00:04:11 --> 00:04:18
implies that the size of E is
at least the size of V minus
51
00:04:18 --> 00:04:22
one.
OK, so if it's connected,
52
00:04:22 --> 00:04:31
meaning, what does it mean to
have a graph that's connected?
53
00:04:31 --> 00:04:37
Yeah, there's a path from any
vertex to any other vertex in
54
00:04:37 --> 00:04:40
the graph.
That's what it means to be
55
00:04:40 --> 00:04:44
connected.
So if that's the case,
56
00:04:44 --> 00:04:50
that a number of edges is at
least the number of vertices
57
00:04:50 --> 00:04:53
minus one, OK?
And so, what that says,
58
00:04:53 --> 00:05:00
so one of the things we'll get
into, a fact that I just wanted
59
00:05:00 --> 00:05:04
to remind you,
is that in that case,
60
00:05:04 --> 00:05:09
if I look at log E,
OK, log of the number of edges,
61
00:05:09 --> 00:05:14
that is O of log V.
And by this,
62
00:05:14 --> 00:05:18
is omega of log V.
So, it's equal to theta of log
63
00:05:18 --> 00:05:21
V.
OK, so basically the number of,
64
00:05:21 --> 00:05:25
in the case of a connected
graph, the number of edges,
65
00:05:25 --> 00:05:30
and the number of vertices are
polynomially related.
66
00:05:30 --> 00:05:36
So, their logs are comparable.
OK, so that's helpful just to
67
00:05:36 --> 00:05:41
know because sometimes I just
get questions later on where
68
00:05:41 --> 00:05:45
people will say,
oh, you showed it was log E but
69
00:05:45 --> 00:05:51
you didn't show it was log V.
And I could point out that it's
70
00:05:51 --> 00:05:55
the same thing.
OK, so there's various ways of
71
00:05:55 --> 00:05:59
representing graphs in
computers, and I'm just going to
72
00:05:59 --> 00:06:04
cover a couple of the important
ones.
73
00:06:04 --> 00:06:11
There's actually more.
We'll see some more.
74
00:06:11 --> 00:06:20
So, the simplest one is what's
called an adjacency matrix.
75
00:06:20 --> 00:06:30
An adjacency matrix of the
graph, G, equals (V,E),
76
00:06:30 --> 00:06:40
where, for simplicity,
I'll let V be the set of
77
00:06:40 --> 00:06:52
integers from one up to n,
OK, is the n by n matrix A
78
00:06:52 --> 00:07:04
given by the ij-th at the entry
is simply one if the edge,
79
00:07:04 --> 00:07:20
ij, is in the edge set and zero
if ij is not in the edge set.
80
00:07:20 --> 00:07:22
OK, so it's simply the matrix
where you say,
81
00:07:22 --> 00:07:25
the ij entry is one if it's in
the matrix.
82
00:07:25 --> 00:07:27
So, this is,
in some sense,
83
00:07:27 --> 00:07:32
giving you the predicate for,
is there an edge from i to j?
84
00:07:32 --> 00:07:35
OK, remember,
predicate is Boolean formula
85
00:07:35 --> 00:07:39
that is either zero or one,
and in this case,
86
00:07:39 --> 00:07:45
you're saying it's one if there
is an edge from i to j and zero
87
00:07:45 --> 00:07:48
otherwise.
OK, sometimes you have edge
88
00:07:48 --> 00:07:52
weighted graphs,
and then sometimes what people
89
00:07:52 --> 00:07:56
will do is replace this by edge
weights.
90
00:07:56 --> 00:08:02
OK, it will be the weight of
the edge from i to j.
91
00:08:02 --> 00:08:12
So, let's just do an example of
that just to make sure that our
92
00:08:12 --> 00:08:22
intuition corresponds to our
mathematical definitions.
93
00:08:22 --> 00:08:33
So, here's an example graph.
Let's say that's our graph.
94
00:08:33 --> 00:08:37
So let's just draw the
adjacency the matrix.
95
00:08:37 --> 00:08:42
OK, so what this says:
is there's an edge from one to
96
00:08:42 --> 00:08:44
one?
And the answer is no.
97
00:08:44 --> 00:08:47
Is there an edge from one to
two?
98
00:08:47 --> 00:08:50
Yes.
Is there an edge from one to
99
00:08:50 --> 00:08:51
three here?
Yep.
100
00:08:51 --> 00:08:54
Is there an edge for one to
four?
101
00:08:54 --> 00:08:58
No.
Is there an edge from two until
102
00:08:58 --> 00:09:00
one?
No.
103
00:09:00 --> 00:09:02
Two to two?
No.
104
00:09:02 --> 00:09:04
Two to three?
Yes.
105
00:09:04 --> 00:09:06
Two to four?
No.
106
00:09:06 --> 00:09:13
No edges going out of three.
Edge from four to three,
107
00:09:13 --> 00:09:19
and that's it.
That's the adjacency matrix for
108
00:09:19 --> 00:09:23
this particular graph,
OK?
109
00:09:23 --> 00:09:33
And so, I can represent a graph
as this adjacency matrix.
110
00:09:33 --> 00:09:43
OK, when I represent it in this
way, how much storage do I need?
111
00:09:43 --> 00:09:52
OK, n^2 or V^2 because the size
is the same thing for V^2
112
00:09:52 --> 00:10:03
storage, OK, and that's what we
call a dense representation.
113
00:10:03 --> 00:10:06
OK, it works well when the
graph is dense.
114
00:10:06 --> 00:10:11
So, the graph is dense if the
number of edges is close to all
115
00:10:11 --> 00:10:14
of the edges possible.
OK, then this is a good
116
00:10:14 --> 00:10:18
representation.
But for many types of graphs,
117
00:10:18 --> 00:10:23
the number of edges is much
less than the possible number of
118
00:10:23 --> 00:10:26
edges, in which case we say the
graph is sparse.
119
00:10:26 --> 00:10:32
Can somebody give me an example
of a sparse graph?
120
00:10:32 --> 00:10:35
A class of graphs:
so, I want a class of graphs
121
00:10:35 --> 00:10:38
that as n grows,
the number of edges in the
122
00:10:38 --> 00:10:42
graph doesn't grow as the
square, but grows rather as
123
00:10:42 --> 00:10:45
something much smaller.
A linked list,
124
00:10:45 --> 00:10:48
so, a chain,
OK, if you look at it from a
125
00:10:48 --> 00:10:52
graph theoretically,
is a perfectly good example:
126
00:10:52 --> 00:10:56
only n edges in the chain for a
chain of length n.
127
00:10:56 --> 00:10:59
So therefore,
the number of edges would be
128
00:10:59 --> 00:11:02
order V.
And in particular,
129
00:11:02 --> 00:11:07
you'd only have one edge per
row here.
130
00:11:07 --> 00:11:10
What other graphs are sparse?
Yeah?
131
00:11:10 --> 00:11:16
Good, a planar graph,
a graph that can be drawn in a
132
00:11:16 --> 00:11:21
plane turns out that if it has V
vertices has,
133
00:11:21 --> 00:11:25
and V is at least three,
then it has,
134
00:11:25 --> 00:11:30
at most, three V minus six
edges.
135
00:11:30 --> 00:11:34
So, it turns out that's order V
edges again.
136
00:11:34 --> 00:11:38
What's another example of a
common graph?
137
00:11:38 --> 00:11:42
Yeah, binary tree,
or even actually any tree,
138
00:11:42 --> 00:11:49
you know, what's called a free
tree if you read the appendix,
139
00:11:49 --> 00:11:54
OK, a tree that just is a
connected graph that has no
140
00:11:54 --> 00:12:00
cycles, OK, is another example.
What's an example of a graph
141
00:12:00 --> 00:12:04
that's dense?
A complete graph,
142
00:12:04 --> 00:12:07
OK: it's all ones,
OK, or if you have edge
143
00:12:07 --> 00:12:11
weights, it would be a
completely filled in matrix.
144
00:12:11 --> 00:12:14
OK, good.
So, this is good for dense
145
00:12:14 --> 00:12:17
representation.
But sometimes you want to have
146
00:12:17 --> 00:12:22
a sparse representation so we
don't have to spend V^2 space to
147
00:12:22 --> 00:12:26
deal with all of the,
where most of it's going to be
148
00:12:26 --> 00:12:28
zeroes.
OK, it's sort of like,
149
00:12:28 --> 00:12:33
if we know why it's zero,
why bother representing it as
150
00:12:33 --> 00:12:40
zero?
So, one such representation is
151
00:12:40 --> 00:12:46
an adjacency list
representation.
152
00:12:46 --> 00:12:56
Actually, adjacency list of a
given vertex is the list,
153
00:12:56 --> 00:13:08
which we denote by Adj of V,
of vertices adjacent to V.
154
00:13:08 --> 00:13:12
OK, just in terms by their
terminology, vertices are
155
00:13:12 --> 00:13:17
adjacent, but edges are incident
on vertices.
156
00:13:17 --> 00:13:22
OK, so the incidence is a
relation between a vertex and an
157
00:13:22 --> 00:13:25
edge.
An adjacency is a relation
158
00:13:25 --> 00:13:31
between two vertices.
OK, that's just the language.
159
00:13:31 --> 00:13:36
Why they use to different
terms, I don't know,
160
00:13:36 --> 00:13:40
but that's what they do.
So, in the graph,
161
00:13:40 --> 00:13:45
for example,
the adjacency list for vertex
162
00:13:45 --> 00:13:52
one is just the list or the set
of two three because one has
163
00:13:52 --> 00:13:57
going out of one are edges to
two and three.
164
00:13:57 --> 00:14:02
The adjacency list for two is
just three, four,
165
00:14:02 --> 00:14:07
three.
It's the empty set,
166
00:14:07 --> 00:14:10
and for four,
it is three.
167
00:14:10 --> 00:14:13
OK, so that's the
representation.
168
00:14:13 --> 00:14:21
Now, if we want to figure out
how much storage is required for
169
00:14:21 --> 00:14:26
this representation,
OK, we need to understand how
170
00:14:26 --> 00:14:35
long the adjacency list is.
So, what is the length of an
171
00:14:35 --> 00:14:40
adjacency list of a vertex,
V?
172
00:14:40 --> 00:14:48
What name do we give to that?
It's the degree.
173
00:14:48 --> 00:14:57
So, in an undirected graph,
we call it the degree of the
174
00:14:57 --> 00:15:02
vertex.
This is undirected.
175
00:15:02 --> 00:15:07
OK, about here,
OK.
176
00:15:07 --> 00:15:12
So that's an undirected case.
In the directed case,
177
00:15:12 --> 00:15:18
OK, actually I guess the way we
should do this is say this.
178
00:15:18 --> 00:15:22
If the degree,
we call it the out degree for a
179
00:15:22 --> 00:15:25
digraph.
OK, so in a digraph,
180
00:15:25 --> 00:15:32
we have an out degree and an in
degree for each vertex.
181
00:15:32 --> 00:15:36
So here, the in degree is
three.
182
00:15:36 --> 00:15:41
Here, the out degree is two,
OK?
183
00:15:41 --> 00:15:50
So, one of the important lemma
that comes up is what's called
184
00:15:50 --> 00:15:56
the handshaking lemma.
OK, it's one of these
185
00:15:56 --> 00:16:01
mathematical lemmas.
186
00:16:01 --> 00:16:11
187
00:16:11 --> 00:16:16
And so, it comes from a story.
Go to a dinner party,
188
00:16:16 --> 00:16:21
and everybody at the dinner
party shakes other people's
189
00:16:21 --> 00:16:24
hands.
Some people may not shake
190
00:16:24 --> 00:16:29
anybody's hand.
Some people may shake several
191
00:16:29 --> 00:16:32
people's hands.
Nobody shakes hands with
192
00:16:32 --> 00:16:37
themselves.
And at some point during the
193
00:16:37 --> 00:16:41
dinner party,
the host goes around and counts
194
00:16:41 --> 00:16:45
up how many, the sum,
of the number of hands that
195
00:16:45 --> 00:16:48
each person has shaken.
OK, so he says,
196
00:16:48 --> 00:16:52
how many did you shake?
How many did you shake?
197
00:16:52 --> 00:16:55
How many did you shake?
He adds them up,
198
00:16:55 --> 00:17:00
OK, and that number is
guaranteed to be even.
199
00:17:00 --> 00:17:03
OK, that's the handshaking
lemma.
200
00:17:03 --> 00:17:08
Or, stated a little bit more
precisely, if I take for any
201
00:17:08 --> 00:17:13
graph the degree of the vertex,
and sum them all up,
202
00:17:13 --> 00:17:20
that's how many hands everybody
shook, OK, that's actually equal
203
00:17:20 --> 00:17:23
to always twice the number of
edges.
204
00:17:23 --> 00:17:26
So, why is that going to be
true?
205
00:17:26 --> 00:17:33
Why is that going to be twice
the number of edges?
206
00:17:33 --> 00:17:33
Yeah?
Yeah.
207
00:17:33 --> 00:17:39
Every time you put in an edge,
you add one to the degree of
208
00:17:39 --> 00:17:44
each person on each end.
So, it's just two different
209
00:17:44 --> 00:17:48
ways of counting up the same
number of edges.
210
00:17:48 --> 00:17:52
OK, I can go around,
and if you imagine that,
211
00:17:52 --> 00:17:56
that every time I count the
degree of the node,
212
00:17:56 --> 00:18:01
I put a mark on every edge.
Then, when I'm done,
213
00:18:01 --> 00:18:07
every edge has two marks on it,
one for each end.
214
00:18:07 --> 00:18:17
OK: a pretty simple theorem.
So, what that says is that for
215
00:18:17 --> 00:18:26
undirected graphs,
that implies that the adjacency
216
00:18:26 --> 00:18:35
list representation,
uses how much storage?
217
00:18:35 --> 00:18:39
OK, at most,
2E, so order E because that's
218
00:18:39 --> 00:18:42
not all.
Yeah, so you have to have the
219
00:18:42 --> 00:18:47
number of vertices plus order
the number of edges,
220
00:18:47 --> 00:18:53
OK, whether it's directed or
undirected because I may have a
221
00:18:53 --> 00:18:59
graph, say it has a whole bunch
of vertices and no edges,
222
00:18:59 --> 00:19:05
that's still going to cost me
order V, OK?
223
00:19:05 --> 00:19:09
So, it uses theta of V plus E
storage.
224
00:19:09 --> 00:19:15
And, it's basically the same
thing asymptotically.
225
00:19:15 --> 00:19:22
In fact, it's easier to see in
some sense for digraphs because
226
00:19:22 --> 00:19:27
for digraphs,
what I do is I just add up the
227
00:19:27 --> 00:19:33
out degrees, and that equal to
E, OK, if I add up the out
228
00:19:33 --> 00:19:39
degrees as equally.
In fact, this is kind of like
229
00:19:39 --> 00:19:41
it amortized analysis,
if you will,
230
00:19:41 --> 00:19:45
a book keeping analysis,
that if I'm adding up the total
231
00:19:45 --> 00:19:48
number of edges,
one way of doing it is
232
00:19:48 --> 00:19:50
accounting for a vertex by
vertex.
233
00:19:50 --> 00:19:54
OK, so for each vertex,
I basically can take each
234
00:19:54 --> 00:19:58
degree, and basically each
vertex, look at the degree,
235
00:19:58 --> 00:20:02
and that allocating of account
per edge, and then ending up
236
00:20:02 --> 00:20:06
with twice the number of edges,
that's exactly accounting type
237
00:20:06 --> 00:20:12
of analysis that we might do for
amortized analysis.
238
00:20:12 --> 00:20:17
OK, so we'll see that.
So, this is a sparse
239
00:20:17 --> 00:20:22
representation,
and it's often better than an
240
00:20:22 --> 00:20:25
adjacency matrix.
For example,
241
00:20:25 --> 00:20:32
you can imagine if the World
Wide Web were done with an
242
00:20:32 --> 00:20:37
adjacency matrix as opposed to,
essentially,
243
00:20:37 --> 00:20:44
with an adjacency list type of
representation.
244
00:20:44 --> 00:20:47
Every link on the World Wide
Web, I had to say,
245
00:20:47 --> 00:20:50
here are the ones that I'm
connected to,
246
00:20:50 --> 00:20:53
and here are all the ones I'm
not connected to.
247
00:20:53 --> 00:20:57
OK, that list of things you're
not connected to for a given
248
00:20:57 --> 00:20:59
page would be pretty
dramatically,
249
00:20:59 --> 00:21:03
show you that there is an
advantage to sparse
250
00:21:03 --> 00:21:06
representation.
On the other hand,
251
00:21:06 --> 00:21:13
one of the nice things about an
adjacency matrix representation
252
00:21:13 --> 00:21:19
is that each edge can be
represented with a single bit,
253
00:21:19 --> 00:21:24
whereas typical when I'm
representing things with an
254
00:21:24 --> 00:21:30
adjacency list representation,
how many bits am I going to
255
00:21:30 --> 00:21:35
need to represent each
adjacency?
256
00:21:35 --> 00:21:39
You'll need order log of V to
be able to name each different
257
00:21:39 --> 00:21:41
vertex.
OK, the log of the number is
258
00:21:41 --> 00:21:46
the number of bits that I need.
So, there are places where this
259
00:21:46 --> 00:21:50
is actually a far more efficient
representation.
260
00:21:50 --> 00:21:53
In particular,
if you have a very dense graph,
261
00:21:53 --> 00:21:56
OK, this may be a better way of
representing it.
262
00:21:56 --> 00:22:01
OK, the other thing I want you
to get, and we're going to see
263
00:22:01 --> 00:22:05
more of this in particular next
week, is that a matrix and a
264
00:22:05 --> 00:22:11
graph, there are two ways of
looking at the same thing.
265
00:22:11 --> 00:22:14
OK, and in fact,
there's a lot of graph theory
266
00:22:14 --> 00:22:17
that when you do things like
multiply the adjacency matrix,
267
00:22:17 --> 00:22:20
OK, and so forth.
So, there's a lot of
268
00:22:20 --> 00:22:24
commonality between graphs and
matrices, a lot of mathematics
269
00:22:24 --> 00:22:28
that if it applies for one,
it applies to the other.
270
00:22:28 --> 00:22:31
Do you have a question,
or just holding your finger in
271
00:22:31 --> 00:22:33
the air?
OK, good.
272
00:22:33 --> 00:22:37
OK, so that's all just review.
Now I want to get onto today's
273
00:22:37 --> 00:22:40
lecture.
OK, so any questions about
274
00:22:40 --> 00:22:42
graphs?
So, this is a good time to
275
00:22:42 --> 00:22:45
review appendix B.
there are a lot of great
276
00:22:45 --> 00:22:48
properties in there,
and in particular,
277
00:22:48 --> 00:22:52
there is a theorem that we're
going to cover today that we're
278
00:22:52 --> 00:22:56
going to talk about today,
which is properties of trees.
279
00:22:56 --> 00:23:00
Trees are very special kinds of
graphs, so I really want you to
280
00:23:00 --> 00:23:05
go and look to see what the
properties are.
281
00:23:05 --> 00:23:08
There is, I think,
something like six different
282
00:23:08 --> 00:23:11
definitions of trees that are
all equivalent,
283
00:23:11 --> 00:23:16
OK, and so, I think a very good
idea to go through and read
284
00:23:16 --> 00:23:20
through that theorem.
We're not going to prove it in
285
00:23:20 --> 00:23:23
class, but really,
provides a very good basis for
286
00:23:23 --> 00:23:27
the thinking that we're going to
be doing today.
287
00:23:27 --> 00:23:30
And we'll see more of that in
the future.
288
00:23:30 --> 00:23:33
OK, so today,
we're going to talk about
289
00:23:33 --> 00:23:38
minimum spanning trees.
OK, this is one of the world's
290
00:23:38 --> 00:23:42
most important algorithms.
OK, it is important in
291
00:23:42 --> 00:23:46
distributed systems.
It's one of the first things
292
00:23:46 --> 00:23:50
that almost any distributed
system tries to find is a
293
00:23:50 --> 00:23:55
minimum spanning tree of the
nodes that happened to be alive
294
00:23:55 --> 00:23:56
at any point,
OK?
295
00:23:56 --> 00:24:01
And one of the people who
developed an algorithm for this,
296
00:24:01 --> 00:24:04
we'll talk about this a little
bit later, OK,
297
00:24:04 --> 00:24:09
it was the basis of the billing
system for AT&T for many years
298
00:24:09 --> 00:24:16
while it was a monopoly.
OK, so very important kind of
299
00:24:16 --> 00:24:21
thing.
It's got a huge number of
300
00:24:21 --> 00:24:25
applications.
So the problem is the
301
00:24:25 --> 00:24:31
following.
You have a connected undirected
302
00:24:31 --> 00:24:35
graph,
G equals (V,E),
303
00:24:35 --> 00:24:42
with an edge weight function,
w, which maps the edges into
304
00:24:42 --> 00:24:50
weights that are real numbers.
And for today's lecture,
305
00:24:50 --> 00:24:56
we're going to make an
important assumption,
306
00:24:56 --> 00:25:02
OK, for simplicity.
The book does not make this
307
00:25:02 --> 00:25:08
assumption.
And so, I encourage you to look
308
00:25:08 --> 00:25:16
at the alternative presentation
or, because what they do in the
309
00:25:16 --> 00:25:22
book is much more general,
but for simplicity and
310
00:25:22 --> 00:25:29
intuition, I'm going to make
this a little bit easier.
311
00:25:29 --> 00:25:37
We're going to assume that all
edge weights are distinct.
312
00:25:37 --> 00:25:39
OK, all edge weights are
distinct.
313
00:25:39 --> 00:25:42
So what does that mean?
What does that mean that this
314
00:25:42 --> 00:25:46
function, w, what property does
the function,
315
00:25:46 --> 00:25:48
w, have if all edge weights are
distinct?
316
00:25:48 --> 00:25:51
Who remembers their discreet
math?
317
00:25:51 --> 00:25:53
It's injective.
OK, it's one to one.
318
00:25:53 --> 00:25:56
OK, it's not one to one and
onto necessarily.
319
00:25:56 --> 00:26:00
In fact, it would be kind of
hard to do that because that's a
320
00:26:00 --> 00:26:05
pretty big set.
OK, but it's one to one.
321
00:26:05 --> 00:26:09
It's injective.
OK, so that's what we're going
322
00:26:09 --> 00:26:14
to assume for simplicity.
OK, and the book,
323
00:26:14 --> 00:26:19
they don't assume that.
It just means that the way you
324
00:26:19 --> 00:26:24
have to state things is just a
little more precise.
325
00:26:24 --> 00:26:28
It has to be more technically
precise.
326
00:26:28 --> 00:26:33
So, that's the input.
The output is--
327
00:26:33 --> 00:26:44
The output is a spanning tree,
T, and by spanning tree,
328
00:26:44 --> 00:26:52
we mean it connects all the
vertices.
329
00:26:52 --> 00:27:02
OK, and it's got to have
minimum weight.
330
00:27:02 --> 00:27:08
OK, so we can write the weight
of the tree is going to be,
331
00:27:08 --> 00:27:14
by that, we meet the sum over
all edges that are in the tree
332
00:27:14 --> 00:27:18
of the weight of the individual
edges.
333
00:27:18 --> 00:27:24
OK, so here I'(V,E) done a
little bit of abusive notation,
334
00:27:24 --> 00:27:31
which is that what I should be
writing is w of the edge (u,v)
335
00:27:31 --> 00:27:37
because this is a mapping from
edges, which would give me a
336
00:27:37 --> 00:27:42
double parentheses.
And, you know,
337
00:27:42 --> 00:27:45
as you know,
I love to abuse notation.
338
00:27:45 --> 00:27:48
So, I'm going to drop that
extra parentheses,
339
00:27:48 --> 00:27:54
because we understand that it's
really the weight of the edge,
340
00:27:54 --> 00:27:57
OK, not the weight of the
ordered pair.
341
00:27:57 --> 00:28:02
So, that's just a little
notational convenience.
342
00:28:02 --> 00:28:05
OK, so one of the things,
when we do the take-home exam,
343
00:28:05 --> 00:28:09
notational convenience can make
the difference between having a
344
00:28:09 --> 00:28:12
horrible time writing up a
problem, and an easy time.
345
00:28:12 --> 00:28:16
So, it's worth thinking about
what kinds of notation you'll
346
00:28:16 --> 00:28:19
use in writing up solutions to
problems, and so forth.
347
00:28:19 --> 00:28:22
OK, and just in general,
a technical communication,
348
00:28:22 --> 00:28:25
you adopt good notation people
understand you.
349
00:28:25 --> 00:28:29
You adopt a poor notation:
nobody pays attention to what
350
00:28:29 --> 00:28:34
you're doing because they don't
understand what you're saying.
351
00:28:34 --> 00:28:38
OK, so let's do an example.
352
00:28:38 --> 00:28:45
353
00:28:45 --> 00:28:52
OK, so here's a graph.
I think for this,
354
00:28:52 --> 00:29:02
somebody asked once if I was
inspired by biochemistry or
355
00:29:02 --> 00:29:09
something, OK,
but I wasn't.
356
00:29:09 --> 00:29:11
I was just writing these things
down, OK?
357
00:29:11 --> 00:29:15
So, here's a graph.
And let's give us some edge
358
00:29:15 --> 00:29:16
weights.
359
00:29:16 --> 00:29:31
360
00:29:31 --> 00:29:34
OK, so there are some edge
weights.
361
00:29:34 --> 00:29:39
And now, what we want is we
want to find a tree.
362
00:29:39 --> 00:29:45
So a connected acyclic graph
such that every vertex is part
363
00:29:45 --> 00:29:49
of the tree.
But it's got to have the
364
00:29:49 --> 00:29:55
minimum weight possible.
OK, so can somebody suggest to
365
00:29:55 --> 00:30:03
me some edges that have to be in
this minimum spanning tree?
366
00:30:03 --> 00:30:05
Yeah, so nine,
good.
367
00:30:05 --> 00:30:09
Nine has to be in there
because, why?
368
00:30:09 --> 00:30:14
It's the only one connecting it
to this vertex,
369
00:30:14 --> 00:30:16
OK?
And likewise,
370
00:30:16 --> 00:30:22
15 has to be in there.
So those both have to be in.
371
00:30:22 --> 00:30:26
What other edges have to be in?
Which one?
372
00:30:26 --> 00:30:33
14 has to be it.
Why does 14 have to be in?
373
00:30:33 --> 00:30:40
Well, one of 14 and three has
to be in there.
374
00:30:40 --> 00:30:50
I want the minimum weight.
The one that has the overall
375
00:30:50 --> 00:30:57
smallest weight.
So, can somebody argue to me
376
00:30:57 --> 00:31:04
that three has to be in there?
Yeah?
377
00:31:04 --> 00:31:09
That's the minimum of two,
which means that if I had a,
378
00:31:09 --> 00:31:14
if you add something you said
was a minimum spanning tree that
379
00:31:14 --> 00:31:19
didn't include three,
right, and so therefore it had
380
00:31:19 --> 00:31:23
to include 14,
then I could just delete this
381
00:31:23 --> 00:31:28
edge, 14, and put in edge three.
And, I have something of lower
382
00:31:28 --> 00:31:34
weight, right?
So, three has to be in there.
383
00:31:34 --> 00:31:37
What other edges have to be in
there?
384
00:31:37 --> 00:31:43
Do a little puzzle logic.
Six and five have to be in
385
00:31:43 --> 00:31:46
there.
Why do they have to be in
386
00:31:46 --> 00:31:48
there?
387
00:31:48 --> 00:32:02
388
00:32:02 --> 00:32:05
Yeah, well, I mean,
it could be connected through
389
00:32:05 --> 00:32:08
this or something.
It doesn't necessarily have to
390
00:32:08 --> 00:32:11
go this way.
Six definitely has to be in
391
00:32:11 --> 00:32:14
there for the same reason that
three had to be,
392
00:32:14 --> 00:32:16
right?
Because we got two choices to
393
00:32:16 --> 00:32:19
connect up this guy.
And so, if everything were
394
00:32:19 --> 00:32:22
connected but it weren't,
12, I mean, and 12 was in
395
00:32:22 --> 00:32:24
there.
I could always,
396
00:32:24 --> 00:32:27
then, say, well,
let's connect them up this way
397
00:32:27 --> 00:32:31
instead.
OK, so definitely that's in
398
00:32:31 --> 00:32:35
there.
I still don't have everything
399
00:32:35 --> 00:32:37
connected up.
400
00:32:37 --> 00:32:50
401
00:32:50 --> 00:33:03
What else has to be in there
for minimum spanning tree?
402
00:33:03 --> 00:33:11
Seven, five,
and eight, why seven,
403
00:33:11 --> 00:33:22
five, and eight?
OK, so can we argue those one
404
00:33:22 --> 00:33:32
at a time?
Why does five have to be in
405
00:33:32 --> 00:33:37
there?
Yeah?
406
00:33:37 --> 00:33:41
OK, so we have four connected
components because we have this
407
00:33:41 --> 00:33:43
one, this one,
we actually have,
408
00:33:43 --> 00:33:46
yeah, this one here,
and this one,
409
00:33:46 --> 00:33:49
good.
We need at least three edges to
410
00:33:49 --> 00:33:53
connect them because each edge
is going to reduce the connected
411
00:33:53 --> 00:33:57
components by one.
OK, so we need three edges,
412
00:33:57 --> 00:33:59
and those are the three
cheapest ones.
413
00:33:59 --> 00:34:04
And they work.
That works, right?
414
00:34:04 --> 00:34:11
Any other edges are going to be
bigger, so that works.
415
00:34:11 --> 00:34:15
Good.
OK, and so, now do we have a
416
00:34:15 --> 00:34:19
spanning tree?
Everything is,
417
00:34:19 --> 00:34:24
we have one big connected graph
here, right?
418
00:34:24 --> 00:34:31
Is that what I got?
Hey, that's the same as what I
419
00:34:31 --> 00:34:35
got.
Life is predictable.
420
00:34:35 --> 00:34:41
OK, so, so everybody had the
idea of what a minimum spanning
421
00:34:41 --> 00:34:44
tree is, then,
out of this,
422
00:34:44 --> 00:34:49
OK, what's going on there?
So, let's first of all make
423
00:34:49 --> 00:34:53
some observations about this
puzzle.
424
00:34:53 --> 00:34:59
And what I want to do is remind
you about the optimal
425
00:34:59 --> 00:35:06
substructure property because it
turns out minimum spanning tree
426
00:35:06 --> 00:35:12
has a great optimal substructure
property.
427
00:35:12 --> 00:35:17
OK, so the setup is going to
be, we're going to have some
428
00:35:17 --> 00:35:22
minimum spanning tree.
Let's call it T.
429
00:35:22 --> 00:35:27
And, I'm going to show that
with the other edges in the
430
00:35:27 --> 00:35:32
graph, are not going to be
shown.
431
00:35:32 --> 00:35:37
OK, so here's a graph.
432
00:35:37 --> 00:35:54
433
00:35:54 --> 00:35:58
OK, so here's a graph.
It looks like the one I have my
434
00:35:58 --> 00:36:01
piece of paper here.
OK, so the idea is,
435
00:36:01 --> 00:36:05
this is some minimum spanning
tree.
436
00:36:05 --> 00:36:09
Now, we want to look at a
property of optimal
437
00:36:09 --> 00:36:13
substructure.
And the way I'm going to get
438
00:36:13 --> 00:36:17
that, is, I'm going to remove
some edge, (u,v),
439
00:36:17 --> 00:36:22
move an arbitrary edge,
(u,v), in the minimum spanning
440
00:36:22 --> 00:36:26
tree.
So, let's call this u and this
441
00:36:26 --> 00:36:29
V.
And so, we're removing this
442
00:36:29 --> 00:36:33
edge.
OK, so when I remove an edge in
443
00:36:33 --> 00:36:36
a tree, what happens to the
tree?
444
00:36:36 --> 00:36:39
What's left?
I have two trees left,
445
00:36:39 --> 00:36:41
OK?
I have two trees left.
446
00:36:41 --> 00:36:45
Now, proving that,
that's basically one of the
447
00:36:45 --> 00:36:50
properties in that appendix,
and the properties of trees
448
00:36:50 --> 00:36:55
that I want you to read,
OK, because you can actually
449
00:36:55 --> 00:37:00
prove that kind of thing rather
than it just being obvious,
450
00:37:00 --> 00:37:05
which is, OK?
OK, so we remove that.
451
00:37:05 --> 00:37:11
Then, T is partitioned into two
subtrees.
452
00:37:11 --> 00:37:15
And, we'll call them T_1 and
T_2.
453
00:37:15 --> 00:37:22
So, here's one subtree,
and here's another subtree.
454
00:37:22 --> 00:37:29
We'(V,E) partitioned it.
No matter what edge I picked,
455
00:37:29 --> 00:37:38
there would be two subtrees
that it's partitioned into.
456
00:37:38 --> 00:37:40
Even if the sub tree is a
trivial subtree,
457
00:37:40 --> 00:37:43
for example,
it just has a single node in it
458
00:37:43 --> 00:37:45
and no edges.
459
00:37:45 --> 00:37:58
460
00:37:58 --> 00:38:11
So, the theorem that we'll
prove demonstrates a property of
461
00:38:11 --> 00:38:24
optimal substructure.
T_1 is a minimum spanning tree
462
00:38:24 --> 00:38:31
for the graph,
G_1, E_1,
463
00:38:31 --> 00:38:43
a subgraph of G induced by the
vertices in T_1.
464
00:38:43 --> 00:38:55
OK, that is,
V_1 is just the vertices in T_1
465
00:38:55 --> 00:39:09
is what it means to be induced.
OK, so V_1 is the vertices in
466
00:39:09 --> 00:39:12
T_1.
So, in this picture,
467
00:39:12 --> 00:39:16
I didn't label it.
This is T_1.
468
00:39:16 --> 00:39:20
This is T_2.
In this picture,
469
00:39:20 --> 00:39:27
these are the vertices of T_1.
So, that's V_1,
470
00:39:27 --> 00:39:32
OK?
And, E_1 is the set of pairs of
471
00:39:32 --> 00:39:39
vertices, x and y,
that are the edges that are in
472
00:39:39 --> 00:39:47
E_1 such that both x and y
belong to V_1.
473
00:39:47 --> 00:39:49
OK, so I haven't shown the
edges of G here.
474
00:39:49 --> 00:39:52
But basically,
if an edge went from here to
475
00:39:52 --> 00:39:57
here, that would be in the E_1.
If it went from here to here,
476
00:39:57 --> 00:40:00
it would not.
And if it went from here to
477
00:40:00 --> 00:40:04
here, it would not.
OK, so the vertices,
478
00:40:04 --> 00:40:10
the subgraph induced by the
vertices of T_1 are just those
479
00:40:10 --> 00:40:17
that connect up things in T_1,
and similarly for T_2.
480
00:40:17 --> 00:40:27
481
00:40:27 --> 00:40:33
So, the theorem says that if I
look at just the edges within
482
00:40:33 --> 00:40:38
the graph here,
G_1, those that are induced by
483
00:40:38 --> 00:40:41
these vertices,
T_1 is, in fact,
484
00:40:41 --> 00:40:46
a minimum spanning tree for
that subgraph.
485
00:40:46 --> 00:40:51
That's what the theorem says.
OK, if I look over here
486
00:40:51 --> 00:40:58
conversely, or correspondingly,
if I look at the set of edges
487
00:40:58 --> 00:41:05
that are induced by this set of
vertices, the vertices in T_2,
488
00:41:05 --> 00:41:13
in fact, T_2 is a minimum
spanning tree on that subgraph.
489
00:41:13 --> 00:41:17
OK, OK, we can even do it over
here.
490
00:41:17 --> 00:41:21
If I took a look,
for example,
491
00:41:21 --> 00:41:27
at these, let's see,
let's say we cut out five,
492
00:41:27 --> 00:41:35
and if I cut out edge five,
that T_1 would be these four
493
00:41:35 --> 00:41:40
vertices here.
And, the point is that if I
494
00:41:40 --> 00:41:44
look at the subgraph induced on
that, that these edges here.
495
00:41:44 --> 00:41:48
In fact, the six,
eight, and three are all edges
496
00:41:48 --> 00:41:52
in a minimum spanning tree for
that subgraph.
497
00:41:52 --> 00:41:54
OK, so that's what the theorem
says.
498
00:41:54 --> 00:41:57
So let's prove it.
499
00:41:57 --> 00:42:09
500
00:42:09 --> 00:42:19
OK, and so what technique are
we going to use to prove it?
501
00:42:19 --> 00:42:28
OK, we learned this technique
last time: hint,
502
00:42:28 --> 00:42:33
hint.
It's something you do it in
503
00:42:33 --> 00:42:39
your text editor all the time:
cut and paste,
504
00:42:39 --> 00:42:45
good, cut and paste.
OK, so the weight of T I can
505
00:42:45 --> 00:42:51
express as the weight of the
edge I removed,
506
00:42:51 --> 00:42:57
plus the weight of T_1,
plus the weight of T_2.
507
00:42:57 --> 00:43:07
OK, so that's the total weight.
So, the argument is pretty
508
00:43:07 --> 00:43:13
simple.
Suppose that there were some
509
00:43:13 --> 00:43:20
T_1 prime that was better than
T_1 for G_1.
510
00:43:20 --> 00:43:31
Suppose I had some better way
of forming a spanning tree.
511
00:43:31 --> 00:43:42
OK, then I would make up a T
prime, which just contained the
512
00:43:42 --> 00:43:48
edges, (u,v),
and T_1 prime,
513
00:43:48 --> 00:43:53
union T_2.
So, I would take,
514
00:43:53 --> 00:44:05
if I had a better spanning
tree, a spanning tree of lower
515
00:44:05 --> 00:44:12
weight for T_1.
And I call that T_1 prime.
516
00:44:12 --> 00:44:17
I just substitute that and make
up a spanning tree that
517
00:44:17 --> 00:44:22
consisted of my edge,
(u,v), whatever works well for
518
00:44:22 --> 00:44:26
T_1 prime and whatever works
well for T.
519
00:44:26 --> 00:44:30
And, that would be a spanning
tree.
520
00:44:30 --> 00:44:36
And it would be better than T
itself was for G,
521
00:44:36 --> 00:44:44
OK, because the weight of these
is just as the weight for this,
522
00:44:44 --> 00:44:50
I now just get to use the
weight of T_1 prime,
523
00:44:50 --> 00:44:54
and that's less.
And so, therefore,
524
00:44:54 --> 00:45:02
the assumption that T was a
minimum spanning tree would be
525
00:45:02 --> 00:45:11
violated if I could find a
better one for the subpiece.
526
00:45:11 --> 00:45:16
So, we have this nice property
of optimal substructure.
527
00:45:16 --> 00:45:20
OK, I have subproblems that
exhibit optimal,
528
00:45:20 --> 00:45:25
if I have a globally optimal
solution to the whole problem
529
00:45:25 --> 00:45:31
within it, I can find optimal
solutions to subproblems.
530
00:45:31 --> 00:45:36
So, now the question is,
that's one hallmark.
531
00:45:36 --> 00:45:41
That's one hallmark of dynamic
programming.
532
00:45:41 --> 00:45:45
What about overlapping
subproblems?
533
00:45:45 --> 00:45:51
Do I have that property?
Do I have overlapping
534
00:45:51 --> 00:45:58
subproblems over here for this
type of problem?
535
00:45:58 --> 00:46:19
536
00:46:19 --> 00:46:20
So, imagine,
for example,
537
00:46:20 --> 00:46:22
that I'm removing different
edges.
538
00:46:22 --> 00:46:26
I look at the space of taking a
given edge, and removing it.
539
00:46:26 --> 00:46:30
It partitions it into two
pieces, and now I have another
540
00:46:30 --> 00:46:32
piece.
And I remove it,
541
00:46:32 --> 00:46:35
etc.
Am I going to end up getting a
542
00:46:35 --> 00:46:38
bunch of subproblems that are
similar in there?
543
00:46:38 --> 00:46:41
Yeah, I am.
OK, if I take out this one,
544
00:46:41 --> 00:46:43
then I take out,
say, this one here,
545
00:46:43 --> 00:46:46
and then I'll have another tree
here and here.
546
00:46:46 --> 00:46:51
OK, that would be the same as
if I had originally taken this
547
00:46:51 --> 00:46:53
out, and then taken that one
out.
548
00:46:53 --> 00:46:57
If I look at simple ordering of
taking out the edges,
549
00:46:57 --> 00:47:00
I'm going to end up with a
whole bunch of overlapping
550
00:47:00 --> 00:47:04
subproblems.
Yeah, OK.
551
00:47:04 --> 00:47:14
So then, what does that suggest
we use as an approach?
552
00:47:14 --> 00:47:18
Dynamic programming,
good.
553
00:47:18 --> 00:47:26
What a surprise!
Yes, OK, you could use dynamic
554
00:47:26 --> 00:47:33
programming.
But it turns out that minimum
555
00:47:33 --> 00:47:41
spanning tree exhibits an even
more powerful property.
556
00:47:41 --> 00:47:48
OK, so we'(V,E) got all the
clues for dynamic programming,
557
00:47:48 --> 00:47:57
but it turns out that there's
an even bigger clue that's going
558
00:47:57 --> 00:48:05
to help us to use an even more
powerful technique.
559
00:48:05 --> 00:48:11
And that, we call,
the hallmark for greedy
560
00:48:11 --> 00:48:13
algorithms.
561
00:48:13 --> 00:48:32
562
00:48:32 --> 00:48:41
And that is,
we have a thing called the
563
00:48:41 --> 00:48:53
greedy choice property,
which says that a locally
564
00:48:53 --> 00:49:03
optimal choice is globally
optimal.
565
00:49:03 --> 00:49:05
And, of course,
as all these hallmarks is the
566
00:49:05 --> 00:49:09
kind of thing you want to box,
OK, because these are the clues
567
00:49:09 --> 00:49:12
that you're going to be able to
do that.
568
00:49:12 --> 00:49:15
So, we have this property that
we call the greedy choice
569
00:49:15 --> 00:49:18
property.
I'm going to show you how it
570
00:49:18 --> 00:49:21
works in this case.
And when you have a greedy
571
00:49:21 --> 00:49:24
choice property,
it turns out you can do even
572
00:49:24 --> 00:49:29
better that dynamic programming.
OK, so when you see the two
573
00:49:29 --> 00:49:33
dynamic programming properties,
there is a clue that says
574
00:49:33 --> 00:49:36
dynamic programming,
yes, but also it says,
575
00:49:36 --> 00:49:41
let me see whether it also has
this greedy property because if
576
00:49:41 --> 00:49:46
it does, you're going to come up
with something that's even
577
00:49:46 --> 00:49:49
better than dynamic programming,
OK?
578
00:49:49 --> 00:49:53
So, if you just have the two,
you can usually do dynamic
579
00:49:53 --> 00:49:56
programming, but if you have
this third one,
580
00:49:56 --> 00:50:00
it's like, whoa!
Jackpot!
581
00:50:00 --> 00:50:04
OK, so here's the theorem we'll
prove to illustrate this idea.
582
00:50:04 --> 00:50:08
Once again, these are not,
all these hallmarks are not
583
00:50:08 --> 00:50:09
things.
They are heuristics.
584
00:50:09 --> 00:50:14
I can't give you an algorithm
to say, here's where dynamic
585
00:50:14 --> 00:50:16
programming works,
or here's where greedy
586
00:50:16 --> 00:50:20
algorithms work.
But I can sort of indicate when
587
00:50:20 --> 00:50:23
they work, the kind of structure
they have.
588
00:50:23 --> 00:50:32
OK, so here's the theorem.
So let's let T be the MST of
589
00:50:32 --> 00:50:40
our graph.
And, let's let A be any subset
590
00:50:40 --> 00:50:49
of V, so, some subset of
vertices.
591
00:50:49 --> 00:51:04
And now, let's suppose that
edge, (u,v), is the least weight
592
00:51:04 --> 00:51:17
edge connecting our set A to A
complement, that is,
593
00:51:17 --> 00:51:27
V minus A.
Then the theorem says that
594
00:51:27 --> 00:51:39
(u,v) is in the minimum spanning
tree.
595
00:51:39 --> 00:51:43
So let's just take a look at
our graph over here and see if
596
00:51:43 --> 00:51:45
that's, in fact,
the case.
597
00:51:45 --> 00:51:49
OK, so let's take,
so one thing I could do for A
598
00:51:49 --> 00:51:53
is just take a singleton node.
So, I take a singleton node,
599
00:51:53 --> 00:51:56
let's say this guy here,
that can be my A,
600
00:51:56 --> 00:52:00
and everything else is V minus
A.
601
00:52:00 --> 00:52:04
And I look at the least weight
edge connecting this to
602
00:52:04 --> 00:52:07
everything else.
Well, there are only two edges
603
00:52:07 --> 00:52:10
that connect it to everything
else.
604
00:52:10 --> 00:52:15
And the theorem says that the
lighter one is in the minimum
605
00:52:15 --> 00:52:17
spanning tree.
Hey, I win.
606
00:52:17 --> 00:52:21
OK, if you take a look,
every vertex that I pick,
607
00:52:21 --> 00:52:25
the latest edge coming out of
that vertex is in the minimum
608
00:52:25 --> 00:52:29
spanning tree.
OK, the lightest weight edge
609
00:52:29 --> 00:52:35
coming out, but that's not all
the edges that are in here.
610
00:52:35 --> 00:52:39
OK, or let's just imagine,
let's take a look at these
611
00:52:39 --> 00:52:43
three vertices connected to this
set of vertices.
612
00:52:43 --> 00:52:46
I have three edges is going
across.
613
00:52:46 --> 00:52:50
The least weight one is five.
That's the minimum spanning
614
00:52:50 --> 00:52:53
tree.
Or, I can cut it this way.
615
00:52:53 --> 00:52:57
OK, the ones above one,
the edges going down are seven,
616
00:52:57 --> 00:53:02
eight, and 14.
Seven is the least weight.
617
00:53:02 --> 00:53:04
It's in the minimum spanning
tree.
618
00:53:04 --> 00:53:08
So, no matter how I choose,
I could make this one in,
619
00:53:08 --> 00:53:10
this one out,
this one in,
620
00:53:10 --> 00:53:12
this one out,
this one in,
621
00:53:12 --> 00:53:14
this one out,
this one in,
622
00:53:14 --> 00:53:18
this one out,
take a look at what all the
623
00:53:18 --> 00:53:21
edges are.
Which ever one to the least
624
00:53:21 --> 00:53:24
weight: it's in the minimum
spanning tree.
625
00:53:24 --> 00:53:28
So, in some sense,
that's a local property because
626
00:53:28 --> 00:53:34
I don't have to look at what the
rest of the tree is.
627
00:53:34 --> 00:53:38
I'm just looking at some small
set of vertices if I wish,
628
00:53:38 --> 00:53:42
and I say, well,
if I wanted to connect that set
629
00:53:42 --> 00:53:46
of vertices to the rest of the
world, what would I pick?
630
00:53:46 --> 00:53:50
I'd pick the cheapest one.
That's the greedy approach.
631
00:53:50 --> 00:53:53
It turns out,
that wins, OK,
632
00:53:53 --> 00:53:57
that picking that thing that's
locally good for that subset,
633
00:53:57 --> 00:54:04
A, OK, is also globally good.
OK, it optimizes the overall
634
00:54:04 --> 00:54:09
function.
That's what the theorem says,
635
00:54:09 --> 00:54:13
OK?
So, let's prove this theorem.
636
00:54:13 --> 00:54:20
Any questions about this?
OK, let's prove this theorem.
637
00:54:20 --> 00:54:27
So, we have (u,v) is the least
weight edge connecting A to D
638
00:54:27 --> 00:54:32
minus A.
So, let's suppose that this
639
00:54:32 --> 00:54:40
edge, (u,v), is not in the
minimum spanning tree.
640
00:54:40 --> 00:54:45
OK, let's suppose that somehow
there is a minimum spanning tree
641
00:54:45 --> 00:54:50
that doesn't include this least
weight edge.
642
00:54:50 --> 00:54:55
OK, so what technique you think
will use to prove to get a
643
00:54:55 --> 00:54:58
contradiction here?
Cut and paste,
644
00:54:58 --> 00:55:04
good.
Yeah, we're going to cut paste.
645
00:55:04 --> 00:55:08
OK, we're going to cut and
paste.
646
00:55:08 --> 00:55:14
So here, I did an example.
OK, so --
647
00:55:14 --> 00:55:40
648
00:55:40 --> 00:55:42
OK, and so I'm going to use the
notation.
649
00:55:42 --> 00:55:44
I'm going to color some of
these in.
650
00:55:44 --> 00:56:05
651
00:56:05 --> 00:56:10
OK, and so my notation here is
this is an element of A,
652
00:56:10 --> 00:56:14
and color it in.
It's an element of V minus A.
653
00:56:14 --> 00:56:18
OK, so if it's not colored it,
that's an A.
654
00:56:18 --> 00:56:21
This is my minimum spanning
tree.
655
00:56:21 --> 00:56:27
Once again, I'm not showing the
overall edges of all the graphs,
656
00:56:27 --> 00:56:30
but they're there,
OK?
657
00:56:30 --> 00:56:33
So, my edge,
(u,v), which is not my minimum
658
00:56:33 --> 00:56:38
spanning tree I say,
let's say is this edge here.
659
00:56:38 --> 00:56:42
It's an edge from u,
u as in A, v as in V minus A.
660
00:56:42 --> 00:56:48
OK, so everybody see the setup?
So, I want to prove that this
661
00:56:48 --> 00:56:52
edge should have been in the
minimum spanning tree,
662
00:56:52 --> 00:56:58
OK, that the contention that
this is a minimum spanning tree,
663
00:56:58 --> 00:57:02
and does include (u,v) is
wrong.
664
00:57:02 --> 00:57:05
So, what I want to do,
that, is I have a tree here,
665
00:57:05 --> 00:57:08
T, and I have two vertices,
u and v, and in a tree,
666
00:57:08 --> 00:57:12
between any two vertices there
is a unique, simple path:
667
00:57:12 --> 00:57:16
simple path meaning it doesn't
go back and forth and repeat
668
00:57:16 --> 00:57:18
edges or vertices.
OK, there's a unique,
669
00:57:18 --> 00:57:23
simple path from u to v.
So, let's consider that path.
670
00:57:23 --> 00:57:42
671
00:57:42 --> 00:57:46
OK, and the way that I know
that that path exists is because
672
00:57:46 --> 00:57:51
I'(V,E) read appendix B of the
textbook, section B.5.1,
673
00:57:51 --> 00:57:56
OK, which has this nice theorem
about properties of trees.
674
00:57:56 --> 00:58:00
OK, so that's how I know that
there exists a unique,
675
00:58:00 --> 00:58:05
simple path.
OK, so now we're going to do is
676
00:58:05 --> 00:58:09
take a look at that path.
So in this case,
677
00:58:09 --> 00:58:13
it goes from here,
to here, to here,
678
00:58:13 --> 00:58:16
to here.
And along that path,
679
00:58:16 --> 00:58:22
there must be a point where I
connect from a vertex in A to a
680
00:58:22 --> 00:58:25
vertex in V minus A.
Why?
681
00:58:25 --> 00:58:32
Well, because this is in A.
This is in V minus A.
682
00:58:32 --> 00:58:42
So, along the path somewhere,
there must be a transition.
683
00:58:42 --> 00:58:52
OK, they are not all in A,
OK, because in particular,
684
00:58:52 --> 00:58:58
V isn't.
OK, so we're going to do is
685
00:58:58 --> 00:59:09
swap (u,v) with the first edge
on this path that connects a
686
00:59:09 --> 00:59:18
vertex in A to a vertex in V
minus A.
687
00:59:18 --> 00:59:20
So in this case,
it's this edge here.
688
00:59:20 --> 00:59:24
I go from A to V minus A.
In general, I might be
689
00:59:24 --> 00:59:28
alternating many times,
OK, and I just picked the first
690
00:59:28 --> 00:59:32
one that I encounter.
OK, that this guy here.
691
00:59:32 --> 00:59:36
And what I do is I put this
edge in.
692
00:59:36 --> 00:59:38
OK, so then,
what happens?
693
00:59:38 --> 00:59:42
Well, the edge,
(u,v), is the lightest thing
694
00:59:42 --> 00:59:46
connecting something in A to
something in V minus A.
695
00:59:46 --> 00:59:49
So that means,
in particular,
696
00:59:49 --> 00:59:53
it's lighter than this edge,
has lower weight.
697
00:59:53 --> 00:59:57
So, by swapping this,
I'(V,E) created a tree with
698
00:59:57 --> 1:00:02.198
lower overall weight,
contradicting the assumption
699
1:00:02.198 --> 1:00:08
that this other thing was a
minimum spanning tree.
700
1:00:08 --> 1:00:14.219
OK: so, a lower weight spanning
tree than T results,
701
1:00:14.219 --> 1:00:18
and that's a contradiction --
702
1:00:18 --> 1:00:25
703
1:00:25 --> 1:00:33.01
-- than T results.
And that's a contradiction,
704
1:00:33.01 --> 1:00:36.57
OK?
How are we doing?
705
1:00:36.57 --> 1:00:44.225
Everybody with me?
OK, now we get to do some
706
1:00:44.225 --> 1:00:46.895
algorithms.
Yea!
707
1:00:46.895 --> 1:00:55.439
So, we are going to do an
algorithm called Prim's
708
1:00:55.439 --> 1:01:01.853
algorithm.
Prim eventually became a very
709
1:01:01.853 --> 1:01:07.069
high-up at AT&T because he
invented this algorithm for
710
1:01:07.069 --> 1:01:12.187
minimum spanning trees,
and it was used in all of the
711
1:01:12.187 --> 1:01:15.73
billing code for AT&T for many
years.
712
1:01:15.73 --> 1:01:21.438
He was very high up at Bell
Labs back in the heyday of Bell
713
1:01:21.438 --> 1:01:24.784
Laboratories.
OK, so it just shows,
714
1:01:24.784 --> 1:01:30
all you have to do is invent an
algorithm.
715
1:01:30 --> 1:01:36.702
You too can be a president of a
corporate monopoly.
716
1:01:36.702 --> 1:01:43.807
Of course, the government can
do things to monopolies,
717
1:01:43.807 --> 1:01:49.438
but anyway, if that's your
mission in life,
718
1:01:49.438 --> 1:01:55.202
invent an algorithm.
OK, so here's the idea.
719
1:01:55.202 --> 1:02:03.648
What we're going to do is we're
going to maintain V minus A as a
720
1:02:03.648 --> 1:02:11.923
priority queue.
We'll call it Q.
721
1:02:11.923 --> 1:02:26.076
And each vertex,
we're going to key each vertex
722
1:02:26.076 --> 1:02:39.923
in Q with the weight of the
least weight edge,
723
1:02:39.923 --> 1:02:53.28
connecting it to a vertex in A.
So here's the code.
724
1:02:53.28 --> 1:03:00
So, we're going to start out
with Q being all vertices.
725
1:03:00 --> 1:03:03.873
So, we start out with A being,
if you will,
726
1:03:03.873 --> 1:03:07.93
the empty set.
OK, and what we're going to do
727
1:03:07.93 --> 1:03:13.095
it is the least weight edge,
therefore, for everything in
728
1:03:13.095 --> 1:03:18.536
the priority queue is basically
going to be infinity because
729
1:03:18.536 --> 1:03:23.7
none of them have any edges.
The least weight edge to the
730
1:03:23.7 --> 1:03:29.325
empty set is going to be empty.
And then, we're going to start
731
1:03:29.325 --> 1:03:33.958
out with one guy.
We'll call him S,
732
1:03:33.958 --> 1:03:39.489
which will set to zero for some
arbitrary S in V.
733
1:03:39.489 --> 1:03:45.135
And then, the main part of the
algorithm kicks in.
734
1:03:45.135 --> 1:03:51.703
So that's our initialization.
OK, when we do the analysis,
735
1:03:51.703 --> 1:03:58.271
I'm going to write some stuff
on the left hand side of the
736
1:03:58.271 --> 1:04:04.406
board.
So if you're taking notes,
737
1:04:04.406 --> 1:04:14.406
you may want to also leave a
little bit of space on the left
738
1:04:14.406 --> 1:04:22.711
hand side of your notes.
So, while Q is not empty,
739
1:04:22.711 --> 1:04:30
we get the smallest element out
of it.
740
1:04:30 --> 1:04:41
741
1:04:41 --> 1:04:43
And then we do some stuff.
742
1:04:43 --> 1:05:19
743
1:05:19 --> 1:05:21.97
That's it.
And the only thing I should
744
1:05:21.97 --> 1:05:25.503
mention here is,
OK, so let's just see what's
745
1:05:25.503 --> 1:05:28.875
going on here.
And then we'll run it on the
746
1:05:28.875 --> 1:05:32.256
example.
OK, so what we do is we take
747
1:05:32.256 --> 1:05:36.609
out the smallest element out of
the queue at each step.
748
1:05:36.609 --> 1:05:40.156
And then for each step in the
adjacency list,
749
1:05:40.156 --> 1:05:43.783
in other words,
everything for which I have an
750
1:05:43.783 --> 1:05:46.846
edge going from v to u,
we take a look,
751
1:05:46.846 --> 1:05:51.44
and if v is still in our set V
minus A, so things we'(V,E)
752
1:05:51.44 --> 1:05:54.261
taken out are going to be part
of A.
753
1:05:54.261 --> 1:05:57.163
OK, every time we take
something out,
754
1:05:57.163 --> 1:06:02
that's going to be a new A that
we construct.
755
1:06:02 --> 1:06:04.258
At every step,
we want to find,
756
1:06:04.258 --> 1:06:08.4
what's the cheapest edge
connecting that A to everything
757
1:06:08.4 --> 1:06:11.035
else?
We basically are going to take
758
1:06:11.035 --> 1:06:15.025
whatever that cheapest thing is,
OK, add that edge in,
759
1:06:15.025 --> 1:06:19.242
and now bring that into A and
find the next cheapest one.
760
1:06:19.242 --> 1:06:22.103
And we just keep repeating the
process.
761
1:06:22.103 --> 1:06:25.567
OK, we'll do it on the example.
And what we do,
762
1:06:25.567 --> 1:06:28.955
is every time we bring it in,
I keep track of,
763
1:06:28.955 --> 1:06:34
what was the vertex responsible
for bringing me in.
764
1:06:34 --> 1:06:43.947
And what I claim is that at the
end, if I look at the set of
765
1:06:43.947 --> 1:06:52.209
these pairs that I'(V,E) made
here, V and pi of V,
766
1:06:52.209 --> 1:06:58.279
that forms the minimum spanning
tree.
767
1:06:58.279 --> 1:07:05.441
So let's just do this.
And, what's that?
768
1:07:05.441 --> 1:07:12.191
We're all set up.
So let's get rid of these guys
769
1:07:12.191 --> 1:07:20.234
here because we are going to
recompute them from scratch.
770
1:07:20.234 --> 1:07:30
OK, so you may want to copy the
graph over again in your notes.
771
1:07:30 --> 1:07:34.84
I was going to do it,
but it turned out,
772
1:07:34.84 --> 1:07:40.797
this is exactly the board is
going to erase this.
773
1:07:40.797 --> 1:07:47.127
OK, well let me just modify it.
OK, so we start out.
774
1:07:47.127 --> 1:07:54.574
We make everything be infinity.
OK, so that's where I'm going
775
1:07:54.574 --> 1:08:01.028
to keep the key value.
OK, and then what I'm going to
776
1:08:01.028 --> 1:08:07.952
do is find one vertex.
And I'm going to call him S.
777
1:08:07.952 --> 1:08:11.749
And I'm going to do this vertex
here.
778
1:08:11.749 --> 1:08:15.018
We'll call that S.
So basically,
779
1:08:15.018 --> 1:08:19.447
I now make him be zero.
And now, what I do,
780
1:08:19.447 --> 1:08:23.454
is I execute extract min.
So basically,
781
1:08:23.454 --> 1:08:28.199
what I'll do is I'll just shade
him like this,
782
1:08:28.199 --> 1:08:34
indicating that he has now
joined the set A.
783
1:08:34 --> 1:08:40.931
So, this is going to be A.
And this is element of V minus
784
1:08:40.931 --> 1:08:44.644
A.
OK, so then what we do is we
785
1:08:44.644 --> 1:08:47.986
take a look.
We extract him,
786
1:08:47.986 --> 1:08:53.433
and then for each edge in the
adjacency list,
787
1:08:53.433 --> 1:08:59.003
OK, so for each vertex in the
adjacency lists,
788
1:08:59.003 --> 1:09:05.315
that these guys here,
OK, we're going to look to see
789
1:09:05.315 --> 1:09:12
if it's still in Q,
that is, in V minus A.
790
1:09:12 --> 1:09:16.795
And if so, and its key value is
less than what the value is at
791
1:09:16.795 --> 1:09:20.254
the edge, there,
we're going to replace it by
792
1:09:20.254 --> 1:09:22.77
the edge value.
So, in this case,
793
1:09:22.77 --> 1:09:25.6
we're going to replace this by
seven.
794
1:09:25.6 --> 1:09:30.317
We're going to replace this by
15, and we're going to replace
795
1:09:30.317 --> 1:09:33.855
this by ten, OK,
because what we're interested
796
1:09:33.855 --> 1:09:39.608
in is, what is the cheapest?
Now, notice that everything in
797
1:09:39.608 --> 1:09:43.782
V minus A, that is,
what's in the priority queue,
798
1:09:43.782 --> 1:09:48.217
everything in there,
OK, now has its cheapest way of
799
1:09:48.217 --> 1:09:53.086
connecting it to the things that
I'(V,E) already removed,
800
1:09:53.086 --> 1:09:57.173
the things that are in A.
OK, and so now I just,
801
1:09:57.173 --> 1:10:01.608
OK, when I actually do that
update, there's actually
802
1:10:01.608 --> 1:10:07
something implicit going on in
this priority queue.
803
1:10:07 --> 1:10:10.636
And that is that I have to do a
decreased key.
804
1:10:10.636 --> 1:10:14.111
So, there's an implicit
decrease of the key.
805
1:10:14.111 --> 1:10:19.121
So, decreased key is a priority
queue operation that lowers the
806
1:10:19.121 --> 1:10:22.191
value of the key in the priority
queue.
807
1:10:22.191 --> 1:10:26.878
And so, that's implicitly going
on when I look at what data
808
1:10:26.878 --> 1:10:31.646
structure I'm going to use to
implement that priority queue.
809
1:10:31.646 --> 1:10:36.171
OK, so common data structures
for implementing a priority
810
1:10:36.171 --> 1:10:41.376
queue are a min heap.
OK, so I have to make sure that
811
1:10:41.376 --> 1:10:43.905
I'm actually doing this
operation.
812
1:10:43.905 --> 1:10:47.355
I can't just change it and not
affect my heap.
813
1:10:47.355 --> 1:10:51.111
So, there is an implicit
operation going on there.
814
1:10:51.111 --> 1:10:54.407
OK, now I repeat.
I find the cheapest thing,
815
1:10:54.407 --> 1:10:58.547
oh, and I also have to set,
now, a pointer from each of
816
1:10:58.547 --> 1:11:02.931
these guys back to u.
So here, this guy sets a
817
1:11:02.931 --> 1:11:07.114
pointer going this way.
This guy sets a pointer going
818
1:11:07.114 --> 1:11:11.298
this way, and this guy sets a
pointer going this way.
819
1:11:11.298 --> 1:11:16.206
That's my pi thing that's going
to keep track of who caused me
820
1:11:16.206 --> 1:11:20.873
to set my value to what it is.
So now, we go in and we find
821
1:11:20.873 --> 1:11:22.885
the cheapest thing,
again.
822
1:11:22.885 --> 1:11:25.62
And we're going to do it fast,
too.
823
1:11:25.62 --> 1:11:32.361
OK, this is a fast algorithm.
OK, so now we're going to go do
824
1:11:32.361 --> 1:11:36.481
this again.
So now, what's the cheapest
825
1:11:36.481 --> 1:11:39.843
thing to extract?
This guy here,
826
1:11:39.843 --> 1:11:42.987
right?
So, we'll take him out,
827
1:11:42.987 --> 1:11:47.542
OK, and now we update all of
his neighbors.
828
1:11:47.542 --> 1:11:51.771
So this guy gets five.
This guy gets 12.
829
1:11:51.771 --> 1:11:56.542
This guy gets nine.
This guy we don't update.
830
1:11:56.542 --> 1:12:02.722
We don't update him because
he's no longer in the priority
831
1:12:02.722 --> 1:12:07.464
queue.
And all of these guys now,
832
1:12:07.464 --> 1:12:12.297
we make point to where they're
supposed to point to.
833
1:12:12.297 --> 1:12:17.983
And, we're done with that step.
Now we find the cheapest one.
834
1:12:17.983 --> 1:12:22.437
What's the cheapest one now?
The five over here.
835
1:12:22.437 --> 1:12:24.807
Good.
So, we take him out.
836
1:12:24.807 --> 1:12:30.019
OK, we update the neighbors.
Here, yep, that goes to six
837
1:12:30.019 --> 1:12:34
now.
And, we have that pointer.
838
1:12:34 --> 1:12:39.684
And, this guy we don't do,
because he's not in there.
839
1:12:39.684 --> 1:12:44.604
This guy becomes 14,
and this guy here becomes
840
1:12:44.604 --> 1:12:47.774
eight.
So, we update that guy,
841
1:12:47.774 --> 1:12:52.803
make him be eight.
Did I do this the right way?
842
1:12:52.803 --> 1:12:57.395
Yeah, because pi is a function
of this guy.
843
1:12:57.395 --> 1:13:00.675
So basically,
this thing, then,
844
1:13:00.675 --> 1:13:04.938
disappears.
Yeah, did I have another one
845
1:13:04.938 --> 1:13:09.258
that I missed?
12, yes, good,
846
1:13:09.258 --> 1:13:12.584
it's removed,
OK, because pi is just a
847
1:13:12.584 --> 1:13:14.741
function.
And now I'm OK.
848
1:13:14.741 --> 1:13:18.516
OK, so now what do I do?
OK, so now my set,
849
1:13:18.516 --> 1:13:23.191
A, consists of these three
things, and now I want the
850
1:13:23.191 --> 1:13:26.786
cheapest edge.
I know it's in the minimum
851
1:13:26.786 --> 1:13:30.561
spanning tree.
So let me just greedily pick
852
1:13:30.561 --> 1:13:34.554
it.
OK, so what's the cheapest
853
1:13:34.554 --> 1:13:37.108
thing now?
This guy appear?
854
1:13:37.108 --> 1:13:39.466
Yeah, six.
So we take it.
855
1:13:39.466 --> 1:13:44.771
We go to update these things,
and nothing matters here.
856
1:13:44.771 --> 1:13:50.175
OK, nothing changes because
these guys are already in A.
857
1:13:50.175 --> 1:13:54.203
OK, so now the cheapest one is
eight here.
858
1:13:54.203 --> 1:13:56.856
Good.
So, we take eight out.
859
1:13:56.856 --> 1:14:01.656
OK, we update this.
Nothing to be done.
860
1:14:01.656 --> 1:14:04.97
This: nothing to be done.
This: oh, no,
861
1:14:04.97 --> 1:14:09.242
this one, instead of 14 we can
make this be three.
862
1:14:09.242 --> 1:14:14.212
So, we get rid of that pointer
and make it point that way.
863
1:14:14.212 --> 1:14:16.915
Now three is the cheapest
thing.
864
1:14:16.915 --> 1:14:21.1
So, we take it out,
and of course there's nothing
865
1:14:21.1 --> 1:14:24.239
to be done over there.
And now, last,
866
1:14:24.239 --> 1:14:26.506
I take nine.
And it's done.
867
1:14:26.506 --> 1:14:32
And 15: it's done.
And the algorithm terminates.
868
1:14:32 --> 1:14:36.972
OK, and as I look at,
now, all the edges that I
869
1:14:36.972 --> 1:14:43.135
picked, those are exactly all
the edges that we had at the
870
1:14:43.135 --> 1:14:48
beginning.
OK, let's do an analysis here.
871
1:14:48 --> 1:14:58
872
1:14:58 --> 1:15:06.316
OK, so let's see,
this part here costs me order
873
1:15:06.316 --> 1:15:11.197
V, right?
OK, and this part,
874
1:15:11.197 --> 1:15:16.983
let's see what we are doing
here.
875
1:15:16.983 --> 1:15:27.107
Well, we're going to go through
this loop how many times?
876
1:15:27.107 --> 1:15:32.711
V times.
It's V elements we put into the
877
1:15:32.711 --> 1:15:35.86
queue.
We are not inserting anything.
878
1:15:35.86 --> 1:15:39.795
We're just taking them out.
This goes V times,
879
1:15:39.795 --> 1:15:43.819
OK, and we do a certain number
of extract Mins.
880
1:15:43.819 --> 1:15:47.492
So, we're going to do order V
extract Mins.
881
1:15:47.492 --> 1:15:52.915
And then we go to the adjacency
list, and we have some constant
882
1:15:52.915 --> 1:15:55.801
things.
But we have these implicit
883
1:15:55.801 --> 1:16:00
decreased keys for this stuff
here.
884
1:16:00 --> 1:16:07.412
That's this thing here.
OK, and so how many implicit
885
1:16:07.412 --> 1:16:14.389
decreased keys do we have?
That's going to be the
886
1:16:14.389 --> 1:16:18.459
expensive thing.
OK, we have,
887
1:16:18.459 --> 1:16:25
in this case,
the degree of u of those.
888
1:16:25 --> 1:16:31.309
OK, so overall,
how many implicit decreased
889
1:16:31.309 --> 1:16:38.218
keys do we have?
Well, we have V times through.
890
1:16:38.218 --> 1:16:43.025
How big could the degree of u
be?
891
1:16:43.025 --> 1:16:48.433
OK, it could be as big as V,
order V.
892
1:16:48.433 --> 1:16:56.995
So, that's V^2 decreased use.
But we can do a better bound
893
1:16:56.995 --> 1:17:04.189
than that.
How many do we really have?
894
1:17:04.189 --> 1:17:11.948
Yeah, at most order E,
OK, because what am I doing?
895
1:17:11.948 --> 1:17:19.086
I'm summing up the degrees of
all the vertices.
896
1:17:19.086 --> 1:17:27
That's how many times I
actually execute that.
897
1:17:27 --> 1:17:34.322
So, I have order E,
implicit decreased keys.
898
1:17:34.322 --> 1:17:44.028
So the time overall is order V
times time for whatever the
899
1:17:44.028 --> 1:17:53.224
extract Min is plus E times the
time for decreased key.
900
1:17:53.224 --> 1:18:02.931
So now, let's look at data
structures, and we can evaluate
901
1:18:02.931 --> 1:18:14
for different data structures
what this formula gives us.
902
1:18:14 --> 1:18:21.492
So, we have different ways of
implementing a data structure.
903
1:18:21.492 --> 1:18:28.222
We have the cost of extract
Min, and of decreased key,
904
1:18:28.222 --> 1:18:32.636
and total.
So, the simplest way of
905
1:18:32.636 --> 1:18:38.369
implementing a data structure is
an unsorted array.
906
1:18:38.369 --> 1:18:44.904
If I have an unsorted array,
how much time does it take me
907
1:18:44.904 --> 1:18:51.668
to extract the minimum element?
If I have an unsorted array?
908
1:18:51.668 --> 1:18:58.433
Right, order V in this case
because it's an array of size V.
909
1:18:58.433 --> 1:19:06
And, to do a decreased key,
OK, I can do it in order one.
910
1:19:06 --> 1:19:14.245
So, the total is V^2,
good, order V^2 algorithm.
911
1:19:14.245 --> 1:19:22.666
Or, as people suggested,
how about a binary heap?
912
1:19:22.666 --> 1:19:33.017
OK, to do an extract Min in a
binary heap will cost me what?
913
1:19:33.017 --> 1:19:38.905
O of log V.
Decreased key will cost me,
914
1:19:38.905 --> 1:19:44.932
yeah, it turns out you can do
that in order log V because
915
1:19:44.932 --> 1:19:49.668
basically you just have to
shuffle the value,
916
1:19:49.668 --> 1:19:54.295
actually shuffle it up towards
the root, OK?
917
1:19:54.295 --> 1:19:58.708
Or at log V.
And, the total cost therefore
918
1:19:58.708 --> 1:20:01.717
is?
E log V, good.
919
1:20:01.717 --> 1:20:06.869
Which of these is better?
It depends, good.
920
1:20:06.869 --> 1:20:12.758
When is one better,
and when is the other better?
921
1:20:12.758 --> 1:20:18.401
Yeah, if it's a dense graph,
E is close to V^2,
922
1:20:18.401 --> 1:20:24.167
the array is better.
But if it's a sparse graph,
923
1:20:24.167 --> 1:20:33
and E is much smaller than V^2,
then the binary heap is better.
924
1:20:33 --> 1:20:37.824
So that motivated the invention
of a data structure,
925
1:20:37.824 --> 1:20:43.216
OK, called a Fibonacci Heap.
So, Fibonacci Heap is covered
926
1:20:43.216 --> 1:20:47.851
in Chapter 20 of CLRS.
We're not going to hold you
927
1:20:47.851 --> 1:20:53.148
responsible for the content,
but it's an interesting data
928
1:20:53.148 --> 1:20:57.878
structure because it's an
amortized data structure.
929
1:20:57.878 --> 1:21:01.851
And it turns out that it is
data structure,
930
1:21:01.851 --> 1:21:08
you can do extract Min in order
log V amortized time.
931
1:21:08 --> 1:21:12.747
And remarkably,
you can do decreased key in
932
1:21:12.747 --> 1:21:17.834
order one amortized.
So, when I plug those in,
933
1:21:17.834 --> 1:21:21
what do I get over here?
934
1:21:21 --> 1:21:34
935
1:21:34 --> 1:21:42.088
What's that going to be?
Plug that it here.
936
1:21:42.088 --> 1:21:52.296
It's going to be V times log V
plus E: E plus V log V.
937
1:21:52.296 --> 1:22:00
These are amortized,
so what's this?
938
1:22:00 --> 1:22:02.317
Trick question.
It's worst-case.
939
1:22:02.317 --> 1:22:05.979
It's not amortized over here.
These are amortized,
940
1:22:05.979 --> 1:22:08.745
but that's the beauty of
amortization.
941
1:22:08.745 --> 1:22:13.006
I can say it's going to be
worst case: E plus V log V over
942
1:22:13.006 --> 1:22:17.715
here, because when I add up the
amortized cost of my operations,
943
1:22:17.715 --> 1:22:20.48
it's an upper bound on the true
costs.
944
1:22:20.48 --> 1:22:24.292
OK, so that's why I say,
one of the beauties of this
945
1:22:24.292 --> 1:22:27.058
amortized analysis,
and in particular,
946
1:22:27.058 --> 1:22:31.692
being able to assign different
costs to different operations is
947
1:22:31.692 --> 1:22:37
I can just add them up and I get
my worst-case costs.
948
1:22:37 --> 1:22:40.565
So this is already V log V.
There are a couple other
949
1:22:40.565 --> 1:22:43.012
algorithms just before I let you
go.
950
1:22:43.012 --> 1:22:47.066
Kruskal's Algorithm in the book
uses another amortized data
951
1:22:47.066 --> 1:22:50.282
structure called a disjoint set
data structure,
952
1:22:50.282 --> 1:22:53.498
which also runs in E log V,
that is, this time:
953
1:22:53.498 --> 1:22:56.574
runs in this time,
the same as using a binary
954
1:22:56.574 --> 1:23:00
heap.
So, I'll refer you to the book.
955
1:23:00 --> 1:23:04.935
The best algorithm to date with
this problem is done by our own
956
1:23:04.935 --> 1:23:09.233
David Karger on the faculty here
with one of our former
957
1:23:09.233 --> 1:23:12.975
graduates, Phil Kline,
who is now a professor at
958
1:23:12.975 --> 1:23:17.353
Brown, and Robert Tarjan,
who is sort of like the master
959
1:23:17.353 --> 1:23:22.368
of all data structures who was a
professor at Princeton in 1993.
960
1:23:22.368 --> 1:23:26.189
OK, it's a randomized
algorithm, and it gives you
961
1:23:26.189 --> 1:23:32
order V plus E expected time.
OK, so that's the best to date.
962
1:23:32 --> 1:23:36.3
It's still open as to whether
there is a deterministic,
963
1:23:36.3 --> 1:23:40.679
there is worst-case bound,
whether there is a worst-case
964
1:23:40.679 --> 1:23:45.059
bound that is linear time.
OK, but there is a randomized
965
1:23:45.059 --> 1:23:47.369
to linear time,
and otherwise,
966
1:23:47.369 --> 1:23:51.509
this is essentially the best
bound without additional
967
1:23:51.509 --> 1:23:54.058
assumptions.
OK, very cool stuff.
968
1:23:54.058 --> 1:23:58.676
Next, we're going to see a lot
of these ideas of greedy and
969
1:23:58.676 --> 1:24:01
dynamic programming in practice.