[SQUEAKING]
[RUSTLING]
[CLICKING]
JASON KU: OK, welcome back to 6.006.
We're going to have a--
at this problem session, we're doing a quiz 2 review.
We're in a much bigger room today.
So I have a little bit more board space.
So I just wanted to go through what's going
to be covered on the exam.
First off, the scope, now, quiz 1 material
will be fair game for this quiz.
But it's not something that's going
to be explicitly emphasized or anything like that.
You should know that when we're storing graph data structures
that they can achieve certain running time
bounds and that kind of thing.
But we're really going to be concentrating on graphs,
the six lectures that we've had on graphs,
the two on unweighted graph algorithms,
and the four that we had on weighted graph algorithms that
covered the material that was covered in the two problem
sets, problem set 5 and problem set 6.
Now, usually, this material covers three problem sets
worth of material.
This term, it's covering two problem sets worth of material.
So just keep that in mind when you're studying
and you want to go look back on previous material.
In general, there's lots of graph problems
that we've talked about how to solve.
There's really a small number of graph algorithms.
But they can solve a lot of different problems.
And so we saw two algorithms to solve the graph reachability
problem, single source, what is reachable from me.
And I can only search a connected component of my graph
from me.
And the connected component from me
is actually upper-bounded, asymptotically
by the number of edges in my graph.
Because a spanning tree of my component has at least,
or has v plus 1 edges.
And so the number of vertices I can reach
is upper bounded by the number of edges in my graph,
asymptotically, anyway.
Then we talked about exploring an entire graph,
even if it's disconnected, not necessarily from a source,
just touching every vertex in a graph.
Of course, we can just touch every vertex in the graph.
i can look at the adjacency representation of my graph
and just go through it.
But this is, really, we're trying
to explore the whole graph, maybe count how many things are
reachable from each other in the graph.
And this is what we had--
we talked about exploring a graph
and counting the size of connected components
in a graph in an unweighted graph.
And we could do this via full BFS or full DFS.
It's basically putting a loop around one of these graph
readability algorithms to explore an entire graph
by exploring component by component.
And when I'm done with the component,
I find a vertex I haven't reached yet before
and explore it again.
And that still gets linear time in the graph V plus E.
Then we had special types of graphs,
directed graphs, directed acyclic graphs,
that we could use--
we proved this property of if we ran DFS, full DFS,
on that graph, we could actually get topological sort
of that graph, basically an ordering of the vertices
in the graph, such that all edges
go in one direction with respect to that ordering,
all forward in that ordering.
And we could actually use that to detect cycles
in a directed graph by just looking at the topological sort
order and seeing if--
look at the finishing time, the reverse finishing time
order of DFS, and just checking to see whether it
was a topological sorter.
Because any back edge there would correspond
to a cycle in our graph.
Because the proposition was that if our graph was acyclic,
doing this procedure would give us a topological order.
Then we had an algorithm, Bellman-Ford,
that was able to detect and find negative weight
cycles in our graph in a way that we've presented it
in lecture.
But normally, we concentrated on these two problems,
single source shortest paths, and some a little bit,
all pair shortest paths, first in the unweighted context,
and then in the weighted context for the majority
of the lectures.
So let's move on to what those single source shortest paths
algorithms were.
We had, kind of, to me, increasing in generality here.
The first restriction is DFS already
solves unweighted shortest paths in linear time.
In the unweighted context, that's all good.
But for weighted graphs, regardless of the weights,
if we had this very strong property on the graph
that the property-- that the graph didn't have any
directed cycles, then we could get
this in linear time via DAG relaxation.
And then for general graphs, we had these increasing--
or decreasing restrictions on the weights.
First, we had the restriction was that they were unweighted.
And that's the BFS constraint.
Or that they're non-negative, that's the Dijkstra constraint.
And if we have no constraints, that gives us Bellman-Ford.
And they increase in time.
In general, you want to choose an algorithm that's
higher on this list.
But sometimes, the algorithms higher on this list
don't apply.
If, on a quiz, you come to a graph for which--
so it's not a DAG, but you use DAG relaxation, that's
no longer a correct algorithm.
And so you're going to get fewer points
than if you happen to use an inefficient algorithm that
is correct.
So if I just-- whenever I saw shortest paths,
I used Bellman-Ford, it's the slowest thing.
That's probably going to be a correct algorithm.
It's not necessarily going to be the most efficient algorithm.
But you'll get more points, because it
is a correct algorithm than if you apply a faster
algorithm that doesn't apply to your problem,
because it's not going to solve it correctly.
Does that make sense?
So and then, in the last lecture we had,
we talked about all pairs shortest paths.
And really running a single source
shortest paths from each vertex is pretty
good in most circumstances.
We don't know how to do a lot better for a lot of these.
And then Johnson gives us, basically,
in this last line of our graph restrictions
and weight restrictions, where Bellman-Ford is right there we,
can actually get a speed up over V times Bellman Ford
by kind of two tricks.
Reweight the-- find if the graph has a negative weight cycles.
And if it doesn't, then there exists a reweighting
of this graph.
So that all the weights are non-negative.
But the shortest paths are preserved.
And so we can use Dijkstra V times
and get that running time instead.
So that's an overview of the content
that we've covered so far.
Just wanted to go--
just a brief overview of what these algorithms actually do.
DAG relaxation finds a topological order
of the thing using DFS looking at the reverse order
of the finishing times.
We proved that that's a reverse topological order.
And then we relax edges forward in that order,
because we know that we'll have found shortest paths distance
to everything before us before.
And we use that invariant to prove that this constructs it
in linear time.
BFS explores things in levels.
Increasing in the number of edges as we go out,
and I just process all of the ones in the same level
at the same time.
And Dijkstra generalizes this notion
by saying, well, I don't know all
of the things that are in the same level
per se, as I'm going.
But I can, using a clever use of a data structure,
find the next one I should process,
in kind of a DAG topological relaxation order,
to find shortest paths when the weights are non-negative.
Because in some sense, I know that once I've reached things
from a short distance, I will never have
to update their distance again.
That's kind of the invariant that we're
having with Dijkstra.
And then Bellman-Ford essentially
duplicates our graph so that each node corresponds
to reaching a vertex using, at most,
a certain number of edges.
And then that duplicated graph is a DAG.
And we can run DAG relaxation.
So that's the basic idea of all these algorithms.
When I approach problems on a quiz,
there's a couple of things to keep in mind.
There's kind of two things that we
have to worry about when you're looking at a graph
problem in this class.
The first thing is I might not see a graph in my problem.
I mean, on quiz 2, you know that there's going
to be a graph in your problem.
Because we covered graph algorithms on this quiz.
But in general, some of the word problems
you've been seeing on your problem sets,
there's no graph defined for you.
They give you an array of things, or a set of things,
or some connections between some things.
And that might be a graph that you want to make.
But kind of defining the graph is an important aspect
of that problem solving that is not necessarily something
that we've covered in lecture.
We've not emphasized that in lecture so much.
But it's something that you've had to do on your problem sets
and something that will appear on the quiz.
So part of this is a modeling context.
Can you look at a real-world situation, or maybe not
so real world, but non-mathematical context,
and you're trying to abstractify, put it
in the language of this class, the mathematics of this class,
make a graph.
So that solving one of the problems
that you know how to solve can adequately solve the word
problem that we gave you.
This is a modeling part.
So I always suggest, when you see a word problem on quiz 2
or on your problem set, it's that--
see if you can state, cleanly, an abstract problem.
Relate it that if you knew the answer
to that abstract problem, you could easily
solve your word problem, can make it a little easier
to decouple the complexity of the word problem.
Then you don't have to think about, I don't know,
various strange characters we come up
with in weird contexts and their weird conditions.
If you can map that to just a graph with a certain--
with certain properties, and solving an abstract problem
on that graph, that might be easier
for you to think about and apply the material in this class.
So you don't have to worry about,
oh, do I have to remember that roads are connected to five
other things, or do I have to--
maybe you're given as the input a sparse graph
or something like that.
That's a little easier to think about when
applying this material.
And so converting your problem into a finding a shortest path
problem, or finding a cycle, or finding
a topological sort, or connected components,
or a negative weight cycle, or any of these kinds of things
can make it easier for you to think about.
It's not fundamental material in this class that's super--
that we need to lecture on, but it is really important for you
when you're out in the real world
looking at problems to be able to make
that transformation from a non-mathematical context
to a mathematical.
I like to think of this as a modeling part of the problem.
But in general, once you've got that nice abstract problem,
then in general, you might have a graph.
But it might not be the graph that you--
the only graph you want when you're solving this problem.
That might be the input graph that you have.
But in general, a lot of the kind of tricks of this thing
is not modifying the algorithms that we gave you.
If you find yourself trying to modify the algorithms
we gave you, that we spent entire lectures on proving
their correctness, and things like that,
that's maybe not something you want to be doing on an exam.
Because then, you're going to be writing
pages of derivation and proof that these algorithms work.
This unit in particular is much more on the let's reduce
to some very powerful black box that we showed you
how it works.
And so because that's the framework here,
the way in which we introduce complexity into problems
is to make the graph non-obvious on the thing
that you're supposed to apply.
And so the graph that we give you is the input
may be different than the graph that you'll want
to use to solve the problem.
And here are some strategies that you
can use to modify a graph.
If you want to store state as you're traversing this graph,
you can expand the number of vertices
in your graph to keep track of what state I'm in.
I can have a different vertex for every possible state
I could be at that vertex.
In your problem session, you had this guy who's
drinking when he got to bars, or every third time,
and you need to remember how many times it's
been since I've been to a bar, I had a drunk in a bar.
And so you can duplicate the vertices to be
able to store that information.
Another thing, if you need to search from multiple locations
at the same time or search two multiple locations
at the same time, you can simulate that
without having to run an algorithm many times.
You can simulate that by adding an auxiliary node,
an extra node, in your graph, with edges to those sources
or to those things.
And run a single source shortest path algorithm
from that super node, sometimes we
call it, to get better performance.
It's kind of an efficiency.
We're adding efficiency by changing our graph
to fit the algorithms that we know how to solve efficiently.
And then the last thing, maybe it
helps to pre-process the graph in some way.
Some edges in the graph that we gave you might be forbidden
or may need to be traversed in one direction rather
than the other, even though the problem statement seems-- kind
of seems like they should be traversable
in either direction.
And doing this pre-processing of the graph could mean that you
break up your graph into--
your connected graph into a set of disconnected components
that you need to find.
Or makes a cyclic graph acyclic.
Or it pruned part of the graph that you don't want to explore.
I never want to touch on my money
my way to get to a location.
So these are all really common strategies that we have,
duplicating graph, adding auxiliary vertices or edges
to the graph.
I don't know the context in which we add edges.
That's an interesting question.
And then pre-processing, kind of filtering out the graph
or transforming it in some way to give it
properties that will allow us to solve the problem better.
So any questions about the problem-solving strategies that
we have or the content--
the kind of baseline content of this class?
This is kind of an overview of the lecture type
material, where we're not necessarily applying
this material in lecture.
The rest of this quiz review session
will be on applying this material to some--
a quiz from a previous term, some of those problems.
Yeah, question?
AUDIENCE: What are some common ways that people lose points
when they write down--
JASON KU: What are some common ways people lose points?
That's a great thing.
I'll add it to the notes when we post.
So common things that people lose points
on in this unit when they're solving problems,
you're given a word problem, and you don't define a graph.
It's as easy as that.
You start solving assuming that we know what
graph you're talking about.
When the implicit graph in the problem may or may not be
correct, but we don't--
there's no graph defined in the problem.
So you need to define a graph in the problem.
So that's the first thing.
The second thing is, a lot of times,
it's really useful, just as a strategy, when
you construct that graph, tell us
how many vertices and edges are in it, tell us if it's acyclic,
tell us what the weights are on each edge.
If you don't tell us these things, it's really hard for us
to base--
to judge your application of algorithms based on that graph.
If there's redundancy, even if you define, for every vertex
in my original graph I have 10 vertices, or blah, blah, blah,
and maybe you're adding a super node, or all these things,
it can be difficult for us to follow how many things are.
So you do that bookkeeping for us, your graders
are going to be a lot happier.
And so common mistakes, not defining a graph.
Not specifying your graph completely.
And then not--
I would also suggest that instead of just applying
an algorithm to a graph, that you clearly state the problem
you're solving on the graph first.
I want to solve this problem, because we've
given you a number of ways to solve
that problem on the graph.
And if you happen to choose the wrong algorithm,
then maybe that's--
separating off the problem from your implementation of how
you solve that problem can maybe help
you get some points for stating the problem
you're solving, even if you choose
the wrong or an inefficient way to solve it.
So that it can really help decouple some of the things
that we're going to give points on in this class.
So usually, what we're breaking up a graph rubric
on grading in did you describe a graph.
Did you modify it in a way that's
going to help you solve the problem?
Did you identify a problem that you
need to solve on this thing?
Did you use a correct algorithm to solve it?
Did you analyze the runtime, usually
involves is the size of my graph not too large,
and what is the running time based on that graph?
And then, argument of correctness in this unit
is basically I constructed a graph that has properties
so that shortest paths in this new graph
correspond to whatever it is that I
want in the original problem.
Some statement that links the problem
you're solving in your problem statement to the problem
you're solving on your graph.
That's a really good statement to have
to bring together correctness.
But aside from that statement, you're
mostly relying on the correctness of the algorithm.
So you don't need to do much on the correctness side.
But forgetting to analyze runtime is a big thing.
So those are a bunch of tips.
I'm going to add them to the end of this slide
after the lecture.
Great question.
Any other questions?
All right, let's get to solving problems.
All right, so these problems that we're going to solve
are from spring '18 Quiz 2, slightly modified.
But we're just going to go through them one at a time.
So the first problem we have, we have an image
of black and white squares.
So it's like a pixel grid.
You think of as a bitmap on your computer.
And what we say is each white pixel is contained in a blob.
But what is a blob?
I don't know.
I kind of am giving you an implicit definition
of what a blob is.
Two white pixels are in the same blob
if they share an edge of the grid.
So this kind of tells me this graph has an edge.
If these pictures are adjacent, they're both white.
That's what it means.
But an interesting part about that definition
is that it kind of is transitive.
If I have a white pixel that shares an edge with--
a white pixel A that shares a--
let's start writing things on the board, shall we?
Instead of me just talking at you.
Right, we have kind of a pixel grid here.
And I don't know how to do this with a chalkboard
because it's white versus black.
I guess I have to color in the white things.
These are all white.
All right, so these guys are in the same blob,
because they share an edge.
These guys are in the same blob.
But because they share an edge in the pixel grid,
these guys are also in the same blob.
Because if these are in the same blob,
and these are in the same blob, there's a transitivity argument
here.
This guy needs to be in the same blob as that guy.
And then it says that black pixels are not in any blob.
And so I'm given an n by m array.
I never remember which one comes first,
but we have dimensions of this thing is n by m.
So we have n times m pixels.
And so we're describing, essentially, a linear time
algorithm to compute the number of blobs in the image.
Why do I say linear time?
It's because for every pixel in my grid,
I needed to give you a specification of whether that
was white or black.
And so if I naively gave you the input
of this algorithm with a word per one of these pixels,
that would be the input size of my--
and so even though this looks quadratic,
the actual input size has--
is what we define as linear.
And so we're looking for a linear time algorithm
to count the number of blobs in the image.
OK, so what is--
this a little underspecified as a problem, I admit.
I hate to admit that I was involved
in this class at that time.
But the idea here is if these are--
share an edge, then everything--
the observation here is if I just draw this picture,
I notice that anything, kind of, that's
reachable through white-white connections
is going to be in the same blob.
So this is a blob, and this is a blob, and this is a blob,
and this is a blob.
But there's no path here.
This black part is not part of the blob.
Now, actually, there's nothing in this specification
that doesn't say--
that says that we couldn't have these things be
in the same blob.
So that's a little confusing, may be a source of error.
This is a source of error that I had when reading this problem
after a couple of years.
But when you are looking at a problem,
if everything could just be in the same blob,
then you just return 1, and this problem is not so interesting.
So the right way to interpret this problem, I mean, I
would not need n times m time.
I could just say 1.
So in some sense, I'd like there to be something
interesting in this problem.
And having these things that are not reachable from each other
be different blobs is kind of the more algorithmically
interesting thing to have.
And so what is this then?
This is just a pixel grid.
There's adjacencies.
There's connections between pixels.
But in particular, I really only care
about the connections between white pixels.
Hard to draw on here, but this component
has a graph that looks like this.
This component is a single vertex.
This one's an edge here.
And here's a singleton there.
And if we were to construct this graph,
we would have an unweighted graph,
such that the number of blobs in my image
would be the number of connected components in this graph.
See how I'm relating the thing that they're
asking for in the problem to a property of a graph
that I'm constructing?
So that's really the key part of argument of correctness
that we're looking for is for you
to make some kind of statement connecting the two.
Otherwise, you're just constructing a graph.
And I have no idea what you're doing to that graph.
You have to tell me.
It's about communication to us.
So how do I construct this graph?
Well, I can just loop through all of the pixels,
look at its four neighbors, at most four,
and if those things share--
are both white, then I add an edge.
We're going to essentially have a graph.
We're going to construct a graph.
I told you to do this.
So what is V here?
Then V is a vertex for each white pixel.
And I can just--
I mean, from the beginning, I can just
walk through all the things, find all the white vertices,
maybe I identify them uniquely by their xy
coordinates in this grid.
That's fine.
So now I have all the vertices.
And now I want to see what the edges are.
I can loop through the pixels again and just
look at it for possible adjacencies, see if any of them
are white, stick that edge in this set.
So edge is any two white pixels that share an edge.
So I can construct both of these things
in order n times m, because there's,
at most, that many vertices.
I just loop through them.
And the edges, for each pixel, I'm
only checking a constant number of things.
And I'm adding them to a set.
So the number of edges-- the size of the number of vertices
in my graph is, at most, n times m, and the number of edges
is, at most, n times m times 4.
It's upper bounded by that, because that's
the number of adjacencies I have in the graph.
You can probably get a better bound
in terms of the number of vertices.
It can be, at most, V times 4.
But that's a little stronger.
It doesn't really matter, we're trying
to get within the order n times m time bound.
So anything's fine here.
So that's the graph we construct.
And then we can run full BFS or full DFS.
We've identified a graph.
We've identified that we want to count
the number of connected components of my graph.
So idea, count connected components.
And then, for example, using full BFS or full DFS.
I wouldn't want you to write both of these algorithms there.
But when we write up our solutions,
we want them to cover the space of student solutions.
And so we will usually mention it.
You only have to mention one of them.
And because these run in linear time,
this also runs in n times m.
So all of these things are n times m.
And we're gold.
Any questions on this question?
Yeah.
AUDIENCE: If I were writing down my proof of correctness
and my proof of efficiency for this problem,
what sort of things would you be looking for?
JASON KU: Right, so when I'm writing down--
I've described to you the algorithm.
And so the question is what kinds of things
do I need to write down when I'm proving--
when I'm arguing running time of my algorithm and I'm arguing
correctness.
For running time, mostly just check out
the size of your graph.
State to me what the size of your graph is here.
In this case, it's order n times m.
And then I state what the running time
is of the algorithm that I have is applied to that.
And so because full BFS runs in O of V plus E time,
it's useful to actually write this down.
Even though right it's not in the terms
of our original problem variables,
it's useful to write this down.
So that if I mess up when plugging these variables in,
that you're showing your steps.
And so if you mess up arithmetically,
then we can still give you points.
But because the number of vertices in the graph
is n times m, the number of edges is n times m,
I add them together, it's still order n times m,
and that would be a sufficient for an argument of running
time.
And then as I was saying for correctness,
most of this, the correctness of this algorithm,
is relying on the fact that this thing
counts connected components correctly in my graph.
The key observation on a word problem that I--
or even a graph transformation problem,
is that the property that you're wanting
of the original graph or the original problem
corresponds to the thing you're solving in a new graph
that you've made.
And so here, an argument of correctness
that I would be looking for, that we might allow some weaker
statements, is that the number of blobs in the image
corresponds to the number of connected components
in this graph that I made.
That's really all it needs.
But I would like a connection between those values.
Now, why why would you be constructing this graph
and finding connected components if that wasn't what
your thought was?
I don't know.
But it's good, when you're communicating,
to make sure that that's abundantly clear that that's
why this is--
I mean, you should be able to argue why these things are--
that is connected component.
You could say something like because anything reachable is
in the same blob or something like that.
So that's problem 1.
You've got these nice mechanical boards.
So that's problem 1.
Problem 2 is a little funky.
It's been reworded a little bit from Spring '18.
So that I could point out some other features of this graph.
We're given a connected--
so connected is in bold, so that might be an important property
of our graph that we're trying to communicate
to you, a connected undirected graph
with strictly positive edge weights.
So they're mapping to the positive integers
where E is the same size as V. So the size of E
is the same as the size of V. So I have the same number of edges
as I have vertices.
We're trying to find a order V time algorithm to determine
a path from some vertex s to some vertex t
with minimum weight.
So what's the first thing I notice?
I notice that on this thing, I've got a graph, problem 2,
we've got a graph.
It's undirected.
It's connected.
It has this weird property that V equals E, or E equals V.
And weights are positive.
And we're asking for a single pair shortest paths.
We want a path, the shortest path, a shortest path
between two vertices.
Now, if we just were given this graph
and we wanted to solve this problem,
a very easy way to do that would be to just say, let's
run Dijkstra on the graph.
This is a graph.
It has only positive edge weights,
run Dijkstra on this graph.
How long does Dijkstra take on this graph?
Idea 1, run Dijkstra.
What's the problem with this?
It applies-- we're in the context of non-negative edge
weigths.
We can find single source paths from s
to everything else in the graph.
And using Dijkstra, it applies, it's a correct algorithm.
What's the difficulty with this algorithm?
AUDIENCE: Too slow.
JASON KU: Too slow, right?
That algorithm would run in O of V log V plus E. In this case,
these are the same.
So this is asymptotically smaller than this one.
It runs in V log V. So we're a little off.
We're off by a logarithmic factor in our running time.
But this would at least be a correct algorithm.
Whenever you approach a problem on an exam
and you see a really stupid polynomial algorithm that still
solves your problem correctly, you
might as well write that write that down in a line.
It doesn't hurt you that much to just write
that down, because it's possible we give you
points for that right.
But on your exam, notice why it is not sufficient.
Notice that this, this is V, notice
that this is not the running time bound we're looking for.
We've got to exploit something different.
Now, this doesn't seem--
this is a weighted context.
We have weighted paths.
It doesn't seem to be in one of the conditions
that we can get a linear time weighted single source shortest
path algorithm.
In particular, using BFS, we saw a transformation
where, as long as the sum of your weights
was linear in the combinatorial size of your graph
we could use BFS by making each edge
a bunch of undirected edges.
We don't have that in this context.
And this graph is undirected.
I mean, so it definitely contains cycles.
So we can't use DAG shortest paths.
So how the heck can we do this?
Well, what does this graph look like?
Here, I'm going to take a look at this condition, V equals E.
So what does this graph look like?
It's connected.
And it's V plus E. Well, how many edges does a tree have?
V minus 1.
So in a sense, a tree is the smallest number
of edges you can have in a connected graph.
So this has one more edge than a tree.
So really, what this looks like, what our graph tree looks
like is some kind of tree.
And somewhere, we've got an extra edge in this graph.
It's a tree plus an extra edge.
That's what our graph is.
So well, let's take a step back.
If I just had a tree, and I had a weighted graph here,
undirected, and the weights are all positive,
if any of the weights were negative,
how could I solve this problem?
Well, every edge is reachable from every vertex.
I can just go to that edge and traverse a negative weight
back and forth.
And my shortest pathway would be infinite for all our vertices.
That's not the case we have here.
We have positive edge weights only.
Which means shortest paths are simple.
And actually, there's only one simple path
between any pair of vertices in a tree.
I basically-- there's one thing I can do.
And in fact, if I took-- if this was s and this was t,
that's an x.
What am I doing?
OK t, if I just ran any unweighted short--
I mean, reachability algorithm, I would get a tree, a BFS
tree or a DFS tree.
It would visit vertices in some order.
Now actually, in a tree, I have to output
a tree that connects all the vertices.
And that would be this tree.
And so, in a sense, the paths that I got from BFS or DFS
in this graph would be exactly shortest paths.
I would just have to then go and add up all the path
edge weights along the edges.
That makes sense?
OK, so BFS or DFS in the unweighted context
can give me the shortest path in the weighted context.
Because there's only one simple path in this graph.
But we have a complication here.
That's not the question that we're asking.
We have an extra edge.
And now, we have a property where there's not just
one simple path to t.
There could be two simple paths.
I could go this way around the cycle.
Or I could go this way around the cycle.
So that's a complication.
But there's only one cycle.
If t is over here, there's only one path.
So if there is only one path, I'll be golden.
But if, basically, the cycle can be
reached between these two things,
I could have two simple paths.
That's the property.
We have the closest vertex on--
so this is the cycle.
There's a cycle here.
If this is the closest vertex to s,
and this is the closest vertex to t on the cycle,
then I could take either path around the cycle
to get from one to the other.
And that gives me my two paths.
But this path and this path, these
are completely edge disjoint.
In other words, any simple path from s to t,
if I find this vertex going through here,
it can only is one of these edges.
Because I can't come back to this vertex.
Once I go into it here, I got to go out one direction,
and I can't come back.
So it's only one of these two edges.
So the idea behind this algorithm
is I'm going to find the cycle.
Or in particular, I'm going to find this thing, s prime,
on the cycle, find the outgoing two edges here, remove one,
and then do my tree search.
Basically find the shortest path by running an undirected--
I mean, an unweighted reachability algorithm,
which will give me a path back to s, the only
simple path in that tree.
I get rid of this edge.
And I do that once.
And I do it again without this edge.
So that's the idea of my algorithm.
So how can I do--
I first have to find s prime.
How can I do that?
Well, I don't know what this edge is.
But if I ran an unweighted shortest path algorithm,
like BFS or DFS on here, I would get back a tree.
Some edge of my graph will not be in my tree.
Something like here, a shortest path to this--
I look through, I run--
algorithm, idea 2, first, find s prime.
OK, and I can find s prime by run, I don't know,
a single source shortest path unweighted--
I guess, run single source reachability
unweighted from s using BFS or DFS to explore
tree of my graph.
Then some edge is not in my tree of the graph, that
will exist on the cycle, kind of by definition.
It's connecting two parts of my tree.
Now, I can look at those two paths from here.
And the last one that they're in common from s is going
to be my split point, s prime.
It's the closest one to my source that is on the cycle.
Because I constructed this cycle here.
So I can find edge u, v, not in the parent tree.
So maybe this is u, v. Not in the parent tree.
And then, find last common vertex in paths from s to u
and s to v. That's going to give me my s prime.
And I can do that by--
I mean, these are each of linear size.
And I can just look at their prefix.
I can start from s.
I can walk forward until they diverge.
And the one before they diverge is s prime.
That's s prime right here.
Once I have s prime, I know what the edges
are when they diverge.
I remove one of those from the graph.
I do the same algorithm again to find a path to t.
And I do the same algorithm again to find a path to t.
And I see which one is shorter.
That's it.
There's only two of them.
And so I check.
Or they could be the same path, in which case
my t is actually before s prime on my cycle.
Does that makes sense?
So that's the idea.
The last thing is remove an edge from s prime.
I don't even have to be picky about this.
It has degree 3.
I can just run single source shortest paths on all of them
and take the min.
Remove each edge from s prime, remove--
that's rigth, for each edge from s, remove and run SSR from s.
And one of the paths there to t will be shortest, my shortest
path in the original graph.
Because it can't use more than two of those edges.
That's the claim.
And this runs in linear time, because what I'm doing
is I'm running single source reachability once, and maybe
two more times, or three more times,
a constant number of times on a graph that
has size v. And this prefix finding also only
takes order v. And so we're done.
OK, any questions about this problem?
No questions, all right, we will move on to--
what's up?
AUDIENCE: There's a hint in the title.
JASON KU: Yeah, there's a hint in the title.
Actually, the original version of this problem
said, instead of this E equals V specification,
it said there's only one cycle in the graph.
But it's in the context of undirected cycles,
as opposed to directed cycles, which is usually what
we talk about in this class.
We say that there's a negative edge weight cycle in the graph
if we can--
usually, we're talking about--
we're allowing non-simple cycles in this class.
So to remember this property about trees
and to enforce this property without talking
about cyclicity, I changed the condition
for this problem session, this review.
Yeah?
AUDIENCE: Could I also just run depth-first search
on this graph?
JASON KU: Could you just run depth-first search
on this graph to do what?
AUDIENCE: To find the shortest path.
JASON KU: To find the shortest path, right?
So depth-first search on this path,
if I ran it from s, when I got to s prime,
I would have a choice on what the next outgoing age to do.
So if I ran depth-first search for one of those choices,
I would find a path to t.
And then I would--
then I could run-- and that would find a path to t.
There's only two of them, or at most, two of them.
But then there's the possibility I missed this other path that
could be shorter.
AUDIENCE: How would I miss it if depth-first search has to go
through the other edge too?
JASON KU: It doesn't go through the other.
That's the point.
AUDIENCE: [INAUDIBLE].
JASON KU: It won't go through.
So depth-first search will actually go through this thing,
traverse an edge, go all the way around this cycle,
because everything here is reachable from here,
because it's an undirected graph.
It will reach back to here and then backtrack all the way.
So it will actually never traverse this last edge
here of the cycle.
That's something you can actually prove with DFS.
Now, you could actually, while you're running DFS,
try every possibility.
Because MY branching factor's at most 3 at some of these things.
So what I could do is--
or it could be at most four.
I could connect two things with the same branching.
But in general, it's a constant.
And with every choice DFS could make,
I could try all possibilities.
How many possibilities would that be?
You get a blow up of the degree of every vertex in my graph.
So the degree multiplied by each other.
That's the number of times I would have to run
DFS, which is exponential.
A constant degree-- so a constant
multiplied like 2 or 3, right, multiplied the times
is 3 to the V, which is exponential in the size
of my graph.
AUDIENCE: Can I have that if the size of E equals the size of V?
JASON KU: Sure, because I could still
have large branching for a large number of vertices.
OK, great question.
All right, cool, so that's that problem.
Problem 3, I have half an hour for the last two problems.
I think that should be fine.
This one's-- OK, this is Doh!-nut is the problem name.
Momer has just finished work at the FingSprield power plant
at a particular location p and needs to drive home
to a known location h.
But along the way, if his driving route ever
comes within driving distance k of a doughnut shop,
he won't be able to resist himself
and will have to go there and eat doughnuts.
And his wife Harge will be angry.
OK, maybe you can get the reference here.
Momer knows the layout of FingSprield,
which can be modeled as a set of n locations
with two-way roads of known driving distance connecting
some pairs of them.
And you may assume that no location is incident
to more than five roads.
So we've got a degree bound here.
As well as the location-- and he knows the locations that--
all the locations that contain doughnut shops.
There's, at most, d of them.
Describe an n log n time algorithm
to find the shortest driving route from the power plant
back to home that avoids driving within distance k of a doughnut
shop.
OK, so we got a couple variables in here.
We've got k.
We've got d.
But a running time bound only relies on n.
OK, I see shortest paths.
I see that--
I don't see an explicit mention of positive distances of the--
I see lengths.
They say he knows the known driving distance connecting
some pairs of locations.
So usually, I think if I were writing this problem now,
I would probably be a little bit more explicit
the distance is positive.
But that's something that you might come into contact with.
Distances are positive.
And so we can have negative distances here.
So I look at n log n.
I'm like, hey, what has a log in it in this unit?
Dijkstra, maybe I can use Dijkstra.
So let's see if we ran Dijkstra from p to h.
So we've got-- we've got a graph here.
We've got our graph.
So this is a word problem.
So there's no graph there.
So I have to define a graph.
So I'm going to define a graph V, E.
And we've got V. That's going to be
my set of locations, locations.
So this says there are order n of them.
There's actually n of them.
And then E, what are we going to have?
We're going to have--
it's a known pair of things, road, roads with weight
equal to driving, driving distance,
which, by my assumption, is going to be greater than 0.
Now, that's not stated explicitly.
But that would be a reasonable assumption for you
to make on your exam.
Because distances are positive.
We would probably be more explicit about that these days.
All right, so this is a graph I could make.
And I have a vertex s or a vertex p and a vertex h.
And I'm trying to find the shortest path between them,
shortest driving distance, driving route.
So I could run Dijkstra--
wait, so what do I know about this?
How many edges do I have in my graph?
I have, at most, five per vertex.
So this is upper bounded by 5 times V, which is order V.
So I have an order V sized graph.
That's a good thing, order n, because n
is the number of vertices.
And so if I were to just run Dijkstra on here
from p, doing Dijkstra on G from any s takes order n log n.
n log n plus n, n log n is bigger than n.
So that's a nice observation.
Maybe-- we have we can at least afford
to use Dijkstra in this problem to find shortest distances.
But what's the problem with a shortest distance found
by Dijkstra in this graph.
Doughnuts, like the entire point of the problem,
I need to avoid being too close to doughnut shops.
So we might have a doughnut shop here.
And we need to stay outside of that distance k.
Or we might have another doughnut shop here.
And so we got to find a path that kind of goes
around these doughnut shops.
So in other words, if I have a vertex in my graph
where I can reach a doughnut shop within--
shop within distance k, I can never visit that vertex.
Because then, Momer will not be able to resist himself
and will have to go eat a doughnut.
So that's the thing we're trying to avoid.
So how can we do this?
Well, here's a silly thing.
I could run Dijkstra from each of these vertices,
these doughnut shops, find all the things reachable in k
driving distance from them.
And then remove those vertices from the graph.
That's an idea.
But how long would that take?
That would need me to run Dijkstra d times,
because there's d doughnut shops.
I could run Dijkstra d times.
So that gives me a running time bound of I run d times
to filter out the graph, to modify this graph,
and then I do one more to find the shortest
path if there is one.
But in general, that's going to take d times n log n,
not n log n.
I have no bound on d, except that it's under n.
So it could be n.
And that would give me a bad running time.
So we're going to use a very similar trick
here to one of your--
I think a previous review session--
stop, there we go.
Is to, when you want to find things,
if we want to prune a graph from multiple locations,
one of the things we can do is, any tricks?
Supernode, right?
I can have a vertex.
Well, maybe I don't want to put it up yet.
OK, if I have all of these doughnut shops, what I can do
is provide a--
I guess, I guess these are unweighted, undirected edges.
Here, we can model all of those directed things
by two undirected edges.
It doesn't really matter.
But here, I don't want to be able to go back
to my supernode.
But what I'm going to do is I'm going
to add a supernode with edge weight, say,
0 to everything else.
And then, if I ran Dijkstra from the supernode,
and found all vertices reachable within distance k, well,
I didn't--
I didn't spend any of that distance going
through this first edge.
And I didn't come back to s, because these things
are directed into the things.
And so anything I write as going to be
within distance k of this doughnut shop,
but for all doughnut shops.
In some sense, I'm doing this search in parallel.
So this is the same trick that we
had in the you're looking through the sewer network,
or something, and they're trying to avoid monitors, or sensors,
or something like that.
We actually did this transformation
and then binary searched on a distance.
It was kind of involved.
But this is an easier example.
Now, you can actually generalize this further.
What if each doughnut shop had an amount that Momer liked it?
So if Momer is within a larger distance of a doughnut
shop he really likes, he still won't be able to resist.
But a doughnut shop that doesn't make very good donuts,
he'll be able to resist a shorter distance without having
to go to that doughnut shop.
So in other words, each one of these doughnut shops
has a different k, a different radius that Momer will allow.
Is there any way to generalize this technique
to be able to prune all of those vertices instead?
AUDIENCE: Put weights on it.
JASON KU: All of these had weight 0 before,
the same weight.
I mean, the algorithm would have worked
for any weight I put on all these edges,
as long as I searched a distance that weight plus k.
Here, I can just make the distance
of this frontier for each one of the doughnut shops
the same by modifying the distance of the incoming edge.
So I can set the length--
the weight to the doughnut shop with the largest radius to 0.
And then put the difference between the largest radius
to all the other ones, I put that is
the weight on the other edges.
And then we still have a graph with positive edge weights.
And I can run Dijkstra from this.
And that would generalize this problem and something
that we've done in some practice exams,
or in exams and problem sets in the past.
So that's another common way.
So we filter forbidden--
there's two d's in forbidden, or two d's.
There's a three d's, vertices, by using supernode
plus one run of Dijkstra.
Those are the extra letters.
So on your exam, you would probably
want to be a little bit more explicit.
This is a summary of the things that we just talked about.
I just talked 10 minutes about the algorithm.
But it doesn't hurt to add a summary at the top of what
you're going to write.
This is the approach that we're going to have.
We're going to filter out the vertices from G, essentially,
by running Dijkstra from each of these doughnut shops.
But we're going to do it in parallel
by adding this supernode.
Actually, on another recommendation
I have for you on an exam is that almost any problem
that we give you in this class, can get 80% to 90%
of the points by writing maybe three lines.
Almost, maybe not some of the data structures problems,
but almost any question in this class
can be solved with not all of the points,
but most of the points, by just writing a couple of lines.
That we know that you know how to solve the problem.
And this would be one of those situations.
Now, I would want, to give you full points,
I would want all the details here.
Did I construct a new graph?
I add this vertex here.
I have to add edges to each of the d things.
But I've only added d more edges and one more vertex.
So it still has this linear size in my input.
And then I want to say that I'm putting the weights on here
based on what the distance is.
Now, they're all the same weight,
because I don't have that generalization.
And then I run this thing.
And I remove all those graphs.
And I construct a new graph from G. That's a third graph
that I'm constructing now.
But notice that the graph that was implicit in my problem
was very different than the graph that I'm ultimately
running a shortest paths algorithm,
like Dijkstra, from p to see if a path exists.
Does that make sense?
Any questions on this problem?
All right, we've got 20 minutes for my last problem.
Man, I'm not using--
I write much less than some of your other instructors.
So I like to talk more than I like to write, apparently.
So problem 4, let's take a look at this.
This one is one I made up last night, kind of fun.
Long shortest paths.
Given a directed graph having arbitrary edge weights,
basically, these could be positive, or negative, or 0,
and 2 vertices from the graph, describe a V cubed time
algorithm to find the minimum weight of any path from s to t.
OK, that sounds like Bellman-Ford right there.
But I have this last condition.
And the last condition is a little weird,
containing at least V edges.
So I want to short path in terms of weight.
But I want a long path, in some sense,
in terms of the number of edges I traverse.
Does that make sense?
So of all paths having at least V edges,
I want a shortest one among them in terms of weight.
This is a weird frickin' problem.
Usually, we're not trying to do this max-min kind of thing.
You've got two different quantities here
we're trying to optimize.
Anyone have any ideas on how I could approach this problem?
What does this sound-- what does the at least V edges sound kind
of similar to that we might have talked about in lecture?
AUDIENCE: [INAUDIBLE].
JASON KU: So when we were talking about Bellman-Ford,
we defined this thing called a k edge weight.
It's the weight of any path using, at most, k edges.
This kind of edge constraint seems similar,
except it's kind of the reverse.
It's not at least, it's most--
it's not at most, it's at least.
Well, here's an observation I have for you.
If I want a path that goes through at least V edges,
some prefix of that path uses exactly V edges.
That makes sense, right?
So maybe it makes sense for me to--
maybe it might make this problem easier
if it's not at least V edges, but if it's exactly V edges.
Maybe I think about it that way.
That seem a reasonable other way to think about this problem?
I knew how to do up to a certain set of edges.
Here, we're asking for at most.
Maybe the thing in between is a little easier to think about.
So what we're doing-- we're given a graph
G. It has any weights.
It's possible that this graph has E lower bounded
by a quadratic in the vertices.
I have no restrictions on how many edges this thing could be.
And so the worst thing I could have is that this thing--
I mean, my graphs are simple.
The worst thing I could do is have
this be quadratic in the number of edges,
say, if it's the complete graph, it's
the maximum number of edges that I could have.
And I'm trying to find, in my graph,
a path that uses a lot of vertices, but has small weight.
Now, what's another thing to notice here
is if I use at least V edges, can my path be simple?
No.
Because I need to use at least V plus 1 vertices.
And there are-- that's more than the vertices
I have in the graph, obviously.
Now, it could go through vertices more than once.
But it's definitely not going to be a simple path.
So what's one thing--
what if there's a negative weight cycle in my graph?
What's the minimum weight of any path from s
to tee if the negative weight cycle is reachable from s--
reachable on a path from s to t.
What is the answer to my problem then?
Negative infinity, right?
Because certainly an infinite length path
is going to use an infinite number of edges.
If it's going arbitrarily long, then I
can just run Bellman-Ford.
So that's one thing I can do.
I can just run Bellman-Ford on this graph.
I have enough time to do that, because E
is upper bounded by V squared.
And I V cubed time.
And if there's a negative weight cycle on my graph,
I can know that the minimum weight of any path
is minus infinity.
I detect that that's the case if, basically, t
is reachable from s with a minimum of shortest path
distance minus infinity.
That the path that achieves that is going
to have more than V edges.
So I'm done.
And no path actually achieves that.
But that's the infimum, supremum, or infimum, sorry,
we're going lower bound.
I'm thinking of long paths.
So the number of edges is approaching infinity.
But in the context where I don't have negative weight cycles,
actually, one of the things we showed
was that if you're reachable not through a negative weight
cycle, or if no negative weight cycles traversable from s to t,
then my shortest path is going to be simple.
But that doesn't seem to apply here either,
because we need to have a non-simple path.
So what do we do?
So let's go back to this idea of trying
to figure out the minimum weight of any path using exactly V
edges.
Can we use some of the tricks that we had in Bellman-Ford,
when we're keeping track of the number of edges we're
going through at a given time.
That's the idea.
If we have a vertex--
if we have a new vertex for each vertex, different versions
of it that talk about exactly how many edges
I went through, then maybe I can keep track of this
while I'm working on this graph.
So let's say I have multiple layers of the graph.
This is the idea.
Maybe we start at level--
level 0, down here to level--
how many edges do I want?
I want V plus 1 vertices.
So I'm going to have V plus 1 levels, which is V. So a level
here, V, how many levels are there?
There's V plus the 0.
And so there--
I'm going to have, for every edge in my graph,
I'm going to take it.
So this is a directed graph.
So I direct it down into the next level.
For each version of this graph that I have,
I take that edge that was originally between u and v
here in the graph.
It was originally here in G. But here, I've
pointed all of those edges downward.
Isn't that what we did in Bellman-Ford?
We made one other addition in Bellman-Ford,
to make it be the at most property.
What was that transformation we did?
AUDIENCE: [INAUDIBLE].
JASON KU: We had 0 weight edges going from each vertex
to another.
It meant that we didn't have to traverse an edge.
But here, if we don't add those edges,
actually, this transformation gives us
that any path that goes through V edges
will be some path from a vertex in layer 0
to a vertex in layer V. Just because, to get down here,
I had to traverse exactly V edges.
And they're edges of my original graph.
Now notice this encodes non-simple paths as well.
Because these things could go--
I could go here and back to u, and back to v, and back to u,
if I had a cycle in my graph.
But actually, what kind of graph is this?
This is a DAG.
So this is DAG.
Maybe, I call this G prime.
How many vertices are in G prime?
v times v plus 1, so I'm going to say order v squared.
And how many edges are in my graph?
V times E, I copied every edge, made it directed down
between each level.
There are V transitions between levels.
And I copy each edge for each of those.
So the number of edges is order V times E. So this graph
is blown up.
There's a lot of things in this graph.
But I notice that this graph has size order V cubed is
what we're going for.
So I can afford to construct this graph,
since V is actually also upper bounded
by V squared by simplicity.
OK, so we have this graph.
We could find our vertex s here, s0 up here.
And we could afford to compute the shortest path distance
to all other vertices using exactly V edges in my graph,
exactly.
That's what we could do.
I can find everything reachable from s0 in this graph
and calculate the shortest path down here at the bottom.
So I can do that in V cubed times
because DAG relaxation is linear in the size of the graph.
But that's not what the problem is asking me, unfortunately.
In particular, I could find the path to t,
to t, v. And that would give me the shortest path using exactly
V edges.
But that's not what I'm asking for.
I am asking for at least.
So it's possible that I get down here to some other vortex.
And maybe there's a negative weight path going to t.
And I want to be able to find that.
So how can I do that?
How can I allow paths to continue
past this an arbitrary amount?
I could have more layers.
Actually, simple paths from any--
I mean, shortest paths that are simple,
that use fewer edges, here, I'm not
restricted on the number of edges I use.
So shortest paths in this graph are
going to be simple, because there's no--
I can already throw away the case
where I have negative cycles, because I ran Bellman-Ford
at the beginning.
I can-- so I know that I'm going to want a short--
a simple path after I've reached V edges,
because it's never going to be beneficial to me
to come back to a vertex, because that will
be a path of longer weight.
This is the kind of surgery argument
we had, both in unweighted and weighted context.
So these are going to be simple.
So I know that I only have to go V more layers at most.
So that's one way to look at it.
I could add more layers of this thing,
find the shortest path distance to all vertices using up
to 2v edges, maybe even 2v minus 1,
but order v. And then for all of the ones down below here,
I just look at each vertex.
And see which weight is the minimum.
Another way-- the way I like to look at it,
which is a little bit more fun, I think, is once I'm down here,
I'm just trying to find simple paths in the graph from this
vertex v--
to this vertex.
So one of the-- so actually, have these go up.
So actually, on this bottom layer,
I want to find short paths to t from, actually, every vertex.
And I actually know what the short-- just from what I did up
here, DAG relaxation on this graph,
I knew what the shortest path distance was from s0
to each of these vertices.
Because I did that in V cubed time up here
with DAG relaxation.
So I could add a super node to this thing with a directed edge
to each vertex with the shortest-- with weighted
by the shortest path distance I found up above.
Now, I have a graph where any path from s0 to tv
here will be a path that uses at least
V edges in my original graph.
because these represent the shortest path
weights of anything using exactly V edges.
And then the path can continue in the original graph.
So now I have a new graph here, such
that every path from here to there corresponds to a path
that I'm looking for.
So I want to find a minimum weight path in this graph.
How can I do that?
Now, this graph might have negative weights.
AUDIENCE: Bellman-Ford.
JASON KU: I can run this with Bellman-Ford, Bellman-Ford.
I can do that again.
Sure, why not?
Now, Jason, why couldn't we just add a bunch of edges here?
Add our original edges here in the bottom layer of this graph
and run b Bellman-Ford on this entire graph?
Why couldn't I do that?
AUDIENCE: Too big.
JASON KU: It's too big, right?
The number of vertices is v squared.
The number of edges is potentially v cubed.
Running Bellman-Ford on that huge, duplicated graph
would give me a v to the fifth running time, which is awful.
In a sense, we're separating out the complexity.
The upper part of the graph has very nice DAG structure.
So let's do shortest paths in that DAG structure.
And then reduce that complexity down to just
being the thing that has the cycles that we
are worried about.
Reduce the complexity down here.
So how big is this graph?
This graph has V plus 1 vertices,
because I only added one supernode here.
And it has E plus order V edges.
I want to be careful here.
But this is linear in the size of the original graph.
So running Bellman-Ford here only takes V times
E time, which is V cubed.
So that's two different ways how to solve this problem.
One using a bunch of graph duplication
and having the insights that going, at most,
v more steps of this graph duplication
could never get a better thing.
So I can stop.
Or recognizing that, well, I have
this very powerful algorithm here
that can find shortest paths, simple paths in a graph
without negative weight cycles.
And I can use this supernode to transfer a part of my graph
with a lot of nice structure down to this other graph.
Any questions about this problem?
So these are some--
we got two abstract problems for you, two word problems for you,
with a lot of different transformations
and a lot of different tricks of trade.
Any of these would be something that
has either appeared on an exam or is
at a level of something that could appear on your exam.
So go ahead and take a look at the practice material
that we've posted and are accessible from previous years'
websites.
And wish you luck in working on graph problems on your exam.