[CREAKING] [CLICKING] [SQUEAKING] [RUSTLING]
[CLICKING]
JUSTIN SOLOMON: We just started a new unit
on graph theory, which is going to be
sort of our focus for the next couple of lectures in 6006.
And so I thought we'd give it a little bit of review
at the beginning the lecture because, as usual,
I've muddled together a lot of notions
in our previous lecture, and then start with some new ideas.
So basically, in our previous lecture,
we talked about an algorithm called breadth-first search.
And then almost always you see that paired
with a second algorithm called depth-first search.
And following tradition, and basically logic,
we'll do the same thing in 006 today.
But in any event, for today we'll
stick to the technical material.
So as a little bit of review, I guess actually,
the one thing I didn't do on this slide
was actually draw a graph.
So we should probably start with that.
So if you recall, graph is a collection of nodes or vertices
depending--
I don't know, is it like a European American thing
or something-- and edges.
So here's an example, which as usual, I'm
not managing to draw particularly clearly.
So this graph is kind of like a cycle.
So I have directed edges here, here, here, and here.
And of course, there are many kind of variations
on the theme, right?
So our basic sort of definition of a graph
is that we have some set V, which
is like the set of vertices.
And then we have a set E, which is set of edges.
And this was a subset of V cross V.
And this is nothing more than fancy notation for saying
that an edge is a pair of vertices, like a from
and a to vertex.
Of course, there are many variations on this theme.
You could have a directed versus an undirected graph.
So this one is directed, meaning the edges look like arrows.
If they didn't have arrowheads, they'd be undirected.
We define something called a simple graph where you have
essentially no repeated edges.
So for instance, you can't do something
like this where you have the same edge twice.
And then there are a couple of different definitions
that were kind of useful.
So in particular-- I'm going to erase this, whoops--
useless edge here.
Maybe make my graph slightly more interesting.
So add another edge going in the reverse direction.
So maybe I have--
I'm going to give my vertices labels. x, y, z, and w.
Then we talked about the neighbors
of a given vertex, which are the vertices that you
can reach by following edges in or out of your vertex.
So in particular, the outgoing neighbors,
which we sort of implicitly defined in our previous lecture
but didn't call it out, we're going to notate with Adj+.
And these are all of the things that you
can reach by going out of a vertex into the next one.
So for example, Adj+ of w is going to be the set
of vertices.
We'll notice I can get from w to y and also from w to z.
Yeah.
So.
Nope, nope.
y comma z.
OK.
So to continue just our tiny amount of review for the day,
remember that a graph-- there are many different ways
to represent a graph.
The sort of brain dead one would be just
like a big long list of edges.
But of course, for our algorithms
it's not a particularly efficient way
to check things like, does this edge exist in my graph.
So the basic representation that I
think we're mostly working from in this course
is to think of a graph like a set of vertices, each of which
maps to another set of vertices.
So roughly every vertex maybe stores its outgoing set
of edges.
And so this is kind of nice because, of course,
very quickly we can answer questions like,
is this edge inside of our graph?
Or we can iterate over the neighbors
of a vertex and so on, which are the kind of typical things
that we do in a lot of graph algorithms.
And then finally, in our previous lecture,
we started talking about paths.
So a path is like a chain of vertices
that can get me from one vertex to the other only following
edges of my graph.
There is a term that I think I forgot
to define last time because it didn't really
matter a ton, which is a simple path, which is just
a path that doesn't have the same vertex more than once.
And then, of course, there are many different questions
you could ask about a graph that are basically
different problems involving computing paths.
So for instance, the shortest path between two vertices
is sort of our canonical one in graph theory.
Or you could ask questions about reachability and so on.
So there's our basic review from our previous lecture.
Does our course staff have any questions about things so far?
Excellent.
OK.
And there's one additional piece of terminology
that I fudged a little bit last time-- or rather,
my co-instructor suggested a bit of an attitude adjustment.
So I thought I'd better clarify really quick.
There's this interesting phrase, linear time,
which we all know and love in computer science theory.
And this sort of implicit thing, especially in this course,
is that when we say linear time, we
mean in the size of the input.
Right?
And so if we have a linear time graph algorithm, well,
how much space does it take to store a graph?
Well, we need a list of vertices and a list of edges,
if nothing else.
So a reasonable way to interpret this phrase linear time
is that it's an algorithm that looks like what
we've shown on the screen.
The times proportional to maybe the sum
of the number of vertices and the number of edges.
If that makes you uncomfortable like it does for me because one
of these can kind of scale on the other,
I think it's always fine to add more detail.
Right?
So if you want to say, linear in the sum
of the number of vertices and edges, that's perfectly fine.
But if you see this phrase, that's
how you should interpret it.
Hopefully that's a fair way to put it.
Excellent.
OK.
So last time, we talked about an algorithm called breadth-first
search--
BFS, for those in the know.
Breadth-first search is an algorithm.
And the reason we use the word breadth
is because it's kind of, remember, we talked about level
sets last time because we talked about breadth-first search
in the context of computing shortest paths.
And in particular, we have our source node all the way
on the left-hand side.
And then breadth-first search constructed all the nodes
that were distance 1 away.
Right.
That's the first level set, and then all the distance 2 away,
and then all the distance 3 away, and so on.
So in particular, the level set L3
isn't visited until we're completely
done with level set L2.
Today, we're going to define another algorithm, which
is called depth-first search, which doesn't do that,
but rather, starts with its first vertex and just
starts walking all the way out until it can't do that anymore.
And then it kind of backtracks.
That's one way to think about it.
And so somehow, in breadth-first search,
we're like, drawing concentric circles.
In depth-first search, we're doing the opposite.
We're like, shooting outward until we
reach the outer boundary, and then exploring
the graph that way.
OK.
And these are sort of the two extremes
in terms of graph search kind of techniques that are typically
used under the basic building blocks for algorithms
in graph theory.
So in order to motivate and think about depth-first search,
we're going to define a second problem, which
is closely related to shortest path, but not exactly the same.
And that's the reachability problem.
So here I have the world's simplest directed graph.
So the black things are the edges.
And the circles are the nodes or the vertices.
And I've marked one special node in blue.
And his name is the source node.
And now the question I want to ask
is, what are all of the other nodes in my graph
that I can reach by following edges--
directed edges-- starting with the source?
So obviously, I can get to the node
in the lower right, no problem.
And of course once I get there, I
can traverse and edge upward to get
to that second green vertex.
Notice that I was really sneaky and evil,
and I drew edges in this graph that might make you
think that the red node is reachable.
The red one being on the upper left.
I'm realizing now that for colorblind people,
this isn't a great slide.
But of course, because all the edges
from the red vertex on the left here point out,
I can't actually reach it from the blue source node.
So the reachability problem is just
asking, which nodes can I reach from a given source?
Pretty straightforward, I think.
Of course, there are many ways to solve this.
Right?
In fact, one way we could do it would
be to use our previous lecture.
We could compute the shortest path distance from the source
to all the other nodes.
And then what would the length of the shortest
path from the source to an unreachable node be?
Any thoughts from our audience here?
Infinity.
Thank you, Professor Demaine.
Right.
So in addition to this, of course, a totally reasonable
question, thinking back to our shortest path lecture,
there are sort of two queries we might make.
Right?
One is just what is the length of the shortest path?
The other is like, what is the actual shortest path
from the source to a given vertex?
We can ask a very similar thing here, which is like, OK.
You tell me that the green guy is reachable, but how?
Give me a path as evidence or a certificate, if you
want to be fancy about it.
So in order to do that, just like last time, remember,
we defined a particular data structure that
was the shortest path tree.
We can do something very similar here.
In particular, this is like the extent of my PowerPoint skills
here.
If I have a reachability problem,
I can additionally store--
I can decorate every node in my graph
with one other piece of information,
which is the previous node along some path
from my source to that thing.
Right?
And just like last time, if I want
to get an actual path from the source to w, what could I do?
I can start with w and then just keep
following those parent relationships
until I get back to the source.
Then if I flip the order of that list of vertices,
I get a path from the source to the target that's valid.
So this object is called a path tree, just
like we talked-- or a parent tree, rather.
Just like we talked about in our last lecture,
there's no reason why this thing should ever have a cycle in it.
It's certainly a tree.
Right.
So that's the basic reachability problem.
And in addition to that, we can compute this object P,
which is going to give me sort of information about how
any given node was reachable.
There's a slight difference between the parent
tree that I've defined here and the shortest path tree, which
I defined last time, which is, I'm not going to require
that the shortest path I get-- oh, man-- the path
I get when I backtrack along my tree P is the shortest path,
it's just a path because for the reachability problem,
I actually don't care.
Like, I could have a weird, circuitous, crazy long path.
And it still tells me that a node is reachable.
Right.
So that's our basic set up and our data structure.
And now we can introduce a problem to solve reachability.
Again, we already have an algorithm
for doing that, which is to compute shortest paths.
And remember that our shortest path algorithm
from previous lecture took linear time
and the size of the input.
It took v plus e time.
Now the question is, can we do a little better?
The answer, obviously, is yes, because I just asked it,
and I gave you this problem.
OK.
And here's a technique for doing that, which unsurprisingly,
is a recursive algorithm.
I'm going to swap my notes for my handwritten notes.
And this algorithm is called depth-first search.
And here's the basic strategy.
I'm going to choose a source node
and label that Node 1 here.
I suppose it actually would have made sense for me
to actually 0 index this.
Maybe in the slides I'll fix it later.
But in any event, I'm going to mark my source node.
And now I'm going to look at every node, every edge coming
out of that node, and I'm going to visit it recursively.
So that's our sort of for loop inside of this function visit.
And then for each neighboring node,
if I haven't visited it before, in other words,
I currently haven't given it a parent.
That's our if statement here.
Then I'm going to say, well, now they do have a parent.
And that parent is me.
And I'm going to recurse.
You guys see what this is doing?
It's kind of crawling outward inside of our graph.
So let's do the example on the screen.
And I purposefully designed this experiment-- or this example--
to look a little bit different from breadth-first search,
at least if you choose to do the ordering that I did.
So here's our graph.
1, 2, 5, 3, 4.
OK.
And let's think about the traversal order
that the depth-first search is going to do.
Right.
So here's our source.
And now what does the source do?
It rec-- so let's think about our recursion tree.
So we have the source all the way up in here.
And now he's going to start calling the visit
function recursively.
So.
And I'll go ahead and number these the same as
on the screen.
Well, he has one outgoing neighbor,
and it hasn't been visited yet.
So of course, the very first recursive call that I'll make
is to that neighbor 2.
Now the neighbor 2 also recurses.
Hopefully this kind of schematic picture
makes some sense, what I'm trying to draw here.
And well now, the 2 has two neighbors, a 3 and a 5.
So let's say that we choose 3 first.
Well, the 3 now recurses and calls 4.
And then the recursion tree is kind of done.
So now it goes back out.
And then finally, well, now, the 3--
or, oh, boy.
Yeah.
The 2 looks at his next neighbor, which is the 5
and visits that recursively.
Notice that this is not following the level sets.
Right?
The depth-first search algorithm got all the way
to the end of my tree in the recursive calls
and then kind of backed its way out to the 2
before calling the 5.
These are not the same technique.
One goes all the way to the end and then kind of backtracks.
When I say backtrack, what I mean
is the recursion is kind of unraveling.
Whereas in breadth-first search, I visit everything in one level
set before I work my way out.
That distinction make sense?
OK.
So of course, we need to prove that this algorithm does
something useful.
So let's do that now.
So in particular, we need a correctness proof.
So our claim is going to be that--
let's see here-- the depth-first search algorithm visits all,
I guess reachable v, and that it correctly
sets the parent in the process.
OK.
So in order to prove this, of course,
as with almost everything in this course,
we're going to use induction.
And in particular, what we're going to do
is do induction on the distance from the source.
So we're going to say that, like,
for all vertices in distance k from the source,
this statement is true.
And then we're going to prove this inductively on k.
OK?
So we want to do induction on k, which
is the distance to the source vertex.
So as with all of our inductive proofs,
we have to do our base case and then our inductive step.
So in the base case, k equals 0.
This is a hella easy case because,
of course, what is the thing that
is distance 0 from the source?
It's the source!
Yeah.
And take a look at our strategy all the way
at the top of the slide.
We explicitly set the correct parent for the source,
and in some sense, visit it because the very first thing
we do is call visit of s.
So there's kind of nothing to say here.
Yeah?
Or there's plenty to say if you write it on your homework.
But your lazy instructor is going
to write a check mark here.
OK.
So now we have to do our inductive step.
So what does that mean?
We're going to assume that our statement is
true for all nodes within a distance k.
And then we're going to prove that the same thing is
true for all nodes within a distance k plus 1.
OK.
So let's do that.
Let's consider a vertex v that's distance k plus 1 away.
So in other words, the distance from the source to v
is equal to k plus 1.
And what's our goal?
Our goal is to show that the parent of v is set correctly.
Yeah?
What was that?
AUDIENCE: [INAUDIBLE].
JUSTIN SOLOMON: Oh, sorry.
I forgot that the distances in this class are in order.
Yeah.
That's absolutely right.
So it should be the distance from s to v. Yeah.
Sorry.
I'm really not used to thinking about directed graphs.
But that's a good fix.
OK.
So now what can we do?
Well, there's this number is distance here.
So in particular, the shortest path from s to v.
So remember our argument last time that essentially,
when we look at shortest path and we kind of truncate by 1,
it's still shortest path?
That property doesn't matter so much here.
But at least we know that there's
another vertex on the path, which is 1 distance from one
less away.
So let's take u, which is also a vertex,
to be the previous node on the shortest path from s
to v. Right.
And so in particular, we know that the distance from s to u
is equal to k.
And conveniently, of course, by our inductive hypothesis
here, we know that our property is true for this guy.
OK.
So now our algorithm, what do we know?
Well, because our property is true,
the visit function at some point in its life
is called on this vertex u.
That's sort of what our induction assumes.
So we have two cases.
Right.
So when we visit u, we know that when we call this visit
function, well, remember that v kind of by definition is
in Adj+ of u.
Right.
So in particular, DGS is going to consider v
when it gets called.
OK.
And now there's two cases.
Right?
So either when this happens, P of v does not equal None.
Right.
Well, what does that mean?
Well, it means that we already kind of found a suitable parent
for v. And we're in good shape.
Otherwise, p of v does equal None.
Well, in this case, the very next line of code
correctly sets the parent.
And we're all set.
So in both of these two cases, we show that the parent of u
was set correctly either by that line of code
right here or just previously.
And so in either case, our induction is done.
All right.
I guess given the feedback I received
from our previous lecture, we now can end our LaTeX suitably.
OK.
So what did we just show?
We showed that the depth-first search algorithm can dig around
in a graph and tell me all of the things that
are searchable, or rather, are reachable from a given source,
just basically by calling visit on that source
and then expanding outward recursively.
OK.
So I think this is certainly straightforward
from an intuitive perspective.
It's easy to get lost when you write
these kind of formal induction proofs
because they always feel a tiny bit like tautology.
So you should go home and kind of convince yourself
that it's not.
OK.
So of course, what do we do in this class?
We always follow the same kind of boring pattern.
The first thing we do, define an algorithm.
Second thing we do, make sure that it's the right algorithm.
What's the third thing we need to do?
AUDIENCE: Analyze it.
JUSTIN SOLOMON: Analyze it.
That's right.
In particular, make sure that it finishes before the heat
death of the universe.
And indeed, depth-first research doesn't really
take all that long, which is a good thing.
So let's think about this a bit.
So what's going to end up happening
in depth-first search, well, we're
going to visit every vertex at most once, kind of
by definition here.
And in each case, we're going to just visit its neighboring
edges.
Can we ever traverse an edge more than one time?
No.
Right.
Because the visit function only ever gets
called one time per vertex.
And our edges are directed.
Right.
So kind think about the from of every edge, the from vertex
is only ever visited one time.
And hence, every edge is only visited one time.
Do we ever visit-- ah, yes.
AUDIENCE: Does DFS work for an undirected graph?
JUSTIN SOLOMON: An undirected graph.
Absolutely.
So there's sort of different ways to think about it.
One is to think of an undirected graph
like a directed graph with two edges
pointed either way, which I think
is in this class how we actually kind of notated it
in the previous lecture.
Yeah.
Actually, that's probably a reasonable way to reduce it.
So let's stick with that.
Right.
Now, does DFS ever visit a vertex
that is not reachable from the source?
Well, the answer is no because all I ever do
is recursively call on my neighbors.
And so kind of by definition, if I'm not reachable,
DFS will never see it.
So if I think about my runtime carefully,
it's not quite the same as breadth-first search.
Remember that breadth-first search took v plus e time.
In depth-first search, it just takes order e time
because I'm expanding outward from the source vertex,
hitting every edge adjacent to every vertex
that I've seen so far.
But I never reach a vertex that I haven't--
that isn't reachable.
Right?
And so because this only ever touches every edge one time,
we're in good shape.
And I see a question here.
Yeah.
AUDIENCE: Does BFS reach vertices
that are not reachable?
JUSTIN SOLOMON: Does BFS reach vertices
that are not reachable?
I guess not, now that you mention it.
But at least in my boring proof of order
v time last time, our very first step of BFS,
reserve space proportional to v, which
is enough to already make that runtime correct.
Good question.
Yeah.
So I guess the way that we've talked about it where
you can stretch one little set after a time, if you
think of that as reachability, then no.
It doesn't reach it in the for loop.
But just by construction, when we started
we already took the time that we're talking about here.
So notice these run times aren't exactly the same.
So for example, if my graph has no edges,
BFS still is going to take time because it still
has to take order v time, at least
in the sort of brain-dead way that we've implemented it
last time.
Obviously, in that case, we could probably
do something better.
Whereas the way that we've defined the DFS algorithm,
it only takes edge time.
I see confusion on my instructor's face.
No?
OK.
Good.
The one thing to notice is that these
are algorithms for slightly different tasks in some sense.
The way that we wrote down breadth-first search last time,
conveniently, it gives us the shortest path.
There are breadth-first search algorithms that doesn't.
I think in this class we kind of think of breadth-first search--
we motivate it in terms of the shortest path problem.
But it's just kind of a strategy of working outwards
from a vertex.
Whereas here, the way we've written down
depth-first search, there's no reason why the path that we get
should be the shortest.
Right?
So to think of a really extreme example,
let's say that I have a cycle graph.
So I get a big loop like this.
Let's say that I do depth-first search
starting from this vertex.
Well, what will happen?
Well, this guy will call its neighbor recursively,
who will then call its neighbor recursively,
who will then call his neighbor recursively, and so on.
So of course, when I do depth-first search, when
I get to this vertex, there's a chain of 1, 2, 3, 4
vertices behind it.
Is that the shortest path from the source to the target here?
Well, clearly not.
Right?
I could have traversed that edge.
I just chose not to.
OK.
So that's the depth-first search algorithm.
It's just essentially a recursive strategy
where I traverse all my neighbors,
and each of my neighbors traverses their neighbors,
and so on.
OK.
So why might we want to use this algorithm?
Well, we've already solved the reachability problem.
So let's solve a few more things using the same basic strategy
here.
So there's some notions that we've sort of--
actually, in some sense, already used in the lecture here.
But we might as well call them out
for what they are, which is this idea of connectivity.
So a graph is connected if there's
a path getting from every vertex to every other vertex.
Right.
Now connectivity in a directed graph
is kind of a weird object.
Like, for instance, think of a directed graph
with just two edges.
And one edge goes from u to v. Then I can get from v to u,
but not vise versa.
That's kind of a weird notion.
So here in 6006 we'll mostly worry about connectivity only
for undirected graphs because they're--
the vertices just basically come in like, big connected clumps.
Or the more technical term for a big connected clump
is a connected component.
Yeah?
So let's see an example.
So let's say that I have a graph, which has
an edge and then a triangle.
This is one graph.
Do you see that?
There's a collection of vertices,
and there's a collection of edges.
But it has two connected components--
the guy on the right and the guy on the left,
meaning that each vertex here is reachable
from every other vertex here.
Each vertex here is reachable from every vertex here.
But there's no edge that goes from the triangle
to the line segment.
Yeah?
And so in the connected components problem,
we're given a graph like this guy.
And initially, we don't, you know--
OK.
When I draw it like this, it's pretty clear
that my graph has two connected components.
Maybe my graph-embedding algorithm failed
and it drew an edge like that.
Well, then maybe--
I don't know-- it's still pretty obvious that there's
two connected components.
But you can imagine a universe where
you don't know that a priori.
And the problem you're trying to solve
is just to enumerate all these clumps of vertices
that are reachable from one another in an undirected graph.
And conveniently, we can use depth-first search
to solve this problem pretty easily.
Right?
So how could we do it?
Well, in some sense how can we find one connected component?
So let's say that I just choose a vertex in my graph.
Well, what do I know about everything
in its connected component?
Well, it's reachable from that vertex.
Remember, we just solved the reachability problem, which
says, if I have a vertex, I can now
tell you all the other vertices that
are reachable from this guy.
So I could call DFS on, well, any vertex of this cycle here.
Call the reachability thing.
And I know that for every vertex there's one of two things.
Either the vertex has a parent in that object P,
or it's the source.
So I can very easily find the connected component
corresponding to that vertex.
Does that makes sense?
Have I found all the connected components?
No.
I found one.
I found the one corresponding to the arbitrary vertex
that I just chose.
So how could I fix this?
Well, it's super simple.
I could put a for loop on the outside, which just loops
over all the vertices, maybe.
And if that vertex is not part of a connected component yet,
then I need to make a new one.
So then I call DFS on that vertex.
I collect all the vertices that I got.
And I iterate.
So this is the algorithm that in this class
we're going to call full DFS.
By the way, you could do the same thing
with full breadth-first search.
That's perfectly fine.
Just kind of by analogy here.
Right.
So what is full D--
oh, this chalk is easier.
Well, I'm going to iterate over all my vertices.
Where I stands for for loop.
Of-- right.
So if v is unvisited, then I'm going
to do to DFS starting at v. I guess we used visit to refer
to this in the previous slide.
And that's going to kind of flood
fill that whole connected component.
And then I can collect that connected component
and continue.
We have to be a little bit careful because, of course,
we don't want like, checking things--
something to be visited to somehow take a bunch of time
and make my algorithm slower than it needs to be.
But of course, we have a set data structure
that we know can do that and order one time at least
in expectation.
OK.
So this is the full DFS algorithm.
It's really simple.
Of DFS because I called DGS on every vertex.
And it's full because I looped over all the vertices.
Right.
And so if we think about it, how much time does this algorithm
take?
It's little bit sneaky because somehow I
have a for loop over all the vertices.
Then I could imagine a universe where I get, like,
vertices times some other number because there's a for loop,
and then there's something inside of it.
I think that's how we're used to thinking
about runtime of for loops.
But in this case, that actually doesn't
happen because there's never a case where an edge gets
traversed more than one time.
Because if I'm in one connected component, then by definition,
I can't be in another connected component.
Right?
And so what happens is, in some sense,
this innocent looking call to DFS--
I suppose if you were like a LISP or a programmer,
you somehow wouldn't like this.
It has a side effect, which is that I marked
all the vertices in that connected component as
"don't touch me again."
Right.
And so implicitly I kind of removed edges in this process.
So if you think through it carefully,
the runtime of this full DFS algorithm
is v plus e time because every edge is
touched no more than one time.
Kind of amortized over all the different calls to DGS here.
And there's this for loop over vertices.
So there's clearly an order v that you need here.
Does that argument make sense?
So again, we call that linear in the size of the input.
I'm going to say it as many times to get it in my own head
correctly.
OK.
Right.
So this is the basic problem.
This comes up all the time, by the way.
Like, it seems like somehow a totally brain dead
weird algorithm.
Like, somehow, why would you want an algorithm that
finds connected components.
Like, why would you even have a graph that's disconnected
or something?
But of course, that can happen a lot.
So for instance, maybe you work at a social media company,
and people have friends.
But like, Eric and I are friends.
And we're not friends with anybody else.
We have a-- there's like, a blood oath kind of thing.
Then that might be not so easy to find in the graph
because, of course, we're just two among a sea of students
in this classroom, all of which have
different interconnections that are just enumerated based
on the list of edges.
And so even though like, pictorially, it's
kind of hard to draw a connecting component algorithm
in a way that doesn't make it sound kind
of like a useless technique from the start,
because it's very clear there are two connected components
there.
Of course, we still have to be able to write code
to solve this sort of thing.
OK.
So for once, I think I'm almost on time in lecture today.
So we have one additional application
of depth-first search in our class today,
which is sort of on the opposite end of the spectrum.
So we just talked about graphs that are undirected
and thinking about cycles.
Now, on the opposite end we might think of a DAG.
So a DAG is a Directed Acyclic Graph.
Can anyone think of a special case of a DAG?
I suppose I should define it first.
And then we'll come back to that question, which means
exactly what it sounds like.
So it's a graph that has directed edges now
and doesn't have any cycles in it.
So actually, the graph I gave you
all the way at the beginning of lecture I think secretly
was an example of one of these.
So let's say that I have directed edges.
Maybe if I make the head a triangle,
it's a little easier to see.
I'm not so sure.
In any event, so I'm going to have an edge up and an edge
to the right, and similarly, an edge down
and an edge to the right.
This graph looks like a cycle.
But it's not because the only direction that I can move
is from the left-hand side to the right-hand side.
So this is a directed graph.
And it doesn't contain any cycles,
meaning there's no path that it can
take from a vertex that gets back to itself
along the directed edges.
OK.
And DAGs show up all the time.
Now that I've defined what a DAG is,
can somebody give me an example of a DAG
that we've already seen in 6006?
AUDIENCE: A tree.
JUSTIN SOLOMON: A tree.
At least if we orient all all the edges kind of pointing
downward in the tree.
Yeah.
Otherwise, it gets kind of debatable over
whether it's a DAG or not.
If there's no direction to the edges, then somehow
the definition just doesn't apply.
OK.
So in processing directed acyclic graphs,
there's a really useful thing that you
can do that's going to show up in this class apparently
quite a bit, which is kind of interesting to me,
I'm curious to see what that looks like,
which is to compute a topological order on the graph.
We're at topologies here.
So as a geometry professor in my day job, I get all excited.
But in this case, a topological order
is a fairly straightforward thing.
Actually, it's defined on the screen,
and I have bad handwriting anyway.
So let's just stick with that.
So topological ordering.
So we think of f as a function that assigns maybe every node
an index in array.
I guess I shouldn't use the word array here.
But just like an index, an ordering.
So like, this is the first vertex.
And this is the second vertex.
And so on.
Then a topological order is one that
has the properties shown here, which
is that if I have a directed edge from u to v, then f of u
is less than f of v. So in other words,
if I look at the ordering that I get on my topological order,
u has to appear before v. Yeah?
Let's look at our example again.
So let's give our nodes names.
So here's A, B, C, D. Well, what clearly
has to be the first node in my topological order?
A. Right.
It goes all the way to the left-hand side.
Yeah.
Well, after that it's a bit of a toss-up.
What do we know?
We know that B and C have to appear before D. So maybe just
to be annoying, I do A, C, B-- that's a B-- and then D.
So it's a topological order.
Notice that things that are on the
left appear in my graph before things that are on the right,
where the word "before" here means
that there's an edge that points from one to the other.
OK.
By the way, are topological orderings unique?
No.
So if we look at our graph example here,
ABCD is also a topological order.
And what that means is somehow very liberating.
It means that when we design an algorithm
for finding a topological order, so there's some design
decisions that we can make.
And we just have to find one among many.
But in any event, we're going to define a slightly different
notion of order.
And then we're going to show that they're closely
linked to each other.
And that is the finishing order.
So in the finishing order, we're going
to call full DFS on our graph.
Remember, that means we iterate over all our nodes.
And if we haven't seen that node yet, we call DFS on it.
And now we're going to make an order in which
as soon as the call to a node in that visit function
is complete, meaning I've already
iterated over all my neighbors, then
I add my node to the ordering.
That make sense?
It's like a little bit backward from what
we're used to thinking about.
So it's the order in which full DFS
finishes visiting each vertex.
Yeah?
And now here's the claim, is that if we
have a reverse finishing order, meaning
that we take the finishing order and then we flip it backward.
That's exactly going to give us a topological ordering
of the vertices in our graph.
Right.
So let's do that really quickly.
So in other words, our claim here--
I think, yeah, let's see-- is that if I
have a directed graph.
So G is a DAG.
Then let's see here.
Then the-- oops.
My notes are backwards.
So I should switch to my--
Jason's notes, which of course, are correct.
Right.
So if I have a graph that's a DAG,
then the reverse of the finishing order
is a topological order.
By the way, we're not going to prove the converse that if I
have a topological order, that somehow that thing is
the reverse of DFS, at least the way that maybe I coded it.
There's a slightly different statement, which is,
does there exist a DFS that has that ordering?
But that's one that we'll worry about another time
around piazza or whatever.
OK.
So let's see here.
So we need to prove this thing.
So what are we going to do?
Well, what do we need to check is the topological order
is that if I look at any edge of my graph,
it obeys the relationship that I have on the screen here.
So in particularly, we're going to take
uv inside of my set of edges.
And then what I need is that u is
ordered before v using the reverse of the finishing order
that we've defined here.
OK.
So let's think back to our call to the DFS algorithm,
where call this visit function.
Right.
So we have two cases.
Either u is visited before v. Or it ain't.
Yeah.
So let's do those two cases.
So case Number 1 is, u is visited before v. OK.
All right.
So what does that mean?
Well, remember that there's an edge.
Like, pictorially, what's going on?
Well, there's all kinds of graph stuff going on.
And then there's u.
And we know that there's a directed edge from u
to v. That's our picture.
Right?
And maybe there's other stuff going on outside of us.
So in particular, well, just by the way
that we've defined that visit function, what do we know?
We know that when we call visit on u, well,
v is one of its outgoing neighbors.
So in particular, a visit on u is
going to call visit v. And we know that because well, u
is visited before v.
So currently, v's parent is l when
I get to you That make sense?
Now, here's where reverse ordering,
we're going have to keep it in our head because now,
well, visit of u calls visit of v. So notice that visit of v
has to complete before visit of u.
Right?
V completes before visit of u.
Well.
So in reverse, sorting--
in reverse finishing order here, what does that mean?
Well, if this completes before the other guy, then they get
flipped backward in the list, which
is exactly what I want because there's an edge from u to v.
OK.
So Case 1 is done.
Now we have Case 2, which is that v is visited before u.
OK.
So now I'm going to make one additional observation.
OK.
So now I'm going to go back to my other notes
because I like my schematic better.
Right.
So what's our basic picture here?
Oh, no.
I-- Oh, you know what it was?
I printed out another copy of this.
That's OK.
I can do it off the top of my head.
OK.
So here's my source vertex.
His name is S. Now, there's a bunch of edges and whatever.
There's a long path.
And now eventually, what happens?
Well.
I have a node v. And somewhere out there
in the universe is another node u.
And what do I know?
I know that by assumption, I know
that there's an edge from u to v. That make sense?
So that's our sort of picture so far.
OK.
So what do we know?
We know that our graph is acyclic.
Yeah?
Kind of by definition, that's our assumption.
So can we reach u from v?
In other words, does there exist a path from v to u?
So that would look like this.
No because our graph is acyclic, and I just drew a cycle.
So this is a big X. There's a frowny face here.
Can't do it.
He has hair, unlike your instructor.
OK.
So right.
So what does this mean?
Well, OK.
So by this picture, I suppose, we know that u cannot be
reached from v.
Yeah.
So what does that mean?
Well, it means that the visit to v
is going to complete and never see u
because remember, the visit to v only ever call things that
are kind of descendants of v. So in other words, visit of v
completes without seeing u.
Well, that's exactly the same thing
that we showed in our first case.
Right?
So by the same reasoning, what does that mean?
In our reverse finishing order, the ordering from u to v
is preserved.
OK.
So that sort of completes our proof
here that reverse finishing order gives me
a topological order, which is kind of nice.
And so this is a nice convenient way
of taking all of the nodes in a directed acyclic graph
and ordering them in a way that respects
the topology or the connectivity of that graph.
So we're going to conclude with one final problem, which
I don't have a slide on.
But that's OK.
And that's cycle detection.
So there's a bit of an exercise left to the reader here.
So the problem that we're looking for now
is that we're given a directed graph.
There's a G in graph, in case you're wondering.
But now, we don't know if it's a DAG or not.
And so the question that we're trying to ask is,
does there exist a cycle in our directed acyclic graph?
So we're just given our graph, and we want to know,
can we do this?
Let's think through the logic of this a bit.
So what do we know?
We know that if our graph were a DAG, then
I could call DGS, get the ordering out,
and then I guess flip its ordering backwards.
So I could compute the reverse finishing order.
And it would give me a topological order of my graph.
So if I were a DAG, I would get a topological order
when I call DFS.
So let's say that I ran DFS.
I got whatever ordering I got.
And now I found an edge the points in the wrong direction.
I can just double check my list of edges,
and I find one that does not respect the relationship that I
see in the second bullet point here.
Can my graph be a DAG?
No.
Because if my graph were a DAG, the algorithm would work.
I just proved it to you.
Right?
So if my graph were a DAG, then I
could do reverse finishing order.
And what I would get back is a topological order.
So if I found a certificate that my order wasn't topological,
something went wrong, and the only thing that could go wrong
is that my graph isn't a DAG.
Yeah.
Isn't a DAG.
In fact, sort of an exercise left to the reader and/or
to your section-- do we still have section?
I think we do, as of now--
is that this is an if and only if, meaning
that the only time that you even have a topological ordering
in your graph is if your graph is a DAG.
This is a really easy fact to sanity check, by the way.
This is not like, a particularly challenging problem.
But you should think through it because it's a good exercise
to make sure you understand the definitions, which
is to say that if you have a topological order,
your graph is a DAG.
If you don't have a topological order, your graph isn't a DAG.
So in other words, we secretly gave you
an algorithm for checking if a graph is a DAG at all.
Right?
What could I do?
I could compute reverse finishing order.
Check if it obeys the relationship
on the second bullet point here for every edge.
And if it does, then we're in good shape.
My graph is a DAG.
If it doesn't, something went wrong.
And the only thing that could have gone wrong
is not being a DAG.
OK.
So in other words, secretly we just solved--
well, I guess the way that I've written it here,
we've solved the cycle detection problem here,
which is to say that, well, I have a cycle if
and only if I'm not a DAG, which I
can check using this technique.
Of course, the word "detection" here probably means
that I actually want to find that cycle,
and I haven't told you how to do that yet.
All we know how to do so far is say, like, somewhere
in this graph there's a cycle.
And that's not so good.
So we can do one additional piece of information
in the two minutes we have remaining to sort of complete
our story here, which is to modify our algorithm ever so
slightly to not only say thumbs up, thumbs down, is there
a cycle in this graph, but also to actually return the vertices
as a cycle.
And here's the property that we're
going to do that, which is following,
which is that if G contains a cycle, right, then
full DFS will traverse an edge from a vertex v
to some ancestor of v. I guess we haven't carefully defined
the term "ancestor" here.
Essentially, if you think of the sort of the running of the DFS
algorithm, then an ancestor is like something
that appears in the recursive call tree before I got to v.
OK.
So how could we prove that?
Well, let's take a cycle.
And we'll give it a name.
In particular, we'll say that it's a cycle from v0 v1 to vk.
And then it's a cycle, so it goes back to v0.
OK.
And I can order this cycle any way I want.
Notice that if I permute the vertices
in this list in a cyclical way, meaning
that I take the last few of them and stick them
at the beginning of the list, it's still a cycle.
That's the nice thing about cycles.
So in particular, without loss of generality,
we're going to assume that v0 is the first vertex visited
by DFS.
What does that mean?
That means, like, when I do my DFS algorithm making
all these recursive calls, the very first vertex
to be touched by this technique is v0.
OK.
Well, now what's going to end up happening?
Well, think about the recursive call tree starting at v0.
By the time that completes, anything
that's reachable from v0 is also going to be complete.
Do you see that?
So for instance, v0 somewhere in its call tree might call v2.
And notice that v2 was not already visited.
So in fact, it will.
For v1 I got to call v2 and so on.
And in particular, we're going to get all the way to vertex k.
Right?
So in other words, we're going to visit a vertex vk.
And notice what's going to happen.
So remember our algorithm.
In fact, we should probably just put it up on the screen
would be easier than talking about it a bunch.
Well, vk is now going to iterate over
every one of the neighbors of vk.
And in particular, it's going to see vertex v0.
Right?
So we're going to see the edge from vk
to v0, which is an edge kind of by definition
because we took this to be a cycle here.
But notice that's exactly the thing we set out to prove,
namely that full DFS traverses an edge from a vertex
to one of its ancestors.
Here's a vertex k.
Here's the ancestor v0.
Why do we know that it's an ancestor?
Well, because v0 was called in our call tree
before any of these other guys.
Right?
So we wanted an algorithm that not only did cycle detection,
but also actually gave me the cycle.
What could I do?
Well, it's essentially a small modification
of what we already have.
Right.
So-- whoops.
Right.
If I want to compute topological order or whatever,
I can just do DFS.
And that'll tell me like, yay or nay, does there exist a cycle.
If I want to actually find that cycle, all I have to do
is check that topological order property at the same time
that it traversed the graph during DFS.
And the second that I find an edge that loops back, I'm done.
And so that's our basic algorithm here.
And this is a technique for actually just finding
the cycle in a graph using the DFS algorithm.