WEBVTT

00:00:00.000 --> 00:00:01.950
[CLICKING]

00:00:01.950 --> 00:00:02.450
[RUSTLING]

00:00:02.450 --> 00:00:04.410
[SQUEAKING]

00:00:04.410 --> 00:00:06.370
[CLICKING]

00:00:25.030 --> 00:00:26.830
MICHAEL SIPSER: OK.

00:00:26.830 --> 00:00:28.130
Hi, everybody.

00:00:28.130 --> 00:00:28.915
Let's get started.

00:00:31.910 --> 00:00:38.050
So, it's been a while since
we came together in a lecture.

00:00:38.050 --> 00:00:39.730
Last week, we had the holiday.

00:00:39.730 --> 00:00:42.220
We had the midterm.

00:00:42.220 --> 00:00:47.320
So with that, what
have we been doing?

00:00:47.320 --> 00:00:52.540
We finished the first half of
the course about two weeks ago,

00:00:52.540 --> 00:00:54.640
where we talked
about-- we were talking

00:00:54.640 --> 00:00:56.410
about computability theory.

00:00:56.410 --> 00:00:58.510
We have shifted into
the second half,

00:00:58.510 --> 00:01:01.040
talking about complexity theory.

00:01:01.040 --> 00:01:04.870
So get your mind back to that.

00:01:04.870 --> 00:01:12.670
We discussed the various
different models and ways

00:01:12.670 --> 00:01:16.570
of measuring complexity
on different models--

00:01:16.570 --> 00:01:19.000
at least in terms of the
amount of time that's used.

00:01:19.000 --> 00:01:22.100
And in the end, we settled on
the one-tape Turing machine,

00:01:22.100 --> 00:01:23.920
which is the same model
we had been working

00:01:23.920 --> 00:01:25.810
with in the first
half of the course,

00:01:25.810 --> 00:01:31.450
and argued that though
the measures of complexity

00:01:31.450 --> 00:01:33.670
are going to differ somewhat
from model to model,

00:01:33.670 --> 00:01:38.050
they're not going to differ by
more than a polynomial amount.

00:01:38.050 --> 00:01:40.420
And so, since the
kinds of questions

00:01:40.420 --> 00:01:42.760
we're going to be
asking are going

00:01:42.760 --> 00:01:47.200
to be, basically, whether
problems are polynomial or not,

00:01:47.200 --> 00:01:49.630
it's not going to really
matter which model we

00:01:49.630 --> 00:01:52.690
pick among reasonable
deterministic models.

00:01:52.690 --> 00:01:57.910
And so, the one-tape Turing
machine is a reasonable choice.

00:01:57.910 --> 00:02:00.930
Given that, we defined
time complexity classes,

00:02:00.930 --> 00:02:02.790
the TIME[T(n)] classes.

00:02:02.790 --> 00:02:07.680
We defined the class P,
which was invariant among all

00:02:07.680 --> 00:02:10.470
of those different deterministic
models in the sense

00:02:10.470 --> 00:02:12.608
that it didn't matter
which model we choose,

00:02:12.608 --> 00:02:14.400
we were going to end
up with the same class

00:02:14.400 --> 00:02:18.090
P. So that argues
for its naturalness.

00:02:18.090 --> 00:02:21.150
And we gave an example of
this path problem being in P.

00:02:21.150 --> 00:02:24.060
And we kind of ended that
lecture before the midterm

00:02:24.060 --> 00:02:26.920
with the discussion of this
Hamiltonian path problem.

00:02:26.920 --> 00:02:29.250
So we're going to come
back to that today.

00:02:29.250 --> 00:02:31.500
So today, we're going to
look at non-non-deterministic

00:02:31.500 --> 00:02:36.840
complexity; define the
classes' non-deterministic time

00:02:36.840 --> 00:02:42.160
or NTIME; talk about the class
NP, the P versus NP problem--

00:02:42.160 --> 00:02:46.230
which one of the very famous
unsolved problems in our field;

00:02:46.230 --> 00:02:50.445
and look at dynamic programming,
one of the most basic algorithm

00:02:50.445 --> 00:02:53.420
- polynomial-time algorithms and
polynomial-time reproducibility

00:02:53.420 --> 00:02:57.310
- moving toward our
discussion of NP completeness,

00:02:57.310 --> 00:03:01.080
which we will
begin next lecture.

00:03:01.080 --> 00:03:06.380
So with that, let's move
into today's content, which

00:03:06.380 --> 00:03:10.280
is, well, just a quick review.

00:03:10.280 --> 00:03:14.420
As I mentioned, we defined
the time complexity class.

00:03:14.420 --> 00:03:16.790
The time complexity
class is a collection

00:03:16.790 --> 00:03:26.960
of all of the
languages that you can

00:03:26.960 --> 00:03:31.040
solve in a certain time
bound, within a certain amount

00:03:31.040 --> 00:03:31.950
of time.

00:03:31.950 --> 00:03:36.290
So the time n-squared,
for example,

00:03:36.290 --> 00:03:39.950
is all of the languages
or all of the problems

00:03:39.950 --> 00:03:42.170
that you can solve
in n-squared time.

00:03:42.170 --> 00:03:45.260
We're identifying problems
with languages here.

00:03:45.260 --> 00:03:47.990
And the class P
is the collection

00:03:47.990 --> 00:03:49.550
of all problems
that you can solve

00:03:49.550 --> 00:03:52.440
or all languages that you
can solve in polynomial time.

00:03:52.440 --> 00:03:55.370
So it's the union over
all time n to the k--

00:03:55.370 --> 00:04:00.380
so n-squared, n-cubed, n to
the fifth power, and so on.

00:04:00.380 --> 00:04:04.630
Union out of all
of those bounds.

00:04:04.630 --> 00:04:08.780
The associated languages,
that's the class P.

00:04:08.780 --> 00:04:11.960
And we gave an example,
this path problem.

00:04:11.960 --> 00:04:14.690
We gave an algorithm for path.

00:04:14.690 --> 00:04:18.779
And then, we introduced this
Hamiltonian path problem.

00:04:18.779 --> 00:04:22.820
So if you remember?

00:04:22.820 --> 00:04:25.850
Instead of just asking
given a graph, whether you

00:04:25.850 --> 00:04:30.380
can get from s to t, now we want
to know can I get from s to t,

00:04:30.380 --> 00:04:34.260
but visit every other
node along the way.

00:04:34.260 --> 00:04:38.720
So find a path that goes
through everything else

00:04:38.720 --> 00:04:40.130
and gets from s to t.

00:04:42.780 --> 00:04:46.450
And I should say also
it's a simple path,

00:04:46.450 --> 00:04:51.330
so you're only allowed to go
through every node just once.

00:04:51.330 --> 00:04:59.320
And now, the question
for this problem

00:04:59.320 --> 00:05:04.240
is can we solve that
problem in polynomial time.

00:05:04.240 --> 00:05:06.790
Can we somehow modify
the algorithm for path

00:05:06.790 --> 00:05:09.850
to give us an algorithm
that solves Hamiltonian path

00:05:09.850 --> 00:05:11.200
in polynomial time?

00:05:11.200 --> 00:05:14.440
Of course, we could
solve Hamiltonian path

00:05:14.440 --> 00:05:18.580
with an exponential algorithm
by trying all possible paths.

00:05:18.580 --> 00:05:21.280
And that will give
a correct algorithm,

00:05:21.280 --> 00:05:25.360
but there are exponentially many
different paths and trying them

00:05:25.360 --> 00:05:28.190
all will not give a
polynomial time algorithm.

00:05:28.190 --> 00:05:29.980
So the interesting
problem is can we

00:05:29.980 --> 00:05:33.070
solve that without doing
that brute force searching

00:05:33.070 --> 00:05:36.070
through all possible paths.

00:05:36.070 --> 00:05:41.390
And that's a problem that
no one knows the answer to.

00:05:41.390 --> 00:05:43.130
Despite lots of
effort, people have not

00:05:43.130 --> 00:05:45.450
succeeded in finding
an algorithm for that.

00:05:45.450 --> 00:05:48.560
But on the other hand,
we don't have any idea

00:05:48.560 --> 00:05:51.742
how do you prove there
is no such algorithm.

00:05:51.742 --> 00:05:54.200
I mean, it's conceivable that
one could prove such a thing,

00:05:54.200 --> 00:05:57.390
but we just don't
know how to do it.

00:05:57.390 --> 00:06:03.980
And so, that problem is
an unsolved problem and I

00:06:03.980 --> 00:06:07.490
just-- this isn't
really a note to myself.

00:06:07.490 --> 00:06:09.410
What's kind of
amazing, and this is

00:06:09.410 --> 00:06:11.990
what we're going to be showing
over the next few lectures,

00:06:11.990 --> 00:06:15.890
that there would be very
surprising consequences if you

00:06:15.890 --> 00:06:18.980
could find a way to
solve Hamiltonian path

00:06:18.980 --> 00:06:20.950
in polynomial time.

00:06:20.950 --> 00:06:24.220
Because what that
would immediately yield

00:06:24.220 --> 00:06:28.210
is the polynomial time way of,
say, factoring large numbers

00:06:28.210 --> 00:06:31.210
or solving a large
number of other problems

00:06:31.210 --> 00:06:35.080
that we don't know how to
solve in polynomial time.

00:06:35.080 --> 00:06:38.980
So as we mentioned,
factoring is a problem

00:06:38.980 --> 00:06:41.170
that we only know at
present how to solve

00:06:41.170 --> 00:06:43.600
with an exponential algorithm.

00:06:43.600 --> 00:06:45.100
And it doesn't seem
to have anything

00:06:45.100 --> 00:06:47.920
to do with the
Hamiltonian path problem.

00:06:47.920 --> 00:06:49.210
Seem very different.

00:06:49.210 --> 00:06:52.180
But, yet, if you can
solve Hamiltonian path

00:06:52.180 --> 00:06:54.580
in polynomial time, then
you can factor numbers

00:06:54.580 --> 00:06:56.000
in polynomial time.

00:06:56.000 --> 00:06:59.800
And so, we'll see how
to make that connection.

00:06:59.800 --> 00:07:01.630
That's what we're
building toward

00:07:01.630 --> 00:07:05.510
with the next few lectures.

00:07:05.510 --> 00:07:07.260
OK.

00:07:07.260 --> 00:07:11.450
So happy to take any comments
and questions on that,

00:07:11.450 --> 00:07:15.080
or we'll just move on,
if you have any questions

00:07:15.080 --> 00:07:18.330
on our little review.

00:07:18.330 --> 00:07:20.280
Well, send questions
along and we

00:07:20.280 --> 00:07:25.230
can stop at the end of various
slides to try to answer them.

00:07:25.230 --> 00:07:27.570
And, of course,
write to the TAs,

00:07:27.570 --> 00:07:32.550
who can take your questions
while I'm lecturing.

00:07:32.550 --> 00:07:34.410
OK.

00:07:34.410 --> 00:07:36.180
So to start this
off, we're going

00:07:36.180 --> 00:07:39.990
to have to talk about
non-deterministic complexity

00:07:39.990 --> 00:07:43.930
as a variation of
deterministic complexity.

00:07:43.930 --> 00:07:48.690
So first of all,
all of the machines

00:07:48.690 --> 00:07:51.480
in this part of the course
and the languages, everything

00:07:51.480 --> 00:07:53.610
is going to be decidable
and all the machines

00:07:53.610 --> 00:07:54.670
are going to be deciders.

00:07:54.670 --> 00:07:57.128
So what do we mean when we have
a non-deterministic machine

00:07:57.128 --> 00:07:59.070
which is a decider?

00:07:59.070 --> 00:08:03.190
And that just simply means
that all of the branches--

00:08:03.190 --> 00:08:05.860
it's not just the machine
halts on every input,

00:08:05.860 --> 00:08:09.490
but all of the branches
halt on every input.

00:08:09.490 --> 00:08:11.740
So the non-deterministic
machine is non-deterministic,

00:08:11.740 --> 00:08:13.620
it has lots of
possible branches.

00:08:13.620 --> 00:08:15.780
They all have to halt--

00:08:15.780 --> 00:08:17.730
all of them-- on every input.

00:08:17.730 --> 00:08:19.660
That's what makes a
non-deterministic machine

00:08:19.660 --> 00:08:20.760
a decider.

00:08:20.760 --> 00:08:24.090
And you're going to convert
a non-deterministic decider

00:08:24.090 --> 00:08:26.490
into a deterministic decider.

00:08:26.490 --> 00:08:29.640
But the question is, how much
time would that introduce?

00:08:29.640 --> 00:08:31.800
How much extra time
is that going to cost?

00:08:31.800 --> 00:08:33.240
And the only way
that people know

00:08:33.240 --> 00:08:36.960
at the present time
for that conversion

00:08:36.960 --> 00:08:41.220
would be to do an
exponential increase.

00:08:41.220 --> 00:08:43.350
Basically, to try all
possible branches.

00:08:43.350 --> 00:08:46.880
And that's, of
course, very slow.

00:08:46.880 --> 00:08:52.040
So first, let's
understand what we

00:08:52.040 --> 00:08:55.970
mean by the time used by a
non-deterministic machine.

00:08:55.970 --> 00:08:59.360
And what we mean
by the time used

00:08:59.360 --> 00:09:02.000
is, we're looking at
each individual branch

00:09:02.000 --> 00:09:05.760
individually, separately.

00:09:05.760 --> 00:09:09.140
So a non-deterministic
machine, we'll say,

00:09:09.140 --> 00:09:13.100
runs in a certain amount of
time if all of the branches

00:09:13.100 --> 00:09:17.110
halt within that amount of time.

00:09:17.110 --> 00:09:23.640
So what we do not mean that
the total amount of usage,

00:09:23.640 --> 00:09:27.150
the total amount of effort
by adding up all the branches

00:09:27.150 --> 00:09:28.950
is at most T of n.

00:09:28.950 --> 00:09:32.430
It's just that each branch
individually uses at most

00:09:32.430 --> 00:09:33.480
T of n.

00:09:33.480 --> 00:09:35.380
That's just going to
be our definition.

00:09:35.380 --> 00:09:38.370
And it's going to turn
out to be the right way

00:09:38.370 --> 00:09:40.290
to look at this to
get something useful.

00:09:46.180 --> 00:09:57.720
So now we're going to define
the analogous complexity

00:09:57.720 --> 00:10:01.950
class associated to
non-deterministic computation,

00:10:01.950 --> 00:10:04.680
which we'll call
non-deterministic time.

00:10:04.680 --> 00:10:10.460
So non-deterministic time T of
n is the set of all languages

00:10:10.460 --> 00:10:13.700
that you can do with a
non-deterministic machine that

00:10:13.700 --> 00:10:17.030
runs in order T of n time.

00:10:17.030 --> 00:10:19.820
Just think back
to the definition

00:10:19.820 --> 00:10:24.530
we had for deterministic
complexity, the time class--

00:10:24.530 --> 00:10:26.240
or sometimes people
call it dtime

00:10:26.240 --> 00:10:27.600
to emphasize the difference.

00:10:27.600 --> 00:10:30.230
But let's just say we're
calling it in this course

00:10:30.230 --> 00:10:32.190
time versus ntime.

00:10:32.190 --> 00:10:35.850
So TIME[T(n)] is all of the
language that you can do with

00:10:35.850 --> 00:10:39.800
the one-tape Turing machine
that's deterministic.

00:10:39.800 --> 00:10:42.020
But this here is a
non-deterministic Turing

00:10:42.020 --> 00:10:46.470
machine for
non-deterministic time.

00:10:46.470 --> 00:10:50.360
So the picture that is
good to have in your head

00:10:50.360 --> 00:10:55.790
here would be if you
think of non-determinism

00:10:55.790 --> 00:10:58.880
in terms of a computation
tree thinking of all

00:10:58.880 --> 00:11:02.780
the different branches
of the non-determinism.

00:11:02.780 --> 00:11:05.540
All of those
branches have to halt

00:11:05.540 --> 00:11:09.220
and they have to halt
within the time bound.

00:11:09.220 --> 00:11:11.790
So imagine, here,
this is T of n time.

00:11:11.790 --> 00:11:15.750
All of the branches have to halt
within T of n steps for this,

00:11:15.750 --> 00:11:18.720
a non-deterministic Turing
machine to be running in T of n

00:11:18.720 --> 00:11:22.620
time and to be doing a language
in the NTIME[T(n)] class.

00:11:25.150 --> 00:11:28.000
And by analogy with
what we did before,

00:11:28.000 --> 00:11:37.150
the class NP is the
collection of all languages

00:11:37.150 --> 00:11:39.310
that you can do
non-deterministically

00:11:39.310 --> 00:11:42.700
in polynomial time.

00:11:42.700 --> 00:11:47.520
So it's the union over all
of the ntime classes where

00:11:47.520 --> 00:11:48.870
the bound is polynomial.

00:11:52.160 --> 00:11:55.880
OK, so a lot of this
should look very familiar,

00:11:55.880 --> 00:11:59.360
but we've just added a
bunch of non-deterministic

00:11:59.360 --> 00:12:02.300
and a bunch of Ns in place.

00:12:02.300 --> 00:12:06.170
But the definitions
are very similar.

00:12:06.170 --> 00:12:10.970
And one of the motivations we
had for looking at the class P

00:12:10.970 --> 00:12:15.260
was that it did not depend
on the choice of model,

00:12:15.260 --> 00:12:18.110
as long as the model was
deterministic and reasonable.

00:12:18.110 --> 00:12:22.040
And the class NP is
also going to not

00:12:22.040 --> 00:12:24.320
depend on the
choice of model, as

00:12:24.320 --> 00:12:28.970
long as it's a reasonable
non-deterministic model.

00:12:28.970 --> 00:12:32.780
So it's again a
very natural class

00:12:32.780 --> 00:12:35.150
to look at from a
mathematical standpoint.

00:12:35.150 --> 00:12:39.680
And it also captures something
interesting, kind of,

00:12:39.680 --> 00:12:41.210
from a practical
standpoint - which

00:12:41.210 --> 00:12:43.700
we're going to talk about
over the next couple of slides

00:12:43.700 --> 00:12:47.330
- which is that it captures the
problems where you can easily

00:12:47.330 --> 00:12:53.110
verify when you're a
member of the language.

00:12:53.110 --> 00:12:54.370
OK, so we'll talk about that.

00:12:54.370 --> 00:12:59.710
But if you take, for example,
the Hamiltonian path problem.

00:12:59.710 --> 00:13:02.840
When you find a member
of the language,

00:13:02.840 --> 00:13:06.760
so that is a graph that
does have a Hamiltonian

00:13:06.760 --> 00:13:10.660
path from s to t,
you can easily verify

00:13:10.660 --> 00:13:14.460
that's true by simply
exhibiting the path.

00:13:14.460 --> 00:13:18.360
Not all problems can be
verified in that way.

00:13:18.360 --> 00:13:22.680
But the problems that are in
NP have that special feature--

00:13:22.680 --> 00:13:25.140
that when you have a
member of the language,

00:13:25.140 --> 00:13:27.517
there's a way to verify
that you're a member.

00:13:27.517 --> 00:13:29.850
So we're going to talk about
that, because that's really

00:13:29.850 --> 00:13:32.820
the key to understanding NP--

00:13:32.820 --> 00:13:35.930
this notion of verification.

00:13:35.930 --> 00:13:39.682
OK, so let me go-- there
was a good question here.

00:13:39.682 --> 00:13:41.390
Let me just see if I
want to answer that.

00:13:44.330 --> 00:13:48.380
Yeah, I mean, this is a little
bit of a longer question

00:13:48.380 --> 00:13:52.790
than I want to fully
respond to but--

00:13:52.790 --> 00:13:55.430
well, let's turn to my
next slide, which maybe

00:13:55.430 --> 00:13:58.568
sort bringing that out anyway.

00:13:58.568 --> 00:14:00.360
Actually, it's a couple
of slides from now.

00:14:00.360 --> 00:14:01.580
But I'll get to that point.

00:14:04.380 --> 00:14:07.280
So let's look at Hamiltonian,
the hampath problem.

00:14:07.280 --> 00:14:10.370
And what I'm going to show is
the Hamiltonian path problem

00:14:10.370 --> 00:14:11.500
is in NP.

00:14:11.500 --> 00:14:13.250
And I'm going to walk
you through this one

00:14:13.250 --> 00:14:14.870
kind of slowly.

00:14:14.870 --> 00:14:16.670
So the Hamiltonian
path problem, remember,

00:14:16.670 --> 00:14:22.340
we don't know if it's
in P. But it is in NP.

00:14:22.340 --> 00:14:23.660
So it's in one of these.

00:14:23.660 --> 00:14:27.440
You can solve Hamiltonian
path in polynomial time

00:14:27.440 --> 00:14:29.330
if you're a
non-deterministic machine.

00:14:32.200 --> 00:14:33.350
Why is that?

00:14:33.350 --> 00:14:41.810
Well, it's because of the
parallelism of non-determinism,

00:14:41.810 --> 00:14:43.210
which allows you
to kind of check

00:14:43.210 --> 00:14:46.930
all of the paths on
different branches.

00:14:46.930 --> 00:14:52.540
So let me first describe how I
would write down the algorithm.

00:14:52.540 --> 00:14:56.200
And then, we'll kind of try
to unpack that and understand

00:14:56.200 --> 00:15:01.630
how that actually looks in
terms of the Turing machine's

00:15:01.630 --> 00:15:03.660
computation.

00:15:03.660 --> 00:15:08.110
So first of all, taking the
Hamiltonian path problem,

00:15:08.110 --> 00:15:10.780
you are given an input
now, which is a graph

00:15:10.780 --> 00:15:14.050
and the nodes s and t where
I'm trying to figure out

00:15:14.050 --> 00:15:16.060
is that Hamiltonian path--

00:15:16.060 --> 00:15:18.340
again, which visits
all the nodes,

00:15:18.340 --> 00:15:22.090
which takes you from s to t.

00:15:22.090 --> 00:15:25.870
And we're trying to make now
a non-deterministic machine,

00:15:25.870 --> 00:15:34.030
which is going to accept all
such inputs which have a path.

00:15:34.030 --> 00:15:36.270
So the way this
non-deterministic machine

00:15:36.270 --> 00:15:39.060
is going to work
is it's basically

00:15:39.060 --> 00:15:44.550
going to use its non-determinism
to try all possible paths

00:15:44.550 --> 00:15:47.300
on the different branches.

00:15:47.300 --> 00:15:50.360
And the way I'll
specify that is to say,

00:15:50.360 --> 00:15:52.070
non-deterministically,
we're going

00:15:52.070 --> 00:15:56.390
to write down a candidate
path which is just

00:15:56.390 --> 00:15:59.600
going to be a
sequence of m nodes,

00:15:59.600 --> 00:16:01.910
where we will say that's
the total number of nodes

00:16:01.910 --> 00:16:02.720
of the graph.

00:16:02.720 --> 00:16:06.470
Remember, a Hamiltonian path,
because it visits every node,

00:16:06.470 --> 00:16:09.320
is going to be a path with
exactly m nodes in it.

00:16:11.903 --> 00:16:13.820
So I'm going to write
down a sequence of nodes

00:16:13.820 --> 00:16:16.230
as a candidate path.

00:16:16.230 --> 00:16:18.030
And I'm
non-deterministically going

00:16:18.030 --> 00:16:22.320
to choose every possible
sequence in this way.

00:16:25.370 --> 00:16:27.680
If you'd like to think
of the guessing metaphor

00:16:27.680 --> 00:16:30.050
for non-determinism,
you can think

00:16:30.050 --> 00:16:33.140
of the non-deterministic
machine as guessing

00:16:33.140 --> 00:16:36.320
the right path, which is going
to be the Hamiltonian path

00:16:36.320 --> 00:16:37.100
from s to t.

00:16:37.100 --> 00:16:39.530
But I think for
this discussion, it

00:16:39.530 --> 00:16:41.840
might be more helpful
to think about all

00:16:41.840 --> 00:16:46.243
of the different branches
of the non-determinism.

00:16:46.243 --> 00:16:47.660
Because that's
perhaps more useful

00:16:47.660 --> 00:16:50.180
when we're thinking about
it in terms of the time.

00:16:50.180 --> 00:16:52.520
I think you'll get used
to thinking about it.

00:16:52.520 --> 00:16:54.380
You should be used
to thinking about it

00:16:54.380 --> 00:16:56.540
in many of the different
ways. but maybe

00:16:56.540 --> 00:16:59.210
the computation
tree of all branches

00:16:59.210 --> 00:17:01.590
might be the more
helpful one here.

00:17:01.590 --> 00:17:05.339
So now after we write
down a candidate path

00:17:05.339 --> 00:17:09.260
sequence of nodes,
now I have to check

00:17:09.260 --> 00:17:12.819
that this really is a path.

00:17:12.819 --> 00:17:15.579
And the way I'm going to do that
is to say, well, now if I have

00:17:15.579 --> 00:17:17.770
just a sequence of
nodes written down, what

00:17:17.770 --> 00:17:21.430
does it mean for it to be a
Hamiltonian path from s to t?

00:17:21.430 --> 00:17:26.410
Well, it better start with s
and end with t, first of all.

00:17:26.410 --> 00:17:30.890
And we have to make sure
that every step of the way

00:17:30.890 --> 00:17:33.590
is actually an edge.

00:17:33.590 --> 00:17:39.105
So each pair vi to vi plus 1
has to be an edge in the graph.

00:17:39.105 --> 00:17:40.480
Otherwise, that
sequence of nodes

00:17:40.480 --> 00:17:44.380
is not going to be a legitimate
Hamiltonian path from s to t.

00:17:44.380 --> 00:17:46.690
And it has to be a simple path.

00:17:46.690 --> 00:17:48.250
You can't be repeating nodes.

00:17:51.100 --> 00:17:53.020
These four conditions
together will

00:17:53.020 --> 00:17:56.470
guarantee that we have
a Hamiltonian path.

00:17:56.470 --> 00:18:01.520
And once we have written
down a candidate sequence,

00:18:01.520 --> 00:18:03.950
we can just check that the
sequence actually works.

00:18:06.750 --> 00:18:11.820
At this second stage
of the algorithm,

00:18:11.820 --> 00:18:13.930
non-determinism isn't necessary.

00:18:13.930 --> 00:18:16.290
This is going to be a
deterministic phase.

00:18:16.290 --> 00:18:19.320
But stage one of
the algorithm is

00:18:19.320 --> 00:18:22.260
going to be a non-deterministic
phase where it's writing down

00:18:22.260 --> 00:18:23.940
all possible paths.

00:18:23.940 --> 00:18:26.340
Now, I'm going to try
to unpack that for you

00:18:26.340 --> 00:18:30.120
so you can actually visualize
how the machine is doing this.

00:18:32.810 --> 00:18:35.750
And then, of course,
you know on each branch

00:18:35.750 --> 00:18:37.730
of the non-determinism,
you're going

00:18:37.730 --> 00:18:41.450
to check to see whether the
conditions have been satisfied.

00:18:41.450 --> 00:18:45.860
And on that branch, if the
conditions were not satisfied,

00:18:45.860 --> 00:18:47.750
that branch is going to reject.

00:18:47.750 --> 00:18:50.000
Of course, one of the other
branches might yet accept,

00:18:50.000 --> 00:18:54.960
so that's how
non-determinism works.

00:18:54.960 --> 00:18:55.460
OK?

00:18:55.460 --> 00:19:12.020
So I'd like to visualize
this as the tree

00:19:12.020 --> 00:19:18.930
of the different branches of the
computation of m on its input.

00:19:18.930 --> 00:19:23.550
So here is our non-deterministic
Turing machine.

00:19:23.550 --> 00:19:24.050
Which?

00:19:24.050 --> 00:19:25.400
This one.

00:19:25.400 --> 00:19:31.040
And you provide it with
the input G, s, and t.

00:19:31.040 --> 00:19:34.680
And how is the machine
actually working?

00:19:34.680 --> 00:19:36.980
So when I say
non-deterministically write

00:19:36.980 --> 00:19:40.310
down a sequence of m nodes--

00:19:40.310 --> 00:19:44.510
look, this is getting into
a little bit more detail

00:19:44.510 --> 00:19:46.430
than I would normally
think about it,

00:19:46.430 --> 00:19:49.310
because we try to tend to think
about things at a higher level.

00:19:49.310 --> 00:19:52.250
But just to get us
started, I think

00:19:52.250 --> 00:19:57.190
it's good to think about
this with a bit more detail.

00:19:57.190 --> 00:20:02.100
So let's think of the m nodes
as being numbered, having

00:20:02.100 --> 00:20:05.632
labels numbered 1 through m.

00:20:05.632 --> 00:20:07.590
And I'm going to think
about them being labeled

00:20:07.590 --> 00:20:09.960
by their binary sequences.

00:20:09.960 --> 00:20:11.760
We're going to write
down those nodes.

00:20:11.760 --> 00:20:14.580
That's how the machine is
going to have to operate

00:20:14.580 --> 00:20:17.160
on those numbers for the nodes.

00:20:17.160 --> 00:20:19.860
We'll think about them as
being written in binary.

00:20:19.860 --> 00:20:22.920
And now, as the machine is
going to be writing down,

00:20:22.920 --> 00:20:26.230
let's say, the node v1.

00:20:26.230 --> 00:20:28.270
So it's
non-deterministically picking

00:20:28.270 --> 00:20:30.130
the first node of the sequence.

00:20:30.130 --> 00:20:32.470
What does that
actually mean in terms

00:20:32.470 --> 00:20:41.140
of the step-by-step processing
of the Turing machine m?

00:20:41.140 --> 00:20:45.670
Well, it's going to be
guessing via a sequence

00:20:45.670 --> 00:20:52.820
of non-deterministic moves,
the bits that represent

00:20:52.820 --> 00:20:57.650
the number of the node for v1.

00:20:57.650 --> 00:21:03.670
For example, v1 might be the
node number 5 in the graph.

00:21:03.670 --> 00:21:05.230
Of course,
non-deterministically,

00:21:05.230 --> 00:21:07.060
the machine is on
different branches,

00:21:07.060 --> 00:21:10.480
picking all different
possible choices for v1.

00:21:10.480 --> 00:21:14.230
Those are going to be the
different branches here.

00:21:14.230 --> 00:21:17.260
But one of the
branches might be--

00:21:17.260 --> 00:21:21.153
and what I'm really
representing here, these are--

00:21:21.153 --> 00:21:23.320
I probably could have written
this down on the slide

00:21:23.320 --> 00:21:25.390
here in tiny font.

00:21:25.390 --> 00:21:28.438
But these are like
the 0, 1 choices.

00:21:28.438 --> 00:21:29.980
That's why it's sort
of a binary tree

00:21:29.980 --> 00:21:37.220
here for writing
down the bits of v1.

00:21:37.220 --> 00:21:44.030
So here maybe this could be
101, representing the number

00:21:44.030 --> 00:21:48.500
5, which might be the very
first node that I'm writing down

00:21:48.500 --> 00:21:50.250
in my sequence.

00:21:50.250 --> 00:21:52.590
Some other branch is going
to write down node number 6.

00:21:52.590 --> 00:21:54.960
Some other branch is going
to write down node number 2.

00:21:54.960 --> 00:21:56.460
Because
non-deterministically, we're

00:21:56.460 --> 00:21:58.860
making all possible
choices for v1.

00:21:58.860 --> 00:22:01.950
That's what I'm trying to
show in this little part

00:22:01.950 --> 00:22:06.990
of the computation
of m on this input.

00:22:06.990 --> 00:22:10.910
So then, after it's
finished writing down

00:22:10.910 --> 00:22:17.660
the description of the
node for its choice for v1,

00:22:17.660 --> 00:22:20.630
it goes down to
choose what v2 is.

00:22:20.630 --> 00:22:22.340
Again, non-deterministically,
so there's

00:22:22.340 --> 00:22:30.330
going to be more branches for
each possible choice of v2.

00:22:30.330 --> 00:22:34.260
And so on, node after node.

00:22:34.260 --> 00:22:36.810
Then, it's going to finally
get to the last node, vm.

00:22:36.810 --> 00:22:39.780
It's going to write down
lots of choices for vm.

00:22:39.780 --> 00:22:44.820
And at this point here, we
have completed the first stage

00:22:44.820 --> 00:22:45.760
of the algorithm.

00:22:45.760 --> 00:22:48.390
Now, there's some
huge tree of all

00:22:48.390 --> 00:22:51.000
of the possible choices
for the V's that

00:22:51.000 --> 00:22:53.400
have shown up at this point.

00:22:53.400 --> 00:22:54.980
OK?

00:22:54.980 --> 00:22:58.850
And now, we're going to
move into the second phase.

00:22:58.850 --> 00:23:03.170
So following this,
there's going to be,

00:23:03.170 --> 00:23:12.090
here, a bunch of deterministic
steps of the machine.

00:23:12.090 --> 00:23:17.190
So no more branching
is needed because here,

00:23:17.190 --> 00:23:19.740
we've written down--
at this point,

00:23:19.740 --> 00:23:25.410
we've reached a point in
each of those locations,

00:23:25.410 --> 00:23:28.680
where we've chosen
one of the candidates,

00:23:28.680 --> 00:23:30.900
one of the possible branches--

00:23:30.900 --> 00:23:37.350
one of the possible paths
through the graph, I'm sorry.

00:23:37.350 --> 00:23:43.960
So here, we're guessing
potential paths in the graph.

00:23:43.960 --> 00:23:46.470
And now, we're going to
check that we actually

00:23:46.470 --> 00:23:52.320
have picked a path that's a
Hamiltonian path from s to t.

00:23:52.320 --> 00:23:52.830
OK?

00:23:52.830 --> 00:23:55.290
So each one of these
branches is now

00:23:55.290 --> 00:23:57.120
going to end up
accepting or rejecting.

00:23:57.120 --> 00:23:59.040
And the whole
overall computation

00:23:59.040 --> 00:24:01.350
is going to accept if
at least one of them

00:24:01.350 --> 00:24:03.570
ended up accepting,
which means you actually

00:24:03.570 --> 00:24:06.410
found a Hamiltonian path.

00:24:06.410 --> 00:24:06.910
OK?

00:24:06.910 --> 00:24:12.610
I don't know if that's helpful
to you or not, but that is--

00:24:12.610 --> 00:24:16.460
if there's any questions
on this, let's see.

00:24:16.460 --> 00:24:19.065
Question on is there something--

00:24:23.750 --> 00:24:28.670
trying to draw a connection here
between this and computation

00:24:28.670 --> 00:24:30.830
histories.

00:24:30.830 --> 00:24:34.310
I mean, there is a pattern
here that does come up often

00:24:34.310 --> 00:24:38.150
where you want to check
that something starts right,

00:24:38.150 --> 00:24:41.030
ends right, and that all of
the intermediates are right.

00:24:41.030 --> 00:24:48.410
So I think there is some
deeper connection here.

00:24:48.410 --> 00:24:52.100
Probably too hard
to explain but that

00:24:52.100 --> 00:24:55.490
has something to do with this
Hamiltonian path problem.

00:24:58.300 --> 00:25:00.390
Why are we using
binary representation?

00:25:00.390 --> 00:25:04.377
Well, we're going
to talk about--

00:25:04.377 --> 00:25:06.210
the algorithm would
have worked equally well

00:25:06.210 --> 00:25:12.630
if we used base three
or base five or base 20

00:25:12.630 --> 00:25:18.000
as a way of writing down
our labels for the nodes.

00:25:18.000 --> 00:25:21.740
But in a sense,
it doesn't matter.

00:25:21.740 --> 00:25:24.830
The alphabet has to be finite
though, so that's true.

00:25:24.830 --> 00:25:28.520
I mean, that's why it's not just
in a single step of the Turing

00:25:28.520 --> 00:25:35.690
machine that you would pick
the node the choice for v1.

00:25:35.690 --> 00:25:37.820
You really have to go
to a sequence of steps.

00:25:37.820 --> 00:25:42.110
Because each of the
branches of the machine

00:25:42.110 --> 00:25:44.190
only has a fixed
number of choices.

00:25:44.190 --> 00:25:47.480
So you can't, in a single
step of the Turing machine,

00:25:47.480 --> 00:25:49.798
pick all the different
possibilities for v1.

00:25:49.798 --> 00:25:51.215
That has to go
through a sequence.

00:25:54.090 --> 00:25:54.590
OK.

00:25:59.150 --> 00:26:03.770
Now, let me do a second example,
the problem of composites.

00:26:03.770 --> 00:26:07.010
So the language
of all composites

00:26:07.010 --> 00:26:10.130
are all of the non-primes,
written as binary numbers

00:26:10.130 --> 00:26:10.740
again.

00:26:10.740 --> 00:26:14.540
So we'll talk about the base and
the representation in a second.

00:26:14.540 --> 00:26:18.410
But just imagine these
are all of the numbers

00:26:18.410 --> 00:26:20.330
that are not prime.

00:26:20.330 --> 00:26:24.260
And that language is easily
seen to be a member of NP.

00:26:26.920 --> 00:26:29.110
Here is, again, the
algorithm for that.

00:26:32.450 --> 00:26:38.460
Given x, we want to accept x,
if it's not a prime number.

00:26:38.460 --> 00:26:40.325
So it has some
non-trivial factor.

00:26:45.020 --> 00:26:47.660
So first, the way the
non-deterministic machine

00:26:47.660 --> 00:26:49.980
is going to work is it's
going to guess that factor.

00:26:49.980 --> 00:26:51.530
So non-deterministically,
it's going

00:26:51.530 --> 00:26:55.310
to try every possible factor.

00:26:55.310 --> 00:26:58.840
Y is going to be a
number between 1 and x.

00:26:58.840 --> 00:27:01.060
But not including 1.

00:27:01.060 --> 00:27:02.740
You have to be an
interesting factor,

00:27:02.740 --> 00:27:05.210
so not including one
in the number itself.

00:27:05.210 --> 00:27:07.730
So something
strictly in between.

00:27:07.730 --> 00:27:08.900
And we're going to then--

00:27:08.900 --> 00:27:11.920
after we've
non-deterministically chosen y,

00:27:11.920 --> 00:27:18.260
then we're going to test to
see if y is really a factor.

00:27:18.260 --> 00:27:23.956
So we'll see if y divides evenly
into x with a remainder of 0.

00:27:23.956 --> 00:27:27.110
If that branch successfully
picked the right y,

00:27:27.110 --> 00:27:29.400
it's going to accept.

00:27:29.400 --> 00:27:32.820
And some other branch where it
might have picked the wrong y,

00:27:32.820 --> 00:27:34.110
will not.

00:27:34.110 --> 00:27:37.800
And if x is really
a composite number,

00:27:37.800 --> 00:27:41.460
some branch will
find the factor.

00:27:41.460 --> 00:27:43.460
Now, the base doesn't matter.

00:27:43.460 --> 00:27:48.160
Could have used base 10, because
you can convert from one base

00:27:48.160 --> 00:27:49.460
to another.

00:27:49.460 --> 00:27:52.330
So this is really in terms
of our representation

00:27:52.330 --> 00:27:54.190
of the number.

00:27:54.190 --> 00:27:58.177
But I do want to make one
point here, that changing--

00:27:58.177 --> 00:28:00.010
we don't want to write
the number in unary--

00:28:03.400 --> 00:28:07.690
writing the number of
k as a sequence of k1s.

00:28:07.690 --> 00:28:09.980
That's not really a base.

00:28:09.980 --> 00:28:12.670
That's just an exponential
representation for the number

00:28:12.670 --> 00:28:14.560
and that changes the game.

00:28:14.560 --> 00:28:20.440
Because if you make the
input exponentially larger,

00:28:20.440 --> 00:28:24.790
then it's going to change
whether the algorithm

00:28:24.790 --> 00:28:28.330
relative to that exponentially
larger input is polynomial

00:28:28.330 --> 00:28:29.560
or not.

00:28:29.560 --> 00:28:34.912
So an algorithm that might have
been exponential originally

00:28:34.912 --> 00:28:36.370
when the number's
written in binary

00:28:36.370 --> 00:28:38.870
might become polynomial if the
numbers are written in unary.

00:28:41.640 --> 00:28:43.620
And I do want to
mention as a side note,

00:28:43.620 --> 00:28:46.860
that the composites language--

00:28:46.860 --> 00:28:48.720
or primes, for that matter--

00:28:48.720 --> 00:28:52.680
both are NP.

00:28:52.680 --> 00:28:54.180
But we won't cover that.

00:28:54.180 --> 00:28:58.290
So whereas the Hamiltonian
path problem is not

00:28:58.290 --> 00:29:02.490
known whether it's NP, the
primes and composites problem

00:29:02.490 --> 00:29:04.440
are NP.

00:29:04.440 --> 00:29:05.190
So that was known.

00:29:05.190 --> 00:29:08.130
That was actually a very
big result in the field.

00:29:08.130 --> 00:29:16.020
Solved by folks at one of the
Indian Institute of Technology

00:29:16.020 --> 00:29:18.691
back about almost
20 years ago now.

00:29:23.440 --> 00:29:27.310
So let's turn here, to
trying to get an intuitive

00:29:27.310 --> 00:29:28.750
feeling for P and NP.

00:29:28.750 --> 00:29:33.160
And we'll return now to this
notion of NP corresponding

00:29:33.160 --> 00:29:34.810
to easy verifiability.

00:29:38.800 --> 00:29:42.490
NP are the languages where you
can easily verify membership

00:29:42.490 --> 00:29:43.630
quickly.

00:29:43.630 --> 00:29:45.700
I'll try to explain
what that means.

00:29:45.700 --> 00:29:50.530
In contrast, P are the languages
where you can test membership

00:29:50.530 --> 00:29:51.820
quickly.

00:29:51.820 --> 00:29:57.940
By quickly, I'm using
polynomial time.

00:29:57.940 --> 00:30:01.150
That's going to be,
for us, that's what

00:30:01.150 --> 00:30:02.770
quickly means in this course.

00:30:07.650 --> 00:30:11.880
In the case of the
Hamiltonian path problem,

00:30:11.880 --> 00:30:13.980
the way you verify
the membership

00:30:13.980 --> 00:30:16.530
is you give the path.

00:30:16.530 --> 00:30:18.238
In the case of the
composites, the way

00:30:18.238 --> 00:30:20.280
you verify the membership
is you give the factor.

00:30:22.960 --> 00:30:27.350
In those two cases,
and in general,

00:30:27.350 --> 00:30:29.920
when we have a
problem that's in NP,

00:30:29.920 --> 00:30:35.350
we think of this
verification as having--

00:30:35.350 --> 00:30:38.260
we give it a special name,
called a certificate,

00:30:38.260 --> 00:30:40.540
or sometimes a
short certificate,

00:30:40.540 --> 00:30:43.870
to emphasize the polynomiality
of the certificate.

00:30:43.870 --> 00:30:46.600
It's like a way of
proving that you're

00:30:46.600 --> 00:30:47.815
a member of the language.

00:30:51.050 --> 00:30:54.550
In the case of COMPOSITES,
the proof is the factor.

00:30:54.550 --> 00:30:56.410
In the case of
HAMPATH, the proof

00:30:56.410 --> 00:31:01.080
is the path, the
Hamiltonian path.

00:31:01.080 --> 00:31:04.125
Contrast that, for example,
if you had a prime number.

00:31:07.570 --> 00:31:10.230
Proving a number is composite
is easy because you just

00:31:10.230 --> 00:31:11.970
exhibit the factor.

00:31:11.970 --> 00:31:15.470
How would you prove
that a number is prime?

00:31:15.470 --> 00:31:20.210
What's the short
certificate of proving

00:31:20.210 --> 00:31:23.150
that some number has no factor?

00:31:23.150 --> 00:31:25.530
That's not so obvious.

00:31:25.530 --> 00:31:27.030
In fact, there are
ways of doing it,

00:31:27.030 --> 00:31:29.900
which I'm not going to get
into in the case of testing

00:31:29.900 --> 00:31:31.910
of numbers prime.

00:31:31.910 --> 00:31:36.470
And now it's even known to be
in P, so that's even better.

00:31:36.470 --> 00:31:40.420
But there's no obvious way
of proving that a number is

00:31:40.420 --> 00:31:42.580
prime with a short certificate.

00:31:45.190 --> 00:31:48.850
This concept of
being able to verify

00:31:48.850 --> 00:31:51.130
when you are a member
of the language, that's

00:31:51.130 --> 00:31:54.870
key to understanding NP.

00:31:54.870 --> 00:31:58.170
That's the intuition you
need to develop and hopefully

00:31:58.170 --> 00:32:00.450
take away from today's
lecture, or at least

00:32:00.450 --> 00:32:03.570
by thinking about it,
reading the book, and so on.

00:32:07.330 --> 00:32:11.536
If you compare these
two classes, P and NP.

00:32:11.536 --> 00:32:15.160
P, first of all, is going
to be a subset of NP,

00:32:15.160 --> 00:32:17.290
both in terms of the
way we defined it

00:32:17.290 --> 00:32:19.240
because, deterministic
machines are

00:32:19.240 --> 00:32:22.670
a special case of
non-deterministic machines.

00:32:22.670 --> 00:32:25.600
But also if you want to think
about testing membership,

00:32:25.600 --> 00:32:27.700
if you can test
membership easily

00:32:27.700 --> 00:32:30.942
then you can certainly verify
it in terms of the certificate.

00:32:30.942 --> 00:32:31.900
You don't even need it.

00:32:31.900 --> 00:32:33.760
The certificate is
irrelevant at that point.

00:32:33.760 --> 00:32:35.860
Because whatever
the certificate is,

00:32:35.860 --> 00:32:40.240
you can still test
yourself whether the input

00:32:40.240 --> 00:32:41.980
is in the language or not.

00:32:47.010 --> 00:32:48.610
The big question,
as I mentioned,

00:32:48.610 --> 00:32:51.640
is whether these two
classes are the same.

00:32:51.640 --> 00:32:56.880
So does being able to verify
membership quickly, say

00:32:56.880 --> 00:32:59.970
with one of these certificates,
allow you to dispense

00:32:59.970 --> 00:33:00.900
with the certificate?

00:33:00.900 --> 00:33:02.850
Not even need a
certificate and just test

00:33:02.850 --> 00:33:06.270
it for yourself whether
you're in the language.

00:33:06.270 --> 00:33:10.570
And do that in polynomial time.

00:33:10.570 --> 00:33:12.310
That's the question.

00:33:12.310 --> 00:33:16.580
For a problem like
Hamiltonian path,

00:33:16.580 --> 00:33:19.280
do you need to
search for the answer

00:33:19.280 --> 00:33:21.050
if you're doing it
deterministically?

00:33:21.050 --> 00:33:26.570
Or can you somehow
avoid that and just

00:33:26.570 --> 00:33:30.950
come up with the answer
directly with a polynomial time

00:33:30.950 --> 00:33:32.780
solution?

00:33:32.780 --> 00:33:35.300
Nobody knows the answer to that.

00:33:35.300 --> 00:33:37.700
And it goes back, at this
point, quite a long time.

00:33:37.700 --> 00:33:39.350
It's almost 60 years now.

00:33:43.100 --> 00:33:46.740
That problem has
been around 60 years.

00:33:46.740 --> 00:33:48.030
No, that would be 50 years.

00:33:48.030 --> 00:33:49.830
No, 50 years, almost 50 years.

00:33:56.770 --> 00:33:59.110
Most people believe that
P is different from NP.

00:33:59.110 --> 00:34:01.360
In other words, that
there are problems in P--

00:34:01.360 --> 00:34:05.080
in NP which are not
in P. A candidate

00:34:05.080 --> 00:34:08.150
would be the Hamiltonian
path problem.

00:34:08.150 --> 00:34:12.230
But it seems to be very
hard to prove that.

00:34:12.230 --> 00:34:13.989
And part of the
reason is, how do you

00:34:13.989 --> 00:34:17.830
prove that a problem
like Hamiltonian path

00:34:17.830 --> 00:34:21.699
does not have a
polynomial time algorithm.

00:34:21.699 --> 00:34:23.920
It's very tricky to do
that, because the class

00:34:23.920 --> 00:34:26.530
of polynomial time algorithms
is a very rich class.

00:34:26.530 --> 00:34:30.130
Polynomial time algorithms
are very powerful.

00:34:30.130 --> 00:34:32.670
And to try to prove--
there's no clever way

00:34:32.670 --> 00:34:35.219
of solving the
Hamiltonian path problem.

00:34:35.219 --> 00:34:38.451
It just seems to be beyond
our present day mathematics.

00:34:38.451 --> 00:34:40.409
I believe someday somebody's
going to solve it.

00:34:40.409 --> 00:34:45.935
But so far, no
one has succeeded.

00:34:45.935 --> 00:34:47.940
So what I thought
we would do is--

00:34:47.940 --> 00:34:49.860
I think I have a check-in here.

00:34:49.860 --> 00:34:50.909
Yes.

00:34:50.909 --> 00:34:52.275
And then we'll stop for a break.

00:34:54.780 --> 00:34:59.280
Let's look at the complementary
problem, HAMPATH complement.

00:35:03.660 --> 00:35:08.180
You're in the language now
if you don't have a path.

00:35:08.180 --> 00:35:10.684
So is that complementary
problem in NP?

00:35:15.390 --> 00:35:17.520
For that to be
the case, we would

00:35:17.520 --> 00:35:22.710
need to have short certificates
of when a graph does not

00:35:22.710 --> 00:35:26.070
have a Hamiltonian path.

00:35:29.500 --> 00:35:30.540
I leave it to you.

00:35:30.540 --> 00:35:31.540
There are three choices.

00:35:34.330 --> 00:35:36.600
OK?

00:35:36.600 --> 00:35:39.660
Going to stop here, so make
sure you get your participation

00:35:39.660 --> 00:35:40.440
credit here.

00:35:42.980 --> 00:35:46.235
I'm going to end
the polling now.

00:35:46.235 --> 00:35:46.735
Interesting.

00:35:46.735 --> 00:35:49.060
[LAUGHS] So the
majority is wrong.

00:35:49.060 --> 00:35:50.790
Well, not wrong, we don't know.

00:35:56.950 --> 00:35:58.900
I think the only fair
answer to this question

00:35:58.900 --> 00:36:04.300
is C. Because we don't
know whether or not

00:36:04.300 --> 00:36:06.370
we can give short
certificates for a graph

00:36:06.370 --> 00:36:08.560
not to have a Hamiltonian path.

00:36:08.560 --> 00:36:15.030
If P equaled NP, then you
can test in polynomial time

00:36:15.030 --> 00:36:17.970
whether a graph has
a Hamiltonian path.

00:36:17.970 --> 00:36:20.610
And then the computation
itself would be a certificate,

00:36:20.610 --> 00:36:23.647
whether it has a path or
whether it doesn't have a path.

00:36:23.647 --> 00:36:25.980
Because it would be something
that you can check easily.

00:36:28.950 --> 00:36:35.700
Since we don't know for sure
that P is different from NP,

00:36:35.700 --> 00:36:38.250
P could be equal to
NP, then it's possible

00:36:38.250 --> 00:36:43.510
that we could give
a short certificate.

00:36:43.510 --> 00:36:47.780
Namely, the computation of
the polynomial algorithm.

00:36:47.780 --> 00:36:54.700
So the only really reasonable
answer to this question

00:36:54.700 --> 00:36:55.630
is that we don't know.

00:36:59.060 --> 00:37:00.470
Just ponder that.

00:37:03.920 --> 00:37:08.900
Those of you who answered
yes, however, need to go back.

00:37:08.900 --> 00:37:12.470
And I put this here
explicitly because I

00:37:12.470 --> 00:37:18.180
know this is a confusion
for, well I can

00:37:18.180 --> 00:37:19.620
see, for quite a few of you.

00:37:25.310 --> 00:37:27.965
When we have
non-deterministic computation

00:37:27.965 --> 00:37:30.020
and you have a
non-deterministic machine,

00:37:30.020 --> 00:37:33.710
you can't simply
invert the answer

00:37:33.710 --> 00:37:35.960
and get back a
non-deterministic machine.

00:37:35.960 --> 00:37:39.240
Non-determinism does
not work that way.

00:37:39.240 --> 00:37:45.800
If you remember, the complement
of a pushdown automaton

00:37:45.800 --> 00:37:47.210
is not a pushdown automaton.

00:37:49.970 --> 00:37:51.980
If you have a
non-deterministic machine

00:37:51.980 --> 00:37:54.620
and you invert all of
the responses on each

00:37:54.620 --> 00:37:59.900
of the branches, it's not going
to be recognizing or deciding

00:37:59.900 --> 00:38:01.055
the complementary language.

00:38:07.150 --> 00:38:09.160
I think that this
is something you--

00:38:09.160 --> 00:38:13.120
if you answered yes, you
need to go back and make

00:38:13.120 --> 00:38:18.430
sure you understand why yes
is not a reasonable answer

00:38:18.430 --> 00:38:19.480
to this question.

00:38:19.480 --> 00:38:22.220
Because that's not how
non-determinism works.

00:38:22.220 --> 00:38:26.630
So you have a not
complete understanding

00:38:26.630 --> 00:38:27.590
of non-determinism.

00:38:27.590 --> 00:38:30.830
And that's going to be really
important for us going forward.

00:38:30.830 --> 00:38:35.840
I really urge you to figure out
and understand why yes is not

00:38:35.840 --> 00:38:38.780
a good answer to this check-in.

00:38:38.780 --> 00:38:40.470
OK, so I think we will--

00:38:40.470 --> 00:38:45.320
we can talk about that
more over the break.

00:38:45.320 --> 00:38:51.125
And so we'll return
here in five minutes.

00:38:55.320 --> 00:39:01.470
Somebody's asking about--
can infinite sequences

00:39:01.470 --> 00:39:02.640
be generated by the machine.

00:39:05.640 --> 00:39:10.520
When we're talking
about, especially

00:39:10.520 --> 00:39:14.000
in the complexity section of the
course, all of the computations

00:39:14.000 --> 00:39:17.102
are going to be bounded in time.

00:39:17.102 --> 00:39:19.310
So we're not going to be
thinking about infinite runs

00:39:19.310 --> 00:39:19.935
of the machine.

00:39:19.935 --> 00:39:21.570
That's not going to
be relevant for us.

00:39:21.570 --> 00:39:24.230
So let's not think about that.

00:39:24.230 --> 00:39:26.135
How does a Turing
machine perform division?

00:39:28.980 --> 00:39:30.870
How does a Turing
machine perform division?

00:39:30.870 --> 00:39:34.590
Well, how do you
perform division?

00:39:34.590 --> 00:39:42.630
[LAUGHS] Long division is an
operation that can run in--

00:39:42.630 --> 00:39:46.530
the long division procedure
that you learn in grade school,

00:39:46.530 --> 00:39:50.100
you can implement that
on a Turing machine.

00:39:50.100 --> 00:39:52.620
Yes, a Turing machine
can definitely perform--

00:39:52.620 --> 00:39:54.750
do long division, or
division of one integer

00:39:54.750 --> 00:39:57.420
by another in polynomial time.

00:40:04.062 --> 00:40:04.770
Another question.

00:40:04.770 --> 00:40:08.760
Can we generally say,
try dividing y by x?

00:40:08.760 --> 00:40:11.370
Or do we have to enumerate
a string of length y

00:40:11.370 --> 00:40:12.455
and cross off every-- no.

00:40:12.455 --> 00:40:13.830
I think that's
the same question.

00:40:18.430 --> 00:40:20.840
I mean, if you have
numbers written in binary,

00:40:20.840 --> 00:40:22.090
how would you do the division?

00:40:22.090 --> 00:40:27.550
You're not going to
use long division.

00:40:27.550 --> 00:40:30.250
Anything such as
the thing that's

00:40:30.250 --> 00:40:34.780
proposed here by the
questioner is going

00:40:34.780 --> 00:40:36.560
to be an exponential algorithm.

00:40:36.560 --> 00:40:37.720
So don't do it that way.

00:40:46.790 --> 00:40:48.830
Why does primes in P--

00:40:48.830 --> 00:40:51.680
composites in P not
imply primes in P?

00:40:51.680 --> 00:40:53.410
It does imply.

00:40:53.410 --> 00:40:57.800
If composites are in P,
then primes is in P as well.

00:40:57.800 --> 00:41:00.620
When you have a
deterministic machine,

00:41:00.620 --> 00:41:02.240
you can flip the answer.

00:41:02.240 --> 00:41:04.430
When you have a
non-deterministic machine,

00:41:04.430 --> 00:41:06.330
you may not be able
to flip the answer.

00:41:06.330 --> 00:41:10.880
So deterministic machine
just having a single branch,

00:41:10.880 --> 00:41:12.650
you can make another
deterministic machine

00:41:12.650 --> 00:41:14.780
that runs in the same
amount of time that does

00:41:14.780 --> 00:41:16.670
the complementary language.

00:41:16.670 --> 00:41:21.170
Because for
deterministic machines,

00:41:21.170 --> 00:41:25.640
just like for deciders in
the computability section,

00:41:25.640 --> 00:41:28.410
you can just invert the answer.

00:41:28.410 --> 00:41:33.500
There is an analogy here
between P and decidability,

00:41:33.500 --> 00:41:35.690
and NP and recognizability.

00:41:35.690 --> 00:41:38.510
It's not an airtight
analogy here,

00:41:38.510 --> 00:41:40.700
but there is some
relationship there.

00:41:44.620 --> 00:41:46.810
Somebody's asking me, what
are the implications of P

00:41:46.810 --> 00:41:48.250
equal to NP?

00:41:48.250 --> 00:41:50.620
Lots of implications.

00:41:50.620 --> 00:41:51.970
Too long to enumerate now.

00:41:51.970 --> 00:41:55.600
But, for example, you
would be able to break

00:41:55.600 --> 00:42:01.325
basically all cryptosystems that
I'm aware of, if P equal to NP.

00:42:01.325 --> 00:42:02.950
So we would have a
lot of consequences.

00:42:05.500 --> 00:42:08.200
Somebody's asking,
so composites--

00:42:08.200 --> 00:42:10.120
primality and
compositeness testing

00:42:10.120 --> 00:42:11.920
is solvable in polynomial time.

00:42:11.920 --> 00:42:14.590
But factoring,
interestingly enough,

00:42:14.590 --> 00:42:16.990
is not known to be solvable
in polynomial time.

00:42:22.187 --> 00:42:24.770
We may talk about this a little
bit toward the end of the term

00:42:24.770 --> 00:42:25.437
if we have time.

00:42:28.830 --> 00:42:32.040
The algorithms for testing
whether a number is

00:42:32.040 --> 00:42:35.190
prime or composite
in polynomial time

00:42:35.190 --> 00:42:37.890
do not operate by
looking for factors.

00:42:37.890 --> 00:42:41.400
They operate in an entirely
different way, basically

00:42:41.400 --> 00:42:46.890
by showing that a number is
prime or composite by looking

00:42:46.890 --> 00:42:48.570
at certain properties
of that number.

00:42:48.570 --> 00:42:51.120
But without testing
whether it has--

00:42:51.120 --> 00:42:53.940
testing, but without
finding a factor.

00:43:03.900 --> 00:43:09.390
Another question
here about asking

00:43:09.390 --> 00:43:11.220
when we talk about
encodings, do we

00:43:11.220 --> 00:43:14.215
have to say how we
encode numbers, values?

00:43:14.215 --> 00:43:15.090
No, we don't have to.

00:43:15.090 --> 00:43:19.770
We usually don't have to get
into spelling out encodings,

00:43:19.770 --> 00:43:22.080
as long as they're
reasonable encodings.

00:43:22.080 --> 00:43:25.028
So you don't have to usually.

00:43:25.028 --> 00:43:27.570
We're going to be talking about
things at a high enough level

00:43:27.570 --> 00:43:32.070
that the specific encodings
are not going to matter.

00:43:32.070 --> 00:43:37.680
Let's return to the
rest of our lecture.

00:43:37.680 --> 00:43:46.440
When we talk about, say,
this P versus NP problem.

00:43:46.440 --> 00:43:53.300
And how do you show
that a problem might not

00:43:53.300 --> 00:43:58.190
be solvable in P, like the
Hamiltonian path problem?

00:43:58.190 --> 00:44:08.810
Many people who are not
practitioners in the field,

00:44:08.810 --> 00:44:11.420
know about the P
versus NP problems--

00:44:11.420 --> 00:44:14.900
over the years, I've
gotten many, many emails

00:44:14.900 --> 00:44:19.475
and physical letters
from people about that.

00:44:22.550 --> 00:44:24.290
Since I've spent
some time thinking--

00:44:24.290 --> 00:44:29.120
I'm known as having spent
some time thinking about it.

00:44:29.120 --> 00:44:31.760
People claim to
solve the problem,

00:44:31.760 --> 00:44:38.630
solve P is NP, the P versus NP
problem, by basically saying,

00:44:38.630 --> 00:44:48.220
problems like Hamiltonian path
or other similar problems,

00:44:48.220 --> 00:44:52.210
basically there's clearly no way
to solve them without searching

00:44:52.210 --> 00:44:53.560
through a lot of possibilities.

00:44:53.560 --> 00:44:55.435
And then they go through
a big, long analysis

00:44:55.435 --> 00:44:56.950
showing that there
are exponentially

00:44:56.950 --> 00:44:58.990
many possibilities.

00:45:08.010 --> 00:45:10.980
A lot of the proofs that
claim to solve P versus NP,

00:45:10.980 --> 00:45:12.240
they all look like that.

00:45:12.240 --> 00:45:13.470
You only have to look--

00:45:13.470 --> 00:45:15.270
somewhere in that
paper, there's going

00:45:15.270 --> 00:45:21.610
to be a statement along the
lines, "to solve this problem,

00:45:21.610 --> 00:45:25.880
clearly you have
to do it this way."

00:45:25.880 --> 00:45:28.610
And that's the flaw
in the reasoning.

00:45:28.610 --> 00:45:32.930
Because just like for
the factoring problem--

00:45:32.930 --> 00:45:36.650
just like for the
compositeness testing problem,

00:45:36.650 --> 00:45:38.570
you don't necessarily
have to solve it

00:45:38.570 --> 00:45:39.945
by searching for factors.

00:45:39.945 --> 00:45:41.570
There might be some
other way to do it.

00:45:41.570 --> 00:45:44.390
You might be able to
solve the Hamiltonian path

00:45:44.390 --> 00:45:46.670
problem without searching
for Hamiltonian paths.

00:45:46.670 --> 00:45:49.140
There might be some other
process that you can use,

00:45:49.140 --> 00:45:52.490
which would give you the answer.

00:45:52.490 --> 00:45:54.250
The class of polynomial
time algorithms

00:45:54.250 --> 00:45:57.480
is very rich, can do
many, many things.

00:45:57.480 --> 00:46:02.600
And I wanted to present to
you one of the most important

00:46:02.600 --> 00:46:04.220
polynomial time algorithms.

00:46:04.220 --> 00:46:07.190
In a sense, you can
make a certain argument

00:46:07.190 --> 00:46:12.800
that this is the most
fundamental polynomial time

00:46:12.800 --> 00:46:13.760
algorithm.

00:46:13.760 --> 00:46:16.790
Some people might
argue with me on that.

00:46:16.790 --> 00:46:20.850
And that's a process called
dynamic programming, which

00:46:20.850 --> 00:46:22.620
I'm sure some of you
have seen already

00:46:22.620 --> 00:46:28.170
in your algorithms classes,
and some of you may not have.

00:46:28.170 --> 00:46:30.340
Since you have a
homework problem on it,

00:46:30.340 --> 00:46:33.910
I want to spend a little
time describing it to you.

00:46:33.910 --> 00:46:39.240
And that's useful for solving
this A CFG problem, which

00:46:39.240 --> 00:46:41.880
you may remember from the
first half of the course,

00:46:41.880 --> 00:46:48.150
involving testing if a
grammar generates a string.

00:46:48.150 --> 00:46:50.100
So you remember
this A CFG problem?

00:46:50.100 --> 00:46:53.770
You're giving a grammar,
context-free grammar,

00:46:53.770 --> 00:46:54.640
and a string.

00:46:54.640 --> 00:46:58.340
And I want to know, is it in
the language of the grammar.

00:46:58.340 --> 00:47:02.420
So that's going to turn out to
be solvable in polynomial time,

00:47:02.420 --> 00:47:05.176
but only with kind of
a clever algorithm.

00:47:09.000 --> 00:47:12.350
Remember, it's decidable.

00:47:12.350 --> 00:47:15.410
We decided it by
making sure that you

00:47:15.410 --> 00:47:18.740
are converting the grammar
in Chomsky normal form.

00:47:18.740 --> 00:47:21.560
Then all the derivations
have a certain length.

00:47:21.560 --> 00:47:24.560
You just try all the possible
derivations of that length,

00:47:24.560 --> 00:47:27.520
and you accept if any of
those derivations generate w.

00:47:31.540 --> 00:47:34.690
You may remember that from
the first half of the course.

00:47:37.660 --> 00:47:41.520
That immediately
gives an NP type

00:47:41.520 --> 00:47:44.280
algorithm for this language.

00:47:44.280 --> 00:47:49.217
Because basically, you
non-deterministically--

00:47:49.217 --> 00:47:51.050
instead of trying the
most sequentially, all

00:47:51.050 --> 00:47:52.580
of these derivations,
you try them

00:47:52.580 --> 00:47:54.950
in parallel
non-deterministically.

00:47:54.950 --> 00:47:58.280
So non-deterministically,
you pick some derivation

00:47:58.280 --> 00:48:00.000
of that length.

00:48:00.000 --> 00:48:03.290
And you accept it if
it generates the input.

00:48:03.290 --> 00:48:12.073
This classically
fits our model of NP.

00:48:12.073 --> 00:48:13.990
You can think of it as
guessing the derivation

00:48:13.990 --> 00:48:15.310
and checking that it works.

00:48:15.310 --> 00:48:19.210
Or, in parallel, writing down
all possible derivations.

00:48:19.210 --> 00:48:26.080
But this A CFG
problem is classically

00:48:26.080 --> 00:48:29.425
an NP problem, a problem in NP.

00:48:32.950 --> 00:48:36.220
And if you just imagine--

00:48:36.220 --> 00:48:38.740
then what's going to
be the certificate?

00:48:38.740 --> 00:48:41.890
If you found an input that's
in the language, that's

00:48:41.890 --> 00:48:45.570
generated by the grammar, the
certificate is the derivation.

00:48:45.570 --> 00:48:47.000
So if you look at
it that way, you

00:48:47.000 --> 00:48:49.310
might think, well, that's
the best you can do.

00:48:49.310 --> 00:48:51.890
This is going to be
an NP problem, in NP,

00:48:51.890 --> 00:48:53.930
and is not going to
be a way of avoiding

00:48:53.930 --> 00:48:55.220
searching for the derivation.

00:48:55.220 --> 00:48:56.667
But that's not true.

00:48:56.667 --> 00:48:59.000
There is a way of avoiding
searching for the derivation.

00:48:59.000 --> 00:49:03.410
You can build up the derivation
using dynamic programming.

00:49:03.410 --> 00:49:07.220
And so that's what I
wanted to describe for you,

00:49:07.220 --> 00:49:07.997
how that works.

00:49:07.997 --> 00:49:09.830
Also partly because
it's a homework problem.

00:49:09.830 --> 00:49:14.143
And I think dynamic programming
is a very important algorithm.

00:49:18.090 --> 00:49:25.940
Before we describe what
dynamic programming is,

00:49:25.940 --> 00:49:28.670
which is very
simple, by the way,

00:49:28.670 --> 00:49:31.460
let's try to work up to
it by making an attempt

00:49:31.460 --> 00:49:34.460
to solve this problem just
using ordinary recursion.

00:49:37.140 --> 00:49:40.120
How would we solve
the A CFG problem?

00:49:40.120 --> 00:49:43.640
So you're given grammar,
you're given an input.

00:49:43.640 --> 00:49:46.790
Let's assume the grammar
is in Chomsky normal form.

00:49:49.610 --> 00:49:51.060
That's going to be useful.

00:49:51.060 --> 00:49:53.210
So it's a Chomsky
normal form grammar.

00:49:53.210 --> 00:49:59.437
And we want to see how to
test if you can generate w.

00:49:59.437 --> 00:50:01.145
And it's going to be
recursive algorithm.

00:50:01.145 --> 00:50:05.690
The recursive algorithm is going
to actually solve something

00:50:05.690 --> 00:50:07.490
slightly more general.

00:50:07.490 --> 00:50:08.960
I'm going to give
you the grammar.

00:50:08.960 --> 00:50:10.500
I'm going to give
you the string.

00:50:10.500 --> 00:50:14.810
And I'm also going to allow you
to start at some other variable

00:50:14.810 --> 00:50:16.550
besides the start variable.

00:50:16.550 --> 00:50:19.340
I'm going to give
you some variable, R,

00:50:19.340 --> 00:50:23.120
and I know I want to know, can
I generate w starting at R?

00:50:26.880 --> 00:50:28.870
So that's my slightly
more general problem,

00:50:28.870 --> 00:50:31.040
which is going to be
useful in the recursion.

00:50:33.720 --> 00:50:36.630
So the input now
to this algorithm

00:50:36.630 --> 00:50:41.128
is the grammar, the input,
and the starting variable.

00:50:41.128 --> 00:50:42.920
And now how is the
algorithm going to work?

00:50:42.920 --> 00:50:46.370
It's going to try to
test, can I get to--

00:50:46.370 --> 00:50:48.920
is there some
derivation, pictured here

00:50:48.920 --> 00:50:53.000
as the parse tree,
for w starting at R?

00:50:53.000 --> 00:50:55.900
That's what the algorithm
is trying to answer.

00:50:55.900 --> 00:50:58.750
Can I get w from R?

00:50:58.750 --> 00:51:04.610
The way it's going to do that
is it's going to try to divide w

00:51:04.610 --> 00:51:10.860
into the two strings
in all possible ways.

00:51:10.860 --> 00:51:15.290
Which sounds like it might
be exponential, but it isn't.

00:51:15.290 --> 00:51:18.800
There's only a polynomial
number of ways to divide

00:51:18.800 --> 00:51:22.730
the string into two substrings.

00:51:22.730 --> 00:51:25.520
Just of order n, just depending
where you make that cut.

00:51:29.480 --> 00:51:30.630
So that's not too bad.

00:51:30.630 --> 00:51:32.660
There's a polynomial,
there's only n ways

00:51:32.660 --> 00:51:33.800
of making that division.

00:51:33.800 --> 00:51:44.430
And also I'm going to try every
possible rule that comes from R

00:51:44.430 --> 00:51:45.950
that generates two variables.

00:51:45.950 --> 00:51:49.290
So these are what's allowed
in Chomsky normal form,

00:51:49.290 --> 00:51:53.280
R goes to ST.

00:51:53.280 --> 00:51:57.480
For each possible way
of cutting w into x, y,

00:51:57.480 --> 00:51:59.190
and for each
possible rule, R goes

00:51:59.190 --> 00:52:04.430
to ST, I'm going to see,
recursively, can I get

00:52:04.430 --> 00:52:08.750
from S to x, and from T to y.

00:52:08.750 --> 00:52:10.460
So I'm going to use
my recursion now.

00:52:10.460 --> 00:52:17.030
Now that I have smaller
strings instead of w,

00:52:17.030 --> 00:52:21.580
I can apply recursion and
try to answer it that way.

00:52:21.580 --> 00:52:23.980
And this algorithm will work.

00:52:23.980 --> 00:52:28.810
If I found a way to cut w
into x, y, and I found a rule,

00:52:28.810 --> 00:52:33.310
R goes to ST such that S
generates x and T generates y,

00:52:33.310 --> 00:52:34.920
then I'm good.

00:52:34.920 --> 00:52:39.850
I know I can generate w from R.

00:52:39.850 --> 00:52:44.050
And if there's no way of
cutting w up to satisfy that,

00:52:44.050 --> 00:52:51.040
or if I can't find any
way to divide w into x, y,

00:52:51.040 --> 00:52:53.350
and a rule R goes
to ST which makes

00:52:53.350 --> 00:52:56.110
this work, if all
possible ways fail

00:52:56.110 --> 00:52:58.300
then you can't get from R to w.

00:53:02.230 --> 00:53:05.440
And then you can decide the
original A CFG problem now,

00:53:05.440 --> 00:53:07.000
by starting from
the start variable,

00:53:07.000 --> 00:53:09.175
instead of just
any old R. You plug

00:53:09.175 --> 00:53:11.320
in the start variable for R.

00:53:11.320 --> 00:53:19.680
So this algorithm works, and
it can be used to solve A CFG.

00:53:19.680 --> 00:53:22.370
But the question is,
is it polynomial?

00:53:22.370 --> 00:53:24.890
And it's not.

00:53:24.890 --> 00:53:30.020
Because every time you're
doing the recursion,

00:53:30.020 --> 00:53:33.960
you're essentially adding
another factor of n.

00:53:33.960 --> 00:53:36.720
Because here, as we pointed
out, this is a factor of n.

00:53:36.720 --> 00:53:40.020
But that's happening every time
you're doing a recursive level.

00:53:40.020 --> 00:53:43.980
And you can imagine, I'm just
doing a very crude analysis

00:53:43.980 --> 00:53:47.160
here, depending upon
how you divide w up.

00:53:47.160 --> 00:53:48.930
But roughly speaking,
it's going to get

00:53:48.930 --> 00:53:51.120
divided in half each
time, so there's

00:53:51.120 --> 00:53:53.220
going to be log n levels.

00:53:53.220 --> 00:53:56.010
So that means you're
going to be multiplying n

00:53:56.010 --> 00:53:58.620
by itself log n
times, or give you

00:53:58.620 --> 00:54:02.030
an n to the log n algorithm.

00:54:02.030 --> 00:54:04.310
That's not polynomial,
because polynomials

00:54:04.310 --> 00:54:07.130
end with a constant,
for some fixed constant.

00:54:07.130 --> 00:54:09.830
n to the log n is not
going to be polynomial.

00:54:09.830 --> 00:54:13.770
This is not a
polynomial algorithm.

00:54:13.770 --> 00:54:17.370
Instead, you're going to have to
do something just a little bit

00:54:17.370 --> 00:54:20.180
more clever.

00:54:20.180 --> 00:54:22.700
It's going to be
the same basic idea,

00:54:22.700 --> 00:54:26.270
but relying on one
little observation.

00:54:26.270 --> 00:54:32.410
And the little
observation is that when--

00:54:32.410 --> 00:54:35.500
this non-polynomial
implementation

00:54:35.500 --> 00:54:40.910
that I've just described
is actually pretty dumb.

00:54:40.910 --> 00:54:43.850
Because it's doing a lot
of recomputation of things

00:54:43.850 --> 00:54:45.380
that it's already solved.

00:54:45.380 --> 00:54:47.770
Why is that?

00:54:47.770 --> 00:54:51.900
Because if you
look at the number

00:54:51.900 --> 00:54:56.250
of possible different
subproblems here,

00:54:56.250 --> 00:55:02.070
once I give you G
and I give you w,

00:55:02.070 --> 00:55:08.700
how many different subproblems
of G, S, and T are there?

00:55:12.140 --> 00:55:15.950
The number of strings
here, all of those strings

00:55:15.950 --> 00:55:17.720
are going to be substrings of w.

00:55:17.720 --> 00:55:22.490
There's only roughly n
squared substrings of w.

00:55:22.490 --> 00:55:24.050
I'm always going
to be generating,

00:55:24.050 --> 00:55:27.140
in a subproblem here,
some substring of w

00:55:27.140 --> 00:55:31.360
from some starting
variable in the grammar.

00:55:31.360 --> 00:55:33.730
So because there aren't that
many different substrings,

00:55:33.730 --> 00:55:37.000
and not that many different
starting variables,

00:55:37.000 --> 00:55:39.640
the total number of
possible problems

00:55:39.640 --> 00:55:42.130
that this algorithm is going
to be called on to solve

00:55:42.130 --> 00:55:44.748
is going to be, in total,
a polynomial number

00:55:44.748 --> 00:55:45.790
of different subproblems.

00:55:45.790 --> 00:55:47.082
There aren't very many of them.

00:55:47.082 --> 00:55:49.840
It's only something
like order of n squared.

00:55:49.840 --> 00:55:54.040
So if the algorithm is running
for exponentially long time,

00:55:54.040 --> 00:55:57.900
it's solving the same
subproblem over and over again.

00:55:57.900 --> 00:56:00.910
That's dumb.

00:56:00.910 --> 00:56:04.000
Why don't you just remember
when you solve the subproblem,

00:56:04.000 --> 00:56:05.290
so you don't solve it again?

00:56:08.480 --> 00:56:12.560
Doing that enhanced
recursion where

00:56:12.560 --> 00:56:16.177
you remember the problems
you've already solved,

00:56:16.177 --> 00:56:17.510
it's called dynamic programming.

00:56:17.510 --> 00:56:22.408
I don't know why it has such
a confusing name like that.

00:56:22.408 --> 00:56:24.450
Actually it's called by
several different things.

00:56:24.450 --> 00:56:28.070
But anyway, that's known
as dynamic programming.

00:56:28.070 --> 00:56:32.400
It's recursion plus the memory.

00:56:32.400 --> 00:56:35.810
And here is just
repeating myself.

00:56:35.810 --> 00:56:39.920
There are not very many
different substrings,

00:56:39.920 --> 00:56:42.957
so every time in your
recursion somewhere,

00:56:42.957 --> 00:56:44.790
you're going to be
working with a substring.

00:56:44.790 --> 00:56:46.730
So there's not that many
different subproblems

00:56:46.730 --> 00:56:48.170
that you can possibly solve.

00:56:48.170 --> 00:56:51.560
And just remember when
you solve the subproblem,

00:56:51.560 --> 00:56:53.720
and not solve it again.

00:56:53.720 --> 00:56:57.470
Let me just show you that
algorithm again here,

00:56:57.470 --> 00:56:59.930
with the little modification.

00:56:59.930 --> 00:57:03.830
So first of all,
let me give you--

00:57:03.830 --> 00:57:05.960
this is the same algorithm
from the previous slide.

00:57:05.960 --> 00:57:08.210
I'm just repeating it here
without all the other stuff

00:57:08.210 --> 00:57:10.520
so we can look at it directly.

00:57:10.520 --> 00:57:15.260
Dividing it into x and y
for each possible rule.

00:57:15.260 --> 00:57:15.980
And then recurse.

00:57:18.680 --> 00:57:26.460
I'm going to add a little
step 0 beforehand, which

00:57:26.460 --> 00:57:32.050
says, if I have G, w, and R,
let me just check first if I've

00:57:32.050 --> 00:57:34.400
already solved that one before.

00:57:34.400 --> 00:57:37.660
So I have to keep track of
the ones I've already solved.

00:57:37.660 --> 00:57:39.160
That's not too bad,
because there's

00:57:39.160 --> 00:57:42.760
only order n squared possible
different ones that I

00:57:42.760 --> 00:57:44.413
could be called on to solve.

00:57:44.413 --> 00:57:46.330
So I'm just going to
have a little table where

00:57:46.330 --> 00:57:47.860
I'm going to remember those.

00:57:47.860 --> 00:57:51.260
And then every time I get a
new one, I get one to solve,

00:57:51.260 --> 00:57:51.760
I'll check.

00:57:51.760 --> 00:57:52.990
Is it on that table?

00:57:52.990 --> 00:57:54.620
And what's the answer?

00:57:54.620 --> 00:57:57.610
So I won't have to rerun those.

00:57:57.610 --> 00:58:01.720
I'm going to be basically
pruning that tree so

00:58:01.720 --> 00:58:03.950
that it has only a
polynomial number of leaves.

00:58:03.950 --> 00:58:05.710
And so the total
size of that tree

00:58:05.710 --> 00:58:09.770
now is going to be polynomial.

00:58:09.770 --> 00:58:12.128
And so that's going to yield
a polynomial running time.

00:58:12.128 --> 00:58:13.670
This, by the way,
I only learned this

00:58:13.670 --> 00:58:16.128
myself, I'm sure you guys all
know this who have taken this

00:58:16.128 --> 00:58:19.160
in the algorithms course,
has a special name called

00:58:19.160 --> 00:58:20.900
memoization.

00:58:20.900 --> 00:58:23.030
Not memorization.

00:58:23.030 --> 00:58:26.060
Came from the same root,
I think, but memoization,

00:58:26.060 --> 00:58:29.748
which is somehow remembering
the results of a computation

00:58:29.748 --> 00:58:31.040
so you don't have to repeat it.

00:58:34.833 --> 00:58:36.250
The total number
of calls is going

00:58:36.250 --> 00:58:39.250
to be, at most, n squared, to
this algorithm, because you're

00:58:39.250 --> 00:58:44.800
never going to be redoing
work that you've done already.

00:58:44.800 --> 00:58:49.135
And when you actually have to go
through it, the running time--

00:58:56.880 --> 00:58:59.010
the total amount of time
that you're going to need

00:58:59.010 --> 00:59:00.720
is going to be
polynomial altogether.

00:59:04.610 --> 00:59:08.340
I don't remember what
my check-in is on this.

00:59:08.340 --> 00:59:09.180
Oh yeah.

00:59:09.180 --> 00:59:10.500
This is somehow related.

00:59:10.500 --> 00:59:12.660
And feel free to
ask questions too,

00:59:12.660 --> 00:59:16.110
while you're thinking
about this check-in.

00:59:16.110 --> 00:59:19.170
But the check-in
says here, we've

00:59:19.170 --> 00:59:22.860
solved the A CFG problem
in polynomial time.

00:59:22.860 --> 00:59:28.820
Does that tell us that every
context-free language itself

00:59:28.820 --> 00:59:31.970
is also solvable
in polynomial time?

00:59:31.970 --> 00:59:37.238
Just mull that over, and
please give me an answer to it.

00:59:37.238 --> 00:59:38.780
I hope you do better
on this check-in

00:59:38.780 --> 00:59:41.630
than you did on the last one.

00:59:41.630 --> 00:59:45.760
But anyway, why don't you go
ahead and think about that.

00:59:45.760 --> 00:59:47.510
I can take some questions
in the meantime.

00:59:51.890 --> 00:59:53.065
Somebody is asking here--

00:59:56.020 --> 00:59:59.140
actually, I'm getting
several questions on this.

00:59:59.140 --> 01:00:02.620
Why isn't it order n
cubed or something greater

01:00:02.620 --> 01:00:06.860
than order n squared
because of the variables?

01:00:06.860 --> 01:00:10.150
The variables don't depend on n.

01:00:10.150 --> 01:00:18.260
When you're given-- well,
actually that's not true.

01:00:18.260 --> 01:00:19.630
No.

01:00:19.630 --> 01:00:20.840
You are right.

01:00:23.870 --> 01:00:28.180
Because the grammar
is part of the input.

01:00:28.180 --> 01:00:31.440
So you might have as many
as n different variables

01:00:31.440 --> 01:00:33.330
in the given grammar.

01:00:33.330 --> 01:00:34.980
So you are right.

01:00:34.980 --> 01:00:40.470
There is potentially--
the grammar might

01:00:40.470 --> 01:00:44.040
be half the size of the input,
and the input to the grammar w

01:00:44.040 --> 01:00:45.718
might be half the
size of the input.

01:00:45.718 --> 01:00:47.760
So I didn't think about
that, but you're correct.

01:00:47.760 --> 01:00:52.290
There are potentially
different numbers

01:00:52.290 --> 01:00:54.100
of variables in
different grammars,

01:00:54.100 --> 01:00:56.550
so you have to add an
extra factor, which

01:00:56.550 --> 01:00:59.010
would be at most the
size of the input,

01:00:59.010 --> 01:01:01.750
because that's as many variables
as you could possibly have.

01:01:01.750 --> 01:01:08.970
So it really should be, I think,
order n cubed to take that

01:01:08.970 --> 01:01:10.440
into account as well.

01:01:10.440 --> 01:01:12.420
Plus all of the work
that needs to happen

01:01:12.420 --> 01:01:14.478
in terms of dividing things up.

01:01:14.478 --> 01:01:16.020
On a one-tape Turing
machine, there's

01:01:16.020 --> 01:01:18.750
going to be some extra
work just to carry out

01:01:18.750 --> 01:01:22.380
some of these individual steps,
because with a single tape

01:01:22.380 --> 01:01:24.030
things are sometimes
a little awkward.

01:01:24.030 --> 01:01:25.200
I think the total
running time is

01:01:25.200 --> 01:01:26.783
going to end up being
something like n

01:01:26.783 --> 01:01:29.190
to the 4th or into the 5th
on a one-tape Turing machine.

01:01:32.670 --> 01:01:33.720
But that's a good point.

01:01:38.070 --> 01:01:41.250
Somebody's saying, how can we
be storing n squared strings

01:01:41.250 --> 01:01:42.570
in finite time?

01:01:42.570 --> 01:01:44.160
I'm not saying finite time.

01:01:44.160 --> 01:01:45.390
We have polynomial time.

01:01:45.390 --> 01:01:48.870
Every stage of this
algorithm is allowed

01:01:48.870 --> 01:01:50.430
to run for polynomially
many steps.

01:01:50.430 --> 01:01:52.260
As long as it's
clearly polynomial,

01:01:52.260 --> 01:01:54.195
we can just write that
down as a single stage.

01:01:58.910 --> 01:02:01.600
Part two should say--

01:02:01.600 --> 01:02:03.070
oh.

01:02:03.070 --> 01:02:03.920
There's a typo here.

01:02:03.920 --> 01:02:06.055
So use D. Thank you.

01:02:06.055 --> 01:02:07.150
That is a typo.

01:02:14.013 --> 01:02:16.180
I'm afraid if I change it
on my original slide here,

01:02:16.180 --> 01:02:18.640
things will break in
some horrible way.

01:02:18.640 --> 01:02:19.630
Let's just see.

01:02:19.630 --> 01:02:22.290
Did I completely wreck my slide?

01:02:22.290 --> 01:02:24.530
No, that's good.

01:02:24.530 --> 01:02:25.520
Yeah, thank you.

01:02:25.520 --> 01:02:26.120
Good point.

01:02:33.080 --> 01:02:35.100
Oops.

01:02:35.100 --> 01:02:38.840
OK, how's our check-in doing?

01:02:38.840 --> 01:02:42.620
I think you're just
about all done.

01:02:42.620 --> 01:02:44.030
Spent a lot of time on this.

01:02:44.030 --> 01:02:44.795
End polling.

01:02:51.460 --> 01:02:54.890
As you may remember from the
first half of the course--

01:02:54.890 --> 01:02:57.640
so the answer is A, indeed.

01:02:57.640 --> 01:03:01.930
Remember that we showed
A CFG is decidable,

01:03:01.930 --> 01:03:03.910
and therefore each
context-free language

01:03:03.910 --> 01:03:06.970
itself is decidable,
just because you

01:03:06.970 --> 01:03:11.440
can plug in a specific grammar
into the A CFG problem.

01:03:11.440 --> 01:03:15.370
The very same
reasoning works here.

01:03:15.370 --> 01:03:19.000
If you have a context-free
language, it has a grammar.

01:03:19.000 --> 01:03:22.240
You can plug that grammar
into the A CFG problem.

01:03:22.240 --> 01:03:24.130
And then, that's
polynomial time,

01:03:24.130 --> 01:03:25.900
you're going to get
a polynomial time

01:03:25.900 --> 01:03:30.700
algorithm for that language.

01:03:30.700 --> 01:03:32.665
Good to review that.

01:03:32.665 --> 01:03:35.290
It's the same thing, same
argument we used before.

01:03:42.327 --> 01:03:44.160
I don't want to spend
a lot of time on this.

01:03:44.160 --> 01:03:46.820
There's another way of looking
at dynamic programming.

01:03:46.820 --> 01:03:50.600
We'll talk about this again
maybe in a lecture, probably

01:03:50.600 --> 01:03:55.220
next lecture, just because I you
have a homework problem on it.

01:03:55.220 --> 01:03:57.358
If you've seen dynamic
programming before,

01:03:57.358 --> 01:03:58.400
this is going to be easy.

01:03:58.400 --> 01:04:00.400
If you haven't seen it
before, it's going to be,

01:04:00.400 --> 01:04:02.450
I think, probably a
little challenging.

01:04:06.340 --> 01:04:08.740
Another way of looking
at dynamic programming

01:04:08.740 --> 01:04:11.950
is the so-called bottom-up
version of dynamic programming.

01:04:11.950 --> 01:04:14.770
And what that would
mean is, you solve

01:04:14.770 --> 01:04:17.080
all of the subproblems first.

01:04:17.080 --> 01:04:19.540
You solve all the smaller
subproblems before you

01:04:19.540 --> 01:04:21.100
solve the larger subproblems.

01:04:25.420 --> 01:04:26.505
It's here on the slide.

01:04:26.505 --> 01:04:28.130
I'm not sure I want
to talk it through.

01:04:28.130 --> 01:04:33.880
But basically, you solve
the subproblems here

01:04:33.880 --> 01:04:39.700
where, start with
strings of length 1,

01:04:39.700 --> 01:04:42.370
and then from that you
build up to subproblems

01:04:42.370 --> 01:04:45.610
with the substrings are
of length 2, and then 3,

01:04:45.610 --> 01:04:46.520
and so on.

01:04:46.520 --> 01:04:50.500
And each of those only relies
on the smaller previously solved

01:04:50.500 --> 01:04:51.550
subproblems.

01:04:51.550 --> 01:04:54.850
So you can, kind of
in a systematic way,

01:04:54.850 --> 01:04:59.260
solve all the larger and
larger subproblems for larger

01:04:59.260 --> 01:05:02.410
and larger substrings.

01:05:02.410 --> 01:05:04.300
That gives kind of a
different perspective

01:05:04.300 --> 01:05:05.860
on dynamic programming.

01:05:05.860 --> 01:05:08.710
And for different
problems, sometimes it's

01:05:08.710 --> 01:05:11.920
better to think about either
this sort of top-down recursion

01:05:11.920 --> 01:05:16.570
based process, or the
bottom-up process that I'm

01:05:16.570 --> 01:05:18.400
describing here.

01:05:18.400 --> 01:05:20.290
They're really
completely equivalent.

01:05:23.600 --> 01:05:27.780
The version that's described
for this particular algorithm,

01:05:27.780 --> 01:05:29.600
which appears in the
textbook, is actually

01:05:29.600 --> 01:05:31.290
the bottom-up algorithm.

01:05:31.290 --> 01:05:33.990
So you shouldn't be confused
if you see something there

01:05:33.990 --> 01:05:35.760
which looks somewhat different.

01:05:38.300 --> 01:05:43.460
You basically solve all
possible subproblems,

01:05:43.460 --> 01:05:45.890
basically filling out a table.

01:05:45.890 --> 01:05:49.850
Let me not say anything
more about that here,

01:05:49.850 --> 01:05:51.890
since we're running a
little short on time.

01:05:54.505 --> 01:05:55.880
There are really
two perspectives

01:05:55.880 --> 01:05:59.230
on dynamic programming.

01:05:59.230 --> 01:06:04.210
So moving on from there,
let's shift gears.

01:06:06.900 --> 01:06:12.030
Leave context-free languages
and dynamic programming behind.

01:06:12.030 --> 01:06:18.530
And so I'm moving toward
understanding P and NP.

01:06:18.530 --> 01:06:23.840
And for that, we will
introduce a new problem called

01:06:23.840 --> 01:06:25.190
the satisfiability problem.

01:06:25.190 --> 01:06:29.780
And that's one we're going
to spend a lot of time on.

01:06:29.780 --> 01:06:34.040
If you tuned out a little bit
during the dynamic programming

01:06:34.040 --> 01:06:37.790
discussion, time to
get back on board.

01:06:41.990 --> 01:06:44.360
The satisfiability
problem is going

01:06:44.360 --> 01:06:50.650
to be a computational
problem that we're

01:06:50.650 --> 01:06:51.730
going to be working on.

01:06:51.730 --> 01:06:54.890
And it has to do
with Boolean formula.

01:06:54.890 --> 01:06:58.510
So these are expressions,
like arithmetical formula,

01:06:58.510 --> 01:07:02.980
like x plus y times
z, but instead

01:07:02.980 --> 01:07:07.210
of using numerical
variables, we're

01:07:07.210 --> 01:07:09.910
going to be using Boolean
variables that take

01:07:09.910 --> 01:07:11.860
on Boolean values, true, false.

01:07:11.860 --> 01:07:14.395
Or sometimes
represented by 1 and 0.

01:07:18.660 --> 01:07:20.670
The operators that
we're going to be using

01:07:20.670 --> 01:07:24.210
are going to be the and,
or, and negation operations.

01:07:24.210 --> 01:07:25.110
And, or, not.

01:07:28.590 --> 01:07:33.230
I'm going to say such a
formula, such a Boolean formula,

01:07:33.230 --> 01:07:34.820
we're going to call
it satisfiable--

01:07:34.820 --> 01:07:36.920
we'll do an example
in a second--

01:07:36.920 --> 01:07:42.440
if that formula value
evaluates to true

01:07:42.440 --> 01:07:47.820
if you make some assignment
of values to its variables.

01:07:47.820 --> 01:07:51.980
So just like arithmetical
formula will have some value

01:07:51.980 --> 01:07:55.970
if you plug in values
for the variables,

01:07:55.970 --> 01:07:58.190
Boolean formula is
going to have some value

01:07:58.190 --> 01:08:01.130
if you plug in Boolean
values for its variables.

01:08:01.130 --> 01:08:02.810
And I want to know,
is there some way

01:08:02.810 --> 01:08:05.480
to plug in values which makes
the whole thing evaluate

01:08:05.480 --> 01:08:07.070
to true.

01:08:07.070 --> 01:08:08.990
The formula itself
is going to evaluate

01:08:08.990 --> 01:08:12.380
to either true or false, and
I wanted to evaluate to true.

01:08:17.840 --> 01:08:19.069
Here is our example.

01:08:19.069 --> 01:08:24.939
Let's take the formula,
phi, which is x or y,

01:08:24.939 --> 01:08:27.760
and x complement--

01:08:27.760 --> 01:08:30.340
or, not x or not y.

01:08:30.340 --> 01:08:35.109
So the notation x with a
bar over it, x complement,

01:08:35.109 --> 01:08:39.939
is just x bar, not x.

01:08:39.939 --> 01:08:42.100
It's just the way
if you're familiar

01:08:42.100 --> 01:08:47.140
with the other notation,
the not operation, which

01:08:47.140 --> 01:08:48.949
just inverts 1s and 0s.

01:08:52.330 --> 01:08:55.149
We're going to write it with
a bar instead of the negation

01:08:55.149 --> 01:08:56.649
symbol.

01:08:56.649 --> 01:08:59.229
I'm assuming that you've
all seen Boolean algebra,

01:08:59.229 --> 01:09:04.660
Boolean arithmetic before,
where the and operation is only

01:09:04.660 --> 01:09:07.660
true if both inputs are true.

01:09:07.660 --> 01:09:10.060
These are going to be binary
and operations and binary

01:09:10.060 --> 01:09:12.160
or operations.

01:09:12.160 --> 01:09:15.010
Or is going to be true
if either input is true.

01:09:15.010 --> 01:09:19.260
And not is true if its
single input is false.

01:09:19.260 --> 01:09:22.380
Oops, just looked at the answer.

01:09:22.380 --> 01:09:25.890
Here I want to know, for
this Boolean formula,

01:09:25.890 --> 01:09:28.410
here is it satisfiable.

01:09:28.410 --> 01:09:32.710
Is there some way to assign
values to the variables

01:09:32.710 --> 01:09:37.479
to make this formula
evaluate to true?

01:09:37.479 --> 01:09:39.450
So for example, let's
just try things.

01:09:39.450 --> 01:09:42.100
Let's make x and y both true.

01:09:42.100 --> 01:09:44.800
So x is true and y is true.

01:09:44.800 --> 01:09:47.910
So x or y, well that's
good, that's true.

01:09:47.910 --> 01:09:52.229
But now we have to do an and, so
we need both sides to be true.

01:09:52.229 --> 01:09:55.900
So now we have x complement--
well we said we said x is true,

01:09:55.900 --> 01:09:58.830
so x complement is false,
y complement is false.

01:09:58.830 --> 01:10:01.270
False or false is false.

01:10:01.270 --> 01:10:03.060
So now we have a true and false.

01:10:03.060 --> 01:10:04.780
That's going to be false.

01:10:04.780 --> 01:10:06.660
We did not find a
satisfying assignment.

01:10:06.660 --> 01:10:08.020
But maybe there's another one.

01:10:08.020 --> 01:10:09.270
And in fact, there is.

01:10:09.270 --> 01:10:13.530
If you make x true and y
false, then both of these parts

01:10:13.530 --> 01:10:16.920
will evaluate to true, and
then you'll have true and true.

01:10:16.920 --> 01:10:19.380
So we found a satisfying
assignment to this formula.

01:10:19.380 --> 01:10:22.570
It is, in fact, satisfiable.

01:10:22.570 --> 01:10:25.470
So if you say x is
1 and y is 0, yes.

01:10:25.470 --> 01:10:28.530
This is satisfiable.

01:10:28.530 --> 01:10:31.010
Now the problem of testing
for a Boolean formula,

01:10:31.010 --> 01:10:34.910
if it is satisfiable, is
going to be the SAT language.

01:10:34.910 --> 01:10:35.550
It's a set.

01:10:35.550 --> 01:10:39.660
It's a collection of
satisfiable Boolean formula.

01:10:39.660 --> 01:10:41.940
And testing whether
you're in SAT or not

01:10:41.940 --> 01:10:46.380
is going to be an important
computational problem.

01:10:49.580 --> 01:10:56.990
There was an amazing
theorem which

01:10:56.990 --> 01:10:59.870
really got this
whole subject going,

01:10:59.870 --> 01:11:05.720
discovered independently by
Steve Cook in North America

01:11:05.720 --> 01:11:10.700
and Leonid Levin in the former
Soviet Union, almost exactly at

01:11:10.700 --> 01:11:14.690
the same time, which made a
connection between this one

01:11:14.690 --> 01:11:18.860
problem and all of
the problems in NP.

01:11:18.860 --> 01:11:22.580
By solving this one
satisfiability problem

01:11:22.580 --> 01:11:28.220
in polynomial
time, it allows you

01:11:28.220 --> 01:11:33.010
to solve all of the problems
in NP in polynomial time.

01:11:33.010 --> 01:11:36.240
So if you could solve
this problem set in P,

01:11:36.240 --> 01:11:41.800
then Hamiltonian path
is also solvable in P.

01:11:41.800 --> 01:11:44.300
If you step back and think about
that, it's kind of amazing.

01:11:46.820 --> 01:11:49.010
And the method that
we're going to introduce

01:11:49.010 --> 01:11:52.940
is called polynomial
time reducibility.

01:11:52.940 --> 01:11:54.590
Let's do a quick
check-in on this.

01:11:54.590 --> 01:11:57.620
This should be an easy one.

01:11:57.620 --> 01:12:00.530
Why don't you just think
about, is SAT, the SAT problem

01:12:00.530 --> 01:12:02.420
that we just described
here, is that in NP?

01:12:06.857 --> 01:12:07.440
Three seconds.

01:12:10.860 --> 01:12:12.570
You all there?

01:12:12.570 --> 01:12:13.500
OK.

01:12:13.500 --> 01:12:14.220
Ending polling.

01:12:21.240 --> 01:12:24.240
Hopefully you're getting
the intuition for NP

01:12:24.240 --> 01:12:28.320
that these are the
problems-- to be in NP means

01:12:28.320 --> 01:12:30.810
that when you're a
member of the language,

01:12:30.810 --> 01:12:34.320
there's a short proof or a
short certificate of membership.

01:12:34.320 --> 01:12:36.240
And in this case,
the short certificate

01:12:36.240 --> 01:12:39.340
that the formula is
satisfiable is the assignment,

01:12:39.340 --> 01:12:43.890
which makes it true, also called
the satisfying assignment.

01:12:43.890 --> 01:12:47.135
So yes, this is an NP language,
language that's in NP.

01:12:51.550 --> 01:12:54.050
There are a lot of things that
we don't know in the subject,

01:12:54.050 --> 01:12:56.090
but this isn't one of them.

01:12:56.090 --> 01:12:59.060
We do know that SAT is in NP.

01:12:59.060 --> 01:13:02.570
So let's talk about
our method for showing

01:13:02.570 --> 01:13:12.150
this remarkable
fact that, if you

01:13:12.150 --> 01:13:15.750
can solve SAT in polynomial
time, then all of NP

01:13:15.750 --> 01:13:17.430
is solvable in polynomial time.

01:13:17.430 --> 01:13:21.450
And it uses this notion of
polynomial time reducibility,

01:13:21.450 --> 01:13:23.940
which is just like
mapping reducibility

01:13:23.940 --> 01:13:26.880
that I hope you've all
grown to know and love

01:13:26.880 --> 01:13:28.620
in the first half of the course.

01:13:28.620 --> 01:13:33.000
But now, the reduction has to
operate in polynomial time.

01:13:33.000 --> 01:13:38.130
So it's the same picture that
we had before, mapping A to B,

01:13:38.130 --> 01:13:40.290
transforming A questions
to B questions.

01:13:40.290 --> 01:13:44.200
But now the transformation
has to operate quickly.

01:13:44.200 --> 01:13:49.530
And we get that if A is
polynomial time reducible to B,

01:13:49.530 --> 01:13:54.000
and B is polynomial time, then
A is also polynomial time.

01:13:54.000 --> 01:13:55.740
Same pattern as before.

01:13:55.740 --> 01:14:01.320
If A is reducible to B and
B is easy, then A is easy.

01:14:01.320 --> 01:14:06.240
Here is kind of the
essence of the idea,

01:14:06.240 --> 01:14:11.280
or at least the outline of the
idea of this Cook and Levin

01:14:11.280 --> 01:14:12.240
theorem.

01:14:12.240 --> 01:14:15.720
That if satisfiable is in
P, then everything in NP

01:14:15.720 --> 01:14:18.480
can be done in P. Which
is because, we will

01:14:18.480 --> 01:14:23.250
show that all problems
in NP are polynomial time

01:14:23.250 --> 01:14:26.640
reducible to SAT.

01:14:26.640 --> 01:14:28.440
That's the amazing fact.

01:14:28.440 --> 01:14:31.830
So therefore, if you
can bring SAT down

01:14:31.830 --> 01:14:35.700
into P by using this reduction,
it brings everything else along

01:14:35.700 --> 01:14:38.670
with it, everything
is reducible to SAT.

01:14:38.670 --> 01:14:42.090
So we just have to
show how to do that.

01:14:42.090 --> 01:14:44.280
There is an analogy that
we had in the first half

01:14:44.280 --> 01:14:46.238
of the course, in one of
our homework problems,

01:14:46.238 --> 01:14:47.520
if you may remember.

01:14:47.520 --> 01:14:53.640
We showed that A TM has
the very special property

01:14:53.640 --> 01:14:56.580
that all Turing
recognizable languages are

01:14:56.580 --> 01:14:59.390
mapping reducible to it.

01:14:59.390 --> 01:15:07.660
I think that was problem 2,
or 2a, either in problem set 3

01:15:07.660 --> 01:15:09.670
or problem set 2.

01:15:09.670 --> 01:15:12.340
I think problem set 3.

01:15:12.340 --> 01:15:15.070
That every Turing
recognizable language

01:15:15.070 --> 01:15:17.590
is polynomial time
reducible to A TM.

01:15:17.590 --> 01:15:20.440
And so, very similar picture.

01:15:20.440 --> 01:15:22.480
And there's a lot
of analogies here

01:15:22.480 --> 01:15:26.920
that you can draw between
the computability section

01:15:26.920 --> 01:15:29.770
and the complexity section.

01:15:29.770 --> 01:15:32.330
With that, I know we're
just about out of time.

01:15:32.330 --> 01:15:38.740
So let's just quick review
of what we've done here.

01:15:38.740 --> 01:15:42.560
I will stick around for
questions for a while.

01:15:42.560 --> 01:15:45.495
Is there a--

01:15:45.495 --> 01:15:46.620
OK, that's a good question.

01:15:46.620 --> 01:15:52.500
Is there a regular
reduction analogy version

01:15:52.500 --> 01:15:54.780
to mapping reducibility?

01:15:54.780 --> 01:16:00.550
We had the general reduction
for the computability section.

01:16:00.550 --> 01:16:03.490
And we had the mapping reduction
for the computability section.

01:16:03.490 --> 01:16:06.910
Here, we're only going
to be focusing now

01:16:06.910 --> 01:16:08.830
on the mapping reduction.

01:16:08.830 --> 01:16:12.220
So polynomial time
reduction is, by assumption,

01:16:12.220 --> 01:16:13.660
going to be a mapping reduction.

01:16:13.660 --> 01:16:18.460
Yes, there is a general
polynomial time reduction

01:16:18.460 --> 01:16:19.225
notion as well.

01:16:22.450 --> 01:16:24.650
This is not required,
but if you are

01:16:24.650 --> 01:16:27.530
curious about the
general reduction

01:16:27.530 --> 01:16:29.570
and how to precisely
formulate that,

01:16:29.570 --> 01:16:34.850
it actually appears in chapter
6 under Turing reducibility.

01:16:34.850 --> 01:16:37.520
That's the general notion
of reducibility spelled out

01:16:37.520 --> 01:16:38.570
in a formal way.

01:16:38.570 --> 01:16:42.180
And there's polynomial time
Turing reducibility as well.

01:16:42.180 --> 01:16:45.365
We're not going to talk
about it in this course.

01:16:49.440 --> 01:16:52.280
Other questions?

01:16:52.280 --> 01:16:56.720
Does NP correspond
exactly to verification

01:16:56.720 --> 01:16:59.900
in polynomial time?

01:16:59.900 --> 01:17:02.120
For me to answer that
as a precise question,

01:17:02.120 --> 01:17:05.180
we have to have a precise
definition of verification.

01:17:05.180 --> 01:17:11.420
But with the right
definition, the answer is yes.

01:17:11.420 --> 01:17:17.690
So you can define a verifier
as a polynomial time algorithm

01:17:17.690 --> 01:17:20.870
that gives a
certificate, that takes

01:17:20.870 --> 01:17:23.150
a certificate and an
input to the language,

01:17:23.150 --> 01:17:30.890
and will accept if
that certificate is

01:17:30.890 --> 01:17:33.710
a valid certificate
for that input.

01:17:33.710 --> 01:17:36.050
This is actually
discussed in chapter,

01:17:36.050 --> 01:17:40.125
I think, 9 of the text.

01:17:47.950 --> 01:17:50.230
Now I'm forgetting already,
what's where in the book.

01:17:50.230 --> 01:17:54.400
But yeah, you can think of
NP in terms of verification

01:17:54.400 --> 01:17:55.540
as the definition.

01:17:59.840 --> 01:18:04.550
Is proving P equal NP the same
as proving that a polynom--

01:18:04.550 --> 01:18:06.500
actually, I can even make the--

01:18:06.500 --> 01:18:11.830
if you want, you can
post public comments too.

01:18:11.830 --> 01:18:14.140
I should have done
that in other cases.

01:18:14.140 --> 01:18:16.750
Is proving P equal NP
the same as proving

01:18:16.750 --> 01:18:19.100
that a polynomial
time non-deterministic

01:18:19.100 --> 01:18:23.410
Turing machine N has a
polynomial time deterministic?

01:18:26.115 --> 01:18:26.615
Yeah.

01:18:29.660 --> 01:18:36.430
Suppose we prove that P equals
NP, which is the minority view,

01:18:36.430 --> 01:18:40.695
I would say, the
small minority view.

01:18:40.695 --> 01:18:42.070
There are some
people who believe

01:18:42.070 --> 01:18:46.960
that that is entirely possible,
and might even be the case.

01:18:46.960 --> 01:18:50.020
But that's a very small group.

01:18:50.020 --> 01:18:52.720
But yeah, if you
prove P equal NP,

01:18:52.720 --> 01:18:55.960
that's the same as saying
that every non-deterministic

01:18:55.960 --> 01:18:58.630
polynomial time Turing
machine is going

01:18:58.630 --> 01:19:02.350
to have a companion
deterministic polynomial

01:19:02.350 --> 01:19:05.020
time Turing machine which
does the same language.

01:19:05.020 --> 01:19:07.530
That's exactly what it means.

01:19:07.530 --> 01:19:09.890
Bye bye, everybody.