1
00:00:00,000 --> 00:00:01,950
[CLICKING]

2
00:00:01,950 --> 00:00:02,450
[RUSTLING]

3
00:00:02,450 --> 00:00:04,410
[SQUEAKING]

4
00:00:04,410 --> 00:00:06,370
[CLICKING]

5
00:00:25,030 --> 00:00:26,830
MICHAEL SIPSER: OK.

6
00:00:26,830 --> 00:00:28,130
Hi, everybody.

7
00:00:28,130 --> 00:00:28,915
Let's get started.

8
00:00:31,910 --> 00:00:38,050
So, it's been a while since
we came together in a lecture.

9
00:00:38,050 --> 00:00:39,730
Last week, we had the holiday.

10
00:00:39,730 --> 00:00:42,220
We had the midterm.

11
00:00:42,220 --> 00:00:47,320
So with that, what
have we been doing?

12
00:00:47,320 --> 00:00:52,540
We finished the first half of
the course about two weeks ago,

13
00:00:52,540 --> 00:00:54,640
where we talked
about-- we were talking

14
00:00:54,640 --> 00:00:56,410
about computability theory.

15
00:00:56,410 --> 00:00:58,510
We have shifted into
the second half,

16
00:00:58,510 --> 00:01:01,040
talking about complexity theory.

17
00:01:01,040 --> 00:01:04,870
So get your mind back to that.

18
00:01:04,870 --> 00:01:12,670
We discussed the various
different models and ways

19
00:01:12,670 --> 00:01:16,570
of measuring complexity
on different models--

20
00:01:16,570 --> 00:01:19,000
at least in terms of the
amount of time that's used.

21
00:01:19,000 --> 00:01:22,100
And in the end, we settled on
the one-tape Turing machine,

22
00:01:22,100 --> 00:01:23,920
which is the same model
we had been working

23
00:01:23,920 --> 00:01:25,810
with in the first
half of the course,

24
00:01:25,810 --> 00:01:31,450
and argued that though
the measures of complexity

25
00:01:31,450 --> 00:01:33,670
are going to differ somewhat
from model to model,

26
00:01:33,670 --> 00:01:38,050
they're not going to differ by
more than a polynomial amount.

27
00:01:38,050 --> 00:01:40,420
And so, since the
kinds of questions

28
00:01:40,420 --> 00:01:42,760
we're going to be
asking are going

29
00:01:42,760 --> 00:01:47,200
to be, basically, whether
problems are polynomial or not,

30
00:01:47,200 --> 00:01:49,630
it's not going to really
matter which model we

31
00:01:49,630 --> 00:01:52,690
pick among reasonable
deterministic models.

32
00:01:52,690 --> 00:01:57,910
And so, the one-tape Turing
machine is a reasonable choice.

33
00:01:57,910 --> 00:02:00,930
Given that, we defined
time complexity classes,

34
00:02:00,930 --> 00:02:02,790
the TIME[T(n)] classes.

35
00:02:02,790 --> 00:02:07,680
We defined the class P,
which was invariant among all

36
00:02:07,680 --> 00:02:10,470
of those different deterministic
models in the sense

37
00:02:10,470 --> 00:02:12,608
that it didn't matter
which model we choose,

38
00:02:12,608 --> 00:02:14,400
we were going to end
up with the same class

39
00:02:14,400 --> 00:02:18,090
P. So that argues
for its naturalness.

40
00:02:18,090 --> 00:02:21,150
And we gave an example of
this path problem being in P.

41
00:02:21,150 --> 00:02:24,060
And we kind of ended that
lecture before the midterm

42
00:02:24,060 --> 00:02:26,920
with the discussion of this
Hamiltonian path problem.

43
00:02:26,920 --> 00:02:29,250
So we're going to come
back to that today.

44
00:02:29,250 --> 00:02:31,500
So today, we're going to
look at non-non-deterministic

45
00:02:31,500 --> 00:02:36,840
complexity; define the
classes' non-deterministic time

46
00:02:36,840 --> 00:02:42,160
or NTIME; talk about the class
NP, the P versus NP problem--

47
00:02:42,160 --> 00:02:46,230
which one of the very famous
unsolved problems in our field;

48
00:02:46,230 --> 00:02:50,445
and look at dynamic programming,
one of the most basic algorithm

49
00:02:50,445 --> 00:02:53,420
- polynomial-time algorithms and
polynomial-time reproducibility

50
00:02:53,420 --> 00:02:57,310
- moving toward our
discussion of NP completeness,

51
00:02:57,310 --> 00:03:01,080
which we will
begin next lecture.

52
00:03:01,080 --> 00:03:06,380
So with that, let's move
into today's content, which

53
00:03:06,380 --> 00:03:10,280
is, well, just a quick review.

54
00:03:10,280 --> 00:03:14,420
As I mentioned, we defined
the time complexity class.

55
00:03:14,420 --> 00:03:16,790
The time complexity
class is a collection

56
00:03:16,790 --> 00:03:26,960
of all of the
languages that you can

57
00:03:26,960 --> 00:03:31,040
solve in a certain time
bound, within a certain amount

58
00:03:31,040 --> 00:03:31,950
of time.

59
00:03:31,950 --> 00:03:36,290
So the time n-squared,
for example,

60
00:03:36,290 --> 00:03:39,950
is all of the languages
or all of the problems

61
00:03:39,950 --> 00:03:42,170
that you can solve
in n-squared time.

62
00:03:42,170 --> 00:03:45,260
We're identifying problems
with languages here.

63
00:03:45,260 --> 00:03:47,990
And the class P
is the collection

64
00:03:47,990 --> 00:03:49,550
of all problems
that you can solve

65
00:03:49,550 --> 00:03:52,440
or all languages that you
can solve in polynomial time.

66
00:03:52,440 --> 00:03:55,370
So it's the union over
all time n to the k--

67
00:03:55,370 --> 00:04:00,380
so n-squared, n-cubed, n to
the fifth power, and so on.

68
00:04:00,380 --> 00:04:04,630
Union out of all
of those bounds.

69
00:04:04,630 --> 00:04:08,780
The associated languages,
that's the class P.

70
00:04:08,780 --> 00:04:11,960
And we gave an example,
this path problem.

71
00:04:11,960 --> 00:04:14,690
We gave an algorithm for path.

72
00:04:14,690 --> 00:04:18,779
And then, we introduced this
Hamiltonian path problem.

73
00:04:18,779 --> 00:04:22,820
So if you remember?

74
00:04:22,820 --> 00:04:25,850
Instead of just asking
given a graph, whether you

75
00:04:25,850 --> 00:04:30,380
can get from s to t, now we want
to know can I get from s to t,

76
00:04:30,380 --> 00:04:34,260
but visit every other
node along the way.

77
00:04:34,260 --> 00:04:38,720
So find a path that goes
through everything else

78
00:04:38,720 --> 00:04:40,130
and gets from s to t.

79
00:04:42,780 --> 00:04:46,450
And I should say also
it's a simple path,

80
00:04:46,450 --> 00:04:51,330
so you're only allowed to go
through every node just once.

81
00:04:51,330 --> 00:04:59,320
And now, the question
for this problem

82
00:04:59,320 --> 00:05:04,240
is can we solve that
problem in polynomial time.

83
00:05:04,240 --> 00:05:06,790
Can we somehow modify
the algorithm for path

84
00:05:06,790 --> 00:05:09,850
to give us an algorithm
that solves Hamiltonian path

85
00:05:09,850 --> 00:05:11,200
in polynomial time?

86
00:05:11,200 --> 00:05:14,440
Of course, we could
solve Hamiltonian path

87
00:05:14,440 --> 00:05:18,580
with an exponential algorithm
by trying all possible paths.

88
00:05:18,580 --> 00:05:21,280
And that will give
a correct algorithm,

89
00:05:21,280 --> 00:05:25,360
but there are exponentially many
different paths and trying them

90
00:05:25,360 --> 00:05:28,190
all will not give a
polynomial time algorithm.

91
00:05:28,190 --> 00:05:29,980
So the interesting
problem is can we

92
00:05:29,980 --> 00:05:33,070
solve that without doing
that brute force searching

93
00:05:33,070 --> 00:05:36,070
through all possible paths.

94
00:05:36,070 --> 00:05:41,390
And that's a problem that
no one knows the answer to.

95
00:05:41,390 --> 00:05:43,130
Despite lots of
effort, people have not

96
00:05:43,130 --> 00:05:45,450
succeeded in finding
an algorithm for that.

97
00:05:45,450 --> 00:05:48,560
But on the other hand,
we don't have any idea

98
00:05:48,560 --> 00:05:51,742
how do you prove there
is no such algorithm.

99
00:05:51,742 --> 00:05:54,200
I mean, it's conceivable that
one could prove such a thing,

100
00:05:54,200 --> 00:05:57,390
but we just don't
know how to do it.

101
00:05:57,390 --> 00:06:03,980
And so, that problem is
an unsolved problem and I

102
00:06:03,980 --> 00:06:07,490
just-- this isn't
really a note to myself.

103
00:06:07,490 --> 00:06:09,410
What's kind of
amazing, and this is

104
00:06:09,410 --> 00:06:11,990
what we're going to be showing
over the next few lectures,

105
00:06:11,990 --> 00:06:15,890
that there would be very
surprising consequences if you

106
00:06:15,890 --> 00:06:18,980
could find a way to
solve Hamiltonian path

107
00:06:18,980 --> 00:06:20,950
in polynomial time.

108
00:06:20,950 --> 00:06:24,220
Because what that
would immediately yield

109
00:06:24,220 --> 00:06:28,210
is the polynomial time way of,
say, factoring large numbers

110
00:06:28,210 --> 00:06:31,210
or solving a large
number of other problems

111
00:06:31,210 --> 00:06:35,080
that we don't know how to
solve in polynomial time.

112
00:06:35,080 --> 00:06:38,980
So as we mentioned,
factoring is a problem

113
00:06:38,980 --> 00:06:41,170
that we only know at
present how to solve

114
00:06:41,170 --> 00:06:43,600
with an exponential algorithm.

115
00:06:43,600 --> 00:06:45,100
And it doesn't seem
to have anything

116
00:06:45,100 --> 00:06:47,920
to do with the
Hamiltonian path problem.

117
00:06:47,920 --> 00:06:49,210
Seem very different.

118
00:06:49,210 --> 00:06:52,180
But, yet, if you can
solve Hamiltonian path

119
00:06:52,180 --> 00:06:54,580
in polynomial time, then
you can factor numbers

120
00:06:54,580 --> 00:06:56,000
in polynomial time.

121
00:06:56,000 --> 00:06:59,800
And so, we'll see how
to make that connection.

122
00:06:59,800 --> 00:07:01,630
That's what we're
building toward

123
00:07:01,630 --> 00:07:05,510
with the next few lectures.

124
00:07:05,510 --> 00:07:07,260
OK.

125
00:07:07,260 --> 00:07:11,450
So happy to take any comments
and questions on that,

126
00:07:11,450 --> 00:07:15,080
or we'll just move on,
if you have any questions

127
00:07:15,080 --> 00:07:18,330
on our little review.

128
00:07:18,330 --> 00:07:20,280
Well, send questions
along and we

129
00:07:20,280 --> 00:07:25,230
can stop at the end of various
slides to try to answer them.

130
00:07:25,230 --> 00:07:27,570
And, of course,
write to the TAs,

131
00:07:27,570 --> 00:07:32,550
who can take your questions
while I'm lecturing.

132
00:07:32,550 --> 00:07:34,410
OK.

133
00:07:34,410 --> 00:07:36,180
So to start this
off, we're going

134
00:07:36,180 --> 00:07:39,990
to have to talk about
non-deterministic complexity

135
00:07:39,990 --> 00:07:43,930
as a variation of
deterministic complexity.

136
00:07:43,930 --> 00:07:48,690
So first of all,
all of the machines

137
00:07:48,690 --> 00:07:51,480
in this part of the course
and the languages, everything

138
00:07:51,480 --> 00:07:53,610
is going to be decidable
and all the machines

139
00:07:53,610 --> 00:07:54,670
are going to be deciders.

140
00:07:54,670 --> 00:07:57,128
So what do we mean when we have
a non-deterministic machine

141
00:07:57,128 --> 00:07:59,070
which is a decider?

142
00:07:59,070 --> 00:08:03,190
And that just simply means
that all of the branches--

143
00:08:03,190 --> 00:08:05,860
it's not just the machine
halts on every input,

144
00:08:05,860 --> 00:08:09,490
but all of the branches
halt on every input.

145
00:08:09,490 --> 00:08:11,740
So the non-deterministic
machine is non-deterministic,

146
00:08:11,740 --> 00:08:13,620
it has lots of
possible branches.

147
00:08:13,620 --> 00:08:15,780
They all have to halt--

148
00:08:15,780 --> 00:08:17,730
all of them-- on every input.

149
00:08:17,730 --> 00:08:19,660
That's what makes a
non-deterministic machine

150
00:08:19,660 --> 00:08:20,760
a decider.

151
00:08:20,760 --> 00:08:24,090
And you're going to convert
a non-deterministic decider

152
00:08:24,090 --> 00:08:26,490
into a deterministic decider.

153
00:08:26,490 --> 00:08:29,640
But the question is, how much
time would that introduce?

154
00:08:29,640 --> 00:08:31,800
How much extra time
is that going to cost?

155
00:08:31,800 --> 00:08:33,240
And the only way
that people know

156
00:08:33,240 --> 00:08:36,960
at the present time
for that conversion

157
00:08:36,960 --> 00:08:41,220
would be to do an
exponential increase.

158
00:08:41,220 --> 00:08:43,350
Basically, to try all
possible branches.

159
00:08:43,350 --> 00:08:46,880
And that's, of
course, very slow.

160
00:08:46,880 --> 00:08:52,040
So first, let's
understand what we

161
00:08:52,040 --> 00:08:55,970
mean by the time used by a
non-deterministic machine.

162
00:08:55,970 --> 00:08:59,360
And what we mean
by the time used

163
00:08:59,360 --> 00:09:02,000
is, we're looking at
each individual branch

164
00:09:02,000 --> 00:09:05,760
individually, separately.

165
00:09:05,760 --> 00:09:09,140
So a non-deterministic
machine, we'll say,

166
00:09:09,140 --> 00:09:13,100
runs in a certain amount of
time if all of the branches

167
00:09:13,100 --> 00:09:17,110
halt within that amount of time.

168
00:09:17,110 --> 00:09:23,640
So what we do not mean that
the total amount of usage,

169
00:09:23,640 --> 00:09:27,150
the total amount of effort
by adding up all the branches

170
00:09:27,150 --> 00:09:28,950
is at most T of n.

171
00:09:28,950 --> 00:09:32,430
It's just that each branch
individually uses at most

172
00:09:32,430 --> 00:09:33,480
T of n.

173
00:09:33,480 --> 00:09:35,380
That's just going to
be our definition.

174
00:09:35,380 --> 00:09:38,370
And it's going to turn
out to be the right way

175
00:09:38,370 --> 00:09:40,290
to look at this to
get something useful.

176
00:09:46,180 --> 00:09:57,720
So now we're going to define
the analogous complexity

177
00:09:57,720 --> 00:10:01,950
class associated to
non-deterministic computation,

178
00:10:01,950 --> 00:10:04,680
which we'll call
non-deterministic time.

179
00:10:04,680 --> 00:10:10,460
So non-deterministic time T of
n is the set of all languages

180
00:10:10,460 --> 00:10:13,700
that you can do with a
non-deterministic machine that

181
00:10:13,700 --> 00:10:17,030
runs in order T of n time.

182
00:10:17,030 --> 00:10:19,820
Just think back
to the definition

183
00:10:19,820 --> 00:10:24,530
we had for deterministic
complexity, the time class--

184
00:10:24,530 --> 00:10:26,240
or sometimes people
call it dtime

185
00:10:26,240 --> 00:10:27,600
to emphasize the difference.

186
00:10:27,600 --> 00:10:30,230
But let's just say we're
calling it in this course

187
00:10:30,230 --> 00:10:32,190
time versus ntime.

188
00:10:32,190 --> 00:10:35,850
So TIME[T(n)] is all of the
language that you can do with

189
00:10:35,850 --> 00:10:39,800
the one-tape Turing machine
that's deterministic.

190
00:10:39,800 --> 00:10:42,020
But this here is a
non-deterministic Turing

191
00:10:42,020 --> 00:10:46,470
machine for
non-deterministic time.

192
00:10:46,470 --> 00:10:50,360
So the picture that is
good to have in your head

193
00:10:50,360 --> 00:10:55,790
here would be if you
think of non-determinism

194
00:10:55,790 --> 00:10:58,880
in terms of a computation
tree thinking of all

195
00:10:58,880 --> 00:11:02,780
the different branches
of the non-determinism.

196
00:11:02,780 --> 00:11:05,540
All of those
branches have to halt

197
00:11:05,540 --> 00:11:09,220
and they have to halt
within the time bound.

198
00:11:09,220 --> 00:11:11,790
So imagine, here,
this is T of n time.

199
00:11:11,790 --> 00:11:15,750
All of the branches have to halt
within T of n steps for this,

200
00:11:15,750 --> 00:11:18,720
a non-deterministic Turing
machine to be running in T of n

201
00:11:18,720 --> 00:11:22,620
time and to be doing a language
in the NTIME[T(n)] class.

202
00:11:25,150 --> 00:11:28,000
And by analogy with
what we did before,

203
00:11:28,000 --> 00:11:37,150
the class NP is the
collection of all languages

204
00:11:37,150 --> 00:11:39,310
that you can do
non-deterministically

205
00:11:39,310 --> 00:11:42,700
in polynomial time.

206
00:11:42,700 --> 00:11:47,520
So it's the union over all
of the ntime classes where

207
00:11:47,520 --> 00:11:48,870
the bound is polynomial.

208
00:11:52,160 --> 00:11:55,880
OK, so a lot of this
should look very familiar,

209
00:11:55,880 --> 00:11:59,360
but we've just added a
bunch of non-deterministic

210
00:11:59,360 --> 00:12:02,300
and a bunch of Ns in place.

211
00:12:02,300 --> 00:12:06,170
But the definitions
are very similar.

212
00:12:06,170 --> 00:12:10,970
And one of the motivations we
had for looking at the class P

213
00:12:10,970 --> 00:12:15,260
was that it did not depend
on the choice of model,

214
00:12:15,260 --> 00:12:18,110
as long as the model was
deterministic and reasonable.

215
00:12:18,110 --> 00:12:22,040
And the class NP is
also going to not

216
00:12:22,040 --> 00:12:24,320
depend on the
choice of model, as

217
00:12:24,320 --> 00:12:28,970
long as it's a reasonable
non-deterministic model.

218
00:12:28,970 --> 00:12:32,780
So it's again a
very natural class

219
00:12:32,780 --> 00:12:35,150
to look at from a
mathematical standpoint.

220
00:12:35,150 --> 00:12:39,680
And it also captures something
interesting, kind of,

221
00:12:39,680 --> 00:12:41,210
from a practical
standpoint - which

222
00:12:41,210 --> 00:12:43,700
we're going to talk about
over the next couple of slides

223
00:12:43,700 --> 00:12:47,330
- which is that it captures the
problems where you can easily

224
00:12:47,330 --> 00:12:53,110
verify when you're a
member of the language.

225
00:12:53,110 --> 00:12:54,370
OK, so we'll talk about that.

226
00:12:54,370 --> 00:12:59,710
But if you take, for example,
the Hamiltonian path problem.

227
00:12:59,710 --> 00:13:02,840
When you find a member
of the language,

228
00:13:02,840 --> 00:13:06,760
so that is a graph that
does have a Hamiltonian

229
00:13:06,760 --> 00:13:10,660
path from s to t,
you can easily verify

230
00:13:10,660 --> 00:13:14,460
that's true by simply
exhibiting the path.

231
00:13:14,460 --> 00:13:18,360
Not all problems can be
verified in that way.

232
00:13:18,360 --> 00:13:22,680
But the problems that are in
NP have that special feature--

233
00:13:22,680 --> 00:13:25,140
that when you have a
member of the language,

234
00:13:25,140 --> 00:13:27,517
there's a way to verify
that you're a member.

235
00:13:27,517 --> 00:13:29,850
So we're going to talk about
that, because that's really

236
00:13:29,850 --> 00:13:32,820
the key to understanding NP--

237
00:13:32,820 --> 00:13:35,930
this notion of verification.

238
00:13:35,930 --> 00:13:39,682
OK, so let me go-- there
was a good question here.

239
00:13:39,682 --> 00:13:41,390
Let me just see if I
want to answer that.

240
00:13:44,330 --> 00:13:48,380
Yeah, I mean, this is a little
bit of a longer question

241
00:13:48,380 --> 00:13:52,790
than I want to fully
respond to but--

242
00:13:52,790 --> 00:13:55,430
well, let's turn to my
next slide, which maybe

243
00:13:55,430 --> 00:13:58,568
sort bringing that out anyway.

244
00:13:58,568 --> 00:14:00,360
Actually, it's a couple
of slides from now.

245
00:14:00,360 --> 00:14:01,580
But I'll get to that point.

246
00:14:04,380 --> 00:14:07,280
So let's look at Hamiltonian,
the hampath problem.

247
00:14:07,280 --> 00:14:10,370
And what I'm going to show is
the Hamiltonian path problem

248
00:14:10,370 --> 00:14:11,500
is in NP.

249
00:14:11,500 --> 00:14:13,250
And I'm going to walk
you through this one

250
00:14:13,250 --> 00:14:14,870
kind of slowly.

251
00:14:14,870 --> 00:14:16,670
So the Hamiltonian
path problem, remember,

252
00:14:16,670 --> 00:14:22,340
we don't know if it's
in P. But it is in NP.

253
00:14:22,340 --> 00:14:23,660
So it's in one of these.

254
00:14:23,660 --> 00:14:27,440
You can solve Hamiltonian
path in polynomial time

255
00:14:27,440 --> 00:14:29,330
if you're a
non-deterministic machine.

256
00:14:32,200 --> 00:14:33,350
Why is that?

257
00:14:33,350 --> 00:14:41,810
Well, it's because of the
parallelism of non-determinism,

258
00:14:41,810 --> 00:14:43,210
which allows you
to kind of check

259
00:14:43,210 --> 00:14:46,930
all of the paths on
different branches.

260
00:14:46,930 --> 00:14:52,540
So let me first describe how I
would write down the algorithm.

261
00:14:52,540 --> 00:14:56,200
And then, we'll kind of try
to unpack that and understand

262
00:14:56,200 --> 00:15:01,630
how that actually looks in
terms of the Turing machine's

263
00:15:01,630 --> 00:15:03,660
computation.

264
00:15:03,660 --> 00:15:08,110
So first of all, taking the
Hamiltonian path problem,

265
00:15:08,110 --> 00:15:10,780
you are given an input
now, which is a graph

266
00:15:10,780 --> 00:15:14,050
and the nodes s and t where
I'm trying to figure out

267
00:15:14,050 --> 00:15:16,060
is that Hamiltonian path--

268
00:15:16,060 --> 00:15:18,340
again, which visits
all the nodes,

269
00:15:18,340 --> 00:15:22,090
which takes you from s to t.

270
00:15:22,090 --> 00:15:25,870
And we're trying to make now
a non-deterministic machine,

271
00:15:25,870 --> 00:15:34,030
which is going to accept all
such inputs which have a path.

272
00:15:34,030 --> 00:15:36,270
So the way this
non-deterministic machine

273
00:15:36,270 --> 00:15:39,060
is going to work
is it's basically

274
00:15:39,060 --> 00:15:44,550
going to use its non-determinism
to try all possible paths

275
00:15:44,550 --> 00:15:47,300
on the different branches.

276
00:15:47,300 --> 00:15:50,360
And the way I'll
specify that is to say,

277
00:15:50,360 --> 00:15:52,070
non-deterministically,
we're going

278
00:15:52,070 --> 00:15:56,390
to write down a candidate
path which is just

279
00:15:56,390 --> 00:15:59,600
going to be a
sequence of m nodes,

280
00:15:59,600 --> 00:16:01,910
where we will say that's
the total number of nodes

281
00:16:01,910 --> 00:16:02,720
of the graph.

282
00:16:02,720 --> 00:16:06,470
Remember, a Hamiltonian path,
because it visits every node,

283
00:16:06,470 --> 00:16:09,320
is going to be a path with
exactly m nodes in it.

284
00:16:11,903 --> 00:16:13,820
So I'm going to write
down a sequence of nodes

285
00:16:13,820 --> 00:16:16,230
as a candidate path.

286
00:16:16,230 --> 00:16:18,030
And I'm
non-deterministically going

287
00:16:18,030 --> 00:16:22,320
to choose every possible
sequence in this way.

288
00:16:25,370 --> 00:16:27,680
If you'd like to think
of the guessing metaphor

289
00:16:27,680 --> 00:16:30,050
for non-determinism,
you can think

290
00:16:30,050 --> 00:16:33,140
of the non-deterministic
machine as guessing

291
00:16:33,140 --> 00:16:36,320
the right path, which is going
to be the Hamiltonian path

292
00:16:36,320 --> 00:16:37,100
from s to t.

293
00:16:37,100 --> 00:16:39,530
But I think for
this discussion, it

294
00:16:39,530 --> 00:16:41,840
might be more helpful
to think about all

295
00:16:41,840 --> 00:16:46,243
of the different branches
of the non-determinism.

296
00:16:46,243 --> 00:16:47,660
Because that's
perhaps more useful

297
00:16:47,660 --> 00:16:50,180
when we're thinking about
it in terms of the time.

298
00:16:50,180 --> 00:16:52,520
I think you'll get used
to thinking about it.

299
00:16:52,520 --> 00:16:54,380
You should be used
to thinking about it

300
00:16:54,380 --> 00:16:56,540
in many of the different
ways. but maybe

301
00:16:56,540 --> 00:16:59,210
the computation
tree of all branches

302
00:16:59,210 --> 00:17:01,590
might be the more
helpful one here.

303
00:17:01,590 --> 00:17:05,339
So now after we write
down a candidate path

304
00:17:05,339 --> 00:17:09,260
sequence of nodes,
now I have to check

305
00:17:09,260 --> 00:17:12,819
that this really is a path.

306
00:17:12,819 --> 00:17:15,579
And the way I'm going to do that
is to say, well, now if I have

307
00:17:15,579 --> 00:17:17,770
just a sequence of
nodes written down, what

308
00:17:17,770 --> 00:17:21,430
does it mean for it to be a
Hamiltonian path from s to t?

309
00:17:21,430 --> 00:17:26,410
Well, it better start with s
and end with t, first of all.

310
00:17:26,410 --> 00:17:30,890
And we have to make sure
that every step of the way

311
00:17:30,890 --> 00:17:33,590
is actually an edge.

312
00:17:33,590 --> 00:17:39,105
So each pair vi to vi plus 1
has to be an edge in the graph.

313
00:17:39,105 --> 00:17:40,480
Otherwise, that
sequence of nodes

314
00:17:40,480 --> 00:17:44,380
is not going to be a legitimate
Hamiltonian path from s to t.

315
00:17:44,380 --> 00:17:46,690
And it has to be a simple path.

316
00:17:46,690 --> 00:17:48,250
You can't be repeating nodes.

317
00:17:51,100 --> 00:17:53,020
These four conditions
together will

318
00:17:53,020 --> 00:17:56,470
guarantee that we have
a Hamiltonian path.

319
00:17:56,470 --> 00:18:01,520
And once we have written
down a candidate sequence,

320
00:18:01,520 --> 00:18:03,950
we can just check that the
sequence actually works.

321
00:18:06,750 --> 00:18:11,820
At this second stage
of the algorithm,

322
00:18:11,820 --> 00:18:13,930
non-determinism isn't necessary.

323
00:18:13,930 --> 00:18:16,290
This is going to be a
deterministic phase.

324
00:18:16,290 --> 00:18:19,320
But stage one of
the algorithm is

325
00:18:19,320 --> 00:18:22,260
going to be a non-deterministic
phase where it's writing down

326
00:18:22,260 --> 00:18:23,940
all possible paths.

327
00:18:23,940 --> 00:18:26,340
Now, I'm going to try
to unpack that for you

328
00:18:26,340 --> 00:18:30,120
so you can actually visualize
how the machine is doing this.

329
00:18:32,810 --> 00:18:35,750
And then, of course,
you know on each branch

330
00:18:35,750 --> 00:18:37,730
of the non-determinism,
you're going

331
00:18:37,730 --> 00:18:41,450
to check to see whether the
conditions have been satisfied.

332
00:18:41,450 --> 00:18:45,860
And on that branch, if the
conditions were not satisfied,

333
00:18:45,860 --> 00:18:47,750
that branch is going to reject.

334
00:18:47,750 --> 00:18:50,000
Of course, one of the other
branches might yet accept,

335
00:18:50,000 --> 00:18:54,960
so that's how
non-determinism works.

336
00:18:54,960 --> 00:18:55,460
OK?

337
00:18:55,460 --> 00:19:12,020
So I'd like to visualize
this as the tree

338
00:19:12,020 --> 00:19:18,930
of the different branches of the
computation of m on its input.

339
00:19:18,930 --> 00:19:23,550
So here is our non-deterministic
Turing machine.

340
00:19:23,550 --> 00:19:24,050
Which?

341
00:19:24,050 --> 00:19:25,400
This one.

342
00:19:25,400 --> 00:19:31,040
And you provide it with
the input G, s, and t.

343
00:19:31,040 --> 00:19:34,680
And how is the machine
actually working?

344
00:19:34,680 --> 00:19:36,980
So when I say
non-deterministically write

345
00:19:36,980 --> 00:19:40,310
down a sequence of m nodes--

346
00:19:40,310 --> 00:19:44,510
look, this is getting into
a little bit more detail

347
00:19:44,510 --> 00:19:46,430
than I would normally
think about it,

348
00:19:46,430 --> 00:19:49,310
because we try to tend to think
about things at a higher level.

349
00:19:49,310 --> 00:19:52,250
But just to get us
started, I think

350
00:19:52,250 --> 00:19:57,190
it's good to think about
this with a bit more detail.

351
00:19:57,190 --> 00:20:02,100
So let's think of the m nodes
as being numbered, having

352
00:20:02,100 --> 00:20:05,632
labels numbered 1 through m.

353
00:20:05,632 --> 00:20:07,590
And I'm going to think
about them being labeled

354
00:20:07,590 --> 00:20:09,960
by their binary sequences.

355
00:20:09,960 --> 00:20:11,760
We're going to write
down those nodes.

356
00:20:11,760 --> 00:20:14,580
That's how the machine is
going to have to operate

357
00:20:14,580 --> 00:20:17,160
on those numbers for the nodes.

358
00:20:17,160 --> 00:20:19,860
We'll think about them as
being written in binary.

359
00:20:19,860 --> 00:20:22,920
And now, as the machine is
going to be writing down,

360
00:20:22,920 --> 00:20:26,230
let's say, the node v1.

361
00:20:26,230 --> 00:20:28,270
So it's
non-deterministically picking

362
00:20:28,270 --> 00:20:30,130
the first node of the sequence.

363
00:20:30,130 --> 00:20:32,470
What does that
actually mean in terms

364
00:20:32,470 --> 00:20:41,140
of the step-by-step processing
of the Turing machine m?

365
00:20:41,140 --> 00:20:45,670
Well, it's going to be
guessing via a sequence

366
00:20:45,670 --> 00:20:52,820
of non-deterministic moves,
the bits that represent

367
00:20:52,820 --> 00:20:57,650
the number of the node for v1.

368
00:20:57,650 --> 00:21:03,670
For example, v1 might be the
node number 5 in the graph.

369
00:21:03,670 --> 00:21:05,230
Of course,
non-deterministically,

370
00:21:05,230 --> 00:21:07,060
the machine is on
different branches,

371
00:21:07,060 --> 00:21:10,480
picking all different
possible choices for v1.

372
00:21:10,480 --> 00:21:14,230
Those are going to be the
different branches here.

373
00:21:14,230 --> 00:21:17,260
But one of the
branches might be--

374
00:21:17,260 --> 00:21:21,153
and what I'm really
representing here, these are--

375
00:21:21,153 --> 00:21:23,320
I probably could have written
this down on the slide

376
00:21:23,320 --> 00:21:25,390
here in tiny font.

377
00:21:25,390 --> 00:21:28,438
But these are like
the 0, 1 choices.

378
00:21:28,438 --> 00:21:29,980
That's why it's sort
of a binary tree

379
00:21:29,980 --> 00:21:37,220
here for writing
down the bits of v1.

380
00:21:37,220 --> 00:21:44,030
So here maybe this could be
101, representing the number

381
00:21:44,030 --> 00:21:48,500
5, which might be the very
first node that I'm writing down

382
00:21:48,500 --> 00:21:50,250
in my sequence.

383
00:21:50,250 --> 00:21:52,590
Some other branch is going
to write down node number 6.

384
00:21:52,590 --> 00:21:54,960
Some other branch is going
to write down node number 2.

385
00:21:54,960 --> 00:21:56,460
Because
non-deterministically, we're

386
00:21:56,460 --> 00:21:58,860
making all possible
choices for v1.

387
00:21:58,860 --> 00:22:01,950
That's what I'm trying to
show in this little part

388
00:22:01,950 --> 00:22:06,990
of the computation
of m on this input.

389
00:22:06,990 --> 00:22:10,910
So then, after it's
finished writing down

390
00:22:10,910 --> 00:22:17,660
the description of the
node for its choice for v1,

391
00:22:17,660 --> 00:22:20,630
it goes down to
choose what v2 is.

392
00:22:20,630 --> 00:22:22,340
Again, non-deterministically,
so there's

393
00:22:22,340 --> 00:22:30,330
going to be more branches for
each possible choice of v2.

394
00:22:30,330 --> 00:22:34,260
And so on, node after node.

395
00:22:34,260 --> 00:22:36,810
Then, it's going to finally
get to the last node, vm.

396
00:22:36,810 --> 00:22:39,780
It's going to write down
lots of choices for vm.

397
00:22:39,780 --> 00:22:44,820
And at this point here, we
have completed the first stage

398
00:22:44,820 --> 00:22:45,760
of the algorithm.

399
00:22:45,760 --> 00:22:48,390
Now, there's some
huge tree of all

400
00:22:48,390 --> 00:22:51,000
of the possible choices
for the V's that

401
00:22:51,000 --> 00:22:53,400
have shown up at this point.

402
00:22:53,400 --> 00:22:54,980
OK?

403
00:22:54,980 --> 00:22:58,850
And now, we're going to
move into the second phase.

404
00:22:58,850 --> 00:23:03,170
So following this,
there's going to be,

405
00:23:03,170 --> 00:23:12,090
here, a bunch of deterministic
steps of the machine.

406
00:23:12,090 --> 00:23:17,190
So no more branching
is needed because here,

407
00:23:17,190 --> 00:23:19,740
we've written down--
at this point,

408
00:23:19,740 --> 00:23:25,410
we've reached a point in
each of those locations,

409
00:23:25,410 --> 00:23:28,680
where we've chosen
one of the candidates,

410
00:23:28,680 --> 00:23:30,900
one of the possible branches--

411
00:23:30,900 --> 00:23:37,350
one of the possible paths
through the graph, I'm sorry.

412
00:23:37,350 --> 00:23:43,960
So here, we're guessing
potential paths in the graph.

413
00:23:43,960 --> 00:23:46,470
And now, we're going to
check that we actually

414
00:23:46,470 --> 00:23:52,320
have picked a path that's a
Hamiltonian path from s to t.

415
00:23:52,320 --> 00:23:52,830
OK?

416
00:23:52,830 --> 00:23:55,290
So each one of these
branches is now

417
00:23:55,290 --> 00:23:57,120
going to end up
accepting or rejecting.

418
00:23:57,120 --> 00:23:59,040
And the whole
overall computation

419
00:23:59,040 --> 00:24:01,350
is going to accept if
at least one of them

420
00:24:01,350 --> 00:24:03,570
ended up accepting,
which means you actually

421
00:24:03,570 --> 00:24:06,410
found a Hamiltonian path.

422
00:24:06,410 --> 00:24:06,910
OK?

423
00:24:06,910 --> 00:24:12,610
I don't know if that's helpful
to you or not, but that is--

424
00:24:12,610 --> 00:24:16,460
if there's any questions
on this, let's see.

425
00:24:16,460 --> 00:24:19,065
Question on is there something--

426
00:24:23,750 --> 00:24:28,670
trying to draw a connection here
between this and computation

427
00:24:28,670 --> 00:24:30,830
histories.

428
00:24:30,830 --> 00:24:34,310
I mean, there is a pattern
here that does come up often

429
00:24:34,310 --> 00:24:38,150
where you want to check
that something starts right,

430
00:24:38,150 --> 00:24:41,030
ends right, and that all of
the intermediates are right.

431
00:24:41,030 --> 00:24:48,410
So I think there is some
deeper connection here.

432
00:24:48,410 --> 00:24:52,100
Probably too hard
to explain but that

433
00:24:52,100 --> 00:24:55,490
has something to do with this
Hamiltonian path problem.

434
00:24:58,300 --> 00:25:00,390
Why are we using
binary representation?

435
00:25:00,390 --> 00:25:04,377
Well, we're going
to talk about--

436
00:25:04,377 --> 00:25:06,210
the algorithm would
have worked equally well

437
00:25:06,210 --> 00:25:12,630
if we used base three
or base five or base 20

438
00:25:12,630 --> 00:25:18,000
as a way of writing down
our labels for the nodes.

439
00:25:18,000 --> 00:25:21,740
But in a sense,
it doesn't matter.

440
00:25:21,740 --> 00:25:24,830
The alphabet has to be finite
though, so that's true.

441
00:25:24,830 --> 00:25:28,520
I mean, that's why it's not just
in a single step of the Turing

442
00:25:28,520 --> 00:25:35,690
machine that you would pick
the node the choice for v1.

443
00:25:35,690 --> 00:25:37,820
You really have to go
to a sequence of steps.

444
00:25:37,820 --> 00:25:42,110
Because each of the
branches of the machine

445
00:25:42,110 --> 00:25:44,190
only has a fixed
number of choices.

446
00:25:44,190 --> 00:25:47,480
So you can't, in a single
step of the Turing machine,

447
00:25:47,480 --> 00:25:49,798
pick all the different
possibilities for v1.

448
00:25:49,798 --> 00:25:51,215
That has to go
through a sequence.

449
00:25:54,090 --> 00:25:54,590
OK.

450
00:25:59,150 --> 00:26:03,770
Now, let me do a second example,
the problem of composites.

451
00:26:03,770 --> 00:26:07,010
So the language
of all composites

452
00:26:07,010 --> 00:26:10,130
are all of the non-primes,
written as binary numbers

453
00:26:10,130 --> 00:26:10,740
again.

454
00:26:10,740 --> 00:26:14,540
So we'll talk about the base and
the representation in a second.

455
00:26:14,540 --> 00:26:18,410
But just imagine these
are all of the numbers

456
00:26:18,410 --> 00:26:20,330
that are not prime.

457
00:26:20,330 --> 00:26:24,260
And that language is easily
seen to be a member of NP.

458
00:26:26,920 --> 00:26:29,110
Here is, again, the
algorithm for that.

459
00:26:32,450 --> 00:26:38,460
Given x, we want to accept x,
if it's not a prime number.

460
00:26:38,460 --> 00:26:40,325
So it has some
non-trivial factor.

461
00:26:45,020 --> 00:26:47,660
So first, the way the
non-deterministic machine

462
00:26:47,660 --> 00:26:49,980
is going to work is it's
going to guess that factor.

463
00:26:49,980 --> 00:26:51,530
So non-deterministically,
it's going

464
00:26:51,530 --> 00:26:55,310
to try every possible factor.

465
00:26:55,310 --> 00:26:58,840
Y is going to be a
number between 1 and x.

466
00:26:58,840 --> 00:27:01,060
But not including 1.

467
00:27:01,060 --> 00:27:02,740
You have to be an
interesting factor,

468
00:27:02,740 --> 00:27:05,210
so not including one
in the number itself.

469
00:27:05,210 --> 00:27:07,730
So something
strictly in between.

470
00:27:07,730 --> 00:27:08,900
And we're going to then--

471
00:27:08,900 --> 00:27:11,920
after we've
non-deterministically chosen y,

472
00:27:11,920 --> 00:27:18,260
then we're going to test to
see if y is really a factor.

473
00:27:18,260 --> 00:27:23,956
So we'll see if y divides evenly
into x with a remainder of 0.

474
00:27:23,956 --> 00:27:27,110
If that branch successfully
picked the right y,

475
00:27:27,110 --> 00:27:29,400
it's going to accept.

476
00:27:29,400 --> 00:27:32,820
And some other branch where it
might have picked the wrong y,

477
00:27:32,820 --> 00:27:34,110
will not.

478
00:27:34,110 --> 00:27:37,800
And if x is really
a composite number,

479
00:27:37,800 --> 00:27:41,460
some branch will
find the factor.

480
00:27:41,460 --> 00:27:43,460
Now, the base doesn't matter.

481
00:27:43,460 --> 00:27:48,160
Could have used base 10, because
you can convert from one base

482
00:27:48,160 --> 00:27:49,460
to another.

483
00:27:49,460 --> 00:27:52,330
So this is really in terms
of our representation

484
00:27:52,330 --> 00:27:54,190
of the number.

485
00:27:54,190 --> 00:27:58,177
But I do want to make one
point here, that changing--

486
00:27:58,177 --> 00:28:00,010
we don't want to write
the number in unary--

487
00:28:03,400 --> 00:28:07,690
writing the number of
k as a sequence of k1s.

488
00:28:07,690 --> 00:28:09,980
That's not really a base.

489
00:28:09,980 --> 00:28:12,670
That's just an exponential
representation for the number

490
00:28:12,670 --> 00:28:14,560
and that changes the game.

491
00:28:14,560 --> 00:28:20,440
Because if you make the
input exponentially larger,

492
00:28:20,440 --> 00:28:24,790
then it's going to change
whether the algorithm

493
00:28:24,790 --> 00:28:28,330
relative to that exponentially
larger input is polynomial

494
00:28:28,330 --> 00:28:29,560
or not.

495
00:28:29,560 --> 00:28:34,912
So an algorithm that might have
been exponential originally

496
00:28:34,912 --> 00:28:36,370
when the number's
written in binary

497
00:28:36,370 --> 00:28:38,870
might become polynomial if the
numbers are written in unary.

498
00:28:41,640 --> 00:28:43,620
And I do want to
mention as a side note,

499
00:28:43,620 --> 00:28:46,860
that the composites language--

500
00:28:46,860 --> 00:28:48,720
or primes, for that matter--

501
00:28:48,720 --> 00:28:52,680
both are NP.

502
00:28:52,680 --> 00:28:54,180
But we won't cover that.

503
00:28:54,180 --> 00:28:58,290
So whereas the Hamiltonian
path problem is not

504
00:28:58,290 --> 00:29:02,490
known whether it's NP, the
primes and composites problem

505
00:29:02,490 --> 00:29:04,440
are NP.

506
00:29:04,440 --> 00:29:05,190
So that was known.

507
00:29:05,190 --> 00:29:08,130
That was actually a very
big result in the field.

508
00:29:08,130 --> 00:29:16,020
Solved by folks at one of the
Indian Institute of Technology

509
00:29:16,020 --> 00:29:18,691
back about almost
20 years ago now.

510
00:29:23,440 --> 00:29:27,310
So let's turn here, to
trying to get an intuitive

511
00:29:27,310 --> 00:29:28,750
feeling for P and NP.

512
00:29:28,750 --> 00:29:33,160
And we'll return now to this
notion of NP corresponding

513
00:29:33,160 --> 00:29:34,810
to easy verifiability.

514
00:29:38,800 --> 00:29:42,490
NP are the languages where you
can easily verify membership

515
00:29:42,490 --> 00:29:43,630
quickly.

516
00:29:43,630 --> 00:29:45,700
I'll try to explain
what that means.

517
00:29:45,700 --> 00:29:50,530
In contrast, P are the languages
where you can test membership

518
00:29:50,530 --> 00:29:51,820
quickly.

519
00:29:51,820 --> 00:29:57,940
By quickly, I'm using
polynomial time.

520
00:29:57,940 --> 00:30:01,150
That's going to be,
for us, that's what

521
00:30:01,150 --> 00:30:02,770
quickly means in this course.

522
00:30:07,650 --> 00:30:11,880
In the case of the
Hamiltonian path problem,

523
00:30:11,880 --> 00:30:13,980
the way you verify
the membership

524
00:30:13,980 --> 00:30:16,530
is you give the path.

525
00:30:16,530 --> 00:30:18,238
In the case of the
composites, the way

526
00:30:18,238 --> 00:30:20,280
you verify the membership
is you give the factor.

527
00:30:22,960 --> 00:30:27,350
In those two cases,
and in general,

528
00:30:27,350 --> 00:30:29,920
when we have a
problem that's in NP,

529
00:30:29,920 --> 00:30:35,350
we think of this
verification as having--

530
00:30:35,350 --> 00:30:38,260
we give it a special name,
called a certificate,

531
00:30:38,260 --> 00:30:40,540
or sometimes a
short certificate,

532
00:30:40,540 --> 00:30:43,870
to emphasize the polynomiality
of the certificate.

533
00:30:43,870 --> 00:30:46,600
It's like a way of
proving that you're

534
00:30:46,600 --> 00:30:47,815
a member of the language.

535
00:30:51,050 --> 00:30:54,550
In the case of COMPOSITES,
the proof is the factor.

536
00:30:54,550 --> 00:30:56,410
In the case of
HAMPATH, the proof

537
00:30:56,410 --> 00:31:01,080
is the path, the
Hamiltonian path.

538
00:31:01,080 --> 00:31:04,125
Contrast that, for example,
if you had a prime number.

539
00:31:07,570 --> 00:31:10,230
Proving a number is composite
is easy because you just

540
00:31:10,230 --> 00:31:11,970
exhibit the factor.

541
00:31:11,970 --> 00:31:15,470
How would you prove
that a number is prime?

542
00:31:15,470 --> 00:31:20,210
What's the short
certificate of proving

543
00:31:20,210 --> 00:31:23,150
that some number has no factor?

544
00:31:23,150 --> 00:31:25,530
That's not so obvious.

545
00:31:25,530 --> 00:31:27,030
In fact, there are
ways of doing it,

546
00:31:27,030 --> 00:31:29,900
which I'm not going to get
into in the case of testing

547
00:31:29,900 --> 00:31:31,910
of numbers prime.

548
00:31:31,910 --> 00:31:36,470
And now it's even known to be
in P, so that's even better.

549
00:31:36,470 --> 00:31:40,420
But there's no obvious way
of proving that a number is

550
00:31:40,420 --> 00:31:42,580
prime with a short certificate.

551
00:31:45,190 --> 00:31:48,850
This concept of
being able to verify

552
00:31:48,850 --> 00:31:51,130
when you are a member
of the language, that's

553
00:31:51,130 --> 00:31:54,870
key to understanding NP.

554
00:31:54,870 --> 00:31:58,170
That's the intuition you
need to develop and hopefully

555
00:31:58,170 --> 00:32:00,450
take away from today's
lecture, or at least

556
00:32:00,450 --> 00:32:03,570
by thinking about it,
reading the book, and so on.

557
00:32:07,330 --> 00:32:11,536
If you compare these
two classes, P and NP.

558
00:32:11,536 --> 00:32:15,160
P, first of all, is going
to be a subset of NP,

559
00:32:15,160 --> 00:32:17,290
both in terms of the
way we defined it

560
00:32:17,290 --> 00:32:19,240
because, deterministic
machines are

561
00:32:19,240 --> 00:32:22,670
a special case of
non-deterministic machines.

562
00:32:22,670 --> 00:32:25,600
But also if you want to think
about testing membership,

563
00:32:25,600 --> 00:32:27,700
if you can test
membership easily

564
00:32:27,700 --> 00:32:30,942
then you can certainly verify
it in terms of the certificate.

565
00:32:30,942 --> 00:32:31,900
You don't even need it.

566
00:32:31,900 --> 00:32:33,760
The certificate is
irrelevant at that point.

567
00:32:33,760 --> 00:32:35,860
Because whatever
the certificate is,

568
00:32:35,860 --> 00:32:40,240
you can still test
yourself whether the input

569
00:32:40,240 --> 00:32:41,980
is in the language or not.

570
00:32:47,010 --> 00:32:48,610
The big question,
as I mentioned,

571
00:32:48,610 --> 00:32:51,640
is whether these two
classes are the same.

572
00:32:51,640 --> 00:32:56,880
So does being able to verify
membership quickly, say

573
00:32:56,880 --> 00:32:59,970
with one of these certificates,
allow you to dispense

574
00:32:59,970 --> 00:33:00,900
with the certificate?

575
00:33:00,900 --> 00:33:02,850
Not even need a
certificate and just test

576
00:33:02,850 --> 00:33:06,270
it for yourself whether
you're in the language.

577
00:33:06,270 --> 00:33:10,570
And do that in polynomial time.

578
00:33:10,570 --> 00:33:12,310
That's the question.

579
00:33:12,310 --> 00:33:16,580
For a problem like
Hamiltonian path,

580
00:33:16,580 --> 00:33:19,280
do you need to
search for the answer

581
00:33:19,280 --> 00:33:21,050
if you're doing it
deterministically?

582
00:33:21,050 --> 00:33:26,570
Or can you somehow
avoid that and just

583
00:33:26,570 --> 00:33:30,950
come up with the answer
directly with a polynomial time

584
00:33:30,950 --> 00:33:32,780
solution?

585
00:33:32,780 --> 00:33:35,300
Nobody knows the answer to that.

586
00:33:35,300 --> 00:33:37,700
And it goes back, at this
point, quite a long time.

587
00:33:37,700 --> 00:33:39,350
It's almost 60 years now.

588
00:33:43,100 --> 00:33:46,740
That problem has
been around 60 years.

589
00:33:46,740 --> 00:33:48,030
No, that would be 50 years.

590
00:33:48,030 --> 00:33:49,830
No, 50 years, almost 50 years.

591
00:33:56,770 --> 00:33:59,110
Most people believe that
P is different from NP.

592
00:33:59,110 --> 00:34:01,360
In other words, that
there are problems in P--

593
00:34:01,360 --> 00:34:05,080
in NP which are not
in P. A candidate

594
00:34:05,080 --> 00:34:08,150
would be the Hamiltonian
path problem.

595
00:34:08,150 --> 00:34:12,230
But it seems to be very
hard to prove that.

596
00:34:12,230 --> 00:34:13,989
And part of the
reason is, how do you

597
00:34:13,989 --> 00:34:17,830
prove that a problem
like Hamiltonian path

598
00:34:17,830 --> 00:34:21,699
does not have a
polynomial time algorithm.

599
00:34:21,699 --> 00:34:23,920
It's very tricky to do
that, because the class

600
00:34:23,920 --> 00:34:26,530
of polynomial time algorithms
is a very rich class.

601
00:34:26,530 --> 00:34:30,130
Polynomial time algorithms
are very powerful.

602
00:34:30,130 --> 00:34:32,670
And to try to prove--
there's no clever way

603
00:34:32,670 --> 00:34:35,219
of solving the
Hamiltonian path problem.

604
00:34:35,219 --> 00:34:38,451
It just seems to be beyond
our present day mathematics.

605
00:34:38,451 --> 00:34:40,409
I believe someday somebody's
going to solve it.

606
00:34:40,409 --> 00:34:45,935
But so far, no
one has succeeded.

607
00:34:45,935 --> 00:34:47,940
So what I thought
we would do is--

608
00:34:47,940 --> 00:34:49,860
I think I have a check-in here.

609
00:34:49,860 --> 00:34:50,909
Yes.

610
00:34:50,909 --> 00:34:52,275
And then we'll stop for a break.

611
00:34:54,780 --> 00:34:59,280
Let's look at the complementary
problem, HAMPATH complement.

612
00:35:03,660 --> 00:35:08,180
You're in the language now
if you don't have a path.

613
00:35:08,180 --> 00:35:10,684
So is that complementary
problem in NP?

614
00:35:15,390 --> 00:35:17,520
For that to be
the case, we would

615
00:35:17,520 --> 00:35:22,710
need to have short certificates
of when a graph does not

616
00:35:22,710 --> 00:35:26,070
have a Hamiltonian path.

617
00:35:29,500 --> 00:35:30,540
I leave it to you.

618
00:35:30,540 --> 00:35:31,540
There are three choices.

619
00:35:34,330 --> 00:35:36,600
OK?

620
00:35:36,600 --> 00:35:39,660
Going to stop here, so make
sure you get your participation

621
00:35:39,660 --> 00:35:40,440
credit here.

622
00:35:42,980 --> 00:35:46,235
I'm going to end
the polling now.

623
00:35:46,235 --> 00:35:46,735
Interesting.

624
00:35:46,735 --> 00:35:49,060
[LAUGHS] So the
majority is wrong.

625
00:35:49,060 --> 00:35:50,790
Well, not wrong, we don't know.

626
00:35:56,950 --> 00:35:58,900
I think the only fair
answer to this question

627
00:35:58,900 --> 00:36:04,300
is C. Because we don't
know whether or not

628
00:36:04,300 --> 00:36:06,370
we can give short
certificates for a graph

629
00:36:06,370 --> 00:36:08,560
not to have a Hamiltonian path.

630
00:36:08,560 --> 00:36:15,030
If P equaled NP, then you
can test in polynomial time

631
00:36:15,030 --> 00:36:17,970
whether a graph has
a Hamiltonian path.

632
00:36:17,970 --> 00:36:20,610
And then the computation
itself would be a certificate,

633
00:36:20,610 --> 00:36:23,647
whether it has a path or
whether it doesn't have a path.

634
00:36:23,647 --> 00:36:25,980
Because it would be something
that you can check easily.

635
00:36:28,950 --> 00:36:35,700
Since we don't know for sure
that P is different from NP,

636
00:36:35,700 --> 00:36:38,250
P could be equal to
NP, then it's possible

637
00:36:38,250 --> 00:36:43,510
that we could give
a short certificate.

638
00:36:43,510 --> 00:36:47,780
Namely, the computation of
the polynomial algorithm.

639
00:36:47,780 --> 00:36:54,700
So the only really reasonable
answer to this question

640
00:36:54,700 --> 00:36:55,630
is that we don't know.

641
00:36:59,060 --> 00:37:00,470
Just ponder that.

642
00:37:03,920 --> 00:37:08,900
Those of you who answered
yes, however, need to go back.

643
00:37:08,900 --> 00:37:12,470
And I put this here
explicitly because I

644
00:37:12,470 --> 00:37:18,180
know this is a confusion
for, well I can

645
00:37:18,180 --> 00:37:19,620
see, for quite a few of you.

646
00:37:25,310 --> 00:37:27,965
When we have
non-deterministic computation

647
00:37:27,965 --> 00:37:30,020
and you have a
non-deterministic machine,

648
00:37:30,020 --> 00:37:33,710
you can't simply
invert the answer

649
00:37:33,710 --> 00:37:35,960
and get back a
non-deterministic machine.

650
00:37:35,960 --> 00:37:39,240
Non-determinism does
not work that way.

651
00:37:39,240 --> 00:37:45,800
If you remember, the complement
of a pushdown automaton

652
00:37:45,800 --> 00:37:47,210
is not a pushdown automaton.

653
00:37:49,970 --> 00:37:51,980
If you have a
non-deterministic machine

654
00:37:51,980 --> 00:37:54,620
and you invert all of
the responses on each

655
00:37:54,620 --> 00:37:59,900
of the branches, it's not going
to be recognizing or deciding

656
00:37:59,900 --> 00:38:01,055
the complementary language.

657
00:38:07,150 --> 00:38:09,160
I think that this
is something you--

658
00:38:09,160 --> 00:38:13,120
if you answered yes, you
need to go back and make

659
00:38:13,120 --> 00:38:18,430
sure you understand why yes
is not a reasonable answer

660
00:38:18,430 --> 00:38:19,480
to this question.

661
00:38:19,480 --> 00:38:22,220
Because that's not how
non-determinism works.

662
00:38:22,220 --> 00:38:26,630
So you have a not
complete understanding

663
00:38:26,630 --> 00:38:27,590
of non-determinism.

664
00:38:27,590 --> 00:38:30,830
And that's going to be really
important for us going forward.

665
00:38:30,830 --> 00:38:35,840
I really urge you to figure out
and understand why yes is not

666
00:38:35,840 --> 00:38:38,780
a good answer to this check-in.

667
00:38:38,780 --> 00:38:40,470
OK, so I think we will--

668
00:38:40,470 --> 00:38:45,320
we can talk about that
more over the break.

669
00:38:45,320 --> 00:38:51,125
And so we'll return
here in five minutes.

670
00:38:55,320 --> 00:39:01,470
Somebody's asking about--
can infinite sequences

671
00:39:01,470 --> 00:39:02,640
be generated by the machine.

672
00:39:05,640 --> 00:39:10,520
When we're talking
about, especially

673
00:39:10,520 --> 00:39:14,000
in the complexity section of the
course, all of the computations

674
00:39:14,000 --> 00:39:17,102
are going to be bounded in time.

675
00:39:17,102 --> 00:39:19,310
So we're not going to be
thinking about infinite runs

676
00:39:19,310 --> 00:39:19,935
of the machine.

677
00:39:19,935 --> 00:39:21,570
That's not going to
be relevant for us.

678
00:39:21,570 --> 00:39:24,230
So let's not think about that.

679
00:39:24,230 --> 00:39:26,135
How does a Turing
machine perform division?

680
00:39:28,980 --> 00:39:30,870
How does a Turing
machine perform division?

681
00:39:30,870 --> 00:39:34,590
Well, how do you
perform division?

682
00:39:34,590 --> 00:39:42,630
[LAUGHS] Long division is an
operation that can run in--

683
00:39:42,630 --> 00:39:46,530
the long division procedure
that you learn in grade school,

684
00:39:46,530 --> 00:39:50,100
you can implement that
on a Turing machine.

685
00:39:50,100 --> 00:39:52,620
Yes, a Turing machine
can definitely perform--

686
00:39:52,620 --> 00:39:54,750
do long division, or
division of one integer

687
00:39:54,750 --> 00:39:57,420
by another in polynomial time.

688
00:40:04,062 --> 00:40:04,770
Another question.

689
00:40:04,770 --> 00:40:08,760
Can we generally say,
try dividing y by x?

690
00:40:08,760 --> 00:40:11,370
Or do we have to enumerate
a string of length y

691
00:40:11,370 --> 00:40:12,455
and cross off every-- no.

692
00:40:12,455 --> 00:40:13,830
I think that's
the same question.

693
00:40:18,430 --> 00:40:20,840
I mean, if you have
numbers written in binary,

694
00:40:20,840 --> 00:40:22,090
how would you do the division?

695
00:40:22,090 --> 00:40:27,550
You're not going to
use long division.

696
00:40:27,550 --> 00:40:30,250
Anything such as
the thing that's

697
00:40:30,250 --> 00:40:34,780
proposed here by the
questioner is going

698
00:40:34,780 --> 00:40:36,560
to be an exponential algorithm.

699
00:40:36,560 --> 00:40:37,720
So don't do it that way.

700
00:40:46,790 --> 00:40:48,830
Why does primes in P--

701
00:40:48,830 --> 00:40:51,680
composites in P not
imply primes in P?

702
00:40:51,680 --> 00:40:53,410
It does imply.

703
00:40:53,410 --> 00:40:57,800
If composites are in P,
then primes is in P as well.

704
00:40:57,800 --> 00:41:00,620
When you have a
deterministic machine,

705
00:41:00,620 --> 00:41:02,240
you can flip the answer.

706
00:41:02,240 --> 00:41:04,430
When you have a
non-deterministic machine,

707
00:41:04,430 --> 00:41:06,330
you may not be able
to flip the answer.

708
00:41:06,330 --> 00:41:10,880
So deterministic machine
just having a single branch,

709
00:41:10,880 --> 00:41:12,650
you can make another
deterministic machine

710
00:41:12,650 --> 00:41:14,780
that runs in the same
amount of time that does

711
00:41:14,780 --> 00:41:16,670
the complementary language.

712
00:41:16,670 --> 00:41:21,170
Because for
deterministic machines,

713
00:41:21,170 --> 00:41:25,640
just like for deciders in
the computability section,

714
00:41:25,640 --> 00:41:28,410
you can just invert the answer.

715
00:41:28,410 --> 00:41:33,500
There is an analogy here
between P and decidability,

716
00:41:33,500 --> 00:41:35,690
and NP and recognizability.

717
00:41:35,690 --> 00:41:38,510
It's not an airtight
analogy here,

718
00:41:38,510 --> 00:41:40,700
but there is some
relationship there.

719
00:41:44,620 --> 00:41:46,810
Somebody's asking me, what
are the implications of P

720
00:41:46,810 --> 00:41:48,250
equal to NP?

721
00:41:48,250 --> 00:41:50,620
Lots of implications.

722
00:41:50,620 --> 00:41:51,970
Too long to enumerate now.

723
00:41:51,970 --> 00:41:55,600
But, for example, you
would be able to break

724
00:41:55,600 --> 00:42:01,325
basically all cryptosystems that
I'm aware of, if P equal to NP.

725
00:42:01,325 --> 00:42:02,950
So we would have a
lot of consequences.

726
00:42:05,500 --> 00:42:08,200
Somebody's asking,
so composites--

727
00:42:08,200 --> 00:42:10,120
primality and
compositeness testing

728
00:42:10,120 --> 00:42:11,920
is solvable in polynomial time.

729
00:42:11,920 --> 00:42:14,590
But factoring,
interestingly enough,

730
00:42:14,590 --> 00:42:16,990
is not known to be solvable
in polynomial time.

731
00:42:22,187 --> 00:42:24,770
We may talk about this a little
bit toward the end of the term

732
00:42:24,770 --> 00:42:25,437
if we have time.

733
00:42:28,830 --> 00:42:32,040
The algorithms for testing
whether a number is

734
00:42:32,040 --> 00:42:35,190
prime or composite
in polynomial time

735
00:42:35,190 --> 00:42:37,890
do not operate by
looking for factors.

736
00:42:37,890 --> 00:42:41,400
They operate in an entirely
different way, basically

737
00:42:41,400 --> 00:42:46,890
by showing that a number is
prime or composite by looking

738
00:42:46,890 --> 00:42:48,570
at certain properties
of that number.

739
00:42:48,570 --> 00:42:51,120
But without testing
whether it has--

740
00:42:51,120 --> 00:42:53,940
testing, but without
finding a factor.

741
00:43:03,900 --> 00:43:09,390
Another question
here about asking

742
00:43:09,390 --> 00:43:11,220
when we talk about
encodings, do we

743
00:43:11,220 --> 00:43:14,215
have to say how we
encode numbers, values?

744
00:43:14,215 --> 00:43:15,090
No, we don't have to.

745
00:43:15,090 --> 00:43:19,770
We usually don't have to get
into spelling out encodings,

746
00:43:19,770 --> 00:43:22,080
as long as they're
reasonable encodings.

747
00:43:22,080 --> 00:43:25,028
So you don't have to usually.

748
00:43:25,028 --> 00:43:27,570
We're going to be talking about
things at a high enough level

749
00:43:27,570 --> 00:43:32,070
that the specific encodings
are not going to matter.

750
00:43:32,070 --> 00:43:37,680
Let's return to the
rest of our lecture.

751
00:43:37,680 --> 00:43:46,440
When we talk about, say,
this P versus NP problem.

752
00:43:46,440 --> 00:43:53,300
And how do you show
that a problem might not

753
00:43:53,300 --> 00:43:58,190
be solvable in P, like the
Hamiltonian path problem?

754
00:43:58,190 --> 00:44:08,810
Many people who are not
practitioners in the field,

755
00:44:08,810 --> 00:44:11,420
know about the P
versus NP problems--

756
00:44:11,420 --> 00:44:14,900
over the years, I've
gotten many, many emails

757
00:44:14,900 --> 00:44:19,475
and physical letters
from people about that.

758
00:44:22,550 --> 00:44:24,290
Since I've spent
some time thinking--

759
00:44:24,290 --> 00:44:29,120
I'm known as having spent
some time thinking about it.

760
00:44:29,120 --> 00:44:31,760
People claim to
solve the problem,

761
00:44:31,760 --> 00:44:38,630
solve P is NP, the P versus NP
problem, by basically saying,

762
00:44:38,630 --> 00:44:48,220
problems like Hamiltonian path
or other similar problems,

763
00:44:48,220 --> 00:44:52,210
basically there's clearly no way
to solve them without searching

764
00:44:52,210 --> 00:44:53,560
through a lot of possibilities.

765
00:44:53,560 --> 00:44:55,435
And then they go through
a big, long analysis

766
00:44:55,435 --> 00:44:56,950
showing that there
are exponentially

767
00:44:56,950 --> 00:44:58,990
many possibilities.

768
00:45:08,010 --> 00:45:10,980
A lot of the proofs that
claim to solve P versus NP,

769
00:45:10,980 --> 00:45:12,240
they all look like that.

770
00:45:12,240 --> 00:45:13,470
You only have to look--

771
00:45:13,470 --> 00:45:15,270
somewhere in that
paper, there's going

772
00:45:15,270 --> 00:45:21,610
to be a statement along the
lines, "to solve this problem,

773
00:45:21,610 --> 00:45:25,880
clearly you have
to do it this way."

774
00:45:25,880 --> 00:45:28,610
And that's the flaw
in the reasoning.

775
00:45:28,610 --> 00:45:32,930
Because just like for
the factoring problem--

776
00:45:32,930 --> 00:45:36,650
just like for the
compositeness testing problem,

777
00:45:36,650 --> 00:45:38,570
you don't necessarily
have to solve it

778
00:45:38,570 --> 00:45:39,945
by searching for factors.

779
00:45:39,945 --> 00:45:41,570
There might be some
other way to do it.

780
00:45:41,570 --> 00:45:44,390
You might be able to
solve the Hamiltonian path

781
00:45:44,390 --> 00:45:46,670
problem without searching
for Hamiltonian paths.

782
00:45:46,670 --> 00:45:49,140
There might be some other
process that you can use,

783
00:45:49,140 --> 00:45:52,490
which would give you the answer.

784
00:45:52,490 --> 00:45:54,250
The class of polynomial
time algorithms

785
00:45:54,250 --> 00:45:57,480
is very rich, can do
many, many things.

786
00:45:57,480 --> 00:46:02,600
And I wanted to present to
you one of the most important

787
00:46:02,600 --> 00:46:04,220
polynomial time algorithms.

788
00:46:04,220 --> 00:46:07,190
In a sense, you can
make a certain argument

789
00:46:07,190 --> 00:46:12,800
that this is the most
fundamental polynomial time

790
00:46:12,800 --> 00:46:13,760
algorithm.

791
00:46:13,760 --> 00:46:16,790
Some people might
argue with me on that.

792
00:46:16,790 --> 00:46:20,850
And that's a process called
dynamic programming, which

793
00:46:20,850 --> 00:46:22,620
I'm sure some of you
have seen already

794
00:46:22,620 --> 00:46:28,170
in your algorithms classes,
and some of you may not have.

795
00:46:28,170 --> 00:46:30,340
Since you have a
homework problem on it,

796
00:46:30,340 --> 00:46:33,910
I want to spend a little
time describing it to you.

797
00:46:33,910 --> 00:46:39,240
And that's useful for solving
this A CFG problem, which

798
00:46:39,240 --> 00:46:41,880
you may remember from the
first half of the course,

799
00:46:41,880 --> 00:46:48,150
involving testing if a
grammar generates a string.

800
00:46:48,150 --> 00:46:50,100
So you remember
this A CFG problem?

801
00:46:50,100 --> 00:46:53,770
You're giving a grammar,
context-free grammar,

802
00:46:53,770 --> 00:46:54,640
and a string.

803
00:46:54,640 --> 00:46:58,340
And I want to know, is it in
the language of the grammar.

804
00:46:58,340 --> 00:47:02,420
So that's going to turn out to
be solvable in polynomial time,

805
00:47:02,420 --> 00:47:05,176
but only with kind of
a clever algorithm.

806
00:47:09,000 --> 00:47:12,350
Remember, it's decidable.

807
00:47:12,350 --> 00:47:15,410
We decided it by
making sure that you

808
00:47:15,410 --> 00:47:18,740
are converting the grammar
in Chomsky normal form.

809
00:47:18,740 --> 00:47:21,560
Then all the derivations
have a certain length.

810
00:47:21,560 --> 00:47:24,560
You just try all the possible
derivations of that length,

811
00:47:24,560 --> 00:47:27,520
and you accept if any of
those derivations generate w.

812
00:47:31,540 --> 00:47:34,690
You may remember that from
the first half of the course.

813
00:47:37,660 --> 00:47:41,520
That immediately
gives an NP type

814
00:47:41,520 --> 00:47:44,280
algorithm for this language.

815
00:47:44,280 --> 00:47:49,217
Because basically, you
non-deterministically--

816
00:47:49,217 --> 00:47:51,050
instead of trying the
most sequentially, all

817
00:47:51,050 --> 00:47:52,580
of these derivations,
you try them

818
00:47:52,580 --> 00:47:54,950
in parallel
non-deterministically.

819
00:47:54,950 --> 00:47:58,280
So non-deterministically,
you pick some derivation

820
00:47:58,280 --> 00:48:00,000
of that length.

821
00:48:00,000 --> 00:48:03,290
And you accept it if
it generates the input.

822
00:48:03,290 --> 00:48:12,073
This classically
fits our model of NP.

823
00:48:12,073 --> 00:48:13,990
You can think of it as
guessing the derivation

824
00:48:13,990 --> 00:48:15,310
and checking that it works.

825
00:48:15,310 --> 00:48:19,210
Or, in parallel, writing down
all possible derivations.

826
00:48:19,210 --> 00:48:26,080
But this A CFG
problem is classically

827
00:48:26,080 --> 00:48:29,425
an NP problem, a problem in NP.

828
00:48:32,950 --> 00:48:36,220
And if you just imagine--

829
00:48:36,220 --> 00:48:38,740
then what's going to
be the certificate?

830
00:48:38,740 --> 00:48:41,890
If you found an input that's
in the language, that's

831
00:48:41,890 --> 00:48:45,570
generated by the grammar, the
certificate is the derivation.

832
00:48:45,570 --> 00:48:47,000
So if you look at
it that way, you

833
00:48:47,000 --> 00:48:49,310
might think, well, that's
the best you can do.

834
00:48:49,310 --> 00:48:51,890
This is going to be
an NP problem, in NP,

835
00:48:51,890 --> 00:48:53,930
and is not going to
be a way of avoiding

836
00:48:53,930 --> 00:48:55,220
searching for the derivation.

837
00:48:55,220 --> 00:48:56,667
But that's not true.

838
00:48:56,667 --> 00:48:59,000
There is a way of avoiding
searching for the derivation.

839
00:48:59,000 --> 00:49:03,410
You can build up the derivation
using dynamic programming.

840
00:49:03,410 --> 00:49:07,220
And so that's what I
wanted to describe for you,

841
00:49:07,220 --> 00:49:07,997
how that works.

842
00:49:07,997 --> 00:49:09,830
Also partly because
it's a homework problem.

843
00:49:09,830 --> 00:49:14,143
And I think dynamic programming
is a very important algorithm.

844
00:49:18,090 --> 00:49:25,940
Before we describe what
dynamic programming is,

845
00:49:25,940 --> 00:49:28,670
which is very
simple, by the way,

846
00:49:28,670 --> 00:49:31,460
let's try to work up to
it by making an attempt

847
00:49:31,460 --> 00:49:34,460
to solve this problem just
using ordinary recursion.

848
00:49:37,140 --> 00:49:40,120
How would we solve
the A CFG problem?

849
00:49:40,120 --> 00:49:43,640
So you're given grammar,
you're given an input.

850
00:49:43,640 --> 00:49:46,790
Let's assume the grammar
is in Chomsky normal form.

851
00:49:49,610 --> 00:49:51,060
That's going to be useful.

852
00:49:51,060 --> 00:49:53,210
So it's a Chomsky
normal form grammar.

853
00:49:53,210 --> 00:49:59,437
And we want to see how to
test if you can generate w.

854
00:49:59,437 --> 00:50:01,145
And it's going to be
recursive algorithm.

855
00:50:01,145 --> 00:50:05,690
The recursive algorithm is going
to actually solve something

856
00:50:05,690 --> 00:50:07,490
slightly more general.

857
00:50:07,490 --> 00:50:08,960
I'm going to give
you the grammar.

858
00:50:08,960 --> 00:50:10,500
I'm going to give
you the string.

859
00:50:10,500 --> 00:50:14,810
And I'm also going to allow you
to start at some other variable

860
00:50:14,810 --> 00:50:16,550
besides the start variable.

861
00:50:16,550 --> 00:50:19,340
I'm going to give
you some variable, R,

862
00:50:19,340 --> 00:50:23,120
and I know I want to know, can
I generate w starting at R?

863
00:50:26,880 --> 00:50:28,870
So that's my slightly
more general problem,

864
00:50:28,870 --> 00:50:31,040
which is going to be
useful in the recursion.

865
00:50:33,720 --> 00:50:36,630
So the input now
to this algorithm

866
00:50:36,630 --> 00:50:41,128
is the grammar, the input,
and the starting variable.

867
00:50:41,128 --> 00:50:42,920
And now how is the
algorithm going to work?

868
00:50:42,920 --> 00:50:46,370
It's going to try to
test, can I get to--

869
00:50:46,370 --> 00:50:48,920
is there some
derivation, pictured here

870
00:50:48,920 --> 00:50:53,000
as the parse tree,
for w starting at R?

871
00:50:53,000 --> 00:50:55,900
That's what the algorithm
is trying to answer.

872
00:50:55,900 --> 00:50:58,750
Can I get w from R?

873
00:50:58,750 --> 00:51:04,610
The way it's going to do that
is it's going to try to divide w

874
00:51:04,610 --> 00:51:10,860
into the two strings
in all possible ways.

875
00:51:10,860 --> 00:51:15,290
Which sounds like it might
be exponential, but it isn't.

876
00:51:15,290 --> 00:51:18,800
There's only a polynomial
number of ways to divide

877
00:51:18,800 --> 00:51:22,730
the string into two substrings.

878
00:51:22,730 --> 00:51:25,520
Just of order n, just depending
where you make that cut.

879
00:51:29,480 --> 00:51:30,630
So that's not too bad.

880
00:51:30,630 --> 00:51:32,660
There's a polynomial,
there's only n ways

881
00:51:32,660 --> 00:51:33,800
of making that division.

882
00:51:33,800 --> 00:51:44,430
And also I'm going to try every
possible rule that comes from R

883
00:51:44,430 --> 00:51:45,950
that generates two variables.

884
00:51:45,950 --> 00:51:49,290
So these are what's allowed
in Chomsky normal form,

885
00:51:49,290 --> 00:51:53,280
R goes to ST.

886
00:51:53,280 --> 00:51:57,480
For each possible way
of cutting w into x, y,

887
00:51:57,480 --> 00:51:59,190
and for each
possible rule, R goes

888
00:51:59,190 --> 00:52:04,430
to ST, I'm going to see,
recursively, can I get

889
00:52:04,430 --> 00:52:08,750
from S to x, and from T to y.

890
00:52:08,750 --> 00:52:10,460
So I'm going to use
my recursion now.

891
00:52:10,460 --> 00:52:17,030
Now that I have smaller
strings instead of w,

892
00:52:17,030 --> 00:52:21,580
I can apply recursion and
try to answer it that way.

893
00:52:21,580 --> 00:52:23,980
And this algorithm will work.

894
00:52:23,980 --> 00:52:28,810
If I found a way to cut w
into x, y, and I found a rule,

895
00:52:28,810 --> 00:52:33,310
R goes to ST such that S
generates x and T generates y,

896
00:52:33,310 --> 00:52:34,920
then I'm good.

897
00:52:34,920 --> 00:52:39,850
I know I can generate w from R.

898
00:52:39,850 --> 00:52:44,050
And if there's no way of
cutting w up to satisfy that,

899
00:52:44,050 --> 00:52:51,040
or if I can't find any
way to divide w into x, y,

900
00:52:51,040 --> 00:52:53,350
and a rule R goes
to ST which makes

901
00:52:53,350 --> 00:52:56,110
this work, if all
possible ways fail

902
00:52:56,110 --> 00:52:58,300
then you can't get from R to w.

903
00:53:02,230 --> 00:53:05,440
And then you can decide the
original A CFG problem now,

904
00:53:05,440 --> 00:53:07,000
by starting from
the start variable,

905
00:53:07,000 --> 00:53:09,175
instead of just
any old R. You plug

906
00:53:09,175 --> 00:53:11,320
in the start variable for R.

907
00:53:11,320 --> 00:53:19,680
So this algorithm works, and
it can be used to solve A CFG.

908
00:53:19,680 --> 00:53:22,370
But the question is,
is it polynomial?

909
00:53:22,370 --> 00:53:24,890
And it's not.

910
00:53:24,890 --> 00:53:30,020
Because every time you're
doing the recursion,

911
00:53:30,020 --> 00:53:33,960
you're essentially adding
another factor of n.

912
00:53:33,960 --> 00:53:36,720
Because here, as we pointed
out, this is a factor of n.

913
00:53:36,720 --> 00:53:40,020
But that's happening every time
you're doing a recursive level.

914
00:53:40,020 --> 00:53:43,980
And you can imagine, I'm just
doing a very crude analysis

915
00:53:43,980 --> 00:53:47,160
here, depending upon
how you divide w up.

916
00:53:47,160 --> 00:53:48,930
But roughly speaking,
it's going to get

917
00:53:48,930 --> 00:53:51,120
divided in half each
time, so there's

918
00:53:51,120 --> 00:53:53,220
going to be log n levels.

919
00:53:53,220 --> 00:53:56,010
So that means you're
going to be multiplying n

920
00:53:56,010 --> 00:53:58,620
by itself log n
times, or give you

921
00:53:58,620 --> 00:54:02,030
an n to the log n algorithm.

922
00:54:02,030 --> 00:54:04,310
That's not polynomial,
because polynomials

923
00:54:04,310 --> 00:54:07,130
end with a constant,
for some fixed constant.

924
00:54:07,130 --> 00:54:09,830
n to the log n is not
going to be polynomial.

925
00:54:09,830 --> 00:54:13,770
This is not a
polynomial algorithm.

926
00:54:13,770 --> 00:54:17,370
Instead, you're going to have to
do something just a little bit

927
00:54:17,370 --> 00:54:20,180
more clever.

928
00:54:20,180 --> 00:54:22,700
It's going to be
the same basic idea,

929
00:54:22,700 --> 00:54:26,270
but relying on one
little observation.

930
00:54:26,270 --> 00:54:32,410
And the little
observation is that when--

931
00:54:32,410 --> 00:54:35,500
this non-polynomial
implementation

932
00:54:35,500 --> 00:54:40,910
that I've just described
is actually pretty dumb.

933
00:54:40,910 --> 00:54:43,850
Because it's doing a lot
of recomputation of things

934
00:54:43,850 --> 00:54:45,380
that it's already solved.

935
00:54:45,380 --> 00:54:47,770
Why is that?

936
00:54:47,770 --> 00:54:51,900
Because if you
look at the number

937
00:54:51,900 --> 00:54:56,250
of possible different
subproblems here,

938
00:54:56,250 --> 00:55:02,070
once I give you G
and I give you w,

939
00:55:02,070 --> 00:55:08,700
how many different subproblems
of G, S, and T are there?

940
00:55:12,140 --> 00:55:15,950
The number of strings
here, all of those strings

941
00:55:15,950 --> 00:55:17,720
are going to be substrings of w.

942
00:55:17,720 --> 00:55:22,490
There's only roughly n
squared substrings of w.

943
00:55:22,490 --> 00:55:24,050
I'm always going
to be generating,

944
00:55:24,050 --> 00:55:27,140
in a subproblem here,
some substring of w

945
00:55:27,140 --> 00:55:31,360
from some starting
variable in the grammar.

946
00:55:31,360 --> 00:55:33,730
So because there aren't that
many different substrings,

947
00:55:33,730 --> 00:55:37,000
and not that many different
starting variables,

948
00:55:37,000 --> 00:55:39,640
the total number of
possible problems

949
00:55:39,640 --> 00:55:42,130
that this algorithm is going
to be called on to solve

950
00:55:42,130 --> 00:55:44,748
is going to be, in total,
a polynomial number

951
00:55:44,748 --> 00:55:45,790
of different subproblems.

952
00:55:45,790 --> 00:55:47,082
There aren't very many of them.

953
00:55:47,082 --> 00:55:49,840
It's only something
like order of n squared.

954
00:55:49,840 --> 00:55:54,040
So if the algorithm is running
for exponentially long time,

955
00:55:54,040 --> 00:55:57,900
it's solving the same
subproblem over and over again.

956
00:55:57,900 --> 00:56:00,910
That's dumb.

957
00:56:00,910 --> 00:56:04,000
Why don't you just remember
when you solve the subproblem,

958
00:56:04,000 --> 00:56:05,290
so you don't solve it again?

959
00:56:08,480 --> 00:56:12,560
Doing that enhanced
recursion where

960
00:56:12,560 --> 00:56:16,177
you remember the problems
you've already solved,

961
00:56:16,177 --> 00:56:17,510
it's called dynamic programming.

962
00:56:17,510 --> 00:56:22,408
I don't know why it has such
a confusing name like that.

963
00:56:22,408 --> 00:56:24,450
Actually it's called by
several different things.

964
00:56:24,450 --> 00:56:28,070
But anyway, that's known
as dynamic programming.

965
00:56:28,070 --> 00:56:32,400
It's recursion plus the memory.

966
00:56:32,400 --> 00:56:35,810
And here is just
repeating myself.

967
00:56:35,810 --> 00:56:39,920
There are not very many
different substrings,

968
00:56:39,920 --> 00:56:42,957
so every time in your
recursion somewhere,

969
00:56:42,957 --> 00:56:44,790
you're going to be
working with a substring.

970
00:56:44,790 --> 00:56:46,730
So there's not that many
different subproblems

971
00:56:46,730 --> 00:56:48,170
that you can possibly solve.

972
00:56:48,170 --> 00:56:51,560
And just remember when
you solve the subproblem,

973
00:56:51,560 --> 00:56:53,720
and not solve it again.

974
00:56:53,720 --> 00:56:57,470
Let me just show you that
algorithm again here,

975
00:56:57,470 --> 00:56:59,930
with the little modification.

976
00:56:59,930 --> 00:57:03,830
So first of all,
let me give you--

977
00:57:03,830 --> 00:57:05,960
this is the same algorithm
from the previous slide.

978
00:57:05,960 --> 00:57:08,210
I'm just repeating it here
without all the other stuff

979
00:57:08,210 --> 00:57:10,520
so we can look at it directly.

980
00:57:10,520 --> 00:57:15,260
Dividing it into x and y
for each possible rule.

981
00:57:15,260 --> 00:57:15,980
And then recurse.

982
00:57:18,680 --> 00:57:26,460
I'm going to add a little
step 0 beforehand, which

983
00:57:26,460 --> 00:57:32,050
says, if I have G, w, and R,
let me just check first if I've

984
00:57:32,050 --> 00:57:34,400
already solved that one before.

985
00:57:34,400 --> 00:57:37,660
So I have to keep track of
the ones I've already solved.

986
00:57:37,660 --> 00:57:39,160
That's not too bad,
because there's

987
00:57:39,160 --> 00:57:42,760
only order n squared possible
different ones that I

988
00:57:42,760 --> 00:57:44,413
could be called on to solve.

989
00:57:44,413 --> 00:57:46,330
So I'm just going to
have a little table where

990
00:57:46,330 --> 00:57:47,860
I'm going to remember those.

991
00:57:47,860 --> 00:57:51,260
And then every time I get a
new one, I get one to solve,

992
00:57:51,260 --> 00:57:51,760
I'll check.

993
00:57:51,760 --> 00:57:52,990
Is it on that table?

994
00:57:52,990 --> 00:57:54,620
And what's the answer?

995
00:57:54,620 --> 00:57:57,610
So I won't have to rerun those.

996
00:57:57,610 --> 00:58:01,720
I'm going to be basically
pruning that tree so

997
00:58:01,720 --> 00:58:03,950
that it has only a
polynomial number of leaves.

998
00:58:03,950 --> 00:58:05,710
And so the total
size of that tree

999
00:58:05,710 --> 00:58:09,770
now is going to be polynomial.

1000
00:58:09,770 --> 00:58:12,128
And so that's going to yield
a polynomial running time.

1001
00:58:12,128 --> 00:58:13,670
This, by the way,
I only learned this

1002
00:58:13,670 --> 00:58:16,128
myself, I'm sure you guys all
know this who have taken this

1003
00:58:16,128 --> 00:58:19,160
in the algorithms course,
has a special name called

1004
00:58:19,160 --> 00:58:20,900
memoization.

1005
00:58:20,900 --> 00:58:23,030
Not memorization.

1006
00:58:23,030 --> 00:58:26,060
Came from the same root,
I think, but memoization,

1007
00:58:26,060 --> 00:58:29,748
which is somehow remembering
the results of a computation

1008
00:58:29,748 --> 00:58:31,040
so you don't have to repeat it.

1009
00:58:34,833 --> 00:58:36,250
The total number
of calls is going

1010
00:58:36,250 --> 00:58:39,250
to be, at most, n squared, to
this algorithm, because you're

1011
00:58:39,250 --> 00:58:44,800
never going to be redoing
work that you've done already.

1012
00:58:44,800 --> 00:58:49,135
And when you actually have to go
through it, the running time--

1013
00:58:56,880 --> 00:58:59,010
the total amount of time
that you're going to need

1014
00:58:59,010 --> 00:59:00,720
is going to be
polynomial altogether.

1015
00:59:04,610 --> 00:59:08,340
I don't remember what
my check-in is on this.

1016
00:59:08,340 --> 00:59:09,180
Oh yeah.

1017
00:59:09,180 --> 00:59:10,500
This is somehow related.

1018
00:59:10,500 --> 00:59:12,660
And feel free to
ask questions too,

1019
00:59:12,660 --> 00:59:16,110
while you're thinking
about this check-in.

1020
00:59:16,110 --> 00:59:19,170
But the check-in
says here, we've

1021
00:59:19,170 --> 00:59:22,860
solved the A CFG problem
in polynomial time.

1022
00:59:22,860 --> 00:59:28,820
Does that tell us that every
context-free language itself

1023
00:59:28,820 --> 00:59:31,970
is also solvable
in polynomial time?

1024
00:59:31,970 --> 00:59:37,238
Just mull that over, and
please give me an answer to it.

1025
00:59:37,238 --> 00:59:38,780
I hope you do better
on this check-in

1026
00:59:38,780 --> 00:59:41,630
than you did on the last one.

1027
00:59:41,630 --> 00:59:45,760
But anyway, why don't you go
ahead and think about that.

1028
00:59:45,760 --> 00:59:47,510
I can take some questions
in the meantime.

1029
00:59:51,890 --> 00:59:53,065
Somebody is asking here--

1030
00:59:56,020 --> 00:59:59,140
actually, I'm getting
several questions on this.

1031
00:59:59,140 --> 01:00:02,620
Why isn't it order n
cubed or something greater

1032
01:00:02,620 --> 01:00:06,860
than order n squared
because of the variables?

1033
01:00:06,860 --> 01:00:10,150
The variables don't depend on n.

1034
01:00:10,150 --> 01:00:18,260
When you're given-- well,
actually that's not true.

1035
01:00:18,260 --> 01:00:19,630
No.

1036
01:00:19,630 --> 01:00:20,840
You are right.

1037
01:00:23,870 --> 01:00:28,180
Because the grammar
is part of the input.

1038
01:00:28,180 --> 01:00:31,440
So you might have as many
as n different variables

1039
01:00:31,440 --> 01:00:33,330
in the given grammar.

1040
01:00:33,330 --> 01:00:34,980
So you are right.

1041
01:00:34,980 --> 01:00:40,470
There is potentially--
the grammar might

1042
01:00:40,470 --> 01:00:44,040
be half the size of the input,
and the input to the grammar w

1043
01:00:44,040 --> 01:00:45,718
might be half the
size of the input.

1044
01:00:45,718 --> 01:00:47,760
So I didn't think about
that, but you're correct.

1045
01:00:47,760 --> 01:00:52,290
There are potentially
different numbers

1046
01:00:52,290 --> 01:00:54,100
of variables in
different grammars,

1047
01:00:54,100 --> 01:00:56,550
so you have to add an
extra factor, which

1048
01:00:56,550 --> 01:00:59,010
would be at most the
size of the input,

1049
01:00:59,010 --> 01:01:01,750
because that's as many variables
as you could possibly have.

1050
01:01:01,750 --> 01:01:08,970
So it really should be, I think,
order n cubed to take that

1051
01:01:08,970 --> 01:01:10,440
into account as well.

1052
01:01:10,440 --> 01:01:12,420
Plus all of the work
that needs to happen

1053
01:01:12,420 --> 01:01:14,478
in terms of dividing things up.

1054
01:01:14,478 --> 01:01:16,020
On a one-tape Turing
machine, there's

1055
01:01:16,020 --> 01:01:18,750
going to be some extra
work just to carry out

1056
01:01:18,750 --> 01:01:22,380
some of these individual steps,
because with a single tape

1057
01:01:22,380 --> 01:01:24,030
things are sometimes
a little awkward.

1058
01:01:24,030 --> 01:01:25,200
I think the total
running time is

1059
01:01:25,200 --> 01:01:26,783
going to end up being
something like n

1060
01:01:26,783 --> 01:01:29,190
to the 4th or into the 5th
on a one-tape Turing machine.

1061
01:01:32,670 --> 01:01:33,720
But that's a good point.

1062
01:01:38,070 --> 01:01:41,250
Somebody's saying, how can we
be storing n squared strings

1063
01:01:41,250 --> 01:01:42,570
in finite time?

1064
01:01:42,570 --> 01:01:44,160
I'm not saying finite time.

1065
01:01:44,160 --> 01:01:45,390
We have polynomial time.

1066
01:01:45,390 --> 01:01:48,870
Every stage of this
algorithm is allowed

1067
01:01:48,870 --> 01:01:50,430
to run for polynomially
many steps.

1068
01:01:50,430 --> 01:01:52,260
As long as it's
clearly polynomial,

1069
01:01:52,260 --> 01:01:54,195
we can just write that
down as a single stage.

1070
01:01:58,910 --> 01:02:01,600
Part two should say--

1071
01:02:01,600 --> 01:02:03,070
oh.

1072
01:02:03,070 --> 01:02:03,920
There's a typo here.

1073
01:02:03,920 --> 01:02:06,055
So use D. Thank you.

1074
01:02:06,055 --> 01:02:07,150
That is a typo.

1075
01:02:14,013 --> 01:02:16,180
I'm afraid if I change it
on my original slide here,

1076
01:02:16,180 --> 01:02:18,640
things will break in
some horrible way.

1077
01:02:18,640 --> 01:02:19,630
Let's just see.

1078
01:02:19,630 --> 01:02:22,290
Did I completely wreck my slide?

1079
01:02:22,290 --> 01:02:24,530
No, that's good.

1080
01:02:24,530 --> 01:02:25,520
Yeah, thank you.

1081
01:02:25,520 --> 01:02:26,120
Good point.

1082
01:02:33,080 --> 01:02:35,100
Oops.

1083
01:02:35,100 --> 01:02:38,840
OK, how's our check-in doing?

1084
01:02:38,840 --> 01:02:42,620
I think you're just
about all done.

1085
01:02:42,620 --> 01:02:44,030
Spent a lot of time on this.

1086
01:02:44,030 --> 01:02:44,795
End polling.

1087
01:02:51,460 --> 01:02:54,890
As you may remember from the
first half of the course--

1088
01:02:54,890 --> 01:02:57,640
so the answer is A, indeed.

1089
01:02:57,640 --> 01:03:01,930
Remember that we showed
A CFG is decidable,

1090
01:03:01,930 --> 01:03:03,910
and therefore each
context-free language

1091
01:03:03,910 --> 01:03:06,970
itself is decidable,
just because you

1092
01:03:06,970 --> 01:03:11,440
can plug in a specific grammar
into the A CFG problem.

1093
01:03:11,440 --> 01:03:15,370
The very same
reasoning works here.

1094
01:03:15,370 --> 01:03:19,000
If you have a context-free
language, it has a grammar.

1095
01:03:19,000 --> 01:03:22,240
You can plug that grammar
into the A CFG problem.

1096
01:03:22,240 --> 01:03:24,130
And then, that's
polynomial time,

1097
01:03:24,130 --> 01:03:25,900
you're going to get
a polynomial time

1098
01:03:25,900 --> 01:03:30,700
algorithm for that language.

1099
01:03:30,700 --> 01:03:32,665
Good to review that.

1100
01:03:32,665 --> 01:03:35,290
It's the same thing, same
argument we used before.

1101
01:03:42,327 --> 01:03:44,160
I don't want to spend
a lot of time on this.

1102
01:03:44,160 --> 01:03:46,820
There's another way of looking
at dynamic programming.

1103
01:03:46,820 --> 01:03:50,600
We'll talk about this again
maybe in a lecture, probably

1104
01:03:50,600 --> 01:03:55,220
next lecture, just because I you
have a homework problem on it.

1105
01:03:55,220 --> 01:03:57,358
If you've seen dynamic
programming before,

1106
01:03:57,358 --> 01:03:58,400
this is going to be easy.

1107
01:03:58,400 --> 01:04:00,400
If you haven't seen it
before, it's going to be,

1108
01:04:00,400 --> 01:04:02,450
I think, probably a
little challenging.

1109
01:04:06,340 --> 01:04:08,740
Another way of looking
at dynamic programming

1110
01:04:08,740 --> 01:04:11,950
is the so-called bottom-up
version of dynamic programming.

1111
01:04:11,950 --> 01:04:14,770
And what that would
mean is, you solve

1112
01:04:14,770 --> 01:04:17,080
all of the subproblems first.

1113
01:04:17,080 --> 01:04:19,540
You solve all the smaller
subproblems before you

1114
01:04:19,540 --> 01:04:21,100
solve the larger subproblems.

1115
01:04:25,420 --> 01:04:26,505
It's here on the slide.

1116
01:04:26,505 --> 01:04:28,130
I'm not sure I want
to talk it through.

1117
01:04:28,130 --> 01:04:33,880
But basically, you solve
the subproblems here

1118
01:04:33,880 --> 01:04:39,700
where, start with
strings of length 1,

1119
01:04:39,700 --> 01:04:42,370
and then from that you
build up to subproblems

1120
01:04:42,370 --> 01:04:45,610
with the substrings are
of length 2, and then 3,

1121
01:04:45,610 --> 01:04:46,520
and so on.

1122
01:04:46,520 --> 01:04:50,500
And each of those only relies
on the smaller previously solved

1123
01:04:50,500 --> 01:04:51,550
subproblems.

1124
01:04:51,550 --> 01:04:54,850
So you can, kind of
in a systematic way,

1125
01:04:54,850 --> 01:04:59,260
solve all the larger and
larger subproblems for larger

1126
01:04:59,260 --> 01:05:02,410
and larger substrings.

1127
01:05:02,410 --> 01:05:04,300
That gives kind of a
different perspective

1128
01:05:04,300 --> 01:05:05,860
on dynamic programming.

1129
01:05:05,860 --> 01:05:08,710
And for different
problems, sometimes it's

1130
01:05:08,710 --> 01:05:11,920
better to think about either
this sort of top-down recursion

1131
01:05:11,920 --> 01:05:16,570
based process, or the
bottom-up process that I'm

1132
01:05:16,570 --> 01:05:18,400
describing here.

1133
01:05:18,400 --> 01:05:20,290
They're really
completely equivalent.

1134
01:05:23,600 --> 01:05:27,780
The version that's described
for this particular algorithm,

1135
01:05:27,780 --> 01:05:29,600
which appears in the
textbook, is actually

1136
01:05:29,600 --> 01:05:31,290
the bottom-up algorithm.

1137
01:05:31,290 --> 01:05:33,990
So you shouldn't be confused
if you see something there

1138
01:05:33,990 --> 01:05:35,760
which looks somewhat different.

1139
01:05:38,300 --> 01:05:43,460
You basically solve all
possible subproblems,

1140
01:05:43,460 --> 01:05:45,890
basically filling out a table.

1141
01:05:45,890 --> 01:05:49,850
Let me not say anything
more about that here,

1142
01:05:49,850 --> 01:05:51,890
since we're running a
little short on time.

1143
01:05:54,505 --> 01:05:55,880
There are really
two perspectives

1144
01:05:55,880 --> 01:05:59,230
on dynamic programming.

1145
01:05:59,230 --> 01:06:04,210
So moving on from there,
let's shift gears.

1146
01:06:06,900 --> 01:06:12,030
Leave context-free languages
and dynamic programming behind.

1147
01:06:12,030 --> 01:06:18,530
And so I'm moving toward
understanding P and NP.

1148
01:06:18,530 --> 01:06:23,840
And for that, we will
introduce a new problem called

1149
01:06:23,840 --> 01:06:25,190
the satisfiability problem.

1150
01:06:25,190 --> 01:06:29,780
And that's one we're going
to spend a lot of time on.

1151
01:06:29,780 --> 01:06:34,040
If you tuned out a little bit
during the dynamic programming

1152
01:06:34,040 --> 01:06:37,790
discussion, time to
get back on board.

1153
01:06:41,990 --> 01:06:44,360
The satisfiability
problem is going

1154
01:06:44,360 --> 01:06:50,650
to be a computational
problem that we're

1155
01:06:50,650 --> 01:06:51,730
going to be working on.

1156
01:06:51,730 --> 01:06:54,890
And it has to do
with Boolean formula.

1157
01:06:54,890 --> 01:06:58,510
So these are expressions,
like arithmetical formula,

1158
01:06:58,510 --> 01:07:02,980
like x plus y times
z, but instead

1159
01:07:02,980 --> 01:07:07,210
of using numerical
variables, we're

1160
01:07:07,210 --> 01:07:09,910
going to be using Boolean
variables that take

1161
01:07:09,910 --> 01:07:11,860
on Boolean values, true, false.

1162
01:07:11,860 --> 01:07:14,395
Or sometimes
represented by 1 and 0.

1163
01:07:18,660 --> 01:07:20,670
The operators that
we're going to be using

1164
01:07:20,670 --> 01:07:24,210
are going to be the and,
or, and negation operations.

1165
01:07:24,210 --> 01:07:25,110
And, or, not.

1166
01:07:28,590 --> 01:07:33,230
I'm going to say such a
formula, such a Boolean formula,

1167
01:07:33,230 --> 01:07:34,820
we're going to call
it satisfiable--

1168
01:07:34,820 --> 01:07:36,920
we'll do an example
in a second--

1169
01:07:36,920 --> 01:07:42,440
if that formula value
evaluates to true

1170
01:07:42,440 --> 01:07:47,820
if you make some assignment
of values to its variables.

1171
01:07:47,820 --> 01:07:51,980
So just like arithmetical
formula will have some value

1172
01:07:51,980 --> 01:07:55,970
if you plug in values
for the variables,

1173
01:07:55,970 --> 01:07:58,190
Boolean formula is
going to have some value

1174
01:07:58,190 --> 01:08:01,130
if you plug in Boolean
values for its variables.

1175
01:08:01,130 --> 01:08:02,810
And I want to know,
is there some way

1176
01:08:02,810 --> 01:08:05,480
to plug in values which makes
the whole thing evaluate

1177
01:08:05,480 --> 01:08:07,070
to true.

1178
01:08:07,070 --> 01:08:08,990
The formula itself
is going to evaluate

1179
01:08:08,990 --> 01:08:12,380
to either true or false, and
I wanted to evaluate to true.

1180
01:08:17,840 --> 01:08:19,069
Here is our example.

1181
01:08:19,069 --> 01:08:24,939
Let's take the formula,
phi, which is x or y,

1182
01:08:24,939 --> 01:08:27,760
and x complement--

1183
01:08:27,760 --> 01:08:30,340
or, not x or not y.

1184
01:08:30,340 --> 01:08:35,109
So the notation x with a
bar over it, x complement,

1185
01:08:35,109 --> 01:08:39,939
is just x bar, not x.

1186
01:08:39,939 --> 01:08:42,100
It's just the way
if you're familiar

1187
01:08:42,100 --> 01:08:47,140
with the other notation,
the not operation, which

1188
01:08:47,140 --> 01:08:48,949
just inverts 1s and 0s.

1189
01:08:52,330 --> 01:08:55,149
We're going to write it with
a bar instead of the negation

1190
01:08:55,149 --> 01:08:56,649
symbol.

1191
01:08:56,649 --> 01:08:59,229
I'm assuming that you've
all seen Boolean algebra,

1192
01:08:59,229 --> 01:09:04,660
Boolean arithmetic before,
where the and operation is only

1193
01:09:04,660 --> 01:09:07,660
true if both inputs are true.

1194
01:09:07,660 --> 01:09:10,060
These are going to be binary
and operations and binary

1195
01:09:10,060 --> 01:09:12,160
or operations.

1196
01:09:12,160 --> 01:09:15,010
Or is going to be true
if either input is true.

1197
01:09:15,010 --> 01:09:19,260
And not is true if its
single input is false.

1198
01:09:19,260 --> 01:09:22,380
Oops, just looked at the answer.

1199
01:09:22,380 --> 01:09:25,890
Here I want to know, for
this Boolean formula,

1200
01:09:25,890 --> 01:09:28,410
here is it satisfiable.

1201
01:09:28,410 --> 01:09:32,710
Is there some way to assign
values to the variables

1202
01:09:32,710 --> 01:09:37,479
to make this formula
evaluate to true?

1203
01:09:37,479 --> 01:09:39,450
So for example, let's
just try things.

1204
01:09:39,450 --> 01:09:42,100
Let's make x and y both true.

1205
01:09:42,100 --> 01:09:44,800
So x is true and y is true.

1206
01:09:44,800 --> 01:09:47,910
So x or y, well that's
good, that's true.

1207
01:09:47,910 --> 01:09:52,229
But now we have to do an and, so
we need both sides to be true.

1208
01:09:52,229 --> 01:09:55,900
So now we have x complement--
well we said we said x is true,

1209
01:09:55,900 --> 01:09:58,830
so x complement is false,
y complement is false.

1210
01:09:58,830 --> 01:10:01,270
False or false is false.

1211
01:10:01,270 --> 01:10:03,060
So now we have a true and false.

1212
01:10:03,060 --> 01:10:04,780
That's going to be false.

1213
01:10:04,780 --> 01:10:06,660
We did not find a
satisfying assignment.

1214
01:10:06,660 --> 01:10:08,020
But maybe there's another one.

1215
01:10:08,020 --> 01:10:09,270
And in fact, there is.

1216
01:10:09,270 --> 01:10:13,530
If you make x true and y
false, then both of these parts

1217
01:10:13,530 --> 01:10:16,920
will evaluate to true, and
then you'll have true and true.

1218
01:10:16,920 --> 01:10:19,380
So we found a satisfying
assignment to this formula.

1219
01:10:19,380 --> 01:10:22,570
It is, in fact, satisfiable.

1220
01:10:22,570 --> 01:10:25,470
So if you say x is
1 and y is 0, yes.

1221
01:10:25,470 --> 01:10:28,530
This is satisfiable.

1222
01:10:28,530 --> 01:10:31,010
Now the problem of testing
for a Boolean formula,

1223
01:10:31,010 --> 01:10:34,910
if it is satisfiable, is
going to be the SAT language.

1224
01:10:34,910 --> 01:10:35,550
It's a set.

1225
01:10:35,550 --> 01:10:39,660
It's a collection of
satisfiable Boolean formula.

1226
01:10:39,660 --> 01:10:41,940
And testing whether
you're in SAT or not

1227
01:10:41,940 --> 01:10:46,380
is going to be an important
computational problem.

1228
01:10:49,580 --> 01:10:56,990
There was an amazing
theorem which

1229
01:10:56,990 --> 01:10:59,870
really got this
whole subject going,

1230
01:10:59,870 --> 01:11:05,720
discovered independently by
Steve Cook in North America

1231
01:11:05,720 --> 01:11:10,700
and Leonid Levin in the former
Soviet Union, almost exactly at

1232
01:11:10,700 --> 01:11:14,690
the same time, which made a
connection between this one

1233
01:11:14,690 --> 01:11:18,860
problem and all of
the problems in NP.

1234
01:11:18,860 --> 01:11:22,580
By solving this one
satisfiability problem

1235
01:11:22,580 --> 01:11:28,220
in polynomial
time, it allows you

1236
01:11:28,220 --> 01:11:33,010
to solve all of the problems
in NP in polynomial time.

1237
01:11:33,010 --> 01:11:36,240
So if you could solve
this problem set in P,

1238
01:11:36,240 --> 01:11:41,800
then Hamiltonian path
is also solvable in P.

1239
01:11:41,800 --> 01:11:44,300
If you step back and think about
that, it's kind of amazing.

1240
01:11:46,820 --> 01:11:49,010
And the method that
we're going to introduce

1241
01:11:49,010 --> 01:11:52,940
is called polynomial
time reducibility.

1242
01:11:52,940 --> 01:11:54,590
Let's do a quick
check-in on this.

1243
01:11:54,590 --> 01:11:57,620
This should be an easy one.

1244
01:11:57,620 --> 01:12:00,530
Why don't you just think
about, is SAT, the SAT problem

1245
01:12:00,530 --> 01:12:02,420
that we just described
here, is that in NP?

1246
01:12:06,857 --> 01:12:07,440
Three seconds.

1247
01:12:10,860 --> 01:12:12,570
You all there?

1248
01:12:12,570 --> 01:12:13,500
OK.

1249
01:12:13,500 --> 01:12:14,220
Ending polling.

1250
01:12:21,240 --> 01:12:24,240
Hopefully you're getting
the intuition for NP

1251
01:12:24,240 --> 01:12:28,320
that these are the
problems-- to be in NP means

1252
01:12:28,320 --> 01:12:30,810
that when you're a
member of the language,

1253
01:12:30,810 --> 01:12:34,320
there's a short proof or a
short certificate of membership.

1254
01:12:34,320 --> 01:12:36,240
And in this case,
the short certificate

1255
01:12:36,240 --> 01:12:39,340
that the formula is
satisfiable is the assignment,

1256
01:12:39,340 --> 01:12:43,890
which makes it true, also called
the satisfying assignment.

1257
01:12:43,890 --> 01:12:47,135
So yes, this is an NP language,
language that's in NP.

1258
01:12:51,550 --> 01:12:54,050
There are a lot of things that
we don't know in the subject,

1259
01:12:54,050 --> 01:12:56,090
but this isn't one of them.

1260
01:12:56,090 --> 01:12:59,060
We do know that SAT is in NP.

1261
01:12:59,060 --> 01:13:02,570
So let's talk about
our method for showing

1262
01:13:02,570 --> 01:13:12,150
this remarkable
fact that, if you

1263
01:13:12,150 --> 01:13:15,750
can solve SAT in polynomial
time, then all of NP

1264
01:13:15,750 --> 01:13:17,430
is solvable in polynomial time.

1265
01:13:17,430 --> 01:13:21,450
And it uses this notion of
polynomial time reducibility,

1266
01:13:21,450 --> 01:13:23,940
which is just like
mapping reducibility

1267
01:13:23,940 --> 01:13:26,880
that I hope you've all
grown to know and love

1268
01:13:26,880 --> 01:13:28,620
in the first half of the course.

1269
01:13:28,620 --> 01:13:33,000
But now, the reduction has to
operate in polynomial time.

1270
01:13:33,000 --> 01:13:38,130
So it's the same picture that
we had before, mapping A to B,

1271
01:13:38,130 --> 01:13:40,290
transforming A questions
to B questions.

1272
01:13:40,290 --> 01:13:44,200
But now the transformation
has to operate quickly.

1273
01:13:44,200 --> 01:13:49,530
And we get that if A is
polynomial time reducible to B,

1274
01:13:49,530 --> 01:13:54,000
and B is polynomial time, then
A is also polynomial time.

1275
01:13:54,000 --> 01:13:55,740
Same pattern as before.

1276
01:13:55,740 --> 01:14:01,320
If A is reducible to B and
B is easy, then A is easy.

1277
01:14:01,320 --> 01:14:06,240
Here is kind of the
essence of the idea,

1278
01:14:06,240 --> 01:14:11,280
or at least the outline of the
idea of this Cook and Levin

1279
01:14:11,280 --> 01:14:12,240
theorem.

1280
01:14:12,240 --> 01:14:15,720
That if satisfiable is in
P, then everything in NP

1281
01:14:15,720 --> 01:14:18,480
can be done in P. Which
is because, we will

1282
01:14:18,480 --> 01:14:23,250
show that all problems
in NP are polynomial time

1283
01:14:23,250 --> 01:14:26,640
reducible to SAT.

1284
01:14:26,640 --> 01:14:28,440
That's the amazing fact.

1285
01:14:28,440 --> 01:14:31,830
So therefore, if you
can bring SAT down

1286
01:14:31,830 --> 01:14:35,700
into P by using this reduction,
it brings everything else along

1287
01:14:35,700 --> 01:14:38,670
with it, everything
is reducible to SAT.

1288
01:14:38,670 --> 01:14:42,090
So we just have to
show how to do that.

1289
01:14:42,090 --> 01:14:44,280
There is an analogy that
we had in the first half

1290
01:14:44,280 --> 01:14:46,238
of the course, in one of
our homework problems,

1291
01:14:46,238 --> 01:14:47,520
if you may remember.

1292
01:14:47,520 --> 01:14:53,640
We showed that A TM has
the very special property

1293
01:14:53,640 --> 01:14:56,580
that all Turing
recognizable languages are

1294
01:14:56,580 --> 01:14:59,390
mapping reducible to it.

1295
01:14:59,390 --> 01:15:07,660
I think that was problem 2,
or 2a, either in problem set 3

1296
01:15:07,660 --> 01:15:09,670
or problem set 2.

1297
01:15:09,670 --> 01:15:12,340
I think problem set 3.

1298
01:15:12,340 --> 01:15:15,070
That every Turing
recognizable language

1299
01:15:15,070 --> 01:15:17,590
is polynomial time
reducible to A TM.

1300
01:15:17,590 --> 01:15:20,440
And so, very similar picture.

1301
01:15:20,440 --> 01:15:22,480
And there's a lot
of analogies here

1302
01:15:22,480 --> 01:15:26,920
that you can draw between
the computability section

1303
01:15:26,920 --> 01:15:29,770
and the complexity section.

1304
01:15:29,770 --> 01:15:32,330
With that, I know we're
just about out of time.

1305
01:15:32,330 --> 01:15:38,740
So let's just quick review
of what we've done here.

1306
01:15:38,740 --> 01:15:42,560
I will stick around for
questions for a while.

1307
01:15:42,560 --> 01:15:45,495
Is there a--

1308
01:15:45,495 --> 01:15:46,620
OK, that's a good question.

1309
01:15:46,620 --> 01:15:52,500
Is there a regular
reduction analogy version

1310
01:15:52,500 --> 01:15:54,780
to mapping reducibility?

1311
01:15:54,780 --> 01:16:00,550
We had the general reduction
for the computability section.

1312
01:16:00,550 --> 01:16:03,490
And we had the mapping reduction
for the computability section.

1313
01:16:03,490 --> 01:16:06,910
Here, we're only going
to be focusing now

1314
01:16:06,910 --> 01:16:08,830
on the mapping reduction.

1315
01:16:08,830 --> 01:16:12,220
So polynomial time
reduction is, by assumption,

1316
01:16:12,220 --> 01:16:13,660
going to be a mapping reduction.

1317
01:16:13,660 --> 01:16:18,460
Yes, there is a general
polynomial time reduction

1318
01:16:18,460 --> 01:16:19,225
notion as well.

1319
01:16:22,450 --> 01:16:24,650
This is not required,
but if you are

1320
01:16:24,650 --> 01:16:27,530
curious about the
general reduction

1321
01:16:27,530 --> 01:16:29,570
and how to precisely
formulate that,

1322
01:16:29,570 --> 01:16:34,850
it actually appears in chapter
6 under Turing reducibility.

1323
01:16:34,850 --> 01:16:37,520
That's the general notion
of reducibility spelled out

1324
01:16:37,520 --> 01:16:38,570
in a formal way.

1325
01:16:38,570 --> 01:16:42,180
And there's polynomial time
Turing reducibility as well.

1326
01:16:42,180 --> 01:16:45,365
We're not going to talk
about it in this course.

1327
01:16:49,440 --> 01:16:52,280
Other questions?

1328
01:16:52,280 --> 01:16:56,720
Does NP correspond
exactly to verification

1329
01:16:56,720 --> 01:16:59,900
in polynomial time?

1330
01:16:59,900 --> 01:17:02,120
For me to answer that
as a precise question,

1331
01:17:02,120 --> 01:17:05,180
we have to have a precise
definition of verification.

1332
01:17:05,180 --> 01:17:11,420
But with the right
definition, the answer is yes.

1333
01:17:11,420 --> 01:17:17,690
So you can define a verifier
as a polynomial time algorithm

1334
01:17:17,690 --> 01:17:20,870
that gives a
certificate, that takes

1335
01:17:20,870 --> 01:17:23,150
a certificate and an
input to the language,

1336
01:17:23,150 --> 01:17:30,890
and will accept if
that certificate is

1337
01:17:30,890 --> 01:17:33,710
a valid certificate
for that input.

1338
01:17:33,710 --> 01:17:36,050
This is actually
discussed in chapter,

1339
01:17:36,050 --> 01:17:40,125
I think, 9 of the text.

1340
01:17:47,950 --> 01:17:50,230
Now I'm forgetting already,
what's where in the book.

1341
01:17:50,230 --> 01:17:54,400
But yeah, you can think of
NP in terms of verification

1342
01:17:54,400 --> 01:17:55,540
as the definition.

1343
01:17:59,840 --> 01:18:04,550
Is proving P equal NP the same
as proving that a polynom--

1344
01:18:04,550 --> 01:18:06,500
actually, I can even make the--

1345
01:18:06,500 --> 01:18:11,830
if you want, you can
post public comments too.

1346
01:18:11,830 --> 01:18:14,140
I should have done
that in other cases.

1347
01:18:14,140 --> 01:18:16,750
Is proving P equal NP
the same as proving

1348
01:18:16,750 --> 01:18:19,100
that a polynomial
time non-deterministic

1349
01:18:19,100 --> 01:18:23,410
Turing machine N has a
polynomial time deterministic?

1350
01:18:26,115 --> 01:18:26,615
Yeah.

1351
01:18:29,660 --> 01:18:36,430
Suppose we prove that P equals
NP, which is the minority view,

1352
01:18:36,430 --> 01:18:40,695
I would say, the
small minority view.

1353
01:18:40,695 --> 01:18:42,070
There are some
people who believe

1354
01:18:42,070 --> 01:18:46,960
that that is entirely possible,
and might even be the case.

1355
01:18:46,960 --> 01:18:50,020
But that's a very small group.

1356
01:18:50,020 --> 01:18:52,720
But yeah, if you
prove P equal NP,

1357
01:18:52,720 --> 01:18:55,960
that's the same as saying
that every non-deterministic

1358
01:18:55,960 --> 01:18:58,630
polynomial time Turing
machine is going

1359
01:18:58,630 --> 01:19:02,350
to have a companion
deterministic polynomial

1360
01:19:02,350 --> 01:19:05,020
time Turing machine which
does the same language.

1361
01:19:05,020 --> 01:19:07,530
That's exactly what it means.

1362
01:19:07,530 --> 01:19:09,890
Bye bye, everybody.