1
00:00:00 --> 00:00:00



2
00:00:00 --> 00:00:02
ANNOUNCER: Open content is
provided under a creative

3
00:00:02 --> 00:00:03
commons license.


4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:10
offer high-quality educational
resources for free.

6
00:00:10 --> 00:00:13
To make a donation, or view
additional materials from

7
00:00:13 --> 00:00:17
hundreds of MIT courses,
visit MIT OpenCourseWare

8
00:00:17 --> 00:00:19
at ocw.mit.edu.


9
00:00:19 --> 00:00:23
PROFESSOR ERIC GRIMSON: Last
time, we ended up, we sort of

10
00:00:23 --> 00:00:25
did this tag team thing,
Professor Guttag did the first

11
00:00:25 --> 00:00:27
half, I did the second half of
the lecture, and the second

12
00:00:27 --> 00:00:30
half of the lecture, we started
talking about complexity.

13
00:00:30 --> 00:00:32
Efficiency.


14
00:00:32 --> 00:00:33
Orders of growth.


15
00:00:33 --> 00:00:34
And that's what we're going
to spend today on, is

16
00:00:34 --> 00:00:36
talking about that topic.


17
00:00:36 --> 00:00:37
I'm going to use it to
build over the next

18
00:00:37 --> 00:00:39
couple of lectures.


19
00:00:39 --> 00:00:41
I want to remind you that we
were talking at a fairly high

20
00:00:41 --> 00:00:42
level about complexity.


21
00:00:42 --> 00:00:45
We're going to get down into
the weeds in a second here.

22
00:00:45 --> 00:00:47
But the things we were trying
to stress were that it's an

23
00:00:47 --> 00:00:51
important design decision, when
you are coming up with a piece

24
00:00:51 --> 00:00:54
of code, as to what kind of
efficiency your code has.

25
00:00:54 --> 00:00:58
And the second thing that we
talked about is this idea that

26
00:00:58 --> 00:01:01
we want you to in fact learn
how to relate a choice you make

27
00:01:01 --> 00:01:05
about a piece of code to what
the efficiency is going to be.

28
00:01:05 --> 00:01:07
So in fact, over the next
thirty or forty minutes, we're

29
00:01:07 --> 00:01:10
going to show you a set of
examples of sort of canonical

30
00:01:10 --> 00:01:13
algorithms, and the different
classes of complexity.

31
00:01:13 --> 00:01:15
Because one of the things that
you want to do as a good

32
00:01:15 --> 00:01:20
designer is to basically map a
new problem into

33
00:01:20 --> 00:01:20
a known domain.


34
00:01:20 --> 00:01:23
You want to take a new
problem and say, what

35
00:01:23 --> 00:01:24
does this most look like?


36
00:01:24 --> 00:01:27
What is the class of algorithm
that's-- that probably applies

37
00:01:27 --> 00:01:30
to this, and how do I pull
something out of that, if you

38
00:01:30 --> 00:01:33
like, a briefcase of possible
algorithms to solve?

39
00:01:33 --> 00:01:36
All right, having said that,
let's do some examples.

40
00:01:36 --> 00:01:39
I'm going to show you a
sequence of algorithms,

41
00:01:39 --> 00:01:41
they're mostly simple
algorithms, that's OK.

42
00:01:41 --> 00:01:44
But I want you to take away
from this how we reason

43
00:01:44 --> 00:01:46
about the complexity
of these algorithms.

44
00:01:46 --> 00:01:47
And I'll remind you, we
said we're going to

45
00:01:47 --> 00:01:48
mostly talk about time.


46
00:01:48 --> 00:01:51
We're going to be counting
the number of basic steps it

47
00:01:51 --> 00:01:53
takes to solve the problem.


48
00:01:53 --> 00:01:55
So here's the first
example I want to do.

49
00:01:55 --> 00:01:59
I'm going to write a function
to compute integer power

50
00:01:59 --> 00:02:02
exponents. a to the b
where b is an integer.

51
00:02:02 --> 00:02:05
And I'm going to do it only
using multiplication and

52
00:02:05 --> 00:02:07
addition and some simple tests.


53
00:02:07 --> 00:02:07
All right?


54
00:02:07 --> 00:02:10
And yeah, I know it comes
built in, that's OK, what

55
00:02:10 --> 00:02:12
we want to do is use it as
an example to look at it.

56
00:02:12 --> 00:02:20
So I'm going to build
something that's going to do

57
00:02:20 --> 00:02:22
iterative exponentiation.


58
00:02:22 --> 00:02:23
OK?


59
00:02:23 --> 00:02:27
And in fact, if you look at the
code up here, and it's on your

60
00:02:27 --> 00:02:30
handout, the very first one, x
1, right here-- if I could ask

61
00:02:30 --> 00:02:33
you to look at it-- is a
piece of code to do it.

62
00:02:33 --> 00:02:35
And I'm less interested in the
code than how we're going

63
00:02:35 --> 00:02:37
to analyze it, but let's
look at it for a second.

64
00:02:37 --> 00:02:42
All right, you can see that
this little piece of code,

65
00:02:42 --> 00:02:45
it's got a loop in there,
and what's it doing?

66
00:02:45 --> 00:02:48
It's basically cycling through
the loop, multiplying

67
00:02:48 --> 00:02:49
by a each time.


68
00:02:49 --> 00:02:51
So first time through the
loop, the answer is 1.

69
00:02:51 --> 00:02:53
Second time it-- sorry, as it
enters the loop, at the time it

70
00:02:53 --> 00:02:56
enter-- exits, the answer is a.


71
00:02:56 --> 00:02:58
Next time through the loop
it goes to a squared.

72
00:02:58 --> 00:02:59
Next time through the
loop it goes to a cubed.

73
00:02:59 --> 00:03:02
And it's just gathering
together the multiplications

74
00:03:02 --> 00:03:05
while counting
down the exponent.

75
00:03:05 --> 00:03:07
And you can see it when we get
down to the end test here,

76
00:03:07 --> 00:03:08
we're going to pop out of there
and we're going to

77
00:03:08 --> 00:03:11
return the answer.


78
00:03:11 --> 00:03:13
I could run it, it'll
do the right thing.

79
00:03:13 --> 00:03:16
What I want to think about
though, is, how much

80
00:03:16 --> 00:03:17
time does this take?


81
00:03:17 --> 00:03:22
How many steps does it take
for this function to run?

82
00:03:22 --> 00:03:23
Well, you can kind of
look at it, right?

83
00:03:23 --> 00:03:26
The key part of that
is that WHILE loop.

84
00:03:26 --> 00:03:27
And what are the steps
I want to count?

85
00:03:27 --> 00:03:29
They're inside that loop-- I've
got the wrong glasses so I'm

86
00:03:29 --> 00:03:32
going to have to squint-- and
we've got one test which is a

87
00:03:32 --> 00:03:34
comparison, we've got another
test which is a

88
00:03:34 --> 00:03:37
multiplication-- sorry, not a
test, we've got another step

89
00:03:37 --> 00:03:39
which is a multiplication-- and
another step that

90
00:03:39 --> 00:03:41
is a subtraction.


91
00:03:41 --> 00:03:44
So each time through the
loop, I'm doing three steps.

92
00:03:44 --> 00:03:46
Three basic operations.


93
00:03:46 --> 00:03:48
How many times do I
go through the loop?

94
00:03:48 --> 00:03:51
Somebody help me out.


95
00:03:51 --> 00:03:51
Hand up?


96
00:03:51 --> 00:03:53
Sorry. b times.


97
00:03:53 --> 00:03:53
You're right.


98
00:03:53 --> 00:03:56
Because I keep counting down
each time around-- mostly I've

99
00:03:56 --> 00:03:59
got to unload this candy, which
is driving me nuts, so--

100
00:03:59 --> 00:04:00
thank you. b times.


101
00:04:00 --> 00:04:03
So I've got to go 3 b steps.


102
00:04:03 --> 00:04:05
All right, I've got to go
through the loop b times, I've

103
00:04:05 --> 00:04:07
got three steps each time, and
then when I pop out of the

104
00:04:07 --> 00:04:09
loop, I've got two more steps.


105
00:04:09 --> 00:04:11
All right, I've got the
initiation of answer

106
00:04:11 --> 00:04:12
and the return of it.


107
00:04:12 --> 00:04:19
So I take 2 plus 3 b steps
to go through this loop.

108
00:04:19 --> 00:04:19
OK.


109
00:04:19 --> 00:04:28
So if b is 300, it takes 902
steps. b is 3000, it takes 9002

110
00:04:28 --> 00:04:35
steps. b is 30,000 you get the
point, it takes 90,002 steps.

111
00:04:35 --> 00:04:35
OK.


112
00:04:35 --> 00:04:38
So the point here is, first of
all, I can count these things,

113
00:04:38 --> 00:04:41
but the second thing you can
see is, as the size of the

114
00:04:41 --> 00:04:46
problems get larger, that
additive constant, that 2,

115
00:04:46 --> 00:04:47
really doesn't matter.


116
00:04:47 --> 00:04:47
All right?


117
00:04:47 --> 00:04:51
The difference between 90,000
steps and 90,002 steps, who

118
00:04:51 --> 00:04:53
cares about the 2, right?


119
00:04:53 --> 00:04:55
So, and typically, we're
not going to worry about

120
00:04:55 --> 00:04:56
those additive constants.


121
00:04:56 --> 00:05:00
The second one is, this
multiplicative constant here

122
00:05:00 --> 00:05:04
is 3, in some sense also
isn't all that crucial.

123
00:05:04 --> 00:05:06
Does it really matter to you
whether your code is going to

124
00:05:06 --> 00:05:10
take 300 years or
900 years to run?

125
00:05:10 --> 00:05:11
Problem is, how big
is that number?

126
00:05:11 --> 00:05:14
So we're going to typically
also not worry about the

127
00:05:14 --> 00:05:15
multiplicative constants.


128
00:05:15 --> 00:05:18
This factor here.


129
00:05:18 --> 00:05:21
What we really want to worry
about is, as the size of the

130
00:05:21 --> 00:05:25
problem gets larger, how
does this thing grow?

131
00:05:25 --> 00:05:27
How does the cost go up?


132
00:05:27 --> 00:05:29
And so what we're going to
primarily talk about as a

133
00:05:29 --> 00:05:46
consequence is the rate of
growth as the size of

134
00:05:46 --> 00:05:47
the problem grows.


135
00:05:47 --> 00:05:53
If it was, how much bigger
does this get as I make

136
00:05:53 --> 00:05:55
the problem bigger?


137
00:05:55 --> 00:05:57
And what that really says is,
that we're going to use this

138
00:05:57 --> 00:06:02
using something we're going to
just call asymptotic notation--

139
00:06:02 --> 00:06:08
I love spelling this word--
meaning, as in the limit as

140
00:06:08 --> 00:06:10
the size of the problem
gets bigger, how do I

141
00:06:10 --> 00:06:12
characterize this growth?


142
00:06:12 --> 00:06:13
All right?


143
00:06:13 --> 00:06:15
You'll find out, if you go on
to some of the other classes

144
00:06:15 --> 00:06:18
in course 6, there are a lot
of different ways that

145
00:06:18 --> 00:06:18
you can measure this.


146
00:06:18 --> 00:06:21
The most common one, and the
one we're going to use, is

147
00:06:21 --> 00:06:27
what's often called
big Oh notation.

148
00:06:27 --> 00:06:30
This isn't big Oh as in, oh my
God I'm shocked the markets are

149
00:06:30 --> 00:06:33
collapsing, This is called big
Oh because we use the Greek

150
00:06:33 --> 00:06:35
letter, capital letter,
omicron to represent it.

151
00:06:35 --> 00:06:38
And the way we're going to do
this, or what this represents,

152
00:06:38 --> 00:06:41
let me write this carefully for
you, big Oh notation is

153
00:06:41 --> 00:06:53
basically going to be an upper
limit to the growth of a

154
00:06:53 --> 00:07:00
function as the input grow--
as the input gets large.

155
00:07:00 --> 00:07:05
Now we're going to see a bunch
of examples, and I know

156
00:07:05 --> 00:07:08
those are words, let me
give you an example.

157
00:07:08 --> 00:07:16
I would write f of x is
in big Oh of n squared.

158
00:07:16 --> 00:07:17
And what does it say?


159
00:07:17 --> 00:07:21
It says that function, f of
x, is bounded above, there's

160
00:07:21 --> 00:07:25
an upper limit on it, that
this grows no faster than

161
00:07:25 --> 00:07:28
quadratic in n, n squared.


162
00:07:28 --> 00:07:29
OK.


163
00:07:29 --> 00:07:31
And first of all, you say,
wait a minute, x and n?

164
00:07:31 --> 00:07:34
Well, one of the things we're
going to see is x is the input

165
00:07:34 --> 00:07:38
to this particular problem, n
is a measure of the size of x.

166
00:07:38 --> 00:07:42
And we're going to talk about
how we come up with that.

167
00:07:42 --> 00:07:46
n measures the size of x.


168
00:07:46 --> 00:07:47
OK.


169
00:07:47 --> 00:07:50
In this example I'd use b.


170
00:07:50 --> 00:07:52
All right, as b get-- b is the
thing that's changing as I go

171
00:07:52 --> 00:07:54
along here, but it could be
things like, how many elements

172
00:07:54 --> 00:07:56
are there in a list if the
input is a list, could be how

173
00:07:56 --> 00:07:59
many digits are there in a
string if the input's a string,

174
00:07:59 --> 00:08:01
it could be the size of the
integer as we go along.

175
00:08:01 --> 00:08:02
All right.?


176
00:08:02 --> 00:08:06
And what we want to do then, is
we want to basically come up

177
00:08:06 --> 00:08:10
with, how do we characterize
the growth-- God bless you-- of

178
00:08:10 --> 00:08:12
this problem in terms of this
quadra-- sorry, terms of

179
00:08:12 --> 00:08:14
this exponential growth


180
00:08:14 --> 00:08:16
Now, one last piece of math.


181
00:08:16 --> 00:08:17
I could cheat.


182
00:08:17 --> 00:08:19
I said I just want
an upper bound.

183
00:08:19 --> 00:08:21
I could get a really big
upper bound, this thing

184
00:08:21 --> 00:08:22
grows exponentially.


185
00:08:22 --> 00:08:23
That doesn't help me much.


186
00:08:23 --> 00:08:26
Usually what I want to talk
about is what's the smallest

187
00:08:26 --> 00:08:30
size class in which
this function grows?

188
00:08:30 --> 00:08:32
With all of that, what that
says, is that this we

189
00:08:32 --> 00:08:35
would write is order b.


190
00:08:35 --> 00:08:40
That algorithm is linear.


191
00:08:40 --> 00:08:41
You can see it.


192
00:08:41 --> 00:08:44
I've said the product
was is 2 plus 3 b.

193
00:08:44 --> 00:08:47
As I make b really large,
how does this thing grow?

194
00:08:47 --> 00:08:48
It grows as b.


195
00:08:48 --> 00:08:50
The 3 doesn't matter, it's
just a constant, it's

196
00:08:50 --> 00:08:51
growing linearly.


197
00:08:51 --> 00:08:55
Another way of saying it is, if
I, for example, increase the

198
00:08:55 --> 00:09:00
size of the input by 10, the
amount of time increases by 10.

199
00:09:00 --> 00:09:03
And that's a sign
that it's linear.

200
00:09:03 --> 00:09:04
OK.


201
00:09:04 --> 00:09:05
So there's one quick example.


202
00:09:05 --> 00:09:07
Let's look at another example.


203
00:09:07 --> 00:09:12
If you look at x 2, this one
right here in your handout.

204
00:09:12 --> 00:09:12
OK.


205
00:09:12 --> 00:09:17
This is another way of doing
exponentiation, but this

206
00:09:17 --> 00:09:18
one's a recursive function.


207
00:09:18 --> 00:09:18
All right?


208
00:09:18 --> 00:09:20
So again, let's look at it.


209
00:09:20 --> 00:09:21
What does it say to do?


210
00:09:21 --> 00:09:23
Well, it's basically
saying a similar thing.

211
00:09:23 --> 00:09:25
It says, if I am in the base
case, if b is equal to

212
00:09:25 --> 00:09:27
1, the answer is just a.


213
00:09:27 --> 00:09:30
I could have used if b is equal
to 0, the answer is 1, that

214
00:09:30 --> 00:09:31
would have also worked.


215
00:09:31 --> 00:09:33
Otherwise, what do I say?


216
00:09:33 --> 00:09:36
I say, ah, I'm in a nice
recursive way, a to the b

217
00:09:36 --> 00:09:41
is the same as a times
a to the b minus 1.

218
00:09:41 --> 00:09:43
And I've just reduced that
problem to a simpler version

219
00:09:43 --> 00:09:45
of the same problem.


220
00:09:45 --> 00:09:47
OK, and you can see that this
thing ought to unwrap, it's

221
00:09:47 --> 00:09:49
going to keep extending out
those multiplications until

222
00:09:49 --> 00:09:51
gets down to the base
case, going to collapse

223
00:09:51 --> 00:09:53
them all together.


224
00:09:53 --> 00:09:53
OK.


225
00:09:53 --> 00:09:57
Now I want to know what's
the order of growth here?

226
00:09:57 --> 00:10:00
What's the complexity of this?


227
00:10:00 --> 00:10:01
Well, gee.


228
00:10:01 --> 00:10:04
It looks like it's pretty
straightforward, right?

229
00:10:04 --> 00:10:06
I've got one test there, and
then I've just got one thing

230
00:10:06 --> 00:10:09
to do here, which has got a
subtraction and a

231
00:10:09 --> 00:10:10
multiplication.


232
00:10:10 --> 00:10:14
Oh, but how do I know how
long it takes to do x 2?

233
00:10:14 --> 00:10:16
All right, we were
counting basic steps.

234
00:10:16 --> 00:10:19
We don't know how long
it takes to do x 2.

235
00:10:19 --> 00:10:21
So I'm going to show
you a little trick for

236
00:10:21 --> 00:10:23
figuring that out.


237
00:10:23 --> 00:10:25
And in particular, I'm going to
cheat slightly, I'm going to

238
00:10:25 --> 00:10:28
use a little bit of abusive
mathematics, but I'm going

239
00:10:28 --> 00:10:30
to show you a trick
to figure it out.

240
00:10:30 --> 00:10:36
In the case of a recursive
exponentiator, I'm going to

241
00:10:36 --> 00:10:36
do the following trick.


242
00:10:36 --> 00:10:41
I'm going to let t of b be the
number of steps it takes to

243
00:10:41 --> 00:10:43
solve the problem of size b.


244
00:10:43 --> 00:10:45
OK, and I can figure this out.


245
00:10:45 --> 00:10:49
I've got one test, I've got
a subtraction, I've got a

246
00:10:49 --> 00:10:55
multiplication, that's three
steps, plus whatever number of

247
00:10:55 --> 00:10:58
steps it takes to solve a
problem of size b minus 1.

248
00:10:58 --> 00:11:02
All right, this is what's
called a recurrence relation,

249
00:11:02 --> 00:11:05
there are actually cool
ways to solve them.

250
00:11:05 --> 00:11:07
We can kind of eyeball it.


251
00:11:07 --> 00:11:10
In particular, how would
I write an expression

252
00:11:10 --> 00:11:12
for t of b minus 1?


253
00:11:12 --> 00:11:13
Well the same way.


254
00:11:13 --> 00:11:18
This is 3 plus 3 plus
t of b minus 2.

255
00:11:18 --> 00:11:18
Right?


256
00:11:18 --> 00:11:21
I'm using exactly the same
form to reduce this.

257
00:11:21 --> 00:11:23
You know, you can see
what's going to happen.

258
00:11:23 --> 00:11:26
If I reduce that, it would be
3 plus t of b minus 3, so in

259
00:11:26 --> 00:11:33
general, this is 3 k
plus t of b minus k.

260
00:11:33 --> 00:11:33
OK.


261
00:11:33 --> 00:11:36
I'm just expanding it out.


262
00:11:36 --> 00:11:38
When am I done?


263
00:11:38 --> 00:11:40
How do I stop this?


264
00:11:40 --> 00:11:43
Any suggestions?


265
00:11:43 --> 00:11:45
Don't you hate it when
professors ask questions?

266
00:11:45 --> 00:11:48
Yeah.


267
00:11:48 --> 00:11:52
Actually, I think I want
b minus k equal to 1.

268
00:11:52 --> 00:11:52
Right?


269
00:11:52 --> 00:11:55
When this gets down to t of
1, I'm in the base case.

270
00:11:55 --> 00:12:00
So I'm done when b
minus k equals 1, or

271
00:12:00 --> 00:12:04
k equals b minus 1.


272
00:12:04 --> 00:12:05
Right, that gets me down to the
base case, I'm solving a

273
00:12:05 --> 00:12:08
problem with size 1, and in
that case, I've got two more

274
00:12:08 --> 00:12:13
operations to do, so I plug
this all back in, I-- t of b is

275
00:12:13 --> 00:12:20
I'm going to put k for b minus
1 I get 3 b minus 1 plus t of

276
00:12:20 --> 00:12:25
1, so t of 1 is 2, so this is 3
b minus 1 plus 2,

277
00:12:25 --> 00:12:30
or 3 b minus 1.


278
00:12:30 --> 00:12:30
OK.


279
00:12:30 --> 00:12:34
A whole lot of work to
basically say, again,

280
00:12:34 --> 00:12:38
order b is linear.


281
00:12:38 --> 00:12:41
But that's also nice, it lets
you see how the recursive thing

282
00:12:41 --> 00:12:43
is simply unwrapping but the
complexity in terms of the

283
00:12:43 --> 00:12:46
amount of time it takes
is going to be the same.

284
00:12:46 --> 00:12:48
I owe you a candy.


285
00:12:48 --> 00:12:50
Thank you.


286
00:12:50 --> 00:12:51
OK.


287
00:12:51 --> 00:12:52
At this point, if we
stop, you'll think all

288
00:12:52 --> 00:12:53
algorithms are linear.


289
00:12:53 --> 00:12:55
This is really boring.


290
00:12:55 --> 00:12:56
But they're not.


291
00:12:56 --> 00:12:56
OK?


292
00:12:56 --> 00:13:00
So let me show you another way
I could do exponentiation.

293
00:13:00 --> 00:13:01
Taking an advantage of a trick.


294
00:13:01 --> 00:13:08
I want to solve a to the b.


295
00:13:08 --> 00:13:10
Here's another way
I could do that.

296
00:13:10 --> 00:13:10
OK.


297
00:13:10 --> 00:13:22
If b is even, then a to the
b is the same as a squared

298
00:13:22 --> 00:13:24
all to the b over 2.


299
00:13:24 --> 00:13:26
All right, just move
the 2's around.

300
00:13:26 --> 00:13:27
It's the same thing.


301
00:13:27 --> 00:13:29
You're saying, OK, so what?


302
00:13:29 --> 00:13:31
Well gee, notice.


303
00:13:31 --> 00:13:33
This is a primitive operation.


304
00:13:33 --> 00:13:35
That's a primitive operation.


305
00:13:35 --> 00:13:38
But in one step, I've reduced
this problem in half.

306
00:13:38 --> 00:13:40
I didn't just make it
one smaller, I made

307
00:13:40 --> 00:13:41
it a half smaller.


308
00:13:41 --> 00:13:43
That's a nice deal.


309
00:13:43 --> 00:13:44
OK.


310
00:13:44 --> 00:13:45
But I'm not always going
to have b as even.

311
00:13:45 --> 00:13:47
If b is odd, what do I do?


312
00:13:47 --> 00:13:56
Well, go back to
what I did before.

313
00:13:56 --> 00:13:59
Multiply a by a to
the b minus 1.

314
00:13:59 --> 00:14:00
You know, that's nice, right?


315
00:14:00 --> 00:14:03
Because if b was odd, then b
minus one is even, which means

316
00:14:03 --> 00:14:07
on the next step, I can cut
the problem in half again.

317
00:14:07 --> 00:14:08
OK?


318
00:14:08 --> 00:14:17
All right. x 3, as you can see
right here, does exactly that.

319
00:14:17 --> 00:14:17
OK?


320
00:14:17 --> 00:14:20
You can take a quick look at
it, even with the wrong glasses

321
00:14:20 --> 00:14:22
on, it says if a-- sorry, b is
equal to 1, I'm just

322
00:14:22 --> 00:14:24
going to return a.


323
00:14:24 --> 00:14:26
Otherwise there's that
funky little test.

324
00:14:26 --> 00:14:28
I'll do the remainder
multiplied by 2, because these

325
00:14:28 --> 00:14:31
are integers, that gives me
back an integer, I just check

326
00:14:31 --> 00:14:32
to see if it's equal to b,
that tells me whether

327
00:14:32 --> 00:14:33
it's even or odd.


328
00:14:33 --> 00:14:38
And in the even case, I'd
square, divide by half, call

329
00:14:38 --> 00:14:41
this again: in the odd
case, I go b minus 1

330
00:14:41 --> 00:14:44
and then multiply by a.


331
00:14:44 --> 00:14:46
I'll let you chase it
through, it does work.

332
00:14:46 --> 00:14:48
What I want to look at
is, what's the order

333
00:14:48 --> 00:14:51
of growth here?


334
00:14:51 --> 00:14:52
This is a little
different, right?

335
00:14:52 --> 00:14:54
It's going to take a little
bit more work, so let's

336
00:14:54 --> 00:14:58
see if we can do it.


337
00:14:58 --> 00:15:01
In the b even case, again
I'm going to let t of b

338
00:15:01 --> 00:15:03
be the number of steps
I want to go through.

339
00:15:03 --> 00:15:05
And we can kind of eyeball
this thing, right?

340
00:15:05 --> 00:15:09
If b is even, I've got a test
to see if b is equal to 1,

341
00:15:09 --> 00:15:11
and then I've got to do the
remainder, the multiplication,

342
00:15:11 --> 00:15:14
and the test, I'm up to four.


343
00:15:14 --> 00:15:15
And then in the even
case, I've got to do a

344
00:15:15 --> 00:15:17
square and the divide.


345
00:15:17 --> 00:15:24
So I've got six steps, plus
whatever it takes to solve the

346
00:15:24 --> 00:15:26
problem size b over 2, right?


347
00:15:26 --> 00:15:33
Because that's the recursive
call. b as odd, well I can go

348
00:15:33 --> 00:15:34
through the same kind of thing.


349
00:15:34 --> 00:15:37
I've got the same first four
steps, I've got a check to see

350
00:15:37 --> 00:15:40
is it 1, I got a check to see
if it's even, and then in the

351
00:15:40 --> 00:15:43
odd case, I've got to subtract
1 from b, that's a fifth step,

352
00:15:43 --> 00:15:45
I've got to go off and solve
the recursive problem, and then

353
00:15:45 --> 00:15:49
I'm going to do one more
multiplication, so it's 6 plus,

354
00:15:49 --> 00:15:52
in this case, t of b minus 1.


355
00:15:52 --> 00:15:55
Because it's now solving
a one-smaller problem.

356
00:15:55 --> 00:16:01
On the next step though, this,
we get substituted by that.

357
00:16:01 --> 00:16:03
Right, on the next step, I'm
back in the even case, it's

358
00:16:03 --> 00:16:09
going to take six more
steps, plus t of b minus 1.

359
00:16:09 --> 00:16:12
Oops, sorry about that, over 2.


360
00:16:12 --> 00:16:15
Because b minus 1 is now even.


361
00:16:15 --> 00:16:17
Don't sweat the details here,
I just want you to see the

362
00:16:17 --> 00:16:17
reason it goes through it.


363
00:16:17 --> 00:16:19
What I now have, though,
is a nice thing.

364
00:16:19 --> 00:16:25
It says, in either case, in
general, t of b-- and this is

365
00:16:25 --> 00:16:27
where I'm going to abuse
notation a little bit-- but I

366
00:16:27 --> 00:16:33
can basically bound it by t,
12 steps plus t of b over 2.

367
00:16:33 --> 00:16:35
And the abuse is, you know,
it's not quite right, it

368
00:16:35 --> 00:16:36
depends upon whether it's all
ready, but you can see in

369
00:16:36 --> 00:16:39
either case, after 12 steps, 2
runs through this and down to

370
00:16:39 --> 00:16:41
a problem size b over 2.


371
00:16:41 --> 00:16:42
Why's that nice?


372
00:16:42 --> 00:16:45
Well, that then says after
another 12 steps, we're

373
00:16:45 --> 00:16:51
down to a problem with
size t of b over 4.

374
00:16:51 --> 00:16:59
And if I pull it out one more
level, it's 12 plus 12 plus t

375
00:16:59 --> 00:17:04
of b over 8, which in general
is going to be, after k steps,

376
00:17:04 --> 00:17:07
12 k because I'll have 12 of
those to add up, plus t

377
00:17:07 --> 00:17:13
of b over 2 to the k.


378
00:17:13 --> 00:17:15
When am I done?


379
00:17:15 --> 00:17:20
When do I get down
to the base case?

380
00:17:20 --> 00:17:21
Somebody help me out.


381
00:17:21 --> 00:17:25
What am I looking for?


382
00:17:25 --> 00:17:25
Yeah.


383
00:17:25 --> 00:17:28
You're jumping slightly ahead
of me, but basically, I'm done

384
00:17:28 --> 00:17:29
when this is equal to 1, right?


385
00:17:29 --> 00:17:32
Because I get down to the base
case, so I'm done when b u is

386
00:17:32 --> 00:17:36
over 2 to the k is equal to 1,
and you're absolutely right,

387
00:17:36 --> 00:17:44
that's when k is
log base 2 of b.

388
00:17:44 --> 00:17:46
You're sitting a long ways
back, I have no idea if I'll

389
00:17:46 --> 00:17:48
make it this far or not.


390
00:17:48 --> 00:17:50
Thank you.


391
00:17:50 --> 00:17:51
OK.


392
00:17:51 --> 00:17:57
There's some constants in
there, but this is order log b.

393
00:17:57 --> 00:17:59
Logarithmic.


394
00:17:59 --> 00:18:01
This matters.


395
00:18:01 --> 00:18:02
This matters a lot.


396
00:18:02 --> 00:18:03
And I'm going to show you an
example in a second, just to

397
00:18:03 --> 00:18:06
drive this home, but notice
the characteristics.

398
00:18:06 --> 00:18:09
In the first two cases,
the problem reduced

399
00:18:09 --> 00:18:11
by 1 at each step.


400
00:18:11 --> 00:18:13
Whether it was recursive
or iterative.

401
00:18:13 --> 00:18:15
That's a sign that
it's probably linear.

402
00:18:15 --> 00:18:19
This case, I reduced the size
of the problem in half.

403
00:18:19 --> 00:18:21
It's a good sign that this is
logarithmic, and I'm going to

404
00:18:21 --> 00:18:25
come back in a second to why
logs are a great thing.

405
00:18:25 --> 00:18:28
Let me show you one more
class, though, about-- sorry,

406
00:18:28 --> 00:18:29
let me show you two more
classes of algorithms.

407
00:18:29 --> 00:18:32
Let's look at the next one g--
and there's a bug in your

408
00:18:32 --> 00:18:36
handout, it should be g of n
and m, I apologize for that, I

409
00:18:36 --> 00:18:40
changed it partway through
and didn't catch it.

410
00:18:40 --> 00:18:40
OK.


411
00:18:40 --> 00:18:45
Order of growth here.


412
00:18:45 --> 00:18:47
Anybody want to
volunteer a guess?

413
00:18:47 --> 00:18:53
Other than the TAs, who know?


414
00:18:53 --> 00:18:54
OK.


415
00:18:54 --> 00:18:56
Let's think it through.


416
00:18:56 --> 00:18:57
I've got two loops.


417
00:18:57 --> 00:18:58
All right?


418
00:18:58 --> 00:19:01
We already saw with one of the
loops, you know, it looked like

419
00:19:01 --> 00:19:03
it might be linear, depending
on what's inside of it, but

420
00:19:03 --> 00:19:04
let's think about this.


421
00:19:04 --> 00:19:06
I got two loops with g.


422
00:19:06 --> 00:19:07
What's g do?


423
00:19:07 --> 00:19:10
I've got an initialization of
x, and then I say, for i in the

424
00:19:10 --> 00:19:12
range, so that's basically
from 0 up to n minus

425
00:19:12 --> 00:19:14
1, what do I do?


426
00:19:14 --> 00:19:17
Well, inside of there, I've got
another loop, for j in the

427
00:19:17 --> 00:19:20
range from 0 up to m minus 1.


428
00:19:20 --> 00:19:24
What's the complexity
of that inner loop?

429
00:19:24 --> 00:19:26
Sorry?


430
00:19:26 --> 00:19:27
OK.


431
00:19:27 --> 00:19:28
You're doing the
whole thing for me.

432
00:19:28 --> 00:19:30
What's the complexity just
of this inner loop here?

433
00:19:30 --> 00:19:31
Just this piece.


434
00:19:31 --> 00:19:36
How many times do I go
through that loop? m.

435
00:19:36 --> 00:19:36
Right?


436
00:19:36 --> 00:19:39
I'm going to get back to your
answer in a second, because

437
00:19:39 --> 00:19:40
you're heading in the
right direction.

438
00:19:40 --> 00:19:43
The inner loop, this part
here, I do m times.

439
00:19:43 --> 00:19:44
There's one step inside of it.


440
00:19:44 --> 00:19:46
Right?


441
00:19:46 --> 00:19:48
How many times do I go
through that loop?

442
00:19:48 --> 00:19:53
Ah, n times, because for each
value of i, I'm going to do

443
00:19:53 --> 00:19:56
that m thing, so that is, close
to what you said, right?

444
00:19:56 --> 00:20:01
The order complexity here, if I
actually write it, would be--

445
00:20:01 --> 00:20:08
sorry, order n times m, and if
m was equal to n, that would be

446
00:20:08 --> 00:20:14
order n squared, and
this is quadratic.

447
00:20:14 --> 00:20:19
And that's a
different behavior.

448
00:20:19 --> 00:20:20
OK.


449
00:20:20 --> 00:20:21
What am I doing?


450
00:20:21 --> 00:20:23
Building up examples
of algorithms.

451
00:20:23 --> 00:20:25
Again, I want you to start
seeing how to map the

452
00:20:25 --> 00:20:27
characteristics of the code--
the characteristics of the

453
00:20:27 --> 00:20:29
algorithm, let's not call it
the code-- to the complexity.

454
00:20:29 --> 00:20:31
I'm going to come back to that
in a second with that, but I

455
00:20:31 --> 00:20:33
need to do one more example,
and I've got to use my

456
00:20:33 --> 00:20:36
high-tech really
expensive props.

457
00:20:36 --> 00:20:38
Right.


458
00:20:38 --> 00:20:39
So here's the fourth or
fifth, whatever we're up

459
00:20:39 --> 00:20:41
to, I guess fifth example.


460
00:20:41 --> 00:20:43
This is an example of a problem
called Towers of Hanoi.

461
00:20:43 --> 00:20:44
Anybody heard about
this problem?

462
00:20:44 --> 00:20:47
A few tentative hands.


463
00:20:47 --> 00:20:49
OK.


464
00:20:49 --> 00:20:51
Here's the story
as I am told it.

465
00:20:51 --> 00:20:53
There's a temple in
the middle of Hanoi.

466
00:20:53 --> 00:20:57
In that temple, there are three
very large diamond-encrusted

467
00:20:57 --> 00:21:01
posts, and on those posts are
sixty-four disks, all

468
00:21:01 --> 00:21:02
of a different size.


469
00:21:02 --> 00:21:05
And they're, you know, covered
with jewels and all sorts

470
00:21:05 --> 00:21:07
of other really neat stuff.


471
00:21:07 --> 00:21:10
There are a set of priests in
that temple, and their task is

472
00:21:10 --> 00:21:14
to move the entire stack of
sixty-four disks from one

473
00:21:14 --> 00:21:17
post to a second post.


474
00:21:17 --> 00:21:20
When they do this, you know,
the universe ends or they solve

475
00:21:20 --> 00:21:22
the financial crisis in
Washington or something like

476
00:21:22 --> 00:21:24
that actually good
happens, right?

477
00:21:24 --> 00:21:26
Boy, none of you have
401k's, you're not even

478
00:21:26 --> 00:21:28
wincing at that thing.


479
00:21:28 --> 00:21:29
All right.


480
00:21:29 --> 00:21:31
The rules, though, are, they
can only move one disk at a

481
00:21:31 --> 00:21:36
time, and they can never
cover up a smaller disk

482
00:21:36 --> 00:21:37
with a larger disk.


483
00:21:37 --> 00:21:38
OK.


484
00:21:38 --> 00:21:40
Otherwise you'd just move
the whole darn stack, OK?

485
00:21:40 --> 00:21:42
So we want to solve
that problem.

486
00:21:42 --> 00:21:43
We want to write a piece of
code that helps these guys

487
00:21:43 --> 00:21:45
out, so I'm going to
show you an example.

488
00:21:45 --> 00:21:47
Let's see if we can figure
out how to do this.

489
00:21:47 --> 00:21:49
So, we'll start
with the easy one.

490
00:21:49 --> 00:21:51
Moving a disk of size 1.


491
00:21:51 --> 00:21:53
OK, that's not so bad.


492
00:21:53 --> 00:21:55
Moving a stack of size 2, if I
want to go there, I need to

493
00:21:55 --> 00:21:57
put this one temporarily over
here so I can move the bottom

494
00:21:57 --> 00:22:00
one before I move it over.


495
00:22:00 --> 00:22:02
Moving a stack of size 3,
again, if I want to go over

496
00:22:02 --> 00:22:04
there, I need to make sure I
can put the spare one over here

497
00:22:04 --> 00:22:06
before I move the bottom one, I
can't cover up any of the

498
00:22:06 --> 00:22:09
smaller ones with the larger
one, but I can get it there.

499
00:22:09 --> 00:22:13
Stack of size 4, again I'm
going there, so I'm going to do

500
00:22:13 --> 00:22:15
this initially, no I'm not,
I'm going to start again.

501
00:22:15 --> 00:22:17
I'm going to go there
initially, so I can move this

502
00:22:17 --> 00:22:20
over here, so I can get the
base part of that over there, I

503
00:22:20 --> 00:22:22
want to put that one there
before I put this over here,

504
00:22:22 --> 00:22:24
finally I get to the point
where I can move the bottom one

505
00:22:24 --> 00:22:26
over, now I've got to be really
careful to make sure that I

506
00:22:26 --> 00:22:29
don't cover up the bottom one
in the wrong way before I get

507
00:22:29 --> 00:22:31
to the stage where I wish they
were posts and there you go.

508
00:22:31 --> 00:22:32
All right?


509
00:22:32 --> 00:22:34
[APPLAUSE]


510
00:22:34 --> 00:22:36
I mean, I can make money
at Harvard Square doing

511
00:22:36 --> 00:22:38
this stuff, right?


512
00:22:38 --> 00:22:41
All right, you
ready to do five?

513
00:22:41 --> 00:22:43
Got the solution?


514
00:22:43 --> 00:22:45
Not so easy to see.


515
00:22:45 --> 00:22:47
All right, but this is
actually a great one of

516
00:22:47 --> 00:22:49
those educational moments.


517
00:22:49 --> 00:22:52
This is a great example
to think recursively.

518
00:22:52 --> 00:22:54
If I wanted to think about this
problem recursively-- what do I

519
00:22:54 --> 00:22:55
mean by thinking recursively?


520
00:22:55 --> 00:22:58
How do I reduce this to
a smaller-size problem

521
00:22:58 --> 00:23:00
in the same instant?


522
00:23:00 --> 00:23:02
And so, if I do that, this
now becomes really easy.

523
00:23:02 --> 00:23:06
If I want to move this stack
here, I'm going to take a stack

524
00:23:06 --> 00:23:11
of size n minus 1, move it to
the spare spot, now I can move

525
00:23:11 --> 00:23:14
the base disk over, and then
I'm going to move that stack

526
00:23:14 --> 00:23:18
of size n minus 1 to there.


527
00:23:18 --> 00:23:21
That's literally
what I did, OK?

528
00:23:21 --> 00:23:22
So there's the code.


529
00:23:22 --> 00:23:23
Called towers.


530
00:23:23 --> 00:23:26
I'm just going to have you--
let you take a look at it.

531
00:23:26 --> 00:23:27
I'm giving it an argument,
which is the size of the

532
00:23:27 --> 00:23:30
stack, and then just labels
for the three posts.

533
00:23:30 --> 00:23:33
A from, a to, and a spare.


534
00:23:33 --> 00:23:36
And in fact, if we look at
this-- let me just pop it over

535
00:23:36 --> 00:23:40
to the other side-- OK, I can
move a tower, I'll say of

536
00:23:40 --> 00:23:48
size 2, from, to, and spare,
and that was what I did.

537
00:23:48 --> 00:23:55
And if I want to move towers,
let's say, size 5, from, to,

538
00:23:55 --> 00:24:01
and spare, there are the
instructions for

539
00:24:01 --> 00:24:02
how to move it.


540
00:24:02 --> 00:24:05
We ain't going to
do sixty-four.

541
00:24:05 --> 00:24:06
OK.


542
00:24:06 --> 00:24:07
All right.


543
00:24:07 --> 00:24:09
So it's fun, and I got a little
bit of applause out of it,

544
00:24:09 --> 00:24:13
which is always nice for me,
but I also showed you how to

545
00:24:13 --> 00:24:14
think about it recursively.


546
00:24:14 --> 00:24:16
Once you hear that description,
it's easy to write

547
00:24:16 --> 00:24:18
the code, in fact.


548
00:24:18 --> 00:24:20
This is a place where the
recursive version of it is

549
00:24:20 --> 00:24:22
much easier to think about
than the iterative one.

550
00:24:22 --> 00:24:24
But what I really want to
talk about is, what's the

551
00:24:24 --> 00:24:25
order of growth here?


552
00:24:25 --> 00:24:28
What's the complexity
of this algorithm?

553
00:24:28 --> 00:24:31
And again, I'm going to do it
with a little bit of abusive

554
00:24:31 --> 00:24:33
notation, and it's a little
more complicated, but we

555
00:24:33 --> 00:24:34
can kind of look at.


556
00:24:34 --> 00:24:35
All right?


557
00:24:35 --> 00:24:40
Given the code up there, if I
want to move a tower of size

558
00:24:40 --> 00:24:43
n, what do I have to do?


559
00:24:43 --> 00:24:47
I've got to test to see if I'm
in the base case, and if I'm

560
00:24:47 --> 00:24:52
not, then I need to move a
tower of size n minus 1, I need

561
00:24:52 --> 00:24:56
to move a tower of size 1, and
I need to move a second--

562
00:24:56 --> 00:25:02
sorry about that-- a second
tower of size n minus 1.

563
00:25:02 --> 00:25:03
OK. t of 1 I can also reduce.


564
00:25:03 --> 00:25:06
In the case of a tower of
size 1, basically there are

565
00:25:06 --> 00:25:07
two things to do, right?


566
00:25:07 --> 00:25:09
I've got to do the test, and
then I just do the move.

567
00:25:09 --> 00:25:16
So the general formula is that.


568
00:25:16 --> 00:25:18
Now.


569
00:25:18 --> 00:25:20
You might look at that and say,
well that's just a lot like

570
00:25:20 --> 00:25:22
what we had over here.


571
00:25:22 --> 00:25:22
Right?


572
00:25:22 --> 00:25:25
We had some additive constant
plus a simpler version of the

573
00:25:25 --> 00:25:28
same problem reduced
in size by 1.

574
00:25:28 --> 00:25:31
But that two matters.


575
00:25:31 --> 00:25:32
So let's look at it.


576
00:25:32 --> 00:25:35
How do I rea-- replace the
expression FOR t of n minus 1?

577
00:25:35 --> 00:25:39
Substitute it in again. t
of n minus 1 is 3 plus

578
00:25:39 --> 00:25:40
2 t of n minus 2.


579
00:25:40 --> 00:25:49
So this is 3, plus 2 times
3, plus 4 t minus 2.

580
00:25:49 --> 00:25:51
OK.


581
00:25:51 --> 00:25:56
And if I substitute it again,
I get 3 plus 2 times 3 plus 4

582
00:25:56 --> 00:26:02
times 3 plus 8 t n minus 3.


583
00:26:02 --> 00:26:04
This is going by a little fast.


584
00:26:04 --> 00:26:05
I'm just substituting in.


585
00:26:05 --> 00:26:08
I'm going to skip some steps.


586
00:26:08 --> 00:26:13
But basically if I do this, I
end up with 3 times 1 plus 2

587
00:26:13 --> 00:26:19
plus 4 to 2 to the k minus 1
for all of those terms, plus

588
00:26:19 --> 00:26:27
2-- I want to do this right, 2
to the k, sorry--

589
00:26:27 --> 00:26:30
t of n minus k.


590
00:26:30 --> 00:26:31
OK.


591
00:26:31 --> 00:26:33
Don't sweat the details,
I'm just expanding it out.

592
00:26:33 --> 00:26:35
What I want you to see
is, because I've got two

593
00:26:35 --> 00:26:38
versions of that problem.


594
00:26:38 --> 00:26:39
The next time down I've
got four versions.

595
00:26:39 --> 00:26:40
Next time down I've
got eight versions.

596
00:26:40 --> 00:26:43
And in fact, if I substitute, I
can solve for this, I'm done

597
00:26:43 --> 00:26:45
when this is equal to 1.


598
00:26:45 --> 00:26:49
If you substitute it all
in, you get basically

599
00:26:49 --> 00:26:54
order 2 to the n.


600
00:26:54 --> 00:26:57
Exponential.


601
00:26:57 --> 00:26:59
That's a problem.


602
00:26:59 --> 00:27:02
Now, it's also the case that
this is fundamentally what

603
00:27:02 --> 00:27:04
class this algorithm falls
into, it is going to take

604
00:27:04 --> 00:27:06
exponential amount of time.


605
00:27:06 --> 00:27:10
But it grows pretty rapidly, as
n goes up, and I'm going to

606
00:27:10 --> 00:27:12
show you an example
in a second.

607
00:27:12 --> 00:27:14
Again, what I want you
to see is, notice the

608
00:27:14 --> 00:27:15
characteristic of that.


609
00:27:15 --> 00:27:20
That this recursive call
had two sub-problems of

610
00:27:20 --> 00:27:22
a smaller size, not one.


611
00:27:22 --> 00:27:23
And that makes a
big difference.

612
00:27:23 --> 00:27:26
So just to show you how big a
difference it makes, let's

613
00:27:26 --> 00:27:29
run a couple of numbers.


614
00:27:29 --> 00:27:33
Let's suppose n is 1000,
and we're running at

615
00:27:33 --> 00:27:37
nanosecond speed.


616
00:27:37 --> 00:27:52
We have seen log, linear,
quadratic, and exponential.

617
00:27:52 --> 00:27:55
So, again, there could be
constants in here, but just

618
00:27:55 --> 00:27:56
to give you a sense of this.


619
00:27:56 --> 00:27:59
If I'm running at nanosecond
speed, n, the size of the

620
00:27:59 --> 00:28:02
problem, whatever it is, is
1000, and I've got a log

621
00:28:02 --> 00:28:08
algorithm, it takes 10
nanoseconds to complete.

622
00:28:08 --> 00:28:12
If you blink, you miss it.


623
00:28:12 --> 00:28:18
If I'm running a linear
algorithm, it'll take one

624
00:28:18 --> 00:28:21
microsecond to complete.


625
00:28:21 --> 00:28:25
If I'm running a quadratic
algorithm, it'll take one

626
00:28:25 --> 00:28:29
millisecond to complete.


627
00:28:29 --> 00:28:33
And if I'm running an
exponential algorithm,

628
00:28:33 --> 00:28:33
any guesses?


629
00:28:33 --> 00:28:50
I hope Washington doesn't take
this long to fix my 401k plan.

630
00:28:50 --> 00:28:51
All right?


631
00:28:51 --> 00:28:54
10 to the 284 years.


632
00:28:54 --> 00:28:56
As Emeril would say, pow!


633
00:28:56 --> 00:28:58
That's a some spicy whatever.


634
00:28:58 --> 00:28:59
All right.


635
00:28:59 --> 00:29:01
Bad jokes aside,
what's the point?

636
00:29:01 --> 00:29:04
You see, these classes have
really different performance.

637
00:29:04 --> 00:29:05
Now this is a
little misleading.

638
00:29:05 --> 00:29:08
These are all really fast, so
just to give you another set of

639
00:29:08 --> 00:29:12
examples, I'm not going to do
the-- If I had a problem where

640
00:29:12 --> 00:29:18
the log one took ten
milliseconds, then the linear

641
00:29:18 --> 00:29:21
one would take a second, the
quadratic one would

642
00:29:21 --> 00:29:26
take 16 minutes.


643
00:29:26 --> 00:29:29
So you can see, even
the quadratic ones can

644
00:29:29 --> 00:29:30
blow up in a hurry.


645
00:29:30 --> 00:29:33
And this goes back to the point
I tried to make last time.

646
00:29:33 --> 00:29:35
Yes, the computers
are really fast.

647
00:29:35 --> 00:29:37
But the problems can grow much
faster than you can get a

648
00:29:37 --> 00:29:39
performance boost out
of the computer.

649
00:29:39 --> 00:29:42
And you really, wherever
possible, want to avoid that

650
00:29:42 --> 00:29:44
exponential algorithm, because
that's really deadly.

651
00:29:44 --> 00:29:48
Yes.


652
00:29:48 --> 00:29:50
All right.


653
00:29:50 --> 00:29:52
The question is, is there
a point where it'll quit.

654
00:29:52 --> 00:29:55
Yeah, when the power goes out,
or-- so let me not answer

655
00:29:55 --> 00:29:57
it quite so facetiously.


656
00:29:57 --> 00:29:58
We'd be mostly
talking about time.

657
00:29:58 --> 00:30:00
In fact, if I ran one of
these things, it would

658
00:30:00 --> 00:30:01
just keep crunching away.


659
00:30:01 --> 00:30:05
It will probably quit at some
point because of space issues,

660
00:30:05 --> 00:30:06
unless I'm writing an
algorithm that is using

661
00:30:06 --> 00:30:08
no additional space.


662
00:30:08 --> 00:30:09
Right.


663
00:30:09 --> 00:30:10
Those things are going to
stack up, and eventually it's

664
00:30:10 --> 00:30:11
going to run out of space.


665
00:30:11 --> 00:30:13
And that's more likely to
happen, but, you know.

666
00:30:13 --> 00:30:17
The algorithm doesn't know that
it's going to take this long to

667
00:30:17 --> 00:30:19
compute, it's just busy
crunching away, trying to see

668
00:30:19 --> 00:30:21
if it can make it happen.


669
00:30:21 --> 00:30:21
OK.


670
00:30:21 --> 00:30:24
Good question, thank you.


671
00:30:24 --> 00:30:26
All right.


672
00:30:26 --> 00:30:29
I want to do one more extended
example here., because we've

673
00:30:29 --> 00:30:31
got another piece to do, but I
want to capture this, because

674
00:30:31 --> 00:30:33
it's important, so let me again
try and say it the

675
00:30:33 --> 00:30:34
following way.


676
00:30:34 --> 00:30:38
I want you to recognize classes
of algorithms and match what

677
00:30:38 --> 00:30:41
you see in the performance
of the algorithm to the

678
00:30:41 --> 00:30:42
complexity of that algorithm.


679
00:30:42 --> 00:30:44
All right?


680
00:30:44 --> 00:30:47
Linear algorithms tend to be
things where, at one

681
00:30:47 --> 00:30:50
pass-through, you reduce the
problem by a constant

682
00:30:50 --> 00:30:51
amount, by one.


683
00:30:51 --> 00:30:53
If you reduce it by two, it's
going to be the same thing.

684
00:30:53 --> 00:30:55
Where you go from problem
of size n to a problem

685
00:30:55 --> 00:30:57
of size n minus 1.


686
00:30:57 --> 00:31:01
A log algorithm typically is
one where you cut the size of

687
00:31:01 --> 00:31:03
the problem down by some
multiplicative factor.

688
00:31:03 --> 00:31:04
You reduce it in half.


689
00:31:04 --> 00:31:05
You reduce it in third.


690
00:31:05 --> 00:31:07
All right?


691
00:31:07 --> 00:31:10
Quadratic algorithms tend to
have this-- I was about to say

692
00:31:10 --> 00:31:13
additive, wrong term-- but
doubly-nested, triply-nested

693
00:31:13 --> 00:31:16
things are likely to be
quadratic or cubic algorithms,

694
00:31:16 --> 00:31:19
all right, because you know--
let me not confuse things--

695
00:31:19 --> 00:31:21
double-loop quadratic
algorithm, because you're doing

696
00:31:21 --> 00:31:24
one set of things and you're
doing it some other number of

697
00:31:24 --> 00:31:26
times, and that's a typical
signal that that's

698
00:31:26 --> 00:31:27
what you have there.


699
00:31:27 --> 00:31:28
OK.


700
00:31:28 --> 00:31:30
And then the exponentials, as
you saw is when typically I

701
00:31:30 --> 00:31:36
reduce the problem of one size
into two or more sub-problems

702
00:31:36 --> 00:31:37
of a smaller size.


703
00:31:37 --> 00:31:39
And you can imagine this gets
complex and there's lots of

704
00:31:39 --> 00:31:41
interesting things to do to
look to the real form, but

705
00:31:41 --> 00:31:43
those are the things
that you should see.

706
00:31:43 --> 00:31:44
Now.


707
00:31:44 --> 00:31:47
Two other things, before
we do this last example.

708
00:31:47 --> 00:31:50
One is, I'll remind you,
what we're interested in

709
00:31:50 --> 00:31:51
is asymptotic growth.


710
00:31:51 --> 00:31:55
How does this thing grow as I
make the problem size big?

711
00:31:55 --> 00:31:56
And I'll also remind you, and
we're going to see this in the

712
00:31:56 --> 00:31:59
next example, we talked about
looking at the worst

713
00:31:59 --> 00:32:00
case behavior.


714
00:32:00 --> 00:32:02
In these cases there's no best
case worst case, it's just

715
00:32:02 --> 00:32:03
doing one computation.


716
00:32:03 --> 00:32:05
We're going to see an example
of that in a second.

717
00:32:05 --> 00:32:07
What we really want to worry
about, what's the worst

718
00:32:07 --> 00:32:09
case that happens.


719
00:32:09 --> 00:32:11
And the third thing I want you
to keep in mind is, remember

720
00:32:11 --> 00:32:14
these are orders of growth.


721
00:32:14 --> 00:32:17
It is certainly possible, for
example, that a quadratic

722
00:32:17 --> 00:32:20
algorithm could run faster
than a linear algorithm.

723
00:32:20 --> 00:32:24
It depends on what the input
is, it depends on, you know,

724
00:32:24 --> 00:32:25
what the particular cases are.


725
00:32:25 --> 00:32:28
So it is not the case that, on
every input, a linear algorithm

726
00:32:28 --> 00:32:30
is always going to be better
than a quadratic algorithm.

727
00:32:30 --> 00:32:33
It is just in general that's
going to hold true, and that's

728
00:32:33 --> 00:32:35
what I want you to see.


729
00:32:35 --> 00:32:36
OK.


730
00:32:36 --> 00:32:37
I want to do one last example.


731
00:32:37 --> 00:32:41
I'm going to take a little bit
more time on it, because it's

732
00:32:41 --> 00:32:43
going to both reinforce these
ideas, but it's also going to

733
00:32:43 --> 00:32:46
show us how we have to think
about what's a primitive step.,

734
00:32:46 --> 00:32:50
and in a particular, how do
data structures interact

735
00:32:50 --> 00:32:51
with this analysis?


736
00:32:51 --> 00:32:53
Here I've just been running
integers, it's pretty simple,

737
00:32:53 --> 00:32:55
but if I have a data structure,
I'm going to have to worry

738
00:32:55 --> 00:32:56
about that a little bit more.


739
00:32:56 --> 00:32:57
So let's look at that.


740
00:32:57 --> 00:33:00
And the example I want to look
at is, suppose I want to search

741
00:33:00 --> 00:33:04
a list that I know is sorted,
to see if an element's

742
00:33:04 --> 00:33:05
in the list.


743
00:33:05 --> 00:33:05
OK?


744
00:33:05 --> 00:33:17
So the example I'm going
to do, I'm going to

745
00:33:17 --> 00:33:19
search a sorted list.


746
00:33:19 --> 00:33:20
All right.


747
00:33:20 --> 00:33:23
If you flip to the second side
of your handout, you'll see

748
00:33:23 --> 00:33:26
that I have a piece of code
there, that does this-- let me,

749
00:33:26 --> 00:33:30
ah, I didn't want to do that,
let me back up slightly-- this

750
00:33:30 --> 00:33:32
is the algorithm called search.


751
00:33:32 --> 00:33:35
And let's take a look at it.


752
00:33:35 --> 00:33:36
OK?


753
00:33:36 --> 00:33:38
Basic idea, before I even look
at the code, is pretty simple.

754
00:33:38 --> 00:33:41
If I've got a list that is
sorted, in let's call it, just

755
00:33:41 --> 00:33:43
in increasing order, and I
haven't said what's in the

756
00:33:43 --> 00:33:45
list, could be numbers, could
be other things, for now, we're

757
00:33:45 --> 00:33:46
going to just assume
they're integers.

758
00:33:46 --> 00:33:50
The easy thing to do would
be the following: start at

759
00:33:50 --> 00:33:53
the front end of the list,
check the first element.

760
00:33:53 --> 00:33:54
If it's the thing I'm
looking for, I'm done.

761
00:33:54 --> 00:33:54
It's there.


762
00:33:54 --> 00:33:57
If not, move on to
the next element.

763
00:33:57 --> 00:33:57
And keep doing that.


764
00:33:57 --> 00:34:00
But if, at any point, I get to
a place in the list where the

765
00:34:00 --> 00:34:04
thing I'm looking for is
smaller than the element in the

766
00:34:04 --> 00:34:07
list, I know everything else in
the rest of the list has to be

767
00:34:07 --> 00:34:09
bigger than that, I don't have
to bother looking anymore.

768
00:34:09 --> 00:34:10
It says the element's
not there.

769
00:34:10 --> 00:34:12
I can just stop.


770
00:34:12 --> 00:34:12
OK.


771
00:34:12 --> 00:34:14
So that's what this piece
of code does here.

772
00:34:14 --> 00:34:15
Right.?


773
00:34:15 --> 00:34:17
I'm going to set up a variable
to say, what's the answer I

774
00:34:17 --> 00:34:19
want to return, is
it there or not.

775
00:34:19 --> 00:34:22
Initially it's got that
funny value none.

776
00:34:22 --> 00:34:25
I'm going to set up an index,
which is going to tell me where

777
00:34:25 --> 00:34:29
to look, starting at the first
part of the list, right?

778
00:34:29 --> 00:34:31
And then, when I got-- I'm
also going to count how many

779
00:34:31 --> 00:34:33
comparisons I do, just so I can
see how much work I do here,

780
00:34:33 --> 00:34:35
and then notice what it does.


781
00:34:35 --> 00:34:40
It says while the index is
smaller than the size of the

782
00:34:40 --> 00:34:43
list, I'm not at the end of
the list, and I don't have

783
00:34:43 --> 00:34:45
an answer yet, check.


784
00:34:45 --> 00:34:49
So I'm going to check to see
if-- really can't read that

785
00:34:49 --> 00:34:51
thing, let me do it this way--
right, I'm going to increase

786
00:34:51 --> 00:34:53
the number of compares, and
I'm going to check to say, is

787
00:34:53 --> 00:34:57
the thing I'm looking for at
the i'th spot in the list?

788
00:34:57 --> 00:34:59
Right, so s of i saying, given
the list, look at the i'th

789
00:34:59 --> 00:35:01
element, is it the same thing?


790
00:35:01 --> 00:35:04
If it is, OK.


791
00:35:04 --> 00:35:06
Set the answer to true.


792
00:35:06 --> 00:35:09
Which means, next time through
the loop, that's going to pop

793
00:35:09 --> 00:35:11
out and return an answer.


794
00:35:11 --> 00:35:17
If it's not, then check to
see, is it smaller than

795
00:35:17 --> 00:35:19
that element in the
current spot of the list?

796
00:35:19 --> 00:35:22
And if that's true, it says
again, everything else in the

797
00:35:22 --> 00:35:24
list has to be bigger than
this, thing can't possibly be

798
00:35:24 --> 00:35:26
in the list, I'm taking
advantage of the ordering, I

799
00:35:26 --> 00:35:30
can set the answer to false,
change i to go to the next one,

800
00:35:30 --> 00:35:31
and next time through the loop,
I'm going to pop out

801
00:35:31 --> 00:35:34
and print it out.


802
00:35:34 --> 00:35:36
OK?


803
00:35:36 --> 00:35:37
Right.


804
00:35:37 --> 00:35:38
Order of growth here.


805
00:35:38 --> 00:35:46
What do you think?


806
00:35:46 --> 00:35:48
Even with these glasses
on, I can see no hands

807
00:35:48 --> 00:35:49
up, any suggestions?


808
00:35:49 --> 00:35:50
Somebody help me out.


809
00:35:50 --> 00:35:51
What do you think the
order of growth is here?

810
00:35:51 --> 00:35:58
I've got a list, walk you
through it an element at a

811
00:35:58 --> 00:36:02
time, do I look at each element
of the list more than once?

812
00:36:02 --> 00:36:04
Don't think so, right?


813
00:36:04 --> 00:36:07
So, what does this suggest?


814
00:36:07 --> 00:36:09
Sorry?


815
00:36:09 --> 00:36:10
Constant.


816
00:36:10 --> 00:36:12
Ooh, constant says, no matter
what the length of the

817
00:36:12 --> 00:36:16
list is, I'm going to take
the same amount of time.

818
00:36:16 --> 00:36:17
And I don't think
that's true, right?

819
00:36:17 --> 00:36:21
If I have a list ten times
longer, it's going to take me

820
00:36:21 --> 00:36:24
more time, so-- not a bad
guess, I'm still reward

821
00:36:24 --> 00:36:27
you, thank you.


822
00:36:27 --> 00:36:29
Somebody else.


823
00:36:29 --> 00:36:30
Yeah.


824
00:36:30 --> 00:36:30
Linear.


825
00:36:30 --> 00:36:31
Why?


826
00:36:31 --> 00:36:39
You're right, by
the way, but why?

827
00:36:39 --> 00:36:40
Yeah.


828
00:36:40 --> 00:36:41
All right, so the answer
was it's linear, which

829
00:36:41 --> 00:36:43
is absolutely right.


830
00:36:43 --> 00:36:46
Although for a reason we're
going to come back in a second.

831
00:36:46 --> 00:36:48
Oh, thank you, I hope your
friends help you out

832
00:36:48 --> 00:36:49
with that, thank you.


833
00:36:49 --> 00:36:50
Right?


834
00:36:50 --> 00:36:52
You can see that this ought
to be linear, because

835
00:36:52 --> 00:36:53
what am I doing?


836
00:36:53 --> 00:36:55
I'm walking down the list.


837
00:36:55 --> 00:36:57
So one of the things I didn't
say, it's sort of implicit

838
00:36:57 --> 00:36:59
here, is what is the thing I
measuring the size

839
00:36:59 --> 00:36:59
of the problem in?


840
00:36:59 --> 00:37:01
What's the size of the list?


841
00:37:01 --> 00:37:08
And if I'm walking down the
list, this is probably order

842
00:37:08 --> 00:37:10
of the length of the list
s, because I'm looking

843
00:37:10 --> 00:37:13
at each element once.


844
00:37:13 --> 00:37:14
Now you might say,
wait a minute.

845
00:37:14 --> 00:37:17
Thing's ordered, if I stop part
way through and I throw away

846
00:37:17 --> 00:37:19
half the list, doesn't
that help me?

847
00:37:19 --> 00:37:22
And the answer is yes, but it
doesn't change the complexity.

848
00:37:22 --> 00:37:23
Because what did we say?


849
00:37:23 --> 00:37:26
We're measuring the worst case.


850
00:37:26 --> 00:37:28
The worst case here is, the
things not in the list, in

851
00:37:28 --> 00:37:29
which case I've got to go
all the way through the

852
00:37:29 --> 00:37:32
list to get to the end.


853
00:37:32 --> 00:37:33
OK.


854
00:37:33 --> 00:37:37
Now, having said that, and I've
actually got a subtlety I'm

855
00:37:37 --> 00:37:39
going to come back to in a
second, there ought to be

856
00:37:39 --> 00:37:40
a better way to do this.


857
00:37:40 --> 00:37:41
OK?


858
00:37:41 --> 00:37:47
And here's the better
way to think about.

859
00:37:47 --> 00:37:54
I'll just draw out sort of a
funny representation of a list.

860
00:37:54 --> 00:37:55
These are sort of the cells, if
you like, in memory that are

861
00:37:55 --> 00:37:57
holding the elements
of the list.

862
00:37:57 --> 00:38:00
What we've been saying is,
I start here and look.

863
00:38:00 --> 00:38:00
If it's there, I'm done.


864
00:38:00 --> 00:38:02
If not, I go there.


865
00:38:02 --> 00:38:04
If it's there, I'm done, if
not, I keep walking down, and I

866
00:38:04 --> 00:38:07
only stop when I get to a place
where the element I'm looking

867
00:38:07 --> 00:38:10
for is smaller than the value
in the list., in which case I

868
00:38:10 --> 00:38:13
know the rest of this is
too big and I can stop.

869
00:38:13 --> 00:38:16
But I still have to
go through the list.

870
00:38:16 --> 00:38:17
There's a better way to think
about this, and in fact

871
00:38:17 --> 00:38:20
Professor Guttag has already
hinted at this in the

872
00:38:20 --> 00:38:22
last couple of lectures.


873
00:38:22 --> 00:38:24
The better way to think about
this is, suppose, rather than

874
00:38:24 --> 00:38:26
starting at the beginning, I
just grabbed some spot at

875
00:38:26 --> 00:38:29
random, like this one.


876
00:38:29 --> 00:38:32
And I look at that value.


877
00:38:32 --> 00:38:34
If it's the value I'm looking
for, boy, I ought to go to

878
00:38:34 --> 00:38:35
Vegas, I'm really lucky.


879
00:38:35 --> 00:38:37
And I'm done, right?


880
00:38:37 --> 00:38:39
If not, what could I do?


881
00:38:39 --> 00:38:41
Well, I could look at the value
here, and compare it to the

882
00:38:41 --> 00:38:46
value I'm trying to find, and
say the following; if the value

883
00:38:46 --> 00:38:51
I'm looking for is bigger
than this value, where

884
00:38:51 --> 00:38:53
do I need to look?


885
00:38:53 --> 00:38:53
Just here.


886
00:38:53 --> 00:38:57
All right?


887
00:38:57 --> 00:38:59
Can't possibly be there,
because I know this

888
00:38:59 --> 00:39:01
thing is over.


889
00:39:01 --> 00:39:04
On the other hand, if the value
I'm looking for here-- sorry,

890
00:39:04 --> 00:39:07
the value I'm looking for is
smaller than the value I see

891
00:39:07 --> 00:39:10
here, I just need to look here.


892
00:39:10 --> 00:39:14
All right?


893
00:39:14 --> 00:39:17
Having done that, I could do
the same thing, so I suppose I

894
00:39:17 --> 00:39:19
take this branch, I can pick a
spot like, say, this

895
00:39:19 --> 00:39:21
one, and look there.


896
00:39:21 --> 00:39:23
Because there, I'm done,
if not, I'm either

897
00:39:23 --> 00:39:27
looking here or there.


898
00:39:27 --> 00:39:31
And I keep cutting
the problem down.

899
00:39:31 --> 00:39:31
OK.


900
00:39:31 --> 00:39:35
Now, having said that,
where should I pick

901
00:39:35 --> 00:39:39
to look in this list?


902
00:39:39 --> 00:39:40
I'm sorry?


903
00:39:40 --> 00:39:41
Halfway.


904
00:39:41 --> 00:39:41
Why?


905
00:39:41 --> 00:39:50
You're right, but why?


906
00:39:50 --> 00:39:50
Yeah.


907
00:39:50 --> 00:39:53
So the answer, in case you
didn't hear it, was, again, if

908
00:39:53 --> 00:39:55
I'm a gambling person, I could
start like a way down here.

909
00:39:55 --> 00:39:57
All right?


910
00:39:57 --> 00:39:59
If I'm gambling, I'm saying,
gee, if I'm really lucky, it'll

911
00:39:59 --> 00:40:01
be only on this side, and I've
got a little bit of work to

912
00:40:01 --> 00:40:04
do, but if I'm unlucky, I'm
scrawed, the past pluperfect of

913
00:40:04 --> 00:40:07
screwed, OK., or a Boston fish.


914
00:40:07 --> 00:40:09
I'll look at the rest of
that big chunk of the

915
00:40:09 --> 00:40:11
list, and that's a pain.


916
00:40:11 --> 00:40:15
So halfway is the right thing
to do, because at each step,

917
00:40:15 --> 00:40:18
I'm guaranteed to throw away
at least half the list.

918
00:40:18 --> 00:40:18
Right?


919
00:40:18 --> 00:40:20
And that's nice.


920
00:40:20 --> 00:40:21
OK.


921
00:40:21 --> 00:40:25
What would you guess the
order of growth here is?

922
00:40:25 --> 00:40:26
Yeah.


923
00:40:26 --> 00:40:29
Why?


924
00:40:29 --> 00:40:29
Good.


925
00:40:29 --> 00:40:30
Exactly.


926
00:40:30 --> 00:40:30
Right?


927
00:40:30 --> 00:40:33
Again, if you didn't hear it,
the answer was it's log.

928
00:40:33 --> 00:40:36
Because I'm cutting down the
problem in half at each time.

929
00:40:36 --> 00:40:38
You're right, but there's
something we have to do to add

930
00:40:38 --> 00:40:40
to that, and that's the last
thing I want to pick up on.

931
00:40:40 --> 00:40:41
OK.


932
00:40:41 --> 00:40:43
Let's look at the code--
actually, let's test this

933
00:40:43 --> 00:40:45
out first before we do it.


934
00:40:45 --> 00:40:47
So I've added, as Professor
Guttag did-- ah, should have

935
00:40:47 --> 00:40:51
said it this way, let's write
the code for it first, sorry

936
00:40:51 --> 00:40:54
about that-- OK, I'm going to
write a little thing

937
00:40:54 --> 00:40:54
called b search.


938
00:40:54 --> 00:40:57
I'm going to call it down here
with search, which is simply

939
00:40:57 --> 00:41:00
going to call it, and then
print an answer out.

940
00:41:00 --> 00:41:03
In binary search-- ah, there's
that wonderful phrase, this is

941
00:41:03 --> 00:41:05
called a version of binary
search, just like you saw bin--

942
00:41:05 --> 00:41:08
or bi-section methods, when we
were doing numerical things--

943
00:41:08 --> 00:41:11
in binary search, I need to
keep track of the starting

944
00:41:11 --> 00:41:14
point and the ending point
of the list I'm looking at.

945
00:41:14 --> 00:41:16
Initially, it's the beginning
and the end of it.

946
00:41:16 --> 00:41:18
And when I do this test, what I
want to do, is say I'm going to

947
00:41:18 --> 00:41:21
pick the middle spot, and
depending on the test, if I

948
00:41:21 --> 00:41:24
know it's in the upper half,
I'm going to set my start at

949
00:41:24 --> 00:41:27
the mid point and the end stays
the same, if it's in the front

950
00:41:27 --> 00:41:29
half I'm going to keep the
front the same and I'm going

951
00:41:29 --> 00:41:30
to change the endpoint.


952
00:41:30 --> 00:41:33
And you can see that
in this code here.

953
00:41:33 --> 00:41:34
Right?


954
00:41:34 --> 00:41:34
What does it say to do?


955
00:41:34 --> 00:41:37
It says, well I'm going to
print out first and last, just

956
00:41:37 --> 00:41:41
so you can see it, and then I
say, gee, if last minus first

957
00:41:41 --> 00:41:43
is less than 2, that is, if
there's no more than two

958
00:41:43 --> 00:41:46
elements left in the list, then
I can just check those

959
00:41:46 --> 00:41:50
two elements, and
return the answer.

960
00:41:50 --> 00:41:52
Otherwise, we find the
midpoint, and notice

961
00:41:52 --> 00:41:53
what it does.


962
00:41:53 --> 00:41:55
First, it's pointing to the
beginning of the list, which

963
00:41:55 --> 00:41:57
initially might be down here
at 0 but after a while,

964
00:41:57 --> 00:41:59
might be part way through.


965
00:41:59 --> 00:42:04
And to that, I simply
add a halfway point,

966
00:42:04 --> 00:42:06
and then I check.


967
00:42:06 --> 00:42:08
If it's at that point, I'm
done, if not, if it's greater

968
00:42:08 --> 00:42:11
than the value I'm looking
for, I either take one

969
00:42:11 --> 00:42:15
half or the other.


970
00:42:15 --> 00:42:15
OK.


971
00:42:15 --> 00:42:17
You can see that thing is
cutting down the problem in

972
00:42:17 --> 00:42:19
half each time, which is good,
but there's one more thing

973
00:42:19 --> 00:42:20
I need to deal with.


974
00:42:20 --> 00:42:22
So let's step through this
with a little more care.

975
00:42:22 --> 00:42:23
And I keep saying, before
we do it, let's just

976
00:42:23 --> 00:42:24
actually try it out.


977
00:42:24 --> 00:42:28
So I'm going to go over here,
and I'm going to type test

978
00:42:28 --> 00:42:32
search-- I can type-- and if
you look at your handout, it's

979
00:42:32 --> 00:42:36
just a sequence of tests
that I'm going to do.

980
00:42:36 --> 00:42:36
OK.


981
00:42:36 --> 00:42:39
So initially, I'm going to
set up the list to be the

982
00:42:39 --> 00:42:40
first million integers.


983
00:42:40 --> 00:42:43
Yeah, it's kind of simple, but
it gives me an ordered list of

984
00:42:43 --> 00:42:44
these things, And let's run it.


985
00:42:44 --> 00:42:46
OK.


986
00:42:46 --> 00:42:48
So I'm first going to look for
something that's not in the

987
00:42:48 --> 00:42:50
list, I'm going to see, is
minus 1 in this list, so it's

988
00:42:50 --> 00:42:52
going to be at the far end, and
if I do that in the

989
00:42:52 --> 00:42:54
basic case, bam.


990
00:42:54 --> 00:42:54
Done.


991
00:42:54 --> 00:42:56
All right?


992
00:42:56 --> 00:42:58
The basic, that primary search,
because it looks at the first

993
00:42:58 --> 00:43:00
element, says it's smaller than
everything else, I'm done.

994
00:43:00 --> 00:43:06
If I look in the binary case,
takes a little longer.

995
00:43:06 --> 00:43:07
Notice the printout here.


996
00:43:07 --> 00:43:10
The printout is simply
telling me, what are the

997
00:43:10 --> 00:43:11
ranges of the search.


998
00:43:11 --> 00:43:15
And you can see it wrapping its
way down, cutting in half at

999
00:43:15 --> 00:43:18
each time until it gets there,
but it takes a while to find.

1000
00:43:18 --> 00:43:18
All right.


1001
00:43:18 --> 00:43:23
Let's search to see though now
if a million is in this list,

1002
00:43:23 --> 00:43:25
or 10 million, whichever way I
did this, it must be

1003
00:43:25 --> 00:43:26
a million, right?


1004
00:43:26 --> 00:43:31
In the basic case, oh,
took a little while.

1005
00:43:31 --> 00:43:35
Right, in the binary case, bam.


1006
00:43:35 --> 00:43:37
In fact, it took the same
number of steps as it did in

1007
00:43:37 --> 00:43:39
the other case, because each
time I'm cutting it

1008
00:43:39 --> 00:43:40
down by a half.


1009
00:43:40 --> 00:43:41
OK.


1010
00:43:41 --> 00:43:42
That's nice.


1011
00:43:42 --> 00:43:44
Now, let's do the following; if
you look right here, I'm going

1012
00:43:44 --> 00:43:47
to set this now to-- I'm going
to change my range to 10

1013
00:43:47 --> 00:43:50
million, I'm going to first
say, gee, is a million in

1014
00:43:50 --> 00:43:54
there, using the basic search.


1015
00:43:54 --> 00:43:56
It is.


1016
00:43:56 --> 00:43:57
Now, I'm going to say,
is 10 million in this,

1017
00:43:57 --> 00:44:01
using the basic search.


1018
00:44:01 --> 00:44:04
We may test your hypothesis,
about how long does it take, if

1019
00:44:04 --> 00:44:06
I time this really well, I
ought to be able to end when it

1020
00:44:06 --> 00:44:10
finds it, which should
be right about now.

1021
00:44:10 --> 00:44:13
That was pure luck.


1022
00:44:13 --> 00:44:14
But notice how much
longer it took.

1023
00:44:14 --> 00:44:16
On the other hand, watch
what happens with binary.

1024
00:44:16 --> 00:44:18
Is the partway one there?


1025
00:44:18 --> 00:44:19
Yeah.


1026
00:44:19 --> 00:44:21
Is the last one there?


1027
00:44:21 --> 00:44:22
Wow.


1028
00:44:22 --> 00:44:25
I think it took one more step.


1029
00:44:25 --> 00:44:27
Man, that's exactly what
logs should do, right?

1030
00:44:27 --> 00:44:29
I make the problem ten
times bigger, it takes

1031
00:44:29 --> 00:44:31
one more step to do it.


1032
00:44:31 --> 00:44:34
Whereas in the linear case, I
make it ten times bigger, it

1033
00:44:34 --> 00:44:36
takes ten times longer to run.


1034
00:44:36 --> 00:44:38
OK.


1035
00:44:38 --> 00:44:40
So I keep saying I've got one
thing hanging, it's the last

1036
00:44:40 --> 00:44:42
thing I want to do, but I
wanted you see how much of

1037
00:44:42 --> 00:44:43
a difference this makes.


1038
00:44:43 --> 00:44:46
But let's look a little more
carefully at the code for

1039
00:44:46 --> 00:44:48
binary search-- for search 1.


1040
00:44:48 --> 00:44:50
What's the complexity
of search 1?

1041
00:44:50 --> 00:44:52
Well, you might say
it's constant, right?

1042
00:44:52 --> 00:44:54
It's only got two things to do,
except what it really says is,

1043
00:44:54 --> 00:44:57
that the complexity of search 1
is the same as the complexity

1044
00:44:57 --> 00:44:59
of b search, because that's
the call it's doing.

1045
00:44:59 --> 00:45:00
So let's look at b search.


1046
00:45:00 --> 00:45:02
All right?


1047
00:45:02 --> 00:45:05
We've got the code for
b search up there.

1048
00:45:05 --> 00:45:08
First step, constant, right?


1049
00:45:08 --> 00:45:09
Nothing to do.


1050
00:45:09 --> 00:45:13
Second step, hm.


1051
00:45:13 --> 00:45:17
That also looks
constant, you think?

1052
00:45:17 --> 00:45:19
Oh but wait a minute.


1053
00:45:19 --> 00:45:21
I'm accessing s.


1054
00:45:21 --> 00:45:22
I'm accessing a list.


1055
00:45:22 --> 00:45:28
How long does it take for me to
get the nth element of a list?

1056
00:45:28 --> 00:45:31
That might not be
a primitive step.

1057
00:45:31 --> 00:45:35
And in fact, it depends
on how I store a list.

1058
00:45:35 --> 00:45:42
So, for example, in this case,
I had lists that I knew

1059
00:45:42 --> 00:45:44
were made out of integers.


1060
00:45:44 --> 00:45:50
As a consequence, I
have a list of ints.

1061
00:45:50 --> 00:45:54
I might know, for example, that
it takes four memory chunks to

1062
00:45:54 --> 00:45:56
represent one int,
just for example.

1063
00:45:56 --> 00:46:01
And to find the i'th element,
I'm simply going to take the

1064
00:46:01 --> 00:46:04
starting point, that point at
the beginning of memory where

1065
00:46:04 --> 00:46:10
the list is, plus 4 times i,
that would tell me how many

1066
00:46:10 --> 00:46:13
units over to go, and that's
the memory location I want

1067
00:46:13 --> 00:46:15
to look for the i'th
element of the list.

1068
00:46:15 --> 00:46:18
And remember, we said we're
going to assume a random access

1069
00:46:18 --> 00:46:22
model, which says, as long as I
know the location, it takes a

1070
00:46:22 --> 00:46:24
constant amount of time
to get to that point.

1071
00:46:24 --> 00:46:28
So if the-- if I knew the lists
were made of just integers,

1072
00:46:28 --> 00:46:30
it'd be really easy
to figure it out.

1073
00:46:30 --> 00:46:32
Another way of saying it is,
this takes constant amount of

1074
00:46:32 --> 00:46:34
time to figure out where to
look, it takes constant amount

1075
00:46:34 --> 00:46:37
of time to get there, so in
fact I could treat indexing

1076
00:46:37 --> 00:46:41
into a list as being
a basic operation.

1077
00:46:41 --> 00:46:44
But we know lists can be
composed of anything.

1078
00:46:44 --> 00:46:47
Could be ints, could be floats,
could be a combination of

1079
00:46:47 --> 00:46:49
things, some ints, some floats,
some lists, some strings, some

1080
00:46:49 --> 00:46:51
lists of lists, whatever.


1081
00:46:51 --> 00:46:58
And in that case, in general
lists, I need to figure out

1082
00:46:58 --> 00:47:01
what's the access time.


1083
00:47:01 --> 00:47:03
And here I've got a choice.


1084
00:47:03 --> 00:47:05
OK, one of the ways I could
do would be the following.

1085
00:47:05 --> 00:47:10
I could have a pointer to the
beginning of the list where the

1086
00:47:10 --> 00:47:17
first element here is the
actual value, and this

1087
00:47:17 --> 00:47:23
would point to the next
element in the list.

1088
00:47:23 --> 00:47:25
Or another way of saying it is,
the first part of the cell

1089
00:47:25 --> 00:47:28
could be some encoding of how
many cells do I need to have to

1090
00:47:28 --> 00:47:31
store the value, and then I've
got some way of telling me

1091
00:47:31 --> 00:47:33
where to get the next
element of the list.

1092
00:47:33 --> 00:47:39
And this would point to value,
and this would point off

1093
00:47:39 --> 00:47:42
someplace in memory.


1094
00:47:42 --> 00:47:44
Here's the problem with that
technique, and by the way, a

1095
00:47:44 --> 00:47:46
number of programming languages
use this, including Lisp.

1096
00:47:46 --> 00:47:48
The problem with that
technique, while it's very

1097
00:47:48 --> 00:47:51
general, is how long does it
take me to find the i'th

1098
00:47:51 --> 00:47:54
element of the list?


1099
00:47:54 --> 00:47:55
Oh fudge knuckle.


1100
00:47:55 --> 00:47:56
OK.


1101
00:47:56 --> 00:47:58
I've got to go to the first
place, figure out how far over

1102
00:47:58 --> 00:48:01
to skip, go to the next place,
figure out how far over to

1103
00:48:01 --> 00:48:02
skip, eventually I'll
be out the door.

1104
00:48:02 --> 00:48:05
I've got to count my way down,
which means that the access

1105
00:48:05 --> 00:48:09
would be linear in the length
of the list to find the i'th

1106
00:48:09 --> 00:48:11
element of the list, and that's
going to increase

1107
00:48:11 --> 00:48:13
the complexity.


1108
00:48:13 --> 00:48:15
There's an alternative, which
is the last point I want to

1109
00:48:15 --> 00:48:22
make, which is instead what I
could do, I should have said

1110
00:48:22 --> 00:48:25
these things are called linked
lists, we'll come back to

1111
00:48:25 --> 00:48:29
those, another way to do it, is
to have the start of the list

1112
00:48:29 --> 00:48:36
be at some point in memory,
and to have each one of the

1113
00:48:36 --> 00:48:40
successive cells in memory
point off to the actual value.

1114
00:48:40 --> 00:48:43
Which may take up some
arbitrary amount of memory.

1115
00:48:43 --> 00:48:47
In that case, I'm back
to this problem.

1116
00:48:47 --> 00:48:51
And as a consequence, access
time in the list is constant,

1117
00:48:51 --> 00:48:56
which is what I want.


1118
00:48:56 --> 00:48:59
Now, to my knowledge, most
implementations of Python use

1119
00:48:59 --> 00:49:02
this way of storing lists,
whereas Lisp and Scheme do not.

1120
00:49:02 --> 00:49:05
The message I'm trying to get
to here, because I'm running

1121
00:49:05 --> 00:49:09
you right up against time, is
I have to be careful about

1122
00:49:09 --> 00:49:11
what's a primitive step.


1123
00:49:11 --> 00:49:14
With this, if I can assume that
accessing the i'th element of a

1124
00:49:14 --> 00:49:18
list is constant, then you
can't see that the rest of that

1125
00:49:18 --> 00:49:21
analysis looks just like the
log analysis I did before, and

1126
00:49:21 --> 00:49:24
each step, no matter which
branch I'm taking, I'm cutting

1127
00:49:24 --> 00:49:25
the problem down in half.


1128
00:49:25 --> 00:49:27
And as a consequence,
it is log.

1129
00:49:27 --> 00:49:35
And the last piece of this, is
as said, I have to make sure I

1130
00:49:35 --> 00:49:41
know what my primitive elements
are, in terms of operations.

1131
00:49:41 --> 00:49:44
Summary: I want you to
recognize different

1132
00:49:44 --> 00:49:45
classes of algorithms.


1133
00:49:45 --> 00:49:46
I'm not going to repeat them.


1134
00:49:46 --> 00:49:48
We've seen log, we've seen
linear, we've seen quadratic,

1135
00:49:48 --> 00:49:50
we've seen exponential.


1136
00:49:50 --> 00:49:53
One of the things you should
begin to do, is to recognize

1137
00:49:53 --> 00:49:57
what identifies those classes
of algorithms, so you can map

1138
00:49:57 --> 00:49:59
your problems into
those ranges.

1139
00:49:59 --> 00:50:01
And with that, good
luck on Thursday.

1140
00:50:01 --> 00:50:02