1
00:00:00 --> 00:00:00



2
00:00:00 --> 00:00:02
The following content is
provided under a creative

3
00:00:02 --> 00:00:03
commons license.


4
00:00:03 --> 00:00:06
Your support will help MIT
OpenCourseWare continue to

5
00:00:06 --> 00:00:10
offer high-quality educational
resources for free.

6
00:00:10 --> 00:00:13
To make a donation or view
additional materials from

7
00:00:13 --> 00:00:16
hundreds of MIT courses,
visit MIT OpenCourseWare

8
00:00:16 --> 00:00:21
at ocw.mit.edu.


9
00:00:21 --> 00:00:25
PROFESSOR: Last time we were
talking about binary search and

10
00:00:25 --> 00:00:29
I sort of left a promise to
you which I need to pick up.

11
00:00:29 --> 00:00:34
I want to remind you, we were
talking about search, which is

12
00:00:34 --> 00:00:36
a very fundamental thing that
we do in a whole lot

13
00:00:36 --> 00:00:37
of applications.


14
00:00:37 --> 00:00:39
We want to go find things
in some data set.

15
00:00:39 --> 00:00:41
And I'll remind you that
we sort of a separated

16
00:00:41 --> 00:00:42
out two cases.


17
00:00:42 --> 00:00:48
We said if we had an
ordered list, we could

18
00:00:48 --> 00:00:50
use binary search.


19
00:00:50 --> 00:00:54
And we said that was log
rhythmic, took log n time where

20
00:00:54 --> 00:00:56
n is the size of the list.


21
00:00:56 --> 00:01:01
If it was an unordered list,
we were basically stuck

22
00:01:01 --> 00:01:03
with linear search.


23
00:01:03 --> 00:01:05
Got to walk through the
whole list to see if

24
00:01:05 --> 00:01:05
the thing is there.


25
00:01:05 --> 00:01:08
So that was of order in.


26
00:01:08 --> 00:01:11
And then one of the things that
I suggested was that if we

27
00:01:11 --> 00:01:15
could figure out some way to
order it, and in particular, if

28
00:01:15 --> 00:01:23
we could order it in n log n
time, and we still haven't done

29
00:01:23 --> 00:01:26
that, but if we could do that,
then we said the complexity

30
00:01:26 --> 00:01:27
changed a little bit.


31
00:01:27 --> 00:01:30
But it changed in a way
that I want to remind you.

32
00:01:30 --> 00:01:33
And the change was, that in
this case, if I'm doing a

33
00:01:33 --> 00:01:36
single search, I've
got a choice.

34
00:01:36 --> 00:01:41
I could still do the linear
case, which is order n or I

35
00:01:41 --> 00:01:44
could say, look, take the
list, let's sort it

36
00:01:44 --> 00:01:45
and then search it.


37
00:01:45 --> 00:01:50
But in that case, we said well
to sort it was going to take n

38
00:01:50 --> 00:01:52
log n time, assuming
I can do that.

39
00:01:52 --> 00:01:59
Once I have it sorted I can
search it in log n time, but

40
00:01:59 --> 00:02:02
that's still isn't as
good as just doing n.

41
00:02:02 --> 00:02:09
And this led to this idea of
amortization, which is I need

42
00:02:09 --> 00:02:12
to not only factor in the cost,
but how am I going to use it?

43
00:02:12 --> 00:02:14
And typically, I'm not going to
just search once in a list, I'm

44
00:02:14 --> 00:02:16
going to search multiple times.


45
00:02:16 --> 00:02:21
So if I have to do k searches,
then in the linear case, I got

46
00:02:21 --> 00:02:23
to do order n things k times.


47
00:02:23 --> 00:02:25
It's order k n.


48
00:02:25 --> 00:02:30
Whereas in the ordered case, I
need to get them sorted, which

49
00:02:30 --> 00:02:34
is still n log n, but then
the search is only log n.

50
00:02:34 --> 00:02:38
I need to do k of those.


51
00:02:38 --> 00:02:44
And we suggested well this
is better than that.

52
00:02:44 --> 00:02:49
This is certainly better than
that. m plus k all times log

53
00:02:49 --> 00:02:51
n is in general going to be
much better than k times n.

54
00:02:51 --> 00:02:54
It depends on n and k but
obviously as n gets big, that

55
00:02:54 --> 00:02:56
one is going to be better.


56
00:02:56 --> 00:03:00
And that's just a way of
reminding you that we want to

57
00:03:00 --> 00:03:02
think carefully, but what are
the things we're trying

58
00:03:02 --> 00:03:04
to measure when we talk
about complexity here?

59
00:03:04 --> 00:03:06
It's both the size of the
thing and how often are

60
00:03:06 --> 00:03:07
we going to use it?


61
00:03:07 --> 00:03:10
And there are some trade offs,
but I still haven't said how

62
00:03:10 --> 00:03:13
I'm going to get an n log n
sorting algorithm, and that's

63
00:03:13 --> 00:03:14
what I want to do today.


64
00:03:14 --> 00:03:17
One of the two things
I want to do today.

65
00:03:17 --> 00:03:19
To set the stage for this,
let's go back just for a

66
00:03:19 --> 00:03:22
second to binary search.


67
00:03:22 --> 00:03:26
At the end of the lecture I
said binary search was an

68
00:03:26 --> 00:03:39
example of a divide and
conquer algorithm.

69
00:03:39 --> 00:03:42
Sort of an Attila the Hun
kind of approach to doing

70
00:03:42 --> 00:03:43
things if you like.


71
00:03:43 --> 00:03:46
So let me say -- boy, I could
have made a really bad

72
00:03:46 --> 00:03:48
political joke there, which
I will forego, right.

73
00:03:48 --> 00:03:50
Let's say what this actually
means, divide and conquer.

74
00:03:50 --> 00:03:53
Divide and conquer says
basically do the following:

75
00:03:53 --> 00:04:14
split the problem into several
sub-problems of the same type.

76
00:04:14 --> 00:04:15
I'll come back in a second
to help binary searches

77
00:04:15 --> 00:04:17
matches in that, but that's
what we're going to do.

78
00:04:17 --> 00:04:22
For each of those sub-problems
we're going to solve them

79
00:04:22 --> 00:04:27
independently, and then we're
going to combine

80
00:04:27 --> 00:04:34
those solutions.


81
00:04:34 --> 00:04:36
And it's called divide and
conquer for the obvious reason.

82
00:04:36 --> 00:04:39
I'm going to divide it up into
sub-problems with the hope that

83
00:04:39 --> 00:04:41
those sub-problems get easier.


84
00:04:41 --> 00:04:43
It's going to be easier to
conquer if you like, and then

85
00:04:43 --> 00:04:45
I'm going to merge them back.


86
00:04:45 --> 00:04:47
Now, in the binary search
case, in some sense, this

87
00:04:47 --> 00:04:51
is a little bit trivial.


88
00:04:51 --> 00:04:52
What was the divide?


89
00:04:52 --> 00:04:56
The divide was breaking a big
search up into half a search.

90
00:04:56 --> 00:04:59
We actually threw half of the
list away and we kept dividing

91
00:04:59 --> 00:05:01
it down, until ultimately
we got something of

92
00:05:01 --> 00:05:03
size one to search.


93
00:05:03 --> 00:05:04
That's really easy.


94
00:05:04 --> 00:05:07
The combination was also sort
of trivial in this case because

95
00:05:07 --> 00:05:11
the solution to the sub-problem
was, in fact, the solution

96
00:05:11 --> 00:05:13
to the larger problem.


97
00:05:13 --> 00:05:15
But there's the idea of
divide and conquer.

98
00:05:15 --> 00:05:18
I'm going to use exactly that
same ideas to tackle sort.

99
00:05:18 --> 00:05:20
Again, I've got an unordered
list of n elements.

100
00:05:20 --> 00:05:24
I want to sort it into a
obviously a sorted list.

101
00:05:24 --> 00:05:27
And that particular algorithm
is actually a really nice

102
00:05:27 --> 00:05:29
algorithm called merge sort.


103
00:05:29 --> 00:05:37
And it's actually a
fairly old algorithm.

104
00:05:37 --> 00:05:41
It was invented in 1945 by
John von Neumann one of the

105
00:05:41 --> 00:05:46
pioneers of computer science.


106
00:05:46 --> 00:05:50
And here's the idea behind
merge sort, actually I'm going

107
00:05:50 --> 00:05:53
to back into it in a funny way.


108
00:05:53 --> 00:05:56
Let's assume that I could
somehow get to the stage where

109
00:05:56 --> 00:05:59
I've got two sorted lists.


110
00:05:59 --> 00:06:02
How much work do I have
to do to actually

111
00:06:02 --> 00:06:04
merge them together?


112
00:06:04 --> 00:06:05
So let me give you an example.


113
00:06:05 --> 00:06:13
Suppose I want to merge two
lists, and they're sorted.

114
00:06:13 --> 00:06:22
Just to give you an example,
here's one list, 3121724

115
00:06:22 --> 00:06:25
Here's another list, 12430.


116
00:06:25 --> 00:06:29
I haven't said how I'm going to
get those sorted lists, but

117
00:06:29 --> 00:06:31
imagine I had two sorted
lists like that.

118
00:06:31 --> 00:06:34
How hard is it to merge them?


119
00:06:34 --> 00:06:35
Well it's pretty easy, right?


120
00:06:35 --> 00:06:38
I start at the beginning of
each list, and I say is

121
00:06:38 --> 00:06:39
one less than three?


122
00:06:39 --> 00:06:41
Sure.


123
00:06:41 --> 00:06:45
So that says one should be the
first element in my merge list.

124
00:06:45 --> 00:06:49
Now, compare the first element
in each of these lists.

125
00:06:49 --> 00:06:53
Two is less than three, so
two ought to be the next

126
00:06:53 --> 00:06:54
element of the list.


127
00:06:54 --> 00:06:55
And you get the idea.


128
00:06:55 --> 00:06:56
What am I going to do next?


129
00:06:56 --> 00:07:00
I'm going to compare
three against four.

130
00:07:00 --> 00:07:04
Three is the smallest one, and
I'm going to compare four games

131
00:07:04 --> 00:07:07
twelve, which is going
to give me four.

132
00:07:07 --> 00:07:07
And then what do?


133
00:07:07 --> 00:07:10
I have to do twelve
against thirty, twelve is

134
00:07:10 --> 00:07:15
smaller, take that out.


135
00:07:15 --> 00:07:25
Seventeen against thirty,
twenty-four against thirty And

136
00:07:25 --> 00:07:27
by this stage I've got nothing
left in this element, so I just

137
00:07:27 --> 00:07:31
add the rest of that list in.


138
00:07:31 --> 00:07:33
Wow I can sort two lists,
so I can merge two lists.

139
00:07:33 --> 00:07:35
I said it poorly.


140
00:07:35 --> 00:07:37
What's the point?


141
00:07:37 --> 00:07:40
How many operations did
it take me to do this?

142
00:07:40 --> 00:07:41
Seven comparisons, right?


143
00:07:41 --> 00:07:41
I've got eight elements.


144
00:07:41 --> 00:07:48
It took me seven comparisons,
because I can take advantage of

145
00:07:48 --> 00:07:50
the fact I know I only ever
have to look at the first

146
00:07:50 --> 00:07:53
element of each sub-list.


147
00:07:53 --> 00:07:54
Those are the only things I
need to compare, and when I run

148
00:07:54 --> 00:07:58
out of one list, I just add
the rest of the list in.

149
00:07:58 --> 00:08:02
What's the order of
complexity of merging?

150
00:08:02 --> 00:08:03
I heard it somewhere
very quietly.

151
00:08:03 --> 00:08:05
STUDENT: n.


152
00:08:05 --> 00:08:06
PROFESSOR: Sorry,
and thank you.

153
00:08:06 --> 00:08:08
Linear, absolutely right?


154
00:08:08 --> 00:08:10
And what's n by the way here?


155
00:08:10 --> 00:08:11
What's it measuring?


156
00:08:11 --> 00:08:14
STUDENT: [UNINTELLIGIBLE]


157
00:08:14 --> 00:08:15
PROFESSOR: In both
lists, right.

158
00:08:15 --> 00:08:21
So this is linear, order n and
n is this sum of the element,

159
00:08:21 --> 00:08:30
or sorry, the number of
elements in each list.

160
00:08:30 --> 00:08:32
I said I was going to
back my way into this.

161
00:08:32 --> 00:08:37
That gives me a way
to merge things.

162
00:08:37 --> 00:08:40
So here's what merge
sort would do.

163
00:08:40 --> 00:08:47
Merge sort takes this idea of
divide and conquer, and it does

164
00:08:47 --> 00:08:58
the following: it says let's
divide the list in half.

165
00:08:58 --> 00:09:00
There's the divide and conquer.


166
00:09:00 --> 00:09:05
And let's keep dividing each of
those lists in half until we

167
00:09:05 --> 00:09:07
get down to something that's
really easy to sort.

168
00:09:07 --> 00:09:08
What's the simplest
thing to sort?

169
00:09:08 --> 00:09:12
A list of size one, right?


170
00:09:12 --> 00:09:26
So continue until we
have singleton lists.

171
00:09:26 --> 00:09:29
Once I got a list of size
one they're sorted,

172
00:09:29 --> 00:09:31
and then combine them.


173
00:09:31 --> 00:09:36
Combine them by doing
emerge the sub-lists.

174
00:09:36 --> 00:09:42
And again, you see that flavor.


175
00:09:42 --> 00:09:45
I'm going to just keep dividing
it up until I get something

176
00:09:45 --> 00:09:47
really easy, and then
I'm going to combine.

177
00:09:47 --> 00:09:49
And this is different than
binary search now, the

178
00:09:49 --> 00:09:50
combine is going to
have to do some work.

179
00:09:50 --> 00:09:55
So, I'm giving you a piece of
code that does this, and I'm

180
00:09:55 --> 00:09:57
going to come back to it in the
second, but it's up there.

181
00:09:57 --> 00:10:00
But what I'd like to do is to
try you sort sort of a little

182
00:10:00 --> 00:10:02
simulation of how
this would work.

183
00:10:02 --> 00:10:03
And I was going to originally
make the TAs come up here and

184
00:10:03 --> 00:10:06
do it, but I don't have enough
t a's to do a full merge sort.

185
00:10:06 --> 00:10:10
So I'm hoping, so I also have
these really high-tech props.

186
00:10:10 --> 00:10:12
I spent tons and tons
of department money on

187
00:10:12 --> 00:10:13
them as you can see.


188
00:10:13 --> 00:10:15
I hope you can see this because
I'm going to try and simulate

189
00:10:15 --> 00:10:16
what a merge sort does.


190
00:10:16 --> 00:10:19
I've got eight things I want to
sort here, and those initially

191
00:10:19 --> 00:10:21
start out here at top level.


192
00:10:21 --> 00:10:23
The first step is
divide them in half.

193
00:10:23 --> 00:10:27
All right?


194
00:10:27 --> 00:10:30
I'm not sure how to mark
it here, remember I need

195
00:10:30 --> 00:10:31
to come back there.


196
00:10:31 --> 00:10:33
I'm not yet done.


197
00:10:33 --> 00:10:33
What do I do?


198
00:10:33 --> 00:10:41
Divide them in half again.


199
00:10:41 --> 00:10:42
You know, if I had like
shells and peas here I

200
00:10:42 --> 00:10:44
could make some more money.


201
00:10:44 --> 00:10:44
What do I do?


202
00:10:44 --> 00:10:50
I divide them in
half one more time.

203
00:10:50 --> 00:10:53
Let me cluster them because
really what I have,

204
00:10:53 --> 00:10:56
sorry, separate them out.


205
00:10:56 --> 00:10:59
I've gone from one problem
size eight down to eight

206
00:10:59 --> 00:11:01
problems of size one.


207
00:11:01 --> 00:11:03
At this stage I'm at
my singleton case.

208
00:11:03 --> 00:11:04
So this is easy.


209
00:11:04 --> 00:11:04
What do I do?


210
00:11:04 --> 00:11:05
I merge.


211
00:11:05 --> 00:11:17
And the merge is,
put them in order.

212
00:11:17 --> 00:11:18
What do I do next?


213
00:11:18 --> 00:11:19
Obvious thing, I merge these.


214
00:11:19 --> 00:11:22
And that as we saw was a
nice linear operation.

215
00:11:22 --> 00:11:31
It's fun to do it upside down,
and then one more merge which

216
00:11:31 --> 00:11:34
is I take the smallest
elements of each one until

217
00:11:34 --> 00:11:40
I get to where I want.


218
00:11:40 --> 00:11:42
Wow aren't you impressed.


219
00:11:42 --> 00:11:45
No, don't please don't
clap, not for that one.

220
00:11:45 --> 00:11:48
Now let me do it a second
time to show you that --

221
00:11:48 --> 00:11:49
I'm saying this poorly.


222
00:11:49 --> 00:11:50
Let me say it again.


223
00:11:50 --> 00:11:52
That's the general idea.


224
00:11:52 --> 00:11:53
What should you
see out of that?

225
00:11:53 --> 00:11:56
I just kept sub-dividing down
until I got really easy

226
00:11:56 --> 00:11:59
problems, and then I
combine them back.

227
00:11:59 --> 00:12:02
I actually misled you slightly
there or maybe a lot, because

228
00:12:02 --> 00:12:03
I did it in parallel.


229
00:12:03 --> 00:12:06
In fact, let me just shuffle
these up a little bit.

230
00:12:06 --> 00:12:08
Really what's going to happen
here, because this is a

231
00:12:08 --> 00:12:11
sequential computer, is that
we're going to start off up

232
00:12:11 --> 00:12:19
here, at top level, we're going
to divide into half, then we're

233
00:12:19 --> 00:12:21
going to do the complete
subdivision and merge

234
00:12:21 --> 00:12:24
here before we ever come
back and do this one.

235
00:12:24 --> 00:12:30
We're going to do a division
here and then a division there.

236
00:12:30 --> 00:12:32
At that stage we can merge
these, and then take this

237
00:12:32 --> 00:12:35
down, do the division merge
and bring them back up.

238
00:12:35 --> 00:12:41
Let me show you an
example by running that.

239
00:12:41 --> 00:12:44
I've got a little list I've
made here called test.

240
00:12:44 --> 00:12:53
Let's run merge sort on it, and
then we'll look at the code.

241
00:12:53 --> 00:12:57
OK, what I would like you to
see is I've been printing out,

242
00:12:57 --> 00:12:59
as I went along, actually let's
back up slightly and

243
00:12:59 --> 00:13:00
look at the code.


244
00:13:00 --> 00:13:03
There's merge sort.


245
00:13:03 --> 00:13:04
Takes in a list.


246
00:13:04 --> 00:13:04
What does it say to do?


247
00:13:04 --> 00:13:07
It says check to see if
I'm in that base case.

248
00:13:07 --> 00:13:10
It's the list of
length less than two.

249
00:13:10 --> 00:13:11
Is it one basically?


250
00:13:11 --> 00:13:16
In which case, just
return a copy the list.

251
00:13:16 --> 00:13:17
That's the simple case.


252
00:13:17 --> 00:13:18
Otherwise, notice
what it says to do.

253
00:13:18 --> 00:13:24
It's says find the mid-point
and split the list in half.

254
00:13:24 --> 00:13:27
Copy of the back end, sorry,
copy of the left side,

255
00:13:27 --> 00:13:28
copy of the right side.


256
00:13:28 --> 00:13:30
Run merge sort on those.


257
00:13:30 --> 00:13:32
By induction, if it does the
right thing, I'm going to get

258
00:13:32 --> 00:13:37
back two lists, and I'm going
to then merge them together.

259
00:13:37 --> 00:13:37
Notice what I'm going to do.


260
00:13:37 --> 00:13:40
I'm going to print here the
list if we go into it, and

261
00:13:40 --> 00:13:44
print of the when we're done
and then just return that.

262
00:13:44 --> 00:13:45
Merge up here.


263
00:13:45 --> 00:13:46
There's a little
more code there.

264
00:13:46 --> 00:13:48
I'll let you just grok it but
you can see it's basically

265
00:13:48 --> 00:13:51
doing what I did over there.


266
00:13:51 --> 00:13:54
Setting up two indices for the
two sub-list, it's just walking

267
00:13:54 --> 00:13:56
down, finding the smallest
element, putting it

268
00:13:56 --> 00:13:57
into a new list.


269
00:13:57 --> 00:14:01
When it gets to the end of one
of the lists, it skips to the

270
00:14:01 --> 00:14:03
next part, and only one of
these two pieces will get

271
00:14:03 --> 00:14:05
called because only one of
them is going to have

272
00:14:05 --> 00:14:06
things leftovers.


273
00:14:06 --> 00:14:08
It's going to add the
other pieces in.

274
00:14:08 --> 00:14:10
OK, if you look at that
then, let's look at what

275
00:14:10 --> 00:14:11
happened when we ran this.


276
00:14:11 --> 00:14:16
We started off with a
call with that list.

277
00:14:16 --> 00:14:19
Ah ha, split it in half.


278
00:14:19 --> 00:14:21
It's going down the
left side of this.

279
00:14:21 --> 00:14:23
That got split in half, and
that got split in half until

280
00:14:23 --> 00:14:27
I got to a list of one.


281
00:14:27 --> 00:14:28
Here's the first
list of size one.

282
00:14:28 --> 00:14:30
There's the second
list of size one.

283
00:14:30 --> 00:14:31
So I merged them.


284
00:14:31 --> 00:14:33
It's now in the right
order, and that's coming

285
00:14:33 --> 00:14:35
from right there.


286
00:14:35 --> 00:14:37
Having done that, it goes
back up and picks the

287
00:14:37 --> 00:14:42
second sub-list, which
came from there.

288
00:14:42 --> 00:14:44
It's a down to base
case, merges it.

289
00:14:44 --> 00:14:48
When these two merges are done,
we're basically at a stage in

290
00:14:48 --> 00:14:51
that branch where we can now
merge those two together, which

291
00:14:51 --> 00:14:56
gives us that, and it goes
through the rest of it.

292
00:14:56 --> 00:15:00
A really nice algorithm.


293
00:15:00 --> 00:15:03
As I said, an example
of divide and conquer.

294
00:15:03 --> 00:15:06
Notice here that it's different
than the binary search case.

295
00:15:06 --> 00:15:10
We're certainly dividing down,
but the combination now

296
00:15:10 --> 00:15:12
actually takes some work.


297
00:15:12 --> 00:15:13
I'll have to actually
figure out how to put

298
00:15:13 --> 00:15:15
them back together.


299
00:15:15 --> 00:15:17
And that's a general thing you
want to keep in mind when

300
00:15:17 --> 00:15:19
you're thinking about designing
a divide and conquer

301
00:15:19 --> 00:15:21
kind of algorithm.


302
00:15:21 --> 00:15:23
You really want to get the
power of dividing things up,

303
00:15:23 --> 00:15:26
but if you end up doing a ton
of work at the combination

304
00:15:26 --> 00:15:28
stage, you may not
have gained anything.

305
00:15:28 --> 00:15:31
So you really want to think
about that trade off.

306
00:15:31 --> 00:15:38
All right, having said that,
what's the complexity here?

307
00:15:38 --> 00:15:40
Boy, there's a dumb question,
because I've been telling you

308
00:15:40 --> 00:15:42
for the last two lectures the
complexity is n log n, but

309
00:15:42 --> 00:15:43
let's see if it really is.


310
00:15:43 --> 00:15:46
What's the complexity here?


311
00:15:46 --> 00:16:01
If we think about it, we start
off with the problem of size n.

312
00:16:01 --> 00:16:02
What do we do?


313
00:16:02 --> 00:16:05
We split it into two
problems of size n over 2.

314
00:16:05 --> 00:16:08
Those get split each into two
problems of size n over 4, and

315
00:16:08 --> 00:16:15
we keep doing that until we get
down to a level in this

316
00:16:15 --> 00:16:20
tree where we have only
singletons left over.

317
00:16:20 --> 00:16:23
Once we're there, we
have to do the merge.

318
00:16:23 --> 00:16:24
Notice what happens here.


319
00:16:24 --> 00:16:30
We said each of the merge
operations was of order n.

320
00:16:30 --> 00:16:31
But n is different.


321
00:16:31 --> 00:16:31
Right?


322
00:16:31 --> 00:16:33
Down here, I've just got two
things to merge, and then

323
00:16:33 --> 00:16:35
I've got things of size two
to merge and then things

324
00:16:35 --> 00:16:37
of size four to merge.


325
00:16:37 --> 00:16:38
But notice a trade off.


326
00:16:38 --> 00:16:43
I have n operations if you
like down there of size one.

327
00:16:43 --> 00:16:47
Up here I have n over two
operations of size two.

328
00:16:47 --> 00:16:50
Up here I've got n over four
operations of size four.

329
00:16:50 --> 00:16:54
So I always have to do
a merge of n elements.

330
00:16:54 --> 00:16:57
How much time does that take?


331
00:16:57 --> 00:17:01
Well, we said it, right?


332
00:17:01 --> 00:17:02
Where did I put it?


333
00:17:02 --> 00:17:04
Right there, order n.


334
00:17:04 --> 00:17:16
So I have order n operations
at each level in the tree.

335
00:17:16 --> 00:17:17
And then how many
levels deep am I?

336
00:17:17 --> 00:17:20
Well, that's the divide, right?


337
00:17:20 --> 00:17:26
So how many levels do I have?


338
00:17:26 --> 00:17:29
Log n, because at each
stage I'm cutting

339
00:17:29 --> 00:17:30
the problem in half.


340
00:17:30 --> 00:17:31
So I start off with n then
it's n over two n over

341
00:17:31 --> 00:17:33
four n over eight.


342
00:17:33 --> 00:17:40
So I have n operations log n
times, there we go, n log n.

343
00:17:40 --> 00:17:42
Took us a long time to get
there, but it's a nice

344
00:17:42 --> 00:17:45
algorithm to have.


345
00:17:45 --> 00:17:51
Let me generalize
this slightly.

346
00:17:51 --> 00:17:56
When we get a problem, a
standard tool to try and attack

347
00:17:56 --> 00:17:59
it with is to say, is there
some way to break this problem

348
00:17:59 --> 00:18:02
down into simpler, I shouldn't
say simpler, smaller versions

349
00:18:02 --> 00:18:05
of the same problem.


350
00:18:05 --> 00:18:07
If I can do that, it's
a good candidate for

351
00:18:07 --> 00:18:08
divide and conquer.


352
00:18:08 --> 00:18:10
And then the things I have
to ask is how much of a

353
00:18:10 --> 00:18:12
division do I want to do?


354
00:18:12 --> 00:18:15
The obvious one is to divide it
in half, but there may be cases

355
00:18:15 --> 00:18:16
where there are different
divisions you want

356
00:18:16 --> 00:18:18
to have take place.


357
00:18:18 --> 00:18:21
The second question I want to
ask is what's the base case?

358
00:18:21 --> 00:18:24
When do I get down to a problem
that's small enough that it's

359
00:18:24 --> 00:18:26
basically trivial to solve?


360
00:18:26 --> 00:18:28
Here it was lists of size one.


361
00:18:28 --> 00:18:30
I could have stopped at
lists of size two right.

362
00:18:30 --> 00:18:31
That's an easy comparison.


363
00:18:31 --> 00:18:34
Do one comparison and return
one of two possible orders on

364
00:18:34 --> 00:18:36
it, but I need to decide that.


365
00:18:36 --> 00:18:39
And the third thing I need to
decide is how do I combine?

366
00:18:39 --> 00:18:42
You know, point out to you
in the binary search case,

367
00:18:42 --> 00:18:44
combination was trivial.


368
00:18:44 --> 00:18:45
The answer to the final
search was just the

369
00:18:45 --> 00:18:47
answer all the way up.


370
00:18:47 --> 00:18:49
Here, a little more work,
and that's why I'll

371
00:18:49 --> 00:18:50
come back to that idea.


372
00:18:50 --> 00:18:54
If I'm basically just squeezing
jello, that is, I'm trying to

373
00:18:54 --> 00:18:56
make the problem simpler, but
the combination turns out to be

374
00:18:56 --> 00:18:59
really complex, I've
not gained anything.

375
00:18:59 --> 00:19:02
So things that are good
candidates for divide and

376
00:19:02 --> 00:19:05
conquer are problems where it's
easy to figure out how to

377
00:19:05 --> 00:19:09
divide down, and the
combination is of

378
00:19:09 --> 00:19:09
little complexity.


379
00:19:09 --> 00:19:12
It would be nice if it was less
than linear, but linear is nice

380
00:19:12 --> 00:19:15
because then I'm going to get
that n log in kind of behavior.

381
00:19:15 --> 00:19:18
And if you ask the TAs in
recitation tomorrow, they'll

382
00:19:18 --> 00:19:20
tell you that you see a lot
of n log n algorithms

383
00:19:20 --> 00:19:21
in computer science.


384
00:19:21 --> 00:19:23
It's a very common class
of algorithms, and it's

385
00:19:23 --> 00:19:28
very useful one to have.


386
00:19:28 --> 00:19:32
Now, one of the questions we
could still ask is, right,

387
00:19:32 --> 00:19:36
we've got binary search, which
has got this nice log behavior.

388
00:19:36 --> 00:19:40
If we can sort things, you
know, we get this n log n

389
00:19:40 --> 00:19:43
behavior, and we got a n
log n behavior overall.

390
00:19:43 --> 00:19:47
But can we actually do better
in terms of searching.

391
00:19:47 --> 00:19:49
I'm going to show you
one last technique.

392
00:19:49 --> 00:19:53
And in fact, we're going to put
quotes around the word better,

393
00:19:53 --> 00:19:58
but it does better than even
this kind of binary search, and

394
00:19:58 --> 00:20:04
that's a method called hashing.


395
00:20:04 --> 00:20:08
You've actually seen hashing,
you just don't know it.

396
00:20:08 --> 00:20:10
Hashing is the the technique
that's used in Python to

397
00:20:10 --> 00:20:12
represent dictionaries.


398
00:20:12 --> 00:20:15
Hashing is used when you
actually come in to Logan

399
00:20:15 --> 00:20:18
Airport and Immigration or
Homeland Security checks your

400
00:20:18 --> 00:20:20
picture against a database.


401
00:20:20 --> 00:20:24
Hashing is used every time you
enter a password into a system.

402
00:20:24 --> 00:20:26
So what in the
world is hashing?

403
00:20:26 --> 00:20:29
Well, let me start with a
simple little example.

404
00:20:29 --> 00:20:35
Suppose I want to represent
a collection of integers.

405
00:20:35 --> 00:20:38
This is an easy little example.


406
00:20:38 --> 00:20:41
And I promise you that the
integers are never going to be

407
00:20:41 --> 00:20:45
anything other than between
the range of zero to nine.

408
00:20:45 --> 00:20:47
OK, so it might be the
collection of one and five.

409
00:20:47 --> 00:20:48
It might be two,
three, four, eight.

410
00:20:48 --> 00:20:50
I mean some collection of
integers, but I guarantee you

411
00:20:50 --> 00:20:52
it's between zero and nine.


412
00:20:52 --> 00:20:55
Here's the trick I can play.


413
00:20:55 --> 00:21:11
I can build -- I can't count --
I could build a list with spots

414
00:21:11 --> 00:21:14
for all of those elements,
zero, one, two, three, four,

415
00:21:14 --> 00:21:16
five, six, seven, eight, nine.


416
00:21:16 --> 00:21:19
And then when I want to create
my set, I could simply put a

417
00:21:19 --> 00:21:26
one everywhere that
that integer falls.

418
00:21:26 --> 00:21:28
So if I wanted to represent,
for example, this is the set

419
00:21:28 --> 00:21:35
two, six and eight, I put
a one in those slots.

420
00:21:35 --> 00:21:37
This seems a little weird, but
bear with me for second, in

421
00:21:37 --> 00:21:40
fact, I've given you a little
piece a code to do it,

422
00:21:40 --> 00:21:45
which is the next piece of
code on the hand out.

423
00:21:45 --> 00:21:48
So let's take a look
at it for second.

424
00:21:48 --> 00:21:53
This little set of code here
from create insert and number.

425
00:21:53 --> 00:21:54
What's create do?


426
00:21:54 --> 00:21:55
It says, given a low and a
high range, in this case

427
00:21:55 --> 00:21:57
it would be zero to nine.


428
00:21:57 --> 00:22:00
I'm going to build a list.


429
00:22:00 --> 00:22:02
Right, you can see that little
loop going through there.

430
00:22:02 --> 00:22:03
What am I doing?


431
00:22:03 --> 00:22:07
I'm creating a list with just
that special symbol none in it.

432
00:22:07 --> 00:22:08
So I'm building the list.


433
00:22:08 --> 00:22:10
I'm returning that as my set.


434
00:22:10 --> 00:22:11
And then to create the
object, I'll simply

435
00:22:11 --> 00:22:13
do a set of inserts.


436
00:22:13 --> 00:22:15
If I want the values two, six
and eight in there, I would do

437
00:22:15 --> 00:22:18
an insert of two into that set,
an insert of six into that

438
00:22:18 --> 00:22:20
set, and an insert of
eight into the set.

439
00:22:20 --> 00:22:21
And what does it do?


440
00:22:21 --> 00:22:25
It marks a one in
each of those spots.

441
00:22:25 --> 00:22:26
Now, what did I want to do?


442
00:22:26 --> 00:22:27
I wanted to check membership.


443
00:22:27 --> 00:22:28
I want to do search.


444
00:22:28 --> 00:22:30
Well that's simple.


445
00:22:30 --> 00:22:32
Given that representation
and some value, I just

446
00:22:32 --> 00:22:36
say gee is it there?


447
00:22:36 --> 00:22:43
What's the order
complexity here?

448
00:22:43 --> 00:22:45
I know I drive you nuts
asking questions?

449
00:22:45 --> 00:22:48
What's the order
complexity here?

450
00:22:48 --> 00:22:57
Quadratic, linear,
log, constant?

451
00:22:57 --> 00:22:58
Any takers?


452
00:22:58 --> 00:23:00
I know I have the wrong glasses
on the see hands up too, but...

453
00:23:00 --> 00:23:04
STUDENT: [UNINTELLIGIBLE]


454
00:23:04 --> 00:23:05
PROFESSOR: Who said it?


455
00:23:05 --> 00:23:06
STUDENT: Constant.


456
00:23:06 --> 00:23:09
PROFESSOR: Constant, why?


457
00:23:09 --> 00:23:09
STUDENT: [UNINTELLIGIBLE]


458
00:23:09 --> 00:23:10
PROFESSOR: Yes, thank you.


459
00:23:10 --> 00:23:11
All right, it is constant.


460
00:23:11 --> 00:23:14
You keep sitting back there
where I can't get to you.

461
00:23:14 --> 00:23:15
Thank you very much.


462
00:23:15 --> 00:23:17
It has a constant.


463
00:23:17 --> 00:23:21
Remember we said we design
lists so that the access, no

464
00:23:21 --> 00:23:24
matter where it was on the
list was of constant time.

465
00:23:24 --> 00:23:27
That is another way of
saying that looking up this

466
00:23:27 --> 00:23:29
thing here is constant.


467
00:23:29 --> 00:23:35
So this is constant
time, order one.

468
00:23:35 --> 00:23:37
Come on, you know, representing
sets of integers,

469
00:23:37 --> 00:23:38
this is pretty dumb.


470
00:23:38 --> 00:23:41
Suppose I want to have
a set of characters.

471
00:23:41 --> 00:23:42
How could I do that?


472
00:23:42 --> 00:23:44
Well the idea of a hash, in
fact, what's called a hash

473
00:23:44 --> 00:23:48
function is to have some way
of mapping any kind of

474
00:23:48 --> 00:23:50
data into integers.


475
00:23:50 --> 00:23:55
So let's look at the second
example, all right, -- I keep

476
00:23:55 --> 00:24:01
doing that -- this piece of
code from here to here gives

477
00:24:01 --> 00:24:06
me a way of now creating
a hash table of size 256.

478
00:24:06 --> 00:24:09
Ord as a built in
python representation.

479
00:24:09 --> 00:24:12
There is lots of them around
that takes any character and

480
00:24:12 --> 00:24:13
gives you back an integer.


481
00:24:13 --> 00:24:17
In fact, just to show that to
you, if I go down here and I

482
00:24:17 --> 00:24:32
type ord, sorry, I
did that wrong.

483
00:24:32 --> 00:24:33
Let me try again.


484
00:24:33 --> 00:24:38
We'll get to exceptions
in a second.

485
00:24:38 --> 00:24:39
I give it some character.


486
00:24:39 --> 00:24:42
It gives me back an
integer representing.

487
00:24:42 --> 00:24:43
It looks weird.


488
00:24:43 --> 00:24:44
Why is three come back
to some other thing?

489
00:24:44 --> 00:24:46
That's the internal
representation that

490
00:24:46 --> 00:24:47
python uses for this.


491
00:24:47 --> 00:24:51
If I give it some other
character, yeah, it would help

492
00:24:51 --> 00:24:54
if I could type, give it
some other character.

493
00:24:54 --> 00:24:57
It gives me back a
representation.

494
00:24:57 --> 00:24:59
So now here's the idea.


495
00:24:59 --> 00:25:04
I build a list 256 elements
long, and I fill it up with

496
00:25:04 --> 00:25:06
those special characters none.


497
00:25:06 --> 00:25:09
That's what create is
going to do right here.

498
00:25:09 --> 00:25:14
And then hash character takes
in any string or character,

499
00:25:14 --> 00:25:16
single character, gives
me back a number.

500
00:25:16 --> 00:25:17
Notice what I do.


501
00:25:17 --> 00:25:21
If I want to create a set or a
sequence representing these

502
00:25:21 --> 00:25:24
things, I simply insert
into that list.

503
00:25:24 --> 00:25:26
It goes through and puts
ones in the right place.

504
00:25:26 --> 00:25:29
And then, if I want to find
out if something's there,

505
00:25:29 --> 00:25:30
I do the same thing.


506
00:25:30 --> 00:25:33
But notice now, hash is
converting the input

507
00:25:33 --> 00:25:38
into an integer.


508
00:25:38 --> 00:25:40
So, what's the idea?


509
00:25:40 --> 00:25:44
If I know what my hash function
does, it maps, in this case

510
00:25:44 --> 00:25:49
characters into a range zero to
256, which is zero to 255,

511
00:25:49 --> 00:25:52
I create a list that long,
and I simply mark things.

512
00:25:52 --> 00:25:55
And my look up is
still constant.

513
00:25:55 --> 00:25:56
Characters are simple.


514
00:25:56 --> 00:25:59
Suppose you want to represent
sets of strings, well you

515
00:25:59 --> 00:26:01
basically just generalize
the hash function.

516
00:26:01 --> 00:26:03
I think one of the classic ones
for strings is called the

517
00:26:03 --> 00:26:06
Rabin-Karp algorithm.


518
00:26:06 --> 00:26:09
And it's simply the same idea
that you have a mapping from

519
00:26:09 --> 00:26:13
your import into a
set of integers.

520
00:26:13 --> 00:26:17
Wow, OK, maybe not so wow,
but this is now constant.

521
00:26:17 --> 00:26:19
This is constant time access.


522
00:26:19 --> 00:26:24
So I can do searching in
constant time which is great.

523
00:26:24 --> 00:26:26
Where's the penalty?


524
00:26:26 --> 00:26:28
What did I trade off here?


525
00:26:28 --> 00:26:31
Well I'm going to suggest
that what I did was I really

526
00:26:31 --> 00:26:40
traded space for time.


527
00:26:40 --> 00:26:44
It makes me sound like an astro
physicist somehow right?

528
00:26:44 --> 00:26:45
What do I mean by that?


529
00:26:45 --> 00:26:49
I have constant time access
which is great, but I paid a

530
00:26:49 --> 00:26:52
price, which is I had
to use up some space.

531
00:26:52 --> 00:26:55
In the case of
integers it was easy.

532
00:26:55 --> 00:26:57
In the case of characters,
so I have to give up a

533
00:26:57 --> 00:26:59
list of 256, no big deal.


534
00:26:59 --> 00:27:01
Imagine now you
want to do faces.

535
00:27:01 --> 00:27:03
You've got a picture of
somebody's face, it's

536
00:27:03 --> 00:27:04
a million pixels.


537
00:27:04 --> 00:27:07
Each pixel has a range of
values from zero to 256.

538
00:27:07 --> 00:27:11
I want to hash a face with some
function into an integer.

539
00:27:11 --> 00:27:13
I may not want to do the full
range of this, but I may

540
00:27:13 --> 00:27:16
decide I have to use a lot
of gigabytes of space in

541
00:27:16 --> 00:27:18
order to do a trade off.


542
00:27:18 --> 00:27:20
The reason I'm showing you this
is it that this is a gain, a

543
00:27:20 --> 00:27:22
common trade off in
computer science.

544
00:27:22 --> 00:27:25
That in many cases, I can gain
efficiency if I'm willing

545
00:27:25 --> 00:27:28
to give up space.


546
00:27:28 --> 00:27:30
Having said that though, there
may still be a problem, or

547
00:27:30 --> 00:27:32
there ought to be a problem
that may be bugging you

548
00:27:32 --> 00:27:37
slightly, which is how do I
guarantee that my hash function

549
00:27:37 --> 00:27:43
takes any input into exactly
one spot in the storage space?

550
00:27:43 --> 00:27:45
The answer is I can't.


551
00:27:45 --> 00:27:48
OK, in the simple case of
integers I can, but in the case

552
00:27:48 --> 00:27:51
of something more complex like
faces or fingerprints or

553
00:27:51 --> 00:27:54
passwords for that matter, it's
hard to design a hash function

554
00:27:54 --> 00:27:57
that has completely even
distribution, meaning that

555
00:27:57 --> 00:28:00
it takes any input into
exactly one output spot.

556
00:28:00 --> 00:28:04
So what you typically do and a
hash case is you design your

557
00:28:04 --> 00:28:05
code to deal with that.


558
00:28:05 --> 00:28:07
You try to design -- actually
I'm going to come back

559
00:28:07 --> 00:28:07
to that in a second.


560
00:28:07 --> 00:28:09
It's like you're trying to use
a hash function that spread

561
00:28:09 --> 00:28:11
things out pretty evenly.


562
00:28:11 --> 00:28:13
But the places you store into
in those lists may have to

563
00:28:13 --> 00:28:16
themselves have a small list in
there, and when you go to check

564
00:28:16 --> 00:28:19
something, you may have to do a
linear search through the

565
00:28:19 --> 00:28:20
elements in that list.


566
00:28:20 --> 00:28:23
The good news is the elements
in any one spot in a hash table

567
00:28:23 --> 00:28:26
are likely to be a small
number, three, four, five.

568
00:28:26 --> 00:28:27
So the search is really easy.


569
00:28:27 --> 00:28:29
You're not searching
a million things.

570
00:28:29 --> 00:28:31
You're searching three or four
things, but nonetheless, you

571
00:28:31 --> 00:28:34
have to do that trade off.


572
00:28:34 --> 00:28:43
The last thing I want to say
about hashes are that they're

573
00:28:43 --> 00:28:46
actually really hard to create.


574
00:28:46 --> 00:28:48
There's been a lot of work done
on these over the years, but

575
00:28:48 --> 00:28:51
in fact, it's pretty hard to
invent a good hash function.

576
00:28:51 --> 00:28:54
So my advice to you is, if you
want to use something was

577
00:28:54 --> 00:28:56
a hash, go to a library.


578
00:28:56 --> 00:28:57
Look up a good hash function.


579
00:28:57 --> 00:29:00
For strings, there's a
classic set of them

580
00:29:00 --> 00:29:01
that work pretty well.


581
00:29:01 --> 00:29:02
For integers, there are
some real simple ones.

582
00:29:02 --> 00:29:05
If there's something more
complex, find a good hash

583
00:29:05 --> 00:29:07
function, but designing a
really good hash function takes

584
00:29:07 --> 00:29:10
a lot of effort because you
want it to have that

585
00:29:10 --> 00:29:11
even distribution.


586
00:29:11 --> 00:29:16
You'd like it to have as few
duplicates if you like in each

587
00:29:16 --> 00:29:22
spot in the hash table for each
one of the things that you use.

588
00:29:22 --> 00:29:25
Let me pull back
for a second then.

589
00:29:25 --> 00:29:28
What have we done over the
last three or four lectures?

590
00:29:28 --> 00:29:31
We've started introducing you
to classes of algorithms.

591
00:29:31 --> 00:29:35
Things that I'd like you to be
able to see are how to do some

592
00:29:35 --> 00:29:37
simple complexity analysis.


593
00:29:37 --> 00:29:40
Perhaps more importantly,
how to recognize a kind of

594
00:29:40 --> 00:29:42
algorithm based on its
properties and know what

595
00:29:42 --> 00:29:43
class it belongs to.


596
00:29:43 --> 00:29:44
This is a hint.


597
00:29:44 --> 00:29:47
If you like, leaning towards
the next quiz, that you oughta

598
00:29:47 --> 00:29:50
be able to say that looks like
a logarithmic algorithm because

599
00:29:50 --> 00:29:51
it's got a particular property.


600
00:29:51 --> 00:29:53
That looks like an n log n
algorithm because it has

601
00:29:53 --> 00:29:54
a particular property.


602
00:29:54 --> 00:29:57
And the third thing we've done
is we've given you now a

603
00:29:57 --> 00:30:01
set of sort of standard
algorithms if you like.

604
00:30:01 --> 00:30:05
Root force, just walk through
every possible case.

605
00:30:05 --> 00:30:08
It works well if the
problem sizes are small.

606
00:30:08 --> 00:30:11
We've had, there are a number
of variants of guess and check

607
00:30:11 --> 00:30:14
or hypothesize and test, where
you try to guess the solution

608
00:30:14 --> 00:30:17
and then check it and use
that to refine your search.

609
00:30:17 --> 00:30:20
Successive approximation,
Newton-Raphson was one nice

610
00:30:20 --> 00:30:23
example, but there's a whole
class of things that get closer

611
00:30:23 --> 00:30:26
and closer, reducing your
errors as you go along.

612
00:30:26 --> 00:30:30
Divide and conquer and actually
I guess in between there

613
00:30:30 --> 00:30:33
bi-section, which is really
just a very difficult of

614
00:30:33 --> 00:30:36
successive approximation, but
divide and conquer is

615
00:30:36 --> 00:30:37
a class of algorithm.


616
00:30:37 --> 00:30:39
These are tools that you
want in your tool box.

617
00:30:39 --> 00:30:41
These are the kinds of
algorithms that you should

618
00:30:41 --> 00:30:42
be able to recognize.


619
00:30:42 --> 00:30:45
And what I'd like you to begin
to do is to look at a problem

620
00:30:45 --> 00:30:49
and say, gee, which kind of
algorithm is most likely to be

621
00:30:49 --> 00:30:54
successful on this problem,
and map it into that case.

622
00:30:54 --> 00:30:56
OK, starting next -- don't
worry I'm not going to quit

623
00:30:56 --> 00:30:58
36 minutes after -- I got
one more topic for today.

624
00:30:58 --> 00:31:02
But jumping ahead, I'm going to
skip in a second now to talk

625
00:31:02 --> 00:31:05
about one last linguistic thing
from Python, but I want to

626
00:31:05 --> 00:31:07
preface Professor Guttag is
going to pick up next week, and

627
00:31:07 --> 00:31:10
what we're going to start doing
then is taking these classes of

628
00:31:10 --> 00:31:13
algorithms and start looking at
much more complex algorithms.

629
00:31:13 --> 00:31:16
Things you're more likely
to use in problems.

630
00:31:16 --> 00:31:19
Things like knapsack
problems as we move ahead.

631
00:31:19 --> 00:31:22
But the tools you've seen so
far are really the things

632
00:31:22 --> 00:31:24
that were going to see as
we build those algorithms.

633
00:31:24 --> 00:31:27
OK, I want to spend the last
portion of this lecture

634
00:31:27 --> 00:31:29
doing one last piece
of linguistics stuff.

635
00:31:29 --> 00:31:32
One last little thing from
Python, and that's to

636
00:31:32 --> 00:31:44
talk about exceptions.


637
00:31:44 --> 00:31:49
OK, you've actually seen
exceptions a lot, you just

638
00:31:49 --> 00:31:51
didn't know that's what they
were, because exceptions show

639
00:31:51 --> 00:31:52
up everywhere in Python.


640
00:31:52 --> 00:31:54
Let me give you a
couple of examples.

641
00:31:54 --> 00:31:58
I'm going to clear
some space here.

642
00:31:58 --> 00:32:01
Before I type in that
expression, I get

643
00:32:01 --> 00:32:02
an error, right?


644
00:32:02 --> 00:32:03
So it's not defined.


645
00:32:03 --> 00:32:06
But in fact, what this did
was it threw an exception.

646
00:32:06 --> 00:32:10
An exception is called a
name error exception.

647
00:32:10 --> 00:32:13
It says you gave me something
I didn't know how to deal.

648
00:32:13 --> 00:32:16
I'm going to throw it, or raise
it, to use the right term to

649
00:32:16 --> 00:32:18
somebody in case they can
handle it, but it's a

650
00:32:18 --> 00:32:21
particular kind of exception.


651
00:32:21 --> 00:32:24
I might do something like,
remind you I have test.

652
00:32:24 --> 00:32:31
If I do this, try and get
the 10th element of a list

653
00:32:31 --> 00:32:32
that's only eight long.


654
00:32:32 --> 00:32:34
I get what looks like an
error, but it's actually

655
00:32:34 --> 00:32:35
throwing an exception.


656
00:32:35 --> 00:32:37
The exception is right there.


657
00:32:37 --> 00:32:41
It's an index error, that is
it's trying to do something

658
00:32:41 --> 00:32:45
going beyond the range of what
this thing could deal with.

659
00:32:45 --> 00:32:47
OK, you say, come on, I've
seen these all the time.

660
00:32:47 --> 00:32:49
Every time I type something
into my program, it does one

661
00:32:49 --> 00:32:50
of these things, right?


662
00:32:50 --> 00:32:54
When we're just interacting
with idol, with the interactive

663
00:32:54 --> 00:32:57
editor or sorry, interactive
environment if you like,

664
00:32:57 --> 00:32:58
that's what you expect.


665
00:32:58 --> 00:33:00
What's happening is that we're
typing in something, an

666
00:33:00 --> 00:33:02
expression it doesn't
know how to deal.

667
00:33:02 --> 00:33:04
It's raising the exception, but
is this simply bubbling up

668
00:33:04 --> 00:33:07
at the top level saying
you've got a problem.

669
00:33:07 --> 00:33:12
Suppose instead you're in the
middle of some deep piece of

670
00:33:12 --> 00:33:15
code and you get one
of these cases.

671
00:33:15 --> 00:33:19
It's kind of annoying if it
throws it all the way back up

672
00:33:19 --> 00:33:21
to top level for you to fix.


673
00:33:21 --> 00:33:23
If it's truly a bug, that's
the right thing to do.

674
00:33:23 --> 00:33:25
You want to catch it.


675
00:33:25 --> 00:33:27
But in many cases exceptions
or things that, in fact,

676
00:33:27 --> 00:33:30
you as a program designer
could have handled.

677
00:33:30 --> 00:33:33
So I'm going to distinguish
in fact between un-handled

678
00:33:33 --> 00:33:39
exceptions, which are the
things that we saw there,

679
00:33:39 --> 00:33:44
and handled exceptions.


680
00:33:44 --> 00:33:45
I'm going to show you in a
second how to handle them, but

681
00:33:45 --> 00:33:46
let's look at an example.


682
00:33:46 --> 00:33:50
What do I mean by a
handled exception?

683
00:33:50 --> 00:33:54
Well let's look at the
next piece of code.

684
00:33:54 --> 00:33:55
OK, it's right here.


685
00:33:55 --> 00:33:56
It's called read float.


686
00:33:56 --> 00:33:58
We'll look at it in a second.


687
00:33:58 --> 00:34:00
Let me sort of set the stage up
for this -- suppose I want to

688
00:34:00 --> 00:34:03
input -- I'm sorry I want
you as a user to input a

689
00:34:03 --> 00:34:05
floating point number.


690
00:34:05 --> 00:34:08
We talked about things
you could do to try

691
00:34:08 --> 00:34:09
make sure that happens.


692
00:34:09 --> 00:34:11
You could run through a little
loop to say keep trying

693
00:34:11 --> 00:34:12
until you get one.


694
00:34:12 --> 00:34:14
But one of the ways I
could deal with it is

695
00:34:14 --> 00:34:15
what's shown here.


696
00:34:15 --> 00:34:17
And what's this little
loop say to do?

697
00:34:17 --> 00:34:21
This little loop says I'm
going to write a function

698
00:34:21 --> 00:34:23
or procedures that
takes in two messages.

699
00:34:23 --> 00:34:25
I'm going to run through a
loop, and I'm going to request

700
00:34:25 --> 00:34:28
some input, which I'm going
to read in with raw input.

701
00:34:28 --> 00:34:30
I'm going to store
that into val.

702
00:34:30 --> 00:34:33
And as you might expect, I'm
going to then try and see if I

703
00:34:33 --> 00:34:36
can convert that into a float.


704
00:34:36 --> 00:34:37
Oh wait a minute, that's a
little different than what

705
00:34:37 --> 00:34:38
we did last time, right?


706
00:34:38 --> 00:34:41
Last time we checked the
type and said if it is

707
00:34:41 --> 00:34:41
a float you're okay.


708
00:34:41 --> 00:34:42
If not, carry on.


709
00:34:42 --> 00:34:43
In this case what would happen?


710
00:34:43 --> 00:34:46
Well float is going to try
and do the cohersion.

711
00:34:46 --> 00:34:50
It's going to try and turn it
into a floating point number.

712
00:34:50 --> 00:34:52
If it does, I'm great, right.


713
00:34:52 --> 00:34:55
And I like just to return val.


714
00:34:55 --> 00:34:59
If it doesn't, floats going to
throw or raise, to use the

715
00:34:59 --> 00:35:00
right term, an exception.


716
00:35:00 --> 00:35:03
It's going to say something
like a type error.

717
00:35:03 --> 00:35:04
In fact, let's try
it over here.

718
00:35:04 --> 00:35:09
I if I go over here, and I say
float of three, it's going

719
00:35:09 --> 00:35:10
to do the conversion.


720
00:35:10 --> 00:35:15
But if I say turn this into
a float, ah it throws a

721
00:35:15 --> 00:35:16
value error exception.


722
00:35:16 --> 00:35:19
It says it's a wrong kind
of value that I've got.

723
00:35:19 --> 00:35:23
So I'm going to write a little
piece of code that says if it

724
00:35:23 --> 00:35:26
gives me a float, I'm set, But
if not, I'd like to have the

725
00:35:26 --> 00:35:29
code handle the exception.


726
00:35:29 --> 00:35:33
And that's what this funky
tri-accept thing does.

727
00:35:33 --> 00:35:43
This is a tri-accept block and
here's the flow of control

728
00:35:43 --> 00:35:45
that takes place in there.


729
00:35:45 --> 00:35:47
When I hit a tri-block.


730
00:35:47 --> 00:35:48
It's going to
literally do that.

731
00:35:48 --> 00:35:51
It's going to try and
execute the instructions.

732
00:35:51 --> 00:35:54
If it can successfully execute
the instructions, it's going to

733
00:35:54 --> 00:35:58
skip past the except block and
just carry on with the

734
00:35:58 --> 00:35:59
rest of the code.


735
00:35:59 --> 00:36:03
If, however, it raises an
exception, that exception, at

736
00:36:03 --> 00:36:07
least in this case where it's a
pure accept with no tags on it,

737
00:36:07 --> 00:36:11
is going to get, be like thrown
directly to the except block,

738
00:36:11 --> 00:36:13
and it's going to try
and execute that.

739
00:36:13 --> 00:36:14
So notice what's going
to happen here, then.

740
00:36:14 --> 00:36:17
If I give it something that can
be turned into a float, I come

741
00:36:17 --> 00:36:20
in here, I read the input, if
it can be turned into a float,

742
00:36:20 --> 00:36:22
I'm going to just return
the value and I'm set.

743
00:36:22 --> 00:36:25
If not, it's basically going to
throw it to this point, in

744
00:36:25 --> 00:36:28
which case I'm going to print
out an error message and oh

745
00:36:28 --> 00:36:30
yeah, I'm still in that while
loop, so it's going

746
00:36:30 --> 00:36:31
to go around.


747
00:36:31 --> 00:36:35
So in fact, if I go here
and, let me un-comment

748
00:36:35 --> 00:36:39
this and run the code.


749
00:36:39 --> 00:36:41
It says enter a float.


750
00:36:41 --> 00:36:48
And if I give it something that
can be -- sorry, I've got, yes,

751
00:36:48 --> 00:36:51
never mind the grades crap.


752
00:36:51 --> 00:36:54
Where did I have that?


753
00:36:54 --> 00:36:58
Let me comment that out.


754
00:36:58 --> 00:37:00
Somehow it's appropriate in the
middle of my lecture for it to

755
00:37:00 --> 00:37:04
say whoops at me but that
wasn't what I intended.

756
00:37:04 --> 00:37:09
And we will try this again.


757
00:37:09 --> 00:37:10
OK, says it says enter a float.


758
00:37:10 --> 00:37:11
I give it something that
can be converted into

759
00:37:11 --> 00:37:13
a float, it says fine.


760
00:37:13 --> 00:37:15
I'm going to go back and
run it again though.

761
00:37:15 --> 00:37:21
If I run it again, it
says enter a float.

762
00:37:21 --> 00:37:24
Ah ha, it goes into that accept
portion, prints out a message,

763
00:37:24 --> 00:37:27
and goes back around the
while loop to say try again.

764
00:37:27 --> 00:37:30
And it's going to keep doing
this until I give it something

765
00:37:30 --> 00:37:33
that does serve as a float.


766
00:37:33 --> 00:37:37
Right, so an exception then
has this format that I can

767
00:37:37 --> 00:37:39
control as a programmer.


768
00:37:39 --> 00:37:40
Why would I want to use this?


769
00:37:40 --> 00:37:44
Well some things I can actually
expect may happen and

770
00:37:44 --> 00:37:45
I want to handle them.


771
00:37:45 --> 00:37:46
The float example
is a simple one.

772
00:37:46 --> 00:37:47
I'm going to generalize
in a second.

773
00:37:47 --> 00:37:48
Here's a better example.


774
00:37:48 --> 00:37:52
I'm writing a piece of code
that wants to input a file.

775
00:37:52 --> 00:37:54
I can certainly imagine
something that says give me

776
00:37:54 --> 00:37:56
the file name, I'm going
to do something with it.

777
00:37:56 --> 00:37:59
I can't guarantee that the file
may exist under that name, but

778
00:37:59 --> 00:38:01
I know that's something
that might occur.

779
00:38:01 --> 00:38:04
So a nice way to handle it is
to write it as an exception

780
00:38:04 --> 00:38:06
that says, here's what I want
to do if I get the file.

781
00:38:06 --> 00:38:10
But just in case the file name
is not there, here's what I

782
00:38:10 --> 00:38:11
want to do in that
case to handle it.

783
00:38:11 --> 00:38:15
Let me specify what the
exception should do.

784
00:38:15 --> 00:38:17
In the example I just wrote
here, this is pretty

785
00:38:17 --> 00:38:17
trivial, right.


786
00:38:17 --> 00:38:19
OK, I'm trying to input floats.


787
00:38:19 --> 00:38:21
I could generalize
this pretty nicely.

788
00:38:21 --> 00:38:24
Imagine the same kind of idea
where I want to simply say I

789
00:38:24 --> 00:38:27
want to take input of anything
and try and see how to make

790
00:38:27 --> 00:38:28
sure I get the right
kind of thing.

791
00:38:28 --> 00:38:35
I want to make it polymorphic.


792
00:38:35 --> 00:38:38
Well that's pretty easy to do.


793
00:38:38 --> 00:38:43
That is basically the next
example, right here.

794
00:38:43 --> 00:38:49
In fact, let me
comment this one out.

795
00:38:49 --> 00:38:50
I can do exactly the
same kind of thing.

796
00:38:50 --> 00:38:55
Now what I'm going to try and
do is read in a set of values,

797
00:38:55 --> 00:38:59
but I'm going to give a type of
value as well as the messages.

798
00:38:59 --> 00:39:00
The format is the same.


799
00:39:00 --> 00:39:02
I'm going to ask for some
input, and then I am going to

800
00:39:02 --> 00:39:07
use that procedure to check, is
this the right type of value.

801
00:39:07 --> 00:39:10
And I'm trying to use that to
do the coercion if you like.

802
00:39:10 --> 00:39:12
Same thing if it works, I'm
going to skip that, if

803
00:39:12 --> 00:39:13
it not, it's going to
throw the exception.

804
00:39:13 --> 00:39:16
Why is this much nice?


805
00:39:16 --> 00:39:18
Well, that's a handy
piece of code.

806
00:39:18 --> 00:39:20
Because imagine I've got that
now, and I can now store that

807
00:39:20 --> 00:39:25
away in some file name, input
dot p y, and import into every

808
00:39:25 --> 00:39:28
one of my procedure functions,
pardon me, my files of

809
00:39:28 --> 00:39:30
procedures, because it's
a standard way of now

810
00:39:30 --> 00:39:33
giving me the input.


811
00:39:33 --> 00:39:35
OK, so far though, I've just
shown you what happens

812
00:39:35 --> 00:39:36
inside a peace a code.


813
00:39:36 --> 00:39:37
It raises an exception.


814
00:39:37 --> 00:39:39
It goes to that accept clause.


815
00:39:39 --> 00:39:42
We don't have to use it
just inside of one place.

816
00:39:42 --> 00:39:44
We can actually use
it more generally.

817
00:39:44 --> 00:39:47
And that gets me to the last
example I wanted to show you.

818
00:39:47 --> 00:39:55
Let me uncomment this.


819
00:39:55 --> 00:39:56
Let's take a look at this code.


820
00:39:56 --> 00:40:00
This looks like a handy piece
of code to have given what we

821
00:40:00 --> 00:40:02
just recently did to you.


822
00:40:02 --> 00:40:03
All right, get grades.


823
00:40:03 --> 00:40:06
It's a little function that's
going to say give me a file

824
00:40:06 --> 00:40:10
name, and I'm going to go off
and open that up and bind

825
00:40:10 --> 00:40:11
it to a local variable.


826
00:40:11 --> 00:40:14
And if it's successful, then
I'd just like to go off and do

827
00:40:14 --> 00:40:17
some things like turn it into a
list so I can compute average

828
00:40:17 --> 00:40:19
score or distributions
or something else.

829
00:40:19 --> 00:40:21
I don't really care
what's going on here.

830
00:40:21 --> 00:40:24
Notice though what I've done.


831
00:40:24 --> 00:40:28
Open, it doesn't succeed is
going to raise a particular

832
00:40:28 --> 00:40:32
kind of exception
called I O error.

833
00:40:32 --> 00:40:33
And so I've done a little bit
different things here which

834
00:40:33 --> 00:40:39
is I put the accept part of
the block with I O error.

835
00:40:39 --> 00:40:40
What does that say?


836
00:40:40 --> 00:40:43
It says if in the code up here
I get an exception of that

837
00:40:43 --> 00:40:46
sort, I'm going to go to
this place to handle it.

838
00:40:46 --> 00:40:49
On the other hand, if I'm
inside this procedure and some

839
00:40:49 --> 00:40:53
other exception is raised, it's
not tagged by that one, it's

840
00:40:53 --> 00:40:55
going to raise it up the chain.


841
00:40:55 --> 00:40:57
If that procedure was called by
some other procedure it's going

842
00:40:57 --> 00:41:00
to say is there an exception
block in there that

843
00:41:00 --> 00:41:01
can handle that.


844
00:41:01 --> 00:41:02
If not, I am going to
keep going up the chain

845
00:41:02 --> 00:41:05
until eventually I
get to the top level.

846
00:41:05 --> 00:41:06
And you can see that down here.


847
00:41:06 --> 00:41:07
I'm going to run
this in a second.

848
00:41:07 --> 00:41:09
This is just a piece of code
where I'm going to say, gee,

849
00:41:09 --> 00:41:15
if I can get the grades, do
something, if not carry on.

850
00:41:15 --> 00:41:19
And if I go ahead and run
this -- now it's going

851
00:41:19 --> 00:41:19
to say woops, at me.


852
00:41:19 --> 00:41:25
What happened?


853
00:41:25 --> 00:41:28
I'm down here and try, I'm
trying do get grades, which is

854
00:41:28 --> 00:41:34
a call to that function, which
is not bound in my computer.

855
00:41:34 --> 00:41:34
That says it's in here.


856
00:41:34 --> 00:41:36
It's in this tri-block.


857
00:41:36 --> 00:41:41
It raised an exception, but
it wasn't and I O error.

858
00:41:41 --> 00:41:45
So it passes it back, past this
exception, up to this level,

859
00:41:45 --> 00:41:48
which gets to that exception.


860
00:41:48 --> 00:41:51
Let me say this a little
bit better then.

861
00:41:51 --> 00:41:54
I can write exceptions
inside a piece of code.

862
00:41:54 --> 00:41:57
Try this, if it doesn't work I
can have an exception that

863
00:41:57 --> 00:41:59
catches any error
at that level.

864
00:41:59 --> 00:42:02
Or I can say catch only these
kinds of errors at that

865
00:42:02 --> 00:42:05
level, otherwise pass
them up the chain.

866
00:42:05 --> 00:42:07
And that exception will keep
getting passed up the chain of

867
00:42:07 --> 00:42:10
calls until it either gets to
the top level, in which case it

868
00:42:10 --> 00:42:11
looks like what you
see all the time.

869
00:42:11 --> 00:42:13
It looks like an error, but it
tells you what the error came

870
00:42:13 --> 00:42:18
from, or it gets an exception
, it can deal with it.

871
00:42:18 --> 00:42:21
OK, so the last thing to say
about this is what's the

872
00:42:21 --> 00:42:37
difference between an
exception and an assert?

873
00:42:37 --> 00:42:39
We introduced
asserts earlier on.

874
00:42:39 --> 00:42:44
You've actually seen them in
some pieces of code, so what's

875
00:42:44 --> 00:42:47
the difference between
the two of them?

876
00:42:47 --> 00:42:49
Well here's my way
of describing it.

877
00:42:49 --> 00:42:53
The goal of an assert, or an
assert statement, is basically

878
00:42:53 --> 00:42:57
to say, look, you can make sure
that my function is going to

879
00:42:57 --> 00:43:01
give this kind of result if you
give me inputs of a

880
00:43:01 --> 00:43:02
particular type.


881
00:43:02 --> 00:43:03
Sorry, wrong way of saying it.


882
00:43:03 --> 00:43:05
If you give me inputs that
satisfy some particular

883
00:43:05 --> 00:43:07
constraints.


884
00:43:07 --> 00:43:08
That was the kind
of thing you saw.

885
00:43:08 --> 00:43:11
Asserts said here are
some conditions to test.

886
00:43:11 --> 00:43:13
If they're true, I'm going to
let the rest of the code run.

887
00:43:13 --> 00:43:16
If not, I'm going
to throw an error.

888
00:43:16 --> 00:43:19
So the assertion is basically
saying we got some

889
00:43:19 --> 00:43:26
pre-conditions, those are the
clauses inside the assert that

890
00:43:26 --> 00:43:34
have to be true, and there's a
post condition. and in essence,

891
00:43:34 --> 00:43:37
what the assert is saying is,
or rather the programmer is

892
00:43:37 --> 00:43:39
saying using the assert is, if
you give me input that

893
00:43:39 --> 00:43:42
satisfies the preconditions,
I'm guaranteeing to you that my

894
00:43:42 --> 00:43:44
code is going to give you
something that meets

895
00:43:44 --> 00:43:45
the post condition.


896
00:43:45 --> 00:43:46
It's going to do
the right thing.

897
00:43:46 --> 00:43:49
And as a consequence, as you
saw with the asserts, if the

898
00:43:49 --> 00:43:52
preconditions aren't true,
it throws an error.

899
00:43:52 --> 00:43:55
It goes back up the top
level saying stop operation

900
00:43:55 --> 00:43:59
immediately and goes
back up the top level.

901
00:43:59 --> 00:44:01
Asserts in fact are nice in the
sense that they let you check

902
00:44:01 --> 00:44:03
conditions at debugging
time or testing time.

903
00:44:03 --> 00:44:07
So you can actually use them to
see where your code is going.

904
00:44:07 --> 00:44:10
An exception, when you use an
exception, basically what

905
00:44:10 --> 00:44:12
you're saying is, look, you can
do anything you want with my

906
00:44:12 --> 00:44:15
function, and you can be sure
that I'm going to tell you if

907
00:44:15 --> 00:44:16
something is going wrong.


908
00:44:16 --> 00:44:19
And in many cases I'm going
to handle it myself.

909
00:44:19 --> 00:44:23
So as much as possible, the
exceptions are going to try to

910
00:44:23 --> 00:44:27
handle unexpected things,
actually wrong term, you

911
00:44:27 --> 00:44:29
expected them, but not
what the user did.

912
00:44:29 --> 00:44:31
It's going to try to handle
conditions other than

913
00:44:31 --> 00:44:33
the normal ones itself.


914
00:44:33 --> 00:44:37
So you can use the
thing in anyway.

915
00:44:37 --> 00:44:39
If it can't, it's going to try
and throw it to somebody else

916
00:44:39 --> 00:44:42
to handle, and only if there
is no handler for that

917
00:44:42 --> 00:44:45
unexpected condition, will
it come up to top level.

918
00:44:45 --> 00:44:48
So, summarizing better, assert
is something you put in to say

919
00:44:48 --> 00:44:51
to the user, make sure you're
giving me input of this type,

920
00:44:51 --> 00:44:53
but I'm going to guarantee you
the rest of the code

921
00:44:53 --> 00:44:54
works correctly.


922
00:44:54 --> 00:44:57
Exceptions and exception
handlers are saying, here are

923
00:44:57 --> 00:45:00
the odd cases that I might see
and here's what I'd like to do

924
00:45:00 --> 00:45:04
in those cases in order to try
and be able to deal with them.

925
00:45:04 --> 00:45:10
Last thing to say is why would
you want to have exceptions?

926
00:45:10 --> 00:45:14
Well, let's go back to that
case of inputting a simple

927
00:45:14 --> 00:45:16
little floating point.


928
00:45:16 --> 00:45:18
If I'm expecting mostly numbers
in, I can certainly try

929
00:45:18 --> 00:45:19
and do the coercion.


930
00:45:19 --> 00:45:22
I could have done that
just doing the coercion.

931
00:45:22 --> 00:45:26
The problem is, I want to know
if, in fact, I've got something

932
00:45:26 --> 00:45:28
that's not of the
form I expect.

933
00:45:28 --> 00:45:30
I'm much better having an
exception get handled at the

934
00:45:30 --> 00:45:34
time of input than to let that
prop -- that value rather

935
00:45:34 --> 00:45:36
propagate through a whole bunch
of code until eventually it

936
00:45:36 --> 00:45:39
hits an error 17 calls later,
and you have no clue

937
00:45:39 --> 00:45:41
where it came from.


938
00:45:41 --> 00:45:44
So the exceptions are useful
when you want to have the

939
00:45:44 --> 00:45:48
ability to say, I expect in
general this kind of behavior,

940
00:45:48 --> 00:45:50
but I do know there are some
other things that might happen

941
00:45:50 --> 00:45:53
and here's what I'd like to do
in each one of those cases.

942
00:45:53 --> 00:45:56
But I do want to make sure that
I don't let a value that I'm

943
00:45:56 --> 00:45:58
not expecting pass through.


944
00:45:58 --> 00:46:01
That goes back to that idea of
sort of discipline coding.

945
00:46:01 --> 00:46:04
It's easy to have assumptions
about what you think are going

946
00:46:04 --> 00:46:06
to come into the program
when you writ it.

947
00:46:06 --> 00:46:09
If you really know what they
are use them as search, but if

948
00:46:09 --> 00:46:11
you think there's going to be
some flexibility, you want to

949
00:46:11 --> 00:46:14
prevent the user getting
trapped in a bad spot, and

950
00:46:14 --> 00:46:17
exceptions as a consequence
are a good thing to use.

951
00:46:17 --> 00:46:18