1
00:00:00,500 --> 00:00:02,800
The following content is
provided under a Creative
2
00:00:02,800 --> 00:00:04,340
Commons license.
3
00:00:04,340 --> 00:00:06,660
Your support will help
MIT OpenCourseWare
4
00:00:06,660 --> 00:00:11,020
continue to offer high quality
educational resources for free.
5
00:00:11,020 --> 00:00:13,640
To make a donation or
view additional materials
6
00:00:13,640 --> 00:00:17,365
from hundreds of MIT courses,
visit MIT OpenCourseWare
7
00:00:17,365 --> 00:00:17,990
at ocw.mit.edu.
8
00:00:22,802 --> 00:00:24,260
TOM LEIGHTON: This
week we're going
9
00:00:24,260 --> 00:00:26,940
to talk about recurrences,
how to set them up,
10
00:00:26,940 --> 00:00:29,570
how to model a problem
as a recurrent problem,
11
00:00:29,570 --> 00:00:31,950
and then how to solve them.
12
00:00:31,950 --> 00:00:35,160
This stuff is really useful when
you get into algorithm design
13
00:00:35,160 --> 00:00:36,770
and algorithm analysis.
14
00:00:36,770 --> 00:00:41,060
6006 and 6046, we you
use this stuff a lot.
15
00:00:41,060 --> 00:00:44,290
We're going to start with a very
famous recurrent problem known
16
00:00:44,290 --> 00:00:46,430
as the Towers of Hanoi problem.
17
00:00:46,430 --> 00:00:47,920
And I'm guessing
that many of you
18
00:00:47,920 --> 00:00:51,300
have seen this problem
in one form or another.
19
00:00:51,300 --> 00:00:54,030
In the Tower of
Hanoi problem, you've
20
00:00:54,030 --> 00:01:00,260
got three pegs and n disks,
these little guys here.
21
00:01:00,260 --> 00:01:02,040
All different sizes.
22
00:01:02,040 --> 00:01:07,340
And the object is to start
with a stack like this
23
00:01:07,340 --> 00:01:09,990
and move the disks
around so they wind up
24
00:01:09,990 --> 00:01:13,640
with the stack on another disk.
25
00:01:13,640 --> 00:01:18,750
Now, the rules are you can
only move one disk at a time.
26
00:01:18,750 --> 00:01:22,270
And you can never put a big
disk over a little disk.
27
00:01:22,270 --> 00:01:24,720
So that you can't do.
28
00:01:24,720 --> 00:01:26,900
So that would have to go there.
29
00:01:26,900 --> 00:01:29,200
Then you could do this.
30
00:01:29,200 --> 00:01:34,220
And now I've gotten the
top two moved over to here.
31
00:01:34,220 --> 00:01:37,650
And the goal is to take
the minimum number of moves
32
00:01:37,650 --> 00:01:40,980
to get the whole stack
moved over to here,
33
00:01:40,980 --> 00:01:45,010
and to even figure
out if that's doable.
34
00:01:45,010 --> 00:01:51,910
Now, this was invented by a
famous French mathematician
35
00:01:51,910 --> 00:01:55,470
named Edward Lucas in 1883.
36
00:01:55,470 --> 00:01:58,750
And in the original
legend around this puzzle,
37
00:01:58,750 --> 00:02:00,870
there were 64 disks.
38
00:02:00,870 --> 00:02:03,770
This one has just seven on it.
39
00:02:03,770 --> 00:02:08,030
And the story was at
the beginning of time,
40
00:02:08,030 --> 00:02:14,390
God placed the 64 disks on
one of the three golden pegs.
41
00:02:14,390 --> 00:02:18,900
And he got an order of
monks whose lifelong mission
42
00:02:18,900 --> 00:02:22,440
was to move the disks one
at a time so at the end
43
00:02:22,440 --> 00:02:26,550
all 64 disks were lined
up on an adjacent peg.
44
00:02:26,550 --> 00:02:29,990
And according to legend,
when that task was done
45
00:02:29,990 --> 00:02:34,540
the tower would crumble
and the world would end.
46
00:02:34,540 --> 00:02:37,520
Sort of explains
why monks are never
47
00:02:37,520 --> 00:02:39,750
smiling, because
they work forever
48
00:02:39,750 --> 00:02:42,530
and then just
something bad happens.
49
00:02:42,530 --> 00:02:44,830
Now, the question we
want to be able to answer
50
00:02:44,830 --> 00:02:48,220
is, how long till
the world ends?
51
00:02:48,220 --> 00:02:53,300
So we can start with
a simpler version.
52
00:02:53,300 --> 00:02:58,940
Say let's take three
disks and start with that.
53
00:02:58,940 --> 00:03:00,890
And let's see, is it doable?
54
00:03:00,890 --> 00:03:02,660
Well, we could.
55
00:03:02,660 --> 00:03:16,740
That's one, that's two,
three, four, five, six, seven.
56
00:03:16,740 --> 00:03:18,820
So it's doable with three disks.
57
00:03:18,820 --> 00:03:20,180
And we took seven moves.
58
00:03:20,180 --> 00:03:22,220
Maybe there's a
better way, and that's
59
00:03:22,220 --> 00:03:25,420
part of what we'll
try to study today.
60
00:03:25,420 --> 00:03:31,512
In fact, what we'd like to know
is how many moves for n disks?
61
00:03:34,820 --> 00:03:36,470
And is it even finite?
62
00:03:36,470 --> 00:03:42,330
So we're going to define t
sub n to be the minimum number
63
00:03:42,330 --> 00:03:49,330
of moves for n disks.
64
00:03:49,330 --> 00:03:51,880
So to take a stack of
n disks and move them
65
00:03:51,880 --> 00:03:54,510
to an adjacent peg.
66
00:03:54,510 --> 00:03:58,010
So, for example, what's t1?
67
00:03:58,010 --> 00:03:58,510
One.
68
00:03:58,510 --> 00:03:59,860
I got one peg, I just move it.
69
00:03:59,860 --> 00:04:01,726
That's easy.
70
00:04:01,726 --> 00:04:05,040
t2, 3.
71
00:04:05,040 --> 00:04:07,870
Move the little
guy, move the base,
72
00:04:07,870 --> 00:04:10,000
move the little guy back on top.
73
00:04:10,000 --> 00:04:12,090
That's the best you can do.
74
00:04:12,090 --> 00:04:16,510
And we've seen that
t3 is at most 7.
75
00:04:16,510 --> 00:04:19,060
Maybe there's a better way.
76
00:04:19,060 --> 00:04:22,930
But let's try to first get
an upper bound on this.
77
00:04:22,930 --> 00:04:26,220
Any thoughts about how we
might go about a general upper
78
00:04:26,220 --> 00:04:31,710
bound for n for this problem?
79
00:04:31,710 --> 00:04:35,270
Like say I had four disks.
80
00:04:35,270 --> 00:04:37,540
Any thoughts about
a strategy where
81
00:04:37,540 --> 00:04:39,510
we might be able to
analyze or get an upper
82
00:04:39,510 --> 00:04:43,570
bound the number of steps
for four disks here?
83
00:04:43,570 --> 00:04:44,400
Or more disks.
84
00:04:44,400 --> 00:04:45,556
Any thoughts?
85
00:04:45,556 --> 00:04:46,056
Yeah.
86
00:04:47,999 --> 00:04:51,412
AUDIENCE: You can look
at the previous problem
87
00:04:51,412 --> 00:04:55,890
and then add a certain
number of [INAUDIBLE].
88
00:04:55,890 --> 00:04:57,780
TOM LEIGHTON: Look at
the three disk problem.
89
00:04:57,780 --> 00:04:59,370
That's a good idea.
90
00:04:59,370 --> 00:05:03,240
In fact, you know, I could
look at the three disk problem
91
00:05:03,240 --> 00:05:04,210
recursively.
92
00:05:04,210 --> 00:05:05,450
I got these three disks.
93
00:05:05,450 --> 00:05:10,160
Say I move them over a series
of steps, seven steps at most,
94
00:05:10,160 --> 00:05:12,035
to put them here.
95
00:05:12,035 --> 00:05:13,282
What would I do next?
96
00:05:16,240 --> 00:05:20,000
Move the lower one and then
do this recurrent problem
97
00:05:20,000 --> 00:05:22,860
again back.
98
00:05:22,860 --> 00:05:26,310
All right, so we can take
a problem with n disks
99
00:05:26,310 --> 00:05:29,900
and solve it recursively.
100
00:05:29,900 --> 00:05:31,698
Let's see what happens
when we do that.
101
00:05:35,087 --> 00:05:36,670
So let's look at a
recursive solution.
102
00:05:43,920 --> 00:05:46,000
And let's draw it for n.
103
00:05:46,000 --> 00:05:50,220
So the first phase
that you described
104
00:05:50,220 --> 00:05:52,430
is I start, here
are my three pegs,
105
00:05:52,430 --> 00:05:55,900
and I start with this,
1, 2, 3 down to end here,
106
00:05:55,900 --> 00:05:58,170
and nothing on these pegs.
107
00:05:58,170 --> 00:06:02,520
And I'm going to use recursion
to take the top and minus 1
108
00:06:02,520 --> 00:06:04,510
and move them out here.
109
00:06:04,510 --> 00:06:06,130
So I leave the nth guy there.
110
00:06:06,130 --> 00:06:07,680
Nothing here.
111
00:06:07,680 --> 00:06:12,870
And I go 1, 2, to n minus 1.
112
00:06:12,870 --> 00:06:17,870
How many steps does that move
take, that series of moves
113
00:06:17,870 --> 00:06:20,650
take, that phase?
114
00:06:20,650 --> 00:06:23,740
In terms of t.
115
00:06:23,740 --> 00:06:28,020
t of n minus 1, because I'm
taking an n minus 1 disk stack
116
00:06:28,020 --> 00:06:28,980
and moving it.
117
00:06:28,980 --> 00:06:32,030
So that is tn minus 1 steps.
118
00:06:36,230 --> 00:06:40,600
All right, now my next
move is to take this guy
119
00:06:40,600 --> 00:06:41,523
and stick him here.
120
00:06:44,120 --> 00:06:46,460
So we'll draw that over here.
121
00:06:46,460 --> 00:06:49,502
So the nth guy moves there and
I leave all these guys here.
122
00:06:52,030 --> 00:06:53,460
That's going to be phase two.
123
00:06:57,840 --> 00:06:59,370
I move the largest disk.
124
00:07:04,120 --> 00:07:07,590
And how many steps
did that take?
125
00:07:07,590 --> 00:07:08,500
One step.
126
00:07:08,500 --> 00:07:09,590
Just move one disk.
127
00:07:13,430 --> 00:07:14,370
One step for that.
128
00:07:14,370 --> 00:07:18,620
And then my last phase
is to take all these
129
00:07:18,620 --> 00:07:19,708
and move them onto here.
130
00:07:30,240 --> 00:07:34,800
And so then it looks like
the final result, 1, 2,
131
00:07:34,800 --> 00:07:36,930
over n minus 1 over n.
132
00:07:36,930 --> 00:07:39,111
And how many steps
this phase three take?
133
00:07:42,200 --> 00:07:43,888
tn minus 1.
134
00:07:43,888 --> 00:07:44,388
Good.
135
00:07:47,710 --> 00:07:49,660
All right, so we've
got an algorithm now,
136
00:07:49,660 --> 00:07:52,080
a recursive algorithm.
137
00:07:52,080 --> 00:08:03,610
And the total time,
the number of moves tn
138
00:08:03,610 --> 00:08:10,820
is at most tn minus 1 plus
tn minus 1 is 2tn minus 1
139
00:08:10,820 --> 00:08:14,970
plus the one step up there.
140
00:08:14,970 --> 00:08:16,930
For example, which
we already know,
141
00:08:16,930 --> 00:08:26,070
t3 is at most 2t2 plus 1,
which is 2 times 3 plus 1 is 7.
142
00:08:26,070 --> 00:08:28,470
So we already knew we
could do it in seven.
143
00:08:28,470 --> 00:08:33,210
What's an upper bound on
the time to move four disks?
144
00:08:33,210 --> 00:08:34,559
15.
145
00:08:34,559 --> 00:08:36,330
OK?
146
00:08:36,330 --> 00:08:41,390
All right, so t4 is at most 15.
147
00:08:41,390 --> 00:08:43,070
But maybe there's a better way.
148
00:08:43,070 --> 00:08:45,740
This is just one way to do it.
149
00:08:45,740 --> 00:08:51,590
How many people think there's a
better way than this approach?
150
00:08:51,590 --> 00:08:53,770
How many people think
there is no better way?
151
00:08:53,770 --> 00:08:55,250
All right, very good.
152
00:08:55,250 --> 00:08:57,570
Yeah, this is the
best you can do.
153
00:08:57,570 --> 00:08:58,710
And let's prove that.
154
00:08:58,710 --> 00:08:59,820
Let's get a lower bound.
155
00:09:07,500 --> 00:09:13,610
All right, to see why this is
optimal, think about the step
156
00:09:13,610 --> 00:09:17,010
where the nth disk moves.
157
00:09:17,010 --> 00:09:23,600
At some point in our procedure,
the biggest disk has to move.
158
00:09:23,600 --> 00:09:25,670
Now, it could move to
this peg or that peg,
159
00:09:25,670 --> 00:09:26,700
doesn't really matter.
160
00:09:26,700 --> 00:09:29,835
But when it moves,
what do I know
161
00:09:29,835 --> 00:09:32,250
had to have happened
before that guy moved?
162
00:09:37,150 --> 00:09:42,360
Everything on top of it had
to move to the other peg.
163
00:09:42,360 --> 00:09:44,840
I started with 1
through n minus 1 here.
164
00:09:44,840 --> 00:09:46,410
None of them can be here.
165
00:09:46,410 --> 00:09:49,680
They all have to be here.
166
00:09:49,680 --> 00:09:50,970
OK?
167
00:09:50,970 --> 00:09:57,330
So that means that I have to
have had tn minus 1 steps.
168
00:09:57,330 --> 00:10:07,792
So at least tn minus 1 steps
before the big disk moves.
169
00:10:12,520 --> 00:10:13,560
All right?
170
00:10:13,560 --> 00:10:18,850
Then there is the step
when the big guy moves.
171
00:10:18,850 --> 00:10:20,570
One step for the
big guy to move.
172
00:10:27,954 --> 00:10:29,578
Because it's got to
move at least once.
173
00:10:33,490 --> 00:10:39,140
And then after it moves, I got
to get everything else, one way
174
00:10:39,140 --> 00:10:42,500
or another, on top of it.
175
00:10:42,500 --> 00:10:46,110
At some point after the
last time the big guy moves,
176
00:10:46,110 --> 00:10:48,890
everything else
has to go on top.
177
00:10:48,890 --> 00:10:51,219
And that's another
tn minus 1 steps.
178
00:10:55,350 --> 00:11:04,160
So after the last time
the big guy moves,
179
00:11:04,160 --> 00:11:05,988
you've got to take
tn minus 1 steps.
180
00:11:10,560 --> 00:11:12,640
OK, so that means
there's a lower bound
181
00:11:12,640 --> 00:11:21,420
that no matter what you do tn
is at least 2tn minus 1 plus 1.
182
00:11:21,420 --> 00:11:24,880
And so in fact, it's equal.
183
00:11:24,880 --> 00:11:30,730
tn equals 2tn minus 1 plus 1.
184
00:11:30,730 --> 00:11:34,750
So t3 is 7, t4 is 15.
185
00:11:34,750 --> 00:11:38,920
Any questions about that?
186
00:11:42,080 --> 00:11:44,590
All right, well we'd like to
know how long till the world
187
00:11:44,590 --> 00:11:47,580
ends, which is t64.
188
00:11:47,580 --> 00:11:50,440
So we'd like to get
a formula for t of n.
189
00:11:50,440 --> 00:11:54,080
We could just compute
it 64 times and see.
190
00:11:54,080 --> 00:11:56,237
But much nicer just
to get a formula.
191
00:11:56,237 --> 00:11:58,570
And that's what we're going
to learn how to do this week
192
00:11:58,570 --> 00:12:01,570
is to get formulas,
closed form expressions,
193
00:12:01,570 --> 00:12:03,304
to solve recurrences.
194
00:12:03,304 --> 00:12:05,470
And there's a bunch of
different methods you can use
195
00:12:05,470 --> 00:12:07,380
and we're going to
talk about them.
196
00:12:07,380 --> 00:12:12,500
The simplest is to guess the
answer and then check it.
197
00:12:12,500 --> 00:12:16,750
It's called guess and
verify, also known
198
00:12:16,750 --> 00:12:18,270
as the substitution method.
199
00:12:33,140 --> 00:12:36,850
Now, what's a good
guess for t of n?
200
00:12:40,070 --> 00:12:41,250
2 to the n minus 1.
201
00:12:41,250 --> 00:12:49,170
You got t of 1 is 1, t of 2 is
3, t of 3 is 7, t of 4 is 15.
202
00:12:49,170 --> 00:12:57,730
You won't compute too many
before you guess that t of n
203
00:12:57,730 --> 00:12:59,291
is 2 to the n minus 1.
204
00:13:02,570 --> 00:13:04,900
Now, that doesn't
mean it's true.
205
00:13:04,900 --> 00:13:08,760
So how are we going
to verify that guess?
206
00:13:08,760 --> 00:13:11,360
What proof technique?
207
00:13:11,360 --> 00:13:12,600
Induction.
208
00:13:12,600 --> 00:13:14,060
It just keeps coming back.
209
00:13:14,060 --> 00:13:16,880
You just never finish
with induction.
210
00:13:16,880 --> 00:13:21,172
So we're going to verify
this by induction.
211
00:13:28,922 --> 00:13:30,255
What's our induction hypothesis?
212
00:13:33,020 --> 00:13:33,638
Any thoughts?
213
00:13:38,280 --> 00:13:39,190
Right there.
214
00:13:39,190 --> 00:13:43,670
p of n is going to be that
tn equals 2 to the n minus 1.
215
00:13:43,670 --> 00:13:44,340
Very typical.
216
00:13:47,390 --> 00:13:52,670
So the predicate is that t
of n is 2 to the n minus 1.
217
00:13:52,670 --> 00:13:53,260
Base case.
218
00:13:57,970 --> 00:14:00,150
We'll pick n equals 1.
219
00:14:00,150 --> 00:14:05,260
T of 1 is 1 and that
is 2 to the 1 minus 1.
220
00:14:05,260 --> 00:14:06,310
So the base case works.
221
00:14:09,120 --> 00:14:10,836
Now we'll do the inductive step.
222
00:14:16,950 --> 00:14:20,330
We're going to assume p of n.
223
00:14:20,330 --> 00:14:24,320
We're going to assume that
tn equals 2 to the n minus 1
224
00:14:24,320 --> 00:14:30,940
to prove pn minus 1, which
is tn plus 1 equals 2
225
00:14:30,940 --> 00:14:33,970
to the n plus 1 minus 1.
226
00:14:33,970 --> 00:14:35,960
So a very standard induction.
227
00:14:35,960 --> 00:14:39,600
So let's look at tn plus 1 now.
228
00:14:39,600 --> 00:14:45,910
tn plus 1 equals 2n plus 1.
229
00:14:45,910 --> 00:14:49,220
Yeah, 2tn plus 1.
230
00:14:49,220 --> 00:14:51,320
Now we plug in here.
231
00:14:51,320 --> 00:14:52,500
That's just the recurrence.
232
00:14:52,500 --> 00:14:56,490
Now we plug in using our
assumption from the induction.
233
00:14:56,490 --> 00:15:04,580
This is 2 to the
n minus 1 plus 1.
234
00:15:04,580 --> 00:15:05,330
Multiply the 2.
235
00:15:05,330 --> 00:15:07,430
That's 2 to the n plus 1.
236
00:15:07,430 --> 00:15:11,730
Minus 2 plus 1 is minus 1.
237
00:15:11,730 --> 00:15:14,651
And so we're done.
238
00:15:14,651 --> 00:15:16,150
All right, so we
guessed the answer.
239
00:15:16,150 --> 00:15:17,600
It was a pretty easy guess.
240
00:15:17,600 --> 00:15:21,087
Very simple to prove
it by induction.
241
00:15:21,087 --> 00:15:21,670
Any questions?
242
00:15:24,250 --> 00:15:25,850
All right, how
long would it take
243
00:15:25,850 --> 00:15:31,010
me to do all seven disks
on this puzzle here?
244
00:15:31,010 --> 00:15:31,795
How many steps?
245
00:15:35,118 --> 00:15:37,060
127 steps.
246
00:15:37,060 --> 00:15:40,350
How long till the
end of the world?
247
00:15:40,350 --> 00:15:41,500
A long time.
248
00:15:41,500 --> 00:15:46,070
2 to the 64th minus 1 is
18 billion, billion moves.
249
00:15:46,070 --> 00:15:50,020
So going to keep those monks
busy for a little while
250
00:15:50,020 --> 00:15:52,180
before the world ends.
251
00:15:52,180 --> 00:15:56,900
Now in general, this method
is sort of the best method
252
00:15:56,900 --> 00:15:58,620
for solving a recurrence.
253
00:15:58,620 --> 00:16:01,950
We'll see lots of examples
where it works great.
254
00:16:01,950 --> 00:16:04,530
The only problem is it
requires that divine insight
255
00:16:04,530 --> 00:16:06,970
thing, that guess to
guess the right answer.
256
00:16:06,970 --> 00:16:10,480
Obviously if we guess the wrong
answer, it's not going to work.
257
00:16:10,480 --> 00:16:15,520
Guessing is many times
easy, but not always.
258
00:16:15,520 --> 00:16:18,220
So you need other methods
when you can't guess it.
259
00:16:18,220 --> 00:16:21,320
The next most common
method has a lot of names.
260
00:16:21,320 --> 00:16:26,780
It's called plug and chug,
expansion, iteration,
261
00:16:26,780 --> 00:16:30,572
brute force, exhaustion.
262
00:16:30,572 --> 00:16:32,530
Let me show you how that
works in this example.
263
00:16:44,640 --> 00:16:46,190
OK, so plug and chug.
264
00:16:51,040 --> 00:16:54,410
So we'll solve the
same recurrence.
265
00:16:54,410 --> 00:16:56,880
So we've got tn
equals, and I'm going
266
00:16:56,880 --> 00:17:01,030
to write it just
reversed, 1 plus 2tn.
267
00:17:01,030 --> 00:17:04,280
Because I'm going to expand
out the recurrent term.
268
00:17:04,280 --> 00:17:08,109
That's why it's
called expansion also.
269
00:17:08,109 --> 00:17:11,150
Well, that's equal to 1 plus 2.
270
00:17:11,150 --> 00:17:14,230
Now I expand this out
using the recurrence.
271
00:17:14,230 --> 00:17:17,066
1 plus 2 tn minus 1.
272
00:17:20,048 --> 00:17:22,250
All right, now I do a
little chugging here.
273
00:17:22,250 --> 00:17:24,700
So I plugged here.
274
00:17:24,700 --> 00:17:27,089
Now I'll do chug.
275
00:17:27,089 --> 00:17:33,420
That's 1 plus 2
plus 4 tn minus 1.
276
00:17:33,420 --> 00:17:34,906
Now I'm going to plug in again.
277
00:17:37,580 --> 00:17:38,738
What's that?
278
00:17:38,738 --> 00:17:39,654
AUDIENCE: [INAUDIBLE].
279
00:17:42,700 --> 00:17:45,290
TOM LEIGHTON: You're right.
280
00:17:45,290 --> 00:17:49,140
You're right, I've got to get
minus 1 here and minus 2 here.
281
00:17:49,140 --> 00:17:50,310
That is correct.
282
00:17:50,310 --> 00:17:50,810
Very good.
283
00:17:50,810 --> 00:17:52,400
I would have run into trouble.
284
00:17:52,400 --> 00:17:53,118
That's good.
285
00:17:55,730 --> 00:17:57,350
Now I plug in here.
286
00:17:57,350 --> 00:18:04,680
1 plus 2 plus 4, 1
plus 2 tn minus 3.
287
00:18:04,680 --> 00:18:05,525
Then we do the chug.
288
00:18:19,230 --> 00:18:26,770
1 plus 2 plus 4
plus 8 tn minus 3.
289
00:18:26,770 --> 00:18:29,270
All right, and now we're
going to try to observe
290
00:18:29,270 --> 00:18:30,412
the pattern we're getting.
291
00:18:30,412 --> 00:18:32,870
So there's a little bit of a
guess going on in this context
292
00:18:32,870 --> 00:18:33,850
still.
293
00:18:33,850 --> 00:18:34,850
But you see the pattern.
294
00:18:34,850 --> 00:18:36,760
We've got powers of 2.
295
00:18:36,760 --> 00:18:37,690
Pretty easy pattern.
296
00:18:37,690 --> 00:18:39,890
So you might say if
I keep on doing this,
297
00:18:39,890 --> 00:18:43,170
I'm going to get 1
plus 2 plus 4 plus
298
00:18:43,170 --> 00:18:48,690
dot dot dot plus 2 to
the i minus 1 plus 2
299
00:18:48,690 --> 00:18:52,964
to the i tn minus i.
300
00:18:52,964 --> 00:18:56,930
Got 2 to the 3 tn minus 3 there.
301
00:18:56,930 --> 00:19:01,770
And then I take this all the way
down to i equals n minus 1, 1
302
00:19:01,770 --> 00:19:10,230
plus 2 plus 4 plus 2
to the n minus 2 plus 2
303
00:19:10,230 --> 00:19:12,910
to the n minus 1 t1.
304
00:19:15,570 --> 00:19:17,300
And t1 is just 1.
305
00:19:21,950 --> 00:19:24,660
So I'm just getting the
sum of the powers of 2
306
00:19:24,660 --> 00:19:26,810
up to the n minus 1.
307
00:19:26,810 --> 00:19:30,180
And we know what that is
from last time, for sure.
308
00:19:30,180 --> 00:19:34,000
That's just 2 to the n minus 1.
309
00:19:34,000 --> 00:19:36,570
So we've derived the answer
with only a little bit
310
00:19:36,570 --> 00:19:38,930
of a-- we had to observe
the pattern here,
311
00:19:38,930 --> 00:19:40,310
which is often easy to do.
312
00:19:40,310 --> 00:19:42,980
But now we've
derived the answer.
313
00:19:42,980 --> 00:19:45,170
Now to be really safe,
we've got to go back
314
00:19:45,170 --> 00:19:47,390
and still do verify,
just to make sure
315
00:19:47,390 --> 00:19:53,580
that we didn't make a mistake in
how we did the derivation here.
316
00:19:53,580 --> 00:19:55,610
OK?
317
00:19:55,610 --> 00:19:59,210
Any questions about
that approach?
318
00:19:59,210 --> 00:20:01,580
You just keep plugging it
back into the recurrence,
319
00:20:01,580 --> 00:20:05,330
look at the pattern you get, and
try to guess it from that point
320
00:20:05,330 --> 00:20:08,040
and then solve.
321
00:20:08,040 --> 00:20:10,916
All right, let's do a little
more interesting recurrence.
322
00:20:13,690 --> 00:20:15,680
This is a fairly famous
one that comes up
323
00:20:15,680 --> 00:20:19,780
in a well known algorithm for
sorting called merge sort.
324
00:20:19,780 --> 00:20:22,890
This is an algorithm
you'll look at in 6046,
325
00:20:22,890 --> 00:20:24,380
one of the many
sorting algorithms
326
00:20:24,380 --> 00:20:27,290
you'll study when you
take an algorithms class.
327
00:20:39,310 --> 00:20:43,740
So you're given n numbers and
your task is to sort them,
328
00:20:43,740 --> 00:20:45,140
put them in order.
329
00:20:45,140 --> 00:20:46,890
And you're allowed
to do comparisons
330
00:20:46,890 --> 00:20:48,120
between the numbers.
331
00:20:48,120 --> 00:20:51,240
You can think of them as
you've got a list of names,
332
00:20:51,240 --> 00:20:53,380
putting them in
alphabetical order.
333
00:20:53,380 --> 00:20:57,260
How many comparisons do you
have to do to make that happen?
334
00:20:57,260 --> 00:21:01,360
And this is one way to do
it that's very efficient.
335
00:21:01,360 --> 00:21:05,250
So to sort n items.
336
00:21:05,250 --> 00:21:07,910
In fact, we're going to look
at n being a power of 2.
337
00:21:07,910 --> 00:21:13,270
Call the items x1, x2, xn.
338
00:21:13,270 --> 00:21:17,348
n's going to be a power of
2 to make it simple for now.
339
00:21:20,186 --> 00:21:21,560
The first thing
we're going to do
340
00:21:21,560 --> 00:21:24,680
is recursively
sort the first half
341
00:21:24,680 --> 00:21:29,860
of the items and
the second half.
342
00:21:29,860 --> 00:21:33,960
So we use recursion to short
the first n over 2 items
343
00:21:33,960 --> 00:21:35,640
and the last n over 2 items.
344
00:21:46,290 --> 00:21:48,668
And then we're going to
merge those two sorted lists
345
00:21:48,668 --> 00:21:49,167
together.
346
00:21:52,830 --> 00:21:54,830
So we're going to merge
this list and that list.
347
00:21:54,830 --> 00:21:59,170
So let's see how this works and
figure out how many comparisons
348
00:21:59,170 --> 00:22:01,630
we have to do to sort the list.
349
00:22:09,580 --> 00:22:16,140
For example, let's say
we're sorting this list.
350
00:22:16,140 --> 00:22:25,260
10, 7, 23, 5, 2, 4, 3, and 9.
351
00:22:25,260 --> 00:22:29,420
So I've got eight numbers
that I'm going to sort.
352
00:22:29,420 --> 00:22:35,110
So the first step is to
recursively sort that list.
353
00:22:35,110 --> 00:22:44,380
And when I do that, I produce,
so I'm sorting 10, 7, 23, 5.
354
00:22:44,380 --> 00:22:48,725
And that produces, let's see,
I'm going to get 5, 7, 10, 23.
355
00:22:55,050 --> 00:22:59,200
And then I recursively
sort these guys.
356
00:22:59,200 --> 00:23:05,430
And that produces
2, 3, 4, and 9.
357
00:23:08,500 --> 00:23:10,850
I've done comparisons,
but haven't counted them
358
00:23:10,850 --> 00:23:12,640
because they're
in the recursion.
359
00:23:12,640 --> 00:23:16,210
Now I'm going to use
comparisons to merge the lists,
360
00:23:16,210 --> 00:23:21,740
And the way I'm
going to do that is
361
00:23:21,740 --> 00:23:28,150
to look at the smallest item
in each list and compare them.
362
00:23:28,150 --> 00:23:30,150
Because I know the
smallest item overall
363
00:23:30,150 --> 00:23:32,220
is either this one or that one.
364
00:23:32,220 --> 00:23:36,870
And when I compare 2 to
5, 2's the smaller one,
365
00:23:36,870 --> 00:23:39,150
so I start with 2.
366
00:23:39,150 --> 00:23:42,520
And I strike it.
367
00:23:42,520 --> 00:23:47,360
What two items do I compare next
to get the next smallest item?
368
00:23:47,360 --> 00:23:48,520
5 and 3.
369
00:23:48,520 --> 00:23:49,520
Smallest remaining guys.
370
00:23:49,520 --> 00:23:51,110
I get 3.
371
00:23:51,110 --> 00:23:52,960
Scratch that guy.
372
00:23:52,960 --> 00:23:55,290
And I compare 5 and 4.
373
00:23:55,290 --> 00:23:58,310
Get 4, scratch him.
374
00:23:58,310 --> 00:24:00,610
And 5 and 9.
375
00:24:00,610 --> 00:24:02,720
5 is the smallest.
376
00:24:02,720 --> 00:24:05,048
7 and 9.
377
00:24:05,048 --> 00:24:05,952
7.
378
00:24:05,952 --> 00:24:07,000
9 and 10.
379
00:24:09,700 --> 00:24:11,920
And actually now I
don't really have
380
00:24:11,920 --> 00:24:13,230
to do any more comparisons.
381
00:24:13,230 --> 00:24:15,360
All I've got is one list left.
382
00:24:15,360 --> 00:24:18,580
So I can just send
these guys down.
383
00:24:21,330 --> 00:24:22,906
And I produced a sorted list.
384
00:24:27,050 --> 00:24:33,410
Now, how many comparisons do
I have to do in the worst case
385
00:24:33,410 --> 00:24:40,620
if this is n over 2 items here
and this is n over 2 here?
386
00:24:40,620 --> 00:24:43,710
How many comparisons
do I have to do
387
00:24:43,710 --> 00:24:46,080
to produce the n item
sorted list after I
388
00:24:46,080 --> 00:24:49,260
have the sub list sorted?
389
00:24:49,260 --> 00:24:50,860
What is it?
390
00:24:50,860 --> 00:24:51,360
n.
391
00:24:51,360 --> 00:24:53,340
Very close.
392
00:24:53,340 --> 00:24:58,260
n minus 1 is right,
because every time I
393
00:24:58,260 --> 00:25:02,420
do a comparison I pull an
item and stick it in the list.
394
00:25:02,420 --> 00:25:05,020
But once I get down
to the very last item,
395
00:25:05,020 --> 00:25:07,820
just one guy sitting there, I
don't have to do a comparison.
396
00:25:07,820 --> 00:25:08,930
It was just the last guy.
397
00:25:08,930 --> 00:25:11,060
I know he's the largest.
398
00:25:11,060 --> 00:25:15,290
So it could go up to n
minus 1 in the worst case.
399
00:25:15,290 --> 00:25:18,340
In this situation, I
actually did n minus 2
400
00:25:18,340 --> 00:25:20,180
because I was left with
two guys at the end,
401
00:25:20,180 --> 00:25:21,350
I just bring them down.
402
00:25:21,350 --> 00:25:24,480
But it might have been that
the next largest thing to 23
403
00:25:24,480 --> 00:25:29,660
was here and then I would
have done n minus 1.
404
00:25:29,660 --> 00:25:31,740
Any questions?
405
00:25:31,740 --> 00:25:32,540
Yeah.
406
00:25:32,540 --> 00:25:35,380
AUDIENCE: [INAUDIBLE].
407
00:25:35,380 --> 00:25:37,145
Would that make it faster?
408
00:25:37,145 --> 00:25:38,770
TOM LEIGHTON: Well,
that's what's going
409
00:25:38,770 --> 00:25:40,872
to happen with the recursion.
410
00:25:40,872 --> 00:25:42,580
Because the first
thing we're going to do
411
00:25:42,580 --> 00:25:44,288
is when we unfold the
recursion is you're
412
00:25:44,288 --> 00:25:47,610
going to take the first
half, split it in half,
413
00:25:47,610 --> 00:25:50,090
recursively sort them
and then merge them.
414
00:25:50,090 --> 00:25:53,020
So the answer is
that's going to happen,
415
00:25:53,020 --> 00:25:56,150
but we're sort of going to
hide it by the recursion.
416
00:25:56,150 --> 00:26:00,430
All right, you didn't see
me produce that sorted list.
417
00:26:00,430 --> 00:26:02,510
But I would have
done it by taking
418
00:26:02,510 --> 00:26:05,240
10 and 7 sorting them, 5
and 23 and sorting them
419
00:26:05,240 --> 00:26:07,680
and then merging them.
420
00:26:07,680 --> 00:26:08,787
OK, any other questions?
421
00:26:11,230 --> 00:26:12,980
All right, well let's
set up the recursion
422
00:26:12,980 --> 00:26:15,510
to count how many
comparisons are
423
00:26:15,510 --> 00:26:19,280
needed for the whole thing,
not just the last merge phase.
424
00:26:34,200 --> 00:26:36,225
All right, so let's define.
425
00:26:39,250 --> 00:26:41,635
And instead of t sub n,
I'm going to write t of n
426
00:26:41,635 --> 00:26:42,620
as a function.
427
00:26:42,620 --> 00:26:44,420
Makes no difference,
but you'll see people
428
00:26:44,420 --> 00:26:46,440
do recurrences both ways.
429
00:26:46,440 --> 00:26:48,477
t of n or t sub n.
430
00:26:48,477 --> 00:26:50,268
It's going to be the
number of comparisons.
431
00:26:56,180 --> 00:27:02,770
Use my merge sort
to sort n numbers.
432
00:27:02,770 --> 00:27:04,853
In the worst case, worst case.
433
00:27:07,650 --> 00:27:14,270
All right, so we already
know that merging
434
00:27:14,270 --> 00:27:23,810
takes n minus 1 comparisons
in the worst case.
435
00:27:23,810 --> 00:27:26,215
And you always want to
prepare for the worst case.
436
00:27:29,860 --> 00:27:35,010
How many comparisons are
used in the recursive
437
00:27:35,010 --> 00:27:41,030
steps to sort this list and
to sort that list, in general.
438
00:27:41,030 --> 00:27:48,400
tn over 2 to do
this list times 2.
439
00:27:48,400 --> 00:27:50,520
Because you've got two
lists of size n over 2.
440
00:27:50,520 --> 00:27:52,430
Two sub problems
besides n over 2.
441
00:27:52,430 --> 00:27:55,650
Each takes t of n over 2.
442
00:27:55,650 --> 00:28:04,030
All right, so you've got
2 tn over 2 comparisons
443
00:28:04,030 --> 00:28:05,852
for the recursive sorting.
444
00:28:13,010 --> 00:28:24,850
That means that t of n equals
2 t of n over 2 plus n minus 1.
445
00:28:24,850 --> 00:28:26,960
And we should say
what t of 1 is.
446
00:28:26,960 --> 00:28:28,808
What's that?
447
00:28:28,808 --> 00:28:31,390
What's the time to sort?
448
00:28:31,390 --> 00:28:31,960
0.
449
00:28:31,960 --> 00:28:34,430
You don't do any comparisons
if there's one item.
450
00:28:34,430 --> 00:28:36,016
So we'll set that
as our base case.
451
00:28:42,180 --> 00:28:46,710
All right, so let's
find the answer.
452
00:28:46,710 --> 00:28:49,892
So the first thing to always
do when you see a recurrence
453
00:28:49,892 --> 00:28:54,560
is to take a few values,
compute a few values,
454
00:28:54,560 --> 00:28:57,560
and try to guess the answer.
455
00:28:57,560 --> 00:28:59,090
So what's t of 2?
456
00:29:03,020 --> 00:29:03,660
1.
457
00:29:03,660 --> 00:29:04,930
Plug it in.
458
00:29:04,930 --> 00:29:08,220
I get 2t of 1, t of
1's 0, plus 2 minus 1.
459
00:29:08,220 --> 00:29:08,740
That's 1.
460
00:29:11,530 --> 00:29:13,490
And we're only doing
powers of 4 here.
461
00:29:13,490 --> 00:29:14,310
Sorry, powers of 2.
462
00:29:14,310 --> 00:29:15,720
So I go to t of 4 next.
463
00:29:15,720 --> 00:29:16,691
What's t of 4?
464
00:29:22,420 --> 00:29:24,280
5.
465
00:29:24,280 --> 00:29:28,670
Right, because I got double
this is 2 plus 4 minus 1.
466
00:29:28,670 --> 00:29:30,080
That's adding 3.
467
00:29:30,080 --> 00:29:32,630
So I get 5.
468
00:29:32,630 --> 00:29:36,500
All right, let's look at t of 8.
469
00:29:36,500 --> 00:29:43,380
That's twice 5 plus 8 minus 1.
470
00:29:43,380 --> 00:29:43,880
17.
471
00:29:48,320 --> 00:29:51,360
Not so clear what to guess here.
472
00:29:51,360 --> 00:29:54,370
We can try one more.
473
00:29:54,370 --> 00:29:59,480
T of 16 equals 2 times
17 plus 16 minus 1
474
00:29:59,480 --> 00:30:07,280
equals 34 and 15 is 49.
475
00:30:07,280 --> 00:30:09,450
Is that right?
476
00:30:09,450 --> 00:30:10,782
34 and 15.
477
00:30:13,460 --> 00:30:14,496
Any guesses?
478
00:30:18,780 --> 00:30:20,729
You're not going to guess this.
479
00:30:20,729 --> 00:30:21,770
That's not going to work.
480
00:30:21,770 --> 00:30:22,994
That happens.
481
00:30:22,994 --> 00:30:25,160
And a miracle could happen,
you get a divine insight
482
00:30:25,160 --> 00:30:28,730
and guess it, but I
think it's not likely.
483
00:30:28,730 --> 00:30:33,180
All right, so let's
go to plug and chug.
484
00:30:33,180 --> 00:30:34,104
See if that works.
485
00:30:44,370 --> 00:30:46,257
All right, so now plug and chug.
486
00:30:51,620 --> 00:30:54,270
All right, so we
write down t of n
487
00:30:54,270 --> 00:30:57,310
is-- and you always write their
current part last for plug
488
00:30:57,310 --> 00:30:59,830
and chug to keep it simpler.
489
00:30:59,830 --> 00:31:03,790
2 t of n over 2.
490
00:31:03,790 --> 00:31:08,310
And then I substitute, I
plug in for t of n over 2.
491
00:31:08,310 --> 00:31:14,290
So I have n minus 1
plus 2 n over 2 minus 1
492
00:31:14,290 --> 00:31:19,410
plus 2 t of n over 4.
493
00:31:19,410 --> 00:31:22,010
Then I chug.
494
00:31:22,010 --> 00:31:28,710
So I have n minus 1
plus that's n minus 2.
495
00:31:28,710 --> 00:31:30,861
Plus 4 tn over 4.
496
00:31:33,950 --> 00:31:35,578
All right, then
we'll plug in here.
497
00:31:41,630 --> 00:31:47,705
And I get n over 4 minus
1 plus 4 tn over 8.
498
00:31:50,510 --> 00:31:51,365
And now we chug.
499
00:31:57,180 --> 00:32:00,150
That's an n here minus a 4.
500
00:32:03,080 --> 00:32:04,580
I did something wrong.
501
00:32:04,580 --> 00:32:06,670
That should be 2.
502
00:32:06,670 --> 00:32:09,652
4 times 2 is 8 tn over 8.
503
00:32:12,590 --> 00:32:17,470
All right, can you start to
guess the pattern here maybe?
504
00:32:17,470 --> 00:32:19,820
We probably should do
one more to be safe.
505
00:32:19,820 --> 00:32:25,850
But you got an n minus 1 plus
an n minus 2 plus an n minus 4.
506
00:32:25,850 --> 00:32:29,820
And if we do one more, what's
the next term going to be?
507
00:32:29,820 --> 00:32:31,720
n minus 8.
508
00:32:31,720 --> 00:32:33,530
And this is looking
pretty simple.
509
00:32:33,530 --> 00:32:39,830
So we could guess now that
the pattern is n minus 1
510
00:32:39,830 --> 00:32:45,070
plus n minus 2 plus
n minus 4 all the way
511
00:32:45,070 --> 00:32:49,290
out to plus n minus
2 to the i minus 1
512
00:32:49,290 --> 00:32:55,350
plus 2 to the i t
n over 2 to the i.
513
00:32:55,350 --> 00:32:57,160
And now we go all
the way down where
514
00:32:57,160 --> 00:33:02,480
i equals log n, base
2, and see what we get.
515
00:33:05,810 --> 00:33:11,390
1 plus n minus 2 plus all
the way down n minus 2
516
00:33:11,390 --> 00:33:22,270
to the log n minus 1 plus
2 to the log n t of 1.
517
00:33:22,270 --> 00:33:25,070
What's t of 1?
518
00:33:25,070 --> 00:33:25,780
0.
519
00:33:25,780 --> 00:33:26,460
That's nice.
520
00:33:26,460 --> 00:33:28,680
That goes away.
521
00:33:28,680 --> 00:33:31,640
All right, so this is
not looking so bad.
522
00:33:31,640 --> 00:33:42,120
This equals the sum i equals 0
to log n minus 1 of n minus 2
523
00:33:42,120 --> 00:33:42,768
to the i.
524
00:33:45,102 --> 00:33:46,810
All right, and this
will be good practice
525
00:33:46,810 --> 00:33:49,850
maybe for towards the test.
526
00:33:49,850 --> 00:33:52,346
Well, I can split that sum up.
527
00:33:52,346 --> 00:34:02,160
i equals 0 to log n minus 1
of n minus the i equals 0 log
528
00:34:02,160 --> 00:34:04,063
n minus 1 of 2 to the i.
529
00:34:06,841 --> 00:34:07,940
Well, this is easy.
530
00:34:07,940 --> 00:34:12,639
I'm just adding n up a
total of log n times.
531
00:34:12,639 --> 00:34:13,659
So I have n log n.
532
00:34:17,150 --> 00:34:19,050
This is just something
the powers of 2
533
00:34:19,050 --> 00:34:20,460
which we know how to do.
534
00:34:20,460 --> 00:34:26,659
So that's going to be
2 to the log n minus 1.
535
00:34:26,659 --> 00:34:33,730
And so we get our answer
is n log n minus 2
536
00:34:33,730 --> 00:34:39,710
the log n is just n
minus 1 is a plus 1.
537
00:34:39,710 --> 00:34:41,072
Well, we derived an answer.
538
00:34:43,670 --> 00:34:47,060
Let's just go back and
check some of the values.
539
00:34:47,060 --> 00:34:48,870
n equals 1.
540
00:34:48,870 --> 00:34:50,850
That's 0 and I get 1 minus 1.
541
00:34:50,850 --> 00:34:52,630
It works.
542
00:34:52,630 --> 00:34:55,159
n equals 2.
543
00:34:55,159 --> 00:35:00,040
2 log 2 is 2 minus 2 plus 1.
544
00:35:00,040 --> 00:35:01,360
Works.
545
00:35:01,360 --> 00:35:07,900
Plug in 4, I get 4 log 4 is
8 minus 4 is 4 plus 1 is 5.
546
00:35:07,900 --> 00:35:10,060
So it works for those examples.
547
00:35:10,060 --> 00:35:14,690
Now, to be really careful we
should prove it by induction.
548
00:35:14,690 --> 00:35:17,170
Because when you do
enough of these equations,
549
00:35:17,170 --> 00:35:19,920
you make mistakes like I
invariably do even up here.
550
00:35:19,920 --> 00:35:21,590
And even though it's
written down here,
551
00:35:21,590 --> 00:35:23,840
I'll make mistakes
writing it on the board.
552
00:35:23,840 --> 00:35:26,245
And so you want to
check by induction.
553
00:35:30,310 --> 00:35:31,880
OK, so this is the answer.
554
00:35:31,880 --> 00:35:36,600
This is how many comparisons
are done in merge sort.
555
00:35:36,600 --> 00:35:40,550
And the nice thing is this
is growing nearly linearly.
556
00:35:40,550 --> 00:35:41,810
It grows as n log n.
557
00:35:41,810 --> 00:35:44,510
And that's why this algorithm
is used a lot in practice.
558
00:35:44,510 --> 00:35:46,330
Much better than
comparing every item
559
00:35:46,330 --> 00:35:50,000
to every other item, which would
be more like n squared steps.
560
00:35:50,000 --> 00:35:54,270
So much better than a
naive sorting algorithm.
561
00:35:54,270 --> 00:35:59,060
Any questions about
what we did there?
562
00:35:59,060 --> 00:36:05,240
Just one rule of thumb when
you're doing the plug and chug.
563
00:36:05,240 --> 00:36:10,870
Notice that I didn't try to
collapse terms along the way.
564
00:36:10,870 --> 00:36:13,470
It makes it easier sometimes
to see the pattern.
565
00:36:13,470 --> 00:36:17,570
I left it as n minus 1 plus n
minus 2 and then plus n minus 4
566
00:36:17,570 --> 00:36:19,350
rather than saying,
oh, this is 3 n
567
00:36:19,350 --> 00:36:23,560
minus 7, which might have made
it harder to guess the pattern.
568
00:36:23,560 --> 00:36:26,130
So you don't always, when you're
doing the chugging, collapse
569
00:36:26,130 --> 00:36:26,810
everything down.
570
00:36:26,810 --> 00:36:28,440
You want to try and
see if there are some.
571
00:36:28,440 --> 00:36:30,050
Because this is the amount
of work done at each step,
572
00:36:30,050 --> 00:36:32,258
and you want to see if you
can see a pattern in that.
573
00:36:37,070 --> 00:36:39,250
All right, so let's write
down our two recurrences
574
00:36:39,250 --> 00:36:42,090
and sort of compare them.
575
00:36:52,870 --> 00:36:57,000
So first we did Towers of
Hanoi, which was t of n
576
00:36:57,000 --> 00:37:02,380
equals 2 t of n minus 1 plus 1.
577
00:37:02,380 --> 00:37:08,455
And the answer was t of
n is tilde 2 to the n.
578
00:37:08,455 --> 00:37:11,680
It was actually 2
to the n minus 1.
579
00:37:11,680 --> 00:37:19,760
Then we did merge sort, which is
t of n equals 2 t of n over 2.
580
00:37:19,760 --> 00:37:21,240
Difference there.
581
00:37:21,240 --> 00:37:23,990
Plus n minus 1.
582
00:37:23,990 --> 00:37:26,970
And now the answer was t of n.
583
00:37:26,970 --> 00:37:29,540
What's the tilde for this one?
584
00:37:29,540 --> 00:37:32,600
For merge sort.
585
00:37:32,600 --> 00:37:33,600
n log n.
586
00:37:33,600 --> 00:37:36,433
Yeah, I can forget the rest
if I know the tilde, n log n.
587
00:37:40,140 --> 00:37:42,360
Wow.
588
00:37:42,360 --> 00:37:45,290
These are pretty
different answers, right?
589
00:37:45,290 --> 00:37:48,170
This one's nearly linear,
that one's exponential.
590
00:37:50,810 --> 00:37:54,320
The recurrences
looks pretty similar,
591
00:37:54,320 --> 00:37:55,970
in fact, this one
sort of looks worse
592
00:37:55,970 --> 00:38:00,960
because I'm adding in n minus
1 every time instead of 1.
593
00:38:00,960 --> 00:38:05,060
But the answer's a lot
better, a lot smaller.
594
00:38:05,060 --> 00:38:09,030
What's the critical difference
between these recurrences?
595
00:38:09,030 --> 00:38:09,530
Yeah?
596
00:38:12,440 --> 00:38:17,201
AUDIENCE: You're dropping the
argument of t a lot faster.
597
00:38:17,201 --> 00:38:18,700
You're cutting it
in half every time
598
00:38:18,700 --> 00:38:19,990
instead of subtracting one.
599
00:38:19,990 --> 00:38:21,010
TOM LEIGHTON: Right.
600
00:38:21,010 --> 00:38:22,430
AUDIENCE: So it's going
to go away faster.
601
00:38:22,430 --> 00:38:23,555
TOM LEIGHTON: That's right.
602
00:38:23,555 --> 00:38:26,560
Here I've got two
problems of size one less.
603
00:38:26,560 --> 00:38:30,230
Here I've got two
problems of half a size.
604
00:38:30,230 --> 00:38:33,450
And that makes an enormous
difference in the answer.
605
00:38:33,450 --> 00:38:36,450
Even though I'm
doing more work after
606
00:38:36,450 --> 00:38:39,700
did the recursive problems,
much faster running
607
00:38:39,700 --> 00:38:45,540
time or answer if these are
algorithms that I'm analyzing.
608
00:38:45,540 --> 00:38:47,520
All right?
609
00:38:47,520 --> 00:38:50,330
In fact, let's
take a look at what
610
00:38:50,330 --> 00:38:55,050
happens if I didn't use that n
minus 1, I just put a 1 there.
611
00:38:55,050 --> 00:38:59,590
Let's see what impact
that n minus 1 had.
612
00:38:59,590 --> 00:39:05,080
So let's do the
following recurrence.
613
00:39:05,080 --> 00:39:09,640
This one is going to be a little
bit different because I'm not
614
00:39:09,640 --> 00:39:10,900
going to stick to powers of 2.
615
00:39:10,900 --> 00:39:13,130
I'm going to get a little
messier, because that often
616
00:39:13,130 --> 00:39:15,130
happens in reality.
617
00:39:15,130 --> 00:39:20,690
And I'm going to
define s of 1 is 0.
618
00:39:20,690 --> 00:39:29,600
s of n is s of the floor
function of n over 2
619
00:39:29,600 --> 00:39:35,540
plus s of the ceiling
function of n over 2 plus 1.
620
00:39:40,120 --> 00:39:45,460
Now, the ceiling function
means the smallest integer
621
00:39:45,460 --> 00:39:46,947
that's at least that big.
622
00:39:46,947 --> 00:39:49,030
So you're sort of rounding
up to the next integer.
623
00:40:00,260 --> 00:40:02,850
And the floor function
means you're rounding down
624
00:40:02,850 --> 00:40:04,430
to the next integer.
625
00:40:04,430 --> 00:40:13,190
So it's the biggest integer
less than equal to n over 2.
626
00:40:13,190 --> 00:40:14,010
All right.
627
00:40:14,010 --> 00:40:17,480
Now, if we had just
powers of 2, this
628
00:40:17,480 --> 00:40:19,380
would be much easier to write.
629
00:40:19,380 --> 00:40:24,985
It would just be sn
equals 2 sn over 2 plus 1.
630
00:40:24,985 --> 00:40:27,610
So it's analogous to what we're
doing up there where we're just
631
00:40:27,610 --> 00:40:29,746
adding 1 instead of n minus 1.
632
00:40:29,746 --> 00:40:30,870
But I'm being careful. now.
633
00:40:30,870 --> 00:40:32,245
I'm going to make
this recurrence
634
00:40:32,245 --> 00:40:35,561
work for all natural numbers.
635
00:40:35,561 --> 00:40:37,060
Because sometimes
in reality, you're
636
00:40:37,060 --> 00:40:39,250
dealing with non powers of 2.
637
00:40:43,440 --> 00:40:47,580
All right, let's see
if we can solve this.
638
00:40:47,580 --> 00:40:50,170
What's the first thing
to try to solve this?
639
00:40:50,170 --> 00:40:50,890
What do you do?
640
00:40:56,520 --> 00:40:59,060
What's the first step
when you get a recurrence?
641
00:40:59,060 --> 00:41:00,142
You try to solve.
642
00:41:03,300 --> 00:41:06,420
What method should you try?
643
00:41:06,420 --> 00:41:07,470
Guess.
644
00:41:07,470 --> 00:41:10,840
And to guess, well, you
got to plug some values in.
645
00:41:10,840 --> 00:41:12,720
It's sort of hard to guess.
646
00:41:12,720 --> 00:41:15,940
I mean, looking at that, I guess
nothing comes to mind other
647
00:41:15,940 --> 00:41:18,820
than panic looking at that.
648
00:41:18,820 --> 00:41:21,470
But if we plug in some values,
then maybe it won't be so bad.
649
00:41:21,470 --> 00:41:25,200
So we've got s1 equals 0.
650
00:41:25,200 --> 00:41:26,026
What's s2?
651
00:41:34,040 --> 00:41:39,380
It's not 3, because this is 2
over 2 is 1, floor of 1 is 1.
652
00:41:39,380 --> 00:41:42,590
s of 1 is 0.
653
00:41:42,590 --> 00:41:45,460
Ceiling of 1 is 1
because it's an integer.
654
00:41:45,460 --> 00:41:48,630
s of 1 is 0 plus 1.
655
00:41:48,630 --> 00:41:51,400
So s of 2 is 1.
656
00:41:51,400 --> 00:41:53,530
What's s of 3?
657
00:41:53,530 --> 00:41:55,510
Yikes, all right.
658
00:41:55,510 --> 00:41:57,510
3 over 2 is 1 and 1/2.
659
00:41:57,510 --> 00:41:59,890
What's the floor of 1 and 1/2?
660
00:41:59,890 --> 00:42:00,460
1.
661
00:42:00,460 --> 00:42:01,360
s of 1 is 0.
662
00:42:01,360 --> 00:42:02,530
That's gone.
663
00:42:02,530 --> 00:42:05,090
What's the ceiling of 1 and 1/2?
664
00:42:05,090 --> 00:42:05,780
2.
665
00:42:05,780 --> 00:42:09,470
s of 2 is 1 plus 1.
666
00:42:09,470 --> 00:42:10,800
What's the answer for s of 3?
667
00:42:14,070 --> 00:42:20,120
2, because I've got 0, 1, and 1.
668
00:42:20,120 --> 00:42:21,710
s of 4.
669
00:42:21,710 --> 00:42:25,540
This one's easy
because it's even.
670
00:42:25,540 --> 00:42:31,500
I get s of 2, which is 1 plus
s of two which is 1 plus 1.
671
00:42:31,500 --> 00:42:33,500
So what's s of 4?
672
00:42:33,500 --> 00:42:36,040
3.
673
00:42:36,040 --> 00:42:38,880
You might try a guess.
674
00:42:38,880 --> 00:42:42,010
What would you guess?
675
00:42:42,010 --> 00:42:43,860
Yeah, n minus 1.
676
00:42:43,860 --> 00:42:51,297
So we're going to guess
s of n equals n minus 1.
677
00:42:51,297 --> 00:42:52,630
And that's not so bad, actually.
678
00:42:52,630 --> 00:42:53,421
It's pretty simple.
679
00:42:53,421 --> 00:42:56,900
Looking at that mess, you'd
never think of that right away.
680
00:42:56,900 --> 00:43:00,250
But once you plug in some
values it's not so bad.
681
00:43:00,250 --> 00:43:02,012
All right, let's verify it.
682
00:43:05,270 --> 00:43:07,261
And we're going to verify
by strong induction.
683
00:43:14,790 --> 00:43:16,720
The induction
hypothesis is our guess.
684
00:43:26,704 --> 00:43:28,560
The base case is easy.
685
00:43:31,870 --> 00:43:36,820
Because I've got s of 1 is 0.
686
00:43:36,820 --> 00:43:40,160
And that equals 1 minus 1.
687
00:43:40,160 --> 00:43:41,970
So that's good.
688
00:43:41,970 --> 00:43:44,124
So let's do the induction step.
689
00:44:00,530 --> 00:44:02,340
All right, so the
induction step.
690
00:44:07,690 --> 00:44:15,850
We're going to assume that
p1 is true, p2 is true,
691
00:44:15,850 --> 00:44:24,128
all the way up to pn, to
prove pn plus 1 is true.
692
00:44:26,820 --> 00:44:28,690
For n better than or equal to 1.
693
00:44:28,690 --> 00:44:32,880
And then we look at
the n plus first term.
694
00:44:32,880 --> 00:44:41,990
That equals s of floor n plus
1 over 2 plus s of ceiling n
695
00:44:41,990 --> 00:44:47,710
plus 1 over 2 plus 1.
696
00:44:47,710 --> 00:44:52,470
All right, now I don't know what
these are exactly depending on
697
00:44:52,470 --> 00:44:55,820
whether n is even a rod.
698
00:44:55,820 --> 00:45:00,070
First let me plug in using
the induction hypothesis.
699
00:45:00,070 --> 00:45:06,040
I know that this is n
plus 1 over 2 minus 1.
700
00:45:06,040 --> 00:45:11,850
This is ceiling n
plus 1 over 2 minus 1.
701
00:45:11,850 --> 00:45:13,400
And then I carry
the plus 1 down.
702
00:45:13,400 --> 00:45:15,350
That's by the
induction hypothesis.
703
00:45:20,760 --> 00:45:23,600
All right, now I don't
know exactly what
704
00:45:23,600 --> 00:45:26,640
either of these values is
unless I know the parity of n.
705
00:45:26,640 --> 00:45:30,680
But I do know the sum
of these two things.
706
00:45:30,680 --> 00:45:34,150
The floor of something plus
the ceiling of that something
707
00:45:34,150 --> 00:45:37,600
is twice the something.
708
00:45:37,600 --> 00:45:42,940
All right, so this plus
that equals n plus 1.
709
00:45:42,940 --> 00:45:46,740
Minus 1 minus 1
plus 1 is minus 1.
710
00:45:46,740 --> 00:45:49,670
And so in fact, we
prove that s of n plus 1
711
00:45:49,670 --> 00:45:54,040
equals n plus 1 minus 1.
712
00:45:54,040 --> 00:45:58,680
So we verified the
induction hypothesis.
713
00:45:58,680 --> 00:45:59,591
Yeah?
714
00:45:59,591 --> 00:46:00,507
AUDIENCE: [INAUDIBLE].
715
00:46:08,040 --> 00:46:09,150
Taking the n [INAUDIBLE].
716
00:46:11,877 --> 00:46:13,460
TOM LEIGHTON: Let's
see, I'm not sure.
717
00:46:13,460 --> 00:46:18,282
So are you asking why I
started sn plus 1 here?
718
00:46:18,282 --> 00:46:19,198
AUDIENCE: [INAUDIBLE].
719
00:46:27,332 --> 00:46:28,540
TOM LEIGHTON: Doesn't matter.
720
00:46:28,540 --> 00:46:33,140
But the expression
is s of whatever
721
00:46:33,140 --> 00:46:36,380
this is, the induction
hypothesis would be,
722
00:46:36,380 --> 00:46:40,240
s of whatever's inside there
is what's inside there minus 1.
723
00:46:40,240 --> 00:46:45,178
And so the induction hypothesis
says s of this is that minus 1.
724
00:46:48,437 --> 00:46:49,270
Any other questions?
725
00:46:55,260 --> 00:46:57,086
So let's write that
on our table up here.
726
00:47:01,950 --> 00:47:07,980
So this is sn equals
s of floor n over 2
727
00:47:07,980 --> 00:47:13,926
plus s of ceiling
n over 2 plus 1.
728
00:47:13,926 --> 00:47:19,020
And the answer is, sn is tilde.
729
00:47:19,020 --> 00:47:20,010
What's the tilde of?
730
00:47:23,140 --> 00:47:23,680
n.
731
00:47:23,680 --> 00:47:24,680
The answer is n minus 1.
732
00:47:24,680 --> 00:47:27,850
So it's tilde of n.
733
00:47:27,850 --> 00:47:32,310
Now, this is the same expression
is this if n's a power of 2.
734
00:47:32,310 --> 00:47:37,440
I'd have 2 sn over 2 plus
1 instead of n minus 1.
735
00:47:37,440 --> 00:47:44,450
And by going from n minus 1 to
1, I got rid of the log term.
736
00:47:44,450 --> 00:47:48,040
But it's still not exponential.
737
00:47:48,040 --> 00:47:50,370
And that's because I'm
cutting down the insides
738
00:47:50,370 --> 00:47:53,388
here by a factor of 2 roughly.
739
00:47:59,260 --> 00:48:03,300
All right, so all the
recurrences we've seen so far
740
00:48:03,300 --> 00:48:05,540
have been pretty simple.
741
00:48:05,540 --> 00:48:08,330
If guess and verify doesn't
work after seeing a few terms,
742
00:48:08,330 --> 00:48:11,750
you can use plug and chug
and you can figure it out.
743
00:48:11,750 --> 00:48:15,640
The good news is, that
happens a lot in practice.
744
00:48:15,640 --> 00:48:18,300
The bad news is, sometimes
it doesn't and you
745
00:48:18,300 --> 00:48:19,630
can get nasty recurrences.
746
00:48:19,630 --> 00:48:27,250
So let me show you a nasty
one that actually was once
747
00:48:27,250 --> 00:48:37,417
asked in 6046, I think, on a
quiz with disastrous results.
748
00:48:46,340 --> 00:48:57,960
All right, so this one is t
of x equals 2 t of x over 2
749
00:48:57,960 --> 00:49:08,260
plus 8/9 t of 3x
over 4 plus x squared
750
00:49:08,260 --> 00:49:11,720
for x bigger and equal
to 1, and otherwise it's
751
00:49:11,720 --> 00:49:14,410
0 when x is less than 1.
752
00:49:18,230 --> 00:49:21,890
Now, this is nasty
for a lot of reasons.
753
00:49:21,890 --> 00:49:25,610
First, it's not just a
find on the integers.
754
00:49:25,610 --> 00:49:28,690
We can make an integer version
by putting floors or ceilings
755
00:49:28,690 --> 00:49:29,500
here.
756
00:49:29,500 --> 00:49:32,730
But the way you set it up now,
it's defined on real numbers.
757
00:49:36,650 --> 00:49:41,150
You can, in theory, solve
it with plug and chug.
758
00:49:41,150 --> 00:49:44,520
The nasty part is, you've got
a bunch of things going down
759
00:49:44,520 --> 00:49:49,410
by powers of 2 and a bunch of
things going down by 3/4 here.
760
00:49:49,410 --> 00:49:51,400
And so you have to keep
track of all the terms
761
00:49:51,400 --> 00:49:54,179
that are decreasing, and that's
a nightmare plug and chug.
762
00:49:54,179 --> 00:49:56,220
Because it's not just one
thing that's unfolding,
763
00:49:56,220 --> 00:49:58,890
it's two things
that are unfolding.
764
00:49:58,890 --> 00:50:01,280
So very painful.
765
00:50:01,280 --> 00:50:05,390
Now, it used to be that
was the state of life.
766
00:50:05,390 --> 00:50:09,080
20 years ago, I think we started
teaching this course sometime
767
00:50:09,080 --> 00:50:11,700
around then, there
just was nothing else
768
00:50:11,700 --> 00:50:14,870
to say other than life's
tough and sometimes you've
769
00:50:14,870 --> 00:50:18,030
got to do these things
and it's a pain.
770
00:50:18,030 --> 00:50:20,140
Just had to make the best of it.
771
00:50:20,140 --> 00:50:24,730
And then in the mid-90s,
one day two students
772
00:50:24,730 --> 00:50:28,900
at the American University
in Beirut came by my office.
773
00:50:28,900 --> 00:50:32,590
They were named Akra and Bazzi.
774
00:50:32,590 --> 00:50:34,130
And they'd actually
come to Boston
775
00:50:34,130 --> 00:50:35,530
to escape the daily bombings.
776
00:50:35,530 --> 00:50:37,900
This was when all heck was
breaking loose in Beirut.
777
00:50:37,900 --> 00:50:39,139
It was a mess.
778
00:50:39,139 --> 00:50:40,680
And they come into
my office and they
779
00:50:40,680 --> 00:50:44,110
claim to have a simple
formula for solving
780
00:50:44,110 --> 00:50:46,460
any recurrence like this.
781
00:50:46,460 --> 00:50:51,590
In fact, you can even
add a 5 t of x over 7.
782
00:50:51,590 --> 00:50:52,580
They said no problem.
783
00:50:52,580 --> 00:50:56,490
Their simple formula
just will always work.
784
00:50:56,490 --> 00:51:00,100
And the general form
that they claim to solve
785
00:51:00,100 --> 00:51:02,020
was the following.
786
00:51:02,020 --> 00:51:06,490
It's the general class of
divide and conquer recurrences.
787
00:51:06,490 --> 00:51:08,460
So any recurrence you
get from an algorithm
788
00:51:08,460 --> 00:51:12,130
where you break it into smaller
pieces by a constant factor,
789
00:51:12,130 --> 00:51:14,390
they claim to solve.
790
00:51:14,390 --> 00:51:16,810
So let me define this class.
791
00:51:16,810 --> 00:51:18,290
It's a little painful.
792
00:51:28,330 --> 00:51:34,268
So a divide and conquer
recurrence has the form.
793
00:51:38,450 --> 00:51:41,290
There's a simple version
and the hard version.
794
00:51:41,290 --> 00:51:45,080
The simple version is
it looks like that.
795
00:51:45,080 --> 00:51:47,210
You got a bunch of terms.
796
00:51:47,210 --> 00:51:50,460
Inside each term, you cut
it down by a constant factor
797
00:51:50,460 --> 00:51:52,290
and then you add
something at the end.
798
00:51:52,290 --> 00:51:55,020
That's the simple version
of what a divide and conquer
799
00:51:55,020 --> 00:51:56,130
recurrence is.
800
00:51:56,130 --> 00:51:59,856
And you can throw in floors and
ceilings and other junk too.
801
00:51:59,856 --> 00:52:01,730
Now I'm going to give
you the formal version.
802
00:52:04,950 --> 00:52:11,120
So it has the form t of x
equals some constant a1 times
803
00:52:11,120 --> 00:52:17,080
t of a constant less than
b1 of x, plus some slop
804
00:52:17,080 --> 00:52:20,170
you're allowed to have
like floors and ceilings
805
00:52:20,170 --> 00:52:22,170
and who knows what else.
806
00:52:22,170 --> 00:52:23,630
And then you get more of these.
807
00:52:23,630 --> 00:52:33,450
You get a2, t of b2x plus
epsilon 2x and so forth.
808
00:52:33,450 --> 00:52:34,950
And you can have
any number of these
809
00:52:34,950 --> 00:52:37,120
you want up to a constant.
810
00:52:37,120 --> 00:52:43,285
So maybe there's a sub k t
b sub kx plus epsilon kx.
811
00:52:45,860 --> 00:52:51,330
Plus some function of x sitting
out here, like the x squared.
812
00:52:51,330 --> 00:52:55,520
And this happens for x
bigger than some fixed value,
813
00:52:55,520 --> 00:52:57,690
some constant.
814
00:52:57,690 --> 00:53:03,096
Now, the constraints are,
well, the ai's are positive
815
00:53:03,096 --> 00:53:03,595
constants.
816
00:53:06,550 --> 00:53:12,650
The bi's are positive
and less than 1.
817
00:53:12,650 --> 00:53:15,532
You gotta cut it down by a
constant factor to be a divide
818
00:53:15,532 --> 00:53:16,490
and conquer recurrence.
819
00:53:16,490 --> 00:53:17,156
That's critical.
820
00:53:19,740 --> 00:53:21,660
k is fixed.
821
00:53:21,660 --> 00:53:22,330
It's constant.
822
00:53:25,720 --> 00:53:29,990
And these epsilon
functions can't be too big.
823
00:53:29,990 --> 00:53:34,390
They can't be bigger than 0
of x over log squared of x.
824
00:53:34,390 --> 00:53:37,100
Almost as big as x, but you
can't let them be too big
825
00:53:37,100 --> 00:53:42,390
or you'll not be cutting things
down by a constant factor.
826
00:53:42,390 --> 00:53:44,260
Now, you don't get
too hung up over this
827
00:53:44,260 --> 00:53:48,030
because you don't get
too deep in the details.
828
00:53:48,030 --> 00:53:53,146
But this is that most 0 of
x over log squared of x.
829
00:53:55,840 --> 00:54:00,550
And finally, the g thing
has got to be polynomial.
830
00:54:00,550 --> 00:54:01,560
It can't be an exponent.
831
00:54:01,560 --> 00:54:03,720
You can't put 2 to the x here.
832
00:54:03,720 --> 00:54:05,510
And the formal way
that's expressed
833
00:54:05,510 --> 00:54:11,720
is the derivative of
g of x is less than x
834
00:54:11,720 --> 00:54:13,820
to the c for some constant c.
835
00:54:18,450 --> 00:54:19,020
That's it.
836
00:54:19,020 --> 00:54:20,900
So this is a mouthful.
837
00:54:20,900 --> 00:54:23,560
But like I said before,
anything that looks
838
00:54:23,560 --> 00:54:26,520
like that, basically.
839
00:54:26,520 --> 00:54:28,270
As long as you don't
throw in exponentials
840
00:54:28,270 --> 00:54:32,370
or do something too
wild to it, it works.
841
00:54:32,370 --> 00:54:36,790
So let's see which of our
guys over here are divide
842
00:54:36,790 --> 00:54:37,790
and conquer recurrences.
843
00:54:43,390 --> 00:54:44,210
How about this one?
844
00:54:44,210 --> 00:54:49,300
Towers of Hanoi, is that a
divide and conquer recurrence?
845
00:54:49,300 --> 00:54:52,350
Some yes, some no.
846
00:54:52,350 --> 00:54:54,540
Who thinks no can
tell me why it's not?
847
00:54:54,540 --> 00:54:55,479
Why isn't that?
848
00:54:55,479 --> 00:54:56,395
AUDIENCE: [INAUDIBLE].
849
00:54:58,936 --> 00:55:00,060
TOM LEIGHTON: That's right.
850
00:55:00,060 --> 00:55:03,290
The recurrence
problem is n minus 1
851
00:55:03,290 --> 00:55:07,300
and we've got to make it down
by a constant factor here.
852
00:55:07,300 --> 00:55:09,080
The b1 is less than 1.
853
00:55:09,080 --> 00:55:10,820
It can't be equal to 1.
854
00:55:10,820 --> 00:55:14,330
And this is not big enough
to make up for the x over 2
855
00:55:14,330 --> 00:55:16,600
term or something we're missing.
856
00:55:16,600 --> 00:55:19,690
So whenever you have an
n minus 1 and minus 2,
857
00:55:19,690 --> 00:55:22,690
that is not divide and conquer.
858
00:55:22,690 --> 00:55:24,860
We'll discuss those
next time, on Thursday,
859
00:55:24,860 --> 00:55:26,510
how to solve those.
860
00:55:26,510 --> 00:55:29,957
So that's not
divide and conquer.
861
00:55:29,957 --> 00:55:30,790
What about this one?
862
00:55:30,790 --> 00:55:32,180
Is that divide and conquer here?
863
00:55:35,480 --> 00:55:36,710
Yes.
864
00:55:36,710 --> 00:55:39,750
You've got two problems
of half the size
865
00:55:39,750 --> 00:55:42,060
and you're adding
a linear function.
866
00:55:42,060 --> 00:55:43,310
It's not exponential.
867
00:55:43,310 --> 00:55:45,130
It's polynomial.
868
00:55:45,130 --> 00:55:47,710
That is divide and conquer.
869
00:55:47,710 --> 00:55:50,822
What about this nasty guy?
870
00:55:50,822 --> 00:55:51,946
Is that divide and conquer?
871
00:55:54,700 --> 00:55:55,940
Yeah.
872
00:55:55,940 --> 00:55:58,400
You've got basically
n over 2 here.
873
00:55:58,400 --> 00:56:00,320
Two n over 2 terms now.
874
00:56:00,320 --> 00:56:03,200
This is where that epsilon
thing comes in handy.
875
00:56:03,200 --> 00:56:05,600
Because epsilon 1
of n, in this case,
876
00:56:05,600 --> 00:56:13,020
is floor n over
2 minus n over 2.
877
00:56:13,020 --> 00:56:14,876
And that's, at most, 1.
878
00:56:18,180 --> 00:56:21,570
So maybe we're adding a half
here or something like that.
879
00:56:21,570 --> 00:56:23,070
And we're allowed
to add anything up
880
00:56:23,070 --> 00:56:27,240
to n over log
squared of n to fit.
881
00:56:27,240 --> 00:56:30,580
So you can mess around with
the low order terms in here.
882
00:56:30,580 --> 00:56:33,560
And adding 1 is fine as well.
883
00:56:33,560 --> 00:56:34,432
So that qualifies.
884
00:56:37,670 --> 00:56:40,830
Let's check that
this thing qualifies.
885
00:56:40,830 --> 00:56:41,866
Let's check that.
886
00:56:50,570 --> 00:56:51,280
Well, yeah.
887
00:56:51,280 --> 00:56:57,840
I got 2 constant a1 t of 1/2 x.
888
00:56:57,840 --> 00:56:59,050
That's OK.
889
00:56:59,050 --> 00:57:01,450
8/9 t of 3/4 x.
890
00:57:01,450 --> 00:57:02,850
That's OK.
891
00:57:02,850 --> 00:57:04,690
My g function is a polynomial.
892
00:57:04,690 --> 00:57:06,020
The derivative is 2x.
893
00:57:06,020 --> 00:57:09,800
That's less than x to a
power or of x to a power.
894
00:57:09,800 --> 00:57:10,455
So this fits.
895
00:57:14,430 --> 00:57:16,860
Any questions about
the definition
896
00:57:16,860 --> 00:57:18,486
of a divide and
conquer recurrence?
897
00:57:23,722 --> 00:57:25,430
And you can see why
sort of they come up,
898
00:57:25,430 --> 00:57:28,800
because whenever you're solving
a problem with recursive pieces
899
00:57:28,800 --> 00:57:30,780
that are a constant
factor smaller,
900
00:57:30,780 --> 00:57:34,140
you're going to have something
that looks like that.
901
00:57:34,140 --> 00:57:36,100
OK, so getting back
to Akra and Bazzi.
902
00:57:36,100 --> 00:57:38,580
B They show up in my office.
903
00:57:38,580 --> 00:57:41,050
They claim to have solved
anything like this.
904
00:57:41,050 --> 00:57:44,817
Now needless to say,
I'm a little skeptical.
905
00:57:44,817 --> 00:57:47,150
I got to tell you, you're a
professor at MIT, especially
906
00:57:47,150 --> 00:57:49,790
mathematics, and
people are always
907
00:57:49,790 --> 00:57:54,020
coming by with proofs of
the Riemann hypothesis,
908
00:57:54,020 --> 00:57:57,210
p equals np, p doesn't equal np.
909
00:57:57,210 --> 00:58:00,020
And this was a classic problem.
910
00:58:00,020 --> 00:58:01,960
Hundreds of
researchers had studied
911
00:58:01,960 --> 00:58:04,960
recurrences for decades.
912
00:58:04,960 --> 00:58:06,670
And not only, it
wasn't even believed
913
00:58:06,670 --> 00:58:08,792
that there was
possibly a solution.
914
00:58:08,792 --> 00:58:11,250
It's not like it's one of these
open questions where we all
915
00:58:11,250 --> 00:58:13,300
believe p doesn't
equal np and it's just
916
00:58:13,300 --> 00:58:14,930
a matter of proving it.
917
00:58:14,930 --> 00:58:18,770
People didn't think a solution
existed that would be clean.
918
00:58:18,770 --> 00:58:20,970
I mean, look at just
the definition's a mess.
919
00:58:20,970 --> 00:58:25,060
And how could there possibly
be a clean solution to that?
920
00:58:25,060 --> 00:58:28,020
So it's sort of
like the holy grail.
921
00:58:28,020 --> 00:58:29,940
It doesn't exist, but
it's fun to think about,
922
00:58:29,940 --> 00:58:35,370
that fantasy, maybe there's a
solution and you could find it.
923
00:58:35,370 --> 00:58:38,380
And much less, these aren't
reputable researchers
924
00:58:38,380 --> 00:58:38,980
in my office.
925
00:58:38,980 --> 00:58:41,860
These are two
students from Beirut
926
00:58:41,860 --> 00:58:44,740
named Akra and Bazzi
claiming they've figured out
927
00:58:44,740 --> 00:58:47,030
this thing that all these
professors been teaching
928
00:58:47,030 --> 00:58:50,830
this stuff in 6046 for
years said you've got to do
929
00:58:50,830 --> 00:58:53,350
plug and chug.
930
00:58:53,350 --> 00:58:56,420
Anyway, they'd been kicked out
of everybody else's office,
931
00:58:56,420 --> 00:58:58,320
so I said, OK, let's do it.
932
00:58:58,320 --> 00:59:00,650
Show me the solution.
933
00:59:00,650 --> 00:59:01,930
And they did.
934
00:59:01,930 --> 00:59:04,280
And I go, holy cow.
935
00:59:04,280 --> 00:59:05,297
It's right.
936
00:59:05,297 --> 00:59:05,880
It looks good.
937
00:59:05,880 --> 00:59:07,296
Now, they had a
few details wrong,
938
00:59:07,296 --> 00:59:09,560
but my goodness, the
answer was correct
939
00:59:09,560 --> 00:59:12,680
and it's amazingly simple.
940
00:59:12,680 --> 00:59:15,260
So now we teach it.
941
00:59:15,260 --> 00:59:18,610
And in fact, you can
write the solution
942
00:59:18,610 --> 00:59:21,800
down easier than you can
write the definition of divide
943
00:59:21,800 --> 00:59:24,870
and conquer recurrence.
944
00:59:24,870 --> 00:59:25,790
Really remarkable.
945
00:59:28,870 --> 00:59:39,950
So the theorem by
Akra and Bazzi in '96.
946
00:59:44,290 --> 00:59:55,240
Set p, real number, So that
the sum i equals 1 to k
947
00:59:55,240 --> 01:00:00,720
of ai times bi to
the p equals 1.
948
01:00:00,720 --> 01:00:04,160
So the ai's are
these things here.
949
01:00:04,160 --> 01:00:07,720
The bi's are the fractions here.
950
01:00:07,720 --> 01:00:13,540
All right, find a p
such that this sum is 1.
951
01:00:13,540 --> 01:00:16,920
Then the answer is this.
952
01:00:16,920 --> 01:00:21,740
Then t of x equals theta.
953
01:00:21,740 --> 01:00:24,030
We're going to use our
asymptotic notation.
954
01:00:24,030 --> 01:00:29,180
x to the p plus x to
the p times the integral
955
01:00:29,180 --> 01:00:33,060
from 1 to x g of u.
956
01:00:33,060 --> 01:00:37,070
G is that polynomial thing,
that thing at the end.
957
01:00:37,070 --> 01:00:40,020
Over u to the p minus 1 du.
958
01:00:43,050 --> 01:00:45,460
That's it.
959
01:00:45,460 --> 01:00:47,910
This is really great
news for you guys,
960
01:00:47,910 --> 01:00:49,940
especially if you
go on to algorithms,
961
01:00:49,940 --> 01:00:51,606
because you don't
have to deal with plug
962
01:00:51,606 --> 01:00:53,460
and chug on nasty things.
963
01:00:53,460 --> 01:00:55,896
You just solve the answer.
964
01:00:58,790 --> 01:01:03,300
Now, the proof is
by guess and verify.
965
01:01:03,300 --> 01:01:04,875
And they made a
really good guess.
966
01:01:07,510 --> 01:01:09,951
You wouldn't think of
looking at that, oh,
967
01:01:09,951 --> 01:01:10,950
I'm going to guess this.
968
01:01:10,950 --> 01:01:13,070
Probably not.
969
01:01:13,070 --> 01:01:16,310
And there's no real numbers
to try to guess this.
970
01:01:16,310 --> 01:01:18,850
It's asymptotic notation.
971
01:01:18,850 --> 01:01:19,860
But it works.
972
01:01:19,860 --> 01:01:21,640
You can verify it by induction.
973
01:01:21,640 --> 01:01:23,920
It's a little painful
to verify by induction,
974
01:01:23,920 --> 01:01:24,880
but you can do it.
975
01:01:24,880 --> 01:01:26,580
We won't do it in class.
976
01:01:26,580 --> 01:01:28,660
I did try one year,
and it was a disaster,
977
01:01:28,660 --> 01:01:31,400
so we don't do it
in class anymore.
978
01:01:31,400 --> 01:01:33,990
But let's see some examples
of how to apply this.
979
01:01:36,750 --> 01:01:42,481
So let's start with merge sort,
which is the second one here.
980
01:01:42,481 --> 01:01:43,064
Let's do that.
981
01:01:53,190 --> 01:01:56,390
All right, so you
got an example here.
982
01:01:56,390 --> 01:02:06,760
t of x equals 2 t of x over 2
plus n minus 1, or x minus 1, x
983
01:02:06,760 --> 01:02:07,860
now.
984
01:02:07,860 --> 01:02:10,630
And I'm not even going to worry
about powers of 2 anymore.
985
01:02:10,630 --> 01:02:11,210
I don't care.
986
01:02:11,210 --> 01:02:14,210
It's going to be fine.
987
01:02:14,210 --> 01:02:17,670
All right, what
is p in this case?
988
01:02:17,670 --> 01:02:18,780
I've only got one term.
989
01:02:18,780 --> 01:02:22,260
I don't even need a sum from
that definition over there.
990
01:02:22,260 --> 01:02:27,400
What's the value
of p for this guy?
991
01:02:27,400 --> 01:02:30,760
So I've got a1 is 2.
992
01:02:30,760 --> 01:02:33,280
b1 is 1/2.
993
01:02:33,280 --> 01:02:34,202
Just one term.
994
01:02:37,320 --> 01:02:42,900
What's the value of p such that
2 times 1/2 to the p equals 1?
995
01:02:42,900 --> 01:02:43,410
1.
996
01:02:43,410 --> 01:02:46,020
Doesn't get much simpler.
997
01:02:46,020 --> 01:02:53,460
So 2 times 1/2 to the p
equals 1 implies p is 1.
998
01:02:53,460 --> 01:02:57,370
All right, now I just plug
into the integral over there.
999
01:02:57,370 --> 01:02:58,476
All right, let's do that.
1000
01:03:12,370 --> 01:03:17,210
So t of x equals theta.
1001
01:03:17,210 --> 01:03:20,620
Well, x to the p is x to
the 1 plus just take off
1002
01:03:20,620 --> 01:03:24,960
the 1, plus x times
the integral 1 to x.
1003
01:03:24,960 --> 01:03:29,220
What's the g function?
1004
01:03:29,220 --> 01:03:31,020
What's g of x in this case?
1005
01:03:34,000 --> 01:03:35,950
x minus 1.
1006
01:03:35,950 --> 01:03:42,730
So I take the integral of
g of u is just u minus 1
1007
01:03:42,730 --> 01:03:53,770
over u to the p
plus 1 p minus 1.
1008
01:03:53,770 --> 01:03:56,310
Did I write the formula right?
1009
01:03:56,310 --> 01:04:00,840
Is it plus 1 or minus 1 here?
1010
01:04:00,840 --> 01:04:01,640
Did I get it?
1011
01:04:01,640 --> 01:04:02,520
p plus 1.
1012
01:04:02,520 --> 01:04:04,970
Sorry, didn't write that right.
1013
01:04:04,970 --> 01:04:07,136
p plus 1 over here.
1014
01:04:10,670 --> 01:04:14,860
All right, so u to the p
plus 1 is u squared du.
1015
01:04:18,410 --> 01:04:27,580
This is theta of x plus x
integral 1 over u minus 1
1016
01:04:27,580 --> 01:04:28,340
over u squared.
1017
01:04:40,650 --> 01:04:42,970
The integral of 1 over
u is the log of u.
1018
01:04:45,500 --> 01:04:50,640
The integral of 1 over
u squared is 1 over u,
1019
01:04:50,640 --> 01:04:53,800
but I changed the sign there.
1020
01:04:53,800 --> 01:04:54,680
1 to x.
1021
01:04:59,860 --> 01:05:08,670
All right, so I get theta of
x plus x times log of x plus 1
1022
01:05:08,670 --> 01:05:12,790
over x minus log of 0.
1023
01:05:12,790 --> 01:05:15,820
That's log of 1 is nothing.
1024
01:05:15,820 --> 01:05:16,758
Minus 1.
1025
01:05:21,160 --> 01:05:27,870
And I'm doing thetas here so I
can forget the low order terms.
1026
01:05:27,870 --> 01:05:32,430
So I get x log x plus 1 minus x.
1027
01:05:32,430 --> 01:05:35,400
This is the only term
that survives, x log x.
1028
01:05:40,040 --> 01:05:42,860
And that's right.
1029
01:05:42,860 --> 01:05:47,956
t of n is in fact tilde n log n,
so in fact, it's theta n log n.
1030
01:05:53,030 --> 01:05:56,150
Now, in this case, going through
the integral and all that,
1031
01:05:56,150 --> 01:05:59,420
maybe that was harder than
just guessing and verifying.
1032
01:05:59,420 --> 01:06:04,060
And by the way, guess and verify
got it really the exact answer.
1033
01:06:04,060 --> 01:06:09,730
The nice thing is this works
for the nasty guys too.
1034
01:06:09,730 --> 01:06:14,390
In fact, let's figure
out the solution
1035
01:06:14,390 --> 01:06:17,950
to this thing, which I
guarantee you is a pain
1036
01:06:17,950 --> 01:06:21,042
to do without this method.
1037
01:06:33,520 --> 01:06:37,050
The first thing to
do is compute p.
1038
01:06:37,050 --> 01:06:42,610
So to do that, we need to
find a p such that 2 times 1/2
1039
01:06:42,610 --> 01:06:55,270
to the p plus 8/9 times
3/4 to the p equals 1.
1040
01:06:55,270 --> 01:06:57,000
I gotta find a p for
which that's true.
1041
01:06:59,700 --> 01:07:01,854
Any guesses what p works?
1042
01:07:06,140 --> 01:07:08,460
p equals 1 not going to do it.
1043
01:07:12,170 --> 01:07:14,330
p equals 2?
1044
01:07:14,330 --> 01:07:17,220
Let's try that.
1045
01:07:17,220 --> 01:07:26,000
I'd have 2 times 1/4
plus 8/9 times 9/16.
1046
01:07:26,000 --> 01:07:27,800
That looks pretty good.
1047
01:07:27,800 --> 01:07:32,310
I get 1/2 plus 1/2 equals 1.
1048
01:07:32,310 --> 01:07:33,430
So p equals 2 works.
1049
01:07:37,500 --> 01:07:39,337
All right, so let's
do the integral.
1050
01:07:42,190 --> 01:07:49,640
tx is theta of x to the p plus
x to the p times the integral.
1051
01:07:49,640 --> 01:07:55,170
All right, what is g
of x in this thing?
1052
01:07:55,170 --> 01:07:56,670
x squared.
1053
01:07:56,670 --> 01:08:00,240
So the integral
is 1 to x. g of u
1054
01:08:00,240 --> 01:08:07,580
is u squared divided
by u to the 1 plus p.
1055
01:08:07,580 --> 01:08:10,276
u to the 1 plus 2
is just u cubed.
1056
01:08:15,860 --> 01:08:20,830
This is theta x squared
plus x squared times, well,
1057
01:08:20,830 --> 01:08:23,140
the integral of 1 over
u is just log of you.
1058
01:08:29,069 --> 01:08:32,529
And that's just log of x.
1059
01:08:32,529 --> 01:08:35,390
So the answer is,
x square ln of x.
1060
01:08:35,390 --> 01:08:38,020
Done.
1061
01:08:38,020 --> 01:08:42,000
So that's pretty good
what these guys did.
1062
01:08:42,000 --> 01:08:47,120
In fact, that is
the correct answer.
1063
01:08:47,120 --> 01:08:50,029
So really easy to pug in.
1064
01:08:50,029 --> 01:08:55,930
Any questions on that?
1065
01:08:55,930 --> 01:08:57,274
Yeah.
1066
01:08:57,274 --> 01:09:03,069
AUDIENCE: [INAUDIBLE] or does
it give the actual number?
1067
01:09:03,069 --> 01:09:05,380
TOM LEIGHTON: It doesn't
give you the actual number.
1068
01:09:05,380 --> 01:09:08,870
It just gives you the
asymptotic growth.
1069
01:09:08,870 --> 01:09:13,609
All you know is that the limit
of tx over x squared ln of x
1070
01:09:13,609 --> 01:09:16,779
is less than infinity
and bigger than 0.
1071
01:09:16,779 --> 01:09:18,760
So it's growing as this.
1072
01:09:18,760 --> 01:09:20,710
You don't know if
it's 10x squared lnx
1073
01:09:20,710 --> 01:09:23,010
or a million squared lnx.
1074
01:09:23,010 --> 01:09:25,604
On the other hand,
for algorithms usually
1075
01:09:25,604 --> 01:09:28,020
you're forgetting the constant
factors anyway because they
1076
01:09:28,020 --> 01:09:29,826
depend on things
to do with the CPU
1077
01:09:29,826 --> 01:09:31,200
you're using and
stuff like that.
1078
01:09:31,200 --> 01:09:33,300
What you really
care about, often,
1079
01:09:33,300 --> 01:09:37,819
when you're studying algorithms
is the asymptotic growth, not
1080
01:09:37,819 --> 01:09:39,460
the actual value.
1081
01:09:39,460 --> 01:09:40,534
Yeah?
1082
01:09:40,534 --> 01:09:43,870
AUDIENCE: [INAUDIBLE]
log base e.
1083
01:09:43,870 --> 01:09:45,620
And then above I
see you using log,
1084
01:09:45,620 --> 01:09:46,825
which I assume is base two?
1085
01:09:46,825 --> 01:09:48,529
Does it matter?
1086
01:09:48,529 --> 01:09:51,560
TOM LEIGHTON: It does up there
because I've got a tilde.
1087
01:09:51,560 --> 01:09:54,310
And in the tilde, the
constant factor matters.
1088
01:09:54,310 --> 01:09:56,810
It doesn't matter
in here because log
1089
01:09:56,810 --> 01:10:00,530
base two and log base E are
within a constant factor,
1090
01:10:00,530 --> 01:10:03,860
namely log of 2, ln of 2
is the constant factor.
1091
01:10:03,860 --> 01:10:06,800
So logs don't
matter what the base
1092
01:10:06,800 --> 01:10:09,910
is once you're inside a theta
or an O or that kind of thing.
1093
01:10:09,910 --> 01:10:13,660
In a tilde, they matter because
the constant factor matters.
1094
01:10:13,660 --> 01:10:14,534
Yeah.
1095
01:10:14,534 --> 01:10:15,450
AUDIENCE: [INAUDIBLE].
1096
01:10:22,685 --> 01:10:23,476
TOM LEIGHTON: Yeah.
1097
01:10:23,476 --> 01:10:25,460
Because x squared
is small compared
1098
01:10:25,460 --> 01:10:29,404
to x squared ln of x because
it's all by the theta.
1099
01:10:29,404 --> 01:10:31,690
AUDIENCE: [INAUDIBLE].
1100
01:10:31,690 --> 01:10:33,040
TOM LEIGHTON: No.
1101
01:10:33,040 --> 01:10:35,840
You could put all sorts
of other stuff in here
1102
01:10:35,840 --> 01:10:38,290
and it doesn't matter
because it still equals
1103
01:10:38,290 --> 01:10:39,910
theta of x squared ln of x.
1104
01:10:39,910 --> 01:10:44,380
There's no hidden meanings by
having extra stuff in here.
1105
01:10:44,380 --> 01:10:45,340
That's important.
1106
01:10:45,340 --> 01:10:48,260
In this case, when you
put more stuff into theta,
1107
01:10:48,260 --> 01:10:51,430
doesn't mean a thing.
1108
01:10:51,430 --> 01:10:52,380
Any other questions?
1109
01:10:52,380 --> 01:10:53,364
Yeah.
1110
01:10:53,364 --> 01:10:56,190
AUDIENCE: [INAUDIBLE].
1111
01:10:56,190 --> 01:10:59,480
TOM LEIGHTON: Ah, good question.
1112
01:10:59,480 --> 01:11:00,915
Well, let's see.
1113
01:11:00,915 --> 01:11:03,553
It goes back to over here.
1114
01:11:13,340 --> 01:11:15,390
See, the b's are
between 0 and 1.
1115
01:11:15,390 --> 01:11:19,000
By making p big enough,
I drive the value down.
1116
01:11:19,000 --> 01:11:22,460
By making p be large and
negative, I drag the value up.
1117
01:11:22,460 --> 01:11:23,710
So there is some value of p.
1118
01:11:23,710 --> 01:11:25,040
But that's a good question.
1119
01:11:25,040 --> 01:11:30,310
Let's see what happens
when p is not so nice.
1120
01:11:30,310 --> 01:11:32,565
Because the examples I gave
you, in one case p was 1
1121
01:11:32,565 --> 01:11:35,220
and one p was 2, sort of nice.
1122
01:11:35,220 --> 01:11:37,175
Let's look at a bad case there.
1123
01:11:43,130 --> 01:11:44,523
Let's look at this recurrence.
1124
01:11:50,790 --> 01:12:01,041
3 t of x over 3 plus 4 t
of x over 4 plus x squared.
1125
01:12:01,041 --> 01:12:02,540
All right, so that's
the recurrence.
1126
01:12:02,540 --> 01:12:04,210
And first step is to compute p.
1127
01:12:04,210 --> 01:12:07,420
And this time it's
not going to be nice.
1128
01:12:07,420 --> 01:12:11,890
So I need to find a p such that
3 times 1/3 of the p plus 4
1129
01:12:11,890 --> 01:12:17,600
times 1/4 to the p equals 1.
1130
01:12:17,600 --> 01:12:20,412
And you might say, OK,
let's try p equals 1.
1131
01:12:24,500 --> 01:12:29,540
I get 1 plus 1 is 2.
1132
01:12:29,540 --> 01:12:31,850
That's too big.
1133
01:12:31,850 --> 01:12:34,010
So which way do I
have to go on p?
1134
01:12:34,010 --> 01:12:37,370
Do I need a bigger
p or a smaller p?
1135
01:12:37,370 --> 01:12:38,230
Bigger.
1136
01:12:38,230 --> 01:12:40,800
Because I got to get
from 2 down to 1.
1137
01:12:40,800 --> 01:12:44,241
So I know that p
is bigger than 1.
1138
01:12:47,190 --> 01:12:49,050
All right, so the
next thing to try
1139
01:12:49,050 --> 01:12:54,440
would be hoping the world
is nice, p equals 2.
1140
01:12:54,440 --> 01:12:55,800
Maybe I'll do that over here.
1141
01:12:55,800 --> 01:12:56,980
Let's try p equals 2.
1142
01:13:00,250 --> 01:13:07,190
So I get three times
1/9 plus 4 times 1/16
1143
01:13:07,190 --> 01:13:12,200
and that equals a 1/3 plus 1/4.
1144
01:13:12,200 --> 01:13:15,590
That's less than 1.
1145
01:13:15,590 --> 01:13:16,300
That didn't work.
1146
01:13:16,300 --> 01:13:17,633
What do I have to do now with p?
1147
01:13:17,633 --> 01:13:20,700
Is it bigger than
2 or less than 2?
1148
01:13:20,700 --> 01:13:21,647
Less than 2.
1149
01:13:24,940 --> 01:13:26,690
That's sort of a pain.
1150
01:13:26,690 --> 01:13:30,270
P is between one and two,
so it's not an integer.
1151
01:13:30,270 --> 01:13:32,400
Now, you can sort of do
divide and conquer here.
1152
01:13:32,400 --> 01:13:36,300
Next I'd try p is 3/2,
get my calculator out
1153
01:13:36,300 --> 01:13:37,870
and see which way it goes.
1154
01:13:37,870 --> 01:13:39,980
And you could sort
of home in on it.
1155
01:13:39,980 --> 01:13:41,360
But there's
something really nice
1156
01:13:41,360 --> 01:13:45,110
that happens with Akra Bazzi.
1157
01:13:45,110 --> 01:13:48,450
And that is if you
ever conclude that p
1158
01:13:48,450 --> 01:13:52,700
is less than your
exponent here, you
1159
01:13:52,700 --> 01:13:55,610
don't even have to compute it.
1160
01:13:55,610 --> 01:13:56,407
Let's see why.
1161
01:13:56,407 --> 01:13:57,240
Let's do an example.
1162
01:14:02,240 --> 01:14:05,870
This will happen a lot.
1163
01:14:05,870 --> 01:14:08,810
There's some p, but it's just
going to be a pain to evaluate.
1164
01:14:08,810 --> 01:14:10,300
But let's see what happens here.
1165
01:14:14,226 --> 01:14:15,600
So I'm just going
to try to solve
1166
01:14:15,600 --> 01:14:17,810
it, write the solution
anyway. t of x
1167
01:14:17,810 --> 01:14:25,420
equals theta x to the p plus
x to the p integral 1 to x.
1168
01:14:25,420 --> 01:14:27,366
What's g of x?
1169
01:14:27,366 --> 01:14:28,400
x squared.
1170
01:14:28,400 --> 01:14:33,354
So that's u squared
over u 1 plus p.
1171
01:14:33,354 --> 01:14:34,770
And I still don't
know what it is,
1172
01:14:34,770 --> 01:14:39,020
but I'm going to
give it a try here.
1173
01:14:39,020 --> 01:14:44,150
That's theta x to the
p plus x to the p.
1174
01:14:44,150 --> 01:14:51,466
Well, this is now the integral
of u to the 1 minus p du.
1175
01:14:51,466 --> 01:14:53,125
And p is bigger than 1.
1176
01:14:53,125 --> 01:14:55,470
I know that.
1177
01:14:55,470 --> 01:15:02,020
This is theta x to
the p plus x to the p.
1178
01:15:02,020 --> 01:15:07,880
Well, the integral here
is x to the 2 minus p.
1179
01:15:07,880 --> 01:15:08,730
Up to constants.
1180
01:15:08,730 --> 01:15:10,830
I don't even care about
dividing by 2 minus p,
1181
01:15:10,830 --> 01:15:13,090
because I got big theta.
1182
01:15:13,090 --> 01:15:15,920
Just put that p up here.
1183
01:15:15,920 --> 01:15:17,250
OK
1184
01:15:17,250 --> 01:15:24,270
Well, this equals theta x to
the p plus cancelled there,
1185
01:15:24,270 --> 01:15:24,790
x squared.
1186
01:15:27,340 --> 01:15:30,280
And what's the
answer going to be?
1187
01:15:30,280 --> 01:15:33,240
x squared, because
p is less than 2.
1188
01:15:33,240 --> 01:15:36,000
So this is a small order term.
1189
01:15:36,000 --> 01:15:37,950
So I didn't even
know what p was, just
1190
01:15:37,950 --> 01:15:40,020
that it was less than two.
1191
01:15:40,020 --> 01:15:41,235
And I got the answer.
1192
01:15:41,235 --> 01:15:42,110
And what do you know?
1193
01:15:42,110 --> 01:15:45,440
The answer was just this thing.
1194
01:15:45,440 --> 01:15:48,070
That's sort of nice and simple.
1195
01:15:48,070 --> 01:15:51,750
And in fact, that's a theorem.
1196
01:15:51,750 --> 01:16:11,900
Which will state, in general, if
g of x equals theta x to the t,
1197
01:16:11,900 --> 01:16:13,502
for some t bigger
and equal to 0.
1198
01:16:16,320 --> 01:16:21,610
And when you plug that
value in and take your sum.
1199
01:16:21,610 --> 01:16:22,860
Trying to compute the p value.
1200
01:16:22,860 --> 01:16:26,320
So I take ai times bi to the t.
1201
01:16:26,320 --> 01:16:28,500
And that comes out
to be less than 1
1202
01:16:28,500 --> 01:16:34,987
like it did here when I plugged
in p equals 2 and I computed it
1203
01:16:34,987 --> 01:16:36,970
and it was less than 1.
1204
01:16:36,970 --> 01:16:42,940
But if that happens in
general, then the answer
1205
01:16:42,940 --> 01:16:50,350
is theta g of x.
1206
01:16:50,350 --> 01:16:51,900
Plain and simple.
1207
01:16:51,900 --> 01:16:58,460
It just turns out to be the g of
x term up to constant factors.
1208
01:16:58,460 --> 01:17:00,430
So you don't even
have to compute it.
1209
01:17:00,430 --> 01:17:03,280
Just do that check on
the power here to see
1210
01:17:03,280 --> 01:17:05,969
is it smaller than that.
1211
01:17:05,969 --> 01:17:07,010
I won't prove that there.
1212
01:17:07,010 --> 01:17:08,310
It's actually not hard to do.
1213
01:17:08,310 --> 01:17:10,980
The proof is pretty much
what I did with this example,
1214
01:17:10,980 --> 01:17:12,082
so it's not too hard.
1215
01:17:15,820 --> 01:17:16,860
OK, any questions?
1216
01:17:19,550 --> 01:17:21,200
I want to show you
one more thing,
1217
01:17:21,200 --> 01:17:24,060
but just make sure that's OK.
1218
01:17:24,060 --> 01:17:30,530
And this is something never to
do with an asymptotic notation,
1219
01:17:30,530 --> 01:17:33,860
like we talked about last time.
1220
01:17:33,860 --> 01:17:36,840
When you're using Akra Bazzi and
when you're doing occurrences
1221
01:17:36,840 --> 01:17:41,630
with algorithm, you always
are using asymptotic notation.
1222
01:17:41,630 --> 01:17:44,490
Your answer is
theta of something.
1223
01:17:44,490 --> 01:17:46,360
And in fact, what will
start happening is
1224
01:17:46,360 --> 01:17:52,470
you'll do this stuff because the
constant factors don't matter.
1225
01:17:52,470 --> 01:17:55,320
Because you're wiping them
out in the theta notation,
1226
01:17:55,320 --> 01:17:56,860
what you'll see is
things like this.
1227
01:17:56,860 --> 01:18:02,300
Your recurrence will be set up
to be t of n is 2 t of n over 2
1228
01:18:02,300 --> 01:18:07,230
plus o of n or plus theta of n.
1229
01:18:07,230 --> 01:18:11,130
And then you'll
conclude that t of n
1230
01:18:11,130 --> 01:18:15,550
is theta of n log
n using Akra Bazzi
1231
01:18:15,550 --> 01:18:18,769
or whatever method you want.
1232
01:18:18,769 --> 01:18:20,310
Because it doesn't
really matter what
1233
01:18:20,310 --> 01:18:26,330
the constant is on g because
the constants disappear.
1234
01:18:26,330 --> 01:18:27,410
Doesn't matter.
1235
01:18:27,410 --> 01:18:31,650
And so people stop
worrying about it early on.
1236
01:18:31,650 --> 01:18:34,830
All right, now here's
the bad example.
1237
01:18:39,394 --> 01:18:44,230
Let me show you a false
proof like last time.
1238
01:18:44,230 --> 01:18:47,342
And this time I think it'll be
pretty easy to spot the flaw.
1239
01:19:01,960 --> 01:19:03,060
Theorem not.
1240
01:19:08,630 --> 01:19:16,050
If t of n equals 2 t of
n over 2 plus n minus 1,
1241
01:19:16,050 --> 01:19:20,590
this is the recurrence we had
for merge sort, and t of 1
1242
01:19:20,590 --> 01:19:29,118
equals 0, then I'm going to
prove that t of n is o event.
1243
01:19:32,360 --> 01:19:38,150
Now, that can't be right
because we've proved it's theta
1244
01:19:38,150 --> 01:19:40,050
then log n, tilde then log n.
1245
01:19:40,050 --> 01:19:42,340
And that is not o of n, right?
1246
01:19:42,340 --> 01:19:46,190
n log n grows faster than n.
1247
01:19:46,190 --> 01:19:47,190
But let's see the proof.
1248
01:19:52,920 --> 01:19:53,911
By strong induction.
1249
01:20:00,720 --> 01:20:04,800
The induction
hypothesis is going
1250
01:20:04,800 --> 01:20:07,240
to be what we're
trying to prove.
1251
01:20:16,770 --> 01:20:19,830
Base case.
1252
01:20:19,830 --> 01:20:21,600
n equals 1.
1253
01:20:21,600 --> 01:20:23,810
t of 1 is 0.
1254
01:20:23,810 --> 01:20:26,050
0 is surely o of 1.
1255
01:20:29,360 --> 01:20:30,873
The induction step.
1256
01:20:36,370 --> 01:20:41,730
We're going to assume p1, p2,
p3 up to pn minus 1 to prove pn.
1257
01:20:55,190 --> 01:20:57,190
And let's look at p of n.
1258
01:20:57,190 --> 01:21:04,820
So I look at tn is 2 t of
n over 2 plus n minus 1.
1259
01:21:04,820 --> 01:21:07,230
Induction hypothesis
on n over 2 says
1260
01:21:07,230 --> 01:21:12,920
this is 2 times o of n
over 2 plus n minus 1.
1261
01:21:12,920 --> 01:21:18,000
Twice o of n over 2 is
o of n plus n is o of n.
1262
01:21:18,000 --> 01:21:19,780
This is o of n.
1263
01:21:19,780 --> 01:21:22,520
And I'm done.
1264
01:21:22,520 --> 01:21:25,580
I just proved that
t of n is o of n.
1265
01:21:28,890 --> 01:21:30,420
What's the bad step?
1266
01:21:30,420 --> 01:21:31,050
What's the bug?
1267
01:21:33,710 --> 01:21:35,150
See the bug?
1268
01:21:35,150 --> 01:21:36,514
What is it?
1269
01:21:36,514 --> 01:21:37,430
AUDIENCE: [INAUDIBLE].
1270
01:21:37,430 --> 01:21:38,890
TOM LEIGHTON:
Induction hypothesis.
1271
01:21:38,890 --> 01:21:40,541
What's wrong with that?
1272
01:21:40,541 --> 01:21:41,457
AUDIENCE: [INAUDIBLE].
1273
01:21:46,017 --> 01:21:46,850
TOM LEIGHTON: Right.
1274
01:21:46,850 --> 01:21:50,260
The end from the predicate
cannot be in the o of n.
1275
01:21:50,260 --> 01:21:54,930
Because remember, o of n means,
this statement here by itself
1276
01:21:54,930 --> 01:22:01,240
means the limit as n goes
to infinity of t of n over n
1277
01:22:01,240 --> 01:22:04,160
is less than infinity.
1278
01:22:04,160 --> 01:22:10,700
But this makes no sense if I
specified n in a predicate.
1279
01:22:10,700 --> 01:22:11,710
So what's the rule?
1280
01:22:11,710 --> 01:22:13,290
Same rule as last time.
1281
01:22:13,290 --> 01:22:14,170
Same bug.
1282
01:22:14,170 --> 01:22:15,075
What's the rule?
1283
01:22:18,360 --> 01:22:21,730
Never do asymptotic notation
in induction hypothesis.
1284
01:22:21,730 --> 01:22:24,430
Never mix big o and a predicate.
1285
01:22:24,430 --> 01:22:25,930
Never, ever.
1286
01:22:25,930 --> 01:22:30,400
It looks so nice to
do, but it is so wrong.
1287
01:22:30,400 --> 01:22:32,910
It just makes nonsense.
1288
01:22:32,910 --> 01:22:34,430
All right, very good.
1289
01:22:34,430 --> 01:22:39,650
So remember ice cream tonight,
recitation optional tomorrow.
1290
01:22:39,650 --> 01:22:42,300
But study session is there.