1
00:00:07 --> 00:00:11
Today we're going to not talk
about sorting.
2
00:00:11 --> 00:00:14
This is an exciting new
development.
3
00:00:14 --> 00:00:18
We're going to talk about
another problem,
4
00:00:18 --> 00:00:23
a related problem,
but a different problem.
5
00:00:23 --> 00:00:35
6
00:00:35 --> 00:00:38
We're going to talk about
another problem that we would
7
00:00:38 --> 00:00:41
like to solve in linear time.
Last class we talked about we
8
00:00:41 --> 00:00:44
could do sorting in linear time.
To do that we needed some
9
00:00:44 --> 00:00:47
additional assumptions.
Today we're going to look at a
10
00:00:47 --> 00:00:51
problem that really only needs
linear time, even though at
11
00:00:51 --> 00:00:54
first glance it might look like
it requires sorting.
12
00:00:54 --> 00:00:56
So this is going to be an
easier problem.
13
00:00:56 --> 00:01:00
The problem is I give you a
bunch of numbers.
14
00:01:00 --> 00:01:06
Let's call them elements.
And they are in some array,
15
00:01:06 --> 00:01:11
let's say.
And they're in no particular
16
00:01:11 --> 00:01:18
order, so unsorted.
I want to find the kth smallest
17
00:01:18 --> 00:01:20
element.
18
00:01:20 --> 00:01:26
19
00:01:26 --> 00:01:30
This is called the element of
rank k.
20
00:01:30 --> 00:01:37
21
00:01:37 --> 00:01:39
In other words,
I have this list of numbers
22
00:01:39 --> 00:01:43
which is unsorted.
And, if I were to sort it,
23
00:01:43 --> 00:01:46
I would like to know what the
kth element is.
24
00:01:46 --> 00:01:50
But I'm not allowed to sort it.
One solution to this problem,
25
00:01:50 --> 00:01:54
this is the naïve algorithm,
is you just sort and then
26
00:01:54 --> 00:01:57
return the kth element.
This is another possible
27
00:01:57 --> 00:02:03
definition of the problem.
And we would like to do better
28
00:02:03 --> 00:02:05
than that.
So you could sort,
29
00:02:05 --> 00:02:10
what's called the array A,
and then return A[k].
30
00:02:10 --> 00:02:16
That is one thing we could do.
And if we use heap sort or
31
00:02:16 --> 00:02:20
mergesort, this will take n lg n
time.
32
00:02:20 --> 00:02:23
We would like to do better than
n lg n.
33
00:02:23 --> 00:02:29
Ideally linear time.
The problem is pretty natural,
34
00:02:29 --> 00:02:34
straightforward.
It has various applications.
35
00:02:34 --> 00:02:39
Depending on how you choose k,
k could be any number between 1
36
00:02:39 --> 00:02:41
and n.
For example,
37
00:02:41 --> 00:02:44
if we choose k=1 that element
has a name.
38
00:02:44 --> 00:02:47
Any suggestions of what the
name is?
39
00:02:47 --> 00:02:50
The minimum.
That's easy.
40
00:02:50 --> 00:02:55
Any suggestions on how we could
find the minimum element in an
41
00:02:55 --> 00:02:59
array in linear time?
Right.
42
00:02:59 --> 00:03:04
Just scan through the array.
Keep track of what the smallest
43
00:03:04 --> 00:03:08
number is that you've seen.
The same thing with the
44
00:03:08 --> 00:03:12
maximum, k=n.
These are rather trivial.
45
00:03:12 --> 00:03:17
But a more interesting version
of the order statistic problem
46
00:03:17 --> 00:03:21
is to find the median.
This is either k equals n plus
47
00:03:21 --> 00:03:26
1 over 2 floor or ceiling.
I will call both of those
48
00:03:26 --> 00:03:29
elements medians.
49
00:03:29 --> 00:03:34
50
00:03:34 --> 00:03:37
Finding the median of an
unsorted array in linear time is
51
00:03:37 --> 00:03:39
quite tricky.
And that sort of is the main
52
00:03:39 --> 00:03:41
goal of this lecture,
is to be able to find the
53
00:03:41 --> 00:03:43
medians.
For free we're going to be able
54
00:03:43 --> 00:03:46
to find the arbitrary kth
smallest element,
55
00:03:46 --> 00:03:48
but typically we're most
interested in finding the
56
00:03:48 --> 00:03:50
median.
And on Friday in recitation
57
00:03:50 --> 00:03:52
you'll see why that is so
useful.
58
00:03:52 --> 00:03:55
There are all sorts of
situations where you can use
59
00:03:55 --> 00:03:58
median for really effective
divide-and-conquer without
60
00:03:58 --> 00:04:02
having to sort.
You can solve a lot of problems
61
00:04:02 --> 00:04:07
in linear time as a result.
And we're going to cover today
62
00:04:07 --> 00:04:10
two algorithms for finding order
statistics.
63
00:04:10 --> 00:04:15
Both of them are linear time.
The first one is randomized,
64
00:04:15 --> 00:04:18
so it's only linear expected
time.
65
00:04:18 --> 00:04:21
And the second one is
worst-case linear time,
66
00:04:21 --> 00:04:25
and it will build on the
randomized version.
67
00:04:25 --> 00:04:31
Let's start with a randomize
divide-and-conquer algorithm.
68
00:04:31 --> 00:04:46
69
00:04:46 --> 00:04:49
This algorithm is called
rand-select.
70
00:04:49 --> 00:05:02
71
00:05:02 --> 00:05:06
And the parameters are a little
bit more than what we're used
72
00:05:06 --> 00:05:08
to.
The order statistics problem
73
00:05:08 --> 00:05:12
you're given an array A.
And here I've changed notation
74
00:05:12 --> 00:05:15
and I'm looking for the ith
smallest element,
75
00:05:15 --> 00:05:18
so i is the index I'm looking
for.
76
00:05:18 --> 00:05:21
And I'm also going to change
the problem a little bit.
77
00:05:21 --> 00:05:25
And instead of trying to find
it in the whole array,
78
00:05:25 --> 00:05:29
I'm going to look in a
particular interval of the
79
00:05:29 --> 00:05:33
array, A from p up to q.
We're going to need that for a
80
00:05:33 --> 00:05:36
recursion.
This better be a recursive
81
00:05:36 --> 00:05:39
algorithm because we're using
divide-and-conquer.
82
00:05:39 --> 00:05:41
Here is the algorithm.
83
00:05:41 --> 00:05:51
84
00:05:51 --> 00:05:54
With a base case.
It's pretty simple.
85
00:05:54 --> 00:06:00
Then we're going to use part of
the quicksort algorithm,
86
00:06:00 --> 00:06:03
randomized quicksort.
87
00:06:03 --> 00:06:09
88
00:06:09 --> 00:06:13
We didn't actually define this
subroutine two lectures ago,
89
00:06:13 --> 00:06:17
but you should know what it
does, especially if you've read
90
00:06:17 --> 00:06:20
the textbook.
This says in the array A[p...q]
91
00:06:20 --> 00:06:24
pick a random element,
so pick a random index between
92
00:06:24 --> 00:06:30
p and q, swap it with the first
element, then call partition.
93
00:06:30 --> 00:06:34
And partition uses that first
element to split the rest of the
94
00:06:34 --> 00:06:39
array into less than or equal to
that random partition and
95
00:06:39 --> 00:06:42
greater than or equal to that
partition.
96
00:06:42 --> 00:06:47
This is just picking a random
partition element between p and
97
00:06:47 --> 00:06:52
q, cutting the array in half,
although the two sizes may not
98
00:06:52 --> 00:06:54
be equal.
And it returns the index of
99
00:06:54 --> 00:07:00
that partition element,
some number between p and q.
100
00:07:00 --> 00:07:08
And we're going to define k to
be this particular value,
101
00:07:08 --> 00:07:15
r minus p plus 1.
And the reason for that is that
102
00:07:15 --> 00:07:21
k is then the rank of the
partition element.
103
00:07:21 --> 00:07:30
This is in A[p...q].
Let me draw a picture here.
104
00:07:30 --> 00:07:34
We have our array A.
It starts at p and ends at q.
105
00:07:34 --> 00:07:38
There is other stuff,
but for this recursive all we
106
00:07:38 --> 00:07:42
care about is p up to q.
We pick a random partition
107
00:07:42 --> 00:07:47
element, say this one,
and we partition things so that
108
00:07:47 --> 00:07:50
everything in here,
let's call this r,
109
00:07:50 --> 00:07:55
is less than or equal to A[r]
and everything up here is
110
00:07:55 --> 00:08:00
greater than or equal to A[r].
And A[r] is our partition
111
00:08:00 --> 00:08:03
element.
After this call,
112
00:08:03 --> 00:08:06
that's what the array looks
like.
113
00:08:06 --> 00:08:09
And we get r.
We get the index of where
114
00:08:09 --> 00:08:14
partition element is stored.
The number of elements that are
115
00:08:14 --> 00:08:20
less than or equal to A[r] and
including r is r minus p plus 1.
116
00:08:20 --> 00:08:23
There will be r minus p
elements here,
117
00:08:23 --> 00:08:28
and we're adding 1 to get this
element.
118
00:08:28 --> 00:08:32
And, if you start counting at
1, if this is rank 1,
119
00:08:32 --> 00:08:35
rank 2, this element will have
rank k.
120
00:08:35 --> 00:08:40
That's just from the
construction in the partition.
121
00:08:40 --> 00:08:46
And now we get to recurse.
And there are three cases --
122
00:08:46 --> 00:08:53
123
00:08:53 --> 00:08:55
-- depending on how i relates
to k.
124
00:08:55 --> 00:08:57
Remember i is the rank that
we're looking for,
125
00:08:57 --> 00:09:01
k is the rank that we happen to
get out of this random
126
00:09:01 --> 00:09:03
partition.
We don't have much control over
127
00:09:03 --> 00:09:07
k, but if we're lucky i=k.
That's the element we want.
128
00:09:07 --> 00:09:13
129
00:09:13 --> 00:09:15
Then we just return the
partition element.
130
00:09:15 --> 00:09:18
More likely is that the element
we're looking for is either to
131
00:09:18 --> 00:09:20
the left or to the right.
And if it's to the left we're
132
00:09:20 --> 00:09:23
going to recurse in the
left-hand portion of the array.
133
00:09:23 --> 00:09:26
And if it's to the right we're
going to recurse in the
134
00:09:26 --> 00:09:28
right-hand portion.
So, pretty straightforward at
135
00:09:28 --> 00:09:30
this point.
136
00:09:30 --> 00:09:45
137
00:09:45 --> 00:09:48
I just have to get all the
indices right.
138
00:09:48 --> 00:10:08
139
00:10:08 --> 00:10:11
Either we're going to recurse
on the part between p and r
140
00:10:11 --> 00:10:14
minus 1, that's this case.
The rank we're looking for is
141
00:10:14 --> 00:10:17
to the left of the rank of
element A[r].
142
00:10:17 --> 00:10:20
Or, we're going to recurse on
the right part between r plus 1
143
00:10:20 --> 00:10:22
and q.
Where we recurse on the left
144
00:10:22 --> 00:10:25
part the rank we're looking for
remains the same,
145
00:10:25 --> 00:10:28
but when we recurse on the
right part the rank we're
146
00:10:28 --> 00:10:33
looking for gets offset.
Because we sort of got rid of
147
00:10:33 --> 00:10:38
the k elements over here.
I should have written this
148
00:10:38 --> 00:10:42
length is k.
We've sort of swept away k
149
00:10:42 --> 00:10:46
ranks of elements.
And now within this array we're
150
00:10:46 --> 00:10:51
looking for the i minus kth
smallest element.
151
00:10:51 --> 00:10:55
That's the recursion.
We only recurse once.
152
00:10:55 --> 00:11:00
And random partition is not a
recursion.
153
00:11:00 --> 00:11:04
That just takes linear time.
And the total amount of work
154
00:11:04 --> 00:11:09
we're doing here should be
linear time plus one recursion.
155
00:11:09 --> 00:11:14
And we'd next like to see what
the total running time is in
156
00:11:14 --> 00:11:19
expectation, but let's first do
a little example --
157
00:11:19 --> 00:11:26
158
00:11:26 --> 00:11:29
-- to make this algorithm
perfectly clear.
159
00:11:29 --> 00:11:33
Let's suppose we're looking for
the seventh smallest element in
160
00:11:33 --> 00:11:35
this array.
161
00:11:35 --> 00:11:50
162
00:11:50 --> 00:11:53
And let's suppose,
just for example,
163
00:11:53 --> 00:11:57
that the pivot we're using is
just the first element.
164
00:11:57 --> 00:12:02
So, nothing fancy.
I would have to flip a few
165
00:12:02 --> 00:12:06
coins in order to generate a
random one, so let's just pick
166
00:12:06 --> 00:12:09
this one.
If I partition at the element
167
00:12:09 --> 00:12:13
6, this is actually an example
we did two weeks ago,
168
00:12:13 --> 00:12:17
and I won't go through it
again, but we get the same
169
00:12:17 --> 00:12:21
array, as we did two weeks ago,
namely 2, 5,
170
00:12:21 --> 00:12:23
3, 6, 8, 13,
10 and 11.
171
00:12:23 --> 00:12:26
If you run through the
partitioning algorithm,
172
00:12:26 --> 00:12:31
that happens to be the order
that it throws the elements
173
00:12:31 --> 00:12:35
into.
And this is our position r.
174
00:12:35 --> 00:12:37
This is p here.
It's just 1.
175
00:12:37 --> 00:12:40
And q is just the end.
And I am looking for the
176
00:12:40 --> 00:12:44
seventh smallest element.
And it happens when I run this
177
00:12:44 --> 00:12:48
partition that 6 falls into the
fourth place.
178
00:12:48 --> 00:12:52
And we know that means,
because all the elements here
179
00:12:52 --> 00:12:56
are less than 6 and all the
elements here are greater than
180
00:12:56 --> 00:13:00
6, if this array were sorted,
6 would be right here in
181
00:13:00 --> 00:13:05
position four.
So, r here is 4.
182
00:13:05 --> 00:13:09
Yeah?
The 12 turned into an 11?
183
00:13:09 --> 00:13:13
This was an 11,
believe it or not.
184
00:13:13 --> 00:13:16
Let me be simple.
Sorry.
185
00:13:16 --> 00:13:20
Sometimes my ones look like
twos.
186
00:13:20 --> 00:13:27
Not a good feature.
That's an easy way to cover.
187
00:13:27 --> 00:13:31
[LAUGHTER]
Don't try that on exams.
188
00:13:31 --> 00:13:33
Oh, that one was just a two.
No.
189
00:13:33 --> 00:13:37
Even though we're not sorting
the array, we're only spending
190
00:13:37 --> 6.
linear work here to partition by
191
6. --> 00:13:39
192
00:13:39 --> 00:13:43
We know that if we had sorted
the array 6 would fall here.
193
00:13:43 --> 00:13:46
We don't know about these other
elements.
194
00:13:46 --> 00:13:49
They're not in sorted order,
but from the properties of
195
00:13:49 --> 00:13:52
partition we know 6 went the
right spot.
196
00:13:52 --> 00:13:56
We now know rank of 6 is 4.
We happened to be looking for 7
197
00:13:56 --> 00:14:00
and we happened to get this
number 4.
198
00:14:00 --> 00:14:03
We want something over here.
It turns out we're looking for
199
00:14:03 --> 00:14:05
10, I guess.
No, 11.
200
00:14:05 --> 00:14:08
There should be eight elements
in this array,
201
00:14:08 --> 00:14:10
so it's the next to max.
Max here is 13,
202
00:14:10 --> 00:14:14
I'm cheating here.
The answer we're looking for is
203
00:14:14 --> 11.
204
11. --> 00:14:16
We know that what we're looking
205
00:14:16 --> 00:14:20
for is in the right-hand part
because the rank we're looking
206
00:14:20 --> 4.
for is 7, which is bigger than
207
4. --> 00:14:22
208
00:14:22 --> 00:14:25
Now, what rank are we looking
for in here?
209
00:14:25 --> 00:14:30
Well, we've gotten rid of four
elements over here.
210
00:14:30 --> 00:14:35
It happened here that k is also
4 because p is 1 in this
211
00:14:35 --> 00:14:38
example.
The rank of 6 was 4.
212
00:14:38 --> 00:14:41
We throw away those four
elements.
213
00:14:41 --> 00:14:46
Now we're looking for rank 7
minus 4 which is 3.
214
00:14:46 --> 00:14:49
And, indeed,
the rank 3 element here is
215
00:14:49 --> 00:14:53
still 11.
So, you recursively find that.
216
00:14:53 --> 00:14:58
That's your answer.
Now that algorithm should be
217
00:14:58 --> 00:15:03
pretty clear.
The tricky part is to analyze
218
00:15:03 --> 00:15:05
it.
And the analysis here is quite
219
00:15:05 --> 00:15:10
a bit like randomized quicksort,
although not quite as hairy,
220
00:15:10 --> 00:15:13
so it will go faster.
But it will be also sort of a
221
00:15:13 --> 00:15:18
nice review of the randomized
quicksort analysis which was a
222
00:15:18 --> 00:15:21
bit tricky and always good to
see a couple of times.
223
00:15:21 --> 00:15:26
We're going to follow the same
kind of outline as before to
224
00:15:26 --> 00:15:31
look at the expected running
time of this algorithm.
225
00:15:31 --> 00:15:34
And to start out we're going
to, as before,
226
00:15:34 --> 00:15:39
look at some intuition just to
feel good about ourselves.
227
00:15:39 --> 00:15:44
Also feel bad as you'll see.
Let's think about two sort of
228
00:15:44 --> 00:15:49
extreme cases,
a good case and the worst case.
229
00:15:49 --> 00:15:54
And I should mention that in
all of the analyses today we
230
00:15:54 --> 00:15:58
assume the elements are
distinct.
231
00:15:58 --> 00:16:04
232
00:16:04 --> 00:16:08
It gets really messy if the
elements are not distinct.
233
00:16:08 --> 00:16:12
And you may even have to change
the algorithms a little bit
234
00:16:12 --> 00:16:16
because if all the elements are
equal, if you pick a random
235
00:16:16 --> 00:16:19
element, the partition does not
do so well.
236
00:16:19 --> 00:16:24
But let's assume they're all
distinct, which is the really
237
00:16:24 --> 00:16:28
interesting case.
A pretty luck case --
238
00:16:28 --> 00:16:32
I mean the best cases we
partition right in the middle.
239
00:16:32 --> 00:16:37
The number of elements to the
left of our partition is equal
240
00:16:37 --> 00:16:42
to the number of elements to the
right of our partition.
241
00:16:42 --> 00:16:47
But almost as good would be
some kind of 1/10 to 9/10 split.
242
00:16:47 --> 00:16:50
Any constant fraction,
we should feel that.
243
00:16:50 --> 00:16:54
Any constant fraction is as
good as 1/2.
244
00:16:54 --> 00:16:58
Then the recurrence we get is,
let's say at most,
245
00:16:58 --> 00:17:01
this bad.
So, it depends.
246
00:17:01 --> 00:17:04
If we have let's say 1/10 on
the left and 9/10 on the right
247
00:17:04 --> 00:17:08
every time we do a partition.
It depends where our answer is.
248
00:17:08 --> 00:17:12
It could be if i is really
small it's in the 1/10 part.
249
00:17:12 --> 00:17:16
If i is really big it's going
to be in the 9/10 part,
250
00:17:16 --> 00:17:19
or most of the time it's going
to be in the 9/10 part.
251
00:17:19 --> 00:17:23
We're doing worst-case analysis
within the lucky case,
252
00:17:23 --> 00:17:25
so we're happy to have upper
bounds.
253
00:17:25 --> 00:17:30
I will say t(n) is at most t of
T(9/10n)+Theta(n).
254
00:17:30 --> 00:17:34
Clearly it's worse if we're in
the bigger part.
255
00:17:34 --> 00:17:38
What is the solution to this
recurrence?
256
00:17:38 --> 00:17:42
Oh, solving recurrence was so
long ago.
257
00:17:42 --> 00:17:47
What method should we use for
solving this recurrence?
258
00:17:47 --> 00:17:51
The master method.
What case are we in?
259
00:17:51 --> 00:17:52
Three.
Good.
260
00:17:52 --> 00:17:55
You still remember.
This is Case 3.
261
00:17:55 --> 00:18:01
We're looking at nlog_b(a).
b here is 10/9,
262
00:18:01 --> 00:18:06
although it doesn't really
matter because a is 1.
263
00:18:06 --> 00:18:11
log base anything of 1 is 0.
So, this is n^0 which is 1.
264
00:18:11 --> 00:18:14
And n is polynomially larger
than 1.
265
00:18:14 --> 00:18:18
This is going to be O(n),
which is good.
266
00:18:18 --> 00:18:21
That is what we want,
linear time.
267
00:18:21 --> 00:18:25
If we're in the lucky case,
great.
268
00:18:25 --> 00:18:30
Unfortunately this is only
intuition.
269
00:18:30 --> 00:18:32
And we're not always going to
get the lucky case.
270
00:18:32 --> 00:18:35
We could do the same kind of
analysis as we did with
271
00:18:35 --> 00:18:38
randomized quicksort.
If you alternate between lucky
272
00:18:38 --> 00:18:41
and unlucky, things will still
be good, but let's just talk
273
00:18:41 --> 00:18:44
about the unlucky case to show
how bad things can get.
274
00:18:44 --> 00:18:48
And this really would be a
worst-case analysis.
275
00:18:48 --> 00:18:53
276
00:18:53 --> 00:19:00
The unlucky case we get a split
of 0:n-1.
277
00:19:00 --> 00:19:04
Because we're removing the
partition element either way.
278
00:19:04 --> 00:19:09
And there could be nothing less
than the partition element.
279
00:19:09 --> 00:19:14
We have 0 on the left-hand side
and we have n-1 on the
280
00:19:14 --> 00:19:18
right-hand side.
Now we get a recurrence like
281
00:19:18 --> 00:19:23
T(n)=T(n-1) plus linear cost.
And what's the solution to that
282
00:19:23 --> 00:19:25
recurrence?
n^2.
283
00:19:25 --> 00:19:27
Yes.
This one you should just know.
284
00:19:27 --> 00:19:33
It's n^2 because it's an
arithmetic series.
285
00:19:33 --> 00:19:38
286
00:19:38 --> 00:19:40
And that's pretty bad.
This is much,
287
00:19:40 --> 00:19:43
much worse than sorting and
then picking the ith element.
288
00:19:43 --> 00:19:46
In the worst-case this
algorithm really sucks,
289
00:19:46 --> 00:19:49
but most of the time it's going
to do really well.
290
00:19:49 --> 00:19:52
And, unless you're really,
really unlucky and every coin
291
00:19:52 --> 00:19:56
you flip gives the wrong answer,
you won't get this case and you
292
00:19:56 --> 00:19:58
will get something more like the
lucky case.
293
00:19:58 --> 00:20:02
At least that's what we'd like
to prove.
294
00:20:02 --> 00:20:05
And we will prove that the
expected running time here is
295
00:20:05 --> 00:20:07
linear.
So, it's very rare to get
296
00:20:07 --> 00:20:09
anything quadratic.
But later on we will see how to
297
00:20:09 --> 00:20:11
make the worst-case linear as
well.
298
00:20:11 --> 00:20:15
This would really,
really solve the problem.
299
00:20:15 --> 00:20:30
300
00:20:30 --> 00:20:34
Let's get into the analysis.
301
00:20:34 --> 00:20:43
302
00:20:43 --> 00:20:47
Now, you've seen an analysis
much like this before.
303
00:20:47 --> 00:20:51
What do you suggest we do in
order to analyze this expected
304
00:20:51 --> 00:20:54
time?
It's a divide-and-conquer
305
00:20:54 --> 00:20:59
algorithm, so we kind of like to
write down the recurrence on
306
00:20:59 --> 00:21:03
something resembling the running
time.
307
00:21:03 --> 00:21:09
308
00:21:09 --> 00:21:12
I don't need the answer,
but what's the first step that
309
00:21:12 --> 00:21:16
we might do to analyze the
expected running time of this
310
00:21:16 --> 00:21:18
algorithm?
Sorry?
311
00:21:18 --> 00:21:20
Look at different cases,
yeah.
312
00:21:20 --> 00:21:22
Exactly.
We have all these possible ways
313
00:21:22 --> 00:21:25
that random partition could
split.
314
00:21:25 --> 00:21:30
It could split 0 to the n-1.
It could split in half.
315
00:21:30 --> 00:21:33
There are n choices where it
could split.
316
00:21:33 --> 00:21:35
How can we break into those
cases?
317
00:21:35 --> 00:21:38
Indicator random variables.
Cool.
318
00:21:38 --> 00:21:41
Exactly.
That's what we want to do.
319
00:21:41 --> 00:21:46
Indicator random variable
suggests that what we're dealing
320
00:21:46 --> 00:21:50
with is not exactly just a
function T(n) but it's a random
321
00:21:50 --> 00:21:53
variable.
This is one subtlety.
322
00:21:53 --> 00:21:57
T(n) depends on the random
choices, so it's really a random
323
00:21:57 --> 00:22:00
variable.
324
00:22:00 --> 00:22:05
325
00:22:05 --> 00:22:08
And then we're going to use
indicator random variables to
326
00:22:08 --> 00:22:10
get a recurrence on T(n).
327
00:22:10 --> 00:22:25
328
00:22:25 --> 00:22:32
So, T(n) is the running time of
rand-select on an input of size
329
00:22:32 --> 00:22:33
n.
330
00:22:33 --> 00:22:40
331
00:22:40 --> 00:22:46
And I am also going to write
down explicitly an assumption
332
00:22:46 --> 00:22:49
about the random numbers.
333
00:22:49 --> 00:22:55
334
00:22:55 --> 00:23:00
That they should be chosen
independently from each other.
335
00:23:00 --> 00:23:03
Every time I call random
partition, it's generating a
336
00:23:03 --> 00:23:07
completely independent random
number from all the other times
337
00:23:07 --> 00:23:10
I call random partition.
That is important,
338
00:23:10 --> 00:23:12
of course, for this analysis to
work.
339
00:23:12 --> 00:23:15
We will see why some point down
the line.
340
00:23:15 --> 00:23:19
And now, to sort of write down
an equation for T(n) we're going
341
00:23:19 --> 00:23:24
to define indicator random
variables, as you suggested.
342
00:23:24 --> 00:23:36
343
00:23:36 --> 00:23:44
And we will call it X_k.
And this is for all k=0...n-1.
344
00:23:44 --> 00:23:50
345
00:23:50 --> 00:23:54
Indicator random variables
either 1 or 0.
346
00:23:54 --> 00:24:00
And it's going to be 1 if the
partition comes out k on the
347
00:24:00 --> 00:24:06
left-hand side.
So say the partition generates
348
00:24:06 --> 00:24:11
a k:n-k-1 split and it is 0
otherwise.
349
00:24:11 --> 00:24:17
We have n of these indicator
random variables between
350
00:24:17 --> 00:24:20
0...n-1.
And in each case,
351
00:24:20 --> 00:24:27
no matter how the random choice
comes out, exactly one of them
352
00:24:27 --> 00:24:32
will be 1.
All the others will be 0.
353
00:24:32 --> 00:24:37
Now we can divide out the
running time of this algorithm
354
00:24:37 --> 00:24:40
based on which case we're in.
355
00:24:40 --> 00:24:49
356
00:24:49 --> 00:24:57
That will sort of unify this
intuition that we did and get
357
00:24:57 --> 00:25:02
all the cases.
And then we can look at the
358
00:25:02 --> 00:25:08
expectation.
T(n), if we just split out by
359
00:25:08 --> 00:25:15
cases, we have an upper bound
like this.
360
00:25:15 --> 00:25:28
361
00:25:28 --> 00:25:33
If we have 0 to n-1 split,
the worst is we have n-1.
362
00:25:33 --> 00:25:38
Then we have to recurse in a
problem of size n-1.
363
00:25:38 --> 00:25:43
In fact, it would be pretty
hard to recurse in a problem of
364
00:25:43 --> 00:25:47
size 0.
If we have a 1 to n-2 split
365
00:25:47 --> 00:25:51
then we take the max of the two
sides.
366
00:25:51 --> 00:25:58
That's certainly going to give
us an upper bound and so on.
367
00:25:58 --> 00:26:03
368
00:26:03 --> 00:26:06
And at the bottom you get an
n-1 to 0 split.
369
00:26:06 --> 00:26:14
370
00:26:14 --> 00:26:16
This is now sort of
conditioning on various events,
371
00:26:16 --> 00:26:19
but we have indicator random
variables to tell us when these
372
00:26:19 --> 00:26:21
events happen.
We can just multiply each of
373
00:26:21 --> 00:26:25
these values by the indicator
random variable and it will come
374
00:26:25 --> 00:26:28
out 0 if that's not the case and
will come out 1 and give us this
375
00:26:28 --> 00:26:31
value if that happens to be the
split.
376
00:26:31 --> 00:26:37
So, if we add up all of those
we'll get the same thing.
377
00:26:37 --> 00:26:45
This is equal to the sum over
all k of the indicator random
378
00:26:45 --> 00:26:52
variable times the cost in that
case, which is t of max k,
379
00:26:52 --> 00:26:57
and the other side,
which is n-k-1,
380
00:26:57 --> 00:27:01
plus theta n.
This is our recurrence,
381
00:27:01 --> 00:27:04
in some sense,
for the random variable
382
00:27:04 --> 00:27:09
representing running time.
Now, the value will depend on
383
00:27:09 --> 00:27:13
which case we come into.
We know the probability of each
384
00:27:13 --> 00:27:19
of these events happening is the
same because we're choosing the
385
00:27:19 --> 00:27:23
partition element uniformly at
random, but we cannot really
386
00:27:23 --> 00:27:29
simplify much beyond this until
we take expectations.
387
00:27:29 --> 00:27:32
We know this random variable
could be as big as n^2.
388
00:27:32 --> 00:27:37
Hopefully it's usually linear.
We will take expectations of
389
00:27:37 --> 00:27:40
both sides and get what we want.
390
00:27:40 --> 00:27:54
391
00:27:54 --> 00:27:58
Let's look at the expectation
of this random variable,
392
00:27:58 --> 00:28:02
which is just the expectation,
I will copy over,
393
00:28:02 --> 00:28:07
summation we have here so I can
work on this board.
394
00:28:07 --> 00:28:30
395
00:28:30 --> 00:28:33
I want to compute the
expectation of this summation.
396
00:28:33 --> 00:28:36
What property of expectation
should I use?
397
00:28:36 --> 00:28:39
Linearity, good.
We can bring the summation
398
00:28:39 --> 00:28:41
outside.
399
00:28:41 --> 00:29:08
400
00:29:08 --> 00:29:09
Now I have a sum of
expectation.
401
00:29:09 --> 00:29:12
Let's look at each expectation
individually.
402
00:29:12 --> 00:29:15
It's a product of two random
variables, if you will.
403
00:29:15 --> 00:29:19
This is an indicator random
variable and this is some more
404
00:29:19 --> 00:29:22
complicated function,
some more complicated random
405
00:29:22 --> 00:29:24
variable representing some
running time,
406
00:29:24 --> 00:29:28
which depends on what random
choices are made in that
407
00:29:28 --> 00:29:31
recursive call.
Now what should I do?
408
00:29:31 --> 00:29:37
I have the expectation of the
product of two random variables.
409
00:29:37 --> 00:29:39
Independence,
exactly.
410
00:29:39 --> 00:29:45
If I know that these two random
variables are independent then I
411
00:29:45 --> 00:29:51
know that the expectation of the
product is the product of the
412
00:29:51 --> 00:29:55
expectations.
Now we have to check are they
413
00:29:55 --> 00:29:58
independent?
I hope so because otherwise
414
00:29:58 --> 00:30:04
there isn't much else I can do.
Why are they independent?
415
00:30:04 --> 00:30:07
Sorry?
Because we stated that they
416
00:30:07 --> 00:30:10
are, right.
Because of this assumption.
417
00:30:10 --> 00:30:14
We assume that all the random
numbers are chosen
418
00:30:14 --> 00:30:17
independently.
We need to sort of interpolate
419
00:30:17 --> 00:30:19
that here.
These X_k's,
420
00:30:19 --> 00:30:21
all the X_k's,
X_0 up to X_n-1,
421
00:30:21 --> 00:30:26
so all the ones appearing in
this summation are dependent
422
00:30:26 --> 00:30:30
upon a single random choice of
this particular call to random
423
00:30:30 --> 00:30:36
partition.
All of these are correlated,
424
00:30:36 --> 00:30:44
because if one of them is 1,
all the others are forced to be
425
00:30:44 --> 0.
426
0. --> 00:30:47
So, there is a lot of
427
00:30:47 --> 00:30:54
correlation among the X_k's.
But with respect to everything
428
00:30:54 --> 00:31:00
that is in here,
and the only random part is
429
00:31:00 --> 00:31:07
this T(max(kn-k-1)).
That is the reason that this
430
00:31:07 --> 00:31:12
random variable is independent
from these.
431
00:31:12 --> 00:31:19
The same thing as quicksort,
but I know some people got
432
00:31:19 --> 00:31:24
confused about it a couple
lectures ago so I am
433
00:31:24 --> 00:31:29
reiterating.
We get the product of
434
00:31:29 --> 00:31:35
expectations,
E[X_k] E[T(max(kn-k-1))].
435
00:31:35 --> 00:31:40
I mean the order n comes
outside, but let's leave it
436
00:31:40 --> 00:31:44
inside for now.
There is no expectation to
437
00:31:44 --> 00:31:49
compute there for order n.
Order n is order n.
438
00:31:49 --> 00:31:55
What is the expectation of X_k?
1/n, because they're all chosen
439
00:31:55 --> 00:32:00
with equal probability.
There is n of them,
440
00:32:00 --> 00:32:04
so the expectation is 1/n.
The value is either 1 or 0.
441
00:32:04 --> 00:32:07
We start to be able to split
this up.
442
00:32:07 --> 00:32:12
We have 1/n times this expected
value of some recursive T call,
443
00:32:12 --> 00:32:15
and then we have plus 1 over n
times order n,
444
00:32:15 --> 00:32:20
also known as a constant,
but everything is summed up n
445
00:32:20 --> 00:32:23
times so let's expand this.
446
00:32:23 --> 00:32:35
447
00:32:35 --> 00:32:42
I have the sum k=0 to n-1.
I guess the 1/n can come
448
00:32:42 --> 00:32:47
outside.
And we have expectation of
449
00:32:47 --> 00:32:54
[T(max(kn-k-1))].
Lots of nifty braces there.
450
00:32:54 --> 00:32:59
And then plus we have,
on the other hand,
451
00:32:59 --> 00:33:06
the sum k=0 to n-1.
Let me just write that out
452
00:33:06 --> 00:33:08
again.
We have a 1/n in front and we
453
00:33:08 --> 00:33:12
have a Theta(n) inside.
This summation is n^2.
454
00:33:12 --> 00:33:16
And then we're dividing by n,
so this whole thing is,
455
00:33:16 --> 00:33:20
again, order n.
Nothing fancy happened there.
456
00:33:20 --> 00:33:25
This is really just saying the
expectation of order n is order
457
00:33:25 --> 00:33:27
n.
Average value of order n is
458
00:33:27 --> 00:33:31
order n.
What is interesting is this
459
00:33:31 --> 00:33:35
part.
Now, what could we do with this
460
00:33:35 --> 00:33:38
summation?
Here we start to differ from
461
00:33:38 --> 00:33:43
randomized quicksort because we
have this max.
462
00:33:43 --> 00:33:48
Randomized quicksort we had the
sum of T(k) plus T(n-k-1)
463
00:33:48 --> 00:33:52
because we were making both
recursive calls.
464
00:33:52 --> 00:33:56
Here we're only making the
biggest one.
465
00:33:56 --> 00:34:03
That max is really a pain for
evaluating this recurrence.
466
00:34:03 --> 00:34:11
How could I get rid of the max?
That's one way to think of it.
467
00:34:11 --> 00:34:13
Yeah?
468
00:34:13 --> 00:34:18
469
00:34:18 --> 00:34:20
Exactly.
I could only sum up to halfway
470
00:34:20 --> 00:34:23
and then double.
In other words,
471
00:34:23 --> 00:34:26
terms are getting repeated
twice here.
472
00:34:26 --> 00:34:30
When k=0 or when k=n-1,
I get the same T(n-1).
473
00:34:30 --> 00:34:33
When k=1 or n-2,
I get the same thing,
474
00:34:33 --> 00:34:37
2 and n-3.
What I will actually do is sum
475
00:34:37 --> 00:34:42
from halfway up.
That's a little bit cleaner.
476
00:34:42 --> 00:34:45
And let me get the indices
right.
477
00:34:45 --> 00:34:49
Floor of n/2 up to n-1 will be
safe.
478
00:34:49 --> 00:34:55
And then I just have E[T(k)],
except I forgot to multiply by
479
00:34:55 --> 00:35:01
2, so I'm going to change this 1
to a 2.
480
00:35:01 --> 00:35:04
And order n is preserved.
This is just because each term
481
00:35:04 --> 00:35:07
is appearing twice.
I can factor it out.
482
00:35:07 --> 00:35:10
And if n is odd,
I'm actually double-counting
483
00:35:10 --> 00:35:13
somewhat, but it's certain at
most that.
484
00:35:13 --> 00:35:17
So, that's a safe upper bound.
And upper bounds are all we
485
00:35:17 --> 00:35:20
care about because we're hoping
to get linear.
486
00:35:20 --> 00:35:24
And the running time of this
algorithm is definitely at least
487
00:35:24 --> 00:35:29
linear, so we just need an upper
bounded linear.
488
00:35:29 --> 00:35:32
So, this is a recurrence.
E[T(n)] is at most 2/n times
489
00:35:32 --> 00:35:36
the sum of half the numbers
between 0 and n of
490
00:35:36 --> 00:35:39
E[T(k)]+Theta(n).
It's a bit of hairy recurrence.
491
00:35:39 --> 00:35:41
We want to solve it,
though.
492
00:35:41 --> 00:35:45
And it's actually a little bit
easier than the randomized
493
00:35:45 --> 00:35:48
quicksort recurrence.
We're going to solve it.
494
00:35:48 --> 00:35:51
What method should we use?
Sorry?
495
00:35:51 --> 00:35:53
Master method?
Master would be nice,
496
00:35:53 --> 00:35:57
except that each of the
recursive calls is with a
497
00:35:57 --> 00:36:01
different value of k.
The master method only works
498
00:36:01 --> 00:36:05
when all the calls are with the
same value, same size.
499
00:36:05 --> 00:36:09
Alas, it would be nice if we
could use the master method.
500
00:36:09 --> 00:36:11
What else do we have?
Substitution.
501
00:36:11 --> 00:36:13
When it's hard,
when in doubt,
502
00:36:13 --> 00:36:16
use substitution.
I mean the good thing here is
503
00:36:16 --> 00:36:20
we know what we want.
From the intuition at least,
504
00:36:20 --> 00:36:23
which is now erased,
we really feel that this should
505
00:36:23 --> 00:36:26
be linear time.
So, we know what we want to
506
00:36:26 --> 00:36:31
prove.
And indeed we can prove it just
507
00:36:31 --> 00:36:35
directly with substitution.
508
00:36:35 --> 00:36:42
509
00:36:42 --> 00:36:46
I want to claim there is some
constant c greater than zero
510
00:36:46 --> 00:36:49
such that E[T(n)],
according to this recurrence,
511
00:36:49 --> 00:36:54
is at most c times n.
Let's prove that over here.
512
00:36:54 --> 00:37:00
513
00:37:00 --> 00:37:04
As we guessed,
the proof is by substitution.
514
00:37:04 --> 00:37:13
515
00:37:13 --> 00:37:18
What that means is we're going
to assume, by induction,
516
00:37:18 --> 00:37:22
that this inequality is true
for all smaller m.
517
00:37:22 --> 00:37:28
I will just say 4 less than n.
And we need to prove it for n.
518
00:37:28 --> 00:37:33
We get E[T(n)].
Now we are just going to expand
519
00:37:33 --> 00:37:36
using the recurrence that we
have.
520
00:37:36 --> 00:37:40
It's at most this.
I will copy that over.
521
00:37:40 --> 00:37:54
522
00:37:54 --> 00:37:57
And then each of these
recursive calls is with some
523
00:37:57 --> 00:38:00
value k that is strictly smaller
than n.
524
00:38:00 --> 00:38:03
Sorry, I copied it wrong,
floor of n over 2,
525
00:38:03 --> 00:38:07
not zero.
And so I can apply the
526
00:38:07 --> 00:38:11
induction hypothesis to each of
these.
527
00:38:11 --> 00:38:16
This is at most c times k by
the induction hypothesis.
528
00:38:16 --> 00:38:20
And so I get this inequality.
529
00:38:20 --> 00:38:37
530
00:38:37 --> 00:38:40
This c can come outside the
summation because it's just a
531
00:38:40 --> 00:38:43
constant.
And I will be slightly tedious
532
00:38:43 --> 00:38:47
in writing this down again,
because what I care about is
533
00:38:47 --> 00:38:50
the summation here that is left
over.
534
00:38:50 --> 00:38:56
535
00:38:56 --> 00:39:01
This is a good old-fashioned
summation.
536
00:39:01 --> 00:39:04
And if you remember back to
your summation tricks or
537
00:39:04 --> 00:39:07
whatever, you should be able to
evaluate this.
538
00:39:07 --> 00:39:11
If we started at zero and went
up to n minus 1,
539
00:39:11 --> 00:39:14
that's just an arithmetic
series, but here we have the
540
00:39:14 --> 00:39:16
tail end of an arithmetic
series.
541
00:39:16 --> 00:39:19
And you should know,
at least up to theta,
542
00:39:19 --> 00:39:21
what this is,
right?
543
00:39:21 --> 00:39:23
n^2, yeah.
It's definitely T(n^2).
544
00:39:23 --> 00:39:26
But we need here a slightly
better upper bond,
545
00:39:26 --> 00:39:31
as we will see the constants
really matter.
546
00:39:31 --> 00:39:35
What we're going to use is that
this summation is at most 3/8
547
00:39:35 --> 00:39:38
times n^2.
And that will be critical,
548
00:39:38 --> 00:39:41
the fact that 3/8 is smaller
than 1/2, I believe.
549
00:39:41 --> 00:39:44
So it's going to get rid of
this 2.
550
00:39:44 --> 00:39:47
I am not going to prove this.
This is an exercise.
551
00:39:47 --> 00:39:52
When you know that it is true,
it's easy because you can just
552
00:39:52 --> 00:39:55
prove it by induction.
Figuring out that number is a
553
00:39:55 --> 00:40:00
little bit more work,
but not too much more.
554
00:40:00 --> 00:40:04
So you should prove that by
induction.
555
00:40:04 --> 00:40:09
Now let me simplify.
This is a bit messy,
556
00:40:09 --> 00:40:15
but what I want is c times n.
Let's write it as our desired
557
00:40:15 --> 00:40:22
value minus the residual.
And here we have some crazy
558
00:40:22 --> 00:40:26
fractions.
This is 2 times 3 which is 6
559
00:40:26 --> 00:40:31
over 8 which is 3/4,
right?
560
00:40:31 --> 00:40:34
Here we have 1,
so we have to subtract up 1/4
561
00:40:34 --> 00:40:37
to get 3/4.
And this should be,
562
00:40:37 --> 00:40:42
I guess, 1/4 times c times n.
And then we have this theta n
563
00:40:42 --> 00:40:45
with double negation becomes a
plus theta n.
564
00:40:45 --> 00:40:49
That should be clear.
I am just rewriting that.
565
00:40:49 --> 00:40:52
So we have what we want over
here.
566
00:40:52 --> 00:40:57
And then we hope that this is
nonnegative because what we want
567
00:40:57 --> 00:41:03
is that this less than or equal
to c times n.
568
00:41:03 --> 00:41:06
That will be true,
provided this thing is
569
00:41:06 --> 00:41:09
nonnegative.
And it looks pretty good
570
00:41:09 --> 00:41:13
because we're free to choose c
however large we want.
571
00:41:13 --> 00:41:17
Whatever constant is imbedded
in this beta notation is one
572
00:41:17 --> 00:41:21
fixed constant,
whatever makes this recurrence
573
00:41:21 --> 00:41:24
true.
We just set c to be bigger than
574
00:41:24 --> 00:41:28
4 times that constant and then
this will be nonnegative.
575
00:41:28 --> 00:41:32
So this is true for c
sufficiently large to dwarf that
576
00:41:32 --> 00:41:36
theta constant.
It's also the base case.
577
00:41:36 --> 00:41:41
I just have to make the cursory
mention that we choose c large
578
00:41:41 --> 00:41:45
enough so that this claim is
true, even in the base case
579
00:41:45 --> 00:41:48
where n is at most some
constant.
580
00:41:48 --> 00:41:52
Here it's like 1 or so because
then we're not making a
581
00:41:52 --> 00:41:55
recursive call.
What we get --
582
00:41:55 --> 00:41:59
This algorithm,
randomize select,
583
00:41:59 --> 00:42:05
has expected running time order
n, Theta(n).
584
00:42:05 --> 00:42:12
585
00:42:12 --> 00:42:15
The annoying this is that in
the worst-case,
586
00:42:15 --> 00:42:19
if you're really,
really unlucky it's n^2.
587
00:42:19 --> 00:42:23
Any questions before we move on
from this point?
588
00:42:23 --> 00:42:29
This finished off the proof of
this fact that we have Theta(n)
589
00:42:29 --> 00:42:32
expected time.
We already saw the n^2
590
00:42:32 --> 00:42:34
worst-case.
All perfectly clear?
591
00:42:34 --> 00:42:37
Good.
You should go over these
592
00:42:37 --> 00:42:39
proofs.
They're intrinsically related
593
00:42:39 --> 00:42:43
between randomized quicksort and
randomized select.
594
00:42:43 --> 00:42:47
Know them in your heart.
This is a great algorithm that
595
00:42:47 --> 00:42:52
works really well in practice
because most of the time you're
596
00:42:52 --> 00:42:54
going to split,
say, in the middle,
597
00:42:54 --> 00:43:00
somewhere between a 1/4 and 3/4
and everything is good.
598
00:43:00 --> 00:43:03
It's extremely unlikely that
you get the n^2 worst-case.
599
00:43:03 --> 00:43:06
It would have to happen with
like 1 over n^n probability or
600
00:43:06 --> 00:43:08
something really,
really small.
601
00:43:08 --> 00:43:10
But I am a theoretician at
least.
602
00:43:10 --> 00:43:14
And it would be really nice if
you could get Theta(n) in the
603
00:43:14 --> 00:43:16
worst-case.
That would be the cleanest
604
00:43:16 --> 00:43:19
result that you could hope for
because that's optimal.
605
00:43:19 --> 00:43:21
You cannot do better than
Theta(n).
606
00:43:21 --> 00:43:23
You've got to look at the
elements.
607
00:43:23 --> 00:43:25
So, you might ask,
can we get rid of this
608
00:43:25 --> 00:43:29
worst-case behavior and somehow
avoid randomization and
609
00:43:29 --> 00:43:33
guarantee Theta(n) worst-case
running time?
610
00:43:33 --> 00:43:39
And you can but it's a rather
nontrivial algorithm.
611
00:43:39 --> 00:43:45
And this is going to be one of
the most sophisticated that
612
00:43:45 --> 00:43:51
we've seen so far.
It won't continue to be the
613
00:43:51 --> 00:43:58
most sophisticated algorithm we
will see, but here it is.
614
00:43:58 --> 00:44:04
Worst-case linear time order
statistics.
615
00:44:04 --> 00:44:09
616
00:44:09 --> 00:44:22
And this is an algorithm by
several, all very famous people,
617
00:44:22 --> 00:44:32
Blum, Floyd,
Pratt, Rivest and Tarjan.
618
00:44:32 --> 00:44:35
I think I've only met the B and
the R and the T.
619
00:44:35 --> 00:44:39
Oh, no, I've met Pratt as well.
I'm getting close to all the
620
00:44:39 --> 00:44:42
authors.
This is a somewhat old result,
621
00:44:42 --> 00:44:46
but at the time it was a major
breakthrough and still is an
622
00:44:46 --> 00:44:50
amazing algorithm.
Ron Rivest is a professor here.
623
00:44:50 --> 00:44:52
You should know him from the R
in RSA.
624
00:44:52 --> 00:44:56
When I took my PhD
comprehensives some time ago,
625
00:44:56 --> 00:45:00
on the cover sheet was a joke
question.
626
00:45:00 --> 00:45:04
It asked of the authors of the
worst-case linear time order
627
00:45:04 --> 00:45:08
statistics algorithm,
which of them is the most rich?
628
00:45:08 --> 00:45:13
Sadly it was not a graded part
of the comprehensive exam,
629
00:45:13 --> 00:45:18
but it was an amusing question.
I won't answer it here because
630
00:45:18 --> 00:45:21
we're on tape,
[LAUGHTER] but think about it.
631
00:45:21 --> 00:45:25
I may not be obvious.
Several of them are rich.
632
00:45:25 --> 00:45:30
It's just the question of who
is the most rich.
633
00:45:30 --> 00:45:33
Anyway, before they were rich
they came up with this
634
00:45:33 --> 00:45:35
algorithm.
They've come up with many
635
00:45:35 --> 00:45:38
algorithms since,
even after getting rich,
636
00:45:38 --> 00:45:42
believe it or not.
What we want is a good pivot,
637
00:45:42 --> 00:45:45
guaranteed good pivot.
Random pivot is going to be
638
00:45:45 --> 00:45:48
really good.
And so the simplest algorithm
639
00:45:48 --> 00:45:52
is just pick a random pivot.
It's going to be good with high
640
00:45:52 --> 00:45:55
probability.
We want to force a good pivot
641
00:45:55 --> 00:45:58
deterministically.
And the new idea here is we're
642
00:45:58 --> 00:46:02
going to generate it
recursively.
643
00:46:02 --> 00:46:04
What else could we do but
recurse?
644
00:46:04 --> 00:46:08
Well, you should know from your
recurrences that if we did two
645
00:46:08 --> 00:46:12
recursive calls on problems of
half the size and we have a
646
00:46:12 --> 00:46:16
linear extra work that's the
mergesort recurrence,
647
00:46:16 --> 00:46:20
T(n)=2[T(n/2)+Theta(n)].
You should recite in your
648
00:46:20 --> 00:46:21
sleep.
That's n lg n.
649
00:46:21 --> 00:46:25
So we cannot recurse on two
problems of half the size.
650
00:46:25 --> 00:46:30
We've got to do better.
Somehow these recursions have
651
00:46:30 --> 00:46:32
to add up to strictly less than
n.
652
00:46:32 --> 00:46:35
That's the magic of this
algorithm.
653
00:46:35 --> 00:46:39
So this will just be called
select instead of rand-select.
654
00:46:39 --> 00:46:44
And it really depends on an
array, but I will focus on the
655
00:46:44 --> 00:46:48
i-th element that we want to
select and the size of the array
656
00:46:48 --> 00:46:53
that we want to select in.
And I am going to write this
657
00:46:53 --> 00:46:57
algorithm slightly less formally
than randomize-select because
658
00:46:57 --> 00:47:02
it's a bit higher level of an
algorithm.
659
00:47:02 --> 00:47:22
660
00:47:22 --> 00:47:31
And let me draw over here the
picture of the algorithm.
661
00:47:31 --> 00:47:36
The first step is sort of the
weirdest and it's one of the key
662
00:47:36 --> 00:47:38
ideas.
You take your elements,
663
00:47:38 --> 00:47:43
and they are in no particular
order, so instead of drawing
664
00:47:43 --> 00:47:47
them on a line,
I am going to draw them in a 5
665
00:47:47 --> 00:47:49
by n over 5 grid.
Why not?
666
00:47:49 --> 00:47:54
This, unfortunately,
take a little while to draw,
667
00:47:54 --> 00:48:00
but it will take you equally
long so I will take my time.
668
00:48:00 --> 00:48:02
It doesn't really matter what
the width is,
669
00:48:02 --> 00:48:06
but it should be width n over 5
so make sure you draw your
670
00:48:06 --> 00:48:08
figure accordingly.
Width n over 5,
671
00:48:08 --> 00:48:10
but the height should be
exactly 5.
672
00:48:10 --> 00:48:13
I think I got it right.
I can count that high.
673
00:48:13 --> 00:48:15
Here is 5.
And this should be,
674
00:48:15 --> 00:48:17
well, you know,
our number may not be divisible
675
00:48:17 --> 00:48:20
by 5, so maybe it ends off in
sort of an odd way.
676
00:48:20 --> 00:48:24
But what I would like is that
these chunks should be floor of
677
00:48:24 --> 00:48:26
n over 5.
And then we will have,
678
00:48:26 --> 00:48:30
at most, four elements left
over.
679
00:48:30 --> 00:48:33
So I am going to ignore those.
They don't really matter.
680
00:48:33 --> 00:48:36
It's just an additive constant.
Here is my array.
681
00:48:36 --> 00:48:39
I just happened to write it in
this funny way.
682
00:48:39 --> 00:48:42
And I will call these vertical
things groups.
683
00:48:42 --> 00:48:45
I would circle them,
and I did that in my notes,
684
00:48:45 --> 00:48:49
but things get really messy if
you start circling.
685
00:48:49 --> 00:48:53
This diagram is going to get
really full, just to warn you.
686
00:48:53 --> 00:48:55
By the end it will be almost
unintelligible,
687
00:48:55 --> 00:49:00
but there it is.
If you are really feeling
688
00:49:00 --> 00:49:03
bored, you can draw this a few
times.
689
00:49:03 --> 00:49:06
And you should draw how it
grows.
690
00:49:06 --> 00:49:10
So there are the groups,
vertical groups of five.
691
00:49:10 --> 00:49:12
Next step.
692
00:49:12 --> 00:49:18
693
00:49:18 --> 00:49:24
The second step is to recurse.
This is where things are a bit
694
00:49:24 --> 00:49:28
unusual, well,
even more unusual.
695
00:49:28 --> 00:49:32
Oops, sorry.
I really should have had a line
696
00:49:32 --> 00:49:37
between one and two so I am
going to have to move this down
697
00:49:37 --> 00:49:40
and insert it here.
I also, in step one,
698
00:49:40 --> 00:49:44
want to find the median of each
group.
699
00:49:44 --> 00:49:53
700
00:49:53 --> 00:49:56
What I would like to do is just
imagine this figure,
701
00:49:56 --> 00:49:59
each of the five elements in
each group gets reorganized so
702
00:49:59 --> 00:50:02
that the middle one is the
median.
703
00:50:02 --> 00:50:05
So I am going to call these the
medians of each group.
704
00:50:05 --> 00:50:10
I have five elements so the
median is right in the middle.
705
00:50:10 --> 00:50:13
There are two elements less
than the median,
706
00:50:13 --> 00:50:15
two elements greater than the
median.
707
00:50:15 --> 00:50:19
Again, we're assuming all
elements are distinct.
708
00:50:19 --> 00:50:21
So there they are.
I compute them.
709
00:50:21 --> 00:50:24
How long does that take me?
N over five groups,
710
00:50:24 --> 00:50:30
each with five elements,
compute the median of each one?
711
00:50:30 --> 00:50:32
Sorry?
Yeah, 2 times n over 5.
712
00:50:32 --> 00:50:34
It's theta n,
that's all I need to know.
713
00:50:34 --> 00:50:38
I mean, you're counting
comparisons, which is good.
714
00:50:38 --> 00:50:42
It's definitely Theta(n).
The point is within each group,
715
00:50:42 --> 00:50:46
I only have to do a constant
number of comparisons because
716
00:50:46 --> 00:50:48
it's a constant number of
elements.
717
00:50:48 --> 00:50:51
It doesn't matter.
You could use randomize select
718
00:50:51 --> 00:50:54
for all I care.
No matter what you do,
719
00:50:54 --> 00:50:59
it can only take a constant
number of comparisons.
720
00:50:59 --> 00:51:03
As long as you don't make a
comparison more than once.
721
00:51:03 --> 00:51:07
So this is easy.
You could sort the five numbers
722
00:51:07 --> 00:51:12
and then look at the third one,
it doesn't matter because there
723
00:51:12 --> 00:51:16
are only five of them.
That's one nifty idea.
724
00:51:16 --> 00:51:21
Already we have some elements
that are sort of vaguely in the
725
00:51:21 --> 00:51:25
middle but just of the group.
And we've only done linear
726
00:51:25 --> 00:51:29
work.
So doing well so far.
727
00:51:29 --> 00:51:33
Now we get to the second step,
which I started to write
728
00:51:33 --> 00:51:36
before, where we recurse.
729
00:51:36 --> 00:51:58
730
00:51:58 --> 00:52:01
So the next idea is,
well, we have these floor over
731
00:52:01 --> 00:52:04
n over 5 medians.
I am going to compute the
732
00:52:04 --> 00:52:07
median of those medians.
I am imagining that I
733
00:52:07 --> 00:52:09
rearranged these.
And, unfortunately,
734
00:52:09 --> 00:52:11
it's an even number,
there are six of them,
735
00:52:11 --> 00:52:15
but I will rearrange so that
this guy, I have drawn in a
736
00:52:15 --> 00:52:18
second box, is the median of
these elements so that these two
737
00:52:18 --> 00:52:22
elements are strictly less than
this guy, these three elements
738
00:52:22 --> 00:52:24
are strictly greater than this
guy.
739
00:52:24 --> 00:52:27
Now, that doesn't directly tell
me anything, it would seem,
740
00:52:27 --> 00:52:31
about any of the elements out
here.
741
00:52:31 --> 00:52:35
We will come back to that.
In fact, it does tell us about
742
00:52:35 --> 00:52:38
some of the elements.
But right now this element is
743
00:52:38 --> 00:52:42
just the median of these guys.
Each of these guys is a median
744
00:52:42 --> 00:52:45
of five elements.
That's all we know.
745
00:52:45 --> 00:52:49
If we do that recursively,
this is going to take T of n
746
00:52:49 --> 00:52:51
over 5 time.
So far so good.
747
00:52:51 --> 00:52:55
We can afford a recursion on a
problem of size n over 5 and
748
00:52:55 --> 00:52:58
linear work.
We know that works out to
749
00:52:58 --> 00:53:00
linear time.
But there is more.
750
00:53:00 --> 00:53:02
We're obviously not done yet.
751
00:53:02 --> 00:53:10
752
00:53:10 --> 00:53:12
The next step is x is our
partition element.
753
00:53:12 --> 00:53:15
We partition there.
The rest of the algorithm is
754
00:53:15 --> 00:53:19
just like randomized partition,
so we're going to define k to
755
00:53:19 --> 00:53:21
be the rank of x.
And this can be done,
756
00:53:21 --> 00:53:25
I mean it's n minus r plus 1 or
whatever, but I'm not going to
757
00:53:25 --> 00:53:30
write out how to do that because
we're at a higher level here.
758
00:53:30 --> 00:53:34
But it can be done.
And then we have the three-way
759
00:53:34 --> 00:53:37
branching.
So if i happens to equal k
760
00:53:37 --> 00:53:41
we're happy.
The pivot element is the
761
00:53:41 --> 00:53:46
element we're looking for,
but more likely i is either
762
00:53:46 --> 00:53:49
less than k or it is bigger than
k.
763
00:53:49 --> 00:53:53
And then we make the
appropriate recursive call,
764
00:53:53 --> 00:54:00
so here we recursively select
the i-th smallest element --
765
00:54:00 --> 00:54:08
766
00:54:08 --> 00:54:11
-- in the lower part of the
array.
767
00:54:11 --> 00:54:16
Left of the partition element.
Otherwise, we recursively
768
00:54:16 --> 00:54:22
select the i minus k-th smallest
element in the upper part of the
769
00:54:22 --> 00:54:25
array.
I am writing this at a high
770
00:54:25 --> 00:54:30
level because we've already seen
it.
771
00:54:30 --> 00:54:36
All of this is the same as the
last couple steps of randomized
772
00:54:36 --> 00:54:37
select.
773
00:54:37 --> 00:54:45
774
00:54:45 --> 00:54:48
That is the algorithm.
The real question is why does
775
00:54:48 --> 00:54:50
it work?
Why is this linear time?
776
00:54:50 --> 00:54:53
The first question is what's
the recurrence?
777
00:54:53 --> 00:54:56
We cannot quite write it down
yet because we don't know how
778
00:54:56 --> 00:55:00
big these recursive subproblems
could be.
779
00:55:00 --> 00:55:03
We're going to either recurse
in the lower part or the upper
780
00:55:03 --> 00:55:07
part, that's just like before.
If we're unlucky and we have a
781
00:55:07 --> 00:55:11
split of like zero to n minus
one, this is going to be a
782
00:55:11 --> 00:55:14
quadratic time algorithm.
The claim is that this
783
00:55:14 --> 00:55:18
partition element is guaranteed
to be pretty good and good
784
00:55:18 --> 00:55:21
enough.
The running time of this thing
785
00:55:21 --> 00:55:24
will be T of something times n,
and we don't know what the
786
00:55:24 --> 00:55:27
something is yet.
How big could it be?
787
00:55:27 --> 00:55:32
Well, I could ask you.
But we're sort of indirect here
788
00:55:32 --> 00:55:34
so I will tell you.
We have already a recursive
789
00:55:34 --> 00:55:38
call of T of n over 5.
It better be that whatever
790
00:55:38 --> 00:55:41
constant, so it's going to be
something times n,
791
00:55:41 --> 00:55:44
it better be that that constant
is strictly less than 4/5.
792
00:55:44 --> 00:55:48
If it's equal to 4/5 then
you're not splitting up the
793
00:55:48 --> 00:55:51
problem enough to get an n lg n
running time.
794
00:55:51 --> 00:55:55
If it's strictly less than 4/5
then you're reducing the problem
795
00:55:55 --> 00:55:59
by at least a constant factor.
In the sense if you add up all
796
00:55:59 --> 00:56:03
the recursive subproblems,
n over 5 and something times n,
797
00:56:03 --> 00:56:07
you get something that is a
constant strictly less than one
798
00:56:07 --> 00:56:09
times n.
That forces the work to be
799
00:56:09 --> 00:56:12
geometric.
If it's geometric you're going
800
00:56:12 --> 00:56:15
to get linear time.
So this is intuition but it's
801
00:56:15 --> 00:56:18
the right intuition.
Whenever you're aiming for
802
00:56:18 --> 00:56:21
linear time keep that in mind.
If you're doing a
803
00:56:21 --> 00:56:24
divide-and-conquer,
you've got to get the total
804
00:56:24 --> 00:56:27
subproblem size to be some
constant less than one times n.
805
00:56:27 --> 00:56:32
That will work.
OK, so we've got to work out
806
00:56:32 --> 00:56:37
this constant here.
And we're going to use this
807
00:56:37 --> 00:56:42
figure, which so far looks
surprisingly uncluttered.
808
00:56:42 --> 00:56:48
Now we will make it cluttered.
What I would like to do is draw
809
00:56:48 --> 00:56:53
an arrow between two vertices,
two points, elements,
810
00:56:53 --> 00:57:00
whatever you want to call them.
Let's call them a and b.
811
00:57:00 --> 00:57:04
And I want to orient the arrow
so it points to a larger value,
812
00:57:04 --> 00:57:06
so this means that a is less
than b.
813
00:57:06 --> 00:57:09
This is notation just for the
diagram.
814
00:57:09 --> 00:57:13
And so this element,
I am going to write down what I
815
00:57:13 --> 00:57:15
know.
This element is the median of
816
00:57:15 --> 00:57:19
these five elements.
I will suppose that it is drawn
817
00:57:19 --> 00:57:22
so that these elements are
larger than the median,
818
00:57:22 --> 00:57:25
these elements are smaller than
the median.
819
00:57:25 --> 00:57:28
Therefore, I have arrows like
this.
820
00:57:28 --> 00:57:33
Here is where I wish I had some
colored chalk.
821
00:57:33 --> 00:57:36
This is just stating this guy
is in the middle of those five
822
00:57:36 --> 00:57:39
elements.
I know that in every single
823
00:57:39 --> 00:57:40
column.
824
00:57:40 --> 00:57:55
825
00:57:55 --> 00:57:58
Here is where the diagram
starts to get messy.
826
00:57:58 --> 00:58:01
I am not done yet.
Now, we also know that this
827
00:58:01 --> 00:58:03
element is the median of the
medians.
828
00:58:03 --> 00:58:06
Of all the squared elements,
this guy is the middle.
829
00:58:06 --> 00:58:10
And I will draw it so that
these are the ones smaller than
830
00:58:10 --> 00:58:13
the median, these are the ones
larger than the median.
831
00:58:13 --> 00:58:15
I mean the algorithm cannot do
this.
832
00:58:15 --> 00:58:18
It doesn't necessarily know how
all this works.
833
00:58:18 --> 00:58:20
I guess it could,
but this is just for analysis
834
00:58:20 --> 00:58:23
purposes.
We know this guy is bigger than
835
00:58:23 --> 00:58:25
that one and bigger than that
one.
836
00:58:25 --> 00:58:29
We don't directly know about
the other elements.
837
00:58:29 --> 00:58:33
We just know that that one is
bigger than both of those and
838
00:58:33 --> 00:58:37
this guy is smaller than these.
Now, that is as messy as the
839
00:58:37 --> 00:58:40
figure will get.
Now, the nice thing about less
840
00:58:40 --> 00:58:43
than is that it's a transitive
relation.
841
00:58:43 --> 00:58:47
If I have a directed path in
this graph, I know that this
842
00:58:47 --> 00:58:51
element is strictly less than
that element because this is
843
00:58:51 --> 00:58:54
less than that one and this is
less than that one.
844
00:58:54 --> 00:58:59
Even though directly I only
know within a column and within
845
00:58:59 --> 00:59:02
this middle row,
I actually know that this
846
00:59:02 --> 00:59:05
element --
This is x, by the way.
847
00:59:05 --> 00:59:10
This element is larger than all
of these elements because it's
848
00:59:10 --> 00:59:15
larger than this one and this
one and each of these is larger
849
00:59:15 --> 00:59:17
than all of those by these
arrows.
850
00:59:17 --> 00:59:22
I also know that all of these
elements in this rectangle here,
851
00:59:22 --> 00:59:27
and you don't have to do this
but I will make the background
852
00:59:27 --> 00:59:32
even more cluttered.
All of these elements in this
853
00:59:32 --> 00:59:37
rectangle are greater than or
equal to this one and all of the
854
00:59:37 --> 00:59:42
elements in this rectangle are
less than or equal to x.
855
00:59:42 --> 00:59:47
Now, how many are there?
Well, this is roughly halfway
856
00:59:47 --> 00:59:52
along the set of groups and this
is 3/5 of these columns.
857
00:59:52 --> 00:59:57
So what we get is that there
are at least --
858
00:59:57 --> 1:00:03.554
We have n over 5 groups and we
have half of the groups that
859
1:00:03.554 --> 1:00:10.222
we're looking at here roughly,
so let's call that floor of n
860
1:00:10.222 --> 1:00:16.664
over 2, and then within each
group we have three elements.
861
1:00:16.664 --> 1:00:23.219
So we have at least 3 times
floor of floor of n over 5 over
862
1:00:23.219 --> 1:00:30
2 n floor elements that are less
than or equal to x.
863
1:00:30 --> 1:00:36.222
And we have the same that are
greater than or equal to x.
864
1:00:36.222 --> 1:00:40.444
Let me simplify this a little
bit more.
865
1:00:40.444 --> 1:00:45.222
I can also give you some more
justification,
866
1:00:45.222 --> 1:00:51.222
and we drew the picture,
but just for why this is true.
867
1:00:51.222 --> 1:00:57.777
We have at least n over 5 over
2 group medians that are less
868
1:00:57.777 --> 1:01:02.622
than or equal to x.
This is the argument we use.
869
1:01:02.622 --> 1:01:05.809
We have half of the group
medians are less than or equal
870
1:01:05.809 --> 1:01:08.59
to x because x is the median of
the group median,
871
1:01:08.59 --> 1:01:11.892
so that is no big surprise.
This is almost an equality but
872
1:01:11.892 --> 1:01:14.905
we're making floors so it's
greater than or equal to.
873
1:01:14.905 --> 1:01:18.034
And then, for each group
median, we know that there are
874
1:01:18.034 --> 1:01:21.568
three elements there that are
less than or equal to that group
875
1:01:21.568 --> 1:01:23.133
median.
So, by transitivity,
876
1:01:23.133 --> 1:01:25.218
they're also less than or equal
to x.
877
1:01:25.218 --> 1:01:30.664
We get this number times three.
This is actually just floor of
878
1:01:30.664 --> 1:01:33.773
n over 10.
I was being unnecessarily
879
1:01:33.773 --> 1:01:38.126
complicated there,
but that is where it came from.
880
1:01:38.126 --> 1:01:43.544
What we know is that this thing
is now at least 3 times n over
881
1:01:43.544 --> 1:01:48.252
10, which is roughly 3/10 of
elements are in one side.
882
1:01:48.252 --> 1:01:53.137
In fact, at least 3/10 of the
elements are in each side.
883
1:01:53.137 --> 1:01:59
Therefore, each side has at
most 7/10 elements roughly.
884
1:01:59 --> 1:02:01.214
So the number here will be
7/10.
885
1:02:01.214 --> 1:02:04.642
And, if I'm lucky,
7/10 plus 1/5 is strictly less
886
1:02:04.642 --> 1:02:06.428
than one.
I believe it is,
887
1:02:06.428 --> 1:02:09.142
but I have trouble working with
tenths.
888
1:02:09.142 --> 1:02:11.357
I can only handle powers of
two.
889
1:02:11.357 --> 1:02:14.857
What we're going to use is a
minor simplification,
890
1:02:14.857 --> 1:02:19.214
which just barely still works,
is a little bit easier to think
891
1:02:19.214 --> 1:02:21.785
about.
It's mainly to get rid of this
892
1:02:21.785 --> 1:02:24.285
floor because the floor is
annoying.
893
1:02:24.285 --> 1:02:28.214
And we don't really have a
sloppiness lemma that applies
894
1:02:28.214 --> 1:02:31.463
here.
It turns out if n is
895
1:02:31.463 --> 1:02:34.975
sufficiently large,
3 times floor of n over 10 is
896
1:02:34.975 --> 1:02:38.707
greater than or equal to 1/4.
Quarters I can handle.
897
1:02:38.707 --> 1:02:42.365
The claim is that each group
has size at least 1/4,
898
1:02:42.365 --> 1:02:46.609
therefore each group has size
at most 3/4 because there's a
899
1:02:46.609 --> 1:02:49.317
quarter on the side.
This will be 3/4.
900
1:02:49.317 --> 1:02:53.048
And I can definitely tell that
1/5 is less than 1/4.
901
1:02:53.048 --> 1:02:57.292
This is going to add up to
something strictly less than one
902
1:02:57.292 --> 1:03:01.292
and then it will work.
How is my time?
903
1:03:01.292 --> 1:03:02.929
Good.
At this point,
904
1:03:02.929 --> 1:03:05.686
the rest of the analysis is
easy.
905
1:03:05.686 --> 1:03:09.993
How the heck you would come up
with this algorithm,
906
1:03:09.993 --> 1:03:14.818
you realize that this is
clearly a really good choice for
907
1:03:14.818 --> 1:03:19.643
finding a partition element,
just barely good enough that
908
1:03:19.643 --> 1:03:22.83
both recursions add up to linear
time.
909
1:03:22.83 --> 1:03:28
Well, that's why it took so
many famous people.
910
1:03:28 --> 1:03:30.78
Especially in quizzes,
but I think in general this
911
1:03:30.78 --> 1:03:34.241
class, you won't have to come up
with an algorithm this clever
912
1:03:34.241 --> 1:03:37.531
because you can just use this
algorithm to find the median.
913
1:03:37.531 --> 1:03:40.312
And the median is a really good
partition element.
914
1:03:40.312 --> 1:03:43.375
Now that you know this
algorithm, now that we're beyond
915
1:03:43.375 --> 1:03:45.815
1973, you don't need to know how
to do this.
916
1:03:45.815 --> 1:03:48.482
I mean you should know how this
algorithm works,
917
1:03:48.482 --> 1:03:51.943
but you don't need to do this
in another algorithm because you
918
1:03:51.943 --> 1:03:55.234
can just say run this algorithm,
you will get the median in
919
1:03:55.234 --> 1:03:58.524
linear time, and then you can
partition to the left and the
920
1:03:58.524 --> 1:04:02.225
right.
And then the left and the right
921
1:04:02.225 --> 1:04:04.737
will have exactly equal size.
Great.
922
1:04:04.737 --> 1:04:07.321
This is a really powerful
subroutine.
923
1:04:07.321 --> 1:04:11.7
You could use this all over the
place, and you will on Friday.
924
1:04:11.7 --> 1:04:14.858
Have I analyzed the running
time pretty much?
925
1:04:14.858 --> 1:04:18.806
The first step is linear.
The second step is T of n over
926
1:04:18.806 --> 5.
927
5. --> 1:04:20.027
The third step,
928
1:04:20.027 --> 1:04:22.037
I didn't write it,
is linear.
929
1:04:22.037 --> 1:04:25.41
And then the last step is just
a recursive call.
930
1:04:25.41 --> 1:04:29
And now we know that this is
3/4.
931
1:04:29 --> 1:04:34
932
1:04:34 --> 1:04:40
I get this recurrence.
T of n is, I'll say at most,
933
1:04:40 --> 1:04:47.079
T of n over 5 plus T of 3/4n.
You could have also used 7/10.
934
1:04:47.079 --> 1:04:54.4
It would give the same answer,
but you would also need a floor
935
1:04:54.4 --> 1:05:01
so we won't do that.
I claim that this is linear.
936
1:05:01 --> 1:05:07
How should I prove it?
Substitution.
937
1:05:07 --> 1:05:12
938
1:05:12 --> 1:05:15.901
Claim that T of n is at most
again c times n,
939
1:05:15.901 --> 1:05:19.891
that will be enough.
Proof is by substitution.
940
1:05:19.891 --> 1:05:23.704
Again, we assume this is true
for smaller n.
941
1:05:23.704 --> 1:05:28.758
And want to prove it for n.
We have T of n is at most this
942
1:05:28.758 --> 1:05:31.489
thing.
T of n over 5.
943
1:05:31.489 --> 1:05:36.489
And by induction,
because n of 5 is smaller than
944
1:05:36.489 --> 1:05:40
n, we know that this is at most
c.
945
1:05:40 --> 1:05:43.723
Let me write it as c over 5
times n.
946
1:05:43.723 --> 1:05:47.765
Sure, why not.
Then we have here 3/4cn.
947
1:05:47.765 --> 1:05:53.085
And then we have a linear term.
Now, unfortunately,
948
1:05:53.085 --> 1:06:00
I have to deal with things that
are not powers of two.
949
1:06:00 --> 1:06:02.447
I will cheat and look at my
notes.
950
1:06:02.447 --> 1:06:06.599
This is also known as 19/20
times c times n plus theta n.
951
1:06:06.599 --> 1:06:10.826
And the point is just that this
is strictly less than one.
952
1:06:10.826 --> 1:06:15.202
Because it's strictly less than
one, I can write this as one
953
1:06:15.202 --> 1:06:19.206
times c of n minus some
constant, here it happens to be
954
1:06:19.206 --> 1:06:22.766
1/20, as long as I have
something left over here,
955
1:06:22.766 --> 1:06:26.622
1/20 times c times n.
Then I have this annoying theta
956
1:06:26.622 --> 1:06:30.923
n term which I want to get rid
of because I want this to be
957
1:06:30.923 --> 1:06:34.783
nonnegative.
But it is nonnegative,
958
1:06:34.783 --> 1:06:38.432
as long as I set c to be
really, really large,
959
1:06:38.432 --> 1:06:41.918
at least 20 times whatever
constant is here.
960
1:06:41.918 --> 1:06:46.216
So this is at most c times n
for c sufficiently large.
961
1:06:46.216 --> 1:06:50.189
And, oh, by the way,
if n is less than or equal to
962
1:06:50.189 --> 1:06:54.405
50, which we used up here,
then T of n is a constant,
963
1:06:54.405 --> 1:06:59.27
it doesn't really matter what
you do, and T of n is at most c
964
1:06:59.27 --> 1:07:03
times n for c sufficiently
large.
965
1:07:03 --> 1:07:06.017
That proves this claim.
Of course, the constant here is
966
1:07:06.017 --> 1:07:08.421
pretty damn big.
It depends exactly what the
967
1:07:08.421 --> 1:07:11.606
constants and the running times
are, which depends on your
968
1:07:11.606 --> 1:07:14.96
machine, but practically this
algorithm is not so hot because
969
1:07:14.96 --> 1:07:18.089
the constants are pretty big.
Even though this element is
970
1:07:18.089 --> 1:07:20.772
guaranteed to be somewhere
vaguely in the middle,
971
1:07:20.772 --> 1:07:23.566
and even though these
recursions add up to strictly
972
1:07:23.566 --> 1:07:26.752
less than n and it's geometric,
it's geometric because the
973
1:07:26.752 --> 1:07:31
problem is reducing by at least
a factor of 19/20 each time.
974
1:07:31 --> 1:07:34.742
So it actually takes a while
for the problem to get really
975
1:07:34.742 --> 1:07:37.106
small.
Practically you probably don't
976
1:07:37.106 --> 1:07:40.782
want to use this algorithm
unless you cannot somehow flip
977
1:07:40.782 --> 1:07:43.146
coins.
The randomized algorithm works
978
1:07:43.146 --> 1:07:46.166
really, really fast.
Theoretically this is your
979
1:07:46.166 --> 1:07:50.237
dream, the best you could hope
for because it's linear time and
980
1:07:50.237 --> 1:07:53.257
you need linear time as
guaranteed linear time.
981
1:07:53.257 --> 1:07:55.161
I will mention,
before we end,
982
1:07:55.161 --> 1:07:57
an exercise.
983
1:07:57 --> 1:08:03
984
1:08:03 --> 1:08:06.375
Why did we use groups of five?
Why not groups of three?
985
1:08:06.375 --> 1:08:09.062
As you might guess,
the answer is because it
986
1:08:09.062 --> 1:08:11.125
doesn't work with groups of
three.
987
1:08:11.125 --> 1:08:13.812
But it's quite constructive to
find out why.
988
1:08:13.812 --> 1:08:17.562
If you work through this math
with groups of three instead of
989
1:08:17.562 --> 1:08:20.25
groups of five,
you will find that you don't
990
1:08:20.25 --> 1:08:23.062
quite get the problem reduction
that you need.
991
1:08:23.062 --> 1:08:27
Five is the smallest number for
which this works.
992
1:08:27 --> 1:08:30.177
It would work with seven,
but theoretically not any
993
1:08:30.177 --> 1:08:32.973
better than a constant factor.
Any questions?
994
1:08:32.973 --> 1:08:35.07
All right.
Then recitation Friday.
995
1:08:35.07 --> 1:08:37.802
Homework lab Sunday.
Problem set due Monday.
996
1:08:37.802 --> 1:08:40
Quiz one in two weeks.