1
00:00:08 --> 00:00:11
OK.
Today we are going to talk
2
00:00:11 --> 00:00:18
about a very interesting
algorithm called Quicksort --
3
00:00:18 --> 00:00:23
4
00:00:23 --> 00:00:30
-- which was invented by Tony
Hoare in 1962.
5
00:00:30 --> 00:00:35
And it has ended up being a
really interesting algorithm
6
00:00:35 --> 00:00:40
from many points of view.
And because of that,
7
00:00:40 --> 00:00:47
it turns out today's lecture is
going to be both hard and fast.
8
00:00:47 --> 00:00:52
If you see the person next to
you sleeping,
9
00:00:52 --> 00:00:56
you will want to say let's get
going.
10
00:00:56 --> 00:01:01
It's a divide-and-conquer
algorithm.
11
00:01:01 --> 00:01:06
12
00:01:06 --> 00:01:08
And it sorts,
as they say,
13
00:01:08 --> 00:01:14
in place, meaning that it just
rearranged the elements where
14
00:01:14 --> 00:01:17
they are.
That is like insertion sort
15
00:01:17 --> 00:01:20
rearranges elements where they
are.
16
00:01:20 --> 00:01:24
Mergesort does not.
Mergesort requires extra
17
00:01:24 --> 00:01:30
storage in order to do the merge
operation.
18
00:01:30 --> 00:01:34
To merge in linear time and
place, it doesn't merge in place
19
00:01:34 --> 00:01:37
in linear time.
It doesn't do it just by
20
00:01:37 --> 00:01:41
rearranging.
It is nice because it is in
21
00:01:41 --> 00:01:45
place, so that means that it is
fairly efficient in its use of
22
00:01:45 --> 00:01:48
storage.
And it also happens to be very
23
00:01:48 --> 00:01:53
practical if you tune it a bit.
The basic algorithm turns out,
24
00:01:53 --> 00:01:58
if you just implement that,
it's not necessarily that
25
00:01:58 --> 00:02:02
efficient.
But if what you do was then do
26
00:02:02 --> 00:02:07
the standard kinds of things you
do to goose up the runtime of
27
00:02:07 --> 00:02:12
something, and we'll talk a
little about what those things
28
00:02:12 --> 00:02:16
are, then it can be very,
very practical.
29
00:02:16 --> 00:02:20
So, it uses divide-and-conquer
paradigm.
30
00:02:20 --> 00:02:33
31
00:02:33 --> 00:02:40
First step is divide.
And to do this basically it
32
00:02:40 --> 00:02:48
does it by partitioning.
So, it partitions the input
33
00:02:48 --> 00:02:59
array into two subarrays around
an element we call the pivot --
34
00:02:59 --> 00:03:04
35
00:03:04 --> 00:03:14
-- such that elements in the
lower subarray are less than or
36
00:03:14 --> 00:03:25
equal to x, are less than or
equal to elements in the upper
37
00:03:25 --> 00:03:31
subarray.
If we draw a picture of the
38
00:03:31 --> 00:03:38
input array, this partition step
basically takes some element x
39
00:03:38 --> 00:03:45
and everything over here is less
than or equal to x after the
40
00:03:45 --> 00:03:52
partition step and everything
over here is greater than or
41
00:03:52 --> 00:03:57
equal to x.
And so now the conquer step is
42
00:03:57 --> 00:04:04
pretty easy.
You just recursively sort the
43
00:04:04 --> 00:04:10
two subarrays.
So, I recursively sort the
44
00:04:10 --> 00:04:19
elements less than or equal to
x, I recursively sort the
45
00:04:19 --> 00:04:24
elements greater than or equal
to x.
46
00:04:24 --> 00:04:32
And then combine is then just
trivial.
47
00:04:32 --> 00:04:36
Because once I have sorted the
things less than or equal to x,
48
00:04:36 --> 00:04:40
then sorted the things greater
than or equal to x,
49
00:04:40 --> 00:04:44
the whole thing is sorted.
So, there is nothing to do
50
00:04:44 --> 00:04:48
really for the combine.
The key step in quicksort is
51
00:04:48 --> 00:04:52
this partition step.
That is the thing that does all
52
00:04:52 --> 00:04:55
of the work.
And so you can view quicksort
53
00:04:55 --> 00:04:57
of just as recursive
partitioning.
54
00:04:57 --> 00:05:06
That's all it is.
Just as mergesort was recursive
55
00:05:06 --> 00:05:18
merging, quicksort sort of goes
the other way around and does
56
00:05:18 --> 00:05:28
recursive partitioning.
The key is the linear time,
57
00:05:28 --> 00:05:40
by which I mean theta n,
partitioning subroutine.
58
00:05:40 --> 00:05:43
And here are some pseudocode
for it.
59
00:05:43 --> 00:05:47
This is actually slightly
different from the book.
60
00:05:47 --> 00:05:51
The book has one.
In fact, there is a nice
61
00:05:51 --> 00:05:55
problem in the book that has
even a different one,
62
00:05:55 --> 00:06:00
but they are all basically the
same idea.
63
00:06:00 --> 00:06:01
Partition (A,
p, q).
64
00:06:01 --> 00:06:06
And what we are looking at,
at this step of the recursion,
65
00:06:06 --> 00:06:11
is the subarray A from p to q.
And basically we pick a pivot,
66
00:06:11 --> 00:06:17
which is we are going to just
pick as the first element of the
67
00:06:17 --> 00:06:19
array A of p.
68
00:06:19 --> 00:06:26
69
00:06:26 --> 00:06:29
And the book,
just for your information,
70
00:06:29 --> 00:06:32
uses A of q.
I use A of p.
71
00:06:32 --> 00:06:37
It doesn't really matter.
And then we set an index to p
72
00:06:37 --> 00:06:40
and then we have a loop.
73
00:06:40 --> 00:07:35
74
00:07:35 --> 00:07:40
This is the code.
Basically the structure of it
75
00:07:40 --> 00:07:47
is a for loop with an "if"
statement in the middle.
76
00:07:47 --> 00:07:54
And so the structure of the
algorithm of this partitioning
77
00:07:54 --> 00:08:00
step looks as follows.
We set the pivot to be the
78
00:08:00 --> 00:08:05
first element.
Here is p and here is q.
79
00:08:05 --> 00:08:10
This is going to be our
invariant for the loop.
80
00:08:10 --> 00:08:15
And, at any time during the
execution of a loop,
81
00:08:15 --> 00:08:22
I essentially have some values
up to i which are already less
82
00:08:22 --> 00:08:28
than or equal to x and then some
values that end at j minus 1
83
00:08:28 --> 00:08:34
that are greater than or equal
to x.
84
00:08:34 --> 00:08:38
And then I don't know about the
rest.
85
00:08:38 --> 00:08:44
And so we start out with i
equal to p and j equal to p plus
86
00:08:44 --> 1.
87
1. --> 00:08:47
It starts out at p plus 1 so
88
00:08:47 --> 00:08:52
that everything is unknown
except for x here.
89
00:08:52 --> 00:08:58
And then the idea is that it is
going to preserve this
90
00:08:58 --> 00:09:02
invariant.
And the way it does it is,
91
00:09:02 --> 00:09:06
as we go through the loop,
it looks at a of j and says is
92
00:09:06 --> 00:09:10
it greater than or equal to x?
Sorry, is it less than or equal
93
00:09:10 --> 00:09:12
to x?
If it is greater than or equal
94
00:09:12 --> 00:09:15
to x it does nothing,
because what can happen?
95
00:09:15 --> 00:09:19
If this is greater than or
equal to x, essentially it just
96
00:09:19 --> 00:09:23
goes to the next iterational
loop which moves this boundary
97
00:09:23 --> 00:09:27
and the invariant is satisfied.
Does everybody see that?
98
00:09:27 --> 00:09:31
Yeah, OK.
But if it is less than or equal
99
00:09:31 --> 00:09:35
to x, I have got a problem if I
want to maintain the invariant
100
00:09:35 --> 00:09:38
if this next element is less
than or equal to x.
101
00:09:38 --> 00:09:43
And so what it does then is it
says oh, let me just move this
102
00:09:43 --> 00:09:47
boundary and swap this element
here, which is greater than or
103
00:09:47 --> 00:09:51
equal to x, with this one here
that is less than or equal to x,
104
00:09:51 --> 00:09:55
thereby increasing the size of
this subarray and then the
105
00:09:55 --> 00:09:59
invariant is satisfied again.
It is a fairly simple
106
00:09:59 --> 00:10:03
algorithm.
And it is actually a very tight
107
00:10:03 --> 00:10:08
and easy algorithm.
That is one reason that this is
108
00:10:08 --> 00:10:13
such a great piece of code
because it is very efficient.
109
00:10:13 --> 00:10:18
Now, in principle,
the running time for this on n
110
00:10:18 --> 00:10:21
elements is order n.
111
00:10:21 --> 00:10:28
112
00:10:28 --> 00:10:31
Because I am basically just
going through the n elements and
113
00:10:31 --> 00:10:35
just doing a constant amount of
work and then just a constant
114
00:10:35 --> 00:10:39
amount of work outside.
This is a clever piece of code.
115
00:10:39 --> 00:10:41
In fact, in principle partition
is easy, right?
116
00:10:41 --> 00:10:44
If I weren't worrying about
doing it in place,
117
00:10:44 --> 00:10:47
it is really a pretty easy
thing to do.
118
00:10:47 --> 00:10:50
I take an element and just
compare every other element with
119
00:10:50 --> 00:10:52
it.
I throw one into one bin and
120
00:10:52 --> 00:10:57
one into the other.
That is clearly linear time.
121
00:10:57 --> 00:11:01
But often what you find is that
just because you can do it that
122
00:11:01 --> 00:11:04
way theoretically doesn't mean
that that is going to end up
123
00:11:04 --> 00:11:08
giving you good code.
And this is a nice piece of
124
00:11:08 --> 00:11:10
code that allows you to do it in
place.
125
00:11:10 --> 00:11:14
And that is one reason why this
is a particularly good
126
00:11:14 --> 00:11:16
algorithm, because the constants
are good.
127
00:11:16 --> 00:11:20
So, yes, when we do asymptotic
analysis we tend to ignore the
128
00:11:20 --> 00:11:24
constants, but when you're
actually building code you care
129
00:11:24 --> 00:11:30
about the constants.
But first you care much more
130
00:11:30 --> 00:11:37
than just about the constants,
is whether overall it is going
131
00:11:37 --> 00:11:43
to be a fast algorithm.
Let's go through an example of
132
00:11:43 --> 00:11:50
this, I guess I will do it over
here, just so we get the gist.
133
00:11:50 --> 00:11:57
Here is a sample array that I
have created out of hallcloth.
134
00:11:57 --> 00:12:05
And here we are going to set x,
the pivot, to be 6.
135
00:12:05 --> 00:12:08
Let's look to see how this
algorithm works.
136
00:12:08 --> 00:12:12
So, i starts out here and j
starts out here if we
137
00:12:12 --> 00:12:15
initialize.
And what we do is start
138
00:12:15 --> 00:12:18
scanning right,
essentially that code is
139
00:12:18 --> 00:12:23
scanning right until it gets
something which is less than or
140
00:12:23 --> 00:12:27
equal to the pivot.
It keeps going here until it
141
00:12:27 --> 00:12:32
finds, j keeps incrementing
until it finds something that is
142
00:12:32 --> 00:12:38
less than or equal to the pivot.
And, in that case,
143
00:12:38 --> 00:12:44
it is the number 5.
Then it says we will swap these
144
00:12:44 --> 00:12:49
two things.
And it does that and we get 6,
145
00:12:49 --> 00:12:52
5, 13, 10, 8,
3, 2, 11.
146
00:12:52 --> 00:12:58
And meanwhile now i gets
incremented and j continues
147
00:12:58 --> 00:13:05
where it left off.
And so now we keep scanning
148
00:13:05 --> 00:13:12
right until we get to something
that is less than or equal to
149
00:13:12 --> 00:13:16
the pivot.
In this case it is 3.
150
00:13:16 --> 00:13:20
We swap 3 and 5 and get 6,
3, etc.
151
00:13:20 --> 00:13:27
And now, at this step we
increment i, we start j out
152
00:13:27 --> 00:13:30
here.
And in this case,
153
00:13:30 --> 00:13:35
right off the bat,
we have something which is less
154
00:13:35 --> 00:13:39
than or equal to x,
so we swap these two.
155
00:13:39 --> 00:13:42
I blew it, didn't I?
Oops.
156
00:13:42 --> 00:13:46
What did I do?
I swapped the wrong thing,
157
00:13:46 --> 00:13:48
didn't I, here?
Ah-ha.
158
00:13:48 --> 00:13:51
That is why I am not a
computer.
159
00:13:51 --> 00:13:54
Good.
We should have swapped this
160
00:13:54 --> 00:13:57
guy, right?
Swapped i plus 1,
161
00:13:57 --> 00:14:01
right?
This was i.
162
00:14:01 --> 00:14:03
We swap i plus 1,
good.
163
00:14:03 --> 00:14:09
So, that's all wrong.
Let's swap the right things.
164
00:14:09 --> 00:14:12
Now we have 6,
5, 3, 10, 8,
165
00:14:12 --> 00:14:16
13, 2, 11.
That even corresponds to my
166
00:14:16 --> 00:14:22
notes for some strange reason.
This is i and now this is j.
167
00:14:22 --> 00:14:28
And now when I look,
I immediately have something
168
00:14:28 --> 00:14:34
that is less than or equal to
the pivot.
169
00:14:34 --> 00:14:41
We swap this and i plus 1,
so now we have 6,
170
00:14:41 --> 00:14:45
5, 3, 2, 8, 13,
10, 11.
171
00:14:45 --> 00:14:52
And we, at that point,
increment i to here.
172
00:14:52 --> 00:15:03
And we have j now going here
and j runs to the end.
173
00:15:03 --> 00:15:08
And the loop terminates.
When the loop terminates there
174
00:15:08 --> 00:15:14
is one less swap that we do,
which is to put our pivot
175
00:15:14 --> 00:15:18
element in the middle between
the two subarrays.
176
00:15:18 --> 00:15:25
Here we swap this one and this
one, and so that gives us then
177
00:15:25 --> 00:15:27
2, 5, 3, 6, 8,
13, 10, 11.
178
00:15:27 --> 00:15:33
And this is the pivot.
And everything over here is
179
00:15:33 --> 00:15:40
less than or equal to the pivot.
And everything over here is
180
00:15:40 --> 00:15:44
greater than or equal to the
pivot.
181
00:15:44 --> 00:15:56
182
00:15:56 --> 00:15:59
OK, so the quicksort routine.
Once we have this partition
183
00:15:59 --> 00:16:03
routine, quicksort is a pretty
easy piece of code to write.
184
00:16:03 --> 00:16:18
185
00:16:18 --> 00:16:21
I should have said return here
i, right?
186
00:16:21 --> 00:16:24
You have got to return with the
pivot.
187
00:16:24 --> 00:16:29
Here I have got to return i
because we want to know where
188
00:16:29 --> 00:16:33
the pivot element is.
Sorry.
189
00:16:33 --> 00:16:46
I will plug in my code.
r gets partition of (A,
190
00:16:46 --> 00:17:03
p, q) and then we quicksort (A,
p, r-1) and quicksort of (A,
191
00:17:03 --> 00:17:09
r+1, q).
And that is it.
192
00:17:09 --> 00:17:17
That's the code.
The initial call is quicksort
193
00:17:17 --> 00:17:24
of (A, 1, n).
Because once we partitioned,
194
00:17:24 --> 00:17:31
we just have to quicksort the
two portions,
195
00:17:31 --> 00:17:39
the left and right portions.
Just the boundary case is
196
00:17:39 --> 00:17:41
probably worth mentioning for a
second.
197
00:17:41 --> 00:17:44
If there are zero or one
elements, that is basically what
198
00:17:44 --> 00:17:47
can possibly happen here,
is that I get zero or one
199
00:17:47 --> 00:17:49
elements here.
Then the point is there is
200
00:17:49 --> 00:17:52
nothing to do because the array
is sorted, either because it is
201
00:17:52 --> 00:17:56
an empty array or because it
only has one element.
202
00:17:56 --> 00:17:59
One of the tricks to making
quicksort go fast,
203
00:17:59 --> 00:18:01
as one tunes this,
is to, in fact,
204
00:18:01 --> 00:18:04
look at having a special
purpose sorting routine for
205
00:18:04 --> 00:18:07
small numbers of elements.
For example,
206
00:18:07 --> 00:18:10
if you get down to five
elements having some straight
207
00:18:10 --> 00:18:14
line piece of code that knows
how to sort five elements
208
00:18:14 --> 00:18:17
sufficiently as opposed to
continuing to go through
209
00:18:17 --> 00:18:20
recursion in order to accomplish
that.
210
00:18:20 --> 00:18:22
And there are a variety of
other things.
211
00:18:22 --> 00:18:27
This is a tail recursive code,
and so you can use certain tail
212
00:18:27 --> 00:18:31
recursion optimizations.
And there are a variety of
213
00:18:31 --> 00:18:35
other kinds of optimizations
that you can use to make this
214
00:18:35 --> 00:18:37
code go fast.
So, yeah, you can tune it up a
215
00:18:37 --> 00:18:40
bit beyond what is there,
but the core of it is this
216
00:18:40 --> 00:18:43
efficient partitioning routine.
217
00:18:43 --> 00:18:53
218
00:18:53 --> 00:18:58
That is the algorithm.
It turns out that looking at
219
00:18:58 --> 00:19:05
how fast it runs is actually a
little bit challenging.
220
00:19:05 --> 00:19:09
In the analysis,
we are going to assume that all
221
00:19:09 --> 00:19:13
elements are distinct.
It turns out that this
222
00:19:13 --> 00:19:19
particular code does not work
very well when you have repeated
223
00:19:19 --> 00:19:25
elements, but Hoare's original
partitioning routine is actually
224
00:19:25 --> 00:19:31
more efficient in that case if
there are duplicates in what you
225
00:19:31 --> 00:19:36
are sorting.
And I encourage you to look at
226
00:19:36 --> 00:19:38
that.
It has a much more complicated
227
00:19:38 --> 00:19:42
invariant for partitioning
routine, but it does a similar
228
00:19:42 --> 00:19:45
kind of thing.
It's just a bit more
229
00:19:45 --> 00:19:48
complicated.
If they weren't all distinct,
230
00:19:48 --> 00:19:52
there are things you can do to
make them distinct or you can
231
00:19:52 --> 00:19:56
just use this code.
The easiest thing to do is just
232
00:19:56 --> 00:20:00
use Hoare's original code
because that works pretty well
233
00:20:00 --> 00:20:07
when they are nondistinct.
But this is a little bit easier
234
00:20:07 --> 00:20:12
to understand.
Let's let T(n) be the
235
00:20:12 --> 00:20:18
worst-case running time on n
elements.
236
00:20:18 --> 00:20:27
And so what is the worse-case?
What is the worse-case going to
237
00:20:27 --> 00:20:31
be for quicksort?
238
00:20:31 --> 00:20:40
239
00:20:40 --> 00:20:43
That's right.
If you always pick the pivot
240
00:20:43 --> 00:20:47
and everything is greater than
or everything is less than,
241
00:20:47 --> 00:20:50
you are not going to partition
the array very well.
242
00:20:50 --> 00:20:54
And when does that happen?
What does the original input
243
00:20:54 --> 00:20:57
look like that makes that
happen?
244
00:20:57 --> 00:21:01
If it is already sorted or
reverse sorted.
245
00:21:01 --> 00:21:05
So, if the input is sorted or
reverse sorted.
246
00:21:05 --> 00:21:09
That is actually kind of
important to understand,
247
00:21:09 --> 00:21:14
because it turns out the most
common thing to sort is
248
00:21:14 --> 00:21:19
something that is already
sorted, surprisingly,
249
00:21:19 --> 00:21:22
or things that are nearly
sorted.
250
00:21:22 --> 00:21:27
But often it is just sorted and
somebody wants to make sure it
251
00:21:27 --> 00:21:33
is sorted.
Well, let's just sort it again
252
00:21:33 --> 00:21:38
rather than checking to see if
it is sorted.
253
00:21:38 --> 00:21:43
And, in those cases,
one side of the partition of
254
00:21:43 --> 00:21:50
each partition has no elements.
Then we can write out what the
255
00:21:50 --> 00:21:54
recursion is for that.
We have T(n).
256
00:21:54 --> 00:22:00
If one side has no elements,
we are going to have T(0) on
257
00:22:00 --> 00:22:06
that side.
And on the other side we are
258
00:22:06 --> 00:22:11
going to have T(n-1).
We are just writing out the
259
00:22:11 --> 00:22:16
recursion for this.
One side has no elements.
260
00:22:16 --> 00:22:19
The other side has n-1
elements.
261
00:22:19 --> 00:22:26
And then partitioning and all
the bookkeeping and so forth is
262
00:22:26 --> 00:22:30
order n.
What is T(0)?
263
00:22:30 --> 00:22:35
What is T(0)?
What is that asymptotically?
264
00:22:35 --> 00:22:38
It's a constant,
order 1.
265
00:22:38 --> 00:22:44
That is just order 1 + T(n-1) +
order n.
266
00:22:44 --> 00:22:50
Well, the order 1 can be
absorbed into the order n,
267
00:22:50 --> 00:22:59
so this is really just saying
it is T(n-1) + order n.
268
00:22:59 --> 00:23:08
And what is that equal to?
That is order n^2.
269
00:23:08 --> 00:23:19
Why is that order n^2?
It is an arithmetic series.
270
00:23:19 --> 00:23:30
Actually, just like we got for
insertion sort.
271
00:23:30 --> 00:23:35
Just like for insertion sort it
is an arithmetic series.
272
00:23:35 --> 00:23:42
Going through all that work and
we have an algorithm called
273
00:23:42 --> 00:23:47
quicksort, and it is no faster
than insertion sort.
274
00:23:47 --> 00:23:51
Nevertheless,
I said it was a good algorithm.
275
00:23:51 --> 00:23:57
The reason it is a good
algorithm is because its average
276
00:23:57 --> 00:24:04
case time, as we are going to
see, is very good.
277
00:24:04 --> 00:24:09
But let's try to understand
this a little bit more just so
278
00:24:09 --> 00:24:15
that we understand the
difference between what is going
279
00:24:15 --> 00:24:20
to happen in the average case
and what is going to happen in
280
00:24:20 --> 00:24:25
the worse-case.
Let's draw a recursion tree for
281
00:24:25 --> 00:24:31
this for T(n) = T(0) + T(n-1) +
and I will make the constant
282
00:24:31 --> 00:24:36
explicit for cn.
So, we get an intuition of what
283
00:24:36 --> 00:24:39
is going on.
Some constant times n.
284
00:24:39 --> 00:24:44
And then we have T(n) is equal
to, and we write it with the
285
00:24:44 --> 00:24:47
constant part here,
cn, and then T(0) here,
286
00:24:47 --> 00:24:51
and then T(n-1) here.
Now, I know that all you folks
287
00:24:51 --> 00:24:56
are really fast and want to jump
immediately to the full-blown
288
00:24:56 --> 00:24:59
tree.
But, let me tell you,
289
00:24:59 --> 00:25:04
my advice is that you spend
just a couple of minutes writing
290
00:25:04 --> 00:25:06
it out.
Since the tree grows
291
00:25:06 --> 00:25:10
exponentially,
it only costs you a constant
292
00:25:10 --> 00:25:14
overhead to write out the small
cases and make sure that you
293
00:25:14 --> 00:25:18
have got the pattern that you
are developing.
294
00:25:18 --> 00:25:21
So, I am going to go one more
step.
295
00:25:21 --> 00:25:26
Here we have T(0) and now this
becomes c(n-1) and now we have
296
00:25:26 --> 00:25:30
another T(0) over here and
T(n-2).
297
00:25:30 --> 00:25:35
And we continue that,
dot, dot, dot.
298
00:25:35 --> 00:25:45
That is all equal to cn with a
T(0) here, c(n-1) with a T(0),
299
00:25:45 --> 00:25:53
c(n-2), T(0) here,
and that goes all the way down
300
00:25:53 --> 00:26:01
until we end up with T(1) down
here.
301
00:26:01 --> 00:26:07
What is the height of this tree?
302
00:26:07 --> 00:26:17
303
00:26:17 --> 00:26:20
What is the height of the tree
here?
304
00:26:20 --> 00:26:22
Yeah, n.
Good.
305
00:26:22 --> 00:26:30
Because every step we are just
decrementing the argument by 1.
306
00:26:30 --> 00:26:35
So, the height is n.
To analyze this,
307
00:26:35 --> 00:26:40
let's first add up everything
that is here.
308
00:26:40 --> 00:26:48
Just so we understand where
these things are coming from,
309
00:26:48 --> 00:26:56
this is just theta of the
summation of k equals 1 to n of
310
00:26:56 --> 00:27:02
k, actually of ck.
That is what is in there.
311
00:27:02 --> 00:27:08
And that is equal to order n^2.
That is where our algorithmatic
312
00:27:08 --> 00:27:12
series is coming from.
So, that is Theta(n^2).
313
00:27:12 --> 00:27:18
And then all of these things
here are all Theta(1).
314
00:27:18 --> 00:27:23
315
00:27:23 --> 00:27:33
And how many of them are there?
There are n Theta(1)'s.
316
00:27:33 --> 00:27:54
So, the total amount is T(n) =
Theta(n) + Theta(n^2) =
317
00:27:54 --> 00:28:03
Theta(n^2).
Just to see what the structure
318
00:28:03 --> 00:28:09
is in terms of the recursion
tree, it is a highly unbalanced
319
00:28:09 --> 00:28:14
recursion tree.
Now I am going to do something
320
00:28:14 --> 00:28:20
that I told you should never do,
which is we are going to be do
321
00:28:20 --> 00:28:25
a best-case analysis.
This is for intuition only.
322
00:28:25 --> 00:28:32
And, in general,
we don't do best-case analyses.
323
00:28:32 --> 00:28:35
It doesn't mean anything,
unless we get some intuition
324
00:28:35 --> 00:28:38
for it maybe.
But basically it means nothing
325
00:28:38 --> 00:28:43
mathematically because it's
providing no guarantee.
326
00:28:43 --> 00:28:55
327
00:28:55 --> 00:29:00
And so this is intuition only.
328
00:29:00 --> 00:29:05
329
00:29:05 --> 00:29:13
If we are really lucky what
happens for partition?
330
00:29:13 --> 00:29:20
What is going to be the lucky
case?
331
00:29:20 --> 00:29:26
332
00:29:26 --> 00:29:28
Yeah, it splits right in the
middle.
333
00:29:28 --> 00:29:31
Which is essentially --
334
00:29:31 --> 00:29:43
335
00:29:43 --> 00:29:46
-- n/2 : n/2.
It is really (n-1)/2 :
336
00:29:46 --> 00:29:51
(n-1)/2, but we're not going to
worry about the details because
337
00:29:51 --> 00:29:56
we're only doing intuition for
the best-case because best-case
338
00:29:56 --> 00:30:01
is not what we want.
If that happened,
339
00:30:01 --> 00:30:11
what is the recurrence I get?
Imagine it split it exactly in
340
00:30:11 --> 00:30:18
the middle every time,
then what happens?
341
00:30:18 --> 00:30:26
You get T(n) = 2T(n/2) + order
n for partitioning and
342
00:30:26 --> 00:30:33
bookkeeping.
And what is the solution of
343
00:30:33 --> 00:30:36
that recurrence?
That is n log n.
344
00:30:36 --> 00:30:42
That is the same as the merge
sort recurrence.
345
00:30:42 --> 00:30:46
It is which case of the master
theorem?
346
00:30:46 --> 00:30:51
Case 2, right?
Because n to the log base 2 of
347
00:30:51 --> 00:30:55
2 is n to the 1,
it is the same,
348
00:30:55 --> 00:31:05
so we tack on the extra log n.
Case 2 of the master theorem.
349
00:31:05 --> 00:31:14
That is pretty good.
That says that in the best-case
350
00:31:14 --> 00:31:24
quicksort is going to do well.
How about let's suppose the
351
00:31:24 --> 00:31:32
split is always let's say 1/10 :
9/10, 1/10n :
352
00:31:32 --> 00:31:35
9/10n.
In that case,
353
00:31:35 --> 00:31:43
are we lucky or are we unlucky?
354
00:31:43 --> 00:31:52
355
00:31:52 --> 00:31:57
I mean, if the split is really
skewed, we clearly are going to
356
00:31:57 --> 00:32:00
be unlucky, right,
because then it's,
357
00:32:00 --> 00:32:04
say, 1 to n.
If it is really in the middle
358
00:32:04 --> 00:32:08
it is n log n.
What do you suppose it is if it
359
00:32:08 --> 00:32:11
is 1/10 : 9/10?
Is that lucky or unlucky?
360
00:32:11 --> 00:32:14
We will have a little democracy
here.
361
00:32:14 --> 00:32:17
Who thinks that that is a lucky
case?
362
00:32:17 --> 00:32:20
It is going to be fast running
time.
363
00:32:20 --> 00:32:23
And who thinks it is an unlucky
case?
364
00:32:23 --> 00:32:26
OK, so we have some brave
souls.
365
00:32:26 --> 00:32:30
And who didn't vote?
Oh, come on.
366
00:32:30 --> 00:32:32
Come on.
It is always better,
367
00:32:32 --> 00:32:35
by the way, to say yes or no
and be right or wrong,
368
00:32:35 --> 00:32:40
because then you have some
emotional commitment to it and
369
00:32:40 --> 00:32:43
we will remember better,
rather than just sitting and
370
00:32:43 --> 00:32:47
being quiet.
You don't manipulate your own
371
00:32:47 --> 00:32:50
emotions well enough to remember
things well.
372
00:32:50 --> 00:32:54
Those people who voted win over
the people who don't vote,
373
00:32:54 --> 00:32:56
whether they are right or
wrong.
374
00:32:56 --> 00:33:02
Well, let's take a look.
Here is the recurrence.
375
00:33:02 --> 00:33:07
T(n) = T(1/10n) + T(9/10n) +
Theta(n).
376
00:33:07 --> 00:33:15
And we will assume that this
part here is less than or equal
377
00:33:15 --> 00:33:19
to some cn in order to analyze
it.
378
00:33:19 --> 00:33:25
We will just do a recursion
tree for this and see.
379
00:33:25 --> 00:33:30
Here is a recursion tree.
380
00:33:30 --> 00:33:35
381
00:33:35 --> 00:33:41
We have T(n) = cn,
T(1/10n), T(9/10n).
382
00:33:41 --> 00:33:46
Now we have again cn at the
top.
383
00:33:46 --> 00:33:51
This gets complicated,
right?
384
00:33:51 --> 00:34:00
This is 1/10cn.
Now, over here we have 1/10.
385
00:34:00 --> 00:34:09
And then we are plugging it
into the recursion again,
386
00:34:09 --> 00:34:20
so we now get T(1/100n) and
over here we get T(9/100n).
387
00:34:20 --> 00:34:26
And over here we have now
9/10cn.
388
00:34:26 --> 00:34:34
And that gives us T(9/100n)
again.
389
00:34:34 --> 00:34:48
And here we get T(81/100n).
And we keep going on.
390
00:34:48 --> 00:34:59
That is equal to cn,
1/10cn here.
391
00:34:59 --> 00:35:08
Down this way we have 1/100cn.
And that keeps going down until
392
00:35:08 --> 00:35:16
we get to order 1 down here.
And over here we have 9/10cn.
393
00:35:16 --> 00:35:24
And here, let's see,
this is 9/100cn and this is now
394
00:35:24 --> 00:35:32
9/100cn and this is 81/100cn.
And these things keep going
395
00:35:32 --> 00:35:36
down until they get down to
order 1.
396
00:35:36 --> 00:35:41
But the leaves are not all at
uniform depth here,
397
00:35:41 --> 00:35:45
right?
This side is way further up
398
00:35:45 --> 00:35:47
than this side,
right?
399
00:35:47 --> 00:35:53
Because here we are only going
down by 9/10 each time.
400
00:35:53 --> 00:35:58
So, in fact,
what is the length of this path
401
00:35:58 --> 00:36:02
here?
What is the length of this path
402
00:36:02 --> 00:36:06
down to this,
if I take the left most spine?
403
00:36:06 --> 00:36:15
404
00:36:15 --> 00:36:23
Somebody raise there hand.
Yeah?
405
00:36:23 --> 00:36:30
Log base 10 of n.
Because I am basically cutting
406
00:36:30 --> 00:36:33
down by a factor of 10 each
time.
407
00:36:33 --> 00:36:35
And how long does it take me to
reduce it to 1?
408
00:36:35 --> 00:36:38
That is the definition,
if you will,
409
00:36:38 --> 00:36:40
of what a log is,
log base 10.
410
00:36:40 --> 00:36:43
What is this one?
What is this path going that
411
00:36:43 --> 00:36:44
way?
412
00:36:44 --> 00:36:53
413
00:36:53 --> 00:37:02
Log of n. Log base 10/9 of n.
Because we're going down by
414
00:37:02 --> 00:37:05
9/10 each time.
Once again, essentially the
415
00:37:05 --> 00:37:09
definition of n.
And everything in between there
416
00:37:09 --> 00:37:14
is somewhere between log base 10
of n and log base 10/9 of n.
417
00:37:14 --> 00:37:17
So, everything is in between
there.
418
00:37:17 --> 00:37:22
Now what I can do is do the
trick that we did for mergesort
419
00:37:22 --> 00:37:26
in looking at what the
evaluation of this is by adding
420
00:37:26 --> 00:37:31
up what is the cost of the total
level.
421
00:37:31 --> 00:37:36
That is just cn.
What is the cost of the next
422
00:37:36 --> 00:37:38
level?
cn.
423
00:37:38 --> 00:37:43
And what is the cost of the
next level?
424
00:37:43 --> 00:37:47
cn.
Every level we are still doing
425
00:37:47 --> 00:37:54
the same amount of work.
And we take that all the way
426
00:37:54 --> 00:38:00
down.
And the last levels --
427
00:38:00 --> 00:38:04
Eventually we hit some point
where it is not equal to cn
428
00:38:04 --> 00:38:09
where we start getting things
that are less than or equal to
429
00:38:09 --> 00:38:14
cn because some of the leaves
start dropping out starting at
430
00:38:14 --> 00:38:17
this level.
Basically this part is going to
431
00:38:17 --> 00:38:21
be log base 10 of n,
and then we start getting
432
00:38:21 --> 00:38:25
things that are less than or
equal to cn, and so forth,
433
00:38:25 --> 00:38:30
until finally we get to add it
all up.
434
00:38:30 --> 00:38:36
T(n) is going to be less than
or equal to cn times,
435
00:38:36 --> 00:38:43
well, what is the longest that
this could possibly be?
436
00:38:43 --> 00:38:49
Log base 10/9 of n.
Plus we have all of the leaves
437
00:38:49 --> 00:38:56
that we have to add in,
but all the leaves together add
438
00:38:56 --> 00:39:03
up to just order n.
All the leaves add up to order
439
00:39:03 --> 00:39:08
n, so we have + Theta(n).
And so this is how much?
440
00:39:08 --> 00:39:14
If I add all of this together,
what is this asymptotically?
441
00:39:14 --> 00:39:18
That is n log n.
So, T(n) is actually bounded by
442
00:39:18 --> 00:39:21
n log n.
We are lucky.
443
00:39:21 --> 00:39:25
Those people who guessed lucky
were right.
444
00:39:25 --> 00:39:30
A 1/10 : 9/10 split is
asymptotically as good as a 50 :
445
00:39:30 --> 00:39:34
50 split.
And, in fact,
446
00:39:34 --> 00:39:41
we can lower bound this by just
looking at these things here and
447
00:39:41 --> 00:39:45
discover that,
in fact, T(n) is lower bounded
448
00:39:45 --> 00:39:49
by cn log base 10 of n + order
n.
449
00:39:49 --> 00:39:55
And so T(n) is lower bounded by
also asymptotically n log n.
450
00:39:55 --> 00:40:00
So, T(n) is actually Theta(n lg
n).
451
00:40:00 --> 00:40:04
Now, this is not really proof.
I generally recommend that you
452
00:40:04 --> 00:40:07
don't do this kind of thing to
do a proof.
453
00:40:07 --> 00:40:10
This is a good intuition of a
recursion tree.
454
00:40:10 --> 00:40:14
The way you prove this is what?
Substitution method.
455
00:40:14 --> 00:40:17
Good.
What you do is use this to get
456
00:40:17 --> 00:40:20
your guess and then use
substitution method to prove
457
00:40:20 --> 00:40:25
that your guess is right.
It is too easy to make mistakes
458
00:40:25 --> 00:40:28
with this method.
It is very easy to make
459
00:40:28 --> 00:40:32
mistakes.
With the substitution method it
460
00:40:32 --> 00:40:37
is harder to make mistakes
because there is just algebra
461
00:40:37 --> 00:40:40
there that you are cranking
through.
462
00:40:40 --> 00:40:43
It is easier to verify rather
than dot, dot,
463
00:40:43 --> 00:40:48
dots and trees that you drew
improperly and wrote in wrong
464
00:40:48 --> 00:40:50
amounts and so forth.
OK?
465
00:40:50 --> 00:40:53
So, this is n log n.
That's pretty good.
466
00:40:53 --> 00:40:56
It is order n log n.
And we are lucky.
467
00:40:56 --> 00:41:01
Now let's try another one.
This is all for intuition
468
00:41:01 --> 00:41:05
because, I will tell you,
by the time we get to the end
469
00:41:05 --> 00:41:10
of this class you folks are
going to bolting for the door
470
00:41:10 --> 00:41:13
because we are going to do some
good math today,
471
00:41:13 --> 00:41:16
actually.
It is actually fun math,
472
00:41:16 --> 00:41:20
I think, but it is challenging.
If you are not awake,
473
00:41:20 --> 00:41:23
you can still sleep now,
but I will tell you when to
474
00:41:23 --> 00:41:26
wake up.
One more bit of intuition.
475
00:41:26 --> 00:41:30
Suppose that we alternate
steps.
476
00:41:30 --> 00:41:33
Suppose we do the partitioning
thing.
477
00:41:33 --> 00:41:38
And it happens that we start
out lucky and then we have a
478
00:41:38 --> 00:41:43
partitioning step that is
unlucky and then we have a step
479
00:41:43 --> 00:41:49
that is lucky and a step that is
unlucky and we do that all the
480
00:41:49 --> 00:41:54
way down the tree.
Suppose we alternate.
481
00:41:54 --> 00:42:07
482
00:42:07 --> 00:42:09
Are we lucky or unlucky if we
do that?
483
00:42:09 --> 00:42:11
This time I want everybody
voting.
484
00:42:11 --> 00:42:13
It doesn't matter what your
answer is.
485
00:42:13 --> 00:42:16
Everybody has to have a stake
in the game.
486
00:42:16 --> 00:42:19
It is sort of like horseracing.
If ever you have watched
487
00:42:19 --> 00:42:23
horseracing, it is really
boring, but if you put a little
488
00:42:23 --> 00:42:26
bit of money down,
a little skin in the game
489
00:42:26 --> 00:42:30
suddenly it is interesting.
The same thing here.
490
00:42:30 --> 00:42:33
I want everybody to put some
skin in the game.
491
00:42:33 --> 00:42:37
Who thinks that this is going
to be lucky?
492
00:42:37 --> 00:42:40
Who thinks it is going to be
unlucky?
493
00:42:40 --> 00:42:42
OK.
Who didn't vote?
494
00:42:42 --> 00:42:46
[LAUGHTER] You guys.
No skin in the game,
495
00:42:46 --> 00:42:48
ha?
Let's analyze this so we can
496
00:42:48 --> 00:42:53
once again write a recurrence.
On the lucky step,
497
00:42:53 --> 00:42:57
we will have L(n) be the
running time on a lucky step of
498
00:42:57 --> 00:43:02
size n.
And that is going to be twice.
499
00:43:02 --> 00:43:07
While the next step is going to
be unlucky.
500
00:43:07 --> 00:43:10
It is two unluckies over 2 plus
order n.
501
00:43:10 --> 00:43:15
That is our lucky step.
And then for the unlucky step
502
00:43:15 --> 00:43:20
it is essentially going to be L
of n minus 1,
503
00:43:20 --> 00:43:25
it is going to be lucky on the
next step, plus order n.
504
00:43:25 --> 00:43:30
That is unlucky.
See how I have described this
505
00:43:30 --> 00:43:36
behavior with a system now of
recurrences that are dependent
506
00:43:36 --> 00:43:41
where the boundary cases,
once again which are unstated,
507
00:43:41 --> 00:43:47
is that the recurrences have a
constant solution with constant
508
00:43:47 --> 00:43:50
input.
Now we just do a little bit of
509
00:43:50 --> 00:43:54
algebra using substitution.
L(n) is then equal to,
510
00:43:54 --> 00:44:00
well, I can just plug in,
for U(n/2) plug in the value of
511
00:44:00 --> 00:44:06
U(n/2).
And that gives me 2[L(n/2-1) +
512
00:44:06 --> 00:44:11
Theta(n) + Theta(n)].
See what I did here?
513
00:44:11 --> 00:44:18
I simply plugged in,
for U(n/2), this recurrence.
514
00:44:18 --> 00:44:27
In fact, technically I guess I
should have said Theta(n/2) just
515
00:44:27 --> 00:44:35
to make this substitution more
straightforward.
516
00:44:35 --> 00:44:41
It is the same thing,
but just to not skip a step.
517
00:44:41 --> 00:44:49
That we can now crank through.
And that is 2L(n/2 - 1) +,
518
00:44:49 --> 00:44:57
and now I have two T(n/2) plus
another one, so all of that is
519
00:44:57 --> 00:45:04
just order n.
And what is the solution to
520
00:45:04 --> 00:45:07
that recurrence?
n log n.
521
00:45:07 --> 00:45:13
Theta(n lg n).
Does everybody see that?
522
00:45:13 --> 00:45:15
OK?
Theta(n lg n).
523
00:45:15 --> 00:45:24
This is basically just,
once again, master theorem with
524
00:45:24 --> 00:45:33
a little bit of jiggering here.
That minus one is only going to
525
00:45:33 --> 00:45:38
help us, actually,
in the solution of the master
526
00:45:38 --> 00:45:41
theorem.
So, it is order n lg n.
527
00:45:41 --> 00:45:45
We are lucky.
If we alternate lucky and
528
00:45:45 --> 00:45:50
unlucky, we are lucky.
How can we insure that we are
529
00:45:50 --> 00:45:55
usually lucky?
If I have the input already
530
00:45:55 --> 00:46:00
sorted, I am going to be
unlucky.
531
00:46:00 --> 00:46:03
Excuse me?
You could randomly arrange the
532
00:46:03 --> 00:46:07
elements, that is one way.
What is another way?
533
00:46:07 --> 00:46:10
That is a perfectly good way,
actually.
534
00:46:10 --> 00:46:13
In fact, it is a common thing
to do.
535
00:46:13 --> 00:46:16
Randomly choose the pivot,
OK.
536
00:46:16 --> 00:46:20
It turns out those are
effectively equivalent,
537
00:46:20 --> 00:46:24
but we are going to do the
randomly choose the pivot
538
00:46:24 --> 00:46:30
because it is a little bit
easier to analyze.
539
00:46:30 --> 00:46:33
But they are effectively
equivalent.
540
00:46:33 --> 00:46:39
That gives us the algorithm
called randomized quicksort.
541
00:46:39 --> 00:46:46
542
00:46:46 --> 00:46:53
And the nice thing about
randomized quicksort is that the
543
00:46:53 --> 00:47:00
running time is independent of
the input ordering.
544
00:47:00 --> 00:47:04
Very much for the same reason
that if I just scramble the
545
00:47:04 --> 00:47:08
input, it would be independent
of the input ordering.
546
00:47:08 --> 00:47:12
If I randomly scramble the
input then it doesn't matter
547
00:47:12 --> 00:47:17
what the order of the input was.
Whereas, original quicksort has
548
00:47:17 --> 00:47:21
some slow cases,
input sorted or reverse sorted,
549
00:47:21 --> 00:47:24
and some fast cases.
In particular,
550
00:47:24 --> 00:47:28
it turns out that if it is
random it is going to be pretty
551
00:47:28 --> 00:47:31
fast.
If I actually randomly scramble
552
00:47:31 --> 00:47:35
the input or pivot on a random
element, it doesn't matter what
553
00:47:35 --> 00:47:38
the input was.
One way of thinking about this
554
00:47:38 --> 00:47:40
is with an adversary.
Imagine your adversary,
555
00:47:40 --> 00:47:44
you are saying I have a good
sorting algorithm and he says I
556
00:47:44 --> 00:47:47
have a good sorting algorithm
and you're trying to sell to a
557
00:47:47 --> 00:47:50
single customer.
And the customer says OK,
558
00:47:50 --> 00:47:53
you guys come up with
benchmarks for each of your
559
00:47:53 --> 00:47:55
algorithms.
And you get to look at his
560
00:47:55 --> 00:47:59
algorithm.
Well, you look and you say oh,
561
00:47:59 --> 00:48:02
he is using quicksort.
I will just give him something
562
00:48:02 --> 00:48:06
that is already sorted.
That is what you could do to
563
00:48:06 --> 00:48:08
him.
If you had quicksort,
564
00:48:08 --> 00:48:10
he would do the same thing to
you.
565
00:48:10 --> 00:48:14
So, how can you defeat him?
Well, one way is with
566
00:48:14 --> 00:48:17
randomization.
Big idea in computer science,
567
00:48:17 --> 00:48:20
use random numbers.
The idea here is if I permute
568
00:48:20 --> 00:48:23
the ordering at random,
as one suggestion,
569
00:48:23 --> 00:48:28
or I pivot at random places,
then the input ordering didn't
570
00:48:28 --> 00:48:32
matter.
And so there is no bad ordering
571
00:48:32 --> 00:48:36
that he can provide that is
going to make my code run
572
00:48:36 --> 00:48:39
slowly.
Now, I might get unlucky.
573
00:48:39 --> 00:48:43
But that is just unlucky in my
use of my random-number
574
00:48:43 --> 00:48:46
generator.
It is not unlucky with respect
575
00:48:46 --> 00:48:50
to what the input was.
What the input was doesn't
576
00:48:50 --> 00:48:52
matter.
Everybody follow that?
577
00:48:52 --> 00:48:55
OK.
The nice thing about randomized
578
00:48:55 --> 00:48:59
quicksort is that it makes no
assumptions about the input
579
00:48:59 --> 00:49:05
distribution.
You don't have to assume that
580
00:49:05 --> 00:49:13
all inputs are equally likely
because either you can make it
581
00:49:13 --> 00:49:20
that way or you pivot in a way
that makes that effectively
582
00:49:20 --> 00:49:23
whole.
And, in particular,
583
00:49:23 --> 00:49:30
there is no specific input that
can elicit the worst-case
584
00:49:30 --> 00:49:45
behavior.
The worst-case is determined
585
00:49:45 --> 00:50:00
only by a random-number
generator.
586
00:50:00 --> 00:50:03
And, therefore,
since it is only determined by
587
00:50:03 --> 00:50:08
a random-number generator,
we can essentially bound the
588
00:50:08 --> 00:50:13
unluckiness mathematically.
We can say what are the odds?
589
00:50:13 --> 00:50:16
So, we are going to analyze
this.
590
00:50:16 --> 00:50:21
And this is where you know if
you belong in this course or
591
00:50:21 --> 00:50:24
not.
If you have skipped 6.042 or
592
00:50:24 --> 00:50:30
whatever, this is a good place
to do the comparison.
593
00:50:30 --> 00:50:34
Since it is going to be a
little bit, why don't people
594
00:50:34 --> 00:50:37
just stand up for a moment and
take a stretch break.
595
00:50:37 --> 00:50:42
Since this is going to be a
nice piece of mathematics we are
596
00:50:42 --> 00:50:47
going to do, you are going to
want to feel fresh for it.
597
00:50:47 --> 00:51:02
598
00:51:02 --> 00:51:04
Stretch break is over.
599
00:51:04 --> 00:51:10
600
00:51:10 --> 00:51:11
Analysis.
Good.
601
00:51:11 --> 00:51:16
I think we are going to make
this.
602
00:51:16 --> 00:51:22
I am sort of racing.
There is a lot of stuff to
603
00:51:22 --> 00:51:24
cover today.
Good.
604
00:51:24 --> 00:51:32
Let's let T(n) now be the
random variable for the running
605
00:51:32 --> 00:51:36
time assuming --
606
00:51:36 --> 00:51:46
607
00:51:46 --> 00:51:47
Wow.
I didn't even write here what
608
00:51:47 --> 00:51:49
we did here.
So, we are going to pivot on a
609
00:51:49 --> 00:51:51
random element.
610
00:51:51 --> 00:51:59
611
00:51:59 --> 00:52:01
That is the basic scheme we are
going to do.
612
00:52:01 --> 00:52:05
And the way I do that,
by the way, is just in the code
613
00:52:05 --> 00:52:07
for partition,
rather than partitioning on the
614
00:52:07 --> 00:52:10
first element,
before I do the partition,
615
00:52:10 --> 00:52:14
I just swap the first element
with some other element in the
616
00:52:14 --> 00:52:16
array chosen at random,
perhaps itself.
617
00:52:16 --> 00:52:19
So, they are all equally likely
to be pivoted on.
618
00:52:19 --> 00:52:23
And then just run the ordinary
partition.
619
00:52:23 --> 00:52:27
This is a random variable for
running in time assuming,
620
00:52:27 --> 00:52:32
we have to make an assumption
for doing probability,
621
00:52:32 --> 00:52:36
the random numbers are
independent.
622
00:52:36 --> 00:52:41
623
00:52:41 --> 00:52:44
So that when I pivot in one
place, it is independent of how
624
00:52:44 --> 00:52:48
I pivoted in some other place as
I am running this algorithm.
625
00:52:48 --> 00:52:52
Then, to analyze this,
what I am going to do is I want
626
00:52:52 --> 00:52:57
to know where we pivoted.
For k = 0, 1,
627
00:52:57 --> 00:53:08
..., n-1, let's let,
for a particular partition,
628
00:53:08 --> 00:53:21
the random variable X_k = 1 if
partition generates a k :
629
00:53:21 --> 00:53:30
n-k-1 split,
and 0 otherwise.
630
00:53:30 --> 00:53:35
631
00:53:35 --> 00:53:39
In the partition routine,
I am picking a random element
632
00:53:39 --> 00:53:42
to pivot on.
And X_k is going to be my
633
00:53:42 --> 00:53:47
random variable that is 1 if it
generates a split that has k
634
00:53:47 --> 00:53:52
elements on the left side and
n-k-1 elements on the right side
635
00:53:52 --> 00:53:54
of the pivot.
Some of those,
636
00:53:54 --> 00:54:00
too, of course are n-1 because
I also have the pivot.
637
00:54:00 --> 00:54:03
And 0 otherwise.
So, I now have n random
638
00:54:03 --> 00:54:09
variables that I have defined
associated with a single
639
00:54:09 --> 00:54:15
partition where all of them are
going to be zero except one of
640
00:54:15 --> 00:54:21
them, whichever one happens to
occur is going to have the value
641
00:54:21 --> 1.
642
1. --> 00:54:22
This is called,
643
00:54:22 --> 00:54:26
by the way.
What is the name of this type
644
00:54:26 --> 00:54:30
of random variable?
645
00:54:30 --> 00:54:37
646
00:54:37 --> 00:54:40
Bernoulli.
Well, Bernoulli has other
647
00:54:40 --> 00:54:43
assumptions.
It is an indicator random
648
00:54:43 --> 00:54:46
variable.
It turns out it is Bernoulli,
649
00:54:46 --> 00:54:50
but that's OK.
It is an indicator random
650
00:54:50 --> 00:54:53
variable.
It just takes on the value of
651
00:54:53 --> 00:54:56
0, 1.
And Bernoulli random variables
652
00:54:56 --> 00:55:02
are a particular type of
indicator random variable.
653
00:55:02 --> 00:55:06
Which it turns out these are.
That is an indicator random
654
00:55:06 --> 00:55:09
variable.
Indicator random variables are
655
00:55:09 --> 00:55:14
a good way when you are trying
to understand what the sum of a
656
00:55:14 --> 00:55:18
bunch of things is.
It is a good way to break apart
657
00:55:18 --> 00:55:22
your big random variables into
smaller ones that can be
658
00:55:22 --> 00:55:25
analyzed.
Let's just take a look at this
659
00:55:25 --> 00:55:30
indicator random variable.
What is the expectation of X_k
660
00:55:30 --> 00:55:32
equal to?
661
00:55:32 --> 00:55:42
662
00:55:42 --> 00:55:45
In other words,
what is the probability that I
663
00:55:45 --> 00:55:48
generate a k :
n-k-1 split?
664
00:55:48 --> 00:56:00
665
00:56:00 --> 00:56:05
X_k is, let's just write out
what that means,
666
00:56:05 --> 00:56:08
just to refresh people's
memory.
667
00:56:08 --> 00:56:15
That is 0 times the probability
that X_k equals 0 plus 1 times
668
00:56:15 --> 00:56:21
the probability that X_k equals
1, which is equal,
669
00:56:21 --> 00:56:27
well, that is all zero.
That is just equal to the
670
00:56:27 --> 00:56:34
probability that X_k equals 1.
And that is a general property
671
00:56:34 --> 00:56:39
of indicator random variables,
is that their expectation is
672
00:56:39 --> 00:56:45
the probability that they are 1.
The nice thing about indicator
673
00:56:45 --> 00:56:51
random variables is it directly
connects the probability to the
674
00:56:51 --> 00:56:55
expectation without any other
terms going on.
675
00:56:55 --> 00:56:58
What is the probability of X_k
equals 1?
676
00:56:58 --> 00:57:02
1/n.
So, all splits are equally
677
00:57:02 --> 00:57:04
likely.
And I have n elements,
678
00:57:04 --> 00:57:10
so each ones has a 1/n chance
of being picked as the pivot.
679
00:57:10 --> 00:57:15
And, once you pick the pivot,
that determines what is on the
680
00:57:15 --> 00:57:19
left and the right and so forth.
So, it is 1/n.
681
00:57:19 --> 00:57:23
Everybody with me so far?
More or less?
682
00:57:23 --> 00:57:26
OK.
As I say, this is going to test
683
00:57:26 --> 00:57:32
whether you're in the class.
If you go home and you study
684
00:57:32 --> 00:57:37
this and you cannot get it,
and you have a deficiency in
685
00:57:37 --> 00:57:41
your math background in trying
to take the course,
686
00:57:41 --> 00:57:46
this is a good indication that
probably you have taken
687
00:57:46 --> 00:57:49
something a little over your
head.
688
00:57:49 --> 00:57:54
Let's write out what T(n) is
equal to here.
689
00:57:54 --> 00:58:00
690
00:58:00 --> 00:58:11
T(n) is going to be equal to
T(0) + T(n-1) + Theta(n) if we
691
00:58:11 --> 00:58:24
get a 0 : n-1 split and is equal
to T(1) + T(n-2) + order n if we
692
00:58:24 --> 00:58:30
have a 1 : n-2 split.
693
00:58:30 --> 00:58:35
694
00:58:35 --> 00:58:46
And now down here it is going
to be T(n-1) + T(0) + Theta(n)
695
00:58:46 --> 00:58:52
if we end up with an n-1 :
0 split.
696
00:58:52 --> 00:59:00
So, this is our recurrence for
T(n).
697
00:59:00 --> 00:59:03
And, unfortunately,
the recurrence is kind of hairy
698
00:59:03 --> 00:59:06
because it has got n cases.
And this is,
699
00:59:06 --> 00:59:11
once again, where the
brilliance of being able to use
700
00:59:11 --> 00:59:13
indicator random variables comes
in.
701
00:59:13 --> 00:59:18
Because we will be able to take
this case analysis and reduce it
702
00:59:18 --> 00:59:23
to mathematics so we don't have
cases using indicator random
703
00:59:23 --> 00:59:29
variables.
And the way we do that is using
704
00:59:29 --> 00:59:37
the following trick of
converting the cases into a
705
00:59:37 --> 00:59:39
summation.
706
00:59:39 --> 00:59:55
707
00:59:55 --> 1:00:00.666
Let's just take a look at why
these two things are the same.
708
1:00:00.666 --> 1:00:06.333
The indicator random variable
is zero, except if you get the
709
1:00:06.333 --> 1:00:10.655
particular split.
Therefore, this summation is
710
1:00:10.655 --> 1:00:14.497
going to be zero,
except for that k which
711
1:00:14.497 --> 1:00:20.355
actually appeared in which case
it is the value that we say it
712
1:00:20.355 --> 1:00:22.468
is.
See the trick using
713
1:00:22.468 --> 1:00:27.079
multiplication by 0,
1 variable to handle all the
714
1:00:27.079 --> 1:00:31.404
cases?
I think that is damn clever.
715
1:00:31.404 --> 1:00:36.213
I think that is damn clever.
And this is like the classic
716
1:00:36.213 --> 1:00:40.421
thing that you do with indicator
random variables.
717
1:00:40.421 --> 1:00:45.144
It's one of the reasons they
are a very powerful method.
718
1:00:45.144 --> 1:00:49.781
Because now we actually have a
mathematical expression,
719
1:00:49.781 --> 1:00:53.559
hairy although it may be,
for our recurrence.
720
1:00:53.559 --> 1:00:58.454
Now, what we are going to
analyze is the expected value of
721
1:00:58.454 --> 1:01:02.977
T(n).
That is what we want to do.
722
1:01:02.977 --> 1:01:06.727
What is the expected value of
T(n)?
723
1:01:06.727 --> 1:01:13.235
To do that, I just write the
expected value of T(n) is equal
724
1:01:13.235 --> 1:01:17.977
to the expected value of this
big summation.
725
1:01:17.977 --> 1:01:24.264
And now we can go ahead and
start to evaluate the expected
726
1:01:24.264 --> 1:01:30
value of that summation.
Everybody with me?
727
1:01:30 --> 1:01:32.486
Yes?
Any questions at this point?
728
1:01:32.486 --> 1:01:35.36
I see a thumbs up.
That's nice to see.
729
1:01:35.36 --> 1:01:39.944
But I generally believe that
what I want to see is no thumbs
730
1:01:39.944 --> 1:01:42.508
down.
It is good to see the thumbs
731
1:01:42.508 --> 1:01:46.704
up, but that means one person
understands, or thinks he
732
1:01:46.704 --> 1:01:48.724
understands.
[LAUGHTER] So,
733
1:01:48.724 --> 1:01:51.832
this is, I claim,
equal to the following.
734
1:01:51.832 --> 1:01:56.571
Actually, I am going to need a
little space here so I am going
735
1:01:56.571 --> 1:02:01
to move the equal sign over a
little bit.
736
1:02:01 --> 1:02:25
737
1:02:25 --> 1:02:27.702
I claim that summation is equal
to that.
738
1:02:27.702 --> 1:02:32
This expectation is equal to
that summation of expectations.
739
1:02:32 --> 1:02:36.042
Why is that?
What are the magic words that
740
1:02:36.042 --> 1:02:40.281
justify this step?
Linearity of expectation.
741
1:02:40.281 --> 1:02:45.704
The expectation of a sum is the
sum of the expectations.
742
1:02:45.704 --> 1:02:49.253
So, that is linearity of
expectation.
743
1:02:49.253 --> 1:02:52.605
I don't need independence for
that.
744
1:02:52.605 --> 1:02:57.83
That is just always true for
expectation of any random
745
1:02:57.83 --> 1:03:04.591
variables.
The sum of the expectations is
746
1:03:04.591 --> 1:03:10.867
the expectation of the sum and
vice versa.
747
1:03:10.867 --> 1:03:19.744
Here we did the vice versa.
That is equal to now the sum of
748
1:03:19.744 --> 1:03:30
k=0 to n-1 of expectation of X_k
[T(k) + T(n-k-1) + Theta(n)].
749
1:03:30 --> 1:03:36
750
1:03:36 --> 1:03:42.272
Why is that true?
What I have done is I've said
751
1:03:42.272 --> 1:03:49.227
the expectation of the product
is the product of the
752
1:03:49.227 --> 1:03:53.454
expectations.
That is because of
753
1:03:53.454 --> 1:04:00
independence.
What is independent of what?
754
1:04:00 --> 1:04:02.492
The X_k here,
random variable,
755
1:04:02.492 --> 1:04:06.962
are independent of any of the
other partitionings in,
756
1:04:06.962 --> 1:04:10.401
if you will,
the X_k that would exist for
757
1:04:10.401 --> 1:04:13.151
any of the other recursive
calls.
758
1:04:13.151 --> 1:04:18.223
So, whatever happens in here is
independent of what happened
759
1:04:18.223 --> 1:04:20.716
there.
We are actually hiding.
760
1:04:20.716 --> 1:04:25.358
Since we have a recurrence,
we are not partitioning the
761
1:04:25.358 --> 1:04:30
same wage time.
We have a different one.
762
1:04:30 --> 1:04:33.168
We actually have something
going on underneath the
763
1:04:33.168 --> 1:04:36.271
mathematics you have to pay
attention to that the
764
1:04:36.271 --> 1:04:40.15
mathematics alone isn't really
showing, which is that in T(k)
765
1:04:40.15 --> 1:04:43.706
there is actually a set of
random choices that are being
766
1:04:43.706 --> 1:04:46.745
made, if you will.
And so you have to understand
767
1:04:46.745 --> 1:04:50.366
that those are independent of
those, in which case we can
768
1:04:50.366 --> 1:04:53.469
multiple the probabilities of
their expectations.
769
1:04:53.469 --> 1:04:55.991
Is everybody with me?
That is a big one,
770
1:04:55.991 --> 1:05:00
independence of X_k from other
random choices.
771
1:05:00 --> 1:05:04.624
That is equal to now,
well, first of all,
772
1:05:04.624 --> 1:05:09.71
this is nice.
What is the expectation of X_k?
773
1:05:09.71 --> 1:05:13.179
1/n.
That actually doesn't even
774
1:05:13.179 --> 1:05:20
belong in the summation.
We will just pop it outside.
775
1:05:20 --> 1:05:35.541
I get 1/n times the sum of k=0
to n-1 of expectation of T(k) +
776
1:05:35.541 --> 1:05:49.808
1/n summation k=0 to n-1 of
expectation of T(n-k-1) + 1/n
777
1:05:49.808 --> 1:06:00
summation k=0 to n-1 up to
Theta(n).
778
1:06:00 --> 1:06:04.838
That is, again,
using linearity of expectation
779
1:06:04.838 --> 1:06:11.075
there this time to split up
these pieces and just factoring
780
1:06:11.075 --> 1:06:15.053
out the expectation of k as
being 1/n.
781
1:06:15.053 --> 1:06:20.43
Everybody with me still?
All of this is elementary.
782
1:06:20.43 --> 1:06:26.989
It is just one of these things
that is hard just because there
783
1:06:26.989 --> 1:06:33.174
are so many steps.
And it takes that you have seen
784
1:06:33.174 --> 1:06:37.986
some of this before.
Now the next observation is
785
1:06:37.986 --> 1:06:43.003
that these two summations are,
in fact, identical.
786
1:06:43.003 --> 1:06:48.532
They are the same summation,
just in a different order.
787
1:06:48.532 --> 1:06:53.549
This is going T(0),
T(1), T(2), T(3) up to T(n-1).
788
1:06:53.549 --> 1:07:00
This one is going T(n-1),
T(n-2), T(n-3) down to T(0).
789
1:07:00 --> 1:07:02.049
These are, in fact,
equal.
790
1:07:02.049 --> 1:07:05
So, therefore,
I have two of them.
791
1:07:05 --> 1:07:19
792
1:07:19 --> 1:07:22
And then what is this term
equal to?
793
1:07:22 --> 1:07:37
794
1:07:37 --> 1:07:40.991
What is that one equal to?
Theta(n).
795
1:07:40.991 --> 1:07:45.438
Let's just see why.
The summation of 0 :
796
1:07:45.438 --> 1:07:50
n of Theta(n) is Theta(n^2)
divided by n.
797
1:07:50 --> 1:07:54.447
Or, if I want to bring the
Theta(n) out,
798
1:07:54.447 --> 1:08:01.631
I have 1 times the summation of
k equals1 to n of Theta(1) or of
799
1:08:01.631 --> 1.
800
1. --> 1:08:04.527
So, once again,
801
1:08:04.527 --> 1:08:07.888
you get n, either way of doing
it.
802
1:08:07.888 --> 1:08:12.981
This is, in some sense,
because the summations have
803
1:08:12.981 --> 1:08:17.157
identical terms,
and this is just algebra.
804
1:08:17.157 --> 1:08:22.861
Now what we are going to do is
do something for technical
805
1:08:22.861 --> 1:08:30.217
convenience.
Because we are going to absorb
806
1:08:30.217 --> 1:08:38.391
the k=0, 1 terms into the
Theta(n) for technical
807
1:08:38.391 --> 1:08:41
convenience.
808
1:08:41 --> 1:08:46
809
1:08:46 --> 1:08:50.432
We have a recurrence here where
I have an order n.
810
1:08:50.432 --> 1:08:54.412
And, if I look at the cases
where k=0 or k=1,
811
1:08:54.412 --> 1:08:59.929
I know what the expectation is.
For 0, 1, the expected cost is
812
1:08:59.929 --> 1:09:04
the worst case cost,
which is constant.
813
1:09:04 --> 1:09:07.92
Because I am only solving the
problem for a constant size.
814
1:09:07.92 --> 1:09:12.253
And we know that for any of the
boundary cases that our solution
815
1:09:12.253 --> 1:09:15.142
of recurrence,
our assumption is that it is
816
1:09:15.142 --> 1:09:18.1
constant time.
So, I basically can just take
817
1:09:18.1 --> 1:09:21.608
those two terms out.
And all that does it accumulate
818
1:09:21.608 --> 1:09:24.291
some more constant here in the
Theta(n).
819
1:09:24.291 --> 1:09:27.798
It is going to make the
solution of the recurrence a
820
1:09:27.798 --> 1:09:33.396
little bit easier.
And, if I do that,
821
1:09:33.396 --> 1:09:43.584
I get expectation of T(n) = 2/n
summation k=2 to n-1 of
822
1:09:43.584 --> 1:09:50
expectation of T(k) + Theta(n).
823
1:09:50 --> 1:10:00
824
1:10:00 --> 1:10:06.864
So, all of that work was to
derive the recurrence.
825
1:10:06.864 --> 1:10:14.57
And now we have to solve it.
Just to review what we did,
826
1:10:14.57 --> 1:10:21.434
we started out with a
recurrence which was for the
827
1:10:21.434 --> 1:10:29
random variable which involved a
case statement.
828
1:10:29 --> 1:10:33.128
We converted that into some
mathematics without the case
829
1:10:33.128 --> 1:10:37.031
statement, just with a product,
and then we derived a
830
1:10:37.031 --> 1:10:41.535
recurrence for the expectation.
And now we are in the process
831
1:10:41.535 --> 1:10:44.087
of trying to solve that
recurrence.
832
1:10:44.087 --> 1:10:47.765
We have done some
simplification of the recurrence
833
1:10:47.765 --> 1:10:52.569
so that we understand what it is
that we are going to solve here.
834
1:10:52.569 --> 1:10:56.472
By the way, I don't give things
like this on quizzes.
835
1:10:56.472 --> 1:11:00
I do expect you to understand
it.
836
1:11:00 --> 1:11:03.12
The elements of this you will
find on a quiz.
837
1:11:03.12 --> 1:11:05.602
This is a lot of work to figure
out.
838
1:11:05.602 --> 1:11:09.148
This took smart people to do.
Even though it is all
839
1:11:09.148 --> 1:11:12.907
elementary, but working out
something like this at the
840
1:11:12.907 --> 1:11:17.375
elementary level is still a bit
of work even for somebody who is
841
1:11:17.375 --> 1:11:23.465
knowledgeable in this area.
Now we are going to solve that
842
1:11:23.465 --> 1:11:30.395
last recurrence over there and
we are going to prove that the
843
1:11:30.395 --> 1:11:37.556
expectation of T(n) is less than
or equal to (an lg n) for some
844
1:11:37.556 --> 1:11:44.139
constant a greater than 0.
That is going to be what we are
845
1:11:44.139 --> 1:11:48.759
going to do.
And so what technique do you
846
1:11:48.759 --> 1:11:52.571
think we should use to prove
this?
847
1:11:52.571 --> 1:11:58
Does this look like a master
method?
848
1:11:58 --> 1:12:03
849
1:12:03 --> 1:12:08.6
It is nothing like the master
method.
850
1:12:08.6 --> 1:12:13.733
So, when in doubt do
substitution.
851
1:12:13.733 --> 1:12:19.333
It is the grand-daddy of all
methods.
852
1:12:19.333 --> 1:12:28.355
What we will do is solve the
base case by simply choosing a
853
1:12:28.355 --> 1:12:37.844
big enough so that (an lg n) is
bigger than the expectation of
854
1:12:37.844 --> 1:12:45
T(n) for sufficiently large
small n.
855
1:12:45 --> 1:12:49.212
So, I just pick a to be big
enough.
856
1:12:49.212 --> 1:12:54.044
And this is,
by the way, why I wanted to
857
1:12:54.044 --> 1:12:59
exclude 0 and 1 from the
recurrence.
858
1:12:59 --> 1:13:03.338
Because, for example,
when n=0, log of 0 is,
859
1:13:03.338 --> 1:13:08.181
it's like dividing by 0,
right, you cannot do it.
860
1:13:08.181 --> 1:13:12.519
Log of 1 is 0.
So here, even if I restricted
861
1:13:12.519 --> 1:13:18.37
it to 1, here I would have a 0,
and I can't ever pick a big
862
1:13:18.37 --> 1:13:24.424
enough to dominate those cases.
What I do is I just say look,
863
1:13:24.424 --> 1:13:30.578
I just absorb whatever the cost
is into the T(n) for technical
864
1:13:30.578 --> 1:13:37.169
convenience.
And that lets me address it as
865
1:13:37.169 --> 1:13:44.119
(an lg n) to be big enough to
handle the base case.
866
1:13:44.119 --> 1:13:50.93
So, that is why we made that
technical assumption.
867
1:13:50.93 --> 1:13:58.714
We are going to use a fact
which is that the summation of
868
1:13:58.714 --> 1:14:06.776
k=2 to n-1 of k lg k is less
than or equal to 1/2n^2 lg n -
869
1:14:06.776 --> 1:14:11.242
1/8n^2.
I am going to leave that as an
870
1:14:11.242 --> 1:14:15.429
exercise for you to workout.
I think it is an exercise in
871
1:14:15.429 --> 1:14:18.644
the book, too.
I want you to go and evaluate
872
1:14:18.644 --> 1:14:21.261
this.
There are two ways to evaluate
873
1:14:21.261 --> 1:14:23.13
it.
One is by using purely
874
1:14:23.13 --> 1:14:27.093
summations and facts about
summations by splitting the
875
1:14:27.093 --> 1:14:31.579
summation into two pieces and
reconstituting it to prove this
876
1:14:31.579 --> 1:14:35.338
bound.
The other way is to use the
877
1:14:35.338 --> 1:14:38.629
integral method for solving
summations.
878
1:14:38.629 --> 1:14:43.22
Either way you can prove.
The integral method actually
879
1:14:43.22 --> 1:14:46.165
gets you a tighter bound than
this.
880
1:14:46.165 --> 1:14:50.582
This is a basic fact,
and you should go off and know
881
1:14:50.582 --> 1:14:55
how to do that.
Now let's do substitution.
882
1:14:55 --> 1:15:06
883
1:15:06 --> 1:15:18.821
The expectation of T(n) is less
than or equal to 2/n times the
884
1:15:18.821 --> 1:15:29.331
summation k=2 to n-1,
now we do the substitution of
885
1:15:29.331 --> 1:15:39
ak lg k, the smaller values plus
Theta(n).
886
1:15:39 --> 1:15:42.416
I might mentioned,
by the way, that the hard part
887
1:15:42.416 --> 1:15:45.334
of doing this,
it is easy to get the bound
888
1:15:45.334 --> 1:15:48.679
without this term,
it is easy to get this bound,
889
1:15:48.679 --> 1:15:51.669
1/2n^2 lg n,
it is harder to get the second
890
1:15:51.669 --> 1:15:54.231
order term.
It turns out you need the
891
1:15:54.231 --> 1:15:59
second order term in order to do
what we are going to do.
892
1:15:59 --> 1:16:05.621
You have to be able to subtract
a quadratic amount of the n^2 lg
893
1:16:05.621 --> 1:16:09.195
n in order to make this proof
work.
894
1:16:09.195 --> 1:16:15.291
And that is the trickier part
in evaluating that summation.
895
1:16:15.291 --> 1:16:20.126
So, we get this.
That is less than or equal to?
896
1:16:20.126 --> 1:16:26.537
Well, I happen to know how much
this is by using that formula.
897
1:16:26.537 --> 1:16:31.792
I use my fact and get 2a/n
(1/2n^2 lg n - 1/8n^2) +
898
1:16:31.792 --> 1:16:43.558
Theta(n).
Did I do something wrong?
899
1:16:43.558 --> 1:16:51.97
There we go.
Very good.
900
1:16:51.97 --> 1:17:05.272
That is equal to -
If I multiply this first part
901
1:17:05.272 --> 1:17:14.363
through that is an lg n.
And now, so I don't make a
902
1:17:14.363 --> 1:17:22.545
mistake, I want to express this
as my desired,
903
1:17:22.545 --> 1:17:32
this is what I want it to be,
minus a residual.
904
1:17:32 --> 1:17:38.702
I am going to write the
residual as this part.
905
1:17:38.702 --> 1:17:47.489
And so, the way to write that
is, that is going to be minus.
906
1:17:47.489 --> 1:17:56.723
And then it is going to be this
term here, which is going to be
907
1:17:56.723 --> 1:18:00
an/4 - Theta(n).
908
1:18:00 --> 1:18:05
909
1:18:05 --> 1:18:12.309
And that is going to be less
than or equal to an lg n if this
910
1:18:12.309 --> 1:18:17.304
part is positive.
And I can make that part
911
1:18:17.304 --> 1:18:24.614
positive by picking a big enough
such that an/4 dominates the
912
1:18:24.614 --> 1:18:34.59
constant in the Theta(n) here.
Whatever the constant is here,
913
1:18:34.59 --> 1:18:45.211
I can find an a that is big
enough so that this term makes
914
1:18:45.211 --> 1:18:54.527
this part positive.
If a is big enough so that an/4
915
1:18:54.527 --> 1:19:01.217
dominates Theta(n).
And so the running time of
916
1:19:01.217 --> 1:19:04.288
randomized quicksort is order n
lg n.
917
1:19:04.288 --> 1:19:09.064
That is what we just proved,
the expected running time is
918
1:19:09.064 --> 1:19:11.623
order n lg n.
Now, in practice,
919
1:19:11.623 --> 1:19:16.741
quicksort is a great algorithm.
It is typically three or more
920
1:19:16.741 --> 1:19:21.688
times faster than mergesort.
It doesn't give you the strong
921
1:19:21.688 --> 1:19:26.464
guarantee necessarily of
mergesort and being worst-case n
922
1:19:26.464 --> 1:19:29.245
lg n.
But in practice,
923
1:19:29.245 --> 1:19:33.136
if you use randomized
quicksort, it is generally as
924
1:19:33.136 --> 1:19:37.573
much as three times faster.
It does require code tuning in
925
1:19:37.573 --> 1:19:40.219
order to get it up to be that
fast.
926
1:19:40.219 --> 1:19:44.966
You do have to go and coarsen
the base cases and do some other
927
1:19:44.966 --> 1:19:47.613
tricks there,
but most good sorting
928
1:19:47.613 --> 1:19:51.66
algorithms that you will find
are based on quicksort.
929
1:19:51.66 --> 1:19:56.018
Also one of the other reasons
it works well is because it
930
1:19:56.018 --> 1:20:01
tends to work well with caches
in virtual memory.
931
1:20:01 --> 1:20:05.431
We are not really talking much
about caching models and so
932
1:20:05.431 --> 1:20:09.941
forth, big topic these days in
algorithms, but it does work
933
1:20:09.941 --> 1:20:12.973
very well with caches in virtual
memory.
934
1:20:12.973 --> 1:20:17.405
It is another reason that this
is a good algorithm to use.
935
1:20:17.405 --> 1:20:20.359
Good recitation,
by the way, on Friday.
936
1:20:20.359 --> 1:20:24.247
We are going to see another n
log n time algorithm,
937
1:20:24.247 --> 1:20:27
a very important one in
recitation on Friday.