WEBVTT

00:00:08.000 --> 00:00:11.000
OK.
Today we are going to talk

00:00:11.000 --> 00:00:18.000
about a very interesting
algorithm called Quicksort --

00:00:23.000 --> 00:00:30.000
-- which was invented by Tony
Hoare in 1962.

00:00:30.000 --> 00:00:35.000
And it has ended up being a
really interesting algorithm

00:00:35.000 --> 00:00:40.000
from many points of view.
And because of that,

00:00:40.000 --> 00:00:47.000
it turns out today's lecture is
going to be both hard and fast.

00:00:47.000 --> 00:00:52.000
If you see the person next to
you sleeping,

00:00:52.000 --> 00:00:56.000
you will want to say let's get
going.

00:00:56.000 --> 00:01:01.000
It's a divide-and-conquer
algorithm.

00:01:06.000 --> 00:01:08.000
And it sorts,
as they say,

00:01:08.000 --> 00:01:14.000
in place, meaning that it just
rearranged the elements where

00:01:14.000 --> 00:01:17.000
they are.
That is like insertion sort

00:01:17.000 --> 00:01:20.000
rearranges elements where they
are.

00:01:20.000 --> 00:01:24.000
Mergesort does not.
Mergesort requires extra

00:01:24.000 --> 00:01:30.000
storage in order to do the merge
operation.

00:01:30.000 --> 00:01:34.000
To merge in linear time and
place, it doesn't merge in place

00:01:34.000 --> 00:01:37.000
in linear time.
It doesn't do it just by

00:01:37.000 --> 00:01:41.000
rearranging.
It is nice because it is in

00:01:41.000 --> 00:01:45.000
place, so that means that it is
fairly efficient in its use of

00:01:45.000 --> 00:01:48.000
storage.
And it also happens to be very

00:01:48.000 --> 00:01:53.000
practical if you tune it a bit.
The basic algorithm turns out,

00:01:53.000 --> 00:01:58.000
if you just implement that,
it's not necessarily that

00:01:58.000 --> 00:02:02.000
efficient.
But if what you do was then do

00:02:02.000 --> 00:02:07.000
the standard kinds of things you
do to goose up the runtime of

00:02:07.000 --> 00:02:12.000
something, and we'll talk a
little about what those things

00:02:12.000 --> 00:02:16.000
are, then it can be very,
very practical.

00:02:16.000 --> 00:02:20.000
So, it uses divide-and-conquer
paradigm.

00:02:33.000 --> 00:02:40.000
First step is divide.
And to do this basically it

00:02:40.000 --> 00:02:48.000
does it by partitioning.
So, it partitions the input

00:02:48.000 --> 00:02:59.000
array into two subarrays around
an element we call the pivot --

00:03:04.000 --> 00:03:14.000
-- such that elements in the
lower subarray are less than or

00:03:14.000 --> 00:03:25.000
equal to x, are less than or
equal to elements in the upper

00:03:25.000 --> 00:03:31.000
subarray.
If we draw a picture of the

00:03:31.000 --> 00:03:38.000
input array, this partition step
basically takes some element x

00:03:38.000 --> 00:03:45.000
and everything over here is less
than or equal to x after the

00:03:45.000 --> 00:03:52.000
partition step and everything
over here is greater than or

00:03:52.000 --> 00:03:57.000
equal to x.
And so now the conquer step is

00:03:57.000 --> 00:04:04.000
pretty easy.
You just recursively sort the

00:04:04.000 --> 00:04:10.000
two subarrays.
So, I recursively sort the

00:04:10.000 --> 00:04:19.000
elements less than or equal to
x, I recursively sort the

00:04:19.000 --> 00:04:24.000
elements greater than or equal
to x.

00:04:24.000 --> 00:04:32.000
And then combine is then just
trivial.

00:04:32.000 --> 00:04:36.000
Because once I have sorted the
things less than or equal to x,

00:04:36.000 --> 00:04:40.000
then sorted the things greater
than or equal to x,

00:04:40.000 --> 00:04:44.000
the whole thing is sorted.
So, there is nothing to do

00:04:44.000 --> 00:04:48.000
really for the combine.
The key step in quicksort is

00:04:48.000 --> 00:04:52.000
this partition step.
That is the thing that does all

00:04:52.000 --> 00:04:55.000
of the work.
And so you can view quicksort

00:04:55.000 --> 00:04:57.000
of just as recursive
partitioning.

00:04:57.000 --> 00:05:06.000
That's all it is.
Just as mergesort was recursive

00:05:06.000 --> 00:05:18.000
merging, quicksort sort of goes
the other way around and does

00:05:18.000 --> 00:05:28.000
recursive partitioning.
The key is the linear time,

00:05:28.000 --> 00:05:40.000
by which I mean theta n,
partitioning subroutine.

00:05:40.000 --> 00:05:43.000
And here are some pseudocode
for it.

00:05:43.000 --> 00:05:47.000
This is actually slightly
different from the book.

00:05:47.000 --> 00:05:51.000
The book has one.
In fact, there is a nice

00:05:51.000 --> 00:05:55.000
problem in the book that has
even a different one,

00:05:55.000 --> 00:06:00.000
but they are all basically the
same idea.

00:06:00.000 --> 00:06:01.000
Partition (A,
p, q).

00:06:01.000 --> 00:06:06.000
And what we are looking at,
at this step of the recursion,

00:06:06.000 --> 00:06:11.000
is the subarray A from p to q.
And basically we pick a pivot,

00:06:11.000 --> 00:06:17.000
which is we are going to just
pick as the first element of the

00:06:17.000 --> 00:06:19.000
array A of p.

00:06:26.000 --> 00:06:29.000
And the book,
just for your information,

00:06:29.000 --> 00:06:32.000
uses A of q.
I use A of p.

00:06:32.000 --> 00:06:37.000
It doesn't really matter.
And then we set an index to p

00:06:37.000 --> 00:06:40.000
and then we have a loop.

00:07:35.000 --> 00:07:40.000
This is the code.
Basically the structure of it

00:07:40.000 --> 00:07:47.000
is a for loop with an "if"
statement in the middle.

00:07:47.000 --> 00:07:54.000
And so the structure of the
algorithm of this partitioning

00:07:54.000 --> 00:08:00.000
step looks as follows.
We set the pivot to be the

00:08:00.000 --> 00:08:05.000
first element.
Here is p and here is q.

00:08:05.000 --> 00:08:10.000
This is going to be our
invariant for the loop.

00:08:10.000 --> 00:08:15.000
And, at any time during the
execution of a loop,

00:08:15.000 --> 00:08:22.000
I essentially have some values
up to i which are already less

00:08:22.000 --> 00:08:28.000
than or equal to x and then some
values that end at j minus 1

00:08:28.000 --> 00:08:34.000
that are greater than or equal
to x.

00:08:34.000 --> 00:08:38.000
And then I don't know about the
rest.

00:08:38.000 --> 00:08:44.000
And so we start out with i
equal to p and j equal to p plus

00:00:01.000 --> 00:08:47.000
It starts out at p plus 1 so
This is called,
So, once again,

00:08:47.000 --> 00:08:52.000
that everything is unknown
except for x here.

00:08:52.000 --> 00:08:58.000
And then the idea is that it is
going to preserve this

00:08:58.000 --> 00:09:02.000
invariant.
And the way it does it is,

00:09:02.000 --> 00:09:06.000
as we go through the loop,
it looks at a of j and says is

00:09:06.000 --> 00:09:10.000
it greater than or equal to x?
Sorry, is it less than or equal

00:09:10.000 --> 00:09:12.000
to x?
If it is greater than or equal

00:09:12.000 --> 00:09:15.000
to x it does nothing,
because what can happen?

00:09:15.000 --> 00:09:19.000
If this is greater than or
equal to x, essentially it just

00:09:19.000 --> 00:09:23.000
goes to the next iterational
loop which moves this boundary

00:09:23.000 --> 00:09:27.000
and the invariant is satisfied.
Does everybody see that?

00:09:27.000 --> 00:09:31.000
Yeah, OK.
But if it is less than or equal

00:09:31.000 --> 00:09:35.000
to x, I have got a problem if I
want to maintain the invariant

00:09:35.000 --> 00:09:38.000
if this next element is less
than or equal to x.

00:09:38.000 --> 00:09:43.000
And so what it does then is it
says oh, let me just move this

00:09:43.000 --> 00:09:47.000
boundary and swap this element
here, which is greater than or

00:09:47.000 --> 00:09:51.000
equal to x, with this one here
that is less than or equal to x,

00:09:51.000 --> 00:09:55.000
thereby increasing the size of
this subarray and then the

00:09:55.000 --> 00:09:59.000
invariant is satisfied again.
It is a fairly simple

00:09:59.000 --> 00:10:03.000
algorithm.
And it is actually a very tight

00:10:03.000 --> 00:10:08.000
and easy algorithm.
That is one reason that this is

00:10:08.000 --> 00:10:13.000
such a great piece of code
because it is very efficient.

00:10:13.000 --> 00:10:18.000
Now, in principle,
the running time for this on n

00:10:18.000 --> 00:10:21.000
elements is order n.

00:10:28.000 --> 00:10:31.000
Because I am basically just
going through the n elements and

00:10:31.000 --> 00:10:35.000
just doing a constant amount of
work and then just a constant

00:10:35.000 --> 00:10:39.000
amount of work outside.
This is a clever piece of code.

00:10:39.000 --> 00:10:41.000
In fact, in principle partition
is easy, right?

00:10:41.000 --> 00:10:44.000
If I weren't worrying about
doing it in place,

00:10:44.000 --> 00:10:47.000
it is really a pretty easy
thing to do.

00:10:47.000 --> 00:10:50.000
I take an element and just
compare every other element with

00:10:50.000 --> 00:10:52.000
it.
I throw one into one bin and

00:10:52.000 --> 00:10:57.000
one into the other.
That is clearly linear time.

00:10:57.000 --> 00:11:01.000
But often what you find is that
just because you can do it that

00:11:01.000 --> 00:11:04.000
way theoretically doesn't mean
that that is going to end up

00:11:04.000 --> 00:11:08.000
giving you good code.
And this is a nice piece of

00:11:08.000 --> 00:11:10.000
code that allows you to do it in
place.

00:11:10.000 --> 00:11:14.000
And that is one reason why this
is a particularly good

00:11:14.000 --> 00:11:16.000
algorithm, because the constants
are good.

00:11:16.000 --> 00:11:20.000
So, yes, when we do asymptotic
analysis we tend to ignore the

00:11:20.000 --> 00:11:24.000
constants, but when you're
actually building code you care

00:11:24.000 --> 00:11:30.000
about the constants.
But first you care much more

00:11:30.000 --> 00:11:37.000
than just about the constants,
is whether overall it is going

00:11:37.000 --> 00:11:43.000
to be a fast algorithm.
Let's go through an example of

00:11:43.000 --> 00:11:50.000
this, I guess I will do it over
here, just so we get the gist.

00:11:50.000 --> 00:11:57.000
Here is a sample array that I
have created out of hallcloth.

00:11:57.000 --> 00:12:05.000
And here we are going to set x,
the pivot, to be 6.

00:12:05.000 --> 00:12:08.000
Let's look to see how this
algorithm works.

00:12:08.000 --> 00:12:12.000
So, i starts out here and j
starts out here if we

00:12:12.000 --> 00:12:15.000
initialize.
And what we do is start

00:12:15.000 --> 00:12:18.000
scanning right,
essentially that code is

00:12:18.000 --> 00:12:23.000
scanning right until it gets
something which is less than or

00:12:23.000 --> 00:12:27.000
equal to the pivot.
It keeps going here until it

00:12:27.000 --> 00:12:32.000
finds, j keeps incrementing
until it finds something that is

00:12:32.000 --> 00:12:38.000
less than or equal to the pivot.
And, in that case,

00:12:38.000 --> 00:12:44.000
it is the number 5.
Then it says we will swap these

00:12:44.000 --> 00:12:49.000
two things.
And it does that and we get 6,

00:12:49.000 --> 00:12:52.000
5, 13, 10, 8,
3, 2, 11.

00:12:52.000 --> 00:12:58.000
And meanwhile now i gets
incremented and j continues

00:12:58.000 --> 00:13:05.000
where it left off.
And so now we keep scanning

00:13:05.000 --> 00:13:12.000
right until we get to something
that is less than or equal to

00:13:12.000 --> 00:13:16.000
the pivot.
In this case it is 3.

00:13:16.000 --> 00:13:20.000
We swap 3 and 5 and get 6,
3, etc.

00:13:20.000 --> 00:13:27.000
And now, at this step we
increment i, we start j out

00:13:27.000 --> 00:13:30.000
here.
And in this case,

00:13:30.000 --> 00:13:35.000
right off the bat,
we have something which is less

00:13:35.000 --> 00:13:39.000
than or equal to x,
so we swap these two.

00:13:39.000 --> 00:13:42.000
I blew it, didn't I?
Oops.

00:13:42.000 --> 00:13:46.000
What did I do?
I swapped the wrong thing,

00:13:46.000 --> 00:13:48.000
didn't I, here?
Ah-ha.

00:13:48.000 --> 00:13:51.000
That is why I am not a
computer.

00:13:51.000 --> 00:13:54.000
Good.
We should have swapped this

00:13:54.000 --> 00:13:57.000
guy, right?
Swapped i plus 1,

00:13:57.000 --> 00:14:01.000
right?
This was i.

00:14:01.000 --> 00:14:03.000
We swap i plus 1,
good.

00:14:03.000 --> 00:14:09.000
So, that's all wrong.
Let's swap the right things.

00:14:09.000 --> 00:14:12.000
Now we have 6,
5, 3, 10, 8,

00:14:12.000 --> 00:14:16.000
13, 2, 11.
That even corresponds to my

00:14:16.000 --> 00:14:22.000
notes for some strange reason.
This is i and now this is j.

00:14:22.000 --> 00:14:28.000
And now when I look,
I immediately have something

00:14:28.000 --> 00:14:34.000
that is less than or equal to
the pivot.

00:14:34.000 --> 00:14:41.000
We swap this and i plus 1,
so now we have 6,

00:14:41.000 --> 00:14:45.000
5, 3, 2, 8, 13,
10, 11.

00:14:45.000 --> 00:14:52.000
And we, at that point,
increment i to here.

00:14:52.000 --> 00:15:03.000
And we have j now going here
and j runs to the end.

00:15:03.000 --> 00:15:08.000
And the loop terminates.
When the loop terminates there

00:15:08.000 --> 00:15:14.000
is one less swap that we do,
which is to put our pivot

00:15:14.000 --> 00:15:18.000
element in the middle between
the two subarrays.

00:15:18.000 --> 00:15:25.000
Here we swap this one and this
one, and so that gives us then

00:15:25.000 --> 00:15:27.000
2, 5, 3, 6, 8,
13, 10, 11.

00:15:27.000 --> 00:15:33.000
And this is the pivot.
And everything over here is

00:15:33.000 --> 00:15:40.000
less than or equal to the pivot.
And everything over here is

00:15:40.000 --> 00:15:44.000
greater than or equal to the
pivot.

00:15:56.000 --> 00:15:59.000
OK, so the quicksort routine.
Once we have this partition

00:15:59.000 --> 00:16:03.000
routine, quicksort is a pretty
easy piece of code to write.

00:16:18.000 --> 00:16:21.000
I should have said return here
i, right?

00:16:21.000 --> 00:16:24.000
You have got to return with the
pivot.

00:16:24.000 --> 00:16:29.000
Here I have got to return i
because we want to know where

00:16:29.000 --> 00:16:33.000
the pivot element is.
Sorry.

00:16:33.000 --> 00:16:46.000
I will plug in my code.
r gets partition of (A,

00:16:46.000 --> 00:17:03.000
p, q) and then we quicksort (A,
p, r-1) and quicksort of (A,

00:17:03.000 --> 00:17:09.000
r+1, q).
And that is it.

00:17:09.000 --> 00:17:17.000
That's the code.
The initial call is quicksort

00:17:17.000 --> 00:17:24.000
of (A, 1, n).
Because once we partitioned,

00:17:24.000 --> 00:17:31.000
we just have to quicksort the
two portions,

00:17:31.000 --> 00:17:39.000
the left and right portions.
Just the boundary case is

00:17:39.000 --> 00:17:41.000
probably worth mentioning for a
second.

00:17:41.000 --> 00:17:44.000
If there are zero or one
elements, that is basically what

00:17:44.000 --> 00:17:47.000
can possibly happen here,
is that I get zero or one

00:17:47.000 --> 00:17:49.000
elements here.
Then the point is there is

00:17:49.000 --> 00:17:52.000
nothing to do because the array
is sorted, either because it is

00:17:52.000 --> 00:17:56.000
an empty array or because it
only has one element.

00:17:56.000 --> 00:17:59.000
One of the tricks to making
quicksort go fast,

00:17:59.000 --> 00:18:01.000
as one tunes this,
is to, in fact,

00:18:01.000 --> 00:18:04.000
look at having a special
purpose sorting routine for

00:18:04.000 --> 00:18:07.000
small numbers of elements.
For example,

00:18:07.000 --> 00:18:10.000
if you get down to five
elements having some straight

00:18:10.000 --> 00:18:14.000
line piece of code that knows
how to sort five elements

00:18:14.000 --> 00:18:17.000
sufficiently as opposed to
continuing to go through

00:18:17.000 --> 00:18:20.000
recursion in order to accomplish
that.

00:18:20.000 --> 00:18:22.000
And there are a variety of
other things.

00:18:22.000 --> 00:18:27.000
This is a tail recursive code,
and so you can use certain tail

00:18:27.000 --> 00:18:31.000
recursion optimizations.
And there are a variety of

00:18:31.000 --> 00:18:35.000
other kinds of optimizations
that you can use to make this

00:18:35.000 --> 00:18:37.000
code go fast.
So, yeah, you can tune it up a

00:18:37.000 --> 00:18:40.000
bit beyond what is there,
but the core of it is this

00:18:40.000 --> 00:18:43.000
efficient partitioning routine.

00:18:53.000 --> 00:18:58.000
That is the algorithm.
It turns out that looking at

00:18:58.000 --> 00:19:05.000
how fast it runs is actually a
little bit challenging.

00:19:05.000 --> 00:19:09.000
In the analysis,
we are going to assume that all

00:19:09.000 --> 00:19:13.000
elements are distinct.
It turns out that this

00:19:13.000 --> 00:19:19.000
particular code does not work
very well when you have repeated

00:19:19.000 --> 00:19:25.000
elements, but Hoare's original
partitioning routine is actually

00:19:25.000 --> 00:19:31.000
more efficient in that case if
there are duplicates in what you

00:19:31.000 --> 00:19:36.000
are sorting.
And I encourage you to look at

00:19:36.000 --> 00:19:38.000
that.
It has a much more complicated

00:19:38.000 --> 00:19:42.000
invariant for partitioning
routine, but it does a similar

00:19:42.000 --> 00:19:45.000
kind of thing.
It's just a bit more

00:19:45.000 --> 00:19:48.000
complicated.
If they weren't all distinct,

00:19:48.000 --> 00:19:52.000
there are things you can do to
make them distinct or you can

00:19:52.000 --> 00:19:56.000
just use this code.
The easiest thing to do is just

00:19:56.000 --> 00:20:00.000
use Hoare's original code
because that works pretty well

00:20:00.000 --> 00:20:07.000
when they are nondistinct.
But this is a little bit easier

00:20:07.000 --> 00:20:12.000
to understand.
Let's let T(n) be the

00:20:12.000 --> 00:20:18.000
worst-case running time on n
elements.

00:20:18.000 --> 00:20:27.000
And so what is the worse-case?
What is the worse-case going to

00:20:27.000 --> 00:20:31.000
be for quicksort?

00:20:40.000 --> 00:20:43.000
That's right.
If you always pick the pivot

00:20:43.000 --> 00:20:47.000
and everything is greater than
or everything is less than,

00:20:47.000 --> 00:20:50.000
you are not going to partition
the array very well.

00:20:50.000 --> 00:20:54.000
And when does that happen?
What does the original input

00:20:54.000 --> 00:20:57.000
look like that makes that
happen?

00:20:57.000 --> 00:21:01.000
If it is already sorted or
reverse sorted.

00:21:01.000 --> 00:21:05.000
So, if the input is sorted or
reverse sorted.

00:21:05.000 --> 00:21:09.000
That is actually kind of
important to understand,

00:21:09.000 --> 00:21:14.000
because it turns out the most
common thing to sort is

00:21:14.000 --> 00:21:19.000
something that is already
sorted, surprisingly,

00:21:19.000 --> 00:21:22.000
or things that are nearly
sorted.

00:21:22.000 --> 00:21:27.000
But often it is just sorted and
somebody wants to make sure it

00:21:27.000 --> 00:21:33.000
is sorted.
Well, let's just sort it again

00:21:33.000 --> 00:21:38.000
rather than checking to see if
it is sorted.

00:21:38.000 --> 00:21:43.000
And, in those cases,
one side of the partition of

00:21:43.000 --> 00:21:50.000
each partition has no elements.
Then we can write out what the

00:21:50.000 --> 00:21:54.000
recursion is for that.
We have T(n).

00:21:54.000 --> 00:22:00.000
If one side has no elements,
we are going to have T(0) on

00:22:00.000 --> 00:22:06.000
that side.
And on the other side we are

00:22:06.000 --> 00:22:11.000
going to have T(n-1).
We are just writing out the

00:22:11.000 --> 00:22:16.000
recursion for this.
One side has no elements.

00:22:16.000 --> 00:22:19.000
The other side has n-1
elements.

00:22:19.000 --> 00:22:26.000
And then partitioning and all
the bookkeeping and so forth is

00:22:26.000 --> 00:22:30.000
order n.
What is T(0)?

00:22:30.000 --> 00:22:35.000
What is T(0)?
What is that asymptotically?

00:22:35.000 --> 00:22:38.000
It's a constant,
order 1.

00:22:38.000 --> 00:22:44.000
That is just order 1 + T(n-1) +
order n.

00:22:44.000 --> 00:22:50.000
Well, the order 1 can be
absorbed into the order n,

00:22:50.000 --> 00:22:59.000
so this is really just saying
it is T(n-1) + order n.

00:22:59.000 --> 00:23:08.000
And what is that equal to?
That is order n^2.

00:23:08.000 --> 00:23:19.000
Why is that order n^2?
It is an arithmetic series.

00:23:19.000 --> 00:23:30.000
Actually, just like we got for
insertion sort.

00:23:30.000 --> 00:23:35.000
Just like for insertion sort it
is an arithmetic series.

00:23:35.000 --> 00:23:42.000
Going through all that work and
we have an algorithm called

00:23:42.000 --> 00:23:47.000
quicksort, and it is no faster
than insertion sort.

00:23:47.000 --> 00:23:51.000
Nevertheless,
I said it was a good algorithm.

00:23:51.000 --> 00:23:57.000
The reason it is a good
algorithm is because its average

00:23:57.000 --> 00:24:04.000
case time, as we are going to
see, is very good.

00:24:04.000 --> 00:24:09.000
But let's try to understand
this a little bit more just so

00:24:09.000 --> 00:24:15.000
that we understand the
difference between what is going

00:24:15.000 --> 00:24:20.000
to happen in the average case
and what is going to happen in

00:24:20.000 --> 00:24:25.000
the worse-case.
Let's draw a recursion tree for

00:24:25.000 --> 00:24:31.000
this for T(n) = T(0) + T(n-1) +
and I will make the constant

00:24:31.000 --> 00:24:36.000
explicit for cn.
So, we get an intuition of what

00:24:36.000 --> 00:24:39.000
is going on.
Some constant times n.

00:24:39.000 --> 00:24:44.000
And then we have T(n) is equal
to, and we write it with the

00:24:44.000 --> 00:24:47.000
constant part here,
cn, and then T(0) here,

00:24:47.000 --> 00:24:51.000
and then T(n-1) here.
Now, I know that all you folks

00:24:51.000 --> 00:24:56.000
are really fast and want to jump
immediately to the full-blown

00:24:56.000 --> 00:24:59.000
tree.
But, let me tell you,

00:24:59.000 --> 00:25:04.000
my advice is that you spend
just a couple of minutes writing

00:25:04.000 --> 00:25:06.000
it out.
Since the tree grows

00:25:06.000 --> 00:25:10.000
exponentially,
it only costs you a constant

00:25:10.000 --> 00:25:14.000
overhead to write out the small
cases and make sure that you

00:25:14.000 --> 00:25:18.000
have got the pattern that you
are developing.

00:25:18.000 --> 00:25:21.000
So, I am going to go one more
step.

00:25:21.000 --> 00:25:26.000
Here we have T(0) and now this
becomes c(n-1) and now we have

00:25:26.000 --> 00:25:30.000
another T(0) over here and
T(n-2).

00:25:30.000 --> 00:25:35.000
And we continue that,
dot, dot, dot.

00:25:35.000 --> 00:25:45.000
That is all equal to cn with a
T(0) here, c(n-1) with a T(0),

00:25:45.000 --> 00:25:53.000
c(n-2), T(0) here,
and that goes all the way down

00:25:53.000 --> 00:26:01.000
until we end up with T(1) down
here.

00:26:01.000 --> 00:26:07.000
What is the height of this tree?

00:26:17.000 --> 00:26:20.000
What is the height of the tree
here?

00:26:20.000 --> 00:26:22.000
Yeah, n.
Good.

00:26:22.000 --> 00:26:30.000
Because every step we are just
decrementing the argument by 1.

00:26:30.000 --> 00:26:35.000
So, the height is n.
To analyze this,

00:26:35.000 --> 00:26:40.000
let's first add up everything
that is here.

00:26:40.000 --> 00:26:48.000
Just so we understand where
these things are coming from,

00:26:48.000 --> 00:26:56.000
this is just theta of the
summation of k equals 1 to n of

00:26:56.000 --> 00:27:02.000
k, actually of ck.
That is what is in there.

00:27:02.000 --> 00:27:08.000
And that is equal to order n^2.
That is where our algorithmatic

00:27:08.000 --> 00:27:12.000
series is coming from.
So, that is Theta(n^2).

00:27:12.000 --> 00:27:18.000
And then all of these things
here are all Theta(1).

00:27:23.000 --> 00:27:33.000
And how many of them are there?
There are n Theta(1)'s.

00:27:33.000 --> 00:27:54.000
So, the total amount is T(n) =
Theta(n) + Theta(n^2) =

00:27:54.000 --> 00:28:03.000
Theta(n^2).
Just to see what the structure

00:28:03.000 --> 00:28:09.000
is in terms of the recursion
tree, it is a highly unbalanced

00:28:09.000 --> 00:28:14.000
recursion tree.
Now I am going to do something

00:28:14.000 --> 00:28:20.000
that I told you should never do,
which is we are going to be do

00:28:20.000 --> 00:28:25.000
a best-case analysis.
This is for intuition only.

00:28:25.000 --> 00:28:32.000
And, in general,
we don't do best-case analyses.

00:28:32.000 --> 00:28:35.000
It doesn't mean anything,
unless we get some intuition

00:28:35.000 --> 00:28:38.000
for it maybe.
But basically it means nothing

00:28:38.000 --> 00:28:43.000
mathematically because it's
providing no guarantee.

00:28:55.000 --> 00:29:00.000
And so this is intuition only.

00:29:05.000 --> 00:29:13.000
If we are really lucky what
happens for partition?

00:29:13.000 --> 00:29:20.000
What is going to be the lucky
case?

00:29:26.000 --> 00:29:28.000
Yeah, it splits right in the
middle.

00:29:28.000 --> 00:29:31.000
Which is essentially --

00:29:43.000 --> 00:29:46.000
-- n/2 : n/2.
It is really (n-1)/2 :

00:29:46.000 --> 00:29:51.000
(n-1)/2, but we're not going to
worry about the details because

00:29:51.000 --> 00:29:56.000
we're only doing intuition for
the best-case because best-case

00:29:56.000 --> 00:30:01.000
is not what we want.
If that happened,

00:30:01.000 --> 00:30:11.000
what is the recurrence I get?
Imagine it split it exactly in

00:30:11.000 --> 00:30:18.000
the middle every time,
then what happens?

00:30:18.000 --> 00:30:26.000
You get T(n) = 2T(n/2) + order
n for partitioning and

00:30:26.000 --> 00:30:33.000
bookkeeping.
And what is the solution of

00:30:33.000 --> 00:30:36.000
that recurrence?
That is n log n.

00:30:36.000 --> 00:30:42.000
That is the same as the merge
sort recurrence.

00:30:42.000 --> 00:30:46.000
It is which case of the master
theorem?

00:30:46.000 --> 00:30:51.000
Case 2, right?
Because n to the log base 2 of

00:30:51.000 --> 00:30:55.000
2 is n to the 1,
it is the same,

00:30:55.000 --> 00:31:05.000
so we tack on the extra log n.
Case 2 of the master theorem.

00:31:05.000 --> 00:31:14.000
That is pretty good.
That says that in the best-case

00:31:14.000 --> 00:31:24.000
quicksort is going to do well.
How about let's suppose the

00:31:24.000 --> 00:31:32.000
split is always let's say 1/10 :
9/10, 1/10n :

00:31:32.000 --> 00:31:35.000
9/10n.
In that case,

00:31:35.000 --> 00:31:43.000
are we lucky or are we unlucky?

00:31:52.000 --> 00:31:57.000
I mean, if the split is really
skewed, we clearly are going to

00:31:57.000 --> 00:32:00.000
be unlucky, right,
because then it's,

00:32:00.000 --> 00:32:04.000
say, 1 to n.
If it is really in the middle

00:32:04.000 --> 00:32:08.000
it is n log n.
What do you suppose it is if it

00:32:08.000 --> 00:32:11.000
is 1/10 : 9/10?
Is that lucky or unlucky?

00:32:11.000 --> 00:32:14.000
We will have a little democracy
here.

00:32:14.000 --> 00:32:17.000
Who thinks that that is a lucky
case?

00:32:17.000 --> 00:32:20.000
It is going to be fast running
time.

00:32:20.000 --> 00:32:23.000
And who thinks it is an unlucky
case?

00:32:23.000 --> 00:32:26.000
OK, so we have some brave
souls.

00:32:26.000 --> 00:32:30.000
And who didn't vote?
Oh, come on.

00:32:30.000 --> 00:32:32.000
Come on.
It is always better,

00:32:32.000 --> 00:32:35.000
by the way, to say yes or no
and be right or wrong,

00:32:35.000 --> 00:32:40.000
because then you have some
emotional commitment to it and

00:32:40.000 --> 00:32:43.000
we will remember better,
rather than just sitting and

00:32:43.000 --> 00:32:47.000
being quiet.
You don't manipulate your own

00:32:47.000 --> 00:32:50.000
emotions well enough to remember
things well.

00:32:50.000 --> 00:32:54.000
Those people who voted win over
the people who don't vote,

00:32:54.000 --> 00:32:56.000
whether they are right or
wrong.

00:32:56.000 --> 00:33:02.000
Well, let's take a look.
Here is the recurrence.

00:33:02.000 --> 00:33:07.000
T(n) = T(1/10n) + T(9/10n) +
Theta(n).

00:33:07.000 --> 00:33:15.000
And we will assume that this
part here is less than or equal

00:33:15.000 --> 00:33:19.000
to some cn in order to analyze
it.

00:33:19.000 --> 00:33:25.000
We will just do a recursion
tree for this and see.

00:33:25.000 --> 00:33:30.000
Here is a recursion tree.

00:33:35.000 --> 00:33:41.000
We have T(n) = cn,
T(1/10n), T(9/10n).

00:33:41.000 --> 00:33:46.000
Now we have again cn at the
top.

00:33:46.000 --> 00:33:51.000
This gets complicated,
right?

00:33:51.000 --> 00:34:00.000
This is 1/10cn.
Now, over here we have 1/10.

00:34:00.000 --> 00:34:09.000
And then we are plugging it
into the recursion again,

00:34:09.000 --> 00:34:20.000
so we now get T(1/100n) and
over here we get T(9/100n).

00:34:20.000 --> 00:34:26.000
And over here we have now
9/10cn.

00:34:26.000 --> 00:34:34.000
And that gives us T(9/100n)
again.

00:34:34.000 --> 00:34:48.000
And here we get T(81/100n).
And we keep going on.

00:34:48.000 --> 00:34:59.000
That is equal to cn,
1/10cn here.

00:34:59.000 --> 00:35:08.000
Down this way we have 1/100cn.
And that keeps going down until

00:35:08.000 --> 00:35:16.000
we get to order 1 down here.
And over here we have 9/10cn.

00:35:16.000 --> 00:35:24.000
And here, let's see,
this is 9/100cn and this is now

00:35:24.000 --> 00:35:32.000
9/100cn and this is 81/100cn.
And these things keep going

00:35:32.000 --> 00:35:36.000
down until they get down to
order 1.

00:35:36.000 --> 00:35:41.000
But the leaves are not all at
uniform depth here,

00:35:41.000 --> 00:35:45.000
right?
This side is way further up

00:35:45.000 --> 00:35:47.000
than this side,
right?

00:35:47.000 --> 00:35:53.000
Because here we are only going
down by 9/10 each time.

00:35:53.000 --> 00:35:58.000
So, in fact,
what is the length of this path

00:35:58.000 --> 00:36:02.000
here?
What is the length of this path

00:36:02.000 --> 00:36:06.000
down to this,
if I take the left most spine?

00:36:15.000 --> 00:36:23.000
Somebody raise there hand.
Yeah?

00:36:23.000 --> 00:36:30.000
Log base 10 of n.
Because I am basically cutting

00:36:30.000 --> 00:36:33.000
down by a factor of 10 each
time.

00:36:33.000 --> 00:36:35.000
And how long does it take me to
reduce it to 1?

00:36:35.000 --> 00:36:38.000
That is the definition,
if you will,

00:36:38.000 --> 00:36:40.000
of what a log is,
log base 10.

00:36:40.000 --> 00:36:43.000
What is this one?
What is this path going that

00:36:43.000 --> 00:36:44.000
way?

00:36:53.000 --> 00:37:02.000
Log of n. Log base 10/9 of n.
Because we're going down by

00:37:02.000 --> 00:37:05.000
9/10 each time.
Once again, essentially the

00:37:05.000 --> 00:37:09.000
definition of n.
And everything in between there

00:37:09.000 --> 00:37:14.000
is somewhere between log base 10
of n and log base 10/9 of n.

00:37:14.000 --> 00:37:17.000
So, everything is in between
there.

00:37:17.000 --> 00:37:22.000
Now what I can do is do the
trick that we did for mergesort

00:37:22.000 --> 00:37:26.000
in looking at what the
evaluation of this is by adding

00:37:26.000 --> 00:37:31.000
up what is the cost of the total
level.

00:37:31.000 --> 00:37:36.000
That is just cn.
What is the cost of the next

00:37:36.000 --> 00:37:38.000
level?
cn.

00:37:38.000 --> 00:37:43.000
And what is the cost of the
next level?

00:37:43.000 --> 00:37:47.000
cn.
Every level we are still doing

00:37:47.000 --> 00:37:54.000
the same amount of work.
And we take that all the way

00:37:54.000 --> 00:38:00.000
down.
And the last levels --

00:38:00.000 --> 00:38:04.000
Eventually we hit some point
where it is not equal to cn

00:38:04.000 --> 00:38:09.000
where we start getting things
that are less than or equal to

00:38:09.000 --> 00:38:14.000
cn because some of the leaves
start dropping out starting at

00:38:14.000 --> 00:38:17.000
this level.
Basically this part is going to

00:38:17.000 --> 00:38:21.000
be log base 10 of n,
and then we start getting

00:38:21.000 --> 00:38:25.000
things that are less than or
equal to cn, and so forth,

00:38:25.000 --> 00:38:30.000
until finally we get to add it
all up.

00:38:30.000 --> 00:38:36.000
T(n) is going to be less than
or equal to cn times,

00:38:36.000 --> 00:38:43.000
well, what is the longest that
this could possibly be?

00:38:43.000 --> 00:38:49.000
Log base 10/9 of n.
Plus we have all of the leaves

00:38:49.000 --> 00:38:56.000
that we have to add in,
but all the leaves together add

00:38:56.000 --> 00:39:03.000
up to just order n.
All the leaves add up to order

00:39:03.000 --> 00:39:08.000
n, so we have + Theta(n).
And so this is how much?

00:39:08.000 --> 00:39:14.000
If I add all of this together,
what is this asymptotically?

00:39:14.000 --> 00:39:18.000
That is n log n.
So, T(n) is actually bounded by

00:39:18.000 --> 00:39:21.000
n log n.
We are lucky.

00:39:21.000 --> 00:39:25.000
Those people who guessed lucky
were right.

00:39:25.000 --> 00:39:30.000
A 1/10 : 9/10 split is
asymptotically as good as a 50 :

00:39:30.000 --> 00:39:34.000
50 split.
And, in fact,

00:39:34.000 --> 00:39:41.000
we can lower bound this by just
looking at these things here and

00:39:41.000 --> 00:39:45.000
discover that,
in fact, T(n) is lower bounded

00:39:45.000 --> 00:39:49.000
by cn log base 10 of n + order
n.

00:39:49.000 --> 00:39:55.000
And so T(n) is lower bounded by
also asymptotically n log n.

00:39:55.000 --> 00:40:00.000
So, T(n) is actually Theta(n lg
n).

00:40:00.000 --> 00:40:04.000
Now, this is not really proof.
I generally recommend that you

00:40:04.000 --> 00:40:07.000
don't do this kind of thing to
do a proof.

00:40:07.000 --> 00:40:10.000
This is a good intuition of a
recursion tree.

00:40:10.000 --> 00:40:14.000
The way you prove this is what?
Substitution method.

00:40:14.000 --> 00:40:17.000
Good.
What you do is use this to get

00:40:17.000 --> 00:40:20.000
your guess and then use
substitution method to prove

00:40:20.000 --> 00:40:25.000
that your guess is right.
It is too easy to make mistakes

00:40:25.000 --> 00:40:28.000
with this method.
It is very easy to make

00:40:28.000 --> 00:40:32.000
mistakes.
With the substitution method it

00:40:32.000 --> 00:40:37.000
is harder to make mistakes
because there is just algebra

00:40:37.000 --> 00:40:40.000
there that you are cranking
through.

00:40:40.000 --> 00:40:43.000
It is easier to verify rather
than dot, dot,

00:40:43.000 --> 00:40:48.000
dots and trees that you drew
improperly and wrote in wrong

00:40:48.000 --> 00:40:50.000
amounts and so forth.
OK?

00:40:50.000 --> 00:40:53.000
So, this is n log n.
That's pretty good.

00:40:53.000 --> 00:40:56.000
It is order n log n.
And we are lucky.

00:40:56.000 --> 00:41:01.000
Now let's try another one.
This is all for intuition

00:41:01.000 --> 00:41:05.000
because, I will tell you,
by the time we get to the end

00:41:05.000 --> 00:41:10.000
of this class you folks are
going to bolting for the door

00:41:10.000 --> 00:41:13.000
because we are going to do some
good math today,

00:41:13.000 --> 00:41:16.000
actually.
It is actually fun math,

00:41:16.000 --> 00:41:20.000
I think, but it is challenging.
If you are not awake,

00:41:20.000 --> 00:41:23.000
you can still sleep now,
but I will tell you when to

00:41:23.000 --> 00:41:26.000
wake up.
One more bit of intuition.

00:41:26.000 --> 00:41:30.000
Suppose that we alternate
steps.

00:41:30.000 --> 00:41:33.000
Suppose we do the partitioning
thing.

00:41:33.000 --> 00:41:38.000
And it happens that we start
out lucky and then we have a

00:41:38.000 --> 00:41:43.000
partitioning step that is
unlucky and then we have a step

00:41:43.000 --> 00:41:49.000
that is lucky and a step that is
unlucky and we do that all the

00:41:49.000 --> 00:41:54.000
way down the tree.
Suppose we alternate.

00:42:07.000 --> 00:42:09.000
Are we lucky or unlucky if we
do that?

00:42:09.000 --> 00:42:11.000
This time I want everybody
voting.

00:42:11.000 --> 00:42:13.000
It doesn't matter what your
answer is.

00:42:13.000 --> 00:42:16.000
Everybody has to have a stake
in the game.

00:42:16.000 --> 00:42:19.000
It is sort of like horseracing.
If ever you have watched

00:42:19.000 --> 00:42:23.000
horseracing, it is really
boring, but if you put a little

00:42:23.000 --> 00:42:26.000
bit of money down,
a little skin in the game

00:42:26.000 --> 00:42:30.000
suddenly it is interesting.
The same thing here.

00:42:30.000 --> 00:42:33.000
I want everybody to put some
skin in the game.

00:42:33.000 --> 00:42:37.000
Who thinks that this is going
to be lucky?

00:42:37.000 --> 00:42:40.000
Who thinks it is going to be
unlucky?

00:42:40.000 --> 00:42:42.000
OK.
Who didn't vote?

00:42:42.000 --> 00:42:46.000
[LAUGHTER] You guys.
No skin in the game,

00:42:46.000 --> 00:42:48.000
ha?
Let's analyze this so we can

00:42:48.000 --> 00:42:53.000
once again write a recurrence.
On the lucky step,

00:42:53.000 --> 00:42:57.000
we will have L(n) be the
running time on a lucky step of

00:42:57.000 --> 00:43:02.000
size n.
And that is going to be twice.

00:43:02.000 --> 00:43:07.000
While the next step is going to
be unlucky.

00:43:07.000 --> 00:43:10.000
It is two unluckies over 2 plus
order n.

00:43:10.000 --> 00:43:15.000
That is our lucky step.
And then for the unlucky step

00:43:15.000 --> 00:43:20.000
it is essentially going to be L
of n minus 1,

00:43:20.000 --> 00:43:25.000
it is going to be lucky on the
next step, plus order n.

00:43:25.000 --> 00:43:30.000
That is unlucky.
See how I have described this

00:43:30.000 --> 00:43:36.000
behavior with a system now of
recurrences that are dependent

00:43:36.000 --> 00:43:41.000
where the boundary cases,
once again which are unstated,

00:43:41.000 --> 00:43:47.000
is that the recurrences have a
constant solution with constant

00:43:47.000 --> 00:43:50.000
input.
Now we just do a little bit of

00:43:50.000 --> 00:43:54.000
algebra using substitution.
L(n) is then equal to,

00:43:54.000 --> 00:44:00.000
well, I can just plug in,
for U(n/2) plug in the value of

00:44:00.000 --> 00:44:06.000
U(n/2).
And that gives me 2[L(n/2-1) +

00:44:06.000 --> 00:44:11.000
Theta(n) + Theta(n)].
See what I did here?

00:44:11.000 --> 00:44:18.000
I simply plugged in,
for U(n/2), this recurrence.

00:44:18.000 --> 00:44:27.000
In fact, technically I guess I
should have said Theta(n/2) just

00:44:27.000 --> 00:44:35.000
to make this substitution more
straightforward.

00:44:35.000 --> 00:44:41.000
It is the same thing,
but just to not skip a step.

00:44:41.000 --> 00:44:49.000
That we can now crank through.
And that is 2L(n/2 - 1) +,

00:44:49.000 --> 00:44:57.000
and now I have two T(n/2) plus
another one, so all of that is

00:44:57.000 --> 00:45:04.000
just order n.
And what is the solution to

00:45:04.000 --> 00:45:07.000
that recurrence?
n log n.

00:45:07.000 --> 00:45:13.000
Theta(n lg n).
Does everybody see that?

00:45:13.000 --> 00:45:15.000
OK?
Theta(n lg n).

00:45:15.000 --> 00:45:24.000
This is basically just,
once again, master theorem with

00:45:24.000 --> 00:45:33.000
a little bit of jiggering here.
That minus one is only going to

00:45:33.000 --> 00:45:38.000
help us, actually,
in the solution of the master

00:45:38.000 --> 00:45:41.000
theorem.
So, it is order n lg n.

00:45:41.000 --> 00:45:45.000
We are lucky.
If we alternate lucky and

00:45:45.000 --> 00:45:50.000
unlucky, we are lucky.
How can we insure that we are

00:45:50.000 --> 00:45:55.000
usually lucky?
If I have the input already

00:45:55.000 --> 00:46:00.000
sorted, I am going to be
unlucky.

00:46:00.000 --> 00:46:03.000
Excuse me?
You could randomly arrange the

00:46:03.000 --> 00:46:07.000
elements, that is one way.
What is another way?

00:46:07.000 --> 00:46:10.000
That is a perfectly good way,
actually.

00:46:10.000 --> 00:46:13.000
In fact, it is a common thing
to do.

00:46:13.000 --> 00:46:16.000
Randomly choose the pivot,
OK.

00:46:16.000 --> 00:46:20.000
It turns out those are
effectively equivalent,

00:46:20.000 --> 00:46:24.000
but we are going to do the
randomly choose the pivot

00:46:24.000 --> 00:46:30.000
because it is a little bit
easier to analyze.

00:46:30.000 --> 00:46:33.000
But they are effectively
equivalent.

00:46:33.000 --> 00:46:39.000
That gives us the algorithm
called randomized quicksort.

00:46:46.000 --> 00:46:53.000
And the nice thing about
randomized quicksort is that the

00:46:53.000 --> 00:47:00.000
running time is independent of
the input ordering.

00:47:00.000 --> 00:47:04.000
Very much for the same reason
that if I just scramble the

00:47:04.000 --> 00:47:08.000
input, it would be independent
of the input ordering.

00:47:08.000 --> 00:47:12.000
If I randomly scramble the
input then it doesn't matter

00:47:12.000 --> 00:47:17.000
what the order of the input was.
Whereas, original quicksort has

00:47:17.000 --> 00:47:21.000
some slow cases,
input sorted or reverse sorted,

00:47:21.000 --> 00:47:24.000
and some fast cases.
In particular,

00:47:24.000 --> 00:47:28.000
it turns out that if it is
random it is going to be pretty

00:47:28.000 --> 00:47:31.000
fast.
If I actually randomly scramble

00:47:31.000 --> 00:47:35.000
the input or pivot on a random
element, it doesn't matter what

00:47:35.000 --> 00:47:38.000
the input was.
One way of thinking about this

00:47:38.000 --> 00:47:40.000
is with an adversary.
Imagine your adversary,

00:47:40.000 --> 00:47:44.000
you are saying I have a good
sorting algorithm and he says I

00:47:44.000 --> 00:47:47.000
have a good sorting algorithm
and you're trying to sell to a

00:47:47.000 --> 00:47:50.000
single customer.
And the customer says OK,

00:47:50.000 --> 00:47:53.000
you guys come up with
benchmarks for each of your

00:47:53.000 --> 00:47:55.000
algorithms.
And you get to look at his

00:47:55.000 --> 00:47:59.000
algorithm.
Well, you look and you say oh,

00:47:59.000 --> 00:48:02.000
he is using quicksort.
I will just give him something

00:48:02.000 --> 00:48:06.000
that is already sorted.
That is what you could do to

00:48:06.000 --> 00:48:08.000
him.
If you had quicksort,

00:48:08.000 --> 00:48:10.000
he would do the same thing to
you.

00:48:10.000 --> 00:48:14.000
So, how can you defeat him?
Well, one way is with

00:48:14.000 --> 00:48:17.000
randomization.
Big idea in computer science,

00:48:17.000 --> 00:48:20.000
use random numbers.
The idea here is if I permute

00:48:20.000 --> 00:48:23.000
the ordering at random,
as one suggestion,

00:48:23.000 --> 00:48:28.000
or I pivot at random places,
then the input ordering didn't

00:48:28.000 --> 00:48:32.000
matter.
And so there is no bad ordering

00:48:32.000 --> 00:48:36.000
that he can provide that is
going to make my code run

00:48:36.000 --> 00:48:39.000
slowly.
Now, I might get unlucky.

00:48:39.000 --> 00:48:43.000
But that is just unlucky in my
use of my random-number

00:48:43.000 --> 00:48:46.000
generator.
It is not unlucky with respect

00:48:46.000 --> 00:48:50.000
to what the input was.
What the input was doesn't

00:48:50.000 --> 00:48:52.000
matter.
Everybody follow that?

00:48:52.000 --> 00:48:55.000
OK.
The nice thing about randomized

00:48:55.000 --> 00:48:59.000
quicksort is that it makes no
assumptions about the input

00:48:59.000 --> 00:49:05.000
distribution.
You don't have to assume that

00:49:05.000 --> 00:49:13.000
all inputs are equally likely
because either you can make it

00:49:13.000 --> 00:49:20.000
that way or you pivot in a way
that makes that effectively

00:49:20.000 --> 00:49:23.000
whole.
And, in particular,

00:49:23.000 --> 00:49:30.000
there is no specific input that
can elicit the worst-case

00:49:30.000 --> 00:49:45.000
behavior.
The worst-case is determined

00:49:45.000 --> 00:50:00.000
only by a random-number
generator.

00:50:00.000 --> 00:50:03.000
And, therefore,
since it is only determined by

00:50:03.000 --> 00:50:08.000
a random-number generator,
we can essentially bound the

00:50:08.000 --> 00:50:13.000
unluckiness mathematically.
We can say what are the odds?

00:50:13.000 --> 00:50:16.000
So, we are going to analyze
this.

00:50:16.000 --> 00:50:21.000
And this is where you know if
you belong in this course or

00:50:21.000 --> 00:50:24.000
not.
If you have skipped 6.042 or

00:50:24.000 --> 00:50:30.000
whatever, this is a good place
to do the comparison.

00:50:30.000 --> 00:50:34.000
Since it is going to be a
little bit, why don't people

00:50:34.000 --> 00:50:37.000
just stand up for a moment and
take a stretch break.

00:50:37.000 --> 00:50:42.000
Since this is going to be a
nice piece of mathematics we are

00:50:42.000 --> 00:50:47.000
going to do, you are going to
want to feel fresh for it.

00:51:02.000 --> 00:51:04.000
Stretch break is over.

00:51:10.000 --> 00:51:11.000
Analysis.
Good.

00:51:11.000 --> 00:51:16.000
I think we are going to make
this.

00:51:16.000 --> 00:51:22.000
I am sort of racing.
There is a lot of stuff to

00:51:22.000 --> 00:51:24.000
cover today.
Good.

00:51:24.000 --> 00:51:32.000
Let's let T(n) now be the
random variable for the running

00:51:32.000 --> 00:51:36.000
time assuming --

00:51:46.000 --> 00:51:47.000
Wow.
I didn't even write here what

00:51:47.000 --> 00:51:49.000
we did here.
So, we are going to pivot on a

00:51:49.000 --> 00:51:51.000
random element.

00:51:59.000 --> 00:52:01.000
That is the basic scheme we are
going to do.

00:52:01.000 --> 00:52:05.000
And the way I do that,
by the way, is just in the code

00:52:05.000 --> 00:52:07.000
for partition,
rather than partitioning on the

00:52:07.000 --> 00:52:10.000
first element,
before I do the partition,

00:52:10.000 --> 00:52:14.000
I just swap the first element
with some other element in the

00:52:14.000 --> 00:52:16.000
array chosen at random,
perhaps itself.

00:52:16.000 --> 00:52:19.000
So, they are all equally likely
to be pivoted on.

00:52:19.000 --> 00:52:23.000
And then just run the ordinary
partition.

00:52:23.000 --> 00:52:27.000
This is a random variable for
running in time assuming,

00:52:27.000 --> 00:52:32.000
we have to make an assumption
for doing probability,

00:52:32.000 --> 00:52:36.000
the random numbers are
independent.

00:52:41.000 --> 00:52:44.000
So that when I pivot in one
place, it is independent of how

00:52:44.000 --> 00:52:48.000
I pivoted in some other place as
I am running this algorithm.

00:52:48.000 --> 00:52:52.000
Then, to analyze this,
what I am going to do is I want

00:52:52.000 --> 00:52:57.000
to know where we pivoted.
For k = 0, 1,

00:52:57.000 --> 00:53:08.000
..., n-1, let's let,
for a particular partition,

00:53:08.000 --> 00:53:21.000
the random variable X_k = 1 if
partition generates a k :

00:53:21.000 --> 00:53:30.000
n-k-1 split,
and 0 otherwise.

00:53:35.000 --> 00:53:39.000
In the partition routine,
I am picking a random element

00:53:39.000 --> 00:53:42.000
to pivot on.
And X_k is going to be my

00:53:42.000 --> 00:53:47.000
random variable that is 1 if it
generates a split that has k

00:53:47.000 --> 00:53:52.000
elements on the left side and
n-k-1 elements on the right side

00:53:52.000 --> 00:53:54.000
of the pivot.
Some of those,

00:53:54.000 --> 00:54:00.000
too, of course are n-1 because
I also have the pivot.

00:54:00.000 --> 00:54:03.000
And 0 otherwise.
So, I now have n random

00:54:03.000 --> 00:54:09.000
variables that I have defined
associated with a single

00:54:09.000 --> 00:54:15.000
partition where all of them are
going to be zero except one of

00:54:15.000 --> 00:54:21.000
them, whichever one happens to
occur is going to have the value

00:54:22.000 --> 00:54:26.000
by the way.
What is the name of this type

00:54:26.000 --> 00:54:30.000
of random variable?

00:54:37.000 --> 00:54:40.000
Bernoulli.
Well, Bernoulli has other

00:54:40.000 --> 00:54:43.000
assumptions.
It is an indicator random

00:54:43.000 --> 00:54:46.000
variable.
It turns out it is Bernoulli,

00:54:46.000 --> 00:54:50.000
but that's OK.
It is an indicator random

00:54:50.000 --> 00:54:53.000
variable.
It just takes on the value of

00:54:53.000 --> 00:54:56.000
0, 1.
And Bernoulli random variables

00:54:56.000 --> 00:55:02.000
are a particular type of
indicator random variable.

00:55:02.000 --> 00:55:06.000
Which it turns out these are.
That is an indicator random

00:55:06.000 --> 00:55:09.000
variable.
Indicator random variables are

00:55:09.000 --> 00:55:14.000
a good way when you are trying
to understand what the sum of a

00:55:14.000 --> 00:55:18.000
bunch of things is.
It is a good way to break apart

00:55:18.000 --> 00:55:22.000
your big random variables into
smaller ones that can be

00:55:22.000 --> 00:55:25.000
analyzed.
Let's just take a look at this

00:55:25.000 --> 00:55:30.000
indicator random variable.
What is the expectation of X_k

00:55:30.000 --> 00:55:32.000
equal to?

00:55:42.000 --> 00:55:45.000
In other words,
what is the probability that I

00:55:45.000 --> 00:55:48.000
generate a k :
n-k-1 split?

00:56:00.000 --> 00:56:05.000
X_k is, let's just write out
what that means,

00:56:05.000 --> 00:56:08.000
just to refresh people's
memory.

00:56:08.000 --> 00:56:15.000
That is 0 times the probability
that X_k equals 0 plus 1 times

00:56:15.000 --> 00:56:21.000
the probability that X_k equals
1, which is equal,

00:56:21.000 --> 00:56:27.000
well, that is all zero.
That is just equal to the

00:56:27.000 --> 00:56:34.000
probability that X_k equals 1.
And that is a general property

00:56:34.000 --> 00:56:39.000
of indicator random variables,
is that their expectation is

00:56:39.000 --> 00:56:45.000
the probability that they are 1.
The nice thing about indicator

00:56:45.000 --> 00:56:51.000
random variables is it directly
connects the probability to the

00:56:51.000 --> 00:56:55.000
expectation without any other
terms going on.

00:56:55.000 --> 00:56:58.000
What is the probability of X_k
equals 1?

00:56:58.000 --> 00:57:02.000
1/n.
So, all splits are equally

00:57:02.000 --> 00:57:04.000
likely.
And I have n elements,

00:57:04.000 --> 00:57:10.000
so each ones has a 1/n chance
of being picked as the pivot.

00:57:10.000 --> 00:57:15.000
And, once you pick the pivot,
that determines what is on the

00:57:15.000 --> 00:57:19.000
left and the right and so forth.
So, it is 1/n.

00:57:19.000 --> 00:57:23.000
Everybody with me so far?
More or less?

00:57:23.000 --> 00:57:26.000
OK.
As I say, this is going to test

00:57:26.000 --> 00:57:32.000
whether you're in the class.
If you go home and you study

00:57:32.000 --> 00:57:37.000
this and you cannot get it,
and you have a deficiency in

00:57:37.000 --> 00:57:41.000
your math background in trying
to take the course,

00:57:41.000 --> 00:57:46.000
this is a good indication that
probably you have taken

00:57:46.000 --> 00:57:49.000
something a little over your
head.

00:57:49.000 --> 00:57:54.000
Let's write out what T(n) is
equal to here.

00:58:00.000 --> 00:58:11.000
T(n) is going to be equal to
T(0) + T(n-1) + Theta(n) if we

00:58:11.000 --> 00:58:24.000
get a 0 : n-1 split and is equal
to T(1) + T(n-2) + order n if we

00:58:24.000 --> 00:58:30.000
have a 1 : n-2 split.

00:58:35.000 --> 00:58:46.000
And now down here it is going
to be T(n-1) + T(0) + Theta(n)

00:58:46.000 --> 00:58:52.000
if we end up with an n-1 :
0 split.

00:58:52.000 --> 00:59:00.000
So, this is our recurrence for
T(n).

00:59:00.000 --> 00:59:03.000
And, unfortunately,
the recurrence is kind of hairy

00:59:03.000 --> 00:59:06.000
because it has got n cases.
And this is,

00:59:06.000 --> 00:59:11.000
once again, where the
brilliance of being able to use

00:59:11.000 --> 00:59:13.000
indicator random variables comes
in.

00:59:13.000 --> 00:59:18.000
Because we will be able to take
this case analysis and reduce it

00:59:18.000 --> 00:59:23.000
to mathematics so we don't have
cases using indicator random

00:59:23.000 --> 00:59:29.000
variables.
And the way we do that is using

00:59:29.000 --> 00:59:37.000
the following trick of
converting the cases into a

00:59:37.000 --> 00:59:39.000
summation.

00:59:55.000 --> 01:00:00.666
Let's just take a look at why
these two things are the same.

01:00:00.666 --> 01:00:06.333
The indicator random variable
is zero, except if you get the

01:00:06.333 --> 01:00:10.655
particular split.
Therefore, this summation is

01:00:10.655 --> 01:00:14.497
going to be zero,
except for that k which

01:00:14.497 --> 01:00:20.355
actually appeared in which case
it is the value that we say it

01:00:20.355 --> 01:00:22.468
is.
See the trick using

01:00:22.468 --> 01:00:27.079
multiplication by 0,
1 variable to handle all the

01:00:27.079 --> 01:00:31.404
cases?
I think that is damn clever.

01:00:31.404 --> 01:00:36.213
I think that is damn clever.
And this is like the classic

01:00:36.213 --> 01:00:40.421
thing that you do with indicator
random variables.

01:00:40.421 --> 01:00:45.144
It's one of the reasons they
are a very powerful method.

01:00:45.144 --> 01:00:49.781
Because now we actually have a
mathematical expression,

01:00:49.781 --> 01:00:53.559
hairy although it may be,
for our recurrence.

01:00:53.559 --> 01:00:58.454
Now, what we are going to
analyze is the expected value of

01:00:58.454 --> 01:01:02.977
T(n).
That is what we want to do.

01:01:02.977 --> 01:01:06.727
What is the expected value of
T(n)?

01:01:06.727 --> 01:01:13.235
To do that, I just write the
expected value of T(n) is equal

01:01:13.235 --> 01:01:17.977
to the expected value of this
big summation.

01:01:17.977 --> 01:01:24.264
And now we can go ahead and
start to evaluate the expected

01:01:24.264 --> 01:01:30.000
value of that summation.
Everybody with me?

01:01:30.000 --> 01:01:32.486
Yes?
Any questions at this point?

01:01:32.486 --> 01:01:35.360
I see a thumbs up.
That's nice to see.

01:01:35.360 --> 01:01:39.944
But I generally believe that
what I want to see is no thumbs

01:01:39.944 --> 01:01:42.508
down.
It is good to see the thumbs

01:01:42.508 --> 01:01:46.704
up, but that means one person
understands, or thinks he

01:01:46.704 --> 01:01:48.724
understands.
[LAUGHTER] So,

01:01:48.724 --> 01:01:51.832
this is, I claim,
equal to the following.

01:01:51.832 --> 01:01:56.571
Actually, I am going to need a
little space here so I am going

01:01:56.571 --> 01:02:01.000
to move the equal sign over a
little bit.

01:02:25.000 --> 01:02:27.702
I claim that summation is equal
to that.

01:02:27.702 --> 01:02:32.000
This expectation is equal to
that summation of expectations.

01:02:32.000 --> 01:02:36.042
Why is that?
What are the magic words that

01:02:36.042 --> 01:02:40.281
justify this step?
Linearity of expectation.

01:02:40.281 --> 01:02:45.704
The expectation of a sum is the
sum of the expectations.

01:02:45.704 --> 01:02:49.253
So, that is linearity of
expectation.

01:02:49.253 --> 01:02:52.605
I don't need independence for
that.

01:02:52.605 --> 01:02:57.830
That is just always true for
expectation of any random

01:02:57.830 --> 01:03:04.591
variables.
The sum of the expectations is

01:03:04.591 --> 01:03:10.867
the expectation of the sum and
vice versa.

01:03:10.867 --> 01:03:19.744
Here we did the vice versa.
That is equal to now the sum of

01:03:19.744 --> 01:03:30.000
k=0 to n-1 of expectation of X_k
[T(k) + T(n-k-1) + Theta(n)].

01:03:36.000 --> 01:03:42.272
Why is that true?
What I have done is I've said

01:03:42.272 --> 01:03:49.227
the expectation of the product
is the product of the

01:03:49.227 --> 01:03:53.454
expectations.
That is because of

01:03:53.454 --> 01:04:00.000
independence.
What is independent of what?

01:04:00.000 --> 01:04:02.492
The X_k here,
random variable,

01:04:02.492 --> 01:04:06.962
are independent of any of the
other partitionings in,

01:04:06.962 --> 01:04:10.401
if you will,
the X_k that would exist for

01:04:10.401 --> 01:04:13.151
any of the other recursive
calls.

01:04:13.151 --> 01:04:18.223
So, whatever happens in here is
independent of what happened

01:04:18.223 --> 01:04:20.716
there.
We are actually hiding.

01:04:20.716 --> 01:04:25.358
Since we have a recurrence,
we are not partitioning the

01:04:25.358 --> 01:04:30.000
same wage time.
We have a different one.

01:04:30.000 --> 01:04:33.168
We actually have something
going on underneath the

01:04:33.168 --> 01:04:36.271
mathematics you have to pay
attention to that the

01:04:36.271 --> 01:04:40.150
mathematics alone isn't really
showing, which is that in T(k)

01:04:40.150 --> 01:04:43.706
there is actually a set of
random choices that are being

01:04:43.706 --> 01:04:46.745
made, if you will.
And so you have to understand

01:04:46.745 --> 01:04:50.366
that those are independent of
those, in which case we can

01:04:50.366 --> 01:04:53.469
multiple the probabilities of
their expectations.

01:04:53.469 --> 01:04:55.991
Is everybody with me?
That is a big one,

01:04:55.991 --> 01:05:00.000
independence of X_k from other
random choices.

01:05:00.000 --> 01:05:04.624
That is equal to now,
well, first of all,

01:05:04.624 --> 01:05:09.710
this is nice.
What is the expectation of X_k?

01:05:09.710 --> 01:05:13.179
1/n.
That actually doesn't even

01:05:13.179 --> 01:05:20.000
belong in the summation.
We will just pop it outside.

01:05:20.000 --> 01:05:35.541
I get 1/n times the sum of k=0
to n-1 of expectation of T(k) +

01:05:35.541 --> 01:05:49.808
1/n summation k=0 to n-1 of
expectation of T(n-k-1) + 1/n

01:05:49.808 --> 01:06:00.000
summation k=0 to n-1 up to
Theta(n).

01:06:00.000 --> 01:06:04.838
That is, again,
using linearity of expectation

01:06:04.838 --> 01:06:11.075
there this time to split up
these pieces and just factoring

01:06:11.075 --> 01:06:15.053
out the expectation of k as
being 1/n.

01:06:15.053 --> 01:06:20.430
Everybody with me still?
All of this is elementary.

01:06:20.430 --> 01:06:26.989
It is just one of these things
that is hard just because there

01:06:26.989 --> 01:06:33.174
are so many steps.
And it takes that you have seen

01:06:33.174 --> 01:06:37.986
some of this before.
Now the next observation is

01:06:37.986 --> 01:06:43.003
that these two summations are,
in fact, identical.

01:06:43.003 --> 01:06:48.532
They are the same summation,
just in a different order.

01:06:48.532 --> 01:06:53.549
This is going T(0),
T(1), T(2), T(3) up to T(n-1).

01:06:53.549 --> 01:07:00.000
This one is going T(n-1),
T(n-2), T(n-3) down to T(0).

01:07:00.000 --> 01:07:02.049
These are, in fact,
equal.

01:07:02.049 --> 01:07:05.000
So, therefore,
I have two of them.

01:07:19.000 --> 01:07:22.000
And then what is this term
equal to?

01:07:37.000 --> 01:07:40.991
What is that one equal to?
Theta(n).

01:07:40.991 --> 01:07:45.438
Let's just see why.
The summation of 0 :

01:07:45.438 --> 01:07:50.000
n of Theta(n) is Theta(n^2)
divided by n.

01:07:50.000 --> 01:07:54.447
Or, if I want to bring the
Theta(n) out,

01:07:54.447 --> 01:08:01.631
I have 1 times the summation of
k equals1 to n of Theta(1) or of

01:08:04.527 --> 01:08:07.888
you get n, either way of doing
it.

01:08:07.888 --> 01:08:12.981
This is, in some sense,
because the summations have

01:08:12.981 --> 01:08:17.157
identical terms,
and this is just algebra.

01:08:17.157 --> 01:08:22.861
Now what we are going to do is
do something for technical

01:08:22.861 --> 01:08:30.216
convenience.
Because we are going to absorb

01:08:30.216 --> 01:08:38.390
the k=0, 1 terms into the
Theta(n) for technical

01:08:38.390 --> 01:08:41.000
convenience.

01:08:46.000 --> 01:08:50.432
We have a recurrence here where
I have an order n.

01:08:50.432 --> 01:08:54.412
And, if I look at the cases
where k=0 or k=1,

01:08:54.412 --> 01:08:59.929
I know what the expectation is.
For 0, 1, the expected cost is

01:08:59.929 --> 01:09:04.000
the worst case cost,
which is constant.

01:09:04.000 --> 01:09:07.920
Because I am only solving the
problem for a constant size.

01:09:07.920 --> 01:09:12.252
And we know that for any of the
boundary cases that our solution

01:09:12.252 --> 01:09:15.142
of recurrence,
our assumption is that it is

01:09:15.142 --> 01:09:18.100
constant time.
So, I basically can just take

01:09:18.100 --> 01:09:21.608
those two terms out.
And all that does it accumulate

01:09:21.608 --> 01:09:24.291
some more constant here in the
Theta(n).

01:09:24.291 --> 01:09:27.798
It is going to make the
solution of the recurrence a

01:09:27.798 --> 01:09:33.395
little bit easier.
And, if I do that,

01:09:33.395 --> 01:09:43.584
I get expectation of T(n) = 2/n
summation k=2 to n-1 of

01:09:43.584 --> 01:09:50.000
expectation of T(k) + Theta(n).

01:10:00.000 --> 01:10:06.864
So, all of that work was to
derive the recurrence.

01:10:06.864 --> 01:10:14.570
And now we have to solve it.
Just to review what we did,

01:10:14.570 --> 01:10:21.434
we started out with a
recurrence which was for the

01:10:21.434 --> 01:10:29.000
random variable which involved a
case statement.

01:10:29.000 --> 01:10:33.128
We converted that into some
mathematics without the case

01:10:33.128 --> 01:10:37.031
statement, just with a product,
and then we derived a

01:10:37.031 --> 01:10:41.535
recurrence for the expectation.
And now we are in the process

01:10:41.535 --> 01:10:44.087
of trying to solve that
recurrence.

01:10:44.087 --> 01:10:47.765
We have done some
simplification of the recurrence

01:10:47.765 --> 01:10:52.568
so that we understand what it is
that we are going to solve here.

01:10:52.568 --> 01:10:56.472
By the way, I don't give things
like this on quizzes.

01:10:56.472 --> 01:11:00.000
I do expect you to understand
it.

01:11:00.000 --> 01:11:03.120
The elements of this you will
find on a quiz.

01:11:03.120 --> 01:11:05.602
This is a lot of work to figure
out.

01:11:05.602 --> 01:11:09.148
This took smart people to do.
Even though it is all

01:11:09.148 --> 01:11:12.907
elementary, but working out
something like this at the

01:11:12.907 --> 01:11:17.375
elementary level is still a bit
of work even for somebody who is

01:11:17.375 --> 01:11:23.465
knowledgeable in this area.
Now we are going to solve that

01:11:23.465 --> 01:11:30.395
last recurrence over there and
we are going to prove that the

01:11:30.395 --> 01:11:37.556
expectation of T(n) is less than
or equal to (an lg n) for some

01:11:37.556 --> 01:11:44.139
constant a greater than 0.
That is going to be what we are

01:11:44.139 --> 01:11:48.759
going to do.
And so what technique do you

01:11:48.759 --> 01:11:52.571
think we should use to prove
this?

01:11:52.571 --> 01:11:58.000
Does this look like a master
method?

01:12:03.000 --> 01:12:08.600
It is nothing like the master
method.

01:12:08.600 --> 01:12:13.733
So, when in doubt do
substitution.

01:12:13.733 --> 01:12:19.333
It is the grand-daddy of all
methods.

01:12:19.333 --> 01:12:28.355
What we will do is solve the
base case by simply choosing a

01:12:28.355 --> 01:12:37.844
big enough so that (an lg n) is
bigger than the expectation of

01:12:37.844 --> 01:12:45.000
T(n) for sufficiently large
small n.

01:12:45.000 --> 01:12:49.212
So, I just pick a to be big
enough.

01:12:49.212 --> 01:12:54.044
And this is,
by the way, why I wanted to

01:12:54.044 --> 01:12:59.000
exclude 0 and 1 from the
recurrence.

01:12:59.000 --> 01:13:03.338
Because, for example,
when n=0, log of 0 is,

01:13:03.338 --> 01:13:08.181
it's like dividing by 0,
right, you cannot do it.

01:13:08.181 --> 01:13:12.519
Log of 1 is 0.
So here, even if I restricted

01:13:12.519 --> 01:13:18.370
it to 1, here I would have a 0,
and I can't ever pick a big

01:13:18.370 --> 01:13:24.424
enough to dominate those cases.
What I do is I just say look,

01:13:24.424 --> 01:13:30.577
I just absorb whatever the cost
is into the T(n) for technical

01:13:30.577 --> 01:13:37.169
convenience.
And that lets me address it as

01:13:37.169 --> 01:13:44.119
(an lg n) to be big enough to
handle the base case.

01:13:44.119 --> 01:13:50.930
So, that is why we made that
technical assumption.

01:13:50.930 --> 01:13:58.714
We are going to use a fact
which is that the summation of

01:13:58.714 --> 01:14:06.776
k=2 to n-1 of k lg k is less
than or equal to 1/2n^2 lg n -

01:14:06.776 --> 01:14:11.242
1/8n^2.
I am going to leave that as an

01:14:11.242 --> 01:14:15.429
exercise for you to workout.
I think it is an exercise in

01:14:15.429 --> 01:14:18.644
the book, too.
I want you to go and evaluate

01:14:18.644 --> 01:14:21.261
this.
There are two ways to evaluate

01:14:21.261 --> 01:14:23.130
it.
One is by using purely

01:14:23.130 --> 01:14:27.093
summations and facts about
summations by splitting the

01:14:27.093 --> 01:14:31.579
summation into two pieces and
reconstituting it to prove this

01:14:31.579 --> 01:14:35.338
bound.
The other way is to use the

01:14:35.338 --> 01:14:38.629
integral method for solving
summations.

01:14:38.629 --> 01:14:43.220
Either way you can prove.
The integral method actually

01:14:43.220 --> 01:14:46.165
gets you a tighter bound than
this.

01:14:46.165 --> 01:14:50.582
This is a basic fact,
and you should go off and know

01:14:50.582 --> 01:14:55.000
how to do that.
Now let's do substitution.

01:15:06.000 --> 01:15:18.821
The expectation of T(n) is less
than or equal to 2/n times the

01:15:18.821 --> 01:15:29.331
summation k=2 to n-1,
now we do the substitution of

01:15:29.331 --> 01:15:39.000
ak lg k, the smaller values plus
Theta(n).

01:15:39.000 --> 01:15:42.416
I might mentioned,
by the way, that the hard part

01:15:42.416 --> 01:15:45.334
of doing this,
it is easy to get the bound

01:15:45.334 --> 01:15:48.679
without this term,
it is easy to get this bound,

01:15:48.679 --> 01:15:51.669
1/2n^2 lg n,
it is harder to get the second

01:15:51.669 --> 01:15:54.231
order term.
It turns out you need the

01:15:54.231 --> 01:15:59.000
second order term in order to do
what we are going to do.

01:15:59.000 --> 01:16:05.621
You have to be able to subtract
a quadratic amount of the n^2 lg

01:16:05.621 --> 01:16:09.195
n in order to make this proof
work.

01:16:09.195 --> 01:16:15.291
And that is the trickier part
in evaluating that summation.

01:16:15.291 --> 01:16:20.126
So, we get this.
That is less than or equal to?

01:16:20.126 --> 01:16:26.537
Well, I happen to know how much
this is by using that formula.

01:16:26.537 --> 01:16:31.792
I use my fact and get 2a/n
(1/2n^2 lg n - 1/8n^2) +

01:16:31.792 --> 01:16:43.558
Theta(n).
Did I do something wrong?

01:16:43.558 --> 01:16:51.970
There we go.
Very good.

01:16:51.970 --> 01:17:05.272
That is equal to -
If I multiply this first part

01:17:05.272 --> 01:17:14.363
through that is an lg n.
And now, so I don't make a

01:17:14.363 --> 01:17:22.545
mistake, I want to express this
as my desired,

01:17:22.545 --> 01:17:32.000
this is what I want it to be,
minus a residual.

01:17:32.000 --> 01:17:38.702
I am going to write the
residual as this part.

01:17:38.702 --> 01:17:47.489
And so, the way to write that
is, that is going to be minus.

01:17:47.489 --> 01:17:56.723
And then it is going to be this
term here, which is going to be

01:17:56.723 --> 01:18:00.000
an/4 - Theta(n).

01:18:05.000 --> 01:18:12.309
And that is going to be less
than or equal to an lg n if this

01:18:12.309 --> 01:18:17.304
part is positive.
And I can make that part

01:18:17.304 --> 01:18:24.614
positive by picking a big enough
such that an/4 dominates the

01:18:24.614 --> 01:18:34.590
constant in the Theta(n) here.
Whatever the constant is here,

01:18:34.590 --> 01:18:45.211
I can find an a that is big
enough so that this term makes

01:18:45.211 --> 01:18:54.527
this part positive.
If a is big enough so that an/4

01:18:54.527 --> 01:19:01.217
dominates Theta(n).
And so the running time of

01:19:01.217 --> 01:19:04.288
randomized quicksort is order n
lg n.

01:19:04.288 --> 01:19:09.064
That is what we just proved,
the expected running time is

01:19:09.064 --> 01:19:11.623
order n lg n.
Now, in practice,

01:19:11.623 --> 01:19:16.741
quicksort is a great algorithm.
It is typically three or more

01:19:16.741 --> 01:19:21.688
times faster than mergesort.
It doesn't give you the strong

01:19:21.688 --> 01:19:26.464
guarantee necessarily of
mergesort and being worst-case n

01:19:26.464 --> 01:19:29.245
lg n.
But in practice,

01:19:29.245 --> 01:19:33.136
if you use randomized
quicksort, it is generally as

01:19:33.136 --> 01:19:37.573
much as three times faster.
It does require code tuning in

01:19:37.573 --> 01:19:40.219
order to get it up to be that
fast.

01:19:40.219 --> 01:19:44.966
You do have to go and coarsen
the base cases and do some other

01:19:44.966 --> 01:19:47.613
tricks there,
but most good sorting

01:19:47.613 --> 01:19:51.660
algorithms that you will find
are based on quicksort.

01:19:51.660 --> 01:19:56.018
Also one of the other reasons
it works well is because it

01:19:56.018 --> 01:20:01.000
tends to work well with caches
in virtual memory.

01:20:01.000 --> 01:20:05.431
We are not really talking much
about caching models and so

01:20:05.431 --> 01:20:09.941
forth, big topic these days in
algorithms, but it does work

01:20:09.941 --> 01:20:12.973
very well with caches in virtual
memory.

01:20:12.973 --> 01:20:17.405
It is another reason that this
is a good algorithm to use.

01:20:17.405 --> 01:20:20.359
Good recitation,
by the way, on Friday.

01:20:20.359 --> 01:20:24.247
We are going to see another n
log n time algorithm,

01:20:24.247 --> 01:20:27.000
a very important one in
recitation on Friday.