WEBVTT

00:00:10.000 --> 00:00:13.000
OK, good morning.
So today we are going to,

00:00:13.000 --> 00:00:17.000
as I mentioned last week,
we've started the part of the

00:00:17.000 --> 00:00:22.000
course where we are doing more
things having to do with design

00:00:22.000 --> 00:00:26.000
than purely analysis.
Today, we're actually going to

00:00:26.000 --> 00:00:32.000
do analysis, but it's the type
of analysis that leads to really

00:00:32.000 --> 00:00:38.000
interesting design issues.
And we're going to follow it up

00:00:38.000 --> 00:00:46.000
on Wednesday with an application
of the methods we're going to

00:00:46.000 --> 00:00:53.000
learn today with a really
interesting and practical

00:00:53.000 --> 00:00:57.000
problem.
So we're talking today about

00:00:57.000 --> 00:01:04.000
amortized analysis.
And I want to motivate this

00:01:04.000 --> 00:01:12.000
topic by asking the question,
how large should a hash table

00:01:12.000 --> 00:01:17.000
be?
So, how large should a hash

00:01:17.000 --> 00:01:20.000
table be?
Any suggestions?

00:01:20.000 --> 00:01:30.000
You have got to make a hash
table, how big should I make it?

00:01:30.000 --> 00:01:38.000
Let's say it's a simple hash
table, resolving collisions with

00:01:38.000 --> 00:01:43.000
chaining.
How big should it be?

00:01:43.000 --> 00:01:50.000
Twice as big as you need:
OK, how big would that be?

00:01:50.000 --> 00:01:57.000
So, twice the number of
elements, for example,

00:01:57.000 --> 00:02:02.000
OK.
As I increase the size of a

00:02:02.000 --> 00:02:10.000
hash table, what happens to the
search time?

00:02:10.000 --> 00:02:14.000
What happens to search time as
I increase the size of the hash

00:02:14.000 --> 00:02:16.000
table?
Yeah, but what does it,

00:02:16.000 --> 00:02:19.000
in general, do you?
It decreases,

00:02:19.000 --> 00:02:21.000
right?
OK, the bigger I make it,

00:02:21.000 --> 00:02:24.000
in fact, if I make it
sufficiently large,

00:02:24.000 --> 00:02:27.000
then I essentially get a direct
access table,

00:02:27.000 --> 00:02:32.000
and everything is worst-case,
order one time.

00:02:32.000 --> 00:02:38.000
So in some sense,
we'll get back to your answer

00:02:38.000 --> 00:02:44.000
in a minute.
We should make it as large as

00:02:44.000 --> 00:02:50.000
possible, OK,
so that the searching is cheap.

00:02:50.000 --> 00:02:58.000
The flipside of that is what?
It takes a lot of space so I

00:02:58.000 --> 00:03:05.000
should make it as small as
possible so as not to waste

00:03:05.000 --> 00:03:10.000
space.
So I want it big,

00:03:10.000 --> 00:03:18.000
and the happy medium,
as we've discussed in our

00:03:18.000 --> 00:03:26.000
analysis, is to make it order n
size for n items,

00:03:26.000 --> 00:03:33.000
OK, because making it larger
than order n,

00:03:33.000 --> 00:03:43.000
the payoff in search time is
not worth the extra amount of

00:03:43.000 --> 00:03:51.000
space that you are paying.
OK, or at least you can view it

00:03:51.000 --> 00:03:54.000
that way.
OK, however,

00:03:54.000 --> 00:03:58.000
this begs the question,
which is, how do I make it,

00:03:58.000 --> 00:04:04.000
if I start out with a hash
table, and I don't know how many

00:04:04.000 --> 00:04:09.000
elements are going to be hashed
into it?

00:04:09.000 --> 00:04:18.000
OK, how big should I make it?
OK, so what if we don't know --

00:04:34.000 --> 00:04:39.000
-- in advance?
OK, what if we don't know n?

00:04:39.000 --> 00:04:47.000
OK, so the solution to this
problem turns out it's fairly

00:04:47.000 --> 00:04:53.000
elegant?
It's a strategy called dynamic

00:04:53.000 --> 00:04:58.000
tables.
OK, and the idea is that

00:04:58.000 --> 00:05:06.000
whenever the table gets too many
elements in it,

00:05:06.000 --> 00:05:13.000
gets too full,
OK, so the idea is --

00:05:26.000 --> 00:05:30.000
-- OK, and we say that says the
table overflows,

00:05:30.000 --> 00:05:34.000
OK, we grow it and make a
bigger table.

00:05:34.000 --> 00:05:38.000
So, for hashing,
although there's going to be no

00:05:38.000 --> 00:05:44.000
point at which you could say
that it overflows in the sense

00:05:44.000 --> 00:05:49.000
that it wouldn't be functional
at least if it was done with

00:05:49.000 --> 00:05:52.000
chaining.
There would be,

00:05:52.000 --> 00:05:57.000
by the way, if you were doing
it with open addressing.

00:05:57.000 --> 00:06:01.000
But let's say with chaining,
when it gets too big,

00:06:01.000 --> 00:06:06.000
say, as many elements as the
size of the table,

00:06:06.000 --> 00:06:11.000
what we do is we grow the
table.

00:06:11.000 --> 00:06:16.000
So, the way we do that as we
allocate using,

00:06:16.000 --> 00:06:22.000
in a language like C,
it's called Malloc,

00:06:22.000 --> 00:06:30.000
or in a language like Java
called New, a larger table.

00:06:30.000 --> 00:06:43.000
So, we create a larger table.
We move the items from the old

00:06:43.000 --> 00:06:53.000
table to the new.
And then, we free the old

00:06:53.000 --> 00:07:01.000
table.
So, let's do an example.

00:07:01.000 --> 00:07:05.000
So, let's say I have,
over here, a table of size one,

00:07:05.000 --> 00:07:09.000
and it's empty to begin with.
And I do an insert.

00:07:09.000 --> 00:07:12.000
So what I do is stick it in the
table.

00:07:12.000 --> 00:07:14.000
It fits.
OK, so here,

00:07:14.000 --> 00:07:17.000
I'm not going to do it with
hashing.

00:07:17.000 --> 00:07:22.000
I'm just going to do it as if I
just had a table that I was

00:07:22.000 --> 00:07:26.000
filling up with elements to
abstract the problem.

00:07:26.000 --> 00:07:33.000
But it would work with hashing.
It would work with any kind of

00:07:33.000 --> 00:07:38.000
fixed size data structure.
I insert again,

00:07:38.000 --> 00:07:42.000
oops, doesn't fit.
I get an overflow.

00:07:42.000 --> 00:07:48.000
OK, so what I do is I create a
new, actually,

00:07:48.000 --> 00:07:53.000
I'm going to need a little more
space than that.

00:07:53.000 --> 00:08:01.000
I create a new table of size
two, doubling the size.

00:08:01.000 --> 00:08:06.000
And, I copy the old value into
the new.

00:08:06.000 --> 00:08:13.000
I freed this one,
and now I can insert item two.

00:08:13.000 --> 00:08:19.000
So, I do it again.
I get another overflow.

00:08:19.000 --> 00:08:24.000
So now, I make a table of size
four.

00:08:24.000 --> 00:08:31.000
I copied these guys in,
and then I insert my number

00:08:31.000 --> 00:08:36.000
three.
I do insert here.

00:08:36.000 --> 00:08:41.000
I do five.
I guess I should be using ditto

00:08:41.000 --> 00:08:46.000
marks.
That would be a lot smarter.

00:08:46.000 --> 00:08:52.000
Whoops, what am I doing?
I overflow.

00:08:52.000 --> 00:09:00.000
And now, I make one of size
eight, OK, copy these over,

00:09:00.000 --> 00:09:07.000
and now I can insert five.
And I can do six,

00:09:07.000 --> 00:09:09.000
seven, etc.,
OK?

00:09:09.000 --> 00:09:15.000
Does everybody understand the
basic idea?

00:09:15.000 --> 00:09:24.000
So, whenever I overflow,
I'm going to create a table of

00:09:24.000 --> 00:09:30.000
twice the size.
OK, so let's do a quick

00:09:30.000 --> 00:09:39.000
analysis of this.
So, we have a sequence of n

00:09:39.000 --> 00:09:49.000
insertion operations.
OK, what is the worst-case cost

00:09:49.000 --> 00:09:58.000
of one insert operation?
What's the worst case for any

00:09:58.000 --> 00:10:04.000
one of these?
Yeah, it's order n,

00:10:04.000 --> 00:10:10.000
whatever the overhead is of
copying; if we counted it as

00:10:10.000 --> 00:10:17.000
one, it would be basically n or
n plus one because we've got to

00:10:17.000 --> 00:10:21.000
copy all those.
OK, so it's order n.

00:10:21.000 --> 00:10:25.000
So therefore,
if I have n of those,

00:10:25.000 --> 00:10:31.000
so the worst-case cost of n
inserts is equal to n times

00:10:31.000 --> 00:10:36.000
order n, which is order n^2.

00:10:44.000 --> 00:10:54.000
Any questions?
Does that make sense?

00:10:54.000 --> 00:10:58.000
Raise hands.

00:11:15.000 --> 00:11:19.000
Yeah, not all of them can be
worst-case, good.

00:11:19.000 --> 00:11:24.000
And in fact,
this is totally wrong analysis.

00:11:24.000 --> 00:11:29.000
Just because one can be
worst-case order n doesn't mean

00:11:29.000 --> 00:11:35.000
n are necessarily order n.
OK, so this is totally wrong

00:11:35.000 --> 00:11:38.000
analysis.
n inserts, in fact,

00:11:38.000 --> 00:11:43.000
take order n time in the worst
case.

00:11:43.000 --> 00:11:48.000
OK, it doesn't take order n^2.
So, the analysis is correct up

00:11:48.000 --> 00:11:52.000
to the point where we set the
worst-case of one insert was

00:11:52.000 --> 00:11:55.000
order n.
Therefore, that's the wrong

00:11:55.000 --> 00:11:58.000
step.
OK, whenever you see bugs in

00:11:58.000 --> 00:12:02.000
proofs, you want to know,
which step is the one that

00:12:02.000 --> 00:12:07.000
failed so you can make sure that
you don't have a confusion

00:12:07.000 --> 00:12:13.000
there?
So, let's do the proper

00:12:13.000 --> 00:12:23.000
analysis, OK?
So let's let c_i be the cost of

00:12:23.000 --> 00:12:32.000
the i'th insert.
OK, so that's equal to i,

00:12:32.000 --> 00:12:42.000
if i minus one is an exact
power of two.

00:12:42.000 --> 00:12:47.000
And it's one otherwise.
OK, so as I was going through

00:12:47.000 --> 00:12:52.000
here, it was only when I
inserted something where the

00:12:52.000 --> 00:12:56.000
previous one had been the exact
power of two,

00:12:56.000 --> 00:13:03.000
because that was my table size.
That's when I got the overflow

00:13:03.000 --> 00:13:09.000
and had to do all that copying.
And otherwise,

00:13:09.000 --> 00:13:13.000
the cost, for example,
for inserting six,

00:13:13.000 --> 00:13:17.000
was just one.
I just inserted it.

00:13:17.000 --> 00:13:23.000
Does everybody see that?
So, let's actually make a

00:13:23.000 --> 00:13:31.000
little table here so we can see
this a little bit more clearly.

00:13:31.000 --> 00:13:36.000
OK, so here's i,
and in the size of the table at

00:13:36.000 --> 00:13:43.000
step i, and the cost at step i.
OK, so let's see,

00:13:43.000 --> 00:13:48.000
the size of i,
let's see, at step one it was

00:13:48.000 --> 00:13:50.000
one.
At step two,

00:13:50.000 --> 00:13:53.000
it was two.
And at step three,

00:13:53.000 --> 00:13:57.000
that's when,
to get three in the table,

00:13:57.000 --> 00:14:03.000
we had to double the size here.
So, this is four,

00:14:03.000 --> 00:14:06.000
and four, it fit.
And then five,

00:14:06.000 --> 00:14:09.000
it had to bump up to eight.
And then, six,

00:14:09.000 --> 00:14:12.000
it was eight.
Seven, it was eight.

00:14:12.000 --> 00:14:16.000
Eight, it was eight.
And nine, it bumps up to 16,

00:14:16.000 --> 00:14:19.000
16, etc.
So that's the size.

00:14:19.000 --> 00:14:22.000
And let's take a look at what
the cost was.

00:14:22.000 --> 00:14:26.000
So, the cost here was one,
OK, to insert one.

00:14:26.000 --> 00:14:29.000
The cost here was,
I had to copy one,

00:14:29.000 --> 00:14:35.000
and then insert one.
So, the cost was two.

00:14:35.000 --> 00:14:38.000
Here, I had to copy two and
insert one.

00:14:38.000 --> 00:14:44.000
So, the cost was three.
Here, I had to just insert one.

00:14:44.000 --> 00:14:49.000
So, the cost was one.
Here, I had to copy four and

00:14:49.000 --> 00:14:52.000
insert one.
So, the cost was five.

00:14:52.000 --> 00:14:55.000
Excuse me?
I think it is.

00:14:55.000 --> 00:14:59.000
Yeah, see, it's i cost.
The cost for five is i,

00:14:59.000 --> 00:15:05.000
OK, is five if this is a power
of two.

00:15:05.000 --> 00:15:07.000
OK, one, one,
and now we paid nine,

00:15:07.000 --> 00:15:11.000
and then one again.
So that's the cost we are

00:15:11.000 --> 00:15:15.000
paying.
It's a little bit easier to see

00:15:15.000 --> 00:15:18.000
what the costs are if I break
them down.

00:15:18.000 --> 00:15:23.000
OK, so let's just redraw this
as two values because there is

00:15:23.000 --> 00:15:28.000
always the cost for inserting
the one thing that I want to

00:15:28.000 --> 00:15:32.000
insert.
And now, the residual amount

00:15:32.000 --> 00:15:35.000
that I have to pay is I have to
pay one here.

00:15:35.000 --> 00:15:38.000
I've got to pay two additional,
four additional,

00:15:38.000 --> 00:15:42.000
eight additional.
That makes the pattern a little

00:15:42.000 --> 00:15:45.000
bit easier to see.
OK, this is the cost of copying

00:15:45.000 --> 00:15:49.000
versus the cost of just doing
the actual insert,

00:15:49.000 --> 00:15:51.000
OK?
Now, if you're taking notes,

00:15:51.000 --> 00:15:55.000
leave some space here because
I'm going to come back to this

00:15:55.000 --> 00:15:58.000
table later.
OK, so leave a little bit of

00:15:58.000 --> 00:16:03.000
space because I'm going to come
back and add some more things at

00:16:03.000 --> 00:16:09.000
there at a later time.
OK, so, I can then just add up

00:16:09.000 --> 00:16:14.000
the cost of n inserts.
That's just the sum,

00:16:14.000 --> 00:16:20.000
I equals one to n of c_i,
which is equal to,

00:16:20.000 --> 00:16:27.000
well, by this analysis it is
essentially n because that's

00:16:27.000 --> 00:16:35.000
what this thing adds up to plus
I just have to add the powers of

00:16:35.000 --> 00:16:42.000
two up to but not exceeding
whatever my n was.

00:16:42.000 --> 00:16:50.000
So, if I do my algebra properly
there, that's up to the floor of

00:16:50.000 --> 00:16:55.000
log n minus one,
OK, of two to the J.

00:16:55.000 --> 00:17:03.000
So, I'm just adding up all the
powers of two up to that aren't

00:17:03.000 --> 00:17:09.000
going to exceed my n.
And, this is what type of

00:17:09.000 --> 00:17:12.000
series?
That's geometric.

00:17:12.000 --> 00:17:18.000
That's geometric,
so it is bounded by its largest

00:17:18.000 --> 00:17:22.000
term.
Its largest term is two to the

00:17:22.000 --> 00:17:27.000
ceiling; it's dominated by its
largest term,

00:17:27.000 --> 00:17:32.000
two to the ceiling of log n
minus one, which is,

00:17:32.000 --> 00:17:38.000
at most, n.
OK, and then all the other

00:17:38.000 --> 00:17:41.000
terms at up to,
at most, n.

00:17:41.000 --> 00:17:47.000
So this is actually less than
or equal to 3n,

00:17:47.000 --> 00:17:51.000
which is order n as we want it
to show.

00:17:51.000 --> 00:17:57.000
OK, that's algebra.
OK, so, thus,

00:17:57.000 --> 00:18:05.000
the average cost per insert is
theta of n over n,

00:18:05.000 --> 00:18:12.000
which is theta one.
So, the average cost of an

00:18:12.000 --> 00:18:21.000
insert is order one,
which is what we would like it

00:18:21.000 --> 00:18:30.000
to be especially if we're
building hash tables.

00:18:30.000 --> 00:18:34.000
Even though sometimes you have
to pay a big price with

00:18:34.000 --> 00:18:38.000
amortized over the previous
insertions that we've done,

00:18:38.000 --> 00:18:42.000
so that the overall cost of n
operations is order n.

00:18:42.000 --> 00:18:46.000
And that's the notion of
amortized analysis,

00:18:46.000 --> 00:18:50.000
OK, that if I look at a
sequence of operations,

00:18:50.000 --> 00:18:55.000
I can spread the cost out over
a whole bunch of operations,

00:18:55.000 --> 00:18:59.000
so that the average cost is
order n.

00:18:59.000 --> 00:19:07.000
So, if we sort of summarize
that, OK, OK,

00:19:07.000 --> 00:19:20.000
with basically an amortized
analysis, we analyze a sequence

00:19:20.000 --> 00:19:34.000
of operations to show that the
average cost per operation is

00:19:34.000 --> 00:19:44.000
small, even though one
operation, or several,

00:19:44.000 --> 00:19:59.000
even, may be expensive.
OK, there's no probability.

00:19:59.000 --> 00:20:05.000
Even though we're doing it with
averages, there's no probability

00:20:05.000 --> 00:20:08.000
going on.
OK, what you do probability,

00:20:08.000 --> 00:20:13.000
and you are looking at means,
there's averages.

00:20:13.000 --> 00:20:17.000
OK, here's average,
but there's no probability

00:20:17.000 --> 00:20:21.000
going on.
It's average performance in the

00:20:21.000 --> 00:20:27.000
worst case because n operations
take me a constant amount of

00:20:27.000 --> 00:20:31.000
time per operation in the worst
case.

00:20:31.000 --> 00:20:36.000
n operations take me order n
time.

00:20:36.000 --> 00:20:42.000
OK, each operation takes order
one time, OK,

00:20:42.000 --> 00:20:49.000
but it's amortized over the n
operations, OK?

00:20:49.000 --> 00:20:52.000
Yeah, question?
Yes.

00:20:52.000 --> 00:21:00.000
Yes, yes, you can mix,
but you don't have to.

00:21:00.000 --> 00:21:06.000
Yeah, but the point is that the
basic amortized analysis is

00:21:06.000 --> 00:21:10.000
actually saying something very
strong.

00:21:10.000 --> 00:21:16.000
It's giving you worst-case
bounds, but over a sequence as

00:21:16.000 --> 00:21:21.000
opposed to looking at each
individual element of the

00:21:21.000 --> 00:21:25.000
sequence.
Now, there are three types of

00:21:25.000 --> 00:21:32.000
amortized arguments that appear
in the literature.

00:21:32.000 --> 00:21:35.000
Maybe there are more.
At one point,

00:21:35.000 --> 00:21:39.000
there were two.
And then, a third one was

00:21:39.000 --> 00:21:43.000
developed.
So, maybe there's a fourth.

00:21:43.000 --> 00:21:49.000
The first one is an aggregate,
what's called an aggregate

00:21:49.000 --> 00:21:53.000
analysis.
And this is what we just saw,

00:21:53.000 --> 00:21:59.000
OK, where basically you just
analyze, what do the n

00:21:59.000 --> 00:22:08.000
operations take?
OK, and then we're going to see

00:22:08.000 --> 00:22:16.000
two more today.
One is called an accounting

00:22:16.000 --> 00:22:25.000
argument, and the other is a
potential argument.

00:22:25.000 --> 00:22:36.000
These two are more precise
because they allocate specific

00:22:36.000 --> 00:22:45.000
amortized costs to each
operation.

00:22:45.000 --> 00:22:49.000
So, one of the things about the
aggregate analysis is that you

00:22:49.000 --> 00:22:53.000
can't really say what the
amortized cost of a single

00:22:53.000 --> 00:22:57.000
operation is easily.
You can in this case.

00:22:57.000 --> 00:23:00.000
You can say it's order one,
OK?

00:23:00.000 --> 00:23:03.000
But, in the accounting and
potential arguments,

00:23:03.000 --> 00:23:07.000
it gives you a much more
precise way of characterizing

00:23:07.000 --> 00:23:11.000
what an amortized cost of a
particular operation is.

00:23:11.000 --> 00:23:16.000
So, let's pitch in and look at
the accounting method as our

00:23:16.000 --> 00:23:19.000
first method.
So, these we're going to go

00:23:19.000 --> 00:23:22.000
through the exact same example.
In some sense,

00:23:22.000 --> 00:23:25.000
this example,
the easiest argument to make is

00:23:25.000 --> 00:23:29.000
the aggregate analysis.
OK, so we're going get into

00:23:29.000 --> 00:23:32.000
arguments that,
in some sense,

00:23:32.000 --> 00:23:37.000
see more complicated.
But it turns out that these

00:23:37.000 --> 00:23:43.000
methods are more powerful in
many circumstances.

00:23:43.000 --> 00:23:49.000
OK, and so I want to do it in a
simple situation where you have

00:23:49.000 --> 00:23:56.000
some sort of appreciation of the
fact that you can look at any

00:23:56.000 --> 00:24:02.000
particular problem and approach
it from different ways.

00:24:02.000 --> 00:24:08.000
OK, so the accounting method is
putting yourself in the position

00:24:08.000 --> 00:24:19.000
of a financial accountant.
So what you do is we are going

00:24:19.000 --> 00:24:33.000
to charge the i'th operation a
fictitious amortized cost.

00:24:33.000 --> 00:24:39.000
We'll call it c hat sub i,
where we are going to use the

00:24:39.000 --> 00:24:44.000
abstraction that $1 pays for one
unit of work.

00:24:44.000 --> 00:24:51.000
There's Time manipulating the
data structure or whatever.

00:24:51.000 --> 00:24:55.000
OK, so the idea is you charge
the cost.

00:24:55.000 --> 00:25:03.000
You say, this operation will
cost you $5 or whatever.

00:25:03.000 --> 00:25:14.000
OK, and that phi is consumed to
perform the operation,

00:25:14.000 --> 00:25:21.000
but there may be some unused
part.

00:25:21.000 --> 00:25:33.000
So, if there's any unused
amount, it's going to be stored

00:25:33.000 --> 00:25:44.000
in the bank for use by later
operations.

00:25:44.000 --> 00:25:49.000
So the idea is that if the phi
that is being paid,

00:25:49.000 --> 00:25:54.000
the c_i hat phi,
isn't sufficient to pay for

00:25:54.000 --> 00:26:01.000
performing the operation,
then you take money out of the

00:26:01.000 --> 00:26:05.000
bank to pay for it.
OK, and so you don't get

00:26:05.000 --> 00:26:09.000
arrested, what's the property
that you've got to have?

00:26:09.000 --> 00:26:12.000
You've got to have the bank
balance.

00:26:12.000 --> 00:26:16.000
What about the bank balance?
What mathematical fact has to

00:26:16.000 --> 00:26:20.000
hold the bank balance?
Yeah, it better be greater than

00:26:20.000 --> 00:26:22.000
or equal to zero,
right?

00:26:22.000 --> 00:26:26.000
Most people are familiar with
that.

00:26:26.000 --> 00:26:30.000
So, the bank balance must not
go negative.

00:26:30.000 --> 00:26:35.000
In other words,
the amortized costs minus the

00:26:35.000 --> 00:26:41.000
costs of the operations up to
that point have to always be

00:26:41.000 --> 00:26:46.000
enough to pay for all the
operations that you're doing.

00:26:46.000 --> 00:26:51.000
Otherwise, you're borrowing on
the future.

00:26:51.000 --> 00:26:56.000
In amortized analysis,
we don't borrow on the future,

00:26:56.000 --> 00:27:04.000
at least not on the simple ones
that we are doing here.

00:27:04.000 --> 00:27:11.000
OK, so that means we must have
that the sum,

00:27:11.000 --> 00:27:19.000
I equals one to n of c_i,
the true costs,

00:27:19.000 --> 00:27:29.000
therefore, if the balance is
not going to ever go negative,

00:27:29.000 --> 00:27:41.000
must be bounded above by the
amortized costs for all n.

00:27:41.000 --> 00:27:45.000
OK, for the bank balance not to
go negative, if I add up the

00:27:45.000 --> 00:27:49.000
true costs, it's got to be the
case that I can always pay for

00:27:49.000 --> 00:27:51.000
them.
This is what I'm charging.

00:27:51.000 --> 00:27:54.000
This is what it actually costs
me.

00:27:54.000 --> 00:27:58.000
So, it better be the case that
whatever I've actually had to

00:27:58.000 --> 00:28:02.000
pay to operate on that data
structure, that's what this is,

00:28:02.000 --> 00:28:07.000
better be covered by the amount
that I've been charging people

00:28:07.000 --> 00:28:12.000
for the use of that data
structure up to that point.

00:28:12.000 --> 00:28:15.000
And that's got to be true for
all n.

00:28:15.000 --> 00:28:19.000
But notice that this now gives
me a way of charging a

00:28:19.000 --> 00:28:23.000
particular operation a certain
amount.

00:28:23.000 --> 00:28:28.000
So, the total amortized costs
provide an upper bound on the

00:28:28.000 --> 00:28:35.000
total true costs.
Total amortized costs are an

00:28:35.000 --> 00:28:45.000
upper bound on the true costs.
Any question about this?

00:28:45.000 --> 00:28:54.000
That we'll do the example of
the dynamic table using this

00:28:54.000 --> 00:29:01.000
methodology.
OK, so, back to the dynamic

00:29:01.000 --> 00:29:08.000
table.
OK, so what we're going to do

00:29:08.000 --> 00:29:19.000
in this case is we're going to
charge an amortized cost of $3

00:29:19.000 --> 00:29:31.000
for the i'th insert for all i.
And the idea is that $1 is

00:29:31.000 --> 00:29:45.000
going to pay for an immediate
insert, and $2 is going to be

00:29:45.000 --> 00:30:00.000
stored for doubling the table.
And, it needs to be expanded.

00:30:00.000 --> 00:30:10.000
When the table doubles,
of the stored dollars,

00:30:10.000 --> 00:30:24.000
we'll use one dollar to move a
recent item, I'll call it,

00:30:24.000 --> 00:30:34.000
and one dollar we'll move an
old item.

00:30:34.000 --> 00:30:42.000
So, let's do the example.
So, imagine that I'm in this

00:30:42.000 --> 00:30:49.000
situation where I have a table
of size eight,

00:30:49.000 --> 00:30:57.000
and I just doubled my table.
So I have four items of the

00:30:57.000 --> 00:31:03.000
table.
What I'm going to do is have no

00:31:03.000 --> 00:31:11.000
dollars in my table.
So, along comes an insertion of

00:31:11.000 --> 00:31:16.000
item number five.
I charge $3 for it.

00:31:16.000 --> 00:31:23.000
$1 lets me put the item in the
table, and I have $2 left over.

00:31:23.000 --> 00:31:31.000
So let me store those $2 in the
slot corresponding to where that

00:31:31.000 --> 00:31:34.000
item is.
Now, item six comes in.

00:31:34.000 --> 00:31:38.000
Once again, $1,
charge $3, $1 is paid for the

00:31:38.000 --> 00:31:42.000
insert, $2 left over,
I'm going to play $2.

00:31:42.000 --> 00:31:45.000
Let me put it down there,
and so forth.

00:31:45.000 --> 00:31:48.000
The next one comes in,
$2, $2 leftover,

00:31:48.000 --> 00:31:51.000
and now the ninth item comes
in.

00:31:51.000 --> 00:31:55.000
So, I double the size of my
table.

00:32:08.000 --> 00:32:13.000
OK, and now I copy all of these
guys and to all of these here.

00:32:13.000 --> 00:32:15.000
And what happens?
Look at that:

00:32:15.000 --> 00:32:19.000
I've got $8,
and I've got eight items that

00:32:19.000 --> 00:32:21.000
have to be copied.
Perfect.

00:32:21.000 --> 00:32:26.000
OK, so one of these dollars
pays for one of the ones that

00:32:26.000 --> 00:32:31.000
was inserted in the last round,
and one of them pays for an old

00:32:31.000 --> 00:32:35.000
one.
OK, and so, I copy them in,

00:32:35.000 --> 00:32:40.000
and now, none of those guys
have any money.

00:32:40.000 --> 00:32:45.000
And the ninth guy comes in:
he has $2 left over.

00:32:45.000 --> 00:32:50.000
And then, we keep going on.
OK, so you see that by that

00:32:50.000 --> 00:32:57.000
argument, if I charge everybody
$3, OK, I can always handle all

00:32:57.000 --> 00:33:01.000
of the table doubling,
the charges for the table

00:33:01.000 --> 00:33:08.000
doubling because the inductive
invariant that I've maintained

00:33:08.000 --> 00:33:13.000
is that after it doubles,
there's nothing in the bank

00:33:13.000 --> 00:33:17.000
account.
And now, I put in $2.

00:33:17.000 --> 00:33:21.000
Well then, I can pay,
and I'm now left in the same

00:33:21.000 --> 00:33:24.000
situation.
OK, and it's the case that the

00:33:24.000 --> 00:33:26.000
bank balance never goes
negative.

00:33:26.000 --> 00:33:31.000
So that's a really important
invariant to verify.

00:33:41.000 --> 00:33:46.000
And so, therefore,
the sum of the true costs,

00:33:46.000 --> 00:33:53.000
or the amortized costs,
upper bound the sum of the true

00:33:53.000 --> 00:33:57.000
costs.
And, since the sum of the

00:33:57.000 --> 00:34:03.000
amortized cost,
here, is, if I go i equals one

00:34:03.000 --> 00:34:09.000
to n, OK, this is 3n.
So, the point is,

00:34:09.000 --> 00:34:16.000
now I bounded the sum of the
true costs by 3n.

00:34:16.000 --> 00:34:25.000
OK, so let's go back to this
table here, and look to see what

00:34:25.000 --> 00:34:35.000
happens, OK, if I put in c_i
hat, and the bank balance.

00:34:35.000 --> 00:34:43.000
OK, so in fact,
so the first thing I do is

00:34:43.000 --> 00:34:53.000
insert; I charge $3,
right, and I do an insert.

00:34:53.000 --> 00:35:01.000
How much do I have left?
I'm going to have $2.

00:35:01.000 --> 00:35:07.000
It turns out I'm actually going
to charge $2 and have only $1

00:35:07.000 --> 00:35:10.000
left.
OK, so I'm actually going to

00:35:10.000 --> 00:35:16.000
under charge the first guy.
I'm going to show you that it

00:35:16.000 --> 00:35:21.000
works if I charge everybody $3.
Except the first guy:

00:35:21.000 --> 00:35:25.000
I charge $2.
I can actually save a little

00:35:25.000 --> 00:35:30.000
bit on number one.
OK, that for this guy I'm going

00:35:30.000 --> 00:35:36.000
to charge $3.
OK, what's the size of my bank

00:35:36.000 --> 00:35:42.000
balance when I'm done?
Well, I have to copy one guy.

00:35:42.000 --> 00:35:47.000
He's all paid for,
so I have $2 left.

00:35:47.000 --> 00:35:51.000
OK, people with me?
OK, the next guy:

00:35:51.000 --> 00:35:56.000
I charge $3.
Actually, I'm going to charge

00:35:56.000 --> 00:36:02.000
all these guys $3.
OK, so here now I basically get

00:36:02.000 --> 00:36:06.000
to, I've got a table of size
four.

00:36:06.000 --> 00:36:09.000
So, I basically have,
I have to copy,

00:36:09.000 --> 00:36:14.000
oh, when I insert the third
guy, I've got to copy two guys.

00:36:14.000 --> 00:36:19.000
That'll use up that,
so I'll have only $2 left in

00:36:19.000 --> 00:36:22.000
the table after I've inserted
him.

00:36:22.000 --> 00:36:27.000
OK, now I insert the fourth
guy, OK, and that's a good one

00:36:27.000 --> 00:36:32.000
because now I've built up a
balance here of $4 because I

00:36:32.000 --> 00:36:41.000
didn't have to copy anybody.
OK, now I insert the fifth guy.

00:36:41.000 --> 00:36:50.000
I've got to copy four items.
So that expends that balance.

00:36:50.000 --> 00:36:54.000
I have, then,
two left.

00:36:54.000 --> 00:37:00.000
OK, and then here basically I
add two to it.

00:37:00.000 --> 00:37:09.000
And then at this point,
I use it all up and go back to

00:37:09.000 --> 00:37:13.000
two, etc.
OK, so you see one of the

00:37:13.000 --> 00:37:18.000
things I want you to notice is I
could have charged three here.

00:37:18.000 --> 00:37:22.000
And then I would've had an
extra dollar lying around

00:37:22.000 --> 00:37:25.000
throughout here.
It wouldn't have mattered.

00:37:25.000 --> 00:37:28.000
It still would be upper bounded
by 3n.

00:37:28.000 --> 00:37:32.000
OK, so the idea is that
different schemes for charging

00:37:32.000 --> 00:37:36.000
amortized costs can work.
They don't all have to be the

00:37:36.000 --> 00:37:39.000
same.
It's not like when you do in

00:37:39.000 --> 00:37:43.000
amortized analysis that there is
one scheme that will work.

00:37:43.000 --> 00:37:45.000
I could have charged $4 to
everybody.

00:37:45.000 --> 00:37:48.000
And it would have worked.
But it turns out,

00:37:48.000 --> 00:37:51.000
I couldn't have charged two
dollars for everybody.

00:37:51.000 --> 00:37:55.000
If I charged $2 for everybody,
my balance would go negative,

00:37:55.000 --> 00:37:57.000
OK?
My balance would go negative,

00:37:57.000 --> 00:38:02.000
but I can charge three dollars,
and that will work.

00:38:02.000 --> 00:38:04.000
OK, four, five,
six, I could charge that.

00:38:04.000 --> 00:38:08.000
The bound that I would get
would be simply a looser bound.

00:38:08.000 --> 00:38:11.000
Instead of it being less than
or equal to 3n,

00:38:11.000 --> 00:38:15.000
it would be less than or equal
to 4n or 5n, or what have you.

00:38:15.000 --> 00:38:19.000
But if I tried to do 2n,
it wouldn't have worked because

00:38:19.000 --> 00:38:22.000
I wouldn't have enough money
left to copy everything.

00:38:22.000 --> 00:38:25.000
What would happen is I would
have only $1 in this,

00:38:25.000 --> 00:38:28.000
and then when it came time to
table double,

00:38:28.000 --> 00:38:30.000
I would need to copy eight
guys.

00:38:30.000 --> 00:38:33.000
And I'd only have built up a
bank account of $4,

00:38:33.000 --> 00:38:38.000
sorry, if I charged $2 and had
$1 left over.

00:38:38.000 --> 00:38:42.000
OK, so to actually make these
things work out,

00:38:42.000 --> 00:38:47.000
you have to play a little bit,
OK, see what works,

00:38:47.000 --> 00:38:53.000
see what doesn't work.
OK, no algorithmic formulas for

00:38:53.000 --> 00:38:55.000
algorithm design.
OK, good.

00:38:55.000 --> 00:38:59.000
In the book,
you can read about table

00:38:59.000 --> 00:39:03.000
contraction.
What happens when you start

00:39:03.000 --> 00:39:06.000
deleting elements?
Now you want to make the table

00:39:06.000 --> 00:39:09.000
be smaller.
Now, you have to be very

00:39:09.000 --> 00:39:12.000
careful because unless you put,
who remembers from physics,

00:39:12.000 --> 00:39:13.000
hysteresis?
Vaguely?

00:39:13.000 --> 00:39:16.000
A couple people?
OK, you have to worry about

00:39:16.000 --> 00:39:18.000
hysteresis.
OK, if you're not careful,

00:39:18.000 --> 00:39:22.000
if whenever it gets to be less
than a power of two,

00:39:22.000 --> 00:39:24.000
you go in the half,
you can find that you're

00:39:24.000 --> 00:39:27.000
thrashing.
So, you need to make it so that

00:39:27.000 --> 00:39:31.000
there is some memory in the
system so that you only collapse

00:39:31.000 --> 00:39:34.000
after you've done a sufficient
number of deletions,

00:39:34.000 --> 00:39:39.000
OK, and so forth.
And the book has analysis of

00:39:39.000 --> 00:39:43.000
the more general case.
OK, so any questions about the

00:39:43.000 --> 00:39:46.000
accounting method?
Accounting method is really

00:39:46.000 --> 00:39:48.000
very cute, OK,
very cute.

00:39:48.000 --> 00:39:51.000
And, it's the one most students
prefer to do,

00:39:51.000 --> 00:39:54.000
OK?
They usually hate the next one

00:39:54.000 --> 00:39:56.000
until they learn it.
Once they learn it,

00:39:56.000 --> 00:40:00.000
they say, ooh,
that's cool.

00:40:00.000 --> 00:40:04.000
OK, but to start out with,
it takes a little bit more

00:40:04.000 --> 00:40:07.000
intestinal fortitude,
OK?

00:40:07.000 --> 00:40:11.000
But it's amazing.
Good potential arguments are

00:40:11.000 --> 00:40:15.000
really sweet.
And we are going to see one

00:40:15.000 --> 00:40:20.000
next time, so you'll definitely
want to review and make sure you

00:40:20.000 --> 00:40:26.000
understand it before Wednesday's
lecture because Wednesday's

00:40:26.000 --> 00:40:32.000
lecture, we're going to assume
we understand potential method.

00:40:32.000 --> 00:40:37.000
OK, so let's do,
enough advertisement.

00:40:37.000 --> 00:40:43.000
I think the potential method is
one of the beautiful results in

00:40:43.000 --> 00:40:48.000
algorithmic analysis,
OK, just beautiful result,

00:40:48.000 --> 00:40:52.000
beautiful set of techniques.
OK, and it's also,

00:40:52.000 --> 00:40:57.000
just in terms of,
I mean, what do you aspire:

00:40:57.000 --> 00:41:02.000
to be a bookkeeper or to be a
physicist?

00:41:02.000 --> 00:41:10.000
OK, so, the idea is we don't
want to be bankers.

00:41:10.000 --> 00:41:16.000
We want to be physicists.

00:41:24.000 --> 00:41:28.000
And so, this bank account,
we are going to say about the

00:41:28.000 --> 00:41:32.000
potential energy of the dynamics
that that we are analyzing.

00:41:32.000 --> 00:41:34.000
OK, because,
why?

00:41:34.000 --> 00:41:38.000
It delivers up work just like a
spring does, for example,

00:41:38.000 --> 00:41:42.000
OK, when you study potential
energy, or putting something up

00:41:42.000 --> 00:41:45.000
high and having gravity pull it
down.

00:41:45.000 --> 00:41:49.000
We convert dynamic to
potential, and that's exactly

00:41:49.000 --> 00:41:53.000
what we're going to be doing
here, and it's similar

00:41:53.000 --> 00:41:58.000
mathematics except that in our
case it turns out to be discrete

00:41:58.000 --> 00:42:04.000
mathematics rather than
continuous math for most of it.

00:42:04.000 --> 00:42:14.000
So here's the framework for the
potential method.

00:42:14.000 --> 00:42:23.000
So, we start with some data
structure, D_0,

00:42:23.000 --> 00:42:35.000
and operation i transforms D_(i
- 1) into D_i.

00:42:35.000 --> 00:42:40.000
So, we view the operation on
the data structure as a mapping,

00:42:40.000 --> 00:42:45.000
mapping one data structure to
another data structure,

00:42:45.000 --> 00:42:48.000
the one from before to the one
after.

00:42:48.000 --> 00:42:51.000
OK, already it's nicely
mathematical.

00:42:51.000 --> 00:42:55.000
OK, and of course,
the costs of operation i

00:42:55.000 --> 00:42:58.000
remains at c_i.

00:43:14.000 --> 00:43:26.000
And, now what we are going to
do is define the potential

00:43:26.000 --> 00:43:35.000
function, phi,
which maps the set of data

00:43:35.000 --> 00:43:46.000
structures into the reals.
So, associated with every data

00:43:46.000 --> 00:43:51.000
structure, now,
is a potential,

00:43:51.000 --> 00:44:00.000
OK, a real-valued potential,
often integer potential such

00:44:00.000 --> 00:44:06.000
that phi of D_0 is equal to
zero.

00:44:06.000 --> 00:44:11.000
So, the initial potential is
zero, and phi of D_i is greater

00:44:11.000 --> 00:44:16.000
than or equal to zero for all i.
So, potential can never be

00:44:16.000 --> 00:44:22.000
nonnegative, just like the bank
account because the potential is

00:44:22.000 --> 00:44:25.000
essentially representing the
bank account,

00:44:25.000 --> 00:44:29.000
if you will,
in the accounting method.

00:44:29.000 --> 00:44:35.000
OK, so we always want the
potential to be nonnegative.

00:44:35.000 --> 00:44:37.000
Now, actually,
there are times where you use

00:44:37.000 --> 00:44:40.000
potential functions where you
violate both of these

00:44:40.000 --> 00:44:43.000
properties.
There's some really interesting

00:44:43.000 --> 00:44:46.000
potential arguments which don't
violate like these,

00:44:46.000 --> 00:44:50.000
but for the simple ones we're
going to be doing in this class,

00:44:50.000 --> 00:44:52.000
we'll just assume that these
tend to be true.

00:44:52.000 --> 00:44:56.000
OK, but we will actually see
some times where phi of D_0

00:44:56.000 --> 00:45:00.000
isn't zero, it doesn't matter.
OK, but generally this is what

00:45:00.000 --> 00:45:03.000
we are going to assume in the
type of potential function

00:45:03.000 --> 00:45:07.000
argument that we are going to be
doing.

00:45:07.000 --> 00:45:12.000
OK, so I just want to let you
know that there are bigger,

00:45:12.000 --> 00:45:18.000
there is a bigger space of
potential function arguments

00:45:18.000 --> 00:45:22.000
than the one that I'm showing
you here.

00:45:22.000 --> 00:45:25.000
OK, so then,
under this circumstance,

00:45:25.000 --> 00:45:31.000
we define the amortized cost
c_i hat with respect to phi as,

00:45:31.000 --> 00:45:36.000
and this is one of these
formulas that if you can't

00:45:36.000 --> 00:45:42.000
remember it, definitely put it
down on your crib sheet for the

00:45:42.000 --> 00:45:51.000
final.
OK, so c_i hat is equal to c_i

00:45:51.000 --> 00:46:01.000
plus phi of D_i minus phi of D_i
minus one.

00:46:01.000 --> 00:46:12.000
OK, so this is the change in
potential difference.

00:46:12.000 --> 00:46:22.000
And, let's call it delta phi i
for shorthand.

00:46:22.000 --> 00:46:33.000
OK, and let's see what it means
to have --

00:46:43.000 --> 00:46:48.000
-- in the different
circumstances.

00:46:48.000 --> 00:46:55.000
So, if delta of phi i is
greater than zero,

00:46:55.000 --> 00:47:03.000
OK, so if this is greater than
zero, then what's the

00:47:03.000 --> 00:47:11.000
relationship between c_i hat and
c_i?

00:47:11.000 --> 00:47:19.000
This is greater than zero.
Yeah, c_i hat is then greater

00:47:19.000 --> 00:47:25.000
than c_i, OK?
Then, c_i hat is greater than

00:47:25.000 --> 00:47:30.000
c_i.
And what does that mean?

00:47:30.000 --> 00:47:39.000
That means when I do operation
I, I charged more than it cost

00:47:39.000 --> 00:47:48.000
me to do the operation.
So, the extra amount that I

00:47:48.000 --> 00:47:58.000
charged beyond what I actually
used is being put into the bank,

00:47:58.000 --> 00:48:04.000
OK, is being stored as
potential energy.

00:48:04.000 --> 00:48:15.000
So, op I stores work in the
data structure for later.

00:48:15.000 --> 00:48:22.000
Similarly, if delta phi i is
less than zero,

00:48:22.000 --> 00:48:33.000
then c_i hat is less than c_i.
And so, the data structure

00:48:33.000 --> 00:48:38.000
delivers up work --

00:48:52.000 --> 00:49:03.000
-- to help pay for op I,
OK, for operation I.

00:49:03.000 --> 00:49:08.000
So, if it's less than zero,
that means that my change in

00:49:08.000 --> 00:49:14.000
potential, that means my bank
account went down as a result,

00:49:14.000 --> 00:49:18.000
and so therefore,
what happens was the data

00:49:18.000 --> 00:49:23.000
structure provided work to be
done in order,

00:49:23.000 --> 00:49:30.000
because the true cost was
bigger than the amortized cost.

00:49:30.000 --> 00:49:33.000
So, if you think about it,
the difference between looking

00:49:33.000 --> 00:49:37.000
at it from the potential
function point of view versus

00:49:37.000 --> 00:49:41.000
the accounting point of view,
the accounting point of view,

00:49:41.000 --> 00:49:44.000
you sort of say,
here is what my amortized cost

00:49:44.000 --> 00:49:46.000
will be.
Now let me analyze my bank

00:49:46.000 --> 00:49:49.000
account, make sure it never went
negative.

00:49:49.000 --> 00:49:52.000
In some sense,
in the potential function

00:49:52.000 --> 00:49:56.000
argument, you're saying,
here's what my bank account is

00:49:56.000 --> 00:50:01.000
all the time.
Now let me analyze what the

00:50:01.000 --> 00:50:06.000
amortized costs are.
So, that's sort of the

00:50:06.000 --> 00:50:12.000
difference in approaches.
One is you are sort of

00:50:12.000 --> 00:50:19.000
specifying the bank account.
The other, you're specifying

00:50:19.000 --> 00:50:24.000
the amortized costs.
So, we look at the,

00:50:24.000 --> 00:50:32.000
why is it that this is a
reasonable way to proceed?

00:50:32.000 --> 00:50:41.000
Well, let's look at the total
amortized cost of n operations.

00:50:41.000 --> 00:50:49.000
OK, that's just the sum,
i equals one to n of c_i hat.

00:50:49.000 --> 00:50:53.000
That's the total amortized
cost.

00:50:53.000 --> 00:50:59.000
And that's equal to,
but substitution,

00:50:59.000 --> 00:51:07.000
just substitute c_i hat for
this formula.

00:51:07.000 --> 00:51:24.000
OK, so that's c_i plus phi of
D_i minus phi of D_i minus one.

00:51:24.000 --> 00:51:36.000
OK, and that's equal to c_i.
And now, what happens when I

00:51:36.000 --> 00:51:41.000
sum up these terms?
What happens when some of these

00:51:41.000 --> 00:51:45.000
terms?
What's the mathematical term we

00:51:45.000 --> 00:51:47.000
use?
It telescopes.

00:51:47.000 --> 00:51:53.000
OK, every term on the left is
added in once when it's I,

00:51:53.000 --> 00:52:00.000
and subtract it out when it's I
minus one, except for the first

00:52:00.000 --> 00:52:08.000
and last terms.
The term for n is only added

00:52:08.000 --> 00:52:16.000
in, and the term for zero is
only subtracted out.

00:52:16.000 --> 00:52:23.000
OK, so that's because it
telescopes.

00:52:23.000 --> 00:52:32.000
OK, so this term is what?
What property do we know of

00:52:32.000 --> 00:52:38.000
this?
It's greater than or equal to

00:52:38.000 --> 00:52:42.000
zero.
And this one equals zero.

00:52:42.000 --> 00:52:48.000
So, therefore,
this is greater than or equal

00:52:48.000 --> 00:52:53.000
to c_i.
And thus, the amortized costs

00:52:53.000 --> 00:53:02.000
are an upper bound on the true
costs, which is what we want.

00:53:02.000 --> 00:53:06.000
Some of the amortized costs is
an upper bound up on the sum of

00:53:06.000 --> 00:53:08.000
the true costs.
OK, but here,

00:53:08.000 --> 00:53:11.000
the way that we define the
amortized costs was by first

00:53:11.000 --> 00:53:15.000
defining the potential function.
OK, so the potential function

00:53:15.000 --> 00:53:19.000
is sort of, as I said,
the difference between the

00:53:19.000 --> 00:53:23.000
accounting and the potential
method is, do you specify the

00:53:23.000 --> 00:53:25.000
bank account or do you specify
the cost?

00:53:25.000 --> 00:53:29.000
OK, do you specify the
potential energy at any point,

00:53:29.000 --> 00:53:33.000
or do you specify the cost at
any point?

00:53:33.000 --> 00:53:38.000
OK, but in any case,
you get, this bound,

00:53:38.000 --> 00:53:44.000
also this math is nicer math.
I like telescopes.

00:53:44.000 --> 00:53:51.000
OK, so the amortized costs
upper bound the true costs.

00:53:51.000 --> 00:53:57.000
OK, let's do table doubling
over here.

00:54:20.000 --> 00:54:24.000
So, to analyze this,
we have to define our

00:54:24.000 --> 00:54:28.000
potential.
OK, if anybody can guess this

00:54:28.000 --> 00:54:35.000
off the top of their head,
they're better than I am.

00:54:35.000 --> 00:54:42.000
I struggled with this for
probably easily a couple hours

00:54:42.000 --> 00:54:48.000
to get it right,
OK, because I'm not too smart.

00:54:48.000 --> 00:54:55.000
OK, that's a potential function
I'm going to use,

00:54:55.000 --> 00:55:02.000
OK, 2i minus two to the ceiling
of log i.

00:55:02.000 --> 00:55:07.000
And, we're going to assume that
two to the ceiling of log of

00:55:07.000 --> 00:55:13.000
zero is equal to zero,
because that's what it is in

00:55:13.000 --> 00:55:15.000
the limit.
For log of zero,

00:55:15.000 --> 00:55:21.000
this becomes minus infinity,
so, two to the minus infinity

00:55:21.000 --> 00:55:25.000
is zero.
So, that's just going to be a

00:55:25.000 --> 00:55:30.000
mathematical convenience.
Assume that.

00:55:30.000 --> 00:55:32.000
OK, so where did I get this
from?

00:55:32.000 --> 00:55:35.000
I played around.
I looked at that sequence that

00:55:35.000 --> 00:55:38.000
I have erased and I said,
OK, because reversing,

00:55:38.000 --> 00:55:41.000
there are some problems for
which defining a potential

00:55:41.000 --> 00:55:45.000
function is fairly easy.
But, defining the amortized

00:55:45.000 --> 00:55:47.000
costs is hard,
OK, to define the accounting.

00:55:47.000 --> 00:55:50.000
So, for this one,
the accounting method is,

00:55:50.000 --> 00:55:53.000
I would say,
an easier method to use.

00:55:53.000 --> 00:55:57.000
However, I'm going to show you
that you still can do it with

00:55:57.000 --> 00:56:02.000
potential method if you come up
with the right potential.

00:56:02.000 --> 00:56:07.000
So, intuitively,
this is basically what's left

00:56:07.000 --> 00:56:14.000
in the bank account at the I'th
operation because I've put in 2i

00:56:14.000 --> 00:56:20.000
things into the bank,
and I've subtracted out this

00:56:20.000 --> 00:56:24.000
many, essentially,
from table doublings,

00:56:24.000 --> 00:56:27.000
OK, up to that point,
OK?

00:56:27.000 --> 00:56:34.000
So, first let's observe,
what is phi of D_0?

00:56:34.000 --> 00:56:36.000
Zero.
So, that's good.

00:56:36.000 --> 00:56:43.000
And, the phi of D_i is greater
than or equal to zero.

00:56:43.000 --> 00:56:45.000
Why is that?

00:56:58.000 --> 00:56:59.000
Why is that?

00:57:11.000 --> 00:57:20.000
So, what's the biggest that
ceiling of log i could be?

00:57:20.000 --> 00:57:30.000
Ceiling of log i is either log
i or log i quantity plus one,

00:57:30.000 --> 00:57:34.000
OK?
So, the biggest it is,

00:57:34.000 --> 00:57:40.000
is log i plus one.
If it's log i plus one,

00:57:40.000 --> 00:57:45.000
two to the log i plus one,
it's just 2i.

00:57:45.000 --> 00:57:50.000
That's the biggest it could be,
right?

00:57:50.000 --> 00:57:55.000
So, two, the log of i,
plus one, is just,

00:57:55.000 --> 00:58:01.000
let's do it the other way,
is i times two,

00:58:01.000 --> 00:58:08.000
OK, i for that part,
two for that part.

00:58:08.000 --> 00:58:12.000
OK, so that the biggest it
could be.

00:58:12.000 --> 00:58:20.000
OK, or it's just log of i.
So, either this is going to be

00:58:20.000 --> 00:58:25.000
2i minus i or 2i minus 2i.
In either case,

00:58:25.000 --> 00:58:30.000
it's bigger than zero,
OK?

00:58:30.000 --> 00:58:34.000
So, those are the two
properties I need for this to be

00:58:34.000 --> 00:58:36.000
a well-defined potential
function.

00:58:36.000 --> 00:58:41.000
Now, that doesn't say that the
amortized costs are going to be

00:58:41.000 --> 00:58:45.000
satisfied the property that
things are going to be cheap,

00:58:45.000 --> 00:58:50.000
OK, that I'm going to be able
to do my analysis and get the

00:58:50.000 --> 00:58:53.000
kind of bounds I want.
But, it sets up to say,

00:58:53.000 --> 00:58:58.000
yes, I've satisfied the syntax
of having a proper potential

00:58:58.000 --> 00:59:03.000
function.
So, let's just do a quick

00:59:03.000 --> 00:59:07.000
example here just to see what
this means.

00:59:07.000 --> 00:59:14.000
So, imagine that I am in the
situation where I have eight,

00:59:14.000 --> 00:59:19.000
did I do that right,
OK, yeah, eight slots,

00:59:19.000 --> 00:59:26.000
and say six of them are full.
So then, phi by this is going

00:59:26.000 --> 00:59:31.000
to be 2i.
That's two times six minus two

00:59:31.000 --> 00:59:35.000
to the 2i.
What's that?

00:59:35.000 --> 00:59:41.000
Sorry, minus two to the ceiling
of log i.

00:59:41.000 --> 00:59:46.000
So, i is six,
right, so log of i is log of

00:59:46.000 --> 00:59:50.000
six.
The ceiling of it is three.

00:59:50.000 --> 00:59:55.000
So, that's minus 2^3,
which is eight.

00:59:55.000 --> 01:00:02.000
So, that's 12 minus eight.
That's four.

01:00:02.000 --> 01:00:05.875
And if you think about this in
the accounting method,

01:00:05.875 --> 01:00:09.228
these would be zeros,
and these would be twos,

01:00:09.228 --> 01:00:13.401
right, for the accounting
method if we do the same thing,

01:00:13.401 --> 01:00:16.233
because this is halfway
through, right,

01:00:16.233 --> 01:00:20.629
all zeros, and that we add two
for each one that we're going

01:00:20.629 --> 01:00:22.194
in.
So, I function is,

01:00:22.194 --> 01:00:25.399
in fact, telling me what the
actual cost is.

01:00:25.399 --> 01:00:29.199
OK, everybody with me?
OK, so that's what we mean by

01:00:29.199 --> 01:00:33.000
this particular potential
function.

01:00:33.000 --> 01:00:37.274
OK, so now let's add up the
amortized cost of the i'th

01:00:37.274 --> 01:00:38.000
insert.

01:01:09.000 --> 01:01:14.190
OK, so that's the amortized
cost of the i'th insert,

01:01:14.190 --> 01:01:18.973
just by definition.
OK, and now that's equal to,

01:01:18.973 --> 01:01:23.756
well, what is c_i?
Do we still have that written

01:01:23.756 --> 01:01:28.336
down somewhere,
or have we erased that at this

01:01:28.336 --> 01:01:32.000
point?
I think we erased it.

01:01:32.000 --> 01:01:37.480
Well, we can write it down
again.

01:01:37.480 --> 01:01:45.530
It is i if i minus one is an
exact power of two.

01:01:45.530 --> 01:01:51.524
And, it's one otherwise.
That's c_i.

01:01:51.524 --> 01:01:58.718
That's this term,
plus, and now phi of D_i:

01:01:58.718 --> 01:02:10.830
so, what is that?
phi of D_i is this business,

01:02:10.830 --> 01:02:26.771
2i minus two ceiling of log i
minus 2i minus one minus two

01:02:26.771 --> 01:02:39.634
ceiling of log of I minus one.
OK, so that's the amortized

01:02:39.634 --> 01:02:42.192
cost.
That's a nice,

01:02:42.192 --> 01:02:45.019
pretty formula,
right?

01:02:45.019 --> 01:02:50.000
OK, let's hope it simplifies a
little.

01:02:50.000 --> 01:02:57.269
OK, so that's equal to,
well, we have the i and the one

01:02:57.269 --> 01:03:01.173
here, if, etc.,
that business,

01:03:01.173 --> 01:03:06.692
plus, OK, well,
we have some things we can

01:03:06.692 --> 01:03:16.662
cancel here, right?
So here, we have 2i minus 2i.

01:03:16.662 --> 01:03:24.668
That cancels.
And then we have what's left

01:03:24.668 --> 01:03:33.846
over here is a minus two.
So, that's a plus two.

01:03:33.846 --> 01:03:44.000
And now, I have minus this term
plus this term.

01:03:44.000 --> 01:03:48.049
That's a lot prettier.
OK, it's still a mess.

01:03:48.049 --> 01:03:53.294
We've got to case analysis.
Why is it suggestive of a case

01:03:53.294 --> 01:03:55.503
analysis?
We have a case.

01:03:55.503 --> 01:03:59.000
OK, so let's do a case
analysis.

01:04:11.000 --> 01:04:17.966
OK, so case one,
I minus one is an exact power

01:04:17.966 --> 01:04:23.538
of two.
So then, c_i hat is equal to,

01:04:23.538 --> 01:04:29.887
well, c_i is now just i.
That's that case.

01:04:29.887 --> 01:04:36.234
And then we have the rest
there, plus two,

01:04:36.234 --> 01:04:44.903
minus two, ceiling of log i
minus two ceiling of log of i

01:04:44.903 --> 01:04:53.089
minus one.
OK, and that's equal to i plus

01:04:53.089 --> 01:04:56.482
two.
Well, let's see.

01:04:56.482 --> 01:05:03.098
If I minus one is an exact
power of two,

01:05:03.098 --> 01:05:12.236
what is this term?
i minus one is an exact power

01:05:12.236 --> 01:05:16.388
of two.
Plus, thank you,

01:05:16.388 --> 01:05:20.000
sorry about that.

01:05:33.000 --> 01:05:36.916
It's good to have students,
let me say, because,

01:05:36.916 --> 01:05:41.250
boy, my math is so bad.
This is actually why I end up

01:05:41.250 --> 01:05:46.333
being a pretty good theoretician
is because I don't ever trust

01:05:46.333 --> 01:05:50.583
what I've written down.
And so, I write it down in a

01:05:50.583 --> 01:05:55.666
way that I can verify it because
otherwise I just am not smart

01:05:55.666 --> 01:06:00.416
enough to carry through five
equations in a row and expect

01:06:00.416 --> 01:06:06.000
that everyone is going to be
transformed appropriately.

01:06:06.000 --> 01:06:11.529
OK, so you write it down.
So I always write it down so I

01:06:11.529 --> 01:06:15.754
can verify it.
And that fortunately has the

01:06:15.754 --> 01:06:21.988
side benefit that other people
can understand what I've done as

01:06:21.988 --> 01:06:24.904
well.
OK, so what is this one?

01:06:24.904 --> 01:06:30.636
This is two to the log of i
minus one because the ceiling,

01:06:30.636 --> 01:06:36.569
if this is an exact power of
two, right, then ceiling of log

01:06:36.569 --> 01:06:42.000
of i minus one is just log of i
minus one.

01:06:42.000 --> 01:06:52.013
So, this is two to the log of i
minus one, which is i minus one,

01:06:52.013 --> 01:06:53.761
right?
Yeah?

01:06:53.761 --> 01:06:58.846
OK.
OK, if it's an exact power of

01:06:58.846 --> 01:07:06.000
two, then the log is an integer,
right?

01:07:06.000 --> 01:07:09.713
So, taking the ceiling,
it doesn't matter,

01:07:09.713 --> 01:07:12.973
get rid of the ceiling.
OK, this one,

01:07:12.973 --> 01:07:16.324
however, is not an exact power
of two.

01:07:16.324 --> 01:07:20.218
But what is it?
It's just one more than this

01:07:20.218 --> 01:07:23.388
guy.
We know that i minus one is not

01:07:23.388 --> 01:07:27.916
an exact power of two,
so it's going to be the next

01:07:27.916 --> 01:07:36.053
bigger one.
OK, so that means this is what?

01:07:36.053 --> 01:07:45.035
So, it's going to be,
how do these two compare?

01:07:45.035 --> 01:07:53.041
How much bigger is this one
than this one?

01:07:53.041 --> 01:08:03.000
It's twice the size.
We know what this one is.

01:08:03.000 --> 01:08:06.656
OK, or it can reason it from
first principles.

01:08:06.656 --> 01:08:10.800
This is going to be the log of
i minus one plus one.

01:08:10.800 --> 01:08:14.131
OK, and so then you can reduce
it to this.

01:08:14.131 --> 01:08:18.600
So, you've got to think about
those floors and ceilings,

01:08:18.600 --> 01:08:21.524
right?
It's like, what's happening in

01:08:21.524 --> 01:08:25.261
the round off there?
OK, so now we can simplify

01:08:25.261 --> 01:08:29.000
this.
OK, so what do we have here?

01:08:29.000 --> 01:08:34.136
We have, OK,
so if I multiply this through,

01:08:34.136 --> 01:08:40.863
I have an i plus two minus 2i
plus two plus i minus one.

01:08:40.863 --> 01:08:46.611
I know a lot of you,
probably 90% of you will do

01:08:46.611 --> 01:08:51.502
this step.
You will go directly from this

01:08:51.502 --> 01:08:57.618
step to the last step.
And that's where 30% of you,

01:08:57.618 --> 01:09:03.000
or some number,
will get it wrong.

01:09:03.000 --> 01:09:06.818
OK, so let me encourage you to
do that step.

01:09:06.818 --> 01:09:11.526
OK, it's easier to find your
bugs if you take it slow.

01:09:11.526 --> 01:09:15.966
Actually, taking it slow is
faster in the long run.

01:09:15.966 --> 01:09:20.140
It's hard to teach young
stallions, or phillies,

01:09:20.140 --> 01:09:21.473
or whatever,
OK?

01:09:21.473 --> 01:09:24.315
Just take it easy.
Just patience,

01:09:24.315 --> 01:09:26.890
just do it slow,
get it right.

01:09:26.890 --> 01:09:32.670
It's actually faster.
OK, everybody knows the

01:09:32.670 --> 01:09:37.432
tortoise and hare story.
Yeah, yeah, yeah,

01:09:37.432 --> 01:09:43.586
OK, but nobody believes it.
OK, so now here we have 2i

01:09:43.586 --> 01:09:47.069
here, an i here,
and an i here.

01:09:47.069 --> 01:09:53.109
And then, that leaves us with
two plus two minus one,

01:09:53.109 --> 01:09:56.593
equals three.
Awesome, awesome.

01:09:56.593 --> 01:10:02.980
OK, amortized cost is three
when i minus one is an exact

01:10:02.980 --> 01:10:08.000
power of two.
OK, case two.

01:10:30.000 --> 01:10:34.829
OK, i minus one is not an exact
power of two.

01:10:34.829 --> 01:10:41.085
So then we have c_i hat is
equal to, now instead of i it's

01:10:41.085 --> 01:10:48.109
one plus, and then the two minus
two to the ceiling of log i plus

01:10:48.109 --> 01:10:52.500
two to the ceiling of log of i
minus one.

01:10:52.500 --> 01:10:59.195
OK, now what can somebody tell
me about these two terms in the

01:10:59.195 --> 01:11:06.000
case where i minus one is not an
exact power of two?

01:11:06.000 --> 01:11:09.216
What are they?
Equal.

01:11:09.216 --> 01:11:15.809
Why is that?
Yeah, the ceiling is going to

01:11:15.809 --> 01:11:24.974
do the same thing to both,
going to take it up to the same

01:11:24.974 --> 01:11:31.085
integer.
So these two things are equal,

01:11:31.085 --> 01:11:38.000
which means this is equal to
three.

01:11:38.000 --> 01:11:42.405
OK, so therefore,
n inserts, OK,

01:11:42.405 --> 01:11:48.657
so now I say,
oh, the amortized cost is three

01:11:48.657 --> 01:11:53.773
for every operation for every
insert.

01:11:53.773 --> 01:11:57.894
So therefore,
n inserts costs,

01:11:57.894 --> 01:12:05.000
well, the amortized cost of
each is three.

01:12:05.000 --> 01:12:08.986
So n of them,
the amortized cost is 3n.

01:12:08.986 --> 01:12:14.230
That's an upper bound on the
worst-case true costs.

01:12:14.230 --> 01:12:18.951
So, n inserts costs order n in
the worst case.

01:12:18.951 --> 01:12:22.622
OK, there is a bug in this
analysis.

01:12:22.622 --> 01:12:27.132
It's a minor bug.
It's the one I pointed out

01:12:27.132 --> 01:12:32.272
before the first insert has
amortized cost of two,

01:12:32.272 --> 01:12:37.182
and not three.
I didn't actually deal with

01:12:37.182 --> 01:12:41.547
that one carefully enough.
OK, so that's an exercise to

01:12:41.547 --> 01:12:46.721
just go and look to see where it
is that that happens and how you

01:12:46.721 --> 01:12:50.763
show that, in fact,
the amortized cost of the first

01:12:50.763 --> 01:12:53.592
one is two, OK,
where that shows up.

01:12:53.592 --> 01:12:57.634
OK, so to summarize,
actually let me summarize over

01:12:57.634 --> 01:13:02.000
here, conclusions about
amortized analysis.

01:13:17.000 --> 01:13:35.000
So amortized costs provide a
clean abstraction for data

01:13:35.000 --> 01:13:45.556
structure performance.
So, what I can tell somebody,

01:13:45.556 --> 01:13:49.699
so suppose I built a dynamic
table, for example,

01:13:49.699 --> 01:13:51.638
OK.
It's easier to say,

01:13:51.638 --> 01:13:55.252
in terms of your own
performance modeling,

01:13:55.252 --> 01:13:59.659
it costs a constant amount of
time for each insert.

01:13:59.659 --> 01:14:04.066
As long as you don't care about
real-time behavior,

01:14:04.066 --> 01:14:08.121
but only the aggregate
behavior, that's a great

01:14:08.121 --> 01:14:14.156
abstraction for the performance
Rather than saying it's got

01:14:14.156 --> 01:14:18.390
that complicated thing which
sometimes cost you a lot,

01:14:18.390 --> 01:14:22.624
how do they reason about that?
But you could say every

01:14:22.624 --> 01:14:26.618
operation costs me order one,
that's really simple.

01:14:26.618 --> 01:14:31.251
But they have to understand,
it's order one in an amortized

01:14:31.251 --> 01:14:34.526
sense, OK.
So, if they do have a real-time

01:14:34.526 --> 01:14:39.000
constraint to make,
amortized doesn't cut it.

01:14:39.000 --> 01:14:42.159
OK, but for many problems,
it's perfectly good.

01:14:42.159 --> 01:14:44.563
It lets me explain it rather
simply.

01:14:44.563 --> 01:14:48.752
OK, we will see some other data
structures that have amortized

01:14:48.752 --> 01:14:53.079
costs where different operations
have different amortized costs.

01:14:53.079 --> 01:14:56.239
And the nice thing about that
is I just add up,

01:14:56.239 --> 01:14:59.398
what's the cost of all my
different operations,

01:14:59.398 --> 01:15:04.000
OK, where there is a different
cost for each operation?

01:15:04.000 --> 01:15:07.068
Some will be log n.
Some will be order one,

01:15:07.068 --> 01:15:08.822
or whatever.
Add them up:

01:15:08.822 --> 01:15:11.672
that's an upper bound on the
true costs.

01:15:11.672 --> 01:15:15.837
OK: tremendous simplification
in abstracting and reasoning

01:15:15.837 --> 01:15:18.686
about those complicated data
structures.

01:15:18.686 --> 01:15:21.755
OK, now, so this is probably,
this is huge.

01:15:21.755 --> 01:15:24.897
OK, abstraction,
you know, computer science,

01:15:24.897 --> 01:15:28.551
what teach you through four
years of undergraduate,

01:15:28.551 --> 01:15:33.008
and another year if you go on
to M.Eng., and then if you get a

01:15:33.008 --> 01:15:39.000
Ph.D., it's another 15 years or
whatever it takes to get a Ph.D.

01:15:39.000 --> 01:15:44.392
OK, all you teach about is
abstraction: abstraction,

01:15:44.392 --> 01:15:46.929
abstraction,
abstraction.

01:15:46.929 --> 01:15:51.792
So, this is a powerful
abstraction: quite good.

01:15:51.792 --> 01:15:58.030
Now, we learned three methods.
In general, any method can be

01:15:58.030 --> 01:16:01.308
used.
You can convert one to the

01:16:01.308 --> 01:16:10.386
other.
But each has situations where

01:16:10.386 --> 01:16:20.320
it is arguably simplest or most
precise.

01:16:20.320 --> 01:16:30.000
So, any of the methods can be
used.

01:16:30.000 --> 01:16:34.299
However, you must learn all of
them, OK, because there are

01:16:34.299 --> 01:16:37.768
going to be some situations
where you need one,

01:16:37.768 --> 01:16:41.464
where it's better to do one,
better to do another.

01:16:41.464 --> 01:16:45.386
If you're reading in the
literature, you want to know

01:16:45.386 --> 01:16:47.800
these different ways of doing
it.

01:16:47.800 --> 01:16:52.401
And that means that even though
you may get really comfortable

01:16:52.401 --> 01:16:54.739
with accounting,
OK, in an exam,

01:16:54.739 --> 01:16:57.681
or whatever,
I may say solve this with a

01:16:57.681 --> 01:17:07.441
potential function argument.
So, you want to be comfortable

01:17:07.441 --> 01:17:12.595
with all the methods,
OK.

01:17:12.595 --> 01:17:22.687
Last point is that,
in fact, different potential

01:17:22.687 --> 01:17:36.000
functions or accounting costs
may yield different bounds.

01:17:36.000 --> 01:17:41.628
OK, so when you do an amortized
analysis, there's nothing to say

01:17:41.628 --> 01:17:45.559
that one set of costs is better
than another.

01:17:45.559 --> 01:17:49.847
So, just as an example,
OK, in any data structure

01:17:49.847 --> 01:17:55.296
generally that supports delete,
I can amortize all the deletes

01:17:55.296 --> 01:17:58.423
against the inserts.
So, in general,

01:17:58.423 --> 01:18:02.265
what I could do is say deletes
are free, OK,

01:18:02.265 --> 01:18:07.000
and charge twice as much for
each insert.

01:18:07.000 --> 01:18:10.338
OK, charging enough when I do
the insert to amortize it

01:18:10.338 --> 01:18:13.059
against the delete.
That you could do with an

01:18:13.059 --> 01:18:15.717
accounting method.
You can also do it with a

01:18:15.717 --> 01:18:17.634
potential method,
the potential,

01:18:17.634 --> 01:18:21.467
then, being the number of items
that I actually have in my data

01:18:21.467 --> 01:18:25.300
structure times the cost of the
delete for each of those items.

01:18:25.300 --> 01:18:28.825
OK, so the point is that I can
allocate costs in different

01:18:28.825 --> 01:18:31.919
ways.
Or I could have amortized costs

01:18:31.919 --> 01:18:33.962
which are equal to the true
costs.

01:18:33.962 --> 01:18:37.491
So, there are different ways
that I could assign amortized

01:18:37.491 --> 01:18:39.101
costs.
There is no one way,

01:18:39.101 --> 01:18:42.692
OK, and choosing different ones
may yield different bounds.

01:18:42.692 --> 01:18:45.478
It may not, but it may yield
different bounds.

01:18:45.478 --> 01:18:48.079
OK, generally it does yield
different ones.

01:18:48.079 --> 01:18:50.617
OK, next time:
an amazing use of potential

01:18:50.617 --> 01:18:52.598
functions.
OK, the stuff is cool,

01:18:52.598 --> 01:18:55.075
but let me tell you,
Wednesday's lecture:

01:18:55.075 --> 01:18:57.490
amazing.
Amazing the type of analysis we

01:18:57.490 --> 01:19:00.000
are going to be able to do.