1
00:00:14 --> 00:00:18
Good morning everyone.
Today we are going to do some
2
00:00:18 --> 00:00:23
algorithms, back to algorithms,
and we are going to use a lot
3
00:00:23 --> 00:00:27
of the, well,
some of the simpler mathematics
4
00:00:27 --> 00:00:31
that we developed last class
like the master theorem for
5
00:00:31 --> 00:00:35
solving recurrences.
We are going to use this a lot.
6
00:00:35 --> 00:00:39
Because we are going to talk
about recursive algorithms
7
00:00:39 --> 00:00:41
today.
And so we will find their
8
00:00:41 --> 00:00:43
running time using the master
theorem.
9
00:00:43 --> 00:00:46
This is just the same as it was
last time, I hope,
10
00:00:46 --> 00:00:49
unless I made a mistake.
A couple of reminders.
11
00:00:49 --> 00:00:52
You should all go to recitation
on Friday.
12
00:00:52 --> 00:00:54
That is required.
If you want to,
13
00:00:54 --> 00:00:56
you can go to homework lab on
Sunday.
14
00:00:56 --> 00:00:59
That may be a good excuse for
you to actually work on your
15
00:00:59 --> 00:01:04
problem set a few hours early.
Well, actually,
16
00:01:04 --> 00:01:07
it's due on Wednesday so you
have lots of time.
17
00:01:07 --> 00:01:10
And there is no class on
Monday.
18
00:01:10 --> 00:01:13
It is the holiday known as
Student Holiday,
19
00:01:13 --> 00:01:16
so don't come.
Today we are going to cover
20
00:01:16 --> 00:01:20
something called Divide and
Conquer.
21
00:01:20 --> 00:01:27
22
00:01:27 --> 00:01:31
Also known as divide and rule
or divide et impera for those of
23
00:01:31 --> 00:01:34
you who know Latin,
which is a tried and tested way
24
00:01:34 --> 00:01:39
of conquering a land by dividing
it into sections of some kind.
25
00:01:39 --> 00:01:43
It could be different political
factions, different whatever.
26
00:01:43 --> 00:01:47
And then somehow making them no
longer like each other.
27
00:01:47 --> 00:01:50
Like starting a family feud is
always a good method.
28
00:01:50 --> 00:01:53
You should remember this on
your quiz.
29
00:01:53 --> 00:01:56
I'm kidding.
And if you can separate this
30
00:01:56 --> 00:02:00
big power structure into little
power structures such that you
31
00:02:00 --> 00:02:03
dominate each little power
structure then you can conquer
32
00:02:03 --> 00:02:06
all of them individually,
as long as you make sure they
33
00:02:06 --> 00:02:09
don't get back together.
That is divide and conquer as
34
00:02:09 --> 00:02:11
practiced, say,
by the British.
35
00:02:11 --> 00:02:14
But today we are going to do
divide and conquer as practiced
36
00:02:14 --> 00:02:17
in Cormen, Leiserson,
Rivest and Stein or every other
37
00:02:17 --> 00:02:20
algorithm textbook.
This is a very basic and very
38
00:02:20 --> 00:02:22
powerful algorithm design
technique.
39
00:02:22 --> 00:02:27
So, this is our first real
algorithm design experience.
40
00:02:27 --> 00:02:30
We are still sort of mostly in
the analysis mode,
41
00:02:30 --> 00:02:33
but we are going to do some
actual design.
42
00:02:33 --> 00:02:37
We're going to cover maybe only
three or four major design
43
00:02:37 --> 00:02:39
techniques.
This is one of them,
44
00:02:39 --> 00:02:43
so it is really important.
And it will lead to all sorts
45
00:02:43 --> 00:02:45
of recurrences,
so we will get to use
46
00:02:45 --> 00:02:49
everything from last class and
see why it is useful.
47
00:02:49 --> 00:02:51
As you might expect,
the first step in
48
00:02:51 --> 00:02:57
divide-and-conquer is divide and
the second step is conquer.
49
00:02:57 --> 00:03:01
But you may not have guessed
that there are three steps.
50
00:03:01 --> 00:03:04
And I am leaving some blank
space here, so you should,
51
00:03:04 --> 00:03:06
too.
Divide-and-conquer is an
52
00:03:06 --> 00:03:10
algorithmic technique.
You are given some big problem
53
00:03:10 --> 00:03:14
you want to solve,
you don't really know how to
54
00:03:14 --> 00:03:18
solve it in an efficient way,
so you are going to split it up
55
00:03:18 --> 00:03:22
into subproblems.
That is the divide.
56
00:03:22 --> 00:03:26
You are going to divide that
problem, or more precisely the
57
00:03:26 --> 00:03:30
instance of that problem,
the particular instance of that
58
00:03:30 --> 00:03:32
problem you have into
subproblems.
59
00:03:32 --> 00:03:36
And those subproblems should be
smaller in some sense.
60
00:03:36 --> 00:03:41
And smaller means normally that
the value of N is smaller than
61
00:03:41 --> 00:03:45
it was in the original problem.
So, you sort of made some
62
00:03:45 --> 00:03:47
progress.
Now you have one,
63
00:03:47 --> 00:03:51
or more likely you have several
subproblems you need to solve.
64
00:03:51 --> 00:03:55
Each of them is smaller.
So, you recursively solve each
65
00:03:55 --> 00:04:00
subproblem.
That is the conquer step.
66
00:04:00 --> 00:04:04
You conquer each subproblem
recursively.
67
00:04:04 --> 00:04:11
And then somehow you combine
those solutions into a solution
68
00:04:11 --> 00:04:14
for the whole problem.
69
00:04:14 --> 00:04:23
70
00:04:23 --> 00:04:26
So, this is the general
divide-and-conquer paradigm.
71
00:04:26 --> 00:04:29
And lots of algorithms fit it.
You have already seen one
72
00:04:29 --> 00:04:33
algorithm that fits this
paradigm, if you can remember.
73
00:04:33 --> 00:04:36
Merge sort, good.
Wow, you are all awake.
74
00:04:36 --> 00:04:39
I'm impressed.
So, we saw merge sort.
75
00:04:39 --> 00:04:42
And, if I am clever,
I could fit it in this space.
76
00:04:42 --> 00:04:44
Sure.
Let's be clever.
77
00:04:44 --> 00:04:48
A quick review on merge sort.
Phrased in this 1,
78
00:04:48 --> 00:04:52
2, 3 kind of method.
The first step was to divide
79
00:04:52 --> 00:04:56
your array into two halves.
This really doesn't mean
80
00:04:56 --> 00:04:59
anything because you just sort
of think, oh,
81
00:04:59 --> 00:05:05
I will pretend my array is
divided into two halves.
82
00:05:05 --> 00:05:07
There is no work here.
This is zero time.
83
00:05:07 --> 00:05:11
You just look at your array.
Here is your array.
84
00:05:11 --> 00:05:14
I guess maybe you compute n/2
and take the floor.
85
00:05:14 --> 00:05:17
That takes constant time.
And you say OK,
86
00:05:17 --> 00:05:21
I am pretending my array is now
divided into the left part and
87
00:05:21 --> 00:05:24
the right part.
And then the interesting part
88
00:05:24 --> 00:05:27
is that you recursively solve
each one.
89
00:05:27 --> 00:05:32
That's the conquer.
We recursively sort each
90
00:05:32 --> 00:05:37
subarray.
And then the third step is to
91
00:05:37 --> 00:05:44
combine those solutions.
And so here we really see what
92
00:05:44 --> 00:05:49
this means.
We now have a sorted version of
93
00:05:49 --> 00:05:55
this array by induction.
We have a sorted version of
94
00:05:55 --> 00:06:02
this array by induction.
We now want the sorted version
95
00:06:02 --> 00:06:07
of the whole array.
And we saw that was the merge
96
00:06:07 --> 00:06:10
problem, merging two sorted
arrays.
97
00:06:10 --> 00:06:15
And that we saw how to do in
linear time, order n time.
98
00:06:15 --> 00:06:20
I am not going to repeat that,
but the point is it falls into
99
00:06:20 --> 00:06:24
that framework.
I want to write the running
100
00:06:24 --> 00:06:29
time and merge sort as a
recurrence.
101
00:06:29 --> 00:06:33
You have already seen the
recurrence, you have already
102
00:06:33 --> 00:06:37
been told the solution,
but now we actually know how to
103
00:06:37 --> 00:06:39
solve it.
And, furthermore,
104
00:06:39 --> 00:06:43
every algorithm that follows
the divide-and-conquer paradigm
105
00:06:43 --> 00:06:47
will have a recurrence of pretty
much the same form,
106
00:06:47 --> 00:06:51
very much like our good friend
the master method.
107
00:06:51 --> 00:06:56
Let's do it for merge sort
where we sort of already know
108
00:06:56 --> 00:07:00
the answer and get a bit of
practice.
109
00:07:00 --> 00:07:02
This is the merge sort
recurrence.
110
00:07:02 --> 00:07:06
You should know and love this
recurrence because it comes up
111
00:07:06 --> 00:07:09
all over the place.
It comes from this general
112
00:07:09 --> 00:07:13
approach by just seeing what are
the sizes of the subproblems you
113
00:07:13 --> 00:07:17
are solving and how many there
are and how much extra work you
114
00:07:17 --> 00:07:20
are doing.
You have here the size of the
115
00:07:20 --> 00:07:22
subproblems.
It happens here that both
116
00:07:22 --> 00:07:25
subproblems have the same size
roughly.
117
00:07:25 --> 00:07:29
There is this sloppiness that
we have, which really should be
118
00:07:29 --> 00:07:34
T of floor of n over 2 plus T of
ceiling of n over 2.
119
00:07:34 --> 00:07:37
And when you go to recitation
on Friday you will see why that
120
00:07:37 --> 00:07:40
is OK, the floors and ceilings
don't matter.
121
00:07:40 --> 00:07:43
There is a theorem you can
prove that that's happy.
122
00:07:43 --> 00:07:47
You can assume that N is a
power of 2, but we are just
123
00:07:47 --> 00:07:51
going to assume that for now.
We just have two problems with
124
00:07:51 --> 00:07:53
size n over 2.
This 2 is the number of
125
00:07:53 --> 00:07:55
subproblems.
126
00:07:55 --> 00:08:00
127
00:08:00 --> 00:08:03
And this order n is all the
extra work we are doing.
128
00:08:03 --> 00:08:07
Now, what is the extra work
potentially?
129
00:08:07 --> 00:08:10
Well, the conquering is always
just recursion.
130
00:08:10 --> 00:08:14
There is sort of no work there
except this lead part.
131
00:08:14 --> 00:08:19
The dividing in this case is
trivial, but in general it might
132
00:08:19 --> 00:08:23
involve some work.
And the combining here involves
133
00:08:23 --> 00:08:25
linear work.
So, this is the
134
00:08:25 --> 00:08:29
divide-and-conquer running
times.
135
00:08:29 --> 00:08:31
So, this is the nonrecursive
work.
136
00:08:31 --> 00:08:36
And that is generally how you
convert a divide-and-conquer
137
00:08:36 --> 00:08:41
algorithm into a recurrence.
It's really easy and you
138
00:08:41 --> 00:08:44
usually get to apply the master
method.
139
00:08:44 --> 00:08:47
Here we are in Case 2.
Very good.
140
00:08:47 --> 00:08:49
This is Case 2.
And k is zero here.
141
00:08:49 --> 00:08:54
And so in the recursion tree,
all of the costs are roughly
142
00:08:54 --> 00:08:58
the same.
They are all n to the log base
143
00:08:58 --> 00:09:02
b of a.
Here n to the log base 2 of 2
144
00:09:02 --> 00:09:04
is just n.
So these are equal.
145
00:09:04 --> 00:09:07
We get an extra log factor
because of the number of levels
146
00:09:07 --> 00:09:11
in the recursion tree.
Remember the intuition behind
147
00:09:11 --> 00:09:13
the master method.
This is n log n,
148
00:09:13 --> 00:09:16
and that is good.
Merge sort is a fast sorting
149
00:09:16 --> 00:09:20
algorithm n log n.
Insertion sort was n squared.
150
00:09:20 --> 00:09:23
In some sense,
n log n is the best you can do.
151
00:09:23 --> 00:09:26
We will cover that in two
lectures from now,
152
00:09:26 --> 00:09:30
but just foreshadowing.
Today we are going to do
153
00:09:30 --> 00:09:32
different divide-and-conquer
algorithms.
154
00:09:32 --> 00:09:36
Sorting is one problem.
There are all sorts of problems
155
00:09:36 --> 00:09:39
we might want to solve.
It turns out a lot of them you
156
00:09:39 --> 00:09:42
can apply divide-and-conquer to.
Not every problem.
157
00:09:42 --> 00:09:45
Like how to wake up in the
morning, it's not so easy to
158
00:09:45 --> 00:09:48
solve a divide-and-conquer,
although maybe that's a good
159
00:09:48 --> 00:09:51
problem set problem.
160
00:09:51 --> 00:09:59
161
00:09:59 --> 00:10:03
The next divide-and-conquer
algorithm we are going to look
162
00:10:03 --> 00:10:09
at is even simpler than sorting,
even simpler than merge sort,
163
00:10:09 --> 00:10:13
but it drives home the point of
when you have only one
164
00:10:13 --> 00:10:16
subproblem.
How many people have seen
165
00:10:16 --> 00:10:19
binary search before?
Anyone hasn't?
166
00:10:19 --> 00:10:22
One, OK.
I will go very quickly then.
167
00:10:22 --> 00:10:27
You have some element X.
You want to find X in a sorted
168
00:10:27 --> 00:10:31
array.
How many people had not seen it
169
00:10:31 --> 00:10:33
before they saw it in
recitation?
170
00:10:33 --> 00:10:34
No one, OK.
Good.
171
00:10:34 --> 00:10:38
You have seen it in another
class, probably 6.001 or
172
00:10:38 --> 00:10:39
something.
Very good.
173
00:10:39 --> 00:10:42
You took the prerequisites.
OK.
174
00:10:42 --> 00:10:46
I just want to phrase it as a
divide-and-conquer because you
175
00:10:46 --> 00:10:50
don't normally see it that way.
The divide step is you compare
176
00:10:50 --> 00:10:53
X with the middle element in
your array.
177
00:10:53 --> 00:10:56
Then the conquer step.
Here is your array.
178
00:10:56 --> 00:11:02
Here is the middle element.
You compare X with this thing
179
00:11:02 --> 00:11:06
if let's say X is smaller than
the middle element in your
180
00:11:06 --> 00:11:09
array.
You know that X is in the left
181
00:11:09 --> 00:11:13
half because it is sorted,
a nice loop invariant there,
182
00:11:13 --> 00:11:17
whatever, but we are just going
to think of that as recursively
183
00:11:17 --> 00:11:22
I am going to solve the problem
of finding X in this subarray.
184
00:11:22 --> 00:11:26
We recurse in one subarray,
unlike merge sort where we had
185
00:11:26 --> 00:11:31
two recursions.
And then the combined step we
186
00:11:31 --> 00:11:35
don't do anything.
I mean if you find X in here
187
00:11:35 --> 00:11:38
then you've found X in the whole
array.
188
00:11:38 --> 00:11:42
There is nothing to bring it
back up really.
189
00:11:42 --> 00:11:46
So, this is just phrasing
binary search in the
190
00:11:46 --> 00:11:51
divide-and-conquer paradigm.
It is kind of a trivial
191
00:11:51 --> 00:11:56
example, but there are lots of
circumstances where you only
192
00:11:56 --> 00:12:02
need to recurse in one side.
And it is important to see how
193
00:12:02 --> 00:12:06
much of a difference making one
recursion versus making two
194
00:12:06 --> 00:12:09
recursions can be.
This is the recurrence for
195
00:12:09 --> 00:12:13
binary search.
We start with a problem size n.
196
00:12:13 --> 00:12:16
We reduce it to 1.
There is an implicit 1 factor
197
00:12:16 --> 00:12:19
here.
One subproblem of size n over 2
198
00:12:19 --> 00:12:21
roughly.
Again, floors and ceilings
199
00:12:21 --> 00:12:24
don't matter.
Plus a constant which is to
200
00:12:24 --> 00:12:28
compare X with the middle
element, so it is actually like
201
00:12:28 --> 00:12:32
one comparison.
This has a solution,
202
00:12:32 --> 00:12:35
log n.
And you all know the running
203
00:12:35 --> 00:12:38
time of binary search,
but here it is at solving the
204
00:12:38 --> 00:12:41
recurrence.
I mean, there are a couple of
205
00:12:41 --> 00:12:44
differences here.
We don't have the additive
206
00:12:44 --> 00:12:48
order n term.
If we did, it would be linear,
207
00:12:48 --> 00:12:51
the running the time.
Still better than n log n.
208
00:12:51 --> 00:12:55
So, we are getting rid of the
2, bringing it down to a 1,
209
00:12:55 --> 00:13:00
taking the n and bringing it
down to a 1.
210
00:13:00 --> 00:13:02
That is making the running time
a lot faster,
211
00:13:02 --> 00:13:05
the whole factor of n faster.
No big surprise there.
212
00:13:05 --> 00:13:08
Let's do some more interesting
algorithms.
213
00:13:08 --> 00:13:18
214
00:13:18 --> 00:13:24
The powering a number problem
is I give you a number X.
215
00:13:24 --> 00:13:31
I give that as like a real
number or floating point number
216
00:13:31 --> 00:13:35
or whatever.
And I give you an integer n,
217
00:13:35 --> 00:13:40
at least zero.
I want to compute X to the
218
00:13:40 --> 00:13:44
power n.
So, it's a very simple problem.
219
00:13:44 --> 00:13:49
It is, in some sense,
even easier than all of these.
220
00:13:49 --> 00:13:54
But here it is.
And divide-and-conquer is sort
221
00:13:54 --> 00:14:00
of the right thing to do.
So, the naïve algorithm is very
222
00:14:00 --> 00:14:04
simple.
How do you compute X to the
223
00:14:04 --> 00:14:07
power n?
Well, the definition of X to
224
00:14:07 --> 00:14:11
the power n is I take X and I
multiply by X n times.
225
00:14:11 --> 00:14:16
So, I take X times X times X
times X where there are n copies
226
00:14:16 --> 00:14:19
of X totally.
And that's X to the n.
227
00:14:19 --> 00:14:22
No big surprise.
That is n multiplications,
228
00:14:22 --> 00:14:27
or n minus 1 multiplications,
theta n time.
229
00:14:27 --> 00:14:34
230
00:14:34 --> 00:14:39
But that's not the best you can
do for this problem.
231
00:14:39 --> 00:14:44
Any suggestions on what we
might do using
232
00:14:44 --> 00:14:49
divide-and-conquer?
Has anyone seen this algorithm
233
00:14:49 --> 00:14:51
before?
A few, OK.
234
00:14:51 --> 00:14:56
For the rest?
Testing on the spot creativity,
235
00:14:56 --> 00:15:04
which is very difficult,
but I always like a challenge.
236
00:15:04 --> 00:15:08
I mean random ideas.
What could we possibly do to
237
00:15:08 --> 00:15:12
solve this problem in less than
linear time?
238
00:15:12 --> 00:15:16
How is this sort of a
divide-and-conquer problem?
239
00:15:16 --> 00:15:19
We have to inputs,
X and n.
240
00:15:19 --> 00:15:22
Yeah?
We could try to divide on X.
241
00:15:22 --> 00:15:25
It seems a bit hard.
It is just some number.
242
00:15:25 --> 00:15:30
Or, we could try to divide on
n.
243
00:15:30 --> 00:15:32
Any guesses?
Look at X to the n over 2,
244
00:15:32 --> 00:15:35
very good.
That is exactly the idea of the
245
00:15:35 --> 00:15:38
divide-and-conquer algorithm.
246
00:15:38 --> 00:15:48
247
00:15:48 --> 00:15:50
We would like to look at X to
the n over 2.
248
00:15:50 --> 00:15:53
This is going to be a little
bit tricky.
249
00:15:53 --> 00:15:56
Now we are going to have to pay
attention to floors and
250
00:15:56 --> 00:15:59
ceilings.
What I would like to say is
251
00:15:59 --> 00:16:04
while X to the n is X to the n
over 2 times X to the n over 2.
252
00:16:04 --> 00:16:07
And this is true if n is even.
If it is odd then I need to be
253
00:16:07 --> 00:16:11
a little bit more careful.
But let's just think about the
254
00:16:11 --> 00:16:14
intuition why this is a good
divide-and-conquer algorithm.
255
00:16:14 --> 00:16:17
We have a problem of size n,
let's say.
256
00:16:17 --> 00:16:20
We convert it into,
it looks like two subproblems
257
00:16:20 --> 00:16:23
of size n over 2,
but in fact they are the same
258
00:16:23 --> 00:16:25
subproblems.
So, I only have to solve one of
259
00:16:25 --> 00:16:28
them.
If I compute X to the n over 2.
260
00:16:28 --> 00:16:33
Yeah, I know X to the n over 2.
So, there is one conversive
261
00:16:33 --> 00:16:38
call, problem of size n over 2,
then I square that number.
262
00:16:38 --> 00:16:44
And that is one computation.
So, exactly the same recurrence
263
00:16:44 --> 00:16:48
as binary search,
log n time much better than n.
264
00:16:48 --> 00:16:51
Cool.
I also have to solve the odd
265
00:16:51 --> 00:16:53
case.
So, n is odd.
266
00:16:53 --> 2.
I will look at n minus 1 over
267
2. --> 00:16:56
268
00:16:56 --> 00:17:02
N minus 1 better be even.
And then I am missing another
269
00:17:02 --> 00:17:04
factor of X.
If n is odd,
270
00:17:04 --> 00:17:09
I am going to have to do one
recursive call and two
271
00:17:09 --> 00:17:12
multiplications.
The same recurrence.
272
00:17:12 --> 00:17:17
One recursive problem of size n
over 2, plus constant time.
273
00:17:17 --> 00:17:21
The dividing work here is
dividing by 2 and the
274
00:17:21 --> 00:17:27
combination work is doing one or
possibly two multiplications.
275
00:17:27 --> 00:17:32
And this is lg n.
And if all you are allowed to
276
00:17:32 --> 00:17:37
do is multiply numbers,
lg n is the best you can do.
277
00:17:37 --> 00:17:40
Good.
Simple but powerful.
278
00:17:40 --> 00:17:44
Whenever you want to compute a
power of a number,
279
00:17:44 --> 00:17:48
now you know what to do.
280
00:17:48 --> 00:17:55
281
00:17:55 --> 00:17:58
Does anyone not know the
definition of Fibonacci numbers
282
00:17:58 --> 00:18:01
and is willing to admit it?
OK, so this is a good old
283
00:18:01 --> 00:18:02
friend.
I will write down the
284
00:18:02 --> 00:18:05
definition as just a reminder.
And, in particular,
285
00:18:05 --> 00:18:07
the base cases.
286
00:18:07 --> 00:18:17
287
00:18:17 --> 00:18:18
Fibonacci numbers,
I will claim,
288
00:18:18 --> 00:18:21
are very important because it
appears throughout nature.
289
00:18:21 --> 00:18:24
You look at certain fruits,
you see the Fibonacci sequence.
290
00:18:24 --> 00:18:28
If you count the number of
little bumps around each ring.
291
00:18:28 --> 00:18:31
If you look at the sand on the
beach and how the waves hit it,
292
00:18:31 --> 00:18:34
it's the Fibonacci sequence I
am told.
293
00:18:34 --> 00:18:39
If you look all over the place,
Fibonacci sequence is there.
294
00:18:39 --> 00:18:43
How does nature compute the
Fibonacci sequence?
295
00:18:43 --> 00:18:46
Well, that is a different
class.
296
00:18:46 --> 00:18:52
But how are we going to compute
the Fibonacci sequence as fast
297
00:18:52 --> 00:18:56
as possible?
You have probably seen two
298
00:18:56 --> 00:18:59
algorithms.
The most naïve algorithm is the
299
00:18:59 --> 00:19:04
recursive algorithm.
Where you say OK,
300
00:19:04 --> 00:19:06
f of n.
I say well, if n is zero,
301
00:19:06 --> 00:19:09
return zero,
if n is 1, return one.
302
00:19:09 --> 00:19:13
Otherwise, recursively compute
f of n minus 1 and f of n minus
303
00:19:13 --> 00:19:17
2, add them together.
How much time does this
304
00:19:17 --> 00:19:20
algorithm take,
for those who have seen it
305
00:19:20 --> 00:19:22
before?
This is not obvious to guess.
306
00:19:22 --> 00:19:26
It doesn't have to be exact.
307
00:19:26 --> 00:19:31
308
00:19:31 --> 00:19:33
OK.
And how many people have seen
309
00:19:33 --> 00:19:36
this algorithm before and
analyzed it?
310
00:19:36 --> 00:19:39
Half, OK.
So what is the running time?
311
00:19:39 --> 00:19:41
Really, really long,
very good.
312
00:19:41 --> 00:19:44
Anymore precise answers?
What's that?
313
00:19:44 --> 00:19:45
Exponential,
yes.
314
00:19:45 --> 00:19:48
That is also correct and more
precise.
315
00:19:48 --> 00:19:52
I will be even more precise.
Maybe you haven't seen this
316
00:19:52 --> 00:19:56
analysis before.
It's phi to the n where phi is
317
00:19:56 --> 00:20:01
the Golden Ratio.
Again, Golden Ratio appears
318
00:20:01 --> 00:20:03
throughout the world in
mathematics.
319
00:20:03 --> 00:20:07
This is probably the only time
in this class,
320
00:20:07 --> 00:20:11
I'm afraid, but there we go.
It made its cameo so we are
321
00:20:11 --> 00:20:14
happy.
This is called exponential
322
00:20:14 --> 00:20:16
time.
This is bigger than one,
323
00:20:16 --> 00:20:20
that's all you need to know.
This is exponential time.
324
00:20:20 --> 00:20:25
Exponential time means
basically some constant to the
325
00:20:25 --> 00:20:28
power n.
Exponential time is a very long
326
00:20:28 --> 00:20:30
time.
It's bad.
327
00:20:30 --> 00:20:35
Polynomial time is good.
[LAUGHTER] This is what we
328
00:20:35 --> 00:20:38
want, polynomial time
algorithms.
329
00:20:38 --> 00:20:42
This class is basically
entirely about polynomial time
330
00:20:42 --> 00:20:44
algorithms.
Question?
331
00:20:44 --> 00:20:47
Oh, say what the algorithm does
again.
332
00:20:47 --> 00:20:53
Define function Fibonacci of n?
I check for the base cases.
333
00:20:53 --> 00:20:59
Otherwise, I recursively call
Fibonacci of n minus 1.
334
00:20:59 --> 00:21:02
I recursively call Fibonacci of
n minus 2 and add those two
335
00:21:02 --> 00:21:04
numbers together.
So, you get this branching
336
00:21:04 --> 00:21:06
tree.
You are solving two subproblems
337
00:21:06 --> 00:21:09
of almost the same size,
just additively smaller by one
338
00:21:09 --> 00:21:10
or two.
I mean you are almost not
339
00:21:10 --> 00:21:13
reducing the problem size at
all, so that's intuitively why
340
00:21:13 --> 00:21:16
it is exponential.
You can draw a recursion tree
341
00:21:16 --> 00:21:18
and you will see how big it gets
and how quickly.
342
00:21:18 --> 00:21:21
I mean by n over two levels,
you've only reduced on one
343
00:21:21 --> 00:21:23
branch the problem from n to n
over 2.
344
00:21:23 --> 00:21:26
The other one,
maybe you've gotten from n down
345
00:21:26 --> 00:21:29
to one, but none of the branches
have stopped after n of two
346
00:21:29 --> 00:21:32
levels.
You have at least 2 to the
347
00:21:32 --> 00:21:37
power n over 2 which is like
square root of 2 to the power n,
348
00:21:37 --> 00:21:40
which is getting close to phi
to the n.
349
00:21:40 --> 00:21:42
So, this is definitely
exponential.
350
00:21:42 --> 00:21:46
And exponential is bad.
We want polynomial.
351
00:21:46 --> 00:21:49
N squared, n cubed,
log n would be nice.
352
00:21:49 --> 00:21:53
Anything that is bounded above
by a polynomial is good.
353
00:21:53 --> 00:21:57
This is an old idea.
It goes back to one of the main
354
00:21:57 --> 00:22:01
people who said polynomial is
good, Jack Edmonds who is famous
355
00:22:01 --> 00:22:06
in the combinatorial
optimization world.
356
00:22:06 --> 00:22:12
He is my academic grandfather
on one side.
357
00:22:12 --> 00:22:18
He is a very interesting guy.
358
00:22:18 --> 00:22:32
359
00:22:32 --> 00:22:34
OK, so that's a really bad
algorithm.
360
00:22:34 --> 00:22:37
You have probably seen a
somewhat better algorithm,
361
00:22:37 --> 00:22:41
which you might think of as the
bottom-up implantation of that
362
00:22:41 --> 00:22:43
recursive algorithm.
363
00:22:43 --> 00:22:49
364
00:22:49 --> 00:22:51
Or, another way to think of it
is if you build out the
365
00:22:51 --> 00:22:54
recursion tree for Fibonacci of
n, you will see that there are
366
00:22:54 --> 00:22:58
lots of common subtrees that you
are just wasting time on.
367
00:22:58 --> 00:23:01
When you solve Fibonacci of n
minus 1, it again solves
368
00:23:01 --> 00:23:04
Fibonacci of n minus 2.
Why solve it twice?
369
00:23:04 --> 00:23:07
You only need to solve it once.
So, it is really easy to do
370
00:23:07 --> 00:23:11
that if you do it bottom-up.
But you could also do it
371
00:23:11 --> 00:23:14
recursively with some cache of
things you have already
372
00:23:14 --> 00:23:16
computed.
So, no big surprise.
373
00:23:16 --> 00:23:19
You compute the Fibonacci
numbers in order.
374
00:23:19 --> 00:23:22
And each time,
when you compute Fibonacci of
375
00:23:22 --> 00:23:24
n, let's say,
you have already computed the
376
00:23:24 --> 00:23:27
previous two,
you add them together,
377
00:23:27 --> 00:23:32
it takes constant time.
So, the running time here is
378
00:23:32 --> 00:23:36
linear, linear in n,
and as our input.
379
00:23:36 --> 00:23:39
Great.
Is that the best we can do?
380
00:23:39 --> 00:23:42
No.
Any ideas on how we could
381
00:23:42 --> 00:23:47
compute Fibonacci of n faster
than linear time?
382
00:23:47 --> 00:23:52
Now we should diverge from what
you have already seen,
383
00:23:52 --> 00:23:56
most of you.
Any ideas using techniques you
384
00:23:56 --> 00:24:00
have already seen?
Yes?
385
00:24:00 --> 00:24:02
Yes.
We can use the mathematical
386
00:24:02 --> 00:24:05
trick of phi and psi to the nth
powers.
387
00:24:05 --> 00:24:09
In fact, you can just use phi,
phi, pi, pho,
388
00:24:09 --> 00:24:13
phum, whatever you want to call
this Greek letter.
389
00:24:13 --> 00:24:16
Good.
Here is the mathematical trick.
390
00:24:16 --> 00:24:18
And, indeed,
this is cheating,
391
00:24:18 --> 00:24:21
as you have said.
This is no good,
392
00:24:21 --> 00:24:24
but so it is.
We will call it naïve recursive
393
00:24:24 --> 00:24:30
squaring and say well,
we know recursive squaring.
394
00:24:30 --> 00:24:32
Recursive squaring takes log n
time.
395
00:24:32 --> 00:24:36
Let's use recursive squaring.
And if you happen to know lots
396
00:24:36 --> 00:24:40
of properties of the Fibonacci
numbers, you don't have to,
397
00:24:40 --> 00:24:44
but here is one of them.
If you take phi to the n
398
00:24:44 --> 00:24:48
divided by root 5 and you round
it to the nearest integer that
399
00:24:48 --> 00:24:51
is the nth Fibonacci number.
400
00:24:51 --> 00:24:58
401
00:24:58 --> 00:25:01
This is pretty cool.
Fibonacci of n is basically phi
402
00:25:01 --> 00:25:03
to the n.
We could apply recursive
403
00:25:03 --> 00:25:07
squaring to compute phi to the n
in log n time,
404
00:25:07 --> 00:25:11
divide by root 5,
assume that our computer has an
405
00:25:11 --> 00:25:16
operation that rounds a number
to its nearest integer and poof,
406
00:25:16 --> 00:25:19
we are done.
That doesn't work for many
407
00:25:19 --> 00:25:22
different reasons.
On a real machine,
408
00:25:22 --> 00:25:26
probably you would represent
phi and root 5 as floating point
409
00:25:26 --> 00:25:32
numbers which have some fixed
amount of precise bits.
410
00:25:32 --> 00:25:35
And if you do this computation,
you will lose some of the
411
00:25:35 --> 00:25:37
important bits.
And when you round to the
412
00:25:37 --> 00:25:39
nearest integer you won't get
the right answer.
413
00:25:39 --> 00:25:42
So, floating point round off
will kill you on a floating
414
00:25:42 --> 00:25:45
point machine.
On a theoretical machine where
415
00:25:45 --> 00:25:48
we magically have numbers that
can do crazy things like this,
416
00:25:48 --> 00:25:51
I mean it really takes more
than constant time per
417
00:25:51 --> 00:25:53
multiplication.
So, we are sort of in a
418
00:25:53 --> 00:25:56
different model.
You cannot multiply phi times
419
00:25:56 --> 00:25:58
phi in constant time.
I mean that's sort of outside
420
00:25:58 --> 00:26:03
the boundaries of this course,
but that's the way it is.
421
00:26:03 --> 00:26:06
In fact, in a normal machine,
some problems you can only
422
00:26:06 --> 00:26:09
solve in exponential time.
In a machine where you can
423
00:26:09 --> 00:26:13
multiply real numbers and round
them to the nearest integers,
424
00:26:13 --> 00:26:15
you can solve them in
polynomial time.
425
00:26:15 --> 00:26:19
So, it really breaks the model.
You can do crazy things if you
426
00:26:19 --> 00:26:22
were allowed to do this.
This is not allowed.
427
00:26:22 --> 00:26:25
And I am foreshadowing like
three classes ahead,
428
00:26:25 --> 00:26:28
or three courses ahead,
so I shouldn't talk more about
429
00:26:28 --> 00:26:32
it.
But it turns out we can use
430
00:26:32 --> 00:26:35
recursive squaring in a
different way if we use a
431
00:26:35 --> 00:26:38
different property of Fibonacci
numbers.
432
00:26:38 --> 00:26:42
And then we will just stick
with integers and everything
433
00:26:42 --> 00:26:45
will be happy.
Don't forget to go to
434
00:26:45 --> 00:26:48
recitation and if you want to
homework lab.
435
00:26:48 --> 00:26:51
Don't come here on Monday.
That is required.
436
00:26:51 --> 00:26:57
This is sort of the right
recursive squaring algorithm.
437
00:26:57 --> 00:27:02
And this is a bit hard to guess
if you haven't already seen it,
438
00:27:02 --> 00:27:07
so I will just give it to you.
I will call this a theorem.
439
00:27:07 --> 00:27:12
It's the first time I get to
use the word theorem in this
440
00:27:12 --> 00:27:15
class.
It turns out the nth Fibonacci
441
00:27:15 --> 00:27:18
number is the nth power of this
matrix.
442
00:27:18 --> 00:27:21
Cool.
If you look at it a little bit
443
00:27:21 --> 00:27:24
you say oh, yeah,
of course.
444
00:27:24 --> 00:27:29
And we will prove this theorem
in a second.
445
00:27:29 --> 00:27:31
But once we have this theorem,
we can compute f of n by
446
00:27:31 --> 00:27:34
computing the nth power of this
matrix.
447
00:27:34 --> 00:27:36
It's a two-by-two matrix.
You multiply two two-by-two
448
00:27:36 --> 00:27:39
matrixes together,
you get a two-by-two matrix.
449
00:27:39 --> 00:27:41
So that is constant size,
four numbers.
450
00:27:41 --> 00:27:44
I can handle four numbers.
We don't have crazy precision
451
00:27:44 --> 00:27:46
problems on the floating point
side.
452
00:27:46 --> 00:27:48
There are only four numbers to
deal with.
453
00:27:48 --> 00:27:52
Matrixes aren't getting bigger.
So, the running time of this
454
00:27:52 --> 00:27:55
divide-and-conquer algorithm
will still be log n because it
455
00:27:55 --> 00:27:57
takes a constant time per
two-by-two matrix
456
00:27:57 --> 00:28:00
multiplication.
Yes?
457
00:28:00 --> 00:28:02
Oh, yes.
Thank you.
458
00:28:02 --> 00:28:06
I have a type error.
Sorry about that.
459
00:28:06 --> 00:28:12
F of n is indeed the upper left
corner, I hope.
460
00:28:12 --> 00:28:17
I better check I don't have it
off by one.
461
00:28:17 --> 00:28:21
I do.
It's F_n upper right corner,
462
00:28:21 --> 00:28:24
indeed.
That's what you said.
463
00:28:24 --> 00:28:27
F of n.
I need more space.
464
00:28:27 --> 00:28:32
Sorry.
I really ought to have a
465
00:28:32 --> 00:28:36
two-by-two matrix on the
left-hand side there.
466
00:28:36 --> 00:28:40
Thank you.
So, I compute this nth power of
467
00:28:40 --> 00:28:44
a matrix in log n time,
I take the upper right corner
468
00:28:44 --> 00:28:48
or the lower left corner,
your choice,
469
00:28:48 --> 00:28:51
that is the nth Fibonacci
number.
470
00:28:51 --> 00:28:56
This implies an order log n
time algorithm with the same
471
00:28:56 --> 00:29:01
recurrence as the last two,
binary search and really the
472
00:29:01 --> 00:29:07
recursive squaring algorithm.
It is log n plus a constant,
473
00:29:07 --> 00:29:10
so log n.
Let's prove that theorem.
474
00:29:10 --> 00:29:30
475
00:29:30 --> 00:29:34
Any suggestions on what
techniques we might use for
476
00:29:34 --> 00:29:37
proving this theorem,
or what technique,
477
00:29:37 --> 00:29:40
singular?
Induction, very good.
478
00:29:40 --> 00:29:44
I think any time I ask that
question the answer is
479
00:29:44 --> 00:29:47
induction.
Hint for the future in this
480
00:29:47 --> 00:29:49
class.
481
00:29:49 --> 00:29:54
482
00:29:54 --> 00:29:56
A friend of mine,
when he took an analysis class,
483
00:29:56 --> 00:29:59
whenever the professor asked,
and what is the answer to this
484
00:29:59 --> 00:30:01
question, the answer was always
zero.
485
00:30:01 --> 00:30:04
If you have taken analysis
class that is funny.
486
00:30:04 --> 00:30:10
[LAUGHTER] Maybe I will try to
ask some questions whose answers
487
00:30:10 --> 00:30:13
are zero just for our own
amusement.
488
00:30:13 --> 00:30:17
We are going to induct on n.
It's pretty much the obvious
489
00:30:17 --> 00:30:20
thing to do.
But we have to check some
490
00:30:20 --> 00:30:23
cases.
So, the base case is we have
491
00:30:23 --> 00:30:27
this to the first power.
And that is itself [(1,
492
00:30:27 --> 00:30:33
1), (1, 0)].
And I should have said n is at
493
00:30:33 --> 00:30:36
least 1.
And you can check.
494
00:30:36 --> 00:30:40
This is supposed to be F_2,
F_1, F_1, F_0.
495
00:30:40 --> 00:30:46
And you can check it is,
F_0 is 0, F_1 is 1 and F_2 is
496
00:30:46 --> 1.
497
1. --> 00:30:46
Good.
498
00:30:46 --> 00:30:52
Base case is correct,
step case is about as exciting,
499
00:30:52 --> 00:30:59
but you've got to prove that
your algorithm works.
500
00:30:59 --> 00:31:03
Suppose this is what we want to
compute.
501
00:31:03 --> 00:31:10
I am just going to sort of,
well, there are many ways I can
502
00:31:10 --> 00:31:14
do this.
I will just do it the fast way
503
00:31:14 --> 00:31:18
because it's really not that
exciting.
504
00:31:18 --> 00:31:23
Which direction?
Let's do this direction.
505
00:31:23 --> 00:31:29
I want to use induction on n.
If I want to use induction on
506
00:31:29 --> 00:31:33
n, presumably I should use what
I already know is true.
507
00:31:33 --> 00:31:37
If I decrease n by 1,
I have this property that this
508
00:31:37 --> 00:31:40
thing is going to be [(1,
1), (1, 0)] to the power n
509
00:31:40 --> 00:31:42
minus 1.
This I already know,
510
00:31:42 --> 00:31:45
by the induction hypothesis,
[(1, 1), (1,
511
00:31:45 --> 00:31:48
0)] to the n minus 1.
So, presumably I should use it
512
00:31:48 --> 00:31:51
in some way.
This equality is not yet true,
513
00:31:51 --> 00:31:54
you may have noticed.
So, I need to add something on.
514
00:31:54 --> 00:31:59
What could I possibly add on to
be correct?
515
00:31:59 --> 00:32:01
Well, another factor of [(1,
1), (1, 0)].
516
00:32:01 --> 00:32:05
The way I am developing this
proof is the only way it could
517
00:32:05 --> 00:32:08
possibly be, in some sense.
If you know its induction,
518
00:32:08 --> 00:32:12
this is all that you could do.
And then you check.
519
00:32:12 --> 00:32:14
Indeed, this equality holds
conveniently.
520
00:32:14 --> 00:32:17
For example,
F_(n plus 1) is the product of
521
00:32:17 --> 00:32:20
these two things.
It is this row times this
522
00:32:20 --> 00:32:22
column.
So, it is F_n times 1 plus F_(n
523
00:32:22 --> 00:32:25
minus 1) times 1,
which is indeed the definition
524
00:32:25 --> 00:32:30
of F_(n plus 1).
And you could check four of the
525
00:32:30 --> 00:32:32
entries.
This is true.
526
00:32:32 --> 00:32:36
Great.
If that is true then I would
527
00:32:36 --> 00:32:39
just put these together.
That is [(1,
528
00:32:39 --> 00:32:43
1), (1, 0)] to the n minus 1
times [(1, 1),
529
00:32:43 --> 00:32:47
(1, 0)], which is [(1,
1), (1, 0)] to the n,
530
00:32:47 --> 00:32:50
end of proof.
A very simple proof,
531
00:32:50 --> 00:32:55
but you have to do that in
order to know if this algorithm
532
00:32:55 --> 00:32:57
really works.
Good.
533
00:32:57 --> 00:33:00
Question?
Oh, yes.
534
00:33:00 --> 00:33:03
Thank you.
This, in the lower right,
535
00:33:03 --> 00:33:08
we should have F_(n minus 1).
This is why you should really
536
00:33:08 --> 00:33:12
check your proofs.
We would have discovered that
537
00:33:12 --> 00:33:16
when I checked that this was
that row times that column,
538
00:33:16 --> 00:33:20
but that is why you are here,
to fix my bugs.
539
00:33:20 --> 00:33:25
That's the great thing about
being up here instead of in a
540
00:33:25 --> 00:33:29
quiz.
But that is a minor mistake.
541
00:33:29 --> 00:33:31
You wouldn't lose much for
that.
542
00:33:31 --> 00:33:34
All right.
More divide-and-conquer
543
00:33:34 --> 00:33:37
algorithms.
Still, we have done relatively
544
00:33:37 --> 00:33:41
simple ones so far.
In fact, the fanciest has been
545
00:33:41 --> 00:33:44
merge sort, which we already
saw.
546
00:33:44 --> 00:33:48
So, that is not too exciting.
The rest have all be log n
547
00:33:48 --> 00:33:51
time.
Let's break out of the log n
548
00:33:51 --> 00:33:54
world.
Well, you all have the master
549
00:33:54 --> 00:33:58
method memorized,
right, so I can erase that.
550
00:33:58 --> 00:34:01
Good.
This will be a good test.
551
00:34:01 --> 00:34:04
Next problem is matrix
multiplication,
552
00:34:04 --> 00:34:08
following right up on this
two-by-two matrix
553
00:34:08 --> 00:34:11
multiplication.
Let's see how we can compute
554
00:34:11 --> 00:34:15
n-by-n matrix multiplications.
Just for a recap,
555
00:34:15 --> 00:34:19
you should know how to multiply
matrixes, but here is the
556
00:34:19 --> 00:34:22
definition so we can turn it
into an algorithm.
557
00:34:22 --> 00:34:26
You have two matrixes,
A and B, which are capital
558
00:34:26 --> 00:34:28
levels.
The ijth entry.
559
00:34:28 --> 00:34:34
Ith row, jth column is called
little a_ij or little b_ij.
560
00:34:34 --> 00:34:40
And your goal is to compute the
products of those matrixes.
561
00:34:40 --> 00:34:45
I should probably say that i
and j range from 1 to n.
562
00:34:45 --> 00:34:51
So, they are square matrixes.
The output is to compute C
563
00:34:51 --> 00:34:57
which has entry c_ij which is
the product of A and B.
564
00:34:57 --> 00:35:01
And, for a recap,
the ijth entry of the product
565
00:35:01 --> 00:35:08
is the inner product of the ith
row of A with the jth column of
566
00:35:08 --> 00:35:13
B.
But you can write that out as a
567
00:35:13 --> 00:35:17
sum like so.
We want to compute this thing
568
00:35:17 --> 00:35:22
for every i and j.
What is the obvious algorithm
569
00:35:22 --> 00:35:27
for doing this?
Well, for every i and j you
570
00:35:27 --> 00:35:32
compute the sum.
You compute all the products.
571
00:35:32 --> 00:35:35
You compute the sum.
So, it's like n operations here
572
00:35:35 --> 00:35:37
roughly.
I mean like 2n minus 1,
573
00:35:37 --> 00:35:40
whatever.
It is order n operations.
574
00:35:40 --> 00:35:43
There are n^2 entries of C that
I need to compute,
575
00:35:43 --> 00:35:47
so that's n^3 time.
I will write this out just for
576
00:35:47 --> 00:35:50
the programmers at heart.
Here is the pseudocode.
577
00:35:50 --> 00:35:53
It's rare that I will write
pseudocode.
578
00:35:53 --> 00:35:57
And this is a simple enough
algorithm that I can write it in
579
00:35:57 --> 00:36:03
gory detail.
But it gives you some basis for
580
00:36:03 --> 00:36:06
this analysis if you like to
program.
581
00:36:06 --> 00:36:10
It is a triply nested for loop.
582
00:36:10 --> 00:36:17
583
00:36:17 --> 00:36:23
And I made a coding error.
Hopefully you haven't written
584
00:36:23 --> 00:36:27
that far yet.
I need c_ij to be initialized
585
00:36:27 --> 00:36:31
to zero.
And then I add to c_ij the
586
00:36:31 --> 00:36:36
appropriate product,
a_ik, b_kj.
587
00:36:36 --> 00:36:39
That is the algorithm.
And the point is you have a
588
00:36:39 --> 00:36:42
nesting of n for loops from 1 to
n.
589
00:36:42 --> 00:36:46
That takes n^3 time because
this is constant and that is
590
00:36:46 --> 00:36:48
constant.
So, very simple,
591
00:36:48 --> 00:36:51
n^3.
Let's do better.
592
00:36:51 --> 00:36:57
593
00:36:57 --> 00:36:58
And, of course,
we are going to use
594
00:36:58 --> 00:36:59
divide-and-conquer.
595
00:36:59 --> 00:37:04
596
00:37:04 --> 00:37:06
Now, how are we going to divide
a matrix?
597
00:37:06 --> 00:37:10
There are a lot of numbers in a
matrix, n^2 of them in each one.
598
00:37:10 --> 00:37:12
There are all sorts of ways you
could divide.
599
00:37:12 --> 00:37:15
So far all of the
divide-and-conquers we have done
600
00:37:15 --> 00:37:19
have been problems of size n
into some number of problems of
601
00:37:19 --> 00:37:21
size n over 2.
I am going to say I start with
602
00:37:21 --> 00:37:25
some matrixes of size n-by-n.
I want to convert it down to
603
00:37:25 --> 00:37:28
something like n/2-by-n/2.
Any suggestions how I might do
604
00:37:28 --> 00:37:30
that?
Yeah?
605
00:37:30 --> 00:37:33
Block form the matrix,
indeed.
606
00:37:33 --> 00:37:38
That is the right answer.
So, this is the first
607
00:37:38 --> 00:37:43
divide-and-conquer algorithm.
This will not work,
608
00:37:43 --> 00:37:47
but it has the first idea that
we need.
609
00:37:47 --> 00:37:51
We have a n-by-n matrix.
We can view it,
610
00:37:51 --> 00:37:55
this equality is more,
you can think of it as,
611
00:37:55 --> 00:38:01
it's really the thing,
a two-by-two block matrix where
612
00:38:01 --> 00:38:06
each entry in this two-by-two
block matrix is a block of
613
00:38:06 --> 00:38:11
n/2-by-n/2 submatrixes.
614
00:38:11 --> 00:38:20
615
00:38:20 --> 00:38:27
I will think of C as being
divided into three parts,
616
00:38:27 --> 00:38:32
r, s, t and u.
Even though I write them as
617
00:38:32 --> 00:38:37
lower case letters they are
really matrixes.
618
00:38:37 --> 00:38:42
Each is n/2-by-n/2.
And A, I will split into a,
619
00:38:42 --> 00:38:46
b, c, d, times B,
I will split into e,
620
00:38:46 --> 00:38:48
f, g, h.
Why not?
621
00:38:48 --> 00:38:53
This is certainly true.
And if you've seen some linear
622
00:38:53 --> 00:39:01
algebra, this is basically what
you can do with matrixes.
623
00:39:01 --> 00:39:05
Now I can pretend these are
two-by-two and sort of forget
624
00:39:05 --> 00:39:09
the fact that these little
letters are matrixes and say
625
00:39:09 --> 00:39:14
well, r is the inner product of
this row with this column.
626
00:39:14 --> 00:39:17
It is ae times bg.
Let me not cheat or else it
627
00:39:17 --> 00:39:23
will be too easy.
r=ae+bg, s=af+bh,
628
00:39:23 --> 00:39:34
t=ce+dh and u=cf+dg.
It's nothing like making it too
629
00:39:34 --> 00:39:41
hard on yourself.
OK, got them right.
630
00:39:41 --> 00:39:48
Good.
I mean this is just a fact
631
00:39:48 --> 00:39:59
about how you would expand out
this product.
632
00:39:59 --> 00:40:01
And so now we have a recursive
algorithm.
633
00:40:01 --> 00:40:04
In fact, we have a
divide-and-conquer algorithm.
634
00:40:04 --> 00:40:07
We start with our n-by-n
matrix.
635
00:40:07 --> 00:40:09
Well, we have two of them
actually.
636
00:40:09 --> 00:40:12
We divide it into eight little
pieces, a, b,
637
00:40:12 --> 00:40:13
c, d, e, f, g,
h.
638
00:40:13 --> 00:40:16
Then we compute these things
and that gives us C,
639
00:40:16 --> 00:40:20
just by sticking them together.
Now, how do we compute these
640
00:40:20 --> 00:40:23
things?
Well, these innocent-looking
641
00:40:23 --> 00:40:27
little products between these
two little numbers are actually
642
00:40:27 --> 00:40:31
recursive matrix
multiplications.
643
00:40:31 --> 00:40:37
Because each of these little
letters is an n/2-by-n/2 matrix
644
00:40:37 --> 00:40:42
so I have to recursively compute
the product.
645
00:40:42 --> 00:40:49
There are like eight recursive
multiplications of n/2-by-n/2
646
00:40:49 --> 00:40:52
matrixes.
That is what bites us.
647
00:40:52 --> 00:40:59
And then there are like four
additions, plus minor work of
648
00:40:59 --> 00:41:05
gluing things together.
How long does it take to add
649
00:41:05 --> 00:41:06
two matrixes together?
n^2.
650
00:41:06 --> 00:41:09
This is cheap.
It just takes n^2.
651
00:41:09 --> 00:41:12
Remember, we are trying to beat
n^3 for our matrix
652
00:41:12 --> 00:41:15
multiplication.
Addition is a really easy
653
00:41:15 --> 00:41:17
problem.
You just have to add every
654
00:41:17 --> 00:41:20
number.
There is no way you can do
655
00:41:20 --> 00:41:23
better than n^2.
So, that is not recursive.
656
00:41:23 --> 00:41:26
That is the nice thing.
But the bad thing is we have
657
00:41:26 --> 00:41:31
eight of these recursions.
We have
658
00:41:31 --> 00:41:39
T(n)=8T(n/2)+Theta(n^2).
And I have erased the master
659
00:41:39 --> 00:41:46
method, but you should all have
it memorized.
660
00:41:46 --> 00:41:52
What is the solution to this
recurrence?
661
00:41:52 --> 00:41:56
Theta(n^3).
That is annoying.
662
00:41:56 --> 00:42:01
All right.
A is 8, b is 2,
663
00:42:01 --> 00:42:07
log base 2 of 8 is 3.
Every computer scientist should
664
00:42:07 --> 00:42:09
know that.
n^log_b(a)=n^3.
665
00:42:09 --> 00:42:15
That is polynomially larger
than n^2, so we are in Case 1.
666
00:42:15 --> 00:42:19
Thank you.
Let's get them upside down.
667
00:42:19 --> 00:42:23
This is n^3,
no better than our previous
668
00:42:23 --> 00:42:26
algorithm.
That kind of sucks.
669
00:42:26 --> 00:42:30
And now comes the divine
inspiration.
670
00:42:30 --> 00:42:34
Let's go over here.
671
00:42:34 --> 00:42:41
672
00:42:41 --> 00:42:44
There are some algorithms like
this Fibonacci algorithm where
673
00:42:44 --> 00:42:47
if you sat down for a little
while, it's no big deal,
674
00:42:47 --> 00:42:51
you would figure it out.
I mean it is kind of clever to
675
00:42:51 --> 00:42:54
look at that matrix and then
everything works happily.
676
00:42:54 --> 00:42:57
It is not obvious but it is not
that amazingly clever.
677
00:42:57 --> 00:43:01
This is an algorithm that is
amazingly clever.
678
00:43:01 --> 00:43:05
You may have seen it before
which steals the thunder a
679
00:43:05 --> 00:43:10
little bit, but it is still
really, really cool so you
680
00:43:10 --> 00:43:15
should be happy to see it again.
And how Strassen came up with
681
00:43:15 --> 00:43:19
this algorithm,
he must have been very clever.
682
00:43:19 --> 00:43:24
The idea is we've got to get
rid of these multiplications.
683
00:43:24 --> 00:43:30
I could do a hundred additions.
That only costs Theta(n^2).
684
00:43:30 --> 00:43:33
I have to reduce this 8 to
something smaller.
685
00:43:33 --> 00:43:35
It turns out,
if you try to split the
686
00:43:35 --> 00:43:39
matrices into three-by-three or
something, that doesn't help
687
00:43:39 --> 00:43:41
you.
You get the same problem
688
00:43:41 --> 00:43:44
because we're using
fundamentally the same
689
00:43:44 --> 00:43:47
algorithm, just in a different
order.
690
00:43:47 --> 00:43:51
We have got to somehow reduce
the number of multiplications.
691
00:43:51 --> 00:43:55
We are going to reduce it to 7.
The claim is that if we have
692
00:43:55 --> 00:43:59
two two-by-two matrices we can
take their product using seven
693
00:43:59 --> 00:44:03
multiplications.
If that were true,
694
00:44:03 --> 00:44:08
we would reduce the 8 to a 7
and presumably make things run
695
00:44:08 --> 00:44:11
faster.
We will see how fast in a
696
00:44:11 --> 00:44:14
moment.
You can compute it in your
697
00:44:14 --> 00:44:16
head.
If you are bored and like
698
00:44:16 --> 00:44:21
computing logs that are
non-integral logs then go ahead.
699
00:44:21 --> 00:44:23
All right.
Here we are.
700
00:44:23 --> 00:44:28
This algorithm is unfortunately
rather long, but it is only
701
00:44:28 --> 00:44:32
seven multiplications.
702
00:44:32 --> 00:44:45
703
00:44:45 --> 00:44:49
Each of these P's is a product
of two things which only
704
00:44:49 --> 00:44:54
involves addition or
subtraction, the same thing.
705
00:44:54 --> 00:45:38
706
00:45:38 --> 00:45:41
Those are seven
multiplications.
707
00:45:41 --> 00:45:44
And I can compute those in
7T(n/2).
708
00:45:44 --> 00:45:47
Oh, indeed it is.
Six was wrong.
709
00:45:47 --> 00:45:51
Six and seven are the same,
very good.
710
00:45:51 --> 00:45:57
You know, you think that
copying something would not be
711
00:45:57 --> 00:46:02
such a challenging task.
But when you become an
712
00:46:02 --> 00:46:07
absent-minded professor like me
then you will know how easy it
713
00:46:07 --> 00:46:07
is.
OK.
714
00:46:07 --> 00:46:10
We have them all correct,
hopefully.
715
00:46:10 --> 00:46:12
We continue.
That wasn't enough.
716
00:46:12 --> 00:46:16
Of course we had seven things.
Clearly we have to reduce this
717
00:46:16 --> 00:46:19
down to four things,
the elements of C.
718
00:46:19 --> 00:46:22
Here they are,
the elements of C,
719
00:46:22 --> 00:46:25
r, s, t, u.
It turns out r=P_5+P_4-P_2+P_6.
720
00:46:25 --> 00:46:27
Of course.
Didn't you all see that?
721
00:46:27 --> 00:46:33
[LAUGHTER]
I mean this one is really easy,
722
00:46:33 --> 00:46:36
s=P_1+P2.
t=P_3+P_4.
723
00:46:36 --> 00:46:41
I mean that is clearly how they
were chosen.
724
00:46:41 --> 00:46:48
And then u is another tricky
one, u=P_5+P_1-P_3-P_7.
725
00:46:48 --> 00:46:52
OK.
Now, which one of these would
726
00:46:52 --> 00:46:57
you like me to check?
Don't be so nice.
727
00:46:57 --> 00:47:03
How about s?
I can show you s is right.
728
00:47:03 --> 00:47:06
Any preferences?
u.
729
00:47:06 --> 00:47:09
Oh, no, sign errors.
OK.
730
00:47:09 --> 00:47:14
Here we go.
The claim that this really
731
00:47:14 --> 00:47:21
works is you have to check all
four of them.
732
00:47:21 --> 00:47:25
And I did in my notes.
u=P_5.
733
00:47:25 --> 00:47:31
P_5=(ae + ah + de + dh).
That is P_5.
734
00:47:31 --> 00:47:35
Check me.
If I screw up,
735
00:47:35 --> 00:47:40
I am really hosed.
(af - ah) = P_1.
736
00:47:40 --> 00:47:48
P_3 has a minus sign in front,
so that is (ce + de).
737
00:47:48 --> 00:47:55
And then we have minus P_7,
which is a big one,
738
00:47:55 --> 00:48:00
(ae + af - ce - cf).
OK.
739
00:48:00 --> 00:48:12
Now I need like the assistant
that crosses off things in
740
00:48:12 --> 00:48:19
parallel like the movie,
right?
741
00:48:19 --> 00:48:25
ah, de, af, ce,
ae, thank you,
742
00:48:25 --> 00:48:37
and hopefully these survive,
dh minus minus cf.
743
00:48:37 --> 00:48:40
And, if we are lucky,
that is exactly what is written
744
00:48:40 --> 00:48:42
here, except in the opposite
order.
745
00:48:42 --> 00:48:45
Magic, right?
Where the hell did Strassen get
746
00:48:45 --> 00:48:47
this?
You have to be careful.
747
00:48:47 --> 00:48:51
It is OK that the plus is in
the wrong order because plus is
748
00:48:51 --> 00:48:54
commutative, but the
multiplications better not be in
749
00:48:54 --> 00:48:58
the wrong order because
multiplication over matrixes is
750
00:48:58 --> 00:49:01
not commutative.
I check cf, OK,
751
00:49:01 --> 00:49:05
dh, they are in the right
order.
752
00:49:05 --> 00:49:11
I won't check the other three.
That is matrix multiplication
753
00:49:11 --> 00:49:17
in hopefully subcubic time.
Let's write down the
754
00:49:17 --> 00:49:19
recurrence.
T(n) is now 7.
755
00:49:19 --> 00:49:25
Maybe I should write down the
algorithm for kicks.
756
00:49:25 --> 00:49:30
Why not?
Assuming I have time.
757
00:49:30 --> 00:49:33
Lots of time.
Last lecture I was ten minutes
758
00:49:33 --> 00:49:36
early.
I ended ten minutes early.
759
00:49:36 --> 00:49:40
I apologize for that.
I know it really upsets you.
760
00:49:40 --> 00:49:46
And I didn't realize exactly
when the class was supposed to
761
00:49:46 --> 00:49:48
end.
So, today, I get to go ten
762
00:49:48 --> 00:49:50
minutes late.
OK.
763
00:49:50 --> 00:49:52
Good.
I'm glad you all agree.
764
00:49:52 --> 00:49:55
[LAUGHTER] I am kidding.
Don't worry.
765
00:49:55 --> 00:49:56
OK.
Algorithm.
766
00:49:56 --> 00:50:02
This is Strassen.
First we divide,
767
00:50:02 --> 00:50:07
then we conquer and then we
combine.
768
00:50:07 --> 00:50:14
As usual, I don't have it
written anywhere here.
769
00:50:14 --> 00:50:17
Fine.
Divide A and B.
770
00:50:17 --> 00:50:27
This is sort of trivial.
Then we compute the terms --
771
00:50:27 --> 00:50:36
772
00:50:36 --> 00:50:39
-- for the products.
This means we get ready to
773
00:50:39 --> 00:50:42
compute all the P's.
We compute a+b,
774
00:50:42 --> 00:50:45
c+d, g-e, a+d,
e+h and so on.
775
00:50:45 --> 00:50:49
All of the terms that appear in
here, we compute those.
776
00:50:49 --> 00:50:54
That takes n2 time because it
is just a bunch of additions and
777
00:50:54 --> 00:50:56
subtractions.
No big deal.
778
00:50:56 --> 00:51:04
A constant number of them.
Then we conquer by recursively
779
00:51:04 --> 00:51:12
computing all the P_i's.
That is each our product of
780
00:51:12 --> 00:51:17
seven of them.
We have P_1,
781
00:51:17 --> 00:51:21
P_2 up to P_7.
And, finally,
782
00:51:21 --> 00:51:30
we combine, which is to compute
r, s, t and u.
783
00:51:30 --> 00:51:36
784
00:51:36 --> 00:51:40
And those are just additions
and subtractions again,
785
00:51:40 --> 00:51:43
so they take n^2 times.
So, here we finally get an
786
00:51:43 --> 00:51:47
algorithm that is nontrivial
both in dividing and in
787
00:51:47 --> 00:51:51
combining.
Recursion is always recursion,
788
00:51:51 --> 00:51:54
but now we have interesting
steps one and three.
789
00:51:54 --> 00:51:59
The recurrence is T(n) is seven
recursive subproblems,
790
00:51:59 --> 00:52:03
each are size n/2 plus order
n^2, to do all this addition
791
00:52:03 --> 00:52:07
work.
Now we need to solve this
792
00:52:07 --> 00:52:11
recurrence.
We compute n^log_b(a),
793
00:52:11 --> 00:52:16
which here is nlog_2(7).
And we know log base 2 of 8 is
794
00:52:16 --> 3.
795
3. --> 00:52:20
Log base 2 of 7 is going be a
796
00:52:20 --> 00:52:26
little bit less than 3 but still
bigger than 2 because log base 2
797
00:52:26 --> 00:52:30
of 4 is 2.
So, it is going to be
798
00:52:30 --> 00:52:36
polynomially larger than n^2 but
polynomially smaller than n^3.
799
00:52:36 --> 00:52:41
We are again in Case 1.
And this is the cheating way to
800
00:52:41 --> 00:52:44
write n log base 2 of 7,
nlg7.
801
00:52:44 --> 00:52:48
lg means log base 2.
You should know that.
802
00:52:48 --> 00:52:53
It is all over the textbook and
in our problem sets and what
803
00:52:53 --> 00:52:56
not, nlg7.
And, in particular,
804
00:52:56 --> 00:53:02
if I have my calculator here.
This is a good old-fashion
805
00:53:02 --> 00:53:04
calculator.
No, that is wrong.
806
00:53:04 --> 00:53:07
Sorry.
It is strictly less than 2.81.
807
00:53:07 --> 00:53:11
It is strictly less than 2.81.
That is cool.
808
00:53:11 --> 00:53:14
I mean it is polynomially
better than n^3.
809
00:53:14 --> 00:53:18
Still not as good as addition,
which is n^2.
810
00:53:18 --> 00:53:23
It is generally believed,
although we don't know whether
811
00:53:23 --> 00:53:29
you can multiply as fast as you
can divide for matrices.
812
00:53:29 --> 00:53:32
We think you cannot get n^2,
but who knows?
813
00:53:32 --> 00:53:36
It could still happen.
There are no lower bounds.
814
00:53:36 --> 00:53:40
This is not the best algorithm
for matrix multiplication.
815
00:53:40 --> 00:53:44
It is sort of the simplest that
beats n^3.
816
00:53:44 --> 00:53:46
The best so far is like
n^2.376.
817
00:53:46 --> 00:53:50
Getting closer to 2.
You might think these numbers
818
00:53:50 --> 00:53:54
are a bit weird.
Maybe the constants out here
819
00:53:54 --> 00:54:00
dominate the improvement you are
getting in the exponent.
820
00:54:00 --> 00:54:03
It turns out improving the
exponent is a big deal.
821
00:54:03 --> 00:54:06
I mean, as n gets larger
exponents really come out to
822
00:54:06 --> 00:54:09
bite you.
So, n^3 is pretty impractical
823
00:54:09 --> 00:54:13
for any very large values of n.
And we known that Strassen will
824
00:54:13 --> 00:54:16
beat normal matrix
multiplication if n is
825
00:54:16 --> 00:54:19
sufficiently large.
The claim is that roughly at
826
00:54:19 --> 00:54:22
about 32 or so already you get
an improvement,
827
00:54:22 --> 00:54:25
for other reasons,
not just because the exponent
828
00:54:25 --> 00:54:30
gets better, but there you go.
So, this is pretty good.
829
00:54:30 --> 00:54:33
This is completely impractical,
so don't use whatever this
830
00:54:33 --> 00:54:36
algorithm is.
I don't have the reference
831
00:54:36 --> 00:54:40
handy, but it is just trying to
get a theoretical improvement.
832
00:54:40 --> 00:54:44
There may be others that are in
between and more reasonable but
833
00:54:44 --> 00:54:46
that is not it.
Wow, lots of time.
834
00:54:46 --> 00:54:49
Any questions?
We're not done yet,
835
00:54:49 --> 00:54:52
but any questions before we
move on for matrix
836
00:54:52 --> 00:54:53
multiplication?
OK.
837
00:54:53 --> 00:54:56
I have one more problem.
838
00:54:56 --> 00:55:15
839
00:55:15 --> 00:55:17
Divide-and-conquer is a pretty
general idea.
840
00:55:17 --> 00:55:20
I mean, you can use it to
dominate countries.
841
00:55:20 --> 00:55:22
You can use it to multiply
matrices.
842
00:55:22 --> 00:55:26
I mean, who would have thought?
Here is a very different kind
843
00:55:26 --> 00:55:30
of problem you can solve with
divide-and-conquer.
844
00:55:30 --> 00:55:32
It is not exactly an
algorithmic problem,
845
00:55:32 --> 00:55:35
although it is computer
science.
846
00:55:35 --> 00:55:37
That is clear.
This is very large-scale
847
00:55:37 --> 00:55:40
integration.
The chips, they are very large
848
00:55:40 --> 00:55:44
scale integrated.
Probably even more these days,
849
00:55:44 --> 00:55:47
but that is the catch phrase.
Here is a problem,
850
00:55:47 --> 00:55:51
and it arises in VLSI layout.
We won't get into too many
851
00:55:51 --> 00:55:54
details why, but you have some
circuit.
852
00:55:54 --> 00:55:58
And here I am going to assume
that the circuit is a binary
853
00:55:58 --> 00:56:02
tree.
This is just part of a circuit.
854
00:56:02 --> 00:56:07
Assume for now here that it is
just a complete binary tree.
855
00:56:07 --> 00:56:11
A complete binary tree looks
like this.
856
00:56:11 --> 00:56:15
In all of my teachings,
I have drawn this figure for
857
00:56:15 --> 00:56:18
sure the most.
It is my favorite figure,
858
00:56:18 --> 00:56:21
the height four complete binary
tree.
859
00:56:21 --> 00:56:25
OK, there it is.
I have some tree like that as
860
00:56:25 --> 00:56:30
some height.
I want to imbed it into some
861
00:56:30 --> 00:56:36
chip layout on a grid.
Let's say it has n leaves.
862
00:56:36 --> 00:56:41
I want to imbed it into a grid
with minimum area.
863
00:56:41 --> 00:56:47
This is a very cute problem and
it really shows you another way
864
00:56:47 --> 00:56:53
in which divide-and-conquer is a
useful and powerful tool.
865
00:56:53 --> 00:57:00
So, I have this tree.
I like to draw it in this way.
866
00:57:00 --> 00:57:02
I want to somehow draw it on
the grid.
867
00:57:02 --> 00:57:06
What that means is the vertices
have to be imbedded onto dots on
868
00:57:06 --> 00:57:09
the grid, and I am talking about
the square grid.
869
00:57:09 --> 00:57:12
It has to go to vertices of the
grid.
870
00:57:12 --> 00:57:15
And these edges have to be
routed as sort of orthogonal
871
00:57:15 --> 00:57:19
paths between one dot and
another, so that should be an
872
00:57:19 --> 00:57:22
edge and they shouldn't cross
and all these good things
873
00:57:22 --> 00:57:25
because wires do not like to
cross.
874
00:57:25 --> 00:57:28
There is the obvious way to
solve this problem and there is
875
00:57:28 --> 00:57:31
the right way.
876
00:57:31 --> 00:57:36
877
00:57:36 --> 00:57:37
And let's talk about both of
them.
878
00:57:37 --> 00:57:41
Neither of them is particularly
obvious, but divide-and-conquer
879
00:57:41 --> 00:57:44
sort of gives you a hint in the
right direction.
880
00:57:44 --> 00:57:47
So, the naïve imbedding.
I seem to like the word naïve
881
00:57:47 --> 00:57:48
here.
882
00:57:48 --> 00:57:54
883
00:57:54 --> 00:57:58
I am going to draw this bottom
up because it is easier,
884
00:57:58 --> 00:58:02
so leave three grid lines and
then start drawing.
885
00:58:02 --> 00:58:05
I don't know how big that is
going to be.
886
00:58:05 --> 00:58:11
Here is the bottom of our tree.
This is like the little three
887
00:58:11 --> 00:58:15
nodes there.
And then I leave a blank column
888
00:58:15 --> 00:58:21
and then a blank column.
I don't actually need to leave
889
00:58:21 --> 00:58:25
those blank columns,
but it makes a prettier
890
00:58:25 --> 00:58:30
drawing.
And then we work our way up.
891
00:58:30 --> 00:58:37
892
00:58:37 --> 00:58:40
There is the tree,
which should be aligned,
893
00:58:40 --> 00:58:42
on a grid.
No crossings.
894
00:58:42 --> 00:58:45
Everything is happy.
How much area does it take?
895
00:58:45 --> 00:58:49
By area, I mean sort of the
area of the bounding box.
896
00:58:49 --> 00:58:54
So, I count this blank space
even though I am not using it
897
00:58:54 --> 00:59:00
and I count all this blank space
even though I am not using it.
898
00:59:00 --> 00:59:04
I want to look at the height.
Let's call this H(n).
899
00:59:04 --> 00:59:08
And to look at the width,
which I will call W(n).
900
00:59:08 --> 00:59:13
Now, it is probably pretty
obvious that H(n) is like log n,
901
00:59:13 --> 00:59:18
W(n) is like n or whatever.
But I want to write it as a
902
00:59:18 --> 00:59:22
recurrence because that will
inspire us to do the right
903
00:59:22 --> 00:59:23
thing.
H(n).
904
00:59:23 --> 00:59:27
Well, if you think of this as a
recursion-tree,
905
00:59:27 --> 00:59:31
in some sense.
We start with the big tree.
906
00:59:31 --> 00:59:35
We split it into two halves,
two subtrees of size n/2 indeed
907
00:59:35 --> 00:59:40
because we are counting leaves.
It is exactly n/2 on each side.
908
00:59:40 --> 00:59:43
Then for height they are in
parallel so it is no big deal.
909
00:59:43 --> 00:59:46
The height is just the height
of this thing,
910
00:59:46 --> 00:59:48
one of these subproblems plus
one.
911
00:59:48 --> 00:59:52
The width, you have to add
together the two widths and also
912
00:59:52 --> 00:59:55
add on 1.
You don't have to add on 1
913
00:59:55 --> 1:00:00
here, but it doesn't matter.
It is certainly at most 1.
914
1:00:00 --> 1:00:10.598
H(n) = H(n/2) + Theta(1),
there you do have to add 1,
915
1:00:10.598 --> 1:00:20.178
and W(n) = 2W(n/2) + O(1).
The usual base cases.
916
1:00:20.178 --> 1:00:32
I mean, these are recurrences
we should know and love.
917
1:00:32 --> 1:00:36.155
This is log n,
I sort of have already given
918
1:00:36.155 --> 1:00:40.409
away the answers,
and this better be linear.
919
1:00:40.409 --> 1:00:45.356
This is again Case 1.
And to the log base 2 of 2 is
920
1:00:45.356 --> 1:00:49.512
n, which is the answer,
much bigger than 1.
921
1:00:49.512 --> 1:00:54.459
And here n to the log base 2 of
1 is n to the zero,
922
1:00:54.459 --> 1:01:00
which is 1, which is the same
so we get log n.
923
1:01:00 --> 1:01:04.085
The area is n log n,
but if you are making chips you
924
1:01:04.085 --> 1:01:08.331
want the area as small as
possible so you can fit more
925
1:01:08.331 --> 1:01:12.256
good stuff in there.
So, we would like to aim for,
926
1:01:12.256 --> 1:01:16.182
well, we certainly cannot do a
better area than n.
927
1:01:16.182 --> 1:01:19.627
You've got to put the leaves
down somewhere,
928
1:01:19.627 --> 1:01:22.11
but this is already pretty
good.
929
1:01:22.11 --> 1:01:26.356
It is only a log factor off,
but we want to aim for n.
930
1:01:26.356 --> 1:01:31.053
How could we get n?
Any guesses on what needs to
931
1:01:31.053 --> 1:01:34.794
change in this layout?
Not how to do it because that
932
1:01:34.794 --> 1:01:37.801
is not obvious,
but in terms of height and
933
1:01:37.801 --> 1:01:41.616
width what should we do?
It is pretty hard to get the
934
1:01:41.616 --> 1:01:44.696
height smaller than log n,
I will tell you,
935
1:01:44.696 --> 1:01:48.584
because this is a tree.
It cannot really get its width
936
1:01:48.584 --> 1:01:52.398
down to less than log n.
What could we do to make the
937
1:01:52.398 --> 1:01:54.819
product linear?
Just random ideas.
938
1:01:54.819 --> 1:01:59
What are two functions whose
product is n?
939
1:01:59 --> 1:02:07
940
1:02:07 --> 1:02:09.227
Square root of n and square
root of n.
941
1:02:09.227 --> 1:02:12.298
That is a good choice.
Were there other suggestions?
942
1:02:12.298 --> 1:02:15.188
n times constant.
Yeah, n times constant would be
943
1:02:15.188 --> 1:02:17.054
nice.
But I claim you cannot get
944
1:02:17.054 --> 1:02:19.704
either of these down to less
than a constant.
945
1:02:19.704 --> 1:02:23.316
You could aim for n over log n
by log n, that is more likely,
946
1:02:23.316 --> 1:02:25.544
but I think that is almost
impossible.
947
1:02:25.544 --> 1:02:30
Root n by root n is the right
answer, so let's go with that.
948
1:02:30 --> 1:02:36.05
So, root n by root n.
We haven't seen any recurrences
949
1:02:36.05 --> 1:02:42.45
whose solution is root n,
but surely they are out there.
950
1:02:42.45 --> 1:02:49.316
Let's say the goal is to get
W(n) = Theta(root n) and to get
951
1:02:49.316 --> 1:02:54.785
H(n) = Theta(root n).
If we did that we would be
952
1:02:54.785 --> 1:03:02
happy, because then the area is
the product is linear.
953
1:03:02 --> 1:03:05.677
How?
What is a recurrence that is in
954
1:03:05.677 --> 1:03:11.245
the usual master method form
whose solution is root n?
955
1:03:11.245 --> 1:03:15.237
I mean, you could think of it
that way.
956
1:03:15.237 --> 1:03:20.49
Recurrence is a bit tricky,
but let's just think of
957
1:03:20.49 --> 1:03:24.377
n^log_b(a).
When is log base b of a ½?
958
1:03:24.377 --> 1:03:29
Because then n^log_b(a) is root
n.
959
1:03:29 --> 1:03:33.726
And there is some hope that I
could get a root n solution to
960
1:03:33.726 --> 1:03:36.77
recurrence.
This is designed by knowing
961
1:03:36.77 --> 1:03:41.095
that it is divide-and-conquer,
and therefore it must be
962
1:03:41.095 --> 1:03:44.94
something like this.
It is easy once you know the
963
1:03:44.94 --> 1:03:49.186
approach you are supposed to
take and you can try this
964
1:03:49.186 --> 1:03:51.989
approach.
When is log base b of a ½?
965
1:03:51.989 --> 1:03:54.633
Lots of solutions,
shout them out.
966
1:03:54.633 --> 1:04:00
4 and 2, that is a good one.
I better get this right.
967
1:04:00 --> 1:04:06.41
Log base 4 of 2 is ½ because
the square root of 4 is 2.
968
1:04:06.41 --> 1:04:10.089
So, let's aim for this.
Why not?
969
1:04:10.089 --> 1:04:14.007
When would we get log base 4 of
2?
970
1:04:14.007 --> 1:04:19.823
This is b, this is a,
so it should be 2T(n/4) plus
971
1:04:19.823 --> 1:04:24.69
something.
And if I want the n^log_b(a) to
972
1:04:24.69 --> 1:04:31.338
dominate, it has got to be
polynomially smaller than root
973
1:04:31.338 --> 1:04:34.388
n.
So, this should be
974
1:04:34.388 --> 1:04:37.32
n^1/2-epsilon.
But it could be smaller.
975
1:04:37.32 --> 1:04:39.867
It could be 1.
Zero would be nice,
976
1:04:39.867 --> 1:04:43.03
but that is probably too much
to hope for.
977
1:04:43.03 --> 1:04:46.966
So, something smaller,
strictly polynomially smaller
978
1:04:46.966 --> 1:04:49.203
than root n.
That is our goal.
979
1:04:49.203 --> 1:04:53.293
And now comes the magic.
If you played with this for a
980
1:04:53.293 --> 1:04:58
while you would find it,
I think, at this point.
981
1:04:58 --> 1:05:02.56
When you know that you are
somehow solve this problem of
982
1:05:02.56 --> 1:05:07.287
size n with two subproblems of
size n/4 what could you do?
983
1:05:07.287 --> 1:05:11.268
Well, if you start thinking of
things as squares,
984
1:05:11.268 --> 1:05:15
this is the natural thing that
happens.
985
1:05:15 --> 1:05:20
986
1:05:20 --> 1:05:30
This is called the H layout.
You can imagine why.
987
1:05:30 --> 1:05:35.578
It would be much easier to draw
if I had a grid board,
988
1:05:35.578 --> 1:05:40
a graph board,
whatever, if that exists.
989
1:05:40 --> 1:05:46
990
1:05:46 --> 1:05:48.983
This is a recursive layout.
I am only going to draw a
991
1:05:48.983 --> 1:05:51.68
couple iterations,
but hopefully you can imagine
992
1:05:51.68 --> 1:05:53
the generalization.
993
1:05:53 --> 1:06:05
994
1:06:05 --> 1:06:08.578
I take four Hs,
a good plan because I want
995
1:06:08.578 --> 1:06:12.156
problems of size n/4.
This has n/4 leaves.
996
1:06:12.156 --> 1:06:15.385
This has n/4 leaves.
This is the root,
997
1:06:15.385 --> 1:06:19.4
by the way, in the middle.
This has n/4 leaves.
998
1:06:19.4 --> 1:06:23.501
This has n/4 leaves.
So, I have four problems of
999
1:06:23.501 --> 1:06:26.905
size n/4.
Somehow I have got to get that
1000
1:06:26.905 --> 1:06:31.471
down to two.
Thankfully, if I look at width
1001
1:06:31.471 --> 1:06:35.92
or if I look at height,
there are only two that matter.
1002
1:06:35.92 --> 1:06:40.286
And these two matter and these
two get along for free.
1003
1:06:40.286 --> 1:06:44.818
They are going in parallel,
just like we had with height
1004
1:06:44.818 --> 1:06:48.031
over here.
But I get that both in height
1005
1:06:48.031 --> 1:06:50.173
and in width.
If I measure,
1006
1:06:50.173 --> 1:06:54.539
well, now they are equal,
so I will just call them the
1007
1:06:54.539 --> 1:06:58
length.
We have L(n)-by-L(n).
1008
1:06:58 --> 1:07:01.117
And if I compute well,
what is L(n)?
1009
1:07:01.117 --> 1:07:06.194
I have here L(n/4) because
there are only a quarter of the
1010
1:07:06.194 --> 1:07:11.271
leaves in this one or in that
one, then I have a constant,
1011
1:07:11.271 --> 1:07:15.813
Theta(1), no big deal,
and then I have L(n/4) again.
1012
1:07:15.813 --> 1:07:20
So, I get the recurrence that I
wanted.
1013
1:07:20 --> 1:07:26
1014
1:07:26 --> 1:07:35.17
L(n) = 2L(n/4) + Theta(1).
And that has solution square
1015
1:07:35.17 --> 1:07:39.976
root of n, as we claimed before.
Again, we are in Case 1 of the
1016
1:07:39.976 --> 1:07:41.759
master method.
Cool, ha?
1017
1:07:41.759 --> 1:07:44.395
This is a much more compact
layout.
1018
1:07:44.395 --> 1:07:47.108
Charles, did you invent this
layout?
1019
1:07:47.108 --> 1:07:49.589
No.
But I know it appears on your
1020
1:07:49.589 --> 1:07:53.62
PhD thesis and you extended it
in various directions.
1021
1:07:53.62 --> 1:07:58.271
So, this is sort of a classic
cool layout of trees into grids
1022
1:07:58.271 --> 1:08:03
and another application of
divide-and-conquer.
1023
1:08:03 --> 1:08:07.789
I mean this is not particularly
useful for algorithms directly.
1024
1:08:07.789 --> 1:08:10.647
It is useful for VLSI layout
directly.
1025
1:08:10.647 --> 1:08:14.663
But it gives you more flavor of
how you should think.
1026
1:08:14.663 --> 1:08:18.371
If you know what running time
you are aiming for,
1027
1:08:18.371 --> 1:08:23.237
like in problem sets in quizzes
often we say here is the running
1028
1:08:23.237 --> 1:08:27.563
time you have got to get,
think about the recurrence that
1029
1:08:27.563 --> 1:08:31.116
will get you there.
And that could inspire you.
1030
1:08:31.116 --> 1:08:33.665
And that is it.
Recitation Friday.
1031
1:08:33.665 --> 1:08:36.446
Homework lab Sunday.
No class Monday.
1032
1:08:36.446 --> 1:08:39
See you Wednesday.