PROFESSOR: Well, last time Gerry really let the cat out
of the bag.
He introduced the idea of assignment.
Assignment and state.
And as we started to see, the implications of introducing
assignment and state into the language are absolutely
frightening.
First of all, the substitution model of
evaluation breaks down.
And we have to use this much more complicated environment
model and this very mechanistic thing with
diagrams, even to say what statements in the programming
language mean.
And that's not a mere technical point.
See, it's not that we had this particular substitution model
and, well, it doesn't quite work, so we have to do
something else.
It's that nothing like the substitution model can work.
Because suddenly, a variable is not just something that
stands for a value.
A variable now has to somehow specify a place
that holds a value.
And the value that's in that place can change.
Or for instance, an expression like f of x might have a side
effect in it.
So if we say f of x and it has some value, and then later we
say f of x again, we might get a different value
depending on the order.
So suddenly, we have to think not only about values but
about time.
And then things like pairs are no longer just their CARs and
their CDRs.
A pair now is not quite its CAR and its CDR. It's rather
its identity.
So a pair has identity.
It's an object.
And two pairs that have the same CAR and CDR might be the
same or different, because suddenly we have to worry
about sharing.
So all of these things enter as soon as we introduce
assignment.
See, this is a really far cry from where we started with
substitution.
It's a technically harder way of looking at things because
we have to think more mechanistically about our
programming language.
We can't just think about it as mathematics.
It's philosophically harder, because suddenly there are all
these funny issues about what does it mean that something
changes or that two things are the same.
And also, it's programming harder, because as Gerry
showed last time, there are all these bugs having to do
with bad sequencing and aliasing that just don't exist
in a language where we don't worry about objects.
Well, how'd we get into this mess?
Remember what we did, the reason we got into this is
because we were looking to build modular systems. We
wanted to build systems that fall apart into chunks that
seem natural.
So for instance, we want to take a random number generator
and package up the state of that random number generator
inside of it so that we can separate the idea of picking
random numbers from the general Monte Carlo strategy
of estimating something and separate that from the
particular way that you work with random numbers in that
formula developed by Cesaro for pi.
And similarly, when we go off and construct some models of
things, if we go off and model a system that we see in the
real world, we'd like our program to break into natural
pieces, pieces that mirror the parts of the system that we
see in the real world.
So for example, if we look at a digital circuit, we say,
gee, there's a circuit and it has a piece and
it has another piece.
And these different pieces sort of have identity.
They have state.
And the state sits on these wires.
And we think of this piece as an object that's different
from that as an object.
And when we watch the system change, we think about a
signal coming in here and changing a state that might be
here and going here and interacting with a state that
might be stored there, and so on and so on.
So what we'd like is we'd like to build in the computer
systems that fall into pieces that mirror our view of
reality, of the way that the actual systems we're modeling
seem to fall into pieces.
Well, maybe the reason that building systems like this
seems to introduce such technical complications has
nothing to do with computers.
See, maybe the real reason that we pay such a price to
write programs that mirror our view of reality is that we
have the wrong view of reality.
See, maybe time is just an illusion, and
nothing ever changes.
See, for example, if I take this chalk, and we say, gee,
this is an object and it has a state.
At each moment it has a position and a velocity.
And if we do something, that state can change.
But if you studied any relativity, for instance, you
know that you don't think of the path of that chalk as
something that goes on instant by instant.
It's more insightful to think of that whole chalk's
existence as a path in space-time.
that's all splayed out.
There aren't individual positions and velocities.
There's just its unchanging existence in space-time.
Similarly, if we look at this electrical system, if we
imagine this electrical system is implementing some sort of
signal processing system, the signal processing engineer who
put that thing together doesn't think of it as, well,
at each instance there's a voltage coming in.
And that translates into something.
And that affects the state over here, which changes the
state over here.
Nobody putting together a signal processing system
thinks about it like that.
Instead, you say there's this signal that's
splayed out over time.
And if this is acting as a filter, this whole thing
transforms this whole thing for some sort of other output.
You don't think of it as what's happening instant by
instant as the state of these things.
And somehow you think of this box as a whole thing, not as
little pieces sending messages of state to each other at
particular instants.
Well, today we're going to look at another way to
decompose systems that's more like the signal processing
engineer's view of the world than it is like thinking about
objects that communicate sending messages.
That's called stream processing.
And we're going to start by showing how we can make our
programs more uniform and see a lot more commonality if we
throw out of these programs what you might say is an
inordinate concern with worrying about time.
Let me start by comparing two procedures.
The first one does this.
We imagine that there's a tree.
Say there's a tree of integers.
It's a binary tree.
So it looks like this.
And there's integers in each of the nodes.
And what we would like to compute is for each odd number
sitting here, we'd like to find the square and then sum
up all those squares.
Well, that should be a familiar kind of thing.
There's a recursive strategy for doing it.
We look at each leaf, and either it's going to
contribute the square of the number if it's odd
or 0 if it's even.
And then recursively, we can say at each tree, the sum of
all of them is the sum coming from the right branch and the
left branch, and recursively down through the nodes.
And that's a familiar way of thinking about programming.
Let's actually look at that on the slide.
We say to sum the odd squares in a tree, well, there's a
test. Either it's a leaf node, and we're going to check to
see if it's an integer, and then either it's odd, in which
we take the square, or else it's 0.
And then the sum of the whole thing is the sum coming from
the left branch and the right branch.
OK, well, let me contrast that with a second problem.
Suppose I give you an integer n, and then some function to
compute of the first of each integer in 1 through n.
And then I want to collect together in a list all those
function values that satisfy some property.
That's a general kind of thing.
Let's say to be specific, let's imagine that for each
integer, k, we're going to compute
the k Fibonacci number.
And then we'll see which of those are odd and assemble
those into a list.
So here's a procedure that does that.
Find the odd Fibonacci numbers among the first n.
And here is a standard loop the way we've been writing it.
This is a recursion.
It's a loop on k, and says if k is bigger than n, it's the
empty list. Otherwise we compute the k-th Fibonacci
number, call that f.
If it's odd, we CONS it on to the list starting
with the next one.
And otherwise, we just take the next one.
And this is the standard way we've been
writing iterative loops.
And we start off calling that loop with 1.
OK, so there are two procedures.
Those procedures look very different.
They have very different structures.
Yet from a certain point of view, those procedures are
really doing very much the same thing.
So if I was talking like a signal processing engineer,
what I might say is that the first procedure enumerates the
leaves of a tree.
And then we can think of a signal coming out of that,
which is all the leaves.
We'll filter them to see which ones are odd, put them through
some kind of filter.
We'll then put them through a kind of transducer.
And for each one of those things, we'll take the square.
And then we'll accumulate all of those.
We'll accumulate them by sticking them together with
addition starting from 0.
That's the first program.
The second program, I can describe in a very, very
similar way.
I'll say, we'll enumerate the numbers on this interval, for
the interval 1 through n.
We'll, for each one, compute the Fibonacci number, put them
through a transducer.
We'll then take the result of that, and we'll
filter it for oddness.
And then we'll take those and put them into an accumulator.
This time we'll build up a list, so we'll accumulate with
CONS starting from the empty list.
So this way of looking at the program makes the two seem
very, very similar.
The problem is that that commonality is completely
obscured when we look at the procedures we wrote.
Let's go back and look at some odd squares again, and say
things like, where's the enumerator?
Where's the enumerator in this program?
Well, it's not in one place.
It's a little bit in this leaf-node test,
which is going to stop.
It's a little bit in the recursive structure of the
thing itself.
Where's the accumulator?
The accumulator isn't in one place either.
It's partly in this 0 and partly in this plus.
It's not there as a thing that we can look at.
Similarly, if we look at odd Fibs, that's also, in some
sense, an enumerator and an accumulator, but
it looks very different.
Because partly, the enumerator is here in this greater than
sign in the test. And partly it's in this whole recursive
structure in the loop, and the way that we call it.
And then similarly, that's also mixed up in there with
the accumulator, which is partly over there and partly
over there.
So these very, very natural pieces, these very natural
boxes here don't appear in our programs. Because they're kind
of mixed up.
The programs don't chop things up in the right way.
Going back to this fundamental principle of computer science
that in order to control something, you need the name
of it, we don't really have control over thinking about
things this way because we don't have our hands in them
explicitly.
We don't have a good language for talking about them.
Well, let's invent an appropriate language in which
we can build these pieces.
The key to the language is these guys, is what is these
things I called signals?
What are these things that are flying on the
arrows between the boxes?
Well, those things are going to be data structures called
streams. That's going to be the key to
inventing this language.
What's a stream?
Well, a stream is, like anything else, a data
abstraction.
So I should tell you what its selectors and
constructors are.
For a stream, we're going to have one constructor that's
called CONS-stream.
CONS-stream is going to put two things together to form a
thing called a stream.
And then to extract things from the stream, we're going
to have a selector called the head of the stream.
So if I have a stream, I can take its head or I
can take its tail.
And remember, I have to tell you George's contract here to
tell you what the axioms are that relate these.
And it's going to be for any x and y, if I form the
CONS-stream and take the head, the head of CONS-stream of x
and y is going to be x and the tail of CONS-stream of x and y
is going to be y.
So those are the constructor, two selectors for
streams, and an axiom.
There's something fishy here.
So you might notice that these are exactly the axioms for
CONS, CAR, and CDR. If instead of writing CONS-stream I wrote
CONS and I said head was the CAR and tail was the CDR,
those are exactly the axioms for pairs.
And in fact, there's another thing here.
We're going to have a thing called the-empty-stream, which
is like the-empty-list.
So why am I introducing this terminology?
Why don't I just keep talking about pairs and lists?
Well, we'll see.
For now, if you like, why don't you just pretend that
streams really are just a terminology for lists.
And we'll see in a little while why we want to keep this
extra abstraction layer and not just call them lists.
OK, now that we have streams, we can start constructing the
pieces of the language to operate on streams. And there
are a whole bunch of very useful things that we could
start making.
For instance, we'll make our map box to take a stream, s,
and a procedure, and to generate a new stream which
has as its elements the procedure applied to all the
successive elements of s.
In fact, we've seen this before.
This is the procedure map that we did with lists.
And you see it's exactly map, except we're testing for
empty-stream.
Oh, I forgot to mention that.
Empty-stream is like the null test. So if it's empty, we
generate the empty stream.
Otherwise, we form a new stream whose first element is
the procedure applied to the head of the stream, and whose
rest is gotten by mapping along with the procedure down
the tail of the stream.
So that looks exactly like the map procedure
we looked at before.
Here's another useful thing.
Filter, this is our filter box.
We're going to have a predicate and a stream.
We're going to make a new stream that consists of all
the elements of the original one
that satisfy the predicate.
That's case analysis.
When there's nothing in the stream, we
return the empty stream.
We test the predicate on the head of the stream.
And if it's true, we add the head of the stream onto the
result of filtering the tail of the stream.
And otherwise, if that predicate was false, we just
filter the tail of the stream.
Right, so there's filter.
Let me run through a couple more rather quickly.
They're all in the book and you can look at them.
Let me just flash through.
Here's accumulate.
Accumulate takes a way of combining things and an
initial value in a stream and sticks them all together.
If the stream's empty, it's just the initial value.
Otherwise, we combine the head of the stream with the result
of accumulating the tail of the stream starting from the
initial value.
So that's what I'd use to add up everything in the stream.
I'd accumulate with plus.
How would I enumerate the leaves of a tree?
Well, if the tree is just a leaf itself, I make something
which only has that node in it.
Otherwise, I append together the stuff of enumerating the
left branch and the right branch.
And then append here is like the ordinary append on lists.
You can look at that.
That's analogous to the ordinary procedure for
appending two lists.
How would I enumerate an interval?
This will take two integers, low and high, and generate a
stream of the integers going from low to high.
And we can make a whole bunch of pieces.
So that's a little language of talking about streams. Once we
have streams, we can build things for manipulating them.
Again, we're making a language.
And now we can start expressing
things in this language.
Here's our original procedure for summing the odd
squares in a tree.
And you'll notice it looks exactly now like the block
diagram, like the signal processing block diagram.
So to sum the odd squares in a tree, we enumerate the leaves
of the tree.
We filter that for oddness.
We map that for squareness.
And we accumulate the result of that using addition,
starting from 0.
So we can see the pieces that we wanted.
Similarly, the Fibonacci one, how do we get the odd Fibs?
Well, we enumerate the interval from 1 to n, we map
along that, computing the Fibonacci of each one.
We filter the result of those for oddness.
And we accumulate all of that stuff using CONS starting from
the empty-list.
OK, what's the advantage of this?
Well, for one thing, we now have pieces that we can start
mixing and matching.
So for instance, if I wanted to change this, if I wanted to
compute the squares of the integers and then filter them,
all I need to do is pick up a standard piece like this in
that square and put it in.
Or if we wanted to do this whole Fibonacci computation on
the leaves of a tree rather than a sequence, all I need to
do is replace this enumerator with that one.
See, the advantage of this stream processing is that
we're establishing--
this is one of the big themes of the course--
we're establishing conventional interfaces that
allow us to glue things together.
Things like map and filter are a standard set of components
that we can start using for pasting together programs in
all sorts of ways.
It allows us to see the commonality of programs.
I just ought to mention, I've only showed you two
procedures.
But let me emphasize that this way of putting things together
with maps, filters, and accumulators
is very, very general.
It's the generate and test paradigm for programs. And as
an example of that, Richard Waters, who was at MIT when he
was a graduate student, as part of his thesis research
went and analyzed a large chunk of the IBM scientific
subroutine library, and discovered that about 60% of
the programs in it could be expressed exactly in terms
using no more than what we've put here--
map, filter, and accumulate.
All right, let's take a break.
Questions?
AUDIENCE: It seems like the essence of this whole thing is
just that you have a very uniform, simple data structure
to work with, the stream.
PROFESSOR: Right.
The essence is that you, again, it's this sense of
conventional interfaces.
So you can start putting a lot of things together.
And the stream is as you say, the uniform data structure
that supports that.
This is very much like APL, by the way.
APL is very much the same idea, except in APL, instead
of this stream, you have arrays and vectors.
And a lot of the power of APL is exactly the same reason of
the power of this.
OK, thank you.
Let's take a break.
All right.
We've been looking at ways of organizing computations using
streams. What I want to do now is just show you two somewhat
more complicated examples of that.
Let's start by thinking about the following kind of utility
procedure that will come in useful.
Suppose I've got a stream.
And the elements of this stream are themselves streams.
So the first thing might be 1, 2, 3.
So I've got a stream.
And each element of the stream is itself a stream.
And what I'd like to do is build a stream that collects
together all of the elements, pulls all of the elements out
of these sub-streams and strings them all
together in one thing.
So just to show you the use of this language, how easy it is,
call that flatten.
And I can define to flatten this stream of streams. Well,
what is that?
That's just an accumulation.
I want to accumulate using append, by
successively appending.
So I accumulate using append streams, starting with
the-empty-stream down that stream of streams.
OK, so there's an example of how you can start using these
higher order things to do some interesting operations.
In fact, there's another useful thing
that I want to do.
I want to define a procedure called flat-map, flat map of
some function and a stream.
And what this is going to do is f will
be a stream of elements.
f is going to be a function that for each element in the
stream produces another stream.
And what I want to do is take all of the elements and all of
those streams and combine them together.
So that's just going to be the flatten of map f down s.
Each time I apply f to an element of s, I get a stream.
If I map it all the way down, I get a stream of streams, and
I'll flatten that.
Well, I want to use that to show you a new way to do a
familiar kind of problem.
The problem's going to be like a lot of problems you've seen,
although maybe not this particular one.
I'm going to give you an integer, n.
And the problem is going to be find all pairs and integers i
and j, between 0 and i, with j less than i, up to n, such
that i plus j is prime.
So for example, if n equals 6, let's make a little table
here, i and j and i plus j.
So for, say, i equals 2 and j equals 1, I'd get 3.
And for i equals 3, I could have j equals 2, and that
would be 5.
And 4 and 1 would be 5 and so on, up until i goes to 6.
And what I'd like to return is to produce a stream of all the
triples like this, let's say i, j, and i plus j.
So for each n, I want to generate this stream.
OK, well, that's easy.
Let's build it up.
We start like this.
We're going to say for each i, we're going
to generate a stream.
For each i in the interval 1 through n, we're going to
generate a stream.
What's that stream going to be?
We're going to start by generating all the pairs.
So for each i, we're going to generate, for each j in the
interval 1 to i minus 1, we'll generate the pair, or the list
with two elements i and j.
So we map along the interval, generating the pairs.
And for each i, that generates a stream of pairs.
And we flatmap it.
Now we have all the pairs i and j, such that i
is less than j.
So that builds that.
Now we're got to test them.
Well, we take that thing we just built, the flatmap, and
we filter it to see whether the i--
see, we had an i and a j.
i was the first thing in the list, j was the second thing
in the list. So we have a predicate which says in that
list of two elements is the sum of the
CAR and the CDR prime.
And we filter that collection of pairs we just built.
So those are the pairs we want.
Now we go ahead and we take the result of that filter and
we map along it, generating the list i and j and i plus j.
And that's our procedure prime-sum-pairs.
And then just to flash it up, here's the whole procedure.
A map, a filter, a flatmap.
There's the whole thing, even though this isn't
particularly readable.
It's just expanding that flatmap.
So there's an example which illustrates the general point
that nested loops in this procedure start looking like
compositions of flatmaps of flatmaps of flatmaps of maps
and things.
So not only can we enumerate individual things, but by
using flatmaps, we can do what would correspond to nested
loops in most other languages.
Of course, it's pretty awful to keep writing these flatmaps
of flatmaps of flatmaps.
Prime-sum-pairs you saw looked fairly complicated, even
though the individual pieces were easy.
So what you can do, if you like, is introduced some
syntactic sugar that's called collect.
And collect is just an abbreviation for that nest of
flatmaps and filters arranged in that particular way.
Here's prime-sum-pairs again, written using collect.
It says to find all those pairs, I'm going to collect
together a result, which is the list i, j, and i plus j,
that's going to be generated as i runs through the interval
from 1 to n and as j runs through the interval from 1 to
i minus 1, such that i plus j is prime.
So I'm not going to say what collect does in general.
You can look at that by looking at it in the book.
But pretty much, you can see that the pieces of this are
the pieces of that original procedure I wrote.
And this collect is just some syntactic sugar for
automatically generating that nest of flatmaps and flatmaps.
OK, well, let me do one more example that shows you the
same kind of thing.
Here's a very famous problem that's used to illustrate a
lot of so-called backtracking computer algorithms. This is
the eight queens problem.
This is a chess board.
And the eight queens problem says, find a way to put down
eight queens on a chess board so that no two are attacking
each other.
And here's a particular solution to the
eight queens problem.
So I have to make sure to put down queens so that no two are
in the same row or the same column or sit
along the same diagonal.
Now, there's sort of a standard way of doing that.
Well, first we need to do is below the
surface, at George's level.
We have to find some way to represent a board, and
represent positions.
And we'll not worry about that.
But let's assume that there's a predicate called safe.
And what safe is going to do is going to say given that I
have a bunch of queens down on the chess board, is it OK to
put a queen in this particular spot?
So safe is going to take a row and a column.
That's going to be a place where I'm going to try and put
down the next queen, and the rest of positions.
And what safe will say is given that I already have
queens down in these positions, is it safe to put
another queen down in that row and that column?
And let's not worry about that.
That's George's problem. and it's not hard to write.
You just have to check whether this thing contains any things
on that row or that column or in that diagonal.
Now, how would you organize the program given that?
And there's sort of a traditional way to organize it
called backtracking.
And it says, well, let's think about all the ways of putting
the first queen down in the first column.
There are eight ways.
Well, let's say try the first column.
Try column 1, row 1.
These branches are going to represent the possibilities at
each level.
So I'll try and put a queen down in the first column.
And now given that it's in the first column, I'll try and put
the next queen down in the first column.
I'll try and put the first queen, the one in the first
column, down in the first row.
I'm sorry.
And then given that, we'll put the next queen down
in the first row.
And that's no good.
So I'll back up to here.
And I'll say, oh, can I put the first queen down in the
second row?
Well, that's no good.
Oh, can I put it down in the third row?
Well, that's good.
Well, now can I put the next queen down
in the first column?
Well, I can't visualize this chess board anymore, but I
think that's right.
And I try the next one.
And at each place, I go as far down this tree as I can.
And I back up.
If I get down to here and find no possibilities below there,
I back all the way up to here, and now start again generating
this sub-tree.
And I sort of walk around.
And finally, if I ever manage to get all the way down, I've
found a solution.
So that's a typical sort of paradigm that's used a lot in
AI programming.
It's called backtracking search.
And it's really unnecessary.
You saw me get confused when I was visualizing this thing.
And you see the complication.
This is a complicated thing to say.
Why is it complicated?
Its because somehow this program is too inordinately
concerned with time.
It's too much--
I try this one, and I try this one, and I go back to the last
possibility.
And that's a complicated thing.
If I stop worrying about time so much, then there's a much
simpler way to describe this.
It says, let's imagine that I have in my hands the tree down
to k minus 1 levels.
See, suppose I had in my hands all possible ways to put down
queens in the first k columns.
Suppose I just had that.
Let's not worry about how we get it.
Well, then, how do I extend that?
How do I find all possible ways to put down queens in the
next column?
It's really easy.
For each of these positions I have, I think about putting
down a queen in each row to make the next thing.
And then for each one I put down, I filter those by the
ones that are safe.
So instead of thinking about this tree as generated step by
step, suppose I had it all there.
And to extend it from level k minus 1 to level k, I just
need to extend each thing in all possible ways and only
keep the ones that are safe.
And that will give me the tree to level k.
And that's a recursive strategy for solving the eight
queens problem.
All right, well, let's look at it.
To solve the eight queens problem on a board of some
specified size, we write a sub-procedure called
fill-columns.
Fill-columns is going to put down queens up
through column k.
And here's the pattern of the recursion.
I'm going to call fill-columns with the size eventually.
So fill-columns says how to put down queens safely in the
first k columns of this chess board with a size number of
rows in it.
If k is equal to 0, well, then I don't have to
put anything down.
So my solution is just an empty chess board.
Otherwise, I'm going to do some stuff.
And I'm going to use collect.
And here's the collect.
I find all ways to put down queens in the
first k minus 1 columns.
And this was just what I set for.
Imagine I have this tree down to k minus 1 levels.
And then I find all ways of trying a row, that's just each
of the possible rows.
They're size rows, so that's enumerate interval.
And now what I do is I collect together the new row I'm going
to try and column k with the rest of the queens.
I adjoin a position.
This is George's problem.
An adjoined position is like safe.
It's a thing that takes a row and a column and the rest of
the positions and makes a new position collection.
So I adjoin a position of a new row and a new column to
the rest of the queens, where the rest of the queens runs
through all possible ways of solving the problem
in k minus 1 columns.
And the new row runs through all possible rows such that it
was safe to put one there.
And that's the whole program.
There's the whole procedure.
Not only that, that doesn't just solve the eight queens
problem, it gives you all solutions to the
eight queens problem.
When you're done, you have a stream.
And the elements of that stream are all possible ways
of solving that problem.
Why is that simpler?
Well, we threw away the whole idea that this is some process
that happens in time with state.
And we just said it's a whole collection of stuff.
And that's why it's simpler.
We've changed our view.
Remember, that's where we started today.
We've changed our view of what it is we're trying to model.
we stop modeling things that evolve in time and have steps
and have state.
And instead, we're trying to model this global thing like
the whole flight of the chalk, rather than its
state at each instant.
Any questions?
AUDIENCE: It looks to me like backtracking would be
searching for the first solution it can find, whereas
this recursive search would be looking for all solutions.
And it seems that if you have a large enough area to search,
that the second is going to become impossible.
PROFESSOR: OK, the answer to that question is the whole
rest of this lecture.
It's exactly the right question.
And without trying to anticipate the lecture too
much, you should start being suspicious at this point, and
exactly those kinds of suspicions.
It's wonderful, but isn't it so terribly inefficient?
That's where we're going.
So I won't answer now, but I'll answer later.
OK, let's take a break.
Well, by now you should be starting to get suspicious.
See, I've showed your this simple, elegant way of putting
programs together, very unlike these other traditional
programs that sum the odd squares or compute the odd
Fibonacci numbers.
Very unlike these programs that mix up the enumerator and
the filter and the accumulator.
And by mixing it up, we don't have all of these wonderful
conceptual advantages of these streams pieces, these
wonderful mix and match components for putting
together lots and lots of programs.
On the other hand, most of the programs you've seen look like
these ugly ones.
Why's that?
Can it possibly be that computer scientists are so
obtuse that they don't notice that if you'd merely did this
thing, then you can get this great programming elegance?
There's got to be a catch.
And it's actually pretty easy to see what the catch is.
Let's think about the following problem.
Suppose I tell you to find the second prime between 10,000
and 1 million, or if your computer's larger, say between
10,000 and 100 billion, or something.
And you say, oh, that's easy.
I can do that with a stream.
All I do is I enumerate the interval
from 10,000 to 1 million.
So I get all those integers from 10,000 to 1 million.
I filter them for prime-ness, so test all of them and see if
they're prime.
And I take the second element.
That's the head of the tail.
Well, that's clearly pretty ridiculous.
We'd not even have room in the machine to store the integers
in the first place, much less to test them.
And then I only want the second one.
See, the power of this traditional programming style
is exactly its weakness, that we're mixing up the
enumerating and the testing and the accumulating.
So we don't do it all.
So the very thing that makes it conceptually ugly is the
very thing that makes it efficient.
It's this mixing up.
So it seems that all I've done this morning so far is just
confuse you.
I showed you this wonderful way that programming might
work, except that it doesn't.
Well, here's where the wonderful thing happens.
It turns out in this game that we really can have our cake
and eat it too.
And what I mean by that is that we really can write
stream programs exactly like the ones I wrote and arrange
things so that when the machine actually runs, it's as
efficient as running this traditional programming style
that mixes up the generation and the test.
Well, that sounds pretty magic.
The key to this is that streams are not lists.
We'll see this carefully in a second, but for now, let's
take a look at that slide again.
The image you should have here of this signal processing
system is that what's going to happen is there's this box
that has the integers sitting in it.
And there's this filter that's connected to it and it's
tugging on them.
And then there's someone who's tugging on this stuff saying
what comes out of the filter.
And the image you should have is that someone says, well,
what's the first prime, and tugs on this filter.
And the filter tugs on the integers.
And you look only at that much, and then say, oh, I
really wanted the second one.
What's the second prime?
And that no computation gets done except when you tug on
these things.
Let me try that again.
This is a little device.
This is a little stream machine invented by Eric
Grimson who's been teaching this course at MIT.
And the image is here's a stream of stuff, like a whole
bunch of the integers.
And here's some processing elements.
And if, say, it's filter of filter of map, or something.
And if I really tried to implement that with streams as
lists, what I'd say is, well, I've got this list of things,
and now I do the first filter.
So do all this processing.
And I take this and I process and I process and I process
and I process.
And now I'm got this new stream.
Now I take that result in my hand someplace.
And I put that through the second one.
And I process the whole thing.
And there's this new stream.
And then I take the result and I put it all the way through
this one the same way.
That's what would happen to these stream programs if
streams were just lists.
But in fact, streams aren't lists, they're streams. And
the image you should have is something a little
bit more like this.
I've got these gadgets connected up by this data
that's flowing out of them.
And here's my original source of the streams. It might be
starting to generate the integers.
And now, what happens if I want a result?
I tug on the end here.
And this element says, gee, I need some more data.
So this one comes here and tugs on that one.
And it says, gee, I need some more data.
And this one tugs on this thing, which might be a
filter, and says, gee, I need some more data.
And only as much of this thing at the end here gets generated
as I tugged.
And only as much of this stuff goes through the processing
units as I'm pulling on the end I need.
That's the image you should have of the difference between
implementing what we're actually going to do and if
streams were lists.
Well, how do we make this thing?
I hope you have the image.
The trick is how to make it.
We want to arrange for a stream to be a data structure
that computes itself incrementally, an on-demand
data structure.
And the basic idea is, again, one of the very basic ideas
that we're seeing throughout the whole course.
And that is that there's not a firm distinction between
programs and data.
So what a stream is going to be is simultaneously this data
structure that you think of, like the stream of the leaves
of this tree.
But at the same time, it's going to be a very clever
procedure that has the method of computing in it.
Well, let me try this.
It's going to turn out that we don't need any more mechanism.
We already have everything we need simply from the fact that
we know how to handle procedures
as first-class objects.
Well, let's go back to the key.
The key is, remember, we had these operations.
CONS-stream and head and tail.
When I started, I said you can think about this as CONS and
think about this as CAR and think about that as
CDR, but it's not.
Now, let's look at what they really are.
Well, CONS-stream of x and y is going to be an abbreviation
for the following thing.
CONS form a pair, ordinary CONS, of x to a thing called
delay of y.
And before I explain that, let me go and write the rest. The
head of a stream is going to be just the CAR.
And the tail of a stream is going to be a thing called
force the CDR of the stream.
Now let me explain this.
Delay is going to be a special magic thing.
What delay does is take an expression and produce a
promise to compute that expression
when you ask for it.
It doesn't do any computation here.
It just gives you a rain check.
It produces a promise.
And CONS-stream says I'm going to put together in a pair x
and a promise to compute y.
Now, if I want the head, that's just the CAR that I put
in the pair.
And the key is that the tail is going to be--
force calls in that promise.
Tail says, well, take that promise and now
call in that promise.
And then we compute that thing.
That's how this is going to work.
That's what CONS-stream, head, and tail really are.
Now, let's see how this works.
And we'll go through this fairly carefully.
We're going to see how this works in this example of
computing the second prime between 10,000 and a million.
OK, so we start off and we have this expression.
The second prime-- the head of the tail of the result of
filtering for primality the integers between
10,000 and 1 million.
Now, what is that?
What that is, that interval between 10,000 and 1 million,
well, if you trace through enumerate interval, there
builds a CONS-stream.
And the CONS-stream is the CONS of 10,000 to a promise to
compute the integers between 10,001 and 1 million.
So that's what this expression is.
Here I'm using the substitution model.
And we can use the substitution model because we
don't have side effects and state.
So I have CONS of 10,000 to a promise to compute the rest of
the integers.
So only one integer, so far, got enumerated.
Well, I'm going to filter that thing for primality.
Again, you go back and look at the filter code.
What the filter will first do is test the head.
So in this case, the filter will test 10,000 and say, oh,
10,000's not prime.
Therefore, what I have to do recursively
is filter the tail.
And what's the tail of it, well, that's the tail of this
pair with a promise in it.
Tail now comes in and says, well, I'm going to force that.
I'm going to force that promise, which means now I'm
going to compute the integers between 10,001 and 1 million.
OK, so this filter now is looking at that.
That enumerate itself, well, now we're back in the original
enumerate situation.
The enumerate is the CONS of the first thing, 10,001, onto
a promise to compute the rest.
So now the primality filter is going to go look at 10,001.
It's going to decide if it likes that or not.
It turns out 10,001 isn't prime.
So it'll force it again and again and again.
And finally, I think the first prime it hits is 10,009.
And at that point, it'll stop.
And that will be the first prime, and then eventually,
it'll need the second prime.
So at that point, it will go again.
So you see what happens is that no more gets generated
than you actually need.
That enumerator is not going to generate any more integers
than the filter asks it for as it's pulling in things to
check for primality.
And the filter is not going to generate any more stuff than
you ask it for, which is the head of the tail.
You see, what's happened is we've put that mixing of
generation and test into what actually happens in the
computer, even though that's not apparently what's
happening from looking at our programs.
OK, well, that seemed easy.
All of this mechanism got put into this magic delay.
So you're saying, gee, that must be where the magic is.
But see there's no magic there either.
You know what delay is.
Delay on some expression is just an abbreviation for--
well, what's a promise to compute an expression?
Lambda of nil, procedure of no arguments, which is that
expression.
That's what a procedure is.
It says I'm going to compute an expression.
What's force?
How do I take up a promise?
Well, force of some procedure, a promise, is just run it.
Done.
So there's no magic there at all.
Well, what have we done?
We said the old style, traditional style of
programming is more efficient.
And the stream thing is more perspicuous.
And we managed to make the stream procedures run like the
other procedures by using delay.
And the thing that delay did for us was to de-couple the
apparent order of events in our programs from the actual
order of events that happened in the machine.
That's really what delay is doing.
That's exactly the whole point.
We've given up the idea that our procedures, as they run,
or as we look at them, mirror some clear notion of time.
And by giving that up, we give delay the freedom to arrange
the order of events in the computation the way it likes.
That's the whole idea.
We de-couple the apparent order of events in our
programs from the actual order of events in the computer.
OK, well there's one more detail.
It's just a technical detail, but it's actually
an important one.
As you run through these recursive programs unwinding,
you'll see a lot of things that look like tail of the
tail of the tail.
That's the kind of thing that would happen as I go CONSing
down a stream all the way.
And if each time I'm doing that, each time to compute a
tail, I evaluate a procedure which then has to go
re-compute its tail, and re-compute its tail and
recompute its tail each time, you can see that's very
inefficient compared to just having a list where the
elements are all there, and I don't have to re-compute each
tail every time I get the next tail.
So there's one little hack to slightly change what delay is,
and make it a thing which is--
I'll write it this way.
The actual implementation, delay is an abbreviation for
this thing, memo-proc of a procedure.
Memo-proc is a special thing that transforms a procedure.
What it does is it takes a procedure of no arguments and
it transforms it into a procedure that'll only have to
do its computation once.
And what I mean by that is, you give it a procedure.
The result of memo-proc will be a new procedure, which the
first time you call it, will run the original procedure,
remember what result it got, and then from ever on after,
when you call it, it just won't have to do the
computation.
It will have cached that result someplace.
And here's an implementation of memo-proc.
Once you have the idea, it's easy to implement.
Memo-proc is this little thing that has two
little flags in there.
It says, have I already been run?
And initially it says, no, I haven't already been run.
And what was the result I got the last time I was run?
So memo-proc takes a procedure called proc, and it returns a
new procedure of no arguments.
Proc is supposed to be a procedure of no arguments.
And it says, oh, if I'm not already run, then I'm going to
do a sequence of things.
I'm going to compute proc, I'm going to save that.
I'm going to stash that in the variable result.
I'm going to make a note to myself that I've already been
run, and then I'll return the result.
So that's if you compute it if it's not already run.
If you call it and it's already been run, it just
returns the result.
So that's a little clever hack called memoization.
And in this case, it short circuits having to re-compute
the tail of the tail of the tail of the tail of the tail.
So there isn't even that kind of inefficiency.
And in fact, the streams will run with pretty much the same
efficiency as the other programs precisely.
And remember, again, the whole idea of this is that we've
used the fact that there's no really good dividing line
between procedures and data.
We've written data structures that, in fact, are sort of
like procedures.
And what that's allowed us to do is take an example of a
common control structure, in this place iteration.
And we've built a data structure which, since itself
is a procedure, kind of has this iteration control
structure in it.
And that's really what streams are.
OK, questions?
AUDIENCE: Your description of tail-tail-tail, if I
understand it correctly, force is actually execution of a
procedure, if it's done without this memo-proc thing.
And you implied that memo-proc gets around that problem.
Doesn't it only get around it if tail-tail-tail is always
executing exactly the same--
PROFESSOR: Oh, that's--
sure.
AUDIENCE: I guess I missed that point.
PROFESSOR: Oh, sure.
I mean the point is--
yeah.
I mean I have to do a computation to get the answer.
But the point is, once I've found the tail of the stream,
to get the tail of the tail, I shouldn't have had to
re-compute the first tail.
See, and if I didn't use memo-proc, that re-computation
would have been done.
AUDIENCE: I understand now.
AUDIENCE: In one of your examples, you mentioned that
we were able to use the substitution model because
there are no side effects.
What if we had a single processing unit--
if we had a side effect, if we had a state?
Could we still practically build the stream model?
PROFESSOR: Maybe.
That's a hard question.
I'm going to talk a little bit later about the places where
substitution and side effects don't really mix very well.
But in general, I think the answer is unless you're very
careful, any amount of side effect is going to mess up
everything.
AUDIENCE: Sorry, I didn't quite understand
the memo-proc operation.
When do you execute the lambda?
In other words, when memo-proc is executed, just this lambda
expression is being generated.
But it's not clear to me when it's executed.
PROFESSOR: Right.
What memo-proc does-- remember, the thing that's
going into memo-proc, the thing proc, is a procedure of
no arguments.
And someday, you're going to call it.
Memo-proc translates that procedure into another
procedure of no arguments, which someday
you're going to call.
That's that lambda.
So here, where I initially built as my tail of the
stream, say, this procedure of no arguments, which
someday I'll call.
Instead, I'm going to have the tail of the stream be
memo-proc of it, which someday I'll call.
So that lambda of nil, that gets called when you call the
memo-proc, when you call the result of that memo-proc,
which would be ordinarily when you would have called the
original thing that you set it.
AUDIENCE: OK, the reason I ask is I had a feeling that when
you call memo-proc, you just return this lambda.
PROFESSOR: That's right.
When you call memo-proc, you return the lambda.
You never evaluate the expression at all, until the
first time that you would have evaluated it.
AUDIENCE: Do I understand it right that you actually have
to build the list up, but the elements of the
list don't get evaluated?
The expressions don't get evaluated?
But at each stage, you actually are building a list.
PROFESSOR: That's--
I really should have said this.
That's a really good point.
No, it's not quite right.
Because what happens is this.
Let me draw this as pairs.
Suppose I'm going to make a big stream, like enumerate
interval, 1 through 1 billion.
What that is, is a pair with a 1 and a promise.
That's exactly what it is.
Nothing got built up.
When I go and force this, and say, what happens?
Well, this thing is now also recursively a CONS.
So that this promise now is the next thing, which is a 2
and a promise to do more.
And so on and so on and so on.
So nothing gets built up until you walk down the stream.
Because what's sitting here is not the list, but a promise to
generate the list. And by promise,
technically I mean procedure.
So it doesn't get built up.
Yeah, I should have said that before this point.
OK.
Thank you.
Let's take a break.