The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare
continue to offer high quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: And so I want to thank you all for coming.
As the holiday season comes along here
I'm guessing that our people are getting
distracted with other things.
So a little bit more of an intimate setting today.
So I urge people to take advantage of that,
ask questions.
It's often a lot easier to do when there aren't
as many people in the room.
So I just really want to encourage you to do that.
So let's get right into this.
All right.
Fortunately the mic actually doesn't pick that
up at all for the most part.
But if you all can hear the drilling of a-- yeah.
All right, so this is Lecture 4.
I kinda went out of order to get our special Halloween lecture
in last time.
But we're going to talking a little bit more
about the analysis of what we call structured data.
And we're doing more structured type analyzes
on unstructured data.
And of course the signal processing on databases,
this is where we bring the ideas of detection theory
and apply them to the kinds of data
we see in databases, strings, and other types of things
there.
And so this lecture is going to talk a little bit more
about-- we're going to get into a little bit
more sophisticated analytics.
I think up to this point we've done a fairly simple thing,
basic queries using the technology, pretty
simple correlations, pretty simple stuff.
Now we're going to begin to kind of go
a little bit more sophisticated in some of the things we do.
And again, I think as things get more complicated the deform
technology becomes, its benefits become even more apparent.
So we're just going to get right into this,
gonna show you our very generic scheme,
gonna talk a little bit more about some of the issues
that we encounter when we deal with particular databases.
But the database we're going to be using for this course
is called accumulo.
And so I'll be talking about some issues that
are more accumulo specific.
So accumulo is a triple store.
And so the way we read data into our triple store
is using what we call this exploded transpose pair schema.
So we have-- [INAUDIBLE].
We have an input data set that might
look like a table like this with maybe our row key
is going to be somehow based on time,
and then we have various columns here, which may or may not
contain various data.
And then the first thing we do is we basically
explode, because our triple stores can hold an arbitrary
number of columns.
It can add columns dynamically without any costs.
We explode this schema by appending
our Column 1 and its value together in this way.
We then just took a value, could be anything,
like one, anything that we wouldn't want a search on.
And then we have a row key here.
I'll get into back a little bit why we've
done the row key in this way.
And so by itself this doesn't give us any real advantage.
But in accumulo or any other triple store end
or in just in D form sociative arrays,
we can store the transpose of the table, which
means that all these columns are now rows.
And this particular database is a row oriented database,
which means it can look up any row key with constant access.
So it can look up row keys very quickly.
And D4M a lot we hide this from you
so that whenever you do inserts it does,
if you want it to happen, it will
store the transpose for you.
And so you look like you have a giant table where
you can look up any row key or any column very, very quickly
and makes it very easy to do very complicated analytics.
One little twist here that you may have noticed
is that I have flipped this time field here,
taking it from essentially little endian
and made it big endian.
And I've always been an advocate of having row keys that
have some meaning in them.
Sometimes in databases they just create your arbitrary random
hashes.
I think we have enough random data in the world
that we don't necessarily need to create more,
and so if we can have meaning in our row keys,
I think it's very useful, because it just
makes it that much easier to debug
data that actually has meaning.
And so I've often advocated for creating having
a row key be a timelike key.
I think it's good to have timelike keys.
And by itself there's nothing wrong
with having a little endian row key,
except when you go parallel.
If this database is actually running on a parallel system,
you can run into some issues.
People have run into these issues, which
is why I now advocate essentially doing
something else with the row key more like this.
In particular, accumulo, like many databases,
when it goes parallel it takes the tables
and it splits them up.
And it splits them up by row keys.
So it creates continuous blocks of row keys
on different processors.
If you have a little endian time row key,
it means every single insert will go to the same processor.
And that will create a bottleneck.
And then what will happen over time
it will then migrate that data to the other processor.
So it can be a real bottleneck if you have essentially
a little endian row key.
If you have a big endian row key, if it just uses,
it will then break up these things.
And then when you insert your data, it will sort of naturally
just cause that to spread out over all the systems.
So that is if your data is coming in in some kind
of time order, which happens.
We definitely see people coming in that yesterday,
you know, today's data comes in today
and tomorrow's data comes in tomorrow.
You don't want to have it all that data just hitting
one processor or one compute node in your parallel database.
Other databases have more sophisticated distribution
things they can use as sort of a round robin
or a modular type of-- they'll create a hash that
does a modular so that it doesn't have that,
eliminates that type of hotspot.
But for now we just recommend that people use this.
This does make it difficult to use the row
key as your actual time value.
And so what you would want to do is also stick,
have a column called time that had the actual time in it.
And then you could actually directly look up
a time in that way.
So that's just a little nuance there,
good sort of design feature.
Probably something that you wouldn't
run into for quite a while, but we've definitely had people,
hey I went off and implemented exactly the way
you suggested, but now I'm going paralyzed
seeing this bottleneck.
And I'm like oh.
So I'm going to try and correct that right now and tell people
that this is the right way to do it.
So starting out simple, we're just
going to talk about kind of one of the simplest analytics you
can do, which is getting basic statistical information
from your data set.
So if this is our data set, now we have a bunch of row keys
here, timelike with some additional stuff
to make them unique, unique rows.
And then we have these various columns
here, some of which-- the gray ones are, I'm saying,
are filled in, and the white ones are empty.
And so what I want to do is just grab a chunk of rows.
OK, and then I'm going to compute basically
how many times each column appears, essentially some.
I'll sum by type.
So I'll show you how we can look at if you
want to compute just how many entries there
were in column 1 or column 2, computing the covariance,
computing type and para covariances are all
different things that we can do.
So I'm going to do all those statistics on the next slide.
All those analytics, which if you
were to do them in another environment would actually be,
you would write a fair amount of code.
But here we can do them and each one of them
is essentially a one liner.
So here's my set of row keys.
OK?
I've just created a list of row keys
that have essentially a comma as the separator here.
And there, service that.
And this is our Table T. So in the notation
we often just refer Table T. This
is in a binding to an Accumulator Table
or it could be any table really, any database that we support.
And so this little bit of code here
says, return me all the rows given by this list of rows here
and all the columns.
So we're using the Matlab notation.
Colon means return all the columns.
This will then return these results
in the form of an associative array.
Now since the values of that are strings,
in this case maybe strings values of 1,
we have this little function here
called double logi, which will just basically
say ignore whatever the value is and just make it--
if it's got an entry, give it a 1 or otherwise ignore it.
So this is a shorthand.
It basically does a logical, and then it does a double,
so we can do math on it.
So this is our query that gets us our rows
and returns them as an associative array
with numeric values.
We then compute the column counts.
So that's just the Matlab sum command,
which says basically this tells you the dimension that's
being compressed.
So it's compressing the first dimension with the rows.
So it's basically collapsing it into a row vector.
So it's just summing.
So I just would tell us now we could then for all those rows
count how many occurrences of each unique column,
of each column type there was.
And then we can then get the covariance,
the type type covariance by just doing A transpose A or square
in.
These do the same thing.
This is usually slightly faster but not a lot.
And so that just does the column type, column type, covariance,
very simple.
And then finally we have this function, which essentially
undoes our exploded schema.
So let's say we wanted to return it
back to the original dense format
where we have essentially four columns, column 1, 2, 3, and 4.
And we wanted to value put back into the value position.
So we have this function call to type with this--
and I don't know if the name makes sense
or not, but that's what we call it-- and basically just says,
oh, this is the limiter.
So it takes each one of those, splits it back out,
and stuffs it back in.
And so then you now have that associative array.
You can then do a sum again on that
to just get like I want to know just how
many column 1 instances, column 2 instance,
column 3s there are.
And likewise, just doing A transpose A or square in
would then do the covariance of that.
And that would tell you of those higher level
types how many of there were.
So this is a lot of high level information.
Really highly recommend that people do this when they first
get their data, because it really
uncovers a lot of bad data right from the get go.
You'll discover columns that are just like,
wow these two column types never appear together.
And that doesn't make sense.
There's something wrong there.
Or why is this exploded column, why is it so common when
that doesn't make sense?
So again, it's very, very useful for doing things.
And we always recommend people start with this analytic.
So again, that's very simple stuff.
We've talked about it before.
Let's talk about some more sophisticated types
of analytics that we can do here.
So I'm going to build what I call a data graph.
So this is just a graph in the data.
It may not actually be a real.
You may have some other graph in mind,
but this is the data supports as a particular kind of graph
in it.
So what we're going to do here is
we're going to set a couple starting column here.
OK, C0, that'll be our set of starting columns.
And we're going to set a column.
I said I've allowed column types.
So we're going to be interested in certain column types.
And we're going to create a set of clutter columns.
These are columns that we want to ignore.
We want to-- they're either very, very large or whatever.
And so the basic algorithm is that we're
going to get all the columns.
Our result is going to be called C1 here.
OK, and that's going to be all rows in C0 that are of type CT,
and excluding columns CL.
So this is a rather complicated joined type of thing
that people often want to do.
They want to do these types of things.
Look, I want to get all these rows,
but I only care about these particular types of columns.
And I don't really care about, or I
want to expressively eliminate certain clutter columns
that I know are just like always pointing everywhere.
So let's go look through the D4M Code that
does this sort of complicated type of join.
So I'm going to specify my C0 here.
And this could be a whole list.
It's just the whole list of starting columns.
I'm going to specify my column types.
In this case I'm going to have two column types.
I'm gonna say, create a string called starts with,
which essentially creates a column range,
one column range around column 1,
and one column range around column 3.
And then I'm going to specify a clutter column here,
which is just this A. And again, it
could be a whole list as well.
All right, so step one here is I'm going to pass.
I'm assuming that this table is bound
to one of these exploded transposed pairs.
So it will know when I give it columns
to look up to then point to the correct table.
So we have a C0 here.
And it will say, all right, please return all rows,
all the data that contains C0, basically all those rows.
I'm then going to say now I don't care about their values,
I just want the rows.
So this is an associative array, and this command row just says,
just give me those rows.
So basically I had a column, I returned all the rows,
but now I just want the row keys.
I'm now going to take those row keys
and pass them back into the row position.
So now I will have gotten the entire row.
So basically I got a column, I took those rows,
and now I'm going and striping and painting and getting
the whole rows.
So now I have every single.
And since I don't care about the actual values,
I just want them to be numeric, I just
use the double logi command.
So basically I've done a rather complicated little piece
of query here in terms of get me all columns that contain,
all rows that contain a certain column.
And so that's now an associative array.
I'm then going to reduce to specific allowed columns.
So I'm going to pass in.
I'm going to say, please give me just
the ones of these particular column types.
So I got the whole row, but now I just want to eliminate those.
I could have probably actually put this in here,
but whether it's more efficient or not to do it here or here,
it's 6 of one, half dozen of the other.
I try and make my table queries just
either only columns or only rows.
It tends to make things simpler.
So now we do that.
We have just those types.
And now I want to eliminate the clutter columns.
So I have A, which is just of these type.
And I want to then eliminate any of the columns that are
in this, basically, column 1.
So I had column 1 as one of my types,
but I don't care about that.
And so I can just do subtraction.
I can just basically say, oh go get the clutter columns
and subtract them from the data that I have.
And now I'm just going to get those columns,
and I have now the set of columns.
I now have C1.
So I've completed the analytic I just
described in the-- if you're grabbing like four lines rather
sophisticating analytic, I could then proceed
to look for additional clutter.
For instance, I could then basically query say,
please give me those, stick the C1 back in and then
sum it up and look for things that had a lot of values
and continue this process.
Just an example of something I might
want to do when I have this data set.
So these show you the kinds of more sophisticated analytics
that you can do.
I want to talk a little bit about data graphs
in terms of what are the things you can do here,
what is supported.
The topology of your data, remember
this edge list has direction.
When you have graphs they can very well
have edges that have direction.
And a lot of times people will say,
well I want to get the whole graph.
I'm like, well, if you're doing essentially things that
are based on breadth first search,
you may not be able to get the full graph,
because you're never going at it from the correct direction.
So you are definitely limited by the natural topology
of your data.
So for example here, let's say I start with C0 as column 1A.
So I now have essentially this vertex A let's call it.
And now here I say, OK, I now want to get all.
Give me the whole row of all I'm going to do.
Give me all the rows that contain that.
All right, so then these two guys pop up.
So I get another A and I get a B. So I got a B here.
That's good.
And then I'm going to go, proceed, go down again.
I'm like, all right.
I'm going to then now say, give me all the rows that
contain those columns.
And I go down again, and I got the C. Did I get a C?
No, I never got a C. I never got a C in any one of these,
even though it's all in the data and probably all connected.
I never actually got a C.
There I got the C. Now when I did it the second time
I got the C, so there we go.
So this is an example of a series
of breadth first searches that result
in getting the whole graph.
But the graph had this typology and wouldn't naturally
admit that.
So certainly in this particular case the data and the queries
were good for this, we would call a star,
because essentially it's a vertex
with everything going into it.
Let's take a different graph.
This is what we call a cycle.
So we see we have a little cycle here going like this.
We start again with A. We get our columns.
We get C1s across here.
And that's kind of the end of the game.
We get A's, we get a B, but we're not
going to get anything else when we column back up.
We're not going to get anything else.
And so we're basically stuck here
at B. We weren't able to get to C or D.
So these are the kind of a little more subtle things
that everyone has to worry about.
And once you see it it's kind of, well, of course.
I can't do-- I'm not going to get the whole graph.
But you'd be amazed how many teams were like,
I wanted the whole graph, and I just can't do it.
It's like, well, you don't have the edges going
in the right direction for you to do that.
You're going to have to think things
through a little bit more.
So I just want to-- it's kind of a little catch
that I want to point out to people, because it's something
that people can run into.
We're going to do a little different type of analytic
here.
I've changed some of my columns here.
I have some.
Let's call these coordinates.
I'm going to have now with my data
set an x and a y-coordinate that I'm storing in different rows
and columns.
I want to do some kind of space windowing.
I want to find all data within a particular x and y-coordinate.
So what I'm going to do is I'm going
to select a set of data here R instead of rows.
And I'm going to give a space polygon.
And I'm going to query, get the data.
And then I'm going to extract the space coordinates
from the values there, and I'm gonna
return all columns that are within my space window here.
And again, this is good for finding columns
in between your space window.
If you're concerned that you're going
to be getting an awful lot of if you have,
let's say you have a coordinate that goes through New York.
And you're concerned that's just going
to-- you don't want to get, you don't want New York,
but you happen to be on the same latitude
and longitude of New York.
You can do something called Mertonization, which basically
is essentially imagine taking your strings
of your coordinates and interleaving them.
And now you've essentially created an Ascii based grid
of the entire planet.
And so that's a way of if you want to quickly filter down,
you can get a box and then go back
and do the detailed coordinates to vent yourself
from having to do.
So that's a standard trick.
And there's a variety of Mertonization schemes
that people use for interleaving coordinates in this way.
I think Google Earth has a standard box now as well.
I find this the simplest, because you literally just
take the two strings and interleave them.
And if you have a space, you can actually then
do with variable precision, because if you just like,
space I don't know.
And it all kind of works out pretty nicely.
And you can read the coordinate right there.
Like, the first one is the first.
And you can even include the plus and minus signs for a lot
if you wanted to.
So maybe not the most efficient scheme, but one that's human
readable.
And I'm a big advocate of human readable types of data schemes.
All right, so let's actually do that analytic.
So again, I created my-- selected my row,
got my x and y-coordinates in those rows,
and then figured out which columns they were that
satisfied that.
So let's do that now in code.
Let's see here.
So we have the-- all right, so we have,
in this case I gave It a row range.
So you can do-- this is what a range query looks like.
If you give any, either an associative array or a table,
something that is a triple, essentially that's a string,
colon, and another string, it will
treat that as a range query.
And we actually support doing if you
have multiple sets of triples, it
should handle that, which is good.
I'm going to specify my bounding box here, essentially a box.
And I happen to do with complex numbers
just for fun, just because complex numbers are a nice way
to store coordinates on a two dimensional plane.
And Matlab supports them very nicely.
Complex numbers are our friends, so there you go.
But I could have just as easily had a set of x and y vectors.
So I'm going to get all the rows.
So I query that.
Very good, we have that.
And then, so that just gives me that set
of data, all those rows.
I then use my starts with my x and y to get just the columns
of those X's and Y's.
And I'm now kind of going to convert those exploded back
into a regular table with this call to type function.
So that basically takes those values.
So it takes those coordinate values, so like we saw.
It takes these coordinate values here like the 0 1,
and it puts it back into the value position.
So now I have, though, it will still
be a string in the value position, which
our associate arrays can handle fine.
But I now want to really treat it like a number.
So we just have overloaded the standard Matlab string
to numb function, which will convert those strings
and will store them back.
You now have an associated array with numbers in them.
So we call this Axy.
And now we can do something.
We basically can extract the x values here.
So we have Axy, and say, all right,
give me the x column, and then Axy and give me the y column.
And Matlab has a built in function called in polygon,
which you give it a polygon.
So I give it the real and the imaginary parts
of my polygon here S and the x and y-coordinates.
And it will return in essentially the value
of is that in there, which is great,
because there are many dissertations written
on the in polygon problem.
And it's nice that we have a nice built in Matlab function
to do that.
And then now I have that, and I can just pass that back
into the original A. So I do find.
This actually returns a logical of zeros and ones.
If I do find, then that will return a set of indices.
And I just pass those indices back into A.
And then I can get those columns and there
we go, all very standard Matlab like syntax.
Again, this is a fairly complicated analytic.
If you were doing this using other technologies,
I mean, I'm sure all of us would be
writing a fair amount of code.
And this is just the kind of thing
that we can do very easily in D4M.
Another analytic, which is probably a bit of a stretch,
but I was just having fun here, is doing convolution
on strings, which is a little odd.
But I gave it I gave it a whirl.
So what we want to do is we want to convolve some of our data
with a filter.
I mean, convolving the filters as a standard type of thing.
So it's the standard way we do detection here.
And so the way we do that is once again,
I give a list of rows that I want here.
I'm going to create a filter, which is just essentially
a box of 4 wide box.
So I get my rows.
And then I convert them to numeric.
And I'm going to do my convolution on the x columns.
So let's see here.
So I'm going to get these.
I'm basically getting all the x-coordinates.
I'm going to sum all of those, so I basically now
have all those.
And now I'm going to pop those back into their values.
And now I can do a convolution.
And this convolution works if one of the axes
is sort of like an integer sequence type of thing.
So you can do-- it tries to extend that naturally.
So something to play around with if you want to do convolutions.
We sort of support it.
And I'm sure if anyone of you do play around it,
we would be glad to hear your experiences,
think about how we should extend it.
So these are all sort of basic standard first order
statistical analytics that one can do on data sets.
And we can support them very, very well.
Let's do some more complicated what I would call second order
analytics.
So I'm going to do something called,
it's a complicated join essentially,
what I call a type pair.
So what I want to do here is I want
to find all rows that contain values.
I want to find rows that have both value of type 1
and of type 2.
So I'm going to specify this to be, basically x to be type 1
and y to be type 2.
And I want to find all data that has
entries in both those very, very standard type
of join type of thing.
And this is done a little bit more complicated
than we need it to be just to show you kind
of some of the richness here.
You can kind of take a fork in any way.
We could probably do this whole thing in about two lines,
but I'm kind of showing you some additional features of D4M
in the spirit of this analytic.
So again, I'm just going to use a range query here.
So I have this range.
I'm going to have my type 1 be starts with x, and my type 2
be starts with y.
So I do my query.
I convert all the string 1's to numeric 1's.
And then what I'm going to do is I'm
going to basically, all right, get me all the columns of type
1, sum them all together, find everything that equals--
and I only care about the 1's that are exactly equal 1.
So like if I had two x's, I'm like no.
I don't want those.
I want exactly one x in this row.
And then I'm going to take those rows that have exactly one x.
I'm going to pass them back into A.
So I now just get the rows that have exactly one x.
I'm going to filter it again with ct1 and ct2,
although, I don't need to do that.
Then I'm going to sum it again.
And now I'm going to say, all right,
give me the only ones that are exactly 2.
So I know my x is exactly 1.
So in order for it to be exactly 2, that means my y also
had to have only exactly one entry in it.
So now I have the data that just has
exactly one of each of those.
Now I want to create sort of like a cross-correlation pair
mapping of this.
So I'm actually to look for x's across columns
that appear say, I want to look for x's
that appear with more than one y or a y that
appears with more than one x.
So there's a variety of ways to do that.
Here what I'm doing is-- so I have gotten the rows of A that
have exactly one x and y.
I now pass that back again to get my C, to get the x's again.
And one of the things that we've done that's kind of nice
is we've overloaded the syntax of this query
on our associated arrays and on our table queries
such that if it only has one argument,
it will return an associative array.
But if you give it three output arguments,
it will return the triple, so in this case, the row, the column,
and the value.
Now I don't care about the row and the value,
I just care about the column.
But that's a nice way.
We're often gonna in certain cases
want to bump back and forth between
the triples representation and the associated array
implementation.
Now you always can use the Find command
around any associated array just as you can in normal Matlab
matrices not return the triple.
The advantage of doing it here is
that it's faster, because what we actually
do when we do the query internally
is we actually get the triples and then convert
to an associative array.
And if you just say I want the triples,
we can just short cut that and give you the triples
right away.
So sometimes if you're dealing with a very large
associative arrays or some operation that's just
I want to get some more performance back then working.
Especially if you like, well, I only care about one thing,
I don't care about all of the values,
I don't really need to be a full associative array,
then that's a great way to sort of short circuit that.
So we do that here.
And now we can construct a new associate array,
which is just taking the x's and the y's and creating
a new associative array with those.
And that just shows me the correlations between
the x's and the y's.
And I can then find ct, basically x's that have more
than one y-- so I've just summed them there--
or y's with more than one x's.
Again, these are very similar to analytics
that people want to do.
And again, very simple to do.
And again, showing you some of the different types of syntax
that are available to you in D4M.
Again, if you're used to using Matlab, these types of tricks
are very natural.
We're just showing you that they also exist within D4M.
So here's another one.
So I wanted to find column of pair set C1 and C2,
and get all columns C1 and C2, and find the rows that have
just one entry in C1 and C2.
And it basically checks to see if data pairs are
present in the same row.
Again, something that people often want to do.
You've got a complicated type of join.
So here we have a set of columns one C1, a set of column C2.
I want to create this, I want to sort of interleave these
together into a pair.
So I want to create some concept of a pair.
And so we have this function here
called cat string, which basically
will take two strings and another delimiter,
and will basically if they are of the same number of strings
will just leave them together.
If one of these is just a single string,
it will just an essentially preappend or append that.
So for instance, if you are wondering how we actually
create these exploded values like call 1 and B,
that's just basically using this function here.
We get the values, we get the columns,
we put essentially the pipe thing in the middle,
and it just merges them together.
So we now sort of interleave these two together.
So we'll now have something like Col1 b pipe Col3
b pipe comma as the separator.
So now I can create a set of pair mappings
from C1 to its pairs.
OK, that's A1 to its pairs and A2 to it's pairs.
I can get the columns of those A1 and A2.
And then I can find all the pairs by essentially
through this combination of matrix multiplies and additions
and etc.
So a very sort of complicated analytic done very nicely.
And then there's a whole bunch of different ones
you can do here.
These are almost semantic extension.
The column types may have several different types,
and you want to do.
So for instance, if I have a pair of columns here,
column 1 and column 3, I could say, well,
that also implies this.
Column three equals column 1.
That's one kind of sort of pair reversal type of thing.
You'll have extensions.
You might say, look if I have a column
1A that also implies that really there
should also be a column 2A, and other types
of things like that.
So these are just types of things
that people do with pairs.
They're often very useful.
And I think that basically brings us
to the end of the lecture portion of class.
So again, just the exploited schema
really allows you to do this very rapidly with your data.
And you can implement a very efficient graph analytics
as a sequence of essentially row and column queries,
because we use this very special exploded transpose pair schema.
And increasingly as you become more and more skilled
with this, you will discover that many, many, many
of your analytics really reduce to matrix matrix multiplies.
That matrix matrix multiply really
captures the whole sort of all correlation
that you want to do without having to kind of figure things
out.
All right, so I'm now going to go and show some
not these specific analytics, but some analytics
that are more sophisticated based on the Reuters data set.
If you remember a few weeks ago, we
worked with the Reuters data set.
And so let's see here.
So we already did the entity analysis application
a few weeks ago.
I'm going to now do basically what
happens when you construct tracks,
which is a more sophisticated structured analytic.
And the assignment I'll send out was basically
doing more cross correlations.
For those of you who have kept it going here this far
and continue to do the homeworks,
I'll send this homework out to you,
which is basically just cross correlating
the data sets that you have.
Again, and not an assignment that really predispose
requires you having done the previous assignments.
Just any data set, pull it into an associate array,
and then do matrix multiplies to figure out
the cross correlations and what they mean.
All right, so with that, why don't we
take a short five minute break.
And then I'll come back and show you the demo.