The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare
continue to offer high quality educational resources for free.
To make a donation or view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at OCW.MIT.edu.
PROFESSOR: So I am going to start this talk
by explaining the title.
So we'll start with active learning.
What is active learning?
In my mind, it's this pretty broad term.
It's a form of learning that attempts to involve students
more directly in the learning process.
And there are all sorts of things
that fall under the umbrella of active learning.
So things like having students engage in group
work, in class-wide discussions, in debates,
acting out certain concepts, group problem solving
sessions, all of these things.
And so in all of those activities,
students are engaged.
They're not passively listening to a lecture,
they're actively participating.
And there is some evidence that this style of learning
is good for a lot of things.
There is evidence to support the effectiveness of student
engagement in exam scores, failure rates, how
well students remember content, student attitudes, study
habits.
And there's also evidence that active learning
has a disproportionate benefit for minorities, students
from disadvantaged backgrounds, female students,
and male dominated fields.
So there's evidence that active learning creates this more
inclusive environment.
So it's this wide, wide variety of techniques reported
to have a lot of benefits.
So that's active learning.
Now, what is 6.033?
So 6.033 is computer systems engineering.
It's a required class for all computer science
majors, which means as a result, at MIT it's quite large.
So like Dipa said, we have about 400 students.
It runs once a year.
So every spring we get 400 new students.
Most of them are juniors and seniors.
And in some ways, 033 is a typical MIT class.
We have lectures twice a week.
We have recitations twice a week.
But in most other ways, 033 is not a typical MIT class.
For one, it's a CIM class.
So it's a Communications Intensive in the Major class.
That means that most of our assignments
are based around writing or oral presentations.
It's not a class where students are implementing or building
a lot of things.
We focus heavily on reading and on writing.
In particular, we have tutorial sections each week,
dedicated to the communication material,
taught by dedicated communication staff.
The major assignment in 033 is a design project.
It takes most of the semester.
The students completed in teams, usually teams of three.
And it's comprised of a few written assignments
and oral presentations.
One of our big focuses for that project is getting students
to recognize the trade-offs they need to make when they're
designing a system, and getting them
to defend those trade-offs.
So this is not just a class about
how computer systems work, but how we design them,
how we decide what choices to make when
we're building those systems.
So the last thing that makes 033 pretty unique,
and is what I'm largely going to talk about here,
is that our recitation sections are pretty intense.
They are taught by faculty, senior research staff.
We have about 25 students per section.
So you are right to think that, oh, my gosh, they have
like 18 sections per semester.
We do.
It's a lot.
Each recitation focuses around a technical paper
that the students have read beforehand.
And the goal for those discussion sections,
those recitations, is for them to be largely discussion based.
And in fact, that has always been the goal of 033.
033 has been around for almost 50 years at this point.
It's been around since before I was born.
And the goal has always been for these recitations
to be discussion based.
During the recitations, we're talking about,
we talk about the system that the paper describes--
does it work?
What were the trade-offs that the authors made?
Do we like this system?
How would you design this system?
All right, so that is 033.
Now, why put active learning into 033?
And you might think that it's just because, well, there's
evidence that active learning can be a good idea,
so why not try it?
That's fair.
We were actually trying to solve a pretty specific problem
in the class though.
So like I said, ideally the recitations
are based heavily around discussion.
And again, that has always been the goal.
Because this is a class about system design, not just about
how these systems work, one way that we teach
that is by having the students analyze and critique
existing systems before they go out
and try to design their own.
And so, what are the goals of the designers?
Does the system meet those goals?
Are those goals important?
If you were redesigning the system, how would you do it?
What would your goals be?
What trade-offs would you have to make?
Things like that.
A large part of that instruction happens
in the recitation, where we're having these discussions.
And they go and apply those skills
to their design projects.
So to get that to happen, it's important that the students are
engaged in recitation, that they're
reading the paper beforehand, they're asking questions,
they're answering questions, they're coming prepared,
things like that.
And for a long time, that was not happening for us.
We had a lot of trouble with this.
We found that students weren't willing to volunteer answers.
When we'd ask them a question, nobody would raise their hand.
We found that some of them weren't even reading
the paper beforehand.
So they're just kind of coming in confused.
As a result of all of this, our instructors
would end up spending the first 15, 20, 25
minutes of the recitation lecturing about the paper,
because so many of the students didn't come prepared.
And then that just made things worse, as you can imagine.
Students know that they're going to come into recitation,
that they're going to get a lecture,
so why bother reading the paper, if your instructor is just
going to explain it to you.
And so it was just kind of making things worse.
We weren't getting to teach the material that we really
needed to teach, because we were spending time
rehashing these papers.
The students weren't learning how
to read these papers, how to analyze them,
how to critique them.
And honestly, it wasn't much fun.
It wasn't all that much fun to teach this way.
Occasionally, we could get a discussion going in a class.
But we would then encounter another problem, which
is pretty common, where we'd have a discussion,
but it would be dominated by two or three students
in the room, the same two or three students.
And then other students are tuning out,
or even becoming sort of visibly uncomfortable,
not wanting to volunteer these answers.
So that's where we were.
And we had this problem.
We really wanted the recitations to be this really fun place,
where the students were coming, and they're
wrestling with all of these new ideas
and proposing these new things.
But we didn't have that.
We had this place where students aren't coming prepared,
or if they're prepared, if we can get a discussion going,
it's not the kind of discussion we want.
It's sort of dominated by one or two people.
So after some thinking, I thought
active learning might be a good approach
to solve both of these problems.
So, as I mentioned earlier, there's
some evidence that active learning
can be good for creating an inclusive environment
in a classroom.
So for instance, one common technique,
which I will talk a bit about later,
is you pose a question to the class,
but put everybody in small groups,
have the students discuss in small groups,
and then kind of bring the classroom back together
for a wide discussion.
Students are in their groups, they
get a chance to kind of vet their ideas with their peers.
It's sort of hard if you're in a group of two or three
to not participate.
You sort of have to say something.
So they'll get to vet their ideas with people,
they'll get some reassurance.
And in theory, they are then more willing to bring that idea
back to the whole class.
That type of activity can also give you a relatively easy way
to make sure one person doesn't dominate the discussion,
by saying, OK, I split you guys up into five groups,
and let's hear from each group now.
Moreover, I thought if we did a lot of active learning,
it might keep the students engaged for longer.
It would help us avoid lecturing.
It would help us set this precedent
that you come to recitation having read the paper,
we're not going to lecture for you.
So that's kind of where we're starting.
That's the baseline for this talk.
Now my goal for all of you is that this talk is, I hope,
exceedingly practical.
I'm going to talk about some specific ways
that I trained my staff to do this,
the types of active learning activities
we do, some lessons we've learned,
how we've evaluated our success.
I hope you'll be able to apply some of these ideas
to your own classes if you want.
I do want to start, though, with a disclaimer.
This is really a case study of how active learning has worked
in 033.
I'm not here to make claims that active learning is
a solution to all of your teaching problems.
I think it's very good.
It's probably not the solution to everything.
Nor am I even--
I'm not even here to really evangelize it
over any other particular pedagogical schemes.
I think it works well in 033, I think we've done some really
cool things.
I hope you're excited to hear about them.
So active learning.
As you've probably picked up at this point,
what I'm going to talk about, we're infusing the recitations
in 033 with active learning.
And so that means that really it is
my recitation instructors who are on the ground doing
a lot of this.
And I will admit, when we were starting out,
I sort of thought--
I had this amazing idea to do active learning recitations.
And I was like, well, I am done.
I have solved the problem.
I just told my instructors to implement it.
It was like, guys, active learning.
It didn't work.
I was perplexed, because I told my instructors correctly
that these types of activities would
help increase participation.
My instructors were the ones complaining about the lack
of participation.
And so I just thought come on, what's the problem.
Well it turns out you cannot tell your instructors to do
a thing that they've never done before and just have them
magically do it.
In particular, you can't tell your instructors
to fundamentally change the way they teach
and magically have that happen.
I would say it's difficult enough
for us to change the way we teach,
much less to get other people to change the way they teach.
So the first part of my talk is changing
the way other people teach in four easy steps.
So step one is get everyone on board.
So this is the first thing that I did, last year in 033.
I will keep saying last semester, but last year.
I presented active learning as our primary goal
for the entire semester.
So we did not wait until the course got settled down
to start doing it.
In fact, our very first staff meeting,
which happens before the semester has even begun,
was about active learning.
We had everyone on the staff there.
So me, all of my recitation instructors,
all of my TAs, all of the communication instructors.
This is about 30 some people here.
And we talked about active learning.
We talked about why it's a good idea, the evidence that it's
a good idea, evidence that it is better than lecturing.
We talked about why running the recitations in this way would
support the other learning objectives in 033,
in particular the objective of getting students to design
their own systems and evaluate their own tradeoffs.
I explained to my instructors that there would
be extensive support for this.
And that we'd be checking in throughout the semester
to make sure this was really working for them
and working for the students.
Now I did expect some pushback on this in this meeting.
Partly, I am asking a lot of my staff here.
And also, I am a lecturer in charge
of a lot of faculty, who have been here for a very long time.
So I was not quite sure how this would go.
I do not know if my staff knew about this.
I actually went so far in preparing for this staff
meeting to not only write out all of my objectives
and why active learning was a good idea,
but to write my own rebuttal to myself and then rebut that.
So I sort of had, OK, if I didn't want to do this,
I'd be asking-- well, I don't think
active learning is a good idea, I don't
think this is going to work.
And I had prepared answers for all of these questions.
I looked at the notes for that meeting to prepare for this.
And it really went on for quite a while.
I did not actually get that much pushback here,
because my staff is amazing.
But also, I talked to a few of my instructors ahead of time
about this, people who had been with the class
for a while, who I knew would be maybe a little bit more game
for this idea.
I kind of got them on board first.
I also appealed to everyone's scientific nature,
explaining that this was somewhat of a research
project, which was true.
I did want to know would this work in 033.
I was heavily biased towards the fact
that I thought it was going to work.
But we had methods to evaluate this throughout the semester.
And I told them if this goes horribly,
we won't keep doing it.
I'm not going to burden you with this.
And I enthusiastically mandated that this is what we
were trained for this semester.
In previous semesters, I had suggested
some of these ideas, encouraged my staff to try it.
And that really wasn't enough.
And so on the one hand, get everyone on board.
It was one staff meeting.
I will say, it was really tempting
for me to not have that staff meeting so soon.
033 is such a big class.
It has so many people involved.
It feels like we're just scrambling
to get the logistics worked out in the first week or two.
Even for us assigning our students to sections
is this whole thing.
But doing this really set the tone for the whole semester.
I was kind of surprised at how helpful this was.
Everybody knew the active learning was important.
Even the people who weren't the recitation instructors,
all the TAs, all the communications instructors,
they knew that it wasn't optional.
It was their primary goal of the semester.
Any other changes that we made in the class
were largely going to be in support of this goal.
So step one, get everybody on board with your mission.
Step two is do a lot of work, do a lot of planning.
So step one was our first staff meeting.
Step two, what's happening in our second staff meeting.
So one of the things that we did is
we talked about a pretty well-known list of something
like 278 different active learning activities.
I knew it was a lot to ask really
anyone to read through all 278 active learning activities.
And so I annotated this list.
And we kind of discussed the highlights
in the staff meeting.
I am going to go into detail in the second part of this class
about some of the specific activities we do in 033,
but I'll give you some examples from this list just
for concreteness.
So one of them, which I really like to do in lecture actually,
is--
they all have funny names--
OK, so fingers on chest.
So this is having students vote on a multiple choice question,
but using like fingers on their chest.
So option one, option two, option three.
And that way, the students can't see what
other people are voting really.
They're not quite like, oh, everybody else
is voting for one.
They can't really see, so they tend to be more truthful.
A lot of times in lecture, I ask them just
do they feel good about something
or bad about something.
Another one, a fairly common one-- think, pair, share.
So posing a question to the class,
asking students to think about it,
then to share their ideas with a partner.
And then maybe go and address the larger class.
And then a third one, which I like just
because it has a nice name--
the name for it is human Tableau.
And so this is having groups create
living scenes which relate to the classroom
concepts or discussion.
So that one's a little bit more out there.
But these are the types of activities
we're thinking about.
And of course, we went through this list.
We looked at a whole bunch of activities.
We're not going to end up using all of them in 033.
But going through those list, it was a really good exercise
for all of us.
It gave everyone a much better sense
of what is active learning or what can active learning be.
But even kind of armed with that list,
it's still a lot to tell your instructors,
here's your 278 activities, now go, now do active learning.
So in addition to this, I pre-planned
a lot of active learning techniques
into each recitation.
So normally, before active learning,
the way that I would plan recitations
was strictly just for technical content.
I would give my instructors a handful
of major technical issues they needed to hit on.
It helps keep our recitations somewhat uniform
so that we can be fair on exams.
We do test the technical material
on exams in this class.
Beyond that though, my instructors
have a lot of leeway to teach these topics however
they want, a lot of leeway on how they teach in general.
I had never before tried to steer them
in a particular direction.
But now, in addition to that technical content,
what I would do is I would put two or three active learning
activities or techniques into each recitation.
So things like pointing out places
where they could have students break into groups
and think about a particular question, pointing out
places where they could hold a debate in the class.
And again, I'm going to talk about these specific activities
we do in the second part of this talk.
But planning activities in this way
meant that my instructors has had
a lot of ideas of how they can make
this work in their sections.
Giving them multiple ideas and multiple types of ideas really
meant that they could pick the ones that they
were the most comfortable with.
So instead of saying, hey, here's this really, really
out there idea, everybody go try it, some of them
might try some of the more extreme ones,
but everybody could kind of find something
that they were comfortable with.
So get everybody on board, plan things,
And the next thing that I did was support my staff
as individuals.
So I didn't plan these activities
and set the staff free.
I actually observed them throughout the semester.
And I made sure to stress that it
wasn't any sort of evaluation.
I was not there to say, oh, boy, Karen,
this is really not working out.
I was just there to see what was working
and what wasn't, so that we could all
iterate on these things.
In practice, this was great.
I, of course, found way more good things
in doing this than anything problematic.
Most of my feedback to my instructors
was, hey, you did this really awesome thing,
you should do more of that, and can we
share it with everybody else.
And so, I really, in the end kind
of thought of myself more as a cheerleader for them
and what they were doing, than someone who was coming in
are really critiquing anything.
I also got a lot of great ideas from them,
both in how to present certain technical content and twists
on some of the active learning techniques
that I had suggested.
So related to that, I'm observing them as individuals,
but we would also talk a lot as a group
about how this was going.
Every staff meeting we'd take time
to talk about what went well the previous week.
Our instructors would share active learning ideas
that they tried, report on how they went.
This was really nice.
It got some of my more hesitant instructors kind
of on board with this, to say like, oh, well,
that person tried it and it seemed like it went really
well.
Maybe I'll try that next time.
And of course, the thing that turned out
to be so great about this, is that all of my instructors
have these amazing ideas on how to teach certain things.
But to them, it doesn't seem special,
like this is just always the way I've taught operating systems.
And so, now we have this space where we're getting
to share all of these ideas.
A lot of them relate to how our active learning has gone.
But sometimes it's just, hey, I taught this paper
this really cool way.
It has also-- the fact that we've now kind of fostered
this space for discussion at every staff meeting,
everybody is pretty willing to bring up
things that aren't going as well, to maybe come in and say,
hey, I tried this active learning thing
and it did not work at all.
What do you all think?
Is there something I could have done differently?
Is this just not something that will work for our class?
And so that has been really, really great,
having this space for my staff to kind of reflect on this
and not feel judged in any way.
So this is the first part of getting active learning to work
in 033, which was a lot of staff training.
But now the second part--
what do we actually do?
What are these activities?
I would like to give you some specific examples from 033,
but I will also try to give you some general feedback as well.
I realize you probably won't be able to take the 033 activities
exactly into your classes, so let me talk about some
of the common things that we do.
So these are things that we've seen work really well
in our classes.
So one of the most common types of active learning that we do
is one I have already referenced a lot, which
is to put students in small groups, have them do something,
and then bring them back for a class-wide discussion.
So let me give you some really specific examples.
We have a recitation where we talk about content distribution
networks.
The paper they read describes various goals
of those networks, so performance, reliability,
scalability, things like this.
And what we'll do in that recitation
is we'll divide the class up into groups.
And we'll have each group think about what
aspects of the system support a particular goal.
So you think about performance, you think about reliability,
you think about scalability, et cetera.
And then we'll bring the class back to a discussion
once they've thought about that for a while.
Another example, when we talk about naming in the Unix file
system, we start by asking them, what things have names?
What things in Unix have names?
And again, we'll put them in groups.
We'll try to get every group to come up with as many things as
possible.
Because when we bring them back together,
we'll try to get a different thing from every group.
So you can't all just say files, you
have to say like files, and directories, and processes.
And there's a lot of things in Unix that have names.
I'm not here to make you Unix experts.
And so, in both of those examples, like I said,
we'll bring the class back together for a discussion.
We'll try to have each group say something.
And as I said before, we've seen very clearly, talking
in these small groups before hand,
really makes the shyer students a lot more confident.
And asking each group to contribute to the discussion
means that we don't end up with something that is dominated
by one or two groups.
Often, we'll even be a little bit more relaxed about it.
We will just ask a big question, some sort of thought question,
not just a tell me this fact from the paper.
We'll break them up in the small groups,
have them think about it, and come back together
for a discussion.
It's nice, it breaks up the recitation.
It gets them kind of talking and doing things.
We occasionally will have our TAs teach in this class.
And we encourage them to use these techniques as well.
And one of them asked me a really good question
that I realized I had never thought about.
She said, what do I do when they're in groups?
Like, do I just stand there and stare at them?
Probably not, that's probably not good.
So to tell you what we do while they're in groups,
sometimes our instructors might actually check in
with each group, to say, oh, hey,
what are you guys thinking about?
Sometimes, you're kind of lightly monitoring,
you're walking around.
I know when I cover recitations, sometimes
I use it as time to prep for the thing that is coming next.
So if I need to draw something on the board that's
going to take a minute, I'll put you all in groups
so you can discuss while I draw that.
So this works really well as a technique
to get students engaged and talking,
and honestly, it's also nice from a classroom management
perspective.
It breaks things up, it gives you a little break from talking
and you can drink some water.
So that is kind of our starting point.
This is one thing where I would say we do something
like this in almost every single recitation, every Tuesday
and Thursday.
The second one is one that I really like,
which is to have the students debate.
It's a little bit harder to work into a recitation.
You need a question that can be debated.
For us, we have one recitation where the students
read two short papers.
And the papers actually come to opposing conclusions.
And so it's an interesting lesson for them,
well, how could this happen in science?
How could data say two different things?
It is mind-blowing for them, honestly, it's pretty wild.
And so we'll split them into two teams.
And we'll give them each a paper, have them debate.
The students usually really like this.
They love to argue, so they're very excited to do this.
It does involve a bit more monitoring.
You want to make sure that one team, or one person,
or one team doesn't dominate.
To combat that, we'll always have the teams converse
beforehand, kind of get your argument together
so you're ready to prepare it for the class.
I personally think something like that could also
work in small groups.
Maybe you have groups of four, teams of two, debating.
I will say, when I've taught this recitation, of these two
papers, there's like a clear winner in my mind.
The last time I taught this, the opposing team
made some very salient points.
So they really do well with this.
So this one is one that I think is really fun,
just a little bit more difficult to work in
because of what it requires.
A third one, something that we've actually
started doing more of this year, is having them draw pictures.
So we'll ask them, in groups, to draw pictures
on the board of whatever system we're talking about,
or maybe some component of that system.
And what we end up with is a wide variety
of drawings of varying levels of artistic skill.
And we can come together and talk about,
what is each drawing showing?
What are the common things that we're seeing
in all of these depictions?
What level of abstraction is each drawing capturing?
That's something that comes up a lot for us in 033.
I like this for a few reasons.
For 033 specifically, part of our communication curriculum
involves teaching the students how to design and draw figures.
So this, for us, this is just a great way for them to practice.
It also forces them to figure out exactly what the system is
doing.
And so we can expose some areas where
they might have missed something,
which always happens.
We expect them to read the papers before they come in.
They should be able to draw something.
It might not be perfect, and that gives us a great space
to explore that, like why was this confusing?
What is it about the paper that made this confusing?
It's also just fun.
So the students get a kick out of it.
Now the last activity that I want to talk about, this is,
I'm going to say that this is one
of our more extreme activities.
We actually do it pretty frequently.
We will have students act things out.
Also, the hardest to make an icon for,
I just didn't know what to do there.
They're acting, you can tell.
Now, I tell people about this a lot, and whenever I say it,
like, oh, we act things out in 033 all the time,
they are for sure intrigued.
They are not sure exactly what that would entail.
So I was going to have you all do this activity,
but we're half going to do this activity.
So I kind of want to go through one of our examples
actually in detail.
We have them act out a system called MapReduce.
But for this to make any sense, you
need to understand a little bit about what MapReduce does.
So you're going to become--
I don't want to say MapReduce experts,
MapReduce novices for the next five minutes.
All right, so, brief aside of technical material.
MapReduce is this system for performing
parallel computation.
And so here's the canonical example.
We have a large piece of text.
The Library of Congress has a lot of books in it.
These books, we're going to digitize this,
and so the data from these books is spread out
on all different machines.
So we have a lot of computers.
Let's say, you all can be computers.
So each one of you has a book from the Library of Congress.
And here's the problem that we want to solve,
the canonical MapReduce problem.
We want to count the number of occurrences
of each word in this entire corpus.
So I want to know, how many times does the word
apple appear?
How many times does the word ball appear?
How many times does the word the appear?
OK?
It's the canonical problem.
You all don't look like you actually want to know that.
You do, you want to know how many times the word
apple occurs in the Library of Congress.
So you can imagine doing that with one computer, where
that computer just reads every book in sequence
and keeps track of these counts.
And that would be very slow.
So instead, what we can do is we can have
multiple computers participate.
You all have a portion of the Library of Congress.
You all can read your data.
You can make accounts for your pieces of data.
This is the map phase.
And then we need some way to kind of combine those results.
So if Dipa finds that the word apple occurred five times
in her text, if Jesse found that it occurred six times,
I need some way to add those up.
That would be the reduce phase.
And so part of the insight of this system
is that there are a lot of common problems in computer
science that can be described this way.
This is a system that came from Google.
So Google is doing this on your data all the time.
They are mapping and reducing everything about you.
So let's see how that would actually work,
how this system would work.
Jessie, come play your role in this thing.
JESSIE: Do I have to wear the hat?
PROFESSOR: You do have to wear the hat, it's active learning.
I've got to write master on it.
JESSIE: I'm ready.
PROFESSOR: I think a lot of things in life
should be more fun.
So there's really no reason for Jessie to wear this hat.
But there's also not a reason for her not to wear this hat.
So--
JESSIE: There might be a reason.
AUDIENCE: This is about who this is fun for.
JESSIE: Fortunately, I can't see you, so now I feel comfortable.
PROFESSOR: Well, you can tilt the hat back a little bit
maybe.
So in MapReduce, there is the one machine
that acts as the coordination for this, the master machine,
as Jessie is so--
doing such a great job acting for us.
And so what the master's job would be--
you don't have-- you're just--
we just want to visualize you.
You're just a prop, yes.
So it would start by assigning jobs to workers.
Now you all, I didn't make hats for all of you, there wasn't--
I know, I'm very sorry, but you're all workers.
Only the master gets a hat.
All right, so you are all workers.
So one of the first things Master Jessie would do,
would be to assign map jobs to workers.
And she would tell you things like exactly what data
to operate on.
So imagine a scenario where you, a worker machine,
has data from a ton of different corpuses on it.
So Jessie actually has to say, OK,
worry about the Library of Congress part of your data.
And you would begin operating on that.
You'd count your words.
And you'd write out some results to your disk,
that you also have as a machine in this.
And then when you were done, you would report back to Jessie.
And you'd tell her some various meta information, kind of like,
these are the words that I found.
You don't have to send her all of your results,
because the Library of Congress is big, you have a lot of data.
But you might tell Jessie, hey, I found apple.
Now, once all of you are finished doing your map jobs,
Jessie has all of this kind of meta information,
she can start assigning the reduce jobs.
And the reduce jobs are a little bit different.
So Dipa if it was your job to reduce, say, the word apple,
you need to know a couple things.
You need to know, well, which one of these machines
actually found the word apple.
It's not necessarily they're going
to be in every piece of data.
And Jessie would be able to tell you that.
She'd basically say, hey, Dipa contact these workers,
they have the data you need, you can add them together,
in this example.
MapReduce functions can be much more complex than this.
I thought let's start slow.
All right, so the reduce workers would go, you'd reduce,
you'd report results back to Jessie,
and then we'd have our full counts.
I'm trying to decide how long I want you to stand here,
and the answer is for the rest of the talk,
but you can almost sit down.
All right, so this is the basic--
JESSIE: I thought I'd have more work to do.
PROFESSOR: I thought that would be a lot.
I didn't prep you for that.
Don't go yet, though.
JESSIE: I'm just moving over.
PROFESSOR: So this is the basic idea of MapReduce.
This is the type of thing we'd actually expect
the students to know coming in.
They've read the paper, and they should understand this sort
of level of coordination.
But there are a lot of details that we left out.
For instance, I didn't ask you all, hey, what
happens if a machine fails?
Worker Dipa crashes, what do we do?
AUDIENCE: [INAUDIBLE]
PROFESSOR: All the time, right?
Is that data that she was storing, is it
anywhere else in the system?
How do we know?
What if she had half finished the job?
What do we do?
Or even, what if there was no crash, but like let's say
the worker Janet is just running really slow, like she's going,
she's working hard, but everyone else is finished
and we're just waiting on Janet.
And these are those types of things.
Those are the interesting things about MapReduce.
That's the point where there are tradeoffs for the students
to think about.
So it's not just important to think about,
oh gosh, what does happen if a machine crashes or is slow,
but why does it happen that way.
So to do this, we actually have them act out this system.
And from the way that I've described it here,
you might think, well, I have one student be the master
and everybody else is a worker, that's
kind of what I was setting you all up for.
This is not actually how we do it.
Normally-- for one, it doesn't work with the size
of our recitations.
Even here, this is a lot of workers for one student
to be in charge of.
And also, if Jessie was maybe more of a shy student,
she's like this is the worst day of my entire life.
Katrina has made me wear a hat, and I have to stand--
I am in charge of everything, it's not good.
It's not good.
So in practice, here's how we do this activity.
You can sit down if you want.
You don't have to be here anymore.
JESSIE: Should I keep the hat on?
PROFESSOR: I think, definitely, yes.
I think absolutely.
So we do start by assigning students roles.
But we will usually have them in teams.
So there will be a master team.
Each worker will be a team of maybe two or three students.
This is good for a lot of reasons.
Again, for the reason of getting the shyer students
to be a little bit more comfortable,
no one student is in charge of something.
From a logistical standpoint, again, I
glazed over many a detail in MapReduce,
but you can imagine that the master has a lot to do,
a lot to keep track of.
And so we will often, we'll have our master team.
And we'll actually give each student in that team a task.
So one might run the clock, timing in the system
is important.
We'll see an example of that in a second.
One keeps track of what map tasks to complete.
One assigns the tasks, things like that.
We have found-- we've actually done this activity
a number of times.
It gets better every time.
One of the things we learned the most quickly
is that the master-- we had to give
the students who were doing the master extremely
detailed instructions.
If we just said do the master's job,
they were not ready for that.
In some sections, in fact, some of my instructors
prefer to do this where the instructor just is the master,
and so they can be kind of in charge of coordination.
Workers in this example get a little bit of an easier job.
Again, we're putting them in teams of two to three.
Sometimes we might give them a sheet with words on it,
and actually ask them to count the words.
Sometimes we'll give them a sheet that kind of just
says here's the words in your data and here are the counts.
That might seem like, oh, gosh what's the point,
they're not actually doing the worker job.
Counting the words in the text is not the interesting part
of this system.
So sometimes we'll just skip that.
It also, again, lets us do some very precise things
with timing.
So once these roles are assigned,
we're going to run the protocol.
So the person keeping track of the master clock
will advance it, one click, two clicks, et cetera.
Various map jobs will complete.
Most of the time, when we give them a paper that says,
here's your words and your counts, we'll also say,
hey, it takes three clock cycles for you to complete.
And so once the clock has ticked three times,
they'll report back to the master team.
Because of how we set up those map tasks, some of them
will also fail.
Some of them will have papers that say, oh, I
failed after click two.
And they have to announce it to the class.
And at that point, we pause.
What happens?
What happens when a machine fails?
What does MapReduce do?
Why does it do that?
The spoiler for you all in MapReduce, the master
will reassign that task to somewhere else.
And part of that challenge is making sure
that the new worker has access to the data that
was on the crashed machine.
MapReduce replicates data, so we talk about how that happens.
And the protocol continues.
The students will find that one map task, it hasn't crashed,
but it's taking a very long time.
And that's another good point for discussion.
What does MapReduce do?
Does it just wait?
In practice, it restarts the task in another machine.
And then, so, oh, there's a question, these two machines
are running the same thing.
What do we do?
Who do we listen to?
How do we get the data?
And this is why the timing of that activity is so important,
because a lot of the discussion points about MapReduce
come from what we do when things are slow or when things fail.
So being in control of that timing is crucial.
We do have some variations on this activity though.
One of our instructors this year had students count characters
instead of words, same idea.
But of the workers was counting Russian characters
instead of English.
So they just organically were much slower than the rest
of the class.
And that was kind of a nice way to show that.
Many of our instructors have the students
count types of candy instead of words.
And we bring them candy.
That's a big hit.
In fact, after we did this recitation this year,
one of our students asked why can't every recitation
be like the MapReduce one.
It is not clear how much the candy
factored into that question.
So it has taken us quite a few iterations
to get that activity working.
And again, I know you're not all teaching
MapReduce in your courses.
You're not going to take that activity to you.
But the things that were interesting for us,
having the students in teams for each of these roles
made a huge difference, both in terms
of making sure they were all doing the correct things,
but just getting them more comfortable with it.
I will tell you, the students go for this.
They are into it.
They map and reduce.
They do not think it's stupid, which is very, very nice.
And making sure that the master has these exceedingly detailed
instructions, figuring out how to make the timing work,
so that all of the things we wanted to discuss
would actually happen.
And so really the activity is the way I describe it--
Karen, this is how Karen does it.
Karen, you're the authoritative source on how we do MapReduce--
yeah.
I want to talk briefly about another system we act out
because it's fun.
It's one of my favorite things that we do.
It's much shorter.
We do this for a system called Raft.
And Raft, like MapReduce is this large distributed system
on a lot of machines.
Timing is also important in Raft.
In particular, machines set randomized timers.
I picked some random number of seconds, milliseconds,
whatever.
And when that timer expires, they do something.
So being able to act that out is actually really crucial.
And we thought, gosh, how are we going to randomize them?
And so one of my instructors came up
with the best idea, which is to have the students spin around
in a circle.
So like literally, they're standing here,
and then they kind of-- they do this, and it takes a while.
Because he figured out that students
just walk around naturally at different paces.
So they've effectively randomized themselves.
Someone will finish first.
So I need you to imagine feel like you're walking around MIT,
and you're just kind of like, oh,
what's going on in this class.
And you see seven or eight students up
at the front of the room just very quietly spinning around
in place, while all of the other students are like--
the person who finishes first wins.
And so who-- who is it?
Who's going to be the next leader of this election
cycle in Raft.
It's really-- it looks ridiculous,
but it works so well.
And it brings me a lot of joy.
So to kind of wrap up our activities
before I get to results, effectively, how
has this worked for us--
again, we do all of these things a lot.
This one in particular, small groups,
a class wide discussion--
although acting things out, we act a lot of things out now.
We've gotten pretty good at that.
And the key component for us has been recognizing
in all of these cases, the students
are more comfortable if they're not doing something alone,
if they have at least one other person on their team doing
whatever it is they need to do in that recitation.
And since we do something like this in every recitation,
the students just come to expect it.
I mean, they go for it.
Sometimes, I'll come in and cover a recitation.
And despite observations, I'm never
exactly sure how my instructors are leaving things.
And so, I always start by asking,
like, hey, I'm going to put you all in groups,
is that something you're used to.
Every time, they're like, yeah, we
do that every day, like of course we can get into groups.
So again, even acting things out, they really go for it.
We try to vary the types of activities
that we do each week, to keep everyone engaged,
to keep things interesting.
It's hard to act ever system out.
And there's occasions where an activity doesn't work.
The first time we tried MapReduce, it was chaos.
But we thought it was a cool thing.
So we just iterated.
We just tried it again the next semester.
And in planning all of these, we're
still very careful to make sure that our activities actually
teach them things.
Like I said, all of the recitations
have technical points that they have to hit on.
A MapReduce activity where they just
act out the system in its normal use case, it would be fun,
but they're supposed to come in knowing that.
So we really want to design these activities
so that we're hitting on the points we need to hit on.
So there is a lot of planning that goes into this.
So now, for the final part, did it work?
What got better?
I'll start with a few actually quantitative things.
Sometimes when I tell people about this endeavor,
they want to know, are grades better?
Have the students' grades improved?
How are they doing on exams?
I don't personally think that that's the right question.
There is one aspect of grading that has improved--
participation grades.
We assign a pretty large portion of the final grade
to participation in recitation.
Again, we're asking a lot of them.
We're expecting them to demonstrate certain skills
in recitation, so they are graded accordingly.
And we're seeing students earn higher participation grades
now.
I'm comfortable interpreting that as they
are participating more.
We are also giving them a lot more opportunities
to participate.
Related to that, one of the things
that we've gained from this experience
is just a better understanding of what it means
to participate in our class.
For a long time to us, it kind of
meant, well, does this student answer questions
when I ask them a question.
But now it means, are they coming prepared to class?
Are they engaged in group work?
Are they listening to other students
and to their instructor?
Are they participating in class wide discussions?
There is more opportunities for students to participate.
We are much clearer about how we assign this grade.
So not only have the grades improved,
but the students are noticeably more relaxed about this grade.
I get fewer complaints about this grade,
which is great for me.
And they know that if they come prepared and ready to do
the work, they'll be fine.
AUDIENCE: The grade for the recitation
is separate from the course grade?
PROFESSOR: It's part of the final grade in the course.
A percentage of the final grade is from participation
in recitation.
So we also surveyed the students during the semester.
And we were largely asking questions about
do you feel comfortable in the recitations.
Largely, yes, they did.
In our final survey, overwhelmingly the students
felt that these activities improved their engagement.
So these things are all good.
I don't think they really captured
though what has been so great about this endeavor
for our class.
And so in preparing this talk, I spoke
to a couple of instructors who have been
with me for a number of years.
So they've seen the class through its non-active learning
era, through the dark times, as well as the current era.
And so I stole a few quotes from them to kind of sum
up what I think is made this approach better
for the students.
So let's turn to number one.
The first is that I think the students actually understand
the algorithms better.
In particular, if we're acting something out,
it gives them a visceral experience of the finer points
and edge conditions.
It's really easy to slide over things
if one reads a paper quickly, and we're
asking them to read a lot.
Second instructor, I think that it's much better when
there are pairs discussing a solution
or when we're doing a game.
People are more likely to notice a flaw in something I've said
or in someone else's answer.
And they ask better questions.
Plus, I think it's reassuring either
to find out that a classmate is just as confused
or to find out that a classmate knows a ton about the subject.
And then the last one, which I think
really sums up how things have been for our students,
is this one.
I think the students are more engaged because they are asked
to be more active participants.
There's a greater sense of camaraderie between them
because of their interactions.
So we're seeing these students understanding
the details of the systems better, while developing
this sense of camaraderie.
And honestly, that has been huge for us.
Largely, I would say, from my perspective
as the person in charge, students
are buying into this class more than they used to.
I get fewer complaints across the board.
We're seeing them work better together
in their team projects.
We're seeing them be more engaged
in lecture and in tutorial, even though recitations was really
where this active learning push happened.
And this whole thing has been beneficial,
not just for the students, but also for the staff.
There was a little hesitance at first.
Active learning can be a hard sell.
It involves more work for everyone.
It takes a lot of planning.
It can be a nerve-wracking way to teach if you've never
taught it before.
Imagine coming into a room of students
you don't know very well, and say, OK, five of your
are going to be on the master team,
let's go, I made you all hats, right?
But now, one of my instructors said,
there's a modest amount of competition
among the recitation instructors to work out schemes and games.
So it feels like there's both a reward for applying myself,
but there's also less need for perfection.
It's helped that we're encouraging
active learning is an approach, because it makes
it feel like less of a risk for me
to try some new unconventional approach that occurs to me.
So I'll tell you one of the highlights for me,
as a person sort of overseeing this operation.
So this happened at the end of last semester.
Again, that's the first time we're really
putting this scheme into place.
And near the end, my instructors--
I got this email from the instructors
about one of the final recitations.
And the recitation was about a system
called DNS SEC, secure DNS.
So it deals with security.
DNS prevents certain types of attacks on a system called DNS.
That's all you need to know DNS SEC.
So the first instructor described an activity
they had done, where they had groups draw diagrams
of their attacks on the board in parallel
and then talked through each diagram,
with the students explaining the problems and the approaches.
And the instructor noted that they engaged,
they laughed, they collaborated, they learned more about
how design figures to tell the story they wanted to tell.
And I got this email, and I was so excited.
Because this was not an activity that I had planned for them.
This is just something they had done.
And I thought, it's happened, finally it's happened.
And then I get a second email.
And so a second instructor responded
saying that that sounded great.
And what they had done, they'd had a group of students
work out how DNS would work for a particular scenario, so
the original system.
And then two additional groups joined that first group,
worked through how different attacks would
affect that scenario.
And then a fourth group acted as DNS SEC,
and explain how DNS SEC would prevent such attacks.
So to brand new ideas, both really good,
both really different.
And this is so exciting.
And then we get to instructor number three.
And I've never been able to figure out
a way to summarize this email.
So I have to quote it for you directly.
"In my recitation, we had a marriage proposal
that went bad when a romantic rival launched an attack
to destroy the relationship.
A second group used a different attack
to induce peasants to send their taxes to a scammer.
All of these attacks were then prevented
by the use of DNS SEC.
After each play, we did a replay with deep structure analysis
that made the analogy between the play and DNS explicit.
Overall, a pretty fun recitation and I
was impressed with the creativity, enthusiasm,
and acting talent the students displayed."
So this was amazing.
This was like such-- this email thread
is just such a triumph for me.
I have it highlighted and saved in my email.
And this has continued on since last semester.
I get like one of these email threads a week
now, with my instructor telling me all of the cool things
that they have done.
So the last thing, which I hope has come through in this talk,
it was a lot of work to get this to happen in 033.
I cannot in good faith stand here tell you active learning
is a breeze, you can just add it in, five minutes of prep time
before the class.
It's a lot of work.
But this class is so much fun now.
It's fun for me to run.
It's fun for instructors to teach.
I don't know how many people would tell you
that their 400-person class is fun to run.
But I have a great time.
And the amount of enjoyment that we get out of teaching 033 this
way really comes through for the students.
And I think that's another component of what has helped
them engage more in this class.
So it has worked really well for us.
I hope that you get some practical things that you can
try out in your own classes.
And I'm happy to take questions or to delegate questions off
to Jesse and Karen.