WEBVTT

00:00:01.040 --> 00:00:03.460
The following content is
provided under a Creative

00:00:03.460 --> 00:00:04.870
Commons license.

00:00:04.870 --> 00:00:07.910
Your support will help MIT
OpenCourseWare continue to

00:00:07.910 --> 00:00:11.560
offer high quality educational
resources for free.

00:00:11.560 --> 00:00:14.460
To make a donation or view
additional materials from

00:00:14.460 --> 00:00:18.390
hundreds of MIT courses, visit
MIT OpenCourseWare at

00:00:18.390 --> 00:00:19.640
ocw.mit.edu.

00:00:31.970 --> 00:00:33.532
PROFESSOR: OK, good afternoon.

00:00:36.970 --> 00:00:39.540
So today we're going to
talk about learning.

00:00:39.540 --> 00:00:50.590
So what can you do now that you
couldn't do as an infant,

00:00:50.590 --> 00:00:52.490
very simply?

00:00:52.490 --> 00:00:53.300
AUDIENCE: [INAUDIBLE]

00:00:53.300 --> 00:00:54.530
PROFESSOR: Talk, that's
a good one.

00:00:54.530 --> 00:00:56.860
Language, that's pretty
good, right?

00:00:56.860 --> 00:00:59.050
Anything else that you
couldn't do, yeah?

00:00:59.050 --> 00:01:01.070
AUDIENCE: [INAUDIBLE]

00:01:01.070 --> 00:01:02.700
PROFESSOR: Yeah, we don't
actually know what the

00:01:02.700 --> 00:01:04.330
internal mental life of
an infant's like.

00:01:04.330 --> 00:01:06.830
But it doesn't seem like they
remember many very specific

00:01:06.830 --> 00:01:07.990
things about their lives.

00:01:07.990 --> 00:01:09.450
You do.

00:01:09.450 --> 00:01:12.160
But you know it's a lot more
than, you know a ton of stuff,

00:01:12.160 --> 00:01:15.860
a lot of facts, right, things
about the world, the values

00:01:15.860 --> 00:01:18.820
you hold, about your family,
your culture, the country or

00:01:18.820 --> 00:01:20.650
countries you've grown up in.

00:01:20.650 --> 00:01:24.650
You just know a ton compared
to an infant.

00:01:24.650 --> 00:01:27.010
And there's only two ways
that you know this.

00:01:27.010 --> 00:01:30.470
It's either in your genes when
you're born or you learned it

00:01:30.470 --> 00:01:30.960
through life.

00:01:30.960 --> 00:01:34.420
And so we're going to talk today
about how scientific

00:01:34.420 --> 00:01:36.720
psychology tries to
approach learning.

00:01:36.720 --> 00:01:38.820
And in a remarkable thing about
learning is we often

00:01:38.820 --> 00:01:40.980
think, well, we're learning
lessons or whatever.

00:01:40.980 --> 00:01:42.150
We're learning material.

00:01:42.150 --> 00:01:43.790
But the important part of
learning the way we think

00:01:43.790 --> 00:01:46.350
about it is that it allows us
to predict the future on the

00:01:46.350 --> 00:01:47.700
basis of the past, right?

00:01:47.700 --> 00:01:50.750
We want to learn lessons in life
to know what's desirable,

00:01:50.750 --> 00:01:53.790
what's dangerous, what's a
smart way to do things so

00:01:53.790 --> 00:01:54.880
we're more effective.

00:01:54.880 --> 00:01:57.080
We choose what we want to do
better in the future to learn

00:01:57.080 --> 00:01:59.390
from the past and predict
the future.

00:01:59.390 --> 00:02:01.630
It imbues our word
with meaning.

00:02:01.630 --> 00:02:04.830
For an infant, not much
means much, right?

00:02:04.830 --> 00:02:05.930
They just look around.

00:02:05.930 --> 00:02:07.160
Who knows what anything means.

00:02:07.160 --> 00:02:11.700
Words, gestures, facial
expressions, there's not much

00:02:11.700 --> 00:02:12.720
meaning of the world
out there.

00:02:12.720 --> 00:02:15.860
For you, the world is full of
meaning, history, social

00:02:15.860 --> 00:02:18.610
interactions, everything that
you're thinking about.

00:02:18.610 --> 00:02:21.950
So how do we acquire the
meanings of things in the

00:02:21.950 --> 00:02:23.590
world that matter to us?

00:02:23.590 --> 00:02:25.930
And it's also had an interesting
kind of a give and

00:02:25.930 --> 00:02:28.790
take with how we learn about
learning in scientific

00:02:28.790 --> 00:02:29.930
psychology and sort
of historically.

00:02:29.930 --> 00:02:32.290
I'll just touch a tiny
bit on that.

00:02:32.290 --> 00:02:35.200
And we'll talk about three
aspects of learning today:

00:02:35.200 --> 00:02:38.150
classical conditioning, operant
conditioning, and then

00:02:38.150 --> 00:02:41.000
various ways in which there are
limits to conditioning.

00:02:41.000 --> 00:02:43.420
Thank you, Todd.

00:02:43.420 --> 00:02:46.800
So let's start with sort of one
of the most famous figures

00:02:46.800 --> 00:02:49.780
in all of conditioning or
learning, Ivan Pavlov, who

00:02:49.780 --> 00:02:52.160
actually won a Nobel Prize,
not for his studies of

00:02:52.160 --> 00:02:56.470
learning, but for his sort of
groundbreaking studies for the

00:02:56.470 --> 00:02:58.000
reflexes of digestion.

00:02:58.000 --> 00:03:00.260
He worked out a lot of the
fundamentals of how, in

00:03:00.260 --> 00:03:04.000
mammals, your stomach breaks
down food and that how food in

00:03:04.000 --> 00:03:06.500
the mouth provokes specific
salivation that begins the

00:03:06.500 --> 00:03:10.640
process by which food is
decomposed and then digested

00:03:10.640 --> 00:03:12.870
in your stomach, the
fundamentals of really how we

00:03:12.870 --> 00:03:15.310
live, right, eat to live.

00:03:15.310 --> 00:03:17.985
And he was very interested in
salivation reflexes, what is

00:03:17.985 --> 00:03:21.090
it that drives that first bit of
breaking down food in your

00:03:21.090 --> 00:03:22.590
mouth with saliva?

00:03:22.590 --> 00:03:27.690
And Pavlov was a pretty intense
researcher, as you

00:03:27.690 --> 00:03:29.360
might be used to at MIT.

00:03:29.360 --> 00:03:31.930
There's a famous story where a
graduate student came somewhat

00:03:31.930 --> 00:03:33.850
late to the lab and said,
but Professor, there's a

00:03:33.850 --> 00:03:36.350
revolution going on with
shooting in the streets.

00:03:36.350 --> 00:03:37.330
This is in Russia.

00:03:37.330 --> 00:03:39.080
And he said, what difference
does it make when you've got

00:03:39.080 --> 00:03:40.160
work to do in the laboratory?

00:03:40.160 --> 00:03:44.560
Next time there's a revolution,
get up earlier; so

00:03:44.560 --> 00:03:49.840
anyway, not an easygoing
supervisor.

00:03:49.840 --> 00:03:52.480
He would do experiments a little
rough in dogs, like cut

00:03:52.480 --> 00:03:55.790
the esophagus, the path by which
food goes from the mouth

00:03:55.790 --> 00:03:59.300
to the stomach, to understand
the role of the esophagus.

00:03:59.300 --> 00:04:01.570
And he found that, even in those
animals, when he placed

00:04:01.570 --> 00:04:04.370
food in the dog's mouth, he
expected, as a fundamental

00:04:04.370 --> 00:04:06.940
biologist studying how digestion
works, that nothing

00:04:06.940 --> 00:04:07.670
would happen.

00:04:07.670 --> 00:04:10.580
But he noticed not only would
the dog salivate, even though

00:04:10.580 --> 00:04:13.090
the food will never make it to
the stomach, but furthermore

00:04:13.090 --> 00:04:15.680
the stomach juices, the gastric
juices that break down

00:04:15.680 --> 00:04:18.810
food, also were released within
the stomach even though

00:04:18.810 --> 00:04:20.610
the food would never
get there.

00:04:20.610 --> 00:04:22.720
And then he got interested in
how it is that these things

00:04:22.720 --> 00:04:25.340
are driven if they're not driven
simply as a direct

00:04:25.340 --> 00:04:28.060
response to the arrival
of the food.

00:04:28.060 --> 00:04:31.890
How is it that the mere sight
of the food or the sight of

00:04:31.890 --> 00:04:35.950
the person bringing the food
would drive the salivation and

00:04:35.950 --> 00:04:38.060
what he called psychic
secretions?

00:04:38.060 --> 00:04:38.900
Because he was a biologist.

00:04:38.900 --> 00:04:41.560
So he expected, basically, you
put the food in the mouth.

00:04:41.560 --> 00:04:44.390
There's some chemistry between
the food and salivation.

00:04:44.390 --> 00:04:47.010
And here he's finding, just
seeing food, just seeing the

00:04:47.010 --> 00:04:49.080
person who would bring the
food, was driving these

00:04:49.080 --> 00:04:52.460
fundamental biological
functions.

00:04:52.460 --> 00:04:54.300
So there's a kind of experiment
that he would set

00:04:54.300 --> 00:04:55.360
up where the dog
would be there.

00:04:55.360 --> 00:04:58.030
And he would be measuring the
amount of salivation in some

00:04:58.030 --> 00:04:59.790
way or another.

00:04:59.790 --> 00:05:02.550
And in these kinds of
experiments-- and I'll go

00:05:02.550 --> 00:05:05.480
through this language
a couple times--

00:05:05.480 --> 00:05:07.165
in conditioning they've come up
with a sort of vocabulary

00:05:07.165 --> 00:05:09.350
to describe the pieces of things
that go into this kind

00:05:09.350 --> 00:05:10.210
of learning.

00:05:10.210 --> 00:05:12.690
So there's an unconditioned
stimulus, which is food.

00:05:12.690 --> 00:05:13.790
I mean you start with that.

00:05:13.790 --> 00:05:15.080
You want to eat food.

00:05:15.080 --> 00:05:16.820
And there's an unconditioned
response, which is you

00:05:16.820 --> 00:05:19.510
salivate when you see the food
or if it's in your mouth.

00:05:19.510 --> 00:05:21.330
That's the beginning.

00:05:21.330 --> 00:05:22.930
Then you're going to add
what's initially

00:05:22.930 --> 00:05:24.270
a meaningless stimulus.

00:05:24.270 --> 00:05:25.830
In this case, it could
be anything.

00:05:25.830 --> 00:05:27.100
But in this case, we'll
say it's a bell.

00:05:27.100 --> 00:05:29.920
You ring a bell before
the food comes.

00:05:29.920 --> 00:05:31.800
And you're going to get to
a conditioned response.

00:05:31.800 --> 00:05:34.020
The animal will say, well, the
bell predicts the food.

00:05:34.020 --> 00:05:35.410
And I'll salivate to
the bell itself.

00:05:35.410 --> 00:05:37.460
Even though I'll never eat the
bell, I learned that it

00:05:37.460 --> 00:05:38.930
predicts the food is coming.

00:05:38.930 --> 00:05:40.940
And so you've created
a new association.

00:05:40.940 --> 00:05:43.980
You learned something new about
the world, that a bell

00:05:43.980 --> 00:05:47.120
signals something that
drives salivation.

00:05:47.120 --> 00:05:49.640
So the two things are sort of
occurring next to each other

00:05:49.640 --> 00:05:51.630
in time or what Aristotle talked
about as the law of

00:05:51.630 --> 00:05:52.340
contiguity.

00:05:52.340 --> 00:05:56.320
The animal learns that the bell
predicts the food coming.

00:05:56.320 --> 00:05:57.860
And so here's the basic idea.

00:05:57.860 --> 00:06:00.750
At first, the tone does not
drive salivation, right?

00:06:00.750 --> 00:06:03.780
There's no reason to salivate
when you hear tone by itself.

00:06:03.780 --> 00:06:05.660
But as the animal discovers
that the tone predicts the

00:06:05.660 --> 00:06:07.630
food, the tone predicts the
food, the tone predicts the

00:06:07.630 --> 00:06:11.290
food, then all of a sudden the
animal starts to learn that

00:06:11.290 --> 00:06:15.020
simply the tone alone will
drive the salivation.

00:06:15.020 --> 00:06:18.040
Now this stimulus
is conditioned.

00:06:18.040 --> 00:06:20.160
And you have a conditioned
response that wasn't there to

00:06:20.160 --> 00:06:21.060
start with.

00:06:21.060 --> 00:06:21.870
It's very simple, right?

00:06:21.870 --> 00:06:23.590
You learn the two
are associated.

00:06:23.590 --> 00:06:27.440
And now you begin to have a
biological response to the

00:06:27.440 --> 00:06:29.890
tone alone.

00:06:29.890 --> 00:06:32.760
And typically what happens is
here's a learning curve.

00:06:32.760 --> 00:06:35.090
Here's how often perhaps
you salivate.

00:06:35.090 --> 00:06:37.810
And across trials you learn
more and more of that.

00:06:37.810 --> 00:06:42.310
And you finally plateau at some
level of performance.

00:06:42.310 --> 00:06:45.540
So a neat thing about Pavlov is,
although there weren't a

00:06:45.540 --> 00:06:47.700
lot of movies at his time, there
is a little bit of film

00:06:47.700 --> 00:06:49.500
of him and his actual dogs.

00:06:49.500 --> 00:06:52.195
So here's Pavlov's
actual dogs.

00:07:04.240 --> 00:07:06.170
So we said conditioning is
going to do two magical

00:07:06.170 --> 00:07:06.850
things, right?

00:07:06.850 --> 00:07:08.380
It's going to let us predict
the future on the

00:07:08.380 --> 00:07:09.260
basis of the past.

00:07:09.260 --> 00:07:11.670
And it is a very simple
experiment.

00:07:11.670 --> 00:07:15.910
Now the bell predicts the food
coming up and the salivation.

00:07:15.910 --> 00:07:17.700
And it imbues the world
with meaning.

00:07:17.700 --> 00:07:20.780
In a sense, the bell now means,
here's dinner, right?

00:07:20.780 --> 00:07:22.030
The bell is not just a bell.

00:07:22.030 --> 00:07:23.425
It's something with meaning
and significance.

00:07:26.856 --> 00:07:30.650
These are another example of
classical conditioning.

00:07:30.650 --> 00:07:33.350
And it's got some sophisticated
properties.

00:07:33.350 --> 00:07:35.990
You could go, well, balloons
popping water in your face,

00:07:35.990 --> 00:07:38.200
food for dinner, that's
pretty basic stuff.

00:07:38.200 --> 00:07:40.610
But here are some sophisticated
properties of

00:07:40.610 --> 00:07:42.490
these as learning mechanisms.

00:07:42.490 --> 00:07:46.600
So one was already talked about,
extinction, that, if

00:07:46.600 --> 00:07:50.940
the bell continues to ring and
no food arrives, the learning

00:07:50.940 --> 00:07:53.000
gets weakened, weakened,
and disappears.

00:07:53.000 --> 00:07:54.590
So you can learn something.

00:07:54.590 --> 00:07:57.380
And you can unlearn something
because it's no longer an

00:07:57.380 --> 00:07:59.900
effective predictor
of the future.

00:07:59.900 --> 00:08:01.990
Generalization is kind
of cued also.

00:08:01.990 --> 00:08:05.680
So imagine the frequency of the
stimulus is this, just the

00:08:05.680 --> 00:08:07.790
specific frequency
of the tone.

00:08:07.790 --> 00:08:13.250
Well, if the tone is a little
bit different, you might still

00:08:13.250 --> 00:08:14.810
think food is coming, right?

00:08:14.810 --> 00:08:18.810
So it's not that you learn
the precise thing.

00:08:18.810 --> 00:08:20.900
You'll still respond if it's
a pretty similar tone.

00:08:20.900 --> 00:08:23.620
And as the tone gets more and
more different, the response

00:08:23.620 --> 00:08:25.090
gets weaker and weaker, right?

00:08:25.090 --> 00:08:27.020
So that's smart generalization.

00:08:27.020 --> 00:08:28.490
I stay pretty close
to the tone.

00:08:28.490 --> 00:08:29.470
I go for it.

00:08:29.470 --> 00:08:31.970
I get further away,
eh, not so sure.

00:08:31.970 --> 00:08:33.370
It's not all or none.

00:08:33.370 --> 00:08:36.610
It's reasonably graded and
sensitive to the specifics of

00:08:36.610 --> 00:08:38.200
the situation.

00:08:38.200 --> 00:08:42.559
There's also one fascinating
thing, which is your acquired

00:08:42.559 --> 00:08:45.150
conditioned response is
extinguished because you're

00:08:45.150 --> 00:08:47.950
only hearing the bell and
there's no more food coming.

00:08:47.950 --> 00:08:53.290
But when you come back in, you
often have a kind of preserved

00:08:53.290 --> 00:08:54.900
recovery function of that CS.

00:08:54.900 --> 00:08:57.050
It's like it's lurking
in you somewhere.

00:08:57.050 --> 00:09:00.480
It's like I have a mental note
that this bell might mean

00:09:00.480 --> 00:09:02.250
food, even though it
was extinguished.

00:09:02.250 --> 00:09:04.590
It's still pretty interesting
stimulus.

00:09:04.590 --> 00:09:07.300
So you'd have some easily
spontaneous recovery of these

00:09:07.300 --> 00:09:11.290
kinds of learned conditioning.

00:09:11.290 --> 00:09:12.990
It can also be sophisticated
in other ways.

00:09:12.990 --> 00:09:16.270
And let me give you a
couple of examples.

00:09:16.270 --> 00:09:17.970
You can learn to discriminate
between two things.

00:09:17.970 --> 00:09:20.670
We've only talked about one
thing so far, bell, food.

00:09:20.670 --> 00:09:23.920
But what if you have a black
patch, for example, that

00:09:23.920 --> 00:09:26.570
predicts food coming, and a gray
one that predicts it's

00:09:26.570 --> 00:09:28.990
not coming, you can distinguish
between those two.

00:09:28.990 --> 00:09:30.720
You can get second order
conditioning.

00:09:30.720 --> 00:09:33.080
And let me show you that.

00:09:33.080 --> 00:09:35.940
The dog learns that the
bell predicts food.

00:09:35.940 --> 00:09:38.180
Now, there's no food presented
for a while.

00:09:38.180 --> 00:09:41.330
But there's a black patch
presented that you see

00:09:41.330 --> 00:09:42.720
presented with the tone.

00:09:42.720 --> 00:09:44.740
Because you know the tone
predicts food, the animal

00:09:44.740 --> 00:09:46.740
figures, well, maybe this
is a pretty good hint

00:09:46.740 --> 00:09:47.840
that food is coming.

00:09:47.840 --> 00:09:50.740
And after a while, it begins to
salivate to this one, even

00:09:50.740 --> 00:09:54.620
though it was never directly
paired with the food itself.

00:09:54.620 --> 00:09:56.550
So it's learning.

00:09:56.550 --> 00:09:59.340
It's a transitive inference
that it's making.

00:09:59.340 --> 00:10:02.490
If the bell predicts it and the
square goes with the bell,

00:10:02.490 --> 00:10:04.570
at some point I'll start
salivating for the square

00:10:04.570 --> 00:10:06.120
because that sounds like a
pretty good candidate for

00:10:06.120 --> 00:10:06.970
predicting the future.

00:10:06.970 --> 00:10:08.590
So that's pretty sophisticated,
too.

00:10:08.590 --> 00:10:10.990
You don't even have
food in this part.

00:10:10.990 --> 00:10:13.730
But it's even more sophisticated
than that.

00:10:13.730 --> 00:10:16.290
So look at these two things.

00:10:16.290 --> 00:10:17.850
And you can think about, I'm
going to let you think about

00:10:17.850 --> 00:10:18.820
this for a moment.

00:10:18.820 --> 00:10:20.110
Here are two groups
of animals.

00:10:20.110 --> 00:10:20.810
Or it could be people.

00:10:20.810 --> 00:10:22.950
It would work just
the same way.

00:10:22.950 --> 00:10:25.610
So here's the conditioned
stimulus and the unconditioned

00:10:25.610 --> 00:10:28.360
stimulus, for example, a bell
and the food, the bell and the

00:10:28.360 --> 00:10:29.920
food, the bell and the food.

00:10:29.920 --> 00:10:32.600
One group only gets
bell, food, bell,

00:10:32.600 --> 00:10:34.940
food kinds of pairings.

00:10:34.940 --> 00:10:36.520
The other group gets
the same pairings.

00:10:36.520 --> 00:10:41.750
But in between, they'll have a
random food reward, two random

00:10:41.750 --> 00:10:45.130
bells, a random food,
random bell.

00:10:45.130 --> 00:10:46.830
Now, the pairings
are identical.

00:10:46.830 --> 00:10:48.560
That's what's shown in orange.

00:10:48.560 --> 00:10:52.410
This group will learn better
and faster, more strongly,

00:10:52.410 --> 00:10:53.290
than this group.

00:10:53.290 --> 00:10:55.380
Even though they've had the
identical number of pairings.

00:10:55.380 --> 00:10:59.210
Why will this group
learn more slowly?

00:10:59.210 --> 00:11:01.560
Why will, even though they have
an identical number of

00:11:01.560 --> 00:11:03.470
times that they got the
tone and the food, the

00:11:03.470 --> 00:11:04.090
tone and the food--

00:11:04.090 --> 00:11:04.385
Yeah?

00:11:04.385 --> 00:11:07.190
AUDIENCE: Is there a little bit
of extinction between--

00:11:07.190 --> 00:11:09.090
PROFESSOR: It's an idea that
is an extinction between--

00:11:09.090 --> 00:11:09.970
I think that's a good idea.

00:11:09.970 --> 00:11:11.940
So that's a very good answer,
the extinction between--

00:11:11.940 --> 00:11:15.650
because, when you get this
CS alone, you're

00:11:15.650 --> 00:11:17.770
fighting the learning.

00:11:17.770 --> 00:11:20.170
But it's telling you the
system is smart, right?

00:11:20.170 --> 00:11:23.420
Because what it's kind of doing
is saying, look, eh, CS

00:11:23.420 --> 00:11:26.550
alone, not always a reliable
predictor.

00:11:26.550 --> 00:11:27.560
So I'll learn it.

00:11:27.560 --> 00:11:30.010
But I'm not going to believe
it so strongly, right?

00:11:30.010 --> 00:11:32.780
It's basically because you're
sticking extinction in between

00:11:32.780 --> 00:11:33.410
those, right?

00:11:33.410 --> 00:11:35.460
You're also having some trials
with the food that aren't

00:11:35.460 --> 00:11:37.000
predicted either, right?

00:11:37.000 --> 00:11:40.300
So it's like the animal or you
are learning what's a really

00:11:40.300 --> 00:11:41.660
excellent signal.

00:11:41.660 --> 00:11:43.750
And if there's other stuff in
there that gives the signal

00:11:43.750 --> 00:11:47.180
not the same predictive power
for the future you learn more

00:11:47.180 --> 00:11:49.360
slowly unless certainly
that relationship.

00:11:49.360 --> 00:11:49.650
OK?

00:11:49.650 --> 00:11:51.385
So that's pretty smart, too.

00:11:51.385 --> 00:11:55.440
Here's another way in
which it's smart.

00:11:55.440 --> 00:11:56.745
And I'll tell you
the phenomenon.

00:11:56.745 --> 00:12:00.360
And you think about why you
think learning might work this

00:12:00.360 --> 00:12:01.940
way in animals and people.

00:12:01.940 --> 00:12:05.110
So pretend there's a tone that
predicts food just like we've

00:12:05.110 --> 00:12:07.280
been talking about all
the way through.

00:12:07.280 --> 00:12:08.600
Food stops.

00:12:08.600 --> 00:12:11.850
And now you get a tone and a
light, a tone and a light, a

00:12:11.850 --> 00:12:12.250
tone and a light.

00:12:12.250 --> 00:12:13.840
The tone predicts it's food.

00:12:13.840 --> 00:12:14.800
You're getting this
part, again,

00:12:14.800 --> 00:12:16.850
without food being presented.

00:12:16.850 --> 00:12:18.870
And now you get the
light alone.

00:12:18.870 --> 00:12:20.340
OK?

00:12:20.340 --> 00:12:24.440
Or everything's the same.

00:12:24.440 --> 00:12:26.620
But there isn't the initial
learning with the tone.

00:12:29.230 --> 00:12:31.250
And then in the end you're
tested for the lights.

00:12:31.250 --> 00:12:33.890
And what happens is there's less
conditioning to the light

00:12:33.890 --> 00:12:37.280
here than here, even though
you've seen a light an exact

00:12:37.280 --> 00:12:38.570
even number of times.

00:12:38.570 --> 00:12:44.170
Why is the conditioning to
the light weaker here?

00:12:44.170 --> 00:12:45.920
Why is it weaker
here than here?

00:12:49.960 --> 00:12:54.260
This is an information problem
for the brain to solve about

00:12:54.260 --> 00:12:57.480
the truth of the world.

00:12:57.480 --> 00:12:59.740
Well, our interpretation
is this.

00:12:59.740 --> 00:13:02.250
Tone, food, tone, food, you
go, good, I got it.

00:13:02.250 --> 00:13:03.530
Easy, right?

00:13:03.530 --> 00:13:06.290
Now you get tone and light
together for a while.

00:13:06.290 --> 00:13:09.930
And you go, OK, I know the
tone is really important.

00:13:09.930 --> 00:13:11.100
The light, I don't know.

00:13:11.100 --> 00:13:12.580
The tone was perfectly good.

00:13:12.580 --> 00:13:16.760
The light seems redundant,
informationally redundant.

00:13:16.760 --> 00:13:19.500
So now when it's a light alone,
you don't have so much

00:13:19.500 --> 00:13:19.920
conditioning.

00:13:19.920 --> 00:13:22.150
Because you didn't care about
the light so much.

00:13:22.150 --> 00:13:23.580
The food had all the information
you needed.

00:13:23.580 --> 00:13:25.250
The light was informationally
redundant, right?

00:13:25.250 --> 00:13:26.880
It was superfluous.

00:13:26.880 --> 00:13:30.590
Here you only got trained with
the tone and the lights.

00:13:30.590 --> 00:13:33.490
So you never made a bet on
what's the critical thing.

00:13:33.490 --> 00:13:35.210
They both predicted evenly.

00:13:35.210 --> 00:13:37.650
So when the light comes by
itself, that was always

00:13:37.650 --> 00:13:40.430
something that seemed like a
pretty good signal equally

00:13:40.430 --> 00:13:41.760
good to the tone.

00:13:41.760 --> 00:13:44.935
So that's very sophisticated
that the animal learning it or

00:13:44.935 --> 00:13:46.930
the human learning it is
deciding what's the

00:13:46.930 --> 00:13:49.880
information that's diagnostic
and what's the information

00:13:49.880 --> 00:13:50.520
that's redundant.

00:13:50.520 --> 00:13:52.560
And it's conditioning
across that, right?

00:13:52.560 --> 00:13:54.700
So it's not just very
simple reflexes.

00:13:54.700 --> 00:13:55.950
It's sophisticated learning.

00:13:59.260 --> 00:14:00.720
So now I'm going to
switch gears.

00:14:00.720 --> 00:14:02.640
So we're going to have a
demonstration that involves

00:14:02.640 --> 00:14:07.660
the opposite of fearful things
like spraying water and

00:14:07.660 --> 00:14:09.610
popping balloons.

00:14:09.610 --> 00:14:11.390
But I can't tell you too
much about this.

00:14:11.390 --> 00:14:14.620
But I can tell you it'll
be pretty pleasant.

00:14:14.620 --> 00:14:16.010
But I need somebody who's
willing to step out of the

00:14:16.010 --> 00:14:17.740
room for a moment to do this.

00:14:17.740 --> 00:14:20.545
OK, thanks.

00:14:20.545 --> 00:14:22.630
Yeah, yeah, you can safely
take off your jacket now.

00:14:28.090 --> 00:14:30.050
And what's your name?

00:14:30.050 --> 00:14:30.730
AUDIENCE: I'm Sam.

00:14:30.730 --> 00:14:32.270
PROFESSOR: Sam, OK.

00:14:32.270 --> 00:14:34.300
Try not to listen
if possible, OK?

00:14:34.300 --> 00:14:35.550
AUDIENCE: Yeah.

00:14:37.596 --> 00:14:40.780
PROFESSOR: So you guys are
going to help me perform

00:14:40.780 --> 00:14:42.580
instrumental conditioning.

00:14:42.580 --> 00:14:44.430
And then we'll discuss
what that is.

00:14:44.430 --> 00:14:48.470
Our job is to have Sam come over
here and pick this up.

00:14:48.470 --> 00:14:50.230
OK?

00:14:50.230 --> 00:14:54.490
Your job, as Sam comes in for
every action he takes, a step

00:14:54.490 --> 00:14:57.100
this way, you played this game
as a kid, or an arm going in

00:14:57.100 --> 00:15:00.230
the right direction, applaud if
he's doing something that

00:15:00.230 --> 00:15:03.460
gets him closer to the goal of
picking up the brain and

00:15:03.460 --> 00:15:06.225
withhold your applause if he's
doing something that's taking

00:15:06.225 --> 00:15:07.170
him away from that.

00:15:07.170 --> 00:15:07.790
Is that OK?

00:15:07.790 --> 00:15:09.040
All right, OK.

00:15:11.530 --> 00:15:15.610
So we're going to discuss what
instrumental conditioning is.

00:15:15.610 --> 00:15:17.740
And then we're going to slightly
think philosophically

00:15:17.740 --> 00:15:20.610
about whether this is how you
learn to do everything.

00:15:20.610 --> 00:15:24.710
You get to choose who the
applause comes from, your

00:15:24.710 --> 00:15:29.530
parents, your family, your
friends, medical schools and

00:15:29.530 --> 00:15:31.790
law schools and graduate
schools.

00:15:31.790 --> 00:15:34.570
You get to choose who the
applause comes from, about how

00:15:34.570 --> 00:15:37.600
much of our lives is responding
to the applause

00:15:37.600 --> 00:15:38.630
that we care about.

00:15:38.630 --> 00:15:41.170
So we'll start more simply,
less philosophically.

00:15:41.170 --> 00:15:41.880
Let's think about this.

00:15:41.880 --> 00:15:44.120
The examples we talked so far in
classical conditioning all

00:15:44.120 --> 00:15:47.900
build on what feels like a basic
reflex, salivating for

00:15:47.900 --> 00:15:50.550
food, flinching for
water, right?

00:15:50.550 --> 00:15:55.310
So if we ask somebody, why do
you work hard, there's not to

00:15:55.310 --> 00:15:57.780
be very good explanation in
terms of those initial

00:15:57.780 --> 00:15:59.240
reflexes, right?

00:15:59.240 --> 00:16:00.010
You're not flinching.

00:16:00.010 --> 00:16:02.820
You're not looking for
food or water.

00:16:02.820 --> 00:16:05.680
And yet much of our life is
involved in endeavors that we

00:16:05.680 --> 00:16:08.850
learn where there's no primary
reflex that we

00:16:08.850 --> 00:16:10.400
can see in the story.

00:16:10.400 --> 00:16:13.320
And so operant or instrumental
conditioning, those two words

00:16:13.320 --> 00:16:16.320
are interchangeable, is the way
that psychologists have

00:16:16.320 --> 00:16:19.250
tried to understand those
forms of learning.

00:16:19.250 --> 00:16:20.990
And it began something
like this.

00:16:20.990 --> 00:16:24.060
And this is from Thorndike.

00:16:24.060 --> 00:16:25.810
He put a cat into
a puzzle box.

00:16:25.810 --> 00:16:28.250
I'll show you what the box looks
like or a sketch of it.

00:16:28.250 --> 00:16:30.890
The cat had to unlatch a door
by pulling a latch.

00:16:30.890 --> 00:16:33.370
So here's, the cat's
in a box like this.

00:16:33.370 --> 00:16:37.250
To get out, which cat wants to
get out, he or she has to push

00:16:37.250 --> 00:16:38.590
this, which will
push the latch.

00:16:38.590 --> 00:16:39.950
Now, that's not obvious
to a cat.

00:16:39.950 --> 00:16:41.570
It wouldn't be necessarily
obvious to a person but

00:16:41.570 --> 00:16:44.670
definitely not obvious
to a cat.

00:16:44.670 --> 00:16:48.650
And the animal learned it by
trial and error sort of like

00:16:48.650 --> 00:16:49.830
Sam did, right?

00:16:49.830 --> 00:16:53.510
And where is the unconditioned
stimulus?

00:16:53.510 --> 00:16:56.230
There's no salivation or
any reward like that.

00:16:56.230 --> 00:17:00.090
So Thorndike proposed that
people and animals learn

00:17:00.090 --> 00:17:02.550
things by the consequence
of their response.

00:17:02.550 --> 00:17:04.970
And the Law of Effect says the
consequence of a response

00:17:04.970 --> 00:17:07.540
determines whether it is
strengthened or weakened.

00:17:07.540 --> 00:17:08.339
You can be rewarded.

00:17:08.339 --> 00:17:09.520
That will strengthen
a response.

00:17:09.520 --> 00:17:10.349
You can have no reward.

00:17:10.349 --> 00:17:11.010
That will weaken it.

00:17:11.010 --> 00:17:11.930
You can be punished.

00:17:11.930 --> 00:17:14.790
That will greatly weaken
the response.

00:17:14.790 --> 00:17:15.859
So here's the cat.

00:17:15.859 --> 00:17:17.500
Here's an example of a cat.

00:17:17.500 --> 00:17:20.260
At first, it takes a long time
for the cat to get out of

00:17:20.260 --> 00:17:22.520
there as it tries different
things.

00:17:22.520 --> 00:17:25.240
But over repeated trials,
the cat can get out

00:17:25.240 --> 00:17:26.440
really, really fast.

00:17:26.440 --> 00:17:30.080
And how Thorndike, by
observation and thinking,

00:17:30.080 --> 00:17:32.330
thought about it was at the
beginning the animal

00:17:32.330 --> 00:17:33.100
wants to get out.

00:17:33.100 --> 00:17:34.670
It has no idea how
it gets out.

00:17:34.670 --> 00:17:36.470
So it does everything it can.

00:17:36.470 --> 00:17:37.700
It scratches at the bars.

00:17:37.700 --> 00:17:38.750
It pushes at the ceiling.

00:17:38.750 --> 00:17:39.520
It digs through the floor.

00:17:39.520 --> 00:17:40.360
It howls.

00:17:40.360 --> 00:17:42.300
And at some moment, it
presses a lever.

00:17:42.300 --> 00:17:44.300
And that's good, right?

00:17:44.300 --> 00:17:46.950
So here's different things the
animal starts with thinking

00:17:46.950 --> 00:17:48.110
might be good ideas.

00:17:48.110 --> 00:17:51.030
But because it discovers with
repeated efforts, repeated

00:17:51.030 --> 00:17:52.250
trials, that pressing the lever

00:17:52.250 --> 00:17:54.390
works, these get weakened.

00:17:54.390 --> 00:17:55.480
This gets strengthened.

00:17:55.480 --> 00:17:58.010
So the consequence of a response
determines whether

00:17:58.010 --> 00:17:59.900
it's strengthened or weakened.

00:17:59.900 --> 00:18:03.450
And now there's no salivating
or reflex or

00:18:03.450 --> 00:18:04.280
anything like that.

00:18:04.280 --> 00:18:05.780
You can learn then
almost anything.

00:18:08.450 --> 00:18:11.420
So this was picked up
a lot in the US.

00:18:11.420 --> 00:18:12.670
John Watson is a famous name.

00:18:12.670 --> 00:18:18.410
You'll see kind of a haunting
film of him performing this

00:18:18.410 --> 00:18:20.730
kinds of stuff with infants in
a kind of a way that we would

00:18:20.730 --> 00:18:21.980
never allow ethically now.

00:18:21.980 --> 00:18:22.530
It's not horrible.

00:18:22.530 --> 00:18:24.590
But it's pretty bad.

00:18:24.590 --> 00:18:25.790
But it's a famous thing.

00:18:25.790 --> 00:18:27.790
And you'll just get a feeling
of the power of this.

00:18:27.790 --> 00:18:31.870
Behaviorism, that if you study
behavior, instead of going for

00:18:31.870 --> 00:18:35.200
things like unobservable
thoughts, that you would just

00:18:35.200 --> 00:18:36.350
do observable actions.

00:18:36.350 --> 00:18:38.310
And you wouldn't make inferences
about the mind that

00:18:38.310 --> 00:18:39.420
you can't measure.

00:18:39.420 --> 00:18:40.600
You make a stimulus.

00:18:40.600 --> 00:18:41.760
You measure the response.

00:18:41.760 --> 00:18:43.400
And that's it.

00:18:43.400 --> 00:18:45.480
No fundamental differences
between animals and humans,

00:18:45.480 --> 00:18:47.570
and you make laws that describe
the relationships

00:18:47.570 --> 00:18:50.260
between stimulating the
environment and physical

00:18:50.260 --> 00:18:53.980
behaviors that you can observe
or responses out in the world

00:18:53.980 --> 00:18:55.830
that an organism has.

00:18:55.830 --> 00:18:57.010
So I'll come back to
this a little bit.

00:18:57.010 --> 00:18:59.620
But one of the ways he wanted
to show that anybody could

00:18:59.620 --> 00:19:02.920
learn anything, because right
now, don't forget, the way we

00:19:02.920 --> 00:19:04.920
set this up, anybody
can learn anything.

00:19:04.920 --> 00:19:08.060
You can applaud anything,
right?

00:19:08.060 --> 00:19:12.230
So you can teach anybody
anything by this perspective.

00:19:12.230 --> 00:19:14.010
And he wanted to show that
he could teach an infant

00:19:14.010 --> 00:19:16.500
initially, whose name
was Little Albert.

00:19:16.500 --> 00:19:18.040
That was his cover name.

00:19:18.040 --> 00:19:22.255
Nobody actually knows his real
name or what happened to him.

00:19:22.255 --> 00:19:24.501
Well, nothing horrible happened
to this person we

00:19:24.501 --> 00:19:25.130
know about.

00:19:25.130 --> 00:19:28.200
And now you would never do
an experiment like this.

00:19:28.200 --> 00:19:29.310
You have this in your notes.

00:19:29.310 --> 00:19:33.530
This is a long version of the
movies taken of this actual

00:19:33.530 --> 00:19:35.900
sort of experiment that
we wouldn't do now.

00:19:35.900 --> 00:19:37.590
And I'll show you a short
version in a couple minutes

00:19:37.590 --> 00:19:40.475
where he got an infant to become
afraid of a rabbit that

00:19:40.475 --> 00:19:44.130
the infant previously
was not afraid of.

00:19:44.130 --> 00:19:46.180
And then the other huge name
in this is B.F. Skinner who

00:19:46.180 --> 00:19:50.810
worked at Harvard, consequence
of a response, again, creates

00:19:50.810 --> 00:19:52.380
the responses that we
make in the world.

00:19:52.380 --> 00:19:55.310
And his sense response is not
just the laboratory one but

00:19:55.310 --> 00:19:58.762
things we do in the world with
each other and to each other.

00:19:58.762 --> 00:20:01.240
And so the classical
conditioning, the

00:20:01.240 --> 00:20:03.130
CS elicits the CR.

00:20:03.130 --> 00:20:05.520
The conditioned stimulus, the
bell, elicits the conditioned

00:20:05.520 --> 00:20:06.820
response, salivation.

00:20:06.820 --> 00:20:09.820
But in instrumental conditioning
CRs are omitted.

00:20:09.820 --> 00:20:12.410
And you can do really anything
that's in the range of what an

00:20:12.410 --> 00:20:14.160
organism can do.

00:20:14.160 --> 00:20:17.410
And then he could show kind of
impressively how you might

00:20:17.410 --> 00:20:19.550
imagine a chain of these
things would

00:20:19.550 --> 00:20:21.440
lead to complex behaviors.

00:20:21.440 --> 00:20:25.010
So he would teach pigeons and
other animals to first to

00:20:25.010 --> 00:20:27.500
click to get a pellet, then
where to click it, then to

00:20:27.500 --> 00:20:29.370
where to face before
you click it.

00:20:29.370 --> 00:20:31.670
And he would just keep adding
little pieces, shape the

00:20:31.670 --> 00:20:34.560
pieces up from the first step
to the second step, until he

00:20:34.560 --> 00:20:38.120
got pretty complex behaviors.

00:20:38.120 --> 00:20:40.050
OK, so now I'm going to show a
slightly longer clip that has

00:20:40.050 --> 00:20:42.580
in it these kinds of experiments
and Little Albert.

00:20:45.700 --> 00:20:54.180
It's kind of cool because in the
sense that we can actually

00:20:54.180 --> 00:20:56.260
see films of these sort of
historical things, which often

00:20:56.260 --> 00:20:57.995
we don't get to do in various
science fields.

00:21:07.800 --> 00:21:13.330
So let's talk about that
for just one moment.

00:21:13.330 --> 00:21:16.315
So we can talk about primary
reinforcers like food, thrist,

00:21:16.315 --> 00:21:18.830
or pain, things that go with
life, right, pain, not being

00:21:18.830 --> 00:21:20.300
injured, food, and thirst.

00:21:20.300 --> 00:21:23.820
We can talk about secondary
reinforcers that we know a lot

00:21:23.820 --> 00:21:28.520
about, money, attention of
other people, praise from

00:21:28.520 --> 00:21:30.820
other people, admissions to
places you want to be admitted

00:21:30.820 --> 00:21:33.000
to, promotions when you
want to get them.

00:21:33.000 --> 00:21:36.020
There can be positive
or negative rewards.

00:21:36.020 --> 00:21:36.890
And I'm going talk
about one thing.

00:21:36.890 --> 00:21:42.280
But just for a moment the
behaviorists said anybody can

00:21:42.280 --> 00:21:45.390
learn anything, right?

00:21:45.390 --> 00:21:46.260
We'll come back to that.

00:21:46.260 --> 00:21:50.050
So that's been a big debate
whether anything can be

00:21:50.050 --> 00:21:51.890
equally learned with
any other thing.

00:21:51.890 --> 00:21:56.180
And the other thing that's
striking is B.F. Skinner

00:21:56.180 --> 00:21:59.070
particularly said that we have
an illusion of free will, that

00:21:59.070 --> 00:22:00.760
all we are, are learning
these things.

00:22:00.760 --> 00:22:04.330
And we might as well get over
our illusion of free will.

00:22:04.330 --> 00:22:07.590
Scientifically, it's very hard
for even the best scientific

00:22:07.590 --> 00:22:09.120
psychologists to tell
you whether free

00:22:09.120 --> 00:22:10.270
will exists or not.

00:22:10.270 --> 00:22:12.290
That's not something that's
easy for us to begin to

00:22:12.290 --> 00:22:14.440
imagine how to measure.

00:22:14.440 --> 00:22:16.230
These kinds of learning
things are very

00:22:16.230 --> 00:22:17.160
impressive in many ways.

00:22:17.160 --> 00:22:18.410
And we'll talk about
some of that.

00:22:20.940 --> 00:22:27.610
So let me talk to you about one
way in which, for example,

00:22:27.610 --> 00:22:31.120
casinos get people to come back
often and other things in

00:22:31.120 --> 00:22:33.930
life, without giving you
money every time.

00:22:33.930 --> 00:22:36.220
Because then they wouldn't
make money, right?

00:22:36.220 --> 00:22:38.320
So your book talks about a
number of examples of this.

00:22:38.320 --> 00:22:41.070
But I'll just pick one,
partial reinforcement.

00:22:41.070 --> 00:22:44.290
So this means when do you get,
in these animals, when do you

00:22:44.290 --> 00:22:45.160
get a reward?

00:22:45.160 --> 00:22:48.070
Every time or only
1/3 of the time?

00:22:48.070 --> 00:22:52.290
So of course, you learn faster
if you get a reward every time

00:22:52.290 --> 00:22:53.350
than if it's 1/3 of the time.

00:22:53.350 --> 00:22:55.020
You're not quite as impressed
and excited.

00:22:55.020 --> 00:22:56.510
But here's the interesting
thing.

00:22:56.510 --> 00:23:00.080
If you stop giving the reward,
these animals will go on for

00:23:00.080 --> 00:23:02.390
quite a while before
they extinguish.

00:23:02.390 --> 00:23:04.590
These animals will extinguish
right away and stop

00:23:04.590 --> 00:23:05.330
responding.

00:23:05.330 --> 00:23:10.360
Why will these animals keep
behaving for a time, even

00:23:10.360 --> 00:23:14.210
though you've entirely
stopped the rewards?

00:23:14.210 --> 00:23:14.560
Yeah?

00:23:14.560 --> 00:23:18.760
AUDIENCE: Since they only got it
30% of the time anyway, it

00:23:18.760 --> 00:23:21.040
seems plausible to them that
it would go a couple more

00:23:21.040 --> 00:23:22.380
times without them getting it.

00:23:22.380 --> 00:23:23.630
PROFESSOR: Yes, since they
only got, the answer was,

00:23:23.630 --> 00:23:26.760
which was right, they only got
it 30% of the time, so the

00:23:26.760 --> 00:23:30.010
first one or two nonpayments,
right, they're going to go,

00:23:30.010 --> 00:23:33.090
well, it's coming up next, OK,
in a couple more, OK, I'm

00:23:33.090 --> 00:23:36.600
coming up soon until they
finally decide that the whole

00:23:36.600 --> 00:23:38.270
thing is over, right?

00:23:38.270 --> 00:23:41.000
So that's very interesting, the
different reinforcement

00:23:41.000 --> 00:23:42.720
schedules, how they relate.

00:23:42.720 --> 00:23:45.390
You might think, well, you
always learn better if you

00:23:45.390 --> 00:23:46.150
reward more.

00:23:46.150 --> 00:23:48.950
But in some cases, if you want
to get people to keep doing

00:23:48.950 --> 00:23:51.810
stuff without direct reward,
partial reinforcement is

00:23:51.810 --> 00:23:54.770
actually more powerful.

00:23:54.770 --> 00:23:57.720
Now we're going to move into
dog experiments and then

00:23:57.720 --> 00:23:59.340
directly to sort of
thinking about

00:23:59.340 --> 00:24:01.260
people and our own lives.

00:24:01.260 --> 00:24:04.910
So this is a famous, famous
experiment at the University

00:24:04.910 --> 00:24:06.200
of Pennsylvania from Seligman.

00:24:06.200 --> 00:24:07.460
In this case, he used dogs.

00:24:07.460 --> 00:24:09.840
It's been done with many species
in many circumstances.

00:24:09.840 --> 00:24:11.070
And he put them in a hammock.

00:24:11.070 --> 00:24:13.430
So there's two dogs on two sides
of a thing that they

00:24:13.430 --> 00:24:14.130
walked around.

00:24:14.130 --> 00:24:16.170
And they don't see each other.

00:24:16.170 --> 00:24:21.850
The dog, A, would periodically
get a shock.

00:24:21.850 --> 00:24:25.670
And it could stop the shock by
pushing a panel near its nose.

00:24:25.670 --> 00:24:27.560
As soon as the shock came,
it could turn its nose

00:24:27.560 --> 00:24:29.390
and stop the shock.

00:24:29.390 --> 00:24:33.930
Dog B on the other side of the
divider would get exactly the

00:24:33.930 --> 00:24:34.910
same shock.

00:24:34.910 --> 00:24:37.370
It would stop at exactly the
moment that the first dog

00:24:37.370 --> 00:24:38.740
stopped it.

00:24:38.740 --> 00:24:42.370
But dog B had no direct control,
equal number of

00:24:42.370 --> 00:24:45.610
shocks, equal duration of
objective pain, if you want to

00:24:45.610 --> 00:24:46.610
call it that way.

00:24:46.610 --> 00:24:49.180
One animal has the capacity
to stop it.

00:24:49.180 --> 00:24:51.000
The other does not.

00:24:51.000 --> 00:24:53.060
So that's the first part
of the experiment.

00:24:53.060 --> 00:24:54.840
And the second part of the
experiment they're put into a

00:24:54.840 --> 00:24:56.090
shuttle box.

00:24:58.420 --> 00:25:01.890
It's just a box with a big
divider in the middle.

00:25:01.890 --> 00:25:03.510
And they're just hanging
out there.

00:25:03.510 --> 00:25:04.510
And then they hear a tone.

00:25:04.510 --> 00:25:07.450
At first, it means
nothing to them.

00:25:07.450 --> 00:25:11.370
And then comes a shock on
the grid on the floor.

00:25:11.370 --> 00:25:13.850
And at first, they don't
know what to do.

00:25:13.850 --> 00:25:15.900
But then after a while, they
get smart like the cat.

00:25:15.900 --> 00:25:18.120
And they jump over the divider
to the other side.

00:25:18.120 --> 00:25:20.020
And it's all OK.

00:25:20.020 --> 00:25:22.680
So you can imagine that when
they hear the tone pretty

00:25:22.680 --> 00:25:24.860
quickly that what do
they learn to do?

00:25:24.860 --> 00:25:27.860
Jump over the divider
and stop the pain.

00:25:27.860 --> 00:25:29.040
And here's the remarkable
thing.

00:25:29.040 --> 00:25:31.910
For the animals who had the
experience that pushing the

00:25:31.910 --> 00:25:35.290
panel stopped the pain, they
learned that pretty quickly.

00:25:35.290 --> 00:25:39.800
For the animals who could not
stop the pain initially, they

00:25:39.800 --> 00:25:42.680
don't learn how to escape
when they could.

00:25:42.680 --> 00:25:46.030
It's as if the animals in group
B, what they learned

00:25:46.030 --> 00:25:48.580
about life, it's painful.

00:25:48.580 --> 00:25:51.820
And you can't do anything
about it.

00:25:51.820 --> 00:25:55.030
The animals in group A
learned pain happens.

00:25:55.030 --> 00:25:58.560
But I can do something
about it.

00:25:58.560 --> 00:26:00.190
So here's the animal.

00:26:00.190 --> 00:26:02.030
At first, it doesn't
know what to do.

00:26:02.030 --> 00:26:02.870
It's getting the shock.

00:26:02.870 --> 00:26:06.400
But with trials, the animal
learns to jump and jump so

00:26:06.400 --> 00:26:07.990
quickly that it doesn't have
almost any problem.

00:26:07.990 --> 00:26:08.720
Here's a tone, boom.

00:26:08.720 --> 00:26:09.710
It's out of there.

00:26:09.710 --> 00:26:11.730
It learns to avoid the pain.

00:26:11.730 --> 00:26:14.620
But what happens to the animals
who either had the

00:26:14.620 --> 00:26:16.240
escapable shock or the ones?

00:26:16.240 --> 00:26:20.400
The ones who could poke their
nose, they also learned this.

00:26:20.400 --> 00:26:23.350
The ones who could jump over,
there's nothing preventing

00:26:23.350 --> 00:26:24.780
them from learning that.

00:26:24.780 --> 00:26:27.490
They just don't escape.

00:26:27.490 --> 00:26:30.950
It's what Seligman called
learned helplessness, that the

00:26:30.950 --> 00:26:34.550
animals learn, from the first
experience, their conclusion

00:26:34.550 --> 00:26:36.350
was pain happens.

00:26:36.350 --> 00:26:37.840
And I can't do anything
about it.

00:26:37.840 --> 00:26:39.600
And then when they get in a
second situation where they

00:26:39.600 --> 00:26:43.060
can do a lot about it, they
continue to feel helpless and

00:26:43.060 --> 00:26:44.220
not do something about it.

00:26:44.220 --> 00:26:46.620
It's like a deep lesson about
life, like when do you think

00:26:46.620 --> 00:26:49.000
you can make your situation
better and when do you think

00:26:49.000 --> 00:26:50.600
there's nothing I
can do about it?

00:26:50.600 --> 00:26:52.140
It's just bad, bad, bad.

00:26:54.700 --> 00:26:57.010
So there's ideas about that some
of this might be related

00:26:57.010 --> 00:26:59.720
to people who struggle with
depression but also other

00:26:59.720 --> 00:27:00.290
things in life.

00:27:00.290 --> 00:27:03.820
So here's the idea from Seligman
as an explanation of

00:27:03.820 --> 00:27:06.460
how we have resilience.

00:27:06.460 --> 00:27:10.310
The idea is that we all get in
sometimes in small doses,

00:27:10.310 --> 00:27:12.590
sometimes in tragically
large doses, difficult

00:27:12.590 --> 00:27:13.940
things into our lives.

00:27:13.940 --> 00:27:15.630
How do we explain those
difficulties that have

00:27:15.630 --> 00:27:17.930
happened to us, the
setbacks we have?

00:27:17.930 --> 00:27:19.780
Do we think they're
internal to us?

00:27:19.780 --> 00:27:21.980
I'm the kind of person
who has bad luck.

00:27:21.980 --> 00:27:24.050
I'm the kind of person that
these things happen to.

00:27:24.050 --> 00:27:25.420
Or is it external?

00:27:25.420 --> 00:27:29.440
Somebody gave me a test.

00:27:29.440 --> 00:27:30.380
What can I do about that?

00:27:30.380 --> 00:27:31.630
But next week the
test is over.

00:27:31.630 --> 00:27:32.990
And life will be better.

00:27:32.990 --> 00:27:34.730
Is it global or specific?

00:27:34.730 --> 00:27:36.610
Does it just seem miserable
all over?

00:27:36.610 --> 00:27:39.960
Or is it just one specific
squirt in the face that this

00:27:39.960 --> 00:27:41.680
faculty member gave me?

00:27:41.680 --> 00:27:43.210
Is it stable or unstable?

00:27:43.210 --> 00:27:45.720
Are bad things just seeming
to happen all the time?

00:27:45.720 --> 00:27:47.340
Or is it one occasion?

00:27:47.340 --> 00:27:49.590
And you can imagine that people
who believe that bad

00:27:49.590 --> 00:27:51.220
things are internal,
global, and stable

00:27:51.220 --> 00:27:53.520
will learn to be helpless.

00:27:53.520 --> 00:27:56.770
Everything seems dark
and hopeless.

00:27:56.770 --> 00:27:59.330
People who believe that things
are external, specific, and

00:27:59.330 --> 00:28:01.830
unstable think, the next
time I can do better.

00:28:01.830 --> 00:28:03.840
I can bounce back from
this and do better.

00:28:03.840 --> 00:28:06.470
And it's not only in terms of
the sort of spirit of people

00:28:06.470 --> 00:28:09.740
and trying to get people to
bounce back from setbacks.

00:28:09.740 --> 00:28:11.770
It's even things like
job applications.

00:28:11.770 --> 00:28:14.050
Seligman did the
following bet.

00:28:14.050 --> 00:28:15.840
I don't even know what
corresponds to it before.

00:28:15.840 --> 00:28:18.290
But there used to be an era when
people would go knock on

00:28:18.290 --> 00:28:19.670
doors and try to sell
you insurance.

00:28:19.670 --> 00:28:20.590
We don't do that
anymore, right?

00:28:20.590 --> 00:28:22.025
Because if somebody comes to our
door, knocks on our door,

00:28:22.025 --> 00:28:23.860
we're calling the police.

00:28:23.860 --> 00:28:26.650
But when things were more
calm, you'd open

00:28:26.650 --> 00:28:27.700
the door and talk.

00:28:27.700 --> 00:28:30.240
Or they would give you cold
calls, which you don't get

00:28:30.240 --> 00:28:31.340
much more anymore these days.

00:28:31.340 --> 00:28:34.760
You guys, have you ever even
heard that phrase?

00:28:34.760 --> 00:28:35.940
You'd work at some insurance
company.

00:28:35.940 --> 00:28:37.590
And you'd go, here's some random
telephone numbers.

00:28:37.590 --> 00:28:38.370
You call them up.

00:28:38.370 --> 00:28:40.430
And you try to sell
them insurance.

00:28:40.430 --> 00:28:42.150
Usually, you get abused
on the phone.

00:28:42.150 --> 00:28:43.440
Or somebody slams the doors.

00:28:43.440 --> 00:28:44.980
But these salespeople, they
make commissions.

00:28:44.980 --> 00:28:45.850
And they keep going.

00:28:45.850 --> 00:28:47.690
And he said, let
me get my test.

00:28:47.690 --> 00:28:49.780
He gave a questionnaire whether
people interpret the

00:28:49.780 --> 00:28:52.750
world one way or the other way,
the resilient way or the

00:28:52.750 --> 00:28:54.190
learned helplessness way.

00:28:54.190 --> 00:28:57.490
And you give your usual
interview way for picking

00:28:57.490 --> 00:28:58.250
sales people.

00:28:58.250 --> 00:29:00.230
And let me see who picks
a better salesperson.

00:29:00.230 --> 00:29:02.810
And his questionnaire picked
a better salesperson.

00:29:02.810 --> 00:29:06.340
Because if you have a sales job
where people often kick

00:29:06.340 --> 00:29:09.970
you out and hang up on you, you
better be pretty external,

00:29:09.970 --> 00:29:12.430
specific, and unstable in your
perception on things, right?

00:29:12.430 --> 00:29:15.300
Because when you have 100 people
who hang up on you or

00:29:15.300 --> 00:29:18.970
close the door on you for one
sale, you need to have a thick

00:29:18.970 --> 00:29:20.500
resilient skin.

00:29:20.500 --> 00:29:23.350
So there's lots of ways in which
I think this speaks to

00:29:23.350 --> 00:29:27.680
how do we respond to the
setbacks in our lives.

00:29:27.680 --> 00:29:30.120
So let me talk about some
limits to conditioning.

00:29:30.120 --> 00:29:32.490
And some of these that I think
touch also on what are the

00:29:32.490 --> 00:29:34.910
values by which we
lead our lives.

00:29:34.910 --> 00:29:36.690
It's not going to be
telling you that.

00:29:36.690 --> 00:29:39.550
But it'll make you think about
some of the mechanics of what

00:29:39.550 --> 00:29:40.850
matters to you.

00:29:40.850 --> 00:29:42.800
So let me slide back
for a moment.

00:29:42.800 --> 00:29:45.050
So the behaviorists said
everybody can learn

00:29:45.050 --> 00:29:45.520
everything.

00:29:45.520 --> 00:29:46.790
Everybody can learn
everything.

00:29:46.790 --> 00:29:48.720
In that sense, it was seen
as a very democratic and

00:29:48.720 --> 00:29:50.510
egalitarian perspective.

00:29:50.510 --> 00:29:52.020
Everybody can learn
everything.

00:29:52.020 --> 00:29:54.270
It's just how you set up the
contingencies in the

00:29:54.270 --> 00:29:55.380
environment.

00:29:55.380 --> 00:29:59.920
And then Garcia did a famous
experiment where he had rats

00:29:59.920 --> 00:30:02.940
get shocks or receive
lithium chloride,

00:30:02.940 --> 00:30:05.830
which made them nauseous.

00:30:05.830 --> 00:30:10.590
And they either had a bright
noisy stimulus.

00:30:10.590 --> 00:30:13.770
Or they tasted some
sweet water.

00:30:13.770 --> 00:30:17.030
And even though these things
were arbitrarily combined, the

00:30:17.030 --> 00:30:21.990
rats learned, who received
shocked, learn to fear the

00:30:21.990 --> 00:30:25.060
bright noise, the bright
light and the noise.

00:30:25.060 --> 00:30:26.860
And the rats who got the lithium
chloride that made

00:30:26.860 --> 00:30:29.360
them nauseous learned to
fear the sweet water.

00:30:29.360 --> 00:30:31.520
Do you see why?

00:30:31.520 --> 00:30:34.720
When we get nauseous, do we
think like, boy, it was noisy

00:30:34.720 --> 00:30:35.150
last night?

00:30:35.150 --> 00:30:37.310
Or do you think, when you wake
up in the morning and you feel

00:30:37.310 --> 00:30:39.890
nauseous, what's the first
thing you think?

00:30:39.890 --> 00:30:42.650
PROFESSOR: Yeah, something I
ate, right, we're prepared for

00:30:42.650 --> 00:30:47.740
internal nausea to think
about food as a risk.

00:30:47.740 --> 00:30:49.830
And if it's a shock, we're
prepared to think of

00:30:49.830 --> 00:30:51.430
something out there.

00:30:51.430 --> 00:30:54.560
So these animals instinctively,
not on the

00:30:54.560 --> 00:30:59.730
basis of the training,
instinctively said, if I feel

00:30:59.730 --> 00:31:02.660
a shock, it's probably related
to this bright noisy stimulus

00:31:02.660 --> 00:31:03.880
that warns me.

00:31:03.880 --> 00:31:08.460
If I feel nauseous, it's related
to the sugar water.

00:31:08.460 --> 00:31:10.190
And they had these opposite
patterns of learning for the

00:31:10.190 --> 00:31:14.780
stimuli as if these animals
were prepared to fear some

00:31:14.780 --> 00:31:15.860
things and not others.

00:31:15.860 --> 00:31:17.670
So it wasn't that everything
could be matched with

00:31:17.670 --> 00:31:18.150
everything.

00:31:18.150 --> 00:31:20.620
And in fact, with humans,
they've done things like show

00:31:20.620 --> 00:31:22.680
you pictures either of
snakes and spiders

00:31:22.680 --> 00:31:24.260
or flowers and mushrooms.

00:31:24.260 --> 00:31:26.420
And then those predicted
a shock, a

00:31:26.420 --> 00:31:27.900
small shock to humans.

00:31:27.900 --> 00:31:29.820
And they measured your galvanic
skin response and

00:31:29.820 --> 00:31:31.020
autonomic response.

00:31:31.020 --> 00:31:32.950
When you think you're going to
get a shock, you start to get

00:31:32.950 --> 00:31:34.790
a little sweaty.

00:31:34.790 --> 00:31:36.170
And what they found was--

00:31:36.170 --> 00:31:38.840
and this will not shock you--
they had better conditioning,

00:31:38.840 --> 00:31:41.330
you more quickly and completely
learned that a

00:31:41.330 --> 00:31:43.810
snake or a spider predicts
a shock than

00:31:43.810 --> 00:31:45.740
a flower or a mushroom.

00:31:45.740 --> 00:31:49.620
Even though, objectively, they
were just as predictive.

00:31:49.620 --> 00:31:52.280
There's something about snakes
and spiders that gets us ready

00:31:52.280 --> 00:31:53.660
for scary stuff.

00:31:53.660 --> 00:31:55.830
And flowers and mushrooms, they
really have to punish us

00:31:55.830 --> 00:31:59.820
before we think they're evil
signals so, again, a human

00:31:59.820 --> 00:32:01.110
example of preparedness.

00:32:01.110 --> 00:32:02.530
We don't learn everything
equally.

00:32:02.530 --> 00:32:05.480
We are prepared to learn some
things and not others.

00:32:05.480 --> 00:32:09.050
Same thing with Little Albert,
the child that you saw, that

00:32:09.050 --> 00:32:10.320
kind of a film has
a funny status.

00:32:10.320 --> 00:32:12.500
And we'll talk about a few
more things this semester

00:32:12.500 --> 00:32:14.920
where that experiment,
why should that

00:32:14.920 --> 00:32:16.520
experiment not be done?

00:32:19.890 --> 00:32:20.420
Yeah?

00:32:20.420 --> 00:32:22.430
AUDIENCE: Nobody knows what
happened to Little Albert.

00:32:22.430 --> 00:32:25.640
PROFESSOR: Yeah, well, you would
never terrorize a child,

00:32:25.640 --> 00:32:28.290
right, to scare the child.

00:32:28.290 --> 00:32:31.860
So weirdly enough, though, these
studies remain in the

00:32:31.860 --> 00:32:34.000
field because since we
can't do them again.

00:32:34.000 --> 00:32:36.470
Because we know they're
not right to do.

00:32:36.470 --> 00:32:38.330
We try to learn lessons
from them so

00:32:38.330 --> 00:32:39.730
that we learn something.

00:32:39.730 --> 00:32:42.190
The rat worked.

00:32:42.190 --> 00:32:43.560
He got scared of a rat.

00:32:43.560 --> 00:32:44.640
And then he got scared
of a bunny.

00:32:44.640 --> 00:32:46.930
He got scared of
a cat, anything

00:32:46.930 --> 00:32:48.390
animal, anything furry.

00:32:48.390 --> 00:32:51.690
But if they gave him a wooden
block, that didn't work.

00:32:51.690 --> 00:32:52.925
It's not everything.

00:32:52.925 --> 00:32:56.550
It has to be prepared
from that.

00:32:56.550 --> 00:33:02.440
Here's another one that also
challenges simple ideas of do

00:33:02.440 --> 00:33:05.700
we always learn simply
to get a reward.

00:33:05.700 --> 00:33:09.170
What kind of learning might
exist that's rewardless?

00:33:09.170 --> 00:33:12.160
So there's three groups of
rats in a goal maze.

00:33:12.160 --> 00:33:14.470
There's food rewards every
day for one group.

00:33:14.470 --> 00:33:15.420
They're the happy group.

00:33:15.420 --> 00:33:18.160
There's no rewards for
a grumbling group.

00:33:18.160 --> 00:33:20.220
There's a third group that gets
no reward for 10 days.

00:33:20.220 --> 00:33:21.630
And then it gets reward.

00:33:21.630 --> 00:33:25.160
So reward all the time, reward
never, no reward for 10 days

00:33:25.160 --> 00:33:27.260
and then you start to get
rewards all the time, and

00:33:27.260 --> 00:33:28.930
here's what happens
to the learning.

00:33:28.930 --> 00:33:31.950
If you look at this green line,
that's the food that

00:33:31.950 --> 00:33:32.970
gets rewarded all the time.

00:33:32.970 --> 00:33:37.010
So they make fewer errors
so they can

00:33:37.010 --> 00:33:39.100
get to the food faster.

00:33:39.100 --> 00:33:41.480
If you look at the group that's
never rewarded, this

00:33:41.480 --> 00:33:42.515
yellow line, it's
pretty steady.

00:33:42.515 --> 00:33:45.510
It's like I'm hanging out,
no reason to go anywhere.

00:33:45.510 --> 00:33:47.080
There's not much going on.

00:33:47.080 --> 00:33:48.720
Look at the orange group.

00:33:48.720 --> 00:33:52.120
For 10 days, up to here, they
look like the yellow line,

00:33:52.120 --> 00:33:53.860
right, not getting rewarded.

00:33:53.860 --> 00:33:55.090
I'm just going to
saunter around.

00:33:55.090 --> 00:33:56.800
There's nothing interesting
to learn here.

00:33:56.800 --> 00:34:01.030
But boom that first day that
it's rewarded, but just one

00:34:01.030 --> 00:34:02.765
day of learning, they're
all caught up.

00:34:02.765 --> 00:34:04.470
They're all caught up.

00:34:04.470 --> 00:34:07.000
So people call this latent
learning, that they were

00:34:07.000 --> 00:34:10.340
really learning a lot about
their environment anyway, even

00:34:10.340 --> 00:34:12.350
though it was of no
use at the moment.

00:34:12.350 --> 00:34:15.540
And then the moment it became
valuable information, boom,

00:34:15.540 --> 00:34:17.100
they're all caught up.

00:34:17.100 --> 00:34:18.414
It's as if they learned
all this information.

00:34:18.414 --> 00:34:19.420
And it was ready to go.

00:34:19.420 --> 00:34:22.510
Latent learning, the information
is stored up just

00:34:22.510 --> 00:34:23.810
because they were in the
environment, even though it

00:34:23.810 --> 00:34:24.800
was unrewarded.

00:34:24.800 --> 00:34:26.980
And as we go around, we often
pick up stuff, even though

00:34:26.980 --> 00:34:28.300
there's no direct reward.

00:34:28.300 --> 00:34:31.460
And sometimes it turns
out to be useful.

00:34:31.460 --> 00:34:33.670
People also love, humans,
including infants,

00:34:33.670 --> 00:34:37.150
contingency, the idea that
we feel like we have

00:34:37.150 --> 00:34:38.909
control over our world.

00:34:38.909 --> 00:34:42.389
So here's a two-month-old
infant, very young infant, in

00:34:42.389 --> 00:34:44.400
a little experiment where they
put the infant in a crib.

00:34:44.400 --> 00:34:47.500
And above is a very colorful
colored mobile.

00:34:47.500 --> 00:34:51.620
And in one version, when the
infant moves his or her head,

00:34:51.620 --> 00:34:53.650
there's a switch
in the pillow.

00:34:53.650 --> 00:34:54.810
The mobile moves.

00:34:54.810 --> 00:34:56.429
And the infants are smiling
and cooing.

00:34:56.429 --> 00:34:56.889
They love it.

00:34:56.889 --> 00:34:57.570
This is awesome.

00:34:57.570 --> 00:34:58.190
They move their head.

00:34:58.190 --> 00:35:00.020
The thing moves, pretty
impressive for a two-month.

00:35:00.020 --> 00:35:01.630
All they can do is wiggle
their head, right?

00:35:01.630 --> 00:35:04.130
They're not walking
or anything.

00:35:04.130 --> 00:35:08.740
They put another group of
infants in who have no switch.

00:35:08.740 --> 00:35:12.940
And just periodically, they
turn on the mobile.

00:35:12.940 --> 00:35:14.570
There's no smiling
and no cooing.

00:35:14.570 --> 00:35:17.190
And they equate the number
of mobile turnings.

00:35:17.190 --> 00:35:18.250
Why is that?

00:35:18.250 --> 00:35:20.730
Well, if it's just because you
like the mobile turning, then

00:35:20.730 --> 00:35:23.050
you should have both groups be
equaling cooing and smiling or

00:35:23.050 --> 00:35:23.760
disinterested.

00:35:23.760 --> 00:35:26.950
But there's something
fundamentally rewarding for

00:35:26.950 --> 00:35:29.280
humans, even at the age of two,
to feel like they have

00:35:29.280 --> 00:35:32.270
control over their environment
and their pleasures.

00:35:32.270 --> 00:35:34.160
And that's not explained
by conditioning.

00:35:34.160 --> 00:35:35.130
Nobody's getting
anything more.

00:35:35.130 --> 00:35:37.690
They're both seeing the equal
number of interesting turns.

00:35:37.690 --> 00:35:39.590
But one group feels like they're
controlling their

00:35:39.590 --> 00:35:41.050
environment.

00:35:41.050 --> 00:35:43.580
Here's another one that I'm
going to show you in an animal

00:35:43.580 --> 00:35:45.890
experiment where you
can control things.

00:35:45.890 --> 00:35:48.020
But you can imagine
in your own life.

00:35:48.020 --> 00:35:51.260
In my life, I see this
all the time.

00:35:51.260 --> 00:35:54.500
It's nice when something
pleasant happens in your life

00:35:54.500 --> 00:35:57.590
and your situation gets better
in some way, right?

00:35:57.590 --> 00:35:59.220
You get a better iPad, right?

00:36:01.840 --> 00:36:04.970
But how hard is it emotionally
when something gets worse?

00:36:04.970 --> 00:36:11.460
They take all your iPads away
or more serious things.

00:36:11.460 --> 00:36:15.060
When things get worse, often
we feel pretty miserable.

00:36:15.060 --> 00:36:16.560
And here's a neat
example of that.

00:36:16.560 --> 00:36:19.400
So these are rats in an
experiment so everything's

00:36:19.400 --> 00:36:20.250
controlled.

00:36:20.250 --> 00:36:21.840
So here's one group of rats.

00:36:21.840 --> 00:36:23.220
This is how quickly
they're running.

00:36:23.220 --> 00:36:25.150
So it's good to be high.

00:36:25.150 --> 00:36:27.060
One group of rats is
constantly getting

00:36:27.060 --> 00:36:28.290
one pellet a day.

00:36:28.290 --> 00:36:29.280
So they learn what to do.

00:36:29.280 --> 00:36:31.650
And they're sort of hanging
out here doing well.

00:36:31.650 --> 00:36:34.500
A second group of rats is the
luckiest group of all.

00:36:34.500 --> 00:36:35.430
They get eight pellets.

00:36:35.430 --> 00:36:36.940
And they learn more.

00:36:36.940 --> 00:36:38.370
But they flatten out.

00:36:38.370 --> 00:36:39.800
So you get more reward.

00:36:39.800 --> 00:36:41.480
You drive the behavior more.

00:36:41.480 --> 00:36:42.770
That's not unexpected.

00:36:42.770 --> 00:36:45.930
The interesting thing are
the two contrast groups.

00:36:45.930 --> 00:36:48.810
The contrast groups start
off with one or eight.

00:36:48.810 --> 00:36:51.030
And then, after a couple days,
they're flipped around.

00:36:51.030 --> 00:36:52.910
So some started with one.

00:36:52.910 --> 00:36:54.370
Then they got eight pellets.

00:36:54.370 --> 00:36:55.930
Others started with eight.

00:36:55.930 --> 00:36:56.680
Then they got one.

00:36:56.680 --> 00:36:57.920
Does that make sense?

00:36:57.920 --> 00:37:01.110
So here's the contrast
group of eight.

00:37:01.110 --> 00:37:02.026
They're learning one.

00:37:02.026 --> 00:37:05.230
They go eight, woo.

00:37:05.230 --> 00:37:06.360
They're happy.

00:37:06.360 --> 00:37:07.230
They're getting eight.

00:37:07.230 --> 00:37:08.670
Life got good.

00:37:08.670 --> 00:37:11.000
Look at the group for
whom life got worse.

00:37:11.000 --> 00:37:11.990
I'm getting eight.

00:37:11.990 --> 00:37:12.890
I'm getting eight.

00:37:12.890 --> 00:37:13.720
I'm getting eight.

00:37:13.720 --> 00:37:14.360
I'm getting one.

00:37:14.360 --> 00:37:15.060
I'm getting one.

00:37:15.060 --> 00:37:15.700
I'm getting one.

00:37:15.700 --> 00:37:17.360
It really stinks.

00:37:17.360 --> 00:37:20.350
They're actually worse than the
groups that got one all

00:37:20.350 --> 00:37:21.920
along, right?

00:37:21.920 --> 00:37:23.560
This is the lowest
line of all.

00:37:23.560 --> 00:37:24.770
They're protesting.

00:37:24.770 --> 00:37:26.420
This stinks.

00:37:26.420 --> 00:37:27.140
I was getting eight.

00:37:27.140 --> 00:37:27.890
Now I'm getting one.

00:37:27.890 --> 00:37:29.120
What a rip off.

00:37:29.120 --> 00:37:31.130
What kind of world
do I live in?

00:37:31.130 --> 00:37:33.080
So the learning is
protesting that.

00:37:33.080 --> 00:37:35.850
Now, by purely conditioning
one, why would

00:37:35.850 --> 00:37:36.920
this group be worse?

00:37:36.920 --> 00:37:39.340
They're still getting
a pellet.

00:37:39.340 --> 00:37:41.610
But because they're interpreting
it as a world

00:37:41.610 --> 00:37:45.570
that's going downhill, as a
worsening situation, they're

00:37:45.570 --> 00:37:46.980
less prone to learn it.

00:37:46.980 --> 00:37:49.520
They're sort of protesting in
their learning, things having

00:37:49.520 --> 00:37:50.950
gotten worse in the world.

00:37:54.820 --> 00:37:57.860
A really interesting theme along
the line of this reward,

00:37:57.860 --> 00:37:59.810
an unbelievable one that has
gotten a lot of attention, is

00:37:59.810 --> 00:38:01.500
delayed gratification.

00:38:01.500 --> 00:38:03.380
If you think about it, and the
analysis is not purely

00:38:03.380 --> 00:38:06.980
scientific but is a theme,
delayed gratification is a

00:38:06.980 --> 00:38:11.550
spectacular part of your life,
right, especially if you go

00:38:11.550 --> 00:38:12.630
through higher education.

00:38:12.630 --> 00:38:14.700
Or maybe everything is
like this, right?

00:38:14.700 --> 00:38:16.760
Do a good job in primary thing
and you'll get to a good

00:38:16.760 --> 00:38:17.180
middle school.

00:38:17.180 --> 00:38:18.740
Do a good job in middle school,
we'll get to a good

00:38:18.740 --> 00:38:19.170
high school.

00:38:19.170 --> 00:38:20.710
Do a good job in high school,
you get to MIT.

00:38:20.710 --> 00:38:23.990
Do a good job at MIT, you go to
med school or law school or

00:38:23.990 --> 00:38:25.120
become a teacher, whatever
you want to do.

00:38:25.120 --> 00:38:27.240
A good job at that and you'll
get to this thing.

00:38:27.240 --> 00:38:29.100
Do a good job of that, finally,
thank you very

00:38:29.100 --> 00:38:29.890
much, you're 95.

00:38:29.890 --> 00:38:31.140
And you're done.

00:38:32.960 --> 00:38:37.640
I mean it's kind of like you
guys are waiting a long time

00:38:37.640 --> 00:38:39.640
between the efforts you're
doing and what you might

00:38:39.640 --> 00:38:43.410
consider a palpable reward.

00:38:43.410 --> 00:38:45.950
So here's a fantastically
interesting study from Walter

00:38:45.950 --> 00:38:48.310
Mischel done at Stanford
some years ago.

00:38:48.310 --> 00:38:51.160
These children were four
to five years old.

00:38:51.160 --> 00:38:52.490
There were about 600
that were studied.

00:38:52.490 --> 00:38:55.110
They were mostly the children
of faculty and graduate

00:38:55.110 --> 00:38:57.680
students at Stanford.

00:38:57.680 --> 00:38:59.430
And they would come
into a room.

00:38:59.430 --> 00:39:00.740
And the experiment, slightly
varied, was

00:39:00.740 --> 00:39:01.490
basically like this.

00:39:01.490 --> 00:39:03.530
They would sit in front
of either a cookie or

00:39:03.530 --> 00:39:04.080
marshmallow.

00:39:04.080 --> 00:39:05.400
They used different rewards.

00:39:05.400 --> 00:39:07.910
But imagine it's
a marshmallow.

00:39:07.910 --> 00:39:09.930
In the movie I'll show you,
it's Oreo cookies.

00:39:09.930 --> 00:39:12.890
And they were told this.

00:39:12.890 --> 00:39:16.170
And they tried to make this,
Walter Mischel who did the

00:39:16.170 --> 00:39:18.390
experiment describes how he
played with kids to try to get

00:39:18.390 --> 00:39:22.010
the marshmallow look
really awesome.

00:39:22.010 --> 00:39:24.620
And the kid would sit there,
the four or five-year-old.

00:39:24.620 --> 00:39:26.670
And they were told, I'm going
to leave the room now.

00:39:26.670 --> 00:39:27.860
And there's a bell here.

00:39:27.860 --> 00:39:30.970
If you really want to eat that
marshmallow, ring the bell and

00:39:30.970 --> 00:39:32.410
just eat it.

00:39:32.410 --> 00:39:34.200
But if you wait a while--

00:39:34.200 --> 00:39:35.800
they didn't tell them
it was 15 minutes.

00:39:35.800 --> 00:39:36.470
It was 15 minutes.

00:39:36.470 --> 00:39:37.050
I'll come back.

00:39:37.050 --> 00:39:38.410
And I'll give you two.

00:39:38.410 --> 00:39:41.680
Delayed gratification, I can
have one marshmallow now.

00:39:41.680 --> 00:39:44.750
Or if I hang in there for 15
minutes and don't touch it, I

00:39:44.750 --> 00:39:47.070
can get two.

00:39:47.070 --> 00:39:49.380
Let me tell you the results.

00:39:49.380 --> 00:39:51.230
And then let me show
a film of this.

00:39:51.230 --> 00:39:52.960
When I was looking on YouTube
and the internet and stuff,

00:39:52.960 --> 00:39:55.540
there's a lot of posed
movies out there.

00:39:55.540 --> 00:39:58.630
This is the only one I found
from Walter Mischel himself

00:39:58.630 --> 00:40:00.970
that's a real movie of real
kids really doing this.

00:40:00.970 --> 00:40:03.700
I don't know why all the ones
are posed out there.

00:40:03.700 --> 00:40:06.340
But here's something
astounding.

00:40:06.340 --> 00:40:07.640
So these are all pretty similar

00:40:07.640 --> 00:40:10.290
children, children of faculty.

00:40:10.290 --> 00:40:12.550
Children varied how long
they could wait.

00:40:12.550 --> 00:40:14.180
And if they took, for example,
the children who

00:40:14.180 --> 00:40:15.290
didn't wait very long.

00:40:15.290 --> 00:40:16.690
They could only wait
30 seconds until

00:40:16.690 --> 00:40:17.910
they ate the cookie.

00:40:17.910 --> 00:40:19.680
And they compared them to the
children who would wait for

00:40:19.680 --> 00:40:22.470
the full 15 minutes
to eat the cookie.

00:40:22.470 --> 00:40:23.470
These are four and
five-year-olds.

00:40:23.470 --> 00:40:27.230
When they looked at them at 18
and their SATs, the children

00:40:27.230 --> 00:40:30.800
who would wait 15 minutes scored
over 200 points higher,

00:40:30.800 --> 00:40:32.210
which you know is a lot.

00:40:32.210 --> 00:40:34.000
That was in the days we just
had two SAT tests.

00:40:34.000 --> 00:40:37.550
So it was 1,600.

00:40:37.550 --> 00:40:41.400
And by many, many measures
in terms of educational

00:40:41.400 --> 00:40:45.320
attainment, in terms of health,
and many measures, the

00:40:45.320 --> 00:40:48.890
longer you waited at age four
or five, the healthier you

00:40:48.890 --> 00:40:51.220
were, the better SAT scores you
were, the further you went

00:40:51.220 --> 00:40:53.790
in school, astounding
correlations

00:40:53.790 --> 00:40:56.380
with real life stuff.

00:40:56.380 --> 00:40:58.630
And people are stunned
by this.

00:40:58.630 --> 00:41:00.700
Partly, it's again, who
knows what goes into

00:41:00.700 --> 00:41:01.540
that at four or five.

00:41:01.540 --> 00:41:02.960
How much is the influence
of parents?

00:41:02.960 --> 00:41:05.330
How much is the influence
of cultures and genes?

00:41:05.330 --> 00:41:06.170
Who knows?

00:41:06.170 --> 00:41:09.010
But that fateful moment when
the child goes for the

00:41:09.010 --> 00:41:12.410
marshmallow or not correlates
with incredibly important

00:41:12.410 --> 00:41:14.370
outcomes in the future.

00:41:14.370 --> 00:41:16.050
So let's see.

00:41:16.050 --> 00:41:17.300
Here we go.

00:41:24.070 --> 00:41:28.220
And you know, right, every
college I remember personally,

00:41:28.220 --> 00:41:30.325
it was always a battle,
like am I going

00:41:30.325 --> 00:41:31.200
to have fun or study?

00:41:31.200 --> 00:41:32.220
Am I going to have
fun or study?

00:41:32.220 --> 00:41:33.600
There's so many chances
to have fun.

00:41:33.600 --> 00:41:36.100
And there's so many pressures
to study, right?

00:41:36.100 --> 00:41:39.480
Well, studying can be fun.

00:41:39.480 --> 00:41:41.030
Learning is fun.

00:41:41.030 --> 00:41:42.220
How about when rewards harm?

00:41:42.220 --> 00:41:44.290
Again, these are things that are
not strictly predicted by

00:41:44.290 --> 00:41:45.160
conditioning.

00:41:45.160 --> 00:41:46.660
I have three or four
more minutes.

00:41:46.660 --> 00:41:50.100
So rats love to run, if you had
a hamster or anything like

00:41:50.100 --> 00:41:51.540
that, they love running
wheels.

00:41:51.540 --> 00:41:56.030
What if you give them, every
time they run, a food reward?

00:41:56.030 --> 00:41:58.540
Then it turns out rats no
longer run for fun.

00:41:58.540 --> 00:42:01.870
They only run when the
food is given.

00:42:01.870 --> 00:42:04.350
If you start to extinguish,
they stop running.

00:42:04.350 --> 00:42:07.270
So what had been a natural thing
that seemed like a whole

00:42:07.270 --> 00:42:11.180
lot of fun, if it became the
price you pay for a reward,

00:42:11.180 --> 00:42:12.490
all a sudden becomes a chore.

00:42:15.780 --> 00:42:20.870
A widely cited experiment
for Mark Lepper, he took

00:42:20.870 --> 00:42:24.220
preschoolers, so four-year-old
children, who love to draw.

00:42:24.220 --> 00:42:27.220
Lots of preschoolers
love to draw.

00:42:27.220 --> 00:42:28.760
Then they started to give
them, every time they

00:42:28.760 --> 00:42:31.300
drew, a gold star.

00:42:31.300 --> 00:42:32.950
Every time they got a reward.

00:42:32.950 --> 00:42:35.570
What happened when they stopped
given the gold stars?

00:42:35.570 --> 00:42:37.470
The kids stopped drawing.

00:42:37.470 --> 00:42:40.350
This is incredibly often cited
in incredibly interesting

00:42:40.350 --> 00:42:44.910
debates about whether children
in impoverished schools should

00:42:44.910 --> 00:42:48.710
be given cash rewards
for studying hard.

00:42:48.710 --> 00:42:50.630
There's lots of debates about
whether that's a good thing or

00:42:50.630 --> 00:42:52.420
not a good thing, an effective
thing or not

00:42:52.420 --> 00:42:53.560
an effective thing.

00:42:53.560 --> 00:42:56.640
This study is often cited for
arguing, if you make something

00:42:56.640 --> 00:42:59.330
that you should love, like
learning, purely rewarded, the

00:42:59.330 --> 00:43:04.250
minute the reward stops, it
becomes a complete chore.

00:43:04.250 --> 00:43:06.750
But we don't know how
that applies.

00:43:06.750 --> 00:43:09.710
Because we might read
about things like

00:43:09.710 --> 00:43:10.970
baseball player salaries.

00:43:10.970 --> 00:43:12.810
And it seems like there's a
pretty good relationship

00:43:12.810 --> 00:43:14.860
between performance
and salary.

00:43:14.860 --> 00:43:18.490
So it's not simply that
way in the world.

00:43:18.490 --> 00:43:21.240
We know that often rewards
are associated with high

00:43:21.240 --> 00:43:23.000
performance and promote
high performance.

00:43:23.000 --> 00:43:24.810
It's pretty complicated.

00:43:24.810 --> 00:43:29.050
The last slide is about a
fantastic debate that occurred

00:43:29.050 --> 00:43:32.140
here in many ways between B.F.
Skinner at Harvard and Noam

00:43:32.140 --> 00:43:36.970
Chomsky at MIT, which is, in
what sense is language learned

00:43:36.970 --> 00:43:39.060
by conditioning?

00:43:39.060 --> 00:43:41.290
So here's an example
against it.

00:43:41.290 --> 00:43:45.330
At one month, there's a switch
inside a rubber nipple hooked

00:43:45.330 --> 00:43:46.690
to a tape recorder.

00:43:46.690 --> 00:43:48.410
And when the baby sucked,
the tape plays.

00:43:48.410 --> 00:43:51.980
This allows the baby to
perform, to behave.

00:43:51.980 --> 00:43:55.480
And they would play syllables
from different languages.

00:43:55.480 --> 00:43:59.950
And they may or may not be
in their own languages.

00:43:59.950 --> 00:44:04.680
And by four days, babies
preferred the languages that

00:44:04.680 --> 00:44:07.670
they are exposed to.

00:44:07.670 --> 00:44:11.970
And one of the questions is,
people said, well, how do we

00:44:11.970 --> 00:44:13.980
get kids to learn language
from a behaviorist

00:44:13.980 --> 00:44:14.580
perspective?

00:44:14.580 --> 00:44:18.700
So one of the things that they
studied children, and they'll

00:44:18.700 --> 00:44:21.480
often, young children will say
wrong things like mama isn't a

00:44:21.480 --> 00:44:22.290
boy, he a girl.

00:44:22.290 --> 00:44:24.140
That's not good grammar,
right?

00:44:24.140 --> 00:44:26.500
And so there was an intuition
that parents would then go,

00:44:26.500 --> 00:44:27.340
that's not good grammar.

00:44:27.340 --> 00:44:28.700
And sometimes parents do that.

00:44:28.700 --> 00:44:31.020
But when they did studies
where they recorded long

00:44:31.020 --> 00:44:33.260
sessions with parents
doing this, parents

00:44:33.260 --> 00:44:35.690
almost never corrected.

00:44:35.690 --> 00:44:37.540
It was very rare.

00:44:37.540 --> 00:44:39.220
You may feel different
from your own record.

00:44:39.220 --> 00:44:43.100
But statistically, it's
very rare to collect.

00:44:43.100 --> 00:44:45.680
They often say that's right
or something like that.

00:44:45.680 --> 00:44:48.920
So how's the child picking up
language if nobody's giving

00:44:48.920 --> 00:44:51.580
them rewards in terms
of whether

00:44:51.580 --> 00:44:52.970
they're right or wrong?

00:44:52.970 --> 00:44:55.540
The idea is that we can say an
infinite number of sentences.

00:44:55.540 --> 00:44:59.580
Well, how can we possibly be
rewarded or conditioned for an

00:44:59.580 --> 00:45:00.820
infinite variety of sentences?

00:45:00.820 --> 00:45:02.070
How could that work?

00:45:04.360 --> 00:45:07.240
And finally, the phenomenon of
overgeneralizing, you may know

00:45:07.240 --> 00:45:10.680
that children will
overgeneralize things.

00:45:10.680 --> 00:45:14.700
Because we usually end E-D to
make a verb past tense, right?

00:45:18.100 --> 00:45:20.360
But some verbs don't
work that way.

00:45:20.360 --> 00:45:22.110
So you would say, my teacher
held the rabbit.

00:45:22.110 --> 00:45:24.300
Children often make these
overgeneralized rules.

00:45:24.300 --> 00:45:26.560
They say, my teacher
holded the rabbit.

00:45:26.560 --> 00:45:29.740
So obviously, they're learning a
rule and then applying it to

00:45:29.740 --> 00:45:32.040
an instance that they
never hear.

00:45:32.040 --> 00:45:34.570
Because almost no
adult says that.

00:45:34.570 --> 00:45:36.520
So they must be producing that,
not on the basis of

00:45:36.520 --> 00:45:40.100
conditioning, but Chomsky argued
on the basis of sort of

00:45:40.100 --> 00:45:45.670
genetically prepotent ways in
which we learn language, rules

00:45:45.670 --> 00:45:47.660
that we develop and figure out
language in the world.

00:45:47.660 --> 00:45:50.500
I will say that, for those of
you happen to be interested in

00:45:50.500 --> 00:45:54.690
this topic, there's been a
striking pushback on that in

00:45:54.690 --> 00:45:57.460
what people call statistical
learning.

00:45:57.460 --> 00:46:02.770
So as much as this seemed like
a done deal 10 years ago, it

00:46:02.770 --> 00:46:05.290
turns out the truth might be
somewhere in the middle with

00:46:05.290 --> 00:46:08.480
some genetic prepotentiziation
but also a lot of learning

00:46:08.480 --> 00:46:12.430
from your environment both.

00:46:12.430 --> 00:46:15.940
So you can think, as you leave
here, what are the applause

00:46:15.940 --> 00:46:17.150
that you listen to?

00:46:17.150 --> 00:46:19.880
Where's your free will and your
choice of what goals you

00:46:19.880 --> 00:46:21.690
pursue and what matters
to you?

00:46:21.690 --> 00:46:25.370
And how does the world attempt
to teach you things that you

00:46:25.370 --> 00:46:27.050
learn about what you
want to do and why

00:46:27.050 --> 00:46:28.300
you want to do them?