1
00:00:00 --> 00:00:01
 
 

2
00:00:01 --> 00:00:02
INTRODUCTION: The following
content is provided by MIT

3
00:00:02 --> 00:00:06
OpenCourseWare under a
Creative Commons license.

4
00:00:06 --> 00:00:09
Additional information
about our license and MIT

5
00:00:09 --> 00:00:15
OpenCourseWare in general is
available at OCW.MIT.edu.

6
00:00:15 --> 00:00:18
PROFESSOR: Anyway.
 

7
00:00:18 --> 00:00:21
If you're sitting there saying,
what are they talking about

8
00:00:21 --> 00:00:22
weird thing in the syllabus.
 

9
00:00:22 --> 00:00:25
This suggests you didn't read
the syllabus, which is why we

10
00:00:25 --> 00:00:27
put the weird line in the
syllabus about -- what is

11
00:00:27 --> 00:00:31
it -- what a warrior your
mother would have been.

12
00:00:31 --> 00:00:33
That sound faintly familiar?
 

13
00:00:33 --> 00:00:34
AUDIENCE: No.
 

14
00:00:34 --> 00:00:35
PROFESSOR: Well it should
sound familiar from reading

15
00:00:35 --> 00:00:36
the syllabus right?
 

16
00:00:36 --> 00:00:38
Nobody's got a clue
where it's from right?

17
00:00:38 --> 00:00:40
AUDIENCE: No.
 

18
00:00:40 --> 00:00:43
PROFESSOR: Except for Rachel.
 

19
00:00:43 --> 00:00:47
I know Rachel knows,
because I told her today.

20
00:00:47 --> 00:00:50
And obviously I cleverly
arranged for it not to

21
00:00:50 --> 00:00:52
be something that
could be Googled.

22
00:00:52 --> 00:00:56
Because several years ago when
Google appeared, I had a line

23
00:00:56 --> 00:00:58
in there to make sure people
were reading it and, you know,

24
00:00:58 --> 00:01:01
within 10 seconds of handing
out the syllabus, half the

25
00:01:01 --> 00:01:03
class said, oh yeah that's
from like Richard the

26
00:01:03 --> 00:01:04
third or something.
 

27
00:01:04 --> 00:01:07
So this one couldn't
be Googled.

28
00:01:07 --> 00:01:14
It is in fact from a
book by John Steinbeck.

29
00:01:14 --> 00:01:17
You probably all read some
Steinbeck in high school.

30
00:01:17 --> 00:01:22
Those of you who are fans of
Arthurian literature might want

31
00:01:22 --> 00:01:27
to read his rewrites of the
King Arthur or chunks of

32
00:01:27 --> 00:01:29
the King Arthur story.
 

33
00:01:29 --> 00:01:32
And when you do, you will
discover that that line about

34
00:01:32 --> 00:01:35
thinking about what kind of a
warrior your mother would

35
00:01:35 --> 00:01:37
have been is in there.
 

36
00:01:37 --> 00:01:40
Great book.
 

37
00:01:40 --> 00:01:43
Any other questions of similar
moment that I should be

38
00:01:43 --> 00:01:46
answering at the present time?
 

39
00:01:46 --> 00:01:47
No.
 

40
00:01:47 --> 00:01:47
OK.
 

41
00:01:47 --> 00:01:52
Well what I'm going to talk
about today is more of

42
00:01:52 --> 00:01:54
the detail, if you like.
 

43
00:01:54 --> 00:01:57
 
 

44
00:01:57 --> 00:02:01
Last time I had up on the board
this business about you're a

45
00:02:01 --> 00:02:04
slave to the environment,
you're a slave to your brain.

46
00:02:04 --> 00:02:08
Well this is the details of
the way in which you are a

47
00:02:08 --> 00:02:10
slave to your environment.
 

48
00:02:10 --> 00:02:15
Now typically that is the
heading of the title of the

49
00:02:15 --> 00:02:17
lecture is something like
learning the chapter

50
00:02:17 --> 00:02:18
is learning.
 

51
00:02:18 --> 00:02:22
You're here thinking that
what you came to do

52
00:02:22 --> 00:02:23
today is to learn.
 

53
00:02:23 --> 00:02:25
And that's not what
we're talking about.

54
00:02:25 --> 00:02:32
The sort of learning that you
are doing here is material that

55
00:02:32 --> 00:02:33
really gets covered in memory.
 

56
00:02:33 --> 00:02:40
I present material to you, you
store it in memories somewhere.

57
00:02:40 --> 00:02:43
At the appropriate time you
retrieve it from memory.

58
00:02:43 --> 00:02:47
We'll talk about that in a
few sessions down the line.

59
00:02:47 --> 00:02:52
What I'm talking about today is
a particular form of learning,

60
00:02:52 --> 00:02:56
which you can think of as
association learning.

61
00:02:56 --> 00:02:59
A very reflective
form of learning.

62
00:02:59 --> 00:03:06
A form of learning that is
phylogenetically very old.

63
00:03:06 --> 00:03:10
It shows up in essentially
any beast you care

64
00:03:10 --> 00:03:11
to try it out in.
 

65
00:03:11 --> 00:03:19
 
 

66
00:03:19 --> 00:03:22
Those of you who are budding
neuroscience sorts, if you were

67
00:03:22 --> 00:03:29
to go to Eric Kandel's website
and look at his Nobel Prize

68
00:03:29 --> 00:03:32
lecture -- because he
won the Nobel Prize.

69
00:03:32 --> 00:03:34
You don't usually poster
your Nobel Prize lecture

70
00:03:34 --> 00:03:35
until you win it.
 

71
00:03:35 --> 00:03:38
Because little
arrogant otherwise.

72
00:03:38 --> 00:03:42
He won the Nobel Prize for
working out the details

73
00:03:42 --> 00:03:44
simple association learning.
 

74
00:03:44 --> 00:03:48
 
 

75
00:03:48 --> 00:03:51
Initially, his work was done in
a beastie known as Aplysia or a

76
00:03:51 --> 00:03:54
plysia, which is a sea slug.
 

77
00:03:54 --> 00:03:57
It looks like a dill pickle
and has just about as much

78
00:03:57 --> 00:03:59
personality and brain.
 

79
00:03:59 --> 00:04:04
But it is capable of
this form of learning.

80
00:04:04 --> 00:04:07
 
 

81
00:04:07 --> 00:04:09
It's a gorgeous lecture.
 

82
00:04:09 --> 00:04:15
Nice art on the website.
 

83
00:04:15 --> 00:04:17
if you're going to give a Nobel
Prize lecture, you kind of

84
00:04:17 --> 00:04:18
don't want to waste it.
 

85
00:04:18 --> 00:04:20
So I heard him give a version
of this it a different

86
00:04:20 --> 00:04:25
meeting in Australia,
and it's a good lecture.

87
00:04:25 --> 00:04:28
All right what kind of
learning is this that

88
00:04:28 --> 00:04:29
we're talking about.
 

89
00:04:29 --> 00:04:35
How many people here
like strawberries?

90
00:04:35 --> 00:04:37
Some people like strawberries
some people don't

91
00:04:37 --> 00:04:38
like strawberries.
 

92
00:04:38 --> 00:04:42
Now you did not come into the
world knowing that fact.

93
00:04:42 --> 00:04:44
You now know that.
 

94
00:04:44 --> 00:04:49
And I do not think that most of
you learned that fact, you

95
00:04:49 --> 00:04:51
know, like in the circle
time at preschool.

96
00:04:51 --> 00:04:54
Hello children today we're
going to decide if you

97
00:04:54 --> 00:04:55
like strawberries.
 

98
00:04:55 --> 00:04:56
You like strawberries.
 

99
00:04:56 --> 00:04:57
You don't like strawberries.
 

100
00:04:57 --> 00:04:59
They make you, you know,
break out in a rash.

101
00:04:59 --> 00:05:00
Whereas no right?
 

102
00:05:00 --> 00:05:04
You learned about your
taste for strawberries

103
00:05:04 --> 00:05:05
in some different way.
 

104
00:05:05 --> 00:05:09
You learn that you should, you
know, dive under the bed when

105
00:05:09 --> 00:05:10
you see a flash of lightning.
 

106
00:05:10 --> 00:05:12
Because you're scared
of the thunder.

107
00:05:12 --> 00:05:14
You learned about that not
because you went to thunder

108
00:05:14 --> 00:05:20
school, but because you learned
about the association.

109
00:05:20 --> 00:05:24
You picked it up from
the environment.

110
00:05:24 --> 00:05:28
And it's that sort of
association learning that I'm

111
00:05:28 --> 00:05:31
going to talk about today.
 

112
00:05:31 --> 00:05:33
I'm going to talk about
two different versions.

113
00:05:33 --> 00:05:36
One of them is learning
associations between stimuli in

114
00:05:36 --> 00:05:40
the world, traditionally known
as classical conditioning or

115
00:05:40 --> 00:05:44
Pavlovian conditioning in
honor of Ivan Pavlov.

116
00:05:44 --> 00:05:47
The other is the learning the
associations between what you

117
00:05:47 --> 00:05:51
do and the consequences of
what you do, often called

118
00:05:51 --> 00:05:52
operant conditioning.
 

119
00:05:52 --> 00:05:55
Sometimes called Skinnerian
conditioning in honor of B.F.

120
00:05:55 --> 00:06:01
Skinner, one of it
great practitioners.

121
00:06:01 --> 00:06:07
Now let's start with this
stimulus form of learning.

122
00:06:07 --> 00:06:10
 
 

123
00:06:10 --> 00:06:14
The basic classical
conditioning experiment is one

124
00:06:14 --> 00:06:16
of those experiments that
people know about before they

125
00:06:16 --> 00:06:19
come into a psych
class typically.

126
00:06:19 --> 00:06:22
That's Pavlov and
his salivating dog.

127
00:06:22 --> 00:06:26
What Pavlov was doing
was studying digestion.

128
00:06:26 --> 00:06:30
And dogs were his
animal of choice.

129
00:06:30 --> 00:06:34
In fact he's also a Nobel Prize
winner, but his Nobel Prize

130
00:06:34 --> 00:06:36
lecture -- well it probably is
posted on the web somewhere.

131
00:06:36 --> 00:06:43
But anyway he won the Nobel
Prize for work on the digestive

132
00:06:43 --> 00:06:47
enzymes in I think the stomach.
 

133
00:06:47 --> 00:06:49
He then decided he was
interested in digestive

134
00:06:49 --> 00:06:51
enzymes in saliva.
 

135
00:06:51 --> 00:06:53
And he needed a
supply of saliva.

136
00:06:53 --> 00:06:55
Where you're going
to get saliva from?

137
00:06:55 --> 00:07:00
Well what he did was he put a
little cannula, a little hole,

138
00:07:00 --> 00:07:04
in the cheek of a dog, and
a little collection tube.

139
00:07:04 --> 00:07:07
And then he took his dog, put
him in a sort of harness and

140
00:07:07 --> 00:07:11
sprayed powdered meat
into the dog's mouth.

141
00:07:11 --> 00:07:13
If you spray powdered
meat into a dog's mouth,

142
00:07:13 --> 00:07:14
the dog does what?
 

143
00:07:14 --> 00:07:15
The dog salivates.
 

144
00:07:15 --> 00:07:18
And since the dog's got a
little hole in here some

145
00:07:18 --> 00:07:19
of the saliva drools out.
 

146
00:07:19 --> 00:07:22
And you can go off
and study it.

147
00:07:22 --> 00:07:28
What Pavlov discovered that was
of interest to him -- it turned

148
00:07:28 --> 00:07:31
out to be of interest to him --
was that he'd get up in the

149
00:07:31 --> 00:07:35
morning, grab the dog, put the
dog in the harness, and the dog

150
00:07:35 --> 00:07:40
would start salivating before
any meat powder showed up.

151
00:07:40 --> 00:07:43
Pavlov's thinking, hmm I
can save on meat powder.

152
00:07:43 --> 00:07:43
AUDIENCE: [LAUGHTER]
 

153
00:07:43 --> 00:07:49
PROFESSOR: But the difference
between the rest of us schlumps

154
00:07:49 --> 00:07:51
and Nobel Prize winning guys is
when they see something

155
00:07:51 --> 00:07:53
interesting, they know
that it's interesting.

156
00:07:53 --> 00:07:56
And what Pavlov figured out
is the dog is anticipating

157
00:07:56 --> 00:07:57
the meat powder.
 

158
00:07:57 --> 00:08:00
The dog has learned that
this situation means meat

159
00:08:00 --> 00:08:02
powder in some fashion.
 

160
00:08:02 --> 00:08:05
Gee that's actually more
interesting said Pavlov

161
00:08:05 --> 00:08:06
than dogs drool.
 

162
00:08:06 --> 00:08:09
I'm going to study
that for a awhile.

163
00:08:09 --> 00:08:13
And so what he did was he set
up the more familiar form

164
00:08:13 --> 00:08:17
of this where -- let's
introduce some terminology.

165
00:08:17 --> 00:08:20
He had food, the meat powder.
 

166
00:08:20 --> 00:08:24
We will call that an
unconditional stimulus.

167
00:08:24 --> 00:08:28
The characteristic of an
unconditional stimulus is that

168
00:08:28 --> 00:08:37
it produces an unconditioned
response, saliva in this case,

169
00:08:37 --> 00:08:38
without you having
to do anything.

170
00:08:38 --> 00:08:40
You know, get your
off the shelf dog.

171
00:08:40 --> 00:08:44
That dog's going to drool for
you if you give it food.

172
00:08:44 --> 00:08:50
Then what he did was to pair
that unconditioned stimulus

173
00:08:50 --> 00:08:53
with a conditioned stimulus,
like a bell or a tone or

174
00:08:53 --> 00:08:55
something like that.
 

175
00:08:55 --> 00:09:01
So now he was going bell, food
and the dog obligingly drooled.

176
00:09:01 --> 00:09:02
Bell, food, drool.
 

177
00:09:02 --> 00:09:05
Boom, boom, boom,
over and over again.

178
00:09:05 --> 00:09:10
Then the critical thing to do
is you do just the bell by

179
00:09:10 --> 00:09:12
itself and skip the food.
 

180
00:09:12 --> 00:09:13
What do you discover?
 

181
00:09:13 --> 00:09:17
You discover that the bell by
itself now produces the saliva.

182
00:09:17 --> 00:09:22
And when the conditioned
stimulus alone is producing

183
00:09:22 --> 00:09:26
the response, we call that
the conditioned response.

184
00:09:26 --> 00:09:30
The animal has learned
an association between

185
00:09:30 --> 00:09:31
bell and food.
 

186
00:09:31 --> 00:09:37
And you're seeing that
learning by measuring this

187
00:09:37 --> 00:09:41
conditioned response.
 

188
00:09:41 --> 00:09:45
That kind of learning, which is
what Kandel was studying in

189
00:09:45 --> 00:09:50
Aplysia, is presumably
reflexive, automatic outside of

190
00:09:50 --> 00:09:55
the realm of consciousness even
if I fall into sort of sloppy

191
00:09:55 --> 00:09:56
kind of language later.
 

192
00:09:56 --> 00:09:59
Understands that the dog is not
presumed to be sitting there

193
00:09:59 --> 00:10:05
thinking hmm bell, what
does that bell mean?

194
00:10:05 --> 00:10:07
That bell symbolizes for
me the appearance of.

195
00:10:07 --> 00:10:09
Nah.
 

196
00:10:09 --> 00:10:11
Reflexive happening
automatically.

197
00:10:11 --> 00:10:15
Doesn't matter what the, you
know, what the dog does

198
00:10:15 --> 00:10:16
or does not think.
 

199
00:10:16 --> 00:10:20
Indeed you can fall victim
to this sort of thing.

200
00:10:20 --> 00:10:23
Quite apart from your
conscious, wishes, desires,

201
00:10:23 --> 00:10:27
whatever, there's a vision
researcher sort of a half of

202
00:10:27 --> 00:10:31
generation older than
me who still mad at me.

203
00:10:31 --> 00:10:37
Because 30 years ago I couldn't
escape being conditioned

204
00:10:37 --> 00:10:39
in this fashion.
 

205
00:10:39 --> 00:10:41
Back in those days the
apparatus when you were

206
00:10:41 --> 00:10:44
putting up a visual
stimulus made a noise.

207
00:10:44 --> 00:10:47
I was supposed to push one
button as quickly as I could.

208
00:10:47 --> 00:10:50
One button if I saw something,
and another button if I

209
00:10:50 --> 00:10:51
didn't see something.
 

210
00:10:51 --> 00:10:54
But every time he put
up a stimulus, the

211
00:10:54 --> 00:10:56
apparatus went click.
 

212
00:10:56 --> 00:10:59
And I just would hit
the stupid button.

213
00:10:59 --> 00:11:06
I had learned the association
between click and the stimulus.

214
00:11:06 --> 00:11:12
And, you know, my little sea
slug brain was refusing to be

215
00:11:12 --> 00:11:16
overridden by this, you know,
great big conscious apparatus

216
00:11:16 --> 00:11:19
of this guy wants me to look
for some really boring thing.

217
00:11:19 --> 00:11:21
And he's still mad at me.
 

218
00:11:21 --> 00:11:23
It's very, very sad.
 

219
00:11:23 --> 00:11:24
OK.
 

220
00:11:24 --> 00:11:30
This is a very ruled
governed behavior.

221
00:11:30 --> 00:11:32
In fact it probably says
that on the handout.

222
00:11:32 --> 00:11:36
No it says what's
being learned here.

223
00:11:36 --> 00:11:43
It's a very ruled
governed behavior.

224
00:11:43 --> 00:11:46
Pavlov himself worked out many
of the rules, and then a large

225
00:11:46 --> 00:11:51
body of research afterwards
continued that effort.

226
00:11:51 --> 00:11:54
And let me tell you about some
of the constraints on this form

227
00:11:54 --> 00:11:58
of learning, and then explain a
bit about how it is that this

228
00:11:58 --> 00:12:01
might actually have anything to
do with learning more

229
00:12:01 --> 00:12:04
interesting things.
 

230
00:12:04 --> 00:12:12
What the sea slug learns is
that gee when I sense shrimp

231
00:12:12 --> 00:12:15
juices I recall in the water,
that means somebody's

232
00:12:15 --> 00:12:17
going to poke me.
 

233
00:12:17 --> 00:12:19
So I should retract my gill.
 

234
00:12:19 --> 00:12:22
That's the response that
the sea slug makes,

235
00:12:22 --> 00:12:23
to retract its gill.
 

236
00:12:23 --> 00:12:25
The response the dog
makes is to drool.

237
00:12:25 --> 00:12:28
I mean that's nice, but how
does that relate to anything

238
00:12:28 --> 00:12:32
like human behavior?
 

239
00:12:32 --> 00:12:35
So let me tell you a few
of the constraints on it.

240
00:12:35 --> 00:12:39
First a constraint that's
really more like an

241
00:12:39 --> 00:12:46
advertisement for next time is
to say that we talk about these

242
00:12:46 --> 00:12:49
conditioned stimuli out there.
 

243
00:12:49 --> 00:12:51
In this case the bell.
 

244
00:12:51 --> 00:12:56
Even something like deciding
what the stimulus is, is not

245
00:12:56 --> 00:12:59
an entirely trivial business.
 

246
00:12:59 --> 00:13:10
So suppose I was to come in
here with a rabbit let's say.

247
00:13:10 --> 00:13:11
Come here with a rabbit.
 

248
00:13:11 --> 00:13:16
Every time I walk in with the
rabbit, I activate the electric

249
00:13:16 --> 00:13:17
grid that's in your sea.
 

250
00:13:17 --> 00:13:20
And you all get a little zips.
 

251
00:13:20 --> 00:13:23
And, you know, you
jump in the air.

252
00:13:23 --> 00:13:27
Eventually I bring the
rabbit in and what happens?

253
00:13:27 --> 00:13:31
You jump without me having to
bother running electric current

254
00:13:31 --> 00:13:33
through your posterior.
 

255
00:13:33 --> 00:13:35
What's the stimulus there?
 

256
00:13:35 --> 00:13:39
Well we sort of naturally
would say it's, you

257
00:13:39 --> 00:13:41
know, it's the rabbit.
 

258
00:13:41 --> 00:13:44
And it could be
the rabbit's ear.

259
00:13:44 --> 00:13:49
It could be rabbit ear
combined with blackboard

260
00:13:49 --> 00:13:50
or something like that.
 

261
00:13:50 --> 00:13:52
It turns out that we have
natural, and animals have

262
00:13:52 --> 00:13:57
natural, ways of carving up
the world into stimuli,

263
00:13:57 --> 00:13:59
into objects perhaps.
 

264
00:13:59 --> 00:14:03
And when you're trying to
figure out what's governing the

265
00:14:03 --> 00:14:07
behavior in animal, it's not
even immediately trivially

266
00:14:07 --> 00:14:10
obvious what that
stimulus might be.

267
00:14:10 --> 00:14:12
That's what we'll end up
talking about for the next

268
00:14:12 --> 00:14:14
few lectures after this.
 

269
00:14:14 --> 00:14:19
But sticking with the realm of
the constraints on the learning

270
00:14:19 --> 00:14:24
itself, one important
constraint is that you have to

271
00:14:24 --> 00:14:29
notice the relationship between
the conditioned stimulus and

272
00:14:29 --> 00:14:31
the unconditioned stimulus.
 

273
00:14:31 --> 00:14:35
One version of that is
completely trivial.

274
00:14:35 --> 00:14:40
If I use a very quiet bell
that you can't hear you don't

275
00:14:40 --> 00:14:43
learn anything about it.
 

276
00:14:43 --> 00:14:44
You know big deal.
 

277
00:14:44 --> 00:14:51
The more interesting case is
illustrated by a phenomenon

278
00:14:51 --> 00:14:53
known as overshadowing.
 

279
00:14:53 --> 00:15:00
Suppose what I do is I have
to stimuli that appear

280
00:15:00 --> 00:15:02
at the same time.
 

281
00:15:02 --> 00:15:07
This bell and a light.
 

282
00:15:07 --> 00:15:10
A nice little, you know, little
LED d or something like that.

283
00:15:10 --> 00:15:13
So bell and light
followed by food.

284
00:15:13 --> 00:15:15
Bell and light
followed by food.

285
00:15:15 --> 00:15:15
Bell and light followed.
 

286
00:15:15 --> 00:15:18
There's a loud bell
and a little light.

287
00:15:18 --> 00:15:20
Got it?
 

288
00:15:20 --> 00:15:25
OK what happens in this
situation is that when you

289
00:15:25 --> 00:15:30
play the bell by itself,
you get salivation.

290
00:15:30 --> 00:15:33
When you play the lights by
itself, even though the light

291
00:15:33 --> 00:15:37
was always followed by the
food, you don't get salivation.

292
00:15:37 --> 00:15:40
Because the light has been
overshadowed -- that's where

293
00:15:40 --> 00:15:43
the jargon comes from -- the
light has been overshadowed

294
00:15:43 --> 00:15:44
by the presence of the bell.
 

295
00:15:44 --> 00:15:47
What's the critical
control experiment in an

296
00:15:47 --> 00:15:49
overshadowing paradigm?
 

297
00:15:49 --> 00:15:52
What do you need to know here
for this to be interesting?

298
00:15:52 --> 00:15:52
Yeah?
 

299
00:15:52 --> 00:15:53
AUDIENCE: [INAUDIBLE]
 

300
00:15:53 --> 00:15:55
PROFESSOR: You have to do a
training session with some

301
00:15:55 --> 00:15:58
other dogs or rat or something
with just the light.

302
00:15:58 --> 00:16:01
You have to go light, food,
light, food, light, food, and

303
00:16:01 --> 00:16:04
show that this relatively wimpy
light -- if presented in

304
00:16:04 --> 00:16:08
isolation -- will produce
perfectly fine learning.

305
00:16:08 --> 00:16:11
In the interesting version
of the experiment,

306
00:16:11 --> 00:16:12
that's in fact the case.
 

307
00:16:12 --> 00:16:18
But when you present, you know,
bell and lights together, the

308
00:16:18 --> 00:16:21
stronger bell captures
the learning and the

309
00:16:21 --> 00:16:22
weaker light loses.
 

310
00:16:22 --> 00:16:24
So, you know, is a silly
example where I was

311
00:16:24 --> 00:16:25
bringing in a rabbit.
 

312
00:16:25 --> 00:16:30
You know, if the rabbit was, oh
I don't know, wearing a little

313
00:16:30 --> 00:16:32
silver ring or something
like that, you'd learn the

314
00:16:32 --> 00:16:36
association between rabbit and
shock, but, not the association

315
00:16:36 --> 00:16:39
between ring and shock perhaps
because it was too small.

316
00:16:39 --> 00:16:44
It was overshadowed
as a stimulus.

317
00:16:44 --> 00:16:53
It is also critical that
the CS predicts the US.

318
00:16:53 --> 00:16:54
Did I get that right?
 

319
00:16:54 --> 00:16:54
Yes.
 

320
00:16:54 --> 00:16:57
The conditioned stimulus
has to predict the

321
00:16:57 --> 00:16:59
unconditioned stimulus.
 

322
00:16:59 --> 00:17:06
It is not adequate simply
for the CS to just show up

323
00:17:06 --> 00:17:09
every time the US shows up.
 

324
00:17:09 --> 00:17:22
So a silly example, everybody
who bombs the midterm in this

325
00:17:22 --> 00:17:27
class, almost everybody maybe
it's not perfect but it's a

326
00:17:27 --> 00:17:30
pretty good association, almost
everybody who bombs the midterm

327
00:17:30 --> 00:17:35
in this class drank
something shortly before

328
00:17:35 --> 00:17:38
doing that right.
 

329
00:17:38 --> 00:17:40
So drink, bomb.
 

330
00:17:40 --> 00:17:42
And it works every year.
 

331
00:17:42 --> 00:17:42
So there's a lot to pairing.
 

332
00:17:42 --> 00:17:44
Drink, bomb.
 

333
00:17:44 --> 00:17:47
Look, I wouldn't on the basis
of that give up drinking.

334
00:17:47 --> 00:17:50
There's no predictive
value there.

335
00:17:50 --> 00:17:52
Because you drink a bunch of
times when nothing has anything

336
00:17:52 --> 00:17:54
to do with the midterm.
 

337
00:17:54 --> 00:17:56
There's no relationships.
 

338
00:17:56 --> 00:17:58
What's important is a
contingent predictive

339
00:17:58 --> 00:17:59
relationship.
 

340
00:17:59 --> 00:18:03
This is a brain mechanisms
that's there to learn what

341
00:18:03 --> 00:18:06
in the world predicts what
other thing in the world.

342
00:18:06 --> 00:18:09
In the standard Pavlovian
set up here, the

343
00:18:09 --> 00:18:10
prediction is perfect.
 

344
00:18:10 --> 00:18:15
Every time you ring the
bell, you get the food.

345
00:18:15 --> 00:18:18
If you cut that down so that
the prediction is imperfect,

346
00:18:18 --> 00:18:21
sort of correlational, you'll
still learn but you'll

347
00:18:21 --> 00:18:24
learn less and you'll
learn more slowly.

348
00:18:24 --> 00:18:29
So if, you know, half the time
the bell rang you got food, and

349
00:18:29 --> 00:18:32
the bell never rang any other
time, the animal would learn

350
00:18:32 --> 00:18:35
that bell more or
less means food.

351
00:18:35 --> 00:18:41
But it would be slower
and less strong.

352
00:18:41 --> 00:18:42
That's not what I
wanted to move.

353
00:18:42 --> 00:18:45
You know, been speaking of
learning, I've been teaching

354
00:18:45 --> 00:18:48
in this room for years, and I
have still never managed to

355
00:18:48 --> 00:18:50
reliably use these boards.
 

356
00:18:50 --> 00:18:52
Always push the wrong button.
 

357
00:18:52 --> 00:18:52
All right.
 

358
00:18:52 --> 00:18:58
So suppose if you've got a good
strong, you know, bell, food,

359
00:18:58 --> 00:19:00
bell, food, bell, food kind of
thing, what you're going to do

360
00:19:00 --> 00:19:09
is over time you build up a
rate of responding to some

361
00:19:09 --> 00:19:12
sort of asintotic level.
 

362
00:19:12 --> 00:19:15
What happens if you
change the contingency?

363
00:19:15 --> 00:19:20
Most particularly what happens
if you stopped feeding

364
00:19:20 --> 00:19:24
the dog after the bell?
 

365
00:19:24 --> 00:19:26
AUDIENCE: The response
will go away.

366
00:19:26 --> 00:19:28
PROFESSOR: Oh the responsible
will go away, not the dog.

367
00:19:28 --> 00:19:30
The dog doesn't have the option
in this particular case.

368
00:19:30 --> 00:19:31
Yeah.
 

369
00:19:31 --> 00:19:36
The response will go away.
 

370
00:19:36 --> 00:19:38
That's known in the
conditioning literature

371
00:19:38 --> 00:19:41
as extinction.
 

372
00:19:41 --> 00:19:45
The response will extinguish.
 

373
00:19:45 --> 00:19:47
I mean that makes sense right?
 

374
00:19:47 --> 00:19:50
You know, if the contingency no
longer applies, why should I

375
00:19:50 --> 00:19:53
continue to respond on the
basis of a contingency

376
00:19:53 --> 00:19:55
that's no longer valid.
 

377
00:19:55 --> 00:20:02
Did you forget, you the dog,
did you forget that contingency

378
00:20:02 --> 00:20:03
or unlearned that contingency?
 

379
00:20:03 --> 00:20:05
Is it gone?
 

380
00:20:05 --> 00:20:06
The answer is no.
 

381
00:20:06 --> 00:20:10
It still seems to be
there in some sense.

382
00:20:10 --> 00:20:13
It's just you're no longer
making the response.

383
00:20:13 --> 00:20:13
How do we know that?
 

384
00:20:13 --> 00:20:19
Well for instance, if you were
to give the dog a little break,

385
00:20:19 --> 00:20:23
send him home, then put
him, back in the apparatus.

386
00:20:23 --> 00:20:26
You would discover that the
response, even if you never

387
00:20:26 --> 00:20:29
presented anymore food, the
response would come back

388
00:20:29 --> 00:20:32
and then extinguish again.
 

389
00:20:32 --> 00:20:35
This is known as
spontaneous recovery.

390
00:20:35 --> 00:20:38
 
 

391
00:20:38 --> 00:20:40
And, you know, to use the
language that we shouldn't be

392
00:20:40 --> 00:20:43
using, it's like the dog
saying, all right, you know,

393
00:20:43 --> 00:20:44
all right bell predicts food.
 

394
00:20:44 --> 00:20:47
Oh bell doesn't
predict food anymore.

395
00:20:47 --> 00:20:49
I mean it's not like I
forgot that the bell used

396
00:20:49 --> 00:20:50
to predict the food.
 

397
00:20:50 --> 00:20:51
I'm not a dumb dog,
I remember that.

398
00:20:51 --> 00:20:53
But it doesn't anymore,
so why should I drool.

399
00:20:53 --> 00:20:54
OK.
 

400
00:20:54 --> 00:21:00
Now so I wonder what
situation we're in here.

401
00:21:00 --> 00:21:01
Are we back here
or are we here?

402
00:21:01 --> 00:21:06
Well we'll drool a
little bit just to see.

403
00:21:06 --> 00:21:10
Again the dog is not presumed
to be doing anything like

404
00:21:10 --> 00:21:11
that kind of thinking.
 

405
00:21:11 --> 00:21:16
But it's not that far off from
more complicated situations

406
00:21:16 --> 00:21:19
that you could imagine
in the real world.

407
00:21:19 --> 00:21:25
So you develop a relationship
with another person that

408
00:21:25 --> 00:21:29
produces some rate of
conditioned response or

409
00:21:29 --> 00:21:31
something like that.
 

410
00:21:31 --> 00:21:35
Because, you know, while you're
responding in any case.

411
00:21:35 --> 00:21:39
Then he decides not to
have anything further

412
00:21:39 --> 00:21:41
to do with you.
 

413
00:21:41 --> 00:21:43
Response disappears.
 

414
00:21:43 --> 00:21:47
Then comes, oh I don't
know, summer vacation.

415
00:21:47 --> 00:21:50
And you see her again
after vacation.

416
00:21:50 --> 00:21:52
Do you remit a response?
 

417
00:21:52 --> 00:21:56
 
 

418
00:21:56 --> 00:22:00
Well are we here, or
maybe we're back here.

419
00:22:00 --> 00:22:01
AUDIENCE: [LAUGHTER]
 

420
00:22:01 --> 00:22:04
PROFESSOR: So you know,
maybe a little response.

421
00:22:04 --> 00:22:10
So that's spontaneous recovery.
 

422
00:22:10 --> 00:22:17
 
 

423
00:22:17 --> 00:22:19
The original notion was sort of
that you were general purpose

424
00:22:19 --> 00:22:22
association learn, you
can learn anything.

425
00:22:22 --> 00:22:25
The evidence from preparedness,
which I talked about before,

426
00:22:25 --> 00:22:27
suggests that you are better
prepared to learn some

427
00:22:27 --> 00:22:29
things than others.
 

428
00:22:29 --> 00:22:32
So you're better prepared to
learn that snakes are scary

429
00:22:32 --> 00:22:36
then that bunnies are
scary, for instance.

430
00:22:36 --> 00:22:40
It's fairly general purpose or
at least there are aspects of

431
00:22:40 --> 00:22:43
association learning that seem
to be general purpose, but not

432
00:22:43 --> 00:22:48
all of it is completely
general purpose.

433
00:22:48 --> 00:22:51
 
 

434
00:22:51 --> 00:22:55
One thing that you're certainly
set up to do is to notice

435
00:22:55 --> 00:23:00
associations only over
a limited time window.

436
00:23:00 --> 00:23:03
 
 

437
00:23:03 --> 00:23:04
Oh I lost underneath there.
 

438
00:23:04 --> 00:23:05
Right.
 

439
00:23:05 --> 00:23:06
Tone followed by food.
 

440
00:23:06 --> 00:23:10
Tone followed by food
that's the CS followed

441
00:23:10 --> 00:23:13
by the US, right?
 

442
00:23:13 --> 00:23:16
So let's take this asintotic
points here and plot

443
00:23:16 --> 00:23:18
that over here.
 

444
00:23:18 --> 00:23:22
Now suppose that we
systematically vary the

445
00:23:22 --> 00:23:26
relationship in time between
the conditioned stimulus and

446
00:23:26 --> 00:23:27
the unconditioned stimulus.
 

447
00:23:27 --> 00:23:31
Well this is on the order
say of one or two seconds.

448
00:23:31 --> 00:23:37
If I ring the bell today and
feed you tomorrow, how much

449
00:23:37 --> 00:23:38
learning you figure we get?
 

450
00:23:38 --> 00:23:40
AUDIENCE: [INAUDIBLE]
 

451
00:23:40 --> 00:23:41
PROFESSOR: Not a lot.
 

452
00:23:41 --> 00:23:44
First of all it's a very long
boring experiment, since

453
00:23:44 --> 00:23:45
we need lots of pairing.
 

454
00:23:45 --> 00:23:48
But basically you don't
have to go that far.

455
00:23:48 --> 00:23:52
But it's going to fade
off in this direction.

456
00:23:52 --> 00:23:55
That you can't have too
long a period between the

457
00:23:55 --> 00:23:59
conditioned stimulus and
the unconditioned stimulus.

458
00:23:59 --> 00:24:01
 
 

459
00:24:01 --> 00:24:05
Zero doesn't produce
much conditioning and

460
00:24:05 --> 00:24:07
negative relationships.
 

461
00:24:07 --> 00:24:12
So I give you food and
then I ring a bell.

462
00:24:12 --> 00:24:14
That says, you know,
food that was food.

463
00:24:14 --> 00:24:17
Food that was food.
 

464
00:24:17 --> 00:24:19
You know OK.
 

465
00:24:19 --> 00:24:23
It doesn't hit zero at zero,
but there's really basically

466
00:24:23 --> 00:24:25
very little happening
down that way.

467
00:24:25 --> 00:24:29
So there's a narrow time window
within which this chunk of you

468
00:24:29 --> 00:24:36
is looking for associations
out there in the world.

469
00:24:36 --> 00:24:38
The the world example
here might be bump

470
00:24:38 --> 00:24:40
signs on the highway.
 

471
00:24:40 --> 00:24:42
When is a bump sign on
the highway useful?

472
00:24:42 --> 00:24:44
If the bump sign shows up
at exactly the same place

473
00:24:44 --> 00:24:48
as the bump, thank you.
 

474
00:24:48 --> 00:24:50
You want it to show up a
little bit before the bump.

475
00:24:50 --> 00:24:54
If it shows up after the bump,
you know, that was a bump.

476
00:24:54 --> 00:24:58
AUDIENCE: [LAUGHTER]
 

477
00:24:58 --> 00:25:01
PROFESSOR: That's
not that useful.

478
00:25:01 --> 00:25:07
So it's limited
constrained in time.

479
00:25:07 --> 00:25:11
Now how can we go and
get from this nice rule

480
00:25:11 --> 00:25:16
governed behavior to more
complicated behaviors?

481
00:25:16 --> 00:25:19
 
 

482
00:25:19 --> 00:25:24
One way to see how that might
work is a paradigm in the

483
00:25:24 --> 00:25:29
conditioning literature known
as century -- did it again

484
00:25:29 --> 00:25:32
didn't I look at that, got it
got it wrong on both

485
00:25:32 --> 00:25:34
dimensions -- called
sensory preconditioning.

486
00:25:34 --> 00:25:40
 
 

487
00:25:40 --> 00:25:43
I'll just restart
this over here.

488
00:25:43 --> 00:25:45
It's easier.
 

489
00:25:45 --> 00:25:47
Here's how a sensory
preconditioning experiment

490
00:25:47 --> 00:25:52
might work if you were a rat or
dog or something like that.

491
00:25:52 --> 00:25:54
So step one.
 

492
00:25:54 --> 00:25:57
I'm going to show you a light.
 

493
00:25:57 --> 00:26:01
And then I'm going
to ring a tone.

494
00:26:01 --> 00:26:02
I'm going to do that
a bunch of times.

495
00:26:02 --> 00:26:04
Light, tone, light,
tone, light, tone.

496
00:26:04 --> 00:26:07
Then I'm going to play
the light alone, or I

497
00:26:07 --> 00:26:08
play the tone alone.
 

498
00:26:08 --> 00:26:10
Do you salivate?
 

499
00:26:10 --> 00:26:11
No, why should you salivate?
 

500
00:26:11 --> 00:26:14
It's got nothing to
do with validation.

501
00:26:14 --> 00:26:14
Step two.
 

502
00:26:14 --> 00:26:19
 
 

503
00:26:19 --> 00:26:22
Tone, food.
 

504
00:26:22 --> 00:26:26
That's just the classical
conditioning paradigm.

505
00:26:26 --> 00:26:27
Tone, food, tone,
food, tone, food.

506
00:26:27 --> 00:26:30
Play the tone alone.
 

507
00:26:30 --> 00:26:31
Well tone food whoops.
 

508
00:26:31 --> 00:26:34
 
 

509
00:26:34 --> 00:26:35
Saliva.
 

510
00:26:35 --> 00:26:41
So now I'm going to
play the tone alone.

511
00:26:41 --> 00:26:44
And low and behold, we know
I'll get the conditioned

512
00:26:44 --> 00:26:47
responses of salivation here.
 

513
00:26:47 --> 00:26:51
The critical test
is now I play.

514
00:26:51 --> 00:26:54
I show the light in isolation.
 

515
00:26:54 --> 00:26:55
What happens?
 

516
00:26:55 --> 00:27:02
Well what happens is that you
get the conditioned response.

517
00:27:02 --> 00:27:12
The animal will now salivate in
response to the light alone.

518
00:27:12 --> 00:27:14
Why is this interesting?
 

519
00:27:14 --> 00:27:15
Well what's the animal doing?
 

520
00:27:15 --> 00:27:19
Well again we can sort of think
in sort of conscious terms

521
00:27:19 --> 00:27:20
that aren't appropriate.
 

522
00:27:20 --> 00:27:22
The animal is saying, light.
 

523
00:27:22 --> 00:27:24
I know about light.
 

524
00:27:24 --> 00:27:26
Light means tone.
 

525
00:27:26 --> 00:27:29
Ah ha tone, I know about
tone, tone means food.

526
00:27:29 --> 00:27:32
So if tone means food, I
should be like salivating.

527
00:27:32 --> 00:27:34
And if light means tone, I
should be salivating there.

528
00:27:34 --> 00:27:36
So I think I'll just start,
you know, drooling all

529
00:27:36 --> 00:27:38
over the place now.
 

530
00:27:38 --> 00:27:42
What's important about this
is actually you can see

531
00:27:42 --> 00:27:43
sort of graphically here.
 

532
00:27:43 --> 00:27:46
I'm managing to get more
remote from the response.

533
00:27:46 --> 00:27:48
I'm working my way backward.
 

534
00:27:48 --> 00:27:52
Suppose it was the case that at
the end of every one of my

535
00:27:52 --> 00:27:59
lectures around 3:23, I
was to say in summary.

536
00:27:59 --> 00:28:01
And that your habit was to
immediately run out of

537
00:28:01 --> 00:28:05
class thereafter and
get yourself a snack.

538
00:28:05 --> 00:28:11
You might discover that as
I said in summary that

539
00:28:11 --> 00:28:12
somebody walked in.
 

540
00:28:12 --> 00:28:13
AUDIENCE: LAUGHTER]
 

541
00:28:13 --> 00:28:17
PROFESSOR: You might discover
that as I said in summary that

542
00:28:17 --> 00:28:22
you start to feel hungry,
because of a learned

543
00:28:22 --> 00:28:23
association.
 

544
00:28:23 --> 00:28:26
It might never occurred to you
until you examined this, why

545
00:28:26 --> 00:28:29
you feel hungry when
I said in summary.

546
00:28:29 --> 00:28:32
But if we look at this timing
thing, it's not the case that I

547
00:28:32 --> 00:28:36
say in summary, and one second
later I blow M&Ms into your

548
00:28:36 --> 00:28:38
mouth or something like that.
 

549
00:28:38 --> 00:28:42
You've got of somehow explain
how you work backwards from

550
00:28:42 --> 00:28:46
the eventual M&Ms here back
to something that I said.

551
00:28:46 --> 00:28:48
And this is one way to do it.
 

552
00:28:48 --> 00:28:51
You imagine that what
you've got is an automatic

553
00:28:51 --> 00:28:53
association learner.
 

554
00:28:53 --> 00:28:59
Its job is to look for
sort of if a then b

555
00:28:59 --> 00:29:02
situations in the world.
 

556
00:29:02 --> 00:29:06
And that it is capable of
chaining those together.

557
00:29:06 --> 00:29:11
That you can go if a then b, if
b then c, if c then, you know.

558
00:29:11 --> 00:29:14
Oh hey look if this then
this thing way down

559
00:29:14 --> 00:29:15
the line somewhere.
 

560
00:29:15 --> 00:29:18
And if you can start chaining
these simple associations

561
00:29:18 --> 00:29:24
together, all of a sudden
you've got the possibility of

562
00:29:24 --> 00:29:31
having a much more complicated
kind of tool that could

563
00:29:31 --> 00:29:37
potentially explain much more
complicated kinds of behavior.

564
00:29:37 --> 00:29:40
So that's classical
conditioning.

565
00:29:40 --> 00:29:45
Good for explaining or good for
talking about relationships

566
00:29:45 --> 00:29:48
between stimuli
out in the world.

567
00:29:48 --> 00:29:51
 
 

568
00:29:51 --> 00:29:55
Another thing that you would
want to learn about is the

569
00:29:55 --> 00:29:59
relationship between what you
do and its consequences.

570
00:29:59 --> 00:30:04
That is what we talked about a
bit last time when I did the

571
00:30:04 --> 00:30:07
Thorndike puzzle box example.
 

572
00:30:07 --> 00:30:16
You know Thorndike's cat
was not so interested in

573
00:30:16 --> 00:30:18
learning about relations
of stimuli in the world.

574
00:30:18 --> 00:30:22
What it was interested in
learning is if I do this

575
00:30:22 --> 00:30:25
then I get that fish.
 

576
00:30:25 --> 00:30:30
And that's the core of
operant conditioning.

577
00:30:30 --> 00:30:38
Now I took this course once
upon a time, but I took

578
00:30:38 --> 00:30:40
it at Princeton where
I was an undergrad.

579
00:30:40 --> 00:30:46
And that allows me to
encapsulate the basic

580
00:30:46 --> 00:30:50
difference between a Princeton
undergraduate education -- look

581
00:30:50 --> 00:30:53
at that, there's a guy wearing
a Princeton sweatshirt

582
00:30:53 --> 00:30:55
right now.
 

583
00:30:55 --> 00:30:57
Don't cover it up.
 

584
00:30:57 --> 00:31:01
You know unless she's no
longer talking to you.

585
00:31:01 --> 00:31:04
That's extinction
curve over there.

586
00:31:04 --> 00:31:07
Anyway where were we.
 

587
00:31:07 --> 00:31:07
Oh yes.
 

588
00:31:07 --> 00:31:10
The difference between a
Princeton education and a MIT

589
00:31:10 --> 00:31:15
education is that at Princeton
this course satisfied

590
00:31:15 --> 00:31:17
the science requirement.
 

591
00:31:17 --> 00:31:20
AUDIENCE: [LAUGHTER]
 

592
00:31:20 --> 00:31:21
PROFESSOR: You laugh.
 

593
00:31:21 --> 00:31:25
But if I was to give this
lecture at Princeton and say at

594
00:31:25 --> 00:31:29
MIT, this course satisfies a
sort of, you know, humanities

595
00:31:29 --> 00:31:32
literature kind of requirement,
they'd all laughs.

596
00:31:32 --> 00:31:39
So the main difference between
the courses was not in what,

597
00:31:39 --> 00:31:41
you know, the guy standing up
front did in the way of

598
00:31:41 --> 00:31:46
lecturing, but here of course
to make it a Hass course

599
00:31:46 --> 00:31:47
you do a lot of writing.
 

600
00:31:47 --> 00:31:52
And at Princeton it
was a lab course.

601
00:31:52 --> 00:31:54
And it's a great
pity in some way.

602
00:31:54 --> 00:31:56
It's excellent that you are
off writing, but it's a

603
00:31:56 --> 00:31:58
great pity not to have a
lab component to this.

604
00:31:58 --> 00:32:04
Because this operant
conditioning thing is a

605
00:32:04 --> 00:32:07
lot cooler if you got
a pigeon of your own.

606
00:32:07 --> 00:32:11
Which is what we had.
 

607
00:32:11 --> 00:32:14
You don't end up with a
salivating dog of your own.

608
00:32:14 --> 00:32:17
But pigeons are OK.
 

609
00:32:17 --> 00:32:27
So the version of an operant
conditioning experiment that

610
00:32:27 --> 00:32:29
people tend to know something
about before they come

611
00:32:29 --> 00:32:31
into a courses, is a
so called Skinner box.

612
00:32:31 --> 00:32:35
Which is, you know, a pigeon.
 

613
00:32:35 --> 00:32:38
 
 

614
00:32:38 --> 00:32:40
AUDIENCE: [LAUGHTER]
 

615
00:32:40 --> 00:32:42
He can be a rat
too if you want.

616
00:32:42 --> 00:32:48
It's just a matter of where
you put the whiskers.

617
00:32:48 --> 00:32:52
Anyway the idea of a Skinner
boxes was you'd have an

618
00:32:52 --> 00:32:56
environment that could really
control the options available

619
00:32:56 --> 00:32:58
to the pigeon or the rat.
 

620
00:32:58 --> 00:33:07
And what you'd have in a pigeon
Skinner box is a box that had

621
00:33:07 --> 00:33:11
nothing much in it except for a
key here that is hooked up to a

622
00:33:11 --> 00:33:12
little micro switch
or something.

623
00:33:12 --> 00:33:14
And that the animal
could peck at.

624
00:33:14 --> 00:33:17
And we could record the pecks.
 

625
00:33:17 --> 00:33:20
And a little bin down
here where the bird

626
00:33:20 --> 00:33:22
seed could show up.
 

627
00:33:22 --> 00:33:25
And stick a pigeon in there,
and this is a hungry pigeon

628
00:33:25 --> 00:33:27
because you haven't fed it.
 

629
00:33:27 --> 00:33:31
And so yes, a
bummed out pigeon.

630
00:33:31 --> 00:33:39
But if he pecks there in the
right ways we'll feed him.

631
00:33:39 --> 00:33:44
So this is simply the somewhat
higher tech version of

632
00:33:44 --> 00:33:47
Thorndike's puzzle box
you'll recognize.

633
00:33:47 --> 00:33:52
The pigeon has to figure
out to peck a the key.

634
00:33:52 --> 00:33:56
If you just stick a pigeon in
a Skinner box, even a hungry

635
00:33:56 --> 00:33:58
pigeon, the pigeon
just sits there.

636
00:33:58 --> 00:34:00
I mean, you know, does stuff.
 

637
00:34:00 --> 00:34:02
But the pigeon doesn't
immediately say, hey I've taken

638
00:34:02 --> 00:34:04
intro psych I know about this,
I'm going to go and

639
00:34:04 --> 00:34:07
peck that thing.
 

640
00:34:07 --> 00:34:11
You have to do what's known
as shaping its behavior.

641
00:34:11 --> 00:34:13
The first thing you have to do
that's important, you have to

642
00:34:13 --> 00:34:16
get your pigeon from the
basement to the fourth

643
00:34:16 --> 00:34:19
floor, where the lab was.
 

644
00:34:19 --> 00:34:21
And should you every need to do
that, the important thing to

645
00:34:21 --> 00:34:25
know is that plastic juice
containers are really good.

646
00:34:25 --> 00:34:27
You take the pigeon sticking
it head first in the

647
00:34:27 --> 00:34:29
plastic juice container.
 

648
00:34:29 --> 00:34:30
Not with the juice in it.
 

649
00:34:30 --> 00:34:31
AUDIENCE: [LAUGHTER]
 

650
00:34:31 --> 00:34:36
PROFESSOR: And then the pigeon
is actually quite calm

651
00:34:36 --> 00:34:38
under those circumstances.
 

652
00:34:38 --> 00:34:43
If you decide how tough can a
pigeon BE, and you don't need

653
00:34:43 --> 00:34:46
to do that because you can't
find your juice container

654
00:34:46 --> 00:34:47
anyway, and you grab your
pigeon because you're

655
00:34:47 --> 00:34:48
bigger than the pigeon.
 

656
00:34:48 --> 00:34:51
Try to carry it upstairs, what
you end up with is lose pigeons

657
00:34:51 --> 00:34:54
unlike the second
and third floor.

658
00:34:54 --> 00:34:57
So things that you're going to
miss out on just because you're

659
00:34:57 --> 00:34:59
writing papers instead.
 

660
00:34:59 --> 00:35:03
What you do once you stick that
pigeon in the Skinner box?

661
00:35:03 --> 00:35:06
If you just sit there, it's
going to be a long day.

662
00:35:06 --> 00:35:09
What you have to do is to
shape the pigeon's behavior.

663
00:35:09 --> 00:35:14
Probably says shaping
somewhere on the handout.

664
00:35:14 --> 00:35:19
Yeah use the law of effect to
shape the animal it says there.

665
00:35:19 --> 00:35:22
That doesn't mean like grr.
 

666
00:35:22 --> 00:35:28
What it means is that the
pigeon's just sitting there

667
00:35:28 --> 00:35:32
moving around doing pigeony
like things, feeling hungry.

668
00:35:32 --> 00:35:36
And the pigeon turns towards
the wall that's got the key on.

669
00:35:36 --> 00:35:39
And you push a little
button that gives him

670
00:35:39 --> 00:35:40
a little bird seed.
 

671
00:35:40 --> 00:35:44
He goes and eats the bird
seed, and goes back to

672
00:35:44 --> 00:35:45
doing pigeony things.
 

673
00:35:45 --> 00:35:47
But now the law of
effect's working.

674
00:35:47 --> 00:35:50
That bird seed is a
positive reinforcer right.

675
00:35:50 --> 00:35:54
So the chunk of the bird's
brain that is doing this

676
00:35:54 --> 00:35:57
association learning between
action and its consequences

677
00:35:57 --> 00:36:00
saying, birds seed that
was good bird seed.

678
00:36:00 --> 00:36:01
I like the bird seed.
 

679
00:36:01 --> 00:36:03
What do I do to get
more bird seed.

680
00:36:03 --> 00:36:05
Well it's not saying that
explicitly, but it's

681
00:36:05 --> 00:36:09
saying we'll do whatever
we were just doing.

682
00:36:09 --> 00:36:10
OK.
 

683
00:36:10 --> 00:36:11
Well now you don't want him
 

684
00:36:11 --> 00:36:12
to just be looking at the wall.
 

685
00:36:12 --> 00:36:16
So now after a couple of rounds
of that, you say, OK bird you

686
00:36:16 --> 00:36:19
only get the bird seed if you
move a little closer

687
00:36:19 --> 00:36:20
to the wall.
 

688
00:36:20 --> 00:36:23
The bird moves closer
to the wall all right.

689
00:36:23 --> 00:36:26
All right now you only get it
if you're looking at the key.

690
00:36:26 --> 00:36:28
Now you're only getting it
if you're right up there.

691
00:36:28 --> 00:36:34
And eventually you get the bird
pecking at the key, and then

692
00:36:34 --> 00:36:37
you can go off and do other
cool experiments from there.

693
00:36:37 --> 00:36:41
But you have to shape the
behavior in the way you want.

694
00:36:41 --> 00:36:42
And the key business.
 

695
00:36:42 --> 00:36:43
I mean there's a key.
 

696
00:36:43 --> 00:36:44
There's the bird food.
 

697
00:36:44 --> 00:36:49
That's somewhat arbitrary.
 

698
00:36:49 --> 00:36:51
It's just, you know, what
Skinner wanted to use to

699
00:36:51 --> 00:36:53
control the birds, you know, to
make it possible to measure

700
00:36:53 --> 00:36:55
the bird's behavior.
 

701
00:36:55 --> 00:36:58
We had a very clever pigeon
when I did this lab.

702
00:36:58 --> 00:37:01
And we got through the stuff
that was in the lab book

703
00:37:01 --> 00:37:02
like really quickly.
 

704
00:37:02 --> 00:37:06
Probably because the pigeon had
been the pigeon for a half

705
00:37:06 --> 00:37:08
a dozen other, you know.
 

706
00:37:08 --> 00:37:11
Oh yeah, yeah I'm supposed to
fake it here and claim I don't

707
00:37:11 --> 00:37:11
know what's going on here.
 

708
00:37:11 --> 00:37:12
All right yeah.
 

709
00:37:12 --> 00:37:15
Oh good we're shape
now give me the food.

710
00:37:15 --> 00:37:15
AUDIENCE: [LAUGHTER]
 

711
00:37:15 --> 00:37:16
PROFESSOR: But anyway.
 

712
00:37:16 --> 00:37:18
We still had some time.
 

713
00:37:18 --> 00:37:21
We thought well we'll
mess with this pigeon.

714
00:37:21 --> 00:37:25
And so we started reinforcing
the pigeon for turns.

715
00:37:25 --> 00:37:27
So, you know, no bird
seed unless you make

716
00:37:27 --> 00:37:30
a quarter turn, OK.
 

717
00:37:30 --> 00:37:32
We eventually had the
bird making three

718
00:37:32 --> 00:37:34
full turns for each.
 

719
00:37:34 --> 00:37:38
And that sort of staggering
over to the bird see.

720
00:37:38 --> 00:37:38
AUDIENCE: [LAUGHTER]
 

721
00:37:38 --> 00:37:43
PROFESSOR: But we had this
great ballet dancing pigeons.

722
00:37:43 --> 00:37:47
And that's in fact the basis of
trained animal acts that you

723
00:37:47 --> 00:37:50
may have seen at
various places.

724
00:37:50 --> 00:37:56
And I should advertised that
it's not confined to pigeons

725
00:37:56 --> 00:37:56
and things like that.
 

726
00:37:56 --> 00:38:00
You can perfectly well
condition human beings.

727
00:38:00 --> 00:38:04
I already think I've used in
the last lecture the example

728
00:38:04 --> 00:38:07
of jokes as a case of
behavior getting shaped.

729
00:38:07 --> 00:38:10
Jokes that get rewarded
with laughs get repeated.

730
00:38:10 --> 00:38:14
Jokes that get rewarded with
smacks don't get repeated.

731
00:38:14 --> 00:38:18
But again going back to my
intro psych class, I should

732
00:38:18 --> 00:38:22
tell you that it is possible
to condition the professor.

733
00:38:22 --> 00:38:24
What you need to do is
figure out well what

734
00:38:24 --> 00:38:27
reinforces professors.
 

735
00:38:27 --> 00:38:31
The answer, at least in this
sort of a setting, is students

736
00:38:31 --> 00:38:35
who look kind of interested,
and smile and write notes.

737
00:38:35 --> 00:38:38
At least that's what my a
professor, who was in fact an

738
00:38:38 --> 00:38:40
important learning theorist in
his own right, that's

739
00:38:40 --> 00:38:41
what he told us.
 

740
00:38:41 --> 00:38:46
He said it's reinforcing if
you're looking like you're

741
00:38:46 --> 00:38:47
interested and you're
writing notes.

742
00:38:47 --> 00:38:53
And so you should, he told us,
be able to pick a behavior that

743
00:38:53 --> 00:38:59
the professor is doing, and
reinforce it by smiling and

744
00:38:59 --> 00:39:01
writing notes at the
appropriate time.

745
00:39:01 --> 00:39:04
So, you know, I was going
to be a psych major.

746
00:39:04 --> 00:39:08
I did what I was told I
was sitting about there.

747
00:39:08 --> 00:39:12
My friend and I every time,
the next psych lecture, every

748
00:39:12 --> 00:39:15
time Professor [? Cayman ?]
 

749
00:39:15 --> 00:39:22
moved to the left, we
smiled and took notes.

750
00:39:22 --> 00:39:27
And by the end of lecture
he was most of the

751
00:39:27 --> 00:39:28
way out the door.
 

752
00:39:28 --> 00:39:29
AUDIENCE: [LAUGHTER]
 

753
00:39:29 --> 00:39:30
PROFESSOR: It was great.
 

754
00:39:30 --> 00:39:34
Now as evidence for the notion
that this is quite unconscious

755
00:39:34 --> 00:39:37
and reflexive kind of learning,
we told him about

756
00:39:37 --> 00:39:38
it afterwards.
 

757
00:39:38 --> 00:39:41
And he absolutely believed
that it was possible that

758
00:39:41 --> 00:39:44
it happened, and absolutely
reported no awareness

759
00:39:44 --> 00:39:46
that it had happened.
 

760
00:39:46 --> 00:39:48
So you're welcome to try this.
 

761
00:39:48 --> 00:39:50
Your even welcome
to try it here.

762
00:39:50 --> 00:39:55
Let me tell you that I have
made this offer in years past,

763
00:39:55 --> 00:39:57
and nobody has reported
a great success.

764
00:39:57 --> 00:40:01
But the great failure was when
one recitation section got

765
00:40:01 --> 00:40:04
together and decided their
strength in numbers.

766
00:40:04 --> 00:40:07
But it turns out that if
everybody in a recitation sits

767
00:40:07 --> 00:40:10
-- I still remember they were
all sitting there -- if you all

768
00:40:10 --> 00:40:13
sit in one place, and I don't
remember what I was

769
00:40:13 --> 00:40:14
supposed to do.
 

770
00:40:14 --> 00:40:16
But when I did it, they
all went [INAUDIBLE].

771
00:40:16 --> 00:40:19
 
 

772
00:40:19 --> 00:40:22
I kind of cracked up, but
I don't believe I learned

773
00:40:22 --> 00:40:23
anything particular.
 

774
00:40:23 --> 00:40:26
AUDIENCE: [LAUGHTER]
 

775
00:40:26 --> 00:40:29
PROFESSOR: But it can sit out
there as a challenge for you.

776
00:40:29 --> 00:40:33
If anybody gets it to work
on, you know, the math

777
00:40:33 --> 00:40:35
guy or something.
 

778
00:40:35 --> 00:40:37
Is Arthur Mattuck
still teaching?

779
00:40:37 --> 00:40:37
AUDIENCE: No.
 

780
00:40:37 --> 00:40:39
PROFESSOR: 18, O whatever?
 

781
00:40:39 --> 00:40:41
AUDIENCE: [UNINTELLIGIBLE]
 

782
00:40:41 --> 00:40:41
PROFESSOR: 1803 in the spring.
 

783
00:40:41 --> 00:40:43
Oh that's too long to wait.
 

784
00:40:43 --> 00:40:45
I've never been sure Mattuck
will notice anyway.

785
00:40:45 --> 00:40:47
But that's a separate issue.
 

786
00:40:47 --> 00:40:48
AUDIENCE: [LAUGHTER]
 

787
00:40:48 --> 00:40:51
PROFESSOR: Anyway if anybody
succeeds in conditioning there

788
00:40:51 --> 00:40:53
professor do let me know.
 

789
00:40:53 --> 00:40:58
 
 

790
00:40:58 --> 00:41:02
The points here is that this is
a form of learning that does

791
00:41:02 --> 00:41:13
apply in the human world as
well as in the pigeon world.

792
00:41:13 --> 00:41:16
Once you've got that animal
shaped to do what you want, you

793
00:41:16 --> 00:41:21
can then start studying the
rules that govern

794
00:41:21 --> 00:41:22
that behavior.
 

795
00:41:22 --> 00:41:24
Let me say a word about
schedules of reinforcements,

796
00:41:24 --> 00:41:26
since I see that it's
there on the handout.

797
00:41:26 --> 00:41:28
Actually it's doesn't say
schedules, oh it does.

798
00:41:28 --> 00:41:32
Fix and variable ratio and
fixed and variable interval

799
00:41:32 --> 00:41:33
schedules reinforcement.
 

800
00:41:33 --> 00:41:36
What that means is if you're
reinforcing the pigeon every

801
00:41:36 --> 00:41:42
time he pecks, he's on a
fixed ratio 1 schedule.

802
00:41:42 --> 00:41:46
 
 

803
00:41:46 --> 00:41:52
You shape up the behavior,
and at some point the

804
00:41:52 --> 00:41:54
pigeon is trained.
 

805
00:41:54 --> 00:41:57
And if you're reinforcing
him every time he makes a

806
00:41:57 --> 00:42:00
peck, he will emit pecks.
 

807
00:42:00 --> 00:42:02
So let's call this
the peck rate.

808
00:42:02 --> 00:42:05
 
 

809
00:42:05 --> 00:42:09
He'll emit pecks at something
some asintotic rate, not

810
00:42:09 --> 00:42:13
unlike the salivation
thing over there.

811
00:42:13 --> 00:42:15
Actually I want to save
myself a little space here.

812
00:42:15 --> 00:42:17
So let's only go this far.
 

813
00:42:17 --> 00:42:23
So an asintotic rate
of responding.

814
00:42:23 --> 00:42:28
Now suppose that you
change the rule.

815
00:42:28 --> 00:42:31
Suppose you say, well you can
say it to the pigeon because

816
00:42:31 --> 00:42:34
the pigeon won't understand
you, say to the pigeon, OK

817
00:42:34 --> 00:42:38
now you get reinforced
only every 10th peck.

818
00:42:38 --> 00:42:42
What's the pigeon going
to do, do you think?

819
00:42:42 --> 00:42:44
How's the behavior
going to change?

820
00:42:44 --> 00:42:47
AUDIENCE: [INAUDIBLE]
 

821
00:42:47 --> 00:42:49
PROFESSOR: It's going to
go squiggly, apparently.

822
00:42:49 --> 00:42:51
 
 

823
00:42:51 --> 00:42:56
No, every 10th peck,
not every 10 seconds.

824
00:42:56 --> 00:42:58
I'm interpreting your
squiggly as the answer

825
00:42:58 --> 00:43:00
to a later question.
 

826
00:43:00 --> 00:43:03
So save the squiggle,
it'll be good.

827
00:43:03 --> 00:43:05
AUDIENCE: Peck faster.
 

828
00:43:05 --> 00:43:06
PROFESSOR: Peck faster.
 

829
00:43:06 --> 00:43:10
Whoever said that,
that sounds good.

830
00:43:10 --> 00:43:12
Well if you've gotta work
harder to get the same

831
00:43:12 --> 00:43:18
reward, you're going to
end up pecking faster.

832
00:43:18 --> 00:43:18
Right?
 

833
00:43:18 --> 00:43:24
So if you sort of think about
it in terms of your once upon

834
00:43:24 --> 00:43:30
a time, you could get
an A in whatever.

835
00:43:30 --> 00:43:32
You know writing.
 

836
00:43:32 --> 00:43:36
For writing like my eight year
old wrote this morning, seven

837
00:43:36 --> 00:43:41
sentences of at least
six words each.

838
00:43:41 --> 00:43:44
Took him all of breakfast
to do that, because

839
00:43:44 --> 00:43:45
he'd lost his book.
 

840
00:43:45 --> 00:43:46
An anyways it's a long story.
 

841
00:43:46 --> 00:43:51
But, you know, so he emitted
his writing behavior to

842
00:43:51 --> 00:43:53
relatively low rate.
 

843
00:43:53 --> 00:43:57
If you emit your writing
behavior at that rate, you

844
00:43:57 --> 00:43:59
guys are going to be in
serious trouble, right?

845
00:43:59 --> 00:44:03
To get the same reward, you now
have to produce more work.

846
00:44:03 --> 00:44:07
And as a result you'll
crank up the level.

847
00:44:07 --> 00:44:08
So that would be a fix ratio.
 

848
00:44:08 --> 00:44:11
What I was describing every
10 pecks, that would be

849
00:44:11 --> 00:44:12
a fixed ratio of 10.
 

850
00:44:12 --> 00:44:18
A variable ratio is on
average every 10 pecks

851
00:44:18 --> 00:44:20
you get reinforced.
 

852
00:44:20 --> 00:44:27
And that actually produces even
higher levels of responding.

853
00:44:27 --> 00:44:31
Now what the squiggle was the
answer to is what happens if

854
00:44:31 --> 00:44:34
instead of a ratio schedule,
you have an interval schedule?

855
00:44:34 --> 00:44:38
Suppose you reinforced the
behavior only for the first

856
00:44:38 --> 00:44:42
peck after every minute?
 

857
00:44:42 --> 00:44:44
Make a different
graph over here.

858
00:44:44 --> 00:44:49
So a bird can peck
it all they want.

859
00:44:49 --> 00:44:55
But only when they peck after
this interval, marker goes by

860
00:44:55 --> 00:44:57
or they're going to
get reinforced.

861
00:44:57 --> 00:44:59
Well I mean a pigeon with a
wrist watch is going to look

862
00:44:59 --> 00:45:01
at his watch and say, hmm.
 

863
00:45:01 --> 00:45:04
Pigeon doesn't have a
wrist watch of course.

864
00:45:04 --> 00:45:06
But they do have an
ability to estimate time.

865
00:45:06 --> 00:45:09
And so what they do, you know,
immediately after getting

866
00:45:09 --> 00:45:11
rewarded, they just
sort of sit there.

867
00:45:11 --> 00:45:13
So this is again going to
be rates of response.

868
00:45:13 --> 00:45:18
 
 

869
00:45:18 --> 00:45:22
They just sit there, and then
they say, it's like a minute

870
00:45:22 --> 00:45:27
gone yet nothing happened.
 

871
00:45:27 --> 00:45:29
Got to be now.
 

872
00:45:29 --> 00:45:30
No, no, it's still not.
 

873
00:45:30 --> 00:45:34
 
 

874
00:45:34 --> 00:45:36
Oh it's coming.
 

875
00:45:36 --> 00:45:36
No, no no.
 

876
00:45:36 --> 00:45:37
Boom.
 

877
00:45:37 --> 00:45:38
AUDIENCE: [LAUGHTER]
 

878
00:45:38 --> 00:45:41
PROFESSOR: And you get this
very scallopy behavior.

879
00:45:41 --> 00:45:45
Now if you don't think
that has anything to do

880
00:45:45 --> 00:45:48
with human behavior.
 

881
00:45:48 --> 00:45:50
Well first of all you
can ask yourself about

882
00:45:50 --> 00:45:53
your writing output.
 

883
00:45:53 --> 00:45:57
So for instance there's
a paper do on a sort

884
00:45:57 --> 00:45:59
of a regular interval.
 

885
00:45:59 --> 00:46:04
AUDIENCE: [LAUGHTER]
 

886
00:46:04 --> 00:46:09
PROFESSOR: A smart pigeon
would be emitting words

887
00:46:09 --> 00:46:12
at about that rate right?
 

888
00:46:12 --> 00:46:14
Well actually I did meet
a student who claimed

889
00:46:14 --> 00:46:15
he was doing that.
 

890
00:46:15 --> 00:46:17
More power to him.
 

891
00:46:17 --> 00:46:22
But I think most of view,
you know, it's do friday?

892
00:46:22 --> 00:46:28
I think this point here is like
Friday 4:00 pm or something.

893
00:46:28 --> 00:46:31
Anyway.
 

894
00:46:31 --> 00:46:35
Or you come to my house where
if you're an eight year old

895
00:46:35 --> 00:46:37
you get your allowance
on the weekend, right.

896
00:46:37 --> 00:46:40
But only if you do your chores.
 

897
00:46:40 --> 00:46:46
So Monday is my kid asking
what chores he should do?

898
00:46:46 --> 00:46:47
No.
 

899
00:46:47 --> 00:46:49
Saturday morning?
 

900
00:46:49 --> 00:46:49
You bet.
 

901
00:46:49 --> 00:46:50
What can I do?
 

902
00:46:50 --> 00:46:50
What can I do?
 

903
00:46:50 --> 00:46:51
You know make my bed.
 

904
00:46:51 --> 00:46:52
I made my bed.
 

905
00:46:52 --> 00:46:54
Can I lick the floor
now, you know.

906
00:46:54 --> 00:46:55
AUDIENCE: [LAUGHTER]
 

907
00:46:55 --> 00:47:01
PROFESSOR: Oh and the downside
of this is one of the reasons

908
00:47:01 --> 00:47:04
for paying people on a Friday.
 

909
00:47:04 --> 00:47:06
Is that if you pay them on
a Thursday, they don't

910
00:47:06 --> 00:47:09
show up on Friday.
 

911
00:47:09 --> 00:47:13
Even though at some level
they're much more likely to

912
00:47:13 --> 00:47:15
be sick on the Friday if you
pay them on the Thursday.

913
00:47:15 --> 00:47:17
So you want to pay them on a
Friday and then they can be

914
00:47:17 --> 00:47:21
sick all weekend on
their own time.

915
00:47:21 --> 00:47:28
So that's what we mean by
schedules of reinforcement.

916
00:47:28 --> 00:47:31
This is also, operant
conditioning, is also a very

917
00:47:31 --> 00:47:33
rule governed behavior.
 

918
00:47:33 --> 00:47:37
It is governed by rules that
are in many ways similar to

919
00:47:37 --> 00:47:39
the rules for classical
conditioning.

920
00:47:39 --> 00:47:43
 
 

921
00:47:43 --> 00:47:44
Let's just do this
one as an example.

922
00:47:44 --> 00:47:47
It's probably later
on in the handout.

923
00:47:47 --> 00:47:52
So I reward little all old
every time he pecks the key.

924
00:47:52 --> 00:47:55
Now I stop rewarding and what
happens to the behavior?

925
00:47:55 --> 00:47:59
 
 

926
00:47:59 --> 00:48:02
I take that boom to mean,
yeah, it extinguishes.

927
00:48:02 --> 00:48:07
If I give him a break, I'll get
some spontaneous recovery and

928
00:48:07 --> 00:48:08
it'll extinguished again.
 

929
00:48:08 --> 00:48:11
Just like you'll get in
classical conditioning.

930
00:48:11 --> 00:48:13
What happens in these?
 

931
00:48:13 --> 00:48:17
How does the extinction compare
in a schedule like this

932
00:48:17 --> 00:48:20
to a schedule like this?
 

933
00:48:20 --> 00:48:21
How about a hand?
 

934
00:48:21 --> 00:48:24
I'm hearing good
mother mutterings but

935
00:48:24 --> 00:48:25
nobody has a hand.
 

936
00:48:25 --> 00:48:27
 
 

937
00:48:27 --> 00:48:28
So slower.
 

938
00:48:28 --> 00:48:30
I heard some more
slower mutterings.

939
00:48:30 --> 00:48:32
It's slower.
 

940
00:48:32 --> 00:48:36
If that's not intuitively
obvious, ask yourself about the

941
00:48:36 --> 00:48:41
difference between a slot
machine and a Coke

942
00:48:41 --> 00:48:44
machine, right?
 

943
00:48:44 --> 00:48:48
You put a buck in the coke
machine, you get a coke right?

944
00:48:48 --> 00:48:50
You put a buck in the coke
machine, you don't get a coke.

945
00:48:50 --> 00:48:52
How many more bucks
do you put in?

946
00:48:52 --> 00:48:54
AUDIENCE: [LAUGHTER]
 

947
00:48:54 --> 00:48:55
PROFESSOR: Well all right.
 

948
00:48:55 --> 00:48:56
So you're not too dumb.
 

949
00:48:56 --> 00:48:57
That's good.
 

950
00:48:57 --> 00:48:57
OK.
 

951
00:48:57 --> 00:49:02
Now you go to Las Vegas you put
a buck in the slot machine and

952
00:49:02 --> 00:49:03
you don't get anything out.
 

953
00:49:03 --> 00:49:05
What do you do?
 

954
00:49:05 --> 00:49:07
Put another buck in.
 

955
00:49:07 --> 00:49:12
Actually Las Vegas is a
beautiful example of applied

956
00:49:12 --> 00:49:15
conditioning theory.
 

957
00:49:15 --> 00:49:17
We can send everybody
on a class trip.

958
00:49:17 --> 00:49:20
Actually we did that
didn't we, sort of.

959
00:49:20 --> 00:49:24
There's the best seller --
that was more applied math

960
00:49:24 --> 00:49:25
than applied psychology.
 

961
00:49:25 --> 00:49:30
The applied psychology thing is
when a slot machine pays off,

962
00:49:30 --> 00:49:33
traditionally what happens is
the coins come out right.

963
00:49:33 --> 00:49:36
Now often they're electronic,
but they simulated anyway.

964
00:49:36 --> 00:49:39
The coins come out.
 

965
00:49:39 --> 00:49:40
What di the coins hit
when they come out?

966
00:49:40 --> 00:49:42
AUDIENCE: Metal.
 

967
00:49:42 --> 00:49:43
PROFESSOR: Metal.
 

968
00:49:43 --> 00:49:44
Why do they hit metal?
 

969
00:49:44 --> 00:49:45
AUDIENCE: To make a sound.
 

970
00:49:45 --> 00:49:47
PROFESSOR: To make a racket.
 

971
00:49:47 --> 00:49:48
Why do you want to
make a racket?

972
00:49:48 --> 00:49:53
Well that racket itself becomes
positively reinforcing.

973
00:49:53 --> 00:49:57
You want to hear
that sound right.

974
00:49:57 --> 00:50:01
You want to put that buck
in and hear crunch crunch.

975
00:50:01 --> 00:50:03
Oh yeah this, is
very good stuff.

976
00:50:03 --> 00:50:05
Now slot machines.
 

977
00:50:05 --> 00:50:07
I mean you don't go out and
play slot machines a lot,

978
00:50:07 --> 00:50:08
but you see the movies.
 

979
00:50:08 --> 00:50:10
I hope.
 

980
00:50:10 --> 00:50:11
Anyway.
 

981
00:50:11 --> 00:50:16
Slot machines are typically
located in small

982
00:50:16 --> 00:50:18
sound proof booth?
 

983
00:50:18 --> 00:50:19
No.
 

984
00:50:19 --> 00:50:20
Where are they located?
 

985
00:50:20 --> 00:50:21
AUDIENCE: [UNINTELLIGIBLE]
 

986
00:50:21 --> 00:50:24
PROFESSOR: On a giant
floor with thousands

987
00:50:24 --> 00:50:25
of slot machines.
 

988
00:50:25 --> 00:50:27
What does that mean?
 

989
00:50:27 --> 00:50:32
That means when somebody
five blocks over gets a

990
00:50:32 --> 00:50:36
pay off, what do you hear?
 

991
00:50:36 --> 00:50:36
You hear the sound.
 

992
00:50:36 --> 00:50:40
 
 

993
00:50:40 --> 00:50:43
You've been conditioned to
find that reinforcing.

994
00:50:43 --> 00:50:44
And you are so dumb.
 

995
00:50:44 --> 00:50:45
AUDIENCE: [LAUGHTER]
 

996
00:50:45 --> 00:50:48
PROFESSOR: They know that
you are willing to being

997
00:50:48 --> 00:50:51
reinforced by somebody
else getting paid off.

998
00:50:51 --> 00:50:54
And you say oh man there's
money happening here let me put

999
00:50:54 --> 00:50:57
some more into this thing.
 

1000
00:50:57 --> 00:50:57
Anyway.
 

1001
00:50:57 --> 00:51:02
Lovely applied psychology
in, you know, the worst

1002
00:51:02 --> 00:51:04
sense of the word.
 

1003
00:51:04 --> 00:51:05
Well maybe not the worst.
 

1004
00:51:05 --> 00:51:06
I can think of a few others.
 

1005
00:51:06 --> 00:51:10
But while I'm thinking of
a few other worse ones,

1006
00:51:10 --> 00:51:13
take a minute to stretch.
 

1007
00:51:13 --> 00:51:16
Reinforce your neighbor.
 

1008
00:51:16 --> 00:51:59
Wake up your neighbor
if that's necessary.

1009
00:51:59 --> 00:52:27
[SIDE___CONVERSA
TION___WITH___STUDENT]

1010
00:52:27 --> 00:52:28
PROFESSOR: OK.
 

1011
00:52:28 --> 00:52:36
 
 

1012
00:52:36 --> 00:52:39
There is a sense in which
you are a general purpose

1013
00:52:39 --> 00:52:43
association learner.
 

1014
00:52:43 --> 00:52:47
And have mechanisms in your
brain that are basically

1015
00:52:47 --> 00:52:51
saying, you know, teach me
about the associations

1016
00:52:51 --> 00:52:51
between stimuli
 

1017
00:52:51 --> 00:52:55
or the association between my
acts and their consequences.

1018
00:52:55 --> 00:52:58
And I can be fairly broad
minded about what those

1019
00:52:58 --> 00:53:00
stimuli are, or what
those acts might be.

1020
00:53:00 --> 00:53:04
And then there are special
purpose versions of these

1021
00:53:04 --> 00:53:09
things designed for
very particular tasks.

1022
00:53:09 --> 00:53:12
Let me fish around for
what's perhaps the

1023
00:53:12 --> 00:53:14
best example of this.
 

1024
00:53:14 --> 00:53:20
Is there anybody here who can
think of in their life of food

1025
00:53:20 --> 00:53:26
that they used to love, but
they now hate or won't touch,

1026
00:53:26 --> 00:53:29
and they know exactly why?
 

1027
00:53:29 --> 00:53:31
Yeah out there in
the cheap seats?

1028
00:53:31 --> 00:53:31
AUDIENCE: I ate a
chocolate frosted cupcake

1029
00:53:31 --> 00:53:31
when I was little.
 

1030
00:53:31 --> 00:53:31
And I went home and I got
the flu, and I don't

1031
00:53:31 --> 00:53:40
eat it anymore.
 

1032
00:53:40 --> 00:53:42
PROFESSOR: Perfect Gee.
 

1033
00:53:42 --> 00:53:45
First try no weird
examples this year.

1034
00:53:45 --> 00:53:47
Now you've got the flu.
 

1035
00:53:47 --> 00:53:49
let's get a little
graphic here.

1036
00:53:49 --> 00:53:51
You've got the flu
and what happened?

1037
00:53:51 --> 00:53:52
AUDIENCE: Vomit a lot.
 

1038
00:53:52 --> 00:53:54
PROFESSOR: Thank you that's
what we needed to know.

1039
00:53:54 --> 00:53:55
AUDIENCE: [LAUGHTER]
 

1040
00:53:55 --> 00:53:59
PROFESSOR: And not only that,
we also need to know do you

1041
00:53:59 --> 00:54:02
believe that the chocolate
cupcake gave you the flu?

1042
00:54:02 --> 00:54:02
AUDIENCE: Not anymore.
 

1043
00:54:02 --> 00:54:07
But I use to definitely
think that.

1044
00:54:07 --> 00:54:07
[LAUGHTER]
 

1045
00:54:07 --> 00:54:08
PROFESSOR: Yeah?
 

1046
00:54:08 --> 00:54:10
AUDIENCE: Alright, when I
was in kindergarten --

1047
00:54:10 --> 00:54:12
PROFESSOR: He doesn't care that
I already have a good example.

1048
00:54:12 --> 00:54:13
He wants to give me
another cool example.

1049
00:54:13 --> 00:54:13
All right.
 

1050
00:54:13 --> 00:54:15
All right.
 

1051
00:54:15 --> 00:54:15
All right.
 

1052
00:54:15 --> 00:54:17
All right.
 

1053
00:54:17 --> 00:54:18
It's not as good?
 

1054
00:54:18 --> 00:54:22
AUDIENCE: When I was in
kindergarten the first day of

1055
00:54:22 --> 00:54:29
school or the first week I ate
lunch in the cafeteria and

1056
00:54:29 --> 00:54:35
didn't eat a school lunch
again five, six years.

1057
00:54:35 --> 00:54:37
PROFESSOR: This is why
he's slim to this day.

1058
00:54:37 --> 00:54:43
AUDIENCE: [LAUGHTER]
 

1059
00:54:43 --> 00:54:45
PROFESSOR: Actually we'll come
back to that example, it's

1060
00:54:45 --> 00:54:51
sort of a useful function to.
 

1061
00:54:51 --> 00:54:53
 
 

1062
00:54:53 --> 00:54:57
So she ate the cupcake,
she got sick.

1063
00:54:57 --> 00:54:59
She knows at some level
it wasn't the cupcake

1064
00:54:59 --> 00:55:00
that made her sick.
 

1065
00:55:00 --> 00:55:05
But there is a chunk of brain
that says, I got sick.

1066
00:55:05 --> 00:55:07
I ate cupcake.
 

1067
00:55:07 --> 00:55:11
Shortly thereafter got sick.
 

1068
00:55:11 --> 00:55:13
Don't want to eat
cupcake anymore.

1069
00:55:13 --> 00:55:18
This is a chunk of your brain
that is quite immune, or is

1070
00:55:18 --> 00:55:22
actually very hard for you to
talk to and say, wait a second

1071
00:55:22 --> 00:55:23
we used to love cupcakes.
 

1072
00:55:23 --> 00:55:24
Cupcakes are really good.
 

1073
00:55:24 --> 00:55:26
It doesn't want to
hear about this.

1074
00:55:26 --> 00:55:28
It says, you ate
it, you got sick.

1075
00:55:28 --> 00:55:31
And this is a form of
association learning.

1076
00:55:31 --> 00:55:36
Well the original version of
this was discovered by some rat

1077
00:55:36 --> 00:55:38
who ate a cupcake and threw up.
 

1078
00:55:38 --> 00:55:43
But in the learning literature,
it was studied by a

1079
00:55:43 --> 00:55:44
guy named Garcia.
 

1080
00:55:44 --> 00:55:47
So it's sometimes goes by the
name of the Garcia effect, what

1081
00:55:47 --> 00:55:52
he did was he had rats, in a
sort of a Skinner

1082
00:55:52 --> 00:55:55
box situation.
 

1083
00:55:55 --> 00:55:59
And he gave them a new flavor
of water like spearmint

1084
00:55:59 --> 00:56:00
or something.
 

1085
00:56:00 --> 00:56:03
And then he made the
poor rats sick.

1086
00:56:03 --> 00:56:09
Strapped it to the turntable
of a record player.

1087
00:56:09 --> 00:56:10
AUDIENCE: [LAUGHTER]
 

1088
00:56:10 --> 00:56:13
PROFESSOR: An experiment you
can't do m because you can't

1089
00:56:13 --> 00:56:16
fit the rat into the CD drive.
 

1090
00:56:16 --> 00:56:18
AUDIENCE: [LAUGHTER]
 

1091
00:56:18 --> 00:56:24
PROFESSOR: And then the next
day you give the rat a choice.

1092
00:56:24 --> 00:56:26
You want spearmint water
or you want this other

1093
00:56:26 --> 00:56:27
new weird flavor.
 

1094
00:56:27 --> 00:56:29
And the rat goes to
the other flavor.

1095
00:56:29 --> 00:56:32
Says forget it, you
know, I'm not going for

1096
00:56:32 --> 00:56:34
the spearmint water.
 

1097
00:56:34 --> 00:56:36
But this is a very specific
form of learning.

1098
00:56:36 --> 00:56:38
It sometimes called
taster version, which

1099
00:56:38 --> 00:56:40
is not quite right.
 

1100
00:56:40 --> 00:56:42
If you happen to be a sensation
and perception person.

1101
00:56:42 --> 00:56:46
Taste, as the sensation that
your tongue does for you, is

1102
00:56:46 --> 00:56:49
really restricted to sweet,
sour, salty and bitter.

1103
00:56:49 --> 00:56:54
What you become averse
to is the flavor, most

1104
00:56:54 --> 00:56:55
of which is smell.
 

1105
00:56:55 --> 00:57:00
Or perhaps in human cases, a
larger context like school

1106
00:57:00 --> 00:57:02
lunch or something like that.
 

1107
00:57:02 --> 00:57:05
But there are strong
restrictions on what it is

1108
00:57:05 --> 00:57:08
that you will develop in
a version two, particular

1109
00:57:08 --> 00:57:11
if you're a rat.
 

1110
00:57:11 --> 00:57:14
So the other version of the
experiment Garcia did was

1111
00:57:14 --> 00:57:18
he gave rats a new
bottled of water.

1112
00:57:18 --> 00:57:23
And this one was
flashing lights right.

1113
00:57:23 --> 00:57:25
Water never did that before.
 

1114
00:57:25 --> 00:57:27
Drink the water.
 

1115
00:57:27 --> 00:57:28
Get sick.
 

1116
00:57:28 --> 00:57:31
The next day you got a choice
between the flashing light one,

1117
00:57:31 --> 00:57:33
and the one that's making
clicking noises or

1118
00:57:33 --> 00:57:34
something like that.
 

1119
00:57:34 --> 00:57:36
Rat doesn't care.
 

1120
00:57:36 --> 00:57:40
This little chunk of brain was
not built to figure out that

1121
00:57:40 --> 00:57:42
flashing water is a problem.
 

1122
00:57:42 --> 00:57:49
This is a chunk of brain that
is there to tell you that it

1123
00:57:49 --> 00:57:50
knows some things about food.
 

1124
00:57:50 --> 00:57:55
Foods have flavors, and if you
get sick some chunk of time

1125
00:57:55 --> 00:58:01
after eating, you should avoid
that flavor the next time, and

1126
00:58:01 --> 00:58:02
things associated
with that flavor.

1127
00:58:02 --> 00:58:03
Also the timing.
 

1128
00:58:03 --> 00:58:05
Where did my timing
parameter go?

1129
00:58:05 --> 00:58:06
The timing is different.
 

1130
00:58:06 --> 00:58:10
If you eat something and
get sick instantly,

1131
00:58:10 --> 00:58:11
nothing happens.
 

1132
00:58:11 --> 00:58:12
You don't learn anything.
 

1133
00:58:12 --> 00:58:15
Because this little box knows
it takes a while to get sick.

1134
00:58:15 --> 00:58:18
I think your story was
ate the cupcake, went

1135
00:58:18 --> 00:58:19
home, got the flu.
 

1136
00:58:19 --> 00:58:23
Hours later she's losing it.
 

1137
00:58:23 --> 00:58:26
And this little junk of brain
is working back in time.

1138
00:58:26 --> 00:58:27
It's got its own time window.
 

1139
00:58:27 --> 00:58:30
It's working back in time
saying, what did we

1140
00:58:30 --> 00:58:31
eat a few hours
 

1141
00:58:31 --> 00:58:31
ago?
 

1142
00:58:31 --> 00:58:32
Cupcakes.
 

1143
00:58:32 --> 00:58:34
Don't want to eat the cupcake.
 

1144
00:58:34 --> 00:58:35
Very useful thing right.
 

1145
00:58:35 --> 00:58:38
If you were grazing, you know,
an omnivore kind of animal,

1146
00:58:38 --> 00:58:43
very useful thing to have a
device that says don't eat it

1147
00:58:43 --> 00:58:46
because, you know, last time
you ate it made you sick.

1148
00:58:46 --> 00:58:49
Oh the other thing when we were
doing this tone, food, tone,

1149
00:58:49 --> 00:58:51
food pairing over
and over again.

1150
00:58:51 --> 00:58:53
How many cupcakes did you eat?
 

1151
00:58:53 --> 00:58:54
One cup cake.
 

1152
00:58:54 --> 00:58:57
And you repeated
this lots of time?

1153
00:58:57 --> 00:58:58
No.
 

1154
00:58:58 --> 00:58:58
Right.
 

1155
00:58:58 --> 00:59:03
Because this is a little
association mechanism

1156
00:59:03 --> 00:59:04
designed to keep you alive.
 

1157
00:59:04 --> 00:59:08
And if, you know, eat
the cupcake almost die.

1158
00:59:08 --> 00:59:12
You know, you don't want
a bunch more pairings of

1159
00:59:12 --> 00:59:14
eat the cupcake, almost
died, eat the cupcake.

1160
00:59:14 --> 00:59:17
Oh man that time dead.
 

1161
00:59:17 --> 00:59:19
AUDIENCE: [LAUGHTER]
 

1162
00:59:19 --> 00:59:22
PROFESSOR: So one try of
learning, which by the way when

1163
00:59:22 --> 00:59:25
Garcia discovered it, it was
thought to be impossible.

1164
00:59:25 --> 00:59:29
But it's a special case of an
association learning that

1165
00:59:29 --> 00:59:34
works in a single trial.
 

1166
00:59:34 --> 00:59:36
It has its pitfalls.
 

1167
00:59:36 --> 00:59:40
And in fact it's sort of
cognitive impenetrability.

1168
00:59:40 --> 00:59:45
The fact that it doesn't care
what you know is a problem.

1169
00:59:45 --> 00:59:54
So except for the seniors
here, of course alcohol

1170
00:59:54 --> 00:59:56
has not past your lips.
 

1171
00:59:56 --> 01:00:00
And even the seniors have
not drunk to excess.

1172
01:00:00 --> 01:00:04
So what may be news to you is
that if you drink too much

1173
01:00:04 --> 01:00:06
alcohol you can get sick.
 

1174
01:00:06 --> 01:00:07
AUDIENCE: [LAUGHTER]
 

1175
01:00:07 --> 01:00:08
PROFESSOR: They knew that.
 

1176
01:00:08 --> 01:00:09
OK.
 

1177
01:00:09 --> 01:00:13
Anyway drink too much
alcohol you get sick right.

1178
01:00:13 --> 01:00:16
The question is what do you
develop the aversion to?

1179
01:00:16 --> 01:00:20
A smart system, rather like
this, you know, the writing

1180
01:00:20 --> 01:00:25
example, the smart system would
say I just drank, you know, 12

1181
01:00:25 --> 01:00:27
screwdrivers in a row or
something like that.

1182
01:00:27 --> 01:00:29
No more vodka for me.
 

1183
01:00:29 --> 01:00:35
The problem is that alcohol has
very minimal flavor to it, and

1184
01:00:35 --> 01:00:38
what happens is you drink a
screwdriver -- orange juice

1185
01:00:38 --> 01:00:41
plus vodka or something like
that -- you get violently sick.

1186
01:00:41 --> 01:00:44
I'm not going to touch that
orange juice anymore.

1187
01:00:44 --> 01:00:47
AUDIENCE: [LAUGHTER]
 

1188
01:00:47 --> 01:00:49
PROFESSOR: You know.
 

1189
01:00:49 --> 01:00:52
Pile of beers and
a pile pretzels.

1190
01:00:52 --> 01:00:55
You're violently ill, and
the pretzels just look

1191
01:00:55 --> 01:00:57
miserable to you.
 

1192
01:00:57 --> 01:00:59
so it is a disadvantage.
 

1193
01:00:59 --> 01:01:02
It's called clever that
this thing is out there

1194
01:01:02 --> 01:01:03
to save your life.
 

1195
01:01:03 --> 01:01:08
Unfortunately in civilization,
we have figured out clever

1196
01:01:08 --> 01:01:11
ways to misuse it I'm afraid.
 

1197
01:01:11 --> 01:01:15
 
 

1198
01:01:15 --> 01:01:18
Well since it says example two
of superstitious behavior, let

1199
01:01:18 --> 01:01:20
me say a quick word about
superstitious behavior.

1200
01:01:20 --> 01:01:21
This is the best time to
see what Skinner called

1201
01:01:21 --> 01:01:24
superstitious behavior,
because all you have to

1202
01:01:24 --> 01:01:26
do is watch baseball.
 

1203
01:01:26 --> 01:01:29
 
 

1204
01:01:29 --> 01:01:32
The Law of Effect said if
something good happens, you're

1205
01:01:32 --> 01:01:35
going to do what you were
doing just beforehand.

1206
01:01:35 --> 01:01:37
Think about baseball.
 

1207
01:01:37 --> 01:01:39
What something good
that can happen?

1208
01:01:39 --> 01:01:40
You get up to the plate.
 

1209
01:01:40 --> 01:01:40
You hit the ball.
 

1210
01:01:40 --> 01:01:42
It goes over the fence.
 

1211
01:01:42 --> 01:01:46
And you get to run
around and stuff.

1212
01:01:46 --> 01:01:49
Well what were you
doing just before?

1213
01:01:49 --> 01:01:52
Well, you know, maybe you
wiped off your shoes or

1214
01:01:52 --> 01:01:54
something like that.
 

1215
01:01:54 --> 01:02:00
So next time you come up to the
plate, you wipe off your shoes.

1216
01:02:00 --> 01:02:02
Don't think anything about it.
 

1217
01:02:02 --> 01:02:04
Pretty soon you're doing
this all the time.

1218
01:02:04 --> 01:02:07
Some sports writer ask you,
you know, how come you wipe

1219
01:02:07 --> 01:02:08
off the shoes every time?
 

1220
01:02:08 --> 01:02:10
The plate's slippery or
some story like that.

1221
01:02:10 --> 01:02:12
You got no idea why it is.
 

1222
01:02:12 --> 01:02:18
But it's probably been shaped
into place by the contingencies

1223
01:02:18 --> 01:02:22
of reinforcement.
 

1224
01:02:22 --> 01:02:25
The second best place
to see this in my

1225
01:02:25 --> 01:02:29
experience is at exams.
 

1226
01:02:29 --> 01:02:30
Look around at exams perhaps.
 

1227
01:02:30 --> 01:02:34
Look at yourself at exams and
say, look at those three

1228
01:02:34 --> 01:02:37
neatly lined up pencils.
 

1229
01:02:37 --> 01:02:40
Points exactly lined up there.
 

1230
01:02:40 --> 01:02:44
The coke has to be exactly to
the left of the test paper,

1231
01:02:44 --> 01:02:47
and the snickers bar has
to be right there yeah.

1232
01:02:47 --> 01:02:50
You know, or I'm
going to flunk.

1233
01:02:50 --> 01:02:51
AUDIENCE: [LAUGHTER]
 

1234
01:02:51 --> 01:02:57
PROFESSOR: Again, if you're
asked about it, the answers oh

1235
01:02:57 --> 01:02:59
I don't know it just makes
me feel comfortable or

1236
01:02:59 --> 01:03:00
something like that.
 

1237
01:03:00 --> 01:03:03
But it's again sort of
superstitious behavior perhaps

1238
01:03:03 --> 01:03:08
shaped into place by this sort
of this Law of Effect

1239
01:03:08 --> 01:03:11
kind of work.
 

1240
01:03:11 --> 01:03:17
 
 

1241
01:03:17 --> 01:03:20
Let me say a bit more about
how this applies out

1242
01:03:20 --> 01:03:21
in the real world.
 

1243
01:03:21 --> 01:03:26
Schedules of reinforcement are
the sorts of things that we do

1244
01:03:26 --> 01:03:28
to ourselves all the time.
 

1245
01:03:28 --> 01:03:29
For example in child rearing.
 

1246
01:03:29 --> 01:03:32
Is that actually the next
example on the handout maybe?

1247
01:03:32 --> 01:03:33
Yeah it says something about
parents and children.

1248
01:03:33 --> 01:03:42
 
 

1249
01:03:42 --> 01:03:47
Suppose that you were a
little kid, which you once

1250
01:03:47 --> 01:03:48
upon a time you were.
 

1251
01:03:48 --> 01:03:52
and you want a cookie.
 

1252
01:03:52 --> 01:03:55
So you say, can I
have a cookie?

1253
01:03:55 --> 01:03:57
And your mom gives
you a cookie.

1254
01:03:57 --> 01:03:58
And that's good.
 

1255
01:03:58 --> 01:04:02
The next time you say, can I
have a cookie, and it's just

1256
01:04:02 --> 01:04:03
before dinner , SO SHE says no.
 

1257
01:04:03 --> 01:04:05
So what do you?
 

1258
01:04:05 --> 01:04:07
Well maybe you extinguish
the behavior immediately.

1259
01:04:07 --> 01:04:09
And you're a perfect model
child, and you end up at,

1260
01:04:09 --> 01:04:12
you know, like MIT
or something like.

1261
01:04:12 --> 01:04:15
Can I have a cookie?
 

1262
01:04:15 --> 01:04:17
Please can I have a cookie?
 

1263
01:04:17 --> 01:04:19
Pretty please can
I have a cookie?

1264
01:04:19 --> 01:04:22
Eventually your mom will say,
oh here have a cookie right.

1265
01:04:22 --> 01:04:23
All right so what
have you done?

1266
01:04:23 --> 01:04:27
We now moved you from an FR1
to an FR something else, or

1267
01:04:27 --> 01:04:30
actually it's probably a
VR schedule of some sort.

1268
01:04:30 --> 01:04:32
So now you're going to be
emitting cookie behavior at

1269
01:04:32 --> 01:04:35
a much more rapid right.
 

1270
01:04:35 --> 01:04:35
Can I have a cookie?
 

1271
01:04:35 --> 01:04:36
Can I have a cookie?
 

1272
01:04:36 --> 01:04:37
Please can I have a cookie?
 

1273
01:04:37 --> 01:04:37
I want a cookie.
 

1274
01:04:37 --> 01:04:38
I want a cookie right now.
 

1275
01:04:38 --> 01:04:41
AUDIENCE: [LAUGHTER]
 

1276
01:04:41 --> 01:04:43
PROFESSOR: Eventually,
you know, your parents

1277
01:04:43 --> 01:04:43
get tired of this.
 

1278
01:04:43 --> 01:04:46
They decide they're going
to cut you off here.

1279
01:04:46 --> 01:04:48
But you're now up
here somewhere.

1280
01:04:48 --> 01:04:51
How's that extinction
curve look?

1281
01:04:51 --> 01:04:54
Oh man, it goes out to
about, you know, age 23

1282
01:04:54 --> 01:04:55
or something like that.
 

1283
01:04:55 --> 01:04:56
AUDIENCE: [LAUGHTER]
 

1284
01:04:56 --> 01:04:59
PROFESSOR: Plus parents
are not good at this.

1285
01:04:59 --> 01:05:01
So what you do is can
I have a cookie?

1286
01:05:01 --> 01:05:01
Can I have a cookie?
 

1287
01:05:01 --> 01:05:02
Can I have a cookie?
 

1288
01:05:02 --> 01:05:03
Can I have a cookie?
 

1289
01:05:03 --> 01:05:04
I really want a cookie.
 

1290
01:05:04 --> 01:05:06
OK have a cookie.
 

1291
01:05:06 --> 01:05:06
OK.
 

1292
01:05:06 --> 01:05:11
So now we're on VR 2064
kind of schedule.

1293
01:05:11 --> 01:05:14
And everybody's going nuts.
 

1294
01:05:14 --> 01:05:17
The other place you see a
similar sort of example is

1295
01:05:17 --> 01:05:18
sleeping through the night.
 

1296
01:05:18 --> 01:05:20
The kid cries right.
 

1297
01:05:20 --> 01:05:21
You go and comfort the kid.
 

1298
01:05:21 --> 01:05:23
Put the kid back
down go to sleep.

1299
01:05:23 --> 01:05:26
Eventually you decide this
little monster needs to sleep

1300
01:05:26 --> 01:05:28
through the night, because
like I need to sleep

1301
01:05:28 --> 01:05:29
through the night.
 

1302
01:05:29 --> 01:05:35
So I'm going to not
get up when he cries.

1303
01:05:35 --> 01:05:39
He's crying, I can't sleep.
 

1304
01:05:39 --> 01:05:41
Crying for like an hour.
 

1305
01:05:41 --> 01:05:43
AUDIENCE: [LAUGHTER]
 

1306
01:05:43 --> 01:05:45
PROFESSOR: Oh maybe
just this time.

1307
01:05:45 --> 01:05:46
All right.
 

1308
01:05:46 --> 01:05:48
Now you've rewarded him for
crying for an hour, rather

1309
01:05:48 --> 01:05:51
than for crying for
a couple of seconds.

1310
01:05:51 --> 01:05:56
And again you end up with a
similar sort of problem.

1311
01:05:56 --> 01:05:59
My eldest actually had
this to an absolute art

1312
01:05:59 --> 01:06:00
when he was little.
 

1313
01:06:00 --> 01:06:01
Of course he didn't
know it again.

1314
01:06:01 --> 01:06:03
Because all nice unconscious
and reflexive stuff.

1315
01:06:03 --> 01:06:09
He came into the world built so
that if he cried for more than

1316
01:06:09 --> 01:06:11
a few minutes, he
also threw up.

1317
01:06:11 --> 01:06:12
AUDIENCE: [LAUGHTER]
 

1318
01:06:12 --> 01:06:12
PROFESSOR: All right.
 

1319
01:06:12 --> 01:06:14
So it's, you know, besides
it's time for him to

1320
01:06:14 --> 01:06:15
sleep through the night.
 

1321
01:06:15 --> 01:06:18
He's crying you're saying, oh
man if he doesn't go back to

1322
01:06:18 --> 01:06:23
sleep in a second not only am I
going to go nuts, but I'm going

1323
01:06:23 --> 01:06:26
to have to do the
laundry again.

1324
01:06:26 --> 01:06:30
But if I reward him then.
 

1325
01:06:30 --> 01:06:35
I don't think they've sleep
through the night yet.

1326
01:06:35 --> 01:06:38
Is this encouraging yet?
 

1327
01:06:38 --> 01:06:40
Mara is getting so
depressed there.

1328
01:06:40 --> 01:06:43
 
 

1329
01:06:43 --> 01:06:45
All right that you're
not babies anymore.

1330
01:06:45 --> 01:06:49
Let's give an example of
the relevance of this that

1331
01:06:49 --> 01:06:54
might have something to do
closer to adult behavior.

1332
01:06:54 --> 01:06:56
One of the interesting realms
where these schedules of

1333
01:06:56 --> 01:07:00
reinforcement may have an
unintentional, well none of it

1334
01:07:00 --> 01:07:03
has intention to it, an
unfortunate negative

1335
01:07:03 --> 01:07:09
consequence is a phenomenon
known in at least in the

1336
01:07:09 --> 01:07:14
research on sexual behavior
as getting to yes.

1337
01:07:14 --> 01:07:16
What is this about?
 

1338
01:07:16 --> 01:07:24
Typically in heterosexual
relationships the request,

1339
01:07:24 --> 01:07:29
verbal or otherwise, for some
sort of positive reinforcement

1340
01:07:29 --> 01:07:33
of a sexual nature comes from
the guy and is delivered to

1341
01:07:33 --> 01:07:36
the woman in some fashion.
 

1342
01:07:36 --> 01:07:41
Typically that request,
verbal or otherwise, is

1343
01:07:41 --> 01:07:43
met with a no initially.
 

1344
01:07:43 --> 01:07:50
If the relationship develops
over time of course, eventually

1345
01:07:50 --> 01:07:51
odds are that there
will be a yes.

1346
01:07:51 --> 01:07:53
Well what does that sound like?
 

1347
01:07:53 --> 01:07:56
That some sort of a variable
schedule of reinforcement.

1348
01:07:56 --> 01:07:59
Oh why might that be the case?
 

1349
01:07:59 --> 01:08:00
We'll talk about this
more later in the term.

1350
01:08:00 --> 01:08:06
But there is interesting
evidence that guys are not

1351
01:08:06 --> 01:08:09
some, you know, unbridal
bag of hormones

1352
01:08:09 --> 01:08:10
AUDIENCE: [LAUGHTER]
 

1353
01:08:10 --> 01:08:13
PROFESSOR: But that they
actually fall in love

1354
01:08:13 --> 01:08:16
faster than women do.
 

1355
01:08:16 --> 01:08:21
So, you know, it's the Romeo
and Juliet thing right.

1356
01:08:21 --> 01:08:25
You know, he's allegedly
in love with Rosalind.

1357
01:08:25 --> 01:08:31
He goes and see Juliet, and two
seconds later it's oh she doth

1358
01:08:31 --> 01:08:33
teaches the torches to burn
bright and all that

1359
01:08:33 --> 01:08:34
sort of stuff.
 

1360
01:08:34 --> 01:08:35
Woo, he's gone.
 

1361
01:08:35 --> 01:08:36
AUDIENCE: [LAUGHTER]
 

1362
01:08:36 --> 01:08:40
PROFESSOR: And when he walks
out of the party later, and

1363
01:08:40 --> 01:08:42
meets his friend the
psychologist who does a

1364
01:08:42 --> 01:08:45
survey kind of thing.
 

1365
01:08:45 --> 01:08:49
He will report he feels
like he's in love.

1366
01:08:49 --> 01:08:52
And I don't know what
yet, Juliet may be a

1367
01:08:52 --> 01:08:52
different example.
 

1368
01:08:52 --> 01:08:54
I mean they all end up dead
and things like that.

1369
01:08:54 --> 01:09:01
But typically sheet will be
slower to report that she

1370
01:09:01 --> 01:09:03
feels like she's in love.
 

1371
01:09:03 --> 01:09:07
So he's sitting there saying,
you know, I'm in love and I'm

1372
01:09:07 --> 01:09:08
going to be with her forever.
 

1373
01:09:08 --> 01:09:11
And if I'm going to be with her
forever, there's certain things

1374
01:09:11 --> 01:09:14
that we might be interested in.
 

1375
01:09:14 --> 01:09:17
And she's saying, I just met
him five minutes ago, I

1376
01:09:17 --> 01:09:19
mean what is this about?
 

1377
01:09:19 --> 01:09:20
AUDIENCE: [LAUGHTER]
 

1378
01:09:20 --> 01:09:21
PROFESSOR: Anyway.
 

1379
01:09:21 --> 01:09:26
The result is some sort of
a variable ratio schedule

1380
01:09:26 --> 01:09:28
of reinforcement.
 

1381
01:09:28 --> 01:09:33
And in most cases this works
itself out just fine.

1382
01:09:33 --> 01:09:36
But suppose the relationship
doesn't go particularly well.

1383
01:09:36 --> 01:09:41
Suppose for example -- well
where's my extinction curve?

1384
01:09:41 --> 01:09:43
Here's a good extension curve.
 

1385
01:09:43 --> 01:09:44
So you're up here somewhere on
this variable variable ratio

1386
01:09:44 --> 01:09:47
schedule of reinforcement.
 

1387
01:09:47 --> 01:09:51
And let's say she decides that
this is not a relationship that

1388
01:09:51 --> 01:09:54
she's interested in anymore.
 

1389
01:09:54 --> 01:10:01
It takes him now an extremely
long time potentially to get

1390
01:10:01 --> 01:10:07
the hint and stop, you know,
pressing that key in

1391
01:10:07 --> 01:10:09
the Skinner box.
 

1392
01:10:09 --> 01:10:14
And we'll talk later about
the ambiguous nature of

1393
01:10:14 --> 01:10:16
communications in this regard.
 

1394
01:10:16 --> 01:10:19
If he's not sure whether she is
saying yes or no or whatever,

1395
01:10:19 --> 01:10:20
you can end up in trouble.
 

1396
01:10:20 --> 01:10:24
And this an example, you know,
where these sort of rules of

1397
01:10:24 --> 01:10:30
association learning suddenly
become relevant in behavior

1398
01:10:30 --> 01:10:33
that's much more complicated
than whether or not you're

1399
01:10:33 --> 01:10:38
pecking a response key
in a Skinner box.

1400
01:10:38 --> 01:10:43
It's that kind of application,
the possibility of going from

1401
01:10:43 --> 01:10:46
these very simple behaviors to
much more complicated, much

1402
01:10:46 --> 01:10:52
more richer behaviors that
drove what was the leading

1403
01:10:52 --> 01:10:55
school of psychology in the
first half the 20th century

1404
01:10:55 --> 01:10:58
in the US, which was
known as behaviorism.

1405
01:10:58 --> 01:11:03
Which was the doctrine that
basically said, look classical

1406
01:11:03 --> 01:11:05
conditioning, operant
conditioning, a couple other

1407
01:11:05 --> 01:11:10
bits, those are the
atoms of behavior.

1408
01:11:10 --> 01:11:14
And we can build up the
molecules and the rest of the

1409
01:11:14 --> 01:11:17
organism out of those atoms.
 

1410
01:11:17 --> 01:11:18
That's all we need.
 

1411
01:11:18 --> 01:11:22
 
 

1412
01:11:22 --> 01:11:25
So I put this quote from John
Watson who's the founder of

1413
01:11:25 --> 01:11:27
American behaviorism on there,
where he says, you know,

1414
01:11:27 --> 01:11:30
we can write a psychology.
 

1415
01:11:30 --> 01:11:32
Define it as the
science of behavior.

1416
01:11:32 --> 01:11:34
That was a move
by itself right.

1417
01:11:34 --> 01:11:39
No science of behavior and
mental life, or no signs of

1418
01:11:39 --> 01:11:40
human mental life or something.
 

1419
01:11:40 --> 01:11:44
The science of behavior of
observable behavior, never

1420
01:11:44 --> 01:11:46
go back on our definition.
 

1421
01:11:46 --> 01:11:49
Never use terms like
consciousness, mental states,

1422
01:11:49 --> 01:11:51
mind, content and so on.
 

1423
01:11:51 --> 01:11:54
It can be done in terms of
habit formation, habit

1424
01:11:54 --> 01:11:57
integration, which are
terms for this sort of

1425
01:11:57 --> 01:11:59
association learning.
 

1426
01:11:59 --> 01:12:06
He is basically saying, that
look if you're taking

1427
01:12:06 --> 01:12:13
chemistry, and you go to the
chemistry prof and say, you

1428
01:12:13 --> 01:12:16
know, these elements you've got
here, they're nice elements.

1429
01:12:16 --> 01:12:23
But I think I want another 16
kind of constructs to explain

1430
01:12:23 --> 01:12:24
what's going on here.
 

1431
01:12:24 --> 01:12:27
The chemistry professor's going
to look at you and say, no

1432
01:12:27 --> 01:12:30
sorry these are the elements.
 

1433
01:12:30 --> 01:12:34
Everything can be
built out this stuff.

1434
01:12:34 --> 01:12:37
You don't like it
you got a problem.

1435
01:12:37 --> 01:12:40
And the behaviorists were
saying essentially the same

1436
01:12:40 --> 01:12:41
thing about psychology.
 

1437
01:12:41 --> 01:12:46
They thought that a very small
set of atoms in this case,

1438
01:12:46 --> 01:12:51
these Laws of Association, were
the elementary properties,

1439
01:12:51 --> 01:12:56
and you could build up
everything out of that.

1440
01:12:56 --> 01:13:01
That carried with it a couple
of an interesting bits of

1441
01:13:01 --> 01:13:05
ideology that will recur
later in the course.

1442
01:13:05 --> 01:13:08
One of them was the idea
that you were basically

1443
01:13:08 --> 01:13:09
an association engine.
 

1444
01:13:09 --> 01:13:11
You were there to
learn associations.

1445
01:13:11 --> 01:13:18
And that everything that was
worth knowing in psychology,

1446
01:13:18 --> 01:13:20
everything that was worth
studying in psychology, was

1447
01:13:20 --> 01:13:22
something that was learnable.
 

1448
01:13:22 --> 01:13:28
That it was not interesting
beyond, you know, for their

1449
01:13:28 --> 01:13:29
purposes kind of
trivial things.

1450
01:13:29 --> 01:13:35
Like you came with eyes
and not radar dishes.

1451
01:13:35 --> 01:13:38
You know beyond that sort of
thing, it was uninteresting

1452
01:13:38 --> 01:13:40
what nature had
provided to you.

1453
01:13:40 --> 01:13:46
Psychology was the effect of
what the interaction of the

1454
01:13:46 --> 01:13:50
environment and these
atoms of learning.

1455
01:13:50 --> 01:13:56
That Everybody was essentially
identical until this

1456
01:13:56 --> 01:13:57
learning started.
 

1457
01:13:57 --> 01:14:01
And what got you to MIT was the
contingencies of reinforcement

1458
01:14:01 --> 01:14:05
over the prior 18
years of your life.

1459
01:14:05 --> 01:14:11
This resonated with a certain
democratic current in

1460
01:14:11 --> 01:14:15
American political thought.
 

1461
01:14:15 --> 01:14:19
In sort of, you know, grade
school civics lessons form.

1462
01:14:19 --> 01:14:21
This was the doctrine
that anybody could grow

1463
01:14:21 --> 01:14:23
up to be president.
 

1464
01:14:23 --> 01:14:29
You didn't actually have to
be a Bush to be president.

1465
01:14:29 --> 01:14:30
Anybody could be president.
 

1466
01:14:30 --> 01:14:36
What was important was how
the environment shaped you.

1467
01:14:36 --> 01:14:41
That's a doctrine in its
strongest form known

1468
01:14:41 --> 01:14:42
as empiricism.
 

1469
01:14:42 --> 01:14:46
 
 

1470
01:14:46 --> 01:14:48
That look about right?
 

1471
01:14:48 --> 01:14:49
OK.
 

1472
01:14:49 --> 01:14:59
The opposite pole is nativism,
the notion that your

1473
01:14:59 --> 01:15:05
genetic innate endowment
is determinitive.

1474
01:15:05 --> 01:15:11
And that you're here, because
you were born with, you know,

1475
01:15:11 --> 01:15:15
you've got the gene for
calculus in there somewhere.

1476
01:15:15 --> 01:15:18
Some of you may have now
decided you lost it somewhere.

1477
01:15:18 --> 01:15:23
But you had once upon a time.
 

1478
01:15:23 --> 01:15:29
So the notion here is what the
behaviorist thought was they

1479
01:15:29 --> 01:15:31
were strongly weighted towards
this empiricist,

1480
01:15:31 --> 01:15:38
environmentally driven
side of things.

1481
01:15:38 --> 01:15:39
In a couple of minutes.
 

1482
01:15:39 --> 01:15:42
Why am I using the past tense?
 

1483
01:15:42 --> 01:15:44
Why aren't we all
behaviorists now?

1484
01:15:44 --> 01:15:47
There are a number of reasons.
 

1485
01:15:47 --> 01:15:51
I suppose the broadest reason
is a course like this is full

1486
01:15:51 --> 01:15:54
of, you know, on the one hand
position on the other

1487
01:15:54 --> 01:15:55
hand, you know.
 

1488
01:15:55 --> 01:15:56
Empiricism and nativism.
 

1489
01:15:56 --> 01:15:59
And I'll give you a
hint for the exam.

1490
01:15:59 --> 01:16:02
The answer always turns
out to be both right.

1491
01:16:02 --> 01:16:05
The extreme positions, the
extreme theoretical positions

1492
01:16:05 --> 01:16:08
in psychology never
seem to quite work out.

1493
01:16:08 --> 01:16:12
But sort of less trivially.
 

1494
01:16:12 --> 01:16:20
 
 

1495
01:16:20 --> 01:16:25
The basics of association
learning theory are absolutely

1496
01:16:25 --> 01:16:27
worth a lecture in this course.
 

1497
01:16:27 --> 01:16:29
And absolutely worth chapter 4.
 

1498
01:16:29 --> 01:16:32
And you'll find on the
handouts, you know, guide

1499
01:16:32 --> 01:16:35
for reading in chapter 4.
 

1500
01:16:35 --> 01:16:39
But they're not all
of psychology.

1501
01:16:39 --> 01:16:45
Watson's statement says, you
know, we can build a science

1502
01:16:45 --> 01:16:48
of behavior without ever
talking about, you know,

1503
01:16:48 --> 01:16:50
consciousness for example.
 

1504
01:16:50 --> 01:16:53
But, you know, that's
an interesting topic.

1505
01:16:53 --> 01:17:02
And if I want to know about it,
I don't want to be told by the

1506
01:17:02 --> 01:17:06
field's idealogues that it's
not a legitimate

1507
01:17:06 --> 01:17:07
area of inquiry.
 

1508
01:17:07 --> 01:17:16
I don't want to be told that
the feeling aspect of

1509
01:17:16 --> 01:17:20
emotion is not a legitimate
topic of psychology.

1510
01:17:20 --> 01:17:22
And that the only thing we can
do is, you know, observe how

1511
01:17:22 --> 01:17:26
many tears are coming out,
something like that.

1512
01:17:26 --> 01:17:29
It proved to be an
incomplete psychology.

1513
01:17:29 --> 01:17:29
In.
 

1514
01:17:29 --> 01:17:33
The same way I should say, that
the occupants of the nativist

1515
01:17:33 --> 01:17:39
poll at the moment are the most
dogmatic of the evolutionary

1516
01:17:39 --> 01:17:40
psychologists.
 

1517
01:17:40 --> 01:17:43
Evolution has wonderful
explanatory power

1518
01:17:43 --> 01:17:44
in psychology.
 

1519
01:17:44 --> 01:17:48
But evolution is not
itself a psychology.

1520
01:17:48 --> 01:17:50
It's not a complete
psychology by itself.

1521
01:17:50 --> 01:17:55
It doesn't give you
the richness of the

1522
01:17:55 --> 01:17:58
field as a whole.
 

1523
01:17:58 --> 01:18:01
Another important reason we're
not all behaviorist is it turns

1524
01:18:01 --> 01:18:07
out to be somewhere between
uninteresting and wrong to

1525
01:18:07 --> 01:18:14
argue that everything that was
of interest was learnable.

1526
01:18:14 --> 01:18:17
Language, we'll see later in
the course, is an example.

1527
01:18:17 --> 01:18:19
Course you learn the
language that you speak.

1528
01:18:19 --> 01:18:22
But you learned it because you
had a brain that was built

1529
01:18:22 --> 01:18:24
to learn a language.
 

1530
01:18:24 --> 01:18:31
And that innate endowment, that
language learning endowment, is

1531
01:18:31 --> 01:18:33
worth study in its own right.
 

1532
01:18:33 --> 01:18:34
And that wasn't something
that fit into the

1533
01:18:34 --> 01:18:37
behaviorist program.
 

1534
01:18:37 --> 01:18:41
So that's why you're
not a behaviorist now.

1535
01:18:41 --> 01:18:43
But even if you're not a
behaviorist, you still

1536
01:18:43 --> 01:18:46
want to read chapter 4.
 

1537
01:18:46 --> 01:18:46