1
00:00:00,000 --> 00:00:01,944
[SQUEAKING]

2
00:00:01,944 --> 00:00:03,888
[RUSTLING]

3
00:00:03,888 --> 00:00:07,290
[CLICKING]

4
00:00:10,115 --> 00:00:11,990
NANCY KANWISHER: Here's
the agenda for today.

5
00:00:11,990 --> 00:00:16,100
As usual, a bunch of
announcements in red.

6
00:00:16,100 --> 00:00:17,210
Assignment 4 was graded.

7
00:00:17,210 --> 00:00:21,470
There will be comments showing
up online on Stellar soon

8
00:00:21,470 --> 00:00:25,160
on any of you who didn't get
a near-perfect score on it.

9
00:00:25,160 --> 00:00:29,528
And I'll also be going over a
little bit of it in a moment.

10
00:00:29,528 --> 00:00:31,070
And then once we do
that, we're going

11
00:00:31,070 --> 00:00:33,740
to talk about navigation--
how we know where we are,

12
00:00:33,740 --> 00:00:36,170
and how to get from here
to someplace else, which

13
00:00:36,170 --> 00:00:39,170
is much more awesome than it
sounds at first, as you will

14
00:00:39,170 --> 00:00:40,020
see.

15
00:00:40,020 --> 00:00:40,520
OK.

16
00:00:40,520 --> 00:00:43,200
So quick review.

17
00:00:43,200 --> 00:00:43,700
OK.

18
00:00:43,700 --> 00:00:45,620
So what was the key point?

19
00:00:45,620 --> 00:00:49,550
Why did I assign the Haxby 2001
article for you guys to read?

20
00:00:49,550 --> 00:00:51,770
It presents this
important challenge

21
00:00:51,770 --> 00:00:54,960
to the functional specificity
of the face area and the place

22
00:00:54,960 --> 00:00:55,460
area.

23
00:00:55,460 --> 00:00:57,110
What was that challenge?

24
00:00:57,110 --> 00:00:59,690
What was Haxby's key point?

25
00:00:59,690 --> 00:01:01,070
Yes, Isabel.

26
00:01:01,070 --> 00:01:06,530
AUDIENCE: Well, he
was just attacked--

27
00:01:06,530 --> 00:01:10,880
it just has a preference
for rectilinear

28
00:01:10,880 --> 00:01:13,300
and not seeing if it's
actual scanning for.

29
00:01:13,300 --> 00:01:17,623
It's not truly detecting
whether it's a face or not.

30
00:01:17,623 --> 00:01:18,540
NANCY KANWISHER: Yeah.

31
00:01:18,540 --> 00:01:20,920
He wasn't worrying about
rectilinearity so much back

32
00:01:20,920 --> 00:01:21,420
then.

33
00:01:21,420 --> 00:01:25,260
But his point was that
we shouldn't care just

34
00:01:25,260 --> 00:01:28,800
about the overall magnitude
of response of a region.

35
00:01:28,800 --> 00:01:30,570
Like, OK, it's nice
if the face area

36
00:01:30,570 --> 00:01:33,870
responds like this to face,
but isn't like that to objects.

37
00:01:33,870 --> 00:01:39,810
But even if it responds low and
the same to cars and chairs,

38
00:01:39,810 --> 00:01:43,200
it might still have information
to enable you to distinguish

39
00:01:43,200 --> 00:01:46,500
cars from chairs if the pattern
of response across boxes

40
00:01:46,500 --> 00:01:50,770
in that region was stably
different for cars and chairs.

41
00:01:50,770 --> 00:01:51,270
OK?

42
00:01:51,270 --> 00:01:52,050
That's really key.

43
00:01:52,050 --> 00:01:53,633
We'll go over it at
a few more points.

44
00:01:53,633 --> 00:01:55,030
But that's essential, right?

45
00:01:55,030 --> 00:01:57,030
A lot of the details that
I'm going to teach you

46
00:01:57,030 --> 00:01:58,768
that go by in
class don't matter,

47
00:01:58,768 --> 00:02:00,810
but I really want you guys
to understand the PPA.

48
00:02:00,810 --> 00:02:01,950
And that's the nub of it.

49
00:02:01,950 --> 00:02:02,640
OK?

50
00:02:02,640 --> 00:02:07,740
So the idea is that
selective-- his claim

51
00:02:07,740 --> 00:02:10,080
is that selective regions,
like the face area,

52
00:02:10,080 --> 00:02:12,930
contain information about
non-preferred stimuli.

53
00:02:12,930 --> 00:02:14,910
That is, like, non-faces
for the face area,

54
00:02:14,910 --> 00:02:17,980
or non-places for
the place area.

55
00:02:17,980 --> 00:02:20,460
And because they
contain information,

56
00:02:20,460 --> 00:02:23,710
those regions don't care only
about their preferred category.

57
00:02:23,710 --> 00:02:26,130
So why does Kanwisher
get off saying

58
00:02:26,130 --> 00:02:29,190
the FFA is only about faces and
the PPA is only about places

59
00:02:29,190 --> 00:02:31,780
if we can see information about
other things in those regions?

60
00:02:31,780 --> 00:02:32,280
OK.

61
00:02:32,280 --> 00:02:33,738
That's a really
important critique.

62
00:02:33,738 --> 00:02:35,340
That's why we're
spending time on it.

63
00:02:35,340 --> 00:02:36,480
OK?

64
00:02:36,480 --> 00:02:37,560
OK.

65
00:02:37,560 --> 00:02:40,320
Next, what kind
of empirical data

66
00:02:40,320 --> 00:02:45,420
might be an answer
to Haxby's challenge?

67
00:02:45,420 --> 00:02:47,520
I presented at least
three different kinds

68
00:02:47,520 --> 00:02:51,210
of data that can address this
and say, hey, wait a minute.

69
00:02:51,210 --> 00:02:54,120
You know, you have a point,
but what kind of data

70
00:02:54,120 --> 00:02:57,228
could speak to that
and respond to Haxby?

71
00:02:57,228 --> 00:02:59,520
We didn't actually talk about
this explicitly in class,

72
00:02:59,520 --> 00:03:00,810
but think about it.

73
00:03:00,810 --> 00:03:03,270
Here's the claim he makes.

74
00:03:03,270 --> 00:03:04,530
What might we say, right?

75
00:03:04,530 --> 00:03:06,390
So that's empirically true.

76
00:03:06,390 --> 00:03:07,650
Like, you look in the FFA.

77
00:03:07,650 --> 00:03:11,370
Even in my own data, I
can distinguish chairs

78
00:03:11,370 --> 00:03:14,860
from shoes a little
teeny bit in the FFA.

79
00:03:14,860 --> 00:03:15,360
OK?

80
00:03:15,360 --> 00:03:17,850
So that empirical claim is true.

81
00:03:17,850 --> 00:03:21,270
Why might it
nonetheless be the case

82
00:03:21,270 --> 00:03:25,350
that the face area is really
only about face recognition?

83
00:03:25,350 --> 00:03:28,380
What other data have
you heard in here

84
00:03:28,380 --> 00:03:30,360
that might make you think that?

85
00:03:30,360 --> 00:03:31,728
Yes, Ben.

86
00:03:31,728 --> 00:03:33,270
AUDIENCE: Because
it's the presence--

87
00:03:33,270 --> 00:03:34,373
NANCY KANWISHER: Speak up.

88
00:03:34,373 --> 00:03:35,040
AUDIENCE: Sorry.

89
00:03:35,040 --> 00:03:38,850
The presence of low-level
stimuli that are generally in

90
00:03:38,850 --> 00:03:45,543
faces, but can also be sparse
on chairs and cars in context.

91
00:03:45,543 --> 00:03:46,710
NANCY KANWISHER: Absolutely.

92
00:03:46,710 --> 00:03:49,440
So yeah, put another
way, even if you

93
00:03:49,440 --> 00:03:52,230
had a perfect coder for faces--

94
00:03:52,230 --> 00:03:56,550
like take your best deep net
for face recognition, VGG face--

95
00:03:56,550 --> 00:03:59,160
it can distinguish chairs
and shoes too, right?

96
00:03:59,160 --> 00:04:02,010
The features that you
use to represent faces

97
00:04:02,010 --> 00:04:05,550
will slightly discriminate
between other non-face objects.

98
00:04:05,550 --> 00:04:08,520
So the fact that we can see
that information in itself

99
00:04:08,520 --> 00:04:13,560
isn't strong evidence
that that region isn't

100
00:04:13,560 --> 00:04:14,820
selective for face perception.

101
00:04:14,820 --> 00:04:15,445
Absolutely.

102
00:04:15,445 --> 00:04:15,945
What else?

103
00:04:19,380 --> 00:04:21,060
Yeah, OK.

104
00:04:21,060 --> 00:04:23,370
AUDIENCE: Like transcranial
magnetic stimulation?

105
00:04:23,370 --> 00:04:25,050
When you stimulate
the epithelial

106
00:04:25,050 --> 00:04:26,692
and you look at it
face it affects it,

107
00:04:26,692 --> 00:04:28,650
but when you are like
looking at other objects,

108
00:04:28,650 --> 00:04:29,700
the effect is no longer.

109
00:04:29,700 --> 00:04:30,742
NANCY KANWISHER: Exactly.

110
00:04:30,742 --> 00:04:32,430
And so what does
that tell you about--

111
00:04:32,430 --> 00:04:33,960
OK, so there's
pattern information

112
00:04:33,960 --> 00:04:37,500
in there about other
things beyond faces.

113
00:04:37,500 --> 00:04:39,720
But?

114
00:04:39,720 --> 00:04:43,140
Apparently it's not used, right?

115
00:04:43,140 --> 00:04:45,870
Now with every bit of evidence,
you can always argue back.

116
00:04:45,870 --> 00:04:48,410
People would say, well,
TMS, those effects are tiny.

117
00:04:48,410 --> 00:04:50,160
Maybe there isn't and
we didn't have power

118
00:04:50,160 --> 00:04:51,910
to detect it, blah,
blah, blah, blah.

119
00:04:51,910 --> 00:04:53,580
But at least,
absolutely you're right.

120
00:04:53,580 --> 00:04:54,870
TMS argues against them.

121
00:04:54,870 --> 00:04:55,933
What else?

122
00:04:55,933 --> 00:04:58,350
Or at least is a way to argue
against it-- and the Pitcher

123
00:04:58,350 --> 00:05:00,540
paper that I assigned
and other papers

124
00:05:00,540 --> 00:05:03,120
that we've talked about in
here provide some evidence

125
00:05:03,120 --> 00:05:06,480
that actually, at least the
occipital face area really

126
00:05:06,480 --> 00:05:09,270
is only causally involved
in face perception

127
00:05:09,270 --> 00:05:13,350
even if there's information
in there about other things.

128
00:05:13,350 --> 00:05:14,340
What else?

129
00:05:14,340 --> 00:05:17,390
What other methods
can address this?

130
00:05:17,390 --> 00:05:17,960
Yeah.

131
00:05:17,960 --> 00:05:19,627
AUDIENCE: That is
pretty new simulation,

132
00:05:19,627 --> 00:05:24,860
and even when you're pressing
hand against your face,

133
00:05:24,860 --> 00:05:26,248
you can perceive faces in it.

134
00:05:26,248 --> 00:05:27,290
NANCY KANWISHER: Exactly.

135
00:05:27,290 --> 00:05:27,860
Exactly.

136
00:05:27,860 --> 00:05:29,540
So these are both
causal tests, right?

137
00:05:29,540 --> 00:05:31,010
OK, there's
information in there.

138
00:05:31,010 --> 00:05:33,980
But is it causally
used in behavior?

139
00:05:33,980 --> 00:05:35,660
TMS suggests not.

140
00:05:35,660 --> 00:05:40,040
The little bit of direct
intracranial stimulation data

141
00:05:40,040 --> 00:05:43,550
that I showed you also suggests
the causal effects when you

142
00:05:43,550 --> 00:05:46,460
stimulate that region are
specific to face perception,

143
00:05:46,460 --> 00:05:48,440
again suggesting that
even if there's pattern

144
00:05:48,440 --> 00:05:50,930
information in there,
it's not doing anything

145
00:05:50,930 --> 00:05:52,550
important because
we can mess it up

146
00:05:52,550 --> 00:05:54,830
and nothing happens to the
perception of things that

147
00:05:54,830 --> 00:05:56,030
aren't faces.

148
00:05:56,030 --> 00:05:56,870
Absolutely.

149
00:05:56,870 --> 00:05:57,530
What else?

150
00:06:00,580 --> 00:06:03,180
We talked about it very
briefly a few weeks ago.

151
00:06:03,180 --> 00:06:03,680
Yeah.

152
00:06:03,680 --> 00:06:06,540
AUDIENCE: So if you
remove the [INAUDIBLE],,

153
00:06:06,540 --> 00:06:08,860
it just completely
makes a person

154
00:06:08,860 --> 00:06:10,670
incapable of perceiving faces.

155
00:06:10,670 --> 00:06:13,120
That is causing--

156
00:06:13,120 --> 00:06:16,000
NANCY KANWISHER: Yes, but
the crucial way-- yes,

157
00:06:16,000 --> 00:06:19,348
but the crucial way
to address Haxby

158
00:06:19,348 --> 00:06:20,890
would be what further
aspect of that?

159
00:06:20,890 --> 00:06:22,480
Yes.

160
00:06:22,480 --> 00:06:25,420
And by the way, we don't
remove the area in humans,

161
00:06:25,420 --> 00:06:26,920
but occasionally,
we find a human

162
00:06:26,920 --> 00:06:29,698
who had a lesion there due to a
stroke and then we study them.

163
00:06:29,698 --> 00:06:31,990
AUDIENCE: So they're still
able to do other categories.

164
00:06:31,990 --> 00:06:33,032
NANCY KANWISHER: Exactly.

165
00:06:33,032 --> 00:06:34,210
Exactly.

166
00:06:34,210 --> 00:06:40,060
So all three lines of evidence
from studies of prosopagnosia,

167
00:06:40,060 --> 00:06:43,780
electrical stimulation directly
on the brain, and TMS, all

168
00:06:43,780 --> 00:06:45,880
can provide evidence
to various degrees.

169
00:06:45,880 --> 00:06:47,260
Again, one can
quibble about each

170
00:06:47,260 --> 00:06:48,760
of these particular studies.

171
00:06:48,760 --> 00:06:52,090
But all of those suggests that
even though there's information

172
00:06:52,090 --> 00:06:54,190
in the pattern,
Haxby's right-- there's

173
00:06:54,190 --> 00:06:57,050
information in there about
other things that aren't faces.

174
00:06:57,050 --> 00:07:00,700
The only causal effects when
you mess up with that region

175
00:07:00,700 --> 00:07:03,010
are on faces, not
on other things.

176
00:07:03,010 --> 00:07:04,810
That suggests that
pattern information

177
00:07:04,810 --> 00:07:07,810
is what they sometimes say
in philosophical circles is

178
00:07:07,810 --> 00:07:08,890
"epiphenomenal."

179
00:07:08,890 --> 00:07:15,100
That is, it's just not related
to behavior and perception.

180
00:07:15,100 --> 00:07:16,480
Make sense?

181
00:07:16,480 --> 00:07:19,480
OK, moving along,
how can we then

182
00:07:19,480 --> 00:07:22,150
use Haxby's method
to not just engage

183
00:07:22,150 --> 00:07:25,870
in this little fight about the
FFA and how specific it is,

184
00:07:25,870 --> 00:07:31,450
but to harness this method and
ask other interesting questions

185
00:07:31,450 --> 00:07:32,710
from functional MRI data.

186
00:07:32,710 --> 00:07:36,400
How can we use it to find out,
for example, does the place

187
00:07:36,400 --> 00:07:39,307
area discriminate, say, beach
scenes from city scenes?

188
00:07:39,307 --> 00:07:41,140
We want to know what's
represented in there.

189
00:07:41,140 --> 00:07:42,848
How could we use this
method to find out?

190
00:07:51,150 --> 00:07:52,710
Yes, Jimmy.

191
00:07:52,710 --> 00:07:55,680
AUDIENCE: If I do what
Haxby kind of did, and try

192
00:07:55,680 --> 00:07:59,520
the decoder, and see if
the decoder could decide

193
00:07:59,520 --> 00:08:02,010
and differentiate between
the city and or like

194
00:08:02,010 --> 00:08:03,818
an acre of shade.

195
00:08:03,818 --> 00:08:04,860
NANCY KANWISHER: Exactly.

196
00:08:04,860 --> 00:08:05,850
Exactly.

197
00:08:05,850 --> 00:08:07,500
So we talked about
decoding methods

198
00:08:07,500 --> 00:08:10,253
last time as a way to
use machine learning

199
00:08:10,253 --> 00:08:11,670
to look at the
pattern of response

200
00:08:11,670 --> 00:08:15,690
in a region of the brain, and
train the decoder so it knows

201
00:08:15,690 --> 00:08:20,070
what the response looks like
during viewing of beach scenes,

202
00:08:20,070 --> 00:08:22,563
train it so it knows what
the response in that region

203
00:08:22,563 --> 00:08:24,480
looks like when you're
looking at city scenes,

204
00:08:24,480 --> 00:08:26,800
and then take a new
pattern, and say,

205
00:08:26,800 --> 00:08:28,380
is this more like
the beach pattern

206
00:08:28,380 --> 00:08:29,970
or is it more like
the city pattern?

207
00:08:29,970 --> 00:08:32,130
And that's how you could
decode from that region.

208
00:08:32,130 --> 00:08:32,700
Yes.

209
00:08:32,700 --> 00:08:36,240
AUDIENCE: That doesn't tell as
much, in the sense that it's

210
00:08:36,240 --> 00:08:37,090
not telling you--

211
00:08:37,090 --> 00:08:39,690
I mean, we know that there
is residue of information

212
00:08:39,690 --> 00:08:41,590
nevertheless, and
that this community

213
00:08:41,590 --> 00:08:46,170
can be varied on any region
considered at any time, always.

214
00:08:46,170 --> 00:08:48,660
NANCY KANWISHER: We have
a true nihilist here.

215
00:08:48,660 --> 00:08:50,980
No, it's a good question.

216
00:08:50,980 --> 00:08:54,570
It's not the case that you can
discriminate anything based

217
00:08:54,570 --> 00:08:56,320
on any region of the brain.

218
00:08:56,320 --> 00:08:57,870
So there are some constraints.

219
00:08:57,870 --> 00:08:59,287
There are some
things you can find

220
00:08:59,287 --> 00:09:02,220
in some places and other things
you can find in other places.

221
00:09:02,220 --> 00:09:04,540
And they're not uniformly
distributed over the brain.

222
00:09:04,540 --> 00:09:07,050
However, the fact we
just-- the point I just

223
00:09:07,050 --> 00:09:09,660
made about yes, there's
discriminative information

224
00:09:09,660 --> 00:09:13,410
in the face area about non-faces
but maybe it's not used,

225
00:09:13,410 --> 00:09:17,310
should raise a huge caveat
about this whole method.

226
00:09:17,310 --> 00:09:18,690
How do we ever know?

227
00:09:18,690 --> 00:09:20,580
We see some discriminative
information.

228
00:09:20,580 --> 00:09:22,080
How do we know
whether it's actually

229
00:09:22,080 --> 00:09:24,300
used by the brain,
part of the brain's

230
00:09:24,300 --> 00:09:28,320
own code for information, or
just epiphenomenal garbage

231
00:09:28,320 --> 00:09:30,240
that's a byproduct
of something else?

232
00:09:30,240 --> 00:09:34,350
It's a really important question
about all of pattern analysis.

233
00:09:34,350 --> 00:09:36,630
We do it anyway
because we're beggars.

234
00:09:36,630 --> 00:09:39,210
We can't be choosers in terms
of methods with human cognitive

235
00:09:39,210 --> 00:09:39,780
neuroscience.

236
00:09:39,780 --> 00:09:41,970
And we want to know
desperately what's

237
00:09:41,970 --> 00:09:43,240
represented in each region.

238
00:09:43,240 --> 00:09:44,040
So we do this.

239
00:09:44,040 --> 00:09:45,510
But whenever you
see these lovely,

240
00:09:45,510 --> 00:09:49,800
"I can decode x from y," things,
you should always be wondering.

241
00:09:49,800 --> 00:09:52,470
Who knows if that fact
that you, the scientist,

242
00:09:52,470 --> 00:09:54,480
can decode it from
that region means

243
00:09:54,480 --> 00:09:58,280
the brain itself is reading that
information out of that region?

244
00:09:58,280 --> 00:09:59,280
Big, important question.

245
00:10:02,310 --> 00:10:04,140
All right, put another way--

246
00:10:04,140 --> 00:10:06,450
so Jimmy mentioned just
decoding in general

247
00:10:06,450 --> 00:10:07,680
and that's absolutely right.

248
00:10:07,680 --> 00:10:10,590
But to directly harness
the Haxby version of this,

249
00:10:10,590 --> 00:10:11,400
what would we do?

250
00:10:11,400 --> 00:10:15,220
First, we would functionally
localize the PPA

251
00:10:15,220 --> 00:10:17,220
by scanning them, looking
at scenes and objects,

252
00:10:17,220 --> 00:10:19,350
find that region
in each subject.

253
00:10:19,350 --> 00:10:21,330
Then we would collect
the pattern of response

254
00:10:21,330 --> 00:10:24,690
across voxels in the
PPA while subjects were

255
00:10:24,690 --> 00:10:26,490
looking at, say, beach scenes.

256
00:10:26,490 --> 00:10:28,800
And so if this is the
PPA, this is the pattern

257
00:10:28,800 --> 00:10:31,680
of response across voxels
in that region when they're

258
00:10:31,680 --> 00:10:32,850
looking at beach scenes--

259
00:10:32,850 --> 00:10:35,910
fake data, obviously,
just to give you the idea.

260
00:10:35,910 --> 00:10:39,390
So we would split the data
in half, even runs, odd runs.

261
00:10:39,390 --> 00:10:41,070
That would be like even runs.

262
00:10:41,070 --> 00:10:43,800
Then we get another
pattern for odd runs.

263
00:10:43,800 --> 00:10:46,650
And then we get another
pattern for when they're

264
00:10:46,650 --> 00:10:48,660
looking at city
scenes with even runs,

265
00:10:48,660 --> 00:10:50,327
and another pattern
when they're looking

266
00:10:50,327 --> 00:10:53,770
at city scenes in odd runs.

267
00:10:53,770 --> 00:10:57,090
So then, once we have
those four patterns,

268
00:10:57,090 --> 00:11:01,800
what is the key prediction
if using Haxby's correlation

269
00:11:01,800 --> 00:11:02,380
method?

270
00:11:02,380 --> 00:11:06,300
What is the key prediction if
the PPA, if pattern of response

271
00:11:06,300 --> 00:11:09,030
in the PPA, can discriminate
beach scenes from city scenes?

272
00:11:09,030 --> 00:11:11,160
What should we see
from these patterns?

273
00:11:11,160 --> 00:11:12,638
What's the key prediction?

274
00:11:15,390 --> 00:11:17,870
Claire.

275
00:11:17,870 --> 00:11:21,200
Key prediction-- you have
these four patterns in the PPA,

276
00:11:21,200 --> 00:11:24,230
and now you want to know is
there information in there

277
00:11:24,230 --> 00:11:27,038
that enables you to discriminate
beach scenes from city scenes?

278
00:11:27,038 --> 00:11:28,830
AUDIENCE: Is that like
beach even and beach

279
00:11:28,830 --> 00:11:31,788
odd are more similar than
beach even and city even?

280
00:11:31,788 --> 00:11:32,830
NANCY KANWISHER: Exactly.

281
00:11:32,830 --> 00:11:33,890
Exactly.

282
00:11:33,890 --> 00:11:35,030
Right.

283
00:11:35,030 --> 00:11:38,430
It sounds all complicated and
it's easy to get confused.

284
00:11:38,430 --> 00:11:40,380
But the nub of the
idea is really simple.

285
00:11:40,380 --> 00:11:43,010
It just says, look, the
beach patterns are stable.

286
00:11:43,010 --> 00:11:46,640
We do beach a few times, we get
the same pattern, more or less.

287
00:11:46,640 --> 00:11:49,220
We do city, we get
a different pattern.

288
00:11:49,220 --> 00:11:52,130
And we keep doing city, we get
the same pattern more or less.

289
00:11:52,130 --> 00:11:56,070
And the beach pattern and the
city pattern are different.

290
00:11:56,070 --> 00:11:57,620
So that's the nub of the idea.

291
00:11:57,620 --> 00:12:00,860
And so you can implement it with
decoding methods or the Haxby

292
00:12:00,860 --> 00:12:05,540
versions, just to ask whether
the correlation between two

293
00:12:05,540 --> 00:12:08,480
beach patterns-- beach
even, beach odd--

294
00:12:08,480 --> 00:12:12,140
is more similar than the pattern
between one of the beaches

295
00:12:12,140 --> 00:12:15,290
and one of the cities.

296
00:12:15,290 --> 00:12:19,490
Just asking, are they stably
similar within a category

297
00:12:19,490 --> 00:12:21,945
and stably different
from another category?

298
00:12:21,945 --> 00:12:22,820
Does that make sense?

299
00:12:26,630 --> 00:12:30,050
This is just a variant of this
thing I showed you guys before.

300
00:12:30,050 --> 00:12:34,190
We just harnessed this to
ask whether that region can

301
00:12:34,190 --> 00:12:35,220
discriminate.

302
00:12:35,220 --> 00:12:37,820
OK, and I just said all of this.

303
00:12:37,820 --> 00:12:39,620
If you still feel
shaky on this, there's

304
00:12:39,620 --> 00:12:41,510
a few things you can do.

305
00:12:41,510 --> 00:12:44,930
A version of my little
lecture on this method

306
00:12:44,930 --> 00:12:47,355
is here at my website.

307
00:12:47,355 --> 00:12:48,230
You can look at that.

308
00:12:48,230 --> 00:12:49,940
It's just like six
minutes and it's basically

309
00:12:49,940 --> 00:12:50,790
what I did before.

310
00:12:50,790 --> 00:12:52,970
But if you want to go over
it again, there it is.

311
00:12:52,970 --> 00:12:56,195
You can reread the Haxby paper,
which I know is not super easy,

312
00:12:56,195 --> 00:12:57,570
but it's actually
nicely written.

313
00:12:57,570 --> 00:12:59,737
And if you read it carefully,
it explains the method

314
00:12:59,737 --> 00:13:00,410
pretty clearly.

315
00:13:00,410 --> 00:13:02,672
You can talk to me or a TA.

316
00:13:02,672 --> 00:13:04,130
And we'll get back
to this question

317
00:13:04,130 --> 00:13:06,590
of whether we should
do a whole MATLAB based

318
00:13:06,590 --> 00:13:09,560
problem set on this.

319
00:13:09,560 --> 00:13:12,410
OK, let's move on and
talk about navigation.

320
00:13:15,760 --> 00:13:17,770
This is a Monarch butterfly.

321
00:13:17,770 --> 00:13:20,860
It weighs about half a gram.

322
00:13:20,860 --> 00:13:25,180
And yet, each fall
the Monarch migrates

323
00:13:25,180 --> 00:13:30,550
over 2,000 miles from the USA
and Canada down to Mexico.

324
00:13:30,550 --> 00:13:37,100
In fact, a single Monarch
flies 50 miles in a single day.

325
00:13:37,100 --> 00:13:39,460
It's pretty amazing for
this tiny, little, beautiful

326
00:13:39,460 --> 00:13:41,200
delicate thing.

327
00:13:41,200 --> 00:13:44,980
Even more amazing-- it flies
to a very specific forest

328
00:13:44,980 --> 00:13:47,740
in Mexico that's just
a few acres in size.

329
00:13:47,740 --> 00:13:50,860
And it arrives at that
particular forest.

330
00:13:50,860 --> 00:13:53,080
Now, that's already
amazing, but here's

331
00:13:53,080 --> 00:13:58,870
the part that is just totally
mind blowing and that is--

332
00:13:58,870 --> 00:14:00,700
and it flies back
north in the spring--

333
00:14:00,700 --> 00:14:04,270
and that is that this whole
cycle takes four generations

334
00:14:04,270 --> 00:14:05,900
to complete.

335
00:14:05,900 --> 00:14:09,220
And that means that the Monarch
that starts up in Canada

336
00:14:09,220 --> 00:14:11,560
and flies down to that
forest in Mexico--

337
00:14:11,560 --> 00:14:13,510
one Monarch does that--

338
00:14:13,510 --> 00:14:17,680
is the great-great-grandkid
of his ancestor

339
00:14:17,680 --> 00:14:20,500
that last went on that route.

340
00:14:20,500 --> 00:14:23,138
Put that in your
head and smoke it.

341
00:14:23,138 --> 00:14:24,055
That's pretty amazing.

342
00:14:28,830 --> 00:14:32,430
Consider the female
loggerhead turtle.

343
00:14:32,430 --> 00:14:37,170
She hatches at a beach,
and goes out in the sea,

344
00:14:37,170 --> 00:14:39,780
and swims around in
the sea for 20 years

345
00:14:39,780 --> 00:14:43,290
before she comes back 20
years later for the first time

346
00:14:43,290 --> 00:14:45,060
to the beach that
she hatched at.

347
00:14:48,690 --> 00:14:52,980
Now, it's pretty amazing, but
some mothers miss by 20 miles.

348
00:14:52,980 --> 00:14:55,500
They go to the wrong
island or the wrong beach

349
00:14:55,500 --> 00:14:57,660
on the same island.

350
00:14:57,660 --> 00:15:00,790
And so you might think,
OK, it's pretty good.

351
00:15:00,790 --> 00:15:02,430
It's not amazing.

352
00:15:02,430 --> 00:15:04,140
But here's the thing--

353
00:15:04,140 --> 00:15:05,700
the wrong beach
that those mothers

354
00:15:05,700 --> 00:15:10,050
go to is the exactly right beach
had the Earth's magnetic field

355
00:15:10,050 --> 00:15:12,660
not shifted slightly
over those 20 years.

356
00:15:12,660 --> 00:15:15,188
They're exactly
precise, but they just

357
00:15:15,188 --> 00:15:17,730
don't compensate for the shift
in the Earth's magnetic field.

358
00:15:22,040 --> 00:15:23,330
Here's a bat.

359
00:15:23,330 --> 00:15:27,020
This bat maintains
its sense of direction

360
00:15:27,020 --> 00:15:30,530
even while it flies 30 to
50 miles in a single night

361
00:15:30,530 --> 00:15:33,800
in the dark catching food.

362
00:15:33,800 --> 00:15:36,140
And it maintains its
sense of direction

363
00:15:36,140 --> 00:15:39,320
even though it's flying around
in all different orientations

364
00:15:39,320 --> 00:15:43,550
in three dimensions, and
even as it flips over

365
00:15:43,550 --> 00:15:47,930
and lands to perch on
the surface of a cave.

366
00:15:47,930 --> 00:15:50,240
It doesn't get confused
by being upside down.

367
00:15:53,620 --> 00:15:56,680
This is Cataglyphis,
the Tunisian desert ant.

368
00:15:56,680 --> 00:15:58,070
These guys are amazing.

369
00:15:58,070 --> 00:16:00,940
They crawl around on the
surface of the Tunisian desert

370
00:16:00,940 --> 00:16:03,970
where it's 140 degrees
in the daytime,

371
00:16:03,970 --> 00:16:06,770
and they have to crawl around
up there to forage for food.

372
00:16:06,770 --> 00:16:09,730
And then because it's so damn
hot, as soon as they find food,

373
00:16:09,730 --> 00:16:12,130
they zoom back to their
nest and go down in the nest

374
00:16:12,130 --> 00:16:13,640
where it's cooler.

375
00:16:13,640 --> 00:16:16,840
So here is a track
of Cataglyphis

376
00:16:16,840 --> 00:16:19,870
starting at point
A and foraging.

377
00:16:19,870 --> 00:16:22,600
He's meandering around
looking for food going along

378
00:16:22,600 --> 00:16:25,930
this whole crazy
path to point B.

379
00:16:25,930 --> 00:16:28,230
And then if he finds
food at point B,

380
00:16:28,230 --> 00:16:32,350
boom-- straight line
back exactly to the nest.

381
00:16:32,350 --> 00:16:34,600
Now we might ask,
how does Cataglyphis

382
00:16:34,600 --> 00:16:38,620
keep track as he's doing all
this stuff of where his heading

383
00:16:38,620 --> 00:16:41,373
is back to his nest?

384
00:16:41,373 --> 00:16:42,790
The first thing
you might think of

385
00:16:42,790 --> 00:16:44,260
is things like
what it looks like.

386
00:16:44,260 --> 00:16:48,040
Maybe there are landmarks,
maybe there are odors.

387
00:16:48,040 --> 00:16:50,900
But no, he doesn't use
any of those things.

388
00:16:50,900 --> 00:16:54,850
And we know that because when
scientists who have set up

389
00:16:54,850 --> 00:16:59,140
this measurement device capture
Cataglyphis after he goes out

390
00:16:59,140 --> 00:17:02,230
on this tortuous path and
finds the feeding station,

391
00:17:02,230 --> 00:17:05,230
they capture him and move them
across the desert-- on which

392
00:17:05,230 --> 00:17:07,480
they've drawn all these grid
lines for the convenience

393
00:17:07,480 --> 00:17:09,910
of their experiment-- and
they release them here.

394
00:17:09,910 --> 00:17:11,839
And what does Cataglyphis do?

395
00:17:11,839 --> 00:17:14,035
He goes on the exactly
correct vector--

396
00:17:16,690 --> 00:17:20,680
no landmarks, no relevant
odors, and yet he's

397
00:17:20,680 --> 00:17:24,970
obviously encoded the exact
vector of how to get home.

398
00:17:24,970 --> 00:17:28,329
Think about what that
entails and what's involved.

399
00:17:28,329 --> 00:17:30,610
AUDIENCE: The same vector
with respect to north?

400
00:17:30,610 --> 00:17:32,290
NANCY KANWISHER:
With respect to--

401
00:17:32,290 --> 00:17:37,750
yes, well, with respect to
absolute external direction,

402
00:17:37,750 --> 00:17:38,560
absolutely.

403
00:17:42,400 --> 00:17:45,520
So that's what I just said.

404
00:17:45,520 --> 00:17:50,770
So these feats of animal
navigation are amazing.

405
00:17:50,770 --> 00:17:53,830
And animals have evolved ways
to solve all these problems

406
00:17:53,830 --> 00:17:55,510
unique to their environment.

407
00:17:55,510 --> 00:17:58,570
They've evolved these
abilities because they really

408
00:17:58,570 --> 00:18:04,150
have to be able to find
food, and mates, and shelter.

409
00:18:04,150 --> 00:18:07,960
And this is not just esoterica
in the natural world.

410
00:18:07,960 --> 00:18:10,810
MIT students, too,
need to be able to find

411
00:18:10,810 --> 00:18:15,610
food, and mates, and shelter.

412
00:18:15,610 --> 00:18:19,030
So what is navigation, anyway?

413
00:18:19,030 --> 00:18:21,040
And what does it entail?

414
00:18:21,040 --> 00:18:23,260
Well, I'll argue over
the next two lectures

415
00:18:23,260 --> 00:18:25,690
that there are two fundamental
questions that organisms

416
00:18:25,690 --> 00:18:28,060
need to solve to be
able to navigate.

417
00:18:28,060 --> 00:18:30,610
First one is, where am I?

418
00:18:30,610 --> 00:18:34,750
And the second one is, how do I
get from here to there, A to B,

419
00:18:34,750 --> 00:18:37,420
wherever there is
that you need to get?

420
00:18:37,420 --> 00:18:38,440
So we'll unpack this.

421
00:18:38,440 --> 00:18:41,020
There are many different
facets of each.

422
00:18:41,020 --> 00:18:45,130
But so for example,
if you see this image,

423
00:18:45,130 --> 00:18:48,310
you immediately
know where you are,

424
00:18:48,310 --> 00:18:52,390
and you also know where
to go if, for example, it

425
00:18:52,390 --> 00:18:53,560
starts raining.

426
00:18:53,560 --> 00:18:57,610
You might rush into lobby
7, or if you're hungry,

427
00:18:57,610 --> 00:19:03,040
you might turn around and go
back to the Student Center.

428
00:19:03,040 --> 00:19:05,500
Same deal here--
if you see this,

429
00:19:05,500 --> 00:19:07,660
then you know where you
are and where you would

430
00:19:07,660 --> 00:19:10,990
go to get to various things.

431
00:19:10,990 --> 00:19:14,050
Now, these judgments rely
on the specific knowledge

432
00:19:14,050 --> 00:19:16,390
you guys have of those
particular places.

433
00:19:16,390 --> 00:19:19,323
You recognize that
exact place, and you

434
00:19:19,323 --> 00:19:20,740
have some kind of
map in your head

435
00:19:20,740 --> 00:19:22,573
that we'll talk more
about in a moment, that

436
00:19:22,573 --> 00:19:25,420
tells you where everything
else is with respect to it.

437
00:19:25,420 --> 00:19:28,420
But even if you're in a
place you don't know at all

438
00:19:28,420 --> 00:19:30,460
you can still extract
some information.

439
00:19:30,460 --> 00:19:34,750
So suppose you miraculously
found yourself-- boom-- here.

440
00:19:34,750 --> 00:19:37,000
I wouldn't mind,
actually, but that's not

441
00:19:37,000 --> 00:19:39,110
in the cards for a while.

442
00:19:39,110 --> 00:19:40,357
So you're here.

443
00:19:40,357 --> 00:19:42,190
Even if you've just
hiked around the corner,

444
00:19:42,190 --> 00:19:44,530
if you've never seen
this place before,

445
00:19:44,530 --> 00:19:47,740
you have some kind of idea of
what sort of place this is.

446
00:19:47,740 --> 00:19:50,410
Where would you pitch your tent?

447
00:19:50,410 --> 00:19:53,110
Where might you try to go
to get out of this valley?

448
00:19:53,110 --> 00:19:54,430
If it was me, I wouldn't.

449
00:19:54,430 --> 00:19:56,500
I have friends who would
go straight up there

450
00:19:56,500 --> 00:19:58,780
and try to drag me
along, complaining.

451
00:19:58,780 --> 00:20:01,750
If it was me, I'd rather
look for some other route.

452
00:20:01,750 --> 00:20:06,713
But you can tell all of that
just by looking at this image--

453
00:20:06,713 --> 00:20:09,130
where you can go from there,
not just what kind of a place

454
00:20:09,130 --> 00:20:13,480
it is, but what are the
possible routes you might take.

455
00:20:13,480 --> 00:20:15,760
So these fundamental
problems that we

456
00:20:15,760 --> 00:20:17,830
solve in navigation,
of knowing where am I

457
00:20:17,830 --> 00:20:20,650
and how do I get
from here to there,

458
00:20:20,650 --> 00:20:22,840
include multiple components.

459
00:20:22,840 --> 00:20:25,600
In terms of where am
I, the first piece

460
00:20:25,600 --> 00:20:29,740
is recognizing a
specific place you know.

461
00:20:29,740 --> 00:20:32,170
So you might open your
eyes and say, OK, this

462
00:20:32,170 --> 00:20:32,980
is my living room.

463
00:20:32,980 --> 00:20:35,710
I know this particular place.

464
00:20:35,710 --> 00:20:38,800
But as I just pointed out, even
if the place is unfamiliar,

465
00:20:38,800 --> 00:20:41,890
we can get a sense of what
kind of place this is.

466
00:20:41,890 --> 00:20:44,350
Am I in an urban environment,
a natural environment,

467
00:20:44,350 --> 00:20:45,910
a living room, a bathroom?

468
00:20:45,910 --> 00:20:48,190
Where am I?

469
00:20:48,190 --> 00:20:50,800
A third aspect of
where am I, a third way

470
00:20:50,800 --> 00:20:52,390
that we might answer
that question,

471
00:20:52,390 --> 00:20:55,790
is something about the geometry
of the environment we're in.

472
00:20:55,790 --> 00:20:59,470
So try this right
now-- close your eyes.

473
00:20:59,470 --> 00:21:03,830
OK, now think about how far
the wall is in front of you.

474
00:21:03,830 --> 00:21:05,980
Don't open your eyes, just
think about how far away

475
00:21:05,980 --> 00:21:10,370
it is, how far away the left
wall is and the right wall is.

476
00:21:10,370 --> 00:21:11,920
And how about the
wall behind you?

477
00:21:11,920 --> 00:21:12,880
Don't open your eyes.

478
00:21:12,880 --> 00:21:14,950
How far back is
the wall behind you

479
00:21:14,950 --> 00:21:17,485
from where you are right now?

480
00:21:17,485 --> 00:21:18,610
OK, you can open your eyes.

481
00:21:18,610 --> 00:21:19,720
It's not rocket science.

482
00:21:19,720 --> 00:21:23,500
I just wanted you to intuit that
even though you're presumably

483
00:21:23,500 --> 00:21:26,660
riveted by this lecture, and
thinking only about navigation,

484
00:21:26,660 --> 00:21:29,230
you sort have a kind of
situational awareness

485
00:21:29,230 --> 00:21:32,560
of the spatial layout
of the space you're in.

486
00:21:32,560 --> 00:21:35,830
So you might have a sense
of I'm in a space like this

487
00:21:35,830 --> 00:21:38,560
and I'm over here in it.

488
00:21:38,560 --> 00:21:41,680
And we'll talk more about
that exact kind of awareness

489
00:21:41,680 --> 00:21:44,890
of your position relative
to the spatial layout

490
00:21:44,890 --> 00:21:46,330
of your immediate environment.

491
00:21:46,330 --> 00:21:50,110
It's something that's very
important in navigation.

492
00:21:50,110 --> 00:21:52,930
And another part of
that is you might think,

493
00:21:52,930 --> 00:21:54,190
how would I get out of here?

494
00:21:54,190 --> 00:21:56,260
If I'm seriously
bored by the lecture

495
00:21:56,260 --> 00:21:58,150
or for any other
reason I urgently

496
00:21:58,150 --> 00:21:59,860
need to get out of
here, you probably

497
00:21:59,860 --> 00:22:02,170
know exactly where the
doors are in this space.

498
00:22:02,170 --> 00:22:07,930
It's just part one of those
things that we keep track of.

499
00:22:07,930 --> 00:22:10,822
So those are aspects of
where am I in this place.

500
00:22:10,822 --> 00:22:12,280
What are the things
we need to know

501
00:22:12,280 --> 00:22:16,090
to know how we would get
from here to someplace else?

502
00:22:16,090 --> 00:22:19,570
Well, the simplest way to
navigate to another location

503
00:22:19,570 --> 00:22:21,880
another goal is
called "beaconing."

504
00:22:21,880 --> 00:22:24,880
And this is a case where
you can directly see or hear

505
00:22:24,880 --> 00:22:26,390
your target location.

506
00:22:26,390 --> 00:22:27,772
So you're sailing in the fog.

507
00:22:27,772 --> 00:22:29,230
You can't see a
damn thing, but you

508
00:22:29,230 --> 00:22:31,000
hear the foghorn over
there, and you know

509
00:22:31,000 --> 00:22:32,480
you're sailing to that point.

510
00:22:32,480 --> 00:22:35,330
So you just go toward the
sound, nice and simple.

511
00:22:35,330 --> 00:22:37,330
You don't need any broader
map of anything else.

512
00:22:37,330 --> 00:22:41,050
You just hear it
and head toward it.

513
00:22:41,050 --> 00:22:44,800
Or if you see
this, and your goal

514
00:22:44,800 --> 00:22:48,760
is to get to the green building,
well, there's a green building

515
00:22:48,760 --> 00:22:49,960
and you just head that way.

516
00:22:49,960 --> 00:22:52,085
Now, you're going to have
to go around a little bit

517
00:22:52,085 --> 00:22:53,770
to get around those
obstacles, but you

518
00:22:53,770 --> 00:22:57,520
know where to head because you
can see your target directly.

519
00:22:57,520 --> 00:22:59,140
These are cases
where you don't need

520
00:22:59,140 --> 00:23:02,620
a broader, long-term knowledge
of the whole environment.

521
00:23:02,620 --> 00:23:06,070
If you can see your target,
you just go straight for it.

522
00:23:06,070 --> 00:23:10,690
So that's beaconing,
simplest kind of A to B.

523
00:23:10,690 --> 00:23:12,970
And it requires no
mental map, no kind

524
00:23:12,970 --> 00:23:16,480
of internal model of the whole
world you're navigating in.

525
00:23:16,480 --> 00:23:19,600
But if you can't see the
place you want to go,

526
00:23:19,600 --> 00:23:23,140
then you need some kind of
mental map of the world.

527
00:23:23,140 --> 00:23:25,780
So what do we mean by a
"mental map of the world?"

528
00:23:25,780 --> 00:23:28,330
Well, this idea was
first articulated

529
00:23:28,330 --> 00:23:31,960
in a classic experiment
way back in the 1940s.

530
00:23:31,960 --> 00:23:35,350
So this was actually one of
the original experiments that

531
00:23:35,350 --> 00:23:38,170
launched the cognitive
revolution, when we emerged

532
00:23:38,170 --> 00:23:42,610
from the scourge of behaviorism
to realize that it was actually

533
00:23:42,610 --> 00:23:45,940
OK, and indeed, of the
essence, to talk about what's

534
00:23:45,940 --> 00:23:47,380
going on in the mind.

535
00:23:47,380 --> 00:23:49,600
And a really
influential study that

536
00:23:49,600 --> 00:23:53,020
launched the cognitive
revolution by Tolman

537
00:23:53,020 --> 00:23:54,290
was done on rats.

538
00:23:54,290 --> 00:23:56,260
And it went like
this-- he trained rats.

539
00:23:56,260 --> 00:23:58,570
He put them down in
this area, and they

540
00:23:58,570 --> 00:24:02,783
had to learn that there would
be food out there at the goal.

541
00:24:02,783 --> 00:24:05,200
And so they just have to make
the series of left and right

542
00:24:05,200 --> 00:24:07,330
turns to find the food.

543
00:24:07,330 --> 00:24:09,580
So you train them
on that for a while

544
00:24:09,580 --> 00:24:11,320
till they're really good at it.

545
00:24:11,320 --> 00:24:15,250
And then he put the rats
in this environment.

546
00:24:15,250 --> 00:24:18,670
Now, the environment is
similar, except there's

547
00:24:18,670 --> 00:24:23,270
multiple paths, one that seems
analogous to the old route.

548
00:24:23,270 --> 00:24:25,570
So what are the rats
do in this situation?

549
00:24:25,570 --> 00:24:28,630
They run down here,
they run into a wall,

550
00:24:28,630 --> 00:24:31,420
and they realize, OK,
that's not going to work.

551
00:24:31,420 --> 00:24:33,070
No surprises yet.

552
00:24:33,070 --> 00:24:36,400
But then, the rats
immediately come back out

553
00:24:36,400 --> 00:24:38,650
and they go straight
out that way.

554
00:24:41,560 --> 00:24:43,900
What does that tell you?

555
00:24:43,900 --> 00:24:45,620
What did they learn?

556
00:24:45,620 --> 00:24:48,400
Did they learn a series of
go straight, and then left,

557
00:24:48,400 --> 00:24:51,190
and then right, and then right,
and then go for a long ways?

558
00:24:51,190 --> 00:24:52,030
No.

559
00:24:52,030 --> 00:24:53,830
That wouldn't work over here.

560
00:24:53,830 --> 00:24:56,110
They learned something
much more interesting.

561
00:24:56,110 --> 00:24:59,560
Even though they were only
being trained on this task here,

562
00:24:59,560 --> 00:25:02,560
they learned some much
more interesting thing

563
00:25:02,560 --> 00:25:07,030
about the kind of vector
average of all of those turns.

564
00:25:07,030 --> 00:25:08,140
Everybody get this?

565
00:25:08,140 --> 00:25:11,350
It's really simple
but really deep.

566
00:25:11,350 --> 00:25:14,440
So from this, Tolman
and others started

567
00:25:14,440 --> 00:25:16,420
talking about cognitive
maps, whatever

568
00:25:16,420 --> 00:25:19,850
it is you have to have learned
in a situation like this

569
00:25:19,850 --> 00:25:23,200
so you can abstract
the general direction.

570
00:25:23,200 --> 00:25:24,760
We don't just learn
specific routes

571
00:25:24,760 --> 00:25:29,100
as a series of
stimulus and responses.

572
00:25:29,100 --> 00:25:31,020
So there must be some
kind of map in your head

573
00:25:31,020 --> 00:25:36,630
to be able to do this, and
rats have that, and so do you.

574
00:25:36,630 --> 00:25:38,880
So let's consider this
question right now.

575
00:25:38,880 --> 00:25:39,720
Where am I?

576
00:25:39,720 --> 00:25:42,240
Where are you?

577
00:25:42,240 --> 00:25:43,980
To answer that
question to yourself,

578
00:25:43,980 --> 00:25:46,650
there's something like
this in your head.

579
00:25:46,650 --> 00:25:49,710
And it probably doesn't look
exactly like that in your head,

580
00:25:49,710 --> 00:25:54,570
but there's some version of this
information that's in your head

581
00:25:54,570 --> 00:25:56,010
that you're using
when you answer

582
00:25:56,010 --> 00:25:57,260
the question of where you are.

583
00:26:00,570 --> 00:26:03,780
And you have some way to say
in that map of the world,

584
00:26:03,780 --> 00:26:06,570
I know not just what the
MIT campus looks like

585
00:26:06,570 --> 00:26:09,450
and how it's arranged, but
I know where I am in it.

586
00:26:12,690 --> 00:26:15,473
Now, if you want to know
how to get somewhere else--

587
00:26:15,473 --> 00:26:16,890
like suppose you're
hungry and you

588
00:26:16,890 --> 00:26:20,620
want to go over to the
Stata Cafeteria over there.

589
00:26:20,620 --> 00:26:23,130
What else do you need to
know besides knowledge

590
00:26:23,130 --> 00:26:26,500
of the map of your environment
and where you are in it?

591
00:26:26,500 --> 00:26:27,750
What else do you need to know?

592
00:26:31,800 --> 00:26:34,380
You know you have this map,
you know where you are,

593
00:26:34,380 --> 00:26:35,790
and you know where your goal is.

594
00:26:35,790 --> 00:26:37,750
Now you have to plan
how to get over there.

595
00:26:37,750 --> 00:26:40,440
What else do you need to know?

596
00:26:40,440 --> 00:26:40,940
Yeah.

597
00:26:40,940 --> 00:26:42,940
AUDIENCE: You have to
know which parts are paths

598
00:26:42,940 --> 00:26:44,490
and which parts are buildings.

599
00:26:44,490 --> 00:26:46,907
NANCY KANWISHER: Yes, exactly--
where can you go in there?

600
00:26:46,907 --> 00:26:49,590
Actually, where can you
physically get through?

601
00:26:49,590 --> 00:26:51,540
Actually, our vector
is right over there,

602
00:26:51,540 --> 00:26:53,370
but you can't go
that way because you

603
00:26:53,370 --> 00:26:55,260
can't go through
that glass, even

604
00:26:55,260 --> 00:26:56,730
though you can see through it.

605
00:26:56,730 --> 00:26:58,830
So knowledge of
physical barriers,

606
00:26:58,830 --> 00:27:01,890
and what's an actual path
and what isn't is crucial.

607
00:27:01,890 --> 00:27:05,030
What else do you need to know?

608
00:27:05,030 --> 00:27:07,050
Suppose we had a
robot in this room,

609
00:27:07,050 --> 00:27:10,370
sitting right here facing the
front of a room like you guys,

610
00:27:10,370 --> 00:27:13,567
and we're programming the
robot on how to get over there.

611
00:27:13,567 --> 00:27:15,650
What are other things we'd
have to tell that robot

612
00:27:15,650 --> 00:27:19,760
to get it to plan how to get
over to the Stata Cafeteria?

613
00:27:22,610 --> 00:27:23,240
Yeah.

614
00:27:23,240 --> 00:27:25,850
AUDIENCE: Things to watch out
for, like cars and traffic.

615
00:27:25,850 --> 00:27:26,660
NANCY KANWISHER: Absolutely.

616
00:27:26,660 --> 00:27:28,580
We'd have to know about
obstacles, like moving

617
00:27:28,580 --> 00:27:30,290
obstacles, not just fixed ones.

618
00:27:30,290 --> 00:27:31,190
Absolutely.

619
00:27:31,190 --> 00:27:32,400
What else?

620
00:27:32,400 --> 00:27:32,900
Yeah.

621
00:27:32,900 --> 00:27:34,575
AUDIENCE: Initial orientation.

622
00:27:34,575 --> 00:27:35,450
NANCY KANWISHER: Yes.

623
00:27:35,450 --> 00:27:37,277
He has to know which
way he's headed.

624
00:27:37,277 --> 00:27:39,110
You're going to give
this robot instructions

625
00:27:39,110 --> 00:27:40,470
on which way to go.

626
00:27:40,470 --> 00:27:43,850
It matters a whole lot if the
robot is starting like this

627
00:27:43,850 --> 00:27:45,170
or starting like that.

628
00:27:45,170 --> 00:27:48,350
The instructions are different
in the two cases, and likewise

629
00:27:48,350 --> 00:27:49,220
for you guys.

630
00:27:49,220 --> 00:27:53,600
To plan a route, you need to
know which way you're heading.

631
00:27:53,600 --> 00:27:55,815
If you guys have ever
been in Manhattan,

632
00:27:55,815 --> 00:27:58,190
and you come up from the
subway, and you see the street's

633
00:27:58,190 --> 00:28:00,148
going like this, and you
know it's north/south,

634
00:28:00,148 --> 00:28:02,690
and you don't know if you're
heading south or north--

635
00:28:02,690 --> 00:28:05,120
really common thing.

636
00:28:05,120 --> 00:28:09,650
It's not enough to know I'm at
the junction of Fifth and 22nd.

637
00:28:09,650 --> 00:28:12,110
You need to know, am I
facing south or north?

638
00:28:12,110 --> 00:28:14,323
Otherwise you can't figure
out which way to go.

639
00:28:14,323 --> 00:28:15,740
That's called
"heading direction."

640
00:28:18,810 --> 00:28:20,390
We just did all that.

641
00:28:20,390 --> 00:28:23,150
You need to know
your current heading.

642
00:28:23,150 --> 00:28:26,690
You also need to know the
direction of your goal

643
00:28:26,690 --> 00:28:29,780
in order to plan a route to it.

644
00:28:29,780 --> 00:28:32,510
So in this kind of
taxonomy of all the things

645
00:28:32,510 --> 00:28:34,850
you need to know
to navigate, we've

646
00:28:34,850 --> 00:28:37,460
just added that if
you're going to navigate

647
00:28:37,460 --> 00:28:39,200
in your own
environment, you need

648
00:28:39,200 --> 00:28:42,020
to know not just
where you are in it,

649
00:28:42,020 --> 00:28:46,310
but which way you are
facing in that mental map.

650
00:28:46,310 --> 00:28:48,290
And we also talked
about this business

651
00:28:48,290 --> 00:28:50,070
of what routes are
possible from here,

652
00:28:50,070 --> 00:28:52,850
how do we move around
obstacles, where

653
00:28:52,850 --> 00:28:58,100
are the doors, where are the
hazards like cars, et cetera.

654
00:28:58,100 --> 00:29:00,380
A final thing you
need to know is

655
00:29:00,380 --> 00:29:02,450
that even if you have
a good system for all

656
00:29:02,450 --> 00:29:05,150
of these other bits, it's
still possible to get

657
00:29:05,150 --> 00:29:06,890
lost in all kinds of ways.

658
00:29:06,890 --> 00:29:10,590
You lose track, you get
confused, you get lost.

659
00:29:10,590 --> 00:29:13,880
So we also need a way
to reorient ourselves

660
00:29:13,880 --> 00:29:15,180
when we're lost.

661
00:29:15,180 --> 00:29:18,020
And we'll talk a lot about
that in the next lecture.

662
00:29:18,020 --> 00:29:19,550
So this is just common sense.

663
00:29:19,550 --> 00:29:21,200
We're doing a kind
of low-tech version

664
00:29:21,200 --> 00:29:24,350
of Marr computational
theory for navigation.

665
00:29:24,350 --> 00:29:26,330
What are the things that
we would need to know

666
00:29:26,330 --> 00:29:29,870
or that a robot would need to
know to be able to navigate?

667
00:29:29,870 --> 00:29:33,860
Just thinking about the
nature of the problem.

668
00:29:33,860 --> 00:29:35,600
So that's what we need.

669
00:29:35,600 --> 00:29:39,170
What's the neural
basis of all of this?

670
00:29:39,170 --> 00:29:41,750
So I'm going to start right in
with the parahippocampal place

671
00:29:41,750 --> 00:29:45,290
area, not to imply it is
the total neural basis

672
00:29:45,290 --> 00:29:46,165
of this whole thing.

673
00:29:46,165 --> 00:29:48,290
It's just one little piece
of a much bigger puzzle.

674
00:29:48,290 --> 00:29:52,940
But we'll start in there
because it's nice and concrete.

675
00:29:52,940 --> 00:29:56,930
All right, so this story
starts about 20 years ago.

676
00:29:56,930 --> 00:29:59,180
I think I mentioned some
of this in the first class

677
00:29:59,180 --> 00:30:00,920
when I talked about
the story of Bob

678
00:30:00,920 --> 00:30:02,430
and I talked about
Russell Epstein,

679
00:30:02,430 --> 00:30:04,160
who was then my post-doc.

680
00:30:04,160 --> 00:30:06,110
And he was doing nice
behavioral experiments,

681
00:30:06,110 --> 00:30:08,880
and thought it was trashy and
cheap to mess around with brain

682
00:30:08,880 --> 00:30:09,380
imaging.

683
00:30:09,380 --> 00:30:10,838
And he was going
to have none of it

684
00:30:10,838 --> 00:30:12,830
until I said, Russell,
just do one experiment.

685
00:30:12,830 --> 00:30:14,660
Scan subjects looking at scenes.

686
00:30:14,660 --> 00:30:16,580
I know it's kind of
stupid, but just do it.

687
00:30:16,580 --> 00:30:18,710
Then you'll have a
slide for your job talk.

688
00:30:18,710 --> 00:30:21,470
And he scanned subjects
looking at scenes

689
00:30:21,470 --> 00:30:22,880
and looking at objects.

690
00:30:22,880 --> 00:30:25,580
And here is one of those
early subjects, probably me--

691
00:30:25,580 --> 00:30:28,430
I don't remember-- with a
bunch of vertical slices

692
00:30:28,430 --> 00:30:31,220
through the brain, near the
back of the brain down there,

693
00:30:31,220 --> 00:30:33,410
moving forward as
we go up to here.

694
00:30:33,410 --> 00:30:36,410
Everybody oriented?

695
00:30:36,410 --> 00:30:38,660
Sorry, it's not showing up
very well in this lighting,

696
00:30:38,660 --> 00:30:41,300
but there's a little
bilateral region

697
00:30:41,300 --> 00:30:44,400
right in the middle there that
shows a stronger response when

698
00:30:44,400 --> 00:30:46,400
people look at pictures
of scenes than when they

699
00:30:46,400 --> 00:30:49,550
look at pictures of objects.

700
00:30:49,550 --> 00:30:52,290
So we hadn't predicted this.

701
00:30:52,290 --> 00:30:52,790
Yeah.

702
00:30:52,790 --> 00:30:54,633
AUDIENCE: Is the
pink the eye color?

703
00:30:54,633 --> 00:30:55,550
NANCY KANWISHER: Yeah.

704
00:30:55,550 --> 00:30:58,970
Yeah, pink is-- all
the colors are--

705
00:30:58,970 --> 00:31:01,160
there's significance
maps or P levels, right.

706
00:31:01,160 --> 00:31:06,680
So pink is higher than blue, but
blue is borderline significant.

707
00:31:06,680 --> 00:31:08,407
So this is kind of dopey.

708
00:31:08,407 --> 00:31:10,490
We didn't actually predict
it for any deep reason.

709
00:31:10,490 --> 00:31:12,793
We hadn't been thinking
about theories of navigation

710
00:31:12,793 --> 00:31:13,710
or anything like that.

711
00:31:13,710 --> 00:31:15,418
It was just one of
those dumb experiments

712
00:31:15,418 --> 00:31:18,290
where we found something
and we followed the data.

713
00:31:18,290 --> 00:31:19,970
So we found this,
and it's, like, OK,

714
00:31:19,970 --> 00:31:22,020
let's try some other subjects.

715
00:31:22,020 --> 00:31:24,590
So here are the first
nine subjects we scanned.

716
00:31:24,590 --> 00:31:28,910
Every single subject had that
kind of signature response

717
00:31:28,910 --> 00:31:33,290
in exactly the same place,
in a part of the brain called

718
00:31:33,290 --> 00:31:36,140
"parahippocampal cortex."

719
00:31:36,140 --> 00:31:37,970
So this is very systematic.

720
00:31:37,970 --> 00:31:40,640
And there's lots of ways to
make progress in science.

721
00:31:40,640 --> 00:31:42,590
One way is to have a
big theory, and use it

722
00:31:42,590 --> 00:31:46,010
to motivate brilliant,
elegantly designed experiments.

723
00:31:46,010 --> 00:31:48,770
And another is you just see
something salient and robust

724
00:31:48,770 --> 00:31:51,200
that you didn't predict,
and you follow your nose,

725
00:31:51,200 --> 00:31:52,470
and try to figure it out.

726
00:31:52,470 --> 00:31:53,928
So that's what we
did in this case.

727
00:31:53,928 --> 00:31:56,810
It's like, OK, what
the hell is that?

728
00:31:56,810 --> 00:31:59,630
So if you think about--

729
00:31:59,630 --> 00:32:02,480
we eventually called it
the "parahippocampal place

730
00:32:02,480 --> 00:32:04,460
area" after a little more work.

731
00:32:04,460 --> 00:32:06,570
If you think about
what we have so far,

732
00:32:06,570 --> 00:32:09,530
we've scanned people looking at
pictures like this and pictures

733
00:32:09,530 --> 00:32:10,327
like that.

734
00:32:10,327 --> 00:32:12,410
And what we've shown is
that little patch of brain

735
00:32:12,410 --> 00:32:16,010
responds a bunch more
to these than those.

736
00:32:16,010 --> 00:32:18,770
So my first question is,
is that a minimal pair?

737
00:32:22,710 --> 00:32:24,390
Tally, is that a minimal pair?

738
00:32:24,390 --> 00:32:26,760
AUDIENCE: Sorry,
I'm about my voice.

739
00:32:26,760 --> 00:32:28,050
NANCY KANWISHER: Sorry.

740
00:32:28,050 --> 00:32:29,250
Simple, simple.

741
00:32:29,250 --> 00:32:31,205
We're contrasting
this with that.

742
00:32:31,205 --> 00:32:33,330
AUDIENCE: Can you remind
me what a minimal pair is?

743
00:32:33,330 --> 00:32:34,705
NANCY KANWISHER:
OK, minimal pair

744
00:32:34,705 --> 00:32:37,320
is this thing we aspire
towards an experimental design,

745
00:32:37,320 --> 00:32:39,000
where we have two
conditions that

746
00:32:39,000 --> 00:32:41,010
are identical except
for one little thing

747
00:32:41,010 --> 00:32:44,255
we're manipulating.

748
00:32:44,255 --> 00:32:46,380
AUDIENCE: I don't really
think it's a minimal pair,

749
00:32:46,380 --> 00:32:48,337
but I'm not really sure.

750
00:32:48,337 --> 00:32:49,920
NANCY KANWISHER:
Well, I even told you

751
00:32:49,920 --> 00:32:51,870
what we were designing
to manipulate, but--

752
00:32:51,870 --> 00:32:54,300
AUDIENCE: There seems to
be too many differences

753
00:32:54,300 --> 00:32:56,730
between a living room and--

754
00:32:56,730 --> 00:32:58,560
NANCY KANWISHER: It's ludicrous.

755
00:32:58,560 --> 00:33:00,960
I mean, it's a million
differences here.

756
00:33:00,960 --> 00:33:03,240
So we don't know that
we have anything yet.

757
00:33:03,240 --> 00:33:05,250
There's all kinds of
uninteresting accounts

758
00:33:05,250 --> 00:33:08,550
of this systematic activation
in that part of the brain.

759
00:33:08,550 --> 00:33:10,530
So just to list a few
that you've probably

760
00:33:10,530 --> 00:33:12,090
already noticed--

761
00:33:12,090 --> 00:33:14,940
these things have rich,
high-level meaning

762
00:33:14,940 --> 00:33:16,200
and complexity.

763
00:33:16,200 --> 00:33:18,370
So you can think
about living rooms,

764
00:33:18,370 --> 00:33:23,610
or where you might sit, or
somebody's aesthetic, home

765
00:33:23,610 --> 00:33:25,740
design, or there's
all kinds of stuff

766
00:33:25,740 --> 00:33:29,520
to think about there, much more
than just, OK, it's a blender.

767
00:33:29,520 --> 00:33:33,430
So there's just complexity
in every possible way.

768
00:33:33,430 --> 00:33:37,200
There are also lots of
objects present here,

769
00:33:37,200 --> 00:33:39,400
and only a single
object over there.

770
00:33:39,400 --> 00:33:41,700
So maybe that region
just represents objects,

771
00:33:41,700 --> 00:33:45,990
and if you have more objects,
you get a higher signal.

772
00:33:45,990 --> 00:33:48,330
There's another
possibility, and that

773
00:33:48,330 --> 00:33:51,390
is that these images
depict spatial layout,

774
00:33:51,390 --> 00:33:53,490
and that one does not.

775
00:33:53,490 --> 00:33:55,890
So you have some sense of
the walls, and the floor,

776
00:33:55,890 --> 00:33:58,230
and the layout of
the local environment

777
00:33:58,230 --> 00:34:01,320
here that you don't
have over there.

778
00:34:01,320 --> 00:34:04,410
And we could probably list
a million other things It's

779
00:34:04,410 --> 00:34:07,350
a very, very sloppy contrast.

780
00:34:07,350 --> 00:34:10,440
So how are we going to
ask which of these things

781
00:34:10,440 --> 00:34:13,469
might be driving the
response of that region?

782
00:34:13,469 --> 00:34:18,820
Well, a natural thing to do is
just deconstruct the stimuli.

783
00:34:18,820 --> 00:34:20,520
So here's what we did--

784
00:34:20,520 --> 00:34:22,520
this is actually way
back 20 years ago.

785
00:34:22,520 --> 00:34:25,330
There were better methods at the
time, but I didn't know them,

786
00:34:25,330 --> 00:34:27,000
so I actually drove
around Cambridge,

787
00:34:27,000 --> 00:34:28,920
photographed my
friends' apartments,

788
00:34:28,920 --> 00:34:31,165
left the camera on
the same tripod,

789
00:34:31,165 --> 00:34:32,790
moved all the furniture
out of the way,

790
00:34:32,790 --> 00:34:34,080
and photographed
the space again.

791
00:34:34,080 --> 00:34:34,590
Ha, ha.

792
00:34:34,590 --> 00:34:36,900
I know.

793
00:34:36,900 --> 00:34:38,550
And then these will
be probably cut out

794
00:34:38,550 --> 00:34:41,909
with some horrific version
of Adobe Photoshop that

795
00:34:41,909 --> 00:34:43,659
existed 20 years ago.

796
00:34:43,659 --> 00:34:46,170
Anyway, we
deconstructed the scenes

797
00:34:46,170 --> 00:34:49,852
into their component objects
and the bare spatial layout.

798
00:34:49,852 --> 00:34:51,060
Everybody get the logic here?

799
00:34:51,060 --> 00:34:54,300
Just to try to make a big cut
in this hypothesis space of what

800
00:34:54,300 --> 00:34:57,850
might be driving that region.

801
00:34:57,850 --> 00:35:01,360
So what do we predict
that the PPA will--

802
00:35:01,360 --> 00:35:02,950
how strongly will it respond?

803
00:35:05,670 --> 00:35:09,330
Oops, how strongly
will it respond

804
00:35:09,330 --> 00:35:10,800
if these two things are true?

805
00:35:10,800 --> 00:35:14,280
If it's the complexity
or multiplicity

806
00:35:14,280 --> 00:35:16,860
of objects that's
driving it, what do you

807
00:35:16,860 --> 00:35:18,390
predict we will see over there?

808
00:35:18,390 --> 00:35:20,520
We already know you get
a high response here.

809
00:35:20,520 --> 00:35:21,750
What will we get over there?

810
00:35:26,630 --> 00:35:27,200
Yeah.

811
00:35:27,200 --> 00:35:29,240
AUDIENCE: Probably get more
biases to the furniture.

812
00:35:29,240 --> 00:35:31,220
NANCY KANWISHER: Yeah, we'll
respond more to this than that.

813
00:35:31,220 --> 00:35:31,940
Right.

814
00:35:31,940 --> 00:35:35,360
It's really simple-minded.

815
00:35:35,360 --> 00:35:39,920
If instead, it responds
more to the spatial layout,

816
00:35:39,920 --> 00:35:41,840
what do we predict?

817
00:35:41,840 --> 00:35:42,380
Isabel.

818
00:35:42,380 --> 00:35:44,713
AUDIENCE: It's going to respond
to the empty rooms more.

819
00:35:44,713 --> 00:35:46,630
NANCY KANWISHER: Yeah.

820
00:35:46,630 --> 00:35:48,460
And that seems like
a weird hypothesis

821
00:35:48,460 --> 00:35:51,070
because these are really boring,
this kind of nothing going on

822
00:35:51,070 --> 00:35:51,440
here.

823
00:35:51,440 --> 00:35:53,330
And there's just lots
of stuff going on here.

824
00:35:53,330 --> 00:35:55,030
I mean, it's not
riveting, but it's

825
00:35:55,030 --> 00:35:57,490
a whole bunch, whole lot
more interesting to look

826
00:35:57,490 --> 00:35:58,730
at these than those.

827
00:35:58,730 --> 00:36:00,880
Believe me, I got scanned
for hours and hours

828
00:36:00,880 --> 00:36:01,927
looking at these things.

829
00:36:01,927 --> 00:36:03,760
And whenever the empty
rooms came on, I was,

830
00:36:03,760 --> 00:36:05,297
like, oh, my god,
I'm just so bored.

831
00:36:05,297 --> 00:36:07,630
There's just nothing here,
whereas here at least there's

832
00:36:07,630 --> 00:36:10,260
stuff.

833
00:36:10,260 --> 00:36:13,560
But that's not what
the PPA thinks.

834
00:36:13,560 --> 00:36:14,970
What the PPA does--

835
00:36:14,970 --> 00:36:19,380
oops, we just did
the localizer--

836
00:36:19,380 --> 00:36:20,445
it responds like this.

837
00:36:20,445 --> 00:36:22,260
This is percent signal
change, a measure

838
00:36:22,260 --> 00:36:27,960
of magnitude of response, to
the full scenes, way down,

839
00:36:27,960 --> 00:36:30,270
less than half the response
to all those objects,

840
00:36:30,270 --> 00:36:34,270
and almost the same response
as the original scene

841
00:36:34,270 --> 00:36:36,270
when all you have is
a bare spatial layout.

842
00:36:39,060 --> 00:36:41,250
Pretty surprising, isn't it?

843
00:36:41,250 --> 00:36:42,090
We were blown away.

844
00:36:42,090 --> 00:36:43,460
We were, like, what?

845
00:36:43,460 --> 00:36:46,110
What?

846
00:36:46,110 --> 00:36:49,440
But can you see how even this
really simple-minded experiment

847
00:36:49,440 --> 00:36:51,630
enables us to just
pretty much rule out

848
00:36:51,630 --> 00:36:53,160
that whole space of hypotheses?

849
00:36:53,160 --> 00:36:55,380
It's not about the
richness, or interest,

850
00:36:55,380 --> 00:36:57,670
or multiplicity of objects.

851
00:36:57,670 --> 00:36:59,670
It's something much
more like spatial layout

852
00:36:59,670 --> 00:37:03,180
because that's kind of all
there is in those empty rooms.

853
00:37:03,180 --> 00:37:05,880
I mean, it could be something
like the texture of wood floors

854
00:37:05,880 --> 00:37:08,370
or something weird like that.

855
00:37:08,370 --> 00:37:10,920
But one's first guess is it's
something about spatial layout.

856
00:37:10,920 --> 00:37:12,140
Does this make sense?

857
00:37:12,140 --> 00:37:15,510
It's just a way to take
a big, sloppy contrast,

858
00:37:15,510 --> 00:37:18,240
and try to formulate
initial hypotheses,

859
00:37:18,240 --> 00:37:21,090
and knock out a whole
big space of hypotheses.

860
00:37:21,090 --> 00:37:21,600
Yes.

861
00:37:21,600 --> 00:37:22,650
Is it Alana?

862
00:37:22,650 --> 00:37:25,720
AUDIENCE: Yeah, I'm sorry, I
might have missed the design.

863
00:37:25,720 --> 00:37:28,410
So people who are
looking at the empty room

864
00:37:28,410 --> 00:37:31,648
would not have the furniture?

865
00:37:31,648 --> 00:37:32,940
NANCY KANWISHER: Good question.

866
00:37:32,940 --> 00:37:35,220
I skipped over all of that.

867
00:37:35,220 --> 00:37:37,650
We did-- yes, that's true.

868
00:37:37,650 --> 00:37:39,840
We did mush them
all together and one

869
00:37:39,840 --> 00:37:42,570
could worry about that,
that when you see this,

870
00:37:42,570 --> 00:37:47,010
you remember that that's
a version of this.

871
00:37:47,010 --> 00:37:48,060
Absolutely.

872
00:37:48,060 --> 00:37:50,640
Absolutely.

873
00:37:50,640 --> 00:37:55,570
And so maybe-- yes, nonetheless,
if what you were doing--

874
00:37:55,570 --> 00:37:58,230
that's absolutely true, but
if what you were doing here

875
00:37:58,230 --> 00:38:02,490
is kind of mentally
recalling this,

876
00:38:02,490 --> 00:38:05,730
then why couldn't you
also do that here?

877
00:38:05,730 --> 00:38:07,794
Maybe you could.

878
00:38:07,794 --> 00:38:10,620
You might argue that this
is more evocative of that

879
00:38:10,620 --> 00:38:15,390
than this is, but it's also got
lots of relevant information.

880
00:38:15,390 --> 00:38:17,102
Yeah, Jimmy.

881
00:38:17,102 --> 00:38:18,810
AUDIENCE: For the
furniture, did you guys

882
00:38:18,810 --> 00:38:22,392
try placing them in the
exact position as the scene

883
00:38:22,392 --> 00:38:23,298
and seeing if that--

884
00:38:23,298 --> 00:38:24,840
NANCY KANWISHER: We
did both versions

885
00:38:24,840 --> 00:38:27,600
for exactly the reasons
you guys are pointing out.

886
00:38:27,600 --> 00:38:30,110
And it didn't make a difference.

887
00:38:30,110 --> 00:38:31,940
Yeah.

888
00:38:31,940 --> 00:38:33,290
Sorry, Cooley.

889
00:38:33,290 --> 00:38:35,840
AUDIENCE: It'd be--
you would transfer

890
00:38:35,840 --> 00:38:40,940
if they were just responding
to the things, like more stuff?

891
00:38:40,940 --> 00:38:44,653
Like in the empty room,
there's more background,

892
00:38:44,653 --> 00:38:46,070
but there's still
more background.

893
00:38:46,070 --> 00:38:46,610
NANCY KANWISHER: Totally.

894
00:38:46,610 --> 00:38:47,660
You're absolutely right.

895
00:38:47,660 --> 00:38:51,387
This is taking us pretty far,
but it's still pretty sloppy.

896
00:38:51,387 --> 00:38:53,720
This stuff goes all the way
up to the edge of the frame,

897
00:38:53,720 --> 00:38:55,070
and here there's
lots of empty space.

898
00:38:55,070 --> 00:38:56,362
Is that what you're getting at?

899
00:38:56,362 --> 00:38:57,445
Absolutely.

900
00:38:57,445 --> 00:38:58,820
I took out those
slides because I

901
00:38:58,820 --> 00:39:02,060
felt I didn't want to spend the
entire lecture doing millions

902
00:39:02,060 --> 00:39:03,560
of controlled
conditions on the PPA.

903
00:39:03,560 --> 00:39:04,643
I thought you'd get bored.

904
00:39:04,643 --> 00:39:06,770
But actually, another
version that we did

905
00:39:06,770 --> 00:39:10,260
was we then took all
of these conditions,

906
00:39:10,260 --> 00:39:13,070
and we chopped them into little
bits and rearranged the bits,

907
00:39:13,070 --> 00:39:15,440
so that you have much
more coverage of stuff

908
00:39:15,440 --> 00:39:19,820
in the chopped-up scenes
than the chopped-up objects.

909
00:39:19,820 --> 00:39:21,752
And in the chopped-up
versions, it

910
00:39:21,752 --> 00:39:23,210
doesn't respond
differently at all.

911
00:39:23,210 --> 00:39:25,910
So it's not the amount of
total spatial coverage.

912
00:39:25,910 --> 00:39:29,957
It's the actual-- something more
like the depiction of space.

913
00:39:29,957 --> 00:39:31,290
Was there a question over there?

914
00:39:31,290 --> 00:39:31,650
Yeah.

915
00:39:31,650 --> 00:39:33,067
AUDIENCE: I was
wondering if there

916
00:39:33,067 --> 00:39:36,120
would be any difference
between looking at images

917
00:39:36,120 --> 00:39:40,470
as 2D or 3D scene, and
actually being there to see

918
00:39:40,470 --> 00:39:41,978
the 3D inside of the scene.

919
00:39:41,978 --> 00:39:43,020
NANCY KANWISHER: Totally.

920
00:39:43,020 --> 00:39:43,520
Totally.

921
00:39:43,520 --> 00:39:44,490
It's a real challenge.

922
00:39:44,490 --> 00:39:46,770
With navigation,
navigation is very

923
00:39:46,770 --> 00:39:50,498
much about being there and
moving around in the space.

924
00:39:50,498 --> 00:39:52,290
And this is just a
pretty rudimentary thing

925
00:39:52,290 --> 00:39:53,850
where you're lying
in the scanner,

926
00:39:53,850 --> 00:39:56,453
and these images are just
flashing, flashing on,

927
00:39:56,453 --> 00:39:57,870
and you're doing
some simple task,

928
00:39:57,870 --> 00:40:00,090
like pressing a button
when consecutive images are

929
00:40:00,090 --> 00:40:00,692
identical.

930
00:40:00,692 --> 00:40:02,400
It's not moving around
in the real world.

931
00:40:02,400 --> 00:40:04,720
You don't think
you're actually there.

932
00:40:04,720 --> 00:40:08,910
But here's where video
games and VR come in

933
00:40:08,910 --> 00:40:12,600
because actually, they produce
a pretty powerful simulation

934
00:40:12,600 --> 00:40:14,490
of knowing your
environment, feeling

935
00:40:14,490 --> 00:40:16,390
you're in a place in it.

936
00:40:16,390 --> 00:40:18,480
And so lots of studies
have used those methods

937
00:40:18,480 --> 00:40:21,810
to give something closer
to the actual experience

938
00:40:21,810 --> 00:40:22,650
of navigation.

939
00:40:26,450 --> 00:40:28,940
So where are we so far?

940
00:40:28,940 --> 00:40:32,030
We've said the PPA seems to
be involved in recognizing

941
00:40:32,030 --> 00:40:33,480
a particular scene.

942
00:40:33,480 --> 00:40:36,500
So this just says it responds
to scenes and something

943
00:40:36,500 --> 00:40:38,570
about spatial layout, maybe.

944
00:40:38,570 --> 00:40:42,980
Does it care about
that particular scene

945
00:40:42,980 --> 00:40:46,220
or do you have to recognize
that particular scene to be

946
00:40:46,220 --> 00:40:47,930
able to use the information?

947
00:40:47,930 --> 00:40:51,110
Now, our subjects mostly didn't
know those particular scenes.

948
00:40:51,110 --> 00:40:53,000
But we wanted to do
a tighter contrast

949
00:40:53,000 --> 00:40:57,030
asking if knowledge of the
particular scene matters.

950
00:40:57,030 --> 00:41:00,020
So what we did was we
took a bunch of pictures

951
00:41:00,020 --> 00:41:03,110
around the MIT campus, and
we took a bunch of pictures

952
00:41:03,110 --> 00:41:04,700
around the Tufts campus.

953
00:41:04,700 --> 00:41:08,240
And we scanned MIT students
looking at MIT pictures

954
00:41:08,240 --> 00:41:10,550
versus Tufts pictures.

955
00:41:10,550 --> 00:41:11,720
And then what else do we do?

956
00:41:14,525 --> 00:41:15,900
AUDIENCE: Get the
Tufts students.

957
00:41:15,900 --> 00:41:18,195
NANCY KANWISHER: Yeah, why?

958
00:41:18,195 --> 00:41:20,070
AUDIENCE: Oh, just to
make sure that it's not

959
00:41:20,070 --> 00:41:22,178
all about that weird
architecture of the set.

960
00:41:22,178 --> 00:41:23,220
NANCY KANWISHER: Exactly.

961
00:41:23,220 --> 00:41:24,120
Exactly.

962
00:41:24,120 --> 00:41:25,590
So this is called--

963
00:41:25,590 --> 00:41:27,090
yes, whose weird architecture?

964
00:41:27,090 --> 00:41:29,400
I think ours is weirder.

965
00:41:29,400 --> 00:41:32,310
So it's not just about
the particular scenes

966
00:41:32,310 --> 00:41:33,820
or the particular subjects.

967
00:41:33,820 --> 00:41:36,180
So everybody get how with
that counterbalanced design,

968
00:41:36,180 --> 00:41:39,990
you can really pull out the
essence of familiarity itself,

969
00:41:39,990 --> 00:41:43,410
unconfounded from the
particular images?

970
00:41:43,410 --> 00:41:48,390
So when we did that, we found a
very similar response magnitude

971
00:41:48,390 --> 00:41:52,020
in the PPA for the
Tufts students,

972
00:41:52,020 --> 00:41:55,570
for the familiar and
unfamiliar scenes.

973
00:41:55,570 --> 00:41:57,810
Really didn't make
much difference.

974
00:41:57,810 --> 00:41:59,280
Yeah.

975
00:41:59,280 --> 00:42:01,320
AUDIENCE: Taking a step
back, so we started off

976
00:42:01,320 --> 00:42:03,495
with the one question
of navigation

977
00:42:03,495 --> 00:42:06,900
and it involving all these
different components.

978
00:42:06,900 --> 00:42:08,730
I just want to place this--

979
00:42:08,730 --> 00:42:09,490
NANCY KANWISHER:
We're getting there.

980
00:42:09,490 --> 00:42:10,080
We're getting there.

981
00:42:10,080 --> 00:42:11,700
There won't be like
a perfect answer.

982
00:42:11,700 --> 00:42:13,470
We're not going to end
up with that slide,

983
00:42:13,470 --> 00:42:16,440
with the exact brain region
of each of those things.

984
00:42:16,440 --> 00:42:21,330
We'll get some gisty, vague
senses of what this is.

985
00:42:21,330 --> 00:42:25,620
OK, so this tells
us it's not about--

986
00:42:25,620 --> 00:42:27,780
whatever the PPA is
responding to in a scene,

987
00:42:27,780 --> 00:42:30,420
it's not something that hinges
on knowing that exact scene.

988
00:42:30,420 --> 00:42:32,467
So it can't be something
like, if I was here

989
00:42:32,467 --> 00:42:34,050
and I wanted to get
coffee, what would

990
00:42:34,050 --> 00:42:36,660
my route from this location
be, given my knowledge

991
00:42:36,660 --> 00:42:37,680
of the environment.

992
00:42:37,680 --> 00:42:39,905
Because otherwise, we
wouldn't get this result.

993
00:42:39,905 --> 00:42:41,280
So whatever it
is, it's something

994
00:42:41,280 --> 00:42:44,925
more immediate and perceptual to
do with just seeing this place.

995
00:42:49,430 --> 00:42:50,580
So where are we?

996
00:42:50,580 --> 00:42:52,550
We've said that there's
this region that

997
00:42:52,550 --> 00:42:55,790
responds more to
scenes than objects,

998
00:42:55,790 --> 00:42:58,310
that when all the objects
are removed from the scenes,

999
00:42:58,310 --> 00:43:02,000
the response barely drops.

1000
00:43:02,000 --> 00:43:03,950
And its response is
pretty much the same

1001
00:43:03,950 --> 00:43:06,890
for familiar and
unfamiliar scenes.

1002
00:43:06,890 --> 00:43:08,990
So all of that
suggests that it's

1003
00:43:08,990 --> 00:43:11,600
involved in something like
perceiving the shape of space

1004
00:43:11,600 --> 00:43:12,860
around you.

1005
00:43:12,860 --> 00:43:15,620
Doesn't nail it yet, but
it kind of pushes you

1006
00:43:15,620 --> 00:43:16,970
towards that hypothesis.

1007
00:43:16,970 --> 00:43:18,950
Yeah, was there a question
here a second ago?

1008
00:43:18,950 --> 00:43:19,130
No?

1009
00:43:19,130 --> 00:43:19,340
OK.

1010
00:43:19,340 --> 00:43:21,048
AUDIENCE: I was talking
about experiment,

1011
00:43:21,048 --> 00:43:24,440
but is it accurate
when you look at a map?

1012
00:43:24,440 --> 00:43:26,220
NANCY KANWISHER:
Oh, great question.

1013
00:43:26,220 --> 00:43:26,825
Not very much.

1014
00:43:31,370 --> 00:43:33,140
Yeah, if you take
pictures of places

1015
00:43:33,140 --> 00:43:35,750
from above versus
this kind of view,

1016
00:43:35,750 --> 00:43:38,810
you get a response in this
kind of view, but not above.

1017
00:43:38,810 --> 00:43:45,560
Yeah, very telling.

1018
00:43:45,560 --> 00:43:47,060
OK, so I'm going to skip.

1019
00:43:47,060 --> 00:43:49,820
We're not going to do
the 30 other experiments.

1020
00:43:49,820 --> 00:43:51,920
We're going to skip to
the general picture, that

1021
00:43:51,920 --> 00:43:54,110
here's the PPA in four
subjects in this very

1022
00:43:54,110 --> 00:43:55,620
stereotyped location.

1023
00:43:55,620 --> 00:43:58,220
And here are some of the
many conditions we've tested.

1024
00:43:58,220 --> 00:44:00,888
It's not just abstract
maps like this.

1025
00:44:00,888 --> 00:44:02,430
They don't produce
a strong response.

1026
00:44:02,430 --> 00:44:04,640
Oh, this is an answer to
Cooley's question way back.

1027
00:44:04,640 --> 00:44:07,470
Here's the scrambled-up
scene-- much lower response.

1028
00:44:07,470 --> 00:44:11,120
So it's not just
coverage of visual junk.

1029
00:44:11,120 --> 00:44:15,170
And it responds pretty strongly
to scenes made out of LEGOs

1030
00:44:15,170 --> 00:44:17,150
compared to objects
made out of LEGOs,

1031
00:44:17,150 --> 00:44:21,020
and various other silly things.

1032
00:44:21,020 --> 00:44:24,260
So all of that seems to suggest
that it's processing something

1033
00:44:24,260 --> 00:44:26,270
like the shape or
geometry of space

1034
00:44:26,270 --> 00:44:31,580
around you-- visible space in
your immediate environment.

1035
00:44:31,580 --> 00:44:34,948
Nonetheless, there's
always pushback.

1036
00:44:34,948 --> 00:44:37,490
And there's pushback on multiple
fronts, and there should be.

1037
00:44:37,490 --> 00:44:38,940
That's proper science.

1038
00:44:38,940 --> 00:44:44,780
So one of the lines of pushback
was this paper by Nasr, et al.

1039
00:44:44,780 --> 00:44:45,763
that I didn't assign.

1040
00:44:45,763 --> 00:44:47,180
I assigned you the
response to it.

1041
00:44:47,180 --> 00:44:48,320
Anyway, what Nasr et al.

1042
00:44:48,320 --> 00:44:53,540
Did was scan people looking at
rectilinear things like cubes

1043
00:44:53,540 --> 00:44:57,600
and pyramids versus curvilinear,
round-y things like cones,

1044
00:44:57,600 --> 00:44:58,760
and spheres.

1045
00:44:58,760 --> 00:45:01,640
And what they showed
is the PPA responds

1046
00:45:01,640 --> 00:45:05,375
more to the rectilinear
than the curvilinear shapes.

1047
00:45:08,380 --> 00:45:11,030
OK, that's the first thing.

1048
00:45:11,030 --> 00:45:15,340
And so then, they argue
that in general, scenes

1049
00:45:15,340 --> 00:45:18,220
have more rectilinear structure
than curvilinear structure.

1050
00:45:18,220 --> 00:45:20,930
And they did a bunch of
math to make that case.

1051
00:45:20,930 --> 00:45:26,450
And so they argue that
maybe the apparent scene

1052
00:45:26,450 --> 00:45:30,340
selectivity of the PPA is
due to a what of scenes

1053
00:45:30,340 --> 00:45:31,480
with rectilinearity?

1054
00:45:37,470 --> 00:45:37,970
Yeah.

1055
00:45:37,970 --> 00:45:38,590
AUDIENCE: Confound.

1056
00:45:38,590 --> 00:45:40,340
NANCY KANWISHER: Yes,
exactly, a confound.

1057
00:45:40,340 --> 00:45:42,820
This is exactly what a confound
is-- something else that

1058
00:45:42,820 --> 00:45:46,060
covaries with the manipulation
you care about that gives you

1059
00:45:46,060 --> 00:45:49,270
an alternative account, namely
it's not scene selectivity.

1060
00:45:49,270 --> 00:45:50,967
It's just rectilinearity.

1061
00:45:50,967 --> 00:45:53,050
I mean, that might be
interesting to other people,

1062
00:45:53,050 --> 00:45:55,300
but it would make it not
very relevant to navigation

1063
00:45:55,300 --> 00:45:59,170
and much less interesting
to me, at least.

1064
00:45:59,170 --> 00:46:01,840
So that's an
important criticism.

1065
00:46:01,840 --> 00:46:03,070
And so then the Bryan et al.

1066
00:46:03,070 --> 00:46:06,100
Paper that you guys read
starts from there and says,

1067
00:46:06,100 --> 00:46:07,210
let's take that seriously.

1068
00:46:07,210 --> 00:46:08,530
Let's find out.

1069
00:46:08,530 --> 00:46:12,230
And so you guys should
have read all of this,

1070
00:46:12,230 --> 00:46:15,490
but just to remind you, they
have a nice, little 2 by 2

1071
00:46:15,490 --> 00:46:16,060
design--

1072
00:46:16,060 --> 00:46:17,920
remember we talked
about 2 by 2 designs--

1073
00:46:17,920 --> 00:46:20,230
where they manipulate
whether the image has

1074
00:46:20,230 --> 00:46:23,680
a lot of rectilinear structure
or less rectilinear structure,

1075
00:46:23,680 --> 00:46:25,630
and whether the image
is a place or a face.

1076
00:46:28,900 --> 00:46:33,430
And what they find in the PPA
is the same response to these.

1077
00:46:33,430 --> 00:46:36,760
And it's higher to the
scenes than the faces,

1078
00:46:36,760 --> 00:46:40,570
and rectilinearity didn't
matter for the scenes.

1079
00:46:40,570 --> 00:46:43,270
So evidently, even
though it does matter

1080
00:46:43,270 --> 00:46:47,605
with these abstract shapes,
in actual scenes and faces,

1081
00:46:47,605 --> 00:46:48,980
it doesn't seem
to be doing much.

1082
00:46:48,980 --> 00:46:51,760
It's not accounting
for this difference.

1083
00:46:51,760 --> 00:46:54,970
Everybody get that?

1084
00:46:54,970 --> 00:46:57,340
OK, let's talk about this graph.

1085
00:46:57,340 --> 00:46:59,230
Are there main effects
or interactions here?

1086
00:46:59,230 --> 00:47:01,570
And what are those main
effects or interactions?

1087
00:47:06,560 --> 00:47:07,880
Yes, Cooley.

1088
00:47:07,880 --> 00:47:10,280
AUDIENCE: There's
many different scenes.

1089
00:47:10,280 --> 00:47:13,280
NANCY KANWISHER: Yeah, of
category, scene versus face.

1090
00:47:13,280 --> 00:47:14,120
Anything else?

1091
00:47:17,460 --> 00:47:20,074
AUDIENCE: What's the first one?

1092
00:47:20,074 --> 00:47:22,930
What was the first thing?

1093
00:47:22,930 --> 00:47:25,618
In PPA category,
what's the subtype?

1094
00:47:25,618 --> 00:47:27,910
NANCY KANWISHER: Oh, wait,
this here-- these are scenes

1095
00:47:27,910 --> 00:47:30,088
and those are faces.

1096
00:47:30,088 --> 00:47:31,630
I'm sorry, and this
is the code here.

1097
00:47:31,630 --> 00:47:35,940
These are rectilinear
versus curvilinear.

1098
00:47:35,940 --> 00:47:38,220
Just one main effect, or
is there an interaction,

1099
00:47:38,220 --> 00:47:40,710
or another main effect?

1100
00:47:40,710 --> 00:47:43,117
Just one main effect.

1101
00:47:43,117 --> 00:47:44,700
These guys are higher
than those guys.

1102
00:47:44,700 --> 00:47:46,380
That's it.

1103
00:47:46,380 --> 00:47:48,990
So that just tells you
there's nothing else going on

1104
00:47:48,990 --> 00:47:52,890
in these data other
than scene selectivity.

1105
00:47:52,890 --> 00:47:55,550
Rectilinearity doesn't
interact with or modify scene

1106
00:47:55,550 --> 00:47:57,675
selectivity, and it doesn't
have a separate effect.

1107
00:48:00,330 --> 00:48:03,120
Nonetheless, as
we've been arguing

1108
00:48:03,120 --> 00:48:05,370
with all the whole
Haxby rigmarole,

1109
00:48:05,370 --> 00:48:09,360
does the fact that there's no
main effect of rectal linearity

1110
00:48:09,360 --> 00:48:12,390
in here mean that the PPA
doesn't have information

1111
00:48:12,390 --> 00:48:13,650
about rectilinearity?

1112
00:48:16,270 --> 00:48:18,810
No, Josh, why?

1113
00:48:18,810 --> 00:48:21,940
AUDIENCE: This little,
tiny moment that could be--

1114
00:48:21,940 --> 00:48:24,430
you know, this is not
the right experiment to--

1115
00:48:24,430 --> 00:48:25,090
NANCY KANWISHER: That's right.

1116
00:48:25,090 --> 00:48:27,423
This is a big-- well, it's
the right experiment, but not

1117
00:48:27,423 --> 00:48:28,720
the right analysis.

1118
00:48:28,720 --> 00:48:32,030
It's the big, average
responses are the same,

1119
00:48:32,030 --> 00:48:34,043
but maybe the patterns
are different.

1120
00:48:34,043 --> 00:48:35,710
That wouldn't directly
engage with this,

1121
00:48:35,710 --> 00:48:37,270
but we wanted to know,
was there information

1122
00:48:37,270 --> 00:48:38,740
in there about rectilinearity.

1123
00:48:42,310 --> 00:48:45,640
So how would we find out?

1124
00:48:45,640 --> 00:48:47,230
So this was your
assignment, and I

1125
00:48:47,230 --> 00:48:48,522
think most people got it right.

1126
00:48:48,522 --> 00:48:51,520
But in case anybody
missed it, we

1127
00:48:51,520 --> 00:48:55,190
were zooming in on
this Figure 4 here.

1128
00:48:55,190 --> 00:48:59,680
So again, this is just the same
basic design of experiment two.

1129
00:48:59,680 --> 00:49:02,450
And now, let's consider
what's going on here.

1130
00:49:02,450 --> 00:49:04,600
So you guys read the
paper and you understood

1131
00:49:04,600 --> 00:49:05,920
what was going on here.

1132
00:49:05,920 --> 00:49:10,393
What's represented in
that cell right there?

1133
00:49:10,393 --> 00:49:11,810
What is the point
of this diagram?

1134
00:49:11,810 --> 00:49:12,852
What are they doing here?

1135
00:49:12,852 --> 00:49:14,965
And what does that cell
mean in that matrix?

1136
00:49:19,180 --> 00:49:23,190
You can't understand the
paper without knowing that.

1137
00:49:23,190 --> 00:49:24,330
Is it Ali?

1138
00:49:24,330 --> 00:49:25,962
No, sorry.

1139
00:49:25,962 --> 00:49:26,670
What's your name?

1140
00:49:26,670 --> 00:49:27,600
AUDIENCE: Sheldon.

1141
00:49:27,600 --> 00:49:29,933
NANCY KANWISHER: Sheldon,
I've only asked you six times.

1142
00:49:29,933 --> 00:49:30,960
Yeah, go ahead.

1143
00:49:30,960 --> 00:49:36,420
AUDIENCE: So they want to
see whether the activation

1144
00:49:36,420 --> 00:49:40,470
patterns can better discriminate
between rectilinearity

1145
00:49:40,470 --> 00:49:44,100
of the same category of things
or between categories of things

1146
00:49:44,100 --> 00:49:45,690
with the same rectilinearity.

1147
00:49:45,690 --> 00:49:53,520
So the first thing I said is
to the left and the second one

1148
00:49:53,520 --> 00:49:54,900
is to the right.

1149
00:49:54,900 --> 00:49:56,187
And they--

1150
00:49:56,187 --> 00:49:58,020
NANCY KANWISHER: Sorry,
wait, here and here?

1151
00:49:58,020 --> 00:49:58,825
No.

1152
00:49:58,825 --> 00:49:59,700
AUDIENCE: Right side.

1153
00:49:59,700 --> 00:50:02,010
Yeah, so that part
is discriminating

1154
00:50:02,010 --> 00:50:05,940
between rectilinearity,
and that side

1155
00:50:05,940 --> 00:50:08,160
is discriminating
between categories.

1156
00:50:08,160 --> 00:50:10,530
And they take the
differences of--

1157
00:50:10,530 --> 00:50:14,435
well, not the differences,
they take how well

1158
00:50:14,435 --> 00:50:16,560
it can distinguish between
each of those categories

1159
00:50:16,560 --> 00:50:18,300
and plot them down there.

1160
00:50:18,300 --> 00:50:20,290
NANCY KANWISHER: Right, OK.

1161
00:50:20,290 --> 00:50:21,360
That's exactly right.

1162
00:50:21,360 --> 00:50:24,270
So this is how well it can
discriminate plotted down here,

1163
00:50:24,270 --> 00:50:27,610
but based on an analysis
that follows this scheme.

1164
00:50:27,610 --> 00:50:30,300
So what does that cell
in there represent,

1165
00:50:30,300 --> 00:50:31,800
that dark green cell?

1166
00:50:31,800 --> 00:50:36,030
What is the number that's going
to be calculated from the data

1167
00:50:36,030 --> 00:50:37,290
corresponding to that cell?

1168
00:50:42,330 --> 00:50:45,270
AUDIENCE: Similar piece
of same rectilinearity

1169
00:50:45,270 --> 00:50:46,268
and same pattern.

1170
00:50:46,268 --> 00:50:47,310
NANCY KANWISHER: Exactly.

1171
00:50:47,310 --> 00:50:48,210
Exactly.

1172
00:50:48,210 --> 00:50:50,820
So just as if you want
to distinguish chairs

1173
00:50:50,820 --> 00:50:53,220
from cars or something
else, if you want to know

1174
00:50:53,220 --> 00:50:56,280
is there information about
rectilinearity in there,

1175
00:50:56,280 --> 00:50:58,440
you take these two
cases which are

1176
00:50:58,440 --> 00:51:02,100
the same in rectilinearity--
both high rectilinear, both low

1177
00:51:02,100 --> 00:51:05,040
rectilinear for run
one and run two--

1178
00:51:05,040 --> 00:51:07,170
and that's the correlation
between run one

1179
00:51:07,170 --> 00:51:08,880
and run two for those cells.

1180
00:51:08,880 --> 00:51:12,480
That's the within
rectilinearity case.

1181
00:51:12,480 --> 00:51:14,700
And if there's information
about rectilinearity,

1182
00:51:14,700 --> 00:51:17,610
the prediction is those
within correlations

1183
00:51:17,610 --> 00:51:20,910
are higher than the
between correlations,

1184
00:51:20,910 --> 00:51:24,060
just as we argued a bit
back with beaches and cities

1185
00:51:24,060 --> 00:51:25,500
and everything else--

1186
00:51:25,500 --> 00:51:26,370
same argument.

1187
00:51:26,370 --> 00:51:29,250
This is just presenting the
data in terms of run one

1188
00:51:29,250 --> 00:51:33,270
and run two, and which cells do
we grab to do this computation.

1189
00:51:37,610 --> 00:51:41,780
So each of the cells in
there-- for each of the cells,

1190
00:51:41,780 --> 00:51:44,510
we're going to calculate
an r value of how

1191
00:51:44,510 --> 00:51:46,070
similar those patterns are.

1192
00:51:50,000 --> 00:51:54,170
A pattern for rectilinear
scenes in run two,

1193
00:51:54,170 --> 00:51:57,470
a pattern for rectilinear
scenes in run one-- this cell

1194
00:51:57,470 --> 00:52:00,140
is a correlation between
those two patterns.

1195
00:52:00,140 --> 00:52:05,420
How stable is that pattern
across repeated measures?

1196
00:52:05,420 --> 00:52:09,110
All right, so that's
what that r value is.

1197
00:52:09,110 --> 00:52:16,250
The two darker blue squares here
are the r values for stimuli

1198
00:52:16,250 --> 00:52:19,130
that differ in rectilinearity.

1199
00:52:19,130 --> 00:52:23,180
And remember that the essence
of the Haxby-style pattern

1200
00:52:23,180 --> 00:52:29,390
analysis is to see if
the within correlations

1201
00:52:29,390 --> 00:52:31,640
are higher than the
between correlations.

1202
00:52:31,640 --> 00:52:33,590
In this case, the
within correlations

1203
00:52:33,590 --> 00:52:38,090
are within rectilinearity
versus between rectilinearity.

1204
00:52:42,590 --> 00:52:48,050
And so then they calculate all
those correlation differences

1205
00:52:48,050 --> 00:52:52,970
and they plot them as
discrimination abilities.

1206
00:52:52,970 --> 00:52:56,690
And so what this is showing
us here is that actually,

1207
00:52:56,690 --> 00:52:58,970
the PPA doesn't
have any information

1208
00:52:58,970 --> 00:53:01,670
in its pattern of response
about the rectilinearity

1209
00:53:01,670 --> 00:53:03,510
of the scene.

1210
00:53:03,510 --> 00:53:06,110
However, if we
take the same data,

1211
00:53:06,110 --> 00:53:10,490
and now choose within category
versus between category,

1212
00:53:10,490 --> 00:53:12,950
ignoring rectilinearity,
and we get

1213
00:53:12,950 --> 00:53:17,840
the same kind of selectivity
correlation difference within

1214
00:53:17,840 --> 00:53:19,910
versus between for
category, there's

1215
00:53:19,910 --> 00:53:22,850
heaps of information
about category.

1216
00:53:22,850 --> 00:53:25,402
Does that make sense?

1217
00:53:25,402 --> 00:53:27,860
Again, if you're fuzzy about
this, look back on that slide.

1218
00:53:27,860 --> 00:53:30,875
I have lots of suggestions for
how to unfuzzy yourself on it.

1219
00:53:33,680 --> 00:53:36,950
So interim summary-- PPA
responds more to scenes

1220
00:53:36,950 --> 00:53:38,810
than objects.

1221
00:53:38,810 --> 00:53:42,020
It seems to like spatial
layout in particular.

1222
00:53:42,020 --> 00:53:44,600
It does respond more
to boxes than circles,

1223
00:53:44,600 --> 00:53:46,850
but that rectilinearity
bias can't

1224
00:53:46,850 --> 00:53:50,120
account for scene selectivity.

1225
00:53:50,120 --> 00:53:53,330
That's all very nice, but
what is a whole other kind

1226
00:53:53,330 --> 00:53:55,070
of fundamental
question we haven't yet

1227
00:53:55,070 --> 00:53:56,320
asked about the PPA?

1228
00:53:59,248 --> 00:54:01,290
So we've been messing
around with functional MRI,

1229
00:54:01,290 --> 00:54:03,240
measuring magnitudes
of response,

1230
00:54:03,240 --> 00:54:07,950
trying to test these kind of
vague or general hypotheses

1231
00:54:07,950 --> 00:54:10,170
about what it might
be responding to.

1232
00:54:10,170 --> 00:54:11,034
Yes.

1233
00:54:11,034 --> 00:54:12,130
AUDIENCE: Causation.

1234
00:54:12,130 --> 00:54:14,130
NANCY KANWISHER: Yes,
what particular causation?

1235
00:54:17,810 --> 00:54:20,870
AUDIENCE: I guess like
how the scenes, like

1236
00:54:20,870 --> 00:54:25,430
with how the PPA with what role
it plays in the person being

1237
00:54:25,430 --> 00:54:26,040
seen.

1238
00:54:26,040 --> 00:54:26,770
NANCY KANWISHER: Exactly.

1239
00:54:26,770 --> 00:54:27,440
Exactly.

1240
00:54:27,440 --> 00:54:30,412
Again, we can test the causal
role of a stimulus on the PPA,

1241
00:54:30,412 --> 00:54:32,120
all of the stuff I
talked about did that.

1242
00:54:32,120 --> 00:54:35,060
Manipulate the stimulus,
find different PPA responses.

1243
00:54:35,060 --> 00:54:37,460
But what we haven't
done yet is ask,

1244
00:54:37,460 --> 00:54:41,390
what is the causal relationship,
if any, between activity

1245
00:54:41,390 --> 00:54:45,830
and the PPA and perception
of scenes or navigation?

1246
00:54:45,830 --> 00:54:47,835
So far, this is all
just suggestive.

1247
00:54:47,835 --> 00:54:49,460
We have no causal
evidence for its role

1248
00:54:49,460 --> 00:54:53,330
in navigation or perception.

1249
00:54:53,330 --> 00:54:55,550
All right, so let's get some.

1250
00:54:55,550 --> 00:54:58,050
I'll show you a few examples.

1251
00:54:58,050 --> 00:54:59,810
So one, as, you guys
have learned by now

1252
00:54:59,810 --> 00:55:01,370
is these rare
cases where there's

1253
00:55:01,370 --> 00:55:04,100
direct electrical
stimulation of a region,

1254
00:55:04,100 --> 00:55:09,200
and there's one patient
in whom this is reported.

1255
00:55:09,200 --> 00:55:13,850
This patient again, is being
mapped out before neurosurgery.

1256
00:55:13,850 --> 00:55:16,520
They did functional MRI
in the patient first.

1257
00:55:16,520 --> 00:55:19,070
This is his functional
MRI response to, I think,

1258
00:55:19,070 --> 00:55:20,660
houses versus objects.

1259
00:55:20,660 --> 00:55:23,232
Houses are not as
strong an activator

1260
00:55:23,232 --> 00:55:25,190
as scenes for the PPA,
but they're pretty good.

1261
00:55:25,190 --> 00:55:28,350
PPA responds much more to
houses than other objects.

1262
00:55:28,350 --> 00:55:31,190
And so that's a nice
activation map showing the PPA.

1263
00:55:31,190 --> 00:55:33,590
And those little circles are
where the electrodes are,

1264
00:55:33,590 --> 00:55:36,080
little, black circles.

1265
00:55:36,080 --> 00:55:39,800
So they know they're in the PPA
because they did functional MRI

1266
00:55:39,800 --> 00:55:41,990
first to localize that region.

1267
00:55:41,990 --> 00:55:44,240
Now those electrodes
are sitting there.

1268
00:55:44,240 --> 00:55:46,170
And so first thing
we do is record--

1269
00:55:46,170 --> 00:55:48,530
or first thing they did--
is record responses.

1270
00:55:48,530 --> 00:55:51,170
They flash up a bunch of
different kind of images,

1271
00:55:51,170 --> 00:55:54,350
and they measure the
response in those electrodes.

1272
00:55:54,350 --> 00:55:56,630
And so what you see
is in those electrodes

1273
00:55:56,630 --> 00:56:00,410
right over there, 1, 2, 3,
that correspond to the PPA,

1274
00:56:00,410 --> 00:56:03,290
you see a higher
response to house images

1275
00:56:03,290 --> 00:56:05,120
than to any of the other images.

1276
00:56:05,120 --> 00:56:08,790
And you see the time course
here over a few seconds.

1277
00:56:08,790 --> 00:56:09,697
Everybody clear?

1278
00:56:09,697 --> 00:56:11,030
This is not causal evidence yet.

1279
00:56:11,030 --> 00:56:14,000
It's just amazing, direct
intracranial recordings

1280
00:56:14,000 --> 00:56:15,500
from the PPA--

1281
00:56:15,500 --> 00:56:17,420
I think the only time
this was ever done,

1282
00:56:17,420 --> 00:56:20,360
because it's pretty rare to
have the electrodes right there

1283
00:56:20,360 --> 00:56:23,060
in a patient who's willing to
look at your silly pictures,

1284
00:56:23,060 --> 00:56:25,850
and all of that.

1285
00:56:25,850 --> 00:56:29,870
But now, what happens
when they stimulate there?

1286
00:56:29,870 --> 00:56:32,840
So let's look at what
happens when they stimulate

1287
00:56:32,840 --> 00:56:38,120
on these sites 4 and 3 that are
off to the side of the scene

1288
00:56:38,120 --> 00:56:39,140
selectivity.

1289
00:56:39,140 --> 00:56:40,718
And this is just a dialogue.

1290
00:56:40,718 --> 00:56:42,260
We don't have a
video, unfortunately.

1291
00:56:42,260 --> 00:56:43,968
The videos are more
fun, but this is just

1292
00:56:43,968 --> 00:56:47,060
a dialogue between the
neurologist and the patient.

1293
00:56:47,060 --> 00:56:51,110
And the neurologist electrically
stimulates that region

1294
00:56:51,110 --> 00:56:54,020
and says, did you
see anything there?

1295
00:56:54,020 --> 00:56:55,460
Patient says, I don't know.

1296
00:56:55,460 --> 00:56:56,960
I started feeling something.

1297
00:56:56,960 --> 00:56:59,060
I don't know, it's
probably just me.

1298
00:56:59,060 --> 00:57:01,310
No, it's not you.

1299
00:57:01,310 --> 00:57:02,600
And then they stimulate again.

1300
00:57:02,600 --> 00:57:03,590
Anything there?

1301
00:57:03,590 --> 00:57:04,400
No.

1302
00:57:04,400 --> 00:57:05,430
Anything here?

1303
00:57:05,430 --> 00:57:06,410
No.

1304
00:57:06,410 --> 00:57:08,300
So that's right next to
the side of the scene

1305
00:57:08,300 --> 00:57:12,920
selective electrodes, right next
door, a few millimeters away.

1306
00:57:12,920 --> 00:57:15,422
Then, they move their
stimulator over here.

1307
00:57:15,422 --> 00:57:16,880
They don't move
anything, they just

1308
00:57:16,880 --> 00:57:18,588
control where they're
going to stimulate.

1309
00:57:18,588 --> 00:57:20,540
Patient, of course, has no idea.

1310
00:57:20,540 --> 00:57:22,400
Neurologist says,
"Anything here?

1311
00:57:22,400 --> 00:57:25,070
Do you see anything,
feel anything?"

1312
00:57:25,070 --> 00:57:27,230
Patient says, "Yeah,
I feel like--"

1313
00:57:27,230 --> 00:57:30,920
he looks perplexed,
puts hand to forehead--

1314
00:57:30,920 --> 00:57:34,820
"I feel like I saw
some other site.

1315
00:57:34,820 --> 00:57:37,370
We were at the train station."

1316
00:57:37,370 --> 00:57:39,140
Neurologist cleverly
says, "So it

1317
00:57:39,140 --> 00:57:41,420
feels like you're
at a train station?"

1318
00:57:41,420 --> 00:57:44,810
Patient says, "Yeah,
outside the train station."

1319
00:57:44,810 --> 00:57:47,780
Neurologist-- "Let me know if
you get any sensation like that

1320
00:57:47,780 --> 00:57:48,950
again."

1321
00:57:48,950 --> 00:57:49,490
Stimulates.

1322
00:57:49,490 --> 00:57:50,760
"Do you feel anything here?"

1323
00:57:50,760 --> 00:57:51,260
"No."

1324
00:57:54,407 --> 00:57:55,490
And then he does it again.

1325
00:57:59,000 --> 00:58:01,700
Did you see the
train station or did

1326
00:58:01,700 --> 00:58:04,550
it feel like you were
at the train station?

1327
00:58:04,550 --> 00:58:07,340
Patient, "I saw it."

1328
00:58:07,340 --> 00:58:11,030
These are very sparse, precious
data, but that's so telling.

1329
00:58:11,030 --> 00:58:14,180
It's not that he knew he was at
the train station abstractly.

1330
00:58:14,180 --> 00:58:17,400
He saw it.

1331
00:58:17,400 --> 00:58:19,940
So then, they
stimulate again, right

1332
00:58:19,940 --> 00:58:22,550
on those
scene-selective regions.

1333
00:58:22,550 --> 00:58:25,850
Patient says again, "I saw
almost like, I don't know,

1334
00:58:25,850 --> 00:58:26,810
like I saw--

1335
00:58:26,810 --> 00:58:28,157
it was very brief."

1336
00:58:28,157 --> 00:58:30,740
Neurologist says, "I'm going to
show it to you one more time."

1337
00:58:30,740 --> 00:58:31,670
Really what he
means is, I'm going

1338
00:58:31,670 --> 00:58:33,768
to stimulate you in the
same place one more time.

1339
00:58:33,768 --> 00:58:35,435
"See if you can
describe it any further.

1340
00:58:38,210 --> 00:58:42,860
And to give you one last
time, what do you think?"

1341
00:58:42,860 --> 00:58:44,840
"I don't really know
what to make of it,

1342
00:58:44,840 --> 00:58:48,740
but I saw, like,
another staircase.

1343
00:58:48,740 --> 00:58:52,700
The rest I couldn't make out,
but I saw a closet space,

1344
00:58:52,700 --> 00:58:53,660
but not this one."

1345
00:58:53,660 --> 00:58:56,450
He points to a closet
door in the room.

1346
00:58:56,450 --> 00:58:59,480
"That one was stuffed
and it was blue."

1347
00:58:59,480 --> 00:59:01,910
"Have you seen it before,"
neurologist, "Have you

1348
00:59:01,910 --> 00:59:04,340
seen it before at some point
in your life, you think?"

1349
00:59:04,340 --> 00:59:07,398
"Yeah, I mean when I
saw the train station."

1350
00:59:07,398 --> 00:59:08,690
"Train station you've been at?"

1351
00:59:08,690 --> 00:59:09,590
"Yeah."

1352
00:59:09,590 --> 00:59:11,280
Et cetera, et cetera.

1353
00:59:11,280 --> 00:59:13,530
So it's not a lot of data.

1354
00:59:13,530 --> 00:59:14,960
But it's very compelling.

1355
00:59:14,960 --> 00:59:16,710
What is the patient describing?

1356
00:59:16,710 --> 00:59:20,580
Places he's in that
he sees, and then he

1357
00:59:20,580 --> 00:59:23,633
describes this closet
space and its colors.

1358
00:59:23,633 --> 00:59:25,050
Interestingly,
colored regions are

1359
00:59:25,050 --> 00:59:28,410
right next to scene regions,
so that's kind of cool, too.

1360
00:59:28,410 --> 00:59:30,750
So it's causal evidence.

1361
00:59:30,750 --> 00:59:31,770
It's sparse.

1362
00:59:31,770 --> 00:59:35,680
Ideally, we'd like more in
science, but it's pretty cool.

1363
00:59:35,680 --> 00:59:36,490
Yeah.

1364
00:59:36,490 --> 00:59:38,323
AUDIENCE: At this point,
the patient is just

1365
00:59:38,323 --> 00:59:39,570
staring at a blank wall?

1366
00:59:39,570 --> 00:59:41,100
NANCY KANWISHER: I actually
forget in the paper.

1367
00:59:41,100 --> 00:59:42,390
I've got to go look that up.

1368
00:59:42,390 --> 00:59:44,640
I forget exactly what the
patient was doing, whether--

1369
00:59:44,640 --> 00:59:46,602
I think he's just in
the room looking out.

1370
00:59:46,602 --> 00:59:48,810
Usually, they don't control
it that much because it's

1371
00:59:48,810 --> 00:59:50,760
done for clinical
reasons, and the patient

1372
00:59:50,760 --> 00:59:52,800
is in their hospital bed,
and they're just stimulating.

1373
00:59:52,800 --> 00:59:54,810
So he's probably just looking
out at the space he's in.

1374
00:59:54,810 --> 00:59:57,060
In fact, he must have
been because at one point,

1375
00:59:57,060 --> 01:00:00,310
he says, "The closet, not
like that one over there."

1376
01:00:00,310 --> 01:00:02,400
So if he was staring
at a blank thing,

1377
01:00:02,400 --> 01:00:04,650
he was also looking
out at his room.

1378
01:00:07,260 --> 01:00:09,135
So yeah.

1379
01:00:09,135 --> 01:00:11,010
AUDIENCE: This may be
a little bit off topic.

1380
01:00:11,010 --> 01:00:12,885
You said that the region
for color perception

1381
01:00:12,885 --> 01:00:15,900
is very close to
this, it seems like.

1382
01:00:15,900 --> 01:00:20,070
Is there any relationship
between functional proximity

1383
01:00:20,070 --> 01:00:20,732
and--

1384
01:00:20,732 --> 01:00:22,440
NANCY KANWISHER: That's
a great question.

1385
01:00:22,440 --> 01:00:24,870
Nobody in the field
has an answer to this.

1386
01:00:24,870 --> 01:00:29,130
People often make hay about
the proximity of two regions,

1387
01:00:29,130 --> 01:00:31,290
like there's some deep
link because this thing is

1388
01:00:31,290 --> 01:00:33,330
next to that thing.

1389
01:00:33,330 --> 01:00:36,840
The body selective region is
right next to, and in fact

1390
01:00:36,840 --> 01:00:40,310
slightly overlapping with, area
MT that responds to motion.

1391
01:00:40,310 --> 01:00:42,000
It's like, bodies move.

1392
01:00:42,000 --> 01:00:43,950
Well, faces move
and cars move, too.

1393
01:00:43,950 --> 01:00:45,930
So I don't know.

1394
01:00:45,930 --> 01:00:47,220
It's tantalizing.

1395
01:00:47,220 --> 01:00:48,930
It feels like it ought
to mean something.

1396
01:00:48,930 --> 01:00:52,170
And people often
talk as if it does.

1397
01:00:52,170 --> 01:00:53,782
And maybe it does,
but nobody's really

1398
01:00:53,782 --> 01:00:55,740
put their finger on what
exactly it would mean.

1399
01:00:58,710 --> 01:00:59,880
But it's useful.

1400
01:00:59,880 --> 01:01:04,140
So when Rosa Lafer-Sousa who
you met in the color demo,

1401
01:01:04,140 --> 01:01:09,060
and I showed that in
humans, you get face, color,

1402
01:01:09,060 --> 01:01:11,970
and place regions right next to
each other in that order, that

1403
01:01:11,970 --> 01:01:13,800
was really cool because
Rosa had previously

1404
01:01:13,800 --> 01:01:17,040
shown that in monkeys, the
monkey brain it goes face,

1405
01:01:17,040 --> 01:01:19,920
color, place in
exactly the same order.

1406
01:01:19,920 --> 01:01:21,900
And so we thought that's
really interesting.

1407
01:01:21,900 --> 01:01:24,210
That suggests common
inheritance because that's

1408
01:01:24,210 --> 01:01:25,350
so weird and arbitrary.

1409
01:01:25,350 --> 01:01:26,590
Why would it be the same?

1410
01:01:26,590 --> 01:01:31,830
So it can be useful in
ways like that, at least.

1411
01:01:31,830 --> 01:01:33,490
So we just went
through all of this.

1412
01:01:33,490 --> 01:01:36,300
So how does this go beyond what
we knew from functional MRI?

1413
01:01:38,397 --> 01:01:39,730
I'm insulting your intelligence.

1414
01:01:39,730 --> 01:01:41,033
You know the answer to this.

1415
01:01:41,033 --> 01:01:42,700
It goes beyond it
because it tells you--

1416
01:01:42,700 --> 01:01:44,260
it implies that
there's a causal role

1417
01:01:44,260 --> 01:01:46,060
of that region in
place perception,

1418
01:01:46,060 --> 01:01:49,660
some aspect of seeing a place.

1419
01:01:49,660 --> 01:01:52,088
Now, all of this
about the PPA I just

1420
01:01:52,088 --> 01:01:54,130
started in there because
it's nice, and concrete,

1421
01:01:54,130 --> 01:01:55,480
and easy to think about.

1422
01:01:55,480 --> 01:01:58,180
But no complex mental
process happens

1423
01:01:58,180 --> 01:01:59,440
in just one brain region.

1424
01:01:59,440 --> 01:02:01,640
Nothing is ever like that.

1425
01:02:01,640 --> 01:02:04,270
And likewise, scene
perception and navigation

1426
01:02:04,270 --> 01:02:07,820
is part of a much
broader set of regions.

1427
01:02:07,820 --> 01:02:10,480
So if you do a contrast,
scan people looking

1428
01:02:10,480 --> 01:02:14,470
at scenes versus objects, you
see not just the PPA in here.

1429
01:02:14,470 --> 01:02:16,090
Again, this is a
folded-up brain,

1430
01:02:16,090 --> 01:02:18,190
and this is the mathematically
unfolded version

1431
01:02:18,190 --> 01:02:19,840
so you can see the whole cortex.

1432
01:02:19,840 --> 01:02:22,750
Dark bits are the bits that
used to be inside a sulcus

1433
01:02:22,750 --> 01:02:24,850
until it was
mathematically unfolded.

1434
01:02:24,850 --> 01:02:27,460
So there's the PPA kind of
hiding up in that sulcus.

1435
01:02:27,460 --> 01:02:31,390
And when you unfold it, you see
this nice, big, huge region.

1436
01:02:31,390 --> 01:02:33,880
But you also see all
these other regions.

1437
01:02:33,880 --> 01:02:36,005
Now there's a bunch of
terminology and don't panic.

1438
01:02:36,005 --> 01:02:37,838
I don't think you should
memorize everything

1439
01:02:37,838 --> 01:02:38,680
about each region.

1440
01:02:38,680 --> 01:02:40,847
You should know that there's
multiple scene regions.

1441
01:02:40,847 --> 01:02:42,580
You should know some
of the kinds of ways

1442
01:02:42,580 --> 01:02:44,913
you tease apart the functions,
and some of the functions

1443
01:02:44,913 --> 01:02:47,500
that have been tested,
and how they're tested.

1444
01:02:47,500 --> 01:02:51,490
But you don't need to
memorize every last detail.

1445
01:02:51,490 --> 01:02:53,470
Because it's going to
get a little hairy.

1446
01:02:53,470 --> 01:02:55,870
So here's a second
scene region right

1447
01:02:55,870 --> 01:03:00,370
there called retrosplenial
cortex or RSC.

1448
01:03:00,370 --> 01:03:02,650
And actually,
Russell Epstein and I

1449
01:03:02,650 --> 01:03:05,440
saw that activation in
the very first experiments

1450
01:03:05,440 --> 01:03:07,690
we did in the
1990s, but we really

1451
01:03:07,690 --> 01:03:09,640
didn't know what we
were doing back then.

1452
01:03:09,640 --> 01:03:13,480
And we knew that this is right
near the calcarine sulcus.

1453
01:03:13,480 --> 01:03:16,162
Remind me, what happens
in the calcarine sulcus?

1454
01:03:16,162 --> 01:03:18,370
What functional region lives
in the calcarine sulcus?

1455
01:03:22,923 --> 01:03:24,590
It's just a weird,
little fact, but it's

1456
01:03:24,590 --> 01:03:29,270
kind of an important one
that we mentioned weeks ago.

1457
01:03:29,270 --> 01:03:32,720
V1, primary visual cortex--

1458
01:03:32,720 --> 01:03:34,920
that's where primary
visual cortex lives.

1459
01:03:34,920 --> 01:03:37,190
And remember,
primary visual cortex

1460
01:03:37,190 --> 01:03:40,460
has a map of retinotopic
space, with next door bits

1461
01:03:40,460 --> 01:03:41,990
of primary visual
cortex responding

1462
01:03:41,990 --> 01:03:44,090
to next door bits of space.

1463
01:03:44,090 --> 01:03:46,190
And in fact, that
map has the center

1464
01:03:46,190 --> 01:03:50,250
of gaze out here and
the periphery out there.

1465
01:03:50,250 --> 01:03:53,390
So when Russell and I
first saw that activation,

1466
01:03:53,390 --> 01:03:56,690
we had the same worry that
Cooley mentioned a while back.

1467
01:03:56,690 --> 01:03:58,610
And that is the scenes
are sticking out.

1468
01:03:58,610 --> 01:03:59,840
There's stuff everywhere.

1469
01:03:59,840 --> 01:04:01,850
The objects, there isn't
that much sticking out.

1470
01:04:01,850 --> 01:04:04,670
And we thought, oh, that's just
peripheral retinotopic cortex.

1471
01:04:04,670 --> 01:04:05,240
But it's not.

1472
01:04:05,240 --> 01:04:07,740
It's right next to there and
it's a totally different thing.

1473
01:04:07,740 --> 01:04:09,680
And it turns out to be
extremely interesting.

1474
01:04:09,680 --> 01:04:11,013
You don't need to know all that.

1475
01:04:11,013 --> 01:04:13,580
It's just silly, little history.

1476
01:04:13,580 --> 01:04:16,760
There's a third region up there
that's on the outer surface

1477
01:04:16,760 --> 01:04:21,955
out there that used to be called
TOS and is now called OPA.

1478
01:04:21,955 --> 01:04:22,830
I'm sorry about that.

1479
01:04:22,830 --> 01:04:24,163
You don't need to remember this.

1480
01:04:24,163 --> 01:04:27,980
Know that there are at
least three regions.

1481
01:04:27,980 --> 01:04:31,430
But TOS slash OPA is
interesting because there's

1482
01:04:31,430 --> 01:04:35,270
a method we can apply to it that
we can't apply to the others.

1483
01:04:35,270 --> 01:04:38,550
What would that method be?

1484
01:04:38,550 --> 01:04:39,470
AUDIENCE: TMS.

1485
01:04:39,470 --> 01:04:40,730
NANCY KANWISHER: Yeah, TMS--

1486
01:04:40,730 --> 01:04:42,470
it's right out on the surface.

1487
01:04:42,470 --> 01:04:44,870
You just stick the coil
there and go "zap."

1488
01:04:44,870 --> 01:04:48,160
So of course, we've
done a lot of that.

1489
01:04:48,160 --> 01:04:50,690
Can't get the coil
into the PPA or RSC.

1490
01:04:50,690 --> 01:04:53,168
It's too medial.

1491
01:04:53,168 --> 01:04:54,710
And there's another
region that we'll

1492
01:04:54,710 --> 01:04:57,020
talk about more next time
called the hippocampus.

1493
01:04:57,020 --> 01:05:00,290
You saw the hippocampus when
Ann Graybiel spent all that time

1494
01:05:00,290 --> 01:05:01,790
digging in the
temporal lobe to find

1495
01:05:01,790 --> 01:05:04,310
that bumpy, little,
dentate gyrus,

1496
01:05:04,310 --> 01:05:05,870
approximately right in there.

1497
01:05:05,870 --> 01:05:06,968
And so all of these--

1498
01:05:06,968 --> 01:05:08,510
and probably other
regions, but these

1499
01:05:08,510 --> 01:05:11,930
are the core elements of
the scene selective regions

1500
01:05:11,930 --> 01:05:15,650
that are implicated in
different aspects of navigation.

1501
01:05:15,650 --> 01:05:18,350
So when you have
multiple regions

1502
01:05:18,350 --> 01:05:21,560
that seem to be part of a
system, that's an opportunity.

1503
01:05:21,560 --> 01:05:23,257
Because now we have
the possibility

1504
01:05:23,257 --> 01:05:25,340
that maybe we could figure
out different functions

1505
01:05:25,340 --> 01:05:26,450
for different regions.

1506
01:05:26,450 --> 01:05:29,210
And then maybe that would really
tell us more than just scenes

1507
01:05:29,210 --> 01:05:30,950
and navigation, end of story.

1508
01:05:30,950 --> 01:05:33,110
It's kind of rudimentary.

1509
01:05:33,110 --> 01:05:36,530
It would be nice if different
aspects of the navigation story

1510
01:05:36,530 --> 01:05:39,350
engage different
parts of the system.

1511
01:05:39,350 --> 01:05:41,040
So really what we
want to know is,

1512
01:05:41,040 --> 01:05:43,670
how does each of these regions
help us navigate and see

1513
01:05:43,670 --> 01:05:45,140
scenes.

1514
01:05:45,140 --> 01:05:47,330
And I'm not going to
answer that fully.

1515
01:05:47,330 --> 01:05:49,550
The field is still trying
to understand all of this,

1516
01:05:49,550 --> 01:05:53,180
but I'll give you a few
tantalizing little snippets.

1517
01:05:53,180 --> 01:05:57,690
So let's take retrosplenial
cortex right here.

1518
01:05:57,690 --> 01:06:02,570
So this is first the
response of the PPA

1519
01:06:02,570 --> 01:06:05,150
right there, and retrosplenial
cortex, which is just

1520
01:06:05,150 --> 01:06:06,520
behind it.

1521
01:06:06,520 --> 01:06:09,020
This is just its mean response
to a bunch of different kinds

1522
01:06:09,020 --> 01:06:11,720
of stimuli, showing
you that it likes

1523
01:06:11,720 --> 01:06:14,930
landscapes and cityscapes,
scenes, more than a bunch

1524
01:06:14,930 --> 01:06:16,370
of other categories of objects.

1525
01:06:16,370 --> 01:06:20,060
And that's true of
both the PPA and RSC.

1526
01:06:20,060 --> 01:06:24,620
No surprises here-- they're
both somewhat scene selective.

1527
01:06:24,620 --> 01:06:26,840
But then in a whole
bunch of other studies

1528
01:06:26,840 --> 01:06:29,930
summarized in this graph
here, Russell Epstein

1529
01:06:29,930 --> 01:06:33,800
and his colleagues had subjects
engage in different tasks

1530
01:06:33,800 --> 01:06:35,240
while they were
looking at scenes.

1531
01:06:35,240 --> 01:06:38,360
In some tasks, they had
to say where they were.

1532
01:06:38,360 --> 01:06:40,700
He's at UPenn, and he
showed his subjects

1533
01:06:40,700 --> 01:06:42,230
pictures of the UPenn campus.

1534
01:06:42,230 --> 01:06:44,210
And they had to answer
all kinds of questions

1535
01:06:44,210 --> 01:06:46,670
about what part of
campus they were,

1536
01:06:46,670 --> 01:06:50,240
where they were on campus,
and also about which way they

1537
01:06:50,240 --> 01:06:53,060
were facing given the view of
the campus they were looking

1538
01:06:53,060 --> 01:06:55,760
at.

1539
01:06:55,760 --> 01:06:58,730
Then he also showed
people familiar scenes

1540
01:06:58,730 --> 01:07:02,300
and unfamiliar scenes, much like
we did with our Tufts study.

1541
01:07:02,300 --> 01:07:04,370
And he had object controls.

1542
01:07:04,370 --> 01:07:07,200
And you can see the PPA
doesn't care about any of that,

1543
01:07:07,200 --> 01:07:09,830
doesn't care, really, if
they're familiar or unfamiliar,

1544
01:07:09,830 --> 01:07:12,140
doesn't care what task
you're doing on the scene.

1545
01:07:12,140 --> 01:07:15,170
You're looking at a
scene, it's just going.

1546
01:07:15,170 --> 01:07:18,410
So we didn't really tease
apart functions there.

1547
01:07:18,410 --> 01:07:23,180
But RSC responds differently
in these conditions.

1548
01:07:23,180 --> 01:07:28,790
It's engaged in both
the location task

1549
01:07:28,790 --> 01:07:30,110
and the orientation task.

1550
01:07:33,380 --> 01:07:36,260
It responds
substantially more when

1551
01:07:36,260 --> 01:07:39,480
you look at images of a familiar
place than an unfamiliar place.

1552
01:07:39,480 --> 01:07:42,350
So this is the first time we've
seen that in the same network.

1553
01:07:42,350 --> 01:07:43,940
And so now, think
about all the things

1554
01:07:43,940 --> 01:07:46,190
you can do when you're looking
at a picture of a scene

1555
01:07:46,190 --> 01:07:48,590
and you know that place.

1556
01:07:48,590 --> 01:07:50,750
You have memories of
having been there.

1557
01:07:50,750 --> 01:07:53,270
You can think about
what you might

1558
01:07:53,270 --> 01:07:54,950
do if you were there,
how you would get

1559
01:07:54,950 --> 01:07:56,158
from there to someplace else.

1560
01:07:56,158 --> 01:07:57,950
All of those things
are possible things

1561
01:07:57,950 --> 01:08:02,450
that might be driving RSC.

1562
01:08:02,450 --> 01:08:05,120
Another thing that
might be driving RSC

1563
01:08:05,120 --> 01:08:09,410
is that if you're looking at
a picture of a familiar place,

1564
01:08:09,410 --> 01:08:12,260
you orient yourself with respect
to the broader environment

1565
01:08:12,260 --> 01:08:14,847
that that view is part of.

1566
01:08:14,847 --> 01:08:17,180
So what I showed you that
picture of the front of Stata,

1567
01:08:17,180 --> 01:08:19,609
you immediately imagine,
I'm out on Vassar Street

1568
01:08:19,609 --> 01:08:24,620
facing that way, roughly
northwest, I think.

1569
01:08:24,620 --> 01:08:26,395
If you look at a
picture of a scene

1570
01:08:26,395 --> 01:08:27,770
and you don't know
that scene, it

1571
01:08:27,770 --> 01:08:30,040
doesn't tell you anything
about your broader heading

1572
01:08:30,040 --> 01:08:31,529
in the broader world.

1573
01:08:31,529 --> 01:08:35,330
So all of those are things
that the RSC, its function

1574
01:08:35,330 --> 01:08:37,085
seems to depend on
knowing that place.

1575
01:08:40,890 --> 01:08:43,170
Perhaps the most
telling case comes

1576
01:08:43,170 --> 01:08:47,279
from a patient who had damage
in retrosplenial cortex.

1577
01:08:47,279 --> 01:08:49,620
And the description
in the paper of this

1578
01:08:49,620 --> 01:08:52,380
says that this patient
could recognize

1579
01:08:52,380 --> 01:08:55,470
buildings and the
landmarks, and therefore,

1580
01:08:55,470 --> 01:08:57,990
understand where he was.

1581
01:08:57,990 --> 01:08:59,430
So lots is intact--

1582
01:08:59,430 --> 01:09:03,689
can recognize scenes
and know where he is.

1583
01:09:03,689 --> 01:09:06,149
But the landmarks he
recognized did not

1584
01:09:06,149 --> 01:09:09,210
provoke directional information
about any other places

1585
01:09:09,210 --> 01:09:12,870
with respect to those landmarks.

1586
01:09:12,870 --> 01:09:15,300
So this person can
look at a picture

1587
01:09:15,300 --> 01:09:16,859
and say, yeah, I
know that place.

1588
01:09:16,859 --> 01:09:18,609
That's the front of my house.

1589
01:09:18,609 --> 01:09:22,470
But then if you say, in
which direction is a coffee

1590
01:09:22,470 --> 01:09:25,170
shop two blocks
away, he doesn't know

1591
01:09:25,170 --> 01:09:28,229
which way it is from there.

1592
01:09:28,229 --> 01:09:32,050
So this should sound familiar.

1593
01:09:32,050 --> 01:09:38,779
This is my guess of the bit that
my friend Bob got messed up.

1594
01:09:38,779 --> 01:09:39,279
Yeah.

1595
01:09:39,279 --> 01:09:40,810
This is exactly
his description--

1596
01:09:40,810 --> 01:09:42,670
he could recognize
places, but it

1597
01:09:42,670 --> 01:09:47,529
wouldn't tell him how to get
from there to somewhere else.

1598
01:09:47,529 --> 01:09:50,649
And so the best current guess
about retrosplenial cortex

1599
01:09:50,649 --> 01:09:54,279
is that it's involved in
anchoring where you are.

1600
01:09:54,279 --> 01:09:57,620
You have this mental map of the
world, and you have a scene,

1601
01:09:57,620 --> 01:09:59,290
and you're trying to
put them together.

1602
01:09:59,290 --> 01:10:02,290
Given that I see this,
where am I on the map,

1603
01:10:02,290 --> 01:10:05,500
and which way am I
heading in that map?

1604
01:10:05,500 --> 01:10:08,050
Again, think about the problem
you face when you emerge

1605
01:10:08,050 --> 01:10:10,420
from the subway in Manhattan.

1606
01:10:10,420 --> 01:10:11,140
You look around.

1607
01:10:11,140 --> 01:10:13,660
Where am I, and which
way am I heading?

1608
01:10:13,660 --> 01:10:15,610
That's what you need
retrosplenial cortex for.

1609
01:10:19,630 --> 01:10:21,990
How about this TOS thing?

1610
01:10:21,990 --> 01:10:23,240
There's lots of studies of it.

1611
01:10:23,240 --> 01:10:26,660
I'll give you just
one little offering.

1612
01:10:26,660 --> 01:10:30,070
So this is a causal
investigation

1613
01:10:30,070 --> 01:10:33,140
because as we discussed, the TOS
is out on the lateral surface.

1614
01:10:33,140 --> 01:10:34,520
So we can zap it.

1615
01:10:34,520 --> 01:10:37,010
And so of course, we do.

1616
01:10:37,010 --> 01:10:40,690
And so in this
study, we were asking

1617
01:10:40,690 --> 01:10:44,680
whether TOS is involved in
perceiving the structure

1618
01:10:44,680 --> 01:10:46,090
of space around you.

1619
01:10:46,090 --> 01:10:49,450
So we took scenes like
this from CAD programs,

1620
01:10:49,450 --> 01:10:51,190
and we just varied
them slightly.

1621
01:10:51,190 --> 01:10:54,130
So for example, the position
of this wall moves around,

1622
01:10:54,130 --> 01:10:57,190
the aspect ratio, the height
of the ceiling moves around,

1623
01:10:57,190 --> 01:11:00,370
and we make this subtle morph
space of different versions

1624
01:11:00,370 --> 01:11:02,933
of this image.

1625
01:11:02,933 --> 01:11:05,350
And then for control condition,
we do the same with faces.

1626
01:11:05,350 --> 01:11:07,360
We morph between this
guy and that guy,

1627
01:11:07,360 --> 01:11:09,430
and make a whole
spectrum in between.

1628
01:11:09,430 --> 01:11:13,480
And then in the task, what
we do is here's one trial.

1629
01:11:13,480 --> 01:11:17,470
One of the scenes or
faces comes on briefly,

1630
01:11:17,470 --> 01:11:20,320
and then shortly thereafter,
you get a choice of two,

1631
01:11:20,320 --> 01:11:24,520
and you have to say which
of these matches that one.

1632
01:11:24,520 --> 01:11:26,890
And then what we do
is we zap people right

1633
01:11:26,890 --> 01:11:30,580
after we present this stimulus.

1634
01:11:30,580 --> 01:11:32,320
And so the idea is
this is as close

1635
01:11:32,320 --> 01:11:35,410
as we can get to a pretty
pure perceptual task.

1636
01:11:35,410 --> 01:11:38,980
How well can you see the
shape of that environment

1637
01:11:38,980 --> 01:11:40,235
or the shape of that face?

1638
01:11:40,235 --> 01:11:42,610
You don't have to remember it
for more than a few hundred

1639
01:11:42,610 --> 01:11:43,490
milliseconds.

1640
01:11:43,490 --> 01:11:46,000
So it's really more of a
perception task than a memory

1641
01:11:46,000 --> 01:11:47,440
task.

1642
01:11:47,440 --> 01:11:50,680
And what we measure
is, we actually

1643
01:11:50,680 --> 01:11:54,490
muck with how different these
two images are in each trial,

1644
01:11:54,490 --> 01:11:56,650
and measure how
far apart they have

1645
01:11:56,650 --> 01:12:00,640
to be in morph space for
you to be about 75% correct.

1646
01:12:00,640 --> 01:12:03,430
That's the standard
psychophysical measure.

1647
01:12:03,430 --> 01:12:04,550
The details don't matter.

1648
01:12:04,550 --> 01:12:07,120
But our dependent measure is,
how different do the stimuli

1649
01:12:07,120 --> 01:12:09,040
have to be for you
to discriminate them

1650
01:12:09,040 --> 01:12:15,400
as a function of whether you're
getting zapped in TOS or not.

1651
01:12:15,400 --> 01:12:17,452
And so here are the data.

1652
01:12:17,452 --> 01:12:18,910
So let's take the
case where you're

1653
01:12:18,910 --> 01:12:21,040
doing the scene task here.

1654
01:12:21,040 --> 01:12:22,780
What this threshold
is, is again,

1655
01:12:22,780 --> 01:12:25,060
how different the
stimuli need to be

1656
01:12:25,060 --> 01:12:26,510
for you to discriminate them.

1657
01:12:26,510 --> 01:12:29,920
So the higher the bar,
the worse performance.

1658
01:12:29,920 --> 01:12:32,720
They have to be really different
or you can't tell them apart.

1659
01:12:32,720 --> 01:12:35,740
And so what you
see is when you zap

1660
01:12:35,740 --> 01:12:39,520
OPA, that lateral
scene selective region,

1661
01:12:39,520 --> 01:12:42,040
discrimination
threshold goes up a bit.

1662
01:12:42,040 --> 01:12:44,140
That means you get worse
at the discrimination.

1663
01:12:44,140 --> 01:12:46,360
The stimuli need to
be more different.

1664
01:12:46,360 --> 01:12:49,393
Compared to zapping
the top of your head--

1665
01:12:49,393 --> 01:12:51,310
remember, you always
want a control condition,

1666
01:12:51,310 --> 01:12:53,110
and there's no perfect
control condition

1667
01:12:53,110 --> 01:12:55,693
because it feels differently to
be zapped in different places.

1668
01:12:55,693 --> 01:13:01,360
But getting zapped up here is
a better than nothing control.

1669
01:13:01,360 --> 01:13:03,430
And then here's the
occipital face area.

1670
01:13:03,430 --> 01:13:06,010
That's the lateral face region
we talked about before when

1671
01:13:06,010 --> 01:13:07,825
I showed you another TMS study.

1672
01:13:07,825 --> 01:13:09,700
Basically, whenever
there's anything lateral,

1673
01:13:09,700 --> 01:13:12,880
we zap it because we can.

1674
01:13:12,880 --> 01:13:14,980
And see, it's not affected here.

1675
01:13:14,980 --> 01:13:17,590
Zapping the occipital face area
does not mess up your ability

1676
01:13:17,590 --> 01:13:19,600
to discriminate the scenes.

1677
01:13:19,600 --> 01:13:23,890
However, in the face task,
we see the opposite pattern.

1678
01:13:23,890 --> 01:13:28,550
For the face task, zapping
the occipital place area

1679
01:13:28,550 --> 01:13:31,120
doesn't do anything compared to
zapping the top of your head,

1680
01:13:31,120 --> 01:13:34,630
but zapping the face area does.

1681
01:13:34,630 --> 01:13:37,570
This is a double dissociation.

1682
01:13:37,570 --> 01:13:42,520
If we just had the scene task,
it would be like, yeah, maybe.

1683
01:13:42,520 --> 01:13:43,810
Who knows.

1684
01:13:43,810 --> 01:13:46,360
Maybe, who knows why.

1685
01:13:46,360 --> 01:13:47,770
But it's not very strong.

1686
01:13:47,770 --> 01:13:50,050
But when you have
these opposite things,

1687
01:13:50,050 --> 01:13:53,500
then we really have much
more strong evidence

1688
01:13:53,500 --> 01:13:55,600
that these two regions
have different functions

1689
01:13:55,600 --> 01:13:56,918
from each other.

1690
01:13:56,918 --> 01:13:58,960
Everybody get that this
is a double dissociation,

1691
01:13:58,960 --> 01:14:01,810
in the same sense of when you
have one patient with damage

1692
01:14:01,810 --> 01:14:03,825
in one location and
another patient with damage

1693
01:14:03,825 --> 01:14:05,200
in another location
and they have

1694
01:14:05,200 --> 01:14:08,680
opposite patterns of deficit,
then we're really in business.

1695
01:14:08,680 --> 01:14:10,240
Then we can draw
strong inferences.

1696
01:14:13,560 --> 01:14:15,090
So we just said all of that.

1697
01:14:15,090 --> 01:14:17,520
So that's just a little snippet.

1698
01:14:17,520 --> 01:14:22,080
These and other data suggest
that that region is strongly

1699
01:14:22,080 --> 01:14:24,420
active when you look
at scenes, and it

1700
01:14:24,420 --> 01:14:27,120
seems to be involved in
something like perceiving--

1701
01:14:27,120 --> 01:14:30,082
just directly online
perceiving the structure

1702
01:14:30,082 --> 01:14:31,290
of the space in front of you.

1703
01:14:36,110 --> 01:14:41,960
So we already did
retrosplenial cortex.

1704
01:14:41,960 --> 01:14:45,530
And next time, we'll talk
about the hippocampus in there,

1705
01:14:45,530 --> 01:14:49,430
and its role in the
whole navigation thing.

1706
01:14:49,430 --> 01:14:55,160
Now, since I have ended early--

1707
01:14:55,160 --> 01:14:55,750
a rare event--

1708
01:14:55,750 --> 01:14:58,250
I actually put together a whole
other piece of this lecture,

1709
01:14:58,250 --> 01:15:01,670
and I thought, no, don't always
have a part you don't get to.

1710
01:15:01,670 --> 01:15:04,910
But then it turns
out we do get to it.

1711
01:15:07,640 --> 01:15:09,560
We're going to go
over this more later,

1712
01:15:09,560 --> 01:15:12,700
but we're going to start with
this business right here.

1713
01:15:12,700 --> 01:15:15,800
So anybody have questions
about this stuff so far?

1714
01:15:15,800 --> 01:15:18,590
OK, so I've spent a
lot of time talking

1715
01:15:18,590 --> 01:15:21,670
about multiple voxel pattern
analysis, because it's

1716
01:15:21,670 --> 01:15:24,380
the only method I've mentioned
so far that enables us to go

1717
01:15:24,380 --> 01:15:27,980
beyond the business of
saying how strongly do

1718
01:15:27,980 --> 01:15:29,990
the neurons fire in
this region to the more

1719
01:15:29,990 --> 01:15:32,510
interesting question of what
information is contained

1720
01:15:32,510 --> 01:15:34,910
in this region.

1721
01:15:34,910 --> 01:15:36,710
But I also ended
the last lecture

1722
01:15:36,710 --> 01:15:38,330
with this kind of
depressive note--

1723
01:15:38,330 --> 01:15:42,740
that you can't see much with
MVPA applied to face patches,

1724
01:15:42,740 --> 01:15:45,260
even when we know there's
information in there

1725
01:15:45,260 --> 01:15:46,933
with electrophysiology data.

1726
01:15:46,933 --> 01:15:48,350
Remember, I showed
you that monkey

1727
01:15:48,350 --> 01:15:51,590
study where they tried MVPA
in the face patches in monkeys

1728
01:15:51,590 --> 01:15:53,970
and they couldn't kind
of read out a damn thing.

1729
01:15:53,970 --> 01:15:57,440
And then they try MVPA on
individual neural responses

1730
01:15:57,440 --> 01:15:59,180
of the same region,
and they can read out

1731
01:15:59,180 --> 01:16:00,710
all kinds of information.

1732
01:16:00,710 --> 01:16:02,810
And that tells you the
information is there

1733
01:16:02,810 --> 01:16:06,230
and we just can't
always see it with MVPA.

1734
01:16:06,230 --> 01:16:08,660
Now today, you've seen
cases where can see stuff

1735
01:16:08,660 --> 01:16:10,370
with MVPA in the scene region.

1736
01:16:10,370 --> 01:16:13,530
So sometimes it works,
sometimes it doesn't.

1737
01:16:13,530 --> 01:16:15,290
And when it doesn't
work, we're left

1738
01:16:15,290 --> 01:16:17,030
in this unsatisfying
situation that we

1739
01:16:17,030 --> 01:16:19,370
don't know if the
information isn't there

1740
01:16:19,370 --> 01:16:22,430
or if the neurons are
just so scrambled together

1741
01:16:22,430 --> 01:16:27,650
that we can't see the
different patterns.

1742
01:16:27,650 --> 01:16:30,270
So bottom line, we
need another method.

1743
01:16:30,270 --> 01:16:32,510
MVPA is a whole lot
better than nothing,

1744
01:16:32,510 --> 01:16:34,970
but we want to be
able to ask, is there

1745
01:16:34,970 --> 01:16:37,730
information present in
this region even when we

1746
01:16:37,730 --> 01:16:42,590
think the relevant neurons are
all spatially intermingled?

1747
01:16:42,590 --> 01:16:44,180
So let me just do a
little bit of this

1748
01:16:44,180 --> 01:16:45,920
and then we'll continue later.

1749
01:16:45,920 --> 01:16:50,210
So goal-- this new method
is called "event-related

1750
01:16:50,210 --> 01:16:53,330
functional MRI adaptation."

1751
01:16:53,330 --> 01:16:55,010
And we use it when
we want to know

1752
01:16:55,010 --> 01:16:57,200
if neural populations
in a particular region

1753
01:16:57,200 --> 01:17:01,050
can discriminate between two
stimuli, two stimulus classes.

1754
01:17:01,050 --> 01:17:04,610
So for example, do
neurons in the FFA

1755
01:17:04,610 --> 01:17:09,830
distinguish between this
image and that image?

1756
01:17:09,830 --> 01:17:12,800
So if we want to
know that, we could

1757
01:17:12,800 --> 01:17:15,170
measure the functional
MRI response in the FFA

1758
01:17:15,170 --> 01:17:18,470
and find this would be an
event-related response,

1759
01:17:18,470 --> 01:17:22,460
similar responses to the two.

1760
01:17:22,460 --> 01:17:25,520
And as I just
mentioned, that wouldn't

1761
01:17:25,520 --> 01:17:28,420
mean that there isn't
information in the FFA

1762
01:17:28,420 --> 01:17:29,420
that discriminates that.

1763
01:17:29,420 --> 01:17:31,790
It just says they have
the same mean response.

1764
01:17:31,790 --> 01:17:35,000
Everybody get that?

1765
01:17:35,000 --> 01:17:39,230
Now, if we zoom in, and think
about what might neurons

1766
01:17:39,230 --> 01:17:42,410
be doing, it's still
possible-- even

1767
01:17:42,410 --> 01:17:44,990
with the same mean
response-- that neurons

1768
01:17:44,990 --> 01:17:47,660
could be organized like this,
with some of them responding

1769
01:17:47,660 --> 01:17:50,180
only to this image and some
of them responding only

1770
01:17:50,180 --> 01:17:51,900
to that image.

1771
01:17:51,900 --> 01:17:54,770
But it's also possible
that all of the neurons

1772
01:17:54,770 --> 01:17:57,320
respond equally to both.

1773
01:17:57,320 --> 01:17:59,570
And we kind of
desperately need to know--

1774
01:17:59,570 --> 01:18:00,720
I mean, not in this case.

1775
01:18:00,720 --> 01:18:02,095
This is a toy
example, obviously.

1776
01:18:02,095 --> 01:18:04,130
But we often, when we're
trying to understand

1777
01:18:04,130 --> 01:18:05,960
a region of the brain, we
need to know which situation

1778
01:18:05,960 --> 01:18:06,590
we're in.

1779
01:18:09,260 --> 01:18:12,800
So that neural population can
discriminate these two and that

1780
01:18:12,800 --> 01:18:15,500
one can't.

1781
01:18:15,500 --> 01:18:17,550
How are we going to
tell which is true?

1782
01:18:17,550 --> 01:18:20,210
Well, we talked before
about multiple voxel pattern

1783
01:18:20,210 --> 01:18:22,370
analysis, but as I
just said, it only

1784
01:18:22,370 --> 01:18:25,820
works when the neurons
are spatially clustered

1785
01:18:25,820 --> 01:18:28,760
on the scale of voxels.

1786
01:18:28,760 --> 01:18:33,522
So imagine you have
these situations here.

1787
01:18:33,522 --> 01:18:35,480
This is getting more and
more of a toy example,

1788
01:18:35,480 --> 01:18:36,900
but just to give you the idea.

1789
01:18:36,900 --> 01:18:40,070
Suppose where those neural
populations land with respect

1790
01:18:40,070 --> 01:18:42,120
to voxels is like this.

1791
01:18:42,120 --> 01:18:44,840
So if each of these is a
voxel in the brain, a little,

1792
01:18:44,840 --> 01:18:47,930
say, 2 by 2 by 3
millimeter chunk of brain

1793
01:18:47,930 --> 01:18:50,180
that we're getting
an MRI signal from,

1794
01:18:50,180 --> 01:18:52,340
if you have the different
neural populations

1795
01:18:52,340 --> 01:18:54,500
spatially segregated
enough that they mostly

1796
01:18:54,500 --> 01:18:59,380
land in different voxels,
then MVPA might work here.

1797
01:18:59,380 --> 01:19:00,320
Is that intuitive?

1798
01:19:00,320 --> 01:19:01,362
Do you guys all see that?

1799
01:19:01,362 --> 01:19:03,845
Then we'd get a different
pattern in these voxels

1800
01:19:03,845 --> 01:19:06,800
if we're looking at those
two different images.

1801
01:19:06,800 --> 01:19:10,680
But even if we have
the situation here,

1802
01:19:10,680 --> 01:19:12,560
which is kind of
informationally the same,

1803
01:19:12,560 --> 01:19:16,070
if they're spatially
scrambled so that they're

1804
01:19:16,070 --> 01:19:21,060
in roughly equal proportion in
each voxel, MVPA won't work.

1805
01:19:21,060 --> 01:19:23,800
Does that make sense?

1806
01:19:23,800 --> 01:19:26,430
And so that's when we need this
other method called "functional

1807
01:19:26,430 --> 01:19:29,130
MRI adaptation."

1808
01:19:29,130 --> 01:19:30,780
Make sense?

1809
01:19:30,780 --> 01:19:33,510
I'm going to go one
minute over probably.

1810
01:19:33,510 --> 01:19:35,670
So the point of
functional MRI adaptation

1811
01:19:35,670 --> 01:19:39,510
is it can work even when
there's no spatial clustering

1812
01:19:39,510 --> 01:19:41,220
of the relevant
neural populations

1813
01:19:41,220 --> 01:19:42,242
on the scale of voxels.

1814
01:19:42,242 --> 01:19:43,950
So let me go through
it quickly and we'll

1815
01:19:43,950 --> 01:19:45,100
come back to it later.

1816
01:19:45,100 --> 01:19:46,500
So here's how it goes--

1817
01:19:46,500 --> 01:19:48,810
the basic idea is,
any measure that's

1818
01:19:48,810 --> 01:19:52,560
sensitive to the sameness versus
difference between two stimuli

1819
01:19:52,560 --> 01:19:57,960
can reveal what that system
takes to be same or different.

1820
01:19:57,960 --> 01:20:01,260
So for example, if a
brain region discriminates

1821
01:20:01,260 --> 01:20:06,930
between two similar stimuli
like these, then if we measure

1822
01:20:06,930 --> 01:20:09,060
the functional MRI
response in that region

1823
01:20:09,060 --> 01:20:11,670
to same versus
different trials--

1824
01:20:11,670 --> 01:20:13,680
so this would be
a different trial.

1825
01:20:13,680 --> 01:20:16,500
You present Trump and then
the chimp back to back.

1826
01:20:16,500 --> 01:20:20,670
That's one trial, compared
to a same trial, chimp

1827
01:20:20,670 --> 01:20:22,020
and then chimp.

1828
01:20:22,020 --> 01:20:24,120
And of course, we
counterbalance everything,

1829
01:20:24,120 --> 01:20:27,030
so we also do chimp and then
Trump in another different case

1830
01:20:27,030 --> 01:20:30,990
and then Trump and then
Trump in another same case.

1831
01:20:30,990 --> 01:20:35,280
If we find that the
neural response is higher

1832
01:20:35,280 --> 01:20:37,800
when the two stimuli are
different than when they're

1833
01:20:37,800 --> 01:20:43,170
same, then we know
that that region

1834
01:20:43,170 --> 01:20:47,370
has neurons that respond
differentially to the two.

1835
01:20:47,370 --> 01:20:48,840
So remember, we
started with a case

1836
01:20:48,840 --> 01:20:51,000
where the mean
response is the same

1837
01:20:51,000 --> 01:20:54,240
to this image and this image
if you just measure them alone.

1838
01:20:54,240 --> 01:20:56,820
But now we want to
know, do we really

1839
01:20:56,820 --> 01:20:58,620
have neurons that
respond differentially?

1840
01:20:58,620 --> 01:21:00,600
So we're using the
fact that neurons

1841
01:21:00,600 --> 01:21:02,830
are like people and muscles.

1842
01:21:02,830 --> 01:21:05,790
If you keep doing the same
thing to them, they get bored.

1843
01:21:05,790 --> 01:21:08,040
Been there, done that.

1844
01:21:08,040 --> 01:21:10,380
So you present
this back to back.

1845
01:21:10,380 --> 01:21:14,220
You get a lower response than if
you present this and then this.

1846
01:21:14,220 --> 01:21:16,140
That's called "functional
MRI adaptation."

1847
01:21:16,140 --> 01:21:18,540
It's like that
waterfall MT adaptation

1848
01:21:18,540 --> 01:21:23,100
we talked about before, but just
crammed into a fine time scale.

1849
01:21:23,100 --> 01:21:26,370
And so then if you do that, you
can ask what a region thinks

1850
01:21:26,370 --> 01:21:28,720
is the same.

1851
01:21:28,720 --> 01:21:33,180
So then, we could ask, what
about these two images?

1852
01:21:33,180 --> 01:21:35,500
Does it think
those are the same?

1853
01:21:35,500 --> 01:21:39,010
And if we find a response like
that, what have we learned?

1854
01:21:39,010 --> 01:21:41,670
So if these two
respond like that,

1855
01:21:41,670 --> 01:21:44,760
what have we learned
about a region that shows?

1856
01:21:44,760 --> 01:21:46,300
This is all fake
data, obviously,

1857
01:21:46,300 --> 01:21:48,450
but if we saw that,
what have we learned?

1858
01:21:48,450 --> 01:21:50,940
And then I'll let
you go, as soon as I

1859
01:21:50,940 --> 01:21:52,110
get a nice answer to this.

1860
01:21:56,020 --> 01:21:56,980
Yeah.

1861
01:21:56,980 --> 01:21:59,500
AUDIENCE: So if it's the
same between two pictures

1862
01:21:59,500 --> 01:22:03,040
of the same stimuli, that
means that it's activated.

1863
01:22:03,040 --> 01:22:03,910
It can discriminate.

1864
01:22:03,910 --> 01:22:08,740
But if the yellow is at
the same degree as the red,

1865
01:22:08,740 --> 01:22:11,567
it would just be the brain
reacting to different pictures.

1866
01:22:11,567 --> 01:22:13,150
NANCY KANWISHER: You
totally get that.

1867
01:22:13,150 --> 01:22:15,400
It's probably right,
and you totally get it.

1868
01:22:15,400 --> 01:22:17,775
Key point-- just because I
don't want to torture you guys

1869
01:22:17,775 --> 01:22:22,340
and go way over-- but key point
is, it's the same response is

1870
01:22:22,340 --> 01:22:23,170
the lower response.

1871
01:22:23,170 --> 01:22:25,880
We tell that with this case, and
we actually give it a same one.

1872
01:22:25,880 --> 01:22:27,380
So same is lower than different.

1873
01:22:27,380 --> 01:22:29,830
That's just how
this method works.

1874
01:22:29,830 --> 01:22:31,240
Then we're basically
asking, does

1875
01:22:31,240 --> 01:22:34,150
that count as the same
to this brain region?

1876
01:22:34,150 --> 01:22:36,040
And we're finding, yes, it does.

1877
01:22:36,040 --> 01:22:38,320
That tells us that
those neurons are

1878
01:22:38,320 --> 01:22:40,450
invariant to all
kinds of things--

1879
01:22:40,450 --> 01:22:44,410
viewpoint, facial
expression, when he last

1880
01:22:44,410 --> 01:22:49,480
dyed his hair, who the hell
knows, all these other things.

1881
01:22:49,480 --> 01:22:51,110
So we'll talk more about this.

1882
01:22:51,110 --> 01:22:53,620
But the idea is, now we have
another method in addition

1883
01:22:53,620 --> 01:22:57,370
to MVPA that can start to tell
us what neurons are actually

1884
01:22:57,370 --> 01:22:58,360
discriminating.

1885
01:22:58,360 --> 01:23:00,720
OK, sorry to go over.