WEBVTT

00:00:01.640 --> 00:00:04.040
The following content is
provided under a Creative

00:00:04.040 --> 00:00:05.580
Commons license.

00:00:05.580 --> 00:00:07.880
Your support will help
MIT OpenCourseWare

00:00:07.880 --> 00:00:12.270
continue to offer high quality
educational resources for free.

00:00:12.270 --> 00:00:14.870
To make a donation or
view additional materials

00:00:14.870 --> 00:00:18.830
from hundreds of MIT courses,
visit MIT OpenCourseWare

00:00:18.830 --> 00:00:21.965
at ocw.mit.edu.

00:00:21.965 --> 00:00:23.960
REBECCA SAXE: I was
supposed to go before Ken,

00:00:23.960 --> 00:00:28.010
and thank goodness Ken insisted
he went before me, because

00:00:28.010 --> 00:00:30.530
in some ways that was the
most amazing introduction

00:00:30.530 --> 00:00:33.910
to my research program that you
could possibly have ever had.

00:00:33.910 --> 00:00:37.970
And it articulated deeply
why social intelligence

00:00:37.970 --> 00:00:39.770
should pervade our
thinking about the mind

00:00:39.770 --> 00:00:42.140
and brain and the
range of phenomena

00:00:42.140 --> 00:00:44.510
that people mean in
social intelligence--

00:00:44.510 --> 00:00:48.080
from extremely complex phenomena
that govern the interactions

00:00:48.080 --> 00:00:51.380
of large groups of
people, like war,

00:00:51.380 --> 00:00:54.260
to incredibly minute
phenomena, like

00:00:54.260 --> 00:00:56.700
whether you can get your
hand to a target in 100

00:00:56.700 --> 00:00:58.110
milliseconds or less.

00:00:58.110 --> 00:01:00.500
I think that when people
talk about social cognition

00:01:00.500 --> 00:01:03.830
they do actually mean
all of those things.

00:01:03.830 --> 00:01:05.990
And that is both
thrilling-- when

00:01:05.990 --> 00:01:08.810
you work in social cognition--
and also terrifying,

00:01:08.810 --> 00:01:13.370
especially when people are
hoping for a coherent theory

00:01:13.370 --> 00:01:14.780
of all of that.

00:01:14.780 --> 00:01:18.341
I think that trying to get a
coherent account of everything

00:01:18.341 --> 00:01:20.840
from your hand motions and your
perception of other people's

00:01:20.840 --> 00:01:26.360
hand motions all the way
to politics and sociology

00:01:26.360 --> 00:01:29.700
is daunting and,
frankly, deeply unlikely.

00:01:29.700 --> 00:01:34.010
And so, by contrast to Ken--

00:01:34.010 --> 00:01:37.130
who starts with, let's
look at social interactions

00:01:37.130 --> 00:01:39.800
and see what's
there, which I think

00:01:39.800 --> 00:01:41.570
is a very awesome approach--

00:01:41.570 --> 00:01:43.820
I'm going to take almost
the opposite approach, which

00:01:43.820 --> 00:01:46.400
is say, there's one thing
that's probably there a lot.

00:01:46.400 --> 00:01:49.790
Let's try to study that one
thing in many different ways

00:01:49.790 --> 00:01:51.110
and contexts.

00:01:51.110 --> 00:01:53.940
And the one thing, as Lou said
that I'm going to talk about--

00:01:53.940 --> 00:01:56.110
although, contrary to
many people's impressions,

00:01:56.110 --> 00:01:58.190
is not the only
thing I work on--

00:01:58.190 --> 00:01:59.870
is this ability that we have.

00:02:03.280 --> 00:02:07.580
OK, so a little demo of the
problem that I work on--

00:02:07.580 --> 00:02:10.190
and because it's early in the
morning and everybody needs

00:02:10.190 --> 00:02:13.020
to wake up, I'm going to get
you guys to do this as a task,

00:02:13.020 --> 00:02:14.145
so as an experiment.

00:02:14.145 --> 00:02:15.770
So in this experiment,
I'm going to ask

00:02:15.770 --> 00:02:17.900
you guys to make a moral
judgment of a character.

00:02:17.900 --> 00:02:19.040
Her name is Grace.

00:02:19.040 --> 00:02:20.630
And the way you're going
to make a moral judgment

00:02:20.630 --> 00:02:22.490
is, I'm going to tell
you something she did,

00:02:22.490 --> 00:02:24.073
and you're going to
say how much blame

00:02:24.073 --> 00:02:26.639
she deserves-- moral
blame, how wrong that was.

00:02:26.639 --> 00:02:28.430
You're going to do so
by raising your hand.

00:02:28.430 --> 00:02:30.620
The more wrong it was,
the higher your hand goes.

00:02:30.620 --> 00:02:31.790
And everybody has to vote.

00:02:31.790 --> 00:02:32.810
OK?

00:02:32.810 --> 00:02:33.500
Yes?

00:02:33.500 --> 00:02:35.600
OK, so this is a
story about Grace.

00:02:35.600 --> 00:02:37.490
She's on a tour of
a chemical factory.

00:02:37.490 --> 00:02:39.500
So they're walking around
being given a tour.

00:02:39.500 --> 00:02:40.700
There's a break in the tour.

00:02:40.700 --> 00:02:41.870
And she goes to make coffee.

00:02:41.870 --> 00:02:44.450
Another girl on the tour
asks for a cup of coffee

00:02:44.450 --> 00:02:45.600
with sugar in it.

00:02:45.600 --> 00:02:48.590
So Grace goes to the coffee
machine to make a cup of coffee

00:02:48.590 --> 00:02:50.390
for herself and for
this other girl.

00:02:50.390 --> 00:02:52.100
Next to the coffee
machine is a jar

00:02:52.100 --> 00:02:54.350
of white powder
labeled sugar, so Grace

00:02:54.350 --> 00:02:55.880
thinks the powder is sugar.

00:02:55.880 --> 00:02:58.490
She puts some of that powder
in the other girl's coffee.

00:02:58.490 --> 00:03:00.950
But it turns out that
powder is contaminated

00:03:00.950 --> 00:03:03.500
by a dangerous toxic
poison, and when the girl

00:03:03.500 --> 00:03:05.270
drinks the coffee, she dies.

00:03:05.270 --> 00:03:06.830
How much blame
does Grace deserve

00:03:06.830 --> 00:03:10.160
for putting the
powder in the coffee?

00:03:10.160 --> 00:03:13.370
OK, now what if I slightly
changed that story?

00:03:13.370 --> 00:03:16.010
So next to the coffee machine
there's a jar of white powder

00:03:16.010 --> 00:03:18.590
and it's labeled
dangerous toxic poison.

00:03:18.590 --> 00:03:20.930
So grace thinks that the
powder is toxic poison.

00:03:20.930 --> 00:03:23.210
And she puts some of the
poison in the coffee,

00:03:23.210 --> 00:03:24.939
and when the girl
drinks it, she dies.

00:03:24.939 --> 00:03:26.480
Now how much blame
does Grace deserve

00:03:26.480 --> 00:03:28.021
for putting the
powder in the coffee?

00:03:28.021 --> 00:03:31.490
So what's characteristic
about these stories is

00:03:31.490 --> 00:03:34.220
that, in the story I
told you, everything

00:03:34.220 --> 00:03:38.270
was the same from the beginning,
the scenario where Grace was,

00:03:38.270 --> 00:03:40.070
to the action and the outcome--

00:03:40.070 --> 00:03:41.590
that the girl died.

00:03:41.590 --> 00:03:43.190
But your moral
judgments differed

00:03:43.190 --> 00:03:45.470
by about the entire
scale that I gave you,

00:03:45.470 --> 00:03:48.170
from saying that she deserved
almost no blame to saying

00:03:48.170 --> 00:03:51.410
that she deserved pretty much as
much blame as you could reach.

00:03:51.410 --> 00:03:53.550
And that's the same
kind of moral judgment

00:03:53.550 --> 00:03:55.330
we get from typical
human subjects

00:03:55.330 --> 00:03:58.130
and also from MIT
undergraduates, which

00:03:58.130 --> 00:04:02.090
say that in scenarios
like the one I gave you,

00:04:02.090 --> 00:04:06.410
what matters most for the moral
blame that we assign is not

00:04:06.410 --> 00:04:08.570
what happened-- did
somebody die or not-- or how

00:04:08.570 --> 00:04:09.530
bad that outcome was.

00:04:09.530 --> 00:04:11.780
But it's what Grace thought
she was doing, whether she

00:04:11.780 --> 00:04:14.690
thought the powder was sugar or
she thought that it was poison.

00:04:14.690 --> 00:04:17.480
I should just say right away
that I set up that scenario.

00:04:17.480 --> 00:04:19.070
I gave you the
best case scenario

00:04:19.070 --> 00:04:20.180
for the role of beliefs.

00:04:20.180 --> 00:04:23.090
It's easy to make these
things way more complicated.

00:04:23.090 --> 00:04:25.820
But that scenario isolates
one important feature

00:04:25.820 --> 00:04:28.160
of our moral judgment and
also an important feature

00:04:28.160 --> 00:04:30.770
of a lot of the rest of
our social cognition.

00:04:30.770 --> 00:04:33.410
It's not how we avoid bumping
into people in subways,

00:04:33.410 --> 00:04:35.610
but a lot of the other
kind of social cognition

00:04:35.610 --> 00:04:38.600
we do about the people that are
around us, which is our ability

00:04:38.600 --> 00:04:43.140
to assign thoughts or internal
mental states to other people.

00:04:43.140 --> 00:04:44.960
So in psychology,
this ability has

00:04:44.960 --> 00:04:47.690
been studied from kind of
relatively simple perceptual

00:04:47.690 --> 00:04:50.840
phenomena like assigning
intentions and goals

00:04:50.840 --> 00:04:53.160
to simple moving
characters in an animation.

00:04:53.160 --> 00:04:55.820
This is the very famous
Heider and Simmel example

00:04:55.820 --> 00:04:57.000
from the '40s.

00:04:57.000 --> 00:04:59.416
This ability has been studied
all the way to understanding

00:04:59.416 --> 00:05:03.080
some of the most complex,
abstract ideas that we ever

00:05:03.080 --> 00:05:06.590
encounter, like the famous
apocryphal statement attributed

00:05:06.590 --> 00:05:09.472
to Alan Greenspan, which is, "I
know you think you understand

00:05:09.472 --> 00:05:11.930
what you thought I said, but
I don't think you realize that

00:05:11.930 --> 00:05:14.000
what you heard was
not what I meant. "

00:05:14.000 --> 00:05:16.610
So to the degree that our minds
let us make any sense of that

00:05:16.610 --> 00:05:18.950
at all, we're using our
ability to make sense of other

00:05:18.950 --> 00:05:20.930
people's minds.

00:05:20.930 --> 00:05:24.244
How many people here have
seen the standard test

00:05:24.244 --> 00:05:26.660
of this ability of thinking
about other people's thoughts,

00:05:26.660 --> 00:05:27.826
which the false belief task?

00:05:27.826 --> 00:05:30.950
How many people have seen
somebody do a false belief task

00:05:30.950 --> 00:05:32.550
or give a false belief task?

00:05:32.550 --> 00:05:34.970
How many people would like
to see a false belief task?

00:05:34.970 --> 00:05:36.910
OK, so then I'm just
going to show you one.

00:05:36.910 --> 00:05:40.900
So as I said, the scope
of tests of our ability

00:05:40.900 --> 00:05:43.730
to think about other people's
thoughts or internal states

00:05:43.730 --> 00:05:45.290
is very large.

00:05:45.290 --> 00:05:48.560
I'm saying that two ways on
purpose because actually,

00:05:48.560 --> 00:05:50.090
although these are
often conflated,

00:05:50.090 --> 00:05:52.298
I think there's a really
important difference between

00:05:52.298 --> 00:05:54.530
thinking about
epistemic states--

00:05:54.530 --> 00:05:56.810
so things like what
you know, what you see,

00:05:56.810 --> 00:05:57.890
and what you think--

00:05:57.890 --> 00:06:01.100
versus states like what
you want and how you feel.

00:06:01.100 --> 00:06:03.350
I think it turns out empirically
that those are really

00:06:03.350 --> 00:06:04.670
different problems.

00:06:04.670 --> 00:06:07.280
And I'm almost exclusively going
to talk about the first one,

00:06:07.280 --> 00:06:12.800
so how we think about what other
people see, think, and know--

00:06:12.800 --> 00:06:14.540
but not want or feel.

00:06:14.540 --> 00:06:16.900
At the end I'll come back
to wanting and feeling.

00:06:16.900 --> 00:06:19.430
OK, so how do we know what
other people have seen

00:06:19.430 --> 00:06:21.650
and what they think
and what they know?

00:06:21.650 --> 00:06:25.550
This problem was set up
as kind of a litmus test

00:06:25.550 --> 00:06:28.010
for our ability to think
about other people's minds,

00:06:28.010 --> 00:06:30.500
starting in the late
'70s and coming out

00:06:30.500 --> 00:06:31.750
of comparative psychology.

00:06:31.750 --> 00:06:35.150
So the origin of this
problem for psychology

00:06:35.150 --> 00:06:37.730
is, everybody knows
humans could do this.

00:06:37.730 --> 00:06:38.960
What about animals?

00:06:38.960 --> 00:06:41.621
And actually, the debate
about whether this capacity

00:06:41.621 --> 00:06:43.370
for thinking about
other people's thoughts

00:06:43.370 --> 00:06:46.670
is or is not shared with
which other animals has

00:06:46.670 --> 00:06:49.160
gone on continuously
since the late '70s

00:06:49.160 --> 00:06:50.430
and has not been resolved.

00:06:50.430 --> 00:06:52.640
That's the origin
of this debate,

00:06:52.640 --> 00:06:54.530
and it's not resolved yet.

00:06:54.530 --> 00:06:57.380
But it led to the construction
of this particular task

00:06:57.380 --> 00:07:00.800
as a litmus test for
what one person knows

00:07:00.800 --> 00:07:03.967
about somebody else's thoughts,
called the false belief task.

00:07:03.967 --> 00:07:06.050
And so here's what a false
belief task looks like.

00:07:06.050 --> 00:07:08.500
This is being given to a
five-year-old human child.

00:07:12.484 --> 00:07:13.860
This is the first pirate.

00:07:13.860 --> 00:07:15.234
His name is Ivan.

00:07:15.234 --> 00:07:17.330
Do you know what
pirates really like?

00:07:17.330 --> 00:07:18.080
CHILD: What?

00:07:18.080 --> 00:07:20.550
REBECCA SAXE: Pirates really
like cheese sandwiches.

00:07:20.550 --> 00:07:21.270
CHILD: Cheese?

00:07:21.270 --> 00:07:23.310
I love cheese!

00:07:23.310 --> 00:07:26.380
REBECCA SAXE: So Ivan has his
cheese sandwich, and he says,

00:07:26.380 --> 00:07:27.840
yum, yum, yum, yum, yum.

00:07:27.840 --> 00:07:30.130
I really love cheese sandwiches.

00:07:30.130 --> 00:07:32.790
And Ivan puts his
sandwich over here

00:07:32.790 --> 00:07:34.670
on top of the pirate chest.

00:07:34.670 --> 00:07:38.160
And Ivan says, you know what,
I need a drink with my lunch.

00:07:38.160 --> 00:07:40.880
So Ivan goes to get a drink.

00:07:40.880 --> 00:07:44.590
And while Ivan is
away, the wind comes,

00:07:44.590 --> 00:07:48.720
and it blows the sandwich
down onto the grass.

00:07:48.720 --> 00:07:51.760
And now, here comes
the other pirate.

00:07:51.760 --> 00:07:54.646
This pirate is called Joshua.

00:07:54.646 --> 00:07:56.845
And Joshua also really
loves cheese sandwiches.

00:07:56.845 --> 00:07:59.570
So Joshua has a cheese
sandwich, and he says,

00:07:59.570 --> 00:08:00.840
yum, yum, yum, yum, yum--

00:08:00.840 --> 00:08:02.100
I love cheese sandwiches.

00:08:02.100 --> 00:08:04.335
And he puts his cheese
sandwich over here

00:08:04.335 --> 00:08:05.460
on top of the pirate chest.

00:08:05.460 --> 00:08:07.246
CHILD: So that one is his.

00:08:07.246 --> 00:08:08.970
REBECCA SAXE: That
one's Joshua's.

00:08:08.970 --> 00:08:11.880
CHILD: And then his
is on the ground.

00:08:11.880 --> 00:08:12.960
REBECCA SAXE: Yeah.

00:08:12.960 --> 00:08:13.890
That's exactly right.

00:08:13.890 --> 00:08:15.956
CHILD: So he won't
know which one is his.

00:08:15.956 --> 00:08:18.884
REBECCA SAXE: Oh-- so now
Joshua goes off to get a drink.

00:08:18.884 --> 00:08:19.550
Ivan comes back.

00:08:19.550 --> 00:08:23.100
And he says, I want
my cheese sandwich.

00:08:23.100 --> 00:08:25.470
So which one do you think
Ivan's going to take?

00:08:25.470 --> 00:08:27.406
CHILD: I think he's
gonna take that one.

00:08:27.406 --> 00:08:28.860
REBECCA SAXE: Yeah, you think
he's gonna take that one.

00:08:28.860 --> 00:08:29.510
All right, let's see.

00:08:29.510 --> 00:08:30.130
CHILD: I told you.

00:08:30.130 --> 00:08:31.713
REBECCA SAXE: Oh
yeah, you were right.

00:08:31.713 --> 00:08:34.230
He took that one.

00:08:34.230 --> 00:08:36.659
OK, so that's called passing
the false belief task.

00:08:36.659 --> 00:08:38.940
And the thing that's
reported in scientific papers

00:08:38.940 --> 00:08:41.280
is that he correctly
predicted that Ivan

00:08:41.280 --> 00:08:42.510
would take Joshua's sandwich.

00:08:42.510 --> 00:08:44.010
Although if you
watch the video, you

00:08:44.010 --> 00:08:46.093
see that the knowledge the
kid is bringing to bear

00:08:46.093 --> 00:08:48.600
is a way richer than just
his correct prediction

00:08:48.600 --> 00:08:51.270
and includes him, in fact,
trying to stop me in the story

00:08:51.270 --> 00:08:52.650
to warn me of what's coming.

00:08:52.650 --> 00:08:55.740
So it's a rich interpretation
of what other people know

00:08:55.740 --> 00:08:59.700
and don't know and will know
and haven't seen and so forth.

00:08:59.700 --> 00:09:03.450
The reason why this
task became so famous

00:09:03.450 --> 00:09:07.340
is that not all participants
perform the same way.

00:09:07.340 --> 00:09:10.050
And so one class of
participants who've

00:09:10.050 --> 00:09:12.510
become the focus
of intense scrutiny

00:09:12.510 --> 00:09:14.552
is slightly younger kids,
namely three-year-olds.

00:09:14.552 --> 00:09:16.593
So I'll give you a sense
of what that looks like.

00:09:16.593 --> 00:09:17.730
This is a three-year-old.

00:09:17.730 --> 00:09:21.084
He's paid equally rapt attention
throughout the entire story.

00:09:21.084 --> 00:09:22.500
And we come to the
crucial moment,

00:09:22.500 --> 00:09:25.000
and he's asked again
the same question.

00:09:25.000 --> 00:09:27.930
And Ivan says, I want
my cheese sandwich.

00:09:27.930 --> 00:09:29.692
Which sandwich is
he going to take?

00:09:29.692 --> 00:09:31.400
Do you think he's
going to take that one?

00:09:31.400 --> 00:09:32.552
Let's see what happens.

00:09:32.552 --> 00:09:33.510
Let's see what he does.

00:09:33.510 --> 00:09:34.660
Here comes Ivan.

00:09:34.660 --> 00:09:37.305
He says, I want my
cheese sandwich.

00:09:37.305 --> 00:09:40.143
And he takes this one.

00:09:40.143 --> 00:09:43.089
Uh oh-- why did
he take that one?

00:09:46.530 --> 00:09:48.570
OK, and so the
traditional read of what

00:09:48.570 --> 00:09:51.990
just happened there is that's
a kid who gets wanting, right.

00:09:51.990 --> 00:09:54.450
Ivan wants his cheese sandwich.

00:09:54.450 --> 00:09:56.940
But he doesn't get believing.

00:09:56.940 --> 00:09:59.940
He doesn't understand that
because Ivan left his cheese

00:09:59.940 --> 00:10:01.584
sandwich on top of
the pirate chest

00:10:01.584 --> 00:10:03.000
and he doesn't
know that it's been

00:10:03.000 --> 00:10:05.910
moved that he'll believe
that that sandwich is his,

00:10:05.910 --> 00:10:08.460
and that his actions
depend on his own beliefs--

00:10:08.460 --> 00:10:10.334
his internal representation
of the world,

00:10:10.334 --> 00:10:12.000
rather than the true
state of the world,

00:10:12.000 --> 00:10:13.890
namely which one is
his cheese sandwich.

00:10:13.890 --> 00:10:17.280
And that's the source of
both this wrong prediction--

00:10:17.280 --> 00:10:20.040
why does he say that he'll
take his cheese sandwich--

00:10:20.040 --> 00:10:21.270
and the wrong explanation.

00:10:21.270 --> 00:10:23.880
So when he goes to take
the other cheese sandwich,

00:10:23.880 --> 00:10:25.960
the one that's
actually Joshua's,

00:10:25.960 --> 00:10:28.890
then we say, why did he do that.

00:10:28.890 --> 00:10:30.510
And again, this is
typical performance

00:10:30.510 --> 00:10:32.550
that the little
kids confabulate.

00:10:32.550 --> 00:10:34.464
They come up with
a reason why he

00:10:34.464 --> 00:10:35.880
might have taken
that other cheese

00:10:35.880 --> 00:10:37.770
sandwich which is
consistent with him

00:10:37.770 --> 00:10:39.390
not wanting his own anymore.

00:10:39.390 --> 00:10:42.370
So in this case, it's that
his fell on the ground.

00:10:42.370 --> 00:10:43.870
He doesn't want his anymore.

00:10:43.870 --> 00:10:46.060
That's why he's taking
Joshua's sandwich.

00:10:46.060 --> 00:10:48.420
And that pattern of
performance was interpreted

00:10:48.420 --> 00:10:51.492
as evidence of conceptual change
and development-- kids going

00:10:51.492 --> 00:10:53.700
from having a partial
understanding of other people's

00:10:53.700 --> 00:10:56.407
minds that included
wanting to a richer

00:10:56.407 --> 00:10:57.990
interpretation of
other people's minds

00:10:57.990 --> 00:11:00.270
that also included believing.

00:11:00.270 --> 00:11:04.470
So what I want to get from this
actually is not whether or not

00:11:04.470 --> 00:11:06.720
it's true that there's
conceptual change between three

00:11:06.720 --> 00:11:09.800
and five, although I
do think it is true,

00:11:09.800 --> 00:11:13.770
but just an idea of what
capacity are we talking about.

00:11:13.770 --> 00:11:15.480
We're talking about
the capacity actually

00:11:15.480 --> 00:11:17.970
that the five-year-old
showed, however they got it

00:11:17.970 --> 00:11:19.320
and whenever they got it.

00:11:19.320 --> 00:11:22.170
It's this capacity to, when
watching other people act

00:11:22.170 --> 00:11:25.620
in the world, bring to bear--
both spontaneously and when

00:11:25.620 --> 00:11:26.510
asked--

00:11:26.510 --> 00:11:28.830
a conception of the
other person having

00:11:28.830 --> 00:11:32.620
beliefs, perceptual
history, knowledge,

00:11:32.620 --> 00:11:36.500
an internal representation of
the world that guides actions.

00:11:36.500 --> 00:11:40.320
And so that is what I'm going
to call thinking about thought.

00:11:40.320 --> 00:11:43.890
And the idea that
this is a domain

00:11:43.890 --> 00:11:45.825
that you could
study on its own--

00:11:45.825 --> 00:11:47.200
well, there's two
questions here.

00:11:47.200 --> 00:11:49.530
One is can you
study this at all.

00:11:49.530 --> 00:11:51.270
And the second one
is can you study

00:11:51.270 --> 00:11:53.490
it separate from the
whole rest of cognition.

00:11:53.490 --> 00:11:56.091
Both of those are related
to Liz, and indeed Nancy,

00:11:56.091 --> 00:11:58.590
and many people's worries that
you could never make progress

00:11:58.590 --> 00:12:01.880
on a problem like
this, which I share.

00:12:01.880 --> 00:12:04.440
I share the worry that you could
never make progress on this.

00:12:04.440 --> 00:12:06.390
And so what I want
to tell you guys

00:12:06.390 --> 00:12:11.940
is two phases of my attempt to
make progress on understanding

00:12:11.940 --> 00:12:13.210
how we do that.

00:12:13.210 --> 00:12:15.090
How do we think
about other people

00:12:15.090 --> 00:12:19.440
as containing internal mental
lives, mental representations.

00:12:19.440 --> 00:12:21.630
I'm going to talk
about just fMRI,

00:12:21.630 --> 00:12:25.230
although I do use other
methods to study this problem.

00:12:25.230 --> 00:12:29.340
But I think fMRI has been
both an incredible gift

00:12:29.340 --> 00:12:31.950
to our ability to
understand the human mind

00:12:31.950 --> 00:12:34.890
and also imposes a huge
number of limitations

00:12:34.890 --> 00:12:36.900
on what we can discover.

00:12:36.900 --> 00:12:39.840
And so what I'm going to
tell you about is just

00:12:39.840 --> 00:12:43.050
a tiny bit of my phase
one investigations using

00:12:43.050 --> 00:12:45.900
the early strategies
that fMRI allowed us

00:12:45.900 --> 00:12:47.460
and then a more
in-depth look at how

00:12:47.460 --> 00:12:51.360
I'm using more modern techniques
in fMRI to try to get further.

00:12:51.360 --> 00:12:53.839
This is partly because
I think it's interesting

00:12:53.839 --> 00:12:54.630
what we've learned.

00:12:54.630 --> 00:12:57.120
But it's mainly because I
think that you guys might not

00:12:57.120 --> 00:12:58.590
actually want to know
about theory of mind,

00:12:58.590 --> 00:12:59.964
but you might want
to know if you

00:12:59.964 --> 00:13:02.580
can fMRI to study interesting
questions about the human mind

00:13:02.580 --> 00:13:03.350
and how.

00:13:03.350 --> 00:13:05.700
And so I'm going to
focus on three ways

00:13:05.700 --> 00:13:08.220
to use modern techniques
in fMRI to study

00:13:08.220 --> 00:13:10.470
interesting representations
in the human mind,

00:13:10.470 --> 00:13:13.260
hoping that either you'll learn
something about theory of mind

00:13:13.260 --> 00:13:15.150
or something about
how you could use fMRI

00:13:15.150 --> 00:13:17.500
to pursue your own interests.

00:13:17.500 --> 00:13:25.770
So phase one in fMRI, which as
Liz said started 15 years ago,

00:13:25.770 --> 00:13:26.340
is--

00:13:26.340 --> 00:13:30.300
OK, thinking about
other people's thoughts,

00:13:30.300 --> 00:13:33.450
is that a thing in the
mind and brain at all?

00:13:33.450 --> 00:13:37.510
So when you go to start studying
something, you want to know,

00:13:37.510 --> 00:13:41.000
am I studying a
part of a problem,

00:13:41.000 --> 00:13:44.580
or am I just studying the
whole mind, our entire capacity

00:13:44.580 --> 00:13:47.960
to think any interesting,
complicated thought.

00:13:47.960 --> 00:13:52.770
And fMRI turns out to
be more useful, mostly,

00:13:52.770 --> 00:13:54.240
when you're studying
something that

00:13:54.240 --> 00:13:57.770
is in some way compartmentalized
from the rest of the mind.

00:13:57.770 --> 00:14:00.090
And so one question is-- is
theory of mind, the ability

00:14:00.090 --> 00:14:02.460
to think about other people's
thoughts, in any sense

00:14:02.460 --> 00:14:04.050
its own problem?

00:14:04.050 --> 00:14:06.510
Or are we just studying
the whole problem

00:14:06.510 --> 00:14:09.330
of human intelligence
and capacity?

00:14:09.330 --> 00:14:11.460
So that was sort of
the first question

00:14:11.460 --> 00:14:12.920
that we set out to answer.

00:14:12.920 --> 00:14:15.060
We and a number of
people did this.

00:14:15.060 --> 00:14:17.130
And the way we did it
is that we had people

00:14:17.130 --> 00:14:19.680
in an fMRI machine
doing basically

00:14:19.680 --> 00:14:23.770
an adult version of the pirates
task that I just showed you.

00:14:23.770 --> 00:14:25.470
So they read short
verbal stories

00:14:25.470 --> 00:14:28.780
that describe somebody who
comes to have a false belief.

00:14:28.780 --> 00:14:29.990
This is an example.

00:14:29.990 --> 00:14:32.427
So Ann puts lasagna
in the blue dish.

00:14:32.427 --> 00:14:34.260
Ian takes the lasagna
out and puts spaghetti

00:14:34.260 --> 00:14:35.410
in the blue dish.

00:14:35.410 --> 00:14:38.420
And then we ask, what does
Ann think is in the blue dish.

00:14:38.420 --> 00:14:41.820
OK, so this is a very simple
encapsulation of our ability

00:14:41.820 --> 00:14:44.100
to represent what
somebody else thinks

00:14:44.100 --> 00:14:47.170
and separate it from
the state of the world.

00:14:47.170 --> 00:14:48.776
So while you were
doing that, you

00:14:48.776 --> 00:14:50.400
were clearly using
your theory of mind.

00:14:50.400 --> 00:14:52.180
But you were clearly
also using many,

00:14:52.180 --> 00:14:54.940
many, many other capacities
of your mind and brain,

00:14:54.940 --> 00:14:56.604
like the capacity
to see those words,

00:14:56.604 --> 00:14:58.020
to know they are
words in English,

00:14:58.020 --> 00:14:59.880
to put them together
in sentences,

00:14:59.880 --> 00:15:02.227
and then to make a response
by pushing a button.

00:15:02.227 --> 00:15:04.560
So we're using everything
from your eyes to your fingers

00:15:04.560 --> 00:15:06.450
and most of the
brain in between.

00:15:06.450 --> 00:15:08.430
And then the
question is, the part

00:15:08.430 --> 00:15:10.770
that required you
thinking about thoughts--

00:15:10.770 --> 00:15:13.350
is there any sense in which
that's special or different

00:15:13.350 --> 00:15:16.230
from the whole rest of the
logical and cognitive capacity

00:15:16.230 --> 00:15:17.370
of your brain?

00:15:17.370 --> 00:15:19.800
So to ask that question
we designed a control

00:15:19.800 --> 00:15:22.950
condition in which
you similarly read

00:15:22.950 --> 00:15:25.440
stories that involve
something that was true

00:15:25.440 --> 00:15:26.676
and becomes false.

00:15:26.676 --> 00:15:28.050
You need to think
about those two

00:15:28.050 --> 00:15:29.970
and respond using
a button press.

00:15:29.970 --> 00:15:32.350
But in this case, what it
is is a state of the world.

00:15:32.350 --> 00:15:35.062
So this is an island and
a photograph taken of it.

00:15:35.062 --> 00:15:37.020
Then the photograph, of
course, stays the same.

00:15:37.020 --> 00:15:38.130
But the world changes.

00:15:38.130 --> 00:15:39.900
So there's a
volcano that erupts.

00:15:39.900 --> 00:15:42.120
And now we can ask you
either about the photograph,

00:15:42.120 --> 00:15:44.070
what's in the
photograph, or what's

00:15:44.070 --> 00:15:46.570
the world actually like now.

00:15:46.570 --> 00:15:49.290
And the idea is that
in this comparison

00:15:49.290 --> 00:15:51.269
you need the ability
to see the stimuli,

00:15:51.269 --> 00:15:53.310
read English, put together
your logical thoughts,

00:15:53.310 --> 00:15:55.620
and choose a button
press in both cases.

00:15:55.620 --> 00:15:58.007
But only in the first
case do you also

00:15:58.007 --> 00:15:59.840
need to think about
other people's thoughts.

00:15:59.840 --> 00:16:02.790
And so that comparison would
let us look for brain regions

00:16:02.790 --> 00:16:05.280
where blood oxygenation
or metabolism is higher

00:16:05.280 --> 00:16:07.710
if you're thinking about
other people's thoughts.

00:16:07.710 --> 00:16:09.690
So that is old news now.

00:16:09.690 --> 00:16:12.530
The simple answer is that we
and many, many other groups that

00:16:12.530 --> 00:16:13.980
tried this in many
different ways

00:16:13.980 --> 00:16:15.750
found a whole group
of brain regions

00:16:15.750 --> 00:16:19.247
where metabolism or blood
oxygenation is higher

00:16:19.247 --> 00:16:21.330
if you need to think about
other people's thoughts

00:16:21.330 --> 00:16:22.375
in the stories.

00:16:22.375 --> 00:16:24.750
Part of what's interesting
though about this brain region

00:16:24.750 --> 00:16:28.230
is not just the claims
about selectivity.

00:16:28.230 --> 00:16:30.600
The other thing that's
interesting-- and extremely

00:16:30.600 --> 00:16:33.360
fortunate for
research purposes--

00:16:33.360 --> 00:16:35.340
is that the signal
is ridiculously

00:16:35.340 --> 00:16:36.327
strong and reliable.

00:16:36.327 --> 00:16:38.910
The difference between thinking
about somebody else's thoughts

00:16:38.910 --> 00:16:41.040
and other logical problems--

00:16:41.040 --> 00:16:43.500
in terms of how
significant, how reliable,

00:16:43.500 --> 00:16:45.570
how similar across
individual subjects--

00:16:45.570 --> 00:16:48.690
is comparable to the difference
between looking at gradings

00:16:48.690 --> 00:16:52.860
and not looking at gratings
in V1, which is nuts.

00:16:52.860 --> 00:16:56.140
That's crazy that something
this complicated and abstract

00:16:56.140 --> 00:17:00.175
would have an unbelievably
large, robust, reliable signal

00:17:00.175 --> 00:17:01.560
in individual subjects.

00:17:01.560 --> 00:17:03.030
I'll give you a
little hint of it.

00:17:03.030 --> 00:17:04.905
But everyone who has
ever come through my lab

00:17:04.905 --> 00:17:07.089
says that they never
believe me until they see it

00:17:07.089 --> 00:17:08.540
in their own data.

00:17:08.540 --> 00:17:10.750
And you can do this in
any individual subject.

00:17:10.750 --> 00:17:13.010
So here's just three
individual participants

00:17:13.010 --> 00:17:15.530
after five minutes to
10 minutes of scanning.

00:17:15.530 --> 00:17:18.470
You need to read only between
10 and 20 total stories

00:17:18.470 --> 00:17:19.849
in literally five to 10 minutes.

00:17:19.849 --> 00:17:21.980
And every individual
subject basically

00:17:21.980 --> 00:17:24.693
shows the same pattern
of brain activation

00:17:24.693 --> 00:17:27.109
for thinking about thoughts
compared to the other stories.

00:17:27.109 --> 00:17:30.320
It's just this unbelievably
strong signal, literally

00:17:30.320 --> 00:17:31.040
unbelievable.

00:17:31.040 --> 00:17:33.980
It should not possibly be
true on any a priori story,

00:17:33.980 --> 00:17:35.900
except for maybe the
story Ken just told you

00:17:35.900 --> 00:17:38.060
about how social cognition
is the fundamental basis

00:17:38.060 --> 00:17:39.770
of everything.

00:17:39.770 --> 00:17:43.100
When you look inside this brain
region, this is in one of them.

00:17:43.100 --> 00:17:45.140
I'm showing you pictures
of the right TPJ.

00:17:45.140 --> 00:17:46.580
It's one of five
cortical regions.

00:17:46.580 --> 00:17:47.960
I'm going to talk
a lot about it,

00:17:47.960 --> 00:17:50.860
because the data from the right
TPJ are particularly clean.

00:17:50.860 --> 00:17:53.720
So in the right TPJ, that's
average percent singal change

00:17:53.720 --> 00:17:55.735
in some of our early
experiments to stories

00:17:55.735 --> 00:17:57.110
about beliefs
compared to control

00:17:57.110 --> 00:17:58.781
stories about photographs.

00:17:58.781 --> 00:17:59.780
Two things are striking.

00:17:59.780 --> 00:18:01.910
One is that it's a
really big difference--

00:18:01.910 --> 00:18:04.032
a big positive signal when
you're reading stories

00:18:04.032 --> 00:18:05.990
about beliefs, and not
much when you're reading

00:18:05.990 --> 00:18:07.970
stories about photographs.

00:18:07.970 --> 00:18:09.574
The other thing
is that it starts

00:18:09.574 --> 00:18:11.240
at the time you start
reading the story.

00:18:11.240 --> 00:18:14.240
So you start reading a story,
and the signal starts to go up.

00:18:14.240 --> 00:18:16.820
This is just showing that
difference in how much

00:18:16.820 --> 00:18:19.070
you think about thoughts
contributes a lot of variance

00:18:19.070 --> 00:18:20.750
across many different
individual stories.

00:18:20.750 --> 00:18:21.890
And if you look
within the story,

00:18:21.890 --> 00:18:23.973
it's the time when you're
thinking about a thought

00:18:23.973 --> 00:18:26.370
that you get activity
in this brain region.

00:18:26.370 --> 00:18:28.190
We also spend a
bunch of time saying

00:18:28.190 --> 00:18:31.730
fMRI, as everybody knows,
is a correlational signal.

00:18:31.730 --> 00:18:33.980
Does this brain region
actually play a causal role

00:18:33.980 --> 00:18:35.870
in letting you think
about thoughts?

00:18:35.870 --> 00:18:38.000
And so we did a version
of the same experiment

00:18:38.000 --> 00:18:41.420
that I gave you guys on
moral reasoning with TMS

00:18:41.420 --> 00:18:44.180
and asked whether using
TMS on the right TPJ

00:18:44.180 --> 00:18:46.010
compared to a
control brain region

00:18:46.010 --> 00:18:47.870
would disproportionately
affect how

00:18:47.870 --> 00:18:50.990
you use people's mental states
in making moral judgments.

00:18:50.990 --> 00:18:54.350
We showed that after TMS to the
right TPJ compared to a control

00:18:54.350 --> 00:18:57.260
brain region, people use
the beliefs of the character

00:18:57.260 --> 00:18:59.930
less in making their
moral judgments.

00:18:59.930 --> 00:19:05.900
And so where we get to after
all of this is a hypothesis.

00:19:05.900 --> 00:19:10.940
This was after about eight
years that I was saying here's

00:19:10.940 --> 00:19:12.300
what we've learned.

00:19:12.300 --> 00:19:14.810
We've learned that the
right TPJ is selectively

00:19:14.810 --> 00:19:18.200
involved in theory
of mind, and so

00:19:18.200 --> 00:19:21.050
selectively depends on all the
experiments I didn't show you.

00:19:21.050 --> 00:19:23.650
That's a claim
about specificity.

00:19:23.650 --> 00:19:28.100
But "involved in"--
that's a euphemism.

00:19:28.100 --> 00:19:29.930
And it's a euphemism
that I think

00:19:29.930 --> 00:19:32.060
a lot of cognitive
neuroscientists

00:19:32.060 --> 00:19:33.900
use and are satisfied with.

00:19:33.900 --> 00:19:37.070
But after a while, I found
it deeply embarrassing,

00:19:37.070 --> 00:19:39.526
like-- what on earth
is "involved in"?

00:19:39.526 --> 00:19:41.150
And so what I want
to talk to you about

00:19:41.150 --> 00:19:44.280
is how to get beyond the
euphemism of "involved in"

00:19:44.280 --> 00:19:46.195
in using fMRI to
understand the mind.

00:19:46.195 --> 00:19:48.320
This is what fMRI in my
hands typically looks like.

00:19:48.320 --> 00:19:52.400
You read a bunch of
stories in the scanner,

00:19:52.400 --> 00:19:54.620
and we record activity in
the brain region-- here,

00:19:54.620 --> 00:19:56.270
for example, the right TPJ--

00:19:56.270 --> 00:19:58.100
while you're reading
those stories.

00:19:58.100 --> 00:20:00.850
And so our traditional
measures are the measures that

00:20:00.850 --> 00:20:03.462
let us estimate
specificity and selectivity

00:20:03.462 --> 00:20:05.420
and answer all the
questions you guys asked me.

00:20:05.420 --> 00:20:07.227
Is it more for
this, less for that?

00:20:07.227 --> 00:20:08.060
What makes it go up?

00:20:08.060 --> 00:20:09.230
What makes it go down?

00:20:09.230 --> 00:20:11.600
Those measures, called
univariate measures,

00:20:11.600 --> 00:20:14.600
measure the amount of
activity in that region,

00:20:14.600 --> 00:20:16.790
on average, as you
read a different story.

00:20:16.790 --> 00:20:19.250
So you get something
that looks like this.

00:20:19.250 --> 00:20:22.100
And what you show is
that this brain region

00:20:22.100 --> 00:20:23.360
responds a certain amount.

00:20:23.360 --> 00:20:27.590
Or there's a certain amount
of activity metabolism

00:20:27.590 --> 00:20:31.060
in this brain region while
you're reading that story.

00:20:31.060 --> 00:20:34.370
And so what we do
with that is we

00:20:34.370 --> 00:20:37.040
make arguments about selectivity
and these kinds of things

00:20:37.040 --> 00:20:39.306
that we've been talking
about this entire time.

00:20:39.306 --> 00:20:41.180
And if you did that in
the reverse direction,

00:20:41.180 --> 00:20:44.510
you'd say, OK, what can we
learn about these stimuli

00:20:44.510 --> 00:20:46.220
or the representation
of these stimlui--

00:20:46.220 --> 00:20:49.260
these two stimuli--
from activity like this?

00:20:49.260 --> 00:20:52.080
Well OK, both of them
are within the set

00:20:52.080 --> 00:20:53.810
that this brain
region cares about.

00:20:53.810 --> 00:20:56.090
They both elicit high activity.

00:20:56.090 --> 00:20:59.264
So both stories involve
thinking about thoughts.

00:20:59.264 --> 00:21:00.930
And one of the things
we showed early on

00:21:00.930 --> 00:21:03.170
is that activity
generalizes in the sense

00:21:03.170 --> 00:21:05.630
that many different stories
about many different kinds

00:21:05.630 --> 00:21:08.007
of thoughts all illicit
activity in this brain region.

00:21:08.007 --> 00:21:10.340
And so from the amount of
activity in this brain region,

00:21:10.340 --> 00:21:15.490
you know something like that
story is about thoughts.

00:21:15.490 --> 00:21:17.540
And I told you that I
think that that is related

00:21:17.540 --> 00:21:19.100
to this idea of involvement.

00:21:19.100 --> 00:21:21.890
This brain region
is involved when

00:21:21.890 --> 00:21:24.020
a story describes thoughts.

00:21:24.020 --> 00:21:26.690
OK, what's wrong with that for
making theoretical progress

00:21:26.690 --> 00:21:28.939
on theory of mind is
that, with respect

00:21:28.939 --> 00:21:30.980
to the representation of
other people's thoughts,

00:21:30.980 --> 00:21:36.080
that doesn't tell us anything
about how our brain does it.

00:21:36.080 --> 00:21:37.940
So for example, what
it doesn't tell us--

00:21:37.940 --> 00:21:40.370
it doesn't tell us how
we know who thinks what.

00:21:40.370 --> 00:21:43.384
It doesn't tell us
why they think that.

00:21:43.384 --> 00:21:45.050
It doesn't tell us
what the consequences

00:21:45.050 --> 00:21:46.460
were of them thinking that.

00:21:46.460 --> 00:21:48.410
It doesn't tell us
how our brains track

00:21:48.410 --> 00:21:50.880
or represent any of
these properties.

00:21:50.880 --> 00:21:53.679
So the things that would make
something a theory of mind--

00:21:53.679 --> 00:21:55.970
a representation of who thinks
what, why, and with what

00:21:55.970 --> 00:21:57.140
consequences--

00:21:57.140 --> 00:21:59.600
we can't see in the
univariate signal.

00:22:02.720 --> 00:22:05.030
So what I would like
to make progress on,

00:22:05.030 --> 00:22:08.510
what I think we're starting to
make progress on using MVPA,

00:22:08.510 --> 00:22:11.510
is getting beyond that this
brain region is involved

00:22:11.510 --> 00:22:13.950
in theory of mind and
trying to ask something

00:22:13.950 --> 00:22:17.110
about what is represented
in this brain region.

00:22:17.110 --> 00:22:21.180
And we're doing this using
a key assumption which

00:22:21.180 --> 00:22:24.240
comes from systems
neuroscience, which

00:22:24.240 --> 00:22:26.970
is that we can think
of representations

00:22:26.970 --> 00:22:30.960
in terms of population codes
of features or dimensions.

00:22:30.960 --> 00:22:34.110
And I want to say that
right now because that

00:22:34.110 --> 00:22:39.270
is an old, discredited
theory of concepts,

00:22:39.270 --> 00:22:42.960
but nevertheless a powerful
strategy in neuroscience,

00:22:42.960 --> 00:22:44.900
including in this context.

00:22:44.900 --> 00:22:48.206
It's another thing I could
talk at greater length about.

00:22:48.206 --> 00:22:49.830
So the idea that
we're going to look at

00:22:49.830 --> 00:22:55.350
is that populations of neurons
will respond differentially

00:22:55.350 --> 00:22:58.050
to features or dimensions
of our stimuli.

00:22:58.050 --> 00:23:01.500
And by figuring out what the
main features or dimensions are

00:23:01.500 --> 00:23:03.630
of our stimuli, we
can infer something

00:23:03.630 --> 00:23:06.082
about the representation
underlying--

00:23:06.082 --> 00:23:08.540
the representation that this
brain region participates in--

00:23:08.540 --> 00:23:10.920
and that is the representation
of theory of mind.

00:23:14.340 --> 00:23:16.570
OK, So what is MVPA?

00:23:16.570 --> 00:23:21.750
I'll briefly say my idea
of how to think about MVPA.

00:23:21.750 --> 00:23:24.330
So a traditional analysis--
the things that we were doing

00:23:24.330 --> 00:23:27.300
mostly for the first
15 years of fMRI--

00:23:27.300 --> 00:23:30.720
are called now
univariate analyses.

00:23:30.720 --> 00:23:33.450
You would take a
patch of cortex,

00:23:33.450 --> 00:23:36.210
as represented by a bunch of
pixels in the brain-- they're

00:23:36.210 --> 00:23:39.030
called voxels, a bunch of
volume elements in the brain

00:23:39.030 --> 00:23:42.340
you're studying-- and look at
the average amount of response.

00:23:42.340 --> 00:23:45.540
The unit of analysis was
the amount of response.

00:23:45.540 --> 00:23:47.160
These experiments
typically proceed

00:23:47.160 --> 00:23:49.950
in what's called now the
forward or encoding direction.

00:23:49.950 --> 00:23:51.960
So that is, you
have some hypothesis

00:23:51.960 --> 00:23:53.850
of what might be represented.

00:23:53.850 --> 00:23:55.830
You vary it in your stimuli.

00:23:55.830 --> 00:23:58.500
And you look at how varying
that dimension in your stimuli

00:23:58.500 --> 00:24:01.050
causes differences in the
magnitude of the thing

00:24:01.050 --> 00:24:03.360
that you're measuring.

00:24:03.360 --> 00:24:07.110
What was most effectively
revealed by these analyzes

00:24:07.110 --> 00:24:09.870
are differences in the cortex
at the scale of regions, what

00:24:09.870 --> 00:24:12.180
one region as opposed
to another region does,

00:24:12.180 --> 00:24:15.400
so what the kind of large-scale
structure of the cortex

00:24:15.400 --> 00:24:18.540
is on the scale maybe
of a centimeter.

00:24:18.540 --> 00:24:21.420
And that turns out
in many contexts--

00:24:21.420 --> 00:24:23.130
especially in the back
half of the brain,

00:24:23.130 --> 00:24:24.810
the representation regions--

00:24:24.810 --> 00:24:27.140
to correspond in some
sense to the stimulus type.

00:24:27.140 --> 00:24:29.220
What kind of thing
were you dealing with?

00:24:29.220 --> 00:24:31.570
What is it that you're
looking at or processing?

00:24:31.570 --> 00:24:34.260
And then this is, I
think, in some ways

00:24:34.260 --> 00:24:36.780
the shortest possible
version of Nancy's

00:24:36.780 --> 00:24:39.840
amazing 30-year research
program of figuring out

00:24:39.840 --> 00:24:42.900
how to parcellate
cortex into chunks

00:24:42.900 --> 00:24:45.540
of about a centimeter that
correspond to something

00:24:45.540 --> 00:24:49.110
about the type of stimulus
that we're presenting to you.

00:24:49.110 --> 00:24:52.350
And divide up in this
forward direction.

00:24:52.350 --> 00:24:53.850
Think of a type of stimulus.

00:24:53.850 --> 00:24:56.850
Find the brain region where
the magnitude of response

00:24:56.850 --> 00:25:00.060
is selective to
that stimulus type.

00:25:00.060 --> 00:25:04.370
MVPA analyses-- so
multivoxel pattern analysis--

00:25:04.370 --> 00:25:06.150
are contrasted to
this in the sense

00:25:06.150 --> 00:25:07.810
that they tend to
be multivariant.

00:25:07.810 --> 00:25:10.500
So that is, you're looking
at not how much on average

00:25:10.500 --> 00:25:13.560
a group of voxels respond,
but the relative response

00:25:13.560 --> 00:25:16.520
between one voxel and
another from trial to trial.

00:25:16.520 --> 00:25:18.090
So you're looking
at which of two

00:25:18.090 --> 00:25:20.340
voxels is higher or
lower than the other,

00:25:20.340 --> 00:25:25.650
rather than what their
overall amount of activity is.

00:25:25.650 --> 00:25:28.530
It has mostly,
though not always,

00:25:28.530 --> 00:25:30.690
been used in the reverse
or decoding direction.

00:25:30.690 --> 00:25:32.670
So the answer at the end is--

00:25:32.670 --> 00:25:35.550
given that I got this
pattern, what can I

00:25:35.550 --> 00:25:37.310
figure out about the stimulus?

00:25:37.310 --> 00:25:39.420
So that's the way many of
these analyzes proceed.

00:25:39.420 --> 00:25:42.120
You ask, having
done all of this,

00:25:42.120 --> 00:25:44.250
now I get a new
pattern of activity.

00:25:44.250 --> 00:25:45.930
What can I decode
about the stimulus

00:25:45.930 --> 00:25:49.500
from the new pattern
of neural activity?

00:25:49.500 --> 00:25:52.500
To me, these analyzes
are most interesting

00:25:52.500 --> 00:25:55.350
when they're looking for
things smaller than a region.

00:25:55.350 --> 00:25:58.920
This is again another
interesting long conversation

00:25:58.920 --> 00:26:00.630
that I would have
got to at the end.

00:26:00.630 --> 00:26:04.590
All the mathematical
techniques of MVPA

00:26:04.590 --> 00:26:07.200
could be used to rediscover
all of the things Nancy already

00:26:07.200 --> 00:26:10.320
discovered using the
traditional analyses.

00:26:10.320 --> 00:26:11.987
And in fact, if you
use them uncarefully

00:26:11.987 --> 00:26:13.528
that's what you're
most likely to do,

00:26:13.528 --> 00:26:15.520
because those are huge
signals in the brain.

00:26:15.520 --> 00:26:17.436
And so if you're not
careful, what you will do

00:26:17.436 --> 00:26:20.490
is just re-go over old
territory with new math.

00:26:20.490 --> 00:26:22.260
I am more interested
in these techniques

00:26:22.260 --> 00:26:24.400
when they let us see things
we could never see before.

00:26:24.400 --> 00:26:26.941
So when, instead of telling us
about region level differences

00:26:26.941 --> 00:26:29.340
or centimeter scale
differences, they're

00:26:29.340 --> 00:26:32.160
telling us about much
smaller and more interleaved

00:26:32.160 --> 00:26:37.770
populations on the spatial
and representational skills

00:26:37.770 --> 00:26:40.470
and when what they're
revealing are not the type

00:26:40.470 --> 00:26:41.925
classifications of stimuli--

00:26:41.925 --> 00:26:43.800
so the things that decide
whether this region

00:26:43.800 --> 00:26:45.680
or that region will
be more activated--

00:26:45.680 --> 00:26:48.210
but for a given type
of stimuli, what

00:26:48.210 --> 00:26:50.560
are the key dimensions
of representation.

00:26:50.560 --> 00:26:55.530
So the reason why I think MVPA
is giving a new life to fMRI

00:26:55.530 --> 00:26:58.230
is because many of the
most interesting questions

00:26:58.230 --> 00:27:00.270
about cognition and
cognitive science

00:27:00.270 --> 00:27:04.050
that we wanted to answer and
that fMRI never let us answer

00:27:04.050 --> 00:27:10.150
were about within-stimulus
type features or dimensions.

00:27:10.150 --> 00:27:14.160
What makes this face look
like person A versus person B?

00:27:14.160 --> 00:27:19.240
What makes this thought predict
moral blame versus not blame?

00:27:19.240 --> 00:27:22.950
So within-type dimensions of
importance-- and MVP I think

00:27:22.950 --> 00:27:26.670
is letting us do that in its
most interesting applications.

00:27:26.670 --> 00:27:28.980
The intuition here is that--

00:27:28.980 --> 00:27:32.070
think about a region, like the
right TPJ, or the face area

00:27:32.070 --> 00:27:33.650
if you think about faces.

00:27:33.650 --> 00:27:37.110
Or V1 is often where I start,
because we know enough about V1

00:27:37.110 --> 00:27:40.120
that I can use it to imagine
what we're talking about.

00:27:40.120 --> 00:27:42.360
So you can think
about that whole area.

00:27:42.360 --> 00:27:45.100
And you think, what can we
learn about what it does.

00:27:45.100 --> 00:27:47.060
So let's talk about V1.

00:27:47.060 --> 00:27:50.190
Does everybody here
have some sense of V1?

00:27:50.190 --> 00:27:52.620
Everyone's had kind of a first
introductory neuroscience

00:27:52.620 --> 00:27:53.670
class, OK.

00:27:53.670 --> 00:27:58.170
So V1 is called V1 because
information goes from your eyes

00:27:58.170 --> 00:28:01.270
to the LGN of your
thalamus to V1.

00:28:01.270 --> 00:28:05.670
It's the first cortical
stop of visual information.

00:28:05.670 --> 00:28:09.384
And one way that we know that
it's very involved in vision

00:28:09.384 --> 00:28:10.800
is that if you
were doing visions,

00:28:10.800 --> 00:28:13.310
if you're seeing visual stimuli,
you get a big response in V1.

00:28:13.310 --> 00:28:14.730
If you're not seeing
visual stimuli,

00:28:14.730 --> 00:28:16.290
like you're hearing
auditory stimuli

00:28:16.290 --> 00:28:19.020
or feeling tactile stimuli, you
don't get a big response in V1.

00:28:19.020 --> 00:28:21.374
So that's a selectivity
type measure.

00:28:21.374 --> 00:28:23.790
It's a univariate measure for
the amount of activity in VI

00:28:23.790 --> 00:28:25.620
that tells you V1 is
in some way involved

00:28:25.620 --> 00:28:31.440
in vision, relative to
audition or somatic sensation.

00:28:31.440 --> 00:28:35.370
But that misses pretty much all
the interesting contributions

00:28:35.370 --> 00:28:37.200
that visual cortex
makes to vision.

00:28:37.200 --> 00:28:39.450
What we want to
know about V1 is not

00:28:39.450 --> 00:28:41.370
that it is involved when
you are doing vision

00:28:41.370 --> 00:28:43.500
and not involved when
you are not doing vision.

00:28:43.500 --> 00:28:45.180
We want to know
what transformations

00:28:45.180 --> 00:28:47.300
over the information
coming from LGN

00:28:47.300 --> 00:28:50.470
is V1 implementing-- what
computational transformations,

00:28:50.470 --> 00:28:51.660
what representations.

00:28:51.660 --> 00:28:54.510
And that's why theories
like Marr's theory-- which

00:28:54.510 --> 00:28:59.370
say that it's edge detection,
or that there are receptive

00:28:59.370 --> 00:29:02.130
fields, that it depends on
the contrast, the position,

00:29:02.130 --> 00:29:05.880
and the orientation
of the information

00:29:05.880 --> 00:29:09.760
in the field that counts as an
account of the representation

00:29:09.760 --> 00:29:13.080
that V1 forms of the image
in the first bottom-up sweep.

00:29:13.080 --> 00:29:15.600
In a way, that's saying
"it's involved in vision"

00:29:15.600 --> 00:29:17.780
doesn't even begin to count.

00:29:17.780 --> 00:29:21.600
OK, so the question is, if
we were going to look at V1,

00:29:21.600 --> 00:29:25.810
could we discover from
fMRI that V1, for example,

00:29:25.810 --> 00:29:28.470
has an orientation
map, that neurons in V1

00:29:28.470 --> 00:29:29.940
have an orientation preference?

00:29:29.940 --> 00:29:32.040
That's a key feature
of neurons in V1.

00:29:32.040 --> 00:29:34.800
It's a key feature of
the computation V1 does.

00:29:34.800 --> 00:29:36.367
Different from the
LGN and the retina

00:29:36.367 --> 00:29:38.700
is the orientation map, a
preference for the orientation

00:29:38.700 --> 00:29:41.490
of a contrast and edge.

00:29:41.490 --> 00:29:45.300
And the answer in
standard analyses

00:29:45.300 --> 00:29:49.000
is-- no, you can't,
because V1 as a whole

00:29:49.000 --> 00:29:52.500
will activate to big images
regardless of the orientation

00:29:52.500 --> 00:29:53.850
of the content of the image.

00:29:53.850 --> 00:29:57.270
So you need to be able to get
to something more fine-grained

00:29:57.270 --> 00:29:57.930
than V1.

00:29:57.930 --> 00:30:00.780
You need to be able to say there
are different subpopulations

00:30:00.780 --> 00:30:03.870
of neurons inside
V1, some of which

00:30:03.870 --> 00:30:06.570
will be responding when a line
is like this, and some of which

00:30:06.570 --> 00:30:08.502
will be responding when
a line is like that.

00:30:08.502 --> 00:30:09.960
And that's the
decoding perspective

00:30:09.960 --> 00:30:12.360
that says, if we
wanted to look at V1

00:30:12.360 --> 00:30:15.210
and know is the line
like this or like that,

00:30:15.210 --> 00:30:19.170
the way we would tell is not how
much activity there is in V1.

00:30:19.170 --> 00:30:21.270
But is there relatively
more activity

00:30:21.270 --> 00:30:23.070
in the population
of neurons that

00:30:23.070 --> 00:30:25.590
responds like this, or in
the subpopulation of neurons

00:30:25.590 --> 00:30:26.820
that responds like that?

00:30:26.820 --> 00:30:30.370
And it's the relative activity
in those two populations

00:30:30.370 --> 00:30:32.550
that would let you say,
is the line like this,

00:30:32.550 --> 00:30:33.660
or is it like that.

00:30:33.660 --> 00:30:37.022
That's population coding
or population decoding.

00:30:37.022 --> 00:30:38.730
And then you take that
to the fMRI level.

00:30:38.730 --> 00:30:42.300
So now you want to
say, can we tell

00:30:42.300 --> 00:30:43.800
which of those
two subpopulations

00:30:43.800 --> 00:30:46.260
is more active in fMRI?

00:30:46.260 --> 00:30:48.404
Now, if you could measure
the individual neurons--

00:30:48.404 --> 00:30:50.820
so if you know these neurons
prefer this and these neurons

00:30:50.820 --> 00:30:53.760
prefer that, and then I measure
your firing patterns-- then

00:30:53.760 --> 00:30:56.010
decoding from the
population is simple.

00:30:56.010 --> 00:30:58.830
What makes it
really hard in fMRI

00:30:58.830 --> 00:31:03.210
is that the unit of measurement
is the blood oxygenation

00:31:03.210 --> 00:31:07.990
in 100,000 neurons, 200,000
neurons, maybe 500,000 neurons.

00:31:07.990 --> 00:31:11.310
And so it seems
potentially really unlikely

00:31:11.310 --> 00:31:14.130
that you would be able to
tell from the fMRI signal

00:31:14.130 --> 00:31:17.134
whether the neurons that
prefer bars like this or bars

00:31:17.134 --> 00:31:19.050
like this are more active,
because they're all

00:31:19.050 --> 00:31:22.810
intermixed inside a single
measurement in fMRI.

00:31:22.810 --> 00:31:26.520
And so it's not stupid that
we used to focus on things

00:31:26.520 --> 00:31:27.870
like how much activity.

00:31:27.870 --> 00:31:31.470
The reason we used to focus
on how much activity with fMRI

00:31:31.470 --> 00:31:33.660
is that it was quite
plausible that that's all fMRI

00:31:33.660 --> 00:31:34.650
could tell us.

00:31:34.650 --> 00:31:37.890
The neural populations, like
orientation preferring neurons

00:31:37.890 --> 00:31:42.750
in V1, were too spatially
mixed to tell the difference

00:31:42.750 --> 00:31:43.920
between them in fMRI.

00:31:43.920 --> 00:31:46.410
And so what we were going to
get was just how much activity

00:31:46.410 --> 00:31:48.340
in the population as a whole.

00:31:48.340 --> 00:31:51.420
So the traditional way of
thinking about what you got out

00:31:51.420 --> 00:31:54.630
of fMRI is, yes, you would see
differences across voxels, so

00:31:54.630 --> 00:31:56.550
these fine spatial patterns.

00:31:56.550 --> 00:31:58.230
But there's so many
things that could

00:31:58.230 --> 00:32:00.870
cause fine spatial patterns
that we don't care about--

00:32:00.870 --> 00:32:02.490
noise, for a start.

00:32:02.490 --> 00:32:05.440
Where the blood vessels
happen to be is another thing.

00:32:05.440 --> 00:32:09.540
And so people assumed,
I think very reasonably,

00:32:09.540 --> 00:32:12.660
that because fMRI is such
a core spatial measure,

00:32:12.660 --> 00:32:14.690
that the only thing
it could tell you

00:32:14.690 --> 00:32:18.290
was the average over
the millions of neurons

00:32:18.290 --> 00:32:20.180
that make up a region.

00:32:20.180 --> 00:32:22.295
And there's a key
intuition underlying MVPA.

00:32:27.180 --> 00:32:29.610
So there's the big signal
which is the regional signal--

00:32:29.610 --> 00:32:32.990
V1 and vision-- and
there's lots of noise.

00:32:32.990 --> 00:32:34.670
But there might
also be inside there

00:32:34.670 --> 00:32:37.800
a tiny bit of spatial pattern
that says something like--

00:32:37.800 --> 00:32:40.380
well, this voxel happens
to have more neurons that

00:32:40.380 --> 00:32:41.910
prefer one orientation.

00:32:41.910 --> 00:32:43.910
And this voxel happens
to have more neurons that

00:32:43.910 --> 00:32:45.630
prefer a different orientation.

00:32:45.630 --> 00:32:48.560
And so from the relative
activity in those two voxels,

00:32:48.560 --> 00:32:50.870
we could still tell
you the orientation--

00:32:50.870 --> 00:32:53.630
even though that would be a
tiny, subtle little signal

00:32:53.630 --> 00:32:56.060
superimposed on top of
this massive signal, which

00:32:56.060 --> 00:32:57.170
is the average of V1.

00:32:57.170 --> 00:33:01.340
That was the intuition behind
multivoxel pattern analysis

00:33:01.340 --> 00:33:03.920
when it was first proposed.

00:33:03.920 --> 00:33:09.080
And it's now sweeping the fMRI
world, many different versions

00:33:09.080 --> 00:33:10.220
of these analyses.

00:33:10.220 --> 00:33:11.540
And so actually what
I'm going to do again--

00:33:11.540 --> 00:33:13.400
to give you a more concrete
sense of what this is

00:33:13.400 --> 00:33:14.900
and how it works--
is I'm just going

00:33:14.900 --> 00:33:17.990
to show you two different
ways MVPA is done concretely

00:33:17.990 --> 00:33:20.689
in my lab to try to get you more
of a sense of what's going on.

00:33:20.689 --> 00:33:22.730
And we can come back to
these more general issues

00:33:22.730 --> 00:33:25.100
of what it's measuring
and what that means.

00:33:25.100 --> 00:33:28.940
OK, so here's what it
looks like when we do MVPA.

00:33:28.940 --> 00:33:31.940
Again, if it helps, think
about the analogy from vision.

00:33:31.940 --> 00:33:35.090
We've gone from saying, is
this vision or audition,

00:33:35.090 --> 00:33:37.440
to trying to say which
orientation is it.

00:33:37.440 --> 00:33:40.600
So we're moving from saying,
is this theory of mind or not,

00:33:40.600 --> 00:33:43.190
to trying to say anything
interesting about the space

00:33:43.190 --> 00:33:45.380
within theory of mind--
some dimension that might

00:33:45.380 --> 00:33:48.140
matter within theory of mind.

00:33:48.140 --> 00:33:50.600
And the first
dimension or potential

00:33:50.600 --> 00:33:54.500
feature that we wanted to look
for we chose because it really

00:33:54.500 --> 00:33:56.204
matters to human judgments.

00:33:56.204 --> 00:33:57.620
And it's the one
that you guys did

00:33:57.620 --> 00:33:58.940
in the very beginning
of my talk--

00:33:58.940 --> 00:34:01.148
telling the difference
between somebody who knowingly

00:34:01.148 --> 00:34:03.840
or unknowingly commits murder.

00:34:03.840 --> 00:34:07.160
That, as you saw, makes a huge,
huge difference in behavior.

00:34:07.160 --> 00:34:10.340
And also, we know it's
represented in the right TPJ

00:34:10.340 --> 00:34:12.020
because of the TMS experiment.

00:34:12.020 --> 00:34:14.210
If we mess up the
signaling in the TPJ,

00:34:14.210 --> 00:34:15.690
we change that judgment.

00:34:15.690 --> 00:34:18.164
And so that was our best
guess, that if any feature

00:34:18.164 --> 00:34:19.580
of other people's
mental states is

00:34:19.580 --> 00:34:22.280
represented in the right TPJ,
it would be that feature.

00:34:22.280 --> 00:34:24.230
If MVPA was ever going
to be able to decode

00:34:24.230 --> 00:34:26.020
a feature of other
people's mental states,

00:34:26.020 --> 00:34:26.989
we should start there.

00:34:26.989 --> 00:34:28.190
That was the idea.

00:34:28.190 --> 00:34:30.530
OK, so here's how
these experiments go.

00:34:30.530 --> 00:34:34.760
In every every trial you read
a long, complicated story

00:34:34.760 --> 00:34:36.250
that sets up a murder.

00:34:36.250 --> 00:34:37.699
So here's an example.

00:34:37.699 --> 00:34:38.949
Your family's over for dinner.

00:34:38.949 --> 00:34:40.894
You want to show off
your culinary skills

00:34:40.894 --> 00:34:41.810
for one of the dishes.

00:34:41.810 --> 00:34:43.420
Adding peanuts will
bring out the flavor.

00:34:43.420 --> 00:34:44.919
So you grind up
peanuts and put them

00:34:44.919 --> 00:34:46.497
in the dish and feed everyone.

00:34:46.497 --> 00:34:48.080
Your cousin, one of
the dinner guests,

00:34:48.080 --> 00:34:50.270
is severely allergic to peanuts.

00:34:50.270 --> 00:34:52.639
You had absolutely no
idea about his allergy

00:34:52.639 --> 00:34:54.699
when you added the peanuts.

00:34:54.699 --> 00:34:57.110
And then at the end we ask
how much blame you should get.

00:34:57.110 --> 00:34:58.401
Somebody asked me this earlier.

00:34:58.401 --> 00:35:00.800
This is in the second
person and doesn't matter.

00:35:00.800 --> 00:35:03.216
Somebody asked me if you could
do it in the second person,

00:35:03.216 --> 00:35:04.760
and you can.

00:35:04.760 --> 00:35:06.770
What's nice about
this experiment

00:35:06.770 --> 00:35:09.500
is that we can do a
relatively minimal pair.

00:35:09.500 --> 00:35:11.060
So in all of our
old experiments we

00:35:11.060 --> 00:35:13.340
wrote one set of stories
about people's mental states

00:35:13.340 --> 00:35:16.100
and a completely different set
of stories about other things.

00:35:16.100 --> 00:35:18.800
And those stories are different
in many, many, many ways.

00:35:18.800 --> 00:35:22.490
In this experiment, we
make one tiny change.

00:35:22.490 --> 00:35:24.380
So we make, for example,
a change from you

00:35:24.380 --> 00:35:26.780
had no idea to you knew.

00:35:26.780 --> 00:35:28.910
We change on average
two to four words

00:35:28.910 --> 00:35:30.950
in this whole long scenario.

00:35:30.950 --> 00:35:32.846
So we can make these
tiny interventions.

00:35:32.846 --> 00:35:34.970
It's a complicated stimulus,
but the change we make

00:35:34.970 --> 00:35:37.580
is very small and
totally changed

00:35:37.580 --> 00:35:39.320
the meaning of the
whole story by just

00:35:39.320 --> 00:35:41.270
changing your mental state.

00:35:41.270 --> 00:35:45.037
OK, what univariate
analyses would say is,

00:35:45.037 --> 00:35:46.370
this is a really important fact.

00:35:46.370 --> 00:35:48.953
Whether you knew or you didn't
know about your cousin's peanut

00:35:48.953 --> 00:35:51.080
allergy is really important
to the moral judgment

00:35:51.080 --> 00:35:52.700
of what happened.

00:35:52.700 --> 00:35:53.760
We know that.

00:35:53.760 --> 00:35:55.380
And it's represented
in the right TPJ,

00:35:55.380 --> 00:35:56.880
because if we TMS
the right TPJ, you

00:35:56.880 --> 00:35:58.940
make your moral judgments
of this distinction

00:35:58.940 --> 00:36:00.750
specifically change.

00:36:00.750 --> 00:36:04.550
But if you ask how much
does the right TPJ respond

00:36:04.550 --> 00:36:06.320
to these stories, the
answer is the right

00:36:06.320 --> 00:36:09.440
TPJ responds exactly equally
to these two conditions.

00:36:09.440 --> 00:36:11.840
And the intuition is,
because in both cases

00:36:11.840 --> 00:36:13.667
it matters what you think.

00:36:13.667 --> 00:36:16.250
It matters that you knew, and
it matters that you didn't know.

00:36:16.250 --> 00:36:19.146
And the right TPJ is tracking
the important information

00:36:19.146 --> 00:36:20.020
about what you think.

00:36:20.020 --> 00:36:22.395
And so it's activated for both
of these kinds of stories.

00:36:22.395 --> 00:36:24.950
So that's a univariate analysis.

00:36:24.950 --> 00:36:27.930
Now what's a
multivariate analysis?

00:36:27.930 --> 00:36:31.560
So here's the key intuition
behind a multivariate analysis.

00:36:31.560 --> 00:36:35.150
The idea is, think in a very
abstract similarity space.

00:36:35.150 --> 00:36:37.330
If we take the two stories--

00:36:37.330 --> 00:36:39.830
and so we take the story you
had no idea about your cousin's

00:36:39.830 --> 00:36:41.288
allergy when you
added the peanuts.

00:36:43.810 --> 00:36:45.050
That story is complicated.

00:36:45.050 --> 00:36:47.680
It has many
important dimensions.

00:36:47.680 --> 00:36:49.280
Now we take a new story.

00:36:49.280 --> 00:36:54.364
This is a story about, for
example, a faulty parachute.

00:36:54.364 --> 00:36:56.780
Within that story there's many,
many different dimensions.

00:36:56.780 --> 00:36:57.790
It's about parachutes.

00:36:57.790 --> 00:36:59.706
There's all kinds of
complicated things going.

00:36:59.706 --> 00:37:01.270
But there's this one feature--

00:37:01.270 --> 00:37:05.140
whether you knew or didn't know
that the parachute was faulty.

00:37:05.140 --> 00:37:08.290
There's another story about
publicly shaming your classmate

00:37:08.290 --> 00:37:10.660
by saying something
embarrassing about their essay.

00:37:10.660 --> 00:37:12.250
So again, that's a
whole new scenario

00:37:12.250 --> 00:37:13.708
with all kinds of
dimensions in it.

00:37:13.708 --> 00:37:14.950
But there's this one feature.

00:37:14.950 --> 00:37:17.800
Did you know or not know that
the person who wrote the essay

00:37:17.800 --> 00:37:20.740
was in the room when you said
that publicly shaming thing?

00:37:20.740 --> 00:37:22.870
A different story is
about demonstrating

00:37:22.870 --> 00:37:26.230
your karate skills and
knocking out your classmate--

00:37:26.230 --> 00:37:28.210
again, totally new
moral scenario.

00:37:28.210 --> 00:37:29.590
But again, this
one feature-- did

00:37:29.590 --> 00:37:31.673
you know or not know that
your classmate was there

00:37:31.673 --> 00:37:33.050
when you did the kick?

00:37:33.050 --> 00:37:34.476
Now here's the idea.

00:37:34.476 --> 00:37:36.100
Even though each of
those new scenarios

00:37:36.100 --> 00:37:38.530
is completely
different, if there

00:37:38.530 --> 00:37:40.930
are different subpopulations
within your right TPJ

00:37:40.930 --> 00:37:43.930
responding when you knew you
were going to cause harm--

00:37:43.930 --> 00:37:46.600
compared to when you didn't know
you were going to cause harm--

00:37:46.600 --> 00:37:49.240
then even though the pattern of
activity in your right TPJ will

00:37:49.240 --> 00:37:51.190
be different on every trial--
because you're representing

00:37:51.190 --> 00:37:53.273
a different person having
a different mental state

00:37:53.273 --> 00:37:54.400
in a different context--

00:37:54.400 --> 00:37:56.840
a little part of that
response will be the same.

00:37:56.840 --> 00:37:58.690
Or it will be different
in the same way,

00:37:58.690 --> 00:38:00.400
right, because the
same cell population

00:38:00.400 --> 00:38:03.990
will be more active for all the
stories that have knowing harm.

00:38:03.990 --> 00:38:06.659
And the other population
will be relatively

00:38:06.659 --> 00:38:08.950
active in all the stories
that have the unknowing harm.

00:38:08.950 --> 00:38:12.430
And so the logic is that if
we could look in the right TPJ

00:38:12.430 --> 00:38:14.840
and measure the
pattern of activity--

00:38:14.840 --> 00:38:18.490
and hope that reflects something
like the relative activation

00:38:18.490 --> 00:38:22.120
of different cell populations
inside the right TPJ--

00:38:22.120 --> 00:38:24.100
that the pattern
of activity would

00:38:24.100 --> 00:38:29.500
be more similar for pairs or
subsets of stories that share

00:38:29.500 --> 00:38:32.890
this one feature, and are
different in every other way,

00:38:32.890 --> 00:38:35.920
compared to pairs that are
different in every other way

00:38:35.920 --> 00:38:37.610
and don't share that feature.

00:38:37.610 --> 00:38:40.480
OK, so this is
the central logic.

00:38:40.480 --> 00:38:42.299
Take any two stories
within the set.

00:38:42.299 --> 00:38:43.090
They're all unique.

00:38:43.090 --> 00:38:44.260
So those two stories
that are different,

00:38:44.260 --> 00:38:46.551
you're representing a new
mental state of a new person.

00:38:46.551 --> 00:38:48.730
You have a new pattern
in your right TPJ.

00:38:48.730 --> 00:38:50.350
But if they share
the feature that you

00:38:50.350 --> 00:38:51.820
knew you were going
to cause harm,

00:38:51.820 --> 00:38:54.236
that would be something a
little bit similar in your right

00:38:54.236 --> 00:38:57.160
TPJ activation compared to if
they don't show that feature.

00:38:57.160 --> 00:38:58.780
Does that logic make sense?

00:38:58.780 --> 00:39:01.870
And so what you get is a
spatial pattern of activation.

00:39:01.870 --> 00:39:05.590
So we're now not looking at how
much the right TPJ responded.

00:39:05.590 --> 00:39:07.920
But within the space
of the right TPJ,

00:39:07.920 --> 00:39:10.300
where was there a little bit
more or a little bit less

00:39:10.300 --> 00:39:10.910
activity?

00:39:10.910 --> 00:39:13.090
And these signals are
tiny compared to the thing

00:39:13.090 --> 00:39:14.210
I showed you before.

00:39:14.210 --> 00:39:17.380
So the amount of activity in
the right TPJ is a big signal.

00:39:17.380 --> 00:39:20.170
The relative activity
between one voxel and another

00:39:20.170 --> 00:39:21.580
is a tiny signal.

00:39:21.580 --> 00:39:23.930
And it's superimposed
on a lot of noise.

00:39:23.930 --> 00:39:25.480
But if there's
anything there at all,

00:39:25.480 --> 00:39:26.980
then you'll still
be able to pick up

00:39:26.980 --> 00:39:29.020
a little more
similarity for pairs

00:39:29.020 --> 00:39:31.010
that are matched on
the feature of interest

00:39:31.010 --> 00:39:32.966
compared to pairs
that are not matched

00:39:32.966 --> 00:39:34.090
on the feature of interest.

00:39:34.090 --> 00:39:37.030
That's the logic behind
a Haxby style analysis.

00:39:37.030 --> 00:39:38.890
And so literally
what you do is, you

00:39:38.890 --> 00:39:41.680
take the vector of responses
across all the voxels

00:39:41.680 --> 00:39:44.380
inside a region, and
you correlate them

00:39:44.380 --> 00:39:46.240
across subsets of your data.

00:39:46.240 --> 00:39:48.910
And you ask whether the
correlation in space--

00:39:48.910 --> 00:39:51.160
so what it looks like,
the spatial pattern

00:39:51.160 --> 00:39:54.370
of activity over those voxels--

00:39:54.370 --> 00:39:57.250
is more similar for
pairs that share

00:39:57.250 --> 00:40:00.209
the feature you're interested
in compared to pairs

00:40:00.209 --> 00:40:02.500
that don't share the feature
that you're interested in.

00:40:02.500 --> 00:40:04.510
And what you get
at is two numbers--

00:40:04.510 --> 00:40:06.959
the correlation
for pairs that do

00:40:06.959 --> 00:40:08.500
share the feature
and the correlation

00:40:08.500 --> 00:40:10.240
for pairs that don't
share the feature

00:40:10.240 --> 00:40:12.640
for each individual subject.

00:40:12.640 --> 00:40:15.370
And the question you ask in
a Haxby style correlation

00:40:15.370 --> 00:40:18.600
is what's called the
within-condition correlation.

00:40:18.600 --> 00:40:20.350
So the spatial correlation
of the response

00:40:20.350 --> 00:40:22.810
to two independent sets
of stories that share

00:40:22.810 --> 00:40:24.280
this one feature--

00:40:24.280 --> 00:40:27.850
is the spatial pattern
more similar in that pair

00:40:27.850 --> 00:40:30.850
compared to a pair that don't
share that feature, when

00:40:30.850 --> 00:40:33.430
everything else is different?

00:40:33.430 --> 00:40:35.800
And so what you get out
of an analysis like this--

00:40:35.800 --> 00:40:38.020
for example, in our first
attempt to do this in these

00:40:38.020 --> 00:40:38.659
stimuli--

00:40:38.659 --> 00:40:39.950
there's these two correlations.

00:40:39.950 --> 00:40:41.050
There's the
within-condition correlation

00:40:41.050 --> 00:40:42.633
and the between-condition
correlation,

00:40:42.633 --> 00:40:44.620
and you ask if
they're different.

00:40:44.620 --> 00:40:47.350
OK, and what we got in
our first experiment

00:40:47.350 --> 00:40:50.260
is that the within-condition
correlation is significantly

00:40:50.260 --> 00:40:57.730
but a tiny bit stronger than the
between-condition correlation.

00:40:57.730 --> 00:41:00.190
So there's a lot of
things to ask about this.

00:41:00.190 --> 00:41:01.840
But the first question is--

00:41:01.840 --> 00:41:03.994
is that real, or is
that a coincidence?

00:41:03.994 --> 00:41:05.660
That is the first
thing you want to know

00:41:05.660 --> 00:41:06.490
when you see data like this.

00:41:06.490 --> 00:41:08.220
Afterwards, we can
ask what does it mean.

00:41:08.220 --> 00:41:10.690
But let's start
with is that real.

00:41:10.690 --> 00:41:12.711
And so the way that
you ask is it real is,

00:41:12.711 --> 00:41:14.710
you just make sure that
it would replicate, that

00:41:14.710 --> 00:41:17.180
in independent data you'd
get the same answer.

00:41:17.180 --> 00:41:20.110
And so just before we set out
to actually replicate this

00:41:20.110 --> 00:41:22.180
experiment, we remember
that we had actually

00:41:22.180 --> 00:41:24.760
already run this experiment
two times before--

00:41:24.760 --> 00:41:27.190
because we were studying
this process of representing

00:41:27.190 --> 00:41:29.440
accidental and intentional
harms for a long time

00:41:29.440 --> 00:41:31.150
before we thought of using MVPA.

00:41:31.150 --> 00:41:32.680
So we had these
two old data sets

00:41:32.680 --> 00:41:35.851
in the lab, two whole
independent experiments

00:41:35.851 --> 00:41:38.350
in which people had read stories
about knowing and unknowing

00:41:38.350 --> 00:41:39.394
harm.

00:41:39.394 --> 00:41:40.810
And the other thing
is that we had

00:41:40.810 --> 00:41:44.020
manipulated this distinction
in different ways

00:41:44.020 --> 00:41:45.190
across the stimuli.

00:41:45.190 --> 00:41:46.669
So in the example
I just told you,

00:41:46.669 --> 00:41:49.210
the way that we did it is, we
said you knew about the allergy

00:41:49.210 --> 00:41:51.160
or you didn't know
about the allergy.

00:41:51.160 --> 00:41:53.304
But in the older experiments,
like this example

00:41:53.304 --> 00:41:54.970
I gave you at the
beginning of the talk,

00:41:54.970 --> 00:41:57.040
we had described two
different beliefs--

00:41:57.040 --> 00:41:58.960
so either believing
that it's sugar

00:41:58.960 --> 00:42:01.282
or believing that it's
poison, so no negation.

00:42:01.282 --> 00:42:03.490
This is just important
because that's a different way

00:42:03.490 --> 00:42:05.442
to create the same distinction.

00:42:05.442 --> 00:42:07.900
And what you want to know is,
are you decoding the abstract

00:42:07.900 --> 00:42:10.120
thing-- that she knew she
was causing harm or not--

00:42:10.120 --> 00:42:12.310
or something less abstract,
like whether the story

00:42:12.310 --> 00:42:13.450
has negation in it.

00:42:13.450 --> 00:42:15.580
That's an alternative
possibility.

00:42:15.580 --> 00:42:19.540
And so in experiments B and
C, we had done it this way.

00:42:19.540 --> 00:42:21.950
It's also in the third
person, not the second person.

00:42:21.950 --> 00:42:23.620
So if we find the
same result, then it

00:42:23.620 --> 00:42:26.110
generalizes across all these
incidental features of the way

00:42:26.110 --> 00:42:27.900
the experiment was run.

00:42:27.900 --> 00:42:30.560
OK, that's experiment two,
and that's experiment three.

00:42:30.560 --> 00:42:33.700
I also want to say that there's
some weird magical property

00:42:33.700 --> 00:42:35.980
of being a scientist,
where if you don't

00:42:35.980 --> 00:42:38.410
know the hypothesis when
you're running the experiment

00:42:38.410 --> 00:42:41.020
and you have all the data and
then you go back and check,

00:42:41.020 --> 00:42:42.570
there's something
more real about it

00:42:42.570 --> 00:42:44.320
than if you knew the
hypothesis before you

00:42:44.320 --> 00:42:47.494
ran the experiment-- even though
that makes no sense whatsoever.

00:42:47.494 --> 00:42:48.910
There's just this
experience like,

00:42:48.910 --> 00:42:50.770
if I had the
hypothesis in my head,

00:42:50.770 --> 00:42:52.930
maybe it somehow got
from my head to the data.

00:42:52.930 --> 00:42:54.580
But when the data
were already there

00:42:54.580 --> 00:42:55.940
and then you went
back and analyzed them

00:42:55.940 --> 00:42:57.481
and the effect was
hiding in the data

00:42:57.481 --> 00:42:59.350
that you'd had on
your server, there's

00:42:59.350 --> 00:43:01.980
something way more real
and magical about that.

00:43:01.980 --> 00:43:04.640
So anyway, because it was
there in all of our old data,

00:43:04.640 --> 00:43:05.930
I just believed it was true.

00:43:05.930 --> 00:43:10.120
The other thing to notice about
this is, to get an MVPA signal,

00:43:10.120 --> 00:43:12.340
we didn't change
anything about the fMRI.

00:43:12.340 --> 00:43:14.470
We didn't change
the resolution--

00:43:14.470 --> 00:43:16.572
the temporal resolution,
the spatial resolution.

00:43:16.572 --> 00:43:18.280
You can know that for
sure, because these

00:43:18.280 --> 00:43:20.710
are our old data that we had
before we started doing MVPA.

00:43:20.710 --> 00:43:23.530
MVPA is not a technique
for collecting better data.

00:43:23.530 --> 00:43:26.230
It's a technique for
getting more information out

00:43:26.230 --> 00:43:27.746
of the same data.

00:43:27.746 --> 00:43:28.870
It's an analysis technique.

00:43:28.870 --> 00:43:30.286
It's a way of
thinking about data,

00:43:30.286 --> 00:43:31.540
not a way of getting data.

00:43:31.540 --> 00:43:38.290
OK, so what this says is
that however similar two

00:43:38.290 --> 00:43:41.350
unrelated stories are about a
case in which somebody kills

00:43:41.350 --> 00:43:44.310
somebody, they are
more similar if they

00:43:44.310 --> 00:43:47.740
are both cases of knowing
murder or both cases

00:43:47.740 --> 00:43:50.910
of unknowing murder than
if you cross that feature.

00:43:50.910 --> 00:43:53.650
So just making that
future match makes

00:43:53.650 --> 00:43:55.720
the pattern of neural
response in the right TPJ

00:43:55.720 --> 00:43:58.870
more similar, suggesting
that which part of the right

00:43:58.870 --> 00:44:02.350
TPJ is more or less active
contains information

00:44:02.350 --> 00:44:03.852
about whether or
not the person who

00:44:03.852 --> 00:44:06.310
committed the murder knew what
they were doing at the time.

00:44:06.310 --> 00:44:07.930
This is specific
to the right TPJ.

00:44:07.930 --> 00:44:09.730
So these are a bunch
of the other brain

00:44:09.730 --> 00:44:13.310
regions involved in theory
of mind and social cognition.

00:44:13.310 --> 00:44:14.980
And none of them
contain any information

00:44:14.980 --> 00:44:16.130
about this dimension at all.

00:44:16.130 --> 00:44:18.171
So this dimension is
represented in the right TPJ

00:44:18.171 --> 00:44:20.057
and not represented
anywhere else.

00:44:20.057 --> 00:44:22.390
There's another thing that
makes these data interesting.

00:44:25.295 --> 00:44:26.670
People are reading
these stories,

00:44:26.670 --> 00:44:28.510
and they're making
moral judgments.

00:44:28.510 --> 00:44:33.130
And moral judgments of these
stories vary across people.

00:44:33.130 --> 00:44:37.420
So some people tend to go more
with what the person thought,

00:44:37.420 --> 00:44:38.920
whereas other people
tend to go more

00:44:38.920 --> 00:44:41.840
with what the person caused.

00:44:41.840 --> 00:44:45.100
It's not extreme
individual variability.

00:44:45.100 --> 00:44:47.350
Everybody agrees that it's
worse to knowingly murder

00:44:47.350 --> 00:44:49.570
than to unknowingly murder.

00:44:49.570 --> 00:44:52.150
But there is variability
in how much worse.

00:44:52.150 --> 00:44:55.540
Some people think that basically
what you thought you were doing

00:44:55.540 --> 00:44:57.460
is all that matters
in these stories,

00:44:57.460 --> 00:44:59.879
whereas other people think
both of those things matter.

00:44:59.879 --> 00:45:02.170
So it matters to some degree
that you caused the murder

00:45:02.170 --> 00:45:03.836
and to some degree
that you didn't know.

00:45:03.836 --> 00:45:05.806
So there's individual
variability.

00:45:05.806 --> 00:45:07.180
And one thing that
we can look at

00:45:07.180 --> 00:45:10.481
is, how does the individual
variability in the behavior

00:45:10.481 --> 00:45:11.980
relate to the
individual variability

00:45:11.980 --> 00:45:13.760
in the representation.

00:45:13.760 --> 00:45:17.290
So what this looks like is,
on the x-axis I measure--

00:45:17.290 --> 00:45:19.890
for you, how much
worse are intentional

00:45:19.890 --> 00:45:21.190
than accidental harms.

00:45:21.190 --> 00:45:23.110
How much worse is
it when you knew

00:45:23.110 --> 00:45:26.060
you were going to cause harm
than when you didn't know?

00:45:26.060 --> 00:45:28.264
So that's always going
to be a positive number.

00:45:28.264 --> 00:45:29.430
Everybody thinks it's worse.

00:45:29.430 --> 00:45:32.200
But for some people,
it's a lot worse

00:45:32.200 --> 00:45:34.870
than it is for other people.

00:45:34.870 --> 00:45:39.250
And then relate that to,
while you were reading

00:45:39.250 --> 00:45:42.280
that story, how different were
the patterns in your brain

00:45:42.280 --> 00:45:43.990
when you were reading
about knowing harm

00:45:43.990 --> 00:45:45.250
compared to unknowing harm.

00:45:45.250 --> 00:45:46.125
Does that make sense?

00:45:48.770 --> 00:45:50.300
They're pretty correlated.

00:45:50.300 --> 00:45:53.360
So the more that you
represented knowing

00:45:53.360 --> 00:45:55.450
harm as different
from unknowing harm

00:45:55.450 --> 00:45:57.830
in your right TPJ,
the more that you

00:45:57.830 --> 00:46:00.487
judged them as different when
we asked you for moral judgment.

00:46:00.487 --> 00:46:02.320
And the pattern difference
in your right TPJ

00:46:02.320 --> 00:46:05.026
accounts for 35% of the variance
in your moral judgment, which

00:46:05.026 --> 00:46:06.400
is pretty amazing,
because that's

00:46:06.400 --> 00:46:08.030
a pretty noisy measurement.

00:46:08.030 --> 00:46:08.950
Actually it's both.

00:46:08.950 --> 00:46:10.390
It's a pretty noisy
measurement of your brain

00:46:10.390 --> 00:46:12.500
and a pretty noisy
measurement of your behavior.

00:46:12.500 --> 00:46:15.086
So it's quite amazing that
those are that correlated.

00:46:17.260 --> 00:46:20.050
So that's what's
cool about the data.

00:46:20.050 --> 00:46:23.244
But we'll get to the method.

00:46:23.244 --> 00:46:25.660
So Haxby style correlations--
these are called Haxby style

00:46:25.660 --> 00:46:28.000
because they were the first
form of MVPA introduced,

00:46:28.000 --> 00:46:30.766
and they were introduced
by Jim Haxby in 2001,

00:46:30.766 --> 00:46:32.621
so actually a long time ago.

00:46:32.621 --> 00:46:34.120
It took a long time
for other people

00:46:34.120 --> 00:46:35.980
to recognize what a
cool technique this was.

00:46:35.980 --> 00:46:39.040
But he had this idea
a very long time ago.

00:46:39.040 --> 00:46:42.475
And the idea is, take
a region you care about

00:46:42.475 --> 00:46:45.100
and ask this basic question.

00:46:45.100 --> 00:46:47.920
For some future that I
wonder if it's represented,

00:46:47.920 --> 00:46:51.580
is the correlation across
neural responses more similar

00:46:51.580 --> 00:46:53.890
when the stimuli share
that feature than when

00:46:53.890 --> 00:46:55.550
they don't share that feature?

00:46:55.550 --> 00:46:57.850
So that gives you a
pretty robust measurement,

00:46:57.850 --> 00:47:00.010
because you're using all
the voxels in the region

00:47:00.010 --> 00:47:02.151
to get one number
out-- the correlation.

00:47:02.151 --> 00:47:04.150
And you're doing it over
partitions of the data,

00:47:04.150 --> 00:47:06.730
often halves of the
data, so many trials

00:47:06.730 --> 00:47:09.050
are going into both
the train and test.

00:47:09.050 --> 00:47:11.980
So in this case we're using
halves of the data, even halves

00:47:11.980 --> 00:47:13.080
and odd halves.

00:47:13.080 --> 00:47:15.520
And so each of the
things we're correlating

00:47:15.520 --> 00:47:18.310
is a relatively less
noisy neural measure

00:47:18.310 --> 00:47:21.250
because it has many
trials averaged into it.

00:47:21.250 --> 00:47:23.410
So it's robust and simple.

00:47:23.410 --> 00:47:26.080
In this case, it can be
sensitive to pretty minimal

00:47:26.080 --> 00:47:27.100
stimulus variations.

00:47:27.100 --> 00:47:29.980
As I showed you, this is a
two- to four-word variation

00:47:29.980 --> 00:47:32.710
on an 80-word story.

00:47:32.710 --> 00:47:36.090
So it's sensitive to small
distinctions in the stimuli.

00:47:36.090 --> 00:47:38.110
Here we showed that
it generalizes.

00:47:38.110 --> 00:47:40.540
So we used totally
independent stories

00:47:40.540 --> 00:47:42.160
in the train and test set.

00:47:42.160 --> 00:47:44.710
And so we're always generalizing
from one set of examples

00:47:44.710 --> 00:47:47.710
to a totally different
set of examples.

00:47:47.710 --> 00:47:51.340
It gave us a measure that was
stable within a participant

00:47:51.340 --> 00:47:53.710
in the sense that the
measure in each individual

00:47:53.710 --> 00:47:55.670
related to that
individual's behavior.

00:47:55.670 --> 00:47:57.280
So it's characterizing
individuals

00:47:57.280 --> 00:47:59.080
in a relatively stable way.

00:47:59.080 --> 00:48:01.259
And we could show that it
differs across regions.

00:48:01.259 --> 00:48:03.550
So we could show that this
was present in the right TPJ

00:48:03.550 --> 00:48:05.470
but not present
in other regions.

00:48:05.470 --> 00:48:08.080
And that's a bunch of stuff
you would want to know.

00:48:08.080 --> 00:48:10.640
That's a whole bunch
of extra information

00:48:10.640 --> 00:48:13.790
than we ever were
able to get before.

00:48:13.790 --> 00:48:16.990
And I'll give you one
more example of the way

00:48:16.990 --> 00:48:19.580
that Haxby correlations
can be used.

00:48:19.580 --> 00:48:22.510
So in this case I showed you,
we hypothesized one dimension.

00:48:22.510 --> 00:48:24.820
And we tried to
decode that dimension.

00:48:24.820 --> 00:48:26.980
Obviously, you don't
only have to do one.

00:48:26.980 --> 00:48:29.680
And so another way to do this
is to build stimulus sets that,

00:48:29.680 --> 00:48:31.690
for example, have two
orthogonal dimensions

00:48:31.690 --> 00:48:34.210
and ask about both of them.

00:48:34.210 --> 00:48:36.160
SO here's an
experiment in which we

00:48:36.160 --> 00:48:38.950
asked about decoding two
orthogonal differences

00:48:38.950 --> 00:48:41.546
within the same set of stimuli.

00:48:41.546 --> 00:48:43.420
So again, you're reading
stories about people

00:48:43.420 --> 00:48:46.150
who are having experiences.

00:48:46.150 --> 00:48:49.225
And some sets of
these stories vary.

00:48:52.960 --> 00:48:54.280
So here's a bunch of stories.

00:48:54.280 --> 00:48:57.065
Leslie has just been in a
big, important interview.

00:48:57.065 --> 00:48:58.690
And he sees himself
in a mirror, and he

00:48:58.690 --> 00:49:01.390
sees that his shirt has a big
coffee stain down the front.

00:49:01.390 --> 00:49:02.320
And another one is--

00:49:02.320 --> 00:49:05.500
Eric gets to a restaurant to
meet his fiance's parents,

00:49:05.500 --> 00:49:07.459
and he sees them and
they're looking happy

00:49:07.459 --> 00:49:09.250
So that's two completely
different stories.

00:49:09.250 --> 00:49:10.510
And then the third story--

00:49:10.510 --> 00:49:12.280
Abigail is painting
her dorm room,

00:49:12.280 --> 00:49:14.590
and she hears somebody's
footsteps down the hallway.

00:49:14.590 --> 00:49:17.572
And the footsteps sound like
her beloved boyfriend's.

00:49:17.572 --> 00:49:19.030
So these stories
are all different.

00:49:19.030 --> 00:49:22.426
Again, they're all But
the first two stories

00:49:22.426 --> 00:49:23.800
I read you share
a feature, which

00:49:23.800 --> 00:49:26.476
is that somebody in the
story is seeing something.

00:49:26.476 --> 00:49:27.850
And they don't
share that feature

00:49:27.850 --> 00:49:30.016
with the third story, in
which somebody in the story

00:49:30.016 --> 00:49:31.320
is hearing something.

00:49:31.320 --> 00:49:33.307
Does that makes sense?

00:49:33.307 --> 00:49:34.390
Compared to, for example--

00:49:34.390 --> 00:49:36.710
Quentin hears a phone
message, and the message

00:49:36.710 --> 00:49:38.320
says she has bad
news to tell him.

00:49:38.320 --> 00:49:40.194
That's another story
that shares this feature

00:49:40.194 --> 00:49:43.390
that somebody in the story
is hearing something.

00:49:43.390 --> 00:49:47.020
And so we can use this
set of stories to ask,

00:49:47.020 --> 00:49:49.450
is the neural
response to stories

00:49:49.450 --> 00:49:51.940
in which somebody
is seeing something

00:49:51.940 --> 00:49:53.287
more similar within that set?

00:49:53.287 --> 00:49:55.120
So one set of stories
about seeing something

00:49:55.120 --> 00:49:57.400
is compared to another set of
stories about seeing something.

00:49:57.400 --> 00:49:59.310
Are those stories more
similar to one another

00:49:59.310 --> 00:50:00.610
than when you
cross that feature?

00:50:00.610 --> 00:50:02.776
So you ask one set of stories
about seeing something

00:50:02.776 --> 00:50:05.090
compared to one set of stories
about hearing something.

00:50:05.090 --> 00:50:08.440
And so in the right
TPJ, what we found

00:50:08.440 --> 00:50:11.740
is that stories about seeing are
more similar to other stories

00:50:11.740 --> 00:50:12.304
about seeing.

00:50:12.304 --> 00:50:13.720
And stories about
hearing are more

00:50:13.720 --> 00:50:15.370
similar to other
stories about hearing

00:50:15.370 --> 00:50:17.594
than when you
cross that feature.

00:50:17.594 --> 00:50:19.510
But, as you may have
noticed, the stimulus set

00:50:19.510 --> 00:50:21.280
had another distinction
in it, which

00:50:21.280 --> 00:50:25.570
is whether the thing is good
or bad that's happening to you.

00:50:25.570 --> 00:50:26.950
So finding out
after an interview

00:50:26.950 --> 00:50:28.450
that you have coffee
down your shirt

00:50:28.450 --> 00:50:30.160
or hearing a message that
says there's bad news,

00:50:30.160 --> 00:50:31.360
those are both bad things.

00:50:31.360 --> 00:50:34.469
Whereas seeing your fiance
looking happy or hearing

00:50:34.469 --> 00:50:36.760
that your beloved boyfriend
is coming down the hallway,

00:50:36.760 --> 00:50:38.320
those are both good things.

00:50:38.320 --> 00:50:40.450
And so we could ask
in the same dataset,

00:50:40.450 --> 00:50:43.210
what about stories that share
this feature of valence.

00:50:43.210 --> 00:50:45.610
The pairs of stories that
are matched on valence,

00:50:45.610 --> 00:50:47.980
do they have a more
similar neural signature

00:50:47.980 --> 00:50:51.900
than the pairs of stories
that are crossed on valence?

00:50:51.900 --> 00:50:53.870
And in the right
TPJ they're not.

00:50:53.870 --> 00:50:56.110
We've actually found this
a whole bunch of times.

00:50:56.110 --> 00:50:59.020
The right TPJ doesn't
care about valence.

00:50:59.020 --> 00:51:01.560
Other regions do-- don't
worry-- we do represent valence.

00:51:01.560 --> 00:51:05.080
But the right TPJ doesn't
represent valence.

00:51:05.080 --> 00:51:08.110
So that's another way that
you can use this method--

00:51:08.110 --> 00:51:10.090
hypothesize two or three
orthogonal dimensions

00:51:10.090 --> 00:51:11.744
within the same stimulus set.

00:51:11.744 --> 00:51:13.660
And then we can get, for
example, interactions

00:51:13.660 --> 00:51:16.180
between these to say, OK,
the right TPJ does represent

00:51:16.180 --> 00:51:18.710
some dimensions, doesn't
represent other dimensions--

00:51:18.710 --> 00:51:21.730
in principle.

00:51:21.730 --> 00:51:24.010
So you can test
potentially multiple

00:51:24.010 --> 00:51:25.660
orthogonal distinctions.

00:51:28.870 --> 00:51:30.370
There's a whole
bunch of limitations

00:51:30.370 --> 00:51:33.310
of Haxby style correlations.

00:51:33.310 --> 00:51:37.690
One of them is that all
the tests are binary.

00:51:37.690 --> 00:51:40.210
The answer you get
for anything you test

00:51:40.210 --> 00:51:42.310
is that there is or
is not information

00:51:42.310 --> 00:51:44.640
about that distinction.

00:51:44.640 --> 00:51:46.380
There's no continuous
measure here.

00:51:46.380 --> 00:51:51.580
It's just that two things are
different from one another

00:51:51.580 --> 00:51:55.150
or they are not different
from one another.

00:51:55.150 --> 00:52:01.069
And so once people started
thinking about this method,

00:52:01.069 --> 00:52:02.610
it became clear that
this is actually

00:52:02.610 --> 00:52:05.430
just a special case of a much
more general way of thinking

00:52:05.430 --> 00:52:07.310
about fMRI data.

00:52:07.310 --> 00:52:10.590
This particular method,
using spatial correlations,

00:52:10.590 --> 00:52:13.790
is very stable and robust.

00:52:13.790 --> 00:52:17.840
But it's a special case of
a much more general set.