The following content is provided under a Creative
Commons license.
Your support will help MIT OpenCourseWare continue to
offer high quality educational resources for free.
To make a donation or view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu.
PROFESSOR: Good afternoon.
Today we're going to go beyond the way we've
spoken about before.
We've talked about how psychologists measure the
mind, how neuroscientists interrogate the brain.
And so now, we'll talk about the first human faculty that
we'll consider in a series of faculties
through the course, Vision--
How We See.
And for today's lecture, I got help from Melissa Troyer, one
of your teaching assistants.
So, one of the things we can think about is what are the
purposes of vision?
What do we use sight for?
And we'll talk about that.
Problems that the visual system has to overcome.
What are the obstacles for vision to work accurately and
efficiently?
And how does the brain organization of vision serve
the human capacity to see?
So human perceptual abilities are amazing, as are perceptual
abilities of many species in different areas.
But humans can detect a candle 30 miles away on
a dark, clear night.
They can detect cochlear displacements
that's in your ear--
equal to the width of a hydrogen atom.
You can taste 1 teaspoon of sugar, even when it's mixed in
two gallons of water.
You can smell a drop of perfume diffused into the
space of a three-bedroom apartment.
So the human capacity to perceive the world in a highly
sensitive way is remarkable.
We're going to focus on vision specifically today.
And think, what do we use vision for?
Well, how does it serve us in dealing with the world?
And researchers have thought about two major reasons that
we see, two purposes of vision.
One of them is object recognition.
Recognizing things in the world.
People we know, words we can read,
chairs, tables, animals--
all kinds of things that we see out there.
And we need to know who they are or what they are in order
to operate in the world.
And the second one is a little bit different, navigation--
getting around in the world.
When you're running, when you're jumping, when you're
reaching to grab something, you're not usually much
interested in what something is, as where you're moving,
where you need to move to avoid something
being thrown at you.
So two big purposes of vision--
what things are and where they are, so you can navigate
around quickly in the world.
And when we think about purposes of vision, for what
things are or where they are, psychologists and people
working in computer vision have recognized a series of
problems that our visual system has to solve.
We have to link unique images.
Because when we see something, it'll almost always look
different from one perceptual moment to the next.
We can see the same person, for example, but under
different conditions of illumination, darkness or
lightness or shadows, the angle we see them at, the
distance we see them at, the expression on their face-- a
smile or frown--
shadows that might be being cast, occlusion, a thing
blocking part of their face.
And yet, we can recognize a person very well under all
these circumstances.
So what arrives in our eye is different from
all of these views.
But what we interpret them is all the same.
That's a person I know.
The same thing with letters.
We can see letters in all kinds of fonts and all kinds
of handwriting, and yet we can interpret what they are in a
uniform way.
Or body.
We can see bodies of all kinds of different positions.
But our visual system lets us know right
away that it's a person.
And so psychologists have tried to formalize these kinds
of problems to ones of equivalents.
So something is the same thing at different views.
Generalization--
something has a very different shape.
Or impoverished input, where there's poor lighting or
something's complicated.
And you still figure out what it is.
For getting around in the world or where things are in
the world, you're moving through space
and tracking things.
And there's many sources of movement.
Your eyes are constantly moving.
Your head is moving.
Your body is moving.
And the world is moving.
And you have to sort all those out.
Run quickly.
Jump [INAUDIBLE]
world spatially.
And so here's some examples.
Here's a shape constancy problem.
Here's two different versions of a cat.
It's trivial for you to know that even though these things
have different shapes, they represent the same thing.
But that's an achievement of your visual system, to easily
make that equivalence across two very different shapes.
And we know, from computer vision, how hard these
problems are.
No computer vision system sees with anything like the
generalized ability of a very young child.
So our visual systems are so brilliant that psychologists
and vision scientists really interrogate our visual system
to figure out, how do we do it.
If people didn't do this, we'd think it would be impossible
to accomplish at all.
Here's another example that's easily solved by people-- not
so easily solved by machines.
If you see something like a car at different angles, we
know that's the same thing, pretty easily, even though,
again, the information that arrives to our eyes is
radically different.
And the same thing can be said of something like all these
different versions of the letter C.
Radically different shapes.
But we can readily interpret almost all them-- this one
won't be too tough--
as the letter C.
So our visual system does this brilliantly.
And also you get very impoverished views.
A key might be covered by something, clothes on a hook.
There might be a book over here.
There might be a fire hydrant over here or a person.
And yet, even though you get only partial information, it's
relatively trivial to discover, for you, what it is
you're looking at.
Or you can see things like you've never seen before.
The pink elephant, this odd picture, this striped ...
you've never seen them before but it's not really ...
So again, your visual system is just so brilliant at
matching what it sees with what you know to figure out
what's in front of you.
So today we're going to talk about how our minds and brains
allow us to see the world vision.
And we'll talk about it in three parts; how philosophers
and psychologists have thought about the idea of seeing; how
when information first enters your eye-- your retina--
which are set of layers in the back of the eye; how that
retina's already organized to help you see the world; and
how that information is then transported into the cortex,
where higher-level processing lets you know where you are
and what you're looking at.
So the ideas about how we learn to see the world have
their roots in philosophy.
For example, John Locke is associated with the idea of
the blank slate or tabula rosa.
He said, "Let us suppose the mind to be a white paper void
of any characters, without any ideas.
How comes it to be furnished?
Whence comes it by that vast store, which the busy and
boundless fancy of man has painted on it with an almost
endless variety?" Where do we get the information for how
the world works when we look at it?
And his answer to that was experience.
That we notice things around us, and experience drives our
understanding of the world.
And that position that there's a blank slate and we learn
objectively from the world around us has been called the
Objectivist view.
And maybe, that we see the world that we build up through
experience, knowledge about how to accurately represent
what's around us.
The alternative you has emphasized how much our mind
organizes what we see, instead of what's out there itself.
And so Gestalt psychologists or Gestalt views on perception
say, it's not what's out there in the world, but the way our
mind organizes and believes what's out there.
And these two things can come together.
So for example, when you think about how you listen to a
piano, music comes from piano the same way a
pianist strikes chords.
There's things out there in the world that you see.
The piano has certain limitations.
It can sound like somethings but not others.
It can't be a clarinet.
There's constraints in the world on what's possible to
see what's out there.
And our percepts are evoked by nature in this way.
But they are personal and not a copy of nature.
In the words, we each, individually, figure out
what's going on there by our own experiences and
interpretation.
And they're exactly that-- an interpretation, not a simple
mirroring or copy of nature.
So I'm going to try and convince you of something that
might not be obvious to people when they first think about
it, that vision is an interpretation of the
world around us.
It's not simply a video camera or mirror, it's an active
calculation of what's out there.
And we see, through inference, what we believe that we see.
And that visual illusions--
and you've seen some before, and I'll show you some again--
are not just illusions to trick people and confuse them.
But they're demonstrations of a gap between what's actually
out there in the world and how we interpret it.
Most of the time we don't have illusions.
We don't walk into walls.
We don't misidentify people we know commonly.
Because illusions are rare, because our minds and brains
have evolved to interpret the world that's out there, and
the system works so brilliantly coupled with the
environment that we actually live in, that we
don't think about it.
So the person on the street says, well, what
do I need to see?
I open my eyes.
But that's because our visual system works so brilliantly,
effortlessly, and automatically.
It's such a calculating machine that that's why it
feels so simple.
So here's one, small example.
And I'll show you a number of visual illusions that show us
how we're constantly interpreting even the most
simple thing.
So for example, here's two people.
And we're not worried that this is one of the shortest
men you've ever seen, because we know, through prospective,
this person's standing back there.
Even though, if they were standing right next to
somebody, you'd worry about the size of that person.
So we're constantly interpreting information by
the context in which we see it.
Here's another example of the choice of interpretations.
You can see a vase.
Or you can see two faces here. , Usually it's hard for people
to see them both simultaneously.
They tend to flip back and forth.
But you can choose the
interpretation of what you see.
So let me pick one example where people have studied a
lot, which is the issue of brightness constancy.
So here's the problem for the visual system.
The ambient brightness, the brightness around us, varies
in staggering ratios.
When we're outdoors in the sun-- and even that's not
constant, right?
Sometimes it's a bright day.
Sometimes it's a cloudy day.
Sometimes we're indoors under bright conditions or less
bright conditions.
So huge changes in the ambient illumination.
Or in shadows-- we'll make it even more complicated.
That means in a purely measurement sense, in a purely
physics sense, a piece of coal in sunlight reflects 10 times
as much light as snow in the shade.
So think about that for a moment.
You're not disturbed if you see a piece of
coal inside or outside.
They both look, to you, like black, dark pieces of coal.
And snow looks, pretty much, the same to you if it's inside
or outside.
But if you're visual system worked like a simple,
objective light meter, you would conclude that the piece
of coal in the sunlight is a much brighter thing-- in an
objective sense--
than snow in the shade.
And yet, you never make that conclusion, because you're
constantly, automatically, without effort, adjusting
mentally for the ambient light condition.
So we use brightness all the time to help us figure out is
that a piece of coal or sunlight?
But that brightness perception is constantly adjusted to
interpret brightness in the context we're sitting in.
And perceptual psychologists have worked on how people
calculate this.
So here's an example of a dark letter T. Again, the black T
would be 10 times brighter outdoors than
white paper is indoors.
And yet, you're never bothered by that.
You can read a newspaper or something else indoors or
outdoors with equal these, because people are calculating
the ratio of the ambient lighting--
lower inside, much brighter outside, from the sun--
and using that information to make inferences about the
relative brightness of a stimulus they're looking at.
And there's all kinds of examples or illusions that
show us this.
So for example, here, you detect a white tile in the
shade and a gray tile in brightness.
And that's how you interpret it.
But that's because you're taking already into account--
without even thinking about it for a moment--
the fact that this is in the shadow, and this is in the
bright light.
You're right in your interpretation.
But if we show a constant background, you could see that
those two are actually identical in brightness.
The same thing-- you could take a look at this cube and
this cube as part of these Rubik's cubes.
This looks orange.
This looks brown.
You're adjusting for light, as you make those
interpretations.
And if you make the background constant, you can see those
are identical in color and luminance.
So this shows you that we're constantly making these
interpretations.
Only these tricks of visual illusion make us realize that
our inferences are occasionally wrong.
But they're incredibly right almost all the time.
So here's another example of taking a local
luminance into account.
So typically, people see this as a lighter gray.
And this is a darker gray.
Again, they're using the contrast that surrounds it to
make that interpretation.
Because if we give it a constant background, you can
see that in an objective sense the two are identical.
People use edges as a very powerful source of information
to make these inferences.
And you can see that in one moment.
Because this boundary, here, is used to make judgments
about the relative brightness around an object.
So you just do this.
The boundary disappears, and the two things look equally
gray, one above the other.
Or no less spectacular, but measured example, here's two
different shades of gray.
And if you're in a light meter, objectively looking at
how much light is reflected from this, here that changes.
This looks almost the same as this.
But really, there's just a manipulation at the
boundaries.
Again, here's a light meter.
Here's a boundary.
So these two sides are really identical.
It's just been manipulated by the boundary.
And you can see that, because if you get rid of that
boundary, now you can see that these two
sides are equally bright.
Another example, again, of how we're constantly using local
information about brightness to make
judgments about objects.
Here, you see a picture of a woman.
Here's her forehead.
Here's her darker hair.
But in fact, that's a judgment in interpretation.
Because if we cover up the surrounding information, the
hair is identical to the face.
Here's some other illusions, not all about brightness.
But, again, they're just reminders that what we see is
constantly a calculation and an
interpretation of many factors.
So here's a fun spiral.
You can see it zooming around, going from inner tightness and
unwinding peripherally.
And so we can add a dot to that.
And now, we can let this travel around to the middle.
You can see it's not quite making the middle.
And we can try that again.
Because in fact, in this picture, every line, every
circle is a circle.
And none of them are connected, one to the other.
They just appear that way.
This illusion, that Richard Gregory describes, was
inspired because he saw a wall in a cafe that looked like it
had not been well assembled by the person building it.
As if they had been consuming something from the cafe as
they built the wall.
And bizarrely, having all kinds of lines that are not
lined up, as you'd expect in
well-constructed tiles on a wall.
But in fact, every line, here, is perfectly straight.
And you're interpreting the black and white edges to
confuse you about the lining up across the horizontal.
This is very striking.
Here's Rubik's cubes that, again, now they're adding one
more story into this, which is color.
But we use that often in every day sight, of course, to make
conclusions about things.
And here, you can see the difference between, for
example, this yellow cube and this brown cube.
What did I want to do?
You can see the difference between this brown cube--
sorry, let me get my illusion correct--
and this darker cube, lighter brown and darker brown.
But if we cover those up, those are identical.
And they're gray.
Here's another one that shows you it's not just the color
itself but the spatial location.
So here's a pink circle, near the middle of this display.
And here's a more orange one.
So we're just simply going to expand those out and get it
bigger and bigger.
And as it's moving across your eye--
even though, objectively, it remains pink and orange, or
the things will lead you to you interpret that--
as it gets big enough, the color difference disappears to
your perception.
And sometimes you can get-- and this is a
very complicated one--
illusions--
I don't know if it's working for you--
that these things are actually turning, even though none of
them are moving at all.
OK.
And one last, simple example.
Here's yellow on gray.
Here's light gray on yellow.
But in fact, these X's are identical.
And you can see that when they meet here.
So again, we're constantly interpreting color, shape,
brightness, by things that are surrounding it.
And everything is being the product of an interpretation,
not what's out there in an objective sense of self.
And so, if we think about this Objectivist view that that
can't be right, that the limit of this objective view would
be that everybody sees everything differently all the
time, that would be a confusing world of
hallucinations.
We don't live there either, mostly.
So we think it's the because there's a relationship between
these, or a synthesis.
We perceive only within limits of our nervous system.
There's a way that we see the world.
But the way that our nervous system computes and makes
inferences about the world reflects many properties of
the world that are efficient and accurate.
So how do we get information into our brains to see?
So, if you were to design a system that was hopelessly
failed, the first thing you would do is think the visual
system is going to fail.
The first thing it does is it takes objects in the world and
it flips them upside down-- and how they're perceived in
the eye, the front of the eye, the back of the eye.
Here's the retina.
And the first place in your brain that sees something is
in the back of your eye.
You'd think you would put it in the front of your eye.
And information bounces back into the
deepest part of the eye.
And then it comes out and leads through the optic nerve.
You also know, from prior classes, that the world is
organized by left and right visual fields, instead of the
way we intuitively think about the world.
Let's take a closer look at the retina.
These layers of cells in your eye that begin vision.
So light comes in and bounces through the back.
And here are the rods and cones where vision begins.
The cones are primarily near the fovea.
The rods are primarily peripheral.
And I'm reminding you, again, as the optic nerve leaves, it
goes through the optic chiasm.
The fibers get sorted out by whether they're having
information from the left or right side of the world, and
then enter the cortex that way.
Within the retina, within your eye, here is a graphic of the
different kinds of cells.
Here's what the cells actually look like.
And here's a blow up of these remarkable cones, these larger
entities and the rods that you can see are very small, but
many of them.
And the big difference between rods and cones is that the
rods are receptive to information at only one
wavelength.
They're effectively color blind.
We'll come back to that.
But that's a good wavelength to pick up information, for
example, in low light.
The cones are selective for blue or red or green.
That's the stuff of seeing color.
But it doesn't operate too well, unless you have a pretty
well illuminated situation.
And the rods and cones have a striking physical
organization.
So here's the distribution of cones for color.
Pretty much, all of them in the fovea in the middle part
of the retina.
And then the rods are spread out peripherally.
So there's this big difference in where they're located.
And another big difference is that the cones, the ones that
respond to color, and that are in the middle of the fovea,
have an almost one-to-one relationship between the
receptors for color information and the neurons
that leave the eye with information that
will go into the brain.
The rods have a many-to-one relationship.
Many rods are contributing for a combined calculation of some
kind to a single neuron that leaves the brain.
So you can already see, at the very first moment of reception
from the rods and from the cons, very different kinds of
processes are being begun, as you begin to see.
Two ideas have been very helpful for understanding how
vision is organized in the brain--
our receptive fields and retinotopy.
Receptive fields are, simply, an area of external space.
It's a physical spot in space in which a
stimulus activates a neuron.
So neurons, early in vision, will respond to specific
locations in space when you're looking straightforward.
One neuron might cover this area.
Another neuron might cover that area and so forth.
Retinotopy refers to topographic or spatial maps of
visual space across a restricted
region of the brain.
And so, retinotopy keeps these receptive fields lined up.
So if the neuron is seeing this spot, this retinotopic
map, and so the next spot stays next to it.
And the next spot stays next to it.
So you can reconstruct what you see, based on a lot of
very local glances at the environment.
And then it travels from the eye, through the optic nerve,
into something called the lateral geniculate nucleus.
The nucleus is simply a collection of cells.
This is part of the thalamus.
And from there, the fibers will travel into the cortex to
begin conscious visual perception.
And you have one on the left and one in the right of the
lateral geniculate nucleon.
And if you look at this blown up--
this is a sample from a monkey, but the human one
looks very similar.
You can see 1, 2, 3, 4, 5, 6 layers of cells in each
lateral geniculate nucleus.
The first two look a little bit different.
If you look carefully, they're comprised of larger cells.
The other four are comprised of smaller cells.
So people describe the cell layer, made up of larger
cells, as magnocellular--
big cells.
And the other four layers as parvocellular, or small cells.
The mangocellular layers tend to have large cells.
A lot of their information comes from the rods, the black
and white elements, good for seeing in low vision.
They have large receptive fields.
They cover relatively large patches of the
world for each neuron.
Three times larger than the parvocellular ones, the
neurons fire rapidly, transiently, are color blind.
They're good in low contrast, low lighting.
The parvocellular ones are smaller.
They get a lot of their input from the cones.
They have smaller receptive fields.
They have slow, sustained responses.
They don't turn on and off very quickly.
They're wavelength sensitive.
So they can respond to color.
They operate best when things are very bright.
They're unique to primates, and there are 10
times more of them.
But again, even before you get to the cortex, there's this
huge organization between two modes or pathways of
perception, so you can construct the
world as you see it.
And then we've gone over before that as you
leave the nuclei --
the lateral geniculate nuclei-- then the next fiber
pathway takes you into the primary visual cortex in the
back of your brain.
Now, vision is so important for primates and people, that
when people estimate how much of the neocortex is devoted to
different modalities, a common estimate is that about half of
the brain is primarily devoted to vision--
11% to touch, 3% to audition.
These are ballpark estimates.
But it shows you how visual we are with our fellow primates.
There's many specialized areas within the
human visual cortex.
They estimated some years ago it was 32 distinct areas, each
performing a different tasks, lots of specialization.
And another fascinating aspect of vision is proliferation as
we go up into higher stations of visual processing.
So in one lateral geniculate nucleus in your brain-- and
you have two of them, one on the left and one on the
right-- you have about a million neurons.
They will send information that will be interpreted by
about 250 million neurons in your visual cortex.
Those will communicate about 400 million neurons in the
next level of processing.
And finally, there'll be something like 1.3 billion
visual cortical neurons in the brain,
overall, in a gross estimate.
It's as if to discover what you're seeing, you're having
larger and larger populations of neurons unpacking the
mystery of what initially came into the brain very
impoverished information, that's expanded to begin to
understand what you need to perceive a face or a word or a
physical movement.
So a single lateral geniculate neuron, it could be estimated,
will take 600 cortical neurons to interpret what this lateral
geniculate neuron has been exposed to.
Here's this area in the visual cortex, where the information
arrives first into your brain and primary visual cortex.
And when we think about cortex in perception, in several
modalities, one of the striking things is how it's
organized to achieve certain goals.
So here, we're going to switch for a moment to the parts of
your brain that move your body--
motor cortex or that have your sense of touch.
And when people stimulate these and look at what they're
related to.
For example, in patients undergoing treatment for
epilepsy or an animal study-- support this where you can do
much more expensive experimentation--
a very striking thing occurs, which is the number of neurons
devoted to different parts of your body are radically
different than your actual body size.
So this funny looking, so-called homunculus
represents how many neurons, for example, in your motor
cortex are devoted to different parts of your body.
So it turns out that we do incredible things with our
hands, so we devote, it seems like, a tremendous amount of
this area to the hand.
That's what that looks so large.
There's a reason we do handshaking and handwriting
and typing with our hands and not with our hips.
Because although our hips are physically larger than our
hand, we devote an incredibly small number of
neurons to do that.
So it's a trick of the brain, given a certain limited size,
to blow up its representation of what needs to be done
brilliantly.
And to reduce its representation of parts of the
body that it doesn't have to do too much with.
So what's blown up?
Your hand?
Things to do with your face, so that you can speak?
Your tongue, because that has to accomplish a lot, in terms
of speech, in terms of eating.
And things like your toes and ankle are gypped of
representation, because you don't do that much that's
brilliance.
There's a very clever strategy, by the brain, to
devote the amount of neural resources to make that part of
your body accomplish either very complicated things or
very simple things.
And if people on the outside looked like this--
this is what they would look like.
Giant hands, giant head, and a very shriveled up trunk.
So that's not what we look like, but that's the way our
brain represents us, our bodies, so that we could have
fantastically complicated control of our hands and
things having to do with speech around our mouth.
It's different for different modalities.
For hearing, things are organized
by frequency, tonotopy.
For vision, things are organized in two ways.
Spatial relations are maintained.
They have to be, so you know, as different neurons saw
different receptive fields, how's that all kept up?
Here's a kind of a rough experiment, where a monkey
saw-- before it was, as they politely say, sacrificed--
a display like this.
This is the brain of the monkey, flattened out.
And you can see it has this representation, like this.
Now you could say, well, that's how we see.
We line everything up.
But this is not the part of our brain that we associate
with conscious perception at all.
But if you didn't keep the spatial information aligned
correctly, you could never interpret things, in the end,
correctly that go together, like different parts of a hand
or different letters within a word.
In humans, it's also in the back of the brain.
It also keeps spatial information.
But what do humans do, and other primates?
They greatly expand the central part of vision.
So here's the central part of vision, the so-called fovea,
which is a small part of the visual display.
Here is everything else that's in peripheral vision.
So this is dark green, dark purple.
But when it comes to the representation in the visual
cortex, this small area of what we see
becomes pretty large.
It's way overrepresented.
So just as the motor cortex over represents the hand, the
visual cortex over represents the cone
foveal part of the brain.
So they can do much more examination, much more
computation on that small foveated area.
So that's why it matters, tremendously, where people put
their eyes.
Where they put their eyes is where neuromachinery is
dedicated to do its most brilliant analysis.
And peripheral regions, which are large, in vision, get
relatively small representation.
All we do, in the periphery, is notice if something is
whizzing at our head.
And for that, we don't need to be that sophisticated.
Now, what do neurons communicate in
primary visual cortex?
Well, this is the, sort of, seminal Nobel Prize-winning
work from Hubel and Wiesel, at Harvard, who discovered that
what neurons respond to in primary visual cortex is the
orientation of a piece of something like this.
So here's a neuron that loves a little bit of a stimulus.
You notice that disorientation?
Here's each of these lines as of the neuron firing.
It generalizes if a bar is close in orientation, it will
still like it, somewhat.
And if the bar moves away from its preferred orientation, it
won't respond and all.
So these neurons are coding something about local
orientation of little lines.
Those little lines will be assembled later on in the
brain to represent a letter, a word, a book, a face, a
chair, and so on.
But these neurons simply know they're seeing a line of this
angle or that angle that will be assembled later on for an
entire conscious percept.
And then that information will go from primary visual cortex,
in two pathways, that have two quite different properties,
but turn out to be the super-information processing
pathways of your brain and my brain.
So where do we first learn about this fundamental
organization of how we see into two giant highways of
information processing?
Well, the first studies where we should study's in monkeys.
And I'll show you work in humans as well.
And the major discovery was this, that in our brains,
there's a so-called where pathway that goes up in the
brain into the parietal cortex and a what pathway that goes
into temporal lobe or lower cortex.
And if you make lesions in these different regions, one
form of vision is spared and one is impaired
in selective ways.
So how did they discover this?
So again, here's the big picture.
Early visual processing information a what pathway to
know what objects are a face, a word, a chair.
Or where things are, that you might run to,
grab, or jump over.
So here was the experiment.
It was pretty simple.
Here were two food wells.
And monkeys that were eager to get food would get rewarded,
if they would pick the correct food well.
They didn't see where the food was that was hidden, but they
would get to pick one or the other.
And in one task they would have to pick which food well
is closer to the stimulus.
That's a where task.
Where is this located?
That's the piece of information that will tell me
where to get my food.
And you can see, here's the performance of the animals,
their errors.
So it's good to be low.
They are eager to get their food.
But if the animal had a surgical lesion in the
parietal cortex, the where pathway, they were very poor
at performing this task.
As if they couldn't appreciate the spatial relation between
where the cylinder was located and the food well.
On the other hand, monkeys who had lesions made in the
temporal cortex were poor when they had to make this task.
They would see two different objects.
And they were taught that one always meant that's
where the food is.
Let's pretend it's the cylinder.
So in order to know the correct answer, they would
have to say, OK, is it next to this object or this object?
That's a what discrimination.
Is it a cylinder or a cube?
And now, the temporal lobe lesions were the ones that
affected performance.
So it's as if one part of the brain tells you where things
are, in vision--
parietal lobe.
And one tells you what they are--
the temporal lobe.
So that's lesion studies in monkeys.
But there's very interesting things about when we look at
what cells communicate in these two pathways that makes
sense in what we understand to be the goals of these systems.
So for example, in the where pathway, in the parietal
cortex, many neurons pick up information from the fovea,
the central part of vision.
But the majority are sensitive to stimuli in the periphery.
So if you imagine, a good thing to know if you're
running, is in the periphery.
Are there things coming at your head, things to avoid?
You might grab something here, if you need to grab it there.
You would want to have a lot of peripheral information for
spatial things around you.
And in fact, if a monkey was looking at this spot, which is
away, entirely, from the stimulus, it would respond to
both a large and small stimulus in the periphery
quite strongly.
So these neurons are responding to a pretty broad
range of space where things could happen.
Neurons in the what pathway have almost their entire
responsiveness in the fovea.
Because you know what something is, if you're
reading a word and looking at it.
If you're looking at a face, they're figuring out
who the person is.
So it's only responding, these kinds of neurons, to
information in the center.
But it has some very interesting properties, even
at the level of singular neurons.
So here's a depiction of an
experiment, now, from a monkey.
You're looking at a single neuron or
small group of neurons.
And you present something like a hand.
And you can see that those neurons really like the hand.
Now, don't forget, for a primary visual cortex, it just
sees lines.
It doesn't think about the entire objects.
But by the time you get to the higher levels of vision, it's
encoding an entire object.
And then, you can do experiments that basically
interrogate, what does this neuron discover in the world?
What is it interested in?
So if you turn the hand over, it's still very interested.
You show it a less detailed hand.
It's still very interested.
Maybe a bit less, but it's still very interested.
A hand, this way, where all the spatial relations have
changed still recognizes a hand.
It still recognizes a hand with a bit less enthusiasm.
But now, a mitten, which kind of looks like a hand.
That barely counts as a hand at all.
That neuron has lost interest in coding that as a hand.
And you're showed other things that have some
similarity in shape.
Because these all have four elements in
them, like four fingers.
This neuron's not responding at all.
It's not that it has four things or lines.
It's a hand.
And then, you could even worry about things.
Well, maybe this, because it's from a person.
But if they see the face, this neuron doesn't care.
So it's nothing about the gross shape
or that it's a person.
This neuron is specialized, foveally, for spotting a hand.
And not only that, these kinds of neurons already have the
properties we described that are big problems for vision.
So here's a neuron that's responding to an upright face.
And then it's responding if the face is turned on its side
pretty well.
The face is turned upside down, a little less well, but
it's still responding to a face.
If the face is distant, it's still responding.
If the face changes expression, it's still
responding.
And if we put a green filter on it, so you have a Martian
person, it's still responding.
So all kinds of ways that a face changes in the world,
this neuron is still firing.
And interestingly, this neuron was firing for one of the
experimenters.
But it wasn't too interested in the other experimenter.
So it's a neuron that's generalizing all the different
views that you might have of a person.
But it's responding to one neuron versus another.
And there's been a lot of fun in monkey
neurophysiology stuff.
There's a so-called Jennifer Aniston cell, where
researchers have discovered neurons, in monkeys, that seem
to respond to Jennifer Aniston, for whatever reason.
And other specific famous people.
Although, not particularly meaningful to the primates.
And of course, in human brain imaging, you can do tasks that
emphasize what an object is or where it is.
And corresponding to the monkey work, we see activation
in this where pathway towards the parietal cortex.
If you have to make a spatial judgment,
where things are located.
Or this lower temporal lobe, what pathway.
You have to make a judgment about what object you're
looking at.
So that lines up in intact humans very much, with what we
see in primates with invasive studies.
So we're going to end today by discussing two kinds of
lesions in humans and some spectacular impairments in the
what or where pathways.
So let me get you ready for these films for a moment.
So one film you're going to see is the patient who has a
great injury to the parts of the brain in the parietal
cortex that serve our where system.
So this is so-called Balint's syndrome, in the Neurology.
These are bilateral injuries.
That means on both sides of the brain, in
the parietal region.
They're pretty good at knowing what something is, because the
temporal lobe pathway, the what pathway, is intact.
But they have problems in knowing where things are,
reaching for things, where to put their gaze, estimating
distances, and navigation, in general.
So I'll show you an example of a patient like this.
And then the converse, a patient who has big problems
in the what system.
And we'll start with that.
And I'll tell you just one last bit of information to
think about.
That the what and where distinction, sometimes, is not
as complete in every way as you might imagine.
And so, sometimes people have said, really, the where system
is better described as not where things are, but the
information you need for physical action in the world,
which is very close to where.
But it's a little bit different.
And let me give you the kind of what type of information
that the where system still seems to have in it.
So here's a patient with extensive damage in the what
system, in the lower parts of the brain and
posteriorly for vision.
Terrible perception of shapes and orientation of shapes.
Very bad what ability.
And she was asked, then, to reach with an envelope to a
slot at different orientations.
And what's striking is even though if her hand was a
distance away-- so if she would have to think, what is
the angle I would need to approach it with--
she was terrible.
When she actually moved her hand towards the slot, she was
surprisingly good.
So let me show you the feeling.
So if this patient would have to put her hand so that this
letter would fit into this slot--
she was very poor at spatial relations and knowing what
things were--
she couldn't line them up at all.
But now, they said, well, please put your hand out and
just stick it in.
And as her hand approaches it, she changes orientations and
is correct.
Because in order to do things, like to grab something--
think about it.
When you grab something, like a pencil or a cup, you need a
little bit of knowledge to know what it is.
You wouldn't grab things similarly, whether they could
spill easily or be pointy or painful.
A little bit what information helps you guide even something
like a simple reaching, which is a special task.
So it's not so much, maybe, that the where
system is only where.
But it has just enough information to guide actions
in the world, which has a tiny bit of information about what
things are that you're jumping over.
If you're going to run into something, is it likely to be
soft or hard.