WEBVTT 00:00:00.000 --> 00:00:00.992 align:middle line:90% [SQUEAKING] 00:00:00.992 --> 00:00:03.452 align:middle line:90% [RUSTLING] 00:00:03.452 --> 00:00:06.404 align:middle line:90% [CLICKING] 00:00:06.404 --> 00:00:09.660 align:middle line:90% 00:00:09.660 --> 00:00:11.640 align:middle line:84% JOSH MCDERMOTT: So today we're going 00:00:11.640 --> 00:00:15.040 align:middle line:84% to finish talking about color perception. 00:00:15.040 --> 00:00:17.400 align:middle line:84% I mentioned I did a little bit of reorganization 00:00:17.400 --> 00:00:20.420 align:middle line:84% of the ordering of things for logistical reasons. 00:00:20.420 --> 00:00:26.440 align:middle line:84% So the sequence of topics on the syllabus has been updated. 00:00:26.440 --> 00:00:29.880 align:middle line:84% But today, we'll talk about color, next time motion. 00:00:29.880 --> 00:00:31.840 align:middle line:84% Last time, we started talking about how 00:00:31.840 --> 00:00:35.402 align:middle line:84% the world is full of colors, how color is important for lots 00:00:35.402 --> 00:00:36.360 align:middle line:90% of things we got to do. 00:00:36.360 --> 00:00:41.760 align:middle line:84% Segregating objects, recognizing objects, 00:00:41.760 --> 00:00:44.760 align:middle line:84% telling what state objects are in, whether fruits are ripe 00:00:44.760 --> 00:00:50.200 align:middle line:84% or not, or whether meat is rare or well done, 00:00:50.200 --> 00:00:54.380 align:middle line:84% learning things about the state of the environment around us, 00:00:54.380 --> 00:00:55.500 align:middle line:90% and so on and so forth. 00:00:55.500 --> 00:01:00.390 align:middle line:84% And so color depends on the fact that light comes 00:01:00.390 --> 00:01:03.310 align:middle line:90% in these different wavelengths. 00:01:03.310 --> 00:01:05.570 align:middle line:84% And if you look at light in isolation, 00:01:05.570 --> 00:01:09.030 align:middle line:84% it will appear to be colored if you have a particular wavelength 00:01:09.030 --> 00:01:10.010 align:middle line:90% that you're looking at. 00:01:10.010 --> 00:01:12.350 align:middle line:84% But really, we think the function of color perceptually 00:01:12.350 --> 00:01:14.950 align:middle line:84% is to provide information about what objects are made of, 00:01:14.950 --> 00:01:15.890 align:middle line:90% pigmentation. 00:01:15.890 --> 00:01:17.670 align:middle line:84% So we talked about the elementary school 00:01:17.670 --> 00:01:20.693 align:middle line:84% theory of color, which is selective reflectance. 00:01:20.693 --> 00:01:23.110 align:middle line:84% So objects appear colored if they reflect some wavelengths 00:01:23.110 --> 00:01:25.150 align:middle line:90% of light better than others. 00:01:25.150 --> 00:01:26.890 align:middle line:84% We all learned this a long time ago, 00:01:26.890 --> 00:01:28.210 align:middle line:90% and there's some truth to it. 00:01:28.210 --> 00:01:30.670 align:middle line:84% So you can see this if you look at the spectral reflectance 00:01:30.670 --> 00:01:32.710 align:middle line:90% of different kinds of surfaces. 00:01:32.710 --> 00:01:35.670 align:middle line:84% The additional wrinkle, which we're going to talk about today, 00:01:35.670 --> 00:01:38.710 align:middle line:84% is that the light that is reflected off of an object 00:01:38.710 --> 00:01:41.930 align:middle line:84% is a function not just of the reflectance of the object, 00:01:41.930 --> 00:01:44.950 align:middle line:84% but also of the spectrum of the illumination. 00:01:44.950 --> 00:01:46.570 align:middle line:84% So those two variables are confounded, 00:01:46.570 --> 00:01:49.070 align:middle line:84% just like they were in lightness perception, where we 00:01:49.070 --> 00:01:50.910 align:middle line:90% were living in a world of gray. 00:01:50.910 --> 00:01:54.590 align:middle line:84% But in order to perceive color, you 00:01:54.590 --> 00:01:58.060 align:middle line:84% have to transduce information about wavelength. 00:01:58.060 --> 00:01:59.840 align:middle line:84% And we do that via the three cones. 00:01:59.840 --> 00:02:01.820 align:middle line:84% And so we talked about how we have these three 00:02:01.820 --> 00:02:04.020 align:middle line:84% types of cones, how they have different spectral 00:02:04.020 --> 00:02:04.840 align:middle line:90% sensitivities. 00:02:04.840 --> 00:02:07.240 align:middle line:84% We talked about what spectral sensitivity means, 00:02:07.240 --> 00:02:10.139 align:middle line:84% how you would measure it, and then spent 00:02:10.139 --> 00:02:12.740 align:middle line:84% a while kind of talking about this idea 00:02:12.740 --> 00:02:14.600 align:middle line:84% that the cone transduction process 00:02:14.600 --> 00:02:18.340 align:middle line:84% you can think of as a mapping of this very high dimensional thing 00:02:18.340 --> 00:02:21.220 align:middle line:84% in the world, which is the spectrum of the incident 00:02:21.220 --> 00:02:24.420 align:middle line:84% light at a given point on the retina, down to a three 00:02:24.420 --> 00:02:27.700 align:middle line:84% dimensional space of cone responses. 00:02:27.700 --> 00:02:31.330 align:middle line:84% And that number three is because of the fact 00:02:31.330 --> 00:02:33.080 align:middle line:84% that there are these three types of cones. 00:02:33.080 --> 00:02:35.330 align:middle line:84% So if we had four cones, it would be four dimensional. 00:02:35.330 --> 00:02:38.820 align:middle line:84% If we had two, it would be two dimensional, and so forth. 00:02:38.820 --> 00:02:41.120 align:middle line:84% So this is the spectral image formation process. 00:02:41.120 --> 00:02:43.080 align:middle line:84% So there's a light source in the world. 00:02:43.080 --> 00:02:46.140 align:middle line:84% That's the illumination of that reflects off 00:02:46.140 --> 00:02:47.820 align:middle line:84% of a surface that is characterized 00:02:47.820 --> 00:02:48.840 align:middle line:90% by some reflectance. 00:02:48.840 --> 00:02:50.887 align:middle line:84% So both of those are functions of wavelength. 00:02:50.887 --> 00:02:52.720 align:middle line:84% So this is the spectrum of the illumination. 00:02:52.720 --> 00:02:55.380 align:middle line:84% This is the reflectance, which varies with wavelength. 00:02:55.380 --> 00:02:57.740 align:middle line:84% Those two things get multiplied, and that 00:02:57.740 --> 00:02:59.580 align:middle line:84% produces the spectrum of the scattered light 00:02:59.580 --> 00:03:01.420 align:middle line:90% at a given point in space. 00:03:01.420 --> 00:03:04.860 align:middle line:84% That is then projected onto the cone sensitivity functions, 00:03:04.860 --> 00:03:08.100 align:middle line:84% and that yields three numbers, the cone responses 00:03:08.100 --> 00:03:11.100 align:middle line:84% or absorptions of the long, medium, and short wavelength 00:03:11.100 --> 00:03:11.980 align:middle line:90% cones. 00:03:11.980 --> 00:03:13.660 align:middle line:84% So those three numbers are what you 00:03:13.660 --> 00:03:17.057 align:middle line:84% got to go on for the basis of color perception. 00:03:17.057 --> 00:03:19.140 align:middle line:84% But the fact that we have got three of those cones 00:03:19.140 --> 00:03:19.720 align:middle line:90% is critical. 00:03:19.720 --> 00:03:21.470 align:middle line:84% If you just had one of them, you would not 00:03:21.470 --> 00:03:24.020 align:middle line:84% be able to distinguish different wavelengths. 00:03:24.020 --> 00:03:26.660 align:middle line:84% But with three, you have some ability 00:03:26.660 --> 00:03:28.800 align:middle line:84% to distinguish different wavelengths, 00:03:28.800 --> 00:03:31.020 align:middle line:84% because different wavelengths will, in general, 00:03:31.020 --> 00:03:35.780 align:middle line:84% produce different ratios of cone excitations. 00:03:35.780 --> 00:03:38.280 align:middle line:84% But you're still throwing out a lot of information, right? 00:03:38.280 --> 00:03:40.860 align:middle line:84% Because you start with this very high dimensional thing 00:03:40.860 --> 00:03:43.380 align:middle line:84% that gets multiplied by the sensitivities 00:03:43.380 --> 00:03:46.718 align:middle line:84% of the photopigments, and you get three numbers out. 00:03:46.718 --> 00:03:48.260 align:middle line:84% So you lose a lot of information when 00:03:48.260 --> 00:03:50.385 align:middle line:84% you go from this really high dimensional thing down 00:03:50.385 --> 00:03:51.200 align:middle line:90% to three numbers. 00:03:51.200 --> 00:03:53.690 align:middle line:84% And the consequence of that is metamers. 00:03:53.690 --> 00:03:58.810 align:middle line:84% So remember, metamers are physically distinct stimuli that 00:03:58.810 --> 00:04:00.308 align:middle line:90% appear the same to an observer. 00:04:00.308 --> 00:04:01.850 align:middle line:84% And in the case of color vision, they 00:04:01.850 --> 00:04:03.558 align:middle line:84% appear the same because of the fact 00:04:03.558 --> 00:04:05.350 align:middle line:84% that we've only got these three cone types. 00:04:05.350 --> 00:04:09.370 align:middle line:84% So these are just two examples of light sources that have, 00:04:09.370 --> 00:04:11.730 align:middle line:84% obviously, really different spectra, 00:04:11.730 --> 00:04:14.450 align:middle line:84% but would produce the same responses in the three cones 00:04:14.450 --> 00:04:16.329 align:middle line:84% and thus would be indistinguishable. 00:04:16.329 --> 00:04:17.785 align:middle line:84% And so here's some other examples. 00:04:17.785 --> 00:04:19.410 align:middle line:84% These are three different light spectra 00:04:19.410 --> 00:04:22.650 align:middle line:84% that produce the same cone excitations. 00:04:22.650 --> 00:04:24.690 align:middle line:84% And so we talked a little bit about that. 00:04:24.690 --> 00:04:27.610 align:middle line:84% And typically, the way to think about metamers 00:04:27.610 --> 00:04:31.050 align:middle line:84% and the conditions in which you would experience metamers really 00:04:31.050 --> 00:04:34.970 align:middle line:84% is to think about color transduction as linear algebra. 00:04:34.970 --> 00:04:38.090 align:middle line:84% So metamers are, on the one hand this kind of pretty cool thing 00:04:38.090 --> 00:04:39.650 align:middle line:90% about perception. 00:04:39.650 --> 00:04:42.290 align:middle line:84% This idea that you can have these things that are physically 00:04:42.290 --> 00:04:46.130 align:middle line:84% really different, but that are indistinguishable to an organism 00:04:46.130 --> 00:04:48.140 align:middle line:84% just because of the way the eye is set up. 00:04:48.140 --> 00:04:49.890 align:middle line:84% They're also practically really important, 00:04:49.890 --> 00:04:54.480 align:middle line:84% because they're the basis of color reproduction in TVs 00:04:54.480 --> 00:04:55.540 align:middle line:90% and so forth. 00:04:55.540 --> 00:04:59.000 align:middle line:84% So what it means is that because you have only three cones, 00:04:59.000 --> 00:05:02.640 align:middle line:84% you can reproduce any color experience just 00:05:02.640 --> 00:05:04.620 align:middle line:90% with three different color guns. 00:05:04.620 --> 00:05:07.000 align:middle line:84% So three different light sources, for instance. 00:05:07.000 --> 00:05:08.600 align:middle line:84% You can combine them in different ways 00:05:08.600 --> 00:05:10.280 align:middle line:90% and create any particular color. 00:05:10.280 --> 00:05:14.220 align:middle line:84% So that turns out to be very convenient. 00:05:14.220 --> 00:05:15.960 align:middle line:84% So then we ended by kind of talking 00:05:15.960 --> 00:05:19.400 align:middle line:84% about how a long time ago, before scientists 00:05:19.400 --> 00:05:22.080 align:middle line:84% had the ability to go poking around in the eye 00:05:22.080 --> 00:05:24.280 align:middle line:84% or in the nervous system and uncover 00:05:24.280 --> 00:05:26.040 align:middle line:84% the responses of the components of the eye 00:05:26.040 --> 00:05:29.440 align:middle line:84% in the nervous system, people had hypothesized 00:05:29.440 --> 00:05:31.680 align:middle line:84% that color vision was trichromatic because 00:05:31.680 --> 00:05:32.600 align:middle line:90% of this fact. 00:05:32.600 --> 00:05:34.093 align:middle line:84% So scientists discovered that you 00:05:34.093 --> 00:05:35.760 align:middle line:84% could take three different light sources 00:05:35.760 --> 00:05:38.480 align:middle line:84% and combine them in different ratios 00:05:38.480 --> 00:05:41.298 align:middle line:84% to create the experience of all these different colors. 00:05:41.298 --> 00:05:42.840 align:middle line:84% So from that, they inferred this idea 00:05:42.840 --> 00:05:44.800 align:middle line:84% that there would be three receptor types. 00:05:44.800 --> 00:05:48.000 align:middle line:84% But then there was controversy about this because 00:05:48.000 --> 00:05:50.230 align:middle line:84% of these phenomenological problems 00:05:50.230 --> 00:05:51.670 align:middle line:84% with trichromatic theory, which is 00:05:51.670 --> 00:05:55.140 align:middle line:84% to say that there are relationships 00:05:55.140 --> 00:05:56.390 align:middle line:90% between these pairs of colors. 00:05:56.390 --> 00:05:58.050 align:middle line:84% And that shows up in colorblindness, 00:05:58.050 --> 00:06:01.150 align:middle line:84% where you tend to be red green colorblind or blue 00:06:01.150 --> 00:06:05.790 align:middle line:84% yellow colorblind, and not just red colorblind to red or not red 00:06:05.790 --> 00:06:07.430 align:middle line:90% blue red colorblind. 00:06:07.430 --> 00:06:09.750 align:middle line:84% And there's also these observations 00:06:09.750 --> 00:06:13.730 align:middle line:84% that you can make a red look more yellowish or more bluish, 00:06:13.730 --> 00:06:15.630 align:middle line:84% but you can't really make it more greenish. 00:06:15.630 --> 00:06:19.770 align:middle line:84% It's just not psychologically possible. 00:06:19.770 --> 00:06:22.430 align:middle line:84% And so Hering, who is this other perceptual scientist from back 00:06:22.430 --> 00:06:28.065 align:middle line:84% in the day, hypothesized that the receptors had this opponent 00:06:28.065 --> 00:06:30.190 align:middle line:84% nature, that they would be measuring the difference 00:06:30.190 --> 00:06:33.830 align:middle line:90% between different colors. 00:06:33.830 --> 00:06:37.570 align:middle line:84% So nowadays, these two views have been reconciled. 00:06:37.570 --> 00:06:39.710 align:middle line:84% And this is where we kind of ended last time, 00:06:39.710 --> 00:06:43.310 align:middle line:84% with this idea that you start out with three cone 00:06:43.310 --> 00:06:46.350 align:middle line:84% types, the S, M, and L cones, and then combine 00:06:46.350 --> 00:06:50.230 align:middle line:84% those in different ways to yield three channels that, 00:06:50.230 --> 00:06:52.830 align:middle line:84% in this case, come out of the retina, as it turns out. 00:06:52.830 --> 00:06:55.310 align:middle line:84% So by taking the M and the L cones 00:06:55.310 --> 00:06:57.750 align:middle line:84% and computing a difference, you can 00:06:57.750 --> 00:07:00.870 align:middle line:84% get what is called a red green opponent channel. 00:07:00.870 --> 00:07:03.470 align:middle line:84% By summing the M and L cones, you 00:07:03.470 --> 00:07:05.110 align:middle line:84% get an achromatic channel, which people 00:07:05.110 --> 00:07:06.530 align:middle line:90% normally think of as luminance. 00:07:06.530 --> 00:07:07.905 align:middle line:84% And then by taking the difference 00:07:07.905 --> 00:07:11.250 align:middle line:84% between the achromatic channel and the S cones, 00:07:11.250 --> 00:07:14.110 align:middle line:84% you get what is referred to as a blue yellow opponent channel. 00:07:14.110 --> 00:07:18.410 align:middle line:84% And so this opponency is set up at the level of the retina. 00:07:18.410 --> 00:07:21.710 align:middle line:84% So if you look at ganglion cells, 00:07:21.710 --> 00:07:23.750 align:middle line:84% you very often see that there will 00:07:23.750 --> 00:07:25.950 align:middle line:84% be a particular type of cone that provides input 00:07:25.950 --> 00:07:28.430 align:middle line:84% to the center and then a different type of cone that 00:07:28.430 --> 00:07:29.970 align:middle line:90% provides input to the surround. 00:07:29.970 --> 00:07:33.310 align:middle line:84% So remember, ganglion cells in the retina 00:07:33.310 --> 00:07:35.480 align:middle line:84% typically have center surround receptive fields. 00:07:35.480 --> 00:07:37.230 align:middle line:84% They come in these two flavors, on center, 00:07:37.230 --> 00:07:40.910 align:middle line:84% off surrounds, and off center, on surround. 00:07:40.910 --> 00:07:45.310 align:middle line:84% And the midget ganglion cells, remember, 00:07:45.310 --> 00:07:48.180 align:middle line:84% which project to the parvocellular layers of the LGN, 00:07:48.180 --> 00:07:50.627 align:middle line:90% tend to be wavelength sensitive. 00:07:50.627 --> 00:07:52.960 align:middle line:84% But they're wavelength sensitive in this particular way. 00:07:52.960 --> 00:07:54.925 align:middle line:84% So they tend to be color opponents. 00:07:54.925 --> 00:07:56.300 align:middle line:84% So there'll be one cone type that 00:07:56.300 --> 00:08:00.133 align:middle line:84% provides input to the center and another to the surround. 00:08:00.133 --> 00:08:01.800 align:middle line:84% And they come in these particular pairs. 00:08:01.800 --> 00:08:05.720 align:middle line:84% So red and green and then blue and yellow. 00:08:05.720 --> 00:08:08.000 align:middle line:84% And in the blue yellow case, it's not one cone type. 00:08:08.000 --> 00:08:14.480 align:middle line:84% It would be the combination of the L and the M. 00:08:14.480 --> 00:08:19.860 align:middle line:84% And so the consequence of that color opponency, we think, 00:08:19.860 --> 00:08:23.680 align:middle line:84% is that color is organized into a 3D opponent space. 00:08:23.680 --> 00:08:26.300 align:middle line:84% So the psychological space of color 00:08:26.300 --> 00:08:29.340 align:middle line:84% kind of has these three opponent axes. 00:08:29.340 --> 00:08:31.580 align:middle line:84% So one that goes from dark to light and then 00:08:31.580 --> 00:08:34.860 align:middle line:84% these two that vary in color from red to green 00:08:34.860 --> 00:08:35.919 align:middle line:90% and blue to yellow. 00:08:35.919 --> 00:08:37.940 align:middle line:84% And as you transition from blue to yellow, 00:08:37.940 --> 00:08:40.220 align:middle line:84% you go through the center of the space 00:08:40.220 --> 00:08:42.240 align:middle line:84% where things are neutral or gray. 00:08:42.240 --> 00:08:44.850 align:middle line:90% 00:08:44.850 --> 00:08:47.610 align:middle line:84% So you can also describe this in polar coordinates 00:08:47.610 --> 00:08:50.410 align:middle line:84% where you have hue, that's like polar angle, 00:08:50.410 --> 00:08:52.070 align:middle line:90% and then saturation. 00:08:52.070 --> 00:08:55.450 align:middle line:84% That would be kind of the distance from the center. 00:08:55.450 --> 00:08:57.890 align:middle line:84% And then value, how light or dark is it. 00:08:57.890 --> 00:08:59.810 align:middle line:84% So just different coordinate systems 00:08:59.810 --> 00:09:02.690 align:middle line:90% for thinking about color. 00:09:02.690 --> 00:09:06.530 align:middle line:90% So here's the interim summary. 00:09:06.530 --> 00:09:08.450 align:middle line:84% The wavelength composition of light 00:09:08.450 --> 00:09:10.710 align:middle line:84% is represented in the cone responses. 00:09:10.710 --> 00:09:12.770 align:middle line:84% So the wavelength composition of light 00:09:12.770 --> 00:09:16.150 align:middle line:84% is the basis of color vision transduced by the cones. 00:09:16.150 --> 00:09:18.530 align:middle line:84% Metamers result from trichromacy. 00:09:18.530 --> 00:09:23.730 align:middle line:84% The cones are then combined to form opponent channels. 00:09:23.730 --> 00:09:26.750 align:middle line:84% But there's this thing that we're going to have to confront, 00:09:26.750 --> 00:09:29.830 align:middle line:84% which is that light that is reflected off of a surface 00:09:29.830 --> 00:09:31.410 align:middle line:84% is the product of the illumination 00:09:31.410 --> 00:09:32.350 align:middle line:90% and the reflectance. 00:09:32.350 --> 00:09:34.650 align:middle line:84% We think of color as being the perceptual correlate 00:09:34.650 --> 00:09:36.690 align:middle line:90% of reflectance. 00:09:36.690 --> 00:09:39.427 align:middle line:84% And the fact that the wavelength that comes off of a surface 00:09:39.427 --> 00:09:41.010 align:middle line:84% is also influenced by the illumination 00:09:41.010 --> 00:09:43.048 align:middle line:84% is a challenge that our perceptual systems 00:09:43.048 --> 00:09:44.340 align:middle line:90% are going to have to deal with. 00:09:44.340 --> 00:09:46.880 align:middle line:84% And the problems are that the cones 00:09:46.880 --> 00:09:49.160 align:middle line:84% alone, they just measure the spectrum of light 00:09:49.160 --> 00:09:51.140 align:middle line:90% entering the eye. 00:09:51.140 --> 00:09:53.560 align:middle line:84% And so if we want to actually disentangle 00:09:53.560 --> 00:09:57.320 align:middle line:84% these two causal factors in the world, the illumination 00:09:57.320 --> 00:10:00.260 align:middle line:84% and the reflectance of the surfaces that reflect light, 00:10:00.260 --> 00:10:03.200 align:middle line:84% we're going to have to do some kind of perceptual inference. 00:10:03.200 --> 00:10:06.600 align:middle line:84% And this is another classic example of an ill posed problem. 00:10:06.600 --> 00:10:09.640 align:middle line:84% Now, the fact that we can successfully 00:10:09.640 --> 00:10:13.160 align:middle line:84% make this inference most of the time 00:10:13.160 --> 00:10:15.743 align:middle line:84% is referred to as color constancy. 00:10:15.743 --> 00:10:17.660 align:middle line:84% And this has been appreciated for a long time. 00:10:17.660 --> 00:10:22.800 align:middle line:84% So this is actually a quote from Hering back in 1872. 00:10:22.800 --> 00:10:27.360 align:middle line:84% So he wrote, "The approximate constancy of the colors of scene 00:10:27.360 --> 00:10:31.800 align:middle line:84% objects, in spite of large quantitative or qualitative 00:10:31.800 --> 00:10:34.080 align:middle line:84% changes of the general illumination of the visual 00:10:34.080 --> 00:10:36.560 align:middle line:84% field, is one of the most noteworthy and most important 00:10:36.560 --> 00:10:39.140 align:middle line:84% facts in the field of physiological optics." 00:10:39.140 --> 00:10:41.160 align:middle line:84% That's what they used to call vision. 00:10:41.160 --> 00:10:42.857 align:middle line:84% "Without this approximate constancy, 00:10:42.857 --> 00:10:45.440 align:middle line:84% a piece of chalk on a cloudy day would manifest the same color 00:10:45.440 --> 00:10:47.780 align:middle line:84% as a piece of coal does on a sunny day." 00:10:47.780 --> 00:10:50.402 align:middle line:84% So that's like lightness constancy. 00:10:50.402 --> 00:10:51.860 align:middle line:84% "And in the course of a single day, 00:10:51.860 --> 00:10:53.960 align:middle line:84% it would have to assume all possible colors that lie between 00:10:53.960 --> 00:10:54.900 align:middle line:90% black and white." 00:10:54.900 --> 00:10:57.940 align:middle line:90% 00:10:57.940 --> 00:10:59.960 align:middle line:84% So here's why this is a big problem. 00:10:59.960 --> 00:11:03.880 align:middle line:84% This is three examples of the spectral composition 00:11:03.880 --> 00:11:05.640 align:middle line:90% of different light sources. 00:11:05.640 --> 00:11:08.040 align:middle line:84% So the y-axis is relative energy. 00:11:08.040 --> 00:11:10.080 align:middle line:90% The x-axis is wavelength. 00:11:10.080 --> 00:11:13.080 align:middle line:84% So the blue curve plots the spectrum 00:11:13.080 --> 00:11:15.920 align:middle line:84% of daylight, some particular time of the day. 00:11:15.920 --> 00:11:18.600 align:middle line:90% So that's what you get. 00:11:18.600 --> 00:11:25.280 align:middle line:84% The red curve plots the spectrum of a tungsten bulb. 00:11:25.280 --> 00:11:27.600 align:middle line:84% So that's the traditional light bulb 00:11:27.600 --> 00:11:29.780 align:middle line:84% that used to be in most kinds of lamps. 00:11:29.780 --> 00:11:33.640 align:middle line:84% And so that's pretty heavily biased towards long wavelengths. 00:11:33.640 --> 00:11:35.640 align:middle line:84% And then that crazy looking green curve 00:11:35.640 --> 00:11:38.390 align:middle line:84% is what you get from a fluorescent light. 00:11:38.390 --> 00:11:43.030 align:middle line:84% So these three spectra are just obviously wildly different. 00:11:43.030 --> 00:11:46.390 align:middle line:84% And the consequence of that is that the spectrum 00:11:46.390 --> 00:11:48.590 align:middle line:84% of light that is reflected from surfaces 00:11:48.590 --> 00:11:50.290 align:middle line:90% will vary quite enormously. 00:11:50.290 --> 00:11:53.270 align:middle line:84% So the top graph is showing the spectrum 00:11:53.270 --> 00:11:57.230 align:middle line:84% of light reflected from surfaces under daylight illumination. 00:11:57.230 --> 00:12:01.010 align:middle line:84% So we've got a leaf, an orange, and a tomato. 00:12:01.010 --> 00:12:04.470 align:middle line:84% And so again, the relative differences between these 00:12:04.470 --> 00:12:06.390 align:middle line:84% is consistent with the elementary school 00:12:06.390 --> 00:12:07.770 align:middle line:90% theory of color vision. 00:12:07.770 --> 00:12:11.270 align:middle line:84% Because the red object, the tomato, is reflecting more long 00:12:11.270 --> 00:12:12.390 align:middle line:90% wavelength light. 00:12:12.390 --> 00:12:14.190 align:middle line:84% The green object, the leaf, is reflecting 00:12:14.190 --> 00:12:16.310 align:middle line:90% more middle wavelength light. 00:12:16.310 --> 00:12:18.910 align:middle line:90% So that kind of makes sense. 00:12:18.910 --> 00:12:23.030 align:middle line:84% But then the bottom graph shows the light 00:12:23.030 --> 00:12:25.590 align:middle line:84% that's reflected off of those same three surfaces 00:12:25.590 --> 00:12:28.142 align:middle line:90% from a fluorescent light source. 00:12:28.142 --> 00:12:30.850 align:middle line:84% This is not fluorescent, but you probably have fluorescent lights 00:12:30.850 --> 00:12:33.730 align:middle line:84% maybe at home in your kitchen, or you go to a department store, 00:12:33.730 --> 00:12:34.970 align:middle line:90% you might encounter it. 00:12:34.970 --> 00:12:37.228 align:middle line:84% And so the wavelength composition 00:12:37.228 --> 00:12:39.020 align:middle line:84% of the light that's coming off the surfaces 00:12:39.020 --> 00:12:41.500 align:middle line:84% is completely different in those cases. 00:12:41.500 --> 00:12:44.260 align:middle line:84% Yet for the most part, the color of these surfaces 00:12:44.260 --> 00:12:45.320 align:middle line:90% looks pretty similar. 00:12:45.320 --> 00:12:47.300 align:middle line:84% Now, you're also probably-- you've 00:12:47.300 --> 00:12:50.400 align:middle line:84% probably encountered failures of color constancy. 00:12:50.400 --> 00:12:53.395 align:middle line:84% So occasionally, you'll be in some department store. 00:12:53.395 --> 00:12:55.520 align:middle line:84% And there's something funny about the light source. 00:12:55.520 --> 00:12:57.700 align:middle line:84% And you either won't quite be sure of what 00:12:57.700 --> 00:12:59.392 align:middle line:84% the color is of some piece of clothing 00:12:59.392 --> 00:13:00.600 align:middle line:90% you're thinking about buying. 00:13:00.600 --> 00:13:02.420 align:middle line:84% Or maybe it looks one way, and then you get home 00:13:02.420 --> 00:13:03.500 align:middle line:90% and it looks different. 00:13:03.500 --> 00:13:06.020 align:middle line:84% So occasionally there are these failures. 00:13:06.020 --> 00:13:08.620 align:middle line:84% And those are interesting to think about and probably 00:13:08.620 --> 00:13:11.500 align:middle line:84% diagnostic of some of the ways in which we're solving 00:13:11.500 --> 00:13:12.800 align:middle line:90% this little posed problem. 00:13:12.800 --> 00:13:16.540 align:middle line:84% But most of the time, color constancy works pretty well. 00:13:16.540 --> 00:13:21.100 align:middle line:90% 00:13:21.100 --> 00:13:22.540 align:middle line:84% So you might say, well, maybe this 00:13:22.540 --> 00:13:25.940 align:middle line:84% is some artifact of the modern world, the fact that we 00:13:25.940 --> 00:13:28.540 align:middle line:84% got fluorescent lights and tungsten light bulbs and stuff. 00:13:28.540 --> 00:13:30.520 align:middle line:84% And that doesn't seem to be the case. 00:13:30.520 --> 00:13:34.690 align:middle line:84% So even daylight can vary across the time of the day and season. 00:13:34.690 --> 00:13:37.930 align:middle line:84% And so these are two different spectra of daylight 00:13:37.930 --> 00:13:40.090 align:middle line:90% at different times of the day. 00:13:40.090 --> 00:13:42.850 align:middle line:84% So the relative proportion of short and long wavelength light 00:13:42.850 --> 00:13:43.830 align:middle line:90% can change. 00:13:43.830 --> 00:13:47.650 align:middle line:90% 00:13:47.650 --> 00:13:51.730 align:middle line:84% And the basis of color vision as we've discussed 00:13:51.730 --> 00:13:52.950 align:middle line:90% is the cone responses. 00:13:52.950 --> 00:13:54.492 align:middle line:84% And so really what ultimately matters 00:13:54.492 --> 00:13:56.575 align:middle line:84% is whether those are going to be affected by this. 00:13:56.575 --> 00:13:57.350 align:middle line:90% And they are. 00:13:57.350 --> 00:13:59.930 align:middle line:84% And so this is showing three different colors 00:13:59.930 --> 00:14:04.810 align:middle line:84% of paper, yellow paper, red paper, blue paper under two 00:14:04.810 --> 00:14:09.370 align:middle line:84% different illuminations from a tungsten bulb and the blue sky. 00:14:09.370 --> 00:14:11.490 align:middle line:84% And what these graphs are showing 00:14:11.490 --> 00:14:13.870 align:middle line:84% is the spectrum of the reflected light. 00:14:13.870 --> 00:14:15.770 align:middle line:84% And then up at the top, those bar graphs 00:14:15.770 --> 00:14:18.130 align:middle line:84% are the responses of the three cone types. 00:14:18.130 --> 00:14:20.130 align:middle line:84% And so the point is that the three cone 00:14:20.130 --> 00:14:22.730 align:middle line:84% types for blue paper under a tungsten bulb 00:14:22.730 --> 00:14:26.690 align:middle line:84% are pretty different from that of blue paper under a blue sky. 00:14:26.690 --> 00:14:30.170 align:middle line:84% So you can see that for the blue paper under the tungsten 00:14:30.170 --> 00:14:32.410 align:middle line:84% bulb, because the tungsten bulb has 00:14:32.410 --> 00:14:34.113 align:middle line:84% got so much power at long wavelengths 00:14:34.113 --> 00:14:35.530 align:middle line:84% and because the blue paper doesn't 00:14:35.530 --> 00:14:38.890 align:middle line:84% reflect very much of those, the overall responses are low. 00:14:38.890 --> 00:14:42.550 align:middle line:84% Under a blue sky that's got more short wavelengths, 00:14:42.550 --> 00:14:45.470 align:middle line:84% you get a big boost in the response of the S cone. 00:14:45.470 --> 00:14:50.070 align:middle line:84% And there are analogous changes in the other colors of paper. 00:14:50.070 --> 00:14:52.490 align:middle line:84% So different illuminations produce different cone 00:14:52.490 --> 00:14:53.690 align:middle line:90% responses. 00:14:53.690 --> 00:14:56.023 align:middle line:84% And yet the pigmentation of the thing in the world 00:14:56.023 --> 00:14:58.190 align:middle line:84% remains the same, and it looks pretty similar to us, 00:14:58.190 --> 00:15:00.810 align:middle line:84% suggesting that we're able to somehow compensate 00:15:00.810 --> 00:15:03.450 align:middle line:90% for that in some way. 00:15:03.450 --> 00:15:07.330 align:middle line:84% And as is the case with lightness constancy, 00:15:07.330 --> 00:15:10.570 align:middle line:84% insight into the nature of color constancy 00:15:10.570 --> 00:15:12.610 align:middle line:90% comes often from illusions. 00:15:12.610 --> 00:15:14.730 align:middle line:90% So this is a pretty famous one. 00:15:14.730 --> 00:15:20.010 align:middle line:84% So these are two images that were kind of constructed so 00:15:20.010 --> 00:15:23.650 align:middle line:84% that different squares here that look different colors 00:15:23.650 --> 00:15:25.370 align:middle line:84% are actually physically the same. 00:15:25.370 --> 00:15:30.320 align:middle line:84% So in particular, compare the blue checks 00:15:30.320 --> 00:15:34.810 align:middle line:84% on the left with the yellow checks on the right. 00:15:34.810 --> 00:15:36.060 align:middle line:90% So they look really different. 00:15:36.060 --> 00:15:38.120 align:middle line:90% So blue and yellow. 00:15:38.120 --> 00:15:41.920 align:middle line:84% But they're actually physically the same shade of gray. 00:15:41.920 --> 00:15:50.600 align:middle line:84% So we just kind of have whited out the rest of the image. 00:15:50.600 --> 00:15:55.400 align:middle line:84% And you can verify that they're the same. 00:15:55.400 --> 00:16:01.720 align:middle line:84% So the point of this is that physically what's coming off 00:16:01.720 --> 00:16:07.380 align:middle line:84% of the page at these points is what we call gray light. 00:16:07.380 --> 00:16:11.040 align:middle line:90% So spectrally, pretty neutral. 00:16:11.040 --> 00:16:14.880 align:middle line:84% But when you see the entire image, 00:16:14.880 --> 00:16:19.200 align:middle line:84% that physically gray light is interpreted either 00:16:19.200 --> 00:16:23.500 align:middle line:84% as something that is blue in the world or yellow in the world. 00:16:23.500 --> 00:16:27.232 align:middle line:90% 00:16:27.232 --> 00:16:30.590 align:middle line:84% So the high level intuition here really 00:16:30.590 --> 00:16:32.830 align:middle line:84% is that this is very much analogous 00:16:32.830 --> 00:16:35.830 align:middle line:84% to all of the principles that we talked about in lightness 00:16:35.830 --> 00:16:36.530 align:middle line:90% constancy. 00:16:36.530 --> 00:16:38.245 align:middle line:84% So remember, in lightness constancy, 00:16:38.245 --> 00:16:40.870 align:middle line:84% we saw these illusions where you could have the same gray patch 00:16:40.870 --> 00:16:43.850 align:middle line:84% and it would look lighter in one condition and darker in another. 00:16:43.850 --> 00:16:46.470 align:middle line:84% And in general, the conditions in which it's lighter 00:16:46.470 --> 00:16:49.950 align:middle line:84% are those in which there's evidence that it's 00:16:49.950 --> 00:16:51.290 align:middle line:90% in lower illumination. 00:16:51.290 --> 00:16:53.870 align:middle line:84% So it's as though you take the same physical stimulus 00:16:53.870 --> 00:16:56.390 align:middle line:84% and compensate for the low or the high illumination. 00:16:56.390 --> 00:16:59.350 align:middle line:84% And so the same kind of thing is thought to happen here. 00:16:59.350 --> 00:17:01.950 align:middle line:84% So stated again kind of informally, 00:17:01.950 --> 00:17:05.510 align:middle line:84% if you add yellow and blue light, you get gray. 00:17:05.510 --> 00:17:08.670 align:middle line:84% So if there is gray light that is projecting to your eye 00:17:08.670 --> 00:17:12.089 align:middle line:84% and your brain thinks that the illumination is very blue, 00:17:12.089 --> 00:17:14.050 align:middle line:84% so biased towards short wavelengths, 00:17:14.050 --> 00:17:16.869 align:middle line:84% then the only way that you could have gray light on your eye 00:17:16.869 --> 00:17:19.510 align:middle line:84% is if the surface patch is yellow. 00:17:19.510 --> 00:17:22.609 align:middle line:84% And the converse holds if the illumination is very yellow. 00:17:22.609 --> 00:17:25.510 align:middle line:84% And so in these kinds of illusions, what's happening here 00:17:25.510 --> 00:17:27.359 align:middle line:84% is there's a lot-- in the one on the left, 00:17:27.359 --> 00:17:29.540 align:middle line:84% there's a lot of evidence that the illumination 00:17:29.540 --> 00:17:30.600 align:middle line:90% is very yellowish. 00:17:30.600 --> 00:17:32.780 align:middle line:84% In the one on the right, there's a lot of evidence 00:17:32.780 --> 00:17:34.322 align:middle line:84% that the illumination is very bluish, 00:17:34.322 --> 00:17:38.560 align:middle line:84% because all this stuff has kind of got a lot of blue tint to it. 00:17:38.560 --> 00:17:43.220 align:middle line:84% So again, unconsciously, without you really thinking about it, 00:17:43.220 --> 00:17:46.500 align:middle line:84% your brain seems to be compensating for the fact 00:17:46.500 --> 00:17:48.620 align:middle line:84% that there's evidence for blue illumination here 00:17:48.620 --> 00:17:50.242 align:middle line:84% and yellow illumination here, and then 00:17:50.242 --> 00:17:51.700 align:middle line:84% taking that thing that's physically 00:17:51.700 --> 00:17:55.660 align:middle line:84% gray and inferring that it's blue and yellow, respectively. 00:17:55.660 --> 00:18:00.120 align:middle line:90% 00:18:00.120 --> 00:18:01.480 align:middle line:90% Here's another example. 00:18:01.480 --> 00:18:06.600 align:middle line:84% This one is a little bit more subtle, but same essential idea. 00:18:06.600 --> 00:18:10.660 align:middle line:84% So the point is that the second card in this person's hand 00:18:10.660 --> 00:18:16.020 align:middle line:84% is physically the same shade of light pink. 00:18:16.020 --> 00:18:21.920 align:middle line:84% But then the one on the top looks really kind of pink 00:18:21.920 --> 00:18:24.420 align:middle line:84% and the one on the bottom looks a lot more white. 00:18:24.420 --> 00:18:27.900 align:middle line:84% And again, if you look at what's going on in the image, 00:18:27.900 --> 00:18:32.660 align:middle line:84% there's evidence here of bluish illumination up top and pinkish 00:18:32.660 --> 00:18:34.300 align:middle line:90% illumination on the bottom. 00:18:34.300 --> 00:18:38.460 align:middle line:84% And so you attribute the pink light here to the illumination 00:18:38.460 --> 00:18:40.580 align:middle line:90% and thus it looks white. 00:18:40.580 --> 00:18:46.340 align:middle line:84% Whereas there, you attribute it to the paper. 00:18:46.340 --> 00:18:49.480 align:middle line:84% Any questions about these kinds of illusions? 00:18:49.480 --> 00:18:55.580 align:middle line:90% 00:18:55.580 --> 00:19:01.520 align:middle line:84% So these illusions reveal color constancy at work. 00:19:01.520 --> 00:19:05.800 align:middle line:84% And again, the logic that's kind of written out in this page, 00:19:05.800 --> 00:19:08.580 align:middle line:84% I think, is now fairly familiar, because we 00:19:08.580 --> 00:19:12.340 align:middle line:84% have leaned on it pretty extensively at various points 00:19:12.340 --> 00:19:13.460 align:middle line:90% throughout the course. 00:19:13.460 --> 00:19:16.340 align:middle line:84% The notion here is that the point of vision 00:19:16.340 --> 00:19:19.560 align:middle line:84% is to recover properties of the world, not the image. 00:19:19.560 --> 00:19:22.580 align:middle line:84% So the visual system is not designed to just go 00:19:22.580 --> 00:19:23.670 align:middle line:90% around measuring light. 00:19:23.670 --> 00:19:26.510 align:middle line:84% It's designed to tell you what's out there in the world. 00:19:26.510 --> 00:19:28.010 align:middle line:84% The information in the image is used 00:19:28.010 --> 00:19:30.530 align:middle line:84% to infer the properties of the world, not the properties 00:19:30.530 --> 00:19:32.150 align:middle line:90% of the images in our eyes. 00:19:32.150 --> 00:19:34.130 align:middle line:84% And so we call these things illusions, 00:19:34.130 --> 00:19:37.930 align:middle line:84% because you're seeing something that at some level 00:19:37.930 --> 00:19:40.647 align:middle line:84% is divorced from the physical thing 00:19:40.647 --> 00:19:42.230 align:middle line:84% that you would measure from the image. 00:19:42.230 --> 00:19:46.930 align:middle line:84% And we think that gives us some understanding of why 00:19:46.930 --> 00:19:48.370 align:middle line:84% the colors of objects don't appear 00:19:48.370 --> 00:19:51.530 align:middle line:84% to change much when they're in different illumination 00:19:51.530 --> 00:19:52.810 align:middle line:90% environments. 00:19:52.810 --> 00:19:56.250 align:middle line:84% And it's very much analogous to lightness 00:19:56.250 --> 00:19:59.950 align:middle line:84% constancy that we discussed in the previous lecture. 00:19:59.950 --> 00:20:03.050 align:middle line:90% So how does all of this work? 00:20:03.050 --> 00:20:06.610 align:middle line:84% So there are similar debates, as with lightness perception, 00:20:06.610 --> 00:20:09.490 align:middle line:84% regarding the role of low level filter based 00:20:09.490 --> 00:20:12.953 align:middle line:84% mechanisms versus mid and high level mechanisms. 00:20:12.953 --> 00:20:14.870 align:middle line:84% And as was the case with lightness perception, 00:20:14.870 --> 00:20:17.610 align:middle line:84% I don't really have a totally nailed down theory 00:20:17.610 --> 00:20:19.910 align:middle line:84% that can account for all of the data that's available. 00:20:19.910 --> 00:20:25.732 align:middle line:84% We have bits and pieces of clues for the ingredients of theories. 00:20:25.732 --> 00:20:27.440 align:middle line:84% And again, some of the pieces of evidence 00:20:27.440 --> 00:20:29.860 align:middle line:84% are very much analogous to things in lightness perception. 00:20:29.860 --> 00:20:34.340 align:middle line:84% So this is a color analog of simultaneous contrast. 00:20:34.340 --> 00:20:36.500 align:middle line:84% So it's a little hard for me to see. 00:20:36.500 --> 00:20:39.240 align:middle line:84% Hopefully you can make out that up top, there 00:20:39.240 --> 00:20:42.400 align:middle line:84% are these little squares kind of in the middle big squares. 00:20:42.400 --> 00:20:46.320 align:middle line:84% And you can probably tell just by looking at it 00:20:46.320 --> 00:20:49.040 align:middle line:84% that the one on the left looks kind of brown 00:20:49.040 --> 00:20:52.920 align:middle line:84% and the one on the right looks kind of green. 00:20:52.920 --> 00:20:54.960 align:middle line:84% Actually, they're physically the same. 00:20:54.960 --> 00:20:58.240 align:middle line:84% So this is a color analog of the simultaneous contrast illusion 00:20:58.240 --> 00:21:00.200 align:middle line:84% that we talked a lot about in the context 00:21:00.200 --> 00:21:01.420 align:middle line:90% of lightness perception. 00:21:01.420 --> 00:21:04.520 align:middle line:84% So the idea is that the surround here 00:21:04.520 --> 00:21:07.800 align:middle line:84% causes that little square in the center to look different. 00:21:07.800 --> 00:21:11.715 align:middle line:90% 00:21:11.715 --> 00:21:15.160 align:middle line:84% So again, with these color illusions, 00:21:15.160 --> 00:21:17.520 align:middle line:84% these things are not happening at the level 00:21:17.520 --> 00:21:18.620 align:middle line:90% of the photoreceptors. 00:21:18.620 --> 00:21:20.830 align:middle line:84% So the photoreceptors are going to give you 00:21:20.830 --> 00:21:24.690 align:middle line:84% the exact same response to those two little squares. 00:21:24.690 --> 00:21:27.190 align:middle line:84% So the fact that they look the same 00:21:27.190 --> 00:21:32.830 align:middle line:84% is the consequence of some subsequent processing. 00:21:32.830 --> 00:21:35.750 align:middle line:84% And some of that, again, could be potentially 00:21:35.750 --> 00:21:38.430 align:middle line:84% center surround receptive field type operations that 00:21:38.430 --> 00:21:40.355 align:middle line:84% look like taking differences between this. 00:21:40.355 --> 00:21:42.230 align:middle line:84% But there's also the possibility that there's 00:21:42.230 --> 00:21:44.750 align:middle line:84% kind of higher level things going on. 00:21:44.750 --> 00:21:48.870 align:middle line:84% And this experiment here is kind of a cool example 00:21:48.870 --> 00:21:51.070 align:middle line:84% of that that provides some evidence for some fairly 00:21:51.070 --> 00:21:52.950 align:middle line:90% sophisticated inferences. 00:21:52.950 --> 00:21:55.730 align:middle line:84% And so, Fernanda, do have those things? 00:21:55.730 --> 00:21:57.070 align:middle line:90% All right. 00:21:57.070 --> 00:21:58.970 align:middle line:84% We can do a little in-class demo of this. 00:21:58.970 --> 00:22:00.850 align:middle line:90% So just distribute those evenly. 00:22:00.850 --> 00:22:04.150 align:middle line:90% You're going to have to share. 00:22:04.150 --> 00:22:06.870 align:middle line:84% So this is a really cool experiment 00:22:06.870 --> 00:22:10.830 align:middle line:90% that was done back in 1999. 00:22:10.830 --> 00:22:18.910 align:middle line:84% And the idea is to look for a role for inferred 3D structure 00:22:18.910 --> 00:22:20.590 align:middle line:90% in color constancy. 00:22:20.590 --> 00:22:24.230 align:middle line:84% And so what some of you are holding in your hands 00:22:24.230 --> 00:22:29.310 align:middle line:84% is a folded piece of paper that is magenta on one side 00:22:29.310 --> 00:22:33.510 align:middle line:90% and white on the other. 00:22:33.510 --> 00:22:34.970 align:middle line:90% And so this is a diagram. 00:22:34.970 --> 00:22:36.690 align:middle line:84% Magenta on this side, white on the other. 00:22:36.690 --> 00:22:39.430 align:middle line:84% So if you just hold the thing up and look at it, 00:22:39.430 --> 00:22:41.810 align:middle line:84% one side looks magenta and the other side looks white. 00:22:41.810 --> 00:22:42.610 align:middle line:90% That's what it is. 00:22:42.610 --> 00:22:46.810 align:middle line:84% So the thing that's happening here, though, 00:22:46.810 --> 00:22:51.590 align:middle line:84% that you're not aware of is light 00:22:51.590 --> 00:22:58.030 align:middle line:84% is coming from a light source and reflecting, to some extent, 00:22:58.030 --> 00:23:01.670 align:middle line:84% off of the magenta surface and then off of the white surface 00:23:01.670 --> 00:23:03.430 align:middle line:90% and then into your eye. 00:23:03.430 --> 00:23:08.110 align:middle line:84% And so the consequence of those different reflections 00:23:08.110 --> 00:23:10.550 align:middle line:84% is that the fact that there's a magenta 00:23:10.550 --> 00:23:14.510 align:middle line:84% surface on one side of the card is causing the white surface 00:23:14.510 --> 00:23:16.180 align:middle line:84% to actually reflect some light that 00:23:16.180 --> 00:23:23.180 align:middle line:84% is tinted towards magenta due to that pattern of reflections. 00:23:23.180 --> 00:23:25.380 align:middle line:84% And so this is actually a physical measurement 00:23:25.380 --> 00:23:28.960 align:middle line:84% with a photometer that demonstrates that. 00:23:28.960 --> 00:23:32.860 align:middle line:84% And so this is the amount of light 00:23:32.860 --> 00:23:35.540 align:middle line:84% that is coming off of the surface at different positions. 00:23:35.540 --> 00:23:36.960 align:middle line:90% So this is the magenta side. 00:23:36.960 --> 00:23:39.127 align:middle line:84% And so there's a lot of pink light coming off there. 00:23:39.127 --> 00:23:40.380 align:middle line:90% But this is the white side. 00:23:40.380 --> 00:23:43.460 align:middle line:84% And you can see that there's a fair amount of pink tinted light 00:23:43.460 --> 00:23:45.440 align:middle line:84% that's physically coming off of that side. 00:23:45.440 --> 00:23:50.500 align:middle line:84% Now, the remarkable thing is that it doesn't look pink, 00:23:50.500 --> 00:23:52.980 align:middle line:90% but you can make it look pink. 00:23:52.980 --> 00:23:54.580 align:middle line:84% And the way that you make it look pink 00:23:54.580 --> 00:23:59.300 align:middle line:84% is by holding it in your hand, closing one eye, 00:23:59.300 --> 00:24:00.440 align:middle line:90% and then waiting. 00:24:00.440 --> 00:24:02.860 align:middle line:84% So when you close one eye, the three dimensional structure 00:24:02.860 --> 00:24:04.440 align:middle line:84% will become a little bit ambiguous. 00:24:04.440 --> 00:24:06.100 align:middle line:90% And it can even become bistable. 00:24:06.100 --> 00:24:08.740 align:middle line:84% And so if you hold it and close one eye, 00:24:08.740 --> 00:24:11.220 align:middle line:84% the three dimensional structure can reverse. 00:24:11.220 --> 00:24:14.930 align:middle line:84% And so you can go from seeing the thing like this 00:24:14.930 --> 00:24:16.970 align:middle line:90% to seeing the thing like that. 00:24:16.970 --> 00:24:22.250 align:middle line:84% And when you do that, when you get the depth to change, 00:24:22.250 --> 00:24:25.110 align:middle line:84% that thing that looked white will start to look pink. 00:24:25.110 --> 00:24:27.910 align:middle line:90% 00:24:27.910 --> 00:24:33.930 align:middle line:84% And the inference is that your visual system is making some 00:24:33.930 --> 00:24:39.250 align:middle line:84% kind of inference based in part on the 3D geometry that somehow 00:24:39.250 --> 00:24:43.650 align:middle line:84% is compensating for the pink light that it knows should be 00:24:43.650 --> 00:24:48.770 align:middle line:84% reflected off of the magenta surface onto the white surface. 00:24:48.770 --> 00:24:51.450 align:middle line:84% But when you reverse the depth, that reflection 00:24:51.450 --> 00:24:54.050 align:middle line:84% is no longer likely to happen, because the surfaces 00:24:54.050 --> 00:24:56.610 align:middle line:84% aren't pointed in the right direction for that. 00:24:56.610 --> 00:24:58.850 align:middle line:84% And so now that pink light that's 00:24:58.850 --> 00:25:01.130 align:middle line:84% actually coming off the surface gets attributed 00:25:01.130 --> 00:25:03.050 align:middle line:90% to the surface itself. 00:25:03.050 --> 00:25:06.950 align:middle line:90% And so the thing looks pink. 00:25:06.950 --> 00:25:09.650 align:middle line:90% And for me, it even works-- 00:25:09.650 --> 00:25:11.770 align:middle line:84% just closing one eye is enough to actually cause 00:25:11.770 --> 00:25:14.800 align:middle line:84% the other thing to start to look a lot more pink. 00:25:14.800 --> 00:25:15.780 align:middle line:90% That work for you? 00:25:15.780 --> 00:25:16.280 align:middle line:90% Yeah. 00:25:16.280 --> 00:25:16.700 align:middle line:90% OK. 00:25:16.700 --> 00:25:16.880 align:middle line:90% Good. 00:25:16.880 --> 00:25:17.860 align:middle line:90% Satisfied customers. 00:25:17.860 --> 00:25:21.240 align:middle line:90% That's what we aim for. 00:25:21.240 --> 00:25:22.745 align:middle line:90% And so this is just some data. 00:25:22.745 --> 00:25:24.620 align:middle line:84% This is the way that they measure this stuff. 00:25:24.620 --> 00:25:27.800 align:middle line:84% So the actual experiment involved 00:25:27.800 --> 00:25:30.760 align:middle line:84% showing people these displays and then giving them 00:25:30.760 --> 00:25:33.060 align:middle line:90% a big set of colored cards. 00:25:33.060 --> 00:25:34.680 align:middle line:84% And they had to pick the one that 00:25:34.680 --> 00:25:37.040 align:middle line:84% was the best match to the surface in these two 00:25:37.040 --> 00:25:38.520 align:middle line:90% different conditions. 00:25:38.520 --> 00:25:41.040 align:middle line:84% And then this is a representation 00:25:41.040 --> 00:25:43.920 align:middle line:90% of the color of the card. 00:25:43.920 --> 00:25:46.640 align:middle line:84% And the different data points here are the different viewing 00:25:46.640 --> 00:25:47.180 align:middle line:90% conditions. 00:25:47.180 --> 00:25:49.388 align:middle line:84% So the point is that the different viewing conditions 00:25:49.388 --> 00:25:51.520 align:middle line:84% produce different sensations of color. 00:25:51.520 --> 00:26:00.680 align:middle line:84% So this is some evidence that there is a fairly sophisticated 00:26:00.680 --> 00:26:05.240 align:middle line:84% element to the inference of color constancy that apparently 00:26:05.240 --> 00:26:08.840 align:middle line:84% to some extent involves 3D geometry in a fairly 00:26:08.840 --> 00:26:09.740 align:middle line:90% sophisticated way. 00:26:09.740 --> 00:26:11.342 align:middle line:84% This idea that you're going to get 00:26:11.342 --> 00:26:12.800 align:middle line:84% inter-reflections and certain three 00:26:12.800 --> 00:26:15.840 align:middle line:84% dimensional configurations and not in others. 00:26:15.840 --> 00:26:17.420 align:middle line:84% What questions you got about this? 00:26:17.420 --> 00:26:28.880 align:middle line:90% 00:26:28.880 --> 00:26:29.720 align:middle line:90% Yeah. 00:26:29.720 --> 00:26:32.417 align:middle line:84% STUDENT: This is slightly related to the previous p-set. 00:26:32.417 --> 00:26:34.000 align:middle line:84% There was one question that was asking 00:26:34.000 --> 00:26:36.860 align:middle line:84% about cortical and subcortical processes. 00:26:36.860 --> 00:26:40.660 align:middle line:84% And it talked about the fact that if you close an eye, 00:26:40.660 --> 00:26:45.920 align:middle line:84% so if you're [INAUDIBLE] monocular vision, that implied 00:26:45.920 --> 00:26:50.080 align:middle line:90% that it was a cortical process. 00:26:50.080 --> 00:26:53.440 align:middle line:84% So is color constancy a cortical process 00:26:53.440 --> 00:26:56.477 align:middle line:84% rather than a higher level thing? 00:26:56.477 --> 00:26:58.560 align:middle line:84% JOSH MCDERMOTT: So the question was whether or not 00:26:58.560 --> 00:27:01.680 align:middle line:84% the fact that making you close one eye here 00:27:01.680 --> 00:27:05.600 align:middle line:84% relates to the monocular manipulation 00:27:05.600 --> 00:27:06.780 align:middle line:90% from the problem set. 00:27:06.780 --> 00:27:10.550 align:middle line:84% And so in particular, on the problem set, 00:27:10.550 --> 00:27:12.390 align:middle line:84% I think what you were asked to do 00:27:12.390 --> 00:27:17.130 align:middle line:84% is to adapt to this display with only one eye open, 00:27:17.130 --> 00:27:19.477 align:middle line:84% and then see if the effects of adaptation 00:27:19.477 --> 00:27:20.810 align:middle line:90% would transfer to the other eye. 00:27:20.810 --> 00:27:22.570 align:middle line:90% Something like that. 00:27:22.570 --> 00:27:24.050 align:middle line:90% I think that's right. 00:27:24.050 --> 00:27:26.190 align:middle line:84% So that was about transfer of adaptation. 00:27:26.190 --> 00:27:28.765 align:middle line:84% So this is really just about-- so the reason, 00:27:28.765 --> 00:27:30.390 align:middle line:84% and I should have said this, the reason 00:27:30.390 --> 00:27:34.510 align:middle line:84% I asked you to close one eye is because that eliminates 00:27:34.510 --> 00:27:37.790 align:middle line:84% binocular depth cues, which would otherwise 00:27:37.790 --> 00:27:41.350 align:middle line:84% make it very difficult for you to see the incorrect three 00:27:41.350 --> 00:27:42.830 align:middle line:90% dimensional structure here. 00:27:42.830 --> 00:27:46.450 align:middle line:84% So you close one eye and you lose stereoscopic depth. 00:27:46.450 --> 00:27:48.862 align:middle line:84% And so depth just becomes a little more ambiguous. 00:27:48.862 --> 00:27:51.070 align:middle line:84% And that enables you to see the thing like this, even 00:27:51.070 --> 00:27:52.362 align:middle line:90% though it's actually like this. 00:27:52.362 --> 00:27:55.110 align:middle line:84% With both eyes open, that would be very hard to achieve. 00:27:55.110 --> 00:27:56.710 align:middle line:84% So the purpose of that manipulation 00:27:56.710 --> 00:27:57.930 align:middle line:90% is different, in this case. 00:27:57.930 --> 00:28:00.350 align:middle line:90% But yeah, it's a good question. 00:28:00.350 --> 00:28:04.430 align:middle line:90% The dress. 00:28:04.430 --> 00:28:07.540 align:middle line:84% So everybody has probably seen the dress. 00:28:07.540 --> 00:28:13.380 align:middle line:84% This was probably the first viral perceptual phenomena 00:28:13.380 --> 00:28:17.020 align:middle line:84% in the era of social media, I think, potentially. 00:28:17.020 --> 00:28:18.460 align:middle line:90% So it's this photo. 00:28:18.460 --> 00:28:23.580 align:middle line:84% And is there anybody here that hasn't seen this before? 00:28:23.580 --> 00:28:24.880 align:middle line:90% So this was viral, like I said. 00:28:24.880 --> 00:28:27.300 align:middle line:84% So everybody knows that the reason this 00:28:27.300 --> 00:28:30.540 align:middle line:84% went viral is that people disagree on what colors 00:28:30.540 --> 00:28:31.980 align:middle line:90% the dress appears to be. 00:28:31.980 --> 00:28:35.540 align:middle line:84% So some people, and in general, people 00:28:35.540 --> 00:28:38.780 align:middle line:84% tend to either see the thing as being white and gold 00:28:38.780 --> 00:28:40.320 align:middle line:90% or as being blue and black. 00:28:40.320 --> 00:28:44.940 align:middle line:84% And this is the result of a big online experiment with 8,000 00:28:44.940 --> 00:28:49.780 align:middle line:84% people where they asked people to label the colors that they 00:28:49.780 --> 00:28:50.280 align:middle line:90% see. 00:28:50.280 --> 00:28:53.140 align:middle line:84% And so you can see 60% of the participants 00:28:53.140 --> 00:28:57.340 align:middle line:84% say that the dress is white and gold, and then almost 30% 00:28:57.340 --> 00:28:58.988 align:middle line:84% of the participants say blue and black. 00:28:58.988 --> 00:29:01.280 align:middle line:84% And then there's assorted others that see other things. 00:29:01.280 --> 00:29:05.320 align:middle line:84% But most people see it as either white or gold or blue or black. 00:29:05.320 --> 00:29:07.780 align:middle line:84% And so it went viral because people 00:29:07.780 --> 00:29:10.340 align:middle line:84% are very confident that it's either white or gold 00:29:10.340 --> 00:29:15.610 align:middle line:84% or blue or black and disagree about what it looks like. 00:29:15.610 --> 00:29:17.860 align:middle line:84% And so the explanation of this, I think at this point, 00:29:17.860 --> 00:29:19.460 align:middle line:90% is pretty well settled. 00:29:19.460 --> 00:29:22.300 align:middle line:84% So this is a quote from one of several papers 00:29:22.300 --> 00:29:26.100 align:middle line:84% that came out shortly after this. 00:29:26.100 --> 00:29:30.620 align:middle line:84% And it's a really nice example of color constancy 00:29:30.620 --> 00:29:33.760 align:middle line:84% and the ill posed nature of the problem. 00:29:33.760 --> 00:29:34.840 align:middle line:90% So here's the idea. 00:29:34.840 --> 00:29:39.580 align:middle line:84% So as it says, those who assume a relatively long wavelength 00:29:39.580 --> 00:29:42.580 align:middle line:84% illuminant, such as from incandescent light, 00:29:42.580 --> 00:29:45.740 align:middle line:84% could be expected to see the dress as black and blue, 00:29:45.740 --> 00:29:48.940 align:middle line:84% as they would subtract long wavelengths from the image 00:29:48.940 --> 00:29:51.740 align:middle line:84% in order to arrive at a relatively shorter wavelength 00:29:51.740 --> 00:29:53.780 align:middle line:90% percept of the object. 00:29:53.780 --> 00:29:57.460 align:middle line:84% Conversely, those who assume a relatively short wavelength 00:29:57.460 --> 00:29:59.700 align:middle line:84% illuminant, such as daylight, could 00:29:59.700 --> 00:30:02.910 align:middle line:84% be expected to see the dress as white and gold, 00:30:02.910 --> 00:30:05.610 align:middle line:84% as they would subtract short wavelengths from the image 00:30:05.610 --> 00:30:08.170 align:middle line:84% in order to arrive at a relatively longer wavelength 00:30:08.170 --> 00:30:10.930 align:middle line:90% percept of the object. 00:30:10.930 --> 00:30:14.290 align:middle line:84% So that's kind of the essence of how perceptual scientists think 00:30:14.290 --> 00:30:14.870 align:middle line:90% about this. 00:30:14.870 --> 00:30:17.290 align:middle line:84% The idea is that different people 00:30:17.290 --> 00:30:20.290 align:middle line:84% are making different assumptions about the spectral content 00:30:20.290 --> 00:30:21.610 align:middle line:90% of the illumination. 00:30:21.610 --> 00:30:24.890 align:middle line:84% And you can also see, if you look at this, 00:30:24.890 --> 00:30:27.250 align:middle line:90% the picture is quite ambiguous. 00:30:27.250 --> 00:30:29.710 align:middle line:84% You could actually imagine that this might be outside. 00:30:29.710 --> 00:30:32.850 align:middle line:84% You could also imagine that it might be inside, taken in a way 00:30:32.850 --> 00:30:36.450 align:middle line:84% that there's not a ton of unambiguous cues 00:30:36.450 --> 00:30:37.890 align:middle line:90% to the illumination. 00:30:37.890 --> 00:30:41.010 align:middle line:84% So that's what people think is going on. 00:30:41.010 --> 00:30:44.410 align:middle line:84% Here's just a graphical depiction of the idea. 00:30:44.410 --> 00:30:47.387 align:middle line:84% And so we've got these two different dresses, one 00:30:47.387 --> 00:30:49.970 align:middle line:84% that is actually blue and black and one that is actually white 00:30:49.970 --> 00:30:51.290 align:middle line:90% and gold. 00:30:51.290 --> 00:30:53.050 align:middle line:84% And then those two different dresses 00:30:53.050 --> 00:30:56.250 align:middle line:84% are under two different illuminations. 00:30:56.250 --> 00:31:00.130 align:middle line:84% So this is kind of a yellower long wavelength illumination, 00:31:00.130 --> 00:31:03.220 align:middle line:84% and this is a shorter wavelength, bluer illumination. 00:31:03.220 --> 00:31:05.880 align:middle line:84% And the point is that those same two dresses under these two 00:31:05.880 --> 00:31:10.980 align:middle line:84% different illuminations create physically identical images. 00:31:10.980 --> 00:31:13.920 align:middle line:84% And that's what this movie is supposed to show. 00:31:13.920 --> 00:31:19.360 align:middle line:84% So you can see this little patch here getting taken out. 00:31:19.360 --> 00:31:25.400 align:middle line:84% So two different combinations of pigmentation and illumination 00:31:25.400 --> 00:31:29.380 align:middle line:90% create the same image. 00:31:29.380 --> 00:31:31.940 align:middle line:90% 00:31:31.940 --> 00:31:36.280 align:middle line:84% So what's the evidence for this kind of explanation? 00:31:36.280 --> 00:31:41.840 align:middle line:84% So this is the result of a experiment in which people 00:31:41.840 --> 00:31:44.840 align:middle line:84% were queried both about what color 00:31:44.840 --> 00:31:47.440 align:middle line:84% they believed the dress to be, as well 00:31:47.440 --> 00:31:49.640 align:middle line:84% as whether they thought the illumination was 00:31:49.640 --> 00:31:51.480 align:middle line:90% natural or artificial. 00:31:51.480 --> 00:31:56.560 align:middle line:84% And so the y-axis here is-- and so people are grouped into three 00:31:56.560 --> 00:31:57.540 align:middle line:90% different groups here. 00:31:57.540 --> 00:31:58.415 align:middle line:90% So this is the group. 00:31:58.415 --> 00:32:00.082 align:middle line:84% This is the people who said they thought 00:32:00.082 --> 00:32:01.150 align:middle line:90% the lighting was natural. 00:32:01.150 --> 00:32:03.920 align:middle line:84% These are the people who thought the light was artificial. 00:32:03.920 --> 00:32:05.670 align:middle line:84% And these were the people who were unsure. 00:32:05.670 --> 00:32:08.990 align:middle line:84% And the y-axis is the proportion of each of these groups 00:32:08.990 --> 00:32:11.830 align:middle line:84% that report the white gold percept. 00:32:11.830 --> 00:32:15.950 align:middle line:84% So what this is showing is that the people who 00:32:15.950 --> 00:32:18.350 align:middle line:84% think that the illumination is natural 00:32:18.350 --> 00:32:22.830 align:middle line:84% are substantially more likely to see the dress as white and gold 00:32:22.830 --> 00:32:27.250 align:middle line:84% than the people who think the illumination is artificial. 00:32:27.250 --> 00:32:29.630 align:middle line:84% Now, again, I mean, this is assuming 00:32:29.630 --> 00:32:34.320 align:middle line:84% that people are actually accurate in reporting 00:32:34.320 --> 00:32:36.570 align:middle line:84% the representation of the illumination in their heads, 00:32:36.570 --> 00:32:37.650 align:middle line:90% and they might not be so. 00:32:37.650 --> 00:32:41.130 align:middle line:84% But it's nonetheless in the direction that you would expect. 00:32:41.130 --> 00:32:44.670 align:middle line:84% Another thing that's pretty interesting 00:32:44.670 --> 00:32:50.030 align:middle line:84% is sleep schedule seems to influence assumptions 00:32:50.030 --> 00:32:51.170 align:middle line:90% about illumination. 00:32:51.170 --> 00:32:54.110 align:middle line:84% So in the same study, they asked people 00:32:54.110 --> 00:32:57.300 align:middle line:84% to characterize their circadian type. 00:32:57.300 --> 00:33:01.580 align:middle line:84% So an owl is somebody who likes to stay up at night 00:33:01.580 --> 00:33:05.580 align:middle line:84% and sleep in late in the morning. 00:33:05.580 --> 00:33:08.020 align:middle line:84% A lark is somebody who wakes up really 00:33:08.020 --> 00:33:09.560 align:middle line:90% early and goes to bed early. 00:33:09.560 --> 00:33:12.940 align:middle line:84% And so the idea is that somebody who's an owl 00:33:12.940 --> 00:33:16.740 align:middle line:84% is going to have more experience with artificial light, 00:33:16.740 --> 00:33:20.260 align:middle line:84% less experience with daylight, and thus more likely 00:33:20.260 --> 00:33:22.380 align:middle line:84% to implicitly assume artificial lighting, 00:33:22.380 --> 00:33:25.120 align:middle line:84% because that's what they experience more of the time. 00:33:25.120 --> 00:33:26.040 align:middle line:90% That's the idea. 00:33:26.040 --> 00:33:28.900 align:middle line:84% Whereas a lark is going to experience more daylight 00:33:28.900 --> 00:33:31.900 align:middle line:84% in their lives and will be plausibly more likely to 00:33:31.900 --> 00:33:35.200 align:middle line:84% assume natural daylight as being the illumination. 00:33:35.200 --> 00:33:41.980 align:middle line:84% And so this is showing for each of these four self 00:33:41.980 --> 00:33:47.460 align:middle line:84% categorizations the percent that people report the white gold 00:33:47.460 --> 00:33:48.580 align:middle line:90% percept. 00:33:48.580 --> 00:33:51.700 align:middle line:84% And so remember, the white gold percept 00:33:51.700 --> 00:33:54.140 align:middle line:84% is the one that you would arrive at if you 00:33:54.140 --> 00:33:58.380 align:middle line:84% assumed a short wavelength dominated illumination, namely 00:33:58.380 --> 00:33:59.080 align:middle line:90% daylight. 00:33:59.080 --> 00:34:01.980 align:middle line:84% And so you can see that the people that self 00:34:01.980 --> 00:34:05.740 align:middle line:84% categorize as larks are more likely to see the white gold 00:34:05.740 --> 00:34:10.300 align:middle line:84% percept than people that self categorize as owls. 00:34:10.300 --> 00:34:14.699 align:middle line:84% So again, it's kind consistent with the idea 00:34:14.699 --> 00:34:19.340 align:middle line:84% that the way you live your life affects the distribution 00:34:19.340 --> 00:34:20.960 align:middle line:84% of illumination that you experience, 00:34:20.960 --> 00:34:24.739 align:middle line:84% that that has been internalized into priors, 00:34:24.739 --> 00:34:29.500 align:middle line:84% and that that influences your inferences about color. 00:34:29.500 --> 00:34:33.580 align:middle line:90% 00:34:33.580 --> 00:34:35.179 align:middle line:90% Another consistency check. 00:34:35.179 --> 00:34:37.780 align:middle line:84% This is the same self-categorization 00:34:37.780 --> 00:34:39.380 align:middle line:90% into owls and larks. 00:34:39.380 --> 00:34:42.260 align:middle line:84% And this is the percent of people 00:34:42.260 --> 00:34:45.260 align:middle line:84% in each group that say they think 00:34:45.260 --> 00:34:47.159 align:middle line:90% there's artificial lighting. 00:34:47.159 --> 00:34:50.100 align:middle line:84% And so the people that are owls are a fair bit more 00:34:50.100 --> 00:34:52.420 align:middle line:84% likely to say that they think there's 00:34:52.420 --> 00:34:57.870 align:middle line:84% artificial lighting than people who say they're larks. 00:34:57.870 --> 00:34:58.810 align:middle line:90% So that's the dress. 00:34:58.810 --> 00:35:02.730 align:middle line:84% The basic story is that this is a really interesting case where 00:35:02.730 --> 00:35:07.290 align:middle line:84% there are individual differences in the implicit assumptions 00:35:07.290 --> 00:35:11.930 align:middle line:84% that people are making about, in this case, illumination. 00:35:11.930 --> 00:35:14.890 align:middle line:84% And you have to make some assumptions about illumination 00:35:14.890 --> 00:35:17.250 align:middle line:84% in some cases in order to resolve the ill posedness 00:35:17.250 --> 00:35:19.410 align:middle line:90% of color perception. 00:35:19.410 --> 00:35:23.502 align:middle line:84% So this is one kind of case where there are 00:35:23.502 --> 00:35:24.710 align:middle line:90% these individual differences. 00:35:24.710 --> 00:35:26.085 align:middle line:84% And this is actually pretty cool, 00:35:26.085 --> 00:35:31.370 align:middle line:84% just because it's an interesting case where priors 00:35:31.370 --> 00:35:36.490 align:middle line:84% can a little bit be linked to everyday experience 00:35:36.490 --> 00:35:39.410 align:middle line:84% and differences in the way that people live their lives. 00:35:39.410 --> 00:35:42.450 align:middle line:84% So a lot of cases, we think that priors are everywhere 00:35:42.450 --> 00:35:43.610 align:middle line:90% in perception. 00:35:43.610 --> 00:35:47.850 align:middle line:84% We typically don't really have hard evidence as to the extent 00:35:47.850 --> 00:35:50.210 align:middle line:84% to which those are hardwired and derived 00:35:50.210 --> 00:35:53.140 align:middle line:84% from an appearance on an evolutionary time scale, 00:35:53.140 --> 00:35:56.240 align:middle line:84% or whether they are learned over the course of development. 00:35:56.240 --> 00:35:57.960 align:middle line:84% But this is one case where there seems 00:35:57.960 --> 00:36:01.280 align:middle line:84% to be some role for the experience 00:36:01.280 --> 00:36:04.720 align:middle line:84% over the course of development, maybe even into adulthood. 00:36:04.720 --> 00:36:07.800 align:middle line:90% So that's kind of cool. 00:36:07.800 --> 00:36:09.340 align:middle line:90% Any questions about the dress? 00:36:09.340 --> 00:36:16.880 align:middle line:90% 00:36:16.880 --> 00:36:17.960 align:middle line:90% Yeah. 00:36:17.960 --> 00:36:22.910 align:middle line:84% STUDENT: Are people able to switch which one they see? 00:36:22.910 --> 00:36:25.160 align:middle line:84% JOSH MCDERMOTT: I think possibly with enough practice, 00:36:25.160 --> 00:36:27.200 align:middle line:84% you might be able to get yourself to. 00:36:27.200 --> 00:36:30.520 align:middle line:84% I think the reason this kind of blew up the way 00:36:30.520 --> 00:36:33.440 align:middle line:84% that it did is it's actually really not that easy to do 00:36:33.440 --> 00:36:34.720 align:middle line:90% that for most people. 00:36:34.720 --> 00:36:37.800 align:middle line:84% And so people are just like, well, that's definitely white 00:36:37.800 --> 00:36:40.880 align:middle line:84% and gold or that's definitely black and blue. 00:36:40.880 --> 00:36:43.200 align:middle line:84% And that's what made it interesting 00:36:43.200 --> 00:36:45.280 align:middle line:84% is they couldn't understand how someone 00:36:45.280 --> 00:36:46.600 align:middle line:90% could see it differently. 00:36:46.600 --> 00:36:50.742 align:middle line:90% 00:36:50.742 --> 00:36:52.200 align:middle line:84% But I think possibly with practice, 00:36:52.200 --> 00:36:55.460 align:middle line:84% you might be able to do some of that. 00:36:55.460 --> 00:36:58.700 align:middle line:90% A lot of times this stuff is-- 00:36:58.700 --> 00:37:00.760 align:middle line:84% it's not very cognitively penetrable. 00:37:00.760 --> 00:37:02.500 align:middle line:84% Your perceptual system does its thing. 00:37:02.500 --> 00:37:05.600 align:middle line:84% You get the result, and it's hard to access 00:37:05.600 --> 00:37:08.500 align:middle line:84% the internal computations that give rise to it. 00:37:08.500 --> 00:37:15.340 align:middle line:90% 00:37:15.340 --> 00:37:20.320 align:middle line:84% So how is color perception implemented in the brain? 00:37:20.320 --> 00:37:22.360 align:middle line:84% There's a variety of pieces of evidence 00:37:22.360 --> 00:37:25.185 align:middle line:84% for some degree of functional specialization, namely the idea 00:37:25.185 --> 00:37:27.560 align:middle line:84% that there are particular parts of the visual system that 00:37:27.560 --> 00:37:29.200 align:middle line:90% mediate color perception. 00:37:29.200 --> 00:37:32.600 align:middle line:84% One really interesting phenomena that provides evidence for this 00:37:32.600 --> 00:37:35.140 align:middle line:84% is the phenomenon of achromatopsia. 00:37:35.140 --> 00:37:37.600 align:middle line:84% So this is a phenomenon that can result 00:37:37.600 --> 00:37:39.400 align:middle line:84% from certain types of brain damage 00:37:39.400 --> 00:37:43.760 align:middle line:84% in which people selectively lose the perception of color. 00:37:43.760 --> 00:37:48.312 align:middle line:84% And so the idea is that if there is some bit of the brain that's 00:37:48.312 --> 00:37:50.270 align:middle line:84% mediating some particular function, if you have 00:37:50.270 --> 00:37:52.925 align:middle line:84% an unfortunate accident, typically a stroke, 00:37:52.925 --> 00:37:55.550 align:middle line:84% part of the brain is deprived of oxygen. That part of the brain 00:37:55.550 --> 00:37:56.310 align:middle line:90% dies. 00:37:56.310 --> 00:37:59.550 align:middle line:84% And you could get some selective damage 00:37:59.550 --> 00:38:01.350 align:middle line:84% to a particular type of function. 00:38:01.350 --> 00:38:04.630 align:middle line:84% So this is a quote from a case study. 00:38:04.630 --> 00:38:08.190 align:middle line:84% This was a person who was an artist who then had this stroke 00:38:08.190 --> 00:38:10.610 align:middle line:84% and lost the ability to perceive color. 00:38:10.610 --> 00:38:14.350 align:middle line:84% "So Mr. I arrived at his studio expecting that the horrible mist 00:38:14.350 --> 00:38:15.590 align:middle line:90% would be gone. 00:38:15.590 --> 00:38:18.367 align:middle line:84% But as soon as he entered, he found his entire studio, 00:38:18.367 --> 00:38:20.950 align:middle line:84% which was hung with brilliantly colored paintings, now utterly 00:38:20.950 --> 00:38:23.710 align:middle line:90% gray and void of color. 00:38:23.710 --> 00:38:25.850 align:middle line:84% He saw people's flesh, his wife's flesh, 00:38:25.850 --> 00:38:27.790 align:middle line:84% his own flesh as an abhorrent gray. 00:38:27.790 --> 00:38:30.390 align:middle line:84% He found foods disgusting in their grayish dead appearance 00:38:30.390 --> 00:38:31.850 align:middle line:84% and had to close his eyes to eat, 00:38:31.850 --> 00:38:34.990 align:middle line:84% but this did not help very much, for the mental image of a tomato 00:38:34.990 --> 00:38:38.870 align:middle line:90% was as black as its appearance." 00:38:38.870 --> 00:38:41.790 align:middle line:90% Yeah, sounds tough. 00:38:41.790 --> 00:38:44.670 align:middle line:84% But scientifically, really interesting, because it 00:38:44.670 --> 00:38:46.900 align:middle line:84% suggests that you can lose the ability 00:38:46.900 --> 00:38:51.620 align:middle line:84% to see color while preserving most other aspects of vision. 00:38:51.620 --> 00:38:53.740 align:middle line:84% A guy can still recognize objects and navigate 00:38:53.740 --> 00:38:55.340 align:middle line:90% around and so forth. 00:38:55.340 --> 00:38:58.580 align:middle line:84% STUDENT: So it seems that his memory was also black and white? 00:38:58.580 --> 00:39:03.460 align:middle line:84% JOSH MCDERMOTT: Saying the image was black and white. 00:39:03.460 --> 00:39:05.707 align:middle line:84% STUDENT: Like if he imagined the tomato, [INAUDIBLE]. 00:39:05.707 --> 00:39:06.540 align:middle line:90% JOSH MCDERMOTT: Yes. 00:39:06.540 --> 00:39:08.940 align:middle line:84% STUDENT: And if he remembered stuff, 00:39:08.940 --> 00:39:11.723 align:middle line:84% like tomato in the past or something ? 00:39:11.723 --> 00:39:13.140 align:middle line:84% JOSH MCDERMOTT: Well, the question 00:39:13.140 --> 00:39:14.440 align:middle line:90% is, is the memory different? 00:39:14.440 --> 00:39:18.280 align:middle line:84% I mean, in order to access a visual memory, 00:39:18.280 --> 00:39:20.080 align:middle line:84% you often have to form a mental image. 00:39:20.080 --> 00:39:25.860 align:middle line:84% And so we often think that the way that imagery works is you 00:39:25.860 --> 00:39:28.300 align:middle line:84% activate the visual representation that 00:39:28.300 --> 00:39:31.620 align:middle line:84% would result from actually looking at something. 00:39:31.620 --> 00:39:36.380 align:middle line:84% And so it could be that the memory is in some sense normal, 00:39:36.380 --> 00:39:40.660 align:middle line:84% but you don't have the part of the visual representation that 00:39:40.660 --> 00:39:41.840 align:middle line:90% represents color anymore. 00:39:41.840 --> 00:39:44.690 align:middle line:84% And so all that's left is the representation 00:39:44.690 --> 00:39:45.750 align:middle line:90% of all the other stuff. 00:39:45.750 --> 00:39:49.090 align:middle line:84% So you can't represent the color. 00:39:49.090 --> 00:39:52.010 align:middle line:90% So that's one interpretation. 00:39:52.010 --> 00:39:54.467 align:middle line:84% STUDENT: But blind people can see images. 00:39:54.467 --> 00:39:56.550 align:middle line:84% Like, when they remember stuff, they can't see it? 00:39:56.550 --> 00:39:58.170 align:middle line:90% It's just like, different? 00:39:58.170 --> 00:39:59.430 align:middle line:90% JOSH MCDERMOTT: I think so. 00:39:59.430 --> 00:40:01.545 align:middle line:84% Yeah, so someone who loses sight later in life. 00:40:01.545 --> 00:40:03.170 align:middle line:84% Yeah, I mean, I think they can remember 00:40:03.170 --> 00:40:06.430 align:middle line:90% what things looked like. 00:40:06.430 --> 00:40:08.870 align:middle line:84% STUDENT: Probably depends on how they lose their sight. 00:40:08.870 --> 00:40:13.570 align:middle line:84% If you just lose the actual, I mean-- 00:40:13.570 --> 00:40:14.990 align:middle line:90% JOSH MCDERMOTT: Oh, absolutely. 00:40:14.990 --> 00:40:16.095 align:middle line:90% Yeah, sure. 00:40:16.095 --> 00:40:18.470 align:middle line:84% STUDENT: [INAUDIBLE] Then you probably wouldn't remember. 00:40:18.470 --> 00:40:20.428 align:middle line:84% JOSH MCDERMOTT: Yeah, and that's probably true. 00:40:20.428 --> 00:40:24.050 align:middle line:84% So I was imagining that there's an issue with the eyes. 00:40:24.050 --> 00:40:27.657 align:middle line:84% So if you had some massive stroke, major lesion 00:40:27.657 --> 00:40:29.490 align:middle line:84% of the occipital cortex, yeah, I don't think 00:40:29.490 --> 00:40:31.730 align:middle line:90% you would have imagery anymore. 00:40:31.730 --> 00:40:34.050 align:middle line:84% And in fact, I believe it's the case 00:40:34.050 --> 00:40:38.970 align:middle line:84% that if you get a big lesion on, say, the right hemisphere, 00:40:38.970 --> 00:40:40.690 align:middle line:90% left hemisphere is intact. 00:40:40.690 --> 00:40:46.020 align:middle line:84% So imagery of things that is on the right would be normal, 00:40:46.020 --> 00:40:49.160 align:middle line:84% but things on the left would not. 00:40:49.160 --> 00:40:51.140 align:middle line:84% That's a whole field which you can check out. 00:40:51.140 --> 00:40:53.112 align:middle line:90% It's pretty interesting. 00:40:53.112 --> 00:40:55.320 align:middle line:84% So this is the location of the lesions that typically 00:40:55.320 --> 00:40:56.700 align:middle line:90% result in achromatopsia. 00:40:56.700 --> 00:40:59.220 align:middle line:90% So they're non-random. 00:40:59.220 --> 00:41:02.000 align:middle line:84% So this is a figure that took a whole bunch of cases 00:41:02.000 --> 00:41:05.760 align:middle line:84% of people that reported achromatopsia following 00:41:05.760 --> 00:41:07.720 align:middle line:90% brain damage. 00:41:07.720 --> 00:41:09.540 align:middle line:84% And this is an overlap of the lesions. 00:41:09.540 --> 00:41:12.760 align:middle line:84% And so there's particular places that 00:41:12.760 --> 00:41:14.100 align:middle line:90% are much more commonly damaged. 00:41:14.100 --> 00:41:17.640 align:middle line:84% So that suggests some localization of function. 00:41:17.640 --> 00:41:19.960 align:middle line:84% So in people with normal visual systems 00:41:19.960 --> 00:41:22.220 align:middle line:84% or with monkeys with normal visual systems, 00:41:22.220 --> 00:41:24.640 align:middle line:84% there have been experiments to try to find regions 00:41:24.640 --> 00:41:26.240 align:middle line:90% for representing color. 00:41:26.240 --> 00:41:29.160 align:middle line:84% So this is an fMRI experiment that 00:41:29.160 --> 00:41:32.680 align:middle line:84% is showing brain regions that respond 00:41:32.680 --> 00:41:34.720 align:middle line:84% more to images that vary in color 00:41:34.720 --> 00:41:38.120 align:middle line:84% than to those that just vary in luminance. 00:41:38.120 --> 00:41:40.320 align:middle line:84% And so you see these little places 00:41:40.320 --> 00:41:43.280 align:middle line:84% that are more active to color than to images 00:41:43.280 --> 00:41:45.600 align:middle line:90% that don't have color. 00:41:45.600 --> 00:41:49.160 align:middle line:84% And for reasons that I think are unclear, 00:41:49.160 --> 00:41:51.600 align:middle line:84% there are multiple such regions, and they 00:41:51.600 --> 00:41:55.360 align:middle line:84% tend to be right next to regions of the ventral pathway that 00:41:55.360 --> 00:41:57.100 align:middle line:90% are very responsive to faces. 00:41:57.100 --> 00:42:02.560 align:middle line:84% So these color biased regions here in the temporal lobe, 00:42:02.560 --> 00:42:05.002 align:middle line:84% and they're right next to these so-called face patches 00:42:05.002 --> 00:42:07.460 align:middle line:84% where you see lots of neurons that are responsive to faces. 00:42:07.460 --> 00:42:10.180 align:middle line:84% So that's just a feature of the visual system. 00:42:10.180 --> 00:42:12.920 align:middle line:84% I don't think we have a very clear explanation for that, 00:42:12.920 --> 00:42:15.280 align:middle line:90% but it is what it is. 00:42:15.280 --> 00:42:20.400 align:middle line:84% So there's been lots of interest in the relationship of color 00:42:20.400 --> 00:42:21.780 align:middle line:90% to other aspects of vision. 00:42:21.780 --> 00:42:24.440 align:middle line:90% But first we got a question. 00:42:24.440 --> 00:42:27.720 align:middle line:84% STUDENT: So it wasn't that the man with the condition 00:42:27.720 --> 00:42:31.160 align:middle line:90% was only using his rods. 00:42:31.160 --> 00:42:34.260 align:middle line:84% His vision didn't get blurry or anything either? 00:42:34.260 --> 00:42:36.920 align:middle line:90% 00:42:36.920 --> 00:42:38.690 align:middle line:84% JOSH MCDERMOTT: Based on the case report, 00:42:38.690 --> 00:42:39.773 align:middle line:90% I believe that to be true. 00:42:39.773 --> 00:42:41.590 align:middle line:84% I mean, yes, but the retina is fine. 00:42:41.590 --> 00:42:46.830 align:middle line:84% So again, you have a stroke and the eye 00:42:46.830 --> 00:42:48.330 align:middle line:90% is not going to be affected. 00:42:48.330 --> 00:42:50.070 align:middle line:84% So it's just the part of the brain that 00:42:50.070 --> 00:42:53.270 align:middle line:84% is deprived of blood flow for that period of time 00:42:53.270 --> 00:42:55.670 align:middle line:90% that would be damaged. 00:42:55.670 --> 00:42:59.090 align:middle line:84% So what is the relationship of color to shape processing? 00:42:59.090 --> 00:43:01.710 align:middle line:84% So there's a bunch of interesting demonstrations here. 00:43:01.710 --> 00:43:05.390 align:middle line:84% So one kind of empirical fact is that the spatial resolution 00:43:05.390 --> 00:43:07.910 align:middle line:84% of color processing is more limited 00:43:07.910 --> 00:43:10.090 align:middle line:84% than the spatial resolution of luminance processing. 00:43:10.090 --> 00:43:11.965 align:middle line:84% So this is just a really simple demonstration 00:43:11.965 --> 00:43:13.230 align:middle line:90% where there's a checkerboard. 00:43:13.230 --> 00:43:17.110 align:middle line:84% The middle part of the checkerboard is blue and yellow. 00:43:17.110 --> 00:43:20.150 align:middle line:84% And you can see that as this is scaled down, 00:43:20.150 --> 00:43:23.030 align:middle line:84% the very small versions, you lose 00:43:23.030 --> 00:43:25.030 align:middle line:90% the sense of color variation. 00:43:25.030 --> 00:43:28.170 align:middle line:84% But you can still see the checks. 00:43:28.170 --> 00:43:30.830 align:middle line:84% So the black and white variation is pretty easy 00:43:30.830 --> 00:43:33.010 align:middle line:84% to discern, even in the smallest one. 00:43:33.010 --> 00:43:35.885 align:middle line:84% But the blue yellow variation, you probably lose. 00:43:35.885 --> 00:43:38.260 align:middle line:84% So this will depend to some extent on how far back you're 00:43:38.260 --> 00:43:41.100 align:middle line:90% sitting in the room. 00:43:41.100 --> 00:43:44.520 align:middle line:84% And some of this probably reflects the receptor lattice. 00:43:44.520 --> 00:43:48.060 align:middle line:84% So for instance, the short wavelength cones 00:43:48.060 --> 00:43:51.080 align:middle line:84% are just kind of spaced pretty far apart in the cone mosaic. 00:43:51.080 --> 00:43:54.620 align:middle line:90% So that's probably part of it. 00:43:54.620 --> 00:43:58.220 align:middle line:84% But there's just also a lot of other pieces of evidence 00:43:58.220 --> 00:44:02.460 align:middle line:84% that color behaves a little bit differently 00:44:02.460 --> 00:44:04.740 align:middle line:90% than luminance variation. 00:44:04.740 --> 00:44:08.660 align:middle line:84% So one such piece of evidence is that you 00:44:08.660 --> 00:44:11.780 align:middle line:84% can come up with images that can purely 00:44:11.780 --> 00:44:13.940 align:middle line:90% contain chromatic variation. 00:44:13.940 --> 00:44:18.580 align:middle line:84% And I'm not going to tell you exactly how you come up 00:44:18.580 --> 00:44:20.843 align:middle line:84% with these things, but you can tell 00:44:20.843 --> 00:44:23.260 align:middle line:84% from looking at these things that they look kind of weird. 00:44:23.260 --> 00:44:25.180 align:middle line:84% And in particular, it's a lot harder 00:44:25.180 --> 00:44:28.680 align:middle line:84% to perceive shape in images that are so-called isoluminant. 00:44:28.680 --> 00:44:31.180 align:middle line:84% It's also much harder to actually perceive motion. 00:44:31.180 --> 00:44:32.620 align:middle line:84% So if you take a isoluminant image 00:44:32.620 --> 00:44:34.550 align:middle line:84% and you and you set it in motion, 00:44:34.550 --> 00:44:37.770 align:middle line:84% the motion is often really hard to perceive. 00:44:37.770 --> 00:44:42.730 align:middle line:84% And this is another example of the poor spatial resolution 00:44:42.730 --> 00:44:45.810 align:middle line:84% of color, where you've got this gray cloud and then 00:44:45.810 --> 00:44:47.850 align:middle line:90% an analogous yellow cloud. 00:44:47.850 --> 00:44:51.210 align:middle line:84% And it's really pretty easy to see the fuzziness of the border 00:44:51.210 --> 00:44:51.790 align:middle line:90% here. 00:44:51.790 --> 00:44:53.510 align:middle line:84% This is actually much harder to see. 00:44:53.510 --> 00:44:55.510 align:middle line:84% And then you superimpose it on the black circle, 00:44:55.510 --> 00:44:58.210 align:middle line:84% and it looks just kind of a yellow, 00:44:58.210 --> 00:45:01.130 align:middle line:84% that the black circle is yellow in the middle. 00:45:01.130 --> 00:45:07.090 align:middle line:84% When in fact, you have this scruffy border all around. 00:45:07.090 --> 00:45:09.490 align:middle line:84% So there's this kind of intuition 00:45:09.490 --> 00:45:14.170 align:middle line:84% that form processing, the processing of shape, 00:45:14.170 --> 00:45:17.050 align:middle line:84% may be mostly luminance based, with color kind of filled 00:45:17.050 --> 00:45:18.250 align:middle line:90% in around the form. 00:45:18.250 --> 00:45:20.530 align:middle line:84% And there's a pretty amazing illusion 00:45:20.530 --> 00:45:23.350 align:middle line:84% that I will show you that is consistent with this. 00:45:23.350 --> 00:45:27.010 align:middle line:90% So let's check this out. 00:45:27.010 --> 00:45:28.770 align:middle line:84% There are these illusion contests 00:45:28.770 --> 00:45:37.250 align:middle line:84% and this is the winning illusion from some year. 00:45:37.250 --> 00:45:42.570 align:middle line:84% And so what you're supposed to do here 00:45:42.570 --> 00:45:48.010 align:middle line:84% is fixate on that little dot in the middle. 00:45:48.010 --> 00:45:55.450 align:middle line:84% And the illusion here consists of this colored pattern 00:45:55.450 --> 00:45:57.290 align:middle line:90% that kind of pops up. 00:45:57.290 --> 00:46:00.570 align:middle line:84% And then after the color pattern pops up, 00:46:00.570 --> 00:46:04.510 align:middle line:84% you see an outline of one kind of part of the pattern. 00:46:04.510 --> 00:46:08.330 align:middle line:84% And alternately, you will see one part and then 00:46:08.330 --> 00:46:09.330 align:middle line:90% the other part. 00:46:09.330 --> 00:46:12.610 align:middle line:84% And they'll be different on the two halves of the screen. 00:46:12.610 --> 00:46:15.450 align:middle line:84% And so the really amazing thing here 00:46:15.450 --> 00:46:19.010 align:middle line:84% is that the color that you perceive 00:46:19.010 --> 00:46:21.170 align:middle line:84% is really dependent on which of the two 00:46:21.170 --> 00:46:23.850 align:middle line:90% outlines you're looking at. 00:46:23.850 --> 00:46:26.890 align:middle line:84% And so the thing kind of flips from being blue to pink and vice 00:46:26.890 --> 00:46:27.390 align:middle line:90% versa. 00:46:27.390 --> 00:46:34.320 align:middle line:84% And what's amazing about this is that the color afterimage, so 00:46:34.320 --> 00:46:38.060 align:middle line:84% the adaptation in the cones should be exactly the same, 00:46:38.060 --> 00:46:41.580 align:middle line:84% irrespective of which outline you're looking at. 00:46:41.580 --> 00:46:43.440 align:middle line:90% So you see that color image. 00:46:43.440 --> 00:46:45.618 align:middle line:84% There's a slight amount of adaptation in the cones. 00:46:45.618 --> 00:46:46.660 align:middle line:90% So you get an afterimage. 00:46:46.660 --> 00:46:49.160 align:middle line:84% But then what you perceive, what you see, 00:46:49.160 --> 00:46:52.440 align:middle line:84% is a function of which outline you get. 00:46:52.440 --> 00:46:56.040 align:middle line:84% And so it's kind of consistent with this idea 00:46:56.040 --> 00:46:59.960 align:middle line:84% that a lot of color is filled in based 00:46:59.960 --> 00:47:04.240 align:middle line:84% on boundaries of objects that are kind of derived 00:47:04.240 --> 00:47:05.880 align:middle line:90% from the luminance system. 00:47:05.880 --> 00:47:08.100 align:middle line:84% And again, this is all a little bit hand-wavy. 00:47:08.100 --> 00:47:11.780 align:middle line:84% We don't have rigorous accounts of this, 00:47:11.780 --> 00:47:14.698 align:middle line:84% but there's these very suggestive and powerful 00:47:14.698 --> 00:47:16.240 align:middle line:84% illusions that are kind of consistent 00:47:16.240 --> 00:47:18.440 align:middle line:90% with that general thing. 00:47:18.440 --> 00:47:19.380 align:middle line:90% Yeah. 00:47:19.380 --> 00:47:21.200 align:middle line:84% STUDENT: [INAUDIBLE] the opposite 00:47:21.200 --> 00:47:24.925 align:middle line:90% of what it is that [INAUDIBLE]? 00:47:24.925 --> 00:47:25.800 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:47:25.800 --> 00:47:29.630 align:middle line:84% So typically, if you generate a color afterimage, 00:47:29.630 --> 00:47:32.030 align:middle line:84% you'll adapt to red, and then you'll 00:47:32.030 --> 00:47:37.310 align:middle line:84% see something that will be more greenish. 00:47:37.310 --> 00:47:41.910 align:middle line:84% So the fact that it looks like the opposite color, that's kind 00:47:41.910 --> 00:47:43.170 align:middle line:90% of a fairly standard thing. 00:47:43.170 --> 00:47:44.950 align:middle line:84% What was new and remarkable about this 00:47:44.950 --> 00:47:47.850 align:middle line:84% was the dependence on the outlines. 00:47:47.850 --> 00:47:51.310 align:middle line:90% 00:47:51.310 --> 00:47:52.730 align:middle line:90% Any other questions about this? 00:47:52.730 --> 00:47:55.950 align:middle line:90% 00:47:55.950 --> 00:47:59.390 align:middle line:84% So this was the illusion of the year in 2008. 00:47:59.390 --> 00:48:00.250 align:middle line:90% It's pretty good. 00:48:00.250 --> 00:48:03.790 align:middle line:90% 00:48:03.790 --> 00:48:07.230 align:middle line:84% So there's this evidence that there's 00:48:07.230 --> 00:48:11.030 align:middle line:84% some separation in some sense between color and form 00:48:11.030 --> 00:48:14.470 align:middle line:84% in the sense that color is not very good at supplying form. 00:48:14.470 --> 00:48:17.870 align:middle line:90% 00:48:17.870 --> 00:48:19.950 align:middle line:84% It tends to follow the luminance structure 00:48:19.950 --> 00:48:22.990 align:middle line:90% in some of hand-wavy way. 00:48:22.990 --> 00:48:25.890 align:middle line:84% But these things are not completely independent. 00:48:25.890 --> 00:48:27.680 align:middle line:84% So if you look in the visual system, 00:48:27.680 --> 00:48:29.980 align:middle line:84% you can find orientation selective neurons that 00:48:29.980 --> 00:48:31.740 align:middle line:90% are also wavelength selective. 00:48:31.740 --> 00:48:34.940 align:middle line:84% So this is an example of a receptive field in V1 00:48:34.940 --> 00:48:37.900 align:middle line:84% where they separately map the receptive 00:48:37.900 --> 00:48:43.860 align:middle line:84% field for L cone stimulation and M cone stimulation. 00:48:43.860 --> 00:48:46.140 align:middle line:84% And you can see that there's opponency here 00:48:46.140 --> 00:48:50.840 align:middle line:84% that's related to the orientation selectivity here. 00:48:50.840 --> 00:48:55.620 align:middle line:84% So you get excitatory input from the M cones in one lobe 00:48:55.620 --> 00:48:59.460 align:middle line:84% and in the other lobe for the L cones. 00:48:59.460 --> 00:49:01.480 align:middle line:84% So this was a physiological discovery. 00:49:01.480 --> 00:49:05.100 align:middle line:84% But what predated that was this very well-known and very 00:49:05.100 --> 00:49:08.060 align:middle line:84% powerful after effect called the McCullough after effect. 00:49:08.060 --> 00:49:10.220 align:middle line:84% And so what I want you to just do here 00:49:10.220 --> 00:49:12.400 align:middle line:90% is start looking at this. 00:49:12.400 --> 00:49:16.210 align:middle line:84% So look alternately between the green and the pink stimuli. 00:49:16.210 --> 00:49:17.960 align:middle line:84% We're going to do this for about a minute. 00:49:17.960 --> 00:49:20.660 align:middle line:84% Just get started on it while I talk to you about it. 00:49:20.660 --> 00:49:23.260 align:middle line:84% So what you're doing while you look at this 00:49:23.260 --> 00:49:26.940 align:middle line:84% is you're adapting to these combinations of color 00:49:26.940 --> 00:49:29.740 align:middle line:84% and orientation, but you're doing it at the same time. 00:49:29.740 --> 00:49:33.460 align:middle line:84% So you're adapting your cones simultaneously 00:49:33.460 --> 00:49:35.340 align:middle line:90% to green and to pink. 00:49:35.340 --> 00:49:37.600 align:middle line:84% So you might think that they would cancel out. 00:49:37.600 --> 00:49:40.580 align:middle line:84% But at the same time, the color is paired 00:49:40.580 --> 00:49:42.500 align:middle line:90% with a particular orientation. 00:49:42.500 --> 00:49:47.500 align:middle line:84% And as you will see, this will produce a color orientation 00:49:47.500 --> 00:49:49.240 align:middle line:90% contingent aftereffect. 00:49:49.240 --> 00:49:51.120 align:middle line:84% It's called the McCullough effect, 00:49:51.120 --> 00:49:54.100 align:middle line:84% named after the person who discovered it. 00:49:54.100 --> 00:49:56.780 align:middle line:84% This was initially taken as evidence 00:49:56.780 --> 00:50:00.340 align:middle line:84% that there are neurons that are tuned to conjunctions of color 00:50:00.340 --> 00:50:02.580 align:middle line:84% and orientation, because that's the traditional logic 00:50:02.580 --> 00:50:03.720 align:middle line:90% of after effects. 00:50:03.720 --> 00:50:05.780 align:middle line:84% So if you get an after effect for something, 00:50:05.780 --> 00:50:07.700 align:middle line:84% that usually is an indication that you 00:50:07.700 --> 00:50:10.540 align:middle line:84% have neurons that are tuned for it, 00:50:10.540 --> 00:50:13.050 align:middle line:84% that you're adapting out when you adapt. 00:50:13.050 --> 00:50:15.300 align:middle line:84% But as we will see, the story may be a little bit more 00:50:15.300 --> 00:50:16.217 align:middle line:90% complicated than that. 00:50:16.217 --> 00:50:17.320 align:middle line:90% So you're adapted out. 00:50:17.320 --> 00:50:18.660 align:middle line:90% Now look at this. 00:50:18.660 --> 00:50:21.900 align:middle line:84% And what you should see-- so first look 00:50:21.900 --> 00:50:26.870 align:middle line:84% at the one on the left and then look at the one on the right. 00:50:26.870 --> 00:50:30.650 align:middle line:84% And these gratings should look slightly tinted, 00:50:30.650 --> 00:50:32.570 align:middle line:90% either pink or green. 00:50:32.570 --> 00:50:35.893 align:middle line:84% But the tint will depend on the orientation. 00:50:35.893 --> 00:50:37.310 align:middle line:84% So if you on the one on the right, 00:50:37.310 --> 00:50:39.185 align:middle line:84% you'll see one particular tint in the middle. 00:50:39.185 --> 00:50:40.893 align:middle line:84% And then you look at the one on the left, 00:50:40.893 --> 00:50:42.810 align:middle line:84% and you'll see a different tint in the middle. 00:50:42.810 --> 00:50:47.590 align:middle line:84% So this is not happening at the retina. 00:50:47.590 --> 00:50:50.770 align:middle line:84% So it's very different from the photoreceptor adaptation 00:50:50.770 --> 00:50:51.790 align:middle line:90% that we've talked about. 00:50:51.790 --> 00:50:54.530 align:middle line:84% It's dependent on the orientation. 00:50:54.530 --> 00:50:58.650 align:middle line:90% So it's something more central. 00:50:58.650 --> 00:51:02.410 align:middle line:84% So the traditional kind of naive interpretation of this 00:51:02.410 --> 00:51:05.690 align:middle line:84% is that there are neurons that are tuned for combinations 00:51:05.690 --> 00:51:08.810 align:middle line:84% of orientation and color and that you just 00:51:08.810 --> 00:51:09.862 align:middle line:90% adapt those neurons. 00:51:09.862 --> 00:51:11.570 align:middle line:84% Now, the reason that people are generally 00:51:11.570 --> 00:51:13.250 align:middle line:84% skeptical of that kind of explanation 00:51:13.250 --> 00:51:15.430 align:middle line:84% is that this particular after effect, 00:51:15.430 --> 00:51:18.890 align:middle line:84% for reasons that are unclear, lasts for a very long time. 00:51:18.890 --> 00:51:23.420 align:middle line:84% And so we'll actually look at it again in a few minutes, 00:51:23.420 --> 00:51:25.537 align:middle line:84% and you'll probably still be able to see it. 00:51:25.537 --> 00:51:27.620 align:middle line:84% If you go home tonight and you look at this again, 00:51:27.620 --> 00:51:29.287 align:middle line:84% you'll probably still be able to see it. 00:51:29.287 --> 00:51:32.040 align:middle line:84% It will eventually go away, but it lasts a really long time. 00:51:32.040 --> 00:51:34.240 align:middle line:84% And so people think that it may have a bit more 00:51:34.240 --> 00:51:37.040 align:middle line:84% to do with perceptual learning or something than a traditional 00:51:37.040 --> 00:51:40.020 align:middle line:84% after effect, but it's a pretty remarkable effect. 00:51:40.020 --> 00:51:43.060 align:middle line:90% It's worth knowing about. 00:51:43.060 --> 00:51:44.400 align:middle line:90% A few other phenomena. 00:51:44.400 --> 00:51:47.440 align:middle line:84% I should also just stop and say that in this lecture 00:51:47.440 --> 00:51:51.080 align:middle line:84% and the next few lectures, I'm going to show 00:51:51.080 --> 00:51:52.580 align:middle line:90% you a bunch of phenomena. 00:51:52.580 --> 00:51:55.553 align:middle line:84% And part of the reason I'm showing you these things, 00:51:55.553 --> 00:51:57.220 align:middle line:84% well, first of all, they're kind of fun, 00:51:57.220 --> 00:52:00.040 align:middle line:84% but also because these are potential food 00:52:00.040 --> 00:52:02.940 align:middle line:84% for thought for the illusion lab that is coming up. 00:52:02.940 --> 00:52:06.110 align:middle line:84% So I don't expect you to memorize all of these things. 00:52:06.110 --> 00:52:08.360 align:middle line:84% I just want to introduce you to a few of these in case 00:52:08.360 --> 00:52:11.140 align:middle line:84% they give you inspiration for your illusion labs. 00:52:11.140 --> 00:52:14.720 align:middle line:84% And they also just provide a little bit more context. 00:52:14.720 --> 00:52:17.600 align:middle line:84% So these are a couple of other interesting cases 00:52:17.600 --> 00:52:20.910 align:middle line:84% where color interacts with shape. 00:52:20.910 --> 00:52:23.450 align:middle line:84% So it contributes to contour grouping. 00:52:23.450 --> 00:52:26.830 align:middle line:84% So we saw some stimuli like this a couple lectures ago, 00:52:26.830 --> 00:52:30.990 align:middle line:84% where you can detect the kind of continuous path of the Gabors. 00:52:30.990 --> 00:52:33.950 align:middle line:84% But if they alternate between being chromatic and achromatic, 00:52:33.950 --> 00:52:38.190 align:middle line:84% like they do here, on the right, it's a lot harder to see. 00:52:38.190 --> 00:52:40.230 align:middle line:84% So that kind of suggests that color 00:52:40.230 --> 00:52:45.152 align:middle line:84% is contributing to contour grouping in some way. 00:52:45.152 --> 00:52:47.610 align:middle line:84% It also interacts with something called shape from shading. 00:52:47.610 --> 00:52:49.810 align:middle line:84% We haven't talked about shape from shading yet. 00:52:49.810 --> 00:52:52.870 align:middle line:84% We're going to talk about it in a fair bit of detail 00:52:52.870 --> 00:52:54.870 align:middle line:90% in about a week or two. 00:52:54.870 --> 00:52:57.110 align:middle line:84% But shape from shading is this phenomenon 00:52:57.110 --> 00:53:01.070 align:middle line:84% whereby if a surface is Lambertian, then 00:53:01.070 --> 00:53:02.630 align:middle line:84% the amount of light that is reflected 00:53:02.630 --> 00:53:04.838 align:middle line:84% will be a function of the orientation of the surface. 00:53:04.838 --> 00:53:07.670 align:middle line:84% And so if you have something that is curved, 00:53:07.670 --> 00:53:12.170 align:middle line:84% there will be a gradient in the luminance as the shape changes. 00:53:12.170 --> 00:53:15.390 align:middle line:84% And so this is a pretty interesting demonstration where 00:53:15.390 --> 00:53:16.770 align:middle line:90% you get a sine wave grating. 00:53:16.770 --> 00:53:19.330 align:middle line:84% So you get this variation in luminance here. 00:53:19.330 --> 00:53:22.970 align:middle line:84% And maybe if you are feeling really good, 00:53:22.970 --> 00:53:27.270 align:middle line:84% you can see that as a corrugated surface. 00:53:27.270 --> 00:53:29.270 align:middle line:84% But then if we take this color grading 00:53:29.270 --> 00:53:32.930 align:middle line:84% and we just superimpose it on the luminance grading, 00:53:32.930 --> 00:53:36.810 align:middle line:84% now you get a pretty compelling sense of a corrugated surface. 00:53:36.810 --> 00:53:41.350 align:middle line:90% 00:53:41.350 --> 00:53:43.710 align:middle line:84% And I think that the idea here is 00:53:43.710 --> 00:53:47.030 align:middle line:84% that you almost never essentially 00:53:47.030 --> 00:53:50.190 align:middle line:84% would get color variation due to shape from shading. 00:53:50.190 --> 00:53:53.310 align:middle line:84% And so if there is color variation in an image 00:53:53.310 --> 00:53:55.550 align:middle line:84% and then you get luminance variation, that's 00:53:55.550 --> 00:53:59.270 align:middle line:84% kind of like uncorrelated with that or in this case orthogonal 00:53:59.270 --> 00:54:02.750 align:middle line:84% with that, it's very likely that that luminance variation is not 00:54:02.750 --> 00:54:04.110 align:middle line:90% due to reflectance. 00:54:04.110 --> 00:54:05.990 align:middle line:84% And so you would interpret that as actually 00:54:05.990 --> 00:54:07.570 align:middle line:90% being more likely due to shape. 00:54:07.570 --> 00:54:10.470 align:middle line:84% So the color here is actually kind of enhancing shape 00:54:10.470 --> 00:54:12.488 align:middle line:90% from shading. 00:54:12.488 --> 00:54:14.030 align:middle line:84% And we'll talk a lot more about shape 00:54:14.030 --> 00:54:16.400 align:middle line:84% from shading when we get to that. 00:54:16.400 --> 00:54:21.220 align:middle line:84% This is just another example of color interfacing with shape. 00:54:21.220 --> 00:54:23.540 align:middle line:84% So although there's a bunch of things about color 00:54:23.540 --> 00:54:26.500 align:middle line:84% that are kind of weird in the sense 00:54:26.500 --> 00:54:29.340 align:middle line:84% that isoluminant images don't do a very 00:54:29.340 --> 00:54:31.580 align:middle line:84% good job of supporting shape, you 00:54:31.580 --> 00:54:34.040 align:middle line:84% tend to fill in color based on luminance boundaries, 00:54:34.040 --> 00:54:37.720 align:middle line:84% there are also these complicated interactions with shape as well. 00:54:37.720 --> 00:54:40.260 align:middle line:90% 00:54:40.260 --> 00:54:44.100 align:middle line:84% Last thing we're going to talk about is color blindness, 00:54:44.100 --> 00:54:49.193 align:middle line:84% scientifically usually called color deficiency. 00:54:49.193 --> 00:54:54.620 align:middle line:84% So most color deficiencies are due to one or more cone 00:54:54.620 --> 00:54:58.520 align:middle line:84% types being either missing or abnormal. 00:54:58.520 --> 00:55:00.800 align:middle line:84% So the most common thing is dichromacy. 00:55:00.800 --> 00:55:03.260 align:middle line:84% So that is when one cone is missing. 00:55:03.260 --> 00:55:05.360 align:middle line:84% Because there are three types of cones. 00:55:05.360 --> 00:55:09.140 align:middle line:84% There are three different varieties of dichromacy. 00:55:09.140 --> 00:55:11.260 align:middle line:84% Deuteranopia is the scientific term 00:55:11.260 --> 00:55:12.890 align:middle line:84% for when you're missing the M cone. 00:55:12.890 --> 00:55:15.210 align:middle line:84% Protanopia means you're missing the L cone. 00:55:15.210 --> 00:55:19.130 align:middle line:84% Tritanopia means you're missing the S cone. 00:55:19.130 --> 00:55:21.970 align:middle line:84% Colorblindness is typically diagnosed 00:55:21.970 --> 00:55:24.130 align:middle line:90% with these color plates. 00:55:24.130 --> 00:55:26.170 align:middle line:84% How many people have had a doctor 00:55:26.170 --> 00:55:27.850 align:middle line:90% show these things to them? 00:55:27.850 --> 00:55:32.330 align:middle line:84% I remember my daughter had a checkup when 00:55:32.330 --> 00:55:34.690 align:middle line:84% she was three or four or something over here 00:55:34.690 --> 00:55:35.522 align:middle line:90% at MIT Medical. 00:55:35.522 --> 00:55:37.230 align:middle line:84% And they brought out these colored plates 00:55:37.230 --> 00:55:38.670 align:middle line:84% and they were asking her what the numbers were, 00:55:38.670 --> 00:55:41.350 align:middle line:84% and she didn't know her numbers very well at that point in time. 00:55:41.350 --> 00:55:42.770 align:middle line:84% And so the test depends on the person 00:55:42.770 --> 00:55:43.710 align:middle line:90% saying the correct number. 00:55:43.710 --> 00:55:45.377 align:middle line:84% And so she kept saying the wrong number, 00:55:45.377 --> 00:55:46.910 align:middle line:90% and I felt really embarrassed. 00:55:46.910 --> 00:55:49.270 align:middle line:84% But she's not actually colorblind. 00:55:49.270 --> 00:55:52.170 align:middle line:84% So that was my experience with these color plates. 00:55:52.170 --> 00:55:54.530 align:middle line:84% But that's how you diagnose colorblindness 00:55:54.530 --> 00:55:56.657 align:middle line:84% by introducing differences that somebody 00:55:56.657 --> 00:55:58.990 align:middle line:84% who's missing one of the cones would not be able to see. 00:55:58.990 --> 00:56:03.950 align:middle line:84% There's also anomalous trichromacy. 00:56:03.950 --> 00:56:06.650 align:middle line:84% So that's when one of the cones has a slightly abnormal 00:56:06.650 --> 00:56:12.040 align:middle line:84% spectral sensitivity relative to the normal population. 00:56:12.040 --> 00:56:14.300 align:middle line:84% That typically occurs in the M or the L cones. 00:56:14.300 --> 00:56:18.560 align:middle line:84% Again, deuteranomaly and protanomaly. 00:56:18.560 --> 00:56:21.240 align:middle line:90% And these are genetic in origin. 00:56:21.240 --> 00:56:26.360 align:middle line:84% So most color, they call them defects here or abnormalities, 00:56:26.360 --> 00:56:29.160 align:middle line:84% they involve the M and the L cones. 00:56:29.160 --> 00:56:31.500 align:middle line:84% And those are coded on the X chromosome. 00:56:31.500 --> 00:56:36.960 align:middle line:84% And so they're much more common in men, 00:56:36.960 --> 00:56:38.500 align:middle line:84% and actually pretty common in men. 00:56:38.500 --> 00:56:43.160 align:middle line:84% So about 8% of men have red green color blindness 00:56:43.160 --> 00:56:45.700 align:middle line:84% because of missing the M or the L cones. 00:56:45.700 --> 00:56:49.880 align:middle line:84% You probably know people who are like this. 00:56:49.880 --> 00:56:53.400 align:middle line:84% So these are like the incidence of these different types 00:56:53.400 --> 00:56:55.220 align:middle line:90% of color abnormalities. 00:56:55.220 --> 00:56:58.560 align:middle line:84% And you can see that they're basically all way more 00:56:58.560 --> 00:57:00.740 align:middle line:90% common in males than females. 00:57:00.740 --> 00:57:03.840 align:middle line:90% 00:57:03.840 --> 00:57:06.480 align:middle line:84% So there's a couple other kinds of things that can happen. 00:57:06.480 --> 00:57:09.840 align:middle line:84% So this is very, very rare, but some people 00:57:09.840 --> 00:57:11.780 align:middle line:90% are rod monochromats. 00:57:11.780 --> 00:57:15.360 align:middle line:84% This means they don't have any cones at all. 00:57:15.360 --> 00:57:17.580 align:middle line:84% So as it says here, we're all rod monochromats 00:57:17.580 --> 00:57:18.620 align:middle line:90% at low light levels. 00:57:18.620 --> 00:57:22.140 align:middle line:84% Because remember, at low light levels, the cones don't operate. 00:57:22.140 --> 00:57:24.400 align:middle line:84% So you're just reliant on your rods. 00:57:24.400 --> 00:57:26.400 align:middle line:84% That means that you don't have color vision. 00:57:26.400 --> 00:57:28.360 align:middle line:84% So rod monochromats typically would 00:57:28.360 --> 00:57:31.480 align:middle line:84% have to wear sunglasses because under normal light levels, 00:57:31.480 --> 00:57:33.812 align:middle line:90% the rods would be saturated. 00:57:33.812 --> 00:57:35.520 align:middle line:84% And then as we've discussed, you can also 00:57:35.520 --> 00:57:37.920 align:middle line:90% get cortical color blindness. 00:57:37.920 --> 00:57:39.140 align:middle line:90% So that's achromatopsia. 00:57:39.140 --> 00:57:40.720 align:middle line:84% So that's if there's brain damage 00:57:40.720 --> 00:57:44.560 align:middle line:84% to the cortex that causes you to not be able to see color. 00:57:44.560 --> 00:57:50.680 align:middle line:84% So two very different sources of individual differences in color. 00:57:50.680 --> 00:57:54.860 align:middle line:90% Any questions about color? 00:57:54.860 --> 00:57:58.080 align:middle line:90% 00:57:58.080 --> 00:57:58.853 align:middle line:90% Yeah? 00:57:58.853 --> 00:58:00.520 align:middle line:84% STUDENT: Would people with the first one 00:58:00.520 --> 00:58:02.640 align:middle line:84% see a little bit blurrier and would glasses 00:58:02.640 --> 00:58:05.200 align:middle line:90% be able to fix that? 00:58:05.200 --> 00:58:06.800 align:middle line:84% JOSH MCDERMOTT: Yeah, the resolution 00:58:06.800 --> 00:58:09.110 align:middle line:84% would be worse because the resolution of the rod system 00:58:09.110 --> 00:58:11.310 align:middle line:90% is a lot worse. 00:58:11.310 --> 00:58:15.850 align:middle line:84% No, glasses are not going to help that. 00:58:15.850 --> 00:58:17.870 align:middle line:90% There would not really be-- 00:58:17.870 --> 00:58:21.830 align:middle line:84% I don't know of any intervention that's currently available 00:58:21.830 --> 00:58:25.330 align:middle line:84% that would be able to address that. 00:58:25.330 --> 00:58:25.830 align:middle line:90% Yeah. 00:58:25.830 --> 00:58:30.390 align:middle line:84% I mean, there's a lot of interest in retinal implants. 00:58:30.390 --> 00:58:35.130 align:middle line:84% And that, in principle, might be able to help with that. 00:58:35.130 --> 00:58:39.110 align:middle line:90% 00:58:39.110 --> 00:58:46.130 align:middle line:84% So how many people still see an aftereffect here? 00:58:46.130 --> 00:58:47.190 align:middle line:90% So it lasts a long time. 00:58:47.190 --> 00:58:48.130 align:middle line:90% It's pretty crazy. 00:58:48.130 --> 00:58:50.710 align:middle line:90% So it will eventually go away. 00:58:50.710 --> 00:58:53.350 align:middle line:90% So high level summary. 00:58:53.350 --> 00:58:55.630 align:middle line:84% Color is the perceptual correlate 00:58:55.630 --> 00:58:57.510 align:middle line:90% of selective reflectance. 00:58:57.510 --> 00:59:01.230 align:middle line:84% It is constrained by the three cones. 00:59:01.230 --> 00:59:03.870 align:middle line:84% Metamers result from trichromacy. 00:59:03.870 --> 00:59:06.892 align:middle line:84% So the fact that you take this very high dimensional thing, 00:59:06.892 --> 00:59:09.100 align:middle line:84% the spectrum of the light that's coming into the eye, 00:59:09.100 --> 00:59:13.140 align:middle line:84% and turn that into three numbers at every point in space. 00:59:13.140 --> 00:59:15.100 align:middle line:84% The cones are subsequently combined 00:59:15.100 --> 00:59:18.808 align:middle line:90% to form opponent channels. 00:59:18.808 --> 00:59:20.600 align:middle line:84% We haven't really talked about why that is. 00:59:20.600 --> 00:59:23.540 align:middle line:84% It's mostly just I've treated it as a fact, 00:59:23.540 --> 00:59:25.140 align:middle line:84% and it has consequences for perception 00:59:25.140 --> 00:59:29.140 align:middle line:84% with the opponent relationships between different colors. 00:59:29.140 --> 00:59:31.660 align:middle line:84% Humans have to discount the illuminant 00:59:31.660 --> 00:59:34.300 align:middle line:90% to achieve color constancy. 00:59:34.300 --> 00:59:36.440 align:middle line:84% The mechanisms of color constancy 00:59:36.440 --> 00:59:39.620 align:middle line:84% and a formal computational level understanding, that 00:59:39.620 --> 00:59:41.620 align:middle line:90% remains a work in progress. 00:59:41.620 --> 00:59:45.180 align:middle line:84% It pretty clearly involves priors over illumination 00:59:45.180 --> 00:59:47.080 align:middle line:84% that can differ between individuals, 00:59:47.080 --> 00:59:49.060 align:middle line:90% as we saw with the dress. 00:59:49.060 --> 00:59:51.780 align:middle line:84% And there's some evidence for specialized brain regions 00:59:51.780 --> 00:59:54.300 align:middle line:84% for color processing, although lots 00:59:54.300 --> 00:59:56.860 align:middle line:84% of different pieces of evidence that color kind of interacts 00:59:56.860 --> 01:00:00.980 align:middle line:84% with other parts of vision, shape, and so forth, 01:00:00.980 --> 01:00:04.050 align:middle line:84% as we saw with some demonstrations. 01:00:04.050 --> 01:00:07.570 align:middle line:84% So motion occurs when things in the world 01:00:07.570 --> 01:00:09.230 align:middle line:90% change position over time. 01:00:09.230 --> 01:00:11.810 align:middle line:84% And today we're going to talk about how the visual system 01:00:11.810 --> 01:00:14.130 align:middle line:90% detects this change. 01:00:14.130 --> 01:00:15.470 align:middle line:90% So here's an example. 01:00:15.470 --> 01:00:18.490 align:middle line:84% So this is a square of random dots. 01:00:18.490 --> 01:00:21.250 align:middle line:84% And there is a square of dots in the center that 01:00:21.250 --> 01:00:26.490 align:middle line:84% will move left or right between frame one and frame two. 01:00:26.490 --> 01:00:30.410 align:middle line:84% And it's very easy for you to see that motion. 01:00:30.410 --> 01:00:35.910 align:middle line:84% And when it stops, you lose the sense of the square. 01:00:35.910 --> 01:00:39.290 align:middle line:84% So people are very sensitive to when things move. 01:00:39.290 --> 01:00:42.450 align:middle line:84% We believe that there are neurons pretty early 01:00:42.450 --> 01:00:45.230 align:middle line:84% in the visual system that are selective for motion. 01:00:45.230 --> 01:00:47.770 align:middle line:84% And as we've just been discussing, 01:00:47.770 --> 01:00:52.068 align:middle line:84% one way that we can probe for the presence of neurons that 01:00:52.068 --> 01:00:53.610 align:middle line:84% are selective for a stimulus property 01:00:53.610 --> 01:00:56.670 align:middle line:84% is to test for aftereffects from adaptation. 01:00:56.670 --> 01:00:58.470 align:middle line:90% So we're going to do this here. 01:00:58.470 --> 01:01:05.170 align:middle line:84% So there's a link here for an opportunity for you 01:01:05.170 --> 01:01:06.810 align:middle line:90% to try this out yourself. 01:01:06.810 --> 01:01:09.090 align:middle line:84% We're going to attempt to do this live. 01:01:09.090 --> 01:01:10.850 align:middle line:90% This is an umbrella. 01:01:10.850 --> 01:01:12.810 align:middle line:90% It's a really old umbrella. 01:01:12.810 --> 01:01:17.810 align:middle line:84% And it was painted a very long time ago with a white spiral. 01:01:17.810 --> 01:01:22.010 align:middle line:84% Unfortunately, the paint has slowly deteriorated over time. 01:01:22.010 --> 01:01:23.690 align:middle line:84% But what I'm going to do here is I'm 01:01:23.690 --> 01:01:26.890 align:middle line:84% going to rotate this for about a minute. 01:01:26.890 --> 01:01:29.050 align:middle line:84% And you all have to stare at the center. 01:01:29.050 --> 01:01:31.630 align:middle line:84% And so you're going to adapt yourself. 01:01:31.630 --> 01:01:33.770 align:middle line:84% So just start doing this, and hopefully 01:01:33.770 --> 01:01:37.090 align:middle line:90% you'll get a sense of-- 01:01:37.090 --> 01:01:39.010 align:middle line:84% I don't know if it's expanding or contracting. 01:01:39.010 --> 01:01:39.950 align:middle line:90% One of the two. 01:01:39.950 --> 01:01:44.950 align:middle line:84% So we're just going to do this for about a minute. 01:01:44.950 --> 01:01:47.030 align:middle line:84% So just stare at the center here. 01:01:47.030 --> 01:01:50.230 align:middle line:84% And this is going to do some motion aftereffect. 01:01:50.230 --> 01:01:54.090 align:middle line:84% The motion aftereffect is one of the most powerful aftereffects 01:01:54.090 --> 01:01:57.850 align:middle line:90% in all of perception. 01:01:57.850 --> 01:02:00.185 align:middle line:90% So keep looking at this. 01:02:00.185 --> 01:02:02.560 align:middle line:84% And hopefully all those little imperfections in the paint 01:02:02.560 --> 01:02:06.600 align:middle line:84% are not going to totally mess this up, but I don't know. 01:02:06.600 --> 01:02:09.040 align:middle line:90% Just keep staring at the middle. 01:02:09.040 --> 01:02:11.560 align:middle line:90% You're getting sleepy. 01:02:11.560 --> 01:02:14.640 align:middle line:90% Keep staring at the middle. 01:02:14.640 --> 01:02:17.660 align:middle line:84% It's kind of hard to turn this thing very consistently, 01:02:17.660 --> 01:02:20.138 align:middle line:90% but I'm doing my best. 01:02:20.138 --> 01:02:22.180 align:middle line:84% We're going to do this for about 10 more seconds. 01:02:22.180 --> 01:02:25.120 align:middle line:84% And then what you're going to do is 01:02:25.120 --> 01:02:29.580 align:middle line:90% you're going to look at my face. 01:02:29.580 --> 01:02:32.760 align:middle line:84% And I'm not going to look the same. 01:02:32.760 --> 01:02:34.060 align:middle line:90% Look at me. 01:02:34.060 --> 01:02:37.230 align:middle line:90% 01:02:37.230 --> 01:02:41.380 align:middle line:90% Did my face distort? 01:02:41.380 --> 01:02:45.600 align:middle line:84% You can also look at the back of your hand. 01:02:45.600 --> 01:02:47.540 align:middle line:84% This effect doesn't last that long. 01:02:47.540 --> 01:02:52.000 align:middle line:84% This goes away on the time scale of normal visual aftereffects. 01:02:52.000 --> 01:02:54.160 align:middle line:90% You just missed a great demo. 01:02:54.160 --> 01:02:55.560 align:middle line:90% It's OK. 01:02:55.560 --> 01:02:57.440 align:middle line:84% So that's the motion aftereffect. 01:02:57.440 --> 01:03:00.163 align:middle line:84% And you can get little tops that have 01:03:00.163 --> 01:03:01.330 align:middle line:90% these spiral things on them. 01:03:01.330 --> 01:03:02.913 align:middle line:84% So you spin the top and you look at it 01:03:02.913 --> 01:03:05.970 align:middle line:84% and then you can get a really nice aftereffect. 01:03:05.970 --> 01:03:09.750 align:middle line:84% So we typically think of these aftereffects 01:03:09.750 --> 01:03:11.250 align:middle line:90% in terms of population codes. 01:03:11.250 --> 01:03:12.625 align:middle line:84% So remember the tilt aftereffect. 01:03:12.625 --> 01:03:14.750 align:middle line:84% The idea that you've got this population of neurons 01:03:14.750 --> 01:03:16.290 align:middle line:84% with different orientation tuning. 01:03:16.290 --> 01:03:17.150 align:middle line:90% You adapt. 01:03:17.150 --> 01:03:19.490 align:middle line:84% The gain of some subset of neurons is reduced, 01:03:19.490 --> 01:03:21.930 align:middle line:84% and thus the inference about the orientation is altered. 01:03:21.930 --> 01:03:23.430 align:middle line:84% So the same thing applies to motion. 01:03:23.430 --> 01:03:25.847 align:middle line:84% The idea is that you have a population of neurons that are 01:03:25.847 --> 01:03:27.250 align:middle line:90% tuned to different directions. 01:03:27.250 --> 01:03:29.250 align:middle line:84% You adapt to a particular direction. 01:03:29.250 --> 01:03:32.810 align:middle line:84% That subset of the population has a reduced response, 01:03:32.810 --> 01:03:35.270 align:middle line:84% and then the inference about the motion is altered. 01:03:35.270 --> 01:03:38.050 align:middle line:84% So you can think through the logic there. 01:03:38.050 --> 01:03:40.750 align:middle line:84% So we got evidence that there are these motion detectors 01:03:40.750 --> 01:03:42.210 align:middle line:90% early in the visual system. 01:03:42.210 --> 01:03:45.190 align:middle line:84% One way to think about detecting image motion 01:03:45.190 --> 01:03:47.230 align:middle line:90% is the Reichardt detector. 01:03:47.230 --> 01:03:48.910 align:middle line:84% So you can construct something that 01:03:48.910 --> 01:03:50.830 align:middle line:84% would be responsive to image motion 01:03:50.830 --> 01:03:56.300 align:middle line:84% by wiring together two neurons that have spatially displaced 01:03:56.300 --> 01:03:58.940 align:middle line:84% receptive fields here and here with a time delay 01:03:58.940 --> 01:04:00.620 align:middle line:90% for one of them. 01:04:00.620 --> 01:04:03.100 align:middle line:84% And so the consequence of the time delay 01:04:03.100 --> 01:04:06.620 align:middle line:84% is that the input from this receptive field and the input 01:04:06.620 --> 01:04:09.540 align:middle line:84% from this receptive field would reach 01:04:09.540 --> 01:04:13.810 align:middle line:84% this at the same time for one direction and not for the other. 01:04:13.810 --> 01:04:16.060 align:middle line:84% So that's the essence of the idea behind the Reichardt 01:04:16.060 --> 01:04:16.560 align:middle line:90% detector. 01:04:16.560 --> 01:04:20.060 align:middle line:84% And if you suppose that the downstream neuron was 01:04:20.060 --> 01:04:22.700 align:middle line:84% kind of performing an and function or a coincidence 01:04:22.700 --> 01:04:25.100 align:middle line:84% detector, that's something that neurons often do, 01:04:25.100 --> 01:04:28.340 align:middle line:84% you could get something that would be responsive to motion. 01:04:28.340 --> 01:04:32.340 align:middle line:84% So you could also take a neuron that is direction selective 01:04:32.340 --> 01:04:34.780 align:middle line:84% and actually look at the receptive field. 01:04:34.780 --> 01:04:38.480 align:middle line:84% So remember the spike triggered average. 01:04:38.480 --> 01:04:43.340 align:middle line:84% This is the stimulus that, on average, precedes the spike. 01:04:43.340 --> 01:04:49.940 align:middle line:84% So we can take some random movie, play it to a neuron, 01:04:49.940 --> 01:04:51.980 align:middle line:84% record the frames, the sequence of frames 01:04:51.980 --> 01:04:54.160 align:middle line:84% that precedes the spike and then average them, 01:04:54.160 --> 01:04:56.420 align:middle line:90% and that gives us a movie. 01:04:56.420 --> 01:04:59.420 align:middle line:84% So it's the movie that was the average thing that 01:04:59.420 --> 01:05:01.600 align:middle line:84% preceded an action potential by the neuron. 01:05:01.600 --> 01:05:04.420 align:middle line:84% So it's one way to think about the receptive field. 01:05:04.420 --> 01:05:07.940 align:middle line:84% And so these are the frames of the movie. 01:05:07.940 --> 01:05:10.540 align:middle line:84% 50 milliseconds, 100 milliseconds, 150 milliseconds, 01:05:10.540 --> 01:05:13.460 align:middle line:84% and 200 milliseconds before the spike. 01:05:13.460 --> 01:05:19.180 align:middle line:84% And if you look closely, you see this orientation selective thing 01:05:19.180 --> 01:05:21.060 align:middle line:84% at every frame of the movie, but you 01:05:21.060 --> 01:05:23.580 align:middle line:84% can see that the phase is different 01:05:23.580 --> 01:05:25.280 align:middle line:90% in the different frames. 01:05:25.280 --> 01:05:30.052 align:middle line:90% 01:05:30.052 --> 01:05:33.460 align:middle line:84% So the exact spatial pattern that the thing 01:05:33.460 --> 01:05:37.680 align:middle line:84% is most responsive to at a given point in time changes over time. 01:05:37.680 --> 01:05:40.020 align:middle line:84% And so this thing, if you played it, 01:05:40.020 --> 01:05:42.340 align:middle line:84% would actually look like a stimulus that was moving. 01:05:42.340 --> 01:05:46.180 align:middle line:90% 01:05:46.180 --> 01:05:50.340 align:middle line:84% So another way to think about motion detection 01:05:50.340 --> 01:05:56.450 align:middle line:84% is that motion is like orientation in space time. 01:05:56.450 --> 01:05:58.890 align:middle line:84% And spatiotemporally oriented filters 01:05:58.890 --> 01:06:01.105 align:middle line:84% can be used to detect and measure it. 01:06:01.105 --> 01:06:02.730 align:middle line:84% So that's kind of weird to think about, 01:06:02.730 --> 01:06:04.355 align:middle line:84% because we haven't really been thinking 01:06:04.355 --> 01:06:05.570 align:middle line:90% about filters in that way. 01:06:05.570 --> 01:06:08.610 align:middle line:84% But time is just another dimension. 01:06:08.610 --> 01:06:12.770 align:middle line:84% And if you've got something that is moving through space, so 01:06:12.770 --> 01:06:16.510 align:middle line:84% in this case, this is one frame of a movie. 01:06:16.510 --> 01:06:18.945 align:middle line:90% So we just have a vertical bar. 01:06:18.945 --> 01:06:20.570 align:middle line:84% So this is the x dimension of the image 01:06:20.570 --> 01:06:22.690 align:middle line:84% and the y dimension of the image. 01:06:22.690 --> 01:06:24.430 align:middle line:90% And now we introduce time. 01:06:24.430 --> 01:06:26.730 align:middle line:84% And if we let that bar translate, 01:06:26.730 --> 01:06:29.930 align:middle line:84% in this case in the x direction over time, 01:06:29.930 --> 01:06:34.850 align:middle line:84% we get this thing that is oriented in space time. 01:06:34.850 --> 01:06:38.670 align:middle line:84% So motion is orientation in space time. 01:06:38.670 --> 01:06:41.490 align:middle line:84% And so if we look at the spike triggered average, 01:06:41.490 --> 01:06:42.870 align:middle line:90% we can actually see this. 01:06:42.870 --> 01:06:45.090 align:middle line:90% So again, this is a movie. 01:06:45.090 --> 01:06:46.070 align:middle line:90% It's three dimensions. 01:06:46.070 --> 01:06:49.210 align:middle line:84% There's the x dimension, the y dimension, and the t dimension. 01:06:49.210 --> 01:06:52.520 align:middle line:84% So now we're just projecting down onto the xt plane. 01:06:52.520 --> 01:06:55.500 align:middle line:90% And we see this oriented thing. 01:06:55.500 --> 01:06:57.740 align:middle line:84% So here it is rotated to make it a little bit easier. 01:06:57.740 --> 01:06:59.380 align:middle line:90% So orientation here. 01:06:59.380 --> 01:07:03.900 align:middle line:90% 01:07:03.900 --> 01:07:06.360 align:middle line:84% So two different ways to think about motion detection. 01:07:06.360 --> 01:07:09.880 align:middle line:84% The more of implementation level explanation 01:07:09.880 --> 01:07:12.120 align:middle line:84% is this Reichardt detector kind of thing 01:07:12.120 --> 01:07:13.880 align:middle line:84% where you can imagine if there was a way 01:07:13.880 --> 01:07:18.640 align:middle line:84% to put in a delay in the connection between one 01:07:18.640 --> 01:07:19.900 align:middle line:90% neuron and another. 01:07:19.900 --> 01:07:23.208 align:middle line:84% You could kind of set something up like this. 01:07:23.208 --> 01:07:25.000 align:middle line:84% It's maybe a little bit less obvious to you 01:07:25.000 --> 01:07:27.720 align:middle line:84% how you would create something that would do this, 01:07:27.720 --> 01:07:31.400 align:middle line:84% but this is a more computational level description of what 01:07:31.400 --> 01:07:33.140 align:middle line:90% the detector ought to be doing. 01:07:33.140 --> 01:07:38.280 align:middle line:84% And if you actually look at this filter kind of characterization 01:07:38.280 --> 01:07:40.760 align:middle line:84% of the receptive fields, you see these things that 01:07:40.760 --> 01:07:43.540 align:middle line:90% are oriented in space and time. 01:07:43.540 --> 01:07:46.423 align:middle line:90% 01:07:46.423 --> 01:07:48.840 align:middle line:84% So there's now lots of evidence that you find these things 01:07:48.840 --> 01:07:51.500 align:middle line:84% in early stages of the visual system, in particular V1. 01:07:51.500 --> 01:07:54.920 align:middle line:84% So one of the problems that you run into 01:07:54.920 --> 01:07:58.240 align:middle line:84% is that the local motion detectors 01:07:58.240 --> 01:08:03.040 align:middle line:84% that you find early in the visual system on their own 01:08:03.040 --> 01:08:05.920 align:middle line:84% can't determine the true direction of motion. 01:08:05.920 --> 01:08:07.960 align:middle line:84% And the essential idea is shown here. 01:08:07.960 --> 01:08:12.120 align:middle line:84% So the motion detector is generated 01:08:12.120 --> 01:08:15.520 align:middle line:84% with these oriented inputs, as we saw in the previous slide. 01:08:15.520 --> 01:08:17.520 align:middle line:84% So if you have two of them that are 01:08:17.520 --> 01:08:20.080 align:middle line:84% at different spatial positions, one with a time delay, 01:08:20.080 --> 01:08:23.080 align:middle line:84% you wire them together, you get your motion detector. 01:08:23.080 --> 01:08:26.840 align:middle line:84% But these two stimuli will both elicit the same response 01:08:26.840 --> 01:08:29.600 align:middle line:84% from the neuron, because you end up 01:08:29.600 --> 01:08:33.880 align:middle line:84% with this stimulus that is overlapping the excitatory lobe 01:08:33.880 --> 01:08:35.800 align:middle line:90% here at this point. 01:08:35.800 --> 01:08:37.560 align:middle line:90% And then it moves over here. 01:08:37.560 --> 01:08:40.120 align:middle line:84% And if it reaches this with the correct time offset, 01:08:40.120 --> 01:08:42.399 align:middle line:90% you're going to get a response. 01:08:42.399 --> 01:08:44.760 align:middle line:84% But this is a different direction of motion 01:08:44.760 --> 01:08:46.359 align:middle line:84% that would produce a similar response, 01:08:46.359 --> 01:08:49.550 align:middle line:84% because it's got the correct spatial offset 01:08:49.550 --> 01:08:51.109 align:middle line:90% in the right direction. 01:08:51.109 --> 01:08:53.029 align:middle line:84% So two different directions of motion. 01:08:53.029 --> 01:08:58.189 align:middle line:84% They're both going to elicit a response in this neuron. 01:08:58.189 --> 01:09:03.149 align:middle line:84% So motion detectors that have these oriented inputs are not 01:09:03.149 --> 01:09:06.590 align:middle line:84% able to determine the specific motion direction and speed 01:09:06.590 --> 01:09:08.010 align:middle line:90% of a 2D signal. 01:09:08.010 --> 01:09:10.950 align:middle line:84% There's a sense in which they can only see the component 01:09:10.950 --> 01:09:13.590 align:middle line:90% at their orientation. 01:09:13.590 --> 01:09:16.430 align:middle line:84% So this raises this question of how then does the visual system 01:09:16.430 --> 01:09:21.270 align:middle line:84% detect the motion of 2D features that are two dimensional? 01:09:21.270 --> 01:09:23.310 align:middle line:84% And so one kind of important idea 01:09:23.310 --> 01:09:27.189 align:middle line:84% is that although the responses of one of these local motion 01:09:27.189 --> 01:09:30.415 align:middle line:84% detectors are ambiguous, they're nonetheless 01:09:30.415 --> 01:09:32.290 align:middle line:84% giving you a lot of information about motion. 01:09:32.290 --> 01:09:35.710 align:middle line:84% And in particular, they're giving you a constraint line 01:09:35.710 --> 01:09:37.090 align:middle line:90% in velocity space. 01:09:37.090 --> 01:09:40.250 align:middle line:84% So if we think about the stimulus in velocity space, 01:09:40.250 --> 01:09:42.149 align:middle line:84% so this is the x component of the velocity 01:09:42.149 --> 01:09:46.779 align:middle line:84% and the y component of the velocity, 01:09:46.779 --> 01:09:49.540 align:middle line:84% a response in one of these detectors 01:09:49.540 --> 01:09:54.040 align:middle line:84% tells you something about a component of the velocity. 01:09:54.040 --> 01:09:56.940 align:middle line:84% And it leaves the other component kind of unconstrained. 01:09:56.940 --> 01:09:59.500 align:middle line:84% But that one component gives you a line 01:09:59.500 --> 01:10:04.260 align:middle line:84% in the space of velocity, which is all the velocities that 01:10:04.260 --> 01:10:07.260 align:middle line:84% could have caused that detector to respond. 01:10:07.260 --> 01:10:09.980 align:middle line:84% And it just kind comes back to this idea that really 01:10:09.980 --> 01:10:13.760 align:middle line:84% what matters is the spatial offset in one direction. 01:10:13.760 --> 01:10:17.420 align:middle line:84% This direction doesn't matter for the response of the neuron. 01:10:17.420 --> 01:10:20.060 align:middle line:84% And so there's this line in the space of velocities, which 01:10:20.060 --> 01:10:22.980 align:middle line:84% is caused by all the different y components that you could add 01:10:22.980 --> 01:10:24.960 align:middle line:90% to a particular x component. 01:10:24.960 --> 01:10:27.880 align:middle line:90% 01:10:27.880 --> 01:10:31.260 align:middle line:84% So any single detector with a particular orientation 01:10:31.260 --> 01:10:36.220 align:middle line:84% in its input gives a constraint line in velocity space. 01:10:36.220 --> 01:10:39.260 align:middle line:84% And so it follows from that if there 01:10:39.260 --> 01:10:44.250 align:middle line:84% were two different detectors that were both responding, 01:10:44.250 --> 01:10:46.350 align:middle line:84% they would each give you a constraint line. 01:10:46.350 --> 01:10:48.910 align:middle line:84% And if you combined those two constraint lines, 01:10:48.910 --> 01:10:52.770 align:middle line:84% the velocity could be determined from their intersection. 01:10:52.770 --> 01:10:56.663 align:middle line:84% So the idea is that if two detectors are responding, 01:10:56.663 --> 01:10:58.330 align:middle line:84% one detector is telling you the stimulus 01:10:58.330 --> 01:10:59.310 align:middle line:90% is somewhere on this line. 01:10:59.310 --> 01:11:00.170 align:middle line:84% The other detector is telling you 01:11:00.170 --> 01:11:02.170 align:middle line:84% the stimulus is somewhere on this line. 01:11:02.170 --> 01:11:04.450 align:middle line:84% There's only one point that's consistent with both, 01:11:04.450 --> 01:11:06.890 align:middle line:84% and that would yield the velocity. 01:11:06.890 --> 01:11:09.770 align:middle line:84% So this is called the intersection of constraints 01:11:09.770 --> 01:11:12.810 align:middle line:90% model of motion detection. 01:11:12.810 --> 01:11:15.570 align:middle line:84% This was popularized by two people back in the early 01:11:15.570 --> 01:11:19.930 align:middle line:84% '80s, Ted Adelson and Tony Movshon. 01:11:19.930 --> 01:11:21.450 align:middle line:84% And so one way to think about this 01:11:21.450 --> 01:11:26.530 align:middle line:84% in terms of receptive fields and in terms of spatial frequency 01:11:26.530 --> 01:11:29.010 align:middle line:84% components is remember how we talked 01:11:29.010 --> 01:11:33.010 align:middle line:84% about how every image can be represented 01:11:33.010 --> 01:11:34.970 align:middle line:90% as a sum of sine wave gratings? 01:11:34.970 --> 01:11:37.610 align:middle line:90% This is just Fourier's theorem. 01:11:37.610 --> 01:11:42.480 align:middle line:84% So every translating pattern can be decomposed as a sum of 1D 01:11:42.480 --> 01:11:43.620 align:middle line:90% sinusoidal components. 01:11:43.620 --> 01:11:46.880 align:middle line:84% So you got that random 2D pattern. 01:11:46.880 --> 01:11:49.400 align:middle line:84% You could generate that by adding together a whole bunch 01:11:49.400 --> 01:11:52.200 align:middle line:90% of different 1D components. 01:11:52.200 --> 01:11:57.000 align:middle line:84% And you can think of any neuron in V1 01:11:57.000 --> 01:11:59.140 align:middle line:84% as kind of tuned for one of those components. 01:11:59.140 --> 01:12:01.900 align:middle line:84% It's tuned to an orientation and a spatial frequency. 01:12:01.900 --> 01:12:04.920 align:middle line:84% And so if you wanted to build a detector for a particular 2D 01:12:04.920 --> 01:12:08.640 align:middle line:84% velocity, all you really need to do is to add up the responses 01:12:08.640 --> 01:12:13.280 align:middle line:84% of the V1 neurons whose preferred orientation and speed, 01:12:13.280 --> 01:12:16.120 align:middle line:84% namely a constraint line, is consistent with that 2D 01:12:16.120 --> 01:12:17.120 align:middle line:90% velocity. 01:12:17.120 --> 01:12:18.840 align:middle line:84% So let's suppose we wanted to build 01:12:18.840 --> 01:12:22.360 align:middle line:84% a detector for something that's moving right with this speed. 01:12:22.360 --> 01:12:25.320 align:middle line:84% So then we just take all of these different constraint lines 01:12:25.320 --> 01:12:26.900 align:middle line:90% that intersect this point. 01:12:26.900 --> 01:12:29.160 align:middle line:84% And each one of those corresponds 01:12:29.160 --> 01:12:32.520 align:middle line:84% to a particular elementary motion detector. 01:12:32.520 --> 01:12:34.840 align:middle line:84% And we just have those kind of converge and project 01:12:34.840 --> 01:12:35.900 align:middle line:90% to the same neuron. 01:12:35.900 --> 01:12:39.560 align:middle line:90% 01:12:39.560 --> 01:12:43.240 align:middle line:84% So this was a very influential idea 01:12:43.240 --> 01:12:47.000 align:middle line:84% that suggested that motion processing might 01:12:47.000 --> 01:12:48.920 align:middle line:90% involve two stages. 01:12:48.920 --> 01:12:52.760 align:middle line:84% So there's this initial stage where the stimulus is decomposed 01:12:52.760 --> 01:12:55.040 align:middle line:90% into a set of 1D components. 01:12:55.040 --> 01:12:58.800 align:middle line:84% So you can think of that as what happens in V1 where you get 01:12:58.800 --> 01:13:01.400 align:middle line:90% Gabor-like receptive fields. 01:13:01.400 --> 01:13:05.800 align:middle line:84% And then a second stage where these 1D components are combined 01:13:05.800 --> 01:13:08.240 align:middle line:84% using a form of these intersection of the intersection 01:13:08.240 --> 01:13:09.880 align:middle line:90% of constraints algorithm. 01:13:09.880 --> 01:13:13.535 align:middle line:84% So this was initially something that was a theoretical idea. 01:13:13.535 --> 01:13:15.660 align:middle line:84% There was then some psychophysical evidence for it, 01:13:15.660 --> 01:13:19.280 align:middle line:84% which we'll discuss, and then some physiology experiments 01:13:19.280 --> 01:13:21.740 align:middle line:84% that provided some confirmation that this occurs. 01:13:21.740 --> 01:13:24.000 align:middle line:84% And this was very influential because it 01:13:24.000 --> 01:13:27.840 align:middle line:84% was a case where there was this theoretical idea that there 01:13:27.840 --> 01:13:30.265 align:middle line:84% might be these two stages of processing. 01:13:30.265 --> 01:13:32.140 align:middle line:84% And then people went and looked in the brain, 01:13:32.140 --> 01:13:33.560 align:middle line:84% and those two stages of processing 01:13:33.560 --> 01:13:35.600 align:middle line:84% seem to map onto two different visual areas 01:13:35.600 --> 01:13:37.310 align:middle line:90% in a relatively clean way. 01:13:37.310 --> 01:13:42.470 align:middle line:84% So this was a very impactful, early demonstration 01:13:42.470 --> 01:13:46.470 align:middle line:84% of how computational stages might be associated with stages 01:13:46.470 --> 01:13:49.750 align:middle line:90% of the visual system. 01:13:49.750 --> 01:13:53.350 align:middle line:84% And the story here involves an area called MT. 01:13:53.350 --> 01:13:58.470 align:middle line:84% So MT is a visual area that's midway up the dorsal or parietal 01:13:58.470 --> 01:13:59.010 align:middle line:90% pathway. 01:13:59.010 --> 01:14:01.310 align:middle line:84% Remember, we can think of the visual system 01:14:01.310 --> 01:14:04.550 align:middle line:84% is kind of coarsely structured into these two pathways, one 01:14:04.550 --> 01:14:06.710 align:middle line:84% that kind of starts out with the midget cells 01:14:06.710 --> 01:14:09.030 align:middle line:84% and then the parvocellular layers of the LGN 01:14:09.030 --> 01:14:12.630 align:middle line:84% all the way up to the inferotemporal cortex. 01:14:12.630 --> 01:14:14.510 align:middle line:84% That's often called the ventral stream. 01:14:14.510 --> 01:14:16.310 align:middle line:84% And then the dorsal stream predominantly 01:14:16.310 --> 01:14:19.470 align:middle line:84% gets input from the parasol cells and the magnocellular 01:14:19.470 --> 01:14:22.110 align:middle line:90% layers of the LGN. 01:14:22.110 --> 01:14:24.630 align:middle line:84% And there are these distinct subregions of V1 and V2 01:14:24.630 --> 01:14:27.570 align:middle line:90% that then project to area MT. 01:14:27.570 --> 01:14:30.470 align:middle line:84% And this is where all this stuff is in the brain. 01:14:30.470 --> 01:14:34.510 align:middle line:84% So MT is very noteworthy in that the responses are 01:14:34.510 --> 01:14:35.570 align:middle line:90% very distinctive. 01:14:35.570 --> 01:14:38.650 align:middle line:84% So most of the neurons in this particular visual area 01:14:38.650 --> 01:14:40.400 align:middle line:84% are selective for the direction of motion. 01:14:40.400 --> 01:14:43.380 align:middle line:84% So this is a graph that shows the proportion of neurons 01:14:43.380 --> 01:14:46.780 align:middle line:84% that are selective for different stimulus attributes. 01:14:46.780 --> 01:14:48.100 align:middle line:90% So there's one bar for MT. 01:14:48.100 --> 01:14:50.180 align:middle line:84% There's another bar for area V4, which 01:14:50.180 --> 01:14:52.420 align:middle line:84% is at the same level of hierarchy 01:14:52.420 --> 01:14:53.600 align:middle line:90% but in the ventral stream. 01:14:53.600 --> 01:14:55.720 align:middle line:84% So those two regions are often compared. 01:14:55.720 --> 01:14:58.460 align:middle line:84% And the point is that the black bar here, which is for MT, 01:14:58.460 --> 01:15:00.720 align:middle line:90% is 95% or something. 01:15:00.720 --> 01:15:03.220 align:middle line:84% So almost all the neurons are selective for the direction 01:15:03.220 --> 01:15:04.140 align:middle line:90% of motion. 01:15:04.140 --> 01:15:07.240 align:middle line:84% By contrast, almost none are selective for color. 01:15:07.240 --> 01:15:09.440 align:middle line:84% And so with V4, you see the opposite. 01:15:09.440 --> 01:15:11.200 align:middle line:90% So some segregation of function. 01:15:11.200 --> 01:15:13.940 align:middle line:84% It's also the case that if you lesion MT, like 01:15:13.940 --> 01:15:19.080 align:middle line:84% in, say, a monkey, that produces deficits in motion perception. 01:15:19.080 --> 01:15:21.380 align:middle line:84% So the basic idea of these experiments 01:15:21.380 --> 01:15:24.860 align:middle line:84% that I'll tell you about are to probe 01:15:24.860 --> 01:15:29.860 align:middle line:84% for this type of processing using stimuli that are composed 01:15:29.860 --> 01:15:32.460 align:middle line:90% of two sine wave gratings. 01:15:32.460 --> 01:15:35.350 align:middle line:84% And these are stimuli that came to be known as plaids. 01:15:35.350 --> 01:15:37.330 align:middle line:84% And so the idea is that each sine wave 01:15:37.330 --> 01:15:40.690 align:middle line:84% grating will stimulate a different elementary motion 01:15:40.690 --> 01:15:41.820 align:middle line:90% detector. 01:15:41.820 --> 01:15:43.570 align:middle line:84% And so this is what the sine wave gratings 01:15:43.570 --> 01:15:45.330 align:middle line:90% look like on their own. 01:15:45.330 --> 01:15:48.130 align:middle line:84% So these are two different gratings, one vertical, one 01:15:48.130 --> 01:15:48.670 align:middle line:90% horizontal. 01:15:48.670 --> 01:15:49.530 align:middle line:90% They're drifting. 01:15:49.530 --> 01:15:50.790 align:middle line:90% It's not very exciting right. 01:15:50.790 --> 01:15:52.548 align:middle line:90% They just move. 01:15:52.548 --> 01:15:54.590 align:middle line:84% But we're going to add these two things together, 01:15:54.590 --> 01:15:56.930 align:middle line:90% and that creates the plaid. 01:15:56.930 --> 01:15:59.050 align:middle line:84% And so the first interesting observation 01:15:59.050 --> 01:16:06.450 align:middle line:84% is that even though we think that this stimulus, when it's 01:16:06.450 --> 01:16:10.330 align:middle line:84% represented in V1, different populations of neurons 01:16:10.330 --> 01:16:12.370 align:middle line:84% will be picking out this component 01:16:12.370 --> 01:16:13.950 align:middle line:90% and picking out this component. 01:16:13.950 --> 01:16:15.825 align:middle line:84% That's what those receptive fields are doing. 01:16:15.825 --> 01:16:19.790 align:middle line:84% They're orientation selective and tuned for spatial frequency. 01:16:19.790 --> 01:16:22.170 align:middle line:84% So the stimulus is going to get decomposed 01:16:22.170 --> 01:16:24.890 align:middle line:84% into these different components, in the same way that the cochlea 01:16:24.890 --> 01:16:26.930 align:middle line:84% takes some sound that's got lots of frequencies 01:16:26.930 --> 01:16:29.090 align:middle line:84% and decomposes it into different frequencies. 01:16:29.090 --> 01:16:31.370 align:middle line:84% So we think that in the early visual system, 01:16:31.370 --> 01:16:32.670 align:middle line:90% this thing gets decomposed. 01:16:32.670 --> 01:16:36.993 align:middle line:84% But when you look at it, you'll see a single thing 01:16:36.993 --> 01:16:39.410 align:middle line:84% that's kind of moving in a different direction than either 01:16:39.410 --> 01:16:40.243 align:middle line:90% of the two gratings. 01:16:40.243 --> 01:16:43.370 align:middle line:84% So drifting downwards and to the left. 01:16:43.370 --> 01:16:44.310 align:middle line:90% So that's a plaid. 01:16:44.310 --> 01:16:46.807 align:middle line:84% So that suggests that even though early 01:16:46.807 --> 01:16:48.890 align:middle line:84% in your visual system, this thing gets taken apart 01:16:48.890 --> 01:16:51.610 align:middle line:84% into these different components, those things are then 01:16:51.610 --> 01:16:54.010 align:middle line:84% reassembled or combined in some way 01:16:54.010 --> 01:16:56.650 align:middle line:84% to infer the direction of motion in the pattern. 01:16:56.650 --> 01:16:58.650 align:middle line:84% And so when we come back, we're going 01:16:58.650 --> 01:17:00.650 align:middle line:84% to talk about physiological evidence 01:17:00.650 --> 01:17:04.210 align:middle line:90% for this happening in area MT. 01:17:04.210 --> 01:17:04.710 align:middle line:90% All right. 01:17:04.710 --> 01:17:06.640 align:middle line:90% Let's break there. 01:17:06.640 --> 01:17:31.000 align:middle line:90%