WEBVTT 00:00:00.000 --> 00:00:10.520 align:middle line:84% [SQUEAKING] [RUSTLING] [CLICKING] 00:00:10.520 --> 00:00:12.143 align:middle line:84% JOSH MCDERMOTT: We got sound working. 00:00:12.143 --> 00:00:14.560 align:middle line:84% And so I'm going to show you a couple of these movies that 00:00:14.560 --> 00:00:16.800 align:middle line:90% didn't quite work last time. 00:00:16.800 --> 00:00:18.920 align:middle line:84% This first one is a demonstration 00:00:18.920 --> 00:00:22.240 align:middle line:84% of a center-surround receptive field in the retina. 00:00:22.240 --> 00:00:25.160 align:middle line:84% So the movie is made in an experiment where 00:00:25.160 --> 00:00:27.440 align:middle line:84% there's an electrode that is recording 00:00:27.440 --> 00:00:29.480 align:middle line:90% from a retinal ganglion cell. 00:00:29.480 --> 00:00:31.600 align:middle line:84% The eye is pointed towards a screen. 00:00:31.600 --> 00:00:33.360 align:middle line:90% This is a video of the screen. 00:00:33.360 --> 00:00:35.440 align:middle line:84% And onto the screen, they project light. 00:00:35.440 --> 00:00:39.280 align:middle line:84% And they will investigate the effect of the pattern of light 00:00:39.280 --> 00:00:40.910 align:middle line:90% on the response of the neuron. 00:00:40.910 --> 00:00:43.160 align:middle line:84% So let's see if we can get a sense of what's going on. 00:00:43.160 --> 00:00:45.920 align:middle line:84% Oh, and the other thing that I should say 00:00:45.920 --> 00:00:49.360 align:middle line:84% is, as is fairly conventional with this kind of thing, 00:00:49.360 --> 00:00:52.600 align:middle line:84% the output of the electrode is being fed 00:00:52.600 --> 00:00:54.047 align:middle line:90% to an amplifier and a speaker. 00:00:54.047 --> 00:00:55.380 align:middle line:90% So you're going to listen to it. 00:00:55.380 --> 00:00:57.520 align:middle line:84% So every time there's an action potential, 00:00:57.520 --> 00:01:00.518 align:middle line:84% it'll show up as a click that you can hear. 00:01:00.518 --> 00:01:02.560 align:middle line:84% So you're going to be listening for the crackling 00:01:02.560 --> 00:01:06.876 align:middle line:90% sound of spiking activity. 00:01:06.876 --> 00:01:08.872 align:middle line:90% [VIDEO PLAYBACK] 00:01:08.872 --> 00:01:12.864 align:middle line:90% [CRACKLING] 00:01:12.864 --> 00:01:21.846 align:middle line:90% 00:01:21.846 --> 00:01:23.988 align:middle line:90% [END PLAYBACK] 00:01:23.988 --> 00:01:24.780 align:middle line:90% JOSH MCDERMOTT: OK. 00:01:24.780 --> 00:01:26.030 align:middle line:90% I'm going to pause real quick. 00:01:26.030 --> 00:01:28.000 align:middle line:84% So one thing that you should have noticed 00:01:28.000 --> 00:01:30.700 align:middle line:84% is that, as this spot of light is moved around, 00:01:30.700 --> 00:01:33.760 align:middle line:84% there's this region of space where, when the spot of light 00:01:33.760 --> 00:01:37.360 align:middle line:84% sweeps through there, the response of the neuron 00:01:37.360 --> 00:01:38.180 align:middle line:90% increases. 00:01:38.180 --> 00:01:42.240 align:middle line:84% But you can also hear that, even when the spot of light 00:01:42.240 --> 00:01:45.860 align:middle line:84% is around in other places, there still sometimes are spikes. 00:01:45.860 --> 00:01:48.440 align:middle line:84% And that's because there's spontaneous activity that's 00:01:48.440 --> 00:01:50.360 align:middle line:90% being emitted by the cell. 00:01:50.360 --> 00:01:52.480 align:middle line:84% So now, they've centered the spot of light 00:01:52.480 --> 00:01:55.920 align:middle line:84% in the approximate location of the receptive field, the place 00:01:55.920 --> 00:01:57.220 align:middle line:90% that's driving the neuron. 00:01:57.220 --> 00:01:59.060 align:middle line:84% And now, it's flashed on and off. 00:01:59.060 --> 00:02:03.300 align:middle line:84% And you can see that the flashes illicit bursts of spikes. 00:02:03.300 --> 00:02:03.800 align:middle line:90% Oops. 00:02:03.800 --> 00:02:08.719 align:middle line:90% 00:02:08.719 --> 00:02:10.192 align:middle line:90% [VIDEO PLAYBACK] 00:02:10.192 --> 00:02:13.138 align:middle line:90% [CRACKLING] 00:02:13.138 --> 00:02:29.880 align:middle line:90% 00:02:29.880 --> 00:02:31.480 align:middle line:84% JOSH MCDERMOTT: So the point here 00:02:31.480 --> 00:02:33.440 align:middle line:84% is that the small circle gives a bigger 00:02:33.440 --> 00:02:35.280 align:middle line:90% response than the big circle. 00:02:35.280 --> 00:02:39.280 align:middle line:90% [CRACKLING] 00:02:39.280 --> 00:02:42.304 align:middle line:84% JOSH MCDERMOTT: That's because there's an inhibitory surround. 00:02:42.304 --> 00:02:45.585 align:middle line:90% [CRACKLING] 00:02:45.585 --> 00:02:46.960 align:middle line:84% JOSH MCDERMOTT: The point of this 00:02:46.960 --> 00:02:48.980 align:middle line:84% is the dark circle kills the response. 00:02:48.980 --> 00:02:52.367 align:middle line:84% And then you get spikes once it disappears. 00:02:52.367 --> 00:02:56.750 align:middle line:90% [CRACKLING] 00:02:56.750 --> 00:03:05.516 align:middle line:90% 00:03:05.516 --> 00:03:06.297 align:middle line:90% [END PLAYBACK] 00:03:06.297 --> 00:03:07.380 align:middle line:90% JOSH MCDERMOTT: All right. 00:03:07.380 --> 00:03:08.422 align:middle line:90% I think you get the idea. 00:03:08.422 --> 00:03:12.680 align:middle line:84% So that's the center-surround receptive field. 00:03:12.680 --> 00:03:14.400 align:middle line:84% There was one other that we were going 00:03:14.400 --> 00:03:16.858 align:middle line:84% to look at last time that did not work, which I'll show you 00:03:16.858 --> 00:03:21.740 align:middle line:84% now, which is a similar kind of experiment, 00:03:21.740 --> 00:03:24.320 align:middle line:84% but mapping out an orientation-selective receptive 00:03:24.320 --> 00:03:24.820 align:middle line:90% field. 00:03:24.820 --> 00:03:26.978 align:middle line:90% So let's check this one out. 00:03:26.978 --> 00:03:28.472 align:middle line:90% [VIDEO PLAYBACK] 00:03:28.472 --> 00:03:31.460 align:middle line:90% [CRACKLING] 00:03:31.460 --> 00:04:23.280 align:middle line:90% 00:04:23.280 --> 00:04:25.760 align:middle line:84% JOSH MCDERMOTT: OK, so the region with the X's, the region 00:04:25.760 --> 00:04:29.100 align:middle line:84% that increases the response when there's light there. 00:04:29.100 --> 00:04:31.760 align:middle line:90% 00:04:31.760 --> 00:04:34.980 align:middle line:84% The O's, or the triangles, whatever that is, 00:04:34.980 --> 00:04:38.968 align:middle line:84% is the region where the response is suppressed by light. 00:04:38.968 --> 00:04:41.896 align:middle line:90% [CRACKLING] 00:04:41.896 --> 00:04:51.168 align:middle line:90% 00:04:51.168 --> 00:04:53.460 align:middle line:84% JOSH MCDERMOTT: You can see it's orientation-selective. 00:04:53.460 --> 00:04:56.920 align:middle line:84% So now, you change the orientation and the response 00:04:56.920 --> 00:04:57.420 align:middle line:90% changes. 00:04:57.420 --> 00:05:01.600 align:middle line:84% And then you hit the right one, you get a bigger response. 00:05:01.600 --> 00:05:04.444 align:middle line:90% [CRACKLING] 00:05:04.444 --> 00:05:07.657 align:middle line:90% 00:05:07.657 --> 00:05:08.240 align:middle line:90% [END PLAYBACK] 00:05:08.240 --> 00:05:10.360 align:middle line:84% JOSH MCDERMOTT: So that's how receptive field 00:05:10.360 --> 00:05:12.620 align:middle line:90% mapping worked in the old days. 00:05:12.620 --> 00:05:15.680 align:middle line:90% 00:05:15.680 --> 00:05:20.500 align:middle line:84% So last time, we talked about primary visual cortex. 00:05:20.500 --> 00:05:22.613 align:middle line:84% So just to remind everybody, so this 00:05:22.613 --> 00:05:23.780 align:middle line:90% is what we're talking about. 00:05:23.780 --> 00:05:25.320 align:middle line:90% It's the visual system. 00:05:25.320 --> 00:05:27.140 align:middle line:90% Light enters the eye. 00:05:27.140 --> 00:05:30.460 align:middle line:84% There's an image formed on the retina. 00:05:30.460 --> 00:05:35.477 align:middle line:84% There's a set of layers of cells, the output of which 00:05:35.477 --> 00:05:37.060 align:middle line:84% comes from the retinal ganglion cells. 00:05:37.060 --> 00:05:39.320 align:middle line:84% That goes through the optic nerve 00:05:39.320 --> 00:05:41.380 align:middle line:84% to the LGN, part of the thalamus. 00:05:41.380 --> 00:05:43.580 align:middle line:84% And from there, it projects to the visual cortex. 00:05:43.580 --> 00:05:46.360 align:middle line:84% And the primary visual cortex is the main projection site 00:05:46.360 --> 00:05:48.680 align:middle line:90% of the thalamus. 00:05:48.680 --> 00:05:50.820 align:middle line:84% It's the biggest region in the visual cortex. 00:05:50.820 --> 00:05:55.010 align:middle line:84% We talked about how one of the key organizational principles 00:05:55.010 --> 00:05:57.050 align:middle line:84% of the visual cortex is retinotopy. 00:05:57.050 --> 00:05:59.140 align:middle line:84% So there are maps of the visual field 00:05:59.140 --> 00:06:00.390 align:middle line:90% that you can see in the brain. 00:06:00.390 --> 00:06:01.850 align:middle line:84% And they distinguish different visual areas, 00:06:01.850 --> 00:06:02.970 align:middle line:90% as we'll see later. 00:06:02.970 --> 00:06:05.310 align:middle line:84% We talked about the principle of cortical magnification. 00:06:05.310 --> 00:06:07.970 align:middle line:84% So one of the consequences for the foveal organization 00:06:07.970 --> 00:06:10.730 align:middle line:84% of the retina, where there's a high density of receptors 00:06:10.730 --> 00:06:14.050 align:middle line:84% at the center of gaze, is that there's 00:06:14.050 --> 00:06:18.490 align:middle line:84% more area in the cortex that's devoted to the center of gaze. 00:06:18.490 --> 00:06:20.970 align:middle line:84% And you can see that in various ways. 00:06:20.970 --> 00:06:23.650 align:middle line:84% We then talked about orientation selectivity, 00:06:23.650 --> 00:06:27.710 align:middle line:84% including this video that we just saw, 00:06:27.710 --> 00:06:30.225 align:middle line:84% and how we can approximate the receptive 00:06:30.225 --> 00:06:31.850 align:middle line:84% fields of orientation-selective neurons 00:06:31.850 --> 00:06:35.330 align:middle line:84% with linear filters that have these kinds of oriented forms. 00:06:35.330 --> 00:06:38.490 align:middle line:84% We then talked about this simple model that's 00:06:38.490 --> 00:06:41.570 align:middle line:84% been proposed for how you might create 00:06:41.570 --> 00:06:46.450 align:middle line:84% an orientation-selective neuron from the responses 00:06:46.450 --> 00:06:49.730 align:middle line:84% of center-surround receptive fields at an earlier 00:06:49.730 --> 00:06:52.750 align:middle line:84% stage of the visual system by having appropriate convergence, 00:06:52.750 --> 00:06:54.930 align:middle line:84% and then talked about an experiment that provides 00:06:54.930 --> 00:06:57.730 align:middle line:84% evidence for this, where the experiment found pairs 00:06:57.730 --> 00:07:01.410 align:middle line:84% of neurons in the LGN and V1 that were likely to be connected 00:07:01.410 --> 00:07:03.567 align:middle line:84% based on the cross-correlogram, and then looked 00:07:03.567 --> 00:07:05.650 align:middle line:84% at the correspondence between the receptive fields 00:07:05.650 --> 00:07:08.350 align:middle line:84% and found that the correspondence was pretty good, 00:07:08.350 --> 00:07:10.858 align:middle line:84% providing evidence for that simple model. 00:07:10.858 --> 00:07:12.650 align:middle line:84% We then talked about the columnar structure 00:07:12.650 --> 00:07:14.090 align:middle line:90% of the visual cortex. 00:07:14.090 --> 00:07:16.850 align:middle line:84% Again, a pretty defining characteristic 00:07:16.850 --> 00:07:19.290 align:middle line:84% of lots of cortical areas is that they're 00:07:19.290 --> 00:07:23.010 align:middle line:84% organized into columns, meaning that, if you are inserting 00:07:23.010 --> 00:07:25.450 align:middle line:84% an electrode perpendicular to the cortical surface, 00:07:25.450 --> 00:07:26.825 align:middle line:84% the neurons that you'll encounter 00:07:26.825 --> 00:07:28.970 align:middle line:84% will tend to have fairly consistent properties. 00:07:28.970 --> 00:07:31.910 align:middle line:84% So in particular, there are orientation columns. 00:07:31.910 --> 00:07:33.930 align:middle line:84% So all of the different cells that 00:07:33.930 --> 00:07:36.930 align:middle line:84% are sampled on electrode penetration 1 00:07:36.930 --> 00:07:38.820 align:middle line:84% have the same orientation preference. 00:07:38.820 --> 00:07:41.070 align:middle line:84% That's what those little line segments are indicating, 00:07:41.070 --> 00:07:43.850 align:middle line:84% whereas if you insert the electrode obliquely 00:07:43.850 --> 00:07:46.830 align:middle line:84% to the cortical surface, you go through a bunch of columns. 00:07:46.830 --> 00:07:49.130 align:middle line:84% And so the orientation preference 00:07:49.130 --> 00:07:52.410 align:middle line:84% changes as a function of the electrode position. 00:07:52.410 --> 00:07:54.390 align:middle line:84% We saw some pictures of orientation columns. 00:07:54.390 --> 00:07:56.410 align:middle line:84% And then we talked about the ambiguity 00:07:56.410 --> 00:07:58.930 align:middle line:84% of the response of a single neuron 00:07:58.930 --> 00:08:01.150 align:middle line:84% and how, in this particular example, 00:08:01.150 --> 00:08:03.190 align:middle line:84% contrast and orientation are confounded, 00:08:03.190 --> 00:08:06.690 align:middle line:84% and how you could resolve that ambiguity using a population 00:08:06.690 --> 00:08:07.270 align:middle line:90% code. 00:08:07.270 --> 00:08:08.730 align:middle line:84% So if you were able to look across 00:08:08.730 --> 00:08:12.770 align:middle line:84% a set of neurons that varied in their preferred orientations, 00:08:12.770 --> 00:08:15.370 align:middle line:84% even in a situation where the contrast is varied, 00:08:15.370 --> 00:08:18.890 align:middle line:84% such that you could get the same response for all 00:08:18.890 --> 00:08:22.130 align:middle line:84% these different orientations in one neuron, 00:08:22.130 --> 00:08:24.730 align:middle line:84% if you were able to analyze the population, in particular 00:08:24.730 --> 00:08:27.930 align:middle line:84% detect the peak of the distribution, 00:08:27.930 --> 00:08:30.490 align:middle line:84% you would be able to infer the correct orientation 00:08:30.490 --> 00:08:32.690 align:middle line:90% despite the change in contrast. 00:08:32.690 --> 00:08:36.090 align:middle line:84% And then we ended class by experiencing the tilt 00:08:36.090 --> 00:08:40.370 align:middle line:84% aftereffect, where we induced a temporary change in your brain 00:08:40.370 --> 00:08:44.330 align:middle line:84% by forcing you or imploring you to stare 00:08:44.330 --> 00:08:48.070 align:middle line:84% at the adapting stimulus, which is this oriented grating. 00:08:48.070 --> 00:08:51.050 align:middle line:84% And you found that if you stared at this thing 00:08:51.050 --> 00:08:54.070 align:middle line:84% for about a minute, and then you looked at the test stimulus, 00:08:54.070 --> 00:08:57.010 align:middle line:84% even though the test stimulus is actually vertical, 00:08:57.010 --> 00:09:00.210 align:middle line:84% it appears to be a little bit off of vertical. 00:09:00.210 --> 00:09:04.010 align:middle line:84% So that's what's known as the tilt aftereffect. 00:09:04.010 --> 00:09:06.130 align:middle line:84% A few details about this-- so you 00:09:06.130 --> 00:09:09.210 align:middle line:84% may recall that, when we were doing the adaptation, 00:09:09.210 --> 00:09:12.170 align:middle line:84% I instructed you to move your eyes around 00:09:12.170 --> 00:09:14.010 align:middle line:90% inside the gray circle. 00:09:14.010 --> 00:09:17.730 align:middle line:84% And the reason for this is that adaptation 00:09:17.730 --> 00:09:20.070 align:middle line:84% happens at all stages of the visual system. 00:09:20.070 --> 00:09:22.530 align:middle line:84% And in particular, it also happens at the retina, 00:09:22.530 --> 00:09:24.510 align:middle line:84% even at the level of the photoreceptors. 00:09:24.510 --> 00:09:28.250 align:middle line:84% And so if you maintain very tight fixation on something, 00:09:28.250 --> 00:09:30.570 align:middle line:84% you will actually get very strong adaptation 00:09:30.570 --> 00:09:31.530 align:middle line:90% of the photoreceptors. 00:09:31.530 --> 00:09:33.230 align:middle line:90% And you'll get an afterimage. 00:09:33.230 --> 00:09:37.570 align:middle line:84% You'll see the negative of the image that you were looking at. 00:09:37.570 --> 00:09:39.850 align:middle line:84% And that might be distracting and might 00:09:39.850 --> 00:09:42.010 align:middle line:84% make it a little harder to actually observe 00:09:42.010 --> 00:09:43.070 align:middle line:90% the orientation. 00:09:43.070 --> 00:09:46.410 align:middle line:84% But by moving your eyes around in that gray circle, 00:09:46.410 --> 00:09:50.090 align:middle line:84% you don't systematically adapt the photoreceptors 00:09:50.090 --> 00:09:54.570 align:middle line:84% because you'll be varying the position of your eyes 00:09:54.570 --> 00:09:57.690 align:middle line:84% by more than the spatial frequency of the grating. 00:09:57.690 --> 00:10:01.130 align:middle line:84% So that's just a trick to get these effects to work 00:10:01.130 --> 00:10:02.050 align:middle line:90% pretty well. 00:10:02.050 --> 00:10:04.270 align:middle line:90% So we got the tilt aftereffect. 00:10:04.270 --> 00:10:07.270 align:middle line:84% And if you would like to, when you go home, 00:10:07.270 --> 00:10:08.883 align:middle line:84% you can try adapting to this one. 00:10:08.883 --> 00:10:11.050 align:middle line:84% And you will find that you would get the aftereffect 00:10:11.050 --> 00:10:12.970 align:middle line:90% in the opposite direction. 00:10:12.970 --> 00:10:13.470 align:middle line:90% All right. 00:10:13.470 --> 00:10:17.250 align:middle line:84% So the question is, how can we understand 00:10:17.250 --> 00:10:18.850 align:middle line:90% these kinds of effects? 00:10:18.850 --> 00:10:23.930 align:middle line:84% What do they tell us about the brain and the visual system? 00:10:23.930 --> 00:10:27.890 align:middle line:84% And so in general, we think that aftereffects 00:10:27.890 --> 00:10:30.730 align:middle line:84% are due to something that happens 00:10:30.730 --> 00:10:32.610 align:middle line:84% in your visual system, whereby there 00:10:32.610 --> 00:10:36.210 align:middle line:84% is a decrease in the responsiveness of neurons 00:10:36.210 --> 00:10:39.450 align:middle line:84% after a period of prolonged activity. 00:10:39.450 --> 00:10:44.030 align:middle line:84% And so the idea is that, when you are adapting yourself-- 00:10:44.030 --> 00:10:46.410 align:middle line:84% so you're staring at this grating. 00:10:46.410 --> 00:10:50.050 align:middle line:84% And so the neurons that are responsive to this grating 00:10:50.050 --> 00:10:53.590 align:middle line:84% are firing pretty hard for an extended period of time, 00:10:53.590 --> 00:10:55.290 align:middle line:90% like a minute. 00:10:55.290 --> 00:10:59.770 align:middle line:84% And for some, as of now, unspecified reason, 00:10:59.770 --> 00:11:02.130 align:middle line:84% we postulate that they become less 00:11:02.130 --> 00:11:05.730 align:middle line:84% responsive as a consequence of having fired pretty intensively 00:11:05.730 --> 00:11:06.990 align:middle line:90% for a period of time. 00:11:06.990 --> 00:11:09.410 align:middle line:84% For instance, you might imagine that there's 00:11:09.410 --> 00:11:12.730 align:middle line:84% a metabolic resource that's needed for spiking. 00:11:12.730 --> 00:11:15.490 align:middle line:84% And that gets temporarily exhausted or depleted 00:11:15.490 --> 00:11:17.730 align:middle line:90% a little bit. 00:11:17.730 --> 00:11:20.870 align:middle line:84% So we believe that that's really what is underlying this. 00:11:20.870 --> 00:11:23.390 align:middle line:84% But how can we understand this in more detail? 00:11:23.390 --> 00:11:24.450 align:middle line:90% So here's the idea. 00:11:24.450 --> 00:11:27.350 align:middle line:84% So remember, we just talked about population codes. 00:11:27.350 --> 00:11:29.330 align:middle line:84% So the idea of the population code 00:11:29.330 --> 00:11:31.570 align:middle line:90% is that you get some stimulus. 00:11:31.570 --> 00:11:33.930 align:middle line:90% There's a population of neurons. 00:11:33.930 --> 00:11:37.970 align:middle line:84% These triangular-looking things are the tuning curves 00:11:37.970 --> 00:11:41.555 align:middle line:84% of a hypothetical set of neurons. 00:11:41.555 --> 00:11:43.930 align:middle line:84% This is plotting the response of the neuron as a function 00:11:43.930 --> 00:11:45.950 align:middle line:90% of orientation on the x-axis. 00:11:45.950 --> 00:11:48.330 align:middle line:84% And then the little line segment at the top 00:11:48.330 --> 00:11:54.050 align:middle line:84% is symbolizing the preferred orientation of a given neuron. 00:11:54.050 --> 00:11:58.850 align:middle line:84% So this is the population response normally 00:11:58.850 --> 00:12:00.392 align:middle line:90% to the vertical grating. 00:12:00.392 --> 00:12:01.850 align:middle line:84% And what you can see here-- so now, 00:12:01.850 --> 00:12:03.850 align:middle line:84% the x-axis here, the line segment 00:12:03.850 --> 00:12:06.970 align:middle line:84% denotes the preferred orientation of a cell. 00:12:06.970 --> 00:12:08.450 align:middle line:84% And you can see that the cell that 00:12:08.450 --> 00:12:11.117 align:middle line:84% prefers the vertical orientation gives you the highest response. 00:12:11.117 --> 00:12:13.830 align:middle line:84% And then the ones on either side give you a lower response. 00:12:13.830 --> 00:12:18.770 align:middle line:84% And you get this bell-shaped curve with a peak at vertical. 00:12:18.770 --> 00:12:21.450 align:middle line:84% And so the hypothesis is that there 00:12:21.450 --> 00:12:23.730 align:middle line:84% is a mechanism that analyzes this population 00:12:23.730 --> 00:12:27.603 align:middle line:84% response, detects the peak, understands that the peak is 00:12:27.603 --> 00:12:29.770 align:middle line:84% occurring at a neuron whose preferred orientation is 00:12:29.770 --> 00:12:30.510 align:middle line:90% vertical. 00:12:30.510 --> 00:12:34.550 align:middle line:84% And that causes you to perceive a vertical orientation. 00:12:34.550 --> 00:12:36.490 align:middle line:84% So that's what we think happens normally. 00:12:36.490 --> 00:12:39.730 align:middle line:84% Now, we did this crazy thing to your visual system, 00:12:39.730 --> 00:12:41.950 align:middle line:84% where we made you stare at that thing for a minute. 00:12:41.950 --> 00:12:45.650 align:middle line:84% And so the hypothesis is that now 00:12:45.650 --> 00:12:50.030 align:middle line:84% the responsiveness of some of the neurons has been altered. 00:12:50.030 --> 00:12:54.890 align:middle line:84% And that's symbolized by the tuning curves being lower 00:12:54.890 --> 00:12:55.930 align:middle line:90% on the graph. 00:12:55.930 --> 00:13:00.330 align:middle line:84% And we hypothesize that the decrease in responsiveness 00:13:00.330 --> 00:13:03.780 align:middle line:84% is proportional to how strongly the neurons were firing when 00:13:03.780 --> 00:13:05.030 align:middle line:90% you were doing the adaptation. 00:13:05.030 --> 00:13:07.530 align:middle line:84% And so in particular, the biggest decrease 00:13:07.530 --> 00:13:12.730 align:middle line:84% here is at the orientation that's equal to the adapting 00:13:12.730 --> 00:13:14.310 align:middle line:90% orientation. 00:13:14.310 --> 00:13:18.590 align:middle line:84% And then there's a graded effect on either side. 00:13:18.590 --> 00:13:21.730 align:middle line:84% So the consequence of that is that, when you subsequently 00:13:21.730 --> 00:13:26.530 align:middle line:84% view the vertical grating again, the population response has now 00:13:26.530 --> 00:13:28.110 align:middle line:90% been altered. 00:13:28.110 --> 00:13:30.650 align:middle line:84% So normally, the peak of the response 00:13:30.650 --> 00:13:33.230 align:middle line:90% occurs in the vertical neuron. 00:13:33.230 --> 00:13:35.130 align:middle line:84% But because of the adaptation, you 00:13:35.130 --> 00:13:37.890 align:middle line:84% can see that's no longer the case. 00:13:37.890 --> 00:13:39.710 align:middle line:84% So we adapted the stuff over here. 00:13:39.710 --> 00:13:42.150 align:middle line:84% And that means that the distribution has shifted. 00:13:42.150 --> 00:13:43.930 align:middle line:84% And now, the peak of the distribution 00:13:43.930 --> 00:13:45.930 align:middle line:84% is this orientation here, which is off 00:13:45.930 --> 00:13:48.310 align:middle line:84% of vertical in the opposite direction. 00:13:48.310 --> 00:13:53.170 align:middle line:84% And so if you believe that there is some mechanism that 00:13:53.170 --> 00:13:56.550 align:middle line:84% can detect the peak and use that to infer the orientation, 00:13:56.550 --> 00:13:58.810 align:middle line:84% then that predicts that the perceived orientation 00:13:58.810 --> 00:14:01.490 align:middle line:84% will be shifted in the direction that you experience 00:14:01.490 --> 00:14:02.970 align:middle line:90% after you've adapted. 00:14:02.970 --> 00:14:06.930 align:middle line:84% So that's typically how we explain aftereffects 00:14:06.930 --> 00:14:09.290 align:middle line:84% and how they are believed to provide evidence 00:14:09.290 --> 00:14:11.370 align:middle line:90% for these population codes. 00:14:11.370 --> 00:14:12.050 align:middle line:90% Yeah? 00:14:12.050 --> 00:14:13.467 align:middle line:84% STUDENT: So in the last slide, why 00:14:13.467 --> 00:14:16.730 align:middle line:84% are all of the peaks at the same height? 00:14:16.730 --> 00:14:19.570 align:middle line:84% JOSH MCDERMOTT: So these are tuning curves. 00:14:19.570 --> 00:14:24.290 align:middle line:84% So that's just telling you how much the cell would respond 00:14:24.290 --> 00:14:26.850 align:middle line:90% to each of these orientations. 00:14:26.850 --> 00:14:30.410 align:middle line:84% This is the actual response of the different cells to that one 00:14:30.410 --> 00:14:32.050 align:middle line:90% orientation. 00:14:32.050 --> 00:14:34.210 align:middle line:84% But the point is that all of these cells 00:14:34.210 --> 00:14:37.130 align:middle line:84% initially are equally responsive. 00:14:37.130 --> 00:14:39.410 align:middle line:84% But then they become differentially responsive 00:14:39.410 --> 00:14:41.430 align:middle line:84% temporarily following adaptation. 00:14:41.430 --> 00:14:44.760 align:middle line:90% 00:14:44.760 --> 00:14:46.790 align:middle line:90% Any other questions about this? 00:14:46.790 --> 00:14:49.610 align:middle line:90% 00:14:49.610 --> 00:14:54.130 align:middle line:84% So there's lots of aftereffects, especially in vision. 00:14:54.130 --> 00:14:56.890 align:middle line:84% They're actually like a lot more powerful in vision 00:14:56.890 --> 00:14:58.332 align:middle line:90% than in audition. 00:14:58.332 --> 00:15:00.790 align:middle line:84% It's interesting to think about why that might be the case. 00:15:00.790 --> 00:15:04.070 align:middle line:84% But we'll encounter lots of other aftereffects. 00:15:04.070 --> 00:15:08.730 align:middle line:84% And they were classically a pretty common tool 00:15:08.730 --> 00:15:10.370 align:middle line:84% that was used by perception scientists 00:15:10.370 --> 00:15:13.470 align:middle line:84% to try to infer things that were happening in the brain. 00:15:13.470 --> 00:15:14.857 align:middle line:90% So the tilt aftereffect is one. 00:15:14.857 --> 00:15:16.690 align:middle line:84% We'll also experience the motion aftereffect 00:15:16.690 --> 00:15:18.670 align:middle line:84% when we start to talk about motion. 00:15:18.670 --> 00:15:22.850 align:middle line:84% That one's really powerful, lots of aftereffects. 00:15:22.850 --> 00:15:27.940 align:middle line:90% 00:15:27.940 --> 00:15:30.990 align:middle line:84% So we've got we've got these orientation-selective neurons. 00:15:30.990 --> 00:15:32.210 align:middle line:90% We can adapt them. 00:15:32.210 --> 00:15:34.330 align:middle line:84% And we discussed how they have these receptive 00:15:34.330 --> 00:15:37.390 align:middle line:84% fields that you can summarize with pictures like this. 00:15:37.390 --> 00:15:39.650 align:middle line:84% And the fact that we can draw a picture like this 00:15:39.650 --> 00:15:41.690 align:middle line:84% really is a reflection of the fact 00:15:41.690 --> 00:15:43.410 align:middle line:84% that the neuron is well-approximated 00:15:43.410 --> 00:15:44.770 align:middle line:90% by a linear filter. 00:15:44.770 --> 00:15:47.890 align:middle line:84% So a linear filter has this property 00:15:47.890 --> 00:15:51.590 align:middle line:84% that you can define coefficients at different positions in space. 00:15:51.590 --> 00:15:55.170 align:middle line:84% And that does a good job of predicting the response. 00:15:55.170 --> 00:15:58.730 align:middle line:84% And so it's pretty common to observe responses that are-- 00:15:58.730 --> 00:16:03.290 align:middle line:84% receptive fields that are either even or odd. 00:16:03.290 --> 00:16:06.090 align:middle line:84% And you might wonder, OK, well, why 00:16:06.090 --> 00:16:08.810 align:middle line:84% do we have these different types of receptive fields? 00:16:08.810 --> 00:16:12.810 align:middle line:84% And so in the early days, the people who initially discovered 00:16:12.810 --> 00:16:14.970 align:middle line:84% orientation selectivity, Hubel and Wiesel, 00:16:14.970 --> 00:16:16.810 align:middle line:84% tended to interpret these receptive 00:16:16.810 --> 00:16:21.430 align:middle line:84% fields with Intuitive verbal descriptions. 00:16:21.430 --> 00:16:25.290 align:middle line:84% So they would talk about these kinds 00:16:25.290 --> 00:16:27.050 align:middle line:84% of receptive fields as edge detectors 00:16:27.050 --> 00:16:31.050 align:middle line:84% and these kinds of receptive fields as bar detectors. 00:16:31.050 --> 00:16:35.140 align:middle line:84% And so there's this very appealing intuitive theory 00:16:35.140 --> 00:16:39.620 align:middle line:84% of vision that of fit into, where the idea is that you're 00:16:39.620 --> 00:16:43.420 align:middle line:84% going to initially detect edges, and then use 00:16:43.420 --> 00:16:48.240 align:middle line:84% those edges to detect and recognize objects. 00:16:48.240 --> 00:16:50.320 align:middle line:84% And so initially, when this was discovered, 00:16:50.320 --> 00:16:53.380 align:middle line:84% these orientation-selective receptive fields 00:16:53.380 --> 00:16:55.260 align:middle line:90% fit into that view. 00:16:55.260 --> 00:16:58.395 align:middle line:84% So the challenge with that way of thinking about things-- well, 00:16:58.395 --> 00:16:59.520 align:middle line:90% there's lots of challenges. 00:16:59.520 --> 00:17:01.820 align:middle line:84% But one of the challenges is that edges 00:17:01.820 --> 00:17:06.180 align:middle line:84% are, in practice, very hard to detect with local operators. 00:17:06.180 --> 00:17:10.740 align:middle line:84% So there was lots of work in the early days of computer vision-- 00:17:10.740 --> 00:17:15.119 align:middle line:84% 1970s, 1980s-- trying to figure out how to detect edges. 00:17:15.119 --> 00:17:18.460 align:middle line:84% And they would include ingredients 00:17:18.460 --> 00:17:20.060 align:middle line:84% like these linear filters, and then 00:17:20.060 --> 00:17:22.980 align:middle line:84% do some other stuff because the raw responses 00:17:22.980 --> 00:17:25.625 align:middle line:84% of the linear filters usually wasn't enough. 00:17:25.625 --> 00:17:27.000 align:middle line:84% And they still didn't work great. 00:17:27.000 --> 00:17:30.380 align:middle line:84% So this is an example of a very well-known classical edge 00:17:30.380 --> 00:17:33.180 align:middle line:84% detector known as the Canny edge detector, 00:17:33.180 --> 00:17:35.980 align:middle line:90% run on this particular image. 00:17:35.980 --> 00:17:39.020 align:middle line:84% And so you can see that you get things out 00:17:39.020 --> 00:17:41.620 align:middle line:90% that are not ridiculous. 00:17:41.620 --> 00:17:45.920 align:middle line:84% But the output really is pretty far from maybe 00:17:45.920 --> 00:17:47.420 align:middle line:84% providing what you might think would 00:17:47.420 --> 00:17:49.620 align:middle line:84% be a complete description of all of the edges 00:17:49.620 --> 00:17:51.280 align:middle line:90% that you can see in the image. 00:17:51.280 --> 00:17:54.320 align:middle line:84% So there's some that get completely missed, 00:17:54.320 --> 00:17:56.580 align:middle line:90% like the edge of the chin. 00:17:56.580 --> 00:17:58.580 align:middle line:84% And so in general, edge detection 00:17:58.580 --> 00:18:01.180 align:middle line:84% is a fairly-- is a pretty hard problem. 00:18:01.180 --> 00:18:06.340 align:middle line:84% And I think one very likely possibility 00:18:06.340 --> 00:18:10.460 align:middle line:84% is that this intuitive idea that one could separate 00:18:10.460 --> 00:18:14.140 align:middle line:84% the stages of detecting edges and detecting objects 00:18:14.140 --> 00:18:16.420 align:middle line:84% is not a great idea, that, in practice, like 00:18:16.420 --> 00:18:20.780 align:middle line:84% the edges are inferred along with the objects. 00:18:20.780 --> 00:18:25.340 align:middle line:84% But that's a longer discussion we can have later. 00:18:25.340 --> 00:18:27.100 align:middle line:84% So that was these original intuitions 00:18:27.100 --> 00:18:29.300 align:middle line:90% about orientation selectivity. 00:18:29.300 --> 00:18:31.940 align:middle line:90% It's got some issues. 00:18:31.940 --> 00:18:34.860 align:middle line:84% Another challenge to that way of thinking about things 00:18:34.860 --> 00:18:38.740 align:middle line:84% is that if you actually look at the responses of the kinds 00:18:38.740 --> 00:18:41.300 align:middle line:84% of filters that you see in the visual system in the way 00:18:41.300 --> 00:18:43.140 align:middle line:84% simple cells, if you look at their responses 00:18:43.140 --> 00:18:45.860 align:middle line:84% to actual edges and actual lines, 00:18:45.860 --> 00:18:48.240 align:middle line:84% the responses are pretty complicated. 00:18:48.240 --> 00:18:52.400 align:middle line:84% And so here's an example, where we've got an input image. 00:18:52.400 --> 00:18:53.347 align:middle line:90% So that's an image. 00:18:53.347 --> 00:18:55.680 align:middle line:84% It's actually a one-dimensional slice through the image. 00:18:55.680 --> 00:18:58.400 align:middle line:84% So the y-axis would be intensity. 00:18:58.400 --> 00:19:01.340 align:middle line:84% And the x-axis would just be one spatial dimension. 00:19:01.340 --> 00:19:03.860 align:middle line:84% And so you can see that there are two step edges where 00:19:03.860 --> 00:19:07.100 align:middle line:84% the intensity goes up, and then two lines-- 00:19:07.100 --> 00:19:09.380 align:middle line:90% one light and one dark. 00:19:09.380 --> 00:19:14.060 align:middle line:84% And then B and C plot the convolution 00:19:14.060 --> 00:19:15.940 align:middle line:84% of an odd symmetric filter-- that's 00:19:15.940 --> 00:19:18.665 align:middle line:84% B-- and an even symmetric filter-- that's C. 00:19:18.665 --> 00:19:20.540 align:middle line:84% So we're measuring the response of the filter 00:19:20.540 --> 00:19:22.200 align:middle line:90% at every position in the image. 00:19:22.200 --> 00:19:25.300 align:middle line:84% And that's what a convolution does. 00:19:25.300 --> 00:19:27.020 align:middle line:84% And again, naively, you might expect 00:19:27.020 --> 00:19:29.660 align:middle line:84% that an edge detector is just going to give you 00:19:29.660 --> 00:19:31.400 align:middle line:90% a delta function at the edges. 00:19:31.400 --> 00:19:32.900 align:middle line:84% And the line detector would give you 00:19:32.900 --> 00:19:35.520 align:middle line:90% a delta function at the lines. 00:19:35.520 --> 00:19:37.100 align:middle line:84% But in actuality, all of the filters 00:19:37.100 --> 00:19:38.800 align:middle line:84% kind of respond at all of the features. 00:19:38.800 --> 00:19:43.220 align:middle line:84% But the responses are complicated and wiggly. 00:19:43.220 --> 00:19:45.097 align:middle line:84% So it's not to say that you might 00:19:45.097 --> 00:19:47.180 align:middle line:84% be able to take these, and then do something else, 00:19:47.180 --> 00:19:48.480 align:middle line:90% and maybe infer something. 00:19:48.480 --> 00:19:50.580 align:middle line:84% But at any rate, the responses don't 00:19:50.580 --> 00:19:54.620 align:middle line:84% completely, straightforwardly detect edges and lines 00:19:54.620 --> 00:19:57.680 align:middle line:90% in an intuitive way. 00:19:57.680 --> 00:20:03.140 align:middle line:84% And so I would say, so there have been multiple iterations 00:20:03.140 --> 00:20:07.300 align:middle line:84% of attempts to explain and understand orientation 00:20:07.300 --> 00:20:10.060 align:middle line:84% selectivity in computational terms, some of which 00:20:10.060 --> 00:20:11.600 align:middle line:84% are beyond the scope of the class. 00:20:11.600 --> 00:20:14.100 align:middle line:84% I would say the relation of simple cells 00:20:14.100 --> 00:20:16.860 align:middle line:84% and orientation selectivity to function 00:20:16.860 --> 00:20:19.660 align:middle line:84% remains challenging to verbalize, 00:20:19.660 --> 00:20:24.100 align:middle line:90% to explain in an intuitive way. 00:20:24.100 --> 00:20:28.540 align:middle line:84% But it's nonetheless the case that similar looking filters 00:20:28.540 --> 00:20:33.140 align:middle line:84% pretty consistently emerge in the initial stages of systems 00:20:33.140 --> 00:20:36.540 align:middle line:84% that are trained on vision tasks. 00:20:36.540 --> 00:20:37.920 align:middle line:90% So this is an example. 00:20:37.920 --> 00:20:39.980 align:middle line:84% So these are the first layer filters 00:20:39.980 --> 00:20:42.180 align:middle line:84% of a convolutional neural network 00:20:42.180 --> 00:20:43.840 align:middle line:90% optimized to recognize objects. 00:20:43.840 --> 00:20:46.600 align:middle line:84% This is a very famous convolutional neural network. 00:20:46.600 --> 00:20:49.460 align:middle line:84% This was the first one that produced really impressive 00:20:49.460 --> 00:20:53.420 align:middle line:84% performance on ImageNet, a classical object recognition 00:20:53.420 --> 00:20:54.780 align:middle line:90% task. 00:20:54.780 --> 00:20:56.040 align:middle line:90% This was back in 2012. 00:20:56.040 --> 00:20:58.660 align:middle line:84% And this opened the gates of deep learning 00:20:58.660 --> 00:21:02.340 align:middle line:84% and got everybody really excited about deep learning. 00:21:02.340 --> 00:21:04.740 align:middle line:84% Each one of these is a different filter. 00:21:04.740 --> 00:21:08.140 align:middle line:84% And you can see that they look a lot like simple cells, 00:21:08.140 --> 00:21:11.300 align:middle line:84% in the sense that they're orientation-selective to tune 00:21:11.300 --> 00:21:13.380 align:middle line:90% to different orientations. 00:21:13.380 --> 00:21:15.660 align:middle line:84% And so this is a very common observation. 00:21:15.660 --> 00:21:18.540 align:middle line:84% So you very often see orientation selectivity emerging 00:21:18.540 --> 00:21:21.540 align:middle line:84% as the initial stage of analysis in systems 00:21:21.540 --> 00:21:25.270 align:middle line:84% that are trained on sensory problems, 00:21:25.270 --> 00:21:26.520 align:middle line:90% in particular vision problems. 00:21:26.520 --> 00:21:29.560 align:middle line:90% 00:21:29.560 --> 00:21:31.180 align:middle line:84% So mathematically, if we actually 00:21:31.180 --> 00:21:36.380 align:middle line:84% want to summarize the form of a simple cell, 00:21:36.380 --> 00:21:40.420 align:middle line:84% it's common to describe them as Gabor functions. 00:21:40.420 --> 00:21:43.380 align:middle line:84% So a Gabor function is the product 00:21:43.380 --> 00:21:47.580 align:middle line:90% of a sinusoid with a Gaussian. 00:21:47.580 --> 00:21:51.080 align:middle line:84% So we've got a sinusoid up here and a Gaussian. 00:21:51.080 --> 00:21:53.060 align:middle line:84% And we just pointwise multiply them. 00:21:53.060 --> 00:21:56.320 align:middle line:84% And you get this thing, where it starts out at 0. 00:21:56.320 --> 00:21:58.660 align:middle line:84% And then the wiggles increase at some rate. 00:21:58.660 --> 00:22:00.540 align:middle line:90% And then they decrease again. 00:22:00.540 --> 00:22:06.400 align:middle line:84% This is a Gabor function displayed as an actual image. 00:22:06.400 --> 00:22:09.020 align:middle line:84% So both of these types of receptive fields, even and odd, 00:22:09.020 --> 00:22:11.860 align:middle line:84% can be described as Gabor functions. 00:22:11.860 --> 00:22:14.440 align:middle line:84% The only difference is whether you use a sine or cosine, 00:22:14.440 --> 00:22:16.420 align:middle line:84% because the even and odd functions are 00:22:16.420 --> 00:22:19.540 align:middle line:90% 90 degrees out of phase. 00:22:19.540 --> 00:22:22.500 align:middle line:84% So that's at least a mathematical description 00:22:22.500 --> 00:22:24.900 align:middle line:84% of what these simple cells are doing. 00:22:24.900 --> 00:22:27.220 align:middle line:84% Now, the other wrinkle to the story 00:22:27.220 --> 00:22:30.640 align:middle line:84% is that, in addition to simple cells, in primary visual cortex, 00:22:30.640 --> 00:22:32.420 align:middle line:84% there's another type of cell that 00:22:32.420 --> 00:22:36.020 align:middle line:90% became known as complex cells. 00:22:36.020 --> 00:22:39.660 align:middle line:84% And complex cells, they differ from simple cells in that they 00:22:39.660 --> 00:22:42.920 align:middle line:84% respond to-- so they're still orientation-selective, 00:22:42.920 --> 00:22:45.060 align:middle line:84% but they respond to an oriented stimulus 00:22:45.060 --> 00:22:47.660 align:middle line:84% no matter where it falls in the receptive field. 00:22:47.660 --> 00:22:49.500 align:middle line:84% So remember in that video that we just saw, 00:22:49.500 --> 00:22:52.420 align:middle line:84% where they were mapping out the orientation-selective receptive 00:22:52.420 --> 00:22:57.267 align:middle line:84% field, it was very sensitive to where that bar was placed. 00:22:57.267 --> 00:22:59.100 align:middle line:84% So if you got it in exactly the right place, 00:22:59.100 --> 00:23:00.320 align:middle line:90% you'd get a big response. 00:23:00.320 --> 00:23:02.360 align:middle line:84% But then if you shifted it a little bit over, 00:23:02.360 --> 00:23:03.880 align:middle line:90% the response would decrease. 00:23:03.880 --> 00:23:06.580 align:middle line:84% So that's characteristic of a simple cell 00:23:06.580 --> 00:23:08.540 align:middle line:84% and consistent with the idea that there's 00:23:08.540 --> 00:23:11.080 align:middle line:84% a linear filter that's got this excitatory region. 00:23:11.080 --> 00:23:13.790 align:middle line:84% And so if you hit that with light, you get a big response. 00:23:13.790 --> 00:23:15.540 align:middle line:84% And if you're in the inhibitory side lobe, 00:23:15.540 --> 00:23:16.920 align:middle line:90% you get a small response. 00:23:16.920 --> 00:23:19.280 align:middle line:84% But complex cells behave very differently. 00:23:19.280 --> 00:23:21.060 align:middle line:84% They respond no matter where the stimulus 00:23:21.060 --> 00:23:22.860 align:middle line:90% falls on the receptive field. 00:23:22.860 --> 00:23:24.700 align:middle line:84% And as a consequence, they can't be 00:23:24.700 --> 00:23:28.725 align:middle line:84% decomposed into these excitatory and inhibitory response zones. 00:23:28.725 --> 00:23:30.100 align:middle line:84% So here's just an example picture 00:23:30.100 --> 00:23:32.700 align:middle line:84% that differentiates the two types of cells. 00:23:32.700 --> 00:23:36.360 align:middle line:84% So we've got the receptive field area in gray. 00:23:36.360 --> 00:23:39.420 align:middle line:90% The stimulus is the orange bar. 00:23:39.420 --> 00:23:41.420 align:middle line:84% And the simple cell response, you can see, 00:23:41.420 --> 00:23:45.220 align:middle line:84% is high when the orange bar is at the right orientation 00:23:45.220 --> 00:23:47.200 align:middle line:84% and in exactly the right position. 00:23:47.200 --> 00:23:50.100 align:middle line:84% And then if you shift the orange bar a little bit to the right, 00:23:50.100 --> 00:23:52.420 align:middle line:84% the simple cell response drops off. 00:23:52.420 --> 00:23:54.560 align:middle line:84% Now, it's also orientation-selective. 00:23:54.560 --> 00:23:56.060 align:middle line:84% So if you get the wrong orientation, 00:23:56.060 --> 00:23:57.700 align:middle line:90% you also get a lower response. 00:23:57.700 --> 00:24:00.120 align:middle line:84% The complex cell response, by contrast, 00:24:00.120 --> 00:24:02.080 align:middle line:84% is high at both of the positions. 00:24:02.080 --> 00:24:05.640 align:middle line:84% But it's still orientation-selective. 00:24:05.640 --> 00:24:08.780 align:middle line:84% So this is, again, an empirical discovery 00:24:08.780 --> 00:24:10.160 align:middle line:90% in primary visual cortex. 00:24:10.160 --> 00:24:12.080 align:middle line:84% You've got these two types of neurons. 00:24:12.080 --> 00:24:14.740 align:middle line:90% 00:24:14.740 --> 00:24:18.620 align:middle line:84% And it turns out that complex cell responses 00:24:18.620 --> 00:24:24.460 align:middle line:84% can be modeled by combining even and odd simple cell responses. 00:24:24.460 --> 00:24:28.060 align:middle line:84% And when you model simple cells as Gabor functions, 00:24:28.060 --> 00:24:30.780 align:middle line:84% this turns out to be mathematically very simple. 00:24:30.780 --> 00:24:35.020 align:middle line:84% So an even receptive field would be a Gabor 00:24:35.020 --> 00:24:36.840 align:middle line:90% that has a cosine in it. 00:24:36.840 --> 00:24:38.820 align:middle line:90% So G here is a Gaussian. 00:24:38.820 --> 00:24:42.500 align:middle line:84% And that's a cosine with some spatial frequency. 00:24:42.500 --> 00:24:44.820 align:middle line:84% That's the odd function with sine. 00:24:44.820 --> 00:24:48.060 align:middle line:84% And so if you just square the responses 00:24:48.060 --> 00:24:49.780 align:middle line:84% of the even and odd receptive fields 00:24:49.780 --> 00:24:51.940 align:middle line:84% and then add them together, because 00:24:51.940 --> 00:24:53.860 align:middle line:84% of the trigonometric identity that sine 00:24:53.860 --> 00:24:56.820 align:middle line:84% squared plus cosine squared equals 1, 00:24:56.820 --> 00:24:59.100 align:middle line:84% you end up getting this very simple expression, 00:24:59.100 --> 00:25:01.940 align:middle line:84% where the sinusoids drop out of the expression. 00:25:01.940 --> 00:25:08.780 align:middle line:84% And you're just left with a Gaussian spatial dependency. 00:25:08.780 --> 00:25:09.280 align:middle line:90% All right. 00:25:09.280 --> 00:25:12.020 align:middle line:84% So this is what is often known as the energy 00:25:12.020 --> 00:25:15.420 align:middle line:90% model of complex cells. 00:25:15.420 --> 00:25:19.940 align:middle line:84% And we now have a second-- or of a third row 00:25:19.940 --> 00:25:24.900 align:middle line:84% to our primary visual cortex response graph 00:25:24.900 --> 00:25:27.900 align:middle line:90% here for edges and lines. 00:25:27.900 --> 00:25:31.340 align:middle line:84% And the bottom row is energy computed 00:25:31.340 --> 00:25:34.020 align:middle line:84% from the combination of the odd symmetric 00:25:34.020 --> 00:25:37.940 align:middle line:84% and the even symmetric responses by squaring them and summing 00:25:37.940 --> 00:25:38.440 align:middle line:90% them. 00:25:38.440 --> 00:25:41.860 align:middle line:84% And now, you get this nice little bump anytime there's 00:25:41.860 --> 00:25:42.920 align:middle line:90% an image feature. 00:25:42.920 --> 00:25:46.680 align:middle line:90% 00:25:46.680 --> 00:25:49.460 align:middle line:84% So this energy model, it provides a reasonable model 00:25:49.460 --> 00:25:51.300 align:middle line:90% of a complex cell. 00:25:51.300 --> 00:25:54.820 align:middle line:84% So there aren't any more of these excitatory and inhibitory 00:25:54.820 --> 00:25:55.940 align:middle line:90% lobes. 00:25:55.940 --> 00:26:00.940 align:middle line:84% It inherits the tuning that is endowed from simple cell input. 00:26:00.940 --> 00:26:04.020 align:middle line:84% And so I should say that what's kind of implicit in here, which 00:26:04.020 --> 00:26:07.020 align:middle line:84% I should make explicit, is the idea 00:26:07.020 --> 00:26:11.900 align:middle line:84% that the complex cell is created by combining 00:26:11.900 --> 00:26:13.680 align:middle line:90% the responses of simple cells. 00:26:13.680 --> 00:26:17.180 align:middle line:84% So just as the simple cell, we believe, 00:26:17.180 --> 00:26:20.180 align:middle line:84% could be created by wiring together 00:26:20.180 --> 00:26:22.460 align:middle line:84% center-surround receptive fields in the LGN, 00:26:22.460 --> 00:26:23.940 align:middle line:84% the idea is that the complex cell 00:26:23.940 --> 00:26:26.860 align:middle line:84% is created by wiring together simple cells that 00:26:26.860 --> 00:26:31.200 align:middle line:84% have the same orientation selectivity and different phase, 00:26:31.200 --> 00:26:34.380 align:middle line:84% or even and odd receptive fields. 00:26:34.380 --> 00:26:37.660 align:middle line:90% So that's the notion. 00:26:37.660 --> 00:26:40.020 align:middle line:84% And the consequence is that they kind of inherit 00:26:40.020 --> 00:26:41.900 align:middle line:90% some of the tuning properties-- 00:26:41.900 --> 00:26:43.620 align:middle line:90% orientation selectivity. 00:26:43.620 --> 00:26:46.980 align:middle line:84% Sometimes, there's also motion selectivity. 00:26:46.980 --> 00:26:49.120 align:middle line:84% But the response becomes nonlinear. 00:26:49.120 --> 00:26:52.020 align:middle line:90% 00:26:52.020 --> 00:26:55.360 align:middle line:84% Any questions about simple versus complex cells? 00:26:55.360 --> 00:27:08.600 align:middle line:90% 00:27:08.600 --> 00:27:13.580 align:middle line:84% So this is a picture that captures like a fairly 00:27:13.580 --> 00:27:16.830 align:middle line:90% standard model of early vision. 00:27:16.830 --> 00:27:19.190 align:middle line:84% When people talk about early vision, 00:27:19.190 --> 00:27:22.990 align:middle line:84% normally they mean roughly up to primary visual cortex. 00:27:22.990 --> 00:27:28.210 align:middle line:84% So it's capturing early vision using convolution. 00:27:28.210 --> 00:27:30.090 align:middle line:90% So we've got an image there. 00:27:30.090 --> 00:27:32.030 align:middle line:90% It's an image of beans. 00:27:32.030 --> 00:27:35.750 align:middle line:84% And then that image is convolved at each step 00:27:35.750 --> 00:27:41.110 align:middle line:84% with an operator that is embodying a particular stage 00:27:41.110 --> 00:27:42.530 align:middle line:90% of visual processing. 00:27:42.530 --> 00:27:45.310 align:middle line:84% So we've got center-surround cells here 00:27:45.310 --> 00:27:48.950 align:middle line:84% that capture what happens in the retina and the LGN, 00:27:48.950 --> 00:27:50.950 align:middle line:84% where the center-surround receptive fields are 00:27:50.950 --> 00:27:52.050 align:middle line:90% different sizes. 00:27:52.050 --> 00:27:55.310 align:middle line:84% Then there's orientation-selective filtering, 00:27:55.310 --> 00:27:59.470 align:middle line:84% and then the energy extraction step. 00:27:59.470 --> 00:28:02.830 align:middle line:84% So again, the idea is that the convolution operation 00:28:02.830 --> 00:28:07.910 align:middle line:84% is used to represent the response of a large population 00:28:07.910 --> 00:28:10.590 align:middle line:84% of neurons that have the same kind of receptive field, 00:28:10.590 --> 00:28:13.770 align:middle line:84% but positioned at different spatial locations. 00:28:13.770 --> 00:28:17.030 align:middle line:84% And so you can think of the response 00:28:17.030 --> 00:28:19.070 align:middle line:84% of a stage of the visual system kind 00:28:19.070 --> 00:28:21.053 align:middle line:90% of as a function of an image. 00:28:21.053 --> 00:28:22.470 align:middle line:84% So you have the image as an input. 00:28:22.470 --> 00:28:26.990 align:middle line:84% And you get this response out that looks like an image. 00:28:26.990 --> 00:28:30.870 align:middle line:84% And the response is high in these different places 00:28:30.870 --> 00:28:33.350 align:middle line:84% where the original image, in this case, 00:28:33.350 --> 00:28:37.710 align:middle line:84% had orientation energy that was at the right orientation 00:28:37.710 --> 00:28:41.173 align:middle line:84% for that cell, and same here, and same here. 00:28:41.173 --> 00:28:42.590 align:middle line:84% And if you look back in the image, 00:28:42.590 --> 00:28:45.470 align:middle line:84% you can probably convince yourself that's the case. 00:28:45.470 --> 00:28:48.330 align:middle line:84% But then when you get to the energy quantity, 00:28:48.330 --> 00:28:53.230 align:middle line:84% you can see that, well, the energy is high any place 00:28:53.230 --> 00:28:58.590 align:middle line:84% that either has high or very low activity in the simple cell 00:28:58.590 --> 00:29:01.950 align:middle line:84% responses, because you get the squaring operation and summing. 00:29:01.950 --> 00:29:04.350 align:middle line:84% And so that causes it to be spatially a little bit 00:29:04.350 --> 00:29:06.030 align:middle line:90% more coarse. 00:29:06.030 --> 00:29:09.550 align:middle line:84% Now, you might wonder, OK, what good is this? 00:29:09.550 --> 00:29:13.230 align:middle line:84% What does this have to do with vision? 00:29:13.230 --> 00:29:15.390 align:middle line:90% That's a legitimate question. 00:29:15.390 --> 00:29:20.190 align:middle line:84% And so I think the broader comment that I would make here 00:29:20.190 --> 00:29:24.830 align:middle line:84% is that there's a bunch of different angles at which to try 00:29:24.830 --> 00:29:26.970 align:middle line:90% to understand perception. 00:29:26.970 --> 00:29:30.310 align:middle line:84% And one of those angles is to go into a sensory system, 00:29:30.310 --> 00:29:33.910 align:middle line:84% and to look at what's there, and to try to measure what's there, 00:29:33.910 --> 00:29:35.410 align:middle line:90% and characterize that. 00:29:35.410 --> 00:29:36.990 align:middle line:84% And that's what we've been talking 00:29:36.990 --> 00:29:38.850 align:middle line:84% about over the past couple of lectures. 00:29:38.850 --> 00:29:42.390 align:middle line:84% And that actually had some success 00:29:42.390 --> 00:29:44.650 align:middle line:84% in the first few stages of the visual system, 00:29:44.650 --> 00:29:47.630 align:middle line:84% in the sense that the measurements often-- they 00:29:47.630 --> 00:29:49.770 align:middle line:84% ended up being mathematically interpretable. 00:29:49.770 --> 00:29:52.910 align:middle line:84% And people were able to come up with mathematical descriptions 00:29:52.910 --> 00:29:56.390 align:middle line:84% of the responses of the neurons that 00:29:56.390 --> 00:30:00.030 align:middle line:84% were pretty good, in the sense of being able to predict 00:30:00.030 --> 00:30:01.830 align:middle line:90% the responses of the neurons. 00:30:01.830 --> 00:30:05.090 align:middle line:84% Now, does that tell us how these things help us see? 00:30:05.090 --> 00:30:05.970 align:middle line:90% No, they don't. 00:30:05.970 --> 00:30:08.370 align:middle line:84% These are just descriptions of parts of the system. 00:30:08.370 --> 00:30:12.575 align:middle line:84% But they are made with the hope and expectation 00:30:12.575 --> 00:30:14.950 align:middle line:84% that they will end up being relevant to our understanding 00:30:14.950 --> 00:30:15.610 align:middle line:90% of how we see. 00:30:15.610 --> 00:30:17.350 align:middle line:84% Now, the other way to get at perception 00:30:17.350 --> 00:30:19.990 align:middle line:84% is to start with perception itself, 00:30:19.990 --> 00:30:24.117 align:middle line:84% to think about the things that we can do, and to measure that, 00:30:24.117 --> 00:30:25.950 align:middle line:84% and to try to come up with theories of that, 00:30:25.950 --> 00:30:28.790 align:middle line:84% and then eventually try to connect that back to neurons. 00:30:28.790 --> 00:30:31.350 align:middle line:84% So remember, we talked about different levels 00:30:31.350 --> 00:30:32.883 align:middle line:90% of understanding. 00:30:32.883 --> 00:30:35.050 align:middle line:84% There's the level of the computation, the algorithm, 00:30:35.050 --> 00:30:36.830 align:middle line:90% and the implementation. 00:30:36.830 --> 00:30:40.030 align:middle line:84% At various points in the history of this field, 00:30:40.030 --> 00:30:42.398 align:middle line:84% sometimes people start with the implementation. 00:30:42.398 --> 00:30:44.190 align:middle line:84% So they're actually measuring from neurons, 00:30:44.190 --> 00:30:45.770 align:middle line:90% trying to document that. 00:30:45.770 --> 00:30:48.930 align:middle line:84% And some people really believe that that's where you start. 00:30:48.930 --> 00:30:51.010 align:middle line:84% And that's the best place to get rolling. 00:30:51.010 --> 00:30:53.183 align:middle line:84% In other cases, people actually start 00:30:53.183 --> 00:30:54.850 align:middle line:84% by trying to understand the computation, 00:30:54.850 --> 00:30:57.017 align:middle line:84% trying to understand the function that the system is 00:30:57.017 --> 00:31:01.190 align:middle line:84% carrying out and how to understand that 00:31:01.190 --> 00:31:02.910 align:middle line:84% in mathematical terms, and then worrying 00:31:02.910 --> 00:31:04.590 align:middle line:90% about the implementation later. 00:31:04.590 --> 00:31:07.470 align:middle line:84% And different sections of this course 00:31:07.470 --> 00:31:09.710 align:middle line:84% will have more of one perspective 00:31:09.710 --> 00:31:10.910 align:middle line:90% and more of the other. 00:31:10.910 --> 00:31:13.750 align:middle line:84% And ideally, all of these things are going to come together. 00:31:13.750 --> 00:31:16.230 align:middle line:84% And they don't always, at least not yet. 00:31:16.230 --> 00:31:20.730 align:middle line:84% But hopefully, eventually, that will be the case. 00:31:20.730 --> 00:31:23.350 align:middle line:90% So big ideas so far-- 00:31:23.350 --> 00:31:26.830 align:middle line:84% we've got these stages of linear filtering in the retina 00:31:26.830 --> 00:31:29.290 align:middle line:84% and in primary visual cortex with simple cells, 00:31:29.290 --> 00:31:33.030 align:middle line:84% and then this one kind of non-linear operation 00:31:33.030 --> 00:31:35.310 align:middle line:84% that happens in complex cells that we can capture 00:31:35.310 --> 00:31:37.670 align:middle line:90% with the notion of energy. 00:31:37.670 --> 00:31:39.650 align:middle line:84% Any questions about that before I move on? 00:31:39.650 --> 00:31:48.150 align:middle line:90% 00:31:48.150 --> 00:31:49.190 align:middle line:90% Yeah? 00:31:49.190 --> 00:31:52.030 align:middle line:84% STUDENT: Can you repeat again what 00:31:52.030 --> 00:31:56.790 align:middle line:84% energy expresses in relation to the original image, 00:31:56.790 --> 00:32:00.150 align:middle line:90% with the beans example? 00:32:00.150 --> 00:32:02.070 align:middle line:84% JOSH MCDERMOTT: Yeah, so the energy 00:32:02.070 --> 00:32:08.670 align:middle line:90% will be high whenever there is-- 00:32:08.670 --> 00:32:11.710 align:middle line:84% whenever an oriented filter that's 00:32:11.710 --> 00:32:16.590 align:middle line:84% at the orientation, which you're measuring the energy, 00:32:16.590 --> 00:32:19.950 align:middle line:84% whenever an oriented filter, whether it's even or odd, 00:32:19.950 --> 00:32:22.130 align:middle line:90% provides a large response. 00:32:22.130 --> 00:32:27.110 align:middle line:90% 00:32:27.110 --> 00:32:32.030 align:middle line:84% Yeah, so I think, here, see how there's a white blob here? 00:32:32.030 --> 00:32:35.630 align:middle line:84% And it's because there's a oriented bean that's 00:32:35.630 --> 00:32:37.110 align:middle line:90% in that place in the image. 00:32:37.110 --> 00:32:41.750 align:middle line:90% 00:32:41.750 --> 00:32:44.390 align:middle line:84% So there's a place where the edge-- unfortunately, I 00:32:44.390 --> 00:32:46.450 align:middle line:90% need a point here. 00:32:46.450 --> 00:32:49.010 align:middle line:84% But there's a place where it goes from dark to light, 00:32:49.010 --> 00:32:51.343 align:middle line:84% and then another place where it goes from light to dark. 00:32:51.343 --> 00:32:53.230 align:middle line:90% And so those will produce-- 00:32:53.230 --> 00:32:55.710 align:middle line:84% one of those would produce a positive response 00:32:55.710 --> 00:32:57.050 align:middle line:90% in an odd receptive field. 00:32:57.050 --> 00:32:58.230 align:middle line:84% The other would produce a negative response 00:32:58.230 --> 00:32:59.355 align:middle line:90% in the odd receptive field. 00:32:59.355 --> 00:33:01.282 align:middle line:84% But the energy squares that and sums that. 00:33:01.282 --> 00:33:02.990 align:middle line:84% So it doesn't really care about the sign. 00:33:02.990 --> 00:33:05.870 align:middle line:84% It's just telling you there's stuff there 00:33:05.870 --> 00:33:08.270 align:middle line:90% that's at that orientation. 00:33:08.270 --> 00:33:10.870 align:middle line:84% So that's one way to maybe intuitively describe it. 00:33:10.870 --> 00:33:11.563 align:middle line:90% Yeah? 00:33:11.563 --> 00:33:13.230 align:middle line:84% STUDENT: So I think it's true that there 00:33:13.230 --> 00:33:18.990 align:middle line:84% are center-around neurons in the LGN that detect color. 00:33:18.990 --> 00:33:22.350 align:middle line:84% How does that get incorporated in the orientation and energy 00:33:22.350 --> 00:33:23.400 align:middle line:90% extraction steps? 00:33:23.400 --> 00:33:25.650 align:middle line:84% JOSH MCDERMOTT: Well, in this model, there's no color. 00:33:25.650 --> 00:33:28.310 align:middle line:90% This is a black and white model. 00:33:28.310 --> 00:33:30.150 align:middle line:84% In the visual system, I mean, we're 00:33:30.150 --> 00:33:33.510 align:middle line:84% going to talk a little bit more about that when we get to color. 00:33:33.510 --> 00:33:34.890 align:middle line:90% But that's part of the story. 00:33:34.890 --> 00:33:38.870 align:middle line:84% And in fact, I mean, one of the things that has been debated 00:33:38.870 --> 00:33:43.910 align:middle line:84% is the extent to which orientation-selective mechanisms 00:33:43.910 --> 00:33:48.950 align:middle line:84% are also color-selective, or whether color vision is maybe 00:33:48.950 --> 00:33:51.058 align:middle line:90% a slightly different thing. 00:33:51.058 --> 00:33:52.350 align:middle line:90% But we'll see evidence of that. 00:33:52.350 --> 00:33:55.110 align:middle line:84% I think we now mostly know that you 00:33:55.110 --> 00:33:57.590 align:middle line:84% can find color selectivity in conjunction 00:33:57.590 --> 00:34:00.995 align:middle line:90% with orientation selectivity. 00:34:00.995 --> 00:34:02.870 align:middle line:84% Yeah, there are some interesting differences, 00:34:02.870 --> 00:34:05.350 align:middle line:84% like the spatial resolution of color vision 00:34:05.350 --> 00:34:07.230 align:middle line:90% tends to be pretty poor. 00:34:07.230 --> 00:34:10.830 align:middle line:84% So there are indications that there is some functional 00:34:10.830 --> 00:34:15.710 align:middle line:84% separation between certain kinds of high resolution, pattern 00:34:15.710 --> 00:34:17.510 align:middle line:90% analysis, and color. 00:34:17.510 --> 00:34:18.690 align:middle line:90% But more to come on that. 00:34:18.690 --> 00:34:24.309 align:middle line:90% 00:34:24.309 --> 00:34:26.409 align:middle line:90% So you get to the visual cortex. 00:34:26.409 --> 00:34:28.830 align:middle line:84% And you see some new stuff that was not 00:34:28.830 --> 00:34:33.826 align:middle line:84% present in the responses of the retina or the thalamus. 00:34:33.826 --> 00:34:35.909 align:middle line:84% The first example of that we've been talking about 00:34:35.909 --> 00:34:38.870 align:middle line:90% is orientation selectivity. 00:34:38.870 --> 00:34:42.429 align:middle line:84% Another example is that you find neurons 00:34:42.429 --> 00:34:46.070 align:middle line:90% that respond to both eyes. 00:34:46.070 --> 00:34:50.550 align:middle line:84% So prior to primary visual cortex, all of the cells 00:34:50.550 --> 00:34:53.469 align:middle line:84% get input from either the right eye or the left eye. 00:34:53.469 --> 00:34:55.510 align:middle line:84% So of course, in the retina, you're 00:34:55.510 --> 00:34:57.350 align:middle line:84% either in the right eye or the left eye. 00:34:57.350 --> 00:35:02.350 align:middle line:84% But then you get to the LGN and different layers of the LGN 00:35:02.350 --> 00:35:04.470 align:middle line:84% are carrying information from either the right 00:35:04.470 --> 00:35:05.830 align:middle line:90% or the left eye. 00:35:05.830 --> 00:35:07.890 align:middle line:84% But then in primary visual cortex, 00:35:07.890 --> 00:35:11.150 align:middle line:84% you see neurons that get input from both eyes 00:35:11.150 --> 00:35:14.230 align:middle line:84% and that will respond more when both eyes are stimulated. 00:35:14.230 --> 00:35:17.750 align:middle line:84% And those are called binocular neurons. 00:35:17.750 --> 00:35:22.230 align:middle line:84% But it's nonetheless the case that many cells are still driven 00:35:22.230 --> 00:35:25.470 align:middle line:90% more by one eye than the other. 00:35:25.470 --> 00:35:30.470 align:middle line:84% And so this is a histogram in two different species 00:35:30.470 --> 00:35:36.010 align:middle line:84% that plots the extent to which there is ocular dominance. 00:35:36.010 --> 00:35:39.190 align:middle line:84% So again, I don't know why this is so blurry. 00:35:39.190 --> 00:35:42.510 align:middle line:84% But on this end would indicate that the neuron is dominated 00:35:42.510 --> 00:35:44.270 align:middle line:90% by the contralateral eye-- 00:35:44.270 --> 00:35:45.993 align:middle line:84% contralateral is on opposite side. 00:35:45.993 --> 00:35:47.910 align:middle line:84% On this end, it would mean that it's dominated 00:35:47.910 --> 00:35:49.130 align:middle line:90% by the ipsilateral eye. 00:35:49.130 --> 00:35:51.390 align:middle line:84% And the point is that there's a lot of mass 00:35:51.390 --> 00:35:53.430 align:middle line:90% out here and out here. 00:35:53.430 --> 00:35:56.390 align:middle line:84% So a lot of neurons are dominated by one eye. 00:35:56.390 --> 00:36:00.510 align:middle line:84% And that shows up in another structural feature 00:36:00.510 --> 00:36:03.750 align:middle line:84% of the visual cortex, which is ocular dominance columns. 00:36:03.750 --> 00:36:05.710 align:middle line:84% And so that is that, again, if you're 00:36:05.710 --> 00:36:10.870 align:middle line:84% looking down at the cortex-- this is the cortical surface. 00:36:10.870 --> 00:36:14.550 align:middle line:84% And this is the result of an experiment where people 00:36:14.550 --> 00:36:17.590 align:middle line:84% were able to image the activity in the visual cortex 00:36:17.590 --> 00:36:19.910 align:middle line:84% while an animal had one eye open. 00:36:19.910 --> 00:36:23.370 align:middle line:84% So one eye is stimulated and the other eye is closed. 00:36:23.370 --> 00:36:26.470 align:middle line:84% And you see this patchy pattern of activation, 00:36:26.470 --> 00:36:29.350 align:middle line:84% where there are bits of the cortex that are a lot more 00:36:29.350 --> 00:36:30.610 align:middle line:90% active than others. 00:36:30.610 --> 00:36:31.850 align:middle line:90% That's ocular dominance. 00:36:31.850 --> 00:36:34.150 align:middle line:84% So the response in the white bits of the cortex 00:36:34.150 --> 00:36:39.230 align:middle line:84% is driven by the eye that's open predominantly. 00:36:39.230 --> 00:36:42.170 align:middle line:84% So we've got orientation columns and ocular dominance columns. 00:36:42.170 --> 00:36:49.030 align:middle line:84% And they're interspersed in some kind of way. 00:36:49.030 --> 00:36:51.347 align:middle line:84% So this is from optical imaging in animals. 00:36:51.347 --> 00:36:52.930 align:middle line:84% You can also see this stuff in humans. 00:36:52.930 --> 00:36:56.550 align:middle line:84% This is an fMRI experiment that measured ocular dominance 00:36:56.550 --> 00:36:57.710 align:middle line:90% columns. 00:36:57.710 --> 00:37:01.830 align:middle line:84% This is two particular subjects, and then overlaid them 00:37:01.830 --> 00:37:05.210 align:middle line:90% on orientation columns. 00:37:05.210 --> 00:37:10.510 align:middle line:84% So these columns gave rise to this cartoon model 00:37:10.510 --> 00:37:13.397 align:middle line:84% of the visual cortex, where the idea 00:37:13.397 --> 00:37:15.230 align:middle line:84% is that you could think of the visual cortex 00:37:15.230 --> 00:37:21.207 align:middle line:84% as a set of hypercolumns, where each hypercolumn contains all 00:37:21.207 --> 00:37:22.790 align:middle line:84% of the different kinds of measurements 00:37:22.790 --> 00:37:25.590 align:middle line:84% that you would want to make at one point in space. 00:37:25.590 --> 00:37:30.750 align:middle line:84% And those measurements are different orientations, 00:37:30.750 --> 00:37:32.350 align:middle line:90% as you can see here. 00:37:32.350 --> 00:37:35.550 align:middle line:84% And also, the left and the right eye. 00:37:35.550 --> 00:37:38.190 align:middle line:84% And this is also sometimes referred to 00:37:38.190 --> 00:37:39.610 align:middle line:90% as the ice cube model. 00:37:39.610 --> 00:37:42.830 align:middle line:90% 00:37:42.830 --> 00:37:46.130 align:middle line:84% So how does all of this hooked up to the LGN? 00:37:46.130 --> 00:37:47.890 align:middle line:90% So remember, this is the LGN. 00:37:47.890 --> 00:37:49.810 align:middle line:90% The LGN's got these six layers. 00:37:49.810 --> 00:37:53.230 align:middle line:84% The top four are the parvocellular layers. 00:37:53.230 --> 00:37:56.310 align:middle line:84% The bottom two are the magnocellular layers. 00:37:56.310 --> 00:38:00.680 align:middle line:84% This is a picture, though, that shows the labeling from one eye. 00:38:00.680 --> 00:38:03.600 align:middle line:84% And so you can see that alternate layers 00:38:03.600 --> 00:38:06.000 align:middle line:90% are getting input from one eye. 00:38:06.000 --> 00:38:08.280 align:middle line:84% So each layer gets input from either the left eye 00:38:08.280 --> 00:38:10.240 align:middle line:90% or the right eye. 00:38:10.240 --> 00:38:14.380 align:middle line:84% And that one, that one, that one are getting it from one eye, 00:38:14.380 --> 00:38:15.420 align:middle line:90% not sure which one. 00:38:15.420 --> 00:38:17.840 align:middle line:90% 00:38:17.840 --> 00:38:18.340 align:middle line:90% All right. 00:38:18.340 --> 00:38:24.140 align:middle line:84% So this is showing how the LGN provides input to V1. 00:38:24.140 --> 00:38:28.320 align:middle line:84% So in general, most cortical areas 00:38:28.320 --> 00:38:32.460 align:middle line:84% have the input coming in at layer 4. 00:38:32.460 --> 00:38:35.940 align:middle line:84% So layer 4 is the input layer for the cortex, typically. 00:38:35.940 --> 00:38:40.360 align:middle line:84% And in V1, a lot of the input comes into layer 4C. 00:38:40.360 --> 00:38:43.560 align:middle line:84% And so we've got these two layers of the LGN-- one 00:38:43.560 --> 00:38:45.680 align:middle line:84% for the right eye one for the left eye. 00:38:45.680 --> 00:38:49.320 align:middle line:84% And then those have these alternating spatial projections 00:38:49.320 --> 00:38:51.780 align:middle line:84% that correspond to ocular dominance columns. 00:38:51.780 --> 00:38:56.360 align:middle line:90% 00:38:56.360 --> 00:38:58.720 align:middle line:84% The one other wrinkle is that-- so 00:38:58.720 --> 00:39:02.760 align:middle line:84% remember how we talked about how we've got parvocellular layers 00:39:02.760 --> 00:39:03.883 align:middle line:90% and magnocellular layers. 00:39:03.883 --> 00:39:05.800 align:middle line:84% Remember, we talked about these lesion studies 00:39:05.800 --> 00:39:09.320 align:middle line:84% where, if you selectively lesion the parvocellular layers, 00:39:09.320 --> 00:39:11.540 align:middle line:84% you get deficits in color perception. 00:39:11.540 --> 00:39:14.320 align:middle line:84% If you selectively lesion the magnocellular layers, 00:39:14.320 --> 00:39:17.840 align:middle line:84% you get deficits in motion perception. 00:39:17.840 --> 00:39:21.520 align:middle line:84% And so, unsurprisingly, the responses 00:39:21.520 --> 00:39:24.040 align:middle line:84% from the parvocellular and the magnocellular layers 00:39:24.040 --> 00:39:25.300 align:middle line:90% remain segregated. 00:39:25.300 --> 00:39:29.040 align:middle line:84% And they, in fact, project to different sublayers of this 00:39:29.040 --> 00:39:30.440 align:middle line:90% layer 4C. 00:39:30.440 --> 00:39:33.820 align:middle line:84% So 4C alpha is where the magnocellular layers end up. 00:39:33.820 --> 00:39:37.560 align:middle line:84% And 4C beta is where the parvocellular layers end up. 00:39:37.560 --> 00:39:40.160 align:middle line:84% So in general, the central theme here 00:39:40.160 --> 00:39:43.760 align:middle line:84% is, especially in the early stages of the visual system, 00:39:43.760 --> 00:39:46.860 align:middle line:84% there's really fairly striking segregation of function. 00:39:46.860 --> 00:39:48.860 align:middle line:84% You've got these distinct cell classes. 00:39:48.860 --> 00:39:51.260 align:middle line:84% There's a small number of cell classes in the retina. 00:39:51.260 --> 00:39:52.760 align:middle line:90% They have different properties. 00:39:52.760 --> 00:39:55.040 align:middle line:84% Those project to different parts of the ln that 00:39:55.040 --> 00:39:58.220 align:middle line:84% have different properties, different impact on behavior. 00:39:58.220 --> 00:40:01.200 align:middle line:84% And then that remains segregated through V1 00:40:01.200 --> 00:40:02.940 align:middle line:90% and also a little bit beyond. 00:40:02.940 --> 00:40:06.480 align:middle line:90% 00:40:06.480 --> 00:40:10.840 align:middle line:84% So a lot of the input comes in here at layer 4C. 00:40:10.840 --> 00:40:12.760 align:middle line:90% And then this is just a diagram. 00:40:12.760 --> 00:40:14.620 align:middle line:84% I don't expect people to memorize this. 00:40:14.620 --> 00:40:21.360 align:middle line:84% This is just included for cultural literacy. 00:40:21.360 --> 00:40:26.100 align:middle line:84% You can see connections between different layers of V1. 00:40:26.100 --> 00:40:28.480 align:middle line:84% So you get the input coming in here at layer 4, 00:40:28.480 --> 00:40:30.277 align:middle line:84% and then projections to other layers. 00:40:30.277 --> 00:40:32.360 align:middle line:84% And then you can see there are certain layers that 00:40:32.360 --> 00:40:36.560 align:middle line:84% are like the output layers that tend to send responses 00:40:36.560 --> 00:40:38.300 align:middle line:90% to other cortical areas. 00:40:38.300 --> 00:40:40.400 align:middle line:84% There's other layers that tend to send responses 00:40:40.400 --> 00:40:42.020 align:middle line:84% to non-cortical brain structures. 00:40:42.020 --> 00:40:46.240 align:middle line:84% So there's just a lot of interesting anatomical 00:40:46.240 --> 00:40:48.080 align:middle line:90% segregation. 00:40:48.080 --> 00:40:51.832 align:middle line:84% And every decade or so, more is learned about this stuff. 00:40:51.832 --> 00:40:53.540 align:middle line:84% And so the pictures get more complicated. 00:40:53.540 --> 00:40:56.960 align:middle line:84% So this was the diagram that you would find in a textbook back 00:40:56.960 --> 00:41:00.520 align:middle line:84% in 1995, where pretty much people just 00:41:00.520 --> 00:41:02.880 align:middle line:84% talked about magnocellular and parvocellular layers 00:41:02.880 --> 00:41:05.120 align:middle line:90% from the LGN. 00:41:05.120 --> 00:41:06.660 align:middle line:90% This is a more recent diagram. 00:41:06.660 --> 00:41:09.160 align:middle line:84% And so now, there are these koniocellular cells 00:41:09.160 --> 00:41:12.040 align:middle line:84% that are in between the parvocellular layers 00:41:12.040 --> 00:41:13.920 align:middle line:90% and the magnocellular layers. 00:41:13.920 --> 00:41:16.600 align:middle line:84% And those project to a different place. 00:41:16.600 --> 00:41:19.720 align:middle line:84% So as time goes on, these things will probably 00:41:19.720 --> 00:41:21.140 align:middle line:90% get more and more complicated. 00:41:21.140 --> 00:41:25.710 align:middle line:90% 00:41:25.710 --> 00:41:27.600 align:middle line:84% Another thing that you see in the cortex 00:41:27.600 --> 00:41:28.740 align:middle line:90% is direction selectivity. 00:41:28.740 --> 00:41:30.520 align:middle line:84% So here's an example of a cell that 00:41:30.520 --> 00:41:33.440 align:middle line:84% is selective for the direction of motion. 00:41:33.440 --> 00:41:38.883 align:middle line:84% So the dashed rectangle here is the approximate location 00:41:38.883 --> 00:41:40.800 align:middle line:84% of the receptive field, the part of space that 00:41:40.800 --> 00:41:42.220 align:middle line:90% causes the neuron to respond. 00:41:42.220 --> 00:41:44.600 align:middle line:84% There's a bar that's being moved back and forth. 00:41:44.600 --> 00:41:47.560 align:middle line:84% You can see that there is a preferred orientation, which 00:41:47.560 --> 00:41:49.580 align:middle line:90% is this oblique orientation. 00:41:49.580 --> 00:41:52.280 align:middle line:84% But you can also see that the response is much bigger when 00:41:52.280 --> 00:41:53.880 align:middle line:84% it moves in one direction than when 00:41:53.880 --> 00:41:56.560 align:middle line:90% it moves in another direction. 00:41:56.560 --> 00:41:59.480 align:middle line:84% So direction selectivity is pretty common 00:41:59.480 --> 00:42:03.560 align:middle line:84% and, again, present in layers that 00:42:03.560 --> 00:42:08.220 align:middle line:84% tend to get input from magnocellular parts of the LGN. 00:42:08.220 --> 00:42:11.060 align:middle line:90% 00:42:11.060 --> 00:42:12.620 align:middle line:84% And there's other complicated stuff. 00:42:12.620 --> 00:42:16.880 align:middle line:84% This is a property that is often called endstopping. 00:42:16.880 --> 00:42:19.520 align:middle line:84% So if you have an orientation-selective neuron 00:42:19.520 --> 00:42:21.720 align:middle line:84% that's, in this case, selective or horizontal, 00:42:21.720 --> 00:42:26.060 align:middle line:84% if you make the line too long, the response actually decreases. 00:42:26.060 --> 00:42:28.840 align:middle line:84% So there's an inhibitory surround 00:42:28.840 --> 00:42:31.360 align:middle line:84% on top of the orientation-selective response. 00:42:31.360 --> 00:42:32.960 align:middle line:84% So that's also a pretty common thing 00:42:32.960 --> 00:42:36.940 align:middle line:84% that you see in the cortex, is inhibitory surrounds, 00:42:36.940 --> 00:42:39.460 align:middle line:84% even though it's not just a center-surround receptive field. 00:42:39.460 --> 00:42:42.040 align:middle line:84% You have some kind of complicated tuning, and then 00:42:42.040 --> 00:42:45.900 align:middle line:90% a surround on top of that. 00:42:45.900 --> 00:42:51.160 align:middle line:84% So the key take-home themes that you should take away 00:42:51.160 --> 00:42:54.880 align:middle line:84% from this lecture are the idea of retinotopy 00:42:54.880 --> 00:42:58.440 align:middle line:84% as an organizational principle for the visual system 00:42:58.440 --> 00:43:01.160 align:middle line:84% and cortical magnification-- so again, in the retina, 00:43:01.160 --> 00:43:03.680 align:middle line:84% we've got the foveal organization 00:43:03.680 --> 00:43:04.740 align:middle line:90% of the receptor lattice. 00:43:04.740 --> 00:43:06.960 align:middle line:84% And that translates into cortical magnification 00:43:06.960 --> 00:43:08.880 align:middle line:90% when you get to the cortex-- 00:43:08.880 --> 00:43:11.880 align:middle line:84% orientation selectivity and the idea 00:43:11.880 --> 00:43:14.680 align:middle line:84% of how you would create orientation selectivity 00:43:14.680 --> 00:43:17.040 align:middle line:84% in a neural network with input from center 00:43:17.040 --> 00:43:20.380 align:middle line:84% surround receptive fields, simple versus complex cells-- 00:43:20.380 --> 00:43:23.560 align:middle line:84% so simple cells are approximately linear. 00:43:23.560 --> 00:43:26.320 align:middle line:84% The receptive field consists of excitatory and inhibitory 00:43:26.320 --> 00:43:28.800 align:middle line:84% regions that are spatially localized. 00:43:28.800 --> 00:43:30.620 align:middle line:90% Complex cells are nonlinear. 00:43:30.620 --> 00:43:35.400 align:middle line:84% They don't have these distinct spatial subregions-- 00:43:35.400 --> 00:43:37.960 align:middle line:84% the idea of population codes and how 00:43:37.960 --> 00:43:39.740 align:middle line:84% they are used to infer aftereffects 00:43:39.740 --> 00:43:42.960 align:middle line:84% and how after effects provide evidence for population codes-- 00:43:42.960 --> 00:43:45.680 align:middle line:84% the idea that if you analyze a population of neurons, 00:43:45.680 --> 00:43:48.320 align:middle line:84% you can estimate stimulus properties-- 00:43:48.320 --> 00:43:52.280 align:middle line:84% the use of convolution to simulate a population of neurons 00:43:52.280 --> 00:43:54.120 align:middle line:84% that has the same kind of receptive field 00:43:54.120 --> 00:43:57.120 align:middle line:84% at different spatial positions, columnar 00:43:57.120 --> 00:44:00.440 align:middle line:84% structure as a central organizational principle 00:44:00.440 --> 00:44:03.480 align:middle line:84% of the brain, these anatomical regularities 00:44:03.480 --> 00:44:06.880 align:middle line:84% that we've been talking about, about how the connections are 00:44:06.880 --> 00:44:11.055 align:middle line:84% very non-random, and then the zoo of response properties 00:44:11.055 --> 00:44:13.680 align:middle line:84% that emerge in the cortex, where there's a whole bunch of stuff 00:44:13.680 --> 00:44:16.460 align:middle line:84% that you see in primary visual cortex that's 00:44:16.460 --> 00:44:20.860 align:middle line:84% not evident or much less evident in the retina and the thalamus. 00:44:20.860 --> 00:44:23.240 align:middle line:84% So we talked about orientation selectivity, 00:44:23.240 --> 00:44:27.840 align:middle line:84% binocularity, direction selectivity, endstopping, 00:44:27.840 --> 00:44:29.200 align:middle line:90% stuff like that. 00:44:29.200 --> 00:44:32.100 align:middle line:84% Any questions on primary visual cortex? 00:44:32.100 --> 00:44:36.800 align:middle line:90% 00:44:36.800 --> 00:44:40.840 align:middle line:84% So we've just been talking about how the receptive fields 00:44:40.840 --> 00:44:43.960 align:middle line:84% that you see in V1, they vary in a bunch of different respects-- 00:44:43.960 --> 00:44:47.280 align:middle line:84% orientation tuning, phase, even versus odd, endstopping, 00:44:47.280 --> 00:44:48.940 align:middle line:84% binocularity, direction selectivity, 00:44:48.940 --> 00:44:50.600 align:middle line:90% whether it's simple or complex. 00:44:50.600 --> 00:44:53.360 align:middle line:84% And they also vary in what's called spatial scale. 00:44:53.360 --> 00:44:59.480 align:middle line:84% So this is a picture that shows receptive fields measured 00:44:59.480 --> 00:45:03.600 align:middle line:84% at a given point in cortex that represents 00:45:03.600 --> 00:45:04.840 align:middle line:90% a particular position. 00:45:04.840 --> 00:45:07.900 align:middle line:84% So each one of these boxes represents a receptive field. 00:45:07.900 --> 00:45:11.240 align:middle line:84% So it's roughly the region of space 00:45:11.240 --> 00:45:13.680 align:middle line:84% that drives the response of the neuron. 00:45:13.680 --> 00:45:16.580 align:middle line:84% And so the point is that the boxes are different sizes. 00:45:16.580 --> 00:45:18.000 align:middle line:84% So some of them are kind of small. 00:45:18.000 --> 00:45:20.160 align:middle line:84% And some of them are kind of big. 00:45:20.160 --> 00:45:22.400 align:middle line:84% So in any given region of the cortex, 00:45:22.400 --> 00:45:25.500 align:middle line:84% the receptive fields will all be at pretty much the same place. 00:45:25.500 --> 00:45:29.520 align:middle line:84% But there's a fair bit of variation in their size. 00:45:29.520 --> 00:45:30.500 align:middle line:90% So what does this mean? 00:45:30.500 --> 00:45:35.280 align:middle line:84% Well, remember back when we were talking about sound and hearing? 00:45:35.280 --> 00:45:37.840 align:middle line:84% We talked about Fourier's theorem and the fact 00:45:37.840 --> 00:45:41.440 align:middle line:84% that Fourier's theorem shows that any signal can 00:45:41.440 --> 00:45:44.480 align:middle line:84% be written as a sum of sine waves of different frequencies 00:45:44.480 --> 00:45:45.440 align:middle line:90% and phases. 00:45:45.440 --> 00:45:47.000 align:middle line:84% And so we were talking about this 00:45:47.000 --> 00:45:52.640 align:middle line:84% in the context of sound, where this dimension was time 00:45:52.640 --> 00:45:55.140 align:middle line:90% and this dimension was pressure. 00:45:55.140 --> 00:45:57.280 align:middle line:84% But Fourier's theorem is just a general fact 00:45:57.280 --> 00:46:00.060 align:middle line:90% about any kind of signal. 00:46:00.060 --> 00:46:03.680 align:middle line:90% 00:46:03.680 --> 00:46:07.120 align:middle line:84% So an image is just an array of numbers. 00:46:07.120 --> 00:46:07.980 align:middle line:90% So it's a signal. 00:46:07.980 --> 00:46:12.040 align:middle line:84% Fourier's theorem applies just as it does to sound. 00:46:12.040 --> 00:46:13.920 align:middle line:84% And so it follows that any image can 00:46:13.920 --> 00:46:16.160 align:middle line:84% be decomposed into a sum of sine wave 00:46:16.160 --> 00:46:19.240 align:middle line:84% gratings of different orientations, frequencies, 00:46:19.240 --> 00:46:21.120 align:middle line:90% and phases. 00:46:21.120 --> 00:46:22.800 align:middle line:84% And the different spatial frequencies 00:46:22.800 --> 00:46:25.600 align:middle line:84% capture what are called different scales of image 00:46:25.600 --> 00:46:27.935 align:middle line:90% structure. 00:46:27.935 --> 00:46:29.060 align:middle line:90% So what do we mean by that? 00:46:29.060 --> 00:46:33.200 align:middle line:84% Well, this is a sinusoidal component of an image. 00:46:33.200 --> 00:46:35.320 align:middle line:90% It's a sinusoidal grating. 00:46:35.320 --> 00:46:38.360 align:middle line:84% So what that means is that the image intensity varies 00:46:38.360 --> 00:46:44.032 align:middle line:84% sinusoidally, in this case across the horizontal dimension. 00:46:44.032 --> 00:46:45.740 align:middle line:84% So back when we were talking about sound, 00:46:45.740 --> 00:46:48.200 align:middle line:84% I was telling you that the y-axis was pressure 00:46:48.200 --> 00:46:50.100 align:middle line:90% and the x-axis was time. 00:46:50.100 --> 00:46:51.780 align:middle line:84% Now that we're talking about images, 00:46:51.780 --> 00:46:55.280 align:middle line:84% the y-axis will be intensity and the x-axis 00:46:55.280 --> 00:46:59.820 align:middle line:84% would be space or position in the image. 00:46:59.820 --> 00:47:01.680 align:middle line:90% So here's one sine wave grating. 00:47:01.680 --> 00:47:04.800 align:middle line:84% These are gratings that have different spatial frequencies. 00:47:04.800 --> 00:47:09.960 align:middle line:84% So you can see that the rate at which the intensity varies 00:47:09.960 --> 00:47:13.800 align:middle line:84% across space is different in the three cases. 00:47:13.800 --> 00:47:15.760 align:middle line:84% So spatial frequency is typically 00:47:15.760 --> 00:47:18.200 align:middle line:90% measured in cycles per degree. 00:47:18.200 --> 00:47:21.180 align:middle line:84% And so this would be a low spatial frequency grating. 00:47:21.180 --> 00:47:23.620 align:middle line:84% And this would be a higher spatial frequency grating. 00:47:23.620 --> 00:47:27.005 align:middle line:90% 00:47:27.005 --> 00:47:29.660 align:middle line:84% The grating can also vary in phase. 00:47:29.660 --> 00:47:33.380 align:middle line:84% So these gratings have the same frequency, but different phases. 00:47:33.380 --> 00:47:35.280 align:middle line:84% So the exact position of the ripples 00:47:35.280 --> 00:47:38.280 align:middle line:90% is different in the three cases. 00:47:38.280 --> 00:47:41.800 align:middle line:84% And so you can take these sine wave grating images 00:47:41.800 --> 00:47:42.860 align:middle line:90% and you can add them. 00:47:42.860 --> 00:47:45.920 align:middle line:90% So here's one grating. 00:47:45.920 --> 00:47:47.335 align:middle line:84% And here's another grating that's 00:47:47.335 --> 00:47:49.460 align:middle line:84% three times the spatial frequency of the first one. 00:47:49.460 --> 00:47:50.293 align:middle line:90% And you add them up. 00:47:50.293 --> 00:47:52.140 align:middle line:84% And you get this more complicated thing. 00:47:52.140 --> 00:47:56.180 align:middle line:90% 00:47:56.180 --> 00:47:58.720 align:middle line:84% So here's what happens when you add up 00:47:58.720 --> 00:48:01.740 align:middle line:84% these two gratings in different phase relationships. 00:48:01.740 --> 00:48:05.260 align:middle line:84% So these are the same two gratings, 00:48:05.260 --> 00:48:07.140 align:middle line:90% but with different phases. 00:48:07.140 --> 00:48:10.660 align:middle line:84% And so here's the image that results in these two cases. 00:48:10.660 --> 00:48:13.100 align:middle line:84% And then here's a slice through the image. 00:48:13.100 --> 00:48:15.480 align:middle line:84% So now, again, the y-axis is intensity 00:48:15.480 --> 00:48:17.020 align:middle line:90% and the x-axis is space. 00:48:17.020 --> 00:48:19.170 align:middle line:84% And so it's a little easier to inspect. 00:48:19.170 --> 00:48:20.920 align:middle line:84% And so you get these different shapes that 00:48:20.920 --> 00:48:24.200 align:middle line:90% result from different phases. 00:48:24.200 --> 00:48:24.740 align:middle line:90% All right. 00:48:24.740 --> 00:48:28.040 align:middle line:84% Now, one of the things that we talked a lot about with sound 00:48:28.040 --> 00:48:30.920 align:middle line:84% and that you got some experience with on the problem sets 00:48:30.920 --> 00:48:33.740 align:middle line:84% and possibly in your illusion labs is filtering. 00:48:33.740 --> 00:48:37.280 align:middle line:84% So filtering is an operation that changes the frequency 00:48:37.280 --> 00:48:38.970 align:middle line:90% content of a signal. 00:48:38.970 --> 00:48:41.490 align:middle line:84% And so images can also be filtered. 00:48:41.490 --> 00:48:47.050 align:middle line:84% Now, there's one wrinkle of doing this in the image domain 00:48:47.050 --> 00:48:50.170 align:middle line:84% that is sometimes a little bit hard to think about. 00:48:50.170 --> 00:48:52.650 align:middle line:84% And that is that images are two dimensional. 00:48:52.650 --> 00:48:55.690 align:middle line:84% And so there's actually two dimensions 00:48:55.690 --> 00:49:01.730 align:middle line:84% of spatial frequency, x and y, whereas with sound, 00:49:01.730 --> 00:49:05.610 align:middle line:84% when we plotted the amplitude spectrum or the power spectrum, 00:49:05.610 --> 00:49:07.510 align:middle line:84% we'd get a one-dimensional thing. 00:49:07.510 --> 00:49:11.370 align:middle line:84% So you have power on the y-axis and frequency on the x-axis. 00:49:11.370 --> 00:49:14.590 align:middle line:84% So now, with images, there's two dimensions of frequency. 00:49:14.590 --> 00:49:17.210 align:middle line:84% And so you have to display the power spectrum 00:49:17.210 --> 00:49:19.450 align:middle line:90% as a three-dimensional plot. 00:49:19.450 --> 00:49:22.210 align:middle line:90% So now, this is amplitude here. 00:49:22.210 --> 00:49:24.350 align:middle line:84% And this is one dimension of spatial frequency. 00:49:24.350 --> 00:49:27.050 align:middle line:84% And this is another dimension of spatial frequency. 00:49:27.050 --> 00:49:31.890 align:middle line:84% And so what this is showing is the frequency representation 00:49:31.890 --> 00:49:34.250 align:middle line:90% of this image of a face. 00:49:34.250 --> 00:49:35.930 align:middle line:90% And what this shows-- 00:49:35.930 --> 00:49:40.530 align:middle line:84% so at the center, that corresponds to the 0 spatial 00:49:40.530 --> 00:49:43.223 align:middle line:90% frequency, or the DC component. 00:49:43.223 --> 00:49:44.890 align:middle line:84% And then the frequencies around that are 00:49:44.890 --> 00:49:46.190 align:middle line:90% the low spatial frequencies. 00:49:46.190 --> 00:49:48.408 align:middle line:84% And then as you get out into the extremes here, 00:49:48.408 --> 00:49:49.950 align:middle line:84% those are higher spatial frequencies. 00:49:49.950 --> 00:49:52.490 align:middle line:84% And so in general, natural images 00:49:52.490 --> 00:49:55.810 align:middle line:84% tend to have frequency spectra that are lowpass. 00:49:55.810 --> 00:49:58.490 align:middle line:84% So they tend to have more power at low spatial frequencies 00:49:58.490 --> 00:49:59.510 align:middle line:90% and less at high. 00:49:59.510 --> 00:50:01.710 align:middle line:84% And so this is just another example of that. 00:50:01.710 --> 00:50:05.870 align:middle line:84% So that's the frequency representation of the image. 00:50:05.870 --> 00:50:09.010 align:middle line:84% And just as if you have an audio waveform, 00:50:09.010 --> 00:50:11.130 align:middle line:84% you can get the power spectrum by taking 00:50:11.130 --> 00:50:13.127 align:middle line:90% the fast Fourier transform. 00:50:13.127 --> 00:50:15.710 align:middle line:84% There's a Fourier transform that can be performed on an image. 00:50:15.710 --> 00:50:20.810 align:middle line:84% It's just a two-dimensional Fourier transform 00:50:20.810 --> 00:50:22.690 align:middle line:90% And so you can have filters. 00:50:22.690 --> 00:50:25.230 align:middle line:84% And these are transfer functions of two filters. 00:50:25.230 --> 00:50:27.370 align:middle line:90% So this is a lowpass filter. 00:50:27.370 --> 00:50:30.650 align:middle line:84% So what this does is it passes all of the stuff that's 00:50:30.650 --> 00:50:36.130 align:middle line:84% right in the center of the frequency plane and attenuates 00:50:36.130 --> 00:50:37.890 align:middle line:90% everything beyond that. 00:50:37.890 --> 00:50:40.890 align:middle line:84% This is a highpass filter that passes just the very high 00:50:40.890 --> 00:50:43.330 align:middle line:84% spatial frequencies and kills off 00:50:43.330 --> 00:50:45.490 align:middle line:90% the lowest spatial frequencies. 00:50:45.490 --> 00:50:51.530 align:middle line:84% And so we can take a lowpass filter, apply it to the image, 00:50:51.530 --> 00:50:53.290 align:middle line:90% and this is what you get. 00:50:53.290 --> 00:50:57.210 align:middle line:84% You get something that is blurry. 00:50:57.210 --> 00:50:59.730 align:middle line:84% This is a highpass filter applied to the image. 00:50:59.730 --> 00:51:01.530 align:middle line:84% And you get something that looks like this, 00:51:01.530 --> 00:51:05.370 align:middle line:84% where a lot of the edges are accentuated. 00:51:05.370 --> 00:51:08.010 align:middle line:84% And so intuitively, what has happened here 00:51:08.010 --> 00:51:11.010 align:middle line:84% is that the lowpass filter is preserving 00:51:11.010 --> 00:51:13.210 align:middle line:90% the low spatial frequencies. 00:51:13.210 --> 00:51:15.170 align:middle line:84% So those are like the parts of the image that 00:51:15.170 --> 00:51:18.650 align:middle line:90% vary slowly across space. 00:51:18.650 --> 00:51:21.410 align:middle line:84% And the highpass filter is eliminating those 00:51:21.410 --> 00:51:24.650 align:middle line:84% and just preserving the parts that vary very rapidly 00:51:24.650 --> 00:51:26.070 align:middle line:90% as a function of space. 00:51:26.070 --> 00:51:30.230 align:middle line:90% 00:51:30.230 --> 00:51:32.530 align:middle line:90% So images can be filtered. 00:51:32.530 --> 00:51:34.530 align:middle line:90% So here are some examples. 00:51:34.530 --> 00:51:38.855 align:middle line:84% Here's another face image with the highpass filtered version 00:51:38.855 --> 00:51:40.230 align:middle line:84% and the low passfiltered version. 00:51:40.230 --> 00:51:44.810 align:middle line:84% Again, the lowpass filtered versions just look blurry. 00:51:44.810 --> 00:51:47.170 align:middle line:84% Here's another image with the low frequency 00:51:47.170 --> 00:51:51.610 align:middle line:84% component and the high frequency component. 00:51:51.610 --> 00:51:54.970 align:middle line:84% And this is an image that is built up 00:51:54.970 --> 00:51:58.610 align:middle line:84% from a lowpass component, a bandpass component-- that's 00:51:58.610 --> 00:52:02.690 align:middle line:84% number 1-- and a high pass component-- that's number 2. 00:52:02.690 --> 00:52:04.730 align:middle line:84% And you get something that looks more 00:52:04.730 --> 00:52:07.170 align:middle line:90% like a full resolution image. 00:52:07.170 --> 00:52:10.770 align:middle line:84% Here's another example of a photograph of flowers that's 00:52:10.770 --> 00:52:14.530 align:middle line:84% been decomposed into a lowpass, bandpass, and highpass 00:52:14.530 --> 00:52:15.890 align:middle line:90% components. 00:52:15.890 --> 00:52:19.090 align:middle line:84% So a lot of these same concepts that we 00:52:19.090 --> 00:52:21.490 align:middle line:84% talked about when we were talking about sound 00:52:21.490 --> 00:52:24.130 align:middle line:90% can also be applied to images. 00:52:24.130 --> 00:52:27.930 align:middle line:84% But now, we're dealing with spatial frequency instead 00:52:27.930 --> 00:52:31.577 align:middle line:84% of audio frequency, where things are varying over time. 00:52:31.577 --> 00:52:33.410 align:middle line:84% And it's two-dimensional, which always makes 00:52:33.410 --> 00:52:35.690 align:middle line:90% everything more confusing. 00:52:35.690 --> 00:52:36.390 align:middle line:90% All right. 00:52:36.390 --> 00:52:38.630 align:middle line:90% So this is math essentially. 00:52:38.630 --> 00:52:41.130 align:middle line:90% So we've got signals. 00:52:41.130 --> 00:52:43.970 align:middle line:84% The Fourier transform lets us decompose them into frequencies. 00:52:43.970 --> 00:52:45.370 align:middle line:90% We can filter them. 00:52:45.370 --> 00:52:47.550 align:middle line:84% So what's the relevance for vision? 00:52:47.550 --> 00:52:51.810 align:middle line:84% So the natural question is, does the visual system 00:52:51.810 --> 00:52:54.830 align:middle line:84% decompose images into spatial frequency components? 00:52:54.830 --> 00:52:58.490 align:middle line:84% So we saw that the cochlea decomposes sound, 00:52:58.490 --> 00:53:01.670 align:middle line:84% to some extent, into audio frequency components. 00:53:01.670 --> 00:53:04.630 align:middle line:84% Maybe the visual system does something similar. 00:53:04.630 --> 00:53:07.690 align:middle line:90% 00:53:07.690 --> 00:53:10.690 align:middle line:84% So to look at this, we are going to get 00:53:10.690 --> 00:53:17.330 align:middle line:84% into the business of measuring contrast detection thresholds. 00:53:17.330 --> 00:53:19.630 align:middle line:84% So for that, we have to define contrast. 00:53:19.630 --> 00:53:25.250 align:middle line:84% So intuitively, contrast is the difference in intensity 00:53:25.250 --> 00:53:27.710 align:middle line:90% across some points in space. 00:53:27.710 --> 00:53:33.050 align:middle line:84% So if we have a sinusoidal grating where the luminance is 00:53:33.050 --> 00:53:36.850 align:middle line:84% varying spatially, sinusoidally, there 00:53:36.850 --> 00:53:42.310 align:middle line:84% will be a minimum luminance and a maximum luminance. 00:53:42.310 --> 00:53:44.330 align:middle line:84% And the contrast of the grating is 00:53:44.330 --> 00:53:47.730 align:middle line:84% defined by the difference between Lmax and Lmin 00:53:47.730 --> 00:53:49.690 align:middle line:90% divided by the sum. 00:53:49.690 --> 00:53:53.830 align:middle line:84% And so, by definition, the upper limit of the contrast can be 1. 00:53:53.830 --> 00:53:56.630 align:middle line:90% 00:53:56.630 --> 00:53:58.690 align:middle line:84% So that's what we're going to define. 00:53:58.690 --> 00:54:02.050 align:middle line:84% And what we want to measure is your contrast detection 00:54:02.050 --> 00:54:02.970 align:middle line:90% threshold. 00:54:02.970 --> 00:54:05.970 align:middle line:84% So how much contrast is needed for you 00:54:05.970 --> 00:54:09.610 align:middle line:84% to be able to detect that there is a grating there-- 00:54:09.610 --> 00:54:11.790 align:middle line:84% that is, that there's variation in intensity? 00:54:11.790 --> 00:54:16.070 align:middle line:84% And so it's kind of obvious that, when the contrast is high, 00:54:16.070 --> 00:54:17.410 align:middle line:90% it's going to be easy. 00:54:17.410 --> 00:54:19.210 align:middle line:84% As the contrast gets lower, it'll 00:54:19.210 --> 00:54:21.330 align:middle line:84% start to get harder and harder to tell that there 00:54:21.330 --> 00:54:22.750 align:middle line:90% are ripples in the image. 00:54:22.750 --> 00:54:25.810 align:middle line:84% And at some point, it will become impossible. 00:54:25.810 --> 00:54:29.250 align:middle line:84% So we'll measure the threshold using the psychophysical methods 00:54:29.250 --> 00:54:32.250 align:middle line:84% that you mastered a few lectures ago. 00:54:32.250 --> 00:54:33.250 align:middle line:90% Yeah? 00:54:33.250 --> 00:54:35.450 align:middle line:84% STUDENT: Does the number of bands 00:54:35.450 --> 00:54:37.537 align:middle line:90% matter in detecting contrast? 00:54:37.537 --> 00:54:39.370 align:middle line:84% JOSH MCDERMOTT: That is one of the questions 00:54:39.370 --> 00:54:40.630 align:middle line:90% we are about to answer. 00:54:40.630 --> 00:54:42.150 align:middle line:90% Yes, it's a great question. 00:54:42.150 --> 00:54:45.850 align:middle line:90% 00:54:45.850 --> 00:54:48.250 align:middle line:84% So we're going to do an experiment where 00:54:48.250 --> 00:54:51.050 align:middle line:84% we will ask, what is the smallest amount of contrast 00:54:51.050 --> 00:54:52.030 align:middle line:90% that is visible? 00:54:52.030 --> 00:54:55.850 align:middle line:84% So these are gratings that vary in their contrast. 00:54:55.850 --> 00:54:57.670 align:middle line:90% And so we measure the threshold. 00:54:57.670 --> 00:55:01.170 align:middle line:84% And then typically, in this particular subfield, 00:55:01.170 --> 00:55:04.490 align:middle line:84% we deal with contrast sensitivity rather than 00:55:04.490 --> 00:55:06.290 align:middle line:90% contrast threshold. 00:55:06.290 --> 00:55:07.630 align:middle line:90% I apologize for the typo here. 00:55:07.630 --> 00:55:10.490 align:middle line:84% So sensitivity is just the reciprocal of threshold. 00:55:10.490 --> 00:55:12.170 align:middle line:84% So in general, if you're more sensitive, 00:55:12.170 --> 00:55:15.450 align:middle line:84% that means your threshold is lower. 00:55:15.450 --> 00:55:18.490 align:middle line:84% So the question was posed, does the threshold 00:55:18.490 --> 00:55:20.770 align:middle line:90% depend on spatial frequency? 00:55:20.770 --> 00:55:24.450 align:middle line:84% And so this is an image that answers that question. 00:55:24.450 --> 00:55:29.010 align:middle line:84% So this is an image that varies in spatial frequency 00:55:29.010 --> 00:55:34.650 align:middle line:84% from very low on the left to very high on the right. 00:55:34.650 --> 00:55:38.690 align:middle line:84% And it varies in contrast, going from high contrast at the bottom 00:55:38.690 --> 00:55:42.980 align:middle line:90% up to low contrast at the top. 00:55:42.980 --> 00:55:47.090 align:middle line:84% And what you should be able to see by looking at this 00:55:47.090 --> 00:55:49.930 align:middle line:84% is that your contrast sensitivity is not 00:55:49.930 --> 00:55:52.230 align:middle line:84% uniform across spatial frequency. 00:55:52.230 --> 00:55:56.270 align:middle line:84% Rather, it's best for medium spatial frequencies 00:55:56.270 --> 00:55:59.290 align:middle line:84% and is worse when the spatial frequencies get too low 00:55:59.290 --> 00:56:01.010 align:middle line:90% or get too high. 00:56:01.010 --> 00:56:03.910 align:middle line:84% Everybody see of inverted U shape? 00:56:03.910 --> 00:56:05.810 align:middle line:90% All right. 00:56:05.810 --> 00:56:09.650 align:middle line:84% So that is what is known as the contrast sensitivity function. 00:56:09.650 --> 00:56:11.490 align:middle line:84% So the contrast sensitivity function 00:56:11.490 --> 00:56:18.290 align:middle line:84% is a function that plots contrast sensitivity 00:56:18.290 --> 00:56:20.490 align:middle line:84% as a function of spatial frequency. 00:56:20.490 --> 00:56:25.950 align:middle line:84% So remember, sensitivity is 1 over the threshold. 00:56:25.950 --> 00:56:29.250 align:middle line:84% So when the number is high, that means your threshold is low. 00:56:29.250 --> 00:56:31.410 align:middle line:84% And you can see that your sensitivity is highest 00:56:31.410 --> 00:56:33.490 align:middle line:84% at these medium spatial frequencies, 00:56:33.490 --> 00:56:35.410 align:middle line:84% like three or four cycles per degree, 00:56:35.410 --> 00:56:38.250 align:middle line:84% and then gets lower if the spatial frequency gets lower 00:56:38.250 --> 00:56:40.050 align:middle line:90% and higher if it gets higher. 00:56:40.050 --> 00:56:42.850 align:middle line:84% And that corresponds to what you saw in that image 00:56:42.850 --> 00:56:45.170 align:middle line:90% that we just demoed. 00:56:45.170 --> 00:56:46.690 align:middle line:84% And so the consequence of that is 00:56:46.690 --> 00:56:50.770 align:middle line:84% that you can see the image variation in this region. 00:56:50.770 --> 00:56:54.490 align:middle line:84% And even though there is physical luminance variation up 00:56:54.490 --> 00:56:58.890 align:middle line:90% here, that is invisible to you. 00:56:58.890 --> 00:57:03.570 align:middle line:84% So this is just a property of your visual system. 00:57:03.570 --> 00:57:07.610 align:middle line:84% But what about our original question 00:57:07.610 --> 00:57:11.210 align:middle line:84% of whether the visual system decomposes images 00:57:11.210 --> 00:57:13.430 align:middle line:84% into different spatial frequency components? 00:57:13.430 --> 00:57:14.930 align:middle line:84% Are there frequency channels, kind 00:57:14.930 --> 00:57:19.010 align:middle line:84% of like we see in the auditory system? 00:57:19.010 --> 00:57:19.510 align:middle line:90% All right. 00:57:19.510 --> 00:57:20.970 align:middle line:84% And so to get at this, we're going 00:57:20.970 --> 00:57:25.210 align:middle line:84% to use a classic tool in the perceptual scientist 00:57:25.210 --> 00:57:27.370 align:middle line:90% toolbox, which is adaptation. 00:57:27.370 --> 00:57:31.250 align:middle line:84% And so what we're going to do is adapt ourselves 00:57:31.250 --> 00:57:33.230 align:middle line:90% to a single spatial frequency. 00:57:33.230 --> 00:57:36.050 align:middle line:90% 00:57:36.050 --> 00:57:38.210 align:middle line:84% So we're going to stare at a sine wave grating 00:57:38.210 --> 00:57:39.230 align:middle line:90% for about a minute. 00:57:39.230 --> 00:57:41.930 align:middle line:90% That's usually the magic number. 00:57:41.930 --> 00:57:45.490 align:middle line:84% And there are naively two hypotheses. 00:57:45.490 --> 00:57:47.850 align:middle line:84% One hypothesis is that the visual system 00:57:47.850 --> 00:57:51.810 align:middle line:84% doesn't have these spatial frequency channels. 00:57:51.810 --> 00:57:55.690 align:middle line:84% And that would predict that, if you're adapting to a grating, 00:57:55.690 --> 00:57:58.750 align:middle line:84% you might just get a general decrease in sensitivity. 00:57:58.750 --> 00:58:02.250 align:middle line:84% So it might get harder to see contrast irrespective 00:58:02.250 --> 00:58:04.210 align:middle line:90% of the spatial frequency. 00:58:04.210 --> 00:58:07.330 align:middle line:84% The other possibility is that, if your visual system does 00:58:07.330 --> 00:58:09.130 align:middle line:84% contain these different channels that 00:58:09.130 --> 00:58:11.990 align:middle line:84% are specific to particular ranges of spatial frequencies, 00:58:11.990 --> 00:58:16.050 align:middle line:84% that if you adapt to one, you will decrease the responsiveness 00:58:16.050 --> 00:58:18.850 align:middle line:84% in that one channel, kind of like what we think 00:58:18.850 --> 00:58:20.890 align:middle line:84% happens when you adapt to an orientation that 00:58:20.890 --> 00:58:24.850 align:middle line:90% causes the tilt aftereffect. 00:58:24.850 --> 00:58:27.010 align:middle line:84% And that would have the consequence then-- so if we 00:58:27.010 --> 00:58:30.930 align:middle line:84% adapt ourselves and then measure contrast sensitivity, 00:58:30.930 --> 00:58:33.010 align:middle line:84% we should see a localized decrease 00:58:33.010 --> 00:58:36.410 align:middle line:84% in contrast sensitivity at the adapting frequency 00:58:36.410 --> 00:58:38.510 align:middle line:90% rather than a global decrease. 00:58:38.510 --> 00:58:39.730 align:middle line:90% So let's try it. 00:58:39.730 --> 00:58:42.770 align:middle line:90% So what I want you to do-- 00:58:42.770 --> 00:58:45.552 align:middle line:84% so first, just glance at B. So the image on B 00:58:45.552 --> 00:58:48.010 align:middle line:84% is the one that we're going to use to measure your contrast 00:58:48.010 --> 00:58:50.090 align:middle line:90% sensitivity function. 00:58:50.090 --> 00:58:54.230 align:middle line:84% Now, let me get out my stopwatch. 00:58:54.230 --> 00:58:56.570 align:middle line:84% I want you to adapt your visual system 00:58:56.570 --> 00:59:01.170 align:middle line:90% by staring at the red circle. 00:59:01.170 --> 00:59:05.250 align:middle line:84% And so again, look around in the red circle, 00:59:05.250 --> 00:59:07.550 align:middle line:84% because we don't want to adapt our photoreceptors. 00:59:07.550 --> 00:59:10.830 align:middle line:84% We're trying to adapt spatial frequency channels. 00:59:10.830 --> 00:59:13.687 align:middle line:84% So just look around at the red circle. 00:59:13.687 --> 00:59:15.270 align:middle line:84% You can think about whatever you want. 00:59:15.270 --> 00:59:17.050 align:middle line:84% Just keep your eyes in the red circle. 00:59:17.050 --> 00:59:19.810 align:middle line:90% 00:59:19.810 --> 00:59:25.940 align:middle line:84% And eventually, we're going to then look over 00:59:25.940 --> 00:59:27.420 align:middle line:90% at that other image. 00:59:27.420 --> 00:59:30.660 align:middle line:84% And the two possibilities that we're going to try to evaluate 00:59:30.660 --> 00:59:34.980 align:middle line:84% is whether the overall the profile of your sensitivity 00:59:34.980 --> 00:59:35.740 align:middle line:90% shifts down. 00:59:35.740 --> 00:59:36.760 align:middle line:90% Again, don't look now. 00:59:36.760 --> 00:59:38.620 align:middle line:84% Just keep looking at the red circle, OK? 00:59:38.620 --> 00:59:41.460 align:middle line:84% We're going to ask ourselves whether the overall sensitivity 00:59:41.460 --> 00:59:44.380 align:middle line:84% drops or whether there's a little bite taken out 00:59:44.380 --> 00:59:46.420 align:middle line:84% of the contrast sensitivity function 00:59:46.420 --> 00:59:51.180 align:middle line:84% around the adaptive spatial frequency. 00:59:51.180 --> 00:59:51.680 align:middle line:90% All right. 00:59:51.680 --> 00:59:53.722 align:middle line:84% We're going to do this for about 15 more seconds. 00:59:53.722 --> 00:59:54.320 align:middle line:90% Hang in there. 00:59:54.320 --> 00:59:56.100 align:middle line:84% Just keep looking at that red circle. 00:59:56.100 --> 00:59:58.728 align:middle line:90% 00:59:58.728 --> 01:00:03.500 align:middle line:90% 5, 4, 3, 2, 1. 01:00:03.500 --> 01:00:04.000 align:middle line:90% All right. 01:00:04.000 --> 01:00:07.540 align:middle line:84% Look over at your contrast sensitivity function. 01:00:07.540 --> 01:00:10.420 align:middle line:84% And what you should notice is that there's now 01:00:10.420 --> 01:00:13.380 align:middle line:84% this little bite taken out of the area 01:00:13.380 --> 01:00:15.440 align:middle line:90% around the adaptive frequency. 01:00:15.440 --> 01:00:17.220 align:middle line:90% Is that what everybody sees? 01:00:17.220 --> 01:00:19.740 align:middle line:84% Raise your hand if that's what you see. 01:00:19.740 --> 01:00:21.280 align:middle line:90% Most of you. 01:00:21.280 --> 01:00:22.460 align:middle line:90% Not everybody. 01:00:22.460 --> 01:00:24.460 align:middle line:90% Yeah, OK. 01:00:24.460 --> 01:00:26.840 align:middle line:84% So we may not have adapted long enough for everyone. 01:00:26.840 --> 01:00:29.940 align:middle line:84% But that's the typically experienced result, 01:00:29.940 --> 01:00:36.180 align:middle line:84% that you should have seen a profile of visibility that 01:00:36.180 --> 01:00:39.380 align:middle line:84% looks more like this, rather than the upside down U shape 01:00:39.380 --> 01:00:40.720 align:middle line:90% that you typically see. 01:00:40.720 --> 01:00:44.320 align:middle line:90% 01:00:44.320 --> 01:00:47.420 align:middle line:84% So this is the result. And the way 01:00:47.420 --> 01:00:50.540 align:middle line:84% that you would do this experiment in the lab 01:00:50.540 --> 01:00:52.760 align:middle line:84% is-- it's a bit of a tedious experiment 01:00:52.760 --> 01:00:55.340 align:middle line:84% because you've got to measure detection thresholds at all 01:00:55.340 --> 01:00:56.840 align:middle line:84% these different spatial frequencies. 01:00:56.840 --> 01:01:00.220 align:middle line:84% And you'll want to measure that under normal conditions, 01:01:00.220 --> 01:01:02.560 align:middle line:84% and then under conditions where you adapt people. 01:01:02.560 --> 01:01:05.820 align:middle line:84% And so to measure this with adaptation, 01:01:05.820 --> 01:01:07.980 align:middle line:84% what you'd have to do is force people 01:01:07.980 --> 01:01:09.660 align:middle line:84% to look at the adapting stimulus, 01:01:09.660 --> 01:01:13.020 align:middle line:84% and then periodically make them do a detection trial, 01:01:13.020 --> 01:01:15.640 align:middle line:84% adapt them some more, have them do some more detection trials. 01:01:15.640 --> 01:01:18.960 align:middle line:84% You have to keep them adapted in order to do these measurements. 01:01:18.960 --> 01:01:20.520 align:middle line:90% So it's a long experiment. 01:01:20.520 --> 01:01:23.620 align:middle line:84% But the outcome of that would be a contrast sensitivity function 01:01:23.620 --> 01:01:25.540 align:middle line:84% that looks like this, where sensitivity 01:01:25.540 --> 01:01:30.200 align:middle line:84% is reduced in a local region of the spatial frequency spectrum. 01:01:30.200 --> 01:01:33.760 align:middle line:90% 01:01:33.760 --> 01:01:36.740 align:middle line:84% So this suggests that this thing that we measure, 01:01:36.740 --> 01:01:39.260 align:middle line:84% that's called the contrast sensitivity function, 01:01:39.260 --> 01:01:41.780 align:middle line:84% represents the envelope of a whole bunch 01:01:41.780 --> 01:01:46.480 align:middle line:84% of different channels rather than a single mechanism, 01:01:46.480 --> 01:01:48.180 align:middle line:90% a single receptive field. 01:01:48.180 --> 01:01:51.620 align:middle line:84% Now, this doesn't tell us where these channels might 01:01:51.620 --> 01:01:53.740 align:middle line:90% originate in the visual system. 01:01:53.740 --> 01:01:57.140 align:middle line:84% And that's a question that we can ask. 01:01:57.140 --> 01:02:01.740 align:middle line:84% So one possibility is that these spatial frequency channels 01:02:01.740 --> 01:02:03.660 align:middle line:84% are actually due to center-surround receptive 01:02:03.660 --> 01:02:07.100 align:middle line:84% fields, like potentially in the retina or the LGN. 01:02:07.100 --> 01:02:09.420 align:middle line:84% And so the idea here is that there's 01:02:09.420 --> 01:02:12.760 align:middle line:84% a center-surround receptive field, shown here. 01:02:12.760 --> 01:02:16.520 align:middle line:84% And it's like a good match to a particular spatial frequency, 01:02:16.520 --> 01:02:18.980 align:middle line:84% where the center of the receptive field 01:02:18.980 --> 01:02:21.600 align:middle line:84% aligns with the peak of a sine wave grating. 01:02:21.600 --> 01:02:24.280 align:middle line:84% And then the surround aligns with the troughs, 01:02:24.280 --> 01:02:27.100 align:middle line:84% whereas if the spatial frequency is lower or higher, 01:02:27.100 --> 01:02:29.220 align:middle line:84% it's a poor match to the receptive field. 01:02:29.220 --> 01:02:31.340 align:middle line:90% And so you get a lower response. 01:02:31.340 --> 01:02:34.340 align:middle line:84% So this is just the same linear filter responses 01:02:34.340 --> 01:02:35.540 align:middle line:90% that we've seen. 01:02:35.540 --> 01:02:38.100 align:middle line:84% So the point is that you could potentially 01:02:38.100 --> 01:02:41.180 align:middle line:84% explain this with these center-surround receptive 01:02:41.180 --> 01:02:41.800 align:middle line:90% fields. 01:02:41.800 --> 01:02:43.940 align:middle line:84% So does anybody have an idea for how 01:02:43.940 --> 01:02:49.340 align:middle line:84% we could test that theory of spatial frequency channels? 01:02:49.340 --> 01:02:54.140 align:middle line:84% How could how could we test whether your spatial frequency 01:02:54.140 --> 01:02:56.340 align:middle line:84% selectivity is due to center-surround receptive 01:02:56.340 --> 01:02:56.920 align:middle line:90% fields? 01:02:56.920 --> 01:03:13.220 align:middle line:90% 01:03:13.220 --> 01:03:16.220 align:middle line:90% OK, I have an idea, actually. 01:03:16.220 --> 01:03:22.300 align:middle line:84% What if we adapt to a horizontal orientation 01:03:22.300 --> 01:03:24.860 align:middle line:84% of the same spatial frequency, and then 01:03:24.860 --> 01:03:28.540 align:middle line:84% measure contrast sensitivity with a vertical orientation? 01:03:28.540 --> 01:03:32.860 align:middle line:84% So if the adaptation is occurring 01:03:32.860 --> 01:03:35.980 align:middle line:84% in a center-surround receptive field, 01:03:35.980 --> 01:03:40.600 align:middle line:84% then, if you adapt to this, it should transfer to this. 01:03:40.600 --> 01:03:47.740 align:middle line:90% 01:03:47.740 --> 01:03:48.600 align:middle line:90% Let's try it. 01:03:48.600 --> 01:03:49.280 align:middle line:90% All right. 01:03:49.280 --> 01:03:52.060 align:middle line:90% So we're going to adapt. 01:03:52.060 --> 01:03:52.800 align:middle line:90% Stare at this. 01:03:52.800 --> 01:03:54.300 align:middle line:84% So the procedure here is we're going 01:03:54.300 --> 01:03:58.140 align:middle line:84% to stare at this red circle for a minute. 01:03:58.140 --> 01:03:59.320 align:middle line:90% So start staring. 01:03:59.320 --> 01:04:01.580 align:middle line:84% And then we're going to look up again and see 01:04:01.580 --> 01:04:03.513 align:middle line:90% if it looks any different. 01:04:03.513 --> 01:04:05.680 align:middle line:84% It's very important to have your eyes open for this. 01:04:05.680 --> 01:04:09.620 align:middle line:84% If your eyes are closed, it will not work. 01:04:09.620 --> 01:04:13.780 align:middle line:84% So stare at the red circle down here. 01:04:13.780 --> 01:04:17.940 align:middle line:84% Again, move your eyes around within the red circle 01:04:17.940 --> 01:04:20.080 align:middle line:84% to avoid getting a big afterimage. 01:04:20.080 --> 01:04:25.140 align:middle line:90% 01:04:25.140 --> 01:04:28.280 align:middle line:84% OK, we've got another 30 seconds to go. 01:04:28.280 --> 01:04:36.900 align:middle line:90% 01:04:36.900 --> 01:04:38.560 align:middle line:90% All right, 20 more seconds. 01:04:38.560 --> 01:04:42.500 align:middle line:90% 01:04:42.500 --> 01:04:43.480 align:middle line:90% Keep staring. 01:04:43.480 --> 01:04:47.340 align:middle line:90% 01:04:47.340 --> 01:04:50.460 align:middle line:90% All right, 10 more seconds. 01:04:50.460 --> 01:04:53.780 align:middle line:90% 8, 7, 6, 5. 01:04:53.780 --> 01:04:57.320 align:middle line:90% 4, 3, 2 1. 01:04:57.320 --> 01:05:01.180 align:middle line:84% Now, measure your contrast sensitivity function. 01:05:01.180 --> 01:05:05.220 align:middle line:84% how many people see the bite taken out of it now? 01:05:05.220 --> 01:05:06.300 align:middle line:90% Nobody? 01:05:06.300 --> 01:05:08.580 align:middle line:90% OK. 01:05:08.580 --> 01:05:13.740 align:middle line:84% So what we conclude from this is that the effect 01:05:13.740 --> 01:05:18.220 align:middle line:84% that we previously witnessed is dependent on some kind 01:05:18.220 --> 01:05:20.740 align:middle line:90% of orientation-tuned mechanism. 01:05:20.740 --> 01:05:24.580 align:middle line:84% So it's not occurring at the site 01:05:24.580 --> 01:05:26.480 align:middle line:84% of center-surround receptive fields. 01:05:26.480 --> 01:05:30.380 align:middle line:84% So it's probably not in the retina or the LGN. 01:05:30.380 --> 01:05:33.100 align:middle line:84% So this suggests that the adaptation effects are probably 01:05:33.100 --> 01:05:35.140 align:middle line:84% mediated cortically because that's where 01:05:35.140 --> 01:05:39.140 align:middle line:90% orientation selectivity arises. 01:05:39.140 --> 01:05:43.180 align:middle line:84% And so the idea here is that these channels correspond 01:05:43.180 --> 01:05:49.800 align:middle line:84% to neurons or groups of neurons that have similar properties. 01:05:49.800 --> 01:05:54.420 align:middle line:84% So we just measured all this stuff with sine wave gratings. 01:05:54.420 --> 01:05:58.420 align:middle line:84% But you can also find evidence for spatial frequency-selective 01:05:58.420 --> 01:05:59.440 align:middle line:90% channels with masking. 01:05:59.440 --> 01:06:01.900 align:middle line:84% So that's how we really investigate frequency 01:06:01.900 --> 01:06:03.713 align:middle line:90% selectivity in hearing. 01:06:03.713 --> 01:06:05.380 align:middle line:84% Remember, we talked a lot about masking, 01:06:05.380 --> 01:06:07.213 align:middle line:84% and their ability to detect a tone in noise, 01:06:07.213 --> 01:06:09.380 align:middle line:84% and how that depended on the bandwidth of the noise, 01:06:09.380 --> 01:06:10.240 align:middle line:90% and stuff like that. 01:06:10.240 --> 01:06:13.600 align:middle line:84% So this is the same idea, but just applied division. 01:06:13.600 --> 01:06:15.780 align:middle line:84% And so what you're seeing here are 01:06:15.780 --> 01:06:19.860 align:middle line:84% letters that are superimposed on noise 01:06:19.860 --> 01:06:23.180 align:middle line:84% that has been filtered into different spatial frequency 01:06:23.180 --> 01:06:25.620 align:middle line:84% ranges, from very low spatial frequencies up 01:06:25.620 --> 01:06:28.480 align:middle line:84% at the top to high spatial frequencies down at the bottom. 01:06:28.480 --> 01:06:31.040 align:middle line:84% And so the letters are exactly the same. 01:06:31.040 --> 01:06:34.380 align:middle line:84% The contrast of the noise is the same across all of the examples. 01:06:34.380 --> 01:06:38.100 align:middle line:84% All that varies is the spatial frequency content of the noise. 01:06:38.100 --> 01:06:40.980 align:middle line:84% And what you are supposed to be able to see 01:06:40.980 --> 01:06:45.100 align:middle line:84% is that it's pretty easy to read the letters at the bottom 01:06:45.100 --> 01:06:46.360 align:middle line:90% and also at the very top. 01:06:46.360 --> 01:06:49.980 align:middle line:84% But then there's this range in the middle where it gets 01:06:49.980 --> 01:06:52.120 align:middle line:90% hard to detect the letters. 01:06:52.120 --> 01:06:54.740 align:middle line:90% So this is evidence that-- 01:06:54.740 --> 01:06:56.380 align:middle line:84% again, more evidence that there is 01:06:56.380 --> 01:06:58.580 align:middle line:84% a spatial frequency-selective channel 01:06:58.580 --> 01:07:03.000 align:middle line:84% that limits, in this case, your ability to detect the letters. 01:07:03.000 --> 01:07:05.890 align:middle line:90% 01:07:05.890 --> 01:07:12.460 align:middle line:84% And you can also see this result in a similar kind of display 01:07:12.460 --> 01:07:14.980 align:middle line:84% that lets you see the contrast sensitivity 01:07:14.980 --> 01:07:16.660 align:middle line:90% function, like structure. 01:07:16.660 --> 01:07:18.740 align:middle line:84% So now, we have these different rows 01:07:18.740 --> 01:07:22.700 align:middle line:84% of letters that are decreasing in contrast. 01:07:22.700 --> 01:07:26.780 align:middle line:84% And the different columns here correspond 01:07:26.780 --> 01:07:28.967 align:middle line:84% to different spatial frequencies of noise. 01:07:28.967 --> 01:07:31.300 align:middle line:84% And so the point is that your ability to see the letters 01:07:31.300 --> 01:07:34.940 align:middle line:84% extends further down on the far left 01:07:34.940 --> 01:07:37.940 align:middle line:84% and the far right, when you're dealing 01:07:37.940 --> 01:07:41.060 align:middle line:84% with low spatial frequency noise or high spatial frequency noise, 01:07:41.060 --> 01:07:45.240 align:middle line:84% and is impaired most by these middle spatial frequencies. 01:07:45.240 --> 01:07:48.140 align:middle line:90% 01:07:48.140 --> 01:07:49.760 align:middle line:84% All right, any questions about that? 01:07:49.760 --> 01:08:00.500 align:middle line:90% 01:08:00.500 --> 01:08:05.320 align:middle line:84% So some nomenclature in terms of talking about spatial frequency 01:08:05.320 --> 01:08:05.820 align:middle line:90% channels-- 01:08:05.820 --> 01:08:07.737 align:middle line:84% I mean, a lot of this will be fairly familiar, 01:08:07.737 --> 01:08:11.740 align:middle line:84% just from the fact that we covered analogous concepts 01:08:11.740 --> 01:08:13.540 align:middle line:90% when we talked about audition. 01:08:13.540 --> 01:08:15.620 align:middle line:84% But filters are usually characterized 01:08:15.620 --> 01:08:17.180 align:middle line:90% in terms of their bandwidth. 01:08:17.180 --> 01:08:19.740 align:middle line:84% So usually, the bandwidth is defined 01:08:19.740 --> 01:08:23.819 align:middle line:84% as the ratio of the frequencies at which the maximum contrast 01:08:23.819 --> 01:08:25.812 align:middle line:90% sensitivity is obtained-- 01:08:25.812 --> 01:08:28.020 align:middle line:84% sorry, at which half the maximum contrast sensitivity 01:08:28.020 --> 01:08:28.600 align:middle line:90% is attained. 01:08:28.600 --> 01:08:31.359 align:middle line:84% Usually, it's measured in octaves. 01:08:31.359 --> 01:08:32.859 align:middle line:84% So remember, an octave is a doubling 01:08:32.859 --> 01:08:36.100 align:middle line:90% in frequency-- so 1 to 2 to 4. 01:08:36.100 --> 01:08:39.740 align:middle line:84% So here's two examples of spatial frequency channels. 01:08:39.740 --> 01:08:43.100 align:middle line:84% And you can see this one here has got peak sensitivity 01:08:43.100 --> 01:08:48.020 align:middle line:84% at 2.3 cycles per degree and a bandwidth of about 2 octaves. 01:08:48.020 --> 01:08:50.319 align:middle line:84% This one's got a bandwidth of 1.3 octaves. 01:08:50.319 --> 01:08:53.060 align:middle line:84% In general, the spatial frequency channels 01:08:53.060 --> 01:08:54.580 align:middle line:84% that you see in the visual system 01:08:54.580 --> 01:08:58.920 align:middle line:84% are wider than the filters of the cochlea. 01:08:58.920 --> 01:09:04.859 align:middle line:84% So the cochlea, roughly speaking, filters 01:09:04.859 --> 01:09:07.180 align:middle line:84% are maybe a third of an octave, depending on where 01:09:07.180 --> 01:09:10.387 align:middle line:90% you are on the cochlea. 01:09:10.387 --> 01:09:12.220 align:middle line:84% And the spatial frequency channels in vision 01:09:12.220 --> 01:09:14.300 align:middle line:84% are typically in excess of an octave. 01:09:14.300 --> 01:09:20.180 align:middle line:84% So the frequency resolution is not nearly as tight. 01:09:20.180 --> 01:09:25.020 align:middle line:90% But it's nonetheless there. 01:09:25.020 --> 01:09:26.753 align:middle line:84% So we just did that experiment that 01:09:26.753 --> 01:09:28.420 align:middle line:84% suggests that the frequency channels are 01:09:28.420 --> 01:09:29.779 align:middle line:90% cortically mediated. 01:09:29.779 --> 01:09:32.460 align:middle line:84% And there's now quite a lot of neurophysiology 01:09:32.460 --> 01:09:33.520 align:middle line:90% that supports that. 01:09:33.520 --> 01:09:37.260 align:middle line:84% So Rus de Valois was the neurophysiologist 01:09:37.260 --> 01:09:40.260 align:middle line:84% who did a lot of work in this particular area. 01:09:40.260 --> 01:09:43.020 align:middle line:84% And these are contrast sensitivity functions 01:09:43.020 --> 01:09:48.580 align:middle line:84% of individual neurons, so single neurons in V1. 01:09:48.580 --> 01:09:52.340 align:middle line:84% And you can see that the best spatial frequency 01:09:52.340 --> 01:09:55.592 align:middle line:84% of the different neurons varies depending on which 01:09:55.592 --> 01:09:56.800 align:middle line:90% neuron you're recording from. 01:09:56.800 --> 01:09:59.860 align:middle line:84% Of course, the overall contrast sensitivity also varies. 01:09:59.860 --> 01:10:04.950 align:middle line:84% But you can think of each neuron as mediating a spatial frequency 01:10:04.950 --> 01:10:07.950 align:middle line:90% channel. 01:10:07.950 --> 01:10:09.790 align:middle line:84% So contrast sensitivity functions 01:10:09.790 --> 01:10:13.610 align:middle line:84% have been measured in lots of people at lots of ages. 01:10:13.610 --> 01:10:16.630 align:middle line:90% 01:10:16.630 --> 01:10:19.270 align:middle line:84% Contrast sensitivity improves pretty dramatically 01:10:19.270 --> 01:10:21.190 align:middle line:84% over the first few months of life, especially 01:10:21.190 --> 01:10:22.610 align:middle line:90% for high spatial frequencies. 01:10:22.610 --> 01:10:25.190 align:middle line:84% So this is the contrast sensitivity function 01:10:25.190 --> 01:10:28.190 align:middle line:84% displayed for an adult at the top, 01:10:28.190 --> 01:10:31.670 align:middle line:84% and then for different ages of life, starting at four weeks. 01:10:31.670 --> 01:10:35.550 align:middle line:84% And so you can see that, in general, sensitivity improves. 01:10:35.550 --> 01:10:38.010 align:middle line:84% But in particular, at these high spatial frequencies, 01:10:38.010 --> 01:10:40.750 align:middle line:84% you can see that a month-old baby just can't really detect 01:10:40.750 --> 01:10:42.910 align:middle line:90% this kind of stuff at all. 01:10:42.910 --> 01:10:48.657 align:middle line:84% And as you mature, you become able to see these higher 01:10:48.657 --> 01:10:49.490 align:middle line:90% spatial frequencies. 01:10:49.490 --> 01:10:52.570 align:middle line:84% So in general, baby vision is fairly blurry. 01:10:52.570 --> 01:10:55.030 align:middle line:90% 01:10:55.030 --> 01:10:55.530 align:middle line:90% All right. 01:10:55.530 --> 01:10:59.070 align:middle line:84% Now, so the idea of these spatial frequency channels 01:10:59.070 --> 01:11:02.670 align:middle line:84% is that you get this image that comes into the eye. 01:11:02.670 --> 01:11:05.110 align:middle line:84% And then it gets decomposed, to some extent, 01:11:05.110 --> 01:11:10.210 align:middle line:90% into these frequency components. 01:11:10.210 --> 01:11:12.710 align:middle line:84% However, there's lots of reasons to think 01:11:12.710 --> 01:11:16.070 align:middle line:84% that those different spatial frequency channels don't 01:11:16.070 --> 01:11:20.330 align:middle line:84% remain independent all the way up to object recognition. 01:11:20.330 --> 01:11:23.390 align:middle line:84% And this is a classic illusion that provides 01:11:23.390 --> 01:11:24.730 align:middle line:90% a demonstration of this. 01:11:24.730 --> 01:11:30.750 align:middle line:84% So how many people know who this is? 01:11:30.750 --> 01:11:31.370 align:middle line:90% A few. 01:11:31.370 --> 01:11:32.310 align:middle line:90% Yeah, OK. 01:11:32.310 --> 01:11:34.470 align:middle line:84% How many people have seen this before? 01:11:34.470 --> 01:11:35.810 align:middle line:90% OK, no one's seen this before. 01:11:35.810 --> 01:11:37.150 align:middle line:90% All right. 01:11:37.150 --> 01:11:42.830 align:middle line:84% So this is a picture of a famous person. 01:11:42.830 --> 01:11:44.470 align:middle line:90% But it's very pixelated. 01:11:44.470 --> 01:11:47.490 align:middle line:84% And that makes it hard to recognize. 01:11:47.490 --> 01:11:49.870 align:middle line:84% But if you squint your eyes, that 01:11:49.870 --> 01:11:52.990 align:middle line:84% has the effect of blurring the image. 01:11:52.990 --> 01:11:57.550 align:middle line:84% And the effect here that you're supposed to note is that, 01:11:57.550 --> 01:12:01.110 align:middle line:84% if you squint your eyes or blur the image, 01:12:01.110 --> 01:12:03.730 align:middle line:84% which is essentially providing a lowpass filter-- 01:12:03.730 --> 01:12:06.390 align:middle line:84% so you're removing the high spatial frequency components-- 01:12:06.390 --> 01:12:08.710 align:middle line:84% the image becomes more recognizable. 01:12:08.710 --> 01:12:12.390 align:middle line:84% And so the inference that we make from this-- 01:12:12.390 --> 01:12:15.190 align:middle line:84% so what's happened here, when the image is pixelated, 01:12:15.190 --> 01:12:17.710 align:middle line:84% is that there are high spatial frequency components that are 01:12:17.710 --> 01:12:21.730 align:middle line:90% introduced that are incorrect. 01:12:21.730 --> 01:12:24.310 align:middle line:84% Those are not the high spatial frequency components 01:12:24.310 --> 01:12:26.630 align:middle line:84% that were in this original painting or photograph 01:12:26.630 --> 01:12:28.030 align:middle line:90% of Abraham Lincoln. 01:12:28.030 --> 01:12:31.070 align:middle line:84% They're just artifacts of the pixelation. 01:12:31.070 --> 01:12:34.190 align:middle line:84% And when you squint your eyes, you 01:12:34.190 --> 01:12:36.870 align:middle line:84% are filtering out some of those high spatial frequency 01:12:36.870 --> 01:12:37.690 align:middle line:90% components. 01:12:37.690 --> 01:12:41.510 align:middle line:84% And the fact that that makes the image more recognizable 01:12:41.510 --> 01:12:44.727 align:middle line:84% is evidence that, although we think early 01:12:44.727 --> 01:12:47.310 align:middle line:84% in the visual system, the high and the low spatial frequencies 01:12:47.310 --> 01:12:50.390 align:middle line:84% are decomposed and representing these different channels, 01:12:50.390 --> 01:12:55.110 align:middle line:84% they then must interact in order to mediate object recognition. 01:12:55.110 --> 01:12:58.470 align:middle line:84% And in this case, the incorrect high spatial frequencies 01:12:58.470 --> 01:13:00.470 align:middle line:84% are interfering with your ability 01:13:00.470 --> 01:13:05.510 align:middle line:84% to extract information from the low spatial frequencies. 01:13:05.510 --> 01:13:06.010 align:middle line:90% All right. 01:13:06.010 --> 01:13:08.910 align:middle line:84% So evidence that these different spatial scales 01:13:08.910 --> 01:13:11.230 align:middle line:84% don't remain independent all the way up 01:13:11.230 --> 01:13:12.870 align:middle line:90% through the visual system. 01:13:12.870 --> 01:13:15.810 align:middle line:90% Any questions about that? 01:13:15.810 --> 01:13:20.790 align:middle line:90% 01:13:20.790 --> 01:13:21.990 align:middle line:90% All right. 01:13:21.990 --> 01:13:25.070 align:middle line:84% So just to summarize what we talked about today, 01:13:25.070 --> 01:13:28.390 align:middle line:84% key ideas are that images can be decomposed 01:13:28.390 --> 01:13:29.610 align:middle line:90% into spatial frequencies. 01:13:29.610 --> 01:13:33.630 align:middle line:90% That's just a mathematical fact. 01:13:33.630 --> 01:13:37.310 align:middle line:84% Evidence that there are multiple spatial frequency channels 01:13:37.310 --> 01:13:40.450 align:middle line:84% comes from experiments on contrast adaptations. 01:13:40.450 --> 01:13:43.510 align:middle line:84% So if you adapt to a sine wave grating, 01:13:43.510 --> 01:13:45.930 align:middle line:84% this results in focal sensitivity decreases. 01:13:45.930 --> 01:13:49.710 align:middle line:84% So specifically, you become less sensitive to spatial frequencies 01:13:49.710 --> 01:13:53.430 align:middle line:84% around that of the adapter rather than just generally less 01:13:53.430 --> 01:13:54.510 align:middle line:90% sensitive. 01:13:54.510 --> 01:13:57.390 align:middle line:84% That adaptation is orientation-specific, 01:13:57.390 --> 01:13:59.810 align:middle line:84% which suggests it probably happens cortically. 01:13:59.810 --> 01:14:02.162 align:middle line:90% 01:14:02.162 --> 01:14:03.870 align:middle line:84% We think these spatial frequency channels 01:14:03.870 --> 01:14:05.990 align:middle line:84% are useful for encoding different spatial scales 01:14:05.990 --> 01:14:08.030 align:middle line:90% in images. 01:14:08.030 --> 01:14:10.310 align:middle line:84% And so the neurons that we see in V1 01:14:10.310 --> 01:14:12.110 align:middle line:84% are tuned for both orientation-- that's 01:14:12.110 --> 01:14:14.050 align:middle line:84% what we talked about in the previous lecture-- 01:14:14.050 --> 01:14:15.790 align:middle line:90% and spatial frequency. 01:14:15.790 --> 01:14:18.710 align:middle line:84% And then we talked more broadly about the contrast sensitivity 01:14:18.710 --> 01:14:23.190 align:middle line:84% function as a thing that you can measure, that is 01:14:23.190 --> 01:14:28.830 align:middle line:84% what is used to actually see these effects of adaptation. 01:14:28.830 --> 01:14:29.830 align:middle line:90% All right. 01:14:29.830 --> 01:14:32.420 align:middle line:90% That's all I've got for you. 01:14:32.420 --> 01:14:56.000 align:middle line:90%