WEBVTT 00:00:00.000 --> 00:00:01.991 align:middle line:90% [SQUEAKING] 00:00:01.991 --> 00:00:03.482 align:middle line:90% [RUSTLING] 00:00:03.482 --> 00:00:04.973 align:middle line:90% [CLICKING] 00:00:04.973 --> 00:00:09.938 align:middle line:90% 00:00:09.938 --> 00:00:11.480 align:middle line:84% JOSH MCDERMOTT: Last time, we started 00:00:11.480 --> 00:00:15.980 align:middle line:84% talking about vision and the eye and the retina. 00:00:15.980 --> 00:00:20.280 align:middle line:84% So the idea is objects in the world reflect light. 00:00:20.280 --> 00:00:22.700 align:middle line:84% The photons are absorbed by the eye. 00:00:22.700 --> 00:00:26.220 align:middle line:84% And from the pattern of photons that are absorbed by the eye, 00:00:26.220 --> 00:00:28.600 align:middle line:84% your job is to figure out what's out there in the world. 00:00:28.600 --> 00:00:32.840 align:middle line:84% So we talked about the eye and the retina 00:00:32.840 --> 00:00:35.640 align:middle line:84% and how the retina is wired up backwards and so how 00:00:35.640 --> 00:00:38.120 align:middle line:84% that has the consequence that you have a blind spot where 00:00:38.120 --> 00:00:40.440 align:middle line:84% the optic nerve exits the eye and that there 00:00:40.440 --> 00:00:42.742 align:middle line:84% are these other adaptations to deal with the fact 00:00:42.742 --> 00:00:45.200 align:middle line:84% that it's backwards, like the cell body is being pushed out 00:00:45.200 --> 00:00:46.600 align:middle line:90% of the way. 00:00:46.600 --> 00:00:48.580 align:middle line:90% We observed our blind spot. 00:00:48.580 --> 00:00:50.840 align:middle line:84% We talked about rods and cones, how 00:00:50.840 --> 00:00:54.120 align:middle line:84% they have different distributions over the eye. 00:00:54.120 --> 00:00:57.480 align:middle line:84% So the cones are very dense in the fovea, the center of gaze, 00:00:57.480 --> 00:01:00.700 align:middle line:84% whereas there are no rods, and then the cones drop off 00:01:00.700 --> 00:01:02.560 align:middle line:90% precipitously after that. 00:01:02.560 --> 00:01:06.070 align:middle line:84% We talked about eccentricity, distance from the fovea. 00:01:06.070 --> 00:01:07.820 align:middle line:84% We talked about how the rods and the cones 00:01:07.820 --> 00:01:09.980 align:middle line:84% differ in their sensitivity to light. 00:01:09.980 --> 00:01:14.720 align:middle line:84% And so under high light conditions, photopic conditions, 00:01:14.720 --> 00:01:17.140 align:middle line:84% your vision is dominated by your cones. 00:01:17.140 --> 00:01:19.240 align:middle line:84% If you walk into a dark room or a movie theater, 00:01:19.240 --> 00:01:22.760 align:middle line:84% you gradually adapt to the low light levels. 00:01:22.760 --> 00:01:25.780 align:middle line:84% And what that means is there's an initial stage 00:01:25.780 --> 00:01:28.820 align:middle line:84% of adaptation within the cones, and then your vision 00:01:28.820 --> 00:01:32.500 align:middle line:84% transitions to being determined by the rods. 00:01:32.500 --> 00:01:35.662 align:middle line:84% And the rods eventually become extremely sensitive to light. 00:01:35.662 --> 00:01:38.120 align:middle line:84% So we talked about differences between rod and cone vision, 00:01:38.120 --> 00:01:40.900 align:middle line:84% how rod vision is more sensitive than cone vision, how 00:01:40.900 --> 00:01:43.580 align:middle line:84% it has lower acuity, and how you're not 00:01:43.580 --> 00:01:46.280 align:middle line:84% able to see color when you are dependent on rods 00:01:46.280 --> 00:01:48.140 align:middle line:84% because there's only one type of rod. 00:01:48.140 --> 00:01:50.097 align:middle line:90% Yeah? 00:01:50.097 --> 00:01:51.680 align:middle line:84% AUDIENCE: I might be misunderstanding, 00:01:51.680 --> 00:01:53.660 align:middle line:90% but can you go back? 00:01:53.660 --> 00:01:55.460 align:middle line:84% So in that one, why is the threshold 00:01:55.460 --> 00:02:00.500 align:middle line:84% of rods higher than the cones for the light, 00:02:00.500 --> 00:02:01.960 align:middle line:90% this kind of light? 00:02:01.960 --> 00:02:05.980 align:middle line:84% JOSH MCDERMOTT: Yeah, so under high light conditions, 00:02:05.980 --> 00:02:09.400 align:middle line:84% so photopic conditions, the rods are essentially not active. 00:02:09.400 --> 00:02:14.240 align:middle line:84% So that's why the sensitivity of the rods is very poor. 00:02:14.240 --> 00:02:20.280 align:middle line:84% And what happens when you enter low light levels is that there's 00:02:20.280 --> 00:02:24.080 align:middle line:84% a biochemical process inside the rod photoreceptor 00:02:24.080 --> 00:02:26.880 align:middle line:90% that restores that sensitivity. 00:02:26.880 --> 00:02:29.220 align:middle line:90% And it builds up over time. 00:02:29.220 --> 00:02:33.060 align:middle line:84% And so their sensitivity increases. 00:02:33.060 --> 00:02:34.625 align:middle line:84% And eventually, there's a transition 00:02:34.625 --> 00:02:37.000 align:middle line:84% where your threshold becomes determined by the rod rather 00:02:37.000 --> 00:02:39.560 align:middle line:90% than the cone. 00:02:39.560 --> 00:02:40.460 align:middle line:90% Does that make sense? 00:02:40.460 --> 00:02:47.160 align:middle line:90% 00:02:47.160 --> 00:02:49.292 align:middle line:84% And so we talked about how one of the things 00:02:49.292 --> 00:02:51.000 align:middle line:84% to remember about the cones is that there 00:02:51.000 --> 00:02:52.583 align:middle line:84% are three different types of them that 00:02:52.583 --> 00:02:54.440 align:middle line:84% have different spectral sensitivities. 00:02:54.440 --> 00:02:56.440 align:middle line:84% And that's critical to color vision, which we'll 00:02:56.440 --> 00:02:59.388 align:middle line:90% get into a little bit later. 00:02:59.388 --> 00:03:01.180 align:middle line:84% And then we talked about the ganglion cells 00:03:01.180 --> 00:03:03.960 align:middle line:84% and how the ganglion cells come in a couple different flavors. 00:03:03.960 --> 00:03:06.340 align:middle line:84% So ganglion cells, remember, are like the output stage 00:03:06.340 --> 00:03:07.160 align:middle line:90% of the retina. 00:03:07.160 --> 00:03:09.180 align:middle line:84% So there's a few different stages 00:03:09.180 --> 00:03:12.263 align:middle line:84% of neurons, starting with the photoreceptors, 00:03:12.263 --> 00:03:13.680 align:middle line:84% culminating in the ganglion cells. 00:03:13.680 --> 00:03:16.380 align:middle line:84% And the ganglion cells then send their axons out 00:03:16.380 --> 00:03:20.540 align:middle line:84% through the optic nerve to the thalamus and then the cortex. 00:03:20.540 --> 00:03:24.300 align:middle line:84% But there are a couple different types of ganglion cells, midget 00:03:24.300 --> 00:03:26.020 align:middle line:90% and parasol cells. 00:03:26.020 --> 00:03:28.660 align:middle line:84% They differ in a whole bunch of different ways, one of which 00:03:28.660 --> 00:03:30.500 align:middle line:90% is the receptive field size. 00:03:30.500 --> 00:03:32.300 align:middle line:84% And we talked about this other kind 00:03:32.300 --> 00:03:35.340 align:middle line:84% of critical property of receptive field size, which 00:03:35.340 --> 00:03:38.340 align:middle line:84% is that receptive fields tend to be small in the fovea 00:03:38.340 --> 00:03:41.800 align:middle line:84% and then get bigger as you move out to the periphery. 00:03:41.800 --> 00:03:44.140 align:middle line:84% So this is a major organizing principle 00:03:44.140 --> 00:03:45.540 align:middle line:90% within the visual system. 00:03:45.540 --> 00:03:47.860 align:middle line:84% But you can see that if you look at the ganglion cells, 00:03:47.860 --> 00:03:51.695 align:middle line:84% there are these two classes of receptive field size. 00:03:51.695 --> 00:03:53.820 align:middle line:84% And those correspond to the midget and the parasol. 00:03:53.820 --> 00:03:56.920 align:middle line:84% So the parasol cells have larger receptive fields. 00:03:56.920 --> 00:04:00.237 align:middle line:84% But overall, there is an increase in the receptive field 00:04:00.237 --> 00:04:01.820 align:middle line:84% size as you move out to the periphery, 00:04:01.820 --> 00:04:05.475 align:middle line:84% and that means a decrease in your spatial resolution. 00:04:05.475 --> 00:04:07.600 align:middle line:84% So spatial resolution is worse out in the periphery 00:04:07.600 --> 00:04:08.900 align:middle line:90% than it is in the fovea. 00:04:08.900 --> 00:04:11.600 align:middle line:84% And you're not normally very aware of this 00:04:11.600 --> 00:04:14.280 align:middle line:84% because typically anytime you want to see something, 00:04:14.280 --> 00:04:15.440 align:middle line:90% you look at it. 00:04:15.440 --> 00:04:18.040 align:middle line:84% And that means that you move your fovea to the thing 00:04:18.040 --> 00:04:19.200 align:middle line:90% that you want to see. 00:04:19.200 --> 00:04:22.170 align:middle line:84% But if you force yourself to fixate, 00:04:22.170 --> 00:04:23.420 align:middle line:90% you'll be able to detect this. 00:04:23.420 --> 00:04:25.480 align:middle line:90% And it's very measurable. 00:04:25.480 --> 00:04:27.320 align:middle line:84% So this is an eye chart that's intended 00:04:27.320 --> 00:04:29.800 align:middle line:84% to compensate for the eccentricity 00:04:29.800 --> 00:04:33.040 align:middle line:90% related changes in resolution. 00:04:33.040 --> 00:04:35.920 align:middle line:84% And so then we ended last time posing 00:04:35.920 --> 00:04:39.880 align:middle line:84% this question of why it is that vision is organized in this way. 00:04:39.880 --> 00:04:43.160 align:middle line:84% So why is it that we've evolved eyes 00:04:43.160 --> 00:04:46.480 align:middle line:84% that have really good resolution at the center of gaze? 00:04:46.480 --> 00:04:48.493 align:middle line:84% And the intuition and the proposal 00:04:48.493 --> 00:04:50.160 align:middle line:84% that was tested in this particular paper 00:04:50.160 --> 00:04:55.140 align:middle line:84% that we started to talk about is that if there 00:04:55.140 --> 00:04:58.980 align:middle line:84% is a limit on the number of spatial samples 00:04:58.980 --> 00:05:00.460 align:middle line:90% that you can make-- 00:05:00.460 --> 00:05:04.900 align:middle line:84% and that limit might be due to the size of the optic nerve-- 00:05:04.900 --> 00:05:07.333 align:middle line:84% then you have some finite number of receptors 00:05:07.333 --> 00:05:09.000 align:middle line:84% that you could kind of place in the eye. 00:05:09.000 --> 00:05:11.458 align:middle line:84% And there's different ways in which you could arrange them. 00:05:11.458 --> 00:05:15.740 align:middle line:84% But one way to do it is to put a very high density of receptors 00:05:15.740 --> 00:05:17.540 align:middle line:84% in one particular part of the eye 00:05:17.540 --> 00:05:20.900 align:middle line:84% and then enable the system to move the eye around so that it 00:05:20.900 --> 00:05:25.200 align:middle line:84% can do this dense sampling at a particular region of the image 00:05:25.200 --> 00:05:27.140 align:middle line:90% and then select where that is. 00:05:27.140 --> 00:05:29.420 align:middle line:84% And so this was a paper that was attempting 00:05:29.420 --> 00:05:31.900 align:middle line:84% to investigate whether that's actually 00:05:31.900 --> 00:05:35.420 align:middle line:84% a good solution to the problem of vision. 00:05:35.420 --> 00:05:37.540 align:middle line:84% And so what they did in this paper 00:05:37.540 --> 00:05:40.540 align:middle line:84% is set up a machine system that had 00:05:40.540 --> 00:05:42.700 align:middle line:84% a receptor lattice the parameters of which 00:05:42.700 --> 00:05:44.000 align:middle line:90% could be optimized. 00:05:44.000 --> 00:05:45.860 align:middle line:84% So there's a lattice of receptors 00:05:45.860 --> 00:05:48.940 align:middle line:84% that are defined by the positions, mu, 00:05:48.940 --> 00:05:51.720 align:middle line:84% and the size of the receptive field of the receptor. 00:05:51.720 --> 00:05:54.400 align:middle line:90% That's sigma. 00:05:54.400 --> 00:05:59.120 align:middle line:84% And this system was trained to perform a visual search task. 00:05:59.120 --> 00:06:00.500 align:middle line:90% So there were images like this. 00:06:00.500 --> 00:06:03.200 align:middle line:84% And the task was to find and identify 00:06:03.200 --> 00:06:06.320 align:middle line:84% the digit on a cluttered background. 00:06:06.320 --> 00:06:09.160 align:middle line:84% And so what they found is that when they optimized 00:06:09.160 --> 00:06:12.000 align:middle line:84% the parameters of the lattice in order 00:06:12.000 --> 00:06:14.280 align:middle line:84% to enable the system to solve this problem, 00:06:14.280 --> 00:06:16.060 align:middle line:84% over the course of the optimization, 00:06:16.060 --> 00:06:20.160 align:middle line:84% you can see that the receptor positions and sizes kind arrange 00:06:20.160 --> 00:06:23.560 align:middle line:84% themselves to resemble the organization that you see 00:06:23.560 --> 00:06:27.120 align:middle line:84% in the retina, which is to say there is a center of gaze, where 00:06:27.120 --> 00:06:30.000 align:middle line:84% the receptors are fairly densely spaced with small receptive 00:06:30.000 --> 00:06:31.627 align:middle line:84% fields, and then a periphery, where 00:06:31.627 --> 00:06:33.960 align:middle line:84% the spacing kind of gets bigger and the receptive fields 00:06:33.960 --> 00:06:37.220 align:middle line:90% get broader. 00:06:37.220 --> 00:06:39.840 align:middle line:84% So that's kind of where we wrapped up. 00:06:39.840 --> 00:06:44.480 align:middle line:84% And so interestingly, in this kind of artificial setting, 00:06:44.480 --> 00:06:48.640 align:middle line:84% they could kind of simulate problem constraints that 00:06:48.640 --> 00:06:51.340 align:middle line:90% don't really exist in biology. 00:06:51.340 --> 00:06:53.180 align:middle line:84% And so one of those problem constraints 00:06:53.180 --> 00:06:57.820 align:middle line:84% was enabling the system to have a zoom function in addition 00:06:57.820 --> 00:07:00.620 align:middle line:84% to simply being able to translate the receptor 00:07:00.620 --> 00:07:01.900 align:middle line:90% lattice around. 00:07:01.900 --> 00:07:06.260 align:middle line:84% And when the system was allowed to zoom, 00:07:06.260 --> 00:07:08.800 align:middle line:84% you get a different kind of solution emerging. 00:07:08.800 --> 00:07:10.820 align:middle line:84% So if it can zoom, then it turns out 00:07:10.820 --> 00:07:13.340 align:middle line:84% to be better to just have the receptors more or less 00:07:13.340 --> 00:07:14.820 align:middle line:90% uniformly spaced. 00:07:14.820 --> 00:07:18.580 align:middle line:84% So that's kind of like a CCD imaging device 00:07:18.580 --> 00:07:20.500 align:middle line:90% like is in your phone. 00:07:20.500 --> 00:07:23.780 align:middle line:84% But when you can translate only, then you 00:07:23.780 --> 00:07:26.880 align:middle line:84% end up with these foveal-like organizations. 00:07:26.880 --> 00:07:29.780 align:middle line:84% And so what these graphs are plotting 00:07:29.780 --> 00:07:32.180 align:middle line:84% is the sampling interval or the spacing 00:07:32.180 --> 00:07:34.380 align:middle line:84% between the receptors as a function 00:07:34.380 --> 00:07:37.620 align:middle line:84% of the distance from what they define to be the center. 00:07:37.620 --> 00:07:39.797 align:middle line:84% And you can see that the spacing kind of increases. 00:07:39.797 --> 00:07:42.380 align:middle line:84% And these are two different data sets that they trained it on, 00:07:42.380 --> 00:07:46.460 align:middle line:84% whereas in the situation where the system can zoom, 00:07:46.460 --> 00:07:48.180 align:middle line:84% you see a much weaker dependence. 00:07:48.180 --> 00:07:51.000 align:middle line:84% And then this is the receptive field size 00:07:51.000 --> 00:07:55.480 align:middle line:90% as a function of eccentricity. 00:07:55.480 --> 00:07:57.800 align:middle line:84% So the other thing that they show in this paper 00:07:57.800 --> 00:08:02.292 align:middle line:84% is how the models end up using their retinas when 00:08:02.292 --> 00:08:03.500 align:middle line:90% they're performing this task. 00:08:03.500 --> 00:08:05.208 align:middle line:84% So remember, this is like a visual search 00:08:05.208 --> 00:08:07.560 align:middle line:84% task, where you have to find and recognize the digit. 00:08:07.560 --> 00:08:12.320 align:middle line:84% And so these are different time intervals in a particular trial. 00:08:12.320 --> 00:08:14.040 align:middle line:90% So this is the stimulus. 00:08:14.040 --> 00:08:16.720 align:middle line:84% And you can see that the model is essentially 00:08:16.720 --> 00:08:19.200 align:middle line:84% moving its receptor lattice so that it's 00:08:19.200 --> 00:08:20.740 align:middle line:90% located over the digit. 00:08:20.740 --> 00:08:27.200 align:middle line:84% And in the case where it's allowed to translate, 00:08:27.200 --> 00:08:29.640 align:middle line:84% you get the fovea positioned over the digit, 00:08:29.640 --> 00:08:32.039 align:middle line:84% whereas when it's allowed to zoom, 00:08:32.039 --> 00:08:37.320 align:middle line:84% it hones in on the digit in a different way. 00:08:37.320 --> 00:08:41.880 align:middle line:84% So the inference from this is that a fovea 00:08:41.880 --> 00:08:46.840 align:middle line:84% like what we have in our eyes is a useful way 00:08:46.840 --> 00:08:51.140 align:middle line:84% to attain high resolution vision given, one, 00:08:51.140 --> 00:08:54.180 align:middle line:84% a limit on the number of samples that can be transmitted 00:08:54.180 --> 00:08:58.260 align:middle line:84% to the brain, in this case from the optic nerve, 00:08:58.260 --> 00:09:02.100 align:middle line:84% and two, the ability to move the eyes to position the fovea 00:09:02.100 --> 00:09:05.800 align:middle line:90% at different locations. 00:09:05.800 --> 00:09:09.940 align:middle line:84% So the idea is that our visual system has kind of arrived 00:09:09.940 --> 00:09:14.580 align:middle line:84% at a good solution to the problem of getting 00:09:14.580 --> 00:09:16.620 align:middle line:84% high resolution without kind of having 00:09:16.620 --> 00:09:19.380 align:middle line:84% this enormous bandwidth that would be difficult to pass on 00:09:19.380 --> 00:09:20.700 align:middle line:90% to the brain. 00:09:20.700 --> 00:09:22.000 align:middle line:90% Any questions about that? 00:09:22.000 --> 00:09:31.620 align:middle line:90% 00:09:31.620 --> 00:09:36.500 align:middle line:84% And so the bigger picture here, again, is the current era 00:09:36.500 --> 00:09:38.380 align:middle line:84% that we're in is giving us the opportunity 00:09:38.380 --> 00:09:40.500 align:middle line:84% to answer some of these questions 00:09:40.500 --> 00:09:42.980 align:middle line:84% about why our sensory systems are 00:09:42.980 --> 00:09:47.747 align:middle line:84% the way they are by taking in silico systems 00:09:47.747 --> 00:09:49.580 align:middle line:84% and being able to optimize them for problems 00:09:49.580 --> 00:09:51.372 align:middle line:84% and then looking at what kinds of solutions 00:09:51.372 --> 00:09:54.280 align:middle line:90% emerge as a consequence of that. 00:09:54.280 --> 00:09:55.980 align:middle line:84% AUDIENCE: So basically, in the bottom 00:09:55.980 --> 00:09:59.600 align:middle line:84% left in examples for those two data sets, 00:09:59.600 --> 00:10:01.620 align:middle line:84% there's different placements of the fovea. 00:10:01.620 --> 00:10:08.240 align:middle line:84% And I'm curious how much we can generalize optimization machine 00:10:08.240 --> 00:10:10.460 align:middle line:84% learning experiments to the real world. 00:10:10.460 --> 00:10:13.095 align:middle line:84% It's so dependent on the type of data that you feed it. 00:10:13.095 --> 00:10:15.220 align:middle line:84% JOSH MCDERMOTT: Yeah, I think it's a good question. 00:10:15.220 --> 00:10:18.140 align:middle line:84% I'm not actually sure that this is data set dependent. 00:10:18.140 --> 00:10:21.080 align:middle line:84% It could just be that you'd get similar variants 00:10:21.080 --> 00:10:25.160 align:middle line:84% with different random initializations. 00:10:25.160 --> 00:10:27.380 align:middle line:84% You'd have to run the experiments multiple times. 00:10:27.380 --> 00:10:31.000 align:middle line:84% But yeah, it is showing you that you 00:10:31.000 --> 00:10:33.220 align:middle line:84% can get-- there's some variability that is emerging. 00:10:33.220 --> 00:10:35.860 align:middle line:84% On the other hand, both of them kind of have a fovea. 00:10:35.860 --> 00:10:38.240 align:middle line:84% It's just like one of them's a little bit lopsided 00:10:38.240 --> 00:10:39.120 align:middle line:90% for whatever reason. 00:10:39.120 --> 00:10:41.120 align:middle line:84% So yeah, you'd have to do some more experiments, 00:10:41.120 --> 00:10:42.100 align:middle line:90% I think, to understand that. 00:10:42.100 --> 00:10:44.183 align:middle line:84% But I think the point is you could understand that 00:10:44.183 --> 00:10:48.220 align:middle line:84% by looking in more detail at the conditions that give rise 00:10:48.220 --> 00:10:49.000 align:middle line:90% to these things. 00:10:49.000 --> 00:10:53.780 align:middle line:90% 00:10:53.780 --> 00:10:57.860 align:middle line:84% So these ganglion cells and other cells 00:10:57.860 --> 00:11:03.300 align:middle line:84% in the retina come in-- there's another set of flavors 00:11:03.300 --> 00:11:04.980 align:middle line:90% that they come in. 00:11:04.980 --> 00:11:08.100 align:middle line:84% And the one that we're going to talk about now, 00:11:08.100 --> 00:11:10.340 align:middle line:84% it has to do with center-surround receptive field 00:11:10.340 --> 00:11:14.400 align:middle line:84% organization and the two different types of that. 00:11:14.400 --> 00:11:17.460 align:middle line:84% And so this was a very early discovery about by the retina 00:11:17.460 --> 00:11:20.780 align:middle line:84% credited to Kuffler in the 1950s, 00:11:20.780 --> 00:11:24.580 align:middle line:84% who recorded from neurons in the retina, from ganglion cells, 00:11:24.580 --> 00:11:28.900 align:middle line:84% and was investigating the patterns of light that would 00:11:28.900 --> 00:11:31.240 align:middle line:90% cause the neurons to fire. 00:11:31.240 --> 00:11:33.860 align:middle line:84% And what he discovered was evidence 00:11:33.860 --> 00:11:38.140 align:middle line:84% that you can think of the neuron as having 00:11:38.140 --> 00:11:40.420 align:middle line:84% this center-surround receptive field, where there 00:11:40.420 --> 00:11:45.120 align:middle line:84% is a center which is defined by the fact 00:11:45.120 --> 00:11:47.640 align:middle line:84% that if you stimulate that with light, 00:11:47.640 --> 00:11:50.200 align:middle line:84% that will increase the response of the cell. 00:11:50.200 --> 00:11:52.960 align:middle line:84% And a surround, which is defined by the fact 00:11:52.960 --> 00:11:55.620 align:middle line:84% that if you stimulate that with light, 00:11:55.620 --> 00:11:58.920 align:middle line:84% that will tend to decrease the response of the cell. 00:11:58.920 --> 00:12:01.040 align:middle line:84% And so the consequence of that is 00:12:01.040 --> 00:12:04.760 align:middle line:84% that the best stimulus for the neuron in this case 00:12:04.760 --> 00:12:08.200 align:middle line:84% is going to be this one here, where you're just stimulating 00:12:08.200 --> 00:12:09.480 align:middle line:90% the center with light. 00:12:09.480 --> 00:12:11.760 align:middle line:84% And if you present an annulus, such 00:12:11.760 --> 00:12:15.320 align:middle line:84% that you're just hitting the surround, you get no response. 00:12:15.320 --> 00:12:17.400 align:middle line:84% And if you have a bigger disc, such that you're 00:12:17.400 --> 00:12:19.275 align:middle line:84% stimulating both the center and the surround, 00:12:19.275 --> 00:12:21.120 align:middle line:84% the response will be weaker than if you just 00:12:21.120 --> 00:12:23.200 align:middle line:90% stimulate the center. 00:12:23.200 --> 00:12:24.960 align:middle line:84% So the neuron is kind of computing 00:12:24.960 --> 00:12:28.480 align:middle line:84% a difference between the center and the surround. 00:12:28.480 --> 00:12:33.700 align:middle line:84% And this is an example of how this used to work. 00:12:33.700 --> 00:12:37.800 align:middle line:84% So what this is, what I would normally show you, 00:12:37.800 --> 00:12:42.120 align:middle line:84% is a video of an experiment where 00:12:42.120 --> 00:12:45.020 align:middle line:84% they were recording from one of these ganglion cells. 00:12:45.020 --> 00:12:48.160 align:middle line:84% And then this is a projector screen, 00:12:48.160 --> 00:12:51.480 align:middle line:84% just like this one, on which light is being shined. 00:12:51.480 --> 00:12:54.060 align:middle line:84% But what you have to really be able to do in order 00:12:54.060 --> 00:12:57.180 align:middle line:84% to appreciate this is hear the sound because what happens 00:12:57.180 --> 00:12:59.940 align:middle line:84% is the electrode that's measuring 00:12:59.940 --> 00:13:02.380 align:middle line:84% the response of the cell is being played out 00:13:02.380 --> 00:13:03.480 align:middle line:90% as an audio signal. 00:13:03.480 --> 00:13:05.397 align:middle line:84% And so every time there's an action potential, 00:13:05.397 --> 00:13:06.880 align:middle line:90% you'll hear a click. 00:13:06.880 --> 00:13:09.300 align:middle line:84% So you have to be able to hear that. 00:13:09.300 --> 00:13:12.400 align:middle line:84% But what you would see if we could play this, 00:13:12.400 --> 00:13:14.540 align:middle line:84% and what you would hear, is that there 00:13:14.540 --> 00:13:17.900 align:middle line:84% is this region of space that corresponds 00:13:17.900 --> 00:13:19.560 align:middle line:84% to the center of the receptive field. 00:13:19.560 --> 00:13:22.500 align:middle line:84% And when there's a spot of light that is in there, 00:13:22.500 --> 00:13:23.640 align:middle line:90% there's a brisk response. 00:13:23.640 --> 00:13:27.060 align:middle line:84% And if you make the disc larger, the response decreases. 00:13:27.060 --> 00:13:30.440 align:middle line:84% So we're going to have to play those next time. 00:13:30.440 --> 00:13:34.680 align:middle line:84% So there's two types of these receptive fields, 00:13:34.680 --> 00:13:39.100 align:middle line:84% one that we describe as an on-center, off-surround 00:13:39.100 --> 00:13:40.260 align:middle line:90% receptive field. 00:13:40.260 --> 00:13:42.080 align:middle line:84% So the center produces an excitatory 00:13:42.080 --> 00:13:45.760 align:middle line:84% response and the surrounded an inhibitory response, and then 00:13:45.760 --> 00:13:49.420 align:middle line:84% another one that we describe as an off-center, on-surround cell, 00:13:49.420 --> 00:13:55.640 align:middle line:84% where it's kind of the opposite, so two different types. 00:13:55.640 --> 00:13:57.960 align:middle line:84% And so you can think of the on-center cells 00:13:57.960 --> 00:14:00.540 align:middle line:84% as responding to local luminance increments. 00:14:00.540 --> 00:14:04.640 align:middle line:84% And the off-center is responding to local luminance decrements. 00:14:04.640 --> 00:14:10.920 align:middle line:84% So this was probably the first example 00:14:10.920 --> 00:14:14.460 align:middle line:84% of a discovery of the structure of receptive fields. 00:14:14.460 --> 00:14:19.400 align:middle line:84% And that kind of led to a pretty large and rich research program 00:14:19.400 --> 00:14:22.560 align:middle line:84% that we'll talk more about when we get into V1 00:14:22.560 --> 00:14:24.800 align:middle line:84% and extended throughout, like a lot of the rest 00:14:24.800 --> 00:14:26.360 align:middle line:90% of the visual system. 00:14:26.360 --> 00:14:29.900 align:middle line:84% But in these early stages of the visual system, 00:14:29.900 --> 00:14:33.440 align:middle line:84% this notion of receptive field is especially powerful, in part 00:14:33.440 --> 00:14:37.000 align:middle line:84% because the neurons can be described in fairly simple terms 00:14:37.000 --> 00:14:38.000 align:middle line:90% mathematically. 00:14:38.000 --> 00:14:40.540 align:middle line:84% And so we have the ability to I'm 00:14:40.540 --> 00:14:43.020 align:middle line:84% going to write down pretty simple mathematical models that 00:14:43.020 --> 00:14:44.700 align:middle line:84% do a really good job of capturing 00:14:44.700 --> 00:14:46.540 align:middle line:90% the responses of the neurons. 00:14:46.540 --> 00:14:52.580 align:middle line:84% So what we're going to do now is get into that just a little bit. 00:14:52.580 --> 00:14:56.110 align:middle line:90% So this is an image. 00:14:56.110 --> 00:14:57.360 align:middle line:90% It's an image of Mickey Mouse. 00:14:57.360 --> 00:14:58.580 align:middle line:90% But it's just an image. 00:14:58.580 --> 00:15:02.500 align:middle line:84% And so what an image is, and for our purposes, 00:15:02.500 --> 00:15:06.320 align:middle line:84% is it's a spatial array of intensities. 00:15:06.320 --> 00:15:08.320 align:middle line:90% So that's I of x, y. 00:15:08.320 --> 00:15:10.100 align:middle line:90% So I is the intensity. 00:15:10.100 --> 00:15:13.460 align:middle line:84% And there will be an intensity value at every x position 00:15:13.460 --> 00:15:16.500 align:middle line:90% and every y position. 00:15:16.500 --> 00:15:20.460 align:middle line:84% Now, we're going to try to describe the neuron and its 00:15:20.460 --> 00:15:25.900 align:middle line:84% receptive field as a filter So the filter will be G. 00:15:25.900 --> 00:15:28.400 align:middle line:84% And G will operate on the input I. 00:15:28.400 --> 00:15:34.180 align:middle line:84% And that will produce a response L. Now, 00:15:34.180 --> 00:15:38.860 align:middle line:84% because these are neurons-- and we talked a little bit about 00:15:38.860 --> 00:15:41.640 align:middle line:84% this a class or two ago in response to a question-- 00:15:41.640 --> 00:15:43.660 align:middle line:84% the output of the neuron is action potentials. 00:15:43.660 --> 00:15:46.860 align:middle line:84% And so in addition to this filtering operation, 00:15:46.860 --> 00:15:49.520 align:middle line:84% there is an output non-linearity that will actually 00:15:49.520 --> 00:15:54.840 align:middle line:84% generate spikes that will turn the response into a firing rate. 00:15:54.840 --> 00:15:59.700 align:middle line:84% And then we'll sample spikes from that firing rate. 00:15:59.700 --> 00:16:01.800 align:middle line:84% So that's going to be the mathematical model 00:16:01.800 --> 00:16:03.180 align:middle line:90% of the neuron. 00:16:03.180 --> 00:16:05.480 align:middle line:84% And we will attempt to see whether this 00:16:05.480 --> 00:16:09.340 align:middle line:84% does a good job of accounting for those responses 00:16:09.340 --> 00:16:12.040 align:middle line:84% So the assumption that we will make here 00:16:12.040 --> 00:16:13.560 align:middle line:84% is that the response of the neuron 00:16:13.560 --> 00:16:16.860 align:middle line:84% is given by a linear filter that acts on the stimulus. 00:16:16.860 --> 00:16:18.200 align:middle line:90% So what does that mean? 00:16:18.200 --> 00:16:21.320 align:middle line:84% It means that the response of the filter 00:16:21.320 --> 00:16:23.380 align:middle line:90% is given by this equation. 00:16:23.380 --> 00:16:24.940 align:middle line:90% So this is the image. 00:16:24.940 --> 00:16:28.440 align:middle line:84% So this is just an array of gray values. 00:16:28.440 --> 00:16:32.400 align:middle line:84% This is the filter, which is another array of numbers 00:16:32.400 --> 00:16:35.520 align:middle line:84% that are a function of spatial position. 00:16:35.520 --> 00:16:37.760 align:middle line:84% And then we're taking the dot product 00:16:37.760 --> 00:16:39.607 align:middle line:90% of the filter in the image. 00:16:39.607 --> 00:16:40.440 align:middle line:90% What does that mean? 00:16:40.440 --> 00:16:44.220 align:middle line:84% It means that at every point in this spatial array, 00:16:44.220 --> 00:16:48.380 align:middle line:84% we take the product of the image intensity and the filter 00:16:48.380 --> 00:16:49.820 align:middle line:90% coefficient. 00:16:49.820 --> 00:16:51.480 align:middle line:84% And then we add up all those products. 00:16:51.480 --> 00:16:56.720 align:middle line:84% That's what this double integral is with respect to both x and y. 00:16:56.720 --> 00:17:00.500 align:middle line:84% So here's an example filter G. That's a center-surround filter. 00:17:00.500 --> 00:17:03.920 align:middle line:84% So if we're looking at it in image coordinates, 00:17:03.920 --> 00:17:05.700 align:middle line:90% that's what it would look like. 00:17:05.700 --> 00:17:07.200 align:middle line:84% If you just take a slice through it, 00:17:07.200 --> 00:17:10.660 align:middle line:84% which is sometimes kind of easier to think about, 00:17:10.660 --> 00:17:14.060 align:middle line:90% you get this looking thing. 00:17:14.060 --> 00:17:16.040 align:middle line:90% That's just in the x direction. 00:17:16.040 --> 00:17:18.040 align:middle line:84% So we're going to look at this in one dimension. 00:17:18.040 --> 00:17:21.780 align:middle line:84% So now, we just have a single interval with respect to x. 00:17:21.780 --> 00:17:23.460 align:middle line:90% So here's our filter. 00:17:23.460 --> 00:17:25.920 align:middle line:90% And this is one example image. 00:17:25.920 --> 00:17:28.820 align:middle line:84% This is a very special image because it has 00:17:28.820 --> 00:17:30.580 align:middle line:90% the same form as the filter. 00:17:30.580 --> 00:17:34.300 align:middle line:84% And the consequence of that is that when 00:17:34.300 --> 00:17:39.800 align:middle line:84% we evaluate the response of the filter, 00:17:39.800 --> 00:17:42.880 align:middle line:84% we take the product here of G of x And I of x. 00:17:42.880 --> 00:17:44.550 align:middle line:84% So at every different position here, 00:17:44.550 --> 00:17:46.800 align:middle line:84% we're taking the number here, and we're multiplying it 00:17:46.800 --> 00:17:48.740 align:middle line:90% by the number here. 00:17:48.740 --> 00:17:52.760 align:middle line:84% And because I of x is the same as G of x, where G of x 00:17:52.760 --> 00:17:55.880 align:middle line:84% is negative, I of x is negative, and you get a positive number. 00:17:55.880 --> 00:17:58.162 align:middle line:84% When G of x is positive, I of x is positive, 00:17:58.162 --> 00:17:59.620 align:middle line:84% and you also get a positive number. 00:17:59.620 --> 00:18:01.840 align:middle line:84% So you have all these positive numbers. 00:18:01.840 --> 00:18:05.000 align:middle line:84% So now, when you add all these things up, 00:18:05.000 --> 00:18:10.200 align:middle line:84% you take the integral, you get something big, a big response. 00:18:10.200 --> 00:18:13.960 align:middle line:84% By comparison, here's the same filter but now applied 00:18:13.960 --> 00:18:16.320 align:middle line:90% to a fairly uniform image. 00:18:16.320 --> 00:18:18.760 align:middle line:84% So what's going to happen here is now the image 00:18:18.760 --> 00:18:21.018 align:middle line:90% is kind of positive everywhere. 00:18:21.018 --> 00:18:23.060 align:middle line:84% And that means that where the filter is negative, 00:18:23.060 --> 00:18:25.945 align:middle line:84% we get negative numbers as the product. 00:18:25.945 --> 00:18:27.320 align:middle line:84% And where the filter is positive, 00:18:27.320 --> 00:18:28.800 align:middle line:90% we get positive numbers. 00:18:28.800 --> 00:18:31.320 align:middle line:84% And so now when we add all this stuff up, 00:18:31.320 --> 00:18:33.160 align:middle line:84% the negative numbers kind of partially 00:18:33.160 --> 00:18:34.500 align:middle line:90% cancel out the positive numbers. 00:18:34.500 --> 00:18:36.290 align:middle line:84% And so we get a smaller response. 00:18:36.290 --> 00:18:40.930 align:middle line:90% 00:18:40.930 --> 00:18:45.950 align:middle line:84% So that's the basic concept for how these filters work. 00:18:45.950 --> 00:18:48.670 align:middle line:84% So how can we measure receptive fields? 00:18:48.670 --> 00:18:50.370 align:middle line:84% Well, remember a couple of lectures ago, 00:18:50.370 --> 00:18:53.150 align:middle line:84% we talked about the idea of the spike-triggered average 00:18:53.150 --> 00:18:55.330 align:middle line:84% as one way to measure these filters. 00:18:55.330 --> 00:19:00.310 align:middle line:84% And so the way that works is you take a random flicker stimulus. 00:19:00.310 --> 00:19:02.770 align:middle line:84% So this is just a sequence of random images. 00:19:02.770 --> 00:19:04.870 align:middle line:90% It's a movie. 00:19:04.870 --> 00:19:08.710 align:middle line:84% | present that to an organism whose retina 00:19:08.710 --> 00:19:11.362 align:middle line:90% you are recording from. 00:19:11.362 --> 00:19:12.570 align:middle line:90% This is the stimulus history. 00:19:12.570 --> 00:19:14.090 align:middle line:90% It's a sequence of frames. 00:19:14.090 --> 00:19:16.990 align:middle line:84% And then every time there's an action potential, 00:19:16.990 --> 00:19:20.490 align:middle line:84% you take the stimulus history that immediately preceded that 00:19:20.490 --> 00:19:23.390 align:middle line:90% and you add that to your buffer. 00:19:23.390 --> 00:19:25.670 align:middle line:84% And you take all of those little sets of frames, 00:19:25.670 --> 00:19:28.010 align:middle line:84% and you get an average sequence of frames. 00:19:28.010 --> 00:19:31.310 align:middle line:84% This is like the average movie that preceded a spike. 00:19:31.310 --> 00:19:33.110 align:middle line:84% So that's the spike-triggered average. 00:19:33.110 --> 00:19:37.690 align:middle line:84% Every time there's a spike, you trigger this averaging process. 00:19:37.690 --> 00:19:41.610 align:middle line:84% So here are examples of these kinds of receptive fields 00:19:41.610 --> 00:19:43.230 align:middle line:84% that you can measure in the retina. 00:19:43.230 --> 00:19:44.470 align:middle line:90% So here's one. 00:19:44.470 --> 00:19:47.610 align:middle line:90% 00:19:47.610 --> 00:19:48.230 align:middle line:90% See that? 00:19:48.230 --> 00:19:49.070 align:middle line:90% I'll try it again. 00:19:49.070 --> 00:19:52.310 align:middle line:90% 00:19:52.310 --> 00:19:55.010 align:middle line:84% So there's a little blob that kind of shows up right. 00:19:55.010 --> 00:19:59.710 align:middle line:84% Here's another, different color blob. 00:19:59.710 --> 00:20:02.890 align:middle line:90% So that's what you get. 00:20:02.890 --> 00:20:07.310 align:middle line:84% If we look at an example of this over time-- 00:20:07.310 --> 00:20:11.230 align:middle line:84% so the spike-triggered average is this movie right. 00:20:11.230 --> 00:20:12.590 align:middle line:90% That's a sequence of frames. 00:20:12.590 --> 00:20:15.290 align:middle line:84% So here's each one of those frames 00:20:15.290 --> 00:20:18.127 align:middle line:90% or some subsampling of them. 00:20:18.127 --> 00:20:20.210 align:middle line:84% And you can see that there's this little blob here 00:20:20.210 --> 00:20:21.810 align:middle line:90% that starts out dark. 00:20:21.810 --> 00:20:24.690 align:middle line:90% It kind of then becomes light. 00:20:24.690 --> 00:20:26.990 align:middle line:84% And this is the time preceding the spike. 00:20:26.990 --> 00:20:32.870 align:middle line:90% 00:20:32.870 --> 00:20:35.630 align:middle line:84% Here is just a quantification of that. 00:20:35.630 --> 00:20:39.110 align:middle line:84% So there's this inner region that's 00:20:39.110 --> 00:20:43.430 align:middle line:84% been defined here by the dashed line, and then an outer region, 00:20:43.430 --> 00:20:46.730 align:middle line:84% and then this region beyond that. 00:20:46.730 --> 00:20:49.830 align:middle line:84% And so this is the plot that kind of averages 00:20:49.830 --> 00:20:53.430 align:middle line:84% all of the pixels inside the inner outline 00:20:53.430 --> 00:20:55.650 align:middle line:84% and then outside the outer outline. 00:20:55.650 --> 00:20:58.270 align:middle line:84% And there's three different plots here 00:20:58.270 --> 00:21:01.170 align:middle line:84% because there are three different color channels. 00:21:01.170 --> 00:21:04.190 align:middle line:84% So this is a movie, each image of which 00:21:04.190 --> 00:21:06.470 align:middle line:90% has R, G, and B channels. 00:21:06.470 --> 00:21:10.390 align:middle line:84% So there's actually three images in every frame. 00:21:10.390 --> 00:21:12.830 align:middle line:84% So we can compute the averages separately 00:21:12.830 --> 00:21:16.155 align:middle line:84% for the red, green, and blue channels. 00:21:16.155 --> 00:21:18.030 align:middle line:84% And so you can see there are some differences 00:21:18.030 --> 00:21:19.550 align:middle line:90% between the channels. 00:21:19.550 --> 00:21:26.590 align:middle line:84% But essentially, in the center, so inside the inner outline, 00:21:26.590 --> 00:21:28.070 align:middle line:84% you get this particular shape where 00:21:28.070 --> 00:21:30.250 align:middle line:84% there's an increase and then a decrease. 00:21:30.250 --> 00:21:32.150 align:middle line:84% And then outside the outer outline, 00:21:32.150 --> 00:21:34.530 align:middle line:84% you get this thing that's kind of the opposite. 00:21:34.530 --> 00:21:38.830 align:middle line:84% So that's the center-surround receptive field organization. 00:21:38.830 --> 00:21:40.205 align:middle line:84% So what's happening in the center 00:21:40.205 --> 00:21:41.580 align:middle line:84% is kind of the opposite of what's 00:21:41.580 --> 00:21:42.790 align:middle line:90% happening in this surround. 00:21:42.790 --> 00:21:46.610 align:middle line:84% But you can also see, there's these temporal dynamics too, 00:21:46.610 --> 00:21:50.210 align:middle line:84% so this sort of picture here with these spatial receptive 00:21:50.210 --> 00:21:53.290 align:middle line:90% fields is part of the story. 00:21:53.290 --> 00:21:57.570 align:middle line:84% But there's also these temporal dynamics. 00:21:57.570 --> 00:22:01.410 align:middle line:84% So this is how you would evaluate a filter at one 00:22:01.410 --> 00:22:02.830 align:middle line:90% particular point in space. 00:22:02.830 --> 00:22:04.890 align:middle line:84% So that would be like one receptive field 00:22:04.890 --> 00:22:07.330 align:middle line:90% at one particular location. 00:22:07.330 --> 00:22:08.950 align:middle line:90% Any questions about that? 00:22:08.950 --> 00:22:17.088 align:middle line:90% 00:22:17.088 --> 00:22:18.630 align:middle line:84% So this was-- and I should have said, 00:22:18.630 --> 00:22:22.350 align:middle line:84% this is how you would evaluate the filter response. 00:22:22.350 --> 00:22:25.090 align:middle line:84% And then this is how you measure it. 00:22:25.090 --> 00:22:31.630 align:middle line:84% So another kind of important idea in this space 00:22:31.630 --> 00:22:34.590 align:middle line:90% is the idea of convolution. 00:22:34.590 --> 00:22:38.590 align:middle line:84% So convolution is an operation that 00:22:38.590 --> 00:22:41.870 align:middle line:84% produces the output of a linear filter 00:22:41.870 --> 00:22:46.310 align:middle line:84% at each location in an image or every time point in some time 00:22:46.310 --> 00:22:47.030 align:middle line:90% series. 00:22:47.030 --> 00:22:52.150 align:middle line:84% And convolution comes up a lot in different contexts 00:22:52.150 --> 00:22:55.350 align:middle line:90% in signal processing. 00:22:55.350 --> 00:22:58.390 align:middle line:84% In how we think about vision, convolution 00:22:58.390 --> 00:23:01.270 align:middle line:84% has a very particular role because it's often 00:23:01.270 --> 00:23:05.310 align:middle line:84% used to simulate the effects of a large set of neurons 00:23:05.310 --> 00:23:07.110 align:middle line:84% with the same type of receptive field 00:23:07.110 --> 00:23:09.150 align:middle line:90% at different spatial positions. 00:23:09.150 --> 00:23:11.110 align:middle line:84% And so the idea is like we've got 00:23:11.110 --> 00:23:13.130 align:middle line:84% these centers-surround receptive fields. 00:23:13.130 --> 00:23:16.910 align:middle line:84% So in this particular example back here, 00:23:16.910 --> 00:23:20.470 align:middle line:84% this is a receptive field that was measured from one neuron, 00:23:20.470 --> 00:23:24.630 align:middle line:84% and it's at a particular location on the retina. 00:23:24.630 --> 00:23:27.610 align:middle line:84% But there's a ton of ganglion cells. 00:23:27.610 --> 00:23:30.090 align:middle line:84% Most of them have these center-surround receptive field 00:23:30.090 --> 00:23:31.130 align:middle line:90% organizations. 00:23:31.130 --> 00:23:34.410 align:middle line:84% And they just differ in where those receptive fields 00:23:34.410 --> 00:23:35.490 align:middle line:90% are located. 00:23:35.490 --> 00:23:38.650 align:middle line:84% So the idea is that the image is being analyzed 00:23:38.650 --> 00:23:40.832 align:middle line:84% by this kind of huge set of neurons 00:23:40.832 --> 00:23:42.290 align:middle line:84% that all have receptive fields that 00:23:42.290 --> 00:23:45.690 align:middle line:84% look like this, that are just at different positions 00:23:45.690 --> 00:23:47.130 align:middle line:90% within the image. 00:23:47.130 --> 00:23:52.890 align:middle line:84% And so one way to capture that population, or to simulate it, 00:23:52.890 --> 00:23:59.290 align:middle line:84% is by taking a particular filter or receptive field type 00:23:59.290 --> 00:24:03.170 align:middle line:84% and convolving it with the image. 00:24:03.170 --> 00:24:04.630 align:middle line:90% So this is what this looks like. 00:24:04.630 --> 00:24:07.790 align:middle line:84% So here's an example grayscale image. 00:24:07.790 --> 00:24:09.430 align:middle line:90% So it's just a black cross. 00:24:09.430 --> 00:24:10.930 align:middle line:84% These are the image intensities that 00:24:10.930 --> 00:24:12.730 align:middle line:90% would correspond to that image. 00:24:12.730 --> 00:24:15.650 align:middle line:84% So each one of these things is like a pixel. 00:24:15.650 --> 00:24:17.390 align:middle line:84% And these are two different filters. 00:24:17.390 --> 00:24:19.050 align:middle line:90% They're really simple filters. 00:24:19.050 --> 00:24:22.330 align:middle line:90% So they're two pixels wide. 00:24:22.330 --> 00:24:26.390 align:middle line:84% One of them is minus 1, plus 1 in this direction. 00:24:26.390 --> 00:24:28.810 align:middle line:84% And this one is plus 1, minus 1 this direction. 00:24:28.810 --> 00:24:33.270 align:middle line:84% And so what is shown at the bottom here 00:24:33.270 --> 00:24:38.790 align:middle line:84% is the convolution of this image with each of these filters. 00:24:38.790 --> 00:24:41.155 align:middle line:84% And so the operation that's being performed here 00:24:41.155 --> 00:24:42.530 align:middle line:84% is the same one that we just saw. 00:24:42.530 --> 00:24:45.230 align:middle line:90% So it's a dot product. 00:24:45.230 --> 00:24:47.030 align:middle line:84% But it's being performed with the filter 00:24:47.030 --> 00:24:49.430 align:middle line:84% at every possible location within the image. 00:24:49.430 --> 00:24:52.430 align:middle line:84% So let's just do a couple of examples to make sure we get it. 00:24:52.430 --> 00:24:55.250 align:middle line:84% So let's start out with this filter here. 00:24:55.250 --> 00:24:57.530 align:middle line:90% We've got minus 1, plus 1. 00:24:57.530 --> 00:25:02.150 align:middle line:84% So when that one is put in the top left corner of the image, 00:25:02.150 --> 00:25:06.630 align:middle line:84% we've got the pixel value 10 occurring here 00:25:06.630 --> 00:25:09.030 align:middle line:90% and 10 occurring here. 00:25:09.030 --> 00:25:13.230 align:middle line:84% So we multiply minus 1 times 10 and plus 1 times 10. 00:25:13.230 --> 00:25:15.050 align:middle line:84% That gives us minus 10 and plus 10. 00:25:15.050 --> 00:25:16.210 align:middle line:90% And we add those together. 00:25:16.210 --> 00:25:18.570 align:middle line:90% And that gives us 0. 00:25:18.570 --> 00:25:20.670 align:middle line:84% And that's the response that we get here. 00:25:20.670 --> 00:25:23.670 align:middle line:84% So if we just shift it over 1 pixel, 00:25:23.670 --> 00:25:30.730 align:middle line:84% so now, we have a intensity value of 10 here and 2 here. 00:25:30.730 --> 00:25:35.010 align:middle line:84% So that gives us minus 1 times 10, which is minus 10, 00:25:35.010 --> 00:25:37.370 align:middle line:90% and plus 1 times 2, which is 2. 00:25:37.370 --> 00:25:41.098 align:middle line:90% Minus 10 plus 2 is minus 8. 00:25:41.098 --> 00:25:42.390 align:middle line:90% And then we move it over again. 00:25:42.390 --> 00:25:43.550 align:middle line:90% And now, we have 2 and 2. 00:25:43.550 --> 00:25:45.410 align:middle line:84% And so those cancel out, and you get 0. 00:25:45.410 --> 00:25:46.710 align:middle line:90% You move it over again. 00:25:46.710 --> 00:25:48.030 align:middle line:90% Now, you get 2 and 10. 00:25:48.030 --> 00:25:51.150 align:middle line:84% And that gives you plus 8 and so on and so forth. 00:25:51.150 --> 00:25:53.770 align:middle line:84% So you evaluate the filter at every different position 00:25:53.770 --> 00:25:54.630 align:middle line:90% within the signal. 00:25:54.630 --> 00:25:56.845 align:middle line:84% So that's the convolution operation. 00:25:56.845 --> 00:25:58.470 align:middle line:84% That's just the mathematical operation. 00:25:58.470 --> 00:25:59.830 align:middle line:90% And that's the definition. 00:25:59.830 --> 00:26:03.530 align:middle line:84% And you can do the same thing with this particular edge 00:26:03.530 --> 00:26:04.030 align:middle line:90% operator. 00:26:04.030 --> 00:26:05.910 align:middle line:84% And it works exactly the same way. 00:26:05.910 --> 00:26:08.250 align:middle line:84% So you get a 0 here because you have 10 and 10. 00:26:08.250 --> 00:26:10.345 align:middle line:84% And then here you get plus 8 because you 00:26:10.345 --> 00:26:12.970 align:middle line:84% have the 10 corresponding to the plus 1 and the 2 corresponding 00:26:12.970 --> 00:26:16.070 align:middle line:84% to the minus 1, and so on and so forth. 00:26:16.070 --> 00:26:18.070 align:middle line:84% So that's a mathematical operation. 00:26:18.070 --> 00:26:20.130 align:middle line:84% It comes up all the time in signal processing. 00:26:20.130 --> 00:26:23.550 align:middle line:84% In the context of thinking about the visual system, as I said, 00:26:23.550 --> 00:26:28.710 align:middle line:84% we use convolution to simulate a whole bunch of neurons 00:26:28.710 --> 00:26:32.410 align:middle line:84% with receptive fields that have the same form 00:26:32.410 --> 00:26:34.670 align:middle line:84% but that are just located at different positions, 00:26:34.670 --> 00:26:37.770 align:middle line:90% as you would have in the retina. 00:26:37.770 --> 00:26:40.070 align:middle line:84% Here's a movie that will give you 00:26:40.070 --> 00:26:42.330 align:middle line:90% another little example of this. 00:26:42.330 --> 00:26:44.910 align:middle line:90% 00:26:44.910 --> 00:26:47.970 align:middle line:84% So what you're going to see here-- so this is a filter. 00:26:47.970 --> 00:26:50.550 align:middle line:84% This is a very simple filter, where the filter 00:26:50.550 --> 00:26:51.770 align:middle line:90% kind of resembles a Gaussian. 00:26:51.770 --> 00:26:55.230 align:middle line:84% So it's essentially taking a local average. 00:26:55.230 --> 00:26:57.830 align:middle line:90% Here's the signal. 00:26:57.830 --> 00:27:01.110 align:middle line:84% And this is the result of the convolution. 00:27:01.110 --> 00:27:07.910 align:middle line:84% And this is a movie that will go like this. 00:27:07.910 --> 00:27:11.690 align:middle line:84% So there's one other little detail here. 00:27:11.690 --> 00:27:15.190 align:middle line:84% So this differs from the example on the previous slide 00:27:15.190 --> 00:27:17.510 align:middle line:84% in that the convolution is actually 00:27:17.510 --> 00:27:20.990 align:middle line:84% being evaluated for positions of the filter 00:27:20.990 --> 00:27:24.810 align:middle line:84% where it doesn't completely overlap with the signal. 00:27:24.810 --> 00:27:28.010 align:middle line:84% And so this is a choice that you make anytime 00:27:28.010 --> 00:27:29.590 align:middle line:84% you want to perform a convolution. 00:27:29.590 --> 00:27:31.090 align:middle line:84% And in different contexts, you might 00:27:31.090 --> 00:27:32.470 align:middle line:90% want to do different things. 00:27:32.470 --> 00:27:37.610 align:middle line:84% So the thing that we were kind of seeing started out like here 00:27:37.610 --> 00:27:40.770 align:middle line:84% and then moved all the way to over here. 00:27:40.770 --> 00:27:43.770 align:middle line:84% And so what happens is that at every position, 00:27:43.770 --> 00:27:46.490 align:middle line:84% the result of applying the filter 00:27:46.490 --> 00:27:49.497 align:middle line:90% is this little moving average. 00:27:49.497 --> 00:27:51.330 align:middle line:84% And so down here, it's negative because most 00:27:51.330 --> 00:27:55.950 align:middle line:84% of the values in the signal are negative and so on and so forth, 00:27:55.950 --> 00:27:56.850 align:middle line:90% so that's the idea. 00:27:56.850 --> 00:27:59.530 align:middle line:90% 00:27:59.530 --> 00:28:03.390 align:middle line:84% So what do we do with all of this? 00:28:03.390 --> 00:28:09.410 align:middle line:84% So one place where this was historically 00:28:09.410 --> 00:28:12.170 align:middle line:84% kind of influential in how people thought about things 00:28:12.170 --> 00:28:17.210 align:middle line:84% was in its application to certain illusions. 00:28:17.210 --> 00:28:22.870 align:middle line:84% So this is a very famous illusion, known as Mach bands, 00:28:22.870 --> 00:28:25.630 align:middle line:90% named after a person named Mach. 00:28:25.630 --> 00:28:28.530 align:middle line:84% So the image here is of this form. 00:28:28.530 --> 00:28:30.270 align:middle line:84% So this is a plot of the image intensity 00:28:30.270 --> 00:28:32.350 align:middle line:90% as a function of position. 00:28:32.350 --> 00:28:34.970 align:middle line:84% And so it starts out at some low intensity. 00:28:34.970 --> 00:28:37.950 align:middle line:84% And then there's a ramp up to some high intensity. 00:28:37.950 --> 00:28:41.270 align:middle line:84% Now, the illusion is that people will typically 00:28:41.270 --> 00:28:47.270 align:middle line:84% report seeing bands, a dark band here and a bright band there. 00:28:47.270 --> 00:28:50.270 align:middle line:84% So for me, the bright band is a lot more salient 00:28:50.270 --> 00:28:53.850 align:middle line:84% than the dark band, but I can see them both. 00:28:53.850 --> 00:28:56.830 align:middle line:84% It's an illusion because there's no actual band in the image. 00:28:56.830 --> 00:28:59.750 align:middle line:84% So the intensity just kind of starts out low and then 00:28:59.750 --> 00:29:03.870 align:middle line:84% increases and then kind of levels off. 00:29:03.870 --> 00:29:06.590 align:middle line:84% So that's just this old illusion. 00:29:06.590 --> 00:29:11.430 align:middle line:84% And people got interested in the fact 00:29:11.430 --> 00:29:16.070 align:middle line:84% that if you take one of these center-surround receptive fields 00:29:16.070 --> 00:29:20.110 align:middle line:84% and convolve it with the image, you 00:29:20.110 --> 00:29:23.610 align:middle line:84% get out a response profile that actually kind of resembles 00:29:23.610 --> 00:29:25.190 align:middle line:90% what you see. 00:29:25.190 --> 00:29:28.970 align:middle line:84% So here's the stimulus, the image. 00:29:28.970 --> 00:29:32.010 align:middle line:84% Here's a one-dimensional version of the center-surround receptive 00:29:32.010 --> 00:29:32.670 align:middle line:90% field. 00:29:32.670 --> 00:29:34.530 align:middle line:84% And the idea here is that when you're 00:29:34.530 --> 00:29:37.170 align:middle line:84% evaluating the response of the filter 00:29:37.170 --> 00:29:41.210 align:middle line:84% at the image, when you get over here, what happens 00:29:41.210 --> 00:29:44.850 align:middle line:84% is that the inhibitory lobe of the receptive field kind of 00:29:44.850 --> 00:29:47.610 align:middle line:84% hits the ramping part of the stimulus. 00:29:47.610 --> 00:29:52.407 align:middle line:84% And so that causes the response to go down a little bit. 00:29:52.407 --> 00:29:54.990 align:middle line:84% And then you get to the ramp, and the thing gradually goes up. 00:29:54.990 --> 00:29:57.090 align:middle line:84% And then you get up here to the top, 00:29:57.090 --> 00:30:00.810 align:middle line:84% and it's the same thing, where you end up 00:30:00.810 --> 00:30:03.010 align:middle line:84% with the inhibitory portion of the receptive field 00:30:03.010 --> 00:30:04.470 align:middle line:90% kind of in the ramp. 00:30:04.470 --> 00:30:06.270 align:middle line:84% And so it's lower than this part. 00:30:06.270 --> 00:30:08.890 align:middle line:84% And that creates an increase in the response. 00:30:08.890 --> 00:30:11.550 align:middle line:90% 00:30:11.550 --> 00:30:15.890 align:middle line:84% So people kind of observed this and thought, well, maybe this 00:30:15.890 --> 00:30:20.190 align:middle line:84% somehow explains the fact that we see these bands when 00:30:20.190 --> 00:30:24.550 align:middle line:90% we look at this image. 00:30:24.550 --> 00:30:26.510 align:middle line:90% Here's a similar thing. 00:30:26.510 --> 00:30:30.110 align:middle line:84% There's lots of illusions that are like this one, 00:30:30.110 --> 00:30:34.143 align:middle line:84% where-- and the illusion here is that you probably, 00:30:34.143 --> 00:30:36.310 align:middle line:84% when you look at this thing, you have the impression 00:30:36.310 --> 00:30:39.590 align:middle line:84% that there are these dark blobs at the intersections 00:30:39.590 --> 00:30:43.230 align:middle line:90% of the white lines. 00:30:43.230 --> 00:30:45.470 align:middle line:90% Everybody see those dark blobs? 00:30:45.470 --> 00:30:47.590 align:middle line:84% You probably don't see them right 00:30:47.590 --> 00:30:49.270 align:middle line:84% at the center of gaze but then kind 00:30:49.270 --> 00:30:50.770 align:middle line:90% of at all the other locations. 00:30:50.770 --> 00:30:53.450 align:middle line:90% 00:30:53.450 --> 00:30:58.910 align:middle line:84% And so the there's this analogous type of explanation 00:30:58.910 --> 00:31:02.950 align:middle line:84% here, which is that if you think about what 00:31:02.950 --> 00:31:04.910 align:middle line:84% the response of a center-surround filter 00:31:04.910 --> 00:31:08.990 align:middle line:84% would be if you convolved it with this image, 00:31:08.990 --> 00:31:15.190 align:middle line:84% the response of an on-center, off-surround receptive field 00:31:15.190 --> 00:31:18.450 align:middle line:84% would be reduced at one of these intersections, 00:31:18.450 --> 00:31:21.770 align:middle line:84% compared to if you're off of one of the intersections. 00:31:21.770 --> 00:31:24.810 align:middle line:84% And so the idea is that when this receptive field is centered 00:31:24.810 --> 00:31:28.930 align:middle line:84% here, you get more white stuff in the surround 00:31:28.930 --> 00:31:31.030 align:middle line:90% than when it's centered here. 00:31:31.030 --> 00:31:34.810 align:middle line:84% And because this is an inhibitory surround, when there 00:31:34.810 --> 00:31:37.930 align:middle line:84% is a high intensity inhibitory surround, 00:31:37.930 --> 00:31:39.710 align:middle line:90% that produces a lower response. 00:31:39.710 --> 00:31:43.490 align:middle line:90% 00:31:43.490 --> 00:31:47.530 align:middle line:84% So there's less dark stuff in the surround 00:31:47.530 --> 00:31:49.550 align:middle line:84% when the receptive field is on an intersection. 00:31:49.550 --> 00:31:52.530 align:middle line:84% So there's a smaller response to the filter. 00:31:52.530 --> 00:31:55.770 align:middle line:84% We've observed that you see a dark spot. 00:31:55.770 --> 00:31:59.810 align:middle line:84% And the hypothesis is that the smaller response of the filter 00:31:59.810 --> 00:32:02.530 align:middle line:84% could explain why you see a dark spot. 00:32:02.530 --> 00:32:03.470 align:middle line:90% Yes? 00:32:03.470 --> 00:32:05.250 align:middle line:84% AUDIENCE: Can you explain why you 00:32:05.250 --> 00:32:08.940 align:middle line:84% don't see dark spot on the one that you're focusing on? 00:32:08.940 --> 00:32:10.690 align:middle line:84% JOSH MCDERMOTT: Does anybody have an idea? 00:32:10.690 --> 00:32:13.250 align:middle line:90% 00:32:13.250 --> 00:32:17.830 align:middle line:84% What do we know to be different about the fovea compared 00:32:17.830 --> 00:32:19.270 align:middle line:90% to the periphery? 00:32:19.270 --> 00:32:20.390 align:middle line:90% Yeah. 00:32:20.390 --> 00:32:23.590 align:middle line:84% AUDIENCE: High density of receptors 00:32:23.590 --> 00:32:25.830 align:middle line:90% and small receptive fields. 00:32:25.830 --> 00:32:27.750 align:middle line:84% JOSH MCDERMOTT: So one possibility 00:32:27.750 --> 00:32:30.558 align:middle line:84% is that at the fovea, the receptive field is so small 00:32:30.558 --> 00:32:31.850 align:middle line:90% that this stuff doesn't happen. 00:32:31.850 --> 00:32:33.290 align:middle line:90% That's at least possible. 00:32:33.290 --> 00:32:36.070 align:middle line:84% But that would be one kind of explanation. 00:32:36.070 --> 00:32:41.830 align:middle line:84% And can anybody explain why would this spot be dark 00:32:41.830 --> 00:32:43.210 align:middle line:90% as opposed to light? 00:32:43.210 --> 00:32:51.550 align:middle line:90% 00:32:51.550 --> 00:32:53.150 align:middle line:90% Yeah. 00:32:53.150 --> 00:32:56.950 align:middle line:84% AUDIENCE: Because the surrounding squares are dark. 00:32:56.950 --> 00:33:00.430 align:middle line:84% If you reversed their colors, would you see spots 00:33:00.430 --> 00:33:03.335 align:middle line:90% or would you see even darker-- 00:33:03.335 --> 00:33:04.210 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:33:04.210 --> 00:33:06.335 align:middle line:84% So if you reverse the colors, it would be reversed. 00:33:06.335 --> 00:33:08.270 align:middle line:90% But I was actually asking-- 00:33:08.270 --> 00:33:10.790 align:middle line:84% I was looking for an answer in terms 00:33:10.790 --> 00:33:13.670 align:middle line:84% of the type of receptive field that we're 00:33:13.670 --> 00:33:16.370 align:middle line:90% trying to explain this with. 00:33:16.370 --> 00:33:19.570 align:middle line:84% And so specifically, what I was going for here 00:33:19.570 --> 00:33:23.190 align:middle line:84% is this is an on-center receptive field. 00:33:23.190 --> 00:33:26.770 align:middle line:84% And so normally, the response of that cell kind of signals 00:33:26.770 --> 00:33:29.273 align:middle line:90% increments in intensity. 00:33:29.273 --> 00:33:30.690 align:middle line:84% And so the idea is that if there's 00:33:30.690 --> 00:33:35.570 align:middle line:84% a smaller response from that cell, that would correspond 00:33:35.570 --> 00:33:38.330 align:middle line:90% to a decrease in intensity. 00:33:38.330 --> 00:33:42.070 align:middle line:84% So that was the underlying logic I was looking for. 00:33:42.070 --> 00:33:44.730 align:middle line:84% But there's something-- so this is like the classic explanation 00:33:44.730 --> 00:33:45.230 align:middle line:90% of stuff. 00:33:45.230 --> 00:33:46.390 align:middle line:90% It's in all the textbooks. 00:33:46.390 --> 00:33:51.930 align:middle line:84% There's something deeply funny about these explanations. 00:33:51.930 --> 00:33:56.810 align:middle line:84% And that is that, essentially, they're 00:33:56.810 --> 00:34:00.210 align:middle line:90% implicitly assuming that-- 00:34:00.210 --> 00:34:04.010 align:middle line:84% it's like there's this notion that your visual system gets 00:34:04.010 --> 00:34:06.370 align:middle line:84% the output of these filters and then somehow 00:34:06.370 --> 00:34:08.230 align:middle line:84% doesn't know where they came from. 00:34:08.230 --> 00:34:09.909 align:middle line:84% So you get the outputs of these filters. 00:34:09.909 --> 00:34:14.429 align:middle line:84% And there's a lower response at one point compared to another. 00:34:14.429 --> 00:34:18.190 align:middle line:84% But if in principle, whatever's happening 00:34:18.190 --> 00:34:21.675 align:middle line:84% downstream knows that it's dealing 00:34:21.675 --> 00:34:23.550 align:middle line:84% with the outputs of center-surround receptive 00:34:23.550 --> 00:34:25.750 align:middle line:84% fields, you might imagine that it 00:34:25.750 --> 00:34:30.350 align:middle line:84% would be able to decode that and not incorrectly think 00:34:30.350 --> 00:34:33.050 align:middle line:84% that there's a dark blob at the center of these intersections. 00:34:33.050 --> 00:34:37.310 align:middle line:84% So essentially, these explanations implicitly 00:34:37.310 --> 00:34:39.929 align:middle line:84% assume suboptimal decoding of the responses, 00:34:39.929 --> 00:34:43.190 align:middle line:90% which is kind of weird. 00:34:43.190 --> 00:34:47.230 align:middle line:84% And so that's the classical explanation. 00:34:47.230 --> 00:34:49.949 align:middle line:90% And it's worth knowing about. 00:34:49.949 --> 00:34:53.550 align:middle line:84% I think we actually don't really understand 00:34:53.550 --> 00:34:56.110 align:middle line:90% why these illusions exist. 00:34:56.110 --> 00:34:59.590 align:middle line:84% My intuition is that it has to do with the fact 00:34:59.590 --> 00:35:01.650 align:middle line:84% that these are very weird images. 00:35:01.650 --> 00:35:03.150 align:middle line:84% They're very unnatural in the sense 00:35:03.150 --> 00:35:06.790 align:middle line:84% that they have these very high contrast edges arranged 00:35:06.790 --> 00:35:08.510 align:middle line:90% in particular ways. 00:35:08.510 --> 00:35:12.470 align:middle line:84% And in general, there's lots of funny illusions 00:35:12.470 --> 00:35:15.170 align:middle line:84% that result from images like this, where 00:35:15.170 --> 00:35:17.090 align:middle line:84% you have lots of high contrast edges 00:35:17.090 --> 00:35:19.570 align:middle line:84% and sharp corners and things like that. 00:35:19.570 --> 00:35:23.530 align:middle line:84% So my suspicion is that it's the kind of image 00:35:23.530 --> 00:35:27.190 align:middle line:84% that pushes the system into a funny operating point. 00:35:27.190 --> 00:35:30.730 align:middle line:84% And that really is critical to actually underlying this. 00:35:30.730 --> 00:35:32.730 align:middle line:84% But really, I would say we actually don't really 00:35:32.730 --> 00:35:36.410 align:middle line:84% have a very good deep theoretical understanding of why 00:35:36.410 --> 00:35:40.610 align:middle line:90% these kinds of illusions exist. 00:35:40.610 --> 00:35:43.050 align:middle line:90% Any questions about that? 00:35:43.050 --> 00:35:48.250 align:middle line:84% AUDIENCE: So basically how do you make the image not appear? 00:35:48.250 --> 00:35:52.170 align:middle line:84% It's not like using your perspective, 00:35:52.170 --> 00:35:55.733 align:middle line:84% if you, say, zoomed in four times? 00:35:55.733 --> 00:35:57.150 align:middle line:84% JOSH MCDERMOTT: You should try it. 00:35:57.150 --> 00:35:59.383 align:middle line:90% That's a great illusion lab. 00:35:59.383 --> 00:36:01.050 align:middle line:84% I can tell you from having played around 00:36:01.050 --> 00:36:04.050 align:middle line:84% with these things in my youth, there 00:36:04.050 --> 00:36:07.250 align:middle line:84% will be effects that relate to that. 00:36:07.250 --> 00:36:10.370 align:middle line:84% I don't know how easy it would be to relate these 00:36:10.370 --> 00:36:12.270 align:middle line:84% to actual receptive field sizes that you 00:36:12.270 --> 00:36:14.437 align:middle line:84% would find at different stages of the visual system. 00:36:14.437 --> 00:36:16.590 align:middle line:84% That would be interesting to try. 00:36:16.590 --> 00:36:21.570 align:middle line:84% But yeah, it's certainly a very powerful effect 00:36:21.570 --> 00:36:23.070 align:middle line:84% that it doesn't happen at the fovea, 00:36:23.070 --> 00:36:26.350 align:middle line:90% and it happens in the periphery. 00:36:26.350 --> 00:36:28.250 align:middle line:84% So yeah, you should play with it. 00:36:28.250 --> 00:36:35.150 align:middle line:90% 00:36:35.150 --> 00:36:36.910 align:middle line:84% Explaining illusions with receptive fields 00:36:36.910 --> 00:36:38.810 align:middle line:90% implicitly involves convolution. 00:36:38.810 --> 00:36:41.790 align:middle line:84% So again, the idea is that the convolution kind of 00:36:41.790 --> 00:36:45.693 align:middle line:84% simulates the response of this big set of neurons 00:36:45.693 --> 00:36:47.610 align:middle line:84% that all has kind of similar receptive fields. 00:36:47.610 --> 00:36:49.830 align:middle line:84% We think that that's effectively what 00:36:49.830 --> 00:36:53.690 align:middle line:84% your retina is sending to the rest of the visual system. 00:36:53.690 --> 00:36:56.590 align:middle line:90% 00:36:56.590 --> 00:37:00.510 align:middle line:84% Another really important thing to know about the visual system 00:37:00.510 --> 00:37:06.070 align:middle line:84% is the way that it's organized with respect to space. 00:37:06.070 --> 00:37:11.170 align:middle line:84% And so this is a diagram that shows the visual system looking 00:37:11.170 --> 00:37:13.730 align:middle line:90% at a screen, essentially. 00:37:13.730 --> 00:37:16.730 align:middle line:84% So the idea is that you're fixating that cross. 00:37:16.730 --> 00:37:21.570 align:middle line:84% And there's a left hemifield and a right hemifield 00:37:21.570 --> 00:37:22.810 align:middle line:90% that are visible. 00:37:22.810 --> 00:37:25.450 align:middle line:84% So the colored part of the screen 00:37:25.450 --> 00:37:27.290 align:middle line:84% is like the extent of visual space 00:37:27.290 --> 00:37:30.610 align:middle line:90% that is visible to your eyes. 00:37:30.610 --> 00:37:34.360 align:middle line:84% And it's colored red when it's on the left and blue when 00:37:34.360 --> 00:37:35.110 align:middle line:90% it's on the right. 00:37:35.110 --> 00:37:38.810 align:middle line:84% And then this shows you where those points in the image end 00:37:38.810 --> 00:37:40.730 align:middle line:84% up being represented at different stages 00:37:40.730 --> 00:37:42.450 align:middle line:90% of the visual system. 00:37:42.450 --> 00:37:44.510 align:middle line:84% And so what do we take away from this? 00:37:44.510 --> 00:37:49.550 align:middle line:84% Well, first of all, both eyes see both hemifields 00:37:49.550 --> 00:37:51.130 align:middle line:90% but to different degrees. 00:37:51.130 --> 00:37:52.250 align:middle line:90% And you can just-- 00:37:52.250 --> 00:37:55.910 align:middle line:84% if you fixate here, you can verify this yourself. 00:37:55.910 --> 00:37:59.490 align:middle line:84% But then something crazy happens. 00:37:59.490 --> 00:38:02.890 align:middle line:84% So you get the optic nerve exiting the eye, and then 00:38:02.890 --> 00:38:05.490 align:middle line:84% the optic nerve fibers kind of split up. 00:38:05.490 --> 00:38:08.970 align:middle line:84% So the fibers that correspond to the left 00:38:08.970 --> 00:38:12.790 align:middle line:84% hemifield all go to the right side of the brain, 00:38:12.790 --> 00:38:14.870 align:middle line:84% irrespective of which eye they come from. 00:38:14.870 --> 00:38:17.550 align:middle line:84% And then the fibers that correspond to the right 00:38:17.550 --> 00:38:22.150 align:middle line:84% hemifield all go to the left side of the brain. 00:38:22.150 --> 00:38:25.330 align:middle line:84% And so the there's a right LGN and a left LGN. 00:38:25.330 --> 00:38:27.230 align:middle line:90% That's the part of the thalamus. 00:38:27.230 --> 00:38:29.910 align:middle line:84% And so those are segregated according to which side of space 00:38:29.910 --> 00:38:32.310 align:middle line:90% we're dealing with. 00:38:32.310 --> 00:38:35.830 align:middle line:84% And then those project to the left and right 00:38:35.830 --> 00:38:40.250 align:middle line:84% primary visual cortex, so really kind of striking segregation. 00:38:40.250 --> 00:38:44.870 align:middle line:84% So what this means is that once you leave the eye, 00:38:44.870 --> 00:38:47.790 align:middle line:84% stuff that's happening on the left side of space 00:38:47.790 --> 00:38:49.870 align:middle line:84% is represented in a different bit of the brain 00:38:49.870 --> 00:38:51.245 align:middle line:84% than the stuff that's represented 00:38:51.245 --> 00:38:54.510 align:middle line:90% in the right side of space. 00:38:54.510 --> 00:38:57.210 align:middle line:84% And of course, this is all relative to the fovea. 00:38:57.210 --> 00:39:00.790 align:middle line:84% So you move your eyes, and what counts as left and right is 00:39:00.790 --> 00:39:01.650 align:middle line:90% going to change. 00:39:01.650 --> 00:39:07.770 align:middle line:90% 00:39:07.770 --> 00:39:11.250 align:middle line:84% So information from each hemifield projects 00:39:11.250 --> 00:39:14.530 align:middle line:90% to the contralateral LGN. 00:39:14.530 --> 00:39:16.970 align:middle line:84% But the information from the two eyes 00:39:16.970 --> 00:39:19.470 align:middle line:84% stays separated until you get to the cortex. 00:39:19.470 --> 00:39:22.130 align:middle line:84% So there are different layers of the LGN 00:39:22.130 --> 00:39:26.810 align:middle line:84% that correspond to the left eye and to the right eye. 00:39:26.810 --> 00:39:29.390 align:middle line:84% And there's an example of that that's shown here. 00:39:29.390 --> 00:39:33.370 align:middle line:84% This is an actual picture of the LGN, 00:39:33.370 --> 00:39:35.550 align:middle line:90% so this little bit of brain. 00:39:35.550 --> 00:39:39.450 align:middle line:84% And you can see that this is a microscope image, where the cell 00:39:39.450 --> 00:39:41.290 align:middle line:90% bodies have been stained. 00:39:41.290 --> 00:39:43.790 align:middle line:84% And you can see that there are these different layers. 00:39:43.790 --> 00:39:46.690 align:middle line:90% There are six layers of the LGN. 00:39:46.690 --> 00:39:51.610 align:middle line:84% The four outer layers are called parvocellular layers. 00:39:51.610 --> 00:39:53.350 align:middle line:84% And as we talked about last time, 00:39:53.350 --> 00:39:55.210 align:middle line:84% they receive input from the midget ganglion 00:39:55.210 --> 00:39:56.650 align:middle line:90% cells in the retina. 00:39:56.650 --> 00:40:01.090 align:middle line:84% And then the two deeper layers are the magnocellular layers. 00:40:01.090 --> 00:40:03.830 align:middle line:84% And they receive input from the parasol ganglion cells. 00:40:03.830 --> 00:40:05.288 align:middle line:84% So there's, again, this really kind 00:40:05.288 --> 00:40:07.670 align:middle line:84% of precise beautiful anatomical organization 00:40:07.670 --> 00:40:10.150 align:middle line:84% where you have these two distinct types of cells 00:40:10.150 --> 00:40:14.010 align:middle line:84% in the retina, midget and parasol, both of which 00:40:14.010 --> 00:40:17.390 align:middle line:84% come in the on-center and off-center receptive field 00:40:17.390 --> 00:40:18.670 align:middle line:90% varieties. 00:40:18.670 --> 00:40:23.110 align:middle line:84% And then they project to these distinct regions of the LGN, 00:40:23.110 --> 00:40:25.870 align:middle line:84% where different layers would get input 00:40:25.870 --> 00:40:28.750 align:middle line:84% from either the left eye or the right eye. 00:40:28.750 --> 00:40:32.550 align:middle line:90% So the eyes remain segregated. 00:40:32.550 --> 00:40:36.910 align:middle line:84% So we've got these two types of cells in this way station 00:40:36.910 --> 00:40:38.550 align:middle line:90% in the visual system. 00:40:38.550 --> 00:40:39.710 align:middle line:90% What's the difference? 00:40:39.710 --> 00:40:43.910 align:middle line:84% So parvocellular cells are notably color opponent. 00:40:43.910 --> 00:40:45.790 align:middle line:84% So that means that they receive inputs 00:40:45.790 --> 00:40:47.950 align:middle line:84% from more than one class of cones 00:40:47.950 --> 00:40:49.550 align:middle line:84% and are typically kind of computing 00:40:49.550 --> 00:40:53.330 align:middle line:84% the difference between those different cone types. 00:40:53.330 --> 00:40:56.428 align:middle line:84% So we have red-green opponency and blue-yellow opponency. 00:40:56.428 --> 00:40:58.470 align:middle line:84% And again, we'll go into that in more detail when 00:40:58.470 --> 00:41:01.190 align:middle line:84% we talk about color, so I'm not going to dwell on it. 00:41:01.190 --> 00:41:04.570 align:middle line:84% But they also give sustained responses to stimuli. 00:41:04.570 --> 00:41:07.870 align:middle line:84% So they tend to respond over extended periods of time, 00:41:07.870 --> 00:41:10.630 align:middle line:84% whereas magnocellular cells, by contrast, are achromatic, 00:41:10.630 --> 00:41:13.530 align:middle line:84% so they tend to not have information about wavelength, 00:41:13.530 --> 00:41:16.930 align:middle line:84% and they give transient responses to inputs. 00:41:16.930 --> 00:41:21.330 align:middle line:84% So these were physiological differences that were observed. 00:41:21.330 --> 00:41:24.988 align:middle line:84% And several decades ago, here in this department-- 00:41:24.988 --> 00:41:27.530 align:middle line:84% you can see this, Department of Brain and Cognitive Sciences, 00:41:27.530 --> 00:41:30.890 align:middle line:84% Massachusetts Institute of Technology-- 00:41:30.890 --> 00:41:33.750 align:middle line:84% this piece of research was done by Peter Schiller. 00:41:33.750 --> 00:41:36.610 align:middle line:84% So Peter is an emeritus professor. 00:41:36.610 --> 00:41:40.770 align:middle line:84% And I did a UROP with him a long time ago, 00:41:40.770 --> 00:41:42.090 align:middle line:90% which was a lot of fun. 00:41:42.090 --> 00:41:44.230 align:middle line:84% So he was a pioneer in the visual system. 00:41:44.230 --> 00:41:46.210 align:middle line:84% And the middle author on this paper 00:41:46.210 --> 00:41:50.290 align:middle line:84% is Nikos Logothetis, who's also now a very famous 00:41:50.290 --> 00:41:51.230 align:middle line:90% neuroscientist. 00:41:51.230 --> 00:41:53.150 align:middle line:84% There's lots of stuff in vision and MRI. 00:41:53.150 --> 00:41:56.548 align:middle line:90% And he's now working in Germany. 00:41:56.548 --> 00:41:57.590 align:middle line:90% So what did they do here? 00:41:57.590 --> 00:42:01.370 align:middle line:84% So this paper used a method that's 00:42:01.370 --> 00:42:03.390 align:middle line:84% no longer really in very widespread use 00:42:03.390 --> 00:42:09.790 align:middle line:84% because we now have more precise ways of doing similar things. 00:42:09.790 --> 00:42:13.670 align:middle line:84% But in this era, it was very common to actually investigate 00:42:13.670 --> 00:42:17.630 align:middle line:84% brain function by lesioning parts of the brain. 00:42:17.630 --> 00:42:20.030 align:middle line:84% So that means typically sometimes you go in, 00:42:20.030 --> 00:42:22.150 align:middle line:84% and you cut out parts of the brain. 00:42:22.150 --> 00:42:24.510 align:middle line:84% Sometimes it means that you would inject acid 00:42:24.510 --> 00:42:27.430 align:middle line:84% to kill the cell bodies in a part of the brain. 00:42:27.430 --> 00:42:29.892 align:middle line:84% And then one could look at the effect on behavior. 00:42:29.892 --> 00:42:31.350 align:middle line:84% And this was one way that you could 00:42:31.350 --> 00:42:35.670 align:middle line:84% infer the role of different parts of the brain in function. 00:42:35.670 --> 00:42:40.830 align:middle line:84% And so in this particular paper, they 00:42:40.830 --> 00:42:45.510 align:middle line:90% lesioned regions of the LGN. 00:42:45.510 --> 00:42:47.490 align:middle line:84% So remember, this is the structure of the LGN. 00:42:47.490 --> 00:42:51.670 align:middle line:84% We got these six layers 1, 2, 3, 4, 5, 6. 00:42:51.670 --> 00:42:54.830 align:middle line:90% The top four are parvocellular. 00:42:54.830 --> 00:42:56.950 align:middle line:84% The bottom two are magnocellular. 00:42:56.950 --> 00:43:00.110 align:middle line:90% So they went in and injected-- 00:43:00.110 --> 00:43:05.270 align:middle line:84% I think this was ibotenic acid into particular regions. 00:43:05.270 --> 00:43:08.790 align:middle line:84% And this is one of the LGNs in one hemisphere. 00:43:08.790 --> 00:43:11.610 align:middle line:84% And this is the other in the other one. 00:43:11.610 --> 00:43:14.250 align:middle line:84% So they injected a ibotenic acid into a particular part 00:43:14.250 --> 00:43:16.490 align:middle line:84% of the parvocellular layers in one hemisphere 00:43:16.490 --> 00:43:18.810 align:middle line:84% and a particular part of the magnocellular layers 00:43:18.810 --> 00:43:20.850 align:middle line:90% in the other hemisphere. 00:43:20.850 --> 00:43:23.210 align:middle line:84% Before they did that, they measured the receptive field 00:43:23.210 --> 00:43:24.570 align:middle line:90% locations. 00:43:24.570 --> 00:43:26.370 align:middle line:84% And this is the receptive field locations 00:43:26.370 --> 00:43:27.750 align:middle line:90% plotted in visual space. 00:43:27.750 --> 00:43:29.730 align:middle line:90% So this is fixation. 00:43:29.730 --> 00:43:33.250 align:middle line:84% And so the region of space in which there was a parvocellular 00:43:33.250 --> 00:43:33.987 align:middle line:90% region is here. 00:43:33.987 --> 00:43:36.570 align:middle line:84% And the region of space where there was a magnocellular lesion 00:43:36.570 --> 00:43:37.610 align:middle line:90% is here. 00:43:37.610 --> 00:43:43.130 align:middle line:84% And so now, the idea was that they could present stimuli 00:43:43.130 --> 00:43:45.310 align:middle line:90% within these regions. 00:43:45.310 --> 00:43:49.450 align:middle line:84% And this was done to monkeys, macaque monkeys, a common animal 00:43:49.450 --> 00:43:50.238 align:middle line:90% model in vision. 00:43:50.238 --> 00:43:52.030 align:middle line:84% They could present stimuli in those regions 00:43:52.030 --> 00:43:54.730 align:middle line:84% and then ask whether the monkey's ability 00:43:54.730 --> 00:43:56.610 align:middle line:84% to perform some kind of discrimination 00:43:56.610 --> 00:43:59.530 align:middle line:84% or other task on the stimuli would be affected by these two 00:43:59.530 --> 00:44:01.270 align:middle line:90% different types of lesions. 00:44:01.270 --> 00:44:05.190 align:middle line:90% 00:44:05.190 --> 00:44:06.890 align:middle line:90% And so these are the findings. 00:44:06.890 --> 00:44:09.950 align:middle line:84% So they measured the ability to discriminate 00:44:09.950 --> 00:44:13.150 align:middle line:84% a whole bunch of different types of basic visual properties. 00:44:13.150 --> 00:44:15.550 align:middle line:84% And this is showing the performance 00:44:15.550 --> 00:44:18.450 align:middle line:84% of these animals on these different discrimination tasks. 00:44:18.450 --> 00:44:20.610 align:middle line:84% So the y-axis here is plotting proportion correct. 00:44:20.610 --> 00:44:25.630 align:middle line:84% And there are three types of positions here. 00:44:25.630 --> 00:44:27.190 align:middle line:90% N is, I guess, normal. 00:44:27.190 --> 00:44:29.630 align:middle line:84% P is the parvocellular lesioned region. 00:44:29.630 --> 00:44:32.430 align:middle line:84% And M is the magnocellular lesioned region. 00:44:32.430 --> 00:44:34.510 align:middle line:84% And what you're supposed to take away from this 00:44:34.510 --> 00:44:38.190 align:middle line:84% is that in the region where there was a parvocellular 00:44:38.190 --> 00:44:42.750 align:middle line:84% lesion, there's a massive deficit in color discrimination 00:44:42.750 --> 00:44:47.190 align:middle line:84% and some deficits in what they call texture and pattern 00:44:47.190 --> 00:44:48.130 align:middle line:90% discrimination. 00:44:48.130 --> 00:44:50.790 align:middle line:84% By comparison, in the region where 00:44:50.790 --> 00:44:53.730 align:middle line:84% there's a magnocellular lesion, these other things are normal. 00:44:53.730 --> 00:44:57.010 align:middle line:84% But motion discrimination is greatly impaired. 00:44:57.010 --> 00:44:59.570 align:middle line:90% 00:44:59.570 --> 00:45:02.030 align:middle line:84% And there's also some other deficits. 00:45:02.030 --> 00:45:03.910 align:middle line:84% But those are the most striking ones. 00:45:03.910 --> 00:45:08.450 align:middle line:84% And so this kind of provided some very compelling evidence 00:45:08.450 --> 00:45:10.330 align:middle line:84% from segregation of function, the idea 00:45:10.330 --> 00:45:15.450 align:middle line:84% that there are these distinct populations in the LGN 00:45:15.450 --> 00:45:19.130 align:middle line:84% that are carrying information that's very critical for color 00:45:19.130 --> 00:45:22.910 align:middle line:84% perception and maybe fine form perception on the one hand 00:45:22.910 --> 00:45:25.400 align:middle line:84% and then motion perception on the other. 00:45:25.400 --> 00:45:27.150 align:middle line:84% And this starts all the way in the retina. 00:45:27.150 --> 00:45:30.270 align:middle line:84% We've got these midget and parasol cells in the retina. 00:45:30.270 --> 00:45:33.130 align:middle line:84% And those project to these distinct regions of the LGN. 00:45:33.130 --> 00:45:36.650 align:middle line:84% And those project to other stuff, as we'll see. 00:45:36.650 --> 00:45:39.970 align:middle line:84% But there's a pretty striking segregation of function. 00:45:39.970 --> 00:45:43.110 align:middle line:84% Any questions about this experiment? 00:45:43.110 --> 00:45:43.610 align:middle line:90% Yeah. 00:45:43.610 --> 00:45:45.450 align:middle line:84% AUDIENCE: Do you what the explanation 00:45:45.450 --> 00:45:47.950 align:middle line:84% is for what's going on between shape one and shape two, 00:45:47.950 --> 00:45:51.850 align:middle line:84% why parvocellular is really detrimental to discriminating 00:45:51.850 --> 00:45:55.142 align:middle line:84% between square and circle but not square and triangle? 00:45:55.142 --> 00:45:56.350 align:middle line:90% JOSH MCDERMOTT: I'm not sure. 00:45:56.350 --> 00:45:57.330 align:middle line:90% Yeah. 00:45:57.330 --> 00:45:58.012 align:middle line:90% Yeah. 00:45:58.012 --> 00:45:58.970 align:middle line:90% That's a good question. 00:45:58.970 --> 00:46:01.630 align:middle line:90% I don't know. 00:46:01.630 --> 00:46:02.730 align:middle line:90% Any other questions? 00:46:02.730 --> 00:46:05.510 align:middle line:90% 00:46:05.510 --> 00:46:06.150 align:middle line:90% Yeah. 00:46:06.150 --> 00:46:09.670 align:middle line:84% AUDIENCE: Do you think that the curvature 00:46:09.670 --> 00:46:13.360 align:middle line:84% is like [INAUDIBLE] like [INAUDIBLE] linearity 00:46:13.360 --> 00:46:14.530 align:middle line:90% ever since? 00:46:14.530 --> 00:46:16.030 align:middle line:84% JOSH MCDERMOTT: It's possible, yeah. 00:46:16.030 --> 00:46:19.110 align:middle line:90% Yeah, it's possible. 00:46:19.110 --> 00:46:24.070 align:middle line:84% Yeah, it's not what I would have predicted, necessarily. 00:46:24.070 --> 00:46:25.650 align:middle line:90% Yeah. 00:46:25.650 --> 00:46:27.990 align:middle line:84% The difference between-- so the parvocellular cells 00:46:27.990 --> 00:46:29.650 align:middle line:84% tend to have smaller receptive fields. 00:46:29.650 --> 00:46:33.665 align:middle line:84% And so yeah, maybe somehow detecting the curvature 00:46:33.665 --> 00:46:35.290 align:middle line:84% is more dependent on that or something. 00:46:35.290 --> 00:46:36.390 align:middle line:90% I don't know. 00:46:36.390 --> 00:46:39.150 align:middle line:90% Hard to say. 00:46:39.150 --> 00:46:41.570 align:middle line:84% So that's just a summary of what we saw there. 00:46:41.570 --> 00:46:43.727 align:middle line:84% So lesions in the parvocellular layers 00:46:43.727 --> 00:46:45.310 align:middle line:84% produce deficits in the discrimination 00:46:45.310 --> 00:46:47.670 align:middle line:84% of color, texture, fine shape, and pattern. 00:46:47.670 --> 00:46:49.590 align:middle line:84% Lesions in the magnocellular layers 00:46:49.590 --> 00:46:52.350 align:middle line:84% primarily resulted in difficulties 00:46:52.350 --> 00:46:55.730 align:middle line:84% in detecting motion, as well as also discriminating flicker. 00:46:55.730 --> 00:46:58.850 align:middle line:84% That's from another related study. 00:46:58.850 --> 00:47:02.085 align:middle line:84% And so the conclusion here is that the magnocellular cells 00:47:02.085 --> 00:47:04.210 align:middle line:84% play a specialized role in transmitting information 00:47:04.210 --> 00:47:05.330 align:middle line:90% about motion. 00:47:05.330 --> 00:47:09.250 align:middle line:84% And that, I think, has held up pretty well over the years. 00:47:09.250 --> 00:47:13.150 align:middle line:84% So we've been talking about the LGN, the lateral geniculate 00:47:13.150 --> 00:47:13.650 align:middle line:90% nucleus. 00:47:13.650 --> 00:47:18.017 align:middle line:84% That projects to the primary visual cortex, also known as V1. 00:47:18.017 --> 00:47:19.850 align:middle line:84% The visual cortex, in general, is divided up 00:47:19.850 --> 00:47:21.725 align:middle line:84% into lots of different areas, as we will see. 00:47:21.725 --> 00:47:23.910 align:middle line:90% But V1 is the largest. 00:47:23.910 --> 00:47:27.030 align:middle line:84% And so we're going to talk about that next. 00:47:27.030 --> 00:47:32.630 align:middle line:84% So onto the rest of the visual system, in particular, 00:47:32.630 --> 00:47:35.030 align:middle line:84% the cortex-- so again, this is where we're at. 00:47:35.030 --> 00:47:37.950 align:middle line:90% 00:47:37.950 --> 00:47:39.950 align:middle line:84% We're going to talk about primary visual cortex. 00:47:39.950 --> 00:47:43.010 align:middle line:90% 00:47:43.010 --> 00:47:46.910 align:middle line:84% One of the most salient features of the visual cortex, 00:47:46.910 --> 00:47:49.930 align:middle line:84% and again, a very fundamental organizational principle 00:47:49.930 --> 00:47:52.650 align:middle line:84% in the visual system, is what's known as retinotopy. 00:47:52.650 --> 00:47:58.310 align:middle line:84% And that refers to the fact that the receptive fields 00:47:58.310 --> 00:48:03.590 align:middle line:84% that you measure in individual neurons in the visual cortex 00:48:03.590 --> 00:48:06.790 align:middle line:84% are organized spatially across the cortex. 00:48:06.790 --> 00:48:09.550 align:middle line:84% So this is an example of an experiment 00:48:09.550 --> 00:48:11.730 align:middle line:90% that measures this in humans. 00:48:11.730 --> 00:48:14.240 align:middle line:90% So this was measured with fMRI. 00:48:14.240 --> 00:48:15.990 align:middle line:84% So this was an experiment where the person 00:48:15.990 --> 00:48:17.350 align:middle line:90% is in an fMRI scanner. 00:48:17.350 --> 00:48:21.547 align:middle line:84% They're looking at stimuli that will vary in eccentricity. 00:48:21.547 --> 00:48:23.630 align:middle line:84% Remember, eccentricity is distance from the fovea. 00:48:23.630 --> 00:48:29.390 align:middle line:84% So imagine you're looking at annuli that gradually get bigger 00:48:29.390 --> 00:48:31.110 align:middle line:90% very slowly. 00:48:31.110 --> 00:48:34.970 align:middle line:84% And then after the fact, you can analyze the data. 00:48:34.970 --> 00:48:37.750 align:middle line:84% And for every point in the cortex, 00:48:37.750 --> 00:48:40.590 align:middle line:84% you can estimate the eccentricity 00:48:40.590 --> 00:48:43.030 align:middle line:84% at which the stimulus produced the biggest response. 00:48:43.030 --> 00:48:44.650 align:middle line:90% And then you color code that. 00:48:44.650 --> 00:48:48.488 align:middle line:84% So this is a flattened map of the cortex. 00:48:48.488 --> 00:48:50.530 align:middle line:84% And what you're supposed to take away from this-- 00:48:50.530 --> 00:48:55.690 align:middle line:90% and so this is the-- 00:48:55.690 --> 00:48:59.530 align:middle line:84% it's a diagram that shows you the relationship 00:48:59.530 --> 00:49:01.970 align:middle line:84% between the color that's being used 00:49:01.970 --> 00:49:04.350 align:middle line:90% on this map and eccentricity. 00:49:04.350 --> 00:49:08.490 align:middle line:84% So the point is that yellow is the fovea, as well 00:49:08.490 --> 00:49:10.750 align:middle line:90% as the furthest eccentricity. 00:49:10.750 --> 00:49:12.330 align:middle line:90% And then red is the parafovea. 00:49:12.330 --> 00:49:14.770 align:middle line:84% Green is a little bit further out and so forth. 00:49:14.770 --> 00:49:19.010 align:middle line:84% And so you can see these stripes in the visual cortex, 00:49:19.010 --> 00:49:23.210 align:middle line:84% from yellow to red to green to blue. 00:49:23.210 --> 00:49:27.110 align:middle line:84% So there's a map of space that's laid out on the cortex. 00:49:27.110 --> 00:49:29.770 align:middle line:90% 00:49:29.770 --> 00:49:31.810 align:middle line:90% So this is in humans. 00:49:31.810 --> 00:49:34.730 align:middle line:84% This is a really famous experiment 00:49:34.730 --> 00:49:37.590 align:middle line:84% that kind of showed additional evidence for this. 00:49:37.590 --> 00:49:39.410 align:middle line:90% This is in macaque. 00:49:39.410 --> 00:49:42.130 align:middle line:84% This is, again, a really old school method 00:49:42.130 --> 00:49:43.970 align:middle line:84% that nobody would use anymore today 00:49:43.970 --> 00:49:46.190 align:middle line:84% because we have more refined methods. 00:49:46.190 --> 00:49:48.450 align:middle line:84% But what was done in these experiments 00:49:48.450 --> 00:49:53.630 align:middle line:84% is a monkey was injected with radioactive glucose. 00:49:53.630 --> 00:49:57.570 align:middle line:84% And then-- so they're anesthetized, of course. 00:49:57.570 --> 00:50:01.790 align:middle line:84% So the monkey's anesthetized and unconscious. 00:50:01.790 --> 00:50:04.230 align:middle line:84% And the monkey's made to look at the screen. 00:50:04.230 --> 00:50:07.530 align:middle line:84% So the idea is that you look at that screen. 00:50:07.530 --> 00:50:09.630 align:middle line:84% The visual cortex, bits of visual cortex 00:50:09.630 --> 00:50:12.310 align:middle line:84% that are processing the image, are active, 00:50:12.310 --> 00:50:15.030 align:middle line:84% taking up glucose, which is radioactive. 00:50:15.030 --> 00:50:17.770 align:middle line:84% And after the monkey's looked at this for a little while, 00:50:17.770 --> 00:50:18.850 align:middle line:90% the animal is sacrificed. 00:50:18.850 --> 00:50:21.670 align:middle line:84% They take the brain out and lay it down 00:50:21.670 --> 00:50:24.070 align:middle line:84% on some kind of photographic plate. 00:50:24.070 --> 00:50:27.070 align:middle line:84% And the idea is that the parts of the brain that 00:50:27.070 --> 00:50:30.610 align:middle line:84% are radioactive are then going to show up in an image. 00:50:30.610 --> 00:50:33.270 align:middle line:84% And so this is the resulting image of a monkey 00:50:33.270 --> 00:50:35.350 align:middle line:90% that looked at that. 00:50:35.350 --> 00:50:39.470 align:middle line:90% And so what you can see-- 00:50:39.470 --> 00:50:41.830 align:middle line:84% so that the image that the monkey was looking at 00:50:41.830 --> 00:50:46.630 align:middle line:84% was a set of rings that were flickering bits of checkerboard, 00:50:46.630 --> 00:50:49.308 align:middle line:84% and they're spaced out eccentrically. 00:50:49.308 --> 00:50:51.850 align:middle line:84% And then so it's kind of like there's polar coordinates here. 00:50:51.850 --> 00:50:53.642 align:middle line:84% And then you have these spokes that kind of 00:50:53.642 --> 00:50:55.470 align:middle line:90% emerge from the wagon wheel. 00:50:55.470 --> 00:50:59.090 align:middle line:84% And you can see that those polar coordinates 00:50:59.090 --> 00:51:00.770 align:middle line:84% kind of get translated into what looks 00:51:00.770 --> 00:51:03.410 align:middle line:84% like Cartesian coordinates, roughly speaking, 00:51:03.410 --> 00:51:04.970 align:middle line:90% in the visual cortex. 00:51:04.970 --> 00:51:08.170 align:middle line:84% But the other thing to take away from this 00:51:08.170 --> 00:51:12.090 align:middle line:84% is that you can see that the spacing of the rings 00:51:12.090 --> 00:51:13.610 align:middle line:90% is not uniform. 00:51:13.610 --> 00:51:16.450 align:middle line:84% The ones that are very close to the fovea 00:51:16.450 --> 00:51:17.590 align:middle line:90% are very closely spaced. 00:51:17.590 --> 00:51:19.340 align:middle line:84% And then as you move out to the periphery, 00:51:19.340 --> 00:51:22.210 align:middle line:84% they become much further apart, whereas when 00:51:22.210 --> 00:51:26.350 align:middle line:84% you look in the visual cortex the spacing is pretty uniform. 00:51:26.350 --> 00:51:34.570 align:middle line:90% 00:51:34.570 --> 00:51:39.170 align:middle line:84% So the stripes on the screen don't 00:51:39.170 --> 00:51:43.570 align:middle line:84% translate to a similar spacing in the visual cortex. 00:51:43.570 --> 00:51:45.310 align:middle line:84% And this represents a phenomenon, 00:51:45.310 --> 00:51:48.970 align:middle line:84% a very important phenomenon, known as cortical magnification. 00:51:48.970 --> 00:51:53.470 align:middle line:84% And the essential idea is that there's 00:51:53.470 --> 00:51:56.150 align:middle line:84% much more cortical real estate that 00:51:56.150 --> 00:52:00.710 align:middle line:84% is devoted to the center of gaze than to the periphery. 00:52:00.710 --> 00:52:03.350 align:middle line:84% So the representation of the center of gaze 00:52:03.350 --> 00:52:06.790 align:middle line:84% has kind of blown up in the cortex. 00:52:06.790 --> 00:52:08.430 align:middle line:84% And so the consequence of that is 00:52:08.430 --> 00:52:12.430 align:middle line:84% that those rings that are very closely spaced on the screen 00:52:12.430 --> 00:52:16.110 align:middle line:84% end up being spread out in the cortex, 00:52:16.110 --> 00:52:17.670 align:middle line:84% whereas the rings that are very far 00:52:17.670 --> 00:52:19.950 align:middle line:84% apart on the screen, because they're out in the periphery, 00:52:19.950 --> 00:52:21.325 align:middle line:84% end up being kind of a little bit 00:52:21.325 --> 00:52:23.910 align:middle line:90% closer together in the cortex. 00:52:23.910 --> 00:52:25.170 align:middle line:90% So what's that all about? 00:52:25.170 --> 00:52:28.190 align:middle line:90% And why does it happen? 00:52:28.190 --> 00:52:31.170 align:middle line:84% So we can get some insight into this from looking at the retina. 00:52:31.170 --> 00:52:34.270 align:middle line:84% So these are two cross-sections of the retina. 00:52:34.270 --> 00:52:37.810 align:middle line:84% So this is the ganglion cell layer. 00:52:37.810 --> 00:52:39.090 align:middle line:90% That's the photoreceptors. 00:52:39.090 --> 00:52:41.950 align:middle line:84% Remember, you've got all these cell layers in the retina. 00:52:41.950 --> 00:52:44.450 align:middle line:84% And you can see here, this is what's called the parafovea. 00:52:44.450 --> 00:52:46.390 align:middle line:84% So that's right next to the fovea. 00:52:46.390 --> 00:52:48.690 align:middle line:84% And you can see there's a lot of ganglion cells, 00:52:48.690 --> 00:52:52.290 align:middle line:84% because remember, near the center of gaze, 00:52:52.290 --> 00:52:53.990 align:middle line:84% the image is sampled very, very densely. 00:52:53.990 --> 00:52:57.390 align:middle line:84% There's a ton of photoreceptors, lots of ganglion cells, 00:52:57.390 --> 00:52:59.690 align:middle line:84% small receptive fields, high resolution. 00:52:59.690 --> 00:53:01.130 align:middle line:90% This is out in the periphery. 00:53:01.130 --> 00:53:03.010 align:middle line:84% And you can see that the ganglion cell layer 00:53:03.010 --> 00:53:05.828 align:middle line:90% has far fewer cell bodies. 00:53:05.828 --> 00:53:07.370 align:middle line:84% And that's because out there, there's 00:53:07.370 --> 00:53:09.722 align:middle line:90% not as many photoreceptors. 00:53:09.722 --> 00:53:11.430 align:middle line:84% Other receptive fields tend to be bigger. 00:53:11.430 --> 00:53:12.638 align:middle line:90% The sampling is much coarser. 00:53:12.638 --> 00:53:13.770 align:middle line:90% Resolution is worse. 00:53:13.770 --> 00:53:17.250 align:middle line:90% 00:53:17.250 --> 00:53:21.190 align:middle line:84% Now, this kind of has to be set up this way 00:53:21.190 --> 00:53:25.570 align:middle line:84% because the retina is physically constrained by the fact 00:53:25.570 --> 00:53:28.310 align:middle line:84% that the image is formed on the retina. 00:53:28.310 --> 00:53:30.810 align:middle line:84% And so your receptors kind of have 00:53:30.810 --> 00:53:34.755 align:middle line:84% to be physically at the image location that they're sampling. 00:53:34.755 --> 00:53:36.130 align:middle line:84% And so if you want to have really 00:53:36.130 --> 00:53:37.672 align:middle line:84% dense sampling at the fovea, you just 00:53:37.672 --> 00:53:39.510 align:middle line:84% have to squeeze everything into the fovea. 00:53:39.510 --> 00:53:42.706 align:middle line:90% 00:53:42.706 --> 00:53:46.890 align:middle line:84% So unlike in the retina, in V1, the cell density 00:53:46.890 --> 00:53:50.667 align:middle line:90% is relatively uniform. 00:53:50.667 --> 00:53:52.750 align:middle line:84% And as a consequence of that, the mapping of space 00:53:52.750 --> 00:53:54.950 align:middle line:90% becomes very non-uniform. 00:53:54.950 --> 00:53:57.410 align:middle line:84% So you take all those ganglion cells from the fovea, 00:53:57.410 --> 00:54:01.230 align:middle line:84% and those get kind of spread out over the visual cortex. 00:54:01.230 --> 00:54:05.830 align:middle line:84% And then you end up needing much less cortex for the periphery. 00:54:05.830 --> 00:54:07.930 align:middle line:84% So that's the idea of cortical magnification. 00:54:07.930 --> 00:54:12.270 align:middle line:84% So it's a consequence of the foveal sampling organization 00:54:12.270 --> 00:54:14.550 align:middle line:84% of the retina that we were just talking about, 00:54:14.550 --> 00:54:18.350 align:middle line:84% where we sample very densely at the center of gaze. 00:54:18.350 --> 00:54:20.230 align:middle line:84% And in the cortex that translates 00:54:20.230 --> 00:54:24.630 align:middle line:84% into a very large region of representation. 00:54:24.630 --> 00:54:25.730 align:middle line:90% Questions about that? 00:54:25.730 --> 00:54:32.350 align:middle line:90% 00:54:32.350 --> 00:54:36.735 align:middle line:84% So cortical magnification, that's the big idea. 00:54:36.735 --> 00:54:38.110 align:middle line:84% Now, there's a lot of other stuff 00:54:38.110 --> 00:54:40.410 align:middle line:84% that you encounter once you get to the cortex. 00:54:40.410 --> 00:54:44.350 align:middle line:84% So the most famous one is a novel feature 00:54:44.350 --> 00:54:47.768 align:middle line:84% of cortical neuronal responses called orientation selectivity. 00:54:47.768 --> 00:54:48.810 align:middle line:90% It's what it sounds like. 00:54:48.810 --> 00:54:51.410 align:middle line:84% It refers to the fact that the neurons 00:54:51.410 --> 00:54:55.070 align:middle line:84% become selective for the orientation of simple stimuli, 00:54:55.070 --> 00:54:56.610 align:middle line:90% like lines and bars. 00:54:56.610 --> 00:54:59.638 align:middle line:84% So when the line is vertical and placed in the receptive field, 00:54:59.638 --> 00:55:01.930 align:middle line:84% you get a brisk response, whereas when it's horizontal, 00:55:01.930 --> 00:55:02.770 align:middle line:90% you don't. 00:55:02.770 --> 00:55:06.530 align:middle line:84% So this is probably one of the most important discoveries 00:55:06.530 --> 00:55:10.930 align:middle line:84% in the history of neuroscience, arguably. 00:55:10.930 --> 00:55:16.910 align:middle line:84% And so we'll talk a little bit about the basis of this. 00:55:16.910 --> 00:55:19.290 align:middle line:84% So what might the receptive field 00:55:19.290 --> 00:55:21.450 align:middle line:84% of an orientation selective neuron look like? 00:55:21.450 --> 00:55:25.850 align:middle line:84% So again, I have a video here of an experiment where 00:55:25.850 --> 00:55:28.290 align:middle line:84% the experimenters are recording from a neuron 00:55:28.290 --> 00:55:31.310 align:middle line:84% in primary visual cortex, flashing light up on a screen. 00:55:31.310 --> 00:55:33.450 align:middle line:84% So the movie shows patterns of light 00:55:33.450 --> 00:55:34.825 align:middle line:84% that they flashed up on a screen. 00:55:34.825 --> 00:55:37.200 align:middle line:84% And you would be listening to the response of the neuron. 00:55:37.200 --> 00:55:38.630 align:middle line:84% Again, we don't have sound today, 00:55:38.630 --> 00:55:41.490 align:middle line:84% so I'm going to skip this, and we'll do it next time. 00:55:41.490 --> 00:55:45.880 align:middle line:84% But what you would see is that you would see evidence 00:55:45.880 --> 00:55:48.750 align:middle line:84% for a receptive field that I think 00:55:48.750 --> 00:55:50.000 align:middle line:90% looks something like this one. 00:55:50.000 --> 00:55:52.040 align:middle line:84% I can't remember for that exact example. 00:55:52.040 --> 00:55:54.560 align:middle line:84% So essentially, there is an elongated region 00:55:54.560 --> 00:56:00.240 align:middle line:84% of space within which light increases the firing 00:56:00.240 --> 00:56:04.800 align:middle line:84% rate of the neuron and then kind of adjacent regions of space 00:56:04.800 --> 00:56:09.690 align:middle line:84% where light actually decreases the response of the neuron. 00:56:09.690 --> 00:56:11.940 align:middle line:84% So you get these receptive fields that look like this. 00:56:11.940 --> 00:56:14.555 align:middle line:84% And so the concept here, it's exactly 00:56:14.555 --> 00:56:16.680 align:middle line:84% analogous to those center-surround receptive fields 00:56:16.680 --> 00:56:17.888 align:middle line:90% that we were just looking at. 00:56:17.888 --> 00:56:19.240 align:middle line:90% It's a spatial filter. 00:56:19.240 --> 00:56:23.080 align:middle line:84% So you take that filter, and you compute the dot product 00:56:23.080 --> 00:56:24.180 align:middle line:90% with respect to the image. 00:56:24.180 --> 00:56:26.640 align:middle line:84% And that predicts the response of the neuron pretty well. 00:56:26.640 --> 00:56:29.880 align:middle line:84% Same idea, it's just that these are not circularly symmetric. 00:56:29.880 --> 00:56:32.260 align:middle line:84% They now have this kind of oriented structure. 00:56:32.260 --> 00:56:36.040 align:middle line:84% And they often kind come in these different flavors 00:56:36.040 --> 00:56:39.240 align:middle line:84% and are often described as even or odd functions. 00:56:39.240 --> 00:56:42.700 align:middle line:84% Remember, an is even function is symmetric about 0, 00:56:42.700 --> 00:56:44.620 align:middle line:90% so f of minus x equals f of x. 00:56:44.620 --> 00:56:47.120 align:middle line:84% And an odd function is antisymmetric. 00:56:47.120 --> 00:56:49.900 align:middle line:90% 00:56:49.900 --> 00:56:54.140 align:middle line:84% So these neurons were discovered by Hubel and Wiesel, 00:56:54.140 --> 00:56:56.940 align:middle line:84% neuroscientists who worked for many years 00:56:56.940 --> 00:57:00.820 align:middle line:84% at the Harvard Medical School and won a Nobel Prize for this 00:57:00.820 --> 00:57:02.700 align:middle line:90% and related discoveries. 00:57:02.700 --> 00:57:06.220 align:middle line:84% And these neurons were dubbed by them simple cells. 00:57:06.220 --> 00:57:08.460 align:middle line:84% And that will be contrasted with complex cells 00:57:08.460 --> 00:57:11.740 align:middle line:90% that we'll see in just a second. 00:57:11.740 --> 00:57:16.140 align:middle line:84% So you see these when you get to primary visual cortex. 00:57:16.140 --> 00:57:21.060 align:middle line:84% And Hubel and Wiesel also had this fairly influential proposal 00:57:21.060 --> 00:57:26.100 align:middle line:84% for how you would build a neuron with this type of orientation 00:57:26.100 --> 00:57:26.800 align:middle line:90% selectivity. 00:57:26.800 --> 00:57:28.820 align:middle line:84% So remember, primary visual cortex 00:57:28.820 --> 00:57:31.060 align:middle line:90% gets its input from the LGN. 00:57:31.060 --> 00:57:33.220 align:middle line:84% In the LGN you have center-surround receptive 00:57:33.220 --> 00:57:36.340 align:middle line:84% fields, just like you have in the retina. 00:57:36.340 --> 00:57:38.700 align:middle line:84% And the idea is that you could create 00:57:38.700 --> 00:57:43.480 align:middle line:84% an orientation selective neuron by setting a neuron 00:57:43.480 --> 00:57:47.960 align:middle line:84% up so that it would get inputs from a set of neurons that 00:57:47.960 --> 00:57:50.680 align:middle line:84% have center-surround receptive fields that are spatially 00:57:50.680 --> 00:57:53.480 align:middle line:90% arranged in an organized way. 00:57:53.480 --> 00:57:55.640 align:middle line:84% So the idea is that each one of these things 00:57:55.640 --> 00:57:57.300 align:middle line:90% corresponds to one neuron. 00:57:57.300 --> 00:57:59.520 align:middle line:84% And so if you took all of these and all 00:57:59.520 --> 00:58:01.520 align:middle line:84% of these and all of these, and you wired them 00:58:01.520 --> 00:58:03.720 align:middle line:84% up so that they would all provide input 00:58:03.720 --> 00:58:05.280 align:middle line:84% to the same neuron, you could think 00:58:05.280 --> 00:58:06.905 align:middle line:84% of the response of that neuron as being 00:58:06.905 --> 00:58:08.740 align:middle line:84% a sum of the response of these neurons. 00:58:08.740 --> 00:58:10.282 align:middle line:84% And the receptive field would loosely 00:58:10.282 --> 00:58:13.160 align:middle line:84% be approximated by the sum of those receptive fields. 00:58:13.160 --> 00:58:16.100 align:middle line:90% So that was their idea. 00:58:16.100 --> 00:58:19.920 align:middle line:84% So in part because this discovery of orientation 00:58:19.920 --> 00:58:22.480 align:middle line:84% selectivity has been such a signature 00:58:22.480 --> 00:58:24.640 align:middle line:84% discovery in neuroscience, there's 00:58:24.640 --> 00:58:26.080 align:middle line:84% been lots and lots of work trying 00:58:26.080 --> 00:58:29.120 align:middle line:84% to unpack the mechanisms underlying orientation 00:58:29.120 --> 00:58:30.040 align:middle line:90% selectivity. 00:58:30.040 --> 00:58:34.180 align:middle line:84% And I would say it still remains fairly controversial. 00:58:34.180 --> 00:58:36.440 align:middle line:84% But I think it's pretty clear that aspects 00:58:36.440 --> 00:58:39.920 align:middle line:84% of the original model proposed by Hubel and Wiesel 00:58:39.920 --> 00:58:41.740 align:middle line:90% are likely to be correct. 00:58:41.740 --> 00:58:43.700 align:middle line:84% And one piece of evidence for that 00:58:43.700 --> 00:58:47.940 align:middle line:84% comes from this paper here from back in 1995. 00:58:47.940 --> 00:58:51.060 align:middle line:84% This was from the lab of Clay Reid, who at the time 00:58:51.060 --> 00:58:52.700 align:middle line:84% was at the Rockefeller University 00:58:52.700 --> 00:58:55.840 align:middle line:84% and then was at Harvard Medical School for a long time. 00:58:55.840 --> 00:58:57.980 align:middle line:90% Now he's at the Allen Institute. 00:58:57.980 --> 00:59:04.060 align:middle line:84% And so the idea here was to try to understand 00:59:04.060 --> 00:59:07.420 align:middle line:84% the way in which orientation selective receptive fields are 00:59:07.420 --> 00:59:11.420 align:middle line:84% wired up by measuring at the same time 00:59:11.420 --> 00:59:14.800 align:middle line:84% from neurons in the LGN and neurons in visual cortex. 00:59:14.800 --> 00:59:16.300 align:middle line:84% So these are super hard experiments. 00:59:16.300 --> 00:59:20.860 align:middle line:84% So you have to have an electrode that is positioned in the LGN 00:59:20.860 --> 00:59:23.460 align:middle line:84% and an electrode that's positioned in V1. 00:59:23.460 --> 00:59:27.140 align:middle line:84% And then you're trying to locate neurons 00:59:27.140 --> 00:59:29.980 align:middle line:84% that have receptive fields that are spatially aligned 00:59:29.980 --> 00:59:33.420 align:middle line:84% because the question was, well, if you find 00:59:33.420 --> 00:59:34.980 align:middle line:84% receptive fields that are kind of 00:59:34.980 --> 00:59:37.900 align:middle line:84% at the appropriate spatial position in the LGN, 00:59:37.900 --> 00:59:41.000 align:middle line:84% would they then be connected to neurons 00:59:41.000 --> 00:59:44.527 align:middle line:84% in V1 that would have the appropriate receptive fields? 00:59:44.527 --> 00:59:46.360 align:middle line:84% So that was what they were trying to get at. 00:59:46.360 --> 00:59:49.440 align:middle line:90% 00:59:49.440 --> 00:59:50.660 align:middle line:90% So how does this work? 00:59:50.660 --> 00:59:54.720 align:middle line:84% Well, you're recording from the LGN and from V1. 00:59:54.720 --> 00:59:57.658 align:middle line:84% These are measured receptive fields. 00:59:57.658 --> 00:59:59.700 align:middle line:84% Think of these as like a spike-triggered average. 00:59:59.700 --> 01:00:02.820 align:middle line:84% So this is your estimate of the receptive field or the filter. 01:00:02.820 --> 01:00:06.600 align:middle line:84% So the LGN has a center-surround receptive field. 01:00:06.600 --> 01:00:08.980 align:middle line:84% This is the V1 neuron, which is kind of oriented. 01:00:08.980 --> 01:00:11.340 align:middle line:84% So this is an odd receptive field. 01:00:11.340 --> 01:00:13.640 align:middle line:84% So there's a lobe here and a lobe here. 01:00:13.640 --> 01:00:16.620 align:middle line:84% This is the actual measured spike-triggered average. 01:00:16.620 --> 01:00:21.393 align:middle line:84% And then for the purposes of the analysis, they would fit models. 01:00:21.393 --> 01:00:23.060 align:middle line:84% This would be a difference of Gaussians. 01:00:23.060 --> 01:00:24.602 align:middle line:84% This was like a Gabor function, which 01:00:24.602 --> 01:00:26.640 align:middle line:90% we'll talk about in a second. 01:00:26.640 --> 01:00:29.060 align:middle line:84% So they would record from these pairs of neurons. 01:00:29.060 --> 01:00:32.380 align:middle line:84% And then the question was, are these neurons connected? 01:00:32.380 --> 01:00:34.240 align:middle line:84% And so one way that you can actually 01:00:34.240 --> 01:00:38.860 align:middle line:84% get some evidence for that is to look at the relationship in time 01:00:38.860 --> 01:00:41.460 align:middle line:84% between spikes that are fired in one neuron and spikes 01:00:41.460 --> 01:00:43.100 align:middle line:90% that are fired in the other. 01:00:43.100 --> 01:00:46.980 align:middle line:84% So the idea is that if the LGN neuron is providing input 01:00:46.980 --> 01:00:50.620 align:middle line:84% to the V1 neuron, that if the LGN neuron spikes, 01:00:50.620 --> 01:00:52.420 align:middle line:84% the V1 neuron should be likely to spike 01:00:52.420 --> 01:00:54.820 align:middle line:90% just a little bit afterwards. 01:00:54.820 --> 01:00:56.340 align:middle line:84% And the way that you actually detect 01:00:56.340 --> 01:00:59.820 align:middle line:84% that is by measuring something called the cross-correlogram. 01:00:59.820 --> 01:01:04.100 align:middle line:84% And what this is, is you record from these two neurons. 01:01:04.100 --> 01:01:07.280 align:middle line:84% And every time you get a pair of spikes, 01:01:07.280 --> 01:01:11.020 align:middle line:84% you measure the time lag between the spike in the V1 cell 01:01:11.020 --> 01:01:12.680 align:middle line:90% and the spike in the LGN cell. 01:01:12.680 --> 01:01:14.560 align:middle line:84% And you just make a histogram of that. 01:01:14.560 --> 01:01:16.800 align:middle line:90% So what does this show us? 01:01:16.800 --> 01:01:18.420 align:middle line:90% So this is 0 here. 01:01:18.420 --> 01:01:22.580 align:middle line:84% And this shows that there's this big peak here right after 0, 01:01:22.580 --> 01:01:25.700 align:middle line:84% so just a millisecond or two after 0. 01:01:25.700 --> 01:01:28.020 align:middle line:84% And that provides evidence of what's 01:01:28.020 --> 01:01:30.700 align:middle line:84% called a monosynaptic connection, that the LGN 01:01:30.700 --> 01:01:33.860 align:middle line:84% cell's directly connecting to the cell in V1. 01:01:33.860 --> 01:01:38.132 align:middle line:84% So that provides some evidence that one neuron is 01:01:38.132 --> 01:01:39.340 align:middle line:90% providing input to the other. 01:01:39.340 --> 01:01:43.120 align:middle line:84% So now the question is, can you explain the receptive field 01:01:43.120 --> 01:01:45.400 align:middle line:84% of the orientation selective neuron 01:01:45.400 --> 01:01:47.960 align:middle line:84% in terms of the receptive field of the cell that 01:01:47.960 --> 01:01:49.460 align:middle line:90% is providing input to it? 01:01:49.460 --> 01:01:52.000 align:middle line:84% And so the way that they address that was 01:01:52.000 --> 01:01:57.320 align:middle line:84% by asking whether the receptive field of the LGN neuron 01:01:57.320 --> 01:02:00.480 align:middle line:84% was appropriately positioned relative to the receptive 01:02:00.480 --> 01:02:01.820 align:middle line:90% field of the V1 neuron. 01:02:01.820 --> 01:02:04.880 align:middle line:84% So in other words, was the excitatory center 01:02:04.880 --> 01:02:09.040 align:middle line:84% kind of overlapped with the excitatory lobe 01:02:09.040 --> 01:02:10.680 align:middle line:90% of the LGN neuron? 01:02:10.680 --> 01:02:14.640 align:middle line:90% 01:02:14.640 --> 01:02:16.660 align:middle line:84% And this is shown here for this one example. 01:02:16.660 --> 01:02:20.820 align:middle line:84% So you see the circle here that's fit around the neuron. 01:02:20.820 --> 01:02:23.560 align:middle line:84% And then they're replotting that circle 01:02:23.560 --> 01:02:25.500 align:middle line:90% around the excitatory lobe. 01:02:25.500 --> 01:02:28.480 align:middle line:84% It's a little hard to see here because the lighting is high. 01:02:28.480 --> 01:02:29.960 align:middle line:90% But that's there. 01:02:29.960 --> 01:02:32.100 align:middle line:84% And then so this is just one example right. 01:02:32.100 --> 01:02:34.040 align:middle line:84% So to see whether this relationship 01:02:34.040 --> 01:02:37.500 align:middle line:84% kind of held over lots of these pairs, what they did 01:02:37.500 --> 01:02:40.020 align:middle line:84% is they recorded from a whole bunch of these pairs 01:02:40.020 --> 01:02:44.140 align:middle line:84% and then shifted and rotated the receptive fields of all 01:02:44.140 --> 01:02:48.300 align:middle line:84% of the oriented receptive fields so that they were aligned 01:02:48.300 --> 01:02:52.340 align:middle line:84% and shifted and oriented the LGN receptive fields 01:02:52.340 --> 01:02:53.380 align:middle line:90% in the same way. 01:02:53.380 --> 01:02:57.940 align:middle line:84% So this is a summary figure of the data from this study. 01:02:57.940 --> 01:03:00.000 align:middle line:84% So you have all these pairs of neurons. 01:03:00.000 --> 01:03:03.380 align:middle line:84% They're now all displayed in this canonical kind 01:03:03.380 --> 01:03:04.540 align:middle line:90% of orientation. 01:03:04.540 --> 01:03:10.100 align:middle line:84% And so the question is, are the excitatory or inhibitory centers 01:03:10.100 --> 01:03:13.180 align:middle line:84% of the LGN receptive fields kind of lining up 01:03:13.180 --> 01:03:15.740 align:middle line:84% with the excitatory and inhibitory lobe of the LNG 01:03:15.740 --> 01:03:16.980 align:middle line:90% receptive field? 01:03:16.980 --> 01:03:19.420 align:middle line:84% And the argument is that they do. 01:03:19.420 --> 01:03:23.160 align:middle line:84% So these red circles predominantly fall here. 01:03:23.160 --> 01:03:25.980 align:middle line:84% And then the blue circles kind of predominantly fall here. 01:03:25.980 --> 01:03:30.660 align:middle line:84% So this provides some evidence that that kind 01:03:30.660 --> 01:03:33.140 align:middle line:84% of original notion from Hubel and Wiesel, 01:03:33.140 --> 01:03:36.080 align:middle line:84% that you could explain an orientation selective receptive 01:03:36.080 --> 01:03:39.200 align:middle line:84% field just by kind of wiring up combinations 01:03:39.200 --> 01:03:42.080 align:middle line:84% of center-surround receptive fields from the LGN. 01:03:42.080 --> 01:03:44.040 align:middle line:84% It provides some evidence that that's possibly 01:03:44.040 --> 01:03:45.340 align:middle line:90% a reasonable account. 01:03:45.340 --> 01:03:48.940 align:middle line:90% 01:03:48.940 --> 01:03:52.160 align:middle line:84% And more generally, this kind of method 01:03:52.160 --> 01:03:55.780 align:middle line:84% is something that you could potentially apply elsewhere 01:03:55.780 --> 01:03:59.560 align:middle line:84% if you wanted to understand how different kinds of structures 01:03:59.560 --> 01:04:02.020 align:middle line:84% in the brain are composed from their inputs. 01:04:02.020 --> 01:04:05.225 align:middle line:84% So you establish that neurons are connected 01:04:05.225 --> 01:04:06.600 align:middle line:84% and then look at the relationship 01:04:06.600 --> 01:04:08.880 align:middle line:90% between the response properties. 01:04:08.880 --> 01:04:09.980 align:middle line:90% Questions about that? 01:04:09.980 --> 01:04:23.680 align:middle line:90% 01:04:23.680 --> 01:04:28.520 align:middle line:84% So we've got orientation selectivity evident 01:04:28.520 --> 01:04:31.720 align:middle line:90% in these neuronal responses. 01:04:31.720 --> 01:04:32.920 align:middle line:90% How is this organized? 01:04:32.920 --> 01:04:37.540 align:middle line:84% And it turns out that it's organized in a very orderly way. 01:04:37.540 --> 01:04:41.597 align:middle line:84% And this gets at another kind of central organizational concept, 01:04:41.597 --> 01:04:43.180 align:middle line:84% really, across the entire brain, which 01:04:43.180 --> 01:04:45.260 align:middle line:84% is the notion that the cortex tends 01:04:45.260 --> 01:04:47.500 align:middle line:90% to be organized in columns. 01:04:47.500 --> 01:04:52.660 align:middle line:84% So what's shown here, it's a schematic result 01:04:52.660 --> 01:04:56.580 align:middle line:84% of an experiment where an electrode would be lowered, 01:04:56.580 --> 01:04:59.540 align:middle line:84% either in this direction or this direction, 01:04:59.540 --> 01:05:01.960 align:middle line:84% and at every position in the brain, 01:05:01.960 --> 01:05:05.820 align:middle line:84% you stop and characterize the response properties of a cell 01:05:05.820 --> 01:05:07.220 align:middle line:90% at that position. 01:05:07.220 --> 01:05:11.340 align:middle line:84% And the line segment that is drawn here 01:05:11.340 --> 01:05:14.180 align:middle line:84% represents the orientation preference 01:05:14.180 --> 01:05:17.380 align:middle line:90% of the cell at that position. 01:05:17.380 --> 01:05:20.020 align:middle line:84% And so what you're supposed to take away from this 01:05:20.020 --> 01:05:22.160 align:middle line:90% is that when the electrode-- 01:05:22.160 --> 01:05:23.820 align:middle line:90% this is the cortex. 01:05:23.820 --> 01:05:27.020 align:middle line:84% You can think of it as a big sheet, kind of a folded sheet. 01:05:27.020 --> 01:05:29.300 align:middle line:84% So when the electrode is inserted perpendicular 01:05:29.300 --> 01:05:33.120 align:middle line:84% to the cortex, you have very similar orientation preferences 01:05:33.120 --> 01:05:37.320 align:middle line:84% all the way down, whereas when you insert the electrode 01:05:37.320 --> 01:05:39.560 align:middle line:84% at an angle, you can see that the orientation 01:05:39.560 --> 01:05:41.640 align:middle line:90% preference changes. 01:05:41.640 --> 01:05:43.240 align:middle line:84% But you can also see that it kind of 01:05:43.240 --> 01:05:47.400 align:middle line:90% changes in a smooth fashion. 01:05:47.400 --> 01:05:49.940 align:middle line:90% And so this hints at two ideas. 01:05:49.940 --> 01:05:53.240 align:middle line:84% One is that the cortex is organized in columns, 01:05:53.240 --> 01:05:55.440 align:middle line:84% so within a column, the neurons tend 01:05:55.440 --> 01:05:57.600 align:middle line:84% to have fairly consistent properties. 01:05:57.600 --> 01:06:02.040 align:middle line:84% And then the other thing is that the columns 01:06:02.040 --> 01:06:04.900 align:middle line:84% tend to be kind of spatially organized in a particular way 01:06:04.900 --> 01:06:07.800 align:middle line:84% so that you have a smooth map of orientation 01:06:07.800 --> 01:06:09.340 align:middle line:90% over the cortical surface. 01:06:09.340 --> 01:06:12.240 align:middle line:90% 01:06:12.240 --> 01:06:15.720 align:middle line:84% So here's another figure that shows this a little bit more 01:06:15.720 --> 01:06:16.260 align:middle line:90% clearly. 01:06:16.260 --> 01:06:19.160 align:middle line:84% So this is, again, an electrode that's being kind of lowered 01:06:19.160 --> 01:06:21.200 align:middle line:90% through the cortex at an angle. 01:06:21.200 --> 01:06:23.760 align:middle line:84% And so this is the track distance in millimeters 01:06:23.760 --> 01:06:26.190 align:middle line:84% I don't know why this is so low resolution. 01:06:26.190 --> 01:06:27.940 align:middle line:84% But you can see the effect really clearly. 01:06:27.940 --> 01:06:30.620 align:middle line:84% And this is the preferred orientation of the neurons 01:06:30.620 --> 01:06:31.560 align:middle line:90% that are encountered. 01:06:31.560 --> 01:06:35.082 align:middle line:84% And so you can see that this is very smoothly over the course 01:06:35.082 --> 01:06:36.040 align:middle line:90% of the electrode track. 01:06:36.040 --> 01:06:38.700 align:middle line:84% So as you move through the cortex, 01:06:38.700 --> 01:06:40.460 align:middle line:84% the dominant preferred orientation 01:06:40.460 --> 01:06:42.560 align:middle line:84% changes in this continuous fashion. 01:06:42.560 --> 01:06:45.880 align:middle line:90% 01:06:45.880 --> 01:06:48.780 align:middle line:84% So this is the result of an experiment where 01:06:48.780 --> 01:06:51.260 align:middle line:84% it's an imaging experiment, where you look down 01:06:51.260 --> 01:06:56.300 align:middle line:84% at the cortex and measure the orientation preferences 01:06:56.300 --> 01:06:58.400 align:middle line:90% at each point along the cortex. 01:06:58.400 --> 01:07:01.720 align:middle line:84% And so you can see the columns here. 01:07:01.720 --> 01:07:04.180 align:middle line:84% So the color here denotes the preferred orientation, 01:07:04.180 --> 01:07:05.340 align:middle line:90% I should have said. 01:07:05.340 --> 01:07:06.715 align:middle line:84% And so you can see that there are 01:07:06.715 --> 01:07:08.540 align:middle line:84% these colored blobs that are distributed 01:07:08.540 --> 01:07:10.260 align:middle line:90% all across the cortex. 01:07:10.260 --> 01:07:13.900 align:middle line:90% These are orientation columns. 01:07:13.900 --> 01:07:16.880 align:middle line:84% So important concept here, columnar structure, 01:07:16.880 --> 01:07:19.820 align:middle line:84% you see that pretty much everywhere in the brain. 01:07:19.820 --> 01:07:23.580 align:middle line:84% Orientation columns are a really robust example of that, 01:07:23.580 --> 01:07:26.880 align:middle line:84% and then also maps, so maps are really common in the brain. 01:07:26.880 --> 01:07:30.560 align:middle line:84% So variables that are very important to be coded, 01:07:30.560 --> 01:07:35.340 align:middle line:84% you tend to see tuning for that very continuously. 01:07:35.340 --> 01:07:38.320 align:middle line:84% So we've got this orientation selectivity. 01:07:38.320 --> 01:07:42.860 align:middle line:84% One important idea is what's shown here. 01:07:42.860 --> 01:07:47.760 align:middle line:84% And that is that just given a single neuron, 01:07:47.760 --> 01:07:50.860 align:middle line:84% it's very difficult to actually estimate what the stimulus is. 01:07:50.860 --> 01:07:53.320 align:middle line:84% And in this particular example, what 01:07:53.320 --> 01:07:58.400 align:middle line:84% you're supposed to note here is that these three stimuli are all 01:07:58.400 --> 01:08:01.280 align:middle line:84% designed to evoke the same response from this filter. 01:08:01.280 --> 01:08:04.120 align:middle line:84% So you can see that this is a neuron that's 01:08:04.120 --> 01:08:06.280 align:middle line:90% got a vertical receptive field. 01:08:06.280 --> 01:08:09.240 align:middle line:84% So the orientation that gives you the best response 01:08:09.240 --> 01:08:10.333 align:middle line:90% is vertical. 01:08:10.333 --> 01:08:12.000 align:middle line:84% But you can also manipulate the response 01:08:12.000 --> 01:08:15.340 align:middle line:84% by changing the image intensity or the contrast. 01:08:15.340 --> 01:08:17.840 align:middle line:84% And so here the orientation is changed, 01:08:17.840 --> 01:08:21.040 align:middle line:84% and the contrast is increased by just the right amount 01:08:21.040 --> 01:08:23.942 align:middle line:90% to give you the same response. 01:08:23.942 --> 01:08:25.359 align:middle line:84% So the orientation of the stimulus 01:08:25.359 --> 01:08:31.340 align:middle line:84% is confounded with the contrast in this case of an edge. 01:08:31.340 --> 01:08:33.500 align:middle line:84% So this is a really important idea. 01:08:33.500 --> 01:08:37.260 align:middle line:84% So this ambiguity between these different stimulus dimensions 01:08:37.260 --> 01:08:41.220 align:middle line:84% can be resolved by employing a population code. 01:08:41.220 --> 01:08:43.000 align:middle line:90% And so this is the idea. 01:08:43.000 --> 01:08:45.220 align:middle line:84% So now, instead of having one neuron 01:08:45.220 --> 01:08:47.260 align:middle line:84% that we're measuring the response for, 01:08:47.260 --> 01:08:48.920 align:middle line:90% we've got a whole population. 01:08:48.920 --> 01:08:51.060 align:middle line:84% The preferred orientations are indicated 01:08:51.060 --> 01:08:53.580 align:middle line:90% by these line segments. 01:08:53.580 --> 01:08:56.340 align:middle line:84% So this is the response across a population 01:08:56.340 --> 01:09:01.340 align:middle line:90% of neurons to this stimulus. 01:09:01.340 --> 01:09:03.420 align:middle line:84% This is the response across that same population 01:09:03.420 --> 01:09:05.654 align:middle line:90% of neurons to this stimulus. 01:09:05.654 --> 01:09:07.779 align:middle line:84% And here's the response across that same population 01:09:07.779 --> 01:09:10.600 align:middle line:90% of neurons to that stimulus. 01:09:10.600 --> 01:09:14.260 align:middle line:84% So what can you see is that as the contrast increases 01:09:14.260 --> 01:09:19.460 align:middle line:84% from here to here to here, the peak response kind of goes up. 01:09:19.460 --> 01:09:23.460 align:middle line:84% And consistent with what I told you, 01:09:23.460 --> 01:09:26.240 align:middle line:84% we've selected the orientations and the contrast 01:09:26.240 --> 01:09:28.760 align:middle line:84% here such that for this particular neuron, the one that 01:09:28.760 --> 01:09:31.399 align:middle line:84% has a preferred orientation of vertical, 01:09:31.399 --> 01:09:35.160 align:middle line:84% you get the same response to the three stimuli. 01:09:35.160 --> 01:09:41.279 align:middle line:84% However, you can also see-- and this intuitively makes sense-- 01:09:41.279 --> 01:09:45.040 align:middle line:84% that if you were somehow able to measure the peak response 01:09:45.040 --> 01:09:47.680 align:middle line:84% over the population, that would tell you 01:09:47.680 --> 01:09:50.439 align:middle line:84% what the orientation of the stimulus is. 01:09:50.439 --> 01:09:54.000 align:middle line:84% So here the largest response is given by the vertical neurons. 01:09:54.000 --> 01:09:56.360 align:middle line:84% Here the largest response is given to the ones that's 01:09:56.360 --> 01:09:57.700 align:middle line:90% a little bit off vertical. 01:09:57.700 --> 01:10:02.380 align:middle line:90% 01:10:02.380 --> 01:10:04.920 align:middle line:84% So the idea of a population code is 01:10:04.920 --> 01:10:08.080 align:middle line:84% that you have the ability to measure the responses 01:10:08.080 --> 01:10:12.340 align:middle line:84% from a population of neurons and perform some computation on that 01:10:12.340 --> 01:10:14.920 align:middle line:84% to estimate some quantity that you're interested in. 01:10:14.920 --> 01:10:17.840 align:middle line:84% And in this particular case, the computation 01:10:17.840 --> 01:10:22.240 align:middle line:84% would be to estimate the peak of the response 01:10:22.240 --> 01:10:25.220 align:middle line:84% across the population and then to estimate the orientation 01:10:25.220 --> 01:10:30.860 align:middle line:84% to correspond to the preferred orientation of that peak. 01:10:30.860 --> 01:10:35.020 align:middle line:84% So where does this end up being useful? 01:10:35.020 --> 01:10:39.460 align:middle line:84% One is to help us understand some funny things that can 01:10:39.460 --> 01:10:41.540 align:middle line:90% happen to our visual system. 01:10:41.540 --> 01:10:44.900 align:middle line:84% So this is going to be a demo of the tilt aftereffect. 01:10:44.900 --> 01:10:47.780 align:middle line:84% So we're going to temporarily change your brain. 01:10:47.780 --> 01:10:48.280 align:middle line:90% Don't worry. 01:10:48.280 --> 01:10:49.440 align:middle line:90% It will just be temporary. 01:10:49.440 --> 01:10:53.380 align:middle line:90% And you will then switch back. 01:10:53.380 --> 01:10:57.300 align:middle line:84% But what I want you to do for now is look at this stimulus. 01:10:57.300 --> 01:10:59.940 align:middle line:84% Everyone can tell if this is vertical. 01:10:59.940 --> 01:11:04.440 align:middle line:84% So what we're going to do is adapt you to this stimulus. 01:11:04.440 --> 01:11:06.660 align:middle line:84% And so what I mean by adapt is I'm 01:11:06.660 --> 01:11:10.140 align:middle line:84% going to ask you to stare at this for about a minute. 01:11:10.140 --> 01:11:12.400 align:middle line:84% So everyone start staring at this. 01:11:12.400 --> 01:11:16.040 align:middle line:84% And what I mean, I want you to look at the gray circle. 01:11:16.040 --> 01:11:20.620 align:middle line:84% And you can look anywhere inside the gray circle, starting now. 01:11:20.620 --> 01:11:25.240 align:middle line:84% So, as you are looking at that stimulus, what's happening 01:11:25.240 --> 01:11:30.200 align:middle line:84% is that there are neurons that are firing in your brain. 01:11:30.200 --> 01:11:33.020 align:middle line:84% And they're going to fire now, and they're going to fire later. 01:11:33.020 --> 01:11:37.960 align:middle line:84% But as they fire, they will adapt. 01:11:37.960 --> 01:11:42.320 align:middle line:84% And so what we think happens is that the response 01:11:42.320 --> 01:11:44.200 align:middle line:90% decreases over time. 01:11:44.200 --> 01:11:46.058 align:middle line:84% And then that will end up changing the way 01:11:46.058 --> 01:11:47.600 align:middle line:84% that things subsequently look, again, 01:11:47.600 --> 01:11:48.933 align:middle line:90% for a very brief period of time. 01:11:48.933 --> 01:11:49.540 align:middle line:90% Don't worry. 01:11:49.540 --> 01:11:52.542 align:middle line:84% So keep looking inside the gray circle. 01:11:52.542 --> 01:11:54.000 align:middle line:84% And it's actually kind of important 01:11:54.000 --> 01:11:56.220 align:middle line:84% that you move your eyes around inside the gray circle. 01:11:56.220 --> 01:11:58.140 align:middle line:84% Otherwise, there's some other funny stuff that will happen, 01:11:58.140 --> 01:11:59.348 align:middle line:90% which we can also talk about. 01:11:59.348 --> 01:12:03.720 align:middle line:84% So move your eyes around inside that little gray disk 01:12:03.720 --> 01:12:05.920 align:middle line:90% just a little bit longer. 01:12:05.920 --> 01:12:09.460 align:middle line:84% And then when I give you the go ahead, 01:12:09.460 --> 01:12:12.310 align:middle line:84% you're going to then move your eyes over to the test stimulus. 01:12:12.310 --> 01:12:14.560 align:middle line:84% And what you should observe is that the test stimulus, 01:12:14.560 --> 01:12:17.580 align:middle line:84% the part in the middle, will no longer look vertical. 01:12:17.580 --> 01:12:21.400 align:middle line:84% It should look tilted a little bit opposite to the test. 01:12:21.400 --> 01:12:22.520 align:middle line:90% Look over here. 01:12:22.520 --> 01:12:25.620 align:middle line:90% 01:12:25.620 --> 01:12:27.820 align:middle line:84% Does it look a little bit off of vertical? 01:12:27.820 --> 01:12:29.220 align:middle line:90% Yeah. 01:12:29.220 --> 01:12:31.200 align:middle line:90% That is the tilt aftereffect. 01:12:31.200 --> 01:12:36.760 align:middle line:84% So we adapted your brain to that leftward grading. 01:12:36.760 --> 01:12:39.300 align:middle line:84% And subsequent to that, a perfectly vertical grading 01:12:39.300 --> 01:12:41.200 align:middle line:90% looked a little bit rightward. 01:12:41.200 --> 01:12:44.120 align:middle line:90% 01:12:44.120 --> 01:12:44.880 align:middle line:90% It's temporary. 01:12:44.880 --> 01:12:47.560 align:middle line:84% So in another-- it's probably even gone now. 01:12:47.560 --> 01:12:50.320 align:middle line:84% But in another 20 seconds, it will be gone. 01:12:50.320 --> 01:12:53.220 align:middle line:90% 01:12:53.220 --> 01:12:55.858 align:middle line:84% And so like I said, the idea behind after effects 01:12:55.858 --> 01:12:57.900 align:middle line:84% is that there are thought to be due to a decrease 01:12:57.900 --> 01:13:01.120 align:middle line:84% in the responsiveness of neurons after prolonged activity. 01:13:01.120 --> 01:13:05.220 align:middle line:84% And it's unclear whether we should think about this 01:13:05.220 --> 01:13:06.740 align:middle line:90% as a feature or a bug. 01:13:06.740 --> 01:13:10.180 align:middle line:84% You can think of implementation level explanations 01:13:10.180 --> 01:13:12.240 align:middle line:84% of this, which is that neurons use glucose. 01:13:12.240 --> 01:13:14.280 align:middle line:84% And then when they fire for a very long time, 01:13:14.280 --> 01:13:17.180 align:middle line:84% the glucose kind of is expended, and there's not 01:13:17.180 --> 01:13:20.140 align:middle line:84% as much of that or some other metabolic resource. 01:13:20.140 --> 01:13:23.440 align:middle line:84% And thus, the response of the neuron decreases. 01:13:23.440 --> 01:13:27.240 align:middle line:84% Other people think that the adaptation actually 01:13:27.240 --> 01:13:30.560 align:middle line:84% is of functional importance, that it actually kind of helps 01:13:30.560 --> 01:13:33.040 align:middle line:84% your visual system get into some optimal state 01:13:33.040 --> 01:13:34.300 align:middle line:90% for the current environment. 01:13:34.300 --> 01:13:35.600 align:middle line:90% That's currently debated. 01:13:35.600 --> 01:13:37.140 align:middle line:90% But adaptation does happen. 01:13:37.140 --> 01:13:41.040 align:middle line:84% And what we will talk about when we resume next time 01:13:41.040 --> 01:13:44.520 align:middle line:84% is how we can explain what just happened 01:13:44.520 --> 01:13:48.120 align:middle line:84% to you in terms of adaptation and population codes, 01:13:48.120 --> 01:13:51.330 align:middle line:90% so to be continued. 01:13:51.330 --> 01:14:20.000 align:middle line:90%