WEBVTT 00:00:00.000 --> 00:00:01.934 align:middle line:90% [SQUEAKING] 00:00:01.934 --> 00:00:04.324 align:middle line:90% [RUSTLING] 00:00:04.324 --> 00:00:07.192 align:middle line:90% [CLICKING] 00:00:07.192 --> 00:00:09.018 align:middle line:90% 00:00:09.018 --> 00:00:10.560 align:middle line:84% JOSH MCDERMOTT: So today, we're going 00:00:10.560 --> 00:00:13.520 align:middle line:84% to finish talking about motion perception. 00:00:13.520 --> 00:00:16.490 align:middle line:84% So we got this started last time. 00:00:16.490 --> 00:00:18.240 align:middle line:84% So motion happens when things in the world 00:00:18.240 --> 00:00:20.120 align:middle line:90% change position over time. 00:00:20.120 --> 00:00:23.120 align:middle line:84% And the problem of motion perception 00:00:23.120 --> 00:00:27.460 align:middle line:84% is understanding how we see that motion. 00:00:27.460 --> 00:00:30.240 align:middle line:84% And so we talked about the evidence 00:00:30.240 --> 00:00:32.920 align:middle line:84% that there are motion detectors in the visual system in the way 00:00:32.920 --> 00:00:34.020 align:middle line:90% of the motion aftereffect. 00:00:34.020 --> 00:00:36.360 align:middle line:84% We experienced a motion after effect. 00:00:36.360 --> 00:00:38.800 align:middle line:84% And we talked about simple models 00:00:38.800 --> 00:00:41.423 align:middle line:84% for how you would construct such motion detectors. 00:00:41.423 --> 00:00:42.840 align:middle line:84% So the simplest and most intuitive 00:00:42.840 --> 00:00:45.200 align:middle line:84% is what's called the Reichardt detector, where there 00:00:45.200 --> 00:00:49.000 align:middle line:84% are spatially displaced inputs that 00:00:49.000 --> 00:00:53.980 align:middle line:84% arrive at a downstream neuron with different time delays, 00:00:53.980 --> 00:00:56.560 align:middle line:84% such that if there's something that's moving through space, 00:00:56.560 --> 00:01:01.070 align:middle line:84% such that the spatial offset matches the temporal offset, 00:01:01.070 --> 00:01:03.070 align:middle line:84% the inputs end up reaching the downstream neuron 00:01:03.070 --> 00:01:03.850 align:middle line:90% at the same time. 00:01:03.850 --> 00:01:05.750 align:middle line:84% And if the downstream neuron is set up 00:01:05.750 --> 00:01:07.350 align:middle line:84% as a coincidence detector, then you'll 00:01:07.350 --> 00:01:10.550 align:middle line:84% get a direction-selective response. 00:01:10.550 --> 00:01:13.790 align:middle line:84% We also looked at an actual direction-selective neuron 00:01:13.790 --> 00:01:16.590 align:middle line:84% and the receptive field that you can 00:01:16.590 --> 00:01:18.830 align:middle line:84% see if you measure the spike triggered 00:01:18.830 --> 00:01:19.950 align:middle line:90% average of such a neuron. 00:01:19.950 --> 00:01:23.710 align:middle line:84% I mean, that consists of, in V1, in this example, 00:01:23.710 --> 00:01:26.630 align:middle line:84% of these orientation selective receptive fields 00:01:26.630 --> 00:01:28.710 align:middle line:90% that kind of shift over time. 00:01:28.710 --> 00:01:31.750 align:middle line:84% And so then we talked about the idea 00:01:31.750 --> 00:01:33.790 align:middle line:84% that another way to think about motion detection 00:01:33.790 --> 00:01:36.470 align:middle line:84% is that you can think of motion as being orientation 00:01:36.470 --> 00:01:38.230 align:middle line:90% in space time. 00:01:38.230 --> 00:01:40.590 align:middle line:84% And so to detect motion you really 00:01:40.590 --> 00:01:43.910 align:middle line:84% need filters that are oriented in space time. 00:01:43.910 --> 00:01:47.430 align:middle line:84% So that's another way to think about this. 00:01:47.430 --> 00:01:51.790 align:middle line:84% And these receptive fields that are measured in actual neurons 00:01:51.790 --> 00:01:53.880 align:middle line:84% show this orientation in space time. 00:01:53.880 --> 00:01:55.630 align:middle line:84% So remember, one of these receptive fields 00:01:55.630 --> 00:01:57.050 align:middle line:90% is like a movie right. 00:01:57.050 --> 00:02:00.200 align:middle line:84% So you got the x and the y dimensions and the t dimensions. 00:02:00.200 --> 00:02:03.260 align:middle line:84% So it's like a bunch of image frames. 00:02:03.260 --> 00:02:05.700 align:middle line:84% But you can project those down to one spatial dimension, 00:02:05.700 --> 00:02:07.360 align:middle line:90% just so it's easy to look at. 00:02:07.360 --> 00:02:09.620 align:middle line:84% And then you see this orientation, in this case, 00:02:09.620 --> 00:02:12.280 align:middle line:90% in x and t. 00:02:12.280 --> 00:02:13.420 align:middle line:90% So then we talked about-- 00:02:13.420 --> 00:02:17.540 align:middle line:84% so this is evidence that these mechanisms 00:02:17.540 --> 00:02:21.100 align:middle line:84% in primary visual cortex for detecting image motion 00:02:21.100 --> 00:02:23.540 align:middle line:90% get oriented inputs. 00:02:23.540 --> 00:02:26.220 align:middle line:84% So you could kind of imagine of building this 00:02:26.220 --> 00:02:28.820 align:middle line:84% from a bunch of simple cell inputs. 00:02:28.820 --> 00:02:31.380 align:middle line:84% And the consequence of them being oriented 00:02:31.380 --> 00:02:36.500 align:middle line:84% is that there's an ambiguity in the motion that they signal. 00:02:36.500 --> 00:02:39.100 align:middle line:84% And so specifically, we talked about the idea 00:02:39.100 --> 00:02:42.080 align:middle line:84% that a motion detector that's set up like this, 00:02:42.080 --> 00:02:43.900 align:middle line:84% it can tell you how much something 00:02:43.900 --> 00:02:47.080 align:middle line:84% is moving, in this case, in the horizontal direction. 00:02:47.080 --> 00:02:49.300 align:middle line:84% It's not going to tell you how much it's moving 00:02:49.300 --> 00:02:50.940 align:middle line:90% in the vertical direction. 00:02:50.940 --> 00:02:53.820 align:middle line:84% So they're measuring a component of the velocity 00:02:53.820 --> 00:02:56.940 align:middle line:84% rather than the velocity in its entirety. 00:02:56.940 --> 00:02:59.420 align:middle line:84% And so the consequence of that is 00:02:59.420 --> 00:03:02.580 align:middle line:84% that if you observe a response in one of these local motion 00:03:02.580 --> 00:03:07.500 align:middle line:84% detectors, that response is giving you a constraint line 00:03:07.500 --> 00:03:09.400 align:middle line:90% in the space of velocities. 00:03:09.400 --> 00:03:12.260 align:middle line:84% So if you think of velocity as a vector quantity, 00:03:12.260 --> 00:03:13.800 align:middle line:90% so it's got two dimensions. 00:03:13.800 --> 00:03:17.100 align:middle line:84% There's an x component and a y component. 00:03:17.100 --> 00:03:21.680 align:middle line:84% And the constraint line means that the data that you observe, 00:03:21.680 --> 00:03:23.500 align:middle line:84% so the response that you observe, 00:03:23.500 --> 00:03:26.160 align:middle line:84% is consistent with each of these possible vectors. 00:03:26.160 --> 00:03:27.560 align:middle line:90% And what do these vectors share? 00:03:27.560 --> 00:03:29.380 align:middle line:84% They share the horizontal component 00:03:29.380 --> 00:03:31.100 align:middle line:90% in this particular example. 00:03:31.100 --> 00:03:35.680 align:middle line:84% But the vertical component is unconstrained. 00:03:35.680 --> 00:03:37.735 align:middle line:84% So each one of these local motion measurements 00:03:37.735 --> 00:03:39.360 align:middle line:84% is giving you information about motion. 00:03:39.360 --> 00:03:42.340 align:middle line:84% But it's not uniquely specifying the 2D motion. 00:03:42.340 --> 00:03:44.060 align:middle line:84% And then we talked about this idea 00:03:44.060 --> 00:03:46.500 align:middle line:84% that if you have multiple detectors that 00:03:46.500 --> 00:03:48.860 align:middle line:84% are tuned to different orientations, 00:03:48.860 --> 00:03:51.400 align:middle line:84% they will give you distinct constraint lines. 00:03:51.400 --> 00:03:53.260 align:middle line:84% And so if both of those detectors 00:03:53.260 --> 00:03:55.770 align:middle line:84% are stimulated by the same thing, 00:03:55.770 --> 00:03:58.090 align:middle line:84% then you can take the intersection of those constraint 00:03:58.090 --> 00:03:58.590 align:middle line:90% lines. 00:03:58.590 --> 00:04:01.450 align:middle line:84% And that would tell you the two-dimensional direction 00:04:01.450 --> 00:04:04.050 align:middle line:84% of motion that's happening in the world. 00:04:04.050 --> 00:04:10.770 align:middle line:84% So you can think about this in terms of neurons as taking 00:04:10.770 --> 00:04:13.790 align:middle line:84% a whole bunch of different local motion detectors, 00:04:13.790 --> 00:04:18.105 align:middle line:84% each one of which is oriented, and adding them all up, 00:04:18.105 --> 00:04:20.730 align:middle line:84% adding all of the ones up that are consistent with a particular 00:04:20.730 --> 00:04:22.230 align:middle line:90% 2D motion direction. 00:04:22.230 --> 00:04:26.030 align:middle line:84% So each one of these things has a particular constraint line. 00:04:26.030 --> 00:04:28.050 align:middle line:84% And so this particular combination 00:04:28.050 --> 00:04:30.130 align:middle line:84% here is all the ones that intersect 00:04:30.130 --> 00:04:31.790 align:middle line:90% at this particular point. 00:04:31.790 --> 00:04:34.650 align:middle line:84% So you could build something that is selective for that 00:04:34.650 --> 00:04:38.810 align:middle line:84% particular 2D velocity with this particular combination of these 00:04:38.810 --> 00:04:40.110 align:middle line:90% simple motion detectors. 00:04:40.110 --> 00:04:41.610 align:middle line:84% And if you wanted something that was 00:04:41.610 --> 00:04:43.830 align:middle line:84% tuned to some other velocity, like up here, 00:04:43.830 --> 00:04:47.690 align:middle line:84% that would give you a different combination. 00:04:47.690 --> 00:04:49.650 align:middle line:84% So this is kind of a theoretical idea, 00:04:49.650 --> 00:04:53.800 align:middle line:84% suggesting that there are these two stages of motion processing, 00:04:53.800 --> 00:04:56.480 align:middle line:84% an initial stage where things are decomposed into one 00:04:56.480 --> 00:05:00.360 align:middle line:84% dimensional components, and then a second stage when those 00:05:00.360 --> 00:05:03.900 align:middle line:84% are combined, potentially using this intersection of constraints 00:05:03.900 --> 00:05:06.580 align:middle line:90% idea that we talked about. 00:05:06.580 --> 00:05:09.680 align:middle line:84% And so then we began to talk about the evidence 00:05:09.680 --> 00:05:12.640 align:middle line:84% that these two stages that are postulated 00:05:12.640 --> 00:05:15.840 align:middle line:84% on theoretical grounds actually have a physiological basis 00:05:15.840 --> 00:05:17.560 align:middle line:90% in the visual system. 00:05:17.560 --> 00:05:20.920 align:middle line:84% And the story here is in this area 00:05:20.920 --> 00:05:23.800 align:middle line:84% called MT, which is a visual area that's kind of midway up 00:05:23.800 --> 00:05:24.940 align:middle line:90% the dorsal pathway. 00:05:24.940 --> 00:05:26.565 align:middle line:84% Remember, we think of the visual system 00:05:26.565 --> 00:05:28.760 align:middle line:84% as being organized into these two pathways. 00:05:28.760 --> 00:05:32.580 align:middle line:84% The ventral pathway mediates object recognition, 00:05:32.580 --> 00:05:35.500 align:middle line:84% the dorsal pathway also often called where pathway. 00:05:35.500 --> 00:05:39.840 align:middle line:84% But it's also where motion analysis seems to take place. 00:05:39.840 --> 00:05:44.080 align:middle line:84% And so this is area MT, just part of the dorsal pathway. 00:05:44.080 --> 00:05:47.200 align:middle line:84% And so we saw how in area MT, virtually all of the neurons 00:05:47.200 --> 00:05:49.540 align:middle line:84% are selective for the direction of motion. 00:05:49.540 --> 00:05:51.810 align:middle line:90% So they're tuned to direction. 00:05:51.810 --> 00:05:53.950 align:middle line:84% So that's one indication that it probably 00:05:53.950 --> 00:05:57.812 align:middle line:84% is important in the perception of motion. 00:05:57.812 --> 00:05:59.770 align:middle line:84% And I didn't show you direct evidence for this. 00:05:59.770 --> 00:06:02.830 align:middle line:84% But I asserted that if you lesion MT, 00:06:02.830 --> 00:06:06.270 align:middle line:84% you get deficits in motion perception. 00:06:06.270 --> 00:06:09.590 align:middle line:84% And so when we left off last time 00:06:09.590 --> 00:06:12.190 align:middle line:84% was by introducing the idea that you 00:06:12.190 --> 00:06:14.710 align:middle line:84% could study this problem using this stimulus, which 00:06:14.710 --> 00:06:16.190 align:middle line:90% is known as a plaid. 00:06:16.190 --> 00:06:19.950 align:middle line:84% So a plaid is a stimulus that is composed of two sine wave 00:06:19.950 --> 00:06:21.190 align:middle line:90% gratings. 00:06:21.190 --> 00:06:22.770 align:middle line:90% So here's one sine wave grating. 00:06:22.770 --> 00:06:23.770 align:middle line:90% And here's another. 00:06:23.770 --> 00:06:26.270 align:middle line:84% And you get the plaid by just adding them together. 00:06:26.270 --> 00:06:28.870 align:middle line:84% And so the idea behind the stimulus 00:06:28.870 --> 00:06:31.790 align:middle line:84% is that each of the sinusoidal components 00:06:31.790 --> 00:06:35.510 align:middle line:84% will stimulate a different elementary motion detector. 00:06:35.510 --> 00:06:38.590 align:middle line:84% And then when the gratings are superimposed, 00:06:38.590 --> 00:06:42.310 align:middle line:84% the question is whether or not the outputs of those two motion 00:06:42.310 --> 00:06:45.830 align:middle line:84% detectors gets combined, and if so, where in the visual system. 00:06:45.830 --> 00:06:50.900 align:middle line:84% So this is what each of those looks like on its own. 00:06:50.900 --> 00:06:52.400 align:middle line:84% And then when you add them together, 00:06:52.400 --> 00:06:56.060 align:middle line:84% the interesting thing is that you see a single pattern that 00:06:56.060 --> 00:06:59.500 align:middle line:84% moves in a direction that's different from the direction 00:06:59.500 --> 00:07:03.315 align:middle line:84% that you see with either of the components on its own. 00:07:03.315 --> 00:07:05.440 align:middle line:84% AUDIENCE: Gratings are a little bit more different. 00:07:05.440 --> 00:07:07.500 align:middle line:84% Like, for example, if you return colors, 00:07:07.500 --> 00:07:12.245 align:middle line:84% [INAUDIBLE], and I guess what might explain those differences. 00:07:12.245 --> 00:07:14.120 align:middle line:84% JOSH MCDERMOTT: Yeah, that's a good question. 00:07:14.120 --> 00:07:18.800 align:middle line:84% So the extent to which the two gratings cohere, 00:07:18.800 --> 00:07:20.700 align:middle line:84% so look like a single pattern, will 00:07:20.700 --> 00:07:22.860 align:middle line:84% depend on the property of those gratings. 00:07:22.860 --> 00:07:25.738 align:middle line:84% So for instance, if the spatial frequency of the gratings 00:07:25.738 --> 00:07:27.780 align:middle line:84% is more different, so if one of them is very low, 00:07:27.780 --> 00:07:29.300 align:middle line:84% and one of them is very high, you'll 00:07:29.300 --> 00:07:32.940 align:middle line:84% be less likely to see the pattern direction of motion. 00:07:32.940 --> 00:07:35.380 align:middle line:84% I don't actually know whether differences in color 00:07:35.380 --> 00:07:37.220 align:middle line:90% will have an effect. 00:07:37.220 --> 00:07:38.980 align:middle line:84% But other things that you might think 00:07:38.980 --> 00:07:44.660 align:middle line:84% would be related to the extent to which those belong together 00:07:44.660 --> 00:07:46.540 align:middle line:90% do affect this. 00:07:46.540 --> 00:07:48.900 align:middle line:84% So it's not like there's some obligatory integration 00:07:48.900 --> 00:07:52.000 align:middle line:84% and it's affected by principles of perceptual organization. 00:07:52.000 --> 00:07:54.960 align:middle line:90% 00:07:54.960 --> 00:07:55.780 align:middle line:90% So yeah. 00:07:55.780 --> 00:08:01.700 align:middle line:84% AUDIENCE: Is there a need for this [INAUDIBLE]? 00:08:01.700 --> 00:08:03.500 align:middle line:90% JOSH MCDERMOTT: Plaid motion. 00:08:03.500 --> 00:08:04.960 align:middle line:90% Yeah. 00:08:04.960 --> 00:08:06.580 align:middle line:84% So this is a very famous stimulus 00:08:06.580 --> 00:08:09.620 align:middle line:84% in the history of vision science. 00:08:09.620 --> 00:08:13.740 align:middle line:84% So the point is that this is a stimulus. 00:08:13.740 --> 00:08:16.940 align:middle line:84% Perceptually, you see it as moving in this direction 00:08:16.940 --> 00:08:17.980 align:middle line:90% of the pattern. 00:08:17.980 --> 00:08:21.560 align:middle line:84% But it's composed of these two components. 00:08:21.560 --> 00:08:23.860 align:middle line:84% And the idea is that those components 00:08:23.860 --> 00:08:27.140 align:middle line:84% are what elementary motion detectors are 00:08:27.140 --> 00:08:30.300 align:middle line:84% going to see because they're tuned to orientation. 00:08:30.300 --> 00:08:33.700 align:middle line:84% And so the question, is there a place in the visual system 00:08:33.700 --> 00:08:36.940 align:middle line:84% where the outputs of these elementary motion detectors 00:08:36.940 --> 00:08:41.559 align:middle line:84% would get combined and respond to the pattern direction? 00:08:41.559 --> 00:08:45.970 align:middle line:84% So just to confirm the intuition or to illustrate the intuition 00:08:45.970 --> 00:08:48.690 align:middle line:84% a little bit more, so we've got this stimulus here 00:08:48.690 --> 00:08:51.050 align:middle line:84% that consists of these two components. 00:08:51.050 --> 00:08:52.730 align:middle line:90% And it creates a plaid. 00:08:52.730 --> 00:08:56.090 align:middle line:84% And so if you have a neuron that is 00:08:56.090 --> 00:08:57.870 align:middle line:84% selective for the direction of motion, 00:08:57.870 --> 00:09:00.550 align:middle line:84% you might, for instance, see tuning like this. 00:09:00.550 --> 00:09:02.670 align:middle line:84% So this is a tuning function in directions. 00:09:02.670 --> 00:09:04.050 align:middle line:90% This is polar coordinates. 00:09:04.050 --> 00:09:06.970 align:middle line:84% So the direction here is indicated by the angle. 00:09:06.970 --> 00:09:10.610 align:middle line:84% And then this is a plot of the response of the neuron 00:09:10.610 --> 00:09:12.270 align:middle line:90% at each direction of motion. 00:09:12.270 --> 00:09:14.850 align:middle line:84% And so the fact that this is way out here for this direction 00:09:14.850 --> 00:09:16.450 align:middle line:84% is like an indication this neuron is 00:09:16.450 --> 00:09:19.970 align:middle line:84% tuned for horizontal motion that is to the right. 00:09:19.970 --> 00:09:23.170 align:middle line:84% So that's a directional tuning curve. 00:09:23.170 --> 00:09:25.750 align:middle line:84% And this is what you would measure with a single grating. 00:09:25.750 --> 00:09:27.770 align:middle line:84% That's why it's called the grating response. 00:09:27.770 --> 00:09:30.650 align:middle line:84% And so there are two possibilities 00:09:30.650 --> 00:09:33.750 align:middle line:84% for what you might get if you show this, a plaid, 00:09:33.750 --> 00:09:35.570 align:middle line:84% should you show this neuron a plaid that's 00:09:35.570 --> 00:09:36.950 align:middle line:90% composed of these two gratings. 00:09:36.950 --> 00:09:39.810 align:middle line:84% So one possibility is that the neuron 00:09:39.810 --> 00:09:44.070 align:middle line:84% is going to respond to each of the gratings individually, 00:09:44.070 --> 00:09:48.060 align:middle line:84% in which case, there will be this bilobed response function, 00:09:48.060 --> 00:09:50.840 align:middle line:84% because what happens is that you rotate the plaid around, 00:09:50.840 --> 00:09:52.980 align:middle line:84% and you move it in every possible direction. 00:09:52.980 --> 00:09:54.560 align:middle line:84% And then there will be two directions 00:09:54.560 --> 00:09:57.120 align:middle line:84% in which one of the gratings is aligned 00:09:57.120 --> 00:10:00.920 align:middle line:90% to the tuning of the neuron. 00:10:00.920 --> 00:10:03.040 align:middle line:84% So this is what you would expect, like, say, 00:10:03.040 --> 00:10:07.320 align:middle line:84% in area of V1, that there's two directions where 00:10:07.320 --> 00:10:09.513 align:middle line:84% you get a big response, where one of the gratings 00:10:09.513 --> 00:10:11.180 align:middle line:84% is aligned with the preferred direction, 00:10:11.180 --> 00:10:14.080 align:middle line:90% or where the other one is. 00:10:14.080 --> 00:10:16.480 align:middle line:84% But the other possibility is that the neuron 00:10:16.480 --> 00:10:20.000 align:middle line:84% might be responsive to the plaid direction, so the direction 00:10:20.000 --> 00:10:22.100 align:middle line:84% that you actually see when you look at this. 00:10:22.100 --> 00:10:23.517 align:middle line:84% And if that was the case, then you 00:10:23.517 --> 00:10:25.360 align:middle line:84% would get a single-lobed response 00:10:25.360 --> 00:10:27.880 align:middle line:84% that would be aligned to the direction of the tuning 00:10:27.880 --> 00:10:31.380 align:middle line:84% that you would get with a single grating. 00:10:31.380 --> 00:10:34.380 align:middle line:84% So those are the two idealized possibilities. 00:10:34.380 --> 00:10:36.000 align:middle line:84% And the question is, what do you see? 00:10:36.000 --> 00:10:41.240 align:middle line:84% So in area V1, you almost exclusively 00:10:41.240 --> 00:10:44.870 align:middle line:84% see neurons that seem to be responsive to the individual 00:10:44.870 --> 00:10:46.270 align:middle line:90% gratings of the plaid. 00:10:46.270 --> 00:10:50.170 align:middle line:84% So in the way this is evaluated with that same experiment, 00:10:50.170 --> 00:10:53.630 align:middle line:84% so you measure the directional tuning with a single grading. 00:10:53.630 --> 00:10:56.950 align:middle line:84% So this is a neuron that prefers this kind of direction that's 00:10:56.950 --> 00:10:58.670 align:middle line:90% downward and to the right. 00:10:58.670 --> 00:11:01.185 align:middle line:84% And then you measure the direction tuning with plaids. 00:11:01.185 --> 00:11:03.310 align:middle line:84% And the directional tuning that you get with plaids 00:11:03.310 --> 00:11:04.790 align:middle line:90% has these two lobes. 00:11:04.790 --> 00:11:07.865 align:middle line:84% And one of these is the actual response. 00:11:07.865 --> 00:11:09.490 align:middle line:84% I think the solid line is the response. 00:11:09.490 --> 00:11:13.150 align:middle line:84% And the dashed line is the prediction of the response, 00:11:13.150 --> 00:11:15.830 align:middle line:84% just from the response to gratings under the assumption 00:11:15.830 --> 00:11:17.770 align:middle line:84% that it just responds to the gratings. 00:11:17.770 --> 00:11:20.530 align:middle line:84% So again, the idea is that these neurons in V1, 00:11:20.530 --> 00:11:22.750 align:middle line:84% they're orientation selective filters. 00:11:22.750 --> 00:11:25.630 align:middle line:84% And so they essentially just see one orientation component 00:11:25.630 --> 00:11:30.270 align:middle line:84% at a time when one of them aligns with the receptive field. 00:11:30.270 --> 00:11:34.630 align:middle line:84% And so this can be quantified by measuring 00:11:34.630 --> 00:11:40.300 align:middle line:84% the correlation of the observed response function with these two 00:11:40.300 --> 00:11:43.860 align:middle line:84% predictions here, with the prediction of what you would 00:11:43.860 --> 00:11:47.583 align:middle line:84% expect if the neuron was just responding to the components, 00:11:47.583 --> 00:11:49.500 align:middle line:84% and by contrast, the prediction if it was just 00:11:49.500 --> 00:11:50.603 align:middle line:90% responding to the plaid. 00:11:50.603 --> 00:11:53.020 align:middle line:84% So you get the correlation between the actual tuning curve 00:11:53.020 --> 00:11:54.580 align:middle line:90% and these two predictions. 00:11:54.580 --> 00:11:57.545 align:middle line:84% And you can plot that in this plane here. 00:11:57.545 --> 00:11:59.920 align:middle line:84% So this is the correlation with the component prediction. 00:11:59.920 --> 00:12:02.283 align:middle line:84% This is the correlation with the pattern prediction. 00:12:02.283 --> 00:12:04.700 align:middle line:84% And so the idea is that if this is a neuron that is really 00:12:04.700 --> 00:12:06.460 align:middle line:84% just responding to the components, 00:12:06.460 --> 00:12:08.380 align:middle line:84% then it's going to be in this region of space 00:12:08.380 --> 00:12:10.755 align:middle line:84% because it should be highly correlated with the component 00:12:10.755 --> 00:12:13.020 align:middle line:84% prediction, whereas, if it's responding to the pattern 00:12:13.020 --> 00:12:14.980 align:middle line:84% direction, it should be kind of up here. 00:12:14.980 --> 00:12:19.660 align:middle line:84% And so this is a plot of the results of doing this analysis 00:12:19.660 --> 00:12:21.520 align:middle line:90% on neurons in area V1. 00:12:21.520 --> 00:12:24.380 align:middle line:90% So each dot here is a neuron. 00:12:24.380 --> 00:12:27.340 align:middle line:84% And what you can see is that pretty much all of the dots 00:12:27.340 --> 00:12:29.220 align:middle line:90% are in this region here. 00:12:29.220 --> 00:12:31.780 align:middle line:84% And what that means is that the directional tuning 00:12:31.780 --> 00:12:37.980 align:middle line:84% curves of V1 neurons to plaids have this bilobed shape that 00:12:37.980 --> 00:12:40.780 align:middle line:84% resembles the prediction if they were just 00:12:40.780 --> 00:12:43.440 align:middle line:90% responding to the components. 00:12:43.440 --> 00:12:45.900 align:middle line:84% So the point is that V1 is doing what you would expect 00:12:45.900 --> 00:12:51.220 align:middle line:84% and not doing something super Interesting. 00:12:51.220 --> 00:12:53.520 align:middle line:84% Now the question is, what happens downstream? 00:12:53.520 --> 00:12:57.260 align:middle line:90% And so this area MT. 00:12:57.260 --> 00:13:01.680 align:middle line:84% And so some of the neurons in MT look just like neurons in V1. 00:13:01.680 --> 00:13:02.573 align:middle line:90% So here's an example. 00:13:02.573 --> 00:13:04.240 align:middle line:84% So this is the tuning curve to gratings. 00:13:04.240 --> 00:13:05.865 align:middle line:84% And this is the tuning curve to plaids. 00:13:05.865 --> 00:13:09.980 align:middle line:84% And you get again this bilobed tuning response function. 00:13:09.980 --> 00:13:13.580 align:middle line:84% But here's a case where the direction tuning to plaids 00:13:13.580 --> 00:13:15.815 align:middle line:84% looks a lot like the direction tuning to gratings, 00:13:15.815 --> 00:13:17.940 align:middle line:84% and in fact, deviates quite significantly from what 00:13:17.940 --> 00:13:19.940 align:middle line:84% would be predicted if it was just 00:13:19.940 --> 00:13:21.780 align:middle line:90% responding to the components. 00:13:21.780 --> 00:13:24.940 align:middle line:84% So this is an example of a neuron that 00:13:24.940 --> 00:13:27.140 align:middle line:84% is tuned to the direction of motion 00:13:27.140 --> 00:13:29.460 align:middle line:84% that you see when you look at that pattern. 00:13:29.460 --> 00:13:33.380 align:middle line:90% 00:13:33.380 --> 00:13:35.220 align:middle line:90% So this is just two examples. 00:13:35.220 --> 00:13:38.570 align:middle line:84% This is the result of this analysis, where 00:13:38.570 --> 00:13:40.250 align:middle line:90% every dot, again, is a neuron. 00:13:40.250 --> 00:13:44.690 align:middle line:84% And although you can see, some of the neurons 00:13:44.690 --> 00:13:47.302 align:middle line:84% here are in this lower region, indicating 00:13:47.302 --> 00:13:49.510 align:middle line:84% that they're responsive to the individual components, 00:13:49.510 --> 00:13:51.010 align:middle line:90% just like in AV1. 00:13:51.010 --> 00:13:53.528 align:middle line:84% There's a bunch that are up here in this top region, 00:13:53.528 --> 00:13:55.570 align:middle line:84% indicating that they're responsive to the pattern 00:13:55.570 --> 00:13:56.390 align:middle line:90% direction. 00:13:56.390 --> 00:14:01.130 align:middle line:90% 00:14:01.130 --> 00:14:04.490 align:middle line:84% So this is evidence for this two-stage model 00:14:04.490 --> 00:14:05.590 align:middle line:90% of motion perception. 00:14:05.590 --> 00:14:09.170 align:middle line:84% There's this initial stage, where you measure responses 00:14:09.170 --> 00:14:12.090 align:middle line:90% to 1D components. 00:14:12.090 --> 00:14:16.650 align:middle line:84% Those then seem to provide inputs to the second stage, 00:14:16.650 --> 00:14:19.410 align:middle line:84% potentially located in area MT, where 00:14:19.410 --> 00:14:21.330 align:middle line:90% those responses get combined. 00:14:21.330 --> 00:14:24.050 align:middle line:84% And you end up with neurons that are responsive to the pattern 00:14:24.050 --> 00:14:24.790 align:middle line:90% direction. 00:14:24.790 --> 00:14:27.780 align:middle line:90% 00:14:27.780 --> 00:14:31.210 align:middle line:84% So as I mentioned last time, this 00:14:31.210 --> 00:14:35.680 align:middle line:84% was a fairly influential story in the history 00:14:35.680 --> 00:14:40.680 align:middle line:84% of systems neuroscience, sensory neuroscience, in that it 00:14:40.680 --> 00:14:42.520 align:middle line:84% was a nice example where there was 00:14:42.520 --> 00:14:45.840 align:middle line:84% convergence between both perception-- 00:14:45.840 --> 00:14:49.720 align:middle line:84% so this stimulus of plaids was introduced and shown 00:14:49.720 --> 00:14:52.020 align:middle line:84% to exhibit these interesting properties, 00:14:52.020 --> 00:14:55.200 align:middle line:84% these theoretical ideas for how you would measure motion-- 00:14:55.200 --> 00:14:57.160 align:middle line:84% and then some experimental evidence 00:14:57.160 --> 00:14:59.560 align:middle line:84% where different stages of the brain 00:14:59.560 --> 00:15:02.960 align:middle line:84% kind of mapped onto these different theoretically 00:15:02.960 --> 00:15:08.560 align:middle line:84% motivated computational stages in a pretty clear way, so 00:15:08.560 --> 00:15:14.880 align:middle line:84% a classic success story in systems neuroscience. 00:15:14.880 --> 00:15:18.240 align:middle line:84% Any questions about how this works? 00:15:18.240 --> 00:15:19.240 align:middle line:90% Yeah. 00:15:19.240 --> 00:15:24.880 align:middle line:84% AUDIENCE: So if the plaid motion response neurons are only 00:15:24.880 --> 00:15:30.320 align:middle line:84% seen half of the grading, will they respond partially or not 00:15:30.320 --> 00:15:31.775 align:middle line:90% at all? 00:15:31.775 --> 00:15:33.400 align:middle line:84% JOSH MCDERMOTT: By half of the grating, 00:15:33.400 --> 00:15:35.010 align:middle line:90% do you mean by only one grading? 00:15:35.010 --> 00:15:36.150 align:middle line:90% AUDIENCE: Yeah. 00:15:36.150 --> 00:15:39.870 align:middle line:84% JOSH MCDERMOTT: Well, that's what's shown here. 00:15:39.870 --> 00:15:44.750 align:middle line:84% So this first column is the tuning function if they're only 00:15:44.750 --> 00:15:48.430 align:middle line:90% presented with one grading. 00:15:48.430 --> 00:15:51.050 align:middle line:84% And so they respond to the single grading. 00:15:51.050 --> 00:15:57.270 align:middle line:84% The key thing that they do is that the direction tuning 00:15:57.270 --> 00:16:01.150 align:middle line:84% to a single grading is the same as the direction tuning 00:16:01.150 --> 00:16:05.310 align:middle line:84% to the plaid, even though when the plaid is moving 00:16:05.310 --> 00:16:08.670 align:middle line:84% in this direction, which is the direction that it responds 00:16:08.670 --> 00:16:11.750 align:middle line:84% a lot to when it gets a grating, the individual gratings 00:16:11.750 --> 00:16:14.690 align:middle line:84% of the plaid, one of them is moving this direction, 00:16:14.690 --> 00:16:16.870 align:middle line:84% and one of them is moving in this direction. 00:16:16.870 --> 00:16:18.990 align:middle line:84% So that's what's kind of remarkable 00:16:18.990 --> 00:16:24.110 align:middle line:84% about it is that it's doing this integrative operation that's 00:16:24.110 --> 00:16:25.510 align:middle line:90% nonlinear. 00:16:25.510 --> 00:16:28.530 align:middle line:84% And that gives you this emergent property. 00:16:28.530 --> 00:16:33.460 align:middle line:90% 00:16:33.460 --> 00:16:36.980 align:middle line:84% All right, so story so far is that we've 00:16:36.980 --> 00:16:40.620 align:middle line:84% got this two-stage process for detecting 00:16:40.620 --> 00:16:42.980 align:middle line:90% the direction of motion. 00:16:42.980 --> 00:16:46.140 align:middle line:84% There's this area called MT, where neurons are able to detect 00:16:46.140 --> 00:16:47.180 align:middle line:90% 2D motion. 00:16:47.180 --> 00:16:48.860 align:middle line:84% And what we're going to talk about now 00:16:48.860 --> 00:16:51.760 align:middle line:84% is this additional challenge that is present. 00:16:51.760 --> 00:16:54.100 align:middle line:84% And so previously, we've been of talking 00:16:54.100 --> 00:16:56.480 align:middle line:84% about this challenge that happened in some sense 00:16:56.480 --> 00:16:58.480 align:middle line:84% because of the way the visual system was set up. 00:16:58.480 --> 00:17:00.620 align:middle line:84% So you start out making these motion measurements 00:17:00.620 --> 00:17:01.540 align:middle line:90% that are oriented. 00:17:01.540 --> 00:17:03.540 align:middle line:84% And then that has this consequence that you have 00:17:03.540 --> 00:17:08.420 align:middle line:84% to combine across orientations in order to get 2D motion 00:17:08.420 --> 00:17:09.260 align:middle line:90% signals. 00:17:09.260 --> 00:17:11.819 align:middle line:84% So there's this other problem, though, 00:17:11.819 --> 00:17:13.540 align:middle line:84% which is that there's an ambiguity that 00:17:13.540 --> 00:17:15.940 align:middle line:90% exists in the world. 00:17:15.940 --> 00:17:18.550 align:middle line:84% And so the issue here is that neurons typically 00:17:18.550 --> 00:17:19.800 align:middle line:90% are making local measurements. 00:17:19.800 --> 00:17:22.339 align:middle line:84% We talked about this idea that the neuron looks at the world 00:17:22.339 --> 00:17:25.319 align:middle line:84% through a receptive field that is restricted in an area. 00:17:25.319 --> 00:17:28.560 align:middle line:84% And on your homework, on your problem sets, 00:17:28.560 --> 00:17:31.300 align:middle line:84% you've done this exercise of looking at images 00:17:31.300 --> 00:17:34.500 align:middle line:84% through apertures and verified how things are frequently 00:17:34.500 --> 00:17:37.480 align:middle line:84% pretty ambiguous when you just have local evidence. 00:17:37.480 --> 00:17:41.420 align:middle line:84% And so another type of this ambiguity is due to the fact 00:17:41.420 --> 00:17:43.680 align:middle line:84% that many things in the world, in particular, 00:17:43.680 --> 00:17:47.060 align:middle line:84% the edges of things, are locally one dimensional. 00:17:47.060 --> 00:17:51.820 align:middle line:84% And as a consequence, they're inherently ambiguous, 00:17:51.820 --> 00:17:53.640 align:middle line:84% their motion is inherently ambiguous, 00:17:53.640 --> 00:17:55.440 align:middle line:84% independent of how you measure it. 00:17:55.440 --> 00:17:57.800 align:middle line:84% And this is known as the aperture problem. 00:17:57.800 --> 00:17:59.400 align:middle line:90% So here's the idea. 00:17:59.400 --> 00:18:03.180 align:middle line:84% So we have an edge here at time 1. 00:18:03.180 --> 00:18:05.400 align:middle line:90% And the edge moves. 00:18:05.400 --> 00:18:08.620 align:middle line:84% So this is where the edge is in the image at time 2. 00:18:08.620 --> 00:18:11.900 align:middle line:84% But if you're viewing it through this aperture, which 00:18:11.900 --> 00:18:15.100 align:middle line:84% could be like a receptive field, again, 00:18:15.100 --> 00:18:17.500 align:middle line:84% all that you can measure in principle 00:18:17.500 --> 00:18:19.900 align:middle line:84% is the component of the velocity that 00:18:19.900 --> 00:18:22.160 align:middle line:84% is perpendicular to the orientation. 00:18:22.160 --> 00:18:25.460 align:middle line:84% And just due to the geometry here, 00:18:25.460 --> 00:18:28.660 align:middle line:84% you don't really know what the component is that's parallel 00:18:28.660 --> 00:18:30.130 align:middle line:90% to the orientation. 00:18:30.130 --> 00:18:32.690 align:middle line:84% And so the consequence is that the edge 00:18:32.690 --> 00:18:36.290 align:middle line:84% could be moving in any of these particular directions. 00:18:36.290 --> 00:18:41.410 align:middle line:84% So this is called the aperture problem. 00:18:41.410 --> 00:18:44.290 align:middle line:84% So here's just an example of this. 00:18:44.290 --> 00:18:47.650 align:middle line:84% So these three motions, they're all different. 00:18:47.650 --> 00:18:52.130 align:middle line:84% But they look the same when you view them through the aperture. 00:18:52.130 --> 00:18:55.210 align:middle line:84% And I can show you some actual examples 00:18:55.210 --> 00:18:59.370 align:middle line:90% of this that involve motion. 00:18:59.370 --> 00:19:01.370 align:middle line:90% Oh, no. 00:19:01.370 --> 00:19:02.930 align:middle line:90% This happened before. 00:19:02.930 --> 00:19:04.430 align:middle line:84% Do you remember this from last year? 00:19:04.430 --> 00:19:05.210 align:middle line:90% What did I-- 00:19:05.210 --> 00:19:07.368 align:middle line:84% AUDIENCE: It's a flash drive plug-in [INAUDIBLE]. 00:19:07.368 --> 00:19:08.910 align:middle line:84% JOSH MCDERMOTT: Yeah, no, this is it. 00:19:08.910 --> 00:19:10.650 align:middle line:90% But it's just that it's-- 00:19:10.650 --> 00:19:11.390 align:middle line:90% oh, there we go. 00:19:11.390 --> 00:19:19.930 align:middle line:84% So this is a demo of a simple scene with some moving objects 00:19:19.930 --> 00:19:22.110 align:middle line:84% that you can view through different apertures. 00:19:22.110 --> 00:19:23.657 align:middle line:90% So this is one aperture. 00:19:23.657 --> 00:19:25.490 align:middle line:84% You're looking at this thing, and it kind of 00:19:25.490 --> 00:19:30.160 align:middle line:84% looks like it's moving diagonally. 00:19:30.160 --> 00:19:33.060 align:middle line:84% And where's my-- there's my mouse. 00:19:33.060 --> 00:19:36.308 align:middle line:90% 00:19:36.308 --> 00:19:42.720 align:middle line:84% So this is what is actually there. 00:19:42.720 --> 00:19:45.980 align:middle line:84% So everything here is moving horizontally. 00:19:45.980 --> 00:19:49.160 align:middle line:90% 00:19:49.160 --> 00:19:52.060 align:middle line:90% But the edges are ambiguous. 00:19:52.060 --> 00:19:54.000 align:middle line:84% Now not everything is ambiguous locally. 00:19:54.000 --> 00:19:58.320 align:middle line:84% So there are certain features that are two dimensional, which 00:19:58.320 --> 00:20:00.680 align:middle line:90% are unambiguous. 00:20:00.680 --> 00:20:03.820 align:middle line:84% But some things are highly ambiguous. 00:20:03.820 --> 00:20:06.640 align:middle line:90% And that's the aperture problem. 00:20:06.640 --> 00:20:11.280 align:middle line:84% So given this issue, that edge motions 00:20:11.280 --> 00:20:13.800 align:middle line:84% are ambiguous, how does the visual system determine 00:20:13.800 --> 00:20:14.460 align:middle line:90% their velocity? 00:20:14.460 --> 00:20:17.480 align:middle line:84% And so there are two main answers that we'll talk about. 00:20:17.480 --> 00:20:21.100 align:middle line:84% The first is to make use of unambiguous 2D signals, 00:20:21.100 --> 00:20:24.000 align:middle line:84% so like the corner that you saw there to generate an unambiguous 00:20:24.000 --> 00:20:24.980 align:middle line:90% solution. 00:20:24.980 --> 00:20:28.825 align:middle line:84% And the second one is to combine edge motions across space. 00:20:28.825 --> 00:20:31.450 align:middle line:84% So if you have different edges that are different orientations, 00:20:31.450 --> 00:20:34.070 align:middle line:84% you can combine those and resolve the ambiguity. 00:20:34.070 --> 00:20:36.030 align:middle line:84% So each edge is ambiguous on its own. 00:20:36.030 --> 00:20:39.350 align:middle line:84% But together, they uniquely determine a velocity. 00:20:39.350 --> 00:20:43.630 align:middle line:84% So this is a really pretty cool example of a real-life 00:20:43.630 --> 00:20:46.670 align:middle line:84% consequence of the aperture problem and our reliance on 2D 00:20:46.670 --> 00:20:47.790 align:middle line:90% features. 00:20:47.790 --> 00:20:52.990 align:middle line:84% So I think this is at a football stadium in Spain 00:20:52.990 --> 00:20:54.890 align:middle line:84% or something, something in Europe. 00:20:54.890 --> 00:20:58.470 align:middle line:84% And these are exit ramps to get out of the stadium. 00:20:58.470 --> 00:21:01.850 align:middle line:84% And so they're just these big long ramps that are spirals. 00:21:01.850 --> 00:21:05.670 align:middle line:84% And so people just walk down the ramps. 00:21:05.670 --> 00:21:09.410 align:middle line:84% But because of the aperture problem, 00:21:09.410 --> 00:21:12.230 align:middle line:84% the motion of these edges is ambiguous. 00:21:12.230 --> 00:21:14.230 align:middle line:84% And it gets captured by the motion 00:21:14.230 --> 00:21:15.890 align:middle line:90% of the people who are walking. 00:21:15.890 --> 00:21:20.830 align:middle line:84% And so the whole thing looks like it's spiraling. 00:21:20.830 --> 00:21:22.470 align:middle line:84% So what's actually happening there 00:21:22.470 --> 00:21:27.238 align:middle line:84% is that this huge gazillion ton structure in the world that, 00:21:27.238 --> 00:21:28.280 align:middle line:90% of course, is not moving. 00:21:28.280 --> 00:21:30.340 align:middle line:90% The people are just walking. 00:21:30.340 --> 00:21:33.140 align:middle line:84% But because of the smoothness of the edge and the aperture 00:21:33.140 --> 00:21:37.355 align:middle line:84% problem, the 2D motions of the people capture the whole thing. 00:21:37.355 --> 00:21:38.980 align:middle line:84% And your visual system ends up thinking 00:21:38.980 --> 00:21:43.260 align:middle line:84% the thing is spiraling down into the ground or something. 00:21:43.260 --> 00:21:45.520 align:middle line:90% So quite amazing. 00:21:45.520 --> 00:21:49.020 align:middle line:90% 00:21:49.020 --> 00:21:52.480 align:middle line:84% Now one of the-- so we often rely on these 2D signals. 00:21:52.480 --> 00:21:55.660 align:middle line:84% Now another challenge that comes up as a consequence of this is 00:21:55.660 --> 00:21:59.020 align:middle line:84% that not all of the 2D motion signals that are present 00:21:59.020 --> 00:22:02.250 align:middle line:84% in images are real motion signals in the sense that they 00:22:02.250 --> 00:22:04.500 align:middle line:84% correspond to an object that's actually moving in that 00:22:04.500 --> 00:22:05.700 align:middle line:90% direction. 00:22:05.700 --> 00:22:09.740 align:middle line:84% And one of the main causes of this is that you can get lots 00:22:09.740 --> 00:22:12.540 align:middle line:84% of 2D motion signals from occlusion. 00:22:12.540 --> 00:22:15.920 align:middle line:84% So this is the same situation where we have two squares. 00:22:15.920 --> 00:22:17.540 align:middle line:90% They're moving horizontally. 00:22:17.540 --> 00:22:21.730 align:middle line:84% But the intersection, the place where one occludes includes, 00:22:21.730 --> 00:22:24.690 align:middle line:84% another will actually be moving up or down. 00:22:24.690 --> 00:22:27.690 align:middle line:84% And so you can see this in that demo 00:22:27.690 --> 00:22:29.390 align:middle line:84% that I was just trying to show you. 00:22:29.390 --> 00:22:35.170 align:middle line:90% 00:22:35.170 --> 00:22:39.230 align:middle line:90% So yeah. 00:22:39.230 --> 00:22:40.930 align:middle line:84% So this thing looks like something 00:22:40.930 --> 00:22:43.410 align:middle line:90% that's moving up and down. 00:22:43.410 --> 00:22:47.370 align:middle line:90% But it's actually just this. 00:22:47.370 --> 00:22:49.910 align:middle line:84% So anytime things occlude one another and move, 00:22:49.910 --> 00:22:53.730 align:middle line:90% this tends to happen. 00:22:53.730 --> 00:22:57.890 align:middle line:84% And so you might expect that in order 00:22:57.890 --> 00:23:03.930 align:middle line:84% to see things moving correctly, then the process of determining 00:23:03.930 --> 00:23:08.050 align:middle line:84% or inferring motion would need to be sensitive to occlusion, 00:23:08.050 --> 00:23:11.558 align:middle line:84% the fact that sometimes one object is in front of another. 00:23:11.558 --> 00:23:13.850 align:middle line:84% And so this is-- and there's now lots of demonstrations 00:23:13.850 --> 00:23:14.767 align:middle line:90% that this is the case. 00:23:14.767 --> 00:23:16.810 align:middle line:84% This is one that's quite powerful, where 00:23:16.810 --> 00:23:20.313 align:middle line:84% I'm going to show you a stimulus that consists of these two 00:23:20.313 --> 00:23:21.730 align:middle line:84% bars, one that's moving vertically 00:23:21.730 --> 00:23:24.170 align:middle line:84% and one that's moving horizontally. 00:23:24.170 --> 00:23:27.750 align:middle line:84% And when you see both of the bars at the same time, 00:23:27.750 --> 00:23:30.430 align:middle line:84% you can interpret the motion in one two different ways, 00:23:30.430 --> 00:23:32.950 align:middle line:84% depending on whether there's evidence for occlusion. 00:23:32.950 --> 00:23:36.222 align:middle line:84% So if you see the bars on their own, 00:23:36.222 --> 00:23:37.930 align:middle line:84% they look like they're moving separately. 00:23:37.930 --> 00:23:39.787 align:middle line:84% One's up and down, and one's back and forth. 00:23:39.787 --> 00:23:41.370 align:middle line:84% But then when we put this little frame 00:23:41.370 --> 00:23:44.330 align:middle line:84% around them, which plausibly could be occluding 00:23:44.330 --> 00:23:47.450 align:middle line:84% the endpoints, you now see the thing 00:23:47.450 --> 00:23:49.730 align:middle line:84% as one object that's kind of moving in a circle. 00:23:49.730 --> 00:23:53.090 align:middle line:84% And so the plausible explanation is that your visual system 00:23:53.090 --> 00:23:56.290 align:middle line:84% supposes that the image motion that's happening here 00:23:56.290 --> 00:23:57.550 align:middle line:90% is due to occlusion. 00:23:57.550 --> 00:24:00.570 align:middle line:84% And that no longer really has a big impact on what you see. 00:24:00.570 --> 00:24:07.970 align:middle line:90% So this is an example of that. 00:24:07.970 --> 00:24:13.890 align:middle line:84% And I should just say that these demos-- 00:24:13.890 --> 00:24:14.950 align:middle line:90% hang on a second. 00:24:14.950 --> 00:24:19.280 align:middle line:90% 00:24:19.280 --> 00:24:20.080 align:middle line:90% There we go. 00:24:20.080 --> 00:24:23.000 align:middle line:90% So this is the basic stimulus. 00:24:23.000 --> 00:24:28.205 align:middle line:84% And so this is what happens if the bars are just on their own. 00:24:28.205 --> 00:24:29.580 align:middle line:84% Hopefully, everybody just agrees, 00:24:29.580 --> 00:24:32.520 align:middle line:84% it looks like two different things, right? 00:24:32.520 --> 00:24:34.780 align:middle line:90% We add this frame around them. 00:24:34.780 --> 00:24:37.160 align:middle line:84% And most of you probably see a single thing that kind of 00:24:37.160 --> 00:24:38.680 align:middle line:90% moves around in a circle. 00:24:38.680 --> 00:24:40.120 align:middle line:84% So the thing to emphasize here is 00:24:40.120 --> 00:24:43.440 align:middle line:84% that the image motion is exactly the same in these two cases. 00:24:43.440 --> 00:24:49.160 align:middle line:84% But you arrive at these two different interpretations, 00:24:49.160 --> 00:24:54.080 align:middle line:84% depending on whether there's evidence for occlusion or not. 00:24:54.080 --> 00:24:56.680 align:middle line:84% I just wanted to mention, these demos, 00:24:56.680 --> 00:25:03.140 align:middle line:84% these were created by an MIT UROP in like 2002 or something. 00:25:03.140 --> 00:25:08.463 align:middle line:84% And there used to be a website that had all of these things. 00:25:08.463 --> 00:25:09.880 align:middle line:84% But they were programmed in flash. 00:25:09.880 --> 00:25:11.860 align:middle line:84% And flash doesn't work in browsers anymore. 00:25:11.860 --> 00:25:14.500 align:middle line:84% So you can download a flash player, and they still work. 00:25:14.500 --> 00:25:16.360 align:middle line:84% But this is just to say that you never know. 00:25:16.360 --> 00:25:17.818 align:middle line:84% The things that you do in your UROP 00:25:17.818 --> 00:25:19.970 align:middle line:84% might still remain relevant in 20 years. 00:25:19.970 --> 00:25:23.750 align:middle line:90% So it's kind of cool. 00:25:23.750 --> 00:25:27.150 align:middle line:84% So the point of this is that motion analysis 00:25:27.150 --> 00:25:29.030 align:middle line:84% seems to be informed by information 00:25:29.030 --> 00:25:30.590 align:middle line:90% about depth and occlusion. 00:25:30.590 --> 00:25:32.343 align:middle line:84% Motions that occur that where you 00:25:32.343 --> 00:25:34.510 align:middle line:84% have evidence that they occur at points of occlusion 00:25:34.510 --> 00:25:36.150 align:middle line:90% tend to be discounted. 00:25:36.150 --> 00:25:38.810 align:middle line:84% So if we go back to the outline that I was just showing you, 00:25:38.810 --> 00:25:41.670 align:middle line:84% we talked about this idea that the aperture problem can 00:25:41.670 --> 00:25:44.470 align:middle line:90% get resolved in two main ways. 00:25:44.470 --> 00:25:48.230 align:middle line:84% One is to make use of unambiguous 2D signals. 00:25:48.230 --> 00:25:49.842 align:middle line:90% We saw some examples of that. 00:25:49.842 --> 00:25:52.550 align:middle line:84% The trick there is that you need to discount 2D signals that come 00:25:52.550 --> 00:25:54.950 align:middle line:84% from occlusion, which means you have to take into account 00:25:54.950 --> 00:25:56.670 align:middle line:90% information about occlusion. 00:25:56.670 --> 00:25:59.253 align:middle line:84% But then the other way you could overcome the aperture problem 00:25:59.253 --> 00:26:01.830 align:middle line:84% is by integrating edge motions across space. 00:26:01.830 --> 00:26:08.762 align:middle line:84% And so the challenge there is that some times, local motions 00:26:08.762 --> 00:26:09.970 align:middle line:90% arise from different objects. 00:26:09.970 --> 00:26:11.803 align:middle line:84% So if you just take this same display again, 00:26:11.803 --> 00:26:15.700 align:middle line:84% with these two squares that are moving in opposite directions, 00:26:15.700 --> 00:26:16.640 align:middle line:90% here's the situation. 00:26:16.640 --> 00:26:19.860 align:middle line:84% So let's suppose you have a receptive field here 00:26:19.860 --> 00:26:22.240 align:middle line:84% and a receptive field here and a receptive field here. 00:26:22.240 --> 00:26:24.980 align:middle line:84% So the consequence of the aperture problem 00:26:24.980 --> 00:26:29.180 align:middle line:84% is that each one of these measurements is ambiguous. 00:26:29.180 --> 00:26:30.000 align:middle line:90% It's not useless. 00:26:30.000 --> 00:26:31.180 align:middle line:90% It's not uninformative. 00:26:31.180 --> 00:26:32.120 align:middle line:90% But it's ambiguous. 00:26:32.120 --> 00:26:34.760 align:middle line:84% And so what that means is it gives you a constraint line. 00:26:34.760 --> 00:26:37.980 align:middle line:84% So with this measurement, you get this particular constraint 00:26:37.980 --> 00:26:38.480 align:middle line:90% line. 00:26:38.480 --> 00:26:42.500 align:middle line:84% So this local motion is consistent with any velocity 00:26:42.500 --> 00:26:45.220 align:middle line:90% anywhere on this line. 00:26:45.220 --> 00:26:47.380 align:middle line:84% And this particular local measurement 00:26:47.380 --> 00:26:49.660 align:middle line:84% is consistent with a different constraint line. 00:26:49.660 --> 00:26:53.340 align:middle line:84% So the intersection of those is unambiguous and gives you 00:26:53.340 --> 00:26:57.240 align:middle line:84% the correct direction of motion, which is this horizontal motion 00:26:57.240 --> 00:26:58.780 align:middle line:90% So life is good. 00:26:58.780 --> 00:27:02.780 align:middle line:84% But let's just suppose that you, instead, 00:27:02.780 --> 00:27:05.020 align:middle line:84% decide to combine the measurement that you make here 00:27:05.020 --> 00:27:06.687 align:middle line:84% with the measurement that you make here. 00:27:06.687 --> 00:27:09.840 align:middle line:84% So this one, again, gives you that same constraint line here. 00:27:09.840 --> 00:27:12.120 align:middle line:84% But this one gives you a different constraint line. 00:27:12.120 --> 00:27:15.570 align:middle line:84% And now the intersection of the constraints is vertical. 00:27:15.570 --> 00:27:19.130 align:middle line:84% And nothing in the scene is moving vertically. 00:27:19.130 --> 00:27:22.370 align:middle line:84% All right, so this business of integrating information really 00:27:22.370 --> 00:27:26.730 align:middle line:84% only makes sense if you know what to combine with what. 00:27:26.730 --> 00:27:30.770 align:middle line:84% And one possibility here is that you 00:27:30.770 --> 00:27:33.870 align:middle line:84% would make use of form information to determine that. 00:27:33.870 --> 00:27:37.110 align:middle line:84% And there's now lots of cool demonstrations of this. 00:27:37.110 --> 00:27:38.110 align:middle line:90% This is a classic one. 00:27:38.110 --> 00:27:42.050 align:middle line:84% This was introduced by Maggie Shiffrar and Jean Lorenceau. 00:27:42.050 --> 00:27:44.090 align:middle line:84% So again, there are these moving bars. 00:27:44.090 --> 00:27:46.610 align:middle line:84% And this is a situation where, when 00:27:46.610 --> 00:27:49.850 align:middle line:84% there is evidence for occlusion, that could potentially 00:27:49.850 --> 00:27:53.870 align:middle line:84% account for the fact that there would be a single diamond here, 00:27:53.870 --> 00:27:56.330 align:middle line:84% just the corners of which happen to be covered. 00:27:56.330 --> 00:27:59.370 align:middle line:84% You tend to see these things as moving as a single thing. 00:27:59.370 --> 00:28:01.110 align:middle line:84% And without that evidence for occlusion, 00:28:01.110 --> 00:28:02.930 align:middle line:90% you tend to see separate things. 00:28:02.930 --> 00:28:04.890 align:middle line:84% And that's a super powerful effect. 00:28:04.890 --> 00:28:07.510 align:middle line:90% So here's how that works. 00:28:07.510 --> 00:28:16.490 align:middle line:90% 00:28:16.490 --> 00:28:19.450 align:middle line:84% OK, so two sets of bars, they look like they're 00:28:19.450 --> 00:28:20.890 align:middle line:90% moving independently. 00:28:20.890 --> 00:28:22.770 align:middle line:90% Then we introduce occlusion. 00:28:22.770 --> 00:28:24.970 align:middle line:84% And suddenly, you can see a single thing 00:28:24.970 --> 00:28:28.450 align:middle line:84% that's kind of moving around in a circle. 00:28:28.450 --> 00:28:32.570 align:middle line:84% Importantly, there's certainly nothing locally 00:28:32.570 --> 00:28:34.290 align:middle line:90% that moves in a circle. 00:28:34.290 --> 00:28:38.050 align:middle line:84% So that circular motion that you perceive 00:28:38.050 --> 00:28:43.610 align:middle line:84% has to result from combining information across the edges. 00:28:43.610 --> 00:28:46.290 align:middle line:84% And you combine it in this setting, 00:28:46.290 --> 00:28:48.550 align:middle line:90% but not in this setting. 00:28:48.550 --> 00:28:54.810 align:middle line:90% 00:28:54.810 --> 00:28:58.250 align:middle line:84% All right, and so intuitively, one possibility 00:28:58.250 --> 00:29:00.930 align:middle line:84% is that this process of integrating 00:29:00.930 --> 00:29:04.390 align:middle line:84% the motion of the edges might be linked to contour completion. 00:29:04.390 --> 00:29:06.970 align:middle line:84% So a couple lectures ago, we talked about the fact 00:29:06.970 --> 00:29:09.130 align:middle line:84% that when things are occluded, there's 00:29:09.130 --> 00:29:11.080 align:middle line:84% this process of amodal completion 00:29:11.080 --> 00:29:13.720 align:middle line:84% that allows you to sense the presence of one 00:29:13.720 --> 00:29:15.140 align:middle line:90% object behind another. 00:29:15.140 --> 00:29:19.880 align:middle line:84% And you might think that is happening here 00:29:19.880 --> 00:29:23.640 align:middle line:84% and that the integration of the motion is related to that. 00:29:23.640 --> 00:29:27.520 align:middle line:84% And in fact, there's a bunch of pieces of evidence for that. 00:29:27.520 --> 00:29:30.640 align:middle line:84% So here's one variant on this stimulus 00:29:30.640 --> 00:29:32.480 align:middle line:90% that can be used to test this. 00:29:32.480 --> 00:29:35.600 align:middle line:84% So here, the idea is that there's 00:29:35.600 --> 00:29:38.560 align:middle line:84% a manipulation of the occluders so that in this case, 00:29:38.560 --> 00:29:41.320 align:middle line:84% there's room beneath the occluders 00:29:41.320 --> 00:29:46.520 align:middle line:84% for there to be a full diamond for these contours to connect. 00:29:46.520 --> 00:29:49.000 align:middle line:84% And the idea is that here, there really isn't room. 00:29:49.000 --> 00:29:52.600 align:middle line:84% And so if you think that really, the process of integrating 00:29:52.600 --> 00:29:54.360 align:middle line:84% and combining the motion of the edges 00:29:54.360 --> 00:29:56.500 align:middle line:84% is dependent on the contour completion, 00:29:56.500 --> 00:29:58.920 align:middle line:84% you might expect that this thing would 00:29:58.920 --> 00:30:02.640 align:middle line:84% cohere as a single object a lot less than that one. 00:30:02.640 --> 00:30:06.540 align:middle line:84% And that is, in fact, what tends to happen. 00:30:06.540 --> 00:30:08.580 align:middle line:84% You can verify this for yourself. 00:30:08.580 --> 00:30:14.430 align:middle line:90% 00:30:14.430 --> 00:30:20.830 align:middle line:84% So this is a case where there's room for the diamond to complete 00:30:20.830 --> 00:30:22.670 align:middle line:90% behind the occluders. 00:30:22.670 --> 00:30:27.270 align:middle line:84% And most people tend to say that they see a single thing moving 00:30:27.270 --> 00:30:28.590 align:middle line:90% around in a circle. 00:30:28.590 --> 00:30:32.110 align:middle line:84% But if we make those occluders much thinner, 00:30:32.110 --> 00:30:35.350 align:middle line:84% that integration probably breaks. 00:30:35.350 --> 00:30:36.478 align:middle line:90% I see some people nodding. 00:30:36.478 --> 00:30:37.770 align:middle line:90% Is that working for all of you? 00:30:37.770 --> 00:30:38.870 align:middle line:90% Yeah, good. 00:30:38.870 --> 00:30:42.830 align:middle line:84% If we make them thick again, you're 00:30:42.830 --> 00:30:46.830 align:middle line:84% probably able to see a single thing moving around in a circle. 00:30:46.830 --> 00:30:51.790 align:middle line:84% We make them thin, possibly breaks. 00:30:51.790 --> 00:30:53.870 align:middle line:90% This is another manipulation. 00:30:53.870 --> 00:30:56.950 align:middle line:84% So if we make a small change here 00:30:56.950 --> 00:30:59.110 align:middle line:84% that now allows you to see the occluders 00:30:59.110 --> 00:31:01.910 align:middle line:84% as these extended surfaces again, 00:31:01.910 --> 00:31:05.150 align:middle line:84% people tend to be able to then see a single object moving 00:31:05.150 --> 00:31:07.980 align:middle line:90% around in a circle again. 00:31:07.980 --> 00:31:09.080 align:middle line:90% Yeah, working? 00:31:09.080 --> 00:31:09.960 align:middle line:90% Yeah, OK. 00:31:09.960 --> 00:31:12.940 align:middle line:90% 00:31:12.940 --> 00:31:16.460 align:middle line:84% And so these are just demonstrations 00:31:16.460 --> 00:31:20.240 align:middle line:84% that you can evaluate from looking at them. 00:31:20.240 --> 00:31:22.640 align:middle line:84% But you can do experiments to measure this. 00:31:22.640 --> 00:31:25.060 align:middle line:84% These are just bar graphs that plot the proportion of time 00:31:25.060 --> 00:31:27.060 align:middle line:84% that people say that they see this, what's 00:31:27.060 --> 00:31:28.880 align:middle line:84% called the coherent interpretation, 00:31:28.880 --> 00:31:32.900 align:middle line:84% which just means that there's a single thing moving in a circle. 00:31:32.900 --> 00:31:34.700 align:middle line:84% And when there's room for the thing 00:31:34.700 --> 00:31:38.877 align:middle line:84% to amodally complete behind the surfaces here or here, 00:31:38.877 --> 00:31:40.960 align:middle line:84% people tend to report the coherent interpretation. 00:31:40.960 --> 00:31:42.780 align:middle line:84% And then when there isn't room, they don't. 00:31:42.780 --> 00:31:45.860 align:middle line:84% Now you may remember, back when we were talking about contour 00:31:45.860 --> 00:31:50.220 align:middle line:84% completion, that another thing that affects completion 00:31:50.220 --> 00:31:52.727 align:middle line:84% is this property of relatability. 00:31:52.727 --> 00:31:55.060 align:middle line:84% So it has to do with whether the contours are positioned 00:31:55.060 --> 00:31:56.400 align:middle line:90% so that they can complete. 00:31:56.400 --> 00:31:58.940 align:middle line:84% And so here, the four white lines 00:31:58.940 --> 00:32:01.180 align:middle line:84% would be considered to be relatable 00:32:01.180 --> 00:32:04.340 align:middle line:84% because you can connect them with a smooth contour. 00:32:04.340 --> 00:32:06.730 align:middle line:84% And here, they're really not relatable. 00:32:06.730 --> 00:32:09.410 align:middle line:84% You have to have a really wacky looking shape in order 00:32:09.410 --> 00:32:12.250 align:middle line:84% for those things to all be connected. 00:32:12.250 --> 00:32:16.490 align:middle line:84% And so if you think that the motion integration would 00:32:16.490 --> 00:32:18.050 align:middle line:84% be affected by this, you might expect 00:32:18.050 --> 00:32:19.550 align:middle line:84% that there would be a big difference 00:32:19.550 --> 00:32:21.927 align:middle line:84% in the extent to which you would see coherent motion. 00:32:21.927 --> 00:32:24.510 align:middle line:84% Even though the local motions in all these cases are the same. 00:32:24.510 --> 00:32:27.570 align:middle line:84% You're just kind of moving them around a little bit. 00:32:27.570 --> 00:32:30.830 align:middle line:84% And so here's an example of that. 00:32:30.830 --> 00:32:37.930 align:middle line:90% 00:32:37.930 --> 00:32:42.530 align:middle line:84% So here's a case where things are relatable. 00:32:42.530 --> 00:32:44.825 align:middle line:84% How many people are seeing this as a single thing 00:32:44.825 --> 00:32:45.950 align:middle line:90% kind of moving in a circle? 00:32:45.950 --> 00:32:48.130 align:middle line:90% Raise your hand. 00:32:48.130 --> 00:32:49.170 align:middle line:90% Most people. 00:32:49.170 --> 00:32:52.090 align:middle line:84% In fact, another thing that affects us a little bit 00:32:52.090 --> 00:32:53.190 align:middle line:90% is the contrast. 00:32:53.190 --> 00:32:56.610 align:middle line:84% So we can even make it a bit lower contrast. 00:32:56.610 --> 00:32:59.810 align:middle line:84% Is everybody seeing that as moving in a circle? 00:32:59.810 --> 00:33:01.690 align:middle line:90% There's one person who doesn't. 00:33:01.690 --> 00:33:06.330 align:middle line:90% But what about this case here? 00:33:06.330 --> 00:33:09.810 align:middle line:90% 00:33:09.810 --> 00:33:12.130 align:middle line:84% Yeah, so that causes it to break. 00:33:12.130 --> 00:33:13.850 align:middle line:84% Now you may say, OK, well, that's just 00:33:13.850 --> 00:33:16.710 align:middle line:84% impossible to see the thing moving in a circle in that case. 00:33:16.710 --> 00:33:19.930 align:middle line:84% But then I would say, well, what about this? 00:33:19.930 --> 00:33:23.170 align:middle line:84% So we add these little 2D-features to the thing. 00:33:23.170 --> 00:33:26.043 align:middle line:84% And now everything looks like it's moving in a circle. 00:33:26.043 --> 00:33:28.210 align:middle line:84% So this is a demonstration that it is, in principle, 00:33:28.210 --> 00:33:32.850 align:middle line:84% possible to see that as a single shape moving around in a circle. 00:33:32.850 --> 00:33:37.570 align:middle line:84% But without that local evidence, you don't really 00:33:37.570 --> 00:33:38.870 align:middle line:90% interpret it in that way. 00:33:38.870 --> 00:33:42.290 align:middle line:90% 00:33:42.290 --> 00:33:50.270 align:middle line:84% We move back to here, much more likely to integrate that. 00:33:50.270 --> 00:33:53.570 align:middle line:84% And so you can do experiments that verify what hopefully 00:33:53.570 --> 00:33:54.450 align:middle line:90% you just saw. 00:33:54.450 --> 00:33:55.870 align:middle line:90% These are big effects. 00:33:55.870 --> 00:33:58.170 align:middle line:90% So they're not very subtle. 00:33:58.170 --> 00:34:03.120 align:middle line:84% So again, this is evidence that these mid-level processes 00:34:03.120 --> 00:34:06.960 align:middle line:84% of contour completion seem to be intimately related 00:34:06.960 --> 00:34:11.040 align:middle line:90% to the interpretation of motion. 00:34:11.040 --> 00:34:12.639 align:middle line:84% Here's just another demonstration 00:34:12.639 --> 00:34:17.480 align:middle line:84% of related things, where information about grouping, 00:34:17.480 --> 00:34:19.900 align:middle line:84% in this case, in depth, really seems to affect things. 00:34:19.900 --> 00:34:24.500 align:middle line:84% So I'm going to show you four stimuli here. 00:34:24.500 --> 00:34:26.920 align:middle line:84% These are each kind of a plaid that 00:34:26.920 --> 00:34:29.420 align:middle line:84% consists of a couple simplified gratings. 00:34:29.420 --> 00:34:30.780 align:middle line:90% This is the basic plaid. 00:34:30.780 --> 00:34:32.560 align:middle line:84% And then we introduce this gray thing 00:34:32.560 --> 00:34:36.679 align:middle line:84% that can be in front or behind or in between the two gratings. 00:34:36.679 --> 00:34:39.760 align:middle line:84% And so the idea is that when you position this thing 00:34:39.760 --> 00:34:42.960 align:middle line:84% in between them, it becomes impossible for the things 00:34:42.960 --> 00:34:46.480 align:middle line:84% to be connected as part of the same thing, or much less likely. 00:34:46.480 --> 00:34:49.400 align:middle line:84% And that ends up having a big effect on the motion 00:34:49.400 --> 00:34:50.639 align:middle line:90% interpretation. 00:34:50.639 --> 00:34:51.900 align:middle line:90% So here's this one. 00:34:51.900 --> 00:34:56.159 align:middle line:90% 00:34:56.159 --> 00:35:00.630 align:middle line:84% So hopefully, most of you are perceiving horizontal motion 00:35:00.630 --> 00:35:01.750 align:middle line:90% here. 00:35:01.750 --> 00:35:05.630 align:middle line:84% So now we just add this other thing in back. 00:35:05.630 --> 00:35:08.430 align:middle line:84% And you still see something horizontal. 00:35:08.430 --> 00:35:09.530 align:middle line:90% Now we put it in front. 00:35:09.530 --> 00:35:11.610 align:middle line:90% Still probably looks horizontal. 00:35:11.610 --> 00:35:18.250 align:middle line:84% But now, you probably see the two things is moving separately. 00:35:18.250 --> 00:35:21.050 align:middle line:90% 00:35:21.050 --> 00:35:24.070 align:middle line:84% So again, the local motion information 00:35:24.070 --> 00:35:26.590 align:middle line:84% is fairly similar in a lot of these cases. 00:35:26.590 --> 00:35:29.310 align:middle line:84% But the way that it gets interpreted 00:35:29.310 --> 00:35:34.030 align:middle line:84% is very differently, depending on the other kinds of evidence 00:35:34.030 --> 00:35:37.210 align:middle line:84% that these two sets of lines could be part of the same thing. 00:35:37.210 --> 00:35:40.350 align:middle line:90% 00:35:40.350 --> 00:35:45.350 align:middle line:84% And again, you can run experiments 00:35:45.350 --> 00:35:48.810 align:middle line:90% to verify all of this. 00:35:48.810 --> 00:35:52.950 align:middle line:84% And things work out more or less like you like you would expect. 00:35:52.950 --> 00:35:58.070 align:middle line:84% So the big picture here, these are all demonstrations 00:35:58.070 --> 00:36:00.580 align:middle line:84% that there are these big interactions 00:36:00.580 --> 00:36:04.260 align:middle line:84% between the perception of motion and the perception of form 00:36:04.260 --> 00:36:07.340 align:middle line:90% and grouping and shape. 00:36:07.340 --> 00:36:10.780 align:middle line:84% And so we classically think of the visual system 00:36:10.780 --> 00:36:13.420 align:middle line:84% as consisting of these two streams. 00:36:13.420 --> 00:36:17.420 align:middle line:84% And the machinery for processing motion 00:36:17.420 --> 00:36:20.780 align:middle line:84% looks like it's in this dorsal stream. 00:36:20.780 --> 00:36:23.420 align:middle line:84% And naively, you might expect that all the stuff 00:36:23.420 --> 00:36:24.900 align:middle line:84% for dealing with shape and so forth 00:36:24.900 --> 00:36:26.400 align:middle line:90% would be in the ventral stream. 00:36:26.400 --> 00:36:28.300 align:middle line:84% And maybe that's true in some way. 00:36:28.300 --> 00:36:31.900 align:middle line:84% But there's pretty clearly some very tight linkages. 00:36:31.900 --> 00:36:34.460 align:middle line:84% And we don't really know how these perceptual phenomena 00:36:34.460 --> 00:36:37.040 align:middle line:84% that I just showed you are actually mediated. 00:36:37.040 --> 00:36:40.340 align:middle line:84% But there's got to be very tight interactions. 00:36:40.340 --> 00:36:42.420 align:middle line:84% And so it seems like the visual system tends 00:36:42.420 --> 00:36:44.740 align:middle line:84% to infer an explanation of motion 00:36:44.740 --> 00:36:47.340 align:middle line:84% in terms of surfaces and objects, 00:36:47.340 --> 00:36:49.500 align:middle line:84% again, based on implicit knowledge of the way 00:36:49.500 --> 00:36:52.260 align:middle line:90% the world works. 00:36:52.260 --> 00:36:57.170 align:middle line:84% All right, any questions about those sorts of phenomena? 00:36:57.170 --> 00:37:01.228 align:middle line:84% All right, so another thing that's worth knowing about-- 00:37:01.228 --> 00:37:02.770 align:middle line:84% and it's thematically related to what 00:37:02.770 --> 00:37:04.170 align:middle line:90% we've just been talking about. 00:37:04.170 --> 00:37:06.050 align:middle line:84% So we just talked about these examples 00:37:06.050 --> 00:37:08.342 align:middle line:84% where form information seems to constrain 00:37:08.342 --> 00:37:09.550 align:middle line:90% the interpretation of motion. 00:37:09.550 --> 00:37:12.730 align:middle line:84% There's also some very famous examples, where motion alone 00:37:12.730 --> 00:37:14.970 align:middle line:90% can define form. 00:37:14.970 --> 00:37:17.930 align:middle line:84% And so the original demonstrations of this 00:37:17.930 --> 00:37:22.370 align:middle line:84% came from experiments where this scientist 00:37:22.370 --> 00:37:26.530 align:middle line:84% put little lights on the joints of humans 00:37:26.530 --> 00:37:28.430 align:middle line:84% and then filmed them in a dark room. 00:37:28.430 --> 00:37:31.130 align:middle line:84% So the idea was that the only thing that was visible 00:37:31.130 --> 00:37:35.510 align:middle line:84% was the motion of these a few points on a person's body. 00:37:35.510 --> 00:37:38.310 align:middle line:84% And when you do this, you can see stuff. 00:37:38.310 --> 00:37:40.730 align:middle line:84% So the first thing I'm going to show you 00:37:40.730 --> 00:37:43.330 align:middle line:84% is this, which is probably not going to look like a whole lot. 00:37:43.330 --> 00:37:47.590 align:middle line:90% This is something meaningful. 00:37:47.590 --> 00:37:50.610 align:middle line:84% But I played a little trick on you. 00:37:50.610 --> 00:37:53.670 align:middle line:84% And this is the one that's probably recognizable. 00:37:53.670 --> 00:37:55.210 align:middle line:84% So you see this, and you immediately 00:37:55.210 --> 00:37:57.190 align:middle line:84% see that there's a person walking. 00:37:57.190 --> 00:37:59.690 align:middle line:84% And the one that I showed you first is the exact same thing, 00:37:59.690 --> 00:38:01.210 align:middle line:90% just flipped upside down. 00:38:01.210 --> 00:38:06.055 align:middle line:84% And so there's a presumably a prior that kind of favors 00:38:06.055 --> 00:38:07.930 align:middle line:84% upright walking just because that's something 00:38:07.930 --> 00:38:10.050 align:middle line:90% you see all the time. 00:38:10.050 --> 00:38:11.430 align:middle line:90% So that kind of constrains this. 00:38:11.430 --> 00:38:15.370 align:middle line:84% But the key point is that this is, again, ill posed. 00:38:15.370 --> 00:38:17.370 align:middle line:84% There's lots and lots of possible explanations 00:38:17.370 --> 00:38:20.150 align:middle line:90% for the motions of these dots. 00:38:20.150 --> 00:38:22.150 align:middle line:84% But they are consistent with the person walking. 00:38:22.150 --> 00:38:24.150 align:middle line:84% And they cause you to see a person walking. 00:38:24.150 --> 00:38:29.670 align:middle line:84% And there's lots of other examples of things like this. 00:38:29.670 --> 00:38:31.170 align:middle line:90% So motion can define form. 00:38:31.170 --> 00:38:33.210 align:middle line:90% This is another cool example. 00:38:33.210 --> 00:38:35.490 align:middle line:84% So this is a little movie that resulted 00:38:35.490 --> 00:38:39.930 align:middle line:84% from running an old-school edge detection algorithm on a movie. 00:38:39.930 --> 00:38:41.970 align:middle line:84% And this is pretty interesting, just 00:38:41.970 --> 00:38:44.810 align:middle line:84% because each individual frame of the movie 00:38:44.810 --> 00:38:46.950 align:middle line:90% is really hard to interpret. 00:38:46.950 --> 00:38:51.370 align:middle line:84% But then when you see them all as a movie, 00:38:51.370 --> 00:38:55.660 align:middle line:84% you can easily recognize that you got some dogs playing. 00:38:55.660 --> 00:39:00.000 align:middle line:84% You can see people in the back, walking along the fence. 00:39:00.000 --> 00:39:03.400 align:middle line:84% But if I like stop it in the middle-- 00:39:03.400 --> 00:39:06.040 align:middle line:90% oops, I shouldn't stop it there. 00:39:06.040 --> 00:39:10.040 align:middle line:84% If I pause it here, each individual frame here 00:39:10.040 --> 00:39:11.540 align:middle line:90% is really hard to interpret. 00:39:11.540 --> 00:39:17.180 align:middle line:84% So lots of cases where motion really helps you see shape. 00:39:17.180 --> 00:39:20.480 align:middle line:84% So what I want to turn to now, though, 00:39:20.480 --> 00:39:22.700 align:middle line:84% is a puzzle in motion perception. 00:39:22.700 --> 00:39:24.200 align:middle line:84% And the answer to that puzzle really 00:39:24.200 --> 00:39:28.480 align:middle line:84% comes in the form of a Bayesian model of motion perception. 00:39:28.480 --> 00:39:31.280 align:middle line:84% And so this turns out to be a pretty nice application 00:39:31.280 --> 00:39:34.080 align:middle line:84% of Bayesian theories of perception 00:39:34.080 --> 00:39:36.040 align:middle line:84% that we've talked about repeatedly 00:39:36.040 --> 00:39:37.880 align:middle line:90% over the course of the class. 00:39:37.880 --> 00:39:41.680 align:middle line:84% So we talked about this idea that you can take local motion 00:39:41.680 --> 00:39:45.560 align:middle line:84% measurements and combine them according to the intersection 00:39:45.560 --> 00:39:46.240 align:middle line:90% of constraints. 00:39:46.240 --> 00:39:48.823 align:middle line:84% There's lots of cases where the local motion measurement gives 00:39:48.823 --> 00:39:50.120 align:middle line:90% you a constraint line. 00:39:50.120 --> 00:39:54.145 align:middle line:84% And to get the true 2D direction of motion, 00:39:54.145 --> 00:39:55.770 align:middle line:84% you would combine the constraint lines. 00:39:55.770 --> 00:39:57.550 align:middle line:84% And so those constraint lines, those could come from different 00:39:57.550 --> 00:40:00.610 align:middle line:84% 1D measurements that are made in the visual system in V1. 00:40:00.610 --> 00:40:03.190 align:middle line:84% They could come from different 1D edges of objects 00:40:03.190 --> 00:40:07.390 align:middle line:84% in the world, situations where you've got the aperture problem, 00:40:07.390 --> 00:40:10.950 align:middle line:90% same issue. 00:40:10.950 --> 00:40:16.433 align:middle line:84% And so this is a case where what you see 00:40:16.433 --> 00:40:18.850 align:middle line:84% is actually not the intersection of constraints direction. 00:40:18.850 --> 00:40:20.017 align:middle line:90% So that's really the puzzle. 00:40:20.017 --> 00:40:22.910 align:middle line:84% There are these instances-- so I should back up and say, 00:40:22.910 --> 00:40:25.510 align:middle line:84% a lot of the time, the direction that people 00:40:25.510 --> 00:40:27.950 align:middle line:84% do see, like when you're looking at a plaid, 00:40:27.950 --> 00:40:31.250 align:middle line:84% is given by the intersection of constraints. 00:40:31.250 --> 00:40:33.355 align:middle line:84% So historically, what happened was 00:40:33.355 --> 00:40:35.730 align:middle line:84% plaids were introduced as a stimulus for studying motion. 00:40:35.730 --> 00:40:37.130 align:middle line:90% This was back in the early '80s. 00:40:37.130 --> 00:40:39.950 align:middle line:84% And there were these initial demonstrations 00:40:39.950 --> 00:40:41.830 align:middle line:84% that the direction of motion people 00:40:41.830 --> 00:40:43.500 align:middle line:84% see when they look at plaid is typically 00:40:43.500 --> 00:40:45.250 align:middle line:84% the intersection of constraints direction. 00:40:45.250 --> 00:40:46.770 align:middle line:90% And then everybody got excited. 00:40:46.770 --> 00:40:48.730 align:middle line:84% There were like a million different experiments on plaids. 00:40:48.730 --> 00:40:51.272 align:middle line:84% And people started to realize that there are conditions where 00:40:51.272 --> 00:40:54.140 align:middle line:84% sometimes, what you see is actually not the direction 00:40:54.140 --> 00:40:58.940 align:middle line:84% it's implied by the intersection of constraints. 00:40:58.940 --> 00:41:04.640 align:middle line:84% And so this is a stimulus that exhibits this, one of several. 00:41:04.640 --> 00:41:08.300 align:middle line:84% So these are rhombuses that come in different variants. 00:41:08.300 --> 00:41:12.100 align:middle line:84% And the idea is that the edges of the rhombus 00:41:12.100 --> 00:41:13.300 align:middle line:90% are each locally ambiguous. 00:41:13.300 --> 00:41:15.900 align:middle line:84% And each one gives you a constraint line. 00:41:15.900 --> 00:41:19.360 align:middle line:84% So for the thin rhombus, you get these two constraint lines. 00:41:19.360 --> 00:41:21.180 align:middle line:84% And the intersection of the constraints 00:41:21.180 --> 00:41:24.022 align:middle line:84% is horizontal for the case where they're moving horizontal. 00:41:24.022 --> 00:41:26.480 align:middle line:84% In this case, you get these two different constraint lines. 00:41:26.480 --> 00:41:28.397 align:middle line:84% But again, the intersection of the constraints 00:41:28.397 --> 00:41:30.220 align:middle line:90% is still horizontal. 00:41:30.220 --> 00:41:33.700 align:middle line:84% But in some conditions, the direction that you see 00:41:33.700 --> 00:41:36.300 align:middle line:84% is actually closer to something that's 00:41:36.300 --> 00:41:38.680 align:middle line:84% pretty far off of the intersection of constraints. 00:41:38.680 --> 00:41:41.840 align:middle line:90% So I'm going to show you that. 00:41:41.840 --> 00:41:48.850 align:middle line:90% 00:41:48.850 --> 00:41:52.010 align:middle line:84% And the thing that we're going to manipulate here 00:41:52.010 --> 00:41:55.210 align:middle line:90% is the contrast. 00:41:55.210 --> 00:41:58.130 align:middle line:84% And so the contrast is being manipulated here 00:41:58.130 --> 00:42:02.490 align:middle line:84% because that helps to control the degree of ambiguity 00:42:02.490 --> 00:42:05.850 align:middle line:90% of the local motion information. 00:42:05.850 --> 00:42:09.890 align:middle line:84% And so this thing is always going to be actually physically 00:42:09.890 --> 00:42:11.250 align:middle line:90% moving horizontally. 00:42:11.250 --> 00:42:13.690 align:middle line:84% And in some cases, like now, you probably 00:42:13.690 --> 00:42:15.330 align:middle line:90% see it as moving horizontally. 00:42:15.330 --> 00:42:20.490 align:middle line:84% But we are going to manipulate some things, 00:42:20.490 --> 00:42:26.410 align:middle line:84% making it thinner and thinner and thinner. 00:42:26.410 --> 00:42:35.130 align:middle line:84% And you'll actually start to see the thing moving diagonally 00:42:35.130 --> 00:42:37.270 align:middle line:90% and downwards. 00:42:37.270 --> 00:42:40.170 align:middle line:84% So even though it's physically moving horizontally, 00:42:40.170 --> 00:42:43.123 align:middle line:84% you're seeing it as moving in a different direction, 00:42:43.123 --> 00:42:43.790 align:middle line:90% moving downward. 00:42:43.790 --> 00:42:45.250 align:middle line:90% Does that work for people? 00:42:45.250 --> 00:42:46.130 align:middle line:90% Yeah, OK. 00:42:46.130 --> 00:42:47.230 align:middle line:90% Good. 00:42:47.230 --> 00:42:48.190 align:middle line:90% So that's the puzzle. 00:42:48.190 --> 00:42:48.530 align:middle line:90% Yes. 00:42:48.530 --> 00:42:49.905 align:middle line:84% AUDIENCE: I was just going to ask 00:42:49.905 --> 00:42:52.930 align:middle line:84% if you can make it dark to see if the effect changes at all. 00:42:52.930 --> 00:42:55.820 align:middle line:90% JOSH MCDERMOTT: My pleasure. 00:42:55.820 --> 00:42:56.570 align:middle line:90% AUDIENCE: Oh, wow. 00:42:56.570 --> 00:42:57.250 align:middle line:90% That's cool. 00:42:57.250 --> 00:42:58.110 align:middle line:90% Yeah. 00:42:58.110 --> 00:43:00.318 align:middle line:84% JOSH MCDERMOTT: So it's actually moving horizontally. 00:43:00.318 --> 00:43:02.050 align:middle line:90% Yeah, thank you for asking. 00:43:02.050 --> 00:43:03.630 align:middle line:90% OK, so that's the puzzle. 00:43:03.630 --> 00:43:08.450 align:middle line:84% And as I said, there were a lot of demonstrations that nature. 00:43:08.450 --> 00:43:12.630 align:middle line:84% And so the question is, how can we understand that? 00:43:12.630 --> 00:43:14.530 align:middle line:84% So to try to make sense of this, I 00:43:14.530 --> 00:43:16.662 align:middle line:84% want to go back to thinking about perception 00:43:16.662 --> 00:43:17.870 align:middle line:90% as a probabilistic inference. 00:43:17.870 --> 00:43:21.810 align:middle line:84% So this is a slide that you have seen several times. 00:43:21.810 --> 00:43:24.730 align:middle line:84% So remember, we think of the task of perception 00:43:24.730 --> 00:43:29.450 align:middle line:84% as inferring the most likely state of the world, given 00:43:29.450 --> 00:43:31.930 align:middle line:84% the sensory data that you observe. 00:43:31.930 --> 00:43:35.130 align:middle line:84% So really, the quantity that we're interested in 00:43:35.130 --> 00:43:36.830 align:middle line:90% is what's called the posterior. 00:43:36.830 --> 00:43:39.490 align:middle line:84% So that's the probability of this variable 00:43:39.490 --> 00:43:42.650 align:middle line:84% here, which indicates something about the state of the world, 00:43:42.650 --> 00:43:44.030 align:middle line:90% given an observation. 00:43:44.030 --> 00:43:45.880 align:middle line:90% That's code. 00:43:45.880 --> 00:43:50.000 align:middle line:84% So the idea is that there's lots of possible states of the world. 00:43:50.000 --> 00:43:52.280 align:middle line:84% You observe O, and you want to figure out 00:43:52.280 --> 00:43:54.040 align:middle line:84% the most probable state of the world, 00:43:54.040 --> 00:43:56.080 align:middle line:84% given O. That's a general framework for thinking 00:43:56.080 --> 00:43:58.340 align:middle line:90% about perception. 00:43:58.340 --> 00:44:03.320 align:middle line:84% So Bayes' rule tells us that this posterior probability 00:44:03.320 --> 00:44:05.360 align:middle line:90% is equal to this. 00:44:05.360 --> 00:44:08.080 align:middle line:90% So there's three terms here. 00:44:08.080 --> 00:44:10.140 align:middle line:90% There's the prior. 00:44:10.140 --> 00:44:12.640 align:middle line:84% So that's how likely the different states of the world 00:44:12.640 --> 00:44:16.440 align:middle line:84% are in the absence of any evidence. 00:44:16.440 --> 00:44:18.360 align:middle line:90% Then there's the likelihood. 00:44:18.360 --> 00:44:21.400 align:middle line:84% And that's the probability of the observed data, 00:44:21.400 --> 00:44:23.060 align:middle line:90% given different hypotheses. 00:44:23.060 --> 00:44:26.120 align:middle line:84% So that's how consistent the observed data 00:44:26.120 --> 00:44:29.600 align:middle line:84% are with what you would expect, given different possible states 00:44:29.600 --> 00:44:30.760 align:middle line:90% of the world. 00:44:30.760 --> 00:44:33.020 align:middle line:84% And then there's a normalization factor, 00:44:33.020 --> 00:44:35.240 align:middle line:84% which is the probability of the observation. 00:44:35.240 --> 00:44:40.000 align:middle line:84% And for a given observation, that would be fixed So then 00:44:40.000 --> 00:44:42.950 align:middle line:84% this is an example of how different 00:44:42.950 --> 00:44:45.110 align:middle line:84% possible perceptual interpretations 00:44:45.110 --> 00:44:47.350 align:middle line:84% can vary in the prior and the likelihood. 00:44:47.350 --> 00:44:49.810 align:middle line:84% So you have, for this observation, 00:44:49.810 --> 00:44:52.670 align:middle line:84% this particular image, the idea is that we intuitively 00:44:52.670 --> 00:44:56.750 align:middle line:84% think of this as being a good explanation of the image. 00:44:56.750 --> 00:44:59.750 align:middle line:84% There's a background with these two colors and then a letter. 00:44:59.750 --> 00:45:02.110 align:middle line:84% But you could also account for the image 00:45:02.110 --> 00:45:05.690 align:middle line:84% with this state of the world, these two objects. 00:45:05.690 --> 00:45:07.770 align:middle line:84% One looks like that and one looks like that. 00:45:07.770 --> 00:45:09.710 align:middle line:84% And that perfectly accounts for the data. 00:45:09.710 --> 00:45:12.010 align:middle line:90% But it's a priori very unlikely. 00:45:12.010 --> 00:45:13.990 align:middle line:90% So the prior is very low here. 00:45:13.990 --> 00:45:16.270 align:middle line:84% But the likelihood is perfectly reasonable. 00:45:16.270 --> 00:45:18.590 align:middle line:84% On the other hand, in this particular case, 00:45:18.590 --> 00:45:21.310 align:middle line:84% the prior probability of this state of the world 00:45:21.310 --> 00:45:23.830 align:middle line:84% is perfectly fine because it's like an intact letter 00:45:23.830 --> 00:45:25.670 align:middle line:84% and a reasonable looking surface. 00:45:25.670 --> 00:45:28.130 align:middle line:84% But it doesn't account for the image data. 00:45:28.130 --> 00:45:29.870 align:middle line:90% So the likelihood would be low. 00:45:29.870 --> 00:45:32.670 align:middle line:84% So a good explanation is something typically 00:45:32.670 --> 00:45:35.070 align:middle line:84% that's got high prior probability and also 00:45:35.070 --> 00:45:36.190 align:middle line:90% high likelihood. 00:45:36.190 --> 00:45:39.110 align:middle line:84% So that's the key idea is that the posterior 00:45:39.110 --> 00:45:42.060 align:middle line:84% the thing that we're trying to maximize 00:45:42.060 --> 00:45:44.900 align:middle line:84% were somehow optimized when we're doing perceptual inference 00:45:44.900 --> 00:45:46.660 align:middle line:90% involves these two quantities. 00:45:46.660 --> 00:45:48.520 align:middle line:90% So that's the general framework. 00:45:48.520 --> 00:45:52.940 align:middle line:84% How can we think about this in the context of motion? 00:45:52.940 --> 00:45:53.820 align:middle line:90% Yeah, all right. 00:45:53.820 --> 00:45:55.905 align:middle line:84% So I want to walk you through this in detail, 00:45:55.905 --> 00:45:57.780 align:middle line:84% just because it's a nice concrete application 00:45:57.780 --> 00:46:00.600 align:middle line:84% of these ideas that otherwise can seem abstract. 00:46:00.600 --> 00:46:04.657 align:middle line:84% So remember, there are these two ingredients in Bayesian accounts 00:46:04.657 --> 00:46:07.240 align:middle line:84% of anything that kind of matter, the likelihood and the prior. 00:46:07.240 --> 00:46:09.340 align:middle line:84% And those get combined to yield the posterior. 00:46:09.340 --> 00:46:11.020 align:middle line:84% So let's talk about the likelihood. 00:46:11.020 --> 00:46:14.060 align:middle line:84% So remember, the likelihood is the probability 00:46:14.060 --> 00:46:18.960 align:middle line:84% of the observed data, given a hypothesis about the world. 00:46:18.960 --> 00:46:21.080 align:middle line:84% So we're dealing with motion perception. 00:46:21.080 --> 00:46:24.440 align:middle line:84% And so now the data is going to be an image sequence. 00:46:24.440 --> 00:46:27.620 align:middle line:84% In the simplest case, it's just two images, frame one 00:46:27.620 --> 00:46:28.880 align:middle line:90% and frame two. 00:46:28.880 --> 00:46:31.760 align:middle line:84% And something moves between frame one and frame two. 00:46:31.760 --> 00:46:34.100 align:middle line:84% The hypothesis space, because we're dealing with motion, 00:46:34.100 --> 00:46:35.600 align:middle line:90% is going to be velocity. 00:46:35.600 --> 00:46:37.460 align:middle line:84% So we're going to think about-- we're 00:46:37.460 --> 00:46:39.460 align:middle line:84% going to pretend that there's only one thing 00:46:39.460 --> 00:46:40.890 align:middle line:90% moving in the world. 00:46:40.890 --> 00:46:43.490 align:middle line:84% It's defined by one velocity that 00:46:43.490 --> 00:46:45.210 align:middle line:90% is a two-dimensional vector. 00:46:45.210 --> 00:46:47.150 align:middle line:90% So it's a point in a 2D space. 00:46:47.150 --> 00:46:48.810 align:middle line:84% There's an x component of the velocity 00:46:48.810 --> 00:46:52.410 align:middle line:84% and a y component of the velocity. 00:46:52.410 --> 00:46:54.850 align:middle line:90% So here's our stimulus. 00:46:54.850 --> 00:46:56.970 align:middle line:84% And we want to look at the likelihood 00:46:56.970 --> 00:47:01.390 align:middle line:84% for one particular measurement that's being made on that image. 00:47:01.390 --> 00:47:03.930 align:middle line:84% And in this case, it's going to be an edge of this moving 00:47:03.930 --> 00:47:08.450 align:middle line:90% diamond where that circle is. 00:47:08.450 --> 00:47:10.790 align:middle line:84% So the image sequence is what's ever observed, 00:47:10.790 --> 00:47:14.330 align:middle line:84% which in this case, would be two frames of that moving diamond. 00:47:14.330 --> 00:47:17.370 align:middle line:84% For a given image sequence, the likelihood 00:47:17.370 --> 00:47:20.850 align:middle line:84% is a function of the unknown velocity. 00:47:20.850 --> 00:47:23.430 align:middle line:84% So we're going to imagine, all right, 00:47:23.430 --> 00:47:28.130 align:middle line:84% for every possible velocity, how probable is the observed image 00:47:28.130 --> 00:47:32.450 align:middle line:84% sequence through that little aperture? 00:47:32.450 --> 00:47:37.650 align:middle line:84% And so what you get here for this particular stimulus 00:47:37.650 --> 00:47:39.130 align:middle line:90% is this. 00:47:39.130 --> 00:47:39.870 align:middle line:90% What is that? 00:47:39.870 --> 00:47:41.850 align:middle line:90% Well, it's a constraint line. 00:47:41.850 --> 00:47:44.810 align:middle line:90% And what this tells us-- 00:47:44.810 --> 00:47:47.090 align:middle line:84% and I should say that the gray level here 00:47:47.090 --> 00:47:49.210 align:middle line:90% represents probability. 00:47:49.210 --> 00:47:51.570 align:middle line:84% So the likelihood is the probability of that image 00:47:51.570 --> 00:47:56.410 align:middle line:84% sequence, given a velocity, which is a point in that 2D 00:47:56.410 --> 00:47:56.913 align:middle line:90% plane. 00:47:56.913 --> 00:47:59.330 align:middle line:84% And so what that is saying is that for all velocities that 00:47:59.330 --> 00:48:03.690 align:middle line:84% are on that line, the probability is pretty high. 00:48:03.690 --> 00:48:07.250 align:middle line:84% You can also see that the line is kind of blurry. 00:48:07.250 --> 00:48:10.830 align:middle line:84% So what that means is that as you move off of the line, 00:48:10.830 --> 00:48:13.410 align:middle line:84% there's this graceful degradation in the probability 00:48:13.410 --> 00:48:14.150 align:middle line:90% of the velocity. 00:48:14.150 --> 00:48:16.030 align:middle line:84% So there's a range of velocities. 00:48:16.030 --> 00:48:19.123 align:middle line:84% So if you're right on the hot spot of the line, 00:48:19.123 --> 00:48:20.790 align:middle line:84% you know the probability is pretty high. 00:48:20.790 --> 00:48:22.410 align:middle line:84% And then you move off of that, and it 00:48:22.410 --> 00:48:26.650 align:middle line:84% goes low and then goes down to something close to zero. 00:48:26.650 --> 00:48:29.530 align:middle line:84% And so what that is saying is that there 00:48:29.530 --> 00:48:32.130 align:middle line:84% is this family of hypotheses that 00:48:32.130 --> 00:48:34.190 align:middle line:84% are all consistent with the image sequence. 00:48:34.190 --> 00:48:36.290 align:middle line:84% And that's because of the aperture problem 00:48:36.290 --> 00:48:41.560 align:middle line:84% and the intrinsic ambiguity of the local evidence there. 00:48:41.560 --> 00:48:43.420 align:middle line:84% So now, what's manipulated here-- 00:48:43.420 --> 00:48:46.040 align:middle line:84% and this is actually because of this projector. 00:48:46.040 --> 00:48:48.040 align:middle line:84% This is not going to be very clear to you. 00:48:48.040 --> 00:48:51.440 align:middle line:84% But what you're supposed to see as you go from this to this 00:48:51.440 --> 00:48:55.092 align:middle line:84% to this is that diamond becomes lower contrast. 00:48:55.092 --> 00:48:57.300 align:middle line:84% So I think, in fact, maybe you can't see this at all. 00:48:57.300 --> 00:48:59.660 align:middle line:84% But in fact, there's a low contrast diamond here. 00:48:59.660 --> 00:49:03.120 align:middle line:84% And there's an even lower contrast diamond here. 00:49:03.120 --> 00:49:06.520 align:middle line:84% And the consequence of decreasing the contrast 00:49:06.520 --> 00:49:09.400 align:middle line:84% is that the sensory evidence becomes noisier. 00:49:09.400 --> 00:49:11.980 align:middle line:84% So there's always noise in the measurement process. 00:49:11.980 --> 00:49:17.440 align:middle line:84% And when the contrast is lower, the image data is less clear. 00:49:17.440 --> 00:49:19.920 align:middle line:84% So when you increase the noise or decrease 00:49:19.920 --> 00:49:21.580 align:middle line:90% the contrast, what happens? 00:49:21.580 --> 00:49:24.480 align:middle line:84% Well, this is what happens to the likelihood function. 00:49:24.480 --> 00:49:27.000 align:middle line:90% So we still have this line here. 00:49:27.000 --> 00:49:30.040 align:middle line:90% But the line is fuzzier. 00:49:30.040 --> 00:49:34.460 align:middle line:84% So because the image data is not as diagnostic, 00:49:34.460 --> 00:49:39.270 align:middle line:84% there's now more hypotheses that can account for the image 00:49:39.270 --> 00:49:42.110 align:middle line:90% data reasonably well. 00:49:42.110 --> 00:49:44.610 align:middle line:84% And if you decrease the contrast a lot, 00:49:44.610 --> 00:49:46.530 align:middle line:90% this thing gets really fuzzy. 00:49:46.530 --> 00:49:49.930 align:middle line:90% 00:49:49.930 --> 00:49:54.070 align:middle line:84% So that's the likelihood at one location. 00:49:54.070 --> 00:49:56.530 align:middle line:84% So if we're going to do Bayesian inference, 00:49:56.530 --> 00:49:59.710 align:middle line:84% so we're going to try to infer the most likely velocity, 00:49:59.710 --> 00:50:03.110 align:middle line:84% given this observed image sequence, 00:50:03.110 --> 00:50:05.510 align:middle line:84% well, we have to take our likelihood 00:50:05.510 --> 00:50:07.780 align:middle line:90% and multiply it by the prior. 00:50:07.780 --> 00:50:10.030 align:middle line:84% And in particular, we also have to take the likelihood 00:50:10.030 --> 00:50:14.570 align:middle line:84% at different locations and combine those. 00:50:14.570 --> 00:50:16.170 align:middle line:90% And so this is how this works. 00:50:16.170 --> 00:50:17.630 align:middle line:90% So we've got our image. 00:50:17.630 --> 00:50:22.750 align:middle line:84% We're making measurements at a bunch of different locations. 00:50:22.750 --> 00:50:25.110 align:middle line:84% Again, the assumption here is that there 00:50:25.110 --> 00:50:27.890 align:middle line:84% is a single thing in the world with a single velocity. 00:50:27.890 --> 00:50:31.350 align:middle line:84% So we're just inferring this one velocity. 00:50:31.350 --> 00:50:34.240 align:middle line:84% So we've got likelihoods at different locations. 00:50:34.240 --> 00:50:37.300 align:middle line:84% This is the high contrast case where the likelihoods 00:50:37.300 --> 00:50:44.140 align:middle line:84% are these very crisp, thin, not very fuzzy constraint lines. 00:50:44.140 --> 00:50:47.180 align:middle line:84% So we've got one at this location, one at that location. 00:50:47.180 --> 00:50:49.440 align:middle line:90% And then we've got a prior. 00:50:49.440 --> 00:50:52.840 align:middle line:84% And so what we're going to do is multiply these things together. 00:50:52.840 --> 00:50:55.760 align:middle line:84% And that's going to give us the posterior. 00:50:55.760 --> 00:50:59.360 align:middle line:84% So the posterior is, again, a function of velocity. 00:50:59.360 --> 00:51:02.580 align:middle line:84% It's telling us the probability of different velocities, given 00:51:02.580 --> 00:51:05.720 align:middle line:90% an image sequence. 00:51:05.720 --> 00:51:06.480 align:middle line:90% So what happens? 00:51:06.480 --> 00:51:08.100 align:middle line:84% Well, in this particular case, when 00:51:08.100 --> 00:51:13.580 align:middle line:84% the contrast is really high, the likelihood is very peaked. 00:51:13.580 --> 00:51:15.820 align:middle line:84% And so the consequence of that is 00:51:15.820 --> 00:51:18.420 align:middle line:84% that when you multiply the likelihood at these two 00:51:18.420 --> 00:51:21.840 align:middle line:84% different locations, you essentially get a dot. 00:51:21.840 --> 00:51:24.840 align:middle line:84% You get a point, because these are really thin lines. 00:51:24.840 --> 00:51:27.100 align:middle line:84% And so it's pretty much just this intersection 00:51:27.100 --> 00:51:30.020 align:middle line:84% of the constraint lines where the probability is significantly 00:51:30.020 --> 00:51:32.050 align:middle line:90% above zero. 00:51:32.050 --> 00:51:33.850 align:middle line:84% So then you multiply by the prior. 00:51:33.850 --> 00:51:36.570 align:middle line:84% And the prior, in this case-- and we'll get back to this-- 00:51:36.570 --> 00:51:39.410 align:middle line:84% is just assumed to be this kind of Gaussian blob here. 00:51:39.410 --> 00:51:42.090 align:middle line:84% And you, again, get this one point 00:51:42.090 --> 00:51:45.950 align:middle line:84% that's kind of significantly above zero. 00:51:45.950 --> 00:51:48.530 align:middle line:84% Now things get more interesting, in this case, 00:51:48.530 --> 00:51:50.550 align:middle line:90% where the contrast is lower. 00:51:50.550 --> 00:51:53.250 align:middle line:84% So when the contrast is lower, the sensory evidence 00:51:53.250 --> 00:51:54.850 align:middle line:90% is not as good. 00:51:54.850 --> 00:51:56.670 align:middle line:90% It's more ambiguous. 00:51:56.670 --> 00:51:59.090 align:middle line:84% So there's a bigger family of velocities 00:51:59.090 --> 00:52:01.470 align:middle line:84% that's consistent with the observed image sequence. 00:52:01.470 --> 00:52:04.090 align:middle line:84% And so now we get these constraint lines 00:52:04.090 --> 00:52:05.390 align:middle line:90% again at the two locations. 00:52:05.390 --> 00:52:06.670 align:middle line:90% But they're now much blurrier. 00:52:06.670 --> 00:52:09.690 align:middle line:84% They're more spread out in space. 00:52:09.690 --> 00:52:13.890 align:middle line:84% So now what happens is that when we multiply the two likelihoods 00:52:13.890 --> 00:52:15.730 align:middle line:84% at the two different locations, you 00:52:15.730 --> 00:52:19.250 align:middle line:84% get a blob, a pretty good sized blob. 00:52:19.250 --> 00:52:23.570 align:middle line:84% And so now, when you multiply by the prior, 00:52:23.570 --> 00:52:25.665 align:middle line:84% the prior actually has a much bigger effect. 00:52:25.665 --> 00:52:28.290 align:middle line:84% So the blob that you're going to get from multiplying these two 00:52:28.290 --> 00:52:31.770 align:middle line:84% things, again, would be centered at the intersection 00:52:31.770 --> 00:52:34.010 align:middle line:84% of the constraint lines, which would be horizontal. 00:52:34.010 --> 00:52:35.570 align:middle line:90% But it's a big blob. 00:52:35.570 --> 00:52:37.730 align:middle line:84% And so now you multiply it by this thing 00:52:37.730 --> 00:52:39.810 align:middle line:90% that's peaked at the origin. 00:52:39.810 --> 00:52:43.930 align:middle line:84% And the estimate gets pulled towards the origin. 00:52:43.930 --> 00:52:46.810 align:middle line:84% So the posterior now has a peak that's 00:52:46.810 --> 00:52:50.430 align:middle line:84% pretty far off of the true horizontal velocity. 00:52:50.430 --> 00:52:55.570 align:middle line:90% 00:52:55.570 --> 00:52:56.890 align:middle line:90% Yes. 00:52:56.890 --> 00:53:03.290 align:middle line:84% AUDIENCE: Does it matter that this is a very thin diamond 00:53:03.290 --> 00:53:03.870 align:middle line:90% shape? 00:53:03.870 --> 00:53:08.330 align:middle line:84% Or if it were a thick line, would the effect not 00:53:08.330 --> 00:53:10.290 align:middle line:90% work anymore? 00:53:10.290 --> 00:53:15.370 align:middle line:84% JOSH MCDERMOTT: So what does matter here is essentially, 00:53:15.370 --> 00:53:19.050 align:middle line:84% the relationship between the constraint lines. 00:53:19.050 --> 00:53:26.730 align:middle line:84% And essentially, yeah, I mean, so as you change the orientation 00:53:26.730 --> 00:53:29.240 align:middle line:84% of that stimulus-- and we saw a demonstration of that 00:53:29.240 --> 00:53:31.720 align:middle line:90% in a little while ago-- 00:53:31.720 --> 00:53:33.088 align:middle line:90% what you see changes. 00:53:33.088 --> 00:53:34.880 align:middle line:84% And that's in part because the relationship 00:53:34.880 --> 00:53:36.580 align:middle line:84% between the constraint line changes. 00:53:36.580 --> 00:53:42.105 align:middle line:84% And so the way that this whole thing works out is different. 00:53:42.105 --> 00:53:43.480 align:middle line:84% But I mean, there's nothing magic 00:53:43.480 --> 00:53:44.940 align:middle line:90% about this particular shape. 00:53:44.940 --> 00:53:48.160 align:middle line:84% It's just, what's actually doing the work here 00:53:48.160 --> 00:53:53.520 align:middle line:84% is the fact that the constraint lines are not that different. 00:53:53.520 --> 00:53:56.760 align:middle line:84% And essentially, what happens is when you multiply the constraint 00:53:56.760 --> 00:54:02.160 align:middle line:84% lines, you get this blurry thing like this. 00:54:02.160 --> 00:54:04.580 align:middle line:84% And that means that when you multiply by that, 00:54:04.580 --> 00:54:06.300 align:middle line:84% you get a point that's down here. 00:54:06.300 --> 00:54:08.343 align:middle line:84% But if they were differently shaped, 00:54:08.343 --> 00:54:10.760 align:middle line:84% then the product of these two things would look different. 00:54:10.760 --> 00:54:12.680 align:middle line:84% And it would interact with the prior in a different way, 00:54:12.680 --> 00:54:13.460 align:middle line:90% essentially. 00:54:13.460 --> 00:54:16.700 align:middle line:90% 00:54:16.700 --> 00:54:18.840 align:middle line:84% So the key thing that is doing the work here 00:54:18.840 --> 00:54:22.800 align:middle line:84% is this idea that when contrast gets low, 00:54:22.800 --> 00:54:25.160 align:middle line:84% the sensory evidence is more ambiguous. 00:54:25.160 --> 00:54:28.630 align:middle line:84% And than the prior exerts a bigger effect. 00:54:28.630 --> 00:54:33.230 align:middle line:84% Now the other key part of the hypothesis that's being tested 00:54:33.230 --> 00:54:35.470 align:middle line:90% here is the prior. 00:54:35.470 --> 00:54:37.570 align:middle line:84% And we haven't talked about that so far. 00:54:37.570 --> 00:54:41.190 align:middle line:84% And so what's being proposed here is that in our heads, 00:54:41.190 --> 00:54:45.430 align:middle line:84% we have a prior that favors slow speeds. 00:54:45.430 --> 00:54:49.890 align:middle line:84% So this is a Gaussian that's centered at zero. 00:54:49.890 --> 00:54:52.790 align:middle line:84% So that means that when speeds are slower, close to the origin, 00:54:52.790 --> 00:54:56.350 align:middle line:84% the probability is higher than when speeds are faster. 00:54:56.350 --> 00:54:58.210 align:middle line:90% So that's a hypothesis. 00:54:58.210 --> 00:55:01.910 align:middle line:84% And I haven't motivated that hypothesis at all. 00:55:01.910 --> 00:55:04.590 align:middle line:84% But that's being proposed as something 00:55:04.590 --> 00:55:07.452 align:middle line:84% that is consistent with the observed data. 00:55:07.452 --> 00:55:09.410 align:middle line:84% And we'll see we'll see some evidence for that. 00:55:09.410 --> 00:55:14.550 align:middle line:84% So other evidence for that, and a prediction of this, 00:55:14.550 --> 00:55:17.430 align:middle line:84% is that if you just take a regular stimulus, 00:55:17.430 --> 00:55:22.430 align:middle line:84% and you reduce the contrast, it should look slower, 00:55:22.430 --> 00:55:23.990 align:middle line:84% because what happens when you reduce 00:55:23.990 --> 00:55:27.960 align:middle line:84% the contrast is that things become more biased by the prior. 00:55:27.960 --> 00:55:30.040 align:middle line:84% And if the prior actually favors slow speeds, 00:55:30.040 --> 00:55:31.400 align:middle line:90% things should look slower. 00:55:31.400 --> 00:55:33.905 align:middle line:84% So again, I'm sorry about the projector here. 00:55:33.905 --> 00:55:35.780 align:middle line:84% The contrast of the projector is not so good. 00:55:35.780 --> 00:55:36.955 align:middle line:90% This looks OK on my screen. 00:55:36.955 --> 00:55:38.580 align:middle line:84% But maybe, when I make this thing move, 00:55:38.580 --> 00:55:40.220 align:middle line:90% you'll be able to see it. 00:55:40.220 --> 00:55:42.220 align:middle line:84% So if you look at the center of fixation-- yeah, 00:55:42.220 --> 00:55:43.473 align:middle line:90% you can see this thing. 00:55:43.473 --> 00:55:45.140 align:middle line:84% So you have a very high contrast grating 00:55:45.140 --> 00:55:47.620 align:middle line:84% on the left and a low contrast grating on the right. 00:55:47.620 --> 00:55:51.220 align:middle line:84% They are physically moving at exactly the same speed. 00:55:51.220 --> 00:55:53.440 align:middle line:84% But if you are looking in the middle, 00:55:53.440 --> 00:55:54.940 align:middle line:84% the high contrast one probably looks 00:55:54.940 --> 00:55:57.580 align:middle line:84% like it's a fair bit faster than the low contrast one. 00:55:57.580 --> 00:55:59.983 align:middle line:84% And so that's just a fact about motion perception. 00:55:59.983 --> 00:56:02.400 align:middle line:84% So if you take patterns, and you make them lower contrast, 00:56:02.400 --> 00:56:04.620 align:middle line:90% they look slower. 00:56:04.620 --> 00:56:06.460 align:middle line:84% And that's consistent with the idea 00:56:06.460 --> 00:56:08.790 align:middle line:84% that there's a prior that favors slow speeds. 00:56:08.790 --> 00:56:10.540 align:middle line:84% And then when the sensory evidence kind of 00:56:10.540 --> 00:56:16.060 align:middle line:84% gets poor at low contrast, you're biased by the prior. 00:56:16.060 --> 00:56:20.460 align:middle line:84% And so this is a graphical depiction of that principle. 00:56:20.460 --> 00:56:23.370 align:middle line:84% So here, it's shown in 1D, just because it's a lot easier 00:56:23.370 --> 00:56:27.130 align:middle line:84% to look at graphs in that setting. 00:56:27.130 --> 00:56:30.430 align:middle line:84% So we've got the prior, the likelihood, and the posterior. 00:56:30.430 --> 00:56:31.910 align:middle line:84% They're both functions of velocity. 00:56:31.910 --> 00:56:34.430 align:middle line:84% In this case, we're just looking at one dimension. 00:56:34.430 --> 00:56:36.250 align:middle line:84% And so the hypothesis here is that we've 00:56:36.250 --> 00:56:38.142 align:middle line:84% got a prior that favors slow speeds. 00:56:38.142 --> 00:56:39.350 align:middle line:90% So that's what this thing is. 00:56:39.350 --> 00:56:44.530 align:middle line:84% It's this bump that's highest at the origin. 00:56:44.530 --> 00:56:48.050 align:middle line:84% We've got a stimulus that's got some particular velocity. 00:56:48.050 --> 00:56:50.210 align:middle line:84% And so there's a likelihood that's 00:56:50.210 --> 00:56:52.810 align:middle line:84% centered on the true velocity of the stimulus. 00:56:52.810 --> 00:56:55.490 align:middle line:84% But then when we multiply the prior and the likelihood 00:56:55.490 --> 00:56:57.310 align:middle line:90% together, we get the posterior. 00:56:57.310 --> 00:57:00.130 align:middle line:84% And so you can see the posterior shifted ever so slightly 00:57:00.130 --> 00:57:06.510 align:middle line:84% in the direction of where the prior has the highest value. 00:57:06.510 --> 00:57:08.970 align:middle line:84% However, because so this is a situation 00:57:08.970 --> 00:57:10.370 align:middle line:90% where the contrast is high. 00:57:10.370 --> 00:57:13.375 align:middle line:84% When the contrast is high, the likelihood is very peaked. 00:57:13.375 --> 00:57:15.250 align:middle line:84% There's a narrow range of velocities that are 00:57:15.250 --> 00:57:16.513 align:middle line:90% consistent with the stimulus. 00:57:16.513 --> 00:57:17.930 align:middle line:84% And because it's very peaked, it's 00:57:17.930 --> 00:57:19.950 align:middle line:84% not really influenced very much by the prior. 00:57:19.950 --> 00:57:21.430 align:middle line:90% It just shifts a little bit. 00:57:21.430 --> 00:57:24.720 align:middle line:84% Whereas, in this case, this is the low contrast case. 00:57:24.720 --> 00:57:27.160 align:middle line:84% So as a consequence of things being low contrast, 00:57:27.160 --> 00:57:29.020 align:middle line:84% the likelihood is now very fuzzy. 00:57:29.020 --> 00:57:30.520 align:middle line:90% It's very spread out. 00:57:30.520 --> 00:57:33.600 align:middle line:84% There's a very wide range of possible velocities that's 00:57:33.600 --> 00:57:36.200 align:middle line:84% consistent with the observed image data. 00:57:36.200 --> 00:57:39.360 align:middle line:84% And so now, when you multiply by the prior, 00:57:39.360 --> 00:57:42.200 align:middle line:84% the posterior gets shifted way over from where the likelihood 00:57:42.200 --> 00:57:43.200 align:middle line:90% peak is. 00:57:43.200 --> 00:57:47.660 align:middle line:84% So the prior has a much bigger effect at low contrast. 00:57:47.660 --> 00:57:50.960 align:middle line:84% So again, a general principle of the way these Bayesian 00:57:50.960 --> 00:57:56.800 align:middle line:84% frameworks work is that when the evidence is degraded or worse, 00:57:56.800 --> 00:57:59.400 align:middle line:84% the prior tends to have a bigger effect. 00:57:59.400 --> 00:58:02.060 align:middle line:84% And so things tend to get pulled towards the mode of the prior. 00:58:02.060 --> 00:58:06.400 align:middle line:90% 00:58:06.400 --> 00:58:09.960 align:middle line:84% And so that's essentially the proposed explanation 00:58:09.960 --> 00:58:13.640 align:middle line:84% for these otherwise funny features of human motion 00:58:13.640 --> 00:58:16.160 align:middle line:84% perception, that in some cases, you actually 00:58:16.160 --> 00:58:18.220 align:middle line:84% don't perceive the intersection of constraints. 00:58:18.220 --> 00:58:21.320 align:middle line:84% And so the explanation is that you're 00:58:21.320 --> 00:58:25.760 align:middle line:84% in a regime where the sensory evidence is fairly ambiguous. 00:58:25.760 --> 00:58:31.160 align:middle line:84% And the prior has the ability to pull things towards its mode. 00:58:31.160 --> 00:58:33.840 align:middle line:84% And that causes you to see a velocity that is 00:58:33.840 --> 00:58:36.160 align:middle line:90% slower than the true velocity. 00:58:36.160 --> 00:58:38.040 align:middle line:84% And in these situations, that actually 00:58:38.040 --> 00:58:40.240 align:middle line:84% causes the direction of the velocity 00:58:40.240 --> 00:58:42.500 align:middle line:84% to be different from what it actually is. 00:58:42.500 --> 00:58:45.340 align:middle line:90% 00:58:45.340 --> 00:58:49.080 align:middle line:84% And so this was a whole thing about 20 years ago. 00:58:49.080 --> 00:58:52.140 align:middle line:84% And there was an extensive set of experiments showing. 00:58:52.140 --> 00:58:54.080 align:middle line:84% So this is like a manipulation of the angle 00:58:54.080 --> 00:58:58.400 align:middle line:84% of the rhombus, which has this particular effect 00:58:58.400 --> 00:59:00.300 align:middle line:84% on the perceived direction of motion. 00:59:00.300 --> 00:59:02.520 align:middle line:84% So you go from seeing the horizontal direction 00:59:02.520 --> 00:59:04.960 align:middle line:84% to seeing something that's kind of diagonal. 00:59:04.960 --> 00:59:06.620 align:middle line:84% And this is the model prediction. 00:59:06.620 --> 00:59:10.560 align:middle line:84% And it matches up pretty well with what people do. 00:59:10.560 --> 00:59:13.400 align:middle line:84% So the take home message here is that this 00:59:13.400 --> 00:59:18.910 align:middle line:84% is a canonical application of Bayesian theories of perception. 00:59:18.910 --> 00:59:22.550 align:middle line:84% So the proposal is that when you are seeing motion, 00:59:22.550 --> 00:59:26.270 align:middle line:84% your visual system is estimating the most likely direction 00:59:26.270 --> 00:59:29.270 align:middle line:84% of motion, given the image evidence. 00:59:29.270 --> 00:59:30.990 align:middle line:84% And in a Bayesian framework, we can 00:59:30.990 --> 00:59:34.510 align:middle line:84% think of that as the product of the prior 00:59:34.510 --> 00:59:37.590 align:middle line:84% over the velocity times the likelihood. 00:59:37.590 --> 00:59:41.690 align:middle line:84% When the image data is very high quality, so high contrast, 00:59:41.690 --> 00:59:43.173 align:middle line:84% the likelihood tends to dominate. 00:59:43.173 --> 00:59:44.590 align:middle line:84% And you'll just tend to see what's 00:59:44.590 --> 00:59:46.030 align:middle line:90% at the peak of the likelihood. 00:59:46.030 --> 00:59:48.690 align:middle line:84% When the quality of the image data is worse, 00:59:48.690 --> 00:59:50.570 align:middle line:90% the prior has a bigger effect. 00:59:50.570 --> 00:59:53.510 align:middle line:84% Things get pulled towards a slower velocity. 00:59:53.510 --> 00:59:57.130 align:middle line:84% And then we saw other evidence for that prior, 00:59:57.130 --> 00:59:59.150 align:middle line:84% in the sense that when you take any pattern, 00:59:59.150 --> 01:00:01.130 align:middle line:84% and you and you lower the contrast, 01:00:01.130 --> 01:00:03.790 align:middle line:90% it tends to look slower. 01:00:03.790 --> 01:00:05.250 align:middle line:90% OK, questions about that. 01:00:05.250 --> 01:00:15.950 align:middle line:90% 01:00:15.950 --> 01:00:18.620 align:middle line:84% So what I haven't told you here, and what's 01:00:18.620 --> 01:00:24.180 align:middle line:84% missing from this story is why we have this prior. 01:00:24.180 --> 01:00:26.180 align:middle line:84% So there's a lot of cases in perception 01:00:26.180 --> 01:00:30.100 align:middle line:84% where the prior is more clearly linked 01:00:30.100 --> 01:00:32.140 align:middle line:90% to properties of the world. 01:00:32.140 --> 01:00:37.060 align:middle line:84% So for instance, like the fact that edge orientations 01:00:37.060 --> 01:00:40.420 align:middle line:84% are more likely to be vertical and horizontal, because 01:00:40.420 --> 01:00:41.480 align:middle line:90% of gravity, essentially. 01:00:41.480 --> 01:00:43.428 align:middle line:84% So that's another example of a prior 01:00:43.428 --> 01:00:44.720 align:middle line:90% that you can link to the world. 01:00:44.720 --> 01:00:47.980 align:middle line:84% In this particular case, the prior 01:00:47.980 --> 01:00:50.460 align:middle line:84% is less obviously related to the world. 01:00:50.460 --> 01:00:53.500 align:middle line:90% It's actually hard to measure. 01:00:53.500 --> 01:00:55.900 align:middle line:84% So it's more like it's a hypothesis that 01:00:55.900 --> 01:01:01.380 align:middle line:84% was proposed because people realized that it would explain 01:01:01.380 --> 01:01:02.480 align:middle line:90% a whole bunch of data. 01:01:02.480 --> 01:01:03.680 align:middle line:90% It's been built into models. 01:01:03.680 --> 01:01:05.240 align:middle line:84% And it does explain quite a lot of data. 01:01:05.240 --> 01:01:06.780 align:middle line:84% And so that's of the evidence that we 01:01:06.780 --> 01:01:07.947 align:middle line:90% have this prior in our head. 01:01:07.947 --> 01:01:10.650 align:middle line:84% It's not as clearly related to statistics of the world. 01:01:10.650 --> 01:01:12.900 align:middle line:84% So that's the piece of the puzzle that's a little less 01:01:12.900 --> 01:01:14.520 align:middle line:90% satisfying than it could be. 01:01:14.520 --> 01:01:18.450 align:middle line:90% 01:01:18.450 --> 01:01:22.410 align:middle line:84% So the last thing that we'll talk about today 01:01:22.410 --> 01:01:24.690 align:middle line:84% is the fact that motion is also very closely 01:01:24.690 --> 01:01:26.710 align:middle line:90% related to depth perception. 01:01:26.710 --> 01:01:29.565 align:middle line:84% And so there's lots of these really cool classic phenomena 01:01:29.565 --> 01:01:31.190 align:middle line:84% and illusions that are related to this. 01:01:31.190 --> 01:01:33.730 align:middle line:84% And I'm going to give you a bunch of these. 01:01:33.730 --> 01:01:36.690 align:middle line:84% And again, this can seem a little bit like a laundry list. 01:01:36.690 --> 01:01:38.690 align:middle line:84% And part of the reason that I'm giving you these 01:01:38.690 --> 01:01:40.607 align:middle line:84% is that I think they are potential inspiration 01:01:40.607 --> 01:01:42.210 align:middle line:90% for your illusion labs. 01:01:42.210 --> 01:01:44.630 align:middle line:84% So they're good things to just know about. 01:01:44.630 --> 01:01:46.170 align:middle line:84% But I don't expect you to memorize 01:01:46.170 --> 01:01:47.350 align:middle line:90% every single one of these. 01:01:47.350 --> 01:01:51.170 align:middle line:84% So we talked about these examples 01:01:51.170 --> 01:01:54.390 align:middle line:84% where you can perceive form from motion, 01:01:54.390 --> 01:01:55.990 align:middle line:84% like with the point light walkers. 01:01:55.990 --> 01:01:59.970 align:middle line:84% But there's also a lot of really interesting cases where you see 01:01:59.970 --> 01:02:02.290 align:middle line:90% 3D shape from motion. 01:02:02.290 --> 01:02:05.010 align:middle line:84% And so this is a pretty simple display, 01:02:05.010 --> 01:02:07.250 align:middle line:84% where you can take an array of random dots. 01:02:07.250 --> 01:02:09.650 align:middle line:84% And you move each row of dots left or right 01:02:09.650 --> 01:02:11.350 align:middle line:90% with a sinusoidal motion. 01:02:11.350 --> 01:02:15.320 align:middle line:84% And if you vary the amplitude of the motion appropriately, 01:02:15.320 --> 01:02:19.040 align:middle line:84% it can be consistent with a 3D surface that oscillates kind 01:02:19.040 --> 01:02:20.500 align:middle line:90% of left and right. 01:02:20.500 --> 01:02:25.200 align:middle line:90% So you can watch that. 01:02:25.200 --> 01:02:27.760 align:middle line:84% And you're probably able to see that-- 01:02:27.760 --> 01:02:32.860 align:middle line:84% you can see this as a corrugated surface that's moving around. 01:02:32.860 --> 01:02:36.280 align:middle line:84% But of course, it's also consistent with something 01:02:36.280 --> 01:02:39.680 align:middle line:84% that's just kind of wiggling around in a 2D plane. 01:02:39.680 --> 01:02:40.420 align:middle line:90% It's ill posed. 01:02:40.420 --> 01:02:42.337 align:middle line:84% There's lots of different ways to explain this 01:02:42.337 --> 01:02:44.440 align:middle line:90% in terms of shape and motion. 01:02:44.440 --> 01:02:46.720 align:middle line:84% People tend to assume interpretations 01:02:46.720 --> 01:02:48.780 align:middle line:90% that involve rigid objects. 01:02:48.780 --> 01:02:51.080 align:middle line:84% So that's a prior that seems to be exerted 01:02:51.080 --> 01:02:54.520 align:middle line:90% in these kinds of cases. 01:02:54.520 --> 01:02:58.880 align:middle line:84% Another related examples is this thing called the kinetic depth 01:02:58.880 --> 01:03:00.200 align:middle line:90% effect. 01:03:00.200 --> 01:03:03.200 align:middle line:84% So here, this is a display where there's a bunch of random dots. 01:03:03.200 --> 01:03:06.540 align:middle line:84% And each one just moves sinusoidally back and forth. 01:03:06.540 --> 01:03:09.920 align:middle line:84% And the consequence of this is that the motion-- 01:03:09.920 --> 01:03:12.960 align:middle line:84% again, I don't know how-- can you see this at all? 01:03:12.960 --> 01:03:15.972 align:middle line:84% Maybe I can turn down the lights a little bit. 01:03:15.972 --> 01:03:16.680 align:middle line:90% Yeah, it's funny. 01:03:16.680 --> 01:03:19.000 align:middle line:90% It's very clear on my screen. 01:03:19.000 --> 01:03:22.760 align:middle line:84% So you look at this thing, and just obvious 01:03:22.760 --> 01:03:25.880 align:middle line:84% that it looks like a sphere that's moving around. 01:03:25.880 --> 01:03:29.760 align:middle line:84% The bottom half is the dots are not moving. 01:03:29.760 --> 01:03:34.160 align:middle line:84% But each one of these things is just rotating sinusoidally. 01:03:34.160 --> 01:03:38.440 align:middle line:84% So again, you tend to interpret the motion as rigid objects. 01:03:38.440 --> 01:03:40.260 align:middle line:84% The direction of motion here is ambiguous. 01:03:40.260 --> 01:03:41.802 align:middle line:84% So if you look at it for a long time, 01:03:41.802 --> 01:03:45.600 align:middle line:84% the direction will sometimes flip. 01:03:45.600 --> 01:03:47.800 align:middle line:90% Here's another one. 01:03:47.800 --> 01:03:50.840 align:middle line:84% This has these pretty striking interactions 01:03:50.840 --> 01:03:52.340 align:middle line:90% between motion and depth. 01:03:52.340 --> 01:03:54.360 align:middle line:84% So this is called the stereo kinetic effect. 01:03:54.360 --> 01:03:59.600 align:middle line:84% So a long time ago, before there was the internet and MP3s 01:03:59.600 --> 01:04:03.330 align:middle line:84% and stuff, people would listen to music on vinyl records. 01:04:03.330 --> 01:04:04.580 align:middle line:90% So you'd have a record player. 01:04:04.580 --> 01:04:05.705 align:middle line:90% It actually have a platter. 01:04:05.705 --> 01:04:06.840 align:middle line:90% It would spin around. 01:04:06.840 --> 01:04:08.840 align:middle line:84% And so mostly, the function of the record player 01:04:08.840 --> 01:04:11.022 align:middle line:84% was supposed to be to help people listen to music. 01:04:11.022 --> 01:04:12.730 align:middle line:84% But you can also do other things with it. 01:04:12.730 --> 01:04:16.090 align:middle line:84% And so people realize that you could create patterns like this 01:04:16.090 --> 01:04:18.590 align:middle line:84% and put them down on the record player and then look at them 01:04:18.590 --> 01:04:19.882 align:middle line:90% and amuse yourself in this way. 01:04:19.882 --> 01:04:22.430 align:middle line:84% And so this is a pattern that is just going 01:04:22.430 --> 01:04:26.010 align:middle line:90% to rotate around in a circle. 01:04:26.010 --> 01:04:27.510 align:middle line:84% But when you look at this thing, you 01:04:27.510 --> 01:04:30.430 align:middle line:84% perceive this three-dimensional structure. 01:04:30.430 --> 01:04:33.950 align:middle line:84% So again, you tend to interpret these patterns 01:04:33.950 --> 01:04:37.190 align:middle line:84% as these rigid objects that are moving around 01:04:37.190 --> 01:04:38.910 align:middle line:90% in particular ways. 01:04:38.910 --> 01:04:40.090 align:middle line:90% So again, it's ill posed. 01:04:40.090 --> 01:04:43.550 align:middle line:84% There's a lot of different ways to account for the image motion 01:04:43.550 --> 01:04:46.150 align:middle line:84% here, in terms of things in the world. 01:04:46.150 --> 01:04:50.070 align:middle line:84% And you tend to arrive at a particular interpretation 01:04:50.070 --> 01:04:54.350 align:middle line:84% by imposing a prior that favors rigidity. 01:04:54.350 --> 01:04:59.322 align:middle line:84% Here's another cool example, the same sort of thing. 01:04:59.322 --> 01:05:01.530 align:middle line:84% So this one's got a lot of different interpretations. 01:05:01.530 --> 01:05:03.238 align:middle line:84% If you stare at these things for a while, 01:05:03.238 --> 01:05:05.630 align:middle line:84% you'll find your visual system flipping back and forth 01:05:05.630 --> 01:05:08.260 align:middle line:84% between different interpretations. 01:05:08.260 --> 01:05:12.360 align:middle line:84% Not all of them are completely rigid. 01:05:12.360 --> 01:05:14.700 align:middle line:84% So the big picture here is that there's 01:05:14.700 --> 01:05:17.640 align:middle line:84% these complicated joint inferences of motion and depth. 01:05:17.640 --> 01:05:18.298 align:middle line:90% Yeah, question. 01:05:18.298 --> 01:05:20.340 align:middle line:84% AUDIENCE: I feel like a well-known reason for why 01:05:20.340 --> 01:05:22.760 align:middle line:84% we see multiple things if we keep staring at it, 01:05:22.760 --> 01:05:25.420 align:middle line:84% wouldn't it make sense that once we see something, 01:05:25.420 --> 01:05:27.100 align:middle line:84% we stick with that interpretation, 01:05:27.100 --> 01:05:28.598 align:middle line:90% unless something were to change? 01:05:28.598 --> 01:05:30.140 align:middle line:84% JOSH MCDERMOTT: Yeah, so the question 01:05:30.140 --> 01:05:33.200 align:middle line:84% is, why is it that sometimes these things are bistable? 01:05:33.200 --> 01:05:36.620 align:middle line:84% And I think the presumptive answer 01:05:36.620 --> 01:05:38.900 align:middle line:84% is that the situations where things are bistable 01:05:38.900 --> 01:05:42.840 align:middle line:84% are those where the posterior is actually multimodal. 01:05:42.840 --> 01:05:46.780 align:middle line:84% So there's multiple good explanations 01:05:46.780 --> 01:05:48.980 align:middle line:84% that both have reasonable prior probability 01:05:48.980 --> 01:05:51.100 align:middle line:90% and that explain the data. 01:05:51.100 --> 01:05:54.480 align:middle line:84% And it tends to be-- the world is very rarely bistable. 01:05:54.480 --> 01:05:56.160 align:middle line:90% You tend to see this one thing. 01:05:56.160 --> 01:05:59.420 align:middle line:84% But like it tends to be these kinds of stimuli, which 01:05:59.420 --> 01:06:02.260 align:middle line:90% are a little bit impoverished. 01:06:02.260 --> 01:06:06.010 align:middle line:84% They're super clean, black and white. 01:06:06.010 --> 01:06:07.990 align:middle line:84% So things are just a little bit more ambiguous. 01:06:07.990 --> 01:06:10.190 align:middle line:84% And so there tend to be multiple good explanations. 01:06:10.190 --> 01:06:11.890 align:middle line:84% And so the question is, OK, well then why not just stick 01:06:11.890 --> 01:06:12.805 align:middle line:90% with one? 01:06:12.805 --> 01:06:15.430 align:middle line:84% And well, there is evidence that you do kind of stick with one. 01:06:15.430 --> 01:06:16.930 align:middle line:84% So the first thing that you see, you 01:06:16.930 --> 01:06:20.063 align:middle line:90% tend to see for a little while. 01:06:20.063 --> 01:06:22.730 align:middle line:84% But some people have postulated that the bistability is actually 01:06:22.730 --> 01:06:23.910 align:middle line:90% related to sampling. 01:06:23.910 --> 01:06:25.570 align:middle line:84% So if you think there's a distribution 01:06:25.570 --> 01:06:27.470 align:middle line:84% in your head of the possible interpretations, 01:06:27.470 --> 01:06:29.830 align:middle line:84% then maybe perception is sampling from that. 01:06:29.830 --> 01:06:32.570 align:middle line:90% That's a proposal. 01:06:32.570 --> 01:06:34.483 align:middle line:84% But yeah, I mean, I think it's not-- 01:06:34.483 --> 01:06:36.650 align:middle line:84% I would say we don't have a definitive understanding 01:06:36.650 --> 01:06:39.210 align:middle line:84% of that, other than the fact that it tends 01:06:39.210 --> 01:06:42.730 align:middle line:84% to happen a lot when you have slightly impoverished stimuli 01:06:42.730 --> 01:06:43.310 align:middle line:90% like this. 01:06:43.310 --> 01:06:46.130 align:middle line:90% 01:06:46.130 --> 01:06:54.330 align:middle line:84% So another respect in which motion interacts with depth 01:06:54.330 --> 01:07:01.770 align:middle line:84% is in the way that the depth layout of the world and our self 01:07:01.770 --> 01:07:04.250 align:middle line:84% motion creates optic flow patterns. 01:07:04.250 --> 01:07:07.200 align:middle line:84% So they're characteristic patterns of motion 01:07:07.200 --> 01:07:11.420 align:middle line:84% that are induced when we move around in an environment. 01:07:11.420 --> 01:07:14.360 align:middle line:84% And this helps us figure out where we're going. 01:07:14.360 --> 01:07:15.940 align:middle line:84% If you're moving in different ways, 01:07:15.940 --> 01:07:18.670 align:middle line:84% you could get a flow pattern like this, or this 01:07:18.670 --> 01:07:19.920 align:middle line:90% if you're moving horizontally. 01:07:19.920 --> 01:07:21.920 align:middle line:84% Or if you're going to crash into the ground, 01:07:21.920 --> 01:07:23.840 align:middle line:84% you get a flow pattern like this. 01:07:23.840 --> 01:07:27.680 align:middle line:84% Here's just an example of that, where 01:07:27.680 --> 01:07:33.520 align:middle line:84% the motion makes it look like you're headed down into doom. 01:07:33.520 --> 01:07:36.080 align:middle line:84% And you can do experiments to actually verify 01:07:36.080 --> 01:07:39.120 align:middle line:84% that the visual information that you get from optic flow 01:07:39.120 --> 01:07:40.660 align:middle line:90% is important for posture. 01:07:40.660 --> 01:07:42.760 align:middle line:84% So here's an experimental setup, where 01:07:42.760 --> 01:07:45.220 align:middle line:84% you got a person who's standing in a room. 01:07:45.220 --> 01:07:49.360 align:middle line:84% But it's an unusual room, because the walls are detached 01:07:49.360 --> 01:07:52.640 align:middle line:84% from the floor so that the walls can be moved independent 01:07:52.640 --> 01:07:53.960 align:middle line:90% of the floor. 01:07:53.960 --> 01:07:55.880 align:middle line:84% And so the consequence of that is 01:07:55.880 --> 01:07:58.472 align:middle line:84% that you can induce this image motion 01:07:58.472 --> 01:08:01.180 align:middle line:84% by moving the walls a little bit towards or away from the person. 01:08:01.180 --> 01:08:03.720 align:middle line:84% You can induce this image motion without introducing 01:08:03.720 --> 01:08:06.480 align:middle line:84% any vestibular cues that you might think would also 01:08:06.480 --> 01:08:07.860 align:middle line:90% play a role in posture. 01:08:07.860 --> 01:08:12.160 align:middle line:84% And so you move the walls a little bit towards the person 01:08:12.160 --> 01:08:13.380 align:middle line:90% or away from the person. 01:08:13.380 --> 01:08:15.922 align:middle line:84% And the idea is that there are these optic flow patterns that 01:08:15.922 --> 01:08:18.920 align:middle line:84% would get created based on what the person is looking at. 01:08:18.920 --> 01:08:21.560 align:middle line:84% And then what the arrows here are indicating 01:08:21.560 --> 01:08:23.859 align:middle line:84% is that the person subtly adjusts their posture. 01:08:23.859 --> 01:08:27.020 align:middle line:84% So when the room moves toward them, 01:08:27.020 --> 01:08:28.787 align:middle line:84% you get this optic flow that, well, it's 01:08:28.787 --> 01:08:30.620 align:middle line:84% consistent with the room moving toward them. 01:08:30.620 --> 01:08:32.560 align:middle line:84% But the natural assumption is that you're actually 01:08:32.560 --> 01:08:33.800 align:middle line:90% falling forward a little bit. 01:08:33.800 --> 01:08:35.720 align:middle line:84% So then the person shifts back on their heels 01:08:35.720 --> 01:08:37.120 align:middle line:90% to try to compensate. 01:08:37.120 --> 01:08:40.640 align:middle line:84% And the same thing if it's the reverse. 01:08:40.640 --> 01:08:43.359 align:middle line:84% And so supposedly, you can do the same experiments 01:08:43.359 --> 01:08:44.319 align:middle line:90% with children. 01:08:44.319 --> 01:08:49.080 align:middle line:84% And you get this horrible result where the kid falls over. 01:08:49.080 --> 01:08:52.000 align:middle line:90% So these are not my experiments. 01:08:52.000 --> 01:08:54.840 align:middle line:84% So optic flow is used to calibrate posture, 01:08:54.840 --> 01:08:58.120 align:middle line:90% even in children. 01:08:58.120 --> 01:09:02.029 align:middle line:84% And so we talked a little bit about this area about area MT. 01:09:02.029 --> 01:09:04.237 align:middle line:84% And you can see, so this is, again, the dorsal stream 01:09:04.237 --> 01:09:05.237 align:middle line:90% of the parietal pathway. 01:09:05.237 --> 01:09:05.870 align:middle line:90% Here's MT. 01:09:05.870 --> 01:09:07.790 align:middle line:84% And there's some downstream areas. 01:09:07.790 --> 01:09:10.350 align:middle line:84% And if you move one step downstream from MT, 01:09:10.350 --> 01:09:12.350 align:middle line:84% you get to this area that's called MST. 01:09:12.350 --> 01:09:15.590 align:middle line:84% And that contains neurons that are selective for optic flow 01:09:15.590 --> 01:09:16.710 align:middle line:90% patterns. 01:09:16.710 --> 01:09:20.310 align:middle line:84% And so this is a paper from Duffy and Wurtz 01:09:20.310 --> 01:09:23.830 align:middle line:90% of an example of neurons. 01:09:23.830 --> 01:09:26.029 align:middle line:84% So A, B, and C are different neurons. 01:09:26.029 --> 01:09:29.229 align:middle line:84% And so A is one that's selective for planar motion. 01:09:29.229 --> 01:09:31.970 align:middle line:84% B is one that is selective for a circular pattern of motion. 01:09:31.970 --> 01:09:33.810 align:middle line:84% And C is selective for radial motion. 01:09:33.810 --> 01:09:36.350 align:middle line:84% And so the arrows off to the side 01:09:36.350 --> 01:09:40.149 align:middle line:84% are depicting the motion flow field of the stimulus. 01:09:40.149 --> 01:09:41.890 align:middle line:90% And then this is a plot. 01:09:41.890 --> 01:09:44.270 align:middle line:84% The graphs here plot the response over time in response 01:09:44.270 --> 01:09:44.970 align:middle line:90% to the stimulus. 01:09:44.970 --> 01:09:48.290 align:middle line:84% And so the leftward motion here produces a big response, 01:09:48.290 --> 01:09:50.910 align:middle line:84% whereas, all these other ones don't. 01:09:50.910 --> 01:09:53.229 align:middle line:84% The circular motion counter-clockwise 01:09:53.229 --> 01:09:54.229 align:middle line:90% produces a big response. 01:09:54.229 --> 01:09:55.604 align:middle line:84% Whereas, all the other ones don't 01:09:55.604 --> 01:09:56.750 align:middle line:90% in this particular example. 01:09:56.750 --> 01:10:01.180 align:middle line:84% And then this particular inward motion gives you a big response. 01:10:01.180 --> 01:10:04.580 align:middle line:84% So you do see sensitivity to these motion patterns 01:10:04.580 --> 01:10:05.280 align:middle line:90% downstream. 01:10:05.280 --> 01:10:08.140 align:middle line:84% And so these are presumably-- the tuning functions 01:10:08.140 --> 01:10:10.980 align:middle line:84% here are presumably the result of neurons 01:10:10.980 --> 01:10:13.700 align:middle line:84% that are getting input from a bunch of different local motions 01:10:13.700 --> 01:10:17.718 align:middle line:84% at different positions that collectively form that pattern. 01:10:17.718 --> 01:10:20.260 align:middle line:84% Last thing that I want to talk about in the context of motion 01:10:20.260 --> 01:10:22.148 align:middle line:90% perception is eye movements. 01:10:22.148 --> 01:10:23.940 align:middle line:84% So we've talked about how eye movements are 01:10:23.940 --> 01:10:27.540 align:middle line:84% necessary to direct the fovea to image regions of interest. 01:10:27.540 --> 01:10:30.820 align:middle line:84% This is the pattern of eye fixation positions 01:10:30.820 --> 01:10:34.900 align:middle line:84% that a person would produce if they were looking 01:10:34.900 --> 01:10:36.300 align:middle line:90% at that image of a face. 01:10:36.300 --> 01:10:38.800 align:middle line:84% So you can see that people are moving their eyes all around. 01:10:38.800 --> 01:10:41.740 align:middle line:84% They look at particular parts of the image more than others. 01:10:41.740 --> 01:10:42.780 align:middle line:84% The reason that we're talking about this 01:10:42.780 --> 01:10:44.238 align:middle line:84% in the context of motion perception 01:10:44.238 --> 01:10:47.620 align:middle line:84% is that making eye movements causes massive retinal image 01:10:47.620 --> 01:10:48.420 align:middle line:90% motion. 01:10:48.420 --> 01:10:51.460 align:middle line:84% And so the question is, how do we 01:10:51.460 --> 01:10:55.220 align:middle line:84% distinguish the retinal motion that is induced by eye movements 01:10:55.220 --> 01:10:57.220 align:middle line:84% from the motion of things in the world? 01:10:57.220 --> 01:10:58.930 align:middle line:84% So anytime you move your eyes, there's 01:10:58.930 --> 01:11:02.393 align:middle line:84% this huge shift that happens across the retina. 01:11:02.393 --> 01:11:03.810 align:middle line:84% And you don't want to mistake that 01:11:03.810 --> 01:11:05.530 align:middle line:90% for something actually moving. 01:11:05.530 --> 01:11:08.050 align:middle line:84% So eye movements come in several flavors. 01:11:08.050 --> 01:11:09.370 align:middle line:90% There are saccades. 01:11:09.370 --> 01:11:13.450 align:middle line:84% So a saccade is a fast movement from one image location 01:11:13.450 --> 01:11:14.310 align:middle line:90% to another. 01:11:14.310 --> 01:11:16.130 align:middle line:84% We make these things all the time, like three or four times 01:11:16.130 --> 01:11:16.590 align:middle line:90% a second. 01:11:16.590 --> 01:11:17.670 align:middle line:90% You typically don't realize it. 01:11:17.670 --> 01:11:19.670 align:middle line:84% You're just constantly looking around the world. 01:11:19.670 --> 01:11:20.670 align:middle line:90% There are microsaccades. 01:11:20.670 --> 01:11:22.712 align:middle line:84% These are small eye movements that occur whenever 01:11:22.712 --> 01:11:23.870 align:middle line:90% you try to fixate a point. 01:11:23.870 --> 01:11:25.070 align:middle line:84% So you think when you're fixating, 01:11:25.070 --> 01:11:25.950 align:middle line:90% your eyes are actually fixed. 01:11:25.950 --> 01:11:26.350 align:middle line:90% But they're not. 01:11:26.350 --> 01:11:28.392 align:middle line:84% They're kind of moving around just very slightly. 01:11:28.392 --> 01:11:30.550 align:middle line:84% And then there are smooth pursuit eye movements. 01:11:30.550 --> 01:11:33.760 align:middle line:84% And so these are made when you track moving image features. 01:11:33.760 --> 01:11:35.510 align:middle line:84% You can't really make these intentionally. 01:11:35.510 --> 01:11:37.968 align:middle line:84% You can only really make these if something actually moves. 01:11:37.968 --> 01:11:41.570 align:middle line:84% This is a pretty crazy movie that I found on the internet, 01:11:41.570 --> 01:11:45.010 align:middle line:84% where this person stuck these things to their eyes. 01:11:45.010 --> 01:11:47.810 align:middle line:84% So you can actually see the eyes move. 01:11:47.810 --> 01:11:49.663 align:middle line:84% And you can observe for yourself. 01:11:49.663 --> 01:11:50.330 align:middle line:90% [VIDEO PLAYBACK] 01:11:50.330 --> 01:11:52.470 align:middle line:84% - You know what's crazy about having these things in my eyes, 01:11:52.470 --> 01:11:53.070 align:middle line:90% right? 01:11:53.070 --> 01:11:55.810 align:middle line:84% It lets you see how the eyes actually work. 01:11:55.810 --> 01:11:57.500 align:middle line:84% Your eyes don't move very smoothly. 01:11:57.500 --> 01:11:59.640 align:middle line:84% Like when I move my head, you can 01:11:59.640 --> 01:12:04.600 align:middle line:84% see how my eyes try to track what I'm looking at. 01:12:04.600 --> 01:12:07.613 align:middle line:84% But sometimes, you can move your eyes smoothly 01:12:07.613 --> 01:12:08.780 align:middle line:90% if it's following something. 01:12:08.780 --> 01:12:12.280 align:middle line:84% So if I hold my finger out, and I move my finger, 01:12:12.280 --> 01:12:14.180 align:middle line:84% you can see my eyes moving smoothly. 01:12:14.180 --> 01:12:18.160 align:middle line:84% But if I try to just move my eyes like that, 01:12:18.160 --> 01:12:20.320 align:middle line:90% you can't do it smoothly. 01:12:20.320 --> 01:12:21.380 align:middle line:90% Isn't that crazy? 01:12:21.380 --> 01:12:22.212 align:middle line:90% [END PLAYBACK] 01:12:22.212 --> 01:12:23.920 align:middle line:84% JOSH MCDERMOTT: All right, yeah, so those 01:12:23.920 --> 01:12:25.020 align:middle line:90% are pursuit eye movements. 01:12:25.020 --> 01:12:27.440 align:middle line:90% They are crazy. 01:12:27.440 --> 01:12:31.000 align:middle line:84% So every time we make an eye movement, huge amount of motion 01:12:31.000 --> 01:12:32.060 align:middle line:90% is induced on the retina. 01:12:32.060 --> 01:12:33.352 align:middle line:90% How do we discount this motion? 01:12:33.352 --> 01:12:35.840 align:middle line:84% All right, in brief, there are two main ways 01:12:35.840 --> 01:12:38.080 align:middle line:84% in which you might expect this would be dealt with. 01:12:38.080 --> 01:12:40.413 align:middle line:84% One is that you would make use of proprioceptive signals 01:12:40.413 --> 01:12:41.960 align:middle line:90% from your eye muscles. 01:12:41.960 --> 01:12:44.420 align:middle line:84% So in other words, anytime that you move your eyes, 01:12:44.420 --> 01:12:46.080 align:middle line:84% you could have a little sensors in the eye muscles 01:12:46.080 --> 01:12:47.720 align:middle line:84% that would send a signal to your brain saying 01:12:47.720 --> 01:12:48.740 align:middle line:90% that things have moved. 01:12:48.740 --> 01:12:51.200 align:middle line:84% Another would be that you would use efference copies 01:12:51.200 --> 01:12:52.960 align:middle line:84% from your motor system, so the thing 01:12:52.960 --> 01:12:55.165 align:middle line:84% that's actually sending motor commands to your eyes. 01:12:55.165 --> 01:12:57.040 align:middle line:84% And you could get a copy of the motor command 01:12:57.040 --> 01:12:58.500 align:middle line:84% that would get sent to your visual system. 01:12:58.500 --> 01:12:59.980 align:middle line:90% It's called an efference copy. 01:12:59.980 --> 01:13:03.350 align:middle line:84% And so we can test the proprioceptive theory 01:13:03.350 --> 01:13:04.100 align:middle line:90% by poking our eye. 01:13:04.100 --> 01:13:05.340 align:middle line:84% We're going to quickly do this experiment. 01:13:05.340 --> 01:13:06.840 align:middle line:84% And then I'm going to send you home. 01:13:06.840 --> 01:13:09.400 align:middle line:90% So close one of your eyes. 01:13:09.400 --> 01:13:12.060 align:middle line:84% And position your finger on the eyelid. 01:13:12.060 --> 01:13:13.060 align:middle line:90% See what I'm doing? 01:13:13.060 --> 01:13:17.200 align:middle line:84% And so you're just going to give your eye a little poke, gentle, 01:13:17.200 --> 01:13:18.800 align:middle line:90% not too hard. 01:13:18.800 --> 01:13:21.840 align:middle line:84% And what you should see when you poke your eye 01:13:21.840 --> 01:13:24.520 align:middle line:90% is the world moves. 01:13:24.520 --> 01:13:25.605 align:middle line:90% So you're moving your eye. 01:13:25.605 --> 01:13:27.980 align:middle line:84% You're causing changes in the tension in the eye muscles. 01:13:27.980 --> 01:13:29.507 align:middle line:84% And so if this was really something 01:13:29.507 --> 01:13:31.340 align:middle line:84% that you would discount with proprioception, 01:13:31.340 --> 01:13:32.520 align:middle line:84% you would expect that you wouldn't really 01:13:32.520 --> 01:13:33.420 align:middle line:90% see that as motion. 01:13:33.420 --> 01:13:37.120 align:middle line:84% So that's evidence that it's not really proprioception. 01:13:37.120 --> 01:13:40.260 align:middle line:84% Efference copies seem to be an important part of the story. 01:13:40.260 --> 01:13:42.302 align:middle line:84% This can be tested by paralyzing the eye muscles. 01:13:42.302 --> 01:13:44.010 align:middle line:84% So when you do that, you tend to get sick 01:13:44.010 --> 01:13:45.940 align:middle line:84% because every time you try to move your eyes, 01:13:45.940 --> 01:13:46.920 align:middle line:90% your eyes don't move. 01:13:46.920 --> 01:13:48.640 align:middle line:84% And it looks like the world moves. 01:13:48.640 --> 01:13:50.720 align:middle line:90% So that's part of the story. 01:13:50.720 --> 01:13:53.910 align:middle line:84% Summary, motion measurements are made by local detectors 01:13:53.910 --> 01:13:56.550 align:middle line:84% early in the visual system tuned to orientation and spatial 01:13:56.550 --> 01:13:57.257 align:middle line:90% frequency. 01:13:57.257 --> 01:13:59.590 align:middle line:84% Local motion measurements are believed to be made in two 01:13:59.590 --> 01:14:03.150 align:middle line:84% stages, 1D measurements in V1, followed by 2D measurements made 01:14:03.150 --> 01:14:03.893 align:middle line:90% in MT. 01:14:03.893 --> 01:14:05.310 align:middle line:84% This has been studied with plaids, 01:14:05.310 --> 01:14:07.670 align:middle line:84% using both psychophysics and physiology. 01:14:07.670 --> 01:14:09.990 align:middle line:84% The aperture problem is a geometric ambiguity 01:14:09.990 --> 01:14:11.970 align:middle line:84% that occurs with the motion of edges. 01:14:11.970 --> 01:14:14.350 align:middle line:84% We believe it's solved using local features, informed 01:14:14.350 --> 01:14:16.108 align:middle line:90% by occlusion and grouping. 01:14:16.108 --> 01:14:18.150 align:middle line:84% The visual system seems to construct explanations 01:14:18.150 --> 01:14:19.958 align:middle line:84% in terms of surfaces and objects. 01:14:19.958 --> 01:14:21.750 align:middle line:84% We discussed how motion perception provides 01:14:21.750 --> 01:14:23.870 align:middle line:84% a key success story of Bayesian models, 01:14:23.870 --> 01:14:26.050 align:middle line:84% using a prior favoring slow speeds. 01:14:26.050 --> 01:14:29.050 align:middle line:84% We saw examples of how motion and depth perception interact, 01:14:29.050 --> 01:14:31.170 align:middle line:84% how optic flow is important for motor control, 01:14:31.170 --> 01:14:33.210 align:middle line:84% and how motion from eye movements is discounted, 01:14:33.210 --> 01:14:34.990 align:middle line:84% apparently using efference copies. 01:14:34.990 --> 01:14:36.170 align:middle line:90% That's all I got for you. 01:14:36.170 --> 01:14:37.290 align:middle line:90% Have a great weekend. 01:14:37.290 --> 01:14:39.110 align:middle line:90% Midterm is on Tuesday. 01:14:39.110 --> 01:14:41.140 align:middle line:90% I'll see you soon. 01:14:41.140 --> 01:14:52.000 align:middle line:90%