WEBVTT 00:00:00.000 --> 00:00:01.949 align:middle line:90% [SQUEAKING] 00:00:01.949 --> 00:00:03.398 align:middle line:90% [RUSTLING] 00:00:03.398 --> 00:00:05.330 align:middle line:90% [CLICKING] 00:00:05.330 --> 00:00:09.518 align:middle line:90% 00:00:09.518 --> 00:00:11.060 align:middle line:84% JOSH MCDERMOTT: Greetings, everybody. 00:00:11.060 --> 00:00:13.080 align:middle line:90% Welcome back to 9.35. 00:00:13.080 --> 00:00:16.160 align:middle line:84% Last time, we started talking about depth perception, 00:00:16.160 --> 00:00:18.978 align:middle line:84% focusing on monocular depth perception. 00:00:18.978 --> 00:00:20.520 align:middle line:84% And we'll wrap that up today and then 00:00:20.520 --> 00:00:24.400 align:middle line:84% talk a little bit about stereopsis, binocular depth 00:00:24.400 --> 00:00:25.320 align:middle line:90% perception. 00:00:25.320 --> 00:00:29.248 align:middle line:84% So just to recap what we talked about last time, 00:00:29.248 --> 00:00:31.040 align:middle line:84% the challenge here that we're talking about 00:00:31.040 --> 00:00:32.479 align:middle line:84% is that the world is three-dimensional. 00:00:32.479 --> 00:00:33.920 align:middle line:84% It's essential for lots of things 00:00:33.920 --> 00:00:35.800 align:middle line:84% that we need to do to be able to recover 00:00:35.800 --> 00:00:37.207 align:middle line:84% that three-dimensional structure. 00:00:37.207 --> 00:00:39.040 align:middle line:84% So that includes the three-dimensional shape 00:00:39.040 --> 00:00:43.320 align:middle line:84% of objects, where things are in depth relative to us. 00:00:43.320 --> 00:00:45.640 align:middle line:84% The problem is nontrivial because the input 00:00:45.640 --> 00:00:49.010 align:middle line:84% to the visual system are two-dimensional images. 00:00:49.010 --> 00:00:50.760 align:middle line:84% Now, we get two of them, but they're still 00:00:50.760 --> 00:00:51.980 align:middle line:90% two-dimensional images. 00:00:51.980 --> 00:00:55.280 align:middle line:90% 00:00:55.280 --> 00:00:58.260 align:middle line:84% So we do take advantage of the fact that we have two eyes. 00:00:58.260 --> 00:01:00.162 align:middle line:84% And we'll talk about that shortly. 00:01:00.162 --> 00:01:01.620 align:middle line:84% But what we talked about last time, 00:01:01.620 --> 00:01:05.200 align:middle line:84% and what we'll finish today, are other ways 00:01:05.200 --> 00:01:08.080 align:middle line:84% in which we recover depth-- typically leveraging assumptions 00:01:08.080 --> 00:01:11.800 align:middle line:84% about the world that often are coupled with knowledge 00:01:11.800 --> 00:01:13.920 align:middle line:84% of the physics and geometry of the way 00:01:13.920 --> 00:01:17.860 align:middle line:84% that light interacts with objects to form images. 00:01:17.860 --> 00:01:19.680 align:middle line:84% And so we talked a lot about cues. 00:01:19.680 --> 00:01:22.900 align:middle line:84% So there are lots of different visual cues to 3D shape 00:01:22.900 --> 00:01:24.960 align:middle line:90% and depth. 00:01:24.960 --> 00:01:26.680 align:middle line:84% And last time, we talked about shape 00:01:26.680 --> 00:01:29.668 align:middle line:84% from shading, linear perspective, texture gradients, 00:01:29.668 --> 00:01:31.460 align:middle line:84% the relationship between distance and size, 00:01:31.460 --> 00:01:36.040 align:middle line:84% height and field, aerial perspective, and more. 00:01:36.040 --> 00:01:42.400 align:middle line:84% So remember, shape from shading is based on the fact 00:01:42.400 --> 00:01:45.280 align:middle line:84% that many surfaces are approximately Lambertian. 00:01:45.280 --> 00:01:48.760 align:middle line:84% So the idea is that the amount of light that a surface reflects 00:01:48.760 --> 00:01:52.320 align:middle line:84% is proportional to the angle between the illumination 00:01:52.320 --> 00:01:54.040 align:middle line:90% and the surface orientation. 00:01:54.040 --> 00:01:56.800 align:middle line:84% And so if you have a surface whose orientation changes 00:01:56.800 --> 00:01:59.510 align:middle line:84% for a fixed illumination direction, 00:01:59.510 --> 00:02:02.250 align:middle line:84% the amount of light that's reflected will vary. 00:02:02.250 --> 00:02:04.563 align:middle line:84% And so if you know the direction of illumination 00:02:04.563 --> 00:02:06.730 align:middle line:84% and you know that the intensity changes in the image 00:02:06.730 --> 00:02:09.009 align:middle line:84% are only due to shading, you can then infer shape. 00:02:09.009 --> 00:02:10.770 align:middle line:84% So we saw a bunch of examples of how 00:02:10.770 --> 00:02:14.770 align:middle line:84% we do this, talked about how the visual system seems to assume-- 00:02:14.770 --> 00:02:16.950 align:middle line:84% in certainly in the absence of other evidence-- 00:02:16.950 --> 00:02:19.050 align:middle line:84% that light tends to come from above. 00:02:19.050 --> 00:02:23.290 align:middle line:84% And the consequence of that assumption is that in some cases 00:02:23.290 --> 00:02:25.610 align:middle line:90% you misperceive depth. 00:02:25.610 --> 00:02:28.650 align:middle line:84% So this thing is a crater but, from this orientation, 00:02:28.650 --> 00:02:30.450 align:middle line:90% looks like a bump. 00:02:30.450 --> 00:02:33.570 align:middle line:84% Here are some sand dunes that only really look like sand dunes 00:02:33.570 --> 00:02:36.650 align:middle line:90% when the light comes from above. 00:02:36.650 --> 00:02:41.250 align:middle line:84% We saw this demonstration of the same kind of thing. 00:02:41.250 --> 00:02:43.050 align:middle line:90% We also talked about shadows. 00:02:43.050 --> 00:02:46.630 align:middle line:84% So shadows are created when an object blocks light. 00:02:46.630 --> 00:02:51.230 align:middle line:84% And so as an object changes in height above a surface, 00:02:51.230 --> 00:02:53.770 align:middle line:84% the position of the shadow will typically change. 00:02:53.770 --> 00:02:59.402 align:middle line:84% And so we use the positioning of shadows to infer depth. 00:02:59.402 --> 00:03:01.610 align:middle line:84% The visual system seems to have internalized the fact 00:03:01.610 --> 00:03:03.890 align:middle line:84% that shadows tend to be dark if you make them light. 00:03:03.890 --> 00:03:05.913 align:middle line:84% You don't really get depth as much. 00:03:05.913 --> 00:03:07.330 align:middle line:84% And then we saw all these examples 00:03:07.330 --> 00:03:10.910 align:middle line:84% where you can find things that either are shadows, 00:03:10.910 --> 00:03:13.210 align:middle line:84% but from some other object, or that 00:03:13.210 --> 00:03:16.210 align:middle line:84% just happen to look like shadows that cause objects to look 00:03:16.210 --> 00:03:19.190 align:middle line:84% like they're elevated up in the air. 00:03:19.190 --> 00:03:23.170 align:middle line:90% 00:03:23.170 --> 00:03:27.210 align:middle line:84% And we revisited this idea that the visual system 00:03:27.210 --> 00:03:31.950 align:middle line:84% seems to model things at a local scale within the image. 00:03:31.950 --> 00:03:36.690 align:middle line:84% So we saw these demonstrations of situations 00:03:36.690 --> 00:03:39.930 align:middle line:84% where you perceive depth, but where it's really is logically 00:03:39.930 --> 00:03:42.130 align:middle line:84% impossible for the shadows to be oriented 00:03:42.130 --> 00:03:44.630 align:middle line:84% the way they are because it implies that there are these two 00:03:44.630 --> 00:03:47.010 align:middle line:84% different light sources that happen to be hitting the two 00:03:47.010 --> 00:03:47.950 align:middle line:90% sets of circles. 00:03:47.950 --> 00:03:49.830 align:middle line:84% But that doesn't really bother you at all. 00:03:49.830 --> 00:03:51.730 align:middle line:84% On the bottom, though, it does seem 00:03:51.730 --> 00:03:54.370 align:middle line:84% to bother you and interfere with the extraction of depth. 00:03:54.370 --> 00:03:56.900 align:middle line:84% So there's some degree of modeling 00:03:56.900 --> 00:04:01.820 align:middle line:84% of the shadows and illumination maybe, but on a local scale. 00:04:01.820 --> 00:04:03.660 align:middle line:84% We talked about these regularities 00:04:03.660 --> 00:04:07.340 align:middle line:84% that have to do with the size of objects or texture elements 00:04:07.340 --> 00:04:09.780 align:middle line:84% and how there's also these relationships between where 00:04:09.780 --> 00:04:13.760 align:middle line:84% things are in the height within your visual field. 00:04:13.760 --> 00:04:15.840 align:middle line:84% So as things extend away from you, 00:04:15.840 --> 00:04:17.290 align:middle line:84% they're often on the ground plane. 00:04:17.290 --> 00:04:19.040 align:middle line:84% So they'll be higher in your visual field. 00:04:19.040 --> 00:04:21.380 align:middle line:84% And so if you get gradients where 00:04:21.380 --> 00:04:23.760 align:middle line:84% the size is varied with height in visual field, 00:04:23.760 --> 00:04:26.420 align:middle line:84% it gives you pretty compelling sense of depth. 00:04:26.420 --> 00:04:29.898 align:middle line:84% So these texture gradients are quite powerful. 00:04:29.898 --> 00:04:31.940 align:middle line:84% We saw some examples where artists have exploited 00:04:31.940 --> 00:04:35.960 align:middle line:84% this to give you the illusion that the floor is not flat, 00:04:35.960 --> 00:04:39.180 align:middle line:84% even though it is, and they just painted it in a funny way. 00:04:39.180 --> 00:04:41.280 align:middle line:84% We then talked about Emmert's law. 00:04:41.280 --> 00:04:43.820 align:middle line:84% So remember, Emmert's law is this relationship 00:04:43.820 --> 00:04:49.380 align:middle line:84% between apparent size, visual angle, and apparent distance. 00:04:49.380 --> 00:04:52.580 align:middle line:84% And we saw this demonstration where we created an after image. 00:04:52.580 --> 00:04:54.580 align:middle line:84% So the idea is that the after image is something 00:04:54.580 --> 00:04:56.120 align:middle line:84% that's a fixed size on the retina, 00:04:56.120 --> 00:04:57.860 align:middle line:90% so the visual angle is fixed. 00:04:57.860 --> 00:04:59.580 align:middle line:84% And you can change the apparent distance 00:04:59.580 --> 00:05:01.580 align:middle line:84% by either looking at something that's very close 00:05:01.580 --> 00:05:03.380 align:middle line:84% or looking at something that's far away. 00:05:03.380 --> 00:05:05.088 align:middle line:84% The consequence of that is that the thing 00:05:05.088 --> 00:05:06.660 align:middle line:90% changes in apparent size. 00:05:06.660 --> 00:05:09.360 align:middle line:84% So in general, when Emmert's law is at work, 00:05:09.360 --> 00:05:11.620 align:middle line:84% it's typically in situations where 00:05:11.620 --> 00:05:15.560 align:middle line:84% either size or distance is less ambiguous than the other one. 00:05:15.560 --> 00:05:18.060 align:middle line:84% And so changes in apparent distance 00:05:18.060 --> 00:05:20.700 align:middle line:84% will cause changes in apparent size or vice versa. 00:05:20.700 --> 00:05:24.540 align:middle line:84% So we saw a bunch of examples like that and examples 00:05:24.540 --> 00:05:26.300 align:middle line:84% in photographs where you misperceive 00:05:26.300 --> 00:05:28.820 align:middle line:84% things because of the application of Emmert's 00:05:28.820 --> 00:05:30.280 align:middle line:90% law in various ways. 00:05:30.280 --> 00:05:32.660 align:middle line:84% So here's a situation where you seem 00:05:32.660 --> 00:05:36.220 align:middle line:84% to implicitly assume that the children are all the same size. 00:05:36.220 --> 00:05:38.540 align:middle line:84% And that means that they're all the same distance away, 00:05:38.540 --> 00:05:41.140 align:middle line:84% and that means that the girl has got to not 00:05:41.140 --> 00:05:44.640 align:middle line:90% have her feet on the ground. 00:05:44.640 --> 00:05:47.820 align:middle line:84% Saw these very large pigeons, where 00:05:47.820 --> 00:05:50.375 align:middle line:84% the distance is erroneously assumed 00:05:50.375 --> 00:05:52.000 align:middle line:84% to be very similar to that of the cars. 00:05:52.000 --> 00:05:54.470 align:middle line:84% And so they look pretty similar in size. 00:05:54.470 --> 00:05:56.670 align:middle line:90% Talked about aerial perspective. 00:05:56.670 --> 00:05:58.590 align:middle line:84% It's an optical effect that happens in the air 00:05:58.590 --> 00:06:02.110 align:middle line:84% that you've learned is associated with distance. 00:06:02.110 --> 00:06:03.990 align:middle line:84% Linear perspective-- due to the fact 00:06:03.990 --> 00:06:07.290 align:middle line:84% that parallel lines in the world will converge in the image, 00:06:07.290 --> 00:06:10.870 align:middle line:84% unless you happen to take the image such 00:06:10.870 --> 00:06:14.418 align:middle line:84% that they lie in a plane that's parallel to the image plane. 00:06:14.418 --> 00:06:16.710 align:middle line:84% And we saw lots of examples of how these depth cues can 00:06:16.710 --> 00:06:21.015 align:middle line:84% interact with Emmert's law to create these funny illusions. 00:06:21.015 --> 00:06:23.390 align:middle line:84% So these two monsters are the same signs, but one of them 00:06:23.390 --> 00:06:25.010 align:middle line:84% looks like it's a lot further away, 00:06:25.010 --> 00:06:26.250 align:middle line:90% and so it looks a lot bigger. 00:06:26.250 --> 00:06:29.830 align:middle line:90% 00:06:29.830 --> 00:06:32.830 align:middle line:84% We then saw some examples of what 00:06:32.830 --> 00:06:36.630 align:middle line:84% we see when we perceive the world are 00:06:36.630 --> 00:06:38.910 align:middle line:84% the three-dimensional structures that we infer. 00:06:38.910 --> 00:06:42.830 align:middle line:84% And so it's very hard to actually undo that inference. 00:06:42.830 --> 00:06:44.992 align:middle line:84% So you get these funny examples like this, 00:06:44.992 --> 00:06:46.950 align:middle line:84% where it's impossible to believe that those two 00:06:46.950 --> 00:06:49.790 align:middle line:84% parallelograms are the same size because you're not seeing them 00:06:49.790 --> 00:06:51.310 align:middle line:90% as they are in the image. 00:06:51.310 --> 00:06:53.470 align:middle line:84% You're seeing them in the way that you infer them 00:06:53.470 --> 00:06:55.910 align:middle line:90% in the world. 00:06:55.910 --> 00:06:58.530 align:middle line:84% And then we wrapped up with these examples of bistability. 00:06:58.530 --> 00:07:02.593 align:middle line:84% So the amazing thing about perception 00:07:02.593 --> 00:07:04.510 align:middle line:84% is that despite the fact that we're constantly 00:07:04.510 --> 00:07:06.450 align:middle line:84% solving these ill-posed problems, 00:07:06.450 --> 00:07:11.870 align:middle line:84% we typically arrive at one pretty convincing solution 00:07:11.870 --> 00:07:13.030 align:middle line:90% to the problem. 00:07:13.030 --> 00:07:14.050 align:middle line:90% It's whatever you see. 00:07:14.050 --> 00:07:17.016 align:middle line:90% And usually, it's correct. 00:07:17.016 --> 00:07:18.990 align:middle line:84% But very occasionally, when you get 00:07:18.990 --> 00:07:21.350 align:middle line:84% these kind of impoverished stimuli, 00:07:21.350 --> 00:07:22.798 align:middle line:90% things can become bistable. 00:07:22.798 --> 00:07:24.590 align:middle line:84% So the Necker cube is a very famous example 00:07:24.590 --> 00:07:29.250 align:middle line:84% of this, where you can see either face as being in front. 00:07:29.250 --> 00:07:31.630 align:middle line:84% And so if you look at it over time, 00:07:31.630 --> 00:07:34.190 align:middle line:84% you'll flip back and forth between seeing one and seeing 00:07:34.190 --> 00:07:34.690 align:middle line:90% another. 00:07:34.690 --> 00:07:37.630 align:middle line:84% And so that phenomenon of bistability 00:07:37.630 --> 00:07:39.410 align:middle line:84% is a pretty common thing in perception. 00:07:39.410 --> 00:07:41.690 align:middle line:84% It's super interesting, as we've commented before, 00:07:41.690 --> 00:07:44.310 align:middle line:84% that we don't actually just see one interpretation 00:07:44.310 --> 00:07:46.230 align:middle line:90% and stick with it. 00:07:46.230 --> 00:07:48.420 align:middle line:84% So some people would argue that that maybe 00:07:48.420 --> 00:07:50.920 align:middle line:84% is related to the fact that there's multiple good solutions, 00:07:50.920 --> 00:07:52.700 align:middle line:84% and you're sampling from a distribution. 00:07:52.700 --> 00:07:56.760 align:middle line:84% We talked about a few different possible explanations of this. 00:07:56.760 --> 00:08:00.440 align:middle line:84% And we saw some classic examples of bistability 00:08:00.440 --> 00:08:05.240 align:middle line:84% where you can interpret things in two different ways 00:08:05.240 --> 00:08:06.860 align:middle line:84% and some jokes that are based on that. 00:08:06.860 --> 00:08:07.360 align:middle line:90% All right. 00:08:07.360 --> 00:08:09.360 align:middle line:84% And so that's where we left things. 00:08:09.360 --> 00:08:11.400 align:middle line:84% So another kind of phenomena that 00:08:11.400 --> 00:08:13.920 align:middle line:84% is related to depth perception and ties 00:08:13.920 --> 00:08:16.880 align:middle line:84% into some of the other themes that we've talked about so far 00:08:16.880 --> 00:08:18.800 align:middle line:84% is the phenomenon of impossible objects. 00:08:18.800 --> 00:08:24.200 align:middle line:84% So these are drawings, or paintings, 00:08:24.200 --> 00:08:31.220 align:middle line:84% or photographs where you glance at the thing, and it looks fine. 00:08:31.220 --> 00:08:33.240 align:middle line:84% But then you inspect it in more detail, 00:08:33.240 --> 00:08:37.200 align:middle line:84% and you realize that it's not a possible object that 00:08:37.200 --> 00:08:39.059 align:middle line:84% could be physically realized in the world. 00:08:39.059 --> 00:08:42.667 align:middle line:84% And so these things in the art of art of Escher 00:08:42.667 --> 00:08:45.000 align:middle line:84% are pretty common, where there are these staircases that 00:08:45.000 --> 00:08:46.940 align:middle line:90% look like they forever go up. 00:08:46.940 --> 00:08:50.200 align:middle line:84% And obviously, it's not really possible. 00:08:50.200 --> 00:08:53.360 align:middle line:84% There's another example where it looks like you have 00:08:53.360 --> 00:08:55.260 align:middle line:90% these arrangements of cubes. 00:08:55.260 --> 00:08:58.840 align:middle line:84% But then if you extrapolate what all the local relationships 00:08:58.840 --> 00:09:03.320 align:middle line:84% would imply, it's not something that could actually happen. 00:09:03.320 --> 00:09:06.520 align:middle line:84% And so these things are kind of fun 00:09:06.520 --> 00:09:09.800 align:middle line:90% to look at and maybe perplexing. 00:09:09.800 --> 00:09:12.500 align:middle line:84% I think they actually tell us a lot about perception. 00:09:12.500 --> 00:09:14.760 align:middle line:84% And specifically, I think they speak to this idea 00:09:14.760 --> 00:09:17.400 align:middle line:84% that we've already hit upon in a bunch of other places 00:09:17.400 --> 00:09:23.520 align:middle line:84% in this class, that the visual system seems to model images 00:09:23.520 --> 00:09:25.220 align:middle line:90% in local neighborhoods. 00:09:25.220 --> 00:09:28.760 align:middle line:84% So you look at a particular point in an image, 00:09:28.760 --> 00:09:31.680 align:middle line:84% and it seems like you create this local 3D model. 00:09:31.680 --> 00:09:33.740 align:middle line:84% As you could tell, the staircase maybe goes up, 00:09:33.740 --> 00:09:35.573 align:middle line:84% and maybe part of it intersects with another 00:09:35.573 --> 00:09:37.440 align:middle line:90% that keeps going up. 00:09:37.440 --> 00:09:40.840 align:middle line:84% And it doesn't seem like you model the entire thing at once, 00:09:40.840 --> 00:09:44.220 align:middle line:84% because if you did, well, I don't know what would happen, 00:09:44.220 --> 00:09:47.610 align:middle line:84% but you'd probably be aware that the thing doesn't make sense. 00:09:47.610 --> 00:09:49.730 align:middle line:84% So it suggests that you're not really 00:09:49.730 --> 00:09:54.690 align:middle line:84% requiring global consistency in the interpretation of the image. 00:09:54.690 --> 00:10:02.610 align:middle line:84% So you can make these funny images that exemplify this idea. 00:10:02.610 --> 00:10:06.490 align:middle line:84% So this is another example of an impossible object 00:10:06.490 --> 00:10:10.810 align:middle line:84% that, at a glance, maybe looks OK, but doesn't really 00:10:10.810 --> 00:10:12.370 align:middle line:90% make sense. 00:10:12.370 --> 00:10:23.050 align:middle line:84% Here's an interesting case of this, where if you look at this, 00:10:23.050 --> 00:10:28.090 align:middle line:84% the local interpretation is forced to change. 00:10:28.090 --> 00:10:31.510 align:middle line:84% It's related to those figure ground illusions that we saw, 00:10:31.510 --> 00:10:34.610 align:middle line:84% where the motion makes it clear that something 00:10:34.610 --> 00:10:36.670 align:middle line:90% is a three-dimensional thing. 00:10:36.670 --> 00:10:39.330 align:middle line:90% 00:10:39.330 --> 00:10:41.470 align:middle line:84% And it's kind of painful to look at. 00:10:41.470 --> 00:10:44.850 align:middle line:90% 00:10:44.850 --> 00:10:46.930 align:middle line:84% Oh, this is another one that I found. 00:10:46.930 --> 00:10:49.770 align:middle line:90% This is really wacky. 00:10:49.770 --> 00:10:54.430 align:middle line:84% So again, at a glance, anywhere that you look, this looks fine. 00:10:54.430 --> 00:10:58.770 align:middle line:90% 00:10:58.770 --> 00:11:03.650 align:middle line:84% But it's doing some things that are not possible, right? 00:11:03.650 --> 00:11:12.870 align:middle line:90% 00:11:12.870 --> 00:11:17.930 align:middle line:84% It's related to depth because the reason that you're 00:11:17.930 --> 00:11:21.370 align:middle line:84% able to render this image is essentially because it's 00:11:21.370 --> 00:11:22.510 align:middle line:90% a two-dimensional image. 00:11:22.510 --> 00:11:25.090 align:middle line:84% And the thing that's not possible 00:11:25.090 --> 00:11:32.290 align:middle line:84% is the depth arrangement that's implied at every point in space. 00:11:32.290 --> 00:11:37.450 align:middle line:84% There isn't a global 3D structure here that is actually 00:11:37.450 --> 00:11:40.770 align:middle line:90% consistent with the images. 00:11:40.770 --> 00:11:41.690 align:middle line:90% OK. 00:11:41.690 --> 00:11:46.460 align:middle line:84% Any questions about impossible objects or anything else 00:11:46.460 --> 00:11:47.560 align:middle line:90% we've talked about so far? 00:11:47.560 --> 00:11:59.060 align:middle line:90% 00:11:59.060 --> 00:12:05.600 align:middle line:84% So why do you suppose that vision might work like this? 00:12:05.600 --> 00:12:07.940 align:middle line:84% Why do you think it might be the case that you 00:12:07.940 --> 00:12:12.900 align:middle line:84% make these inferences locally without really 00:12:12.900 --> 00:12:14.760 align:middle line:90% enforcing global consistency? 00:12:14.760 --> 00:12:19.060 align:middle line:90% 00:12:19.060 --> 00:12:19.840 align:middle line:90% Any ideas? 00:12:19.840 --> 00:12:30.140 align:middle line:90% 00:12:30.140 --> 00:12:32.340 align:middle line:84% So one kind of idea that I think explains 00:12:32.340 --> 00:12:34.420 align:middle line:84% a lot of things about perception is 00:12:34.420 --> 00:12:38.500 align:middle line:90% that the world is always there. 00:12:38.500 --> 00:12:40.915 align:middle line:84% And so anytime you really you want 00:12:40.915 --> 00:12:43.040 align:middle line:84% to see something or know something about the world, 00:12:43.040 --> 00:12:45.100 align:middle line:84% typically what you do is you look at that. 00:12:45.100 --> 00:12:47.880 align:middle line:84% So we have this receptor lattice. 00:12:47.880 --> 00:12:50.360 align:middle line:84% It's very dense in this one part of the lattice. 00:12:50.360 --> 00:12:52.660 align:middle line:84% And so you typically look around the world, 00:12:52.660 --> 00:12:56.380 align:middle line:84% and you move that lattice onto the stuff that you want to see. 00:12:56.380 --> 00:12:59.480 align:middle line:84% And because the world tends to be pretty stable-- 00:12:59.480 --> 00:13:02.730 align:middle line:84% and if you want to understand what the 3D geometry is, like, 00:13:02.730 --> 00:13:03.980 align:middle line:90% say here, you just look there. 00:13:03.980 --> 00:13:05.230 align:middle line:90% Or there, you just look there. 00:13:05.230 --> 00:13:07.320 align:middle line:84% Or there, you just look there, right? 00:13:07.320 --> 00:13:11.242 align:middle line:84% So there's a lot of aspects of vision 00:13:11.242 --> 00:13:12.700 align:middle line:84% that seem consistent with this idea 00:13:12.700 --> 00:13:14.722 align:middle line:84% that you only compute things as needed. 00:13:14.722 --> 00:13:16.680 align:middle line:84% So there's something that you're interested in. 00:13:16.680 --> 00:13:18.680 align:middle line:84% You look at it, and then you do some computation 00:13:18.680 --> 00:13:22.100 align:middle line:84% that's related to the thing that you look at without necessarily 00:13:22.100 --> 00:13:24.660 align:middle line:84% analyzing, or representing, or remembering 00:13:24.660 --> 00:13:26.080 align:middle line:90% the entirety of the scene. 00:13:26.080 --> 00:13:28.660 align:middle line:84% And so we'll come back to that idea 00:13:28.660 --> 00:13:30.560 align:middle line:90% little bit later in the class. 00:13:30.560 --> 00:13:36.400 align:middle line:90% 00:13:36.400 --> 00:13:38.020 align:middle line:84% So one other kind of thing that's 00:13:38.020 --> 00:13:43.870 align:middle line:84% related to depth perception is the hollow-face illusion. 00:13:43.870 --> 00:13:48.510 align:middle line:90% So this is a hollow mask. 00:13:48.510 --> 00:13:53.230 align:middle line:84% So you look at it, and it looks like the shape 00:13:53.230 --> 00:13:54.670 align:middle line:90% of someone's face. 00:13:54.670 --> 00:13:57.910 align:middle line:90% But it's actually a hollow mask. 00:13:57.910 --> 00:14:03.710 align:middle line:84% And so these are cases where you misperceive the depth, in part 00:14:03.710 --> 00:14:05.310 align:middle line:84% because you have a pretty strong prior 00:14:05.310 --> 00:14:08.950 align:middle line:84% for what the three-dimensional structure of faces should be. 00:14:08.950 --> 00:14:11.790 align:middle line:84% And that has some pretty interesting consequences 00:14:11.790 --> 00:14:14.090 align:middle line:90% on the motion that you see. 00:14:14.090 --> 00:14:16.590 align:middle line:84% So this is another really interesting example 00:14:16.590 --> 00:14:18.670 align:middle line:84% where motion and depth seem to be really tightly 00:14:18.670 --> 00:14:19.370 align:middle line:90% Yoked. 00:14:19.370 --> 00:14:24.490 align:middle line:84% So I'll show you this one video about this. 00:14:24.490 --> 00:14:29.590 align:middle line:84% And then we'll experience this ourselves in a sense. 00:14:29.590 --> 00:14:30.810 align:middle line:90% One second. 00:14:30.810 --> 00:14:36.425 align:middle line:90% 00:14:36.425 --> 00:14:37.830 align:middle line:90% [VIDEO PLAYBACK] 00:14:37.830 --> 00:14:40.190 align:middle line:84% - We've had an Einstein mask for many years. 00:14:40.190 --> 00:14:43.577 align:middle line:84% Now we have one that's fully painted, decorated by an artist. 00:14:43.577 --> 00:14:45.910 align:middle line:84% And in fact, with all the flesh tones and the white hair 00:14:45.910 --> 00:14:48.030 align:middle line:84% and so on, it's remarkable because it really does 00:14:48.030 --> 00:14:49.770 align:middle line:90% look like Einstein himself. 00:14:49.770 --> 00:14:52.008 align:middle line:84% But this is a hollow face, don't forget. 00:14:52.008 --> 00:14:53.550 align:middle line:84% Here is the piece being turned around 00:14:53.550 --> 00:14:55.870 align:middle line:84% very slowly to show that it's hollow that side. 00:14:55.870 --> 00:14:58.153 align:middle line:84% And then this side is all in gray. 00:14:58.153 --> 00:15:00.070 align:middle line:84% You're not supposed to see this side normally. 00:15:00.070 --> 00:15:02.152 align:middle line:90% It's just the back of the mask. 00:15:02.152 --> 00:15:03.110 align:middle line:90% We keep turning around. 00:15:03.110 --> 00:15:07.070 align:middle line:84% And there's a nice effect here as he suddenly pops out to you 00:15:07.070 --> 00:15:08.570 align:middle line:84% because although it's a hollow face, 00:15:08.570 --> 00:15:10.550 align:middle line:84% he's going to suddenly pop in your imagination 00:15:10.550 --> 00:15:13.150 align:middle line:90% to be a full face. 00:15:13.150 --> 00:15:14.710 align:middle line:84% An effect like that, also, you see 00:15:14.710 --> 00:15:17.310 align:middle line:84% very remarkable is when you tilt the piece back and forth 00:15:17.310 --> 00:15:17.810 align:middle line:90% like that. 00:15:17.810 --> 00:15:19.830 align:middle line:84% You get some very, very strange effects 00:15:19.830 --> 00:15:22.650 align:middle line:84% of the head tilting upwards and downwards. 00:15:22.650 --> 00:15:25.750 align:middle line:84% And the eyes seem to follow you up and down. 00:15:25.750 --> 00:15:28.990 align:middle line:84% And now if we just leave the mask on the table 00:15:28.990 --> 00:15:31.750 align:middle line:84% and move the camera, you'll get some very interesting 00:15:31.750 --> 00:15:33.750 align:middle line:90% extra effects as well-- 00:15:33.750 --> 00:15:34.970 align:middle line:90% hollow-face illusion. 00:15:34.970 --> 00:15:37.730 align:middle line:90% 00:15:37.730 --> 00:15:38.640 align:middle line:90% [END PLAYBACK] 00:15:38.640 --> 00:15:43.400 align:middle line:84% JOSH MCDERMOTT: Greatly enhanced by the dramatic exposition. 00:15:43.400 --> 00:15:44.780 align:middle line:90% OK, so what's going on there? 00:15:44.780 --> 00:15:50.560 align:middle line:84% So this is a case where you look at this thing 00:15:50.560 --> 00:15:53.920 align:middle line:84% where the depth is actually the opposite of what 00:15:53.920 --> 00:15:55.180 align:middle line:90% you see it to be, right? 00:15:55.180 --> 00:15:57.280 align:middle line:84% So the actual structure in the world 00:15:57.280 --> 00:16:00.080 align:middle line:84% is the opposite of what you see it to be. 00:16:00.080 --> 00:16:04.190 align:middle line:90% And then it moves. 00:16:04.190 --> 00:16:07.260 align:middle line:84% And what's so freaky about looking at this thing 00:16:07.260 --> 00:16:12.200 align:middle line:84% is that the face appears to move in the direction that's 00:16:12.200 --> 00:16:15.100 align:middle line:84% opposite to the direction it is actually rotating. 00:16:15.100 --> 00:16:20.720 align:middle line:84% And so this, at some level, is a simple consequence 00:16:20.720 --> 00:16:22.960 align:middle line:84% of geometry, which is that if you have 00:16:22.960 --> 00:16:26.940 align:middle line:84% a three-dimensional thing and it rotates in one direction, 00:16:26.940 --> 00:16:29.737 align:middle line:84% the points at the back are going to move physically in one 00:16:29.737 --> 00:16:31.320 align:middle line:84% direction, and the points at the front 00:16:31.320 --> 00:16:33.180 align:middle line:84% will move physically in the other direction. 00:16:33.180 --> 00:16:37.888 align:middle line:84% And so if you take an image of this, 00:16:37.888 --> 00:16:39.680 align:middle line:84% that stimulus-- and we saw examples of this 00:16:39.680 --> 00:16:41.180 align:middle line:84% where-- remember that stimulus where 00:16:41.180 --> 00:16:43.880 align:middle line:84% you have that thing that looks like a sphere of dots that's 00:16:43.880 --> 00:16:45.120 align:middle line:90% rotating around? 00:16:45.120 --> 00:16:46.660 align:middle line:90% And it's bistable, right? 00:16:46.660 --> 00:16:49.160 align:middle line:84% You can see it as rotating either direction. 00:16:49.160 --> 00:16:52.120 align:middle line:84% But when it switches direction, the depth also flips. 00:16:52.120 --> 00:16:55.080 align:middle line:84% So in that case, it's not very striking 00:16:55.080 --> 00:16:57.877 align:middle line:84% because both depth interpretations basically 00:16:57.877 --> 00:16:58.460 align:middle line:90% look the same. 00:16:58.460 --> 00:16:59.620 align:middle line:90% It's just a sphere of dots. 00:16:59.620 --> 00:17:04.598 align:middle line:84% But in this case, that has the consequence that-- 00:17:04.598 --> 00:17:06.640 align:middle line:84% and particularly, in this case, part of the thing 00:17:06.640 --> 00:17:08.920 align:middle line:84% is reversing, which is the part that's the face. 00:17:08.920 --> 00:17:13.599 align:middle line:84% And so because of this kind of geometric ambiguity 00:17:13.599 --> 00:17:16.760 align:middle line:84% about the depth of the points and the direction of motion, 00:17:16.760 --> 00:17:19.560 align:middle line:84% there are these two possible interpretations. 00:17:19.560 --> 00:17:25.335 align:middle line:84% One is the actual depth of the thing-- depth interpretation 00:17:25.335 --> 00:17:26.960 align:middle line:84% of the thing, which is that it's hollow 00:17:26.960 --> 00:17:28.210 align:middle line:90% and rotating in one direction. 00:17:28.210 --> 00:17:30.700 align:middle line:84% The other is the depth that you see, 00:17:30.700 --> 00:17:32.360 align:middle line:90% which is where it's poking out. 00:17:32.360 --> 00:17:34.830 align:middle line:84% But in order for it to be poking out 00:17:34.830 --> 00:17:37.370 align:middle line:84% and for the image motion to be what it is, 00:17:37.370 --> 00:17:39.250 align:middle line:84% the actual face has to be rotating 00:17:39.250 --> 00:17:42.770 align:middle line:84% in the opposite direction from the direction it actually is 00:17:42.770 --> 00:17:45.310 align:middle line:84% and from the direction that the frame is. 00:17:45.310 --> 00:17:48.112 align:middle line:84% So that's essentially what is going on here. 00:17:48.112 --> 00:17:50.070 align:middle line:84% And so what does this tell us about perception? 00:17:50.070 --> 00:17:54.290 align:middle line:84% Well it's an additional piece of evidence 00:17:54.290 --> 00:17:56.290 align:middle line:84% that the three-dimensional motion interpretation 00:17:56.290 --> 00:17:58.450 align:middle line:84% that we make about the world is very coupled 00:17:58.450 --> 00:17:59.650 align:middle line:90% to the depth interpretation. 00:17:59.650 --> 00:18:01.530 align:middle line:90% Those things go hand in hand. 00:18:01.530 --> 00:18:03.290 align:middle line:84% So the visual system in this case 00:18:03.290 --> 00:18:06.490 align:middle line:84% is tricked into the wrong depth interpretation. 00:18:06.490 --> 00:18:10.130 align:middle line:84% And the way that it's tricked is in part because you have 00:18:10.130 --> 00:18:13.130 align:middle line:84% a very strong prior that faces are sticking out 00:18:13.130 --> 00:18:14.530 align:middle line:90% rather than hollow. 00:18:14.530 --> 00:18:16.930 align:middle line:84% And in the version in the video, the face 00:18:16.930 --> 00:18:19.370 align:middle line:84% was painted to also accentuate the three-dimensional 00:18:19.370 --> 00:18:20.610 align:middle line:90% interpretation. 00:18:20.610 --> 00:18:23.070 align:middle line:84% So you get tricked into the wrong depth interpretation. 00:18:23.070 --> 00:18:26.010 align:middle line:84% And then the 3D motion interpretation also has to be 00:18:26.010 --> 00:18:29.290 align:middle line:84% incorrect in order to be inconsistent. 00:18:29.290 --> 00:18:29.790 align:middle line:90% All right. 00:18:29.790 --> 00:18:36.062 align:middle line:84% So I'm going to show you a real-life example of this. 00:18:36.062 --> 00:18:38.390 align:middle line:84% In this class, you learn a lot of party tricks, 00:18:38.390 --> 00:18:40.370 align:middle line:90% and this is yet another one. 00:18:40.370 --> 00:18:44.690 align:middle line:84% So what I have for you here is the wire frame 00:18:44.690 --> 00:18:47.810 align:middle line:84% that is affixed to the end of a champagne bottle. 00:18:47.810 --> 00:18:50.010 align:middle line:90% I have three of them. 00:18:50.010 --> 00:18:52.110 align:middle line:84% That's all I could find in my office. 00:18:52.110 --> 00:18:54.730 align:middle line:84% We keep these every time someone graduates. 00:18:54.730 --> 00:18:59.110 align:middle line:84% We open champagne, and then I keep these to show all of you. 00:18:59.110 --> 00:19:01.190 align:middle line:90% So why is this important? 00:19:01.190 --> 00:19:04.450 align:middle line:84% Well, so this thing, you can think 00:19:04.450 --> 00:19:06.990 align:middle line:84% of this as like a three-dimensional Necker cube. 00:19:06.990 --> 00:19:10.610 align:middle line:84% So it's got two faces and these edges. 00:19:10.610 --> 00:19:12.210 align:middle line:84% Normally, you hold this thing up, 00:19:12.210 --> 00:19:13.930 align:middle line:84% and you look at it with two eyes. 00:19:13.930 --> 00:19:16.270 align:middle line:84% And so the depth interpretation is unambiguous. 00:19:16.270 --> 00:19:21.070 align:middle line:84% So right now, this thing is closer to me than this thing. 00:19:21.070 --> 00:19:25.490 align:middle line:84% But if you close one eye and you wait a little while, 00:19:25.490 --> 00:19:27.550 align:middle line:90% the depth will reverse. 00:19:27.550 --> 00:19:31.700 align:middle line:84% Now, when you rotate this thing, the most psychedelic thing 00:19:31.700 --> 00:19:33.820 align:middle line:84% you can imagine will happen, which 00:19:33.820 --> 00:19:37.380 align:middle line:84% is that you will see the wireframe as rotating 00:19:37.380 --> 00:19:39.760 align:middle line:84% in the opposite direction as your hand. 00:19:39.760 --> 00:19:42.180 align:middle line:84% So the wire will look like it's non-rigid. 00:19:42.180 --> 00:19:44.940 align:middle line:84% So there are three examples here. 00:19:44.940 --> 00:19:46.780 align:middle line:84% I'm going to pass this around, and you 00:19:46.780 --> 00:19:47.915 align:middle line:90% can try this for yourself. 00:19:47.915 --> 00:19:49.040 align:middle line:90% It may take a couple tries. 00:19:49.040 --> 00:19:51.140 align:middle line:84% So again, the method here is you're 00:19:51.140 --> 00:19:53.580 align:middle line:84% going to look at this with one eye 00:19:53.580 --> 00:19:57.620 align:middle line:84% and wait for the depth to reverse and then rotate it. 00:19:57.620 --> 00:19:59.920 align:middle line:90% And try not to scream, OK? 00:19:59.920 --> 00:20:03.260 align:middle line:90% 00:20:03.260 --> 00:20:04.780 align:middle line:90% There you go. 00:20:04.780 --> 00:20:06.500 align:middle line:90% And pass those around. 00:20:06.500 --> 00:20:07.000 align:middle line:90% OK. 00:20:07.000 --> 00:20:13.500 align:middle line:90% 00:20:13.500 --> 00:20:16.720 align:middle line:84% So I'm going to keep lecturing while you all try this yourself. 00:20:16.720 --> 00:20:18.360 align:middle line:90% It may take a couple tries. 00:20:18.360 --> 00:20:21.613 align:middle line:90% 00:20:21.613 --> 00:20:23.280 align:middle line:84% And I should just say, just to be clear, 00:20:23.280 --> 00:20:25.340 align:middle line:84% this is exactly the same phenomena 00:20:25.340 --> 00:20:27.300 align:middle line:90% as this hollow-face illusion. 00:20:27.300 --> 00:20:30.500 align:middle line:84% What's happening is that we are waiting 00:20:30.500 --> 00:20:34.380 align:middle line:84% for your brain to arrive at the incorrect depth interpretation. 00:20:34.380 --> 00:20:35.780 align:middle line:90% We're then moving the thing. 00:20:35.780 --> 00:20:38.900 align:middle line:90% 00:20:38.900 --> 00:20:42.620 align:middle line:84% Your brain is then forced to make the correct motion 00:20:42.620 --> 00:20:45.380 align:middle line:84% interpretation to be consistent with the incorrect depth 00:20:45.380 --> 00:20:48.880 align:middle line:84% interpretation, even though it seems physically impossible. 00:20:48.880 --> 00:20:50.220 align:middle line:90% Does that work for you? 00:20:50.220 --> 00:20:53.100 align:middle line:90% Yeah, OK. 00:20:53.100 --> 00:20:54.900 align:middle line:90% OK. 00:20:54.900 --> 00:20:59.900 align:middle line:84% Pop quiz here-- name some depth cues and depth-related phenomena 00:20:59.900 --> 00:21:00.480 align:middle line:90% in here. 00:21:00.480 --> 00:21:06.200 align:middle line:90% 00:21:06.200 --> 00:21:06.700 align:middle line:90% Yep? 00:21:06.700 --> 00:21:07.620 align:middle line:90% AUDIENCE: Object size. 00:21:07.620 --> 00:21:12.068 align:middle line:90% 00:21:12.068 --> 00:21:12.860 align:middle line:90% JOSH MCDERMOTT: OK. 00:21:12.860 --> 00:21:15.980 align:middle line:90% Are you referring to that one? 00:21:15.980 --> 00:21:18.140 align:middle line:84% AUDIENCE: I'm referring to all of them, some 00:21:18.140 --> 00:21:21.260 align:middle line:84% being the object's closer or bigger than the objects 00:21:21.260 --> 00:21:22.200 align:middle line:90% further away. 00:21:22.200 --> 00:21:23.125 align:middle line:90% And that one, that's-- 00:21:23.125 --> 00:21:24.000 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:21:24.000 --> 00:21:25.940 align:middle line:84% So that's kind of like Emmert's law, where 00:21:25.940 --> 00:21:28.690 align:middle line:84% you know that all of these things are the same size, 00:21:28.690 --> 00:21:31.350 align:middle line:84% and so that kind of implies that those things are further away. 00:21:31.350 --> 00:21:34.350 align:middle line:84% But down here, we have the opposite application 00:21:34.350 --> 00:21:36.950 align:middle line:84% of Emmert's law, where the inference, 00:21:36.950 --> 00:21:40.950 align:middle line:84% if this is a lot closer than that, is that that's very close. 00:21:40.950 --> 00:21:43.010 align:middle line:84% And thus, it must be a toy or something. 00:21:43.010 --> 00:21:45.850 align:middle line:84% So that's Emmert's law applied that way. 00:21:45.850 --> 00:21:50.070 align:middle line:84% What are some other depth cues that you can see here? 00:21:50.070 --> 00:21:51.043 align:middle line:90% Yep? 00:21:51.043 --> 00:21:52.210 align:middle line:90% AUDIENCE: I forgot the name. 00:21:52.210 --> 00:21:54.470 align:middle line:84% But in the background, the mountains are faded. 00:21:54.470 --> 00:21:54.930 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:21:54.930 --> 00:21:56.550 align:middle line:84% So that's what people call aerial perspective, yeah. 00:21:56.550 --> 00:21:58.650 align:middle line:84% So the stuff gets blurry when it's further away. 00:21:58.650 --> 00:21:59.870 align:middle line:90% Yep. 00:21:59.870 --> 00:22:02.568 align:middle line:90% Anything else? 00:22:02.568 --> 00:22:03.910 align:middle line:90% AUDIENCE: Linear perspective. 00:22:03.910 --> 00:22:04.230 align:middle line:90% JOSH MCDERMOTT: Yep. 00:22:04.230 --> 00:22:05.930 align:middle line:84% You've got some converging lines there. 00:22:05.930 --> 00:22:06.430 align:middle line:90% Yep. 00:22:06.430 --> 00:22:07.370 align:middle line:90% OK, good. 00:22:07.370 --> 00:22:10.790 align:middle line:90% 00:22:10.790 --> 00:22:12.430 align:middle line:90% How about here? 00:22:12.430 --> 00:22:14.050 align:middle line:90% What are some depth cues here? 00:22:14.050 --> 00:22:17.110 align:middle line:90% 00:22:17.110 --> 00:22:17.670 align:middle line:90% Yep? 00:22:17.670 --> 00:22:19.070 align:middle line:90% AUDIENCE: Shadow and light. 00:22:19.070 --> 00:22:19.590 align:middle line:90% JOSH MCDERMOTT: Shadow? 00:22:19.590 --> 00:22:20.130 align:middle line:90% Yep. 00:22:20.130 --> 00:22:22.550 align:middle line:84% So we got some shadows here that tell you that that's off 00:22:22.550 --> 00:22:24.790 align:middle line:90% of the ground plane. 00:22:24.790 --> 00:22:25.650 align:middle line:90% Anything else? 00:22:25.650 --> 00:22:33.070 align:middle line:90% 00:22:33.070 --> 00:22:38.070 align:middle line:84% How are you inferring the 3D shape of those objects? 00:22:38.070 --> 00:22:40.230 align:middle line:90% AUDIENCE: Shading. 00:22:40.230 --> 00:22:41.590 align:middle line:90% AUDIENCE: Shadow [INAUDIBLE]. 00:22:41.590 --> 00:22:42.150 align:middle line:90% JOSH MCDERMOTT: Shadows. 00:22:42.150 --> 00:22:43.710 align:middle line:84% And somebody said something else? 00:22:43.710 --> 00:22:44.150 align:middle line:90% AUDIENCE: Shading. 00:22:44.150 --> 00:22:44.450 align:middle line:90% JOSH MCDERMOTT: Shading, yeah. 00:22:44.450 --> 00:22:46.117 align:middle line:84% So there's a lot of shading here, right? 00:22:46.117 --> 00:22:48.850 align:middle line:90% 00:22:48.850 --> 00:22:50.930 align:middle line:84% And then there's also occlusion-- 00:22:50.930 --> 00:22:52.790 align:middle line:84% so the fact that this object looks 00:22:52.790 --> 00:22:54.450 align:middle line:90% like it's going under that one. 00:22:54.450 --> 00:22:58.510 align:middle line:84% So we didn't really talk a whole lot about that. 00:22:58.510 --> 00:23:00.130 align:middle line:90% What are some depth cues here? 00:23:00.130 --> 00:23:05.390 align:middle line:90% 00:23:05.390 --> 00:23:08.030 align:middle line:90% AUDIENCE: Shadow. 00:23:08.030 --> 00:23:10.150 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:23:10.150 --> 00:23:10.830 align:middle line:90% What else? 00:23:10.830 --> 00:23:16.710 align:middle line:90% 00:23:16.710 --> 00:23:17.585 align:middle line:90% AUDIENCE: Size. 00:23:17.585 --> 00:23:18.710 align:middle line:90% JOSH MCDERMOTT: Size, yeah. 00:23:18.710 --> 00:23:20.670 align:middle line:84% So this is kind of like a texture gradient. 00:23:20.670 --> 00:23:22.650 align:middle line:90% What else? 00:23:22.650 --> 00:23:23.150 align:middle line:90% Anything? 00:23:23.150 --> 00:23:25.720 align:middle line:90% 00:23:25.720 --> 00:23:28.160 align:middle line:84% Linear perspective because the knobs 00:23:28.160 --> 00:23:30.380 align:middle line:84% are arranged in these lines, and those converge. 00:23:30.380 --> 00:23:33.080 align:middle line:90% 00:23:33.080 --> 00:23:35.000 align:middle line:84% This is actually a very interesting case 00:23:35.000 --> 00:23:39.540 align:middle line:84% where there's some shading on these objects. 00:23:39.540 --> 00:23:42.040 align:middle line:84% But these objects are not Lambertian. 00:23:42.040 --> 00:23:44.480 align:middle line:84% So you can see specular reflections here. 00:23:44.480 --> 00:23:47.200 align:middle line:84% And this is, in practice, what makes the problem of shape 00:23:47.200 --> 00:23:52.200 align:middle line:84% from shading very challenging, is that a lot of surfaces 00:23:52.200 --> 00:23:54.120 align:middle line:90% are not purely Lambertian. 00:23:54.120 --> 00:23:55.720 align:middle line:84% You get specular reflections, so you 00:23:55.720 --> 00:23:58.600 align:middle line:84% have to distinguish that from shading and from reflectance 00:23:58.600 --> 00:24:01.435 align:middle line:90% and so forth. 00:24:01.435 --> 00:24:02.060 align:middle line:90% How about here? 00:24:02.060 --> 00:24:03.380 align:middle line:90% What are some depth cues here? 00:24:03.380 --> 00:24:07.860 align:middle line:90% 00:24:07.860 --> 00:24:08.360 align:middle line:90% Yep? 00:24:08.360 --> 00:24:11.847 align:middle line:84% AUDIENCE: There's the aerial view of that or something. 00:24:11.847 --> 00:24:12.680 align:middle line:90% JOSH MCDERMOTT: Yep. 00:24:12.680 --> 00:24:14.480 align:middle line:90% Aerial perspective, yep. 00:24:14.480 --> 00:24:15.480 align:middle line:90% Yep? 00:24:15.480 --> 00:24:18.447 align:middle line:84% AUDIENCE: Linear perspective or [INAUDIBLE]. 00:24:18.447 --> 00:24:19.280 align:middle line:90% JOSH MCDERMOTT: Yep. 00:24:19.280 --> 00:24:20.340 align:middle line:90% Yep, linear perspective. 00:24:20.340 --> 00:24:20.840 align:middle line:90% Yep. 00:24:20.840 --> 00:24:21.460 align:middle line:90% Anything else? 00:24:21.460 --> 00:24:22.240 align:middle line:90% Yeah? 00:24:22.240 --> 00:24:27.020 align:middle line:84% AUDIENCE: I think you can also assume that things are the same, 00:24:27.020 --> 00:24:30.043 align:middle line:84% so they get smaller and further away. 00:24:30.043 --> 00:24:31.460 align:middle line:84% JOSH MCDERMOTT: Yeah, that's true. 00:24:31.460 --> 00:24:33.160 align:middle line:90% Yeah. 00:24:33.160 --> 00:24:33.860 align:middle line:90% What else? 00:24:33.860 --> 00:24:39.120 align:middle line:90% 00:24:39.120 --> 00:24:42.713 align:middle line:90% AUDIENCE: The back is lighter. 00:24:42.713 --> 00:24:44.380 align:middle line:84% JOSH MCDERMOTT: Yes, aerial perspective. 00:24:44.380 --> 00:24:45.180 align:middle line:90% Yeah, yeah. 00:24:45.180 --> 00:24:46.860 align:middle line:84% And then also some texture gradients. 00:24:46.860 --> 00:24:48.860 align:middle line:84% So the bricks gives you a texture gradient. 00:24:48.860 --> 00:24:49.360 align:middle line:90% Yeah. 00:24:49.360 --> 00:24:51.600 align:middle line:90% OK. 00:24:51.600 --> 00:24:55.320 align:middle line:84% So just to summarize what we talked about in terms 00:24:55.320 --> 00:24:58.080 align:middle line:90% of monocular depth perception-- 00:24:58.080 --> 00:25:00.080 align:middle line:84% so as you could probably tell from the lecture, 00:25:00.080 --> 00:25:02.760 align:middle line:84% I mean, our understanding of these issues, I would say, 00:25:02.760 --> 00:25:04.580 align:middle line:90% is still mostly qualitative. 00:25:04.580 --> 00:25:09.300 align:middle line:84% There's some sets of cues and associated phenomena. 00:25:09.300 --> 00:25:11.280 align:middle line:84% We don't yet have working models that can 00:25:11.280 --> 00:25:13.200 align:middle line:90% account for human perception. 00:25:13.200 --> 00:25:15.240 align:middle line:84% This is something that I think merits 00:25:15.240 --> 00:25:18.543 align:middle line:90% revisiting in the current era. 00:25:18.543 --> 00:25:19.960 align:middle line:84% So we talked about a bunch of cues 00:25:19.960 --> 00:25:23.850 align:middle line:84% and a bunch of phenomena that give us 00:25:23.850 --> 00:25:26.768 align:middle line:84% some clues to what an eventual working model ought 00:25:26.768 --> 00:25:27.310 align:middle line:90% to look like. 00:25:27.310 --> 00:25:29.130 align:middle line:84% So we talked about shape from shading. 00:25:29.130 --> 00:25:31.690 align:middle line:84% Key ideas there is that shape from shading typically 00:25:31.690 --> 00:25:34.650 align:middle line:84% assumes Lambertian surfaces as well as no variation 00:25:34.650 --> 00:25:37.570 align:middle line:84% in reflectance and known illumination. 00:25:37.570 --> 00:25:39.770 align:middle line:84% We often assume that the illumination 00:25:39.770 --> 00:25:42.010 align:middle line:90% is coming from above. 00:25:42.010 --> 00:25:44.090 align:middle line:84% Shadows are a powerful depth cue. 00:25:44.090 --> 00:25:45.730 align:middle line:90% They have to be dark. 00:25:45.730 --> 00:25:47.650 align:middle line:84% It can help for them to be blurry. 00:25:47.650 --> 00:25:49.690 align:middle line:84% We talked about textures gradients. 00:25:49.690 --> 00:25:52.790 align:middle line:84% Talked about Emmert's law that relates size, distance, 00:25:52.790 --> 00:25:55.290 align:middle line:84% and visual angle, and different ways 00:25:55.290 --> 00:25:58.170 align:middle line:84% in which that can explain illusions that are 00:25:58.170 --> 00:26:00.170 align:middle line:90% related to depth perception. 00:26:00.170 --> 00:26:04.930 align:middle line:84% Talked about how what we see are the inferred 3D shapes that we 00:26:04.930 --> 00:26:06.150 align:middle line:90% infer from 2D images. 00:26:06.150 --> 00:26:09.130 align:middle line:84% It's very hard to access the 2D image that we use to infer 00:26:09.130 --> 00:26:10.370 align:middle line:90% the shape. 00:26:10.370 --> 00:26:12.195 align:middle line:84% We saw examples of impossible objects 00:26:12.195 --> 00:26:13.570 align:middle line:84% and talked about how they suggest 00:26:13.570 --> 00:26:15.930 align:middle line:84% that depth representations are computed locally. 00:26:15.930 --> 00:26:17.530 align:middle line:84% And then we saw these phenomena that 00:26:17.530 --> 00:26:19.900 align:middle line:84% illustrate that motion and depth are jointly inferred. 00:26:19.900 --> 00:26:21.650 align:middle line:84% And we also saw some examples of that back 00:26:21.650 --> 00:26:25.970 align:middle line:84% in the lecture on motion perception. 00:26:25.970 --> 00:26:27.150 align:middle line:90% What questions you got? 00:26:27.150 --> 00:26:31.090 align:middle line:90% 00:26:31.090 --> 00:26:34.050 align:middle line:84% Is everyone getting the wireframe thing to work? 00:26:34.050 --> 00:26:36.770 align:middle line:90% Did anybody not get it to work? 00:26:36.770 --> 00:26:37.530 align:middle line:90% All right, good. 00:26:37.530 --> 00:26:38.947 align:middle line:84% That's a pretty good success rate. 00:26:38.947 --> 00:26:40.390 align:middle line:90% Are you impressed? 00:26:40.390 --> 00:26:40.890 align:middle line:90% Yeah? 00:26:40.890 --> 00:26:44.370 align:middle line:90% OK, good. 00:26:44.370 --> 00:26:44.990 align:middle line:90% All right. 00:26:44.990 --> 00:26:51.482 align:middle line:84% So let's turn to binocular stereopsis. 00:26:51.482 --> 00:26:53.690 align:middle line:84% So we just talked about how there's lots of monocular 00:26:53.690 --> 00:26:56.272 align:middle line:90% accused of 3D shape and depth. 00:26:56.272 --> 00:26:57.730 align:middle line:84% But another big piece of the puzzle 00:26:57.730 --> 00:26:59.490 align:middle line:84% is the fact that you have two eyes. 00:26:59.490 --> 00:27:04.610 align:middle line:84% So if you look across the animal kingdom, 00:27:04.610 --> 00:27:07.210 align:middle line:84% organisms tend to come in two flavors 00:27:07.210 --> 00:27:09.430 align:middle line:84% when it comes to the positions of the eyes. 00:27:09.430 --> 00:27:12.570 align:middle line:84% So there are some critters that have the eyes positioned 00:27:12.570 --> 00:27:15.290 align:middle line:90% on the sides of the head. 00:27:15.290 --> 00:27:17.530 align:middle line:90% So here are a couple examples. 00:27:17.530 --> 00:27:20.520 align:middle line:84% There are others that have the eyes positioned at the front. 00:27:20.520 --> 00:27:24.260 align:middle line:90% 00:27:24.260 --> 00:27:27.300 align:middle line:84% In general, the animals that have eyes in the front 00:27:27.300 --> 00:27:30.340 align:middle line:84% are what we consider to be predators. 00:27:30.340 --> 00:27:34.340 align:middle line:90% They hunt their prey. 00:27:34.340 --> 00:27:36.900 align:middle line:84% Animals that have eyes on the side of their head 00:27:36.900 --> 00:27:39.020 align:middle line:84% are often the ones that are probably 00:27:39.020 --> 00:27:40.700 align:middle line:90% worried about getting eaten. 00:27:40.700 --> 00:27:46.180 align:middle line:84% So the notion here is that animals 00:27:46.180 --> 00:27:51.540 align:middle line:84% that are in the business of hunting for subsistence 00:27:51.540 --> 00:27:55.500 align:middle line:84% need to know how far away things are from them very accurately. 00:27:55.500 --> 00:27:58.160 align:middle line:84% And so they have two eyes on the front of their head 00:27:58.160 --> 00:28:00.640 align:middle line:84% because two eyes, as we will discover, 00:28:00.640 --> 00:28:05.632 align:middle line:84% really helps to give you precise depth perception. 00:28:05.632 --> 00:28:07.840 align:middle line:84% Whereas animals that are worried about getting eaten, 00:28:07.840 --> 00:28:09.257 align:middle line:84% it's more important for them to be 00:28:09.257 --> 00:28:12.360 align:middle line:84% able to see as much as they can at any one time, 00:28:12.360 --> 00:28:18.460 align:middle line:84% and so the eyes are positioned to give panoramic vision. 00:28:18.460 --> 00:28:20.900 align:middle line:84% So there's a real cost to having two eyes in the front 00:28:20.900 --> 00:28:22.700 align:middle line:90% of our head like we do. 00:28:22.700 --> 00:28:25.520 align:middle line:84% So it greatly reduces the field of view. 00:28:25.520 --> 00:28:29.020 align:middle line:84% So if you just look ahead and you 00:28:29.020 --> 00:28:32.940 align:middle line:84% look at what the most peripheral thing is that you can see, 00:28:32.940 --> 00:28:37.480 align:middle line:84% you get, I don't know, a good chunk. 00:28:37.480 --> 00:28:40.927 align:middle line:84% But you definitely can't really see more than maybe 150 degrees, 00:28:40.927 --> 00:28:41.760 align:middle line:90% something like that. 00:28:41.760 --> 00:28:44.220 align:middle line:84% So there's this huge part of the visual world 00:28:44.220 --> 00:28:45.980 align:middle line:90% that is not visible to you. 00:28:45.980 --> 00:28:49.980 align:middle line:84% But what you get in return for that 00:28:49.980 --> 00:28:53.240 align:middle line:84% is disparities between the two eyes. 00:28:53.240 --> 00:28:56.000 align:middle line:84% So your two eyes are in slightly different positions. 00:28:56.000 --> 00:28:57.540 align:middle line:84% And so the consequence is that they 00:28:57.540 --> 00:29:00.740 align:middle line:84% take slightly different images of the world. 00:29:00.740 --> 00:29:03.180 align:middle line:84% And those differences between the two eyes 00:29:03.180 --> 00:29:05.420 align:middle line:84% provide information that allow us 00:29:05.420 --> 00:29:10.890 align:middle line:84% to discriminate very small differences in relative depth. 00:29:10.890 --> 00:29:13.760 align:middle line:84% So here's a demo that we're all going to do. 00:29:13.760 --> 00:29:17.190 align:middle line:84% So you're going to hold your two fingers out in front of you. 00:29:17.190 --> 00:29:22.150 align:middle line:84% Fixate the one that is close to you 00:29:22.150 --> 00:29:27.010 align:middle line:84% and then alternately close and open the two eyes. 00:29:27.010 --> 00:29:30.590 align:middle line:84% So you're going to look at it with one eye and then the other. 00:29:30.590 --> 00:29:34.870 align:middle line:84% And what you will see is that the distance between the two 00:29:34.870 --> 00:29:37.910 align:middle line:84% eyes in the image will be different for the left eye 00:29:37.910 --> 00:29:39.070 align:middle line:90% and the right eye. 00:29:39.070 --> 00:29:45.070 align:middle line:84% So that's an example of binocular disparity. 00:29:45.070 --> 00:29:47.390 align:middle line:84% And so in this case, that gives you information 00:29:47.390 --> 00:29:51.330 align:middle line:84% that one of the fingers is further away than the other. 00:29:51.330 --> 00:29:57.350 align:middle line:90% 00:29:57.350 --> 00:29:57.850 align:middle line:90% All right. 00:29:57.850 --> 00:30:02.630 align:middle line:84% So binocular disparity refers to differences in positions 00:30:02.630 --> 00:30:04.790 align:middle line:90% between the two eyes. 00:30:04.790 --> 00:30:10.710 align:middle line:84% So in a situation where one object is being fixated-- let's 00:30:10.710 --> 00:30:13.590 align:middle line:90% say that's object a here. 00:30:13.590 --> 00:30:18.710 align:middle line:84% If b is not being fixated and is at a different depth, 00:30:18.710 --> 00:30:20.510 align:middle line:84% then the relative position of b will 00:30:20.510 --> 00:30:23.510 align:middle line:84% be different in the left compared to the right eye, 00:30:23.510 --> 00:30:26.090 align:middle line:90% and that's the disparity. 00:30:26.090 --> 00:30:28.110 align:middle line:84% And so we typically measure the disparity 00:30:28.110 --> 00:30:32.470 align:middle line:84% in degrees of visual angle or minutes of visual angle. 00:30:32.470 --> 00:30:34.230 align:middle line:90% So here's a schematic. 00:30:34.230 --> 00:30:36.330 align:middle line:90% So here are your two eyes. 00:30:36.330 --> 00:30:38.358 align:middle line:84% You're fixating this red object here. 00:30:38.358 --> 00:30:40.150 align:middle line:84% Remember, what it means to fixate something 00:30:40.150 --> 00:30:42.770 align:middle line:84% is that your fovea is directed towards that object. 00:30:42.770 --> 00:30:47.870 align:middle line:84% And so the red point is projecting onto the fovea 00:30:47.870 --> 00:30:49.550 align:middle line:90% a portion of the retina. 00:30:49.550 --> 00:30:55.910 align:middle line:84% So this dark-blue thing, which is further away, 00:30:55.910 --> 00:30:58.550 align:middle line:84% also projects onto the left and the right eye. 00:30:58.550 --> 00:31:02.150 align:middle line:84% But you can see that it's at a different place in the left eye 00:31:02.150 --> 00:31:03.670 align:middle line:90% compared to the right eye. 00:31:03.670 --> 00:31:05.970 align:middle line:90% So that creates a disparity. 00:31:05.970 --> 00:31:09.570 align:middle line:90% 00:31:09.570 --> 00:31:11.950 align:middle line:84% So when fixating one object, the other object 00:31:11.950 --> 00:31:16.560 align:middle line:84% can be in front of or behind the focused or fixated object. 00:31:16.560 --> 00:31:20.860 align:middle line:84% If b is further away, we say that the disparity is uncrossed. 00:31:20.860 --> 00:31:23.300 align:middle line:84% By convention, we just consider that to be positive. 00:31:23.300 --> 00:31:25.780 align:middle line:84% If it's closer than a, we say that it's crossed. 00:31:25.780 --> 00:31:27.800 align:middle line:84% And by convention, we say that it's negative. 00:31:27.800 --> 00:31:29.480 align:middle line:84% And so the idea is that you would 00:31:29.480 --> 00:31:32.800 align:middle line:84% have to cross your eyes in one case to look at b. 00:31:32.800 --> 00:31:33.960 align:middle line:90% So here's the idea. 00:31:33.960 --> 00:31:37.720 align:middle line:84% So in this case, the blue object on the left 00:31:37.720 --> 00:31:39.440 align:middle line:90% has crossed disparity. 00:31:39.440 --> 00:31:42.940 align:middle line:84% And on the right, the red object has uncrossed disparity. 00:31:42.940 --> 00:31:45.420 align:middle line:84% So you'd have to uncross or cross your eyes to fixate it. 00:31:45.420 --> 00:31:48.980 align:middle line:90% 00:31:48.980 --> 00:31:52.460 align:middle line:84% So objects can also give rise to zero disparity or no disparity. 00:31:52.460 --> 00:31:55.640 align:middle line:84% So this happens when the lateral separation between objects 00:31:55.640 --> 00:31:57.600 align:middle line:90% is the same in both eyes. 00:31:57.600 --> 00:32:02.440 align:middle line:84% And the set of points that has this particular property 00:32:02.440 --> 00:32:04.520 align:middle line:84% is known as the horopter, also known 00:32:04.520 --> 00:32:06.200 align:middle line:90% as the Vieth-Muller circle. 00:32:06.200 --> 00:32:08.360 align:middle line:90% So there's this. 00:32:08.360 --> 00:32:11.740 align:middle line:84% And if you take a 2D slice here-- 00:32:11.740 --> 00:32:15.080 align:middle line:84% In reality, this is a 3D world-- but in this 2D slice, 00:32:15.080 --> 00:32:17.020 align:middle line:90% you get this circle here. 00:32:17.020 --> 00:32:21.280 align:middle line:84% And every point on this circle will 00:32:21.280 --> 00:32:25.320 align:middle line:84% project to corresponding points on the retina. 00:32:25.320 --> 00:32:28.360 align:middle line:84% And that would cause-- who would be willing to volunteer 00:32:28.360 --> 00:32:29.380 align:middle line:90% for this purpose? 00:32:29.380 --> 00:32:32.600 align:middle line:90% 00:32:32.600 --> 00:32:33.720 align:middle line:90% Preston. 00:32:33.720 --> 00:32:34.840 align:middle line:90% All right. 00:32:34.840 --> 00:32:35.960 align:middle line:90% Thank you. 00:32:35.960 --> 00:32:38.960 align:middle line:84% So you're going to need two pencils or two pens-- 00:32:38.960 --> 00:32:42.440 align:middle line:90% two things that are pointy. 00:32:42.440 --> 00:32:44.280 align:middle line:90% Do you have that? 00:32:44.280 --> 00:32:45.380 align:middle line:90% Maybe somebody can lend. 00:32:45.380 --> 00:32:48.200 align:middle line:90% I should have brought those. 00:32:48.200 --> 00:32:49.020 align:middle line:90% Yeah, that'll do. 00:32:49.020 --> 00:32:50.520 align:middle line:90% OK. 00:32:50.520 --> 00:32:52.900 align:middle line:90% OK, so just sit right here. 00:32:52.900 --> 00:32:55.460 align:middle line:90% 00:32:55.460 --> 00:32:59.600 align:middle line:84% So I want you to grab one in each hand. 00:32:59.600 --> 00:33:02.300 align:middle line:90% Point them towards each other. 00:33:02.300 --> 00:33:05.680 align:middle line:84% Now, using both eyes, I want you to bring them 00:33:05.680 --> 00:33:06.900 align:middle line:90% into contact with each other. 00:33:06.900 --> 00:33:08.560 align:middle line:90% Touch the tips. 00:33:08.560 --> 00:33:09.220 align:middle line:90% You're good. 00:33:09.220 --> 00:33:09.910 align:middle line:90% You're good. 00:33:09.910 --> 00:33:11.810 align:middle line:90% OK, do it again. 00:33:11.810 --> 00:33:14.170 align:middle line:90% Start further away. 00:33:14.170 --> 00:33:16.770 align:middle line:90% Now bring them into contact. 00:33:16.770 --> 00:33:17.270 align:middle line:90% Very good. 00:33:17.270 --> 00:33:19.210 align:middle line:90% OK, now bring them apart. 00:33:19.210 --> 00:33:20.370 align:middle line:90% Close one eye. 00:33:20.370 --> 00:33:23.370 align:middle line:90% Now do it again. 00:33:23.370 --> 00:33:24.150 align:middle line:90% Oh, my goodness. 00:33:24.150 --> 00:33:25.490 align:middle line:90% It's a miracle. 00:33:25.490 --> 00:33:26.410 align:middle line:90% [LAUGHTER] 00:33:26.410 --> 00:33:27.190 align:middle line:90% Try it again. 00:33:27.190 --> 00:33:30.890 align:middle line:90% 00:33:30.890 --> 00:33:31.590 align:middle line:90% Try it again. 00:33:31.590 --> 00:33:34.830 align:middle line:90% 00:33:34.830 --> 00:33:35.330 align:middle line:90% Yeah. 00:33:35.330 --> 00:33:37.110 align:middle line:90% So it's much harder, right? 00:33:37.110 --> 00:33:37.610 align:middle line:90% OK. 00:33:37.610 --> 00:33:40.770 align:middle line:84% So what you get from having two eyes 00:33:40.770 --> 00:33:43.410 align:middle line:84% is very, very, very accurate depth perception. 00:33:43.410 --> 00:33:46.130 align:middle line:84% So you have depth perception with one eye, 00:33:46.130 --> 00:33:48.313 align:middle line:90% But the acuity is much less. 00:33:48.313 --> 00:33:49.230 align:middle line:90% Preston, thanks a lot. 00:33:49.230 --> 00:33:50.850 align:middle line:90% Appreciate it. 00:33:50.850 --> 00:33:52.610 align:middle line:84% Let's give him round of applause. 00:33:52.610 --> 00:33:55.710 align:middle line:90% [APPLAUSE] 00:33:55.710 --> 00:33:57.550 align:middle line:84% And you should all try this yourselves. 00:33:57.550 --> 00:33:59.130 align:middle line:90% It works on everyone. 00:33:59.130 --> 00:34:02.810 align:middle line:84% So there's a really big difference here. 00:34:02.810 --> 00:34:06.410 align:middle line:84% So just to give you a quantitative example-- 00:34:06.410 --> 00:34:10.929 align:middle line:84% so using binocular stereopsis, you can discriminate 00:34:10.929 --> 00:34:14.330 align:middle line:84% a 1-millimeter difference in relative depth at a distance 00:34:14.330 --> 00:34:15.650 align:middle line:90% of 1 meter. 00:34:15.650 --> 00:34:17.770 align:middle line:90% So that's very, very small. 00:34:17.770 --> 00:34:20.969 align:middle line:84% And so the disparity that is created by that difference 00:34:20.969 --> 00:34:22.070 align:middle line:90% is tiny. 00:34:22.070 --> 00:34:25.030 align:middle line:84% It's that really small number of millimeters. 00:34:25.030 --> 00:34:28.482 align:middle line:84% So it's less than the diameter of a single photoreceptor-- 00:34:28.482 --> 00:34:30.690 align:middle line:84% so these very, very small differences between the two 00:34:30.690 --> 00:34:34.489 align:middle line:84% eyes that the brain is really attuned to because they tell you 00:34:34.489 --> 00:34:39.290 align:middle line:84% about fine-grained differences in depth. 00:34:39.290 --> 00:34:43.409 align:middle line:84% So one very compelling demonstration 00:34:43.409 --> 00:34:47.850 align:middle line:84% of binocular stereopsis makes use of this device here, 00:34:47.850 --> 00:34:49.989 align:middle line:90% called a stereoscope. 00:34:49.989 --> 00:34:55.370 align:middle line:84% So this was invented in the mid-1800s by Wheatstone. 00:34:55.370 --> 00:34:59.370 align:middle line:84% And so what the stereoscope does is 00:34:59.370 --> 00:35:03.090 align:middle line:84% it's a device for taking two images 00:35:03.090 --> 00:35:05.490 align:middle line:84% and projecting one to the left eye, one to the right eye. 00:35:05.490 --> 00:35:07.782 align:middle line:84% And so the two images are taken from slightly different 00:35:07.782 --> 00:35:11.580 align:middle line:84% positions, so they have slight disparities in them. 00:35:11.580 --> 00:35:14.900 align:middle line:84% And they cause you to see things in 3D. 00:35:14.900 --> 00:35:17.340 align:middle line:84% So this was a demonstration that you 00:35:17.340 --> 00:35:20.440 align:middle line:84% can get a vivid impression of depth from two flat pictures. 00:35:20.440 --> 00:35:25.605 align:middle line:84% And so here's another diagram of this. 00:35:25.605 --> 00:35:27.980 align:middle line:84% This particular version of it actually works a little bit 00:35:27.980 --> 00:35:28.480 align:middle line:90% differently. 00:35:28.480 --> 00:35:30.380 align:middle line:90% This uses prisms. 00:35:30.380 --> 00:35:32.800 align:middle line:84% But the classic version involves mirrors. 00:35:32.800 --> 00:35:36.780 align:middle line:84% So you have two images on either side and then two mirrors. 00:35:36.780 --> 00:35:39.120 align:middle line:90% SIRI: [INAUDIBLE] 00:35:39.120 --> 00:35:41.460 align:middle line:90% 00:35:41.460 --> 00:35:44.700 align:middle line:84% JOSH MCDERMOTT: My computer thinks I asked Siri about 00:35:44.700 --> 00:35:47.400 align:middle line:90% something, which I didn't. 00:35:47.400 --> 00:35:49.140 align:middle line:90% OK. 00:35:49.140 --> 00:35:51.480 align:middle line:84% OK, I must have said something that sounds like Siri. 00:35:51.480 --> 00:35:53.563 align:middle line:84% So you can tell that machine speech recognition is 00:35:53.563 --> 00:35:56.780 align:middle line:90% not quite where it needs to be. 00:35:56.780 --> 00:35:57.820 align:middle line:90% All right. 00:35:57.820 --> 00:36:01.160 align:middle line:84% So we've got these two images and two mirrors. 00:36:01.160 --> 00:36:03.360 align:middle line:84% And so each eye sees a different image. 00:36:03.360 --> 00:36:07.180 align:middle line:84% So in this particular version, there are two prisms here. 00:36:07.180 --> 00:36:10.020 align:middle line:84% And then there are these cards, each of which 00:36:10.020 --> 00:36:13.120 align:middle line:84% has a different image taken from a slightly different position. 00:36:13.120 --> 00:36:15.920 align:middle line:84% And so you look at it, and each eye gets an image. 00:36:15.920 --> 00:36:18.820 align:middle line:84% And you can see this as being 3D. 00:36:18.820 --> 00:36:22.620 align:middle line:84% So I'll pass this around in just a second. 00:36:22.620 --> 00:36:26.340 align:middle line:84% So stereoscopes were invented around the same time as cameras. 00:36:26.340 --> 00:36:30.700 align:middle line:84% And so they became a popular fad. 00:36:30.700 --> 00:36:34.940 align:middle line:84% So they were very common back in the 1800s. 00:36:34.940 --> 00:36:38.180 align:middle line:84% People were once crazy about stereo views. 00:36:38.180 --> 00:36:40.580 align:middle line:84% Many families had a stereoscope and a collection 00:36:40.580 --> 00:36:44.220 align:middle line:84% of stereo views, which they used to entertain friends, educate 00:36:44.220 --> 00:36:48.820 align:middle line:84% their children, and see the world. 00:36:48.820 --> 00:36:52.060 align:middle line:84% Oliver Wendell Holmes called the stereoscopic photograph 00:36:52.060 --> 00:36:55.067 align:middle line:84% the card of introduction to make all acquaintances. 00:36:55.067 --> 00:36:57.400 align:middle line:84% So this was like, you'd have someone over to your house, 00:36:57.400 --> 00:37:00.460 align:middle line:84% and then you'd show them stuff on your stereoscope. 00:37:00.460 --> 00:37:02.520 align:middle line:90% Yeah, old school. 00:37:02.520 --> 00:37:04.270 align:middle line:84% So I'm going to pass this around. 00:37:04.270 --> 00:37:07.110 align:middle line:84% So you can still get these things on eBay, 00:37:07.110 --> 00:37:09.230 align:middle line:84% and Amazon, and things like that. 00:37:09.230 --> 00:37:12.930 align:middle line:84% So here's just one example, and you can try it out for yourself. 00:37:12.930 --> 00:37:15.370 align:middle line:90% So you just look through. 00:37:15.370 --> 00:37:16.050 align:middle line:90% There you go. 00:37:16.050 --> 00:37:18.670 align:middle line:90% 00:37:18.670 --> 00:37:22.830 align:middle line:84% So that's how they did it back in the day. 00:37:22.830 --> 00:37:25.210 align:middle line:84% Nowadays, with modern technology, 00:37:25.210 --> 00:37:29.430 align:middle line:84% we have other ways to experience binocular stereopsis. 00:37:29.430 --> 00:37:32.910 align:middle line:84% One such way is called an anaglyph. 00:37:32.910 --> 00:37:36.750 align:middle line:84% So an anaglyph uses colored ink and colored glasses 00:37:36.750 --> 00:37:39.870 align:middle line:84% to present different images to the eyes. 00:37:39.870 --> 00:37:45.150 align:middle line:84% And so I'm going to hand these things out to you. 00:37:45.150 --> 00:37:51.800 align:middle line:84% And you can try this out for yourself. 00:37:51.800 --> 00:37:54.310 align:middle line:84% Some of these are actually-- you want one 00:37:54.310 --> 00:37:57.670 align:middle line:90% that has one red and one blue. 00:37:57.670 --> 00:38:00.042 align:middle line:84% Actually, TAs, maybe I can put you 00:38:00.042 --> 00:38:02.250 align:middle line:84% in charge of making sure everybody gets one of these. 00:38:02.250 --> 00:38:03.010 align:middle line:90% There's some more in here. 00:38:03.010 --> 00:38:04.635 align:middle line:84% I don't know how many of these we need. 00:38:04.635 --> 00:38:07.550 align:middle line:90% 00:38:07.550 --> 00:38:08.250 align:middle line:90% There you go. 00:38:08.250 --> 00:38:09.990 align:middle line:90% OK. 00:38:09.990 --> 00:38:12.530 align:middle line:90% So we'll pass this out. 00:38:12.530 --> 00:38:14.370 align:middle line:90% And you're going to put them on. 00:38:14.370 --> 00:38:16.030 align:middle line:84% And the little thing in the bottom 00:38:16.030 --> 00:38:18.990 align:middle line:84% is supposed to tell you which side the red goes on 00:38:18.990 --> 00:38:21.550 align:middle line:90% and which side the blue goes on. 00:38:21.550 --> 00:38:22.550 align:middle line:90% So here's the idea. 00:38:22.550 --> 00:38:25.550 align:middle line:84% So we've got these two different images. 00:38:25.550 --> 00:38:27.510 align:middle line:90% These are just synthesized. 00:38:27.510 --> 00:38:29.570 align:middle line:84% So this one is supposed to go to the left eye. 00:38:29.570 --> 00:38:30.990 align:middle line:84% This one is supposed to go to the right eye. 00:38:30.990 --> 00:38:32.510 align:middle line:84% You can see that what distinguishes 00:38:32.510 --> 00:38:36.110 align:middle line:84% them is that in the left eye, the letter A is 00:38:36.110 --> 00:38:38.310 align:middle line:84% a little bit to the left of the letter B. 00:38:38.310 --> 00:38:40.390 align:middle line:84% And the letter C is a little bit to the right. 00:38:40.390 --> 00:38:42.830 align:middle line:84% Whereas in the right eye, that's reversed. 00:38:42.830 --> 00:38:45.650 align:middle line:84% So that's going to create disparity. 00:38:45.650 --> 00:38:48.590 align:middle line:84% And so now the anaglyph just puts 00:38:48.590 --> 00:38:51.450 align:middle line:84% one of these images in the red channel 00:38:51.450 --> 00:38:53.670 align:middle line:84% and one of these in the blue channel. 00:38:53.670 --> 00:38:55.710 align:middle line:84% And then these colored glasses are 00:38:55.710 --> 00:38:59.430 align:middle line:84% going to pass one of those images to one eye and the other 00:38:59.430 --> 00:39:00.670 align:middle line:90% to the other eye. 00:39:00.670 --> 00:39:04.730 align:middle line:84% And when you do that, lo and behold, 00:39:04.730 --> 00:39:10.030 align:middle line:84% if you're not stereo blind, you should see depth differences. 00:39:10.030 --> 00:39:11.318 align:middle line:90% AUDIENCE: What's stereo blind? 00:39:11.318 --> 00:39:13.110 align:middle line:84% JOSH MCDERMOTT: Stereo blind means that you 00:39:13.110 --> 00:39:15.190 align:middle line:90% can't make use of stereo. 00:39:15.190 --> 00:39:17.670 align:middle line:84% So we'll talk about that in a minute. 00:39:17.670 --> 00:39:22.270 align:middle line:84% So 5% of people supposedly are stereo blind. 00:39:22.270 --> 00:39:24.390 align:middle line:90% Everybody, keep these on. 00:39:24.390 --> 00:39:25.930 align:middle line:84% Let me take a quick picture here. 00:39:25.930 --> 00:39:30.070 align:middle line:90% 00:39:30.070 --> 00:39:33.150 align:middle line:84% Everybody looks very stylish and hopefully is 00:39:33.150 --> 00:39:37.470 align:middle line:84% experiencing very vivid impressions of depth. 00:39:37.470 --> 00:39:38.990 align:middle line:90% Does this work for you? 00:39:38.990 --> 00:39:41.270 align:middle line:90% Raise your hand if this works. 00:39:41.270 --> 00:39:45.310 align:middle line:90% Most people, maybe. 00:39:45.310 --> 00:39:45.810 align:middle line:90% Yep. 00:39:45.810 --> 00:39:46.518 align:middle line:90% OK, good. 00:39:46.518 --> 00:39:47.810 align:middle line:90% All right, we'll see some more. 00:39:47.810 --> 00:39:50.830 align:middle line:84% So this one is not that exciting. 00:39:50.830 --> 00:39:52.670 align:middle line:90% This one is a little bit better. 00:39:52.670 --> 00:39:53.670 align:middle line:90% AUDIENCE: Ooh. 00:39:53.670 --> 00:39:55.670 align:middle line:90% [LAUGHTER] 00:39:55.670 --> 00:39:56.950 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:39:56.950 --> 00:39:59.600 align:middle line:90% Yeah, I like it. 00:39:59.600 --> 00:40:01.880 align:middle line:84% So if you flip the glasses, the depth 00:40:01.880 --> 00:40:05.680 align:middle line:84% will switch because you're going to reverse the disparity. 00:40:05.680 --> 00:40:08.116 align:middle line:90% AUDIENCE: Whoa. 00:40:08.116 --> 00:40:10.536 align:middle line:90% [INTERPOSING VOICES] 00:40:10.536 --> 00:40:19.400 align:middle line:90% 00:40:19.400 --> 00:40:22.520 align:middle line:84% JOSH MCDERMOTT: These are just generated synthetically. 00:40:22.520 --> 00:40:24.500 align:middle line:90% You can do this with photos. 00:40:24.500 --> 00:40:31.240 align:middle line:90% 00:40:31.240 --> 00:40:32.800 align:middle line:84% These are like the photos that go 00:40:32.800 --> 00:40:36.080 align:middle line:84% into the stereoscope-- so two different photos from two 00:40:36.080 --> 00:40:37.220 align:middle line:90% different positions. 00:40:37.220 --> 00:40:40.097 align:middle line:90% 00:40:40.097 --> 00:40:41.680 align:middle line:84% I'm not entirely sure what will happen 00:40:41.680 --> 00:40:44.160 align:middle line:90% if you flip the glasses here. 00:40:44.160 --> 00:40:46.360 align:middle line:84% AUDIENCE: How does this work if it's all gray? 00:40:46.360 --> 00:40:47.640 align:middle line:90% There's a little bit of-- 00:40:47.640 --> 00:40:48.550 align:middle line:90% AUDIENCE: It's not all gray. 00:40:48.550 --> 00:40:49.740 align:middle line:90% AUDIENCE: [INAUDIBLE] the red. 00:40:49.740 --> 00:40:50.500 align:middle line:90% It's very subtle. 00:40:50.500 --> 00:40:51.375 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:40:51.375 --> 00:40:54.760 align:middle line:84% I mean, if you add the blue and the red together, 00:40:54.760 --> 00:40:59.780 align:middle line:84% you're going to see something close to gray. 00:40:59.780 --> 00:41:01.700 align:middle line:84% But the glasses separate them out. 00:41:01.700 --> 00:41:02.200 align:middle line:90% Yeah. 00:41:02.200 --> 00:41:05.586 align:middle line:90% 00:41:05.586 --> 00:41:08.472 align:middle line:90% [SIDE CONVERSATION] 00:41:08.472 --> 00:41:11.840 align:middle line:90% 00:41:11.840 --> 00:41:15.440 align:middle line:84% JOSH MCDERMOTT: I see murmurs of something-- 00:41:15.440 --> 00:41:17.360 align:middle line:90% or hear murmurs. 00:41:17.360 --> 00:41:18.320 align:middle line:90% What's that? 00:41:18.320 --> 00:41:19.140 align:middle line:90% AUDIENCE: It looks a lot better. 00:41:19.140 --> 00:41:20.265 align:middle line:90% AUDIENCE: Oh, no, these do. 00:41:20.265 --> 00:41:21.960 align:middle line:90% JOSH MCDERMOTT: Oh, I see. 00:41:21.960 --> 00:41:22.700 align:middle line:90% Yeah, OK. 00:41:22.700 --> 00:41:26.800 align:middle line:84% Our projector here is possibly not helping us too much. 00:41:26.800 --> 00:41:28.960 align:middle line:90% Here's a Mars rock. 00:41:28.960 --> 00:41:30.842 align:middle line:90% JOSH MCDERMOTT: Whoa. 00:41:30.842 --> 00:41:33.297 align:middle line:90% [LAUGHTER] 00:41:33.297 --> 00:41:35.760 align:middle line:90% 00:41:35.760 --> 00:41:38.440 align:middle line:90% [SIDE CONVERSATION] 00:41:38.440 --> 00:41:47.320 align:middle line:90% 00:41:47.320 --> 00:41:49.200 align:middle line:90% JOSH MCDERMOTT: Here, show me. 00:41:49.200 --> 00:41:52.145 align:middle line:84% Yeah, no, it is definitely better. 00:41:52.145 --> 00:41:54.020 align:middle line:84% I don't know what's wrong with our projector. 00:41:54.020 --> 00:41:58.390 align:middle line:90% 00:41:58.390 --> 00:42:02.170 align:middle line:84% So just to be clear, the reason why this might work better 00:42:02.170 --> 00:42:06.610 align:middle line:84% on your iPad or your laptop screen than the projector 00:42:06.610 --> 00:42:10.810 align:middle line:84% is that all of this depends on these things being 00:42:10.810 --> 00:42:13.830 align:middle line:84% the right colors relative to these glasses. 00:42:13.830 --> 00:42:15.970 align:middle line:84% And so if the projector is a little bit off, 00:42:15.970 --> 00:42:18.210 align:middle line:90% then it may work less well. 00:42:18.210 --> 00:42:20.930 align:middle line:90% This one is pretty cool. 00:42:20.930 --> 00:42:22.050 align:middle line:90% Yeah, Christa? 00:42:22.050 --> 00:42:24.850 align:middle line:84% AUDIENCE: Would these types of 3D effects work if you had 00:42:24.850 --> 00:42:25.790 align:middle line:90% different glasses? 00:42:25.790 --> 00:42:29.068 align:middle line:90% 00:42:29.068 --> 00:42:30.110 align:middle line:90% JOSH MCDERMOTT: Oh, yeah. 00:42:30.110 --> 00:42:33.250 align:middle line:84% There's nothing special about red and blue. 00:42:33.250 --> 00:42:35.210 align:middle line:84% Yeah, you just need filters that will 00:42:35.210 --> 00:42:38.190 align:middle line:84% get one image to one eye and another to the other eye. 00:42:38.190 --> 00:42:39.130 align:middle line:90% Yeah. 00:42:39.130 --> 00:42:40.650 align:middle line:90% Yeah. 00:42:40.650 --> 00:42:44.770 align:middle line:84% So I think the moral of all of this is that people like 3D. 00:42:44.770 --> 00:42:50.530 align:middle line:84% And that's consistent with the fact that 3D movies are 00:42:50.530 --> 00:42:53.290 align:middle line:90% a recurring fad. 00:42:53.290 --> 00:42:59.370 align:middle line:84% So nowadays, if you go and you see Avatar or whatever in 3D, 00:42:59.370 --> 00:43:00.678 align:middle line:90% the glasses are not like these. 00:43:00.678 --> 00:43:01.970 align:middle line:90% They're a little bit different. 00:43:01.970 --> 00:43:05.210 align:middle line:84% So the glasses actually use polarizing filters. 00:43:05.210 --> 00:43:09.050 align:middle line:84% So you can present two projectors 00:43:09.050 --> 00:43:11.330 align:middle line:84% that polarize light at different angles, 00:43:11.330 --> 00:43:17.770 align:middle line:84% and then that light will be passed by one lens or the other. 00:43:17.770 --> 00:43:21.510 align:middle line:84% And so the advantage of that is that the film can be in color. 00:43:21.510 --> 00:43:24.270 align:middle line:84% So if you're going to use this anaglyph technology, 00:43:24.270 --> 00:43:26.910 align:middle line:84% then you're stuck with a black-and-white film, 00:43:26.910 --> 00:43:27.910 align:middle line:90% which people don't like. 00:43:27.910 --> 00:43:28.410 align:middle line:90% Yeah? 00:43:28.410 --> 00:43:29.330 align:middle line:84% AUDIENCE: This this the cool thing. 00:43:29.330 --> 00:43:30.490 align:middle line:84% After wearing these for a few minutes 00:43:30.490 --> 00:43:31.865 align:middle line:84% and then taking them off, and you 00:43:31.865 --> 00:43:35.070 align:middle line:84% close one eye, when I close my left eye, 00:43:35.070 --> 00:43:36.970 align:middle line:84% I feel like my right eye sees more red. 00:43:36.970 --> 00:43:39.415 align:middle line:84% Then when I close my right eye, the world went blue. 00:43:39.415 --> 00:43:40.290 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:43:40.290 --> 00:43:42.230 align:middle line:84% So that's photoreceptor adaptation. 00:43:42.230 --> 00:43:44.090 align:middle line:90% Yeah, yeah. 00:43:44.090 --> 00:43:45.170 align:middle line:90% Yeah, yeah. 00:43:45.170 --> 00:43:46.830 align:middle line:90% Yeah, the miracles of vision. 00:43:46.830 --> 00:43:47.330 align:middle line:90% OK. 00:43:47.330 --> 00:43:50.610 align:middle line:90% 00:43:50.610 --> 00:43:51.770 align:middle line:90% OK. 00:43:51.770 --> 00:43:53.578 align:middle line:84% So binocular disparity-- so that's 00:43:53.578 --> 00:43:55.620 align:middle line:84% the basis of all of this, these small differences 00:43:55.620 --> 00:43:56.412 align:middle line:90% between the images. 00:43:56.412 --> 00:43:59.080 align:middle line:84% So the intuition here is obvious, right? 00:43:59.080 --> 00:44:01.540 align:middle line:84% So the question is, how does this actually work? 00:44:01.540 --> 00:44:03.200 align:middle line:84% How do you actually compute disparity? 00:44:03.200 --> 00:44:07.980 align:middle line:84% So one of the big challenges is what's called the correspondence 00:44:07.980 --> 00:44:08.560 align:middle line:90% problem. 00:44:08.560 --> 00:44:12.100 align:middle line:84% So in order to measure disparity, 00:44:12.100 --> 00:44:15.580 align:middle line:84% that assumes that you know what in the left image 00:44:15.580 --> 00:44:17.800 align:middle line:84% goes with what in the right image. 00:44:17.800 --> 00:44:21.440 align:middle line:84% And so when I show you these diagrams, 00:44:21.440 --> 00:44:23.620 align:middle line:84% it's obvious because there's only two things, 00:44:23.620 --> 00:44:25.520 align:middle line:90% and they're lines and stuff. 00:44:25.520 --> 00:44:28.700 align:middle line:84% But in actual images, it may not necessarily 00:44:28.700 --> 00:44:31.880 align:middle line:84% be obvious what in one eye corresponds 00:44:31.880 --> 00:44:33.280 align:middle line:90% to what in the left eye. 00:44:33.280 --> 00:44:38.460 align:middle line:84% And so in particular, what are the features 00:44:38.460 --> 00:44:39.600 align:middle line:90% that would be matched? 00:44:39.600 --> 00:44:42.420 align:middle line:84% And where in the visual system might this occur? 00:44:42.420 --> 00:44:49.540 align:middle line:84% So for a long time, people supposed that stereopsis 00:44:49.540 --> 00:44:51.700 align:middle line:90% happened after shape analysis. 00:44:51.700 --> 00:44:53.260 align:middle line:84% So the way that the vision would work 00:44:53.260 --> 00:44:55.635 align:middle line:84% is you'd get the left image, and you get the right image, 00:44:55.635 --> 00:44:57.960 align:middle line:84% and you do some analysis of each of the two images 00:44:57.960 --> 00:44:59.900 align:middle line:84% to figure out where the shapes were. 00:44:59.900 --> 00:45:01.700 align:middle line:84% And then on the basis of that, you 00:45:01.700 --> 00:45:03.220 align:middle line:90% would then compute disparities. 00:45:03.220 --> 00:45:06.300 align:middle line:84% So the idea with stereopsis happened late. 00:45:06.300 --> 00:45:08.478 align:middle line:84% So the modern view is the opposite 00:45:08.478 --> 00:45:11.020 align:middle line:84% of this, where we now think that stereopsis is something that 00:45:11.020 --> 00:45:14.500 align:middle line:84% happens earlier, probably prior to what you might normally 00:45:14.500 --> 00:45:15.800 align:middle line:90% think of as shape analysis. 00:45:15.800 --> 00:45:17.220 align:middle line:84% And really, the key demonstration 00:45:17.220 --> 00:45:21.860 align:middle line:84% that caused this shift in how people think about this 00:45:21.860 --> 00:45:24.240 align:middle line:90% was random-dot stereogram. 00:45:24.240 --> 00:45:25.900 align:middle line:84% So this was invented by Bela Julesz, 00:45:25.900 --> 00:45:28.358 align:middle line:84% who was a perceptual scientist who was working at Bell Labs 00:45:28.358 --> 00:45:29.580 align:middle line:90% at the time. 00:45:29.580 --> 00:45:35.380 align:middle line:90% And they had just started the-- 00:45:35.380 --> 00:45:37.940 align:middle line:84% it had only been a few years that computers had come online 00:45:37.940 --> 00:45:40.607 align:middle line:84% and it was possible to actually generate these synthetic images. 00:45:40.607 --> 00:45:42.980 align:middle line:84% And so the random-dot stereogram consists 00:45:42.980 --> 00:45:45.560 align:middle line:90% of images that are random dots. 00:45:45.560 --> 00:45:48.940 align:middle line:84% So each dot in the image is randomly assigned black or white 00:45:48.940 --> 00:45:50.590 align:middle line:90% with 50% probability. 00:45:50.590 --> 00:45:54.550 align:middle line:84% And then you take that image and then take a square 00:45:54.550 --> 00:45:57.210 align:middle line:84% and shift it over by a dot or two. 00:45:57.210 --> 00:45:59.130 align:middle line:84% And so that gives you the other image. 00:45:59.130 --> 00:46:04.470 align:middle line:90% And now we have our anaglyph. 00:46:04.470 --> 00:46:08.970 align:middle line:90% And you should-- oh, yeah. 00:46:08.970 --> 00:46:12.590 align:middle line:90% 00:46:12.590 --> 00:46:15.790 align:middle line:84% You can flip the glasses either way, 00:46:15.790 --> 00:46:23.510 align:middle line:84% and you'll either see a square in front or behind. 00:46:23.510 --> 00:46:25.310 align:middle line:84% And yeah, this definitely does not 00:46:25.310 --> 00:46:26.910 align:middle line:84% work quite as well as it should, I 00:46:26.910 --> 00:46:29.470 align:middle line:84% think because there's a mismatch between the color guns 00:46:29.470 --> 00:46:32.770 align:middle line:84% and the projector and these glasses. 00:46:32.770 --> 00:46:34.290 align:middle line:90% But you hopefully get the idea. 00:46:34.290 --> 00:46:37.030 align:middle line:90% 00:46:37.030 --> 00:46:41.930 align:middle line:84% So each eye by itself gets an array of random dots. 00:46:41.930 --> 00:46:45.230 align:middle line:84% And so the point of this is that there are no obvious objects 00:46:45.230 --> 00:46:47.830 align:middle line:84% or features that can be identified and matched 00:46:47.830 --> 00:46:49.090 align:middle line:90% between the eyes. 00:46:49.090 --> 00:46:51.290 align:middle line:84% There's just these random-looking textures. 00:46:51.290 --> 00:46:52.630 align:middle line:90% One second. 00:46:52.630 --> 00:46:55.390 align:middle line:84% But when the eyes are combined, there's 00:46:55.390 --> 00:46:57.510 align:middle line:84% a square that becomes evident, either floating 00:46:57.510 --> 00:47:00.610 align:middle line:84% above the background or behind it. 00:47:00.610 --> 00:47:01.270 align:middle line:90% Yeah? 00:47:01.270 --> 00:47:04.650 align:middle line:84% AUDIENCE: I was just curious how eye dominance plays into this 00:47:04.650 --> 00:47:07.110 align:middle line:84% because I noticed that when I flip it over, 00:47:07.110 --> 00:47:09.850 align:middle line:84% it's definitely whatever color it is on my dominant eye. 00:47:09.850 --> 00:47:11.590 align:middle line:84% My right eye is where I'm seeing more. 00:47:11.590 --> 00:47:14.130 align:middle line:84% Does eye dominance also play into depth at all? 00:47:14.130 --> 00:47:17.350 align:middle line:84% Or is it just the color that I'm seeing? 00:47:17.350 --> 00:47:18.750 align:middle line:84% JOSH MCDERMOTT: It probably does. 00:47:18.750 --> 00:47:23.310 align:middle line:84% And when you say eye dominance, what does that mean to you/ 00:47:23.310 --> 00:47:27.950 align:middle line:84% AUDIENCE: When on the team, you have to see which eye your brain 00:47:27.950 --> 00:47:28.533 align:middle line:90% is using more. 00:47:28.533 --> 00:47:29.575 align:middle line:90% JOSH MCDERMOTT: Oh, yeah. 00:47:29.575 --> 00:47:30.137 align:middle line:90% OK, yeah. 00:47:30.137 --> 00:47:32.470 align:middle line:84% AUDIENCE: If I close my left eye, my view doesn't shift. 00:47:32.470 --> 00:47:33.410 align:middle line:90% But if I close my right eye-- 00:47:33.410 --> 00:47:35.785 align:middle line:84% JOSH MCDERMOTT: Yeah, yeah, yeah, yeah, yeah, yeah, yeah, 00:47:35.785 --> 00:47:36.350 align:middle line:90% yeah. 00:47:36.350 --> 00:47:38.910 align:middle line:84% I don't know how that's going to affect stereopsis. 00:47:38.910 --> 00:47:41.450 align:middle line:84% I mean, my intuition is that it might make it weaker. 00:47:41.450 --> 00:47:43.470 align:middle line:84% So there is an extreme case-- and we're going 00:47:43.470 --> 00:47:45.550 align:middle line:90% to talk about this at the end. 00:47:45.550 --> 00:47:48.760 align:middle line:84% There's a phenomenon called amblyopia, where one eye 00:47:48.760 --> 00:47:50.220 align:middle line:90% completely dominates the other. 00:47:50.220 --> 00:47:53.720 align:middle line:84% And this typically happens when the eyes are misaligned, usually 00:47:53.720 --> 00:47:54.960 align:middle line:90% at birth. 00:47:54.960 --> 00:47:57.040 align:middle line:90% And if that's not corrected-- 00:47:57.040 --> 00:47:58.740 align:middle line:90% there's a critical period. 00:47:58.740 --> 00:48:02.240 align:middle line:84% And if it's not corrected before the critical period is over, 00:48:02.240 --> 00:48:03.920 align:middle line:84% you get this phenomenon called amblyopia 00:48:03.920 --> 00:48:07.160 align:middle line:84% where one eye dominates, and you don't have stereopsis 00:48:07.160 --> 00:48:07.700 align:middle line:90% after that. 00:48:07.700 --> 00:48:10.632 align:middle line:84% But there's probably gradations of that, 00:48:10.632 --> 00:48:12.340 align:middle line:84% and that may be what you're experiencing. 00:48:12.340 --> 00:48:13.923 align:middle line:84% So yeah, I don't know how that's going 00:48:13.923 --> 00:48:16.480 align:middle line:90% to affect normal stereopsis. 00:48:16.480 --> 00:48:17.560 align:middle line:90% Yeah. 00:48:17.560 --> 00:48:20.220 align:middle line:84% OK, so this is the random-dot stereogram. 00:48:20.220 --> 00:48:24.468 align:middle line:90% 00:48:24.468 --> 00:48:26.260 align:middle line:84% And so you can look at this as an anaglyph. 00:48:26.260 --> 00:48:28.280 align:middle line:84% You can also look at it as a stereo pair. 00:48:28.280 --> 00:48:31.240 align:middle line:84% So people who study stereo vision 00:48:31.240 --> 00:48:33.720 align:middle line:90% train themselves to free fuse. 00:48:33.720 --> 00:48:37.740 align:middle line:84% So what that means is that you train yourself 00:48:37.740 --> 00:48:41.740 align:middle line:84% so that your eyes can diverge or converge at will. 00:48:41.740 --> 00:48:44.560 align:middle line:84% So normally, when you look at something like a screen, 00:48:44.560 --> 00:48:46.840 align:middle line:84% you just fixate the screen because normally, 00:48:46.840 --> 00:48:49.440 align:middle line:84% when you fixate something, you want to bring that into focus. 00:48:49.440 --> 00:48:52.120 align:middle line:84% But you can train yourself to have your eyes actually 00:48:52.120 --> 00:48:54.180 align:middle line:84% not fixate the screen, but to fixate 00:48:54.180 --> 00:48:55.680 align:middle line:84% a point either in back of the screen 00:48:55.680 --> 00:48:57.060 align:middle line:90% or in front of the screen. 00:48:57.060 --> 00:49:00.080 align:middle line:84% And that can cause the left image 00:49:00.080 --> 00:49:05.760 align:middle line:84% to be registered at the same location as the other image 00:49:05.760 --> 00:49:07.360 align:middle line:90% in the other eye. 00:49:07.360 --> 00:49:09.760 align:middle line:84% And that will then give you disparity 00:49:09.760 --> 00:49:11.380 align:middle line:84% that you can then use to see depth. 00:49:11.380 --> 00:49:14.505 align:middle line:84% And so, yeah, you can try holding up your thumb 00:49:14.505 --> 00:49:15.380 align:middle line:90% and get this to work. 00:49:15.380 --> 00:49:16.672 align:middle line:90% It usually takes some practice. 00:49:16.672 --> 00:49:19.120 align:middle line:84% I never got to be very good at this, even back when I was 00:49:19.120 --> 00:49:20.880 align:middle line:90% of doing vision for a living. 00:49:20.880 --> 00:49:23.420 align:middle line:84% But some people are very skilled at this. 00:49:23.420 --> 00:49:27.040 align:middle line:90% 00:49:27.040 --> 00:49:28.220 align:middle line:90% Yeah, it's a little tricky. 00:49:28.220 --> 00:49:30.360 align:middle line:90% You can try it at home. 00:49:30.360 --> 00:49:31.885 align:middle line:84% So this is the actual stimulus just 00:49:31.885 --> 00:49:33.260 align:middle line:84% to make sure everybody gets this. 00:49:33.260 --> 00:49:35.060 align:middle line:84% So here's the stimulus to the left eye, 00:49:35.060 --> 00:49:38.300 align:middle line:84% and here's the stimulus to the right eye. 00:49:38.300 --> 00:49:41.680 align:middle line:84% So it's just the square of dots that's offset. 00:49:41.680 --> 00:49:44.090 align:middle line:84% And each image on its own doesn't really 00:49:44.090 --> 00:49:48.170 align:middle line:84% have any kind of meaningful features or structure to it. 00:49:48.170 --> 00:49:52.250 align:middle line:84% So the significance here is that it suggests that you certainly 00:49:52.250 --> 00:49:54.530 align:middle line:84% don't have to do anything like object recognition 00:49:54.530 --> 00:49:57.370 align:middle line:84% before matching things between the two eyes 00:49:57.370 --> 00:49:59.190 align:middle line:90% and getting binocular disparity. 00:49:59.190 --> 00:50:01.190 align:middle line:84% So maybe there's some kind of shape analysis, 00:50:01.190 --> 00:50:03.850 align:middle line:84% but it's not the interpretable form analysis 00:50:03.850 --> 00:50:06.010 align:middle line:90% that you might think of. 00:50:06.010 --> 00:50:07.890 align:middle line:84% And in particular, this is also important 00:50:07.890 --> 00:50:11.050 align:middle line:84% because it demonstrates that you can get depth 00:50:11.050 --> 00:50:13.470 align:middle line:90% from binocular cues alone. 00:50:13.470 --> 00:50:15.570 align:middle line:84% So even like the stereoscope, it's 00:50:15.570 --> 00:50:18.010 align:middle line:84% showing that the depth kind of gets enhanced just 00:50:18.010 --> 00:50:19.990 align:middle line:84% from these two images into the two eyes. 00:50:19.990 --> 00:50:20.890 align:middle line:84% But these are actual photographs, 00:50:20.890 --> 00:50:23.140 align:middle line:84% and they got a million different monocular depth cues. 00:50:23.140 --> 00:50:31.010 align:middle line:84% And so the random-dot stereogram isolates binocular cues. 00:50:31.010 --> 00:50:37.690 align:middle line:84% So here's just another example to illustrate the correspondence 00:50:37.690 --> 00:50:38.930 align:middle line:90% problem. 00:50:38.930 --> 00:50:41.410 align:middle line:84% I've been showing you all of these pictures of how 00:50:41.410 --> 00:50:45.370 align:middle line:84% points in the world project to points on the retina. 00:50:45.370 --> 00:50:48.690 align:middle line:84% And it's a little bit misleading in the sense 00:50:48.690 --> 00:50:51.650 align:middle line:84% that the visual system, when it is 00:50:51.650 --> 00:50:54.190 align:middle line:84% making use of binocular information, 00:50:54.190 --> 00:50:55.770 align:middle line:90% it doesn't know this thing. 00:50:55.770 --> 00:50:59.050 align:middle line:90% All you get is this. 00:50:59.050 --> 00:51:01.690 align:middle line:84% So you get these two images and the two eyes, 00:51:01.690 --> 00:51:05.510 align:middle line:84% and then you're trying to infer what's out there in the world. 00:51:05.510 --> 00:51:08.530 align:middle line:84% And so you could create these three dots 00:51:08.530 --> 00:51:11.790 align:middle line:84% from three dots that are in a row at the same depth. 00:51:11.790 --> 00:51:13.730 align:middle line:84% But there's other kinds of arrangements 00:51:13.730 --> 00:51:17.390 align:middle line:84% of dots at different depths that could produce the same thing. 00:51:17.390 --> 00:51:20.330 align:middle line:90% 00:51:20.330 --> 00:51:24.040 align:middle line:84% And so, in general, I mean, this is another area of vision 00:51:24.040 --> 00:51:26.290 align:middle line:84% where I would say we don't really have complete models 00:51:26.290 --> 00:51:27.310 align:middle line:90% or theories. 00:51:27.310 --> 00:51:28.930 align:middle line:84% I mean, the correspondence problem 00:51:28.930 --> 00:51:30.970 align:middle line:90% is acknowledged as a challenge. 00:51:30.970 --> 00:51:33.273 align:middle line:84% The random-dot stereogram places some constraints 00:51:33.273 --> 00:51:34.690 align:middle line:84% on how that might work, because it 00:51:34.690 --> 00:51:38.210 align:middle line:84% shows that you don't have to have interpretable shape. 00:51:38.210 --> 00:51:43.300 align:middle line:84% One idea for helping solve the correspondence problem 00:51:43.300 --> 00:51:48.200 align:middle line:84% is to do the matching on low spatial frequencies first. 00:51:48.200 --> 00:51:51.180 align:middle line:84% So the idea is that lower spatial frequencies 00:51:51.180 --> 00:51:54.160 align:middle line:84% don't have as many possible matches between the two images. 00:51:54.160 --> 00:51:56.400 align:middle line:84% And so you can of settle on a good match, 00:51:56.400 --> 00:51:58.980 align:middle line:84% maybe with a easier search, and then 00:51:58.980 --> 00:52:03.180 align:middle line:84% hone in on a more fine-grained match 00:52:03.180 --> 00:52:07.660 align:middle line:84% that would cause the higher spatial frequencies to line up 00:52:07.660 --> 00:52:08.340 align:middle line:90% as well. 00:52:08.340 --> 00:52:10.260 align:middle line:84% And so that's what's often referred to 00:52:10.260 --> 00:52:12.000 align:middle line:90% as a coarse-to-fine strategy. 00:52:12.000 --> 00:52:14.500 align:middle line:84% You see this in lots of areas of vision 00:52:14.500 --> 00:52:19.060 align:middle line:84% where you first try to solve a problem at coarse spatial scales 00:52:19.060 --> 00:52:24.040 align:middle line:84% and then refine the solution with higher spatial frequencies. 00:52:24.040 --> 00:52:26.920 align:middle line:90% 00:52:26.920 --> 00:52:30.060 align:middle line:84% So there's lots of funny phenomena 00:52:30.060 --> 00:52:32.740 align:middle line:84% that are associated with binocular vision. 00:52:32.740 --> 00:52:35.020 align:middle line:84% One that you may have encountered 00:52:35.020 --> 00:52:36.403 align:middle line:84% is called the wallpaper illusion. 00:52:36.403 --> 00:52:37.820 align:middle line:84% Maybe you haven't encountered this 00:52:37.820 --> 00:52:41.500 align:middle line:84% because wallpaper is a thing that people don't do anymore. 00:52:41.500 --> 00:52:44.340 align:middle line:84% But the previous generation, they 00:52:44.340 --> 00:52:47.600 align:middle line:84% used to decorate their houses with wallpaper. 00:52:47.600 --> 00:52:48.975 align:middle line:84% So instead of painting the walls, 00:52:48.975 --> 00:52:50.725 align:middle line:84% you'd put paper up on the walls that would 00:52:50.725 --> 00:52:52.120 align:middle line:90% have particular patterns in it. 00:52:52.120 --> 00:52:54.640 align:middle line:84% So maybe your grandparents had had this. 00:52:54.640 --> 00:52:55.870 align:middle line:90% Mine did. 00:52:55.870 --> 00:52:58.120 align:middle line:84% And there would typically be these repeating patterns. 00:52:58.120 --> 00:53:00.420 align:middle line:84% And so this is a situation where there 00:53:00.420 --> 00:53:02.040 align:middle line:90% are lots of possible matches. 00:53:02.040 --> 00:53:08.700 align:middle line:84% So if you're fixating the screen and you-- 00:53:08.700 --> 00:53:12.020 align:middle line:84% this particular feature is matched 00:53:12.020 --> 00:53:13.980 align:middle line:84% between the left eye and the right eye, 00:53:13.980 --> 00:53:16.300 align:middle line:90% then everything is good. 00:53:16.300 --> 00:53:20.140 align:middle line:84% But if, by chance, your eyes diverge or converge 00:53:20.140 --> 00:53:23.780 align:middle line:84% a little bit because of the ambiguity and this feature 00:53:23.780 --> 00:53:26.540 align:middle line:84% ends up being at the same location in the left eye 00:53:26.540 --> 00:53:29.340 align:middle line:84% as this in the right eye, then you'll 00:53:29.340 --> 00:53:32.380 align:middle line:84% get disparity with respect to the rest of the world. 00:53:32.380 --> 00:53:36.340 align:middle line:84% And that will cause the wallpaper to jump out in depth 00:53:36.340 --> 00:53:37.640 align:middle line:90% or jump back in depth. 00:53:37.640 --> 00:53:39.390 align:middle line:84% So that's the wallpaper illusion. 00:53:39.390 --> 00:53:41.917 align:middle line:90% And that'll happen sometimes. 00:53:41.917 --> 00:53:43.750 align:middle line:84% Another thing that you may-- how many people 00:53:43.750 --> 00:53:46.130 align:middle line:84% have encountered in autostereogram? 00:53:46.130 --> 00:53:46.630 align:middle line:90% Yeah. 00:53:46.630 --> 00:53:50.510 align:middle line:84% So these were a really popular thing when I was a kid. 00:53:50.510 --> 00:53:58.630 align:middle line:84% So these patterns that allow for the possibility 00:53:58.630 --> 00:54:03.950 align:middle line:84% that you could get an aligned image if your eyes converge 00:54:03.950 --> 00:54:05.050 align:middle line:90% or diverge a little bit. 00:54:05.050 --> 00:54:07.790 align:middle line:84% So again, you look at this in the right way 00:54:07.790 --> 00:54:09.450 align:middle line:84% where your eyes diverge a little bit. 00:54:09.450 --> 00:54:12.170 align:middle line:84% So instead of actually fixating the depth of the screen, 00:54:12.170 --> 00:54:14.790 align:middle line:84% you're fixating a point that is in back of the screen 00:54:14.790 --> 00:54:16.630 align:middle line:90% or in front of the screen. 00:54:16.630 --> 00:54:20.410 align:middle line:84% And if you get your eyes in exactly the right position, 00:54:20.410 --> 00:54:22.197 align:middle line:84% the images will come into register. 00:54:22.197 --> 00:54:24.030 align:middle line:84% And you'll get something that's a little bit 00:54:24.030 --> 00:54:26.310 align:middle line:84% like a random-dot dot stereogram, 00:54:26.310 --> 00:54:28.250 align:middle line:90% but it just looks different. 00:54:28.250 --> 00:54:31.950 align:middle line:84% In this case, it'll look like the shape of a horse. 00:54:31.950 --> 00:54:35.150 align:middle line:84% So you might have to look at it for a long time. 00:54:35.150 --> 00:54:37.090 align:middle line:84% We can come back to this at the end of class. 00:54:37.090 --> 00:54:39.950 align:middle line:90% 00:54:39.950 --> 00:54:40.450 align:middle line:90% All right. 00:54:40.450 --> 00:54:43.950 align:middle line:84% So another pretty cool and important phenomena 00:54:43.950 --> 00:54:46.510 align:middle line:84% that's related to binocular vision 00:54:46.510 --> 00:54:51.170 align:middle line:84% occurs when you can't match the images between the two eyes. 00:54:51.170 --> 00:54:54.590 align:middle line:84% So if the two images in the two eyes are too different, 00:54:54.590 --> 00:54:59.370 align:middle line:84% you get what is called binocular rivalry. 00:54:59.370 --> 00:55:03.590 align:middle line:84% So hopefully the projector is good enough for us to see this. 00:55:03.590 --> 00:55:06.110 align:middle line:90% So the left eye here-- 00:55:06.110 --> 00:55:06.610 align:middle line:90% oh, yeah. 00:55:06.610 --> 00:55:07.990 align:middle line:90% This is good. 00:55:07.990 --> 00:55:11.830 align:middle line:84% The left eye is going to get one orientation, 00:55:11.830 --> 00:55:16.390 align:middle line:84% and the right eye is going to get another orientation. 00:55:16.390 --> 00:55:20.350 align:middle line:84% And if you look at this, what will happen 00:55:20.350 --> 00:55:25.870 align:middle line:84% is that, over time, you will see one image and then another, 00:55:25.870 --> 00:55:27.830 align:middle line:84% and then back to the other image, 00:55:27.830 --> 00:55:30.510 align:middle line:90% and then back to the other. 00:55:30.510 --> 00:55:32.010 align:middle line:90% Is this working for everybody? 00:55:32.010 --> 00:55:32.510 align:middle line:90% Yeah? 00:55:32.510 --> 00:55:32.770 align:middle line:90% Good. 00:55:32.770 --> 00:55:33.270 align:middle line:90% All right. 00:55:33.270 --> 00:55:34.940 align:middle line:90% So that's binocular rivalry. 00:55:34.940 --> 00:55:38.880 align:middle line:90% And it's an interesting-- 00:55:38.880 --> 00:55:42.680 align:middle line:84% and I can play another example here. 00:55:42.680 --> 00:55:45.800 align:middle line:84% This one is a lot of fun to look at. 00:55:45.800 --> 00:55:48.720 align:middle line:84% It's interesting because the retinal stimulus is constant, 00:55:48.720 --> 00:55:51.920 align:middle line:84% but your percept-- what you're consciously aware of-- 00:55:51.920 --> 00:55:54.120 align:middle line:90% will change over time. 00:55:54.120 --> 00:55:57.320 align:middle line:84% This one is interesting because there are these two 00:55:57.320 --> 00:56:00.200 align:middle line:84% different words, but you may actually mix up 00:56:00.200 --> 00:56:02.000 align:middle line:90% the letters from time to time. 00:56:02.000 --> 00:56:07.440 align:middle line:84% So it's blue and red, but you might see bled sometimes. 00:56:07.440 --> 00:56:09.160 align:middle line:90% Yeah, question? 00:56:09.160 --> 00:56:11.520 align:middle line:84% AUDIENCE: What the glasses work if you were colorblind? 00:56:11.520 --> 00:56:12.140 align:middle line:90% JOSH MCDERMOTT: What's that? 00:56:12.140 --> 00:56:13.515 align:middle line:84% AUDIENCE: If you were colorblind, 00:56:13.515 --> 00:56:14.920 align:middle line:90% would this work at all or no? 00:56:14.920 --> 00:56:17.520 align:middle line:90% 00:56:17.520 --> 00:56:24.560 align:middle line:84% JOSH MCDERMOTT: I think it should because this is-- yeah, 00:56:24.560 --> 00:56:26.060 align:middle line:90% it's an optical effect. 00:56:26.060 --> 00:56:29.440 align:middle line:84% So it just relies on the fact that these 00:56:29.440 --> 00:56:33.960 align:middle line:84% are filters that are going to absorb certain wavelengths 00:56:33.960 --> 00:56:36.200 align:middle line:90% and let others pass. 00:56:36.200 --> 00:56:38.220 align:middle line:84% So it doesn't depend on having color vision. 00:56:38.220 --> 00:56:38.720 align:middle line:90% Yeah. 00:56:38.720 --> 00:56:43.160 align:middle line:90% 00:56:43.160 --> 00:56:46.020 align:middle line:84% All right, so binocular rivalry, it's a thing. 00:56:46.020 --> 00:56:47.740 align:middle line:84% It's something that the brain does. 00:56:47.740 --> 00:56:50.100 align:middle line:84% It's an interesting case of bistability, right? 00:56:50.100 --> 00:56:52.640 align:middle line:90% Really well-known case. 00:56:52.640 --> 00:56:55.000 align:middle line:84% One reason why this is important is 00:56:55.000 --> 00:56:57.200 align:middle line:84% it's been a tool that people have 00:56:57.200 --> 00:57:02.340 align:middle line:84% used to look at the basis of conscious awareness. 00:57:02.340 --> 00:57:06.160 align:middle line:84% So this is a case where you can have the stimulus be completely 00:57:06.160 --> 00:57:09.960 align:middle line:84% fixed, but what you will see, what you perceive, 00:57:09.960 --> 00:57:11.680 align:middle line:90% fluctuates over time. 00:57:11.680 --> 00:57:14.080 align:middle line:84% So you can ask the question, what 00:57:14.080 --> 00:57:18.320 align:middle line:84% happens in the brain when your percept changes like this? 00:57:18.320 --> 00:57:22.040 align:middle line:84% And in particular, you could say, 00:57:22.040 --> 00:57:26.000 align:middle line:84% record from neurons in different stages of the visual system 00:57:26.000 --> 00:57:29.480 align:middle line:84% and ask whether the responses of those neurons 00:57:29.480 --> 00:57:33.630 align:middle line:84% is changing when what you see changes, 00:57:33.630 --> 00:57:36.850 align:middle line:84% even though the input of the visual system remains the same. 00:57:36.850 --> 00:57:38.810 align:middle line:84% And so that's, in fact, something 00:57:38.810 --> 00:57:41.390 align:middle line:84% that people have done fairly extensively. 00:57:41.390 --> 00:57:47.070 align:middle line:84% And we will talk about that in a couple lectures. 00:57:47.070 --> 00:57:51.810 align:middle line:84% Any questions about binocular rivalry or matching? 00:57:51.810 --> 00:57:52.610 align:middle line:90% Yeah? 00:57:52.610 --> 00:57:54.950 align:middle line:84% AUDIENCE: This might not be a very answerable question. 00:57:54.950 --> 00:58:01.370 align:middle line:84% But if we had eyes like prey that didn't necessarily-- 00:58:01.370 --> 00:58:04.330 align:middle line:84% or like older or black, would binocular 00:58:04.330 --> 00:58:06.070 align:middle line:90% rivalry what we perceive? 00:58:06.070 --> 00:58:09.650 align:middle line:84% Or would we just have a much broader view? 00:58:09.650 --> 00:58:11.770 align:middle line:90% What would that be? 00:58:11.770 --> 00:58:15.010 align:middle line:84% JOSH MCDERMOTT: Yeah, that's a great question. 00:58:15.010 --> 00:58:17.835 align:middle line:90% So I would have to assume-- 00:58:17.835 --> 00:58:19.210 align:middle line:84% I mean, it's a difficult question 00:58:19.210 --> 00:58:21.730 align:middle line:84% to answer because we can't really 00:58:21.730 --> 00:58:23.230 align:middle line:90% talk to squirrels very easily. 00:58:23.230 --> 00:58:27.570 align:middle line:84% But I would have to assume that they just would not experience 00:58:27.570 --> 00:58:32.370 align:middle line:84% binocular rivalry and that they just 00:58:32.370 --> 00:58:35.290 align:middle line:84% see a much wider field of view without just 00:58:35.290 --> 00:58:39.870 align:middle line:90% using monocular depth cues. 00:58:39.870 --> 00:58:43.350 align:middle line:84% I mean, it would be dysfunctional, I think, 00:58:43.350 --> 00:58:46.017 align:middle line:84% if they experienced rivalry, so yeah. 00:58:46.017 --> 00:58:47.850 align:middle line:84% So I think this is something that's probably 00:58:47.850 --> 00:58:50.010 align:middle line:84% specific to people that have stereo to organisms 00:58:50.010 --> 00:58:52.310 align:middle line:90% that have stereopsis. 00:58:52.310 --> 00:58:54.590 align:middle line:84% So in order to do the experience that I talked about-- 00:58:54.590 --> 00:58:56.570 align:middle line:84% I mean, some of those experiments 00:58:56.570 --> 00:58:58.770 align:middle line:84% have been done with humans using fMRI 00:58:58.770 --> 00:59:01.130 align:middle line:84% by our own Nancy Kanwisher, who some of you may know. 00:59:01.130 --> 00:59:05.730 align:middle line:84% But others were done on macaque monkeys, 00:59:05.730 --> 00:59:08.290 align:middle line:84% who were reporting what they saw at every moment in time. 00:59:08.290 --> 00:59:12.710 align:middle line:84% And prior to engaging in the physiology work, 00:59:12.710 --> 00:59:15.970 align:middle line:84% they made sure, as best they could, 00:59:15.970 --> 00:59:18.850 align:middle line:84% that the properties of the rivalry that the monkeys were 00:59:18.850 --> 00:59:22.590 align:middle line:84% reporting was similar to the rivalry that you see in humans. 00:59:22.590 --> 00:59:25.650 align:middle line:84% And so in particular, if you measure 00:59:25.650 --> 00:59:27.410 align:middle line:84% how long it takes to flip between one 00:59:27.410 --> 00:59:30.803 align:middle line:84% percept than another, that has pretty established parameters. 00:59:30.803 --> 00:59:32.220 align:middle line:84% So there are certain distributions 00:59:32.220 --> 00:59:35.100 align:middle line:84% for the duration of each percept, 00:59:35.100 --> 00:59:38.600 align:middle line:84% and it changes the stimulus parameters and predictable ways. 00:59:38.600 --> 00:59:41.560 align:middle line:84% And so I think they checked that that happens in monkeys as well. 00:59:41.560 --> 00:59:43.500 align:middle line:84% So as far as we could tell, or they 00:59:43.500 --> 00:59:45.580 align:middle line:84% could tell in those studies, monkeys 00:59:45.580 --> 00:59:48.320 align:middle line:84% have a fairly similar experience of binocular rivalry. 00:59:48.320 --> 00:59:52.220 align:middle line:90% 00:59:52.220 --> 00:59:53.060 align:middle line:90% OK. 00:59:53.060 --> 00:59:57.860 align:middle line:84% Another cool phenomena that is related to binocular vision 00:59:57.860 --> 00:59:59.380 align:middle line:90% is luster. 00:59:59.380 --> 01:00:04.580 align:middle line:84% So luster occurs when the two eyes 01:00:04.580 --> 01:00:06.540 align:middle line:90% receive different luminances. 01:00:06.540 --> 01:00:10.860 align:middle line:84% So when that happens, you get this shimmery appearance 01:00:10.860 --> 01:00:13.060 align:middle line:90% that's known as luster. 01:00:13.060 --> 01:00:18.880 align:middle line:84% So in this particular case, the two middle squares-- 01:00:18.880 --> 01:00:20.980 align:middle line:84% the one on the left and the one on the right-- 01:00:20.980 --> 01:00:24.100 align:middle line:84% will generate different luminances in the two eyes 01:00:24.100 --> 01:00:26.640 align:middle line:84% because one is bluish and one is reddish. 01:00:26.640 --> 01:00:29.700 align:middle line:84% And so they'll be filtered differently by the glasses. 01:00:29.700 --> 01:00:33.820 align:middle line:84% And you should see those as having a funny shimmery 01:00:33.820 --> 01:00:35.900 align:middle line:90% appearance. 01:00:35.900 --> 01:00:37.700 align:middle line:90% Does that kind of work? 01:00:37.700 --> 01:00:38.220 align:middle line:90% Yeah? 01:00:38.220 --> 01:00:39.220 align:middle line:90% OK. 01:00:39.220 --> 01:00:40.140 align:middle line:90% So here's the idea. 01:00:40.140 --> 01:00:40.640 align:middle line:90% All right. 01:00:40.640 --> 01:00:45.180 align:middle line:84% So these are two different photographs taken 01:00:45.180 --> 01:00:47.120 align:middle line:90% from two different positions. 01:00:47.120 --> 01:00:50.520 align:middle line:84% The idea is one camera is at the position of the left eye. 01:00:50.520 --> 01:00:53.040 align:middle line:84% One camera is at the position of the right eye of some keys. 01:00:53.040 --> 01:00:54.360 align:middle line:90% So keys are metal. 01:00:54.360 --> 01:00:58.360 align:middle line:84% Metal is shiny and generates specular reflections. 01:00:58.360 --> 01:01:00.780 align:middle line:90% So it's not Lambertian. 01:01:00.780 --> 01:01:03.900 align:middle line:84% And the consequence of the specular reflections 01:01:03.900 --> 01:01:07.300 align:middle line:84% is that there can be pretty significant differences 01:01:07.300 --> 01:01:12.620 align:middle line:84% in the luminance that hits the two eyes just because they're 01:01:12.620 --> 01:01:15.300 align:middle line:84% at slightly different angles with respect to the image. 01:01:15.300 --> 01:01:19.580 align:middle line:84% So the idea is that what typically 01:01:19.580 --> 01:01:21.980 align:middle line:84% causes these differences in luminance 01:01:21.980 --> 01:01:25.100 align:middle line:84% would be the fact that there's a shiny surface in the world that 01:01:25.100 --> 01:01:27.210 align:middle line:84% is generating a specular reflection. 01:01:27.210 --> 01:01:29.830 align:middle line:90% 01:01:29.830 --> 01:01:33.070 align:middle line:90% So here's just those two images. 01:01:33.070 --> 01:01:36.030 align:middle line:84% And I can flip back and forth between them. 01:01:36.030 --> 01:01:38.670 align:middle line:84% So you can see that the camera's at these two 01:01:38.670 --> 01:01:42.478 align:middle line:84% slightly different positions, and there's 01:01:42.478 --> 01:01:44.270 align:middle line:84% some slight shifts in the position and then 01:01:44.270 --> 01:01:46.170 align:middle line:90% this big change in luminance. 01:01:46.170 --> 01:01:53.790 align:middle line:90% 01:01:53.790 --> 01:01:57.870 align:middle line:84% I mean, the perception of shininess is super interesting. 01:01:57.870 --> 01:02:01.190 align:middle line:84% But you're getting this cue in the image, which is these two 01:02:01.190 --> 01:02:02.290 align:middle line:90% different luminances. 01:02:02.290 --> 01:02:05.270 align:middle line:84% You've learned to associate that with material. 01:02:05.270 --> 01:02:09.070 align:middle line:84% And then that gives it a certain look, the look of gloss. 01:02:09.070 --> 01:02:11.690 align:middle line:84% That's what gloss is in some way. 01:02:11.690 --> 01:02:17.070 align:middle line:90% 01:02:17.070 --> 01:02:21.670 align:middle line:84% So you're always fixating something, typically, right? 01:02:21.670 --> 01:02:24.630 align:middle line:84% So you fixate a particular object. 01:02:24.630 --> 01:02:26.970 align:middle line:90% That induces this horopter. 01:02:26.970 --> 01:02:30.030 align:middle line:84% It's the set of all points that project a corresponding points 01:02:30.030 --> 01:02:32.030 align:middle line:90% on the two retina-- 01:02:32.030 --> 01:02:33.510 align:middle line:90% zero disparity. 01:02:33.510 --> 01:02:37.270 align:middle line:84% And so when you fixate an object at a certain depth, 01:02:37.270 --> 01:02:40.350 align:middle line:84% everything on the horopter, plus and minus a small amount, 01:02:40.350 --> 01:02:42.230 align:middle line:90% will appear to be fused. 01:02:42.230 --> 01:02:44.650 align:middle line:90% So you see a single thing. 01:02:44.650 --> 01:02:47.070 align:middle line:84% So that's called Panum's fusional area. 01:02:47.070 --> 01:02:48.950 align:middle line:84% So objects that are outside this region 01:02:48.950 --> 01:02:50.290 align:middle line:90% will appear as double images. 01:02:50.290 --> 01:02:51.915 align:middle line:84% And so you can check this out yourself. 01:02:51.915 --> 01:02:54.470 align:middle line:84% So if you fixate one finger and then you 01:02:54.470 --> 01:02:58.270 align:middle line:84% move the other finger sufficiently far away, 01:02:58.270 --> 01:03:00.390 align:middle line:90% you'll see two of it. 01:03:00.390 --> 01:03:03.710 align:middle line:84% So if it's right next to the finger that you fixate, 01:03:03.710 --> 01:03:05.810 align:middle line:90% you see one thing. 01:03:05.810 --> 01:03:07.250 align:middle line:84% And then you start to see double. 01:03:07.250 --> 01:03:10.830 align:middle line:90% 01:03:10.830 --> 01:03:14.190 align:middle line:84% How many of you have noticed that before? 01:03:14.190 --> 01:03:15.530 align:middle line:90% A lot of you haven't. 01:03:15.530 --> 01:03:19.710 align:middle line:84% So it's this thing that you're always seeing. 01:03:19.710 --> 01:03:21.430 align:middle line:84% You're constantly seeing double images. 01:03:21.430 --> 01:03:23.097 align:middle line:84% Most of the time, you don't notice them. 01:03:23.097 --> 01:03:27.040 align:middle line:84% You have to really pay attention to them. 01:03:27.040 --> 01:03:28.860 align:middle line:84% And so this gets back to this thing 01:03:28.860 --> 01:03:30.860 align:middle line:84% that we were talking about a little bit earlier. 01:03:30.860 --> 01:03:32.780 align:middle line:84% So why don't you notice this crazy thing 01:03:32.780 --> 01:03:35.280 align:middle line:84% about your vision, which is that you have two copies of most 01:03:35.280 --> 01:03:38.040 align:middle line:90% of the objects in your world? 01:03:38.040 --> 01:03:39.575 align:middle line:84% And it's probably the same reason 01:03:39.575 --> 01:03:41.700 align:middle line:84% that you don't notice that the periphery is blurry. 01:03:41.700 --> 01:03:43.283 align:middle line:84% It's probably the same reason that you 01:03:43.283 --> 01:03:45.280 align:middle line:84% don't notice that some kinds of paintings 01:03:45.280 --> 01:03:46.960 align:middle line:90% depict impossible objects. 01:03:46.960 --> 01:03:50.320 align:middle line:84% Whenever you're interested in something, you move your eyes, 01:03:50.320 --> 01:03:52.040 align:middle line:84% and you place the thing that you're 01:03:52.040 --> 01:03:53.460 align:middle line:90% interested in at the fovea. 01:03:53.460 --> 01:03:56.820 align:middle line:84% And then, in this case, because of binocular convergence, 01:03:56.820 --> 01:03:58.020 align:middle line:90% the two images line up. 01:03:58.020 --> 01:04:00.270 align:middle line:84% You're always fixating the stuff that you want to see. 01:04:00.270 --> 01:04:03.340 align:middle line:84% So typically, what you're paying attention to will be fused. 01:04:03.340 --> 01:04:06.955 align:middle line:84% And the stuff that you're not fixating won't be fused, 01:04:06.955 --> 01:04:09.580 align:middle line:84% but you're not paying attention to it, and you don't notice it. 01:04:09.580 --> 01:04:13.480 align:middle line:90% 01:04:13.480 --> 01:04:15.800 align:middle line:84% OK, so how does this stuff work in the brain? 01:04:15.800 --> 01:04:22.360 align:middle line:84% So remember that the two eyes is remain segregated 01:04:22.360 --> 01:04:24.200 align:middle line:90% at the level of the LGN. 01:04:24.200 --> 01:04:27.080 align:middle line:84% So the LGN consists of these six layers. 01:04:27.080 --> 01:04:30.200 align:middle line:84% Remember, the top four layers are parvocellular. 01:04:30.200 --> 01:04:32.720 align:middle line:90% Bottom two are magnocellular. 01:04:32.720 --> 01:04:36.600 align:middle line:84% But then three of those layers are for the left eye, 01:04:36.600 --> 01:04:39.960 align:middle line:84% and the other three are for the right eye 01:04:39.960 --> 01:04:43.000 align:middle line:90% in each of the two hemifields. 01:04:43.000 --> 01:04:46.260 align:middle line:84% So there's strict segregation in the layers of the LGN. 01:04:46.260 --> 01:04:48.240 align:middle line:84% A neuron will exclusively get input 01:04:48.240 --> 01:04:49.995 align:middle line:84% from the left eye or the right eye. 01:04:49.995 --> 01:04:52.120 align:middle line:84% Now, when you get to V1, the left and the right eye 01:04:52.120 --> 01:04:53.240 align:middle line:90% begin to mix. 01:04:53.240 --> 01:04:56.000 align:middle line:84% Now, the cells have varying weights from the two eyes. 01:04:56.000 --> 01:04:57.450 align:middle line:90% So some are purely monocular. 01:04:57.450 --> 01:04:58.700 align:middle line:90% Some have equal contributions. 01:04:58.700 --> 01:05:01.460 align:middle line:84% Remember, we talked about ocular dominance columns. 01:05:01.460 --> 01:05:04.860 align:middle line:84% So even though you begin to see binocularity in neurons, 01:05:04.860 --> 01:05:08.320 align:middle line:84% it's still the case that at a given point in the cortex, 01:05:08.320 --> 01:05:10.960 align:middle line:84% that column will tend to be dominated 01:05:10.960 --> 01:05:12.660 align:middle line:90% by one eye or the other. 01:05:12.660 --> 01:05:14.640 align:middle line:84% It's called ocular dominance column. 01:05:14.640 --> 01:05:16.480 align:middle line:84% So here's an image of looking down 01:05:16.480 --> 01:05:19.930 align:middle line:84% at the cortex of where the animal's got one of its eyes 01:05:19.930 --> 01:05:20.430 align:middle line:90% open. 01:05:20.430 --> 01:05:22.680 align:middle line:84% It's being stimulated with checkerboards or something, 01:05:22.680 --> 01:05:23.450 align:middle line:90% probably. 01:05:23.450 --> 01:05:26.050 align:middle line:84% And these are the bits of the visual cortex that 01:05:26.050 --> 01:05:27.810 align:middle line:90% are activated by that one eye. 01:05:27.810 --> 01:05:31.570 align:middle line:84% So those are ocular dominance columns. 01:05:31.570 --> 01:05:34.130 align:middle line:84% But it's nonetheless the case that you 01:05:34.130 --> 01:05:37.810 align:middle line:84% see plenty of neurons that get input from both eyes. 01:05:37.810 --> 01:05:40.310 align:middle line:84% And some of them are disparity selective. 01:05:40.310 --> 01:05:42.890 align:middle line:84% So here's an example of a neuron that 01:05:42.890 --> 01:05:46.330 align:middle line:84% is selective for both orientation and disparity. 01:05:46.330 --> 01:05:51.690 align:middle line:84% So what you're seeing here is the position 01:05:51.690 --> 01:05:54.290 align:middle line:84% of the stimulus in the left eye and the right eye. 01:05:54.290 --> 01:05:59.050 align:middle line:84% And so these numbers here are the minutes of arc 01:05:59.050 --> 01:06:01.850 align:middle line:84% that separate the images in the two eyes. 01:06:01.850 --> 01:06:05.410 align:middle line:84% So that zero, that would correspond to zero disparity-- 01:06:05.410 --> 01:06:08.830 align:middle line:84% 15 minutes, 30 minutes, 45, 60, and so forth. 01:06:08.830 --> 01:06:11.730 align:middle line:84% And then this is the right eye and the left eye on its own. 01:06:11.730 --> 01:06:13.330 align:middle line:84% So this is an oriented bar, and it's 01:06:13.330 --> 01:06:16.250 align:middle line:84% being swept in one direction and another. 01:06:16.250 --> 01:06:18.990 align:middle line:84% And this is a trace from a recording. 01:06:18.990 --> 01:06:21.610 align:middle line:84% So these will show you the action potentials that result. 01:06:21.610 --> 01:06:25.490 align:middle line:84% So they've already figured out what the preferred orientation 01:06:25.490 --> 01:06:29.810 align:middle line:84% is for the cell, and so the bar is fixed at that orientation. 01:06:29.810 --> 01:06:33.210 align:middle line:84% But you can see that there is a disparity that 01:06:33.210 --> 01:06:36.330 align:middle line:84% elicits the biggest response where the left and the right eye 01:06:36.330 --> 01:06:39.670 align:middle line:84% images don't correspond perfectly. 01:06:39.670 --> 01:06:42.570 align:middle line:84% So they're offset by a little bit. 01:06:42.570 --> 01:06:45.010 align:middle line:84% And if you present either the right or the left eye 01:06:45.010 --> 01:06:49.170 align:middle line:84% on its own, you also get a weak response. 01:06:49.170 --> 01:06:51.590 align:middle line:84% So this happens even in early visual cortex. 01:06:51.590 --> 01:06:55.230 align:middle line:84% And again, that's consistent with the random-dot stereogram 01:06:55.230 --> 01:06:59.850 align:middle line:84% and the idea that stereopsis is something that at least starts 01:06:59.850 --> 01:07:03.410 align:middle line:90% early in the visual system. 01:07:03.410 --> 01:07:07.250 align:middle line:84% And so it's easy to imagine how you might wire up 01:07:07.250 --> 01:07:09.570 align:middle line:90% a disparity-tuned neuron. 01:07:09.570 --> 01:07:11.130 align:middle line:84% So the idea is just that you would 01:07:11.130 --> 01:07:17.660 align:middle line:84% have neurons that have receptive fields that 01:07:17.660 --> 01:07:20.660 align:middle line:84% are in different locations in the two eyes 01:07:20.660 --> 01:07:22.720 align:middle line:90% and that get wired together. 01:07:22.720 --> 01:07:27.860 align:middle line:90% 01:07:27.860 --> 01:07:29.740 align:middle line:84% You have a thing in the world that 01:07:29.740 --> 01:07:33.980 align:middle line:84% happens to project to the right combination of points in the two 01:07:33.980 --> 01:07:36.760 align:middle line:84% eyes, and those converge on some downstream neuron 01:07:36.760 --> 01:07:39.540 align:middle line:84% that's disparity selective, then you get a response. 01:07:39.540 --> 01:07:43.020 align:middle line:90% 01:07:43.020 --> 01:07:43.520 align:middle line:90% All right. 01:07:43.520 --> 01:07:47.300 align:middle line:84% So as I was mentioning earlier, there 01:07:47.300 --> 01:07:49.820 align:middle line:84% are some fairly common problems that can 01:07:49.820 --> 01:07:51.900 align:middle line:90% occur with binocular vision. 01:07:51.900 --> 01:07:56.420 align:middle line:84% So strabismus is a condition that people are often 01:07:56.420 --> 01:07:59.380 align:middle line:84% born with where the eyes are slightly misaligned. 01:07:59.380 --> 01:08:01.260 align:middle line:90% That can produce double vision. 01:08:01.260 --> 01:08:05.540 align:middle line:84% And so typically, in the United States, 01:08:05.540 --> 01:08:09.980 align:middle line:84% this would be detected very early on and corrected. 01:08:09.980 --> 01:08:11.480 align:middle line:84% Sometimes I think it just goes away. 01:08:11.480 --> 01:08:13.063 align:middle line:84% Sometimes there's a surgical procedure 01:08:13.063 --> 01:08:16.020 align:middle line:84% that is performed to get the eyes in alignment. 01:08:16.020 --> 01:08:19.580 align:middle line:84% But if it's not corrected, you develop 01:08:19.580 --> 01:08:21.779 align:middle line:84% something called amblyopia, where 01:08:21.779 --> 01:08:23.859 align:middle line:90% one eye suppresses the other. 01:08:23.859 --> 01:08:25.819 align:middle line:84% It's typically caused by strabismus. 01:08:25.819 --> 01:08:29.380 align:middle line:84% So strabismus is usually correctable by exercises 01:08:29.380 --> 01:08:31.100 align:middle line:90% or by surgery, like I said. 01:08:31.100 --> 01:08:32.600 align:middle line:90% So this is what it looks like. 01:08:32.600 --> 01:08:37.500 align:middle line:84% The eyes just won't quite be at the right relative positions. 01:08:37.500 --> 01:08:40.444 align:middle line:84% And childhood amblyopia is a big deal 01:08:40.444 --> 01:08:41.819 align:middle line:84% because if it's not corrected, it 01:08:41.819 --> 01:08:45.220 align:middle line:84% can cause permanent losses in binocular vision 01:08:45.220 --> 01:08:49.580 align:middle line:84% because it interferes with the development of binocular cells 01:08:49.580 --> 01:08:52.479 align:middle line:84% in the visual cortex that are selective for disparity. 01:08:52.479 --> 01:08:56.300 align:middle line:84% And so there's a critical period up to about six years of age. 01:08:56.300 --> 01:08:59.779 align:middle line:84% So the idea is that if you can correct the problem prior 01:08:59.779 --> 01:09:04.460 align:middle line:84% to six years, the person will be able to develop normal binocular 01:09:04.460 --> 01:09:04.960 align:middle line:90% vision. 01:09:04.960 --> 01:09:08.279 align:middle line:84% But if you wait beyond that, even if it's corrected, 01:09:08.279 --> 01:09:12.260 align:middle line:84% there's less plasticity after that point in time, 01:09:12.260 --> 01:09:16.330 align:middle line:84% and you don't develop the necessary abilities. 01:09:16.330 --> 01:09:17.452 align:middle line:90% Yeah? 01:09:17.452 --> 01:09:19.910 align:middle line:84% AUDIENCE: Under the stereo part, is that the suppressed eye 01:09:19.910 --> 01:09:24.029 align:middle line:84% development longer if someone had anybody at 20 years old 01:09:24.029 --> 01:09:27.149 align:middle line:84% lost their eye that was suppressing? 01:09:27.149 --> 01:09:31.010 align:middle line:84% Do they just continued normal vision without the stereopsy? 01:09:31.010 --> 01:09:35.557 align:middle line:84% Or is that the weaker and they'll be able to see less? 01:09:35.557 --> 01:09:36.890 align:middle line:90% JOSH MCDERMOTT: So I don't know. 01:09:36.890 --> 01:09:40.069 align:middle line:84% My guess would be that you would be 01:09:40.069 --> 01:09:45.510 align:middle line:84% able to recover some other aspects of vision 01:09:45.510 --> 01:09:47.890 align:middle line:84% and that it would just be stereo that you would lose, 01:09:47.890 --> 01:09:49.848 align:middle line:90% but I don't know for sure. 01:09:49.848 --> 01:09:52.390 align:middle line:84% I'm sure the answer is known because this is something that's 01:09:52.390 --> 01:09:53.490 align:middle line:90% been pretty well studied. 01:09:53.490 --> 01:09:57.750 align:middle line:84% It's just not my area of expertise. 01:09:57.750 --> 01:10:00.550 align:middle line:84% And so there are tests that will actually diagnose whether people 01:10:00.550 --> 01:10:01.810 align:middle line:90% have normal binocular vision. 01:10:01.810 --> 01:10:05.870 align:middle line:84% You can give them displays of that we've been looking at. 01:10:05.870 --> 01:10:09.230 align:middle line:84% And by the standard diagnosis, about 5% of people 01:10:09.230 --> 01:10:11.310 align:middle line:84% are what's considered to be stereoblind. 01:10:11.310 --> 01:10:15.357 align:middle line:84% So you give them moderate binocular disparities, 01:10:15.357 --> 01:10:17.690 align:middle line:84% and they won't be able to perceive depth based on those. 01:10:17.690 --> 01:10:20.230 align:middle line:84% So that's a pretty good percentage. 01:10:20.230 --> 01:10:27.230 align:middle line:84% Any questions about binocular vision or stereopsis? 01:10:27.230 --> 01:10:27.870 align:middle line:90% Yeah? 01:10:27.870 --> 01:10:31.630 align:middle line:84% AUDIENCE: If you did have amblyopia, for example, like, 01:10:31.630 --> 01:10:36.390 align:middle line:84% you were given a 3D image and these glasses have particular 01:10:36.390 --> 01:10:39.910 align:middle line:84% distance, would you still be able to have 3D effect? 01:10:39.910 --> 01:10:43.490 align:middle line:84% Or are you just not able to see everything with the glasses? 01:10:43.490 --> 01:10:46.590 align:middle line:90% 01:10:46.590 --> 01:10:49.456 align:middle line:90% JOSH MCDERMOTT: Yeah. 01:10:49.456 --> 01:10:51.670 align:middle line:84% These anaglyphs just wouldn't work. 01:10:51.670 --> 01:10:55.310 align:middle line:84% So I mean, you'd presumably have normal monocular depth 01:10:55.310 --> 01:10:57.350 align:middle line:84% perception, but you just wouldn't really 01:10:57.350 --> 01:11:00.590 align:middle line:90% get the benefit of stereopsis. 01:11:00.590 --> 01:11:04.550 align:middle line:84% I mean, these glasses are not doing-- 01:11:04.550 --> 01:11:07.270 align:middle line:84% what the glasses are doing is essentially recreating what 01:11:07.270 --> 01:11:11.940 align:middle line:84% happens normally, where there's a 3D world, 01:11:11.940 --> 01:11:13.760 align:middle line:84% and you get two slightly different images. 01:11:13.760 --> 01:11:15.760 align:middle line:84% Because the two eyes are in different positions, 01:11:15.760 --> 01:11:16.900 align:middle line:90% then the world is 3D. 01:11:16.900 --> 01:11:19.680 align:middle line:84% So what's happening is that what's useful about the glasses 01:11:19.680 --> 01:11:24.840 align:middle line:84% is you can look at a 2D screen and still get disparity. 01:11:24.840 --> 01:11:26.398 align:middle line:84% If you're looking at things normally, 01:11:26.398 --> 01:11:28.440 align:middle line:84% there would be no disparity looking at the screen 01:11:28.440 --> 01:11:30.260 align:middle line:84% because everything is at the same depth. 01:11:30.260 --> 01:11:36.360 align:middle line:84% So the anaglyphs are just a way to artificially get disparity 01:11:36.360 --> 01:11:39.785 align:middle line:84% that you would normally get from three dimensions. 01:11:39.785 --> 01:11:41.160 align:middle line:84% AUDIENCE: So as a follow-up, does 01:11:41.160 --> 01:11:45.000 align:middle line:84% that mean that this type of binocular depth perception 01:11:45.000 --> 01:11:48.840 align:middle line:90% is more cognitive than physical? 01:11:48.840 --> 01:11:51.880 align:middle line:84% JOSH MCDERMOTT: No, I think it's pretty automatic. 01:11:51.880 --> 01:11:53.800 align:middle line:84% I wouldn't call it cognitive at all. 01:11:53.800 --> 01:11:57.160 align:middle line:84% It just happens, and you're not even-- you don't really 01:11:57.160 --> 01:11:59.200 align:middle line:90% think about it. 01:11:59.200 --> 01:12:01.040 align:middle line:90% Yeah. 01:12:01.040 --> 01:12:01.582 align:middle line:90% Yeah? 01:12:01.582 --> 01:12:03.040 align:middle line:84% AUDIENCE: For the stereo blindness, 01:12:03.040 --> 01:12:08.680 align:middle line:84% is it only caused by problems with the development 01:12:08.680 --> 01:12:10.160 align:middle line:90% of the different context? 01:12:10.160 --> 01:12:16.960 align:middle line:84% Or does it also include problems caused by strabismus? 01:12:16.960 --> 01:12:19.098 align:middle line:90% Is that a level eye or complex? 01:12:19.098 --> 01:12:20.640 align:middle line:84% JOSH MCDERMOTT: Well, it can be both. 01:12:20.640 --> 01:12:23.565 align:middle line:84% So with strabismus, if the eyes kind 01:12:23.565 --> 01:12:25.440 align:middle line:84% don't align in the right way, then, yeah, you 01:12:25.440 --> 01:12:27.480 align:middle line:84% wouldn't necessarily be able to normally get 01:12:27.480 --> 01:12:29.760 align:middle line:84% images that are in register, which 01:12:29.760 --> 01:12:31.520 align:middle line:84% is why one-- and the solution to that 01:12:31.520 --> 01:12:33.460 align:middle line:84% is that one eye ends up dominating. 01:12:33.460 --> 01:12:35.167 align:middle line:84% That's how the system copes with it. 01:12:35.167 --> 01:12:37.000 align:middle line:84% AUDIENCE: So it's like colorblindness that's 01:12:37.000 --> 01:12:39.842 align:middle line:90% a cortical cause [INAUDIBLE]? 01:12:39.842 --> 01:12:41.300 align:middle line:84% JOSH MCDERMOTT: Yeah, I would say-- 01:12:41.300 --> 01:12:44.240 align:middle line:84% OK, so the way that I would think about this 01:12:44.240 --> 01:12:50.280 align:middle line:84% is that there's this cortical mechanism that normally develops 01:12:50.280 --> 01:12:54.120 align:middle line:84% if the eyes are positioned in the correct way 01:12:54.120 --> 01:12:56.340 align:middle line:84% such that the depth cue is available. 01:12:56.340 --> 01:12:58.280 align:middle line:84% So if that depth cue is there in the input, 01:12:58.280 --> 01:13:00.560 align:middle line:84% then your cortex develops, and you 01:13:00.560 --> 01:13:03.200 align:middle line:90% learn to detect the disparity. 01:13:03.200 --> 01:13:05.640 align:middle line:84% Now, if the eyes are misaligned and the depth cue is not 01:13:05.640 --> 01:13:08.330 align:middle line:84% available, then the cortex doesn't 01:13:08.330 --> 01:13:10.410 align:middle line:90% develop the disparity analysis. 01:13:10.410 --> 01:13:13.110 align:middle line:84% Now, subsequent to that, if you're, say, 10 years old, 01:13:13.110 --> 01:13:15.090 align:middle line:84% you can then correct the strabismus. 01:13:15.090 --> 01:13:17.870 align:middle line:84% So now you have disparities between the two eyes. 01:13:17.870 --> 01:13:20.650 align:middle line:84% So the cue is available, but the cortex 01:13:20.650 --> 01:13:23.490 align:middle line:84% is past its critical period for this particular thing. 01:13:23.490 --> 01:13:28.250 align:middle line:84% And so the cortex can no longer learn to develop the disparity 01:13:28.250 --> 01:13:30.010 align:middle line:90% sensitivity. 01:13:30.010 --> 01:13:30.790 align:middle line:90% Make sense? 01:13:30.790 --> 01:13:31.570 align:middle line:90% AUDIENCE: Yeah. 01:13:31.570 --> 01:13:33.290 align:middle line:90% JOSH MCDERMOTT: OK. 01:13:33.290 --> 01:13:34.770 align:middle line:90% OK, have a great weekend. 01:13:34.770 --> 01:13:39.050 align:middle line:84% Problem set's been extended to Monday, as you probably heard. 01:13:39.050 --> 01:13:42.000 align:middle line:84% Have a great weekend, and we'll see you Tuesday. 01:13:42.000 --> 01:14:07.000 align:middle line:90%