WEBVTT 00:00:00.000 --> 00:00:02.448 align:middle line:90% [SQUEAKING] 00:00:02.448 --> 00:00:03.909 align:middle line:90% [RUSTLING] 00:00:03.909 --> 00:00:06.344 align:middle line:90% [CLICKING] 00:00:06.344 --> 00:00:10.477 align:middle line:90% 00:00:10.477 --> 00:00:11.560 align:middle line:90% JOSH MCDERMOTT: All right. 00:00:11.560 --> 00:00:13.680 align:middle line:90% Let's talk about perception. 00:00:13.680 --> 00:00:15.840 align:middle line:84% So perception is the task of determining 00:00:15.840 --> 00:00:19.060 align:middle line:84% what is out there in the world from sensory input. 00:00:19.060 --> 00:00:21.120 align:middle line:84% So as an organism, you need to know 00:00:21.120 --> 00:00:22.340 align:middle line:90% what's happening around you. 00:00:22.340 --> 00:00:25.280 align:middle line:84% And things in the world, they give off different types 00:00:25.280 --> 00:00:28.000 align:middle line:90% of clues to their existence. 00:00:28.000 --> 00:00:30.973 align:middle line:84% And organisms have sensory organs that detect these clues. 00:00:30.973 --> 00:00:32.640 align:middle line:84% And the nature of the clues is different 00:00:32.640 --> 00:00:33.700 align:middle line:90% for the different senses. 00:00:33.700 --> 00:00:36.240 align:middle line:84% So in vision, photons that originate 00:00:36.240 --> 00:00:38.960 align:middle line:84% at some kind of light source reflect off objects 00:00:38.960 --> 00:00:41.140 align:middle line:90% and are absorbed by the eye. 00:00:41.140 --> 00:00:42.640 align:middle line:84% And the pattern of photons gives you 00:00:42.640 --> 00:00:45.840 align:middle line:84% information about the objects that they reflected off. 00:00:45.840 --> 00:00:49.073 align:middle line:84% In hearing, objects cause vibrations in the air. 00:00:49.073 --> 00:00:50.240 align:middle line:90% They travel through the air. 00:00:50.240 --> 00:00:54.300 align:middle line:84% And they're absorbed and measured by the ear. 00:00:54.300 --> 00:00:56.540 align:middle line:84% With the sense of touch, we bump into things, 00:00:56.540 --> 00:00:57.980 align:middle line:84% intentionally or unintentionally. 00:00:57.980 --> 00:01:01.730 align:middle line:84% And that stimulates receptors in your skin 00:01:01.730 --> 00:01:05.209 align:middle line:84% that respond to pressure and other things. 00:01:05.209 --> 00:01:07.610 align:middle line:84% With the sense of taste, we lick things. 00:01:07.610 --> 00:01:10.810 align:middle line:84% And molecules that are contained in the substances that we lick 00:01:10.810 --> 00:01:13.270 align:middle line:84% interact with the taste receptors in your tongue. 00:01:13.270 --> 00:01:15.890 align:middle line:84% And turns out they also interact with the olfactory receptors 00:01:15.890 --> 00:01:18.090 align:middle line:90% in your nose, as we'll learn. 00:01:18.090 --> 00:01:20.970 align:middle line:84% And with the sense of smell, substances in the world 00:01:20.970 --> 00:01:23.650 align:middle line:84% give off molecules that float through the air 00:01:23.650 --> 00:01:26.430 align:middle line:84% and interact with the receptors that are in your nose. 00:01:26.430 --> 00:01:30.050 align:middle line:90% 00:01:30.050 --> 00:01:30.550 align:middle line:90% OK. 00:01:30.550 --> 00:01:32.050 align:middle line:84% So we've got these sensory organs 00:01:32.050 --> 00:01:33.850 align:middle line:84% that measure these different types of clues 00:01:33.850 --> 00:01:35.142 align:middle line:90% that are coming from the world. 00:01:35.142 --> 00:01:38.170 align:middle line:84% Now, the task of perception is to take input 00:01:38.170 --> 00:01:41.670 align:middle line:84% from these sensory receptors, and then, with that input, 00:01:41.670 --> 00:01:44.790 align:middle line:84% figure out what is out there in the world. 00:01:44.790 --> 00:01:48.152 align:middle line:90% 00:01:48.152 --> 00:01:49.610 align:middle line:84% So the first important point I want 00:01:49.610 --> 00:01:51.027 align:middle line:84% you to take away from this lecture 00:01:51.027 --> 00:01:55.150 align:middle line:84% is that perception is deceptively hard. 00:01:55.150 --> 00:02:00.546 align:middle line:90% 00:02:00.546 --> 00:02:02.500 align:middle line:84% Now, normally, you just open your eyes 00:02:02.500 --> 00:02:05.660 align:middle line:84% or you listen and you effortlessly 00:02:05.660 --> 00:02:07.700 align:middle line:90% apprehend the world around you. 00:02:07.700 --> 00:02:09.253 align:middle line:90% I look out at you all. 00:02:09.253 --> 00:02:11.420 align:middle line:84% And I'm seeing all these chairs, and people in them, 00:02:11.420 --> 00:02:13.020 align:middle line:90% and people looking at me. 00:02:13.020 --> 00:02:16.080 align:middle line:84% And I don't really have to try to do it most of the time. 00:02:16.080 --> 00:02:17.280 align:middle line:90% It just kind of happens. 00:02:17.280 --> 00:02:23.180 align:middle line:84% And this often makes it non-obvious, 00:02:23.180 --> 00:02:25.100 align:middle line:84% all of the complicated stuff is going 00:02:25.100 --> 00:02:28.700 align:middle line:84% on that actually enables us to derive that kind of information 00:02:28.700 --> 00:02:29.980 align:middle line:90% about the world. 00:02:29.980 --> 00:02:32.860 align:middle line:84% And one way to get some perspective on this 00:02:32.860 --> 00:02:35.820 align:middle line:84% is to actually view the sensory input 00:02:35.820 --> 00:02:37.380 align:middle line:90% in a slightly different way. 00:02:37.380 --> 00:02:41.420 align:middle line:84% And so here, what we have is an image. 00:02:41.420 --> 00:02:46.860 align:middle line:84% But instead of representing it like a normal image, where there 00:02:46.860 --> 00:02:52.700 align:middle line:84% will be pigmentation on the screen at a particular point, 00:02:52.700 --> 00:02:54.352 align:middle line:90% here we have numbers. 00:02:54.352 --> 00:02:55.560 align:middle line:90% So this is a grayscale image. 00:02:55.560 --> 00:02:58.350 align:middle line:84% And the number here represents the gray level 00:02:58.350 --> 00:03:00.030 align:middle line:84% at a particular point in the image. 00:03:00.030 --> 00:03:02.510 align:middle line:84% So you can think of this as if you snapped a picture 00:03:02.510 --> 00:03:04.430 align:middle line:90% with your phone. 00:03:04.430 --> 00:03:06.910 align:middle line:84% And if it was a black and white camera, 00:03:06.910 --> 00:03:12.490 align:middle line:84% this is what the CCD from your phone camera would output. 00:03:12.490 --> 00:03:14.370 align:middle line:90% It's an array of numbers. 00:03:14.370 --> 00:03:17.870 align:middle line:90% 00:03:17.870 --> 00:03:20.322 align:middle line:84% And so your task here is to take input like this, 00:03:20.322 --> 00:03:22.530 align:middle line:84% and then to figure out what's out there in the world. 00:03:22.530 --> 00:03:24.072 align:middle line:84% And so when you look at it like this, 00:03:24.072 --> 00:03:25.990 align:middle line:84% it makes it clear that, well, this is actually 00:03:25.990 --> 00:03:28.750 align:middle line:84% kind of non-obvious, because normally what happens 00:03:28.750 --> 00:03:30.830 align:middle line:84% is when you look at the image, well, your brain 00:03:30.830 --> 00:03:32.065 align:middle line:90% is doing its thing. 00:03:32.065 --> 00:03:34.190 align:middle line:84% So there's all this complicated stuff that happens. 00:03:34.190 --> 00:03:36.110 align:middle line:90% And that causes you to see. 00:03:36.110 --> 00:03:39.170 align:middle line:90% But it all starts with this. 00:03:39.170 --> 00:03:41.550 align:middle line:84% Now, when you talk to people on the street 00:03:41.550 --> 00:03:44.550 align:middle line:84% and you tell them that you study, for instance, vision, 00:03:44.550 --> 00:03:47.390 align:middle line:84% they often think that you must work on the eye. 00:03:47.390 --> 00:03:50.150 align:middle line:90% So here's the eye. 00:03:50.150 --> 00:03:51.950 align:middle line:90% So the eye is amazing. 00:03:51.950 --> 00:03:56.150 align:middle line:84% The eye is this incredible device 00:03:56.150 --> 00:04:01.150 align:middle line:84% that evolved to form an image and to measure that image. 00:04:01.150 --> 00:04:03.390 align:middle line:90% So we've got a lens there. 00:04:03.390 --> 00:04:06.330 align:middle line:84% Light passes through the cornea, and then the lens. 00:04:06.330 --> 00:04:08.870 align:middle line:84% And it gets focused onto an image on the back 00:04:08.870 --> 00:04:11.030 align:middle line:90% of the eye called the retina. 00:04:11.030 --> 00:04:15.070 align:middle line:84% So this is a close-up schematic view 00:04:15.070 --> 00:04:16.630 align:middle line:90% of a little piece of the retina. 00:04:16.630 --> 00:04:20.070 align:middle line:84% So the light rays come through here. 00:04:20.070 --> 00:04:21.838 align:middle line:84% So one weird thing about the retina 00:04:21.838 --> 00:04:23.630 align:middle line:84% is that it actually is, in a certain sense, 00:04:23.630 --> 00:04:24.770 align:middle line:90% wired up backwards. 00:04:24.770 --> 00:04:26.690 align:middle line:84% And so the light has to pass through a whole bunch of stuff. 00:04:26.690 --> 00:04:28.690 align:middle line:84% But then it eventually gets to the things up at the top. 00:04:28.690 --> 00:04:29.898 align:middle line:90% Those are the photoreceptors. 00:04:29.898 --> 00:04:32.190 align:middle line:84% And so the photoreceptors are a special type of cell 00:04:32.190 --> 00:04:36.550 align:middle line:84% that absorb photons, and then take that absorbed photon 00:04:36.550 --> 00:04:40.270 align:middle line:84% and turn that into a change in voltage. 00:04:40.270 --> 00:04:44.110 align:middle line:84% So they take light and they turn it into an electrical signal. 00:04:44.110 --> 00:04:44.650 align:middle line:90% All right. 00:04:44.650 --> 00:04:49.430 align:middle line:84% And so this is happening across the entire retinal image. 00:04:49.430 --> 00:04:52.250 align:middle line:84% And so you effectively get a spatial array of voltages. 00:04:52.250 --> 00:04:55.520 align:middle line:84% So it's really analogous to this array of numbers 00:04:55.520 --> 00:05:00.292 align:middle line:84% here, except that it's happening in your eye. 00:05:00.292 --> 00:05:02.000 align:middle line:84% So you can think of your eye as something 00:05:02.000 --> 00:05:06.200 align:middle line:84% that is measuring light, turning that into a bunch of numbers, 00:05:06.200 --> 00:05:09.740 align:middle line:84% and then sending that through the optic nerve to your brain. 00:05:09.740 --> 00:05:13.220 align:middle line:90% 00:05:13.220 --> 00:05:16.340 align:middle line:84% So that's just the start of it, though. 00:05:16.340 --> 00:05:18.327 align:middle line:90% So the eyes just measure light. 00:05:18.327 --> 00:05:19.660 align:middle line:90% They don't interpret it for you. 00:05:19.660 --> 00:05:22.097 align:middle line:84% And that's really the job of the brain. 00:05:22.097 --> 00:05:23.680 align:middle line:84% And so one piece of evidence that this 00:05:23.680 --> 00:05:25.960 align:middle line:84% is a pretty complicated thing comes from the fact 00:05:25.960 --> 00:05:27.760 align:middle line:84% that really a large fraction of the brain 00:05:27.760 --> 00:05:30.940 align:middle line:84% is devoted to seeing, so roughly 50%, 00:05:30.940 --> 00:05:34.760 align:middle line:84% depending on how you measure it in humans, more in monkeys. 00:05:34.760 --> 00:05:39.480 align:middle line:84% There's another pretty big piece that's devoted to hearing. 00:05:39.480 --> 00:05:42.820 align:middle line:84% So there's very similar issues with audition. 00:05:42.820 --> 00:05:45.300 align:middle line:84% So just listen to this particular sound signal. 00:05:45.300 --> 00:05:55.943 align:middle line:90% 00:05:55.943 --> 00:05:57.392 align:middle line:90% [AUDIO PLAYBACK] 00:05:57.392 --> 00:05:58.358 align:middle line:90% [INTERPOSING VOICES] 00:05:58.358 --> 00:06:01.090 align:middle line:90% [END PLAYBACK] 00:06:01.090 --> 00:06:03.030 align:middle line:84% JOSH MCDERMOTT: OK, so what did you just hear? 00:06:03.030 --> 00:06:06.290 align:middle line:90% 00:06:06.290 --> 00:06:07.765 align:middle line:90% Yeah, just shout it out. 00:06:07.765 --> 00:06:08.890 align:middle line:90% AUDIENCE: Background noise. 00:06:08.890 --> 00:06:09.830 align:middle line:84% JOSH MCDERMOTT: Some background noise? 00:06:09.830 --> 00:06:11.413 align:middle line:84% Yeah, there was some background noise. 00:06:11.413 --> 00:06:12.230 align:middle line:90% Yeah, what else? 00:06:12.230 --> 00:06:13.410 align:middle line:90% AUDIENCE: A question. 00:06:13.410 --> 00:06:14.710 align:middle line:84% JOSH MCDERMOTT: Yeah, somebody asking a question. 00:06:14.710 --> 00:06:15.210 align:middle line:90% Yeah. 00:06:15.210 --> 00:06:17.307 align:middle line:84% How many people were talking in that? 00:06:17.307 --> 00:06:17.890 align:middle line:90% AUDIENCE: Two. 00:06:17.890 --> 00:06:19.025 align:middle line:90% AUDIENCE: At least two. 00:06:19.025 --> 00:06:19.650 align:middle line:90% AUDIENCE: Four. 00:06:19.650 --> 00:06:20.030 align:middle line:90% JOSH MCDERMOTT: Two or three. 00:06:20.030 --> 00:06:21.350 align:middle line:90% OK, yeah. 00:06:21.350 --> 00:06:23.870 align:middle line:84% What kind of-- where do you think that was recorded? 00:06:23.870 --> 00:06:24.830 align:middle line:90% AUDIENCE: A restaurant. 00:06:24.830 --> 00:06:26.450 align:middle line:84% JOSH MCDERMOTT: A restaurant, OK. 00:06:26.450 --> 00:06:28.690 align:middle line:84% All right, so just from listening to that, 00:06:28.690 --> 00:06:30.690 align:middle line:84% you just immediately know all these things that 00:06:30.690 --> 00:06:34.330 align:middle line:90% are happening in the world. 00:06:34.330 --> 00:06:37.189 align:middle line:90% But the sensory input that-- 00:06:37.189 --> 00:06:37.856 align:middle line:90% [AUDIO PLAYBACK] 00:06:37.856 --> 00:06:38.790 align:middle line:90% [INTERPOSING VOICES] 00:06:38.790 --> 00:06:39.170 align:middle line:90% [END PLAYBACK] 00:06:39.170 --> 00:06:40.503 align:middle line:90% JOSH MCDERMOTT: Oops, kill that. 00:06:40.503 --> 00:06:41.970 align:middle line:84% The sensory input that you received 00:06:41.970 --> 00:06:43.410 align:middle line:90% was a pressure waveform. 00:06:43.410 --> 00:06:45.590 align:middle line:90% So there was a sound signal. 00:06:45.590 --> 00:06:46.850 align:middle line:90% It started off in my computer. 00:06:46.850 --> 00:06:50.500 align:middle line:84% And it traveled to the speakers in this room. 00:06:50.500 --> 00:06:52.753 align:middle line:84% And there was a diaphragm that wiggled back and forth. 00:06:52.753 --> 00:06:54.420 align:middle line:84% The sound wave traveled through the air. 00:06:54.420 --> 00:06:56.760 align:middle line:84% And it caused pressure variation at your eardrum. 00:06:56.760 --> 00:06:58.760 align:middle line:84% And that made your eardrum wiggle back and forth 00:06:58.760 --> 00:07:00.200 align:middle line:90% in some particular pattern. 00:07:00.200 --> 00:07:02.100 align:middle line:84% So you can think of this waveform 00:07:02.100 --> 00:07:07.260 align:middle line:84% here as the eardrum displacement as a function of time. 00:07:07.260 --> 00:07:12.300 align:middle line:84% But really, at some level, it's just a time series of numbers. 00:07:12.300 --> 00:07:14.700 align:middle line:84% And so from that time series of numbers, 00:07:14.700 --> 00:07:17.060 align:middle line:84% you were able to determine all those things about what 00:07:17.060 --> 00:07:20.020 align:middle line:84% was going on in the world to cause that signal when 00:07:20.020 --> 00:07:21.040 align:middle line:90% it was recorded. 00:07:21.040 --> 00:07:24.420 align:middle line:84% And that's your brain doing its thing. 00:07:24.420 --> 00:07:26.740 align:middle line:84% So the ear, kind of analogous to the eye, 00:07:26.740 --> 00:07:30.340 align:middle line:84% is a really remarkable device for transducing 00:07:30.340 --> 00:07:34.460 align:middle line:84% the mechanical energy from sound into electrical signals that 00:07:34.460 --> 00:07:35.463 align:middle line:90% get sent to the brain. 00:07:35.463 --> 00:07:36.880 align:middle line:84% So this is a schematic of the ear. 00:07:36.880 --> 00:07:38.240 align:middle line:90% We've got the eardrum here. 00:07:38.240 --> 00:07:40.900 align:middle line:84% And then there's an organ called the cochlea 00:07:40.900 --> 00:07:45.700 align:middle line:84% that ends up turning the sound energy into electrical signals. 00:07:45.700 --> 00:07:48.670 align:middle line:84% But your brain then-- so this is the auditory nerve, 00:07:48.670 --> 00:07:50.910 align:middle line:90% analogous to the optic nerve. 00:07:50.910 --> 00:07:53.790 align:middle line:84% And your brain then gets the signals from the auditory nerve 00:07:53.790 --> 00:07:55.390 align:middle line:84% and does all this complicated stuff 00:07:55.390 --> 00:07:59.870 align:middle line:84% to cause you to hear what is there. 00:07:59.870 --> 00:08:00.370 align:middle line:90% All right. 00:08:00.370 --> 00:08:01.745 align:middle line:84% The second important point that I 00:08:01.745 --> 00:08:04.230 align:middle line:84% want you to come out of this lecture with 00:08:04.230 --> 00:08:08.390 align:middle line:84% is that perceptual problems are usually ill-posed. 00:08:08.390 --> 00:08:12.830 align:middle line:84% So ill-posed means that there's not enough information 00:08:12.830 --> 00:08:16.430 align:middle line:84% to uniquely determine the answer to the problem. 00:08:16.430 --> 00:08:21.250 align:middle line:84% And most perceptual problems actually are of this nature. 00:08:21.250 --> 00:08:24.070 align:middle line:84% So one classic example derives from the fact 00:08:24.070 --> 00:08:27.670 align:middle line:84% that the world is three dimensional. 00:08:27.670 --> 00:08:30.350 align:middle line:84% And we usually are pretty good at correctly 00:08:30.350 --> 00:08:32.669 align:middle line:84% perceiving its three-dimensional structure. 00:08:32.669 --> 00:08:35.390 align:middle line:84% So I can reach and grab this coffee cup. 00:08:35.390 --> 00:08:37.530 align:middle line:84% And that requires that I know the shape of the cup. 00:08:37.530 --> 00:08:38.549 align:middle line:90% Otherwise, I'd knock it over. 00:08:38.549 --> 00:08:40.929 align:middle line:84% Or my hand wouldn't be closed tight enough, and so forth. 00:08:40.929 --> 00:08:45.510 align:middle line:84% And that's primarily something that happens visually. 00:08:45.510 --> 00:08:49.520 align:middle line:84% Similarly, I know roughly how far away each of you 00:08:49.520 --> 00:08:50.823 align:middle line:90% are from me and so forth. 00:08:50.823 --> 00:08:52.240 align:middle line:84% So we're pretty good at perceiving 00:08:52.240 --> 00:08:53.920 align:middle line:90% three-dimensional structure. 00:08:53.920 --> 00:08:57.080 align:middle line:84% But the input to perception, in particular to vision, 00:08:57.080 --> 00:08:58.720 align:middle line:90% is two-dimensional. 00:08:58.720 --> 00:09:00.540 align:middle line:84% We form an image on the back of our eye. 00:09:00.540 --> 00:09:03.420 align:middle line:84% Now, we have two images, one for the left and one for the right. 00:09:03.420 --> 00:09:07.800 align:middle line:84% And that actually is part of the solution. 00:09:07.800 --> 00:09:10.040 align:middle line:84% But you can close one eye and depth perception 00:09:10.040 --> 00:09:12.680 align:middle line:90% is still pretty good right. 00:09:12.680 --> 00:09:13.180 align:middle line:90% All right. 00:09:13.180 --> 00:09:16.960 align:middle line:84% So the depth information that is there in the third dimension 00:09:16.960 --> 00:09:21.400 align:middle line:84% is lost in the projection that forms images. 00:09:21.400 --> 00:09:23.777 align:middle line:84% And so there's lots of different shapes, for instance, 00:09:23.777 --> 00:09:25.360 align:middle line:84% that have the same three-dimensional-- 00:09:25.360 --> 00:09:27.340 align:middle line:84% have the same 2D projection onto an image. 00:09:27.340 --> 00:09:29.760 align:middle line:84% So each of these is like a different three-dimensional 00:09:29.760 --> 00:09:32.920 align:middle line:84% shape, but they line up in just the right way 00:09:32.920 --> 00:09:35.288 align:middle line:84% and would all cause the same image. 00:09:35.288 --> 00:09:36.580 align:middle line:90% There's lots of other examples. 00:09:36.580 --> 00:09:39.360 align:middle line:84% This is another one that's kind of interesting. 00:09:39.360 --> 00:09:43.040 align:middle line:84% So this is a bunch of moving dots. 00:09:43.040 --> 00:09:47.810 align:middle line:84% And the same dot motion is consistent with a whole bunch 00:09:47.810 --> 00:09:49.770 align:middle line:90% of different possible objects. 00:09:49.770 --> 00:09:51.530 align:middle line:90% So here, we have triangles. 00:09:51.530 --> 00:09:55.250 align:middle line:90% Here, we've got some squares. 00:09:55.250 --> 00:09:56.550 align:middle line:90% This is something else. 00:09:56.550 --> 00:09:59.330 align:middle line:90% 00:09:59.330 --> 00:10:01.050 align:middle line:90% This is something else. 00:10:01.050 --> 00:10:03.910 align:middle line:84% So it's exactly the same the same motions. 00:10:03.910 --> 00:10:06.370 align:middle line:84% But they could be grouped differently 00:10:06.370 --> 00:10:09.930 align:middle line:84% and either represent fixed points of objects or things 00:10:09.930 --> 00:10:13.890 align:middle line:84% that are moving along other things. 00:10:13.890 --> 00:10:19.650 align:middle line:84% Another classic example is that of auditory scenes. 00:10:19.650 --> 00:10:22.050 align:middle line:84% So usually in the world, there will 00:10:22.050 --> 00:10:25.210 align:middle line:84% be more than one thing that's making sound at the same time. 00:10:25.210 --> 00:10:28.770 align:middle line:84% And the vibrations that are caused by different sound 00:10:28.770 --> 00:10:32.010 align:middle line:84% sources-- so my voice and the rustling of your neighbor 00:10:32.010 --> 00:10:34.970 align:middle line:84% turning the page of their notes, for instance-- 00:10:34.970 --> 00:10:36.587 align:middle line:90% they sum together at the ear. 00:10:36.587 --> 00:10:38.170 align:middle line:84% So the signal that you get at your ear 00:10:38.170 --> 00:10:40.890 align:middle line:84% is a mixture of the sounds that would 00:10:40.890 --> 00:10:45.850 align:middle line:84% have been caused by the individual events on their own. 00:10:45.850 --> 00:10:48.250 align:middle line:84% But as an organism, typically, what you want to hear 00:10:48.250 --> 00:10:50.870 align:middle line:84% and what you need to hear are the individual sounds. 00:10:50.870 --> 00:10:52.770 align:middle line:84% You need to understand what I'm saying. 00:10:52.770 --> 00:10:55.570 align:middle line:84% You want to understand whether somebody is turning the page, 00:10:55.570 --> 00:10:58.890 align:middle line:84% or walking close to you, or whatever it may be. 00:10:58.890 --> 00:11:00.410 align:middle line:84% And so the problem that you really 00:11:00.410 --> 00:11:05.170 align:middle line:84% have to solve there is akin to me giving you this equation 00:11:05.170 --> 00:11:07.755 align:middle line:90% and asking you to solve for x. 00:11:07.755 --> 00:11:09.130 align:middle line:84% And so if I put that on the exam, 00:11:09.130 --> 00:11:10.410 align:middle line:84% you're all going to complain because there 00:11:10.410 --> 00:11:11.750 align:middle line:90% isn't a unique solution. 00:11:11.750 --> 00:11:14.490 align:middle line:84% There's lots of different combinations of x and y 00:11:14.490 --> 00:11:16.530 align:middle line:84% that could sum to the same number-- 00:11:16.530 --> 00:11:18.672 align:middle line:90% one equation, two unknowns. 00:11:18.672 --> 00:11:20.130 align:middle line:84% But that's exactly the problem that 00:11:20.130 --> 00:11:23.010 align:middle line:84% is happening when you have a mixture of sounds 00:11:23.010 --> 00:11:27.730 align:middle line:84% and you have to understand one or more of them. 00:11:27.730 --> 00:11:31.950 align:middle line:84% And so somehow, in the case of this auditory scene problem, 00:11:31.950 --> 00:11:35.570 align:middle line:84% we can usually hear the constituent sounds reasonably 00:11:35.570 --> 00:11:36.188 align:middle line:90% well. 00:11:36.188 --> 00:11:37.730 align:middle line:84% And maybe the classic example of this 00:11:37.730 --> 00:11:41.420 align:middle line:84% is what's called the cocktail party problem. 00:11:41.420 --> 00:11:45.435 align:middle line:84% How many people know what movie this is from? 00:11:45.435 --> 00:11:47.220 align:middle line:84% AUDIENCE: Breakfast at Tiffany's. 00:11:47.220 --> 00:11:48.200 align:middle line:84% JOSH MCDERMOTT: Breakfast at Tiffany's. 00:11:48.200 --> 00:11:48.908 align:middle line:90% Yeah, you got it. 00:11:48.908 --> 00:11:50.660 align:middle line:90% Yeah, good. 00:11:50.660 --> 00:11:56.420 align:middle line:84% Yeah, it's classic, Audrey Hepburn. 00:11:56.420 --> 00:12:00.255 align:middle line:84% So at a cocktail party, you're often 00:12:00.255 --> 00:12:01.380 align:middle line:90% trying to talk to somebody. 00:12:01.380 --> 00:12:03.338 align:middle line:84% So there's somebody who you want to understand. 00:12:03.338 --> 00:12:05.753 align:middle line:84% And so maybe it's somebody saying this-- 00:12:05.753 --> 00:12:06.420 align:middle line:90% [AUDIO PLAYBACK] 00:12:06.420 --> 00:12:08.300 align:middle line:90% - She argues with her sister. 00:12:08.300 --> 00:12:09.483 align:middle line:90% [END PLAYBACK] 00:12:09.483 --> 00:12:11.400 align:middle line:84% JOSH MCDERMOTT: And next to that is a picture. 00:12:11.400 --> 00:12:13.855 align:middle line:84% It's a way of turning a sound into a picture, very 00:12:13.855 --> 00:12:15.480 align:middle line:84% similar to what's called a spectrogram. 00:12:15.480 --> 00:12:18.800 align:middle line:84% So we have frequency on the y-axis and time on the x-axis. 00:12:18.800 --> 00:12:21.100 align:middle line:84% So there's all this structure in that speech signal 00:12:21.100 --> 00:12:22.552 align:middle line:84% that allows you to understand what 00:12:22.552 --> 00:12:23.760 align:middle line:90% the person was trying to say. 00:12:23.760 --> 00:12:26.580 align:middle line:84% But the problem that you might encounter 00:12:26.580 --> 00:12:29.153 align:middle line:84% is that what actually enters your ears might be this. 00:12:29.153 --> 00:12:29.820 align:middle line:90% [AUDIO PLAYBACK] 00:12:29.820 --> 00:12:30.653 align:middle line:90% [INTERPOSING VOICES] 00:12:30.653 --> 00:12:31.580 align:middle line:90% [END PLAYBACK] 00:12:31.580 --> 00:12:33.788 align:middle line:84% JOSH MCDERMOTT: There's another person talking there. 00:12:33.788 --> 00:12:36.745 align:middle line:84% And so now, that picture is kind of complicated, 00:12:36.745 --> 00:12:38.620 align:middle line:84% because it's got these two sound signals that 00:12:38.620 --> 00:12:40.050 align:middle line:90% are on top of each other. 00:12:40.050 --> 00:12:43.150 align:middle line:84% But it could also be that there's even 00:12:43.150 --> 00:12:44.790 align:middle line:90% more people talking. 00:12:44.790 --> 00:12:45.650 align:middle line:90% [AUDIO PLAYBACK] 00:12:45.650 --> 00:12:46.080 align:middle line:90% [INTERPOSING VOICES] 00:12:46.080 --> 00:12:46.663 align:middle line:90% [END PLAYBACK] 00:12:46.663 --> 00:12:49.870 align:middle line:84% JOSH MCDERMOTT: Or even seven other people talking-- 00:12:49.870 --> 00:12:50.557 align:middle line:90% [AUDIO PLAYBACK] 00:12:50.557 --> 00:12:51.390 align:middle line:90% [INTERPOSING VOICES] 00:12:51.390 --> 00:12:52.030 align:middle line:90% [END PLAYBACK] 00:12:52.030 --> 00:12:54.450 align:middle line:84% JOSH MCDERMOTT: So by the time you get down to here, 00:12:54.450 --> 00:12:56.370 align:middle line:84% it's really a pretty serious mess. 00:12:56.370 --> 00:12:58.870 align:middle line:84% But you could hopefully probably tell 00:12:58.870 --> 00:13:02.430 align:middle line:84% that you had a pretty good ability to actually hear out 00:13:02.430 --> 00:13:06.650 align:middle line:84% that target voice throughout these examples. 00:13:06.650 --> 00:13:09.530 align:middle line:90% 00:13:09.530 --> 00:13:12.110 align:middle line:84% So in general, in many cases, speech 00:13:12.110 --> 00:13:15.652 align:middle line:84% remains intelligible, despite the presence of other speakers. 00:13:15.652 --> 00:13:17.110 align:middle line:84% This is a problem that humans still 00:13:17.110 --> 00:13:19.950 align:middle line:84% solve substantially better than machines. 00:13:19.950 --> 00:13:22.070 align:middle line:84% So present day speech recognition algorithms, 00:13:22.070 --> 00:13:24.870 align:middle line:84% like in your iPhone, they work pretty great now, 00:13:24.870 --> 00:13:26.350 align:middle line:90% if you're in a quiet room. 00:13:26.350 --> 00:13:29.950 align:middle line:84% But in a situation where there's a lot of other people talking, 00:13:29.950 --> 00:13:33.470 align:middle line:84% they'll typically still perform poorly. 00:13:33.470 --> 00:13:39.560 align:middle line:84% OK, so that's another example of an ill-posed problem. 00:13:39.560 --> 00:13:42.240 align:middle line:84% So one of the amazing things about perception 00:13:42.240 --> 00:13:44.960 align:middle line:84% is that, despite the fact that we're constantly confronted 00:13:44.960 --> 00:13:47.840 align:middle line:84% with these ill-posed problems, which means that there's not 00:13:47.840 --> 00:13:51.280 align:middle line:84% usually a unique solution, usually 00:13:51.280 --> 00:13:53.920 align:middle line:84% we arrive at a single unambiguous interpretation 00:13:53.920 --> 00:13:56.560 align:middle line:90% of a stimulus. 00:13:56.560 --> 00:13:58.360 align:middle line:90% So you just open your eyes. 00:13:58.360 --> 00:13:59.580 align:middle line:90% And you see what's there. 00:13:59.580 --> 00:14:00.820 align:middle line:90% And it's usually correct. 00:14:00.820 --> 00:14:03.320 align:middle line:84% Again, this is what enables you to pick things up, and avoid 00:14:03.320 --> 00:14:06.880 align:middle line:84% running into things, and so forth. 00:14:06.880 --> 00:14:11.080 align:middle line:84% There are, however, interesting cases where 00:14:11.080 --> 00:14:12.800 align:middle line:90% perception can be ambiguous. 00:14:12.800 --> 00:14:16.220 align:middle line:84% And those often suddenly give you this insight into, oh yeah, 00:14:16.220 --> 00:14:19.160 align:middle line:84% it's not always completely determined. 00:14:19.160 --> 00:14:22.180 align:middle line:84% So this kind of looks like a cat. 00:14:22.180 --> 00:14:23.380 align:middle line:90% Wait, actually, no, hang on. 00:14:23.380 --> 00:14:25.800 align:middle line:90% That's a crow, right? 00:14:25.800 --> 00:14:26.940 align:middle line:90% I don't know which it is. 00:14:26.940 --> 00:14:28.400 align:middle line:90% It could be either. 00:14:28.400 --> 00:14:30.220 align:middle line:84% This one kind of makes your brain hurt. 00:14:30.220 --> 00:14:32.940 align:middle line:90% 00:14:32.940 --> 00:14:34.500 align:middle line:90% Which way is this person facing? 00:14:34.500 --> 00:14:39.833 align:middle line:90% 00:14:39.833 --> 00:14:41.250 align:middle line:84% AUDIENCE: I'm going to say they're 00:14:41.250 --> 00:14:43.653 align:middle line:84% facing forward and holding up a cutout piece of paper. 00:14:43.653 --> 00:14:45.070 align:middle line:84% JOSH MCDERMOTT: It could be, yeah. 00:14:45.070 --> 00:14:45.670 align:middle line:90% It could be. 00:14:45.670 --> 00:14:48.730 align:middle line:84% But I kind of feel like the person's looking that way. 00:14:48.730 --> 00:14:51.290 align:middle line:84% Wait, actually, no, they're looking that way. 00:14:51.290 --> 00:14:51.790 align:middle line:90% Yeah. 00:14:51.790 --> 00:14:53.110 align:middle line:84% And if you look at this for a while, it may. 00:14:53.110 --> 00:14:54.110 align:middle line:90% It may change. 00:14:54.110 --> 00:14:57.770 align:middle line:90% 00:14:57.770 --> 00:15:00.460 align:middle line:90% What is this? 00:15:00.460 --> 00:15:02.890 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:15:02.890 --> 00:15:05.583 align:middle line:84% JOSH MCDERMOTT: Is it a dog or is it a person? 00:15:05.583 --> 00:15:08.541 align:middle line:90% [INTERPOSING VOICES] 00:15:08.541 --> 00:15:16.570 align:middle line:90% 00:15:16.570 --> 00:15:19.312 align:middle line:84% JOSH MCDERMOTT: And are this person's legs shiny? 00:15:19.312 --> 00:15:21.970 align:middle line:90% [INTERPOSING VOICES] 00:15:21.970 --> 00:15:24.570 align:middle line:84% JOSH MCDERMOTT: How many people think this looks shiny? 00:15:24.570 --> 00:15:26.673 align:middle line:90% Yeah? 00:15:26.673 --> 00:15:28.090 align:middle line:84% But I think what actually happened 00:15:28.090 --> 00:15:29.590 align:middle line:90% is this is suntan lotion. 00:15:29.590 --> 00:15:33.770 align:middle line:90% 00:15:33.770 --> 00:15:36.500 align:middle line:84% So these, again, are interesting examples 00:15:36.500 --> 00:15:40.420 align:middle line:84% in the sense that they highlight the fact that these problems are 00:15:40.420 --> 00:15:42.700 align:middle line:84% ill-posed, because there are these two 00:15:42.700 --> 00:15:46.180 align:middle line:84% interpretations of the image, things that could actually 00:15:46.180 --> 00:15:49.520 align:middle line:84% be happening in the world, that are consistent with the image. 00:15:49.520 --> 00:15:52.980 align:middle line:84% So this person could have put suntan lotion on their legs. 00:15:52.980 --> 00:15:57.060 align:middle line:84% Or their legs could be like wrapped in cellophane. 00:15:57.060 --> 00:16:00.060 align:middle line:84% This could be a photo of a dog or of a person running 00:16:00.060 --> 00:16:02.600 align:middle line:84% into the woods with some kind of backpack on. 00:16:02.600 --> 00:16:05.270 align:middle line:90% 00:16:05.270 --> 00:16:08.640 align:middle line:84% This could be a face looking forward or a face to the side 00:16:08.640 --> 00:16:09.640 align:middle line:90% and so on and so forth. 00:16:09.640 --> 00:16:12.836 align:middle line:90% 00:16:12.836 --> 00:16:17.260 align:middle line:84% So there's also these unusual cases where individuals 00:16:17.260 --> 00:16:20.040 align:middle line:84% may be confident in their interpretation, 00:16:20.040 --> 00:16:22.360 align:middle line:84% but they'll disagree with other people. 00:16:22.360 --> 00:16:25.140 align:middle line:84% So this was an example that took the internet 00:16:25.140 --> 00:16:27.780 align:middle line:90% by storm several years ago now. 00:16:27.780 --> 00:16:28.820 align:middle line:90% It's the dress. 00:16:28.820 --> 00:16:31.420 align:middle line:84% And there were all these fights over whether the dress is 00:16:31.420 --> 00:16:34.780 align:middle line:84% actually white and gold or blue and black. 00:16:34.780 --> 00:16:36.900 align:middle line:84% How many people think it's white and gold? 00:16:36.900 --> 00:16:38.820 align:middle line:84% How many people think it's blue and black? 00:16:38.820 --> 00:16:39.900 align:middle line:90% OK, yeah. 00:16:39.900 --> 00:16:41.620 align:middle line:90% Disagreements. 00:16:41.620 --> 00:16:44.060 align:middle line:84% So this is actually really interesting because the fact 00:16:44.060 --> 00:16:49.020 align:middle line:84% that people disagree about this indicates or suggests that there 00:16:49.020 --> 00:16:53.180 align:middle line:84% are different assumptions that the white/gold people are making 00:16:53.180 --> 00:16:55.180 align:middle line:84% about the world compared with the black/blue. 00:16:55.180 --> 00:16:59.360 align:middle line:84% And these assumptions are made to resolve the ambiguity. 00:16:59.360 --> 00:17:04.260 align:middle line:84% And in fact, that's one of the central themes of perception, 00:17:04.260 --> 00:17:08.300 align:middle line:84% is that the way that we are able to solve these problems 00:17:08.300 --> 00:17:12.900 align:middle line:84% is by making assumptions about what the world is like. 00:17:12.900 --> 00:17:17.339 align:middle line:84% And that constrains the solution space hopefully enough 00:17:17.339 --> 00:17:20.057 align:middle line:84% that you can reach a unique solution. 00:17:20.057 --> 00:17:22.099 align:middle line:84% The third important point I want you to walk away 00:17:22.099 --> 00:17:25.660 align:middle line:84% from this lecture with is that perceptual systems 00:17:25.660 --> 00:17:28.920 align:middle line:90% must be invariant. 00:17:28.920 --> 00:17:31.190 align:middle line:84% And this is because the sensory input that 00:17:31.190 --> 00:17:34.270 align:middle line:84% is caused by a single type of thing in the world 00:17:34.270 --> 00:17:36.870 align:middle line:90% typically varies enormously. 00:17:36.870 --> 00:17:40.567 align:middle line:84% So these are all images of a car-- 00:17:40.567 --> 00:17:42.150 align:middle line:84% a car viewed from different distances, 00:17:42.150 --> 00:17:44.670 align:middle line:84% and different viewpoints, and on different backgrounds. 00:17:44.670 --> 00:17:47.190 align:middle line:90% They're all cars. 00:17:47.190 --> 00:17:52.790 align:middle line:84% But the actual image that is here and here and here 00:17:52.790 --> 00:17:53.810 align:middle line:90% is totally different. 00:17:53.810 --> 00:17:55.370 align:middle line:90% So that array of numbers-- 00:17:55.370 --> 00:17:57.852 align:middle line:84% If you think of this, that image as an array of numbers, 00:17:57.852 --> 00:17:59.310 align:middle line:84% it's going to be totally different. 00:17:59.310 --> 00:18:00.950 align:middle line:84% So the question is, how do you actually 00:18:00.950 --> 00:18:04.350 align:middle line:84% build something that can take that array of numbers 00:18:04.350 --> 00:18:08.630 align:middle line:84% and tell you that that is a car given that the numbers are 00:18:08.630 --> 00:18:11.030 align:middle line:84% changing so much from this instance to this instance 00:18:11.030 --> 00:18:12.130 align:middle line:90% to this instance? 00:18:12.130 --> 00:18:15.850 align:middle line:84% So that's one of the key problems of invariance. 00:18:15.850 --> 00:18:18.110 align:middle line:84% And so somehow or another, you're 00:18:18.110 --> 00:18:19.870 align:middle line:84% immediately able to look at these things 00:18:19.870 --> 00:18:22.495 align:middle line:84% and tell each of these things is a car and each of these things 00:18:22.495 --> 00:18:24.850 align:middle line:90% is not. 00:18:24.850 --> 00:18:27.750 align:middle line:84% The same problem exists in speech. 00:18:27.750 --> 00:18:29.760 align:middle line:84% So I'm going to play you an example of what 00:18:29.760 --> 00:18:33.133 align:middle line:84% we call dry speech, so without a whole lot of reverberation. 00:18:33.133 --> 00:18:33.800 align:middle line:90% [AUDIO PLAYBACK] 00:18:33.800 --> 00:18:35.480 align:middle line:90% - They ate the lemon pie. 00:18:35.480 --> 00:18:36.817 align:middle line:90% Father forgot the bread. 00:18:36.817 --> 00:18:37.400 align:middle line:90% [END PLAYBACK] 00:18:37.400 --> 00:18:39.600 align:middle line:84% JOSH MCDERMOTT: So this now, the second thing 00:18:39.600 --> 00:18:41.420 align:middle line:84% is going to be speech and reverberation. 00:18:41.420 --> 00:18:42.087 align:middle line:90% [AUDIO PLAYBACK] 00:18:42.087 --> 00:18:43.500 align:middle line:90% - They ate the lemon pie. 00:18:43.500 --> 00:18:45.020 align:middle line:90% Father forgot the bread. 00:18:45.020 --> 00:18:45.603 align:middle line:90% [END PLAYBACK] 00:18:45.603 --> 00:18:48.103 align:middle line:84% JOSH MCDERMOTT: That's like in a subway station or something 00:18:48.103 --> 00:18:49.420 align:middle line:90% or a really big bathroom. 00:18:49.420 --> 00:18:52.920 align:middle line:90% 00:18:52.920 --> 00:18:56.060 align:middle line:84% The sound waveforms there are now very, very different. 00:18:56.060 --> 00:18:57.648 align:middle line:84% If you actually look at spectrograms, 00:18:57.648 --> 00:18:59.440 align:middle line:84% you can see the effect of the reverberation 00:18:59.440 --> 00:19:02.160 align:middle line:84% is to smear the structure out in time. 00:19:02.160 --> 00:19:04.200 align:middle line:84% So it's like a massive distortion. 00:19:04.200 --> 00:19:05.742 align:middle line:84% It's totally different sound signals. 00:19:05.742 --> 00:19:07.742 align:middle line:84% But you can listen to them and tell that they're 00:19:07.742 --> 00:19:08.780 align:middle line:90% saying the same thing. 00:19:08.780 --> 00:19:11.585 align:middle line:90% 00:19:11.585 --> 00:19:14.200 align:middle line:84% Now, the ill-posed nature of the problems 00:19:14.200 --> 00:19:16.880 align:middle line:84% that your perceptual systems have to solve 00:19:16.880 --> 00:19:20.280 align:middle line:84% and the difficulty of acquiring the right type of invariance 00:19:20.280 --> 00:19:22.640 align:middle line:84% is why perceptual problems present 00:19:22.640 --> 00:19:24.600 align:middle line:90% a computational challenge. 00:19:24.600 --> 00:19:28.730 align:middle line:84% And so for really many decades, these 00:19:28.730 --> 00:19:32.050 align:middle line:84% were computational challenges that were insurmountable. 00:19:32.050 --> 00:19:35.270 align:middle line:84% So I taught this class when I was a PhD student, 00:19:35.270 --> 00:19:38.970 align:middle line:90% which was in the early 2000s. 00:19:38.970 --> 00:19:41.810 align:middle line:84% And one of the things that you would always comment on 00:19:41.810 --> 00:19:44.930 align:middle line:84% is how amazing human perceptual systems are compared 00:19:44.930 --> 00:19:47.810 align:middle line:84% to computer vision systems or speech recognition systems 00:19:47.810 --> 00:19:48.590 align:middle line:90% and so forth. 00:19:48.590 --> 00:19:51.930 align:middle line:84% And things have changed a little bit. 00:19:51.930 --> 00:19:53.905 align:middle line:84% So in particular, in the last five to 10 years, 00:19:53.905 --> 00:19:55.530 align:middle line:84% contemporary machine perception systems 00:19:55.530 --> 00:19:57.930 align:middle line:84% have become pretty good at certain types 00:19:57.930 --> 00:19:59.930 align:middle line:84% of perceptual tasks, specifically 00:19:59.930 --> 00:20:02.850 align:middle line:90% classification tasks. 00:20:02.850 --> 00:20:06.430 align:middle line:84% So object and face recognition now work pretty well. 00:20:06.430 --> 00:20:10.210 align:middle line:84% There's arguments over do they work as well as humans. 00:20:10.210 --> 00:20:12.510 align:middle line:84% Those are interesting questions we could talk about. 00:20:12.510 --> 00:20:13.830 align:middle line:84% Speech recognition works pretty well. 00:20:13.830 --> 00:20:15.970 align:middle line:84% Again, most of us talk to our phones all the time. 00:20:15.970 --> 00:20:20.050 align:middle line:84% You can dictate emails, and texts, and things like that. 00:20:20.050 --> 00:20:23.410 align:middle line:84% So that's remarkable and a game changer. 00:20:23.410 --> 00:20:25.050 align:middle line:84% And of course, that's been a big deal 00:20:25.050 --> 00:20:27.180 align:middle line:90% in the world of engineering. 00:20:27.180 --> 00:20:29.140 align:middle line:84% It's also been a very interesting development 00:20:29.140 --> 00:20:32.660 align:middle line:84% for the study of perception, because the resulting systems 00:20:32.660 --> 00:20:36.548 align:middle line:84% now give-- you can treat those as models of perceptual systems. 00:20:36.548 --> 00:20:38.340 align:middle line:84% And they exhibit many interesting parallels 00:20:38.340 --> 00:20:40.080 align:middle line:90% with human perception. 00:20:40.080 --> 00:20:42.620 align:middle line:84% So this has given rise to a new generation of models 00:20:42.620 --> 00:20:45.605 align:middle line:90% of perceptual systems. 00:20:45.605 --> 00:20:47.980 align:middle line:84% And so that's going to be a theme that we will talk about 00:20:47.980 --> 00:20:50.500 align:middle line:84% periodically throughout the class-- namely, 00:20:50.500 --> 00:20:52.860 align:middle line:84% can we obtain better models of the brain, 00:20:52.860 --> 00:20:55.060 align:middle line:84% in particular of sensory systems in the brain, 00:20:55.060 --> 00:20:58.100 align:middle line:84% using contemporary technology from AI? 00:20:58.100 --> 00:21:00.900 align:middle line:84% So one of the main engines of all of this 00:21:00.900 --> 00:21:02.860 align:middle line:90% is artificial neural networks. 00:21:02.860 --> 00:21:05.340 align:middle line:84% So these are systems that consist of the repeated 00:21:05.340 --> 00:21:07.780 align:middle line:84% application of pretty simple operations, all of which 00:21:07.780 --> 00:21:09.820 align:middle line:84% were loosely inspired by things that people 00:21:09.820 --> 00:21:13.250 align:middle line:84% saw in the brain-- filtering, pooling, normalization. 00:21:13.250 --> 00:21:15.500 align:middle line:84% We'll talk more about what each of those things means. 00:21:15.500 --> 00:21:17.167 align:middle line:84% And we now have really effective methods 00:21:17.167 --> 00:21:19.700 align:middle line:84% to optimize the parameters of systems like this 00:21:19.700 --> 00:21:23.740 align:middle line:84% to cause them to correctly classify input signals. 00:21:23.740 --> 00:21:27.910 align:middle line:84% So just to give you a little example of something that 00:21:27.910 --> 00:21:31.830 align:middle line:84% was really unimaginable back when I was a student, 00:21:31.830 --> 00:21:34.910 align:middle line:84% I'm going to show you a comparison of speech recognition 00:21:34.910 --> 00:21:37.830 align:middle line:84% by humans and by an artificial neural network. 00:21:37.830 --> 00:21:39.990 align:middle line:84% And so this is an experiment where 00:21:39.990 --> 00:21:43.390 align:middle line:84% humans are played, short excerpts of speech superimposed 00:21:43.390 --> 00:21:45.037 align:middle line:90% on background noise. 00:21:45.037 --> 00:21:46.870 align:middle line:84% And they just have to say what the words are 00:21:46.870 --> 00:21:48.030 align:middle line:90% that the person is saying. 00:21:48.030 --> 00:21:50.330 align:middle line:84% So the y-axis here plots the proportion of words. 00:21:50.330 --> 00:21:54.170 align:middle line:84% And the x-axis is the signal to noise ratio. 00:21:54.170 --> 00:21:56.910 align:middle line:84% So as you move from left to right, 00:21:56.910 --> 00:21:58.630 align:middle line:84% the speech becomes louder relative 00:21:58.630 --> 00:21:59.850 align:middle line:90% to the background noise. 00:21:59.850 --> 00:22:02.630 align:middle line:84% And so you expect that people will get better. 00:22:02.630 --> 00:22:03.730 align:middle line:90% And indeed, they do. 00:22:03.730 --> 00:22:05.910 align:middle line:84% But you can see that, in this experiment, 00:22:05.910 --> 00:22:08.230 align:middle line:84% there were four different types of background noise. 00:22:08.230 --> 00:22:10.790 align:middle line:90% So the green one is music. 00:22:10.790 --> 00:22:13.290 align:middle line:84% For instance, the purple one is what's called speech babble. 00:22:13.290 --> 00:22:14.290 align:middle line:90% That's like crowd noise. 00:22:14.290 --> 00:22:16.750 align:middle line:84% And you can see that some types of noise for humans 00:22:16.750 --> 00:22:18.550 align:middle line:84% are much easier to recognize speech 00:22:18.550 --> 00:22:20.230 align:middle line:90% in than other types of noise. 00:22:20.230 --> 00:22:21.550 align:middle line:90% So that's just what people do. 00:22:21.550 --> 00:22:24.680 align:middle line:84% And so next to it, this is the results 00:22:24.680 --> 00:22:26.760 align:middle line:84% of running a neural network model 00:22:26.760 --> 00:22:28.800 align:middle line:90% on the exact same experiment. 00:22:28.800 --> 00:22:31.360 align:middle line:84% And there's two main things to take away from this. 00:22:31.360 --> 00:22:34.140 align:middle line:84% One is that the model is doing about as well as humans. 00:22:34.140 --> 00:22:37.340 align:middle line:84% That's the thing that was inconceivable 15 years ago. 00:22:37.340 --> 00:22:39.380 align:middle line:84% And nowadays, it's pretty commonplace. 00:22:39.380 --> 00:22:41.120 align:middle line:84% But the other thing that's interesting 00:22:41.120 --> 00:22:43.840 align:middle line:84% is that the conditions that are easy for humans 00:22:43.840 --> 00:22:45.920 align:middle line:84% are easy for the model and vice versa. 00:22:45.920 --> 00:22:49.840 align:middle line:84% So the phenotype of speech recognition 00:22:49.840 --> 00:22:54.000 align:middle line:84% seems to be shared across humans in this model. 00:22:54.000 --> 00:22:56.190 align:middle line:84% And so the question is, well, can we 00:22:56.190 --> 00:22:57.940 align:middle line:84% use these things to model sensory systems? 00:22:57.940 --> 00:23:00.440 align:middle line:84% And that's just something that we will talk about 00:23:00.440 --> 00:23:01.933 align:middle line:90% throughout the class. 00:23:01.933 --> 00:23:03.100 align:middle line:90% AUDIENCE: I have a question. 00:23:03.100 --> 00:23:04.320 align:middle line:90% JOSH MCDERMOTT: Yes? 00:23:04.320 --> 00:23:07.640 align:middle line:84% AUDIENCE: You mentioned that 50% of the brain is [INAUDIBLE]. 00:23:07.640 --> 00:23:09.240 align:middle line:90% Is that by mass or volume? 00:23:09.240 --> 00:23:13.280 align:middle line:84% And is that the entire brain or just the neocortex? 00:23:13.280 --> 00:23:15.920 align:middle line:84% JOSH MCDERMOTT: I was actually talking about the cortex. 00:23:15.920 --> 00:23:18.640 align:middle line:84% That's a very approximate number. 00:23:18.640 --> 00:23:22.200 align:middle line:84% And it would be both mass and volume. 00:23:22.200 --> 00:23:28.540 align:middle line:84% So mostly, it's pretty much the back half of the brain, 00:23:28.540 --> 00:23:29.540 align:middle line:90% more or less-- 00:23:29.540 --> 00:23:31.040 align:middle line:84% I mean, this is very, very crude-- 00:23:31.040 --> 00:23:33.420 align:middle line:84% more or less is involved in vision. 00:23:33.420 --> 00:23:36.153 align:middle line:84% And again, it's a little complicated 00:23:36.153 --> 00:23:37.820 align:middle line:84% to give you a very precise number there, 00:23:37.820 --> 00:23:39.362 align:middle line:84% because the question is, what does it 00:23:39.362 --> 00:23:40.760 align:middle line:90% mean to be involved in vision? 00:23:40.760 --> 00:23:44.920 align:middle line:84% And there's lots of parts of the brain that 00:23:44.920 --> 00:23:48.980 align:middle line:84% respond when you're looking at things, 00:23:48.980 --> 00:23:51.640 align:middle line:84% but have other functions as well. 00:23:51.640 --> 00:23:55.080 align:middle line:84% But yeah, roughly half is what I would say. 00:23:55.080 --> 00:23:56.880 align:middle line:90% Thanks for the question. 00:23:56.880 --> 00:23:57.380 align:middle line:90% All right. 00:23:57.380 --> 00:23:59.160 align:middle line:84% So the fourth important point that I 00:23:59.160 --> 00:24:02.840 align:middle line:84% want you to walk away from is that perception 00:24:02.840 --> 00:24:06.520 align:middle line:90% is unconscious inference. 00:24:06.520 --> 00:24:07.020 align:middle line:90% All right. 00:24:07.020 --> 00:24:09.400 align:middle line:84% So we've talked about how one of the key things 00:24:09.400 --> 00:24:12.640 align:middle line:84% to know about perception is that the problems that we're solving 00:24:12.640 --> 00:24:13.960 align:middle line:90% are ill-posed. 00:24:13.960 --> 00:24:16.720 align:middle line:84% So that usually means there's not like a unique solution. 00:24:16.720 --> 00:24:19.490 align:middle line:84% So the information in the sensory input 00:24:19.490 --> 00:24:21.910 align:middle line:84% does not uniquely specify the structure of the world. 00:24:21.910 --> 00:24:23.570 align:middle line:84% And so the consequence of that is 00:24:23.570 --> 00:24:25.250 align:middle line:84% that the brain has to make its best 00:24:25.250 --> 00:24:29.050 align:middle line:90% guess as to what is out there. 00:24:29.050 --> 00:24:30.270 align:middle line:90% This is inference. 00:24:30.270 --> 00:24:32.290 align:middle line:84% So when you see or when you hear, 00:24:32.290 --> 00:24:37.770 align:middle line:84% we think your brain is choosing the most probable interpretation 00:24:37.770 --> 00:24:41.470 align:middle line:84% of the sensory input that you are getting. 00:24:41.470 --> 00:24:42.687 align:middle line:90% Now, so this is inference. 00:24:42.687 --> 00:24:44.270 align:middle line:84% But you're not aware of the inference. 00:24:44.270 --> 00:24:46.210 align:middle line:90% So it's very different from-- 00:24:46.210 --> 00:24:48.210 align:middle line:84% some kinds of inferences you make consciously. 00:24:48.210 --> 00:24:50.410 align:middle line:84% You might reason about a problem to work out 00:24:50.410 --> 00:24:52.990 align:middle line:84% what might likely have happened to explain something. 00:24:52.990 --> 00:24:56.130 align:middle line:84% So the inferences that your perceptual systems make, 00:24:56.130 --> 00:24:58.930 align:middle line:84% they just happen automatically, really 00:24:58.930 --> 00:25:00.730 align:middle line:90% without you being aware of them. 00:25:00.730 --> 00:25:03.950 align:middle line:84% And so that's why we call it unconscious inference. 00:25:03.950 --> 00:25:05.870 align:middle line:84% So that is a term that is due to Helmholtz. 00:25:05.870 --> 00:25:10.610 align:middle line:84% So Helmholtz was a giant of 19th century science, 00:25:10.610 --> 00:25:14.050 align:middle line:84% did lots of stuff, and had made many important contributions 00:25:14.050 --> 00:25:17.220 align:middle line:84% to perception, including this idea that perception is 00:25:17.220 --> 00:25:18.440 align:middle line:90% unconscious inference. 00:25:18.440 --> 00:25:21.740 align:middle line:90% 00:25:21.740 --> 00:25:24.540 align:middle line:84% So let me give you an example or two of that. 00:25:24.540 --> 00:25:28.388 align:middle line:84% So I'm going to play you a bunch of moving dots. 00:25:28.388 --> 00:25:30.180 align:middle line:84% And so this is a little bit like that thing 00:25:30.180 --> 00:25:32.305 align:middle line:84% that I showed you a little bit earlier in the sense 00:25:32.305 --> 00:25:35.060 align:middle line:84% that the moving dots-- because really all 00:25:35.060 --> 00:25:37.180 align:middle line:84% that you can see there are these moving dots, 00:25:37.180 --> 00:25:40.180 align:middle line:84% there's lots of potential explanations of the motion. 00:25:40.180 --> 00:25:40.700 align:middle line:90% OK? 00:25:40.700 --> 00:25:41.200 align:middle line:90% Shh. 00:25:41.200 --> 00:25:45.980 align:middle line:90% 00:25:45.980 --> 00:25:48.260 align:middle line:90% So you look at this. 00:25:48.260 --> 00:25:51.070 align:middle line:90% And you can see the dots moving. 00:25:51.070 --> 00:25:55.540 align:middle line:84% It may not really be completely obvious what caused those dots. 00:25:55.540 --> 00:25:57.580 align:middle line:84% But now, what I'm going to do is show you 00:25:57.580 --> 00:26:00.580 align:middle line:84% the same thing flipped upside down. 00:26:00.580 --> 00:26:06.320 align:middle line:84% And when you see it in this orientation, 00:26:06.320 --> 00:26:10.340 align:middle line:84% it becomes quite obvious that this is a person walking. 00:26:10.340 --> 00:26:13.540 align:middle line:84% And so in fact, this stimulus is an example 00:26:13.540 --> 00:26:16.230 align:middle line:84% of a very famous type of stimulus known as a point light 00:26:16.230 --> 00:26:16.890 align:middle line:90% walker. 00:26:16.890 --> 00:26:20.790 align:middle line:84% And so originally, the way that they would generate something 00:26:20.790 --> 00:26:22.590 align:middle line:84% like this is by putting these little lights 00:26:22.590 --> 00:26:25.048 align:middle line:84% on the joints of a person, and putting them in a dark room, 00:26:25.048 --> 00:26:26.470 align:middle line:90% and then filming them. 00:26:26.470 --> 00:26:30.013 align:middle line:84% So now, of course, we can do this with computers. 00:26:30.013 --> 00:26:31.430 align:middle line:84% And so the really remarkable thing 00:26:31.430 --> 00:26:34.050 align:middle line:84% is that, when the orientation is correct, 00:26:34.050 --> 00:26:36.110 align:middle line:84% which means what you're used to, you 00:26:36.110 --> 00:26:39.910 align:middle line:84% can take that pattern of motion, and then perceive a form. 00:26:39.910 --> 00:26:45.790 align:middle line:84% But when it's not what you're used to, 00:26:45.790 --> 00:26:47.290 align:middle line:90% that's a lot harder to do. 00:26:47.290 --> 00:26:51.640 align:middle line:90% 00:26:51.640 --> 00:26:53.010 align:middle line:90% So what does this mean? 00:26:53.010 --> 00:26:58.310 align:middle line:84% Well, OK, so it's an example of an ill-posed problem, 00:26:58.310 --> 00:27:00.310 align:middle line:84% because there are lots of potential explanations 00:27:00.310 --> 00:27:01.850 align:middle line:90% for this motion. 00:27:01.850 --> 00:27:03.670 align:middle line:84% So one explanation is that there's 00:27:03.670 --> 00:27:06.200 align:middle line:90% a person walking upside down. 00:27:06.200 --> 00:27:08.450 align:middle line:84% And we verified that because you saw it right side up. 00:27:08.450 --> 00:27:10.533 align:middle line:84% And you can tell that could be caused by a person. 00:27:10.533 --> 00:27:12.990 align:middle line:84% But another explanation is that these dots are just 00:27:12.990 --> 00:27:13.590 align:middle line:90% moving around. 00:27:13.590 --> 00:27:15.480 align:middle line:84% Maybe they're fireflies or something. 00:27:15.480 --> 00:27:21.030 align:middle line:84% They're not necessarily related to a single thing. 00:27:21.030 --> 00:27:23.100 align:middle line:84% And there are these multiple interpretations. 00:27:23.100 --> 00:27:26.560 align:middle line:84% And then your brain is choosing one of those interpretations 00:27:26.560 --> 00:27:29.160 align:middle line:84% based on what it thinks is likely. 00:27:29.160 --> 00:27:31.920 align:middle line:84% And so when there's an interpretation 00:27:31.920 --> 00:27:33.840 align:middle line:84% that there's a person that's right side up, 00:27:33.840 --> 00:27:35.880 align:middle line:84% that seems to be what you see, presumably 00:27:35.880 --> 00:27:37.942 align:middle line:84% because usually when people walk, 00:27:37.942 --> 00:27:39.400 align:middle line:84% they're not walking on the ceiling. 00:27:39.400 --> 00:27:40.320 align:middle line:90% They're walking on the ground. 00:27:40.320 --> 00:27:41.620 align:middle line:90% And you're mostly right side up. 00:27:41.620 --> 00:27:43.203 align:middle line:84% And so most of the time, you're seeing 00:27:43.203 --> 00:27:44.580 align:middle line:90% people walk right side up. 00:27:44.580 --> 00:27:48.120 align:middle line:90% 00:27:48.120 --> 00:27:50.920 align:middle line:84% So that, in some sense, is an unconscious inference. 00:27:50.920 --> 00:27:54.540 align:middle line:90% Any questions about that? 00:27:54.540 --> 00:27:55.040 align:middle line:90% Yeah? 00:27:55.040 --> 00:27:57.520 align:middle line:84% AUDIENCE: Is there the nature/nurture thing going on 00:27:57.520 --> 00:28:01.640 align:middle line:84% in the fact that unconscious versus conscious-- is 00:28:01.640 --> 00:28:05.560 align:middle line:84% any of this things that we expect 00:28:05.560 --> 00:28:10.440 align:middle line:84% are hardwired versus learned by experience of like watching 00:28:10.440 --> 00:28:13.090 align:middle line:90% thousands of people walk? 00:28:13.090 --> 00:28:15.690 align:middle line:84% JOSH MCDERMOTT: Yeah, that's a great question. 00:28:15.690 --> 00:28:19.970 align:middle line:84% So you will be able to ask that question of many of the things 00:28:19.970 --> 00:28:23.130 align:middle line:84% that we will talk about in this class. 00:28:23.130 --> 00:28:25.730 align:middle line:84% And in most cases, we don't really 00:28:25.730 --> 00:28:29.420 align:middle line:90% know the answer very clearly. 00:28:29.420 --> 00:28:34.370 align:middle line:84% So there's bits of evidence that young infants 00:28:34.370 --> 00:28:36.890 align:middle line:84% can experience some of the kinds of phenomena 00:28:36.890 --> 00:28:38.370 align:middle line:90% that we'll talk about. 00:28:38.370 --> 00:28:40.610 align:middle line:84% It's not that easy to actually determine 00:28:40.610 --> 00:28:42.503 align:middle line:84% whether a baby can tell if this is 00:28:42.503 --> 00:28:44.170 align:middle line:84% somebody walking because they can't talk 00:28:44.170 --> 00:28:45.270 align:middle line:90% and so on and so forth. 00:28:45.270 --> 00:28:49.190 align:middle line:84% So you have to do really clever experiments to assess that. 00:28:49.190 --> 00:28:55.050 align:middle line:90% 00:28:55.050 --> 00:28:56.810 align:middle line:84% These assumptions that your brain 00:28:56.810 --> 00:28:59.930 align:middle line:84% brings to bear to constrain these ill-posed problems, 00:28:59.930 --> 00:29:04.073 align:middle line:84% they could be something that you acquire through evolution, 00:29:04.073 --> 00:29:04.990 align:middle line:90% that you're born with. 00:29:04.990 --> 00:29:06.990 align:middle line:84% It could be something that you acquire over development. 00:29:06.990 --> 00:29:08.370 align:middle line:90% It's probably some of both. 00:29:08.370 --> 00:29:10.290 align:middle line:84% In most cases, we don't really know. 00:29:10.290 --> 00:29:12.700 align:middle line:90% Yeah. 00:29:12.700 --> 00:29:14.540 align:middle line:84% There are a handful of cases where 00:29:14.540 --> 00:29:20.960 align:middle line:84% there is very clear evidence that you can learn assumptions. 00:29:20.960 --> 00:29:23.620 align:middle line:90% 00:29:23.620 --> 00:29:25.420 align:middle line:84% So part of why this is hard to study 00:29:25.420 --> 00:29:30.075 align:middle line:84% is it's not that easy to have somebody inhabit a world where 00:29:30.075 --> 00:29:31.200 align:middle line:90% people walk on the ceiling. 00:29:31.200 --> 00:29:34.100 align:middle line:84% So just in practice, the interventions 00:29:34.100 --> 00:29:36.705 align:middle line:84% that you would need to do the experiments are impractical. 00:29:36.705 --> 00:29:38.580 align:middle line:84% But there are cases where those interventions 00:29:38.580 --> 00:29:39.680 align:middle line:90% are less impractical. 00:29:39.680 --> 00:29:42.020 align:middle line:84% And in some of those cases, people have done them. 00:29:42.020 --> 00:29:43.700 align:middle line:84% And we'll talk about some of those. 00:29:43.700 --> 00:29:47.060 align:middle line:90% 00:29:47.060 --> 00:29:49.380 align:middle line:84% So here's another interesting example 00:29:49.380 --> 00:29:51.620 align:middle line:90% of an unconscious inference. 00:29:51.620 --> 00:29:55.380 align:middle line:84% So I'm going to play you a sound that consists of something 00:29:55.380 --> 00:29:56.760 align:middle line:90% called a sound texture. 00:29:56.760 --> 00:30:00.340 align:middle line:84% This is the sound of a lot of people clapping, applause. 00:30:00.340 --> 00:30:03.370 align:middle line:84% And then that will be interrupted by noise. 00:30:03.370 --> 00:30:05.620 align:middle line:84% And then there'll be a little bit more of the texture. 00:30:05.620 --> 00:30:08.805 align:middle line:84% And the amazing thing about this particular sound 00:30:08.805 --> 00:30:10.180 align:middle line:84% is you're going to listen to this 00:30:10.180 --> 00:30:12.580 align:middle line:84% and you will have the sense that texture 00:30:12.580 --> 00:30:15.540 align:middle line:90% continues during the noise. 00:30:15.540 --> 00:30:17.800 align:middle line:84% But in fact, it's completely not present. 00:30:17.800 --> 00:30:21.220 align:middle line:84% So listen to this and see what you think. 00:30:21.220 --> 00:30:22.216 align:middle line:90% [AUDIO PLAYBACK] 00:30:22.216 --> 00:30:24.208 align:middle line:90% [APPLAUSE] 00:30:24.208 --> 00:30:25.204 align:middle line:90% [STATIC NOISE] 00:30:25.204 --> 00:30:26.117 align:middle line:90% [APPLAUSE] 00:30:26.117 --> 00:30:26.700 align:middle line:90% [END PLAYBACK] 00:30:26.700 --> 00:30:28.408 align:middle line:84% JOSH MCDERMOTT: Did you hear it continue? 00:30:28.408 --> 00:30:30.620 align:middle line:90% Yeah? 00:30:30.620 --> 00:30:31.543 align:middle line:90% Again. 00:30:31.543 --> 00:30:32.210 align:middle line:90% [AUDIO PLAYBACK] 00:30:32.210 --> 00:30:33.150 align:middle line:90% [APPLAUSE] 00:30:33.150 --> 00:30:34.090 align:middle line:90% [STATIC NOISE] 00:30:34.090 --> 00:30:35.030 align:middle line:90% [APPLAUSE] 00:30:35.030 --> 00:30:37.455 align:middle line:90% [END PLAYBACK] 00:30:37.455 --> 00:30:40.080 align:middle line:84% JOSH MCDERMOTT: So it's so crazy that you might not believe me. 00:30:40.080 --> 00:30:41.455 align:middle line:84% You might say, well, hang on, how 00:30:41.455 --> 00:30:44.340 align:middle line:84% do I know that that thing is not actually there? 00:30:44.340 --> 00:30:47.200 align:middle line:84% So this is a variant that will hopefully convince you of this. 00:30:47.200 --> 00:30:49.520 align:middle line:84% So in this particular case, it's exactly the same. 00:30:49.520 --> 00:30:52.180 align:middle line:84% But there's a little gap here between this texture 00:30:52.180 --> 00:30:53.882 align:middle line:90% and the noise. 00:30:53.882 --> 00:30:55.340 align:middle line:84% And so now, I think your experience 00:30:55.340 --> 00:30:57.440 align:middle line:90% will be very different. 00:30:57.440 --> 00:30:58.440 align:middle line:90% [AUDIO PLAYBACK] 00:30:58.440 --> 00:30:59.440 align:middle line:90% [APPLAUSE] 00:30:59.440 --> 00:31:00.940 align:middle line:90% [STATIC NOISE] 00:31:00.940 --> 00:31:01.940 align:middle line:90% [APPLAUSE] 00:31:01.940 --> 00:31:02.940 align:middle line:90% [END PLAYBACK] 00:31:02.940 --> 00:31:04.700 align:middle line:84% JOSH MCDERMOTT: So it wasn't there then. 00:31:04.700 --> 00:31:06.260 align:middle line:90% All right, so what's going on? 00:31:06.260 --> 00:31:11.510 align:middle line:84% Well, it seems to be the case that this, what we call, 00:31:11.510 --> 00:31:13.990 align:middle line:84% illusory continuity-- the fact that you hear that sound 00:31:13.990 --> 00:31:16.950 align:middle line:84% continue during the noise-- one second-- 00:31:16.950 --> 00:31:21.110 align:middle line:84% is an unconscious inference about what most likely is 00:31:21.110 --> 00:31:23.590 align:middle line:90% happening during the noise. 00:31:23.590 --> 00:31:24.610 align:middle line:90% You had a question. 00:31:24.610 --> 00:31:25.110 align:middle line:90% Yeah? 00:31:25.110 --> 00:31:26.850 align:middle line:84% AUDIENCE: So you can hear the texture in the noise. 00:31:26.850 --> 00:31:29.557 align:middle line:84% But why can't you hear the noise in the texture afterwards? 00:31:29.557 --> 00:31:31.390 align:middle line:84% JOSH MCDERMOTT: That's a very good question. 00:31:31.390 --> 00:31:35.230 align:middle line:84% The presumptive reason is that the noise is actually 00:31:35.230 --> 00:31:36.290 align:middle line:90% more intense. 00:31:36.290 --> 00:31:39.710 align:middle line:84% So the sound level or the intensity or the loudness 00:31:39.710 --> 00:31:41.190 align:middle line:90% is higher. 00:31:41.190 --> 00:31:43.270 align:middle line:84% And so what we think actually happens 00:31:43.270 --> 00:31:44.950 align:middle line:84% in these examples of illusory continuity 00:31:44.950 --> 00:31:46.630 align:middle line:84% is that the reason that you actually 00:31:46.630 --> 00:31:51.350 align:middle line:84% hear the texture during the noise in this example 00:31:51.350 --> 00:31:55.270 align:middle line:84% is that the noise is sufficiently 00:31:55.270 --> 00:32:01.590 align:middle line:84% intense that a phenomena called masking would occur 00:32:01.590 --> 00:32:03.630 align:middle line:90% if the texture were there. 00:32:03.630 --> 00:32:05.390 align:middle line:84% And specifically, what that means 00:32:05.390 --> 00:32:08.840 align:middle line:84% is that when two sounds play at the same time, if one of them 00:32:08.840 --> 00:32:10.600 align:middle line:84% is loud enough relative to the other, 00:32:10.600 --> 00:32:14.520 align:middle line:84% you will not be able to detect the presence of the other one. 00:32:14.520 --> 00:32:15.840 align:middle line:90% That's known as masking. 00:32:15.840 --> 00:32:17.740 align:middle line:90% But that's just a phenomenon. 00:32:17.740 --> 00:32:22.680 align:middle line:84% And so the consequence of that is that, 00:32:22.680 --> 00:32:26.280 align:middle line:84% when the noise is very high in intensity, 00:32:26.280 --> 00:32:29.200 align:middle line:84% you actually wouldn't-- the stimulus would not be any 00:32:29.200 --> 00:32:31.785 align:middle line:84% different whether or not the texture actually was there. 00:32:31.785 --> 00:32:33.660 align:middle line:84% And when I say it would not be any different, 00:32:33.660 --> 00:32:36.077 align:middle line:84% it means it would not be any different from the standpoint 00:32:36.077 --> 00:32:36.800 align:middle line:90% of the ear. 00:32:36.800 --> 00:32:38.560 align:middle line:84% The stimulation at the ear would be 00:32:38.560 --> 00:32:41.200 align:middle line:84% the same due to this phenomenon of masking. 00:32:41.200 --> 00:32:46.400 align:middle line:84% And the consequence of that is that this is the situation 00:32:46.400 --> 00:32:47.220 align:middle line:90% is ill-posed. 00:32:47.220 --> 00:32:52.040 align:middle line:84% So the stimulus that you would get if the texture was there 00:32:52.040 --> 00:32:53.640 align:middle line:84% is basically the same as the stimulus 00:32:53.640 --> 00:32:55.440 align:middle line:84% that you would get if it wasn't there. 00:32:55.440 --> 00:32:57.560 align:middle line:84% And so now, the brain just has to make its best 00:32:57.560 --> 00:32:59.020 align:middle line:90% guess as to what is happening. 00:32:59.020 --> 00:33:01.840 align:middle line:84% And presumably with these texture sounds, 00:33:01.840 --> 00:33:05.730 align:middle line:84% they're of that they just tend to go on and on and on and on. 00:33:05.730 --> 00:33:07.770 align:middle line:84% And so your brain infers that it's 00:33:07.770 --> 00:33:10.410 align:middle line:84% a good bet that that sound actually continued. 00:33:10.410 --> 00:33:12.530 align:middle line:84% You had a question there, and then there. 00:33:12.530 --> 00:33:17.650 align:middle line:84% AUDIENCE: Yeah, I was wondering, does the type of noise-- 00:33:17.650 --> 00:33:19.610 align:middle line:84% let's say it was a trumpet instead. 00:33:19.610 --> 00:33:22.290 align:middle line:84% Would you still have the same phenomenon, 00:33:22.290 --> 00:33:25.010 align:middle line:90% like if the texture was really-- 00:33:25.010 --> 00:33:29.370 align:middle line:84% let's say clapping, but it's a different type or intensity 00:33:29.370 --> 00:33:31.435 align:middle line:90% of noise? 00:33:31.435 --> 00:33:32.810 align:middle line:84% JOSH MCDERMOTT: So are you asking 00:33:32.810 --> 00:33:35.930 align:middle line:84% whether the effect would be the same if this was a trumpet? 00:33:35.930 --> 00:33:37.070 align:middle line:90% AUDIENCE: No, the noise. 00:33:37.070 --> 00:33:41.308 align:middle line:84% Like, if the noise was not the static that it was. 00:33:41.308 --> 00:33:42.350 align:middle line:90% JOSH MCDERMOTT: Yeah, OK. 00:33:42.350 --> 00:33:42.910 align:middle line:90% Got it. 00:33:42.910 --> 00:33:45.050 align:middle line:84% So the thing that seems to be really 00:33:45.050 --> 00:33:47.770 align:middle line:84% important for this illusion to happen 00:33:47.770 --> 00:33:51.890 align:middle line:84% is that the frequencies in the noise 00:33:51.890 --> 00:33:54.390 align:middle line:84% have to overlap with the frequencies in the textures. 00:33:54.390 --> 00:33:56.650 align:middle line:84% And they have to be kind of loud enough that they 00:33:56.650 --> 00:33:58.530 align:middle line:90% could mask the texture. 00:33:58.530 --> 00:34:02.290 align:middle line:84% And so in fact, what happens is that if you actually 00:34:02.290 --> 00:34:04.860 align:middle line:84% make the noise quiet, if you lower 00:34:04.860 --> 00:34:07.040 align:middle line:84% the amplitude of the noise, at some point, 00:34:07.040 --> 00:34:09.100 align:middle line:90% this stops happening. 00:34:09.100 --> 00:34:11.940 align:middle line:84% And you stop hearing the texture there. 00:34:11.940 --> 00:34:15.583 align:middle line:84% You had a question, [? Hannah? ?] In the back? 00:34:15.583 --> 00:34:17.500 align:middle line:84% AUDIENCE: If you were to just play the texture 00:34:17.500 --> 00:34:19.167 align:middle line:84% and then play the noise for a long time, 00:34:19.167 --> 00:34:21.380 align:middle line:84% would you eventually stop hearing the texture? 00:34:21.380 --> 00:34:21.699 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:34:21.699 --> 00:34:24.318 align:middle line:84% In fact, so Richard McWalter, who was a postdoc in my lab, 00:34:24.318 --> 00:34:25.360 align:middle line:90% he did these experiments. 00:34:25.360 --> 00:34:28.900 align:middle line:84% And so the amount of time that the texture lasts 00:34:28.900 --> 00:34:30.000 align:middle line:90% depends on the texture. 00:34:30.000 --> 00:34:33.580 align:middle line:84% But in some cases, it can go on for three or four seconds. 00:34:33.580 --> 00:34:36.659 align:middle line:90% But it fades out eventually. 00:34:36.659 --> 00:34:38.280 align:middle line:90% Yeah, yeah. 00:34:38.280 --> 00:34:40.944 align:middle line:90% 00:34:40.944 --> 00:34:42.080 align:middle line:90% Oh, yeah? 00:34:42.080 --> 00:34:44.080 align:middle line:84% AUDIENCE: This question is asking about priming. 00:34:44.080 --> 00:34:46.980 align:middle line:84% And it reads, would I hear the same clapping in the noise 00:34:46.980 --> 00:34:50.400 align:middle line:84% if Josh didn't tell me that you will hear in the classroom? 00:34:50.400 --> 00:34:54.040 align:middle line:84% Thinking back to before page 25 of the slides, 00:34:54.040 --> 00:34:56.120 align:middle line:84% when I was listening to the 7-people overlay, 00:34:56.120 --> 00:34:59.060 align:middle line:84% I didn't hear the initial sentence before realizing that 00:34:59.060 --> 00:35:02.630 align:middle line:84% the sentence was uttered when Josh said you can still hear 00:35:02.630 --> 00:35:04.030 align:middle line:90% the sentence. 00:35:04.030 --> 00:35:05.530 align:middle line:84% JOSH MCDERMOTT: Yeah, good question. 00:35:05.530 --> 00:35:07.990 align:middle line:84% So for this phenomenon, the priming-- 00:35:07.990 --> 00:35:10.410 align:middle line:84% I believe the priming has a very small effect. 00:35:10.410 --> 00:35:12.830 align:middle line:84% And the reason I think that we've done lots of experiments 00:35:12.830 --> 00:35:15.163 align:middle line:84% where we didn't tell people that they were going to hear 00:35:15.163 --> 00:35:16.870 align:middle line:90% this thing in the noise. 00:35:16.870 --> 00:35:20.190 align:middle line:84% In fact, we don't even tell them that the texture is not there. 00:35:20.190 --> 00:35:21.737 align:middle line:90% Sometimes, the texture is there. 00:35:21.737 --> 00:35:22.570 align:middle line:90% Sometimes, it's not. 00:35:22.570 --> 00:35:23.930 align:middle line:84% And we ask them whether it's there. 00:35:23.930 --> 00:35:26.305 align:middle line:84% And when it's not there, they say it's there essentially. 00:35:26.305 --> 00:35:28.430 align:middle line:84% So the illusion is real in that sense. 00:35:28.430 --> 00:35:32.510 align:middle line:84% And it's certainly true that, with that cocktail party effect, 00:35:32.510 --> 00:35:36.735 align:middle line:84% part of what helps you hear what we call the target voice-- so 00:35:36.735 --> 00:35:38.110 align:middle line:84% the very first one that I played. 00:35:38.110 --> 00:35:39.830 align:middle line:84% Part of what helps you hear that is that you heard it 00:35:39.830 --> 00:35:41.390 align:middle line:90% in isolation the first time. 00:35:41.390 --> 00:35:42.950 align:middle line:90% So that is priming. 00:35:42.950 --> 00:35:44.390 align:middle line:84% So in that situation, the priming 00:35:44.390 --> 00:35:47.555 align:middle line:84% actually does have a fairly substantial effect. 00:35:47.555 --> 00:35:49.930 align:middle line:84% So that can definitely be important in a lot of settings. 00:35:49.930 --> 00:35:54.530 align:middle line:90% 00:35:54.530 --> 00:35:56.430 align:middle line:84% So the fifth important point I want 00:35:56.430 --> 00:36:01.310 align:middle line:84% you to leave the lecture with is that illusions 00:36:01.310 --> 00:36:03.550 align:middle line:84% illustrate perceptual mechanisms at work 00:36:03.550 --> 00:36:04.810 align:middle line:90% and can help us study them. 00:36:04.810 --> 00:36:06.790 align:middle line:90% So illusions are fun. 00:36:06.790 --> 00:36:10.630 align:middle line:84% We love looking at illusions because they 00:36:10.630 --> 00:36:13.730 align:middle line:84% make us realize that the world is not always how it seems. 00:36:13.730 --> 00:36:17.150 align:middle line:90% And they're lots of fun. 00:36:17.150 --> 00:36:19.830 align:middle line:84% But they're are also scientific tools. 00:36:19.830 --> 00:36:23.190 align:middle line:84% And so a lot of what happens in perception research 00:36:23.190 --> 00:36:25.210 align:middle line:90% actually utilizes illusion. 00:36:25.210 --> 00:36:28.030 align:middle line:84% That's partly why studying perception is so much fun. 00:36:28.030 --> 00:36:29.990 align:middle line:84% And so throughout this class, we're 00:36:29.990 --> 00:36:34.550 align:middle line:84% going to be looking and listening to and feeling 00:36:34.550 --> 00:36:36.170 align:middle line:90% a lot of perceptual illusions. 00:36:36.170 --> 00:36:38.670 align:middle line:84% And you'll be making some yourself. 00:36:38.670 --> 00:36:40.330 align:middle line:90% So this is an example. 00:36:40.330 --> 00:36:44.510 align:middle line:84% This is a classic illusion, one of my favorites, 00:36:44.510 --> 00:36:50.030 align:middle line:84% where shadows are being used to manipulate 00:36:50.030 --> 00:36:53.610 align:middle line:84% your perception of depth and thus your perception of motion. 00:36:53.610 --> 00:36:56.830 align:middle line:84% And so what is going to happen here 00:36:56.830 --> 00:37:00.440 align:middle line:84% when I play this is you will see this ball 00:37:00.440 --> 00:37:01.800 align:middle line:90% move across the screen. 00:37:01.800 --> 00:37:04.440 align:middle line:90% It'll go from here to here. 00:37:04.440 --> 00:37:07.060 align:middle line:84% And you're going to see two different versions of this. 00:37:07.060 --> 00:37:09.720 align:middle line:84% And the versions will be differentiated 00:37:09.720 --> 00:37:13.120 align:middle line:84% by the trajectory of this dark spot that 00:37:13.120 --> 00:37:15.640 align:middle line:84% is supposed to be-- it's supposed to look like a shadow. 00:37:15.640 --> 00:37:18.400 align:middle line:84% And so in one case, the shadow is 00:37:18.400 --> 00:37:20.020 align:middle line:90% going to move in one direction. 00:37:20.020 --> 00:37:20.680 align:middle line:84% And in the other case, it's going 00:37:20.680 --> 00:37:21.888 align:middle line:90% to move in another direction. 00:37:21.888 --> 00:37:24.400 align:middle line:84% And that will cause you to perceive a totally different 00:37:24.400 --> 00:37:28.520 align:middle line:84% trajectory for this ball, even though the physical trajectory 00:37:28.520 --> 00:37:30.520 align:middle line:84% along the image is exactly the same. 00:37:30.520 --> 00:37:31.980 align:middle line:90% So check it out. 00:37:31.980 --> 00:37:33.740 align:middle line:84% And I'll loop this a couple times. 00:37:33.740 --> 00:37:40.440 align:middle line:90% 00:37:40.440 --> 00:37:43.600 align:middle line:90% So it's on the floor, right? 00:37:43.600 --> 00:37:45.100 align:middle line:90% Whoa. 00:37:45.100 --> 00:37:47.890 align:middle line:90% 00:37:47.890 --> 00:37:49.380 align:middle line:90% All right, let's do it again. 00:37:49.380 --> 00:37:53.440 align:middle line:90% 00:37:53.440 --> 00:37:58.430 align:middle line:90% On the floor, up in the air. 00:37:58.430 --> 00:38:01.010 align:middle line:90% 00:38:01.010 --> 00:38:02.190 align:middle line:90% All right. 00:38:02.190 --> 00:38:03.190 align:middle line:90% So what's going on here? 00:38:03.190 --> 00:38:06.290 align:middle line:84% So I think I love this particular effect 00:38:06.290 --> 00:38:10.090 align:middle line:84% because it illustrates a whole bunch of important things. 00:38:10.090 --> 00:38:12.010 align:middle line:84% So one of the things that illustrates 00:38:12.010 --> 00:38:14.950 align:middle line:84% is the ill-posedness of depth perception. 00:38:14.950 --> 00:38:19.450 align:middle line:84% So the image and the trajectory of the ball 00:38:19.450 --> 00:38:22.530 align:middle line:84% is exactly the same for those two totally different 00:38:22.530 --> 00:38:24.730 align:middle line:90% trajectories in the world. 00:38:24.730 --> 00:38:26.610 align:middle line:84% Three-dimensional trajectories, they 00:38:26.610 --> 00:38:30.070 align:middle line:84% project to the same two-dimensional image sequence. 00:38:30.070 --> 00:38:32.930 align:middle line:90% 00:38:32.930 --> 00:38:39.370 align:middle line:84% However, there is a relationship between the location 00:38:39.370 --> 00:38:43.410 align:middle line:90% of a shadow and an object. 00:38:43.410 --> 00:38:46.650 align:middle line:84% So in general, if something is sitting on a surface, 00:38:46.650 --> 00:38:48.370 align:middle line:84% the shadow that it casts will be right 00:38:48.370 --> 00:38:50.890 align:middle line:84% next to that object, just like the geometry 00:38:50.890 --> 00:38:52.070 align:middle line:90% of the way the optics work. 00:38:52.070 --> 00:38:55.480 align:middle line:84% If something is way off of a surface, 00:38:55.480 --> 00:38:59.140 align:middle line:84% then the shadow will be more distant in the image. 00:38:59.140 --> 00:39:02.240 align:middle line:84% And so your brain, either over evolution or development, 00:39:02.240 --> 00:39:07.540 align:middle line:84% has learned the relationship between shadows and depth. 00:39:07.540 --> 00:39:10.900 align:middle line:84% And it just automatically estimates 00:39:10.900 --> 00:39:13.740 align:middle line:84% the three-dimensional structure of the world using 00:39:13.740 --> 00:39:15.740 align:middle line:84% the shadow and its knowledge of the way 00:39:15.740 --> 00:39:17.920 align:middle line:90% that optics and geometry work. 00:39:17.920 --> 00:39:21.180 align:middle line:90% 00:39:21.180 --> 00:39:26.070 align:middle line:90% And this illusion really-- 00:39:26.070 --> 00:39:28.780 align:middle line:84% it demonstrates the role of shadows 00:39:28.780 --> 00:39:31.580 align:middle line:84% in depth perception in a way that you wouldn't really 00:39:31.580 --> 00:39:35.540 align:middle line:84% have known otherwise, because you have this stimulus that's 00:39:35.540 --> 00:39:36.480 align:middle line:90% otherwise the same. 00:39:36.480 --> 00:39:38.180 align:middle line:84% And you just manipulate the shadow. 00:39:38.180 --> 00:39:40.000 align:middle line:90% And it changes what you see. 00:39:40.000 --> 00:39:43.260 align:middle line:90% 00:39:43.260 --> 00:39:47.820 align:middle line:84% Another great example of this is lightness perception. 00:39:47.820 --> 00:39:50.340 align:middle line:84% So as organisms, we typically want 00:39:50.340 --> 00:39:53.110 align:middle line:84% to infer what things in the world are made of. 00:39:53.110 --> 00:39:55.790 align:middle line:84% And that is partially signaled by their pigmentation. 00:39:55.790 --> 00:39:58.510 align:middle line:84% Now, the problem that we face is that the light that 00:39:58.510 --> 00:40:01.350 align:middle line:84% reaches your eye from an object, it 00:40:01.350 --> 00:40:04.850 align:middle line:84% depends not just on the pigment of things in the world, 00:40:04.850 --> 00:40:06.690 align:middle line:84% but also on the amount of illumination. 00:40:06.690 --> 00:40:10.330 align:middle line:84% So right now, the light level in here is kind of moderate. 00:40:10.330 --> 00:40:11.842 align:middle line:90% And so this surface is white. 00:40:11.842 --> 00:40:13.550 align:middle line:84% And I'm getting a certain amount of light 00:40:13.550 --> 00:40:14.998 align:middle line:90% that is reflected off of it. 00:40:14.998 --> 00:40:17.290 align:middle line:84% That's a function of the fact that this thing is white. 00:40:17.290 --> 00:40:19.918 align:middle line:84% And that means that it reflects a high proportion of light, 00:40:19.918 --> 00:40:22.210 align:middle line:84% but also the fact that the illumination here is modest. 00:40:22.210 --> 00:40:24.550 align:middle line:84% So if we walked out into the center of the atrium, 00:40:24.550 --> 00:40:26.670 align:middle line:84% where the light level is much, much higher, 00:40:26.670 --> 00:40:29.950 align:middle line:84% the number of photons that will be coming off of the surface 00:40:29.950 --> 00:40:31.690 align:middle line:90% will go way up. 00:40:31.690 --> 00:40:34.030 align:middle line:84% Now, the amazing thing about your visual system 00:40:34.030 --> 00:40:37.150 align:middle line:84% is that the color of this thing doesn't change by and large. 00:40:37.150 --> 00:40:40.310 align:middle line:84% Most of the time, we have what's called color constancy 00:40:40.310 --> 00:40:41.490 align:middle line:90% or lightness constancy. 00:40:41.490 --> 00:40:44.190 align:middle line:84% And so in order to do that, your visual system 00:40:44.190 --> 00:40:48.350 align:middle line:84% has to somehow discount the illumination. 00:40:48.350 --> 00:40:51.240 align:middle line:84% And so illusions are one way that people have gotten 00:40:51.240 --> 00:40:53.320 align:middle line:90% a lot of insight into this. 00:40:53.320 --> 00:40:56.360 align:middle line:84% And so my PhD advisor, Ted Adelson, 00:40:56.360 --> 00:40:59.480 align:middle line:84% did a lot of the pioneering work on this. 00:40:59.480 --> 00:41:01.920 align:middle line:90% And this is one of my favorites. 00:41:01.920 --> 00:41:08.400 align:middle line:84% So this is an illusion because these ovals and these ovals 00:41:08.400 --> 00:41:09.860 align:middle line:90% are the same shade of gray. 00:41:09.860 --> 00:41:14.120 align:middle line:90% 00:41:14.120 --> 00:41:15.880 align:middle line:90% You look skeptical. 00:41:15.880 --> 00:41:17.800 align:middle line:90% So they are. 00:41:17.800 --> 00:41:22.600 align:middle line:84% And so this little test patch can be dragged over here, 00:41:22.600 --> 00:41:25.520 align:middle line:84% and then dragged up here to verify that they are actually 00:41:25.520 --> 00:41:27.032 align:middle line:90% the same. 00:41:27.032 --> 00:41:29.240 align:middle line:84% Do people actually want to-- do you want to see that? 00:41:29.240 --> 00:41:29.400 align:middle line:90% AUDIENCE: Yes. 00:41:29.400 --> 00:41:30.942 align:middle line:84% JOSH MCDERMOTT: You don't believe me? 00:41:30.942 --> 00:41:34.580 align:middle line:84% OK, well, we're going to have to satisfy your curiosity. 00:41:34.580 --> 00:41:37.112 align:middle line:90% 00:41:37.112 --> 00:41:38.820 align:middle line:84% All right, so let's see if I can do this. 00:41:38.820 --> 00:41:40.620 align:middle line:84% So we've got our test patch here. 00:41:40.620 --> 00:41:45.320 align:middle line:90% 00:41:45.320 --> 00:41:46.300 align:middle line:90% That's the same, right? 00:41:46.300 --> 00:41:46.800 align:middle line:90% OK. 00:41:46.800 --> 00:41:51.690 align:middle line:90% 00:41:51.690 --> 00:41:54.130 align:middle line:90% Oh. 00:41:54.130 --> 00:41:55.970 align:middle line:90% I tell you the truth, right? 00:41:55.970 --> 00:41:57.710 align:middle line:84% I only say true things in this class. 00:41:57.710 --> 00:42:03.610 align:middle line:90% 00:42:03.610 --> 00:42:04.390 align:middle line:90% All right. 00:42:04.390 --> 00:42:09.054 align:middle line:90% 00:42:09.054 --> 00:42:12.810 align:middle line:90% So why do they look different? 00:42:12.810 --> 00:42:13.650 align:middle line:90% Yeah? 00:42:13.650 --> 00:42:16.390 align:middle line:84% AUDIENCE: Because we make a lot of inferences based on shadows. 00:42:16.390 --> 00:42:19.190 align:middle line:84% And if we think that even in the shadow 00:42:19.190 --> 00:42:21.930 align:middle line:84% the bottom part is so light, we automatically 00:42:21.930 --> 00:42:25.490 align:middle line:84% associate higher than [INAUDIBLE] the shadow 00:42:25.490 --> 00:42:28.890 align:middle line:90% was typically with [INAUDIBLE]? 00:42:28.890 --> 00:42:30.970 align:middle line:84% JOSH MCDERMOTT: Yeah, so that's basically the way 00:42:30.970 --> 00:42:37.170 align:middle line:84% that we think about this, that this contains a lot of evidence 00:42:37.170 --> 00:42:40.490 align:middle line:84% that the illumination level down here is very, very low. 00:42:40.490 --> 00:42:43.610 align:middle line:84% And there's a lot of evidence that the illumination there 00:42:43.610 --> 00:42:44.710 align:middle line:90% is actually very high. 00:42:44.710 --> 00:42:47.090 align:middle line:84% So it's on the top side of this thing. 00:42:47.090 --> 00:42:49.250 align:middle line:84% You can see a little highlight here indicates 00:42:49.250 --> 00:42:50.530 align:middle line:90% there's a light source. 00:42:50.530 --> 00:42:54.650 align:middle line:84% This is in the lower region of this thing. 00:42:54.650 --> 00:42:57.490 align:middle line:84% So all this evidence for different illumination levels. 00:42:57.490 --> 00:43:00.250 align:middle line:84% And so the same physical amount of light in these two 00:43:00.250 --> 00:43:03.130 align:middle line:84% different cases is best explained 00:43:03.130 --> 00:43:04.970 align:middle line:84% by their being very different colors 00:43:04.970 --> 00:43:09.370 align:middle line:84% of paint on the object in these two settings. 00:43:09.370 --> 00:43:11.513 align:middle line:90% So again, it's an illusion. 00:43:11.513 --> 00:43:12.430 align:middle line:90% Why is it an illusion? 00:43:12.430 --> 00:43:14.510 align:middle line:84% It's an illusion because these things are physically the same, 00:43:14.510 --> 00:43:16.090 align:middle line:90% but they look really different. 00:43:16.090 --> 00:43:19.690 align:middle line:84% And it teaches us something about the way 00:43:19.690 --> 00:43:20.830 align:middle line:90% your visual system works. 00:43:20.830 --> 00:43:24.950 align:middle line:84% And so you often think of illusions as, in some sense, 00:43:24.950 --> 00:43:27.170 align:middle line:84% your perceptual system is making a mistake. 00:43:27.170 --> 00:43:30.370 align:middle line:84% But as a perceptual scientist, we typically 00:43:30.370 --> 00:43:32.850 align:middle line:84% think that the illusion represents 00:43:32.850 --> 00:43:36.850 align:middle line:84% a sensible engineering solution that is in your brain 00:43:36.850 --> 00:43:39.130 align:middle line:84% that most of the time is going to cause 00:43:39.130 --> 00:43:41.610 align:middle line:90% you to see the world as it is. 00:43:41.610 --> 00:43:44.797 align:middle line:84% So we think that the reason that this works the way that it does 00:43:44.797 --> 00:43:46.380 align:middle line:84% is, most of the time, this is actually 00:43:46.380 --> 00:43:49.900 align:middle line:84% helping you to correctly tell that this thing is white. 00:43:49.900 --> 00:43:51.300 align:middle line:90% Yeah? 00:43:51.300 --> 00:43:54.620 align:middle line:84% AUDIENCE: I'm just interested in knowing or seeing 00:43:54.620 --> 00:43:58.340 align:middle line:84% your opinion on how much of the other information 00:43:58.340 --> 00:44:04.600 align:middle line:84% plays a role in causing dilution because there 00:44:04.600 --> 00:44:06.140 align:middle line:90% are squiggly lines around it. 00:44:06.140 --> 00:44:09.840 align:middle line:84% And there's stuff under, on the ground. 00:44:09.840 --> 00:44:14.500 align:middle line:84% And so how much did that really contribute to-- if it was just 00:44:14.500 --> 00:44:19.180 align:middle line:84% a plain oval, would we still have that sense of illusion? 00:44:19.180 --> 00:44:21.453 align:middle line:84% JOSH MCDERMOTT: Yeah, so Ted and other people 00:44:21.453 --> 00:44:24.120 align:middle line:84% who have worked on these things have looked at things like that. 00:44:24.120 --> 00:44:26.740 align:middle line:84% And in general, there's lots and lots 00:44:26.740 --> 00:44:29.180 align:middle line:84% of little cues in these displays that 00:44:29.180 --> 00:44:32.080 align:middle line:84% add up to create this, like incredible illusion. 00:44:32.080 --> 00:44:34.900 align:middle line:84% And so if you take away some of those clues, 00:44:34.900 --> 00:44:37.643 align:middle line:84% it'll get a little bit weaker and weaker and weaker. 00:44:37.643 --> 00:44:39.060 align:middle line:84% In fact, when we start, we'll have 00:44:39.060 --> 00:44:41.940 align:middle line:84% a lecture on the problem of lightness perception 00:44:41.940 --> 00:44:43.460 align:middle line:90% in a couple of months. 00:44:43.460 --> 00:44:45.550 align:middle line:84% And we'll see the original version 00:44:45.550 --> 00:44:49.670 align:middle line:84% of this that was discovered 150 years ago is way simpler. 00:44:49.670 --> 00:44:52.710 align:middle line:84% It's essentially you take the same gray square 00:44:52.710 --> 00:44:56.150 align:middle line:84% and you surround it either with white or something dark. 00:44:56.150 --> 00:44:58.090 align:middle line:84% And there, you get a small effect. 00:44:58.090 --> 00:45:00.050 align:middle line:84% That's called simultaneous contrast. 00:45:00.050 --> 00:45:02.447 align:middle line:84% And this effect is huge because there's 00:45:02.447 --> 00:45:04.030 align:middle line:84% lots and lots of these other cues that 00:45:04.030 --> 00:45:05.770 align:middle line:90% have been added to the thing. 00:45:05.770 --> 00:45:08.470 align:middle line:84% And so your brain is just using all the different bits 00:45:08.470 --> 00:45:11.093 align:middle line:84% of information it can when it does this, 00:45:11.093 --> 00:45:12.010 align:middle line:90% as far as we can tell. 00:45:12.010 --> 00:45:12.610 align:middle line:90% Yeah? 00:45:12.610 --> 00:45:14.110 align:middle line:84% AUDIENCE: And actually, going off of 00:45:14.110 --> 00:45:19.150 align:middle line:84% her question, I'm wondering, is there-- 00:45:19.150 --> 00:45:20.790 align:middle line:84% because for the top dots, they're 00:45:20.790 --> 00:45:23.975 align:middle line:84% on the light portion of the wavy lines, 00:45:23.975 --> 00:45:25.350 align:middle line:84% whereas for the bottom dots, they 00:45:25.350 --> 00:45:26.690 align:middle line:90% correspond to a dark portion. 00:45:26.690 --> 00:45:27.940 align:middle line:90% Does that contribute anything? 00:45:27.940 --> 00:45:29.168 align:middle line:90% Or is it unrelated? 00:45:29.168 --> 00:45:30.210 align:middle line:90% JOSH MCDERMOTT: Yeah, no. 00:45:30.210 --> 00:45:34.270 align:middle line:84% That's one of those cues that will have some effect. 00:45:34.270 --> 00:45:36.430 align:middle line:84% Yeah, so there's these local cues. 00:45:36.430 --> 00:45:40.543 align:middle line:84% There's these geometric cues related to shape, like the fact 00:45:40.543 --> 00:45:42.960 align:middle line:84% that there's this blurriness here because a lot of shadows 00:45:42.960 --> 00:45:43.980 align:middle line:90% tend to be blurry. 00:45:43.980 --> 00:45:44.740 align:middle line:90% That also helps. 00:45:44.740 --> 00:45:49.520 align:middle line:84% There's a lot of stuff that's working towards this. 00:45:49.520 --> 00:45:54.720 align:middle line:84% OK, here's another example in the motion domain, which 00:45:54.720 --> 00:45:57.420 align:middle line:90% I will show you real quick. 00:45:57.420 --> 00:46:01.240 align:middle line:90% 00:46:01.240 --> 00:46:08.520 align:middle line:84% So here, we have some line segments. 00:46:08.520 --> 00:46:11.160 align:middle line:84% And it looks like there are these two pairs of line segments 00:46:11.160 --> 00:46:14.180 align:middle line:84% that are moving more or less independently. 00:46:14.180 --> 00:46:15.680 align:middle line:84% So what I'm going to do is I'm going 00:46:15.680 --> 00:46:18.520 align:middle line:84% to add some other shapes to this display. 00:46:18.520 --> 00:46:21.800 align:middle line:84% And it's going to totally change the motion that you see. 00:46:21.800 --> 00:46:25.760 align:middle line:84% So now, you look at this and you probably 00:46:25.760 --> 00:46:30.560 align:middle line:84% see a diamond that's moving around in a circle. 00:46:30.560 --> 00:46:32.220 align:middle line:84% Are people seeing that, more or less? 00:46:32.220 --> 00:46:32.720 align:middle line:90% Yeah? 00:46:32.720 --> 00:46:33.440 align:middle line:90% OK. 00:46:33.440 --> 00:46:35.720 align:middle line:84% And so if I get rid of them, you're 00:46:35.720 --> 00:46:41.170 align:middle line:84% going to go back to seeing these line segments. 00:46:41.170 --> 00:46:43.930 align:middle line:84% So again, it's another example of an ill-posed problem 00:46:43.930 --> 00:46:46.370 align:middle line:84% because it's the same image motion. 00:46:46.370 --> 00:46:49.450 align:middle line:84% And it can be explained in two different ways. 00:46:49.450 --> 00:46:51.970 align:middle line:84% It can be explained by these two pairs of line segments 00:46:51.970 --> 00:46:55.370 align:middle line:84% that are moving separately or by a single shape that's 00:46:55.370 --> 00:46:58.250 align:middle line:90% moving around in a circle. 00:46:58.250 --> 00:47:01.462 align:middle line:90% 00:47:01.462 --> 00:47:05.010 align:middle line:84% And in these two different settings, 00:47:05.010 --> 00:47:07.130 align:middle line:84% your brain is preferentially choosing 00:47:07.130 --> 00:47:09.210 align:middle line:84% one of those two interpretations, 00:47:09.210 --> 00:47:11.470 align:middle line:84% presumably based on what it thinks is most likely. 00:47:11.470 --> 00:47:14.010 align:middle line:84% So in this particular case, you've 00:47:14.010 --> 00:47:16.250 align:middle line:84% got these surfaces that could be occluding 00:47:16.250 --> 00:47:18.010 align:middle line:90% the corner of a diamond. 00:47:18.010 --> 00:47:21.570 align:middle line:84% And so the diamond perception seems pretty reasonable. 00:47:21.570 --> 00:47:26.530 align:middle line:84% But without that, it seems like the most likely explanation 00:47:26.530 --> 00:47:28.890 align:middle line:90% is something different. 00:47:28.890 --> 00:47:31.750 align:middle line:84% And so again, what do we learn from this particular illusion? 00:47:31.750 --> 00:47:36.690 align:middle line:84% Well, this shows us that, when interpreting motion, 00:47:36.690 --> 00:47:39.740 align:middle line:84% your brain seems to be taking into account information 00:47:39.740 --> 00:47:42.860 align:middle line:84% about occlusion, what other surfaces there 00:47:42.860 --> 00:47:45.420 align:middle line:84% are, and their potential depth relationships to things. 00:47:45.420 --> 00:47:48.540 align:middle line:90% 00:47:48.540 --> 00:47:49.740 align:middle line:90% OK. 00:47:49.740 --> 00:47:55.180 align:middle line:84% So to give you a firsthand taste of the role of illusions 00:47:55.180 --> 00:47:58.040 align:middle line:84% and perception, we have these illusion laboratories. 00:47:58.040 --> 00:48:00.780 align:middle line:84% This is a web page that we made for the version of this 00:48:00.780 --> 00:48:02.340 align:middle line:90% a couple of years ago. 00:48:02.340 --> 00:48:05.940 align:middle line:84% And so what you will do is you'll be in these small teams. 00:48:05.940 --> 00:48:08.180 align:middle line:84% And you will make your own illusion 00:48:08.180 --> 00:48:10.800 align:middle line:84% to try to answer some kind of question about perception. 00:48:10.800 --> 00:48:12.500 align:middle line:84% And so usually, what people do is 00:48:12.500 --> 00:48:15.800 align:middle line:84% they riff on things that they encounter during the class. 00:48:15.800 --> 00:48:17.460 align:middle line:84% And so over the course of the class, 00:48:17.460 --> 00:48:19.200 align:middle line:84% you'll encounter lots of these effects. 00:48:19.200 --> 00:48:21.233 align:middle line:84% And so many of you have asked me questions 00:48:21.233 --> 00:48:22.400 align:middle line:90% about how these things work. 00:48:22.400 --> 00:48:24.660 align:middle line:84% Well, that's an experiment that you can do. 00:48:24.660 --> 00:48:26.080 align:middle line:84% So you can try this out yourself. 00:48:26.080 --> 00:48:27.980 align:middle line:84% And then you look at it or you listen to it. 00:48:27.980 --> 00:48:29.022 align:middle line:90% And you ask your friends. 00:48:29.022 --> 00:48:33.900 align:middle line:84% And then you learn something about how you see or hear. 00:48:33.900 --> 00:48:36.105 align:middle line:90% AUDIENCE: I have a question. 00:48:36.105 --> 00:48:36.980 align:middle line:90% JOSH MCDERMOTT: Yeah? 00:48:36.980 --> 00:48:39.750 align:middle line:84% AUDIENCE: Would a person who has an unknown broken link between 00:48:39.750 --> 00:48:41.330 align:middle line:84% sound input and sound processing-- 00:48:41.330 --> 00:48:43.170 align:middle line:84% so someone that could generally hear fine, 00:48:43.170 --> 00:48:45.110 align:middle line:84% but maybe misinterprets what they're hearing-- 00:48:45.110 --> 00:48:47.110 align:middle line:84% would they still be able to perceive an illusion 00:48:47.110 --> 00:48:50.310 align:middle line:90% the same as a hearing person? 00:48:50.310 --> 00:48:53.110 align:middle line:90% JOSH MCDERMOTT: I don't know. 00:48:53.110 --> 00:48:57.330 align:middle line:84% So sorry, the question is somebody whose ears work fine, 00:48:57.330 --> 00:48:59.430 align:middle line:84% but maybe the downstream auditory system 00:48:59.430 --> 00:49:01.130 align:middle line:90% is not working normally? 00:49:01.130 --> 00:49:05.830 align:middle line:90% 00:49:05.830 --> 00:49:08.350 align:middle line:84% So yeah, to my knowledge, that has not really 00:49:08.350 --> 00:49:09.450 align:middle line:90% been studied very much. 00:49:09.450 --> 00:49:12.750 align:middle line:90% 00:49:12.750 --> 00:49:15.030 align:middle line:84% The main thing-- I mean, one of the main things 00:49:15.030 --> 00:49:18.390 align:middle line:84% that can cause the downstream auditory system to malfunction 00:49:18.390 --> 00:49:21.448 align:middle line:84% is brain damage that would typically happen after a stroke 00:49:21.448 --> 00:49:23.490 align:middle line:84% or if you hit your head really hard, potentially. 00:49:23.490 --> 00:49:27.910 align:middle line:84% And so there is lots of instances 00:49:27.910 --> 00:49:32.790 align:middle line:84% where people will report difficulty understanding speech. 00:49:32.790 --> 00:49:36.430 align:middle line:84% And it might well be that if you tested people 00:49:36.430 --> 00:49:38.290 align:middle line:84% on some of these kinds of examples, 00:49:38.290 --> 00:49:39.730 align:middle line:84% that would also work differently. 00:49:39.730 --> 00:49:41.870 align:middle line:84% But in practice, most of the time, 00:49:41.870 --> 00:49:45.170 align:middle line:84% you're worried about trying to get them to communicate better. 00:49:45.170 --> 00:49:48.130 align:middle line:84% And so you focus on the speech deficit. 00:49:48.130 --> 00:49:52.170 align:middle line:84% So I'm not aware of that having been studied very much, 00:49:52.170 --> 00:49:52.930 align:middle line:90% to be honest. 00:49:52.930 --> 00:49:55.990 align:middle line:90% 00:49:55.990 --> 00:49:56.830 align:middle line:90% OK. 00:49:56.830 --> 00:50:00.183 align:middle line:84% So why would you want to study perception? 00:50:00.183 --> 00:50:02.350 align:middle line:84% Well, one thing that's kind of cool about it is it's 00:50:02.350 --> 00:50:04.130 align:middle line:90% very relevant to everyday life. 00:50:04.130 --> 00:50:05.755 align:middle line:84% So there's just lots of things that you 00:50:05.755 --> 00:50:09.533 align:middle line:84% experience on a daily basis that really relate to perception. 00:50:09.533 --> 00:50:10.950 align:middle line:84% And then as a consequence of that, 00:50:10.950 --> 00:50:13.228 align:middle line:84% everyday experience can give you insights. 00:50:13.228 --> 00:50:14.270 align:middle line:90% You'll be walking around. 00:50:14.270 --> 00:50:14.990 align:middle line:90% And you'll see something. 00:50:14.990 --> 00:50:16.323 align:middle line:90% And it looks a little bit funny. 00:50:16.323 --> 00:50:21.350 align:middle line:84% And then you realize that it's not necessarily how it seems. 00:50:21.350 --> 00:50:23.990 align:middle line:84% There's also lots of exciting applications. 00:50:23.990 --> 00:50:27.768 align:middle line:84% So one important application domain is prosthetics. 00:50:27.768 --> 00:50:29.810 align:middle line:84% So we're going to talk about this in a few weeks. 00:50:29.810 --> 00:50:33.310 align:middle line:84% But lots of people lose their hearing as they age, 00:50:33.310 --> 00:50:35.360 align:middle line:90% in particular. 00:50:35.360 --> 00:50:37.840 align:middle line:84% And so hearing aids are a huge industry. 00:50:37.840 --> 00:50:39.020 align:middle line:90% They work OK, but not great. 00:50:39.020 --> 00:50:40.480 align:middle line:84% We'd like to make them work better. 00:50:40.480 --> 00:50:42.560 align:middle line:84% There's also cochlear implants for someone 00:50:42.560 --> 00:50:47.380 align:middle line:84% who's deaf, retinal implants for someone who's blind. 00:50:47.380 --> 00:50:50.560 align:middle line:90% 00:50:50.560 --> 00:50:52.622 align:middle line:84% There's lots of exciting new directions 00:50:52.622 --> 00:50:55.080 align:middle line:84% in all of those domains because of some of the developments 00:50:55.080 --> 00:50:57.600 align:middle line:84% that have been happening in terms of computational models 00:50:57.600 --> 00:50:59.320 align:middle line:90% of sensory systems. 00:50:59.320 --> 00:51:02.260 align:middle line:84% It's also very relevant to the design of displays. 00:51:02.260 --> 00:51:04.990 align:middle line:90% 00:51:04.990 --> 00:51:07.408 align:middle line:84% Right now, in particular, it's kind of an exciting time 00:51:07.408 --> 00:51:09.200 align:middle line:84% to be working on perception because there's 00:51:09.200 --> 00:51:13.040 align:middle line:84% lots of interesting analogs to AI systems. 00:51:13.040 --> 00:51:16.760 align:middle line:84% And the other thing to note is that sensory systems, 00:51:16.760 --> 00:51:23.440 align:middle line:84% they have homologues, typically, in non-human animals. 00:51:23.440 --> 00:51:28.640 align:middle line:84% So other things that we do with our brains 00:51:28.640 --> 00:51:30.120 align:middle line:90% are fairly uniquely human. 00:51:30.120 --> 00:51:32.250 align:middle line:90% Most animals don't talk. 00:51:32.250 --> 00:51:34.930 align:middle line:84% It's not clear how much they think. 00:51:34.930 --> 00:51:38.670 align:middle line:84% But most animals see, and hear, and smell, and taste, and touch. 00:51:38.670 --> 00:51:41.330 align:middle line:84% So sensory systems have classically 00:51:41.330 --> 00:51:43.350 align:middle line:84% been model systems for neuroscience. 00:51:43.350 --> 00:51:47.770 align:middle line:84% And so knowing about perception is important in that sense. 00:51:47.770 --> 00:51:48.270 align:middle line:90% All right. 00:51:48.270 --> 00:51:52.130 align:middle line:84% So just to summarize what we talked about, 00:51:52.130 --> 00:51:55.330 align:middle line:84% we talked about how perceptual systems measure energy 00:51:55.330 --> 00:51:58.050 align:middle line:84% from different sources in the world, 00:51:58.050 --> 00:52:01.590 align:middle line:84% and then try to infer what caused that in the world. 00:52:01.590 --> 00:52:04.350 align:middle line:84% We talked about how perception is deceptively hard. 00:52:04.350 --> 00:52:06.753 align:middle line:84% So you've got these sensory organs that measure energy. 00:52:06.753 --> 00:52:08.170 align:middle line:84% But then the rest of your brain is 00:52:08.170 --> 00:52:10.530 align:middle line:84% there to really take that information 00:52:10.530 --> 00:52:13.447 align:middle line:84% and figure out what caused it in the world. 00:52:13.447 --> 00:52:16.030 align:middle line:84% We talked about how perceptual problems are usually ill-posed. 00:52:16.030 --> 00:52:19.410 align:middle line:84% That means there's typically not a unique solution. 00:52:19.410 --> 00:52:22.030 align:middle line:84% And as a consequence, perception is unconscious inference, 00:52:22.030 --> 00:52:23.992 align:middle line:84% where you are unconsciously trying 00:52:23.992 --> 00:52:25.450 align:middle line:84% to choose what we think is the best 00:52:25.450 --> 00:52:28.890 align:middle line:84% explanation of the sensory data that you get. 00:52:28.890 --> 00:52:31.660 align:middle line:84% And we talked about how illusions illustrate 00:52:31.660 --> 00:52:33.120 align:middle line:90% perceptual mechanisms at work. 00:52:33.120 --> 00:52:35.537 align:middle line:84% So they're fun to look at, but they're also powerful tools 00:52:35.537 --> 00:52:37.980 align:middle line:84% that we can use to help understand your brain. 00:52:37.980 --> 00:52:43.060 align:middle line:84% Any questions about what I've talked about so far? 00:52:43.060 --> 00:52:43.908 align:middle line:90% Yeah? 00:52:43.908 --> 00:52:45.700 align:middle line:84% AUDIENCE: Well, we talked about perceptions 00:52:45.700 --> 00:52:47.340 align:middle line:90% of unconscious inference. 00:52:47.340 --> 00:52:49.680 align:middle line:84% But sometimes, especially with illusions, 00:52:49.680 --> 00:52:52.660 align:middle line:84% sometimes when we notice an illusion or see it, 00:52:52.660 --> 00:52:56.200 align:middle line:84% we can understand it consciously as well, 00:52:56.200 --> 00:52:57.620 align:middle line:90% like in a different way. 00:52:57.620 --> 00:53:01.940 align:middle line:84% And is that why it's in parentheses, like unconscious 00:53:01.940 --> 00:53:02.440 align:middle line:90% inference? 00:53:02.440 --> 00:53:05.620 align:middle line:84% Because sometimes it is kind of conscious? 00:53:05.620 --> 00:53:07.940 align:middle line:84% JOSH MCDERMOTT: That's not what I meant. 00:53:07.940 --> 00:53:13.577 align:middle line:84% I was just-- yeah, I could take away the parentheses. 00:53:13.577 --> 00:53:15.160 align:middle line:84% And that would be equally appropriate. 00:53:15.160 --> 00:53:17.580 align:middle line:84% But the point is perception is inference. 00:53:17.580 --> 00:53:19.420 align:middle line:84% But it's a particular type of inference 00:53:19.420 --> 00:53:20.760 align:middle line:90% that is typically unconscious. 00:53:20.760 --> 00:53:25.820 align:middle line:84% And so sometimes when you're made aware of an illusion, 00:53:25.820 --> 00:53:28.930 align:middle line:84% you can then think about it and try to understand it. 00:53:28.930 --> 00:53:30.930 align:middle line:84% They're often actually quite hard to understand. 00:53:30.930 --> 00:53:33.790 align:middle line:84% And part of that is because the perceptual systems are 00:53:33.790 --> 00:53:35.842 align:middle line:84% often fairly, what we call, encapsulated. 00:53:35.842 --> 00:53:37.550 align:middle line:84% You don't really have a tremendous amount 00:53:37.550 --> 00:53:39.610 align:middle line:84% of conscious insight into why does that look light 00:53:39.610 --> 00:53:40.777 align:middle line:90% and why does that look dark. 00:53:40.777 --> 00:53:44.030 align:middle line:84% You have to think about it and stuff. 00:53:44.030 --> 00:53:48.190 align:middle line:84% So that's just an interesting characteristic of perception, 00:53:48.190 --> 00:53:49.790 align:middle line:90% that you're-- 00:53:49.790 --> 00:53:52.750 align:middle line:84% yeah, it's often fairly impenetrable. 00:53:52.750 --> 00:53:56.790 align:middle line:84% But there are interesting cases of what people will often 00:53:56.790 --> 00:53:59.750 align:middle line:84% refer to as top-down effects, where maybe 00:53:59.750 --> 00:54:04.550 align:middle line:84% what you think then influences what you see or hear. 00:54:04.550 --> 00:54:10.910 align:middle line:84% So there are interesting cases of that, yeah. 00:54:10.910 --> 00:54:14.030 align:middle line:90% OK. 00:54:14.030 --> 00:54:20.045 align:middle line:84% So the sense of hearing exists to measure sound and use 00:54:20.045 --> 00:54:22.170 align:middle line:84% it to infer what's happening in the ear around you. 00:54:22.170 --> 00:54:25.610 align:middle line:84% So sound is produced when objects in the world vibrate. 00:54:25.610 --> 00:54:28.600 align:middle line:84% They transmit acoustic energy through a surrounding medium 00:54:28.600 --> 00:54:30.960 align:middle line:90% in the form of a wave. 00:54:30.960 --> 00:54:34.280 align:middle line:84% The task of the ears is to measure sound and transmit that 00:54:34.280 --> 00:54:36.060 align:middle line:84% to the brain as an electrical signal. 00:54:36.060 --> 00:54:38.540 align:middle line:84% And the task of the brain is to interpret that signal, 00:54:38.540 --> 00:54:43.360 align:middle line:84% and then to use it to figure out what is out there in the world. 00:54:43.360 --> 00:54:45.140 align:middle line:90% So sound waves are longitudinal. 00:54:45.140 --> 00:54:46.760 align:middle line:84% So that means they consist of regions 00:54:46.760 --> 00:54:49.343 align:middle line:84% of high and low pressure that move away from the sound source. 00:54:49.343 --> 00:54:52.540 align:middle line:84% Here, we have a tuning fork that's been whacked. 00:54:52.540 --> 00:54:54.000 align:middle line:90% So it's vibrating. 00:54:54.000 --> 00:54:56.120 align:middle line:84% And there'll be this longitudinal wave 00:54:56.120 --> 00:54:58.400 align:middle line:84% that moves away from it with regions 00:54:58.400 --> 00:55:00.100 align:middle line:90% of compression and rarefaction. 00:55:00.100 --> 00:55:02.960 align:middle line:90% 00:55:02.960 --> 00:55:03.460 align:middle line:90% All right. 00:55:03.460 --> 00:55:04.877 align:middle line:84% So some facts about sound-- so one 00:55:04.877 --> 00:55:07.040 align:middle line:84% is that it needs a medium to carry it. 00:55:07.040 --> 00:55:10.800 align:middle line:84% How many people have seen this movie? 00:55:10.800 --> 00:55:11.940 align:middle line:90% Every year, it's a couple. 00:55:11.940 --> 00:55:13.860 align:middle line:84% Yeah, you should all go watch this movie. 00:55:13.860 --> 00:55:15.280 align:middle line:90% It's one of my favorites, OK? 00:55:15.280 --> 00:55:19.600 align:middle line:84% So Alien is a movie that came out in 1979. 00:55:19.600 --> 00:55:23.020 align:middle line:84% And it takes place far in the future. 00:55:23.020 --> 00:55:26.680 align:middle line:84% There's a whole bunch of people that are on this spaceship. 00:55:26.680 --> 00:55:28.813 align:middle line:84% And they get an alert beacon from this planet 00:55:28.813 --> 00:55:29.480 align:middle line:90% they're passing. 00:55:29.480 --> 00:55:32.980 align:middle line:84% And they go down there and some bad things happen. 00:55:32.980 --> 00:55:36.560 align:middle line:84% And they end up with this monster on the spaceship. 00:55:36.560 --> 00:55:39.660 align:middle line:84% And then they have to deal with that. 00:55:39.660 --> 00:55:42.300 align:middle line:90% And lots of bad things happen. 00:55:42.300 --> 00:55:45.400 align:middle line:84% And so this is the poster that was originally 00:55:45.400 --> 00:55:48.100 align:middle line:90% used to publicize the movie. 00:55:48.100 --> 00:55:51.900 align:middle line:84% And you can see that it's got this slogan, "In space, 00:55:51.900 --> 00:55:55.320 align:middle line:90% no one can hear you scream." 00:55:55.320 --> 00:55:56.920 align:middle line:90% So why is that? 00:55:56.920 --> 00:55:57.923 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:55:57.923 --> 00:55:59.340 align:middle line:84% JOSH MCDERMOTT: There's no medium. 00:55:59.340 --> 00:55:59.882 align:middle line:90% That's right. 00:55:59.882 --> 00:56:00.920 align:middle line:90% Space is a vacuum. 00:56:00.920 --> 00:56:02.760 align:middle line:90% So there's no sound in space. 00:56:02.760 --> 00:56:05.340 align:middle line:84% So they won't be able to hear you scream. 00:56:05.340 --> 00:56:07.120 align:middle line:84% So sound needs a medium to carry it. 00:56:07.120 --> 00:56:10.840 align:middle line:90% 00:56:10.840 --> 00:56:12.680 align:middle line:84% So sound travels through the medium. 00:56:12.680 --> 00:56:14.920 align:middle line:84% The speed of sound is proportional to the density 00:56:14.920 --> 00:56:16.920 align:middle line:90% of a medium. 00:56:16.920 --> 00:56:19.480 align:middle line:84% So sound in air at room temperature moves 00:56:19.480 --> 00:56:22.360 align:middle line:90% at 343 meters per second. 00:56:22.360 --> 00:56:25.210 align:middle line:84% So in water, because water is denser, the speed is faster, 00:56:25.210 --> 00:56:26.810 align:middle line:90% 1,500 meters per second. 00:56:26.810 --> 00:56:28.970 align:middle line:84% In solid objects, it gets really fast. 00:56:28.970 --> 00:56:31.955 align:middle line:90% 00:56:31.955 --> 00:56:34.570 align:middle line:84% So the intensity of a sound is the energy 00:56:34.570 --> 00:56:38.330 align:middle line:84% that's transmitted per second through a unit area. 00:56:38.330 --> 00:56:40.670 align:middle line:84% But the speed is independent of a sound's intensity. 00:56:40.670 --> 00:56:44.610 align:middle line:84% So whether you speak quietly or yell really loudly, 00:56:44.610 --> 00:56:47.930 align:middle line:84% the sound will travel at the same rate away from you. 00:56:47.930 --> 00:56:52.590 align:middle line:84% What does change with distance is intensity. 00:56:52.590 --> 00:56:55.330 align:middle line:84% So it's often approximately the case 00:56:55.330 --> 00:56:57.140 align:middle line:84% that, when something is making sound, 00:56:57.140 --> 00:56:58.890 align:middle line:84% the waves that travel away from the source 00:56:58.890 --> 00:57:01.410 align:middle line:90% are approximately spherical. 00:57:01.410 --> 00:57:03.010 align:middle line:84% And so the consequence of that is 00:57:03.010 --> 00:57:05.650 align:middle line:84% that, as you move away from the source, 00:57:05.650 --> 00:57:07.890 align:middle line:84% by conservation of energy, the energy that's 00:57:07.890 --> 00:57:10.130 align:middle line:84% in the sphere of remains the same. 00:57:10.130 --> 00:57:11.690 align:middle line:90% But the sphere gets bigger. 00:57:11.690 --> 00:57:16.370 align:middle line:84% And so the energy per unit area drops as you move away 00:57:16.370 --> 00:57:17.730 align:middle line:90% from the source. 00:57:17.730 --> 00:57:19.810 align:middle line:84% And it drops with the square of the distance 00:57:19.810 --> 00:57:21.730 align:middle line:84% because the area of that sphere is 00:57:21.730 --> 00:57:24.020 align:middle line:84% proportional to the square of the distance. 00:57:24.020 --> 00:57:26.260 align:middle line:84% And so that's known as the inverse square law. 00:57:26.260 --> 00:57:28.420 align:middle line:84% So there are these very predictable relationships 00:57:28.420 --> 00:57:32.280 align:middle line:84% between how close you are to something and how loud it is. 00:57:32.280 --> 00:57:35.900 align:middle line:90% 00:57:35.900 --> 00:57:36.440 align:middle line:90% All right. 00:57:36.440 --> 00:57:39.260 align:middle line:84% So the last thing we're going to talk about today 00:57:39.260 --> 00:57:40.940 align:middle line:90% is the decibel scale. 00:57:40.940 --> 00:57:43.780 align:middle line:84% So sound, sound level specifically, 00:57:43.780 --> 00:57:45.485 align:middle line:90% is measured in decibels. 00:57:45.485 --> 00:57:47.860 align:middle line:84% So one of the remarkable things about the auditory system 00:57:47.860 --> 00:57:51.460 align:middle line:84% is that it can deal with a huge range of sound intensities. 00:57:51.460 --> 00:57:54.960 align:middle line:84% So typically, instead of talking about intensity directly, 00:57:54.960 --> 00:57:57.580 align:middle line:84% we use a logarithmic scale of the ratio 00:57:57.580 --> 00:58:00.300 align:middle line:90% between two sound intensities. 00:58:00.300 --> 00:58:06.220 align:middle line:84% So a bel is defined as a ratio of 10 to 1. 00:58:06.220 --> 00:58:08.780 align:middle line:84% So if you want to know the number of bels that 00:58:08.780 --> 00:58:12.300 align:middle line:84% differentiate two sounds-- one with having pressure P1 00:58:12.300 --> 00:58:14.940 align:middle line:84% and the other having pressure P0-- 00:58:14.940 --> 00:58:18.580 align:middle line:84% you take the log to the base 10 of their ratio. 00:58:18.580 --> 00:58:22.310 align:middle line:84% So it turns out that bels are impractically large. 00:58:22.310 --> 00:58:24.910 align:middle line:84% So a lot of the differences that actually matter in sound 00:58:24.910 --> 00:58:26.330 align:middle line:90% are smaller than that. 00:58:26.330 --> 00:58:29.450 align:middle line:84% And so we use the decibel scale instead. 00:58:29.450 --> 00:58:31.170 align:middle line:84% So a decibel is a tenth of a bel. 00:58:31.170 --> 00:58:34.830 align:middle line:84% And so the number of decibels by which two sounds are different-- 00:58:34.830 --> 00:58:36.710 align:middle line:90% the pressures P1 and P2-- 00:58:36.710 --> 00:58:41.110 align:middle line:84% is 10 to the log 10 of that ratio. 00:58:41.110 --> 00:58:46.490 align:middle line:84% So if you increase the sound level by 10 decibels-- 00:58:46.490 --> 00:58:48.170 align:middle line:84% this stands for power, not pressure. 00:58:48.170 --> 00:58:49.110 align:middle line:90% I misspoke. 00:58:49.110 --> 00:58:50.990 align:middle line:90% So these are power measurements. 00:58:50.990 --> 00:58:54.510 align:middle line:84% So if you increase the sound level by 10 dB, 10 decibels, 00:58:54.510 --> 00:58:57.470 align:middle line:84% that means a 10-fold increase in power because log to the base 10 00:58:57.470 --> 00:58:58.930 align:middle line:90% of 10 is 1. 00:58:58.930 --> 00:59:01.890 align:middle line:84% 20 dB is a 100-fold increase in power. 00:59:01.890 --> 00:59:06.070 align:middle line:90% 00:59:06.070 --> 00:59:11.120 align:middle line:84% So typically, we will describe particular things 00:59:11.120 --> 00:59:12.370 align:middle line:90% in terms of their sound level. 00:59:12.370 --> 00:59:14.787 align:middle line:84% So I might say, well, I went to a rock concert last night. 00:59:14.787 --> 00:59:18.390 align:middle line:90% And it was 120 dB. 00:59:18.390 --> 00:59:22.800 align:middle line:84% Whenever we say that, we are implicitly 00:59:22.800 --> 00:59:27.040 align:middle line:84% measuring things with reference to an agreed upon reference 00:59:27.040 --> 00:59:28.280 align:middle line:90% sound level. 00:59:28.280 --> 00:59:31.200 align:middle line:90% Here, it's called P0. 00:59:31.200 --> 00:59:33.960 align:middle line:84% So there is a Bureau of Standards 00:59:33.960 --> 00:59:37.240 align:middle line:84% that's responsible for all these kinds of references. 00:59:37.240 --> 00:59:39.080 align:middle line:84% And if you go to the Bureau of Standards 00:59:39.080 --> 00:59:41.320 align:middle line:84% and you look up this reference, you'll 00:59:41.320 --> 00:59:43.880 align:middle line:84% find there's this particular number that's 00:59:43.880 --> 00:59:46.800 align:middle line:84% been chosen as the reference level for what 00:59:46.800 --> 00:59:49.220 align:middle line:84% are called sound pressure level measurements. 00:59:49.220 --> 00:59:52.680 align:middle line:84% And that particular value, P0, is 00:59:52.680 --> 00:59:55.440 align:middle line:84% chosen to be close to the minimum detectable sound 00:59:55.440 --> 00:59:58.000 align:middle line:90% level for humans. 00:59:58.000 --> 01:00:01.820 align:middle line:84% So 0 dB is defined to be that reference sound level. 01:00:01.820 --> 01:00:04.200 align:middle line:84% So if you have a sound that is at 0 dB, 01:00:04.200 --> 01:00:08.640 align:middle line:84% that means that ratio in there is 1 Because log of 1 is 0. 01:00:08.640 --> 01:00:12.900 align:middle line:84% And so that means that sound is at that power level. 01:00:12.900 --> 01:00:15.520 align:middle line:84% And so that's intended to be the quietest thing 01:00:15.520 --> 01:00:17.380 align:middle line:90% that you can hear. 01:00:17.380 --> 01:00:19.770 align:middle line:84% Of course, this is all approximate, because everybody's 01:00:19.770 --> 01:00:21.145 align:middle line:84% hearing's a little bit different. 01:00:21.145 --> 01:00:23.370 align:middle line:84% But roughly speaking, something that is 0 dB SPL 01:00:23.370 --> 01:00:25.770 align:middle line:84% is supposed to be just the quietest thing that you could 01:00:25.770 --> 01:00:27.353 align:middle line:84% detect under perfect conditions if you 01:00:27.353 --> 01:00:30.590 align:middle line:90% were in a really quiet room. 01:00:30.590 --> 01:00:33.090 align:middle line:84% And so when we measure sound levels in this way with respect 01:00:33.090 --> 01:00:35.730 align:middle line:84% to that standard reference-- so if you have a sound meter, 01:00:35.730 --> 01:00:37.970 align:middle line:90% like we have in our lab-- 01:00:37.970 --> 01:00:44.210 align:middle line:84% we will say that the sound is, in this case, 120 dB SPL. 01:00:44.210 --> 01:00:46.730 align:middle line:84% So when you say dB SPL, it means that you're 01:00:46.730 --> 01:00:48.730 align:middle line:84% using this special reference that 01:00:48.730 --> 01:00:53.650 align:middle line:84% is close to the threshold of human hearing. 01:00:53.650 --> 01:00:54.150 align:middle line:90% All right. 01:00:54.150 --> 01:00:56.190 align:middle line:84% So here are some example SPL values. 01:00:56.190 --> 01:00:58.690 align:middle line:84% As we said, 0 is designed to be close to the threshold 01:00:58.690 --> 01:01:00.370 align:middle line:90% of hearing. 01:01:00.370 --> 01:01:03.770 align:middle line:84% Normal breathing would be around 10 dB. 01:01:03.770 --> 01:01:06.690 align:middle line:84% A very, very soft whisper would be 30 dB; 01:01:06.690 --> 01:01:12.030 align:middle line:84% quiet conversation, 50 dB; busy traffic on Massachusetts Avenue, 01:01:12.030 --> 01:01:14.210 align:middle line:84% for instance, might be around 70 dB. 01:01:14.210 --> 01:01:16.520 align:middle line:84% Somebody shouts right next to you, 01:01:16.520 --> 01:01:19.180 align:middle line:90% it'd be close to maybe 90 dB. 01:01:19.180 --> 01:01:22.300 align:middle line:84% Once you get into the hundreds of decibels, 01:01:22.300 --> 01:01:25.468 align:middle line:84% you're getting sound levels where prolonged exposure can 01:01:25.468 --> 01:01:27.260 align:middle line:84% cause hearing loss, where you should really 01:01:27.260 --> 01:01:30.020 align:middle line:90% be wearing earplugs. 01:01:30.020 --> 01:01:33.180 align:middle line:84% 120 decibels is a propeller plane at takeoff. 01:01:33.180 --> 01:01:38.405 align:middle line:84% And 140 decibels is roughly the sound level of a jet at takeoff. 01:01:38.405 --> 01:01:40.280 align:middle line:84% And supposedly, that's the threshold of pain. 01:01:40.280 --> 01:01:41.800 align:middle line:84% I've never experienced that myself. 01:01:41.800 --> 01:01:42.760 align:middle line:90% And I don't intend to. 01:01:42.760 --> 01:01:45.900 align:middle line:90% 01:01:45.900 --> 01:01:48.897 align:middle line:84% So this is the last thing that we're going to talk about. 01:01:48.897 --> 01:01:49.980 align:middle line:90% And then I'm going to end. 01:01:49.980 --> 01:01:52.940 align:middle line:84% So another reason to use the decibel scale, 01:01:52.940 --> 01:01:55.020 align:middle line:84% and the main one that is very, very common 01:01:55.020 --> 01:01:58.140 align:middle line:84% in perceptual science, is that human discrimination 01:01:58.140 --> 01:02:01.980 align:middle line:84% thresholds-- so that means the smallest change in intensity 01:02:01.980 --> 01:02:03.900 align:middle line:90% that you can detect. 01:02:03.900 --> 01:02:07.460 align:middle line:84% Human discrimination thresholds are roughly constant when 01:02:07.460 --> 01:02:09.200 align:middle line:90% you measure them in decibels. 01:02:09.200 --> 01:02:11.220 align:middle line:84% And usually, they're on the order 01:02:11.220 --> 01:02:14.302 align:middle line:84% of 1 decibel in optimal conditions. 01:02:14.302 --> 01:02:15.260 align:middle line:90% So what does that mean? 01:02:15.260 --> 01:02:17.980 align:middle line:84% That means that if we have a pretty quiet sound, so 01:02:17.980 --> 01:02:22.600 align:middle line:84% like, say, 40 or dB, you'll just be able-- so if it's 40 dB, 01:02:22.600 --> 01:02:25.200 align:middle line:84% you'll just be able to detect a change in intensity 01:02:25.200 --> 01:02:27.860 align:middle line:90% if I move it up to 41. 01:02:27.860 --> 01:02:30.780 align:middle line:84% Similarly, if I have a pretty loud sound, 90 dB, 01:02:30.780 --> 01:02:33.440 align:middle line:84% you'll just be able to detect a change in intensity 01:02:33.440 --> 01:02:35.580 align:middle line:90% if I move it up to 91 dB. 01:02:35.580 --> 01:02:38.060 align:middle line:84% And so the decibel scale ends up being 01:02:38.060 --> 01:02:41.860 align:middle line:84% pretty convenient because the changes that you can detect 01:02:41.860 --> 01:02:48.060 align:middle line:84% are pretty constant as a function of sound intensity. 01:02:48.060 --> 01:02:48.560 align:middle line:90% All right. 01:02:48.560 --> 01:02:50.560 align:middle line:84% So I'm going to quickly play you some demos that 01:02:50.560 --> 01:02:52.620 align:middle line:90% illustrate the decibel scale. 01:02:52.620 --> 01:02:53.713 align:middle line:90% Here you go. 01:02:53.713 --> 01:02:54.380 align:middle line:90% [AUDIO PLAYBACK] 01:02:54.380 --> 01:02:57.020 align:middle line:90% - The decibel scale. 01:02:57.020 --> 01:03:03.180 align:middle line:84% Broadband noise is reduced in 10 steps of six decibels. 01:03:03.180 --> 01:03:05.160 align:middle line:84% Demonstrations are repeated once. 01:03:05.160 --> 01:03:05.743 align:middle line:90% [END PLAYBACK] 01:03:05.743 --> 01:03:06.618 align:middle line:90% JOSH MCDERMOTT: Oops. 01:03:06.618 --> 01:03:07.240 align:middle line:90% OK, sorry. 01:03:07.240 --> 01:03:09.740 align:middle line:84% So I wanted to pause this just to say so this is going to be 01:03:09.740 --> 01:03:11.740 align:middle line:90% 6-decibel steps. 01:03:11.740 --> 01:03:14.210 align:middle line:84% So I told you that your threshold is around 1 dB. 01:03:14.210 --> 01:03:16.650 align:middle line:84% So these 6-decibel steps will be very, very obvious. 01:03:16.650 --> 01:03:19.410 align:middle line:90% OK, let's do it. 01:03:19.410 --> 01:03:20.790 align:middle line:90% [AUDIO PLAYBACK] 01:03:20.790 --> 01:03:26.310 align:middle line:84% - The decibel is reduced in 10 steps of six decibels. 01:03:26.310 --> 01:03:29.238 align:middle line:84% Demonstrations are repeated once. 01:03:29.238 --> 01:03:32.166 align:middle line:90% [PULSING STATIC] 01:03:32.166 --> 01:03:39.974 align:middle line:90% 01:03:39.974 --> 01:03:41.158 align:middle line:90% [END PLAYBACK] 01:03:41.158 --> 01:03:41.950 align:middle line:90% JOSH MCDERMOTT: OK. 01:03:41.950 --> 01:03:45.030 align:middle line:84% Now, in this example, the next one, 01:03:45.030 --> 01:03:47.950 align:middle line:84% I think this is in 1-decibel steps. 01:03:47.950 --> 01:03:50.507 align:middle line:84% So these steps should just barely be discriminable. 01:03:50.507 --> 01:03:52.590 align:middle line:84% And a lot of times, you might not be 100% positive 01:03:52.590 --> 01:03:53.810 align:middle line:90% if the level is changing. 01:03:53.810 --> 01:03:56.483 align:middle line:84% But you'll be able to then hear it over a couple steps. 01:03:56.483 --> 01:03:57.150 align:middle line:90% [AUDIO PLAYBACK] 01:03:57.150 --> 01:04:02.396 align:middle line:84% - Broadband noise is reduced in 20 steps of one decibel. 01:04:02.396 --> 01:04:05.312 align:middle line:90% [PULSING STATIC] 01:04:05.312 --> 01:04:13.988 align:middle line:90% 01:04:13.988 --> 01:04:14.780 align:middle line:90% JOSH MCDERMOTT: OK. 01:04:14.780 --> 01:04:17.940 align:middle line:84% So that's very close to your discriminant-- 01:04:17.940 --> 01:04:19.410 align:middle line:90% [PULSING STATIC] 01:04:19.410 --> 01:04:20.880 align:middle line:90% [LAUGHTER] 01:04:20.880 --> 01:04:23.820 align:middle line:90% [END PLAYBACK] 01:04:23.820 --> 01:04:26.257 align:middle line:90% 01:04:26.257 --> 01:04:27.340 align:middle line:90% JOSH MCDERMOTT: All right. 01:04:27.340 --> 01:04:29.440 align:middle line:84% So those steps were very close to your discrimination 01:04:29.440 --> 01:04:29.940 align:middle line:90% threshold. 01:04:29.940 --> 01:04:31.080 align:middle line:84% And you could probably just barely tell 01:04:31.080 --> 01:04:32.220 align:middle line:90% that that was changing. 01:04:32.220 --> 01:04:34.760 align:middle line:90% So that's the decibel scale. 01:04:34.760 --> 01:04:36.733 align:middle line:84% That's how we measure sound intensities. 01:04:36.733 --> 01:04:37.900 align:middle line:90% It's always based on ratios. 01:04:37.900 --> 01:04:39.680 align:middle line:90% There's got to be a reference. 01:04:39.680 --> 01:04:42.800 align:middle line:84% And we use it because it's pretty closely related 01:04:42.800 --> 01:04:44.180 align:middle line:90% to discrimination thresholds. 01:04:44.180 --> 01:04:45.520 align:middle line:90% We're going to end there. 01:04:45.520 --> 01:04:46.652 align:middle line:90% Welcome to 935. 01:04:46.652 --> 01:04:47.860 align:middle line:90% We're happy to have you here. 01:04:47.860 --> 01:04:50.060 align:middle line:84% I'm going to have office hours upstairs. 01:04:50.060 --> 01:04:52.280 align:middle line:84% You're welcome to come by, no obligation. 01:04:52.280 --> 01:04:54.710 align:middle line:90% And we will see you Tuesday. 01:04:54.710 --> 01:05:10.000 align:middle line:90%