WEBVTT 00:00:00.000 --> 00:00:01.979 align:middle line:90% [SQUEAKING] 00:00:01.979 --> 00:00:03.951 align:middle line:90% [RUSTLING] 00:00:03.951 --> 00:00:06.909 align:middle line:90% [CLICKING] 00:00:06.909 --> 00:00:09.020 align:middle line:90% 00:00:09.020 --> 00:00:10.520 align:middle line:84% JOSH MCDERMOTT: There was a question 00:00:10.520 --> 00:00:14.080 align:middle line:84% at the end of-- or in the middle of last lecture 00:00:14.080 --> 00:00:17.240 align:middle line:84% about the relationship of loudness to frequency, 00:00:17.240 --> 00:00:20.160 align:middle line:84% and I completely forgot to mention something that's worth 00:00:20.160 --> 00:00:22.360 align:middle line:84% knowing about, which is this thing that's 00:00:22.360 --> 00:00:25.800 align:middle line:84% shown here, which are known as equal loudness contours. 00:00:25.800 --> 00:00:30.040 align:middle line:84% And it's also sometimes called the Fletcher-Munson curves. 00:00:30.040 --> 00:00:34.160 align:middle line:84% And what this graph is showing is 00:00:34.160 --> 00:00:37.920 align:middle line:84% what the intensity of different frequencies has to be for them 00:00:37.920 --> 00:00:39.880 align:middle line:90% to sound equally loud. 00:00:39.880 --> 00:00:43.700 align:middle line:84% So the bottom graph is our detection thresholds. 00:00:43.700 --> 00:00:44.740 align:middle line:90% That's the audiogram. 00:00:44.740 --> 00:00:46.895 align:middle line:84% That's the thing that we've seen before. 00:00:46.895 --> 00:00:48.520 align:middle line:84% And that's the thing that kind of looks 00:00:48.520 --> 00:00:49.895 align:middle line:84% like more or less like a U shape. 00:00:49.895 --> 00:00:53.040 align:middle line:84% So your detection threshold is higher for low frequencies, 00:00:53.040 --> 00:00:56.200 align:middle line:84% and then rises again at higher frequencies, and it's best here 00:00:56.200 --> 00:00:57.880 align:middle line:90% in the mids. 00:00:57.880 --> 00:01:00.060 align:middle line:84% And then what each of these curves 00:01:00.060 --> 00:01:05.000 align:middle line:84% is showing is for higher intensities, 00:01:05.000 --> 00:01:07.060 align:middle line:84% it's like what the different frequencies 00:01:07.060 --> 00:01:08.280 align:middle line:90% would need to be set at. 00:01:08.280 --> 00:01:10.780 align:middle line:84% So everything that would fall on one of these red curves 00:01:10.780 --> 00:01:14.680 align:middle line:84% supposedly sounds equally loud to a person with normal hearing. 00:01:14.680 --> 00:01:20.900 align:middle line:84% And so you can see that as the overall intensity increases, 00:01:20.900 --> 00:01:23.900 align:middle line:84% you retain traces of the audiogram, which 00:01:23.900 --> 00:01:27.500 align:middle line:84% is to say lower frequencies and to some extent higher 00:01:27.500 --> 00:01:29.900 align:middle line:84% frequencies, they have to be higher in intensity 00:01:29.900 --> 00:01:34.060 align:middle line:84% to sound equally loud to a middle frequency. 00:01:34.060 --> 00:01:35.880 align:middle line:84% But that difference gets attenuated. 00:01:35.880 --> 00:01:40.220 align:middle line:84% So the U is kind of deeper down here when 00:01:40.220 --> 00:01:43.760 align:middle line:84% the level is lower compared to when the level is higher. 00:01:43.760 --> 00:01:47.620 align:middle line:84% And so what this means in practice, 00:01:47.620 --> 00:01:50.820 align:middle line:84% it actually has a lot of relevance for mixing music. 00:01:50.820 --> 00:01:54.260 align:middle line:84% So what it means is that a recording is not 00:01:54.260 --> 00:01:56.460 align:middle line:84% going to sound the same when you play it 00:01:56.460 --> 00:01:59.380 align:middle line:84% at a low level compared to when you played at a high level. 00:01:59.380 --> 00:02:04.260 align:middle line:84% And so in general, at a very low level, 00:02:04.260 --> 00:02:06.420 align:middle line:84% the mids are going to be dominant, 00:02:06.420 --> 00:02:09.680 align:middle line:84% because you can see that in order to make something 00:02:09.680 --> 00:02:12.220 align:middle line:84% equally loud, if it's a low frequency or a high frequency, 00:02:12.220 --> 00:02:14.095 align:middle line:84% you've got to increase the level of fair bit. 00:02:14.095 --> 00:02:18.400 align:middle line:84% So the mid frequencies will be fairly dominant at low levels, 00:02:18.400 --> 00:02:21.760 align:middle line:84% and then the highs and the lows become 00:02:21.760 --> 00:02:24.200 align:middle line:84% more prominent as the overall level of the recording 00:02:24.200 --> 00:02:25.020 align:middle line:90% increases. 00:02:25.020 --> 00:02:27.520 align:middle line:84% So equal loudness contours, and there 00:02:27.520 --> 00:02:31.160 align:middle line:84% are these interactions with intensity and with frequency. 00:02:31.160 --> 00:02:31.780 align:middle line:90% OK. 00:02:31.780 --> 00:02:32.280 align:middle line:90% Yeah? 00:02:32.280 --> 00:02:33.040 align:middle line:90% AUDIENCE: [INAUDIBLE] 00:02:33.040 --> 00:02:33.957 align:middle line:90% JOSH MCDERMOTT: I did. 00:02:33.957 --> 00:02:35.680 align:middle line:90% Yeah. 00:02:35.680 --> 00:02:39.700 align:middle line:84% So I just wanted to mention that. 00:02:39.700 --> 00:02:42.540 align:middle line:84% And the-- yeah, I think that's it. 00:02:42.540 --> 00:02:47.400 align:middle line:84% So let's start talking about auditory scene analysis. 00:02:47.400 --> 00:02:50.240 align:middle line:84% So we got started on this last time. 00:02:50.240 --> 00:02:52.920 align:middle line:84% We talked about how the problem is 00:02:52.920 --> 00:02:56.160 align:middle line:84% that there are very often multiple acoustic events that 00:02:56.160 --> 00:02:58.500 align:middle line:84% are happening in the world at the same time. 00:02:58.500 --> 00:03:01.300 align:middle line:84% Your ear receives the single sound waveform, 00:03:01.300 --> 00:03:05.460 align:middle line:84% and you typically will need to estimate or comprehend something 00:03:05.460 --> 00:03:09.140 align:middle line:84% about one or more of the individual sources. 00:03:09.140 --> 00:03:11.320 align:middle line:84% Classic example being the cocktail party problem, 00:03:11.320 --> 00:03:13.380 align:middle line:84% where you're trying to understand one person 00:03:13.380 --> 00:03:16.180 align:middle line:84% and there are other voices that are present at the same time. 00:03:16.180 --> 00:03:18.940 align:middle line:84% The problem is kind of interesting, 00:03:18.940 --> 00:03:21.400 align:middle line:84% like many perceptual problems, because it's ill posed. 00:03:21.400 --> 00:03:27.340 align:middle line:84% So you receive this one variable, which is the-- 00:03:27.340 --> 00:03:30.340 align:middle line:84% one signal, which is the waveform 00:03:30.340 --> 00:03:34.100 align:middle line:84% corresponding to the mixture of a bunch of signals. 00:03:34.100 --> 00:03:36.980 align:middle line:84% That one signal is generated by distinct source 00:03:36.980 --> 00:03:38.080 align:middle line:90% signals in the world. 00:03:38.080 --> 00:03:40.500 align:middle line:84% So it's akin to me giving you that equation 00:03:40.500 --> 00:03:43.460 align:middle line:90% and asking you to solve for x. 00:03:43.460 --> 00:03:45.877 align:middle line:84% And we talked about how the only way that you're 00:03:45.877 --> 00:03:47.460 align:middle line:84% able to solve these ill-posed problems 00:03:47.460 --> 00:03:50.900 align:middle line:84% is by leveraging the fact that the sources that occur 00:03:50.900 --> 00:03:52.580 align:middle line:90% in the world are not random. 00:03:52.580 --> 00:03:54.940 align:middle line:84% And in fact, they're very, very far from random. 00:03:54.940 --> 00:03:56.640 align:middle line:84% And so we believe that what has happened 00:03:56.640 --> 00:03:58.900 align:middle line:84% is that over evolution and development, 00:03:58.900 --> 00:04:02.960 align:middle line:84% your brain has internalized the statistical regularities 00:04:02.960 --> 00:04:04.040 align:middle line:90% of the world. 00:04:04.040 --> 00:04:07.360 align:middle line:84% And that's what enables you to solve these ill-posed problems. 00:04:07.360 --> 00:04:10.920 align:middle line:84% And then we talked a little bit about the idea of perception 00:04:10.920 --> 00:04:15.440 align:middle line:84% as unconscious inference, and how we can formalize that 00:04:15.440 --> 00:04:19.920 align:middle line:84% with Bayes' rule, where the task of perception 00:04:19.920 --> 00:04:22.200 align:middle line:84% can be considered that of finding 00:04:22.200 --> 00:04:26.680 align:middle line:84% the hypothesis about the world that is most probable given some 00:04:26.680 --> 00:04:29.720 align:middle line:84% observed signal O, and that can be 00:04:29.720 --> 00:04:32.960 align:middle line:84% decomposed into the prior and the likelihood. 00:04:32.960 --> 00:04:36.520 align:middle line:84% We talked a little bit about levels of analysis, 00:04:36.520 --> 00:04:41.288 align:middle line:84% and then discussed how the only way 00:04:41.288 --> 00:04:43.080 align:middle line:84% that you can solve these ill-posed problems 00:04:43.080 --> 00:04:44.400 align:middle line:90% is by having priors. 00:04:44.400 --> 00:04:48.300 align:middle line:84% The priors kind of capture your knowledge of regularities, 00:04:48.300 --> 00:04:50.760 align:middle line:84% in this case, of real world sounds, 00:04:50.760 --> 00:04:55.720 align:middle line:84% and how illusions can reveal the aspects of these priors. 00:04:55.720 --> 00:04:58.300 align:middle line:84% They reveal the constraints on sound generation internalized 00:04:58.300 --> 00:05:01.640 align:middle line:84% by the brain that make it possible to solve the problem. 00:05:01.640 --> 00:05:03.440 align:middle line:90% And so that's where we left off. 00:05:03.440 --> 00:05:06.380 align:middle line:84% And so what we're going to do today 00:05:06.380 --> 00:05:08.820 align:middle line:84% is go through a bunch of examples 00:05:08.820 --> 00:05:12.140 align:middle line:84% of these classic illusions that illustrate principles 00:05:12.140 --> 00:05:13.680 align:middle line:90% of auditory scene analysis. 00:05:13.680 --> 00:05:18.140 align:middle line:84% So properties of sound that seem to provide constraints 00:05:18.140 --> 00:05:19.680 align:middle line:90% on how we solve the problem. 00:05:19.680 --> 00:05:22.820 align:middle line:84% And I'll note that many of these could actually 00:05:22.820 --> 00:05:26.600 align:middle line:84% be useful jumping-off points for your illusion lab. 00:05:26.600 --> 00:05:29.120 align:middle line:84% So just be thinking about these as we go through. 00:05:29.120 --> 00:05:31.020 align:middle line:84% And if you're wondering about anything, 00:05:31.020 --> 00:05:33.900 align:middle line:84% write that down as something that maybe you could investigate 00:05:33.900 --> 00:05:35.140 align:middle line:90% for the illusion lab. 00:05:35.140 --> 00:05:37.820 align:middle line:90% 00:05:37.820 --> 00:05:40.940 align:middle line:84% So oftentimes these phenomena are 00:05:40.940 --> 00:05:44.000 align:middle line:84% discussed in the context of what are called grouping cues. 00:05:44.000 --> 00:05:47.740 align:middle line:84% So we've talked about cues as being properties 00:05:47.740 --> 00:05:51.100 align:middle line:84% of a stimulus that are informative about some aspect 00:05:51.100 --> 00:05:51.800 align:middle line:90% of the world. 00:05:51.800 --> 00:05:55.260 align:middle line:84% And so when we're talking about sound localization, 00:05:55.260 --> 00:05:56.860 align:middle line:84% we talked about localization cues. 00:05:56.860 --> 00:05:58.800 align:middle line:84% Now we're talking about auditory scene 00:05:58.800 --> 00:06:01.940 align:middle line:84% analysis, which you can think of as a problem of grouping. 00:06:01.940 --> 00:06:05.420 align:middle line:84% So you get this sound signal here. 00:06:05.420 --> 00:06:08.240 align:middle line:84% It's got all these little bits of acoustic energy scattered 00:06:08.240 --> 00:06:10.280 align:middle line:84% across frequency and across time. 00:06:10.280 --> 00:06:13.103 align:middle line:84% We often will represent it with a picture like this, 00:06:13.103 --> 00:06:14.520 align:middle line:84% because we think that this picture 00:06:14.520 --> 00:06:17.020 align:middle line:84% kind of captures what your ear is sending to your brain. 00:06:17.020 --> 00:06:19.680 align:middle line:84% The ear, you can think of that as doing a frequency 00:06:19.680 --> 00:06:21.080 align:middle line:90% decomposition. 00:06:21.080 --> 00:06:24.280 align:middle line:84% And then there are responses at different frequencies over time. 00:06:24.280 --> 00:06:27.200 align:middle line:84% So one way to think of the auditory scene analysis problem 00:06:27.200 --> 00:06:30.960 align:middle line:84% is as a problem of grouping the bits of energy 00:06:30.960 --> 00:06:35.000 align:middle line:84% into sources that would have a common cause in the world. 00:06:35.000 --> 00:06:37.440 align:middle line:84% And so we'll talk about grouping cues. 00:06:37.440 --> 00:06:39.600 align:middle line:84% And what is typically meant by that 00:06:39.600 --> 00:06:41.800 align:middle line:84% is a stimulus property that is informative 00:06:41.800 --> 00:06:44.440 align:middle line:84% about the organization of sound into sources. 00:06:44.440 --> 00:06:48.640 align:middle line:84% And so some standard examples would be common onset 00:06:48.640 --> 00:06:50.300 align:middle line:84% and offset, which you can see here, 00:06:50.300 --> 00:06:52.100 align:middle line:84% and we'll see some examples in a second. 00:06:52.100 --> 00:06:55.428 align:middle line:84% And then harmonicity, the fact that some sounds in the world 00:06:55.428 --> 00:06:57.720 align:middle line:84% tend to have frequencies that are harmonically related. 00:06:57.720 --> 00:07:00.640 align:middle line:90% 00:07:00.640 --> 00:07:04.140 align:middle line:84% So this is an initial kind of classic demonstration 00:07:04.140 --> 00:07:07.620 align:middle line:84% that the brain seems to interpret abrupt amplitude 00:07:07.620 --> 00:07:11.220 align:middle line:84% changes as the onset or offset of a distinct source 00:07:11.220 --> 00:07:12.980 align:middle line:90% in the world. 00:07:12.980 --> 00:07:15.500 align:middle line:84% By contrast, gradual amplitude changes 00:07:15.500 --> 00:07:17.460 align:middle line:84% are instead interpreted as a single source 00:07:17.460 --> 00:07:18.573 align:middle line:90% that changes over time. 00:07:18.573 --> 00:07:20.240 align:middle line:84% So this is a super simple demonstration. 00:07:20.240 --> 00:07:21.615 align:middle line:84% So what's going to happen here is 00:07:21.615 --> 00:07:23.420 align:middle line:84% we're going to have a noise signal. 00:07:23.420 --> 00:07:25.500 align:middle line:84% And then the amplitude of the noise signal 00:07:25.500 --> 00:07:28.060 align:middle line:90% will change in one of two ways. 00:07:28.060 --> 00:07:31.720 align:middle line:84% In one case, it will change gradually. 00:07:31.720 --> 00:07:33.700 align:middle line:84% So this is a graph that shows the amplitude 00:07:33.700 --> 00:07:35.260 align:middle line:90% as a function of time. 00:07:35.260 --> 00:07:38.560 align:middle line:84% So in one case, the amplitude will start at a value, 00:07:38.560 --> 00:07:40.820 align:middle line:84% and then it'll ramp up, and then it'll ramp down. 00:07:40.820 --> 00:07:43.842 align:middle line:84% And in another case, it'll kind of shoot up very abruptly. 00:07:43.842 --> 00:07:45.300 align:middle line:84% But in both cases, essentially what 00:07:45.300 --> 00:07:48.380 align:middle line:84% happened is there's a noise signal and a little volume knob, 00:07:48.380 --> 00:07:50.160 align:middle line:84% and that volume knob is just getting 00:07:50.160 --> 00:07:53.400 align:middle line:90% manipulated in one of two ways. 00:07:53.400 --> 00:07:55.037 align:middle line:84% But what you will see is that what 00:07:55.037 --> 00:07:57.120 align:middle line:84% you hear when you listen to this is actually quite 00:07:57.120 --> 00:07:58.800 align:middle line:90% different in the two cases. 00:07:58.800 --> 00:08:00.342 align:middle line:84% In this case, it'll really just sound 00:08:00.342 --> 00:08:02.092 align:middle line:84% like there's kind of a single thing that's 00:08:02.092 --> 00:08:03.780 align:middle line:84% kind of waxing and waning in amplitude. 00:08:03.780 --> 00:08:06.400 align:middle line:84% You might even hear it as kind of coming closer to you 00:08:06.400 --> 00:08:07.500 align:middle line:90% or further from you. 00:08:07.500 --> 00:08:11.200 align:middle line:84% Whereas in this case, you'll hear a new sound source 00:08:11.200 --> 00:08:14.300 align:middle line:84% when you get this abrupt increase in amplitude. 00:08:14.300 --> 00:08:16.080 align:middle line:90% Let's check it out. 00:08:16.080 --> 00:08:16.880 align:middle line:90% [AUDIO PLAYBACK] 00:08:16.880 --> 00:08:20.800 align:middle line:84% - Homophonic continuity and rise time. 00:08:20.800 --> 00:08:24.400 align:middle line:84% A noise repeatedly rises and falls in intensity. 00:08:24.400 --> 00:08:28.320 align:middle line:84% If the changes occur slowly, you hear a single sound 00:08:28.320 --> 00:08:31.440 align:middle line:90% waxing and waning in intensity. 00:08:31.440 --> 00:08:35.000 align:middle line:84% If the changes are abrupt, you hear two sounds, 00:08:35.000 --> 00:08:39.440 align:middle line:84% a soft one continuing unchanged, and an additional noise burst 00:08:39.440 --> 00:08:41.880 align:middle line:90% added to it periodically. 00:08:41.880 --> 00:08:47.052 align:middle line:84% First we hear the gradual changes, then the abrupt ones. 00:08:47.052 --> 00:08:50.454 align:middle line:90% [PULSING WHITE NOISE] 00:08:50.454 --> 00:08:55.314 align:middle line:90% 00:08:55.314 --> 00:08:58.716 align:middle line:90% [CLICKING WHITE NOISE] 00:08:58.716 --> 00:09:02.604 align:middle line:90% 00:09:02.604 --> 00:09:03.427 align:middle line:90% [END PLAYBACK] 00:09:03.427 --> 00:09:04.510 align:middle line:90% JOSH MCDERMOTT: All right. 00:09:04.510 --> 00:09:06.730 align:middle line:84% So what you probably heard in the second case 00:09:06.730 --> 00:09:10.650 align:middle line:84% is that it sounds like there's a low-intensity noise that's 00:09:10.650 --> 00:09:12.610 align:middle line:84% kind of just on continuously, and then 00:09:12.610 --> 00:09:16.630 align:middle line:90% these pulses of louder noise. 00:09:16.630 --> 00:09:18.890 align:middle line:84% Now in actuality, there's just one noise signal 00:09:18.890 --> 00:09:21.350 align:middle line:84% where the amplitude gets modulated, whereas in this case, 00:09:21.350 --> 00:09:26.070 align:middle line:84% it kind of sounds like one thing that's changing in intensity. 00:09:26.070 --> 00:09:29.210 align:middle line:84% So these very sudden changes in intensity 00:09:29.210 --> 00:09:31.930 align:middle line:84% are interpreted by your brain as the presence 00:09:31.930 --> 00:09:34.930 align:middle line:90% of a distinct source. 00:09:34.930 --> 00:09:37.850 align:middle line:84% And this can also happen kind of at the level 00:09:37.850 --> 00:09:40.490 align:middle line:84% of individual frequency channels. 00:09:40.490 --> 00:09:42.610 align:middle line:84% So sudden onsets of new frequencies 00:09:42.610 --> 00:09:44.888 align:middle line:84% are typically interpreted as new separate events, 00:09:44.888 --> 00:09:46.430 align:middle line:84% whereas frequencies that were present 00:09:46.430 --> 00:09:48.190 align:middle line:90% before are heard as continuing. 00:09:48.190 --> 00:09:52.190 align:middle line:84% And this is often referred to as the old-plus-new heuristic. 00:09:52.190 --> 00:09:55.910 align:middle line:84% So this is a diagram that shows how this works. 00:09:55.910 --> 00:09:59.910 align:middle line:84% So you're going to compare two stimuli. 00:09:59.910 --> 00:10:02.742 align:middle line:84% So previous graph showed amplitude versus time, 00:10:02.742 --> 00:10:04.450 align:middle line:84% whereas these are schematic spectrograms. 00:10:04.450 --> 00:10:06.670 align:middle line:84% So now we have frequency, and the gray level 00:10:06.670 --> 00:10:10.030 align:middle line:84% is kind of supposed to indicate the intensity at that frequency. 00:10:10.030 --> 00:10:12.530 align:middle line:90% And the x-axis is time. 00:10:12.530 --> 00:10:15.990 align:middle line:84% So in this case, there's a stimulus that is constructed 00:10:15.990 --> 00:10:18.610 align:middle line:84% by concatenating low-pass noise-- 00:10:18.610 --> 00:10:21.390 align:middle line:84% so it's low-pass because it contains low frequencies-- 00:10:21.390 --> 00:10:23.250 align:middle line:90% with a broadband noise. 00:10:23.250 --> 00:10:28.010 align:middle line:84% That's the B. So low-pass noise, broadband noise, low-pass noise. 00:10:28.010 --> 00:10:31.270 align:middle line:84% Whereas this one, you have high-pass noise alternated 00:10:31.270 --> 00:10:33.370 align:middle line:90% with broadband noise. 00:10:33.370 --> 00:10:36.830 align:middle line:84% So the idea here is that the broadband noise in the two 00:10:36.830 --> 00:10:39.270 align:middle line:90% stimuli is the same. 00:10:39.270 --> 00:10:43.070 align:middle line:84% But the perceptual organization that you 00:10:43.070 --> 00:10:46.150 align:middle line:84% will hear in the stimulus is very different. 00:10:46.150 --> 00:10:48.090 align:middle line:84% So in this case, even though this 00:10:48.090 --> 00:10:51.690 align:middle line:84% was constructed by alternating low-pass and broadband noise, 00:10:51.690 --> 00:10:53.730 align:middle line:84% you're going to interpret the stimulus 00:10:53.730 --> 00:10:56.290 align:middle line:84% as a single low-pass noise that's 00:10:56.290 --> 00:11:00.010 align:middle line:84% continuous with these intermittent high-frequency 00:11:00.010 --> 00:11:01.570 align:middle line:90% pulses of noise. 00:11:01.570 --> 00:11:03.370 align:middle line:84% Whereas in this case, you'll hear 00:11:03.370 --> 00:11:06.690 align:middle line:84% a continuous high-pass noise with these intermittent 00:11:06.690 --> 00:11:10.270 align:middle line:90% low-frequency bursts of noise 00:11:10.270 --> 00:11:13.490 align:middle line:84% So same broadband noise will get taken apart 00:11:13.490 --> 00:11:15.490 align:middle line:84% by your auditory system, or explained 00:11:15.490 --> 00:11:19.610 align:middle line:84% by your auditory system, in two different ways 00:11:19.610 --> 00:11:22.950 align:middle line:84% I'm going to let you see the spectrogram while this plays. 00:11:22.950 --> 00:11:24.650 align:middle line:90% And let's listen to it. 00:11:24.650 --> 00:11:25.450 align:middle line:90% [AUDIO PLAYBACK] 00:11:25.450 --> 00:11:29.330 align:middle line:84% - Demonstration 34, capturing a low band of noise 00:11:29.330 --> 00:11:33.890 align:middle line:84% from a wider band, leaving a high band as a residual. 00:11:33.890 --> 00:11:37.530 align:middle line:84% Please refer to the figure in the booklet. 00:11:37.530 --> 00:11:40.570 align:middle line:84% A short band of noise, B, is alternated 00:11:40.570 --> 00:11:45.830 align:middle line:84% with a longer band that contains only B's lower frequencies. 00:11:45.830 --> 00:11:48.750 align:middle line:84% These frequencies group with the low ones of B 00:11:48.750 --> 00:11:52.750 align:middle line:84% to form one continuous low noise, and the high components 00:11:52.750 --> 00:11:55.830 align:middle line:84% of B are heard as an intermittent high sound. 00:11:55.830 --> 00:11:59.470 align:middle line:84% This is the residual formed when the lower frequencies of B 00:11:59.470 --> 00:12:01.470 align:middle line:90% are captured. 00:12:01.470 --> 00:12:03.310 align:middle line:84% Similarly, the high frequencies of B 00:12:03.310 --> 00:12:06.670 align:middle line:84% can be captured by a high noise, leaving a low noise 00:12:06.670 --> 00:12:08.550 align:middle line:90% as the residual. 00:12:08.550 --> 00:12:12.430 align:middle line:84% These two cases are presented twice in alternation. 00:12:12.430 --> 00:12:17.660 align:middle line:84% Remember that B is physically identical in the two cases. 00:12:17.660 --> 00:12:21.620 align:middle line:90% [PULSING WHITE NOISE] 00:12:21.620 --> 00:12:26.075 align:middle line:90% [CLICKING WHITE NOISE] 00:12:26.075 --> 00:12:30.530 align:middle line:90% [PULSING WHITE NOISE] 00:12:30.530 --> 00:12:33.500 align:middle line:90% [CLICKING WHITE NOISE] 00:12:33.500 --> 00:12:34.518 align:middle line:90% [END PLAYBACK] 00:12:34.518 --> 00:12:35.310 align:middle line:90% JOSH MCDERMOTT: OK. 00:12:35.310 --> 00:12:36.830 align:middle line:90% Everybody get that? 00:12:36.830 --> 00:12:37.910 align:middle line:90% All right. 00:12:37.910 --> 00:12:40.150 align:middle line:84% So what it sounds like subjectively 00:12:40.150 --> 00:12:43.260 align:middle line:84% is completely different in the two cases. 00:12:43.260 --> 00:12:45.510 align:middle line:84% And at the moment when those intermittent noises play, 00:12:45.510 --> 00:12:47.730 align:middle line:90% the stimulus is identical. 00:12:47.730 --> 00:12:52.250 align:middle line:84% And that's because your brain is interpreting that 00:12:52.250 --> 00:12:55.490 align:middle line:84% as being two different combinations of sources. 00:12:55.490 --> 00:13:03.450 align:middle line:90% 00:13:03.450 --> 00:13:05.690 align:middle line:84% So there's kind of a related phenomena where-- 00:13:05.690 --> 00:13:11.250 align:middle line:84% so in those examples, we were just looking at these cases 00:13:11.250 --> 00:13:13.950 align:middle line:84% where you would really describe the stimulus in terms of onsets. 00:13:13.950 --> 00:13:16.810 align:middle line:84% It's like these frequencies, or the sound, kind of abruptly 00:13:16.810 --> 00:13:17.570 align:middle line:90% starts. 00:13:17.570 --> 00:13:23.530 align:middle line:84% But it's also often pretty common for the modulation 00:13:23.530 --> 00:13:25.210 align:middle line:90% to be more gradual. 00:13:25.210 --> 00:13:29.050 align:middle line:84% And we call that modulation, or amplitude modulation. 00:13:29.050 --> 00:13:32.650 align:middle line:84% And so in particular, there's a pretty well-known effect 00:13:32.650 --> 00:13:35.730 align:middle line:84% that seems to indicate that common modulation 00:13:35.730 --> 00:13:38.933 align:middle line:84% across frequencies tends to promote grouping. 00:13:38.933 --> 00:13:39.850 align:middle line:90% And this is actually-- 00:13:39.850 --> 00:13:42.810 align:middle line:84% I like featuring this particular example, 00:13:42.810 --> 00:13:46.477 align:middle line:84% because it kind of builds on the masking experiments 00:13:46.477 --> 00:13:48.810 align:middle line:84% that we talked about a little bit earlier in the course. 00:13:48.810 --> 00:13:52.550 align:middle line:84% So in this particular phenomenon, 00:13:52.550 --> 00:13:56.150 align:middle line:84% your task is to detect a pure tone in noise, 00:13:56.150 --> 00:13:58.310 align:middle line:84% like one of those classical masking experiments. 00:13:58.310 --> 00:14:00.870 align:middle line:84% And we're going to measure your threshold for detecting 00:14:00.870 --> 00:14:02.110 align:middle line:90% the tone in the noise. 00:14:02.110 --> 00:14:06.350 align:middle line:84% And we will, like in the classic band widening 00:14:06.350 --> 00:14:09.310 align:middle line:84% experiment of Fletcher, we're going to manipulate 00:14:09.310 --> 00:14:11.670 align:middle line:90% the bandwidth of the noise. 00:14:11.670 --> 00:14:13.790 align:middle line:84% But the catch here is that unlike the classical 00:14:13.790 --> 00:14:15.910 align:middle line:84% experiments, in this experiment, the noise 00:14:15.910 --> 00:14:18.790 align:middle line:84% can either be fully random, as in the classical experiments, 00:14:18.790 --> 00:14:20.570 align:middle line:90% or comodulated. 00:14:20.570 --> 00:14:23.910 align:middle line:84% That means multiplied by a common envelope. 00:14:23.910 --> 00:14:26.410 align:middle line:84% So that's what is schematized here. 00:14:26.410 --> 00:14:31.150 align:middle line:84% The idea is that we've got all these different bands of noise, 00:14:31.150 --> 00:14:33.070 align:middle line:90% and they can be comodulated. 00:14:33.070 --> 00:14:36.510 align:middle line:84% And the central phenomena that we will witness 00:14:36.510 --> 00:14:37.990 align:middle line:84% is that this kind of strange thing 00:14:37.990 --> 00:14:41.090 align:middle line:84% happens, where this is supposed to be the case, where there's 00:14:41.090 --> 00:14:45.610 align:middle line:84% just a fairly narrow band of noise around the tone. 00:14:45.610 --> 00:14:47.530 align:middle line:84% You could, in principle, adjust the amplitudes 00:14:47.530 --> 00:14:50.070 align:middle line:84% of the tone and the noise so that you can't detect the tone. 00:14:50.070 --> 00:14:51.330 align:middle line:90% That's why the person is sad. 00:14:51.330 --> 00:14:52.510 align:middle line:90% It means they can't hear it. 00:14:52.510 --> 00:14:55.530 align:middle line:84% And the phenomenon here that we will witness 00:14:55.530 --> 00:14:58.490 align:middle line:84% is that we can add these additional bands of noise 00:14:58.490 --> 00:14:59.710 align:middle line:90% at adjacent frequencies. 00:14:59.710 --> 00:15:02.410 align:middle line:84% And if they're comodulated, that will actually 00:15:02.410 --> 00:15:03.510 align:middle line:90% lower your threshold. 00:15:03.510 --> 00:15:04.890 align:middle line:90% That's why the person's happy. 00:15:04.890 --> 00:15:08.270 align:middle line:90% They can now hear the tone. 00:15:08.270 --> 00:15:10.250 align:middle line:84% So when the noise is comodulated, 00:15:10.250 --> 00:15:14.210 align:middle line:84% the thresholds are better when noise that is far from the tone 00:15:14.210 --> 00:15:15.530 align:middle line:90% is added. 00:15:15.530 --> 00:15:18.023 align:middle line:84% So here's the result of the experiment, 00:15:18.023 --> 00:15:19.690 align:middle line:84% and then we'll listen to a demo, and you 00:15:19.690 --> 00:15:20.982 align:middle line:90% can see whether you believe it. 00:15:20.982 --> 00:15:23.630 align:middle line:84% So this is same kind of graph that we saw earlier. 00:15:23.630 --> 00:15:27.250 align:middle line:84% So we got a threshold on the y-axis, 00:15:27.250 --> 00:15:30.270 align:middle line:84% and the masker bandwidth on the x-axis. 00:15:30.270 --> 00:15:35.150 align:middle line:84% So the graph that has the R symbol, that stands for random. 00:15:35.150 --> 00:15:37.630 align:middle line:84% And so that's the kind of classical band-widening result. 00:15:37.630 --> 00:15:38.590 align:middle line:90% What does it show? 00:15:38.590 --> 00:15:41.450 align:middle line:84% It shows that as we increase the bandwidth of the masker, 00:15:41.450 --> 00:15:42.892 align:middle line:90% the thresholds kind go up. 00:15:42.892 --> 00:15:44.350 align:middle line:84% And then you hit the critical band, 00:15:44.350 --> 00:15:46.110 align:middle line:90% and they kind of level off. 00:15:46.110 --> 00:15:50.790 align:middle line:84% That's the thing that we looked at a few lectures ago. 00:15:50.790 --> 00:15:53.190 align:middle line:84% So the new thing here is what happens 00:15:53.190 --> 00:15:54.850 align:middle line:90% when the noise is comodulated. 00:15:54.850 --> 00:15:57.213 align:middle line:90% And that's the M symbols. 00:15:57.213 --> 00:15:59.630 align:middle line:84% And so what happens there is actually different and pretty 00:15:59.630 --> 00:16:03.990 align:middle line:84% interesting, which is initially, the threshold goes up, 00:16:03.990 --> 00:16:05.910 align:middle line:84% but then it goes back down again. 00:16:05.910 --> 00:16:08.110 align:middle line:84% And so you end up with a pretty big difference 00:16:08.110 --> 00:16:09.950 align:middle line:84% between the thresholds that you get 00:16:09.950 --> 00:16:13.650 align:middle line:84% when the noise is fully random versus when it's modulated. 00:16:13.650 --> 00:16:15.390 align:middle line:90% Let's see if we can hear this. 00:16:15.390 --> 00:16:17.690 align:middle line:84% So this is what happens with purely random noise. 00:16:17.690 --> 00:16:19.470 align:middle line:84% So what you're going to hear in this demo 00:16:19.470 --> 00:16:23.710 align:middle line:84% is there's a noise signal and then a sequence of tones. 00:16:23.710 --> 00:16:29.010 align:middle line:84% And I think in this demo, the tones increase in level. 00:16:29.010 --> 00:16:31.370 align:middle line:84% And so you're supposed to notice when you can hear them. 00:16:31.370 --> 00:16:33.430 align:middle line:84% And it's possible that you won't be able to hear any of them 00:16:33.430 --> 00:16:35.050 align:middle line:84% in this particular case, but let's see. 00:16:35.050 --> 00:16:35.717 align:middle line:90% [AUDIO PLAYBACK] 00:16:35.717 --> 00:16:38.292 align:middle line:90% [PULSING WHITE NOISE] 00:16:38.292 --> 00:16:39.210 align:middle line:90% [END PLAYBACK] 00:16:39.210 --> 00:16:41.210 align:middle line:84% JOSH MCDERMOTT: I heard the last two. 00:16:41.210 --> 00:16:41.863 align:middle line:90% Try it again. 00:16:41.863 --> 00:16:42.530 align:middle line:90% [AUDIO PLAYBACK] 00:16:42.530 --> 00:16:45.488 align:middle line:90% [PULSING WHITE NOISE] 00:16:45.488 --> 00:16:46.970 align:middle line:90% [END PLAYBACK] 00:16:46.970 --> 00:16:48.810 align:middle line:84% JOSH MCDERMOTT: Can you hear a tone there? 00:16:48.810 --> 00:16:49.450 align:middle line:90% Not sure? 00:16:49.450 --> 00:16:49.950 align:middle line:90% All right. 00:16:49.950 --> 00:16:50.670 align:middle line:90% That's OK. 00:16:50.670 --> 00:16:52.030 align:middle line:84% Well, let's see if you can hear it here. 00:16:52.030 --> 00:16:52.697 align:middle line:90% [AUDIO PLAYBACK] 00:16:52.697 --> 00:16:54.518 align:middle line:90% [LOWER FREQUENCY PULSING] 00:16:54.518 --> 00:16:56.009 align:middle line:90% [TONE BEEPING] 00:16:56.009 --> 00:16:57.010 align:middle line:90% [END PLAYBACK] 00:16:57.010 --> 00:16:58.350 align:middle line:84% JOSH MCDERMOTT: Did you hear a tone at the end? 00:16:58.350 --> 00:16:58.850 align:middle line:90% OK. 00:16:58.850 --> 00:17:00.650 align:middle line:90% So that's this effect. 00:17:00.650 --> 00:17:03.390 align:middle line:84% So when that noise is comodulated, 00:17:03.390 --> 00:17:05.883 align:middle line:84% it becomes easier to detect the tone, 00:17:05.883 --> 00:17:07.050 align:middle line:90% and your threshold is lower. 00:17:07.050 --> 00:17:09.869 align:middle line:84% In this case, by about 10 dB, so it's a pretty big effect. 00:17:09.869 --> 00:17:12.510 align:middle line:84% So this is called comodulation masking release. 00:17:12.510 --> 00:17:16.050 align:middle line:84% It's a super well-known effect in the world of psychoacoustics. 00:17:16.050 --> 00:17:19.010 align:middle line:84% And it is considered to be evidence 00:17:19.010 --> 00:17:20.990 align:middle line:90% for grouping via comodulation. 00:17:20.990 --> 00:17:24.530 align:middle line:84% So the idea being that the fact that the noise has 00:17:24.530 --> 00:17:28.010 align:middle line:84% the same envelope at all these different frequency channels 00:17:28.010 --> 00:17:30.810 align:middle line:84% helps you to separate the tone from the noise 00:17:30.810 --> 00:17:35.568 align:middle line:84% because it's helping you distinguish 00:17:35.568 --> 00:17:37.110 align:middle line:84% which parts of those signals are tone 00:17:37.110 --> 00:17:39.270 align:middle line:84% and which parts are noise, which in turn makes it easier 00:17:39.270 --> 00:17:41.030 align:middle line:84% for you to tell whether the tone is there. 00:17:41.030 --> 00:17:45.950 align:middle line:90% 00:17:45.950 --> 00:17:48.350 align:middle line:84% Another very basic property of natural sounds 00:17:48.350 --> 00:17:50.630 align:middle line:84% that we have already talked about at some length 00:17:50.630 --> 00:17:52.290 align:middle line:90% is harmonicity. 00:17:52.290 --> 00:17:54.910 align:middle line:84% So sounds in the world are often harmonic. 00:17:54.910 --> 00:17:57.390 align:middle line:84% And the brain seems to assume that frequencies 00:17:57.390 --> 00:17:59.070 align:middle line:84% that are harmonically related belong 00:17:59.070 --> 00:18:00.770 align:middle line:90% to the same acoustic event. 00:18:00.770 --> 00:18:02.270 align:middle line:84% And one piece of evidence for this 00:18:02.270 --> 00:18:06.070 align:middle line:84% is that when one frequency out of a bunch 00:18:06.070 --> 00:18:08.730 align:middle line:84% is not harmonically related to a bunch of others, 00:18:08.730 --> 00:18:10.710 align:middle line:84% it tends to segregate perceptually. 00:18:10.710 --> 00:18:13.110 align:middle line:84% And this is often known as harmonic mistuning. 00:18:13.110 --> 00:18:16.858 align:middle line:84% And so we'll hear a demonstration 00:18:16.858 --> 00:18:18.650 align:middle line:84% where you can listen for this for yourself. 00:18:18.650 --> 00:18:20.608 align:middle line:84% And so what's going to happen here-- so, again, 00:18:20.608 --> 00:18:22.590 align:middle line:84% these are schematic spectrograms. 00:18:22.590 --> 00:18:26.070 align:middle line:84% So this is a complex tone that has harmonic frequencies. 00:18:26.070 --> 00:18:30.590 align:middle line:84% And all of the harmonics except the third one 00:18:30.590 --> 00:18:34.050 align:middle line:84% here are going to shift down over successive cycles 00:18:34.050 --> 00:18:34.990 align:middle line:90% of the stimulus. 00:18:34.990 --> 00:18:38.850 align:middle line:84% And so this third harmonic is going 00:18:38.850 --> 00:18:42.670 align:middle line:84% to stop being harmonically related to all the others. 00:18:42.670 --> 00:18:46.530 align:middle line:84% And so at the beginning, this is going to sound like one tone, 00:18:46.530 --> 00:18:48.388 align:middle line:90% like one thing. 00:18:48.388 --> 00:18:49.930 align:middle line:84% But over time, what's going to happen 00:18:49.930 --> 00:18:52.370 align:middle line:84% is you'll actually start to hear two tones. 00:18:52.370 --> 00:18:54.650 align:middle line:84% So it's all just a bunch of frequencies, 00:18:54.650 --> 00:18:57.190 align:middle line:84% and they all start and stop at the same time, 00:18:57.190 --> 00:19:00.170 align:middle line:84% but the deviation from these harmonic frequency relations 00:19:00.170 --> 00:19:03.170 align:middle line:84% is going to cause you to hear two things. 00:19:03.170 --> 00:19:05.070 align:middle line:90% Let's check it out. 00:19:05.070 --> 00:19:07.170 align:middle line:90% [AUDIO PLAYBACK] 00:19:07.170 --> 00:19:10.107 align:middle line:90% - Demonstration 18, isolation-- 00:19:10.107 --> 00:19:10.690 align:middle line:90% [END PLAYBACK] 00:19:10.690 --> 00:19:12.690 align:middle line:84% JOSH MCDERMOTT: In fact, I what I'm going to do, 00:19:12.690 --> 00:19:15.590 align:middle line:84% I'm going to play the intro and let you look at the picture. 00:19:15.590 --> 00:19:16.263 align:middle line:90% It seems better. 00:19:16.263 --> 00:19:16.930 align:middle line:90% [AUDIO PLAYBACK] 00:19:16.930 --> 00:19:21.090 align:middle line:84% - Demonstration 18, isolation of a frequency component 00:19:21.090 --> 00:19:23.290 align:middle line:90% based on mistuning. 00:19:23.290 --> 00:19:26.930 align:middle line:84% You are to listen for the third harmonic of a complex tone. 00:19:26.930 --> 00:19:30.650 align:middle line:84% First, this component is played alone as a standard. 00:19:30.650 --> 00:19:33.070 align:middle line:84% Then over a series of repetitions, 00:19:33.070 --> 00:19:35.510 align:middle line:84% it remains at a constant frequency, 00:19:35.510 --> 00:19:38.510 align:middle line:84% while the rest of the components are gradually lowered 00:19:38.510 --> 00:19:42.446 align:middle line:90% as a group in steps of 1%. 00:19:42.446 --> 00:19:43.670 align:middle line:90% [HIGH-FREQUENCY BEEP] 00:19:43.670 --> 00:19:44.753 align:middle line:90% JOSH MCDERMOTT: All right. 00:19:44.753 --> 00:19:48.620 align:middle line:90% Now, let's see if I can-- 00:19:48.620 --> 00:19:51.590 align:middle line:90% [MID-FREQUENCY BEEP] 00:19:51.590 --> 00:19:52.770 align:middle line:90% Sorry, I messed it up. 00:19:52.770 --> 00:19:54.550 align:middle line:90% [HIGH-FREQUENCY BEEP] 00:19:54.550 --> 00:19:56.950 align:middle line:84% My technique here is questionable. 00:19:56.950 --> 00:19:57.710 align:middle line:90% Let's see. 00:19:57.710 --> 00:20:01.085 align:middle line:90% - The group in steps of 1% 00:20:01.085 --> 00:20:04.550 align:middle line:90% [HIGH-FREQUENCY BEEPING] 00:20:04.550 --> 00:20:08.015 align:middle line:90% [MID-FREQUENCY BEEPING] 00:20:08.015 --> 00:20:09.995 align:middle line:90% 00:20:09.995 --> 00:20:13.460 align:middle line:90% [DUAL TONES BEEPING] 00:20:13.460 --> 00:20:16.652 align:middle line:90% 00:20:16.652 --> 00:20:19.110 align:middle line:84% JOSH MCDERMOTT: Raise your hand if you're hearing two tones 00:20:19.110 --> 00:20:20.090 align:middle line:90% at this point. 00:20:20.090 --> 00:20:20.590 align:middle line:90% Yeah. 00:20:20.590 --> 00:20:21.650 align:middle line:90% So almost everybody. 00:20:21.650 --> 00:20:23.130 align:middle line:90% So this works. 00:20:23.130 --> 00:20:32.630 align:middle line:84% [DUAL TONES BEEPING LOWER IN FREQUENCY] 00:20:32.630 --> 00:20:35.170 align:middle line:84% - Now, after two more presentations of the target 00:20:35.170 --> 00:20:38.130 align:middle line:84% component alone, the other partials of the tone 00:20:38.130 --> 00:20:42.850 align:middle line:84% are raised in steps of 1% until the target component 00:20:42.850 --> 00:20:47.690 align:middle line:84% loses its identity and once more forms part of the complex tone. 00:20:47.690 --> 00:20:49.970 align:middle line:84% JOSH MCDERMOTT: So what this is going to show-- so now 00:20:49.970 --> 00:20:53.530 align:middle line:84% you know that there's this separate thing there. 00:20:53.530 --> 00:20:56.530 align:middle line:84% That third harmonic is going to sound like it's a second tone. 00:20:56.530 --> 00:20:59.730 align:middle line:84% But as it approaches being harmonically related 00:20:59.730 --> 00:21:03.350 align:middle line:84% to everything else, it's going to fuse back into them. 00:21:03.350 --> 00:21:05.183 align:middle line:84% And even though you know it's there, 00:21:05.183 --> 00:21:07.350 align:middle line:84% you're going to stop hearing it as a separate thing. 00:21:07.350 --> 00:21:10.850 align:middle line:90% 00:21:10.850 --> 00:21:14.320 align:middle line:90% [HIGH FREQUENCY BEEPING] 00:21:14.320 --> 00:21:18.240 align:middle line:90% [DUAL TONES BEEPING] 00:21:18.240 --> 00:21:21.670 align:middle line:90% 00:21:21.670 --> 00:21:35.297 align:middle line:90% [HIGHER FREQUENCY DUAL TONES] 00:21:35.297 --> 00:21:35.880 align:middle line:90% [END PLAYBACK] 00:21:35.880 --> 00:21:38.505 align:middle line:84% JOSH MCDERMOTT: So at the very end there, you probably lost it. 00:21:38.505 --> 00:21:40.550 align:middle line:84% Raise your hand if you lost it at the end. 00:21:40.550 --> 00:21:41.330 align:middle line:90% Yeah, most of you. 00:21:41.330 --> 00:21:43.690 align:middle line:90% OK. 00:21:43.690 --> 00:21:45.210 align:middle line:90% So that's harmonic mistuning. 00:21:45.210 --> 00:21:49.550 align:middle line:90% 00:21:49.550 --> 00:21:52.670 align:middle line:84% One way to measure this is to ask people 00:21:52.670 --> 00:21:55.390 align:middle line:84% whether the harmonic is mistuned or not. 00:21:55.390 --> 00:21:57.910 align:middle line:84% And the idea is that the way that you actually 00:21:57.910 --> 00:22:02.030 align:middle line:84% perform that judgment is by hearing out whether it sounds 00:22:02.030 --> 00:22:03.670 align:middle line:90% like there's a separate tone. 00:22:03.670 --> 00:22:05.970 align:middle line:84% And so people are actually pretty sensitive to this, 00:22:05.970 --> 00:22:09.430 align:middle line:84% so you can mistune the frequency of that harmonic by 2% 00:22:09.430 --> 00:22:12.850 align:middle line:90% and measure thresholds for that. 00:22:12.850 --> 00:22:14.830 align:middle line:84% So this is just four different people. 00:22:14.830 --> 00:22:16.210 align:middle line:90% That's the threshold. 00:22:16.210 --> 00:22:19.043 align:middle line:84% This is the number of the harmonic. 00:22:19.043 --> 00:22:20.210 align:middle line:90% None of this really matters. 00:22:20.210 --> 00:22:22.470 align:middle line:84% The point is like the numbers are small right, 2%. 00:22:22.470 --> 00:22:24.410 align:middle line:84% So people are very sensitive to this. 00:22:24.410 --> 00:22:27.530 align:middle line:90% 00:22:27.530 --> 00:22:30.430 align:middle line:84% So here's something that is related to this 00:22:30.430 --> 00:22:31.630 align:middle line:90% that is really pretty cool. 00:22:31.630 --> 00:22:35.890 align:middle line:84% And it's natural to wonder like, well, what-- 00:22:35.890 --> 00:22:40.870 align:middle line:84% so you mistune the harmonic and you hear the second thing. 00:22:40.870 --> 00:22:42.330 align:middle line:90% So that seems significant. 00:22:42.330 --> 00:22:43.930 align:middle line:84% You've changed your interpretation 00:22:43.930 --> 00:22:46.890 align:middle line:84% in terms of what the sounds are in the world. 00:22:46.890 --> 00:22:49.810 align:middle line:84% But it's natural to wonder like the extent to which 00:22:49.810 --> 00:22:52.850 align:middle line:84% that subjective sense that there are two things there, 00:22:52.850 --> 00:22:55.770 align:middle line:84% like whether that affects other things that you 00:22:55.770 --> 00:22:57.830 align:middle line:90% do with your auditory system. 00:22:57.830 --> 00:22:59.970 align:middle line:84% And so one of the things that we often 00:22:59.970 --> 00:23:01.930 align:middle line:84% do when we listen to a sound like that is we 00:23:01.930 --> 00:23:04.570 align:middle line:90% get a sense of the pitch. 00:23:04.570 --> 00:23:09.250 align:middle line:84% Maybe because you want to sing it back or accompany it 00:23:09.250 --> 00:23:10.090 align:middle line:90% in some way. 00:23:10.090 --> 00:23:14.290 align:middle line:84% And so it turns out that when you mistune 00:23:14.290 --> 00:23:17.970 align:middle line:84% one harmonic of a complex tone just a little bit, 00:23:17.970 --> 00:23:21.370 align:middle line:84% that will very slightly change your sense 00:23:21.370 --> 00:23:23.130 align:middle line:90% of the pitch of the tone. 00:23:23.130 --> 00:23:25.330 align:middle line:84% And so the way that this is measured 00:23:25.330 --> 00:23:29.190 align:middle line:84% is in an experiment where you perform a matching task. 00:23:29.190 --> 00:23:31.820 align:middle line:84% So on every trial of this experiment, 00:23:31.820 --> 00:23:33.570 align:middle line:84% you're going to hear a stimulus like this. 00:23:33.570 --> 00:23:35.910 align:middle line:84% It's a complex tone where one of the harmonics 00:23:35.910 --> 00:23:38.110 align:middle line:90% will be slightly mistuned. 00:23:38.110 --> 00:23:41.350 align:middle line:84% And then following that, there will be a match tone, 00:23:41.350 --> 00:23:42.930 align:middle line:84% and that's a purely harmonic tone. 00:23:42.930 --> 00:23:46.070 align:middle line:84% And your job is to adjust the fundamental frequency 00:23:46.070 --> 00:23:48.190 align:middle line:84% of the match tone to make it sound 00:23:48.190 --> 00:23:54.350 align:middle line:84% like it has exactly the same pitch as the initial tone. 00:23:54.350 --> 00:23:55.430 align:middle line:90% Yeah? 00:23:55.430 --> 00:23:58.910 align:middle line:84% AUDIENCE: Does the graph change depending on which [INAUDIBLE]? 00:23:58.910 --> 00:24:00.310 align:middle line:90% JOSH MCDERMOTT: Yes. 00:24:00.310 --> 00:24:02.930 align:middle line:84% But as long as it's what we call a resolved harmonic, 00:24:02.930 --> 00:24:04.470 align:middle line:90% it looks pretty similar. 00:24:04.470 --> 00:24:06.253 align:middle line:84% So in this case, it's the third harmonic. 00:24:06.253 --> 00:24:08.670 align:middle line:84% It would look the same if it was the second or the fourth, 00:24:08.670 --> 00:24:10.630 align:middle line:90% for instance. 00:24:10.630 --> 00:24:12.810 align:middle line:84% And that's what's shown in this graph. 00:24:12.810 --> 00:24:15.030 align:middle line:84% So this graph is showing the mean shift 00:24:15.030 --> 00:24:19.390 align:middle line:84% in the pitch match in percent of the fundamental frequency 00:24:19.390 --> 00:24:23.730 align:middle line:84% as a function of the shift in that mistuned harmonic. 00:24:23.730 --> 00:24:24.960 align:middle line:90% So what does this show? 00:24:24.960 --> 00:24:27.840 align:middle line:84% Well, it shows that as you mistune the harmonic, 00:24:27.840 --> 00:24:30.800 align:middle line:84% there's a small shift in the perceived 00:24:30.800 --> 00:24:32.640 align:middle line:90% pitch of the complex tone. 00:24:32.640 --> 00:24:33.440 align:middle line:90% It's very small. 00:24:33.440 --> 00:24:37.000 align:middle line:84% This is like a fraction of a percent, but it's measurable. 00:24:37.000 --> 00:24:40.960 align:middle line:84% However, something pretty interesting 00:24:40.960 --> 00:24:46.040 align:middle line:84% happens, which is that once the harmonic gets shifted too much, 00:24:46.040 --> 00:24:51.540 align:middle line:84% it stops changing the pitch of the tone that it's part of. 00:24:51.540 --> 00:24:54.980 align:middle line:84% And in fact, the pitch shift kind of drops back down. 00:24:54.980 --> 00:24:57.760 align:middle line:90% 00:24:57.760 --> 00:25:01.480 align:middle line:84% So the interpretation here is that when 00:25:01.480 --> 00:25:04.780 align:middle line:84% the shift of the harmonic is very small, 00:25:04.780 --> 00:25:07.760 align:middle line:84% you don't really hear it as a separate sound, 00:25:07.760 --> 00:25:10.580 align:middle line:84% and it gets incorporated into the calculation of the pitch, 00:25:10.580 --> 00:25:12.880 align:middle line:90% and so changes the pitch. 00:25:12.880 --> 00:25:14.820 align:middle line:84% When the shift kind of gets too big, 00:25:14.820 --> 00:25:18.120 align:middle line:84% though, you start hearing it as a separate sound, 00:25:18.120 --> 00:25:20.800 align:middle line:84% and your auditory system kind of excludes it 00:25:20.800 --> 00:25:26.580 align:middle line:84% from the computation of the pitch of the tone. 00:25:26.580 --> 00:25:29.900 align:middle line:84% So this is evidence that the estimation 00:25:29.900 --> 00:25:31.980 align:middle line:84% of the properties of sound sources 00:25:31.980 --> 00:25:35.580 align:middle line:84% is kind of influenced by these grouping phenomena. 00:25:35.580 --> 00:25:40.140 align:middle line:84% It's as though the pitch is kind of calculated 00:25:40.140 --> 00:25:45.300 align:middle line:84% on your estimates of what the sources are that explain a given 00:25:45.300 --> 00:25:47.940 align:middle line:90% stimulus. 00:25:47.940 --> 00:25:49.560 align:middle line:84% What questions you got about this? 00:25:49.560 --> 00:25:55.820 align:middle line:90% 00:25:55.820 --> 00:25:56.500 align:middle line:90% Yeah. 00:25:56.500 --> 00:25:59.940 align:middle line:84% AUDIENCE: So does this graph change 00:25:59.940 --> 00:26:04.680 align:middle line:84% as frequency goes up based on pitch perception in general? 00:26:04.680 --> 00:26:07.980 align:middle line:90% 00:26:07.980 --> 00:26:09.960 align:middle line:84% JOSH MCDERMOTT: It doesn't change a whole lot. 00:26:09.960 --> 00:26:13.660 align:middle line:84% I mean, I've only seen it measured 00:26:13.660 --> 00:26:15.828 align:middle line:90% over a couple octave range. 00:26:15.828 --> 00:26:17.620 align:middle line:84% So you can change the fundamental frequency 00:26:17.620 --> 00:26:20.220 align:middle line:84% of the tone, it doesn't really change very much. 00:26:20.220 --> 00:26:23.300 align:middle line:84% As I was saying in response to the earlier question, 00:26:23.300 --> 00:26:25.580 align:middle line:84% it will be affected by which harmonic you do this to. 00:26:25.580 --> 00:26:27.413 align:middle line:84% So if you do this to an unresolved harmonic, 00:26:27.413 --> 00:26:29.400 align:middle line:84% you actually don't get this effect. 00:26:29.400 --> 00:26:31.440 align:middle line:84% So the harmonic has to be resolved. 00:26:31.440 --> 00:26:33.160 align:middle line:84% It sort of makes sense, because remember 00:26:33.160 --> 00:26:35.640 align:middle line:84% how we talked about how pitch perception really 00:26:35.640 --> 00:26:38.500 align:middle line:84% seems to be mostly dominated by these low-numbered harmonics? 00:26:38.500 --> 00:26:40.980 align:middle line:90% So that really is a big factor. 00:26:40.980 --> 00:26:44.040 align:middle line:90% 00:26:44.040 --> 00:26:45.180 align:middle line:90% Any other questions? 00:26:45.180 --> 00:26:47.820 align:middle line:90% 00:26:47.820 --> 00:26:48.320 align:middle line:90% OK. 00:26:48.320 --> 00:26:51.420 align:middle line:90% 00:26:51.420 --> 00:26:54.360 align:middle line:84% So this effect of grouping via harmonicity 00:26:54.360 --> 00:26:57.140 align:middle line:84% is not specific to these kind of funny synthetic tones. 00:26:57.140 --> 00:26:58.960 align:middle line:90% It also works for speech. 00:26:58.960 --> 00:27:02.572 align:middle line:84% So this is a spectrogram of a speech signal. 00:27:02.572 --> 00:27:04.280 align:middle line:84% And you can see that there are these kind 00:27:04.280 --> 00:27:05.260 align:middle line:90% of horizontal stripes. 00:27:05.260 --> 00:27:10.480 align:middle line:84% So these are the harmonics that exist in voiced speech. 00:27:10.480 --> 00:27:13.180 align:middle line:84% And again, speech is usually pretty fused. 00:27:13.180 --> 00:27:14.933 align:middle line:90% So here's how it sounds. 00:27:14.933 --> 00:27:15.600 align:middle line:90% [AUDIO PLAYBACK] 00:27:15.600 --> 00:27:17.877 align:middle line:84% - Academic aptitude guarantees your diploma. 00:27:17.877 --> 00:27:18.460 align:middle line:90% [END PLAYBACK] 00:27:18.460 --> 00:27:19.835 align:middle line:84% JOSH MCDERMOTT: Good news, right? 00:27:19.835 --> 00:27:21.753 align:middle line:84% Academic aptitude guarantees your diploma. 00:27:21.753 --> 00:27:22.920 align:middle line:90% That's just a speech signal. 00:27:22.920 --> 00:27:26.920 align:middle line:84% That speech signal is composed of harmonics. 00:27:26.920 --> 00:27:29.440 align:middle line:84% And we can actually-- using modern techniques, 00:27:29.440 --> 00:27:33.268 align:middle line:84% we can take the sound apart and isolate particular harmonics. 00:27:33.268 --> 00:27:34.560 align:middle line:90% And this is the third harmonic. 00:27:34.560 --> 00:27:36.120 align:middle line:84% This is what it sounds like on its own. 00:27:36.120 --> 00:27:36.787 align:middle line:90% [AUDIO PLAYBACK] 00:27:36.787 --> 00:27:39.590 align:middle line:90% [HIGH-FREQUENCY WHISTLING] 00:27:39.590 --> 00:27:40.580 align:middle line:90% [END PLAYBACK] 00:27:40.580 --> 00:27:41.560 align:middle line:84% JOSH MCDERMOTT: Can you hear that? 00:27:41.560 --> 00:27:42.852 align:middle line:90% Sounds like a whistle, kind of. 00:27:42.852 --> 00:27:46.080 align:middle line:84% So that's what the third harmonic sounds like on its own. 00:27:46.080 --> 00:27:47.900 align:middle line:84% And it's always there, but it gets 00:27:47.900 --> 00:27:49.760 align:middle line:84% grouped with the rest of the speech signal. 00:27:49.760 --> 00:27:53.480 align:middle line:84% So what we did here is something kind of funny. 00:27:53.480 --> 00:27:55.300 align:middle line:84% You can see how the third harmonic is now 00:27:55.300 --> 00:27:58.300 align:middle line:84% kind of shifted up relative to where it should 00:27:58.300 --> 00:27:59.440 align:middle line:90% be in the harmonic series. 00:27:59.440 --> 00:28:01.660 align:middle line:84% So it's been mistuned by a fair bit. 00:28:01.660 --> 00:28:05.600 align:middle line:84% And so now we're going to listen to what that sounds like. 00:28:05.600 --> 00:28:08.252 align:middle line:90% And you should be able to hear-- 00:28:08.252 --> 00:28:09.960 align:middle line:84% it'll sound like there's a speech signal, 00:28:09.960 --> 00:28:12.220 align:middle line:84% but then you'll hear this whistle that's 00:28:12.220 --> 00:28:15.060 align:middle line:84% kind of on top of it, which is the third harmonic, which you're 00:28:15.060 --> 00:28:17.365 align:middle line:84% now hearing out as a separate thing 00:28:17.365 --> 00:28:19.740 align:middle line:84% because it's not harmonically related to everything else. 00:28:19.740 --> 00:28:21.200 align:middle line:90% [AUDIO PLAYBACK] 00:28:21.200 --> 00:28:23.857 align:middle line:84% - Academic aptitude guarantees your diploma. 00:28:23.857 --> 00:28:24.440 align:middle line:90% [END PLAYBACK] 00:28:24.440 --> 00:28:25.960 align:middle line:84% JOSH MCDERMOTT: Now, lest you think 00:28:25.960 --> 00:28:27.600 align:middle line:84% this is all like me hypnotizing you, 00:28:27.600 --> 00:28:29.600 align:middle line:84% let's go back to the original, and now you won't 00:28:29.600 --> 00:28:30.680 align:middle line:90% be able to hear it, I think. 00:28:30.680 --> 00:28:31.347 align:middle line:90% [AUDIO PLAYBACK] 00:28:31.347 --> 00:28:34.900 align:middle line:84% - Academic aptitude guarantees your diploma. 00:28:34.900 --> 00:28:35.920 align:middle line:90% [END PLAYBACK] 00:28:35.920 --> 00:28:38.240 align:middle line:84% JOSH MCDERMOTT: So these harmonic frequency relations 00:28:38.240 --> 00:28:44.320 align:middle line:84% are really quite powerful and play a pretty big role 00:28:44.320 --> 00:28:47.640 align:middle line:90% in the grouping of sound. 00:28:47.640 --> 00:28:50.160 align:middle line:90% But they're not omnipotent. 00:28:50.160 --> 00:28:54.880 align:middle line:84% So onset differences can cause harmonics to segregate even 00:28:54.880 --> 00:28:56.380 align:middle line:90% when they're not mistuned. 00:28:56.380 --> 00:28:57.920 align:middle line:84% So this is a classic demonstration 00:28:57.920 --> 00:29:03.840 align:middle line:84% where we're going to play you complex tones that are harmonic, 00:29:03.840 --> 00:29:07.240 align:middle line:84% and individual harmonics are going to get gated on and off. 00:29:07.240 --> 00:29:09.200 align:middle line:84% And what that's going to do is it's 00:29:09.200 --> 00:29:13.440 align:middle line:84% going to cause you to hear out that harmonic that's 00:29:13.440 --> 00:29:15.320 align:middle line:90% gated as a separate thing. 00:29:15.320 --> 00:29:18.140 align:middle line:84% And what's going to happen is iteratively, 00:29:18.140 --> 00:29:20.440 align:middle line:84% we're going to step through the harmonics. 00:29:20.440 --> 00:29:22.985 align:middle line:84% So here, you've got like the second harmonic being gated. 00:29:22.985 --> 00:29:24.360 align:middle line:84% Next, it's going to be the third. 00:29:24.360 --> 00:29:26.280 align:middle line:84% Next, it's going to be the fourth. 00:29:26.280 --> 00:29:28.460 align:middle line:84% And as we step through that, you'll 00:29:28.460 --> 00:29:30.740 align:middle line:84% hear each of these harmonics individually as 00:29:30.740 --> 00:29:32.180 align:middle line:90% like separate things. 00:29:32.180 --> 00:29:34.180 align:middle line:84% And the one that was gated on the previous trial 00:29:34.180 --> 00:29:35.920 align:middle line:90% will then just blend back in. 00:29:35.920 --> 00:29:39.100 align:middle line:90% 00:29:39.100 --> 00:29:41.400 align:middle line:90% So I'll let you watch this one. 00:29:41.400 --> 00:29:42.396 align:middle line:90% [AUDIO PLAYBACK] 00:29:42.396 --> 00:29:50.828 align:middle line:84% [FLAT TONE WITH LOW-FREQUENCY BEEPING] 00:29:50.828 --> 00:29:58.764 align:middle line:84% [FLAT TONE WITH MID-FREQUENCY BEEPING] 00:29:58.764 --> 00:30:06.700 align:middle line:84% [FLAT TONE WITH HIGHER-FREQUENCY BEEPING] 00:30:06.700 --> 00:30:14.636 align:middle line:84% [FLAT TONE WITH HIGHER-FREQUENCY BEEPING] 00:30:14.636 --> 00:30:22.572 align:middle line:84% [FLAT TONE WITH HIGH-FREQUENCY BEEPING] 00:30:22.572 --> 00:30:24.080 align:middle line:90% [END PLAYBACK] 00:30:24.080 --> 00:30:25.900 align:middle line:84% JOSH MCDERMOTT: We could do this forever, 00:30:25.900 --> 00:30:27.608 align:middle line:84% going through the entire harmonic series. 00:30:27.608 --> 00:30:29.160 align:middle line:90% But you get the idea. 00:30:29.160 --> 00:30:31.960 align:middle line:84% And so that illustrates a whole bunch of things. 00:30:31.960 --> 00:30:35.800 align:middle line:84% Again, it's this-- similar to some of the other examples 00:30:35.800 --> 00:30:39.460 align:middle line:84% that we've seen, it's kind of cognitively impenetrable. 00:30:39.460 --> 00:30:43.720 align:middle line:84% So even though you know that those harmonics are there, 00:30:43.720 --> 00:30:47.152 align:middle line:84% and you were hearing them on the previous iteration, 00:30:47.152 --> 00:30:49.360 align:middle line:84% once they stopped being gated, they just kind of fuse 00:30:49.360 --> 00:30:51.880 align:middle line:84% back in with the rest of the tone. 00:30:51.880 --> 00:30:55.780 align:middle line:90% And that's the effect of onset. 00:30:55.780 --> 00:31:01.080 align:middle line:90% 00:31:01.080 --> 00:31:04.760 align:middle line:84% So this is another pretty famous example 00:31:04.760 --> 00:31:07.400 align:middle line:84% where you can take a harmonic tone 00:31:07.400 --> 00:31:09.360 align:middle line:84% and cause it to sound like two things 00:31:09.360 --> 00:31:11.500 align:middle line:84% by frequency modulating part of it. 00:31:11.500 --> 00:31:15.900 align:middle line:84% So this starts out as a harmonic tone, 00:31:15.900 --> 00:31:18.780 align:middle line:84% and the even harmonics over time are 00:31:18.780 --> 00:31:22.860 align:middle line:84% given frequency modulation, and common frequency modulation. 00:31:22.860 --> 00:31:26.160 align:middle line:84% And so what happens is that at the very start of the stimulus, 00:31:26.160 --> 00:31:27.940 align:middle line:90% you hear this one thing. 00:31:27.940 --> 00:31:30.460 align:middle line:84% And then it'll kind of split into these two tones. 00:31:30.460 --> 00:31:32.740 align:middle line:84% And so this is called the Reynolds-McAdams Oboe 00:31:32.740 --> 00:31:36.740 align:middle line:84% because it was created by two people, Reynolds and McAdams. 00:31:36.740 --> 00:31:38.980 align:middle line:84% And it's called the Reynolds-McAdams Oboe, 00:31:38.980 --> 00:31:40.700 align:middle line:84% because the oboe is an instrument that 00:31:40.700 --> 00:31:43.020 align:middle line:84% has a lot of power at the odd harmonics. 00:31:43.020 --> 00:31:46.720 align:middle line:84% And so when you start to modulate the even harmonics, 00:31:46.720 --> 00:31:48.680 align:middle line:84% and those are heard as a separate tone, 00:31:48.680 --> 00:31:52.080 align:middle line:84% kind of an octave up, what's left is the odd harmonics. 00:31:52.080 --> 00:31:54.340 align:middle line:84% And that arguably sounds a little bit like an oboe. 00:31:54.340 --> 00:31:59.640 align:middle line:90% So let's listen to that. 00:31:59.640 --> 00:32:01.125 align:middle line:90% [AUDIO PLAYBACK] 00:32:01.125 --> 00:32:04.590 align:middle line:90% [DUAL TONES RESONATING] 00:32:04.590 --> 00:32:17.377 align:middle line:90% 00:32:17.377 --> 00:32:17.960 align:middle line:90% [END PLAYBACK] 00:32:17.960 --> 00:32:18.200 align:middle line:90% JOSH MCDERMOTT: Yeah. 00:32:18.200 --> 00:32:19.380 align:middle line:90% So we're struggling here. 00:32:19.380 --> 00:32:22.360 align:middle line:84% The spectrogram of this is pretty underwhelming 00:32:22.360 --> 00:32:25.360 align:middle line:84% because there's a lot of noise right in back 00:32:25.360 --> 00:32:27.920 align:middle line:84% of me, which is kind of overlapping with some 00:32:27.920 --> 00:32:29.140 align:middle line:90% of the frequencies here. 00:32:29.140 --> 00:32:33.540 align:middle line:84% So let's just look at this example. 00:32:33.540 --> 00:32:36.000 align:middle line:84% And hopefully that's kind of what you heard. 00:32:36.000 --> 00:32:38.060 align:middle line:84% So it separates out into these two tones. 00:32:38.060 --> 00:32:41.880 align:middle line:84% So naively, you look at this and it seems very plausible 00:32:41.880 --> 00:32:44.600 align:middle line:84% that what's going on here is that the common frequency 00:32:44.600 --> 00:32:47.320 align:middle line:84% modulation, like the changes in frequency that 00:32:47.320 --> 00:32:49.620 align:middle line:84% are shared across all those even harmonics, 00:32:49.620 --> 00:32:51.700 align:middle line:84% are what's causing them to group together. 00:32:51.700 --> 00:32:55.420 align:middle line:84% And that seems plausible, but you could also, in principle, 00:32:55.420 --> 00:32:58.740 align:middle line:84% just kind explain this by deviations from harmonicity. 00:32:58.740 --> 00:33:00.840 align:middle line:84% And the idea there is that when you're 00:33:00.840 --> 00:33:04.160 align:middle line:84% changing the frequencies of those second harmonics, 00:33:04.160 --> 00:33:08.200 align:middle line:84% they're at least instantaneously no longer harmonically related 00:33:08.200 --> 00:33:09.580 align:middle line:90% to the odd harmonics. 00:33:09.580 --> 00:33:11.300 align:middle line:84% So in principle, you could just have 00:33:11.300 --> 00:33:13.260 align:middle line:84% something that's kind of very sensitive to the harmonic 00:33:13.260 --> 00:33:15.300 align:middle line:84% relations that would then tell you that you have 00:33:15.300 --> 00:33:16.580 align:middle line:90% two different things there. 00:33:16.580 --> 00:33:21.740 align:middle line:84% And so that's left ambiguous by this demo. 00:33:21.740 --> 00:33:25.720 align:middle line:84% And you could play around with this and try to test this. 00:33:25.720 --> 00:33:26.220 align:middle line:90% Yeah. 00:33:26.220 --> 00:33:29.020 align:middle line:84% AUDIENCE: Can you have multiple modulation frequencies 00:33:29.020 --> 00:33:32.420 align:middle line:90% when you go to three or four? 00:33:32.420 --> 00:33:33.940 align:middle line:84% JOSH MCDERMOTT: You should try it. 00:33:33.940 --> 00:33:36.847 align:middle line:90% I think it will-- 00:33:36.847 --> 00:33:39.180 align:middle line:84% it'll probably start to sound more like a texture, would 00:33:39.180 --> 00:33:40.080 align:middle line:90% be my prediction. 00:33:40.080 --> 00:33:44.100 align:middle line:84% But it's not that easy to pay attention 00:33:44.100 --> 00:33:45.780 align:middle line:90% to lots of things at once. 00:33:45.780 --> 00:33:49.540 align:middle line:84% And so with just two, you can do it, 00:33:49.540 --> 00:33:52.940 align:middle line:84% but if you push it up to four, I think it's going to-- 00:33:52.940 --> 00:33:56.380 align:middle line:84% you might be able to pay attention one of them at a time, 00:33:56.380 --> 00:33:58.920 align:middle line:84% but feeling like you hear all of them at the same time, 00:33:58.920 --> 00:34:01.160 align:middle line:84% I think that will be challenging, is my guess. 00:34:01.160 --> 00:34:01.660 align:middle line:90% Yeah. 00:34:01.660 --> 00:34:06.580 align:middle line:90% 00:34:06.580 --> 00:34:10.040 align:middle line:84% But can anybody think of how you might 00:34:10.040 --> 00:34:13.760 align:middle line:84% test whether this effect might not 00:34:13.760 --> 00:34:17.440 align:middle line:90% be driven by harmonicity alone? 00:34:17.440 --> 00:34:19.280 align:middle line:84% What could we do to the tone that 00:34:19.280 --> 00:34:23.760 align:middle line:84% might give us a purer test of whether or not common frequency 00:34:23.760 --> 00:34:27.080 align:middle line:84% modulation is enough to make you hear something 00:34:27.080 --> 00:34:30.239 align:middle line:84% as separate from a bunch of other frequencies? 00:34:30.239 --> 00:34:31.679 align:middle line:90% Yeah. 00:34:31.679 --> 00:34:34.780 align:middle line:84% AUDIENCE: Do you use inharmonic tones [INAUDIBLE]? 00:34:34.780 --> 00:34:35.780 align:middle line:90% JOSH MCDERMOTT: Exactly. 00:34:35.780 --> 00:34:36.360 align:middle line:90% Yeah. 00:34:36.360 --> 00:34:36.860 align:middle line:90% Yeah. 00:34:36.860 --> 00:34:37.610 align:middle line:90% You should try it. 00:34:37.610 --> 00:34:38.880 align:middle line:90% Let me know what happens. 00:34:38.880 --> 00:34:39.760 align:middle line:90% Yeah. 00:34:39.760 --> 00:34:42.420 align:middle line:84% I know what happens, actually, but you should try it. 00:34:42.420 --> 00:34:42.920 align:middle line:90% Yeah. 00:34:42.920 --> 00:34:44.380 align:middle line:90% Yeah, that's exactly right. 00:34:44.380 --> 00:34:44.880 align:middle line:90% Yeah. 00:34:44.880 --> 00:34:48.480 align:middle line:84% So the idea is that if the tones are not harmonically related, 00:34:48.480 --> 00:34:51.280 align:middle line:84% then when you comodulate a bunch of them, 00:34:51.280 --> 00:34:53.824 align:middle line:84% they're still just as inharmonic as they were before, 00:34:53.824 --> 00:34:56.199 align:middle line:84% and you might be able to tell whether the comodulation is 00:34:56.199 --> 00:34:59.200 align:middle line:84% enough to really drive the effect. 00:34:59.200 --> 00:35:03.040 align:middle line:84% Another thing that could potentially 00:35:03.040 --> 00:35:05.520 align:middle line:84% be a powerful grouping cue, and turns out 00:35:05.520 --> 00:35:08.440 align:middle line:84% to be a powerful grouping cue, is repetition. 00:35:08.440 --> 00:35:13.860 align:middle line:84% So this is a graph that's-- it's showing you a time frequency 00:35:13.860 --> 00:35:15.440 align:middle line:90% decomposition of sound. 00:35:15.440 --> 00:35:17.340 align:middle line:84% And if you just look at this, you 00:35:17.340 --> 00:35:20.960 align:middle line:84% can notice that there's some repeating structure. 00:35:20.960 --> 00:35:23.540 align:middle line:84% You see those like blobs up at the top that seem like they're 00:35:23.540 --> 00:35:25.120 align:middle line:90% happening kind of regularly. 00:35:25.120 --> 00:35:30.300 align:middle line:84% And lots of natural sounds kind of repeat. 00:35:30.300 --> 00:35:34.480 align:middle line:84% Not so much speech, but lots of animal vocalizations. 00:35:34.480 --> 00:35:38.460 align:middle line:84% So most nonhuman animals, they pretty much can say one thing. 00:35:38.460 --> 00:35:40.860 align:middle line:90% It means like, hey, it's me. 00:35:40.860 --> 00:35:43.640 align:middle line:84% So they have this one call, but typically, if they vocalize, 00:35:43.640 --> 00:35:45.900 align:middle line:84% they'll do it several times in a row. 00:35:45.900 --> 00:35:50.380 align:middle line:84% So repetition is actually very common in animal vocalizations. 00:35:50.380 --> 00:35:53.260 align:middle line:84% So it seems plausible that the brain 00:35:53.260 --> 00:35:56.060 align:middle line:84% might have learned to leverage the fact that repetition 00:35:56.060 --> 00:35:59.380 align:middle line:84% is a powerful cue that you have some sound source out there 00:35:59.380 --> 00:36:00.080 align:middle line:90% in the world. 00:36:00.080 --> 00:36:03.900 align:middle line:84% So these are some experiments to try to test that. 00:36:03.900 --> 00:36:07.760 align:middle line:84% So in these experiments, we will present people 00:36:07.760 --> 00:36:10.720 align:middle line:84% with target sounds and distractor sounds. 00:36:10.720 --> 00:36:13.160 align:middle line:84% So these are modulated noises that 00:36:13.160 --> 00:36:15.920 align:middle line:90% will just sound kind of funny. 00:36:15.920 --> 00:36:18.400 align:middle line:84% So you add these two together, and you get a mixture. 00:36:18.400 --> 00:36:20.480 align:middle line:84% And so the question is whether or not 00:36:20.480 --> 00:36:22.520 align:middle line:84% you can actually segregate that mixture 00:36:22.520 --> 00:36:25.108 align:middle line:84% and estimate the constituent sources. 00:36:25.108 --> 00:36:27.400 align:middle line:84% And so the idea behind making these kind of funny noise 00:36:27.400 --> 00:36:29.440 align:middle line:84% sources is that they don't really 00:36:29.440 --> 00:36:31.667 align:middle line:84% have any grouping cues of their own. 00:36:31.667 --> 00:36:33.500 align:middle line:84% There's not much in the way of common onset, 00:36:33.500 --> 00:36:35.880 align:middle line:84% there's no harmonicity, none of the classical stuff 00:36:35.880 --> 00:36:37.320 align:middle line:84% that we've just been talking about 00:36:37.320 --> 00:36:38.620 align:middle line:90% is present in these things. 00:36:38.620 --> 00:36:44.320 align:middle line:84% And so the idea was to see whether repetition alone 00:36:44.320 --> 00:36:47.880 align:middle line:84% would be enough to allow you to actually segregate 00:36:47.880 --> 00:36:50.000 align:middle line:90% mixtures in this situation. 00:36:50.000 --> 00:36:53.920 align:middle line:84% And so the way that you test that is by presenting a mixture, 00:36:53.920 --> 00:36:56.280 align:middle line:84% and then presenting a probe sound. 00:36:56.280 --> 00:36:59.480 align:middle line:84% And the probe sound can either be 00:36:59.480 --> 00:37:02.460 align:middle line:84% one of the sounds in the mixture, or something different. 00:37:02.460 --> 00:37:06.500 align:middle line:84% And the idea is that if you are able to correctly segregate 00:37:06.500 --> 00:37:08.820 align:middle line:84% the mixture into the constituent sources, 00:37:08.820 --> 00:37:11.940 align:middle line:84% you should be able to solve that task of saying 00:37:11.940 --> 00:37:16.740 align:middle line:84% whether the probe was one of the sources in the mixture. 00:37:16.740 --> 00:37:18.940 align:middle line:84% But it turns out that, for these sounds, if you just 00:37:18.940 --> 00:37:20.760 align:middle line:84% get a single mixture this is really hard. 00:37:20.760 --> 00:37:21.800 align:middle line:90% So it sounds like this. 00:37:21.800 --> 00:37:22.300 align:middle line:90% [AUDIO PLAYBACK] 00:37:22.300 --> 00:37:23.260 align:middle line:90% [MID-FREQUENCY STATIC] 00:37:23.260 --> 00:37:23.860 align:middle line:90% [LOWER-FREQUENCY STATIC] 00:37:23.860 --> 00:37:23.980 align:middle line:90% [END PLAYBACK] 00:37:23.980 --> 00:37:24.720 align:middle line:84% JOSH MCDERMOTT: Let me do it again. 00:37:24.720 --> 00:37:25.220 align:middle line:90% [AUDIO PLAYBACK] 00:37:25.220 --> 00:37:26.080 align:middle line:90% [MID-FREQUENCY STATIC] 00:37:26.080 --> 00:37:27.020 align:middle line:90% [LOWER-FREQUENCY STATIC] 00:37:27.020 --> 00:37:27.240 align:middle line:90% [END PLAYBACK] 00:37:27.240 --> 00:37:28.380 align:middle line:84% JOSH MCDERMOTT: So the question is, 00:37:28.380 --> 00:37:30.780 align:middle line:84% that sound at-- the second sound, that's the probe. 00:37:30.780 --> 00:37:32.540 align:middle line:90% Was that present in the mixture. 00:37:32.540 --> 00:37:35.620 align:middle line:84% And you don't know because for these kinds of sounds, 00:37:35.620 --> 00:37:37.200 align:middle line:90% they just really fuse together. 00:37:37.200 --> 00:37:40.620 align:middle line:84% You don't really hear them as separate things. 00:37:40.620 --> 00:37:42.500 align:middle line:84% And it turns out that probe sound actually 00:37:42.500 --> 00:37:43.320 align:middle line:90% was in the mixture. 00:37:43.320 --> 00:37:44.840 align:middle line:84% So the sounds have some structure, 00:37:44.840 --> 00:37:47.940 align:middle line:84% but not enough to produce the segregation of a single mixture. 00:37:47.940 --> 00:37:52.080 align:middle line:84% However, if you present the target sound kind of repeatedly, 00:37:52.080 --> 00:37:55.340 align:middle line:84% and these are schematics of how this works, 00:37:55.340 --> 00:37:57.900 align:middle line:84% you're able to hear it out actually really well. 00:37:57.900 --> 00:37:59.340 align:middle line:90% So you just heard this. 00:37:59.340 --> 00:38:02.660 align:middle line:84% So we have the target sound in red, a distractor in blue, 00:38:02.660 --> 00:38:04.520 align:middle line:90% and then a probe sound in black. 00:38:04.520 --> 00:38:07.680 align:middle line:84% And this was actually pretty hard. 00:38:07.680 --> 00:38:09.640 align:middle line:84% But let's think about what would happen here 00:38:09.640 --> 00:38:11.700 align:middle line:84% if we have the target repeated twice, 00:38:11.700 --> 00:38:13.840 align:middle line:84% each time paired with a different distractor, 00:38:13.840 --> 00:38:17.440 align:middle line:84% or three times, or five times, or 10 times. 00:38:17.440 --> 00:38:21.400 align:middle line:84% And so this graph is plotting a measure of how well people 00:38:21.400 --> 00:38:22.700 align:middle line:90% can perform this task. 00:38:22.700 --> 00:38:24.560 align:middle line:84% The measure here is what's called the ROC 00:38:24.560 --> 00:38:28.420 align:middle line:84% area, which is related to d prime, 00:38:28.420 --> 00:38:29.740 align:middle line:90% which we talked about earlier. 00:38:29.740 --> 00:38:31.573 align:middle line:84% So the point is that chance performance here 00:38:31.573 --> 00:38:34.920 align:middle line:84% is going to be 0.5, and values above that 00:38:34.920 --> 00:38:37.520 align:middle line:84% will mean that you're able to perform the task. 00:38:37.520 --> 00:38:40.407 align:middle line:84% And so this graph is showing that as you increase 00:38:40.407 --> 00:38:42.240 align:middle line:84% the number of mixtures, which, in this case, 00:38:42.240 --> 00:38:45.680 align:middle line:84% means hearing repetitions of the target sound, 00:38:45.680 --> 00:38:49.320 align:middle line:84% you're able to hear the target out pretty well. 00:38:49.320 --> 00:38:51.365 align:middle line:84% This is pretty easy to verify for ourselves. 00:38:51.365 --> 00:38:52.740 align:middle line:84% So I'm going to play these demos. 00:38:52.740 --> 00:38:54.880 align:middle line:84% And in all of these cases, there'll 00:38:54.880 --> 00:38:56.220 align:middle line:90% be a probe sound at the end. 00:38:56.220 --> 00:39:00.580 align:middle line:84% And the probe sound will be the same as the target sound. 00:39:00.580 --> 00:39:02.360 align:middle line:84% And so what you need to ask yourself 00:39:02.360 --> 00:39:05.220 align:middle line:84% is whether the sound at the end resembles 00:39:05.220 --> 00:39:07.200 align:middle line:84% something that you heard in the mixture, 00:39:07.200 --> 00:39:09.048 align:middle line:90% or the sequence of mixtures. 00:39:09.048 --> 00:39:10.840 align:middle line:84% And so when you just have a single mixture, 00:39:10.840 --> 00:39:12.320 align:middle line:84% it's not going to sound very similar. 00:39:12.320 --> 00:39:12.820 align:middle line:90% [AUDIO PLAYBACK] 00:39:12.820 --> 00:39:13.295 align:middle line:90% [MID-FREQUENCY STATIC] 00:39:13.295 --> 00:39:14.720 align:middle line:90% [HIGHER-FREQUENCY STATIC] 00:39:14.720 --> 00:39:16.063 align:middle line:90% [END PLAYBACK] 00:39:16.063 --> 00:39:17.480 align:middle line:84% JOSH MCDERMOTT: Let's do it again. 00:39:17.480 --> 00:39:18.440 align:middle line:90% [AUDIO PLAYBACK] 00:39:18.440 --> 00:39:19.360 align:middle line:90% [MID-FREQUENCY STATIC] 00:39:19.360 --> 00:39:20.280 align:middle line:90% [HIGHER-FREQUENCY STATIC] 00:39:20.280 --> 00:39:20.740 align:middle line:90% [END PLAYBACK] 00:39:20.740 --> 00:39:21.620 align:middle line:84% JOSH MCDERMOTT: But with two, maybe you 00:39:21.620 --> 00:39:22.880 align:middle line:90% can hear a little something. 00:39:22.880 --> 00:39:23.380 align:middle line:90% [AUDIO PLAYBACK] 00:39:23.380 --> 00:39:24.220 align:middle line:90% [MIXED FREQUENCY STATIC] 00:39:24.220 --> 00:39:24.640 align:middle line:90% [MID-FREQUENCY STATIC] 00:39:24.640 --> 00:39:25.060 align:middle line:90% [END PLAYBACK] 00:39:25.060 --> 00:39:27.060 align:middle line:84% JOSH MCDERMOTT: Probably heard something repeat. 00:39:27.060 --> 00:39:27.740 align:middle line:90% Let's try three. 00:39:27.740 --> 00:39:29.198 align:middle line:90% [MIXED FREQUENCY STATIC] 00:39:29.198 --> 00:39:30.170 align:middle line:90% [MID-FREQUENCY STATIC] 00:39:30.170 --> 00:39:30.660 align:middle line:90% [END PLAYBACK] 00:39:30.660 --> 00:39:32.240 align:middle line:84% JOSH MCDERMOTT: There you get it pretty clearly. 00:39:32.240 --> 00:39:32.953 align:middle line:90% And with five. 00:39:32.953 --> 00:39:33.620 align:middle line:90% [AUDIO PLAYBACK] 00:39:33.620 --> 00:39:36.882 align:middle line:84% [MIXED FREQUENCY STATIC REPEATING] 00:39:36.882 --> 00:39:37.588 align:middle line:90% [END PLAYBACK] 00:39:37.588 --> 00:39:38.880 align:middle line:90% JOSH MCDERMOTT: And 10 is easy. 00:39:38.880 --> 00:39:39.547 align:middle line:90% [AUDIO PLAYBACK] 00:39:39.547 --> 00:39:44.030 align:middle line:84% [MIXED FREQUENCY STATIC REPEATING] 00:39:44.030 --> 00:39:45.340 align:middle line:90% [END PLAYBACK] 00:39:45.340 --> 00:39:47.780 align:middle line:84% JOSH MCDERMOTT: So the point is that there's 00:39:47.780 --> 00:39:52.580 align:middle line:84% this latent repetition in the signal that kind of pops out 00:39:52.580 --> 00:39:57.180 align:middle line:84% to your auditory system, even though any single one 00:39:57.180 --> 00:40:00.620 align:middle line:84% of these mixtures is kind of impossible to parse. 00:40:00.620 --> 00:40:03.670 align:middle line:84% And so this is a demonstration that kind of repetition 00:40:03.670 --> 00:40:08.590 align:middle line:84% alone is a pretty powerful grouping cue. 00:40:08.590 --> 00:40:11.910 align:middle line:90% Any questions about that? 00:40:11.910 --> 00:40:15.830 align:middle line:84% And this is actually probably very, very important in music, 00:40:15.830 --> 00:40:18.630 align:middle line:84% where there's also sounds that kind of repeat, 00:40:18.630 --> 00:40:21.310 align:middle line:90% probably helps you hear rhythms. 00:40:21.310 --> 00:40:24.870 align:middle line:84% And my guess is it has its roots as something that 00:40:24.870 --> 00:40:27.480 align:middle line:84% co-evolved with animal vocalizations 00:40:27.480 --> 00:40:29.730 align:middle line:84% because these repeating vocalizations are very common. 00:40:29.730 --> 00:40:33.330 align:middle line:90% 00:40:33.330 --> 00:40:35.510 align:middle line:84% So a couple lectures ago, we were talking 00:40:35.510 --> 00:40:37.790 align:middle line:90% about sound localization. 00:40:37.790 --> 00:40:41.090 align:middle line:84% Sounds often occur at different locations in the world, 00:40:41.090 --> 00:40:42.710 align:middle line:84% and it's kind of natural to wonder 00:40:42.710 --> 00:40:45.550 align:middle line:84% whether spatial location would really help you 00:40:45.550 --> 00:40:47.130 align:middle line:90% with auditory scene analysis. 00:40:47.130 --> 00:40:52.750 align:middle line:84% And there's a bunch of examples of a role for spatial cues. 00:40:52.750 --> 00:40:55.310 align:middle line:84% This is one that's pretty famous, where 00:40:55.310 --> 00:40:58.350 align:middle line:84% spatial cues actually improve your ability 00:40:58.350 --> 00:41:00.470 align:middle line:90% to detect masked sounds. 00:41:00.470 --> 00:41:03.290 align:middle line:84% So again, we have the happy and sad people 00:41:03.290 --> 00:41:06.090 align:middle line:84% that are indicating conditions in which the tone is 00:41:06.090 --> 00:41:08.050 align:middle line:90% masked or not. 00:41:08.050 --> 00:41:12.793 align:middle line:84% And we know from the lecture on masking that you can take a tone 00:41:12.793 --> 00:41:14.210 align:middle line:84% and play it with noise, and we can 00:41:14.210 --> 00:41:16.148 align:middle line:84% adjust the levels of the tone and the noise 00:41:16.148 --> 00:41:17.690 align:middle line:84% such that the tone is masked, meaning 00:41:17.690 --> 00:41:18.910 align:middle line:90% that people can't detect it. 00:41:18.910 --> 00:41:20.410 align:middle line:90% So the person is unhappy. 00:41:20.410 --> 00:41:23.210 align:middle line:90% 00:41:23.210 --> 00:41:27.335 align:middle line:84% Turns out, though, that if you add an identical noise-- 00:41:27.335 --> 00:41:29.210 align:middle line:84% so, OK, so this is a situation where you just 00:41:29.210 --> 00:41:30.610 align:middle line:90% play them to ear ear. 00:41:30.610 --> 00:41:32.892 align:middle line:84% So it's just going into the right ear. 00:41:32.892 --> 00:41:34.850 align:middle line:84% Turns out that you can take the identical noise 00:41:34.850 --> 00:41:37.930 align:middle line:84% and put it into the opposite ear and that will 00:41:37.930 --> 00:41:39.470 align:middle line:90% render the tone detectable. 00:41:39.470 --> 00:41:42.770 align:middle line:90% 00:41:42.770 --> 00:41:44.930 align:middle line:84% Anybody have any idea, like why would that 00:41:44.930 --> 00:41:48.470 align:middle line:84% happen in terms of what we know about sound localization? 00:41:48.470 --> 00:42:00.070 align:middle line:90% 00:42:00.070 --> 00:42:01.550 align:middle line:90% So answer me this. 00:42:01.550 --> 00:42:03.550 align:middle line:84% In the bottom scenario, where is it 00:42:03.550 --> 00:42:05.770 align:middle line:84% going to sound like the noise is coming from? 00:42:05.770 --> 00:42:10.790 align:middle line:84% Where will the noise be situated spatially? 00:42:10.790 --> 00:42:12.990 align:middle line:84% AUDIENCE: Probably like in front of the person. 00:42:12.990 --> 00:42:14.970 align:middle line:84% JOSH MCDERMOTT: Yeah, like at the midline. 00:42:14.970 --> 00:42:15.470 align:middle line:90% because? 00:42:15.470 --> 00:42:17.677 align:middle line:84% We got exactly the same noise in the two ears, 00:42:17.677 --> 00:42:20.010 align:middle line:84% that means that the interaural level difference is zero. 00:42:20.010 --> 00:42:22.390 align:middle line:84% It means that the interaural time difference is zero. 00:42:22.390 --> 00:42:24.690 align:middle line:84% That's consistent with it being on the midline. 00:42:24.690 --> 00:42:29.470 align:middle line:84% Whereas the tone is exclusively present 00:42:29.470 --> 00:42:30.770 align:middle line:90% in this case in the right ear. 00:42:30.770 --> 00:42:34.205 align:middle line:84% So you have a huge ILD, and the tone 00:42:34.205 --> 00:42:36.330 align:middle line:84% will sound like it's kind of coming from the right. 00:42:36.330 --> 00:42:39.110 align:middle line:84% So you've got very different spatial cues, 00:42:39.110 --> 00:42:42.350 align:middle line:84% and that tends to have a pretty big effect on your ability 00:42:42.350 --> 00:42:44.070 align:middle line:90% to detect the tone. 00:42:44.070 --> 00:42:49.710 align:middle line:84% So this is known as the binaural masking-level difference. 00:42:49.710 --> 00:42:51.750 align:middle line:90% This has been studied a lot. 00:42:51.750 --> 00:42:53.370 align:middle line:84% This is one of many, many graphs. 00:42:53.370 --> 00:42:55.190 align:middle line:84% So what this graph is showing you-- 00:42:55.190 --> 00:42:56.890 align:middle line:84% this graph is like from a paper that 00:42:56.890 --> 00:42:59.810 align:middle line:84% was measuring this effect as a function 00:42:59.810 --> 00:43:02.115 align:middle line:84% of a lot of different stimulus parameters. 00:43:02.115 --> 00:43:03.490 align:middle line:84% Those effects don't really matter 00:43:03.490 --> 00:43:04.782 align:middle line:90% for the purposes of this class. 00:43:04.782 --> 00:43:07.670 align:middle line:84% I'm just giving you this as an example of the effect. 00:43:07.670 --> 00:43:10.398 align:middle line:84% And the key comparison here is-- so these are thresholds. 00:43:10.398 --> 00:43:12.190 align:middle line:84% In this case, it's a function of bandwidth. 00:43:12.190 --> 00:43:13.070 align:middle line:90% That doesn't matter. 00:43:13.070 --> 00:43:15.190 align:middle line:84% But the key comparison here is between the circles 00:43:15.190 --> 00:43:16.565 align:middle line:84% and the diamonds and the squares. 00:43:16.565 --> 00:43:22.130 align:middle line:84% So the circles are where the tone and the noise both 00:43:22.130 --> 00:43:24.690 align:middle line:84% have the same phase, so they're both in one ear. 00:43:24.690 --> 00:43:27.650 align:middle line:84% And then the other symbols here are 00:43:27.650 --> 00:43:29.970 align:middle line:84% when you introduce a phase difference 00:43:29.970 --> 00:43:32.310 align:middle line:84% between the tone and the noise, for instance, 00:43:32.310 --> 00:43:34.950 align:middle line:84% by putting the noise in both ears. 00:43:34.950 --> 00:43:37.530 align:middle line:84% And you can see that there's a big difference in thresholds 00:43:37.530 --> 00:43:40.650 align:middle line:84% here, up to possibly 20 dB, so it's a huge effect. 00:43:40.650 --> 00:43:45.510 align:middle line:90% 00:43:45.510 --> 00:43:52.130 align:middle line:84% So it kind of follows, though, that if we 00:43:52.130 --> 00:43:57.787 align:middle line:84% were to add the tone back in to the other ear, 00:43:57.787 --> 00:43:58.870 align:middle line:90% it's now going to become-- 00:43:58.870 --> 00:44:00.620 align:middle line:84% it's now going to be hard to detect again, 00:44:00.620 --> 00:44:02.287 align:middle line:84% because we set the signal-to-noise ratio 00:44:02.287 --> 00:44:04.245 align:middle line:84% of the tone and the noise such that if you just 00:44:04.245 --> 00:44:06.410 align:middle line:84% play it for one year, it will be below threshold. 00:44:06.410 --> 00:44:09.430 align:middle line:84% So if you have the identical tone and the identical noise 00:44:09.430 --> 00:44:11.650 align:middle line:84% in the other ear, that would also be below threshold. 00:44:11.650 --> 00:44:14.245 align:middle line:84% So now subject to the constraint that the tone and the noise 00:44:14.245 --> 00:44:15.870 align:middle line:84% have to be in both ears, what could you 00:44:15.870 --> 00:44:20.070 align:middle line:84% do to the tone or the noise in order 00:44:20.070 --> 00:44:23.173 align:middle line:84% to render the tone audible again, 00:44:23.173 --> 00:44:25.090 align:middle line:84% based on what you know about binaural hearing? 00:44:25.090 --> 00:44:25.590 align:middle line:90% Yes, Krista. 00:44:25.590 --> 00:44:28.790 align:middle line:84% AUDIENCE: You would phase shift one-- like either the noise 00:44:28.790 --> 00:44:30.117 align:middle line:90% or the tone of one ear. 00:44:30.117 --> 00:44:30.950 align:middle line:90% JOSH MCDERMOTT: Yep. 00:44:30.950 --> 00:44:31.690 align:middle line:90% Exactly. 00:44:31.690 --> 00:44:34.470 align:middle line:84% So you introduce a phase difference 00:44:34.470 --> 00:44:36.150 align:middle line:90% for one of the two signals. 00:44:36.150 --> 00:44:39.790 align:middle line:84% That will introduce a interaural time difference 00:44:39.790 --> 00:44:44.990 align:middle line:84% and the signal would be detectable again. 00:44:44.990 --> 00:44:46.890 align:middle line:90% Cool. 00:44:46.890 --> 00:44:49.790 align:middle line:84% Any questions about the binaural masking-level difference? 00:44:49.790 --> 00:44:55.470 align:middle line:90% 00:44:55.470 --> 00:44:59.850 align:middle line:84% So one example of a case where spatial cues seems to influence 00:44:59.850 --> 00:45:01.930 align:middle line:90% auditory scene analysis. 00:45:01.930 --> 00:45:05.730 align:middle line:90% So let's keep it moving. 00:45:05.730 --> 00:45:08.210 align:middle line:84% There's another really cool spatial effect 00:45:08.210 --> 00:45:09.978 align:middle line:84% on grouping that relates to reverberation. 00:45:09.978 --> 00:45:11.770 align:middle line:84% So we were just talking about reverberation 00:45:11.770 --> 00:45:13.990 align:middle line:84% in the context of distance perception. 00:45:13.990 --> 00:45:16.010 align:middle line:84% So reverberation refers to all the reflections 00:45:16.010 --> 00:45:18.470 align:middle line:84% that occur when you're in a natural environment. 00:45:18.470 --> 00:45:20.450 align:middle line:84% So this is like a schematic of what 00:45:20.450 --> 00:45:23.190 align:middle line:84% happens to sound when you're listening in a room. 00:45:23.190 --> 00:45:24.870 align:middle line:84% So the idea is there's a person here, 00:45:24.870 --> 00:45:27.410 align:middle line:84% there's a speaker here that's playing sound. 00:45:27.410 --> 00:45:29.330 align:middle line:84% There's this direct path from the speaker 00:45:29.330 --> 00:45:31.110 align:middle line:90% to the two ears of the person. 00:45:31.110 --> 00:45:32.510 align:middle line:90% Those are the green lines. 00:45:32.510 --> 00:45:35.050 align:middle line:84% But then there are all these other indirect paths. 00:45:35.050 --> 00:45:37.010 align:middle line:84% So the blue lines are the paths that 00:45:37.010 --> 00:45:40.090 align:middle line:90% involve a single reflection. 00:45:40.090 --> 00:45:43.142 align:middle line:84% The red paths are the paths that involve two reflections. 00:45:43.142 --> 00:45:44.350 align:middle line:90% There's quite a lot of those. 00:45:44.350 --> 00:45:48.027 align:middle line:84% And then for the purposes of reducing chaos, 00:45:48.027 --> 00:45:49.610 align:middle line:84% the picture doesn't show you the paths 00:45:49.610 --> 00:45:51.735 align:middle line:84% that involve three reflections, and four, and five, 00:45:51.735 --> 00:45:53.990 align:middle line:90% and so forth. 00:45:53.990 --> 00:45:58.150 align:middle line:84% And so what happens is that these alternative 00:45:58.150 --> 00:46:01.890 align:middle line:84% paths that the sound can take, well, they're longer, 00:46:01.890 --> 00:46:04.190 align:middle line:84% and so the sound arrives a little bit later. 00:46:04.190 --> 00:46:08.453 align:middle line:84% And the sound also gets filtered because any time you reflect off 00:46:08.453 --> 00:46:10.870 align:middle line:84% of a surface, the surface absorbs some of the sound energy 00:46:10.870 --> 00:46:12.630 align:middle line:84% and typically filters it a little bit. 00:46:12.630 --> 00:46:18.630 align:middle line:84% And so what happens in practice is shown here. 00:46:18.630 --> 00:46:22.710 align:middle line:84% So we can characterize the reverberation in any setting 00:46:22.710 --> 00:46:24.950 align:middle line:84% with what's called the impulse response. 00:46:24.950 --> 00:46:28.390 align:middle line:84% So this is the sound that would be produced 00:46:28.390 --> 00:46:30.390 align:middle line:84% at a microphone at a particular position 00:46:30.390 --> 00:46:32.170 align:middle line:84% if the source was at another position, 00:46:32.170 --> 00:46:37.690 align:middle line:84% and if the sound source played an impulse that's like a click. 00:46:37.690 --> 00:46:39.630 align:middle line:84% So there's an impulse, and then this 00:46:39.630 --> 00:46:44.010 align:middle line:84% is what's recorded at the microphone in a room. 00:46:44.010 --> 00:46:46.870 align:middle line:84% And so what happens here, well, so this 00:46:46.870 --> 00:46:48.710 align:middle line:84% is the time at which the click is played 00:46:48.710 --> 00:46:51.310 align:middle line:90% from the source, the speaker. 00:46:51.310 --> 00:46:53.650 align:middle line:84% And you can see that initially the microphone 00:46:53.650 --> 00:46:54.930 align:middle line:90% doesn't have any signal. 00:46:54.930 --> 00:46:56.930 align:middle line:90% This is the propagation time. 00:46:56.930 --> 00:47:01.030 align:middle line:84% So then you get this initial impulse in the impulse response. 00:47:01.030 --> 00:47:03.070 align:middle line:90% That's the direct sound right. 00:47:03.070 --> 00:47:05.130 align:middle line:84% That's the sound that's reaching the microphone 00:47:05.130 --> 00:47:08.632 align:middle line:90% direct from the source. 00:47:08.632 --> 00:47:10.090 align:middle line:84% But then you can see that there are 00:47:10.090 --> 00:47:12.090 align:middle line:84% all these subsequent little things that 00:47:12.090 --> 00:47:13.510 align:middle line:90% happen in the impulse response. 00:47:13.510 --> 00:47:17.850 align:middle line:84% And so initially, these are discrete reflections 00:47:17.850 --> 00:47:20.330 align:middle line:84% that are happening from some of the shortest paths. 00:47:20.330 --> 00:47:23.530 align:middle line:84% And then after that, the reflections get sufficiently 00:47:23.530 --> 00:47:26.650 align:middle line:84% dense, so you just kind of have this long tail that slowly 00:47:26.650 --> 00:47:30.495 align:middle line:84% decays, because every time there's a reflection, 00:47:30.495 --> 00:47:32.870 align:middle line:84% the sound gets absorbed a little bit, it gets attenuated. 00:47:32.870 --> 00:47:35.290 align:middle line:84% And so eventually, this kind of fades down 00:47:35.290 --> 00:47:38.090 align:middle line:84% into something that's undetectable. 00:47:38.090 --> 00:47:42.410 align:middle line:84% So this is how you characterize reverberation. 00:47:42.410 --> 00:47:46.250 align:middle line:84% And these impulse responses have been measured in lots of spaces. 00:47:46.250 --> 00:47:49.210 align:middle line:84% And the key thing to note here is that you always 00:47:49.210 --> 00:47:55.610 align:middle line:84% see this direct sound followed by these later reflections. 00:47:55.610 --> 00:47:59.590 align:middle line:84% Now, one of the big challenges that this poses 00:47:59.590 --> 00:48:01.210 align:middle line:90% is for sound localization. 00:48:01.210 --> 00:48:03.043 align:middle line:84% Because the other thing that you should note 00:48:03.043 --> 00:48:07.070 align:middle line:84% here is that the sound that is traveling the direct path 00:48:07.070 --> 00:48:10.270 align:middle line:84% is coming from the direction where the sound source actually 00:48:10.270 --> 00:48:12.070 align:middle line:90% is in the world. 00:48:12.070 --> 00:48:15.350 align:middle line:84% By comparison, the sound that is reflected 00:48:15.350 --> 00:48:19.550 align:middle line:84% is actually coming from other directions, which is not where 00:48:19.550 --> 00:48:22.112 align:middle line:90% the sound actually is from. 00:48:22.112 --> 00:48:24.070 align:middle line:84% So you're essentially getting all these delayed 00:48:24.070 --> 00:48:27.370 align:middle line:84% copies of the sound coming from the wrong direction. 00:48:27.370 --> 00:48:32.670 align:middle line:84% So it's a major challenge for sound localization. 00:48:32.670 --> 00:48:34.750 align:middle line:84% And there's this effect that's known 00:48:34.750 --> 00:48:38.390 align:middle line:84% as the precedence effect that seems to be related to this. 00:48:38.390 --> 00:48:41.050 align:middle line:84% So this was discovered by Hans Wallach, 00:48:41.050 --> 00:48:43.610 align:middle line:84% who was an early Gestalt psychologist, 00:48:43.610 --> 00:48:47.150 align:middle line:84% did lots of really interesting things back in the '40s. 00:48:47.150 --> 00:48:49.550 align:middle line:84% And the setup for measuring this is shown here. 00:48:49.550 --> 00:48:52.047 align:middle line:84% So you have the person, they're in an anechoic chamber. 00:48:52.047 --> 00:48:54.130 align:middle line:84% Remember, that's one of those special rooms that's 00:48:54.130 --> 00:48:56.090 align:middle line:84% kind of treated so that the room itself really 00:48:56.090 --> 00:48:58.750 align:middle line:90% doesn't have many reflections. 00:48:58.750 --> 00:49:01.210 align:middle line:84% And there are two speakers, one to the left 00:49:01.210 --> 00:49:02.150 align:middle line:90% and one to the right. 00:49:02.150 --> 00:49:04.730 align:middle line:90% That's minus 45 and plus 45. 00:49:04.730 --> 00:49:07.250 align:middle line:90% And the speakers emit clicks. 00:49:07.250 --> 00:49:09.610 align:middle line:84% But critically, there can be a time difference 00:49:09.610 --> 00:49:10.890 align:middle line:90% between the clicks. 00:49:10.890 --> 00:49:14.178 align:middle line:84% And so one of them is designated as the lagging speaker, 00:49:14.178 --> 00:49:15.470 align:middle line:90% and one is the leading speaker. 00:49:15.470 --> 00:49:18.350 align:middle line:90% That means it comes first. 00:49:18.350 --> 00:49:22.370 align:middle line:84% And there's two phenomena that happen in this setting. 00:49:22.370 --> 00:49:24.830 align:middle line:84% The first is what's shown in this graph. 00:49:24.830 --> 00:49:28.370 align:middle line:84% So this is a graph that plots the perceived location 00:49:28.370 --> 00:49:32.410 align:middle line:84% as a function of the delay between the clicks. 00:49:32.410 --> 00:49:34.490 align:middle line:84% So if the delay is zero, that means 00:49:34.490 --> 00:49:36.790 align:middle line:90% the two clicks are synchronous. 00:49:36.790 --> 00:49:39.792 align:middle line:84% And what happens then is because of the symmetry, 00:49:39.792 --> 00:49:41.250 align:middle line:84% you just hear a single sound that's 00:49:41.250 --> 00:49:42.750 align:middle line:90% kind of coming from the middle. 00:49:42.750 --> 00:49:44.690 align:middle line:84% So the perceived location is zero. 00:49:44.690 --> 00:49:48.590 align:middle line:84% But if there's a short delay of a few milliseconds, what happens 00:49:48.590 --> 00:49:51.430 align:middle line:84% is that what you hear is a single sound 00:49:51.430 --> 00:49:55.950 align:middle line:84% at the location of the first click, the leading click. 00:49:55.950 --> 00:49:58.230 align:middle line:84% So that's why it's called the precedence effect. 00:49:58.230 --> 00:50:00.950 align:middle line:84% The location that you hear is dominated 00:50:00.950 --> 00:50:03.790 align:middle line:84% by the initial bit of the sound, the sound that 00:50:03.790 --> 00:50:08.150 align:middle line:84% precedes the rest of the sound-- precedence effect 00:50:08.150 --> 00:50:10.310 align:middle line:84% So this happens for short delays. 00:50:10.310 --> 00:50:13.130 align:middle line:84% And then there's an echo threshold, 00:50:13.130 --> 00:50:15.330 align:middle line:90% which is maybe 5 milliseconds. 00:50:15.330 --> 00:50:18.030 align:middle line:84% And at that point, once the delay gets too long, 00:50:18.030 --> 00:50:20.270 align:middle line:84% you start to hear two separate sounds 00:50:20.270 --> 00:50:24.210 align:middle line:90% at the two actual locations. 00:50:24.210 --> 00:50:26.910 align:middle line:84% So this is a grouping effect because there's two clicks that 00:50:26.910 --> 00:50:28.550 align:middle line:90% are actually being produced. 00:50:28.550 --> 00:50:32.430 align:middle line:84% And under some conditions, you hear them as a single thing. 00:50:32.430 --> 00:50:36.110 align:middle line:84% And the location that you hear is attributed 00:50:36.110 --> 00:50:40.280 align:middle line:90% to the leading click. 00:50:40.280 --> 00:50:45.490 align:middle line:84% And so it seems that the representation of the lagging 00:50:45.490 --> 00:50:47.410 align:middle line:84% click is actually kind of suppressed 00:50:47.410 --> 00:50:48.550 align:middle line:90% in the auditory system. 00:50:48.550 --> 00:50:52.490 align:middle line:84% And evidence for this comes from this related effect here called 00:50:52.490 --> 00:50:54.750 align:middle line:90% discrimination suppression. 00:50:54.750 --> 00:50:57.190 align:middle line:84% So here, the situation is a little more complicated. 00:50:57.190 --> 00:50:58.550 align:middle line:90% We now have three speakers. 00:50:58.550 --> 00:51:00.790 align:middle line:84% So we still have the one that's off to the right. 00:51:00.790 --> 00:51:02.410 align:middle line:90% That's the leading speaker. 00:51:02.410 --> 00:51:06.950 align:middle line:84% But now we have two, one at minus 55, and one at minus 35. 00:51:06.950 --> 00:51:10.370 align:middle line:84% And so these are clicks that are coming second. 00:51:10.370 --> 00:51:13.690 align:middle line:90% And so the task here is-- 00:51:13.690 --> 00:51:16.010 align:middle line:84% so what I should have told you is that on every trial, 00:51:16.010 --> 00:51:18.770 align:middle line:84% there's a click that comes from here first, and then 00:51:18.770 --> 00:51:21.490 align:middle line:84% either a click from here, or a click from here. 00:51:21.490 --> 00:51:25.570 align:middle line:84% And your job is to judge whether the second click came 00:51:25.570 --> 00:51:28.970 align:middle line:90% from minus 55 or minus 35. 00:51:28.970 --> 00:51:32.290 align:middle line:84% And so what this graph is plotting here 00:51:32.290 --> 00:51:37.170 align:middle line:84% is your performance on this task as a function of the delay 00:51:37.170 --> 00:51:39.270 align:middle line:90% between the clicks. 00:51:39.270 --> 00:51:41.690 align:middle line:84% And so the point is that when the delay is short, 00:51:41.690 --> 00:51:44.390 align:middle line:84% you're not able to perform this task. 00:51:44.390 --> 00:51:46.430 align:middle line:84% Once the delay gets longer, you start 00:51:46.430 --> 00:51:49.430 align:middle line:84% being able to discriminate between this location 00:51:49.430 --> 00:51:51.110 align:middle line:90% and this one. 00:51:51.110 --> 00:51:53.150 align:middle line:84% So it's as though that click is kind 00:51:53.150 --> 00:51:55.110 align:middle line:84% of being suppressed by your auditory system 00:51:55.110 --> 00:51:58.910 align:middle line:84% and you lose access to the location information. 00:51:58.910 --> 00:52:02.625 align:middle line:90% Question? 00:52:02.625 --> 00:52:05.110 align:middle line:84% AUDIENCE: If it was just undiscernible, wouldn't 00:52:05.110 --> 00:52:11.910 align:middle line:84% like the bottom be around like 50%, which is like just chance? 00:52:11.910 --> 00:52:15.308 align:middle line:84% Or when the delay is small, are they actually, like... 00:52:15.308 --> 00:52:16.350 align:middle line:90% JOSH MCDERMOTT: Yeah, I-- 00:52:16.350 --> 00:52:18.065 align:middle line:84% AUDIENCE: [INAUDIBLE] perceive it to be-- 00:52:18.065 --> 00:52:19.690 align:middle line:84% JOSH MCDERMOTT: I think you're correct. 00:52:19.690 --> 00:52:21.982 align:middle line:84% I was thinking that to myself as I was explaining this. 00:52:21.982 --> 00:52:24.550 align:middle line:84% I think this is a mistake on the graph, 00:52:24.550 --> 00:52:27.210 align:middle line:84% because you're absolutely right that this should be at chance. 00:52:27.210 --> 00:52:32.970 align:middle line:84% And if this is a 2AFC task, then that would be 50%. 00:52:32.970 --> 00:52:37.530 align:middle line:84% So yeah, this is a schematic summary from a review paper, 00:52:37.530 --> 00:52:39.230 align:middle line:90% which may have an error in it. 00:52:39.230 --> 00:52:41.470 align:middle line:84% But the main thing to take away from this 00:52:41.470 --> 00:52:45.170 align:middle line:84% is that you are unable to discriminate 00:52:45.170 --> 00:52:48.712 align:middle line:90% between these locations when-- 00:52:48.712 --> 00:52:50.170 align:middle line:84% under conditions in which you would 00:52:50.170 --> 00:52:51.878 align:middle line:84% experience the precedence effect, so when 00:52:51.878 --> 00:52:53.490 align:middle line:90% the delay is short. 00:52:53.490 --> 00:52:55.450 align:middle line:84% And I can go hunt down the original graph 00:52:55.450 --> 00:52:57.850 align:middle line:90% and send that to you. 00:52:57.850 --> 00:53:01.690 align:middle line:90% So that's the precedence effect. 00:53:01.690 --> 00:53:03.970 align:middle line:84% And the precedence effect has some other kind 00:53:03.970 --> 00:53:06.690 align:middle line:90% of interesting properties. 00:53:06.690 --> 00:53:09.450 align:middle line:84% One is that it exhibits this phenomenon 00:53:09.450 --> 00:53:11.970 align:middle line:90% of what's called build-up. 00:53:11.970 --> 00:53:15.770 align:middle line:84% And that refers to the fact that you can be out in this regime 00:53:15.770 --> 00:53:18.410 align:middle line:84% here, where the delay between the clicks 00:53:18.410 --> 00:53:20.970 align:middle line:90% is above your threshold. 00:53:20.970 --> 00:53:24.130 align:middle line:84% But if the clicks are presented repeatedly 00:53:24.130 --> 00:53:26.690 align:middle line:84% in the same spatial configuration, what 00:53:26.690 --> 00:53:28.843 align:middle line:84% happens over time is the precedence effect 00:53:28.843 --> 00:53:31.010 align:middle line:84% builds up even in these settings where you initially 00:53:31.010 --> 00:53:31.870 align:middle line:90% didn't hear it. 00:53:31.870 --> 00:53:33.770 align:middle line:84% And so that's what's indicated here. 00:53:33.770 --> 00:53:35.727 align:middle line:84% So you repeatedly present the lead and then 00:53:35.727 --> 00:53:38.310 align:middle line:84% the lag, the lead and then the lag, the lead and then the lag. 00:53:38.310 --> 00:53:39.810 align:middle line:84% And so initially, this is supposed 00:53:39.810 --> 00:53:42.370 align:middle line:84% to be indicating that you're hearing both of them. 00:53:42.370 --> 00:53:44.830 align:middle line:84% And then what happens over time is that you 00:53:44.830 --> 00:53:47.510 align:middle line:90% stop hearing the second click. 00:53:47.510 --> 00:53:50.910 align:middle line:84% And then what you can do if you're an experimenter, 00:53:50.910 --> 00:53:54.230 align:middle line:84% is this sort of tricky thing of actually switching the order. 00:53:54.230 --> 00:53:56.230 align:middle line:84% So now the leading click is on the left, 00:53:56.230 --> 00:53:57.610 align:middle line:90% and the lagging is on the right. 00:53:57.610 --> 00:54:00.510 align:middle line:84% And you now hear both of them, and then it builds up again. 00:54:00.510 --> 00:54:04.910 align:middle line:84% And so here's a graph that shows that effect. 00:54:04.910 --> 00:54:06.390 align:middle line:84% So this is plotting the proportion 00:54:06.390 --> 00:54:10.470 align:middle line:84% of trials on which you report hearing two sounds. 00:54:10.470 --> 00:54:12.910 align:middle line:84% And this is the presentation number. 00:54:12.910 --> 00:54:16.510 align:middle line:84% So initially, there's this particular arrangement 00:54:16.510 --> 00:54:19.410 align:middle line:84% of lead and lag, and then it switches here, 00:54:19.410 --> 00:54:22.652 align:middle line:84% and you initially shoot up to hearing both sounds. 00:54:22.652 --> 00:54:24.110 align:middle line:84% And then it kind of builds up again 00:54:24.110 --> 00:54:26.790 align:middle line:90% over the course of a few trials. 00:54:26.790 --> 00:54:31.150 align:middle line:84% And so the kind of naive interpretation of this 00:54:31.150 --> 00:54:33.910 align:middle line:84% is that this is an adaptation that 00:54:33.910 --> 00:54:37.430 align:middle line:84% helps us localize sounds correctly in environments 00:54:37.430 --> 00:54:39.090 align:middle line:90% that have reflections. 00:54:39.090 --> 00:54:42.690 align:middle line:84% And one explanation for this build up effect 00:54:42.690 --> 00:54:47.170 align:middle line:84% is that if you're in a situation where you're hearing these two 00:54:47.170 --> 00:54:51.370 align:middle line:84% sounds that are always occurring with this very 00:54:51.370 --> 00:54:54.250 align:middle line:84% precise and consistent temporal relationship between the two 00:54:54.250 --> 00:54:57.890 align:middle line:84% of them, and they're otherwise identical, like two clicks, 00:54:57.890 --> 00:55:00.010 align:middle line:84% odds are that relationship is kind of caused 00:55:00.010 --> 00:55:01.230 align:middle line:90% by something in the world. 00:55:01.230 --> 00:55:02.690 align:middle line:84% And a plausible candidate for that 00:55:02.690 --> 00:55:05.090 align:middle line:84% is the positioning of surfaces in the world. 00:55:05.090 --> 00:55:07.610 align:middle line:84% And so the brain is presumably interpreting that these 00:55:07.610 --> 00:55:09.750 align:middle line:90% are likely to be reflections. 00:55:09.750 --> 00:55:11.250 align:middle line:84% So it's almost as though you're kind 00:55:11.250 --> 00:55:16.330 align:middle line:84% of building up a little model of the world's reflections. 00:55:16.330 --> 00:55:20.730 align:middle line:84% So this is kind of-- like the story that we associated 00:55:20.730 --> 00:55:24.510 align:middle line:84% with this effect is a nice story. 00:55:24.510 --> 00:55:26.230 align:middle line:90% It seems plausible. 00:55:26.230 --> 00:55:30.770 align:middle line:84% But it's often really hard to test 00:55:30.770 --> 00:55:33.010 align:middle line:84% whether this is, in fact, like an adaptation 00:55:33.010 --> 00:55:35.010 align:middle line:90% to deal with reflections. 00:55:35.010 --> 00:55:36.050 align:middle line:90% Question over there. 00:55:36.050 --> 00:55:38.070 align:middle line:84% AUDIENCE: So if we get better at this 00:55:38.070 --> 00:55:41.390 align:middle line:84% if it's in a room where there is reflection, 00:55:41.390 --> 00:55:45.110 align:middle line:90% would the delay be less? 00:55:45.110 --> 00:55:47.190 align:middle line:84% JOSH MCDERMOTT: Well, what this is suggesting 00:55:47.190 --> 00:55:51.990 align:middle line:84% is that, no, actually the echo threshold increases. 00:55:51.990 --> 00:55:53.190 align:middle line:90% You're getting worse. 00:55:53.190 --> 00:55:56.430 align:middle line:84% And you're getting worse at detecting 00:55:56.430 --> 00:55:57.970 align:middle line:84% the presence of the lagging sound, 00:55:57.970 --> 00:56:01.430 align:middle line:84% but better from the standpoint of correctly interpreting 00:56:01.430 --> 00:56:04.650 align:middle line:84% the world, like if, in fact, this is caused by one sound, 00:56:04.650 --> 00:56:06.830 align:middle line:90% and these are reflections. 00:56:06.830 --> 00:56:08.450 align:middle line:90% Yeah. 00:56:08.450 --> 00:56:12.110 align:middle line:84% So the question is like, can we actually give this story 00:56:12.110 --> 00:56:16.150 align:middle line:84% that we want to tell about the function of this any teeth? 00:56:16.150 --> 00:56:19.050 align:middle line:84% And again, computational models can help us with this. 00:56:19.050 --> 00:56:20.750 align:middle line:84% So you recall from when we were talking 00:56:20.750 --> 00:56:23.270 align:middle line:84% about sound localization, we discussed 00:56:23.270 --> 00:56:25.870 align:middle line:84% how, if you take artificial neural networks 00:56:25.870 --> 00:56:29.070 align:middle line:84% and train them to localize sounds from binaural audio, 00:56:29.070 --> 00:56:32.030 align:middle line:84% they end up exhibiting some of the behavioral phenomena 00:56:32.030 --> 00:56:36.240 align:middle line:84% that we observe in human listeners. 00:56:36.240 --> 00:56:39.840 align:middle line:84% And it turns out that they also exhibit the precedence effect. 00:56:39.840 --> 00:56:44.520 align:middle line:84% And you may recall how this model 00:56:44.520 --> 00:56:49.880 align:middle line:84% is trained in a simulated world, where we render spatially, 00:56:49.880 --> 00:56:52.640 align:middle line:84% the binaural audio that would enter the ears of a person 00:56:52.640 --> 00:56:55.500 align:middle line:84% in a room listening to sounds at different positions. 00:56:55.500 --> 00:56:58.840 align:middle line:84% And so there are reflections in this world. 00:56:58.840 --> 00:57:02.780 align:middle line:84% So you can then go and test the model on the precedence effect. 00:57:02.780 --> 00:57:04.863 align:middle line:84% So you can render this situation where 00:57:04.863 --> 00:57:06.280 align:middle line:84% you have a click coming from here, 00:57:06.280 --> 00:57:07.800 align:middle line:90% and a click coming from here. 00:57:07.800 --> 00:57:09.503 align:middle line:84% And so this is the binaural audio. 00:57:09.503 --> 00:57:11.920 align:middle line:84% The signal for the left ear, the signal for the right ear. 00:57:11.920 --> 00:57:15.360 align:middle line:84% And so the leading click here, the first one, 00:57:15.360 --> 00:57:16.575 align:middle line:90% is coming from the right. 00:57:16.575 --> 00:57:19.200 align:middle line:84% So you can see this small time difference between the right ear 00:57:19.200 --> 00:57:19.925 align:middle line:90% and the left ear. 00:57:19.925 --> 00:57:21.800 align:middle line:84% The lagging click, the one that comes second, 00:57:21.800 --> 00:57:22.860 align:middle line:90% is coming from the left. 00:57:22.860 --> 00:57:25.920 align:middle line:84% So you can see the left ear signal arrives a little bit 00:57:25.920 --> 00:57:27.760 align:middle line:90% before the right ear. 00:57:27.760 --> 00:57:30.180 align:middle line:84% And then this is the precedence effect in the model. 00:57:30.180 --> 00:57:32.480 align:middle line:84% So this is the model's judged location 00:57:32.480 --> 00:57:35.300 align:middle line:84% as a function of the interclick delay. 00:57:35.300 --> 00:57:39.360 align:middle line:84% Now, this model is constrained to report a single location. 00:57:39.360 --> 00:57:41.840 align:middle line:84% So it can't tell us that it hears two things. 00:57:41.840 --> 00:57:45.700 align:middle line:84% But the location that it does ascribe to the stimulus 00:57:45.700 --> 00:57:50.800 align:middle line:84% is that of the leading click when the delay is short, 00:57:50.800 --> 00:57:52.540 align:middle line:90% and then that kind of goes away. 00:57:52.540 --> 00:57:54.900 align:middle line:84% So it looks an awful lot like the precedence effect. 00:57:54.900 --> 00:57:56.860 align:middle line:90% So that's nice. 00:57:56.860 --> 00:57:59.980 align:middle line:84% But the other thing that we can do with a model like this is we 00:57:59.980 --> 00:58:02.300 align:middle line:84% can ask, well, how would the model 00:58:02.300 --> 00:58:05.180 align:middle line:84% behave if it had evolved and developed 00:58:05.180 --> 00:58:07.420 align:middle line:90% in a different kind of world? 00:58:07.420 --> 00:58:10.140 align:middle line:84% And in particular, because this model 00:58:10.140 --> 00:58:16.060 align:middle line:84% was trained in a simulator, we can mess with the simulator 00:58:16.060 --> 00:58:19.820 align:middle line:84% to make it deviate from the natural world in different ways. 00:58:19.820 --> 00:58:23.320 align:middle line:84% So we can train the model in anechoic conditions. 00:58:23.320 --> 00:58:26.300 align:middle line:84% So this is a situation in which every surface absorbs 00:58:26.300 --> 00:58:28.040 align:middle line:90% all the sound that it receives. 00:58:28.040 --> 00:58:32.060 align:middle line:84% It's like if you grew up and evolved in anechoic chambers, 00:58:32.060 --> 00:58:34.240 align:middle line:84% we can get rid of background noise, make the sounds 00:58:34.240 --> 00:58:37.920 align:middle line:84% unnatural, different kinds of alternative training regimes. 00:58:37.920 --> 00:58:40.920 align:middle line:84% And so then we can optimize the model 00:58:40.920 --> 00:58:43.120 align:middle line:84% in these different kinds of worlds, 00:58:43.120 --> 00:58:44.760 align:middle line:84% and then test it behaviorally and see 00:58:44.760 --> 00:58:47.100 align:middle line:90% if it exhibits this phenomenon. 00:58:47.100 --> 00:58:49.320 align:middle line:84% And what this graph is showing you 00:58:49.320 --> 00:58:51.840 align:middle line:84% is that in several of these alternative training 00:58:51.840 --> 00:58:54.840 align:middle line:84% conditions-- so I should say the natural one, or the normal one, 00:58:54.840 --> 00:58:55.590 align:middle line:90% is the light blue. 00:58:55.590 --> 00:58:58.215 align:middle line:84% And then in two of these other alternative training conditions, 00:58:58.215 --> 00:59:00.160 align:middle line:84% you still see the precedence effect. 00:59:00.160 --> 00:59:03.080 align:middle line:84% But look what happens in anechoic conditions. 00:59:03.080 --> 00:59:05.800 align:middle line:84% So if the model is trained in conditions without reflections, 00:59:05.800 --> 00:59:07.640 align:middle line:90% that effect goes away. 00:59:07.640 --> 00:59:12.080 align:middle line:84% So that's kind consistent with the kind of classical story 00:59:12.080 --> 00:59:14.520 align:middle line:84% that this is a mechanism that has evolved 00:59:14.520 --> 00:59:17.680 align:middle line:84% to help us cope with reflections and the challenge 00:59:17.680 --> 00:59:19.920 align:middle line:84% that they pose to sound localization. 00:59:19.920 --> 00:59:23.460 align:middle line:90% 00:59:23.460 --> 00:59:26.402 align:middle line:84% So that's the precedence effect, and evidence 00:59:26.402 --> 00:59:28.860 align:middle line:84% that this is something that helps us deal with reflections. 00:59:28.860 --> 00:59:32.860 align:middle line:84% The bigger picture here is that this new era of machine 00:59:32.860 --> 00:59:35.740 align:middle line:84% learning-driven modeling can give us some ways 00:59:35.740 --> 00:59:39.140 align:middle line:84% to ask why our perceptual systems are the way they are, 00:59:39.140 --> 00:59:42.020 align:middle line:84% and what are the problems that they were actually 00:59:42.020 --> 00:59:44.137 align:middle line:90% designed to solve. 00:59:44.137 --> 00:59:45.720 align:middle line:84% What questions do you have about that? 00:59:45.720 --> 00:59:51.660 align:middle line:90% 00:59:51.660 --> 00:59:52.500 align:middle line:90% Yeah. 00:59:52.500 --> 00:59:55.540 align:middle line:84% AUDIENCE: How does that even model sound waves? 00:59:55.540 --> 00:59:57.820 align:middle line:84% Like when it comes to [INAUDIBLE]. 00:59:57.820 --> 00:59:59.900 align:middle line:84% I mean, all the presentations we've 00:59:59.900 --> 01:00:02.940 align:middle line:90% seen [INAUDIBLE] single lines. 01:00:02.940 --> 01:00:06.260 align:middle line:84% So what's the intuition for how do you manage that in a 3D 01:00:06.260 --> 01:00:07.180 align:middle line:90% space? 01:00:07.180 --> 01:00:09.940 align:middle line:84% JOSH MCDERMOTT: Yeah, I mean, so this is probably 01:00:09.940 --> 01:00:11.380 align:middle line:90% a respect in which-- 01:00:11.380 --> 01:00:13.960 align:middle line:84% I mean, so there's a simulator that kind of does it. 01:00:13.960 --> 01:00:19.020 align:middle line:84% And it treats them as though they are traveling along lines, 01:00:19.020 --> 01:00:20.520 align:middle line:84% and that that's an approximation. 01:00:20.520 --> 01:00:23.580 align:middle line:90% It's not fully accurate. 01:00:23.580 --> 01:00:26.020 align:middle line:84% All this stuff is suggesting that it's probably 01:00:26.020 --> 01:00:29.420 align:middle line:84% good enough for our purposes, but it's not perfect. 01:00:29.420 --> 01:00:31.900 align:middle line:84% And those are things that will get better with time. 01:00:31.900 --> 01:00:37.020 align:middle line:90% 01:00:37.020 --> 01:00:40.720 align:middle line:84% So here's a quick summary of these auditory grouping cues 01:00:40.720 --> 01:00:41.900 align:middle line:90% that we've discussed. 01:00:41.900 --> 01:00:45.280 align:middle line:84% We talked about common onset, comodulation, harmonicity, 01:00:45.280 --> 01:00:47.840 align:middle line:84% we talked about how there's this ambiguous role for frequency 01:00:47.840 --> 01:00:49.920 align:middle line:84% modulation, talked about repetition, 01:00:49.920 --> 01:00:54.060 align:middle line:84% and then some role for spatial cues. 01:00:54.060 --> 01:00:57.040 align:middle line:84% And all of the things that we've been discussing so far really 01:00:57.040 --> 01:01:00.160 align:middle line:84% relate to how the auditory system assigns energy that's 01:01:00.160 --> 01:01:04.100 align:middle line:84% kind of received in of local neighborhood in time, 01:01:04.100 --> 01:01:05.485 align:middle line:90% different sources in the world. 01:01:05.485 --> 01:01:07.360 align:middle line:84% But there are also a whole bunch of phenomena 01:01:07.360 --> 01:01:10.080 align:middle line:84% that relate to the grouping of sound energy 01:01:10.080 --> 01:01:11.820 align:middle line:90% over longer periods of time. 01:01:11.820 --> 01:01:14.320 align:middle line:84% And this is often referred to as streaming. 01:01:14.320 --> 01:01:18.700 align:middle line:84% And so I'm going to give you a few examples of that. 01:01:18.700 --> 01:01:22.720 align:middle line:84% So this is the most classic streaming phenomenon. 01:01:22.720 --> 01:01:25.580 align:middle line:84% This was really introduced back in the 1970s. 01:01:25.580 --> 01:01:28.208 align:middle line:84% And so back then, really people were very limited 01:01:28.208 --> 01:01:30.500 align:middle line:84% in terms of the kinds of sounds that you could actually 01:01:30.500 --> 01:01:31.940 align:middle line:84% do experiments on, and so there were 01:01:31.940 --> 01:01:33.060 align:middle line:84% lots of experiments that were done 01:01:33.060 --> 01:01:34.360 align:middle line:90% on beeps and boops and noises. 01:01:34.360 --> 01:01:35.972 align:middle line:90% That was pretty much it. 01:01:35.972 --> 01:01:38.680 align:middle line:84% But they nonetheless, discovered some fairly interesting effects. 01:01:38.680 --> 01:01:41.100 align:middle line:84% And so in this particular stimulus, 01:01:41.100 --> 01:01:44.380 align:middle line:84% you will be presented with alternating high and low tones. 01:01:44.380 --> 01:01:47.160 align:middle line:84% So a high tone, a low tone, a high tone, a high tone, 01:01:47.160 --> 01:01:48.420 align:middle line:90% a low tone, a high tone. 01:01:48.420 --> 01:01:51.540 align:middle line:90% 01:01:51.540 --> 01:01:54.420 align:middle line:84% This sequence of tones can be perceived 01:01:54.420 --> 01:01:56.340 align:middle line:84% as part of a single process, kind 01:01:56.340 --> 01:01:59.060 align:middle line:84% think of that as a single melody, in which case 01:01:59.060 --> 01:02:00.780 align:middle line:90% there's a galloping rhythm. 01:02:00.780 --> 01:02:05.158 align:middle line:84% 1, 2, 3; 1, 2, 3; 1, 2, 3; or as two separate streams, 01:02:05.158 --> 01:02:06.700 align:middle line:84% one with high-frequency tones and one 01:02:06.700 --> 01:02:09.140 align:middle line:90% with low-frequency tones. 01:02:09.140 --> 01:02:13.580 align:middle line:84% So in general, if the difference in the frequency gets too great, 01:02:13.580 --> 01:02:16.160 align:middle line:84% or if the time between the tones gets too short, 01:02:16.160 --> 01:02:18.540 align:middle line:84% people will hear two separate streams. 01:02:18.540 --> 01:02:22.380 align:middle line:84% Now, you might think that this distinction between one and two 01:02:22.380 --> 01:02:27.588 align:middle line:84% streams is kind of like a weird and subjective distinction. 01:02:27.588 --> 01:02:29.880 align:middle line:84% It's like, well, obviously like this could be one thing 01:02:29.880 --> 01:02:31.300 align:middle line:90% or it could be two things. 01:02:31.300 --> 01:02:34.040 align:middle line:84% But it just turns out that, in practice, 01:02:34.040 --> 01:02:38.760 align:middle line:84% when you listen to this, there's a pretty clear, subjective sense 01:02:38.760 --> 01:02:42.280 align:middle line:84% that it either sounds like one thing or two. 01:02:42.280 --> 01:02:44.860 align:middle line:84% And it's very consistent, and it works for lots of people. 01:02:44.860 --> 01:02:48.853 align:middle line:90% So let's have a listen. 01:02:48.853 --> 01:02:49.520 align:middle line:90% [AUDIO PLAYBACK] 01:02:49.520 --> 01:02:52.600 align:middle line:84% - Demonstration three, loss of rhythmic information 01:02:52.600 --> 01:02:55.120 align:middle line:84% as a result of stream segregation. 01:02:55.120 --> 01:02:58.280 align:middle line:84% You will hear a cycle of two high tones and one low tone 01:02:58.280 --> 01:03:00.000 align:middle line:90% in a galloping pattern. 01:03:00.000 --> 01:03:02.760 align:middle line:84% Listen to what happens to the experience of the gallop 01:03:02.760 --> 01:03:06.120 align:middle line:84% as we speed up the sequence by shortening the tones. 01:03:06.120 --> 01:03:10.240 align:middle line:84% First, the tones are far apart in frequency. 01:03:10.240 --> 01:03:16.240 align:middle line:84% [GALLOPING HIGH AND LOW TONES INCREASING IN SPEED] 01:03:16.240 --> 01:03:17.233 align:middle line:90% [END PLAYBACK] 01:03:17.233 --> 01:03:18.900 align:middle line:84% JOSH MCDERMOTT: And I should have said-- 01:03:18.900 --> 01:03:21.040 align:middle line:90% so we got the tones far apart. 01:03:21.040 --> 01:03:22.840 align:middle line:84% Initially, they're probably going 01:03:22.840 --> 01:03:24.840 align:middle line:84% to sound like they're two distinct things. 01:03:24.840 --> 01:03:28.860 align:middle line:84% What probably will happen is as they speed up at the very end-- 01:03:28.860 --> 01:03:31.540 align:middle line:84% or sorry, initially, because they're so slow, 01:03:31.540 --> 01:03:34.360 align:middle line:84% you will hear them kind of potentially as a single melody. 01:03:34.360 --> 01:03:37.560 align:middle line:84% But what will happen as they speed up is it's going to split. 01:03:37.560 --> 01:03:40.060 align:middle line:84% And you'll start to hear this high beep-beep-beep 01:03:40.060 --> 01:03:41.173 align:middle line:90% and a low beep-beep-beep 01:03:41.173 --> 01:03:41.840 align:middle line:90% [AUDIO PLAYBACK] 01:03:41.840 --> 01:04:03.260 align:middle line:84% [HIGH AND LOW TONES BEEPING WITH INCREASING SPEED] 01:04:03.260 --> 01:04:06.750 align:middle line:84% - Next the frequency separation is small. 01:04:06.750 --> 01:04:07.600 align:middle line:90% [END PLAYBACK] 01:04:07.600 --> 01:04:10.200 align:middle line:84% JOSH MCDERMOTT: And so now the frequency separation is small, 01:04:10.200 --> 01:04:11.940 align:middle line:84% and so there's going to be a tendency for you 01:04:11.940 --> 01:04:13.120 align:middle line:90% to hear this as one stream. 01:04:13.120 --> 01:04:15.560 align:middle line:84% So you're going to still get this effect of speeding up, 01:04:15.560 --> 01:04:17.340 align:middle line:84% but you'll probably hear it as one thing 01:04:17.340 --> 01:04:18.480 align:middle line:90% until the very, very end. 01:04:18.480 --> 01:04:19.593 align:middle line:90% So see what happens. 01:04:19.593 --> 01:04:20.260 align:middle line:90% [AUDIO PLAYBACK] 01:04:20.260 --> 01:04:45.160 align:middle line:84% [HIGH AND LOW TONES BEEPING WITH INCREASING SPEED] 01:04:45.160 --> 01:04:46.200 align:middle line:90% [END PLAYBACK] 01:04:46.200 --> 01:04:48.980 align:middle line:84% JOSH MCDERMOTT: So at the very end, they split for me. 01:04:48.980 --> 01:04:52.360 align:middle line:84% I don't know what happened to you. 01:04:52.360 --> 01:04:55.600 align:middle line:84% So this seems like a very subjective phenomenon, 01:04:55.600 --> 01:04:59.160 align:middle line:84% but it turns out that there's this fairly interesting 01:04:59.160 --> 01:05:03.920 align:middle line:84% objective consequence to this difference in subjective state, 01:05:03.920 --> 01:05:05.880 align:middle line:90% and that's what's stated here. 01:05:05.880 --> 01:05:09.280 align:middle line:84% So when a sequence is parsed by your auditory system 01:05:09.280 --> 01:05:12.480 align:middle line:84% into two streams, it becomes very difficult for you 01:05:12.480 --> 01:05:15.880 align:middle line:84% to judge temporal relations between the tones 01:05:15.880 --> 01:05:19.360 align:middle line:84% or the elements of the two streams. 01:05:19.360 --> 01:05:21.920 align:middle line:84% I mean, it's a very interesting and important phenomenon 01:05:21.920 --> 01:05:24.420 align:middle line:84% because it suggests that there is 01:05:24.420 --> 01:05:26.700 align:middle line:84% some fundamental change in the way 01:05:26.700 --> 01:05:28.680 align:middle line:90% that the sound is represented. 01:05:28.680 --> 01:05:32.780 align:middle line:90% So here's the essential idea. 01:05:32.780 --> 01:05:34.980 align:middle line:90% So you're going to-- 01:05:34.980 --> 01:05:36.158 align:middle line:90% there's my cursor. 01:05:36.158 --> 01:05:38.200 align:middle line:84% So you're going to hear these high and low tones. 01:05:38.200 --> 01:05:41.535 align:middle line:84% And in one case, the frequency separation will be small, 01:05:41.535 --> 01:05:43.660 align:middle line:84% and so you're going to be a lot more likely to hear 01:05:43.660 --> 01:05:44.712 align:middle line:90% this as one thing. 01:05:44.712 --> 01:05:46.420 align:middle line:84% In another case, it'll get big, so you're 01:05:46.420 --> 01:05:49.220 align:middle line:84% going to tend to hear it as two streams. 01:05:49.220 --> 01:05:51.220 align:middle line:84% And what we're going to be looking at 01:05:51.220 --> 01:05:55.220 align:middle line:84% is the extent to which you can tell whether the low tone is 01:05:55.220 --> 01:05:57.620 align:middle line:84% kind of centered right between the two high tones 01:05:57.620 --> 01:05:59.380 align:middle line:90% or offset a little bit. 01:05:59.380 --> 01:06:03.500 align:middle line:84% And the phenomenon that you'll probably notice 01:06:03.500 --> 01:06:06.040 align:middle line:84% is that when the frequency separation is small, 01:06:06.040 --> 01:06:08.140 align:middle line:84% so that you tend to hear this as one stream, 01:06:08.140 --> 01:06:11.100 align:middle line:84% it's pretty easy to tell the difference between those two 01:06:11.100 --> 01:06:11.980 align:middle line:90% rhythms. 01:06:11.980 --> 01:06:15.215 align:middle line:84% Whereas when the frequency separation gets big, 01:06:15.215 --> 01:06:17.340 align:middle line:84% that distinction kind of gets lost because you just 01:06:17.340 --> 01:06:18.900 align:middle line:90% hear it as two streams. 01:06:18.900 --> 01:06:20.900 align:middle line:84% So you still have that same physical time 01:06:20.900 --> 01:06:23.000 align:middle line:84% offset between the two tones, but you're not going 01:06:23.000 --> 01:06:25.020 align:middle line:90% to really hear it very clearly. 01:06:25.020 --> 01:06:28.000 align:middle line:90% 01:06:28.000 --> 01:06:29.480 align:middle line:84% And actually, I'm not going to show 01:06:29.480 --> 01:06:30.720 align:middle line:84% you the spectrogram for this because you 01:06:30.720 --> 01:06:31.660 align:middle line:90% could use it to cheat. 01:06:31.660 --> 01:06:33.413 align:middle line:90% So let's listen. 01:06:33.413 --> 01:06:34.080 align:middle line:90% [AUDIO PLAYBACK] 01:06:34.080 --> 01:06:35.360 align:middle line:90% - Demonstration 13. 01:06:35.360 --> 01:06:37.000 align:middle line:84% The effects of stream segregation 01:06:37.000 --> 01:06:38.960 align:middle line:90% on the judgment of timing. 01:06:38.960 --> 01:06:41.920 align:middle line:84% We start with regular repetitions of a single pitch 01:06:41.920 --> 01:06:44.320 align:middle line:84% and occasionally add a lower tone 01:06:44.320 --> 01:06:46.640 align:middle line:90% to create a galloping rhythm. 01:06:46.640 --> 01:06:48.760 align:middle line:84% Sometimes the low tone is halfway 01:06:48.760 --> 01:06:51.640 align:middle line:84% in time between the high ones, and sometimes closer 01:06:51.640 --> 01:06:53.040 align:middle line:90% to the second one. 01:06:53.040 --> 01:06:54.480 align:middle line:84% In each sequence, you are to judge 01:06:54.480 --> 01:06:59.000 align:middle line:84% whether the low tone is exactly halfway between the high ones. 01:06:59.000 --> 01:07:00.880 align:middle line:84% First, we present two examples in which 01:07:00.880 --> 01:07:04.960 align:middle line:84% the low tone is close in frequency to the high one. 01:07:04.960 --> 01:07:08.240 align:middle line:84% Next, we present two examples in which the low tone is far 01:07:08.240 --> 01:07:11.255 align:middle line:90% in frequency from the high one. 01:07:11.255 --> 01:07:27.218 align:middle line:84% [GALLOPING RHYTHM OF LOW AND HIGH TONES] 01:07:27.218 --> 01:07:29.260 align:middle line:84% JOSH MCDERMOTT: OK, could you hear the difference 01:07:29.260 --> 01:07:32.020 align:middle line:90% between those two rhythms? 01:07:32.020 --> 01:07:35.660 align:middle line:84% Yeah like one-- it's a little bit subtle, but one, just-- 01:07:35.660 --> 01:07:37.553 align:middle line:90% the timing sounds different. 01:07:37.553 --> 01:07:39.220 align:middle line:84% So now we're going to do the same thing, 01:07:39.220 --> 01:07:41.860 align:middle line:84% but with this bigger frequency separation. 01:07:41.860 --> 01:07:56.837 align:middle line:84% [HIGH-FREQUENCY RHYTHM WITH LOW TONE SEPARATION] 01:07:56.837 --> 01:07:57.420 align:middle line:90% [END PLAYBACK] 01:07:57.420 --> 01:08:00.180 align:middle line:84% JOSH MCDERMOTT: So the argument is that in this setting 01:08:00.180 --> 01:08:03.360 align:middle line:84% here, where we have this big frequency separation, 01:08:03.360 --> 01:08:05.420 align:middle line:84% the subjective difference between the timing 01:08:05.420 --> 01:08:06.160 align:middle line:90% is very subtle. 01:08:06.160 --> 01:08:08.220 align:middle line:84% And objectively, you can measure people's ability 01:08:08.220 --> 01:08:11.480 align:middle line:84% to discriminate and show that they're worse. 01:08:11.480 --> 01:08:12.920 align:middle line:84% So that's just an important fact, 01:08:12.920 --> 01:08:17.240 align:middle line:84% and an objective consequence to the streaming phenomenon. 01:08:17.240 --> 01:08:20.548 align:middle line:90% 01:08:20.548 --> 01:08:22.420 align:middle line:90% So there's this kind of-- 01:08:22.420 --> 01:08:23.880 align:middle line:84% one way to think about the problem 01:08:23.880 --> 01:08:26.800 align:middle line:84% that we've been talking about, where you get 01:08:26.800 --> 01:08:28.420 align:middle line:90% some auditory input like this. 01:08:28.420 --> 01:08:32.439 align:middle line:84% It's just energy that's spread across frequency and over time. 01:08:32.439 --> 01:08:35.439 align:middle line:84% And you could think of the scene analysis problem 01:08:35.439 --> 01:08:38.840 align:middle line:84% as possibly initially requiring you 01:08:38.840 --> 01:08:41.819 align:middle line:84% to determine the grouping of the observed sound elements. 01:08:41.819 --> 01:08:45.200 align:middle line:84% And so what's shown here is mixtures 01:08:45.200 --> 01:08:48.279 align:middle line:84% of the target talker with a bunch of additional talkers. 01:08:48.279 --> 01:08:53.319 align:middle line:84% And here, the pixels in the cochleagram representation 01:08:53.319 --> 01:08:55.800 align:middle line:84% have been colored red if they correspond 01:08:55.800 --> 01:08:57.500 align:middle line:90% to distractor sources. 01:08:57.500 --> 01:09:00.300 align:middle line:84% So this is energy from the target talker, 01:09:00.300 --> 01:09:02.460 align:middle line:84% and then this is from some other talker. 01:09:02.460 --> 01:09:04.000 align:middle line:84% So you might imagine that you need 01:09:04.000 --> 01:09:07.080 align:middle line:84% to be able to figure out which bits of energy 01:09:07.080 --> 01:09:10.640 align:middle line:84% belong to one source and which belong to another source. 01:09:10.640 --> 01:09:13.600 align:middle line:84% But there's this other thing that happens. 01:09:13.600 --> 01:09:16.500 align:middle line:84% You can see that a lot of the pixels are colored green. 01:09:16.500 --> 01:09:20.540 align:middle line:84% And so those are places where the target talker had 01:09:20.540 --> 01:09:24.340 align:middle line:84% a fair bit of energy, but where all the other talkers actually 01:09:24.340 --> 01:09:25.899 align:middle line:90% have more energy. 01:09:25.899 --> 01:09:29.340 align:middle line:84% And so that's a situation where the target 01:09:29.340 --> 01:09:32.840 align:middle line:90% talker is going to be masked. 01:09:32.840 --> 01:09:35.660 align:middle line:84% The other sources will make it difficult to actually detect 01:09:35.660 --> 01:09:39.040 align:middle line:84% the content of the target source at those frequencies 01:09:39.040 --> 01:09:40.740 align:middle line:90% at those moments in time. 01:09:40.740 --> 01:09:43.819 align:middle line:84% And as we've discussed, because of the fact that 01:09:43.819 --> 01:09:48.600 align:middle line:84% sounds add in the air, masking is ubiquitous in the world. 01:09:48.600 --> 01:09:51.460 align:middle line:84% Things are always masking other things. 01:09:51.460 --> 01:09:55.140 align:middle line:84% And it seems plausible that you might 01:09:55.140 --> 01:09:58.300 align:middle line:84% want to be able to infer the content of sources 01:09:58.300 --> 01:09:59.880 align:middle line:84% in these places where they're masked. 01:09:59.880 --> 01:10:01.860 align:middle line:84% And in fact, there's a fair bit of evidence 01:10:01.860 --> 01:10:04.620 align:middle line:90% that this happens all the time. 01:10:04.620 --> 01:10:07.220 align:middle line:84% So this is a classic effect that's often 01:10:07.220 --> 01:10:08.760 align:middle line:90% known as the continuity effect. 01:10:08.760 --> 01:10:12.100 align:middle line:84% And it consists of a stimulus where 01:10:12.100 --> 01:10:16.300 align:middle line:84% you have tones that are separated by a gap, 01:10:16.300 --> 01:10:19.840 align:middle line:84% and, in some cases, the gap will be filled by noise. 01:10:19.840 --> 01:10:23.600 align:middle line:84% And the phenomenon is that if the noise is sufficiently high 01:10:23.600 --> 01:10:27.560 align:middle line:84% in intensity and contains frequencies close to those 01:10:27.560 --> 01:10:30.640 align:middle line:84% of the tone, you will subjectively 01:10:30.640 --> 01:10:33.520 align:middle line:84% hear the tone as continuing during the noise, 01:10:33.520 --> 01:10:36.380 align:middle line:84% even though the tone is not physically present there. 01:10:36.380 --> 01:10:41.240 align:middle line:84% And the idea is that the brain is inferring 01:10:41.240 --> 01:10:43.360 align:middle line:84% that the tone's absence during the noise 01:10:43.360 --> 01:10:47.600 align:middle line:84% is best explained by masking, and so it gets filled in. 01:10:47.600 --> 01:10:51.460 align:middle line:84% So the idea is that here's a stimulus-- again, 01:10:51.460 --> 01:10:52.750 align:middle line:90% this is sort of spectrogram. 01:10:52.750 --> 01:10:53.500 align:middle line:90% You got your tone. 01:10:53.500 --> 01:10:54.840 align:middle line:90% You got your noise. 01:10:54.840 --> 01:10:57.860 align:middle line:84% And there's two possible explanations for the stimulus-- 01:10:57.860 --> 01:10:59.120 align:middle line:90% at least two. 01:10:59.120 --> 01:11:01.720 align:middle line:84% One is that you have two tones that just kind of happen 01:11:01.720 --> 01:11:04.660 align:middle line:90% to be sandwiching this noise. 01:11:04.660 --> 01:11:07.680 align:middle line:84% The other is that the noise is continuous-- sorry, 01:11:07.680 --> 01:11:10.840 align:middle line:84% the tone is continuous, and that the reason that you don't 01:11:10.840 --> 01:11:13.080 align:middle line:84% actually have physical evidence for the tone here 01:11:13.080 --> 01:11:16.260 align:middle line:84% is because it gets masked by the noise. 01:11:16.260 --> 01:11:18.460 align:middle line:84% And most of the time, this is what 01:11:18.460 --> 01:11:22.160 align:middle line:84% you hear if the noise is sufficiently high in intensity 01:11:22.160 --> 01:11:24.540 align:middle line:84% so as to plausibly mask the tone. 01:11:24.540 --> 01:11:25.680 align:middle line:90% let's check this out. 01:11:25.680 --> 01:11:29.800 align:middle line:90% 01:11:29.800 --> 01:11:30.980 align:middle line:90% [AUDIO PLAYBACK] 01:11:30.980 --> 01:11:34.620 align:middle line:90% - Pulsation threshold. 01:11:34.620 --> 01:11:38.900 align:middle line:84% You will hear a 2,000 hertz tone alternating with a band of noise 01:11:38.900 --> 01:11:41.700 align:middle line:90% centered around 2,000 hertz. 01:11:41.700 --> 01:11:46.260 align:middle line:84% The tone intensity decreases one decibel after every four tone 01:11:46.260 --> 01:11:48.060 align:middle line:90% presentations. 01:11:48.060 --> 01:11:52.943 align:middle line:84% Notice when the tone begins to appear continuous. 01:11:52.943 --> 01:11:54.860 align:middle line:84% JOSH MCDERMOTT: So just to make sure everybody 01:11:54.860 --> 01:11:56.180 align:middle line:84% gets what's happening, you're going 01:11:56.180 --> 01:11:58.263 align:middle line:84% to hear this tone-- beep, beep, beep, beep, beep-- 01:11:58.263 --> 01:11:59.420 align:middle line:90% alternating with noise. 01:11:59.420 --> 01:12:01.920 align:middle line:84% Initially, the tone is going to be pretty high in intensity. 01:12:01.920 --> 01:12:03.740 align:middle line:84% It's going to gradually decrease. 01:12:03.740 --> 01:12:07.020 align:middle line:84% It will reach a point at which the tone is low enough in level 01:12:07.020 --> 01:12:09.873 align:middle line:84% that it could plausibly be masked by the noise. 01:12:09.873 --> 01:12:11.540 align:middle line:84% And at that point, you're going to start 01:12:11.540 --> 01:12:14.430 align:middle line:84% to hear this thing as continuous, 01:12:14.430 --> 01:12:16.950 align:middle line:84% even though the tone is always going to be pulsing. 01:12:16.950 --> 01:12:20.373 align:middle line:90% [HIGH-FREQUENCY TONE PULSES] 01:12:20.373 --> 01:12:32.598 align:middle line:90% 01:12:32.598 --> 01:12:37.488 align:middle line:90% [TONE PULSE CONTINUOUS] 01:12:37.488 --> 01:12:38.138 align:middle line:90% [END PLAYBACK] 01:12:38.138 --> 01:12:40.430 align:middle line:84% JOSH MCDERMOTT: At the end, were you hearing that beep? 01:12:40.430 --> 01:12:40.930 align:middle line:90% Yeah. 01:12:40.930 --> 01:12:41.870 align:middle line:90% OK. 01:12:41.870 --> 01:12:46.510 align:middle line:84% So that's what's known as the continuity effect. 01:12:46.510 --> 01:12:49.350 align:middle line:90% And it happens for tones. 01:12:49.350 --> 01:12:50.950 align:middle line:90% It also happens for speech. 01:12:50.950 --> 01:12:56.350 align:middle line:84% And speech is often referred to as phonemic restoration. 01:12:56.350 --> 01:13:00.230 align:middle line:84% And this is a famous picture that 01:13:00.230 --> 01:13:03.067 align:middle line:84% is thought of as a visual analog of this, where 01:13:03.067 --> 01:13:04.650 align:middle line:84% what we're going to be comparing here, 01:13:04.650 --> 01:13:07.790 align:middle line:84% it's just like going to be like the tone, where we can have 01:13:07.790 --> 01:13:11.030 align:middle line:84% tones separated by gaps that can be filled with noise or not. 01:13:11.030 --> 01:13:13.330 align:middle line:84% And in this case, we've got these letters that 01:13:13.330 --> 01:13:15.275 align:middle line:90% are occluded by of splatter. 01:13:15.275 --> 01:13:16.650 align:middle line:84% And if you just get the fragments 01:13:16.650 --> 01:13:19.937 align:middle line:84% of the letters on their own, it's kind of hard to interpret. 01:13:19.937 --> 01:13:22.270 align:middle line:84% But when you have visual evidence that they're occluded, 01:13:22.270 --> 01:13:24.890 align:middle line:84% in this case, it's a lot easier to see that they're Bs. 01:13:24.890 --> 01:13:27.070 align:middle line:84% So the speech demo is kind of the analog of that. 01:13:27.070 --> 01:13:28.695 align:middle line:84% So we're going to have a speech signal. 01:13:28.695 --> 01:13:31.950 align:middle line:84% Little bits of it will be deleted, replaced by silence. 01:13:31.950 --> 01:13:33.930 align:middle line:84% And then in some cases, those silences 01:13:33.930 --> 01:13:35.370 align:middle line:90% will be filled by noise. 01:13:35.370 --> 01:13:39.010 align:middle line:84% And we're going to ask whether it's easier to hear the speech 01:13:39.010 --> 01:13:40.570 align:middle line:90% when the noise is present. 01:13:40.570 --> 01:13:43.855 align:middle line:90% 01:13:43.855 --> 01:13:45.290 align:middle line:90% [AUDIO PLAYBACK] 01:13:45.290 --> 01:13:47.010 align:middle line:90% - Demonstration 31. 01:13:47.010 --> 01:13:49.850 align:middle line:84% The picket fence effect with speech. 01:13:49.850 --> 01:13:53.690 align:middle line:84% We play a sentence in which half of the sound has been eliminated 01:13:53.690 --> 01:13:58.250 align:middle line:84% by taking out every other 1/6 second segment and replacing it 01:13:58.250 --> 01:13:59.810 align:middle line:90% with a silence. 01:13:59.810 --> 01:14:03.170 align:middle line:84% Then the same sentence is played with loud noise bursts 01:14:03.170 --> 01:14:05.370 align:middle line:90% replacing the silences. 01:14:05.370 --> 01:14:08.810 align:middle line:84% Does this make the speech sound more complete? 01:14:08.810 --> 01:14:12.616 align:middle line:84% Finally, you will hear the intact sentence. 01:14:12.616 --> 01:14:14.550 align:middle line:84% - (CHOPPILY) Auditory scene analysis 01:14:14.550 --> 01:14:16.590 align:middle line:90% involves a grouping of sounds. 01:14:16.590 --> 01:14:20.736 align:middle line:84% The principle of similarity is very important. 01:14:20.736 --> 01:14:22.710 align:middle line:84% (WITH STATIC PULSES) Auditory scene analysis 01:14:22.710 --> 01:14:24.710 align:middle line:90% involves a grouping of sounds. 01:14:24.710 --> 01:14:28.910 align:middle line:84% The principle of similarity is very important. 01:14:28.910 --> 01:14:32.790 align:middle line:84% (NORMAL) Auditory scene analysis involves the grouping of sounds. 01:14:32.790 --> 01:14:35.690 align:middle line:84% The principle of similarity is very important. 01:14:35.690 --> 01:14:38.470 align:middle line:90% 01:14:38.470 --> 01:14:41.630 align:middle line:84% - Again, half the speech is deleted, but this time, 01:14:41.630 --> 01:14:45.630 align:middle line:84% every other quarter-second segment has been deleted. 01:14:45.630 --> 01:14:48.829 align:middle line:84% Again, judge the apparent completeness of the sentence. 01:14:48.829 --> 01:14:50.246 align:middle line:84% - (CHOPPILY WITH LONGER DELETIONS) 01:14:50.246 --> 01:14:53.950 align:middle line:84% Auditory scene analysis involves grouping of sounds. 01:14:53.950 --> 01:14:58.025 align:middle line:84% The principle of similarity is very important. 01:14:58.025 --> 01:15:00.150 align:middle line:84% (WITH LONGER STATIC PULSES) Auditory scene analysis 01:15:00.150 --> 01:15:02.110 align:middle line:90% involves grouping of sounds. 01:15:02.110 --> 01:15:05.047 align:middle line:84% The principle of similarity is very important. 01:15:05.047 --> 01:15:05.630 align:middle line:90% [END PLAYBACK] 01:15:05.630 --> 01:15:06.550 align:middle line:84% JOSH MCDERMOTT: So did you feel like you 01:15:06.550 --> 01:15:08.910 align:middle line:84% were hearing the speech when the noise was is present? 01:15:08.910 --> 01:15:09.410 align:middle line:90% Yeah. 01:15:09.410 --> 01:15:11.010 align:middle line:90% So that's phonemic restoration. 01:15:11.010 --> 01:15:13.730 align:middle line:84% And did you have the sense that it worked better 01:15:13.730 --> 01:15:14.970 align:middle line:90% the second time? 01:15:14.970 --> 01:15:15.490 align:middle line:90% Yeah. 01:15:15.490 --> 01:15:20.050 align:middle line:84% So I think there's some optimal stimulus parameters for this 01:15:20.050 --> 01:15:23.210 align:middle line:84% that are partially a function of how much context 01:15:23.210 --> 01:15:27.330 align:middle line:84% there is to be able to fill in the stuff in the noise, 01:15:27.330 --> 01:15:31.530 align:middle line:84% and some evidence that there's actually linguistic constraints 01:15:31.530 --> 01:15:34.170 align:middle line:90% on what you fill in. 01:15:34.170 --> 01:15:36.570 align:middle line:90% It's an interesting effect. 01:15:36.570 --> 01:15:38.170 align:middle line:90% Phonemic restoration. 01:15:38.170 --> 01:15:41.930 align:middle line:84% The last thing I'll leave you with 01:15:41.930 --> 01:15:45.590 align:middle line:84% is an example where this happens with sound textures. 01:15:45.590 --> 01:15:47.610 align:middle line:84% So textures are sounds that result 01:15:47.610 --> 01:15:49.790 align:middle line:84% from large numbers of similar acoustic events. 01:15:49.790 --> 01:15:57.470 align:middle line:84% So things like rain or swarms of insects, fire, wind, 01:15:57.470 --> 01:15:59.490 align:middle line:90% things like that. 01:15:59.490 --> 01:16:03.650 align:middle line:84% And there seems to be this pretty powerful inference 01:16:03.650 --> 01:16:06.110 align:middle line:90% of texture when it's masked. 01:16:06.110 --> 01:16:12.310 align:middle line:90% And so I'll play this example. 01:16:12.310 --> 01:16:14.510 align:middle line:84% So what you're going to hear is two seconds 01:16:14.510 --> 01:16:18.403 align:middle line:84% of texture followed by two seconds of masking noise. 01:16:18.403 --> 01:16:20.570 align:middle line:84% And I'm actually going to turn the volume down here. 01:16:20.570 --> 01:16:22.870 align:middle line:84% Just don't want anybody to lose their hearing. 01:16:22.870 --> 01:16:25.190 align:middle line:84% And you will probably have the sense-- 01:16:25.190 --> 01:16:27.065 align:middle line:84% and the texture is the sound of applause. 01:16:27.065 --> 01:16:29.690 align:middle line:84% You probably have the sense that it continues during the noise. 01:16:29.690 --> 01:16:30.357 align:middle line:90% [AUDIO PLAYBACK] 01:16:30.357 --> 01:16:31.606 align:middle line:90% [APPLAUSE] 01:16:31.606 --> 01:16:33.494 align:middle line:90% [STATIC] 01:16:33.494 --> 01:16:34.910 align:middle line:90% [APPLAUSE] 01:16:34.910 --> 01:16:35.810 align:middle line:90% [END PLAYBACK] 01:16:35.810 --> 01:16:36.790 align:middle line:84% JOSH MCDERMOTT: So it's kind of interesting 01:16:36.790 --> 01:16:38.665 align:middle line:84% because the noise is a full two seconds long, 01:16:38.665 --> 01:16:40.310 align:middle line:84% and you feel like you hear the thing 01:16:40.310 --> 01:16:42.143 align:middle line:84% throughout the whole thing, unless you think 01:16:42.143 --> 01:16:44.390 align:middle line:90% it's the power of suggestion. 01:16:44.390 --> 01:16:47.670 align:middle line:84% This is a version where there's a small gap 01:16:47.670 --> 01:16:50.252 align:middle line:84% between the end of the texture and the start of the noise, 01:16:50.252 --> 01:16:51.710 align:middle line:84% which makes it a lot less plausible 01:16:51.710 --> 01:16:55.083 align:middle line:84% that the texture continues, and that tends to kill the effect. 01:16:55.083 --> 01:16:55.750 align:middle line:90% [AUDIO PLAYBACK] 01:16:55.750 --> 01:16:57.610 align:middle line:90% [APPLAUSE] 01:16:57.610 --> 01:16:59.470 align:middle line:90% [STATIC] 01:16:59.470 --> 01:17:00.287 align:middle line:90% [APPLAUSE] 01:17:00.287 --> 01:17:00.870 align:middle line:90% [END PLAYBACK] 01:17:00.870 --> 01:17:04.070 align:middle line:84% JOSH MCDERMOTT: And so the big idea here 01:17:04.070 --> 01:17:07.290 align:middle line:84% is that these processes of filling in, we think 01:17:07.290 --> 01:17:10.650 align:middle line:84% are going on all the time, like behind the scenes without you 01:17:10.650 --> 01:17:11.230 align:middle line:90% knowing it. 01:17:11.230 --> 01:17:13.537 align:middle line:84% So you probably notice there's this kind 01:17:13.537 --> 01:17:15.370 align:middle line:84% of annoying humming sound that kind of comes 01:17:15.370 --> 01:17:17.330 align:middle line:90% from the cabinet in back of me. 01:17:17.330 --> 01:17:21.170 align:middle line:84% But that is continually masked intermittently by my voice. 01:17:21.170 --> 01:17:23.650 align:middle line:84% So when I talk, there will be periods of time 01:17:23.650 --> 01:17:26.330 align:middle line:84% where that noise is fully masked. 01:17:26.330 --> 01:17:28.830 align:middle line:84% Yet it probably sounds like the noise is totally continuous. 01:17:28.830 --> 01:17:31.090 align:middle line:84% And of course, physically it is, but the fact 01:17:31.090 --> 01:17:33.890 align:middle line:84% that it sounds continuous is because your brain is just kind 01:17:33.890 --> 01:17:36.570 align:middle line:90% of filling it in subjectively. 01:17:36.570 --> 01:17:39.930 align:middle line:84% So that you're kind of hearing your estimate of what's actually 01:17:39.930 --> 01:17:42.090 align:middle line:84% out there in the world, and that estimate 01:17:42.090 --> 01:17:45.250 align:middle line:84% is intended to compensate, in part, for the fact 01:17:45.250 --> 01:17:47.150 align:middle line:84% that things are masked all the time. 01:17:47.150 --> 01:17:50.810 align:middle line:84% Similarly, I can talk, and I can clap my hands like this. 01:17:50.810 --> 01:17:54.650 align:middle line:84% So my clapping is going to be masking my speech 01:17:54.650 --> 01:17:57.350 align:middle line:84% and it sounds just continuous to you. 01:17:57.350 --> 01:18:00.330 align:middle line:84% So this stuff goes on all the time, kind of behind the scenes. 01:18:00.330 --> 01:18:00.830 align:middle line:90% Yeah. 01:18:00.830 --> 01:18:03.730 align:middle line:90% 01:18:03.730 --> 01:18:06.510 align:middle line:84% AUDIENCE: Would this still work with like a texture that 01:18:06.510 --> 01:18:10.070 align:middle line:90% doesn't sound like the noise? 01:18:10.070 --> 01:18:13.750 align:middle line:84% If I really think about it, the applause 01:18:13.750 --> 01:18:17.250 align:middle line:84% does sound a bit like static noise. 01:18:17.250 --> 01:18:22.470 align:middle line:84% But if you replace the applause with popping bubbles, 01:18:22.470 --> 01:18:27.433 align:middle line:84% for instance, would this effect still hold? 01:18:27.433 --> 01:18:29.350 align:middle line:84% JOSH MCDERMOTT: Yeah, so there's a whole paper 01:18:29.350 --> 01:18:33.450 align:middle line:84% that explores the dependence on stuff like that. 01:18:33.450 --> 01:18:36.350 align:middle line:84% And in general, the effect is strongest 01:18:36.350 --> 01:18:41.110 align:middle line:84% when the sounds are stationary, which is to say kind of stable 01:18:41.110 --> 01:18:42.190 align:middle line:90% over time. 01:18:42.190 --> 01:18:44.950 align:middle line:84% So something that has these intermittent popping bubbles 01:18:44.950 --> 01:18:48.550 align:middle line:84% gets a lot harder to hear in this way. 01:18:48.550 --> 01:18:52.263 align:middle line:84% And in particular-- so if you do this exact same-- set 01:18:52.263 --> 01:18:53.930 align:middle line:84% up this exact same thing, but with, say, 01:18:53.930 --> 01:18:56.510 align:middle line:84% speech or music instead of the texture, 01:18:56.510 --> 01:18:58.670 align:middle line:84% you don't hear the speech or music kind 01:18:58.670 --> 01:19:00.090 align:middle line:90% of continuing over two seconds. 01:19:00.090 --> 01:19:05.630 align:middle line:84% So if you shorten the noise to, say, like 200 milliseconds, 01:19:05.630 --> 01:19:08.390 align:middle line:84% you'll get the continuity effect with speech, just like we heard. 01:19:08.390 --> 01:19:10.370 align:middle line:84% You get the continuity effect with music. 01:19:10.370 --> 01:19:12.090 align:middle line:84% But it doesn't last for that long. 01:19:12.090 --> 01:19:15.090 align:middle line:84% So the situations in which you subjectively 01:19:15.090 --> 01:19:16.850 align:middle line:84% hear this thing lasting for two seconds 01:19:16.850 --> 01:19:20.290 align:middle line:84% are those where the sound is stationary. 01:19:20.290 --> 01:19:22.250 align:middle line:84% And one reason for that is potentially 01:19:22.250 --> 01:19:24.990 align:middle line:84% because those are sounds that are, A, very predictable, 01:19:24.990 --> 01:19:28.570 align:middle line:84% but, B, also very likely to continue on very long 01:19:28.570 --> 01:19:29.910 align:middle line:90% timescales in the world. 01:19:29.910 --> 01:19:31.930 align:middle line:84% So textures are typically generated 01:19:31.930 --> 01:19:34.770 align:middle line:90% by lots of similar events. 01:19:34.770 --> 01:19:37.093 align:middle line:84% Probably makes it unlikely that they stop abruptly. 01:19:37.093 --> 01:19:39.010 align:middle line:84% And so you probably have a pretty strong prior 01:19:39.010 --> 01:19:40.970 align:middle line:84% that they just go on for long periods of time 01:19:40.970 --> 01:19:44.082 align:middle line:84% would be my guess, whereas for things like speech or music, 01:19:44.082 --> 01:19:45.790 align:middle line:84% you're not going to have that same prior. 01:19:45.790 --> 01:19:49.650 align:middle line:90% 01:19:49.650 --> 01:19:51.760 align:middle line:90% That's all for today. 01:19:51.760 --> 01:20:02.000 align:middle line:90%