1 00:00:01,640 --> 00:00:04,040 The following content is provided under a Creative 2 00:00:04,040 --> 00:00:05,580 Commons license. 3 00:00:05,580 --> 00:00:07,880 Your support will help MIT OpenCourseWare 4 00:00:07,880 --> 00:00:12,270 continue to offer high-quality educational resources for free. 5 00:00:12,270 --> 00:00:14,870 To make a donation or view additional materials 6 00:00:14,870 --> 00:00:18,830 from hundreds of MIT courses, visit MIT OpenCourseWare 7 00:00:18,830 --> 00:00:21,400 at ocw.mit.edu. 8 00:00:21,400 --> 00:00:23,150 NANCY KANWISHER: I'll just be brief today, 9 00:00:23,150 --> 00:00:28,070 but you can check out some of my stuff at the website up there. 10 00:00:28,070 --> 00:00:30,260 If you're confused by my appearance, 11 00:00:30,260 --> 00:00:33,710 if you've met me before, yes, I used to look like that. 12 00:00:33,710 --> 00:00:37,360 But at a deeper level, really, I look like this. 13 00:00:37,360 --> 00:00:41,682 This is me, and you look like that, too, inside. 14 00:00:41,682 --> 00:00:43,390 And these are parts of-- is that showing? 15 00:00:43,390 --> 00:00:44,187 Yeah. 16 00:00:44,187 --> 00:00:45,770 These are parts of my brain that we've 17 00:00:45,770 --> 00:00:48,890 mapped with functional MRI that were either discovered 18 00:00:48,890 --> 00:00:50,805 in my lab, or that my colleagues discovered, 19 00:00:50,805 --> 00:00:54,480 and that we then ran-- they're kinds of scans in my lab. 20 00:00:54,480 --> 00:00:58,160 These are all regions that do very specific things and that 21 00:00:58,160 --> 00:01:04,040 to me, are a big part of the story of how we are so smart. 22 00:01:04,040 --> 00:01:06,860 So my interests, at a very general level, 23 00:01:06,860 --> 00:01:11,510 are to answer things like what is the architecture 24 00:01:11,510 --> 00:01:12,320 of the human mind? 25 00:01:12,320 --> 00:01:14,162 What are its fundamental components? 26 00:01:14,162 --> 00:01:15,620 And there are lots and lots of ways 27 00:01:15,620 --> 00:01:17,600 to find those fundamental components. 28 00:01:17,600 --> 00:01:20,040 Functional MRI, which is how we made this picture, 29 00:01:20,040 --> 00:01:21,960 is just one of a huge number. 30 00:01:21,960 --> 00:01:24,950 I loved Patrick's comment that you should find questions, not 31 00:01:24,950 --> 00:01:25,605 hammers. 32 00:01:25,605 --> 00:01:28,680 I kind of like my hammer, I have to confess. 33 00:01:28,680 --> 00:01:31,070 But questions are more important. 34 00:01:31,070 --> 00:01:33,980 And there are lots of ways to approach 35 00:01:33,980 --> 00:01:39,740 this question of the basic architecture of the human mind. 36 00:01:39,740 --> 00:01:41,990 I also want to know how this structure, 37 00:01:41,990 --> 00:01:44,000 which is present in every normal person-- 38 00:01:44,000 --> 00:01:45,590 I could pop any of you in the scanner 39 00:01:45,590 --> 00:01:47,548 and make a picture like this of your brain, OK, 40 00:01:47,548 --> 00:01:52,500 it would take a little while, but wouldn't take that long. 41 00:01:52,500 --> 00:01:57,110 How does that structure arise over development? 42 00:01:57,110 --> 00:01:59,660 How does your genetic code and your experience 43 00:01:59,660 --> 00:02:01,730 work together to wire that up when 44 00:02:01,730 --> 00:02:04,460 you're an infant and a child? 45 00:02:04,460 --> 00:02:07,178 How did it evolve over human evolution? 46 00:02:10,310 --> 00:02:13,040 This is sort of what's sometimes called a mesoscale, this really 47 00:02:13,040 --> 00:02:16,310 macroscopic picture of the major components of the human mind 48 00:02:16,310 --> 00:02:17,720 and brain. 49 00:02:17,720 --> 00:02:20,720 But of course, we also want to know how each of those bits 50 00:02:20,720 --> 00:02:21,355 work. 51 00:02:21,355 --> 00:02:22,730 What are the representations that 52 00:02:22,730 --> 00:02:24,230 live in each of those regions? 53 00:02:24,230 --> 00:02:25,340 And how are they computed? 54 00:02:25,340 --> 00:02:27,230 And what are the neural circuits that 55 00:02:27,230 --> 00:02:30,470 implement those computations? 56 00:02:30,470 --> 00:02:33,530 And of course, cognition doesn't happen in just one 57 00:02:33,530 --> 00:02:35,960 little machine in there. 58 00:02:35,960 --> 00:02:39,040 It's a product of all of these bits working together. 59 00:02:39,040 --> 00:02:41,150 We want to understand how all of that works, too, 60 00:02:41,150 --> 00:02:45,230 and how all of that goes together to make us so smart. 61 00:02:45,230 --> 00:02:48,510 And that's related to a question that I'm deeply interested in, 62 00:02:48,510 --> 00:02:52,490 which is what is so special about this machine that looks 63 00:02:52,490 --> 00:02:54,350 a lot like a rodent brain? 64 00:02:54,350 --> 00:02:57,450 And it's smaller than a whale brain or a Neanderthal brain, 65 00:02:57,450 --> 00:02:59,930 so it's not just that we have more of it. 66 00:02:59,930 --> 00:03:04,190 What is so special about this thing that has put all of us 67 00:03:04,190 --> 00:03:06,320 here, interacting with each other 68 00:03:06,320 --> 00:03:07,760 and studying this thing, something 69 00:03:07,760 --> 00:03:11,450 that no other brain is doing, no other species brain? 70 00:03:11,450 --> 00:03:12,830 So are there special bits? 71 00:03:12,830 --> 00:03:14,602 Do those bits work differently? 72 00:03:14,602 --> 00:03:16,060 Are there special kinds of neurons? 73 00:03:16,060 --> 00:03:18,530 I don't think so, some people do. 74 00:03:18,530 --> 00:03:22,400 What is it about this that has brought us all right here? 75 00:03:22,400 --> 00:03:25,460 OK, so that, at a top level, are some of the questions 76 00:03:25,460 --> 00:03:27,167 I would most like to answer. 77 00:03:27,167 --> 00:03:29,000 Not that I know how to approach any of them, 78 00:03:29,000 --> 00:03:31,430 but I think it's important to keep an eye on those goals, 79 00:03:31,430 --> 00:03:34,910 even when you don't quite see how you're going to get there. 80 00:03:34,910 --> 00:03:40,700 My particular focus in the CBMM Project 81 00:03:40,700 --> 00:03:42,530 is to look at social intelligence, which 82 00:03:42,530 --> 00:03:45,210 is one piece of that puzzle. 83 00:03:45,210 --> 00:03:47,460 And so, why social intelligence? 84 00:03:47,460 --> 00:03:50,300 Well, just briefly, I think social cognition 85 00:03:50,300 --> 00:03:53,030 is in many ways the crux of human intelligence. 86 00:03:53,030 --> 00:03:56,000 OK, and it's a crux in a whole bunch of different senses. 87 00:03:56,000 --> 00:03:58,620 One is it's just the source of how we're so smart. 88 00:03:58,620 --> 00:04:00,620 Like, if you think about all the stuff you know, 89 00:04:00,620 --> 00:04:02,374 OK, do a quick mental inventory. 90 00:04:02,374 --> 00:04:03,790 OK, what's all the stuff you know? 91 00:04:03,790 --> 00:04:05,062 Like, make a little taxonomy. 92 00:04:05,062 --> 00:04:06,770 There's this kind of stuff, it's all lots 93 00:04:06,770 --> 00:04:08,700 of different kinds of stuff you know. 94 00:04:08,700 --> 00:04:10,630 OK, now how much of that stuff that you 95 00:04:10,630 --> 00:04:13,280 know would you know if you had never 96 00:04:13,280 --> 00:04:15,746 interacted with another person? 97 00:04:15,746 --> 00:04:17,480 A lot of it, you wouldn't know, right? 98 00:04:17,480 --> 00:04:20,149 So a lot of the stuff we know and a lot of the ways 99 00:04:20,149 --> 00:04:21,890 that we're smart are things that we 100 00:04:21,890 --> 00:04:23,850 get from interacting with other people. 101 00:04:23,850 --> 00:04:25,820 That's social cognition. 102 00:04:25,820 --> 00:04:26,930 OK. 103 00:04:26,930 --> 00:04:29,270 Another sense in which social cognition 104 00:04:29,270 --> 00:04:30,980 is the crux of human intelligence 105 00:04:30,980 --> 00:04:34,730 is many people think that the primary driver of the evolution 106 00:04:34,730 --> 00:04:36,920 of the human brain has been the requirement 107 00:04:36,920 --> 00:04:38,910 to interact with other people who are, 108 00:04:38,910 --> 00:04:41,425 after all, very complex entities, 109 00:04:41,425 --> 00:04:43,550 and to be able to understand how to work with them, 110 00:04:43,550 --> 00:04:45,216 and what they're doing, and what they'll 111 00:04:45,216 --> 00:04:48,320 do next is very cognitively demanding. 112 00:04:48,320 --> 00:04:52,820 And so that may be one of the major forces that has driven 113 00:04:52,820 --> 00:04:55,130 the evolution of our brain. 114 00:04:55,130 --> 00:04:56,930 Another sense in which social intelligence 115 00:04:56,930 --> 00:04:59,240 is the crux of human intelligence 116 00:04:59,240 --> 00:05:03,160 is that it's just plain a large percent of human cognition. 117 00:05:03,160 --> 00:05:09,310 OK, so we do versions of social cognition much of every day. 118 00:05:09,310 --> 00:05:12,160 Right now, I'm having these thoughts in my head. 119 00:05:12,160 --> 00:05:14,346 God knows what that looks like neurally. 120 00:05:14,346 --> 00:05:16,720 I'm translating that into some noises that are coming out 121 00:05:16,720 --> 00:05:18,984 of my mouth, you're hearing those noises, 122 00:05:18,984 --> 00:05:21,400 and you're getting-- let's hope-- kind of similar thoughts 123 00:05:21,400 --> 00:05:22,840 in your head. 124 00:05:22,840 --> 00:05:24,580 That is a miracle. 125 00:05:24,580 --> 00:05:28,990 Nobody has the foggiest idea how that works at a neural level. 126 00:05:28,990 --> 00:05:32,754 Nobody can even make up a sketch of a hypothesis 127 00:05:32,754 --> 00:05:34,420 of a bunch of neural circuits that might 128 00:05:34,420 --> 00:05:35,800 be able to make that happen. 129 00:05:35,800 --> 00:05:37,420 Right? 130 00:05:37,420 --> 00:05:42,340 That's a fascinating puzzle, and it's also 131 00:05:42,340 --> 00:05:44,500 of the essence in human intelligence. 132 00:05:44,500 --> 00:05:47,560 And we do it all the time, not just speaking per se, 133 00:05:47,560 --> 00:05:50,050 but all the other ways that we share information 134 00:05:50,050 --> 00:05:51,470 with each other. 135 00:05:51,470 --> 00:05:54,070 So one, social cognition is just what 136 00:05:54,070 --> 00:05:56,560 we do all day long every day. 137 00:05:56,560 --> 00:06:00,380 It's also a big part of the surface area of the cortex. 138 00:06:00,380 --> 00:06:04,480 So this cartoon here shows-- with some major poetic 139 00:06:04,480 --> 00:06:05,494 license-- 140 00:06:05,494 --> 00:06:07,660 brain regions that are involved in different aspects 141 00:06:07,660 --> 00:06:08,650 of social cognition. 142 00:06:08,650 --> 00:06:12,820 And it's just a big part of the cortical area as well. 143 00:06:12,820 --> 00:06:14,560 OK. 144 00:06:14,560 --> 00:06:16,930 Another sense in which social cognition 145 00:06:16,930 --> 00:06:18,790 is of the essence in human intelligence 146 00:06:18,790 --> 00:06:22,060 is that many of the greatest things that humanity 147 00:06:22,060 --> 00:06:25,960 has accomplished are products of people working together. 148 00:06:25,960 --> 00:06:27,610 So all of that is the big picture 149 00:06:27,610 --> 00:06:31,480 on why social cognition is cool, and important, and fundamental. 150 00:06:31,480 --> 00:06:34,840 The part of it that we're focusing on in our thrust 151 00:06:34,840 --> 00:06:37,180 within this NSF grant is something 152 00:06:37,180 --> 00:06:39,160 I call social perception. 153 00:06:39,160 --> 00:06:41,950 OK, so by social perception, I mean 154 00:06:41,950 --> 00:06:45,490 this spectacularly impressive human ability 155 00:06:45,490 --> 00:06:48,340 to extract rich, multidimensional 156 00:06:48,340 --> 00:06:53,060 social information from a brief glimpse of a scene. 157 00:06:53,060 --> 00:06:54,800 From a brief glimpse at a person, 158 00:06:54,800 --> 00:06:58,820 you can tell not just who that person is, you can 159 00:06:58,820 --> 00:07:02,250 tell what they're trying to do. 160 00:07:02,250 --> 00:07:04,700 You can tell how they feel. 161 00:07:04,700 --> 00:07:08,180 You can tell what they're paying attention to. 162 00:07:08,180 --> 00:07:12,090 You can tell what they know and who they like. 163 00:07:12,090 --> 00:07:13,250 OK? 164 00:07:13,250 --> 00:07:14,720 And that's just the beginning. 165 00:07:14,720 --> 00:07:15,219 OK? 166 00:07:15,219 --> 00:07:18,020 So the work in our thrust tries to approach 167 00:07:18,020 --> 00:07:20,210 all of these different kinds of questions 168 00:07:20,210 --> 00:07:24,800 that we are calling as part of our PR of this NSF grant. 169 00:07:24,800 --> 00:07:26,960 It's kind of an organizing principle. 170 00:07:26,960 --> 00:07:30,620 The Turing questions, these demanding, difficult 171 00:07:30,620 --> 00:07:33,860 computational problems of social perception. 172 00:07:33,860 --> 00:07:34,790 Who is that person? 173 00:07:34,790 --> 00:07:36,206 What are they paying attention to? 174 00:07:36,206 --> 00:07:37,320 What are they feeling? 175 00:07:37,320 --> 00:07:38,172 What are they like? 176 00:07:38,172 --> 00:07:39,630 Are they interacting with somebody? 177 00:07:39,630 --> 00:07:41,255 What is the nature of that interaction? 178 00:07:41,255 --> 00:07:42,190 And so on. 179 00:07:42,190 --> 00:07:43,340 OK? 180 00:07:43,340 --> 00:07:47,570 So the general plan of action in how 181 00:07:47,570 --> 00:07:50,120 to approach this in our thrust is 182 00:07:50,120 --> 00:07:54,140 first to study these abilities in the computational system 183 00:07:54,140 --> 00:07:56,420 that's best at them, namely this one-- 184 00:07:56,420 --> 00:07:58,880 and those out there, yours, too-- 185 00:07:58,880 --> 00:08:01,310 the human brain. 186 00:08:01,310 --> 00:08:05,210 And so the roadmap here is to first do psychophysics, 187 00:08:05,210 --> 00:08:07,970 characterize simple behavioral measurements-- 188 00:08:07,970 --> 00:08:10,300 what can people do, what can't they do-- 189 00:08:10,300 --> 00:08:16,010 from simple stimuli, and quantify that in detail. 190 00:08:16,010 --> 00:08:17,390 Ask, how good are we at it? 191 00:08:17,390 --> 00:08:19,970 Maybe some of these things that we think we can do, like 192 00:08:19,970 --> 00:08:22,280 size up somebody's personality in three seconds 193 00:08:22,280 --> 00:08:23,450 when we first meet them-- 194 00:08:23,450 --> 00:08:25,565 feels like you can do that, or at least 195 00:08:25,565 --> 00:08:27,884 you get a read on them-- 196 00:08:27,884 --> 00:08:29,300 I mean, is that based on anything? 197 00:08:29,300 --> 00:08:30,450 Is that just garbage? 198 00:08:30,450 --> 00:08:31,810 Right? 199 00:08:31,810 --> 00:08:36,770 Are we actually tapping into real information there? 200 00:08:36,770 --> 00:08:39,679 What cues are we using when we make those high level 201 00:08:39,679 --> 00:08:41,900 social inferences? 202 00:08:41,900 --> 00:08:43,669 What is the input that we get, that we 203 00:08:43,669 --> 00:08:47,720 use as a basis for analyzing this particular percept 204 00:08:47,720 --> 00:08:49,880 or throughout life that we've used 205 00:08:49,880 --> 00:08:53,145 to train up our brains to be able to do this? 206 00:08:53,145 --> 00:08:55,520 So the second approach is once we have some kind of sense 207 00:08:55,520 --> 00:08:58,190 of what are those abilities-- that's sometimes called Marr 208 00:08:58,190 --> 00:09:02,650 theory level, characterizing what can we do, right-- 209 00:09:02,650 --> 00:09:05,749 is we can then try to computationally model this. 210 00:09:05,749 --> 00:09:07,790 And so there's lots of different ways to do this, 211 00:09:07,790 --> 00:09:09,956 and many of the other thrusts that you'll hear about 212 00:09:09,956 --> 00:09:12,580 are really tackling that problem. 213 00:09:12,580 --> 00:09:14,360 Another thing we can do is, of course, 214 00:09:14,360 --> 00:09:17,256 characterize the brain basis of these abilities, 215 00:09:17,256 --> 00:09:19,130 and we can do that with all kinds of methods. 216 00:09:19,130 --> 00:09:22,730 We're using, in our thrust, functional MRI, 217 00:09:22,730 --> 00:09:26,930 intracranial recordings, something called NIRS. 218 00:09:26,930 --> 00:09:31,070 This is the ability to make measurements of blood flow 219 00:09:31,070 --> 00:09:34,760 changes in very young infants. 220 00:09:34,760 --> 00:09:36,950 And so we can characterize these brain systems 221 00:09:36,950 --> 00:09:38,300 in adults and infants. 222 00:09:38,300 --> 00:09:41,210 And that gives you a leg up in understanding 223 00:09:41,210 --> 00:09:42,890 these other broader questions about how 224 00:09:42,890 --> 00:09:46,280 the whole system works in a number of different ways. 225 00:09:46,280 --> 00:09:48,650 Just seeing how the brain carves up 226 00:09:48,650 --> 00:09:51,380 the problem of social perception into pieces 227 00:09:51,380 --> 00:09:53,360 already gives you some clues about the kinds 228 00:09:53,360 --> 00:09:56,900 of computations that may go on in each of those pieces. 229 00:09:56,900 --> 00:09:57,960 OK? 230 00:09:57,960 --> 00:09:58,640 OK. 231 00:09:58,640 --> 00:10:00,654 So that's the overview. 232 00:10:00,654 --> 00:10:02,320 There's many, many ways you can do this, 233 00:10:02,320 --> 00:10:04,050 and of course, people all over the place are doing this. 234 00:10:04,050 --> 00:10:06,260 There's nothing all that unique about it. 235 00:10:06,260 --> 00:10:09,230 This is just our framework here. 236 00:10:09,230 --> 00:10:12,050 Some of the specific projects that are going on 237 00:10:12,050 --> 00:10:15,410 include some work on face recognition, which of course, 238 00:10:15,410 --> 00:10:17,540 a really classic question that many people 239 00:10:17,540 --> 00:10:18,950 have been approaching. 240 00:10:18,950 --> 00:10:23,270 My post-doc, Matt Peterson, here has done some very lovely work 241 00:10:23,270 --> 00:10:26,990 where he's shown that, actually, where you look on a face 242 00:10:26,990 --> 00:10:28,220 is very systematic. 243 00:10:28,220 --> 00:10:30,110 You don't just look anywhere, right? 244 00:10:30,110 --> 00:10:32,120 When you first make us saccade into a face, 245 00:10:32,120 --> 00:10:34,730 somebody appears in your visual periphery, right, of course, 246 00:10:34,730 --> 00:10:38,750 all the high-resolution visual abilities are all right near 247 00:10:38,750 --> 00:10:40,490 the center of gaze around the fovea, 248 00:10:40,490 --> 00:10:43,640 where you have a high density of photo receptors and a shitload 249 00:10:43,640 --> 00:10:45,800 of cortex-- to be technical about it-- 250 00:10:45,800 --> 00:10:47,970 devoted to allocating center of gaze. 251 00:10:47,970 --> 00:10:52,310 Right back here, in primary visual cortex and with 252 00:10:52,310 --> 00:10:54,170 the first few retinotopic regions, 253 00:10:54,170 --> 00:10:57,890 you have 20 square centimeters-- that's like that-- 254 00:10:57,890 --> 00:11:01,200 of cortex allocated to just the central two degrees of vision. 255 00:11:01,200 --> 00:11:01,700 Right? 256 00:11:01,700 --> 00:11:04,550 So you have a lot of computational machinery doing 257 00:11:04,550 --> 00:11:06,140 just that bit right there. 258 00:11:06,140 --> 00:11:08,120 Well, when a face appears in your periphery, 259 00:11:08,120 --> 00:11:10,190 you move that bit of your cortex, boom, 260 00:11:10,190 --> 00:11:11,207 right on top of it. 261 00:11:11,207 --> 00:11:13,040 So you have all that computational machinery 262 00:11:13,040 --> 00:11:15,710 to dig in on the face, right? 263 00:11:15,710 --> 00:11:20,120 OK, so what Matt has shown is that the particular way 264 00:11:20,120 --> 00:11:23,390 that you allocate that computational machinery, namely 265 00:11:23,390 --> 00:11:25,940 by making an eye movement to put that stimulus 266 00:11:25,940 --> 00:11:30,060 right on your fovea, people do that slightly differently. 267 00:11:30,060 --> 00:11:32,720 Some people fixate on a face up here, 268 00:11:32,720 --> 00:11:35,360 some people fixate on a face down there, 269 00:11:35,360 --> 00:11:37,621 and most people fixate someplace in the middle. 270 00:11:37,621 --> 00:11:38,120 OK? 271 00:11:38,120 --> 00:11:39,560 Well, so why is it interesting? 272 00:11:39,560 --> 00:11:40,970 Here's why it's interesting. 273 00:11:40,970 --> 00:11:42,830 People do that in very systematic ways. 274 00:11:42,830 --> 00:11:44,780 And if you look up here, you pretty much 275 00:11:44,780 --> 00:11:45,817 always look up there. 276 00:11:45,817 --> 00:11:47,900 And if you look down there, you pretty much always 277 00:11:47,900 --> 00:11:49,310 look down there. 278 00:11:49,310 --> 00:11:51,650 And this has computational consequences. 279 00:11:51,650 --> 00:11:53,330 If we brought you guys into the lab 280 00:11:53,330 --> 00:11:55,664 and ran you on an eye tracker for 15 minutes, 281 00:11:55,664 --> 00:11:57,330 we'd find out which of you look up there 282 00:11:57,330 --> 00:11:58,705 and which of you look down there. 283 00:11:58,705 --> 00:12:01,490 And if we took those of you who look up here, 284 00:12:01,490 --> 00:12:04,250 and we presented a face by flashing it briefly 285 00:12:04,250 --> 00:12:06,410 while you're fixating so that the face landed 286 00:12:06,410 --> 00:12:08,510 in your not-preferred looking position, 287 00:12:08,510 --> 00:12:10,520 your accuracy at recognizing that face 288 00:12:10,520 --> 00:12:12,799 would be much lower, and vice versa. 289 00:12:12,799 --> 00:12:14,840 If you're one of the people who looks down there, 290 00:12:14,840 --> 00:12:16,700 and we flash up a face so that it 291 00:12:16,700 --> 00:12:18,200 lands right there on your retina, 292 00:12:18,200 --> 00:12:20,450 you're much worse at recognizing it. 293 00:12:20,450 --> 00:12:23,079 And what that means is that this fundamental problem 294 00:12:23,079 --> 00:12:24,620 that you'll hear about in the course, 295 00:12:24,620 --> 00:12:26,360 that Tommy has worked at in many people, 296 00:12:26,360 --> 00:12:29,900 it's one of the central problems in vision research of how 297 00:12:29,900 --> 00:12:32,900 we deal with the many different ways an object-- 298 00:12:32,900 --> 00:12:35,330 the many different kinds of images an object 299 00:12:35,330 --> 00:12:38,630 can make on our retina by where it 300 00:12:38,630 --> 00:12:40,100 lands on the retina, how close it 301 00:12:40,100 --> 00:12:44,180 is to you, the orientation, the lighting, all these things that 302 00:12:44,180 --> 00:12:47,090 create this central problem in vision 303 00:12:47,090 --> 00:12:50,690 of the variable ways an object can look. 304 00:12:50,690 --> 00:12:53,510 A big part of how we solve that for face recognition is we 305 00:12:53,510 --> 00:12:55,670 just move our eyes to the same place. 306 00:12:55,670 --> 00:12:58,705 Position and variance problem solved, mostly. 307 00:12:58,705 --> 00:13:00,300 OK, it's kind of a low-tech solution. 308 00:13:00,300 --> 00:13:02,120 It's a good one. 309 00:13:02,120 --> 00:13:04,880 OK, anyway, so Matt has been working on that for a while, 310 00:13:04,880 --> 00:13:06,830 and so now, most of that is lab studies. 311 00:13:06,830 --> 00:13:11,030 Now what he's done is he's using mobile eye trackers, which 312 00:13:11,030 --> 00:13:14,224 look like this, and a GoPro attached to his head, 313 00:13:14,224 --> 00:13:16,640 because the mobile eye trackers don't have very good image 314 00:13:16,640 --> 00:13:17,782 resolution. 315 00:13:17,782 --> 00:13:19,740 And so he's sending people around in the world, 316 00:13:19,740 --> 00:13:21,800 and he's finding that, first of all, yes, in fact, 317 00:13:21,800 --> 00:13:23,508 when you're walking around in the world-- 318 00:13:23,508 --> 00:13:26,125 not just when you're on a bike bar in a lab, you know, 319 00:13:26,125 --> 00:13:28,400 with a tracker and a screen-- 320 00:13:28,400 --> 00:13:31,010 the people who look up here also look up there in the world, 321 00:13:31,010 --> 00:13:32,242 right? 322 00:13:32,242 --> 00:13:33,950 So that's just a reality check that shows 323 00:13:33,950 --> 00:13:35,390 that our technology is working. 324 00:13:35,390 --> 00:13:38,960 And now Matt is using this to ask all kinds of questions. 325 00:13:38,960 --> 00:13:41,387 For example, social interactions, 326 00:13:41,387 --> 00:13:43,220 where do people look in social interactions? 327 00:13:43,220 --> 00:13:45,770 Can you tell stuff about what they 328 00:13:45,770 --> 00:13:48,200 think about each other based on where they look on faces, 329 00:13:48,200 --> 00:13:49,370 right? 330 00:13:49,370 --> 00:13:50,300 We want to run-- 331 00:13:50,300 --> 00:13:51,110 this is fruity. 332 00:13:51,110 --> 00:13:52,651 We haven't set it up yet, but we want 333 00:13:52,651 --> 00:13:55,730 to run speed dating experiments in the lab 334 00:13:55,730 --> 00:13:57,290 with people wearing eye trackers. 335 00:13:57,290 --> 00:13:59,437 I bet in the first few fixation positions, 336 00:13:59,437 --> 00:14:01,520 you can tell who's going to want to recontact who. 337 00:14:01,520 --> 00:14:02,061 I don't know. 338 00:14:02,061 --> 00:14:03,230 We haven't done that yet. 339 00:14:03,230 --> 00:14:07,550 OK, that's a little trashy, but it's kind of interesting. 340 00:14:07,550 --> 00:14:10,730 Some interesting scientific questions are a little bit 341 00:14:10,730 --> 00:14:11,660 trashy, you know. 342 00:14:11,660 --> 00:14:14,150 Some trashy questions are not scientifically interesting. 343 00:14:14,150 --> 00:14:16,870 I think that's one of those rare that's actually both. 344 00:14:16,870 --> 00:14:18,050 Anyway. 345 00:14:18,050 --> 00:14:21,127 We also want to characterize-- a whole other part of this 346 00:14:21,127 --> 00:14:23,210 is this question that people have been considering 347 00:14:23,210 --> 00:14:26,580 for a few decades now of natural image statistics, right? 348 00:14:26,580 --> 00:14:29,420 So people have done all this stuff, collecting images, 349 00:14:29,420 --> 00:14:31,400 and at first, they did it really low-tech, 350 00:14:31,400 --> 00:14:32,535 and then the web appeared. 351 00:14:32,535 --> 00:14:34,910 And it's like, oh, now there's a lot of images out there, 352 00:14:34,910 --> 00:14:36,574 and we can just collect them easily. 353 00:14:36,574 --> 00:14:37,740 And let's characterize them. 354 00:14:37,740 --> 00:14:39,390 What are natural images like? 355 00:14:39,390 --> 00:14:41,240 So it's a whole set of math where 356 00:14:41,240 --> 00:14:43,100 people have looked at those natural images, 357 00:14:43,100 --> 00:14:44,475 and characterized them, and tried 358 00:14:44,475 --> 00:14:49,430 to ask how the statistical properties of natural images 359 00:14:49,430 --> 00:14:51,340 have-- 360 00:14:51,340 --> 00:14:55,160 how we have adjusted our visual systems to deal with the images 361 00:14:55,160 --> 00:14:56,600 that we confront. 362 00:14:56,600 --> 00:14:59,400 And that's a cool and important area of research. 363 00:14:59,400 --> 00:15:02,730 But in all of that work, nobody's 364 00:15:02,730 --> 00:15:05,190 actually used real natural images, right? 365 00:15:05,190 --> 00:15:07,800 The images on the web, somebody stuck a camera 366 00:15:07,800 --> 00:15:11,300 and put it there, and then they threw away 367 00:15:11,300 --> 00:15:12,966 most of the pictures they took. 368 00:15:12,966 --> 00:15:14,340 The ones that land on the web are 369 00:15:14,340 --> 00:15:17,369 the ones that have good resolution, where 370 00:15:17,369 --> 00:15:19,410 people weren't moving in and out of frame, things 371 00:15:19,410 --> 00:15:20,240 weren't occluded. 372 00:15:20,240 --> 00:15:22,410 They're not at all like the actual images 373 00:15:22,410 --> 00:15:23,940 that land on your retina. 374 00:15:23,940 --> 00:15:25,620 So we're collecting the actual images 375 00:15:25,620 --> 00:15:26,817 that land on your retina. 376 00:15:26,817 --> 00:15:28,650 And we're doing it with mobile eye trackers, 377 00:15:28,650 --> 00:15:31,140 sending people around in the world using these nice GoPro 378 00:15:31,140 --> 00:15:33,480 systems to give us high resolution. 379 00:15:33,480 --> 00:15:36,780 And importantly, not only are we collecting real natural image 380 00:15:36,780 --> 00:15:39,900 statistics from these real natural images, 381 00:15:39,900 --> 00:15:42,810 we know, for each frame, where the person was looking. 382 00:15:42,810 --> 00:15:45,360 And that's important for the reason I mentioned a while ago, 383 00:15:45,360 --> 00:15:47,640 that most of your high-resolution information 384 00:15:47,640 --> 00:15:49,560 is right at the center of gaze. 385 00:15:49,560 --> 00:15:53,520 And the information out in the periphery is pretty lousy. 386 00:15:53,520 --> 00:15:56,190 OK, so that's one project that I described too long, 387 00:15:56,190 --> 00:16:00,030 so I'll whip through the others more briefly. 388 00:16:00,030 --> 00:16:03,870 We want to know how well people can read each other's direction 389 00:16:03,870 --> 00:16:05,140 of attention. 390 00:16:05,140 --> 00:16:09,300 OK, so when I'm lecturing now, if you guys get bored and look 391 00:16:09,300 --> 00:16:12,814 at the clock, I will see it, right? 392 00:16:12,814 --> 00:16:14,730 And that's just one of these things, you know? 393 00:16:14,730 --> 00:16:17,580 We're very attuned to where each other are looking, 394 00:16:17,580 --> 00:16:19,464 and that's very useful information. 395 00:16:19,464 --> 00:16:20,880 You meet somebody at a conference, 396 00:16:20,880 --> 00:16:23,130 and you see them make a saccade down to your name tag, 397 00:16:23,130 --> 00:16:26,190 and it's like, damn it, doesn't this person remember who I am? 398 00:16:26,190 --> 00:16:26,870 You know? 399 00:16:26,870 --> 00:16:29,330 I'm very aware of this because I'm mildly prosopagnosic. 400 00:16:29,330 --> 00:16:31,719 So if I've met you before, and I'm slow to register, 401 00:16:31,719 --> 00:16:32,760 don't take it personally. 402 00:16:32,760 --> 00:16:33,600 I'm just lousy. 403 00:16:33,600 --> 00:16:36,690 It takes me a long time to encode a face. 404 00:16:36,690 --> 00:16:40,330 Anyway, we're very attuned at where each other are looking. 405 00:16:40,330 --> 00:16:42,960 And so there's been a lot of work on how precisely 406 00:16:42,960 --> 00:16:45,780 we can tell whether somebody is looking right at you versus off 407 00:16:45,780 --> 00:16:46,800 to the side. 408 00:16:46,800 --> 00:16:47,580 Try this at lunch. 409 00:16:47,580 --> 00:16:50,010 When you're in the middle of a conversation with somebody, 410 00:16:50,010 --> 00:16:52,800 fixate on just the side of their face, not way off to the side, 411 00:16:52,800 --> 00:16:56,400 just like here, and just do that for a few seconds. 412 00:16:56,400 --> 00:16:59,160 It's deeply weird. 413 00:16:59,160 --> 00:17:01,587 The person you're talking to will detect it immediately, 414 00:17:01,587 --> 00:17:04,170 will feel uncomfortable, until they realize what you're doing, 415 00:17:04,170 --> 00:17:06,869 and then you guys will have a good laugh. 416 00:17:06,869 --> 00:17:10,740 And that will show you how exquisitely precise 417 00:17:10,740 --> 00:17:12,839 your ability to read another person's gaze is. 418 00:17:12,839 --> 00:17:15,000 It's really very precisely tuned. 419 00:17:15,000 --> 00:17:15,545 OK. 420 00:17:15,545 --> 00:17:16,920 So there's a lot of work on that, 421 00:17:16,920 --> 00:17:19,859 but there's less work on how well I 422 00:17:19,859 --> 00:17:22,817 can tell what exactly you're looking at if it's not me. 423 00:17:22,817 --> 00:17:24,900 That is, I can tell if you're looking at me or off 424 00:17:24,900 --> 00:17:27,450 to the side, or this side, or that side. 425 00:17:27,450 --> 00:17:29,130 But what we're looking at is how well 426 00:17:29,130 --> 00:17:32,280 can I tell what object you're looking at? 427 00:17:32,280 --> 00:17:35,400 And that's an important question because many people 428 00:17:35,400 --> 00:17:40,080 have pointed out that a central little microcosm, kind 429 00:17:40,080 --> 00:17:42,390 of a unit of social interaction, is something 430 00:17:42,390 --> 00:17:43,880 called joint attention. 431 00:17:43,880 --> 00:17:47,060 And joint attention is when you're looking at this thing, 432 00:17:47,060 --> 00:17:49,500 and I'm looking at it, and I know you're looking at it, 433 00:17:49,500 --> 00:17:51,420 and you know I'm looking at it. 434 00:17:51,420 --> 00:17:53,280 That's a cosmic little thing. 435 00:17:53,280 --> 00:17:55,290 Like, we can have this little moment, right? 436 00:17:55,290 --> 00:17:56,820 Joint attention, OK? 437 00:17:56,820 --> 00:17:58,410 And people have argued that that's 438 00:17:58,410 --> 00:18:01,920 of the essence in children learning language. 439 00:18:01,920 --> 00:18:05,460 It's of the essence in all kinds of social interactions. 440 00:18:05,460 --> 00:18:08,010 And by most accounts, no other species 441 00:18:08,010 --> 00:18:09,510 has it, not even chimps. 442 00:18:09,510 --> 00:18:10,010 OK? 443 00:18:10,010 --> 00:18:12,030 I mean, there's still some debate about this, 444 00:18:12,030 --> 00:18:14,030 and people niggle and stuff, but basically, they 445 00:18:14,030 --> 00:18:16,560 don't have it in anything like the way we have it. 446 00:18:16,560 --> 00:18:19,352 So we want to know, what is the acuity of joint attention? 447 00:18:19,352 --> 00:18:21,060 OK, so I was supposed to do that briefly. 448 00:18:21,060 --> 00:18:22,710 I can't seem to be brief. 449 00:18:22,710 --> 00:18:24,810 OK. 450 00:18:24,810 --> 00:18:27,240 So that's a whole project that's going on with Danny 451 00:18:27,240 --> 00:18:28,980 Harari and Tao Gao. 452 00:18:28,980 --> 00:18:30,480 We're also asking how well people 453 00:18:30,480 --> 00:18:34,080 can predict the target of another person's action, right? 454 00:18:34,080 --> 00:18:36,304 So if I go out to reach this, at one point-- well, 455 00:18:36,304 --> 00:18:37,720 there's only one thing there-- but 456 00:18:37,720 --> 00:18:39,136 if we had a whole array of things, 457 00:18:39,136 --> 00:18:41,070 at one point when I'm reaching for an object, 458 00:18:41,070 --> 00:18:43,170 can you extrapolate my trajectory, 459 00:18:43,170 --> 00:18:45,120 look at my eye gaze, and use all of those cues 460 00:18:45,120 --> 00:18:48,930 to figure out what is the goal of my action? 461 00:18:48,930 --> 00:18:52,822 Here's a cool way to look at how well people can 462 00:18:52,822 --> 00:18:54,030 predict each other's actions. 463 00:18:54,030 --> 00:18:56,520 This is work by Maryam Vaziri-Pashkam, shown here, 464 00:18:56,520 --> 00:19:00,450 who's a post-doc at Harvard working with Ken Nakayama, who 465 00:19:00,450 --> 00:19:03,580 will give a lecture later in the course. 466 00:19:03,580 --> 00:19:05,250 And what they're trying to do is get 467 00:19:05,250 --> 00:19:07,702 an online read of how well people can 468 00:19:07,702 --> 00:19:08,910 predict each other's actions. 469 00:19:08,910 --> 00:19:11,830 And so obviously, this happens in all kinds of situations, 470 00:19:11,830 --> 00:19:13,475 especially in sports, right? 471 00:19:13,475 --> 00:19:15,690 If you're playing basketball or ultimate frisbee, 472 00:19:15,690 --> 00:19:17,940 it's all about predicting who's going to go where when 473 00:19:17,940 --> 00:19:21,420 and trying to take that into account with your actions. 474 00:19:21,420 --> 00:19:23,130 So they've set this up in the lab. 475 00:19:23,130 --> 00:19:25,050 And they have a piece of glass here, 476 00:19:25,050 --> 00:19:29,100 and there's two Post-its on this piece of glass. 477 00:19:29,100 --> 00:19:32,700 And one person's task is to reach out and touch 478 00:19:32,700 --> 00:19:35,350 one of those targets quickly. 479 00:19:35,350 --> 00:19:38,310 And the other person who's the goalie 480 00:19:38,310 --> 00:19:40,110 watches them through the glass and tries 481 00:19:40,110 --> 00:19:42,060 to touch that target as soon as possible 482 00:19:42,060 --> 00:19:43,410 after the first one does. 483 00:19:43,410 --> 00:19:44,390 OK? 484 00:19:44,390 --> 00:19:46,810 And so it's just a basic little game. 485 00:19:46,810 --> 00:19:51,500 And so they have little sensors on each person's finger 486 00:19:51,500 --> 00:19:53,190 so they can track the exact trajectories 487 00:19:53,190 --> 00:19:54,529 and get reaction times. 488 00:19:54,529 --> 00:19:56,070 They're just behavioral measurements, 489 00:19:56,070 --> 00:19:57,450 but they're very cool. 490 00:19:57,450 --> 00:19:59,070 So what they find first of all is 491 00:19:59,070 --> 00:20:00,444 that the goalie, the person who's 492 00:20:00,444 --> 00:20:02,730 trying to reach to respond to the other person, 493 00:20:02,730 --> 00:20:05,280 can do that extremely fast, right? 494 00:20:05,280 --> 00:20:08,820 They launch their hand to the correct target 495 00:20:08,820 --> 00:20:11,931 within 150 milliseconds. 496 00:20:11,931 --> 00:20:14,430 Well, you should immediately realize that something's fishy. 497 00:20:14,430 --> 00:20:15,210 You can't do that. 498 00:20:15,210 --> 00:20:18,600 It takes about 100 milliseconds just to get to V1. 499 00:20:18,600 --> 00:20:21,300 It takes, I forget how long, but a few tens of milliseconds 500 00:20:21,300 --> 00:20:24,620 to send the signal out from your brain 501 00:20:24,620 --> 00:20:26,520 out your arm to initiate the movement. 502 00:20:26,520 --> 00:20:29,490 So how could you possibly do all of that in that time? 503 00:20:29,490 --> 00:20:30,550 Well, you can't. 504 00:20:30,550 --> 00:20:33,420 And what that means is that people are actually 505 00:20:33,420 --> 00:20:37,530 launching the hand action, the goalie's 506 00:20:37,530 --> 00:20:39,960 launching the action before the other person has actually 507 00:20:39,960 --> 00:20:41,126 started moving their finger. 508 00:20:41,126 --> 00:20:42,990 They've started processing it before. 509 00:20:42,990 --> 00:20:46,620 And the way they've done that is before this person starts, 510 00:20:46,620 --> 00:20:49,230 before their hand moves at all, they've 511 00:20:49,230 --> 00:20:52,110 subtly changed their body configuration in ways 512 00:20:52,110 --> 00:20:54,060 that the other person can read. 513 00:20:54,060 --> 00:20:55,350 OK? 514 00:20:55,350 --> 00:20:57,429 Now, on the one hand, OK, duh. 515 00:20:57,429 --> 00:20:58,470 You're playing this game. 516 00:20:58,470 --> 00:20:59,940 You learn to exploit cues. 517 00:20:59,940 --> 00:21:03,300 We're really great at figuring out cues quickly, and using 518 00:21:03,300 --> 00:21:05,580 them, and learning to use them. 519 00:21:05,580 --> 00:21:07,110 But here's the-- one second-- here's 520 00:21:07,110 --> 00:21:09,540 the cool thing about this task is 521 00:21:09,540 --> 00:21:12,870 that this immediate, ultrafast reaction time happens 522 00:21:12,870 --> 00:21:15,670 on the very first few trials. 523 00:21:15,670 --> 00:21:18,480 So the ability that this task is tapping into 524 00:21:18,480 --> 00:21:22,470 is not that the goalie can learn what cues are predictive given 525 00:21:22,470 --> 00:21:24,250 enough trials and feedback. 526 00:21:24,250 --> 00:21:26,160 No, they do it right off the bat. 527 00:21:26,160 --> 00:21:28,050 This task is tapping into an ability 528 00:21:28,050 --> 00:21:30,630 that we all have already, right now, 529 00:21:30,630 --> 00:21:35,016 to read each other's actions and predict each other's behavior. 530 00:21:35,016 --> 00:21:37,140 And so people with no instruction and no experience 531 00:21:37,140 --> 00:21:40,080 whatsoever in this novel task know 532 00:21:40,080 --> 00:21:43,410 that this subtle little cue of the way the body is moving 533 00:21:43,410 --> 00:21:46,740 a little bit before the person's finger even starts to move, 534 00:21:46,740 --> 00:21:48,900 they can tell what it's predictive of. 535 00:21:48,900 --> 00:21:54,450 So that's just another way to characterize people's abilities 536 00:21:54,450 --> 00:21:56,280 in social perceptions, so one of some 537 00:21:56,280 --> 00:21:59,040 of the many different things that we just see really 538 00:21:59,040 --> 00:22:01,440 well in other people's actions. 539 00:22:01,440 --> 00:22:05,510 OK, that's what I just said, all right. 540 00:22:05,510 --> 00:22:06,120 All right. 541 00:22:06,120 --> 00:22:07,536 I'm going to skip over some stuff. 542 00:22:07,536 --> 00:22:09,900 We're looking at perception of emotional expressions. 543 00:22:09,900 --> 00:22:12,060 Almost the entire literature is based 544 00:22:12,060 --> 00:22:16,080 on staged emotional expressions on faces, 545 00:22:16,080 --> 00:22:18,360 huge literature with neuroimaging and behavior, 546 00:22:18,360 --> 00:22:20,940 and it goes back forever. 547 00:22:20,940 --> 00:22:24,000 But my colleague Elinor McKone has pointed out 548 00:22:24,000 --> 00:22:28,260 that actually, it would be important to look 549 00:22:28,260 --> 00:22:29,886 at real emotional expressions on faces. 550 00:22:29,886 --> 00:22:31,385 Maybe that's different behaviorally. 551 00:22:31,385 --> 00:22:33,600 It turns out it's very different behaviorally. 552 00:22:33,600 --> 00:22:37,470 One, you can tell if somebody's faking an emotional expression 553 00:22:37,470 --> 00:22:38,580 or if it's a real one. 554 00:22:38,580 --> 00:22:41,160 Like, OK, which of these is real fear, and which of these 555 00:22:41,160 --> 00:22:42,503 is staged fear? 556 00:22:42,503 --> 00:22:43,390 Duh! 557 00:22:43,390 --> 00:22:45,450 OK, so one, we're really attuned to that. 558 00:22:45,450 --> 00:22:46,970 I think that's really interesting. 559 00:22:46,970 --> 00:22:49,590 Just as a social perceptual ability, 560 00:22:49,590 --> 00:22:52,110 we spend a lot of time trying to figure out who's sincere, 561 00:22:52,110 --> 00:22:53,700 who's genuine, who's faking something, 562 00:22:53,700 --> 00:22:54,790 what's for real, right? 563 00:22:54,790 --> 00:22:55,290 You know? 564 00:22:55,290 --> 00:22:57,392 There's all kinds of shades of that. 565 00:22:57,392 --> 00:22:59,100 And here's one little piece of it, right? 566 00:22:59,100 --> 00:23:00,558 So I think that's very interesting. 567 00:23:00,558 --> 00:23:03,079 And they've shown that behaviorally, these phenomenon 568 00:23:03,079 --> 00:23:03,870 are very different. 569 00:23:03,870 --> 00:23:05,690 Just one example. 570 00:23:05,690 --> 00:23:09,300 A prior literature had shown that people with schizophrenia 571 00:23:09,300 --> 00:23:12,600 are particularly bad at reading facial expressions, 572 00:23:12,600 --> 00:23:17,580 using the standard measures, a standard stimuli, the Ekman six 573 00:23:17,580 --> 00:23:20,070 facial expressions. 574 00:23:20,070 --> 00:23:23,730 These guys replicated that finding and then showed 575 00:23:23,730 --> 00:23:26,530 that when you run the same experiment, 576 00:23:26,530 --> 00:23:29,830 but using not staged but real emotional expressions, 577 00:23:29,830 --> 00:23:32,870 schizophrenics are better than everyone else. 578 00:23:32,870 --> 00:23:35,810 OK, so it matters behaviorally, and it's interesting. 579 00:23:35,810 --> 00:23:37,440 OK. 580 00:23:37,440 --> 00:23:40,050 All right. 581 00:23:40,050 --> 00:23:42,420 Other things that we're doing-- 582 00:23:42,420 --> 00:23:43,050 right. 583 00:23:43,050 --> 00:23:47,280 Leyla, your TA here, who's done beautiful work on her thesis 584 00:23:47,280 --> 00:23:50,740 work with Tommy using MEG and other methods, 585 00:23:50,740 --> 00:23:53,710 is now working with me and Gabriel, 586 00:23:53,710 --> 00:23:56,170 using some of this magnificent data 587 00:23:56,170 --> 00:23:58,780 that Gabriel has collected over a bunch of years, where he's 588 00:23:58,780 --> 00:24:02,890 got intracranial recordings from human brains 589 00:24:02,890 --> 00:24:04,930 while people watch movies. 590 00:24:04,930 --> 00:24:06,340 This is so precious. 591 00:24:06,340 --> 00:24:08,680 These data are like a dream to me, 592 00:24:08,680 --> 00:24:10,600 as somebody who's been using functional 593 00:24:10,600 --> 00:24:14,260 MRI as my main hammer for the last 15 years. 594 00:24:14,260 --> 00:24:17,020 Functional MRI is magnificent, it's wonderful, it's fun, 595 00:24:17,020 --> 00:24:19,100 but it has fundamental limits. 596 00:24:19,100 --> 00:24:23,270 One, it has no time information worth a damn. 597 00:24:23,270 --> 00:24:25,720 And the computations that make up perception, 598 00:24:25,720 --> 00:24:28,900 including social perception, and language processing, and most 599 00:24:28,900 --> 00:24:30,730 of the interesting aspects of cognition, 600 00:24:30,730 --> 00:24:33,319 happen on the order of tens of milliseconds. 601 00:24:33,319 --> 00:24:34,360 We can't see any of that. 602 00:24:34,360 --> 00:24:37,000 It's all just squashed together like a pancake, right, 603 00:24:37,000 --> 00:24:38,770 with functional MRI. 604 00:24:38,770 --> 00:24:41,590 With intracranial recordings, you have exquisite time 605 00:24:41,590 --> 00:24:44,470 information, and you can see computations unfold over time. 606 00:24:44,470 --> 00:24:47,320 That's very precious. 607 00:24:47,320 --> 00:24:53,170 Second of all, in principle, with intracranial electrodes, 608 00:24:53,170 --> 00:24:54,820 you can test causality, something you 609 00:24:54,820 --> 00:24:56,470 can't do with functional MRI. 610 00:24:56,470 --> 00:25:00,680 You can stimulate and ask what tasks are disrupted. 611 00:25:00,680 --> 00:25:01,180 All right? 612 00:25:01,180 --> 00:25:03,580 So there's a huge number of cool things 613 00:25:03,580 --> 00:25:06,900 you can do with intracranial recordings. 614 00:25:06,900 --> 00:25:08,970 Leyla is looking at some of the data 615 00:25:08,970 --> 00:25:11,400 that Gabriel has been collecting, 616 00:25:11,400 --> 00:25:14,380 with intracranial recordings of people watching movies. 617 00:25:14,380 --> 00:25:18,210 And because these are rich, complex social stimuli, 618 00:25:18,210 --> 00:25:20,640 she's going to look at all kinds of things 619 00:25:20,640 --> 00:25:23,700 that we can try to extract from those data. 620 00:25:23,700 --> 00:25:27,630 Like, can you tell the identity of the person 621 00:25:27,630 --> 00:25:29,790 who's on the screen right now? 622 00:25:29,790 --> 00:25:33,570 Can you tell from their face, their voice, their body? 623 00:25:33,570 --> 00:25:35,970 Can you tell what action they're carrying out? 624 00:25:35,970 --> 00:25:38,010 Can you tell if the person on the screen right 625 00:25:38,010 --> 00:25:39,650 now is a good guy or a bad guy? 626 00:25:39,650 --> 00:25:41,220 Right? 627 00:25:41,220 --> 00:25:43,950 Can you tell what kind of social interactions are going on? 628 00:25:43,950 --> 00:25:46,170 So we know all of this stuff, all this information 629 00:25:46,170 --> 00:25:48,600 is extracted in the brain, because people are good at it. 630 00:25:48,600 --> 00:25:51,240 But to get a handle on the actual neural basis of how 631 00:25:51,240 --> 00:25:55,190 we carry out those perceptual processes, 632 00:25:55,190 --> 00:25:57,060 this will be a really cool tool. 633 00:25:57,060 --> 00:26:00,150 So that project is just starting now. 634 00:26:00,150 --> 00:26:03,870 And in other projects going on, Lindsey Powell, shown here, 635 00:26:03,870 --> 00:26:08,670 who's working with Rebecca Saxe, and Liz Spelke, and others, 636 00:26:08,670 --> 00:26:10,710 is using this NIRS method to look 637 00:26:10,710 --> 00:26:13,860 at blood flow changes in response 638 00:26:13,860 --> 00:26:15,720 to neural activity in infant brains. 639 00:26:15,720 --> 00:26:18,090 She's looking at some of those specializations 640 00:26:18,090 --> 00:26:20,400 that I showed you in my brain at the beginning 641 00:26:20,400 --> 00:26:22,740 and asking, which of those are present in infancy, 642 00:26:22,740 --> 00:26:25,740 a totally cool question. 643 00:26:25,740 --> 00:26:29,400 And Ben Deen, and Rebecca Saxe, and me, and a bunch of others 644 00:26:29,400 --> 00:26:33,660 are looking at a big chunk of the human brain that was one 645 00:26:33,660 --> 00:26:35,280 of my colored patches before. 646 00:26:35,280 --> 00:26:39,150 This whole dark gray region here is called 647 00:26:39,150 --> 00:26:41,760 the superior temporal sulcus. 648 00:26:41,760 --> 00:26:43,770 This is an inflated picture of the brain. 649 00:26:43,770 --> 00:26:45,300 That means-- usually, the cortexes 650 00:26:45,300 --> 00:26:46,820 are all folded up inside the head. 651 00:26:46,820 --> 00:26:48,870 You have to do that to fit it in there. 652 00:26:48,870 --> 00:26:50,620 But if you want to see the whole thing, 653 00:26:50,620 --> 00:26:52,229 you can mathematically inflate it. 654 00:26:52,229 --> 00:26:53,520 So that's what's happened here. 655 00:26:53,520 --> 00:26:56,040 And the dark bits are the bits that were inside of folds 656 00:26:56,040 --> 00:26:57,030 before it was inflated. 657 00:26:57,030 --> 00:26:58,560 So they're inside a sulcus, but now 658 00:26:58,560 --> 00:27:01,540 shown blown out to the surface. 659 00:27:01,540 --> 00:27:04,350 So this superior temporal sulcus running down here 660 00:27:04,350 --> 00:27:07,580 is one of the longest sulci in the human brain 661 00:27:07,580 --> 00:27:08,850 and one of the coolest. 662 00:27:08,850 --> 00:27:12,340 And an awful lot of social perception goes on right there. 663 00:27:12,340 --> 00:27:16,290 Ben Deen has a paper in press and some ongoing work 664 00:27:16,290 --> 00:27:18,720 where he shows that lots of different kinds 665 00:27:18,720 --> 00:27:22,350 of social, cognitive, and perceptual abilities 666 00:27:22,350 --> 00:27:26,460 actually inhabit somewhat distinct regions 667 00:27:26,460 --> 00:27:28,650 along the superior temporal sulcus. 668 00:27:28,650 --> 00:27:30,099 They're not perfectly discrete. 669 00:27:30,099 --> 00:27:31,890 Nothing is a neat little oval in the brain. 670 00:27:31,890 --> 00:27:33,723 Actually, they somewhat overlap, but there's 671 00:27:33,723 --> 00:27:35,352 a lot of organization in there. 672 00:27:35,352 --> 00:27:36,810 And that's cool because it gives us 673 00:27:36,810 --> 00:27:39,480 a lever to try to understand this whole big space 674 00:27:39,480 --> 00:27:41,420 of cognition.