WEBVTT 00:00:00.000 --> 00:00:02.490 align:middle line:84% The following content is provided under a Creative 00:00:02.490 --> 00:00:03.730 align:middle line:90% Commons license. 00:00:03.730 --> 00:00:06.120 align:middle line:84% Your support will help MIT OpenCourseWare 00:00:06.120 --> 00:00:10.090 align:middle line:84% continue to offer high quality educational resources for free. 00:00:10.090 --> 00:00:12.720 align:middle line:84% To make a donation or to view additional materials 00:00:12.720 --> 00:00:16.650 align:middle line:84% from hundreds of MIT courses, visit MIT OpenCourseWare 00:00:16.650 --> 00:00:21.030 align:middle line:90% at ocw.mit.edu. 00:00:21.030 --> 00:00:22.970 align:middle line:84% RAMESH RASKAR: And has done just terrific, 00:00:22.970 --> 00:00:23.970 align:middle line:90% terrific amount of work. 00:00:23.970 --> 00:00:26.790 align:middle line:84% He was also [INAUDIBLE] program manager 00:00:26.790 --> 00:00:28.860 align:middle line:84% where he led the effort that we have 00:00:28.860 --> 00:00:31.680 align:middle line:84% studied in the class, the origami lens 00:00:31.680 --> 00:00:33.668 align:middle line:90% and many interesting projects. 00:00:33.668 --> 00:00:35.460 align:middle line:84% So here, today's going to work on something 00:00:35.460 --> 00:00:37.410 align:middle line:90% that hopefully get us thinking. 00:00:37.410 --> 00:00:39.240 align:middle line:90% It's just at the right time. 00:00:39.240 --> 00:00:41.070 align:middle line:84% It's the middle of a semester, and we have 00:00:41.070 --> 00:00:43.290 align:middle line:90% learned about a lot of ideas. 00:00:43.290 --> 00:00:45.450 align:middle line:84% And all of us enjoy talking to Ravi 00:00:45.450 --> 00:00:48.005 align:middle line:84% about where the field is going and how 00:00:48.005 --> 00:00:54.720 align:middle line:84% we can classify and understand the field itself 00:00:54.720 --> 00:00:58.120 align:middle line:84% of computational imaging and computational camera better. 00:00:58.120 --> 00:01:00.010 align:middle line:90% So looking forward to it. 00:01:00.010 --> 00:01:00.990 align:middle line:90% RAVI ATHALE: Thank you. 00:01:00.990 --> 00:01:01.620 align:middle line:90% Thanks, Ramesh. 00:01:01.620 --> 00:01:05.940 align:middle line:84% It's a pleasure to come here and talk about this thing 00:01:05.940 --> 00:01:07.920 align:middle line:90% that all of us love so much-- 00:01:07.920 --> 00:01:08.970 align:middle line:90% cameras. 00:01:08.970 --> 00:01:12.960 align:middle line:84% And this, I worked a lot on this type of camera, 00:01:12.960 --> 00:01:14.310 align:middle line:90% as we cannot picture. 00:01:14.310 --> 00:01:19.260 align:middle line:84% So anyway, I sent the slides to Ramesh ahead of time 00:01:19.260 --> 00:01:20.010 align:middle line:90% to make sure. 00:01:20.010 --> 00:01:24.450 align:middle line:84% I didn't know how to calibrate where to go, 00:01:24.450 --> 00:01:26.950 align:middle line:84% but he said that these slides are just perfect, 00:01:26.950 --> 00:01:28.710 align:middle line:90% so I trust his judgment. 00:01:28.710 --> 00:01:32.700 align:middle line:84% And I wanted not just the seminar or lecture, 00:01:32.700 --> 00:01:34.180 align:middle line:90% but a lot more interactive. 00:01:34.180 --> 00:01:36.120 align:middle line:84% So we can go through some of these 00:01:36.120 --> 00:01:38.160 align:middle line:84% very quickly if I see from your faces 00:01:38.160 --> 00:01:43.200 align:middle line:84% that, why is he telling us these things, but be as it may. 00:01:43.200 --> 00:01:44.700 align:middle line:90% So history of camera-- 00:01:44.700 --> 00:01:48.330 align:middle line:84% I think this was meant more for general audience, so I really 00:01:48.330 --> 00:01:51.150 align:middle line:84% focused a lot on history and how we 00:01:51.150 --> 00:01:54.810 align:middle line:84% are at a dramatic breakpoint, how that breakpoint is 00:01:54.810 --> 00:01:57.640 align:middle line:84% likely to impact the future, and things like that. 00:01:57.640 --> 00:02:00.780 align:middle line:84% So interesting historical milestones 00:02:00.780 --> 00:02:02.940 align:middle line:84% that you may or may not have covered in the class. 00:02:02.940 --> 00:02:04.840 align:middle line:90% Let's go with that. 00:02:04.840 --> 00:02:07.210 align:middle line:84% But this is one thing, of course, 00:02:07.210 --> 00:02:11.430 align:middle line:84% now, if you need to be told about this, 00:02:11.430 --> 00:02:14.100 align:middle line:90% you all are carrying cameras. 00:02:14.100 --> 00:02:16.050 align:middle line:84% If you lived in London, you probably 00:02:16.050 --> 00:02:19.980 align:middle line:84% got photographed about 30 to 40 times during a typical workday 00:02:19.980 --> 00:02:23.430 align:middle line:84% as you stepped outside, and all these things. 00:02:23.430 --> 00:02:25.080 align:middle line:90% I just wanted to point out to-- 00:02:25.080 --> 00:02:30.330 align:middle line:84% I don't know how many of you will know what that image is. 00:02:30.330 --> 00:02:32.220 align:middle line:84% In order to know what that image is, 00:02:32.220 --> 00:02:37.500 align:middle line:84% you need to focus on the date, April 19, 1995. 00:02:37.500 --> 00:02:40.170 align:middle line:84% That date has a special significance. 00:02:40.170 --> 00:02:43.620 align:middle line:84% It was the Oklahoma City bombing, and this is the truck. 00:02:43.620 --> 00:02:46.320 align:middle line:84% This is the Ryder truck that carried the bomb. 00:02:46.320 --> 00:02:49.290 align:middle line:84% And this is a picture from a security camera 00:02:49.290 --> 00:02:51.450 align:middle line:90% in the bank across the street. 00:02:51.450 --> 00:02:55.560 align:middle line:84% And this image has been used by many, many people 00:02:55.560 --> 00:02:59.070 align:middle line:84% to justify the work on high dynamic imaging. 00:02:59.070 --> 00:03:01.230 align:middle line:84% Everything inside is completely underexposed. 00:03:01.230 --> 00:03:03.360 align:middle line:84% Everything outside is completely overexposed. 00:03:03.360 --> 00:03:07.090 align:middle line:84% As a result, this image, by itself, is pretty much useless. 00:03:07.090 --> 00:03:10.290 align:middle line:84% So things come in two sizes-- too big and too small. 00:03:10.290 --> 00:03:13.170 align:middle line:90% Similarly, this is the tape. 00:03:13.170 --> 00:03:15.730 align:middle line:84% This is a very interesting thing, 00:03:15.730 --> 00:03:20.160 align:middle line:84% and I also had another image of, somewhere in Iraq, 00:03:20.160 --> 00:03:23.705 align:middle line:84% an IED going off and blowing up, and some person holding 00:03:23.705 --> 00:03:24.330 align:middle line:90% the cell phone. 00:03:24.330 --> 00:03:28.710 align:middle line:84% And so I'm sure you're aware that, over the last few years, 00:03:28.710 --> 00:03:32.580 align:middle line:84% any significant news [INAUDIBLE] that happened in most places 00:03:32.580 --> 00:03:35.040 align:middle line:84% around the world, the first reports and the first images 00:03:35.040 --> 00:03:38.100 align:middle line:84% to come out of that were cell phone cameras. 00:03:38.100 --> 00:03:40.920 align:middle line:90% So everybody is a reporter. 00:03:40.920 --> 00:03:43.920 align:middle line:84% So anyway, one of the interesting things 00:03:43.920 --> 00:03:48.990 align:middle line:84% is this business about natural observation, 00:03:48.990 --> 00:03:52.230 align:middle line:84% gaps in the leaves projecting images 00:03:52.230 --> 00:03:54.300 align:middle line:90% of a partial solar eclipse. 00:03:54.300 --> 00:03:58.800 align:middle line:84% So our ancestors must have observed these things 00:03:58.800 --> 00:04:00.850 align:middle line:90% and said something is going on. 00:04:00.850 --> 00:04:04.230 align:middle line:84% So exactly what is going on, and how can we replicate it? 00:04:04.230 --> 00:04:07.650 align:middle line:84% How can we understand what's going on over there? 00:04:07.650 --> 00:04:09.930 align:middle line:84% This is another very beautiful photograph. 00:04:09.930 --> 00:04:13.500 align:middle line:84% I'm sure you have see waterdrop acting as a lens, 00:04:13.500 --> 00:04:14.590 align:middle line:90% forming the image. 00:04:14.590 --> 00:04:17.880 align:middle line:84% So just before there was any technology, 00:04:17.880 --> 00:04:24.630 align:middle line:84% before there was even a formal modern scientific process, 00:04:24.630 --> 00:04:28.170 align:middle line:84% our ancestors, a couple of thousand years ago, since they 00:04:28.170 --> 00:04:31.200 align:middle line:84% didn't have the distractions of cable TV and cell phones 00:04:31.200 --> 00:04:32.700 align:middle line:84% and all that, they had a lot of time 00:04:32.700 --> 00:04:37.530 align:middle line:84% to really very carefully observe nature and observe it 00:04:37.530 --> 00:04:40.140 align:middle line:84% over a long period of time from many different angles. 00:04:40.140 --> 00:04:45.480 align:middle line:84% So as an outcome of this was the observation 00:04:45.480 --> 00:04:48.810 align:middle line:84% of pinhole images going back several thousand years, 00:04:48.810 --> 00:04:50.400 align:middle line:90% and I believe-- 00:04:50.400 --> 00:04:52.620 align:middle line:84% at least I understand that this was 00:04:52.620 --> 00:04:57.300 align:middle line:84% the first comprehensive study on optics, Alhazen. 00:04:57.300 --> 00:05:01.317 align:middle line:84% And this is the drawing of a pinhole camera. 00:05:01.317 --> 00:05:03.150 align:middle line:84% I think, Ramesh, you have probably told them 00:05:03.150 --> 00:05:04.350 align:middle line:90% about the origin of the word-- 00:05:04.350 --> 00:05:04.970 align:middle line:90% RAMESH RASKAR: I have not. 00:05:04.970 --> 00:05:05.503 align:middle line:90% Go ahead. 00:05:05.503 --> 00:05:06.420 align:middle line:90% RAVI ATHALE: You have? 00:05:06.420 --> 00:05:07.503 align:middle line:90% RAMESH RASKAR: I have not. 00:05:07.503 --> 00:05:09.300 align:middle line:90% RAVI ATHALE: Oh, you have not? 00:05:09.300 --> 00:05:11.520 align:middle line:84% Camera, the literal meaning of the word, 00:05:11.520 --> 00:05:16.620 align:middle line:84% is room because the first camera was indeed a room with a hole 00:05:16.620 --> 00:05:20.310 align:middle line:84% there, and the image projected back there, so you can trace it 00:05:20.310 --> 00:05:23.100 align:middle line:90% or something like that. 00:05:23.100 --> 00:05:28.290 align:middle line:84% Now, what is interesting is, fast forward 1,000 years, 00:05:28.290 --> 00:05:30.540 align:middle line:84% and the world's largest pinhole camera 00:05:30.540 --> 00:05:33.540 align:middle line:90% was constructed in July 2006. 00:05:33.540 --> 00:05:37.380 align:middle line:84% The El Toro Marine Corps Air Station in Irvine 00:05:37.380 --> 00:05:38.340 align:middle line:90% was decommissioned. 00:05:38.340 --> 00:05:43.080 align:middle line:84% They wanted to preserve it, so they took an airplane hangar, 00:05:43.080 --> 00:05:45.030 align:middle line:84% closed the door, and drilled a hole 00:05:45.030 --> 00:05:49.210 align:middle line:84% the size of [INAUDIBLE] tennis ball or something. 00:05:49.210 --> 00:05:52.320 align:middle line:84% And on the backside, had a photographic film 00:05:52.320 --> 00:05:56.590 align:middle line:84% eight stories tall and 1/3 the length of a football field. 00:05:56.590 --> 00:05:59.670 align:middle line:90% And here it is. 00:05:59.670 --> 00:06:03.360 align:middle line:84% This is the image of the base upside down projected 00:06:03.360 --> 00:06:05.080 align:middle line:90% on the back wall. 00:06:05.080 --> 00:06:08.430 align:middle line:84% This is this the photographic emulsion, 00:06:08.430 --> 00:06:10.470 align:middle line:84% and there were a lot of numbers as 00:06:10.470 --> 00:06:13.800 align:middle line:84% to how many gallons of developer solution 00:06:13.800 --> 00:06:17.160 align:middle line:84% they needed and basically throw it over a barrel 00:06:17.160 --> 00:06:20.160 align:middle line:84% and take a broom and spread it over and develop it. 00:06:20.160 --> 00:06:22.440 align:middle line:84% And I don't know where that image is now. 00:06:22.440 --> 00:06:25.170 align:middle line:84% I think it is in California, some art museum 00:06:25.170 --> 00:06:26.670 align:middle line:90% somewhere in Los Angeles. 00:06:26.670 --> 00:06:30.150 align:middle line:84% The point of that is, over 1,000 years, 00:06:30.150 --> 00:06:33.210 align:middle line:84% and exactly the same operating principle. 00:06:33.210 --> 00:06:37.020 align:middle line:84% It's just a room, except a much, much larger room, 00:06:37.020 --> 00:06:39.652 align:middle line:84% and taking a picture of a much, much larger area. 00:06:39.652 --> 00:06:41.610 align:middle line:84% STUDENT: Right, you do make a good point there. 00:06:41.610 --> 00:06:43.068 align:middle line:84% They also [INAUDIBLE] how much time 00:06:43.068 --> 00:06:47.400 align:middle line:84% they had to spend to make the [INAUDIBLE].. 00:06:47.400 --> 00:06:50.730 align:middle line:90% RAMESH RASKAR: Yes, indeed. 00:06:50.730 --> 00:06:53.790 align:middle line:84% RAVI ATHALE: The first optical instrument are spectacles, 00:06:53.790 --> 00:06:56.070 align:middle line:84% and I think that really, the point-- 00:06:56.070 --> 00:06:57.840 align:middle line:90% this is an interesting point. 00:06:57.840 --> 00:06:59.640 align:middle line:90% Spectacles are glasses. 00:06:59.640 --> 00:07:03.000 align:middle line:84% It really is a mechanical packaging solution. 00:07:03.000 --> 00:07:06.510 align:middle line:84% So people realize that having these specifically shaped 00:07:06.510 --> 00:07:10.300 align:middle line:84% lenses improves your vision, but how do you do that? 00:07:10.300 --> 00:07:13.200 align:middle line:84% How do you make it convenient for you to wear? 00:07:13.200 --> 00:07:15.510 align:middle line:84% So people realize you have two years, 00:07:15.510 --> 00:07:17.280 align:middle line:84% and they're shaped in this manner. 00:07:17.280 --> 00:07:19.920 align:middle line:84% And there is the bridge of the nose, so 3 point support 00:07:19.920 --> 00:07:21.030 align:middle line:90% like this. 00:07:21.030 --> 00:07:24.060 align:middle line:84% Of course, then there were all the elaborations 00:07:24.060 --> 00:07:26.850 align:middle line:84% of the [INAUDIBLE] glasses sitting 00:07:26.850 --> 00:07:30.300 align:middle line:84% or the monocle that you can screw into your eye. 00:07:30.300 --> 00:07:32.910 align:middle line:90% But this is basically-- 00:07:32.910 --> 00:07:36.600 align:middle line:84% 1270 is the-- taking these optical phenomena 00:07:36.600 --> 00:07:39.480 align:middle line:84% and constructing a useful instrument that can 00:07:39.480 --> 00:07:43.260 align:middle line:90% be portable, and that is used. 00:07:43.260 --> 00:07:49.590 align:middle line:84% Beyond that, almost a 300 year gap between a single lens 00:07:49.590 --> 00:07:52.230 align:middle line:84% optical instrument and a compound optical instrument-- 00:07:52.230 --> 00:07:55.120 align:middle line:84% microscope and then telescope and everything. 00:07:55.120 --> 00:07:57.930 align:middle line:84% You might have known this the 400th anniversary 00:07:57.930 --> 00:08:01.770 align:middle line:84% of the Galilean telescope revolution, 00:08:01.770 --> 00:08:07.890 align:middle line:84% and this is the diagram [INAUDIBLE] multiple lenses. 00:08:07.890 --> 00:08:13.200 align:middle line:84% So going forward, the next history is photography. 00:08:13.200 --> 00:08:19.470 align:middle line:84% This is the first photograph ever by Niepce, 1827, 00:08:19.470 --> 00:08:24.330 align:middle line:84% and it's basically the [INAUDIBLE] sensitive media 00:08:24.330 --> 00:08:26.940 align:middle line:84% in various forms, and it just evolved. 00:08:26.940 --> 00:08:31.440 align:middle line:84% One of the big milestone was this sort of glass. 00:08:31.440 --> 00:08:34.909 align:middle line:84% It was coated in a paper and rolled in other things. 00:08:34.909 --> 00:08:38.340 align:middle line:90% So there's another tidbit-- 00:08:38.340 --> 00:08:40.169 align:middle line:90% Eastman Kodak. 00:08:40.169 --> 00:08:43.500 align:middle line:90% What does Kodak mean? 00:08:43.500 --> 00:08:46.170 align:middle line:90% Anybody knows? 00:08:46.170 --> 00:08:50.700 align:middle line:84% Eastman wanted a word that was a very crisp word, 00:08:50.700 --> 00:08:55.620 align:middle line:84% and that was very positive, forward-leaning kind of a word. 00:08:55.620 --> 00:08:59.970 align:middle line:84% And so apparently, Kodak is the sound 00:08:59.970 --> 00:09:01.950 align:middle line:84% that the shutter made when you push the button. 00:09:01.950 --> 00:09:03.270 align:middle line:90% [STUDENTS LAUGHING] 00:09:03.270 --> 00:09:06.960 align:middle line:84% So it was not George Eastman and his colleague Kodak Eastman 00:09:06.960 --> 00:09:08.520 align:middle line:90% Kodak was named after. 00:09:08.520 --> 00:09:11.440 align:middle line:84% But the slogan was, you push the button, and we do the rest. 00:09:11.440 --> 00:09:14.100 align:middle line:90% And I think $1 Brownie camera-- 00:09:14.100 --> 00:09:16.410 align:middle line:84% if you put it in historical context, 00:09:16.410 --> 00:09:18.300 align:middle line:84% that was around the same time when 00:09:18.300 --> 00:09:22.140 align:middle line:84% Model-T, maybe a few years earlier, 10, 15 years earlier. 00:09:22.140 --> 00:09:25.950 align:middle line:84% But it's taking essentially really high technology 00:09:25.950 --> 00:09:32.310 align:middle line:84% and using some innovation to bring it down to masses, 00:09:32.310 --> 00:09:33.870 align:middle line:90% so everybody can use it. 00:09:33.870 --> 00:09:37.500 align:middle line:84% So you can say that Model-T and the Brownie camera 00:09:37.500 --> 00:09:42.000 align:middle line:84% was the first age of modern consumer technological culture. 00:09:42.000 --> 00:09:45.000 align:middle line:90% 00:09:45.000 --> 00:09:46.890 align:middle line:84% More than three-color color emulsion, 00:09:46.890 --> 00:09:50.820 align:middle line:84% kodachrome stacked with doping and everything, 00:09:50.820 --> 00:09:52.650 align:middle line:90% and Polaroid instant camera. 00:09:52.650 --> 00:09:55.350 align:middle line:84% Of course, in 1945, the definition of instant 00:09:55.350 --> 00:09:57.000 align:middle line:90% was a little bit different. 00:09:57.000 --> 00:10:00.450 align:middle line:84% 30 seconds to two minutes was considered to be instant. 00:10:00.450 --> 00:10:02.310 align:middle line:90% And this, I was intrigued. 00:10:02.310 --> 00:10:05.730 align:middle line:84% This, of course, I found it on Wikipedia. 00:10:05.730 --> 00:10:08.070 align:middle line:84% But Maxwell, whom we don't normally 00:10:08.070 --> 00:10:10.748 align:middle line:84% associate with doing experimental work, 00:10:10.748 --> 00:10:12.165 align:middle line:84% but this was the color photograph. 00:10:12.165 --> 00:10:14.123 align:middle line:84% RAMESH RASKAR: Do you know how it was captured? 00:10:14.123 --> 00:10:17.100 align:middle line:84% RAVI ATHALE: At three separate exposures, and then stack them. 00:10:17.100 --> 00:10:20.110 align:middle line:84% Three separate films through three separate filters. 00:10:20.110 --> 00:10:22.110 align:middle line:84% RAMESH RASKAR: But how did he put them together? 00:10:22.110 --> 00:10:23.832 align:middle line:84% RAVI ATHALE: I guess outlined it by hand. 00:10:23.832 --> 00:10:25.290 align:middle line:84% RAMESH RASKAR: After developing it? 00:10:25.290 --> 00:10:26.580 align:middle line:90% RAVI ATHALE: Yeah. 00:10:26.580 --> 00:10:30.090 align:middle line:84% And of course, Abraham Lincoln was the first president 00:10:30.090 --> 00:10:34.230 align:middle line:84% to be photographed, and lots of beautiful photographs 00:10:34.230 --> 00:10:35.520 align:middle line:90% of Civil War. 00:10:35.520 --> 00:10:38.190 align:middle line:84% RAMESH RASKAR: The photograph looks still crisp. 00:10:38.190 --> 00:10:40.320 align:middle line:90% RAVI ATHALE: Yeah. 00:10:40.320 --> 00:10:40.935 align:middle line:90% I don't know. 00:10:40.935 --> 00:10:43.546 align:middle line:84% I don't if it was subsequently touched up. 00:10:43.546 --> 00:10:45.296 align:middle line:84% STUDENT: They used the light field camera. 00:10:45.296 --> 00:10:46.448 align:middle line:90% [STUDENTS LAUGHING] 00:10:46.448 --> 00:10:48.240 align:middle line:84% RAMESH RASKAR: Actually, light field camera 00:10:48.240 --> 00:10:50.220 align:middle line:90% was invented back in 1908. 00:10:50.220 --> 00:10:51.410 align:middle line:90% RAVI ATHALE: Yeah. 00:10:51.410 --> 00:10:53.452 align:middle line:84% STUDENT: It looks like it's a long exposure maybe 00:10:53.452 --> 00:10:55.493 align:middle line:84% because the trees in the background are kind of-- 00:10:55.493 --> 00:10:56.250 align:middle line:90% RAVI ATHALE: Yes. 00:10:56.250 --> 00:10:56.970 align:middle line:90% [INTERPOSING VOICES] 00:10:56.970 --> 00:10:57.240 align:middle line:90% RAVI ATHALE: I think-- 00:10:57.240 --> 00:10:58.350 align:middle line:90% STUDENT: Using a pinhole. 00:10:58.350 --> 00:11:00.150 align:middle line:84% RAVI ATHALE: All exposures were long. 00:11:00.150 --> 00:11:08.580 align:middle line:84% But-- Another historic milestone that captured [INAUDIBLE] 00:11:08.580 --> 00:11:13.410 align:middle line:84% display that [INAUDIBLE] so mechanical scanning 00:11:13.410 --> 00:11:17.820 align:middle line:84% and all kinds of other things, CCD. 00:11:17.820 --> 00:11:22.050 align:middle line:84% Just, I gave this seminar about a month 00:11:22.050 --> 00:11:25.380 align:middle line:84% and a half ago, and two weeks later, the CCD Nobel Prize 00:11:25.380 --> 00:11:26.020 align:middle line:90% was announced. 00:11:26.020 --> 00:11:28.620 align:middle line:84% And I don't know-- many of you are probably 00:11:28.620 --> 00:11:32.490 align:middle line:84% aware of the controversy whether these people really deserve 00:11:32.490 --> 00:11:37.290 align:middle line:84% the Nobel Prize or not because they were working on shift 00:11:37.290 --> 00:11:38.760 align:middle line:90% registered memory. 00:11:38.760 --> 00:11:41.070 align:middle line:84% And it was somebody else who said, hey, 00:11:41.070 --> 00:11:42.150 align:middle line:90% it's light sensitive. 00:11:42.150 --> 00:11:44.520 align:middle line:84% Anyway, but most of these things, 00:11:44.520 --> 00:11:48.450 align:middle line:84% there is always a controversy about including MRI 00:11:48.450 --> 00:11:50.460 align:middle line:90% and all kinds of other things. 00:11:50.460 --> 00:11:52.830 align:middle line:90% But this is kind of interesting. 00:11:52.830 --> 00:11:53.940 align:middle line:90% Here. 00:11:53.940 --> 00:11:56.730 align:middle line:84% Look at this, the first Fairchild camera. 00:11:56.730 --> 00:12:00.150 align:middle line:90% So I think 88 pounds. 00:12:00.150 --> 00:12:04.920 align:middle line:90% 0.01 megapixels, 23 seconds. 00:12:04.920 --> 00:12:07.740 align:middle line:84% And actually, if you look at the date, December '75. 00:12:07.740 --> 00:12:09.240 align:middle line:90% That's not that far back. 00:12:09.240 --> 00:12:10.290 align:middle line:90% RAMESH RASKAR: Yeah. 00:12:10.290 --> 00:12:12.660 align:middle line:84% RAVI ATHALE: That's not that far back at all. 00:12:12.660 --> 00:12:17.820 align:middle line:84% Then the subsequent digital camera [INAUDIBLE] Sony 00:12:17.820 --> 00:12:24.705 align:middle line:84% [INAUDIBLE],, quarter megapixel DCS 100, 1 megapixel $20,000. 00:12:24.705 --> 00:12:29.730 align:middle line:84% And '97 was the first transmission of picture 00:12:29.730 --> 00:12:31.800 align:middle line:84% from cell phone and cell phone camera. 00:12:31.800 --> 00:12:33.600 align:middle line:84% And this last one is very interesting. 00:12:33.600 --> 00:12:39.190 align:middle line:84% Very recently, Kodak stopped production of kodachrome film, 00:12:39.190 --> 00:12:43.200 align:middle line:84% so if anybody has film cameras, well, you better start 00:12:43.200 --> 00:12:45.330 align:middle line:90% stocking up on old film. 00:12:45.330 --> 00:12:49.260 align:middle line:84% And 2006, the 10 megapixel cell phone. 00:12:49.260 --> 00:12:52.500 align:middle line:84% I think with this camera, pixel cell phone cameras and all 00:12:52.500 --> 00:12:55.900 align:middle line:84% that, instead of a cell phone with a camera, 00:12:55.900 --> 00:12:57.720 align:middle line:84% this is a camera with a cell phone. 00:12:57.720 --> 00:13:01.530 align:middle line:84% So the bulk of it is probably the optics and everything else. 00:13:01.530 --> 00:13:05.550 align:middle line:84% Optics, mostly, that goes with it. 00:13:05.550 --> 00:13:07.320 align:middle line:90% You have heard of Moore's law? 00:13:07.320 --> 00:13:09.810 align:middle line:84% There is something called Kennedy's law. 00:13:09.810 --> 00:13:11.670 align:middle line:90% Empirical observation. 00:13:11.670 --> 00:13:13.320 align:middle line:90% How many pixels per dollar? 00:13:13.320 --> 00:13:15.330 align:middle line:84% And that's of course, a large scale, 00:13:15.330 --> 00:13:18.630 align:middle line:84% and this is a linear scale in years. 00:13:18.630 --> 00:13:25.110 align:middle line:84% From this DCS 460, 100 pixels per dollar, 00:13:25.110 --> 00:13:26.490 align:middle line:90% to all the way here. 00:13:26.490 --> 00:13:27.990 align:middle line:90% And we will probably-- 00:13:27.990 --> 00:13:32.190 align:middle line:84% I don't know, this is still an old girl. 00:13:32.190 --> 00:13:36.540 align:middle line:84% A megapixel camera, I think $5, and I 00:13:36.540 --> 00:13:38.700 align:middle line:84% think the goal of many of these companies 00:13:38.700 --> 00:13:40.560 align:middle line:90% is megapixel camera for a buck. 00:13:40.560 --> 00:13:44.280 align:middle line:84% Everything-- the lens, the focal plane array, the PC board, 00:13:44.280 --> 00:13:46.310 align:middle line:90% and everything ready to plug in. 00:13:46.310 --> 00:13:48.060 align:middle line:84% RAMESH RASKAR: With a wafer level cameras. 00:13:48.060 --> 00:13:48.900 align:middle line:84% RAVI ATHALE: With the wafer level cameras. 00:13:48.900 --> 00:13:49.890 align:middle line:84% RAMESH RASKAR: It's already about $1. 00:13:49.890 --> 00:13:50.100 align:middle line:90% RAVI ATHALE: Yeah. 00:13:50.100 --> 00:13:51.410 align:middle line:90% RAMESH RASKAR: Less than $1. 00:13:51.410 --> 00:13:54.120 align:middle line:84% RAVI ATHALE: So megapixel per dollar, so it's already 00:13:54.120 --> 00:13:55.632 align:middle line:90% out there somewhere. 00:13:55.632 --> 00:13:58.230 align:middle line:90% 00:13:58.230 --> 00:14:02.310 align:middle line:84% And this is the picture gallery, so the [INAUDIBLE] 100, 00:14:02.310 --> 00:14:03.720 align:middle line:90% Samsung. 00:14:03.720 --> 00:14:08.520 align:middle line:84% This is the Omnivision's focal plane array, 00:14:08.520 --> 00:14:11.550 align:middle line:90% 2.1 by 2.3 millimeters. 00:14:11.550 --> 00:14:14.010 align:middle line:84% And this is a very interesting thing. 00:14:14.010 --> 00:14:16.840 align:middle line:90% [INTERPOSING VOICES] 00:14:16.840 --> 00:14:21.930 align:middle line:84% For endoscopy, video endoscopy, 1.8 millimeter diameter 00:14:21.930 --> 00:14:27.600 align:middle line:84% includes lens, the focal plane, the cable for sending power, 00:14:27.600 --> 00:14:29.970 align:middle line:90% sending the signals back. 00:14:29.970 --> 00:14:36.450 align:middle line:84% Not great resolution, 326 by 382, but if something 00:14:36.450 --> 00:14:39.130 align:middle line:84% needs to be stuck inside your body, smaller the better, 00:14:39.130 --> 00:14:45.000 align:middle line:84% and 1.8 millimeter is definitely, definitely nice. 00:14:45.000 --> 00:14:50.280 align:middle line:84% This chip is the one that's used in the pill camera 00:14:50.280 --> 00:14:52.575 align:middle line:84% that you swallow to image your insides. 00:14:52.575 --> 00:14:56.940 align:middle line:90% 00:14:56.940 --> 00:15:01.440 align:middle line:84% I thought of something-- that imaging case study, 00:15:01.440 --> 00:15:04.950 align:middle line:84% studying the inner space versus studying the outer space. 00:15:04.950 --> 00:15:09.810 align:middle line:84% So the camera pill and the Hubble Space Telescope. 00:15:09.810 --> 00:15:11.620 align:middle line:84% RAMESH RASKAR: Both are shaped like a pill. 00:15:11.620 --> 00:15:13.320 align:middle line:84% RAVI ATHALE: (LAUGHING) Both are shaped like a pill. 00:15:13.320 --> 00:15:14.700 align:middle line:90% RAMESH RASKAR: Just a little. 00:15:14.700 --> 00:15:16.367 align:middle line:84% RAVI ATHALE: Both are shaped like a pill 00:15:16.367 --> 00:15:20.550 align:middle line:84% in the sense this was actually in the montage program 00:15:20.550 --> 00:15:21.990 align:middle line:90% that I started at DARPA. 00:15:21.990 --> 00:15:26.640 align:middle line:84% That was one of the motivations that, over the past 30, 40 00:15:26.640 --> 00:15:31.410 align:middle line:84% years, the displays have become thinner, 00:15:31.410 --> 00:15:35.380 align:middle line:84% from bulky CRTs to thin, flat panel displays. 00:15:35.380 --> 00:15:38.250 align:middle line:84% But the cameras, the aspect ratio of the cameras, 00:15:38.250 --> 00:15:41.970 align:middle line:84% have not changed in the past however many thousand years. 00:15:41.970 --> 00:15:45.930 align:middle line:84% They are always tube like, and so the goal was, can you 00:15:45.930 --> 00:15:47.820 align:middle line:90% squash the cameras. 00:15:47.820 --> 00:15:51.120 align:middle line:84% Can you make a camera that is five millimeters thick, 00:15:51.120 --> 00:15:54.090 align:middle line:84% but the effective focal length is 50 millimeters, 00:15:54.090 --> 00:15:57.570 align:middle line:84% and the light gathering power also [INAUDIBLE]?? 00:15:57.570 --> 00:16:00.180 align:middle line:84% And there are various tricks that you can play. 00:16:00.180 --> 00:16:02.190 align:middle line:84% The origami lens that you probably studied 00:16:02.190 --> 00:16:04.210 align:middle line:90% is one of the examples. 00:16:04.210 --> 00:16:07.000 align:middle line:84% A multi aperture is another example. 00:16:07.000 --> 00:16:08.850 align:middle line:84% But one of the things about-- and I'm 00:16:08.850 --> 00:16:11.850 align:middle line:84% not sure, in this presentation, there are the technical details 00:16:11.850 --> 00:16:13.270 align:middle line:90% or specifications. 00:16:13.270 --> 00:16:14.880 align:middle line:84% But it's very interesting, if you 00:16:14.880 --> 00:16:17.340 align:middle line:84% look at the specifications of, say, Hubble, a pill 00:16:17.340 --> 00:16:22.080 align:middle line:84% camera, or take one of the modern lithography lenses, 00:16:22.080 --> 00:16:24.360 align:middle line:84% I don't think I have a picture of that, 00:16:24.360 --> 00:16:29.490 align:middle line:84% but the model lithography lenses are a marvel of technology. 00:16:29.490 --> 00:16:34.080 align:middle line:84% That these things-- getting-- you remember how much-- 00:16:34.080 --> 00:16:36.147 align:middle line:84% they weigh several thousand pounds. 00:16:36.147 --> 00:16:37.230 align:middle line:90% RAMESH RASKAR: [INAUDIBLE] 00:16:37.230 --> 00:16:42.240 align:middle line:84% RAVI ATHALE: Some obscenely heavy, huge number 00:16:42.240 --> 00:16:46.890 align:middle line:84% of elements in this, and cost multiple million dollars. 00:16:46.890 --> 00:16:50.790 align:middle line:84% And the reason is that it's absolutely diffraction 00:16:50.790 --> 00:16:54.660 align:middle line:84% limited performance over a fairly wide field of view 00:16:54.660 --> 00:16:57.840 align:middle line:84% with zero distortion and all the aberrations, 00:16:57.840 --> 00:16:59.070 align:middle line:90% everything corrected. 00:16:59.070 --> 00:17:02.790 align:middle line:90% 00:17:02.790 --> 00:17:06.000 align:middle line:84% Now, multi-scale images broad, wide, and deep. 00:17:06.000 --> 00:17:09.300 align:middle line:84% Now everybody, gigapixel imaging, 00:17:09.300 --> 00:17:12.780 align:middle line:84% since the inauguration, it has really taken off. 00:17:12.780 --> 00:17:17.819 align:middle line:84% There are so many images like that, having the wide panorama 00:17:17.819 --> 00:17:21.750 align:middle line:84% and finding the details over here somewhere. 00:17:21.750 --> 00:17:22.900 align:middle line:90% The eagle. 00:17:22.900 --> 00:17:30.420 align:middle line:84% And so can we get here from where we are right now, 00:17:30.420 --> 00:17:36.300 align:middle line:84% and in a realistic size, weight, cost thing? 00:17:36.300 --> 00:17:40.590 align:middle line:90% Can we get a gigapixel camera? 00:17:40.590 --> 00:17:43.110 align:middle line:84% Can we get a 10 or 100 gigapixel camera? 00:17:43.110 --> 00:17:44.340 align:middle line:90% Very wide field of view. 00:17:44.340 --> 00:17:45.870 align:middle line:90% Very high resolution. 00:17:45.870 --> 00:17:48.480 align:middle line:84% These are some of the challenges that will be addressed, 00:17:48.480 --> 00:17:51.010 align:middle line:90% and not doing brute force. 00:17:51.010 --> 00:17:56.190 align:middle line:84% So it's an interesting prospect, and it's 00:17:56.190 --> 00:17:58.682 align:middle line:90% an interesting direction. 00:17:58.682 --> 00:18:01.140 align:middle line:84% RAMESH RASKAR: But in terms of the cameras that individuals 00:18:01.140 --> 00:18:01.650 align:middle line:90% will carry? 00:18:01.650 --> 00:18:03.108 align:middle line:84% Or this will be installed, and then 00:18:03.108 --> 00:18:06.330 align:middle line:84% you have access to it, so that you 00:18:06.330 --> 00:18:10.130 align:middle line:84% can [INAUDIBLE] of interest, you can capture what you want? 00:18:10.130 --> 00:18:13.200 align:middle line:84% RAVI ATHALE: This is kind of an interesting thing. 00:18:13.200 --> 00:18:19.290 align:middle line:84% You can imagine an individual, like special operations 00:18:19.290 --> 00:18:24.360 align:middle line:84% soldiers, carrying binoculars with the standard binocular 00:18:24.360 --> 00:18:29.400 align:middle line:84% form factor that will be equivalent to a gigapixel. 00:18:29.400 --> 00:18:32.580 align:middle line:84% But there are no gigapixel displays. 00:18:32.580 --> 00:18:36.060 align:middle line:84% Human vision doesn't have the space bandwidth for gigapixel. 00:18:36.060 --> 00:18:40.440 align:middle line:84% So within that gigapixel, you should be able to [INAUDIBLE] 00:18:40.440 --> 00:18:43.470 align:middle line:84% and zoom in and look at the details. 00:18:43.470 --> 00:18:45.630 align:middle line:84% There is an ongoing DARPA program-- 00:18:45.630 --> 00:18:49.650 align:middle line:84% I don't know what phase it is at-- 00:18:49.650 --> 00:18:54.630 align:middle line:84% some cognitive technology for warfighters, blah, blah. 00:18:54.630 --> 00:18:57.900 align:middle line:84% It was colloquially called Luke's Binoculars. 00:18:57.900 --> 00:18:59.880 align:middle line:84% So in Star Wars, Luke has these binoculars, 00:18:59.880 --> 00:19:01.110 align:middle line:90% and you're looking at it. 00:19:01.110 --> 00:19:05.400 align:middle line:84% And it could put a cursor that alien there, or that's that. 00:19:05.400 --> 00:19:08.310 align:middle line:84% It was primarily focused on the processing 00:19:08.310 --> 00:19:13.510 align:middle line:84% based on neuromorphic algorithms or something like that. 00:19:13.510 --> 00:19:20.760 align:middle line:84% So the idea will be, if you can design the camera or an imaging 00:19:20.760 --> 00:19:25.660 align:middle line:84% sensor that can capture a gigapixel worth of information, 00:19:25.660 --> 00:19:29.820 align:middle line:84% and if you have this back end, this neuromorphically induced 00:19:29.820 --> 00:19:33.420 align:middle line:84% processing, that can cue the soldiers to look here, there, 00:19:33.420 --> 00:19:38.700 align:middle line:84% and the soldier can appropriately focus there. 00:19:38.700 --> 00:19:40.470 align:middle line:84% In terms of what kind of things that 00:19:40.470 --> 00:19:43.410 align:middle line:84% will be really useful for warfighter right now, 00:19:43.410 --> 00:19:45.000 align:middle line:90% that could be one of the things. 00:19:45.000 --> 00:19:48.480 align:middle line:84% And then of course, you have platforms at various levels, 00:19:48.480 --> 00:19:51.570 align:middle line:84% from micro UAVs with about nine inch wingspan, 00:19:51.570 --> 00:19:53.370 align:middle line:84% to Predators to all the [INAUDIBLE] 00:19:53.370 --> 00:19:54.780 align:middle line:90% space based imaging. 00:19:54.780 --> 00:19:59.400 align:middle line:84% And the field of view, and the pixel resolution requirements, 00:19:59.400 --> 00:20:02.060 align:middle line:84% and everything-- it spans all over the space. 00:20:02.060 --> 00:20:06.420 align:middle line:90% 00:20:06.420 --> 00:20:09.400 align:middle line:84% So I'm switching gears a little bit, 00:20:09.400 --> 00:20:15.570 align:middle line:84% and previous was, you can say, an anecdotal or event 00:20:15.570 --> 00:20:17.670 align:middle line:90% based vision of the history. 00:20:17.670 --> 00:20:22.800 align:middle line:84% Now, we'll go into a little bit of a technical angle 00:20:22.800 --> 00:20:24.910 align:middle line:90% of looking at history. 00:20:24.910 --> 00:20:28.290 align:middle line:84% So I think, first, one of the questions-- and this 00:20:28.290 --> 00:20:33.360 align:middle line:84% also gets into where we should be headed anyway. 00:20:33.360 --> 00:20:38.730 align:middle line:84% So imaging sensors, or you can say sensors in general, 00:20:38.730 --> 00:20:40.570 align:middle line:90% why do we do this? 00:20:40.570 --> 00:20:42.510 align:middle line:84% What it is, and why do we do this? 00:20:42.510 --> 00:20:47.460 align:middle line:84% So questions we ask, and this is something-- these 00:20:47.460 --> 00:20:48.990 align:middle line:90% are very generic questions. 00:20:48.990 --> 00:20:52.300 align:middle line:84% And we constantly are asking these questions, who, what, 00:20:52.300 --> 00:20:54.030 align:middle line:90% where, when. 00:20:54.030 --> 00:20:56.490 align:middle line:84% That's the sensing front end part, 00:20:56.490 --> 00:21:00.550 align:middle line:84% and how and why is the analysis and exploration part. 00:21:00.550 --> 00:21:07.290 align:middle line:84% So when you reduce it down to its sense, 00:21:07.290 --> 00:21:11.850 align:middle line:84% all we care about is answering these questions. 00:21:11.850 --> 00:21:16.890 align:middle line:84% When it comes to using imaging as spatial sensors, 00:21:16.890 --> 00:21:21.720 align:middle line:84% not imaging to take photographs of your vacation, where 00:21:21.720 --> 00:21:24.870 align:middle line:84% you can view this and recapture the experience. 00:21:24.870 --> 00:21:27.660 align:middle line:84% That's a different world, and in some sense, 00:21:27.660 --> 00:21:31.170 align:middle line:84% computational photography and computational imaging, 00:21:31.170 --> 00:21:34.170 align:middle line:84% that's where-- when I say computational imaging, 00:21:34.170 --> 00:21:36.420 align:middle line:84% you are looking at imaging, you are considering 00:21:36.420 --> 00:21:38.970 align:middle line:90% that as a spatial sensor. 00:21:38.970 --> 00:21:41.670 align:middle line:84% A spatial sensor means a system that 00:21:41.670 --> 00:21:44.580 align:middle line:84% is designed to answer these questions. 00:21:44.580 --> 00:21:50.310 align:middle line:84% Identifying various constituent elements in the scene, 00:21:50.310 --> 00:21:52.950 align:middle line:84% specifying their location and their time. 00:21:52.950 --> 00:21:56.250 align:middle line:90% 00:21:56.250 --> 00:22:00.510 align:middle line:84% Now, within this particular realm, 00:22:00.510 --> 00:22:04.860 align:middle line:84% you have two different sensory modalities. 00:22:04.860 --> 00:22:07.560 align:middle line:84% One, you can say proximate sensor, 00:22:07.560 --> 00:22:09.750 align:middle line:84% and the other is standard sensor. 00:22:09.750 --> 00:22:13.260 align:middle line:84% So proximate sensor, I want to know what 00:22:13.260 --> 00:22:14.820 align:middle line:90% is the texture of a surface. 00:22:14.820 --> 00:22:17.640 align:middle line:84% I put my hand at the sensor are move around. 00:22:17.640 --> 00:22:20.250 align:middle line:90% I say, is it rough or smooth? 00:22:20.250 --> 00:22:21.750 align:middle line:90% Is it uniform? 00:22:21.750 --> 00:22:23.550 align:middle line:90% What is the granularity? 00:22:23.550 --> 00:22:26.790 align:middle line:84% In other words, your sensors are spatially 00:22:26.790 --> 00:22:32.700 align:middle line:84% co-located with the object that you are trying to measure. 00:22:32.700 --> 00:22:35.340 align:middle line:84% And the other thing is your sensors 00:22:35.340 --> 00:22:37.200 align:middle line:90% are separated by some distance. 00:22:37.200 --> 00:22:43.560 align:middle line:84% Now, that standoff is a malleable soft concept. 00:22:43.560 --> 00:22:49.260 align:middle line:84% Say, for example, take the magnetic hard drive disk head. 00:22:49.260 --> 00:22:53.310 align:middle line:84% So for a magnetic storage, the heads will be floating, 00:22:53.310 --> 00:22:57.300 align:middle line:84% I don't know, a few microns or a few hundred nanometers 00:22:57.300 --> 00:22:58.870 align:middle line:90% above the disk surface. 00:22:58.870 --> 00:23:02.550 align:middle line:84% So if, when you go to the optical disks, 00:23:02.550 --> 00:23:05.820 align:middle line:84% and the optical head is floating two millimeters away 00:23:05.820 --> 00:23:08.650 align:middle line:84% from the surface, that's a huge distance. 00:23:08.650 --> 00:23:13.620 align:middle line:84% So that's a standoff for writing and reading, 00:23:13.620 --> 00:23:17.100 align:middle line:84% whereas a standoff, if you're talking about a Hubble Space 00:23:17.100 --> 00:23:18.780 align:middle line:84% Telescope that's looking at the farthest 00:23:18.780 --> 00:23:23.460 align:middle line:84% galaxy in the universe, that standoff distance is however 00:23:23.460 --> 00:23:25.810 align:middle line:90% many billion light years away. 00:23:25.810 --> 00:23:30.030 align:middle line:84% But the point is, where you're making the measurements is not 00:23:30.030 --> 00:23:33.240 align:middle line:84% co-located with what it is that you are trying to measure. 00:23:33.240 --> 00:23:35.850 align:middle line:90% 00:23:35.850 --> 00:23:37.680 align:middle line:84% If you are doing standoff sensing, 00:23:37.680 --> 00:23:40.290 align:middle line:84% it has to involve real propagation. 00:23:40.290 --> 00:23:42.840 align:middle line:90% No ifs, ands, or buts. 00:23:42.840 --> 00:23:48.390 align:middle line:84% And that's almost contained in the definition of wave motion. 00:23:48.390 --> 00:23:51.540 align:middle line:84% Wave is defined as a phenomenon that 00:23:51.540 --> 00:23:55.590 align:middle line:84% carries energy and information over distance 00:23:55.590 --> 00:23:59.170 align:middle line:90% without material transport. 00:23:59.170 --> 00:24:03.180 align:middle line:84% So that's the difference between a soundwave and wind. 00:24:03.180 --> 00:24:04.920 align:middle line:90% Wind will carry leaves with it. 00:24:04.920 --> 00:24:07.140 align:middle line:84% Soundwave will not carry leaves with it. 00:24:07.140 --> 00:24:10.470 align:middle line:84% Both involve movement of air molecules, 00:24:10.470 --> 00:24:15.930 align:middle line:84% but soundwave is capable of carrying energy and information 00:24:15.930 --> 00:24:19.770 align:middle line:84% over a distance without a bulk transport. 00:24:19.770 --> 00:24:22.920 align:middle line:84% And then of course, you have the electromagnetic waves, 00:24:22.920 --> 00:24:25.530 align:middle line:84% then you have seismic waves, all kinds of waves. 00:24:25.530 --> 00:24:28.290 align:middle line:90% But nonetheless, wave motion-- 00:24:28.290 --> 00:24:33.480 align:middle line:84% it is at the heart of any kind of a standoff sensing activity. 00:24:33.480 --> 00:24:38.810 align:middle line:84% So when you have wave, and you have standoff distance, 00:24:38.810 --> 00:24:44.870 align:middle line:84% you have wave propagation that involves diffraction. 00:24:44.870 --> 00:24:47.750 align:middle line:84% And that diffraction essentially strangles 00:24:47.750 --> 00:24:50.820 align:middle line:84% the spatial organization of signals. 00:24:50.820 --> 00:24:55.790 align:middle line:84% So in other words, this particular scene, 00:24:55.790 --> 00:24:58.250 align:middle line:84% there is ambient light that is reflected off 00:24:58.250 --> 00:25:01.500 align:middle line:84% your face as individual spots on your face. 00:25:01.500 --> 00:25:04.880 align:middle line:84% And so right there, if I put a photodetector 00:25:04.880 --> 00:25:08.210 align:middle line:84% and move it around your face, it can measure 00:25:08.210 --> 00:25:10.430 align:middle line:90% the reflectance of the surface. 00:25:10.430 --> 00:25:13.410 align:middle line:84% If I move that photodetector over here, 00:25:13.410 --> 00:25:16.040 align:middle line:84% and if I put a photographic film or a CCD 00:25:16.040 --> 00:25:18.620 align:middle line:84% without any optical element, I'm just 00:25:18.620 --> 00:25:21.290 align:middle line:84% going to get basically a uniform exposure. 00:25:21.290 --> 00:25:24.480 align:middle line:84% All that spatial organization is lost. 00:25:24.480 --> 00:25:32.850 align:middle line:84% So we effectively, when we are doing the standoff sensing, 00:25:32.850 --> 00:25:35.040 align:middle line:90% we have two parts. 00:25:35.040 --> 00:25:38.540 align:middle line:84% One, you can call it the source encoding. 00:25:38.540 --> 00:25:44.750 align:middle line:84% That is the phenomenology by which the objects in the scene 00:25:44.750 --> 00:25:48.140 align:middle line:84% impress information about themselves 00:25:48.140 --> 00:25:50.310 align:middle line:84% on the electromagnetic radiation, 00:25:50.310 --> 00:25:53.000 align:middle line:84% whether it's by emission or reflection or absorption 00:25:53.000 --> 00:25:54.050 align:middle line:90% or scattering. 00:25:54.050 --> 00:25:58.070 align:middle line:84% And the full domain of the electromagnetic wavelength, 00:25:58.070 --> 00:26:02.060 align:middle line:84% the amplitude and wavelength and polarization and everything, 00:26:02.060 --> 00:26:04.470 align:middle line:84% is used to encode that information. 00:26:04.470 --> 00:26:08.600 align:middle line:84% And the other is channel encoding or channel distortion. 00:26:08.600 --> 00:26:11.030 align:middle line:84% That is the result of the propagation 00:26:11.030 --> 00:26:16.130 align:middle line:84% of that wave between the objects and your entrance pupil, 00:26:16.130 --> 00:26:17.960 align:middle line:90% or where you're entering. 00:26:17.960 --> 00:26:19.320 align:middle line:90% So these are the two things. 00:26:19.320 --> 00:26:22.550 align:middle line:84% So when we say about processing, we 00:26:22.550 --> 00:26:25.220 align:middle line:84% are talking about removing the channel distortion 00:26:25.220 --> 00:26:28.520 align:middle line:84% or undoing the effects of diffraction. 00:26:28.520 --> 00:26:32.090 align:middle line:84% So what are the components of an imaging sensor? 00:26:32.090 --> 00:26:36.330 align:middle line:84% And I'm sure you have done this in this class. 00:26:36.330 --> 00:26:38.930 align:middle line:84% So there is the front end element 00:26:38.930 --> 00:26:41.630 align:middle line:84% that operates on the electromagnetic wavelength 00:26:41.630 --> 00:26:44.610 align:middle line:90% of the optical wafer directly. 00:26:44.610 --> 00:26:46.130 align:middle line:84% Then there is the transduction that 00:26:46.130 --> 00:26:47.840 align:middle line:84% takes that electromagnetic energy 00:26:47.840 --> 00:26:50.150 align:middle line:84% and transduces it into some other form. 00:26:50.150 --> 00:26:52.760 align:middle line:84% In photographic film, it was chemical. 00:26:52.760 --> 00:26:56.090 align:middle line:84% In CCDs and CMOS, it's the electrical energy. 00:26:56.090 --> 00:26:58.460 align:middle line:84% And then there is the storage display processing 00:26:58.460 --> 00:26:59.700 align:middle line:90% and exploitation. 00:26:59.700 --> 00:27:03.140 align:middle line:84% So these are the three stages, and I'm using the [INAUDIBLE].. 00:27:03.140 --> 00:27:08.360 align:middle line:84% This color is not that strong, but blue means biology and pink 00:27:08.360 --> 00:27:09.510 align:middle line:90% means technology. 00:27:09.510 --> 00:27:12.110 align:middle line:84% So in the pre-prehistoric period, 00:27:12.110 --> 00:27:14.330 align:middle line:84% all these elements were biological. 00:27:14.330 --> 00:27:17.090 align:middle line:84% The front end element was the optics of your eye. 00:27:17.090 --> 00:27:18.980 align:middle line:84% The transduction was your retina, 00:27:18.980 --> 00:27:21.650 align:middle line:84% and the subsequent processing going into [INAUDIBLE] 00:27:21.650 --> 00:27:24.440 align:middle line:84% went all the way up to the prefrontal cortex. 00:27:24.440 --> 00:27:28.640 align:middle line:84% But some recognition or some processing exploitation 00:27:28.640 --> 00:27:29.450 align:middle line:90% took place. 00:27:29.450 --> 00:27:31.430 align:middle line:84% It was probably before language, so we 00:27:31.430 --> 00:27:35.930 align:middle line:84% don't know whether pre-prehistoric primates, 00:27:35.930 --> 00:27:37.490 align:middle line:90% and what terms they thought. 00:27:37.490 --> 00:27:44.330 align:middle line:84% So then we come forward to prehistoric and historic period 00:27:44.330 --> 00:27:46.550 align:middle line:90% down to cave paintings. 00:27:46.550 --> 00:27:51.440 align:middle line:84% So there, the storage and display "technology," 00:27:51.440 --> 00:27:53.510 align:middle line:90% in quotes, was introduced. 00:27:53.510 --> 00:27:56.180 align:middle line:84% Everything else is still biological, 00:27:56.180 --> 00:27:59.450 align:middle line:84% but capturing and storing that experience-- 00:27:59.450 --> 00:28:03.080 align:middle line:84% and beyond this prehistoric and historic period, 00:28:03.080 --> 00:28:07.340 align:middle line:84% you can imagine the technology getting more sophisticated, 00:28:07.340 --> 00:28:10.290 align:middle line:84% but the principle did not change. 00:28:10.290 --> 00:28:14.360 align:middle line:84% And it stayed that way through the prehistoric and historic 00:28:14.360 --> 00:28:16.910 align:middle line:90% period. 00:28:16.910 --> 00:28:20.730 align:middle line:84% Now, as we go forward to pre-industrial period, 00:28:20.730 --> 00:28:24.270 align:middle line:84% and as I said, the optical elements were introduced. 00:28:24.270 --> 00:28:27.710 align:middle line:84% So the optics of the eye, or the front end element 00:28:27.710 --> 00:28:32.960 align:middle line:84% of the entire sensing chain, was augmented by technology. 00:28:32.960 --> 00:28:35.840 align:middle line:90% And what were the consequences? 00:28:35.840 --> 00:28:38.435 align:middle line:84% You could see farther, and you could see smaller. 00:28:38.435 --> 00:28:41.150 align:middle line:90% 00:28:41.150 --> 00:28:45.720 align:middle line:84% Industrial period was basically the invention of photography. 00:28:45.720 --> 00:28:50.450 align:middle line:84% So in addition to the eye, the transduction element 00:28:50.450 --> 00:28:55.040 align:middle line:84% and the storage and display element was now technological. 00:28:55.040 --> 00:28:56.960 align:middle line:90% What are the consequences? 00:28:56.960 --> 00:29:00.080 align:middle line:84% We could expand into invisible spectrum. 00:29:00.080 --> 00:29:03.980 align:middle line:84% Now, one has to remember, without photographic film, 00:29:03.980 --> 00:29:08.060 align:middle line:84% x-rays would never have been invented because [INAUDIBLE] 00:29:08.060 --> 00:29:10.520 align:middle line:84% figured it out that there was something going on because he 00:29:10.520 --> 00:29:14.390 align:middle line:84% had this film that was [INAUDIBLE],, and it still-- 00:29:14.390 --> 00:29:15.410 align:middle line:90% something happened. 00:29:15.410 --> 00:29:16.070 align:middle line:90% It exposed. 00:29:16.070 --> 00:29:19.820 align:middle line:84% So invisible spectrum, then second thing 00:29:19.820 --> 00:29:23.750 align:middle line:84% is time sequence recording allowing 00:29:23.750 --> 00:29:26.343 align:middle line:84% the analysis of very fast or very slow events. 00:29:26.343 --> 00:29:28.010 align:middle line:84% Time lapse photography and other things, 00:29:28.010 --> 00:29:30.770 align:middle line:84% so detailed study of motion that went well 00:29:30.770 --> 00:29:33.960 align:middle line:90% beyond the human time phase. 00:29:33.960 --> 00:29:37.340 align:middle line:84% However, everything was still non-real time. 00:29:37.340 --> 00:29:40.340 align:middle line:84% You still required chemical processing, 00:29:40.340 --> 00:29:44.720 align:middle line:84% and processing and exploitation was still by humans. 00:29:44.720 --> 00:29:50.210 align:middle line:84% So as we move forward to the modern era, 20th Century, 00:29:50.210 --> 00:29:55.640 align:middle line:84% it really is a revolution in imaging sensors, where 00:29:55.640 --> 00:30:01.490 align:middle line:84% the processing and exploitation is still primarily by humans, 00:30:01.490 --> 00:30:05.370 align:middle line:84% but that slowly is moving into a completely automated image 00:30:05.370 --> 00:30:06.350 align:middle line:90% exploitation system. 00:30:06.350 --> 00:30:09.060 align:middle line:84% And I just thought this is an interesting example. 00:30:09.060 --> 00:30:12.080 align:middle line:84% I don't think the Pee Wee soccer league still 00:30:12.080 --> 00:30:14.900 align:middle line:84% has a goal line camera or something to decide 00:30:14.900 --> 00:30:17.520 align:middle line:84% whether somebody was offside or whatever, 00:30:17.520 --> 00:30:20.990 align:middle line:84% but it's really a matter of time. 00:30:20.990 --> 00:30:24.590 align:middle line:84% And why is it a dramatic break from the past? 00:30:24.590 --> 00:30:28.970 align:middle line:84% Real time acquisition, no chemical processing, 00:30:28.970 --> 00:30:30.530 align:middle line:84% real time processing, and [INAUDIBLE] 00:30:30.530 --> 00:30:33.320 align:middle line:84% scalable manufacturing, at least as far as sensors 00:30:33.320 --> 00:30:33.930 align:middle line:90% are concerned. 00:30:33.930 --> 00:30:37.430 align:middle line:84% And now, with the wafer level camera and new designs 00:30:37.430 --> 00:30:41.120 align:middle line:84% for front end optical elements, that scalable manufacturing 00:30:41.120 --> 00:30:45.110 align:middle line:90% is also moving towards optics. 00:30:45.110 --> 00:30:48.378 align:middle line:90% And let me see, this-- 00:30:48.378 --> 00:30:48.920 align:middle line:90% I don't know. 00:30:48.920 --> 00:30:51.530 align:middle line:84% Did you show this [INAUDIBLE] in the class? 00:30:51.530 --> 00:30:54.330 align:middle line:90% RAMESH RASKAR: No, I haven't. 00:30:54.330 --> 00:30:58.160 align:middle line:84% RAVI ATHALE: As I was working on some of this presentation, 00:30:58.160 --> 00:31:02.180 align:middle line:84% from this very interesting article from Nathan Myhrvold-- 00:31:02.180 --> 00:31:05.030 align:middle line:84% people know about Nathan Myhrvold, and his-- 00:31:05.030 --> 00:31:08.390 align:middle line:84% I think he was employee number 15 or something at Microsoft. 00:31:08.390 --> 00:31:10.910 align:middle line:84% And from what I understand, his primary 00:31:10.910 --> 00:31:13.430 align:middle line:84% contribution to the thing at that time, 00:31:13.430 --> 00:31:17.120 align:middle line:84% to have a standalone company that is solely 00:31:17.120 --> 00:31:20.180 align:middle line:84% for the purpose of developing systems software 00:31:20.180 --> 00:31:21.540 align:middle line:90% and making money on that. 00:31:21.540 --> 00:31:25.460 align:middle line:84% So his contribution was the business model for Microsoft, 00:31:25.460 --> 00:31:31.550 align:middle line:84% and he's currently either the most famous or most reviled 00:31:31.550 --> 00:31:34.430 align:middle line:84% person in Silicon Valley for having 00:31:34.430 --> 00:31:36.830 align:middle line:84% started a company, Intellectual Ventures. 00:31:36.830 --> 00:31:38.810 align:middle line:84% And again, he's trying to do a business model, 00:31:38.810 --> 00:31:43.130 align:middle line:84% that a company can exist solely for the purpose of inventing 00:31:43.130 --> 00:31:45.590 align:middle line:84% things, never commercializing, never 00:31:45.590 --> 00:31:49.280 align:middle line:84% prioritizing, but essentially filing the patents 00:31:49.280 --> 00:31:53.240 align:middle line:84% and sitting on them, and either licensing it or suing people 00:31:53.240 --> 00:31:55.640 align:middle line:84% who they think violated the patent. 00:31:55.640 --> 00:31:58.320 align:middle line:84% It's a very interesting business model. 00:31:58.320 --> 00:32:00.560 align:middle line:84% So obviously, he knows something about these things. 00:32:00.560 --> 00:32:05.270 align:middle line:84% He made this comment about, cameras will also change form. 00:32:05.270 --> 00:32:08.030 align:middle line:84% They are film cameras without the film, 00:32:08.030 --> 00:32:09.980 align:middle line:84% basically like the early automobiles 00:32:09.980 --> 00:32:12.110 align:middle line:90% were a horseless carriage. 00:32:12.110 --> 00:32:15.980 align:middle line:84% So his point is that camera stores of the future 00:32:15.980 --> 00:32:18.450 align:middle line:90% will surprise us just as much. 00:32:18.450 --> 00:32:22.410 align:middle line:84% And I was making this comment, somebody said, 00:32:22.410 --> 00:32:24.770 align:middle line:84% oh what the hell is he talking about? 00:32:24.770 --> 00:32:27.635 align:middle line:84% The automobiles are still the same as in 1910. 00:32:27.635 --> 00:32:29.280 align:middle line:84% 100 years later, nothing has changed. 00:32:29.280 --> 00:32:31.520 align:middle line:84% It still is internal combustion engine 00:32:31.520 --> 00:32:33.080 align:middle line:90% and transmission and wheels. 00:32:33.080 --> 00:32:34.250 align:middle line:90% Well, yes and no. 00:32:34.250 --> 00:32:38.090 align:middle line:84% So now, we thought that we'll take this analogy a little bit 00:32:38.090 --> 00:32:45.050 align:middle line:84% forward and say, so horse drawn carriage, horseless carriage, 00:32:45.050 --> 00:32:46.430 align:middle line:90% to the family station wagon. 00:32:46.430 --> 00:32:49.700 align:middle line:84% Up to here, you can say really, it's more sophisticated, 00:32:49.700 --> 00:32:51.120 align:middle line:90% but nothing has changed. 00:32:51.120 --> 00:32:54.330 align:middle line:84% But then you say, what else has happened? 00:32:54.330 --> 00:33:00.830 align:middle line:84% And automotives, tractors, [INAUDIBLE],, a tank, 00:33:00.830 --> 00:33:04.550 align:middle line:84% to that [INAUDIBLE] that's allowed to split the lid. 00:33:04.550 --> 00:33:09.050 align:middle line:84% And this, of course, you recognize that as Stanley, 00:33:09.050 --> 00:33:11.120 align:middle line:90% autonomous vehicle. 00:33:11.120 --> 00:33:15.170 align:middle line:90% So if we say that film cameras-- 00:33:15.170 --> 00:33:18.260 align:middle line:84% filmless cameras-- I really love this picture because both 00:33:18.260 --> 00:33:19.400 align:middle line:90% of them are cameras. 00:33:19.400 --> 00:33:20.160 align:middle line:90% Identical. 00:33:20.160 --> 00:33:21.860 align:middle line:84% This-- these are all film backplane. 00:33:21.860 --> 00:33:23.480 align:middle line:90% That's a CCD backplane. 00:33:23.480 --> 00:33:25.580 align:middle line:84% So where is it going in the future? 00:33:25.580 --> 00:33:27.260 align:middle line:90% What are the future directions? 00:33:27.260 --> 00:33:31.640 align:middle line:84% And at least our projection is where 00:33:31.640 --> 00:33:35.510 align:middle line:84% it's going in the future is specialization and [INAUDIBLE],, 00:33:35.510 --> 00:33:37.980 align:middle line:84% that the same concept, of course, 00:33:37.980 --> 00:33:40.700 align:middle line:84% there is some energy source, and there 00:33:40.700 --> 00:33:44.660 align:middle line:84% is some mechanism to convert that energy 00:33:44.660 --> 00:33:46.650 align:middle line:90% into mechanical motion. 00:33:46.650 --> 00:33:48.990 align:middle line:84% So in that sense nothing has changed. 00:33:48.990 --> 00:33:53.300 align:middle line:84% But if you look at this and this and this and this, 00:33:53.300 --> 00:33:55.490 align:middle line:84% each one is designed for a different purpose. 00:33:55.490 --> 00:33:59.490 align:middle line:84% Each one is specialized for that particular purpose. 00:33:59.490 --> 00:34:02.090 align:middle line:84% So similarly, cameras-- right now, 00:34:02.090 --> 00:34:04.710 align:middle line:84% a camera is a camera that takes picture, 00:34:04.710 --> 00:34:07.790 align:middle line:84% whether it's the Hubble, or whether it is your cell phone 00:34:07.790 --> 00:34:11.210 align:middle line:84% camera or anything, or the pill camera. 00:34:11.210 --> 00:34:16.250 align:middle line:84% The structure and function, so far, hasn't changed at all. 00:34:16.250 --> 00:34:22.790 align:middle line:84% And that is where we feel a dramatic break coming 00:34:22.790 --> 00:34:26.120 align:middle line:84% in the future because of the flexibility of the technology, 00:34:26.120 --> 00:34:28.610 align:middle line:84% because of very specialized applications, 00:34:28.610 --> 00:34:31.620 align:middle line:84% very specialized constraints for things. 00:34:31.620 --> 00:34:35.310 align:middle line:90% So why if shouldn't there be-- 00:34:35.310 --> 00:34:39.659 align:middle line:84% you can say, just like there are ASICs, Application Specific 00:34:39.659 --> 00:34:41.699 align:middle line:84% Integrated Circuits, why shouldn't there 00:34:41.699 --> 00:34:47.070 align:middle line:84% be Applications Specific Imaging Systems, ASIS? 00:34:47.070 --> 00:34:54.719 align:middle line:84% So [INAUDIBLE] autonomy that an imaging sensor that decides, 00:34:54.719 --> 00:34:58.170 align:middle line:84% on its own, without any manual intervention, 00:34:58.170 --> 00:35:03.620 align:middle line:84% where to focus on, how to interrogate the scene? 00:35:03.620 --> 00:35:09.320 align:middle line:84% And this is sort of reworking the biological inspiration. 00:35:09.320 --> 00:35:12.380 align:middle line:84% The cameras are really replacing the human eye 00:35:12.380 --> 00:35:16.740 align:middle line:84% with trying to form a similar kind of representation, 00:35:16.740 --> 00:35:18.950 align:middle line:84% whether it should go to the biological world, 00:35:18.950 --> 00:35:25.940 align:middle line:84% it's a vast array of different types of imaging sensors. 00:35:25.940 --> 00:35:28.820 align:middle line:84% It's probably not even proper to call them imaging sensors. 00:35:28.820 --> 00:35:30.890 align:middle line:84% They are more like spatial sensors. 00:35:30.890 --> 00:35:33.260 align:middle line:84% The most primitive form, an earthworm 00:35:33.260 --> 00:35:35.750 align:middle line:84% with a light sensitive skin that simply 00:35:35.750 --> 00:35:37.370 align:middle line:84% is able to distinguish there is more 00:35:37.370 --> 00:35:39.680 align:middle line:84% light on this side versus that side 00:35:39.680 --> 00:35:42.260 align:middle line:84% and can steer itself accordingly. 00:35:42.260 --> 00:35:47.960 align:middle line:84% And then a variety of techniques for the insects 00:35:47.960 --> 00:35:53.360 align:middle line:84% and other creatures, and these, their spectral response, 00:35:53.360 --> 00:35:56.990 align:middle line:84% the processing, the functionality just exquisitely 00:35:56.990 --> 00:36:00.895 align:middle line:84% evolved for the specific evolutionary needs 00:36:00.895 --> 00:36:02.270 align:middle line:84% of the animal for their survival. 00:36:02.270 --> 00:36:06.120 align:middle line:90% 00:36:06.120 --> 00:36:09.350 align:middle line:90% And I don't know if I really-- 00:36:09.350 --> 00:36:14.396 align:middle line:84% I think I would like to sort of-- 00:36:14.396 --> 00:36:18.500 align:middle line:90% the optics is now getting there. 00:36:18.500 --> 00:36:21.560 align:middle line:84% [INAUDIBLE] plastic lenses, diamond [INAUDIBLE],, 00:36:21.560 --> 00:36:24.020 align:middle line:84% microfabrication, wafer scale assembly. 00:36:24.020 --> 00:36:26.300 align:middle line:84% So the grind and polish technique 00:36:26.300 --> 00:36:28.520 align:middle line:84% for making optical elements really 00:36:28.520 --> 00:36:34.910 align:middle line:84% is, now, relegated to very high quality specialized. 00:36:34.910 --> 00:36:39.620 align:middle line:84% And processing embedded in the cameras, that's the first step 00:36:39.620 --> 00:36:41.630 align:middle line:84% right now-- red eye removal, face detection, 00:36:41.630 --> 00:36:43.490 align:middle line:90% smile detection. 00:36:43.490 --> 00:36:46.490 align:middle line:84% And this is, of course, illumination. 00:36:46.490 --> 00:36:50.460 align:middle line:84% Right now, we are still using this dumb flash that this goes. 00:36:50.460 --> 00:36:53.510 align:middle line:84% So spatial temporal coding of illumination 00:36:53.510 --> 00:36:57.860 align:middle line:84% in order to enhance your ability to extract information 00:36:57.860 --> 00:36:59.360 align:middle line:90% from the scene. 00:36:59.360 --> 00:37:04.220 align:middle line:84% Different spectral regimes for exploiting that information. 00:37:04.220 --> 00:37:07.760 align:middle line:84% And this is some picture gallery of new optics. 00:37:07.760 --> 00:37:12.260 align:middle line:84% That's the Kodak microlens array, folded optics, 00:37:12.260 --> 00:37:16.310 align:middle line:84% diamond turn optics, and free form multi scale conformal 00:37:16.310 --> 00:37:17.563 align:middle line:90% optics. 00:37:17.563 --> 00:37:19.730 align:middle line:84% RAMESH RASKAR: What do you use it for, the last one? 00:37:19.730 --> 00:37:20.300 align:middle line:90% RAVI ATHALE: I'm sorry? 00:37:20.300 --> 00:37:22.460 align:middle line:84% RAMESH RASKAR: What is the use of the last one? 00:37:22.460 --> 00:37:26.408 align:middle line:84% RAVI ATHALE: I don't if this is any use. 00:37:26.408 --> 00:37:28.028 align:middle line:90% [STUDENTS LAUGHING] 00:37:28.028 --> 00:37:28.820 align:middle line:90% STUDENT: It's cool. 00:37:28.820 --> 00:37:29.653 align:middle line:90% STUDENT: It is cool. 00:37:29.653 --> 00:37:33.260 align:middle line:84% RAVI ATHALE: Yeah, this is just a technology that's there. 00:37:33.260 --> 00:37:39.120 align:middle line:84% You saw, in Charlotte, they had this machine there. 00:37:39.120 --> 00:37:40.010 align:middle line:90% They can do this. 00:37:40.010 --> 00:37:42.500 align:middle line:84% RAMESH RASKAR: They can etch away optics using it. 00:37:42.500 --> 00:37:44.930 align:middle line:84% RAVI ATHALE: I think it's a six degrees of freedom 00:37:44.930 --> 00:37:47.210 align:middle line:84% machine for turning, and then you 00:37:47.210 --> 00:37:50.960 align:middle line:84% can do that on a conformant surface. 00:37:50.960 --> 00:37:57.020 align:middle line:90% And these, I think-- 00:37:57.020 --> 00:38:01.580 align:middle line:84% RAMESH RASKAR: Just going back to your previous comment, 00:38:01.580 --> 00:38:05.810 align:middle line:84% so everything is moving in a direction 00:38:05.810 --> 00:38:09.530 align:middle line:84% where the things are more programmable, more specialized, 00:38:09.530 --> 00:38:11.260 align:middle line:84% but in the basic laws of physics, 00:38:11.260 --> 00:38:12.260 align:middle line:90% are not going to change. 00:38:12.260 --> 00:38:15.680 align:middle line:84% You're still going to convert photons and electrons and so 00:38:15.680 --> 00:38:16.430 align:middle line:90% on. 00:38:16.430 --> 00:38:20.060 align:middle line:90% And so are there some-- 00:38:20.060 --> 00:38:22.580 align:middle line:84% if you want to build an aircraft, 00:38:22.580 --> 00:38:24.710 align:middle line:84% you're going to use Bernoulli principle. 00:38:24.710 --> 00:38:27.440 align:middle line:84% If you're going to build a vehicle, 00:38:27.440 --> 00:38:29.240 align:middle line:90% it's Newton's third law. 00:38:29.240 --> 00:38:31.190 align:middle line:84% It's just going to use those laws of physics. 00:38:31.190 --> 00:38:31.940 align:middle line:90% RAVI ATHALE: Yeah. 00:38:31.940 --> 00:38:33.815 align:middle line:84% RAMESH RASKAR: Is there something fundamental 00:38:33.815 --> 00:38:37.065 align:middle line:84% about imaging that we should be challenging? 00:38:37.065 --> 00:38:39.695 align:middle line:90% 00:38:39.695 --> 00:38:41.945 align:middle line:84% Oh, should we continue it as the same laws of physics? 00:38:41.945 --> 00:38:46.190 align:middle line:90% 00:38:46.190 --> 00:38:47.910 align:middle line:84% RAVI ATHALE: And what laws of physics? 00:38:47.910 --> 00:38:50.800 align:middle line:84% I think it may not be like challenging laws of physics, 00:38:50.800 --> 00:38:53.910 align:middle line:84% but what are the laws of physics that we can bend, 00:38:53.910 --> 00:38:56.100 align:middle line:90% or that are various qualifiers? 00:38:56.100 --> 00:38:58.950 align:middle line:84% So one of the things here, resolution 00:38:58.950 --> 00:39:01.470 align:middle line:84% is limited by the wavelength of light. 00:39:01.470 --> 00:39:05.580 align:middle line:84% Yes, as long as you are in the far field regime, 00:39:05.580 --> 00:39:07.980 align:middle line:84% as long as you are in the linear regime, 00:39:07.980 --> 00:39:11.370 align:middle line:84% you highlight one of these, and you can do whatever you want. 00:39:11.370 --> 00:39:14.100 align:middle line:84% If you are proximity near-field scanning optical microscope, 00:39:14.100 --> 00:39:15.480 align:middle line:84% you can get as small as you want. 00:39:15.480 --> 00:39:20.670 align:middle line:84% But if you have nonlinear, these fluorescence microscopes, 00:39:20.670 --> 00:39:23.820 align:middle line:84% you can get 50 nanometer resolution with visible light. 00:39:23.820 --> 00:39:30.150 align:middle line:84% So that is some ways in which you can really carefully 00:39:30.150 --> 00:39:35.070 align:middle line:84% look at what we consider to be fundamental limit imposed 00:39:35.070 --> 00:39:37.830 align:middle line:84% by physics and say, what is the fine print, 00:39:37.830 --> 00:39:38.850 align:middle line:90% and can we violate that? 00:39:38.850 --> 00:39:40.765 align:middle line:84% So that would be very interesting to look at. 00:39:40.765 --> 00:39:42.390 align:middle line:84% RAMESH RASKAR: Yeah, but I mean, can we 00:39:42.390 --> 00:39:45.420 align:middle line:84% start using our own bodies as optics? 00:39:45.420 --> 00:39:48.810 align:middle line:84% Or can we start using biological principles 00:39:48.810 --> 00:39:56.080 align:middle line:84% to solve the five w and one h problems? 00:39:56.080 --> 00:39:59.370 align:middle line:84% I mean, in the film world, we actually 00:39:59.370 --> 00:40:01.690 align:middle line:84% use the chemistry to solve a lot of the problems. 00:40:01.690 --> 00:40:03.690 align:middle line:84% Now, somehow we have forgotten chemistry, 00:40:03.690 --> 00:40:07.200 align:middle line:84% and we are more back to the physics now. 00:40:07.200 --> 00:40:08.970 align:middle line:90% RAVI ATHALE: Yeah, I think-- 00:40:08.970 --> 00:40:10.095 align:middle line:90% I don't know. 00:40:10.095 --> 00:40:12.060 align:middle line:84% One of the things I wondered, can we 00:40:12.060 --> 00:40:16.980 align:middle line:84% have a three dimensional detector, a volume detector? 00:40:16.980 --> 00:40:21.270 align:middle line:84% And if we can store it, what will it record? 00:40:21.270 --> 00:40:22.470 align:middle line:90% How will we read it out? 00:40:22.470 --> 00:40:25.440 align:middle line:90% How will we process it? 00:40:25.440 --> 00:40:29.010 align:middle line:84% Some people have talked about directly recording 00:40:29.010 --> 00:40:32.160 align:middle line:84% the coherence function instead of just measuring 00:40:32.160 --> 00:40:32.970 align:middle line:90% the intensity. 00:40:32.970 --> 00:40:36.120 align:middle line:90% 00:40:36.120 --> 00:40:40.200 align:middle line:90% And you can imagine that now-- 00:40:40.200 --> 00:40:42.570 align:middle line:84% of course, you can wave the wand and say nano, 00:40:42.570 --> 00:40:45.280 align:middle line:84% and everything goes away or everything is different. 00:40:45.280 --> 00:40:47.490 align:middle line:84% But to some extent, that may be true, 00:40:47.490 --> 00:40:51.900 align:middle line:84% that if you are able to control the material 00:40:51.900 --> 00:40:54.660 align:middle line:84% at deep, deep, subwavelength level, 00:40:54.660 --> 00:40:58.082 align:middle line:84% you can do some pretty ridiculous things. 00:40:58.082 --> 00:41:03.510 align:middle line:84% So that's the direction that we haven't really explored. 00:41:03.510 --> 00:41:08.880 align:middle line:84% And another interesting thing is that, if you have the ability 00:41:08.880 --> 00:41:14.130 align:middle line:84% to manipulate matter and control it at deep subwavelength level, 00:41:14.130 --> 00:41:16.080 align:middle line:84% then it's really incumbent on you 00:41:16.080 --> 00:41:20.760 align:middle line:84% to abandon your other cherished notions about, oh, 00:41:20.760 --> 00:41:24.570 align:middle line:84% a spherical lens forming a well focused image on a focal point. 00:41:24.570 --> 00:41:27.370 align:middle line:84% Why are you sticking to that part of it? 00:41:27.370 --> 00:41:31.440 align:middle line:84% So if you really fully want to exploit the ability 00:41:31.440 --> 00:41:33.720 align:middle line:84% to manipulate the electromagnetic wavefront 00:41:33.720 --> 00:41:38.070 align:middle line:84% in very novel ways using very novel materials, 00:41:38.070 --> 00:41:40.290 align:middle line:84% you should also start thinking about what are we 00:41:40.290 --> 00:41:44.010 align:middle line:84% going to measure, and how are we going to measure it? 00:41:44.010 --> 00:41:47.515 align:middle line:84% So I don't know if I answered your question. 00:41:47.515 --> 00:41:49.640 align:middle line:84% RAMESH RASKAR: Yeah, I mean, it's an open question. 00:41:49.640 --> 00:41:50.623 align:middle line:90% [INTERPOSING VOICES] 00:41:50.623 --> 00:41:52.040 align:middle line:84% RAMESH RASKAR: Yeah, I don't think 00:41:52.040 --> 00:41:53.670 align:middle line:90% there's a clear direction. 00:41:53.670 --> 00:41:55.640 align:middle line:84% But I mean, the chart that you showed 00:41:55.640 --> 00:42:00.080 align:middle line:84% earlier of different stages where biological-- 00:42:00.080 --> 00:42:02.570 align:middle line:84% I mean, not biological, but human involved, 00:42:02.570 --> 00:42:04.580 align:middle line:84% and then we got more and more technological. 00:42:04.580 --> 00:42:06.890 align:middle line:84% Now, everything that was blue has become pink. 00:42:06.890 --> 00:42:07.640 align:middle line:90% RAVI ATHALE: Yeah. 00:42:07.640 --> 00:42:10.250 align:middle line:90% RAMESH RASKAR: So what's next? 00:42:10.250 --> 00:42:12.740 align:middle line:84% RAVI ATHALE: I think maybe just what kind of technologies? 00:42:12.740 --> 00:42:15.830 align:middle line:84% I mean, you can think of some of the things like, 00:42:15.830 --> 00:42:18.830 align:middle line:84% can we think of self-assembled imaging sensors 00:42:18.830 --> 00:42:20.750 align:middle line:84% that are not lithographically driven 00:42:20.750 --> 00:42:23.880 align:middle line:84% or that are not mechanically assembled? 00:42:23.880 --> 00:42:24.380 align:middle line:90% Yeah. 00:42:24.380 --> 00:42:26.930 align:middle line:90% 00:42:26.930 --> 00:42:28.460 align:middle line:90% I don't know what else. 00:42:28.460 --> 00:42:31.730 align:middle line:84% RAMESH RASKAR: It's like the discussion on similarity. 00:42:31.730 --> 00:42:34.160 align:middle line:84% It's almost easier to say it's coming tomorrow, 00:42:34.160 --> 00:42:36.410 align:middle line:84% but when you go back, it seems like a lot of stuff 00:42:36.410 --> 00:42:43.220 align:middle line:84% has to happen before we reach that point of-- 00:42:43.220 --> 00:42:44.085 align:middle line:90% [INTERPOSING VOICES] 00:42:44.085 --> 00:42:46.460 align:middle line:84% RAVI ATHALE: And many of the turning points in technology 00:42:46.460 --> 00:42:48.200 align:middle line:84% are turning points only in retrospect. 00:42:48.200 --> 00:42:49.010 align:middle line:90% RAMESH RASKAR: Yes. 00:42:49.010 --> 00:42:52.250 align:middle line:84% RAVI ATHALE: So you say, ha, that was the day world changed. 00:42:52.250 --> 00:42:54.080 align:middle line:84% You didn't realize that time, but-- 00:42:54.080 --> 00:42:57.200 align:middle line:84% I mean, the 40 year anniversary of the internet, 00:42:57.200 --> 00:43:00.320 align:middle line:84% and there was a story about what was the first message sent? 00:43:00.320 --> 00:43:02.330 align:middle line:90% First message was "log in." 00:43:02.330 --> 00:43:04.640 align:middle line:84% And after L-O-G, the computers crashed, 00:43:04.640 --> 00:43:06.140 align:middle line:90% and they had to reboot. 00:43:06.140 --> 00:43:07.765 align:middle line:84% And so they asked the person from UCLA, 00:43:07.765 --> 00:43:09.598 align:middle line:84% so what did you do after you send a message? 00:43:09.598 --> 00:43:11.120 align:middle line:84% Oh, I went, had a burger, and went 00:43:11.120 --> 00:43:15.138 align:middle line:84% to sleep because they didn't know that revolutionized 00:43:15.138 --> 00:43:15.680 align:middle line:90% or something. 00:43:15.680 --> 00:43:17.060 align:middle line:90% It's only in retrospect. 00:43:17.060 --> 00:43:20.000 align:middle line:90% 00:43:20.000 --> 00:43:24.950 align:middle line:84% And these are some interesting examples just 00:43:24.950 --> 00:43:27.630 align:middle line:84% to say application specific imaging, 00:43:27.630 --> 00:43:33.560 align:middle line:84% and this is one example from a surgeon in Naval Medical 00:43:33.560 --> 00:43:34.610 align:middle line:90% Center. 00:43:34.610 --> 00:43:38.570 align:middle line:84% They're using the three CCD camera on a laparoscopic tower, 00:43:38.570 --> 00:43:41.150 align:middle line:84% and basically, his point is that, when 00:43:41.150 --> 00:43:43.310 align:middle line:84% you are doing surgery, it's really important 00:43:43.310 --> 00:43:46.610 align:middle line:84% to know which is the artery and which is the vein. 00:43:46.610 --> 00:43:48.380 align:middle line:90% You treat it very differently. 00:43:48.380 --> 00:43:50.450 align:middle line:90% And you start off, say-- 00:43:50.450 --> 00:43:53.090 align:middle line:90% I mean, hey, color coding wires. 00:43:53.090 --> 00:43:58.380 align:middle line:84% Unfortunately, in the body, it's not that straightforward. 00:43:58.380 --> 00:44:05.720 align:middle line:84% So taking these three colors and then doing the processing, 00:44:05.720 --> 00:44:09.590 align:middle line:84% and using many of the things that you saw here, 00:44:09.590 --> 00:44:11.130 align:middle line:90% that you can really-- 00:44:11.130 --> 00:44:14.810 align:middle line:84% a very distinct spectrum for the hemoglobin and oxygenated 00:44:14.810 --> 00:44:15.750 align:middle line:90% hemoglobin. 00:44:15.750 --> 00:44:18.800 align:middle line:84% And so by doing the processing, you create that map, 00:44:18.800 --> 00:44:22.340 align:middle line:90% and you project it back. 00:44:22.340 --> 00:44:23.600 align:middle line:90% You show it. 00:44:23.600 --> 00:44:29.540 align:middle line:84% Veins are colored blue, and the arteries are colored red. 00:44:29.540 --> 00:44:32.160 align:middle line:84% Now, here in this particular case, 00:44:32.160 --> 00:44:34.700 align:middle line:84% the surgeon looks at the monitor and says, ah, 00:44:34.700 --> 00:44:37.740 align:middle line:84% this is the one that I should be careful about, or whatever. 00:44:37.740 --> 00:44:40.370 align:middle line:90% But then some of the things-- 00:44:40.370 --> 00:44:41.930 align:middle line:90% I don't know. 00:44:41.930 --> 00:44:45.500 align:middle line:84% How many of you have seen this thing called Vein Viewer? 00:44:45.500 --> 00:44:47.210 align:middle line:90% Have you seen the product? 00:44:47.210 --> 00:44:54.180 align:middle line:84% I don't if there is a wireless connection is good enough. 00:44:54.180 --> 00:44:54.680 align:middle line:90% We'll see. 00:44:54.680 --> 00:44:56.300 align:middle line:90% It might be working. 00:44:56.300 --> 00:44:56.890 align:middle line:90% Yeah. 00:44:56.890 --> 00:44:58.417 align:middle line:90% [INTERPOSING VOICES] 00:44:58.417 --> 00:44:59.750 align:middle line:90% RAVI ATHALE: You have seen that? 00:44:59.750 --> 00:45:08.140 align:middle line:90% 00:45:08.140 --> 00:45:10.510 align:middle line:84% And if you look at the technology, 00:45:10.510 --> 00:45:14.220 align:middle line:84% it is just absolutely stupid technically. 00:45:14.220 --> 00:45:18.880 align:middle line:84% It's 780 nanometer light illumination that you do that, 00:45:18.880 --> 00:45:23.260 align:middle line:84% and that penetrates the skin and is absorbed by your blood. 00:45:23.260 --> 00:45:25.870 align:middle line:84% And so you can see the veins deep inside, 00:45:25.870 --> 00:45:28.970 align:middle line:84% and the beauty about this is a camera is up there, 00:45:28.970 --> 00:45:30.610 align:middle line:90% and so is a DLP projector. 00:45:30.610 --> 00:45:34.540 align:middle line:84% And the processor basically scales and aligns the images 00:45:34.540 --> 00:45:39.370 align:middle line:84% so that it is projected back exactly aligned like that. 00:45:39.370 --> 00:45:41.412 align:middle line:90% But the use is phenomenal. 00:45:41.412 --> 00:45:43.120 align:middle line:84% RAMESH RASKAR: Yeah, it's one of the best 00:45:43.120 --> 00:45:44.315 align:middle line:90% forms of augmented reality. 00:45:44.315 --> 00:45:45.190 align:middle line:90% RAVI ATHALE: Exactly. 00:45:45.190 --> 00:45:48.040 align:middle line:84% RAMESH RASKAR: So I always show it in my augmented reality 00:45:48.040 --> 00:45:49.090 align:middle line:90% presentations. 00:45:49.090 --> 00:45:51.720 align:middle line:90% 00:45:51.720 --> 00:45:54.560 align:middle line:84% RAVI ATHALE: Yeah, and I think the rest of the thing may be-- 00:45:54.560 --> 00:45:59.690 align:middle line:90% 00:45:59.690 --> 00:46:01.340 align:middle line:90% I was talking to Ramesh. 00:46:01.340 --> 00:46:07.190 align:middle line:84% This is Doug Hart's company, Brontes Technology for the 3D. 00:46:07.190 --> 00:46:07.967 align:middle line:90% I think you saw-- 00:46:07.967 --> 00:46:09.050 align:middle line:90% RAMESH RASKAR: We saw it-- 00:46:09.050 --> 00:46:12.620 align:middle line:84% RAVI ATHALE: You saw that, and [INAUDIBLE] 00:46:12.620 --> 00:46:14.420 align:middle line:90% chair side oral scanners. 00:46:14.420 --> 00:46:16.190 align:middle line:90% I think there was a video. 00:46:16.190 --> 00:46:19.028 align:middle line:84% Did you see that video of the doctor manipulating in the-- 00:46:19.028 --> 00:46:20.570 align:middle line:84% RAMESH RASKAR: Did it show the video? 00:46:20.570 --> 00:46:21.320 align:middle line:90% I don't remember. 00:46:21.320 --> 00:46:23.630 align:middle line:90% [INAUDIBLE]? 00:46:23.630 --> 00:46:24.180 align:middle line:90% I forget. 00:46:24.180 --> 00:46:25.263 align:middle line:90% RAVI ATHALE: I don't know. 00:46:25.263 --> 00:46:26.270 align:middle line:90% It's not that important. 00:46:26.270 --> 00:46:27.355 align:middle line:90% You know what it is-- 00:46:27.355 --> 00:46:29.480 align:middle line:84% RAMESH RASKAR: Yeah, it's amazing how simple it is. 00:46:29.480 --> 00:46:31.460 align:middle line:84% RAVI ATHALE: Yeah, and I think, this 00:46:31.460 --> 00:46:34.100 align:middle line:84% is one of the things that maybe-- 00:46:34.100 --> 00:46:35.810 align:middle line:84% and the refocus imaging, of course. 00:46:35.810 --> 00:46:38.940 align:middle line:84% You don't need to, right now, know anything more about it. 00:46:38.940 --> 00:46:43.220 align:middle line:84% But in some ways, when you talk about application 00:46:43.220 --> 00:46:48.110 align:middle line:84% specific cameras, the key is understanding 00:46:48.110 --> 00:46:51.575 align:middle line:84% the nature of the problem and finding the simplest solution 00:46:51.575 --> 00:46:53.970 align:middle line:90% to that specific problem. 00:46:53.970 --> 00:46:57.740 align:middle line:84% And one of the examples, you saw that Vein Viewer, all of you 00:46:57.740 --> 00:47:01.040 align:middle line:84% know the pulse oximeter that is clipped to your finger. 00:47:01.040 --> 00:47:02.120 align:middle line:90% It's relatively new. 00:47:02.120 --> 00:47:05.948 align:middle line:84% I think maybe 20, 25 years old technology. 00:47:05.948 --> 00:47:07.490 align:middle line:84% Again, if you look at the technology, 00:47:07.490 --> 00:47:10.280 align:middle line:84% it's absolutely dirt simple technology. 00:47:10.280 --> 00:47:12.980 align:middle line:84% Two different ladies looking at the differentiable absorption 00:47:12.980 --> 00:47:15.410 align:middle line:84% through your finger to measure the oxygenation 00:47:15.410 --> 00:47:16.880 align:middle line:90% level of the blood. 00:47:16.880 --> 00:47:19.890 align:middle line:84% And before, they had to draw the blood and send it for analysis, 00:47:19.890 --> 00:47:22.520 align:middle line:84% and by the time it came back, maybe bad things happened. 00:47:22.520 --> 00:47:27.050 align:middle line:84% But this one is now, you saw in all the TV shows anywhere, 00:47:27.050 --> 00:47:29.270 align:middle line:84% the first thing they do is clip this on your finger. 00:47:29.270 --> 00:47:31.658 align:middle line:84% And I don't know how much it costs-- 00:47:31.658 --> 00:47:33.200 align:middle line:84% RAMESH RASKAR: It still costs $1,000. 00:47:33.200 --> 00:47:33.908 align:middle line:90% I don't know why. 00:47:33.908 --> 00:47:36.158 align:middle line:84% RAVI ATHALE: Because somebody's making a lot of money. 00:47:36.158 --> 00:47:36.740 align:middle line:90% That's why. 00:47:36.740 --> 00:47:37.452 align:middle line:90% I think-- 00:47:37.452 --> 00:47:38.690 align:middle line:90% STUDENT: They have a patent. 00:47:38.690 --> 00:47:40.400 align:middle line:90% RAVI ATHALE: Yeah. 00:47:40.400 --> 00:47:44.330 align:middle line:84% But that's some of the research projects 00:47:44.330 --> 00:47:46.760 align:middle line:84% that I was involved in monitoring. 00:47:46.760 --> 00:47:50.420 align:middle line:84% To talk about, can you do a spacial map 00:47:50.420 --> 00:47:54.410 align:middle line:84% of oxygenation levels using these multi spectral cameras 00:47:54.410 --> 00:47:55.790 align:middle line:90% and then processing? 00:47:55.790 --> 00:47:57.650 align:middle line:84% And the doctors got really excited 00:47:57.650 --> 00:48:02.390 align:middle line:84% for applications like burn assessment, 00:48:02.390 --> 00:48:04.610 align:middle line:84% whether it's a first or second or third degree burn. 00:48:04.610 --> 00:48:07.340 align:middle line:84% The treatment differs dramatically, 00:48:07.340 --> 00:48:11.630 align:middle line:84% and instead of relying on the doctor's visual assessment 00:48:11.630 --> 00:48:16.100 align:middle line:84% of what degree burn it is, if you can take a camera that 00:48:16.100 --> 00:48:20.330 align:middle line:84% can do that blood oxygenation map, 00:48:20.330 --> 00:48:25.370 align:middle line:84% or if you can map the retina and a lot 00:48:25.370 --> 00:48:28.420 align:middle line:84% of the diabetic retinopathy and other diseases, 00:48:28.420 --> 00:48:33.260 align:middle line:84% there are precursors are in, what is the oxygenation 00:48:33.260 --> 00:48:35.090 align:middle line:90% level of the blood [INAUDIBLE]. 00:48:35.090 --> 00:48:38.120 align:middle line:84% So these kind of things, just like the distinction 00:48:38.120 --> 00:48:42.410 align:middle line:84% between arteries and veins or the Vein Viewer and all that-- 00:48:42.410 --> 00:48:46.100 align:middle line:84% you don't need to invent new nanotechnologies or something. 00:48:46.100 --> 00:48:48.860 align:middle line:84% You need to understand in depth what the problem is 00:48:48.860 --> 00:48:50.550 align:middle line:90% and what the human factors is. 00:48:50.550 --> 00:48:52.760 align:middle line:90% Again, this thing, Vein Viewer-- 00:48:52.760 --> 00:48:57.140 align:middle line:84% the cool thing is not just that IR LED, 00:48:57.140 --> 00:49:00.830 align:middle line:84% but you can imagine everything else the same 00:49:00.830 --> 00:49:03.530 align:middle line:84% except that image being shown on TV, 00:49:03.530 --> 00:49:06.320 align:middle line:84% and it will nowhere be as dramatic 00:49:06.320 --> 00:49:08.900 align:middle line:84% if the doctor has to manipulate the needle here 00:49:08.900 --> 00:49:10.190 align:middle line:90% while looking at the monitor. 00:49:10.190 --> 00:49:12.710 align:middle line:84% It's a human factor, so right there. 00:49:12.710 --> 00:49:17.090 align:middle line:90% So that that's really where-- 00:49:17.090 --> 00:49:18.960 align:middle line:90% so this is the last slide. 00:49:18.960 --> 00:49:24.920 align:middle line:84% So just a little bit science fantasy, not just fiction. 00:49:24.920 --> 00:49:27.050 align:middle line:84% A personal imaging assistant, something 00:49:27.050 --> 00:49:29.690 align:middle line:84% that just like you carry your pen every day, 00:49:29.690 --> 00:49:33.150 align:middle line:84% or your wallet or credit card or your cell phone. 00:49:33.150 --> 00:49:36.260 align:middle line:84% The personal imaging assistant-- what all would it do? 00:49:36.260 --> 00:49:37.820 align:middle line:90% Health care. 00:49:37.820 --> 00:49:39.770 align:middle line:84% Checking for sunburns, or if you get 00:49:39.770 --> 00:49:43.340 align:middle line:84% scraped, hey, is it getting infected, this wound? 00:49:43.340 --> 00:49:44.420 align:middle line:90% You have an attachment? 00:49:44.420 --> 00:49:48.380 align:middle line:84% You check the ear infection of your children or whatever. 00:49:48.380 --> 00:49:49.640 align:middle line:90% Look at your tongue. 00:49:49.640 --> 00:49:52.640 align:middle line:90% Are you getting upset stomach? 00:49:52.640 --> 00:49:56.180 align:middle line:84% Wardrobe matching, of course, color and styles, especially. 00:49:56.180 --> 00:49:59.990 align:middle line:84% What kind of makeup will go with this, and other things. 00:49:59.990 --> 00:50:02.330 align:middle line:84% Hygiene, cleanliness of surroundings, 00:50:02.330 --> 00:50:07.190 align:middle line:84% presence of bacteria, water/food safety, quality of what-- 00:50:07.190 --> 00:50:09.950 align:middle line:84% The last one, it's really getting a little bit science 00:50:09.950 --> 00:50:13.370 align:middle line:84% fictiony, but maybe not given that sixth sense thing. 00:50:13.370 --> 00:50:16.430 align:middle line:84% So you're wearing these glasses, or this personal [INAUDIBLE] 00:50:16.430 --> 00:50:19.130 align:middle line:84% assistant with a headphone, and somebody walks-- 00:50:19.130 --> 00:50:20.540 align:middle line:90% Hey, Ramesh, how are you? 00:50:20.540 --> 00:50:24.470 align:middle line:84% Somebody, this person, [INAUDIBLE] 00:50:24.470 --> 00:50:25.940 align:middle line:84% It's like the politicians who have 00:50:25.940 --> 00:50:30.170 align:middle line:84% these assistants at their elbow who whisper in their ears who 00:50:30.170 --> 00:50:34.250 align:middle line:84% is the person walking in, and discerning move, that's 00:50:34.250 --> 00:50:36.260 align:middle line:90% again very science fictiony. 00:50:36.260 --> 00:50:38.450 align:middle line:90% Can the image tell you that? 00:50:38.450 --> 00:50:39.620 align:middle line:90% This guy is very skeptical. 00:50:39.620 --> 00:50:43.190 align:middle line:84% He's not buying anything [INAUDIBLE] of course, 00:50:43.190 --> 00:50:45.800 align:middle line:84% taking pictures, and how it's basically 00:50:45.800 --> 00:50:49.640 align:middle line:84% exploit the electromagnetic domain to the hilt. 00:50:49.640 --> 00:50:56.170 align:middle line:84% Combine the processing right there and have an adaptive-- 00:50:56.170 --> 00:50:58.370 align:middle line:90% one analogy that people make. 00:50:58.370 --> 00:51:02.480 align:middle line:84% Right now, it's a sensor with a total one directional flow 00:51:02.480 --> 00:51:06.020 align:middle line:84% of information from the sensor to the processor to it. 00:51:06.020 --> 00:51:07.970 align:middle line:90% How about going back? 00:51:07.970 --> 00:51:14.270 align:middle line:84% I heard somewhere about human visual pathways from the retina 00:51:14.270 --> 00:51:16.700 align:middle line:84% to the lateral geniculate nucleus to V1 00:51:16.700 --> 00:51:19.280 align:middle line:84% and all the inferior temporal cortex and all that. 00:51:19.280 --> 00:51:21.860 align:middle line:84% Everybody analyzes that feedforward 00:51:21.860 --> 00:51:24.530 align:middle line:84% and what kind of features are extracted. 00:51:24.530 --> 00:51:26.930 align:middle line:84% And I heard that from V1 to [INAUDIBLE],, 00:51:26.930 --> 00:51:30.080 align:middle line:84% there is a huge amount of feedback connection. 00:51:30.080 --> 00:51:31.880 align:middle line:90% What's the role of that? 00:51:31.880 --> 00:51:36.400 align:middle line:84% I don't think the neuroscientists know yet. 00:51:36.400 --> 00:51:38.630 align:middle line:90% But that kind of a thing-- 00:51:38.630 --> 00:51:41.720 align:middle line:84% an imaging sensor that analyzes the scene, 00:51:41.720 --> 00:51:46.220 align:middle line:84% says now, next, I should look in this direction, 00:51:46.220 --> 00:51:48.680 align:middle line:84% in this spectral domain, this place of something. 00:51:48.680 --> 00:51:52.280 align:middle line:84% And in a few of these adaptive steps, 00:51:52.280 --> 00:51:56.510 align:middle line:84% can learn a hell of a lot about what is out there. 00:51:56.510 --> 00:52:00.230 align:middle line:84% So last thing, unobtrusive or most covert form factor, 00:52:00.230 --> 00:52:02.330 align:middle line:84% and it's literally part of getting dressed. 00:52:02.330 --> 00:52:04.280 align:middle line:84% Just like carry your wallet, your cell 00:52:04.280 --> 00:52:06.530 align:middle line:84% phone, carry your imaging assistant 00:52:06.530 --> 00:52:08.690 align:middle line:84% that can do all of these things for you. 00:52:08.690 --> 00:52:10.190 align:middle line:84% RAMESH RASKAR: But what form factor? 00:52:10.190 --> 00:52:12.170 align:middle line:90% Is it in the clothing or is it-- 00:52:12.170 --> 00:52:15.050 align:middle line:84% RAVI ATHALE: That's interesting in your eyeglasses, 00:52:15.050 --> 00:52:17.330 align:middle line:90% eyewear kind of a product. 00:52:17.330 --> 00:52:20.690 align:middle line:84% Or I was, this morning at [INAUDIBLE],, 00:52:20.690 --> 00:52:25.490 align:middle line:84% I was looking at some of these glossy magazines for Army, 00:52:25.490 --> 00:52:27.740 align:middle line:84% and I noticed that all the soldiers 00:52:27.740 --> 00:52:31.580 align:middle line:84% in Iraq and Afghanistan, their helmet had something there. 00:52:31.580 --> 00:52:33.200 align:middle line:90% And I asked what is this? 00:52:33.200 --> 00:52:35.510 align:middle line:84% They said, of course, it's a video camera, 00:52:35.510 --> 00:52:38.630 align:middle line:84% so it's part of a standard military equipment. 00:52:38.630 --> 00:52:43.580 align:middle line:84% So I don't know how unobtrusive you can make it. 00:52:43.580 --> 00:52:45.020 align:middle line:90% Is it in the clothing? 00:52:45.020 --> 00:52:50.418 align:middle line:84% Is it something that you clip on your shirt, or is it here? 00:52:50.418 --> 00:52:51.710 align:middle line:90% Hey, you are in the media labs. 00:52:51.710 --> 00:52:52.460 align:middle line:90% You figure it out. 00:52:52.460 --> 00:52:55.160 align:middle line:90% 00:52:55.160 --> 00:52:56.540 align:middle line:90% So well, that's it. 00:52:56.540 --> 00:52:57.080 align:middle line:90% Thank you. 00:52:57.080 --> 00:53:01.457 align:middle line:90% 00:53:01.457 --> 00:53:03.040 align:middle line:84% RAMESH RASKAR: Any questions for Ravi? 00:53:03.040 --> 00:53:08.380 align:middle line:90% 00:53:08.380 --> 00:53:11.110 align:middle line:84% And we can make your slides available? 00:53:11.110 --> 00:53:11.860 align:middle line:90% RAVI ATHALE: Yeah. 00:53:11.860 --> 00:53:12.980 align:middle line:90% RAMESH RASKAR: OK. 00:53:12.980 --> 00:53:14.230 align:middle line:90% STUDENT: Can I ask a question? 00:53:14.230 --> 00:53:15.280 align:middle line:90% RAMESH RASKAR: Yes. 00:53:15.280 --> 00:53:17.620 align:middle line:84% STUDENT: So there is this problem 00:53:17.620 --> 00:53:20.980 align:middle line:84% in biometrics that you want to capture-- 00:53:20.980 --> 00:53:26.290 align:middle line:84% where you want to recognize a person at a distance, and maybe 00:53:26.290 --> 00:53:30.100 align:middle line:84% they are pushing programs of iris at a distance or face 00:53:30.100 --> 00:53:34.670 align:middle line:84% recognition at a distance in [INAUDIBLE] things like that. 00:53:34.670 --> 00:53:37.240 align:middle line:84% And there are two problems in this. 00:53:37.240 --> 00:53:40.720 align:middle line:84% One is atmosphere between the subject and you, 00:53:40.720 --> 00:53:44.350 align:middle line:90% and the other is subject is-- 00:53:44.350 --> 00:53:46.640 align:middle line:84% he does not want you to recognize him. 00:53:46.640 --> 00:53:48.520 align:middle line:90% He's uncooperative. 00:53:48.520 --> 00:53:50.410 align:middle line:84% What would be your ideas in terms 00:53:50.410 --> 00:53:55.960 align:middle line:84% of how our image sensors should go to capture that problem? 00:53:55.960 --> 00:53:57.890 align:middle line:84% Because if we have a really good image, 00:53:57.890 --> 00:54:00.160 align:middle line:84% then we can run our recognition algorithms, 00:54:00.160 --> 00:54:02.890 align:middle line:84% and then we can focus more on recognition aspect 00:54:02.890 --> 00:54:08.560 align:middle line:84% rather than dealing with issues like atmospheric issues 00:54:08.560 --> 00:54:09.490 align:middle line:90% and stuff like that. 00:54:09.490 --> 00:54:13.738 align:middle line:90% 00:54:13.738 --> 00:54:16.450 align:middle line:84% RAVI ATHALE: And that's a very broad question. 00:54:16.450 --> 00:54:19.690 align:middle line:84% It also touches upon some very sensitive topics, 00:54:19.690 --> 00:54:21.550 align:middle line:90% as you can imagine. 00:54:21.550 --> 00:54:26.260 align:middle line:84% All I can say is that that program announcement is public. 00:54:26.260 --> 00:54:26.770 align:middle line:90% Right? 00:54:26.770 --> 00:54:27.430 align:middle line:90% RAVI ATHALE: That is-- 00:54:27.430 --> 00:54:29.763 align:middle line:84% STUDENT: [INAUDIBLE] not the only one that's interested. 00:54:29.763 --> 00:54:33.425 align:middle line:90% [INAUDIBLE] 00:54:33.425 --> 00:54:35.800 align:middle line:84% RAVI ATHALE: There are all kinds of agencies interested-- 00:54:35.800 --> 00:54:36.978 align:middle line:90% [INTERPOSING VOICES] 00:54:36.978 --> 00:54:39.520 align:middle line:84% RAVI ATHALE: Yeah, there is a program enhancement called Best 00:54:39.520 --> 00:54:42.957 align:middle line:84% Biometric Exploitation sponsored by-- 00:54:42.957 --> 00:54:44.290 align:middle line:90% STUDENT: Biometric exploitation? 00:54:44.290 --> 00:54:45.850 align:middle line:84% RAVI ATHALE: Science and technology. 00:54:45.850 --> 00:54:49.810 align:middle line:84% Out of this agency called IARPA, Intelligence Advanced Research 00:54:49.810 --> 00:54:51.520 align:middle line:90% Projects Agency. 00:54:51.520 --> 00:54:54.580 align:middle line:84% It's a multidimensional problem, and there are all kinds 00:54:54.580 --> 00:54:56.110 align:middle line:90% of ideas that are proposed. 00:54:56.110 --> 00:55:01.270 align:middle line:84% Hopefully, within few months, the contracts will be awarded. 00:55:01.270 --> 00:55:04.400 align:middle line:90% We'll find out what. 00:55:04.400 --> 00:55:07.200 align:middle line:84% But I think somewhat going beyond that, 00:55:07.200 --> 00:55:08.810 align:middle line:84% I think really in imaging senses-- 00:55:08.810 --> 00:55:12.170 align:middle line:84% and this is a debate that we are constantly having. 00:55:12.170 --> 00:55:16.710 align:middle line:84% Is it necessary to form an image? 00:55:16.710 --> 00:55:20.450 align:middle line:84% And if you don't form an image, if you just 00:55:20.450 --> 00:55:23.838 align:middle line:84% map directly into features, number one, can you do that? 00:55:23.838 --> 00:55:25.130 align:middle line:90% Number two, should you do that? 00:55:25.130 --> 00:55:27.350 align:middle line:84% Is it advantageous in some form or fashion? 00:55:27.350 --> 00:55:30.080 align:middle line:90% 00:55:30.080 --> 00:55:32.870 align:middle line:84% It applies to biometrics as well. 00:55:32.870 --> 00:55:38.490 align:middle line:84% Right now, for iris recognition, there is the dogman algorithm. 00:55:38.490 --> 00:55:42.920 align:middle line:84% So that is geared upon you sampling the iris 00:55:42.920 --> 00:55:45.840 align:middle line:84% at a certain resolution in a certain way. 00:55:45.840 --> 00:55:48.890 align:middle line:84% But if you tie the front end acquisition 00:55:48.890 --> 00:55:53.750 align:middle line:84% to the exploitation, could you get into a more robust, simpler 00:55:53.750 --> 00:55:56.160 align:middle line:90% system? 00:55:56.160 --> 00:55:59.670 align:middle line:84% That's an interesting question, but it's 00:55:59.670 --> 00:56:04.140 align:middle line:84% a little difficult for sponsors to launch in that direction 00:56:04.140 --> 00:56:07.440 align:middle line:84% because too much risk, and if something changes on one side, 00:56:07.440 --> 00:56:11.330 align:middle line:90% the whole thing may collapse. 00:56:11.330 --> 00:56:13.830 align:middle line:90% But that's an interesting point. 00:56:13.830 --> 00:56:15.750 align:middle line:84% STUDENT: Yeah, I think the point I'd make 00:56:15.750 --> 00:56:20.310 align:middle line:84% is that, as long as you continue to rely on collecting 00:56:20.310 --> 00:56:25.290 align:middle line:84% a high quality image to do this, I think doing it at a distance 00:56:25.290 --> 00:56:26.940 align:middle line:90% is going to pose a challenge. 00:56:26.940 --> 00:56:29.460 align:middle line:84% And the dogman algorithm is a perfect example. 00:56:29.460 --> 00:56:31.712 align:middle line:84% You start out with an image that's 00:56:31.712 --> 00:56:33.420 align:middle line:84% several hundred kilobytes, and you end up 00:56:33.420 --> 00:56:36.760 align:middle line:84% with a template that's a couple 100 bytes. 00:56:36.760 --> 00:56:39.420 align:middle line:84% So obviously, you don't need all that data 00:56:39.420 --> 00:56:42.030 align:middle line:84% that you collected in that high resolution image 00:56:42.030 --> 00:56:44.940 align:middle line:84% to do iris recognition, so why do 00:56:44.940 --> 00:56:49.080 align:middle line:84% you need to start with that image in the first place? 00:56:49.080 --> 00:56:53.240 align:middle line:84% And if you could develop a compressed representation, 00:56:53.240 --> 00:56:57.980 align:middle line:84% if you could sense that compressed representation 00:56:57.980 --> 00:57:00.890 align:middle line:84% directly, would it lead to a simpler system? 00:57:00.890 --> 00:57:03.080 align:middle line:84% Would it lead to a simpler and more robust system? 00:57:03.080 --> 00:57:07.070 align:middle line:84% That last question is very tricky to answer, 00:57:07.070 --> 00:57:10.400 align:middle line:84% and the three of us said [INAUDIBLE] 00:57:10.400 --> 00:57:13.730 align:middle line:84% when he was with us four of us [INAUDIBLE] gone over this over 00:57:13.730 --> 00:57:18.380 align:middle line:84% and over, and we still haven't come to a conclusion. 00:57:18.380 --> 00:57:19.240 align:middle line:90% STUDENT: Thanks. 00:57:19.240 --> 00:57:19.990 align:middle line:90% RAVI ATHALE: Sure. 00:57:19.990 --> 00:57:20.823 align:middle line:90% RAMESH RASKAR: Good. 00:57:20.823 --> 00:57:26.260 align:middle line:84% So hopefully, as you already saw, a lot of the project plans 00:57:26.260 --> 00:57:28.870 align:middle line:90% are also about not-- 00:57:28.870 --> 00:57:32.290 align:middle line:84% not about standard cameras and then exploiting specialization 00:57:32.290 --> 00:57:36.278 align:middle line:84% and autonomy and trying to change a lot of the rules here. 00:57:36.278 --> 00:57:37.820 align:middle line:84% RAVI ATHALE: And as a matter of fact, 00:57:37.820 --> 00:57:41.830 align:middle line:84% many of the projects we looked at each other and said, hm, 00:57:41.830 --> 00:57:43.860 align:middle line:84% should be talking to these people? 00:57:43.860 --> 00:57:45.610 align:middle line:84% STUDENT: I'm going to come back and visit. 00:57:45.610 --> 00:57:46.570 align:middle line:90% RAVI ATHALE: Yeah, exactly. 00:57:46.570 --> 00:57:47.110 align:middle line:90% RAMESH RASKAR: Yeah, welcome-- 00:57:47.110 --> 00:57:49.810 align:middle line:84% you're welcome to sign up as a mentor for the class. 00:57:49.810 --> 00:57:51.850 align:middle line:84% So we have several mentors for the class, 00:57:51.850 --> 00:57:53.810 align:middle line:84% and you're welcome to sign up for that. 00:57:53.810 --> 00:57:57.880 align:middle line:84% So excellent, so next week we have the midterm exam. 00:57:57.880 --> 00:58:01.720 align:middle line:84% It's open book, open laptop, open internet, open everything, 00:58:01.720 --> 00:58:04.220 align:middle line:90% so don't study for it. 00:58:04.220 --> 00:58:07.180 align:middle line:84% And it will be mostly about drawing diagrams and explaining 00:58:07.180 --> 00:58:11.590 align:middle line:84% things and problems that will make you think. 00:58:11.590 --> 00:58:14.050 align:middle line:84% And eventually, after that, we'll 00:58:14.050 --> 00:58:17.835 align:middle line:84% be studying animal eyes, which is 00:58:17.835 --> 00:58:21.650 align:middle line:90% [INAUDIBLE] 13th of November.